# A Deep Dive into NFT Whales: A Longitudinal Study of the NFT Trading
Ecosystem

Na Hyeon Park1§ Hanna Kim1§ Chanhee Lee2 Changhoon Yoon2
Seunghyeon Lee2 Youngjin jin1 Seungwon Shin1
1KAIST, Daejeon, South Korea
2S2W Inc., Seongnam, South Korea
1{julia19, gkssk3654<EMAIL_ADDRESS>
2{leemember, cy, minsu<EMAIL_ADDRESS>
(2018; 20 February 2007; 12 March 2009; 5 June 2009)

###### Abstract.

NFT (Non-fungible Token) has drastically increased in its size, accounting for
over $16.9B of total market capitalization. Despite the rapid growth of NFTs,
this market has not been examined thoroughly from a financial perspective. In
this paper, we conduct methodical analyses to identify NFT market movers who
play a significant role in potentially manipulating and oscillating NFT
values. We collect over 3.8M NFT transaction data from the Ethereum Blockchain
from January 2021 to February 2022 to extract trading information in line with
the NFT lifecycle: (i) mint, (ii) transfer/sale, and (iii) burn. Based on the
size of held NFT values, we classify NFT traders into three groups (whales,
dolphins, and minnows). In total, we analyze 430K traders from 91 different
NFT collection sources. We find that the top 0.1% of NFT traders (i.e.,
whales) drive the NFT market with consistent, high returns. We then identify
and characterize the NFT whales’ unique investment strategies (e.g., mint/sale
patterns, wash trading) to empirically understand the whales in the NFT market
for the first time.

NFT, Economics, Whales, Market, Cryptocurrency

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Applied computing Digital
cash††ccs: Social and professional topics User characteristics§§footnotetext:
Both authors contributed equally to this research.

##  1\. Introduction

Cryptocurrency, which has recorded a phenomenal growth in the past decade, has
now become one of the most popular investment options. Successfully
demonstrating the potential of its underlying blockchain technology as well as
its enormous profitability, other blockchain-based digital assets have also
emerged as alternative, life-changing investment opportunities.

Following the huge success of cryptocurrencies, public attention toward NFTs
has exploded. Many investors have traded NFT collectibles for financial gain,
and eventually, the _whales_ appeared on the market. The term _whale_ has
become a commonly accepted designation in the blockchain community. For
example, in the cryptocurrency market, Satoshi Nakamoto and the Winkelvoss
Brothers are often referred to as _Bitcoin whales_. They own an exceptionally
large amount of Bitcoins enough to impact the entire market as previously
measured. (Banton, 2021; Gupta, 2022; Onanuga, 2021).

NFT whales can be easily monitored on commercial NFT analytics platforms
(NFTGo, 2022b; NFTBank, 2022) available today. For instance, NFTGo (NFTGo,
2022b), one of the most popular NFT analytics platforms, evaluates the total
value of the NFTs in each NFT holder’s wallet and publishes a list of whales.
The purpose of this feature is to provide useful investment information for
potential NFT buyers who wish to follow the whales’ investment pattern. As
such, the movement of NFT whales, which may be potentially driving the market,
may provide useful insights or lessons; however, neither whale-following
investment strategy nor other aspects of NFT whales have been thoroughly
studied.

To gain a deeper understanding of obscure NFT whales, we ask the following
questions:

  * •

What are the characteristics of NFT whales?

  * •

What are the investment strategies of NFT whales?

  * •

Are NFT whales successful investors?

  * •

Are NFT whales really powerful enough to impact the entire NFT market?

This paper dives deeply into the newly emerging NFT market to reveal and
understand NFT whales for the first time. We collect the transaction logs from
NFT smart contracts (Hollander, 2020) and align the data according to the NFT
life-cycle: (i) mint, (ii) transfer/sale, and (iii) burn. Then, we estimate
how much value has been transferred through NFT trades based on the market
price at the time of each transaction. As a result, we are able to identify
430K traders involved in the top 91 NFT collections.

Below are the significant findings of this paper:

  * •

Most high-value NFTs are owned by NFT whales.

  * •

NFT whales rarely sell NFTs while eagerly buying the highest priced NFTs (top
1 percent)

  * •

NFT whales are indeed the most profitable group.

  * •

NFT whales take advantage of wash trading.

The findings of this longitudinal measurement study have given us a lesson
regarding the NFT market. Our study has shown that NFT whales have been
powerful enough to manipulate the entire market. In other words, the NFT
market itself is still immature and unstable. Furthermore, we also observe
that the whales have been abusing weaknesses of the NFT market to inflate the
value of the NFTs they own. In order to establish a mature and stable NFT
market, the existing regulatory and legal environments must adapt to include
NFT. It is also recommended to implement security countermeasures to protect
the market from any abusive behaviors.

##  2\. Background

Non-fungible token. An NFT is unique, irreplaceable, and its value is decided
individually depending on demand. An NFT is a digital asset that represents
objects like art, video, in-game entities, etc. NFTs are stored in blockchain,
with Ethereum being the representative choice. NFTs are categorized in a
collection, which refers to a set of NFTs. Commonly, NFTs in the same
collection share common features. For example, one of most popular collection
named Bored Ape Yacht Club (BAYC) is a collection consisting of 10,000 ape
tokens. The price of NFTs varies widely even within the same collection,
although the range varies depending on the collection.

NFT life-cycle. Due to the anonymous nature of cryptocurrency wallet, we
define a unique wallet address as a trader. Here, we detail the procedures for
how an NFT transaction operates, which consists of three steps: (i) mint, (ii)
transfer/sale, and (iii) burn. Mint refers to the process of offering newly
created tokens to the public. Generally, NFT creators post the detailed
schedule and price on various channels, such as discord, which is then
conducted by a smart contract on a blockchain. Transfer is an act of simple
value transfer which only passes ownership of NFT tokens to another wallet.
Sale is the process of transferring ownership of NFTs for a price, and is
commonly held in NFT markets (ope, 2022a; loo, 2022). Burn refers to the
method of removing ownership of NFT tokens on purpose. Once an NFT is burned,
no one is able to gain its control.

Classifying NFT traders. There has been no official term for traders
(individiuals or entities) that hold differing amounts of NFTs. Thus, we
consider each wallet as an individual trader and classify the traders into
three groups according to their holding values of NFTs, for every time range
in our dataset:

  * •

Whale \- top 0.1% traders

  * •

Dolphin \- top 10% traders excluding whales

  * •

Minnow \- all other traders excluding whales and dolphins (89.9%)

To clearly identify the trader groups, we define holding value to be used as a
concrete criteria for our classification. The estimation of the current market
price of NFTs may be imprecise due to the high volatility of the market value
of NFTs compared to traditional economics. Instead, we believe that the latest
trading price of each NFT could represent its value well. We define holding
value of a trader as the sum of the last traded price for each token held by
the trader.

To focus on the top holding value traders, we refer to whales and dolphins
collectively as holding value leaders. The number of traders in each group is
shown in Table 1 and is further described in Table 2 of the Appendix.

##  3\. Tracing NFT Transactions on the blockchain

In Ethereum, there are two types of transactions: external and internal
transactions. Ether (ETH) transfers between users is recorded as external
transactions. External transactions have information such as receiver’s and
sender’s addresses and transferred amount in ETH is recorded in the blockchain
and readily available to anyone for reference.

On the other hand, transferring tokens, such as NFTs or fungible tokens, is a
type of an internal transaction, which is not stored on the blockchain.
Instead, we can use the token transfer log, which is recorded by token
contracts when token transferring occurs. By collecting transfer logs from NFT
contracts, we can trace how NFT transactions work in the NFT ecosystem.

We collect external transactions and token transfer logs from the Ethereum
blockchain to track NFT ownership changes and subsequent payments. More
details can be found in Section A.2.

For the NFT ecosystem, 2021 marks the first year when the market began to grow
rapidly with public attention (Mattei, 2021), with NFT trading volumes showing
an increase of 21,000% from 2020 (Browne, 2022). As a result, we focus on data
with block_timestamp from the first day of 2021 to February 28, 2022 (14
months). During this collection period, we obtain 3,838,587 transactions for
over a million NFT items in total. Also, the unique number of accounts
participating in a transaction at least once is 430.2K. Our data collection is
summarized in Table 1.

Table 1. Summary of data collection. The accounts are divided by whale, dolphin, minnow (February, 2022) Type | Collection
---|---
NFT | 1,129,6967
Transaction | 3,838,587
Account | 3,086,046
| Whale | 430
| Dolphin | 42,593
| Minnow | 387,204
| Total accounts | 430,277
Period |  January 1,2021 $\sim$ February 28, 2022

##  4\. Characteristics of NFT WHALES

In this section, we characterize NFT whales through a deep analysis of their
behavior patterns such as unique trading methods for NFT items and portfolio
management.

Figure 1. Changes in the group of traders in whale from January 2021 (t = 1)
to February 2022 (t = 14). The top and bottom only depict traders that have
belonged to the whale group at least once, until August and February 2022,
respectively. Transitions from one color to another depict a quantity of
traders with group changes.

###  4.1. Changes in Whales’ Composition

The NFT market has changed dramatically over the past 14 months with the
influx of new traders and the advent of various types of new collections. In
this rapidly changing market, how do traders become whales? To answer this
question, we study how traders move across between groups during 14 months,
especially focusing on whales. Figure 1 shows the changes in composition of
the whale group until February 2022.

In the beginning, since the size of whale group is small, the composition of
the group is highly volatile as the market size increases. However, from June
2021, more than 80% of whales remain in the next whale group, except in August
2021. August is when the most significant change in the whale group occurs.
The surge of public interest at that time (Major, 2021) caused the largest
number of new traders compared to the previous months. To understand how
traders become whales and how they maintain their status, we investigate the
characteristics according to the whales’ origin as follows. The percentage of
each group where the whales belonged to is summarized in Table 4 of the
Appendix.

Whales from whales. Whales maintain their status through various ways: buying,
receiving, and minting. In the first half, more than half of whales buy NFTs
in order to increase the holding value. However, in early 2022, whales who
mainly receive tokens grow in number. Meanwhile, some whales actively mint
NFTs. For example, one whale minted a tremendous number of tokens in several
collections and sold them to dolphins and minnows. There also exist whales who
were inactive for more than a few months. Their extremely expensive tokens let
them stay in the whale group.
Whales from dolphins. As the size of the whale group increases, it is natural
for dolphins to become whales. In August, when the market size grows most
rapidly, dolphins take up 39% of the new whale group. Dolphins raise their
holding value in various ways like whales from whales. However, in early 2022,
the percentage of new whales from dolphins see a decrease to 15%.

Whales from minnows. We rarely observe whales who formerly belonged to
minnows. They increase their NFT holding value through buying or minting. They
barely receive tokens from other traders, which indicates that they do not
form particular relationships with other traders. This is completely different
to first-time trading whales, which is described in detail below.

First-time trading whales. There are a lot of traders who become whales as
soon as they begin participating in the NFT market. When the size of the
market increases rapidly, whales from this group sometimes outnumber the
whales from the dolphin group. They usually become whales through buying or
receiving, but many of them become whales only through the act of receiving
tokens. Usually, the tokens they receive are NFTs of popular collections with
high trading volume (e.g. CrytoPunks). Interestingly, most of the NFTs they
received come from the former whales, implying a close relationship between
them.
Findings and Insight. Although the NFT ecosystem is rapidly growing, whales
maintain their place firmly. In addition, newly emerging whales are often
associated with former whales. This clearly shows that it is hard for minnows
to become whales.

Figure 2. Price distribution of tokens held by each group. Dark-blue, sky-
blue, olive each represents holdings of whales, dolphins, and minnows. Figure
3. Monthly statistics of each transaction type of each group on average.
(stacked plot)

###  4.2. Whales’ Portfolio Holdings

Common participants in the NFT market pursue financial gains. In this regard,
examining their portfolio from various angles is highly valuable for measuring
significant financial factors on the NFT ecosystem. In this section, we
examine the whales’ NFT portfolio, preferences, and dominance in NFT
collections.

Top collection holdings. We begin by searching for top 10 collections that
whales mostly hold. Interestingly, we discover that the majority of the
collections in the top 10 are popular collections with large trading volume.
The whales’ most held collection is Art Blocks (9.6K tokens held).
Interestingly, whales hold 1.6K CryptoPunks tokens which accounts for 16% of
total tokens in this collection. The number of tokens held by Whales in other
collections can be found on Table 3 of the Appendix.

Price distribution. To look into the price of each group’s holdings, we divide
the token price into six price points, ranging from $0 to the maximum price in
our data, which is $23.3M. Figure 2 describes the price distribution of tokens
that each group held on February 28, 2022. Figure 2(a) shows the distribution
of tokens over total 91 collections. In addition, to look at distribution
within a collection, 2(b) shows the holding range of the most popular NFT
project, CryptoPunks. Note that the price distribution of popular collections
(e.g., BAYC, Art Blocks) also resembles the distribution of CryptoPunks.
Interestingly, almost all of the tokens with prices ranging over $10M are in
hands of whales. Whales also hold most of the tokens in the price range of $1M
to $10M across all collections. On the whole, Figure 2 indicates that whales
are the only holders of the high-price tokens, both for overall collections
and the top collections.
Findings and Insight. Generally speaking, price of goods in a market is an
indicator of its ‘value’. The result obtained in this section suggests that
whales gain dominance over almost all highly valuable tokens.

##  5\. Whales’ Impact on the NFT Market

To examine how influential whales are in the NFT ecosystem, we analyze the
whales’ trading behavior in terms of their impact on market sentiment and
traders.

In Figure 3, we observe that the average number of transactions by whales is
overwhelmingly large. We observe larger fluctuations in the trading activities
of whales than dolphins and minnows. To understand the whales’ behavior, we
study how they respond to real-world events related to NFT.

Impact on market volatility during liquidation. Although the number of whales
makes up a small portion of all traders, whales impact market volatility when
liquidating NFT collections. Liquidation by whales has drastically increased
market volatility. We discuss important events (E1$\sim$E3) in which the
whales had a huge impact on the market.

First, we observe a rapid but short-lived peak in February 2021 in all groups.
With the growing popularity of one collection, Hashmasks (Google, 2022),
traders buy a huge number of tokens from the collection, which accounts for
more than half of their purchases.

(E1) However, in April, NFT market conditions cooled abruptly. Traders in all
groups barely participated in transactions due to a sharp decline (nearly 70%)
in the average price of NFTs (Shaw, 2021). In fact, before the market crash,
we observe whale activities drop to its lowest position in March. This implies
that the decrease in whale activities has had a big impact on the NFT market
sentiment.

(E2) In May, we observe an uplift in the number of sells only in the
transaction graph for whales. In particular, the whales sell tokens that they
have minted on this month. Interestingly, 95% of such tokens are from two new
collections, Meebits and Bored Ape Yacht Club (BAYC). Meebits received the
spotlight even before launch, since it is made by the creators behind
CryptoPunks. On the other hand, BAYC suddenly gained popularity due to a large
quantity of whales’ mintings (Trash, 2021). This promoted the sales of BAYC
tokens, especially in the dolphins and minnows group. In fact, BAYC accounts
for 40% and 54% of dolphins’ and minnows’ sales, respectively.

(E3) We observe a steady increase in overall sales volume in July 2021, which
increases dramatically by August. This is related to a surge in price of
CryptoPunks caused by whales. One day in August, a whale bought over 100
CryptoPunks NFTs worth more than $6M in total. Since then, the whales’
purchase of the collection continued. Such whale activity sparked public
interest in the NFT market, resulting in the influx of new traders (Li, 2021).

Except for specific periods in which whales actively liquidate their assets,
whales tend to transfer (specifically, the act of receiving) instead of
selling assets on the market. They prefer to hold assets relatively longer
compared to dolphins and minnows. This is discussed in more detail in Section
6.1.

Leading investment trends.  Whales have actively participated in mint as early
stage investors. Also, they prefer to invest specific collections that they
are interested in, while other groups participate in a variety of collections.
Here, we uncover several remarkable events as follows.

In January 2021, whales mint about four times more than any other transaction
types, and most of them are Hashmasks. Their mintings surged again in May, due
to Meebits and BAYC as mentioned in E2. In the later months including August,
some collections (e.g.,Punks Comic) embed governance rights on tokens (a.k.a.
DAO tokens), that increased the demand for mint. This suggests that whales are
interested in exercising governance rights on a collection.

NFT market capitalization is increasing due to rise in the number of
collections and traders. However, we see a drop in practical trader activities
compared to the early months. In fact, only 20% of minnows participates in
trading in the last month (see Table 5 in the Appendix). This implies that the
NFT market is mainly operated by whales and dolphins. In addition, we cannot
observe any noticeable increase in whale transactions after August, which
suggests the lack of emergence of influential collections (e.g., CryptoPunks).
Despite the large number of collection launches, only a few major collections
account for the majority of the NFT market capitalization (NFTGo, 2022a).
Therefore, holding major collection NFTs allows the whales to maintain their
position as whales.
Findings and Insight. The NFT market is predominantly being driven by whales.
Whales have distinct trading patterns compared to that of other groups; whales
have a huge impact on the market and often alter market sentiment.

##  6\. Deep-dive into Whales’ investing strategies

In this section, we discuss several investment strategies that whales utilize
to achieve high financial gains. We discover three strategies: (i) minting or
buying expensive NFTs and holding them for a long time until liquidation, (ii)
intensively investing during the minting period, and (iii) taking advantage of
self-trading.

###  6.1. Long-term Investment

In a market, the value of a token is decided by its price; valuable tokens are
limited but takes up a major portion of the market cap. Therefore, we define
and track most-valuable NFTs as the top 1% most expensive NFTs from each
collection. By accumulating such tokens, we obtain 14,192 most-valuable NFTs.

To investigate the strategies whales use on the most-valuable NFTs, we begin
by calculating how long each group holds the tokens before selling them.
Figure 4 shows the holding time of each group on most-valuable NFTs. Here,
holding time is the duration between the last buy/receive time of each token
and the last day of our data (February 28, 2022). As clearly shown from the
graph, whales are likely to hold tokens longer than any other group. It turns
out that some whales even hold tokens for the entire period of our data. On
the other hand, for minnows, the peak in the left range of the graph is due to
their relatively short holding period on collections that have low average
token price. While the holding time distributions of minnows and whales are
very distinct from each other, the distribution of dolphins resemble that of
whales. Hence, the graph indicates that holding value leaders tend to hold the
valuable tokens longer.

These observations bring about a question related to the liquidity of high-
value tokens; do the other traders have any chance at acquiring such tokens?
To answer this, we track the sale history of these tokens. Figure 5
illustrates the number of sales by whales on most-valuable NFTs. The graph
clearly shows that the number of buys across all months heavily outweigh the
number of sells. This indicates that high-value NFTs have low-liquidity.

These two strategies are likely to be long-term tactics by whales to wait for
valuable tokens to reach their desired price. Indeed, the results suggest that
whales consistently collect most-valuable NFTs but barely sell them for a long
duration.
Findings and Insight. Whales are patient; they invest by holding valuable NFTs
for a long period and do not sell them to other traders. NFT traders must be
aware of low-liquidity of high-value tokens caused by whales.

Figure 4. The holding time of most-valuable tokens per group Figure 5.
Buy/sell plot by whales on most-valuable tokens (stacked plot)

###  6.2. Deliberate Investment During the Minting Phase

We find out that whales take advantage of minting via unique methods. They use
a number of strategies to raise the price of tokens they minted. We describe
each strategy throughout this section.
Strategy 2-A. The first strategy is related to the number of transfers between
the first minting period and the first sale. Around 10% of tokens minted by
each group are transferred at least once before the first sale. To compare
each group’s transfer tendency, we look into those tokens in detail. Figure 6
shows the number of transfers between the mint and the first sale, where the
number is shown in percentage. As shown in the figure, a large portion of NFTs
minted by whales are transferred many times (up to 25 times). 72.0% of tokens
minted by whales are transferred at least two times whereas the percentages
are much smaller in dolphins and minnows. Furthermore, 13.8% of tokens minted
by whales go through transfer at least 5 times.

Strategy 2-B. Another noticeable strategy is relevant to time duration before
the first sale. This can be seen from Figure 7 where the time duration is
divided into four ranges: within a day, a day to a week, a week to a month and
more than a month. Surprisingly, whales tend to wait for long periods of time
before the first sale; 82.6% of tokens take more than a week and 58.5% take
more than a month. In contrast, dolphins and minnows wait less and the
majority of tokens are sold within a week for both groups.

Strategy 2-C. This strategy is used by some of the minters. Usual sales after
minting do not involve the address (i.e., minter) that minted the token.
However, we find that some minters later receive back the token that they
minted. To give an instance that occurred in September 2021, the minter first
minted the token and sold it to another wallet address for 6.25 ETH. Then, the
token was transferred back to the minter. Finally, the minter sold the token
to another trader for 6 ETH. Indeed, we find that 13.7K tokens are involved in
this type of transaction pattern. Whales and dolphins are involved in 10.4K
tokens which accounts for 77.4% of such tokens. This is a significant number
considering the fact that usually the total number of tokens for a collection
is around 10K.

All three strategies are effectively utilized to maximize the profit of
whales. We find out that the profit of the first sale is proportional to all
of these strategies. Details of these profits can be found in Table 6 and 7.
Further details for mint profits from whales are discussed in Section 10
Findings and Insight. Whales use their own special strategies to maximize the
profit of tokens they minted.

Figure 6. Number of transfers between mint and first sell Figure 7. Time
duration between mint and first sell

###  6.3. Wash Trading

Figure 8. Number of wash trading instances(stacked plot)

Wash trading is a collusion by the buyer and the seller to artificially
inflate the trading volume of an asset. They repeatedly trade their assets
between them, which results in cycles in the sale graph, where nodes are
traders and edges are sales between them. Therefore, wash tradings are
captured by finding strongly connected components (SCCs) in each token sale
graph (Victor and Weintraud, 2021). Indeed, multiple unconditional token
transfers can be used in trading malpractices to avoid monitoring (Das et al.,
2021; Victor and Weintraud, 2021). Thus, we construct each NFT transaction
graph, where nodes are traders and edges are transactions between them, and
find SCCs in each graph including sales at least once.

We detect 3,558 instances of wash trading in 1,676 NFTs across 82 out of 91
collections including 3,676 traders in our data. Note that the collected NFT
transaction data is different for each period and for each collection, the
number of wash trading instances can be different from the work done by Victor
et al. (Victor and Weintraud, 2021). Surprisingly, the holding value leaders
perform 69% of wash trading. Furthermore, 42 whales, which makes up 10% of all
whales, are involved. We find that popular collections with high trading
volume (e.g., CryptoPunks, BAYC) also belong to manipulations done by the
holding value leaders.

To understand when and why wash trading occurs over time, we locate SCCs in
each token transaction graph from the monthly transaction records. Figure 8
shows the number of wash trading over time. First, we observe a noticeable
peak in October. Wash trading for The n project started in September (when it
was launched) and recorded 506 cases in October. We confirm that the average
price of the collection tokens traded sharply rises whenever washing trading
is involved as shown in Figure 12 of the Appendix.

This is a common phenomenon in newly launched collections. In order for a
newly-launched collection to be verified by Opensea (the largest NFT
marketplace), the trading volume must be over 100 ETH (ope, 2022b). For this
reason, many traders are tempted to perform wash trading. Similarly, we can
see a increase in the number of wash trading in August 2021, when many new
collections emerged.

The number of wash trading shows another small peak in January 2022, which is
related to the policy of LooksRare (Williams, 2022), a newly-launched NFT
marketplace. It rewards valuable tokens to traders according to each trader’s
trading volume, which lures traders into wash trading.
Findings and Insight. Whales and dolphins actively participate in wash trading
to receive verification of newly-launched collections in NFT matketplaces or
earn rewards by abusing NFT market policy. NFT prices can be driven up and
down along with their wash trading. Popular collections (e.g., CryptoPunks,
BAYC) are also unavoidable.

##  7\. Evaluating Investment Performance

In this section, we evaluate the investment performance of each group and
discuss how whales achieve high returns compared to other market participants.

###  7.1. Investment Performance

Figure 9 illustrates the investment performance of each trading group. We
divide the profit range with a log scale due to the wide range of profit among
traders. Note that we do not include traders who have never participated in
profit activity (i.e., zero profit). The trader who gained the maximum profit
is a whale ($18.9M) and the trader with the largest loss is a dolphin
(-$2.7M). The most noticeable observation is that 35.9% of traders who belong
to whales make profits larger than $1M while there hardly exists any from
dolphins and minnows. Moreover, while 79.8% of whales produced profits greater
than $100K, the percentages are much lower in other groups. This suggests that
a large fraction of whales produces a considerably higher profit compared to
any other group.
Findings and Insight. We identify that whales have achieved significant
financial gain. A large fraction of whales achieved profit over $1M which
shows great contrast with other groups.

###  7.2. Sources of Financial Gains from Whales

Finally, we scrutinize the source of profit from whales. We divide the profit
source into two parts: buy profit and mint profit.

Buy profit. This type of profit occurs from ordinary sales, except the very
first sale of a token. For profit generated from buy, we focus on the
collections that whales take advantage of. Whales gained profit from 74
collections out of 91 in total. Among them, we sort the top collections in
which the whales were most profitable. If a collection does not generate any
profit for several months (e.g., launching in the middle of the entire
period), we only mark zero profit once for graph’s visibility.

Figure 10 shows the average profit of whales from the top 7 collections (top
10% of 74 collections) for each month. Overall, the majority of these
collections are well-known for their large trading volume. Among them, profits
from CryptoPunks, Art Blocks and BAYC are consistently large, which indicates
that whales continuously gain profit from very popular collections. For the
peak of the rest of the collections, the profit ranks the top due to the
selling of several expensive tokens by small number of whales. For example, a
peak of ASM AIFA Genesis on December 2021 was generated by a single whale that
sold 56 tokens.

Mint profit. Whales consistently gained profit from mint profit, which is the
profit from the first sale just after mint. Surprisingly, although mint profit
occurs only once per token, mint profits are comparable to buy profits and
even are larger on some months. This is noteworthy as, 1) the number of mint
logs accounts for 28.2% of total transfer logs in our data (1.1M out of 3.9M)
and 2) most collections have a limited number of tokens available for mint
(e.g., only 10K tokens for BAYC).

Until July 2021, whales sell the tokens that they have minted from a limited
number of collections. For example, The mint profit of whales consistently
occur from Hashmasks in the first half of the year. However, starting from
August where the number of traders in the NFT market drastically increased,
whales start to gain mint profit from a wide range of collections. This is
likely to be due to the launch of many collections after August. Nevertheless,
most of the mint profit are obtained from popular collections mentioned above,
e.g., Art Blocks, Meebits, BAYC until the last month (February 2022).

Meanwhile, some collections utilize airdrop as a way to advertise themselves
during the launch period; they let traders mint the tokens without paying any
mint fees (Graham, 2022). To amplify the effects of advertising, some
collections even choose to send their tokens to popular wallet addresses in a
unilateral way. Since whale accounts are easily available in NFT platforms
(e.g.,NFTGo (NFTGo, 2022b)), it is likely for whales to receive lots of tokens
through airdrop. Therefore, we investigate the profits produced by whales
through airdrop.

Figure 11 shows the top 5 collections in which the total profit of whales is
the largest. The number of airdrop tokens sold by whales take up 9 to 20% of
airdrop tokens in these collections. This is a large portion considering the
small number of whales and implies that whales actively participate in selling
airdrop tokens. Interestingly, the five collections in Figure 11 are very
popular collections with large trading volumes. Overall, among the 41
collections from which the whales obtained profit through airdrop, we find
that the tokens sold by whales are generally more expensive than tokens by
other groups on 36 collections. This indicates two possibilities: 1) whales
already have the public’s trust in the NFT market and the traders are willing
to pay high prices for the tokens minted by whales or 2) whales receive
relatively high-value tokens of a collection via airdrop. Still, in either
case, it is an undeniable fact that whales have power in the minting process
that can lead to larger profit while no others can.
Findings and Insight. Whales are mainly lucrative through selling NFTs of
popular collections. More importantly, they actively participate in minting
process, sell NFTs at higher price than any other group. This allows whales to
become successful investors.

Figure 9. Investment performance per trading group Figure 10. Average Profit
of whales per collection Figure 11. Average Profit of airdrop tokens (MAYC and
CLONE X are abbreviations for Mutant Ape Yacht Club and RTFKT CLONE X +
Murakami, respectively.)

##  8\. Discussion

Throughout this paper, we find out that whales are highly influential traders
in the NFT ecosystem. Whales hold a mere 5 percent of all NFT tokens on the
Ethereum blockchain, but their worth accounts for nearly 20 percent of the
entire NFT market value. In addition, although the NFT market features
relatively wide variations in prices, nearly all of the high value items worth
more than one million dollars belong to the whales. This means that whales
exclusively own almost all of the dominant NFT items. Considering that a small
number of NFT projects with a high market cap (e.g., CryptoPunks, BAYC,
Doodles) is bringing success to the NFT ecosystem, to say that the NFT market
is being driven by whales is not an overstatement.

In summary, the average profit from whales is far greater than that of the
other groups. In addition, the most highly profitable traders (i.e., make
profits of more than $1M) make up over 35% of all whale traders, while there
such traders only make up under one percent of dolphins and minnows. A large
portion of traders from minnows did not make profits, with some even faced
with considerable losses. This indicates that a small number of whales (note
that we only consider the top 0.1 percent of traders as whales) takes most of
the rewards for success in the NFT market. In other words, the NFT ecosystem
is a harsh environment to find success for the majority of the traders
involved (non-whales), and stakes are high for the minority traders.
Therefore, it is imperative that various studies about the NFT ecosystem are
conducted and measures to protect investors are established.

##  9\. Related Work

To the best our knowledge, this is the first longitudinal study of NFT
ecosystem that focuses on identifying and characterizing market movers from a
financial point of view. Wang et al. (Wang et al., 2021) provides an overview
of the NFT ecosystem with regard to the technical components, etc. Our work is
different in that we analyze how NFT market operates from a financial
standpoint. Dowling et al. (Dowling, 2022) and Ante et al. (Ante, 2021) study
correlation between price of NFTs and cryptocurrencies, and shows that they
have a low correlation. Distinct from their works, we focus on the NFT market
driven by traders. Brunet et al. (Casale-Brunet et al., 2021) and Nandi et al.
(Nadini et al., 2021) study the basic topological structure of NFT trading
networks and show that each is similar to other social networks. Das et al.
(Das et al., 2021) presents a systematic overview of how the NFT ecosystem
works and uncovers potential security issues. Wachter et al. (von Wachter et
al., 2022) also examines potentially illicit trading patterns in the NFT
market with two detection algorithms and their effect on price. Still, there
is no understanding on how malicious behaviors have changed over time and
analysis on the performers of wash trading.

##  10\. Conclusion

Following the rapid growth of cryptocurrencies, NFTs have recently received
wide attention from the public. Many participants have successfully traded NFT
items with financial gain, leading to the emergence of NFT whales. However,
many participants suffer from the lack of a clear viewpoint on the NFT
ecosystem. In this research, we perform the first longitudinal study to
construct in-depth analysis on the NFT market, focusing on unique activities
of market participants including whales and market movements. Specifically, we
reveal whales on the NFT ecosystem and discover that they possess unique
strategies to maximize their earnings from NFT trades and greatly impact the
entire market. Consequently, we believe that our findings and insight in this
work shed light on the NFT ecosystem, which has been minimally investigated to
date.

## References

  * (1)
  * loo (2022) 2022\.  Looksrare.  https://looksrare.org/.  Accessed: 2022-05-19.
  * ope (2022a) 2022a.  Opensea.  https://opensea.io/.  Accessed: 2022-05-19.
  * ope (2022b) 2022b.  What is a verified account or badged collection?  https://support.opensea.io/hc/en-us/articles/360063519133-What-is-a-verified-account-or-badged-collection-.
  * Ante (2021) Lennart Ante. 2021\.  The non-fungible token (NFT) market and its relationship with Bitcoin and Ethereum.  _Available at SSRN 3861106_ (2021).
  * Banton (2021) Caroline Banton. 2021\.  Bitcoin Whale.  https://www.investopedia.com/terms/b/bitcoin-whale.asp.
  * Browne (2022) Ryan Browne. 2022\.  Trading in NFTs spiked 21,000% to more than $17 billion in 2021, report says.  https://www.cnbc.com/2022/03/10/trading-in-nfts-spiked-21000percent-to-top-17-billion-in-2021-report.html.
  * Casale-Brunet et al. (2021) S Casale-Brunet, P Ribeca, P Doyle, and M Mattavelli. 2021\.  Networks of Ethereum Non-Fungible Tokens: A graph-based analysis of the ERC-721 ecosystem. In _2021 IEEE International Conference on Blockchain (Blockchain)_. IEEE, 188–195.
  * Das et al. (2021) Dipanjan Das, Priyanka Bose, Nicola Ruaro, Christopher Kruegel, and Giovanni Vigna. 2021.  Understanding Security Issues in the NFT Ecosystem.  _arXiv preprint arXiv:2111.08893_ (2021).
  * Dowling (2022) Michael Dowling. 2022\.  Is non-fungible token pricing driven by cryptocurrencies?  _Finance Research Letters_ 44 (2022), 102097.
  * Google (2022) Google. 2022.  Google Search Trend - Hashmasks.  https://trends.google.com/trends/explore?date=2021-01-01%202022-02-28&q=Hashmasks.
  * Graham (2022) Michelai Graham. 2022\.  Crypto Airdrops Explained.  https://boardroom.tv/airdrop-crypto-nft-explained/.
  * Gupta (2022) Alinda Gupta. 2022\.  What Are Whales and How Do They Manipulate Cryptocurrency?  https://www.jumpstartmag.com/what-are-whales-and-how-do-they-manipulate-cryptocurrency/.
  * Hollander (2020) Luit Hollander. 2020\.  Understanding event logs on the Ethereum blockchain.  https://medium.com/mycrypto/understanding-event-logs-on-the-ethereum-blockchain-f4ae7ba50378.
  * Li (2021) Joyce Li. 2021.  CryptoPunk Prices See 53% Increase After Ethereum Whales’ Purchase Frenzy.  https://hypebeast.com/2021/8/cryptopunk-price-rises-after-ethereum-whales-purchase-frenzy.
  * Major (2021) Jordan Major. 2021\.  Global interest in NFTs soars by 426% in a month, data shows.  https://finbold.com/global-interest-in-nfts-soars-by-426-in-a-month-data-shows/.
  * Mattei (2021) Shanti Escalante-De Mattei. 2021\.  The Year of the NFT: How an Emerging Medium Went Mainstream in 2021.  https://www.artnews.com/list/art-news/artists/2021-year-of-the-nft-1234614022/.
  * Nadini et al. (2021) Matthieu Nadini, Laura Alessandretti, Flavio Di Giacinto, Mauro Martino, Luca Maria Aiello, and Andrea Baronchelli. 2021.  Mapping the NFT revolution: market trends, trade networks, and visual features.  _Scientific reports_ 11, 1 (2021), 1–11.
  * NFTBank (2022) NFTBank. 2022.  NFTBank.  https://nftbank.ai/.  Accessed: 2022-05-18.
  * NFTGo (2022a) NFTGo. 2022a.  Market Overview.  https://nftgo.io/analytics/market-overview/.  Accessed: 2022-05-18.
  * NFTGo (2022b) NFTGo. 2022b.  NFTGo.  https://nftgo.io/.  Accessed: 2022-05-18.
  * Onanuga (2021) Tola Onanuga. 2021\.  Bitcoin whales: what are they – and how are they affecting the cryptocurrency’s price?  https://www.businessinsider.com/bitcoin-whales-the-key-facts-figures-you-need-to-know-2021-1.
  * Shaw (2021) Anny Shaw. 2021\.  Sorry to burst your bubble: NFT prices slump 70%.  https://www.theartnewspaper.com/2021/04/08/sorry-to-burst-your-bubble-nft-prices-slump-70percent.
  * Trash (2021) Corporate Trash. 2021\.  What is Bored Ape Yacht Club? The Ape NFT Transforming NFTs.  https://momentranks.com/blog/what-is-bored-ape-yacht-club-the-ape-nft-transforming-nfts.
  * Victor and Weintraud (2021) Friedhelm Victor and Andrea Marie Weintraud. 2021.  Detecting and quantifying wash trading on decentralized cryptocurrency exchanges. In _Proceedings of the Web Conference 2021_. 23–32.
  * von Wachter et al. (2022) Victor von Wachter, Johannes Rude Jensen, Ferdinand Regner, and Omri Ross. 2022.  NFT Wash Trading: Quantifying suspicious behaviour in NFT markets.  _arXiv preprint arXiv:2202.03866_ (2022).
  * Wang et al. (2021) Qin Wang, Rujia Li, Qi Wang, and Shiping Chen. 2021\.  Non-fungible token (NFT): Overview, evaluation, opportunities and challenges.  _arXiv preprint arXiv:2105.07447_ (2021).
  * Williams (2022) Chris Williams. 2022\.  A Meebit NFT Sold for $49.5M in Ethereum—But There’s a Catch.  https://cryptobriefing.com/meebit-nft-sold-for-49-5m-ethereum-with-catch/.

##  Appendix A Appendix

###  A.1. Ethics

Due to anonymous nature of crpytocurrency wallet addresses, this work does not
raise any ethical issues.

###  A.2. NFT transaction data collection approach

In Ethereum, there are two types of transactions: external and internal
transactions. External transactions are initiated by external accounts.
Transaction information such as receiver’s and sender’s addresses and
transferred amount in ETH is recorded in the blockchain and readily available
to anyone for reference (e.g., etherscan.io).

Internal transactions are results of preceding transactions like byproducts of
smart contract functionality, and they are not recorded. In this case, the
transaction only records the truth that a message sender made a call on smart
contract functionality, and not detailed behaviors of what the contract did.
Unfortunately, NFT functionalities are implemented in the manner of smart
contracts, making it infeasible for the extraction of information on NFT
transactions through simple referencing of transaction data. Instead, we can
use the event logs, which are the result of LOG opcodes being executed in the
EVM (Ethereum Virtual Machine) to gain a clear view of smart contract
behaviors. Commonly, smart contracts record their activities as event logs,
and NFT contracts are not exceptions. For instance, when there is an ownership
change for an NFT item, the details about the transmission such as owner
address and token identifier are recorded as an event log. By collecting such
events from NFT contracts, we can trace how NFT transactions are working in
the NFT ecosystem. The transfer log, which is a type of the event log,
explains how many NFTs are transferred and who sends the tokens to whom
consisting of a number of entities. From the transfer log, we can extract
information of ownership change for an NFT item, including the token contract
address, the sender’s and receiver’s accounts, and transferred token number. A
transfer log from one of the most popular NFT projects is described in Listing
LABEL:l:transfer_log as an example. Diving into the specifics of each of the
log entities, the ‘address’ indicates the token contract address to launch the
transfer. ‘Topics’ refers to a combination of four entities including the
sender’s and receiver’s accounts and transferred token number. According to
the NFT standard, one NFT is transferred at a time.

Most NFT buyers make payments in ETH or using several fungible tokens.
However, the actual transaction price for an NFT sale is not included in a NFT
transfer log. To figure out the transaction price for NFT sales, it is
necessary to refer to additional data. More specifically, when the NFT buyer
pays in ETH, the payment of sales is recorded in the external transaction. If
the buyer paid in fungible tokens, a transfer log of fungible token explains
how much tokens are transferred as payment for the NFT sales. Thus, if the
transfer value of a transaction lists a value greater than 0 ETH, we consider
the amount of transferred ETH as the actual price of NFT sales. If not, we
look up transfer logs from any of fungible tokens transferred from an NFT
buyer, and take the amount of transferred fungible tokens as the transaction
price of NFT sales. Lastly, if there are no cryptocurrency payments from an
NFT buyer for NFT transaction, the transaction is considered to just a
transfer, not sale.

###  A.3. Supplementary information of analysis

Year/Month | # whales | # dolphins | # minnows | Total
---|---|---|---|---
21/01 | 6 | 590 | 5,368 | 5,964
21/02 | 13 | 1,237 | 11,255 | 12,505
21/03 | 20 | 1,977 | 17,973 | 19,970
21/04 | 25 | 2,465 | 22,412 | 24,902
21/05 | 38 | 3,755 | 34,137 | 37,930
21/06 | 45 | 4,456 | 40,601 | 45,012
21/07 | 61 | 5,995 | 54,499 | 60,555
21/08 | 117 | 11,578 | 105,257 | 116,952
21/09 | 179 | 17,745 | 161,318 | 179,242
21/10 | 248 | 24,530 | 223,001 | 247,779
21/11 | 292 | 28,936 | 263,047 | 292,275
21/12 | 362 | 35,804 | 325,396 | 361,662
22/01 | 403 | 39,868 | 362,443 | 402,714
22/02 | 430 | 42,593 | 387,204 | 430,227
Table 2. Number of each trader type on each month Figure 12. Average price of The n Project NFTs traded at that time. Red line indicates when malicious behaviors occurred. Rank | Collection | # tokens
---|---|---
1 | Art Blocks | 9,548
2 | The Sandbox | 9,549
3 | LOSTPOETS | 4,766
4 | Meebits | 2,744
5 | Hashmasks | 2,671
6 | Mutant Ape Yacht Club | 1,621
7 | RTFKT CLONE X + Murakami | 1,610
8 | CryptoPunks | 1,599
9 | CyberKongz VX | 1,558
10 | Punks Comic | 1,528
Table 3. Top 10 collections and corresponding number of tokens held by whales t = YY/MM | % (t-1) whale remains in (t) whale | % (t) whale comes from
---|---|---
Whale | Dolphin | Minnow | No activity before
21/02 | 50% | 23% (3) | 38% (5) | 0% (0) | 38% (5)
21/03 | 62% | 40% (8) | 25% (5) | 0% (0) | 35% (7)
21/04 | 70% | 56% (14) | 28% (7) | 0% (0) | 16% (4)
21/05 | 76% | 50% (19) | 37% (14) | 3% (1) | 11% (4)
21/06 | 100% | 84% (38) | 13% (6) | 0% (0) | 2% (1)
21/07 | 80% | 59% (36) | 21% (13) | 3% (2) | 16% (10)
21/08 | 59% | 31% (36) | 39% (46) | 2% (2) | 28% (33)
21/09 | 86% | 56% (101) | 31% (56) | 2% (4) | 10% (18)
21/10 | 89% | 65% (160) | 29% (73) | 0% (1) | 6% (14)
21/11 | 93% | 79% (230) | 17% (49) | 1% (3) | 3% (10)
21/12 | 84% | 67% (244) | 27% (96) | 1% (2) | 6% (20)
22/01 | 88% | 79% (319) | 15% (60) | 0% (1) | 6% (23)
22/02 | 89% | 83% (357) | 11% (48) | 0% (0) | 6% (25)
Table 4. Percentage of each group where whale belonged to YY/MM | Whale | Dolphin | Minnow
---|---|---|---
# actives (%) | Avg # transactions | # actives (%) | Avg # transactions | # actives (%) | Avg # transactions
21/01 | 6 (100%) | 218 | 586 (100%) | 28 | 5323 (100%) | 7
21/02 | 13 (100%) | 138 | 1170 (95%) | 19 | 8448 (76%) | 5
21/03 | 19 (95%) | 19 | 1475 (75%) | 13 | 10599 (59%) | 4
21/04 | 22 (88%) | 30 | 1322 (54%) | 12 | 8839 (40%) | 4
21/05 | 36 (95%) | 154 | 2872 (76%) | 14 | 17009 (50%) | 5
21/06 | 26 (58%) | 90 | 2607 (59%) | 18 | 14164 (35%) | 6
21/07 | 53 (87%) | 43 | 4373 (73%) | 20 | 24765 (46%) | 7
21/08 | 114 (97%) | 123 | 11036 (95%) | 27 | 75551 (72%) | 7
21/09 | 148 (83%) | 58 | 14874 (84%) | 17 | 99257 (62%) | 5
21/10 | 189 (76%) | 41 | 18211 (74%) | 10 | 120058 (54%) | 3
21/11 | 196 (68%) | 34 | 18583 (64%) | 7 | 95574 (36%) | 3
21/12 | 258 (72%) | 46 | 24175 (68%) | 8 | 132279 (41%) | 3
22/01 | 292 (73%) | 34 | 27361 (69%) | 6 | 109103 (30%) | 2
22/02 | 275 (64%) | 26 | 21472 (50%) | 4 | 78029 (20%) | 2
Table 5. Transaction statistics of active users on each month # transfer | 1 | 2 | 3 | 4 |  $\geq$ 5
---|---|---|---|---|---
Avg profit | 7848.8 | 8604.7 | 10750.1 | 9274.1 | 14675.6
Table 6. Average Profit from mint according to number of transfer Time duration | ¡ 1 day | 1 day - 1 week | 1 week - 1 month |  $\geq$ 1 month
---|---|---|---|---
Avg profit | 3372.3 | 4285.7 | 4660.6 | 10585.3
Table 7. Average Profit from mint according time duration before first sell


# Learning Emergent Gaits with Decentralized Phase Oscillators:
on the role of Observations, Rewards, and Feedback

Jenny Zhang, Steve Heim, Se Hwan Jeon, Sangbae Kim All authors are with the
Biomimetic Robotics Lab, Massachusetts Institute of Technology, Cambridge,
Massachusetts, USA (e-mails<EMAIL_ADDRESS><EMAIL_ADDRESS>{sehwan<EMAIL_ADDRESS>
###### Abstract

We present a minimal phase oscillator model for learning quadrupedal
locomotion. Each of the four oscillators is coupled only to itself and its
corresponding leg through local feedback of the ground reaction force, which
can be interpreted as an observer feedback gain. We interpret the oscillator
itself as a latent contact state-estimator. Through a systematic ablation
study, we show that the combination of phase observations, simple phase-based
rewards, and the local feedback dynamics induces policies that exhibit
emergent gait preferences, while using a reduced set of simple rewards, and
without prescribing a specific gait. The code is open-source, and a video
synopsis available at https://youtu.be/1NKQ0rSV3jU.

##  I INTRODUCTION

Quadrupedal animals exhibit a variety of gaits, or pattern of footfalls, and
the choice of gaits has been linked to energetics, speed, morphology, etc. [1,
2, 3]. Quadrupedal robot controllers, on the other hand, are typically
designed around a fixed contact sequence, for a number of reasons. First, the
energetic difference between gaits for robots has been shown to be
inconsequential [4, 5]. Perhaps more importantly, using a single gait greatly
simplifies the controller design; for conventional model-predictive control,
pre-specifying the contact sequence allows a significantly simpler problem to
be solved [6, 7, 8]. Model-free reinforcement learning (RL) side-steps the
computational burden of reasoning over different contact sequences.
Nonetheless, RL typically requires extensive reward shaping and
regularization, which is often encoded in a time-indexed reference trajectory
based on a fixed gait, either as a nominal trajectory [9, 10, 11] or a reward
[12]. A periodic clock observation is then necessary to maintain the Markov
property of the reference.

Figure 1:  We augment the robot state with four decentralized phase
oscillators, one per leg. Blue arrows in the diagram indicate the three
oscillator-related signals: first, observations of the oscillator phase to the
policy to make feed-forward Markov. Second, the phase-based reward encodes the
general properties of gaits. Finally, the ground reaction force
($F^{\text{GRF}}$) is used as feedback, which we view as the observer feedback
that allows us to interpret the phase oscillators as a state observer of
whether each foot should be in stance or swing. The scissors represent our
ablation study.

It is generally difficult to design shaping rewards that capture a general
high-level notion, in our context “locomote with a regular gait”, without
over-specifying the solution. [13] proposed a simple phase-based reward to
encourage stance or swing at any phase difference for their bipedal robot
Cassie, and demonstrated this policy can then track any desired gait by simply
adjusting the phase difference between the legs accordingly. Similar work has
applied this type of reward to quadrupeds as well [14, 15]. In these cases,
the actual gait choice is removed from the policy, which is essentially
treated as a low-level policy. Instead, the phase difference needs to be
specified by the user or a high-level policy [16].

A popular approach to achieving specific phase differences is to augment the
state space with a network of coupled phase oscillators called central pattern
generators (CPGs) [17]. In its simplest form, the dynamics of the oscillators
are designed to exhibit stable limit cycles, and the resulting phases are
mapped to values relevant to the robot controller, such as desired kinematics.
In essence, this is a generalization of time-indexed trajectories. Despite the
often simpler and lower-dimensional design space of the phase oscillators
(compared to the space of reference trajectories in joint space), it can still
be difficult to design the oscillator dynamics and mapping [18]. Recent works
have instead opted to learn the coupling, while fixing the mapping from the
phase to robot states [19, 16]. This approach complements the phase-based low-
level policies, and CPGs are often interpreted as feed-forward reference
generators [20, 21]. An alternative view, recently proposed by [22],
interprets the CPG as an observer, with reflex-like sensory feedback playing
the role of the observer gain.

We present an implementation of _decentralized_ phase oscillators based on the
work of [23], and, based on the observer-interpretation of [22], treat the
oscillator phase as a loose estimate of whether each leg should be in swing or
stance. Based on this interpretation, we use a reward similar to [13] to
encourage the policy and phase oscillators to entrain. We further enrich the
oscillator dynamics with an offset term to entrain the phases for standing
still while retaining the same reward structure.

The three key signals afforded by our architecture, represented in Fig. 1 with
blue arrows, are the _phase observation_ which renders feed-forward policies
Markov, the _phase rewards_ that encode the high-level gait properties of duty
factor and nominal frequency, and the _local feedback coupling_ through the
ground reaction force coupling the oscillator dynamics and the policy. We will
show the importance of each of these through a systematic ablation.

##  II PRELIMINARIES

We introduce gaits and pattern generation based on phase oscillators. We
assume working knowledge of reinforcement learning and quadrupedal robots; for
the interested reader, we recommend [24, 25].

###  II-A Gaits

For the scope of this paper, we restrict ourselves to quadrupedal gaits, which
are defined by the pattern of stance phases [3]. In the rest of this paper, we
will define gaits by the relative phase difference (RPD); using the right
front (RF) foot as the reference, we first define the gait cycle length as the
time between consecutive RF foot touchdowns, normalized to $2\pi$. We then
calculate the RPD as a 3D vector, composed of the time difference between
touchdowns of the left front (LF), right hind (RH), and left hind (LH) feet to
the RF reference foot, normalized by the gait cycle length. Each gait is fully
defined by these three values. The ideal symmetric gaits Trot $(\pi,\pi,0)$,
Pace $(\pi,0,\pi)$, Bound $(0,\pi,\pi)$, Pronk $(0,0,0)$ are marked in Fig. 4.

We classify gaits by averaging the RPD over two gait cycles every 5 seconds,
then taking the closest ideal gait within the set of symmetric quadruped gaits
by Euclidean distance in RPD space. If the distance to the closest ideal gait
exceeds 2, we classify the RPD as being in Transition.

###  II-B Phase-based Pattern Generation

We will distinguish between clocks, central pattern generators (CPGs) without
feedback, CPGs with feedback, and decentralized oscillators (which are driven
by feedback by definition). Each of these is a special case of a system of
oscillators with state $\phi\in\left[0,2\pi\right)^{n}$, where
$n\in\mathbb{N}$ is the number of oscillators, and

$\begin{split}\dot{\phi}_{i}&=\omega+f(\phi_{j},x_{j}),\ j\in C(i)\\\
u&=g(\phi_{j},x_{j})\\\ \text{for }i&\in\\{1,\ldots,n\\},\text{ and
}C(i)\subseteq\\{1,\ldots,n\\}\end{split}$ (1)

where $\omega$ is a nominal frequency, $x$ is the system state, $f()$ is a
function that determines the dynamics properties, $g()$ is a mapping function
to some control-relevant input $u$, and the set $C()$ indicates which
oscillators are directly coupled. We will generally consider the case where
there is one oscillator per leg, that is $n=4$ for a quadruped.
Clocks: $f()=0$
This degenerate choice for $f()$ reduces the oscillator to a clock with a
constant growth rate $\omega$. Though sometimes called CPGs [9], we
distinguish this setting as it makes no use of the state and dynamics of the
oscillator: the burden is placed on designing the map $g()$. This is the
minimal form needed to make a cyclic feed-forward pattern Markov. [13] use
this setup, and learn the mapping function $g(\phi,x_{j})$, using the clock as
both an observation and to design a simple reward function.
CPGs without feedback: $f()=f(\phi_{j})$
This form provides a pure feed-forward pattern, and allows the engineer to
design the oscillator dynamics, such as limit cycles and convergence
properties [18, 26], unencumbered by the physical dynamics of the robot. This
can significantly simplify the design of a high-level controller that switches
between multiple $f()$ [27]: the engineer can ensure smooth transitions by
simply enforcing the desired properties in the phase oscillator space, and
given a smooth mapping $g(\phi)$, those properties are retained in the
generated reference trajectory. However, once the phase has converged to the
limit cycle, this setup effectively acts as a clock, as the phase oscillator
state cannot be perturbed off the limit cycle.
CPGs with feedback: $f()=f(\phi_{j},x_{j})$
Coupling the phase oscillator and physical states fully exploits the dynamics
properties of the phase oscillator, but also makes it more difficult to design
useful dynamics. Nonetheless, even relatively simple, local feedback has been
shown to greatly improve performance [28, 29]. Several studies have relied on
RL to learn $f()$ [19, 16]. [22] also use this form, but re-interpret it as a
state estimator, with the state feedback acting as the observer gain. We rely
heavily on this interpretation.
Decentralized Oscillators: $C(i)=\left\\{i\right\\}$
This is a special case where $f()=f(\phi_{i},x_{i})$: each oscillator is only
affected by feedback from itself and sensory information from its
corresponding leg. Despite lacking direct coupling between oscillators, such
systems can still synchronize due to the indirect feedback coupling through
local physical interaction [23, 30]. Our work uses this setup, and in
particular a form based on the so-called _Tegotae_ feedback model proposed by
[23]:

$f()=-\sigma F^{\text{GRF}}_{i}\left(\cos\left(\phi_{i}\right)\right)$ (2)

where $\sigma$ is a feedback gain, and $F^{\text{GRF}}_{i}$ is the ground
reaction force felt at foot $i$.

This model captures the essence of a gait (frequency, swing/stance phases)
without specifying any details. Indeed, it would be difficult to predict any
specific gait when looking at Eq. (2). This type of decentralized oscillators
has been shown to not only converge to gaits, but also switch gaits depending
on forward velocity [23, 31].

We use a variation of this oscillator with the interpretation proposed by
[22], that each phase is a latent observation of whether the corresponding leg
should be in stance or swing, and $\sigma$ in Eq. (2) is an observer gain.

##  III Implementation details

We train policies and collect simulation data in IsaacGym, using a fork of
legged_gym by [24], and proximal policy optimization. Our code is open-sourced
[32] and can be referenced for implementation details, hyperparameters, and
instructions to reproduce our results.

###  III-A Robot

We use the MIT Mini Cheetah robot [33] in simulation and for hardware
transfer. For all results presented in the paper, which rely on large-scale
data collection for statistical analysis, we use simulation results.
Preliminary tests on hardware can be seen in the supplementary video.
Observation space: The policy is given standard state observations composed of
the base angular velocity, projected gravity vector, joint positions and
velocities, the previous actions, and the desired base linear and angular
velocity commands. When not ablated, the observations also include the phase
oscillator observations
$\left[\sin\left(\phi_{i}\right),\cos\left(\phi_{i}\right)\right],i\in\\{\text{FR},\text{FL},\text{HR},\text{HL}\\}$.
The critic is additionally given the current oscillator velocity as privileged
information.
Action space: The policy outputs desired joint positions at 100 Hz, which are
fed into a low-gain PD-controller with $K_{p}=20$ and $K_{d}=0.5$ running at
500 Hz.
Rewards: Following standard conventions, we use positive rewards for the
command tracking error, orientation error, and a minimum base-height error,
all passed through a squared exponential function. In addition, we regularize
with negative rewards on the square of the torques, first and second-order
action smoothness, and the hip abduction/adduction joints deviating from the
resting position. Finally, body collisions with the ground terminate the
episode and incur a flat penalty.

When not ablated, we also add the phase-based reward

$r_{\text{gait}}(x)=-F^{\text{GRF}}_{i}\sin\left(\phi_{i}\right)$ (3)

which penalizes foot contact when $\phi_{i}\in\left[0,\pi\right)$ and
encourages contact during $\phi_{i}\in\left[\pi,2\pi\right)$. Due to the
dynamics of the phase oscillator, this simple reward encourages two effects:
first, to learn a policy with roughly the oscillator nominal frequency
$\omega$, and second, to have consistent periods of contact with all feet. We
will see that the policy also learns to actively use the $F^{\text{GRF}}$ to
‘guide’ the oscillators into stable gaits if the coupling is included during
training.
Ground Reaction Force Estimation: Each $F^{\text{GRF}}_{i}$ value is
normalized by robot mass and clamped between $\left[0,1\right]$, such that
$\sigma$ values can be kept consistent for robots of different sizes. We also
found this mitigated issues caused by the highly inaccurate contact force
estimates in IsaacGym.

###  III-B Decentralized phase oscillators

We add an offset term $\xi$ to the original _Tegotae_ feedback model from Eq.
(2), so our oscillator dynamics become

$\dot{\phi}_{i}=2\pi\left(\omega-\sigma
F^{\text{GRF}}_{i}\left(\cos\left(\phi_{i}\right)+\xi\right)\right)$ (4)

Based on biological observations [34] and trial and error, the parameters
$\omega$, $\sigma$, and $\xi$ are set to

$[\omega,\,\sigma,\,\xi](v_{x})=\begin{cases}[1,\,4,\,1]&\text{if }|v_{x}|\leq
0.5\\\ [\min\\{1.5+|v_{x}|,4\\},\,1,\,0]&\text{otherwise}\end{cases}$ (5)

where $v_{x}$ is the commanded forward velocity. Each phase $\phi_{i}$ is
uniformly randomized at the start of each episode.

Standing still is a special case for which we want all oscillators to settle
to a stable point in stance. Rather than activating special rewards to
minimize joint velocities when the commanded velocity is low, we change the
oscillator parameters to smoothly introduce a stable fixed point. This allows
the fixed point to be closer to the middle of the stance range
$\left[\pi,2\pi\right)$ without using an excessively large $\sigma$ value,
which can destabilize the oscillator by chattering between stance and swing
phases. This smoothing effect is illustrated in Fig. 2, with the blue line
showing the actual phase velocity vs. phase function used during training for
zero velocity commands.

Figure 2:  To illustrate the oscillator fixed points (where $\dot{\phi}=0$),
we graph the oscillator dynamics with $F^{\text{GRF}}=0.25$, where each leg
supports a quarter of the body weight. For the curve with coupling $\sigma=4$
and offset $\xi=0$, the point at phase $\phi=2\pi$ is only marginally stable,
and would not settle in stance. When $\xi=0$, the limit of the fixed point as
$\sigma$ approaches $+\infty$ is $3\pi/2$, but drastically increasing $\sigma$
alone introduces discrete jumps in $\phi$ that are destabilizing. Setting
$\xi=1$ with $\sigma=4$ caps $\dot{\phi}$ at the nominal $2\pi\omega$, and
places the fixed point directly in the middle of the stance phase. This
formulation helps to achieve standing without stepping in place.

##  IV Observation, Reward, Coupling Ablation

We perform an ablation of the phase-based observations, rewards, and coupling
by cutting their respective signals during training, see Fig. 1. We will refer
to each permutation as ORC(xxx), where each entry is a boolean indicating
whether the corresponding signal is present or not during training. For
example, ORC(110) indicates a policy that was trained with the phase
observations and phase-based rewards, but with the coupling term $\sigma=0$.
Note that ORC(110) is essentially the setting used by [13].

We skip the permutation ORC(001) as it is equivalent to ORC(000). For all
other permutations, we train and statistically analyze 10 policies each to
answer three questions:
IV-A) _How do the signals influence leg-load distribution?_
IV-B) _How do the signals influence gait emergence?_
IV-C) _How does each signal affect overall stability?_

###  IV-A Balanced Leg Use

Ideally, a policy should use all four legs in a consistent and balanced
manner. ORC(x0x) policies either don’t have oscillator observations or are not
encouraged to use them in any particular way. ORC(01x) policies are non-Markov
as it appears to receive inconsistent rewards for performing the same action
given its observable state. ORC(11x) policies are trained with both oscillator
observations and rewards to encourage swing and stance during specific ranges
of the oscillators, and have enough information to match phases with rewards
related to ground contact. We expect to see more consistent leg use with the
policies trained using ORC(11x), and more variability in the other policies.

For each policy, 50 robots are initialized with random oscillator phases and
rolled out in simulation with 1 m/s forward velocity command. We calculate the
average $F^{\text{GRF}}_{i}$ over the entire 10 second episode for each leg
separately. Randomly initialized oscillator phases result in different
behavior in the 50 agents for ORC(1xx) policies. Results from all 500 runs
belonging to each ORC configuration are aggregated into the same dataset. Each
violin plot in Fig. 3 shows the distribution of average $F^{\text{GRF}}_{i}$
experienced by each leg. Since $F^{\text{GRF}}_{i}$ is normalized by body
weight, perfectly balanced leg use yields average $F^{\text{GRF}}_{i}=0.25$
for all legs.

Figure 3: Each ORC configuration has 500 agents (50 per re-trained policy),
and $F^{\text{GRF}}$ is averaged for each leg across the entire episode.
ORC(11x) policies show much more consistent and balanced leg use compared to
all other configurations, which tend to exhibit 2 or 3 legged gaits.

We can see that for all ORC configurations without observations and rewards
working together, the distributions of $F^{\text{GRF}}_{i}$ are very wide and
have significant clusters around 0 for at least one leg out of four. This
agrees with our observation that the policies trained with those ORC
configurations often result in 2 or 3 legged gaits, where some feet are
dragging or always kept in the air, and therefore experiencing little to no
$F^{\text{GRF}}_{i}$ over the roll-out. The pattern of two clusters around the
extremities of the distributions for each leg arises from inconsistent
policies after repeated training with the same setting. In some policies, the
RF foot might always be in the air, while in others the LF foot might always
be in the air.

The phase observations and rewards strongly encourage ORC(11x) policies to use
all four legs cyclically without specifying the exact desired gait. Both
configurations yield distributions that are roughly centered around average
$F^{\text{GRF}}_{i}=0.25$, showing that the leg use is well balanced over all
trials and policies.

ORC(110) experiments have slightly larger range compared to ORC(111), which
may be attributed to some randomly initialized asymmetric gaits requiring
higher $F^{\text{GRF}}_{i}$ on some legs compared to the others to track,
since it cannot converge to a more symmetric gait without coupling.

###  IV-B Emergence of Gaits

To evaluate gait emergence, we focus only on ORC(110) and ORC(111) policies,
since we see from Subsection IV-A that other permutations rarely yield well-
defined gaits. Each experiment evaluates a single policy chosen at random, and
includes 500 runs with randomized initial oscillator values, rolled out over
40 seconds with a 1 m/s forward velocity command. We calculate the RPD as
described in Subsection II-A, and assess gait preference by evaluating the RPD
distribution at the end of the roll-out. We verify that ORC(111) policies
don’t ignore the phase observation by deactivating the coupling at execution
time. We further probe the role of the feedback coupling by activating it for
an ORC(110) policy, which was trained with no coupling.

Both ORC(111), visualized in Fig. 4a), and ORC(110) policies match their RPD
to the oscillator phases when the coupling is set to $\sigma=0$, as expected.

When evaluating ORC(111) with coupling $\sigma=1$ (same as during training),
the gaits converge within 10 seconds to their final preferred state, and we
observe mostly trotting and pronking gaits in tight clusters on Fig. 4b).
Since the policy’s actions influences the oscillators through ground contacts,
it can learn to manipulate the randomly initialized oscillators and phase lock
into desirable gaits faster.

Figure 4: Initial and final relative phase difference RPD points are shown for
500 randomly initialized runs in each experiment. ORC(111) evaluated with
coupling $\sigma=0$ tracks the initial phases and cannot converge to any
specific gait. ORC(111) evaluated with $\sigma=1$ exhibits strong convergence
to trot and pronk, while ORC(110) evaluated with $\sigma=1$ exhibits some
convergence around trot, but is more spread out compared to the final ORC(111)
RPD.

Figure 5: The distribution of gaits for 500 runs of ORC(111) with $\sigma=1$
settles into both trot and bound quickly within 10 seconds.

When evaluating ORC(110) with $\sigma=1$, the policy continues to track the
oscillators, which are now also modulated by the feedback term that was not
seen during training. The gaits that emerge from this experiment have regions
of attraction dictated by the oscillator dynamics, which highlights the role
of the oscillator in determining the preferred gait. However, as shown in Fig.
4c), the final gaits are clustered further away from the ideal phase
differences of symmetric gaits compared to the ORC(111) policy shown in Fig.
4b). We also see in Fig. 5 and Fig. 6 that the ORC(110) (evaluated with
coupling) takes nearly 40 seconds to settle into its preferred gait, whereas
the ORC(111) policy settles in roughly 10 seconds.

Figure 6: The distribution of gaits for 500 runs of ORC(110) with coupling
$\sigma=1$ settles into trot slowly, with more environments transitioning at
the beginning but others continuing to slowly converge toward trot as runs
with RPD initialized further away become more trot-like over time due to the
oscillator dynamics.

Fig. 7 shows a pace to trot transition of one of the runs from Fig. 5 by
plotting all $F^{\text{GRF}}_{i}$ from 0-10 seconds with respect to the
oscillator phases of the RF reference leg. The thick blue lines show
$F^{\text{GRF}}_{i}$ during the first gait cycle and the thick orange lines
show $F^{\text{GRF}}_{i}$ during the last gait cycle. As more gait cycles
occur, the phase differences between the leg oscillators change, indicated by
the red and green dots showing the RF phase value at the time-step when the
corresponding leg’s oscillator crosses 0 and $\pi$ respectively. Those dots do
not exactly overlap for the RF leg because of discrete time-step errors. The
$F^{\text{GRF}}_{i}$ of each leg follows its own phases, and over time settles
into a trotting gait, with diagonal legs being in phase.

Figure 7: The ground reaction force $F^{\text{GRF}}_{i}$ is plotted for each
foot relative to $\phi_{\text{RF}}$, with initial gait cycles in blue and
progressing through time to orange. The phase $0$ and $\pi$ crossings of each
leg are shown with red and green dots, respectively. The bold cycles are the
initial and final cycles, which show this run starting in pace with lateral
feet in phase and ending in trot with diagonal feet in phase. All swing and
stance behavior obeys each leg’s respective oscillator phase well, with non-
zero $F^{\text{GRF}}$ falling between the green dot $\pi$ crossings and red
dot $0$ crossings in every gait cycle.

###  IV-C Disturbance Rejection

To compare the overall effect on stability, we test how well each policy can
reject a planar velocity impulse applied to the body while being commanded to
run at 3 $\mathrm{m}\mathrm{/}\mathrm{s}$, with oscillators being initialized
at random and allowed a 5 second settling period. To ensure that we apply
perturbations at all phases of the gait, we run 1800 trials per policy, and
stagger the perturbations: every 0.01 seconds, a ball of perturbations spaced
at 10 degree intervals is applied to a different set of 36 robots, over a
period of 0.5 seconds. As all policies exhibit a frequency of roughly 4 Hz at
this commanded speed111We verified this frequency via a fast Fourier transform
on the joint positions for policies without phase observations., staggering
the perturbations ensures that we apply perturbations at different phases of
the gait, with different numbers of feet on the ground. We then calculate the
mean and standard deviation of the failure rate across all ten policies
learned for each permutation of ORC(xxx).

The failure rate means and standard deviations, reported in Table I, show that
policies trained under ORC(111) consistently have a lower failure rate
compared to all other permutations. Surprisingly, toggling on phase-based
rewards has a stronger impact than the phase-based observations, even for
cases such as ORC(010) and ORC(011) where the reward is not Markov. We
conjecture that, when the reward is not Markov, it should be viewed as a
stochastic reward that still discourages chattering contacts or dragging feet,
despite not signaling any specific gait schedule.

TABLE I: Failure rate after velocity impulse disturbance (All values in %) ORC |  | 000 | 100 | 101 | 010 | 110 | 011 | 111
---|---|---|---|---|---|---|---|---
Impulse magnitude [$\mathrm{m}\mathrm{/}\mathrm{s}$] | 1.5 | 0.0 $\pm$ 0.0 | 0.0 $\pm$ 0.0 | 0.1 $\pm$ 0.2 | 0.0 $\pm$ 0.0 | 0.0 $\pm$ 0.0 | 0.0 $\pm$ 0.0 | 0.0 $\pm$ 0.0
2.0 | 1.7 $\pm$ 2.5 | 4.1 $\pm$ 5.2 | 0.9 $\pm$ 1.2 | 0.0 $\pm$ 0.0 | 0.0 $\pm$ 0.1 | 0.0 $\pm$ 0.0 | 0.0 $\pm$ 0.1
2.5 | 11.0 $\pm$ 13.0 | 10.6 $\pm$ 9.1 | 7.2 $\pm$ 6.3 | 0.7 $\pm$ 0.7 | 0.7 $\pm$ 1.0 | 0.3 $\pm$ 0.4 | 0.4 $\pm$ 0.4
3.0 | 16.3 $\pm$ 12.4 | 17.5 $\pm$ 8.9 | 16.0 $\pm$ 10.2 | 3.2 $\pm$ 3.0 | 4.1 $\pm$ 3.6 | 3.2 $\pm$ 3.1 | 2.2 $\pm$ 1.7
3.5 | 25.0 $\pm$ 13.5 | 29.7 $\pm$ 12.7 | 24.0 $\pm$ 17.1 | 10.6 $\pm$ 8.0 | 13.8 $\pm$ 7.8 | 13.1 $\pm$ 9.2 | 7.4 $\pm$ 4.0

##  V Discussion

We presented an augmentation of the quadruped robot state space using one
decentralized phase oscillator per leg, with a simple feedback coupling to the
ground reaction force of the corresponding leg, which can be interpreted as an
observer gain. Through a systematic ablation study, we investigated the
importance of each phase-related signal: observations of the oscillator phase,
phase-based rewards to encourage distinct swing and stance phases, and
feedback coupling.

Overall, ORC(111) policies trained with all three signals demonstrated the
fastest convergence to well-defined gaits, and were consistently the most
robust to large impulse perturbations. We did not find significant differences
in local stability222Analysis of Floquet multipliers and rate of entropy decay
were evaluated. between the policies, which matches our experience in hardware
that local stability is not a useful proxy for legged system ‘stability’.
Nonetheless, preliminary sim-to-real trials (see supplementary video) show
significantly better transfer with the observation signal. [19] also reported
more reliable sim-to-real transfer when learning a CPG with feedback, although
in their case the phase is directly mapped to desired kinematics with a pre-
designed mapping.

ORC(110) policies trained with phase observations and rewards but no coupling
showed only slightly worse performance to those with coupling, while tracking
the gait defined by whichever phase-difference the oscillators are initialized
in, similar to the results of [13]. However, although activating the coupling
during evaluation does cause these policies to converge toward symmetric
gaits, the convergence time is nearly four times slower. This observation
suggests that the dual roles of control and estimation are not fully separated
between the oscillators and the policy, as it is in the linearized model
presented by [22]: the policy is affected by the oscillator state, but can
also learn to actively drive it towards a more stable gait if trained with the
coupling active. This also matches the observation of [20] that CPGs may act
as both an observer and a pattern generator.

Surprisingly, we found that the reward signal has a stronger effect on
stability than the phase observations, despite being non-Markov in some
ablations. Nonetheless, only when both observation and reward signals were
present during training, did policies consistently train to exhibit gaits with
balanced load distribution among the legs.

Anecdotally, before we introduced the offset term in equation (4), policies
did not settle into standing as well, but did appear to favor gaits other than
pronking more compared to the results presented. We conjecture that frequent
standing causes the oscillators to all sync to stance, and thus biases
training data towards pronking gaits. We also observed different gaits to
emerge more frequently at different velocities, or with different
morphologies. Although quantifying the exact convergence and transition
patterns is out of the scope of this paper, future work studying the effects
of changing oscillator parameters along with physical parameters such as
center of mass location and leg length on gait emergence for a quadruped robot
could yield connections to gait patterns observed in nature for animals of
different sizes [34].

Another avenue we find intriguing is the role oscillators may play in a
hierarchical RL setting. Higher levels of hierarchy typically reason about
task-level objectives in both a lower-dimensional space and at a slower
timescale; the phase oscillators could be interpreted as a latent state with
cyclic dynamics [35]. A latent state space with cyclic dynamics could serve
for temporal abstraction, multi-joint coordination and amortized control for
cyclic behavior [36], a direction we find very promising.

## Acknowledgements

We thank Aditya Mehrotra for helping with robot maintenance, filming, and
interesting discussions along the way. This work was supported by NAVER Labs.

## References

  * [1] Donald F Hoyt and C Richard Taylor  “Gait and the energetics of locomotion in horses”  In _Nature_ 292.5820 Nature Publishing Group UK London, 1981
  * [2] Simon Wilshin, Michelle A Reeve, G Clark Haynes, Shai Revzen, Daniel E Koditschek and Andrew J Spence  “Longitudinal quasi-static stability predicts changes in dog gait on rough terrain”  In _Journal of Experimental Biology_ 220.10 The Company of Biologists Ltd, 2017, pp. 1864–1874
  * [3] Milton Hildebrand  “Symmetrical gaits of dogs in relation to body build”  In _Journal of Morphology_ 124.3 Wiley Online Library, 1968
  * [4] Gerardo Bledt, Matthew J Powell, Benjamin Katz, Jared Di Carlo, Patrick M Wensing and Sangbae Kim  “Mit cheetah 3: Design and control of a robust, dynamic quadruped robot”  In _IEEE International Conference on Intelligent Robots and Systems (IROS)_ , 2018
  * [5] Weitao Xi, Yevgeniy Yesilevskiy and C David Remy  “Selecting gaits for economical locomotion of legged robots”  In _The International Journal of Robotics Research_ 35.9 SAGE Publications Sage UK: London, England, 2016
  * [6] Donghyun Kim, Jared Di Carlo, Benjamin Katz, Gerardo Bledt and Sangbae Kim  “Highly dynamic quadruped locomotion via whole-body impulse control and model predictive control”  In _arXiv preprint arXiv:1909.06586_ , 2019
  * [7] Hua Chen, Zejun Hong, Shunpeng Yang, Patrick M Wensing and Wei Zhang  “Quadruped capturability and push recovery via a switched-systems characterization of dynamic balance”  In _IEEE Transactions on Robotics_ IEEE, 2023
  * [8] Nathan J Kong, Chuanzheng Li, George Council and Aaron M Johnson  “Hybrid iLQR model predictive control for contact implicit stabilization on legged robots”  In _IEEE Transactions on Robotics_ , 2023
  * [9] Takahiro Miki, Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladlen Koltun and Marco Hutter  “Learning robust perceptive locomotion for quadrupedal robots in the wild”  In _Science Robotics_ 7.62, 2022, pp. eabk2822
  * [10] Gilbert Feng, Hongbo Zhang, Zhongyu Li, Xue Bin Peng, Bhuvan Basireddy, Linzhu Yue, Zhitao Song, Lizhi Yang, Yunhui Liu, Koushil Sreenath and Sergey Levine  “Genloco: Generalized locomotion controllers for quadrupedal robots”  In _Conference on Robot Learning (CoRL)_ , 2023
  * [11] Zhaoming Xie, Xingye Da, Michiel Van de Panne, Buck Babich and Animesh Garg  “Dynamics randomization revisited: A case study for quadrupedal locomotion”  In _IEEE International Conference on Robotics and Automation (ICRA)_ , 2021
  * [12] Yongbin Jin, Xianwei Liu, Yecheng Shao, Hongtao Wang and Wei Yang  “High-speed quadrupedal locomotion by imitation-relaxation reinforcement learning”  In _Nature Machine Intelligence_ , 2022
  * [13] Jonah Siekmann, Yesh Godse, Alan Fern and Jonathan Hurst  “Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition”  In _IEEE International Conference on Robotics and Automation (ICRA)_ , 2021
  * [14] Gabriel B. Margolis and Pulkit Agrawal  “Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior”  In _Conference on Robot Learning (CoRL)_ , 2023
  * [15] Yecheng Shao, Yongbin Jin, Xianwei Liu, Weiyan He, Hongtao Wang and Wei Yang  “Learning free gait transition for quadruped robots via phase-guided controller”  In _IEEE Robotics and Automation Letters_ , 2021
  * [16] Yuxiang Yang, Tingnan Zhang, Erwin Coumans, Jie Tan and Byron Boots  “Fast and efficient locomotion via learned gait transitions”  In _Conference on Robot Learning (CoRL)_ , 2022
  * [17] Auke Jan Ijspeert  “Central pattern generators for locomotion control in animals and robots: a review”  In _Neural networks_ 21.4 Elsevier, 2008, pp. 642–653
  * [18] Jonas Buchli, Ludovic Righetti and Auke Jan Ijspeert  “Engineering entrainment and adaptation in limit cycle systems: From biological inspiration to applications in robotics”  In _Biological Cybernetics_ Springer, 2006
  * [19] Guillaume Bellegarda and Auke Ijspeert  “CPG-RL: Learning central pattern generators for quadruped locomotion”  In _IEEE Robotics and Automation Letters_
  * [20] Auke J Ijspeert and Monica A Daley  “Integration of feedforward and feedback control in the neuromechanics of vertebrate locomotion: a review of experimental, simulation and robotic studies”  In _Journal of Experimental Biology_ 226.15 The Company of Biologists, 2023
  * [21] Antonin Raffin, Olivier Sigaud, Jens Kober, Alin Albu-Schäffer, João Silvério and Freek Stulp  “A Simple Open-Loop Baseline for Reinforcement Learning Locomotion Tasks”  In _International Conference on Learning Representations_ PMLR, 2024
  * [22] Hansol X Ryu and Arthur D Kuo  “An optimality principle for locomotor central pattern generators”  In _Scientific Reports_ Nature Publishing Group UK London, 2021
  * [23] Dai Owaki, Takeshi Kano, Ko Nagasawa, Atsushi Tero and Akio Ishiguro  “Simple robot suggests physical interlimb communication is essential for quadruped walking”  In _Journal of The Royal Society Interface_ 10.78 The Royal Society, 2013
  * [24] Nikita Rudin, David Hoeller, Philipp Reist and Marco Hutter  “Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning”  In _Conference on Robot Learning (CoRL)_ , Proceedings of Machine Learning Research  PMLR, 2022
  * [25] Jens Kober, J Andrew Bagnell and Jan Peters  “Reinforcement learning in robotics: A survey”  In _The International Journal of Robotics Research_ 32 SAGE Publications Sage UK: London, England, 2013
  * [26] Mostafa Ajallooeian, Jesse Kieboom, Albert Mukovskiy, Martin A Giese and Auke J Ijspeert  “A general family of morphed nonlinear phase oscillators with arbitrary limit cycle shape”  In _Physica D: Nonlinear Phenomena_ Elsevier, 2013
  * [27] Auke Jan Ijspeert, Alessandro Crespi, Dimitri Ryczko and Jean-Marie Cabelguen  “From swimming to walking with a salamander robot driven by a spinal cord model”  In _science_ American Association for the Advancement of Science, 2007
  * [28] Mostafa Ajallooeian, Sébastien Gay, Alexandre Tuleu, Alexander Spröwitz and Auke J Ijspeert  “Modular control of limit cycle locomotion over unperceived rough terrain”  In _IEEE International Conference on Intelligent Robots and Systems_ , 2013
  * [29] Shinya Aoi, Daiki Katayama, Soichiro Fujiki, Nozomi Tomita, Tetsuro Funato, Tsuyoshi Yamashita, Kei Senda and Kazuo Tsuchiya  “A stability-based mechanism for hysteresis in the walk–trot transition in quadruped locomotion”  In _Journal of The Royal Society Interface_ , 2013
  * [30] Malte Schilling and Holk Cruse  “Decentralized control of insect walking: A simple neural network explains a wide range of behavioral and neurophysiological results”  In _PLoS computational biology_ Public Library of Science San Francisco, CA USA, 2020
  * [31] Dai Owaki and Akio Ishiguro  “A quadruped robot exhibiting spontaneous gait transitions from walking to trotting to galloping”  In _Scientific Reports_ Nature Publishing Group UK London, 2017
  * [32] MIT Biomimetics  “ORCAgym”, https://github.com/mit-biomimetics/ORCAgym, 2023
  * [33] Benjamin Katz, Jared Di Carlo and Sangbae Kim  “Mini cheetah: A platform for pushing the limits of dynamic quadruped control”  In _IEEE International Conference on Robotics and Automation (ICRA)_ , 2019
  * [34] Norman C Heglund, C Richard Taylor and Thomas A McMahon  “Scaling stride frequency and gait to animal size: mice to horses”  In _Science_ American Association for the Advancement of Science, 1974
  * [35] Sebastian Starke, Ian Mason and Taku Komura  “Deepphase: Periodic autoencoders for learning motion phase manifolds”  In _ACM Transactions on Graphics (TOG)_ 41.4 ACM New York, NY, USA, 2022, pp. 1–13
  * [36] Josh Merel, Matthew Botvinick and Greg Wayne  “Hierarchical motor control in mammals and machines”  In _Nature Communications_ Nature Publishing Group UK London, 2019