# Learn to Sing by Listening: Building Controllable Virtual Singer by
Unsupervised Learning from Voice Recordings

Wei Xue1, Yiwen Wang2, Qifeng Liu3,1, Yike Guo1
1 Hong Kong University of Science and Technology, 2 Hong Kong Chu Hai College,
3 Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences

###### Abstract

The virtual world is being established in which digital humans are created
indistinguishable from real humans. Producing their audio-related capabilities
is crucial since voice conveys extensive personal characteristics. We aim to
create a controllable audio-form virtual singer; however, supervised modeling
and controlling all different factors of the singing voice, such as timbre,
tempo, pitch, and lyrics, is extremely difficult since accurately labeling all
such information needs enormous labor work. In this paper, we propose a
framework that could digitize a person’s voice by simply “listening” to the
clean voice recordings of any content in a fully unsupervised manner and
predict singing voices even only using speaking recordings. A variational
auto-encoder (VAE) based framework is developed, which leverages a set of pre-
trained models to encode the audio as various hidden embeddings representing
different factors of the singing voice, and further decodes the embeddings
into raw audio. By manipulating the hidden embeddings for different factors,
the resulting singing voices can be controlled, and new virtual singers can
also be further generated by interpolating between timbres. Evaluations of
different types of experiments demonstrate the proposed method’s
effectiveness. The proposed method is the critical technique for producing the
AI choir, which empowered the human-AI symbiotic orchestra in Hong Kong in
July 2022.

##  1 Introduction

We are entering a new era in which the boundary between real and virtual
worlds is increasingly blurred, eliminating the geographical barriers between
people and the gaps between humans and AI. This further facilitates co-
inspiring and co-creation between humans and AI to push the boundaries of
science and art. Digital humans can be created indistinguishable from real
humans; generating natural and personalized voices is essential since the
voice conveys not only the content information for communication but also
personalized information such as timbre, accent, and cadence. In this paper,
we aim to produce audio-form digital humans capable of singing, i.e., virtual
singers, with wide-ranging applications in entertainment, virtual assistants,
cultural preservation, and digital immortality.

Creating natural voices from the machine is conventionally tackled by the
problem of text-to-speech (TTS), which syntheses speaking speech waveforms
according to the text specifying the content. Early TTS approaches seek to
estimate the over-simplified linear filters modeling the physical structure of
vocal organs [1; 2; 3], and current mainstream approaches train the deep
neural networks (DNNs) in a supervised manner to model the speech signals for
different contents and dynamics while preserving the timbre. Typically, these
approaches first utilize an acoustic modeling network to transform the text
into the time-frequency mel-spectrogram [4; 5; 6; 7; 8; 9], and then adopt a
vocoder [10; 11; 12; 13] to transform the mel-spectrogram into the time-domain
waveforms. Widely used acoustic modeling networks include Tacotron [4], DurIAN
[6], and FastSpeech [8; 9], and vocoders include WaveNet [10], HiFi-GAN [14]
and MelGAN [15]. To achieve singing voice synthesis (SVS), the lyrics and
melody information are jointly used as the input of the acoustic modeling
network [16; 17; 18; 19], and vocoders similar to the TTS are adopted to
produce the audio signals finally.

A major problem for supervised SVS is that a large annotated dataset is
required, which includes the audio and the corresponding scripts indicating
the content and melody. Although there are some public SVS datasets such as
OpenCpop [20] and VocalSet [21], building such datasets needs extensive labor
work for recording and annotating. This hinders flexibly building the voice
models for an arbitrary person and also prevents simulating the singing skills
of highly professional singers because of the cost of inviting singers for
recording. To eliminate the reliance on annotation, singing voice conversion
(SVC) can be used to essentially perform style transfer on audios, where the
timbre is defined as the “style”. Many methods [22; 23; 24; 25; 26] are
proposed to disentangle the content and timbre information with unsupervised
learning and then replace the timbre with the target speaker. However, as only
the timbre is replaced with the target singer, it is difficult to model the
unique singing skills of the singer, and precisely controlling other
diversified singing characteristics is also not straightforward.

In this paper, we propose a new framework to digitize the voice in a fully
unsupervised manner. The proposed method could simply rely on the audio
recordings of any content and language to build a flexible voice model and
make it possible to control the detailed characteristics such as singing
pitch, melody, and lyrics. In this way, the proposed method can even generate
singing from datasets of speaking voices.

A variational auto-encoder (VAE) based framework is developed, which leverages
a set of pre-trained models to encode the audio as various hidden embeddings
representing different factors of the singing voice, and further decodes the
embeddings into raw audio. By manipulating the hidden embeddings for different
factors, the resulting singing voices can be controlled, and new virtual
singers can also be further generated by interpolating between timbres.
Furthermore, by training on large-scale data, the model can also learn to
model other unique skills, including the accent and emotional expression of
the singer. We conduct experiments on different datasets for various tasks,
and the results demonstrate the effectiveness of the proposed method. The
proposed method is also the key technique for producing the AI choir, which
empowered the human-AI symbiotic orchestra in Hong Kong in July 2022 [27].

The rest of this paper is organized as follows. In Section 2, we review the
related works. The proposed framework for unsupervised voice modeling will be
introduced in Section 3, including the encoder and decoder models, as well as
the end-to-end adaptation. How to manipulate the representations to achieve
controllable singing will also be described. In Section 4, we explain how to
generate the AI choir based on the proposed framework. Experimental results
will be shown and discussed in Section 5.

##  2 Related Work

###  2.1 Speech Synthesis

A conventional way to digitize the human voice is speech synthesis, or TTS,
which generates natural speech according to the text inputs. The state-of-the-
art TTS systems are based on neural networks to model the complicated dynamics
of natural speech, and datasets with pairwise audios and texts are used to
train the models in the supervised learning scheme. The pipeline of TTS
generally consists of an acoustic model which converts the textual information
into the audio mel-spectrogram and a vocoder that further generates the
audible waveform. The acoustic model mainly focuses on learning the low-level
speech representations from text, and the vocoder aims at generating ultra-
long signals (e.g., 24,000 samples per second) with high fidelity [11].

Early DNN-based approaches for acoustic modeling generally use RNN to model
the temporal dependencies of speech, such as Tacotron [4] and DurIAN [6], and
produce the mel-spectrogram in an autoregressive way. Transformer-based
methods, including [28; 29; 8; 9], are then proposed, significantly speeding
up the generation process by parallel computation in the attention.
Variational autoencoder (VAE) is also used for acoustic modeling, typical
methods include GMVAE-Tacotron [30] and VAE-TTS [31]. Recently, diffusion-
based acoustic models, such as [32; 33], have been further developed to
improve the quality of synthesized speeches. When using the acoustic model for
SVS [16; 17; 18; 19], the main modification is that the pitch and duration of
the lyrics are explicitly given rather than being estimated by separate
modules in the speaking speech synthesis.

The vocoder is typically regarded as a sequence-to-sequence modeling problem.
An important work is WaveNet [10], which uses the autoregressive (AR)
convolutional neural network for sequence modeling. Parallel WaveNet [11]
further uses the inverse autoregressive flow (IAF) to distill the knowledge
from a pre-trained WaveNet teacher, substantially improving the inference
speed. GAN-based methods are also proposed to improve speech quality,
including MelGAN [15], GAN-TTS [34], and HiFi-GAN [14]. Diffusion models
further improve speech quality, and typical works include SpecGrad [35] and
DiffWave [36]. Differential signal processing is also used to design the
vocoders, for instance, Source-Filter HiFi-GAN [37] and SawSing [38].

Separately optimizing the acoustic modeling and vocoder helps to improve the
stability of model training, and the universal vocoders can also be trained to
synthesize voices of different timbres. Nevertheless, many end-to-end methods
have been developed to produce the audio waveforms directly from the textual
inputs. Famous works include Char2Wav [39], ClariNet [40], and FastSpeech 2s
[9; 41], which generally combine acoustic modeling and vocoder into a large
encoder-decoding framework.

###  2.2 Voice Conversion

The significant reliance on the large and carefully annotated dataset is a
primary obstacle to exploiting speech synthesis to digitize the human voice.
The dataset includes textual information about the content of speaking, and
for SVS, the duration, and pitch of each word in the lyrics should be
additionally precisely given. Several hours of training data are needed to
produce high-quality audio. Such a requirement significantly increases the
difficulty of training a voice model for the ordinary person since specialized
voice recording and annotation are costly. For SVS, the singing skills of the
produced voice models are also limited since inviting many pop stars for
recording is nearly impossible.

By applying style transfer, which is extensively studied in computer vision
[42; 43; 44; 45; 46], to the audios, the problems of voice conversion (VC) and
SVC are formulated, which treats the timbre as the style of audio and the rest
information as the content. In this way, inspired by methods such as CycleGAN
[45] and StarGAN [46] for images, unsupervised learning can disentangle the
hidden audio representations on nonparallel data from different people.
Further, VC can be achieved by changing the timbre embedding, which yields
CycleGAN-VC [47], StarGAN-VC [48], and StarGANv2-VC [49]. The main focus has
been disentangling linguistic features from the speaker representation, such
as the VAE-based methods [50; 51], phonetic posteriorgram (PPG)-based methods
[52; 53], as well as the vector quantification (VQ) based methods [54; 26;
55]. Although these methods can change the timbre of the generated audio to
the target, they cannot fully express other unique characteristics of the
human voice, such as accent, emotion, and rhythm. On the other hand, although
many unsupervised methods have been developed to disentangle the audio
representations, explicitly disentangling different characteristics remains
challenging.

##  3 Proposed Framework for Voice Digitization

Figure 1: Diagram of the overall framework.

In this paper, we propose a new method that can digitize anyone’s voice based
on their recordings without annotation. The recordings can be of arbitrary
content and language, and the proposed method can predict singing voices only
from speaking recordings. As shown in Fig. 1, an unsupervised VAE-based
framework is developed. Similar to humans using existing skills to learn new
tasks, we largely leverage the skills of a set of pre-trained audio models to
decouple different characteristics of sounds. Similar to the pipeline of TTS,
the proposed framework consists of an acoustic model which maps different
controlling factors to the mel-spectrogram and a vocoder that converts the
mel-spectrogram to audio waveforms. After training the acoustic model and
vocoder separately, the end-to-end training is further adopted to finetune the
two networks. Details of the proposed framework will be described below.

###  3.1 Pre-training Audio Models

To identify detailed audio characteristics, we aim to figure out a) what is
the audio content, b) who is speaking or singing, and c) what is the melody of
the speaking or singing. We note that training data for the target speaker
containing pairs of audio and textual information is not available. Therefore,
instead of performing end-to-end unsupervised learning to disentangle audio
representations, we rely on models trained on other annotated datasets to
identify distinctive characteristics. In this way, the reliance on an
annotated target speaker dataset is eliminated, while discriminative
embeddings can be generated.

The automatic speech recognition (ASR) model trained on the large-scale ASR
dataset can identify the content information. The Wenetspeech dataset [56],
consisting of over $10,000$ hours of accurately labeled Mandarin speeches, is
used for training, and the pre-trained model [57] based on the U2++ network
[58] is utilized to generate the audio content embedding, which is the output
of the Conformer [59] encoding block. As shown in Fig. 2, the audio content
embedding is essentially the two-dimensional PPG, and each column indicates
the probability distribution of the current frame on different phones. When
feeding the unannotated audio from the target speaker to the pre-trained ASR
model, its content information can be represented by the PPG. The frame size
for the ASR processing is $25$ ms, and the hop size is $10$ ms. Assuming $T$
frames are contained in the utterance, to accelerate the decoding, $T/3$ PPG
frames are obtained with the subsampling factor as $3$ in the ASR decoding.
The resulting PPG is then up-sampled by $3$ times to match the original number
of frames.

Figure 2: Illustration of PPG. Each column of the two-dimensional PPG
represents the probability distribution over different phonemes.

Similarly, the speaker identity is obtained using the VoxCeleb2 [60] dataset
for speaker identification. An ECAPA-TDNN model [61] pre-trained using the
SpeechBrain [62] is adopted to generate the speaker embeddings, which are the
outputs of the model. For one utterance, only one speaker embedding is
obtained, and the embeddings of all utterances for the same speaker are
averaged to finally represent the timbre. The resulting embedding is further
replicated over $T$ times to represent the identity of each frame.

We finally identify the melody of the speech and singing voice by estimating
the pitch contours in the voiced audio. The pre-trained CREPE model [63] is
exploited, and the resulting contour for the $T$-frame utterance is a $T\times
1$ signal, indicating the pitch of each frame.

Figure 3: Diagram of the acoustic model, which converts embeddings extracted
from pre-trained audio models to the mel-spectrogram of the input signal.

###  3.2 Acoustic Model

With pre-trained audio models, we can convert the audio signal without any
annotation into a set of embeddings representing the different
characteristics. An acoustic model is further trained to convert the obtained
embeddings into the mel-spectrogram, which will be used to synthesize audible
waveforms later. Since the mel-spectrogram has much richer information than
the embeddings from pre-trained models, the acoustic model learns to represent
the uniqueness of the speaker, thereby digitizing the person’s voice. In
addition, the embeddings also provide the interface to control the resulting
mel-spectrogram.

The diagram of the proposed acoustic model is shown in Fig. 3. Given the
extracted embeddings representing the content, identity, and melody
information, a Conformer-based decoding network is designed to reconstruct the
mel-spectrogram of the input utterance. The mel-spectrogram can be directly
computed by applying the Mel filterbanks to the input signal in the short-time
Fourier Transform (STFT) domain. Therefore, the acoustic model can be seen as
an auto-encoder, and the training can be conducted fully unsupervised.

The details of the Conformer-based decoder are illustrated in Fig. 4. In Fig.
4 (a), three linear layers first process different audio embeddings separately
to ensure all embeddings have the same dimension. Then, the projected
embeddings are summed up and further transformed by a linear layer before
being fed into the Conformer block. The acoustic model finally predicts the
mel-spectrogram of the utterance producing the embeddings, and similar to [8],
a PostNet is used as an auxiliary network to improve the prediction
performance.

The Conformer block is widely used for ASR and TTS [59; 8; 64], which
basically integrates a convolution block into the Transformer, and its
structure is shown in Fig. 4 (b). The structure of the PostNet is shown in
Fig. 4 (c), which adopts the convolutional RNN (CRNN) with skip connections to
model the temporal and structural information of the mel-spectrogram.

Figure 4: Details of the Conformer-based decoder in the acoustic model. (a)
Overall structure; (b) Conformer Block; (c) PostNet.

During training, only the Conformer-based decoder is optimized using the
combination of $L_{1}$ and $L_{2}$ norms:

$\displaystyle
L(\mathbf{S}_{\textrm{linear}},\mathbf{S}_{\textrm{PostNet}},\mathbf{S}_{\textrm{gt}})$
$\displaystyle=||\mathbf{S}_{\textrm{linear}}-\mathbf{S}_{\textrm{gt}}||_{1}+||\mathbf{S}_{\textrm{linear}}-\mathbf{S}_{\textrm{gt}}||_{2}+||\mathbf{S}_{\textrm{PostNet}}-\mathbf{S}_{\textrm{gt}}||_{1}+||\mathbf{S}_{\textrm{PostNet}}-\mathbf{S}_{\textrm{gt}}||_{2},$
(1)

where $\mathbf{S}_{\textrm{linear}}$,$\mathbf{S}_{\textrm{PostNet}}$ are the
mel-spectrograms output by the last linear layer and PostNet, respectively,
and $\mathbf{S}_{\textrm{gt}}$ is the ground-truth mel-spectrogram.

###  3.3 Vocoder

Figure 5: Illustration of the vocoder training.

The mel-spectrogram is further converted into the audio waveform through a
vocoder. Here, the widely used HiFi-GAN [14] vocoder is adopted, which uses
multi-scale and multi-period discriminators to ensure the fidelity of the
produced waveforms. As illustrated in Fig. 5, for each utterance, the ground-
truth mel-spectrogram is first computed and is then fed into the vocoder to
produce the original signal.

###  3.4 End-to-end Training

We note that in practice, the mel-spectrogram predicted by the acoustic model
rather than the ground-truth one will be used to generate the audio signal.
However, there will always be an error between the predicted mel-spectrogram
and the ground truth, which will finally affect the quality of the produced
audio. In Fig. 1, the whole framework aims to perform end-to-end VAE over the
audio waveforms of the target person. Therefore, end-to-end training over the
acoustic model and vocoder will be performed to finetune both models further.

In the end-to-end training, the acoustic model and vocoder are combined to
transform the embeddings extracted from the pre-trained audio models directly
to the raw waveforms, and they are alternatively optimized by freezing the
other model when updating the weights. The loss function of the HiFi-GAN is
used to optimize the end-to-end framework.

###  3.5 Controllable Audio Generation

Given the acoustic model and vocoder, as shown in Fig. 1, controllable audio
signals can be generated by manipulating the audio embeddings fed into the
acoustic model, and we can produce singing voices even from the speaking
recordings.

The PPG representing the audio contents can be obtained by applying the ASR
model on existing speech recordings with target contents, e.g., a real or
synthesized reading speech or the vocal track of a song. Forced alignment (FA)
[56] can be conducted to determine the interval of each phoneme, and the PPG
corresponding to each phoneme can be re-sampled to adjust to the target
duration.

The pitch contour can also be extracted from existing audio which can be
either speaking or singing and can be further edited in an explainable way. If
the training data has large variations on the pitch, even when only the
speaking data is contained, the model can learn how to generate the sounds for
the target pitches in a singing melody, such that singing voices can be
produced.

We can produce either a personalized model for each person, corresponding to
the audio-form digital twin, or a general model that can change the timbres to
create non-existing humans. A pair of specialized acoustic models and vocoder
can be trained to fully model the uniqueness of an individual’s voice, such as
the timbre, accent, and subtle rhythm control of speaking and singing. In this
case, the speaker identity embedding in can be fixed to a zero-valued vector
or be removed in the framework shown in Fig. 3. To produce variable timbres,
audio data from different people can be jointly used to train the acoustic
model, in which case the speaker identity embedding can be manipulated to
control the timbre. A universal vocoder can also be trained to convert any
mel-spectrogram to an audible signal.

##  4 Case Study: AI Choir Generation

This section explains how to generate an AI choir composed of hundreds of
virtual singers based on the proposed method, which empowered the human-AI
symbiotic orchestra in Hong Kong in July 2022. Different from digitizing the
voice of a single person, producing the AI choir raises the problem of the
trade-off between the convergence and diversity of the generated singing
voices.

To produce a satisfactory choral effect, real singers must perform in a highly
coordinated manner in terms of timbre, rhythm, and expressiveness. It is worth
noting that the combination of the same voices does not produce a choral
effect, and the choral effect actually results from the carefully crafted
diversity of timbre, rhythm, and expressiveness of each singer in the
collective performance. This yields a control problem for the joint generation
of multiple singing voices: the coherence of the singers’ voices and the
diversity of timbres in the choir must be jointly optimized.

Figure 6: Generating interpolated virtual singers from prototype singers.

We develop a two-stage method to produce the AI choir with hundreds of virtual
singers. First, several “prototype” singers with similar timbres are produced
in the first stage. Then, in the second stage, as shown in Fig. 6, hundreds of
new virtual singers are produced by interpolating between the timbres of these
prototype singers. The harmonic choral effect is obtained by carefully
controlling the rhythm and pitch of each virtual singer.

Figure 7: AI choir generation with timbre interpolation.

###  4.1 Prototype Singers

The prototype singer is generated by digitizing the voices of an existing
singer, given the clean vocal recordings. In practice, although many public-
domain songs of a singer are available online, the vocal signals are mixed
with accompaniments. To collect the large-scale training data, a pre-trained
speech separation model, Demucs [65], is used to extract the vocals from the
original sounds. With the extracted vocal recordings from multiple prototype
singers, we train a general model which relies on the speaker embedding to
control the timbre of the produced singing voice. To produce the choir, eight
prototype singers whose similar timbres are carefully selected by human
evaluators are generated.

###  4.2 Interpolated Singers

The prototype singers can be seen as audio-form digital twins of existing
singers, and new virtual singers can be created by performing timbre
interpolation. As shown in Fig. 7, speaker embeddings of different new virtual
singers are generated by performing linear interpolation among the prototype
singers, and these embeddings are utilized to produce the mel-spectrograms.

The same PPG and pitch contour embeddings are used for all different virtual
singers, so the produced singing voices have the same lyrics and rhythm. The
generated mel-spectrograms are finally converted to audio waveforms using the
universal vocoder trained on datasets combining all singers. In total, 320
Virtual singers are produced to finally generate a choir.

Since prototype singers with similar timbres are used, although the timbres of
all interpolated virtual singers are different, they will still be similar,
which ensures collaborative performances in the choir. The human evaluator
also plays an essential role in combining the virtual singers to produce the
choir. The combination of a large number of virtual singers to produce the
choir appears to be straightforward. However, human evaluators must examine
whether the choir has a good combined timbre and adjust the proportions of the
prototype singers.

##  5 Experiments

###  5.1 Datasets

Three different datasets are used to evaluate the performances of singing
voice generation, speaking-to-singing, and AI choir generation, respectively.

OpenCpop. The publicly-available high-quality Mandarin singing corpus,
OpenCpop [20], is adopted to examine the capability of the proposed method to
generate high-quality singing audio. The corpus consists of $100$ songs
without accompaniments recorded by a professional female singer, and the
audios are segmented into 3,756 utterances with a total duration of $5.2$
hours. Although the note and phoneme information is included in the original
dataset, the proposed method uses only the audio waveforms to digitize the
singer’s voice.

Speaking Audios. To test the speaking-to-singing performances, we constructed
a $3.7$ hours speaking audio dataset by ourselves based on the recordings of
one male colleague in the daily Zoom meetings. We note that many high-quality
publicly-available datasets (e.g., LibriTTS [66]) for the reading speech
synthesis are available. However, using the data collected from Zoom meetings
helps to examine the feasibility of using daily normal-quality speeches to
achieve voice digitization. To facilitate discussion, we denote this dataset
as “Speaking” in the rest of the experiments.

Audios from Multiple Singers. To produce the choir, we further collected the
songs of eight professional singers online (Youtube, Spotify, etc.), with four
male singers and four female singers included. For each singer, nearly $4$
hours of data are collected, and for all songs, Demucs [65] is used to extract
the vocal tracks. We denote this dataset as “Multi-Sing” in the following.

For all datasets, the audios are converted into the single-channel with a $24$
kHz sampling rate. No other information is required to conduct the
unsupervised training. The obtained models are tested by synthesizing singing
voices according to the “straight” excerpts of the VocalSet [21], with matched
genders for the training dataset and testing utterances.

###  5.2 Implementation

Acoustic model. The structure of the acoustic model is shown in Fig. 4. The
dimensions of PPG, speaker identity and pitch contour embeddings are $320$,
$256$, and $1$, respectively, and they all are transformed by linear layers to
the dimension of $320$. $6$ Conformer blocks are included in the acoustic
model, and for each Conformer block, we use $2$ attention heads.
$80$-dimension mel-spectrogram is used as the training target, and the
analysis frame size is $25$ ms with $10$ ms hop size. The maximum length of
the Conformer is $1000$ frames, corresponding to $10$ s for the $10$ ms hop
size. The model is trained with the batch size as $24$ with the learning rate
as $1\times 10^{-3}$ and the weight decay as $1\times 10^{-6}$. The training
is conducted for $2000$ epochs.

The resulting model is speaker-dependent for the OpenCpop and Speaking
datasets since only one person is included in the training dataset. When
training on the Multi-Sing dataset, the timbre of the output audio can be
controlled by the speaker identity embedding.

Vocoder. To produce audio waveforms at $24$ kHz sampling rate from the
$80$-dimension mel-spectrogram, $4$ transposed convolution-based upsampling
blocks are included in the HiFi-GAN generator, with upsampling rates as
$\\{8,8,2,2\\}$ and upsampling kernel sizes as $\\{16,16,4,4\\}$,
respectively. The segment size for sequence-to-sequence modeling is 8192
samples. We used the batch size as $32$ for training with the learning rate as
$2\times 10^{-4}$, and the model was trained for $200,000$ steps.

Similar to the acoustic model, speaker-dependent models are obtained for the
OpenCpop and Speaking datasets, and a universal vocoder is trained by using
all the data in the Multi-Sing dataset.

End-to-end Training. After training the acoustic model and vocoder for each
dataset, we finally perform end-to-end training to finetune both models
further. We first freeze the acoustic model and update the vocoder for $5,000$
steps, then freeze the resulting vocoder to update the acoustic model for
$100$ epochs. Such process is repeated 5 times.

Figure 8: The SVE as a function of acceptance rate for the OpenCpop dataset.
The acceptance rate is used to determine the threshold for speaker
verification. Figure 9: Comparison between the pitch contours in the generated
audio and the ground truth for singing voice generated by the OpenCpop singer.

###  5.3 Results

Singing Voice Generation. In this experiment, we digitize the OpenCpop singer
based on her singing recordings.

We first check the speaker identity similarity of the produced audios. With
the pre-trained ECAPA-TDNN model [61], the speaker embeddings of all
utterances in the OpenCpop dataset are computed and then averaged to get the
overall speaker embedding of OpenCpop singer, which is denoted by
$\mathbf{S}_{\textrm{cpop}}$ here. In the following, the
$\mathbf{S}_{\textrm{cpop}}$ will be used as an anchor to examine whether the
produced audio is similar to the OpenCpop singer according to a threshold. For
all utterances in the OpenCpop, the cosine similarities between their speaker
embeddings and $\mathbf{S}_{\textrm{cpop}}$ are computed, then the threshold
is determined by accepting“ $p\%$” utterances after sorting the similarity
scores. It can be seen that the higher $p$, the lower the threshold.

Figure 10: Linear spectrograms of the ground truth and generated signal for
the OpenCpop singer. Figure 11: The SVE as a function of acceptance rate for
the “Speaking” dataset. The acceptance rate is used to determine the threshold
for speaker verification.

As described above, 25 utterances are generated by using the 25 “straight”
excerpts of the VocalSet from the female singers. The cosine similarities
between the generated audios and $\mathbf{S}_{\textrm{cpop}}$ are computed,
and the Speaker Verification Error (SVE) is obtained by comparing the
similarities with the threshold. For comparison, the same metric for the
original excerpts of the VocalSet is also calculated. Fig. 8 shows the SVE of
the generated audios and the VocalSet audios as a function of the acceptance
rate $p\%$. For all acceptance rates, the SVE of the generated audios is zero
compared to the VocalSet audios having SVE near to one, which indicates that
the proposed method can effectively learn the timbre of the OpenCpop singer.

Then we evaluate the pitch accuracy of the generated audios. Fig. 9
illustrates an example of the pitch contour of the generated audio and the
pitch contour of the input audio from the VocalSet. By taking the pitch
contour as the controlling factor of the acoustic model, the generated audio
is expected to follow the specified melody. It is shown that in almost all
cases, the generated audio produces the desired pitches, except for some
errors on onset and offset.

We further show one example of the spectrograms of the generated audio and the
ground truth in Fig. 10. The generated audio is obtained by first extracting
the embeddings from the ground truth audio and then using the unaltered
embeddings to reconstruct the waveform signal. We can observe that the
generated audio can effectively capture the temporal-frequency characteristics
of the original singing audio, indicating the capability to produce high-
quality audio outputs.

Figure 12: Comparison between the singing pitch contours in the generated
audio and the ground truth for speaking to singing by a male speaker. Figure
13: Linear spectrograms of the reference and generated signal for the
“Speaking” dataset. A singing voice is produced. We note that the reference
and generated audios correspond to different speakers.

Speaking to Singing. Using the “Speaking” dataset, we further evaluate whether
the proposed method could generate singing audios based on the dataset only
containing the speaking voices. Similar to the experiments for the OpenCpop
dataset, we use the $29$ “straight” excerpts of the VocalSet from the male
singers as the input signals to the models. The results are shown in Fig. 11.
We can see that when rejecting $10\%$ training data according to the speaker
embedding similarity, nearly $27\%$ generated audios are classified as
different from the target speaker. By increasing the acceptance rate to
$99\%$, almost all generated samples are regarded as the target, indicating
that the largest divergence of the generated audios to the average speaker
embedding is comparable to the training set. We also notice that the SVE
values for all cases are apparently higher than the OpenCpop cases, showing
that generating the singing audios from speaking-only data is more
challenging.

Figure 14: Linear spectrograms of the ground-truth and generated signal for
the “Speaking” dataset. Speaking voice is reconstructed. We note that the
ground truth and generated audios correspond to the same speaker. Figure 15:
Linear spectrograms of the reference and a generated choir consisting of 120
male singers.

The pitch contours of the generated singing audio and ground truth are
compared in Fig. 12. We can see that even though the model is trained on the
speaking-only dataset, the resulting model can still produce singing pitch
contours with high precision. In Fig. 13, we compare the spectrograms of the
reference audio from the VocalSet and the generated audio. It can be noticed
that although the produced signal can effectively follow the melody and rhythm
of the reference signal, their frequency distributions are different. This, in
turn, shows that the produced audio has exhibited a different formant from the
reference one, indicating a successful modification of the speaker identity.
The proposed method can achieve accurate reconstruction performance simply
based on the extracted embeddings when reconstructing the speaking audio from
the training speaker, as shown in Fig. 14.

Figure 16: Linear spectrograms of the reference and a generated choir
consisting of 120 female singers.

Choir Generation. We finally evaluate the performance of the proposed method
of generating the choir, which consists of tens to hundreds of virtual
singers. By using identity interpolation, new virtual singers can be produced
from the prototype singers, who are trained using the data collected online.
We note that the singing voice performances for each singer are similar to the
OpenCpop singer, and the same conclusions can be drawn; therefore, evaluations
on each generated audio are not presented here. In Fig. 15, we show an AI
choir consisting of 160 male virtual singers. We notice that since each singer
has a unique formant, the pitch harmonics become indistinguishable in the
combined choir signal, which leads to the choir effect. Similar results can be
observed from the female choir shown in Fig. 16, where the frequency
distribution is much broader than the male choir due to the higher diversity
of female timbres.

##  6 Conclusion

This paper presents a novel framework for unsupervised voice modeling that
enables the creation of digital singing humans. Our method relies on a
variational auto-encoder (VAE) that encodes audio recordings as various hidden
embeddings representing different factors of the singing voice, which can then
be manipulated to control various singing characteristics such as pitch,
melody, and lyrics. By training on large-scale data, the proposed method can
also learn to model other unique skills, including the accent and emotional
expression of the singer. Experimental results on different datasets
demonstrate the effectiveness of the proposed method.

## References

  * [1] J. Thyssen, H. Nielsen, and S. D. Hansen, “Non-linear short-term prediction in speech coding,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , Apr. 1994, pp. 185–188.
  * [2] J. D. Markel and A. H. Gray, Jr., _Linear Prediction of Speech_. Springer-Verlag, 1976.
  * [3] L. R. Rabiner and R. W. Schafer, Eds., _Theory and Applications of Digital Signal Processing_. Pearson, 2010.
  * [4] Y. Wang, R. Skerry-Ryan, D. Stanton, Y. Wu, R. J. Weiss, N. Jaitly, Z. Yang, Y. Xiao, Z. Chen, S. Bengio _et al._ , “Tacotron: Towards end-to-end speech synthesis,” _arXiv preprint arXiv:1703.10135_ , 2017.
  * [5] J. Shen, R. Pang, R. J. Weiss, M. Schuster, N. Jaitly, Z. Yang, Z. Chen, Y. Zhang, Y. Wang, R. Skerrv-Ryan _et al._ , “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2018, pp. 4779–4783.
  * [6] C. Yu, H. Lu, N. Hu, M. Yu, C. Weng, K. Xu, P. Liu, D. Tuo, S. Kang, G. Lei, D. Su, and D. Yu, “Durian: Duration informed attention network for speech synthesis,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2020.
  * [7] S. Vasquez and M. Lewis, “Melnet: A generative model for audio in the frequency domain,” _arXiv preprint arXiv:1906.01083_ , 2019.
  * [8] Y. Ren, Y. Ruan, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Liu, “Fastspeech: Fast, robust and controllable text to speech,” in _Proc. Advances in Neural Information Processing Systems (NeurIPS)_ , 2019, pp. 3165–3174.
  * [9] Y. Ren, C. Hu, X. Tan, T. Qin, S. Zhao, Z. Zhao, and T. Liu, “Fastspeech 2: Fast and high-quality end-to-end text to speech,” in _Proc. Intl. Conf. on Learning Representations (ICLR)_ , 2021.
  * [10] A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. W. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” in _ISCA Speech Synthesis Workshop_ , 2016.
  * [11] A. van den Oord, Y. Li, I. Babuschkin, K. Simonyan, O. Vinyals, K. Kavukcuoglu, G. van den Driessche, E. Lockhart, L. C. Cobo, F. Stimberg, N. Casagrande, D. Grewe, S. Noury, S. Dieleman, E. Elsen, N. Kalchbrenner, H. Zen, A. Graves, H. King, T. Walters, D. Belov, and D. Hassabis, “Parallel wavenet: Fast high-fidelity speech synthesis,” in _Proc. Intl. Conf. Machine Learning (ICML)_ , 2018, pp. 3915–3923.
  * [12] S. Mehri, K. Kumar, I. Gulrajani, R. Kumar, S. Jain, J. Sotelo, A. C. Courville, and Y. Bengio, “Samplernn: An unconditional end-to-end neural audio generation model,” in _Proc. Intl. Conf. on Learning Representations (ICLR)_ , 2017.
  * [13] J. Valin and J. Skoglund, “LPCNET: improving neural speech synthesis through linear prediction,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2019, pp. 5891–5895.
  * [14] J. Kong, J. Kim, and J. Bae, “Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis,” in _Proc. Advances in Neural Information Processing Systems (NeurIPS)_ , 2020.
  * [15] K. Kumar, R. Kumar, T. de Boissiere, L. Gestin, W. Z. Teoh, J. Sotelo, A. de Brébisson, Y. Bengio, and A. C. Courville, “Melgan: Generative adversarial networks for conditional waveform synthesis,” in _Proc. Advances in Neural Information Processing Systems (NeurIPS)_ , 2019.
  * [16] P. Lu, J. Wu, J. Luan, X. Tan, and L. Zhou, “Xiaoicesing: A high-quality and integrated singing voice synthesis system,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2020.
  * [17] Y. Song, W. Song, W. Zhang, Z. Zhang, D. Zeng, Z. Liu, and Y. Yu, “Singing voice synthesis with vibrato modeling and latent energy representation,” in _Proc. IEEE Intl. Workshop on Multimedia Signal Processing (MMSP)_ , 2022, pp. 1–6.
  * [18] J. Liu, C. Li, Y. Ren, F. Chen, and Z. Zhao, “Diffsinger: Singing voice synthesis via shallow diffusion mechanism,” in _Proc. AAAI Conference on Artificial Intelligence (AAAI)_ , 2022, pp. 11 020–11 028.
  * [19] R. Huang, C. Cui, F. Chen, Y. Ren, J. Liu, Z. Zhao, B. Huai, and Z. Wang, “Singgan: Generative adversarial network for high-fidelity singing voice generation,” in _Proc. ACM Intl. Conf. on Multimedia (ACM MM)_ , 2022, pp. 2525–2535.
  * [20] Y. Wang, X. Wang, P. Zhu, J. Wu, H. Li, H. Xue, Y. Zhang, L. Xie, and M. Bi, “Opencpop: A high-quality open source chinese popular song corpus for singing voice synthesis,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2022, pp. 4242–4246.
  * [21] J. Wilkins, P. Seetharaman, A. Wahl, and B. Pardo, “Vocalset: A singing voice dataset,” in _Proc. Intl. Soc. for Music Information Retrieval Conf. (ISMIR)_ , 2018.
  * [22] J. Chou, C. Yeh, H. Lee, and L. Lee, “Multi-target voice conversion without parallel data by adversarially learning disentangled audio representations,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2018, pp. 501–505.
  * [23] J. Chou and H. Lee, “One-shot voice conversion by separating speaker and content representations with instance normalization,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2019, pp. 664–668.
  * [24] Y. Wang, D. Stanton, Y. Zhang, R. J. Skerry-Ryan, E. Battenberg, J. Shor, Y. Xiao, Y. Jia, F. Ren, and R. A. Saurous, “Style tokens: Unsupervised style modeling, control and transfer in end-to-end speech synthesis,” in _Proc. Intl. Conf. Machine Learning (ICML)_ , 2018, pp. 5167–5176.
  * [25] H. Lu, Z. Wu, D. Dai, R. Li, S. Kang, J. Jia, and H. Meng, “One-shot voice conversion with global speaker embeddings,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , G. Kubin and Z. Kacic, Eds., 2019, pp. 669–673.
  * [26] D. Wang, L. Deng, Y. T. Yeung, X. Chen, X. Liu, and H. Meng, “VQMIVC: vector quantization and mutual information-based unsupervised speech representation disentanglement for one-shot voice conversion,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2021, pp. 1344–1348.
  * [27] Y. Guo, Q. Liu, J. Chen, W. Xue, H. Jensen, F. Rosas, J. Shaw, X. Wu, J. Zhang, and J. Xu, “Pathway to future symbiotic creativity,” _arXiv preprint arXiv:2209.02388_ , 2022.
  * [28] N. Li, S. Liu, Y. Liu, S. Zhao, and M. Liu, “Neural speech synthesis with transformer network,” in _Proc. AAAI Conference on Artificial Intelligence (AAAI)_ , 2019.
  * [29] M. Chen, X. Tan, Y. Ren, J. Xu, H. Sun, S. Zhao, and T. Qin, “Multispeech: Multi-speaker text to speech with transformer,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2020.
  * [30] W. Hsu, Y. Zhang, R. J. Weiss, H. Zen, Y. Wu, Y. Wang, Y. Cao, Y. Jia, Z. Chen, J. Shen, P. Nguyen, and R. Pang, “Hierarchical generative modeling for controllable speech synthesis,” in _Proc. Intl. Conf. on Learning Representations (ICLR)_ , 2019.
  * [31] Y. Zhang, S. Pan, L. He, and Z. Ling, “Learning latent representations for style control and transfer in end-to-end speech synthesis,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2019\.
  * [32] M. Jeong, H. Kim, S. J. Cheon, B. J. Choi, and N. S. Kim, “Diff-tts: A denoising diffusion model for text-to-speech,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2021, pp. 3605–3609.
  * [33] V. Popov, I. Vovk, V. Gogoryan, T. Sadekova, and M. A. Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” in _Proc. Intl. Conf. Machine Learning (ICML)_ , 2021, pp. 8599–8608.
  * [34] M. Binkowski, J. Donahue, S. Dieleman, A. Clark, E. Elsen, N. Casagrande, L. C. Cobo, and K. Simonyan, “High fidelity speech synthesis with adversarial networks,” in _Proc. Intl. Conf. on Learning Representations (ICLR)_ , 2020\.
  * [35] Y. Koizumi, H. Zen, K. Yatabe, N. Chen, and M. Bacchiani, “Specgrad: Diffusion probabilistic model based neural vocoder with adaptive noise spectral shaping,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2022, pp. 803–807.
  * [36] Z. Kong, W. Ping, J. Huang, K. Zhao, and B. Catanzaro, “Diffwave: A versatile diffusion model for audio synthesis,” in _Proc. Intl. Conf. on Learning Representations (ICLR)_ , 2021.
  * [37] R. Yoneyama, Y. Wu, and T. Toda, “Source-filter hifi-gan: Fast and pitch controllable high-fidelity neural vocoder,” _arXiv.2210.15533_ , 2022.
  * [38] D. Wu, W. Hsiao, F. Yang, O. Friedman, W. Jackson, S. Bruzenak, Y. Liu, and Y. Yang, “Ddsp-based singing vocoders: A new subtractive-based synthesizer and A comprehensive evaluation,” _arXiv.2208.04756_ , 2022.
  * [39] J. Sotelo, S. Mehri, K. Kumar, J. F. Santos, K. Kastner, A. C. Courville, and Y. Bengio, “Char2wav: End-to-end speech synthesis,” in _Proc. Intl. Conf. on Learning Representations (ICLR)_ , 2017.
  * [40] W. Ping, K. Peng, and J. Chen, “Clarinet: Parallel wave generation in end-to-end text-to-speech,” in _Proc. Intl. Conf. on Learning Representations (ICLR)_ , 2019.
  * [41] D. Lim, S. Jung, and E. Kim, “JETS: jointly training fastspeech2 and hifi-gan for end to end text to speech,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2022, pp. 21–25.
  * [42] L. A. Gatys, A. S. Ecker, and M. Bethge, “A neural algorithm of artistic style,” _arXiv.1508.06576_ , 2015.
  * [43] ——, “Image style transfer using convolutional neural networks,” in _Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)_ , 2016\.
  * [44] Y. Li, N. Wang, J. Liu, and X. Hou, “Demystifying neural style transfer,” in _Proc. Intl Joint Conf. on Artifi. Intelli. (IJCAI)_ , 2017, pp. 2230–2236.
  * [45] J. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _Proc. IEEE Intl. Conf. on Computer Vision (ICCV)_ , 2017, pp. 2242–2251.
  * [46] Y. Choi, M. Choi, M. Kim, J. Ha, S. Kim, and J. Choo, “Stargan: Unified generative adversarial networks for multi-domain image-to-image translation,” in _Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)_ , 2018, pp. 8789–8797.
  * [47] T. Kaneko and H. Kameoka, “CycleGAN-vc: Non-parallel voice conversion using cycle-consistent adversarial networks,” in _Proc. European Signal Processing Conf. (EUSIPCO)_ , 2018, pp. 2100–2104.
  * [48] H. Kameoka, T. Kaneko, K. Tanaka, and N. Hojo, “Stargan-vc: non-parallel many-to-many voice conversion using star generative adversarial networks,” 2018, pp. 266–273.
  * [49] Y. A. Li, A. Zare, and N. Mesgarani, “Starganv2-vc: A diverse, unsupervised, non-parallel framework for natural-sounding voice conversion,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2021.
  * [50] H. Lu, D. Wang, X. Wu, Z. Wu, X. Liu, and H. Meng, “Disentangled speech representation learning for one-shot cross-lingual voice conversion using ß-vae,” in _IEEE Spoken Language Technology Workshop, SLTs_ , 2022, pp. 814–821.
  * [51] J. Williams, Y. Zhao, E. Cooper, and J. Yamagishi, “Learning disentangled phone and speaker representations in a semi-supervised VQ-VAE paradigm,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2021, pp. 7053–7057.
  * [52] Z. Li, B. Tang, X. Yin, Y. Wan, L. Xu, C. Shen, and Z. Ma, “Ppg-based singing voice conversion with adversarial representation learning,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2021, pp. 7073–7077.
  * [53] L. Sun, K. Li, H. Wang, S. Kang, and H. M. Meng, “Phonetic posteriorgrams for many-to-one voice conversion without parallel data training,” in _Proc. Intl. Conf. Multimedia and Expo (ICME)_ , 2016.
  * [54] H. Tang, X. Zhang, J. Wang, N. Cheng, and J. Xiao, “Avqvc: One-shot voice conversion by vector quantization with applying contrastive learning,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2022, pp. 4613–4617.
  * [55] D. Wu and H. Lee, “One-shot voice conversion by vector quantization,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020.
  * [56] B. Zhang, H. Lv, P. Guo, Q. Shao, C. Yang, L. Xie, X. Xu, H. Bu, X. Chen, C. Zeng, D. Wu, and Z. Peng, “WENETSPEECH: A 10000+ hours multi-domain mandarin corpus for speech recognition,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2022, pp. 6182–6186.
  * [57] “Pretrained models in wenet,” https://github.com/wenet-e2e/wenet/blob/main/docs/pretrained_models.md, 2022\.
  * [58] B. Zhang, D. Wu, Z. Peng, X. Song, Z. Yao, H. Lv, L. Xie, C. Yang, F. Pan, and J. Niu, “Wenet 2.0: More productive end-to-end speech recognition toolkit,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2022, pp. 1661–1665.
  * [59] A. Gulati, J. Qin, C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang, “Conformer: Convolution-augmented transformer for speech recognition,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2020, pp. 5036–5040.
  * [60] J. S. Chung, A. Nagrani, and A. Zisserman, “Voxceleb2: Deep speaker recognition,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2018, pp. 1086–1090.
  * [61] B. Desplanques, J. Thienpondt, and K. Demuynck, “ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2020, pp. 3830–3834.
  * [62] M. Ravanelli, T. Parcollet, P. Plantinga, A. Rouhe, S. Cornell, L. Lugosch, C. Subakan, N. Dawalatabad, A. Heba, J. Zhong, J. Chou, S. Yeh, S. Fu, C. Liao, E. Rastorgueva, F. Grondin, W. Aris, H. Na, Y. Gao, R. D. Mori, and Y. Bengio, “Speechbrain: A general-purpose speech toolkit,” _arXiv Preprint: arXiv.2106.04624_ , 2021.
  * [63] J. W. Kim, J. Salamon, P. Li, and J. P. Bello, “Crepe: A convolutional representation for pitch estimation,” in _Proc. IEEE Intl. Conf. on Acoustics, Speech and Signal Processing (ICASSP)_ , 2018, pp. 161–165.
  * [64] O. Chang, H. Liao, D. Serdyuk, A. Shah, and O. Siohan, “Conformers are all you need for visual speech recogntion,” _arXiv Preprint arXiv.2302.10915_ , 2023\.
  * [65] A. Défossez, N. Usunier, L. Bottou, and F. R. Bach, “Demucs: Deep extractor for music sources with extra unlabeled data remixed,” _arXiv Preprint arXiv.1909.01174_ , 2019.
  * [66] H. Zen, V. Dang, R. Clark, Y. Zhang, R. J. Weiss, Y. Jia, Z. Chen, and Y. Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” in _Proc. Conf. of Intl. Speech Commun. Assoc. (INTERSPEECH)_ , 2019, pp. 1526–1530.


# Predicting the evolution Of SARS-Covid-2 in Portugal using an adapted SIR
Model previously used in South Korea for the MERS outbreak

Pedro Teles
Departamento de Física e Astronomia
Faculdade de Ciências da Universidade do Porto
Rua do Campo Alegre s/n, 4169-007 Porto
<EMAIL_ADDRESS>

###### Abstract

Since original reports in Wuhan, China, the new coronavirus covid-19 has
spread very quickly worldwide, leading the World Health Organization (WHO) to
declare a state of pandemic. Moreover, as of the 13th of March 2020, the WHO
has announced that the European continent is now the main centre of the
pandemic. Many European governments have already implemented harsh measures to
attempt to contain the spread of the virus. In Portugal, there are, as of the
20th of March 2020, 785 confirmed cases. One of the questions many policy
makers, and governments are asking themselves is how the spread is going to
evolve in time. A timely idea of the amount of cases that will exist in a near
future can allow governments and policy makers to act accordingly. In this
study, I applied an adapted SIR model previously used in South Korea to model
the MERS outbreak, which is also caused by a coronavirus, to estimate the
evolution of the curve of active cases in the case of the Portuguese
situation. As some of the parameters were unknown, and the data for Portugal
is still scarce, given that the outbreak started later (first case on the 2nd
of March) I used Italian data (first reported case in Italy on the 31st of
January) to predict them. I then construct five different scenarios for the
evolution of covid-19 in Portugal, considering both the effectiveness of the
mitigation measurements implemented by the government, and the self-protective
measures taken by the population, as explained in the South Korean model. In
the out of control scenario, the number of active cases could reach as much as
40,000 people on the 5th of April (out-of-control scenario). If the self-
protective and control are taken in the same level as what was considered for
the South Korean model, this number could have be reduced to about 800 cases
(scenario 1). Considering that this scenario is now unrealistic. Three other
scenarios were devised. In all these scenarios, the government measures had a
50% effectiveness when compared to the measures in Korea. But in scenario 2
the transmission rate $\beta$ was effectively reduced to 50%, In this scenario
active cases could reach circa 7,000 people. In scenario 3, the transmission
rate $\beta$ was reduced to 70% of its initial value, in which the number of
cases would reach a peak of 11,000 people. And finally in scenario 4, $\beta$
was reduced to 80%. In this scenario, the peak would be reached at about
13,000 cases. These 4 scenarios clearly demonstrate that self-protective
measures can indeed have the much important effect of flattening the curve. In
the government and the people of Portugal manage to sustain a level of control
and self-protectiveness, the actual figures probably lie between the interval
( 7,000-13,000) and the peak will be reached between 10th and the 20nd of
April 2020. Without control and self-protective measures, this model predicts
that the figures of active cases of SARS-covid-2 would reach a staggering
40,000 people It shows the importance of control and self-protecting measure
to bring down the number of affected people by following the recommendations
of the WHO and health authorities. With the appropriate measures, this number
can be brought down to 7,000-13,000 people. Hopefully that will be the case
not just in Portugal, but in the rest of the World.

##  1 Introduction.

There is already abundant information on the new coronavirus and its spread
worldwide [1, 2, 3], as well as the risks it poses to the population. The
person infected with the severe acute respiratory syndrome coronavirus 2
(SARS-cov-2)[4] may develop the disease called COVID-19, which manifests with
mild to severe symptoms of fever, cough, shortness of breath and other signs
[5]. In the most severe cases, the infection can lead to the development of
acute respiratory distress syndrome (ARDS) causing respiratory failure, septic
shock, or multi-organ failure, and even death [6]. There is yet a lot about
this new coronavirus that is still unknown. Studies suggest that the case
fatality rate of the virus is of about 3.5% in mainland China, but the authors
suggest considering the range 0.2-3.0% [7]. However, this value seems to be
much higher in Italy [8], suggesting its strong dependence on demographics. In
fact, the Chinese Center for Disease Control, released an analysis that showed
that the case fatality rate ranged from 0% in children aged 0-9 years old to
14.8% in those over 80 [9], which implies that countries with a high number of
elderly citizens, as is the case of Italy and Portugal, are more prone to
having higher death rates. The quick spread of the virus in Europe has led the
WHO to declare Europe as the new epicentre of the disease on the 13th of March
of 2020[10]. The rapid growth of the number of active cases presenting severe
symptoms has saturated the health services in most countries in the continent,
especially in Italy[11]. It is yet unknown when the epidemic will reach an
end, but governments throughout Europe have implemented harsh measures to
prevent and mitigate the spread of the virus. Yet, as of today there are as
many as 77,394 confirmed cases on the European continent, in 46 territories,
resulting in 3,457 deaths, and rising[2, 3]. In Portugal, the two first
confirmed cases in the Portuguese territory were announced on the 2nd of
March, which puts the country slightly behind the curve in the outbreak in the
European continent. As of today, there are 642 confirmed cases, 2 deaths, and
3 recovered [12]. Also, the country is on lockdown since the 13th of March
2020, and the Portuguese Parliament declared a State of Emergency on the 18th
of March 2020. One of the questions many policy makers, and governments are
asking themselves is how the spread is going to evolve in time. A timely idea
of the amount of cases that will exist in a near future can allow governments
and policy makers to act accordingly. In this study, I adapted a model of the
SIR type used in South-Korea to model the evolution of the active cases of the
MERS outbreak in the country back in 2015 [13]. Although there are other
models being used [14, 15], this model seemed most appropriate given that the
MERS and the SARS-cov-2 are related illnesses, both provoked by a coronavirus
strand, and with similar symptoms, although MERS was much deadlier. In order
to fit the parameters and given that there is still very little data in the
case of Portugal, I used the data of Italy instead. This allowed me to fit the
curve of current active cases in Portugal with a model, which I then use, by
implementing the control measure parameters predicted in the model to predict
the future number of cases, in five different scenarios (the out-of-control
scenario, a scenario in which measures were the same as in the original model
(scenario 1), a scenario where government measures are 50% as effective as
those in South Korea, and self-protective measures reduce the transmission
rate by 50% (scenario 2), a similar scenario but in this case the transmission
rate was only reduced to 70% (scenario 3), and a fourth scenario, similar to
the two previous ones but in which the transmission rate was reduced to 80%.

##  2 Methodology.

The model used is the model described in Xia et al[13], which can be
understood by the flow diagram shown in figure 1 (which was taken from the
paper, page 4). In this model, S corresponds to the number of susceptible
individuals; E, the number of exposed individuals; A, the number of
asymptomatic infected cases; I, the total number of mild-to-severe infected
patients. H, the number of hospitalized cases; R, the number of removed cases,
and finally N, the total population of Portugal.

$\beta_{1}$ is the transmission coefficient of the asymptomatic infected
cases, $\beta_{2}$ is the transmission coefficient of the symptomatic infected
cases (mild infected person and severe patients) to the susceptible,
$\beta_{3}$ is the transmission coefficient of the hospitalized cases to the
susceptible, $\sigma^{-1}$ is the mean incubation period, $\lambda^{-1}$ is
the mean time between symptom onset to hospitalization, $k_{1}^{-1}$ is the
mean infectious period of asymptomatic infected person for survivors,
$k_{2}^{-1}$ is the mean duration for hospitalized cases for survivors,
$\delta^{-1}$ is the mean time from hospitalization to death, $\gamma$ is the
clinical outbreak rate in all the infected cases. The time unit is 1 day.

Figure 1: Flow chart of the SIR-type model used in this work, taken from [13]

The set of differential equations can then be written as:

$\systeme{\frac{\mathrm{d}S(t)}{\mathrm{d}t}=-\beta_{1}\cdot\frac{S(t)\cdot
A(t)}{N}-\beta_{2}\cdot\frac{S(t)\cdot I(t)}{N}-\beta_{3}\cdot\frac{S(t)\cdot
H(t)}{N},\frac{\mathrm{d}E(t)}{\mathrm{d}t}=\beta_{1}\cdot\frac{S(t)\cdot
A(t)}{N}+\beta_{2}\cdot\frac{S(t)\cdot I(t)}{N}+\beta_{3}\cdot\frac{S(t)\cdot
H(t)}{N}-\sigma\cdot
E(t),\frac{\mathrm{d}A(t)}{\mathrm{d}t}=(1-\gamma)\cdot\sigma\cdot
E(t()-k_{1}\cdot
A(t),\frac{\mathrm{d}I(t)}{\mathrm{d}t}=\gamma\cdot\sigma\cdot
E(t()-\lambda\cdot I(t),\frac{\mathrm{d}H(t)}{\mathrm{d}t}=\lambda\cdot
I(t)-k_{2}\cdot H(t)-\delta\cdot
H(t),\frac{\mathrm{d}R(t)}{\mathrm{d}t}=k_{1}\cdot A(t)-k_{2}\cdot
H(t)+\delta\cdot H(t)}$ (1)

Given that the available data for covid-19 is still very preliminary, the
model was further adapted. the only parameters that were considered “known”
were the mean incubation time, which was taken as $\sigma^{-1}$ =5.1 days from
the literature (here using the median as equal to the mean) [16], and the mean
infectious period of asymptomatic people, where a conservative estimate of
$k_{1}^{-1}=14$ days was used. All other parameters were taken to be
“unknown”. Furthermore, and because the fitting of the model was difficult to
achieve, I took $\beta_{1}=\beta_{2}=\beta_{3}=\beta$ as a first
approximation, meaning that the mean transmission coefficient is the same
independently of type and setting of the transmission. Finally, unlike Xia et
al[13], I consider that the epidemic starts with only one person infected
(I0=1), being the rest of the values 0, except for $S_{0}$ which is simply
N-1. See table 1 for a breakdown of the different parameters of this model.

The solving of the system of differential equations was performed using the
Mathematica code [17], which also allows for the use of fittable parameters,
using the function “NonLinearModelFit”[18]. Given that there is still very
little data for Portugal, the data for Italy was used as proxy. This seems
like the most reasonable approach (instead of using data from China or another
Asian country), given the cultural and health system proximity between the two
countries. In order to determine the parameters $\lambda$ and $\delta$, I used
the number of deaths, as reported by the Italian ministry of Health and
available online [19]. For the estimation of the value of $k_{2}$, the number
of recovered was used. All data was taken until the 19th of March 2020.

Table 1: The different parameters of the model described by Eq. 1 parameter | Xia et al values [13] | this work
---|---|---
$\sigma^{-1}$ | 5.2 (3$\sim$8) (days) | 5.1 (days)
$\lambda^{-1}$ | 5(3$\sim$7) (days) | To be fitted
$\delta^{-1}$ | 15.16(0$\sim$42) (days) | To be fitted
$k_{1}^{-1}$ | 5 (days) |
$k_{2}^{-1}$ | 7 (days) | To be fitted
$\beta_{1}$ | 0.8756(0.853$\sim$0.9324) ($days^{-1}$) | To be fitted
$\beta_{2}$ | 0.7833(0.5925$\sim$0.8592) ($days^{-1}$) | To be fitted
$\beta_{3}$ | 0.4568(0.3839$\sim$0.6751) ($days^{-1}$) | To be fitted
$\gamma$ | 0.0348 (0.0285$\sim$0.353) ($days^{-1}$) | To be fitted
N | 49,520,000 | 10,290,000
$S_{0}$ | 49,519,960 | 10,289,999
$E_{0}$ | 16$\sim$32 | 0
$A_{0}$ | 16$\sim$36 | 0
$I_{0}$ | 1 | 1
$H_{0}$ | 0 | 0
$R_{0}$ | 0 | 0

After determining the values for these parameters, they were used to fit the
curve of infected cases for Portugal, where the values of
$\beta_{1}=\beta_{2}=\beta_{3}=\beta$ and $\gamma$ were fitted to obtain a
model. This model was then used to estimate the evolution of future cases in
Portugal, according to five different scenarios:

  * •

Out-of-control scenario (nothing done, the virus is free to spread (out-of-
control scenario);

  * •

Scenario in which the government takes severe mitigating measures and the
population adheres to self-protecting measures equivalent to the original
model (scenario 1)

  * •

Scenario where government measures are 50% as effective as those in South
Korea, and self-protective measures reduce the transmission rate by 50%
(scenario 2)

  * •

Scenario where government measures are 50% as effective as those in South
Korea, and self-protective measures reduce the transmission rate by 70%
(scenario 3)

  * •

Scenario where government measures are 50% as effective as those in South
Korea, and self-protective measures reduce the transmission rate by 80%
(scenario 4)

##  3 Results.

###  3.1  Fitting of parameters $\lambda$ and $\delta$ (Italy).

The number of deaths in Italy was taken from [19], and the set of differential
equations described in eq. 1 fitted to obtain the best possible fit for
parameters $\lambda$ and $\delta$. An $R^{2}$ value of 0.99255 was obtained
with the fit. The rest of the parameters are presented in table 2. Results are
shown in figure 2.

Table 2: Table of fitted parameters $\lambda$ and $\delta$ as obtained using “NonLinearCurveFit” in Mathematica parameter | estimate | standard error | t-statistic | p-value
---|---|---|---|---
$\lambda$ | 0.2548 | 0.0002 | 879 | 9.83$\times 10^{-70}$
$\delta$ | 0.0168 | 0.0002 | 73 | 3.02$\times 10^{-36}$
Figure 2: Graphical representation of the fitted model to the Italian
government official death toll.

###  3.2 Fitting of parameter $k_{2}$ (Italy).

Now, using the values for $\lambda$ and $\delta$ found in 3.1, parameter
$k_{2}$ was obtained with a new fit of Eq. 1, but now to the number of
recovered cases in Italy, which again was taken from [19]. The best possible
fit for $k_{2}$ gave an $R^{2}$ value of 0.86178. The rest of the parameters
are presented in table 3.

Table 3: Table of fitted parameter $k_{2}$ as obtained using “NonLinearCurveFit” in Mathematica parameter | estimate | standard error | t-statistic | p-value
---|---|---|---|---
$k_{2}$ | 0.0875 | 0.012 | 7.32 | 3.09$\times 10^{-9}$

As can be seen, in the case of recovered cases, the fit has not the same
quality as the fit in 3.1. This is further evidenced by figure 3, where a
graphical representation of the fit is shown.

Figure 3: Graphical representation of the fitted model to the Italian
government official recoveries.

###  3.3 Fitting of parameters $\beta$ and $\gamma$ (Portugal).

After the two previous procedures, we now have all the parameters we need to
fit Eq. 1 to the number of Portuguese active cases until the 19th of March, in
order to determine the value of $\beta$ and $\gamma$. The number of Portuguese
active cases was taken from [20], which is updated daily.

The values previously obtained of parameters $\lambda$, $\delta$, and $k_{2}$
were used. As explained previously, because the fitting was difficult to
obtain, as a first approximation, we
considered$\beta_{1}=\beta_{2}=\beta_{3}=\beta$. The obtained values for
$\beta$ and $\gamma$ are given in table 5.

Table 4: Table of fitted parameters $\lambda$ and $\delta$ as obtained using “NonLinearCurveFit” in Mathematica parameter | estimate | standard error | t-statistic | p-value
---|---|---|---|---
$\beta$ | 1.16 | 0.033 | 34.997 | 1.51$\times 10^{-16}$
$\gamma$ | 0.0158 | 0.0034 | 4.634 | 2.74$\times 10^{-5}$

A graphical depiction of the fitted model is given in figure 4.

Figure 4: Graphical representation of the fitted model to the Portuguese
government official active cases.

In this scenario, the out-of-control scenario, in which the corona virus would
be “free” to spread within the country would lead to a peak of 40,000 infected
around the 5th of April 2020. A scenario that will very likely be avoided
given the strong containment measures implemented by the government.

###  3.4 Discussion of the obtained values for $\lambda$, $\delta$, $k_{2}$,
$\beta$, and $\gamma$

The fitting of these parameters was difficult to obtain. There is enough
systematic error and lack of statistics to make the present results very
unreliable. In order to obtain the parameters for $\lambda$, $\delta$, and
$k_{2}$, an ansatz value for $\gamma$ had to be given. In this model, $\gamma$
is originally defined as “the clinical outbreak rate in all the infected
cases”. In the cases of the fits of $\lambda$, $\delta$, and $k_{2}$, the
ersatz value used for $\gamma$ had a very strong impact in the results of the
other parameters. In fact, as it comes multiplied with $\sigma$ in the
transfer from compartment E(t) to compartments A(t) and S(t), I think the high
variation in this value could very well be due to the huge uncertainty there
is at the moment for the appropriate incubation time of this virus, which some
authors claim can be of up to 27 days [21], although this number is probably
an outlier. Also, a case of an incubation period of 18 days seems to have been
confirmed by a recent study[22]. The other parameters are somewhat less
elusive. The obtained value for $\delta=0.0168$ is 50% the value obtained by
Xia et al[13] of 0.0348, s 24% the value obtained by Xia et al[13] of 0.0660,
which shows the difference in case fatality rates between the SARS-covid-2
(7.2% currently for Italy), and MERS (estimated at 34%) which is 20%. This
seems reasonable, although the case fatality rate of the SARS-covid-2 is still
difficult to ascertain.

Finally, from the obtained values it is also possible to determine a value for
the basic reproduction number $R_{0}$, which here I take as being simply the
value $R_{0}=2\frac{\beta}{k_{1}+k_{2}}=14.5$ which demonstrates how strong
the capacity for spreading of this particular virus is. In table 5, the final
values of the obtained parameters are given.

Table 5: The obtained parameters of the model described in the previous sections parameter | this work ($days^{-1}$)
---|---
$\sigma$ | 5.1(fixed)
$\lambda$ | 0.2548$\pm$0.0002
$\delta$ | 0.0168$\pm$0.0002
$k_{1}$ | 0.0714 (fixed)
$k_{2}$ | 0.0875$\pm$0.012
$\beta$ | 1.16$\pm$0.033
$\gamma$ | 0.0158$\pm$0.034 (Portugal)***This value could go as high as 35 as an ersatz for the determination of the other values for the Italian data. Please see discussion.

##  4 Five different scenarios for the evolution of SARS-covid-2 in portugal.

The paper by Xia et al[13] further presents an extra 7 parameters to take into
account control measures in time evolution of the epidemic. These parameters
are meant to take into account two things:

  * •

Isolation and monitoring measures taken by the government (parameters $d_{1}$,
$d_{2}$, $d_{3}$, and $d_{4}$);

  * •

Self-protection measures taken by the population (parameters $l_{1}$, $l_{2}$,
and $l_{3}$).

The way these parameters are introduced in the model are as follows:

$\systeme{\frac{\mathrm{d}S(t)}{\mathrm{d}t}=-\beta_{1}\cdot
l_{1}\cdot\frac{S(t)\cdot A(t)}{N}-\beta_{2}\cdot l_{2}\cdot\frac{S(t)\cdot
I(t)}{N}-\beta_{3}\cdot l_{3}\cdot\frac{S(t)\cdot
H(t)}{N},\frac{\mathrm{d}E(t)}{\mathrm{d}t}=\beta_{1}\cdot
l_{1}\cdot\frac{S(t)\cdot A(t)}{N}+\beta_{2}\cdot l_{2}\cdot\frac{S(t)\cdot
I(t)}{N}+\beta_{3}\cdot l_{3}\cdot\frac{S(t)\cdot H(t)}{N}-(\sigma+d_{1})\cdot
E(t),\frac{\mathrm{d}A(t)}{\mathrm{d}t}=(1-\gamma)\cdot\sigma\cdot
E(t()-(k_{1}+d_{2})\cdot
A(t),\frac{\mathrm{d}I(t)}{\mathrm{d}t}=\gamma\cdot\sigma\cdot
E(t()-(\lambda+d_{3})\cdot
I(t),\frac{\mathrm{d}H(t)}{\mathrm{d}t}=\lambda\cdot I(t)-k_{2}\cdot
H(t)-(\delta+d_{4})\cdot H(t),\frac{\mathrm{d}R(t)}{\mathrm{d}t}=k_{1}\cdot
A(t)-k_{2}\cdot H(t)+\delta\cdot H(t)+d_{1}\cdot E(t)+d_{2}\cdot
A(t)+d_{3}\cdot I(t)+d_{4}\cdot H(t)}$ (2)

The $l_{1}$, $l_{2}$, and $l_{3}$ parameters are basically multiplication
factors for the transmission parameter $\beta$, so in our model we only
consider one value for it as we did in the previous sections. The $d_{1}$,
$d_{2}$, $d_{3}$, and $d_{4}$ parameters are summed to the parameters of
spread between the infected, asymptomatic and hospitalized cases, further
stressing their connection to implemented government measures. The values
provided by Xia et al[13], when applied to the Portuguese case, seem to
provide a very unrealistic scenario, in which the values would immediately
start to diminish. Perhaps this made sense in the case of the MERS epidemic,
given that it was much more localised that the one provoked by this virus. As
such, and for the lack of a better model, I devised four different scenarios
in addition to the “out-of-control” scenario described in the previous
section:

  * •

Scenario in which the government takes severe mitigating measures and the
population adheres to self-protecting measures equivalent to the original
model (scenario 1)

  * •

Scenario where government measures are 50% as effective as those in South
Korea, and self-protective measures reduce the transmission rate by 50%
(scenario 2)

  * •

Scenario where government measures are 50% as effective as those in South
Korea, and self-protective measures reduce the transmission rate by 70%
(scenario 3)

  * •

Scenario where government measures are 50% as effective as those in South
Korea, and self-protective measures reduce the transmission rate by 80%
(scenario 4)

Obviously other values could have been used, but there was no clear indication
of what these values could be, therefore I just assumed, as a first
approximation, that the protective measures would manage to diminish the
transmission of the contamination by 50%. For the government related measures
I used the same value as the ones from Xia et al[13]. In the second scenario I
keep the 50% reduction in the transmission rate but diminish the efforts of
the government by half. In the third and fourth scenarios I use 70% and 80% of
the transmission rate respectively.

The values for these scenarios are given in table 6.

Table 6: Control parameters of the model described by Eq. 2 parameter | Xia et al values[13] | scenario 1 | scenario 2 | scenario 3 | scenario 4
---|---|---|---|---|---
$d_{1}$ | 0.1922 | 0.1922 | 0.0961 | 0.0961 | 0.0961
$d_{2}$ | 0.1922 | 0.1922 | 0.0961 | 0.0961 | 0.0961
$d_{3}$ | 0.1922 | 0.1922 | 0.0961 | 0.0961 | 0.0961
$d_{4}$ | 0.1922 | 0.1922 | 0.0961 | 0.0961 | 0.0961
$l_{1}$ | 0.3239 | 0.5 | 0.5 | 0.7 | 0.8
$l_{2}$ | 0.9175 | 0.5 | 0.5 | 0.7 | 0.8
$l_{3}$ | 0.5316 | 0.5 | 0.5 | 0.7 | 0.8

The day considered for the initiation of these measures is day 33
corresponding to the 18th of March 2020 which is 5 days after the government
implemented the measures, so giving the virus another cycle of infection
before the measures take effect. Results are shown in figure 5.

Figure 5: Graphical representation of the different scenarios considered in
this study.

##  5 Conclusions.

Results show that measures can be effective in “flattening the curve”, as it
has come to be called this effect in recent times. In the first scenario the
effectiveness of the control measures seems to be at a maximum, and a maximum
of just about 800 cases would be reached. This has proven to be an unrealistic
scenarion given that as of today there are 1600 active cases in the country.
In scenario 2, the maximum is shifted to the 20th of April, reaching a maximum
number of 7,000 active cases in the country. In this scenario, the
transmission rate has been cut to a half, but the government’s measures
effectiveness has been cut to a half of the value from the original model used
in Korea. In scenarios 3 and 4, the measures of the government’s effectiveness
are kept at the same value, but the transmission rate is only cut to 70% and
80%, respectively. In scenario 3 the number of infected woulf reach a peak of
about 11,000 people, on the 12th of April, and finally in scenario 4, the peak
would be reached at 13,000 cases, on the 9th of April. In the out-of-control
scenario, the peak would be reached at about 40,000 people on the 5th of
April.

##  6 Limitations of this study and scope of application

Given the huge uncertainties concerning these numbers, and the current lack of
knowledge of the ongoing epidemic, it is very hard to make sense of its
evolution, even using well-established SIR methods. The current work makes a
series of assumptions which may or may not be proven correct in the future,
including the incubation period of the virus, which is still not well-known.
This proved to be a major source of uncertainty to fit the results, because it
had a great influence in the other parameters, especially parameter $\gamma$,
which seems to describe a probability of the person being exposed to the virus
being symptomatic or not. Instead this parameter seems to behave in this study
as an indicator of the huge uncertainty of the value for the incubation
period. Finally, epidemiologists and pathologists will tell you that these
parameters, unlike what is considered in this study, not only are not
constant, but dependent on time, location, culture, and other factors. It is
therefore particularly hard to model the current epidemic of covid-19 at this
stage. Also, this study takes into account the particular situation for one
country (Portugal), and we know that in globalised times, the likelihood that
new foci of infections arise on a multi-country scale is also very high.
Although currently most government have implemented serious travel
restrictions, it is now well-known exactly how this parameter could affect
this model, which does not take that into account. In fact, one of the major
issues with the SARS-covid-2 virus is that, unlike the other SARS, it seems to
be much less contained locally, as is evident by the amount of countries
afflicted. The amount of actual active cases is another clear source of
uncertainty. This model takes the values of active cases and compares them
with official confirmed numbers, which not only may be smaller than the actual
cases, as are released at an administratively defined time, which probably
does not correspond to something realistic.

The author believes that the values obtained in this study should be taken
qualitatively rather than quantitatively. One can think of the scenarios as
defining two limits of a likelihood interval, and as such, the actual figure
of maximum number of active cases in Portugal will likely be in the interval
7,000-13,000 between the 9th-20th of April, if the government and the people
of Portugal manage to keep implementing self-protective and control measures.
This can perhaps serve to advise the competent authorities in taking the
appropriate measures. It also serves to show the importance of control and
self-protecting measures. It is possible to bring down the number of affected
people by following the recommendations of the WHO and health authorities.
Without them, this model predicts that the figures of active cases would reach
a staggering 40,000 people. It also shows that with the appropriate measures
this can be indeed brought down, thus flattening the curve. Hopefully that
will be the case not just in Portugal, but in the rest of the World.

## References

  * [1] Novel Coronavirus (2019-nCoV) situation reports - World Health Organization (WHO) http://tiny.cc/bnyqlz
  * [2] Worldometer (https://www.worldometers.info/coronavirus/) accessed 18th March 2020
  * [3] Coronavirus Disease (COVID-19) – Research and Statistics, Max Roser, Hannah Ritchie and Esteban Ortiz-Ospina (2020), ourworldindata.org/coronavirus, accessed 18 March 2020
  * [4] The species Severe acute respiratory syndrome-related coronavirus: classifying 2019-nCoV and naming it SARS-CoV-2, Gobalenya AE, Baker SC, Baric RS, et al. (March 2020). Nature Microbiology: 1–9. doi:10.1038/s41564-020-0695-z. PMID 32123347.
  * [5] Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study Chen N, Zhou M, Dong X, Qu J, Gong F, Han Y, et al. (February 2020). Lancet. 395 (10223): 507–13. doi:10.1016/S0140-6736(20)30211-7. PMID 32007143
  * [6] COVID-19: what is next for public health?. Heymann DL, Shindo N, et al. (WHO Scientific and Technical Advisory Group for Infectious Hazards) (February 2020). Lancet. Elsevier BV. 395 (10224): 542–545. doi:10.1016/s0140-6736(20)30374-3. PMID 32061313.
  * [7] Case-fatality estimates for COVID-19 calculated by using a lag time for fatality Wilson N, Kvalsvig A, Telfar Barnard L, Baker MG.. Emerg Infect Dis. 2020 Jun [date cited]. https://doi.org/10.3201/eid2606.200320
  * [8] Why Deaths from Coronavirus Are So High in Italy, Rachael Rettner, LiveScience on March 10, 2020 http://tiny.cc/uoyqlz accessed 18 March 2020
  * [9] Novel Coronavirus Pneumonia Emergency Response Epidemiology Teamexternal icon. [The Epidemiological Characteristics of an Outbreak of 2019 Novel Coronavirus Diseases (COVID-19) in China]. Zhonghua Liu Xing Bing Xue Za Zhi. 2020;41(2):145–151. DOI:10.3760/cma.j.issn.0254-6450.2020.02.003.
  * [10] WHO Director-General’s opening remarks at the media briefing on COVID-19 - 13 March 2020 World Health Organization http://tiny.cc/cqyqlz accessed 18 March 2020
  * [11] The Extraordinary Decisions Facing Italian Doctors, Yascha Mounk, The Atlantic, 11 March 2020 http://tiny.cc/9qyqlz accessed 18 March 2020
  * [12] Ponto da Situação atual em Portugal, Direcção Geral de Saúde (in Portuguese), http://tiny.cc/esyqlz accessed 18 March 2020
  * [13] Modeling the Transmission of Middle East Respirator Syndrome Corona Virus in the Republic of Korea Xia Z-Q, Zhang J, Xue Y-K, Sun G-Q, Jin Z (2015). PLoS ONE 10(12): e0144778. doi:10.1371/ journal.pone.0144778
  * [14] A Time-dependent SIR model for COVID-19, Yi-Cheng Chen, Ping-En Lu, Cheng-Shang Chang arXiv:2003.00122v1 (28 Feb 2020)
  * [15] Risk Assessment of Novel Coronavirus COVID-19 Outbreaks Outside China. Boldog, P.; Tekeli, T.; Vizi, Z.; Dénes, A.; Bartha, F.A.; Röst, G. J. Clin. Med. 2020, 9, 571.
  * [16] The incubation period of coronavirus disease 2019 (COVID-19) from publicly reported confirmed cases: Estimation and application. Lauer SA et al. Ann Intern Med 2020 Mar 10; [e-pub]. (https://doi.org/10.7326/M20-0504)
  * [17] Wolfram Research, Inc., Mathematica, Version 12.1, Champaign, IL (2020).
  * [18] NonLinearModelFit https://reference.wolfram.com/language/ref/NonlinearModelFit.html accessed 18 March 2020
  * [19] Dipartimento della Protezione Civile COVID-19 Italia - Monitoraggio della situazione (in Italian) http://tiny.cc/5syqlz accessed 18 March 2020
  * [20] Ponto de Situação Atual em Portugal, Direção Geral de Saúde (in Portuguese) http://tiny.cc/1tyqlz accessed 18 March 2020
  * [21] Coronavirus incubation could be as long as 27 days, Chinese provincial government says - Reuters, Feb. 22, 2020
  * [22] Presumed Asymptomatic Carrier Transmission of COVID-19. JAMA. Bai Y, Yao L, Wei T, et al., JAMA. Published online February 21, 2020. doi:10.1001/jama.2020.2565