FitToWidth[1][]

# Enhancing Child Vocalization Classification in Multi-Channel Child-Adult
Conversations Through Wav2vec2 Children ASR Features

###### Abstract

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that often
emerges in early childhood. ASD assessment typically involves an observation
protocol including note-taking and ratings of child’s social behavior
conducted by a trained clinician. A robust machine learning (ML) model that is
capable of labeling adult and child audio has the potential to save
significant time and labor in manual coding children’s behaviors. This may
assist clinicians capture events of interest, better communicate with parents,
and educate new clinicians. In this study, we leverage the self-supervised
learning model, Wav2Vec 2.0 (W2V2), pretrained on 4300-hour of home recordings
of children under 5 years old, to build a unified system that performs both
speaker diarization (SD) and vocalization classification (VC) tasks. We apply
this system to two-channel audio recordings of brief 3-5 minute clinician-
child interactions using the Rapid-ABC corpus. We propose a novel technique by
introducing auxiliary features extracted from W2V2-based automatic speech
recognition (ASR) system for children under 4 years old to improve children’s
VC task. We test our proposed method of improving children’s VC task on two
corpora (Rapid-ABC and BabbleCor) and observe consistent improvements.
Furthermore, we reach, or perhaps outperform, the state-of-the-art performance
of BabbleCor.

Index Terms—  self-supervised learning, children ASR, vocalization
classification, Wav2vec 2.0, autism

††footnotetext: This work has been funded by the Jump ARCHES endowment through
the Health Care Engineering Systems Center at UIUC.

##  1 Introduction

ASD is a neurodevelopmental disorder characterized by deficits in social
communication and the presence of restricted interests and repetitive
behaviors [1]. ASD often begins in early childhood. According to estimates
from the Centers for Disease Control and Prevention, 1 in 36 children has been
identified with ASD in 2020 in the U.S [2]. ASD is a prevalent disorder in
children, but acquiring ASD diagnostic outcomes through conventional
evaluations may be protracted due to the limited available healthcare
professional services and the complexity of examination procedures [3]. Hence,
a robust machine learning (ML) model, capable of automatically detecting
features that clinicians consider relevant to the early diagnosis of ASD, may
be helpful for clinicians to better explain critical events to parents for
early intervention and educate new clinicians for future assessment.

Previous work has built ML-based speaker diarization (SD) models for Autism
Diagnostic Observation Schedule (ADOS) [4] interviews. ADOS is a well-
established protocol administrated by a trained clinician to elicit children’s
speech. Samples of ADOS interviews between clinicians and children aged 12-16
years were used in the series of DIHARD SD challenges [5, 6, 7]. Other related
studies performed child-adult SD [8, 9] or predicted atypical speech for
autistic children [10] using ADOS interviews.

To help diagnose toddlers aged 1-2, who are in the early stages of language
development and often can’t express themselves through complete sentences,
it’s crucial for the ML model to perform children’s vocalization
classification (VC), in addition to the child-adult SD task. ASD researchers
have reported that a low frequency of verbalization and non-verbal cues
(vocalization, laughter, crying, etc.) is one of the early behavioral signs
for autistic children [11]. Past work has studied automatic classification of
autistic children with various levels of severity based on their vocalizations
[12].

ASD researchers have relied on manually coding children’s social behaviors for
assessment, but coding is a time- and labor-intensive task. To combat data
sparsity issues, self-supervised learning models, such as wav2vec 2.0 (W2V2)
[13], have been recently proposed and have performed well on several speech
processing downstream tasks, including speech-to-text [14] and emotion
recognition [15], given a limited amount of labeled data. W2V2 first uses
large-scale unlabeled data for pre-training followed by fine-tuning on a small
amount of labeled data. A few recent studies also explored using self-
supervised learning models for child-adult SD and VC tasks [16, 17].

Most of the past work has primarily studied child-adult recordings on a single
audio channel. However, using multi-channel recordings of adult and children’s
speech is potentially beneficial for individually analyzing children’s
vocalizations. In this study, we explore building a unified end-to-end
W2V2-based system for SD and VC tasks on two-channel clinician-child
recordings for ASD assessment using Rapid-ABC (RABC) corpus. We propose a
novel technique incorporating auxiliary features extracted from a W2V2-based
automatic speech recognition (ASR) system for children under 4 years old to
improve children’s VC task. To the best of our knowledge, this is the first
study that leverages a self-supervised learning model to perform ASR mainly
targeting children under 4 years old. We further demonstrate the superiority
of our proposed method in children’s VC on two corpora that shared similar
annotation protocols (RABC and BabbleCor).

##  2 Data

In this study, we experiment with RABC for child-adult SD and VC task, and we
further validate our method of improving children’s VC task on BabbleCor. We
use MyST and Providence for building children’s ASR.

Rapid-ABC (RABC) [18] contains both video and audio recordings of 3-5 mins
brief interactive assessment between a child of 1-2 years old and a clinician.
Two audio streams are recorded separately, each from a lapel mic placed in
front of a speaker’s chest. Two types of annotations are available: protocol
segment, and child vocalization. Protocol segment annotations mark the times
at which the adult first speaks each of five key phrases, marking the start of
five parts of the diagnostic protocol. Child vocalization carefully labels
child audio as one of non-lexical vocalization (VOC), verbalization that
contains words (VERB), cry (CRY), or laugh (LAU). To evaluate SD, we manually
label adult audio as one of VOC or LAU. Ten percent of adult audio was double-
coded, and inter-coder reliability (Cohen’s kappa score) was 0.90 at a
precision of 0.2s. In this study, we analyze 51 sessions of audio recordings
from part of the RABC corpus available for research use, where a total of 4
clinicians and 43 children participated (with 8 children being assessed
twice). To prevent over-fitting, we use 3-fold cross-validation for
evaluation. We follow leave-one-child-out partition to ensure training and
testing set in each fold doesn’t have overlapped child participants. Table 1
presents details of partition of each fold. For fine-tuning W2V2, we label
audio stream in frames of 2s starting every 0.1s; the label is determined by
the centered 0.1s of each frame. The mean and standard deviation (std) of the
child utterances is 1.54s $\pm$ 2.30s.

[]  ADU m. CHI m. Total Fold # # of ID VOC LAU VOC VERB LAU CRY Time m. 1 28 /
15 38.6/20.3 2.4/1.3 4.4/1.4 2.3/1.5 1.4/.38 7.0/.53 93.9/48.3 2 29 / 14
39.4/19.5 2.7/1.0 4.0/1.8 2.3/1.6 1.1/.73 1.7/5.9 93.9/48.4 3 29 / 14
39.9/19.1 2.3/1.4 3.1/2.6 3.1/.77 1.1/.68 6.5/1.1 96.7/45.6

Table 1: Number of participants (“# of ID”) and duration of audio (“m.”=minutes) in each of six categories in each fold of cross-validation. Symbol “/” separates training and testing durations. “Total time” counts the length of entire recordings, including silence and non-speech events. partition | Non-CAN | CAN | LAU | CRY | JUNK | Total
---|---|---|---|---|---|---
training | 1324 | 410 | 42 | 223 | 1651 | 3650
development | 1521 | 352 | 37 | 151 | 1242 | 3303
testing | 1247 | 545 | 55 | 235 | 1252 | 3334
Table 2: Number of samples per class of BabbleCor.

BabbleCor [19] contains 11k short audio clips of 52 healthy children (2-36
month) without speech delay. The audio clips were collected from day-long
recordings of children at home using LENA [20], an infant-wearable audio
recording device. Annotators label the clips as one of canonical, (containing
a consonant to vowel transition), non-canonical, (not containing a consonant
to vowel transition), CRY, LAU, or junk (JUNK, same as silent or non-speech).
The current BabbleCor distribution includes about 90% of the data used for the
Baby Sounds (BS) sub-challenge in the 2019 Interspeech Paralinguistic
Challenge [21]. We use the speaker split of the BS challenge. Table 2 presents
details of our data partition. Mean duration of clips is 0.36s $\pm$ 0.08s
std.

My Science Tutor (MyST) [22] contains conversational speech of 1371 students
between third and fifth grades with a virtual tutor. We select short
transcribed utterances ($<15$s) for training and evaluating ASR to align with
shorter vocalizations produced by younger children. We have 84.2h/14.1h/15.1h
for training/development/testing set respectively. The mean and std of the
speech is 6.6s $\pm$ 3.6s.

Providence [23] contains longitudinal audio and video recordings of six
English-speaking children (3 boys: Alex, Ethan, and William, and 3 girls:
Lily, Naima, and Violet) from ages 1 to 4 interacting with their mothers at
home. Annotators transcribed children’s speech using SAMPA phonetic symbols.
We manually filter out some of the highly noisy recordings and use transcribed
child audio for fine-tuning W2V2 ASR. In total, we train on 84.0h long
utterances of four children (Ethan, William, Lily, and Naima) and test on
24.8h utterances of the other two (Alex and Violet). We also examine ASR
performance on utterances $<2s$ on the test set ($\sim$4.1h). The mean and std
of the utterances is 3.3s $\pm$ 2.1s.

Fig. 1: (a): W2V2 model architecture combining adult audio & child audio
with/without auxiliary W2V2-Children’s ASR (W2V2-CASR) features. MP=mean
pooling, WA=weighted average, and FFN=feed-forward network. ADU and CHI denote
adult and child VC tiers respectively. (b): Illustration of four combination
modules. Symbol “$+$” means summation and “$\bigoplus$” means concatenation.
For combination $C1$ and $C3$, $\alpha_{i}+\beta_{i}=1$ for $i\in\\{1,3\\}$.
(c): Explanation of feature dimension letters.

##  3 Methodology

###  3.1 Baseline W2V2 systems for child-adult SD and VC

Detailed model architecture and training procedures of W2V2 are described in
[13]. Briefly, W2V2 encodes raw audio into latent embeddings and learns to
predict masked embeddings of quantized speech units from contextual embeddings
during pretraining. We use W2V2-base (hidden feature size 768 with 12
transformer (TF) layers) in our study. For the baseline W2V2-based model on
RABC, we adapt the model from our previous work [24]. Specifically, input
waveforms from adult and child audio channels are separately fed into the W2V2
model to extract hidden features from 12 TF layers. Mean pooling (MP) across
the duration of the utterance is followed by a weighted average (WA) across
layers. Two feed-forward networks (FFN) are used as output tiers for
classifying adult and child audio as silence, or one of their VC types. Cross-
entropy losses of two output tiers are averaged as the overall loss. We test
individual vs. joint learning of the two audio channels, fine-tuning either
W2V2-LL4300h (pretrained on 4300h large-scale daylong home recordings of
children under 5 years old [24]) or W2V2-base (pretrained on 52k-hour of
unlabeled adult audio [13]). Figure 1 shows baseline joint learning using
W2V2-LL4300h plus the combination module described in Section 3.4.

###  3.2 Energy thresholding on two audio channels for SD

For RABC, we also test two energy thresholding (ET) baselines for SD. Each
baseline finds thresholds of the child and adult microphones using training
set of each fold, labels vocalization when the energy spectrum is above
threshold at frames of 0.1s, and smooths the results with 11-frame median
filter. Unsupervised ET: For each audio stream, we test multiple ETs and find
the one that best matches the Pydub voice activity detector [25] . Weak
supervised ET: Given labels, for each audio stream, we find an ET where
foreground speaker is always louder than the background speaker.

###  3.3 Learning children’s phonetics using W2V2

To train ASR for children under 4 years old, we use two-level fine-tuning to
gradually reduce age mismatch from adult ASR to child ASR.

W2V2-MyST: We build a CTC-based [26] ASR by integrating one linear layer of
hidden dimension 384 followed by Leaky Relu activation and softmax on top of
W2V2-Libri960h (W2V2-base fine-tuned on 960h LibriSpeech of adult speech). We
fine-tune W2V2-Libri960h using MyST data to minimize character-level CTC loss
for 40 epochs. For inference, we use CTC greedy decoding without language
model, obtaining 11.2% word error rate on the test set.

W2V2-Pro: We follow similar training and inference setup to fine-tune
W2V2-MyST using Providence for 5 epochs. We test two types of learning
outputs: sequences of phones, and sequences of binary consonant/vowel
indicators (CV). CV is a coarse quantization of the phone sequence, and we
hypothesize that CV may provide a useful training signal even when clips are
too short for the ASR to learn to generate phone sequences. We change the
number of output nodes to three (CTC $<$blank$>$, beginning-of-sentence, and
ending-of-sentence) plus the number of output targets (47 phones or 2 CV),
then re-initialize the weights of the top linear layer. We denote W2V2-Pro
(Pho/CV) as W2V2 trained on phonetic/CV sequences respectively for the rest of
the paper. For learning phone sequences, we obtain phone error rate of 61.3%
on the test set and 56.0% for utterances $<2s$. For learning CV sequence, we
obtain unit error rate of 28.1% on the test set and 22.3% for utterances
$<2s$. Note our goal of training children’s ASR is not getting highly accurate
transcripts but encouraging ASR to learn children’s phonetics.

###  3.4 Auxiliary W2V2 children’s ASR features

To leverage background speech information in RABC, we introduce a combination
(comb) module to fuse utterance-level W2V2 features from both audio channels
via summation or concatenation. Figure 1 presents the detailed model
architecture. For summation (comb $C1$), we set $\alpha_{1}=0.8$ and
$\beta_{1}=0.2$ for ADU tier, and $\alpha_{1}=0.2$ and $\beta_{1}=0.8$ for CHI
tier. Therefore, for each tier, features of foreground speaker have larger
weight than background speaker. We also explore combining W2V2-Children ASR
(W2V2-CASR) features with W2V2-LL4300h features in the child channel (comb
$C3$ and $C4$). During fine-tuning of W2V2-LL4300h, we freeze W2V2-CASR and
only extract the last TF layer features as auxiliary input. The last TF layer
may encode richer phonetics information as it’s closest to the output layer.
For both summation modules (comb $C1$ and $C3$), we ensure the weights of both
input features sum up to 1.

###  3.5 Experimental Setup

All waveforms are downsampled to 16 kHz. For RABC, each fine-tuning experiment
is trained 10 epochs with batch size 32 on single NVIDIA 1080 Ti for about 10
hours. Adam optimizer sets learning rates (LR) of 1e-4 for the output linear
layer and 1e-5 for the W2V2; scheduler with the new-bob technique adjusts LR
based on the test set performance (average of unweighted F1-scores over two
channels) after each epoch. Best performance of the test set is used for each
fold. SD outputs are computed by merging non-silent VC outputs then applying
an 11-frame median filter. SD task is evaluated using diarization error rate
(DER). We follow a common standard with 0.25s collar forgiveness over
reference segments. We diarize from scratch without knowing the reference
voiced segments. Overlapped speech segments are included for evaluating DER.
VC task is evaluated using unweighted F1-scores for each speaker over all
classes. We use similar setup on BabbleCor by ignoring the adult audio
channel. We tune LR as 3e-5/1e-5 for the output linear layer/W2V2
respectively. Optimal results for summation (comb $C3$) are obtained using
$\alpha_{3}=0.8$ and $\beta_{3}=0.2$. We evaluate children’s VC task using
unweighted average recall (UAR), the same metric used in BS challenge, and
unweighted F1-scores over all classes. We implement all experiments using
SpeechBrain [27]. Code and model weights are
available††https://huggingface.co/lijialudew/.

##  4 Results

###  4.1 Baseline models

Table 5 summarizes the baseline results. Weak supervised ET improves over
unsupervised ET by correcting errors related to noises and background speech
($\mathtt{B_{1}}$ vs. $\mathtt{B_{2}}$). Jointly fine-tuning on W2V2-LL4300h
greatly improves the performance of W2V2-base ($\mathtt{B_{3}}$ vs.
$\mathtt{B_{4}}$) because W2V2-base is pretrained on adult speech only and
fails to capture children’s phonetics. Separately learning two audio streams
yields slight benefit for SD task but not for VC task ($\mathtt{B_{4}}$ vs.
$\mathtt{B_{5}}$). Thus, we use joint modeling on W2V2-LL4300h for the rest of
our experiments.

[]  Exp ID Method DER ADU-F1 CHI-F1 $\mathtt{B_{1}}$ ET (unsupervised) 64.4
$\pm$ 6.3 - - $\mathtt{B_{2}}$ ET (weak supervised) 41.4 $\pm$ 4.0 - -
$\mathtt{B_{3}}$ joint (W2V2-base) 26.7 $\pm$ 15.1 81.7 $\pm$ 0.9 46.8 $\pm$
3.5 $\mathtt{B_{4}}$ joint (W2V2-LL4300h) 19.3 $\pm$ 5.1 83.3 $\pm$ 0.2 55.7
$\pm$ 1.7 $\mathtt{B_{5}}$ separate W2V2 19.2 $\pm$ 4.3 82.6 $\pm$ 0.1 54.4
$\pm$ 1.7 []  Exp ID comb ID W2V2-CASR type DER ADU-F1 CHI-F1 $\mathtt{E_{1}}$
$C1$ - 18.7 $\pm$ 4.8 83.5 $\pm$ 2.1 55.1 $\pm$ 1.3 $\mathtt{E_{2}}$ $C2$ -
17.3 $\pm$ 5.0 83.5 $\pm$ 1.5 56.7 $\pm$ 0.6 $\mathtt{E_{3}}$ $C3$ W2V2-Pro
($\beta_{3}$=0.5) 18.1 $\pm$ 5.0 83.4 $\pm$ 1.1 58.2 $\pm$ 1.2
$\mathtt{E_{4}}$ $C4$ W2V2-Pro 18.1 $\pm$ 5.2 83.4 $\pm$ 1.1 57.3 $\pm$ 1.1
$\mathtt{E_{5}}$ $C3$ W2V2-MyST ($\beta_{3}$=0.5) 17.8 $\pm$ 5.0 83.1 $\pm$
1.1 57.6 $\pm$ 0.1 $\mathtt{E_{6}}$ $C3$ W2V2-Pro ($\beta_{3}$=0.2) 17.8 $\pm$
3.7 83.4 $\pm$ 1.3 57.1 $\pm$ 1.1 $\mathtt{E_{7}}$ $C3$ W2V2-Pro
($\beta_{3}$=0.8) 18.3 $\pm$ 4.6 82.5 $\pm$ 2.2 58.4 $\pm$ 0.0 []  Exp ID
Method Dev-UAR Dev-F1 Test-UAR Test-F1 ComPare2019 baseline [21] 54.0 - 58.7 -
Gosztolya [28] 58.7 - 59.5 - Heysem Kaya et al. [29] 60.1 - 61.4 - Sung-Lin
Yeh et al. [30] 61.3 - 62.4 - $\mathtt{BC_{0}}$ Fine-tuning W2V2-LL4300h 67.6
64.1 62.9 64.7 $\mathtt{BC_{1}}$ $\bigoplus$ W2V2-Pro (Pho) 66.7 64.3 63.2
65.0 $\mathtt{BC_{2}}$ $+$ W2V2-Pro (Pho) ($\beta_{3}$=0.2) 70.4 64.1 62.2
64.5 $\mathtt{BC_{3}}$ $\bigoplus$ W2V2-Pro (CV) 69.2 64.3 64.9 66.3
$\mathtt{BC_{4}}$ $+$ W2V2-Pro (CV) ($\beta_{3}$=0.2) 69.2 65.1 62.6 65.1

Table 3: Mean and Std of DER and F1 scores of ADU and CHI VC tasks, in
percent, over 3-fold cross-validation on RABC corpus trained on baseline
models. ET=energy thresholding. Table 4: Mean and Std of DER and F1 scores of
ADU and CHI VC tasks on RABC corpus trained on different combinations.
W2V2-Pro is trained on phonetic sequences. Table 5: UAR and F1 scores for
development and testing set of BabbleCor in past studies (top 4 rows) and our
studies ($\mathtt{BC_{0}}$-$\mathtt{BC_{4}}$). Our tests lack 10% of the data
from BS challenge.

###  4.2 Auxiliary W2V2-CASR features

Table 5 shows the relevant results of combining two audio channels. We observe
combining both audio channels ($\mathtt{E_{1}}$ and $\mathtt{E_{2}}$) improves
DER over all baselines ($\mathtt{B_{1}}-\mathtt{B_{5}}$). We obtain optimal
DER using concatenation module ($\mathtt{E_{2}}$). By introducing W2V2-Pro
(Pho) features trained on phonetic sequences, $\mathtt{E_{3}}$ system
consistently improves children’s VC over 3 folds (not present in Table 5 for
brevity) while maintaining a comparable DER. This illustrates the advantages
of using children’s phonetics to improve children’s VC task. We find W2V2-Pro
(CV) is not as helpful as W2V2-Pro (Pho), so we only report results for
W2V2-Pro (Pho) in Table 5. We perform ablation studies based on comb $C3$ by
replacing W2V2-Pro (Pho) with W2V2-MyST ($\mathtt{E_{5}}$), reducing
($\mathtt{E_{6}}$) or increasing ($\mathtt{E_{7}}$) the weights of W2V2-Pro
(Pho) features. Children in the RABC test corpus are similar in age to the 6
Providence children than the 1371 MyST children, which may explain superior
performance of W2V2-Pro over W2V2-MyST. We also find increasing the weights of
W2V2-Pro (Pho) features leads to optimal CHI VC but slightly hurts ADU VC.
Perhaps overweighting children’s phonetic features may add variability to
adult speech embeddings and thus hurt ADU performance.

###  4.3 BabbleCor

Table 5 shows the results for BabbleCor. Fine-tuning W2V2-LL4300h yields a
competitive baseline ($\mathtt{BC_{0}}$), compared with previous works.
Interpolation with W2V2-Pro (Pho) features gives the best UAR on the dev set
($\mathtt{BC_{2}}$), while concatenation with W2V2-Pro (CV) gives the best
results on the test set ($\mathtt{BC_{3}}$). Possibly, BabbleCor’s clips may
be too short for W2V2-Pro (Pho) to extract a useful phonetic signal, but since
W2V2-Pro (CV) is trained using a simpler binary target sequence, it is able to
extract useful phonetic information even from these short clips. Though the
distributed BabbleCor does not contain all data from the BS challenge, results
suggest that our proposed method achieves or surpasses state-of-the-art
performance with almost identical data partition.

##  5 Conclusion & Future Work

The use of W2V2 features, pre-trained using 4300h of home recordings, improves
child-adult SD and VC on two-channel audio. With children’s ASR embeddings as
auxiliary features, we validate our proposed method on two corpora with
different lengths of child utterances. One limitation of our study is that,
for privacy reasons, the RABC corpus does not specify each child’s diagnosis
(ASD vs. non-ASD), so it is not possible to measure the quality of results
across diagnostic categories. In the future, we aim to extend our approach to
ASD children’s vocalizations when relevant data are available.

## References

  * [1] H. Hodges _et al._ , “Autism spectrum disorder: definition, epidemiology, causes, and clinical evaluation,” _Translational pediatrics_ , vol. 9, no. Suppl 1, p. S55, 2020.
  * [2] M. J. Maenner _et al._ , “Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, united states, 2020,” _MMWR Surveillance Summaries_ , 2023.
  * [3] E. Gordon-Lipkin _et al._ , “Whittling down the wait time: exploring models to minimize the delay from initial concern to diagnosis and treatment of autism spectrum disorder,” _Pediatric Clinics_ , 2016.
  * [4] C. Lord _et al._ , “The autism diagnostic observation schedule—generic: A standard measure of social and communication deficits associated with the spectrum of autism,” _Journal of autism and developmental disorders_ , vol. 30, pp. 205–223, 2000.
  * [5] N. Ryant _et al._ , “The third dihard diarization challenge,” _arXiv preprint arXiv:2012.01477_ , 2020.
  * [6] F. Landini _et al._ , “But system description for the third dihard speech diarization challenge,” in _Proc. 3rd DIHARD Speech Diarization Challenge Workshop_ , 2021.
  * [7] P. Singh _et al._ , “Leap diarization system for the second dihard challenge,” in _Interspeech_ , 2019, pp. 983–987.
  * [8] N. R. Koluguri _et al._ , “Meta-learning for robust child-adult classification from speech,” in _ICASSP_ , 2020.
  * [9] S. Krishnamachari _et al._ , “Developing neural representations for robust child-adult diarization,” in _2021 IEEE Spoken Language Technology Workshop (SLT)_. IEEE, 2021.
  * [10] M. Li _et al._ , “An automated assessment framework for atypical prosody and stereotyped idiosyncratic phrases related to autism spectrum disorder,” _Computer Speech & Language_, 2019.
  * [11] S. Ozonoff _et al._ , “A prospective study of the emergence of early behavioral signs of autism,” _Journal of the American Academy of Child & Adolescent Psychiatry_, 2010.
  * [12] A. Baird _et al._ , “Automatic classification of autistic child vocalisations: A novel database and results,” _Interspeech_ , 2017.
  * [13] A. Baevski _et al._ , “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in _NeurIPS_ , 2020.
  * [14] S. Schneider _et al._ , “wav2vec: Unsupervised pre-training for speech recognition,” _Interspeech_ , 2019.
  * [15] L. Pepino _et al._ , “Emotion recognition from speech using wav2vec 2.0 embeddings,” _Interspeech_ , 2021.
  * [16] A. Xu _et al._ , “Understanding spoken language development of children with asd using pre-trained speech embeddings,” _Interspeech_ , 2023.
  * [17] R. Lahiri _et al._ , “Robust self supervised speech embeddings for child-adult classification in interactions involving children with autism,” _Interspeech_ , 2023.
  * [18] J. Rehg _et al._ , “Decoding children’s social behavior,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2013.
  * [19] M. Cychosz _et al._ , “Vocal development in a large-scale crosslinguistic corpus,” _Developmental science_ , 2021.
  * [20] J. Gilkerson _et al._ , “The lena natural language study,” _Boulder, CO: LENA Foundation. Retrieved March_ , vol. 3, p. 2009, 2008.
  * [21] B. W. Schuller _et al._ , “The interspeech 2019 computational paralinguistics challenge: Styrian dialects, continuous sleepiness, baby sounds & orca activity,” _Interspeech 2019_ , 2019.
  * [22] W. Ward _et al._ , “My science tutor: A conversational multimedia virtual tutor for elementary school science,” _ACM Transactions on Speech and Language Processing (TSLP)_ , 2011.
  * [23] K. Demuth _et al._ , “Word-minimality, epenthesis and coda licensing in the early acquisition of english,” _Language and speech_ , 2006.
  * [24] J. Li _et al._ , “Towards robust family-infant audio analysis based on unsupervised pretraining of wav2vec 2.0 on large-scale unlabeled family audio,” _Interspeech_ , 2023.
  * [25] J. Robert _et al._ , “Pydub,” 2018. [Online]. Available: http://pydub.com/
  * [26] A. Graves _et al._ , “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in _ICML_ , 2006, pp. 369–376.
  * [27] M. Ravanelli _et al._ , “SpeechBrain: A general-purpose speech toolkit,” 2021, arXiv:2106.04624.
  * [28] G. Gosztolya, “Using fisher vector and bag-of-audio-words representations to identify styrian dialects, sleepiness, baby & orca sounds,” 2019.
  * [29] H. Kaya _et al._ , “Combining clustering and functionals based acoustic feature representations for classification of baby sounds,” in _Companion publication of the 2020 international conference on multimodal interaction_ , 2020, pp. 509–513.
  * [30] S.-L. Yeh _et al._ , “Using attention networks and adversarial augmentation for styrian dialect continuous sleepiness and baby sound recognition.” in _Interspeech_ , 2019, pp. 2398–2402.