# Data-driven Parsing Evaluation for Child-Parent Interactions

Zoey Liu
Boston College
<EMAIL_ADDRESS>&Emily Prud’hommeaux
Boston College
<EMAIL_ADDRESS>
###### Abstract

We present a syntactic dependency treebank for naturalistic child and child-
directed speech in English MacWhinney (2000). Our annotations largely followed
the guidelines of the Universal Dependencies project (UD (Zeman et al.,
2022)), with detailed extensions to lexical/syntactic structures unique to
conversational speech (in opposition to written texts). Compared to existing
UD-style spoken treebanks as well as other dependency corpora of child-parent
interactions specifically, our dataset is of (much) larger size ($N$ of
utterances = 44,744; $N$ of words = 233, 907) and contains speech from a total
of 10 children covering a wide age range (18-66 months). With this dataset, we
ask: (1) How well would state-of-the-art dependency parsers, tailored for the
written domain, perform for speech of different interlocutors in spontaneous
conversations? (2) What is the relationship between parser performance and the
developmental stage of the child? To address these questions, in ongoing work,
we are conducting thorough dependency parser evaluations using both graph-
based and transition-based parsers with different hyperparameterization,
trained from three different types of out-of-domain written texts: news,
tweets, and learner data. Our dataset and code are available at
https://github.com/zoeyliu18/childes_parsing.

##  1 Introduction

Research on syntactic dependency parsing has undoubtedly enjoyed tremendous
progress with the continuous development of the Universal Dependencies project
(Zeman et al., 2022) (hereafter UD). That said, of the 228 treebanks in the
latest version of UD (v2.10 at the time of writing), only 12 are treebanks of
fully spoken data (see also Dobrovoljc (2022)), while the rest focus on
different genres within the written domain (e.g. news, wikipedia texts). This
means that most (if not all) off-the-shelf dependency parsers that are
considered state-of-the-art are oriented towards written texts, rather than
tailored specifically for spontaneous speech. Therefore a natural question
arises: how well would parsing systems developed for written data perform when
it comes to spontaneous speech?

Over the past decade, there have been efforts devoted into dependency parsing
for the spoken domain, especially for (a subset of) the Switchboard corpus
(Godfrey et al., 1992), which contains transcripts of telephone conversations
in English; while some focused on parsing the full subset (Yoshikawa et al.,
2016; Rasooli and Tetreault, 2013; Miller and Schuler, 2008), others attended
to specific phenomena common in spoken data, such as speech repairment
(Miller, 2009a, b). In addition to English, dependency treebanks have been
developed for speech of other languages, including but not limited to French
Gerdes and Kahane (2009); Bazillon et al. (2012), Czech (Mikulová et al.,
2017), Russian (Kovriguina et al., 2018), Japanese (Ohno et al., 2005) and
Mandarin Chinese (He et al., 2018). These treebanks, including Switchboard,
however, were not always built on the basis of the UD guidelines; the
annotations and trained parsers are not always publicly available, making it
less straightforward to perform parsing evaluation with the data, especially
given the fact that the majority of dependency parsers are specific to UD-
formatted treebanks.

This paper presents a wide-coverage dataset of spontaneous child-parent
interactions (MacWhinney, 2000), annotated with syntactic dependencies largely
following the UD standards. We illustrate careful annotation guidelines that
hopefully would be useful to other developments of spoken dependency
treebanks. Compared to previous studies, our work goes beyond in several
respects (see Section 2). First, compared to most of the other spoken
dependency treebanks that contain telephone conversations (Bechet et al.,
2014), interactions between adults (Dobrovoljc and Martinc, 2018; Dobrovoljc
and Nivre, 2016), or user-generated content (Davidson et al., 2019), our
dataset attends to child and child-directed speech.

Second, in contrast to other spoken treebanks in the UD project, our dataset
as a whole is of considerable size: there are 26,098 utterances ($N$ of words
= 116,428) for child speech, and 18,646 utterances ($N$ of words = 117,479)
for parent speech.

Third, while there are some dependency corpora of child-parent interactions in
English (Sagae et al., 2010), Japanese (Miyata et al., 2013), and Hebrew
(Gretz et al., 2013), they only include data from one or two children, whereas
we provide annotations for the speech of 10 children across a wider age range,
therefore covering in more details lexical/syntactic phenomena that are more
unique to child speech (e.g., repetition, speech disfluency) and spoken
language more broadly.

With this dataset, in ongoing work, we ask two additional questions: (1) How
do state-of-the-art dependency parsing architectures trained on out-of-domain
data perform when it comes to naturalistic speech of separate interlocutors?
To address this question, we evaluate parsers trained on three genres within
the written domain: news texts, tweets and learner data, all in English. (2)
What is the relationship, if any, between parser performance and the
developmental stage of the child? In a way one might foresee a positive
correlation between the two, with the expectation that as the child continues
to develop their language skills, they would utilize more and more cohesive
syntactic structures, instead of, for example, produce speech with
unintelligible speech or word omissions. On the other hand, it is possible
that parser performance would increase as the child reaches a certain
developmental stage, then start decreasing, since the child might start
articulating sentences with more complex or expressive syntactic structures,
which thereby are potentially harder to analyze.

Child | 18-24 | 24-30 | 30-36 | 36-42 | 42-48 | 48-54 | 54-60 | 60-66
---|---|---|---|---|---|---|---|---
Violet | 1,730 | 1,855 | 1,889 | 1,886 | 1,894 |  |  |
Parent | 2,007 | 1,995 | 2,025 | 2,044 | 2,030 |  |  |
Naima | 1,748 | 1,752 | 1,814 | 1,902 | 1,943 |  |  |
Parent | 2,019 | 2,007 | 2,005 | 2,032 | 2,019 |  |  |
Thomas |  | 1,969 | 1,965 | 1,972 | 1,984 | 2,009 | 1,992 |
Parent |  | 2,013 | 2,035 | 2,050 | 2,016 | 2,025 | 2,010 |
Adam |  | 2,012 | 2,004 | 2,016 | 2,021 | 2,012 | 2,026 | 2,025
Parent |  | 2,012 | 2,009 | 2,020 | 2,008 | 2,015 | 2,007 | 1,234
Lily | 2,004 | 1,788 | 1,908 | 1,981 | 1,957 | 1,161 |  |
Parent | 1,996 | 2,016 | 1,999 | 2,038 | 2,006 | 2,000 |  |
Laura | 1,930 | 1,925 | 1,955 | 1,912 | 1,956 | 1,943 | 1,954 |
Parent | 2,022 | 2,019 | 2,013 | 1,991 | 2,023 | 1,984 | 2,015 |
Roman |  | 1,980 | 1,975 | 1,990 | 2,004 | 1,999 | 2,009 |
Parent |  | 1,712 | 2,017 | 2,012 | 2,020 | 2,013 | 2,027 |
Emma |  |  | 2,007 | 2,014 | 2,029 | 2,025 |  |
Parent |  |  | 2,020 | 2,038 | 2,015 | 1,455 |  |
Sarah |  | 2,014 | 2,008 | 2,011 | 2,007 | 2,020 | 2,015 | 2,028
Parent |  | 2,019 | 2,043 | 2,037 | 2,036 | 2,026 | 2,032 | 2,049
Abe |  | 2,007 | 2,007 | 2,022 | 2,020 | 2,036 | 2,021 | 1,386
Parent |  | 2,028 | 2,040 | 2,025 | 2,020 | 2,017 | 2,019 |
Table 1: $N$ of words for child and parent speech at different age ranges of
the children.

##  2 Related Work

Earlier work on dependency parsing of child-parent conversations (Sagae et
al., 2001, 2004) focused on the Eve corpus (Brown, 1973) from the CHILDES
project (MacWhinney, 2000), though the annotations did not follow those of UD.
Subsequent research extended the annotation guidelines for the Eve corpus
(Sagae et al., 2010) to child and child-directed speech in Japanese (Miyata et
al., 2013) and Hebrew (Gretz et al., 2013).

We note two studies that carried out UD-style dependency parse annotations for
child and/or child-directed speech.. Liu and Prud’hommeaux (2021) took a semi-
automatic approach to convert part of the existing dependency parses from the
Eve corpus (Brown, 1973) to UD standards; specifically, they focused on child
and parent utterances when the child is within 18-27 months old. Concurrent
work by Szubert et al. (2021) annotated dependency parses and semantic logical
forms for two languages: English (a large portion from the Adam corpus (Brown,
1973)) and Hebrew (The Hagar corpus (BERMAN, 1990)), although they only looked
at child-directed speech.

##  3 Meet the Data

For dataset construction, we borrowed transcripts of English naturalistic
parent-child interactions from the childes-db interface (Sanchez et al.,
2019), which contains data from the CHILDES (MacWhinney, 2000) database. As we
are interested in how parser evaluation results change at different
developmental stages of a child, we used age as a proxy for developmental
stage and set 6-month as one age bin. For every unique child from each corpus
of English, we calculated the total number of words produced by the child
($N\\_child$) and by the parent(s) ($N\\_parent$; excluding data from other
care-givers) respectively within each age bin of the child. From each age bin,
for both child and parent speech, we randomly sampled a number of utterances
(mostly without replacement) that amounted to approximately 2,000 words; the
criteria was relaxed somewhat in order to include data across a wide range of
age bins. This resulted in spoken data of ten children from 6 corpora (Table
1). In what follows, we briefly describe each of the selected corpora.

Kuczaj (Kuczaj II, 1977) The Kuczaj corpus includes speech from diary study of
the child, Abe; each original recording lasts $\sim$30 minutes.

Brown (Brown, 1973) From the Brown corpus we included naturalistic speech from
Adam and Sarah.

Thomas (Lieven et al., 2009) The Thomas corpus contains spoken interactions
between the child, Thomas, and primarily his mother at their house; each
initial audio recording is about an-hour long.

Weist (Weist and Zevenbergen, 2008) The data of Emma and Roman came from the
Weist corpus, which includes caregiver-child interactions recorded in either a
laboratory setting or in their own homes twice a month for around 30 minutes.

Braunwald We used recorded longitudinal speech of Laura when interacting with
different interlocutors; the speech was produced under naturalistic
environments.

Providence We took data of three children, Naima, Lily and Violet, from the
Providence corpus (Demuth et al., 2006); the data contains longitudinal
spontaneous interactions at home between the children and mostly their
mothers.

##  4 Annotation process

Our annotations largely followed those of UD (Zeman et al., 2022). Annotator
A, who has advanced training in dependency syntax, initially annotated data of
age 18-24 months and 24-30 months from Abe and Sarah, as a way to take notes
of any domain-specific phenomenon or cases that might not be straightforward
to annotate (see Section 5). These guidelines were discussed with annotator B
and modified if needed. Then given each age bin of every child, the two
annotators annotated 10% of data from both child and parent speech. We
calculated agreement scores using Cohen’s Kappa Artstein and Poesio (2008).
The overall agreement score taking into account all syntactic head and
dependency relation annotations of all data is 0.97; the average agreement
score across each dependency parse is 0.96. (We also computed agreement scores
for each child and the results are around 0.97). Final annotations of all data
were performed and checked by annotator A.

##  5 Annotation guidelines

Here we describe in details our approach to transcription orthography,
tokenization and dependency annotations for syntactic constructions that are
unique or more common in child speech and spoken data more broadly.

###  5.1 Orthography and tokenization

Regarding orthography of the transcripts, we made four decisions, all of which
are on the basis of a principle that we call “annotate what is actually
there". First, we did not perform any orthographic normalization of most
intelligible words in the speech (e.g., she wana eat); in other words, these
words stayed true to their original forms taken from CHILDES. That said, the
tokenizations of certain cases were updated following UD. These cases include:
(1) possessives (e.g., Daddy’s $\rightarrow$ Daddy ’s); (2) shortened copulas
(e.g., I’m eating $\rightarrow$ I ’m eating); (3) combined conjunctives (e.g.,
in_spite_of $\rightarrow$ in spite of ); (4) combined adverbs (e.g., as_well
$\rightarrow$ as well); (5) negation (e.g., don’t $\rightarrow$ do n’t); (6)
other informal contraction (e.g., gonna $\rightarrow$ gon na); (7) childish
expressions (e.g., poo_poo, choo_choo)

Second, unintelligible speech words (e.g., xxx, yyy) were removed as it is
hard to tell whether the words existed in the first place and whether they
have any syntactic roles in the utterances.

Third, we kept the initial capitalization in the transcripts, since most of
the times only proper names or words such as Mommy were capitalized.

Lastly, we omitted all punctuations with the exceptions of apostrophe (to
abide with UD’s standards), since punctuations tend to not be explicitly
articulated in actual spontaneous speech; they are instead added manually
during the transcription process based on judgments by the annotators.
Admittedly, under certain circumstances it is relatively easy to make choices
of punctuations; for instance, one could use a question mark at the end of an
utterance with a raising pitch contour. That said, deciding which punctuation
to use is not always easy and could be quite time-consuming; for example, for
spoken data, there could always be the problem of setting utterance boundary
(see Section 5.2), and accordingly, whether to insert, say, a period or a
semi-colon is not as simple as one might think. Therefore the process of
adding punctuations in transcripts is quite subjective itself.

###  5.2 (Vague) utterance boundary

Each instance in the dataset was originally treated as one utterance in
CHILDES, therefore we tried to annotate them as one sentence most of the
times. That said, the initial utterance boundaries are not always adequate.
This means that one extracted instance can be considered as having “side-by-
side" sentences (Figure 1) 111Examples presented in this paper are often
modified from the original utterances for ease of presentation. ; we abided by
the instructions of UD and annotated the first sentence to be the root; then
later sentences in the instance were treated as parataxis of the root.

[column sep=1em] she & ’s & up & she & ’s & swinging

3root 31nsubj 36parataxis 32cop 64nsubj 65aux

(a) Figure 1: Examples of (vague) utterance boundary.

###  5.3 Creative lexical usage

It is common that children make certain lexical choices that do not
necessarily follow the standards of parent speech or (formal) written data
(e.g., mine pillow; my mummy telephones me). On the other hand, these cases in
a way reflect children’s world, in the sense that they capture children’s own
understanding of these words and their lexical (and syntactic) development at
different stages. Therefore here we refer to such cases as creative usage; for
each case, we analyzed their syntactic usage given the remaining structure of
the sentence Lee et al. (2017); Santorini (1990), then assigned dependency
parses accordingly. For example, in Figure 2, the word magicked is creatively
used as a verb that links the subject I and object it.

[column sep=1em] I & magicked & it

2root 21nsubj 23obj

(a)

[column sep=1em] I & wan & na & pencil

2root 21nsubj 24obj 43det

(b)

Figure 2: Examples of creative lexical usage.

For some instances, it is relatively difficult to decide whether the utterance
contains the child’s creative usage of some lexical items, or potential
transcription errors induced by annotators. Given that transcribing spoken
data manually requires large amounts of time and energy, it is not against
expectations that the resulting transcriptions might have errors. For
instance, with Figure 2, it is not exactly clear whether the child really said
I wan na pencil, or the transcription should have been I want a pencil. Our
approach is that we compared how often each alternative occurs in our dataset,
then made the final decision. Therefore between the two alternatives above, we
chose to annotate na as the determiner of pencil.

###  5.4 Possible lexical omission

As we are taking a data-driven approach, we tried to perform annotations based
on how words or phrases are used within an utterance; this means that we
avoided assuming potential word omissions as much as possible. We assigned
dependency structures to an utterance if a reasonable parse (given the
context) could be derived without the assumption that certain words are
missing.

In other cases, we deemed the utterance as having lexical omission if the
omitted word could be automatically retrievable; this way other researchers
could formulate a different analysis for the utterance as they see fit. In our
annotations, we considered two types of lexical omissions. The first type is
copula omission, where the syntactic head of the copula is mostly a noun
(Figure 3), an adjective ((Figure 3), or an adposition ((Figure 4); other
times we assumed that a copula is omitted if the utterance can be interpreted
as an expletive structure (e.g., there book, with there as the expl dependent
of book). The second type is adposition omission, mostly the adposition that
is the function head of an oblique phrase (Figure 4) or the infinitival-to in
a complement clause (Figure 4).

[column sep=1em] I & girl

2root 21nsubj

(a)

[column sep=1em] he & heavy

2root 21nsubj

(b)

Figure 3: Examples of possible lexical omission: omission of copula.

[column sep=1em] I & like & to & go & Texas

2root 21nsubj 24xcomp 43mark 45obl

(a)

[column sep=1em] I & want & dad & come

2root 21nsubj 23obj 24xcomp

(b)

Figure 4: Examples of possible lexical omission: omission of adposition.

###  5.5 Nominal phrases

For certain nominal phrases that serve as adverbial modifiers in a given
utterance (e.g., Figure 5), and/or express time and dates, we tried to
annotate them more carefully using subtypes of specific dependency relations
(e.g., nmod:tmod or obl:tmod) (Schneider and Zeldes, 2021). For example, in
Figure 5, depending on their respective role, morning should be an oblique
phrase of go whereas tomorrow modifies morning).

[column sep=1em] wait & a & while

1root 13obl:npmod 32det

(a)

[column sep=1em] tomorrow & morning & we & can & go

5root 52obl:tmod 53nsubj 54aux 21nmod:tmod

(b)

Figure 5: Examples of nominal phrases.

###  5.6 Ambiguity

The syntactic structure of a sentence can be ambiguous when looking at the
sentence by itself. Therefore we took into account the surrounding context of
an utterance when performing annotations. In some cases, contexts could be
helpful; for example, in (1), like can be treated as the verb of the sentence,
rather than an adposition.

  1. (1)

Parent: do you like this

  2. Child: like this

In other (rare) cases, context might not be quite useful; for instance, in
(2), it is not clear whether rain should be a verb and the relation between
the two words is obl:tmod, or a noun and the dependency relation is nmod:tmod.
For these examples we opted for the simpler analysis given characteristics of
child speech and treated rain as a noun.

  1. (2)

Parent: eat your soup please

  2. Child: rain tonight

Another source of ambiguity comes from whether to treat proper names or words
like mommy and daddy as vocative or not, e.g., Momma try it. For these cases,
we decided to consider them as vocative if this interpretation is reasonable,
since subject omission is common in early child speech (Hughes and Allen,
2006).

###  5.7 Non-canonical word order

Often times in child speech or spoken data more broadly, an utterance does not
have the canonical word order that is presumed in (formal) written data. In
our dataset, cases as such usually involve post-posed subject and (copula)
verb (Figure 6). For these cases, we assigned dependency relations based on
the syntactic function of a word/phrase which is not constrained by their
relative orderings.

[column sep=1em] a & fork & this & is

2root 21det 23nsubj 24cop

(a) Figure 6: Examples of non-canonical word order.

###  5.8 Speech repairment

For speech repair, which captures one type of disfluency (Ferreira and Bailey,
2004), we used reparandum in the same way as suggested by the UD guidelines,
that is, the speech repair is the syntactic head of the subtree that
constitues the disfluent speech (e.g., seven in Figure 7). If the disfluent
speech contains discourse fillers or editing terms such as uh or um, these
elements are annotated to be the syntactic dependents of the repair with the
relation discourse (also to avoid unnecessary crossing dependencies). In some
more complicated cases, the disfluency subtrees are word fragments that do not
form a whole coherent phrase together (e.g., grab the in Figure 7); for these
cases, we used the principle of promotion to analyze elements within the
subtree structure of the disfluency if needed. For example, with Figure 7, the
word grab is most likely to be the head within the disfluency subtree,
therefore we promoted the following the to be the object of grab, then
analyzed the dependency relations of the residual structures in the instance.

To separate repairment from speech restart or abandonment (Section 5.9), we
only categorized an utterance as having speech repairment if the repairment
occurs in-between the utterance / is sentence-medial (as opposed to the
beginning of the sentence).

[column sep=1em] there & are & seven & six & dishes

2root 21expl 25nsubj 54nummod 43reparandum

(a)

[column sep=1em] well & grab & the & use & the & spoon

4root 41discourse 23obj 42reparandum 46obj 65det

(b)

Figure 7: Examples of repairment; the dependency relation for speech repair in
each example is in blue.

###  5.9 Speech restart

Another type of disfluency is speech restart. We generally considered an
instance as having speech restart or abandonment if the abandoned elements
occur at the beginning of the instance and do not form a coherent phrase
together; what’s more, the abandoned elements need to be different from the
speech restart. For these cases, given that speech restart falls broadly under
the umbrella of disfluency, and in order to distinguish restart from
repairment above, we extended reparandum with a new dependency relation
subtype: reparandum:restart to connect the abandoned elements as the
dependents of the speech restart. This way the dependency relation will also
go “right-to-left" (Dobrovoljc, 2022), following the usage of reparandum.

[column sep=1em] I & just & you & ate & it & too

4root 41reparandum:restart 43nsubj 45obj 46advmod 12advmod

(a) Figure 8: Examples of restart; the dependency relation between repeated
elements in each example is in orange.

###  5.10 Repetition

Overall we identify three major kinds of repetitions. For the first type, an
utterance consists of repetitions of the same dependency subtree and the
repeated subtree is a coherent phrase by itself. Examples include cases such
as discursive repetition (e.g., no no mommy; Figure 9), onomatopoeia (e.g.,
honk honk), or repetition of other kinds of word or phrase (e.g., this is my
truck my truck; Figure 9). For these cases, we treated the first appearance of
the repeated subtree as the syntactic head with the following repetitions as
the dependents connected with the relation conj. A special case is when the
instance repeats a full sentence (or just containing a verbal phrase?), e.g.,
I did it I did it (Figure 9); for these examples we used parataxis to adhere
to the annotations of side-by-side sentences noted by UD.

For the second type, repetition is used to emphasize the characteristics of
certain objects or conditions (or serves as an intensifier (Szubert et al.,
2021)); in these cases the repeated element is usually a single word with a
POS tag of adjective or adverb. These cases were annotated similarly as those
from the first type above (Figure 9).

The third type of repetition pertains disfluency. Whether to interpret an
instance as disfluent repetitions or not could be challenging when the
instance does not have a corresponding audio, where the prosody could help
guide the interpretation. Therefore to distinguish the two types mentioned
before, we considered an instance as having disfluent repetitions if the
repetition appears at the beginning or in-between a sentence; in addition, the
repeated element has to be either word fragments that do not form a whole
coherent phrase together when taking the sentential context into account
(Figure 9), or a single word and its POS tag is neither an adjective or an
adverb (Figure 9). For these cases, in order to not go against the fact that
conj is usually applied left-to-right (i.e., syntactic head precedes its
dependents), we again extended the usage of reparandum and applied a new
dependency relation subtype: reparandum:repetition to describe repetition in
disfluency; speech repairment, restart and disfluent repetition hence would
also be easily distinguishable in an automatic fashion.

[column sep=1em] no & no & mommy

1root 12conj 13vocative

(a)

[column sep=1em] this & is & my & truck & my & truck

4root 41nsubj 42cop 65nmod 46conj

(b)

[column sep=1em] I & did & it & I & did & it

2root 21nsubj 23obj 25parataxis 54nsubj 56obj

(c)

[column sep=1em] she & saw & a & big & big & ballon

2root 21nsubj 26obj 36det 64amod 45conj

(d)

[column sep=1em] I & ’m & in & the & in & the & park

7root 71nsubj 72cop 43case 74reparandum:repetition 75case 76det

(e)

[column sep=1em] his & his & book

3root 32nmod:poss 21reparandum:repetition

(f)

Figure 9: Examples of repetition; the dependency relation between repeated
elements in each example is in teal.

###  5.11 Other structures

The last three types of structures to be mentioned are serial verb
construction (SVC), tag question and unintelligible structure where we could
not decide on a clear (or sometimes any) interpretation. For SVC, we followed
Szubert et al. (2021); for an utterance such as he came see me play, see was
treated as the dependent of came and the relation is compound:svc.

Tag questions are mostly used in parent speech (e.g., you like that book do
n’t you); for these examples we abided by the UD annotations and used
parataxis to connect the tag question to the main clause of the utterance.

Lastly, for utterances without a clear syntactic structure, we used dep to
connect the individual word fragments. Given that UD advises to avoid applying
this relation as much as possible, we restricted its usage to mostly
utterances produced in early developmental stages of the child (e.g., 18-24
months), where the utterances are composed of usually two to three words in
total.

##  6 Experiments

###  6.1 Meet the parsers

We experimented with two graph-based parsers and one transition-based parser
using their default parameters: (1) Diaparser (Attardi et al., 2021): A graph-
based biaffine parser model (Dozat and Manning, 2017) which uses a language
model (LM) with three BiLSTM layers as encoder. (2) MaChamp (van der Goot et
al., 2021): A multi-task learning toolkit which includes the option of
dependency parsing; the parser is also a graph-based biaffine parser (Dozat
and Manning, 2017). To increase the variety of parser hyperparameterization,
for both Diaparser and MaChamp, we explored three LMs as encoders: bert-base-
cased (bert) (Devlin et al., 2018), roberta-base (roberta) (Liu et al., 2019),
and twitter-roberta-base (twitter) (Barbieri et al., 2020). (3) UUParser (de
Lhoneux et al., 2017; Smith et al., 2018): A greedy transition-based parser
that extends the model from Kiperwasser and Goldberg (2016) via allowing
predictions to have non-projective dependency trees (Nivre, 2009).

###  6.2 Out-of-domain data

Data | Set |  $N$ of sentences |  $N$ of words
---|---|---|---
EWT | train | 12,543 | 204,579
| dev | 2,001 | 25,149
| test | 2,077 | 25,097
Tweebank | train | 1,639 | 24,753
| dev | 710 | 11,742
| test | 1,201 | 19,112
ESL | train | 4,124 | 78,541
| dev | 500 | 9,549
| test | 500 | 9,591
Table 2: Descriptive statistics for out-of-domain data sets. Data | Parser | LM | LAS
---|---|---|---
EWT | MaChamp | bert | 90.49
|  | roberta | 91.27
|  | twitter | 90.84
| Diaparser | bert | 89.89
|  | roberta | 88.40
|  | twitter | 88.26
| UUParser | - | 82.68
Tweebank | MaChamp | bert | 79.57
|  | roberta | 80.49
|  | twitter | 80.26
| Diaparser | bert | 80.01
|  | roberta | 79.48
|  | twitter | 79.65
| UUParser | 69.54 |
ESL | MaChamp | bert | 91.70
|  | roberta | 92.27
|  | twitter | 91.94
| Diaparser | bert | 91.48
|  | roberta | 90.74
|  | twitter | 90.60
| UUParser | - | 86.57
Table 3: Parser evaluation results for out-of-domain data sets.

We used three out-of-domain datasets of written English (Table 2): (1)
UD_English-EWT (EWT) (Silveira et al., 2014), which contains written texts
from five genres of web media, including weblogs, newsgroups, emails, reviews,
and Yahoo! answers; (2) UD_Tweebank (Tweebank) (Liu et al., 2018), which
consists of English tweets; (3) UD_English-ESL (ESL) (Berzak et al., 2016),
which includes writing of learner English taken from the Cambridge Learner
Corpus First Certificate in English dataset (Yannakoudakis et al., 2011).

For each of the aforementioned datasets, we trained the parsers described in
Section 6.1 with their default parameters. We calculated micro unlabeled
attachment scores and labeled attachment scores (LAS) to evaluate parser
performance. Due to space limitation, throughout this paper we focus on
reporting LAS. Parser evaluation results across three random seeds for all
out-of-domain datasets are reported in Table 3.

###  6.3 Evaluation of out-of-domain parsers

We first performed automatic part-of-speech tagging for all child and parent
speech using the publicly-open NLP library Stanza (Qi et al., 2020). We then
applied each of the out-of-domain parsers to child and parent speech within
each age range of the child; parser performance was again index by LAS across
3 random seeds. We foresee two directions regarding the parsing results. On
one hand, EWT has significantly much more data in contrast to Tweebank and
ESL, which might lead to overall better results on child and parent speech.
That said, among the three out-of-domain datasets, the domains of Tweebank and
ESL are possibly most relevant or more similar to child-parent interactions,
in the sense that they are less “formal" compared to EWT. This potentially
means that parsers trained from Tweebank or ESL might outperform those based
on EWT.

Note that to be fair for the out-of-domain parsers, here the evaluations did
not consider new dependency relation subtypes that were introduced in our
annotations, which are reparandum:restart and reparandum:repetition.

(a) MaChamp

(b) UUParser

Figure 10: Evaluation of out-of-domain Tweebank parsers; at each age range of
the children, the parser score for child (or parent) speech was averaged
across all children (or parents) within that age range.

##  7 Results and analysis

We present analysis mainly for results based on MaChamp; though we performed
the same analysis for the other two parsers as well and there were no
noticeable discrepancies in the patterns. (While there are differences in
numerical results given that UUParser was not trained with neural LMs).

###  7.1 Parent’s speech

When looking at parser performance for all parents across different age ranges
of their children, on average parsers trained on the out-of-domain ESL dataset
with twitter achieved the best result (83.88). By contrast, the best parsers
from EWT, which were also trained with twitter notably performed worse; the
average difference in LAS ranges from 2.11 between the age range of 48-54
months, to 3.95 between the age range of 18 to 24. On the other hand, the best
parsers from Tweebank, trained with bert, achieved comparable performance
(83.48) to the best parsers based on the ESL dataset. What’s worth noting
here, is that comparing the parsing results of different LMs, the
discrepancies in LAS for Tweebank are smaller than those for ESL. For example,
the biggest difference in LAS for Tweebank is 0.80 at the age range of 60-66
months between bert and roberta; whereas for ESL, the smallest discrepancy is
1.51 (48-54 months), and can be as large as 3.42 (18-24 months). This means
that parsers trained from Tweebank, which contains less than 1/8 of the amount
of data from EWT, and around 1/3 from ESL, are able to lead to good and
reliable performance.

So where do the discrepancies between parsers trained from different out-of-
domain treebanks come from, especially during early ages of the children?
Comparing the best performing parsers based on EWT and those from Tweebank and
ESL, it appears that for some utterances where the copula takes the form of
‘’s’, parsers trained from EWT errorneously annotated the copula as the
subject, therefore assigning two subjects to the same syntactic head; this
accounts for around 17.07% of all differences between the parsers’ and our
manual annotations. This raises the worrisome question of why a structure
where the syntactic head has two subjects would arise. The most plausible
answer is that such structures exist in the training set. To check that, we
looked into the training data of the EWT treebank and found five relevant
sentences. In these cases, the first subject between the two were incorrectly
annotated as headed by the verb of the subordinate clause at a lower level
(e.g., in it is not about how much you earn (adapted), it was annotated as the
subject of earn). Similarly, in cases where ‘’s’ has a dependency relation of
aux, the parsers trained from EWT also tended to assign it a relation of nsubj
instead; this led to around 7.69% of all the discrepancies between parser and
manual annotations. The patterns described above seem to be the main and
consistent explanation for the performance discrepancies between the best
parsers trained from different out-of-domain treebanks, even when the children
get older; between the age range of 54-60 months, around 11.30% of all
differences when comparing parsers’s annotations to those done manually came
from the parsers assigning nsubj instead of cop; and the number is
approximately 11.99% when the age range is between 60 and 66 months.

Now let us turn to the question of where do best-performing parsers trained
from out-of-domain treebanks fall short in general? We note four cases here.
During early ages of the children, for parent speech, the dependency relation
that results in the biggest discrepancy between parser and manual annotations
is nmod:poss, e.g., my book, where the parsers annotated the relation as nmod,
which is less preferred in latest annotation guidelines of UD. The second case
that caused confusion for the parsers is when the sentences contain elements
that should be annotated as discourse and/or vocative (e.g., hahaha I see,
Roman oh that is beautiful) whereas the parsers appeared to more consistently
annotate the first word in these instance as the root of the sentence.

One other noteworthy example is when the parsers think of a vocative as the
subject (e.g., Adam eat your soupe) or sometimes object (e.g., sit Sarah) of
the utterance, since there is no punctuation in the annotations and the main
clause of the sentence is subjectless or should not take an object at all. The
last one is conj, which we used for cases of repetition or when the speaker
appears to be listing individual nouns (e.g., orange grape apple); the
parsers, however, annotated the relation between pairs of nouns to be
compound, which was common in the training sets of the out-of-domain treebanks
given that they were annotated (largely) based on UD guidelines.

As the children age, the aforementioned dependency relations still turned out
to be the main reasons to explain parser performance, though to lesser
extents. Note this was not because the parsers started correctly annotating
these relations, but due to the fact that the relevant instances occur less
frequently in parent speech at later age ranges of the children, at least in
the data samples that we selected and annotated. In addition, another
dependency relation that the parsers tended to not annotate correctly at later
developmental stages of the children is compound:prt, in cases such as hurry
up; instead the parsers assigned compound, which is not entirely wrong, but
less precise.

###  7.2 Child’s speech

Overall parser performance for child’s speech demonstrates similar patterns to
those found for parent’s speech. Across all age ranges, again the best parsers
trained on the ESL treebank using twitter and the best parsers trained on the
Tweebank with bert arrived at comparable performance (79.35 vs. 79.39); these
parsers also outperformed the best parsers based on the EWT treebank (76.48),
indicating that ESL and Tweebank, especially the latter, suffice for yielding
reasonable parser performance despite their much smaller dataset scales in
contrast to EWT.

In terms of what dependency relations resulted in the discrepancies between
parser performance and manual annotations, we identified four main categories
that are applicable especially during early ages of the children, which again
are similar to those noted for parent speech. The first one is conj; the
parsers tended to assign compound for an utterance that consists mostly of
individual nouns (book table pencil) where none of the nouns modifies each
other.

The other three categories appear to be mainly caused by the utterances
potentially having word omissions or ambiguous structures. For instance, when
the utterances consist of two words, where the first one was treated as the
subject of the second (Adam home) in our annotations, given the length of the
utterances, the parsers again preferred to annotate Adam as the compound of
home. Another case pertains nmod:poss; in addition to assigning nmod for
utterances that contain possessives such as my, the parsers also used nmod for
utterances that possibly involve the omission of the possisive marker ‘’s’.
The last one concerns the treatment of vocative, which again sometimes were
annotated by the parsers as nsubj or obj.

Notice there is small fluctuation in LAS between the age of 48-54 months. When
looking at the differences between parser and manual annotations for this age
range, we see an additional type of instance where the parsers committed
errors, which involve a head verb with an adverbial modifier, such as bring
them back; rather than parsing back as advmod of bring, the parsers assigned
aux.

In all, contrasting results for child speech to those for parent speech, not
so surprisingly, parser performance is better overall for the latter. That
said, when children reach later ages, the parsers’ evaluation results
($\sim$87.83) approach those for parent speech ($\sim$89.13), suggesting that
children’s development of syntactic structures are becoming more parent-like.
While parser scores for parent speech slowly increase between the age range of
18-66 months, this progress is much more pronounced for child speech; in other
words, we do see an overall improvement of parser performance as children
progress along the syntactic developmental trajectory, particularly within
18-42 months.

##  8 Ongoing in-domain parser training and evaluation

After observing the performance of out-of-domain parsers, we now turn to
evaluating the performance of parsers trained with in-domain data of child-
parent interactions; in particular, these parsers are expected to pick up the
new dependency relations for disfluencies that we introduced, at least to some
extent. Based on results from Section 7, on average parsers trained with
MaChamp using the twitter LM seemed to achieve the most stable performance,
therefore we adopted the same parser setup here.

Our training scheme is as follows. Say we want to evaluate parser performance
for Adam from the Brown Corpus. We first trained parsers using all the data
from the other nine child-parent pairs; we then measured parser performance
for both child and parent speech at each age range of the child using LAS
averaged across 3 random seeds.

##  9 Conclusion

We present a wide-coverage dataset of child-parent interactions annotated with
syntactic dependencies, along with detailed annotation guidelines extending
the Universal Dependencies project. The dataset covers child and parent speech
from the age range of 18-66 months for the children. Evaluations from graph-
based and transition based dependency parsers with varying hyperparameters
demonstrate that parsers trained using a relatively small amount of English
tweets (Tweebank) are able to yield comparable or even outperform parsers
trained from much larger dependency treebanks. In addition, we observed the
general trend that on average, parser performance increases as the children
reach older ages, indicating that as children progress along their syntactic
developmental trajectory, they start producing more cohesive structures but
not too complex for the parsers to handle. We wait to verify this again with
our ongoing work of in-domain parser training and evaluation..

## References

  * Artstein and Poesio (2008) Ron Artstein and Massimo Poesio. 2008.  Inter-coder agreement for computational linguistics.  _Computational linguistics_ , 34(4):555–596.
  * Attardi et al. (2021) Giuseppe Attardi, Daniele Sartiano, and Maria Simi. 2021.  Biaffine Dependency and Semantic Graph Parsing for Enhanced Universal Dependencies.  In _Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021)_ , pages 184–188, Online. Association for Computational Linguistics.
  * Barbieri et al. (2020) Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020.  TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1644–1650, Online. Association for Computational Linguistics.
  * Bazillon et al. (2012) Thierry Bazillon, Melanie Deplano, Frederic Bechet, Alexis Nasr, and Benoit Favre. 2012.  Syntactic annotation of spontaneous speech: application to call-center conversation data.  In _Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12)_ , pages 1338–1342, Istanbul, Turkey. European Language Resources Association (ELRA).
  * Bechet et al. (2014) Frederic Bechet, Alexis Nasr, and Benoit Favre. 2014.  Adapting dependency parsing to spontaneous speech for open domain spoken language understanding.  In _Fifteenth Annual Conference of the International Speech Communication Association_.
  * BERMAN (1990) RUTH A. BERMAN. 1990.  On acquiring an (S)VO language: subjectless sentences in children’s Hebrew.  28(6):1135–1166.
  * Berzak et al. (2016) Yevgeni Berzak, Jessica Kenney, Carolyn Spadine, Jing Xian Wang, Lucia Lam, Keiko Sophie Mori, Sebastian Garza, and Boris Katz. 2016.  Universal Dependencies for Learner English.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 737–746. Association for Computational Linguistics.
  * Brown (1973) Roger Brown. 1973.  _A first language: The early stages._ Harvard University Press.
  * Davidson et al. (2019) Sam Davidson, Dian Yu, and Zhou Yu. 2019.  Dependency parsing for spoken dialog systems.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 1513–1519, Hong Kong, China. Association for Computational Linguistics.
  * Demuth et al. (2006) Katherine Demuth, Jennifer Culbertson, and Jennifer Alter. 2006.  Word-minimality, epenthesis and coda licensing in the early acquisition of English.  _Language and speech_ , 49(2):137–173.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * Dobrovoljc (2022) Kaja Dobrovoljc. 2022.  Spoken Language Treebanks in Universal Dependencies: an Overview.  In _Proceedings of the 13th Language Resources and Evaluation Conference_ , page 1798–1806, Marseille, France. European Language Resources Association.
  * Dobrovoljc and Martinc (2018) Kaja Dobrovoljc and Matej Martinc. 2018.  Er … well, it matters, right? On the role of data representations in spoken language dependency parsing.  In _Proceedings of the Second Workshop on Universal Dependencies (UDW 2018)_ , pages 37–46, Brussels, Belgium. Association for Computational Linguistics.
  * Dobrovoljc and Nivre (2016) Kaja Dobrovoljc and Joakim Nivre. 2016.  The Universal Dependencies treebank of spoken Slovenian.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 1566–1573, Portorož, Slovenia. European Language Resources Association (ELRA).
  * Dozat and Manning (2017) Timothy Dozat and Christopher D. Manning. 2017.  Deep Biaffine Attention for Neural Dependency Parsing.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Ferreira and Bailey (2004) Fernanda Ferreira and Karl G.D. Bailey. 2004.  Disfluencies and human language comprehension.  _Trends in Cognitive Sciences_ , 8(5):231–237.
  * Gerdes and Kahane (2009) Kim Gerdes and Sylvain Kahane. 2009.  Speaking in piles: Paradigmatic annotation of French spoken corpus.  In _Fifth Corpus Linguistics Conference_ , pages 1–15, Liverpool, United Kingdom.
  * Godfrey et al. (1992) John J Godfrey, Edward C Holliman, and Jane McDaniel. 1992.  Switchboard: Telephone speech corpus for research and development.  In _Acoustics, Speech, and Signal Processing, IEEE International Conference on_ , volume 1, pages 517–520. IEEE Computer Society.
  * van der Goot et al. (2021) Rob van der Goot, Ahmet Üstün, Alan Ramponi, Ibrahim Sharaf, and Barbara Plank. 2021.  Massive choice, ample tasks (MaChAmp): A toolkit for multi-task learning in NLP.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations_ , pages 176–197, Online. Association for Computational Linguistics.
  * Gretz et al. (2013) Shai Gretz, Alon Itai, Brian MacWhinney, Bracha Nir, and Shuly Wintner. 2013.  Parsing Hebrew CHILDES transcripts.  _Language Resources and Evaluation_ , 49(1):107–145.
  * He et al. (2018) Ruifang He, Yaru Wang, Dawei Song, Peng Zhang, Yuan Jia, and Aijun Li. 2018.  A dependency parser for spontaneous Chinese spoken language.  _ACM Transactions on Asian and Low-Resource Language Information Processing_ , 17(4):1–13.
  * Hughes and Allen (2006) Mary Hughes and Shanley Allen. 2006.  A discourse-pragmatic analysis of subject omission in child english.  In _Proceedings of the 30th Annual Boston University Conference on Language Development_ , volume 1, pages 293–304. Cascadilla Press Somerville, MA.
  * Kiperwasser and Goldberg (2016) Eliyahu Kiperwasser and Yoav Goldberg. 2016.  Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations.  _Transactions of the Association for Computational Linguistics_ , 4:313–327.
  * Kovriguina et al. (2018) Liubov Kovriguina, Ivan Shilin, Alina Putintseva, and Alexander Shipilo. 2018.  Multilevel annotation in the corpus for parsing russian spontaneous speech.  In _Speech and Computer_ , pages 311–320, Cham. Springer International Publishing.
  * Kuczaj II (1977) Stan A Kuczaj II. 1977.  The acquisition of regular and irregular past tense forms.  _Journal of verbal learning and verbal behavior_ , 16(5):589–600.
  * Lee et al. (2017) John Lee, Herman Leung, and Keying Li. 2017.  Towards Universal Dependencies for Learner Chinese.  In _Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017)_ , pages 67–71, Gothenburg, Sweden. Association for Computational Linguistics.
  * de Lhoneux et al. (2017) Miryam de Lhoneux, Sara Stymne, and Joakim Nivre. 2017.  Arc-hybrid non-projective dependency parsing with a static-dynamic oracle.  In _Proceedings of the The 15th International Conference on Parsing Technologies (IWPT)._ , Pisa, Italy.
  * Lieven et al. (2009) Elena Lieven, Dorothé Salomo, and Michael Tomasello. 2009.  Two-year-old children’s production of multiword utterances: A usage-based analysis.  _Cognitive Linguistics_ , 20(3):481–508.
  * Liu et al. (2018) Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A. Smith. 2018\.  Parsing tweets into Universal Dependencies.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 965–975, New Orleans, Louisiana. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized BERT pretraining approach.  _CoRR_ , abs/1907.11692.
  * Liu and Prud’hommeaux (2021) Zoey Liu and Emily Prud’hommeaux. 2021.  Dependency parsing evaluation for low-resource spontaneous speech.  In _Proceedings of the Second Workshop on Domain Adaptation for NLP_ , pages 156–165, Kyiv, Ukraine. Association for Computational Linguistics.
  * MacWhinney (2000) Brian MacWhinney. 2000.  _The CHILDES Project: Tools for analyzing talk. transcription format and programs_ , volume 1.  Psychology Press.
  * Mikulová et al. (2017) Marie Mikulová, Jiří Mírovský, Anja Nedoluzhko, Petr Pajas, Jan Štěpánek, and Jan Hajič. 2017.  Pdtsc 2.0 - spoken corpus with rich multi-layer structural annotation.  In _Text, Speech, and Dialogue_ , pages 129–137, Cham. Springer International Publishing.
  * Miller (2009a) Tim Miller. 2009a.  Improved syntactic models for parsing speech with repairs.  In _Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics_ , pages 656–664, Boulder, Colorado. Association for Computational Linguistics.
  * Miller (2009b) Tim Miller. 2009b.  Improved syntactic models for parsing speech with repairs.  In _Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics_ , pages 656–664, Boulder, Colorado. Association for Computational Linguistics.
  * Miller and Schuler (2008) Tim Miller and William Schuler. 2008.  A unified syntactic model for parsing fluent and disfluent speech.  In _Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers_ , pages 105–108, Columbus, Ohio. Association for Computational Linguistics.
  * Miyata et al. (2013) Susanne Miyata, Kenji Sagae, and Brian MacWhinney. 2013.  The syntax parser GRASP for CHILDES (in Japanese).  _Journal of Health and Medical Science_ , (3):45–62.
  * Nivre (2009) Joakim Nivre. 2009.  Non-projective dependency parsing in expected linear time.  In _Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP_ , pages 351–359, Suntec, Singapore. Association for Computational Linguistics.
  * Ohno et al. (2005) Tomohiro Ohno, Shigeki Matsubara, Nobuo Kawaguchi, and Yasuyoshi Inagaki. 2005.  Robust Dependency Parsing of Spontaneous Japanese Spoken Language.  _IEICE - Trans. Inf. Syst._ , E88-D(3):545–552.
  * Qi et al. (2020) Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020\.  Stanza: A Python Natural Language Processing Toolkit for Many Human Languages.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_ , pages 101–108, Online. Association for Computational Linguistics.
  * Rasooli and Tetreault (2013) Mohammad Sadegh Rasooli and Joel Tetreault. 2013.  Joint Parsing and Disfluency Detection in Linear Time.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 124–129, Seattle, Washington, USA. Association for Computational Linguistics.
  * Sagae et al. (2010) Kenji Sagae, Eric Davis, Alon Lavie, Brian Macwhinney, and Shuly Wintner. 2010.  Morphosyntactic annotation of CHILDES transcripts.  _Journal of Child Language_ , 37(3):705–729.
  * Sagae et al. (2001) Kenji Sagae, Alon Lavie, and Brian MacWhinney. 2001.  Parsing the CHILDES database: Methodology and lessons learned.  In _Proceedings of the Seventh International Workshop on Parsing Technologies_ , pages 166–176, Beijing, China.
  * Sagae et al. (2004) Kenji Sagae, Brian MacWhinney, and Alon Lavie. 2004.  Adding syntactic annotations to transcripts of parent-child dialogs.  In _Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)_ , Lisbon, Portugal. European Language Resources Association (ELRA).
  * Sanchez et al. (2019) Alessandro Sanchez, Stephan C Meylan, Mika Braginsky, Kyle E MacDonald, Daniel Yurovsky, and Michael C Frank. 2019.  childes-db: A flexible and reproducible interface to the child language data exchange system.  _Behavior research methods_ , 51(4):1928–1941.
  * Santorini (1990) Beatrice Santorini. 1990.  Part-of-speech tagging guidelines for the penn treebank project.
  * Schneider and Zeldes (2021) Nathan Schneider and Amir Zeldes. 2021.  Mischievous nominal constructions in Universal Dependencies.  In _Proceedings of the Fifth Workshop on Universal Dependencies (UDW, SyntaxFest 2021)_ , pages 160–172, Sofia, Bulgaria. Association for Computational Linguistics.
  * Silveira et al. (2014) Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Christopher D. Manning. 2014.  A gold standard dependency corpus for English.  In _Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014)_.
  * Smith et al. (2018) Aaron Smith, Bernd Bohnet, Miryam de Lhoneux, Joakim Nivre, Yan Shao, and Sara Stymne. 2018.  82 treebanks, 34 models: Universal Dependency parsing with multi-treebank models.  In _Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies_ , pages 113–123, Brussels, Belgium. Association for Computational Linguistics.
  * Szubert et al. (2021) Ida Szubert, Omri Abend, Nathan Schneider, Samuel Gibbon, Sharon Goldwater, and Mark Steedman. 2021.  Cross-linguistically consistent semantic and syntactic annotation of child-directed speech.  _arXiv preprint arXiv:2109.10952_.
  * Weist and Zevenbergen (2008) Richard M Weist and Andrea A Zevenbergen. 2008.  Autobiographical memory and past time reference.  _Language Learning and Development_ , 4(4):291–308.
  * Yannakoudakis et al. (2011) Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011.  A new dataset and method for automatically grading esol texts.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1_ , pages 180–189. Association for Computational Linguistics.
  * Yoshikawa et al. (2016) Masashi Yoshikawa, Hiroyuki Shindo, and Yuji Matsumoto. 2016.  Joint transition-based dependency parsing and disfluency detection for automatic speech recognition texts.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1036–1041, Austin, Texas. Association for Computational Linguistics.
  * Zeman et al. (2022) Daniel Zeman, Joakim Nivre, Mitchell Abrams, Elia Ackermann, Noëmi Aepli, Hamid Aghaei, Željko Agić, Amir Ahmadi, Lars Ahrenberg, Chika Kennedy Ajede, Gabrielė Aleksandravičiūtė, Ika Alfina, Avner Algom, Erik Andersen, Lene Antonsen, Katya Aplonova, Angelina Aquino, Carolina Aragon, Glyd Aranes, Maria Jesus Aranzabe, Bilge Nas Arıcan, H͡órunn Arnardóttir, Gashaw Arutie, Jessica Naraiswari Arwidarasti, Masayuki Asahara, Deniz Baran Aslan, Cengiz Asmazoğlu, Luma Ateyah, Furkan Atmaca, Mohammed Attia, Aitziber Atutxa, Liesbeth Augustinus, Elena Badmaeva, Keerthana Balasubramani, Miguel Ballesteros, Esha Banerjee, Sebastian Bank, Verginica Barbu Mititelu, Starkaður Barkarson, Rodolfo Basile, Victoria Basmov, Colin Batchelor, John Bauer, Seyyit Talha Bedir, Kepa Bengoetxea, Yifat Ben Moshe, Gözde Berk, Yevgeni Berzak, Irshad Ahmad Bhat, Riyaz Ahmad Bhat, Erica Biagetti, Eckhard Bick, Agnė Bielinskienė, Kristín Bjarnadóttir, Rogier Blokland, Victoria Bobicev, Loïc Boizou, Emanuel Borges Völker, Carl Börstell, Cristina Bosco, Gosse Bouma, Sam Bowman, Adriane Boyd, Anouck Braggaar, Kristina Brokaitė, Aljoscha Burchardt, Marie Candito, Bernard Caron, Gauthier Caron, Lauren Cassidy, Tatiana Cavalcanti, Gülşen Cebiroğlu Eryiğit, Flavio Massimiliano Cecchini, Giuseppe G. A. Celano, Slavomír Čéplö, Neslihan Cesur, Savas Cetin, Özlem Çetinoğlu, Fabricio Chalub, Shweta Chauhan, Ethan Chi, Taishi Chika, Yongseok Cho, Jinho Choi, Jayeol Chun, Juyeon Chung, Alessandra T. Cignarella, Silvie Cinková, Aurélie Collomb, Çağrı Çöltekin, Miriam Connor, Daniela Corbetta, Marine Courtin, Mihaela Cristescu, Philemon Daniel, Elizabeth Davidson, Mathieu Dehouck, Martina de Laurentiis, Marie-Catherine de Marneffe, Valeria de Paiva, Mehmet Oguz Derin, Elvis de Souza, Arantza Diaz de Ilarraza, Carly Dickerson, Arawinda Dinakaramani, Elisa Di Nuovo, Bamba Dione, Peter Dirix, Kaja Dobrovoljc, Timothy Dozat, Kira Droganova, Puneet Dwivedi, Hanne Eckhoff, Sandra Eiche, Marhaba Eli, Ali Elkahky, Binyam Ephrem, Olga Erina, Tomaž Erjavec, Aline Etienne, Wograine Evelyn, Sidney Facundes, Richárd Farkas, Federica Favero, Jannatul Ferdaousi, Marília Fernanda, Hector Fernandez Alcalde, Jennifer Foster, Cláudia Freitas, Kazunori Fujita, Katarína Gajdošová, Daniel Galbraith, Federica Gamba, Marcos Garcia, Moa Gärdenfors, Sebastian Garza, Fabrício Ferraz Gerardi, Kim Gerdes, Filip Ginter, Gustavo Godoy, Iakes Goenaga, Koldo Gojenola, Memduh Gökırmak, Yoav Goldberg, Xavier Gómez Guinovart, Berta González Saavedra, Bernadeta Griciūtė, Matias Grioni, Loïc Grobol, Normunds Grūzītis, Bruno Guillaume, Céline Guillot-Barbance, Tunga Güngör, Nizar Habash, Hinrik Hafsteinsson, Jan Hajič, Jan Hajič jr., Mika Hämäläinen, Linh Hà Mỹ, Na-Rae Han, Muhammad Yudistira Hanifmuti, Takahiro Harada, Sam Hardwick, Kim Harris, Dag Haug, Johannes Heinecke, Oliver Hellwig, Felix Hennig, Barbora Hladká, Jaroslava Hlaváčová, Florinel Hociung, Petter Hohle, Jena Hwang, Takumi Ikeda, Anton Karl Ingason, Radu Ion, Elena Irimia, Ọlájídé Ishola, Kaoru Ito, Siratun Jannat, Tomáš Jelínek, Apoorva Jha, Anders Johannsen, Hildur Jónsdóttir, Fredrik Jørgensen, Markus Juutinen, Sarveswaran K, Hüner Kaşıkara, Andre Kaasen, Nadezhda Kabaeva, Sylvain Kahane, Hiroshi Kanayama, Jenna Kanerva, Neslihan Kara, Ritván Karahóǧa, Boris Katz, Tolga Kayadelen, Jessica Kenney, Václava Kettnerová, Jesse Kirchner, Elena Klementieva, Elena Klyachko, Arne Köhn, Abdullatif Köksal, Kamil Kopacewicz, Timo Korkiakangas, Mehmet Köse, Natalia Kotsyba, Jolanta Kovalevskaitė, Simon Krek, Parameswari Krishnamurthy, Sandra Kübler, Oğuzhan Kuyrukçu, Aslı Kuzgun, Sookyoung Kwak, Veronika Laippala, Lucia Lam, Lorenzo Lambertino, Tatiana Lando, Septina Dian Larasati, Alexei Lavrentiev, John Lee, Phuong Lê Hồng, Alessandro Lenci, Saran Lertpradit, Herman Leung, Maria Levina, Cheuk Ying Li, Josie Li, Keying Li, Yuan Li, KyungTae Lim, Bruna Lima Padovani, Krister Lindén, Nikola Ljubešić, Olga Loginova, Stefano Lusito, Andry Luthfi, Mikko Luukko, Olga Lyashevskaya, Teresa Lynn, Vivien Macketanz, Menel Mahamdi, Jean Maillard, Aibek Makazhanov, Michael Mandl, Christopher Manning, Ruli Manurung, Büşra Marşan, Cătălina Mărănduc, David Mareček, Katrin Marheinecke, Stella Markantonatou, Héctor Martínez Alonso, Lorena Martín Rodríguez, André Martins, Jan Mašek, Hiroshi Matsuda, Yuji Matsumoto, Alessandro Mazzei, Ryan McDonald, Sarah McGuinness, Gustavo Mendonça, Tatiana Merzhevich, Niko Miekka, Karina Mischenkova, Margarita Misirpashayeva, Anna Missilä, Cătălin Mititelu, Maria Mitrofan, Yusuke Miyao, AmirHossein Mojiri Foroushani, Judit Molnár, Amirsaeid Moloodi, Simonetta Montemagni, Amir More, Laura Moreno Romero, Giovanni Moretti, Keiko Sophie Mori, Shinsuke Mori, Tomohiko Morioka, Shigeki Moro, Bjartur Mortensen, Bohdan Moskalevskyi, Kadri Muischnek, Robert Munro, Yugo Murawaki, Kaili Müürisep, Pinkey Nainwani, Mariam Nakhlé, Juan Ignacio Navarro Horñiacek, Anna Nedoluzhko, Gunta Nešpore-Bērzkalne, Manuela Nevaci, Luong Nguyễn Thị, Huyền Nguyễn Thị Minh, Yoshihiro Nikaido, Vitaly Nikolaev, Rattima Nitisaroj, Alireza Nourian, Hanna Nurmi, Stina Ojala, Atul Kr. Ojha, Adédayọ Olúòkun, Mai Omura, Emeka Onwuegbuzia, Noam Ordan, Petya Osenova, Robert Östling, Lilja Øvrelid, Şaziye Betül Özateş, Merve Özçelik, Arzucan Özgür, Balkız Öztürk Başaran, Teresa Paccosi, Alessio Palmero Aprosio, Hyunji Hayley Park, Niko Partanen, Elena Pascual, Marco Passarotti, Agnieszka Patejuk, Guilherme Paulino-Passos, Giulia Pedonese, Angelika Peljak-Łapińska, Siyao Peng, Cenel-Augusto Perez, Natalia Perkova, Guy Perrier, Slav Petrov, Daria Petrova, Andrea Peverelli, Jason Phelan, Jussi Piitulainen, Tommi A Pirinen, Emily Pitler, Barbara Plank, Thierry Poibeau, Larisa Ponomareva, Martin Popel, Lauma Pretkalniņa, Sophie Prévost, Prokopis Prokopidis, Adam Przepiórkowski, Tiina Puolakainen, Sampo Pyysalo, Peng Qi, Andriela Rääbis, Alexandre Rademaker, Mizanur Rahoman, Taraka Rama, Loganathan Ramasamy, Carlos Ramisch, Fam Rashel, Mohammad Sadegh Rasooli, Vinit Ravishankar, Livy Real, Petru Rebeja, Siva Reddy, Mathilde Regnault, Georg Rehm, Ivan Riabov, Michael Rießler, Erika Rimkutė, Larissa Rinaldi, Laura Rituma, Putri Rizqiyah, Luisa Rocha, Eiríkur Rögnvaldsson, Mykhailo Romanenko, Rudolf Rosa, Valentin Rosca, Davide Rovati, Ben Rozonoyer, Olga Rudina, Jack Rueter, Kristján Rúnarsson, Shoval Sadde, Pegah Safari, Benoît Sagot, Aleksi Sahala, Shadi Saleh, Alessio Salomoni, Tanja Samardžić, Stephanie Samson, Manuela Sanguinetti, Ezgi Sanıyar, Dage Särg, Baiba Saulīte, Yanin Sawanakunanon, Shefali Saxena, Kevin Scannell, Salvatore Scarlata, Nathan Schneider, Sebastian Schuster, Lane Schwartz, Djamé Seddah, Wolfgang Seeker, Mojgan Seraji, Syeda Shahzadi, Mo Shen, Atsuko Shimada, Hiroyuki Shirasu, Yana Shishkina, Muh Shohibussirri, Dmitry Sichinava, Janine Siewert, Einar Freyr Sigurðsson, Aline Silveira, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simkó, Mária Šimková, Kiril Simov, Maria Skachedubova, Aaron Smith, Isabela Soares-Bastos, Shafi Sourov, Carolyn Spadine, Rachele Sprugnoli, Vivian Stamou, Steinh͡ór Steingrímsson, Antonio Stella, Milan Straka, Emmett Strickland, Jana Strnadová, Alane Suhr, Yogi Lesmana Sulestio, Umut Sulubacak, Shingo Suzuki, Daniel Swanson, Zsolt Szántó, Chihiro Taguchi, Dima Taji, Yuta Takahashi, Fabio Tamburini, Mary Ann C. Tan, Takaaki Tanaka, Dipta Tanaya, Mirko Tavoni, Samson Tella, Isabelle Tellier, Marinella Testori, Guillaume Thomas, Sara Tonelli, Liisi Torga, Marsida Toska, Trond Trosterud, Anna Trukhina, Reut Tsarfaty, Utku Türk, Francis Tyers, Sumire Uematsu, Roman Untilov, Zdeňka Urešová, Larraitz Uria, Hans Uszkoreit, Andrius Utka, Elena Vagnoni, Sowmya Vajjala, Rob van der Goot, Martine Vanhove, Daniel van Niekerk, Gertjan van Noord, Viktor Varga, Uliana Vedenina, Eric Villemonte de la Clergerie, Veronika Vincze, Natalia Vlasova, Aya Wakasa, Joel C. Wallenberg, Lars Wallin, Abigail Walsh, Jing Xian Wang, Jonathan North Washington, Maximilan Wendt, Paul Widmer, Shira Wigderson, Sri Hartati Wijono, Seyi Williams, Mats Wirén, Christian Wittern, Tsegay Woldemariam, Tak-sum Wong, Alina Wróblewska, Mary Yako, Kayo Yamashita, Naoki Yamazaki, Chunxiao Yan, Koichi Yasuoka, Marat M. Yavrumyan, Arife Betül Yenice, Olcay Taner Yıldız, Zhuoran Yu, Arlisa Yuliawati, Zdeněk Žabokrtský, Shorouq Zahra, Amir Zeldes, He Zhou, Hanzhi Zhu, Anna Zhuravleva, and Rayan Ziane. 2022.  Universal dependencies 2.10.  LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University.


for $i=p+1,\ldots,2p-1$, by $(\mathrm{e}^{-1}\nu)^{\nu}/(y_{i}\Gamma(\nu))$
(see Lemma 1). Recall that these are non-outliers such that
$|\mathbf{x}_{i}^{T}\boldsymbol{\beta}|<\log(\omega)/\gamma$. In Step $c$, we
simplify each $\log(b_{i}\omega)^{\lambda_{\text{r}}}$ and
$\log(b_{i}\omega)^{\lambda_{\text{l}}}$ by multiplying each of these terms by
the product of densities of
$\lceil\lambda_{\text{l}}/\lambda_{\text{r}}\rceil$ extreme non-outliers and
by bounding the resulting product by
$\left(4\gamma\right)^{\lambda_{\text{r}}}$ or
$\left(4\gamma\right)^{\lambda_{\text{l}}}$, as we have explained earlier. In
Step $d$, we bound the rest of the non-outlier terms
$f_{\nu,c}(y_{i}/\mu_{i})/\mu_{i}$ by
$(\mathrm{e}^{-1}\nu)^{\nu}/(y_{i}\Gamma(\nu))$. If we consider all the $k$
non-outliers, $p$ non-outliers were used for the change of variables and to
integrate over $\boldsymbol{\beta}$ at the beginning, we bounded $p-1$ terms
of non-extreme non-outliers, and
$\lceil\lambda_{\text{l}}/\lambda_{\text{r}}\rceil(\ell+s)$ were used to
offset the outliers. After Steps (a)-(c), there are thus possibly still
$k-p-(p-1)-\lceil\lambda_{\text{l}}/\lambda_{\text{r}}\rceil(\ell+s)=k-2p-\lceil\lambda_{\text{l}}/\lambda_{\text{r}}\rceil(\ell+s)+1$
non-outliers left, that need to be considered, which is what was done in Step
d. The condition of this theorem
$k\geq\lceil\lambda_{\text{l}}/\lambda_{\text{r}}\rceil(\ell+s)+2p-1$ is to
make sure that we have enough non-outlying points to bound the whole product.
The proof is simpler and still valid if there is no outlier left after Steps
(a)-(c); Step (d) is simply skipped. In Step $e$, every $y_{i}$ in the
expression is a non-outlying observation, and is thus equal to $a_{i}$.
Finally, in Step $f$, the whole expression is finite given that all terms are
constants.

Therefore, $h(\omega)=A\times B\times C$ is bounded. This completes the proof
of Result (a).

We now turn to the proof of Result (b). We have that

$\displaystyle\pi(\boldsymbol{\beta}\mid\mathbf{y})=\pi(\boldsymbol{\beta}\mid\mathbf{y}_{k})\,\frac{m(\mathbf{y}_{k})\prod_{i=1}^{n}f_{\nu,c}(y_{i})^{s_{i}+\ell_{i}}}{m(\mathbf{y})}\prod_{i=1}^{n}\left[\frac{f_{\nu,c}(y_{i}/\mu_{i})/\mu_{i}}{f_{\nu,c}(y_{i})}\right]^{s_{i}+\ell_{i}},$

and

$\displaystyle\frac{m(\mathbf{y}_{k})\prod_{i=1}^{n}f_{\nu,c}(y_{i})^{s_{i}+\ell_{i}}}{m(\mathbf{y})}\prod_{i=1}^{n}\left[\frac{f_{\nu,c}(y_{i}/\mu_{i})/\mu_{i}}{f_{\nu,c}(y_{i})}\right]^{s_{i}+\ell_{i}}\rightarrow
1,$

as $\omega\rightarrow\infty$, for any
$\boldsymbol{\beta}\in\operatorname{\mathbb{R}}^{p}$, using Result (a) and
Proposition 3. We also showed that $\pi(\beta\mid\mathbf{y}_{k})$ is proper.
This concludes the proof of Result (b).

We finish with the proof of Result (c). This result is a direct consequence of
Result (b) using Scheffé’s theorem (Scheffé, 1947). ∎

##  Appendix B Supplementary material for Section 4.2

With gamma GLM, the posterior density $\pi(\,\cdot\,,\cdot\mid\mathbf{y})$ is
such that

$\pi(\boldsymbol{\beta},\nu\mid\mathbf{y})\propto\pi(\boldsymbol{\beta},\nu)\prod_{i=1}^{n}\frac{1}{\mu_{i}}f_{\nu}\left(\frac{y_{i}}{\mu_{i}}\right),$

with

$f_{\nu}(z):=\frac{\exp(-\nu z)\,z^{\nu-1}\nu^{\nu}}{\Gamma(\nu)},\quad z>0.$

The posterior estimates are computed using Hamiltonian Monte Carlo (Duane et
al., 1987). We apply a change of variables $\eta:=\log\nu$ so that the
resulting posterior distribution has $\operatorname{\mathbb{R}}^{p+1}$ for
support. The resulting posterior density is such that

$\pi(\boldsymbol{\beta},\eta\mid\mathbf{y})\propto\mathrm{e}^{\eta}\,\pi(\mathrm{e}^{\eta})\prod_{i=1}^{n}\frac{1}{\mu_{i}}f_{\mathrm{e}^{\eta}}\left(\frac{y_{i}}{\mu_{i}}\right);$

recall the prior on the parameters. The log density is such that (if we omit
the normalization constant)

$\log\pi(\boldsymbol{\beta},\eta\mid\mathbf{y})=\eta+\log\pi(\mathrm{e}^{\eta})+\sum_{i=1}^{n}\log
f_{\mathrm{e}^{\eta}}\left(\frac{y_{i}}{\mu_{i}}\right)-\log\mu_{i}.$

We have that

$\frac{\partial}{\partial\eta}\log\pi(\mathrm{e}^{\eta})=(\alpha-1)-\frac{\mathrm{e}^{\eta}}{\theta},$

where $\alpha>0$ and $\theta>0$ are the shape and scale parameters of the
prior distribution. Also,

$\displaystyle\frac{\partial}{\partial\boldsymbol{\beta}}\log
f_{\mathrm{e}^{\eta}}\left(\frac{y_{i}}{\mu_{i}}\right)-\log\mu_{i}=\mathrm{e}^{\eta}\left(\frac{y_{i}}{\mu_{i}}-1\right)\,\mathbf{x}_{i},$
$\displaystyle\frac{\partial}{\partial\eta}\log
f_{\mathrm{e}^{\eta}}\left(\frac{y_{i}}{\mu_{i}}\right)-\log\mu_{i}=\mathrm{e}^{\eta}\left(-\frac{y_{i}}{\mu_{i}}+\log\left(\frac{y_{i}}{\mu_{i}}\right)+\eta+1-\frac{\Gamma^{\prime}(\mathrm{e}^{\eta})}{\Gamma(\mathrm{e}^{\eta})}\right),$

where $\Gamma^{\prime}$ is the derivative of $\Gamma$.

With the proposed GLM, the posterior density
$\pi(\,\cdot\,,\cdot\mid\mathbf{y})$ (with the same prior as before) is such
that

$\pi(\boldsymbol{\beta},\nu\mid\mathbf{y})\propto\pi(\nu)\prod_{i=1}^{n}\frac{1}{\mu_{i}}f_{\nu,c}\left(\frac{y_{i}}{\mu_{i}}\right).$

We also use Hamiltonian Monte Carlo to compute the posterior estimates.

The posterior density resulting from the change of variables $\eta:=\log\nu$
is such that

$\pi(\boldsymbol{\beta},\eta\mid\mathbf{y})\propto\mathrm{e}^{\eta}\,\pi(\mathrm{e}^{\eta})\prod_{i=1}^{n}\frac{1}{\mu_{i}}f_{\mathrm{e}^{\eta},c}\left(\frac{y_{i}}{\mu_{i}}\right).$

The log density is such that (if we omit the normalization constant)

$\log\pi(\boldsymbol{\beta},\nu\mid\mathbf{y})=\eta+\log\pi(\mathrm{e}^{\eta})+\sum_{i=1}^{n}\log
f_{\mathrm{e}^{\eta},c}\left(\frac{y_{i}}{\mu_{i}}\right)-\log\mu_{i}.$

We have that

$\frac{\partial}{\partial\boldsymbol{\beta}}\log
f_{\mathrm{e}^{\eta},c}\left(\frac{y_{i}}{\mu_{i}}\right)-\log\mu_{i}=\begin{cases}\mathrm{e}^{\eta}\left(\frac{y_{i}}{\mu_{i}}-1\right)\,\mathbf{x}_{i}\quad\text{if}\quad
z_{\text{l}}\leq\frac{y_{i}}{\mu_{i}}\leq
z_{\text{r}},\cr\frac{\lambda_{\text{r}}}{\log(y_{i}/\mu_{i})}\,\mathbf{x}_{i}\quad\text{if}\quad\frac{y_{i}}{\mu_{i}}>z_{\text{r}},\cr\frac{\lambda_{\text{l}}}{\log(y_{i}/\mu_{i})}\,\mathbf{x}_{i}\quad\text{if}\quad\frac{y_{i}}{\mu_{i}}<z_{\text{l}},\end{cases}$

and

$\frac{\partial}{\partial\eta}\log
f_{\mathrm{e}^{\eta},c}\left(\frac{y_{i}}{\mu_{i}}\right)-\log\mu_{i}=\begin{cases}\mathrm{e}^{\eta}\left(-\frac{y_{i}}{\mu_{i}}+\log\left(\frac{y_{i}}{\mu_{i}}\right)+\eta+1-\frac{\Gamma^{\prime}(\mathrm{e}^{\eta})}{\Gamma(\mathrm{e}^{\eta})}\right)\quad\text{if}\quad
z_{\text{l}}\leq\frac{y_{i}}{\mu_{i}}\leq
z_{\text{r}},\cr-\frac{c}{2}\,\mathrm{e}^{\eta/2}\left(1+\frac{1}{1+c\,\mathrm{e}^{-\eta/2}}\right)+\mathrm{e}^{\eta}\left(\log(1+c\,\mathrm{e}^{-\eta/2})+\eta-\frac{\Gamma^{\prime}(\mathrm{e}^{\eta})}{\Gamma(\mathrm{e}^{\eta})}\right)\cr\quad+\lambda_{\text{r}}^{\prime}\left(\log(\log(z_{\text{r}}))-\log\left(\log\left(\frac{y_{i}}{\mu_{i}}\right)\right)\right)-\frac{\lambda_{\text{r}}}{\log(1+c\,\mathrm{e}^{-\eta/2})}\frac{c\,\mathrm{e}^{-\eta/2}}{2(1+c\,\mathrm{e}^{-\eta/2})}\quad\text{if}\quad\frac{y_{i}}{\mu_{i}}>z_{\text{r}},\cr\frac{c}{2}\,\mathrm{e}^{\eta/2}\left(1+\frac{1}{1-c\,\mathrm{e}^{-\eta/2}}\right)+\mathrm{e}^{\eta}\left(\log(1-c\,\mathrm{e}^{-\eta/2})+\eta-\frac{\Gamma^{\prime}(\mathrm{e}^{\eta})}{\Gamma(\mathrm{e}^{\eta})}\right)\cr\quad+\lambda_{\text{l}}^{\prime}\left(\log(-\log(z_{\text{l}}))-\log\left(-\log\left(\frac{y_{i}}{\mu_{i}}\right)\right)\right)+\frac{\lambda_{\text{l}}}{\log(1-c\,\mathrm{e}^{-\eta/2})}\frac{c\,\mathrm{e}^{-\eta/2}}{2(1-c\,\mathrm{e}^{-\eta/2})}\quad\text{if}\quad
y_{i}/\mu_{i}<z_{\text{l}},\end{cases}$

where
$\lambda_{\text{r}}^{\prime}:=\frac{\partial}{\partial\eta}\lambda_{\text{r}}$
and
$\lambda_{\text{l}}^{\prime}:=\frac{\partial}{\partial\eta}\lambda_{\text{l}}$
which are evaluated numerically.