# A study of vowel nasalization using instantaneous spectra

RaviShankar Prasad†111email<EMAIL_ADDRESS>& B. Yegnanarayana  †Idiap
Research Institute, Martigny, CH-1920
‡International Institute of Information Technology, Hyderabad, India - 500032

###### Abstract

Nasalization of vowels is a phenomenon where oral and nasal tracts participate
simultaneously for the production of speech. Acoustic coupling of oral and
nasal tracts results in a complex production system, which is subjected to a
continuous changes owing to glottal activity. Identification of the duration
of nasalization in vowels, and the extent of coupling of oral and nasal
tracts, is a challenging task. The present research focuses on the changes in
instantaneous vocal tract system response to study the effects of
co–articulatory load of nasals on vowels. The dominant resonance frequency
(DRF) contour derived from the spectra illustrates the contribution of oral
and nasal resonances during nasalization. The extent of coupling determines
the dominance of these resonances during glottal open and closed phases. A
higher extent leads to more decay of energy for the oral resonances, and hence
the vowel spectra exhibits a dominant nasal resonance. A comparison of the
proposed parameters is made with the previously suggested measures. Several
examples of segments of vowels in the context of nasal consonants for English
language for both male and female speakers of English are used to illustrate
different aspects of the proposed analysis method.

###### keywords:

nasalized vowels; oral–nasal coupling; zero time windowing; HNGD spectra;
dominant resonance frequency

††journal: Computer Speech and Language

##  1 Introduction

Nasalization of vowels occurs when the air flow takes place through both oral
and nasal tracts during production of vowels. Oral vowels are produced by
exciting an open oral tract with pseudo–periodic vibration of the vocal folds
at the glottis. Nasal consonants are produced with lowering the velum and
closing the oral tract, resulting in airflow only through the nasal tract.
Coupling of an open oral and nasal tracts results in presence of a nasal
signature in the vowel spectrum. Extent of this coupling dictates the volume
of airflow through each of these tracts. The nasal tract consists of multiple
sinus cavities, and acts as a branched cavity to the open oral tract during
nasalization. This overall system is further subjected to glottal activity,
resulting in a complex production system changing continuously with time. A
dynamic and involuntary nature of glottal and velar movement makes the
identification of presence of nasalization in vowels a challenging task.
Understanding the phenomena of vowel nasalization in speech is important for
the improvement of several speech applications, such as automatic speech
recognition, speech pathology, language identification, and interactive voice
response systems.

Study of the phenomenon of vowel nasalization has been motivated to highlight
the contrast between oral and nasal vowels. Previous studies explored the
spectro–temporal characteristics of vowel segments derived across a variety of
acoustic and linguistic contexts. Early developments towards the task have
highlighted the presence of a low frequency spectral peak (within $250$–$300$
Hz range) in vowel spectra, as a characteristic signature of coupling of nasal
and oral tracts [1, 2, 3, 4]. These studies also emphasized on the role of
spectral zero in $700$–$1800$ Hz range, and widening of the first formant
($F$_1_) bandwidth, as supportive evidences of nasalization in vowels.
Presence of a pole–zero pair around first formant ($F$_1_) of the vowel, and a
spectral peak around $1$ kHz, is also noted for several cases of nasalized
vowel segments [5]. The effect of introduction of a synthetic pole–zero pair
in low frequency range, and increasing the amplitude of the first harmonic, is
studied for its contribution towards perception of nasality in vowels [1, 6].
Further studies have discussed the importance of spectral peak in $250$–$450$
Hz range as an important cue to determine the presence of nasalization in
vowel spectra [7, 8, 9, 10].

These studies have propelled the task of automatic identification of
nasalization in speech, based on parameters derived from vowel spectra.
Spectral correlates $A$_1_ –$P$_1_ and $A$_1_ –$P$_0_ have been proposed to
highlight the contrast within oral and nasalized vowels [11, 12]. $A$_1_ is
defined as amplitude of the harmonic peak closest to an estimated location of
$F$_1_. $P$_1_ is amplitude of the nasal peak in vicinity of $F$_1_ , whereas
$P$_0_ is amplitude of first resonance peak at low frequencies. A large shape
of the nasal tract contributes towards broadening of $F$_1_ , consequently
lowering its amplitude in nasalized vowels. An average value of both these
correlates were found to be lower for nasalized vowels than for oral vowel
segments. The study presents a comparison between nasalized vowels for
anticipatory (occurring after) and carryover (occurring before) nasal
contexts, for these correlates. An _`orality threshold'_ measure is proposed
based on the extent of lowering of velum, to distinguish nasalized vowels from
oral vowels [5]. However, a high degree of variability in the nasalization
phenomena across utterances, speakers and languages mitigates weakens the
reliability on such a measure [13]. A set of nine acoustic features have been
proposed to detect the presence of nasalization in vowel segments [14]. These
features capture the low frequency spectral behavior of nasalized vowels, such
as change in energy and bandwidth of $F$_1_ , and other spectral peaks, and
changes in energy profile of different bands in low frequency range. All these
parameters are derived on the short–time spectral representation of vowel
spectrum obtained using short time Fourier transform (STFT). Dominant peaks in
group delay (GD) spectrum have been utilized to detect hypernasality in
pathological speech [15]. The GD spectrum is derived as derivative of the
phase component of STFT with respect to frequency. Advantages of the GD
spectrum is that it provides sharper spectral peaks compared to the STFT based
spectrum [16].

Importance of the low frequency resonance occurring below or in the range of
$F$_1_ , for identification of nasalization in vowels, has consistently been
highlighted by several studies. The present study utilizes this knowledge to
derive correlates from the instantaneous spectra to address the identification
of presence of nasalization, along with its duration and extent, in vowel
segments. The spectral representation obtained from the zero time windowing
(ZTW) method gives a good resolution in temporal and spectral domains [17].
The method, therefore, proves helpful to study the dynamic characteristics of
the production system during coupling of the oral and nasal tracts, subjected
to the glottal activity. The paper explores the effects of contextual load of
nasals on vowels, for different CV/VC pairs in the English language. The paper
is organized as follows: Section 2 presents the zero time windowing method,
and the motivation to the study. Section 3 discusses the analysis of oral and
nasalized vowels using dominant resonance frequencies (DRFs) derived from the
instantaneous spectra. Section 4 illustrates the behavior of DRF contour for
different extent of coupling of oral and nasal tracts. Section 5 discusses the
observations based on proposed hypothesis for the behavior of DRF contour for
oral and nasalized vowel segments for several CV/VC pairs. Section 6 presents
a summary to the paper.

##  2 Background and motivation

This section discusses the zero time windowing method to obtain the spectral
characteristics at good temporal resolution. Distinction in the dominant
spectral behavior of oral vowels and nasal consonants provide the necessary
motivation to explore vowel nasalization phenomena using ZTW.

###  2.1 Zero time windowing method

The zero time windowing (ZTW) method utilizes a heavily decaying window,
giving more weightage to samples near the point of application of the window,
called the zero time [17]. This windowing is motivated by the zero frequency
filter (ZFF) which is a sharp resonator centered around $0$ Hz [18]. The time
domain analog of this operation is a heavily decaying window function given
by,

$w_{1}[n]=\begin{cases}0,\quad\emph{n}=0,\\\ 1/4sin^{2}(\pi
n/2N),\quad\emph{n}=1,2,\ldots,\emph{N}-1,\end{cases}$ (1)

where $N$ is the length of the window in samples corresponding to a duration
of $l$ ms. The windowed signal is given as $x[n]=s[n]w[n]$, where $s[n]$ is
the speech signal, and the window function is given as
$w[n]=w_{1}^{2}[n]w_{2}[n]$. $w_{2}[n]$ is another window which helps to
reduce the ripple effect due to truncation, and is given by

$w_{2}[n]=4cos^{2}(\pi n/2N),\quad n=0,1,\ldots,N-1,$ (2)

Application of the window function $w_{1}[n]$ can be interpreted as an
integration operation performed twice in the frequency domain [17]. Spectral
characteristics of the windowed segment $x[n]$ are thus obtained by successive
differentiation. The spectrum is represented using the Hilbert envelope of the
differenced numerator group delay (HNGD) function. The numerator of GD (NGD)
function is given by,

$\tau(\omega)=X_{R}(\omega)Y_{R}(\omega)+X_{I}(\omega)Y_{I}(\omega).$ (3)

where $X(\omega)$=$X_{R}(\omega)$+$jX_{I}(\omega)$ is the discrete–time
Fourier transform (DTFT) of $x[n]$, and
$Y(\omega)$=$Y_{R}(\omega)$+$jY_{I}(\omega)$ is the DTFT of $nx[n]$. The
Hilbert envelope of the twice differenced NGD (HNGD) shows the formant peaks
with a good resolution [19]. A window with duration $l\leq$ average pitch
period gives the changes in the acoustic system response within a glottal
cycle [20]. The window $w[n]$ is shifted by one sample, to obtain the spectrum
at every sampling instant.

Figure 1 illustrates the contrast in spectral estimates obtained using HNGD
and STFT methods. Figure 1(b) shows the HNGD spectrogram for a segment of
vowel /e/ uttered by a male speaker given in Fig. 1(a). The HNGD spectrogram
is obtained across an analysis segment duration $l=4$ ms, shifted at every
sample. Changes in the system response appear in an instantaneous manner in
the HNGD spectrum. Figure 1(c) shows the STFT spectrogram of the segment. The
STFT is computed at every sampling instant using $l=4$ ms _Hann_ window. The
averaging effect on the spectral details can be seen in the STFT spectrogram.
Movement of the spectral resonances can be seen better in the HNGD
spectrogram.

Figure 1: Estimated spectrograms for a vowel segment using ZTW and STFT
methods. (a) Segment of speech signal. (b) HNGD spectrogram using ZTW analysis
method with $l=4$ ms. (c) STFT spectrogram using a _Hann_ window of $4$ ms.

###  2.2 Dynamics of coupling of oral and nasal tracts

The phenomenon of nasalization is categorized into three major types [21]:

  * 1.

Co–articulatory nasalization: The cases where nasal consonants appear either
in the pre–vocalic or post–vocalic or both contexts with a vowel. Production
of a nasal requires complete closure of the oral tract. Delay in the closure
of the oral tract, while opening the velopharyngeal section, leads to
nasalization of the vowel preceding a nasal consonant. On the other hand,
opening of the oral tract before closure of the velopharyngeal section leads
to nasalization of the vowel following a nasal consonant.

  * 2.

Phonemic nasalization: The cases where vowels are distinctively nasalized,
independent of any contextual proximity to nasals. Oral and nasalized vowels
form a minimal pair for such cases, which are linguistically different from
each other and therefore convey different meaning.

  * 3.

Functional nasalization: The cases where characteristics of nasalization are
introduced in vowels due to dysfunctional velar mechanism.

Co–articulatory nasalization is the most dynamic of the three types of
nasalization discussed above, in terms of production system complexity,
duration, dependence on factors related to contextual load, muscular inertia,
articulatory constraints, speaking rate etc. Vowels appearing in context of
nasal consonants are expected to be nasalized, at least for part of their
duration. This assumption may hold true across different utterances and
speakers in different languages. Furthermore, the degree of nasalization may
also vary for different utterances of same VC/CV pair.

Recent studies show that acoustic correlates $A$_1_ –$P$_1_ and $A$_1_ –$P$_0_
are popular in detecting the presence of nasalization in vowels [22]. However,
derivation of these parameters requires manual demarcation of locations of
$F$_0_ , $P$_0_ and $P$_1_ in STFT, which is a tedious process. Furthermore,
the harmonic structure of spectrum and the analysis window response pose
difficulty in resolving the nasal formant and other spectral peaks in STFT.
The dominant resonance frequency (DRF) contour derived from the HNGD spectra
are therefore utilized to identify the presence of nasal resonance in vowel
spectra. DRFs have proven to be a consistent and concise representation of the
spectral characteristics in speech, and efficiently reflect changes in
production system due to coupling/decoupling of cavities to the oral tract
[20]. Ability of DRFs to capture distinction in the production system response
for oral and nasalized vowels also serves as motivation to this study.

##  3 Analysis of oral and nasalized vowels using DRFs

The section describes behavior of DRF contours for oral vowels and nasal
segments. The behavior is further investigated for nasalized vowel segments.
Previous studies hypothesized that vowels present in these contexts are always
nasalized. The present section investigates the behavior of DRFs in oral
vowels and nasal segments. A hypothesis is suggested on the basis of this
behavior, to study the behavior of DRF contour during vowel nasalization. This
hypothesis is validated based on previously proposed spectral correlates. The
validation is conducted for vowels in English language, obtained from
utterances recorded by male and female speakers of English language in TIMIT
database [23].

###  3.1 Distinction in behaviour of DRFs for oral vowel and nasal segments

Distinction in the spectral structure among nasal and oral vowel segments has
been widely discussed in literature. Presence of a characteristic resonance in
lower frequencies, and a spectral null following it, is attributed to the
presence of a longer nasal cavity coupled with a closed oral cavity, during
production of nasals. Production of oral vowels is usually characterized by
spectral resonances in relatively higher frequency range. These spectral
characteristics are derived at a frame duration $20$–$30$ ms with an
underlying assumption of stationarity of the production system. Such analysis
averages the behavior of speech production system, which otherwise is
continuously changing in nature.

Figure 2: $\mu_{D}$ vs $\sigma_{D}$ across glottal cycles for (a) nasal
consonants /m/ and /n/, and (b) vowels /æ/, /e/, /2/, /i/, and /u/ .

The present study uses HNGD spectrum, obtained over short duration of $4$ ms,
to derive the DRF contours reflecting instantaneous changes in production
system response. The glottal activity during production of oral and nasal
vowels leads to a fluctuation of DRFs within high and low frequency range for
each glottal cycle [20]. These fluctuations are characterized using mean
($\mu_{D}$) and standard deviation ($\sigma_{D}$) derived over the DRF contour
across every glottal cycle, for vowel and nasal segments. $\mu_{D}$ indicates
the centroid and $\sigma_{D}$ indicates the bounds of fluctuation in DRF
contours for respective utterances. Fig. 2 shows the distribution of these
values in spectral plane, obtained over multiple instances of oral vowel and
nasal segments in continuous speech in English language, for utterances chosen
from TIMIT dataset [23]. Fig. 2(a) gives location of $\mu_{D}$ vs.
$\sigma_{D}$ values for nasal (/m/ and /n/) segments, and Fig. 2(b) gives
these for oral vowel (/æ/, /e/, /2/, /i/, and /u/) segments. Oral vowels are
chosen as vowel segments appearing in context with fricative and stop
consonants. Except for /i/, the non–overlapping clusters for oral vowels and
nasals show distinction in the production system characteristics for oral
vowels and nasals. Literature have highlighted the difficulties in identifying
nasalization within front vowels [11]. Thus, the study will not cover
instances of /i/ for analysis. For other oral vowels and nasals, clusters of
$\mu_{D}$ vs. $\sigma_{D}$ help in deriving bounds for the expected range of
occurrence and fluctuations of DRF contours. Given the figure, the $300$–$400$
Hz range (= $B_{N}$) is understood as expected range of fluctuation of DRF
contour for nasal segments (Fig. 2 (a)), and the $450$–$850$ Hz range (=
$B_{V}$) for oral vowels (Fig. 2 (b)).

Figure 3: DRF contour obtained for VC transition segment. (a) Speech with the
GCI locations (dotted). (b) DRF contour. (c) $\mu_{D}$ (dots) and $\sigma_{D}$
(bars) within each glottal cycle. Figure 4: Comparison of DFT and HNGD spectra
for OV, NV and NC segments. (a1), (b1) and (c1) DFT spectra in OV, NV and NC
regions, respectively, with the $A$_1_ –$P$_0_. (a2), (b2) and (c2) HNGD
spectra.

###  3.2 DRF contour for vowels appearing in nasal context

Vowels appearing in CV/VC context with nasal consonants experience a
contextual load, which dictates the graduation of instantaneous spectral
behavior from $B_{V}$ to $B_{N}$, towards the conjunction boundary of these
segments. A sudden transition reflects the absence of overlap of their
respective spectral characteristics. A gradual transition of DRFs within
$B_{V}$, towards the characteristics of $B_{N}$, within the vowel segment
indicates an overlap of nasal spectral characteristics. Fig. 3 illustrates one
such example of graduation in DRF contour across the length of a vowel
segment, present with a nasal context. Fig. 3(a) shows the signal for a VC
pair /æ/ and /n/, in the utterance _`man'_ spoken by a female speaker of
English language. The DRF contour obtained from the HNGD spectra using the ZTW
method with an analysis window of $4$ ms is shown in Fig. 3(b). The VC
boundary appears at $300$ ms as given by the annotations provided with the
database. Changes in the behavior and fluctuations of DRF contour across the
length of vowel can be noted in the figure. Glottal cycles are identified
using glottal closure instants (GCIs) derived using ZFF method [18] (Fig.
3(a)).

Three locations, $w1$ (near the vowel onset), $w2$ (near the VC conjunction
region), and $w3$ (post the conjunction region) are chosen to illustrate the
distinction and similarity, in spectral behavior across the vowel segment with
the nasal spectra, for different instances. The locations, $w1$ and $w2$ lie
in the vowel segment whereas $w3$ lies in the nasal segment. Figs. 4(a1),
4(b1) and 4(c1) show the DFT spectrum obtained at $w1,w2$ and $w3$ in the
segment. The oral formant can be resolved around $550$ Hz as the strongest
harmonic peak, in Fig. 4(a1), with an amplitude value understood as $A$_1_.
The characteristic low frequency nasal resonance is resolved around
$300$–$350$ Hz, in Fig. 4(c1), with an amplitude value $P$_0_. The parameter
$A$_1_ –$P$_0_ exhibits relatively a higher value at the window location $w1$
(Fig. 4(a1)) as compared to location $w2$ (Fig. 4(b1)). The value of $P$_0_
appears larger than $A$_1_ for the case of location $w2$, which is closer to
the VC conjunction region, resulting in a negative value of the parameter.
Another limitation with the parameter $A$_1_ –$P$_0_ , apart from resolving
the locations of spectral peaks, is that it needs to be observed in a relative
frame of reference. It is therefore difficult to determine the presence of
nasalization for segments with few instances of the same VC/CV pair.

Figs. 4(a2), 4(b2) and 4(c2), show the HNGD spectrum obtained at $w1,w2,$ and
$w3$ using $l=5$ ms, respectively. The oral and nasal resonances in
$550$–$600$ and $300$–$350$ Hz regions are easily resolved in the HNGD spectra
in Figs. 4(a2) and 4(b2), respectively. The HNGD spectrum at $w2$ exhibits
both oral and nasal resonances with similar dominant behavior. This asserts
the simultaneous presence of both these cavities during the production, which
is also verified with a relatively lower value of $A$_1_ –$P$_0_ in this
region. The DRF contour in Fig. 3(b) illustrates the distinction in oral and
nasalized segments based on the bound of their fluctuation within each glottal
cycle. The oral vowel and nasal segments exhibit DRFs in $B_{V}$ and $B_{N}$
ranges, respectively, as suggested by the $\mu_{D}$ and $\sigma_{D}$ values in
Fig. 3(c). The nasalized segments exhibit DRFs fluctuating between $B_{V}$ and
$B_{N}$ ranges, owing to presence of a coupled oral and nasal tract, leading
to a shift in $\mu_{D}$ and higher $\sigma_{D}$ values. Due to a high temporal
resolution, it is easier to demarcate the presence of nasalization for each
glottal cycle with an improved accuracy.

##  4 Behavior of DRF contour for a variability in coupling of oral and nasal
tracts

DRFs appear as distinctive feature of the HNGD spectrum to represent the
instantaneous vocal tract system response. Adduction of the nasal tract to the
oral tract results in the dominance of a characteristic low frequency nasal
resonance, which is efficiently captured by the DRFs in HNGD spectrum. The
presence section studies the characteristics of DRF contours for different
cases of oral–nasal coupling. The effect of a higher extent of this coupling
on DRF behavior is explored in detail. The section also highlights the
importance of segment duration, and hence determination of the glottal open
phase, towards identification of nasalization in vowels.

###  4.1 Behavior of DRF for vowels nasalized at different extents

A significant transition in the behavior of DRFs in vowel segments is
illustrated by deviation in their fluctuation from $B_{V}$ range. There can be
several factors to cause such a deviation, but for the study of nasalization
in vowels, such a behavior is largely dependent on the extent of coupling of
the oral and nasal tracts. Presence of nasalization can easily be tracked
using the $\mu_{D}$ and $\sigma_{D}$ parameters derived for the DRF contours.
A change in the extent of coupling, however, dictates the extent of shift in
values of these parameters.

Figure 5: DRF contours obtained using ZTW analysis, for different extents of
nasalization. (a1)–(f1) Speech signals for syllable _/_ me _/_ , _/_ m2 _/_ ,
_/_ æn _/_ , _/_ m2 _/_ , _/_ mi _/_ and _/_ n1 _/_ , respectively. (a2)–(f2)
DRF contours for the corresponding signals.

Figure 5 shows the DRF contour for VC/CV segments of English words uttered by
different male and female speakers of English, to illustrate the difference in
extent of oral–nasal coupling. The different VC/CV pairs in the figures are as
follows: Fig. 5(a1) for _/_ me _/_ (word `_me lody'_, gender: male), Fig.
5(b1) for _/_ m2 _/_ (word `_mo nday'_, gender: male), Fig. for 5(c1) _/_ æn
_/_ (word `_an '_, gender: female), Fig. 5(d1) for _/_ m2 _/_ (word `_le
mon'_, gender: female), Fig. 5(e1) for _/_ mi _/_(word `_me '_, gender: male)
and Fig. 5(f1) for _/_ ni _/_ (word `_mo ney'_, gender: male). The VC/CV
transition boundary can be located around $50$ ms for all the segments as
given by respective annotations. In Figs. 5(a2) and 5(b2), the DRF contours
corresponding to oral and nasal segments appear bounded within the $B_{V}$ and
$B_{N}$ range, respectively. Figs. 5(c2) and 5(d2) show the cases where the
DRF contours for vowel segments, fluctuate between the $B_{V}$ and $B_{N}$
range. This reflects the simultaneous dominance of oral and nasal resonances,
and hence the vowel is hypothesized as partially nasalized.

Figs. 5(e2) and 5(f2) show the DRF contours appearing completely in the
$B_{N}$ range for the vowel segment, with a behavior similar to nasal segment.
The VC/CV transition boundaries occur around $50$ ms (Figs. 5(e1) and 5(f1)) ,
which cannot be observed in the respective DRF contours. This behavior
reflects the presence of a dominant nasal resonance for the entire duration of
vowel segment, which is attributed to a larger extent of coupling of oral and
nasal tracts. Such a coupling masks oral resonance, and hence the DRF contours
shift completely towards $B_{N}$ range. The following Secs. 4.2 and 4.3
present further discussions on this.

###  4.2 Significance of glottal open region for studying degree of
nasalization

Relation between glottal open phase and degree of nasalization, with a higher
open quotient leading to a higher degree of nasalization, has been studied in
literature [11]. It has also been reported that spectral behavior for a
nasalized vowel is similar to addition of the transfer function of oral and
nasal cavities, along with a shift and broadening of the formant peaks [24].
The present section examines these factors, based on the behavior of DRF
contour in nasalized vowel segments.

Fig. 6 illustrates a case of partial extent of nasalization of a vowel
segment. Once again, this claim of extent is based on the behavior of DRF
contour which fluctuates between $B_{V}$ and $B_{N}$ range. Fig. 6(a) shows
the speech signal along with the GCI locations for the VC segment (/on _/_
word: _` only'_ in English language, uttered by English male speaker). Fig.
6(b) shows the DRF contour obtained from the HNGD spectra with $l=6$ ms. The
regions of nasal consonant (NC), nasalized vowel (NV) and oral vowel (OV)
segments are marked manually across the CV segment based on the DRF contour
behavior, with NV exhibiting a partial degree of nasalization in $70$–$130$ ms
duration. The $B_{V}$ and $B_{N}$ ranges can be identified from OV and NC
segments in range $500$–$650$ Hz and $200$–$400$ Hz, respectively. A close
observation suggests that the DRFs in vicinity of GCIs appear in the $B_{V}$
range, whereas those in the middle of two GCI locations appear transit to the
$B_{N}$ range.

---
Figure 6: Illustration of DRF contours and HNGD spectra for the partial
opening of the velopharyngeal port during the production of the vowel segment.
(a) speech signal for VC /on/ (word: only) with GCI. (b) DRF contour. (c)
Magnified DRF contour from $10$–$100$ ms with analysis regions R$1$ and R$2$.
(d1)–(d4) HNGD spectra at equidistant windows in region R$1$. (e1)–(e4) HNGD
spectra for equidistant windows in region R$2$

Fig. 6(c) shows a magnified view of the DRF contour for the segment from OV
and NV regions, in $10$–$100$ ms duration. The figure also marks two regions
$R1$ and $R2$ within OV and NV regions, demarcated manually, which essentially
span from the onset of the glottal open phase to part of the close phase, as
derived by the behavior of DRF contour [20]. These regions are identified to
illustrate the evolution of spectral behavior across a glottal cycle for OV
segments, and NV segments owing to a partial degree of nasalization. The
closed region behavior is nearly identical for both OV and NV segments. The
spectrum within $R1$ and $R2$, sampled at four equidistant locations, is
showed in Figs. 6(d1)–(d4) and 6(e1)–(e4), respectively. Figs. 6(d1)–(d2)
correspond to glottal open region for R$1$. A spectral dominance can be noted
around $450$ Hz, which further can be seen shifting to higher frequency range
($\sim 600$ Hz) during the glottal closed phases, in Figs. 6(d3)–(d4). During
the glottal open phase for OV segments, this close phase resonance shifts to a
higher $700$–$750$ Hz as a secondary peak, in Fig. 6(d2).

A shift in location of oral vowel formants during nasalization has already
been discussed in the literature [11]. Figs. 6(e1)–(e4) show the spectral
evolution during $R2$, present in NV segment. Dominance of low frequency
characteristic nasal resonance at $200$ Hz during glottal open phase (Figs.
6(e1)–(e3)), along with the shifted oral resonance ($\sim 600$–$800$ Hz)
impress upon the coupling of oral and nasal tracts. The dominant behavior of
the low frequency resonance during the glottal open phase transits back to
oral formant location during glottal closed phase, in Fig. 6(e4). Similarity
in DRF during glottal closed phase for OV and NV segments can be noted in
Figs. 6(d4) and 6(e4). The oral resonances are weaker and hence masked by
presence of a stronger nasal zero in the vicinity of first formant, during low
SNR glottal open phase. The figure illustrates the alternating dominance of
oral and nasal resonances for a partial extent of nasalization. This verifies
the hypothesis of a partial coupling of oral–nasal tracts, as claimed earlier
for Figs. 5(c2) and 5(d2).

The effect of analysis window duration on identification of glottal open and
closed phases in speech has been discussed in the literature [20]. The
dominant resonances during the glottal open phase are weaker and hence are
masked if the size of the analysis window is longer than few pitch periods. It
is therefore necessary to maintain the analysis window duration comparable to
the pitch duration to study nasalization in vowels.

###  4.3 Full extent of coupling of oral–nasal tracts

It has been discussed in the literature that a higher extent of coupling of
oral–nasal tracts has a more drastic effect on the vowel spectra [24]. A
larger opening of the velopharyngeal section results in a stronger nasal
resonance which dominates the oral resonance during entire glottal cycle. Oral
resonance experiences significant decay due to a higher extent of coupling.
Fig. 7 illustrates this phenomenon with the help of strength of resonances.

Fig. 7(a) shows a vowel segment _/_ æ _/_ , present in context of nasal
consonants _/_ m _/_ and _/_ n _/_ (word: _` man'_ in English language by a
female English speaker). The GCI locations for the segment are shown by dotted
vertical lines, along with demarcations for nasal segments (NC1 and NC2), oral
vowel (OV) and nasalized segments (NV1 and NV2). Fig. 7(b) shows the DRF
contour for the segment obtained using the ZTW method with $l=6$ ms. DRFs in
NC1 and NC2, and OV segments appear in $B_{N}$ and $B_{V}$ range.

Figure 7: DRF and DRF2 contours illustrating the variation in degree of
coupling of nasal and oral tracts. (a) Speech segment with the GCI locations
(dotted). (b) DRF contour. (c1) and (c2) DRF (solid line) along with DRF2
(**). (d1) and (d2) $|1-a2/a1|$.

The NV1 and NV2 segments ($80$–$260$ ms duration) are nasalized. Splitting of
nasalized vowel segment within NV1 and NV2 is done to illustrate the
difference of extent of coupling of oral–nasal tracts. Regions within NV1 and
NV2 are chosen to illustrate the difference in behavior of DRF contour for
different extent of nasalization. Figs. 7(c1) and 7(c2) show the DRF (solid
line) and DRF2 (**) contours within NV1 and NV2 segments, respectively. The
DRF2 are the secondary dominant resonances in the spectrum and are used in
this context to illustrate the shift in dominance. These are identified as
resonances with strength next to DRF in the HNGD spectrum. Figs. 7(d1) and
7(d2) show the relative difference $(\alpha=|1-(a2/a1)|)$ of amplitudes $a1$
and $a2$ of DRF and DRF2, respectively, within NV1 and NV2 segments,
respectively.

The NV1 segment exhibits a partial extent of nasalization as suggested by the
DRF contour. The DRF2 contour (Fig. 7(c1)) appears appropriately in the gaps
in DRF contour, suggesting the simultaneous presence of resonances in $B_{V}$
and $B_{N}$ range, across the vowel segment. DRF contour in NV2 segment shifts
completely to $B_{N}$ range reflecting the increased dominance of nasal
resonance over the oral resonance. The DRF2 also shifts appropriately to
$B_{V}$ range (Fig. 7(c2)). Average value of $\alpha$ can also be observed
rising with each glottal cycle while transiting from NV1 to NV2 segments (Fig.
7(d2)). This reflects a stronger nasal resonance compared to the oral
resonance in NV2 segments. A stronger nasal characteristic resonance within
vowel segments signifies a larger extent of velopharyngeal opening.

##  5 Observations at CV/VC boundaries for vowel–nasal pairs

The section validates the relation between $\sigma_{D}$ and $A1-P0$ for
different sets of oral and nasalized vowels. The section also examines the
presence of nasalization owing to co–articulatory load across multiple
instances of CV/VC and CVC utterances in English.

###  5.1 Relation in $A$_1_ –$P$_0_ and $\sigma_{D}$

Figure 8: $A$_1_ –$P$_0_ vs $\sigma_{\text{DRF}}$ for OV ($\circ$) and NV
($\times$) segments.

Values of $A$_1_ –$P$_0_ , and their relation with $\sigma_{D}$ values, is
explored for OV and NV segments in TIMIT database [23]. The OV and NV segments
are identified using the proposed hypothesis based on the DRF contours. Fig. 8
gives values obtained for $A$_1_ –$P$_0_ against $\sigma_{D}$ for different OV
and NV segments obtained across several speakers in TIMIT. The OV segments are
obtained from vowels appearing in CV/VC clusters with fricative or stop
consonants. The NV segments are obtained from vowels appearing in context of a
nasal consonant with their DRFs fluctuating to $B_{N}$ range. The $A$_1_
–$P$_0_ values are derived over a $20$ ms frame duration at VC/CV transition
region for vowel segments. Peaks corresponding to $A$_1_ and $P$_0_ are marked
manually in the normalized log spectrum obtained using DFT. $\sigma_{D}$
values obtained with ZTW using analysis window of $4$ ms, are averaged across
a duration of $20$ ms at VC/CV transition boundary. It is observed that $A$_1_
–$P$_0_ values for oral vowel segments ($\circ$) are higher and positive .
This signifies a stronger $F_{1}$ as compared to low frequency resonance. The
corresponding $\sigma_{D}$ values are also found to be relatively lower ($\sim
70-130$ Hz). The NV segments on the other hand, result in negative $A$_1_
–$P$_0_ values ($\times$) appearing in distinct cluster. Values of
$\sigma_{D}$ are also relatively higher ($\sim 320-360$ Hz) for NV segments.
These two non–overlapping clusters suggests that proposed analysis aligns with
previously suggested spectral correlates to capture nasalization in vowels.
The proposed method based on DRF contour however, reflects the spectral
information at a better resolution, and makes it relatively easier to be
characterised. .

###  5.2 Study of vowels at VC/CV transition boundaries

The proposed method is employed to examine the behavior of vowel segment at
CV/VC transition boundaries when present in context with nasal consonants. The
$\mu_{D}$ and $\sigma_{D}$ values are derived from DRF contours, obtained
using ZTW analysis with $l=5$ms. The VC/CV segments are obtained from
utterances by different male and female speakers in TIMIT. Values of $\mu_{D}$
and $\sigma_{D}$, averaged across a duration of $5$ glottal cycles in the
vowel segment, at the CV/VC transition boundary are reported. Figs. 9 and 10
show observations for vowels /æ/ and /Y/ present in context with nasal
consonants /m/ and /n/, in utterances obtained across different speakers.

Figure 9: Errorbars for average $\mu_{D}$ and $\sigma_{D}$ values obtained
across VC/CV boundaries for vowel /æ/.  Figure 10: Errorbars for average
$\mu_{D}$ and $\sigma_{D}$ values obtained across VC/CV boundaries for vowel
/Y/.

Few segments exhibit a similarity in behavior for $\mu_{D}$ values, in /æ/,
where these values are present in the range $B_{V}$, as seen in Figs.
9(a1)–9(b2). Some of these values for /Y/, especially for VC transition,
appear in range $B_{N}$ which shows the affinity of this vowel to get
nasalized, as seen in Figs. 10(a2)–10(b2). Few instances for /æ/ at VC
boundary with /m/ (Fig. 9(a1)) show increased value of $\sigma_{D}$, which
also illustrates the overlap of $B_{V}$ and $B_{N}$ and hence presence of
nasalization. The figures illustrate a dynamic nature of nasalization due to
contextual load across similar CV/VC clusters, for different speakers.

##  6 SUMMARY

This paper presents a new way to study the phenomena of nasalization of vowels
based on dominant behavior of the instantaneous spectral characteristics. The
problems of identification of duration and extent of nasalization in vowels
are addressed in this paper. The DRFs obtained from HNGD spectrum fluctuate
within characteristic $B_{V}$ and $B_{N}$ range for oral vowels and nasal
consonants, respectively. Changes in the spectral structure of vowels due to
coupling of oral–nasal tracts are captured based on the average location and
spread of DRFs. This behavior is characterized across each glottal cycle using
the $\mu_{D}$ and $\sigma_{D}$ parameters. A partial extent of nasalization
examined across glottal open and closed phase, and is illustrated through
fluctuations in DRFs within $B_{V}$ and $B_{N}$ range. The paper also explains
the shift in dominance within oral and nasal resonances across glottal phases,
for a partial nasalization. A higher extent of coupling is illustrated by a
stronger nasal resonance dominating across the entire glottal cycle.
Examination of secondary dominant resonance proves a continuum in DRF contours
in $B_{V}$ and $B_{N}$ range during nasalization. Comparison of strengths of
DRF and DRF2 shows the transition of dominance within oral and nasal
resonances.

The study is carried out over nasal–vowel pairs occurring words in the English
language, spoken by different male and female English speakers. The behavior
of $\sigma_{D}$ for nasalized vowel segments is also validated using spectral
correlate $A1-P0$ derived across VC/CV boundaries. The DRF2 contour helps to
illustrate the simultaneous presence of oral and nasal resonances in the
spectrum for nasalized vowels, and the fluctuation in the dominant behavior
between the two. The study can easily be implemented across different
languages and speakers, and can help in studying speech pathology.

## References

  * [1] A. S. House and K. N. Stevens, ``Analog studies of the nasalization of vowels,'' Journal of Speech and Hearing Disorders, vol. 21, no. 2, pp. 218–232, 1956.
  * [2] G. Fant, Acoustic theory of speech production.  Mouton & Co. The Hague, Netherlands, 1958.
  * [3] S. Hawkins and K. N. Stevens, ``Acoustic and perceptual correlates of the non-nasal–nasal distinction for vowels,'' The Journal of the Acoustical Society of America, vol. 77, no. 4, pp. 1560–1575, 1985.
  * [4] J. Glass and V. Zue, ``Detection of nasalized vowels in american english,'' in Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP'85, vol. 10, pp. 1569–1572, 1985.
  * [5] M. K. Huffman, Implementation of nasal: timing and articulatory landmarks.  Phonetics Laboratory, Department of Linguistics, UCLA, 1990.
  * [6] D. H. Klatt and L. C. Klatt, ``Analysis, synthesis, and perception of voice quality variations among female and male talkers,'' The Journal of the Acoustical Society of America, vol. 87, no. 2, pp. 820–857, 1990.
  * [7] O. Fujimura and J. Lindqvist, ``Sweep-tone measurements of vocal-tract characteristics,'' The Journal of the Acoustical Society of America, vol. 49, no. 2B, pp. 541–558, 1971.
  * [8] J. Lindqvist-Gauffin and J. Sundberg, ``Acoustic properties of the nasal tract,'' Phonetica, vol. 33, no. 3, pp. 161–168, 1976.
  * [9] M. Båvegård, G. Fant, J. Gauffin, and J. Liljencrants, ``Vocal tract sweeptone data and model simulations of vowels, laterals and nasals,'' STL-QPSR, vol. 4, pp. 43–76, 1993.
  * [10] J. Dang, K. Honda, and H. Suzuki, ``Morphological and acoustical analysis of the nasal and the paranasal cavities,'' The Journal of the Acoustical Society of America, vol. 96, no. 4, pp. 2088–2100, 1994.
  * [11] M. Y. Chen, ``Acoustic parameters of nasalized vowels in hearing–impaired and normal–hearing speakers,'' The Journal of the Acoustical Society of America, vol. 98, no. 5, pp. 2443–2453, 1995.
  * [12] M. Y. Chen, ``Acoustic correlates of English and French nasalized vowels,'' The Journal of the Acoustical Society of America, vol. 102, no. 4, pp. 2360–2370, 1997.
  * [13] S. Ploch, Nasals on my mind: the phonetic and the cognitive approach to the phonology of nasality. PhD thesis, School of Oriental and African Studies (University of London), 1999.
  * [14] T. Pruthi and C. Y. Espy-Wilson, ``Acoustic parameters for the automatic detection of vowel nasalization.,'' in Proc. INTERSPEECH, 2007 (Antwerp, Belgium), pp. 1925–1928, Citeseer, 2007.
  * [15] P. Vijayalakshmi, M. R. Reddy, and D. O'Shaughnessy, ``Acoustic analysis and detection of hypernasality using a group delay function,'' IEEE Transactions on Biomedical Engineering, vol. 54, no. 4, pp. 621–629, 2007.
  * [16] B. Yegnanarayana, ``Formant extraction from linear-prediction phase spectra,'' The Journal of the Acoustical Society of America, vol. 63, no. 5, pp. 1638–1640, 1978.
  * [17] B. Yegnanarayana and N. Dhananjaya, ``Spectro-temporal analysis of speech signals using zero-time windowing and group delay function,'' Speech Communication, vol. 55, no. 6, pp. 782–795, 2013.
  * [18] K. S. R. Murty and B. Yegnanarayana, ``Epoch extraction from speech signals,'' IEEE Transactions on Audio, Speech, and Language Processing, vol. 16, no. 8, pp. 1602–1613, 2008.
  * [19] M. A. Joseph, S. Guruprasad, and B. Yegnanarayana, ``Extracting formants from short segments of speech using group delay functions,'' in ICSLP-IS, (Pittsburgh PA, USA), pp. 1009–1012, Sept. 2006.
  * [20] R. S. Prasad and B. Yegnanarayana, ``Determination of glottal open regions by exploiting changes in the vocal tract system characteristics,'' The Journal of the Acoustical Society of America, vol. 140, no. 1, pp. 666–677, 2016\.
  * [21] P. Tarun, Analysis, vocal-tract modeling and automatic detection of vowel nasalization.  PhD thesis, Uiversity of Maryland, Tech. Rep, 2007.
  * [22] W. Styler, ``On the acoustical features of vowel nasality in english and french,'' The Journal of the Acoustical Society of America, vol. 142, no. 4, pp. 2469–2482, 2017.
  * [23] J. S. Garofalo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett, and N. L. Dahlgren, ``The darpa timit acoustic-phonetic continuous speech corpus cdrom,'' Linguistic Data Consortium, 1993.
  * [24] K. N. Stevens, Acoustic phonetics, vol. 30.  MIT press, 2000.