# Second harmonic light scattering induced by defects in the twist-bend
nematic phase of achiral liquid crystal dimers

Shokir A. Pardaev  Department of Physics, Kent State University, Kent, Ohio
44242, USA
S. M. Shamid  Department of Physics, Kent State University, Kent, Ohio 44242,
USA
M. G. Tamba  Department of Chemistry, University of Hull, Hull HU6 7RX, UK
C. Welch  Department of Chemistry, University of Hull, Hull HU6 7RX, UK
G. H. Mehl  Department of Chemistry, University of Hull, Hull HU6 7RX, UK
J. T. Gleeson  Department of Physics, Kent State University, Kent, Ohio 44242,
USA
D. W. Allender  Department of Physics, Kent State University, Kent, Ohio
44242, USA
J. V. Selinger  Chemical Physics Interdisciplinary Program and Liquid Crystal
Institute, Kent State University, Kent, Ohio 44242, USA
B. Ellman  Department of Physics, Kent State University, Kent, Ohio 44242, USA
A. Jakli  Chemical Physics Interdisciplinary Program and Liquid Crystal
Institute, Kent State University, Kent, Ohio 44242, USA
S. Sprunt<EMAIL_ADDRESS>Department of Physics, Kent State University,
Kent, Ohio 44242, USA

###### Abstract

The nematic twist-bend ($\mathrm{N_{TB}}$) phase, exhibited by certain
thermotropic liquid crystalline (LC) dimers, represents a new orientationally
ordered mesophase – the first distinct nematic variant discovered in many
years. The $\mathrm{N_{TB}}$ phase is distinguished by a heliconical winding
of the average molecular long axis (director) with a remarkably short
(nanoscale) pitch and, in systems of achiral dimers, with an equal probability
to form right- and left-handed domains. The $\mathrm{N_{TB}}$ structure thus
provides another fascinating example of spontaneous chiral symmetry breaking
in nature. The order parameter driving the formation of the heliconical state
has been theoretically conjectured to be a polarization field, deriving from
the bent conformation of the dimers, that rotates helically with the same
nanoscale pitch as the director field. It therefore presents a significant
challenge for experimental detection. Here we report a second harmonic light
scattering (SHLS) study on two achiral, $\mathrm{N_{TB}}$-forming LCs, which
is sensitive to the polarization field due to micron-scale distortion of the
helical structure associated with naturally-occurring textural defects. These
defects are parabolic focal conics of smectic-like “pseudo-layers”, defined by
planes of equivalent phase in a coarse-grained description of the
$\mathrm{N_{TB}}$ state. Our SHLS data are explained by a coarse-grained free
energy density that combines a Landau-deGennes expansion of the polarization
field, the elastic energy of a nematic, and a linear coupling between the two.

###### pacs:

61.30.Eb,61.30.Dk,64.70.M-

††preprint: APS/123-QED

##  I Introduction

Polar orientational order in liquid crystals (LCs) is traditionally associated
with the hindered rotation of chiral molecules arranged in layers (smectic
phase). In conventional ferroelectric LCs, this hindrance is caused by tilt of
molecules with respect to the layer normal (smectic-C phase), which leads to
broken inversion symmetry and a spontaneous polarization parallel to the
layers. The development and exploration in the past 20 years of mesogens with
a bent-shaped (rather than rod-like) rigid core structure has seen the
requirement of molecular chirality eliminated: In these systems of achiral
molecules, both smectic-C and smectic-A (untilted) phases exhibit in-layer
polarization and, perhaps more interestingly, in the smectic-C case
spontaneously separate into domains of opposite structural chirality. Layer
tilt plus layer polarization therefore induce chirality.

So far, however, the realization of polar orientational order without layering
or constituent chirality has proven elusive. While simple nematics exhibit a
typically weak polarization under an applied orientational stress (the so-
called flexoelectric effect Meyer_flexo ), in equilibrium the magnitude of the
vector order parameter representing the dipole moment of the molecules
vanishes. The recent discovery of the twist-bend nematic ($\mathrm{N_{TB}}$)
phase Cestari_PRE ; Borshch_Nature ; Chen_PNAS , originally predicted by Meyer
Meyer_TB , may have fundamentally altered this situation.

The $\mathrm{N_{TB}}$ phase typically occurs at temperatures below the
ordinary uniaxial nematic phase, in achiral liquid crystal dimers composed of
a pair of rodlike mesogenic units connected by an odd-numbered hydrocarbon
linkage. This linkage favors an overall bent conformation (Fig. 1(a)). As
shown in Fig. 1(b), the $\mathrm{N_{TB}}$ state is characterized by a
modulated orientational structure in which the average molecular long axis
(local director $\hat{\mathbf{n}}$) simultaneously bends and twists in space
in a periodic fashion, producing a heliconical configuration. The typical cone
angle (average tilt of the molecules away from the helical axis, inferred from
optical birefringence measurements) Borshch_Nature ; CMeyer3 , is $\beta\sim
10^{\circ}$. The pitch ($t_{0}$) of the modulation, measured directly by
freeze-fracture TEM Chen_PNAS , is surprisingly short – of order a few
molecular lengths (i.e., $t_{0}\sim 10$ nm). Both parameters differ markedly
from the usual chiral nematic (cholesteric), where $\beta=90^{\circ}$ and
$t_{0}\gtrsim 100$ nm.

Various theoretical models Dozov_TB ; CMeyer2 ; Shamid_PRE ; Kats ; Virga ;
Greco ; Barbero ; Lelidis ; Vanakaras  have been proposed to account for the
properties of the $\mathrm{N_{TB}}$ phase and the nature of the
nematic-$\mathrm{N_{TB}}$ transition. Dozov Dozov_TB  and Meyer et al CMeyer2
developed a model based on the Frank elasticity for bend distortions of
$\hat{\mathbf{n}}$ becoming negative at the transition; in this thoery, the
order parameter for the $\mathrm{N_{TB}}$ state is $\sin^{2}\beta$. Two other
models – those of Kats and Lebedev Kats  and Shamid et al Shamid_PRE  –
introduce a vector order parameter and expand the Landau-deGennes free energy
density in terms of this order parameter and $\hat{\mathbf{n}}$. In
particular, Shamid et al Shamid_PRE  propose that the order parameter is a
helical polarization wave, $\mathbf{P}$, which is orthogonal to, but has the
same nanoscale pitch as, the heliconical director $\hat{\mathbf{n}}$.
($\mathbf{P}$ may represent a shape polarization in lieu of, or in addition
to, an electric polarization.) A linear coupling of $\mathbf{P}$ to the curl
of $\hat{\mathbf{n}}$ drives the heliconical modulation of $\hat{\mathbf{n}}$,
and, on the nematic side of the transition, renormalizes the bend elastic
constant to zero, destabilizing the nematic phase in agreement with the theory
of Dozov Dozov_TB  and with experiment Adlem . While studies of fluctuations
in the $\mathrm{N_{TB}}$ phase are consistent with the presence of a short-
pitch polarization wave Parsouzi_PRX , more direct confirmation, by a method
sensitive to broken centrosymmetry due to $\mathbf{P}$, is clearly desirable.
Given the absence of any mass density wave (Bragg peaks) in X-ray diffraction
measurements Chen_PNAS ; Mehl_M37 ; Ahmed , confirmation of $\mathbf{P}$ would
establish the $\mathrm{N_{TB}}$ state as the first example of a locally polar
LC phase without smectic-like molecular layering – a prospect of fundamental
significance.

This paper presents the results of a second harmonic light scattering (SHLS)
study designed to test this prospect. In order to detect at optical
wavelengths a polarization field that averages out over a nanoscale helical
pitch, our approach takes advantage of another interesting feature of the
$\mathrm{N_{TB}}$ phase – the presence of topological defects, analogous to
the classical parabolic focal conic (PFC) defects in the layer structure of
simple smectic LCs, that distort the polarization field over a length scale
much larger than $t_{0}$ and thereby produce a nonzero $\mathbf{P}$ on the
optical scale suited to the SHLS technique Pardaev_PRE .

Figure 1: (a): Structure of LC dimers that form the $\mathrm{N_{TB}}$ phase in
the mixture KA(0.2) and the single compound DTC5C7. (b): Schematic rendering
of the heliconical molecular orientation in the $\mathrm{N_{TB}}$ phase, with
cone angle $\beta$ and helical wavenumber $q_{0}$. Each cylinder represents
the average molecular long axis of a dimer, and the vectors $\hat{\mathbf{n}}$
and $\mathbf{P}$ correspond to the local director and polarization fields,
respectively, while $\hat{\mathbf{t}}$ is the average or “coarse-grained”
director. (c): 3D CAD rendering of the experimental apparatus.

Specifically, we demonstrate an SHLS signal in the $\mathrm{N_{TB}}$ phase,
whose key features (optical polarization selection rules, in particular) are
explained by a model in which slabs of the $\mathrm{N_{TB}}$ system, defined
by planes of constant helical phase, are treated as “pseudo-layers”. When
their spacing is reduced due to a temperature dependent helical pitch, these
“layers” exhibit PFC defects in an a analogous manner as true smectics do when
placed under a dilatory strain. The concept of “pseudo-layers” was previously
introduced in the development of a “coarse-grain” theory of cholesterics
deGennes ; Lubensky_PRA ; Radzihovsky_PRE .

This concept has recently been applied to the $\mathrm{N_{TB}}$ phase to
describe smectic-like elastic properties CMeyer3 ; Parsouzi_PRX  and
fluctuation modes Parsouzi_PRX , strong shear thinning behavior Jakli1 , and
optical stripe and focal textures Jakli2 . Starting from a theoretically
proposed free energy density coupling $\hat{\mathbf{n}}$ and $\mathbf{P}$
Shamid_PRE , we show that “pseudo-layer” deformations associated with PFCs
naturally lead to a distortion of the helical field $\mathbf{P}$ that breaks
centrosymmetry at length scales comparable to (or greater than) optical
wavelengths. We also verify that the SHLS signal is absent in an ordinary
calamitic smectic liquid crystal containing a comparable density of PFCs, but
lacking any underlying polar structure. Our results and analysis thus provide
strong evidence of an underlying helical polarization field that characterizes
the $\mathrm{N_{TB}}$ phase.

##  II Experimental Details

The $\mathrm{N_{TB}}$ materials studied are: (1) a mixture [denoted KA(0.2)]
Adlem  of five odd-membered dimers with ether linkages between mesogenic
groups plus one dimer with a methylene linkage, CBF9CBF (Fig. 1(a)), which is
considered to be the active component in inducing the $\mathrm{N_{TB}}$ phase;
and (2) a pure dimer Mehl_M37 ; Tamba  (labeled DTC5C7 and also shown in Fig.
1(a)). The phase sequences of these materials (in cooling) are
isotropic-($77^{\circ}$)-N-($37.4^{\circ}$)-$\mathrm{N_{TB}}$-($22^{\circ}$C)-crystal
[KA(0.2)] and
isotropic-($156.8^{\circ}$)-N-($127.8^{\circ}$)-$\mathrm{N_{TB}}$-($96.6^{\circ}$)-$\mathrm{SmX}$-($77.4^{\circ}$C)-crystal
(DTC5C7). As a control compound, we chose the conventional, rodlike
thermotropic, 4-n-octyl-4’-cyanobiphenyl (8CB).

The liquid crystals were either capillary filled into sandwich cells made of
fused silica (FS) substrates with a 0.1 mm spacer or loaded into FS cuvettes
with a 1 mm path length. In both cases the substrate surfaces were thoroughly
cleaned but otherwise untreated. The loaded cells were placed in a
temperature-regulated oven with optical access on opposite sides and with
$\sim 0.01$ K stability. The oven fitted between the pole faces of a
laboratory electromagnet, with the direction of the field oriented
perpendicular to the substrate normal (Fig. 1(c)). The director
$\hat{\mathbf{n}}$ in the nematic phase or direction $\hat{\mathbf{t}}$,
corresponding to the average of $\hat{\mathbf{n}}$ over the heliconical pitch
in the $\mathrm{N_{TB}}$ phase (see Fig. 1(b)), was magnetically aligned by
slowly cooling from the isotropic phase in a 1.13 T field produced by the
magnet.

The samples were illuminated at normal incidence to the substrates and to the
direction of $\hat{\mathbf{t}}$ with polarized 1064 nm light from a Nd:YAG
laser (Continuum Minilite II, outputting 5-7 ns, $\sim 2$ mJ pulses at 10 Hz).
The beam diameter at the sample was reduced to 1 mm by a fixed aperture. At
the incident pulse energy used, no evidence of damage to the samples was
detected during or after the measurements. The incident polarization could be
rotated between vertical ($V$, perpendicular to $\hat{\mathbf{t}}$) and
horizontal ($H$, parallel to $\hat{\mathbf{t}}$) orientations. The directions
$V$, $H$ also refer to vertical and horizontal with respect to the plane of
the apparatus in Fig. 1(c).

The scattered second harmonic (SH) light from the samples was collected in a
cone of $15^{\circ}$ opening angle around the forward (transmitted light)
direction by a specially constructed optical telescope, which imaged the
scattering pattern onto the sensor of a cooled CCD camera (Princeton
Instruments, model ProEM512). An analyzer was used to set the $V$ and $H$
polarization state of the SH light, while a series of broad and narrow band
filters (the latter having a passband of 1 nm FWHM at 532 nm) removed the
transmitted fundamental light.

The telescope could be interchanged with a polarizing optical microscope for
in-situ examination of the sample textures over the region illuminated by the
laser beam and for confirmation of uniform director alignment. SH scattering
patterns and optical micrographs were recorded for various polarizer/analyzer
combinations at fixed temperatures in the $\mathrm{N_{TB}}$ phase of the
samples.

Figure 2: Polarizing microscope images of parabolic focal conic defect arrays
in the $\mathrm{N_{TB}}$ phase of the studied LC compounds (polarizer along
horizontal direction, analyzer at $10^{\circ}$ angle from horizontal). (a) and
(b): Images taken in the mid-plane of a 0.1 mm thick sample of DTC5C7 at
temperatures $7.8^{\circ}$C and $13.8^{\circ}$C below the unixial nematic to
$\mathrm{N_{TB}}$ transition. (These temperatures correspond to two of the
data sets for DTC5C7 in Fig. 3.) (c): Image taken near the surface of a 1 mm
cell of KA(0.2), $0.9^{\circ}$C below the transition. There are no observable
PFCs and no SH signal (see Fig. 3). (d): At $\sim 2^{\circ}$C below the
transition, PFCs appear and an SH signal is observed. All of the images were
recorded after cooling through the transition in a 1.13 T magnetic field
(horizontal in the figure). Figure 3: Angular distribution of SH light
recorded in the $\mathrm{N_{TB}}$ phase of KA(0.2) and DTC5C7. The thumbnail
images (insets) show the raw data from the CCD detector, color coded for
intensity (light blue representing the background level, dark red representing
peak level). The axis of the applied magnetic field (and average LC director)
is horizontal in the thumbnails. The main figures display plots SH power vs
scattering angle for a horizontal cut (40 pixels wide) through the central
peak. The polarizer and analyzer combinations for the normally incident
fundamental light (wavevector $\mathbf{k}_{\omega}$) and the second harmonic
light collected from the sample are indicated in each panel; $H$ (horizontal)
corresponds to polarization parallel to the average LC director
$\hat{\mathbf{t}}$ and also to the axis through the foci of the parabolae
delineating the PFC defects in Fig. 2, while $V$ (vertical) applies to the
direction perpendicular to $\hat{\mathbf{t}}$ and to $\mathbf{k}_{\omega}$.
First two columns (from left): Data for KA(0.2) in thick (1 mm) and thin (0.1
mm) optical cells: Note the approximately twofold decrease in signal, despite
a tenfold decrease in sample thickness. Data, taken in cooling, are shown for
several temperatures relative to the nematic to $\mathrm{N_{TB}}$ transition:
$-0.9^{\circ}$C (gray), $-3.4^{\circ}$C (blue), $-5.4^{\circ}$C (green), and
$-7.4^{\circ}$C (red points). For the temperatures close to the transition, no
PFCs are observed (Fig. 2), and there is no SH signal. Note that the central
peak is split when the incident (fundamental light) polarization is switched
from $H$ to $V$ (for $V$ SH output), but there is no $H$ output for either $H$
or $V$ input polarization (bottom panels). Third column: Similar results for a
thin cell containing DTC5C7, at temperatures $-3.8^{\circ}$C (gray),
$-7.8^{\circ}$C (blue), $-9.8^{\circ}$C (green), and $-13.8^{\circ}$C (red)
relative to the nematic to NTB transition. Again there is no $H$ polarized SH
output.

##  III Results

###  III.1 Microscopy

First we consider the results from polarizing microscopy displayed in Fig. 2;
the images shown were obtained after cooling the samples in the applied
magnetic field to temperatures below the transition temperature ($T_{TB}$) to
the $\mathrm{N_{TB}}$ phase. In the images, the polarizer and analyzer axes
are approximately parallel to the field. The effective director
$\hat{\mathbf{t}}$ is uniformly oriented along the field, except in the
immediate vicinity of parabolic focal conic (PFC) defects. These are
delineated in Fig. 2 by dark lines tracing out parabolic trajectories, which
are paired together in orthogonal planes such that each parabola in a pair
passes through the other’s focus. The paired parabolae resemble the classic
PFC texture observed in ordinary smectic LCs under a dilatory strain of the
smectic layers Rosenblatt_PFC .

In the images of Fig. 2, typically one parabola lies in the focal plane
(parallel to the cell substrates), while the second of the pair occupies a
plane normal to the viewing direction. The spacing between the two opposite
“arms” within each PFC is $\sim 50\mu$m, while the separation between foci in
the core region is about $6~{}\mu$m. The symmetry axes of the PFCs are well-
aligned, and parallel to the $\mathrm{N_{TB}}$ optical axis
($\hat{\mathbf{t}}$) direction. In the thicker (1 mm) cells, the core regions
of the PFCs could be resolved in two distinct planes close to the opposing
cell boundaries, whereas in the 0.1 mm cells a single focal plane was located
at essentially the mid-plane of the cell. Thus, two layers of PFCs form near
the opposing substrates in sufficiently thick samples, but are squeezed into a
single layer when the cell thickness becomes comparable to the dimensions of
the parabolae.

As Fig. 2(c) reveals, the PFCs are absent at temperatures close to (within
$\sim 1^{\circ}$ of $T_{TB}$); they appear and their population increases in
the range $T_{TB}-T\simeq 1$ to $3^{\circ}$C, and then saturates. By analogy
with ordinary smectics, we interpret the PFCs as defects in a “pseudo-layer”
structure defined by $\sim 10$ nm thick slabs between planes of constant phase
in the heliconical $\mathrm{N_{TB}}$ structure. As the temperature decreases
below $T_{TB}$, the “layer” spacing (pitch $t_{0}$) shrinks, resulting in PFC
formation at a threshold $t_{0}$ just as a critical dilatory strain does in
the case of true smectic layers.

###  III.2 Second harmonic light scattering

We now turn to our results from SHLS measurements in the $\mathrm{N_{TB}}$
phase. These are summarized in Fig. 3. First consider the case where the
incident (fundamental) light is polarized along $\hat{\mathbf{t}}$ and the SH
output is polarized perpendicular to $\hat{\mathbf{t}}$ (an
$H_{\omega}\rightarrow V_{2\omega}$ process, following the nomenclature
introduced above). In both $\mathrm{N_{TB}}$ materials, for $T$ below but
within $1^{\circ}$C of the transition ($T_{TB}$) – the range where PFCs are
absent – no SH output is detected. Below this range and in the presence of
PFCs, a clear SH signal is observed, with a peak in the forward direction. In
the KA(0.2) samples, the signal level increases slightly with decreasing $T$
in both thick and thin samples, whereas the signal approximately doubles for
DTC5C7. The angular FWHM of the peak (using data from the thin cell where one
expects less broadening due to multiple scattering) corresponds to a length
scale of $1.22\lambda_{2\omega}/$FWHM $\sim 7~{}\mu$m, which is comparable to
the spacing between foci of the PFCs. The SH signal is clearly associated with
the appearance of PFCs in the $\mathrm{N_{TB}}$ phase.

For $V$ polarized input and $V$ output ($V_{\omega}\rightarrow V_{2\omega}$
process), the angular distribution of SH light is again concentrated around
the forward direction. However, the peak is split approximately symmetrically
about zero scattering angle (compare the thumbnail images in Fig. 3 showing
the raw SHLS patterns recorded on the CCD). Also, the temperature dependence
is weaker than for the $H_{\omega}\rightarrow V_{2\omega}$ process. On the
other hand, for $H$ polarized output ($H_{\omega}\rightarrow H_{2\omega}$ and
$V_{\omega}\rightarrow H_{2\omega}$ processes), there is no SH signal. Thus,
all detected SH output is polarized perpendicular to the average helical axis
of the TB structure or, equivalently, to the axis of the PFC defects.

The peak SH intensity from the 0.1 mm thick samples of KA(0.2) is
approximately two times lower than in the 1 mm sample. For incoherent SH
generation from the bulk NTB phase, we would normally expect a factor $\sim
10$ decrease. The discrepancy is explained by the SHG originating from the PFC
defects: As noted above, there are two “layers” of PFCs, localized near the
surfaces, in the thick cell, but only one in the thin cell – hence the factor
of $\sim 2$ change observed in SH intensity.

In all cases, the SH signal is quite weak – of order $\sim 10^{-7}$ the level
from a 0.5 mm thick Z-cut single crystal quartz reference. Several factors,
beyond the possibility of an intrinsically small nonlinear susceptibility, may
contribute to this weak level. First, the scattering is incoherent, and there
is no phase matching between the fundamental and second harmonic waves.
Second, the signal comes only from the PFCs, and the region of strong “pseudo-
layer” deformation (defined by the parabolic lines in the images in Fig. 2)
occupies only a small fraction of the total volume illuminated by the
fundamental laser beam. Third, as we shall describe later in the Discussion
section, cancellations in the induced polarization (and effective non-
centrosymmetry) occur both at points on the parabolae and between points on
opposite sides of them, reducing the SH output.

To confirm that the SH signal arises specifically from deformation of
$\mathrm{N_{TB}}$ “pseudo-layers”, we performed similar measurements on the
smectic-A phase of an ordinary rod-like LC, octyl cyanobiphenyl (8CB), which
also exhibits classical PFCs. As shown in Fig. 4, for both 1 mm and 0.1 mm
cells populated with PFC defects of an ordinary smectic layer structure, there
is no detectable SH signal under experimental conditions identical to those
used for the $\mathrm{N_{TB}}$ samples.

Figure 4: Results for 1 and 0.1 mm thick cells of the control sample of 8CB in
the smectic-A phase. No SH output is observed despite the presence of PFCs in
the smectic layer structure (see accompanying microscope images).

##  IV Discussion

Let us now consider mechanisms for second harmonic generation due to the
presence of PFCs, which could potentially account for the experimental results
described in the previous section. Before beginning, it may be useful to
summarize the following key points of the analysis developed in the
subsections below:

  1. (1)

Deformation of the $\mathrm{N_{TB}}$ “pseudo-layers” and the average director,
due to PFCs, distorts the helical polarization field associated with
$\mathrm{N_{TB}}$ order, and gives rise to a net polarization $\mathbf{P}$
that lies principally in the plane perpendicular to the PFC axis.

  2. (2)

In a coarse-grained picture, the process is analogous to the electroclinic
effect in a chiral smectic-A phase: Tilting of the average director (helical
axis) away from the “pseudo-layer” normal induces a $\mathbf{P}$ orthogonal to
the tilt plane (with the direction in this plane depending on the sign of the
helicity).

  3. (3)

The locally broken centrosymmetry, together with the electroclinic analogy,
determine the structure of the second-order nonlinear susceptibility tensor
$\mathbf{\chi}^{(2)}$.

  4. (4)

The mirror symmetry of the “pseudo-layer” displacement through planes
containing the PFC axis and a pair of parabolic “arms” leads to cancellations
of $\mathbf{P}$ between opposing “arms”. While this complicates development of
a quantitative relation between the local magnitude of $\mathbf{P}$ and the
observed SH signal, it still allows an explanation of the main qualitative
features (e.g., polarization selection rules and angular distribution) of the
SH signal.

###  IV.1 Theoretical model for induced polarization

Fig. 5(a) shows a three-dimensional rendering of deformed layers in the core
region of a PFC, which is based upon the geometrical construction given in
Ref. Rosenblatt_PFC . Although this construction strictly applies in the limit
of incompressible layers (i.e., infinite elastic constant for layer
compression), it provides a useful starting point.

In a normal smectic-A, curvature of the layers implies a splay of the director
field, and thus in principle a flexoelectric polarization normal to the layers
and in the direction of
$\hat{\mathbf{n}}(\mathbf{\nabla}\cdot\hat{\mathbf{n}})$ Meyer_flexo .
However, as already pointed out, no SHG from such a mechanism was observed due
to PFCs in the smectic-A phase of the control sample 8CB. For the layer
distortion in a PFC defect, the maximum curvature occurs at the conical
“cusps”, where the layers “pinch” down to points located along the parabolae
(dashed lines in Fig. 5) that define the defect. In order to avoid an infinite
splay energy, the molecules must either rotate off the layer normal (relaxing
the splay) and adopt a more uniform orientation at the cusp, or the vertex of
the cusps must relax from a conical tip (infinite layer curvature) to a bowl
shape (finite curvature). In the former case, the splay flexoelectric
polarization would be significantly reduced, while in the latter case the
curvature of the layer approaching a cusp would have the opposite sign of the
curvature at the bottom of the bowl (that replaces the sharp tip of the cusp),
again tending to cancel the splay flexoelectricity. These effects can explain
the absence of a detectable SH signal in the control sample (ordinary
smectic-A).

In the dimers forming the pseudo-layered $\mathrm{N_{TB}}$ phase, the same
terminal groups on identical mesogenic units are connected to opposite ends of
the flexible spacer; this architecture tends to eliminate an overall
longitudinal molecular dipole, and thus negate the conventional splay
flexoelectric effect at the molecular level. As will be made clear below, the
polarization selection of the SH light observed in our experiment also cannot
be explained by conventional splay flexoelectric polarization, which would
have its major component along the PFC axis ($\hat{\mathbf{z}}$ in Fig. 5).
Moreover, any bend flexoelectricity associated with the equilibrium
(heliconical) NTB structure averages out over a length scale much shorter than
an optical wavelength.

Figure 5: (a): 3D rendering of pseudo-layer distortion surrounding the core
region of a PFC in the limit of infinite layer compression elastic constant.
The two overlapping parabolae (dashed lines) pass through each other’s focus
and lie in orthogonal planes, whose intersection defines the axis of the PFC.
The parabolae are the loci of conical cusps (“pinch” points) in the
deformation of the layer planes. Since these are points of infinite curvature
energy, they will relax into rounded “bowls” in the actual case of finite
compression constant. The PFC axis corresponds to the direction of average
director alignment (applied magnetic field direction $\mathbf{H}$) in our
experiment. An orthogonal direction, perpendicular to the plane of one
parabola, corresponds to the wavevector of the incident (fundamental light),
$\hat{\mathbf{k}}_{\omega}$ and to the forward direction of the transmitted SH
light. (b): Two distorted “pseudolayers”, extracted from the stack in (a) and
located at equal distances ($\pm z_{0}$) from the center ($z=0$) of the PFC.
The components of the induced polarization field $\mathbf{p}$ due to an
imposed layer displacement $u$ along $\hat{\mathbf{z}}$, as predicted from the
model discussed in the text, are shown in the cusp regions (assumed rounded in
the limit of finite layer compression constant). Note in particular that the
model predicts that the component of $\mathbf{p}$ perpendicular to the PFC
axis is substantially larger than that along it. The fundamental and second
harmonic polarization axes ($V$ and $H$) are also indicated; these correspond
to vertical and horizontal directions with respect to the
$\hat{\mathbf{k}}_{\omega}-\mathbf{H}$ plane of the experiment (and not with
respect to those directions along the page.

Thus, we turn to a model that explicitly couples long wavelength distortions
of the heliconical director structure to distortions of a helical vector order
parameter $\mathbf{P}$ (where $\mathbf{P}$ may originate either from a net
transverse dipole moment, or simply from the “shape” polarization, associated
with the bent conformation of LC dimers. It is convenient to introduce a
dimensionless form for this order parameter, $\mathbf{p}=\mathbf{P}/P_{sat}$,
where $P_{sat}$ corresponds to the saturated polarization at low temperature.

The $\mathrm{N_{TB}}$ free energy density may then be expanded in terms of the
uniaxial director field $\hat{\mathbf{n}}$ and the polarization field
$\mathbf{p}$ as Shamid_PRE

$\displaystyle F_{NTB}$ $\displaystyle=$
$\displaystyle\frac{K_{1}}{2}(\nabla\cdot\hat{\mathbf{n}})^{2}+\frac{K_{2}}{2}(\hat{\mathbf{n}}\cdot\nabla\times\hat{\mathbf{n}})^{2}$
$\displaystyle+\frac{K_{3}}{2}[\hat{\mathbf{n}}\times(\nabla\times\hat{\mathbf{n}})]^{2}+\frac{\mu}{2}|\mathbf{p}|^{2}+\frac{\nu}{4}|\mathbf{p}|^{4}$
$\displaystyle+\frac{\kappa}{2}(\nabla\mathbf{p})^{2}-\Lambda[\hat{\mathbf{n}}\times(\nabla\times\hat{\mathbf{n}})]\cdot\mathbf{p}+\eta(\hat{\mathbf{n}}\cdot\mathbf{p})^{2}.$

Here, $K_{1}$, $K_{2}$, and $K_{3}$ are the usual Frank elastic constants for
splay, twist, and bend distortions of the nematic director $\hat{\mathbf{n}}$.
The coefficient $\mu=\mu_{0}(T-T_{0})$ is the temperature-dependent Landau
coefficient for the polarization $\mathbf{p}$ ($\mu_{0}$ being a constant),
while $\nu>0$ is a higher-order, temperature-independent Landau coefficient.
The elastic constant $\kappa$ penalizes spatial distortions in $\mathbf{p}$,
and the coefficient $\Lambda$ couples $\mathbf{p}$ with bend distortions. The
last term (not included in Ref. Shamid_PRE ), with $\eta>0$, penalizes any
component of $\mathbf{p}$ that lies along $\hat{\mathbf{n}}$. Because
$\mathbf{p}$ is defined as a dimensionless quantity, the Landau coefficients
$\mu$ and $\nu$ carry the same units, and $\kappa$ has the same units as the
Frank constants.

In the $\mathrm{N_{TB}}$ phase, the director field has the equilibrium
heliconical modulation
$\hat{\mathbf{n}}=\hat{\mathbf{z}}\cos\beta+\hat{\mathbf{x}}\sin\beta\cos(q_{0}z)+\hat{\mathbf{y}}\sin\beta\sin(q_{0}z)$,
with pitch wavenumber $q_{0}$ and cone angle $\beta$. Likewise, the
polarization field has the equilibrium helical modulation
$\mathbf{p}=\hat{\mathbf{x}}p_{0}\sin(q_{0}z)-\hat{\mathbf{y}}p_{0}\cos(q_{0}z)$,
with magnitude $p_{0}$, perpendicular to $\hat{\mathbf{n}}$ and to the pitch
axis $\hat{\mathbf{z}}$, as shown in Fig. 1(b). In the nematic phase, $\beta$
and $p_{0}$ are both zero while $q_{0}$ is undefined; in the $\mathrm{N_{TB}}$
phase, these quantities all become non-zero.

Let us now consider an imposed spatial variation in the phase, $\Phi$, of the
helicoidal modulation. This corresponds to a displacement, $u=-\Phi/q_{0}$, of
the $\mathrm{N_{TB}}$ “pseudo-layers” along the equilibrium layer normal
$\hat{\mathbf{z}}$. Under the imposed distortion, the layer plane varies in
orientation, and the directions of $\hat{\mathbf{n}}$ and $\mathbf{p}$ also
generally change in order to minimize $F_{NTB}$. To describe such a change in
$\hat{\mathbf{n}}$, we introduce the orthogonal triad of unit vectors
$\hat{\mathbf{e}}_{1}$, $\hat{\mathbf{e}}_{2}$, $\hat{\mathbf{t}}$, where
$\hat{\mathbf{t}}$ is obtained by rotation of $\hat{\mathbf{z}}$ about an
arbitrary axis in the $x$-$y$ plane, and $\hat{\mathbf{e}}_{1}$ and
$\hat{\mathbf{e}}_{2}$ are the transformed $\hat{\mathbf{x}}$ and
$\hat{\mathbf{y}}$ under this rotation. Then the distorted $\hat{\mathbf{n}}$
is

$\displaystyle\hat{\mathbf{n}}$ $\displaystyle=$
$\displaystyle\hat{\mathbf{e}}_{1}\sin\beta\cos(q_{0}z-q_{0}u)+\hat{\mathbf{e}}_{2}\sin\beta\sin(q_{0}z-q_{0}u)+$
(2) $\displaystyle\hat{\mathbf{t}}\cos\beta.$

The rotated coordinate unit vectors may be expressed in the $xyz$ system as

$\displaystyle\hat{\mathbf{t}}=t_{x}\hat{\mathbf{x}}+t_{y}\hat{\mathbf{y}}+t_{z}\hat{\mathbf{z}},$
$\displaystyle\hat{\mathbf{e}}_{1}=\frac{t_{y}^{2}+t_{x}^{2}t_{z}}{t_{x}^{2}+t_{y}^{2}}\hat{\mathbf{x}}+\frac{t_{x}t_{y}(-1+t_{z})}{t_{x}^{2}+t_{y}^{2}}\hat{\mathbf{y}}-t_{x}\hat{\mathbf{z}},$
$\displaystyle\hat{\mathbf{e}}_{2}=\frac{t_{x}t_{y}(-1+t_{z})}{t_{x}^{2}+t_{y}^{2}}\hat{\mathbf{x}}+\frac{t_{x}^{2}+t_{y}^{2}t_{z}}{t_{x}^{2}+t_{y}^{2}}\hat{\mathbf{y}}-t_{y}\hat{\mathbf{z}},$
(3)

where $t_{z}=\sqrt{1-t_{x}^{2}-t_{y}^{2}}$, and $t_{x}$, $t_{y}$ are the $x$,
$y$ components of $\hat{\mathbf{t}}$.

Since $\mathbf{p}$ is not a unit vector and need not, in general, remain
orthogonal to $\hat{\mathbf{n}}$ under the imposed distortion, we write,

$\displaystyle\mathbf{p}$ $\displaystyle=$
$\displaystyle\hat{\mathbf{e}}_{1}p_{0}\sin(q_{0}z-q_{0}u)-\hat{\mathbf{e}}_{2}p_{0}\cos(q_{0}z-q_{0}u)+$
(4)
$\displaystyle+p_{x}\hat{\mathbf{x}}+p_{y}\hat{\mathbf{y}}+p_{z}\hat{\mathbf{z}}.$

Here $p_{x}$, $p_{y}$, $p_{z}$ are the $xyz$ components of the distortion in
$\mathbf{p}$ that does not remain perpendicular to $\hat{\mathbf{n}}$.

The calculation of these components for an imposed “pseudo-layer” displacement
$u=u(x)$, which varies along an axis in the equilibrium layer plane (and
therefore describes layer bending), and for $u=u(z)$, which varies along the
layer normal (describing layer compression), is outlined in the Appendix. For
simplicity, the calculation is done only to linear order in
$\\{p_{x},p_{y},p_{z},t_{x},t_{y}\\}$.

The results, assuming large $\eta$ Parsouzi_PRX  and small $\beta$
($\beta\lesssim 10^{\circ}$ close to the $\mathrm{N_{TB}}$-N transition
Borshch_Nature ; CMeyer3 , are:

$\displaystyle p_{x}\approx-\frac{\Lambda q_{0}}{2\eta}t_{y},$ (5)
$\displaystyle p_{y}\approx\frac{\Lambda
q_{0}}{2\eta}\left(t_{x}+\frac{du}{dx}\right),$ (6) $\displaystyle
p_{z}\approx\frac{\Lambda\beta^{2}}{4\eta}\left(\frac{dt_{x}}{dx}-q_{0}t_{y}\frac{du}{dx}\right),$
(7)

for $u=u(x)$, and

$\displaystyle\mathbf{p}_{\perp}\approx-\frac{\sqrt{2}\Lambda}{\eta\beta^{2}}\frac{d\mathbf{t}_{\perp}}{dz},$
(8) $\displaystyle p_{z}\approx 0,$ (9)

for $u=u(z)$. Here
$\mathbf{p}_{\perp}=p_{x}\hat{\mathbf{x}}+p_{y}\hat{\mathbf{y}}$ and similarly
for $\mathbf{t}_{\perp}$. The approximate equality signs in Eqs. (5)-(9)
reflect the approximation associated with our linear analysis in
$\\{p_{x},p_{y},p_{z},t_{x},t_{y}\\}$.

Now let us apply these results to the cusp region of a PFC observed in the
$\mathrm{N_{TB}}$ phase. We employ them with the understanding that they are
based on a linear analysis, and thus only represent leading order
contributions to $\mathbf{p}$ in the case of “pseudo-layer” distortion
associated with a PFC. We also assume that the size of the cusp region is
small compared to the size of a homochiral domain (i.e., domain with single
handedness of the underlying heliconical director structure) CMeyer2 , so that
the polarization $\mathbf{p}$ is not washed out in the cusp region due to
mixed left and right-handed helical domains.

As mentioned above and particularly in the case of the $\mathrm{N_{TB}}$
phase, where the “pseudo-layer” compression elastic constant is substantially
lower than that for a typical smectic LC Challa ; CMeyer1 , the sharp tips of
the cusps should realistically be envisioned as rounded caps. This also means
that in addition to “pseudo layer” bending, there will be a compression of the
layers in the cusp region. However, as described in the Appendix, for
$du/dz\ll 1$, the Euler-Lagrange equation for $\mathbf{t}_{\perp}$ gives
$\mathbf{t}_{\perp}=0$ to lowest order, and in that case Eqs. (8) and (9)
imply $\mathbf{p}\approx 0$ for “pseudo-layer” compression. Therefore, we will
focus on the effect of layer bending.

Consider then a pair of distorted layers equidistant from the center of the
PFC, at $+z_{0}$ and $-z_{0}$, along the $z$ axis, as shown in Fig. 5(b), and
consider first the behavior in a slice taken through the cusp in the $x-z$
plane for the layer at $-z_{0}$. Along $x$, the displacement $u$ is asymmetric
through the cusp region and thus $t_{x}$ will be also; this will lead to a net
$p_{y}$ and $p_{z}$, according to Eqs. (6)–(7). On the other hand, for
$u=u(x)$, $t_{y}=0$ to lowest order (see Appendix), and thus Eq. (5) implies
$p_{x}\approx 0$ in the $-z_{0}$ layer. On opposite sides of the $x$ axis,
i.e., $x\rightarrow-x$, the direction of $p_{y}$ is reversed.

Next, for a slice taken in the $y-z$ plane, we may interchange $x$ and $y$ in
Eqs. (5)–(7); however, the mirror symmetry across this plane means that only
the $p_{z}$ component of the polarization will survive. The net components of
$\mathbf{p}$ in the cusp region are shown qualitatively in Fig. 5(b); the
relative magnitudes shown for $p_{y}$ and $p_{z}$ will be explained shortly.

For the layer at $+z_{0}$, the results are the same as for $-z_{0}$, except
for a $90^{\circ}$ rotation around the $z$ axis, equivalent to an exchange of
$p_{x}$ and $p_{y}$ and a reversal in the direction of $p_{z}$ (see Fig.
5(b)).

The relative magnitudes of $p_{y}$ and $p_{z}$ in the $-z_{0}$ layer (or
$p_{x}$ and $p_{z}$ in the $+z_{0}$ layer) may be estimated if we consider a
harmonic form for $u$ – specifically, a Fourier component whose wavenumber
$q_{x}=2\pi/\delta$ corresponds to the length scale of the cusp region of the
PFC. That scale can be estimated Rosenblatt_PFC  from the width $\delta\simeq
2\mu$m of the birefringent arms of the parabolae (the loci of PFC cusps) in
the textures presented in Fig. 2. Then taking $u=u(x)=u_{0}\exp(iq_{x}x)$, the
Euler Lagrange equation for $t_{x}$ yields (see Appendix)

$t_{x}\approx-\frac{K_{3}q_{0}^{2}\beta^{2}}{K_{1}q_{x}^{2}+K_{3}q_{0}^{2}\beta^{2}}\frac{du}{dx}.$
(10)

Typically, $\beta=0.18$ rad, $q_{0}=2\pi/(0.01\mu\mathrm{m})$, and
$q_{x}=2\pi/(2\mu\mathrm{m})$, so $q_{x}^{2}\ll q_{0}^{2}\beta^{2}$. Then the
right hand side of the expression for $t_{x}$ may be expanded, yielding

$t_{x}\approx\left(-1+\frac{K_{1}q_{x}^{2}}{K_{3}q_{0}^{2}\beta^{2}}\right)\frac{du}{dx}$
(11)

Inserting this result into Eqs. (6) and (7) with $u=u_{0}\exp(iq_{x}x)$, we
obtain to lowest order

$\displaystyle p_{y}$ $\displaystyle\approx$ $\displaystyle
i\frac{\Lambda}{2\eta}\frac{K_{1}}{K_{3}}\frac{q_{x}^{3}}{q_{0}\beta^{2}}u,$
(12) $\displaystyle p_{z}$ $\displaystyle\approx$
$\displaystyle\frac{\Lambda}{4\eta}\beta^{2}q_{x}^{2}u.$ (13)

Then, using the numerical values given above, and taking $K_{1}\simeq 2K_{3}$
(for the bare nematic values) Borshch_Nature , we estimate

$\left|\frac{p_{z}}{p_{y}}\right|\approx\frac{\beta^{4}}{2}\frac{K_{3}}{K_{1}}\frac{q_{0}}{q_{x}}\approx
0.05$ (14)

Since the SH signal should scale as the square of $\mathbf{p}$ (assuming the
components of the second order susceptibility $\mathbf{\chi}^{(2)}$ scale with
the magnitude of $\mathbf{p}$), the contribution from $p_{z}$ is negligible. A
similar result applies to the layer at $+z_{0}$. In the following subsection,
we will therefore concentrate only on the component of $\mathbf{p}$
perpendicular to the axis of the PFC.

###  IV.2 Nonlinear susceptibility in the $\mathrm{N_{TB}}$ phase

Although SHG could possibly arise due to chiral symmetry breaking in the
undistorted $\mathrm{N_{TB}}$ state, this structural chirality averages out
over a molecular length scale (the heliconical pitch) that is much shorter
than the optical wavelength. We detected SH signal from our samples only in
the presence of pseudo-layer distortion specifically associated with PFCs. The
situation is similar to the chiral smectic-C* phase, where the helical
polarization must be partly unwound by an applied field in order to observe a
signal FLC_NLO .

Therefore, we propose that the broken centrosymmetry produced by the induced
polarization $\mathbf{p}$, over length scales comparable to or longer than the
optical scale, is responsible for the observed SHG. As shown in Fig. 5(b), and
based on the arguments above, the dominant component of the distortion-induced
polarization is normal to the axis of the PFC and to the direction of the
applied magnetic field, both of which are along $\hat{\mathbf{z}}$. Eq. (6)and
the discussion above reveal that the polarization arises essentially from an
electroclinic effect, where the coarse-grain director tilts with respect to
the pseudo-layer normal and the polarization develops along the axis
perpendicular to the tilt plane. Indeed, for small distortions, the tilt angle
is given by the sum $t_{x}+du/dx$ in Eq. (6).

The local symmetry is therefore that of a tilted chiral smectic – namely,
$C_{2}$ symmetry, with symmetry axis ($\mathbf{p}$) perpendicular to the axis
of the PFC ($\hat{\mathbf{z}}$). As evidenced in Fig. 2, the plane of one
parabola of the PFCs is oriented parallel to the substrates. We take this as
the $y-z$ plane and the incident (fundamental) light propagation direction as
$\hat{\mathbf{k}}_{\omega}=\hat{\mathbf{x}}$. Then the polarizations $V$ or
$H$ of the fundamental or second harmonic waves are $V=\hat{\mathbf{y}}$ and
$H=\hat{\mathbf{z}}$, and the second harmonic intensities for the various
polarization combinations are,

$\displaystyle I(H_{\omega}\rightarrow
V_{2\omega})\propto|\chi_{VHH}^{(2)}|^{2}=|\chi_{yzz}^{(2)}|^{2},$
$\displaystyle I(H_{\omega}\rightarrow
H_{2\omega})\propto|\chi_{HHH}^{(2)}|^{2}=|\chi_{zzz}^{(2)}|^{2},$
$\displaystyle I(V_{\omega}\rightarrow
H_{2\omega})\propto|\chi_{HVV}^{(2)}|^{2}=|\chi_{zyy}^{(2)}|^{2},$
$\displaystyle I(V_{\omega}\rightarrow
V_{2\omega})\propto|\chi_{VVV}^{(2)}|^{2}=|\chi_{yyy}^{(2)}|^{2}.$ (15)

Here $\mathbf{\chi}^{(2)}$ is the second order susceptibility for $C_{2}$
symmetry, with symmetry axis = $\hat{\mathbf{y}}$ ($-z_{0}$ layer in Fig.
5(b)) or $\hat{\mathbf{x}}$ ($+z_{0}$ layer). Assuming Kleinman’s symmetry,
the non-vanishing components of $\mathbf{\chi}^{(2)}$ with recurring indices
are Boyd_NLO ,

$\chi_{xxy}^{(2)}=\chi_{xyx}^{(2)}=\chi_{yxx}^{(2)}~{},~{}\chi_{zzy}^{(2)}=\chi_{zyz}^{(2)}=\chi_{yzz}^{(2)}~{},~{}\chi_{yyy}^{(2)}$

for $-z_{0}$ layers, and

$\chi_{yyx}^{(2)}=\chi_{yxy}^{(2)}=\chi_{xyy}^{(2)}~{},~{}\chi_{zzx}^{(2)}=\chi_{zxz}^{(2)}=\chi_{xzz}^{(2)}~{},~{}\chi_{xxx}^{(2)}$

for $+z_{0}$ layers. From these lists and the relations for SH intensity in
Eq. (15), we conclude that the intensity for $H$ polarized output vanishes, in
agreement with the polarization selection observed in our experiment, and that
the SH signal is generated predominantly by the induced polarization in the
$-z_{0}$ layers (where $\mathbf{p}\parallel\hat{\mathbf{y}}$) in Fig. 5(b).

Fig. 3 reveals that the SH intensity is systematically lower for a
$V_{\omega}\rightarrow V_{2\omega}$ process than for $H_{\omega}\rightarrow
V_{2\omega}$, indicating $|\chi_{yzz}^{(2)}|^{2}>|\chi_{yyy}^{(2)}|^{2}$
according to Eq. (15). We can then suggest the following explanation for the
splitting of the forward peak in the $V_{\omega}\rightarrow V_{2\omega}$ data:
In linear light scattering, we have observed strong depolarized scattering
from defect structure at small angles in the $\mathrm{N_{TB}}$ phase. Consider
then a process by which small-angle linear scattering converts polarization
$V_{\omega}$ to $H_{\omega}$, and then a SH process that converts $H_{\omega}$
to $V_{2\omega}$. This could lead to a slight splitting of the forward peak,
as observed in Fig. 3 for the $V_{\omega}\rightarrow V_{2\omega}$ data. On the
other hand, the sequence $H_{\omega}\rightarrow V_{\omega}\rightarrow
V_{2\omega}$ would be less efficient, producing perhaps a broadening but not a
splitting of the peak in the $H_{\omega}\rightarrow V_{2\omega}$ data.
Moreover, the peak for $V_{\omega}\rightarrow V_{2\omega}$ splits along the
magnetic-field alignment direction – i.e., parallel to the axis of the PFCs.
This is because the distortion-induced polarization near the core region will
be concentrated more along the PFC axis than normal it.

Finally, we note that the major components of the induced polarization are
oppositely directed on opposite sides of the parabola axis (Fig. 5(b)), even
if the entire PFC is in a homochiral domain. In this case, there will be no
net SH output when the distance across the parabola yields a destructive
interference condition, or when the vector sum of polarizations cancels out
over a length scale small compared to the coherence length
$\lambda_{2\omega}/(n_{2\omega}-n_{\omega})\simeq 5~{}\mu$m ($n=$ refractive
index) Boyd_NLO . As mentioned in the Results section, this effect accounts
for the relatively weak signal recorded in the experiment. The phase factor is
important, and consequently the signal level is not a direct measure of the
amplitude of the helical polarization field.

##  V Conclusion

In this paper, we have used second harmonic light scattering to probe polar
molecular organization in the nematic twist-bend phase. The SH signal arises
from parabolic focal conic defects in the “pseudo-layer” structure of the
$\mathrm{N_{TB}}$ state. Its key features can be explained by a coarse-grained
free energy that couples the “layer” deformation to long wavelength
distortions of a helical polarization wave, which serves as the
$\mathrm{N_{TB}}$ order parameter. In the future, we hope to employ focused
fundamental radiation in order to generate SH light from specific regions of
an individual PFC defect in a thin sample. This approach, combined with a more
detailed treatment of the PFC structure, could provide a direct measure of the
magnitude of the $\mathrm{N_{TB}}$ polarization field.

###### Acknowledgements.

We are indebted to our colleague Prof. O. D. Lavrentovich for several
informative conversations, and to Dr. Owain Parri for providing us with the
material KA(0.2) used in this study. This work was supported by the NSF under
grants DMR-1307674 (SP, JG, AJ, SS) and DMR-1409658 (JVS); the EPSRC under
grant EP/J004480/1 (GM and CW); and the EU under project 216025 (MGT).

## Appendix: Induced polarization due to “pseudo-layer” distortion

In this appendix, we sketch the details leading to Eqs. (5)–(10) of the text.
We first consider a “pseudo-layer” displacement $u$ that varies along a single
direction in the plane of the equilibrium “pseudo-layers” – e.g., $u=u(x)$.
This corresponds to a bending of the layers without compression along the
average layer normal ($\hat{\mathbf{z}}$).

Expressions for the induced $\\{p_{x},p_{y},p_{z},t_{x},t_{y}\\}$ that
minimize the free energy under the imposed $u(x)$ may be obtained by the
following procedure: (1) Combine Eqs. (2)–(4) in the text and insert the
result into the free energy density, Eq. (1); (2) “Coarse-grain” the free
energy density by averaging with respect to $z$ over a helicoidal pitch,
$2\pi/q_{0}$, based on the assumption that $u$ and the variables
$\\{p_{x},p_{y},p_{z},t_{x},t_{y}\\}$ vary slowly over the length scales
comparable to the pitch; (3) Obtain and solve the Euler-Lagrange equations for
the set $\\{p_{x},p_{y},p_{z},t_{x},t_{y}\\}$ that minimize the coarse-grained
free enrgy density. In step (2), the coarse-grained $F$ is evaluated as
$F_{cg}(\hat{\mathbf{t}},u,\mathbf{p};x)=\frac{q_{0}}{2\pi}\int_{0}^{2\pi/q_{0}}F(\hat{\mathbf{n}},\mathbf{p};x,z)\,dz$.
The assumption on relative length scales is reasonable, since the pitch is on
the molecular scale, while the features associated with the PFC textures
observed in the $\mathrm{N_{TB}}$ phase have sizes $\gtrsim 1\mu$m.

After implementing this program, and assuming that terms containing the
elastic constant $\kappa$ are neglgible (i.e., long wavelength limit for the
distortion $u(x)$ and small polarization elastic constant $\kappa$
Parsouzi_PRX , we find to lowest order in
$\\{p_{x},p_{y},p_{z},t_{x},t_{y}\\}$,

$\displaystyle p_{x}$ $\displaystyle=$ $\displaystyle-\frac{\Lambda
q_{0}\sin^{2}\beta}{2(\mu+2\nu p_{0}^{2}+\eta\sin^{2}\beta)}t_{y},$ (16)
$\displaystyle p_{y}$ $\displaystyle=$ $\displaystyle\frac{\Lambda
q_{0}\sin^{2}\beta}{2(\mu+2\nu
p_{0}^{2}+\eta\sin^{2}\beta)}\left(t_{x}+\frac{du}{dx}\right),$ (17)
$\displaystyle p_{z}$ $\displaystyle=$
$\displaystyle\frac{\Lambda\sin^{2}\beta}{2(2\eta\cos^{2}\beta+\mu+\nu
p_{0}^{2})}\times$ (18)
$\displaystyle\left(\frac{dt_{x}}{dx}-q_{0}t_{y}\frac{du}{dx}\right).$

For $u=u(x)$, $\\{p_{x},p_{y},p_{z},t_{x},t_{y}\\}$ are functions of $x$.
Taking the small $\beta$ limit of Eqs. (A1)-(A3) yields Eqs. (5)-(7) of the
text.

To linear order in $\\{t_{x},t_{y}\\}$, the Euler-Lagrange equations that must
be solved to obtain $\hat{\mathbf{t}}$ in terms of a given $u(x)$, and thus
complete the solution for $\mathbf{p}$, are

$\displaystyle-\frac{A_{x}}{2}\frac{d^{2}t_{x}}{dx^{2}}+\frac{B_{x}}{2}\left(t_{x}+\frac{du}{dx}\right)=0,$
(19)
$\displaystyle-\frac{A_{y}}{2}\frac{d^{2}t_{y}}{dx^{2}}+\frac{B_{y}}{2}t_{y}=0,$
(20)

where the coefficients (together with their small $\beta$ limits) are given by

$\displaystyle A_{x}$ $\displaystyle=$ $\displaystyle K_{1}(\cos
2\beta+1)+\frac{K_{2}}{32}(3-4\cos 2\beta+\cos 4\beta)+$ (21)
$\displaystyle\frac{K_{3}}{32}(28+4\cos 2\beta)\sin^{2}\beta-$
$\displaystyle\frac{\Lambda^{2}\sin^{4}\beta}{2(2\eta\cos^{2}\beta+\mu+\nu
p_{0}^{2})}$ $\displaystyle\approx$ $\displaystyle 2K_{1},$ $\displaystyle
B_{x}$ $\displaystyle=$ $\displaystyle B_{y}=\Lambda p_{0}q_{0}\sin
2\beta+\frac{q_{0}^{2}\sin^{2}\beta}{2}\biggl{[}2K_{1}+$ (22)
$\displaystyle(3\cos 2\beta-1)K_{2}-\frac{\Lambda^{2}\sin^{2}\beta}{2(\mu+2\nu
p_{0}^{2}+\eta\sin^{2}\beta)}\biggr{]}$ $\displaystyle\approx$ $\displaystyle
2\Lambda p_{0}q_{0}\beta,$ $\displaystyle A_{y}$ $\displaystyle=$
$\displaystyle\frac{41+20\cos 2\beta+3\cos 4\beta}{32}K_{2}+$ (23)
$\displaystyle\frac{7-4\cos 2\beta-3\cos 4\beta}{32}K_{3}$
$\displaystyle\approx$ $\displaystyle 2K_{2}.$

Note that Eq. (20) is independent of the imposed distortion $u$. Thus, to
linear order the only physically acceptable solution is $t_{y}=0$, implying
$p_{x}=0$ in Eq. (16).

Next we consider a “pseudo-layer” displacement that varies along the axis
normal to the equilibrium layer planes – i.e., a layer compression, with
$u=u(z)$. Carrying through a similar analysis as above, we find to lowest
order in
$\\{\mathbf{p}_{\perp}=p_{x}\hat{\mathbf{x}}+p_{y}\hat{\mathbf{y}}~{},~{}\mathbf{t}_{\perp}=t_{x}\hat{\mathbf{x}}+t_{y}\hat{\mathbf{y}}\\}$,
and neglecting terms containing $\kappa$,

$\displaystyle\mathbf{p}_{\perp}=-\frac{\sqrt{2}\Lambda\cos^{2}\beta}{\mu+2\nu
p_{0}^{2}+\eta\sin^{2}\beta}\frac{d\mathbf{t}_{\perp}}{dz},$ (24)
$\displaystyle p_{z}=0,$ (25)

Here $\\{p_{\perp},t_{\perp}\\}$ depend on $z$. Eq. (8) in the text
corresponds to the large $\eta\sin^{2}\beta$ and small $\beta$ limit of Eq.
(24).

The Euler-Lagrange equation for $\mathbf{t}_{\perp}$ (linearized in
$\mathbf{t}_{\perp}$) is

$-\frac{A_{\perp}}{2}\frac{d^{2}\mathbf{t}_{\perp}}{dz^{2}}+\left(B_{\perp}+C_{\perp}\frac{du}{dz}\right)\left(1-\frac{du}{dz}\right)\mathbf{t}_{\perp}=0,$
(26)

where

$\displaystyle A_{\perp}$ $\displaystyle=$ $\displaystyle
K_{1}+\frac{K_{2}}{4}+\frac{7K_{3}}{4}+(2K_{3}-K_{1})\times$ (27)
$\displaystyle\cos 2\beta+\frac{(K_{3}-K_{2})\cos 4\beta}{4}+$
$\displaystyle\frac{2\Lambda^{2}\cos^{4}\beta}{\mu+2\nu
p_{0}^{2}+\eta\sin^{2}\beta},$ $\displaystyle B_{\perp}$ $\displaystyle=$
$\displaystyle C_{\perp}+\Lambda p_{0}q_{0}\sin 2\beta,$ (28) $\displaystyle
C_{\perp}$ $\displaystyle=$
$\displaystyle-2q_{0}^{2}\sin^{2}\beta\biggl{[}K_{1}+\frac{3\cos
2\beta-1}{2}\times$ (29) $\displaystyle(K_{2}-K_{3})\biggr{]}.$

For $du/dz\ll 1$, Eq. (26) reduces to
$-(A_{\perp}/2)d^{2}\mathbf{t}_{\perp}/dz^{2}+B_{\perp}\mathbf{t}_{\perp}=0$,
the only physical solution to which is $\mathbf{t}_{\perp}=0$.

Finally, if we consider a simple harmonic form for $u(x)$,
$u(x)=u_{0}\exp(iq_{x}x)$, then Eqs. (19), (21) and (22) yield (for small
$\beta$),

$t_{x}=-\frac{p_{0}q_{0}\Lambda\beta}{K_{1}q_{x}^{2}+p_{0}q_{0}\Lambda\beta}\frac{du}{dx}$
(30)

Since for small $\beta$ the polarization magnitude $p_{0}$ in the
$\mathrm{N_{TB}}$ phase is Parsouzi_PRX  $p_{0}\approx
K_{3}q_{0}\beta/\Lambda$, Eq. (30) produces Eq. (10) of the text.

## References

  * (1) R. B. Meyer, Phys. Rev. Lett. 22, 918 (1969).
  * (2) M. Cestari, S. Diez-Berart, D. A. Dunmur, A. Ferrarini, M. R. de la Fuente, D. J. B. Jackson, D. O. Lopez, G. R. Luckhurst, M. A. Perez-Jubindo, R. M. Richardson, J. Salud, B. A. Timimi, and H. Zimmermann, Phys. Rev. E 84, 031704 (2011).
  * (3) V. Borshch, Y.-K. Kim, J. Xiang, M. Gao, A. Jakli, V. P. Panov, J. K. Vij, C. T. Imrie, M. G. Tamba, G. H. Mehl, and O. D. Lavrentovich, Nat. Commun. 4, 2635 (2013).
  * (4) D. Chen, J. H. Porada, J. B. Hooper, A. Klittnick, Y. Shen, M. R. Tuchband, E. Korblova, D. Bedrov, D. M. Walba, M. A. Glaser, J. E. Maclennan, and N. A. Clark, Proc. Natl. Acad. Sci. USA 110, 15931 (2013).
  * (5) R. B. Meyer in Molecular Fluids. Les Houches Lectures, 1973, R. Balian and G. Weill, eds (Gordon and Breach, 1976), pp. 271-343.
  * (6) C. Meyer, G. R. Luckhurst, and I. Dozov. J. Mater. Chem. C 3, 318 (2015).
  * (7) I. Dozov, Europhys. Lett. 56, 247 (2001).
  * (8) C. Meyer, G. R. Luckhurst, and I. Dozov, Phys. Rev. Lett. 111, 067801 (2013).
  * (9) S. M. Shamid, S. Dhakal, and J. V. Selinger, Phys. Rev. E 87, 052503 (2013).
  * (10) E. I. Kats and V. V. Lebedev, JETP Lett. 100, 110 (2014).
  * (11) E. Virga, Double-well elastic theory for twist-bend nematic phases, Phys. Rev. E 89, 052502 (2014).
  * (12) G. Barbero, L. R. Evangelista, M. P. Rosseto, R. S. Zola, and I Lelidis, Elastic continuum theory: Toward understanding of the twist-bend nematic phases, Phys. Rev. E 92, 0350501(R) (2015).
  * (13) C. Greco, G. R. Luckhurst, and A. Ferrarini, Molecular geometry, twist-bend nematic phases and unconventional elasticity: a generalized Maier-Saupe theory, Soft Matter 10, 9318 (2014).
  * (14) I. Lelidis and G. Barbero, Nematic phases with spontaneous splay-bend deformation: standard elastic description, Liq. Cryst. 43, 208 (2016).
  * (15) A. G. Vanakaras and D. J. Photinos, Soft Matter 12, 2208 (2016).
  * (16) K. Adlem, M. Copic, G. R. Luckhurst, A. Mertelj, O. Parri, R. M. Richardson, B. D. Snow, B. A. Timimi, R. P. Tuffin, and D. Wilkes, Phys. Rev. E 88, 022503 (2013).
  * (17) Z. Parsouzi.A.Sh, S. M. Shamid, V. Borshch, P. K. Challa, M. G. Tamba, C. Welch, G. H. Mehl, J. T. Gleeson, A. Jakli, O. D. Lavrentovich, D. W. Allender, J. V. Selinger, S. Sprunt, submitted to Phys. Rev. X (2016).
  * (18) V. P. Panov, M. Nagaraj, J. K. Vij, Yu. P. Panarin, A. Kohlmeier, M. G. Tamba, R. A. Lewis, and G. H. Mehl, Phys. Rev. Lett. 105, 167801 (2010).
  * (19) Z. Ahmed, C. Welch, and G. H. Mehl, RSC Adv. 5, 93513 (2015).
  * (20) A similar approach was recently utilized to study flexoelectric polarization induced by topological defects in ordinary nematics – see Shokir A. Pardaev, J. C. Williams, R. J. Twieg, A. Jakli, J. T. Gleeson, B. Ellman, and S. Sprunt, Phys. Rev. E 91, 032501 (2015).
  * (21) P.-G. de Gennes and J. Prost, The Physics of Liquid Crystals, second edition (Oxford, 1993), section 6.3.1.3.
  * (22) T. C. Lubensky, Phys. Rev. A 6, 452 (1972).
  * (23) L. Radzihovsky and T. C. Lubensky, Phys. Rev. E 83, 051701 (2011).
  * (24) S. M. Salili, C. Kim, S. Sprunt, J.T. Gleeson, O. Parri, A. Jakli, RSC Advances 4, 57419 (2014).
  * (25) P. K. Challa, V. Borshch, O. Parri, C. T. Imrie S.N. Sprunt, J.T. Gleeson, O.D. Lavrentovich, and A. Jakli, Phys. Rev. E. 89, 060501(R) (2014).
  * (26) M. G. Tamba, S. M. Salili, C. Zhang, A. J kli, G. H. Mehl, R. Stannarius, and A. Eremin, RSC Adv. 5, 11207 (2015).
  * (27) C. S. Rosenblatt, R. Pindak, N. A. Clark, R. B. Meyer, Journal de Physique, 38, 1105 (1977).
  * (28) P. K. Challa, V. Borshch, O. Parri, C. T. Imrie, S. N. Sprunt, J. T. Gleeson, O. D. Lavrentovich, and A. Jakli, Phys. Rev. E 89, 060501(R) (2014).
  * (29) C. Meyer and I. Dozov, Soft Matter 12, 574 (2016).
  * (30) M. I. Barnik, L. M. Blinov, and N. M. Shtykov, JETP 86, 1681 (1984).
  * (31) R. W. Boyd, Nonlinear Optics, Third edition (Academic Press, Burlington, MA, 2008), Chapter 2.


# Empirical Study of Straggler Problem in Parameter Server on Iterative
Convergent Distributed Machine Learning

Benjamin Wong Benjamin Wong is with Computer Science, Chinese University of
Hong Kong, e-mail<EMAIL_ADDRESS>(Corresponding author).

###### Abstract

The purpose of this study is to test the effectiveness of current straggler
mitigation techniques over different important iterative convergent machine
learning(ML) algorithm including Matrix Factorization (MF), Multinomial
Logistic Regression (MLR), and Latent Dirichlet Allocation (LDA) . The
experiment was conducted to implemented using the FlexPS system, which is the
latest system implementation that employ parameter server architecture [1],
[2]. The experiment employed the Bulk Synchronous Parallel (BSP) computational
model to examine the straggler problem in Parameter Server on Iterative
Convergent Distributed Machine Learning. Moreover, the current research
analyzes the experimental arrangement of the parameter server strategy
concerning the parallel learning problems by injecting universal straggler
patterns and executing latest mitigation techniques. The findings of the study
are significant in that as they will provide the necessary platform for
conducting further research into the problem and allow the researcher to
compare different methods for various applications. The outcome is therefore
expected to facilitate the development of new techniques coupled with new
perspectives in addressing this problem.

##  I Introduction

###  I-A The Straggler Problem

Different applications employ iterative convergent algorithms, especially
machine learnings. Parallel execution of such algorithms is typically
expressed through the Bulk Synchronous Parallel (BSP) model for computation
[3]. Nonetheless, implementation through this model is characterized by what
is referred as the straggler problem in computing terminology. [4] This means
that every brief delay for a given worker can result in a slowdown of the
whole process or iteration. This problem significantly reduces the overall
completion time and performance, and the problem gets worse with increased
parallelism. The diagram below is a visual representation of the straggler
problem.

Figure 1: The Straggler Problem

The training time of a single iteration is estimated depending on the worker
with the longest computation time and communication time on each iteration, as
expressed in the following:

$\displaystyle t_{iteration}=\sum\left(max_{1\leq i\;\leq
n\;}Comp\left(i\right)+max_{1\leq i\leq n}Comm(i)\right)$

where $n$ is the number of workers, and $Compute(i)$ and $Communicate(i)$ are
computation time and communication time of the worker $i$ respectively.

We can see that any delay in computation or communication of any worker may
delay the overall process time. We can further derive the wait time, as
expressed in the following:

$\displaystyle t_{waste}=\;{\sum_{i=0}^{n}{t_{waste}(i)}}$

Where $n$ is the number of workers, and $t_{waste}(i)$ is the time to wait
synchronization barrier of the worker $i$, i.e. wasted time. To address the
straggler problem, we have to minimize the total waste time, $t_{waste}$.

###  I-B Mitigate the Straggler Problem

Stragglers problem have long overwhelmed parallel computing, and numerous
technique have been researched to moderate them. Following table summarize
current techniques for mitigate straggler problem.

TABLE I:  Mitigation Type |  Implementations
---|---
Speculative execution |  ML-NA [5], FlexRR [6]
Task Cloning |  Wrangler [7],Full Clone [8]
Job Shedding and Job Stealing |  FlexRR
Relaxed progress synchronization |  Stale Synchronization Parallel (SSP) [9]

Summary of Current mitigation techniques

In addressing the straggler problem, new significant data structures have
adopted different reactive or preventive speculation approaches through which
a system initiates backup or extra copies for tasks on other machines via a
prudent manner. Currently, there are 4 popular type of straggler mitigation
methods, these including the Speculative execution, task cloning, Job Shedding
and Job Stealing and Relaxed Progress Synchronization.

In the Speculative execution, the development of distinct tasks is examined
through the system as well as backup copies when the stragglers are detected.
In the cloning approach, on the other hand, extra task copies are organized in
a parallel manner with the first job if only the anticipated cost involved in
computation is low and the availability of system resources is guaranteed. In
the Job Shedding and Job Stealing approach, the system adaptively rebalance
work among the workers. In Relaxed progress synchronization approach, the
system allow certain level of inconsistency of synchronization and retain the
convergence guarantee.

##  II Current Straggler Mitigation Technique

###  II-A Speculative execution & task cloning (Predict Straggler and task
cloning)

Research on large scale computing clusters has shown that the completion time
of a particular task [10]is often unnecessarily delayed by one or a few
“stragglers” or straggling task that are assigned with either a failing or
overloaded server, but definitely there may have a better choice within the
cluster. This finding triggered the study of speculative execution so that to
diminish stragglers in the processing of data systems like Hadoop, MapReduce,
and Spark. Wrangler [7] described an approach that predict when the stragglers
are going take place and make scheduling decisions to circumvent such
situation.

To mitigate stragglers, recent big data framework, as, for example; the
MapReduce system or its alternatives have implemented numerous reactive or
proactive approaches under which the system will add additional duplicates of
a job on other machines in a judicious method. This technique can
significantly reduce delays as due to stragglers, as the output from the
faster worker can be used without waiting for slower ones, however, with the
trade-off of consuming additional resources.

Full cloning of small jobs, without waiting and speculation completely [11].
Cloning small jobs increase the machine utilization since workload are small
if jobs is small slightly and consumes a small portion of the resources. [12]

###  II-B Using relaxed progress synchronization

In Bulk synchronous parallel (BSP) model, the straggler problem is significant
due to the costly barrier synchronization.

However, the Bulk Synchronous Parallel (BSP) model represents one of the
common ways to implement distributed applications of iterative convergent
algorithms underlined by an “input-data-parallel” methodology. Typically, the
model parameter will be stored in the shared data store so that every worker
will update throughout the iteration. There exists design of BSP model
guarantees the model data will be available to the worker after
synchronization instead of immediately available to all worker, and it would
enable worker efficiently to use the cached copy. Another design is to ensure
the allocation of work to the worker are similar across the iteration, and it
would enable the worker to reuse the already loaded data, and thus reduce the
overhead of input data movement.

Figure 2: Visual representation of Stale Synchronization model

For some ML Task, strict barriers of BSP can be relaxed with alternate
progress synchronization model. Flexible steadiness confines that have been
lately proposed suggested and considered two diverse names that are “Bounded
Delay consistency” and “Stale Synchronous Parallel” (SSP) [13]. These bonds
guarantee convergence proof, as, for example; the SSP has been publicized to
moderate insignificant transitory effects of straggler effects. The diagram
above shows the visual representation of SSP model. [4]

###  II-C Work Shedding and Work Stealing

Work stealing and shedding of work reflects methodologies for adaptively
rebalance work among the workers. The latest research by FlexRR [6] proposed a
method of work shedding, dedicated to the scope of data-parallel iterative ML.
FlexRR employs two mechanisms to mitigate the problem: temporary work
reassignment through RapidReassignment (RR), and adjustable consistency
parameters through the State Synchronous Parallel (SSP) model. SSP allows
individual workers threads to lead the slowest worker by a quantified slack-
bound iteration number. This flexibility permits mitigation of the straggler
problem to s certain degree but more fundamentally, provides sufficient
flexibility to RapidReassignment (RR) making it highly efficient. RR employs
peer-to-peer communication thus enabling the self-identification of workers as
stragglers. FlexRR designed to use the parameter server method to store and
distributing normal states among worker threads. During execution, it will
comprise of one process prompting a worker thread for respective cores on the
nodes as well as the number of threads in the background for internal
functionality. Following sequence diagram visualizing the RapidReassignment
protocol when a slow worker is behind the progress.

Figure 3: Work Shedding and Work Stealing using Rapidreassignment(RR)

###  II-D Predicting Straggler with ML Technique

Besides Job rescheduling, speculation [14] is another standard method for
straggler mitigation. It predicts stragglers based on progress score of the
task and launch a clone of the job on different machines. Most recent research
ML-NA [5] is intended on an ML-based Node Performance Analyzer. By leveraging
actual task execution log, ML-NA will categorize cluster nodes into different
classifications and predict their performance in the following task, so that
scheduler can be better and minimize the possibility of task straggler.

##  III Methodology and Experimental Set-Up

To study the straggler problem, we built a prototype system using FlexPS
system, which setup with the Bulk Synchronous Parallel model to support
distributed machine learning applications. FlexPS is a “parameter server” that
provides the abstraction of a set of shared sparse matrices that all processes
can access. These matrices are stored in memory, distributed across a set of
servers.

FlexPS provides a flexible parallel control API similar to the Bösen system,
abstracts consistency management and net- working operations away from
application, and presents a simple key-value interface, as shown below table.
It provides read and update operations including Get(), Add() and Clock().

TABLE II: FlexPS Client API  Get(key) |  Read a parameter indexed by key.
---|---
Add(key,value) |  Update/Add the parameter key by value
Clock() |  Signal end of iteration.

###  III-A Injected Straggler Framework

The experiment will test the effects of a straggler beyond what ascended for
the duration of the precise instances of the trials. It will also measure
attributable delays directly. The research will intentionally inject straggler
into the execution, including transient as well as persistent stragglers. The
experiment will include different injected frameworks, while the control setup
will involve testing the straggler mitigation technique without injected
delays. In other words, the control experiment will encompass the Ideal
situation representing perfect cases that assume every worker have no wait.
The injected straggler pattern are listed in the table below:

TABLE III: Injected Straggler Framework  Straggler Pattern |  Description
---|---
Slow Worker Pattern |  Random Transient delay, which within iteration job
Disrupted Machine Pattern |  Transient delay simulated by system disruptor method for specific time
Power Law Pattern |  The Random Transient delay within iteration job that varies with execution time
Persistent Straggler |  A particular node that persistently performs slowly
Ideal |  perfect cases that assume every worker have no wait

####  III-A1  Slow Worker Pattern

Through this pattern, it models transient worker by manually add sleep to the
worker, in application or engine. We will add all possibly delay point (any
point that may incur waiting) within each iteration, and each worker decide
its possibility (x%) to be delayed independently. It can be delayed from 0 – 2
times of the duration of the actual iteration time. It is possible to have no
or multiple straggler in each iteration. the transient delay % will be used to
determine the level of worker slowed (e.g., 100% delay means runs 2 times
slower than original one).

The simulation will be implemented by dividing the overall sleep into multiple
millisecond sleep, and set to all possibly delay point, (e.g. for a 50% delay
within a 1 second iteration, and there are potential 10 delay points, it can
insert a 1ms sleep at each delay point).

####  III-A2  Disrupted Machine Pattern

Through this methodology approach, the study will model transient resource
contention by simulating a high priority disruptor procedure that takes affect
significant Memory or CPU resources. It can be duplicated by initiating a
disruptor process with a probability of x% every d seconds. The disruptor
process can be implemented by launching multiple threads that work on heavy
calculation work (or run infinite loop). This implies that for d seconds, then
a temporary delay concentration, which is a component of d, with a machine of
p core running p worker threads, the disruptor therefor commences at d
$\times$ p processes. For instance, 200% delay on an 8-core machine running
eight worker thread will run 16 threads disruptor process [1].

####  III-A3  Power-Law Pattern

The selection of this methodology was informed by the real-world straggler
patterns as established by existing research. The gist of this approach is
predicated on the understanding a straggler may occur with a power-law
distribution [2] to finish an iteration. For instance,

$\displaystyle p\;(t)\;\propto\;t\;\times\;\alpha$

Where $\alpha$ represents an element of the distribution “skewness.” Sleep
will, therefore, be employed with the comparable execution in the Slow Worker
Pattern, with $t\;\times\;\alpha$ added to the delay time. Lesser $\alpha$
leads to a “flatter” distribution. Each iteration employs the same $\alpha$,
and without considering the actual completion time of the iteration.

####  III-A4  Persistent Straggler Pattern

This pattern is caused by a particular node that insistently performs slow.
The study also incorporates the concept of persistent straggler pattern
whereby some the machines receive 75% or even 50% of the work per iteration.
Such unbalanced assignments could trigger the case where processing skew of
the data is co-related with the data placement.

####  III-A5  Implementation of Straggler injection framework

Leveraging the FlexPS API, the straggler injection shall be implemented inside
the application code. The following operations has implemented to add
straggler inside the algorithm implementation. A global configurable switch in
the application to control whether current execution will inject straggler
simulation.

TABLE IV: Straggler injection framework  InjectStraggler(delay_percent) |  Inject Straggler for specific worker, with its delay setting. It will inject delay based on current actual process timer.
---|---
CheckPermanentStraggler(worker_id)
|  Check whether current worker is elected as permanent straggler
CheckTranisentStraggler(worker_id, iteration_count)
|  Check whether current worker is elected as transient straggler

Following Benchmark framework has implemented to extend current FlexPS system
in order to measure the performance of the parallel execution.

TABLE V: Straggler injection framework  Benchmark<TimeT> A C++ Template Object
that encapsulate the benchmark utilities
---
Benchmark<TimeT>::measure Measure the block of code and return the time_t
Benchmark<TimeT>::measure_r Measure the block of code and return the block’s
output
Benchmark<TimeT>::total Return the total time logged into benchmark object
Benchmark<TimeT>::reset Reset the time logged into benchmark object

###  III-B Benchmark with selected ML Algorithm

In order to study the behaviour of different mitigation technique over
different type of workload, the experiment is designed to conduct using
various ML algorithm implemented, and instrument with customized C++ library
implemented in FlexPS application code base. Following table describe the
dataset used for different ML Algorithm selected in the experiment.

TABLE VI: Experiment dataset used against different algorithm  Application |  Experiment dataset
---|---
Logistric Regression (LR) |  Avazu dataset, containing 40,428,967 training size with a feature dimension of 1,000,000 and 2 classes
Matrix Factorization (MF) |  Netflix dataset, which is a 480k-by-18k sparse matrix with 100m known features.
Latent Dirichlet Allocation (LDA) |  New York Times dataset, which is a 1.8 million articles that appeared between January 1, 1987 and June 19, 2007

####  III-B1  Logistic regression (LR)

This is a regression model where DV is unconditional. Specifically, the model
employed by this experiment is a binary dependent variable, this means that
only two values such as "0" and "1", representing output such as boy/girl, or
else win/lose can be used.

The MLR investigations use the Avazu dataset, containing 40,428,967 training
size underlined by a variable dimension of 1,000,000 as well as two
classifications. The investigations will also encompass ImageNet datasets
coupled with LLC elements comprising 64k experimental observations defined by
1000 classes and feature dimensions of 21,504.

####  III-B2  Matrix factorization (MF)

This is a procedure employed in reference systems, such as endorsing flicks to
consumers in networks such as Netflix. It is also referred to as collaborative
filtering. In numerical analysis, given a moderately complete matrix X, matrix
factorization factorizes X into factor matrices $L$ and $R$. consequently,
their respective product estimates X as $X\;\approx\;LR$. We can, therefore,
execute MF through the “stochastic gradient descent” (SGD) algorithm with the
reference of other systems and individual worker threads.

The experiment will employ the “Netflix dataset” underlined by “a 480k-by-18k
sparse matrix with 100m known features”. These elements are configured in such
a way that they factor the data set into the result generated by matrix
underlined by rank 100. This version is a “7683k-by-284l sparse matrix”
characterized by 4.24 billion recognized features underlined by rank 100.

####  III-B3  Latent Dirichlet Allocation (LDA)

LDA represents a generative probabilistic corpus model. The essence of this ML
strategy is predicated on the need to have documents served in random
assortments over the possible topic.

This experiment will employ the Nytimes dataset comprising 100m words within
1.8 million articles containing 100k vocabulary size. They will be configured
to categorize words, as well as reports into 1000.

###  III-C Experiment result

We performed experiments on Cluster in University to evaluate all straggler
technique in the presence of all straggler patterns injected during the
particular times of the experiments.

TABLE VII: Experiment result  Injected Pattern |  Figures
---|---
Slow worker Pattern |  Figure 4,5
Disrupted Machine |  Figure 6
Power Law Pattern |  Figure 7,8

##  IV Results for Slow Worker pattern

Figure 4: Total waste time for Slow worker patter against Technique/ML Figure
5: Average iteration time for Slow Worker Pattern against Technique/ML

The outcomes of the LR, MF and LDA are presented in figures above. The results
show that FlexRR decreases the time-per-iteration or total waste time by 35%
about Matrix factorization and 34% for Latent Dirichlet Allocation relative to
the Bulk Synchronous Parallel (BSP) and 25% and 34% reductions relative to SSP
respectively. As noted in the two experiments, the increase is more
significant for LR since the less costly LR undergo more transient effects of
the straggler problem.

The enhancements in cluster were observed despite implementing comparatively
short experiments on relatively costly situations expected minimal exhibit
sharing of resources with other tenant actions while at the same time
highlighting the realness of transient stragglers within infrastructures.

##  V Result for Disrupted Machine Pattern

Figure 6: Total Waste time for Disrupted machine pattern

We compare the average time-per-iteration (20 iterations) and total waste time
of alternative modes for the Disrupted Machine Pattern. Following diagram
shows results for MF—results for LDA and MLR are qualitatively similar. SSP
and BSP RR individually reduce the delay experienced by BSP by up to 49% and
42%, respectively. The combination of the two techniques in FlexRR matches
Ideal, reducing the run-time by up to 63%, very close to ideal waste time.

##  VI Result for Power Law Machine Pattern

Figure 7: Iteration Time for power law machine Figure 8: Total Waste time for
power law machine

We compare the average time-per-iteration (20 iterations) and total waste time
of alternative modes for the Power-Law Pattern. We present results on each of
our applications, setting $\alpha$ to 4, 7, and 11. Figure above shows the
results for MF. For $\alpha$ = 11, SSP and Speculative are faster than BSP by
39% and 40%, respectively. When the two techniques are combined in FlexRR, the
run-time is 48% faster than BSP. Similarly, to experiments conducted in
earlier sections, with increasing delays (smaller $\alpha$), the other three
modes experienced significant increases in run-times, while FlexRR experienced
only slight increases.

The results for MLR and LDA show similar trends. For $\alpha$ = 11, SSP and
BSP RR were 36% and 31% respectively faster than BSP for MLR and 37% and 42%
respectively faster than BSP for LDA. FlexRR was 43% and 52% faster than BSP
on LR and LDA respectively. With increasing delays (smaller $\alpha$), the
other three modes experienced significant increases in run-times for both LR
and LDA. FlexRR experienced only modest delays for LR and somewhat larger
delays for LDA. In all cases, FlexRR significantly outperforms the other three
modes.

##  VII Conclusion

The experiments described herein were designed to address the straggler
problem of the Parameter Server for important Machine Learning strategies such
as Matrix Factorization (MF), Logistic Regression (LR), and Latent Dirichlet
Allocation (LDA), which use iterative approaches to achieve convergence.

Experiments with custom implemented ML applications under a variety of
injected straggler behaviors confirm that FlexRR as a straggler mitigation
technique compared to the methods such as the cloning and speculative
execution is more effective. The results show that FlexRR decreases the time-
per-iteration and wasted time significantly under our experiment environment.
The study thus successfully conclude that FlexRR is the best algorithm as a
straggler mitigation technique in common cases.

## References

  * [1] Y. Huang, “FlexPS : Flexible Parallelism Control in Parameter Server Architecture,” _Proceedings of the VLDB Endowment_ , vol. 11, no. 5, pp. 566–579, 2018.
  * [2] M. Li, “Scaling Distributed Machine Learning with the Parameter Server,” in _11th{USENIX} Symposium on Operating Systems Design and Implementation, {OSDI} ’14_, 2014, pp. 583–598.
  * [3] L. G. Valiant, “A bridging model for parallel computation,” _Commun. ACM_ , vol. 33, no. 8, pp. 103–111, 1990.
  * [4] G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica, “Effective Straggler Mitigation: Attack of the Clones,” _Presented as part of the 10th{USENIX} Symposium on Networked Systems Design and Implementation_, pp. 185–198, 2013.
  * [5] X. Ouyang, C. Wang, R. Yang, G. Yang, P. Townend, and J. Xu, 2017.
  * [6] A. Harlap, “Addressing the straggler problem for iterative convergent parallel {ML},” in _Proceedings of the Seventh{ACM} Symposium on Cloud Computing_, 2016, pp. 98–111.
  * [7] N. J. Yadwadkar, G. Ananthanarayanan, and R. Katz, “Wrangler: Predictable and Faster Jobs using Fewer Resources,” _SoCC_ , pp. 1–14, 2014.
  * [8] M. F. Aktas, P. Peng, and E. Soljanin, pp. 12–14, 2017.
  * [9] J. Cipar, J. K. Kim, and S. Lee, “Solving the Straggler Problem with Bounded Staleness,” _HotOS_ , 2013.
  * [10] E. Krevat, J. Tucek, and G. R. Ganger, “Disks Are Like Snowflakes : No Two Are Alike,” _ACM HotOS_ , pp. 5–5, 2011.
  * [11] G. Ananthanarayanan, “Reining in the Outliers in Map-Reduce Clusters using Mantri,” _Proc. 9th USENIX Conf. Oper. Syst. Des. Implement_ , pp. 1–16, 2010.
  * [12] F. Pop, J. Kolodziej, and B. D. Martino, 2016.
  * [13] T. M. Chilimbi, Y. Suzue, J. Apacible, and K. Kalyanaraman, “Project Adam: Building an Efficient and Scalable Deep Learning Training System,” in _11th{USENIX} Symposium on Operating Systems Design and Implementation, {OSDI} ’14_, 2014, pp. 571–582.
  * [14] T. White, _Hadoop - The Definitive Guide: Storage and Analysis at Internet Scale{(4.} ed., revised & updated)_, O’Reilly, 2015.