# An Evaluation of Multi-Component Weft-Knitted Twill Structures for Sensing
Tensile Force

Roland Aigner
Media Interaction Lab
4232 Hagenberg, Austria
<EMAIL_ADDRESS>0000-0002-8503-4335 Frank Hepper
Karl Mayer Stoll GmbH
72770 Reutlingen, Germany
<EMAIL_ADDRESS>
###### Abstract

We present multi-component knitted resistive sensors for tracking tensile
force. The knits were fabricated using a Twill structure, which is a simple
pattern featuring anisotropic elastic behavior, providing high stability along
course-direction. Our sensors are made of two commercially available
conductive yarn types, with highly different linear resistance. We present a
variety of integration methods using the proposed Twill structure, all of
which can be easily replicated on a two-bed weft-knitting machine. We evaluate
the performance of the resulting sensor variations, with respect to
consistency, hysteresis, short-term and long-term relaxation and drift, among
other metrics. We found that particulars of the knit’s loop composition have a
crucial effect on the consistency of the sensor readings. Furthermore, we show
that knitting resistive yarn more tightly than the substrate material gives
superior results and that improving elastic recoil by adding Lycra to the
supporting substrate can considerably improve performance.

## Glossary

The following is a short and arguably incomplete description of the terms used
in the text, however we refrain to go into more detail, since this should be
sufficient for the scope of the paper. For more details please refer to [1].

knit, tuck, float: different stitch types performed by the needles. While a
knit operation forms a new loop by pulling the new yarn through the currently
held loop, a tuck just adds the yarn to the current loop, i.e.,
holding/securing the new yarn. In contrast, in a float (aka. ”miss”), the yarn
is guided behind the needle and not held at all.

wale, course: terms describing the dimensions of a knit. Oversimplified but
adequate for the scope of this paper, wales and courses can be considered the
”columns” and ”rows” in a knit, when using matrices as an analogy.

##  1 Introduction

Textile based sensors provide beneficial features such as high flexibility and
breathability, which can make them comfortable for wearing them on skin, e.g.,
when compared to foil-based solutions [2, 3, 4, 5]. This is attractive for use
cases requiring long duration of direct skin contact, such as therapy
scenarios via bio-monitoring [6] and activity tracking [7, 8], but also for
user interfaces such as data gloves, for tracking of hand posture [9, 10] or
gesture detection [11]. Many of those use cases are already implemented with
knitted fabrics, since they are particularly suitable for sensing strain due
to their innate stretchability.

Although there already is a large body of work focusing on knitted strain
sensors, we found that most of them are based on highly stretchable fabrics,
which are not always desirable. Many scenarios, also beyond garments, require
solutions that provide higher tensile stability, or even anisotropic
elasticity. Those properties are mostly subject to the geometric composition
of the knit, i.e, the knitting pattern. In weft-knitting, those pattern can be
thoroughly engineered down to loop level, which is in contrast to warp-
knitting [12]. Examples for patterns with relatively high extensibility are
Plain Knit (aka. Single Jersey) [13, 14, 15, 16, 17], Double Jersey [13], and
rib structures [18, 13, 19]. In contrast, patterns with higher stability are
relatively rare in related literature, examples for those are Interlock used
by Atalay et al. [6], and Cardigan used by Ehrmann et al. [20], which showed
they provide better sensitivity in low-elongation ranges, when compared to
Double Jersey. However, both Interlock and Cardigan represent patterns that
occupy both of the machine’s needle beds at all times, which is a potential
limitation in flexibility for fabrication and design. Therefore, and in
contrast to the stated works, we use a pattern that is inconsistently called
Twill in the textile industry, due to its structural similarity with the weave
pattern of the same name. It is a generally widespread and simple pattern,
however not touched in textile sensor literature, to our knowledge. As
illustrated in Figure 1(a), it consists of courses of alternating knit and
float stitches, while the sequence is shifted by one needle for every other
course. Due to this high number floats, it provides exceptional stability in
course-direction (i.e., ”horizontally”), while being more extensible along
wale-direction (i.e., ”vertically”), when compared to Plain or Double Jersey
Knits. This represents a distinct property of a Twill, as opposed to Cardigan
or Interlock, with exactly opposite behavior, as a preliminary study confirmed
(see supplement). Note that manufacturing of a Twill only requires one needle
bed, which increases flexibility and can be a design advantage over other
patterns. For example, using a two-bed machine, the entirety of the sensing
part can be hidden away to one face of the knit, as done in the work presented
in this paper, which can be of aesthetic preference and/or protect it from
exposure and therefore from abrasion and damage.

(a)

(b)

(c)

Figure 1: Illustration of a Twill knitting pattern (a). Where current flows
along the yarn, loop intermeshing points act as variable resistors (b),
increasing conductivity with physical stress at the contact positions. The
overall knit geometry can therefore be modeled as a network of variable
resistors (c). We maximize the ratio of sensor yarn resistivity and connector
yarn resistivity, so changes in connecting parts’ resistance $R_{C}$ are
negligible over the much higher absolute values from sensor loops $R_{S}$.

In contrast to most of related work, which focuses on sensing strain [17, 9,
13, 21, 22, 18, 23], our primary interest is in sensing force, which cannot be
trivially inferred from strain directly, due to short-term wear-out effects,
exposing hysteresis. Furthermore, knits are subject to considerable structure-
dependent relaxation, causing a gradual, non-linear decrease of force at
constant elongation. Based on our observations during this work, we noticed
that recorded displacement values (and thus inferred strain values) are not
entirely adequate to reflect the state of the fabric, since it may be slack
when the actuator returns to its initial position. We argue that due to these
effects, it would be necessary to record and reconstruct the true fabric
lengths by different means, e.g., by optically tracking its geometric state,
which would complicate not only the setup, but also reporting and its
comprehensibility. Within our work, we consequently investigate the sensors’
response to force directly and thus avoid this issue. For the sake of
comparibility with related work however, we still include strain data in this
paper.

By combining two types of conductive yarn that are knitted directly into the
fabric, we produce a fully functional textile force sensor without requiring
manual finishing steps. This is opposed to augmenting pre-existing knits by
embroidering [9] or printing [24] functional parts, or by sewing patches of
conductive fabric[25]. Other works demonstrate the method of polymerizing
parts of textiles with Polypyrrole, e.g., the seminal work of DeRossi et al.
[26] showed a data glove with resistive sensing areas. However, this process
is challenging to do in a consistent manner, when compared to computerized
flatbed knitting, which provides loop-level control. Hence, in contrast to the
stated works, our method enables to precisely design and tune the sensor
structure even to create highly intricate sensor shapes and complex connector
traces (cf. [27]).

The goal of our work was to find the optimal variant with respect to general
sensor consistency (for repeated actuation with equal or varying force, as
well as different actuation speeds), hysteresis, dynamic range, offset,
relaxation, drift, and anisotropic behavior. We therefore explored different
implementations and contrasted their behavior in a systematic evaluation.

In a nutshell, the top contributions of this paper are:

  * •

Three methods of integrating a Twill-based resistive sensor on a Twill
substrate fabric, including conductive knit connector traces for attaching
electronics at remote positions.

  * •

10 variations of these sensor designs, using different substrate material
compositions and yarn tensions.

  * •

An in-depth evaluation of those 10 variations and our findings regarding
consistency, relaxation, offset, and drift in different scenarios.

Figure 2: Sample of one of our sensor patches (left), with conductive yarn
traces connecting the resistive area (black) on both upper and lower ends. We
evaluated our sensors using a custom-built tensile tester with an integrated
force cell (right).

##  2 Sensor Implementation

(a)

(b)

(c)

Figure 3: Twill based knitting structures for T (tubular, a), P$\leftarrow$R
(Resistat tucked to front-face PES, b), and P$\rightarrow$R (PES tucked to
back-face Resistat, c). Note the connecting front bed tucks at the beginning
of each Resistat row (red) which secures the edges of the sensor area with the
substrate knit (purple). Connector traces (blue) are knit on the back bed for
connecting to the Resistat loops, and on the front bed otherwise. Images at
the bottom show closeups of front bed (PES) and back bed (Resistat) faces of
the resulting knit structures. In particular, the closeups show PTm (a), PRP
(b), and PPR (c).

In the following chapter, we outline the sensor design, including potential
for slight modifications that we expected to have an impact on sensor
performance. We present our knitted samples and specify manufacturing details.
All our patches where knit on a flat-bed knitting machine of type ADF 530-32
KI W Multi Gauge from KARL MAYER STOLL, at gauge E 7.2. Knitting programs were
created with Patternsoftware M1 PLUS Version 7.5.

###  2.1 General Sensor Design and Sensing Principle

The functional principle of our knit force sensors is according to Holm’s
Theory [28], which states that contact resistance is depending on material
resistivity $\rho$ and hardness $H$, as well as contact point count $n$ and
pressure $P$, with

$R=\frac{\rho}{2}\sqrt{\frac{\pi H}{nP}}\,.$

Since contact pressure between loops is varying (and may even be zero when
loops lose contact), the resistance drops when force is applied. Consequently,
each intermeshing point in the sensor knit can be considered a variable
resistor (cf. Figure 1(b)), and moreover, the overall structure can be modeled
as a network of resistors, as done for analytical solutions by [29, 30, 22].

Similar to Baribina et al. [31] and Semjonova et al. [27], we utilize a multi-
material sensor layout, combining conductive and resistive yarn, i.e., two
types with largely different linear resistance, which provides several
advantages:

  * •

Conductive parts can be utilized to knit connector traces, that enable to
comfortably attach readout electronics at remote positions of the fabric. This
is unlike other work that requires attaching of connecting wires directly at
the sensing structures, such as [11, 8].

  * •

In increasing the resistance ratio of sensor area to connector trace we ensure
that the sensor area operates in vastly different absolute value ranges, when
compared to the connecting parts. Hence, resistance changes caused by
deformation of the connectors are minor and therefore negligible, when
compared to the sensor’s operational range. This is similar to [31] and
addresses an issue often ignored in related work, while furthermore enabling
to create more explicit and localized force sensors on a textile.

  * •

Connecting the resistive part with conductive yarn along the entire width (cf.
Figure 2 left) provides more uniform current flow across all wales (cf. Figure
1(c)), since RC is insignificant against RS. This should improve sensitivity
consistency across the whole sensor area, which could be particularly relevant
when the number of sensor wales is much higher than the number of its courses.

###  2.2 Materials

As a resistive yarn for the sensor areas, we used Shakespeare® Resistat P6204
H100i111https://shakespeare-pf.com/product/polyester/, which is a den 100/24
Polyester fiber with Carbon sheath, providing relatively high linear
electrical resistance of $\scriptstyle\mathtt{\sim}$10 M$\Omega$/m. We twisted
four den 100/24 threads with 30S in 1st stage and 50Z in 2nd stage, to achieve
adequate yarn count for a balanced knit when combined with our PES yarn,
yielding a den 400 thread with den 100/24x4 and $\scriptstyle\mathtt{\sim}$2.5
M$\Omega$/m.

The conductive traces for providing connections were knit with Shieldex®
Madeira HC40222https://www.shieldex.de/products/madeira-hc-40/, which is a
silver-coated PA yarn with den 260 and electrical resistance of $<$300
$\Omega$/m and proved highly durable during previous work [32, 33].

For the surrounding substrate base structure, we used a PES with den 150/30
(TWD Fibres GmbH). The Lycra that we plated along the PES for improving the
fabric’s elastic recoil was a den 140 Lycra core covered with PES den 150/20
(Jörg Lederer GmbH).

###  2.3 Knit Structure and Manufacturing

As any pattern that is knit on a single bed, internal forces on the Twill are
unbalanced, meaning it shows inherent curling tendency. For many use case
scenarios, where the knit is tailored together with other parts, this may not
be an issue. Otherwise, it can be counteracted by framing with a more stable
knit structure.

Instead of implementing the sensor area as an Intarsia field within a
surrounding PES structure (cf. [29, 14, 10, 34, 11]), we knit the resistive
yarn on the opposite needle bed and connect it to the PES face, which is knit
as a continuous Twill. Apart from a more straight-forward integration into a
knit, this provides better control about the force distribution throughout the
structure. Due to different properties of PES and Resistat, an Intarsia field
requires proper tuning of yarn count and stitch settings, to prevent an
unbalanced and non-uniform surface. By knitting two faces on opposite beds, we
gain more flexibility in tuning the Resistat tightness – and therefore the
sensor’s responsiveness – without introducing areas of considerable physical,
visual, and haptic inconsistency. A side-effect is also that the functional
parts can be hidden away and are therefore protected from abrasion by the
covering PES layer, which may be a benefit in some use cases.

For connecting the sensor face with the base structure, we tuck the Resistat
to the opposite bed at the beginning of each knit course. On upper and lower
courses, the Resistat is knit to the conductive yarn which provide connector
traces. Figure 3 provides a detailed knitting diagram. For knitting, we plied
2 threads of den 400 Resistat for the resistive parts, 2 threads of den 260
Madeira HC40 for the conductive parts, and 6 threads of den 150 PES for the
substrate.

Table 1: Overview of our sensor variations. We varied structure composition (Type), ratio of PES threads vs. Lycra threads, nominal stitch length (NP) of substrate material (PES+Lycra), Resistat material (Res), and tuck stitches that connect front and back faces. Note that NP are a measure of yarn usage per loop, i.e., lower numbers represent tighter knits. Name | Type∗ | PES | Lycra | NP PES | NP Res | NP tuck | notes
---|---|---|---|---|---|---|---
PTl | T | 6 | - | 13.1 | 13.5 | - | tubular, low tension for Resistat
PTm | T | 6 | - | 13.1 | 12.5 | - | tubular, medium tension for PES/Resistat
PTh | T | 6 | - | 13.1 | 11.5 | - | tubular, high tension for Resistat
PRP | P$\leftarrow$R | 6 | - | 13.1 | 12.0 | 9.0 | Resistat tucked to PES
PPR | P$\rightarrow$R | 6 | - | 13.1 | 12.0 | 9.0 | PES tucked to Resistat
PL1m | P$\rightarrow$R | 5 | 1 | 12.5 | 12.5 | 9.0 | 1$\times$Lycra + medium tension Resistat
PL1h | P$\rightarrow$R | 5 | 1 | 12.5 | 11.5 | 9.0 | 1$\times$Lycra + high tension Resistat
PL1ml | P$\rightarrow$R | 5 | 1 | 12.5 | 12.5 | 9.5 | 1$\times$Lycra + medium tension Resistat, low tension tuck
PL2m+ | P$\rightarrow$R | 4 | 2 | 12.0 | 11.8 | 9.0 | 2$\times$Lycra + medium-high tension Resistat
PL2hl | P$\rightarrow$R | 4 | 2 | 12.0 | 11.5 | 9.5 | 2$\times$Lycra + high tension Resistat, low tension tuck
∗ Types: T = tubular knit; P$\leftarrow$R = Resistat is tucked to front-face
PES; P$\rightarrow$R = PES is tucked to back-face Resistat

###  2.4 Variations

As mentioned above, we knitted non-functional PES and functional Resistat on
opposite needle beds, yielding two knit faces that need to be fixated so they
do not fall apart. We investigated three options of doing so: the most
straight-forward one is to connect both faces along the sensor’s outer wales
(cf. Figure 3(a)), by tucking at the respective outer needles. This results in
a ”tubular” knit structure, which we henceforth will address with ”T”. Note
that both faces are completely detached in this knit, which could lead to
erratic behaviour, depending on the fabric’s firmness. We therefore tried
three variations with the sensor parts knitted with different tightness: one
with the Resistat knit with lower tension than the PES (”Tl”), one with medium
tension, meaning PES and Resistat tightness balanced (”Tm”), and one with
Resistat knit with higher tension than PES (”Th”).

We furthermore created variations that kept both faces closely attached, by
tucking the Resistat to the PES (”P$\leftarrow$R”) across the entire courses
(cf. Figure 3(b)), as well as the opposite, tucking the PES to the Resistat
loops (”P$\rightarrow$R”, cf. Figure 3(c)).

From handling with the resulting knits, we could subjectively see that our
first variations with 6 threads of PES (”P”) were prone to short-term wear-out
and we therefore expected poor elastic recoil. For this reason, we also
created samples that combined PES with Lycra (”PL”), to encounter this aspect
(cf. Table 1). Hence, in addition to using 6$\times$PES for the surrounding
substrate, we also created patches where we plated 5$\times$PES together with
1$\times$Lycra (”PL1”), as well as 4$\times$PES with 2$\times$Lycra (”PL2”).
We already saw during our first evaluation, that lower-tension Resistat
patches performed poorly, as well as connecting front and back faces
outperforms tubular structures, we therefore chose to focus on those for our
Lycra variations, hence, all of them were knit of type P$\rightarrow$R with
medium to high Resistat tension.

##  3 Evaluation

###  3.1 Apparatus

For evaluation, we used a custom tensile tester, which we built from an
obsolete CNC milling machine (cf. Figure 2 right). The machine was operated by
Art-Soft Mach4 CNC Control Software (v4.2.0), running on a Windows 10 PC. We
attached mounts to clamp the textiles on both ends, incorporated needles at 2
cm distance additionally secured the textile so it would not slip. The clamp
attached to the moving part was equipped with a single-point load-cell of type
Sauter CP P1-Ba-d-1810333https://www.kern-sohn.com/shop/en/measuring-
instruments/measuring-cells/CP-P1/ which was sampled at
$\scriptstyle\mathtt{\sim}$80 Hz with an ADS 1231 24-bit Delta-Sigma
ADC444https://www.ti.com/product/ADS1231. We acquired the sensors’ resistance
values using a simple voltage divider with a reference resistor of 606
k$\Omega$ and sampled using an Adafruit ADS1115 16-bit
ADC555https://www.adafruit.com/product/1085 at $\scriptstyle\mathtt{\sim}$128
Hz. At $\scriptstyle\mathtt{\sim}$40 Hz, we averaged the samples of the
previous period, and captured the results into CSV files for later analysis,
along with timestamp, and actuator displacement. Sampling, recording, as well
as remote control of Mach4 via RS232, was performed by an ESP32 on an Adafruit
HUZZAH32 Feather board666https://www.adafruit.com/product/3405.

Note that our tensile tester is able to move along three axes, which enables
testing for shearing effects, however, this would require a modification for
omni-directional force measurement. Although this would greatly complicate the
procedure and evaluation, we see potential for future work, in order to
simulate more generic actuation which may be closer to many real-life
scenarios. Since this this work is focusing on variations of knit structures,
we explicitly performed orthogonal actuation in our tests.

non-Lycra Lycra

(a)

(b)

(c)

(d)

(e)

(f)

Figure 4: Characteristics and timelines of non-Lycra (left) and Lycra (right)
variants: Plots of correlation between strain $e$ and force $F$ (a,c), as well
as strain and relative resistance change $\Delta R/R_{0}$ (b,d). ’x’ marks
initial values at beginning of recording. Timeline plots of all variations
(e,f), overlaying strain $e$ (dashed, black) and sensor conductivity $G$, show
respective conformity of our variations.

###  3.2 Procedure

All of our knitted sensor variations share a similar design (cf. Figure 2
left), i.e., a 4 cm $\times$ 4 cm square field of resistive yarn, which is
connected with conductive traces along the entire upper and lower courses.
These connector traces were knit beyond the sensor area, leading to the edge
of the textile sample, where we attached crocodile clips to connect our
measurement electronics. We chose strong clips to avoid their slipping during
the procedure and ensured adequate overlap with the conductive yarn. We
refrained from testing different sensor dimensions, since we know from [35]
that sensor resistance is directly proportional to height and inversely
proportional to width, with $R=\rho\frac{h}{w}$, where $\rho$ is a material-
specific constant. We were able to verify this correlation in a preliminary
evaluation (see supplementary material).

Each sensor variation was knitted three times. We performed an ex ante
evaluation to get an estimate regarding consistency and to identify outliers.
We found good consistency overall and a very low number of outlier sensors
(see supplement). We did however not perform an in-depth formal evaluation
regarding consistency at this point.

For each sample, a single 5-cycle procedure was recorded. For the testing
procedure, we marked the textiles at 5 cm distance with the 4 cm sensor areas
centered, giving 5 mm extra on each side for mounting. The samples were then
punched through the mounting needles at the marks, so the tests would start
from consistent initial lengths of 5 cm. Tested patches were not previously
ironed or otherwise chemically, mechanically, or thermally treated.

####  3.2.1 Pulling with equal force

To observe correlation of sensor reading and applied force, as well as sensor
offset and general consistency, we performed a test procedure repetitively
applying force along wale direction, and releasing again. We chose the force
based on an informal initial test, where we estimated the upper working limit
of most of our sensor variants with $\scriptstyle\mathtt{\sim}$20 N, and
repeated for 5 cycles with a jog rate of 1.333 mm/s. Note that, due to a
communication lag in between the ESP32, Mach4, and the testing machine, we
were slightly overshooting the target forces, however this does not undermine
the general point of our results. Also note that, since different samples had
different elastic behavior, this resulted in different strain ranges.
Moreover, since we are not observing strain but force, we returned to F=0 N
after each cycle, which does not align with the initial actuator position of
d=0 mm, due to fabric extension from wear-out. As a result, this offset in
strain could be considered a metric for the fabric’s wear-out.

####  3.2.2 Pulling with dwell

In order to investigate drift and relaxation effects, we conducted a test
similar to our initial one, however instead of switching actuation direction
at 20 N and 0 N immediately, we dwelt for 5 seconds at each position. Note
that due to ongoing fabric relaxation, the force was not constant at this
point. We refrained from readjustment motions, since we judged this would
introduce considerable jerkiness in the data and complicate analysis.

####  3.2.3 Pulling with varying speed

Since we noticed during ex-ante experiments, that actuation speed can have a
profound impact on the sensor reading – most notably on the spikes in
resistance after starting and stopping – we repeated our initial tests (5
cycles at 20 N, no dwell) with half and twice the baseline speed, hence,
speeds were 0.667 m/s, 1.333 m/s, and 2.667 m/s.

####  3.2.4 Pulling with increasing force

To find upper sensing range limit and to inspect consistency when pulled with
different amplitudes, we varied pulling force. For this test, we started at an
initial 5 N and increased in steps of 5 N up until 40 N, with returning to 0 N
after each cycle. The test was again based on our initial one, i.e., we did
not dwell before switching direction and moved with 1.333 m/s.

####  3.2.5 Long-term pull

To observe long-term drift and relaxation effects, we performed a test pulling
the samples to 20 N and returning to 0 N, dwelling for 15 minutes at each end.

####  3.2.6 Course-directional pull

In related work, knit strain sensors are frequently tested along a single
direction [13, 22, 18]. However, like most knitting structures such as Jersey
and Rib patterns, a Twill is subject to anisotropic behavior in terms of
physical properties, such as elasticity and recovery. We therefore
investigated the behavior orthogonal to our primary testing direction as well,
by mounting the sample rotated accordingly in our testing apparatus.

Table 2: The majority of our results are gathered in this table. Note that we progressively excluded sensors that were performing badly from subsequent evaluation steps. Best values for non-Lycra and Lycra versions are put in bold. | non-Lycra | Lycra
---|---|---
| PTl | PTm | PTh | PRP | PPR | PL1m | PL1h | PL1ml | PL2m+ | PL2hl
$\Delta$d0,5 [%] | 11.7 | 11.8 | 7.9 | 11.4 | 10.2 | 11.0 | 10.6 | 9.1 | 5.4 | 5.2
$\Delta$d1,5 [%] | 2.3 | 2.6 | 2.0 | 2.3 | 1.6 | 1.8 | 1.9 | 2.0 | 1.3 | 1.3
$r^{2}$ | 0.65 | 0.90 | 0.91 | -1.94 | 0.90 | 0.92 | 0.90 | 0.91 | 0.92 | 0.93
hR [%] | 14.9 | 27.1 | 10.7 | - | 25.4 | 45.6 | 63.3 | 24.8 | 5.8 | 4.1
Fh [N] | 0.5 | 0.0 | 0.0 | - | 0.0 | 0.0 | 0.0 | 0.0 | 9.2 | 11.1
$\Delta$Rrel [%] | 24.6 | 46.5 | 56.7 | - | 64.0 | 63.1 | 65.9 | 56.7 | 35.4 | 34.5
offset [%] | - | - | -1.62 | - | -2.14 | -1.62 | - | - | -3.45 | -3.24
relaxation [%] | - | - | 7.32 | - | 5.80 | - | - | - | 2.59 | 2.45
drift [%] | - | - | 23.29 | - | 30.38 | - | - | - | 8.51 | 7.93
Tr [s] | - | - | 22.9 | - | 630.4 | - | - | - | 15.1 | 15.1
Td [s] | - | - | 24.6 | - | 23.7 | - | - | - | 10.7 | 10.2
jog x 0.5 $r^{2}$ | - | - | 0.84 | - | 0.94 | - | - | - | 0.92 | 0.20
jog x 2.0 $r^{2}$ | - | - | 0.90 | - | 0.87 | - | - | - | 0.94 | 0.84
course-dir hG [%] | - | - | 4.3 | - | 8.7 | - | - | - | - | -
F [N] | - | - | 6.9 | - | 5.5 | - | - | - | - | -

##  4 Results and Discussion

In the following, we summarize our main findings. Note that to save space and
reduce complexity, we narrow down our subset of evaluated patches, by
progressively excluding poorly performing sensor variations. Refer to Table 2,
which sums up the majority of our results.

###  4.1 General performance

####  4.1.1 Non-Lycra variants

Non-Lycra variants show almost linear relation between strain $e$ and applied
force $F$ for the pulling segments (cf. Figure 4(a)), however during release
phase, we see considerable lag throughout all variations, which is due to poor
elastic recoil. This effect inherently translates into hysteresis in sensor
characteristics (cf. Figure 4(b)), since releasing does not reflect in the
knit mesh immediately and instead exhibits noticeable delay for recovery of
the structure. We consider this an innate limitation of knits, however this
can be counteracted to some degree, which was our main motivation to add Lycra
into the supporting base knit, as outlined in Section 2.4.

We noticed that the resting state (i.e., at $F$=0 N) of the sensor is highly
different from the remaining iterations, hinting towards sensor offset. To
quantify this wear-out effect, we calculated the relative extension $\Delta
d_{0,5}$, i.e., the change in length in between before 1st and after 5th
pulling iteration. We see that for the non-Lycra variants, PTh performs best,
with 7.9% extension. However, the first pulling iteration can be considered an
outlier and may be irrelevant in many use case scenarios, e.g., when the
fabric is draped and therefore permanently stretched. Therefore, we also
report relative extension $\Delta d_{1,5}$, which excludes the first iteration
by calculating relative length change in between _after_ 1st and 5th pulling
iterations. There, the results are different, with PPR clearly outperforming
PTh.

When comparing sensor response by relative change in resistance $\Delta
R/R_{0}$ (cf. Figure 4(b)), we see that for tubular structures (i.e., PTl,
PTm, and PTh), tighter knit Resistat areas result in superior characteristics
(i.e., PTh, with better correlation between strain and resistance change, less
hysteresis, less noisy signal). Unexpectedly, there is considerable difference
between the two connected variants PRP and PPR. While the patch with PES
tucked to the Resistat (P$\rightarrow$R) clearly outperforms its tubular
equivalent PTm, tucking the other way around (P$\leftarrow$R) results in a
defective sensor. We can eliminate the possibility of manufacturing flaws,
since all of our three specimen of type PRP showed this erratic behavior. We
exclude PRP from further evaluations, due to the bad performance.

The difference in sensor performance is also clearly visible on the timeline
plots (cf. Figure 4(e)), where conductivity $G$ of PTh and PPR goes well in
line with strain $e$. Note that since we alternate between 0 N and 20 N, the
values of $e$ drift slightly upwards due to wear-out effects. As mentioned
above, we present strain-related data in the paper for sake of comparability,
however we refer the interested reader to the supplement, which shows that the
conductivity is well in line with the amplitude of force $F$ in most
variations. We quantify conformance between the two trends of $F$ and $G$
using the Coefficient of Determination777We use the less-common lower case
notation to mitigate confusion with electrical resistance $R$
$r^{2}=1-\textrm{SS}_{res}/\textrm{SS}_{tot}$, where $\textrm{SS}_{res}$ is
the residual sum of squares and $\textrm{SS}_{tot}$ is the total sum of
squares. Both data series are first normalized using the
preprocessing.StandardScaler from Python package scikit-
learn888https://scikit-
learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html,
which transforms all values with $y=(x-\mu)/\sigma$. Results show that PTm,
PTh, and PPR perform best in that regard.

####  4.1.2 Lycra variants

From Figure 4(c), we see that elastic recoil was slightly improved by adding
Lycra, however, the striking linearity we observed for the pulling-segments
earlier seems to suffer from the boost in elasticity in general. For all of
the Lycra variants PL*, conformity between force and conductivity is also
slightly improved (cf. Figure 4(f), and $r^{2}$ values). Most notably the
variants with 2$\times$Lycra (PL2*) show best linearity in e/R correlation
with least hysteresis (cf. Figure 4(d)). Drift also appears to be less severe
for those variations, which also reflects on relative extension values $\Delta
d_{0,5}$ and $\Delta d_{1,5}$ (cf. Table 2).

###  4.2 Hysteresis & Dynamic Range

In order to objectively compare hysteresis, we separated data into pulling and
releasing segments and fit exponential functions to the data sets using SciPy
function
optimize.curve_fit999https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html
(for further details refer to the supplementary material). We excluded the
first pull/release cycle as an outlier for this curve fitting procedure and
normalized $R$ values by scaling with $1/R_{0}$. We then searched for
positions of maximum distance between pulling and releasing curves. Results
are reported in Table 2, with resistance hysteresis $h_{R}$, at respective
locations $F_{h}$. We see that the variants with two Lycra threads PL2*
outperform all others, including non-Lycra knits. Furthermore, their maximum
hysteresis is found at $F$$\scriptstyle\mathtt{\sim}$10 N, unlike most
patches, which show considerable differences at 0 N, as results of strong
settling effects.

We report dynamic range as relative difference in sensor resistance in between
$F$=0 N and $F$=20 N, i.e., $\Delta
R=\mathrm{abs}\left(R_{0}-R_{20}\right)/R_{0}$. To find $R_{0}$ and $R_{20}$,
we used the curves fit for finding hysteresis, and evaluated them at $F$=0 N
and $F$=20 N: for non-Lycra versions, the connected sensor shows higher range
than the tubular knits. The versions with two strands of Lycra PL2* show least
range.

Even though the PL2* variants were inferior in terms of range, we decided to
exclude PL1* sensors for further evaluation, since we judged low hysteresis
and better consistency far more important than range. We furthermore tried to
keep a balanced set of Lycra and non-Lycra patches with the two best-
performing each.

Figure 5: Results from short-term dwell with repetitive actuation (left) and
long-term dwell with single actuation (right) show all sensors are subject to
relaxation and sensor drift.

(a)

(b)

Figure 6: Results of our jog-rate test (a) showed that PL2m+ is most immune
against variations in actuation speed. Cascading tests (b) showed that
characteristics lines of Lycra-variants follow largely different trends,
depending on the force applied.

###  4.3 Offset, Relaxation, & Drift

We use the terminology of [36], with offset being the change in resting-state
resistance after each pulling iteration, relaxation being the continuous
change in resistance at constant strain, and drift being the continuous change
in resistance when returned to resting state. We calculate offset as relative
change in resistance between before and after each cycle. To quantify
relaxation and drift, we calculated resistance change relative to the initial
value of the respective dwelling segment. For calculating all three metrics,
we again excluded the first pull/release iteration for each sets, since
sensors initiate form long-term settled states (cf. initial trends in Figures
4(e) and 4(f)); averaged values of the remaining 4 segments are presented in
Table 2.

Offset values show the remaining non-Lycra variants are superior over our
Lycra versions. Furthermore, the tubular structure PTh outperforms the
connected version PPR. However, tests with dwelling at 0 N and 20 N for 5
seconds each showed that Lycra variants are by far superior in terms of both
relaxation and drift, as can be seen also in Figure 5 (left). For Lycra
variants, PL2hl is only slightly superior, while non-Lycra versions tie for
both metrics: the tubular sensor shows less drift but worse relaxation
behavior.

Results from long-term test (cf. Figure 5, right) show similar results. It is
clearly visible that PPR exhibits highest relative noise and PTh shows lowest.
To compare settling of conductivity values, we calculated RSD over time
windows of the past 10 seconds with
$\textrm{RSD}_{10}(t)=\sigma_{10}(t)/\mu_{10}(t)$, where $\sigma_{10}$ and
$\mu_{10}$ denote the SD and arithmetic mean of $G$ values in the period
$[t-10\textrm{s},t]$ respectively. In Table 2, we specify periods for RSD10 to
permanently drop below 1%. We report one value for relaxation (i.e., when the
sensor patch is pulled, Tr) and one for drift (i.e., after sensor patch is
released again, Td). Similar to our short-term dwelling tests, we observed
that both Lycra variants are by far superior for both of relaxation and drift.
Furthermore, the Lycra-versions’ advantage is backed by long-term actuation
tests, straining the sensors 2,200 times with e=20% over a time-span of 5.6
hours (see supplement).

###  4.4 Actuation speed

To compare recordings with different lengths resulting from different jog-
rates, we again discarded the first pull/release iterations as outliers,
normalized along time-axes, and downsampled our data to equal sample count
with
scipy.interpolate.interp1d101010https://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.interp1d.html.
We again scaled our samples using sklearn.preprocessing.StandardScaler with
$y=(x-\mu)/\sigma$, this time not individually for each recording, but using
$\mu$ and $\sigma$ of our baseline set (1.333 mm/s) for all three speeds, to
preserve relative deviations. We then determined conformity of half and double
speed from the baseline set by calculating $r^{2}$, which can be found in
Table 2.

We see that for the non-Lycra versions, PTh deviates more for half speed,
however less for double speed. For Lycra patches, we see clearly that PL2m+ is
superior to PL2hl (cf. Figure 6(a)), which implies that tighter tuck tension
(i.e., tighter connection between front and back faces) should be preferred.

###  4.5 Increasing force

Figure 6(b) shows data collected from continuously increasing force for each
iteration in steps of 5 N. We see that both Lycra variants perform very poorly
and follow a different trend for each repetition, returning to largely
different sensor values when released. Note that this is most severe when we
went beyond our previous upper testing limit of 20 N. Non-Lycra patches show
much more consistent trends, with PTh achieving the best results, even in
highly-strained states. We see furthermore, that PPR seems to reach saturation
more quickly.

###  4.6 Anisotropy

Our main testing direction so far was along wales, where the knit provides
good stability. Many use cases cause actuation along a single direction,
however, given the anisotropic nature of most knitting structures, it is
reasonable to also consider the orthogonal direction. Although for our Twill
structure equal force causes higher strain in course-direction, first
observations with non-Lycra patches did not hint towards considerable
differences in terms of e/R correlation. Our tests however show that the
Lycra-variants exhibit significant erratic behavior when actuated along
courses. In particular, characteristics and timelines in Figure 7 show erratic
sensor values for Lycra-variants, similar to PRP, which we discarded earlier.
We therefore infer that patches with Lycra additives are only of limited value
for use cases that involve omni-directional strain and/or shear. Hysteresis
data for non-Lycra patches can be found in Table 2. As earlier with PRP, we
refrained from calculating hysteresis for Lycra-variants since we found curve
fitting unreasonable due to their overall erratic behavior and high offsets in
between pulling iterations; more details on curve fitting and hysteresis can
be again found in the supplementary material.

Figure 7: Course-directional strain behavior is most consistent for non-Lycra
variants, while Lycra-versions act highly erratic.

###  4.7 Discussion

Summarizing, we found that when manufacturing our sensors as tubular knits,
the resistive part should exhibit slightly higher tension than the substrate.
Connecting front and back faces can yield better results in many instances,
however an unexpected finding was that the tucking direction is crucial: if
the resistive yarn is tucked to the substrate, the resulting sensor is
defective. If however the tucks are performed the other way around, the result
is among the best-performing sensors, however gives noisier values when
compared to the tubular versions. Lycra-variants produced more consistent
results overall, at least within our testing range of 0 to 20 N. They
subjectively showed less wear-out effects and better elastic recoil. In terms
of quantitative measures, they are less prone to relaxation and drift, both in
short- and in long-term. Knit using two threads of Lycra for the substrate
showed best linearity, least hysteresis, and are least affected by variations
in actuation speed; using higher tuck tension yielded slightly better results.

Still, we see that the best choice of materials depends largely on the
specific use case at hand. We noticed that Lycra-enhanced variants perform
well for wale-directional strain, however, the results were sobering when we
varied actuation amplitudes, especially beyond our usual testing range with
huge offsets as there was considerable offset and hysteresis visible in the
plots. In contrast, the PES-only-patches show much less anisotropic behaviour
and could be used even beyond 20 N, in particular the tubular version PTh.

##  5 Conclusion, Limitations, & Future Work

In this paper, we presented three means of fabrication for implementing a
resistive force sensor in flatbed weft knitting machines with a minimum of two
beds. The chosen knitting pattern enables to knit the sensing part on one bed
entirely, which allows for combinations with a supporting substrate. Our
method therefore provides the possibility to hide away the functional part for
aesthetic and/or protective purposes. Based on these methods, we presented and
evaluated 10 variations, 5 of which used a PES-only substrate material. The
remaining 5 combined PES with Lycra to improve their physical properties.

We do acknowledge a few limitations of this study: first, we did not evaluate
all possible combinations of nominal stitch numbers (cf. Table 1), instead we
chose settings driven by subjective measures of quality. We did this to keep
the number of patches reasonably low. We did however experiment with other
compositions and plies which were moderately successful, and only presented
the most relevant ones in this paper.

Second, the stitch numbers for PES differ slightly for PL1* patches, which
seems like they are not objectively comparable. We justify this with the
change in haptic quality when new material of a different type is introduced
into the composition, which requires stitch numbers to be adjusted
accordingly, hence, we went for comparable haptic quality.

Third, we did not test all of our sensors to their full saturation, i.e., did
not cover the entire working range. We did this since in our use case, 20 N
was way beyond the expected upper limit, however we noticed especially during
the tests with increasing actuation force, that some variations perform badly
beyond this value. This connects to a limitation of our Lycra-patches: offsets
and hysteresis depending on actuation amplitude pose a challenge in general,
since there seems to be temporal data (i.e., the degree of ”past actuation”)
required to infer correct force and/or strain at all times. We plan to
investigate this aspect in future work using a specially trained Artificial
Neuronal Network to act as a special filter. First steps into this direction
already yielded promising results for compensating those temporal effects.

Finally, tests in harsh environmental conditions, such as in highly humid and
extreme temperature scenarios were not performed at this point. Related work
showed that conductive polymer composites can be affected in particular by
high humidity [37], however, we expect that since this is a property of the
material, the knitting structure does not have a profound influence in that
regard. Since the sensing parts are entirely replaceable by similar products
and our work was focusing on structural compositions and consistency benefits
from adding Lycra material, we refrained from evaluating the specific
materials that we used for our implementation, since we expect our results
would reasonably translate to arbitrary resistive and conductive yarn.

## Acknowledgment

This research is part of the COMET project TextileUX (No. 865791, which is
funded within the framework of COMET – Competence Centers for Excellent
Technologies by BMVIT, BMDW, and the State of Upper Austria. The COMET program
is handled by the FFG.

## References

  * [1] D. J. Spencer, Knitting Technology: A Comprehensive Handbook and Practical Guide.  Woodhead Publishing, 3 ed., 2001.
  * [2] Y. Luo, X. Chen, X. Li, H. Tian, S. Li, L. Wang, J. He, Z. Yang, and J. Shao, “Heterogeneous strain distribution based programmable gated microchannel for ultrasensitive and stable strain sensing,” Advanced Materials, vol. 35, no. 2, p. 2207141, 2023.
  * [3] S. Li, X. Chen, X. Li, H. Tian, C. Wang, B. Nie, J. He, and J. Shao, “Bioinspired robot skin with mechanically gated electron channels for sliding tactile perception,” Science Advances, vol. 8, no. 48, p. eade0720, 2022.
  * [4] X. Chen, J. Shao, H. Tian, X. Li, C. Wang, Y. Luo, and S. Li, “Scalable imprinting of flexible multiplexed sensor arrays with distributed piezoelectricity-enhanced micropillars for dynamic tactile sensing,” Advanced Materials Technologies, vol. 5, no. 7, p. 2000046, 2020.
  * [5] J. Shao, X. Chen, X. Li, H. Tian, C. Wang, and B. Lu, “Nanoimprint lithography for the manufacturing of flexible electronics,” Science China Technological Sciences, vol. 62, no. 2, pp. 175–198, 2019.
  * [6] O. Atalay, W. R. Kennon, and E. Demirok, “Weft-knitted strain sensor for monitoring respiratory rate and its electro-mechanical modeling,” IEEE Sensors Journal, vol. 15, no. 1, pp. 110–122, 2015.
  * [7] E. Ayodele, T. Bao, S. A. R. Zaidi, A. M. A. Hayajneh, J. Scott, Z.-Q. Zhang, and D. McLernon, “Grasp classification with weft knit data glove using a convolutional neural network,” IEEE Sensors Journal, vol. 21, no. 9, pp. 10824–10833, 2021.
  * [8] Y. Li, X. Miao, J. Y. Chen, G. Jiang, and Q. Liu, “Sensing performance of knitted strain sensor on two-dimensional and three-dimensional surfaces,” Materials & Design, vol. 197, p. 109273, 2021.
  * [9] M. Åkerfeldt, A. Lund, and P. Walkenström, “Textile sensing glove with piezoelectric pvdf fibers and printed electrodes of pedot:pss,” Textile Research Journal, vol. 85, no. 17, pp. 1789–1799, 2015.
  * [10] X. Han, X. Miao, X. Chen, G. Jiang, and L. Niu, “Research on finger movement sensing performance of conductive gloves,” Journal of Engineered Fibers and Fabrics, vol. 14, p. 1558925019887622, 2019.
  * [11] S. Lee, Y. Choi, M. Sung, J. Bae, and Y. Choi, “A knitted sensing glove for human hand postures pattern recognition,” Sensors, vol. 21, no. 4, 2021\.
  * [12] A. Grassi, F. Cecchi, M. Maselli, M. Röling, C. Laschi, and M. Cianchetti, “Warp-knitted textile as a strain sensor: Characterization procedure and application in a comfortable wearable goniometer,” IEEE Sensors Journal, vol. 17, no. 18, pp. 5927–5936, 2017.
  * [13] A. Liang, R. Stewart, and N. Bryan-Kinns, “Analysis of sensitivity, linearity, hysteresis, responsiveness, and fatigue of textile knit stretch sensors.,” Sensors (Basel), 8 2019.
  * [14] X. Lin and B.-C. Seet, “A linear wide-range textile pressure sensor integrally embedded in regular fabric,” IEEE Sensors Journal, vol. 15, no. 10, pp. 5384–5385, 2015.
  * [15] S. Dinparast Tohidi, A. Zille, A. P. Catarino, and A. M. Rocha, “Effects of base fabric parameters on the electro-mechanical behavior of piezoresistive knitted sensors,” IEEE Sensors Journal, vol. 18, no. 11, pp. 4529–4535, 2018.
  * [16] I. Wicaksono, P. G. Hwang, S. Droubi, F. X. Wu, A. N. Serio, W. Yan, and J. A. Paradiso, “3dknits: Three-dimensional digital knitting of intelligent textile sensor for activity recognition and biomechanical monitoring,” in 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 2403–2409, 2022.
  * [17] H. Zhang, X. Tao, T. Yu, and S. Wang, “Conductive knitted fabric as large-strain gauge under high temperature,” Sensors and Actuators A: Physical, vol. 126, no. 1, pp. 129–140, 2006.
  * [18] B. Bozali, J. J. F. van Dam, L. Plaude, and K. M. B. Jansen, “Development of hysteresis-free and linear knitted strain sensors for smart textile applications,” in 2021 IEEE Sensors, pp. 1–4, 2021.
  * [19] K. Yang, G.-l. Song, L. Zhang, and L.-w. Li, “Modelling the electrical property of 1×1 rib knitted fabrics made from conductive yarns,” in 2009 Second International Conference on Information and Computing Science, vol. 4, pp. 382–385, 2009.
  * [20] A. Ehrmann, F. Heimlich, A. Brücken, M. Weber, and R. Haug, “Suitability of knitted fabrics as elongation sensors subject to structure, stitch dimension and elongation direction,” Textile Research Journal, vol. 84, no. 18, pp. 2006–2012, 2014.
  * [21] X. Chen, B. Li, Y. Qiao, and Z. Lu, “reparing polypyrrole-coated stretchable textile via low-temperature interfacial polymerization for highly sensitive strain sensor.,” Micromachines (Basel), 11 2019.
  * [22] E. Ayodele, S. A. R. Zaidi, J. Scott, Z. Zhang, M. Hafeez, and D. McLernon, “The effect of miss and tuck stitches on a weft knit strain sensor,” Sensors, vol. 21, no. 2, 2021.
  * [23] X. Liang, H. Cong, Z. Dong, and G. Jiang, “Size prediction and electrical performance of knitted strain sensors,” Polymers, vol. 14, no. 12, 2022\.
  * [24] S. Jang, J. Y. Choi, E. S. Yoo, D. Y. Lim, J. Y. Lee, J. K. Kim, and C. Pang, “Printable wet-resistive textile strain sensors using bead-blended composite ink for robustly integrative wearable electronics,” Composites Part B: Engineering, vol. 210, p. 108674, 2021.
  * [25] Y. Teyeme, B. Malengier, T. Tesfaye, and L. Van Langenhove, “A fabric-based textile stretch sensor for optimized measurement of strain in clothing,” Sensors, vol. 20, no. 24, 2020.
  * [26] D. De Rossi, F. Carpi, F. Lorussi, A. Mazzoldi, E. Scilingo, and A. Tognetti, “Electroactive fabrics for distributed, conformable and interactive systems,” in SENSORS, 2002 IEEE, vol. 2, pp. 1608–1613 vol.2, 2002.
  * [27] G. Semjonova, A. Davidovica, N. Kozlovskis, A. Okss, and A. Katashevs, “Smart textile sock system for athletes’ self-correction during functional tasks: Formative usability evaluation,” Sensors, vol. 22, no. 13, 2022.
  * [28] ”Holm, Ragnar”, ”Electric Contacts: Theory and Application”.  ”Heidelberg”: ”Springer Berlin”, 4 ed., 1967.
  * [29] L. Li, W. M. Au, K. M. Wan, S. H. Wan, W. Y. Chung, and K. S. Wong, “A resistive network model for conductive knitting stitches,” Textile Research Journal, vol. 80, no. 10, pp. 935–947, 2010.
  * [30] S. Liu, C. Yang, Y. Zhao, X. ming Tao, J. Tong, and L. Li, “The impact of float stitches on the resistance of conductive knitted structures,” Textile Research Journal, vol. 86, no. 14, pp. 1455–1473, 2016.
  * [31] N. Baribina, A. Oks, I. Baltina, and P. Eizentals, “Comparative analysis of knitted pressure sensors,” 05 2018.
  * [32] R. Aigner, M. A. Haberfellner, and M. Haller, “SpaceR: Knitting ready-made, tactile, and highly responsive spacer-fabric force sensors for continuous input,” in Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22, (New York, NY, USA), Association for Computing Machinery, 2022.
  * [33] A. Pointner, T. Preindl, S. Mlakar, R. Aigner, M. A. Haberfellner, and M. Haller, “Knitted force sensors,” in Adjunct Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST ’22 Adjunct, (New York, NY, USA), Association for Computing Machinery, 2022.
  * [34] C. Isaia, S. A. McMaster, and D. McNally, “Study of performance of knitted conductive sleeves as wearable textile strain sensors for joint motion tracking,” in 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 4555–4558, 2020.
  * [35] J. Ou, D. Oran, D. D. Haddad, J. Paradiso, and H. Ishii, “Sensorknit: Architecting textile sensors with machine knitting,” 3D Printing and Additive Manufacturing, vol. 6, no. 1, pp. 1–11, 2019.
  * [36] K. M. Jansen, “Performance evaluation of knitted and stitched textile strain sensors,” Sensors, vol. 20, no. 24, 2020.
  * [37] C. Cochrane, V. Koncar, M. Lewandowski, and C. Dufour, “Design and development of a flexible strain sensor for textile structures based on a conductive polymer composite,” Sensors, vol. 7, no. 4, pp. 473–492, 2007.


# SU-NLP at SemEval-2022 Task 11: Complex Named Entity Recognition with Entity
Linking

Buse Çarık∗, Fatih Beyhan∗, Reyyan Yeniterzi
Sabancı University
İstanbul, Turkey
<EMAIL_ADDRESS>
###### Abstract

This paper describes the system proposed by Sabancı University Natural
Language Processing Group in the SemEval-2022 MultiCoNER task. We developed an
unsupervised entity linking pipeline that detects potential entity mentions
with the help of Wikipedia and also uses the corresponding Wikipedia context
to help the classifier in finding the named entity type of that mention. Our
results showed that our pipeline improved performance significantly,
especially for complex entities in low-context settings.

**footnotetext: These authors contributed equally to this work.

##  1 Introduction

Named Entity Recognition (NER) is a widely studied task of Natural Language
Processing. Recent leading architectures achieved impressive performances,
especially in formal contexts (like news articles, etc.) and for commonly
worked named entity types (like Person, Organization, etc.). In Multilingual
Complex Named Entity Recognition task, organizers introduced a dataset which
is a large collection of short texts that includes complex entities. As they
stated, complex entities can be noun phrases, gerunds, or full clauses Malmasi
et al. (2022). The participants were challenged to develop a NER system for
identifying these complex entities in short and low-context settings.

Recent contextual neural architectures like transformers work quite well on a
variety of NLP tasks and datasets. However, they are strongly dependent on the
context of the given text. In case of lack of context, these models alone may
not gather enough evidence to make a correct prediction. When that is the
case, getting help from knowledge bases can be useful. Dataset provided with
this task has similar low-context characteristics together with some complex
named entity types to be resolved. Therefore, in this work, we investigated
whether performing entity linking to provide additional context for a
transformer model can be useful for identifying complex named entities. We
initially used an unsupervised entity linking approach to identify the
possible entity mentions and their corresponding Wikipedia pages. Later on, we
used the additional context retrieved from Wikipedia to predict the types of
the identified entities. Our results show that performing entity linking prior
to the classifier is beneficial in the case of complex entities in short and
low-context settings.

##  2 Related Work

NER is an important and also well-studied task in NLP. Similar to other NLP
tasks, recent neural architectures provided significant improvements in NER as
well. However, since these models are heavily dependent on context
information, applying them to short, noisy texts such as very short social
media posts or web queries results in significant performance degradation Meng
et al. (2021).

To overcome the lack of context in these short texts, as well as the code-
mixed terms, it has been prominent to use external knowledge in neural
approaches. Recent studies Meng et al. (2021); Fetahu et al. (2021) combining
multilingual transformer models with gazetteers have shown remarkable
improvement in performance. Another study Yamada et al. (2015) utilized the
Entity Linking task to address the issues of Twitter NER by detecting entity
mentions from knowledge bases like Wikipedia.

In this study, we also focused on external information due to the lack of
context. However, instead of using a static auxiliary like gazetteers, we
tried to detect entities dynamically from the continuously extending Wikipedia
by using a search engine architecture. We integrated the retrieved Wikipedia
content into the input representation in order to improve the performance of
entity type classification.

##  3 Data

Malmasi et al. (2022) introduced a new multilingual NER dataset consisting of
short texts for eleven languages. The dataset consists of 6 named entity
classes which are Person, Location, Group, Corporation, Product, and Creative
Work.

In this paper, even though we propose a language-independent approach, due to
limited resources we focused on the Turkish part of the dataset only.
Statistics of the dataset for Turkish are shown in Table 1. As can be seen
from Table 1, even though the number of samples is much higher, the length of
the samples is significantly lower in the test set compared to train and
validation splits. This short length in context may cause recent state-of-the-
art contextual models to perform worse than expected.

| # Samples | Avg. # Tokens
---|---|---
Train | 15,300 | 14.27
Validation | 800 | 14.27
Test | 136,935 | 5.28
Table 1: Distribution of samples and average number of tokens per sample for
Turkish part

An issue we found with the dataset is that in some examples, a phrase is
labeled as OTHER although there are similar phrases that are labeled as a
named entity. Consider the following example:

"Nokia 1011, 1994’e kadar, Nokia 2010 ve Nokia 2110’un halefler olarak
tanıtılmasıyla üretime devam etti." English: "Nokia 1011 continued in
production until 1994, with the introduction of the Nokia 2010 and Nokia 2110
as successors."

The text includes three different cellphone models produced by Nokia. Two of
these models (the blue ones) were annotated as PRODUCT; however, Nokia 1011
was mislabeled as OTHER. A more major issue is when the same named entity
occurs more than once in a sentence, but their annotations are different. An
example is illustrated below:

"Whitney Houston yine bu sene içinde New Jack Swing’e adını yazdırdı, I’m Your
Baby Tonight adlı şarkı Whitney Houston’a başarı üstüne başarı getirmiştir."
English: "Whitney Houston made her name on New Jack Swing again this year, the
song I’m Your Baby Tonight brought Whitney Houston success after success."

In the example, Whitney Houston in blue was labeled as PERSON; but, the red
Whitney Houston was labeled as OTHER. These types of wrong annotations cause
ambiguities and learning problems in supervised approaches.

##  4 Methodology

In order to overcome the challenges described in previous sections, we propose
an unsupervised entity linking pipeline followed by a classifier as our
proposed NER architecture. The entity linking pipeline is designed to detect
potential entity mentions and retrieve corresponding candidate documents for
each mention. Later, these detected mentions and the linked content is used
together in a classifier to detect the named entity type of the entity
mention. A pre-trained BERT model was also fine-tuned for the Named Entity
Recognition task in order to create a strong baseline for comparison.

Figure 1: Overall pipeline of the system which is showing the path of sentence
$S_{i}$.

###  4.1 BERT

The BERT Devlin et al. (2018) model was adopted as a baseline approach since
transformer models outperform other approaches in many NLP tasks. We fine-
tuned the BERTurk Schweter (2020) model pre-trained on large Turkish corpora
with a feed-forward layer on top to classify named entities. Base model, in
which each feed-forward layer has 12 encoder layers and 768 hidden units, was
used. Since every query in all datasets consists of lowercase letter, uncased
version of BERTurk was preferred.

###  4.2 NER with Entity Linking

BERT like transformer models heavily depend on the context. Therefore having a
very short context like a couple of words, not even a sentence will cause
problems with these models. Especially when the task is to identify complex
named entities like creative work or product, it is much more challenging. For
very short context, even human beings may have hard time resolving entities
and their types.

In order to overcome this challenge an additional help from a gazetteer or a
knowledge base will be very useful. Therefore in this paper we propose using
Wikipedia in order to both identify entities and also their types. Our
proposed architecture which is depicted in Figure 1 consists of three steps.
The first two steps work towards identifying possible entities and linking
them to their corresponding Wikipedia articles. In this unsupervised entity
linking approach as the first step the original input text is used to retrieve
relevant Wikipedia pages which are possible candidate pages for entities. In
the second step content within the retrieved documents (like their titles) are
used to identify the entity mentions in the original text. Finally, in the
third step each identified mention and its linked Wikipedia article is used
together to predict the type of the entity.

####  4.2.1 Candidate Generation

ElasticSearch***https://www.elastic.co/ was used as the underlying search
engine architecture. The most recent Wikipedia dump was used to create an
index. For each document $D_{i}$, the title, referred_by and interwikies were
indexed to create the following four fields:

  * •

title: This is the title of the document $D_{i}$.

  * •

referred_by: This is a set of text spans which are parsed from other Wikipedia
pages where $D_{i}$ is being referred with a interwiki link. This set includes
all of these text spans sorted from longest to shortest. The title is added to
the set as well.

  * •

interwikies: This is the list of interwikies***Interwikies are the types of
hyperlinks which is linking a text span from a Wikipedia page to another
Wikipedia page. For instance, Wikipedia pages of Earth, Mercury and Venus are
in the interwikies section of the Sun’s Wikipedia page, since they are
mentioned within its text. in the document $D_{i}$.

  * •

all_text: This one is a concatenation of all fields (title, text content,
interwikies, referred_by, and categories***Categories section of a Wikipedia
page includes a list of topics which is related to the Wikipedia page.) of the
document $D_{i}$.

Each sample was queried and the most relevant 200 articles were retrieved.
Later on for each query, documents retrieved with different field searches
were pooled together. The reason we are pooling these different field results
is to make sure we do not miss a possibly relevant article which could have
been linked.

For the train and validation tests, around 199 documents were retrieved for
each field. After pooling, we ended up with a pool of size 578 documents on
average. Since test queries are shorter, the average number of documents
returned got as low as 161 with the title field and 465 over the pool. Short
queries also affected the empty ones, in other words, no document retrieved
samples. After pooling there was not such a case for train and validation,
however even pooling could not help to the 169 queries over 136,935 samples,
and no Wikipedia articles were retrieved for those.

Field | Train | Validation
---|---|---
title | 96.52 | 97.00
interwikies | 57.16 | 61.26
referred_by | 96.46 | 96.11
all_text | 86.50 | 86.68
After pooling | 98.89 | 98.85
Table 2: Recall Scores over Detected Mentions

####  4.2.2 Mention Detection and Linking

After generating a set of candidate documents for each input text, the next
step is to map these documents to the possible entity mentions in the input.
For each input, our proposed algorithm iterated over the retrieved and pooled
documents {D}, and cross-checked each item in the referred_by field of the
document to see if there was an exact match to any phrase in the input text.
In case there was an exact match, then that matched phrase was tagged as a
possible entity and linked to the corresponding Wikipedia page. If there were
multiple matches for a span of text, then the longest match was considered and
shorter ones were ignored. Furthermore, if there was a tie in terms of length,
then the relevance score of the Wikipedia document was used. That relevance
score was calculated after pooling by doing a summation of the relevance
scores from different field retrievals.

In order to see which fields in Wikipedia are more useful for mention
detection, recall was calculated over detected entity mentions, without
considering the type of entity. The results are summarized in Table 2. Based
on the results title and referred_by are the most useful fields and pooling
all retrievals is the best over both train and development sets.

####  4.2.3 Named Entity Type Detection

After linking the entities, the only thing remaining is to find their NER
type. In order to do that we used the original context where the entity was
mentioned in the input text and the linked Wikipedia article content. We
combined these two as follows:

  * [CLS]ctxl[SEP]M[SEP]ctxr[SEP]WP[SEP]

where ctxl and ctxr are tokens for left and right context of mention, M is
mention itself and WP is the first two paragraphs of a Wikipedia page. [SEP]
tokens are used to tag the mention. The maximum length of the representation
is set 256. This representation is given to BERT for encoding.

We used two classifier heads following the BERT encoder for classifying the
pairs. The architectural design is presented in Figure 2. The first one is a
binary classifier head which aims to solve an auxiliary task of whether the
given candidate entity mention is a real entity or not with the help of input
context tokens and Wikipedia content. The second classifier head is trying to
learn the type of the candidate named entity. The loss function for our model
is the summation of the two losses from these classifier heads. In case the
binary classifier head returns O (which means the given candidate entity
mention is not an entity), the second classifier head’s decision is ignored,
and given candidate entity mention is directly predicted as O (which stands
for Other, not an entity). In order to test the effectiveness of multiple
heads, we created another model which has a single classifier head only for
identifying named entity types.

Figure 2: The architecture of the classifier. NE in the binary classifier
head’s output stands for Named Entity and O for Other.

##  5 Experimental Setup

Models | Validation | Test
---|---|---
Precision | Recall | F1 | Precision | Recall | F1
BERT | 85.84 | 85.01 | 85.39 | 58.02 | 58.39 | 57.82
EL_BERT | 84.59 | 79.40 | 81.74 | 74.78 | 59.03 | 65.47
EL_MultiBERT | 86.18 | 77.56 | 81.60 | 80.69 | 65.13 | 71.91
EnsembleToken+EL | 83.78 | 85.35 | 84.47 | 74.23 | 59.44 | 65.66
EnsembleToken+EL_Multi | 83.66 | 85.51 | 84.49 | 78.86 | 66.43 | 72.02
Table 3: Results of our proposed methods on development and test data

Due to limited resources and time, we only experimented with the Turkish part
of the dataset and therefore indexed 749,204 Turkish Wikipedia pages. During
the retrieval OR operator was used as part of the query and the default BM25
was used for ranking.

We fine-tuned the BERTurk model***https://huggingface.co/dbmdz/bert-base-
turkish-uncased. As the optimizer, we used AdamW Loshchilov and Hutter (2017)
with a fixed learning rate in all our BERT models as 3 x $10^{-5}$. We also
applied dropout before the feed-forward layer with 0.3 probability. Our batch
size was 32, with a maximum sequence length of 128 for the baseline BERT. We
extended the maximum sequence length to 256 for EL_MultiBERT and reduced the
batch size to 8. All experiments were conducted with seed 22.

##  6 Results

The official evaluation metric was chosen as macro-average F1-score by the
organizers. In Table 3, we present our results with models described in
Section 4. Our baseline BERT, the one fine-tuned directly for NER, worked
quite well on the validation set but performed badly on the test set. This
substantial difference in these two sets shows that contextual models like
BERT do not perform well in low-context settings.

On the other hand, although our proposed entity linking approaches fell
slightly behind the baseline model in the development set, they performed
significantly better than the baseline BERT in testing. Among these two entity
linking methods, the model with the two classification heads performed better.
While determining the named entity type, training with this auxiliary task
even using the same data seems to be useful.

Entity Type | BERT | EL_MultiBERT
---|---|---
Person | 70.72 | 72.43
Location | 59.73 | 57.42
Group | 51.49 | 76.51
Corporation | 57.69 | 73.79
Product | 63.13 | 76.93
Creative Work | 44.14 | 74.40
Table 4: F1-Scores by NER type on Test Set

Table 4 illustrates the performance of BERT and EL_MultiBERT models for each
named entity class. As can be seen from the table, our entity linking approach
improves the performance, especially in complex entities like Creative Work,
Group and Product. Two approaches returned similar results only for Person and
Location, but since we do not have the gold standard labels, we cannot perform
any detailed analysis of the reason.

###  6.1 Ensemble Approach

Although the majority of the samples in the test set are short texts, there
are some longer sentences as well. From the validation set experiments, we
already know that when there is enough context, transformer models like BERT
perform quite well. Hence, we decided to see if we can use our proposed
architecture as a fall back mechanism when there is not enough context
available, but otherwise use BERT. We applied an ensemble approach relying on
BERT for samples with more than 11 tokens and EL_MultiBERT for shorter ones.
So when sentence length is less than 11, we depended on our entity linking
architecture and Wikipedia for retrieving and using additional useful context
about the input. As shown in Table 3, The ensemble of BERT and EL_MultiBERT
achieved the best result with 72.02% on the test set, and we ranked as third
in the Turkish track.

##  7 Conclusion

In this work, we proposed a language-independent method for the Complex NER
task in low-context settings. Our results showed that utilizing the use of
entity linking for NER provides a significant improvement over state-of-the-
art transformer models when there is not enough context. Yet, since our
resources are limited, we experimented with only the Turkish part. In future
work, we will extend this approach in a multilingual setting.

## References

  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * Fetahu et al. (2021) Besnik Fetahu, Anjie Fang, Oleg Rokhlenko, and Shervin Malmasi. 2021.  Gazetteer Enhanced Named Entity Recognition for Code-Mixed Web Queries.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , pages 1677–1681.
  * Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_.
  * Malmasi et al. (2022) Shervin Malmasi, Anjie Fang, Besnik Fetahu, Sudipta Kar, and Oleg Rokhlenko. 2022\.  MultiCoNER: a Large-scale Multilingual dataset for Complex Named Entity Recognition.
  * Meng et al. (2021) Tao Meng, Anjie Fang, Oleg Rokhlenko, and Shervin Malmasi. 2021.  GEMNET: Effective gated gazetteer representations for recognizing complex entities in low-context input.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1499–1512.
  * Schweter (2020) Stefan Schweter. 2020.  Berturk - bert models for turkish.
  * Yamada et al. (2015) Ikuya Yamada, Hideaki Takeda, and Yoshiyasu Takefuji. 2015.  Enhancing named entity recognition in twitter messages using entity linking.  In _Proceedings of the Workshop on Noisy User-generated Text_ , pages 136–140.