# BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named
Entity Recognition

Yinghao Li1, Pranav Shetty1, Lucas Liu1, Chao Zhang1 Le Song2
1 Georgia Institute of Technology, Atlanta, USA
2 Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United
Arab Emirates
{yinghaoli, pranav.shetty, lucasliu<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

We study the problem of learning a named entity recognition (NER) tagger using
noisy labels from multiple weak supervision sources. Though cheap to obtain,
the labels from weak supervision sources are often incomplete, inaccurate, and
contradictory, making it difficult to learn an accurate NER model. To address
this challenge, we propose a conditional hidden Markov model (CHMM), which can
effectively infer true labels from multi-source noisy labels in an
unsupervised way. CHMM enhances the classic hidden Markov model with the
contextual representation power of pre-trained language models. Specifically,
CHMM learns token-wise transition and emission probabilities from the BERT
embeddings of the input tokens to infer the latent true labels from noisy
observations. We further refine CHMM with an alternate-training approach
(CHMM-ALT). It fine-tunes a BERT-NER model with the labels inferred by CHMM,
and this BERT-NER’s output is regarded as an additional weak source to train
the CHMM in return. Experiments on four NER benchmarks from various domains
show that our method outperforms state-of-the-art weakly supervised NER models
by wide margins.

##  1 Introduction

Named entity recognition (NER), which aims to identify named entities from
unstructured text, is an information extraction task fundamental to many
downstream applications such as event detection (Li et al., 2012),
relationship extraction (Bach and Badaskar, 2007), and question answering
(Khalid et al., 2008). Existing NER models are typically supervised by a large
number of training sequences, each pre-annotated with token-level labels. In
practice, however, obtaining such labels could be prohibitively expensive. On
the other hand, many domains have various knowledge resources such as
knowledge bases, domain-specific dictionaries, or labeling rules provided by
domain experts (Farmakiotou et al., 2000; Nadeau and Sekine, 2007). These
resources can be used to match a corpus and quickly create large-scale noisy
training data for NER from multiple views.

Learning an NER model from multiple weak supervision sources is a challenging
problem. While there are works on distantly supervised NER that use only
knowledge bases as weak supervision (Mintz et al., 2009; Shang et al., 2018;
Cao et al., 2019; Liang et al., 2020), they cannot leverage complementary
information from multiple annotation sources. To handle multi-source weak
supervision, several recent works (Nguyen et al., 2017; Safranchik et al.,
2020; Lison et al., 2020) leverage the hidden Markov model (HMM), by modeling
true labels as hidden variables and inferring them from the observed noisy
labels through unsupervised learning. Though principled, these models fall
short in capturing token semantics and context information, as they either
model input tokens as one-hot observations (Nguyen et al., 2017) or do not
model them at all (Safranchik et al., 2020; Lison et al., 2020). Moreover, the
flexibility of HMM is limited as its transitions and emissions remain constant
over time steps, whereas in practice they should depend on the input words.

We propose the conditional hidden Markov model (CHMM) to infer true NER labels
from multi-source weak annotations. CHMM conditions the HMM training and
inference on BERT by predicting token-wise transition and emission
probabilities from the BERT embeddings. These token-wise probabilities are
more flexible than HMM’s constant counterpart in modeling how the true labels
should evolve according to the input tokens. The context representation
ability they inherit from BERT also relieves the Markov constraint and expands
HMM’s context-awareness.

Further, we integrate CHMM with a supervised BERT-based NER mode with an
alternate-training method (CHMM-ALT). It fine-tunes BERT-NER with the denoised
labels generated by CHMM. Taking advantage of the pre-trained knowledge
contained in BERT, this process aims to refine the denoised labels by
discovering the entity patterns neglected by all of the weak sources. The
fine-tuned BERT-NER serves as an additional supervision source, whose output
is combined with other weak labels for the next round of CHMM training. CHMM-
ALT trains CHMM and BERT-NER alternately until the result is optimized.

Our contributions include:

  * •

A multi-source label aggregator CHMM with token-wise transition and emission
probabilities for aggregating multiple sets of NER labels from different weak
labeling sources.

  * •

An alternate-training method CHMM-ALT that trains CHMM and BERT-NER in turn
utilizing each other’s outputs for multiple loops to optimize the multi-source
weakly supervised NER performance.

  * •

A comprehensive evaluation on four NER benchmarks from different domains
demonstrates that CHMM-ALT achieves a $4.83$ average F1 score improvement over
the strongest baseline models.

The code and data used in this work are available at github.com/Yinghao-
Li/CHMM-ALT.

##  2 Related Work

#### Weakly Supervised NER

There have been works that train NER models with different weak supervision
approaches. Distant supervision, a specific type of weak supervision,
generates training labels from knowledge bases (Mintz et al., 2009; Yang et
al., 2018; Shang et al., 2018; Cao et al., 2019; Liang et al., 2020). But such
a method is limited to one source and falls short of acquiring supplementary
annotations from other available resources. Other works adopt multiple
additional labeling sources, such as heuristic functions that depend on
lexical features, word patterns, or document information (Nadeau and Sekine,
2007; Ratner et al., 2016), and unify their results through multi-source label
denoising. Several multi-source weakly supervised learning approaches are
designed for sentence classification (Ratner et al., 2017, 2019; Ren et al.,
2020; Yu et al., 2020). Although these methods can be adapted for sequence
labeling tasks such as NER, they tend to overlook the internal dependency
relationship between token-level labels during the inference. Fries et al.
(2017) target the NER task, but their method first generates candidate named
entity spans and then classifies each span independently. This independence
makes it suffer from the same drawback as sentence classification models.

A few works consider label dependency while dealing with multiple supervision
sources. Lan et al. (2020) train a BiLSTM-CRF network (Huang et al., 2015)
with multiple parallel CRF layers, each for an individual labeling source, and
aggregate their transitions with confidence scores predicted by an attention
network (Bahdanau et al., 2015; Luong et al., 2015). HMM is a more principled
model for multi-source sequential label denoising as the true labels are
implicitly inferred through unsupervised learning without deliberately
assigning any additional scores. Following this track, Nguyen et al. (2017)
and Lison et al. (2020) use a standard HMM with multiple observed variables,
each from one labeling source. Safranchik et al. (2020) propose linked HMM,
which differs from ordinary HMM by introducing unique linking rules as an
adjunct supervision source additional to general token labels. However, these
methods fail to utilize the context information embedded in the tokens as
effectively as CHMM, and their NER performance is further constrained by the
Markov assumption.

#### Neuralizing the Hidden Markov Model

Some works attempt to neuralize HMM in order to relax the Markov assumption
while maintaining its generative property (Kim et al., 2018). For example, Dai
et al. (2017) and Liu et al. (2018) incorporate recurrent units into the
hidden semi-Markov model (HSMM) to segment and label high-dimensional time
series; Wiseman et al. (2018) learn discrete template structures for
conditional text generation using neuralized HSMM. Wessels and Omlin (2000)
and Chiu and Rush (2020) factorize HMM with neural networks to scale it and
improve its sequence modeling capacity. The work most related to ours
leverages neural HMM for sequence labeling (Tran et al., 2016). CHMM differs
from neural HMM in that the tokens are treated as a dependency term in CHMM
instead of the observation in neural HMM. Besides, CHMM is trained with
generalized EM, whereas neural HMM optimizes the marginal likelihood of the
observations.

Figure 1:  An example of label aggregation with two weak labeling sources. We
use BIO labeling scheme. PER represents person; LOC is location.

Figure 2:  The illustration of the alternate-training method. Phase I is
acyclic, starting from getting $K$ weak labels from supervision sources and
ending at the fine-tuning of BERT-NER with CHMM’s denoised output. Phase II
contains several loops, each trains CHMM with $K+1$ sources, including the
additional BERT predictions from the previous loop, and fine-tunes BERT-NER
using the updated denoised labels.

##  3 Problem Setup

In this section, we formulate the multi-source weakly supervised NER problem.
Consider an input sentence that contains $T$ tokens $\bm{w}^{(1:T)}$, NER can
be formulated as a sequence labeling task that assigns a label to each token
in the sentence.111We represent vectors, matrices or tensors with bold fonts
and scalars with regular fonts; $1:a\triangleq\\{1,2,\dots,a\\}$. Assuming the
set of target entity types is $\mathcal{E}$ and the tagging scheme is BIO
(Ramshaw and Marcus, 1995), NER models assign one label from the label set
$l\in\mathcal{L}$ to each token, where the size of the label set is
$|\mathcal{L}|=2|\mathcal{E}|+1$, _e.g._ , if
$\mathcal{E}=\\{\texttt{PER},\texttt{LOC}\\}$, then
$\mathcal{L}=\\{\texttt{O},\texttt{B-PER},\texttt{I-PER},\texttt{B-LOC},\texttt{I-LOC}\\}$.

Suppose we have a sequence with $K$ weak sources, each of which can be a
heuristic rule, knowledge base, or existing out-of-domain NER model. Each
source serves as a labeling function that generates token-level weak labels
from the input corpus, as shown in Figure 1. For the input sequence
$\bm{w}^{(1:T)}$, we use $\bm{x}^{(1:T)}_{k},k\in\\{1,\dots,K\\}$ to represent
the weak labels from the source $k$, where
$\bm{x}^{(t)}_{k}\in\mathbb{R}^{|\mathcal{L}|},t\in\\{1,\dots,T\\}$ is a
probability distribution over $\mathcal{L}$. Multi-source weakly supervised
NER aims to find the underlying true sequence of labels
$\hat{\bm{y}}^{(1:T)},\hat{y}^{(t)}\in\mathcal{L}$ given
$\\{\bm{w}^{(1:T)},\bm{x}^{(1:T)}_{1:K}\\}$.

##  4 Methodology

In this section, we describe our proposed method CHMM-ALT. We first sketch the
alternate-training procedure (§ 4.1), then explain the CHMM component (§ 4.2)
and how BERT-NER is involved (§ 4.3).

###  4.1 Alternate-Training Procedure

The alternate-training method trains two models—a multi-source label
aggregator CHMM and a BERT-NER model—in turn with each other’s output. CHMM
aggregates multiple sets of labels from different sources into a unified
sequence of labels, while BERT-NER refines them by its language modeling
ability gained from pre-training. The training process is divided into two
phases.

  * •

In phase I, CHMM takes the annotations $\bm{x}^{(1:T)}_{1:K}$ from existing
sources and gives a set of denoised labels ${\bm{y}^{*}}^{(1:T)}$, which are
used to fine-tune the BERT-NER model. Then, we regard the fine-tuned model as
an additional labeling source, whose outputs ${\tilde{\bm{y}}}^{(1:T)}$ are
added into the original weak label sets to give the updated observation
instances:
$\bm{x}^{(1:T)}_{1:K+1}=\\{\bm{x}^{(1:T)}_{1:K},{\tilde{\bm{y}}}^{(1:T)}\\}$.

  * •

In phase II, CHMM and BERT-NER mutually improve each other iteratively in
several loops. Each loop first trains CHMM with the observation
$\bm{x}^{(1:T)}_{1:K+1}$ from the previous one. Then, its predictions are
adopted to fine-tune BERT-NER, whose output updates $\bm{x}^{(1:T)}_{K+1}$.

Figure 2 illustrates the alternate-training method. In general, CHMM gives
high precision predictions, whereas BERT-NER trades recall with precision. In
other words, CHMM can classify named entities with high accuracy but is
slightly disadvantaged in discovering all entities. BERT-NER increases the
coverage with a certain loss of accuracy. Combined with the alternate-training
approach, this complementarity between these models further increases the
overall performance.

Figure 3:  An illustration of CHMM’s architecture. Shaded circles are observed
elements; white circles are hidden elements; rectangles are matrices. Rounded
rectangles are multi-layer perceptrons containing the trainable parameters.
The arrows between $\bm{w}^{(t)}$ and $\bm{e}^{(t)}$ denote the context
representation ability of BERT. MLP denotes the “multi-layer perceptron”.

###  4.2 Conditional Hidden Markov Model

The conditional hidden Markov model is an HMM variant for multi-source label
denoising. It models true entity labels as hidden variables and infers them
from the observed noisy labels. Traditionally, discrete HMM uses one
transition matrix to model the probability of hidden label transitioning and
one emission matrix to model the probability of the observations from the
hidden labels. These two matrices are constant, _i.e._ , their values do not
change over time steps. CHMM, on the contrary, conditions both its transition
and emission matrices on the BERT embeddings $\bm{e}^{(1:T)}$ of the input
tokens $\bm{w}^{(1:T)}$. This design not only allows CHMM to leverage the rich
contextual representations of the BERT embeddings but relieves the constant
matrices constraint as well.

In phase I, CHMM takes $K$ sets of weak labels from the provided $K$ weak
labeling sources. In phase II, in addition to the existing sources, it takes
another set of labels from the previously fine-tuned BERT-NER, making the
total number of sources $K+1$. For convenience, we use $K$ as the number of
weak sources below.

#### Model Architecture

Figure 3 shows a sketch of CHMM’s architecture.222We relax plate notation here
to present details. $\bm{z}^{(1:T)}$ denotes the discrete hidden states of
CHMM with $z^{(t)}\in\mathcal{L}$, representing the underlying true labels to
be inferred from multiple weak annotations.
$\bm{\Psi}^{(t)}\in\mathbb{R}^{|\mathcal{L}|\times|\mathcal{L}|}$ is the
transition matrix, whose element
$\Psi^{(t)}_{i,j}=p(z^{(t)}=j|z^{(t-1)}=i,\bm{e}^{(t)}),i,j\in\\{1,\dots,|\mathcal{L}|\\}$
denotes the probability of moving from label $i$ to label $j$ at time step
$t$. $\bm{\Phi}^{(t)}_{k}\in\mathbb{R}^{|\mathcal{L}|\times|\mathcal{L}|}$ is
the emission matrix of weak source $k$, each element in which
$\Phi^{(t)}_{i,j,k}=p(x^{(t)}_{j,k}=1|z^{(t)}=i,\bm{e}^{(t)})$ represents the
probability of source $k$ observing label $j$ when the hidden label is $i$ at
time step $t$.

For each step, $\bm{e}^{(t)}\in\mathbb{R}^{d_{\rm emb}}$ is the output of a
pre-trained BERT with $d_{\rm emb}$ being its embedding dimension.
$\bm{\Psi}^{(t)}$ and $\bm{\Phi}^{(t)}_{1:K}$ are calculated by applying a
multi-layer perceptron (MLP) to $\bm{e}^{(t)}$:

$\displaystyle\bm{s}^{(t)}\in\mathbb{R}^{|\mathcal{L}|^{2}}={\rm
MLP}(\bm{e}^{(t)}),$ (1)
$\displaystyle\bm{h}^{(t)}\in\mathbb{R}^{|\mathcal{L}|\cdot|\mathcal{L}|\cdot
K}={\rm MLP}(\bm{e}^{(t)}).$ (2)

Since the MLP outputs are vectors, we need to reshape them to matrices or
tensors:

$\displaystyle\bm{S}^{(t)}\in\mathbb{R}^{|\mathcal{L}|\times|\mathcal{L}|}={\rm
reshape}(\bm{s}^{(t)}),$ (3)
$\displaystyle\bm{H}^{(t)}\in\mathbb{R}^{|\mathcal{L}|\times|\mathcal{L}|\times
K}={\rm reshape}(\bm{h}^{(t)}).$ (4)

To achieve the proper probability distributions, we apply the Softmax function
along the label axis so that these values are positive and sum up to $1$:

$\bm{\Psi}^{(t)}_{i,1:|\mathcal{L}|}=\sigma({\bm{S}}^{(t)}_{i,1:|\mathcal{L}|}),\
\bm{\Phi}^{(t)}_{i,1:|\mathcal{L}|,k}=\sigma({\bm{H}}^{(t)}_{i,1:|\mathcal{L}|,k}),$

where

$\sigma(\bm{a})_{i}=\frac{\exp{(a_{i})}}{\sum_{j}\exp{(a_{j})}}.$ (5)

$\bm{a}$ is an arbitrary vector. The formulae in the following discussion
always depend on $\bm{e}^{(1:T)}$, but we will omit the dependency term for
simplicity.

#### Model Training

According to the generative process of CHMM, the joint distribution of the
hidden states and the observed weak labels for one sequence
$p(\bm{z}^{(0:T)},\bm{x}^{(1:T)}|\bm{\theta})$ can be factorized as:

$\displaystyle
p(\bm{z}^{(0:T)},\bm{x}^{(1:T)}|\bm{\theta})=p(z^{(0)})p(\bm{x}^{(1:T)}|\bm{z}^{(1:T)})$
(6)
$\displaystyle\quad=p(z^{(0)})\prod_{t=1}^{T}p(z^{(t)}|z^{(t-1)})\prod_{t=1}^{T}p(\bm{x}^{(t)}|z^{(t)}),$

where $\bm{\theta}$ represents all the trainable parameters.

HMM is generally trained with an expectation-maximization (EM, also known as
Baum-Welch) algorithm. In the expectation step (E-step), we compute the
expected complete data log likelihood:

$Q(\bm{\theta},\bm{\theta}^{\rm
old})\triangleq\mathbb{E}_{\bm{z}}[\ell_{c}(\bm{\theta})|\bm{\theta}^{\rm
old}].$ (7)

$\bm{\theta}^{\rm old}$ is the parameters from the previous training step,
$\mathbb{E}_{\bm{z}}[\cdot]$ is the expectation over variable $\bm{z}$, and

$\ell_{c}(\bm{\theta})\triangleq\log
p(\bm{z}^{(0:T)},\bm{x}^{(1:T)}|\bm{\theta})$

is the comptelete data log likelihood. Let
$\bm{\varphi}^{(t)}\in\mathbb{R}^{|\mathcal{L}|}$ be the observation
likelihood where

$\varphi^{(t)}_{i}\triangleq
p(\bm{x}^{(t)}|z^{(t)}=i)=\prod_{k=1}^{K}\sum_{j=1}^{|\mathcal{L}|}\Phi^{(t)}_{i,j,k}x^{(t)}_{j,k}.$
(8)

Combining (6)–(8) together, we have

$\displaystyle\ Q(\bm{\theta},\bm{\theta}^{\rm
old})=\sum_{i=1}^{|\mathcal{L}|}\gamma^{(0)}_{i}\log\pi_{i}+$ (9)
$\displaystyle\sum_{t=1}^{T}\sum_{i=1}^{|\mathcal{L}|}\sum_{j=1}^{|\mathcal{L}|}\xi^{(t)}_{i,j}\log\Psi^{(t)}_{i,j}+\sum_{t=1}^{T}\sum_{i=1}^{|\mathcal{L}|}\gamma^{(t)}_{i}\log\varphi^{(t)}_{i},$

where $\pi_{1}=1,\bm{\pi}_{2:|\mathcal{L}|}=\bm{0}$;333This assumes the
initial hidden state is always O. In practice, we set
$\pi_{\ell}=\epsilon,\forall\ell\in 2:|\mathcal{L}|$ and
$\pi_{1}=1-(|\mathcal{L}|-1)\epsilon$, where $\epsilon$ is a small value, to
avoid getting $-\infty$ from log. $\gamma^{(t)}_{i}\triangleq
p(z^{(t)}=i|\bm{x}^{(1:T)})$ is the smoothed marginal;
$\xi^{(t)}_{i,j}\triangleq p(z^{(t-1)}=i,z^{(t)}=j|\bm{x}^{(1:T)})$ is the
expected number of transitions. These parameters are computed using the
forward-backward algorithm.444Details are presented in § A.1.

In the maximization step (M-step), traditional HMM updates parameters
${\bm{\theta}}_{\rm HMM}=\\{\bm{\Psi},\bm{\Phi},\bm{\pi}\\}$ by optimizing (7)
with pseudo-statistics.555Details are presented in § A.2. However, as the
transitions and emissions in CHMM are not standalone parameters, we cannot
directly optimize CHMM by this method. Instead, we update the model parameters
through gradient descent _w.r.t._ $\bm{\theta}_{\rm{CHMM}}$ using (9) as the
objective function:

$\nabla\bm{\theta}_{\rm{CHMM}}=\frac{\partial
Q(\bm{\theta}_{\rm{CHMM}},\bm{\theta}_{\rm{CHMM}}^{\rm
old})}{\partial\bm{\theta}_{\rm{CHMM}}}.$ (10)

In practice, the calculation is conducted in the logarithm domain to avoid the
loss of precision issue that occurs when the floating-point numbers become too
small.

To solve the label sparsity issue, _i.e._ , some entities are only observed by
a minority of the weak sources, we modify the observations $\bm{x}^{(1:T)}$
before training. If one source $k$ observes an entity at time step $t$:
$x^{(t)}_{j\neq 1,k}>0$, the observation of non-observing sources at $t$ will
be modified to $\bm{x}^{(t)}_{1,\kappa}=\epsilon;\bm{x}^{(t)}_{j\neq
1,\kappa}=(1-\epsilon)/|\mathcal{L}|,\forall\kappa\in\\{1,\dots,K\\}\backslash
k$, where $\epsilon$ is an arbitrary small value. Note that
$x_{1,\kappa}^{(t)}$ corresponds to the observed label O.

#### CHMM Initialization

Generally, HMM has its transition and emission probabilities initialized with
the statistics $\bm{\Psi}^{*}$ and $\bm{\Phi}^{*}$ computed from the
observation set. But it is impossible to directly set $\bm{\Psi}^{(t)}$ and
$\bm{\Phi}^{(t)}$ in CHMM to these values, as these matrices are the output of
the MLPs rather than standalone parameters. To address this issue, we choose
to pre-train the MLPs before starting CHMM’s training by minimizing the mean
squared error (MSE) loss between their outputs and the target statistics:

$\ell_{\rm
MSE}=\frac{1}{T}\sum_{t}\|\bm{\Psi}^{*}-\bm{S}^{(t)}\|_{F}^{2}+\|\bm{\Phi}^{*}-\bm{H}^{(t)}\|_{F}^{2},$

where $\|\cdot\|_{F}$ is the Frobenius norm. Right after initialization, MLPs
can only output similar probabilities for all time steps:
$\bm{\Psi}^{(t)}\approx\bm{\Psi}^{*},\ \bm{\Phi}^{(t)}\approx\bm{\Phi}^{*},\
\forall t\in\\{1,2,\dots,T\\}$. But their token-wise prediction divergence
will emerge when CHMM has been trained. The initial hidden state $z^{(0)}$ is
fixed to O as it has no corresponding token.

#### Inference

Once trained, CHMM can provide the most probable sequence of hidden labels
$\hat{\bm{z}}^{(1:T)}$ along with the probabilities of all labels
${\bm{y}^{*}}^{(1:T)}$.

$\displaystyle\hat{\bm{z}}^{(1:T)}=\operatorname*{arg\,max}_{{\bm{z}}^{(1:T)}}p_{\hat{\bm{\theta}}_{\rm{CHMM}}}({\bm{z}}^{(1:T)}|\bm{x}^{(1:T)}_{1:K},\bm{e}^{(1:T)}),$
$\displaystyle{y^{*}}^{(t)}_{i}=p_{\hat{\bm{\theta}}_{\rm{CHMM}}}(z^{(t)}=i|\bm{x}^{(1:T)}_{1:K},\bm{e}^{(1:T)}),$

where $\hat{\bm{\theta}}_{\rm{CHMM}}$ represents the trained parameters. These
results can be calculated by either the Viterbi decoding algorithm (Viterbi,
1967) or directly maximizing the smoothed marginal $\bm{\gamma}^{(1:T)}$.

###  4.3 Improving Denoised Labels with BERT

The pre-trained BERT model encodes semantic and structural knowledge, which
can be distilled to further refine the denoised labels from CHMM.
Specifically, we construct the BERT-NER model by stacking a feed-forward layer
and a Softmax layer on top of the original BERT to predict the probabilities
of the classes that each token belongs to (Sun et al., 2019). The probability
predictions of CHMM, ${\bm{y}^{*}}^{(1:T)}$, often referred to as soft labels,
are chosen to supervise the fine-tuning procedure. Compared with the hard
labels $\hat{\bm{z}}^{(1:T)}$, soft labels lead to a more stable training
process and higher model robustness (Thiel, 2008; Liang et al., 2020).

We train BERT-NER by minimizing the Kullback-Leibler divergence (KL
divergence) between the soft labels $\bm{y}^{*}$ and the model output
$\bm{y}$:

$\displaystyle\hat{\bm{\theta}}_{\rm BERT}$
$\displaystyle=\operatorname*{arg\,min}_{{\bm{\theta}}_{\rm
BERT}}\mathcal{D}[{\bm{y}^{*}}^{(1:T)}\|\bm{y}^{(1:T)}]$ (11)
$\displaystyle=\operatorname*{arg\,min}_{{\bm{\theta}}_{\rm
BERT}}\sum_{t=1}^{T}\sum_{i=1}^{|\mathcal{L}|}{y^{*}}^{(t)}_{i}\log\frac{{y^{*}}^{(t)}_{i}}{{y}_{i}^{(t)}},$

where ${\bm{\theta}}_{\rm BERT}$ denotes all the trainable parameters in the
BERT model. BERT-NER does not update the embeddings $\bm{e}^{(1:T)}$ that CHMM
depends on.

We obtain the refined labels
${\tilde{\bm{y}}}^{(1:T)}\in\mathbb{R}^{T\times|\mathcal{L}|}$ from the fine-
tuned BERT-NER directly through a forward pass. Different from CHMM, we
continue BERT-NER’s training with parameter weights from the last loop’s
checkpoint so that the model is initialized closer to the optimum.
Correspondingly, phase II trains BERT-NER with a smaller learning rate, fewer
epoch iterations, and batch gradient descent instead of the mini-batch
version.666Hyper-parameter values are listed in appendix C. This strategy
speeds up phase II training without sacrificing the model performance as
${\bm{y}^{*}}^{(1:T)}$ does not change significantly from loop to loop.

##  5 Experiments

We benchmark CHMM-ALT on four datasets against state-of-the-art weakly
supervised NER baselines, including both distant learning models and multi-
source label aggregation models. We also conduct a series of ablation studies
to evaluate the different components in CHMM-ALT’s design.

| Co03 | NCBI | CDR | LR
---|---|---|---|---
# Instance | $22137$ | $793$ | $1500$ | $3845$
# Training | $14041$ | $593$ | $500$ | $2436$
# Development | $3250$ | $100$ | $500$ | $609$
# Test | $3453$ | $100$ | $500$ | $800$
Ave# Tokens | $14.5$ | $219.8$ | $217.7$ | $16.4$
# Entities | $4$ | $1$ | $2$ | $1$
# Sources | $13$ | $5$ | $8$ | $4$
Table 1:  Dataset statistics. Co03 is CoNLL 2003; LR is LaptopReview; CDR is
BC5CDR. “# Sources” indicates the number of labeling sources for each dataset.

###  5.1 Setup

#### Datasets

We consider four NER datasets covering the general, technological and
biomedical domains: 1) CoNLL 2003 (English subset) (Tjong Kim Sang and De
Meulder, 2003) is a general domain dataset containing $22137$ sentences
manually labelled with $4$ entity types. 2) LaptopReview dataset (Pontiki et
al., 2014) consists of $3845$ sentences with laptop-related entity mentions.
3) NCBI-Disease dataset (Dogan et al., 2014) contains $793$ PubMed abstracts
annotated with disease mentions. 4) BC5CDR (Li et al., 2016), the dataset
accompanies the BioCreative V CDR challenge, consists of $1500$ PubMed
articles, annotated with chemical disease mentions.

Table 1 shows dataset statistics, including the average number of tokens,
entities and weak labeling sources. We use the original word tokens in the
dataset if provided and use NLTK (Bird and Loper, 2004) otherwise for sentence
tokenization.

For weak labeling sources, we use the ones from Lison et al. (2020) for CoNLL
2003, and the ones from Safranchik et al. (2020) for LaptopReview, NCBI-
Disease and BC5CDR.777Details are presented in appendix B.

Models | CoNLL 2003 | NCBI-Disease | BC5CDR | LaptopReview
---|---|---|---|---
Supervised BERT-NER $\ddagger$ $\natural$ | 90.74 (90.37/91.10) | 88.89 (87.05/90.82) | 88.81 (87.12/90.57) | 81.34 (82.02/80.67)
best consensus $\natural$ | 89.18 (100.0/80.47) | 81.60 (100.0/68.91) | 87.58 (100.0/77.89) | 77.72 (100.0/63.55)
SwellShark (noun-phrase) $\dagger\ddagger$ | - | 67.10 (64.70/69.70) | 84.23 (84.98/83.49) | -
SwellShark (hand-tuned) $\dagger\ddagger$ | - | 80.80 (81.60/80.10) | 84.21 (86.11/82.39) | -
AutoNER $\dagger\ddagger$ | 67.00 (75.21/60.40) | 75.52 (79.42/71.98) | 82.13 (83.23/81.06) | 65.44 (72.27/59.79)
Snorkel $\dagger\ddagger$ | 66.40 (71.40/62.10) | 73.41 (71.10/76.00) | 82.24 (80.23/84.35) | 63.54 (64.09/63.09)
Linked HMM $\dagger\ddagger$ | - | 79.03 (83.46/75.05) | 82.96 (82.65/83.28) | 69.04 (77.74/62.11)
BOND-MV $\dagger\ddagger$ $\natural$ | 65.96 (64.22/67.82) | 80.33 (84.77/76.34) | 83.18 (82.90/83.49) | 67.19 (68.90/65.75)
Majority Voting $\dagger$ $\natural$ | 58.40 (49.01/72.24) | 73.94 (79.76/68.91) | 80.73 (83.79/77.88) | 67.92 (72.93/63.55)
HMM $\dagger$ $\natural$ | 68.84 (70.80/66.98) | 73.06 (83.88/64.70) | 80.57 (88.75/73.76) | 66.96 (77.46/58.96)
CHMM-i.i.d. $\dagger$ $\natural$ | 68.57 (69.67/67.50) | 71.69 (83.49/62.87) | 79.37 (85.68/73.92) | 65.89 (75.70/58.34)
CHMM $\dagger$ $\natural$ | 70.11 (72.98/67.47) | 78.88 (93.37/68.28) | 82.39 (89.93/76.02) | 73.02 (87.23/62.79)
CHMM + BERT-NER $\dagger\ddagger$ $\natural$ | 74.30 (75.02/73.58) | 82.87 (89.42/77.22) | 84.33 (85.58/83.12) | 69.67 (75.48/64.70)
CHMM-ALT $\dagger\ddagger$ $\natural$ |  75.54 (76.22/74.86) |  85.02 (87.92/82.47) |  85.12 (84.97/85.28) |  76.55 (81.39/72.32)
Table 2:  Evaluation results on four datasets. The results are presented in
the “F1 (Precision/Recall)” format. “CHMM + BERT-NER” is essentially CHMM-
ALT’s phase I output. “BOND-MV” is the BOND model trained with majority voted
labels. $\dagger$ indicates unsupervised label denoiser; $\ddagger$ represents
fully supervised models. A model with $\dagger\ddagger$ is either distantly
supervised or trains a supervised by labels from the denoiser. $\natural$
signifies the results from our experiments. In addition to models with
$\natural$, Snorkel and Linked HMM also share our labeling sources.

#### Baselines

We compare our model to the following state-of-the-art baselines: 1) Majority
Voting returns the label for a token that has been observed by most of the
sources and randomly chooses one if it’s a tie; 2) Snorkel (Ratner et al.,
2017) treats each token in a sequence as i.i.d. and conducts the label
classification without considering its context; 3) SwellShark (Fries et al.,
2017) improves Snorkel by predicting all the target entity spans before
classifying them using naïve Bayes; 4) AutoNER (Shang et al., 2018) augments
distant supervision by predicting whether two consecutive tokens should be in
the same entity span; 5) BOND (Liang et al., 2020) adopts self-training and
high-confidence selection to further boost the distant supervision
performance. 6) HMM is the multi-observation generative model used in Lison et
al. (2020) that does not have the integrated neural network; 7) Linked HMM
(Safranchik et al., 2020) uses linking rules to provide additional inter-token
structural information to the HMM model.

For the ablation study, we modify CHMM to another type of i.i.d. model by
taking away its transition matrices. This model, named CHMM-i.i.d., directly
predicts the hidden steps from the BERT embeddings, while otherwise identical
to CHMM. We also investigate how CHMM-ALT performs with other aggregators
other than CHMM.

We also introduce two upper bounds from different aspects: 1) a fully
supervised BERT-NER model trained with manually labeled data is regarded as a
supervised reference; 2) the best possible consensus of the weak sources. The
latter assumes an oracle that always selects the correct annotations from
these weak supervision sources. According to the definition, its precision is
always $100\%$ and its recall is non-decreasing with the increase of the
number of weak sources.

#### Evaluation Metrics

We evaluate the performance of NER models using entity-level precision,
recall, and F1 scores. All scores are presented as percentages. The results
come from the average of $5$ trials with different random seeds.

#### Implementation Details

We use BERT pre-trained on different domains for different datasets, both for
embedding construction and as the component of the supervised BERT-NER model.
The original BERT (Devlin et al., 2019) is used for CoNLL 2003 and
LaptopReview datasets, bioBERT (Lee et al., 2019) for NCBI-Disease and SciBERT
(Beltagy et al., 2019) for BC5CDR. Instances with lengths exceeding BERT’s
maximum length limitation ($512$) are broken into several shorter segments.

The only tunable hyper-parameter in CHMM is the learning rate. But its
influence is negligible—benefitted from the stability of the generalized EM,
the model is guaranteed to converge to a local optimum if the learning rate is
small enough. For all the BERT-NER models used in our experiments, the hyper-
parameters except the batch size are fixed to the default values (appendix C).

To prevent overfitting, we use a two-scale early stopping strategy for model
choosing at two scales based on the development set. The micro-scale early
stopping chooses the best model parameters for each individual training
process of both CHMM and BERT-NER; the macro-scale early stopping selects the
best-performing model in phase II iterations, which reports the test results.
In our experiments, phase II exits if the macro-scale development score has
not increased in $5$ loops or the maximum number of loops ($10$) is reached.

(a)

(b)

(c)

(d)

Figure 4:  F1 score evolution across the alternate-training phases. “PI” is
phase I; “PII-$i$” is the $i$th loop of phase II. The “strongest baseline”
reports the result from the best-performed baseline in Table 2 for each
dataset.

###  5.2 Main Results

Table 2 presents the model performance from different domains. We find that
our alternate-training framework outperforms all weakly supervised baseline
models. In addition, CHMM-ALT approaches or even exceeds the best source
consensus, which sufficiently proves the effectiveness of the design. For
general HMM-based label aggregators such as CHMM, it is impossible to exceed
the best consensus since they can only predict an entity observed by at least
one source. Based on this fact, CHMM is designed to select the most accurate
observations from the weak sources without shrinking their coverage. In
comparison, BERT’s language representation ability enables it to generalize
the entity patterns and successfully discovers those entities annotated by
none of the sources. Comparing CHMM + BERT to CHMM, we can conclude that BERT
basically exchanges recall with precision, and its high-recall predictions can
improve the result of CHMM in return. The complementary nature of these two
models is why CHMM-ALT improves the overall performance of weakly supervised
NER.

###  5.3 Analysis of CHMM

Looking at Table 2, we notice that CHMM performs the best amongst all
generative models including majority voting, HMM and CHMM-i.i.d. The
performance of conventional HMM is largely limited by the Markov assumption
with the unchanging transition and emission probabilities. The results in the
table validate that conditioning the model on BERT embedding alleviates this
limitation. However, the transition matrices in HMM are indispensable, implied
by CHMM-i.i.d.’s results, as they provide supplemental information about how
the underlying true labels should evolve.

Models | Co03 | NCBI | CDR | Laptop
---|---|---|---|---
MV $\dagger$ $\natural$ | 58.40 | 73.94 | 80.73 | 67.92
MV-ALT $\dagger\ddagger$ $\natural$ | 66.64 | 80.83 | 82.78 | 70.45
HMM $\dagger$ $\natural$ | 68.84 | 73.06 | 80.57 | 66.96
HMM-ALT $\dagger\ddagger$ $\natural$ | 74.04 | 82.99 | 83.34 | 72.90
i.i.d. $\dagger$ $\natural$ | 68.57 | 71.69 | 79.37 | 65.89
i.i.d.-ALT $\dagger\ddagger$ $\natural$ | 73.84 | 83.15 | 83.17 | 72.61
CHMM $\dagger$ $\natural$ | 70.11 | 78.88 | 82.39 | 73.02
CHMM-ALT $\dagger\ddagger$ $\natural$ | 75.54 | 85.02 | 85.12 | 76.55
Table 3:  Alternate-training F1 scores with different label aggregators. MV
denotes Majority voting; i.i.d. represents CHMM-i.i.d. The model names without
the “ALT” suffix are the multi-source label aggregators whereas the suffix
indicates that the result comes from the alternate-training framework with the
corresponding model as the label aggregator.

###  5.4 Analysis of Alternate-Training

#### Performance Evolution

Figure 4 reveals the details of the alternate-training process. For less
ambiguous tasks including NCBI-Disease, BC5CDR and LaptopReview with fewer
entity types, BERT generally has better performance in phase I but gets
surpassed in phase II. Interestingly, BERT’s performance never exceeds that of
CHMM on the LaptopReview dataset. This may be because BERT fails to construct
sufficiently representative patterns from the denoised labels for this
dataset. For CoNLL 2003, where it is harder for the labeling sources to model
the language structures, the strength of a pre-trained language model in
pattern recognition becomes more prominent. From the results it seems that the
performance increment of the denoised labels ${\bm{y}^{*}}^{(1:T)}$ provides
marginally extra information to BERT after phase II, as most of the increment
comes from the information provided by BERT itself. Even so, keeping phase II
is reasonable when we want to get the best out of the weak labeling sources
and the pre-trained BERT.

#### BERT-NER Initialization

CHMM-ALT initializes BERT-NER’s parameters from its previous checkpoint at the
beginning of each loop in phase II to reduce training time (§ 4.3). If we
instead fine-tune BERT-NER from the initial parameters of the pre-trained BERT
model for each loop, CHMM-ALT gets $84.30$, $84.71$, and $76.68$ F1 scores on
NCBI-Disease, BC5CDR, and LaptopReview datasets. These scores are close to the
results in Table 2, but the training takes much longer. Consequently, our
BERT-NER initialization strategy is a more practical choice overall.

#### Applying Alternate-Training to Other Methods

Table 3 shows the alternate-training performance acquired with different label
aggregators. The accompanying BERT-NER models are identical to those described
in § 5.1. The results in the table suggest that the performance improvement
obtained by using alternate-training on the label aggregators is stable and
generalizable to any other models yet to be proposed.

##  6 Conclusion

In this work, we present CHMM-ALT, a multi-source weakly supervised approach
that does not depend on manually labeled data to learn an accurate NER tagger.
It integrates a label aggregator—CHMM and a supervised model—BERT-NER together
into an alternate-training procedure. CHMM conditions HMM on BERT embeddings
to achieve greater flexibility and stronger context-awareness. Fine-tuned with
CHMM’s prediction, BERT-NER discovers patterns unobserved by the weak sources
and complements CHMM. Training these models in turn, CHMM-ALT uses the
knowledge encoded in both the weak sources and the pre-trained BERT model to
improve the final NER performance. In the future, we will consider imposing
more constraints on the transition and emission probabilities, or manipulating
them according to sophisticated domain knowledge. This technique could be also
extended to other sequence labeling tasks such as semantic role labeling or
event extraction.

## Acknowledgments

This work was supported by ONR MURI N00014-17-1-2656, NSF III-2008334, Kolon
Industries, and research gifts from Google and Amazon. In addition, we would
like to thank Yue Yu for his insightful suggestions for this work.

## References

  * Bach and Badaskar (2007) Nguyen Bach and Sameer Badaskar. 2007.  A review of relation extraction.  _Literature review for Language and Statistics II_ , 2:1–15.
  * Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural machine translation by jointly learning to align and translate.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Beltagy et al. (2019) Iz Beltagy, Kyle Lo, and Arman Cohan. 2019.  SciBERT: A pretrained language model for scientific text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3615–3620, Hong Kong, China. Association for Computational Linguistics.
  * Bird and Loper (2004) Steven Bird and Edward Loper. 2004.  NLTK: The natural language toolkit.  In _Proceedings of the ACL Interactive Poster and Demonstration Sessions_ , pages 214–217, Barcelona, Spain. Association for Computational Linguistics.
  * Cao et al. (2019) Yixin Cao, Zikun Hu, Tat-seng Chua, Zhiyuan Liu, and Heng Ji. 2019.  Low-resource name tagging learned with weakly labeled data.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 261–270, Hong Kong, China. Association for Computational Linguistics.
  * Chiu and Rush (2020) Justin Chiu and Alexander Rush. 2020.  Scaling hidden Markov language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1341–1349, Online. Association for Computational Linguistics.
  * Dai et al. (2017) Hanjun Dai, Bo Dai, Yan-Ming Zhang, Shuang Li, and Le Song. 2017.  Recurrent hidden semi-markov model.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dogan et al. (2014) Rezarta Islamaj Dogan, Robert Leaman, and Zhiyong Lu. 2014.  NCBI disease corpus: A resource for disease name recognition and concept normalization.  _J. Biomed. Informatics_ , 47:1–10.
  * Farmakiotou et al. (2000) Dimitra Farmakiotou, Vangelis Karkaletsis, John Koutsias, George Sigletos, Constantine D. Spyropoulos, and Panagiotis Stamatopoulos. 2000.  Rule-based named entity recognition for greek financial texts.  In _In Proceedings of the Workshop on Computational Lexicography and Multimedia Dictionaries (COMLEX 2000_ , pages 75–78.
  * Fries et al. (2017) Jason A. Fries, Sen Wu, Alexander Ratner, and Christopher Ré. 2017.  Swellshark: A generative model for biomedical named entity recognition without labeled data.  _CoRR_ , abs/1704.06360.
  * Huang et al. (2015) Zhiheng Huang, Wei Xu, and Kai Yu. 2015.  Bidirectional LSTM-CRF models for sequence tagging.  _CoRR_ , abs/1508.01991.
  * Khalid et al. (2008) Mahboob Alam Khalid, Valentin Jijkoun, and Maarten De Rijke. 2008.  The impact of named entity normalization on information retrieval for question answering.  In _Proceedings of the IR Research, 30th European Conference on Advances in Information Retrieval_ , ECIR’08, pages 705–710, Berlin, Heidelberg. Springer-Verlag.
  * Kim et al. (2018) Yoon Kim, Sam Wiseman, and Alexander M. Rush. 2018.  A tutorial on deep latent variable models of natural language.  _CoRR_ , abs/1812.06834.
  * Lan et al. (2020) Ouyu Lan, Xiao Huang, Bill Yuchen Lin, He Jiang, Liyuan Liu, and Xiang Ren. 2020\.  Learning to contextually aggregate multi-source supervision for sequence labeling.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2134–2146, Online. Association for Computational Linguistics.
  * Lee et al. (2019) Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2019.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ , 36(4):1234–1240.
  * Li et al. (2012) Chenliang Li, Aixin Sun, and Anwitaman Datta. 2012.  Twevent: Segment-based event detection from tweets.  In _Proceedings of the 21st ACM International Conference on Information and Knowledge Management_ , CIKM ’12, pages 155–164, New York, NY, USA. Association for Computing Machinery.
  * Li et al. (2016) Jiao Li, Yueping Sun, Robin J. Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Thomas C. Wiegers, and Zhiyong Lu. 2016.  Biocreative V CDR task corpus: a resource for chemical disease relation extraction.  _Database J. Biol. Databases Curation_ , 2016.
  * Liang et al. (2020) Chen Liang, Yue Yu, Haoming Jiang, Siawpeng Er, Ruijia Wang, Tuo Zhao, and Chao Zhang. 2020.  Bond: Bert-assisted open-domain named entity recognition with distant supervision.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, KDD ’20, pages 1054–1064, New York, NY, USA. Association for Computing Machinery.
  * Lison et al. (2020) Pierre Lison, Jeremy Barnes, Aliaksandr Hubin, and Samia Touileb. 2020.  Named entity recognition without labelled data: A weak supervision approach.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1518–1533, Online. Association for Computational Linguistics.
  * Liu et al. (2018) Hao Liu, Lirong He, Haoli Bai, Bo Dai, Kun Bai, and Zenglin Xu. 2018.  Structured inference for recurrent hidden semi-markov model.  In _Proceedings of the 27th International Joint Conference on Artificial Intelligence_ , IJCAI’18, page 2447–2453. AAAI Press.
  * Luong et al. (2015) Thang Luong, Hieu Pham, and Christopher D. Manning. 2015.  Effective approaches to attention-based neural machine translation.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 1412–1421, Lisbon, Portugal. Association for Computational Linguistics.
  * Mintz et al. (2009) Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009.  Distant supervision for relation extraction without labeled data.  In _Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP_ , pages 1003–1011, Suntec, Singapore. Association for Computational Linguistics.
  * Nadeau and Sekine (2007) David Nadeau and Satoshi Sekine. 2007.  A survey of named entity recognition and classification.  _Linguisticae Investigationes_ , 30(1):3–26.  Publisher: John Benjamins Publishing Company.
  * Nguyen et al. (2017) An Thanh Nguyen, Byron Wallace, Junyi Jessy Li, Ani Nenkova, and Matthew Lease. 2017\.  Aggregating and predicting sequence labels from crowd annotations.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 299–309, Vancouver, Canada. Association for Computational Linguistics.
  * Pontiki et al. (2014) Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014.  SemEval-2014 task 4: Aspect based sentiment analysis.  In _Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)_ , pages 27–35, Dublin, Ireland. Association for Computational Linguistics.
  * Ramshaw and Marcus (1995) Lance Ramshaw and Mitch Marcus. 1995.  Text chunking using transformation-based learning.  In _Third Workshop on Very Large Corpora_.
  * Ratner et al. (2017) Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. 2017.  Snorkel: Rapid training data creation with weak supervision.  _Proc. VLDB Endow._ , 11(3):269–282.
  * Ratner et al. (2019) Alexander Ratner, Braden Hancock, Jared Dunnmon, Frederic Sala, Shreyash Pandey, and Christopher Ré. 2019.  Training complex models with multi-task weak supervision.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33:4763–4771.
  * Ratner et al. (2016) Alexander Ratner, Christopher De Sa, Sen Wu, Daniel Selsam, and Christopher Ré. 2016.  Data programming: Creating large training sets, quickly.  In _Proceedings of the 30th International Conference on Neural Information Processing Systems_ , NIPS’16, pages 3574–3582, Red Hook, NY, USA. Curran Associates Inc.
  * Ren et al. (2020) Wendi Ren, Yinghao Li, Hanting Su, David Kartchner, Cassie Mitchell, and Chao Zhang. 2020.  Denoising multi-source weak supervision for neural text classification.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3739–3754, Online. Association for Computational Linguistics.
  * Safranchik et al. (2020) Esteban Safranchik, Shiying Luo, and Stephen H. Bach. 2020.  Weakly supervised sequence tagging from noisy rules.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 5570–5578. AAAI Press.
  * Shang et al. (2018) Jingbo Shang, Liyuan Liu, Xiaotao Gu, Xiang Ren, Teng Ren, and Jiawei Han. 2018\.  Learning named entity tagger using domain-specific dictionary.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2054–2064, Brussels, Belgium. Association for Computational Linguistics.
  * Sun et al. (2019) Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019.  How to fine-tune bert for text classification? _Chinese Computational Linguistics_ , page 194–206.
  * Thiel (2008) Christian Thiel. 2008.  Classification on soft labels is robust against label noise.  In _Proceedings of the 12th International Conference on Knowledge-Based Intelligent Information and Engineering Systems, Part I_ , KES ’08, pages 65–73, Berlin, Heidelberg. Springer-Verlag.
  * Tjong Kim Sang and De Meulder (2003) Erik F. Tjong Kim Sang and Fien De Meulder. 2003.  Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition.  In _Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003_ , pages 142–147.
  * Tran et al. (2016) Ke M. Tran, Yonatan Bisk, Ashish Vaswani, Daniel Marcu, and Kevin Knight. 2016.  Unsupervised neural hidden Markov models.  In _Proceedings of the Workshop on Structured Prediction for NLP_ , pages 63–71, Austin, TX. Association for Computational Linguistics.
  * Viterbi (1967) A. Viterbi. 1967.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm.  _IEEE Trans. Inf. Theor._ , 13(2):260–269.
  * Wessels and Omlin (2000) T. Wessels and Christian W. Omlin. 2000.  Refining hidden markov models with recurrent neural networks.  In _Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, IJCNN 2000, Neural Computing: New Challenges and Perspectives for the New Millennium, Como, Italy, July 24-27, 2000, Volume 2_ , pages 271–278. IEEE Computer Society.
  * Wiseman et al. (2018) Sam Wiseman, Stuart Shieber, and Alexander Rush. 2018.  Learning neural templates for text generation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3174–3187, Brussels, Belgium. Association for Computational Linguistics.
  * Yang et al. (2018) Yaosheng Yang, Wenliang Chen, Zhenghua Li, Zhengqiu He, and Min Zhang. 2018.  Distantly supervised NER with partial annotation learning and reinforcement learning.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 2159–2169, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Yu et al. (2020) Yue Yu, Simiao Zuo, Haoming Jiang, Wendi Ren, Tuo Zhao, and Chao Zhang. 2020.  Fine-tuning pre-trained language model with weak supervision: A contrastive-regularized self-training approach.

##  Appendix A Technical Details

###  A.1 CHMM Training

Following the discussion in § 4.2, we use the forward-backward algorithm to
calculate the smoothed marginal $\gamma^{(t)}_{i}\triangleq
p(z^{(t)}=i|\bm{x}^{(1:T)}),i\in\\{1,2,\dots,|\mathcal{L}|\\},t\in\\{1,2,\dots,T\\}$
and the expected number of transitions $\xi^{(t)}_{i,j}\triangleq
p(z^{(t-1)}=i,z^{(t)}=j|\bm{x}^{(1:T)}),i,j\in\\{1,2,\dots,|\mathcal{L}|\\}$.888Same
as § 4.2, we omit the dependency term $\bm{e}^{(1:T)}$. $|\mathcal{L}|$ is the
number of BIO formatted entity labels, which are regarded as hidden states;
$T$ is the total number of hidden steps in a sequence, which equals the number
of tokens.

Defining $\alpha^{(t)}_{i}\triangleq p(z^{(t)}=i|\bm{x}^{(1:t)})$ and
$\beta^{(t)}_{i}\triangleq p(\bm{x}^{(t+1:T)}|z^{(t)}=i)$, $\gamma^{(t)}_{i}$
and $\xi^{(t)}_{i,j}$ can be represented by $\alpha$ and $\beta$ using the
Bayes’ rule and Markov assumption:

$\displaystyle\begin{aligned} \gamma^{(t)}_{i}&\triangleq
p(z^{(t)}=i|\bm{x}^{(1:T)})\\\
&=\frac{p(\bm{x}^{(t+1:T)},z^{(t)}=i|\bm{x}^{(1:t)})}{p(\bm{x}^{(t+1:T)}|\bm{x}^{(1:T)})}\\\
&\propto p(z^{(t)}=i|\bm{x}^{(1:t)})p(\bm{x}^{(t+1:T)}|z^{(t)}=i)\\\
&=\alpha^{(t)}_{i}\beta^{(t)}_{i},\end{aligned}$ (12)
$\displaystyle\begin{aligned} \xi^{(t)}_{i,j}&\triangleq
p(z^{(t-1)}=i,z^{(t)}=j|\bm{x}^{(1:T)})\\\ &\propto
p(z^{(t-1)}=i|\bm{x}^{(1:t-1)})\\\ &\quad\
p(z^{(t)}=j|z^{(t-1)}=i,\bm{x}^{(t:T)})\\\ &\propto
p(z^{(t-1)}=i|\bm{x}^{(1:t-1)})p(\bm{x}^{(t)}|z^{(t)}=j)\\\ &\quad\
p(\bm{x}^{(t+1:T)}|z^{(t)}=j)p(z^{(t)}=j|z^{(t-1)}=i)\\\
&=\alpha^{(t-1)}_{i}\varphi^{(t)}_{j}\beta^{(t)}_{j}\Psi^{(t)}_{i,j}.\end{aligned}$
(13)

$\varphi^{(t)}_{i}\in\mathbb{R}^{|\mathcal{L}|}\triangleq
p(\bm{x}^{(t)}|z^{(t)}=i)$ is the likelihood of the observation when the
hidden state is $i$ (§ 4.2).

Written in the matrix form, (12) and (13) become:

$\displaystyle\bm{\gamma}^{(t)}\propto\bm{\alpha}^{(t)}\odot\bm{\beta}^{(t)},$
(14)
$\displaystyle\bm{\xi}^{(t)}\propto\bm{\Psi}^{(t)}\odot(\bm{\alpha}^{(t-1)}(\bm{\varphi}^{(t)}\odot\bm{\beta}^{(t)})^{\sf
T}),$ (15)

where $\odot$ is the element-wise product. Note that the elements in both
$\bm{\gamma}^{(t)}$ and $\bm{\xi}^{(t)}$ should sum up to $1$.

#### The Forward Pass

The filtered marginal $\alpha^{(t)}_{i}$ can be computed iteratively:

$\displaystyle\alpha^{(t)}_{i}$ $\displaystyle\triangleq
p(z^{(t)}=i|\bm{x}^{(1:t)})$ (16)
$\displaystyle=p(z^{(t)}=i|\bm{x}^{(t)},\bm{x}^{(1:t-1)})$
$\displaystyle\propto p(\bm{x}^{(t)}|z^{(t)}=i)p(z^{(t)}=i|\bm{x}^{(1:t-1)})$
$\displaystyle=\sum_{j}\varphi^{(t)}_{i}\Psi^{(t)}_{j,i}\alpha^{(t-1)}_{j}.$

Written in the matrix form, (16) becomes

$\bm{\alpha}^{(t)}\propto\bm{\varphi}^{(t)}\odot({\bm{\Psi}^{(t)}}^{\sf
T}\bm{\alpha}^{(t-1)}).$ (17)

We initialize $\bm{\alpha}$ with $\bm{\alpha}^{(0)}=\bm{\pi}$ (§ 4.2) since we
have no observation at time step $0$. As $\bm{\alpha}^{(t)}$ is a probability
distribution, the elements in it sum up to $1$. The calculation of
$\bm{\alpha}$ is the forward pass.

#### The Backward Pass

In the same way, we do the backward pass to compute the conditional future
evidence $\beta^{(t)}_{i}\triangleq p(\bm{x}^{(t+1:T)}|z^{(t)}=i)$:

$\displaystyle\beta^{(t-1)}_{i}$ $\displaystyle\triangleq
p(\bm{x}^{(t+1:T)}|z^{(t)}=j)$ (18)
$\displaystyle=\sum_{j}p(z^{(t)}=j,\bm{x}^{(t)},\bm{x}^{(t+1:T)}|z^{(t-1)}=i)$
$\displaystyle=\sum_{j}[p(\bm{x}^{(t+1:T)}|z^{(t)}=j)$
$\displaystyle\qquad\quad p(\bm{x}^{(t)},z^{(t)}=j|z^{(t-1)}=i)]$
$\displaystyle=\sum_{j}\beta^{(t)}_{j}\varphi^{(t)}_{j}\Psi^{(t)}_{i,j}.$

In the matrix form, (18) becomes:

$\bm{\beta}^{(t-1)}=\bm{\Psi}^{(t)}(\bm{\varphi}^{(t)}\odot\bm{\beta}^{(t)}),$
(19)

whose base case is

$\displaystyle\beta^{(T)}_{i}=p(\bm{x}^{(T+1:T)}|z^{(T)}=i)$
$\displaystyle=1,$ $\displaystyle\forall i$
$\displaystyle\in\\{1,\dots,|\mathcal{L}|\\}.$

###  A.2 The Maximization step for Unsupervised HMM

For traditional unsupervised HMM, the expected complete data log likelihood is
maximized by updating the matrices with the approximated pseudo-statistics.
different from CHMM, HMM has constant transition and emission for all time
steps, _i.e._ :

$\displaystyle\bm{\Psi}^{(1)}=\bm{\Psi}^{(t)};\bm{\Phi}^{(1)}=\bm{\Phi}^{(t)};\quad\forall
t\in\\{2,\dots,T\\}.$

For simplicity, we remove the term $t$ for the transition and emission
matrices. Suppose we are updating HMM based on one instance with $t$ starting
from $1$:

$\displaystyle\pi_{i}=\gamma^{(1)}_{i};$ (20)
$\displaystyle\Psi_{i,j}=\frac{\sum_{t=2}^{T}\xi^{(t)}_{i,j}}{\sum_{t=2}^{T}\sum_{\ell=1}^{|\mathcal{L}|}\xi^{(t)}_{i,\ell}};$
(21)
$\displaystyle\Phi_{i,j,k}=\frac{\sum_{t=1}^{T}\gamma^{(t)}_{i}x^{(t)}_{j,k}}{\sum_{t=1}^{T}\gamma^{{t}}_{i}}.$
(22)

Note that the observation has property $0\leq x^{(t)}_{j,k}\leq 1$ and
$\sum_{j=1}^{|\mathcal{L}|}x^{(t)}_{j,k}=1$, where $k\in\\{1,\dots K\\}$ is
the index of the weak labeling source.

##  Appendix B Labeling Source Performance

The weak labeling sources of the CoNLL 2003 dataset come from Lison et al.
(2020), whereas Safranchik et al. (2020) provide the sources for the
LaptopReview, NCBI-Disease and BC5CDR dataset. For Safranchik et al. (2020)’s
labeling sources, we apply a majority voting using their tagging results to
the spans detected by their linking rules to convert the linking results to
token annotations. In consideration of the training time and resource
consumption, we only adopt a subset of the labeling sources provided by the
authors. The performance of the labeling sources is presented in the tables
below.

source name | precision | recall | f1
---|---|---|---
CoreDictionaryUncased | 81.03 | 41.41 | 5.48
CoreDictionaryExact | 80.69 | 17.18 | 28.32
CancerLike | 34.88 | 1.58 | 3.02
BodyTerms | 68.52 | 3.90 | 7.38
ExtractedPhrase | 97.12 | 32.03 | 48.18
Table 4: The performance of the labeling sources used in the NCBI-Disease dataset. source name | precision | recall | f1
---|---|---|---
DictCore-Chemical | 91.81 | 29.55 | 44.7
DictCore-Chemical-Exact | 85.88 | 3.16 | 6.1
DictCore-Disease | 81.57 | 26.32 | 39.8
DictCore-Disease-Exact | 81.4 | 1.09 | 2.16
Organic Chemical | 92.67 | 30.07 | 45.4
Disease or Syndrome | 77.36 | 11.67 | 20.28
PostHyphen | 84.47 | 08.07 | 14.74
ExtractedPhrase | 86.8 | 17.96 | 29.76
Table 5: The performance of the labeling sources used in the BC5CDR dataset. source name | precision | recall | f1
---|---|---|---
CoreDictionary | 72.63 | 51.61 | 60.34
iStuff | 26.67 | 0.61 | 1.2
ExtractedPhrase | 97.45 | 29.25 | 45.0
ConsecutiveCapitals | 35.29 | 0.92 | 1.8
Table 6: The performance of the labeling sources used in the LaptopReview dataset. source name | precision | recall | f1
---|---|---|---
BTC+c | 61.56 | 46.35 | 52.88
SEC+c | 39.54 | 24.59 | 30.32
core_web_md+c | 69.53 | 60.04 | 64.44
crunchbase_cased | 38.26 | 5.59 | 9.76
crunchbase_uncased | 37.88 | 6.2 | 10.66
doc_majority_cased | 65.81 | 40.21 | 49.92
doc_majority_uncased | 61.69 | 40.17 | 48.66
full_name_detector | 87.79 | 11.33 | 20.06
geo_cased | 68.16 | 15.35 | 25.06
geo_uncased | 65.1 | 18.89 | 29.28
misc_detector | 85.14 | 21.51 | 34.34
wiki_cased | 75.27 | 32.65 | 45.54
wiki_uncased | 72.26 | 35.61 | 47.7
Table 7: The performance of the labeling sources used in the CoNLL 2003
dataset.

Please refer to Lison et al. (2020) for the information about the construction
of the labeling sources on the CoNLL 2003 dataset; please refer to Safranchik
et al. (2020) for the labeling sources on other three datasets.

##  Appendix C Hyper-Parameters

The experiments are conducted on one GeForce RTX 2080 Ti GPU. For NCBI-
Disease, BC5CDR and LaptopReview datasets, CHMM is pre-trained for $5$ epochs
and trained for $20$ epochs. The learning rates for these three datasets are
$5\times 10^{-4}$, $10^{-3}$ and $10^{-4}$, respectively, and the batch sizes
are $64$, $64$ and $128$. In phase I, BERT-NER is trained with the default
learning rate ($5\times 10^{-5}$) for $100$ epochs. The batch sizes are $8$,
$8$, and $48$, respectively. Note that for LaptopReview, the maximum length
limitation of BERT-NER is set to $128$ whereas the limitation is $512$ for the
other two datasets. In phase II, we use half the learning rate with $20$
epochs for each loop.

For CoNLL 2003, CHMM has the same number of training epochs as for other
datasets. The batch size is $32$, and the learning rate is $10^{-5}$. BERT-NER
has a maximum sequence length of $256$. It is trained for $15$ epochs in phase
I and $5$ epochs in phase II. Other hyper-parameters are identical to other
BERT-NER models’.