# An Overview of the State of the Art for Practical Quantum Key Distribution

Daniel D. Moskovich  Center for Quantum Information Science and Technology,
Ben-Gurion University of the Negev, Israel

(Date: 9 April, 2015)

###### Abstract.

This is an overview of the state of the art for quantum key distribution (QKD)
as of March 2015. It is written by a non-expert for non-experts. Additions and
corrections are welcome.

##  1\. Introduction

The goal of this overview to concisely summarize, in a way that is accessible
to a non-expert, where practical Quantum Key Distribution (QKD) stands now in
early 2015 and what seem to be promising directions for the near future to the
best of the author’s knowledge and understanding.

We begin with a general overview of what QKD is, followed by a discussion of
the major practical QKD players at the moment, a discussion of protocols, and
a discussion of photon sources, transmission, and detection. We conclude with
a section on attacks against QKD.

##  2\. What is quantum key distribution?

Quantum key distribution (QKD) uses principles of quantum information theory
to ensure secure communication (Weisner, 1983; Bennett & Brassard, 1984;
Ekert, 1991). Its goal is for two parties called (A)lice and (B)ob to share a
secret key made up of $0$ s and $1$ s which they will later use to encrypt and
decrypt communications between them. The information used to compose the key
is carried between Alice and Bob on qubits (two state quantum systems).

The _SECOCQ White Paper_ convincingly argued that QKD is a form of _trusted
courier_ (Alice hands a message to somebody she trusts, who carries that to
Bob), so that it is useful in contexts in which a trusted courier would be
useful (SECOQC, 2007). With idealized hardware and with perfect accuracy, the
advantages of QKD over classical trusted courier methods would be:

  * •

Mathematically-proven security against all classical and quantum attacks.

  * •

Alice and Bob can detect any active attempt by an eavesdropper (E)ve to
eavesdrop on the key distribution process.

The _Black Paper of Quantum Cryptography_ convincingly argued that real-life
QKD security is less than perfect, so that each different QKD setup should be
carefully and individually studied to assure its security (Scarani &
Kurtsiefer, 2014). All security threats discovered so far have been
redeemable, and we have no reason to believe that any given QKD setup cannot
be made perfectly secure against all known attacks in principle.

Real world QKD has become a focus of interest for industrial players, for
governments, and for security agencies.

##  3\. Fundamental challenges

A number of fundamental challenges to the widescale use of QKD have been
identified (Pritchard & Till, 2010).

  1. (1)

Limited transmission rate and range. Both the range and the maximal bit-rate
of QKD are low compared to classical communications. It is considered
technologically impossible, for instance, to transmit a polarized photon
reliably over more than 400km of fiber, although quantum repeaters will allow
for longer range QKD.

  2. (2)

QKD protocols are fundamentally point-to-point, and do not integrate with
packet-based protocols such as those used on the internet.

  3. (3)

QKD requires expensive special-purpose hardware such as single-photon sources
and detectors. Such hardware is difficult to upgrade and to maintain.

  4. (4)

QKD addresses only one aspect of the security problem. For example
authentication and integrity are not covered and must be handled classically.

  5. (5)

There is nothing fundamentally wrong with existing classical cryptographic
techniques. Even if some classical ciphers (e.g. RSA) may be cracked using
quantum algorithms at some unspecified point in the future, other classical
ciphers are being developed that would be immune to quantum attacks.

  6. (6)

Because it is a new technology, there are potential discovered and
undiscovered vulnerabilities in practical QKD systems. Indeed, several
proposed conditions for unconditional security of practical implementations
were found wanting and had to be revised, e.g. because key-length is finite
while many security proofs assume infinite-length keys (Inamori, Lütkenhaus &
Mayers, 2007; Tomamichel et al., 2012), or because the quantum effect of
_locking_ might allow unexpectedly large information leak during the error-
correction and privacy amplification steps (König et al., 2007; Iwakoshi &
Hirota, 2014; Yuen, 2013; Portmann & Renner, 2014). Commercial QKD systems
have been successfully hacked (Section 12.4). Because QKD is unconditionally
secure while the security of any quantum protocol implementation is a
probabilistic quantitative matter, these vulnerabilities will in principle
never be fatal flaws. But each new vulnerability might require re-tuning of
parameters and modifications to technological implementation.

##  4\. Advantages of QKD

  1. (1)

QKD provides the possibility to establish a secret key in a way that is
provable secure against eavesdropping. Moreover, QKD can be composed with
other encryptions, so as to provide an additional layer of security for an
already secure message. For example a message encrypted using an RSA public
key may be once again encrypted using a quantum key. To intercept the message,
an attacker would have to break both the quantum key and the classical key.

  2. (2)

Eavesdropping can be detected, following which countermeasures may be adopted.
This capability distinguished QKD from among all encryption methods.

  3. (3)

Expertise and knowledge gained in QKD research will, in large part, be useful
for developing future technologies in future manifestations of the coming
quantum revolution predicted by Michael Berry when he said, “It is easy to
predict that in the twenty-first century, it will be quantum mechanics that
influences all our lives.” Berry (1998).

##  5\. World QKD projects

###  5.1. Large scale networks

Microwave apparatus used in quantum experiments. Retrieved from
http://www.foxnews.com/tech/2013/05/08/quantum-network-secretly-running-
for-2-years/

In the last 10 years, a number of multi-user QKD networks have been
constructed. All use relay between trusted nodes and optical switching. The
first of these was the $10$–node DARPA Quantum Network which has been
operating since 2004 (Elliot et al., 2005). It uses active optical switching
(i.e. an electrically powered switching device similar to a router) to
distribute the key between pairs of nodes. It is being developed by BBN
Technologies, Harvard University, Boston University and QinetiQ, with the
support of the US Defense Advanced Research Projects Agency (DARPA). The
SECOQC (Secure Communication Based on Quantum Cryptography) Quantum Network is
an EU project which integrated several different QKD systems into one quantum
backbone (QBB) network, developing a cross-platform interface
(http://www.secoqc.net/). This provided impetus for the European
Telecommunications Standards Institute (ETSI) to launch an industry
specification group for QKD (ISG-QKD)in order to create universally accepted
QKD standards (ETSI, 2015). The Swiss Quantum Network and the Durban Network
are testing long-term QKD operation in field environments
(http://swissquantum.idquantique.com and (Mirza & Petruccione, 2010)).
Transparent network implementation of QKD using only beam splitters, which
facilitate secure communication without requiring clients to be reconfigured,
have been demonstrated by several groups (Telecordia Technologies, Universidad
Politécnica de Madrid and Telefónica Investigación y Desarrollo, and two teams
from the University of Science and Technology of China). The Tokyo QKD network
used a central Key Management Service (KMS) and newer technologies to increase
its speed to the point of transmitting a QKD-secured live teleconference
between two nodes (Sasaki et al., 2011). This is suitable for government or
municipal settings in which one central body controls the flow of information.
Mitsubishi combined this system with an application for secure telephony to
demonstrate QKD-secured mobile telephony (Mitsubishi, 2015). Finally, Los
Alamos National Laboratory (LANL) runs a hub-and-spokes one-to-many quantum
network (Hughes et al., 2013). The LANL photon generator has been miniaturized
to around the size of a house key.

China is currently constructing a 1200-mile line between Beijing and Shanghai
as part of a proposed $20$-node QKD network which it aims to complete in 2016.
Its current network, the Hefei-Chaohu-Wuhu wide area QKD network, is the
largest in the world (Wang et al., 2014). To overcome the need to use trusted
nodes, where one compromised node could impact the security of the entire
network, there has been work aimed at using techniques of classical multiple
access optical communication in the quantum context (Sarwar Pasha & Bala Ram,
2014). Such technologies have been applied for one part of the DARPA network,
and also for an experimental three-node network at NIST (see
http://www.nist.gov/itl/quantum/threeusernetwork.cfm).

Taking the above technologies into account, the Engineering Science and
Research Council (EPSRC) estimated in their 2014 report that hand-held QKD
systems should be commercially available “with sufficient investment and
encouragement” within 4–7 years, and that long-range highly-connected quantum
networks should become available within 10–25 years (EPSRC, 2014).

###  5.2. University Centres

There are a growing number of university centers in the world which specialize
in quantum communication. We list a few of the most active.

The world’s foremost dedicated quantum communications center is the Group of
Applied Physics (GAP) at Geneva University (http://www.unige.ch/gap/quantum/)
and their commercial spinoff company Id Quantique
(http://www.idquantique.com/). They have developed what is today the world’s
best single photon detector (Korzh _et al._ , 2014) with which they have
achieved the current world record distance for QKD through fiber (Korzh _et
al._ , 2015). They also produce and sell photon detectors and random number
generators using patented technologies.

The Centre for Quantum Technologies (CQT) in Singapore, directed by Ekert who
developed the E91 protocol, specializes in quantum hacking
http://www.quantumlah.org/. They have developed several successful attacks,
which have taken advantage both of side-channels (e.g. (Lamas-Linares &
Kurtsiefer, 2007)) and of erroneous parameters in security proofs (e.g.
(Gerhardt et al., 2011)).

The Institute of Quantum Computing (IQC) in Waterloo also has a research group
working on QKD. It is directed by Norbert Lütkenhaus, who previously worked at
MagiQ to develop practical QKD. Vadim Makarov of that group discovered some
successful side-channel attacks against QKD (e.g. (Makarov _et al._ , 2006;
Makarov, 2009)).

The Key Laboratory of Quantum Information is China’s leading quantum
information center, which is creating the world’s longest and most
sophisticated QKD networks
http://en.physics.ustc.edu.cn/research_9/Quantum/201107/t20110728_116550.html.

Artist’s conception of quantum key distribution over free space ground to
satellite quantum information links. Retrieved from http://www.esa.int/ESA

###  5.3. Commercial companies

A number of commercial companies sell QKD systems and related devices. MagiQ
Technologies in the US sells the QPN-8505, a QKD system which combines BB84
QKD with classical 3DES and AES encryption (http://www.magiqtech.com/). It
works using decoy-state optimized BB84, with a secure key rate of 256Hz over
100km, or 140km with decoy states. In Europe the leading QKD company is Id
Quantique, whose flagship product is the Clavis2, a pure QKD system
(http://www.idquantique.com/). Clavis2 implements both BB84 and SARG04, with
secure key-rates of around 1KHz on a 25km fiber. SeQureNet is a Paris-based
company that produces QKD parts and that specialized is continuous variable
(CV) QKD (http://sequrenet.com). Quintessence Labs in Australia provides true
random number generators (http://www.quintessencelabs.com/).

##  6\. Protocols

###  6.1. BB84

The most widely used QKD protocol, which was also the world’s first QKD
protocol, was developed by Bennett and Brassard in 1984, and is called BB84.
It is typically divided into three layers: The physical layer in which the
quantum communication is carried out,the key-extraction in which the actual
key is extracted from the qubits that Alice sent to Bob, and the key-
application layer where the secret key is used to encode a communication such
as a telephone or a video conversation (Bennett et al., 1992; Gisin et al.,
2002).

In the physical layer, Alice sends random photons, $1$ with 50% probability
and $0$ with 50% probability, either in the so-called $X$ basis or in the so-
called $Z$ basis, each with 50% probability. Bob measures each bit he receives
in a random basis, either in the $X$ basis with 50% probability, or in the $Y$
basis with 50% probability. This is the hardware-intensive portion of the
protocol, for which good random-number generators, single-photon sources, and
single-photon detectors are required.

In the key-extraction layer, BB84 becomes classical. The first classical
sublevel is called _sifting_. Alice and Bob both reveal which bases they used
over a public channel. They then discard the bits which they measured in
different bases. The second sublevel is called _authentication_. In it, Alice
and Bob compare some of their sifted bits over the public channel to determine
whether eavesdropping has occurred. If the bits they compare are more
different than can be accounted by from random noise, then they can guess that
Eve has eavesdropped, and adopt countermeasures. The reason that they can make
this deduction is that Eve’s direct attack, intercept–resend REF, would
involve measuring some of the bits sent by Alice, and sending them on to Bob.
But since Eve does not know which basis was originally used by Alice, she will
choose the wrong basis with 50% probability, and if she chooses the wrong
basis then she will send the wrong qubit to Bob, and that incorrect qubit will
survive into the sifted key with 50% probability. The third sublevel is called
_error correction_. In it, Alice and Bob apply classical error-correcting
algorithms to remedy the effect of random errors caused by channel noise and
by the fact that equipment is non-ideal. The fourth and final level is called
_privacy amplification_ , in which Alice and Bob apply classical cryptography
algorithms to minimize the effect on the final key of any under-the-radar
eavesdropping by Eve. In other words, security of a QKD key is always a
quantitative affair because of non-ideal equipment and channel noise, so some
non-trivial information might have been picked-up by Eve without being
detected in the authentication phase. But the amount of leaked information is
guaranteed to be below a certain threshold, and privacy amplification can
negate the knowledge about the final key which that partial information
imparts.

The BB84 protocol. Figure retrieved from
http://swissquantum.idquantique.com/?Key-Sifting.

###  6.2. Modified BB84 protocols

The best known modification of BB84 is SARG04, which adapts it for use with
attenuated laser pulses (Scarani et al., 2004). SARG04 is more robust than
BB84 against so-called ‘coherent attacks’, but unfortunately it performs worse
against certain ‘incoherent attacks’ (Branciard et al., 2005).

Lo, Chau, and Ardehali presents a modification of BB84 which essentially
doubles its efficiency (Lo _et al._ , 2005b). The key differences are that
significantly different probabilities are assigned to the two bases so that
few bits are discarded, and that key extraction is performed separately for
data in each of the bases. The Cambridge–Toshiba team further improved
efficiency and included decoy states, developing a new protocol called T12
(Lucamarini _et al._ , 2013). The authors prove it to be unconditionally
secure. As of February 2015, this is the protocol with which the highest
ranges and secure key rates have been obtained (Korzh _et al._ , 2015).

Decoy state QKD comes to solve the problem that the secure key rate of a
quantum key from a coherent source scales like the square of the transmittance
(the proportion of photons that make it through from Alice to Bob) of the
medium, and thus a secure key becomes too long to be practical when it must be
transmitted for long distances. For decoy-state QKD, the key length scales
like the transmittance. Three-source decoy-state QKD was what was used in
(Lucamarini _et al._ , 2013). Decoy state QKD works also with non-coherent
sources such as PDC sources (Ma & Lo, 2008).

Additionally, there has been work on measuring-device independent (MDI) QKD,
in which Alice and Bob independently prepare phase randomized coherent pulses
in one of the four BB84 states (with decoy states) and send these to an
untrusted third party, Charlie. Charlie then performs Bell state measurements
(BSM), and announces to Alice and Bob over a public channel the successful BSM
events. Alice and Bob can get a sifted key by dropping events where they sent
pulses in different bases (Wang, 2013). This has been implemented and gives
good key rates in the laboratory (Tang et al., 2014). A further improvement
has been examined, using four-source decoy states (Jiang et al., 2015).

There has been recent work to modify the BB84 protocol to deal with higher bit
error rates on the sifted key, in order to distribute quantum keys for longer
distances without using repeaters in a way that is compatible with optical
amplification (Hughes & Norholdt, 2014).

###  6.3. Continuous Variable (CV) QKD

Continuous-variable (CV) QKD protocols employ continuous or discrete
modulations of the quadratures of an electromagnetic field. CV-QKD setups rely
on a coherent detection between the quantum signal and a classical reference
signal, and their implementation requires only standard telecom components.
They are compatible with wavelength division multiplexing, which greatly eases
their deployment into telecommunication networks. They should be easier to
integrate on silicon photonics chips (Jouguet et al., 2013; Kumar, Qin, &
Alléaume, 2014).

The bottleneck for CV-QKD is a classical cryptography problem, that of error-
correction. For a long time, the range of CV-QKD was limited to 25-30km. New
error-correcting codes have improved this range to 80km (Jouguet et al.,
2012). SeQureNet’s Cygnus module for CV-QKD features this range
(http://sequrenet.com/products.html). Currently, CV-QKD keyrates are
competitive with DV-QKD keyrates up to about 30km. But it may be more
difficult to increase ranges for CV-QKD than for DV-QKD because security
proofs for CV-QKD are penalized heavily for finite size effects. An additional
concern is that CV-QKD is a newer technology, and therefore has different
vulnerabilities, some of which may be unmapped. Several potential
vulnerabilities have been identified and addressed in (Jouguet, Kunz-Jacques,
& Diamanti, 2013; Huang et al., 2014; Kunz-Jacques & Jouguet, 2015).

Considering the above, CV-QKD should be considered a promising future
technology for medium-range QKD. The state of the art for CV-QKD is surveyed
in (Jouguet et al., 2014).

###  6.4. Entanglement-based protocols

There are a number of protocols involving entangled pairs of photons, chief
among these being E91 (Ekert, 1991). In E91, Alice and Bob each have half of
an entangled state (EPR pair, or singlet). The working concept of this scheme
is that there is nothing for Eve to intercept, as the qubit state manifests
only after a measurement has been made. If Eve attempts an intercept-resend
attack, her measurement will break the entanglement between the photons.

The protocol proceeds as follows. Alice and Bob each choose one of two
different bases to measure, with 50% probability of choosing one basis and 50%
probability of choosing the other. After having performed their measurements,
they disclose which bases they used over a public channel. If the results of
measurements which were made in different bases violate Bell inequalities,
then the state is still entangled and there has been no eavesdropping.

Despite being theoretically more secure than BB84 and its variants (fewer
side-channels and thus fewer bits required for a secure key), entanglement-
based protocols are not currently considered to be practical for long-range
large-scale systems because of the difficulty of controlling entangled pairs
caused by decoherence (Scarani & Kurtsiefer, 2014).

###  6.5. Counterfactual QKD

Tae-Gon Noh has demonstrated that a QKD can be achieved ostensibly without
sending the key through the quantum channel (Noh, 2009). The quantum principle
in play is that the possibility of sending a photon can be detected even if
the photon is seemingly not actually sent. Counterfactual QKD has been
demonstrated experimentally in the laboratory (Liu et al., 2011).

##  7\. Real-time key extraction

Key generation bandwidth in a pure CPU-based implementation has been shown to
saturate at rates of around 1MHz (Restelli et al., 2009). High speed QKD
networks routinely exceed this data rate— for example, the NIST system
generates sifted keys at around 2MHz and has a maximal capacity above 30MHz.
For secure real-time practical applications, GHz data rates are anticipated.
In order to shift the bottleneck from the classical computation layer back to
the physical layer where it should be, hardware-based implementations have
become necessary.

Sifting is computationally straightforward, and is relatively easy to perform
at high speed. There is good privacy-amplification software which can work
directly on a CPU-based system (Zhang et al., 2014), and the Wegman–Carter
strongly universal hashing method, as used by e.g. Id Quantique, is also good.
It is the error correction step which is complicated and which sets a hard
upper limit on the secure key rate.

The Cascade error-correction algorithm, developed for QKD in (Brassard &
Salvail, 1994), is the fastest at current data rates, and is best implemented
in a Field Programmable Gate Array (FPGA) because it requires many simple but
different logical bit-level operations. When the NIST QKD network began to
exceed data rates of 1MHz, implementation of the Cascade algorithm was moved
to hardware (Mink et al., 2006). The maximal throughput they were able to
achieve was 12MHz in theory, but they were not able to approach that limit in
a practical system due to timing jitter in their photon detectors (Mink &
Bienfang, 2013). The Wuhu metropolitan area QKD network uses a similar FPGA-
based system (Zhang et al., 2012).

For next-generation real-time error correction as data rates push towards the
GHz mark, the Low-Density Parity-Check (LDPC) algorithm is expected to replace
the Cascade algorithm for error correction (Elkouss et al., 2009). The LDPC
algorithm requires 20 to 30 bytes of memory per bit of data being corrected,
as opposed to 1 or 2 bytes for the Cascade algorithm. On an FPGA, LDPC
implementation rates of up to 607MHz have been reported (Mhaske et al., 2015).
The current fastest implementation of the LDPC algorithm runs on a GPU-based
system (Falcão et al., 2009) and has been tested for QKD (Martinez-Mateo et
al., 2013; Dixon & Sato, 2014). For even faster rates, LDPC performance of 47
GHz has been reported for a custom chip implementation but not in the context
of QKD (Zhang et al., 2009).

##  8\. Hardware: Photon sources

A common method of encoding qubits is the use of polarized photons (less
common methods include time-bin encoding (Marcikic et al., 2002) and frequency
encoding (Zhu et al., 2011)). To preclude photon number splitting attacks,
each qubit should be sent on a single photon.

The ideal single-photon source would send a single photon 100% of the time
whenever the user wishes (“on demand”), would send multiple photons 0% of the
time, and the photons it sends would be indistinguishable.

If a photon cannot be sent 100% of the time on demand, the detector might have
to be left on for a longer time, increasing ‘dark count’ (detection of photons
which were not sent to it) and thus increasing noise. If the source were to
send multiple photons, then Eve would be able to intercept one photon and
transmit the remaining photons to Bob, executing a _photon number splitting
attack_. And if photons were distinguishable, then interception one photon
could give non-trivial information about another photon.

Photon sources are classified as _deterministic_ versus _probabilistic_. A
deterministic single photon source emits a single photon on demand, whereas a
probabilistic source might emit more than one photon, and its photon emission
timing not be entirely on demand. One should note, however, that even the most
‘deterministic’ photon source might in practice exhibit probabilistic
behaviour because, for example, photons might get lost during emission with
some probability (“ _extraction loss_ ”).

A common measure for the efficiency of a single-photon source is the _$2$ nd
order correlation function_ $g^{(2)}$. If $g^{(2)}=1$, this means that the
number of photons emitted by the source follows a Poisson distribution, which
is the distribution one would expect from completely random and uncorrelated
emissions. It is usually assumed that $g^{(2)}=1$ for attenuated lasers,
although, as pointed out by the European Telecom Standards Institute, standard
number GS QKD 003 Section 6.4.1 (ETSI, 2015), experiment hasn’t always born
this out and perhaps further study is necessary e.g. when the diode is biased
close to the lasing threshold (Dixon _et al._ , 2009). The $g^{(2)}<1$
situation is referred to as _photon antibunching_. In this case the
probability of emitting one photon relative to the probability of emitting
multiple photons is higher than in a Poisson process. The ideal state is
$g^{(2)}=0$, which means that we get a single photon $100\%$ of the time.

The most common single photon sources are attenuated lasers, in which a laser
beam is sent through a powerful attenuator which weakens it to the point that
the probability of emitting one photon is greater than the probability of
emitting multiple photons. Attenuated lasers are relatively cheap, convenient,
and robust.

When higher performance (lower $g^{(2)}$) is desired, the most common single-
photon sources make use of parametric down-conversion (PDC). This type of
source is not on-demand, but it probabilistically produces a pair of photons,
one of which can be used as a _heralding photon_ to instruct the detector to
activate. This is a major advantage in QKD where it is important to minimize
detector dark-count. The heralding photon could also be used as an entangled
pair with the first photon, although here PDC makes it difficult in general to
obtain the desired wavelength and phase-matching for the pair (Eisaman _et
al._ , 2011).

A promising future technology is the use of nitrogen vacancy (NV) color
centers in diamond as single photon sources. An NV center is a defect in a
diamond lattice which occurs when a nitrogen atom is substituted for a carbon
atom, leaving a vacancy next to it. As single photon sources, NV centers are
on-demand and exhibit low $g^{(2)}$. The current challenges are that they are
not identical, although some tunability has been demonstrated (Tamarat et al.,
2006), and that the ‘shelving level’ reduces their efficiency. There are
several proposed approaches to solving these problems (e.g. (Babinec _et al._
, 2010)); but it is the promise of a single photon coupled with a long-lived
spin qubit (the vacancy itself is an excellent qubit) that makes NV centers
especially promising. Note also that must be thousands of optical defects in
solids which could potentially be used for single-photon generation; only two
of these have so far been seriously studied in this context (Santori, Fattal,
& Yamamoto, 2010).

An NV center used as a single photon source. Retrieved from
http://xqp.physik.uni-muenchen.de/research/single_photon/index.html

There are many other single-photon sources, including single atoms, ions, and
molecules, ensembles, quantum dots, nanowires, four-wave mixing, and
mesoscopic quantum wells, but these do not currently seem as suitable for QKD
as the sources discussed above.

##  9\. Transmission

Quantum key distribution can be performed through fiber, through free space,
or (experimentally) bounced off a satellite. The principles of sending photons
through fiber and through free space are the same, but fiber provides a
channel in which the amount of noise can be determined and even controlled to
some extent, whereas the amount noise in a free-space channel is unknown
(although sometimes one may try to estimate it as in e.g. (Gabay & Arnon,
2005)) and typically is changing. Frequencies used to send photons are
typically around 800nm for free space and around 1550nm for fibers.
Experimental QKD usually uses dark fibers with no other signals passing
through it, but real-world applications will typically involve sending
messages through bright fibers which are carrying other signals. Scattering
effects in bright fibers will raise the BER of Alice’s transmissions, and will
cause more of Alice’s photons to ‘get lost on the way’. Despite this, by
smartly time-filtering QKD photon and other communication photons, in 1992 a
team from Toshiba was able to obtain a secure bit-rate of 507KHz over a 95km
bright fiber, several factors of 10 over what had been achieved previously
(Patel et al., 2012).

The greatest distance positive key rates have been experimentally obtained
through fiber is 307km (Korzh _et al._ , 2015) and through free space is 144km
between two Canary Islands (Ursin _et al._ , 2007). The problem with free
space transmission is atmospheric turbulence— random fluctuations in the
refractive index of air. One potential solution is to bounce the polarized
photons off satellites. The distance to the International Space Station is
400km, but the atmospheric thickness is an order of magnitude smaller than the
Canary Islands experiment. In 2014 a team from the University of Padua bounced
photons off four satellites to show feasibility (Vallone _et al._ , 2014), and
China claims to have done so as well, and aims to have a dedicated QKD
satellite in orbit by 2016 (Yikra, 2014). When such technologies become
practically viable, they will significantly increase free space QKD ranges.

##  10\. Hardware: Photon detection

For Bob to receive qubits from Alice in the form of single photon
polarizations, Bob needs to have a good single photon detector. The main
technological bottleneck in the development of practical and secure QKD
systems for short to medium distances is the development of good single photon
detectors. Thus, in the last few years, any improvement to single photon
detection technology has immediately led to improved QKD capabilities. Our
main reference for this section are (Eisaman _et al._ , 2011) and (Hadfield,
2009).

Single crystal diamond nanowires for photon detection. Retrieved from
http://www.osa-opn.org/opn/media/Images/photocontests/gallery09_36.jpg

An ideal single photon detector for QKD should have 100% _detection
efficiency_ (every photon sent to the detector should be successfully
detected), 0% _dark count_ (the detector should not detect photons which were
not sent to it), no _dead time_ (the recovery time for the detector after it
has detected a photon until it had detected another photon), and no _timing
jitter_ (the time between the photon’s arrival and its registration by the
detector). Additionally, an ideal detector would have complete _photon number
resolution_ , meaning that it would be able to count the number of photons it
had received. It would also be _asynchronous_ , meaning that it need not know
the arrival times of photons in advance.

Low detection capacities and high dark counts create noise in the
communication channel, reducing its capacity. A low capacity channel is
vulnerable to an intercept-resent attack, because it is difficult to detect
eavesdropping in the presence of random noise (Section 12.1). High dead time
reduces the channel bit rate, and creates a vulnerability to faked-state
attacks and to time-shift attacks (Makarov _et al._ , 2006; Burenkov _et al._
, 2010; Makarov, 2009). Timing jitter can lead to a leak of secret key
information (Lamas-Linares & Kurtsiefer, 2007). Poor or nonexistent photon
number resolution creates a vulnerability to photon number splitting attacks.

A single photon detector typically works by converting a photon into a charge
carrier which in turn triggers an avalanche process in a physical system which
is held very close to a critical state, leading to a macroscopic current
pulse.

Superconducting nanowire single photon detectors (SNSPD) are the best single
photon detectors known currently. They were first developed in 2001. They have
high detection efficiency (ten times better than the best semiconductor
detectors), low dark count, low dead time, and low time jitter. They are also
fully asynchronous. Their drawback is that they require cryogenic temperatures
($<3K$) to operate, which makes them bulky and expensive, and has limited
their uses outside the lab. There is work being done, however, on variations
of SNSPD that can function at temperatures of over $20K$, making them a
promising future technology for military and government applications (Wang,
Miki, & Fujiwara, 2009).

Single photon avalanche detectors (SPADs) are the single photon detectors
which are most currently used in practice. They are cheap and compact, with
high detection efficiency and low time jitter. They are also fully
asynchronous. The challenge in building good SPADs has been _afterpulsing_ ,
which is the phenomenon of a spontaneous dark count occurring shortly after a
photon detection. If we wait until afterpulsing ends before reactivating the
SPAD, then we increase the dead time.

There has recently been dramatic progress in SPAD design. In 2013, the
University of Geneva Applied Physics team developed an InGaAs negative
feedback avalanche diode (NFAD) single photon detector whose performance
rivals many SNSPD systems, but which operates at temperatures of approximately
150–220 K (as opposed to $<3K$ for SNSPD systems) (Korzh _et al._ , 2014).
Using these InGaAs NFADs, the same team were able in February 2015 to
demonstrate provably secure QKD transmission over 307km of optical fiber,
which is the current record (Korzh _et al._ , 2015).

##  11\. Auxiliary systems

###  11.1. Random number generators

A QKD system is only as good as its random number generator. If Eve can
predict Alice and Bob’s random choices, they she can read the entire key. The
entire selling point of Quintessence Technologies is their random number
generators.

CPU-based random number generators are trusted for many classical cryptography
tasks, and are implemented in most operating systems. The numbers they produce
are not truly random, however, and therefore they are usually referred to as
_pseudorandom number generators_. When higher speeds are required and when
stronger random numbers are needed, hardware-based implementations are
preferred. These come in two flavours— they either use filtered random
physical processes within the FPGA as random number seeds (Tsoi et-al., 2007;
Kwok & Lam, 2006), or they use less random seeds and strong permutations
(Alimohammad et al., 2008; Cheung et al., 2007; Xiang & Benkrid, 2009).
Currently, both alternatives are considered cryptographically equivalent.

In QKD, in order to physically guarantee unconditional security, quantum
effects are desired for use as true random number generators (TRNG). A quick
and dirty way to do this, for Alice at least, is to send an unpolarized single
photon through a beam splitter— if it comes out one end then count that as a
zero, and if it comes out of the other end count it as a 1. A more
sophisticated version of this scheme which eliminates this bias is marketed by
Id Quantique, and another by Quintessence Technologies, and reaches rates of
16 KHz. Looking into the future, an experimental idea with great promise is to
use quantum vacuum fluctuations for high bandwidth truly random number
generation of up to 100GHz (Jofre et al., 2011).

We note that TRNGs arise as commercial spinoffs of QKD projects.

###  11.2. Memories and repeaters

The concept of a quantum repeater. Retrieved from
http://www.uqcc.org/research/index.html

To extend the range of QKD beyond a few hundred kilometers, quantum repeaters
will be necessary, which in turn will require quantum memories. A quantum
memory absorbs a photon, stores its quantum states for as long as possible,
and releases it on demand. A key feature is that it does not break
entanglement. The primary candidates for practical quantum memories for QKD in
the near future are Raman gas based quantum memories (Simon et al., 2010) and
quantum memories using NV centers (de Riedmatten & Afzelius, 2015). The
advantages of the former include its greater capacity, while the advantages of
the latter include that it is solid-state and that it allows longer storage
times. It is still unclear which of these approaches will be best.

It is still unclear which repeater technology will be best, although the first
quantum repeaters which outperform direct transmission will probably be based
on atomic ensembles, linear optics, and photon counting (Sangouard et al.,
2011).

Quantum memories and repeaters are expected to become a commercial technology
within 10-15 years.

##  12\. Attacks

While the protocols of QKD operating under certain conditions are
unconditionally secure, practical implementations have been successfully
attacked. While none of these attacks is fatal to the QKD concept— effective
countermeasures to each attack have been devised— it is generally agreed that
the security of each setup should be the object of a dedicated study whose
goal is to find and patch up all vulnerabilities (Scarani & Kurtsiefer, 2014).

###  12.1. Intercept-resend

The simplest and most direct attack against BB84 and its relatives is for Eve
to intercept a photon sent by Alice to Bob, to measure that photon, to prepare
her own photon encoding the bit which she measured, and to send that photon
off to Bob. Because Eve doesn’t know in which basis Alice’s photon was sent,
she’ll measure in the wrong basis approximately half of the time and she will
send a photon in the wrong basis to Bob approximately half of the time. An
intercept-resend attack thus introduces a bit error rate (BER) of around 25%,
although weaknesses in certain practical systems allow modified versions of
this attack to introduce BERs of 19.7% (Xi, Qi, & Lo, 2010). Because
acceptably BERs in commercial systems are around 8%, practical QKD is indeed
secure against pure intercept-resent attacks, which are caught during the
error-correction key-establishment phase— if the error rate is too high then
Eve has been there.

###  12.2. Photon Number Splitting (PNS)

Due to hardware limitations, most photon sources used in QKD are not true
single photon sources in the sense that there is a non-negligible probability
that they will generate multiple photons to transmit a single qubit. If Alice
sends two or more identical photons to Bob, then Eve can split off one photon
and send the remaining photons through. Eve stores the qubit she has learnt in
quantum memory until Alice has revealed her encoding basis. Then Eve measures
her photons in the correct basis and gains information about the key.

A successful photon number splitting attack requires sophisticated equipment—
Eve must be able to count photons and to split off just one to quantum memory
while sending others through. Moreover, various countermeasures have been
developed. Better single-photon sources and modifications of the BB84
protocol, such as for instance SARG04, make successful PNS attacks much more
difficult to carry out. Another solution is to use decoy states, in which
photons are randomly sent at different intensities. The security of decoy
state QKD against PNS attacks was proven in (Lo _et al._ , 2005a). Because a
successful photon number splitting attack is much more difficult to carry out
against decoy-state QKD, we can use attenuated lasers as photon sources when
transmitting keys, increasing secure key-rates (Yuan, 2007). The current
state-of-the-art for decoy-state QKD is 320MHz over a 200km fiber, yielding a
15Hz secure key rate (Liu _et al._ , 2010).

###  12.3. Timing attacks

When different light sources are used for beams in different polarizations,
and/or different detectors are used to make different measurements, it may be
possible to ‘listen in’ to which bit was sent or to which bases was used
without actually intercepting a photon. Such side-channels were evident
already in the first implementations QKD, as noted by Brassard (Brassard,
2005):

> The funny thing is that, while our theory had been serious, our prototype
> was mostly a joke. Indeed, the largest piece in the prototype was the power
> supply needed to feed in the order of one thousand volts to Pockels cells,
> used to turn photon polarization. But power supplies make noise, and not the
> same noise for the different voltages needed for different polarizations.
> So, we could literally hear the photons as they flew, and zeroes and ones
> made different noises. Thus, our prototype was unconditionally secure
> against any eavesdropper who happened to be deaf ! :-)

It is therefore critical to the security of the QKD system that different
light-sources and detectors be indistinguishable to Eve. One particular
vulnerability is that different light sources and detectors may not be
perfectly synchronized, so that Eve can figure out which detector clicked, for
example, by examining the time signature publicly announced by Bob in order to
determine which photon he detected of the photons sent by Alice. Such an
attack could read-off $\geq 25\%$ of the key for a detector mismatch of $0.5$
nanoseconds, an amount that could easily go unnoticed (Lamas-Linares &
Kurtsiefer, 2007). An attempt to carry out such an attack against a commercial
system was unsuccessful because of several practical difficulties (Zhao et
al., 2008).

###  12.4. Trojan attacks

The logo of the Quantum Hacking Lab. Retrieved from http://www.vad1.com/lab/

In a Trojan attack, Eve shines bright light at either Alice or Bob,
determining which base was used by analyzing the reflection. The Trojan attack
has successfully read off complete keys both in the lab (Gerhardt et al.,
2011) and also of commercial QKD systems, QPN-5505 from MagiQ Technologies and
Clavis2 of IP Quantique (Lydersen et al., 2010). This has been the most
powerful and the best-performing hack on QKD so far. Although these specific
attacks can be protected against, Trojan attacks using pulses of different
wavelengths may still be able to hack complete keys, and we still do not know
the full scope of the vulnerability of practical QKD systems to Trojan attacks
(Jain et al., 2014, 2015).

###  12.5. Other side-channel attacks

Many other attacks against QKD systems have been investigated, and new
vulnerabilities are periodically discovered. Some of these attacks (denial of
service, man-in-the-middle,…) can be carried out against classical systems as
well, and the vulnerability of QKD to these attacks is identical to the
vulnerability of any classical protocol. Other attacks which take advantage of
a weakness in an auxiliary system— e.g. a randomization attacks, in which the
random bases are successfully computed by Eve because the random number
generator is faulty— can be counteracted by using better hardware. Of course
each side-channel attack must be investigated and ruled out for each setup.

##  13\. Conclusion

Quantum Key Distribution (QKD) is a modern form of trusted courier, which in
principle allows Alice to communicate a message to Bob with complete
confidence that the message will not be eavesdropped on during transmission.
Real-life QKD security, however, is a quantitative issue, and each setup
should be individually studied to ensure its security. QKD is currently a
focus of interest for many private, governmental, and military groups all over
the world.

Current state of the art setups still use the first QKD protocol, BB84, and
its variants. The Cascade protocol is still the fastest for error correction,
but LDPC is expected to overtake it as key rates rise. In both cases, hardware
implementation using FPGA’s is the current state of the art and is likely to
remain so for the next decade at least.

Qubits are typically transmitted as polarized photons. Decoy-state QKD using
attenuated laser pulses are the current state of the art photon sources,
despite not being true single photon sources. NV centers are a promising
future technology. Polarized photons can be transmitted through fiber, through
air (free space), or bounced off satellites. There are various attempts to
send polarized photons via bright fibers through which other messages are
travelling, but the key rates being obtained are still quite low.

Photon detectors are the main technological bottleneck for practical QKD. The
current state of the art are InGaAs NFAD’s. A promising future technology are
SNSPD’s, which currently require cryogenic temperatures to operate, but future
SNSPD’s be able to operate at above 20K.

Memories and repeaters, which are thought to be required for QKD at ranges
over around 400km, are still in the experimental stage, and it is too early to
say which technology will be best.

Security of QKD is a well-studied field, and there have been numerous attempts
to attack QKD implementations both using standard attacks and also using side-
channel attacks. Only one of these attacks, a Trojan attack, has successfully
stolen a secret key, and the vulnerability it highlights can be plugged. QKD
of course has the same vulnerability to classical attacks such as denial-of-
service and man-in-the-middle as classical implementations.

QKD is an exciting emerging technology which is beginning to enter the
marketplace. We expect it to be successful in its own right, and also to serve
as a stepping stone towards greater and higher goals in quantum communication
and computation.

##  14\. Acknowledgement

The author wishes to thank Judith Kupferman for useful comments, and wishes to
thank Paul Jouguet at Takehisa Iwakoshi for corrections and references.

## References

  * Alimohammad et al. (2008) Alimohammad, A., Fard, S.F., Cockburn, B.F., & Schlegel, C. 2008 A Compact and Accurate Gaussian Variate Generator.  _IEEE T. VLSI Syst._ , 16(5), 517–527.
  * Babinec _et al._ (2010) Babinec, T.M., Hausmann, B.J., Khan, M., Zhang, Y., Maze, J.R., Hemmer, P.R., & Lončar, M. 2010 A diamond nanowire single-photon source.  _Nat. Nanotechnol._ , 5(3), 195–199.
  * Bennett & Brassard (1984) Bennett, C.H. & Brassard, G. 1984 Quantum cryptography: Public key distribution and coin tossing.  In _Proceedings of IEEE International Conference on Computers, Systems and Signal Processing_ , volume 175, page 8.  http://researcher.watson.ibm.com/researcher/files/us-bennetc/BB84highest.pdf
  * Bennett et al. (1992) Bennett, C.H., Bessette, F., Brassard, G., Salvail, L., & Smolin, J. 1992 Experimental quantum cryptography.  _J. of Cryptography_ , 5(1) 3–28.
  * Berry (1998) Berry, M. 1998 Foreword, in _Introduction to Quantum Computation and Information_ eds. Lo, H.-K., Popescu S., & Spiller, T. World Scientific, Singapore, hardcover 1998, paperback 2001.
  * Branciard et al. (2005) Branciard, C., Gisin, N., Kraus, B., & Scarani, V. 2005 Security of two quantum cryptography protocols using the same four qubit states.  _Phys. Rev. A_ , 72(3), 032301.  arXiv:quant-ph/0505035
  * Brassard & Salvail (1994) Brassard, G., & Salvail, L. 1994 Secret-key reconciliation by public discussion.  In _Advances in Cryptology EUROCRYPT93_ , Springer Berlin Heidelberg, 410–423.
  * Brassard (2005) Brassard, G. 2005 Brief history of quantum cryptography: A personal perspective.  In _Theory and Practice in Information-Theoretic Security, 2005_. IEEE Information Theory Workshop, 19–23.  arXiv:quant-ph/0604072
  * Burenkov _et al._ (2010) Burenkov, V., Qi, B., Fortescue, B., & Lo H.-K. 2010 Security of high speed quantum key distribution with finite detector dead time.  _Quantum Inf. Comput._ , 14(3-4), 217–235.  arXiv:1005.0272
  * Cheung et al. (2007) Cheung, R.C.C., Lee, D.-U., Luk, W., & Villasenor, J.D. 2007 Hardware Generation of Arbitrary Random Number Distributions From Uniform Distributions Via the Inversion Method.  _IEEE T. VLSI Syst._ , 15(8), 952–962.
  * Comandar _et al._ (2014) Comandar, L.C., Fröhlich, B., Lucamarini, M., Patel, K.A., Sharpe, A.W., Dynes, J.F., Yuan, Z.L., Penty, R.V., & Shields, A.J. 2014 Room temperature single-photon detectors for high bit rate quantum key distribution.  _Appl. Phys. Lett._ , 104, 021101.  arXiv:1402.2210
  * de Riedmatten & Afzelius (2015) de Riedmatten, H., & Afzelius, M. 2015 Quantum Light Storage in Solid State Atomic Ensembles.  Preprint.  arXiv:1502.00307
  * Dixon _et al._ (2009) Dixon, A.R., Dynes, J.F., Yuan, Z.L., Sharpe, A.W., Bennett, A.J., & Shields, A.J. 2009 Ultrashort dead time of photon-counting InGaAs avalanche photodiodes  _Appl. Phys. Lett._ , 94(23), 231113.  arXiv:0905.2931
  * Dixon & Sato (2014) Dixon, A.R., & Sato, H. 2014 High speed and adaptable error correction for megabit/s rate quantum key distribution. _Scientific Reports_ , 4.
  * Eisaman _et al._ (2011) Eisaman, M.D., Fan, J., Migdall, A., & Polyakov, S.V. 2011 Single-photon sources and detectors.  _Rev. Sci. Instrum._ , 82(7), 071101.
  * Ekert (1991) Ekert, A.K. 1991 Quantum cryptography based on Bell’s theorem.  _Phys. Rev. Lett._ 67(6), 661.
  * Elkouss et al. (2009) Elkouss, D., Leverrier, A., Alléaume, R., & Boutros, J.J. 2009 Efficient reconciliation protocol for discrete-variable quantum key distribution.  In _IEEE International Symposium on Information Theory, 2009_ , ISIT 2009, 1879–1883).
  * Elliot et al. (2005) Elliott, C., Colvin, A., Pearson, D., Pikalo, O., Schlafer, J., & Yeh, H. 2005 Current status of the DARPA quantum network.  In _Defense and Security_ , International Society for Optics and Photonics, 138–149.  arXiv:quant-ph/0503058
  * EPSRC (2014) UK Quantum Technology Landscape, 2014.  Retrieved from http://www.epsrc.ac.uk/newsevents/pubs/dstl-uk-quantum-technology-landscape-2014/
  * ETSI (2015) European Telecom Standards Institute, Standards for Quantum Key Distribution.  Retrieved from http://www.etsi.org/technologies-clusters/technologies/quantum-key-distribution
  * Falcão et al. (2009) Falcão, G., Silva, V., & Sousa, L. 2009 How GPUs can outperform ASICs for fast LDPC decoding.  In _Proceedings of the 23rd international conference on Supercomputing_ , ACM, 390–399.
  * Gabay & Arnon (2005) Gabay, M., & Arnon, S. 2005 Effect of turbulence on a quantum-key distribution scheme based on transformation from the polarization to the time domain: laboratory experiment.  _Opt. Engineering_ , 44(4), 045002.
  * Gerhardt et al. (2011) Gerhardt, I., Liu, Q., Lamas-Linares, A., Skaar, J., Kurtsiefer, C., & Makarov, V. 2011 Full-field implementation of a perfect eavesdropper on a quantum cryptography system.  _Nat. Commun._ , 2, 349.  arXiv:1011.0105
  * Gisin et al. (2002) Gisin, N., Ribordy, G., Tittel, W., & Zbinden, H. 2002 Quantum cryptography.  _Rev.Mod.Phys._ , 74, 145–190.
  * Hadfield (2009) Hadfield, R.H. 2009 Single-photon detectors for optical quantum information applications.  _Nat. Photonics_ , 3(12), 696–705.
  * Huang et al. (2014) Huang, J.Z., Kunz-Jacques, S., Jouguet, P., Weedbrook, C., Yin, Z.Q., Wang, S., Chen, W., Guo, G.C., & Han, Z.F. (2014). Quantum hacking on quantum key distribution using homodyne detection.  _Phys. Rev. A_ , 89(3), 032304.  arXiv:1402.6921
  * Hughes et al. (2013) Hughes, R.J., Nordholt, J.E., McCabe, K.P., Newell, R.T., Peterson, C.G., & Somma, R.D. 2013 Network-Centric Quantum Communications.  In _Frontiers in Optics_ , Optical Society of America.  arXiv:1305.0305
  * Hughes & Norholdt (2014) Hughes, R.J., & Nordholt, J.E. 2014 Long-range quantum cryptography: Amplified quantum key distribution (AQKD).  Preprint.  arXiv:1406.6990
  * Inamori, Lütkenhaus & Mayers (2007) Inamori, H., Lütkenhaus, N., & Mayers, D. 2007 Unconditional security of practical quantum key distribution.  _Eur. Phys. J. D._ , 41(3), 599–627.  arXiv:quant-ph/0107017
  * Iwakoshi & Hirota (2014) Iwakoshi, T., & Hirota, O. 2014 Misinterpretation of statistical distance in security of quantum key distribution shown by simulation.  In _SPIE Security + Defence_ , International Society for Optics and Photonics, 92540L–92540L.
  * Jain et al. (2014) Jain, N., Anisimova, E., Khan, I., Makarov, V., Marquardt, Ch., & Leuchs, G. 2014 Trojan-horse attacks threaten the security of practical quantum cryptography.  _New J. Phys._ , 16(12), 123030.  arXiv:1406.5813
  * Jain et al. (2015) Jain, N., Stiller, B., Khan, I., Makarov, V., Marquardt, Ch., & Leuchs, G. 2015 Risk analysis of Trojan-horse attacks on practical quantum key distribution systems.  _IEEE J. Sel. Top. Quantum Electron._ 21, 6600710.  arXiv:1408.0492
  * Jiang et al. (2015) Jiang, H., Gao, M., Wang, H., Li, H., & Ma, Z. 2015 Four-intensity Decoy-state Quantum Key Distribution with enhanced resistance against statistical fluctuation.  Preprint.  arXiv:1502.02249
  * Jofre et al. (2011) Jofre, M., Curty, M., Steinlechner, F., Anzolin, G., Torres, J.P., Mitchell, M.W. & Pruneri, V. 2011 True random numbers from amplified quantum vacuum.  _Opt. Express_ , 19(21), 20665–20672.
  * Jouguet et al. (2012) Jouguet, P., Kunz-Jacques, S., Leverrier, A., Grangier, P., & Diamanti, E. 2013 Experimental demonstration of long-distance continuous-variable quantum key distribution.  _Nat. Photonics_ , 7(5), 378–381.  arXiv:1210.6216
  * Jouguet, Kunz-Jacques, & Diamanti (2013) Jouguet, P., Kunz-Jacques, S., & Diamanti, E. 2013 Preventing calibration attacks on the local oscillator in continuous-variable quantum key distribution.  _Phys. Rev. A_ , 87(6), 062313.  arXiv:1304.7024
  * Jouguet et al. (2013) Jouguet, P., Kunz-Jacques, S., Kumar, R., Qin, H., Gabet, R., Diamanti, E., & Alléaume, R. 2013 Experimental demonstration of the coexistence of continuous-variable quantum key distribution with an intense DWDM classical channel.  In _Proc. 3rd Ann. Conf. Quantum Cryptography_.
  * Jouguet et al. (2014) Jouguet, P., Elkouss, D., & Kunz–Jacques, S. 2014 High-bit-rate continuous-variable quantum key distribution.  _Phys. Rev. A_ 90, 042329.
  * König et al. (2007) König, R., Renner, R., Bariska, A., & Maurer, U. 2007 Small accessible quantum information does not imply security.  _Phys. Rev. Lett._ , 98(14), 140502.  arXiv:quant-ph/0512021
  * Korzh _et al._ (2014) Korzh, B., Walenta, N., Lunghi, T., Gisin, N., & Zbinden, H. 2014 Free-running InGaAs single photon detector with 1 dark count per second at 10% efficiency.  _Appl. Phys. Lett._ , 104, 081108.  arXiv:1312.2636
  * Korzh _et al._ (2015) Korzh, B., Lim, C.C.W., Houlmann, R., Gisin, N., Li M.J., Nolan, D., Sanguinetti, B., Thew, R., & Zbiden, H. 2015 Provably secure and practical quantum key distribution over 307km of optical fibre.  _Nat. Photonics_.  arXiv:1407.7427
  * Kumar, Qin, & Alléaume (2014) Kumar, R., Qin, H., & Alléaume, R. 2014 Coexistence of continuous variable QKD with intense DWDM classical channels.  Preprint.  arXiv:1412.1403
  * Kunz-Jacques & Jouguet (2015) Kunz-Jacques, S., & Jouguet, P. 2015 Robust Shot Noise Measurement for Continuous Variable Quantum Key Distribution.  _Phys. Rev. A_ 91, 022307.  arXiv:1406.7554
  * Kwok & Lam (2006) Kwok, S.H.M., & Lam, E.Y. 2006 FPGA–based High-speed True Random Number Generator for Cryptographic Applications.  _TENCON 2006. 2006 IEEE Region 10 Conference_ , 1–4 & 14–17.
  * Lamas-Linares & Kurtsiefer (2007) Lamas-Linares, A., & Kurtsiefer, C. 2011 Breaking a quantum key distribution system through a timing side channel.  _Opt. Express_ , 15, 9388.  arXiv:0704.3297
  * Liu _et al._ (2010) Liu, Y., Chen, T.-Y., Wang, J., Cai, W.-Q., Wan, X., Chen, L.-K., Wang, J.-H., Liu, S.-B. Liang, H., Yang, L., Peng, C.-Z. Chen, K., Chen, Z.-B. & Pan J.-W. 2010 Decoy–state quantum key distribution with polarized photons over 200 km.  _Opt. Express_ , 18 (2010) 8587–8594.  arXiv:0908.4063
  * Liu et al. (2011) Liu, Y., Ju, L., Liang, X.L., Tang, S.B., Tu, G.L.S., Zhou, L., Peng, C.-Z., Chen, K. Chen, T.-Y. Chen, Z.-B. & Pan, J.W. 2012 Experimental demonstration of counterfactual quantum communication.  _Phys. Rev. Lett._ , 109(3), 030501.  arXiv:1107.5754
  * Lo _et al._ (2005a) Lo, H.K., Ma, X., & Chen, K. 2005 Decoy state quantum key distribution.  _Phys. Rev. Lett._ , 94(23), 230504.  arXiv:quant-ph/0411004
  * Lo _et al._ (2005b) Lo, H.-K., Chau, H.F. & Ardehali, M. 2005 Efficient quantum key distribution scheme and proof of its unconditional security.  _J. of Crypt._ , 18, 133.  arXiv:quant-ph/0011056
  * Lucamarini _et al._ (2013) Lucamarini, M., Patel, K.A., Dynes, J.F., Fröhlich, B., Sharpe, A.W., Dixon, A.R., Yuan, Z.L., Penty, R.V., & Shields, A. J. 2013 Efficient decoy-state quantum key distribution with quantified security.  _Opt. Express_ , 21(21), 24550–24565.  http://arxiv.org/abs/1310.0240
  * Lydersen et al. (2010) Lydersen, L., Wiechers, C., Wittmann, C., Elser, D., Skaar, J., & Makarov, V. 2010 Hacking commercial quantum cryptography systems by tailored bright illumination.  _Nat. Photonics_ , 4(10), 686–689.  arXiv:1008.4593
  * Ma & Lo (2008) Ma, X., & Lo, H.K. 2008 Quantum key distribution with triggering parametric down-conversion sources.  _New J. Phys._ , 10(7), 073018.
  * Makarov _et al._ (2006) Makarov, V., Anisimov, A. & Skaar, J. 2006 Effects of detector efficiency mismatch on security of quantum cryptosystems.  _Phys. Rev. A_ , 74, 022313; Erratum _Phys. Rev. A_ , 78, 019905.  arXiv:quant-ph/0511032
  * Makarov (2009) Makarov, V. 2009 Controlling passively quenched single photon detectors by bright light.  _New J. Phys._ , 11(6), 065003.  arXiv:quant-ph/0707.3987
  * Marcikic et al. (2002) Marcikic, I., De Riedmatten, H., Tittel, W., Scarani, V., Zbinden, H., & Gisin, N. 2002 Time-bin entangled qubits for quantum communication created by femtosecond pulses.  _Phys. Rev. A_ , 66(6), 062308.  arXiv:quant-ph/0205144
  * Martinez-Mateo et al. (2013) Martinez-Mateo, J., Elkouss, D., & Martin, V. 2013 Key Reconciliation for High Performance Quantum Key Distribution.  _Scientific Reports_ , 3.  http://www.nature.com/srep/2013/130402/srep01576/full/srep01576.html
  * Mhaske et al. (2015) Mhaske, S., Kee, H., Ly, T., Aziz, A., & Spasojevic, P. 2015 High-throughput FPGA-based QC-LDPC decoder architecture.  Preprint.  arXiv:quant-ph/1503.02986.
  * Mink et al. (2006) Mink, A., Tang, X., Ma, L., Nakassis, T., Hershman, B., Bienfang, J.C., Su, D., Boisvert, R., Clark, C.W., & Williams, C. J. 2006 High speed quantum key distribution system supports one-time pad encryption of real-time video.  In _Defense and Security Symposium_ , International Society for Optics and Photonics, 62440M.
  * Mink & Bienfang (2013) Mink, A., & Bienfang, J. 2013 QKD on a Board Limited by Detector Rates in a Free-Space Environment.  In _ICQNM 2013, The Seventh International Conference on Quantum, Nano and Micro Technologies_ , 28–33.
  * Mirza & Petruccione (2010) Mirza, A., & Petruccione, F. 2010 Realizing long-term quantum cryptography.  _JOSA B_ , 27(6), A185–A188.
  * Mitsubishi (2015) Quantum Cryptography: Realizing next-generation security that cannot be broken. [R& D Highlights] 2015.  Retrieved from http://www.mitsubishielectric.com/company/rd/research/highlights/communications/quantum.html
  * Nazarathy et al. (2006) Nazarathy, M., Tselniker, I., Regev, Y., Orenstein, M., & Katz, M. 2006 Integrated–optical realizations of quantum key distribution over maximally unbiased bases.  _IEEE J. Sel. Top. Quant._ , 12(4), 897–913.
  * Noh (2009) Noh, T.-G. 2009 Counterfactual quantum cryptography.  _Phys. Rev. Lett._ , 103(23), 230501.  arXiv:0809.3979
  * Patel et al. (2012) Patel, K.A., Dynes, J.F., Choi, I., Sharpe, A.W., Dixon, A.R., Yuan, Z.L., Penty, R.V. & Shields, A.J. 2012 Coexistence of high-bit-rate quantum key distribution and data on optical fiber.  _Phys. Rev. X_ , 2(4), 041010.  arXiv:1212.0033
  * Portmann & Renner (2014) Portmann, C., & Renner, R. 2014 Cryptographic security of quantum key distribution.  Preprint.  arXiv:1409.3525
  * Pritchard & Till (2010) Pritchard, J. & Till, S. 2014 UK Quantum Technology Landscape 2014.  Engineering and Physical Sciences Research Council Report DSTL/PUB75620  Retrieved from http://www.epsrc.ac.uk/newsevents/pubs/dstl-uk-quantum-technology-landscape-2014/
  * Restelli et al. (2009) Restelli, A., Bienfang, J.C., Mink, A., & Clark, C.W. 2009 Quantum key distribution at GHz transmission rates.  In _SPIE OPTO: Integrated Optoelectronic Devices_ , International Society for Optics and Photonics, 72360L.
  * Sangouard et al. (2011) Sangouard, N., Simon, C., De Riedmatten, H., & Gisin, N. 2011 Quantum repeaters based on atomic ensembles and linear optics.  _Rev. Mod. Phys._ , 83(1), 33.  http://www.arxiv.org/abs/quant-ph/0906.2699
  * Santori, Fattal, & Yamamoto (2010) Santori, C., Fattal, D., & Yamamoto, Y. 2010 _Single-photon Devices and Applications_.  John Wiley & Sons.
  * Sarwar Pasha & Bala Ram (2014) Sarwar Pasha M.D. & Bala Ram, A. 2014 Key pre-distribution using quantum key channel— A survey.  _International Journal of Computer Science and Mobile Applications_ , 2(3) 85–101.
  * Sasaki et al. (2011) Sasaki, M., Fujiwara, M., Ishizuka, H., Klaus, W., Wakui, K., Takeoka, M., Tanaka, A., Yoshino, K., Nambu, Y., Takahashi, S., Tajima, A., Tomita, A., Domeki, T., Hasegawa, T., Sakai, Y., Kobayashi, H., Asai, T., Shimizu, K., Tokura, T., Tsurumaru, T., Matsui, M., Honjo, T., Tamaki, K., Takusue, H., Tokura, Y., Dynes, J.F., Dixon, A.R., Sharpe, A.W., Yuan, Z.-L., Shields, A.J., Uchikoga, S., Legré, M., Robyr, S., Trinkler, P., Monat, L., Page, J.-B., Ribordy, G., Poppe, A., Allacher, A., Maurhart, O., Länger, T., Peev, M., & Zeilinger, A. 2011 Field test of quantum key distribution in the Tokyo QKD Network.  _Opt. Express_ , 19(11), 10387–10409.  arXiv:1103.3566
  * Scarani et al. (2004) Scarani, V., Acín, A., Ribordy, G. & Gisin, N. 2004 Quantum Cryptography Protocols Robust against Photon Number Splitting Attacks for Weak Laser Pulse Implementations.  _Phys. Rev. Lett._ 92, 057901.
  * Scarani & Kurtsiefer (2014) Scarani, V., & Kurtsiefer, C. 2014 The black paper of quantum cryptography: Real implementation problems.  _Theor. Comput. Sci._ , 560, 27–32.  arXiv:0906.4547
  * SECOQC (2007) Alléaume, R., Bouda, J., Branciard, C., Debuisschert, T., Dianati, M., Gisin, N., Godfrey, M., Grangier, P., Länger, T., Leverrier, A., Lütkenhaus, N., Painchault, P., Peev, M., Poppe, A., Pornin, T., Rarity, J., Renner, R., Ribordy, G., Riguidel, M., Salvail, L., Shields, A., Weinfurter, H., & Zeilinger, A. 2007 SECOQC White Paper on Quantum Key Distribution and Cryptography.  Preprint.  arXiv:quant-ph/0701168
  * Simon et al. (2010) Simon, C., Afzelius, M., Appel, J., Boyer de La Giroday, A., Dewhurst, S.J., Gisin, N., Hu., C.-Y., Jelezko, F., Kröhl, S., Müller, J.H., Nunn, J., Polzik, E., Rarity, J., de Riedmatten, H., Rosenfeld, W., Shields, A.J., Sköld, N., Stevenson, R.M., Thew, R., Walmsley, I., Weber, M., Weinfurter, H., Wrachtrup, J., & Young, R.J. 2010 Quantum memories.  _Eur. Phys. J. D._ , 58(1), 1–22.  arXiv:1003.1107
  * Tamarat et al. (2006) Tamarat, Ph., Gaebel, T., Rabeau, J.R., Khan, M., Greentree, A.D, Wilson, H., Hollenberg, L.C.L., Prawer, S., Hemmer, P., Jelezko, F., & Wrachtrup J. 2006 Stark shift control of single optical centers in diamond.  _Phys. Rev. Lett._ , 97(8), 083002.  arXiv:quant-ph/0607170
  * Tang et al. (2014) Tang, Z., Liao, Z., Xu, F., Qi, B., Qian, L., & Lo, H.K. 2014 Experimental demonstration of polarization encoding measurement-device-independent quantum key distribution.  _Phys. Rev. Lett._ , 112(19), 190503.  http://2013.qcrypt.net/contributions/Tang-abstract.pdf
  * Tomamichel et al. (2012) Tomamichel, M., Lim, C.C.W., Gisin, N., & Renner, R. 2012 Tight finite-key analysis for quantum cryptography.  _Nat. Commun._ , 3, 634.
  * Tsoi et-al. (2007) Tsoi, K.H., Leung, K.H., & Leong, P.H.W. 2007 High performance physical random number generator.  _IET Comput. Digit. Tech._ , 1(4), 349–352.
  * Ursin _et al._ (2007) Ursin, R., Tiefenbacher, F., Schmitt-Manderbach, T., Weier, H., Scheidl, T., Lindenthal, M., Blauensteiner, B., Jennewein, T., Perdigues, J., Trojek, P., Ömer, B., Fürst, M., Meyenburg, M., Rarity, J., Sodnik, Z., Barbieri, C., Weinfurter, H.& Zeilinger, A. (2007) Entanglement-based quantum communication over 144 km.  _Nature Phys._ , 3(7), 481–486.
  * Vallone _et al._ (2014) Vallone, G., Bacco, D., Dequal, D., Galarin, S., Luceri, V., Bianco, G., & Villoresi, P. 2014 Experimental Satellite Quantum Communications  Preprint.  arXiv:quant-ph/1406.4051
  * Wang, Miki, & Fujiwara (2009) Wang, Z., Miki, S., & Fujiwara, M. 2009 Superconducting nanowire single-photon detectors for quantum information and communications.  _IEEE J. Sel. Top. Quantum Electron._ , 15(6), 1741–1747.
  * Wang (2013) Wang, X.B. 2013 Three-intensity decoy-state method for device-independent quantum key distribution with basis-dependent errors.  _Phys. Rev. A_ , 87(1), 012320.  arXiv:1308.5677
  * Wang et al. (2014) Wang, S., Chen, W., Yin, Z.-Q., Li, H.-W., He, D.-Y., Li, Y.-H., Wang, D., Chen, H., Han, Y.-G., Huang, J.-Z., Guo, J.-F., Hao, P.-L., Li, M., Zhang, C.-M., Liu, D., Liang, W.-Y., Miao, C.-H., Wu, P., Guo, G.-C., & Han, Z.F. 2014 Field and long-term demonstration of a wide area quantum key distribution network.  _Opt. Express_ , 22(18), 21739–21756.  arXiv:1409.1568
  * Weisner (1983) Wiesner, S. 1981 Conjugate coding.  _SIGACT News_ , 15(1), 78–88.  Original manuscript written circa 1969.
  * Xi, Qi, & Lo (2010) Xu, F., Qi, B., & Lo, H.K. 2010 Experimental demonstration of phase-remapping attack in a practical quantum key distribution system.  _New J. Phys._ , 12(11), 113026.  arXiv:1005.2376
  * Xiang & Benkrid (2009) Xiang, T., & Benkrid, K. 2009 Mersenne Twister Random Number Generation on FPGA, CPU and GPU.  _NASA/ESA Conference on Adaptive Hardware and Systems_ , 460–464.
  * Yikra (2014) Yikra,B. 2014 Researchers bounce polarized photons off satellites to show feasibility of space based quantum communications.  _Phys.org_. Retrieved from http://phys.org/news/2014-06-polarized-photons-satellites-feasibility-space.html
  * Yuan (2007) Yuan, Z.L., Sharpe, A.W., Shields, A.J. 2007 Unconditionally secure one-way quantum key distribution using decoy pulses.  _Appl. Phys. Lett._ , 90(1), 011118.  arXiv:quant-ph/0610015
  * Yuen (2013) Yuen, H.P. 2013 Essential elements lacking in security proofs for quantum key distribution.  In _Proceedings of SPIE- The International Society for Optical Engineering_.  arXiv:quant-ph/1310.0842
  * Zhang et al. (2009) Zhang, Z., Anantharam, V., Wainwright, M.J., & Nikolic, B. 2009 A 47 Gb/s LDPC decoder with improved low error rate performance.  In _2009 Symposium on VLSI Circuits_ , IEEE, 286–287.
  * Zhang et al. (2012) Zhang, H.F., Wang, J., Cui, K., Luo, C.L., Lin, S.Z., Zhou, L., Liang, H., Chen, T.-Y., Chen, K., & Pan, J.W. 2012 A real-time QKD system based on FPGA.  _J. Lightwave Technol._ , 30(20), 3226–3234.  arXiv:quant-ph/1301.2383
  * Zhang et al. (2014) Zhang, C.-M., Li M., Huang J.-Z., Treeviriyanupab, P., Li, H.-W., Li, F.-Y., Wang, C., Yin, Z.-Q., Chen, W., Sripimanwat, K., & Han, Z.-F., 2013 Fast implementation of length-adaptive privacy amplification in quantum key distribution.  _Chinese Phys. B_ , 23(9), 090310.
  * Zhao et al. (2008) Zhao, Y., Fung, C.H.F., Qi, B., Chen, C., & Lo, H.K. 2008 Quantum hacking: Experimental demonstration of time-shift attack against practical quantum-key-distribution systems.  _Phys. Rev. A_ , 78(4), 042333.  arXiv:0704.3253
  * Zhu et al. (2011) Zhu, C.-H., Pei, C.-X., Quan, D.-X., Gao, J.-L., Chen, N., Yi, Y.-H., 2010 A new quantum key distribution scheme based on frequency and time coding.  _Chinese Phys. Lett._ , 27(9), 090301.  arXiv:1003.2472


# PP-MeT: a Real-world Personalized Prompt based Meeting Transcription System

###### Abstract

Speaker-attributed automatic speech recognition (SA-ASR) improves the accuracy
and applicability of multi-speaker ASR systems in real-world scenarios by
assigning speaker labels to transcribed texts. However, SA-ASR poses unique
challenges due to factors such as speaker overlap, speaker variability,
background noise, and reverberation. In this study, we propose PP-MeT system,
a real-world personalized prompt based meeting transcription system, which
consists of a clustering system, target-speaker voice activity detection (TS-
VAD), and TS-ASR. Specifically, we utilize target-speaker embedding as a
prompt in TS-VAD and TS-ASR modules in our proposed system. In constrast with
previous system, we fully leverage pre-trained models for system
initialization, thereby bestowing our approach with heightened
generalizability and precision. Experiments on M2MeT2.0 Challenge dataset show
that our system achieves a cp-CER of 11.27% on the test set, ranking first in
both fixed and open training conditions.

Index Terms—  SA-ASR, TS-VAD, TS-ASR, personalized prompt, M2MeT2.0 Challenge

##  1 Introduction

The rapid advancements in deep learning have led to remarkable strides in
automatic speech recognition (ASR), substantially enhancing its overall
performance. Despite these achievements, ASR systems continue to face
challenges in real-world far-field scenarios, such as meetings or home
parties, where background noise, unavoidable reverberation, and overlapping
speech from multiple speakers can significantly degrade their performance. In
order to develop a robust ASR system in such challenging acoustic
environments, numerous research studies have concentrated on multi-channel
multi-party speech recognition and diarization within dinner party scenarios
[1, 2].

The objective of the M2MeT2.0 challenge [3, 4] is to address the ASR task in
multi-party meetings, which involves providing precise transcriptions and
identifying the corresponding speakers. To advance the practical application
of current multi-speaker speech recognition systems, the M2MET 2.0 Challenge
evaluates the task of Speaker-attributed ASR (SA-ASR). Additionally, the
challenge includes two sub-tracks: fixed training condition track and open
training condition track. Speaker-attributed ASR (SA-ASR) poses several
challenges due to the complexity of accurately attributing speech to specific
speakers. The SA-ASR task improves the accuracy and applicability of multi-
speaker ASR systems in real-world scenarios by assigning speaker labels to
transcribed texts. Unlike traditional ASR systems that transcribe speech
without considering speaker identities, SA-ASR goes a step further by
associating each recognized word or phrase with the corresponding speaker.

SA-ASR faces unique challenges due to factors like speaker overlap, speaker
variability, background noise, and reverberation. Overcoming these challenges
involves developing advanced algorithms and techniques for speaker
diarization, speech separation, and speaker recognition to accurately
attribute spoken words to their respective speakers. The development of SA-ASR
systems has the potential to improve the performance and usability of speech
recognition in scenarios where multiple speakers are present, enabling
applications that require speaker-specific information and analysis.

In this study, we present the PP-MeT system, a personalized-prompt based
meeting transcription system designed to address the ASR task in multi-party
meetings. Our approach comprises three essential components: a clustering
system, target-speaker voice activity detection (TS-VAD), and target-speaker
ASR (TS-ASR). To enhance the system’s performance and applicability, we
integrate target-speaker embeddings as prompts within the TS-VAD and TS-ASR
modules. Leveraging pre-trained models during system initialization further
empowers our approach, granting it superior generalizability and precision. In
experiments conducted on the M2MeT2.0 dataset, our integrated PP-MeT system
achieves a concatenated minimum permutation character error rate (cp-CER) of
only 11.27% on the test set, achieving the top position in both fixed and open
training conditions. We also release our inference system with pre-trained
models at website111https://github.com/XimalayaEverestIntelligentLab/M2MET2.0.

The rest of this paper is organized as follows. In Section 2, we detail the
architecture of the PP-MeT system. Datasets and experimental setup are
described in Section 3. Section 4 presents the experimental results of
M2MeT2.0 Challenge test set and our ablation study. Finally, we conclude in
Section 5.

##  2 Proposed System Description

The overview of our proposed PP-MeT system for the M2MeT2.0 Challenge is shown
in Figure 1.

Fig. 1: The overview of our proposed PP-MeT system.

###  2.1 Speaker Embedding System

As M2MeT2.0 encourage the participants to use pre-trained models, we use two
pre-trained
models222https://github.com/wenet-e2e/wespeaker/blob/master/docs/pretrained.md
from Wespeaker toolkit [5, 6]. One is Resnet34 from the CN-Celeb example, and
another is ResNet34-LM, which is obtained by further training ResNet34 with a
large-margin technique. We also train a ResNet34 model with Speechbrain
toolkit333https://github.com/speechbrain/speechbrain/tree/develop to introduce
diversity to our speaker embedding model. We will refer to these three speaker
embedding models as SV-1/2/3 and the corresponding personalized prompt as
Prompt-1/2/3 for simplicity.

###  2.2 Clustering System

Before proceeding to TS-VAD and TS-ASR systems, we need to estimate the number
of speakers and initialize personalized prompts using clustering algorithm.

First, we extract voice speech segments based on VAD results for each session.
Then we split each segment into subsegments using a fixed 3s window size and
1.5s window-shift. After that, we use speaker embedding model to extract
embedding for each subsegment. Finally, we feed the L2-normalized embedding
into the clustering algorithm and obtain the speaker number for each session,
as well as the label for each subsegment.

We use DOVER-lap toolkit [7]444https://github.com/desh2608/dover-lap to merge
clustering results from different channel and speaker embedding models. We
compare Auto-tuning Spectral Clustering with Normalized Eigen Gap [8](NME-SC)
with Agglomerative Hierarchical Clustering(AHC) algorithm. As NME-SC
outperforms AHC by a large margin, we use NME-SC algorithm results to
initialize personalized prompts.

After obtaining the clustering system result, for each speaker, we extract
speech that contains only the targeted speaker as personalized speech. Then we
repeat the speaker embedding extraction steps over the personalized speech and
use the mean-pooled L2-normalized speaker embedding as the personalized
prompt.

###  2.3 TS-VAD System

As the clustering system can not handle overlap speech, it results in a high
miss error in multi-party meeting scenarios. To further reduce DER, we use TS-
VAD system to give a more accurate estimate of speaker labels.

We use ResNet34 model as backbone for our TS-VAD system, which is the same as
that of speaker embedding model. First, we extract the pooling layer input as
frame-level speaker embedding. Then we do a stats-pooling with 3-second stride
to extract the frame level mean and std feature, and concatenate it with the
original frame-level speaker embedding. We do mean-pooling and attention-
pooling for frame-level speaker embedding and personalized prompts,
respectively. After that, we use a conformer decoder layer to explore the
relationship between the frame-level speaker embedding and personalized
prompt. We feed the frame-level speaker embedding feature as conformer decoder
input, and each personalized prompt as decoder memory. Finally, we concatenate
the conformer decoder layer output and use a BiLSTM layer to explore the
relationship among each speaker. The BiLSTM output is fed into a fully-
connect(FC) layer with a sigmoid activation function to generate the final TS-
VAD probability[9, 10]. The detailed TS-VAD model structure is shown in Figure
2.

Fig. 2: TS-VAD model structure

###  2.4 TS-ASR system

Far-Field ASR poses a greater challenge compared to ASR of speech captured by
a close-proximity microphone due to the degraded quality of the signal. To
address this, we endeavor to engage in speech enhancement. In our practical
pursuit, there exist two pivotal components. Firstly, we employ a
sophisticated dereverberation method based on weighted prediction error (WPE)
[11] to mitigate the effects of late reverberation. In the challenge, we
utilize an accelerated GPU-version of WPE, incorporating the following
parameters: taps=12, delay=2, iterations=3. Secondly, in order to further
attenuate late reverberation and minimize noise interference, the weighted
delay-and-sum acoustic beamforming (BeamformIt) method [12] is employed.

As M2MeT2.0 requires participants to give transcription for each speaker, we
upgrade the traditional ASR model into TS-ASR system with personalized prompt
module, which enables it to yield different transcription given different
personalized prompt [13, 14]. We feed the personalized prompt into a FC layer,
and do Hadamard product with the output from the first layer of both asr
encoder and decoder. As our TS-ASR model makes little modification to the
traditional ASR model, we can easily adapt a pre-trained ASR model into a TS-
ASR model. We use Unfied-Conformer [15] model pretrained on
wenetspeech555https://github.com/wenet-e2e/wenet/blob/main/docs/pretrained_models.md
from [16] as the TS-ASR model backbone. The detailed TS-ASR model structure is
shown in Figure 3.

Fig. 3: TS-ASR model structure

##  3 Experimental Setup

###  3.1 Datasets

The original M2MeT1.0 dataset [3] contains 118.75 hours of speech data in
total. The dataset is divided into 104.75 hours for training, 4 hours for
development (denoted as Dev 1.0), and 10 hours as test set (denoted as Test
1.0) for scoring and ranking in M2MeT1.0 Challenge. Test 1.0 is used as
development set in M2MeT2.0 Challenge. M2MeT2.0 uses a new 10 hours dataset
(denote as Test 2.0) as test set. AISHELL4 [17] is a real-recorded Mandarin
speech dataset collected by 8-channel circular microphone array for speech
processing in a conference scenario. This dataset consists of 211 recorded
meeting sessions, each containing 4 to 8 speakers, with a total length of 120
hours, aiming to bridge the advanced research on multi-speaker processing and
the practical application scenarios. CN-Celeb [18] is a large-scale speaker
recognition dataset collected ‘in the wild’. This dataset contains more than
130, 000 utterances from 1, 000 Chinese celebrities, and covers 11 different
genres in the real world.

Both M2MeT and AISHELL4 datasets are far-field multi-channel datasets, while
the CN-Celeb dataset is a near-field dataset. Figure 4 shows the data
preparation. By Oracle VAD, the non-overlap speech of each speaker is obtained
from both near-field and far-field data. Then the personalized prompt is
extracted. The M2MeT dataset is processed according to the given prior
information into continuous voice speech. All far-field multi-channel datasets
are pre-processed to generate enhanced data by WPE and BF. The original far-
field 8-channel data and the enhanced data compose the speech of each speaker,
which is used in the next training process.

Fig. 4: Data preparation before training.

The data flow of each training process is shown in Figure 5. The near-field
data is processed into 3-second uniform segments and used in speaker embedding
training. In TS-VAD model training, the continuous voice speech and non-
overlap speech with online augmentation are processed into 16-second uniform
segments, and the target-speaker embedding is used as a prompt. Moreover, the
speech segment of each speaker and the personalized prompt are used in TS-ASR
model training.

Fig. 5: Data flow in each training process.

###  3.2 System Setup

For all systems, we use 80-dimension log-mel filter bank (Fbank) feature as
input. The Fbank feature is extracted using a 25ms window length and 10ms
window shift.

####  3.2.1 Speaker embedding system

we use CN-Celeb data [18] to train our speaker embedding model and split each
utterance into 3s uniform length segments. When iterating over all segments,
we introduce diversity by randomly offsetting the start frame of the segments
from -1.5s to 1.5s. All these three speaker embedding models are trained using
AAM softmax loss [19] and generate 256 dimension speaker embedding as output.
We use a cyclical learning rate policy to dynamically adjust the lr for 16
epochs.

####  3.2.2 TS-VAD system

we use M2MeT2.0 training data and Aishell-4 data for training. For each
session, first, we extract and combine all voiced speech as our real training
data. Then, for each speaker, we extract and combine speech that contains only
the target speaker as personalized speech. Finally, we initialize Prompt-1/2/3
using personalized speech. If the number of speaker is less than 4, we pad
Prompt-1/2/3 using zero vectors.

During training, we split the real training data into 16s segments and iterate
over each segment. We also do an online data simulation by choosing
personalized speech from random speakers to fill up the voiced region of real
data[20]. It is important that the randomly chosen speakers are from the same
session, in case the model learns background noise feature of each session,
rather than the essential difference of each speaker.

We train three TS-VAD models based on SV-1/2/3 and Prompt-1/2/3. For all TS-
VAD models, we use 2 layers, 256-dimension input, 512-dimension hidden
dimension, and 8 heads for the conformer decoder setup. We use 2 layers, 1024
dimension input, and 512 hidden dimensions for BiLSTM setup.

TS-VAD training consists three key stages. In stage 1, we copy the pre-trained
speaker embedding parameter into the TS-VAD model, freeze the backbone part
and train the model using real and simulated data until convergence with 1e-3
lr. In stage 2, we train the whole model using real and simulated data until
convergence with 1e-4 lr. In stage 3, we finetune the whole model only using
real data with 1e-5 lr. We choose the model with the lowest DER on Test 1.0
for decoding.

During TS-VAD decoding, we initialize Prompt-1/2/3 from clustering system. We
can iterate over the TS-VAD system by re-initialize Prompt-1/2/3 using TS-VAD
system outputs.

####  3.2.3 TS-ASR system

We use the WeNet toolkit and its pre-trained Unified-Conformer model on
WeNetSpeech as backbone. Since M2MeT2.0 and Aishell-4 training data comprise
multiple channels, on one hand, we directly feed the model with raw mean-
pooled data, on the other hand, we feed the model with enhanced single-channel
data. Additionally, we incorporate speed augmentation techniques during the
training process. It is imperative to note that when the audio speed is
altered, the corresponding personalized prompt for that particular speed
variation should be rendered.

We also train three TS-ASR systems based on Prompt-1/2/3. For all TS-ASR
models, we use 12-layer conformer encoder with 512 dimension output, 2048
dimension linear units, and 8 attention heads. We use a 3-layer bi-transformer
decoder with 2048 dimension linear units and 8 attention heads. For the
personalized prompt module, we feed the 256 dimension personalized prompt into
a FC layer, project it into a 512 dimension vector and do a Hadamard product
with the first layer output of both encoder and decoder.

TS-ASR training also consists three key stages. In stage 1, we freeze the
Unified-Conformer backbone, and only train the personalized prompt module
using raw data and enhanced data. In stage 2, we train the whole model with
1e-4 lr. In stage 3, we finetune the whole model with 1e-5 lr using enhanced
data.

##  4 Experimental Results

###  4.1 Results on M2MeT2.0 Challenge

M2MeT2.0 challenge uses concatenated minimum permutation character error rate
(cp-CER) as the evaluation metric. It computes the minimum CER given all
speaker permutations, which requires the system to give the correct
transcription and speaker label. The calculation of cp-CER is divided into
three steps. First, recognition results and reference transcriptions belonging
to the same speaker are concatenated on the timeline in a session. Second, the
character error rate (CER) of all permutations of speakers is calculated.
Finally, the lowest CER is selected as the cp-CER.

Table 1 presents the cp-CER results of the official baseline and each
competition system. Our system achieves 15.05%, 16.84%, and 11.27% cp-CER on
Dev 1.0, Test 1.0, and Test 2.0, respectively. Notice that cp-CER on Dev 1.0
and Test 1.0 is achieved using oracle diarization result. We can observe that
our PP-MeT model gives better results over the official baseline and achieve
up to 30.28% absolute cp-CER improvement due to the enhanced dataset and
advanced model architectures. achieving first place in the challenge.

System | Dev 1.0 | Test 1.0 | Test 2.0
---|---|---|---
PP-MeT (Rank 1st) | 15.05 | 16.84 | 11.27
Rank 2nd Team | – | – | 18.64
Rank 3rd Team | – | – | 22.83
Rank 4th Team | – | – | 23.51
Rank 5th Team | – | – | 24.82
Official Baseline | 47.4 | 52.57 | 41.55
Table 1: The cp-CER (%)$\downarrow$ results of each competition system on the
M2MeT Dev 1.0, Test 1.0, and Test 2.0.

###  4.2 Ablation Study

We conduct a detailed ablation study to better understand the contribution of
cp-CER from each system, and the significance of pre-trained models.

Clustering Method | SV Model | Channel 1-8 DER(%) $\downarrow$ | DER (%)_Channel $\downarrow$ | DER (%)_Model $\downarrow$
---|---|---|---|---
SC | SV-1 | 16.87 | 16.21 | 16.89 | 17.49 | 18.61 | 18.00 | 16.09 | 18.85 | 16.40 | 15.22
SV-2 | 15.96 | 16.22 | 16.41 | 16.39 | 16.86 | 16.57 | 16.55 | 17.99 | 15.22
SV-3 | 17.26 | 16.18 | 17.16 | 16.97 | 17.07 | 16.55 | 16.52 | 17.11 | 15.75
AHC | SV-1 | 28.22 | 27.11 | 26.92 | 25.79 | 25.90 | 24.52 | 26.61 | 26.25 | 22.95 | 22.43
SV-2 | 26.48 | 23.85 | 26.13 | 26.70 | 25.28 | 25.61 | 24.58 | 26.65 | 22.43
SV-3 | 25.90 | 26.74 | 29.33 | 27.19 | 27.51 | 27.02 | 26.09 | 25.87 | 22.94
Table 2: DER Results for each clustering system on Test 1.0

####  4.2.1 Clustering System

As clustering system gives the estimate of speaker number and rough speaker
label, its performance determines the superior limit of the whole PP-MeT
system. In Table 2, we study the impact of different speaker embedding models
and clustering algorithms in clustering systems.

SV-1/2/3 achieves 7.13%, 6.49%, and 7.06% on CN-Celeb dev trials,
respectively. The threshold for AHC clustering is tuned on Dev 1.0. Results
show that given each model and channel, NME-SC outperforms AHC significantly.
DOVER-lap makes the clustering result more stable by leveraging clustering
results from different channels and models.

As the accuracy of speaker embedding directly affects the quality of speaker
embedding, DER relates to speaker embedding performance evidently. The lowest
DER is achieved by SV-2, which also achieves the lowest EER on CN-Celeb
trials.

####  4.2.2 TS-VAD system

In Table 3, we study the impact of pre-trained speaker embedding model and
different model architectures in TS-VAD system.

Results show that pre-trained model contributed heavily to the performance of
TS-VAD system. If TS-VAD model backbone parameter is randomly initialized, it
only achieves 13.28% DER on Test 1.0, which is only slightly better than that
of clustering system. Also, TS-VAD model backbone should match that of the
personalized prompt. If we initialize the TS-VAD model backbone parameter
using pre-trained ECAPA-TDNN speaker embedding model and train with Prompt-1.
It achieves 7.68% DER, which is much worse than its counterpart using matched
speaker embedding model and prompt. The above results demonstrate the
importance of pre-trained models in TS-VAD system, and using matched speaker
embedding model for initialization and personalized prompt makes it easier to
explore the relationship between frame-level speaker embedding and
personalized prompt.

We can also observe that the DER drops moderately if we iterate the TS-VAD
system by refining Prompt-1/2/3 using TS-VAD system output.

Initialization Model Parameter | Personalized Prompt | DER (%)_Iter0 $\downarrow$ | DER (%)_Model $\downarrow$ | DER (%)_Iter1 $\downarrow$ | DER (%)_Model $\downarrow$
---|---|---|---|---|---
SV-1 | Prompt-1 | 5.22 | 3.19 | 4.87 | 2.99
SV-2 | Prompt-2 | 4.52 | 4.25
SV-3 | Prompt-3 | 4.02 | – | 3.64
Random | Prompt-1 | 13.28 | –
ECAPA | Prompt-1 | 7.68 | –
Table 3: DER Results for each TS-VAD model on Test 1.0

####  4.2.3 TS-ASR System

In Table 4, we study the impact of pre-trained models and personalized prompts
in TS-ASR system.

First, we try to finetune the pre-trained unified-conformer ASR model directly
without any structure modification. Results show that the pre-trained ASR
model achieves 32.63% and 35.89% cp-CER on Dev 1.0 and Test 1.0. After
finetuning the model on M2MET2.0 and Aishell-4 data, the cp-CER drops to
22.55% and 26.43%, respectively. However, the cp-CER improvement is largely
due to the model performance on nonoverlap speech. It fails to decrease
further because the traditional ASR model cannot handle overlap speech.

Then, we try to train the TS-ASR model from scratch with Prompt-1. However,
the TS-ASR model with a unified-conformer backbone fails to converge. This
demonstrates the necessity of pre-trained ASR model backbone in our TS-ASR
system.

Finally, we train three TS-ASR models based on the Prompt-1/2/3. cp-CER on TS-
ASR model with pretrained ASR model backbone and Prompt-1/2/3 drops
dramatically both on Dev 1.0 and Test 1.0. The result shows that pre-trained
ASR model with Prompt-2 achieves the lowest cp-CER, which means that the
performance of pre-trained speaker embedding model also affects the
performance of TS-ASR on overlapped speeches.

We also try to finetune the TS-ASR model further using LF-MMI with k2
toolkit666https://github.com/k2-fsa/k2, and introducing LM information by
decoding with HLG. However, the cp-CER fails to drop on both Dev 1.0 and Test
1.0. This is due to the fact that in multi-party meeting scenario, the
transcription from each session is highly irrelevant. External LM information
can not help to decrease cp-CER.

In Table 4, Test 1.0 cp-CER is calculated using segments and prompts from TS-
VAD system. The gap between cp-CER of Test 1.0 and Test 1.0 means the
degradation introduced by TS-VAD system, which is approximately 2%. We obtain
final results by leveraging each system results using SCTK rover
toolkit777https://github.com/usnistgov/SCTK.

Personalized Prompt | Dev 1.0 | Test 1.0 | Test 1.0
---|---|---|---
– | 22.55 | 26.43 | –
Prompt-1 | 15.35 | 17.20 | 19.45
Prompt-2 | 15.13 | 17.08 | 19.06
Prompt-3 | 15.28 | 17.16 | 19.06
Rover | 15.05 | 16.84 | 18.92
Table 4: cp-CER (%) results for each TS-ASR model on Dev 1.0, Test 1.0 and
Test 1.0.

##  5 Conclusion

In this paper, we present our PP-MET system for the Multi-channel Multi-party
Meeting Transcription Challenge 2.0 (M2MeT2.0) to address the ASR task in a
multi-party meeting scenario. Compared with the other conventional systems, we
incorporate target-speaker embedding as a personalized prompt in both TS-VAD
and TS-ASR stage. Moreover, to further enhance the system’s robustness and
reduce the training cost, pre-trained models are used in our system’s
initialization, enabling fast adaptation across all modules. Experimental
results shows proposed system outperforms conventional systems by a large
margin.

In future work, we plan to explore the potential of expanding personalized
prompts on the time axis. Additionally, we aim to enhance the TS-ASR model by
jointly training the speaker embedding module with the ASR backbone, further
improving its performance.

## References

  * [1] Jon Barker, Shinji Watanabe, Emmanuel Vincent, and Jan Trmal,  “The fifth’chime’speech separation and recognition challenge: dataset, task and baselines,”  arXiv preprint arXiv:1803.10609, 2018.
  * [2] Shinji Watanabe, Michael Mandel, Jon Barker, Emmanuel Vincent, Ashish Arora, Xuankai Chang, Sanjeev Khudanpur, Vimal Manohar, Daniel Povey, Desh Raj, et al.,  “Chime-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings,”  arXiv preprint arXiv:2004.09249, 2020.
  * [3] Fan Yu, Shiliang Zhang, Yihui Fu, Lei Xie, Siqi Zheng, Zhihao Du, Weilong Huang, Pengcheng Guo, Zhijie Yan, Bin Ma, et al.,  “M2met: The icassp 2022 multi-channel multi-party meeting transcription challenge,”  in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 6167–6171.
  * [4] Yuhao Liang, Mohan Shi, Fan Yu, Yangze Li, Shiliang Zhang, Zhihao Du, Qian Chen, Lei Xie, Yanmin Qian, Jian Wu, Zhuo Chen, Kong Aik Lee, Zhijie Yan, and Hui Bu,  “The second multi-channel multi-party meeting transcription challenge (m2met) 2.0): A benchmark for speaker-attributed asr,”  in arxiv preprint arxiv: 2309.13573, 2023.
  * [5] Hongji Wang, Chengdong Liang, Shuai Wang, Zhengyang Chen, Binbin Zhang, Xu Xiang, Yanlei Deng, and Yanmin Qian,  “Wespeaker: A research and production oriented speaker embedding learning toolkit,”  in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5.
  * [6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Deep residual learning for image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [7] Desh Raj, Leibny Paola Garcia-Perera, Zili Huang, Shinji Watanabe, Daniel Povey, Andreas Stolcke, and Sanjeev Khudanpur,  “Dover-lap: A method for combining overlap-aware diarization outputs,”  in 2021 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2021, pp. 881–888.
  * [8] Tae Jin Park, Kyu J Han, Manoj Kumar, and Shrikanth Narayanan,  “Auto-tuning spectral clustering for speaker diarization using normalized maximum eigengap,”  IEEE Signal Processing Letters, vol. 27, pp. 381–385, 2019.
  * [9] Weiqing Wang, Xiaoyi Qin, and Ming Li,  “Cross-channel attention-based target speaker voice activity detection: Experimental results for the m2met challenge,”  in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 9171–9175.
  * [10] Ivan Medennikov, Maxim Korenevsky, Tatiana Prisyach, Yuri Khokhlov, Mariya Korenevskaya, Ivan Sorokin, Tatiana Timofeeva, Anton Mitrofanov, Andrei Andrusenko, Ivan Podluzhny, et al.,  “Target-speaker voice activity detection: a novel approach for multi-speaker diarization in a dinner party scenario,”  arXiv preprint arXiv:2005.07272, 2020.
  * [11] Tomohiro Nakatani, Takuya Yoshioka, Keisuke Kinoshita, Masato Miyoshi, and Biing-Hwang Juang,  “Speech dereverberation based on variance-normalized delayed linear prediction,”  IEEE Transactions on Audio, Speech, and Language Processing, vol. 18, no. 7, pp. 1717–1731, 2010.
  * [12] Xavier Anguera, Chuck Wooters, and Javier Hernando,  “Acoustic beamforming for speaker diarization of meetings,”  IEEE Transactions on Audio, Speech, and Language Processing, vol. 15, no. 7, pp. 2011–2022, 2007.
  * [13] Takafumi Moriya, Hiroshi Sato, Tsubasa Ochiai, Marc Delcroix, and Takahiro Shinozaki,  “Streaming target-speaker asr with neural transducer,”  Proc. Interspeech 2022, pp. 2673–2677, 2022.
  * [14] Hiroshi Sato, Tsubasa Ochiai, Marc Delcroix, Keisuke Kinoshita, Takafumi Moriya, and Naoyuki Kamo,  “Should we always separate?: Switching between enhanced and observed signals for overlapping speech recognition,”  arXiv preprint arXiv:2106.00949, 2021.
  * [15] Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al.,  “Conformer: Convolution-augmented transformer for speech recognition,”  arXiv preprint arXiv:2005.08100, 2020.
  * [16] Binbin Zhang, Di Wu, Zhendong Peng, Xingchen Song, Zhuoyuan Yao, Hang Lv, Lei Xie, Chao Yang, Fuping Pan, and Jianwei Niu,  “Wenet 2.0: More productive end-to-end speech recognition toolkit,”  arXiv preprint arXiv:2203.15455, 2022.
  * [17] Yihui Fu, Luyao Cheng, Shubo Lv, Yukai Jv, Yuxiang Kong, Zhuo Chen, Yanxin Hu, Lei Xie, Jian Wu, Hui Bu, et al.,  “Aishell-4: An open source dataset for speech enhancement, separation, recognition and speaker diarization in conference scenario,”  arXiv preprint arXiv:2104.03603, 2021.
  * [18] Yue Fan, JW Kang, LT Li, KC Li, HL Chen, ST Cheng, PY Zhang, ZY Zhou, YQ Cai, and Dong Wang,  “Cn-celeb: a challenging chinese speaker recognition dataset,”  in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 7604–7608.
  * [19] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou,  “Arcface: Additive angular margin loss for deep face recognition,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690–4699.
  * [20] Weiqing Wang, Danwei Cai, Qingjian Lin, Lin Yang, Junjie Wang, Jin Wang, and Ming Li,  “The dku-dukeece-lenovo system for the diarization task of the 2021 voxceleb speaker recognition challenge,”  arXiv preprint arXiv:2109.02002, 2021.