# A Soft Robotic Gripper with Active Palm for In-Hand Object Reorientation

Thomas Mack, Ketao Zhang, Kaspar Althoefer, Senior Member, IEEE This work was
supported in part by an Alan Turing Institute funded project on Intuitive
human-robot interaction in work environments.The first author is funded by an
iCASE EPSRC PhD studentship.For the purpose of open access, the authors have
applied a Creative Commons Attribution (CC BY) license to any Accepted
Manuscript version arising.Authors are with the Centre for Advanced Robotics @
Queen Mary, School of Engineering and Materials Science, Queen Mary University
of London, United Kingdom.

###### Abstract

The human hand has an inherent ability to manipulate and re-orientate objects
without external assistance. As a consequence, we are able to operate tools
and perform an array of actions using just one hand, without having to
continuously re-grasp objects. Emulating this functionality in robotic end-
effectors remains a key area of study with efforts being made to create
advanced control systems that could be used to operate complex manipulators.
In this paper, a three fingered soft gripper with an active rotary palm is
presented as a simpler, alternative method of performing in-hand rotations.
The gripper, complete with its pneumatic suction cup to prevent object
slippage, was tested and found to be able to effectively grasp and rotate a
variety of objects both quickly and precisely.

##  I Introduction

The act of repositioning objects within our grasp comes so naturally to humans
that we often barely notice ourselves doing it. When lifting a pen we usually
have to reorient it before we start writing - a precise action that most
humans are able to execute in half a second. Some are even able to deftly
twirl pens and similar objects between their fingers. Removing the pen lid is
more challenging, but most people are able to accomplish this task single-
handedly using three fingers to grip the pen and the thumb and index finger to
lift the lid off. While writing, the fingers accurately move the nib in a
plane to trace out letters legible enough to be read by others.

Figure 1: The gripper using the soft, 3D printed fingers

Replicating this level of dexterity, either fully or partially, in a robotic
manipulator is clearly a huge challenge for the industry, on account of the
myriad of applications it would proffer. Assembly lines and packing robots
would benefit enormously from this technology as items with arbitrary shapes
and weights could be easily reoriented by the gripper. Potentially it could
even remove the need for specialised manipulators for different stages of
production.

Most environments and implements encountered in everyday life are designed for
humans use. Underlying any design is an inherent assumption that the humans
working in those environments or using those implements will have the level of
dexterity that comes so naturally. A general purpose collaborative robot
would, of course, benefit from human-like dexterity in most basic tasks such
as opening doors, or reorienting objects to make them easier for a human to
receive. Advanced generalised grippers would be able to handle tools, enabling
them perform the same tasks as humans without the need of specialised end
effectors.

During interactions with a robot, a human would find it most useful if
objects, such as tools, were presented to them in a convenient manner. A tool,
when picked up by a robot gripper with the intention to pass this on to a
human, would rarely be in an orientation that can be received easily and
safely.

Here, a robotic system capable of rearranging the grasped tool (e.g., a
screwdriver) such that when the handover to the human occurs, the human can
receive it handle first, to conduct the task (e.g., fastening screws). It
would immediately, make a difference in many industrial settings such as
manufacturing, assembly, maintenance and inspection.

The development of robotic in-hand manipulation would also enhance prosthetics
and other assistive technologies. Current commercial robotic prosthetic hands
are very expensive and rely mostly on an external interface setting different
poses that can be proportionally controlled or toggled [3] using
electromyography (EMG) [14]. However, the muscles required for EMG are not
always intact. Even if finger control via EMG is possible, it may not offer
the precision required to manipulate objects within the hand. A method of
automatically grasping everyday objects would be a great enhancement in terms
of quality of life, eliminating the need to changing the grasp pose for each
different object. Indeed the ability to reorient an object within the grasp
using a simple control scheme would be the next step and an ideal solution if
only a limited number of EMG inputs were available.

Many different approaches to creating grippers capable of in-hand manipulation
have been adopted in the past. As the most prominent example of a high
precision, fast manipulator is the human hand, many anthropomorphic designs
have been created that try to closely mimic it [9][12][15] or that are at
least inspired by it. However, their inherent complexity makes them difficult
to build. A human hand has over 23 degrees of freedom [5] and their positions
and torque can be sensed with inbuilt proprioception [6]. Tendons are often
used for actuation in robotic hands as fitting so many actuators and force and
position sensors in to the fingers is almost impossible when working at a
scale similar to a human hand at least with today’s technology; using tendons,
sensors can be situated remotely, e.g., in the arm that the gripper is mounted
on.

Human skin is a soft, thin, waterproof covering filled with biological sensors
that provide us a wealth of information when grasping. Robotic skin is a field
of study in its own right [7] and has many applications beyond in-hand
manipulation. Our skin has directional force sensors that let us gauge not
just the weight of an object but the torque that it exerts on our hands, which
we can use to infer its centre of mass - a function which has been implemented
in some grippers [16]. These force sensors can also detect texture when moved
over a material. Combined with the ability to sense temperature, a human can
make a reasonable estimate as to the material properties through touch alone.
This would all be useful information to a robotic gripper, potentially
allowing it to adjust its grasp for individual objects without visual
information.

Controlling such a complex system to perform in-hand manipulation presents its
own set of challenges. While the human hand can be used as an example for
manipulation strategies, it is computationally expensive to model an object
with five distinct points of contact. Most finger based in-hand manipulation
is performed with the fingertips grasping objects from the side, incurring a
risk of slipping which would need to be mitigated by the robot. To do this it
would need to have knowledge of the object’s shape and properties to
successfully plan movements. This information may need to be gathered using
computer vision or through contact sensors on the robot which adds another
layer of complexity.

Non-anthropomorphic systems have also been explored as simpler solutions to
in-hand manipulation [10][17][18]. They will often use fewer fingers or non-
standard methods to manipulate objects. Instead of trying to achieve a wide
variety of manipulation tasks, these grippers may be designed to effectively
perform just one or a few tasks. Using fewer fingers makes the robot much
easier and cheaper to build and reduces the computational load when modelling
and planning movements. However, using the minimum number of fingers for
manipulation means there are no extra redundant fingers that can be used to
adapt the grasping style mid reorientation. As few as two fingers can achieve
limited human-like in-hand rotation and translation [17].

The recent advance in soft robotics has prompted its incorporation into in-
hand manipulation. A soft fingertip will deform during a grasp, distributing
the force over a wider area which can reduce the risk of damaging fragile
objects. Materials like silicone which are often used in soft robotics also
increase friction, reducing the risk of slipping. Whole fingers can be made of
soft materials with chambers that can be pressurised to cause continuous
bending throughout the structure [2][4][11], similar to a continuum robot
[13]. These compliant fingers can be designed to grasp a wide variety of
shapes as they deform and adapt if too much air pressure is applied. This
means soft grippers can often grasp objects without needing their exact
dimensions - a key advantage. As the applied force is distributed, fragile
objects will also tolerate more excessive force than they would with a rigid
gripper. However, deformable materials and the use of pneumatics can introduce
non-linearities and make soft actuators more difficult to control, sometimes
requiring the use of machine learning [8]. They can also be prone to leaks and
can be easily damaged.

Rigid solutions [10][18] are still being explored as the actuators and
construction methods are well established. The kinematics are comparatively
simple and can allow for far greater precision than an entirely soft robot.
However, the rigidity requires precise knowledge of the manipulated object and
the system must control the trajectories of all the fingers to ensure stable
movement without excessive force.

##  II Conceptual Design of the 3-Fingered Soft Gripper

Some of the more successful grippers utilise a palm built into them for
stability. [2] employed a palm which acted as a flat surface on which objects
were rotated and translated. [11] used an in-built suction cup to support
objects while the fingers release and re-position them. In both cases, as with
many others, these palms are in an effective place to enable in-hand rotation
as they lie in the axis of rotation. They can also have adhesive materials or
devices attached to stabilise the objects they handle.

Here, a gripper is proposed that utilises a rotating palm with an integrated
suction cup for in-hand rotation. As with other solutions, the palm is located
centrally, and is surrounded by fingers used for grasping.

The normal use case for this gripper would not involve the fingers during
rotation. They would be used to initially grasp the object from above. The
entire gripper would then rotate 180∘ to place the palm below the object. The
object would then be released and dropped on to the palm so that it could then
be rotated to the desired position. The suction cup would help secure the
object to the palm during rotation. The object would then be re-grasped for
placing back down onto a surface (Figure 2).

(a)

(b)

(c)

Figure 2: The process by which objects are rotated. The chassis is shown in
grey, the fingers in green and the palm in red: a) The fingers grasp and lift
the object; b) The gripper and object are turned 180∘ to face up; c) The
object is dropped on to the palm for rotation.

Using a motor for rotation would allow an object to be turned quickly and
precisely. It would also be largely unaffected by interactions between fingers
and the object’s geometry during rotation in the way that finger based
manipulators are. Another advantage over finger based rotation is that no re-
grasping needs to take place to achieve large changes in angle, allowing the
target orientation to be reached in a single smooth twist.

Because the fingers are not the source of rotation they only need to be able
to reliably grasp objects. Soft pneumatic fingers are ideal as they can
achieve effective grasping using a minimal control scheme [4]. They could
simply be designed to open and close, utilising their compliance rather than
any specific knowledge of the objects’ dimensions. This would however limit
the freedom of movement of the fingers, preventing the gripper from
translating objects.

The proposed in-hand rotation could still be used in many applications such as
in human-robot interaction. Objects could be re-positioned in the manipulator
to render them easier to receive by a human. The fingers would also be soft,
making any contact with a human safer. This kind of in-hand rotation could
also be very useful for delicate tasks like fruit picking or packing. The only
force being applied to the fruit would be from suction cup, and that would
only be a marginal force, just enough to prevent the fruit from falling off
the palm.

##  III Design and Prototyping

(a)

(b)

Figure 3: The CAD model shown of the gripper: a) Isometric view; b) Top view

The gripper uses a motorised platform for the palm with an integrated
pneumatic suction cup (Figure 4), attached directly at the base to an XL-320
servo motor by ROBOTIS. The suction cup is moulded from the softest variety of
Ecoflex silicone from SmoothOn in the shape of a half torus shell. It is
hollow to make it more deformable so as to form a better seal with non flat
objects. A channel that passes from the inside of the suction cup to below the
outside of the palm is designed house a silicone tube that flexes as the palm
twists (Figure 4(b)). This tube is fed through the chassis and used to
propagate the vacuum that is toggled with a valve.

(a)

(b)

Figure 4: From the top can be seen the soft silicon suction cup, the palm with
integrated tubing and a piece that attaches it to the servo motor: a)
Exploded; b) Exploded cross section.

The palm is then surrounded by three soft pneumatic fingers for grasping that
all flex simultaneously. They are mounted on the chassis and spread out at 25∘
to increase the size of objects that they are able to handle.

Different types of continuous pneumatic silicone fingers were designed and
evaluated to find an optimal version that would allow most objects to be
grasped. The soft nature of the fingers meant that they would naturally adapt
to different sizes and shapes, which then left maximum liftable weight as the
principal challenge.

Each finger was moulded from Dragonskin 30. A pneu-net type design inspired by
[2] and softroboticstoolkit.com [1] was first implemented, with a single set
of chambers that inflate to continuously flex the finger (Figure 7(a)). The
actuators were fabricated in a single step using a 3D printed, Polylactic Acid
(PLA) mould and a Polyvynil Acetate sacrificial insert. Because pneu-net style
actuators often flex to extreme angles and curl over 180∘, the fingertips were
made cylindrical. This was to encourage the fingertips to roll over objects as
the fingers curled, bringing it closer to the palm. A layer of fabric was
glued to the inside edge of the finger to prevent extension. However, when
grasping was attempted, they would often deform laterally and rotate past
objects, dropping them.

An attempt was then made to stiffen the finger laterally. The design was
modified to have a thicker base in which a polyethylene strip could be
embedded (Figure 7(b)). Both of these increased the stiffness of the finger,
and the polyethylene strip resisted lateral deformation.

(a)

(b)

(c)

(d)

Figure 5: The CAD designs of the fingers: a) the first set of moulded fingers
with the fingertips attached; b) a cross section of the first set of moulded
fingers; c) a cross section of the oval fingers; d) the cavity of the oval
fingers.

(a)

(b)

(c)

Figure 7: Each finger designed and moulded for the gripper: a) The fabric
reinforced finger; b) The plastic reinforcement can be seen in the second
finger when illuminated; c) The oval shaped finger.

Another pneumatic finger was designed with an oval cross section to encourage
bending over its shortest length. A semicircular cavity was placed in the
outer side of the finger. It was noticed that due to variations in the curing
conditions each finger had a slightly different response to pressure which was
rectified by having a separate SMC ITV2050 pressure regulator with its own
control curve.

All of the silicone fingers were attached to 3D printed PLA inlets using Sil-
Poxy with 4mm pipes glued in to connect them to the pressure regulators. The
inlets were also used to hold the fingers on to the chassis.

They were later replaced by a set of three 3D printed soft fingers similar
manufactured by Inkbit. These had a uniform response to pressure so could be
actuated using a single regulator. They were also much lighter and would not
deform under their own weight, though they were shorter, which limited the
size of graspable objects.

The chassis was 3D printed from PLA in two parts. The base plate had bolt
holes to attach the servo motor (Figure 8(a)). The cover had mount-points for
each finger recessed into slanted edges that were designed for easy
replacement (Figure 8(b)). The palm attached directly to the servo motor
through a hole in the cover.

(a)

(b)

Figure 8: The chassis was 3D printed from PLA in two parts to allow access to
the motor: a) The base plate of the chassis where the motor was mounted; b)
The cover of the chassis that supported the fingers on the sloped exterior.

The control system consisted of an Arduino, a Raspberry Pi and a PC linked
together using a ROS framework. The Raspberry Pi and Arduino provided a 0-5V
pulse-width-modulated output to each pressure controller, a serial output to
the motor, and toggled the 24V supply to the valve for the vacuum pump, while
the PC was used as the controller.

When the moulded fingers were used, the Arduino was programmed to map a single
linear input from the PC to the three separate voltage ranges for the pressure
controllers. The ranges were manually set to align the responses, allowing the
fingers to bend simultaneously. When using the 3D printed fingers, a single
actuation single going to all three fingers in parallel was sufficient.

##  IV Experimental Study

Grasping and rotating were tested with a variety of different weights and
shapes to ascertain how well the gripper would cope with manipulating real-
world objects. The sizes of objects used were restricted to those within the
grasping capabilities of the fingers. They had to be small enough to be
grasped and below 80g for the fingers to be able to lift them.

The tests were carried out by performing the full grasping process on each
object five times. All of the objects were tested with both the oval moulded
fingers and the 3D printed fingers (Table I). Each one was grasped from above,
rotated 180∘ to above the gripper, dropped on the palm, re-oriented, re-
grasped and placed back down. Because the whole sequence had to be completed
without error for an object to be deemed as having been successfully
manipulated, the individual stages were carefully observed to ascertain how a
particular object’s properties affected the process. If, for example, the
process repeatedly failed at a particular stage, the test would be continued
at the start of the subsequent stage.

Object | Mass (g)
---|---
Styrofoam Egg | 1
Cylindrical container | 33
Glove | 40
Tape | 50
Tennis ball | 62
TABLE I: A list of the objects used to test the gripper and their associated
weights

A styrofoam egg weighing 1g was tested first as it was an ideal shape to be
grasped and rotated (Figure 9). The moulded fingers had some trouble grasping
it as the non-uniform bending would push it out of the centre of the grasping
area. They also visibly sagged under their own weight while the gripper was
turning face up. The 3D printed fingers grasped and rotated the egg with no
issues. Both sets of fingers were able to drop the egg on to the palm easily.
The 3D printed fingers held it very close to the palm, so there was little
room for deviation. The conical shape the oval fingers that formed when
relaxed guided it towards the palm. Once on the palm, the egg could be freely
rotated within the servo motor’s range of movement. Re-grasping the egg was
only possible with the 3D printed fingers as the moulded fingers converged
above the egg when flexing.

Figure 9: The 3D printed fingers holding the styrofoam egg

A cylindrical container was then tested as although it was still an easy shape
to grasp, it would have a different weight distribution. Both sets of fingers
reliably grasped the container but while rotating it above the palm, the
moulded fingers would deform and lose grip. The 3D printed fingers were
shorter and could not grasp the cylinder around its center of mass, so the
cylinder sometimes twisted out of their grasp. When it was dropped on to the
palm it rotated easily. Both sets of fingers were also able to re-grasp the
cylinder as it was long enough to pass through the aperture created when the
moulded fingers converged. When rotating the palm face down, the moulded
fingers were able to support it as they pressed it against the palm. It acted
as a fourth point of contact, preventing the cylinder from twisting.

A roll of tape weighing 58g was tested as a heavy object. It was possible to
lift with both sets of fingers, but only when held by the gripper with the 3D
printed fingers, the tape could be consistently rotated without it dropping.
Because it was wide, when it was dropped by the silicone fingers, the
differential responses would cause one finger to stay in contact, tipping it
as it fell. The roll of tape would then hit the palm at an angle, and either
bounce off, falling between the fingers or rest partially on the palm with one
edge on a finger.

A rubber glove was also tested to simulate cloth-like items. These were
grasped successfully by the 3D printed fingers. The moulded fingers only
successfully grasped the glove after pinching it, raising part of the glove
off the table to a height at which the cylindrical fingertips could then grasp
it. Dropping on to the palm was difficult as the glove would end up draped
over the fingers. When it was successfully dropped on to the palm, the glove
would fall into the gaps between the fingers prevent it from being turned,
even with the suction cup active.

The whole manipulation process was consistently performed successfully on a
tennis ball weighing 62g. The fingers interacted well with it as they could
reach under it, cage it, and pull it onto the palm before the gripper was
rotated to the face up position. This was advantageous as having the center of
mass of the object closer to the palm exerted a smaller force on the fingers.
As a consequence the ball was never dropped, despite being the heaviest object
tested.

##  V Conclusion

A soft gripper with rotary palm was designed and implemented. It was tested
with a variety of objects and found to be effective at grasping and
reorienting them. After successfully grasping, objects weighing up to 62g were
easily rotated above 600∘ per second without slipping. Once on the palm, the
objects’ dimensions and shape did not effect accuracy - a common occurrence in
other fingered grippers. The exceptions were cloth-like objects and large
objects which were both obstructed by the surrounding fingers.

The suction cup on the palm was demonstrably powerful enough to lift objects
without using the fingers, which adds extra redundancy to the design. It could
also be used in tandem alongside the fingers to increase the maximum lifting
capacity.

In future work, the fingers should be made reorientable and retractable to
allow for larger objects to be rotated, as the base of the fingers can
obstruct larger objects during rotation. The ability to change the angle
between the fingers and the centre of the gripper would enable us to address
issues such as flexible, cloth-like objects partially falling down in-between
the fingers, and wide objects resting on only the palm and not the fingers. A
wider spreading of pressurised fingers would also allow wider objects to be
grasped.

It may also be necessary to find a method of preventing the fingers from
displacing non-circular objects when re-grasping them. One possible method
would be to add contact sensors and individually inflating the fingers until
they applied pressure to the object while it is held in place by the vacuum
palm. An alternative would be to make the fingers reorient themselves to re-
grasp the object in a way that does not disturb it, but that would
significantly increase the complexity of the gripper and would require
knowledge of the object’s shape.

The fingers of the gripper are currently too weak for practical application as
they can only reliably lift about 80g. This could be improved by using fibre
reinforced silicone or fabric based fingers. Adding electroadhesion would also
be a space efficient addition for improving grasping quality.

## Acknowledgment

Thank you very much to Mish Toszeghi for proof reading.

## References

  * [1] In _Softroboticstoolkit.com_ , 2022  URL: https://softroboticstoolkit.com/book/pneunets-bending-actuator
  * [2] Sylvain Abondance, Clark B. Teeple and Robert J. Wood  “A Dexterous Soft Robotic Hand for Delicate In-Hand Manipulation”  In _IEEE Robotics and Automation Letters_ 5.4, 2020, pp. 5502–5509  DOI: 10.1109/lra.2020.3007411
  * [3] Cosimo Gentile, Francesca Cordella and Loredana Zollo  “Hierarchical Human-Inspired Control Strategies for Prosthetic Hands”  In _Sensors_ 22.7, 2022, pp. 2521  DOI: 10.3390/s22072521
  * [4] Benjamin Gorissen et al.  “Elastic Inflatable Actuators for Soft Robotic Applications”  In _Advanced Materials_ 29.43, 2017, pp. 1604977  DOI: 10.1002/adma.201604977
  * [5] Agneta Gustus et al.  “Human hand modelling: kinematics, dynamics, applications”  In _Biological Cybernetics_ 106.11-12, 2012, pp. 741–755  DOI: 10.1007/s00422-012-0532-4
  * [6] Martin E. Héroux et al.  “Proprioception: a new look at an old concept”  In _Journal of Applied Physiology_ 132.3, 2022, pp. 811–814  DOI: 10.1152/japplphysiol.00809.2021
  * [7] William R. Johnson, Joran Booth and Rebecca Kramer-Bottiglio  “Integrated Sensing in Robotic Skin Modules”  In _2021 IEEE Sensors_ , 2021  DOI: 10.1109/sensors47087.2021.9639529
  * [8] Daekyum Kim et al.  “Review of machine learning methods in soft robotics”  In _PLOS ONE_ 16.2, 2021, pp. e0246102  DOI: 10.1371/journal.pone.0246102
  * [9] Anna Kochan  “Shadow delivers first hand”  In _Industrial Robot: An International Journal_ 32.1, 2005, pp. 15–16  DOI: 10.1108/01439910510573237
  * [10] Connor McCann, Vatsal Patel and Aaron Dollar  “The Stewart Hand: A Highly Dexterous, Six-Degrees-of-Freedom Manipulator Based on the Stewart-Gough Platform”  In _IEEE Robotics & Automation Magazine_ 28.2, 2021, pp. 23–36  DOI: 10.1109/mra.2021.3064750
  * [11] Amir Pagoli et al.  “A Soft Robotic Gripper With an Active Palm and Reconfigurable Fingers for Fully Dexterous In-Hand Manipulation *”  In _Ieeexplore.ieee.org_ , 2022  URL: https://ieeexplore.ieee.org/document/9495192
  * [12] Steffen Puhlmann, Jason Harris and Oliver Brock  “RBO Hand 3: A Platform for Soft Dexterous Manipulation”  In _IEEE Transactions on Robotics_ , 2022, pp. 1–16  DOI: 10.1109/tro.2022.3156806
  * [13] S.. Sadati et al.  “Mechanics of Continuum Manipulators, a comparative study of five methods with experiments”  In _Towards Autonomous Robotic Systems_ , 2017, pp. 686–702  DOI: 10.1007/978-3-319-64107-2_56
  * [14] Muhammad Mukhlis Suhaimi, Aimi Shazwani Ghazali, Ahmad Jazlan and Naim Sidek  “Analysis of High-Density Surface Electromyogram (HD-sEMG) signal for thumb posture classification from extrinsic forearm muscles”  In _Cogent Engineering_ 9.1, 2022  DOI: 10.1080/23311916.2022.2055445
  * [15] Li Tian et al.  “Design of a Single-Material Complex Structure Anthropomorphic Robotic Hand”  In _Micromachines_ 12.9, 2021, pp. 1124  DOI: 10.3390/mi12091124
  * [16] Chao Wang et al.  “Parameter estimation and object gripping based on fingertip force/torque sensors”  In _Measurement_ 179, 2021, pp. 109479  DOI: 10.1016/j.measurement.2021.109479
  * [17] Wenpei Zhu et al.  “A Soft-Rigid Hybrid Gripper With Lateral Compliance and Dexterous In-Hand Manipulation”  In _IEEE/ASME Transactions on Mechatronics_ , 2022, pp. 1–12  DOI: 10.1109/tmech.2022.3195985
  * [18] Shiping Zuo, Jianfeng Li and Mingjie Dong  “Design, modeling, and manipulability evaluation of a novel four-DOF parallel gripper for dexterous in-hand manipulation”  In _Journal of Mechanical Science and Technology_ 35.7, 2021, pp. 3145–3160  DOI: 10.1007/s12206-021-0636-7


Murali Karthick Baskar, Andrew Rosenberg, Bhuvana Ramabhadran, Kartik
Audhkhasi

# O-1: Self-training with Oracle and 1-best Hypothesis

###### Abstract

We introduce O-1, a new self-training objective to reduce training bias and
unify training and evaluation metrics for speech recognition. O-1 is a faster
variant of Expected Minimum Bayes Risk (EMBR), that boosts the oracle
hypothesis and can accommodate both supervised and unsupervised data. We
demonstrate the effectiveness of our approach in terms of recognition on
publicly available SpeechStew datasets and a large-scale, in-house data set.
On Speechstew, the O-1 objective closes the gap between the actual and oracle
performance by 80% relative compared to EMBR which bridges the gap by 43%
relative. O-1 achieves 13% to 25% relative improvement over EMBR on the
various datasets that SpeechStew comprises of, and a 12% relative gap
reduction with respect to the oracle WER over EMBR training on the in-house
dataset. Overall, O-1 results in a 9% relative improvement in WER over EMBR,
thereby speaking to the scalability of the proposed objective for large-scale
datasets.

Index Terms:Self-training, EMBR, O-1, ASR, speech recognition, discriminative
training.

##  1 Introduction

The use of labeled and unlabeled data in training has led to a suite of co-
training and self-training [1] methods in speech and language processing. The
underlying principle in these algorithms is an iterative learning paradigm
where a model learns from a ground-truth distribution as well as from its own
predictions at the previous iteration with appropriate regularization [2].
These algorithms serve to mitigate three key problems. First, exposure bias in
autoregressive models [3], where the model is conditioned only from the
supervised training data (teacher forcing) [4] and thus never learns from its
own predictions. This leads to early convergence but is mismatched with the
inference step, where no ground-truth is available. Scheduled sampling [5]
mitigates this issue by exposing the model to its past/own predictions at
random instances. Second, mismatches between training objectives and
evaluation metrics such as, maximum likelihood estimation (MLE) vs Word Error
Rate (WER)) [6, 3] which cause performance degradation of the model. While the
former results in error accumulation during inference, the latter ignores the
lattice of predictions which can be utilized to learn from unsupervised data.
Thirdly, label bias in e2e systems occurs when the model cannot relearn from
new labels and remains biased to past labels. This label bias is mostly subtle
and is studied and addressed in [7] with the use of global normalization.

Prior work on Automated Speech Recognition (ASR) has shown that, Minimum Bayes
risk (MBR) approach is capable of handling the exposure bias [8, 9], training-
inference criterion mismatch [10, 11] and the label bias problem [7, 12]. MBR
is a self-training paradigm (commonly referred to as discriminative training)
[13] and is either termed as expected MBR (EMBR) [14, 15, 16] when used to
reduce expected word errors or state MBR (sMBR) when minimizing the state
level expected errors [17, 18, 10, 19, 20]. Despite the benefits of EMBR, it
suffers from two main drawbacks:

  * •

EMBR formulation does not directly impact the oracle hypothesis (best WER),
but instead focusses on expected error reduction by boosting multiple
hypotheses scores.

  * •

The training duration is relatively high compared to MLE training as batch
size gets scaled by the beam size.

In this work, we propose a novel self-training objective, referred to as O-1.
It is a variant of EMBR formulated to discriminatively boost only the Oracle
hypothesis from the 1-best hypothesis. The primary contributions of this work
are:

  * •

We introduce, the O-1 objective which allows the efficient use of both
supervised data and unsupervised data. O-1 is integrated with hard
distillation [21] for unsupervised training. To the best of our knowledge, O-1
is the first variant of EMBR which can perform unsupervised learning.

  * •

Optimized training time (and computational costs) compared to EMBR, thereby
paving the way to train for longer duration and more data.

  * •

Effective self-training paradigm that uses WER to to select the best sequence
predicted by the model as the target label sequence.

  * •

The proposed objective results in a significant reduction of the gap between
1-best WER and the oracle WER. Experiments conducted on the set of public
corpora in Speechstew show consistent reduction in WER, by 13-25 % relative.

##  2 Related Work

Exposure bias [5, 22, 23] is well-known as a central problem for auto-
regressive models such as language models. Scheduled sampling [5] is a form of
self-training proposed to alleviate this training-inference bias [24] caused
by the maximum-likelihood (MLE) based training objective. Beam search
optimization was proposed to reduce exposure bias in [25]. It collects and
improves the scores of each ground truth token and the previous token leading
to incorrect prediction in each beam search step. Recent work on reducing
exposure bias for ASR using recurrent neural network transducers (RNN-T)
claims that the direct application of scheduled sampling to RNN-T
architectures is challenging as it is not a generative architecture [26],
unlike self-attention based transformer like encoder-decoder architectures
[27]. To mitigate this bias, the authors sample the target label from the
top-k candidates from a language model instead of the model itself. In [28],
the EMBR computation is optimized for RNN-T architectures by using alignment
scores for target hypotheses. The bias introduced during training-inference
with LM rescoring is addressed in [16] using the EMBR objective. Prefix level
EMBR is proposed in [29, 30, 15] to impose correct prefixes in the beam by
boosting prefixes with less errors. Scheduled sampling uses a single
hypothesis and EMBR uses multiple hypotheses in their objectives, but neither
of the above approaches select the oracle hypothesis and boosts its score. The
proposed O-1 mitigates this constraint and has shown consistent improvements
in recognition performance.

Figure 1: Visualization of O-1 training defined in Equation (4) that boosts
the score of the oracle hypothesis to the top of the beam.

##  3 O-1: Proposed Approach

In this work, we use Recurrent Neural Network Transducers [31] are used as the
framework for EMBR and the proposed O-1 objectives. Figure 1 visualizes the
training procedure corresponding to Equation (4).

###  3.1 EMBR

Given an input speech sequence $\mathbf{X}$, the encoder network, prediction
and joint network in an RNN-T model help to compute the alignment probability
$p(\mathbf{a}\mid\mathbf{X})$ with the help of forward-backward algorithm. The
RNN-T training objective $\mathcal{L}_{rnnt}$ is defined as:

$\begin{split}\mathcal{L}_{rnnt}=-\log
p(\mathbf{Y}\mid\mathbf{X})=-\log\sum_{\mathbf{a}\in\mathcal{B}^{-1}(\mathbf{Y})}p(\mathbf{a}\mid\mathbf{X})\end{split}$
(1)

Here, $\mathcal{B}$ operation removes all blank labels in a given sequence and
$p(\mathbf{Y}\mid\mathbf{X})$ is the probability of the predicted hypothesis
$\mathbf{Y}$. Based on the objective in Equation (1), the EMBR loss [28] for
RNNT is computed as:

$\begin{split}\mathcal{L}_{embr}=\sum_{\mathbf{Y}_{i}\in\mathcal{Y}}\hat{p}(\mathbf{Y}_{i}\mid\mathbf{X})W(\mathbf{Y}_{i},\mathbf{Y}^{*})\end{split}$
(2)

where
$\hat{p}(\mathbf{Y}_{i}\mid\mathbf{X})=\textrm{softmax}(p(\mathbf{Y}_{i}\mid\mathbf{X}))$
is the probability of $i^{\textrm{th}}$ hypothesis in the beam obtained by
applying a softmax function on $\log p$ of the RNN-T model to normalize over
the n-best hypotheses. Here, $\mathcal{Y}$ denotes the n-best list and
$W(\mathbf{Y}_{i},\mathbf{Y}^{*})$ is the number of erroneous words computed
using an edit distance between each hypothesis in the beam $\mathbf{Y}_{i}$
and the ground-truth $\mathbf{Y}^{*}$. In the partial derivative of the EMBR
objective below,

$\begin{split}\frac{\partial\mathcal{L}_{embr}}{\partial\log
p(\mathbf{Y}_{i}\mid\mathbf{X})}&=\hat{p}(\mathbf{Y}_{i}\mid\mathbf{X})(W(\mathbf{Y}_{i},\mathbf{Y}^{*})-\bar{W})\\\
\end{split}$ (3)

$\bar{W}=\underset{\mathbf{Y}_{j}\in\textrm{n-best}}{\sum}\hat{p}(\mathbf{Y}_{j}\mid\mathbf{X})W(\mathbf{Y}_{j},\mathbf{Y}^{*})$
denotes the expected word errors or risk among all the hypotheses for training
an utterance. Monte-Carlo approximation of the n-best list results in
$\bar{W}\approx\underset{\mathbf{Y}_{j}\in\textrm{n-best}}{\sum}W(\mathbf{Y}_{j},\mathbf{Y}^{*})$
as the average number of word errors. The subtraction of $\bar{W}$ in (3)
performs variance reduction [32, 14]. In [8], the authors applied $\bar{W}$
directly in (2) also as a form of variance reduction, but computing $\bar{W}$
is empirical and tricky [22].

###  3.2 O-1 Training Procedure

The O-1 training objective $\mathcal{L}(\mathbf{X},\mathbf{Y},\mathbf{Y}^{*})$
is defined as:

$\begin{split}\mathcal{L}_{O1}=-\log
p(\mathbf{Y}_{oracle}\mid\mathbf{X})\cdot(1-\textrm{WER}(\mathbf{Y}_{oracle},\mathbf{Y}^{*}))\\\
+\log
p(\mathbf{Y}_{1-best}\mid\mathbf{X})\cdot\textrm{WER}(\mathbf{Y}_{1-best},\mathbf{Y}^{*})\end{split}$
(4)

The training steps to implement Equation (4) are enumerated below:

  1. 1.

Beam-search decoding is performed during training to extract n-best hypotheses
$\mathbf{Y}_{i=1:n}$.

  2. 2.

Each hypothesis $\mathbf{Y}_{i}$ is scored against the ground-truth or pseudo-
label from the teacher to compute WER.

  3. 3.

The hypothesis with the best WER serves as the oracle hypothesis
$\mathbf{Y}_{oracle}$ and the hypothesis with the top probability becomes the
1-best hypothesis $\mathbf{Y}_{1-best}$. Both oracle and 1-best hypotheses are
chosen from the beam while dropping the rest of the hypotheses.

  4. 4.

$-\log p(\mathbf{Y}_{i,j}\mid\mathbf{X})$ is the RNN-T loss for $i=oracle$
hypothesis and $j=1-best$ hypothesis computed using (1).

  5. 5.

Each of these two RNN-T losses is scaled by its corresponding WER as described
in Equation (4). This idea of scaling objectives with the evaluation metric
has also been explored for pre-training in [33].

  6. 6.

When the scores of 1-best and oracle hypotheses are equal,
$\mathcal{L}_{O1}\rightarrow 0$ in (4) and the auxiliary $\mathcal{L}_{rnn-t}$
loss will overcome this zero loss scenario as given in Equation (5).

It can be seen that O-1 training simplifies EMBR training by eliminating the
need to consider all the alternate hypotheses and eliminates the computation
of $\bar{W}$. O-1 objective also does not require a softmax computation that
EMBR requires (Equation (2)). The final EMBR and O-1 losses used in our
experiments is a multitask training objective given by:

$\begin{split}\mathcal{L}_{embr}^{{}^{\prime}}=\mathcal{L}_{embr}+\gamma\mathcal{L}_{rnnt}\\\
\mathcal{L}_{O1}^{{}^{\prime}}=\mathcal{L}_{O1}+\lambda\mathcal{L}_{rnnt}\end{split}$
(5)

O-1 explicitly targets the hypothesis with less WER to be boosted without
relying on a error threshold $W-\bar{W}$ as in Equation (3). O-1 performs
discriminative training between oracle and 1-best scores. This formulation of
O-1 naturally addresses the label bias problem as well; when the model is
pushing the oracle hypothesis score to be on top of the beam, the 1-best
hypothesis score has to come down to respect the discriminative aspect of O-1.

###  3.3 O-1 Unsupervised training

Hard distillation [34, 35] involves training a student model with the target
labels obtained on-the-fly from the teacher model as in (6). Given the reduced
computational complexity compared to EMBR, we choose to employ O-1 in an
unsupervised setting using the distillation loss given in [34]. Thus, O-1 loss
with hard distillation becomes:

$\begin{split}\mathcal{L}_{distill}&=\mathcal{L}_{rnnt}(\mathbf{X},\mathbf{Y}^{{}^{\prime}})\\\
\mathcal{L}_{O1\\_distill}&=\mathcal{L}_{O1}(\mathbf{X},\mathbf{Y},\mathbf{Y}^{{}^{\prime}})+\lambda\,\mathcal{L}_{distill},\end{split}$
(6)

where, $\mathbf{Y}^{{}^{\prime}}$ denotes the pseudo-label generated by the
teacher model during training.

##  4 Experiments

###  4.1 Datasets

We conduct experiments on both publicly available and in-house corpora.
SpeechStew, described in [36], a publicly available and well-benchmarked
corpus, composed of multiple datasets ($\approx$5000 hours) spanning several
acoustic conditions and domains. It comprises of 960 hrs of Librispeech, 100
hrs of AMI, 1.5k hrs of CommonVoice corpus, 50 hrs of English broadcast news,
2k hrs of Switchboard/Fisher, 450 hrs of TED-lium and 80 hrs of WSJ corpus.
The in-house corpus consists of Google's short Voice Search (VS) utterances in
U.S. English. This supervised ASR training data is anonymized and hand-
transcribed. The development and test sets are a small fraction of the
training set held out for validation and evaluation. Also included in the
training are nearly 300 thousand hours of multi-domain utterances spanning
domains such as Dictation, YouTube and Telephony for US English [37]. The
unsupervised dataset contains 500M short search utterances. While the public
corpora allows us to benchmark against state-of-the-art algorithms, the in-
house corpora allows us to study performance on tail distributions and
evaluate the scalability of the algorithm to large-scale training data and
streaming E2E setups.

###  4.2 Model Description

The model architecture used in this work is based on the RNN-T architecture
first described in [31]. It consists of a conformer encoder [38] and an LSTM
decoder. The encoder comprises of several blocks, with each block including a
series of multi-headed self attention, depth-wise convolution and feed-forward
layers. For our experiments with SpeechStew, we use two different
configurations, namely a 100M and 600M parameter model with 17 and 24
conformer layers with 512 and 1024 encoder dimensions, respectively. Both
models use relative attention. All models are trained on use 80-dimensional
log-mel filter bank coefficients. We use SpecAugment [39] with time and
frequency masking for all experiments. For SpeechStew, the number of frequency
masking bins $F$ = 27 and time bins $T$ = 10 with a maximum mask ratio set to
0.05. For the in-house dataset, $F$ = 27, $T$ = 50 and mask ratio to 0.1. For
experiments with the in-house corpus, we use a 120M parameter, streaming
Hybrid Autoregressive Transducer (HAT) [40] architecture, comprising of a
conformer encoder with 12 layers and a step decoder with V2 embedding as
described in [16]. In addition to SpecAugment, we use multistyle training
similar to [41]. The experiments on SpeechStew use 1024 word-piece targets and
the in-house experiments on the Voice Search task use 4K word-piece targets
derived using the algorithm presented. in [42].

###  4.3 O-1 Supervised Self Training

We use the following training strategy to study the impact of the proposed
algorithm with supervised-only training data. A 100M RNN-T model is trained
from scratch using the supervised, SpeechStew corpora. The 600M parameter
model is finetuned from a wav2vec2 [43] encoder that is pretrained with
Librilight data. The model is trained to convergence (approximately 300K
iterations) and serves as the baseline model for subsequent EMBR and O-1
training (100K iterations). Based on empirical evidence we set the RNN-T loss
scaling factor as $\lambda=0.1,\gamma=0.1$ for the EMBR $\mathcal{L}_{embr}$
and the O-1 training objectives $\mathcal{L}_{O1}$. In all our O-1
experiments, the RNN-T loss in Equation (1) is normalized with the target
label sequence length before using for O-1 loss computation in (4). SpeechStew
test sets [36] are used for evaluation.

###  4.4 O-1 Unsupervised Learning

In the unsupervised learning scenario, the teacher is a non-causal model with
right context fixed to 64 and the student is a streaming model with zero right
context. The teacher model is trained with the supervised data alone and the
baseline student model is trained with the same supervised data and additional
unsupervised data. For unsupervised learning, we use a teacher model's
transcripts as pseudo labels. We use the hard distillation (Equation (6)), to
train both student and teacher models jointly. Typically CE training converges
after 500K iterations. This serves as the CE baseline to continue with EMBR
training (Equation (2)). For O-1 training (Equation (4)), the teacher and
student are initialized in the same way the baseline, i.e., we do not require
the additional CE training step that conventional EMBR training requires. As
explained in Section 3, CE objective is included as an auxiliary loss in the
O-1 objective. All experiments related to unsupervised learning are conducted
on the in-house corpora, since vast amounts of untranscribed data is
available. Each training batch contains equal amounts of transcribed and
untranscribed data. Beam-search is done with a beam size of 8 for hypotheses
generation in EMBR training and inference. In the case of O-1, beam size of 18
is set for SpeechStew and 8 for in-house dataset during training. O-1
inference is done using beam size of 8 to perform fair comparison with EMBR.

Figure 2: [A] Average WER of all test sets in SpeechStew evaluated at
different checkpoints trained with different beam sizes using the proposed O-1
objective and EMBR. [B] Plot showing the number of hypotheses required to
reach the optimal performance in EMBR and O-1. Beam search is fixed to compute
8 hypotheses. EMBR is computed from top8 hypotheses. O-1's WER is close to the
oracle WER (1.67%) compared to EMBR on librispeech test clean. Model is
trained on SpeechStew datasets. [C] 1-best WER vs oracle WER on Librispeech
test-clean using O-1, baseline and EMBR

##  5 Results and Analysis

Table 1 shows the performance comparison of O-1 with the baseline and the EMBR
models. EMBR shows inconsistent behavior in different testsets. For instance,
EMBR degrades on AMI-IHM, Librispeech test and testother, TED-LIUM and WSJ-
eval92. EMBR shows gains on AMI-SDM, CHiME and Commonvoice which are
relatively harder tasks over rest of the datasets. O-1 provides consistent
improvement in all testsets over both the baseline and the EMBR models with a
maximum of 25% relative gain on WSJ-eval92. The primary reason for O-1's
improved performance is learning from the best hypothesis targets. This
enables the model to achieve closer to oracle performance during training as
shown in Figure 2C and gets reflected in evaluation.

Table 1: Recognition performance on Speechstew evaluation sets with 100M and
600M parameter models

#Params | % WER | AMI | CHiME | CV | Libri-test | TED | WSJ
---|---|---|---|---|---|---|---
IHM | SDM | clean | other
100M | Baseline | 10.2 | 24.8 | 34.9 | 11.5 | 2.3 | 4.9 | 4.8 | 2.6
EMBR | 11.3 | 23.7 | 33.2 | 11.2 | 2.4 | 5.4 | 4.7 | 2.7
O-1 | 9.8 | 23.6 | 31.6 | 11.1 | 2.1 | 4.6 | 4.4 | 2.0
600M | Baseline | 9.6 | 23.8 | 33.5 | 11.5 | 1.8 | 3.6 | 5.6 | 1.4
EMBR | 9.1 | 22.7 | 29.7 | 10.2 | 1.9 | 3.6 | 4.8 | 1.4
O-1 | 8.7 | 20.4 | 25.3 | 8.9 | 1.7 | 3.4 | 4.3 | 1.3

We also conducted the same experiment on a 600M parameter model to investigate
if the performance of the new objective is complementary to the increase in
model complexity. Table 1 shows that on average, O-1 achieves 8.4% rel.
improvement over EMBR and 13% relative improvement over the state-of-the-art
baseline system in [36]. O-1 shows relative improvements of up to 25% on WSJ,
15% on CHiME and greater than 10% on TED, AMI-SDM and CommonVoice.

###  5.1 Comparison with Scheduled Sampling

Analogous to our approach, scheduled sampling also allows the model to train
from its own predictions. However, O-1 differs from scheduled sampling in two
aspects: (1) The oracle hypothesis serves as the target score to be improved,
while the 1-best hypothesis score is suppressed. In the case of scheduled
sampling, the 1-best hypothesis is chosen as the target based on the selection
ratio. (2) The oracle hypothesis score is scaled by the corresponding WER in
O-1 which is not done in scheduled sampling. Empirical analysis is conducted
on Librispeech test set to evaluate the behavior of O-1 and scheduled sampling
approach. Here we perform scheduled sampling at the utterance level without
any language model [26]. O-1 obtains 8% relative improvement over scheduled
sampling approach on test-other and 12.5% relative gain on test.

###  5.2 Training duration, Performance and Beam size

During O-1 training all the samples in the beam are not involved in loss
computation which reduces the exposure to erroneous samples. Batch size is not
impacted by the number of samples in the beam during O-1 training. These two
factors allow the model tained with O-1 to use a larger beam size unlike EMBR
training. Figure 2A shows the effect of increasing the beam size during
training. "O1 - 18" (O-1 trained with beam size of 18) shows substantial gains
over EMBR. We also conducted an experiment by training using the oracle
hypothesis only and this achieved a train time similar to the baseline model.
Efficient inferencing on TPUs [44] results in maximum computation resting on
the N-best hypothesis generation. On average, the O-1 processes 945
examples/sec and EMBR processes 754 examples/sec for the SpeechStew data with
a total of 3.6 million examples, which makes it faster over EMBR training.

###  5.3 Are all hypothesis needed during training ?

Figure 2B shows the performance of EMBR and O-1 based on the number of
hypotheses involved in loss computation. Here, we argue that the reason for
EMBR attaining the best performance with 8 hypotheses is that on average the
oracle hypothesis is between 5${}^{\textrm{th}}$ and 8${}^{\textrm{th}}$
position within the top 8 hypotheses. O-1 is capable of retrieving the oracle
hypothesis with a single hypothesis used in its training, and attains best
performance threshold with the inclusion of 1-best.

To further evaluate the performance difference between actual and oracle WER,
the O-1 and EMBR models were evaluated at each training checkpoint as shown in
Figure 2C. O-1's 1-best WER gets closer to the oracle WER as training
progresses and matches the oracle performance at 100K iterations, indicating
that use of all n-best hypotheses is unnecessary.

Figure 3: Comparison of 1best WER with oracle WER on librispeech test set
using our proposed O-1 approach, baseline and EMBR

###  5.4 Results on O-1 Unsupervised Learning

The experiments on a large-scale in-house data set comprising of short
utterances up to 4 seconds long, shows a similar trend to the public data
sets. Two audio sources were used for this evaluation, real and TTS generated
audio. The test sets include several named entities. For example, the RPN
datasets contains proper nouns, place names, music genres, artists and other
media related entities. Each test set contains 10K utterances. From Table 2 we
can see that O-1 attains 8% and 9% relative improvement over EMBR on two of
the four testsets while yielding modest improvements on the remaining test
sets. This indicates that the O-1 objective handles streaming recognition well
compared to EMBR and scales to large-scale data sets.

Table 2: Recognition performance on in-house data sets in U.S. English
including VS and rare proper nouns (RPN-) from multiple application sources.

% WER | VS | RPN-M | RPN-N | RPN-P | RPN-Q | RPN-Y
---|---|---|---|---|---|---
Baseline | 6.0 | 13.2 | 10.1 | 37.3 | 20.6 | 24.1
EMBR | 5.8 | 13.1 | 9.9 | 36.9 | 20.3 | 24.0
O-1 | 5.8 | 12.4 | 9.6 | 35.5 | 18.5 | 22.2

##  6 Conclusion

We have proposed O-1, a novel self-training objective to learn from oracle
hypothesis. Our experiments shows that the O-1 reduces the difference between
1-best and oracle performance by a significant margin in multiple evaluation
sets. O-1 shows promise by performing well on tasks where EMBR fails to
improve. O-1 enjoys additional benefit of optimized training time over EMBR.
In addition the results show that O-1 shows gains in both supervised and
unsupervised setting under various test conditions. O-1 achieves 13%, 15% and
25% relative improvement over EMBR on AMI-HM, Librispeech-testother and WSJ-
eval92 testsets respectively. In a real-time, streaming setting, O-1 provides
9% relative improvement over EMBR on a large scale in-house test set. O-1 can
be further improved in the future using multiple hypotheses from the teacher
model and applying advanced distillation approaches.

## References

  * [1] Colin Wei, Kendrick Shen, et al.,  ``Theoretical analysis of self-training with deep networks on unlabeled data,''  in ICLR, 2021.
  * [2] Yves Grandvalet and Yoshua Bengio,  ``Semi-supervised learning by entropy minimization,''  Advances in NeurIPS, vol. 17, 2004.
  * [3] Marc'Aurelio Ranzato, Sumit Chopra, et al.,  ``Sequence level training with recurrent neural networks,''  in ICLR, 2016.
  * [4] Ronald J Williams and David Zipser,  ``A learning algorithm for continually running fully recurrent neural networks,''  Neural computation, vol. 1, no. 2, pp. 270–280, 1989.
  * [5] Samy Bengio, Oriol Vinyals, et al.,  ``Scheduled sampling for sequence prediction with recurrent neural networks,''  Advances in NeurIPS, vol. 28, 2015.
  * [6] Dzmitry Bahdanau, Philemon Brakel, et al.,  ``An actor-critic algorithm for sequence prediction,''  ArXiv, vol. abs/1607.07086, 2016.
  * [7] Ronan Collobert, Awni Hannun, et al.,  ``A fully differentiable beam search decoder,''  in Proceedings of the 36th ICML, Kamalika Chaudhuri and Ruslan Salakhutdinov, Eds. 09–15 Jun 2019, vol. 97 of PMLR, pp. 1341–1350, PMLR.
  * [8] Rohit Prabhavalkar, Tara N Sainath, et al.,  ``Minimum word error rate training for attention-based sequence-to-sequence models,''  in IEEE ICASSP. IEEE, 2018, pp. 4839–4843.
  * [9] Liang Lu, Zhong Meng, et al.,  ``On Minimum Word Error Rate Training of the Hybrid Autoregressive Transducer,''  in Proc. Interspeech 2021, 2021, pp. 3435–3439.
  * [10] Daniel Povey,  Discriminative training for large vocabulary speech recognition,  Ph.D. thesis, University of Cambridge, 2005.
  * [11] Mathias Müller and Rico Sennrich,  ``Understanding the properties of minimum Bayes risk decoding in neural machine translation,''  in International Joint Conference on NLP (Volume 1: Long Papers), Online, Aug. 2021, pp. 259–272, Association for Computational Linguistics.
  * [12] Zhiyun Lu, Yanwei Pan, et al.,  ``Input length matters: Improving rnn-t and mwer training for long-form telephony speech recognition,''  arXiv preprint arXiv:2110.03841, 2021.
  * [13] Vaibhava Goel and William J Byrne,  ``Minimum bayes-risk automatic speech recognition,''  Computer Speech & Language, vol. 14, no. 2, pp. 115–135, 2000\.
  * [14] Matt Shannon,  ``Optimizing expected word error rate via sampling for speech recognition,''  arXiv preprint arXiv:1706.02776, 2017.
  * [15] Murali Karthick Baskar, Lukáš Burget, et al.,  ``Promising accurate prefix boosting for sequence-to-sequence asr,''  in IEEE ICASSP, 2019, pp. 5646–5650.
  * [16] Wang Weiran, Tongzhou Chen, et al.,  ``Improving Rare Word Recognition with LM-aware MWER Training,''  in Proc. Interspeech 2022, 2022, pp. 1031–1035.
  * [17] Karel Veselỳ, Arnab Ghoshal, et al.,  ``Sequence-discriminative training of deep neural networks.,''  in Interspeech, 2013, vol. 2013, pp. 2345–2349.
  * [18] Daniel Povey and P Woodland,  ``Improved discriminative training techniques for large vocabulary continuous speech recognition,''  in ICASSP. IEEE, 2001, vol. 1, pp. 45–48.
  * [19] Brian Kingsbury,  ``Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling,''  in 2009 IEEE ICASSP. IEEE, 2009, pp. 3761–3764.
  * [20] Daniel Povey and Brian Kingsbury,  ``Evaluation of proposed modifications to mpe for large scale discriminative training,''  in 2007 IEEE ICASSP'07. IEEE, 2007, vol. 4, pp. IV–321.
  * [21] Mohammad Zeineldeen, Kartik Audhkhasi, et al.,  ``Robust knowledge distillation from rnn-t models with noisy training labels using full-sum loss,''  in IEEE ICASSP, 2023, pp. 1–5.
  * [22] Steven J Rennie, Etienne Marcheret, et al.,  ``Self-critical sequence training for image captioning,''  in Proceedings of the IEEE conference on CVPR, 2017, pp. 7008–7024.
  * [23] Tianxing He, Jingzhao Zhang, et al.,  ``Exposure bias versus self-recovery: Are distortions really incremental for autoregressive text generation?,''  arXiv preprint arXiv:1905.10617, 2019.
  * [24] Abhishek Arun, Barry Haddow, et al.,  ``A unified approach to minimum risk training and decoding,''  in Proceedings on Statistical Machine Translation and MetricsMATR, 2010, pp. 365–374.
  * [25] Sam Wiseman and Alexander M. Rush,  ``Sequence-to-sequence learning as beam-search optimization,''  in Proceedings of the 2016 Conference on EMNLP, Austin, Texas, Nov. 2016, pp. 1296–1306, Association for Computational Linguistics.
  * [26] Xiaodong Cui, Brian Kingsbury, et al.,  ``Reducing Exposure Bias in Training Recurrent Neural Network Transducers,''  in Proc. Interspeech 2021, 2021, pp. 1802–1806.
  * [27] Dzmitry Bahdanau, Jan Chorowski, et al.,  ``End-to-end attention-based large vocabulary speech recognition,''  in IEEE ICASSP. IEEE, 2016, pp. 4945–4949.
  * [28] Jinxi Guo, Gautam Tiwari, et al.,  ``Efficient minimum word error rate training of rnn-transducer for end-to-end speech recognition,''  in Interspeech 2020, 2020.
  * [29] V. Doumpiotis, S. Tsakalidis, et al.,  ``Discriminative training for segmental minimum bayes risk decoding,''  in ICASSP Proceedings, 2003, vol. 1, pp. I–I.
  * [30] Sara Sabour, William Chan, et al.,  ``Optimal completion distillation for sequence learning,''  in ICLR, 2019.
  * [31] Alex Graves,  ``Sequence transduction with recurrent neural networks,''  arXiv preprint arXiv:1211.3711, 2012.
  * [32] Richard S Sutton, David McAllester, et al.,  ``Policy gradient methods for reinforcement learning with function approximation,''  NeurIPS, vol. 12, 1999.
  * [33] Murali Karthick Baskar, Andrew Rosenberg, et al.,  ``Ask2mask: Guided data selection for masked speech modeling,''  IEEE JSTSP, vol. 16, no. 6, pp. 1357–1366, 2022.
  * [34] Daniel S. Park, Yu Zhang, et al.,  ``Improved Noisy Student Training for Automatic Speech Recognition,''  in Proc. Interspeech 2020, 2020, pp. 2817–2821.
  * [35] Dongseong Hwang, Khe Chai Sim, et al.,  ``Comparison of soft and hard target rnn-t distillation for large-scale asr,''  arXiv preprint arXiv:2210.05793, 2022.
  * [36] William Chan, Daniel Park, et al.,  ``Speechstew: Simply mix all available speech recognition data to train one large neural network,''  arXiv preprint arXiv:2104.02133, 2021.
  * [37] Arun Narayanan, Rohit Prabhavalkar, et al.,  ``Recognizing long-form speech using streaming end-to-end models,''  in 2019 IEEE ASRU. IEEE, 2019, pp. 920–927.
  * [38] Anmol Gulati, James Qin, et al.,  ``Conformer: Convolution-augmented transformer for speech recognition,''  arXiv preprint arXiv:2005.08100, 2020.
  * [39] Daniel S. Park, William Chan, et al.,  ``SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition,''  in Proc. Interspeech 2019, 2019, pp. 2613–2617.
  * [40] Ehsan Variani, David Rybach, et al.,  ``Hybrid autoregressive transducer (hat),''  in IEEE ICASSP. IEEE, 2020, pp. 6139–6143.
  * [41] Chanwoo Kim, Ananya Misra, et al.,  ``Generation of large-scale simulated utterances in virtual rooms to train deep-neural networks for far-field speech recognition in google home,''  in Interspeech, 2017.
  * [42] Taku Kudo and John Richardson,  ``Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,''  arXiv preprint arXiv:1808.06226, 2018.
  * [43] Alexei Baevski, Yuhao Zhou, et al.,  ``wav2vec 2.0: A framework for self-supervised learning of speech representations,''  in Advances in NeurIPS, 2020.
  * [44] Norman P Jouppi, Doe Hyun Yoon, et al.,  ``Ten lessons from three generations shaped google's tpuv4i: Industrial product,''  in 2021 ACM/IEEE ISCA. IEEE, 2021, pp. 1–14.