# A3VLM: Actionable Articulation-Aware Vision Language Model

Siyuan Huang1,2,∗, Haonan Chang3,∗, Yuhan Liu3, Yimeng Zhu4, Hao Dong5,
Peng Gao2 ✉, Abdeslam Boularias3 ✉, Hongsheng Li6 ✉
1SJTU, 2Shanghai AI Lab, 3 Rutgers University, 4 Yuandao AI, 5 PKU, 6 CUHK
MMLab
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Vision Language Models (VLMs) have received significant attention in recent
years in the robotics community. VLMs are shown to be able to perform complex
visual reasoning and scene understanding tasks, which makes them regarded as a
potential universal solution for general robotics problems such as
manipulation and navigation. However, previous VLMs for robotics such as RT-1
[4], RT-2 [3], and ManipLLM [21] have focused on directly learning robot-
centric actions. Such approaches require collecting a significant amount of
robot interaction data, which is extremely costly in the real world. Thus, we
propose A3VLM, an object-centric, actionable, articulation-aware vision
language model. A3VLM focuses on the articulation structure and action
affordances of objects. Its representation is robot-agnostic and can be
translated into robot actions using simple action primitives. Extensive
experiments in both simulation benchmarks and real-world settings demonstrate
the effectiveness and stability of A3VLM. We release our code and other
materials at https://github.com/changhaonan/A3VLM.

††∗ Equal Contribution.

##  1 Introduction

Figure 1: Sequential inference with prompts. To answer the first question,
A3VLM identifies the corresponding action type and bounding box for the
movable part of the scissors. Then, the second answer reveals the articulation
structure of the scissor leg. A manipulation action can be performed based on
the answer.

The use of Large Language Models (LLMs) [6] for robotics applications has
recently attracted significant attention. Combined with robot control APIs
[23, 16], open-vocabulary scene graphs [7], or motion planners [8], LLMs have
demonstrated an extraordinary ability to understand users’ commands, reason
about the environment, and select the correct action from skill pools. In
these works, LLMs usually rely on external vision tools such as an open-
vocabulary detector to convert scene information into text. This
straightforward combination, however, prevents LLMs from acquiring further
scene understanding and certain important details about the environment.
Subsequently, researchers have begun to directly train Vision Language Models
(VLMs) for manipulation and navigation tasks. Compared to methods that combine
LLMs with external vision tools, native VLMs can capture detailed visual data
and perform complex visual reasoning, making it possible to be an all-in-one
solution for solving general manipulation tasks. However, the training of VLMs
is extremely data-hungry, and the collection of image-text pair data for
robotics VLMs can be a significant problem. Furthermore, the output of VLMs is
purely text-based, which is fundamentally different from the standard robot
action representation as trajectories. Thus, robot VLMs need to define a
tailored and precise action representation for a robotic system.

There have been several attempts to address the data gathering and the action
representation problems. RT-1 [4] and RT-2 [3] directly represent the robot’s
end-effector pose using a discretized 6D pose and collect a large amount of
image-action pair data using teleoperation. ManipLLM [21] simplifies this
action representation by replacing the parallel gripper with a suction
gripper, requiring only the computation of a contact point and the gripper’s
direction. Interaction data is then collected in simulation. Such large
amounts of robot interaction data are expensive to collect in the real world.

To address this issue, we shift our focus from directly learning actions to
learning an object-centeric representation that is independent of the robot’s
configuration and that can easily be translated into low-level manipulation
actions. We propose the Actionable Articulation-Aware VLM (A3VLM), with a
representation that describes the object’s articulation structure and action
affordance simultaneously. Compared to previous robot-centric action
representations [3, 21], A3VLM’s representation is object-centric, which makes
it possible to learn actionable models of objects without collecting expensive
robot interaction data, and the same learned object models can be used by
various robots.

Given a single RGB image of an unknown object and a language task description,
A3VLM locates an actionable part of the object and provides necessary
articulation information for a manipulation action. Results on the PartNet-
Mobility simulation benchmark show that our proposed A3VLM outperforms
previous related models by a large margin. Extensive real-world experiments
also demonstrate A3VLM’s excellent robustness and potential for real-world
robot manipulation applications.

##  2 Related Work

Manipulation of Articulated Objects. An articulated object refers to an object
composed of multiple rigid parts connected by movable joints, such as a drawer
or dishwasher. Articulated object manipulation is an important topic in
robotics. A common practice for articulated object manipulation is to
determine its articulation structure first and then manipulate it with
predefined action primitives. Some works [15, 20, 33] use part-based pose
estimation methods to locate different articulation structures. Other methods,
such as FlowBot3D [10], predict the per-point articulation movement of every
point on the object’s 3D point cloud. A subsequent technique, FlowBot++ [38],
predicts for each point an articulation parameter instead of a movement.
Although effective, these methods only perform articulation estimation.
GaPartNet [13] shares similar insights with our work by combining articulation
and affordance. Object links in GaPartNet are classified into nine different
articulation prototypes, each with a different canonical pose and affordance.
Link poses and affordances are detected from a point cloud. Inspired by
GaPartNet, A3VLM also predicts the articulation of each part of the object, as
well as its corresponding action prototype. A3VLM simplifies GaPartNet’s nine
prototypes to only two types, prismatic and revolute. Lastly, most existing
articulated object detection and manipulation methods [10, 38, 13] are based
on 3D point clouds. 3D point-cloud data in real-world environments can be
noisy and inaccurate due to reflections and transparency, as illustrated in
Fig. 6. With the support of a strong VLM backbone, A3VLM is able to predict 3D
articulation structures directly from a single RGB image, without any depth
data.

LLMs/VLMs for Manipulation. Existing LLMs/VLMs for manipulation can be divided
into three main categories. The first type, such as Code-as-Policies [22],
Instruct2Act [16], SayCan [5], and others, uses LLMs/VLMs to generate high-
level semantic action plans in pure text or code. The system then relies on
external low-level action models or hand-crafted APIs to execute these plans.
This approach heavily depends on the implementation of the low-level skills
and APIs, and it is primarily limited to simple tasks such as pick-and-place.
In contrast, methods like RT-1 [3], RT-2 [3], and ManipLLM [21] aim to
generate robot actions directly, which enables them to handle more complex
manipulation tasks, such as opening drawers and closing doors. However,
action-based LLMs/VLMs typically require a substantial amount of robot-
environment interaction data, which can be costly to collect in the real
world. Additionally, since this data is often collected from specific types of
robots, it cannot be directly used for different robots. The third main
category involves using LLMs/VLMs to generate intermediate representations,
such as cost maps (VoxPoser [18]), action constraints (MOKA [25]), or
affordances (ManipVQA [17]), which are then translated into robot actions
using simple action primitives or controls. Our A3VLM falls into this third
category. Unlike previous methods, A3VLM focuses on the articulation structure
of objects, which enables complex manipulation actions. To the best of our
knowledge, A3VLM is the first VLM capable of accurately and consistently
locating and understanding the articulation structure for robot manipulation.

##  3 Method

###  3.1 Proposed Articulation Representation

Figure 2: Articulation Representation in A3VLM

Unlike the robot-centric action representation in RT-1 [4], RT-2 [3] and
ManipLLM [21], A3VLM uses an object-centric representation that focuses on the
articulations and affordances of the movable parts within an object. Compared
to a single articulation detection pipeline such as FlowBot++ [38], A3VLM can
predict the affordance of each part of the object and locate the appropriate
part to manipulate based on the desired task. GapartNet [13] has a similar
representation. However, GapartNet uses nine different types of articulation
prototypes, whereas we unify articulation structures into two basic types:
prismatic articulation and revolute articulation.

Actionable parts, affordances, and articulation structures in A3VLM are
represented as a triad: (Bounding box $\mathcal{B}$, Axis $\mathcal{A}$,
Semantic label $\mathcal{S}$). Bounding box $\mathcal{B}$ locates an
actionable part of interest in the given image. Axis $\mathcal{A}$ represents
the articulation structure of the part. Semantic label $\mathcal{S}$ refers to
the articulation type (prismatic or revolute), the link name and the action
type. Examples of this representation are shown in Fig. 2.

In practice, the 3D bounding box $\mathcal{B}$ is represented by its eight
vertices $\\{(x_{i},y_{i},z_{i})\\}_{i=1,\ldots,8}$. Here, $(x_{i},y_{i})$ is
the 2D projected position of vertex $i$ in the given image’s plane, and
$z_{i}$ is normalized to the range (0,1) using the maximum and minimum depth
values. Axis $\mathcal{A}$ is represented using two edge points
$\\{(\alpha_{i},\beta_{i},\gamma_{i})\\}_{i=1,2}$ in the 3D space, and it is
normalized using the same method used for the bounding box $\mathcal{B}$.

Figure 3: Annotations used for training A3VLM on the PartNet-Mobility dataset.

###  3.2 Instruction-following Dataset Construction

Capabilities | Tasks | Examples of Task Templates | Num.
---|---|---|---
Partial Object Understand. | Detection |  |  User: Detect all manipulable object parts and provide their 3D bounding boxes.
---
A3VLM: There is one manipulable object parts with their 3d bounding boxes:
object name and its BBox $\mathbf{\mathcal{B}}$.
43K
Partial Object Understand. | REC-Link |  |  User: Please provide the 3D bounding box of the region this sentence describes: lid.
---
A3VLM: BBox $\mathbf{\mathcal{B}}$.
178K
Articulation Understand. | REG-Joint |  |  User: Please provide the joint’s type and its 3D axis linked to the object part: BBox $\mathbf{\mathcal{B}}$ or Link Name $\mathbf{\mathcal{S}}$.
---
A3VLM: Joint type $\mathbf{\mathcal{S}}$ and its Axis $\mathbf{\mathcal{A}}$.
18K
Action Grounding | REC-Action |  |  User: Please execute the task described with 3D rotated bounding box representations by the following instruction: Open the storage.
---
A3VLM: Action type $\mathbf{\mathcal{S}}$ and targeted object’s BBox
$\mathbf{\mathcal{B}}$.
15K

Table 1: Overview of the instruction-following dataset, including task
templates, associated capabilities, examples, and sample counts.

As training a VLM requires colossal resources in terms of data and
computation, we do not train A3VLM from scratch. Instead, we fine-tune an
established VLM. To fine-tune a VLM, we need to build an instruction-following
dataset, where the input is an image and a text prompt, and the answer should
be structured text. In this section, we will discuss how to construct this
instruction-following dataset. As we mentioned in Sec. 3.1, we use a triad
$(\mathcal{B},\mathcal{A},\mathcal{S})$ to describe the location, articulation
structure, and action affordance of a movable part. In practice, we do not ask
the VLM to generate everything in one inference step but separate the tasks
into four different types of sub-tasks. This separation allows the VLM to
focus on one concept during each inference.

Raw A3 Annotation Generation. The first step in generating an instruction-
following dataset is to create object-level raw annotations. We use PartNet-
Mobility, which provides more than 2,000 different articulated objects across
46 categories in URDF format. First, we render these objects into RGB images
using PyRender, incorporating random camera positions, lighting, and joint
values to generate 40 different images for each object. For each image, we use
ControlNet [39] to generate augmented images for data augmentation (see Sec.
3.3 for more details on data augmentation).

Within each image, we provide an annotation
$(\mathcal{B},\mathcal{A},\mathcal{S})$ for each visible and movable link. We
categorize all links into prismatic and revolute types. For revolute links,
axis $\mathcal{A}$ is the rotation axis provided in the URDF. For prismatic
links, we use the prismatic direction provided in the URDF as the axis
direction, ensuring that axis $\mathcal{A}$ passes through the 3D center of
the link. After determining axis $\mathcal{A}$, we project the link points
along $\mathcal{A}$ and compute a minimal 2D bounding box for the projected
shape. We use the longer edge of this bounding box as the x-axis, the shorter
edge as the y-axis, and axis $\mathcal{A}$ as the z-axis for bounding box
$\mathcal{B}$. The center of bounding box $\mathcal{B}$ is the 3D center of
the link. The width, height, and length of bounding box $\mathcal{B}$ are
computed based on the distance between the furthest points of the link and the
center. Semantic information $S$ stores the articulation type, name, and
affordable action of the link. Fig. 3 shows multiple annotation examples from
different categories in the PartNet-Mobility dataset.

Noticeably, the affordable actions of the links in the PartNet-Mobility
dataset are not provided. We therefore select the actions from a robotic skill
library as defined in [2]. To ensure that the selected skills are compatible
with the corresponding links of the objects, we use GPT-4 for skill selection.
The prompt used during this process can be found in Appendix A.

Sub-tasks Construction. To fit into the established VLM training pipeline, we
follow the widely used VLM task templates: Referring Expression Comprehension
(REC) and Referring Expression Generation (REG). In a nutshell, REC requires
the VLM to provide a bounding box according to a text description, while REG
asks the VLM to provide a description of a region within an image referred to
by a bounding box. The definitions of REG and REC can be found in [9].

Following these definitions, we construct four different sub-tasks: (1)
Detection, (2) REC-Link, (3) REG-Joint, and (4) REC-Action. Each sub-task
consists of an image, a text question, and a text answer. Examples of each
sub-task question can be found in Tab. 1. The detection task requires the VLM
to locate all manipulable parts within an image by outputting a bounding box
$\mathcal{B}$ for each part. Since this task generates more than one bounding
box, it does not follow the traditional definition of REC tasks. The REC-Link
task involves locating a part/link based on a description. The REG-Joint task
asks the VLM to provide the joint axis $\mathcal{A}$ and the joint type
$\mathcal{S}$ for a part specified by a bounding box $\mathcal{B}$. REC-Action
requires the VLM to locate a movable part by providing a bounding box
$\mathcal{B}$ and corresponding action type $\mathcal{S}$ according to an
action task, such as “Open the storage.”

###  3.3 Data Augmentation Strategy

A limitation of the original PartNet-Mobility dataset is the absence of
texture details. As depicted in Figure 3, most objects are rendered in plain
gray, which does not reflect real-world conditions. To address the simulation-
to-reality (Sim2Real) gap, we employed ControlNet [39] to generate more
realistic images, using depth maps as the primary control signal due to their
ability to convey both geometric and semantic information. For objects with
minimal depth variance, we utilized semantic segmentation as the control
signal. To enhance the diversity of the generated images using Stable
Diffusion, we employed ChatGPT to generate a broader range of detailed
descriptions, enriching the contextual input necessary for producing varied
visual outputs. Specific prompts used with ChatGPT and examples of the
augmented data are detailed in Appendix E.

###  3.4 Model and Training

Figure 4: The A3VLM architecture.

Model Architecture. A3VLM is developed based on the SPHINX-X [12] with LLaMA2
serving as the language backbone. We select this model as it is uniquely
tailored to focus on the partial or regional details of target objects,
necessitating fine-grained visual analysis. Our architecture is shown in Fig.
4. Following the SPHINX “any resolution” approach [24, 12], the input image is
first partitioned into sub-images and then visual encoders are applied to
extract visual features. Moreover, given the necessity for both global and
local visual grounding ability in manipulation tasks, we integrate the visual
encoder from CLIP [32], DINOv2 [29] to extract local semantic features, and
Q-Former [1] for the global visual features summarization. Then, local and
global features are channel-wise concatenated. The spatial alignment between
visual tokens and language tokens is achieved with projection layers. Bounding
box values are normalized to the range $(0,1)$ and expressed with precision up
to two decimal places.

Fine-tuning Strategy. As discussed in Section 3.2, our training paradigm
follows the conventional Visual Question Answering (VQA) framework and
encapsulates all information about articulations within a natural language
framework. As a result, the training objective only employs the cross-entropy
loss, which is a departure from previous works [21, 31]. To bridge the visual
disparity between our specialized dataset and generic natural imagery, we
employ a two-stage fine-tuning strategy. Initially, the visual projection
layers are fine-tuned using straightforward image caption tasks, utilizing a
basic template such as “This is a [OBJ]” to generate naive captions. Then, we
fine-tune the visual projection layers and LLM simultaneously on the
instruction-following dataset.

###  3.5 Action Primitives

Figure 5: Action primitives for sliding, rotating and scrolling

As previously mentioned, A3VLM utilizes an object-centric representation. To
translate this into a robot movement, we need to define specific action
primitives. A3VLM is designed for use with various types of robots; therefore,
it is not optimized for any particular type of manipulator, such as a parallel
or a suction gripper. An independent generic grasp-pose proposer is required
to generate a list of grasp-pose candidates. During manipulation, we utilize
the triad $(\mathcal{B},\mathcal{A},\mathcal{S})$ generated by A3VLM along
with the grasp pose candidates.

We define three types of action primitives: Rotate, Slide, and Scroll. For a
given link, if its corresponding joint type is prismatic, we select the slide
action; if it is revolute, we choose the rotate action, unless the target link
is semantically labeled as a bottle cap or scroll button, in which case we opt
for the scroll action. If the action selected is “scroll”, we ensure that the
grasp pose overlaps with the rotation axis $\mathcal{A}$. Otherwise, we
randomly select a grasp pose within the bounding box $\mathcal{B}$ to serve as
the contact point $\mathcal{C}$. We then generate a trajectory using
$\mathcal{C}$ and $\mathcal{A}$ for each action type, as illustrated in Fig 5.
These trajectories constitute our generated actions.

##  4 Experiment

###  4.1 Implementation Details

We fine-tuned the A3VLM model using the SPHINX framework [12] on eight NVIDIA
A100 (80 GB) GPUs. The fine-tuning was completed in three epochs, which took
approximately 24 hours. The visual encoders were kept frozen throughout the
fine-tuning phase to maintain the integrity of the pre-trained features. We
utilized the SPHINX-1K model, sourced directly from the official repository,
as our pre-trained base. Training was conducted with a batch size of 4 and a
learning rate set to $2\times 10^{-5}$.

###  4.2 Qualitative Evaluation

To evaluate the action capabilities of A3VLM, we modified upon the settings of
ManipLLM [21]. This benchmark utilizes Sapien [34] as the simulator and
PartNet-Mobility [28] objects as the target objects. Object’s joint values
will be initialized to the middle value. We use a flying Franka Panda Robot’s
gripper with suction ability as the manipulator.

Evaluation Metric. The goal of this benchmark is to evaluate the model’s
ability to interact with the environment. In each task, we will load an
articulated object from PartNet-Mobility dataset. Then, we will drive the
manipulator to interact with the articulated object. The base of the object is
fixed, so the robot needs to locate a movable part and understand the correct
direction to drive the part. Regarding the definition of successful task, we
follow the definition in ManipLLM, where we measure the movement of a part in
the articulated objects as $d$. If, after manipulation, the part’s movement
$d$ exceeds a threshold $\sigma$, we define it as a success. We use the same
threshold as ManipLLM, where $\sigma=0.01$222ManipLLM uses two thresholds
$\sigma=0.01$ and $\sigma=0.1$. But their main evaluation is performed under
$\sigma=0.01$..

Baselines. The robot manipulation success rates are compared for 20 training
categories (used during training) and 10 testing categories (excluded from
training). We compare our proposed method with five different baselines: (1).
Where2Act [27]: This method processes input from point-clouds to evaluate
scores for each point. The highest-scoring point is chosen as the contact
point. Additionally, it predicts 100 possible orientations for the end-
effector, selecting the orientation with the top score to establish the
contact pose. To ensure a fair comparison, we modified the originally used
parallel gripper into a suction gripper. (2). UMPNet [36]: In alignment with
UMPNet’s methodology, we carry out manipulations at the predicted contact
point, positioning the end-effector’s orientation perpendicular to the surface
of the object. (3). Flowbot3D [10]: This approach identifies the motion
direction within a point cloud, referred to as “flow”. The point demonstrating
the greatest flow magnitude is selected as the point of interaction, and the
flow’s direction dictates the orientation of the end-effector. (4). Implicit3D
[40]: This framework formulates a manipulation strategy for subsequent tasks
by employing the Transporter network to identify keypoints on 3D articulated
objects. These keypoints facilitate the determination of the end-effector’s
pose. (5). ManipLLM [21]: ManipLLM is the current state-of-the-art method on
this benchmark. It uses a vision language model to predict the contact point
and forward direction of a suction gripper given an RGBD image and a language
prompt.

Action Primitives Details. Different from the mentioned baselines, A3VLM
models the action in an object-centric way. To be more specific, for each
object, we first detect a list of action parts with corresponding bounding
boxes $\mathcal{B}$, axes $\mathcal{A}$, joint types, and link names
$\mathcal{S}$. We select a random action part from the list and use its
bounding box $\mathcal{B}$ and axis $\mathcal{A}$ to generate two robot
trajectories. For example, for the handle of a faucet, we will generate
trajectories to rotate it clockwise and counter-clockwise. We will execute
these trajectories in two attempts. The task is regarded as successful if it
succeeds in either try.

Result. Table. 2 shows the performance of A3VLM and five other baselines. From
the table, we can see that A3VLM outperforms all baselines by a large margin
for most object categories. We think this improvement comes from two folds.
One fold comes from the accurate grounding of actionable parts and
articulation structure and the other fold is the introduce of action
primitives. Action primitives enable A3VLM to perform different actions
towards different articulated objects. This result demonstrates the
effectiveness of A3VLM for manipulating articulated objects.

Method |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Where2Act | 0.26 | 0.36 | 0.19 | 0.27 | 0.23 | 0.11 | 0.15 | 0.47 | 0.14 | 0.24 | 0.13 | 0.12 | 0.56 | 0.68 | 0.07 | 0.40
UMPNet | 0.46 | 0.43 | 0.15 | 0.28 | 0.54 | 0.32 | 0.28 | 0.56 | 0.44 | 0.40 | 0.10 | 0.23 | 0.18 | 0.54 | 0.20 | 0.42
FlowBot3D | 0.67 | 0.55 | 0.20 | 0.32 | 0.27 | 0.31 | 0.61 | 0.68 | 0.15 | 0.28 | 0.36 | 0.18 | 0.21 | 0.70 | 0.18 | 0.26
Implicit3D | 0.53 | 0.58 | 0.35 | 0.55 | 0.28 | 0.66 | 0.58 | 0.51 | 0.52 | 0.57 | 0.45 | 0.34 | 0.41 | 0.54 | 0.39 | 0.43
ManipLLM | 0.68 | 0.64 | 0.36 | 0.77 | 0.43 | 0.62 | 0.65 | 0.61 | 0.65 | 0.52 | 0.53 | 0.40 | 0.64 | 0.71 | 0.60 | 0.64
Ours | 0.90 | 0.82 | 0.94 | 0.90 | 0.49 | 0.70 | 0.87 | 0.35 | 0.86 | 0.79 | 1.00 | 0.70 | 0.83 | 0.97 | 0.34 | 0.40

Method |  |  |  |  | AVG |  |  |  |  |  |  |  |  |  |  | AVG
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Where2Act | 0.13 | 0.18 | 0.13 | 0.40 | 0.26 | 0.18 | 0.35 | 0.38 | 0.28 | 0.05 | 0.21 | 0.17 | 0.20 | 0.15 | 0.15 | 0.21
UMPNet | 0.22 | 0.33 | 0.26 | 0.64 | 0.35 | 0.42 | 0.20 | 0.35 | 0.42 | 0.29 | 0.20 | 0.26 | 0.28 | 0.25 | 0.15 | 0.28
FlowBot3D | 0.17 | 0.53 | 0.29 | 0.42 | 0.37 | 0.23 | 0.10 | 0.60 | 0.39 | 0.27 | 0.42 | 0.28 | 0.51 | 0.13 | 0.23 | 0.32
Implicit3D | 0.27 | 0.65 | 0.20 | 0.33 | 0.46 | 0.45 | 0.17 | 0.80 | 0.53 | 0.15 | 0.69 | 0.41 | 0.31 | 0.30 | 0.31 | 0.41
ManipLLM | 0.41 | 0.75 | 0.44 | 0.67 | 0.59 | 0.38 | 0.22 | 0.81 | 0.86 | 0.38 | 0.85 | 0.42 | 0.83 | 0.26 | 0.38 | 0.54
Ours | 0.62 | 0.50 | 0.90 | 0.73 | 0.91 | 0.88 | 0.76 | 0.74 | 0.86 | 0.79 | 0.67 | 0.96 | 0.50 | 0.62 | 0.72 | 0.76

Table 2: Comparisons of our method against baseline methods on PartNet-
Mobility. The first 20 categories are training categories and the following 10
categories are testing categories.

###  4.3 Real-World Application

Real-World Inference Test. To test the stability and accuracy of A3VLM in the
real world, we perform an inference test for A3VLM on many objects across
different categories. It turns out A3VLM is able to correctly detect the
movable parts of the objects and accurately recognize the articulation
structure of those parts. As shown in Fig. 6, A3VLM consistently generates
correct inferences on 20 different real-world objects. It is worth-noting that
A3VLM is able to correctly perform inference on objects with reflecting or
transparent surface (e.g. microwave oven, pot and coke bottle), which are very
challenging for point-cloud-based methods due to their inaccurate depth.

Figure 6: A3VLM predictions on various real-world objects using a single RGB image. Table 3: Manipulation success rates on the real-world objects Object |  |  |  |  |
---|---|---|---|---|---
Success | 5/5 | 5/5 | 5/5 | 5/5 | 4/5

Real-World Robot Manipulation. To test the effectiveness of the manipulation
ability of A3VLM, we selected five different objects from the aforementioned
twenty tested categories. We used a Kuka robot equipped with a RealSense D415
depth camera and a Robotiq three-finger gripper. We followed the action
primitive mentioned in Sec. 3.5. Five trials with different initial positions
are performed for each object. Manipulation is considered successful if the
manipulated part has moved a sufficient distance. Success rate results can be
found in Tab. 3, and experiment start and end states are shown in Fig. 7. As
shown in Tab. 3 and Fig. 7, A3VLM successfully manipulated the articulated
objects in the real world with simple action primitives. Further
implementation details on the robot experiments can be found in Appendix B.

Figure 7: Real-world manipulation experiments using a Kuka robot with a
Robotiq hand.

##  5 Conclusion

We present the Actionable Articulation-Aware Vision Language Model (A3VLM), an
object-centric robot VLM designed to understand the articulation and action
affordances of articulated objects. Unlike previous action-centric robot VLMs,
A3VLM does not require any robot interaction data and can be adapted to
various robot configurations. Although it is trained solely on simulated data,
A3VLM demonstrates significant manipulation capabilities and inference
stability across both simulation benchmarks and real-world robot experiments,
establishing a new state of the art in this area. We believe that A3VLM
represents a promising direction for future research in robot VLMs.

##  Appendix A Prompt Engineering for action affordance generation

⬇

Role and Task Description:

Develop a systematic approach for generating grounding

tasks involving object links, where each task involves

a limited number of steps and utilizes predefined

action primitives from a robot skill library.

Robot Skill Library::

Actions available include:

"slide_open", "slide_close", "flap_open", "flap_close", "cap", "uncap",
"pick", "place", "slide_in", "slide_out", "wipe", "press", "rotate",
"StatusComplete".

Requirements and Constraints:

1\. Tasks and actions must be tailored based on the

current status of the link .

2\. Links may have different joint types: prismatic,

revolute, static, etc.

3\. All actions must be sourced exclusively from the

provided skill library.

4\. Provide the list of tasks and corresponding

actions in JSON format.

5\. The generated actions should vary in sequence

length, order, and semantics.

6\. Create tasks involving both single and multiple

links where applicable.

7\. Do not assume or add components not

explicitly specified.

Examples:

examples ...

avilible links information ...

Instruction:

Now please generate the tasks and actions for the

OBJECT_CLASS’s link part with the links

LINK_INFO.

You have generated tasks and actions in the previous as

following HISTORY_GENERATION, please make

sure the tasks and actions are different from the‘

previous ones.

Please ONLY generate the tasks and actions in the valid

json format.

Listing 1: An example prompt for guiding ChatGPT to generate the action
grounding task for a specific object.

##  Appendix B Real-world Experiments Details

Since A3VLM is trained on purely simulated data, it is necessary to address
the discrepancies between simulation and real-world data. In simulations, the
image resolution is 960x960, the focal length is 1000 pixels, and the rendered
object images lack backgrounds. To align real-world images with those of the
simulation, we first tuned the real-world camera’s intrinsics to match those
of the simulation using a homography transformation. Then, we used the
Segment-Anything-Model (SAM) [19] to segment out the background.

It is worth noting that the output z-value of the 3D bounding box is not an
absolute value but a normalized one. To denormalize it, we need to know the
minimal and maximal depths of the scene. For this purpose, we use the depth
images from the depth camera. Although real-world depth images are full of
noise and inaccuracies, we can still use the minimal and maximal depth values,
as the relative error is small.

Figure 8: Robot setup: we use a Kuka robot with a three-fingers Robotiq hand.
A realsense d415 camera is placed at the side to capture RGBD images. The
depth camera on the table is not used for this experiment.

Regarding the robot experiments, the robot setup is illustrated in Fig. 8. We
translate A3VLM’s output into actions using the primitives described in
Section 3.5. For generating grasp pose candidates, we can employ tools like
the Grasp Pose Generator (GPG) [14], GraspNet [11], or define them manually.
Since this is not the primary focus of our work, we have simplified the
process by manually providing a list of grasp poses.

##  Appendix C Discussion on Input Modality

A3VLM takes RGB images as input. Compared with other methods that rely on
depth images or point clouds, RGB suffers less noise in real-world
experiments. However, as A3VLM is learning a 3D bounding box, a natural
concern is whether pure RGB input is less accurate compared with depth input.
To explore this problem, we trained a new version of A3VLM, i.e. A3VLM-depth,
using depth images as input. To adapt the depth images for the A3VLM pipeline,
we first normalized the depth values to the range [0, 1] and then converted
them to RGB values.

We evaluated the performance of A3VLM and A3VLM-depth on the PartNet-Mobility
simulation benchmark as described in Section 4.2. The performance is shown in
Tab. 4. From the results, we can see that A3VLM and A3VLM-depth perform
similarly on the training categories. However, in the testing categories,
A3VLM shows a significant improvement, indicating that the pure RGB input is
actually better for generalization. This ablation study confirms our initial
hypothesis of using pure RGB as the input modality.

Method |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
A3VLM | 0.90 | 0.82 | 0.94 | 0.90 | 0.49 | 0.70 | 0.87 | 0.35 | 0.86 | 0.79 | 1.00 | 0.70 | 0.83 | 0.97 | 0.34 | 0.40
A3VLM-depth | 0.92 | 0.89 | 0.93 | 1.00 | 0.35 | 0.56 | 1.00 | 0.63 | 0.72 | 0.87 | 1.00 | 0.67 | 0.80 | 0.95 | 0.29 | 0.36

Method |  |  |  |  | AVG |  |  |  |  |  |  |  |  |  |  | AVG
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
A3VLM | 0.62 | 0.50 | 0.90 | 0.73 | 0.91 | 0.88 | 0.76 | 0.74 | 0.86 | 0.79 | 0.67 | 0.96 | 0.50 | 0.62 | 0.72 | 0.76
A3VLM-depth | 0.83 | 0.65 | 0.85 | 0.76 | 0.90 | 0.82 | 0.77 | 0.68 | 0.73 | 0.79 | 0.48 | 0.94 | 0.67 | 0.57 | 0.50 | 0.70

Table 4: Comparisons of A3VLM against A3VLM-depth. The first 20 categories are
training categories and the following 10 categories are testing categories.

##  Appendix D Exploration on More Input Modalities

In addition to the modalities discussed in Section C, we also explored
expanding our experiments to include RGB-D and point cloud modalities.

###  D.1 RGB-D Modality

For the RGB-D experiments, RGB and depth images were processed separately
using distinct visual encoders. The outputs, termed visual tokens, were
appended sequentially—with special tokens $<img>$ and $<depth>$ indicating
their respective modalities—and then fed into the Large Language Model (LLM).
However, the LLM struggled with instruction following, likely due to two main
challenges:

  * •

Domain Gap: The visual foundation models, originally pre-trained only on
natural RGB images, fail to reliably extract features from depth images which
lack visual textures.

  * •

Token Length: The combination of tokens from both modalities resulted in
excessively long input sequences, which the LLM could not process effectively
due to its limitations in handling long sequences.

###  D.2 Point Cloud Modality

For point cloud inputs, we utilized PointBert [37] and RECON [30] as the point
encoders, following practices established in ShapeLLM [30] and PointLLM [35].
The point features were aligned with language features using the Cap3D [26]
dataset during fine-tuning of the projection layers. Despite successful
training where the Modified Large Language Model (MLLM) produced high-quality
captions, the LLM failed to predict partial object bounding box coordinates.
This difficulty is attributed to:

  * •

Lack of Visual Texture: Point clouds inherently contain fewer visual texture
features, complicating the task of partial-level object detection.

  * •

Model and Data Limitations: The point cloud models, having fewer parameters
and a smaller training dataset compared to visual foundation models, exhibit
weaker performance capabilities.

##  Appendix E Data Augmentation Examples

In this section, we displayed more data augmentation examples in Figure 9. The
first row and third rows display the raw images rendered directly from the
PartNet Mobility, while the second row and the last row display the generated
ones. And we used the prompts in the List. LABEL:lst:gpt_prompt_for_sd to
guide the ChatGPT to generate more diverse texture descriptions for the target
object.

Figure 9: Comparison between raw images with the Stable Diffusion generated
ones.

⬇

Role and Task Description:

You are a good assistant, skilled in providing accurate

prompts for stable diffusion.

I want to use stable diffusion to draw a category,

please give me ten prompts with different styles.

Note that the target object is the daily-use item.

I already have the descriptions like

previous_description, please avoid repeats.

Only give me the newly generated prompts in a list, and nothing else.

Listing 2: An example prompt for guiding ChatGPT to generate diverse texture
description for a specific object.

## References

  * [1] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al.  Flamingo: a visual language model for few-shot learning.  Advances in Neural Information Processing Systems, 35:23716–23736, 2022.
  * [2] Homanga Bharadhwaj, Jay Vakil, Mohit Sharma, Abhinav Gupta, Shubham Tulsiani, and Vikash Kumar.  Roboagent: Generalization and efficiency in robot manipulation via semantic augmentations and action chunking.  arXiv preprint arXiv:2309.01918, 2023.
  * [3] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al.  Rt-2: Vision-language-action models transfer web knowledge to robotic control.  arXiv preprint arXiv:2307.15818, 2023.
  * [4] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al.  Rt-1: Robotics transformer for real-world control at scale.  arXiv preprint arXiv:2212.06817, 2022.
  * [5] Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, et al.  Do as i can, not as i say: Grounding language in robotic affordances.  In Conference on Robot Learning, pages 287–318. PMLR, 2023.
  * [6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [7] Haonan Chang, Kowndinya Boyalakuntla, Shiyang Lu, Siwei Cai, Eric Jing, Shreesh Keskar, Shijie Geng, Adeeb Abbas, Lifeng Zhou, Kostas Bekris, et al.  Context-aware entity grounding with open-vocabulary 3d scene graphs.  arXiv preprint arXiv:2309.15940, 2023.
  * [8] Haonan Chang, Kai Gao, Kowndinya Boyalakuntla, Alex Lee, Baichuan Huang, Harish Udhaya Kumar, Jinjin Yu, and Abdeslam Boularias.  Lgmcts: Language-guided monte-carlo tree search for executable semantic object rearrangement.  arXiv preprint arXiv:2309.15821, 2023.
  * [9] Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao.  Shikra: Unleashing multimodal llm’s referential dialogue magic.  arXiv preprint arXiv:2306.15195, 2023.
  * [10] Ben Eisner, Harry Zhang, and David Held.  Flowbot3d: Learning 3d articulation flow to manipulate articulated objects.  arXiv preprint arXiv:2205.04382, 2022.
  * [11] Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu.  Graspnet-1billion: A large-scale benchmark for general object grasping.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11444–11453, 2020.
  * [12] Peng Gao, Renrui Zhang, Chris Liu, Longtian Qiu, Siyuan Huang, Weifeng Lin, Shitian Zhao, Shijie Geng, Ziyi Lin, Peng Jin, et al.  Sphinx-x: Scaling data and parameters for a family of multi-modal large language models.  arXiv preprint arXiv:2402.05935, 2024.
  * [13] Haoran Geng, Helin Xu, Chengyang Zhao, Chao Xu, Li Yi, Siyuan Huang, and He Wang.  Gapartnet: Cross-category domain-generalizable object perception and manipulation via generalizable and actionable parts.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7081–7091, 2023.
  * [14] Marcus Gualtieri, Andreas ten Pas, Kate Saenko, and Robert Platt.  High precision grasp pose detection in dense clutter, 2016.
  * [15] Ruizhen Hu, Wenchao Li, Oliver Van Kaick, Ariel Shamir, Hao Zhang, and Hui Huang.  Learning to predict part mobility from a single static snapshot.  ACM Trans. Graph., 36(6), nov 2017.
  * [16] Siyuan Huang, Zhengkai Jiang, Hao Dong, Yu Qiao, Peng Gao, and Hongsheng Li.  Instruct2act: Mapping multi-modality instructions to robotic actions with large language model.  arXiv preprint arXiv:2305.11176, 2023.
  * [17] Siyuan Huang, Iaroslav Ponomarenko, Zhengkai Jiang, Xiaoqi Li, Xiaobin Hu, Peng Gao, Hongsheng Li, and Hao Dong.  Manipvqa: Injecting robotic affordance and physically grounded information into multi-modal large language models.  arXiv preprint arXiv:2403.11289, 2024.
  * [18] Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei.  Voxposer: Composable 3d value maps for robotic manipulation with language models.  arXiv preprint arXiv:2307.05973, 2023.
  * [19] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.  Segment anything.  arXiv preprint arXiv:2304.02643, 2023.
  * [20] Xiaolong Li, He Wang, Li Yi, Leonidas J. Guibas, A. Lynn Abbott, and Shuran Song.  Category-level articulated object pose estimation.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2020.
  * [21] Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yuxing Long, Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong.  Manipllm: Embodied multimodal large language model for object-centric robotic manipulation.  arXiv preprint arXiv:2312.16217, 2023.
  * [22] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng.  Code as policies: Language model programs for embodied control.  arXiv preprint arXiv:2209.07753, 2022.
  * [23] Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng.  Code as policies: Language model programs for embodied control.  In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 9493–9500. IEEE, 2023.
  * [24] Ziyi Lin, Chris Liu, Renrui Zhang, Peng Gao, Longtian Qiu, Han Xiao, Han Qiu, Chen Lin, Wenqi Shao, Keqin Chen, et al.  Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models.  arXiv preprint arXiv:2311.07575, 2023.
  * [25] Fangchen Liu, Kuan Fang, Pieter Abbeel, and Sergey Levine.  Moka: Open-vocabulary robotic manipulation through mark-based visual prompting, 2024.
  * [26] Tiange Luo, Chris Rockwell, Honglak Lee, and Justin Johnson.  Scalable 3d captioning with pretrained models.  Advances in Neural Information Processing Systems, 36, 2024.
  * [27] Kaichun Mo, Leonidas J Guibas, Mustafa Mukadam, Abhinav Gupta, and Shubham Tulsiani.  Where2act: From pixels to actions for articulated 3d objects.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6813–6823, 2021.
  * [28] Kaichun Mo, Shilin Zhu, Angel X Chang, Li Yi, Subarna Tripathi, Leonidas J Guibas, and Hao Su.  Partnet: A large-scale benchmark for fine-grained and hierarchical part-level 3d object understanding.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 909–918, 2019.
  * [29] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.  Dinov2: Learning robust visual features without supervision.  arXiv preprint arXiv:2304.07193, 2023.
  * [30] Zekun Qi, Runpei Dong, Shaochen Zhang, Haoran Geng, Chunrui Han, Zheng Ge, Li Yi, and Kaisheng Ma.  Shapellm: Universal 3d object understanding for embodied interaction.  arXiv preprint arXiv:2402.17766, 2024.
  * [31] Shengyi Qian, Weifeng Chen, Min Bai, Xiong Zhou, Zhuowen Tu, and Li Erran Li.  Affordancellm: Grounding affordance from vision language models.  arXiv preprint arXiv:2401.06341, 2024.
  * [32] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  * [33] Xiaogang Wang, Bin Zhou, Yahao Shi, Xiaowu Chen, Qinping Zhao, and Kai Xu.  Shape2motion: Joint analysis of motion parts and attributes from 3d shapes.  In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2019.
  * [34] Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al.  Sapien: A simulated part-based interactive environment.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11097–11107, 2020.
  * [35] Runsen Xu, Xiaolong Wang, Tai Wang, Yilun Chen, Jiangmiao Pang, and Dahua Lin.  Pointllm: Empowering large language models to understand point clouds.  arXiv preprint arXiv:2308.16911, 2023.
  * [36] Zhenjia Xu, Zhanpeng He, and Shuran Song.  Universal manipulation policy network for articulated objects.  IEEE robotics and automation letters, 7(2):2447–2454, 2022.
  * [37] Xumin Yu, Lulu Tang, Yongming Rao, Tiejun Huang, Jie Zhou, and Jiwen Lu.  Point-bert: Pre-training 3d point cloud transformers with masked point modeling.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 19313–19322, 2022.
  * [38] Harry Zhang, Ben Eisner, and David Held.  Flowbot++: Learning generalized articulated objects manipulation via articulation projection.  arXiv preprint arXiv:2306.12893, 2023.
  * [39] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.  Adding conditional control to text-to-image diffusion models.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023.
  * [40] Chengliang Zhong, Yuhang Zheng, Yupeng Zheng, Hao Zhao, Li Yi, Xiaodong Mu, Ling Wang, Pengfei Li, Guyue Zhou, Chao Yang, et al.  3d implicit transporter for temporally consistent keypoint discovery.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3869–3880, 2023.


The Belle Collaboration

# Measurement of $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ with a
semileptonic tagging method

G. Caria  School of Physics, University of Melbourne, Victoria 3010  P.
Urquijo  School of Physics, University of Melbourne, Victoria 3010  I. Adachi
High Energy Accelerator Research Organization (KEK), Tsukuba 305-0801
SOKENDAI (The Graduate University for Advanced Studies), Hayama 240-0193  H.
Aihara  Department of Physics, University of Tokyo, Tokyo 113-0033  S. Al Said
Department of Physics, Faculty of Science, University of Tabuk, Tabuk 71451
Department of Physics, Faculty of Science, King Abdulaziz University, Jeddah
21589  D. M. Asner  Brookhaven National Laboratory, Upton, New York 11973  H.
Atmacan  University of South Carolina, Columbia, South Carolina 29208  T.
Aushev  Moscow Institute of Physics and Technology, Moscow Region 141700  V.
Babu  Deutsches Elektronen–Synchrotron, 22607 Hamburg  I. Badhrees  Department
of Physics, Faculty of Science, University of Tabuk, Tabuk 71451  King
Abdulaziz City for Science and Technology, Riyadh 11442  S. Bahinipati  Indian
Institute of Technology Bhubaneswar, Satya Nagar 751007  A. M. Bakich  School
of Physics, University of Sydney, New South Wales 2006  P. Behera  Indian
Institute of Technology Madras, Chennai 600036  C. Beleño  II. Physikalisches
Institut, Georg-August-Universität Göttingen, 37073 Göttingen  J. Bennett
University of Mississippi, University, Mississippi 38677  B. Bhuyan  Indian
Institute of Technology Guwahati, Assam 781039  T. Bilka  Faculty of
Mathematics and Physics, Charles University, 121 16 Prague  J. Biswal  J.
Stefan Institute, 1000 Ljubljana  A. Bozek  H. Niewodniczanski Institute of
Nuclear Physics, Krakow 31-342  M. Bračko  University of Maribor, 2000 Maribor
J. Stefan Institute, 1000 Ljubljana  T. E. Browder  University of Hawaii,
Honolulu, Hawaii 96822  M. Campajola  INFN - Sezione di Napoli, 80126 Napoli
Università di Napoli Federico II, 80055 Napoli  D. Červenkov  Faculty of
Mathematics and Physics, Charles University, 121 16 Prague  P. Chang
Department of Physics, National Taiwan University, Taipei 10617  R. Cheaib
University of Mississippi, University, Mississippi 38677  V. Chekelian  Max-
Planck-Institut für Physik, 80805 München  A. Chen  National Central
University, Chung-li 32054  B. G. Cheon  Department of Physics and Institute
of Natural Sciences, Hanyang University, Seoul 04763  K. Chilikin  P.N.
Lebedev Physical Institute of the Russian Academy of Sciences, Moscow 119991
H. E. Cho  Department of Physics and Institute of Natural Sciences, Hanyang
University, Seoul 04763  K. Cho  Korea Institute of Science and Technology
Information, Daejeon 34141  Y. Choi  Sungkyunkwan University, Suwon 16419  S.
Choudhury  Indian Institute of Technology Hyderabad, Telangana 502285  D.
Cinabro  Wayne State University, Detroit, Michigan 48202  S. Cunliffe
Deutsches Elektronen–Synchrotron, 22607 Hamburg  N. Dash  Indian Institute of
Technology Bhubaneswar, Satya Nagar 751007  G. De Nardo  INFN - Sezione di
Napoli, 80126 Napoli  Università di Napoli Federico II, 80055 Napoli  F. Di
Capua  INFN - Sezione di Napoli, 80126 Napoli  Università di Napoli Federico
II, 80055 Napoli  S. Di Carlo  LAL, Univ. Paris-Sud, CNRS/IN2P3, Université
Paris-Saclay, Orsay 91898  Z. Doležal  Faculty of Mathematics and Physics,
Charles University, 121 16 Prague  T. V. Dong  Key Laboratory of Nuclear
Physics and Ion-beam Application (MOE) and Institute of Modern Physics, Fudan
University, Shanghai 200443  S. Eidelman  Budker Institute of Nuclear Physics
SB RAS, Novosibirsk 630090  Novosibirsk State University, Novosibirsk 630090
P.N. Lebedev Physical Institute of the Russian Academy of Sciences, Moscow
119991  D. Epifanov  Budker Institute of Nuclear Physics SB RAS, Novosibirsk
630090  Novosibirsk State University, Novosibirsk 630090  J. E. Fast  Pacific
Northwest National Laboratory, Richland, Washington 99352  T. Ferber
Deutsches Elektronen–Synchrotron, 22607 Hamburg  D. Ferlewicz  School of
Physics, University of Melbourne, Victoria 3010  B. G. Fulsom  Pacific
Northwest National Laboratory, Richland, Washington 99352  R. Garg  Panjab
University, Chandigarh 160014  V. Gaur  Virginia Polytechnic Institute and
State University, Blacksburg, Virginia 24061  N. Gabyshev  Budker Institute of
Nuclear Physics SB RAS, Novosibirsk 630090  Novosibirsk State University,
Novosibirsk 630090  A. Garmash  Budker Institute of Nuclear Physics SB RAS,
Novosibirsk 630090  Novosibirsk State University, Novosibirsk 630090  A. Giri
Indian Institute of Technology Hyderabad, Telangana 502285  P. Goldenzweig
Institut für Experimentelle Teilchenphysik, Karlsruher Institut für
Technologie, 76131 Karlsruhe  D. Greenwald  Department of Physics, Technische
Universität München, 85748 Garching  O. Grzymkowska  H. Niewodniczanski
Institute of Nuclear Physics, Krakow 31-342  Y. Guan  University of
Cincinnati, Cincinnati, Ohio 45221  O. Hartbrich  University of Hawaii,
Honolulu, Hawaii 96822  K. Hayasaka  Niigata University, Niigata 950-2181  H.
Hayashii  Nara Women’s University, Nara 630-8506  T. Higuchi  Kavli Institute
for the Physics and Mathematics of the Universe (WPI), University of Tokyo,
Kashiwa 277-8583  W.-S. Hou  Department of Physics, National Taiwan
University, Taipei 10617  C.-L. Hsu  School of Physics, University of Sydney,
New South Wales 2006  T. Iijima  Kobayashi-Maskawa Institute, Nagoya
University, Nagoya 464-8602  Graduate School of Science, Nagoya University,
Nagoya 464-8602  K. Inami  Graduate School of Science, Nagoya University,
Nagoya 464-8602  G. Inguglia  Institute of High Energy Physics, Vienna 1050
A. Ishikawa  High Energy Accelerator Research Organization (KEK), Tsukuba
305-0801  SOKENDAI (The Graduate University for Advanced Studies), Hayama
240-0193  R. Itoh  High Energy Accelerator Research Organization (KEK),
Tsukuba 305-0801  SOKENDAI (The Graduate University for Advanced Studies),
Hayama 240-0193  M. Iwasaki  Osaka City University, Osaka 558-8585  Y. Iwasaki
High Energy Accelerator Research Organization (KEK), Tsukuba 305-0801  W. W.
Jacobs  Indiana University, Bloomington, Indiana 47408  H. B. Jeon  Kyungpook
National University, Daegu 41566  S. Jia  Beihang University, Beijing 100191
Y. Jin  Department of Physics, University of Tokyo, Tokyo 113-0033  D. Joffe
Kennesaw State University, Kennesaw, Georgia 30144  K. K. Joo  Chonnam
National University, Gwangju 61186  A. B. Kaliyar  Indian Institute of
Technology Madras, Chennai 600036  K. H. Kang  Kyungpook National University,
Daegu 41566  G. Karyan  Deutsches Elektronen–Synchrotron, 22607 Hamburg  T.
Kawasaki  Kitasato University, Sagamihara 252-0373  H. Kichimi  High Energy
Accelerator Research Organization (KEK), Tsukuba 305-0801  C. H. Kim
Department of Physics and Institute of Natural Sciences, Hanyang University,
Seoul 04763  D. Y. Kim  Soongsil University, Seoul 06978  H. J. Kim  Kyungpook
National University, Daegu 41566  K. T. Kim  Korea University, Seoul 02841  S.
H. Kim  Department of Physics and Institute of Natural Sciences, Hanyang
University, Seoul 04763  K. Kinoshita  University of Cincinnati, Cincinnati,
Ohio 45221  P. Kodyš  Faculty of Mathematics and Physics, Charles University,
121 16 Prague  S. Korpar  University of Maribor, 2000 Maribor  J. Stefan
Institute, 1000 Ljubljana  D. Kotchetkov  University of Hawaii, Honolulu,
Hawaii 96822  P. Križan  Faculty of Mathematics and Physics, University of
Ljubljana, 1000 Ljubljana  J. Stefan Institute, 1000 Ljubljana  R. Kroeger
University of Mississippi, University, Mississippi 38677  J.-F. Krohn  School
of Physics, University of Melbourne, Victoria 3010  P. Krokovny  Budker
Institute of Nuclear Physics SB RAS, Novosibirsk 630090  Novosibirsk State
University, Novosibirsk 630090  T. Kuhr  Ludwig Maximilians University, 80539
Munich  R. Kumar  Punjab Agricultural University, Ludhiana 141004  Y.-J. Kwon
Yonsei University, Seoul 03722  J. S. Lange  Justus-Liebig-Universität Gießen,
35392 Gießen  I. S. Lee  Department of Physics and Institute of Natural
Sciences, Hanyang University, Seoul 04763  J. K. Lee  Seoul National
University, Seoul 08826  S. C. Lee  Kyungpook National University, Daegu 41566
L. K. Li  Institute of High Energy Physics, Chinese Academy of Sciences,
Beijing 100049  Y. B. Li  Peking University, Beijing 100871  L. Li Gioi  Max-
Planck-Institut für Physik, 80805 München  J. Libby  Indian Institute of
Technology Madras, Chennai 600036  K. Lieret  Ludwig Maximilians University,
80539 Munich  D. Liventsev  Virginia Polytechnic Institute and State
University, Blacksburg, Virginia 24061  High Energy Accelerator Research
Organization (KEK), Tsukuba 305-0801  T. Luo  Key Laboratory of Nuclear
Physics and Ion-beam Application (MOE) and Institute of Modern Physics, Fudan
University, Shanghai 200443  C. MacQueen  School of Physics, University of
Melbourne, Victoria 3010  M. Masuda  Earthquake Research Institute, University
of Tokyo, Tokyo 113-0032  T. Matsuda  University of Miyazaki, Miyazaki
889-2192  D. Matvienko  Budker Institute of Nuclear Physics SB RAS,
Novosibirsk 630090  Novosibirsk State University, Novosibirsk 630090  P.N.
Lebedev Physical Institute of the Russian Academy of Sciences, Moscow 119991
M. Merola  INFN - Sezione di Napoli, 80126 Napoli  Università di Napoli
Federico II, 80055 Napoli  F. Metzner  Institut für Experimentelle
Teilchenphysik, Karlsruher Institut für Technologie, 76131 Karlsruhe  K.
Miyabayashi  Nara Women’s University, Nara 630-8506  G. B. Mohanty  Tata
Institute of Fundamental Research, Mumbai 400005  T. J. Moon  Seoul National
University, Seoul 08826  T. Mori  Graduate School of Science, Nagoya
University, Nagoya 464-8602  R. Mussa  INFN - Sezione di Torino, 10125 Torino
K. R. Nakamura  High Energy Accelerator Research Organization (KEK), Tsukuba
305-0801  M. Nakao  High Energy Accelerator Research Organization (KEK),
Tsukuba 305-0801  SOKENDAI (The Graduate University for Advanced Studies),
Hayama 240-0193  K. J. Nath  Indian Institute of Technology Guwahati, Assam
781039  M. Nayak  Wayne State University, Detroit, Michigan 48202  High Energy
Accelerator Research Organization (KEK), Tsukuba 305-0801  N. K. Nisar
University of Pittsburgh, Pittsburgh, Pennsylvania 15260  S. Nishida  High
Energy Accelerator Research Organization (KEK), Tsukuba 305-0801  SOKENDAI
(The Graduate University for Advanced Studies), Hayama 240-0193  K. Nishimura
University of Hawaii, Honolulu, Hawaii 96822  K. Ogawa  Niigata University,
Niigata 950-2181  H. Ono  Nippon Dental University, Niigata 951-8580  Niigata
University, Niigata 950-2181  Y. Onuki  Department of Physics, University of
Tokyo, Tokyo 113-0033  P. Oskin  P.N. Lebedev Physical Institute of the
Russian Academy of Sciences, Moscow 119991  P. Pakhlov  P.N. Lebedev Physical
Institute of the Russian Academy of Sciences, Moscow 119991  Moscow Physical
Engineering Institute, Moscow 115409  G. Pakhlova  P.N. Lebedev Physical
Institute of the Russian Academy of Sciences, Moscow 119991  Moscow Institute
of Physics and Technology, Moscow Region 141700  B. Pal  Brookhaven National
Laboratory, Upton, New York 11973  T. Pang  University of Pittsburgh,
Pittsburgh, Pennsylvania 15260  H. Park  Kyungpook National University, Daegu
41566  S.-H. Park  Yonsei University, Seoul 03722  S. Patra  Indian Institute
of Science Education and Research Mohali, SAS Nagar, 140306  S. Paul
Department of Physics, Technische Universität München, 85748 Garching  T. K.
Pedlar  Luther College, Decorah, Iowa 52101  R. Pestotnik  J. Stefan
Institute, 1000 Ljubljana  L. E. Piilonen  Virginia Polytechnic Institute and
State University, Blacksburg, Virginia 24061  V. Popov  P.N. Lebedev Physical
Institute of the Russian Academy of Sciences, Moscow 119991  Moscow Institute
of Physics and Technology, Moscow Region 141700  E. Prencipe
Forschungszentrum Jülich, 52425 Jülich  M. T. Prim  Institut für
Experimentelle Teilchenphysik, Karlsruher Institut für Technologie, 76131
Karlsruhe  A. Rabusov  Department of Physics, Technische Universität München,
85748 Garching  P. K. Resmi  Indian Institute of Technology Madras, Chennai
600036  M. Ritter  Ludwig Maximilians University, 80539 Munich  M. Rozanska
H. Niewodniczanski Institute of Nuclear Physics, Krakow 31-342  G. Russo
Università di Napoli Federico II, 80055 Napoli  D. Sahoo  Tata Institute of
Fundamental Research, Mumbai 400005  Y. Sakai  High Energy Accelerator
Research Organization (KEK), Tsukuba 305-0801  SOKENDAI (The Graduate
University for Advanced Studies), Hayama 240-0193  S. Sandilya  University of
Cincinnati, Cincinnati, Ohio 45221  L. Santelj  High Energy Accelerator
Research Organization (KEK), Tsukuba 305-0801  T. Sanuki  Department of
Physics, Tohoku University, Sendai 980-8578  V. Savinov  University of
Pittsburgh, Pittsburgh, Pennsylvania 15260  O. Schneider  École Polytechnique
Fédérale de Lausanne (EPFL), Lausanne 1015  G. Schnell  University of the
Basque Country UPV/EHU, 48080 Bilbao  IKERBASQUE, Basque Foundation for
Science, 48013 Bilbao  J. Schueler  University of Hawaii, Honolulu, Hawaii
96822  C. Schwanda  Institute of High Energy Physics, Vienna 1050  A. J.
Schwartz  University of Cincinnati, Cincinnati, Ohio 45221  Y. Seino  Niigata
University, Niigata 950-2181  K. Senyo  Yamagata University, Yamagata 990-8560
M. E. Sevior  School of Physics, University of Melbourne, Victoria 3010  V.
Shebalin  University of Hawaii, Honolulu, Hawaii 96822  J.-G. Shiu  Department
of Physics, National Taiwan University, Taipei 10617  B. Shwartz  Budker
Institute of Nuclear Physics SB RAS, Novosibirsk 630090  Novosibirsk State
University, Novosibirsk 630090  F. Simon  Max-Planck-Institut für Physik,
80805 München  A. Sokolov  Institute for High Energy Physics, Protvino 142281
E. Solovieva  P.N. Lebedev Physical Institute of the Russian Academy of
Sciences, Moscow 119991  M. Starič  J. Stefan Institute, 1000 Ljubljana  Z. S.
Stottler  Virginia Polytechnic Institute and State University, Blacksburg,
Virginia 24061  T. Sumiyoshi  Tokyo Metropolitan University, Tokyo 192-0397
W. Sutcliffe  Institut für Experimentelle Teilchenphysik, Karlsruher Institut
für Technologie, 76131 Karlsruhe  M. Takizawa  Showa Pharmaceutical
University, Tokyo 194-8543  J-PARC Branch, KEK Theory Center, High Energy
Accelerator Research Organization (KEK), Tsukuba 305-0801  Theoretical
Research Division, Nishina Center, RIKEN, Saitama 351-0198  U. Tamponi  INFN -
Sezione di Torino, 10125 Torino  K. Tanida  Advanced Science Research Center,
Japan Atomic Energy Agency, Naka 319-1195  F. Tenchini  Deutsches
Elektronen–Synchrotron, 22607 Hamburg  K. Trabelsi  LAL, Univ. Paris-Sud,
CNRS/IN2P3, Université Paris-Saclay, Orsay 91898  M. Uchida  Tokyo Institute
of Technology, Tokyo 152-8550  T. Uglov  P.N. Lebedev Physical Institute of
the Russian Academy of Sciences, Moscow 119991  Moscow Institute of Physics
and Technology, Moscow Region 141700  S. Uno  High Energy Accelerator Research
Organization (KEK), Tsukuba 305-0801  SOKENDAI (The Graduate University for
Advanced Studies), Hayama 240-0193  Y. Usov  Budker Institute of Nuclear
Physics SB RAS, Novosibirsk 630090  Novosibirsk State University, Novosibirsk
630090  S. E. Vahsen  University of Hawaii, Honolulu, Hawaii 96822  R. Van
Tonder  Institut für Experimentelle Teilchenphysik, Karlsruher Institut für
Technologie, 76131 Karlsruhe  G. Varner  University of Hawaii, Honolulu,
Hawaii 96822  K. E. Varvell  School of Physics, University of Sydney, New
South Wales 2006  A. Vossen  Duke University, Durham, North Carolina 27708  E.
Waheed  School of Physics, University of Melbourne, Victoria 3010  B. Wang
Max-Planck-Institut für Physik, 80805 München  C. H. Wang  National United
University, Miao Li 36003  M.-Z. Wang  Department of Physics, National Taiwan
University, Taipei 10617  P. Wang  Institute of High Energy Physics, Chinese
Academy of Sciences, Beijing 100049  X. L. Wang  Key Laboratory of Nuclear
Physics and Ion-beam Application (MOE) and Institute of Modern Physics, Fudan
University, Shanghai 200443  S. Watanuki  Department of Physics, Tohoku
University, Sendai 980-8578  J. Wiechczynski  H. Niewodniczanski Institute of
Nuclear Physics, Krakow 31-342  E. Won  Korea University, Seoul 02841  H.
Yamamoto  Department of Physics, Tohoku University, Sendai 980-8578  S. B.
Yang  Korea University, Seoul 02841  H. Ye  Deutsches Elektronen–Synchrotron,
22607 Hamburg  J. H. Yin  Institute of High Energy Physics, Chinese Academy of
Sciences, Beijing 100049  C. Z. Yuan  Institute of High Energy Physics,
Chinese Academy of Sciences, Beijing 100049  Z. P. Zhang  University of
Science and Technology of China, Hefei 230026  V. Zhilich  Budker Institute of
Nuclear Physics SB RAS, Novosibirsk 630090  Novosibirsk State University,
Novosibirsk 630090  V. Zhukova  P.N. Lebedev Physical Institute of the Russian
Academy of Sciences, Moscow 119991  V. Zhulanov  Budker Institute of Nuclear
Physics SB RAS, Novosibirsk 630090  Novosibirsk State University, Novosibirsk
630090

###### Abstract

The experimental results on the ratios of branching fractions
$\mathcal{R}(D)={\cal B}(\bar{B}\to D\tau^{-}\bar{\nu}_{\tau})/{\cal
B}(\bar{B}\to D\ell^{-}\bar{\nu}_{\ell})$ and $\mathcal{R}(D^{\ast})={\cal
B}(\bar{B}\to D^{*}\tau^{-}\bar{\nu}_{\tau})/{\cal B}(\bar{B}\to
D^{*}\ell^{-}\bar{\nu}_{\ell})$, where $\ell$ denotes an electron or a muon,
show a long-standing discrepancy with the Standard Model predictions, and
might hint to a violation of lepton flavor universality. We report a new
simultaneous measurement of $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$,
based on a data sample containing $772\times 10^{6}$ $B\bar{B}$ events
recorded at the
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}$
resonance with the Belle detector at the KEKB $e^{+}e^{-}$ collider. In this
analysis the tag-side $B$ meson is reconstructed in a semileptonic decay mode
and the signal-side $\tau$ is reconstructed in a purely leptonic decay. The
measured values are $\mathcal{R}(D)\ =0.307\pm 0.037\pm 0.016$ and
$\mathcal{R}(D^{\ast})\ =0.283\pm 0.018\pm 0.014$, where the first
uncertainties are statistical and the second are systematic. These results are
in agreement with the Standard Model predictions within $0.2$, $1.1$ and $0.8$
standard deviations for $\mathcal{R}(D)$, $\mathcal{R}(D^{\ast})$ and their
combination, respectively. This work constitutes the most precise measurements
of $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ performed to date as well as
the first result for $\mathcal{R}(D)$ based on a semileptonic tagging method.

###### pacs:

13.20.He, 14.40.Nd

††preprint:  Version 4.2 Belle Preprint 2019-18 KEK Preprint 2019-40 Intended
for PRL

Semitauonic $B$ meson decays, involving the transition $b\to c\tau\nu_{\tau}$,
are sensitive probes for physics beyond the Standard Model (SM). Any
difference in the branching fraction of these processes with respect to the SM
prediction would violate lepton flavor universality, which enforces equal
coupling of the gauge bosons to the three lepton generations. Indeed, in many
models beyond the SM, new interactions with enhanced coupling to the third
family are postulated. Among such new mediators, charged Higgs bosons, which
appear in supersymmetry Martin (1998) and other models with two Higgs doublets
Gunion  _et al._ (2000), may contribute measurably to the $b\to
c\tau\nu_{\tau}$ decay rate due to the large masses of the $\tau$ and the $b$
quark. Similarly, leptoquarks Buchmuller  _et al._ (1987), which carry both
lepton and baryon numbers, may also contribute to this process.

The ratios of branching fractions,

$\displaystyle\mathcal{R}(D^{(\ast)})=\frac{{\cal B}(\bar{B}\to
D^{(*)}\tau^{-}\bar{\nu}_{\tau})}{{\cal B}(\bar{B}\to
D^{(*)}\ell^{-}\bar{\nu}_{\ell})}$ (1)

where the denominator represents the average of electron and muon modes, are
typically measured instead of the absolute branching fractions of $\bar{B}\to
D^{(*)}\tau^{-}\bar{\nu}_{\tau}$ to reduce common systematic uncertainties,
such as those due to the detection efficiency, the magnitude of the quark-
mixing matrix element $|V_{cb}|$, and the semileptonic decay form factors.
Hereafter, $\bar{B}\to D^{(*)}\tau^{-}\bar{\nu}_{\tau}$ not  and $\bar{B}\to
D^{(*)}\ell^{-}\bar{\nu}_{\ell}$ will be referred to as the signal and
normalization modes, respectively. The SM calculations for these ratios,
performed by several groups Bigi and Gambino (2016); Bernlochner  _et al._
(2017); Bigi  _et al._ (2017); Jaiswal  _et al._ (2017), are averaged by HFLAV
Amhis  _et al._ (2019) to obtain $\mathcal{R}(D)=0.299\pm 0.003$ and
$\mathcal{R}(D^{\ast})=0.258\pm 0.005$.

Semitauonic $B$ decays were first observed by Belle in 2007 Matyja  _et al._
(2007), with subsequent studies reported by Belle Bozek  _et al._ (2010);
Huschle  _et al._ (2015); Sato  _et al._ (2016); Hirose  _et al._ (2017);
*Hirose:2016wfn, BaBar Lees  _et al._ (2012); *lees2013, and LHCb Aaij  _et
al._ (2015, 2018). The average values of the experimental results, excluding
the result presented in this Letter, are $\mathcal{R}(D)=0.407\pm 0.039\pm
0.024$ and $\mathcal{R}(D^{\ast})=0.306\pm 0.013\pm 0.007$ Amhis  _et al._
(2019), where the first uncertainty is statistical and the second is
systematic. These values exceed SM predictions by $2.1\sigma$ and $3.0\sigma$,
respectively, where $\sigma$ denotes the standard deviation. A combined
analysis of $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ taking correlations
into account finds that the deviation from the SM prediction is approximately
$3.8\sigma$ Amhis  _et al._ (2019). This large discrepancy must be
investigated with complementary and more precise measurements.

Measurements at the $e^{+}e^{-}$ “$B$-factory” experiments Belle and BaBar,
are commonly performed by first reconstructing one of the $B$ mesons in the
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}\to
B\bar{B}$ decay, denoted as ${B}_{\text{tag}}$, using a dedicated tagging
algorithm. So far, simultaneous measurements of $\mathcal{R}(D)$ and
$\mathcal{R}(D^{\ast})$ at Belle and BaBar have been performed using hadronic
tagging methods on both $B^{0}$ and $B^{+}$ decays Huschle  _et al._ (2015);
Lees  _et al._ (2012, 2013), while only $\mathcal{R}(D^{*+})$ was measured
with a semileptonic tagging method Sato  _et al._ (2016). In this Letter, we
report the first measurement of $\mathcal{R}(D)$ using the semileptonic
tagging method, and we update or measurement of $\mathcal{R}(D^{\ast})$ by
combining results of $B^{0}$ and $B^{+}$ decays with a more efficient tagging
algorithm. Our previous measurement of $\mathcal{R}(D^{*+})$ with a
semileptonic tagging method is therefore superseded by this work.

We use the full
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}$
data sample containing $772\times 10^{6}$ $B\bar{B}$ events recorded with the
Belle detector Abashian  _et al._ (2002) at the KEKB $e^{+}e^{-}$ collider
Kurokawa and Kikutani (2003). Belle was a general-purpose magnetic
spectrometer, which consisted of a silicon vertex detector (SVD), a 50-layer
central drift chamber (CDC), an array of aerogel threshold Cherenkov counters
(ACC), time-of-flight scintillation counters (TOF), and an electromagnetic
calorimeter (ECL) comprising CsI(Tl) crystals. These components were located
inside a superconducting solenoid coil that provided a 1.5 T magnetic field.
An iron flux-return yoke located outside the coil was instrumented to detect
$K_{L}^{0}$ mesons and muons (KLM). The detector is described in detail
elsewhere Abashian  _et al._ (2002).

To determine the reconstruction efficiency and probability density functions
(PDFs) for signal, normalization, and background modes, we use Monte Carlo
(MC) simulated events generated with the EvtGen event generator Lange (2001).
The detector response is simulated with the GEANT3 package Brun  _et al._ .

Semileptonic $B\to D^{(*)}\ell\nu$ decays are generated with the HQET2 EvtGen
package, based on the CLN parametrization Caprini  _et al._ (1998). As the
measured parameters of the model have been updated since our MC sample was
generated, we apply an event-by-event correction factor obtained by taking the
ratio of differential decay rates in the updated CLN parameters compared to
those used in the MC. For the MC samples of $B\to D^{**}\ell\nu$ decays, we
used the ISGW2 EvtGen package, based on the quark model described in Ref.
Scora and Isgur (1995). This model has been superseded by the LLSW model
Leibovich  _et al._ (1998); thus we weight events with a correction factor
based on the ratio of the analytic predictions of LLSW and MC distributions
generated with ISGW2. Here, $D^{**}$ denotes the orbitally excited states
$D_{1}$, $D_{2}^{*}$, $D_{1}^{\prime}$, and $D_{0}^{*}$. We consider $D^{**}$
decays to a $D^{(*)}$ and a pion, a $\rho$ or an $\eta$ meson, or a pair of
pions, where branching fractions are based on quantum number, phase-space, and
isospin arguments. The sizes of the inclusive
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}\to
B\bar{B}$ MC sample and the dedicated $B\to D^{**}\ell\nu$ MC sample
correspond to about 10 times and 5 times the integrated luminosity of the
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}$
data sample, respectively.

The ${B}_{\text{tag}}$ is reconstructed using a hierarchical algorithm based
on boosted decision trees (BDT) Keck  _et al._ (2019) in
$D\ell\bar{\nu}_{\ell}$ and $D^{*}\ell\bar{\nu}_{\ell}$ channels, where
$\ell=e,\mu$. The BDT classifier assigns to each ${B}_{\text{tag}}$ candidate
a probability of representing a well-reconstructed $B$ meson. The range of the
BDT classifier extends from 0 to 1, with well-reconstructed candidates having
the highest values. We select ${B}_{\text{tag}}$ candidates with a BDT
classifier output greater than $10^{-1.5}$. We suppress ${B}\to
D^{\ast}\tau(\to\ell\nu\nu)\nu$ events on the ${B}_{\text{tag}}$ side by
applying a selection on $\cos\theta_{B,D^{(\ast)}\ell}$. This variable
corresponds to the cosine of the angle between the momenta of the $B$ meson
and the $D^{(*)}\ell$ system in the
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}$
rest frame, under the assumption that only one massless particle is not
reconstructed:

$\displaystyle\cos\theta_{B,D^{(\ast)}\ell}\equiv\frac{2E_{\rm
beam}E_{D^{(*)}\ell}-m_{B}^{2}-m_{D^{(*)}\ell}^{2}}{2|\bm{p}_{B}||\bm{p}_{D^{(*)}\ell}|}.$
(2)

Here $E_{\rm beam}$ is the beam energy, and $E_{D^{*}\ell}$,
$\bm{p}_{D^{*}\ell}$, and $m_{D^{*}\ell}$ are the energy, momentum, and mass,
of the $D^{*}\ell$ system, respectively. The quantities $m_{B}$ and
$|\bm{p}_{B}|$ are the nominal $B$ meson mass Tanabashi  _et al._ (2018) and
momentum, respectively. All quantities are evaluated in the
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}$
rest frame.

Correctly reconstructed $B\to D^{(*)}\ell\nu$ decays are expected to have a
value of $\cos\theta_{B,D^{(\ast)}\ell}$ between $-1$ and $+1$. Correctly
reconstructed as well as misreconstructed $B\to D^{(*)}\tau\nu$ decays
generally have $\cos\theta_{B,D^{(\ast)}\ell}$ values below $-1$ due to the
presence of additional missing particles. To account for detector resolution
effects we apply the requirement $-2.0<\cos\theta_{B,D^{(\ast)}\ell}<1.0$ for
the ${B}_{\text{tag}}$.

In each event with a selected ${B}_{\text{tag}}$ candidate, we search for the
opposite-flavor signature $D^{(*)}\ell$ among the remaining tracks and
calorimeter clusters, since we only reconstruct pure leptonic tau decays
$\tau\to\ell\bar{\nu}\nu$. We define four disjoint data samples, denoted
$D^{+}\ell^{-}$, $D^{0}\ell^{-}$, $D^{*+}\ell^{-}$, and $D^{*0}\ell^{-}$.

Charged particle tracks are reconstructed with the SVD and CDC by requiring a
point of closest approach to the interaction point smaller than 5.0 cm along
the direction of the $e^{+}$ beam and 2.0 cm in the direction perpendicular to
it. These requirements do not apply to the pions from $K_{S}^{0}$ decays.
Electrons are identified by a combination of the specific ionization ($dE/dx$)
in the CDC, the ratio of the cluster energy in the ECL to the track momentum
measured with the CDC, the response of the ACC, the cluster shape in the ECL,
and the match between positions of the cluster and the track at the ECL. To
recover bremsstrahlung photons from electrons, we add the four-momentum of
each photon detected within a cone of 0.05 rad of the original track direction
to the electron momentum. Muons are identified by the track penetration depth
and hit distribution in the KLM. Charged kaons are identified by combining
information from the $dE/dx$ measured in the CDC, the flight time measured
with the TOF, and the response of the ACC. We do not apply any particle
identification criteria for charged pion candidates.

Candidate $K_{S}^{0}$ mesons are formed by combining two oppositely charged
tracks with pion mass hypotheses. We require their invariant mass to lie
within $\pm$15 MeV/$c^{2}$ of the nominal $K^{0}$ mass Tanabashi  _et al._
(2018), which corresponds to approximately seven times the reconstructed mass
resolution. Further selection is performed with an algorithm based on a neural
network Feindt and Kerzel (2006).

Photons are measured as an electromagnetic cluster in the ECL with no
associated charged track. Neutral pions are reconstructed in the
$\pi^{0}\to\gamma\gamma$ channel, and their energy resolution is improved by
performing a mass-constrained fit of the two photon candidates to the nominal
$\pi^{0}$ mass Tanabashi  _et al._ (2018). For neutral pions from $D$ decays,
we require the daughter photon energies to be greater than 50 MeV and their
asymmetry to be less than 0.6 in the laboratory frame, the cosine of the angle
between two photons to be greater than zero, and the $\gamma\gamma$ invariant
mass to be within $[-15,+10]$ MeV/$c^{2}$ of the nominal $\pi^{0}$ mass, which
corresponds to approximately $\pm 1.8$ times the resolution. Low-energy
$\pi^{0}$ candidates from $D^{*}$ are reconstructed using less restrictive
energy requirements: one photon must have an energy of at least 50 MeV, while
the other must have a minimum energy of 20 MeV. We also require a narrower
window around the diphoton invariant mass to compensate for the lower photon-
energy requirement: within 10 MeV/$c^{2}$ of the nominal $\pi^{0}$ mass, which
corresponds to approximately $\pm 1.6$ times the resolution.

Neutral $D$ mesons are reconstructed in the following decay modes: $D^{0}\to
K^{-}\pi^{+}\pi^{0}$, $K^{-}\pi^{+}\pi^{+}\pi^{-}$, $K^{-}\pi^{+}$,
$K_{S}^{0}\pi^{+}\pi^{-}$, $K_{S}^{0}\pi^{0}$, $K_{S}^{0}K^{+}K^{-}$,
$K^{+}K^{-}$, and $\pi^{+}\pi^{-}$. Similarly, charged $D$ mesons are
reconstructed in the following modes: $D^{+}\to K^{-}\pi^{+}\pi^{+}$,
$K_{S}^{0}\pi^{+}\pi^{0}$, $K_{S}^{0}\pi^{+}\pi^{+}\pi^{-}$,
$K_{S}^{0}\pi^{+}$, $K^{-}K^{+}\pi^{+}$, and $K_{S}^{0}K^{+}$. The combined
branching fractions for reconstructed channels are 30% and 22% for $D^{0}$ and
$D^{+}$, respectively. For $D$ decays without a $\pi^{0}$ in the final state,
we require the invariant mass of the reconstructed candidates to be within 15
MeV/$c^{2}$ of the nominal $D^{0}$ or $D^{+}$ mass, which corresponds to a
window of approximately $\pm 2.8$ times the resolution. In the case of
channels with a $\pi^{0}$ in the final state, which have worse mass
resolution, we require a wider window: from $-45$ to $+30$ MeV/$c^{2}$ around
the nominal $D^{0}$ mass, and from $-36$ to $+24$ MeV/$c^{2}$ around the
nominal $D^{+}$ mass. These windows correspond to approximately [$-1.1,+1.6$]
and [$-1.0,+1.4$] times the resolution, respectively. Candidate $D^{*+}$
mesons are reconstructed in the channels $D^{0}\pi^{+}$ and $D^{+}\pi^{0}$,
and $D^{*0}$ in the channel $D^{0}\pi^{0}$. We do not consider the $D^{*0}\to
D^{0}\gamma$ decay channel due to its higher background level.

We require the mass difference $D^{*}-D$ be within 2.5 MeV/$c^{2}$ for the
$D^{*+}\to D^{0}\pi^{+}$ decay mode, and within 2.0 MeV/$c^{2}$ for the
$D^{*+}\to D^{+}\pi^{0}$ and $D^{*0}\to D^{0}\pi^{0}$ decay modes. These
windows correspond to $\pm 3.0$ and $\pm 1.9$ times the resolution,
respectively. We require a tighter mass window in the $D^{*}$ modes that
contain a low-momentum (“slow”) $\pi^{0}$ to suppress the large background
arising from misreconstructed neutral pions.

On the signal side, we require $\cos\theta_{B,D^{(\ast)}\ell}$ to be less than
1.0 and the $D^{(*)}$ momentum in the
${\mathit{{\Upsilon}{}_{\mspace{-3.0mu}\scriptstyle{}}^{\mspace{-1.0mu}\scriptstyle{}}}}{\mathit{{\left({4S}\right)}}}$
rest frame to be less than 2.0 GeV/$c$. Finally, we require that events
contain no extra prompt charged tracks, $K_{S}^{0}$ candidates, or $\pi^{0}$
candidates, which are reconstructed with the same criteria as those used for
the $D$ candidates. All selection criteria used for event reconstruction have
been the subject of optimization studies. When multiple ${B}_{\text{tag}}$ or
${B}_{\text{sig}}$ candidates are found in an event, we first select the
${B}_{\text{tag}}$ candidate with the highest tagging classifier output, and
then the ${B}_{\text{sig}}$ candidate with the highest p-value from the vertex
fit of the $B$ candidate’s charm daughter.

To distinguish signal and normalization events from background processes, we
use the sum of the energies of neutral clusters detected in the ECL that are
not associated with any reconstructed particles, denoted as $E_{\text{ECL}}$.
To mitigate the varying effects of photons related to beam background in the
calculation of $E_{\text{ECL}}$, we only include clusters with energies
greater than 50, 100, and 150 MeV, respectively, from the barrel, forward, and
backward ECL regions Abashian  _et al._ (2002). Signal and normalization
events peak near zero in $E_{\rm ECL}$, while background events populate a
wider range. We require that $E_{\text{ECL}}$ be less than 1.2 GeV.

To separate reconstructed signal and normalization events, we employ a BDT
based on the `XGBoost` package Chen and Guestrin (2016). The input variables
to the BDT are $\cos\theta_{B,D^{(\ast)}\ell}$; the approximate missing mass
squared $m_{\rm miss}^{2}=(E_{\rm
beam}-E_{D^{(*)}}-E_{\ell})^{2}-(\bm{p}_{D^{(*)}}+\bm{p}_{\ell})^{2}$; the
visible energy $E_{\rm vis}=\sum_{i}E_{i}$, where $(E_{i},\bm{p}_{i})$ is the
four-momentum of particle $i$. We do not apply any selection on the BDT
classifier output, denoted as $O_{\text{cls}}$; instead we use it as one of
the fitting variables for the extraction of $\mathcal{R}(D^{(\ast)})$. Signal
events have $O_{\text{cls}}$ values near 1, while normalization events have
values near 0.

We extract the yields of signal and normalization modes from a two-dimensional
(2D) extended maximum-likelihood fit to the variables $O_{\text{cls}}$ and
$E_{\text{ECL}}$. The fit is performed simultaneously to the four
$D^{(*)}\ell$ samples, and exploits the isospin constraint
$R(D^{(*)0})=R(D^{(*)+})$. The distribution of each sample is described as the
sum of several components: $D^{(*)}\tau\nu$, $D^{(*)}\ell\nu$, feed-down from
$D^{*}\ell(\tau)\nu$ to $D\ell(\tau)\nu$, $D^{**}\ell(\tau)\nu$, and other
backgrounds. The PDFs of these components are determined from MC simulations
as 2D histogram templates. A large fraction of $B\to D^{*}\ell\nu$ decays from
both $B^{0}$ and $B^{+}$ are reconstructed in the $D\ell$ samples (denoted
feed-down). We leave these two contributions free in the fit and use their
fitted yields to correct the MC estimated feed-down rate of $B\to
D^{*}\tau\nu$ decays. The events of the $D^{*}\ell$ samples that appear as
feed-down are treated as a component of the signal or normalization yields. As
the probability of $B\to D\ell(\tau)\nu$ decays contributing to the
$D^{*}\ell$ samples is very small, the relative rates of these contributions
are fixed to the MC expected values.

The free parameters in the final fit are the yields of signal, normalization,
$B\to D^{**}\ell\nu_{\ell}$, and feed-down from $D^{*}\ell$ to $D\ell$
components. The yields of other backgrounds are fixed to their MC expected
values. The ratios $\mathcal{R}(D^{(\ast)})$ are given by the formula:

$\displaystyle\mathcal{R}(D^{(\ast)})$ $\displaystyle=$
$\displaystyle\frac{1}{2{\cal
B}(\tau^{-}\to\ell^{-}\bar{\nu}_{\ell}\nu_{\tau})}\cdot\frac{\varepsilon_{\rm
norm}}{\varepsilon_{\rm sig}}\cdot\frac{N_{\rm sig}}{N_{\rm norm}},$ (3)

where $\varepsilon_{\rm sig(norm)}$ and $N_{\rm sig(norm)}$ are the detection
efficiency including tagging efficiency and yields of signal (normalization)
modes and ${\cal B}(\tau^{-}\to\ell^{-}\bar{\nu}_{\ell}\nu_{\tau})$ is the
average of the world-average branching fractions for $\ell=e$ and $\ell=\mu$.

Figure 1: $E_{\text{ECL}}$ fit projections and data points with statistical
uncertainties in the $D^{+}\ell^{-}$ (top left), $D^{0}\ell^{-}$ (top right),
$D^{*+}\ell^{-}$ (bottom left) and $D^{*0}\ell^{-}$ (bottom right) samples,
for the full classifier region. The signal region, defined by the selection
$O_{\text{cls}}>0.9$, is shown in the inset.

To improve the accuracy of the MC simulation, we apply a series of correction
factors determined from control sample measurements, such as those associated
to lepton and hadron identification efficiencies as well as slow pion tracking
efficiencies. Correction factors for the lepton efficiencies are evaluated as
a function of the lepton momentum and direction using $e^{+}e^{-}\to
e^{+}e^{-}\ell^{+}\ell^{-}$ and $J/\psi\to\ell^{+}\ell^{-}$ decays.
Furthermore, to determine the expected yield of fake and misreconstructed
$D^{(*)}$ mesons, treated as background, we use data sidebands of difference
between their nominal and reconstructed mass, and we correct for differences
in the reconstruction efficiency of the tagging algorithm between data and MC
simulation.

The $E_{\text{ECL}}$ projections of the fit are shown in Fig. 1. The fit finds
$\mathcal{R}(D)=0.307\pm 0.037$ and $\mathcal{R}(D^{\ast})=0.283\pm 0.018$,
where the error is statistical.

To estimate various systematic uncertainties contributing to
$\mathcal{R}(D^{(\ast)})$, we vary each fixed parameter 500 times, sampling
from a Gaussian distribution built using the value and uncertainty of the
parameter. For each variation, we repeat the fit. The associated systematic
uncertainty is taken as the standard deviation of the resulting distribution
of fitted results. The systematic uncertainties are listed in Table 1.

In Table 1 the label “$D^{**}$ composition” refers to the uncertainty
introduced by the branching fractions of the $B\to D^{**}\ell\nu_{\ell}$
channels and the decays of the $D^{**}$ mesons, which are not well known and
hence contribute significantly to the total PDF uncertainty. The uncertainties
on the branching fraction of $B\to D^{**}\ell\nu_{\ell}$ are assumed to be
$\pm 6\%$ for $D_{1}$, $\pm 10\%$ for $D_{2}^{*}$, $\pm 83\%$ for
$D_{1}^{\prime}$, and $\pm 100\%$ for $D_{0}^{*}$, while the uncertainties on
each of the $D^{**}$ decay branching fractions are conservatively assumed to
be $\pm 100\%$.

A large systematic uncertainty arises from the limited size of the MC samples.
Firstly, this is reflected in the uncertainty of the PDF shapes. To estimate
this contribution, we recalculate PDFs for signal, normalization, fake
$D^{(*)}$ events, $B\to D^{**}\ell\nu_{\ell}$, feed-down, and other
backgrounds by generating toy MC samples from the nominal PDFs according to
Poisson statistics, and then repeating the fit with the new PDFs. Secondly,
the reconstruction efficiency of feed-down events, together with the
efficiency ratio of signal to normalization events, are varied within their
uncertainties, which are limited by the size of the MC samples as well.

The efficiency factors for the fake $D^{(*)}$ and ${B}_{\text{tag}}$
reconstruction are calibrated using collision data. The uncertainties on these
factors are affected by the size of the samples used in the calibration. We
vary the factors within their errors and extract associated systematic
uncertainties.

The effect of the lepton efficiency and fake rate, as well as that due to the
slow pion efficiency, do not cancel out in the $\mathcal{R}(D^{(*)})$ ratios.
This is due to the different momentum spectra of leptons and charm mesons in
the normalization and signal modes. The uncertainties introduced by these
factors are included in the total systematic uncertainty.

We include minor systematic contributions from other sources: one related to
the parameters that are used for re-weighting the semileptonic $B\to
D^{(*)}\ell\nu$ and $B\to D^{**}\ell\nu$ decays; and others from the
integrated luminosity, the $B$ production fractions at the $\Upsilon(4S)$,
$f^{+-}$ and $f^{00}$, and the branching fractions of $B\to D^{(*)}\ell\nu$,
$D$, $D^{*}$ and $\tau^{-}\to\ell^{-}\bar{\nu}_{\ell}\nu_{\tau}$ decays
Tanabashi  _et al._ (2018). The total systematic uncertainty is estimated by
summing the aforementioned contributions in quadrature.

Table 1: Systematic uncertainties contributing to the $\mathcal{R}(D^{(\ast)})$ results, together with their correlation. Source | $\Delta\mathcal{R}(D)\ (\%)$ | $\Delta\mathcal{R}(D^{\ast})\ (\%)$ | Correlation
---|---|---|---
$D^{**}$ composition | 0.76 | 1.41 | $-0.41$
PDF shapes | 4.39 | 2.25 | $-0.55$
Feed-down factors | 1.69 | 0.44 | $\quad 0.53$
Efficiency factors | 1.93 | 4.12 | $-0.57$
Fake $D^{(*)}$ calibration | 0.19 | 0.11 | $-0.76$
${B}_{\text{tag}}$ calibration | 0.07 | 0.05 | $-0.76$
Lepton efficiency | 0.36 | 0.33 | $-0.83$
and fake rate |  |  |
Slow pion efficiency | 0.08 | 0.08 | $-0.98$
$B$ decay form factors | 0.55 | 0.28 | $-0.60$
Luminosity, $f^{+-}$, $f^{00}$ | 0.10 | 0.04 | $-0.58$
and $\mathcal{B}(\Upsilon(4S))$ |  |  |
$\mathcal{B}(B\to D^{(*)}\ell\nu)$ | 0.05 | 0.02 | $-0.69$
$\mathcal{B}(D)$ | 0.35 | 0.13 | $-0.65$
$\mathcal{B}(D^{*})$ | 0.04 | 0.02 | $-0.51$
$\mathcal{B}(\tau^{-}\to\ell^{-}\bar{\nu}_{\ell}\nu_{\tau})$ | 0.15 | 0.14 | $-0.11$
Total | 5.21 | 4.94 | $-0.52$

In conclusion, we have measured the ratios $\mathcal{R}(D^{(\ast)})={\cal
B}(\bar{B}\to D^{(*)}\tau^{-}\bar{\nu}_{\tau})/{\cal B}(\bar{B}\to
D^{(*)}\ell^{-}\bar{\nu}_{\ell})$, where $\ell$ denotes an electron or a muon,
using a semileptonic tagging method and a data sample containing $772\times
10^{6}B\bar{B}$ events collected with the Belle detector. The results are

$\displaystyle\mathcal{R}(D)$ $\displaystyle=$ $\displaystyle 0.307\pm
0.037\pm 0.016$ (4) $\displaystyle\mathcal{R}(D^{\ast})$ $\displaystyle=$
$\displaystyle 0.283\pm 0.018\pm 0.014,$ (5)

where the first uncertainties are statistical and the second are systematic.
These results are in agreement with the SM predictions within $0.2\sigma$ and
$1.1\sigma$, respectively. The combined result agrees with the SM predictions
within $0.8\sigma$. This work constitutes the most precise measurements of
$\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ performed to date and the first
result for $\mathcal{R}(D)$ based on a semileptonic tagging method. The
results of this analysis, together with the most recent Belle results on
$\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ (Huschle  _et al._ (2015); Hirose
_et al._ (2018)) obtained using a hadronic tag, are combined to provide the
Belle combination, which yields $\mathcal{R}(D)=0.326\pm 0.034,\
\mathcal{R}(D^{*})=0.283\pm 0.018$ with a correlation equal to $-0.47$ between
the $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ values. This combined result
is in agreement with the SM predictions within 1.6 standard deviations.

We thank the KEKB group for excellent operation of the accelerator; the KEK
cryogenics group for efficient solenoid operations; and the KEK computer
group, the NII, and PNNL/EMSL for valuable computing and SINET5 network
support. We acknowledge support from MEXT, JSPS and Nagoya’s TLPRC (Japan);
ARC (Australia); FWF (Austria); NSFC and CCEPP (China); MSMT (Czechia); CZF,
DFG, EXC153, and VS (Germany); DST (India); INFN (Italy); MOE, MSIP, NRF,
RSRI, FLRFAS project, GSDC of KISTI and KREONET/GLORIAD (Korea); MNiSW and NCN
(Poland); MSHE, Agreement 14.W03.31.0026 (Russia); ARRS (Slovenia); IKERBASQUE
(Spain); SNSF (Switzerland); MOE and MOST (Taiwan); and DOE and NSF (USA). We
acknowledge the support provided by the Albert Shimmins Fund for the writing
of this Letter.

## References

  * Martin (1998) S. P. Martin, Adv. Ser. Direct. High Energy Phys. 18, 1 (1998), arXiv:hep-ph/9709356 [hep-ph] .
  * Gunion  _et al._ (2000) J. F. Gunion, H. E. Haber, G. L. Kane, and S. Dawson, Front. Phys. 80, 1 (2000).
  * Buchmuller  _et al._ (1987) W. Buchmuller, R. Ruckl, and D. Wyler, Phys. Lett. B191, 442 (1987), [Erratum: Phys. Lett. B448, 320 (1999)].
  * (4) Throughout this paper, the inclusion of the charge-conjugate decay mode is implied.
  * Bigi and Gambino (2016) D. Bigi and P. Gambino, Phys. Rev. D94, 094008 (2016), arXiv:1606.08030 [hep-ph] .
  * Bernlochner  _et al._ (2017) F. U. Bernlochner, Z. Ligeti, M. Papucci, and D. J. Robinson, Phys. Rev. D95, 115008 (2017), [erratum: Phys. Rev. D97, 059902 (2018)], arXiv:1703.05330 [hep-ph] .
  * Bigi  _et al._ (2017) D. Bigi, P. Gambino, and S. Schacht, JHEP 11, 061 (2017), arXiv:1707.09509 [hep-ph] .
  * Jaiswal  _et al._ (2017) S. Jaiswal, S. Nandi, and S. K. Patra, JHEP 12, 060 (2017), arXiv:1707.09977 [hep-ph] .
  * Amhis  _et al._ (2019) Y. Amhis _et al._ (Heavy Flavor Averaging Group), “Averages of $b$-hadron, $c$-hadron, and $\tau$-lepton properties as of 2018,” (2019), arXiv:1909.12524 [hep-ex] .
  * Matyja  _et al._ (2007) A. Matyja _et al._ (Belle Collaboration), Phys. Rev. Lett. 99, 191807 (2007), arXiv:0706.4429 [hep-ex] .
  * Bozek  _et al._ (2010) A. Bozek _et al._ (Belle Collaboration), Phys. Rev. D82, 072005 (2010), arXiv:1005.2302 [hep-ex] .
  * Huschle  _et al._ (2015) M. Huschle _et al._ (Belle Collaboration), Phys. Rev. D92, 072014 (2015), arXiv:1507.03233 [hep-ex] .
  * Sato  _et al._ (2016) Y. Sato _et al._ (Belle Collaboration), Phys. Rev. D94, 072007 (2016), arXiv:1607.07923 [hep-ex] .
  * Hirose  _et al._ (2017) S. Hirose _et al._ (Belle Collaboration), Phys. Rev. Lett. 118, 211801 (2017), arXiv:1612.00529 [hep-ex] .
  * Hirose  _et al._ (2018) S. Hirose _et al._ (Belle Collaboration), Phys. Rev. D97, 012004 (2018), arXiv:1709.00129 [hep-ex] .
  * Lees  _et al._ (2012) J. P. Lees _et al._ (BaBar Collaboration), Phys. Rev. Lett. 109, 101802 (2012), arXiv:1205.5442 [hep-ex] .
  * Lees  _et al._ (2013) J. P. Lees _et al._ (BaBar Collaboration), Phys. Rev. D88, 072012 (2013), arXiv:1303.0571 [hep-ex] .
  * Aaij  _et al._ (2015) R. Aaij _et al._ (LHCb Collaboration), Phys. Rev. Lett. 115, 111803 (2015), arXiv:1506.08614 [hep-ex] .
  * Aaij  _et al._ (2018) R. Aaij _et al._ (LHCb Collaboration), Phys. Rev. D97, 072013 (2018), arXiv:1711.02505 [hep-ex] .
  * Abashian  _et al._ (2002) A. Abashian _et al._ (Belle Collaboration), Nucl. Instrum. Methods Phys. Res., Sect. A 479, 117 (2002), also see detector section in J.Brodzicka et al., Prog. Theor. Exp. Phys. 2012, 04D001 (2012).
  * Kurokawa and Kikutani (2003) S. Kurokawa and E. Kikutani, Nucl. Instrum. Methods Phys. Res., Sect. A 499, 1 (2003), and other papers included in this Volume; T.Abe et al., Prog. Theor. Exp. Phys. 2013, 03A001 (2013) and references therein.
  * Lange (2001) D. J. Lange, Nucl. Instrum. Methods Phys. Res., Sect. A 462, 152 (2001).
  * (23) R. Brun _et al._ , GEANT3, CERN Report No. DD/EE/84-1, (1987).
  * Caprini  _et al._ (1998) I. Caprini, L. Lellouch, and M. Neubert, Nucl. Phys. B530, 153 (1998), arXiv:hep-ph/9712417 [hep-ph] .
  * Scora and Isgur (1995) D. Scora and N. Isgur, Phys. Rev. D52, 2783 (1995).
  * Leibovich  _et al._ (1998) A. K. Leibovich, Z. Ligeti, I. W. Stewart, and M. B. Wise, Phys. Rev. D57, 308 (1998).
  * Keck  _et al._ (2019) T. Keck _et al._ , Computing and Software for Big Science 3, 6 (2019), arXiv:1807.08680 [hep-ex] .
  * Tanabashi  _et al._ (2018) M. Tanabashi _et al._ (Particle Data Group), Phys. Rev. D 98, 030001 (2018).
  * Feindt and Kerzel (2006) M. Feindt and U. Kerzel, Nucl. Instrum. Methods Phys. Res., Sect. A 559, 190 (2006), proceedings of the X International Workshop on Advanced Computing and Analysis Techniques in Physics Research.
  * Chen and Guestrin (2016) T. Chen and C. Guestrin,  (2016), proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, arXiv:1603.02754 [cs.LG] .