# OralViewer: 3D Demonstration of Dental Surgeries for Patient Education with
Oral Cavity Reconstruction from a 2D Panoramic X-ray

Yuan Liang  University of California, Los Angeles<EMAIL_ADDRESS>,
Liang Qiu  University of California, Los Angeles<EMAIL_ADDRESS>,
Tiancheng Lu  University of Pittsburgh<EMAIL_ADDRESS>,  Zhujun Fang
University of California, Los Angeles<EMAIL_ADDRESS>,  Dezhan Tu
University of California, Los Angeles<EMAIL_ADDRESS>,  Jiawei Yang
University of California, Los Angeles<EMAIL_ADDRESS>,  Tiandong Zhao
University of California, Los Angeles<EMAIL_ADDRESS>,  Yiting Shao
University of California, Los Angeles<EMAIL_ADDRESS>,  Kun Wang
University of California, Los Angeles<EMAIL_ADDRESS>,  Xiang ‘Anthony’ Chen
University of California, Los Angeles<EMAIL_ADDRESS>and  Lei He  University of
California, Los Angeles<EMAIL_ADDRESS>
###### Abstract.

Patient’s understanding on forthcoming dental surgeries is required by
patient-centered care and helps reduce fear and anxiety. Due to the gap of
expertise between patients and dentists, conventional techniques of patient
education are usually not effective for explaining surgical steps. In this
paper, we present OralViewer—the first interactive application that enables
dentist’s demonstration of dental surgeries in 3D to promote patients’
understanding. OralViewer takes a single 2D panoramic dental X-ray to
reconstruct patient-specific 3D teeth structures, which are then assembled
with registered gum and jaw bone models for complete oral cavity modeling.
During the demonstration, OralViewer enables dentists to show surgery steps
with virtual dental instruments that can animate effects on a 3D model in
real-time. A technical evaluation shows our deep learning based model achieves
a mean Intersection over Union (IoU) of 0.771 for 3D teeth reconstruction. A
patient study with 12 participants shows OralViewer can improve patients’
understanding of surgeries. An expert study with 3 board-certified dentists
further verifies the clinical validity of our system.

neural networks, 3D visualization, patient education

††ccs: Human-centered computing Human computer interaction (HCI) Figure 1.
Overview of OralViewer. The system takes a patient’s 2D X-ray as input (a),
and reconstructs the 3D teeth structure (b) with a novel deep learning model.
The system then generates the complete oral cavity model (c) by registering
the pre-defined models of jaw bone and gum to the dental arch curve. Finally,
a dentist can demonstrate the forthcoming surgeries to a patient by animating
the steps with our virtual dental instruments (d).

##  1\. Introduction

Patient-dentist communication is a core requirement of patient-centered care
(Asimakopoulou et al., 2014). According to (Travaline et al., 2005; Mills et
al., 2014; Stewart, 1995; Rozier et al., 2011), patients who understand their
dentists and procedures are more likely to follow medication schedules, feel
satisfied about treatments and have better oral health outcomes. Moreover,
many patients about to undergo oral surgeries can experience anxiety — up to
every fourth adult reported dental fear (Oosterink et al., 2009) and it has
been cited as the fifth-most common cause of anxiety (Agras et al., 1969)
among all kinds of anxiety. To manage the dental fear, one solution is to
unveil the surgical steps with patient education to decrease patients’ fear of
the unknown (Johnson et al., 1984; Armfield and Heaton, 2013; Bailey, 2010).
Previous studies have observed shorter duration of surgery (Elter et al.,
1997), lower level of post-operative pain (Kain et al., 2000), and smoother
recovery (Kazancioglu et al., 2015) with a reduced dental fear.

Currently, dentists mostly perform pre-operative patient education via verbal
explanation, and possibly with the aid of hand-drawn diagrams (Burghardt et
al., 2018), audiovisual slides (Choi et al., 2015) and video clips
(Kazancioglu et al., 2015). Meanwhile, the recent advent of 3D demonstration,
which illustrates complex procedures with dynamic visuals in 3D, has shown an
increasing potential in patient education since it is more intuitive and
complete than verbal description and static images. Indeed, existing studies
have explored the 3D demonstration for cardiac surgeries (Capuano et al.,
2019; Loke et al., 2019), condylar reconstruction (Yang et al., 2011) and
pancreatectomy (Endo et al., 2014) to assist the pre-operative communications.
However, to the best of our knowledge, the use of 3D demonstration for dental
clinics is still an underexplored area.

To fill this gap, we present OralViewer, a web-based system to enable dentists
to virtually demonstrate dental surgeries on 3D oral cavity models for patient
education. To inform the design of OralViewer, we interviewed three dentists
and elicited key system requirements from a clinical point of view: (i)
providing a patient-specific 3D teeth model, (ii) modeling the complete oral
cavity of teeth, gums and jaw bones, , and (iii) demonstrating surgery steps
using simple operations. In terms of 3D oral cavity modeling, OralViewer goes
beyond existing work (Badash et al., 2016; Lin et al., 2019; Welker, 2014)
that extracts a patient’s anatomy models from high-cost 3D scanning, e.g.,
computerized tomography (CT) scans. Instead, we enable the generation of 3D
models from a single 2D panoramic X-ray image with a novel deep learning
model. This approach lowers the barrier of obtaining a 3D model as the 2D
panoramic X-ray is the most common modality in dentistry and the only required
imaging for many dental surgeries (Langland et al., 2002). In terms of surgery
demonstration, OralViewer implements virtual dental instruments that are
simple to operate with a mouse and illustrative with real-time effects on oral
cavity models for patients to understand. Figure 1 shows the workflow: the
system first takes a patient’s panoramic X-ray to generate the 3D teeth
structure (a$\rightarrow$b); then pre-defined gum and jaw bone models are
registered to the dental arch, and assembled with the teeth structure for the
complete oral cavity model (c); finally, a dentist operates virtual dental
instruments on the reconstructed oral cavity to demonstrate a forthcoming
surgery to a patient (d).

We validate OralViewer for the demonstration of two common dental surgeries:
crown lengthening and apicoectomy. Note that the design and implementation of
OralViewer (e.g., the 3D reconstruction process and virtual operation
techniques) are expected to generalize to other dental surgeries as well.
These two surgeries were selected because each of them involves multiple steps
and and requires various commonly-used dental instruments, thus are ideal for
testing the demonstration capability of OralViewer. We conducted three
evaluations: (i) A technical evaluation of 3D teeth reconstruction from 2D
panoramic X-ray shows our model achieves an average IoU of 0.771$\pm$0.062.
(ii) A study with 12 patient indicates that our system leads to patients’
improved understanding of dental surgeries. (iii) An expert study with 3
board-certificated dentists suggests that the demonstration using our system
is clinically valid, can improve the efficiency of patient education, yet
there remain areas for improvement in the ease of operation of the virtual
tools.

Contributions of this paper include:

  * •

OralViewer — the first solution that enables 3D demonstration of dental
surgeries for patient education;

  * •

3D modeling — the first 3D reconstruction technique of oral cavity from a
single 2D panoramic X-ray;

  * •

Evaluation — a patient study and an expert study validate the feasibility and
usability of educating patients with 3D simulative demonstration.

##  2\. Background

In this section, we briefly introduce the common steps in typical apicoectomy
and crown lengthening surgeries. Detailed information about both dental
surgeries can be found in (Gutmann and Lovdahl, 2010), and more descriptions
with figures are included in Supplementary Material 1.1.

Apicoectomy is the removal of the root tip and surrounding tissues of a tooth
with periapical inflammation. A dentist first performs a periodontal flap —
incises and flaps the gum tissue for unveiling the underlying bone structure
with scalpels. Next, the apex is exposed by creating an peripheral opening on
the buccal (jaw) bone with a round bur and a steady stream of saline solution.
After that, the inflamed root tip can be resected with a handpiece, followed
by filling material into the tooth cavity created to seal it. Then, bone
grafting materials can be injected into the jaw bone hole for rehabilitation,
and finally the periodontal flap being repositioned and sutured.

Crown Lengthening can be applied for restoring cavities and tooth fractures
that happen below the gum tissue. To start, a dentist incises and flaps the
gum tissue to unveil the target structure. Next, the jaw bone’s height at a
surrounding area is often reduced with a bur, in order to support the
repositioned gum in a lower position below the cavity/fracture. Then, the
cavity is removed (or fracture shaped) with a handpiece, and restored with
grafting materials. For better protecting the tooth, the restored crown is
sometimes further shaped (with handpieces), and cemented with an artifact
crown.

##  3\. Related Work

In this section, we first review existing work on 3D surgical visualization
and simulation. Then, we summarize the 3D-based Computer-Aided Design (CAD)
technologies for dentistry. We also include a review on the deep learning
based algorithms for 3D reconstruction from a single 2D view.

###  3.1. Surgical Visualization and Simulation

Conventional techniques of delivering education to patients through verbal
instructions may not be effective to explain surgical procedures due to the
educational barriers between the patients and the clinicians. Researches have
shown 3D anatomy visualization can improve the patients’ understanding of
surgeries, where example systems include those for abdominal (Lin et al.,
2019), cardiac system (Whitman and Dufeau, 2017; Olivieri et al., 2018), and
more (Welker, 2014). Moreover, with the advent of computer graphics,
interactive manipulation of virtual 3D models has shown to help patients
acquire a more satisfactory level of knowledge (Prinz et al., 2005; Endo et
al., 2014; Eschweiler et al., 2016). Recently, studies (Shirk et al., 2019;
Satava, 1993) have also incorporated virtual reality (VR) to enable more
intuitive anatomy viewing.

In comparison, OralViewer is different from all the aforementioned work on two
aspects. First, all the 3D anatomy models used are either captured from 3D
scanning, e.g., CT, or utilizing a one-size-fits-all standard model. However,
considering the limited availability of 3D imaging for dental surgeries,
OralViewer generates the detailed oral cavity model from a single 2D panoramic
X-ray with a novel 3D reconstruction algorithm. Second, to the best of our
knowledge, no existing study has enabled the 3D demonstration of dental
surgeries for patient education, which we explore in the design and
implementation of OralViewer.

###  3.2. Computer-Aided Design (CAD) for Dentistry

CAD tools have been widely applied in dentistry to improve the design of
dental restorations, e.g. crowns, dental implants and orthodontic appliances
(Davidowitz and Kotick, 2011; Rekow, 1987). Specifically, models of patients’
oral cavity are created from digital 3D scanning, based on which dentists
produce a virtual design of restorations for manufacturing (Oen et al., 2014).
However, all the CAD tools are aimed to guide a clinician through restoration
designing (Strub et al., 2006; Taneva et al., 2015), rather than patient
education, which is the focus of OralViewer. Thus, oral cavity visualization
and surgical step simulation have not been considered in existing CAD tools
when it comes to patient education. Moreover, 3D imaging of patient’s oral
cavity, e.g., CT and intra-oral scanning (Davidowitz and Kotick, 2011; Raja’a
and Farid, 2016), is almost always required by the existing CAD tools. In
contrast, OralViewer reconstructs the patient’s 3D oral cavity from the 2D
panoramic X-ray, which is one of the most common imaging modalities in
dentistry (Whaites and Drage, 2013), in order to enable the application of the
system for a wide range of dental surgeries.

###  3.3. Single-View 3D Reconstruction

Single-view 3D reconstruction aims at generating the 3D model of an object
based on a single 2D projection of it. Currently, deep Convolutional Neural
Networks (ConvNets) based methods have achieved the highest accuracy in
various benchmarks by using both low-level image cues, e.g., texture, and
high-level semantic information (Tatarchenko et al., 2019; Xian et al., 2018).
According to the representation type of 3D outputs, most existing work can be
categorized into: (i) voxel-based (Choy et al., 2016; Girdhar et al., 2016;
Tatarchenko et al., 2017), (ii) mesh-based (Wang et al., 2018; Groueix et al.,
2018; Mescheder et al., 2019), and (iii) point-cloud-based (Tatarchenko et
al., 2016; Lin et al., 2017). A detailed review of the above categories of
methods can be found in (Tatarchenko et al., 2019). Our work targets at
generating the voxel-based representation of teeth volumes, which estimates a
voxel occupancy grid for indicating if voxels are within the space of an
object. The representation selection mainly considers the need for smooth and
closed-surface models, even with the presence of complex typologies on
occluded surfaces. A few existing work (Abdelrahim et al., 2012; Abdelrehim et
al., 2013; Liang et al., 2020) explored teeth reconstruction from X-ray,
however, they either targeted at single tooth or worked with synthesized
images only, which cannot serve our propose of patient-specific modeling and
demonstration. To the best of our knowledge, ours is the first work on
exploring 3D reconstruction of teeth structures from clinical 2D panoramic
X-rays.

##  4\. Formative Study

To understand the system requirements of OralViewer from a clinical point of
view, we conducted interviews with three dentists (two female and one male).
We started by asking for the method he/she applies to perform patient
education. We then described the motivation and goal of OralViewer,
emphasizing on using a 3D model to visualize and simulate surgical steps to
laymen patients. We gathered and built on dentists’ feedback to formulate the
below system requirements that ensure clinically-valid demonstration and user-
friendliness.

R1. Providing patient-specific teeth model. Surgical steps, e.g. how a
fractured tooth is extracted or repaired, often depend on individual’s teeth
condition. Thus, patient-specific teeth model should be provided to make
demonstrations contextualized to the patient’s conditions. Moreover, compared
to panoramic oral X-ray, 3D screening of oral cavity is not a standard
practice for the clinical diagnosis of many common surgeries, e.g.
apicoectomy, root canal treatment, and crown lengthening, for its higher
radiation and cost. As such, it is preferred to generate a patient’s 3D teeth
model from his/her 2D X-ray image to enable the widely available application
of the system.

R2. Modeling complete oral cavity. Both the target oral structure of a surgery
and its nearby anatomies need to be incorporated into a surgical
demonstration. For example, when dentist removes a root tip in apicoectomy,
procedures on other structures should be simulated as well, e.g. some gum
tissue will be lifted from an area near the root tip and some surrounding bone
will be removed. Thus, to help patients understand what to expect in a
surgery, complete oral cavity including teeth, gum, and jaw bones should be
modeled.

R3. Demonstration in simple operations. Dentists consider it important to show
for each surgery step: (i) how the step is performed — illustrating the
applied instruments, and (ii) what happens in the step — animating the dental
structure changes upon the application of instruments. Moreover, the
demonstration should be carried out by dentists using simple interaction
techniques, which is more important than having to achieve realistic effects
with a high fidelity. For example, to demonstrate shaping a tooth with a
dental handpiece, dentists prefer a simple operation, e.g., pressing and
dragging the cursor on desired places of a tooth with customizable effect to
simulatively perform a grinding as in an actual surgery.

##  5\. OralView Design and Implementation

Guided by the aforementioned requirements, we designed and implemented
OralViewer for 3D demonstration of dental surgery to patients. The OralViewer
consists of two cascaded parts: (i) a 3D reconstruction pipeline for
generating a patient’s oral cavity from a single 2D panoramic X-ray, and (ii)
a demonstration tool for dentist’s animating steps on the 3D model with
virtual dental instruments.

###  5.1. 3D Reconstruction of Oral Cavity

OralViewer reconstructs a complete oral cavity model consisting of teeth, gum
and jaw bones, all of which are vital for the demonstration of surgical
procedures (R2). Importantly, to reflect the patient-specific dental
condition, we estimate the patient’s 3D teeth structures from a single 2D
panoramic X-ray with an end-to-end trainable deep ConvNet model (R1). Since
the 3D structures of soft tissues, i.e., gum, and jaw bones cannot be well
reflected from X-ray (Haring and Jansen, 2000), their 3D templates are pre-
defined and can be registered to tailor for specific patients’ oral anatomy.

##### Model Architecture.

Our task of teeth reconstruction has two unique challenges from the existing
voxel-based work. (i) The reconstruction contains multiple objects (teeth)
rather than a single object as in (Tatarchenko et al., 2019; Xian et al.,
2018; Choy et al., 2016). (ii) The input image of X-ray has a higher
resolution than existing work (e.g., 128$\times$128 (Choy et al., 2016)),
which calls for higher computational and memory efficiency of model. To tackle
both challenges, we decompose the task into two sub-tasks of teeth
localization and patch-wise single tooth reconstruction. Figure 2 shows the
overall architecture of our model. The input 2D panoramic X-ray
$X\in\mathbb{R}^{H\times W}$(Figure 2(a)), where $H$ and $W$ are image height
and width, is fed to a feature extraction subnet (Figure 2(a)) of a 2D
encoder-decoder structure for capturing a deep feature map of the same
resolution as the input X-ray. Given the feature map, a segmentation subnet
(Figure 2(b)), which consists of stacked convolutional layers and a $Sigmoid$
activation, maps it into a categorical mask
$Y_{\text{seg}}\in\mathbb{Z}^{H\times W\times K}$, where $K=32$ denotes the
maximum number of tooth category. Figure 4(a) demonstrates the tooth numbering
rule we used by following the World Dental Federation Notion (Hickel et al.,
2010). Moreover, the tooth localization of bounding boxes is further derived
from the segmentation map by keeping the largest island per tooth, as shown in
Figure 2(3). According to the tooth localization, a tooth reconstruction
subnet (Figure 2(c)) performs patch-sampling for all teeth from the
aforementioned deep feature map (Figure 2(2)), and back projects them into 3D
tooth shapes represented in 3D occupancy map (Figure 2(4)), using a
2D-encoder-3D-decoder structure similar to (Choy et al., 2016). Note that the
deep feature map (Figure 2(2)) is shared for both segmentation and tooth
reconstruction sub-tasks in order to increase the compactness and
generalization of the model. By assembling the predicted tooth volumes (Figure
2(4)) according to their estimated localization from X-ray segmentation
(Figure 2(3)), we can achieve a flatten 3D reconstruction of teeth. The
flatten reconstruction is then bent to an estimated dental arch curve (Figure
2(5)) for the final 3D teeth reconstruction as shown in Figure 2(6)). The
parameters in all the subnets of the model (Figure 2(a,b,c)) can be optimized
in an end-to-end fashion for the optimal performance. The training strategy
and dataset are described as below. More details about the model can be found
in Supplementary Material 1.2.

Figure 2. Overall architecture for reconstructing 3D tooth structures from a
panoramic X-ray.  Figure 3. Semi-automatic dental arch curve estimation from
occlusion photos.

##### Unsupervised Dental Arch Curve Estimation.

The dental arch curve needs to be estimated since such information is lost
during the circular rotational screening process of a panoramic X-ray imaging
system (Haring and Jansen, 2000). Multiple methods can be applied for the
estimation, e.g., using average shape from general population (Muhamad et al.,
2015), and $\beta-$curve fitting with measured width and depth of oral cavity
(Braun et al., 1998). In this work, we propose a semi-automatic pipeline to
accurately extract dental arch curve from occlusal surface photos without
supervision, as shown in Figure 3. First, macro shots of upper and lower
occlusal surfaces are taken from a patient (Figure 3(a)), and are roughly
labeled for teeth regions with simple sketching (Figure 3(b). The dental arch
area can then be obtained with the active contour algorithm (Williams and
Shah, 1992) applied on the sketches as priors (Figure 3(c)), and further
skeletonlized for an initial dental arch curve (Figure 3(d)). The final smooth
dental arch curve is achieved by fitting a cubic curve to the uniformly
sampled data points from the initial curve (Figure 3(e)).

##### Complete Oral Cavity Model.

To achieve a complete oral cavity model, OralViewer is embedded with a set of
pre-built template models for gums and jaw bones. As shown in Figure 4(b), to
build the templates, a Cone Beam CT from an adult male was collected 4(b)(a),
and pre-processed with intensity thresholding (Naumovich et al., 2015) for
extracting the skull structure 4(b)(b). Then, the upper jaw (Figure 4(b)(c1))
and lower jaw (Figure 4(b)(d1)) bone models were constructed by removing tooth
structures, hole filling, and smoothing, while the upper gum (Figure 4(b)(e1))
and low gum (Figure 4(b)(f1)) models were constructed as the smooth volumes
embodying the corresponding jaw bones. For the reconstruction for each patient
in the deployment stage, the pre-built gum and jaw bone models are first
registered and aligned to the estimated dental arch curves (Figure 4(b)(g)).
Then the deformed gum and jaw bones model (Figure 4(b)(c2,d2,e2,f2)) are
assembled with the 3D reconstructed teeth for the complete oral cavity model
as the example shown in Figure 4(b)(i). We expect the averaging models of gum
and jaw bone from CT scans of multiple individuals can further improve the
reconstruction quality, while the current templates have been shown valid for
the surgical demonstration purpose according to dentists as detailed in the
Expert Study section.

(a)

(b)

Figure 4.  (a) Teeth naming, numbering and coloring rule. (b) Gum and jaw bone
templates building method, and their application in the deployment stage for a
complete oral cavity model.

##### Training Dataset.

OralViewer utilizes a deep ConvNet for estimating patient-specific 3D teeth
structure from a 2D panoramic X-ray. Intuitively, the model can be trained
with the supervision of patients’ paired data of 3D teeth structures obtained
from the teeth labeling of CT and panoramic X-ray. However, tooth structures
from X-ray and CT are misaligned due to different postures during screening,
e.g. head directions and occlusion condition. As such, we propose to collect
CT scans, and synthesize their corresponding panoramic X-rays as the model
input. The synthesis is valid since the CT scans contain full 3D information
of oral cavity, while panoramic radiographs are the 2D projections of them.
Several previous work has demonstrated promising results for high quality
X-ray synthesis from CT, and we employed the method from Yun et al. (Luo et
al., 2016) in this work. In total, our in-house dataset contains 23 pairs of
3D CT scans and synthesized panoramic X-rays. Moreover, another 39 real
panoramic X-ray scans have also been included for augmenting the training
process of the segmentation subnet of the model. All CBCT scans and panoramic
X-rays have been first labeled with pixel-wise tooth masks by 3 research team
members, and then reviewed by 2 board-certificated dentists. We randomly split
the paired data into 21 scans for training, and 2 scans as the validation set
for determining the early stopping of the training process.

##### Training Strategy.

We employed a two-stage training paradigm. In the first stage, we train the
deep feature extraction subnet and segmentation subnet (Figure 2(a,b)) for
tooth localization, which servers as the starting point for the training of
tooth reconstruction subnet since it requires tooth patch sampling ability.
For the segmentation loss $L_{\text{seg}}$, we define it to be the average of
dice loss across all tooth categories. In the second stage, we train the whole
model including tooth reconstruction subnet (Figure 2(c)) for the optimal
performance of both tooth localization and reconstruction, with the loss
defined as the sum of the aforementioned segmentation loss $L_{\text{seg}}$
and a reconstruction loss $L_{\text{recon}}$. The $L_{\text{recon}}$ was set
as the 3D dice loss between the ground-truth volume and the predicted volume.
More details about model training can be found in Supplementary Material 1.3.

###  5.2. Demonstrating Surgery with Virtual Dental Instruments

OralViewer provides a web-based 3D demonstration tool for dentists to
demonstrate surgery steps on a patient’s oral cavity model with virtual dental
instruments. The dental instruments allow dentists to express what and where
an action is applied to the oral cavity, and demonstrate the effect on the
model in real-time (R3). Moreover, dentists can use simple sliders to
customize the animation effect of the instruments to better suit their
particular needs and preferences. By discussing with the dentists, the current
tool’s implementation consists of six common dental instruments : (1) surgical
scalpel, (2) fissure bur, (3) handpiece, (4) syringe, (5) curette, and (6)
artifacts. In this section, we first show the overall workflow of using the
demonstration tool. We then describe the technical details of each virtual
dental instrument, followed by the tool implementation.

##### Overall Workflow.

Figure 5. Overall workflow for demonstrating dental surgeries.

As shown in Figure 5(a), a dentist start with importing a reconstructed 3D
oral cavity model generated from the aforementioned pipeline (Figure 5(1)),
which can be viewed freely with rotation and scaling. To apply a virtual
dental instrument, the dentist selects the instrument from a list (Figure
5(2)). Upon the selection, the corresponding instrument model (Figure 5(4,6))
is visualized, and can be controlled by using the mouse to move and operate on
the oral cavity model. For instruments 1-5, e.g., scalpel as shown in Figure
5(b), their animation effect on dental structures can be customized by
changing a set of parameters (Figure 5(3)); while for dental artifacts, e.g.,
implant as shown in Figure 5(c), their shapes and directions can also be
adjusted to tailor for the patient’s condition (Figure 5(5)). The selected
instrument can be directly applied to a dental structure for demonstrating
effects with clicking, pressing, and dragging (Figure 5(d)). The oral cavity
model can also be freely rotated by press-and-drag and scaled with wheel to
adjust an optimal view for the demonstration and manipulation. The effect can
be dynamically reflected on the structure in real-time, with the operated
structure highlighted for visualization (Figure 5(7)). A typical dental
surgery consists of several sequential steps using multiple dental
instruments, which can be demonstrated on a 3D oral cavity model by operating
with each dental instrument following the aforementioned steps of instrument
selection (Figure 5(a)), adjusting (Figure 5(b,d)), and animating (Figure
5(d)).

##### #1 Surgical Scalpel & #2 Fissure Bur.

Figure 6. Example use cases of virtual surgical scalpel (top left), fissure
bur (bottom left), handpiece for tooth tip resection (top right) and jaw bone
lowering (bottom right).

Both tools are used to incise dental structures: surgical scalpel can be
applied to gum tissue for the creation of periodontal flap; while fissure bur
can create holes on jaw bones for the exposure of the root tip. To use the
tools, a dentist describes the desired incision location and its size by
creating a region from a sequence of mouse clicks, where the created boundary
is visualized with red lines (Figure 6(b,f)). Upon a closed-loop boundary is
formed, the incised region is highlighted in green (Figure 6(c,g)), and the
corresponding part of the dental structure is then removed upon the dentist’s
confirmation. The design allows dentists to perform any type of incision
according to a patient’s condition, e.g., semilunar and triangular types of
periodontal flap. Figure 6(a$\rightarrow$d) shows a periodontal flap example
using surgical scalpel, and Figure 6(e$\rightarrow$h) shows the creation of
jaw bone opening based on a flapped gum.

##### #3 Handpiece.

Handpiece is widely used for shaping bone structures, e.g., tooth and jaw
bones. A dentist can move the virtual handpiece as mouse cursor to any desired
location of grinding. The grinding effect takes place once with a mouse click,
or continuously by pressing and dragging the mouse. The size and intensity of
the grinding effect can be customized using sliders. For example, Figure
6(i$\rightarrow$j) shows the resection of an exposed root tip in an
apicoectomy; while Figure 6(k$\rightarrow$l) demonstrates it is applied to
reduce the upper jaw bone height in a crown lengthening surgery.

##### #4 Syringe & #5 Curette.

Figure 7. Example use cases of syringe for injecting grafting materials (top
left), curette for restoring fractured crown (bottom left), applying an
artifact implant (top right), and cementing an artifact crown (bottom right).

Both instruments can be used to fill materials into holes of bone structures,
e.g., teeth and jaw bones. Similarly, a dentist can move either instrument as
a cursor to the desired location on the tooth or jaw bone structures and click
or press-and-drag for the filling effect. The size and intensity of the
filling effect can be customized using sliders. For example, Figure
7(a$\rightarrow$b) shows the syringe is used to inject bone grafting materials
to fill the jaw bone opening, and Figure 7(c$\rightarrow$d) demonstrates a
curette is applied on a fractured crown for restoration.

##### #6 Artifacts.

Artifact crown and implant are included in the demonstration tools. To apply
them, a dentist starts with importing a pre-defined artifact model, followed
by specifying the dental structure of the artifact to be applied on. The
artifact model and the dental structure are visualized in blue and red (Figure
7(e,g)) for assisting the operations and demonstration. The 3D location of the
artifact can be adapted with dragging in 2D viewing planes , while both the
orientation and size of the artifact can be modified to match the patient’s
condition using sliders from X, Y, and Z axis. Once the artifact has been
customized with confirmation from dentists, the artifact model and the
operated dental structure are merged as one object. Figure 7 (e$\rightarrow$f,
g$\rightarrow$h) shows the cementation of an implant on a resected root and a
dental crown on a prepared tooth, respectively, as indicated with red arrows.

##### Implementation.

The demonstration tool of OralViewer was implemented using OpenGL, JavaScript
and three.js. The tool can run readily inside a modern browser. The effect of
gum/jaw bone incision with surgical scalpel/fissure bur and artifact
implanting was implemented with the Constructive Solid Geometry (CSG)
operations111https://github.com/oathihs/ThreeCSG between: (i) generated 3D
convex geometry from the input trajectories or pre-built artifact models, and
(ii) corresponding dental structures. The effect of shaping/filling with
handpiece/syringe/curette was implemented with mesh sculpting
operations222https://stephaneginier.com/sculptgl/ including flattening,
filling, and scraping.

##  6\. Evaluation

We conducted (i) a technical evaluation of the reconstruction pipeline for
generating a 3D model of a patient’s oral cavity, (ii) a patient study with 12
participants, and (iii) an expert study with three board-certified dentists.

###  6.1. Technical Evaluation

##### Dataset.

We built an in-house testing dataset collected from 10 patients from a local
orthodontics hospital. Each patient was screened for both panoramic X-ray and
Cone Beam CT. Moreover, since our reconstruction pipeline estimates patient-
specific dental arch curves, two photos of occlusion macro photos (for upper
and low jaws respectively) of each patient were captured by dentists in the
clinic. To quantitatively evaluate the reconstruction accuracy, patients’
tooth structures were manually labeled from CT scans by two research team
members and reviewed by one dentist.

##### Results & Analysis.

We applied our reconstruction pipeline with the panoramic X-ray and occlusion
photos as input to generate the complete 3D oral cavity model. We first
evaluate the teeth reconstruction accuracy for reflecting patient-specific
condition by quantitatively comparing the generated model with labeled 3D
structure from CT. We used the Intersection over Union (IoU) between the
predicted volume $P$ and the ground-truth volume $G$ of each tooth as the
performance metric, where $IoU=|P\cap G|/(|P\cup G|)$, and $|.|$ denotes the
cardinality of a voxel set. To overcome the misalignment of teeth between
X-ray and CT caused by patients’ gestures, the upper and lower teeth labeling
from CT (3D) was rigidly aligned (using ANTs (Avants et al., 2009)) to the
corresponding part of the reconstructed 3D teeth (3D) from the model before
the IoU calculation. Figure 8(a) reveals the IoUs for all the 32 categories of
tooth, where our model achieves a mean IoU of 0.771, with patient-wise
$std=0.031$ and tooth category-wise $std=0.062$. The result indicates the
effectiveness of our method for reconstructing 3D teeth model from 2D x-ray
image. Yet, we also find that the method has a consistent lower accuracy for
wisdom teeth (numbering 18, 28, 38, and 48) than the others, which can be
caused by: (i) the shape of wisdom teeth is more variant across different
persons, and (ii) relatively fewer samples are available for ConvNet training
(16 out of 23 cases in the training dataset containing one or more wisdom
teeth).

We then qualitatively evaluate the 3D reconstruction of the complete oral
cavity model. Figure 9 visualizes an example case from the testing dataset:
the input panoramic X-ray (a), the semi-automatic dental arch extraction
results for both upper and lower jaws (b), the reconstructed 3D teeth model
from the 2D X-ray and the estimated dental arch curves (c), the ground-truth
teeth structures extracted from the CT image (d), and the complete oral cavity
model with the pre-defined jaw bone and gum models registered and assembled
with the reconstructed teeth (e). Note that the shown case has 24 teeth,
rather than 28 teeth as a normal adult. This is possibly because 4 teeth have
been extracted during a previous orthodontic operation. The reconstructed
results clearly reflect this individual’s condition, and show the
effectiveness of our reconstruction pipeline for generating 3D oral cavity
models.

(a)

(b)

Figure 8.  (a): 3D teeth reconstruction accuracy. (b) A participant was
demonstrated a dental surgery using OralViewer by a dentist.  Figure 9.
Reconstruction example from testing dataset.

###  6.2. Patient Study

OralViewer enables a dentist to help laymen patients understand procedures of
dental surgeries by 3D demonstration. To validate the feasibility of our
approach, we investigate the research question concerning the effect of
OralViewer:

  * •

RQ1: can OralViewer improve patients’ understanding of a surgery?

####  6.2.1. Participants

We recruited 12 patients from the clinic of an orthodontics hospital (4
females and 8 males, aged between 21 to 64 years). Each participant was
demonstrated with one surgery of crown lengthening or apicoectomy. None of the
participants had received the dental surgery before. Note that due to limited
patient resources, we were only able to recruit participants who came in for
regular dental check-up but did not actually undergo such dental surgeries.
The detailed demographic information of our participants can be found in Table
1 from Supplementary Material 1.4.

####  6.2.2. Procedure

The patient study consists of the following key activities:

Tutoring dentists how to use the system. To clinically validate OralViewer, we
collaborated with three board-certified dentists (E1: male, 25 years of
practice; E2: male, 17 years of practice, E3: female, 11 years of practice)
for carrying out the surgery demonstrations. We first introduced OralViewer,
let each dentist follow a step-by-step onboarding tutorial, and answered their
questions about utilizing the program. Then dentists were free to continue
trying out OralViewer’s virtual dental instruments until they felt they were
able to use the system independently.

In-clinic study. In order to compare OralViewer with current patient education
method, we randomly split the participants into an experiment group of 7,
which was demonstrated OralViewer, and a control group of 5, which was
demonstrated an X-ray and verbal descriptions as per the dentists’ regular
practice. Participants were randomly assigned one of the two dental surgeries,
e.g., apicoectomy or crown lengthening, to receive a demonstration: 4
participants in the experiment group and 3 in the control group were
demonstrated apicoectomy; while 3 in the experiment group and 2 in the control
group were demonstrated for crown lengthening. Each study happened in one of
the three dentists’ clinics and OralViewer was accessed as a web app using
dentists’ own computers (Figure 8(b)) . Details on the surgery and dentist
assignment for each participant can be found in Table 1 from Supplementary
Material 1.4.

Exit interview. After the explanation, we interviewed each participant to
verbally describe the surgery procedures by focusing on: (i) what steps are
included, and (ii) how (by using what instrument) a procedure is applied.
Their answers were recorded and later compiled for evaluating their
understanding of surgeries.

####  6.2.3. Analysis & Results

We scored the participants’ understanding on a surgery based on whether the
key steps of the surgery were described in their answers. Specifically, there
are five key steps for apicoectomy: (S1) periodontal flap, (S2) jaw bone
opening, (S3) root tip removal, (S4) root tip sealing, and (S5) grafting
material injection; while five key procedures were considered for crown
lengthening: (S1) periodontal flap, (S2) jaw bone shaping, (S3) tooth
preparing, and (S4) artifact implanting. An answer regarding a step is scored
as: 0 if the step was not described, 1 if the step was described but the
applied dental instrument was not, and 2 if both the step and its dental
instrument were mentioned or described. Note that the exact names of
step/instrument were not required to be mentioned in an answer for a score — a
step/instrument was counted if it was described or indicated from a patient’s
answer. Figure 10 shows the average score for each step within the experiment
and control group for the apicoectomy (Figure 10(a)) and crown lengthening
(Figure 10(c)). We can see that OralViewer significantly improves patients’
understanding in three out of the five steps for apicoectomy and two out of
the four steps in crown lengthening, while the improvement in the other steps
is not statistically significant. Moreover, the overall average score among
all steps between the experiment and control group was 1.36 vs. 0.85, which
also indicates OralViewer can significantly improve the patients’
understanding with $p<0.05$.

Figure 10. Participants’ understanding score distribution for (a) apicoectomy
and (b) crown lengthening. Sx represents key steps within the two surgeries.

###  6.3. Expert Study

To clinically validate OralViewer, we interviewed 3 dentists from the patient
study after they finished all the assigned patient demonstrations. Each
dentist had done at least one demonstration using OralViewer for both types
surgeries (E1: 3 times; E2: 2 times; E3 2 times. Details see Supplementary
Material 1.4). We investigate the following questions:

  * •

RQ2: Usability — do dentists have difficulty using virtual dental instruments
in OralViewer?

  * •

RQ3: Validity — is OralViewer’s demonstration effect clinically valid for
patient education?

  * •

RQ4: Preference — do dentists prefer a system like OralViewer as a tool for
performing patient education for surgeries?

We asked each dentist to rate their agreement (from 1-strongly disagree to
7-strongly agree) with statements about the usability and demonstration effect
of the instruments and the resultant demonstration effect of 3D oral cavity
models (RQ2 & 3), as well as their preference (RQ4).

Figure 11. Experts’ scores for OralViewer about (a) usability, (b)
demonstration effect validity, and (c) preference.

####  6.3.1. Usability

We measured the usability for the four types of virtual instruments involved
in the demonstration of apicoectomy and crown lengthening: (S1) surgical
scalpel and fissure bur for gum/jaw incision, (S2) handpiece for jaw/tooth
shaping, (S3) syringe and curette for tooth filling/material injection, and
(S4) dental artifacts for implanting. Figure 11(a) shows the questions and
experts’ scores. While all the experts successfully carried out all the
demonstrations with patients using OralViewer, a major issue raised by experts
was that the virtual instrument control with mouse was unfamiliar to dentists:
it is different from the way dentists using real dental instruments in a
surgery, which can lead to a steep learning curve (E2,E3). E3 suggested that
an implementation of OralViewer on touch screen, e.g., iPad, with the control
of virtual instruments using stylus should be more intuitive to dentists.

####  6.3.2. Validity

The experts rated the demonstration effect of OralViewer for the 3D oral
cavity model (S1) and the four types of virtual instruments (S2-S5). Figure
11(b) shows the questions and experts’ ratings, where OralViewer achieves a
mean score of 5.67 out of 7. All experts agreed that our reconstructed oral
cavity model and virtual instruments are clinically valid for patient
education. Regarding the oral cavity model, experts confirmed that it
contributes to the surgical demonstration because (i) patients are able to see
structures that they cannot observe from a mirror, e.g., molar teeth, using
rotation in 3D (E1, E2, E3); and (ii) the patient-specific teeth can not only
let patients understand their conditions better (E2, E3) but also raise their
interests in learning more about operating on such conditions (E3). Besides,
E3 suggested that oral cavity model can be improved by modeling root canals
within tooth (detail in the Discussion section). Regarding virtual
instruments, experts agreed that they are valid for the patient education
purpose (E1, E2, E3), and preferred the visualization of the instruments,
which can help patients comprehensively understand what to expect during a
surgery (E3). Moreover, experts also suggested that the appearance of virtual
instruments can be dynamic altered upon users customizing their effect to
further improve the visualization (detailed in the Discussion section).

####  6.3.3. Preference

We asked experts about their preference of OralViewer from two perspectives:
(S1) OralViewer enables effective patient education and (S2) I would integrate
OralViewer into my existing practice. As shown in Figure 11(c), the experts
agreed that OralViewer enables effective patient education with a mean score
of 6.00 out of 7. The experts also rated agreement of 5.00 out of 7 for
integrating OralViewer into their existing practices. As mentioned by E2, the
tool can be very necessary with the patients’ recently growing need for
improved dentist visit experience and their willingness to involve in
treatment planning. He also pointed out that animating procedures on patient-
specific model can possibly contribute to higher patient satisfaction because
of the personalized communications. While agreeing on OralViewer’s
effectiveness of patient education, E3 mentioned that an improved virtual
instrument control design can gain a higher preference from dentists for more
fluently utilizing the system.

##  7\. Limitations, Discussions, and Future Work

###  7.1. Improving Oral Cavity Modeling

We enable 3D oral cavity modeling by assembling patient-specific teeth
structures estimated from X-ray and registered gum and jaw bone templates.
However, two improvements can be made according to interviews from the expert
study e.g., the current gum templates as the smooth volumes embodying a set of
CT-extracted jaw bones are reported to be coarse in form (E3). Although such
approximation is sufficient for patient education, E3 suggested that improved
gum templates should be pre-built from an existing intra-oral scanning, which
is a dental imaging modality that is capable of capturing soft tissue.
Furthermore, the current reconstructed teeth do not model root canals, which
can be useful in certain surgeries, e.g., root canal treatment (E3). Future
work can enable the root canal modeling by either augmenting the current solid
teeth model with artifact canals or including the root canal modeling in the
ConvNet training process.

###  7.2. Extending Virtual Instrument Set for All Dental Surgeries

The current implementation of OralViewer includes 6 common dental instruments,
which can be applied to conduct various virtual surgeries including wisdom
tooth extraction, apicoectomy, crown restoration and lengthening, etc.
However, more virtual instruments are required towards a comprehensive system
for all dental surgeries, e.g., endodontic files for root canal treatment.
Since most virtual instrument effect can be simulated with CSG (Constructive
Solid Geometry) operations, e.g., Boolean operation between two models, and
mesh sculpting, e.g., smoothing, creasing, and flattening, we suggest that
OralViewer should be extended to allow dentists to register new virtual dental
instruments. Specifically, various CSG and mesh sculpting functions can be
implemented and a dentist can add a new instrument by importing a instrument
model for visualization and its parameters for configuring the operating
effects.

###  7.3. Improving Dental Instrument Visualization

OralViewer visualizes a model of the selected dental instrument for enhancing
patients’ understanding. However, expert E3 suggested that the visualization
should be improved from two aspects. First, when the animation effect of the
instrument is changed, the appearance of the instrument can be dynamically
altered to reflect the effect change. For example, the head of a handpiece can
become larger when the effecting size of grinding is set to be larger. Second,
different models of an instrument can be pre-defined for dentists’ selection
to reflect the real surgery situation. An example as E3 mentioned is that,
there are multiple periapical curette tips of different shapes, each of which
is applied according to periapical cavity access condition in real surgeries.

## References

  * (1)
  * Abdelrahim et al. (2012) Aly S Abdelrahim, Moumen T El-Melegy, and Aly A Farag. 2012\.  Realistic 3d reconstruction of the human teeth using shape from shading with shape priors. In _2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops_. IEEE, 64–69.
  * Abdelrehim et al. (2013) Aly S Abdelrehim, Aly A Farag, Ahmed M Shalaby, and Moumen T El-Melegy. 2013.  2D-PCA shape models: Application to 3D reconstruction of the human teeth from a single image. In _International MICCAI Workshop on Medical Computer Vision_. Springer, 44–52.
  * Agras et al. (1969) Stewart Agras, David Sylvester, and Donald Oliveau. 1969\.  The epidemiology of common fears and phobia.  _Comprehensive psychiatry_ 10, 2 (1969), 151–156.
  * Armfield and Heaton (2013) Jason M Armfield and LJ Heaton. 2013.  Management of fear and anxiety in the dental clinic: a review.  _Australian dental journal_ 58, 4 (2013), 390–407.
  * Asimakopoulou et al. (2014) Koula Asimakopoulou, Adyya Gupta, and Sasha Scambler. 2014\.  Patient-centred care: barriers and opportunities in the dental surgery.  _Community dentistry and oral epidemiology_ 42, 6 (2014), 603–610.
  * Avants et al. (2009) Brian B Avants, Nick Tustison, and Gang Song. 2009\.  Advanced normalization tools (ANTS).  _Insight j_ 2, 365 (2009), 1–35.
  * Badash et al. (2016) Ido Badash, Karen Burtt, Carlos A Solorzano, and Joseph N Carey. 2016.  Innovations in surgery simulation: a review of past, current and future techniques.  _Annals of translational medicine_ 4, 23 (2016).
  * Bailey (2010) Laila Bailey. 2010\.  Strategies for decreasing patient anxiety in the perioperative setting.  _AORN journal_ 92, 4 (2010), 445–460.
  * Braun et al. (1998) Stanley Braun, William P Hnat, Dana E Fender, and Harry L Legan. 1998.  The form of the human dental arch.  _The Angle Orthodontist_ 68, 1 (1998), 29–36.
  * Burghardt et al. (2018) Sophia Burghardt, Susan Koranyi, Gabriel Magnucki, Bernhard Strauss, and Jenny Rosendahl. 2018\.  Non-pharmacological interventions for reducing mental distress in patients undergoing dental procedures: Systematic review and meta-analysis.  _Journal of Dentistry_ 69 (2018), 22–31.
  * Capuano et al. (2019) Francesco Capuano, Yue-Hin Loke, Ileen Cronin, Laura J Olivieri, and Elias Balaras. 2019.  Computational study of pulmonary flow patterns after repair of transposition of great arteries.  _Journal of biomechanical engineering_ 141, 5 (2019).
  * Choi et al. (2015) Sung-Hwan Choi, Ji-Hoon Won, Jung-Yul Cha, and Chung-Ju Hwang. 2015.  Effect of audiovisual treatment information on relieving anxiety in patients undergoing impacted mandibular third molar removal.  _Journal of Oral and Maxillofacial Surgery_ 73, 11 (2015), 2087–2092.
  * Choy et al. (2016) Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 2016.  3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In _European conference on computer vision_. Springer, 628–644.
  * Davidowitz and Kotick (2011) Gary Davidowitz and Philip G Kotick. 2011.  The use of CAD/CAM in dentistry.  _Dental Clinics_ 55, 3 (2011), 559–570.
  * Elter et al. (1997) John R Elter, RONALD P STRAUSS, and JAMES D BECK. 1997\.  Assessing dental anxiety, dental care use and oral status in older adults.  _The Journal of the American Dental Association_ 128, 5 (1997), 591–597.
  * Endo et al. (2014) Kazuhiro Endo, Naohiro Sata, Yasunao Ishiguro, Atsushi Miki, Hideki Sasanuma, Yasunaru Sakuma, Atsushi Shimizu, Masanobu Hyodo, Alan Lefor, and Yoshikazu Yasuda. 2014\.  A patient-specific surgical simulator using preoperative imaging data: an interactive simulator using a three-dimensional tactile mouse.  _Journal of Computational Surgery_ 1, 1 (2014), 1–8.
  * Eschweiler et al. (2016) Jörg Eschweiler, Jan-Philipp Stromps, Maximilian Fischer, Fabian Schick, Björn Rath, Norbert Pallua, and Klaus Radermacher. 2016\.  A biomechanical model of the wrist joint for patient-specific model guided surgical therapy: Part 2.  _Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine_ 230, 4 (2016), 326–334.
  * Girdhar et al. (2016) Rohit Girdhar, David F Fouhey, Mikel Rodriguez, and Abhinav Gupta. 2016.  Learning a predictable and generative vector representation for objects. In _European Conference on Computer Vision_. Springer, 484–499.
  * Groueix et al. (2018) Thibault Groueix, Matthew Fisher, Vladimir G Kim, Bryan C Russell, and Mathieu Aubry. 2018.  A papier-mâché approach to learning 3d surface generation. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 216–224.
  * Gutmann and Lovdahl (2010) James L Gutmann and Paul E Lovdahl. 2010.  _Problem Solving in Endodontics-E-Book: Prevention, Identification and Management_.  Elsevier Health Sciences.
  * Haring and Jansen (2000) Joen Iannucci Haring and Laura Jansen. 2000.  _Dental radiography: principles and techniques_.  WB Saunders.
  * Hickel et al. (2010) Reinhard Hickel, Arnd Peschke, Martin Tyas, Ivar Mjör, Stephen Bayne, Mathilde Peters, Karl-Anton Hiller, Ross Randall, Guido Vanherle, and Siegward D Heintze. 2010.  FDI World Dental Federation: clinical criteria for the evaluation of direct and indirect restorations—update and clinical examples.  _Clinical oral investigations_ 14, 4 (2010), 349–366.
  * Johnson et al. (1984) Steven Johnson, Kathleen Chapman, and Gene Huebner. 1984\.  Stress reduction prior to oral surgery.  _Anesthesia progress_ 31, 4 (1984), 165.
  * Kain et al. (2000) Zeev N Kain, Ferne Sevarino, Gerianne M Alexander, Sharon Pincus, and Linda C Mayes. 2000\.  Preoperative anxiety and postoperative pain in women undergoing hysterectomy: A repeated-measures design.  _Journal of psychosomatic research_ 49, 6 (2000), 417–422.
  * Kazancioglu et al. (2015) Hakki Oguz Kazancioglu, Mustafa Tek, Seref Ezirganli, and Nihat Demirtas. 2015.  Does watching a video on third molar surgery increase patients’ anxiety level?  _Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology_ 119, 3 (2015), 272–277.
  * Langland et al. (2002) Olaf E Langland, Robert P Langlais, and John W Preece. 2002\.  _Principles of dental imaging_.  Lippincott Williams & Wilkins.
  * Liang et al. (2020) Yuan Liang, Weinan Song, Jiawei Yang, Liang Qiu, Kun Wang, and Lei He. 2020\.  X2Teeth: 3D Teeth Reconstruction from a Single Panoramic Radiograph. In _International Conference on Medical Image Computing and Computer-Assisted Intervention_. Springer, 400–409.
  * Lin et al. (2019) Chen Lin, Junyi Gao, Hua Zheng, Jun Zhao, Hua Yang, Yue Zheng, Yihan Cao, Yufei Chen, Guoliang Wu, Guole Lin, et al. 2019\.  When to introduce three-dimensional visualization technology into surgical residency: a randomized controlled trial.  _Journal of medical systems_ 43, 3 (2019), 71\.
  * Lin et al. (2017) Chen-Hsuan Lin, Chen Kong, and Simon Lucey. 2017.  Learning efficient point cloud generation for dense 3d object reconstruction.  _arXiv preprint arXiv:1706.07036_ (2017).
  * Loke et al. (2019) Yue-Hin Loke, Francesco Capuano, Jason Mandell, Russell R Cross, Ileen Cronin, Paige Mass, Elias Balaras, and Laura J Olivieri. 2019\.  Abnormal Pulmonary Artery Bending Correlates With Increased Right Ventricular Afterload Following the Arterial Switch Operation.  _World Journal for Pediatric and Congenital Heart Surgery_ 10, 5 (2019), 572–581.
  * Luo et al. (2016) Ting Luo, Changrong Shi, Xing Zhao, Yunsong Zhao, and Jinqiu Xu. 2016.  Automatic synthesis of panoramic radiographs from dental cone beam computed tomography data.  _PloS one_ 11, 6 (2016), e0156976.
  * Mescheder et al. (2019) Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. 2019\.  Occupancy networks: Learning 3d reconstruction in function space. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 4460–4470.
  * Mills et al. (2014) Ian Mills, Julia Frost, Chris Cooper, David R Moles, and Elizabeth Kay. 2014.  Patient-centred care in general dental practice-a systematic review of the literature.  _BMC Oral Health_ 14, 1 (2014), 64.
  * Muhamad et al. (2015) Abu-Hussein Muhamad, Watted Nezar, and Abdulgani Azzaldeen. 2015\.  The curve of dental arch in normal occlusion.  _Open Science Journal of Clinical Medicine_ 3, 2 (2015), 47–54.
  * Naumovich et al. (2015) SS Naumovich, SA Naumovich, and VG Goncharenko. 2015\.  Three-dimensional reconstruction of teeth and jaws based on segmentation of CT images using watershed transformation.  _Dentomaxillofacial Radiology_ 44, 4 (2015), 20140313\.
  * Oen et al. (2014) Kay T Oen, Analia Veitz-Keenan, Silvia Spivakovsky, Y Jo Wong, Eman Bakarman, and Julie Yip. 2014\.  CAD/CAM versus traditional indirect methods in the fabrication of inlays, onlays, and crowns.  _Cochrane Database of Systematic Reviews_ 2014, 4 (2014), CD011063.
  * Olivieri et al. (2018) Laura J Olivieri, David Zurakowski, Karthik Ramakrishnan, Lillian Su, Fahad A Alfares, Matthew R Irwin, Jenna Heichel, Axel Krieger, and Dilip S Nath. 2018.  Novel, 3D display of heart models in the postoperative care setting improves CICU caregiver confidence.  _World Journal for Pediatric and Congenital Heart Surgery_ 9, 2 (2018), 206–213.
  * Oosterink et al. (2009) Floor MD Oosterink, Ad De Jongh, and Johan Hoogstraten. 2009\.  Prevalence of dental fear and phobia relative to other fear and phobia subtypes.  _European journal of oral sciences_ 117, 2 (2009), 135–143.
  * Prinz et al. (2005) A Prinz, M Bolz, and O Findl. 2005.  Advantage of three dimensional animated teaching over traditional surgical videos for teaching ophthalmic surgery: a randomised study.  _British Journal of Ophthalmology_ 89, 11 (2005), 1495–1499.
  * Raja’a and Farid (2016) M Raja’a and Farzaneh Farid. 2016.  Computer-based technologies in dentistry: types and applications.  _Journal of dentistry (Tehran, Iran)_ 13, 3 (2016), 215\.
  * Rekow (1987) Dianne Rekow. 1987\.  Computer-aided design and manufacturing in dentistry: a review of the state of the art.  _Journal of Prosthetic Dentistry_ 58, 4 (1987), 512–516.
  * Rozier et al. (2011) R Gary Rozier, Alice M Horowitz, and Gary Podschun. 2011\.  Dentist-patient communication techniques used in the United States: the results of a national survey.  _The Journal of the American Dental Association_ 142, 5 (2011), 518–530.
  * Satava (1993) Richard M Satava. 1993\.  Virtual reality surgical simulator.  _Surgical endoscopy_ 7, 3 (1993), 203–205.
  * Shirk et al. (2019) Joseph D Shirk, David D Thiel, Eric M Wallen, Jennifer M Linehan, Wesley M White, Ketan K Badani, and James R Porter. 2019.  Effect of 3-dimensional virtual reality models for surgical planning of robotic-assisted partial nephrectomy on surgical outcomes: a randomized clinical trial.  _JAMA network open_ 2, 9 (2019), e1911598–e1911598.
  * Stewart (1995) Moira A Stewart. 1995\.  Effective physician-patient communication and health outcomes: a review.  _CMAJ: Canadian medical association journal_ 152, 9 (1995), 1423\.
  * Strub et al. (2006) Joerg R Strub, E Dianne Rekow, and Siegbert Witkowski. 2006\.  Computer-aided design and fabrication of dental restorations: current systems and future possibilities.  _The Journal of the American Dental Association_ 137, 9 (2006), 1289–1296.
  * Taneva et al. (2015) Emilia Taneva, Budi Kusnoto, and Carla A Evans. 2015\.  3D scanning, imaging, and printing in orthodontics.  _Issues in contemporary orthodontics_ 148 (2015).
  * Tatarchenko et al. (2016) Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. 2016\.  Multi-view 3d models from single images with a convolutional network. In _European Conference on Computer Vision_. Springer, 322–337.
  * Tatarchenko et al. (2017) Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. 2017\.  Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. In _Proceedings of the IEEE International Conference on Computer Vision_. 2088–2096.
  * Tatarchenko et al. (2019) Maxim Tatarchenko, Stephan R Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox. 2019.  What do single-view 3d reconstruction networks learn?. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 3405–3414.
  * Travaline et al. (2005) John M Travaline, Robert Ruchinskas, and Gilbert E D’Alonzo Jr. 2005\.  Patient-physician communication: why and how.  _Journal of the American Osteopathic Association_ 105, 1 (2005), 13.
  * Wang et al. (2018) Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. 2018\.  Pixel2mesh: Generating 3d mesh models from single rgb images. In _Proceedings of the European Conference on Computer Vision (ECCV)_. 52–67.
  * Welker (2014) Samantha A Welker. 2014\.  _UROGYNECOLOGY PATIENT EDUCATION: VISUALIZING SURGICAL MANAGEMENT OF PELVIC ORGAN PROLAPSE_.  Ph.D. Dissertation. Johns Hopkins University.
  * Whaites and Drage (2013) Eric Whaites and Nicholas Drage. 2013.  _Essentials of dental radiography and radiology_.  Elsevier Health Sciences.
  * Whitman and Dufeau (2017) Rachel Marie Whitman and David Dufeau. 2017.  Visualization of Cardiac Anatomy: New Approaches for Medical Education.  _The FASEB Journal_ 31, 1_supplement (2017), 736–8.
  * Williams and Shah (1992) Donna J Williams and Mubarak Shah. 1992.  A fast algorithm for active contours and curvature estimation.  _CVGIP: Image understanding_ 55, 1 (1992), 14–26.
  * Xian et al. (2018) Ke Xian, Chunhua Shen, Zhiguo Cao, Hao Lu, Yang Xiao, Ruibo Li, and Zhenbo Luo. 2018\.  Monocular relative depth perception with web stereo data supervision. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 311–320.
  * Yang et al. (2011) Xiaoqin Yang, Jing Hu, Songsong Zhu, Xinhua Liang, Jihua Li, and En Luo. 2011\.  Computer-assisted surgical planning and simulation for condylar reconstruction in patients with osteochondroma.  _British Journal of Oral and Maxillofacial Surgery_ 49, 3 (2011), 203–208.