# GAT-CADNet: Graph Attention Network
for Panoptic Symbol Spotting in CAD Drawings

Zhaohua Zheng $\dagger$
Technical University of Munich
<EMAIL_ADDRESS>Jianfang Li $\dagger$
Alibaba Inc.
<EMAIL_ADDRESS>Lingjie Zhu
Alibaba Inc.
<EMAIL_ADDRESS>Honghua Li
Alibaba Inc.
<EMAIL_ADDRESS>Frank Petzold
Technical University of Munich
<EMAIL_ADDRESS>Ping Tan $\ddagger$
Simon Fraser University
<EMAIL_ADDRESS>
###### Abstract

Spotting graphical symbols from the computer-aided design (CAD) drawings is
essential to many industrial applications. Different from raster images, CAD
drawings are vector graphics consisting of geometric primitives such as
segments, arcs, and circles. By treating each CAD drawing as a graph, we
propose a novel graph attention network GAT-CADNet to solve the panoptic
symbol spotting problem: vertex features derived from the GAT branch are
mapped to semantic labels, while their attention scores are cascaded and
mapped to instance prediction. Our key contributions are three-fold: 1) the
instance symbol spotting task is formulated as a subgraph detection problem
and solved by predicting the adjacency matrix; 2) a relative spatial encoding
(RSE) module explicitly encodes the relative positional and geometric relation
among vertices to enhance the vertex attention; 3) a cascaded edge encoding
(CEE) module extracts vertex attentions from multiple stages of GAT and treats
them as edge encoding to predict the adjacency matrix. The proposed GAT-CADNet
is intuitive yet effective and manages to solve the panoptic symbol spotting
problem in one consolidated network. Extensive experiments and ablation
studies on the public benchmark show that our graph-based approach surpasses
existing state-of-the-art methods by a large margin.

##  1 Introduction

(a) CAD drawing

(b) Panoptic symbol spotting

(c) Reconstructed BIM model

Figure 1: A patch of floor plan (a) and its panoptic symbol spotting results
(b), where line semantics are color coded and instances are presented by
translucent rectangles. The BIM model (c) with complete semantic and accurate
geometry can be reconstructed from such annotated floor plan. We only show 3D
model of wall, windows and doors for the sake of clarity.

Computer-aided design (CAD) is the use of computers to generate digital 2D or
3D illustrations of a product, aiding the creation, modification, analysis, or
optimization process during designing and manufacturing. This technology has
been widely used in modern architecture, engineering and construction (AEC)
industries. The CAD drawings usually convey accurate geometry, rich semantic,
and domain-specific knowledge of a product design, with basic geometric
primitives, such as segments, arcs, circles, and ellipses, as illustrated in
Figs. 1(a) and 1(b).

Spotting and recognizing symbols from the CAD drawings is the first step
towards understanding its content, which is crucial to many real-world
industrial applications. For example, building information modeling (BIM) has
growing demand in various architecture engineering areas such as pipe
arrangement, construction inspection and equipment maintenance. A floor plan
usually contains complete details of a storey in an orthogonal top-down view.
Therefore, a BIM model can be precisely reconstructed from a group of 2D floor
plans with accurate semantic and instance annotations, as demonstrated in Fig.
1(c).

(a)

(b)

(c)

Figure 2: Illustration of the panoptic symbol spotting in a bathroom. Symbols
of countable things (a), and uncountalbe stuff, e.g., wall (b). The panoptic
symbol spotting proposed by [15] considers both types of symbols in a unified
scheme (c).

Traditional symbol spotting usually deals with instance symbols representing
countable things [32], like table, sofa, and bed. Following the idea in [23],
Fan _et al_. [15] extended the definition with recognizing semantic of
uncountable stuff, and named it panoptic symbol spotting, as shown in Fig. 2.
Therefore, all components in a CAD drawing are covered in one task altogether.
For example, the wall represented by a group of parallel lines was properly
handled by [15], which however was treated as background by [28, 27, 35, 33].

Large-scale dataset of high quality annotations is the fundamental ingredient
to recent advances in supervised methods with deep learning, _e.g_., ImageNet
[10] for image classification, COCO [26] for image detection, and ShapeNet [2]
for 3D shape analysis. Existing datasets for symbol spotting on floor plan,
i.e., SESYD [9] and FPLAN-POLY [34], are either synthetic, or inaccurate, both
with only a few hundreds of samples. Fan _et al_. [15] built the first large-
scale real-world FloorPlanCAD dataset of over $10,000$ floor plans in the form
of vector graphics, and provided line-grained panoptic annotations.

CAD drawings are composed of domain-specific items, which are usually
represented by abstract symbols. Human perception of CAD drawings is usually a
multi-modal cross-context reference process requiring strong domain related
knowledge. Meanwhile, the large intra-class variance and small inter-class
dissimilarity of symbols make it a more challenging task for computers, as
shown in Fig. 3.

(a) Tables.

(b) Wall, window, bay window, and curtain wall are high lighted in red.

Figure 3: The inter-class variance (a) and intra-class similarity (b) in the
public FloorPlanCAD dataset.

Representing a CAD drawing as a graph of primitives is an intuitive way to
retain the property of vector graphics, and has been proven effective for the
semantic symbol spotting task in [15]. In this work, we present a novel graph
attention network GAT-CADNet to solve the panoptic symbol spotting problem.
The network achieved state-of-the-art performance and our main contributions
are:

  * •

We formulate the instance symbol spotting task as a subgraph detection
problem, and solve it by predicting the adjacency matrix.

  * •

We explicitly encode the relative relation among vertices, using a relative
spatial encoding (RSE) module, to enhance the vertex attention.

  * •

We treat the vertex attention as edge encoding for predicting the adjacency
matrix, and design a cascaded edge encoding (CEE) module to aggregate vertex
attentions from multiple GAT stages.

##  2 Related Work

In this section we briefly summarize methods in related areas, including
symbol spotting, panoptic segmentation, graph neural networks, and attention.

#### Symbol spotting.

It is the process of finding target symbols from an image or a document [35,
37]. Optical character recognition (OCR) can be viewed as a specific case
where symbols are from a standard character set. Traditional non-data-driven
methods usually design hand-crafted descriptors [28, 27, 35], then the query
symbol is matched to the document by sliding window or graph matching
approaches [13, 12, 14]. With recent development in deep learning, data-driven
approaches [33, 15] reported better results on various datasets [9, 34].

#### Panoptic segmentation.

In the computer vision community, object detection often refers to identifying
countable things from an image such as cats, dogs, and cars [17, 24, 25]. On
the other hand, semantic segmentation is partitioning an image into multiple
regions without distinguishing instances with the same semantic [5, 40].
However, there is uncountable stuff that has no instance but only semantic,
such as sky, road, and pavement [4, 5, 36]. Panoptic segmentation is first
introduced by Kirillov et al. [23], which treated countable instance things
and uncountable stuff as one visual recognition task [23, 22, 44]. Chen _et
al_. [6] improved the panoptic segmentation quality with a bidirectional path
between the semantic and instance segmentation branches. Wu _et al_. [43]
constructed modular graph structure to reason their relations. Inspired by
[23], Fan _et al_. [15] generalized the traditional symbol spotting problem
and considered both countable things and uncountable stuff symbols as one
recognition task. They also provided a reasonable evaluation metric and a
well-annotated public dataset.

#### Graph neural networks.

The graph convolutional networks (GCNs) proposed by Thomas _et al_. [20]
operated directly on graphs via a local first-order approximation of spectral
graph convolutions. To enable the training of traditional neural networks on
the graphs, Zhang _et al_. [47] sorted graph vertices in a consistent order.
Ying _et al_. [46] introduced a differentiable graph pooling module that can
generate hierarchical representations of graphs. Some works [41, 16, 15] tried
to fuse image features to enhance the GCNs. Thomas _et al_. [21] proposed
graph auto encoders (GAE) and variational graph auto encoders (VGAE), where
vertex features are used to restore adjacency matrix.

#### Attention.

Transformers have brought the machine translation and natural language
processing to a higher level [19, 8, 42, 45]. The success has stimulated the
development of self-attention networks for various image perception tasks [18,
29, 48, 11]. Bello _et al_. [1] augmented CNN with relative self-attention to
integrate global information to the network. Dosovitskiy _et al_. [11] cut
images into grid patches and apply attention on the sequence. Vaswani _et al_.
[39] proposed self attention which is permutation invariant for sequence data.
In the same paper, they added positional embedding to the networks. In long
sequence cases, Dai _et al_. [8] found attention matrix is usually sparse and
local focused. Hence, they proposed a method to encode not absolute but
relative position.

Figure 4: The segment approximation for graphic primitives (arc, circle, and
ellipse) are shown as dash lines.  Figure 5: Graph construction: blue dots
represent vertices ${v_{i}}$, and red arrows are edges staring from ${v_{0}}$.
Note that $v_{1},v_{2},v_{3}$ are connected to ${v_{0}}$ due to their
closeness (the orange area is the $\epsilon$ envelope of $v_{0}$), while
$v_{6}$ is connected due to their collinearity. Figure 6: Architecture of the
proposed GAT-CADNet. The middle branch includes the main GAT stages of gray
blocks followed by the semantic and instance symbol spotting heads. The upper
blue branch is the RSE module, and the lower orange branch is the CEE module.

##  3 Methodology

Our GAT architecture takes CAD drawings of vector graphics as input and
predicts the semantic and instance attributes of every geometric primitive in
it.

###  3.1 Graph Construction

A graph $\mathcal{G}=(\mathcal{V},\mathcal{E})$ is constructed for one input
CAD drawing, where vertex $v_{i}\in\mathcal{V}$ is the segment approximation
of a geometric primitive. The segment approximation of an arc is the line
connecting its start and end points, while the horizontal diameter or major
axis are approximations for a circle and ellipse respectively, see Fig. 4 for
illustrations. Such simplifications are acceptable, because segments are the
majority in CAD drawings.

An edge connecting two vertices $v_{i}$ and $v_{j}$ is added if their distance
$d_{ij}$ is below certain threshold $\epsilon$, where:

$d_{ij}=\text{min}_{\boldsymbol{p}\in v_{i},\boldsymbol{q}\in
v_{j}}\|\boldsymbol{p}-\boldsymbol{q}\|.$ (1)

Since CAD drawings are usually drawn by professionals to depict man-made
objects with strong regularity, we add extra edges for collinear primitives.
To keep the graph complexity low, at most $K$ edges are allowed for every
vertex by random dropping. Fig. 5 demonstrates the graph construction around a
door symbol, where only edges starting from $v_{0}$ are illustrated. In the
following experiments, we set $\epsilon=300\text{mm}$ and $K=30$.

#### Instance and subgraph.

An instance symbol of countable things, e.g., tables or doors, usually
consists of a set of locally connected primitives. Naturally, an instance
corresponds to an connected subgraph $\mathcal{G}_{k}\subset\mathcal{G}$.
Therefore we formulate the instance symbol spotting task as a subgraph
detection problem, which can be solved by predicting the adjacency matrix.

#### Vertex feature.

We define the vertex features $\boldsymbol{v}_{i}\in\mathbb{R}^{7}$ as:

$\boldsymbol{v}_{i}=\left[cos(2\alpha_{i}),sin(2\alpha_{i}),l_{i},\boldsymbol{t}_{i}\right],$
(2)

where $\alpha_{i}\in[0,\pi)$ is the clockwise angle from the $x$ positive axis
to $v_{i}$, and $l_{i}$ measures the length of $v_{i}$. Note that our
direction features are continuous when $\alpha$ jumps between $0$ and $\pi$.
We encode the primitive type (segment, arc, circle, or ellipse) into a one hot
vector $\boldsymbol{t}_{i}\in\mathbb{R}^{4}$ to make up the missing
information of segment approximations.

#### Edge features.

Besides vertex features, we explicitly encode relation between two vertices as
edge features. The positional offset $\boldsymbol{\delta}_{ij}$ from $v_{i}$
to $v_{j}$ is defined as:

$\boldsymbol{\delta}_{ij}=\boldsymbol{m}_{j}-\boldsymbol{m}_{i},$ (3)

where $\boldsymbol{m}_{i}$ is the middle point of $v_{i}$. The directional
offset $\measuredangle_{ij}$ is defined as the acute angle between $v_{i}$ and
$v_{j}$. The length ratio between $v_{i}$ and $v_{j}$ is computed as:

$r_{ij}=\frac{l_{i}}{l_{i}+l_{j}}.$ (4)

As illustrated in Fig. 1 and reported in [15], the parallelism and
orthogonality between two line segments are common and play crucial role in
CAD drawings. We add three binary indicators to emphasize such regularities:

$\boldsymbol{g}_{ij}=\left[\parallel_{ij},\perp_{ij},\neg_{ij}\right],$ (5)

where $\parallel_{ij}$ and $\perp_{ij}$ indicates whether $v_{i}$ is parallel
or orthogonal to $v_{j}$, and $\neg_{ij}$ is used to indicate whether $v_{i}$
and $v_{j}$ share a same end point. Putting the aforementioned terms together,
we obtain the edge features $\boldsymbol{e}_{ij}\in\mathbb{R}^{7}$ as:

$\boldsymbol{e}_{ij}=\left[\boldsymbol{\delta}_{ij},\measuredangle_{ij},r_{ij},\boldsymbol{g}_{ij}\right].$
(6)

In our experiments, the angle and distance threshold used in
$\boldsymbol{g}_{ij}$ are set to $5^{\circ}$ and $100\text{mm}$ respectively.

###  3.2 Network Architecture

Based on the graph constructed from the CAD drawing in Sec. 3.1, we propose a
novel GAT-CADNet to solve the panoptic symbol spotting problem, as shown in
Fig. 6. The network 1) formulates the instance symbol spotting task as an
adjacency matrix prediction problem, 2) enhances the vertex attention with
edge feature encoding, 3) aggregates vertex attentions from multiple GAT
stages for predicting the sparse adjacency matrix.

The initial vertex features $\boldsymbol{v}_{i}$ and edge features
$\boldsymbol{e}_{ij}$ are embedded to $\hat{\boldsymbol{v}}_{i}$ and
$\hat{\boldsymbol{e}}_{ij}$ with two separate multilayer perceptron (MLP)
blocks. For each vertex $v_{i}$, we enhance its features by its connected
edges as:

$\boldsymbol{v}^{0}_{i}=\text{Concat}(\hat{\boldsymbol{v}}_{i},\text{MaxPooling}(\\{\hat{\boldsymbol{e}}_{ij}\\})).$
(7)

Vertex features are stacked to $V^{0}\in\mathbb{R}^{N\times 128}$,
$N=|\mathcal{V}|$, as the input for the following GAT stages.

#### Relative spatial encoding (RSE).

When processing point cloud [49] or natural language [39], researchers often
use relative position encoding to make the network invariant to translation
and aware of distance. Similarly, we pass the initial edge features through
another MLP block to encode the relative spatial relations among vertices:

$R=\text{MLP}(E),$ (8)

where $E\in\mathbb{R}^{N\times N\times 7}$ is the edge features by expanding
$|\mathcal{E}|$ edges to $N\times N$. The RSE encoding
$R\in\mathbb{R}^{N\times N\times H}$ is then fed to every stage of the main
GAT branch, where $H$ is the number of heads in the GAT statge.

Figure 7: A GAT stage in Fig. 6, where the gray area contains the multihead
attention [39]. The vertex attention score is fed to the CEE module as edge
encoding (orange arrow), and then enhanced with edge encoding from the RSE
module (blue arrow).

#### Graph attention stage.

The stem of our network is the GAT branch of $S$ stages, as illustrated in
Fig. 7. The $s\text{-th}$ stage takes vertex features $V^{s-1}$ from previous
stage and outputs vertex features $V^{s}$ of the same dimension. In the
$h\text{-th}$ head of the GAT block, we project $V^{s}$ to a query matrix
$Q_{h}\in\mathbb{R}^{N\times d}$, a key matrix $K_{h}\in\mathbb{R}^{N\times
d}$, and a value matrix $V_{h}\in\mathbb{R}^{N\times d}$. Then the multihead
attention score $A^{s}\in\mathbb{R}^{N\times N\times H}$ can be formulated as:

$\displaystyle A^{s}_{h}$ $\displaystyle=Q_{h}K^{T}_{h},$ (9) $\displaystyle
A^{s}$ $\displaystyle=\text{Concat}(A^{s}_{1},\ldots,A^{s}_{H}).$ (10)

Note that $A^{s}$ expresses the relation among vertices in the embedding
space. Similar to the relative position encoding in [49, 39], we add our
relative spatial encoding $R$ to $A^{s}$ to enhance their attention
explicitly. Therefore the aggregated value matrix
$V^{\prime}_{h}\in\mathbb{R}^{N\times d}$ is obtained by:

$V^{\prime}_{h}=\text{Softmax}(A^{s}+R)V_{h},$ (11)

which is passed through a MLP block and added to $V^{s-1}$, producing the
output vertex features $V^{s}$ of current stage. The semantic symbol spotting
head maps vertex features from the final stage to the classification
prediction:

$Y=\text{Softmax}(\text{MLP}(V^{S})),$ (12)

with the semantic loss as:

$loss_{\text{sem}}=\text{CrossEntropy}(Y,Y^{gt}).$ (13)

#### Cascaded edge encoding (CEE).

Recall that vertex attentions $A^{s}$ can be viewed as relational intensity
among vertices, which are good choice for predicting the adjacency matrix.
Therefore, we cascade attention scores from all GAT stages $\\{A^{s}\\}$ as
implicit edge encoding to capture local and global vertex connectivity:

$C=\sum_{s=1}^{S}{A^{s}}.$ (14)

Each valid edge encoding $\boldsymbol{c}_{ij}$ in $C\in\mathbb{R}^{N\times
N\times H}$ is then concatenated with vertex features of its two endpoints
from the last GAT stage to form the final edge feature:

$\boldsymbol{\tilde{e}}_{ij}=\text{Concat}(\boldsymbol{c}_{ij},\boldsymbol{v}^{S}_{i},\boldsymbol{v}^{S}_{j}).$
(15)

Finally, the adjacency matrix prediction $Z\in\mathbb{R}^{N\times N}$ is given
by the instance symbol spotting head:

$Z=\text{Sigmod}(\text{MLP}(\tilde{E})),$ (16)

where $\tilde{E}\in\mathbb{R}^{N\times N\times(H+256)}$ denotes the stacked
final edge features $\\{\boldsymbol{\tilde{e}}_{ij}\\}$. The loss for instance
symbol spotting is defined as:

$loss_{\text{ins}}=\text{BinaryCrossEntropy}(Z,Z^{gt},\boldsymbol{w}),$ (17)

where weights $\boldsymbol{w}$ for punishing incorrect predictions are defined
as:

$\begin{array}[]{c|cc}\boldsymbol{w}&Z^{gt}_{ij}=0&Z^{gt}_{ij}=1\\\ \hline\cr
Y^{gt}_{i}=Y^{gt}_{j}&20&2\\\ Y^{gt}_{i}\neq Y^{gt}_{j}&1&0\end{array}.$

Note that an edge connecting two vertices with the same semantic label
($Y^{gt}_{i}=Y^{gt}_{j}$) but belong to different instances ($Z^{gt}_{ij}=0$)
has largest weight of 20.

#### Panoptic loss.

The panoptic symbol spotting loss of our network is the linear combination of
the semantic and instance loss terms:

$loss_{\text{pan}}=loss_{\text{sem}}+\lambda loss_{\text{ins}}.$ (18)

In our implementation, the attention is conducted within the one-ring
neighbors and our $N\times N$ matrices are sparse.

(a) Input CAD drawing

(b) GT

(c) DeepLabv3 [5]

(d) HRNetV2 [40]

(e) PanCADNet [15]

(f) Ours

Figure 8: Qualitative comparison of semantic symbol spotting results on the
FloorPlanCAD [15] dataset. Two close-ups of region A (upper row) and B (lower
row) are listed from left to right.

##  4 Experiment

Qualitative and quantitative evaluations of our GAT-CADNet are conducted for
the panoptic symbol spotting task on the public CAD drawing dataset. We also
compare our method with typical image-based instance detection [30, 31, 38]
and semantic segmentation methods [5, 40]. Extensive ablation study is
performed to validate the design choice of our network.

#### Dataset and panoptic metric.

Although there are several small vector graphics datasets [9, 34] for
traditional symbol spotting, we use the latest large-scale FloorPlanCAD [15]
dataset in our experiment, which has $11,602$ CAD drawings of various floor
plans with segment-grained panoptic annotation. The dataset consists of
$10\text{m}\times 10\text{m}$ squared blocks covering $30$ things and $5$
stuff classes. Similar to [23], it also provides a panoptic metric defined on
vector graphics:

$\displaystyle PQ$ $\displaystyle=RQ\times SQ$
$\displaystyle=\frac{\sum_{(s^{p},s^{g})\in
TP}\text{IoU}(s^{p},s^{g})}{|TP|+\frac{1}{2}|FP|+\frac{1}{2}|FN|},$ (19)

where $RQ$ is the $F_{1}$ score measuring the recognition quality and $SQ$ is
the segmentation quality computed by averaging IoUs of matched symbols. For
the detailed IoU evaluation of a predicted symbol $s^{p}$ and the ground truth
symbol $s^{g}$ at primitive level, please refer to [15].

#### Implementation.

In the following experiments, our GAT-CADNet is configured with 8 GAT stages
and $H=8$, $\lambda=2$ if not specified. We use the Adam optimizer with
$\beta_{1}=0.9$, $\beta_{2}=0.99$, $lr=0.001$ and set the decay rate to $0.7$
for every 20 epochs. We train our GAT-CADNet for 100 epochs and take the best
model on the validation split. The number of graph vertices and their
neighbours are limited to $4096$ and $30$ respectively for each CAD drawing to
fit graphics card memory. All other image-based networks are trained with the
latest release of OpenMMLab [3, 7].

During inference, we prune the resulted adjacency matrix by a threshold of
$0.7$, producing a directed graph. Vertices of the same semantics are grouped
first, and then instances are found by searching connected components within
each group. Please refer to the supplementary material for more results and
feel free to zoom in since they are vector graphics.

###  4.1 Quantitative Evaluation

Methods | F1 | length-weighted F1
---|---|---
HRNetsV2 W18 [40] | 0.656 | 0.683
HRNetsV2 W48 [40] | 0.666 | 0.693
DeepLabv3+R50 [4] | 0.680 | 0.705
DeepLabv3+R101 [4] | 0.688 | 0.714
PanCADNet [15] | 0.806 | 0.798
Ours | 0.850 | 0.823
Table 1: Statistical results of different image semantic segmentation models
and our GAT-CADNet.

(a) GT

(b) FRCNN [31]

(c) Ours

Figure 9: Prediction quality comparison. Our primitive-level prediction
produces clearer boundary and can exclude background (grey lines) in an
instance symbol.

#### Semantic symbol spotting.

To compare with existing image segmentation methods, the CAD drawings are
rendered as images with line width of 2 pixels. The semantic of a primitive in
$\mathcal{G}$ is then retrieved by sampling on the predicted mask with a
majority voting strategy. PanCADNet [15] is a GCN architecture for semantic
symbol spotting and relies on image features from a CNN backbone. Tab. 1
compares the results of popular segmentation methods [4, 40] with different
configurations. Qualitative comparion are shown in Fig. 8 where DeepLabv3 [5]
and HRNetV2 [40] are with the W48 and R01 configuration in Tab. 1
respectively. While our GAT-CADNet is built on the graph entirely and requires
geometric features only, it manages to outperform other image-based methods.

(a) Input CAD drawing

(b) GT

(c) FRCNN [31]

(d) YOLOv3 [30]

(e) FCOS [38]

(f) Ours

Figure 10: Instance symbol spotting comparison with image detection methods.
(a) The input CAD drawing with close-ups of regions in blue rectangles. Two
close-ups of region A (upper row) and B (lower row) are listed from (c) to
(f). Wrong predictions are marked by red rectangle with dash lines. Faster
R-CNN [31] and YOLOv3 [30] mistakenly recognize two more sliding doors in
region A. Both YOLOv3 [30] and FCOS [38] fail to recognize some windows at
bottom left in region B. Compared to the image based methods, our GAT-CADNet
gives closer bounding boxes to ground truth.

#### Instance symbol spotting.

As reported in [33, 15], traditional symbol spotting algorithms [28, 27, 35]
have lower generalization ability and are omitted in the comparison. By
rendering CAD drawings into images, our GAT-CADNet is compared with various
image detection methods, including the two stage Faster-RCNN [31], the one
stage YOLOv3 [30] and the more recent FCOS [38]. Note that the instance head
in PanCADNet [15] is from Faster-RCNN and is not listed here.

The image based detection methods [31, 30, 38] predict bounding boxes
directly, while we predict instance labels for each geometric primitive. For a
fair comparison, we compute the bounding box of each instance symbol and use
its averaged connection intensity as the confidence score. Quantitative
comparison are listed in Tab. 2 and our GAT-CADNet surpasses other methods by
a large margin.

One thing noteworthy is that our average precision (AP) does not drop
dramatically when increasing the IoU threshold and has a much higher mAP
score. Since CNNs rely on local patch texture for recognition and may ignore
features at border, it is not a surprise that their box predictions are less
accurate due to the low texture in CAD drawings. Such phenomenon can be
observed in Figs. 9 and 10 where our primitive-level prediction has clearer
bounding boxes.

Methods | AP50 | AP75 | mAP
---|---|---|---
Faster R-CNN [31] | 0.693 | 0.631 | 0.568
YOLOv3 [30] | 0.656 | 0.431 | 0.395
FCOS [38] | 0.648 | 0.572 | 0.525
Ours | 0.735 | 0.680 | 0.690
Table 2: Comparison on instance symbol spotting with typical image detection
methods.

(a) GT

(b) Panoptic prediction.

Figure 11: Visual results of our network on various scenes. Missing symbols
are highlighted with rectangles of red dash lines. For more results, please
refer to the supplementary materials.

#### Panoptic symbol spotting.

Converting CAD drawings into images and applying panoptic segmentation
algorithms on them is a straightforward approach. However, as demonstrated in
the aforementioned comparison sections, the image based methods are less
capable of recognizing abstract symbol at geometric primitive level. PanCADNet
[15] provides a CNN-GCN architecture for the panoptic symbol spotting. It
constructs a graph on the CAD drawing first, then fetches CNN multi-layer
features to each vertex and uses a simple GCN structure for recognition. Since
PanCADNet [15] adopts Faster-RCNN as its backbone and detection head, there is
no surprise that it has much lower recognition quality than our model, second
and last row in Tab. 3. In addition, it does not encode inter-vertex relation
explicitly and even has lower recognition and segmentation than our baseline
model, third row in Tab. 3.

###  4.2 Ablation study

Figure 12: Evaluation on different numbers of GAT stages. Model | RSE | CEE | RQ | SQ | PQ
---|---|---|---|---|---
PanCADNet [15] | - | - | 0.660 | 0.838 | 0.553
baseline |  |  | 0.687 | 0.875 | 0.602
b. + RSE | ✓ |  | 0.734 | 0.891 | 0.654
b. + CEE |  | ✓ | 0.749 | 0.896 | 0.671
| ✓ | 2nd | 0.761 | 0.903 | 0.687
| ✓ | 4th | 0.768 | 0.903 | 0.694
| ✓ | 6th | 0.768 | 0.904 | 0.695
| ✓ | 8th | 0.786 | 0.908 | 0.714
Ours | ✓ | ✓ | 0.807 | 0.914 | 0.737
Table 3: Ablation study of different network configurations. Numbers in the
CEE column represent the $n$th GAT stage.

Various controlled experiments are conducted to verify specific design
decisions in our GAT-CADNet architecture. Discussion about initial geometric
feature selection and the number of GAT stages are also included.

#### The RSE module.

The baseline architecture of our model is the multi-stage GAT branch in the
middle of Fig. 6. Following the black arrows in Fig. 6, it takes initial
vertex and edge features and maps to the semantic and instance heads. The blue
branch in Fig. 6 is the RSE module that attaches relative spatial relation to
the vertex attention in every GAT stage. Adding the RSE module to the baseline
shows clear improvement in both recognition and segmentation quality by 4 and
5 percentage points respectively, as shown in the third row in Tab. 3. It is
evident that the explicitly encoded primitive spacial relations, _e.g_.
parallelism and orthogonality, enhances vertex attention and thus yields
better performance in the panoptic recognition.

#### The CEE module.

Our CEE module is the orange branch in Fig. 6, which views attention among
vertices as affinity in feature space and cascades them to predict instance
adjacency matrix. Adding the CEE module to the baseline boosts the $RQ$ metric
up to 6 percentage points as shown in the fifth row in Tab. 3. It proves that
the CEE module is able to gather connections between vertices effectively and
assist in collecting primitives of the same instance. If we add both RSE and
CEE modules to the baseline, our method achieves state-of-the-art performance,
which exceeds PanCADNet [15] in $RQ$, $SQ$ and $PQ$ metrics by $14.7$, $7.6$
and $18.4$ percentage points respectively.

To further verify the cascaded structure in CEE, we take attention score from
only one GAT stage and test their performance. Specifically, the attention in
the 2nd, 4th, 6th and 8th GAT stage are fed to the instance head separately.
Statistics listed in Tab. 3 (sixth to eighth row) show steady improvement in
the $RQ$ metric, indicating the higher level information is gathered form
deeper GAT stage. Our cascaded structure is able to merge multi-stage local
and global features for instance symbol spotting.

#### Edge regularity features.

Theoretically, the parallel and orthogonal indicators in Eq. 6 are redundant
if we have the angle between two vertices. However, if we drop the regularity
term in the initial edge features, the $RQ$, $SQ$ and $PQ$ metrics decrease to
$0.58$, $0.85$ and $0.49$ respectively. This suggests that the regularities in
CAD drawings are essential to recognizing symbols and our extra geometric
regularity properties help the network to find a better solution.

#### Number of GAT stages.

We also test the effect on different number of GAT stages. The number of GAT
stage is configured from 2 to 16 and the results are plotted in Fig. 12. As
the number of stages increases, the performance gets better. However, if the
number of stages reaches to 16, our network does not benefit from it.

##  5 Conclusion

In this work we present an intuitive yet effective architecture named GAT-
CADNet for panoptic symbol spotting on CAD drawings. It formulates the
instance symbol spotting task as an adjacency matrix prediction problem. The
relative spatial encoding module explicitly encodes the relative relation
among vertices to enhance their attention. The cascaded edge encoding module
extracts vertex attentions from multiple GAT stages capturing both local and
global connectivity information. With the help of the RSE and CEE modules, our
GAT-CADNet surpasses other approaches by a large margin.

#### Limitation and future work.

It is undeniable that our method is still far from perfection, and the
panoptic symbol spotting remains an open problem. One shortcoming of our
network is that it can only process drawings with a limited number of
primitives, otherwise it will suffer from GPU memory shortage. A possible
solution is cutting the drawing into smaller blocks and fuse the results. We
will keep exploring more efficient networks to alleviate such issue.

## References

  * [1] Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, and Quoc V Le.  Attention augmented convolutional networks.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 3286–3295, 2019.
  * [2] Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu.  ShapeNet: An Information-Rich 3D Model Repository.  Technical Report arXiv:1512.03012 [cs.GR], Stanford University — Princeton University — Toyota Technological Institute at Chicago, 2015.
  * [3] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al.  Mmdetection: Open mmlab detection toolbox and benchmark.  arXiv preprint arXiv:1906.07155, 2019.
  * [4] Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam.  Rethinking atrous convolution for semantic image segmentation.  arXiv preprint arXiv:1706.05587, 2017.
  * [5] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam.  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In Proceedings of the European conference on computer vision (ECCV), pages 801–818, 2018.
  * [6] Yifeng Chen, Guangchen Lin, Songyuan Li, Omar Bourahla, Yiming Wu, Fangfang Wang, Junyi Feng, Mingliang Xu, and Xi Li.  Banet: Bidirectional aggregation network with occlusion handling for panoptic segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3793–3802, 2020.
  * [7] MMSegmentation Contributors.  MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark.  https://github.com/open-mmlab/mmsegmentation, 2020.
  * [8] Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov.  Transformer-xl: Attentive language models beyond a fixed-length context.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2978–2988, 2019.
  * [9] Mathieu Delalandre, Ernest Valveny, Tony Pridmore, and Dimosthenis Karatzas.  Generation of synthetic documents for performance evaluation of symbol recognition & spotting systems.  International Journal on Document Analysis and Recognition (IJDAR), 13(3):187–207, 2010.
  * [10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In International Conference on Learning Representations, 2020.
  * [12] Anjan Dutta, Josep Lladós, Horst Bunke, and Umapada Pal.  Near convex region adjacency graph and approximate neighborhood string matching for symbol spotting in graphical documents.  In 2013 12th International Conference on Document Analysis and Recognition, pages 1078–1082. IEEE, 2013.
  * [13] Anjan Dutta, Josep Lladós, and Umapada Pal.  Symbol spotting in line drawings through graph paths hashing.  In 2011 International Conference on Document Analysis and Recognition, pages 982–986. IEEE, 2011.
  * [14] Anjan Dutta, Josep Lladós, and Umapada Pal.  A symbol spotting approach in graphical documents by hashing serialized graphs.  Pattern Recognition, 46(3):752–768, 2013.
  * [15] Zhiwen Fan, Lingjie Zhu, Honghua Li, Xiaohao Chen, Siyu Zhu, and Ping Tan.  Floorplancad: A large-scale cad drawing dataset for panoptic symbol spotting.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
  * [16] Georgia Gkioxari, Jitendra Malik, and Justin Johnson.  Mesh r-cnn.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9785–9795, 2019.
  * [17] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.
  * [18] Han Hu, Zheng Zhang, Zhenda Xie, and Stephen Lin.  Local relation networks for image recognition.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3464–3473, 2019.
  * [19] Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of NAACL-HLT, pages 4171–4186, 2019.
  * [20] Thomas N Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  arXiv preprint arXiv:1609.02907, 2016.
  * [21] Thomas N Kipf and Max Welling.  Variational graph auto-encoders.  arXiv preprint arXiv:1611.07308, 2016.
  * [22] Alexander Kirillov, Ross Girshick, Kaiming He, and Piotr Dollár.  Panoptic feature pyramid networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6399–6408, 2019.
  * [23] Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollár.  Panoptic segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9404–9413, 2019.
  * [24] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2117–2125, 2017.
  * [25] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017.
  * [26] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In European conference on computer vision, pages 740–755. Springer, 2014.
  * [27] Thi-Oanh Nguyen, Salvatore Tabbone, and Alain Boucher.  A symbol spotting approach based on the vector model and a visual vocabulary.  In 2009 10th International Conference on Document Analysis and Recognition, pages 708–712. IEEE, 2009.
  * [28] Thi Oanh Nguyen, Salvatore Tabbone, and O Ramos Terrades.  Symbol descriptor based on shape context and vector model of information retrieval.  In 2008 The Eighth IAPR International Workshop on Document Analysis Systems, pages 191–197. IEEE, 2008.
  * [29] Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello, Anselm Levskaya, and Jon Shlens.  Stand-alone self-attention in vision models.  Advances in Neural Information Processing Systems, 32, 2019.
  * [30] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement.  arXiv preprint arXiv:1804.02767, 2018.
  * [31] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: towards real-time object detection with region proposal networks.  IEEE transactions on pattern analysis and machine intelligence, 39(6):1137–1149, 2016.
  * [32] Alireza Rezvanifar, Melissa Cote, and Alexandra Branzan Albu.  Symbol spotting for architectural drawings: state-of-the-art and new industry-driven developments.  IPSJ Transactions on Computer Vision and Applications, 11(1):2, 2019\.
  * [33] Alireza Rezvanifar, Melissa Cote, and Alexandra Branzan Albu.  Symbol spotting on digital architectural floor plans using a deep learning-based framework.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 568–569, 2020.
  * [34] Marçal Rusiñol, Agnés Borràs, and Josep Lladós.  Relational indexing of vectorial primitives for symbol spotting in line-drawing images.  Pattern Recognition Letters, 31(3):188–201, 2010.
  * [35] Marçal Rusiñol, Josep Lladós, and Gemma Sánchez.  Symbol spotting in vectorized technical drawings through a lookup table of region strings.  Pattern Analysis and Applications, 13(3):321–331, 2010.
  * [36] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018.
  * [37] KC Santosh.  Document image analysis: Current trends and challenges in graphics recognition.  2018\.
  * [38] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He.  Fcos: Fully convolutional one-stage object detection.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9627–9636, 2019.
  * [39] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in neural information processing systems, pages 5998–6008, 2017.
  * [40] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al.  Deep high-resolution representation learning for visual recognition.  IEEE transactions on pattern analysis and machine intelligence, 2020\.
  * [41] Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang.  Pixel2mesh: Generating 3d mesh models from single rgb images.  In ECCV, 2018.
  * [42] Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli.  Pay less attention with lightweight and dynamic convolutions.  In International Conference on Learning Representations, 2018.
  * [43] Yangxin Wu, Gengwei Zhang, Yiming Gao, Xiajun Deng, Ke Gong, Xiaodan Liang, and Liang Lin.  Bidirectional graph reasoning network for panoptic segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9080–9089, 2020.
  * [44] Yuwen Xiong, Renjie Liao, Hengshuang Zhao, Rui Hu, Min Bai, Ersin Yumer, and Raquel Urtasun.  Upsnet: A unified panoptic segmentation network.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8818–8826, 2019.
  * [45] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le.  Xlnet: Generalized autoregressive pretraining for language understanding.  Advances in neural information processing systems, 32, 2019.
  * [46] Rex Ying, Jiaxuan You, Christopher Morris, Xiang Ren, William L Hamilton, and Jure Leskovec.  Hierarchical graph representation learning with differentiable pooling.  In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 4805–4815, 2018.
  * [47] Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen.  An end-to-end deep learning architecture for graph classification.  In Thirty-Second AAAI Conference on Artificial Intelligence, 2018\.
  * [48] Hengshuang Zhao, Jiaya Jia, and Vladlen Koltun.  Exploring self-attention for image recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10076–10085, 2020.
  * [49] Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun.  Point transformer.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16259–16268, 2021.

GAT-CADNet: Graph Attention Network
for Panoptic Symbol Spotting in CAD Drawings
Supplementary Material

As the space limitation in the paper, more quantitative and qualitative
results are illustrated in supplementary material part.

##  Appendix A Quantitative results

#### More ablation study.

Two more extra experiments are conducted to further prove the superiority of
our model. One is using graph convolution network(GCN) as baseline. Similar to
GAT, GCN is also a widely used graph neural network. Thus, we replace GAT
stages in our model with GCN stages and take the normalized Laplacian matrix
equivalently to one head attention score in our RSE and CEE modules. Another
is using vertices features only. Once the center coordinates are added to
current vertices features, spacial relationship can be figured out with any
two given segments. To verify the necessity of explicitly encoded edge
features in our model, we conduct another experiment with only vertices
features . As shown in Tab. 4, neither of two extra experiments reaches the
performance of our baseline, let alone our best model.

#### Quantitative results.

As the space limitation in main body, only the total evaluation results of
panoptic quality(PQ), segmentation quality(SQ) and recognition quality(RQ) are
shown ahead. Here we provide the evaluation results of each class in Tab. 5.

##  Appendix B Qualitative results

Visualized results of more cases are illustrated in this section. Fig. 13 and
Fig. 14 show the cases of residential buildings and core of towers, in which
things and stuff are usually in regular layout, and are also the cases in
which our model gives best results. Fig. 15 are plans of shopping malls, which
have lager amount of stuff including parking and curtain wall. Our model also
performs well in these cases. Fig. 16 and Fig. 17 are cases of schools, in
which tables and chairs are arrayed orderly and densely. Although some
instances have lower confidence, most instances are well segmented.

Furthermore,all figures are illustrated as vector graph, such that details can
be shown clearly after zooming in. Annotations and confidence are labeled on
the upper left corner of the instances blocks. Segments belong to different
classes are drawn with different colors, while background segments are drawn
in light gray.

##  Appendix C limitations

Our GAT-CADNet treats the instance symbol spotting as a subgraph detection
problem, with proposed RSE and CEE modules, surpasses existing state-of-the-
art methods by a large margin. There are still limitations. Two failed cases
are shown in Fig. 18. For some cases, simple symbols could be missing or
wrongly recognized with mistaken labeled or lager variation in graph, e.g. our
model misses all L shape tables (upper) and recognizes all windows as curtain
wall (lower) by mistake. Future work would be focusing on failed cases and
improving the robustness of our model.

Model | RQ | SQ | PQ
---|---|---|---
our baseline | 0.687 | 0.875 | 0.602
GCN based | 0.655 | 0.859 | 0.563
w/o edge features | 0.599 | 0.850 | 0.509
ours | 0.807 | 0.914 | 0.737
Table 4: Extra experiments. GCN based model and GAT without edge features show inferior results. class | Baseline | Basline + RSE | Basline + CEE | Basline + RSE + CEE
---|---|---|---|---
RQ SQ PQ | RQ SQ PQ | RQ SQ PQ | RQ SQ PQ
single door | 0.78 0.91 0.71 | 0.84 0.93 0.78 | 0.88 0.93 0.82 |  0.91 0.95 0.86
double door | 0.82 0.91 0.75 | 0.86 0.93 0.80 | 0.84 0.93 0.79 |  0.89 0.94 0.83
sliding door | 0.89 0.94 0.83 | 0.90 0.94 0.85 | 0.90 0.95 0.85 |  0.94 0.96 0.91
folding door | 0.34 0.85 0.29 |  0.46 0.90 0.42 |  0.39 0.91 0.35 | 0.45 0.89 0.40
revolving door |  0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00
shutter door |  0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00
window | 0.69 0.81 0.56 | 0.71 0.82 0.58 | 0.74 0.81 0.60 |  0.79 0.84 0.66
bay window |  0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00
shutter window | 0.69 0.82 0.56 | 0.75 0.85 0.64 | 0.74 0.84 0.62 |  0.76 0.87 0.66
opening symbol |  0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00 | 0.00 0.00 0.00
sofa | 0.40 0.81 0.32 | 0.47 0.89 0.42 | 0.36 0.91 0.33 |  0.61 0.96 0.59
bed | 0.68 0.90 0.61 | 0.68 0.88 0.60 | 0.64 0.91 0.58 |  0.78 0.91 0.72
chair | 0.38 0.84 0.32 | 0.58 0.85 0.49 |  0.66 0.94 0.62 |  0.84 0.93 0.78
table | 0.36 0.88 0.32 | 0.31 0.88 0.27 | 0.40 0.93 0.37 |  0.57 0.94 0.53
TV cabinet | 0.45 0.84 0.38 | 0.54 0.83 0.45 | 0.39 0.87 0.34 |  0.73 0.94 0.69
wardrobe | 0.75 0.81 0.61 | 0.67 0.81 0.55 | 0.71 0.87 0.62 |  0.84 0.92 0.77
cabinet | 0.20 0.79 0.16 | 0.21 0.82 0.18 | 0.18 0.80 0.14 |  0.44 0.85 0.37
gas stove | 0.92 0.96 0.88 | 0.81 0.95 0.77 | 0.92 0.96 0.88 |  0.94 0.98 0.92
sink | 0.75 0.93 0.69 | 0.78 0.93 0.72 | 0.77 0.94 0.73 |  0.81 0.95 0.77
refrigerator | 0.72 0.81 0.58 | 0.72 0.87 0.62 | 0.76 0.85 0.64 |  0.88 0.93 0.82
air conditioning | 0.46 0.89 0.41 | 0.59 0.92 0.55 |  0.68 0.96 0.66 |  0.66 0.97 0.64
bath | 0.24 0.77 0.19 | 0.31 0.77 0.24 |  0.36 0.80 0.29 |  0.46 0.79 0.36
bathtub | 0.51 0.78 0.40 | 0.54 0.85 0.46 | 0.63 0.81 0.51 |  0.68 0.88 0.59
washing machine | 0.74 0.85 0.63 | 0.61 0.86 0.53 | 0.69 0.86 0.59 |  0.75 0.94 0.70
urinal | 0.91 0.97 0.89 | 0.92 0.97 0.89 | 0.92 0.98 0.90 |  0.94 0.99 0.92
squat toilet | 0.88 0.90 0.79 | 0.91 0.93 0.85 | 0.77 0.93 0.72 |  0.92 0.96 0.88
toilet | 0.88 0.96 0.84 | 0.89 0.97 0.87 | 0.91 0.96 0.88 |  0.94 0.99 0.93
stairs | 0.53 0.83 0.44 | 0.62 0.86 0.53 | 0.68 0.86 0.58 |  0.74 0.90 0.66
elevator | 0.76 0.92 0.70 | 0.82 0.93 0.76 | 0.81 0.93 0.75 |  0.84 0.95 0.80
escalator | 0.17 0.73 0.12 | 0.23 0.74 0.17 |  0.25 0.74 0.18 |  0.22 0.78 0.17
row seat | 0.26 0.90 0.23 | 0.46 0.92 0.42 | 0.33 0.92 0.30 |  0.49 0.93 0.45
parking |  0.83 0.88 0.73 | 0.82 0.90 0.74 | 0.77 0.86 0.66 |  0.82 0.90 0.74
wall | 0.66 0.71 0.47 | 0.72 0.74 0.53 | 0.68 0.72 0.48 |  0.76 0.76 0.58
curtain wall | 0.28 0.77 0.22 | 0.33 0.74 0.24 | 0.21 0.75 0.15 |  0.40 0.78 0.32
handrail | 0.12 0.65 0.08 | 0.14 0.72 0.10 | 0.10 0.69 0.07 |  0.27 0.78 0.21
total | 0.69 0.87 0.60 | 0.73 0.90 0.65 | 0.75 0.90 0.67 |  0.80 0.91 0.74
Table 5: Quantitative results for semantic symbol spotting, instance symbol
spotting and panoptic symbol spotting of each class. There are classes in test
split with few instances leading to zeros and very low values in results. The
total result are weighed with the number of each class.

(a) GT

(b) Panoptic prediction

Figure 13: Results of GAT-CADNet on FloorPlanCAD, see the main body for
annotation details. The images are part of our test split of residential
building CAD drawings.

(a) GT

(b) Panoptic prediction

Figure 14: Results of GAT-CADNet on FloorPlanCAD, see the main body for
annotation details. The images are part of our test split of tower CAD
drawings.

(a) GT

(b) Panoptic prediction

Figure 15: Results of GAT-CADNet on FloorPlanCAD, see the main body for
annotation details. The images are part of our test split of shopping mall CAD
drawings.

(a) GT

(b) Panoptic prediction

Figure 16: Results of GAT-CADNet on FloorPlanCAD, see the main body for
annotation details. The images are part of our test split of school CAD
drawings.

(a) GT

(b) Panoptic prediction

Figure 17: Results of GAT-CADNet on FloorPlanCAD, see the main body for
annotation details. The images are part of our test split of public building
CAD drawings

(a) GT

(b) Panoptic prediction

Figure 18: Two typical failed cases of GAT-CADNet. All L shape tables in upper
are missing and all windows in lower are recognized as curtain wall by
mistake.