# LInK: Learning Joint Representations of Design and
Performance Spaces through Contrastive Learning for
Mechanism Synthesis

Amin Heyrani Nobari
<EMAIL_ADDRESS>
Department of Mechanical Engineering
Massachusetts Institute of Technology Akash Srivastava
MIT-IBM Watson AI Lab
IBM Research
Cambridge, Massachusetts 02142
Dan Gutfreund
MIT-IBM Watson AI Lab
IBM Research
Cambridge, Massachusetts 02142
Kai Xu
MIT-IBM Watson AI Lab
IBM Research
Cambridge, Massachusetts 02142
Faez Ahmed
Department of Mechanical Engineering
Massachusetts Institute of Technology

###### Abstract

In this paper, we introduce LInK, a novel framework that integrates
contrastive learning of performance and design space with optimization
techniques for solving complex inverse problems in engineering design with
discrete and continuous variables. We focus on the path synthesis problem for
planar linkage mechanisms. By leveraging a multi-modal and transformation-
invariant contrastive learning framework, LInK learns a joint representation
that captures complex physics and design representations of mechanisms,
enabling rapid retrieval from a vast dataset of over 10 million mechanisms.
This approach improves precision through the warm start of a hierarchical
unconstrained nonlinear optimization algorithm, combining the robustness of
traditional optimization with the speed and adaptability of modern deep
learning methods. Our results on an existing benchmark demonstrate that LInK
outperforms existing methods with 28 times less error compared to a state-of-
the-art approach while taking 20 times less time on an existing benchmark.
Moreover, we introduce a significantly more challenging benchmark, named LINK-
ABC, which involves synthesizing linkages that trace the trajectories of
English capital alphabets—an inverse design benchmark task that existing
methods struggle with due to large non-linearities and tiny feasible space.
Our results demonstrate that LInK not only advances the field of mechanism
design but also broadens the applicability of contrastive learning and
optimization to other areas of engineering.

##  1 Introduction

Linkage mechanisms play a pivotal role in mechanical engineering, serving as
fundamental components in applications ranging from the automation of
manufacturing processes to the development of robotic systems. These
mechanisms are crucial for translating input motions—often rotational—into
desired output trajectories or paths, thereby enabling the execution of
complex tasks through relatively simple inputs. The design of linkage
mechanisms relies heavily on kinematics—the study of motion without
considering the forces involved—and kinematic synthesis, the design process of
mechanisms to achieve specific motions.

Despite the focused interest in kinematics and kinematic synthesis, designing
complex kinematic systems remains a challenge Lipson (2008), often requiring
trial and error, specialized expertise, or heuristic approaches to find
effective solutions. The complexity arises because kinematic synthesis
requires defining both the discrete components of a mechanism and their
connections, as well as determining their continuous spatial locations, making
it a mixed combinatorial and continuous problem.

In this paper, we focus on the inverse design of planar linkage mechanisms,
where the design involves determining how many joints a mechanism has, what
type of joints they are (either fixed or revolute joints, and how these joints
are connected to one another using rigid linkages. On top of this, the initial
position of the joints in space must also be determined. At each of these
levels, mechanisms can be infeasible, and the majority of randomly sampled
mechanisms tend to be invalid Heyrani Nobari et al. (2022). By examining the
continuous values of initial positions, the non-linear and discontinuous
nature of the problem becomes apparent. For instance, Figure 7 illustrates a
mechanism that writes the letter “B”, where all design variables are held
constant except for one joint (highlighted in red), which is moved in the x
and y directions to demonstrate the feasible regions for these variables. The
red-shaded area indicates where the mechanism fails to function. The three
grey patches of feasibility for just one joint underscore the problem’s non-
linearity and discontinuity, featuring multiple feasible regions. Furthermore,
adjusting one joint alters the shape and number of feasible regions for all
others. This discussion has focused solely on the continuous variables of
joint positioning, yet the combinatorial challenge of defining the mechanism’s
structure also plays a critical role. Not surprisingly, kinematic synthesis is
sometimes regarded more as an art than a science due to its complexities 10\.
(2018).

In this paper, we introduce a contrastive learning and optimization framework
to address such inverse design challenges, exemplifying broader issues in
kinematic synthesis and engineering design that involve integrating continuous
and discrete variables within functional and engineering constraints. Our
approach not only addresses existing difficulties in kinematic design but also
leverages deep learning to drive advancements in mechanical systems
(Regenwetter et al., 2022). A significant challenge in applying deep learning
to engineering is the accurate capture of physics-based performance, achieving
precision in meeting requirements, effective representation of design and
performance spaces, and the mitigation of data scarcity.

To demonstrate the effectiveness of our proposed framework, we focus on a
specific type of kinematic synthesis, called, the path synthesis problem. This
problem involves designing the aforementioned linkage mechanisms such that
they can trace a desired curve with one circular motion of a single actuator.
Planar linkage mechanisms use revolute and prismatic pairs to produce
rotating, oscillating, or reciprocating motions which can combine into complex
motions that trace desired curves. These mechanisms are commonly found in
everyday machinery, such as in the moving parts of printing presses, auto
engines, and robotics, making their design critical to a wide range of
applications.

Most research on the inverse design of planar linkage mechanisms has applied
deep learning to a limited problem of only generating a specific set of
mechanism types (typically, four-bar and six-bar mechanisms) (Cabrera et al.,
2002; Varedi-Koulaei & Rezagholizadeh, 2020; Ebrahimi & Payvandy, 2015; 10.,
2010; Khan et al., 2015b; McGarva, 1994; Chu & Sun, 2010a; Deshpande & Purwar,
2019a; Vasiliu & Yannou, 2001; Deshpande & Purwar, 2019b; 2020; Khan et al.,
2015a). In other cases, deep learning frameworks such as deep reinforcement
learning, are applied to one task (a single target curve) at a time, only to
find a single solution for a given desired curve (Fogelson et al., 2023)
instead of quickly generating a set of solutions for multiple target curves.
Additionally, as we show later, existing methods lack high precision and have
large errors in achieving the target.

To tackle these challenges of path synthesis in planar linkage mechanisms, we
introduce Learning-accelerated Inverse Kinematics (LInK), which synergizes
optimization with deep learning—termed “deep optimization”–to accelerate the
process and surpass traditional methods in performance. Our approach utilizes
a multi-modal, transformation-invariant contrastive learning framework to
accurately represent and integrate the different modalities of performance and
design spaces, enabling rapid retrieval of mechanisms from extensive datasets.
Specifically, we train a multimodal contrastive learning model on a dataset of
10 million linkage mechanisms and their simulated kinematics, pre-compute
embeddings for these mechanisms, and later retrieve them using embeddings of
target curves produced by the same model (see Figure 3). Additionally, we
refine these mechanisms through a hierarchical optimization algorithm,
leveraging optimization’s inherent robustness and accuracy to improve the
precision of our generated mechanisms. This integration of advanced
engineering design techniques allows us to excel in designing linkage
mechanisms, significantly surpassing existing methods in both efficiency and
precision with 28 times better performance and a 20 times increase in speed
compared to prior works on an established benchmark. We also create a
significantly more challenging benchmark and show that our method can find
solutions for all tasks in it too.

Figure 1: This figure illustrates the LInK algorithm’s workflow. LInK first
precomputes joint embeddings for mechanisms and curves, allowing it to
retrieve numerous candidate mechanisms for any new target curve within
seconds. With these high-quality initial candidates, a rapid BFGS optimization
process efficiently converges to path synthesis solutions. These solutions
significantly outperform existing methods in both speed and performance.

Our key contributions in this paper can be summarized as follows:

  * •

We introduce a contrastive learning approach to learn joint embeddings of
design and performance space for problems with both continuous and discrete
parameters.

  * •

We introduce a hierarchical deep optimization framework for integrating multi-
modal contrastive learning-based warm-start with traditional optimization for
inverse kinematic design of planar linkage mechanisms. We show that the
proposed approach significantly outperforms the state-of-the-art in both speed
and performance.

  * •

We develop a graph neural network framework named ’Hop Attention,’ designed to
effectively capture features sensitive to the number of hops within a graph
neural network. This feature is essential for analyzing planar linkage
mechanisms.

  * •

We propose a new set of benchmark problems for inverse kinematics in planar
linkage mechanisms, advancing beyond existing benchmarks with two new problems
that introduce significantly higher levels of complexity and exceed the simple
shapes produced by less complex mechanisms.

##  2 Background and Related Works

This section outlines relevant literature and contextualizes the research
presented here. We begin with an overview of the specific problem we aim to
address and its inherent complexities. We then review traditional
computational methods and recent learning-based advancements applied to this
problem. Finally, we discuss prior works that have influenced our framework
and the methodologies it incorporates.

###  2.1 Inverse Kinematics & The Path Synthesis Problem

Figure 2: Left: Demonstrates the path synthesis problem which involves a mixed
combinatorial and conoth the combinatorial and continuous values. This figure
also shows how mechanisms are represented as graphs. Right: This figure
illustrates the path the solver takes to solve the kinematics of a given
mechanism, showing how the skeleton of a mechanism (combinatorial design
variables) is involved in the mechanism solution. Initially, the solver starts
with the known joints (i.e., fixed and actuated joints highlighted red), and
step by step the solver solves joints with two solved neighbors (Eqn. 10). In
thtinuous nature and involves a non-linear solver whose solutions depend on
bis example, three steps are taken to solve the mechanism. The numbered joints
indicate the order of solution.

##### Inverse Kinematics:

Inverse kinematics is a computational technique primarily used in robotics,
computer graphics, and biomechanics to determine the positions and angles of a
mechanism’s joints, such as in a robotic arm. This method specifically targets
positioning the end effector, the terminal component of the mechanism, at a
desired location or along a predetermined trajectory. While standard
applications involve adjusting joint angles within an existing configuration,
inverse kinematic synthesis involves designing new mechanisms to achieve
specified motion objectives. The process is challenging due to its nonlinear
and combinatorial nature, which often leads to multiple viable solutions for a
given target. This complexity underscores the importance of inverse kinematics
as a field that requires precise, controlled movements and positioning.

##### Planar Linkage Mechanisms:

In our study of inverse kinematics, we focus on planar linkage mechanisms,
which consist of interconnected rigid links joined by joints that move within
a two-dimensional plane (see Fig. 2). These joints include revolute joints,
which allow free rotation while keeping the linkage endpoints fixed, and fixed
joints, which restrict the linkage end to a stationary position in 2D space.
The mechanisms under study have a single degree of freedom, controlled by a
rotary actuator connected to a fixed joint. This actuator’s movement dictates
the entire mechanism’s configuration, determining the positions of all other
joints and linkages based on its rotation.

##### Graph Representation and Optimization Formulation:

We model mechanisms as graphs. Let $M=(J,L)$ represent an undirected graph of
a planar linkage mechanism, where $J=\\{j_{1},j_{2},\ldots,j_{n}\\}$ is the
set of nodes (joints), and $L=\\{(j_{i},j_{j})|j_{i},j_{j}\in J\\}$ is the set
of edges (linkages). Each node $j_{i}$ has a 2D position in space and a binary
feature describing its type (fixed or free joint), denoted as
$(x_{i},y_{i},T_{i})$, where $x_{i}$ and $y_{i}$ are the positions of joint
$i$ in space and $T_{i}=1$ for fixed joints and $T_{i}=0$ for free joints (see
Fig. 2). In our representation, joint $j_{0}$ is always fixed and joint
$j_{1}$ is always free and there always exists a linkage $(0,1)\in L$ which
connects joints $j_{0}$ and $j_{1}$ which is the driven linkage actuated
around joint $j_{0}$ (see Fig. 2). In the specific problem we tackle in this
paper, the goal is to generate a mechanism of any size ($n\geq 2$) such that
the path traced by the final joint matches a given target path (see Fig. 2).
This problem can be formulated as an optimization problem:

$\begin{split}\min_{M,\theta_{start},\theta_{end}}\quad&||F(M,\theta_{start},\theta_{end})-P_{target}||_{2}\\\
\text{s.t.}\quad&DOF(M)=1\\\
&\text{Singularity}(M,\theta_{start},\theta_{end})=0\end{split}$ (1)

Where $M$ is the graph describing the mechanism and $\theta_{start}$ and
$\theta_{end}$ describe the range of motion for the actuator, $F(M)$ is the
path traced by the mechanism $M$ and $P_{target}$ is the target path , while
$DOF(M)$ is the degrees of freedom for mechanism $M$, which has to be always 1
(to be exactly defined by the motion of the single actuator), and
$\text{Singularity}(M,\theta_{start},\theta_{end})$ is a function that
determines if there are any singularities in a given mechanism $M$ for the
actuator range of motion from $\theta_{start}$ to $\theta_{end}$.

##### Singularity Constraints:

Singularities occur when the mechanism $M$ cannot be solved, leading to a
locked state where the actuator cannot complete its intended range of motion
(as shown in Figure 7 with the red region). The challenge involves defining
the mechanism $M$, which includes specifying the number of joints (integer),
their positions (continuous), their types (binary), and the linkage
arrangement $L$. This arrangement $L$ can be represented by the adjacency
matrix of $M$, containing $\frac{(n^{2}-n)}{2}$ binary values for $n$ joints.
The problem is non-linear and involves different variable types and
singularities are all non-linear and non-differentiable with respect to all or
some of the optimization variables. Graph structure also adds combinatorial
complexity. This complexity makes traditional optimization infeasible for
large $n$, offering an opportunity for deep learning to address these
difficult problems.

###  2.2 Computational Design of Planar Linkage Mechanisms

This section outlines significant research efforts aimed at solving inverse
kinematics problems similar to the one discussed in this paper. The literature
on path synthesis methods falls into three main categories: Numerical atlas-
based, Optimization-based, and Deep learning-based approaches, with detailed
discussions provided in Appendix A.

Numerical Atlas-Based Approaches: These involve creating a database of
mechanisms and their paths, which serves as a "numerical atlas" to find the
best match for a target path. However, comparing targets to large databases
can be computationally expensive, so these databases are often limited in size
or to specific mechanism types, like four-bar or six-bar mechanisms McGarva
(1994); Chu & Sun (2010b); Sun et al. (2015).

Optimization-Based Approaches: These involve mostly modified versions of
common conventional algorithms. Methods include genetic algorithms Lipson
(2008); Khan et al. (2015b), Fourier descriptor-based optimization Ullah &
Kota (1997); Wu et al. (2011), and mixed integer conic/non-linear programming
(MICP, MINLP) Pan et al. (2023). These approaches often refine existing
solutions Bächer et al. (2015); Thomaszewski et al. (2014) or are limited to
specific problems Lipson (2008); Baskar & Plecnik (2021). Despite their
generalizability, the effectiveness of these methods is limited by their
computational intensity and they often struggle (as our results also show) to
work for large mechanisms with many joints and target points due to the
highly-constrained nature of the problem.

Deep Learning-Based Approaches: These aim to accelerate and enhance path
synthesis by integrating traditional methods into data-driven frameworks.
Techniques involve using generative models such as variational autoencoders
(VAEs)Kingma & Welling (2014), clustering-based searchesDeshpande & Purwar
(2019a; b; 2020), and reinforcement learning (RL) Fogelson et al. (2023).
While promising, these methods either require retraining (RL-based methods)
for new targets and can be limited by dataset size and mechanism type
Deshpande & Purwar (2021); Vasiliu & Yannou (2001).

In this paper, we propose a hybrid method that combines the speed of deep
learning with the precision of optimization, aiming to surpass current state-
of-the-art methods in both inference time and performance.

###  2.3 Contrastive Learning and Retrieval

Retrieval, in the context of machine learning, refers to the process of
finding and returning relevant information from a dataset in response to a
query. This can involve matching query features with features of items in the
dataset to identify the best matches based on similarity metrics. While most
deep learning retrieval research has focused on vision applications like
content-based image retrieval (CIBR) Dubey (2022); Chen et al. (2023), our
work shifts attention to cross-modal retrieval, particularly retrieving graphs
of mechanisms based on their 2D kinematic paths. See Appendix B for more
details.

Most cross-modal works involve retrieving images based on multi-modal
information, such as text and other images. Early works like the
correspondence autoencoder (Corr-AE) Feng et al. (2014) train autoencoders for
text and images to create embeddings close to each other. The Deep Visual-
Semantic Hashing (DVSH) model Cao et al. (2016) and similar methods like
Textual-Visual Deep Binaries (TVDB) Shen et al. (2017) create joint embedding
spaces for text-based retrieval. Adversarial learning approaches, such as
Self-Supervised Adversarial Hashing (SSAH) Li et al. (2018), use adversarial
networks to learn features and hash codes for different modalities Wang et al.
(2017); Zhang et al. (2018); Gu et al. (2019); Ji et al. (2019). These methods
primarily specialize in text and image, making them less adaptable to general
cross-modal tasks. For our work, we need methods robust to general cross-modal
retrieval, leading us to contrastive learning-based retrieval.

Contrastive learning is a technique in machine learning where the model learns
to distinguish between similar (positive) and dissimilar (negative) pairs of
data points Jaiswal et al. (2021). This method is often used in unsupervised
or self-supervised settings to effectively encode information about data
without requiring explicit labels for every example. It focuses on pulling
representations of similar items closer and pushing representations of
dissimilar items farther apart in the embedding space. Seminal works in this
field have demonstrated its effectiveness in both supervised and unsupervised
tasks Hjelm et al. (2018); van den Oord et al. (2019); Chen et al. (2020a),
with foundational works like SimCLR Chen et al. (2020a; b) establishing a
general framework for contrastive learning. This inspired further developments
such as the Contrastive Language-Image Pretraining (CLIP) model, which
generates a cross-modal embedding space for text and images through a
generalized loss function:

$\displaystyle\mathcal{L}_{\text{CLIP}}$
$\displaystyle=-\log\frac{\exp(\text{sim}(f_{1},f_{2})/\tau)}{\sum_{j=1}^{N}\exp(\text{sim}(f_{1},f_{j})/\tau)},$
(2)

where $\text{sim}(f_{i},f_{j})$ measures the cosine similarity between feature
embeddings $f_{i}$ and $f_{j}$, and $\tau$ is the temperature parameter. This
generalizability makes CLIP suitable for downstream tasks, such as multi-modal
retrieval applications used in our methodology. Shared cross-modal embedding
spaces have been shown to be effective for retrieval in such cases. For
instance, Izacard et al. (2022) use contrastive embedding spaces for document
retrieval. Similar approaches have been explored for tasks like temporal
moment retrieval from videos based on text Zhang et al. (2021) and text-based
molecular retrieval for molecule design Liu et al. (2023). Contrastive
learning has immense potential for simultaneous modeling of design and
performance spaces in engineering applications. For the current problem, we
create a contrastive learning-enabled cross-modal approach for retrieving
mechanisms based on their kinematic paths, with more details provided in
subsequent sections.

##  3 Methodology

Our methodology is illustrated in Figure 1, where we begin by retrieving
potential matches from a dataset of 10 million mechanisms using a processed
version of the input curve. These initial matches seed our optimization
algorithm to refine the designs further. This section elaborates on our dual
approach. Firstly, we train a contrastive learning model that bridges the
physics underlying the mechanisms with their design representations. This
model is crucial for accurately identifying relevant mechanisms from the
extensive dataset. Secondly, we detail our optimization algorithm, termed the
Learning-accelerated Inverse Kinematics (LInK) algorithm, which builds upon
the foundations laid by the contrastive retrieval model. This ’deep
optimization’ strategy enhances our capability to fine-tune and improve the
designs based on the initial matches identified by the retrieval process.

###  3.1 Contrastive Learning Framework

Figure 3: Overview of the cross-modal contrastive learning approach in LInK.
Three different representations across multiple modalities are brought into
the same embedding space using contrastive learning. The bottom figure also
demonstrates the GHop architecture for hop attention that enables us to
capture the kinematics of mechanisms.

The first stage of our framework involves retrieving candidates from a massive
dataset of linkage mechanisms for a given target curve such that these
candidates are best suited to be refined for the given input. We use the LINKS
dataset proposed by Heyrani Nobari et al. (2022), which includes 100 million
mechanisms with up to 20 joints. Searching through such a massive dataset of
this size by comparing curves and using conventional methods would be
prohibitively expensive and time-consuming. Furthermore, most conventional
metrics for searching in such datasets are sensitive to rotations, scale, and
other spatial transformations, which makes them less useful. In our approach
to facilitate this, we build a cross-modal contrastive embedding to quickly
search through a massive dataset in practical time. In this section, we
describe the details of how we built this model and how we build robustness to
different spatial and geometric transformations.

####  3.1.1 Mechanism Representation & Model

We employ a subset of 10 million mechanisms from the LINKS dataset, which
undergoes extensive preprocessing for normalization. To achieve scale
invariance, we standardize the actuator arm length to 0.05 units. We further
normalize these mechanisms with respect to translation by centering the
actuators at the origin. Additionally, the mechanisms are aligned such that
the actuator angle is set to zero, standardizing both position and rotation.
This normalization process is critical for effective training of our
contrastive learning model. Furthermore, we meticulously process the dataset
to eliminate any redundant joints i.e., there is one joint in the mechanism
whose kinematic solution depends on the kinematics of all the other joints.

In our approach, we model planar linkage mechanisms as undirected graphs $M$
as we described prior with slight variation. The only difference in the graphs
we use for training is that instead of only two joint types and initial
positions, the node features of the graph are defined such that The first
element row indicates if a node is fixed or moving (i.e. 0 for moving joints
and 1 for fixed joints), the second element indicates whether a joint is
actuated and the third indicates if a given joint is the terminal joint (i.e.,
the joint which should produce the target curve), and the two other elements
in each row are set to be the initial positions of the nodes at time zero. In
training our model, we employ a sophisticated multi-layer, multi-level
architecture inspired by the Graph Isomorphism Network (GIN) approach proposed
by Xu et al. (2019). Off-the-shelf graph models proved inadequate due to the
intricate relationship between mechanism configurations and their kinematics,
which standard message-passing techniques struggle to capture. To address
this, we devised a novel architecture, which we term Graph Isomorphism Hop-
Attention (GHop), inspired by mechanism solvers. This architecture enables
both shallow and deep message passing within the same graph structure,
optimizing computational efficiency. Our approach modifies the traditional GIN
computation as follows:

$h_{v}^{(k)}=\operatorname{MLP}^{(k)}\left(\left(1+\epsilon^{(k)}\right)\cdot
h_{v}^{(k-1)}+\sum_{u\in\mathcal{N}(v)}h_{u}^{(k-1)}\right),$ (3)

where $h_{v}^{(k)}$ is the output features for node/joint $v$ at layer $k$ and
$\epsilon^{(k)}$ is a learnable parameter as described in the original work by
Xu et al. (2019) and $\mathcal{N}(v)$ refers to the set of neighbors of node
$v$. In our approach, we introduce hop-attention, where at the end of
computing all layers we compute a final feature for each node through a graph
attention (GAT) driven attention mechanism. To do this we take the outputs of
all layers concatenate them for each node and perform a single GAT convolution
on the resulting graph:

$h_{v}^{(GAT)}=\underset{n=1}{\overset{n=N}{\|}}\sigma\left(\sum_{j\in\mathcal{N}_{i}}\alpha_{ij}^{n}\mathbf{W}^{n}\underset{k=1}{\overset{k=K}{\|}}h_{v}^{(k)}\right),$
(4)

where $h_{v}^{(GAT)}$ is the output of the graph attention convolution and $N$
is the number of attention heads in the GAT and $W^{n}$ is the weights of each
attention head linear transformation while $\sigma$ is a non-linearity
function and $h_{v}^{(k)}$ is the output of the $k$-th layer of the GIN model.
Then we use this output of the GAT as the query and the outputs of each layer
for a given node as the keys and values for a multi-headed dot-product
attention which is applied to obtain the final output of each layer. This
means that at the end of each node/joint in the graph/mechanism, we have the
following:

$h_{v}=\left(\underset{i=1}{\overset{i=N}{\|}}\operatorname{softmax}\left(\frac{\mathbf{W}^{Q}_{i}\left[h_{v}^{(GAT)}\right]\left(\mathbf{W}^{K}_{i}\left[h_{v}^{(1)}h_{v}^{(2)}\cdots
h_{v}^{(M)}\right]^{T}\right)^{T}}{\sqrt{d_{k}}}\right)\left(\mathbf{W}^{V}_{i}\left[h_{v}^{(1)}h_{v}^{(2)}\cdots
h_{v}^{(M)}\right]^{T}\right)\right)\mathbf{W}^{O},$ (5)

where $h_{v}$ is the final output for node/joint $v$ and $N$ is the number of
attention heads and $\left[h_{v}^{(GAT)}\right]$ is a single row matrix with
$h_{v}^{(GAT)}$ as its only row and $\left[h_{v}^{(1)}h_{v}^{(2)}\cdots
h_{v}^{(M)}\right]$ is a matrix whose columns are assembled by the output of
each layer of the GIN model.
$\mathbf{W}^{Q}_{i},\mathbf{W}^{K}_{i},\mathbf{W}^{V}_{i}$ refer to the weight
of the linear transformation applied at each attention head, and
$\mathbf{W}^{O}$ is the final linear transformation applied to the
concatenated attention head outputs resulting the final representation for
each node/joint. The intuition behind this kind of architecture is evident in
the solver’s approach to computing the kinematics of the mechanisms. In the
solver we start with the known joints (i.e. 0-1 hops), then propagate the
solution to a joint with two known neigbours (1st hop) and repeat this process
until all joints are solved (variable number of hops depending on the
mechanism skeleton). This means to correctly describe the kinematics of the
mechanism each joint requires a different number of hops (see figure 2) and
conventional graph convolution approaches simply do not account for this as
the nature of problems they encounter are not like the ones we have in our
problem. Given this, it is intuitive to add a mechanism to account for the
variable number of hops needed for each joint, and our GHop approach enables
this through a simple yet effective attention mechanism. One thing to note is
the hop attention mechanism (HAT) is independent of the graph convolution
mechanism used and any graph convolution or message passing approach can be
used with HAT, which should enable improvement in existing models in problems
that involve similarly hop-count sensitive challenges (see figure 3 for
details on the architecture).

The final stage of our methodology involves employing an 8-layer model and a
hidden size of 768 to compute the final features for each node and take an
attention-pooled embedding for each mechanism as the overall mechanism
embedding.

####  3.1.2 Path Representation & Building Robust Invariances for Retrieval

In our approach to mechanism retrieval, the goal is to identify mechanisms
that accurately trace a target curve without concern for the curve’s scale,
location, orientation, or the tracing velocity at different parts of its
motion. To achieve invariance to these factors, we employ specific
preprocessing steps:

  * •

Normalization for Scale and Translation: We utilize the first two steps of
Procrustes analysis to normalize the scale and translation of the curves. This
is done by centering the curve around its mean and scaling it by its standard
deviation:

$X_{\text{normalized}}=\frac{X-\overline{X}}{\sqrt{\frac{\sum_{i=1}^{N}(X_{i}^{1}-\overline{X^{1}})^{2}+(X_{i}^{2}-\overline{X^{2}})^{2}}{N}}},$
(6)

Where $X$ is the curve being normalized represented as a $N\times 2$ matrix,
and $\overline{X}$ is the average value across each column of $X$, and
$X_{i}^{1}$ refers to the x value of the $i$-th row of $X$, and $X_{i}^{2}$
refers to the y value of the same row of $X$.

  * •

Building Rotation Invariance: To build invariance to rotation, we randomly
rotate curves during training.

  * •

Timing Invariance: We also need to build invariance to timing; i.e., the
curves should be made up of points that are equidistant from one another. To
do this, we simply take any given curve and interpolate points across the
curves such that the points are equidistant.

These steps, illustrated in Figure 3, ensure that our contrastive learning
model becomes robust to irrelevant geometric transformations, focusing purely
on the kinematic compatibility of the mechanisms with the target curves.

##### Modeling Partial Target Curves:

The curves in the LINKS dataset are always closed. However, the target curves
that one can give to a model during testing may not always be closed. To
effectively manage both closed and open target curves in our contrastive
cross-modal space, we implement a robust preprocessing strategy that
accommodates varying curve types. To do this, we randomly generate partial
curves of the original curves that a mechanism generates, normalize them
exactly like we do for the full curves, and remove timing from them (see
Figure 3). We then introduce partial curves as a separate module into the
overall contrastive learning space to ensure that open curves are also handled
by the contrastive representation space. This is a unique contribution of this
approach, as previous approaches are limited to closed curves.

####  3.1.3 Consolidated Methodology

In this section, we consolidate our methodologies for training contrastive
learning models and searching within the dataset. Our approach maps both
mechanisms and curves into a unified representation space using contrastive
learning. This mapping allows any introduced curve to be accurately paired
with corresponding mechanism candidates based on its spatial representation.
We employ a dual-model strategy for curves alongside a specialized graph-based
model for mechanisms. Specifically, the mechanism model utilizes the
previously discussed GI-HAT architecture. For curve processing, we utilize two
ResNet50 models (He et al., 2015): one handling full curves and another for
partial curves. The training leverages two distinct contrastive losses, akin
to the methodology used in the CLIP model (Radford et al., 2021):

$\mathcal{L}_{\text{LInK}}=\mathcal{L}_{\text{CLIP1}}+\gamma\mathcal{L}_{\text{CLIP2}}$
(7)

where $\mathcal{L}_{\text{CLIP1}}$ is the loss for the contrastive signal
between the mechanism models output, and $\mathcal{L}_{\text{CLIP2}}$ is the
contrastive loss for the partial and full curves models, and $\gamma$ is a
hyperparameter for applying different weights to these losses.

The individual CLIP-based loss terms, $\mathcal{L}_{\text{CLIP1}}$ and
$\mathcal{L}_{\text{CLIP2}}$, are computed as follows:

$\displaystyle\mathcal{L}_{\text{CLIP1}}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}-\log\frac{\exp(\text{sim}(f_{i},g_{i})/\tau)}{\sum_{j=1}^{N}\exp(\text{sim}(f_{i},g_{j})/\tau)}$
(8) $\displaystyle\mathcal{L}_{\text{CLIP2}}$
$\displaystyle=\frac{1}{N}\sum_{i=1}^{N}-\log\frac{\exp(\text{sim}(g_{i},h_{i})/\tau)}{\sum_{j=1}^{N}\exp(\text{sim}(g_{i},h_{j})/\tau)}$
(9)

where $\text{sim}(a,b)$ measures the cosine similarity between feature
embeddings $a$, and $b$ extracted by any of the models, and $\tau$ is the
temperature parameter controlling the sharpness of the similarity scores.
Importantly, the output of the mechanism model corresponds to $f_{i}$, the
output of the full curve model corresponds to $g_{i}$, and the output for the
partial curve model corresponds to $h_{i}$, and $N$ is the batch size.
Notably, mechanisms are matched to their corresponding curves, presumed
dissimilar to others in the batch. While this assumption holds given our
diverse dataset, no specific measures have been implemented to handle
potential similarities between different mechanisms’ curves. This setup is
visualized in figure 3, detailing the interaction between these components in
our training framework.

Once these models are trained, we have effectively established an approach to
map mechanisms (design space) and curves (performance space) into a unified
space. This mapping enables rapid searches across our extensive dataset for
any given curve by leveraging precomputed embeddings of all mechanisms. For
retrieval, we compute cosine similarities between the target curve’s embedding
and those of the mechanisms, subsequently ranking them by similarity.

###  3.2 Path Synthesis Framework

Our framework for path synthesis includes three stages.

  1. 1.

Initial Search: In the first stage, we perform a search on a 10 million sample
subset of the LINKS dataset and find an initial pool of candidates based on
the input curves using a contrastive learning-based search.

  2. 2.

Batch Optimization: In the second stage, we perform a gradient-based
optimization based on the BFGS BROYDEN (1970); Fletcher (1970); Goldfarb
(1970); Shanno (1970) method upon all of the mechanisms retrieved in the first
stage. To accelerate the process, we implement a batch BFGS optimization on
GPU which performs the BFGS optimization on multiple mechanisms
simultaneously. We describe this in more detail in this section. In this stage
we only perform 10 steps of optimization as optimizing the larger initial pool
of candidates can be slow.

  3. 3.

Final Refinement: After the batch optimization of the candidates, we then re-
evaluate the mechanisms and pick the top 10% of mechanisms amongst the
candidates to move to the next stage. In the final stage of optimization, we
further refine the smaller candidate pool using the BFGS optimization for 150
steps. at which point the final design is output to the user.

Each step of this framework is crafted to ensure a fast yet accurate synthesis
of paths, optimizing performance and computational efficiency. The following
sections will delve deeper into the specifics of these optimization
techniques.

####  3.2.1 GPU-Accelerated Batch Simulation of Linkage Mechanisms &
Differentiation

In our gradient-based optimization step, we focus only on optimizing the
positions of joints in mechanisms as connectivity parameters, which determine
how joints are linked, are not differentiable. To efficiently conduct these
optimizations, we employ a differentiable and vectorized solver that allows
for rapid simulation and differentiation, enabling practical batch processing
on GPUs.

We write a solver to simulate mechanisms, similar to the one proposed in
Heyrani Nobari et al. (2022) for dataset generation. This solver handles the
kinematics of mechanisms geometrically. While it is not suitable for the rare
instances of mechanisms with complex kinematic loops—a limitation discussed
further in the prior work Heyrani Nobari et al. (2022)—it suits our needs. The
advantage of this solver is twofold. The entire process is differentiable with
respect to the initial positions of the joints in a given mechanism and the
process can be vectorized for a batch of simulations which makes this solver
the perfect candidate for a differentiable GPU accelerated framework like
ours. Here we will briefly discuss the solver and how we vectorized the solver
for batches of simulations.

The operation of this solver involves a process akin to a graph traversal
algorithm (See Figure 2). Starting with joints of known positions (such as the
actuator arm at a specific angle or fixed joints), the solver iteratively
computes the positions of adjacent joints that connect to two already
determined positions. This sequence continues until the positions of all
joints in the mechanism are established, ensuring efficient computation across
batches of mechanism simulations.

To vectorize and parallelize this process we have to perform two things:

  * •

Pre-Sorting Mechanisms: To facilitate vectorization and batch processing, we
first establish a consistent solution path for all mechanisms in the dataset.
By pre-sorting the joints of each mechanism in a solution-ready order, we
eliminate the need to identify the solution path in real time during
simulations. This sorting, conveniently, has already been handled by Heyrani
Nobari et al. (2022) method, as proposed by the creators.

  * •

Resizing Mechanisms: The other necessary step that allows for the batch
solution by a vectorized GPU solver is that all of the mechanisms must have
the same size (i.e., the same number of joints). To do this, we add
unconnected fixed joints to all mechanisms that are smaller than the largest
mechanism in the dataset such that the mechanisms in the dataset all have the
same number of joints. This step ensures that multiple mechanisms can be
processed simultaneously without discrepancies in size.

With the mechanisms appropriately sorted and resized, the next step is to
implement a vectorized solver that can efficiently handle batches of
mechanisms. This solver operates on the principle of solving for any given
joint based on the positions of two known adjacent joints at each timestep
using the followingequation Bächer et al. (2015):

$\mathbf{X}_{i}^{T}=\mathbf{X}_{j}^{T}+||X_{i}^{T}-X_{j}^{T}||\times\mathbf{R}(\phi)\frac{X_{k}^{T}-X_{j}^{T}}{||X_{k}^{T}-X_{j}^{T}||}$
(10)

where $\mathbf{X}_{i}^{T}$ is the position of joint $i$ at timestep $T$ and
$\mathbf{R}(\phi)$ is the 2D rotation matrix for $\phi$ which is computed as:

$\begin{split}\phi=&\text{
Sign}\left[\left(X_{j,y}^{0}-X_{i,y}^{0}\right)\times\left(X_{j,x}^{0}-X_{k,x}^{0}\right)-\left(X_{j,y}^{0}-X_{k,y}^{0}\right)\times\left(X_{j,x}^{0}-X_{i,x}^{0}\right)\right]\times\\\
&\cos^{-1}{\frac{||X_{j}^{T}-X_{k}^{T}||^{2}+||X_{i}^{T}-X_{j}^{T}||^{2}-||X_{i}^{T}-X_{k}^{T}||^{2}}{2\times||X_{j}^{T}-X_{k}^{T}||^{2}\times||X_{i}^{T}-X_{j}^{T}||}}\end{split}$
(11)

where $\mathbf{X}_{i}^{0}$ is the initial positions of joint $i$. As can be
seen, these equations are easily vectorizable for multiple mechanisms
(essentially by turning the $X$ vectors into tensors with an additional
dimension with the same size as the batch size) and multiple time steps (all
$T$s that need to be solved for). Furthermore, all of the above-mentioned
equations are differentiable (except the sign function which will not be
differentiable at zero), which allows us to use backpropagation to compute the
gradients with respect to the initial positions $X^{0}$ and run the simulation
much faster in a vectorized fashion for a batch of mechanisms and for all
timesteps simultaneously. We implement such a GPU-accelerated solver using
Pytorch and use the automatic differentiation features of Pytorch to obtain
the gradients of the solution with respect to the initial positions of the
joints.

To optimize mechanisms for path synthesis we need to use this differentiable
solver to compute gradients for an objective function. In our approach, we
look at two metrics simultaneously. The first metric we look at is the bi-
directional Chamfer distance (CD) defined by:

$d_{CD}\left(S_{1},S_{2}\right)=\frac{1}{|S_{1}|}\sum_{x\in S_{1}}\min_{y\in
S_{2}}\|x-y\|_{2}+\frac{1}{|S_{2}|}\sum_{y\in S_{2}}\min_{x\in
S_{1}}\|x-y\|_{2}$ (12)

where $S_{1}$ is the set of points in the target curve and $S_{2}$ is the set
of points in the curve traced by a mechanism. Chamfer distance is a well-
established point cloud-based shape comparison metric and as we have seen this
plays a major role in our approach. Although Chamfer distance provides a great
metric for comparing the general shape of the curves, it lacks information
about path connectivity or ordering. As such we also have to look at some
metric that encompasses this. For this purpose, we use the objective function
proposed by Pan et al. (2023) in their MICP optimization approach. This metric
is defined as the ordered distance (OD):

$d_{OD}=\min_{o_{1}\in
O_{1}}\frac{2\pi}{N}\sum_{i=1}^{N}\left\|X^{\text{coupler}}_{o_{1}(i)}-X^{\text{target}}_{i}\right\|^{2}$
(13)

where $N$ is the number of points sampled in both the target curve and
mechanism traced curve and $X^{\text{coupler}}_{i}$ is the $i$-th point on the
curve traced by the mechanisms and $X^{\text{target}}_{i}$ is the $i$-th point
on the target curve. This while $o_{1}(i)$ is the $i-th$ point in the optimal
ordering to match the curves and $O_{1}$ is the set of all possible clockwise
and counter-clockwise orderings of the curve traced by a mechanism. The
constant $2\pi$ is kept for consistency across different works to allow for
meaningful comparison of results to prior works that use this metric. Finally,
in our gradient-based optimization, we have the following objective function:

$d=\gamma_{1}\times d_{OD}+\gamma_{2}\times d_{CD},$ (14)

In our approach, we assign weights to each metric in the optimization process,
with $\gamma_{1}=0.25$ for Chamfer distance and $\gamma_{2}=1.0$ for ordered
distance. Using Chamfer distance alone sometimes results in solutions that
disregard the ordering of points, focusing merely on matching point clouds.
Incorporating ordered distance helps prevent this issue by maintaining the
sequence integrity of points, leading to more accurate path tracing. However,
relying solely on ordered distance can yield suboptimal solutions and slow
down the convergence rate of the optimization process. Typically, a
combination of both metrics tends to produce robust solutions.

####  3.2.2 Batch BFGS Optimization on GPU

The BFGS approach for gradient-based optimization has been established as a
robust method for unconstrained non-linear gradient-based optimization.
However, there are two important matters that we must address before being
able to apply this method to our problem. First, our problem is not
unconstrained. This is because if the initial positions of the mechanism being
optimized are moved to a position that leads to the term in the inverse cosine
seen in equation 11, becoming greater than 1 or less than -1, we will have a
locking/infeasible mechanism. As such, instead of using a simple line search
for the optimization steps in BFGS, we use a heuristic approach to prevent
infeasible and locking configurations. We thus use the common Wolfe conditions
(Wolfe, 1971) for inexact line search to find the step size in each iteration
of optimization; however, on top of the Wolfe conditions, we also check that a
given stepsize does not lead to infeasible mechanisms by checking if the
simulation has resulted in any NaN values.

The traditional approach of performing BFGS optimization individually for each
mechanism is inefficient and time-consuming, particularly when leveraging CPU
computation. To address this and take advantage of GPU acceleration, we have
developed a new batch BFGS scheme that vectorizes the BFGS Hessian updates and
line searches across multiple mechanisms simultaneously. This adjustment
allows us to process all optimizations in parallel, significantly speeding up
the refinement of candidates and speeding up the generation of optimal
solutions. For the sake of brevity, further details on how this is done are
omitted here; interested readers are referred to our publicly available code
for the detailed implementation.

###  3.3 Implementation Details of the Optimization Framework

Our methodology begins by precomputing embeddings for 10 million mechanisms
from the LINKS dataset using our contrastive learning model tailored for
mechanisms. For any provided target curve, we normalize it as outlined
earlier, including a smoothing step for hand-drawn or rough curves. This
smoothing involves a fast Fourier transform to retain only the first seven
frequencies, which we use to reconstruct and compute the curve’s embedding.
All further optimization processes utilize the original, unsmoothed curves.
This step is only done for computing the embedding of the target curve; the
other optimization steps use the original input curves directly.

Given the embedding for the target curve, we then measure the cosine distance
between the target embedding and all of the precomputed embeddings of the
mechanisms in our dataset and pick the top 500 most similar (i.e., lowest
cosine distance) mechanisms for the optimization. As mentioned earlier, we
then perform 10 steps of batch BFGS on all of the 500 mechanisms and the
target curves. To do this, we normalize the target curve and the paths
generated by the candidate mechanisms, then find the optimal orientation of
the target for the curves the candidate mechanisms produce. We then perform a
brute-force grid search with 200 equally spaced rotation angles (equally
spaced from $0$ to $2\pi$), and identify the optimal rotation that minimizes
the Chamfer distance. This rotation is identified for each mechanism in the
candidate set. The target curves are then rotated to match the candidate paths
as well as possible. Then we perform the BFGS with curves that have been
oriented properly. Once the initial 10 steps of BFGS are performed, we then
pick the top 50 best-performing at this stage of optimization and perform an
additional 150 steps of BFGS on them. We call this ‘deep optimization’
framework for path synthesis the ‘Learning-accelerated Inverse Kinematics
(LInK)‘. We look at LInK’s performance and efficiency in the following
sections.

###  3.4 Manufacturability Considerations

So far we have only focused on the topic of kinematics and path synthesis.
However, not all mechanisms that can be kinematically solved can additionally
be manufactured. These mechanisms involve many linkages that move in space
that have to be staked in layers, and these linkages can collide with each
other; since each joint can be attached to multiple linkages, the joints may
have to exist in many layers, and can therefore collide with linkages in each
layer. This issue can be overcome with exotic designs for joints that enable
circumventing collisions, but that would be costly and difficult to achieve in
practice. As such, we must establish if any given mechanism is manufacturable
with conventional parts. We can thus formulate an optimization problem to
obtain the optimal configuration for a given linkage mechanism such that the
total number of layers needed to manufacture a mechanism is minimized.

Each linkage will have a $z$ value, which we denote with $z_{i}$ for the
$i$-th linkage in the set of linkages $L$. Each joint has an attachment to
some number of linkages and collides with a set number of linkages as well.
For each joint $j$ in the set of all joints $J$, we denote the set of linkages
attached to it as $A_{j}$ and the set of linkages colliding with the joint as
$C_{j}$ for the $j$-th joint. Finally, each linkage is in collision with some
other linkages. We denote the set of linkages colliding with the $i$-th
linkage as $O_{i}$, which excludes self-collision. Finally, a subset of $J$
denoted $G\subset J$ is the set of grounded joints. Our objective is to
minimize the maximum of $z_{i}$ for all linkages. To obtain $O_{i}$, $A_{j}$
and $C_{j}$, we simulate the mechanism and find the colliding geometries.
Given the above notation, we have the following design variables (the set $X$
contains all design variables):

$\displaystyle\text{Design Variables }X:$ (15) $\displaystyle
z_{i}\quad\forall i\in L$ $\displaystyle u_{j}\quad\forall j\in J$
$\displaystyle v_{j}\quad\forall j\in J$ $\displaystyle
y_{ji}\in\\{0,1\\}\quad\forall j\in J-G\text{ and }\forall i\in C_{j}$
$\displaystyle x_{ik}\in\\{0,1\\}\quad\forall i\in L\text{ and }\forall k\in
O_{i}$ $\displaystyle M\text{ which will represent the maximum value amongst
}z_{i}$

With these design variables and assuming $N$ is a very large constant, we have
the following optimization problem:

$\displaystyle\min_{X}M$ (16) $\displaystyle\begin{aligned} &\text{ s.t.
}0\leq z_{i}<|L|,\quad\forall i\in L\\\ &z_{i}-z_{j}+N*x_{ij}\geq
1,\quad\forall i\in L\text{ and }\forall j\in O_{i}\\\
&z_{j}-z_{i}+N*\left(1-x_{ij}\right)\geq\text{1},\quad\forall i\in L\text{ and
}\forall j\in O_{i}\\\ &u_{j}\leq z_{i},\forall i\in A_{j}\\\ &v_{j}\geq
z_{i},\forall i\in A_{j}\\\ &z_{i}\geq v_{j}+1-N\times y_{ji},\quad\forall
j\in J-G\text{ and }\forall i\in C_{j}\\\ &z_{i}\leq
u_{j}-1+N\times\left(1-y_{ji}\right),\quad\forall j\in J-G\text{ and }\forall
i\in C_{j}\\\ &M\geq z_{i},\quad\forall i\in L\end{aligned}$

This optimization simulation will yield z values which will be integers
ranging from 0 to a maximum of $|L|-1$, which would be equivalent to having
one layer per linkage. It is also important to note that the collisions for
fixed joints are not considered. This is because each linkage can be grounded
in space without the need for a joint pass through the mechanism. This
formulation lends itself very well to mixed-integer linear programming (MILP)
algorithms. In this case, we use Gurobi to solve the problem and determine if
any given mechanism yields a feasible MILP problem; if so, what the optimal
configuration for that given mechanism is. Note that since all of the
operations in our approach happen in batches, at every step, we obtain many
candidates, and as such we easily check the manufacturability of each
mechanism from best to worst performing until the most manufacturable
mechanism is found. This is an important aspect of the process that prior
methods have ignored; few check for manufacturability at all. In our work from
here on to the end, all mechanisms we identify as solutions are
manufacturable; the above MILP is feasible for the mechanisms that are shown
and discussed in this paper. For more details on this aspect of the work and
visualizations of linkage assemblies please see Appendix C.

##  4 Results & Discussion

In this section, we conduct a few different sets of experiments using our
model and compare the results of our approach to the state of the art. Before
discussing the details of the experiment, it is important to first discuss our
experimental settings and the evaluation metrics we use to evaluate both our
method and other approaches.

###  4.1 Evaluation Metrics & Experiment Details

As we discussed before in path synthesis the primary aim is to come up with
mechanisms that allow for the tracing of a desired target path. As such to
measure the performance of any model for this task we need to look at metrics
that allow for comparing two given paths purely based on their shape/path.
Here we track two highly correlated metrics but each provides slightly
different insight. These metrics are the Chamfer distance and ordered distance
as discussed in Equation 12 and Equation 13 respectively. We also discussed
the trade-offs of each metric with regards to comparing paths in Section
3.2.1. However, as far as comparison

Finally, it is important to note that in all of our tests, we resample all
curves, both target and traced curves by the mechanisms, by interpolating the
curves to 2000 equidistant points. This is necessary to remove timing bias in
the curves traced by mechanisms since the motion of the mechanism is not at a
constant velocity. Removing timing is needed since the objective of path
synthesis is to trace a path without any prescribed kinematics such as speed
and acceleration. Now that the metrics and how we measure them are established
we will conduct some tests to compare our method to prior works and finally,
we will introduce two new benchmarks for this problem to establish a concrete
benchmark for future works to try to handle.

###  4.2 Path Synthesis Case Studies

Figure 4: This figure shows the 8 test curves used in prior works (Left) and
24 random samples from the LINKS test subset (Middle) and the alphabet test
curves (Right).

We will first test our model on three sets of problems and report the results
of our approach on each of these test cases. Currently, the existing
literature lacks standardized test sets. As such, it is hard for us to compare
our model against the state of the art as these methods have limited code
availability, and replicating them can be complicated. Despite this, a
specific set of test curves have been proposed by Pan et al. (2023), which
have been evaluated using multiple methods. However, this set is very small
(i.e., only 8 curves), and is not well suited for rigorous testing of path
synthesis methods such as ours. However, we use this dataset as it provides an
avenue for comparison to the state of the art.

Table 1: This table presents the quantitative results of our model on the
three different test sets we use to evaluate our model. For each test case, we
run our model 5 times and report the best solution found within the 5 runs.
The numbers following $\pm$ are the standard deviation across the samples in
each test case. Note that although we normalize the LINKS test curves and the
alphabet test curves we use the original scaling on the 8 test curves as they
were measured in prior works at that scale.

Test Set | Curve Types | Chamfer Distance | Ordered Distance
---|---|---|---
LINKS Test Subset | Curves Generated By Random Planar Mechanisms | 0.0057 $\pm$ 0.0039 | 0.0016 $\pm$ 0.0042
Popular 8 Curves Test | Hand-Drawn Curves | 0.034 $\pm$ 0.029 | 0.015 $\pm$ 0.021
Alphabet Benchmark | Curves Mimicking Capital Letters of The English Alphabet | 0.051 $\pm$ 0.041 | 0.229 $\pm$ 0.671

Moving beyond, we also introduce two sets of benchmark problems, each of which
provides a different insight into the performance of any method on this
problem. First, we use in our approach 200 curves traced by a subset of the
LINKS dataset Heyrani Nobari et al. (2022), which we set aside during training
for the purpose of testing our method. These curves represent problems that we
call ‘in-distribution,‘ as these curves are part of the same distribution of
data used for training the model. However, it is important to test any method
upon curves that are very different from the distribution of the data and
provide an additional challenge by being particularly exotic shapes; curves
which planar linkage mechanisms would be ordinarily considered capable of
producing. In line with this, we introduce a rather challenging problem:
tracing the capital letters in the English alphabet. To do this, we provide 26
continuous curves that align with the shapes of the capital letters in the
English alphabet. This set includes shapes that are drastically different from
the typical curves in the dataset, hence giving us an ‘out-of-distribution’
test case. Furthermore, these curves are riddled with sharp corners and
features that are particularly difficult for planar linkage mechanisms to
handle.

Figure 5: This figure shows the best results from LInK on the 8 test curves
used in prior works and the overlayed resulting path traced by the mechanism
and the target. As it is visible here LInK has matched the target curves
effectively without much challenge. For visualization of the underlying
mechanisms please see Appendix C. Figure 6: This figure shows 20 random
results from LInK on the larger LINKS test subset. We do not visualize
mechanisms and only visualize the curve traced by the mechanisms to save
space. As it can be seen in almost all cases the solution found is nearly
identical to the target curve, which shows that when it comes to in-
distribution target curves LInK performs exceptionally well. Figure 7: Right:
This figure shows results from LInK on the larger alphabet test set. As can be
seen, the paths produced by the generated mechanisms do not capture the
alphabet shapes as well as we see in the other test sets. This shows that
despite LInK performing well even on this test this test set presents a more
challenging benchmark for path synthesis and can serve as a good benchmark for
future methods. The ordered distance value for each curve is displayed above
the curves for each alphabet path. Left: Showing the mechanism that traces B
and the feasible region highlighted in blue where the highlighted joint can
move without causing a singularity. This shows the non-linear and
discontinuous nature of the problem and highlights the challenge of performing
kinematic synthesis. Note that this is only assuming all other joints keep the
same positions. Changing the position of any joint could change the feasible
region for all other joints making the problem very challenging from the
perspective of conventional optimization.

Each of the three cases discussed above is visualized in figure 4, which
displays all of the curves in the alphabet and the test curves proposed by Pan
et al. (2023), while showing a random subset of the dataset test curves. We
run our model for all three test cases and display the results visually in
figures 7 and 5 for the alphabet and 8 test curves respectively. Finally, we
show a few random example solutions from the LINKS test set in figure 6. As it
can be seen when it comes to the 8 test curves our model performs
exceptionally well and as we will see later when compared to the state of the
art our approach significantly outperforms prior methods. Furthermore, we see
that the model’s performance on the in-distribution test case is also very
good, which should be expected as these samples are similar to the training
curves. However, we can see that the curves produced for the alphabet test
case do not follow a similar trend. Although for some letters the model has
performed well, for many of the complex curves the results are under-whelming
and do not trace the curves as expected. This test set, therefore, serves as a
great benchmark for future methods as it presents a more demanding challenge,
given the other challenges we test our model seem to be handled relatively
easily.

Beyond the qualitative results discussed so far, we measure the performance of
our model on each of the test sets and report both the ordered distance and
Chamfer distance in table 1. As it can be seen as we saw earlier the
qualitative observations we had are indeed confirmed with both metrics, with
the model performing well on the in-distribution test and the 8 test curves,
while performing notably worse on the challenging alphabet curves (see the
ordered distance of the alphabet test).

###  4.3 Comparison To State of Art Methods

Figure 8: This figure shows how the performance of the LInK method changes with the maximum number of nodes for each curve. As expected, we see a general decrease in error with an increase in complexity, with diminishing returns. Table 2: This table presents the performance metrics for our method LInK compared to the state-of-the-art deep learning model GCP-HOLO proposed by Fogelson et al. (2023) and optimization-based methods MINLP and MICP proposed by Pan et al. (2023). The values reported here are ordered distance in Eqn.13. The values for GCP-HOLO, MICP, and MINLP are reported based on the measurements in the original papers Fogelson et al. (2023); Pan et al. (2023). We observe that on average, LInK has 61X less error than GCP-HOLO, while being 33X faster (shown in Table 3) Target Curve | LInK | LInK (8) | LInK (12) | LInK (16) | GCP-HOLO | MINLP | MICP
---|---|---|---|---|---|---|---
| 0.0094 | 2.53 | 0.079 | 0.026 | 2.73 | 0.71 | 2.58
| 0.0086 | 0.42 | 0.012 | 0.012 | 0.32 | 1.25 | 1.37
| 0.013 | 0.21 | 0.015 | 0.013 | 0.44 | 0.70 | 0.77
| 0.0041 | 2.01 | 0.23 | 0.061 | 1.58 | 1.39 | 2.52
| 0.018 | 0.85 | 0.28 | 0.024 | 0.32 | 1.25 | 1.37
| 0.0039 | 0.34 | 0.0063 | 0.0056 | 1.07 | 0.77 | 1.28
| 0.0043 | 0.16 | 0.010 | 0.010 | 0.87 | 1.36 | 1.36
| 0.060 | 0.45 | 0.090 | 0.072 | 0.38 | 0.36 | 0.32
Mean | 0.015 | 0.87 | 0.089 | 0.029 | 0.92 | 0.97 | 1.45

In their work, Pan et al. (2023) proposed 8 curves for which they used MICP to
perform path synthesis and reported their results. Since then, the current
state-of-the-art method in path synthesis using deep learning (i.e., GCP-HOLO
Fogelson et al. (2023)), has evaluated their method on this set as well. As
such, we will test our method on this set of target paths as well, and compare
our performance to the state of the art based on these paths.

In Table 2, we present the results of our approach for each of the 8 curves:
both GCP-HOLO and MICP are used, and as we can see, LInK significantly out-
performs these approaches by an order of magnitude demonstrating more than 94%
improvement over the state of the art. One of the main explanations for the
shortcomings of the approaches mentioned in this study is that these
approaches are limited to smaller mechanisms, due to the high computational
cost of dealing with increasingly larger mechanisms with these models. Despite
limiting their size, these models are still significantly slower than our
proposed framework. We discuss the efficiency and inference speed in later
sections. Regardless, we run LInK on limited-sized datasets to demonstrate
that LInK outperforms the state-of-the-art even when limited to smaller
mechanisms. For these runs, we only conduct the contrastive search on the
subset of mechanisms that are smaller than a certain threshold (See LInK(8)
through LInK(16) in table 2).

###  4.4 Inference Speed & Scalability

We have so far only looked at the pure performance of the mentioned methods in
terms of how well they can perform path synthesis. However, an important
aspect of path synthesis algorithms is their limitations in terms of inference
speed and scalability to large, complex mechanisms. To test the scalability
and inference speed of different methods, we investigate both the inference
time and size of the mechanisms each method can produce.

Table 3: This table shows the average inference time for each method. For SOTA
methods we report the rough inference time reported by the authors (they
report a range, and we report the middle of that range here), as the code for
the work by Pan et al. (2023) is not publicly available. For GCP-HOLO, we run
both of their variants on our hardware using an Intel i9-14900K and an RTX
4090 GPU, which provides better inference speed than the authors reported. We
also run our model on the same hardware. In each case, 10 tests are run, and
the average time is reported below. The percentages are reported based on the
inference speed up compared to MINLP. We observe that on average, LInK is 33X
faster than the fastest GCP-HOLO.

Model | Maximum of Joints | Solver timesteps | Inference Speed (s)
---|---|---|---
MINLP Pan et al. (2023) | 7 | 20 | 36000 (00.00%)
MICPPan et al. (2023) | 7 | 20 | 27000 (25.00%)
GCP-HOLOFogelson et al. (2023) | 11 | 20 | 2475.12 (93.12%)
GCP-HOLO + CMAE-ESFogelson et al. (2023) | 11 | 20 | 2498.23 (93.06%)
LInK (Ours) | 20 | 2000 | 75.18 (99.79%)

Table 3 presents the inference speed and maximum mechanism sizes each method
is capable of producing within that inference time. Furthermore, we show in
this table the simulation fidelity, which determines the number of points
sampled for each method. We can see that LInK is not only 99.67% faster than
the MINLP method, but it is also more than 95% faster than the fastest method
in the state of the art capable of producing mechanisms that go beyond 4-bar
and 6-bar mechanisms, GCP-HOLO. Beyond this, we see that LInK is capable of
producing much more complex mechanisms with up to 20 joints and simulating
with 2000 timesteps during optimization. This is in contrast with the 11
joints and 20 timesteps that the best competing method is working with while
having 95% slower inference. These results demonstrate that frameworks like
LInK can be very powerful for speeding up these kinds of optimization-based
methods by removing the majority of the burden from the optimizer, enabling
faster and more precise optimization.

###  4.5 Validating The Effectiveness of GHop Architecture

To validate the superiority of our GHop architecture over standard GNN
architectures, we conducted a comparative performance analysis. We trained the
LInK framework using baseline Graph Convolutional Network (GCN) and Graph
Isomorphism Networks (GIN) models, each configured with an equivalent number
of layers, hops, and hidden sizes, resulting in approximately the same
parameter count: 38M for both GCN and GIN, and 31M for GHop.

We evaluated the effectiveness of each model by monitoring the
$\mathcal{L}_{\text{CLIP1}}$ loss on validation data throughout the training
period. The results indicated that the GHop architecture consistently achieved
lower validation losses compared to the GCN and GIN models, which failed to
reduce the loss to comparable levels. This discrepancy in performance was
significant, with GHop’s validation loss being less than half that of the GCN,
and the naive GIN model even showed a higher loss, suggesting a poor capture
of the contrastive relationships between mechanisms and curves.

These findings are graphically represented in Figure 9, where GHop’s
superiority is clearly evident, illustrating its enhanced capability in the
LInK framework compared to traditional GNN architectures.

Figure 9: This figure shows how the validation performance
($\mathcal{L}_{\text{CLIP1}}$) of each GNN architecture changes in the LInK
training. We see here that the proposed GHop architecture is performing
significantly better than alternative GNN architectures.

##  5 Conclusion & Future Work

In this work, we introduced LInK, a novel framework that combines contrastive
learning with optimization techniques for effectively solving complex
engineering design problems, with a specific focus on the path synthesis
problem for planar linkage mechanisms. By leveraging a vast dataset and
employing a multi-modal, transformation-invariant contrastive learning
framework, LInK adeptly captures intricate physics and design representations,
enabling the rapid and precise retrieval and optimization of mechanisms. This
approach not only significantly expedites the search process, but also
enhances precision through a hierarchical optimization algorithm. The results
demonstrate the effectiveness of LInK through improvement in both speed and
performance by orders of magnitude.

For future work, the adaptability and robustness of LInK open new avenues for
exploring its applicability across a broader spectrum of engineering
challenges beyond linkage synthesis. As demonstrated, the integration of
contrastive learning-based joint design and performance space representation
and optimization has the potential to facilitate advancements in areas where
traditional methods falter due to the complexity or combinatorial nature of
design spaces. Further research could explore the extension of LInK’s
methodology to more complex mechanisms, the incorporation of dynamic
simulation for moving mechanisms, and a fully end-to-end generative model.

## References

  * 10\. (2010) _On the Extension of a Fourier Descriptor Based Method for Four-Bar Linkage Synthesis for Generation of Open and Closed Paths_ , volume Volume 2: 34th Annual Mechanisms and Robotics Conference, Parts A and B of _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference_ , 08 2010.  doi: 10.1115/DETC2010-29028.  URL https://doi.org/10.1115/DETC2010-29028.
  * 10\. (2018) _Kinematic Synthesis Using Reinforcement Learning_ , volume Volume 2A: 44th Design Automation Conference of _International Design Engineering Technical Conferences and Computers and Information in Engineering Conference_ , 08 2018.  doi: 10.1115/DETC2018-85529.  URL https://doi.org/10.1115/DETC2018-85529.  V02AT03A009.
  * Bächer et al. (2015) Moritz Bächer, Stelian Coros, and Bernhard Thomaszewski.  Linkedit: Interactive linkage editing using symbolic kinematics.  _ACM Trans. Graph._ , 34(4), jul 2015.  ISSN 0730-0301.  doi: 10.1145/2766985.  URL https://doi-org.libproxy.mit.edu/10.1145/2766985.
  * Baskar & Plecnik (2021) Aravind Baskar and Mark Plecnik.  Synthesis of Watt-Type Timed Curve Generators and Selection From Continuous Cognate Spaces.  _Journal of Mechanisms and Robotics_ , 13(5):051003, 05 2021.  ISSN 1942-4302.  doi: 10.1115/1.4050197.  URL https://doi.org/10.1115/1.4050197.
  * BROYDEN (1970) C. G. BROYDEN.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations.  _IMA Journal of Applied Mathematics_ , 6(1):76–90, 03 1970.  ISSN 0272-4960.  doi: 10.1093/imamat/6.1.76.  URL https://doi.org/10.1093/imamat/6.1.76.
  * Cabrera et al. (2002) J.A. Cabrera, A. Simon, and M. Prado.  Optimal synthesis of mechanisms with genetic algorithms.  _Mechanism and Machine Theory_ , 37(10):1165–1177, 2002.  ISSN 0094-114X.  doi: https://doi.org/10.1016/S0094-114X(02)00051-4.  URL https://www.sciencedirect.com/science/article/pii/S0094114X02000514.
  * Cao et al. (2016) Yue Cao, Mingsheng Long, Jianmin Wang, Qiang Yang, and Philip S. Yu.  Deep visual-semantic hashing for cross-modal retrieval.  In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , KDD ’16, pp. 1445–1454, New York, NY, USA, 2016. Association for Computing Machinery.  ISBN 9781450342322.  doi: 10.1145/2939672.2939812.  URL https://doi.org/10.1145/2939672.2939812.
  * Chen et al. (2020a) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pp. 1597–1607. PMLR, 2020a.
  * Chen et al. (2020b) Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton.  Big self-supervised models are strong semi-supervised learners.  _Advances in neural information processing systems_ , 33:22243–22255, 2020b.
  * Chen et al. (2023) Wei Chen, Yu Liu, Weiping Wang, Erwin M. Bakker, Theodoros Georgiou, Paul Fieguth, Li Liu, and Michael S. Lew.  Deep learning for instance retrieval: A survey.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 45(6):7270–7292, 2023.  doi: 10.1109/TPAMI.2022.3218591.
  * Chu & Sun (2010a) Jinkui Chu and Jianwei Sun.  A New Approach to Dimension Synthesis of Spatial Four-Bar Linkage Through Numerical Atlas Method.  _Journal of Mechanisms and Robotics_ , 2(4), 08 2010a.  ISSN 1942-4302.  doi: 10.1115/1.4001774.  URL https://doi.org/10.1115/1.4001774.  041004\.
  * Chu & Sun (2010b) Jinkui Chu and Jianwei Sun.  Numerical atlas method for path generation of spherical four-bar mechanism.  _Mechanism and Machine Theory_ , 45(6):867–879, 2010b.  ISSN 0094-114X.  doi: https://doi.org/10.1016/j.mechmachtheory.2009.12.005.  URL https://www.sciencedirect.com/science/article/pii/S0094114X09002286.
  * Deshpande & Purwar (2019a) Shrinath Deshpande and Anurag Purwar.  A Machine Learning Approach to Kinematic Synthesis of Defect-Free Planar Four-Bar Linkages.  _Journal of Computing and Information Science in Engineering_ , 19(2), 02 2019a.  ISSN 1530-9827.  doi: 10.1115/1.4042325.  URL https://doi.org/10.1115/1.4042325.  021004\.
  * Deshpande & Purwar (2019b) Shrinath Deshpande and Anurag Purwar.  Computational Creativity Via Assisted Variational Synthesis of Mechanisms Using Deep Generative Models.  _Journal of Mechanical Design_ , 141(12), 09 2019b.  ISSN 1050-0472.  doi: 10.1115/1.4044396.  URL https://doi.org/10.1115/1.4044396.  121402\.
  * Deshpande & Purwar (2020) Shrinath Deshpande and Anurag Purwar.  An Image-Based Approach to Variational Path Synthesis of Linkages.  _Journal of Computing and Information Science in Engineering_ , 21(2), 10 2020.  ISSN 1530-9827.  doi: 10.1115/1.4048422.  URL https://doi.org/10.1115/1.4048422.  021005\.
  * Deshpande & Purwar (2021) Shrinath Deshpande and Anurag Purwar.  An image-based approach to variational path synthesis of linkages.  _Journal of Computing and Information Science in Engineering_ , 21(2), 2021.
  * Dubey (2022) Shiv Ram Dubey.  A decade survey of content based image retrieval using deep learning.  _IEEE Trans. Cir. and Sys. for Video Technol._ , 32(5):2687–2704, may 2022.  ISSN 1051-8215.  doi: 10.1109/TCSVT.2021.3080920.  URL https://doi.org/10.1109/TCSVT.2021.3080920.
  * Ebrahimi & Payvandy (2015) Saeed Ebrahimi and Pedram Payvandy.  Efficient constrained synthesis of path generating four-bar mechanisms based on the heuristic optimization algorithms.  _Mechanism and Machine Theory_ , 85:189–204, 2015.  ISSN 0094-114X.  doi: https://doi.org/10.1016/j.mechmachtheory.2014.11.021.  URL https://www.sciencedirect.com/science/article/pii/S0094114X14003036.
  * Feng et al. (2014) Fangxiang Feng, Xiaojie Wang, and Ruifan Li.  Cross-modal retrieval with correspondence autoencoder.  In _Proceedings of the 22nd ACM International Conference on Multimedia_ , MM ’14, pp. 7–16, New York, NY, USA, 2014. Association for Computing Machinery.  ISBN 9781450330633.  doi: 10.1145/2647868.2654902.  URL https://doi.org/10.1145/2647868.2654902.
  * Fletcher (1970) R. Fletcher.  A new approach to variable metric algorithms.  _The Computer Journal_ , 13(3):317–322, 01 1970.  ISSN 0010-4620.  doi: 10.1093/comjnl/13.3.317.  URL https://doi.org/10.1093/comjnl/13.3.317.
  * Fogelson et al. (2023) Mitchell B. Fogelson, Conrad Tucker, and Jonathan Cagan.  GCP-HOLO: Generating High-Order Linkage Graphs for Path Synthesis.  _Journal of Mechanical Design_ , 145(7):073303, 04 2023.  ISSN 1050-0472.  doi: 10.1115/1.4062147.  URL https://doi.org/10.1115/1.4062147.
  * Goldfarb (1970) Donald Goldfarb.  A family of variable-metric methods derived by variational means.  _Mathematics of computation_ , 24(109):23–26, 1970.
  * Gu et al. (2019) Wen Gu, Xiaoyan Gu, Jingzi Gu, Bo Li, Zhi Xiong, and Weiping Wang.  Adversary guided asymmetric hashing for cross-modal retrieval.  In _Proceedings of the 2019 on International Conference on Multimedia Retrieval_ , ICMR ’19, pp. 159–167, New York, NY, USA, 2019. Association for Computing Machinery.  ISBN 9781450367653.  doi: 10.1145/3323873.3325045.  URL https://doi.org/10.1145/3323873.3325045.
  * He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition, 2015.
  * Heyrani Nobari et al. (2022) Amin Heyrani Nobari, Akash Srivastava, Dan Gutfreund, and Faez Ahmed.  Links: A dataset of a hundred million planar linkage mechanisms for data-driven kinematic design.  _Volume 3A: 48th Design Automation Conference (DAC)_ , Aug 2022.  doi: 10.1115/detc2022-89798.
  * Hjelm et al. (2018) R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio.  Learning deep representations by mutual information estimation and maximization.  In _International Conference on Learning Representations_ , 2018.
  * Izacard et al. (2022) Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave.  Unsupervised dense information retrieval with contrastive learning.  _Transactions on Machine Learning Research_ , 2022.  ISSN 2835-8856.  URL https://openreview.net/forum?id=jKN1pXi7b0.
  * Jaiswal et al. (2021) Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon.  A survey on contrastive self-supervised learning.  _Technologies_ , 9(1), 2021.  ISSN 2227-7080.  doi: 10.3390/technologies9010002.  URL https://www.mdpi.com/2227-7080/9/1/2.
  * Ji et al. (2019) Zhenyan Ji, Weina Yao, Wei Wei, Houbing Song, and Huaiyu Pi.  Deep multi-level semantic hashing for cross-modal retrieval.  _IEEE Access_ , 7:23667–23674, 2019.  doi: 10.1109/ACCESS.2019.2899536.
  * Khan et al. (2015a) N Khan, I Ullah, and M Al-Grafi.  Dimensional synthesis of mechanical linkages using artificial neural networks and fourier descriptors.  _Mechanical Sciences_ , 6(1):29–34, 2015a.
  * Khan et al. (2015b) N. Khan, I. Ullah, and M. Al-Grafi.  Dimensional synthesis of mechanical linkages using artificial neural networks and fourier descriptors.  _Mechanical Sciences_ , 6(1):29–34, 2015b.  doi: 10.5194/ms-6-29-2015.  URL https://ms.copernicus.org/articles/6/29/2015/.
  * Kingma & Welling (2014) Diederik P Kingma and Max Welling.  Auto-encoding variational bayes, 2014.
  * Li et al. (2018) Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, and Dacheng Tao.  Self-supervised adversarial hashing networks for cross-modal retrieval.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 4242–4251, 2018.
  * Lipson (2008) Hod Lipson.  Evolutionary synthesis of kinematic mechanisms.  _Artificial Intelligence for Engineering Design, Analysis and Manufacturing_ , 22(3):195–205, 2008.  doi: 10.1017/S0890060408000139.
  * Liu et al. (2023) Shengchao Liu, Weili Nie, Chengpeng Wang, Jiarui Lu, Zhuoran Qiao, Ling Liu, Jian Tang, Chaowei Xiao, and Animashree Anandkumar.  Multi-modal molecule structure–text model for text-based retrieval and editing.  _Nature Machine Intelligence_ , 5(12):1447–1457, Dec 2023.  ISSN 2522-5839.  doi: 10.1038/s42256-023-00759-6.  URL https://doi.org/10.1038/s42256-023-00759-6.
  * McGarva (1994) John R McGarva.  Rapid search and selection of path generating mechanisms from a library.  _Mechanism and Machine Theory_ , 29(2):223–235, 1994.  ISSN 0094-114X.  doi: https://doi.org/10.1016/0094-114X(94)90032-9.  URL https://www.sciencedirect.com/science/article/pii/0094114X94900329.
  * Mnih et al. (2013) Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.  Playing atari with deep reinforcement learning, 2013.
  * Pan et al. (2023) Zherong Pan, Min Liu, Xifeng Gao, and Dinesh Manocha.  Joint search of optimal topology and trajectory for planar linkages.  _The International Journal of Robotics Research_ , 42(4-5):176–195, 2023.  doi: 10.1177/02783649211069156.  URL https://doi.org/10.1177/02783649211069156.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision, 2021.
  * Regenwetter et al. (2022) Lyle Regenwetter, Amin Heyrani Nobari, and Faez Ahmed.  Deep Generative Models in Engineering Design: A Review.  _Journal of Mechanical Design_ , 144(7):071704, 03 2022.  ISSN 1050-0472.  doi: 10.1115/1.4053859.  URL https://doi.org/10.1115/1.4053859.
  * Shanno (1970) David F Shanno.  Conditioning of quasi-newton methods for function minimization.  _Mathematics of computation_ , 24(111):647–656, 1970.
  * Shen et al. (2017) Yuming Shen, Li Liu, Ling Shao, and Jingkuan Song.  Deep binaries: Encoding semantic-rich cues for efficient textual-visual cross retrieval.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 4097–4106, 2017.
  * Sohn et al. (2015) Kihyuk Sohn, Honglak Lee, and Xinchen Yan.  Learning structured output representation using deep conditional generative models.  In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), _Advances in Neural Information Processing Systems_ , volume 28. Curran Associates, Inc., 2015.  URL https://proceedings.neurips.cc/paper/2015/file/8d55a249e6baa5c06772297520da2051-Paper.pdf.
  * Sun et al. (2015) Jianwei Sun, He Lu, and Jinkui Chu.  Variable step-size numerical atlas method for path generation of spherical four-bar crank-slider mechanism.  _Inverse Problems in Science and Engineering_ , 23(2):256–276, 2015.  doi: 10.1080/17415977.2014.890615.  URL https://doi.org/10.1080/17415977.2014.890615.
  * Thomaszewski et al. (2014) Bernhard Thomaszewski, Stelian Coros, Damien Gauge, Vittorio Megaro, Eitan Grinspun, and Markus Gross.  Computational design of linkage-based characters.  _ACM Trans. Graph._ , 33(4), jul 2014.  ISSN 0730-0301.  doi: 10.1145/2601097.2601143.  URL https://doi-org.libproxy.mit.edu/10.1145/2601097.2601143.
  * Ullah & Kota (1997) Irfan Ullah and Sridhar Kota.  Optimal Synthesis of Mechanisms for Path Generation Using Fourier Descriptors and Global Search Methods.  _Journal of Mechanical Design_ , 119(4):504–510, 12 1997.  ISSN 1050-0472.  doi: 10.1115/1.2826396.  URL https://doi.org/10.1115/1.2826396.
  * van den Oord et al. (2019) Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding, 2019.
  * Varedi-Koulaei & Rezagholizadeh (2020) SM Varedi-Koulaei and H Rezagholizadeh.  Synthesis of the four-bar linkage as path generation by choosing the shape of the connecting rod.  _Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science_ , 234(13):2643–2652, 2020.  doi: 10.1177/0954406220908616.  URL https://doi.org/10.1177/0954406220908616.
  * Vasiliu & Yannou (2001) Adrian Vasiliu and Bernard Yannou.  Dimensional synthesis of planar mechanisms using neural networks: Application to path generator linkages.  _Mechanism and Machine Theory_ , 36:299–310, 02 2001.  doi: 10.1016/S0094-114X(00)00037-9.
  * Wang et al. (2017) Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen.  Adversarial cross-modal retrieval.  In _Proceedings of the 25th ACM International Conference on Multimedia_ , MM ’17, pp. 154–162, New York, NY, USA, 2017. Association for Computing Machinery.  ISBN 9781450349062.  doi: 10.1145/3123266.3123326.  URL https://doi.org/10.1145/3123266.3123326.
  * Wei et al. (2017) Yunchao Wei, Yao Zhao, Canyi Lu, Shikui Wei, Luoqi Liu, Zhenfeng Zhu, and Shuicheng Yan.  Cross-modal retrieval with cnn visual features: A new baseline.  _IEEE Transactions on Cybernetics_ , 47(2):449–460, 2017.  doi: 10.1109/TCYB.2016.2519449.
  * Wolfe (1971) Philip Wolfe.  Convergence conditions for ascent methods. ii: Some corrections.  _SIAM Review_ , 13(2):185–188, 1971.  doi: 10.1137/1013035.  URL https://doi.org/10.1137/1013035.
  * Wu et al. (2011) Jun Wu, Q. J. Ge, Feng Gao, and W. Z. Guo.  On the Extension of a Fourier Descriptor Based Method for Planar Four-Bar Linkage Synthesis for Generation of Open and Closed Paths.  _Journal of Mechanisms and Robotics_ , 3(3), 07 2011.  ISSN 1942-4302.  doi: 10.1115/1.4004227.  URL https://doi.org/10.1115/1.4004227.  031002\.
  * Wu et al. (2017) Yiling Wu, Shuhui Wang, and Qingming Huang.  Online asymmetric similarity learning for cross-modal retrieval.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 3984–3993, 2017.  doi: 10.1109/CVPR.2017.424.
  * Xu et al. (2019) Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka.  How powerful are graph neural networks?, 2019.
  * Yang et al. (2017) Erkun Yang, Cheng Deng, Wei Liu, Xianglong Liu, Dacheng Tao, and Xinbo Gao.  Pairwise relationship guided deep hashing for cross-modal retrieval.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 31(1), Feb. 2017.  doi: 10.1609/aaai.v31i1.10719.  URL https://ojs.aaai.org/index.php/AAAI/article/view/10719.
  * Zhang et al. (2021) Hao Zhang, Aixin Sun, Wei Jing, Guoshun Nan, Liangli Zhen, Joey Tianyi Zhou, and Rick Siow Mong Goh.  Video corpus moment retrieval with contrastive learning.  In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_ , SIGIR ’21, pp. 685–695, New York, NY, USA, 2021. Association for Computing Machinery.  ISBN 9781450380379.  doi: 10.1145/3404835.3462874.  URL https://doi.org/10.1145/3404835.3462874.
  * Zhang et al. (2018) Xi Zhang, Hanjiang Lai, and Jiashi Feng.  Attention-aware deep adversarial hashing for cross-modal retrieval.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , September 2018.

##  Appendix A Broader Discussion On Computational Kinematic Synthesis of
Planar Linkage Mechanisms

As discussed prior most methods in kinematic synthesis fall into 3 primary
categories: Numerical atlas-based approaches, Optimization-based approaches,
and Deep learning-based approaches.

##### Numerical Atlas:

In numerical atlas approaches first a database of mechanisms is created or
gathered and simulated to obtain the paths that each mechanism traces in the
database. Then these mechanisms and their corresponding paths are used as a
“numerical atlas”. Given a target path, one can look up all of the mechanisms
in a database to find the best matching candidate in the database McGarva
(1994); Chu & Sun (2010b); Sun et al. (2015). It can be computationally
expensive to compare a target to a large number of mechanisms therefore in the
majority of these approaches, the database is usually limited in size or
limited to a handful of mechanism types, such as a four-bar or a six-bar
mechanism.

##### Optimization-Based Kinematic Synthesis:

A different kind of approach to solving this kind of problem is applying
different optimization approaches. Some use genetic algorithms to the problem
Lipson (2008); Khan et al. (2015b), While others have developed Fourier
descriptor-based optimization algorithms Ullah & Kota (1997); Wu et al.
(2011). At the same time, many approaches focus on specific problems without
generalizability or refining existing solutionsLipson (2008); Bächer et al.
(2015). Most of the aforementioned methods commonly work with existing
solutions Bächer et al. (2015); Thomaszewski et al. (2014) or are limited to
specific kinds of mechanisms or problems Lipson (2008); Baskar & Plecnik
(2021). Most promising amongst the recent developments is the method proposed
by Pan et al. (2023), which formulates the optimization problem discussed
prior as a mixed integer conic-programming (MICP) and mixed-integer nonlinear
programming (MINLP). However, these methods use a branch and bounds method to
optimize the problem, which limits their capacity in the number of joints and
the number of points in their target curves Pan et al. (2023). Furthermore,
even for simple problems the time to solve any given input is significant (in
the order of 5-10 hours) which makes these kinds of methods not only limited
by mechanism size and target path point count but prohibitive for generating
multiple candidates for the same problem or generating solutions for complex
targets that require many points to describe.

##### Deep Learning And Kinematic Synthesis:

Given these limitations in the conventional approaches, data-driven and deep
learning approaches have been proposed to accelerate and improve path
synthesis results. These studies often integrate traditional methods, such as
the “numerical atlas” and optimization techniques, into data-driven
frameworks. For instance, Deshpande & Purwar (2019a) have refined a method
that merges the numerical atlas concept with optimization strategies
(Deshpande & Purwar, 2019a; b; 2020), utilizing variational autoencoders
(VAEs) (Kingma & Welling, 2014) and a clustering-based search to identify
suitable candidates for generating the required coupler curves. Additionally,
their subsequent research applies VAEs and conditional VAEs (Sohn et al.,
2015) to create mechanisms. However, the datasets employed in these studies
are typically small and restricted to certain mechanism types (e.g., four-bar,
six-bar, etc.), with one instance being a dataset comprising 6818 linkage
mechanisms (Deshpande & Purwar, 2021). Other data-driven studies have explored
mechanism generation by conditioning on specific paths (Vasiliu & Yannou,
2001), yet this approach is also mainly limited to four-bar mechanisms. In
contrast to these strategies, methods are attempting to apply machine learning
to emulate optimization techniques. An example of this is a study employing
deep Q learning (Mnih et al., 2013). More recently Fogelson et al. (2023) have
introduced a novel reinforcement learning approach that accelerates the
process of optimization in comparison to MICP-based approaches mentioned
before. Although these reinforcement learning-based strategies are not
confined to particular mechanisms, they require retraining for each new target
shape, thus limiting their generality and introducing issues similar to the
aforementioned MICP-based methods.

It is clear that machine learning techniques hold significant potential in
this domain, yet current methodologies exhibit two main shortcomings. Firstly,
many techniques are constrained to specific types of mechanisms and problems.
Secondly, when these methods are not limited by mechanism type, they mimic
optimization and perform the task rather slowly. In our work, we developed a
hybrid method that combined the acceleration that is associated with deep
learning frameworks and the precision and robustness that is associated with
optimization to strike a balance and significantly outperform the state of the
art in both inference time and raw performance.

##  Appendix B Detailed Review of Deep Learning-Based Retrieval and
Contrastive Learning

Here we provide a more in-depth review of the works we discussed in the main
body of the paper for further context around the topics of deep learning-based
retrieval and contrastive learning.

###  B.1 Learning Based Retrieval

When it comes to deep learning-based approaches for retrieval the majority of
research has been dedicated to vision applications, specifically content-based
image retrieval (CIBR). Dubey (2022) and Chen et al. (2023) have conducted
comprehensive surveys on the deep learning developments for different types
and applications of CIBR, which provide a more in-depth discussion on the
topic for readers seeking a more detailed review of the topic. Here we will
focus on discussing some of the more directly related works of research on the
topic. In our work, we aim to retrieve mechanisms (represented as graphs)
given target kinematics (2D paths), which makes the approach one that involves
different modalities of data representation. As such here we focus on cross-
modal retrieval methods, discuss some of the ways that others have approached
problems of this nature, and refer readers to the aforementioned surveys for a
review of other types of retrieval models.

Cross-modal image retrieval refers to the task of retrieving images in a
dataset based on information involving more than one modality. A prominent
example of this is text and image-based retrieval, which comes up in text
query-based image retrieval and image labeling based on text retrieval. One of
the earliest works on the topic by Feng et al. (2014) called correspondence
autoencoder (Corr-AE) trains two autoencoders simultaneously one for text and
one for images and attempts to build correspondence by encouraging the
embeddings of the two autoencoders to be close to each other through an L2
loss function between the text and image embeddings produced by each
autoencoder. Cao et al. (2016) introduce a novel Deep Visual-Semantic Hashing
(DVSH) model that introduces an end-to-end deep learning architecture called
visual-semantic fusion network, which simultaneously captures spatial
dependencies in images and temporal dynamics in text. This network is
responsible for learning a joint embedding space that mitigates the
heterogeneity between the two modalities, thereby generating compact,
similarity-preserving hash codes, which can be used for text-based retrieval.
Others have looked at similar approaches with slight differences in
architecture and implementation such as Textual-Visual Deep Binaries (TVDB)
Shen et al. (2017) and many more approaches that have been explored for this
kind of cross-modal task with similar overall methodologies Wei et al. (2017);
Yang et al. (2017); Wu et al. (2017) with slight nuances the detailed
description of which is outside the scope of this section. Furthermore, some
research has been focused on performing the same task through adversarial
learning approaches. For example Li et al. (2018) introduces Self-Supervised
Adversarial Hashing (SSAH). The methodology combines adversarial learning with
a self-supervised semantic network to achieve this goal. Two adversarial
networks are employed to learn the high-dimensional features and corresponding
hash codes for different modalities. Simultaneously, a self-supervised
semantic network, leveraging multi-label annotations, guides the adversarial
learning process to maximize semantic relevance and feature distribution
consistency between modalities. Again various works of research have been
proposed along the same lines of using adversarial networks Wang et al.
(2017); Zhang et al. (2018); Gu et al. (2019); Ji et al. (2019) for this kind
of text-to-image or image-to-text retrieval.

The common pattern across the aforementioned works and the majority of works
that surround the topic is the fact that these methods all focus primarily on
text and image, which means the authors often tailor their approaches and
architecture to specialize in this kind of task specifically. This makes these
models much better at performing cross-modal retrieval on images and text but
makes the approaches very difficult to adapt to general cross-modal retrieval
tasks such as the one we are performing in this work. This means that we must
look at methods that are robust to cross-modal retrieval tasks which do not
specialize at their core to deal specifically with image and text and this is
where contrastive learning-based retrieval methods come into play.

###  B.2 Contrastive Learning and Contrastive Learning Based Retrieval

As discussed before most of the methods built for retrieval using deep
learning were built to be specialized and included mechanisms specific to text
and image datasets which are not adaptable to general tasks like the one we
intend to perform here. Given this and the fact that the inverse kinematic
problem involves a very general and context-free nature, the kinematics
simulations are the only directly available information for any given
mechanism and this exists in isolation, unsupervised and self-supervised
approaches are best suited for building a robust retrieval mechanism for
kinematic synthesis. These requirements lend themselves rather well to
contrastive learning.

Contrastive learning at its core refers to unsupervised or self-supervised
learning methods that aim to develop models that learn to distinguish between
similar and dissimilar data points Jaiswal et al. (2021). These methods gained
popularity as a result of the early seminal works in deep contrastive learning
Hjelm et al. (2018); van den Oord et al. (2019); Chen et al. (2020a) that
demonstrated the powerful capabilities of these approaches for both
conventionally supervised deep learning problems (e.g. contrastive learning
feature extraction for classification and regression) and unsupervised
representation learning. Among these early approaches, the work by Chen et al.
(2020a; b), known as an effective framework for contrastive learning (SimCLR)
stands out as a rather generalizable and robust framework. Although SimCLR
itself was built and tested on a single modality of data, primarily images,
similar approaches soon emerged in other works of research that adapted
similar techniques to contrastive learning on multiple modalities. Most
notably the work by Radford et al. (2021), introduces the Contrastive
Language-Image Pretraining (CLIP) model, which creates a crossmodal embedding
space for text and images. However, unlike highly specialized text-image
retrieval models the CLIP approach is rather generalizable to other cross-
modal embedding spaces. This is evident in the loss function used by CLIP
(Equation 2). This means that so long as trainable models can be implemented
for data representations in each modality the CLIP approach applies to the
problem. Given this robustness and generalizability, we incorporate the CLIP
loss in our approach for unsupervised representation learning in the inverse
kinematics problem.

The concept of a shared cross-modal embedding space is highly analogous to the
retrieval problem discussed before. This is because these models map data from
different modalities into the same space based on similarity, which makes
these embedding spaces ripe for retrieval problems. As it turns out, this has
been explored in many works of research surrounding cross-modal retrieval. In
an approach very similar to what we do here Izacard et al. (2022) uses this
concept of contrastive embedding spaces for retrieval for the task of document
retrieval. These kinds of approaches have also been explored for cross-modal
retrieval tasks as well, such as temporal moment retrieval from videos based
on text Zhang et al. (2021) or text-based molecular retrieval for molecule
design Liu et al. (2023), which demonstrates the effectiveness of cross-modal
embedding spaces built using contrastive learning for retrieval tasks. In our
approach, we employ contrastive learning-enabled cross-modal spaces for
mechanism retrieval based on kinematic objectives. The details of our
implementation are discussed in later sections.

##  Appendix C Additional Visualizations

Here we will visualize the mechanisms that trace the curves in the 8 test
curves and alphabet tests. For each set of tests, we will visualize the
kinematics of the entire mechanisms in 2D graph and path figures and provide
the optimal assembly of the mechanisms in 3D as well to visualize what the
actual assembled mechanism would look like. Note that these solutions are the
best perfoming solutions as such they tend to have more joints since higher
complexity mechanisms tend to produce more precise solutions.

###  C.1 Eight Test Curves

Figure 10 shows the mechanisms associated with the solutions found by LInK for
each of the 8 test curves. This figure shows the mechanisms and their
kinematic in 2D while Figures 11, 12 show the optimal assemblies for each of
these mechanisms to avoid collisions while using the fewest layers in the
z-axis according to the optimization scheme described in Section 3.4.

Figure 10: This figure shows the best results from LInK on the 8 test curves.
As it is visible here LInK has matched the target curves effectively without
much challenge. Here we visualize the mechanisms in 2D and also plot the paths
traced by all the joints in each mechanism. Figure 11: This figure shows the
3D optimal assemblies for mechanisms displayed in Figure 10(first four
curves). Figure 12: This figure shows the 3D optimal assemblies for mechanisms
displayed in Figure 10(last four curves).

###  C.2 Alphabet Test Curves

The figures in this subsection are similar in nature to the figures in the
prior subsection. Again it is important to note the fact that these are the
best perfoming solutions which means that even for simpler curves such the
letter ‘O’ the produced mechanism is more complex because the more complex
mechanisms achieve an exceptionally accurate solution. Note that the LInK
approach can be set to be limited any number of joint count as we mentioned in
the main text to obtain simpler mechanisms.

Figure 13: This figure shows the best results from LInK on the first 9
alphabet test curves. As it is visible here LInK has matched the target curves
effectively without much challenge. Here we visualize the mechanisms in 2D and
also plot the paths traced by all the joints in each mechanism. Figure 14:
This figure shows the best results from LInK on the 9-18 alphabet test curves.
As it is visible here LInK has matched the target curves effectively without
much challenge. Here we visualize the mechanisms in 2D and also plot the paths
traced by all the joints in each mechanism. Figure 15: This figure shows the
best results from LInK on the last 8 alphabet test curves. As it is visible
here LInK has matched the target curves effectively without much challenge.
Here we visualize the mechanisms in 2D and also plot the paths traced by all
the joints in each mechanism. Figure 16: This figure shows the 3D optimal
assemblies for mechanisms displayed in Figure prior figures for the alphabet
mechanisms (A-D). Figure 17: This figure shows the 3D optimal assemblies for
mechanisms displayed in Figure prior figures for the alphabet mechanisms
(E-H). Figure 18: This figure shows the 3D optimal assemblies for mechanisms
displayed in Figure prior figures for the alphabet mechanisms (I-L). Figure
19: This figure shows the 3D optimal assemblies for mechanisms displayed in
Figure prior figures for the alphabet mechanisms (M-P). Figure 20: This figure
shows the 3D optimal assemblies for mechanisms displayed in Figure prior
figures for the alphabet mechanisms (Q-T). Figure 21: This figure shows the 3D
optimal assemblies for mechanisms displayed in Figure prior figures for the
alphabet mechanisms (U-X). Figure 22: This figure shows the 3D optimal
assemblies for mechanisms displayed in Figure prior figures for the alphabet
mechanisms (Y, Z).