# Transfer learning for improved generalizability in causal physics-informed
neural networks for beam simulations ††thanks: _Preprint submitted to
Elsevier_

Taniya Kapoor, Hongrui Wang, Alfredo Núñez, Rolf Dollevoet
Department of Engineering Structures
Faculty of Civil Engineering and Geosciences
Delft University of Technology
The Netherlands
{t.kapoor, h.wang-8, a.a.nunezvicencio<EMAIL_ADDRESS>

###### Abstract

This paper introduces a novel methodology for simulating the dynamics of beams
on elastic foundations. Specifically, Euler-Bernoulli and Timoshenko beam
models on the Winkler foundation are simulated using a transfer learning
approach within a causality-respecting physics-informed neural network (PINN)
framework. Conventional PINNs encounter challenges in handling large space-
time domains, even for problems with closed-form analytical solutions. A
causality-respecting PINN loss function is employed to overcome this
limitation, effectively capturing the underlying physics. However, it is
observed that the causality-respecting PINN lacks generalizability. We propose
using solutions to similar problems instead of training from scratch by
employing transfer learning while adhering to causality to accelerate
convergence and ensure accurate results across diverse scenarios. Numerical
experiments on the Euler-Bernoulli beam highlight the efficacy of the proposed
approach for various initial conditions, including those with noise in the
initial data. Furthermore, the potential of the proposed method is
demonstrated for the Timoshenko beam in an extended spatial and temporal
domain. Several comparisons suggest that the proposed method accurately
captures the inherent dynamics, outperforming the state-of-the-art physics-
informed methods under standard $L^{2}$-norm metric and accelerating
convergence.

_K_ eywords Transfer learning $\cdot$ Causality $\cdot$ Physics-informed
neural networks (PINNs) $\cdot$ Biharmonic equations $\cdot$ Euler-Bernoulli
beam $\cdot$ Timoshenko beam $\cdot$ Elastic foundation

##  1 Introduction

Beams on elastic foundations (Fig. 1) are a fundamental and indispensable
structural component in civil engineering, providing critical support and
stability to different and diverse structures ([1, 2, 3, 4]). Due to their
characteristic to distribute loads, mitigate deformations, and enhance
structural stability, these beams are extensively utilized in various
structures, such as railway tracks ([1]), pile foundations embedded in soils
([5]), and longitudinal fibers in a composite elastomer ([3]), among others.
Understanding their dynamics is essential for ensuring the structural
integrity of these systems, developing effective maintenance strategies,
optimizing machine performance, refining design methodologies, and enabling
precise control mechanisms. These issues highlight the need for advanced
methodologies to simulate and predict the underlying dynamics of beams on
elastic foundations, facilitating safer, more efficient, and reliable
structures and systems.

However, accurately predicting the dynamics of beams on elastic foundations
through experiments and measurements could be infeasible ([6]). Conducting
many experiments with varying materials, conditions, and prototypes becomes
impractical and prohibitively costly. In practice, finite element-based
software provides a viable alternative for simulating such scenarios ([7]).
However, these software solutions are restricted in generalization. For
instance, even a slight change in the problem domain requires performing the
entire new simulation from scratch, including mesh creation and adjustments
([8]). This non-generalization becomes particularly problematic when different
aspects of the system need to be investigated separately or when multiple
design iterations are required. The number of simulations necessary for
tackling a design problem can quickly escalate into thousands, making the task
laborious and time-consuming.

Figure 1: Simply supported beam on an elastic foundation under varying
transverse force

Recently, deep learning and neural networks, in particular, have been used
extensively as surrogates to model the underlying physical phenomenon ([9,
10]). However, even state-of-the-art supervised machine-learning approaches
encounter similar challenges as traditional experimental methods, requiring
substantial input-output data at various fidelities to learn the underlying
dynamics effectively. This large data requirement poses a significant hurdle,
as obtaining such vast data can be arduous and resource-intensive ([8]).

One potential approach to mitigate the need for an enormous amount of data is
to incorporate the underlying physics into the learning procedure, thereby
guiding the neural network based on physics principles as presented by [11,
12, 13, 14], among others. One popular class of methods that adopts this
approach is physics-informed neural networks (PINNs) proposed by [11]. PINNs
are a form of semi-supervised learning where the boundary and initial
conditions serve as input-output pairs while the solution is regularized by
the governing partial differential equations (PDEs). However, several
challenges for PINNs have emerged, including spectral bias ([15]), shock
learning ([16]), generalization with even slight changes in physical
parameters and computational domain ([17, 18, 19]), and difficulties dealing
with large coefficients ([20, 21, 22]).

One possible reason for such challenges encountered by vanilla PINNs is its
causality-agnostic training. PINNs trained by gradient descent are implicitly
biased towards first approximating PDE solutions at later times before
resolving the initial conditions ([15]), therefore profoundly violating
temporal causality. Physical systems are known to possess an inherent causal
structure. For instance, the deflection of the beam at any point in time is
causally linked to the previous state of the system (deflection), the physical
properties of the beam, and the external forces acting on it. This causality
is a fundamental aspect of how the beam equations accurately model the
behavior of beams in response to loads, making it a useful tool in engineering
and physics. The PINN model could learn complicated solutions to PDEs when the
causality is considered, enabling progressive sequential-time learning of the
solution.

Our work proposes to train PINN while respecting causality in the context of
structural engineering, referred to as causal PINN hereafter. However, as we
present in this work, even after employing causal PINN, the models lack
generalizability to different initial conditions and computational domains,
requiring each new problem to be solved from scratch. This limitation reverts
the problem to the need for extensive simulations for each problem. To
mitigate this issue, we propose employing transfer learning (TL) ([23]) in
conjunction with causal PINN. The idea of transfer learning is to utilize the
knowledge acquired from solving one problem in the form of trained model
parameters to be utilized in a similar or related problem, accelerating the
training process.

We examine the application of PINNs on well-known Euler-Bernoulli and
Timoshenko beam models on elastic foundations, specifically the Winkler
foundation ([1, 24]). Numerical experiments reveal that the standard PINN
approach struggles to capture the underlying physics accurately, particularly
for large space-time domains, even in cases with closed-form analytical
solutions. This paper proposes a novel approach to simulate beams on elastic
foundations using the Euler-Bernoulli and Timoshenko theories, employing a
transfer learning-based causal PINN framework to conduct comprehensive
experiments. Specifically, transferring knowledge from one initial condition
to another, handling noisy initial conditions, transferring knowledge for
beams of different lengths, and systems with significant time dependencies are
addressed. The primary contributions of this research paper are as follows:

A causality-respecting PINN loss function addresses the aforementioned
limitations and effectively enforces the relevant physics. However,
implementing this modified causal loss function requires a denser neural
network with more parameters. Considering the importance of various factors in
engineering structure design and the impracticality of simulating every
instant, transfer learning is proposed within the causal PINN architecture. By
incorporating transfer learning, the parameters of the previously trained
model are leveraged to initialize and train new models. Consequently, this
reduces the computational burden and enables faster convergence for subsequent
tasks, improving the efficiency of simulating the dynamics of beams on elastic
foundations.

The rest of the paper is structured as follows: Section 2 presents related
works to this manuscript. Section 3 provides a detailed discussion of vanilla
and causal PINN. Section 4 introduces the proposed framework of fusing
transfer learning with causal PINN to train different models. Section 5
presents the numerical experiments results, showcasing the effectiveness of
our methodology in addressing challenging beam problems where the vanilla PINN
([11]) and self-adaptive PINN ([25]) formulation fails. Finally, the main
findings are summarized, and conclusions drawn from this study are presented
in Section 6.

##  2 Related works

This section outlines the pertinent studies conducted within the domain of
transfer learning-driven PINNs, causal PINNs, and physics-informed
methodologies for the simulation of beam models.

Applying transfer learning within PINNs has garnered significant attention
([26]). Notably, [27] predicted laser deposition temperature fields accurately
without labeled data, using physical losses and transfer learning. In another
work, [28] utilized transfer learning for accurate temperature field inversion
with limited observations, employing a PINN and optimal position selection.
[29] developed a multi-objective loss function and transfer learning for
accurate elastoplastic solid mechanics solutions through PINN. In a different
study, [30] proposed a transfer learning-based PINN framework for efficient
stress-strain constitutive modeling. While our research aligns with the
fundamental principle of leveraging transfer learning, a distinguishing aspect
lies in our consideration of causality during the training of the models.

In the literature, research has been conducted to enforce causality in the
PINN framework without incorporating transfer learning ([31]). In another
work, [32] proposed a causal framework incorporating transfer learning to
simulate time-dependent PDEs. Although our work shares similar ideas of
incorporating causality and utilizing transfer learning within the PINN
framework, we employ transfer learning to train distinct models under diverse
conditions. Conversely, [32] employs transfer learning within a particular
problem by segmenting the domain into multiple subdomains and leveraging
insights from one subdomain to another, employing the concepts of domain
decomposition and PINNs ([33]).

Recently, beam simulations have concentrated on physics-informed
methodologies, largely omitting the considerations of causality and transfer
learning. Noteworthy works include [34], which utilized PINNs for estimating
nonlinear bending behavior within a confined domain. Similarly, [35, 36]
delved into applying PINNs for the system of beam models and moving load
problems, albeit within the limited domain confines. [37] introduced a spatio-
temporal PINN tailored for analyzing the dynamics of cantilever beams. In
[38], a self-adaptive PINN framework capable of accommodating varying load
conditions is presented. Additionally, [39] sought to enhance predictions by
incorporating supplementary data, all still constrained within the confined
domain bounded by the capabilities of PINNs. This work aims to enhance the
potential of physics-informed methodologies for simulating beam dynamics.

##  3 Vanilla and causal PINN

This section is structured into two subsections. First, we provide an overview
of the architecture of the vanilla PINN ([11]). Second, a modification in the
PINN loss function leading to the incorporation of causality in the PINN loss
function, as proposed by [15], is presented.

###  3.1 Vanilla PINN

Recently, PINNs have been widely used for solving PDEs across diverse domains,
including but not limited to works by [40, 41, 42]. PINNs are based on deep
neural network (DNN) architecture, and the idea of PINN is to incorporate
physical knowledge in the loss function of DNN. The loss function consists of
two terms - a data term and a physics term. The data term ensures that the
neural network fits the provided data points, while the physics term enforces
the PDE constraints. Here, the data term refers to the value of the quantity
of interest at initial and boundary points. Minimizing the data term amounts
to measuring the discrepancy between the predicted solution of the PINN and
the measured data points. The physics term incorporates the PDE constraints
into the loss function, evaluating the differential operator of the PDE using
automatic differentiation ([43]). The resulting equation is then included as a
penalty term in the loss function. To elucidate these terms, we consider an
abstract PDE as,

$D(u(x,t,k))=f(x,t),\quad(x,t)\in\mathcal{D}\times\mathcal{T}\\\ $ (1)

where $D$ is the differential operator, $\mathcal{D}$ is the spatial domain,
and $\mathcal{T}$ is the temporal domain. The unknown solution is $u$
depending on independent space ($x$) and time ($t$) variables. A constant
parameter is $k$, and $f$ is the source term. To ensure the uniqueness of the
solution, appropriate initial and boundary conditions are necessary for the
considered PDE.

$\begin{gathered}u(x,0)=g(x),\quad(x,0)\in\mathcal{D}\times\Gamma\\\
u(x_{\mathrm{b}},t)=\bar{g}(x_{\mathrm{b}},t),\quad(x_{\mathrm{b}},t)\in\Omega\times\mathcal{T}\end{gathered}$
(2)

here, $g(x)$ and $\bar{g}(x_{\mathrm{b}},t)$ are the initial and boundary
conditions, respectively. The initial temporal region and spatial boundary are
$\Gamma$ and $\Omega$, respectively. The loss function of PINNs is defined as
follows

$L(\mu)=\lambda_{1}L_{\mathrm{PDE}}(\mu)+\lambda_{2}L_{\mathrm{IC}}(\mu)+\lambda_{3}L_{\mathrm{BC}}(\mu)$
(3)

here, $\mu$ represents the trainable network parameters. The individual loss
terms weighted by the hyperparameters $\lambda_{i}$, $i=1,2,3$, are defined
as,

$\begin{gathered}L_{\mathrm{PDE}}(\mu)=\frac{1}{N_{\mathrm{int}}}\sum_{n=1}^{N_{\mathrm{int}}}||D(u^{*}(x^{(n)},t^{(n)},k))-f(x^{(n)},t^{(n)})||^{p}\end{gathered}$
(4)

The loss terms for initial and boundary conditions in (3) are defined as
follows,

$\begin{gathered}L_{\mathrm{IC}}(\mu)=\frac{1}{N_{\mathrm{i}}}\sum_{n=1}^{N_{\mathrm{i}}}||u^{*}(x^{(n)},0)-g(x^{(n)})||^{p}\\\
L_{\mathrm{BC}}(\mu)=\frac{1}{N_{\mathrm{b}}}\sum_{n=1}^{N_{\mathrm{b}}}||u^{*}(x^{(n)}_{\mathrm{b}},t^{(n)})-\bar{g}(x_{\mathrm{b}}^{(n)},t^{(n)})||^{p}\end{gathered}$
(5)

here, $N$ is the total number of training points, which is the sum of interior
training points ($N_{\mathrm{int}}$), initial training points
($N_{\mathrm{i}}$), and boundary training points $N_{\mathrm{b}}$. The
approximation of $u$ by the neural network is denoted by $u^{*}$. Training
with $L^{2}$-norm amounts to $p$ $=$ $2$. The primary objective is minimizing
(3) and obtaining optimal parameters ($\mu$). These optimized parameters are
then utilized for predicting the PDE solution
$u(x,t),\forall(x,t)\in\mathcal{D}\times\mathcal{T}$.

###  3.2 Causal PINN

This subsection presents causal PINN, modifying the PINN loss function ([15]).
The notion of causal PINNs is inspired by traditional numerical methods for
solving differential equations that prioritize resolving the solution at lower
times before approximating the solution at higher times. The modification in
the loss function pertains to the PDE term $L_{\mathrm{PDE}}(\mu)$, while the
initial $L_{\mathrm{IC}}(\mu)$ and boundary $L_{\mathrm{BC}}(\mu)$ loss terms
remain unchanged. The causal PDE loss term $L_{\mathrm{PDE}}(\mu)$ is defined
as

$\begin{gathered}L_{\mathrm{PDE}}(\mu)=\sum_{i=1}^{N_{\mathrm{t}}}w_{i}L_{\mathrm{PDE}}(t_{i},\mu)\\\
w_{i}=e^{-\epsilon\sum_{k=1}^{i-1}L_{\mathrm{PDE}}(t_{k},\mu)},\quad
i=2,3,\ldots N_{\mathrm{t}}\end{gathered}$ (6)

Here, $N_{\mathrm{t}}$ is the number of timesteps in which the computational
domain has been divided. The causality hyperparameter $\epsilon$ controls the
steepness of the weights. The modification introduces a weighting factor,
$w_{i}$, for loss at each time level $t_{i}$. The weight $w_{i}$ depends on
the accumulated PDE loss up to time $t_{i}$. The weights are adjusted to
prioritize the fully resolved solution at lower time levels by exponentiating
the negative of this accumulated loss. To summarize, the modified loss
function for causal PINN could be written as

$L_{\mathrm{PDE}}(\mu)=\frac{1}{N_{t}}\sum_{i=1}^{N_{t}}e^{-\epsilon\sum_{k=1}^{i-1}L_{\mathrm{PDE}}(t_{k},\mu)}L_{\mathrm{PDE}}(t_{i},\mu)$
(7)

In the following section, the proposed transfer learning method is presented
along with the underlying motivation.

##  4 Transfer learning for causal PINN

Figure 2: Proposed Transfer Learning Framework in Causal PINN: The top
horizontal block outlines the training process of the causal PINN for the
parent model, which is the primary beam model under consideration. The model
parameters, initialized using Xavier initialization within the first neural
network, undergo training while adhering to causality. The resulting
parameters of the trained model serve as the initialization for $J$ subsequent
tasks shown by the bottom vertical blocks ($1\leq j\leq J$). These tasks
pertain to different initial conditions and extensions of both spatial and
temporal domains. The training of these subtasks is also performed to adhere
to causality.

Several factors are crucial for designing an engineering structure, and
solving the problem for each case is important. However, training the neural
network for every case is time-consuming and laborious. Here, we propose to
utilize transfer learning for beam problems on the Winkler foundation. The
idea is to train the parent beam model for one case, for instance, to train an
Euler-Bernoulli beam for a specific initial condition and then utilize the
parameters for different initial conditions. The aim is to reduce the training
time for the transfer learning case compared to the case without transfer
learning. This reduction in computational time in terms of epochs is done by
utilizing the previously trained model parameters and using them as
initialization for subsequent cases.

The proposed approach incorporates transfer learning for different scenarios
for the same physical beam equation. Fig. 2 visually demonstrates the steps:
initially, the parent model is trained using causal PINN for a significant
number of epochs ($n_{1}$). Subsequently, the trained parameters are utilized
as an initialization for the training of other problems of the physical
equation with different initial conditions or for an extended domain, which is
trained for a reduced number of epochs ($n_{2}$), where $n_{2}$ $<<$ $n_{1}$,
reducing the computational cost of training the model again from the start.
The step-by-step illustration is provided in Fig. 2.

In Fig. 2, the top horizontal block illustrates the training of causal PINN
for the parent model, specifically the primary beam model, either the Euler-
Bernoulli or the Timoshenko beam model. The model parameters, generated using
Xavier initialization ([44]) for the initial neural network, undergo a
training process adhering to the causal loss function. This training involves
the resolution of solutions at lower times prior to approximating at higher
times, as shown by the snapshots of the resolved solution. As the number of
epochs increases, the model prediction at higher time levels improves only
when the solution at lower time levels has been resolved up to a certain
accuracy. The resulting parameters from this training serve as the
initialization for subsequent $j$ tasks presented by the bottom vertical
blocks in Fig. 2. These tasks involve diverse initial conditions and
extensions of both spatial and temporal domains. Notably, the training of
these subtasks is also performed by minimizing the loss terms (7) and (5) in
the loss function (3), ensuring a coherent and principled transfer learning
framework.

(a)  (b)  (c)  (d)

Figure 3: Euler-Bernoulli beam displacement on the Winkler foundation (a.)
Predicted solution using PINN (b.) Predicted solution using SA-PINN (c.)
Predicted solution using causal PINN (d.) Reference solution

In the next section, we perform a series of experiments to demonstrate the
efficacy of the proposed framework.

##  5 Numerical experiments

This section presents the numerical experiments for simulating the dynamics of
the Euler-Bernoulli and Timoshenko beam models using the proposed framework.
The proposed method is compared with vanilla PINN and self-adaptive PINN (SA-
PINN) proposed by [25] and utilized in several works, for instance, [45] to
improve the performance of PINNs. In addition, leveraging transfer learning,
several other experiments are performed for noisy data, different initial
conditions, and extrapolation in both spatial and temporal domains for the
beam models.

The experimental setup involves first simulating the parent case and utilizing
the trained parameters for various subcases. Specifically, transfer learning
is employed for these subcases. The main model utilizes a neural network
architecture comprising four hidden layers with 200 neurons each. The
activation function employed is the hyperbolic tangent (tanh), and the
limited-memory Broyden-Fletcher-Goldfarb-Shanno (LBFGS) optimizer is utilized
with a learning rate of $0.1$. The parent model is trained for a total of
$10,000$ epochs. Within the causal-respecting PINN function, the causality
hyperparameter ($\epsilon$) is set to $5$ and the number of timesteps $N_{t}$
is taken to be 100. During the training process, $N_{\mathrm{i}}=500$ initial
points, $N_{\mathrm{b}}=1000$ boundary points, and $N_{\mathrm{int}}=10,000$
interior points are considered. The weight hyperparameters $\lambda_{1}$,
$\lambda_{2}$ and $\lambda_{3}$ are taken to be $1$ each. The selected
evaluation metric is the $L^{2}$ relative error percentage ($\mathcal{R}$)
defined as

$\displaystyle\mathcal{R}=\frac{||u^{*}-u||_{2}}{||u||_{2}}\times 100$ (8)

where $u^{*}$ is the approximated PDE solution by the neural network, and $u$
refers to the ground truth. We utilize the trained parameters $(\mu)$ of the
main model as initialization for training the subcase neural networks for only
$1500$ epochs, achieving the same level of accuracy as the main model.

Figure 4: Euler-Bernoulli beam equation on the Winkler foundation with noise
in the initial condition of displacement of the beam Left: Predicted solution
at final time ($t=1$) with $10\%$ Gaussian noise; Right: Predicted solution at
final time ($t=1$) with $20\%$ Gaussian noise

###  5.1 Euler-Bernoulli beam

The Euler-Bernoulli beam model is a mathematical framework used to analyze the
behavior of beams when subjected to loads. It is derived from the three-
dimensional elasticity theory or through principles such as Newton’s second
law or the generalized Hamiltonian Principle ([46]). The model assumes certain
simplifications: it neglects the effects of rotary inertia and transverse
shear deformations. The Euler-Bernoulli beam equation describes the behavior
of a beam subjected to bending (Fig. 1). When the beam is supported on a
Winkler foundation, representing an elastic foundation, the Euler-Bernoulli
beam equation is modified to account for the interaction between the beam and
the foundation. This modified equation considers the stiffness of the
foundation and its influence on the behavior of the beam. The mathematical
model of a simply supported Euler-Bernoulli beam on a Winkler foundation is
described by ([24])

$u_{\mathrm{tt}}+u_{\mathrm{xxxx}}+p(x,t)=f(x,t),\quad x\in[0,8\pi],t\in[0,1]$
(9)

where $u$ represents the vertical displacement of the beam. $u_{\mathrm{tt}}$,
and $u_{\mathrm{xxxx}}$ represent the two times partial derivative of $u$ with
respect to $t$, and four times partial derivative with respect to $x$,
respectively. The loading on the beam is defined by
$f(x,t)=(2-\pi^{2})\sin{(x)}\cos(\pi t)$. The initial and boundary conditions
are given as

$\begin{gathered}u(x,0)=\sin(x),\quad u_{\mathrm{t}}(x,0)=0\\\
u(0,t)=u(8\pi,t)=u_{\mathrm{xx}}(0,t)=u_{\mathrm{xx}}(8\pi,t)=0\end{gathered}$
(10)

The foundation reaction, $p(x,t)$, assumes that the reaction at every location
is proportional to the displacement at a particular location, and the springs
are linear and independent, as described in (9). The reaction force of the
foundation is given by $p(x,t)=ku(x,t)$, where $u(x,t)$ is vertical
displacement and $k$ is the stiffness of linear springs. The exact solution
for this problem is given by $u(x,t)=\sin(x)\cos(\pi t)$.

Solving (9), one can determine the vertical displacement of the beam at any
point along its length and other important quantities of interest, such as
bending moments and beam acceleration. These quantities help engineers
understand how the beam will perform structurally and ensure it meets the
desired design criteria. By calculating the displacement, engineers can check
whether the beam deflects within acceptable limits under the applied loads.

Table 1: Euler-Bernoulli beam: $\mathcal{R}$ at $t=1$ for $k=1$ | PINN | SA-PINN | Causal PINN
---|---|---|---
$\mathcal{R}$ | 5.33732 | 5.15410 | 0.03618

We simulate (9) with three different methods to establish that incorporating
causality provides more accuracy in the predicted solution than vanilla PINN
and the SA-PINN for our problem. The results presented in Table 1 indicate
that vanilla PINN and SA-PINN provide less accurate displacement predictions
at $t=1$ for the Euler-Bernoulli equation for stiffness $k=1$. In contrast,
causal PINN yields more accurate displacement predictions as the relative
percent error is $0.036$. This observation is further supported by the
findings depicted in Fig. 3 (a), (b), which demonstrates that PINN and SA-PINN
models are not accurate, particularly during the initial time, highlighted by
the white rectangular box in Fig. 3. However, this challenge is effectively
overcome by incorporating a causality-respecting loss function (Fig. 3(c)),
which facilitates training the solution at lower time levels before training
at higher times.

The parameters from PINN and SA-PINN are not used subsequently to avoid
incomplete or bad knowledge transfer. Only the trained parameters from the
causal PINN formulation are transferred to the subsequent experiments
presented in the next two subsections, fostering convergence by effectively
reducing the training epochs.

####  5.1.1 Noisy initial conditions

This subsection presents the performance of the proposed method with noisy
initial conditions. Initial conditions may not be perfectly known in real-
world scenarios or contain uncertainties or noise. By learning displacements
for noisy initial conditions, we can develop models that accurately represent
the system’s behaviour under such realistic conditions, allowing us to account
for uncertainties and better understand the actual response of the system. To
observe the dynamics of beam models under these conditions, we introduce
Gaussian noise in the initial condition ranging from $5{\%}$ to $20{\%}$. The
hyperparameter selection is the same as the main model, except for the number
of epochs. With transfer learning, we perform $1500$ epochs instead of
$10000$.

Table 2 presents the results from $5{\%}$ to $20{\%}$ Gaussian noise levels in
the initial conditions for the displacement of the beam with and without (w/o)
using transfer learning. The proposed method predicts $u$ with less relative
error percent. This prediction is significantly more accurate compared to the
case without transfer learning. Also, Fig. 4 shows the results for $10{\%}$
and $20{\%}$ noise levels in the initial conditions for the displacement of
the beam using transfer learning, demonstrating the computational efficiency
of the proposed method.

Fig. 5 illustrates the comparison of relative error percentages concerning the
noise percentage for both methods, one with transfer learning and the other
without it. In the transfer learning scenario, it becomes apparent that an
increase in the noise percentage results in a corresponding increase in the
relative error percentage. When the subcases use the trained parameters for
initialization, noise and error percentages exhibit a direct proportional
relationship. However, in cases where trained parameters are not utilized, no
discernible pattern emerges due to the non-convergence in minimizing the loss
function.

Figure 5: $\mathcal{R}$ vs noise percentage in the initial condition for the Euler Bernoulli beam for both approaches - with and without transfer learning. Table 2: Euler Bernoulli Beam: $\mathcal{R}$ at $t=1$ for different percentages of noise in the initial condition. "TL" refers to transfer learning, and "w/o" refers to without. Abbreviations are used consistently for the following tables. | 5% | 10% | 12.5% | 15% | 17.5% | 20%
---|---|---|---|---|---|---
with TL | 0.03063 | 0.03198 | 0.04180 | 0.06937 | 0.222182 | 0.23296
w/o TL | 117.7389 | 45.65849 | 59.42882 | 19.7473 | 48.75515 | 29.50691

####  5.1.2 Different initial displacements and velocities

In this section, we present the results of the Euler-Bernoulli beam for
different initial conditions characterized by the change in initial
displacements and velocities of the beam. Learning deflections for different
initial conditions and force functions allows for generalization. Beams or
structures can have varying initial conditions, such as different magnitudes,
positions, or load distributions. By learning the deflections for a diverse
set of initial conditions, we can develop models that capture the underlying
patterns and behavior of the system, enabling accurate predictions for unseen
or novel initial conditions.

Figure 6: Euler-Bernoulli beam on the Winkler foundation for initial velocity
for case 1: Causal PINN prediction at final time $t=1$ with and without
transfer learning

Here, we consider different initial conditions compared to the parent model.
The initial conditions for this case are $u(x,0)=a\sin(x)$ and
$u_{t}(x,t=0)=a\sin(x)$. The analytical solution for the corresponding problem
is $u(x,t)=a\sin(x)e^{t}$. We utilize the trained parameters of the Euler-
Bernoulli beam model as an initialization for training this problem with
different initial conditions considering $a=1,2$ (representing case $1$ and
case $2$ in Table 3). The hyperparameters remain unchanged; the only change is
the number of epochs, which is only 3000. Relative error percentages of
displacement are presented in Table 3, which shows a large difference in
relative percent errors. From Fig. 6, it is evident for the first case that
the transfer learning approach achieves accurate predictions in a fewer number
of epochs.

Table 3: Euler Bernoulli Beam: $\mathcal{R}$ at $t=1$ for different velocities $u^{*}$ |  $\mathcal{R}$ (case 1) |  $\mathcal{R}$ (case 2)
---|---|---
with TL | 0.00105 | 0.02188
w/o TL | 70.72229 | 193.85024

###  5.2 Timoshenko beam

The Timoshenko beam theory considers the shear deformation and rotational
effects neglected in the Euler-Bernoulli beam equation ([46]). Hence, in
addition to the quantity vertical displacement ($u$), Timoshenko’s theory
considers the cross-sectional rotation ($\theta$) as another unknown variable.
The mathematical model for a beam resting on a Winkler foundation and
subjected to an external load based on the Timoshenko beam theory is given as
follows ([24])

$\displaystyle\theta_{\mathrm{tt}}-\theta_{\mathrm{xx}}+(\theta-
u_{\mathrm{x}})=0;$ (11) $\displaystyle u_{\mathrm{tt}}+(\theta-
u_{\mathrm{x}})_{\mathrm{x}}+ku=h(x,t)$

where the symbols have their usual meaning, as in the case of the Euler-
Bernoulli beam model. We consider $h(x,t)$ = $\cos(t)$ and the computational
domain to be $x\in[0,3\pi]$ and $t\in[0,1]$. The supporting initial and
boundary conditions are given as

(a)  (b)  (c)  (d)  (e)  (f)  (g)  (h)

Figure 7: Timoshenko beam on the Winkler foundation Top: Predicted
Displacement ($u^{*}$) (a.) Using PINN (b.) Using SA-PINN (c.) Using causal
PINN (d.) Reference solution Bottom: Predicted Rotation ($\theta^{*}$) (e.)
Using PINN (f.) Using SA-PINN (g.) Using causal PINN (h.) Reference solution

Figure 8: Timoshenko beam on the Winkler foundation at final time $t=1$. Left:
Absolute error in predicting displacement ($|u-u^{*}|$); Right: Absolute error
in predicting rotation ($|\theta-\theta^{*}|$)

$\begin{gathered}\theta(x,0)=\frac{3\pi}{2}\cos(x)+\left(x-\frac{3\pi}{2}\right),\quad\theta_{t}(x,0)=0\\\
u(x,0)=\frac{3\pi}{2}\sin(x),\quad u_{t}(x,0)=0\\\
\theta(0,t)=\theta(3\pi,t)=u(0,t)=u(3\pi,t)=0\end{gathered}$ (12)

The analytic solution for the rotation and vertical displacement is given as
follows

$\begin{gathered}\theta(x,t)=\left(\frac{3\pi}{2}\cos(x)+\left(x-\frac{3\pi}{2}\right)\right)\cos(t)\\\
u(x,t)=\frac{3\pi}{2}\sin(x)\cos(t)\end{gathered}$ (13)

Solving the Timoshenko beam model (11) - (12) would help engineers obtain more
accurate predictions of beam deflections and rotations, especially for beams
with high aspect ratios or subjected to high shear forces. This accuracy is
crucial for assessing structural integrity, ensuring compliance with design
criteria, and preventing potential failures.

Fig. 7 illustrates the predicted displacement and rotation throughout the
entire space-time domain. Fig. 7(c) and Fig. 7(g) depict the displacement and
rotation prediction using the causal PINN loss function. Fig. 7(a-b) and Fig.
7(e-f) depict the displacement and rotation prediction using vanilla PINN and
SA-PINNs, respectively, illustrating its failure in prediction. In addition,
Fig. 8 presents the absolute error in displacement and rotation resulting from
the causal PINN loss function. The maximum error magnitude falls below
$10^{-2}$, clearly indicating the accuracy of causal PINN.

Table 4: Timoshenko Beam: $\mathcal{R}$ at $t=1$ for $k=1$ | PINN | SA-PINN | Causal PINN
---|---|---|---
$u^{*}$ | 240.05908 | 137.150752 | $1.20048\times 10^{-6}$
$\theta^{*}$ | 9.18397 | 6.56500 | $7.70788\times 10^{-6}$

Table 4 presents the relative percentage error in predicting displacement and
rotation for vanilla PINN, SA-PINN, and causal PINN. In the case of causal
PINN, both quantities of interest, $u$, and $\theta$ exhibit errors in the
magnitude of $10^{-6}$, demonstrating its accuracy. Conversely, vanilla PINN
fails to adequately approximate the quantities of interest, as evidenced by a
relative error percent of over $200{\%}$ for displacement and an error of
approximately $9{\%}$ for rotation. Also, for the case of SA-PINN, the
relative error percent for displacement is over $100{\%}$ and around $6{\%}$
for rotation. The results show that Causal PINN accurately predicts
displacement and rotation for the Timoshenko beam model.

###  5.3 Large space-time horizon

In the following two experiments, we show the potential of transfer learning
and predict the displacement and cross-sectional rotation in a larger domain.
We utilize transfer learning for extrapolating. There are several benefits to
knowing deflections on larger domains. Firstly, it provides a better
understanding of the structural behavior of the beam under different loading
conditions. By analyzing the deflection over larger lengths, engineers can
assess the beam’s overall stability and structural integrity, which is crucial
for designing safe and reliable structures.

Secondly, calculating the deflection for extended domains allows for more
accurate predictions of the behaviour of the beam in real-world scenarios.
This information is valuable in various engineering applications such as
building design, bridge construction, and aerospace engineering, where
accurate deflection predictions are essential for ensuring the structural
performance and safety of the final product.

Also, studying the deflection of the beam over a larger domain can help
identify potential areas of weakness or excessive deformation. This knowledge
enables engineers to make informed decisions about reinforcing certain
sections or implementing design modifications to improve the overall
performance and durability of the structure.

Furthermore, studying larger domains can optimize material usage and cost-
effectiveness in construction projects. By accurately predicting deflection,
engineers can optimize the size, shape, and materials used to construct beams,
leading to more efficient designs and reduced material waste.

Figure 9: Timoshenko beam on the Winkler foundation: Top Prediction for
extended domain in space, $x\in[0,7\pi]$ Left: Displacement ($u^{*}$); Right:
Rotation ($\theta^{*}$). Bottom Prediction for extended domain in time for
$t=7$ Left: Displacement ($u^{*}$); Right: Rotation ($\theta^{*}$).

####  5.3.1 Extended spatial domain

In this section, we consider the Timoshenko beam model in an extended domain
in space. The spatial domain for the parent model is $x\in[0,3\pi]$. Here, we
utilize the parameters of the parent model and train the subsequent models for
different spatial domains, in particular $x\in[0,5\pi]$, $x\in[0,6\pi]$, and
$x\in[0,7\pi]$. The aim is to observe the method’s potential in a larger
domain, indicating that the method generalizes well. The results obtained with
and without transfer learning are presented in Table 5, highlighting the
superior accuracy achieved by the proposed method when utilizing parameters
from the main model compared to training the model with Xavier initialization.
Fig. 9 top row presents the proposed method’s predictions of displacement and
rotation, indicating that the model generalizes well across the spatial
domain, inheriting the underlying structure and symmetry of the solution.

Table 5: Timoshenko Beam: $\mathcal{R}$ for extension in the spatial domain with 3000 epochs | With TL | w/o TL
---|---|---
$x$ | $u^{*}$ | $\theta^{*}$ | $u^{*}$ | $\theta^{*}$
$[0,5\pi]$ | $6.6\times 10^{-5}$ | 0.00011 | 2.34306 | 3.51362
$[0,6\pi]$ | 0.00653 | 0.00097 | 21.81964 | 30.67853
$[0,7\pi]$ | 1.52043 | 0.61573 | 11.00256 | 8.90537

####  5.3.2 Extended temporal domain

We now extend our investigation to the temporal domain based on successfully
generalizing the proposed method in the spatial domain. By employing the
trained parameters obtained from the parent model, we train the same model
with an extension in time, considering different temporal domains,
$t\in[0,4]$, $t\in[0,6]$, and $t\in[0,7]$. The relative error percentage for
all cases of the extended temporal domains is presented in Table 6. We observe
that the proposed method accurately predicts displacement and rotation, while
the approach without transfer learning fails to provide the same level of
accuracy. Fig. 9 bottom row shows the predictions obtained by the proposed
method for displacement and rotation in an extended temporal domain. The
results show that utilizing transfer learning for extended domains in space
and time provides accurate results, conserving the structure and symmetry of
the solution.

##  6 Conclusions

This paper introduced a methodology for simulating the dynamics of beam models
based on Euler-Bernoulli and Timoshenko’s theories on the Winkler foundation.
By incorporating transfer learning within a causality-respecting PINN
framework, we addressed the need for re-training the network when there are
modifications to the initial conditions or computational domain.

Numerical experiments demonstrated the effectiveness of the proposed approach.
For the Euler-Bernoulli beam, we utilized the trained parameters from the
parent model to simulate sub-cases with different initial conditions,
including noisy ones. For the Timoshenko beam, we investigated its behavior in
an extended spatial and temporal domain. These experiments showcased the
generalization potential of the proposed method.

Table 6: Timoshenko Beam: $\mathcal{R}$ for extension in the temporal domain with 3000 epochs | With TL | w/o TL
---|---|---
$t$ | $u^{*}$ | $\theta^{*}$ | $u^{*}$ | $\theta^{*}$
$[0,4]$ | 9.7e-6 | 2.4e-5 | 7.9e-5 | 0.00026
$[0,6]$ | 0.00111 | 0.00085 | 0.01627 | 0.12266
$[0,7]$ | 0.89122 | 0.05554 | 4.92954 | 2.50340

We also performed comparisons of the proposed method with SA-PINNs and vanilla
PINNs. Results show that the causality-respecting PINN with transfer learning
reduces computational costs and improves convergence. The results indicate
that the method struggled to approximate the solutions accurately without
transfer learning.

Overall, our findings highlight the efficacy of the proposed methodology in
simulating beam dynamics under diverse engineering scenarios. By leveraging
transfer learning and a causality-respecting PINN framework, we can reduce
training requirements while achieving accurate results for various cases. This
research opens up new possibilities for efficiently predicting the dynamics of
structural elements, leading to advancements in structural engineering design,
optimization, and control.

Future research directions involve extending the methodology to other
structural elements like systems of beams, strings and plates. An alternative
research trajectory may involve training a family of PDE models and applying
meta-learning techniques to derive a universal set of parameters applicable
across diverse models. This unified parameter set could potentially be
employed to test novel models, contributing to a generalized and efficient
approach in the field. The codes will be made available upon publication.

## CRediT authorship contribution statement

Taniya Kapoor: Conceptualization, Methodology, Software, Validation, Formal
analysis, Investigation, Writing - Original Draft, Writing - Review & Editing,
Visualization. Hongrui Wang: Writing - Review & Editing, Supervision, Project
administration. Alfredo Núñez: Writing - Review & Editing, Supervision,
Project administration. Rolf Dollevoet: Resources, Funding acquisition,
Supervision, Project administration.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Acknowledgment

The authors would like to acknowledge the support and computational resources
provided by the DelftBlue Cluster contributing to this research.

## References

  * [1] Angie C Lamprea-Pineda, David P Connolly, and Mohammed FM Hussein.  Beams on elastic foundations–a review of railway applications and solutions.  Transportation Geotechnics, 33:100696, 2022.
  * [2] Jian Deng, Mohammadmehdi Shahroudi, and Kefu Liu.  Dynamic stability and responses of beams on elastic foundations under a parametric load.  International Journal of Structural Stability and Dynamics, 23(02):2350018, 2023.
  * [3] Edward Tsudik.  Analysis of structures on elastic foundations.  J. Ross Publishing, 2012.
  * [4] Miklós Hetényi and Miklbos Imre Hetbenyi.  Beams on elastic foundation: theory with applications in the fields of civil and mechanical engineering, volume 16.  University of Michigan press Ann Arbor, MI, 1946.
  * [5] Levon G Petrosian.  Analysis of Structures on Elastic Foundation: Incorporating the Spectral Method of Boundary Elements.  CRC Press, 2022.
  * [6] P. B. Chang, B. J. Williams, T. J. Santner, W. I. Notz, and D. L. Bartel.  Robust Optimization of Total Joint Replacements Incorporating Environmental Variables.  Journal of Biomechanical Engineering, 121(3):304–310, 06 1999.
  * [7] Erdogan Madenci and Ibrahim Guven.  The finite element method and applications in engineering using ANSYS®.  Springer, 2015.
  * [8] George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang.  Physics-informed machine learning.  Nature Reviews Physics, 3(6):422–440, 2021.
  * [9] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al.  Highly accurate protein structure prediction with alphafold.  Nature, 596(7873):583–589, 2021.
  * [10] Giuseppe Carleo, Ignacio Cirac, Kyle Cranmer, Laurent Daudet, Maria Schuld, Naftali Tishby, Leslie Vogt-Maranto, and Lenka Zdeborová.  Machine learning and the physical sciences.  Reviews of Modern Physics, 91(4):045002, 2019.
  * [11] Maziar Raissi, Paris Perdikaris, and George E Karniadakis.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  Journal of Computational physics, 378:686–707, 2019.
  * [12] Anuj Karpatne, William Watkins, Jordan Read, and Vipin Kumar.  Physics-guided neural networks (PGNN): An application in lake temperature modeling.  arXiv preprint arXiv:1710.11431, 2, 2017.
  * [13] Anuj Karpatne, Gowtham Atluri, James H Faghmous, Michael Steinbach, Arindam Banerjee, Auroop Ganguly, Shashi Shekhar, Nagiza Samatova, and Vipin Kumar.  Theory-guided data science: A new paradigm for scientific discovery from data.  IEEE Transactions on knowledge and data engineering, 29(10):2318–2331, 2017.
  * [14] Anuj Karpatne, Ramakrishnan Kannan, and Vipin Kumar.  Knowledge Guided Machine Learning: Accelerating Discovery Using Scientific Knowledge and Data.  CRC Press, 2022.
  * [15] Sifan Wang, Shyam Sankaran, and Paris Perdikaris.  Respecting causality is all you need for training physics-informed neural networks.  arXiv preprint arXiv:2203.07404, 2022.
  * [16] Olga Fuks and Hamdi A Tchelepi.  Limitations of physics informed machine learning for nonlinear two-phase transport in porous media.  Journal of Machine Learning for Modeling and Computing, 1(1), 2020.
  * [17] Jungeun Kim, Kookjin Lee, Dongeun Lee, Sheo Yon Jhin, and Noseong Park.  Dpm: a novel training method for physics-informed neural networks in extrapolation.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8146–8154, 2021.
  * [18] Taniya Kapoor, Abhishek Chandra, Daniel M Tartakovsky, Hongrui Wang, Alfredo Nunez, and Rolf Dollevoet.  Neural oscillators for generalization of physics-informed machine learning.  arXiv preprint arXiv:2308.08989, 2023.
  * [19] Taniya Kapoor, Abhishek Chandra, Daniel Tartakovsky, Hongrui Wang, Alfredo Núñez, and Rolf Dollevoet.  Neural oscillators for generalizing parametric PDEs.  In The Symbiosis of Deep Learning and Differential Equations III, 2023.
  * [20] Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney.  Characterizing possible failure modes in physics-informed neural networks.  Advances in Neural Information Processing Systems, 34:26548–26560, 2021.
  * [21] Aleksandr Dekhovich, Marcel HF Sluiter, David MJ Tax, and Miguel A Bessa.  iPINNs: Incremental learning for physics-informed neural networks.  arXiv preprint arXiv:2304.04854, 2023.
  * [22] Tim De Ryck, Florent Bonnet, Siddhartha Mishra, and Emmanuel de Bézenac.  An operator preconditioning perspective on training in physics-informed machine learning.  arXiv preprint arXiv:2310.05801, 2023.
  * [23] Shuteng Niu, Yongxin Liu, Jian Wang, and Houbing Song.  A decade survey of transfer learning (2010–2020).  IEEE Transactions on Artificial Intelligence, 1(2):151–166, 2020.
  * [24] Davood Younesian, Ali Hosseinkhani, Hassan Askari, and Ebrahim Esmailzadeh.  Elastic and viscoelastic foundations: a review on linear and nonlinear vibration modeling and applications.  Nonlinear Dynamics, 97(1):853–895, 2019.
  * [25] Levi D McClenny and Ulisses M Braga-Neto.  Self-adaptive physics-informed neural networks.  Journal of Computational Physics, 474:111722, 2023.
  * [26] Somdatta Goswami, Cosmin Anitescu, Souvik Chakraborty, and Timon Rabczuk.  Transfer learning enhanced physics informed neural network for phase-field modeling of fracture.  Theoretical and Applied Fracture Mechanics, 106:102447, 2020.
  * [27] Shilin Li, Gang Wang, Yuelan Di, Liping Wang, Haidou Wang, and Qingjun Zhou.  A physics-informed neural network framework to predict 3D temperature field without labeled data in process of laser metal deposition.  Engineering Applications of Artificial Intelligence, 120:105908, 2023.
  * [28] Xu Liu, Wei Peng, Zhiqiang Gong, Weien Zhou, and Wen Yao.  Temperature field inversion of heat-source systems via physics-informed neural networks.  Engineering Applications of Artificial Intelligence, 113:104902, 2022.
  * [29] Arunabha M Roy and Suman Guha.  A data-driven physics-constrained deep learning computational framework for solving von mises plasticity.  Engineering Applications of Artificial Intelligence, 122:106049, 2023.
  * [30] Ehsan Haghighat, Sahar Abouali, and Reza Vaziri.  Constitutive model characterization and discovery using physics-informed deep learning.  Engineering Applications of Artificial Intelligence, 120:105828, 2023.
  * [31] Bin Mu, Bo Qin, Shijin Yuan, Xin Wang, and Yuxuan Chen.  PIRT: A physics-informed red tide deep learning forecast model considering causal-inferred predictors selection.  IEEE Geoscience and Remote Sensing Letters, 20:1–5, 2023.
  * [32] Michael Penwarden, Ameya D Jagtap, Shandian Zhe, George Em Karniadakis, and Robert M Kirby.  A unified scalable framework for causal sweeping strategies for physics-informed neural networks (pinns) and their temporal decompositions.  arXiv preprint arXiv:2302.14227, 2023.
  * [33] Ameya D Jagtap and George E Karniadakis.  Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations.  In AAAI spring symposium: MLPS, volume 10, 2021.
  * [34] Maziyar Bazmara, Mohammad Silani, Mohammad Mianroodi, et al.  Physics-informed neural networks for nonlinear bending of 3D functionally graded beam.  In Structures, volume 49, pages 152–162. Elsevier, 2023.
  * [35] Taniya Kapoor, Hongrui Wang, Alfredo Núñez, and Rolf Dollevoet.  Physics-informed neural networks for solving forward and inverse problems in complex beam systems.  IEEE Transactions on Neural Networks and Learning Systems, pages 1–15, 2023.
  * [36] Taniya Kapoor, Hongrui Wang, Alfredo Núñez, and Rolf Dollevoet.  Physics-informed machine learning for moving load problems.  arXiv preprint arXiv:2304.00369, 2023.
  * [37] Jeongsu Lee.  Physics informed neural networks for extreme mechanics problems.  Available at SSRN 4362563.
  * [38] Chen Xu, Ba Trung Cao, Yong Yuan, and Günther Meschke.  Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios.  Computer Methods in Applied Mechanics and Engineering, 405:115852, 2023.
  * [39] Ali Fallah and Mohammad Mohammadi Aghdam.  Physics-informed neural network for bending and free vibration analysis of three-dimensional functionally graded porous beam resting on elastic foundation.  Engineering with Computers, pages 1–18, 2023.
  * [40] Qijing Chen, Qi Ye, Weiqi Zhang, He Li, and Xiaoning Zheng.  Tgm-nets: A deep learning framework for enhanced forecasting of tumor growth by integrating imaging and modeling.  Engineering Applications of Artificial Intelligence, 126:106867, 2023.
  * [41] Navid Zobeiry and Keith D Humfeld.  A physics-informed machine learning approach for solving heat transfer equation in advanced manufacturing and engineering applications.  Engineering Applications of Artificial Intelligence, 101:104232, 2021.
  * [42] Sheng Shen, Hao Lu, Mohammadkazem Sadoughi, Chao Hu, Venkat Nemani, Adam Thelen, Keith Webster, Matthew Darr, Jeff Sidon, and Shawn Kenny.  A physics-informed deep learning approach for bearing fault detection.  Engineering Applications of Artificial Intelligence, 103:104295, 2021.
  * [43] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer.  Automatic differentiation in pytorch.  2017\.
  * [44] Xavier Glorot and Yoshua Bengio.  Understanding the difficulty of training deep feedforward neural networks.  In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
  * [45] Yi Ding, Su Chen, Xiaojun Li, Suyang Wang, Shaokai Luan, and Hao Sun.  Self-adaptive physics-driven deep learning for seismic wave modeling in complex topography.  Engineering Applications of Artificial Intelligence, 123:106425, 2023.
  * [46] Andreas Öchsner.  Classical beam theories of structural mechanics, volume 42.  Springer, 2021.