# AU-Guided Unsupervised Domain Adaptive Facial Expression Recognition

Kai Wang∗ Alibaba DAMO Acadmey  Yuxin Gu Equally-contributed first authors
Alibaba DAMO Acadmey  Xiaojiang Peng∗ Shenzhen Technology University, China
Panpan Zhang  Fudan University, China  Baigui Sun  Alibaba DAMO Acadmey  Hao
Li Corresponding author<EMAIL_ADDRESS>Alibaba DAMO Acadmey

###### Abstract

The domain diversities including inconsistent annotation and varied image
collection conditions inevitably exist among different facial expression
recognition (FER) datasets, which pose an evident challenge for adapting the
FER model trained on one dataset to another one. Recent works mainly focus on
domain-invariant deep feature learning with adversarial learning mechanism,
ignoring the sibling facial action unit (AU) detection task which has obtained
great progress. Considering AUs objectively determine facial expressions, this
paper proposes an AU-guided unsupervised Domain Adaptive FER (AdaFER)
framework to relieve the annotation bias between different FER datasets. In
AdaFER, we first leverage an advanced model for AU detection on both source
and target domain. Then, we compare the AU results to perform AU-guided
annotating, i.e., target faces that own the same AUs with source faces would
inherit the labels from source domain. Meanwhile, to achieve domain-invariant
compact features, we utilize an AU-guided triplet training which randomly
collects anchor-positive-negative triplets on both domains with AUs. We
conduct extensive experiments on several popular benchmarks and show that
AdaFER achieves state-of-the-art results on all these benchmarks.

##  1 Introduction

Facial expression is one of the most important modalities in human emotional
communication. Accurately recognizing facial expressions helps understand
various human emotions and intents, which is applied in wide-range
applications, such as human-computer interaction [6], service robots [12], and
medicine treatment [17]. Both in industry and academy areas, in the past
decades, many well-labeled datasets [27, 38, 51, 10, 3, 32, 11, 23] and high-
performance algorithms [41, 24, 40] have been proposed to automatically
recognize facial expressions in the past decades.

Figure 1: Illustration of unsupervised domain adaptive facial expression
recognition. A model trained on source domain usually fails on target domain
due to subjective inconsistent annotations and different imaging conditions.
AdaFER leverages the objective facial action units for an auxiliary training,
effectively relieve the annotation bias between source and target domains.

In general, deep learning based facial expression recognition (FER) methods
[32, 11, 23, 41, 24, 40] achieve high performance only when the training
domain is identical or similar to the testing domain. However, due to the
diversities of inconsistent annotation and different image collection
condition, there inevitably exists annotation bias (domain gap) among
different datasests, which poses an evident challenge for adapting the FER
model trained on one dataset to another one. As shown in Figure 1, a naive
cross-domain method that deploying the trained model on different target
domain often fails. To this end, many methods have been presented to mitigate
the domain shift in FER [47, 43, 56], while almost all the methods follow
general domain adaption algorithms and ignore the sibling facial action unit
(AU) detection task. Action Units (AUs) represent the movement of the facial
muscles, which have lower bias than subjective facial expression annotations.
Intuitively, AUs can be regarded as auxiliary cues to alleviate the annotation
bias and data bias (domain gap) among different FER datasets.

In this paper, considering the remarkable progress and stable performance of
AU detection [34, 16], we propose an AU-guided unsupervised Domain Adaptive
FER (AdaFER). The AdaFER consists of two crucial modules: AU-Guided Annotating
(AGA) and AU-Guided Triplet Training (AGT). Given two groups of images from
source domain and target domain, we first utilize an advanced pretrained AU
detection model to extract AUs coding on both domains. Then, we perform AU-
guided annotating as following. For each image on target domain, we use its
AUs coding to query the source domain. All these images on source domain with
the same AUs coding as the query image will be used for annotating. By
default, the query image is assigned with a soft label which is the statistic
label distribution of the retrieval images. Further, to achieve structure-
reliable and compact facial expression features, we perform the AU-guided
triplet training. Each triplet is generated by comparing the AUs coding of
source and target images. For example, when a source image is used as an
anchor, we randomly sample a positive(negative) sample that has
same(different) AUs as the anchor from all target images. With AdaFER, we are
able to fine-tune a source-pretrained model on target domain with both pseudo
soft labels and triplet loss, which effectively prevent FER performance
degradation on target domain.

Overall, our contributions can be summarised as follow,

  * •

We heuristically utilize the relationship between action units and facial
expressions for cross-domain facial expression recognition, and propose AU-
guided unsupervised Domain Adaptive FER (AdaFER).

  * •

We elaborately design an AU-Guided Annotating module to assign soft labels for
target domain and an AU-Guided Triplet Training module to learn structure-
reliable and compact facial expression features.

  * •

We conduct extensive experiments on several popular benchmarks and
significantly outperform the state-of-the-art methods.

##  2 Related Work

###  2.1 Facial Expression Recognition

Recently, facial expression recognition has achieved a significant progress
due to the well-designed feature extraction methods and high-performance
algorithms. They first detect and align faces using several popular face
detectors like MTCNN [49] and Dlib [1]. For feature extraction, a large number
of methods focus on modeling the facial geometry and appearance features to
help facial expression recognition. From the feature type view, these features
can be generally divided into hand-craft feature and deep-learning feature.
The hand-craft feature usually contains texture-based features and geometry-
based features [33, 36, 8]. Sometimes they are used in a combination called
hybrid features. For deep-learning feature, Tang [37] and Kahou et al.[18]
utilize deep CNNs for feature extraction, and win the FER2013 and Emotiw2013
challenge, respectively. Zhou et al. [54] achieve a remarkable result in
Emotiw2019 multi-modal emotion recognition challenge by using audio-video deep
fusion method. To address the pose variant and occlusion in FER, Wang et al.
[41] and Li et al. [24] design region-based attention network. Wang et al.
[40] propose Self-Cure Network to suppress uncertainty samples in FER
datasets. Liu et al. [25] introduce a Facial Action Units (FAUs) based network
for expression recognition. Daniel et al. [29] present a cross-platform real-
time multi-face expression recognition toolkit using the (FAUs), which shows
the robustness of FAUs in FER system.

Figure 2: The pipeline of our AdaFER. We first utilize a pretrained AU
detection model to extract facial action units for both source and target
domain images. Then an AU-Guided Annotating (AGA) module assigns a soft/hard
pseudo label for each image in target domain by comparing the AUs between
source and target domain. Meanwhile, we mine triplets among source and target
domains according AUs and apply Triplet loss for training. We jointly train
the FER model on source domain with ground-truths and target domain with
pseudo labels and triplets. AdaFER makes full use of the objective AUs to
bridge the gap of FER datasets caused by subjective inconsistent annotation
and different imaging condition.

###  2.2 Cross-Domain FER

It is inevitable to exist distribution divergences among different facial
expression datasets due to the variant collecting conditions and annotating
subjectiveness. In past decades, cross-domain FER (CD-FER) has got more
attention [5, 22, 30, 35, 45, 46, 47, 52, 55, 56]. Generally, the CD-FER can
be divided into semi-supervised-based, unsupervised-based, and generate-based
methods. Semi-supervised-based methods [20, 45] apply Convolutional Neural
Network (CNN) model to train a classification model using limited labeled
samples from target domain. For unsupervised-based methods, Valstar et al.
[39] use a Gabor-feature based landmarks detector to localize facial and track
these points in facial sequences to model temporal facial activation for
facial expression recognition. They trained the recognition model using the CK
database and performed the test in the MMI database for cross-validation.
Zheng et al.[53] propose a transductive transfer subspace learning method
using labeled source domain images and unlabelled auxiliary target domain
images to jointly learn a discriminative subspace. For generate-based methods,
Zong et al. [57] propose a domain regeneration framework (DR) that aims at
learning a domain regenerator to regenerate samples from source and target
databases, respectively. Wang et al. [42] introduce an unsupervised domain
adaptation method using generative adversarial network (GAN) on the target
dataset and give the unlabelled GAN generated samples distributed pseudo
labels dynamically according to the current prediction probabilities. In order
to understand the conditional probability distributions differences between
source and target datasets, Li et al. [22] develop a deep emotion-conditional
adaption network that simultaneously considers conditional distribution bias
and expression class imbalance problem in CD-FER. Chen et al. [43] propose a
Adversarial Graph Representation Adaptation (AGRA) framework that unifies
graph representation propagation with adversarial learning for cross-domain
holistic-local feature co-adaptation. Different from the above works, our work
utilizes AU information as auxiliary cues to bridge the gap of different FER
datasets, which is expected to learn a generic feature space for source
dataset and target dataset.

##  3 Methodology

In this section, we first overview the AdaFER, and then present its key
modules. We finally present the details of training and inference.

###  3.1 Overview of AdaFER

The goal of our method is to mitigate the domain gap including inconsistent
annotations and different imaging conditions. Considering the relationship
between subjective facial expression and objective Action Units, we propose a
simple yet efficient AU-guided unsupervised Domain Adaptive Facial Expression
Recognition (AdaFER) method. Figure 2 illustrates the pipeline of our AdaFER
which mainly consists of two crucial modules: i) AU-Guided Annotating (AGA)
and ii) AU-Guided Triplet Training (AGT). Given images from source domain and
target domain, we first utilize a pretrained AU detection model to extract the
AUs coding for images from both domains. Then the AGA module assigns a
soft/hard pseudo label for each image in target domain by comparing the AUs
between source and target domain. For example, one of the AGA strategies is to
assign a target image with a hard label that is the same as a source image if
both images have equal AUs. Meanwhile, we mine triplets among source and
target domains according AUs. For example, given an anchor image in target
domain, a positive image in source domain is the one with equal AUs and a
negative image is the one with different AUs. We jointly train the FER model
on source domain with ground-truths and target domain with pseudo labels and
triplets.

###  3.2 The AU Distributions of FER Datasets

To check whether AUs are with low bias among FER datasets (RAF-DB, FERPlus,
ExpW, CK+), we visualize the AU distributions of each facial expression
category. Specifically, we first utilize a pretrained AU detection model to
extract AUs for each image. Then, we make statistics of AU occurrence numbers
over each category. After normalizing, we show the AU statistics in Figure 3.
We observe that i) the AU distributions of same categories are very similar
among different datasets and ii) different facial expressions own very
different AU distributions which indicates that AUs offer discriminate cues.

Figure 3: The distributions of AUs in different FER datasets.

###  3.3 AU-Guided Annotating

We introduce the AU-Guided Annotating (AGA) module to assign pseudo labels for
target domain images according to AU detection results. Specifically, we
elaborately design several assignment schemes as follows.

Source-based hard label assignment (S-hard). Given an image $X_{s}^{i}$ and
its detected AUs $E_{s}^{i}$ from source domain, the S-hard scheme utilizes
$E_{s}^{i}$ as a query to search target domain images that have same AUs with
$E_{s}^{i}$. This process can be defined as follows,

$[X_{t}^{1},\ldots,X_{t}^{k}]=Q_{\rm{S-hard}}(E_{s}^{i})=\\{X_{t}^{j}~{}~{}\textrm{if}~{}~{}E_{t}^{j}\equiv
E_{s}^{i}\\},$ (1)

where $[X_{t}^{1},\ldots,X_{t}^{k}]$ are the retrieval images from target
domain, $Q_{\rm{S-hard}}$ represents the S-hard operation. S-hard assigns all
the retrieval images with the label of $X_{s}^{i}$.

Target-based hard label assignment (T-hard). Different from S-hard, the T-hard
scheme uses target domain images as query images. Given an image $X_{t}^{i}$
and its detected AUs $E_{t}^{i}$ from target domain, the query process can be
formulated as follows,

$[X_{s}^{1},\ldots,X_{s}^{k}]=Q_{\rm{T-hard}}(E_{t}^{i})\\{X_{s}^{j}~{}~{}\textrm{if}~{}~{}E_{s}^{j}\equiv
E_{t}^{i}\\},$ (2)

where $[X_{s}^{1},\ldots,X_{s}^{k}]$ represents the retrieval images from
source domain. With the ground truths of source domain images, T-hard assigns
the most frequent label to the target domain image $X_{t}^{i}$.

Target-based soft label assignment (T-soft). For a query image from target
domain, besides the hard assignment scheme, the T-soft directly uses the label
distribution of retrieval samples to assign each target domain image a soft
label vector. It is worth noting that there does not exist a source-based soft
label assignment since we do not have the labels of target domain images.

Learning with AGA. After assigning pseudo labels for target domain, we can
train FER models in traditional ways. Suppose $Y_{s}\in\mathcal{R}^{{1}\times
M}$ represents the labels of $M$ source domain images,
$Y_{\rm{S-hard}}\in\mathcal{R}^{{1}\times N}$,
$Y_{\rm{T-hard}}\in\mathcal{R}^{{1}\times N}$,
$Y_{\rm{T-soft}}\in\mathcal{R}^{{C}\times N}$ denote the labels of $N$ target
domain images in S-hard, T-hard, and T-Soft schemes, we use the following loss
function by default to train with AGA module,

$L_{c}=\rm{CE}(P_{s},Y_{s})+\beta(\rm{CE}(P_{t},Y_{\rm{S-hard}})\\\
+\rm{KL}(P_{t},Y_{\rm{T-soft}})),$ (3)

where CE denotes the Cross-Entropy loss function, KL is the KL Divergence loss
function, $P_{t}$ and $P_{s}$ represent the predictions of target and source
domains images. $\beta$ is the trade-off ratio between the two loss values
that calculate by pseudo labels and ground truths.

###  3.4 AU-Guided Triplet Training

To achieve structure-reliable and compact facial features, we perform the AU-
Guided Triplet Training (AGT) to further narrow the gap among different
domains. The key step is to sample triplets from source and target domain.

Triplet Selection. Intuitively, we can select triplets from the union of
source domain and target domain. However, considering CD-FER is a
classification task, we ignore the triplets in source domain since ground
truths are available. Specifically, we only keep those cross-domain triplets.
Given an anchor in source domain ($X_{s}^{a}$) or target domain ($X_{t}^{a}$),
we first use it to retrieve the images of target domain or source domain that
own the same AUs, and then we randomly select a positive sample from target
domain ($X_{t}^{p}$) or source domain ($X_{s}^{p}$) according to the retrieval
samples, and a negative sample $X_{t}^{n}$ or $X_{s}^{n}$ according to the
rest samples of target domain or source domain. Thus, we mainly select two
kinds of cross-domain triplets: ($X_{s}^{a}$,$X_{t}^{p}$,$X_{t}^{n}$) and
($X_{t}^{a}$,$X_{s}^{p}$,$X_{s}^{n}$). We conduct triplet selection in an
offline manner. In addition, we also perform hard negative mining by sorting
the similarities between the AU scores of anchor and all negative samples. We
randomly select a sample from these with AU similarities larger than a
threshold $\tau_{n}$ (0.5 by default) as a negative sample.

AU-Guided Triplet Loss. After the selection of triplets, we use triplet loss
to learn discriminative and compact features as follows,

$L_{tri}=max\\{0,\gamma-(\left\|F_{a}-F_{n}\right\|-\left\|F_{a}-F_{p}\right\|)\\},$
(4)

where $F_{a}$, $F_{p}$, and $F_{n}$ represent the L2-normalized features of
anchor, positive, and negative images, respectively. $\gamma$ is a margin
which can be a fixed hyper parameter or a learnable parameter. We evaluate it
in the experiment section. Training with these cross-domain triplets, we can
obtain a cross-domain common feature space which makes similar facial images
close and dissimilar ones far away. Considering both pseudo annotations and
triplets, the total loss function is $L_{all}=L_{c}+\epsilon L_{tri}$ where
$\epsilon$ is a trade-off ratio.

###  3.5 Implementation Details

AU detection and FER backbone. Face images are detected and aligned by
Retinaface [9] and further resized to 224 $\times$ 224 pixels. We utilize an
advanced AU detector that pretrained on EmotiNet dataset using MLCR[34]
algorithm, to extract AUs for each image. We then evaluate the effectiveness
of AdaFER using ResNet-18 [15] with Pytorch toolbox. The ResNet-18 is pre-
trained on the MS-Celeb-1M face recognition dataset and the facial features
for triplet training are extracted from the last pooling layer.

Training. We train our AdaFER in an end-to-end manner with 1 Tesla V100, and
set the batch size as 128, i.e., 128 triplets with ground-truth labels or
pseudo labels. In each iteration, all the images are optimized by Cross-
Entropy loss, KLDiv loss and AU-Guided Triplet Loss. The ratio $\beta$ and is
defaulted as 1 and evaluated in the ensuing Experiments. The triplet loss
margin $\gamma$ is set as 0.5 by default. The ratio of $L_{c}$ and $L_{tri}$
is empirically set at 1:1, and its influence will be studied in the ensuing
ablation study of Experiments. The leaning rate is initialized as 0.001 with
Adam optimizer using Exponential (gamma =0.9) scheme to reduce the learning
rate. We stop training at 40-th epoch.

##  4 Experiments

In this section, we first describe employed datasets. We then demonstrate the
robustness of our AdaFER in cross-domain facial expression recognition task.
Further, we conduct ablation studies to show the effectiveness of each module
and the settings of hyper-parameters in AdaFER. After that, We compare AdaFER
to related state-of-the-art methods. Finally, to obtain a better understanding
of AdaFER, we visualize the statistical distributions of CK+ and FERPlus
datasets.

###  4.1 Datasets

RAF-DB. [23] consists of 30,000 facial images annotated with seven basic and
14 compound facial expressions by 40 trained students. In our experiments, we
default set RAF-DB as source dataset and only images with seven basic
expressions (neutral, happiness, surprise, sadness, anger, disgust, fear,
neutral) are used which leads to 12,271 images for training.

FER2013 & FERPlus [2] contain 28,709 training images, 3,589 validation images
and 3,589 test images, all of which are resized to 48$\times$48 pixels.
FER2013 is collected by Google Image Search API and annotated by seven facial
expressions. However, the annotation of FER2013 is not accurate because the
there are only two annotators. Therefore, FERPlus is extended from FER2013 as
used in the ICML 2013 Challenges, it is annotated by 10 annotators and added
contempt category.

CK+ [32] contains 593 video sequences from 123 subjects. Among these videos,
327 sequences from 118 subjects are labeled with seven expressions (except
neutral), i.e. anger, contempt, disgust, fear, happiness, sadness, and
surprise. All the subjects start from neutral and increase their expression
intensity to seven expressions. Therefore, we select the last three frames
with peak formation from each sequence and the first frame (neutral face) of
each sequence, resulting in 1236 images. We follow previous work [32] to
choose 1108 images for training and 128 images for testing.

ExpW [50] contains 91793 images that are annotated by one of the seven
expressions. Since the official ExpW dataset does not provide training/testing
splits, we follow [43] to select 28848 for training, 28853 for validating and
28848 for testing.

JAFFE [28] collects 213 images from 10 Japanese females in lab-controlled
condition. We here choose 170 images for training and 43 images for testing.

Table 1: Performance (%) comparison between the proposed AdaFER and baseline
methods

Method | CK+ | JAFFE | ExpW | FER2013 | FERPlus
---|---|---|---|---|---
#1 | 70.54 | 46.51 | 61.53 | 52.91 | 62.40
#2 | 65.11 | 27.50 | 66.52 | 46.31 | 68.35
#3 | 74.42 | 55.81 | 68.13 | 55.89 | 63.81
#4 | 73.64 | 58.14 | 69.41 | 54.81 | 70.02
#5 | 71.32 | 53.49 | 73.58 | 57.15 | 77.99
AdaFER | 81.40 | 61.37 | 70.86 | 57.29 | 78.22

###  4.2 AdaFER for Unsupervised CD-FER

To evaluate the effectiveness of AdaFER, we compare several baseline methods
with our proposed AdaFER using RAF-DB as source dataset and testing on CK+,
JAFFE, ExpW, FER2013, and FERPlus.We totally implement five baseline methods
as follows.

  * •

#1: We train a model on source data and directly test on target data.

  * •

#2: We first extract AUs for both source and target data, and then use the AUs
of each image in target data to query source data, and finally we assign the
most frequent category of retrieval images to the target image.

  * •

#3: We first use the trained model on source data to predict hard pseudo
labels of target data, then fine-tune the model on target data.

  * •

#4: It is identical with #3 except for the predicted pseudo labels are kept as
vectors (i.e. soft labels).

  * •

#5: We use both the image and detected AUs as inputs to train a classify on
the source set and then fine-tune the model on pseudo soft labelled target
data.

The results are shown in Table 1. Several observations can be concluded as
follows. First, our AdaFER almost outperforms all other baseline methods with
a large margin especially when testing on lab-controlled CK+ and JAFFE
datasets. Second, using pseudo labels (#3 and #4) to fine-tune model on target
data also achieves large improvement over #1. Third, the naive AU-based method
(#2) performs better than #1 on FERPlus and ExpW which indicates AUs are
useful among similar data. Last but not the least, the naive AU-based method
degrades in lab-controlled FER datasets which suggests there exists AU domain
gap between in-the-wild datasets and lab-controlled datasets. Nevertheless,
our AdaFER mitigates the domain gap and achieves large improvements on both
in-the-wild and lab-controlled datasets

Visualization of AU-Guided Annotating. To further investigate AdaFER, we
visualize the pseudo labels of target images annotated by AGA module and
baseline #3. We use our T-soft assignment and list the top-3 categories. As
shown in Fig 4, AGA achieves better results than baseline #3. In the first 6
images, AGA assigns the highest weights on the ground-truth category. The last
two bad cases are extremely ambiguous even for human, and our AGA seems to
assign reasonable categories for them.

Figure 4: Visualization of pseudo labels from baseline #3 and our AGA module. BSA represents baseline #3. Table 2: Evaluation the influences of anchor images for AGA module. Source | Target | CK+ | ExpW | FER2013 | FERPlus
---|---|---|---|---|---
$\surd$ | $\times$ | 80.62 | 69.90 | 54.36 | 77.61
$\times$ | $\surd$ | 71.23 | 69.73 | 55.45 | 77.93
$\surd$ | $\surd$ | 81.40 | 70.86 | 57.29 | 78.22

###  4.3 Ablation Studies

We conduct ablation studies for the modules of our AdaFER and other hyper
parameters.

Figure 5: Evaluation of pseudo assignment strategies . Table 3: Evaluation of
AU-Guided Annotating (AGA) and AU-Guided Triplet Training (AGT) on CK+, ExpW,
FER2013, and FERPlus.

AGA | AGT | CK+ | ExpW | FER2013 | FERPlus
---|---|---|---|---|---
$\times$ | $\times$ | 70.54 | 61.53 | 52.91 | 62.40
$\surd$ | $\times$ | 80.62 | 68.45 | 55.20 | 77.42
$\times$ | $\surd$ | 80.28 | 69.80 | 56.45 | 74.50
$\surd$ | $\surd$ | 81.40 | 70.86 | 57.29 | 78.22

The three types of label assignments. In AGA, we introduce three kinds of
assignment, namely S-hard, T-hard, and T-soft. We explore individual types and
their combination on CK+, ExpW, FERPlus, and FER2013. The results are shown in
Figure 5. For individual assignments, we observe that T-soft strategy performs
best in average following by S-hard strategy. Combining T-soft and S-hard
strategies further boost performance in most cases. We use this combination
strategy by default in the following experiments.

Anchor images in AGT module. In AU-guided triplet training, both source and
target data can be used as anchor images. We evaluate the effect of anchor
images in Table 2. As can be observed, using source data as anchor images
largely outperforms the target anchor scheme on CK+ which may be explained by
that the target CK+ dataset is too small to collect enough triplet samples.
Both individual anchor schemes perform similarly on large-scale datasets, and
combining both schemes consistently boosts performance on all datasets.

Figure 6: Evaluation of the margin parameter in AU-Guided Triplet Loss. Figure
7: Evaluation of the trade-off ratios $\epsilon$ and $\beta$.

AU-Guided Annotating (AGA) and AU-Guided Triplet Training (AGT) are two
crucial modules in AdaFER which respectively leverages pseudo labels for
target domain images and constraints the distance structure of each triplet
tuple. To explore the effectiveness of each module, we conduct the evaluation
on CK+, ExpW, FER2013, and FERPlus. As shown in Table 3, both AGA and AGT can
individually improve the baseline by a large margin, and they perform
similarly on most of datasets. This may be explained by that both AGA and AGT
essentially resorts to AU information, and the only difference is that AGA
performs supervision on classifier while AGT on feature mapping. Nevertheless,
from the results of AGA+AGT, they are complementary with each other on all the
datasets.

The margin $\gamma$ of AGT. $\gamma$ is a margin parameter to control the
distance between anchor-positive pair and anchor-negative pair. Theoretically,
it can be a learnable parameter in the end-to-end framework. We evaluate it
with both fixed mode and learnable mode. The results are illustrated in Figure
6. For the fixed mode, we evaluate margin from 0.25 to 1.5 with 0.25 as the
interval. On all the 4 FER datasets, our default margin $\gamma=0.5$ achieves
the highest performance. Larger margin makes training harder which degrades
performance on CK+ and ExpW. For the learnable mode, $\gamma$ respectively
converges to 0.62 ($\pm 0.034$), 0.37 ($\pm 0.067$), 0.71($\pm 0.026$), and
0.42($\pm 0.033$) on CK+, ExpW, FERPlus, and FER2013. Meanwhile, the learnable
$\gamma$ also obtains competitive results.

The trade-off ratios $\beta$ and $\epsilon$. $\beta$ is the trade-off ratio
between two loss values calculated by pseudo labels and ground truths in
Equation (3). $\epsilon$ is the trade-off ratio between $L_{c}$ and $L_{tri}$.
We evaluate them from 0.0 to 2.0 with 0.25 as the interval on FERPlus dataset
and present the results in Figure 7 . For both $\beta$ and $\epsilon$, the
final performance increases gradually and reaches the peak in value 1.0.
Larger values degrade performance dramatically which illustrates that all the
loss items are almost equally important.

Table 4: Evaluation of the threshold for hard negative mining. Threshold | CK+ | ExpW | FER2013 | FERPlus
---|---|---|---|---
0 | 68.99 | 65.24 | 52.21 | 66.68
0.25 | 72.03 | 68.23 | 56.23 | 74.23
0.5 | 81.40 | 70.86 | 57.29 | 78.22
0.75 | 80.96 | 69.12 | 55.80 | 77.72
Table 5: Comparison to the state-of-the-art methods on CK+, JAFFE, FER2013,
ExpW datasets. The results of upper part are taken from the corresponding
papers, and the results of the bottom part are taken from a cross-domain
facial expression recognition benchmark’s [4] implementations. The last column
shows the mean accuracy of performances on all the datasets.

Methods | Source Dataset | Backbones | CK+ | JAFFE | FER2013 | ExpW | Mean
---|---|---|---|---|---|---|---
Da et al.[7] | BOSPHORUS | HOG & Gabor Filters | 57.60 | 36.2 | - | - | -
Hasani et al.[13] | MMI&FERA&DISFA | Inception-ResNet | 67.52 | - | - | - | -
Hasani et al.[14] | MMI&FERA | Inception-ResNet | 73.91 | - | - | - | -
Zavarez et al.[48] | Six Datasets | VGG-Net | 88.58 | 44.32 | - | - | -
Mollahosseini et al.[31] | Six Datasets | Inception | 64.20 | - | 34.00 | - | -
DETN[21] | RAF-DB | Manually-Designed Net | 78.83 | 57.75 | 52.37 | - | -
ECAN[22] | RAF-DB 2.0 | VGG-Net | 86.49 | 61.94 | 58.21 | - | -
CADA[26] | RAF-DB | ResNet-18 | 73.64 | 55.40 | 54.71 | 63.74 | 61.87
SAFN[44] | RAF-DB | ResNet-18 | 68.99 | 49.30 | 53.31 | 68.32 | 59.98
SWD[19] | RAF-DB | ResNet-18 | 72.09 | 53.52 | 53.70 | 65.85 | 61.29
LPL[23] | RAF-DB | ResNet-18 | 72.87 | 53.99 | 53.61 | 68.35 | 62.20
DETN[21] | RAF-DB | ResNet-18 | 64.19 | 52.11 | 42.01 | 43.92 | 50.55
ECAN[22] | RAF-DB | ResNet-18 | 66.51 | 52.11 | 50.76 | 48.73 | 54.52
AGRA[43] | RAF-DB | ResNet-18 | 77.52 | 61.03 | 54.94 | 69.70 | 65.79
AdaFER | RAF-DB | ResNet-18 | 81.40 | 61.37 | 57.29 | 70.86 | 67.73
Baseline (#3) | FERPlus | ResNet-18 | 64.34 | 41.86 | - | 66.64 | 57.87
AdaFER | FERPlus | ResNet-18 | 65.12 | 46.51 | - | 73.58 | 61.47

The threshold for hard negative mining of triplet samples. In triplet
selection, we sort the AU scores and set a threshold for negative mining. We
evaluate the threshold from 0 to 0.75 in Table 4. On all datasets, increasing
the threshold from 0 to 0.5 boosts performance largely. Too small threshold
could make zero triplet loss since triplet samples are too easy. Too large
threshold may introduce hard positive samples as negative ones which is
harmful for training.

Figure 8: Visualization of feature distributions on CK+ and FERPlus datasets.
The top and bottom parts show the feature distributions using baseline method
and AdaFER, respectively.

###  4.4 Comparison to state-of-the-art methods

Table 5 compares our AdaFER with 13 state-of-the-art (SOTA) methods in CD-FER
task. The methods from the bottom table use RAF-DB as source dataset, and
these from the upper table use other datasets. As shown in Table 5, our AdaFER
achieves competitive results compared to SOTA methods. Zavarez et al.[48]
achieve the SOTA on CK+ using six datasets. ECAN[22] obtain SOTA on JAFFE and
FER2013 using a model pretrained on VGGFace2 and then fine-tune on RAF-DB2.0
which is not publicly available. For fair comparison, the methods in the
bottom part of Table 5 use the same source dataset and backbone. The mean
accuracy over all target datasets is also computed for easy comparison. Our
AdaFER obtains accuracy 81.40%, 61.37%, 57.29%, and 70.86% on CK+, JAFFE,
FER2013, and ExpW datasets, which are the new state-of-the-art CD-FER results
on these datasets. Moreover, our AdaFER does not increase any computing cost
in inference phase. AGRN [43] needs to extract holistic and local features to
initialize the nodes of target domain in inference stage, which is time-
cosuming. LPL [23] relies on the assumption of both source and target domains.

To evaluate the robustness of our method on different source datasets, we also
show the CD-FER performance with FERPlus as source dataset. Note that the
FER2013 dataset has the same images as FERPlus, therefore we ignore the
performance on FER2013. Our AdaFER also improve the baseline (#3) with large
margin in term of mean accuracy.

Understanding AU-Guided Learning. To better understand the differences between
baseline (#3) and our AdaFER, we utilize the T-SNE method to illustrate the
feature distributions of the test sets of CK+ and FERPlus. The results are
shown in Figure 8. We can find that our AdaFER can learn more compact features
than baseline in unsupervised CD-FER tasks. For CK+, ‘Neutral’ samples are the
most scattered ones for the baseline method, AdaFER can cluster them
dramatically. For FERPlus, the samples of same categories are clustered
compactly while these of different categories are largely separated. These
illustrates the reason why AdaFER improves baseline dramatically.

##  5 Conslusion

In this paper, we address the unsupervised domain adaptive facial expression
recognition task with the auxiliary facial action units. Our method is very
different from existing cross-domain FER methods which typically follow
generic cross-domain methods. Specifically, we proposed an AU-guided
unsupervised Domain Adaptive FER (AdaFER) framework which includes an AU-
guided annotating module and an AU-guided triplet training module. We
evaluated several AU-guided annotating strategies and triplet selection
methods. Extensive experiments on several popular benchmarks show that i)
AdaFER achieves state-of-the-art results and ii) obtains discriminative and
compact facial expression features.

## References

  * [1] Brandon Amos, Bartosz Ludwiczuk, Mahadev Satyanarayanan, et al.  Openface: A general-purpose face recognition library with mobile applications.  CMU School of Computer Science, 6(2), 2016.
  * [2] Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang.  Training deep networks for facial expression recognition with crowd-sourced label distribution.  In ACM ICMI, 2016.
  * [3] Emad Barsoum, Cha Zhang, Cristian Canton Ferrer, and Zhengyou Zhang.  Training deep networks for facial expression recognition with crowd-sourced label distribution.  In Proceedings of the 18th ACMMM, pages 279–283, 2016.
  * [4] Tianshui Chen, Yuan Xie, Tao Pu, Hefeng Wu, and Liang Lin.  Cross-domain facial expression recognition:a unified evaluation benchmark and adversarial graph learning.  In Arxiv, pages 1255–1264, 2020.
  * [5] Wen-Sheng Chu, Fernando De la Torre, and Jeffrey F Cohn.  Selective transfer machine for personalized facial expression analysis.  IEEE TPAMI, 39(3):529–545, 2016.
  * [6] Roddy Cowie, Ellen Douglas-Cowie, Nicolas Tsapatsoulis, George Votsis, Stefanos Kollias, Winfried Fellenz, and John G Taylor.  Emotion recognition in human-computer interaction.  IEEE Signal processing magazine, 18(1):32–80, 2001.
  * [7] Flávio Altinier Maximiano da Silva and Helio Pedrini.  Effects of cultural characteristics on building an emotion classifier through facial expression analysis.  Journal of Electronic Imaging, 24(2):023015, 2015.
  * [8] Navneet Dalal and Bill Triggs.  Histograms of oriented gradients for human detection.  In 2005 IEEE CVPR, volume 1, pages 886–893. IEEE, 2005.
  * [9] Jiankang Deng, Jia Guo, Yuxiang Zhou, Jinke Yu, Irene Kotsia, and Stefanos Zafeiriou.  Retinaface: Single-stage dense face localisation in the wild.  arXiv preprint arXiv:1905.00641, 2019.
  * [10] Abhinav Dhall, Roland Goecke, Simon Lucey, and Tom Gedeon.  Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark.  In 2011 IEEE ICCV Workshops, pages 2106–2112. IEEE, 2011.
  * [11] C Fabian Benitez-Quiroz, Ramprakash Srinivasan, and Aleix M Martinez.  Emotionet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild.  In Proceedings of the IEEE CVPR, pages 5562–5570, 2016.
  * [12] Geovanny Giorgana and Paul G Ploeger.  Facial expression recognition for domestic service robots.  In Robot Soccer World Cup, pages 353–364. Springer, 2011.
  * [13] Behzad Hasani and Mohammad H Mahoor.  Facial expression recognition using enhanced deep 3d convolutional neural networks.  In Proceedings of the IEEE CVPR Workshops, pages 30–40, 2017.
  * [14] Behzad Hasani and Mohammad H Mahoor.  Spatio-temporal facial expression recognition using convolutional neural networks and conditional random fields.  In 2017 12th IEEE FG 2017, pages 790–795. IEEE, 2017.
  * [15] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE CVPR, pages 770–778, 2016.
  * [16] Sijie Ji, Kai Wang, Xiaojiang Peng, Jianfei Yang, Zhaoyang Zeng, and Yu Qiao.  Multiple transfer learning and multi-label balanced training strategies for facial au detection in the wild.  In Proceedings of the IEEE CVPR Workshops, pages 414–415, 2020\.
  * [17] Wenyan Jiang, Zhongmin Yin, Yixin Pang, Feng Wu, Lingtao Kong, and Ke Xu.  Brain functional changes in facial expression recognition in patients with major depressive disorder before and after antidepressant treatment: A functional magnetic resonance imaging study.  Neural Regeneration Research, 7(15):1151, 2012.
  * [18] Samira Ebrahimi Kahou, Christopher Pal, Xavier Bouthillier, Pierre Froumenty, Çaglar Gülçehre, Roland Memisevic, Pascal Vincent, Aaron Courville, Yoshua Bengio, Raul Chandias Ferrari, et al.  Combining modality specific deep neural networks for emotion recognition in video.  In Proceedings of the 15th ACM on International conference on multimodal interaction, pages 543–550, 2013.
  * [19] Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel Ulbricht.  Sliced wasserstein discrepancy for unsupervised domain adaptation.  In Proceedings of the IEEE CVPR, pages 10285–10295, 2019.
  * [20] Gil Levi and Tal Hassner.  Emotion recognition in the wild via convolutional neural networks and mapped binary patterns.  In Proceedings of the 2015 ACM ICMI, pages 503–510, 2015.
  * [21] Shan Li and Weihong Deng.  Deep facial expression recognition: A survey.  IEEE TAC, 2020.
  * [22] Shan Li and Weihong Deng.  A deeper look at facial expression dataset bias.  IEEE TAC, 2020.
  * [23] Shan Li, Weihong Deng, and JunPing Du.  Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild.  In Proceedings of the CVPR, pages 2852–2861, 2017.
  * [24] Yong Li, Jiabei Zeng, Shiguang Shan, and Xilin Chen.  Occlusion aware facial expression recognition using cnn with attention mechanism.  IEEE TIP, 28(5):2439–2450, 2018.
  * [25] Mengyi Liu, Shaoxin Li, Shiguang Shan, and Xilin Chen.  Au-inspired deep networks for facial expression feature learning.  Neurocomputing, 159:126–136, 2015.
  * [26] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan.  Conditional adversarial domain adaptation.  In NIPS, pages 1640–1650, 2018.
  * [27] Patrick Lucey, Jeffrey F Cohn, Takeo Kanade, Jason Saragih, Zara Ambadar, and Iain Matthews.  The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression.  In 2010 CVPR-workshops, pages 94–101. IEEE, 2010.
  * [28] Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba.  Coding facial expressions with gabor wavelets.  In Proceedings Third IEEE international conference on automatic face and gesture recognition, pages 200–205. IEEE, 1998.
  * [29] Daniel McDuff, Abdelrahman Mahmoud, Mohammad Mavadati, May Amr, Jay Turcot, and Rana el Kaliouby.  Affdex sdk: A cross-platform real-time multi-face expression recognition toolkit.  CHI EA ’16, page 3723–3726. Association for Computing Machinery, 2016\.
  * [30] Yun-Qian Miao, Rodrigo Araujo, and Mohamed S Kamel.  Cross-domain facial expression recognition using supervised kernel mean matching.  In 2012 11th International Conference on Machine Learning and Applications, volume 2, pages 326–332. IEEE, 2012.
  * [31] Ali Mollahosseini, David Chan, and Mohammad H Mahoor.  Going deeper in facial expression recognition using deep neural networks.  In 2016 IEEE WACV, pages 1–10. IEEE, 2016.
  * [32] Ali Mollahosseini, Behzad Hasani, Mohammad H Mahoor, and Mohammad H Mahoor.  Affectnet: A database for facial expression, valence, and arousal computing in the wild.  TAC, 10(1):18–31, 2017.
  * [33] Pauline C Ng and Steven Henikoff.  Sift: Predicting amino acid changes that affect protein function.  Nucleic acids research, 31(13):3812–3814, 2003.
  * [34] Xuesong Niu, Hu Han, Shiguang Shan, and Xilin Chen.  Multi-label co-regularization for semi-supervised facial action unit recognition.  CoRR, abs/1910.11012, 2019.
  * [35] Enver Sangineto, Gloria Zen, Elisa Ricci, and Nicu Sebe.  We are not all equal: Personalizing models for facial expression analysis with transductive parameter transfer.  In Proceedings of the 22nd ACMMM, pages 357–366, 2014.
  * [36] Caifeng Shan, Shaogang Gong, and Peter W McOwan.  Facial expression recognition based on local binary patterns: A comprehensive study.  Image and vision Computing, 27(6):803–816, 2009.
  * [37] Yichuan Tang.  Deep learning using linear support vector machines.  arXiv preprint arXiv:1306.0239, 2013.
  * [38] Michel Valstar and Maja Pantic.  Induced disgust, happiness and surprise: an addition to the mmi facial expression database.  In Proc. 3rd Intern. Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect, page 65. Paris, France., 2010.
  * [39] M. F. Valstar and M. Pantic.  Fully automatic recognition of the temporal phases of facial actions.  IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(1):28–43, 2012.
  * [40] Kai Wang, Xiaojiang Peng, Jianfei Yang, Shijian Lu, and Yu Qiao.  Suppressing uncertainties for large-scale facial expression recognition.  In Proceedings of the IEEE CVPR, pages 6897–6906, 2020.
  * [41] Kai Wang, Xiaojiang Peng, Jianfei Yang, Debin Meng, and Yu Qiao.  Region attention networks for pose and occlusion robust facial expression recognition.  IEEE TIP, 29:4057–4069, 2020.
  * [42] Xiaoqing Wang, Xiangjun Wang, and Yubo Ni.  Unsupervised domain adaptation for facial expression recognition using generative adversarial networks.  Computational intelligence and neuroscience, 2018, 2018.
  * [43] Yuan Xie, Tianshui Chen, Tao Pu, Hefeng Wu, and Liang Lin.  Adversarial graph representation adaptation for cross-domain facial expression recognition.  In Proceedings of the 28th ACMMM, pages 1255–1264, 2020.
  * [44] Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin.  Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation.  In Proceedings of the IEEE ICCV, pages 1426–1435, 2019.
  * [45] Haibin Yan.  Transfer subspace learning for cross-dataset facial expression recognition.  Neurocomputing, 208:165–173, 2016.
  * [46] Keyu Yan, Wenming Zheng, Zhen Cui, and Yuan Zong.  Cross-database facial expression recognition via unsupervised domain adaptive dictionary learning.  In International Conference on Neural Information Processing, pages 427–434. Springer, 2016.
  * [47] Keyu Yan, Wenming Zheng, Tong Zhang, Yuan Zong, Chuangao Tang, Cheng Lu, and Zhen Cui.  Cross-domain facial expression recognition based on transductive deep transfer learning.  IEEE Access, 7:108906–108915, 2019.
  * [48] Marcus Vinicius Zavarez, Rodrigo F Berriel, and Thiago Oliveira-Santos.  Cross-database facial expression recognition based on fine-tuned deep convolutional network.  In 2017 30th SIBGRAPI, pages 405–412. IEEE, 2017.
  * [49] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao.  Joint face detection and alignment using multitask cascaded convolutional networks.  IEEE Signal Processing Letters, 23(10):1499–1503, 2016.
  * [50] Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang.  From facial expression recognition to interpersonal relation prediction.  IJCV, 126(5):550–569, 2018.
  * [51] Guoying Zhao, Xiaohua Huang, Matti Taini, Stan Z Li, and Matti PietikäInen.  Facial expression recognition from near-infrared videos.  Image and Vision Computing, 29(9):607–619, 2011.
  * [52] Wenming Zheng, Yuan Zong, Xiaoyan Zhou, and Minghai Xin.  Cross-domain color facial expression recognition using transductive transfer subspace learning.  IEEE transactions on Affective Computing, 9(1):21–37, 2016.
  * [53] W. Zheng, Y. Zong, X. Zhou, and M. Xin.  Cross-domain color facial expression recognition using transductive transfer subspace learning.  IEEE Transactions on Affective Computing, 9(1):21–37, 2018.
  * [54] Hengshun Zhou, Debin Meng, Yuanyuan Zhang, Xiaojiang Peng, Jun Du, Kai Wang, and Yu Qiao.  Exploring emotion features and fusion strategies for audio-video emotion recognition.  In 2019 ICMI, pages 562–566, 2019.
  * [55] Ronghang Zhu, Gaoli Sang, and Qijun Zhao.  Discriminative feature adaptation for cross-domain facial expression recognition.  In 2016 International Conference on Biometrics (ICB), pages 1–7. IEEE, 2016.
  * [56] Yuan Zong, Wenming Zheng, Xiaohua Huang, Jingang Shi, Zhen Cui, and Guoying Zhao.  Domain regeneration for cross-database micro-expression recognition.  IEEE TIP, 27(5):2484–2498, 2018.
  * [57] Y. Zong, W. Zheng, X. Huang, J. Shi, Z. Cui, and G. Zhao.  Domain regeneration for cross-database micro-expression recognition.  IEEE Transactions on Image Processing, 27(5):2484–2498, 2018.


# Variational energy based XPINNs for phase field analysis in brittle fracture

Ayan Chakraborty  Cosmin Anitescu  Somdatta Goswami  Xiaoying Zhuang and Timon
Rabczuk Ayan Chakraborty is with the Institute of Photonics, Leibniz
University, Hannover, Germany (e-mail<EMAIL_ADDRESS>hannover.de).Cosmin Anitescu is with the Institute of Structural Mechanics,
Bauhaus University-Weimar, Germany. (e-mail: cosmin.anitescu@uni-
weimar.de).Somdatta Goswami is with the Division of Applied Mathematics, Brown
University, USA. (e-mail: somdatta_goswami@brown.edu).Xiaoying Zhuang is with
the Department of Geotechnical Engineering,College of Civil Engineering,
Tongji University, Shanghai, China and is also the Chair of Computational
Science and Simulation Technology, Institute of Photonics, Faculty of
Mathematics and Physics,Leibniz University, Hannover, Germany. (e-mail:
zhuang@iop.uni-hannover.de).Timon Rabczuk is with the Division of
Computational Mechanics and the Faculty of Civil Engineering, Ton Duc Thang
University, Ho Chi Minh City, Vietnam. (e-mail: timon.rabczuk@uni-weimar.de).

###### Abstract

Modeling fracture is computationally expensive even in computational
simulations of two-dimensional problems. Hence, scaling up the available
approaches to be directly applied to large components or systems crucial for
real applications become challenging. In this work. we propose domain
decomposition framework for the variational physics-informed neural networks
to accurately approximate the crack path defined using the phase field
approach. We show that coupling domain decomposition and adaptive refinement
schemes permits to focus the numerical effort where it is most needed: around
the zones where crack propagates. No a priori knowledge of the damage pattern
is required. The ability to use numerous deep or shallow neural networks in
the smaller subdomains gives the proposed method the ability to be
parallelized. Additionally, the framework is integrated with adaptive non-
linear activation functions which enhance the learning ability of the
networks, and results in faster convergence. The efficiency of the proposed
approach is demonstrated numerically with three examples relevant to
engineering fracture mechanics. Upon the acceptance of the manuscript, all the
codes associated with the manuscript will be made available on Github.

Machine learning techniques have been increasingly used for modeling of
engineering problems. In particular, physics informed neural networks (PINNs)
have been shown to be a promising approach for discretizing and solving
partial differential equations. However, PINNs are best suited for smooth
function approximations and have some difficulties dealing with
discontinuities and rapidly changing gradients in the solution. Here, we
propose a framework for the simulation of nucleation and propagation of cracks
under brittle fracture using a subdomain-based phase-field approach. By
subdividing the domain into smaller regions and considering an energy
minimization formulation, the discontinuous displacements and singular stress
fields can be more accurately represented compared to the residual-based
formulation.

Adaptive activation function, Domain decomposition, Extended physics-informed,
Phase field

##  1 Introduction

In recent years, scientific machine learning has emerged as an essential tool
to address domain specific challenges in computational mechanics and extract
insights from scientific datasets and governing partial differential equations
(PDEs) through innovative methodological solutions. The theory of employing a
neural networks to solve an initial and a boundary value problem defined by a
PDE was proposed as early as $1990$’s [20, 19]. However, this discovery
remained theoretical due to the requirement of enormous compute power. The
latest resurrection of neural networks — the deep-learning revolution — is the
direct outcome of the modern GPUs that has turned the one-layer networks of
the $1960$s and the two- to three-layer networks of the $1980$s into the
$10$-, $15$-, and even $50$-layer networks of today. The advent of physics-
informed neural networks (PINNs) [24] served as the catalyst for this
revolution for computational mechanics engineers, where the hyperparameters of
the DNN are optimized by minimizing the residual of the governing PDE,
ensuring that the outputs of the network will necessarily satisfy the physics
of the problem. Almost during the same time, a variant of PINNs, the
variational energy based PINNs (VE-PINNs) [25], was proposed to solve the
governing PDE in its weak form. Authors in [9, 8] demonstrated a domain
adaptation transfer learning approach and goal functional error estimation
technique to solve PDES based on complex geometry. In [25], the authors showed
that for approximating the growth of fracture and to obtain the crack path, it
is obligatory to use the variational formulation since fracture is an energy-
driven phenomenon. VE-PINNs was able to alleviate the curse of dimensionality
with some trade-offs related to the degree of accuracy and relative ease of
modeling.

Fracture modeling is a computationally expensive phenomenon as it demands a
very fine mesh to resolve the damage region. To design a computationally
efficient approach, the same group also proposed an adaptive refinement scheme
within the framework of VE-PINNs to locally refine the domain along the path
of the growth of the crack [14]. To that end, another promising alternative
can be the integration of domain decomposition methods with the VE-PINNs
framework. The basic idea behind domain decomposition technique is to divide
the global domain into subdomains that can be solved independently and then
reconnected by interface conditions. These approaches are naturally applicable
to the solution of large-scale problems; their goal is to significantly
increase the computational efficiency of simulations. Along the same lines and
motivated by the extended physics informed neural networks (XPINNs) framework
[15], in this work we propose a domain decomposition framework for variational
energy based XPINNs (VE-XPINNs) to resolve crack paths in brittle materials.

The developed framework employs phase field modeling approach, a popular
continuous fracture modeling technique. VE-XPINNs are trained on the governing
coupled PDE (variational form) of the phase field approach to encode the
vector valued elastic field and the scalar valued phase field based on the
initial crack location, material properties and the characteristic width of
the crack. One significant advantage of the proposed variational energy
formulation is that it requires derivatives one order lower than the
conventional residual minimization approach, which results in better
computational efficiency. Additionally, motivated by the previous works of
V-PINNs [17], we use Gauss quadrature points to evaluate the integrals, over a
domain. To begin with, the computational domain is divided into a number of
elements and then, the quadrature points are generated within each element,
for an efficient integration of non-smooth functions like fracture.

In the following, we summarize the new contributions of the current work:

  * •

a generalized domain decomposition framework for variational energy based
PINNs to estimate the crack path defined using the phase field approach.

  * •

the proposed method employs separate neural networks for each subdomain, which
allows the use of tailored NNs and optimizers specifically catering to the
needs of a domain, improving computing efficiency and avoiding problems
related to overfitting.

  * •

the use of adaptive activation functions in this framework allows for better
learning and faster convergence of the solution.

The remainder of the paper is organized as follows. In subsection 2.1, we
discuss the problem statement for phase-field modeling of brittle fracture
using VE-XPINN. Implementation of the domain decomposition method for solving
fracture mechanics problems using phase field method is discussed in
subsection 2.2. Numerical examples illustrating the performance of the
proposed approach are presented in section 3. Finally, section 4 presents the
concluding remarks.

##  2 Methodology

###  2.1 Phase-field based fracture modelling

In this section, we briefly put forth the phase field formulation for brittle
fracture analysis. The integration of two fields, the vector-valued elastic
field and the scalar-valued phase field, is required to model fracture using
the phase field approach. While crack nucleation may be influenced by stress,
crack propagation necessitates an increase in the fracture energy or surface
energy of a solid, $\Psi_{c}$ [1]. As a result, the energy criteria is used in
the phase field approach to analyse the growth of fracture.

Let us begin with the linear elastic problem on an arbitrary body, $\Omega$,
with external boundary $\partial\Omega$. The displacement at each material
point $\boldsymbol{x}$ is denoted by $\boldsymbol{u}(\boldsymbol{x})$. In
addition, we assume a small strain tensor, $\boldsymbol{\epsilon}$ at each
material point defined as,

$\boldsymbol{\epsilon}(\boldsymbol{u})=\frac{1}{2}\left(\nabla\boldsymbol{u}+\nabla\boldsymbol{u}^{T}\right).$
(1)

The displacement field satisfies the Dirichlet and Neumann boundary
conditions. Considering the material is isotropic and is linearly elastic, the
stored elastic energy, $\Psi_{c}$ at any material point in the body is
described by the energy density function, $\Psi_{0}(\boldsymbol{\epsilon})$
as:

$\Psi_{0}(\boldsymbol{\epsilon})=\frac{1}{2}\lambda\text{tr}(\boldsymbol{\epsilon})^{2}+\mu\text{tr}(\boldsymbol{\epsilon}^{2}),$
(2)

where tr$(\cdot)$ denotes the trace of the strain tensor and $\lambda$ and
$\mu$ are the Lamé constants. The Cauchy stress tensor, $\boldsymbol{\sigma}$,
at any material point on the domain $\Omega$ can be computed as:

$\boldsymbol{\sigma}=\partial_{\boldsymbol{\epsilon}}\Psi_{0}(\boldsymbol{\epsilon})=\lambda\text{tr}(\boldsymbol{\epsilon})\boldsymbol{I}+2\mu\boldsymbol{\epsilon}.$
(3)

The momentum-balance equation for the elastic field, considering an isotropic
solid, can be written as:

$\begin{split}&\nabla\cdot\boldsymbol{\sigma}=\boldsymbol{f}(\boldsymbol{x})\quad\quad\,\;\;\text{in
}\Omega,\\\
&\boldsymbol{u}=\overline{\boldsymbol{u}}\qquad\qquad\;\;\;\;\;\;\text{on
}\partial\Omega_{D},\\\ &\boldsymbol{\sigma}\cdot
n=\boldsymbol{t}_{N}\qquad\,\;\;\;\;\;\;\text{on
}\partial\Omega_{N},\end{split}$ (4)

where the Dirichlet and Neumann boundaries are represented by
$\partial\Omega_{D}$ and $\partial\Omega_{N}$, respectively,
$\boldsymbol{t}_{N}$ is the prescribed boundary force and
$\boldsymbol{\overline{u}}$ is the prescribed displacement. The stored
internal potential energy of the body, for homogeneous Neumann boundary
conditions, is given by:

$\Psi_{int}(\boldsymbol{u})=\int_{\Omega}\Psi_{0}(\boldsymbol{\epsilon})\,d\boldsymbol{x},$
(5)

and the external energy is given by,

$\Psi_{ext}(u)=\int_{\Omega}\boldsymbol{f}\cdot\boldsymbol{u}~{}\,d\boldsymbol{x}+\int_{\partial\Omega_{N}}\boldsymbol{t}_{N}\cdot\boldsymbol{u}~{}\,d\gamma,$
(6)

where $\boldsymbol{f}$ is the body force. Therefore, the optimization problem
for linear elasticity is defined as:

$\min:\Psi_{e}=\Psi_{int}-\Psi_{ext},$ (7)

subjected to the pre-defined boundary conditions.

The equilibrium equation for the elastic field for an isotropic model,
considering the evolution of crack, involves the degradation of the stiffness
of the material around the area of the crack by penalizing the Cauchy stress
tensor, $\boldsymbol{\sigma}$ with a monotonically decreasing stress-
degradation function, $g(\phi)$. A common form of the stress-degradation
function in the literature is [23]

$g(\phi)=(1-\phi)^{2}.$

With evolving damage, only the tensile component of the principal stress
degrades while the compressive component remains invariant [22]. Therefore, in
order to prevent the growth of crack inside regions under compression, a
tension-compression split of $\Psi_{0}(\boldsymbol{\epsilon})$ is considered.
With evolving damage, $g(\phi)$ is applied only to the tensile component of
the principal strain. Consequently, due to the growth of fracture, the
anisotropic constitutive assumption for the degradation of the elastic strain
energy, can be stated as:

$\Psi_{e}(\boldsymbol{\epsilon}):=g(\phi)\Psi_{0}^{+}(\boldsymbol{\epsilon})+\Psi_{0}^{-}(\boldsymbol{\epsilon}),$
(8)

where $\Psi_{0}^{+}(\boldsymbol{\epsilon})$ and
$\Psi_{0}^{-}(\boldsymbol{\epsilon})$ denote the strain energies computed from
the positive and negative components of the principal strains, respectively.

The governing equation for the phase-field is written as [2, 7]:

$\frac{G_{c}}{l_{0}}\phi-
G_{c}l_{0}\nabla^{2}\phi=-g^{\prime}(\phi)\mathcal{H}(\boldsymbol{x},t)\text{
on }\Omega,$ (9)

where $G_{c}$ represents the critical energy release rate (property of
material), $\mathcal{H}(\boldsymbol{x},t)$ is a local strain-history function,
$\ell_{0}$ is the length scale parameter that controls the width of the
diffusive region of the crack. The effect of $l_{0}$ has been verified by a
series of numerical simulations [28, 27], demonstrating that the crack region
has a larger width with an increasing $\ell_{0}$ while the phase field
represents a sharp crack topology when $\ell_{0}\to 0$. The phase field is
used to smear out the crack surface over the domain $\Omega$. In the
regularized model, correspondingly, the phase field must satisfy the following
condition:

$\phi(\boldsymbol{x},t)=\left\\{\begin{array}[]{ll}0&\mbox{the material is
intact, }\\\ 1&\mbox{the material is completely cracked. }\end{array}\right.$
(10)

The local strain-history functional, $\mathcal{H}(\boldsymbol{x},t)$ contains
the maximum positive tensile energy, $\Psi_{0}^{+}$ in the history of
deformation of the system [23] and is defined as:

$\mathcal{H}(\boldsymbol{x},t)=\max_{s\in[0,t]}\Psi_{0}^{+}(\boldsymbol{\epsilon}(\boldsymbol{x},s)).$
(11)

The strain-history function can also be used to initialize or nucleate the
crack [22]. In particular, we set the initial strain-history function as [5]:

$\mathcal{H}(\boldsymbol{x},0)=\left\\{\begin{array}[]{ll}\frac{BG_{c}}{2\ell_{0}}\left(1-\frac{2d(\boldsymbol{x})}{\ell_{0}}\right)&\mbox{if
}d(\boldsymbol{x})\leq\frac{\ell_{0}}{2},\\\ 0&\mbox{otherwise.
}\end{array}\right.$ (12)

where $d(\boldsymbol{x})$ is the distance from $\boldsymbol{x}$ to the crack
tip, $B$ is a scalar parameter controlling the magnitude of the scalar history
field and is computed as:

$B(\phi)=\frac{1}{1-\phi}\;\;\mbox{for }\phi<1.$ (13)

The fracture energy, $\Psi_{c}$ of the newly formed cracks is expressed as:

$\Psi_{c}=\int_{\Omega}\left(G_{c}\Gamma_{n}(\phi)+g(\phi)\mathcal{H}(\boldsymbol{x},t)\right)\,d\Omega.$
(14)

Here, $\Gamma_{n}(\phi)$ represents the crack density functional and $n$ is
the order of the corresponding phase field model. For the second-order phase
field model, $n=2$, while for the fourth-order phase field model, $n=4$. The
fourth order phase model includes higher-order derivatives of $\phi$, leading
to greater regularity in the exact solution of the phase field. In this case
the cracked surface can be captured more accurately and fewer degrees of
freedom are required to resolve the crack path relative to the second-order
model. For the second and fourth order phase field model the crack density
functionals, $\Gamma$ are defined as [4]:

$\begin{split}\Gamma_{2}(\phi)&=\frac{1}{2\ell_{0}}\int_{\Omega}\left(\phi^{2}+\frac{\ell_{0}^{2}}{2}|\nabla\phi|^{2}\right)\,d\Omega,\\\
\Gamma_{4}(\phi)&=\frac{1}{2\ell_{0}}\int_{\Omega}\left(\phi^{2}+\frac{\ell_{0}^{2}}{2}|\nabla\phi|^{2}+\frac{\ell_{0}^{4}}{16}|\Delta\phi|^{2}\right)\,d\Omega.\end{split}$
(15)

In this work we study the growth of fracture employing the variational energy
based XPINN approach. To that end, we solve the optimization problem defined
as [1, 6, 3]:

$\begin{split}\textrm{Minimize }&:\mathcal{E}=\Psi_{e}+\Psi_{c},\\\
\textrm{constrained to }&:\boldsymbol{u}=\bar{\boldsymbol{u}}~{}~{}\text{on
}\partial\Omega_{D},\\\ \textrm{such that
}&:\Psi_{e}=\int_{\Omega}\left(g(\phi)\Psi_{0}^{+}+\Psi_{0}^{-}\right)\,d\Omega,\\\
&\;\;\;\;\Psi_{c}=\int_{\Omega}G_{c}\Gamma_{n}+g(\phi)\mathcal{H}(\boldsymbol{x},t)\,d\Omega.\end{split}$
(16)

The homogeneous Neumann boundary conditions are automatically satisfied when
the variational energy principle is used.

###  2.2 Variational energy based XPINNs

When performing a traditional numerical model analysis, like FEM and IGA, to
simulate the mechanical response of solids under loading dynamics and crack
propagation, as described in subsection 2.1, the resulting computational
burden can be very high, especially when addressing a fully three-dimensional
problem. In fact, a very refined mesh is required to properly describe the
discrete crack boundaries and resolve the crack tip. This results in a large
number of degrees of freedom that needs to be solved to compute the quantities
of interest. Hence, the high computational burden of a three-dimensional
simulation remains a major disadvantage. To alleviate the burden, in this
section we introduce the domain decomposition method.

Suppose the computational domain $\Omega$ is partitioned into $p$ subdomains
$\\{\Omega_{s}:s=1,2\ldots,p\\}$ such that
$\Omega=\bigcup_{s=1}^{p}\Omega_{s}$. By partitioning, we imply that the
division is non overlapping, and the subdomains intersect only on their common
boundaries which is also called as _interface_. To simplify the discussion,
consider the linear elastic problem defined in Equation 4 as an example, where
on a particular subdomain $\Omega_{s}$ we have:

$-\nabla\cdot\boldsymbol{\sigma}_{s}(\boldsymbol{x})=\boldsymbol{f}_{s}(\boldsymbol{x})\;\;\text{in
}\Omega_{s},$ (17)

where $\boldsymbol{f}_{s}$ is the body force in the $s$-th subdomain, and
$\boldsymbol{\sigma}_{s}$ is the Cauchy stress tensor associated with the
material point in the $s$-th domain. Each subdomain is constrained by the
continuity condition:

$\boldsymbol{u}_{i}(\boldsymbol{x})=\boldsymbol{u}_{j}(\boldsymbol{x})\qquad\text{on
}\Gamma_{ij},\\\ $ (18)

where $\Gamma_{ij}$ is the interface between the $i$-th and $j$-th subdomains,
and $\boldsymbol{u}_{i}$ and $\boldsymbol{u}_{j}$ are the displacements on
$\Omega_{i}$ and $\Omega_{j}$ respectively. The above boundary conditions also
represent the interface transmission conditions between the neighbouring
subdomains. For each of these subdomains, a separate deep neural network (DNN)
is deployed.

A DNN [9], $\mathfrak{N}_{\mathrm{L}}:\mathbb{R}^{d}\mapsto\mathbb{R}^{p}$ is
a non linear function defined as concatenations of affine maps with point-wise
non linearities, in the form:

$\mathfrak{N}_{\mathrm{L}}(\boldsymbol{x})=\mathcal{W}_{\mathrm{L}}\tau_{\mathrm{L}}\left(\ldots\tau_{3}\left(\mathcal{W}_{2}\tau_{2}\left(\mathcal{W}_{1}\boldsymbol{x}+\mathbb{\beta}_{1}\right)+\mathbb{\beta}_{2}\right)\ldots\right)+\mathbb{\beta}_{\mathrm{L}}$
(19)

where, $\mathcal{W}_{i}$’s are weight matrices may not necessarily be square
and $\beta_{i}$’s are referred as bias vectors. $\mathrm{L}$ is the depth (the
number of layers) of the network and $\mathrm{L}\geq 3$. The input vector is
$\boldsymbol{x}$ and the output vector of any arbitrary $\ell$-th layer is
denoted by $\mathfrak{N}_{\ell}(\boldsymbol{x})$ in particular,
$\mathfrak{N}_{0}(\boldsymbol{x})=\boldsymbol{x}$. The non-linear monotonic
function $\tau$ is known as activation function applied layer-wise to any
vector, however the dimension of the input vector may vary depending on
layers. However, regardless of input dimension, the activation function
performs the same operations on all input entries. The activation function in
the final layer is linear.

Recently, the implementation of adaptive activation functions [18, 16] (AAF)
have been receiving more attention, i.e, function possess trainable parameters
that changes their shape in the training process. AAFs have shown advantages
such as reducing training times while significantly improving the rate of
convergence and increased accuracy of the solution. Therefore the model ends
up with better learning capabilities than the the model with a fixed
activation function. To expand upon these notions, an adaptive model that can
easily be linked with existing methodologies to increase overall accuracy of
VE-XPINNs has been investigated in this work. The simplest form is just to add
a parameter to a particular neural network that controls one of its
properties, e.g. slope. Following the approach in [16]. in this work we
introduced two parameters $\mathfrak{c}$ (scaling parameter) and $\alpha$
(hyperparameter to be optimized) in the activation function as:

$\tau_{\mathrm{L}}(\ldots\mathfrak{c}\cdot\alpha~{}\tau_{3}(\mathfrak{c}\cdot\alpha~{}\cdot\tau_{2}(\cdot))).$
(20)

The primary goal of fracture mechanics is to determine the crack path. In this
work, we have studied displacement-controlled fracture, i.e. this is growth of
the crack by applying a displacement increment until failure. In addition, we
assumed a constant displacement step, $\Delta u$. The proposed VE-XPINN is
trained and the strain-history function of each sub-domain is updated at each
displacement increment in this configuration. In the beginning, the network
parameters are initialized using a Gaussian distribution with Xavier
initialization technique [10]. Let us first use a DNN to represent the
displacement field, $\boldsymbol{U}_{s}$, and the phase-field, $\phi_{s}$ of
the $s$-th subdomain, such that

$\left(\boldsymbol{U}_{s},\phi_{s}\right)=\mathfrak{N}_{s}\left(\boldsymbol{x}_{s};[\mathcal{W}_{s},\beta_{s}]\right).$
(21)

Each subdomain is discretised into $n_{e}^{s}$ elements, and we generate the
Gauss points within each element and their corresponding weights. The total
variational energy is calculated at the Gauss points to approximate the
integral. The displacements and the phase field at the common interfaces are
assumed to be continuous. Therefore, for solving the PDE as a whole, we
defining the loss function separately for the interior of each subdomain and
the interfaces as follows:

$\begin{split}\mathcal{L}_{s}^{int}(\mathcal{W},\beta_{s})&=\Psi_{e}(\epsilon(U_{s}),\phi_{s})+\Psi_{c}(G_{c},\boldsymbol{x}_{s})\\\
\mathcal{L}_{ij}^{iface}(\mathcal{W},\beta_{s})&=\frac{W_{1}}{N_{\Gamma_{ij}}}\sum_{k=1}^{N_{\Gamma_{ij}}}|\boldsymbol{u}_{i}(\boldsymbol{x}_{k})-\boldsymbol{u}_{j}(\boldsymbol{x}_{k})|^{2}\\\
&+\frac{W_{2}}{N_{\Gamma_{ij}}}\sum_{k=1}^{N_{\Gamma_{ij}}}\left|\phi_{i}(\boldsymbol{x}_{k})-\phi_{j}(\boldsymbol{x}_{k})\right|^{2}\end{split}$

where $N_{\Gamma_{s}}$ represents the total training points scattered over its
interfaces, respectively. The interface condition endowed with the loss
function plays an important role for stitching the subdomains together. It
ensures that the data information should propagate among the neighbouring
domains. A sufficient number of collocation points needs to be generated on
the interface leading to faster convergence of the algorithm. In subsection
2.2, $W_{i}$ $\forall\;\;i={1,2}$ are penalizing parameters, which are chosen
manually for balancing each of the loss terms and also for faster convergence.
From the viewpoint of optimization it could be more sophisticated to multiply
the constraints with suitable penalty coefficients. A method for adaptively
choosing these penalty parameters has been proposed in [21].

The analysis of fracture requires a very fine mesh in the vicinity of the
crack, to capture local quantities of interest. To that end, we could either
pre-refine the expected path of crack growth based on prior information
available from the literature, or need an efficient adaptive refinement scheme
that generates more quadrature points along the path that the crack grows.
Since knowledge to pre-refine the domain is strictly limited, in this work we
propose an adaptive $\it{h}$-refinement scheme for VE-XPINN to enhance the
formulation. The refinement scheme locally refines the crack path as the crack
grows.

One of the most difficult challenges for adaptive refinement schemes is
determining the appropriate refinement criterion, which determines the
location and level of refinement. The resulting refinement scheme has two
components: (i) a model training status derived from the loss function of each
subdomain that serves as an error indicator, and (ii) an algorithmic component
that performs domain refinement. Since the values of the phase-field variable
$(\phi)$ increase from zero to one from the intact domains to the crack
surface, both the phase field variable and its gradient are used as error
estimators to determine the intensity of training points around crack
surfaces. Initially the domain is divided into several subdomain and we
consider a threshold $\phi_{thres}$ and hence the expression
$\phi>\phi_{thres}$ is used as a refinement criteria to improve the domain
refinement around crack surfaces especially the areas near the crack-tips. The
region chosen for refinement should not be finer than a chosen maximum
refinement level. On the other hand, for the phase field area a recovery-based
posteriori error estimator [29] is used to drive adaptivity. The refinement
strategy is based on refining the region that contributes the most to error.
The process of generating fine refinements in order to achieve optimal results
in which the error norm is distributed evenly across the decomposed
subdomains. Precisely, the underlying idea is to minimize the loss function in
each subdomain, prior to the resulting error function meets a predefined
tolerance.

##  3 Numerical Examples

As a demonstration of the effectiveness of our proposed model, we study
several benchmark problems from fracture mechanics in this section. The first
example illustrates a one-dimensional elastic bar with a crack in the center
subjected to sinusoidal loading. An analytical solution to this problem
exists. Thus, the proposed adaptive refinement approach can be validated. In
addition, we have considered two other examples; the single-edge notched
tension test and a specimen with an eccentric hole under tension. All the
experiments have been carried out using the _TensorFlow_ framework on Google
Colab Pro GPU.

###  3.1 One dimensional cracked elastic bar

We consider a one-dimensional bar that is fixed at both the ends $(x=-1,x=1)$
and is subjected to a sinusoidal load. The bar has a crack at the center,
$x=0$. The geometrical setup is presented in Figure 1.

Figure 1: Geometrical setup of one-dimensional elastic bar with crack.

For simplicity $E$ is assumed to be unity and the strain ($\epsilon$) is
assumed to be non-negative in the crack zone. Hence, the stress-strain
relation is obtained by:

$\sigma=(1-\phi)^{2}\epsilon.$ (22)

The crack at center is imposed by the following step function:

$\mathcal{H}(x,0):=\left\\{\begin{array}[]{ll}1000&\mbox{if
}d(x)\leq\ell_{0}\\\ 0&\mbox{if }d(x)>\ell_{0},\end{array}\right.$ (23)

where $\ell_{o}$ is known as length-scale parameter. In particular we choose
$\ell_{0}=\frac{1}{80}$. The displacement field satisfies Dirichlet boundary
condition, ı.e,

$u(-1)=u(1)=0.$ (24)

The analytical solution for the displacement field [26] is discontinuous and
is given by,

$u_{ex}:=\left\\{\begin{array}[]{ll}\frac{1}{\pi^{2}}\sin(\pi
x)-\frac{1+x}{\pi}&\mbox{if }x<0\\\ \frac{1}{\pi^{2}}\sin(\pi
x)+\frac{1-x}{\pi}&\mbox{if }x\geq 0\end{array},\right.$ (25)

and solution for the phase field [23] is,

$\phi_{ex}:=\exp\left(\frac{-|x|}{\ell_{0}}\right).$ (26)

In this example, we have considered two possibilities. The first setup uses
two subdomains such that

$\underbrace{[-1.0,1.0]}_{\Omega}=\underbrace{[-1.0,0.0]}_{\Omega_{1}}\cup\underbrace{[0.0,1.0]}_{\Omega_{2}},$

while the second setup employs the computational domain $\Omega$ is subdivided
into four subdomains, such that

$\underbrace{[-1,1]}_{\Omega}=\underbrace{[-1,-0.5]}_{\Omega_{1}}\cup\underbrace{[-0.5,0]}_{\Omega_{2}}\cup\underbrace{[0,0.5]}_{\Omega_{3}}\cup\underbrace{[0.5,1]}_{\Omega_{4}}.$

We have used adaptive tanh activation in the hidden layers whereas for the
final layer, linear activation function has been considered and
$\epsilon_{tol}=1e-12$. The learning rate $\alpha=0.001$ is used in the _BFGS_
optimizer following _Adam_. The region to be refined is decided based on the
critical threshold, $\phi_{thres}$ and percentage of total estimated error for
elasticity, $\rho$. In this problem, $\rho=25\%$ and $\phi_{thres}$ is fixed
at 0.2. To ensure that the neural network outputs exactly satisfies the
Dirichlet boundary conditions, we set the following:

$\boldsymbol{u}=[(x+1)(x-1)]\boldsymbol{u}_{\theta},$

where $\boldsymbol{u}_{\theta}$ is the displacement field obtained as output
from the neural network. In the proposed VE-XPINN approach, the network is
trained by minimizing the total variational energy of the system as defined in
Equation 16. The adaptive refinement scheme generates more quadrature points
in the vicinity of the crack. For the setup with four subdomains, the
refinement is carried out in the intervals $\Omega_{2}$ and $\Omega_{3}$. We
start training the network with $100-250-250-100$ Gauss quadrature points and
to achieve a relative $L_{2}$ prediction error of less than $1.5\%$ in $\phi$,
the domain is refined adaptively and we finally end the training with
$250-600-600-250$ quadrature points in the four domains, respectively. To
quantify the accuracy of the results obtained using the proposed approach, the
relative $L_{2}$ error corresponding to $u$ and $\phi$ are reported in Table
1. To illustrate the superiority of using the fourth-order phase field model
over the second-order model, we simulate the second order model and report a
prediction error of $3.4\%$ for $u$ and $2.51\%$ for $\phi$.

Table 1: Summary of results corresponding to subsection 3.1, when the domain is discretized in $2$ and $4$ subdomains and using the fourth-order phase field model. # of | DNN Architecture | Integration | Prediction error
---|---|---|---
domains | points | $\mathcal{L}_{2}^{rel,u}$ | $\mathcal{L}_{2}^{rel,\phi}$
$2$ | $\left[1,10,10,10,2\right]$ | $800,800$ | $3.4\%$ | $2.51\%$
$4$ | $\left[1,10,10,10,2\right]$ | $250,600,600,250$ | $1.19\%$ | $<1\%$

Figure 2 (a) and (b) depicts the solution plots of $\boldsymbol{u}$ and $\phi$
using the fourth-order phase field model with $4$ subdomains. As expected, in
Figure 2 (g), we see a sharp decrease in the percentage error with an increase
in the number of quadrature points in the vicinity of the crack. In Figure 2
(c)-(f), we present the loss plots and the relative $\mathcal{L}_{2}$ error
plots obtained during the training for a framework integrated with the
adaptive activation function and also with a constant activation function. The
plots show that the simulation with adaptive activation function converges to
a lower $\mathcal{L}_{2}$ error, which shows that the adaptive activation
function provides advantages like an increase in accuracy and fast convergence
rate, especially in the early training period. We noted that for unity scaling
factor (when $\mathfrak{c}=1)$ the optimization process is slower but the
tuning process speeds up once the scaling factor increases from
$\mathfrak{c}=1,\ldots 10$. The inclusion of scalable hyperparameter
dynamically changes the topology of loss values and thereby attaining faster
convergence towards the minima.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

Figure 2: Results for one-dimensional elastic bar with crack using the domain
decomposition framework with the variational energy based PINN (proposed
approach). (a) and (b) show the agreement of the proposed approach with the
exact solution computed using Equation 25 and Equation 26. (c)-(f) show the
loss plot and the $\mathcal{L}_{2}$ error plots for an adaptive activation
function vs a constant activation function. Finally, (g) shows the convergence
of the solution with the increase in the number of quadrature points.

Finally, we compare our results with one of the earlier work from this group
on adaptive fourth-order phase field using VE-PINNs [14]. The work reports a
relative $\mathcal{L}_{2}$ error of $2.19\%$ and is $1.6\%$ of
$\boldsymbol{u}$ and $\phi$, respectively. With these results, we summarize
that the proposed approach approximates the solution better than the work in
the existing literature.

###  3.2 Single-edge notched tension example

As the next example, we consider a unit square plate with a horizontal crack
from the midpoint of the left outer edge to the center of the plate. The
geometrical setup and the boundary conditions of the problem are shown in
Figure 3(a). We consider $\lambda=121.15$ kN/mm2, $\mu=80.77$ kN/mm2 and
$G_{c}=2.7\times 10^{-3}$ kN/mm. The computation is performed by applying a
constant displacement increment of $\Delta u$ = $1\times 10^{-3}$ mm. In this
example, we consider $l_{0}=0.0125$.

(a)

(b)

(c)

(d)

Figure 3: (a) Geometrical setup and boundary conditions for the single-edge
notched tension test. All the units are in mm. (b)-(d) Distribution of
training data over four subdomains, eight subdomains and twelve subdomains.

The crack is initiated using the strain-history functional as defined in
Equation 12. The Dirichlet boundary conditions are:

$\boldsymbol{u}(0,y)=0,\;\;\boldsymbol{v}(x,0)=0,\;\;\boldsymbol{v}(x,1)=\Delta
u,$ (27)

where $\boldsymbol{u}$ and $\boldsymbol{v}$ are the solutions of the elastic
field in coordinate axes. In order to exactly satisfy the Dirichlet boundary
conditions, the neural network outputs for the elastic field are modified as:

$\begin{split}\boldsymbol{u}&=[x(1-x)]\boldsymbol{u}_{\theta},\\\
\boldsymbol{v}&=[y(y-1)]\boldsymbol{v}_{\theta}+y\Delta u,\end{split}$ (28)

where $\boldsymbol{u}_{\theta}$ and $\boldsymbol{v}_{\theta}$ are the neural
network approximations of the displacement field in $x-$ and $y-$ axes. In
this example an adaptive refinement scheme has been used. For robust
implementations, subdomains within the cracked zones are refined and hence
sub-divided into several sub-elements. We carried out experiments using multi
subdomains, 4, 8 and 12 to understand the behaviour of the solution as the
number of subdomains increases. Table 2 gives the details of training points
used in various experiments. The subdomain arrangement for all the experiments
is shown in Figure 3(b)-(d).

Table 2: Summary of the subdomain arrangements studied in problem 2. # of | # elements | Integration | Interface points
---|---|---|---
domains | per subdomain | points per element | per subdomain
$4$ | $11^{2}$ | $4$ | $1600$
$8$ |  $9^{2}$,$16^{2}$ | $4$ | $1000$
$12$ |  $6^{2}$,$8^{2}$, $14^{2}$ | $4$ | $800$

In all the cases, the networks has $4$\- hidden layers with $50$ neurons in
the model architecture, employing one DNN for each subdomain. As expected, the
crack is less diffused for a smaller length scale parameter. The crack path
obtained using $12$ subdomains and the fourth-order phase field model is shown
in Figure 4. The presented crack patterns are the same as those reported in
[12]. The cracks propagate in a horizontal direction. The load-displacement
curves computed at the top edge of the plate are also shown in Figure 5 with a
comparison between fourth order and second order phase field models. Based on
the plots, we can conclude that using the fourth-order model, we obtain a more
linear response before failure. Also the post-failure behavior of the fourth-
order model agrees well with the standard FEM and IGA [13]. Based on the plots
we can also conclude that as the crack progresses, the refinement procedure
refines its path locally. By increasing the density of the quadrature and
interface points in each sub elements the imprinting effect of the interfaces
has been significantly reduced. Moreover, we have used BFGS optimizer
following SGD with learning rate $\in(0.001,0.09]$. Although large scaling
factor speeds up the convergence rate but because of SGD optimizer one should
avoid using larger scaling factor otherwise the model may encounter gradient
explode or doesn’t even converges at all. To overcome this difficulty we have
added a regularization term so that it suppress the oscillations in the loss
functions. In practice we rely on trial and error analysis approach to learn a
suitable value of regularization weight.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 4: The experiment with $12$ subdomains with adaptive refinement for
prescribed displacements of (from (a) to (c)) $10^{-3}$, $7\times 10^{-3}$ and
$10^{-2}$ mm using the fourth-order phase field model in the domain
decomposition framework of variational energy based XPINN approach.

(a)

(b)

(c)

Figure 5: Force-displacement graph for second order and fourth order phase
field model using the proposed approach for (a) $4$ subdomains, (b) $8$
subdomains, and (c) $12$ subdomains.

###  3.3 Square plate with eccentric hole

This numerical example is based on a unit square plate embedded with an
eccentric hole, as shown in Figure 6(a). The bottom boundary remains fixed. At
the top boundary, a quasi-static load is applied in the form of a prescribed
displacement increment $\bar{\boldsymbol{u}}$. The material properties
considered for this problem are: $\lambda=121.154$ kN/mm2, $\mu=80.77$ kN/mm2,
$G_{c}=2.7\times 10^{3}$ kN/mm. For this example, we have considered
$l_{0}=0.02$. A displacement increment of $10^{-3}$ mm is applied at each
incremental step.

(a)

(b)

(c)

Figure 6: (a) Geometrical setup and boundary conditions for the square plate with eccentric hole test. All the units are in mm. (b) and (c) Distribution of training data over four and eight subdomains, respectively. Table 3: Summary of the subdomain arrangements studied in problem 3. # of | # elements | Integration | Interface points
---|---|---|---
domains | per subdomain | points per element | per subdomain
$4$ | $12^{2}$ | $4$ | $1600$
$8$ |  $8^{2}$,$14^{2}$ | $4$ | $1000$

In Table 3, the details of the experimental setup of $4$ and $8$ subdomains
are provided. A $3$-layers fully-connected neural networks with $50$ neurons
per hidden layer with $swish$ activation is employed. The final layer uses
linear activation function. The outputs for the elastic field are altered to
exactly match the Dirichlet boundary conditions, following:

$\begin{split}\boldsymbol{u}&=x\boldsymbol{u}_{\theta},\\\
\boldsymbol{v}&=y(y-1)\boldsymbol{v}_{\theta}+y\Delta u,\end{split}$ (29)

where $\boldsymbol{u}$ and $\boldsymbol{v}$ are the solutions of the elastic
field in coordinate axes, and $\boldsymbol{u}_{\theta}$ and
$\boldsymbol{v}_{\theta}$ are the neural network approximations of the
displacement field in $x-$ and $y-$ axes. Figure 7 demonstrates the results of
the fourth-order phase-field model obtained using the proposed adaptive XPINN
based approach. The load-displacement curves for both the experiments are
shown in Figure 8. In the specimen $8$ subdomains, we observe a linear pre-
peak behavior while this linear stage is missing for the experiment with $4$
subdomains.

(a)

(b)

(c)

Figure 7: Crack growth in the square plate with eccentric hole, using the
fourth-order phase field model and $8$ subdomains.

(a)

(b)

Figure 8: Load displacement plots of square plate with eccentric hole. (a) $4$
subdomains, and (b) $8$ subdomains.

##  4 Conclusion

In this work, we have proposed a domain decomposition approach for variational
physics informed neural networks (VE-XPINNs), that aims to accurately resolve
the crack path employing the phase field modeling approach. The proposed
approach is more efficient and accurate than its predecessor VE-PINNS [11,
14]. The proposed method can be used to solve any differential equation in its
weak form, much like the VE-PINN method can. This is accomplished by enforcing
the residual continuity requirement along the shared subdomain interfaces. VE-
XPINNs has the ability to be scaled up to solve real world probelms because of
the deployment of separate neural network in each subdomain (which can be
easily parallelized), and efficient and tailored hyper-parameter adjustment
based on domain requirements. To speed up the learning ability of the networks
and for faster convergence, we have employed adaptive activation functions for
the mapping of the neural networks. The proposed approach is applied for
solving three fracture mechanics examples. For all the examples, we observe
that the results obtained using the proposed approach match closely with
results from the literature.

## Acknowledgment

This work was supported in part by the Carl Zeiss Foundation (project
“Functionalisation of smart materials under multi-field requirements for
transport infrastructure”) and ERC Starting Grant no: 802205.

## References

  * [1] A. Griffith.  The Phenomena of Rupture and Flow in Solids.  Philosophical Transactions of the Royal Society of London, 221(Series A):163–198, 1921.
  * [2] H. Amor, J.-J. Marigo, and C. Maurini.  Regularized formulation of the variational brittle fracture with unilateral contact: Numerical experiments.  Journal of the Mechanics and Physics of Solids, 57(8):1209–1229, 2009.
  * [3] M. J. Borden, T. J. Hughes, C. M. Landis, and C. V. Verhoosel.  A higher-order phase-field model for brittle fracture: Formulation and analysis within the isogeometric analysis framework.  Computer Methods in Applied Mechanics and Engineering, 273:100–118, may 2014.
  * [4] M. J. Borden, T. J. Hughes, C. M. Landis, and C. V. Verhoosel.  A higher-order phase-field model for brittle fracture: Formulation and analysis within the isogeometric analysis framework.  Computer Methods in Applied Mechanics and Engineering, 273:100–118, 2014.
  * [5] M. J. Borden, C. V. Verhoosel, M. A. Scott, T. J. Hughes, and C. M. Landis.  A phase-field description of dynamic brittle fracture.  Computer Methods in Applied Mechanics and Engineering, 217-220:77–95, apr 2012.
  * [6] B. Bourdin, G. Francfort, and J.-J. Marigo.  Numerical experiments in revisited brittle fracture.  Journal of the Mechanics and Physics of Solids, 48(4):797–826, apr 2000.
  * [7] B. Bourdin, C. J. Larsen, and C. L. Richardson.  A time-discrete model for dynamic fracture based on crack regularization.  International journal of fracture, 168(2):133–143, 2011.
  * [8] A. Chakraborty, C. Anitescu, X. Zhuang, and T. Rabczuk.  Domain adaptation based transfer learning approach for solving pdes on complex geometries.  Engineering with Computers, pages 1–20, 2022.
  * [9] A. Chakraborty, T. Wick, X. Zhuang, and T. Rabczuk.  Multigoal-oriented dual-weighted-residual error estimation using deep neural networks.  arXiv e-prints, pages arXiv–2112, 2021.
  * [10] X. Glorot and Y. Bengio.  Understanding the difficulty of training deep feedforward neural networks.  In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
  * [11] S. Goswami, C. Anitescu, S. Chakraborty, and T. Rabczuk.  Transfer learning enhanced physics informed neural network for phase-field modeling of fracture.  Theoretical and Applied Fracture Mechanics, 106:102447, 2020.
  * [12] S. Goswami, C. Anitescu, and T. Rabczuk.  Adaptive phase field analysis with dual hierarchical meshes for brittle fracture.  Engineering Fracture Mechanics, 218:106608, 2019.
  * [13] S. Goswami, C. Anitescu, and T. Rabczuk.  Adaptive fourth-order phase field analysis for brittle fracture.  Computer Methods in Applied Mechanics and Engineering, 361:112808, 2020.
  * [14] S. Goswami, C. Anitescu, and T. Rabczuk.  Adaptive fourth-order phase field analysis using deep energy minimization.  Theoretical and Applied Fracture Mechanics, 107:102527, 2020.
  * [15] A. D. Jagtap and G. E. Karniadakis.  Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations.  Communications in Computational Physics, 28(5):2002–2041, 2020\.
  * [16] A. D. Jagtap, K. Kawaguchi, and G. E. Karniadakis.  Adaptive activation functions accelerate convergence in deep and physics-informed neural networks.  Journal of Computational Physics, 404:109136, 2020.
  * [17] E. Kharazmi, Z. Zhang, and G. E. Karniadakis.  hp-vpinns: Variational physics-informed neural networks with domain decomposition.  Computer Methods in Applied Mechanics and Engineering, 374:113547, 2021.
  * [18] V. Kunc and J. Kléma.  On transformative adaptive activation functions in neural networks for gene expression inference.  PloS one, 16(1):e0243915, 2021.
  * [19] I. E. Lagaris, A. Likas, and D. I. Fotiadis.  Artificial neural networks for solving ordinary and partial differential equations.  IEEE transactions on neural networks, 9(5):987–1000, 1998.
  * [20] H. Lee and I. S. Kang.  Neural algorithm for solving differential equations.  Journal of Computational Physics, 91(1):110–131, 1990.
  * [21] L. McClenny and U. Braga-Neto.  Self-adaptive physics-informed neural networks using a soft attention mechanism.  arXiv preprint arXiv:2009.04544, 2020.
  * [22] C. Miehe, M. Hofacker, and F. Welschinger.  A phase field model for rate-independent crack propagation: Robust algorithmic implementation based on operator splits.  Computer Methods in Applied Mechanics and Engineering, 199(45-48):2765–2778, 2010.
  * [23] C. Miehe, F. Welschinger, and M. Hofacker.  Thermodynamically consistent phase-field models of fracture: Variational principles and multi-field fe implementations.  International journal for numerical methods in engineering, 83(10):1273–1311, 2010.
  * [24] M. Raissi, P. Perdikaris, and G. E. Karniadakis.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  Journal of Computational Physics, 378:686–707, 2019.
  * [25] E. Samaniego, C. Anitescu, S. Goswami, V. M. Nguyen-Thanh, H. Guo, K. Hamdia, X. Zhuang, and T. Rabczuk.  An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications.  Computer Methods in Applied Mechanics and Engineering, 362:112790, 2020.
  * [26] D. Schillinger, M. J. Borden, and H. K. Stolarski.  Isogeometric collocation for phase-field fracture models.  Computer Methods in Applied Mechanics and Engineering, 284:583–610, 2015.
  * [27] S. Zhou, T. Rabczuk, and X. Zhuang.  Phase field modeling of quasi-static and dynamic crack propagation: Comsol implementation and case studies.  Advances in Engineering Software, 122:31–49, 2018.
  * [28] S. Zhou, X. Zhuang, H. Zhu, and T. Rabczuk.  Phase field modelling of crack propagation, branching and coalescence in rocks.  Theoretical and Applied Fracture Mechanics, 96:174–192, 2018.
  * [29] O. C. Zienkiewicz and J. Z. Zhu.  The superconvergent patch recovery and a posteriori error estimates. part 2: Error estimates and adaptivity.  International Journal for Numerical Methods in Engineering, 33(7):1365–1382, 1992.