# Exploring Key Point Analysis with
Pairwise Generation and Graph Partitioning

Xiao Li⋄, Yong Jiang†$\ast$, Shen Huang†, Pengjun Xie†
Gong Cheng⋄, Fei Huang†
⋄State Key Laboratory for Novel Software Technology  Yong Jiang and Gong Cheng
are the corresponding authors. Nanjing University
†Institute for Intelligent Computing  Alibaba Group
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Key Point Analysis (KPA), the summarization of multiple arguments into a
concise collection of key points, continues to be a significant and unresolved
issue within the field of argument mining. Existing models adapt a two-stage
pipeline of clustering arguments or generating key points for argument
clusters. This approach rely on semantic similarity instead of measuring the
existence of shared key points among arguments. Additionally, it only models
the intra-cluster relationship among arguments, disregarding the inter-cluster
relationship between arguments that do not share key points. To address these
limitations, we propose a novel approach for KPA with pairwise generation and
graph partitioning. Our objective is to train a generative model that can
simultaneously provide a score indicating the presence of shared key point
between a pair of arguments and generate the shared key point. Subsequently,
to map generated redundant key points to a concise set of key points, we
proceed to construct an arguments graph by considering the arguments as
vertices, the generated key points as edges, and the scores as edge weights.
We then propose a graph partitioning algorithm to partition all arguments
sharing the same key points to the same subgraph. Notably, our experimental
findings demonstrate that our proposed model surpasses previous models when
evaluated on both the ArgKP and QAM datasets.

Exploring Key Point Analysis with
Pairwise Generation and Graph Partitioning

##  1 Introduction

Analyzing a large number of arguments and making decisions is a long standing
issue present in real life scenarios such as product review analysis Bar-Haim
et al. (2021); Cattan et al. (2023), opinion surveys Bar-Haim et al. (2020a),
etc. However, manually reading through a vast amount of arguments is not a
feasible solution. In this context, key point analysis (KPA, Bar-Haim et al.,
2020a) has been proposed. KPA involves distilling a multitude of arguments
into a concise set of key points, followed by a quantitative evaluation,
typically by calculating the frequency of each key point related to arguments.
An example is shown in Figure 1.

Figure 1: An example in the ArgKP dataset. Only a subset of arguments and key
points are shown due to space limitation. Our goal is to generate key points
(right side) within a specific topic based on the provided arguments (left
side). A single argument may correspond to multiple key points.

##### Existing Methods and Limitations.

The generation of key points from arguments represents the most pivotal and
challenging aspect of the KPA task. Initially, a straightforward approach
Cattan et al. (2023) involves selecting a few short arguments as key points
and then determining whether other arguments match these key points.
Subsequently, advanced model Li et al. (2023) divides the KPA task into two
stages. Firstly, a semantic feature based clustering model is utilized to
cluster arguments into multiple clusters. Then, a generation model is employed
to generate corresponding key points for arguments in each cluster by
concatenating the arguments. However, these existing methods possess certain
limitations. Cattan et al., 2023 posits that the mere presence of appropriate
key points within arguments is insufficient. Li et al., 2023’s approach is
based on clustering arguments using their semantic features and then positing
that arguments within a cluster share the same key point. However, this
approach does not fully align with the objectives of the KPA task. It is
possible for arguments that are not semantically similar to still share a key
point, as an argument can correspond to more than one key point. Moreover, Li
et al., 2023’s approach only models the relationships between arguments that
share a key point (i.e., intra-cluster arguments) and neglects the
relationships between arguments that do not share a key point (i.e., inter-
cluster arguments).

##### Our Approach.

In order to address the deficiencies mentioned above, we present an approach
which trains a generative model to simultaneously utilize the relationships
between intra-cluster and inter-cluster arguments. This model generates a key
point for a pair of inter-cluster arguments and provided a score indicating
whether there exists shared key points between the pair of arguments. During
the prediction phase, we utilize the score provided by the generative model,
which indicates whether a pair of arguments is inter-cluster arguments, rather
than the semantic similarity of the arguments, to partition the arguments into
multiple clusters. This score is provided by a fine-tuned generative model,
which is aligned with the data distribution within the domain, rather than
employing a clustering model that is unrelated to the domain. We then
construct an argument graph and formulate the relationships among arguments as
a graph-based problem. Specifically, we represent arguments as vertices in the
graph, the shared key point between two arguments as edges, and the scores as
edge weights, resulting in a weighted argument graph. We introduce an
iterative graph partitioning algorithm, which generates several subgraphs,
each representing a collection of arguments that share the same or similar key
point. From these subgraphs, we select a representative key point for further
analysis and utilization.

To summarize, our contributions include

  * •

proposing a pairwise generation approach that simultaneously utilizes the
information between intra-cluster arguments and inter-cluster arguments and
providing a score to indicate the presence of the shared key point between a
pair of arguments, and

  * •

introducing a weighted graph partition algorithm designed specifically for the
KPA task. Our algorithm evaluates the connections between arguments by the
scores provided by the generative model, as semantic similarity may not fully
align with the objectives of the KPA task.

##### Outline.

We elaborate our approach in Section 2, present experiments in Section 3,
discuss related work in Section 4, and conclude in Section 5.

##### Code.

Our code and data are available on GitHub: https://github.com/Alibaba-NLP/Key-
Point-Analysis.

##  2 Approach

On the KPA task, the input consists of a set of arguments
$\mathbb{A}=\\{(\texttt{arg}_{1},\texttt{stance}_{1}),(\texttt{arg}_{2},\texttt{stance}_{2}),\cdots\\}$
under a given topic $T$, where $\texttt{arg}_{i}$ is the argument text and
$\texttt{stance}_{i}$ is the stance for $\texttt{arg}_{i}$. The stance (“pro”
or “con”) indicate the stance of argument towards the topic. The ideal outcome
is a concise set of key points
$\mathbb{K}=\\{\texttt{kp}_{1},\texttt{kp}_{2},\cdots\\}$, where each key
point in $\mathbb{K}$ corresponds to one or more arguments in $\mathbb{A}$.
That is, our objective is to learn a surjective function
$f:\mathbb{A},T\rightarrow\mathbb{K}$. Afterwards, the importance of each key
point can be expressed by calculating the percentage of arguments
corresponding to each key point.

Figure 2: Overview of our approach. Each argument possesses a stance
representing its polarity (“pro” or “con”). For the sake of brevity, it has
been omitted in the figure.

Figure 2 provides an overview of our proposed method, which comprises two
principal components. The initial component is dedicated to the generation of
key points, as depicted at the bottom of Figure 2. During this phase, we
process the given topic $T$ and pairs of arguments
$\mathbb{A}\times\mathbb{A}$ to ascertain if each pair shares a key point and,
if so, generate the corresponding key point. The subsequent component entails
graph partitioning, illustrated at the top of Figure 2. In this stage, we
construct a graph of arguments, key points, and their associated scores. We
then apply our specially developed graph partitioning algorithm to divide the
graph into multiple subgraphs, allowing for the possibility of the same
argument being present in more than one subgraph, as a single argument may
relate to multiple key points. Finally, we select a representative key point
from each subgraph to form the final concise set of key points $\mathbb{K}$.
Notably, unlike previous pipeline approaches, we transmit the scores generated
by the first stage to the second stage, avoiding issues such as feature
propagation interruption and error accumulation that may arise in multi-stage
pipelines.

###  2.1 Key Point Generation

Inspired by contrastive learning Khosla et al. (2020); Gao et al. (2021), our
approach takes a fine-grained perspective by pairing arguments and determining
whether a pair of arguments shares the same key point. We posit that it is
equally important to model the relationships between inter-cluster arguments
that possess different key points and intra-cluster arguments that share a key
point. The previous models either mapped each argument to a single key point
Alshomary et al. (2021); Kapadnis et al. (2021) or clustered arguments
together and concatenated multiple arguments within a cluster to generate a
key point Li et al. (2023).

##### Construction of Training Data.

We pack a pair of arguments into an input text

$\texttt{input}_{ij}=\texttt{topic}\ |\ \texttt{stance}_{i}\ \texttt{arg}_{i}\
|\ \texttt{stance}_{j}\ \texttt{arg}_{j},$

and prefix the output text with a special word to indicate whether the input
pair of arguments are intra-cluster arguments or inter-cluster arguments,
which is

$\texttt{output}_{ij}=\begin{cases}\text{Yes.}\
\\{\texttt{kp}_{ij}\\},&\text{for intra-cluster arguments,}\\\
\text{No.},&\text{for inter-cluster arguments.}\end{cases}$

We construct training data by pairing arguments, disregarding the order
between a pair of arguments. That is, we consider $\texttt{input}_{ij}$ while
discarding $\texttt{input}_{ji}$ (where $i<j$). We determine whether a pair of
arguments are intra-cluster arguments or inter-cluster arguments by assessing
if the corresponding key points of the arguments are identical. To avoid
overfitting, we limit the occurrence of each argument as an intra-cluster
argument in the training data to no more than five times.

input A |  We should adopt atheism. | positive. if we adopt atheism then maybe people will start believing in the scientific community again. | positive. we should adopt atheism because science can explain how we got here without needing a god to explain it.
---|---
output A |  Yes. Science can adequately explain the Universe
input B |  We should ban the use of child actors. | positive. child actors do not get to live a proper young persons life and experience childhood play, education and social interaction, so should be banned | positive. the use of child actors should be banned as children are not capable of making important decisions and some do not have trustworthy parents to make rationale decisions.
output B |  No.

Table 1: Training examples. The two arguments in example A share a key point,
whereas the two arguments in example B do not.

To facilitate a better understanding of the construction of the training data,
we present two examples in Table 1.

##### Generation Model.

Subsequently, we feed $\texttt{input}_{ij}$ to the generative model 111In our
paper, we exemplify our approach by employing a model based on the encoder-
decoder architecture. However, our approach can also be applied to decoder-
only models.,

$\mathbf{h}_{1},\mathbf{h}_{2},\cdots,\mathbf{h}_{n}=\mathtt{Decoder}(\mathtt{Encoder}(\texttt{input}_{ij})),$

where $\mathbf{h}_{k}$ denotes the representation obtained from the
$\mathtt{Decoder}$ during the $k$-th decoding step and $n$ is the decoder
output length. During the decoding phase, a linear transformation
$\mathtt{LM\\_Head}$ is employed to map the representations outputted by the
$\mathtt{Decoder}$ to the probability distribution for the tokens in the
vocabulary at step $k$, denoted as

$S_{k}=\mathtt{LM\\_Head}(\mathbf{h}_{k}).$

The $k$-th token is decoded by

$\texttt{token}_{k}=\mathtt{vocab}[\mathtt{argmax}(S_{k})],$

where $\mathtt{vocab}$ is the vocabulary.

We extract scores for token “Yes” and “No” from $S_{1}$ (i.e.,
$S_{1}[\text{Yes}]$ and $S_{1}[\text{No}]$, respesctively) and obtain the
score indicating whether $\texttt{arg}_{i}$ and $\texttt{arg}_{j}$ sharing the
same key point by

$score_{ij}=\frac{\mathtt{exp}(S_{1}[\text{Yes}])}{\mathtt{exp}(S_{1}[\text{Yes}])+\mathtt{exp}(S_{1}[\text{No}])}.$
(1)

The loss during the training process is

$\mathcal{L}=-\frac{1}{n}\sum_{1\leq k\leq
n}\log\frac{\mathtt{exp}(S_{k}[\texttt{output}_{ij}[k]])}{\sum_{t\in\texttt{vocab}}\mathtt{exp}(S_{k}[t])},$

where $\texttt{output}_{ij}[k]$ is the $k$-th token in $\texttt{output}_{ij}$.

By employing this approach, we have integrated the binary classification task
of determining the presence of shared key points and the task of generating
key points into a single generative model, thereby mitigating potential
information loss that may arise from coupling two different models.

###  2.2 Graph Partitioning

After the generation of key points, it is necessary to aggregate all intra-
cluster arguments and determine the shared key point. We transform this issue
into a graph partitioning problem, such that each partition yields a subgraph
representing a cluster of arguments that share key points. Previous approaches
Li et al. (2023) relied on semantic similarity-based clustering methods.
However, it is important to recognize that two arguments sharing the same key
point may not always exhibit high semantic similarity, since an argument can
encompass multiple key points. An alternative direct approach involves
clustering the key points directly. However, the semantic gap between the
generative models and the clustering models may introduce performance drop in
this approach. We have mitigated this issue by transferring $score_{ij}$
between the two stages. Specifically, in Equation 1, we calculate the score
$score_{ij}$ to quantify the presence of a shared key point between argument
$\mathtt{kp}_{i}$ and argument $\mathtt{kp}_{j}$. The score $score_{ij}$ is
then used to guide the partitioning process.

##### Argument Graph Construction.

We employ arguments $\mathbb{A}$ as vertices and each edge possesses two
attributes, an edge weight

$score\in\mathbb{S}=\\{score_{i,j}\lvert 1\leq
i,j\leq\lvert\mathbb{A}\rvert,i\neq j\\},$

which indicates the probability of the presence of a shared key point, and the
generated shared key point

$\texttt{kp}\in\hat{\mathbb{K}}_{\text{Gen}}=\\{\texttt{kp}_{ij}\lvert 1\leq
i,j\leq\lvert\mathbb{A}\rvert,i<j\\}.$

We have removed the edges for which the model outputs “No”, corresponding to
the edges between argument vertices that do not share a key point.

In doing so, we construct a weighted argument graph

$G=\langle\mathbb{A},(\mathbb{S},\hat{\mathbb{K}}_{\text{Gen}})\rangle.$

##### Partitioning by Local Search.

We employ local search algorithm to partition argument graph. We use the BAAI-
bge-large Xiao et al. (2023) model to obtain representations of arguments, and
then apply k-means clustering Lloyd (1982) to cluster the arguments, which
serves as the initial partitioning for our graph. We then obtain the initial
subgraphs $\mathbb{G}_{\text{subgraph}}=\\{g_{1},g_{2},\cdots,g_{s}\\}$, where
$s$ is a hyperparameter. Subsequently, we iteratively move vertices across the
current subgraphs by optimizing a cost function, until no improvement of cost
is possible or the limit of iteration steps denoted by $l$ is reached. We
finally obtain subgraphs $\mathbb{G}^{\prime}_{\text{subgraph}}$ and form the
concise set of key points $\mathbb{K}_{\text{Gen}}$ from each subgraph by
choosing the key points with the highest edge weights.

##### Cost Function.

We define the weight $\mathtt{wt}(g)$ of a subgraph $g$ as the average weight
of all edges in $g$. For a randomly selected subgraph and within it, a
randomly selected vertex arg, we calculate the change in the graph weight
resulting from the movement of vertex arg from the subgraph $g_{\text{out}}$
to another subgraph $g$ by our cost function

$\displaystyle\mathtt{cost}$ $\displaystyle(g,g_{\text{out}},\texttt{arg})=$
$\displaystyle\mathtt{wt}(g_{\text{out}}\setminus\\{\texttt{arg}\\})-\mathtt{wt}(g_{\text{out}})+$
$\displaystyle\mathtt{wt}(g\cup\\{\texttt{arg}\\})-\mathtt{wt}(g).$

We then search for a target subgraph $g_{\text{in}}$ by

$\displaystyle g_{\text{in}}=\mathop{\mathtt{arg\
max}}_{g\in\mathbb{G}_{\text{subgraph}}\setminus\\{g_{\text{out}}\\}}\mathtt{cost}(g,g_{\text{out}},\texttt{arg}),$

which means that moving the selected vertex arg from $g_{\text{out}}$ to
$g_{\text{in}}$ would result in the maximum change in graph weight and the
selected vertex will only be moved when the weight change is positive.

##### Soft Partition.

Furthermore, since an argument may encompass multiple key points, it is
permissible to allow an argument to belong to multiple subgraphs
simultaneously, which we refer to as soft partition. We set a threshold $h$,
such that if the removal of the selected argument from its original subgraph
would cause the weight of the subgraph to decrease beyond the threshold $h$,
we will retain the argument within the original subgraph. This means that the
selected argument can contribute a weight gain greater than $h$ in the
original subgraph.

It is worth noting that we have not imposed any restrictions on the
connectivity of subgraphs in our algorithm. This is because, given a specified
number of subgraphs, we cannot guarantee that all the resulting subgraphs will
be connected (as the number of connected components in $G$ may be less than
the number of subgraphs). However, it is important to note that our
optimization objective (i.e., graph weight) tends to favor connected subgraphs
or subgraphs with higher connectivity.

##  3 Experiments

| ArgKP | QAM
---|---|---
Model |  $\mathtt{Rouge}$-1 |  $\mathtt{Rouge}$-2 | sP | sR | sF1 |  $\mathtt{Rouge}$-1 |  $\mathtt{Rouge}$-2 | sP | sR | sF1
Previous SOTA Approaches
GBS | 19.60 | 3.40 | 53.00 | 52.00 | 52.00 | - | - | - | - | -
Enigma${}_{\text{PEGASUS}}$ | 20.00 | 4.80 | 58.00 | 57.00 | 57.00 | 32.61 | 7.17 | 46.96 | 44.43 | 45.51
SKPM${}_{\text{Flan-T5-base}}$ | 30.30 | 8.90 | 59.00 | 58.00 | 59.00 | - | - | - | - | -
SKPM${}_{\text{Flan-T5-large}}$ | 31.40 | 9.10 | 57.00 | 62.00 | 60.00 | - | - | - | - | -
Ours
w/ Flan-T5-base | 37.38 | 11.39 | 61.29 | 58.08 | 59.59 | 30.31 | 7.44 | 48.62 | 43.81 | 45.96
w/ Flan-T5-large | 50.91 | 21.23 | 62.57 | 60.26 | 61.34 | 30.44 | 8.02 | 49.38 | 45.27 | 47.11

Table 2: Comparison with previous state-of-the-art approaches. The best
results are in bold and the second best results are underlined. The dash
signifies that we did not obtain the results due to issues such as code
availability. The results of previous state-of-the-art approaches on ArgKP are
taken from Li et al., 2023.

###  3.1 Datasets

We conducted experiments on ArgKP Friedman et al. (2021) and QAM Guo et al.
(2023). ArgKP is a KPA dataset annotated by crowd sourcing, with
training/validation/test sets consisting of 24/4/3 topics, 5583/932/724
arguments, and corresponding 207/36/33 key points. Each argument and key point
possess a stance attribute, either “pro” (i.e., positive) or “con” (i.e.,
negative), indicating whether they support the current topic. The QAM dataset
is originally designed for extracting quadruples $\langle$key point, stance,
argument, evidence type$\rangle$ from documents with a given topic. QAM is
annotated through crowdsourcing, and we have transformed the QAM dataset into
the format required for the KPA task. The training/validation/test sets
consists 96/52/53 topics, 6635/808/948 arguments, and corresponding
2366/314/337 key points.

###  3.2 Evaluation Metrics

We employed $\mathtt{Rouge}$ Lin (2004) and soft-Precision/Recall/F1 as
metrics. Following Li et al., 2023, we obtained $\mathtt{Rouge}$ scores by
comparing the concatenation of generated key points and the concatenation of
reference key points. We reported the scores of $\mathtt{Rouge}$-1 and
$\mathtt{Rouge}$-2. For soft-Precision/Recall/F1 (denoted by sP, sR and sF1,
respectively), we employed $\mathtt{BLEURT}$ Sellam et al. (2020) to measure
the similarity between individual generated key point and the reference key
point. The sP, sR and sF1 to compare the similarity between the sets of key
points were given by

sP
$\displaystyle=\frac{1}{\lvert\mathbb{K}_{\text{Gen}}\rvert}\mathop{\sum}_{\texttt{kp}_{\text{Gen}}\in\mathbb{K}_{\text{Gen}}}\max_{\texttt{kp}\in\mathbb{K}}\mathtt{BLEURT}(\texttt{kp},\texttt{kp}_{\text{Gen}}),$
sR
$\displaystyle=\frac{1}{\lvert\mathbb{K}\rvert}\sum_{\texttt{kp}\in\mathbb{K}}\max_{\texttt{kp}_{\text{Gen}}\in\mathbb{K}_{\text{Gen}}}\mathtt{BLEURT}(\texttt{kp},\texttt{kp}_{\text{Gen}})\
\text{and}$ sF1
$\displaystyle=\frac{2\times\textbf{sP}\times\textbf{sR}}{\textbf{sP}+\textbf{sR}}.$

###  3.3 Implement Details

We conducted experiments on NVIDIA A100 (80GB) GPUs. Our implementations were
based on PyTorch 2.0.1 and HuggingFace Transformers 4.30.2. We set
$\textit{learning rate}=1e-5$ selected from $\\{1e-5,3e-5,6e-5\\}$,
$\textit{batch size}=64$ selected from $\\{32,64,128,256\\}$, $\textit{random
seed}=42$, $\textit{maximum sequence length}=512$, $\textit{epoch}=5$. We set
$h=0.008$ for ArgKP, $h=0.006$ for QAM and $l=200$ for ArgKP, $l=80$ for QAM.
In order to ensure a fair comparison with previous work Li et al. (2023), we
set the number of selected key points, which is the number of subgraphs $s$,
to be equal to the number of reference key point set.

###  3.4 Previous State-of-the-Art Approaches

We compared with three state-of-the-art models: graph based summarization
(GBS, Alshomary et al., 2021), Enigma Kapadnis et al. (2021) and supervised
key point modelling (SKPM, Li et al., 2023). The GBS model was a
representative of the extractive key point generation models. It employed
arguments to establish a graph, where each argument vertex was examined to
determine their suitability as a key point. The PageRank Page et al. (1999)
algorithm was then utilized to choose the key points. Enigma generated a key
point for each argument based on PEGASUS Zhang et al. (2020) and selected the
highest-scoring key point using the precision of $\mathtt{Rouge}$-1 compared
with reference key point set. SKPM was a two-stage model that first clusterd
arguments and then generated key points for arguments within each cluster.

###  3.5 Comparison with Previous State-of-the-Art Approaches

In comparison with the existing methods as shown in Table 2, our approach
demonstrated advantages over the previous SOTA approaches. Specifically, on
the ArgKP dataset, we observed improvements of 19.51–31.31 points in
$\mathtt{Rouge}$-1 and 1.34–9.34 points in sF1. On the QAM dataset, our
approach yielded an improvement of 1.60 points in sF1. The results
demonstrated the effectiveness of our method, which were analyzed in detail in
the subsequent sections.

###  3.6 Effectiveness of Key Point Generation

We conducted several experiments to evaluate the effectiveness of our
generative model in identifying and generating the shared key point between a
pair of arguments. These experiments were structured around two research sub-
questions. SQ1 (Shared Key Point Detection) focused on the ability of our
model to determine the presence of a shared key point, which was essentially a
binary classification task. SQ2 (Key Point Generation) assessed the model’s
capability to generate high-quality key points that effectively covered the
reference key point set. In tackling these research sub-questions, we
prioritized the recall metric because our subsequent step involves selecting a
concise subset from an extensive set of key points, as explained in Section
3.7).

####  3.6.1 SQ1: Shared Key Point Detection

We applied a representative model RoBERTa-large Liu et al. (2019) to classify
whether two arguments shared a key point and compared its performance with
that of our generative model. In our generative model, if a key point was
generated for a pair of arguments, the output was classified as positive
(i.e., the output began with “Yes”). Conversely, if no key point was
generated, the output was classified as negative (i.e., the output began with
“No”). For the RoBERTa-large model, we utilized the CLS vector, followed by a
linear layer that produced two scores, one for positive and one for negative,
to serve as the classifier.

Figure 3: Shared key point detection on ArgKP.

We evaluated the recall and accuracy of two models in a binary classification
task aimed at identifying whether a pair of arguments shared a key point on
ArgKP. As indicated in Figure 3, our approach outperformed by 10.56 points in
recall, the metric of greater significance in our scenario. Generating key
points may also help in determining whether there are shared key points among
arguments. With pairwise generation, despite the Flan-T5-large being an
encoder-decoder model primarily intended for generative tasks, it still
delivered satisfactory performance in the classification task.

####  3.6.2 SQ2: Key Point Generation

In this section, we designed an experiment to verify that the relationship
between a pair of intra-cluster arguments with shared key points and inter-
cluster arguments without shared key points is beneficial for key point
generation. We removed the pairs of arguments that did not share key points
from our training data as ablation experiment, which is the result of the
“Ours w/o inter-cluster argument pairs” row in Figure 4. To verify the
effectiveness of pairwise generation, we trained a Flan-T5-large model with
the concatenation of the topic, stance and a single argument (i.e.,
$\texttt{input}=\texttt{topic}\ |\ \texttt{stance}\ |\ \texttt{arg}$) as the
input, and an output of concatenated key points implied by the single
argument. We denoted this approach as “One2KP”. Here, we also focused on the
recall metric (i.e., sR), which reflected the coverage of the reference key
point set during the key point generation stage.

| ArgKP | QAM
---|---|---
Model |  $\mathtt{Rouge}$-1 |  $\mathtt{Rouge}$-2 | sP | sR | sF1 |  $\mathtt{Rouge}$-1 |  $\mathtt{Rouge}$-2 | sP | sR | sF1
Clustering Methods (on arguments)
k-means${}_{\text{RoBERTa-large}}$ | 50.70 | 21.01 | 59.38 | 59.49 | 59.34 | 29.30 | 7.26 | 48.30 | 44.67 | 46.28
k-means${}_{\text{BAAI-bge-large}}$ | 49.01 | 18.64 | 60.94 | 60.08 | 60.47 | 30.13 | 7.85 | 47.75 | 44.99 | 46.21
Clustering Methods (on key points)
k-means${}_{\text{RoBERTa-large}}$ | 52.71 | 20.16 | 61.30 | 60.21 | 60.74 | 28.28 | 6.77 | 48.35 | 43.48 | 45.67
k-means${}_{\text{BAAI-bge-large}}$ | 51.75 | 19.57 | 61.42 | 58.80 | 59.99 | 29.40 | 7.50 | 48.79 | 44.70 | 46.53
Graph Partitioning Methods
Greedy Modularity Maximization | 40.85 | 14.75 | 56.15 | 54.04 | 55.07 | 18.59 | 4.79 | 39.34 | 33.85 | 36.19
Louvain Community Detection | 43.01 | 18.03 | 56.81 | 56.65 | 56.69 | 17.96 | 4.68 | 38.28 | 32.97 | 35.23
Asynchronous Label Propagation | 17.32 | 5.54 | 63.76 | 53.31 | 58.01 | 14.71 | 4.41 | 54.40 | 40.62 | 46.04
Ours | 50.91 | 21.23 | 62.57 | 60.26 | 61.34 | 30.44 | 8.02 | 49.38 | 45.27 | 47.11

Table 3: Comparison with clustering and graph partitioning methods with
Flan-T5-large. The best results are in bold and the second best results are
underlined. Figure 4: Comparison of different models on key point generation
on ArgKP. All the models were based on Flan-T5-large.

From Figure 4, it can be seen that our method had an advantage over the One2KP
method, leading by 9.21 points on the sR metric. After separating inter-
cluster argument pairs from the training set, a decrease of 5.84 point on the
sR metric was observed. Experiments in Figure 4 validated that generating key
points from pairwise arguments was effective. It was not only necessary to
consider the relationships among intra-cluster arguments, but also inter-
cluster arguments.

Additionally, addressing two related tasks simultaneously within the same
model did not introduce a drop on the model performance. In our task, as seen
in Figure 3 and Figure 4, the carefully designed tasks and the use of pairwise
generations method brought about a gain in model performance. This allowed our
model to not only exhibit superior performance in key point generation but
also achieve satisfactory discriminative ability in intra-cluster arguments
and inter-cluster arguments classification.

###  3.7 Effectiveness of Graph Partitioning

In this section, we aimed to validate the effectiveness of our graph
partitioning method. We compared our graph partitioning method with existing
clustering methods and graph partitioning methods in Section 3.7.1.
Additionally, we investigated the influence of the number of iterations of our
graph partitioning method on the performance in Section 3.7.2.

####  3.7.1 Comparison with Clustering and Graph Partitioning Methods

We substituted our graph partitioning algorithm with either clustering methods
or graph partitioning methods. Our key point generation model, exemplified by
the Flan-T5-large model, produced key points for these methods and provided
scores to determine if two arguments shared a key point for graph partitioning
methods. The clustering algorithms we compared were of two types, one
clustered arguments to obtain argument subgraphs and then selected key points
with highest edge weights, and the other directly clustered key points and
then selected the key point at the center of the cluster. We utilized well-
established clustering algorithms, namely k-means Lloyd (1982) , for our
analysis. Two models were utilized to derive representations for arguments: 1)
the renowned RoBERTa-large Liu et al. (2019) and 2) BAAI-bge-large Xiao et al.
(2023), which is the state-of-the-art model in sentence representation.
Furthermore, we employed three well-known graph partitioning methods,
including

  * •

Greedy Modularity Maximization Clauset et al. (2004), which is a graph
partitioning algorithm that utilizes a greedy strategy to maximize modularity
and identify community structures,

  * •

Louvain Community Detection Blondel et al. (2008), which is a heuristic method
based on modularity optimization by iteratively merging similar vertices to
uncover community structures, and

  * •

Asynchronous Label Propagation Raghavan et al. (2007), which relies on label
propagation iterations to divide the graph into communities by updating vertex
labels.

From Table 3, our model exhibited superior performance compared to the three
sets of baseline models on $\mathtt{Rouge}$-2, sR and sF1 metrics.
Specifically, on the ArgKP dataset, our model exhibited a lead of 0.60–6.27
points in sF1. On the QAM dataset, our model demonstrated a lead of 0.58–11.88
points in sF1.

Our approach amalgamated the strengths of both by identifying shared key
points through our generative model, taking into account semantic information,
while simultaneously utilizing graph structure information via the graph
partitioning method, resulting in improved outcomes.

####  3.7.2 Analysis of Iterations Steps ($l$)

#steps |  $\mathtt{Rouge}$-1 |  $\mathtt{Rouge}$-2 | sP | sR | sF1
---|---|---|---|---|---
0 | 49.01 | 18.64 | 60.94 | 60.08 | 60.47
50 | 51.46 | 20.55 | 61.38 | 60.63 | 60.96
100 | 47.53 | 19.77 | 60.92 | 59.25 | 60.03
150 | 52.13 | 22.39 | 61.22 | 60.52 | 60.85
200 | 50.91 | 21.23 | 62.57 | 60.26 | 61.34
250 | 49.23 | 19.45 | 62.32 | 59.93 | 61.09

Table 4: Analysis of iterations steps ($l$).

We conducted an analysis on the impact of the number of iterations steps in
the graph partition algorithm. When the number of steps equaled 0, it
represented a ablation experiment, indicating the exclusion of any execution
of the graph partition algorithm. As shown in Table 4, it could be seen that,
excluding the subtle fluctuations in the data, the overall trend of sF1 was
first an increase and then a stabilization. Additionally, it could be seen
that as the algorithm iterations progress, sR did not change much overall, and
our graph partitioning algorithm improved the overall performance by improving
sP.

##  4 Related Work

###  4.1 Key Point Analysis

Unlike multi-document summarization Mani and Bloedorn (1997); Xiao et al.
(2022), KPA Bar-Haim et al. (2020a) requires the production of a concise
collection of key points rather than a cohesive paragraph. Furthermore, KPA
entails a quantitative analysis of these key points, involving quantifying
their prevalence within the arguments.

Existing methods for key points generation in KPA rely on extractive
summarization and abstractive summarization techniques. Extractive
summarization methods Bar-Haim et al. (2020b, 2021); Alshomary et al. (2021)
involve selecting short and representative arguments as key points. However,
extractive summarization approaches are limited by the quality of the
arguments and do not ensure the presence of an optimal key point within the
arguments. On the other hand, abstractive summarization methods utilize
generative models to produce key points for the arguments. Kapadnis et al.,
2021 generates key points for each argument individually and then select a
collection of key points based on the Rouge Lin (2004) scores. Li et al., 2023
first employs clustering methods to group semantically similar arguments into
the same cluster, and then concatenates the arguments within the cluster to
generate key points.

In contrast, our approach simultaneously takes into account the relationships
between arguments with shared key points and those without shared key points.
Additionally, by replacing semantic similarity scores with scores for the
presence of shared key points between arguments, we address the shortcomings
of existing models.

###  4.2 Graph Partitioning

Unlike clustering methods which depends on the similarity or distance of
clustering objects, graph partitioning aims to optimize a certain metric of
the resulting subgraphs Buluç et al. (2013), which is the probability that the
arguments within a common subgraph share a common key point in the KPA
scenario. The algorithms Kernighan and Lin (1970); Krauthgamer et al. (2009);
Margo and Seltzer (2015) based on optimizing the minimum vertex/edge cuts has
been extensively utilized. Additionally, algorithms Clauset et al. (2004);
Blondel et al. (2008); Dugué and Perez (2015); Traag et al. (2018) have
approached graph partitioning by optimizing the modularity of subgraphs.
Another avenue Raghavan et al. (2007); Cordasco and Gargano (2010) involves
achieving graph partitioning through vertex labels propagation.

In the KPA task, our objective is to maximize the probability that arguments
within a subgraph share the same key point while allowing arguments that
contain more than one key point to exist in multiple subgraphs simultaneously.
Given this requirement, the aforementioned graph partitioning algorithms are
not suitable for the KPA task.

##  5 Conclusion

This paper presents a method that merges pairwise generation with graph
partitioning to efficiently produce key points within arguments. Our
integrated approach models both the relationships between arguments sharing
key points and those that do not, aiding in key point generation. We employ an
iterative algorithm to partition the argument graph and extract representative
key points from subgraphs, yielding a concise set of key points. In the
future, we will continue to explore better ways to utilize generative models
to model the relationships among arguments. For instance, we will investigate
improved methods of outputting scores indicating whether arguments share key
points and develop graph partition algorithms that better align with
generative models. Our approach may provide inspiration for tasks including
clustering tasks based on generative models Zhang et al. (2023); Viswanathan
et al. (2023), set generation tasks Madaan et al. (2022), and so forth.

## Limitations

The input consists of pairwise arguments, which increases the amount of
training data and the computational cost of training. Currently, training time
of Flan-T5-large (780M) on ArgKP is approximately 12 hours, and on QAM it is
about 6 hours. Our model does not fully utilize the graph structure during the
graph partitioning process, and we plan to incorporate more graph structural
information into the model in the future. The datasets currently used for
experiments are related to survey review topics, and there is a lack of
datasets from other fields (such as product reviews in the e-commerce
scenario) in the research community. Future research could explore the
effectiveness on datasets from different domains.

## References

  * Alshomary et al. (2021) Milad Alshomary, Timon Gurcke, Shahbaz Syed, Philipp Heinisch, Maximilian Spliethöver, Philipp Cimiano, Martin Potthast, and Henning Wachsmuth. 2021.  Key point analysis via contrastive learning and extractive argument summarization.  In _Proceedings of the 8th Workshop on Argument Mining_ , pages 184–189, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Bar-Haim et al. (2020a) Roy Bar-Haim, Lilach Eden, Roni Friedman, Yoav Kantor, Dan Lahav, and Noam Slonim. 2020a.  From arguments to key points: Towards automatic argument summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4029–4039, Online. Association for Computational Linguistics.
  * Bar-Haim et al. (2021) Roy Bar-Haim, Lilach Eden, Yoav Kantor, Roni Friedman, and Noam Slonim. 2021.  Every bite is an experience: Key Point Analysis of business reviews.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3376–3386, Online. Association for Computational Linguistics.
  * Bar-Haim et al. (2020b) Roy Bar-Haim, Yoav Kantor, Lilach Eden, Roni Friedman, Dan Lahav, and Noam Slonim. 2020b.  Quantitative argument summarization and beyond: Cross-domain key point analysis.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 39–49, Online. Association for Computational Linguistics.
  * Blondel et al. (2008) Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008.  Fast unfolding of communities in large networks.  _Journal of Statistical Mechanics: Theory and Experiment_ , 2008:P10008.
  * Buluç et al. (2013) Aydın Buluç, Henning Meyerhenke, Ilya Safro, Peter Sanders, and Christian Schulz. 2013.  Recent advances in graph partitioning.  In _Algorithm Engineering_.
  * Cattan et al. (2023) Arie Cattan, Lilach Eden, Yoav Kantor, and Roy Bar-Haim. 2023.  From key points to key point hierarchy: Structured and expressive opinion summarization.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 912–928, Toronto, Canada. Association for Computational Linguistics.
  * Clauset et al. (2004) Aaron Clauset, Mark E. J. Newman, and Cristopher Moore. 2004.  Finding community structure in very large networks. _Physical review. E, Statistical, nonlinear, and soft matter physics_ , 70 6 Pt 2:066111.
  * Cordasco and Gargano (2010) Gennaro Cordasco and Luisa Gargano. 2010.  Community detection via semi-synchronous label propagation algorithms.  In _2010 IEEE International Workshop on: Business Applications of Social Network Analysis (BASNA)_ , pages 1–8.
  * Dugué and Perez (2015) Nicolas Dugué and Anthony Perez. 2015.  Directed louvain : maximizing modularity in directed networks.
  * Friedman et al. (2021) Roni Friedman, Lena Dankin, Yufang Hou, Ranit Aharonov, Yoav Katz, and Noam Slonim. 2021.  Overview of the 2021 key point analysis shared task.  In _Proceedings of the 8th Workshop on Argument Mining_ , pages 154–164, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple contrastive learning of sentence embeddings.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Guo et al. (2023) Jia Guo, Liying Cheng, Wenxuan Zhang, Stanley Kok, Xin Li, and Lidong Bing. 2023.  AQE: Argument quadruplet extraction via a quad-tagging augmented generative approach.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 932–946, Toronto, Canada. Association for Computational Linguistics.
  * Kapadnis et al. (2021) Manav Kapadnis, Sohan Patnaik, Siba Panigrahi, Varun Madhavan, and Abhilash Nandy. 2021.  Team enigma at ArgMining-EMNLP 2021: Leveraging pre-trained language models for key point matching.  In _Proceedings of the 8th Workshop on Argument Mining_ , pages 200–205, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Kernighan and Lin (1970) B. W. Kernighan and S. Lin. 1970.  An efficient heuristic procedure for partitioning graphs.  _The Bell System Technical Journal_ , 49(2):291–307.
  * Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020.  Supervised contrastive learning.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 18661–18673. Curran Associates, Inc.
  * Krauthgamer et al. (2009) Robert Krauthgamer, Joseph (Seffi) Naor, and Roy Schwartz. 2009.  Partitioning graphs into balanced components.  In _Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms_ , SODA ’09, page 942–949, USA. Society for Industrial and Applied Mathematics.
  * Li et al. (2023) Hao Li, Viktor Schlegel, Riza Batista-Navarro, and Goran Nenadic. 2023.  Do you hear the people sing? key point analysis via iterative clustering and abstractive summarisation.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 14064–14080, Toronto, Canada. Association for Computational Linguistics.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _ArXiv_ , abs/1907.11692.
  * Lloyd (1982) Stuart P. Lloyd. 1982.  Least squares quantization in pcm.  _IEEE Trans. Inf. Theory_ , 28:129–136.
  * Madaan et al. (2022) Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, and Antoine Bosselut. 2022.  Conditional set generation using seq2seq models.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 4874–4896, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Mani and Bloedorn (1997) Inderjeet Mani and Eric Bloedorn. 1997.  Multi-document summarization by graph search and matching.  In _AAAI/IAAI_.
  * Margo and Seltzer (2015) Daniel Margo and Margo Seltzer. 2015.  A scalable distributed graph partitioner.  _Proc. VLDB Endow._ , 8(12):1478–1489.
  * Page et al. (1999) Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999.  The pagerank citation ranking : Bringing order to the web.  In _The Web Conference_.
  * Raghavan et al. (2007) Usha Nandini Raghavan, Réka Albert, and Soundar Kumara. 2007.  Near linear time algorithm to detect community structures in large-scale networks.  _Phys. Rev. E_ , 76:036106.
  * Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020.  Bleurt: Learning robust metrics for text generation.  In _Proceedings of ACL_.
  * Traag et al. (2018) Vincent Antonio Traag, Ludo Waltman, and Nees Jan van Eck. 2018.  From louvain to leiden: guaranteeing well-connected communities.  _Scientific Reports_ , 9.
  * Viswanathan et al. (2023) Vijay Viswanathan, Kiril Gashteovski, Carolin (Haas) Lawrence, Tongshuang Sherry Wu, and Graham Neubig. 2023.  Large language models enable few-shot clustering.  _ArXiv_ , abs/2307.00524.
  * Xiao et al. (2023) Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023.  C-pack: Packaged resources to advance general chinese embedding.  _ArXiv_ , abs/2309.07597.
  * Xiao et al. (2022) Wen Xiao, Iz Beltagy, Giuseppe Carenini, and Arman Cohan. 2022.  PRIMERA: Pyramid-based masked sentence pre-training for multi-document summarization.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 5245–5263, Dublin, Ireland. Association for Computational Linguistics.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  In _Proceedings of the 37th International Conference on Machine Learning_ , ICML’20. JMLR.org.
  * Zhang et al. (2023) Yuwei Zhang, Zihan Wang, and Jingbo Shang. 2023.  Clusterllm: Large language models as a guide for text clustering.  _ArXiv_ , abs/2305.14871.


# A Deep Neural Network Deployment Based on Resistive Memory Accelerator
Simulation

Tejaswanth Maram, Ria Barnwal, Dr. Bindu B

###### Abstract

The objective of this study is to illustrate the process of training a Deep
Neural Network (DNN) within a Resistive RAM (ReRAM) Crossbar-based simulation
environment using CrossSim, an Application Programming Interface (API)
developed for this purpose. The CrossSim API is designed to simulate neural
networks while taking into account factors that may affect the accuracy of
solutions during training on non-linear and noisy ReRAM devices. ReRAM-based
neural cores that serve as memory accelerators for digital cores on a chip can
significantly reduce energy consumption by minimizing data transfers between
the processor and SRAM and DRAM. CrossSim employs lookup tables obtained from
experimentally derived datasets of real fabricated ReRAM devices to digitally
reproduce noisy weight updates to the neural network. The CrossSim directory
comprises eight device configurations that operate at different temperatures
and are made of various materials. This study aims to analyze the results of
training a Neural Network on the Breast Cancer Wisconsin (Diagnostic) dataset
using CrossSim, plotting the innercore weight updates and average training and
validation loss to investigate the outcomes of all the devices.

##  I Introduction

When we talk about the existing memory technologies, we mainly talk about
SRAM, DRAM and flash memory. While all of them are charge based storage
devices, they are used in different places according to their varied
performances in different areas. SRAM devices are high speed, low power
consuming devices which are often used as small cache memory units in the
computers because of it’s complexity to scale in size and costlier to
manufacture. DRAM being a cheaper technology and better scalable, it is used
for bigger RAM units which stores the temporary data that is required by the
computer for the current program at volatile level. Flash memory is a highly
scalable and low-power-consuming non-volatile memory technology, which makes
it an ideal candidate for use as a secondary storage unit in computer systems.

Figure 1: Existing memory technologies and their properties.

It is desired by the industry to create an ideal memory technology which has
most of the upsides and very few if not none of the downsides of the current
memory technologies. Hence with the aim of reinventing the current memory
storage techniques, the industry is actively researching the development of a
new hierarchy of memory often known as emerging memory technologies. The
purpose of these emerging memory technologies is to enable the strengths of
existing memory technologies like the rapid toggling capability of SRAMs, high
storage capacity equivalent to DRAMs, and non-volatile capability of Flash
memory. thus having the potential to be highly appealing alternatives to the
next-generation memory hierarchy. [1]
In contrast to charge-based memories technologies, these memory technologies
works on the principle of making changes in the resistance of the cells to
store the information. Few examples of this emerging memory technologies are :
(i) PCM or phase change memory, (ii) STT-MRAM or spin-transfer torque
magnetoresistive random access memory, and (iii) ReRAM or resistive random
access memory. ReRAM is characterized as a non-volatile memory type that has
the ability to preserve data even during the power absence. ReRAM stores the
data in form of Low and High resistive states. The Metal Insulator Metal (MIM)
structure of this memory type allows the fabrication to be simple. It
comprises an insulating layer (I) positioned between two metal (M) electrodes.
The storage and retrieval of data is dependent on the creation and breaking of
a conductive filament between the electrodes, resulting in the low resistance
state (LRS) and high resistance state (HRS) respectively. These states will be
retained until it is changed with a writing voltage thus making the ReRAMS non
volatile.

Figure 2: Advantages of ReRAM [2]

A ReRAM uses a set voltage (Vs) to form a conductive filament between the
electrodes representing the Low Resistive State(LRS). And a reset voltage (Vr)
to rupture the filament representing the High Reistive State (HRS).

Figure 3: Switching mechanism of ReRAM [2]

In comparison with a conventional CPU, resistive memory crossbars can reduce
the energy required to perform computations in neural algorithms by five
orders of magnitude or more. Data movement between the processor, SRAM (static
random access memory), and DRAM (dynamic random access memory) dominates
computational energy in data intensive applications. Memristor and resistive
crossbars can effectively handle large data sets by minimizing data movement,
leveraging analog operations, and increasing memory density on a single chip.
This newer approach of computing with analog memory based neural cores acting
as an accelerator to the digital core will make the data intensive
applications much more power efficient, compact and faster.
Resistive memory devices are essentially two terminal programmable resistors.
The device’s resistive state has no impact with a low input voltage and can
essentially be used to read the resistance of the memory cell. While as to
write a value to the device, a higher voltage is applied for a set period of
time which will be stored as an analog value of resistance in the memory cell.
When a crossbar of resistive memory cells is applied as a neural network, the
resistance of each cell behaves as a weight simulating a neural synaptic
connection. As a result, neuromorphic systems utilizing such tools have gained
popularity. In theory, the resistive memory would react in a linearly
predictable and controllable manner, enabling them to be set to any given
analog value.

The use of resistive memory crossbar has the potential to accelerate two
important operations in many neural algorithms: parallel read (or vector
matrix multiply) and parallel write (or rank one outer product). Sparse
coding, restricted Boltzmann machines, and backpropagation are examples of
neural algorithms that depend significantly on these two operations. The
processing of inputs and outputs on a crossbar varies based on the specific
algorithm used.

Figure 4: A parallel read (or) vector matrix multiply on crossbar array [3]
Figure 5: A parallel write (or) rank one outerproduct [3]

##  II Materials and Methodology

The Breast Cancer Wisconsin (Diagnostic) Data Set [4] was utilized the DNN for
this paper. This dataset includes 569 instances of features derived from
samples obtained by inserting a thin needle into a breast mass and extracting
cells called FNA for analysis. The features record the characteristics of cell
nuclei in the images. The dataset includes the mean, standard error, and mean
of the three largest values for 10 attributes computed for each image,
resulting in a total of 30 features. These attributes include radius (the
average distance from the center to points on the perimeter), texture (the
standard deviation of gray-scale values), perimeter, area, smoothness (the
local variation in radius lengths), compactness (square of the perimeter,
divided by area, minus 1.0), concavity (the severity of concave portions of
the contour), concave points (the number of concave portions of the contour),
symmetry, and fractal dimension (which approximates the coastline). A deep
neural net is to be trained on these 569 instances of 30 features to classify
the given set of features into either one of the two classes: Malignant and
Benign.

Exploratory data analysis will give us a summary of the main characteristics
of the dataset in the form of visual designs, graphs, etc. To check the
balance of the dataset we visualized the count of each class as a seaborn
count plot. And, The correlation between the attributes can be visualized as a
heatmap.

Figure 6: Count plot of the two classes benign and malignant Figure 7: Heatmap
representing the correlation between the attributes

To simulate this famous dataset on ReRAM crossbars, a tool called CrossSim has
been used for modeling analog in-memory computing for neural networks and
linear algebra operations. It is designed to simulate the impact of analog
hardware effects in resistive crossbars on the algorithm’s accuracy. The
simulator takes into account various factors that affect the accuracy of
training a neural network, such as the non-linear conductance update, write
symmetry, write stochasticity, and device-to-device variability. CrossSim
generates lookup tables from experimental data of a ReRAM device to digitally
simulate these factors. Its accuracy simulation and co-design capabilities
make it a valuable tool for designing and optimizing analog hardware for
neural network applications. The details of this tool and its applications are
discussed in a published research paper [3].

(a) Decreasing Conductance vs CDF

(b) Decreasing Conductance vs Delta G

(c) Increasing Conductance vs CDF

(d) Increasing Conductance vs Delta G

Figure 8: Conductance plots for TaOx Device

CrossSim GitHub repository is provided with more than 180 device
configurations of various materials, mechanisms ,and temperature conditions.
The experimental data of the devices used to generate the lookup tables can be
visualized as conductance plots. Figures 8a, 8b, 8c ,and 8d represent the
conductance plots of TaOx ReRAM devices fabricated at Sandia, ca. 2016 [5]

This paper utilizes the broadly classified 10 configurations of devices
available on the CrossSim which are: DWMTJ SOT 0K, DWMTJ SOT 300K, DWMTJ SOT
400K, DWMTJ STT 0K, DWMTJ STT 300K, DWMTJ STT 400K, ENODe, and TaOx.

A simple deep neural network is built to train the dataset simulated on the
above 8 device configurations. The simulation and model parameters of the
CrossSim are set by default. The DNN consists of 3 layers, first layer is the
input layer with 30 neurons for the input of 30 features. The input layer uses
the sigmoid function for the activation purpose. The second layer is the
hidden layer with 15 neurons and follows the sigmoid function as the
activation purpose. The nonlinearity of the sigmoid function can capture the
complex relationship between the input and the output variables. The “S” –
shaped curve bounded between the range of 0 and 1 will be very sensitive to
small changes. Figure 10 represents the logistic curve which is an example of
the sigmoid function on the XY-axis graph.

Figure 9: logistic function [6]

The third layer of the DNN is the output layer consisting of 2 nodes for the
binary classification of the output into either of the two classes namely
Malignant and Benign. The output layer uses the SoftMax function as the
activation function. By applying the SoftMax function, a set of n actual
numbers is converted into n real numbers whose sum is equal to 1. The neural
net then turns on the node with higher probability thus classifying the given
input into either of the 2 classes. Following are the equations of the sigmoid
and the SoftMax functions respectively.

Sigmoid Function:

$S(x)=\frac{1}{1+e^{-x}}=\frac{e^{x}}{e^{x}+1}=1-S(-x)$ (1)

Softmax Function:

$\sigma(z)_{i}=\frac{e^{z_{i}}}{\sum_{j=1}^{K}e^{z_{j}}}\qquad\text{for
}i=1,\dots,K$ (2)

During the backpropagation of the network, the network uses a cost function
C(w) to calculate the error in the prediction and backpropagate to adjust the
weights of the neurons in order to minimize the error. One widely used cost
function is the quadratic cost function. The neural net applies a gradient
descent algorithm to the cost function C(w) and finds the minima which would
result in the least value of error possible in the prediction made by the DNN.
The step of the gradient also plays an important role in the performance of a
neural network. A smaller step value would result in higher time consumption
to reach the minima while a bigger step value could result in inaccurate
minima. An optimizer algorithm would help find the right step size for the
gradient descent algorithm and one such algorithm that is used is Adam’s
algorithm. Figure 10 shows the performance of Adam algorithm on compared to
other famous optimization algorithms.

Figure 10: Adam algorithm performance comparison on the MNIST dataset [7]

Quadratic Cost Function:

$\sigma(z)_{i}=\frac{1}{2n}\sum_{x}{[y(x)-a^{L}(x)]}^{2}$ (3)

##  III Results and Discussion

The neural net with the above mentioned architecture and parameters has been
trained and validated on the 8 multiple device configurations. The dataset was
partitioned into three subsets, with 40% allocated for training, 30% for
validation, and 30% for testing purposes. The observations in the innercore
update errors have been visualized on Target update vs Real update plot and a
update error vs probability density plot. The plots have been mapped for the
numeric, standard and multi update configurations
The training was done for 80 epochs on each device. The training curves are
plotted against the average loss in the predictions made for the training data
and the average loss in the predictions made for the validation data. The plot
for training loss starts from the epoch 0 while the plot for validation loss
starts from epoch 1 for the better scaling of the graph pictures. The
innercore updates plots shows the weight updates happened in each epoch on a
scatter plot. The plot displays the real update vs target update in the inner
core and also the update error and its probability density.
The DWMTJ: domain-wall magnetic tunnel junction is a spintronic device that
stores data inform of magnetic spins. This device has been simulated with the
data in three different temperatures: 0K, 300K and 400K, and in two different
DW propagation mechanisms: spin transfer torque (STT) and spin orbit torque
(SOT) [8]. The ENODe device refers to the electrochemical random access memory
(ECRAM). The multi model of ENODe uses lookup table based on 9 devices[9]. The
TaOx device uses the lookup table from the experimental data generated by the
TaOx ReRAM devices fabricated at Sandia. The multi model for TaOx listed uses
the medium set from CrossSim [5].
Observations show that all the devices perform with accuracies well above 92%.
It also can be clearly seen from the weight update plots of DWMTJ that the
lesser temperatures will decrease the noise in the updates causing less error.
The higher temperature induces more noise and error in the weight updates of a
device. DWMTJ SOT at 400K has the highest accuracy of 97.8%. ENODe displayed
the least accuracy of 92.9% in the standard and multi modes.

Device Configuration | Training Accuracy
---|---
| Numeric | Standard | Multi
DWMTJ SOT 0K | 96.4% | 97.8% | 95.7%
DWMTJ SOT 300k | 97.1% | 97.1% | 97.1%
DWMTJ SOT 400k | 97.8% | 97.8% | 97.8%
DWMTJ STT 0k | 96.4% | 97.8% | 97.8%
DWMTJ STT 300k | 97.1% | 96.4% | 95.7%
DWMTJ STT 400k | 96.4% | 97.1% | 97.8%
TaOx | 95.7% | 96.4% | 97.1%
ENODe 8 | 96.4% | 92.9% | 92.9%

###  III-A DWMTJ SOT 0k

Figure 11: innercore update error Balanced DWMTJ SOT 0K numeric Figure 12: avg
training loss vs avg validation loss Figure 13: innercore update error
Balanced DWMTJ SOT 0K standard Figure 14: avg training loss vs avg validation
loss Figure 15: innercore update error Balanced DWMTJ SOT 0K multi Figure 16:
avg training loss vs avg validation loss

###  III-B DWMTJ SOT 300k

Figure 17: innercore update error Balanced DWMTJ SOT 300K numeric Figure 18:
avg training loss vs avg validation loss Figure 19: innercore update error
Balanced DWMTJ SOT 300K standard Figure 20: avg training loss vs avg
validation loss Figure 21: innercore update error Balanced DWMTJ SOT 300K
multi Figure 22: avg training loss vs avg validation loss

###  III-C DWMTJ SOT 400k

Figure 23: innercore update error Balanced DWMTJ SOT 400K numeric Figure 24:
avg training loss vs avg validation loss Figure 25: innercore update error
Balanced DWMTJ SOT 400K standard Figure 26: avg training loss vs avg
validation loss Figure 27: innercore update error Balanced DWMTJ SOT 400K
multi Figure 28: avg training loss vs avg validation loss

###  III-D DWMTJ STT 0k

Figure 29: innercore update error Balanced DWMTJ STT 0K numeric Figure 30: avg
training loss vs avg validation loss Figure 31: innercore update error
Balanced DWMTJ STT 0K standard Figure 32: avg training loss vs avg validation
loss Figure 33: innercore update error Balanced DWMTJ STT 0K multi Figure 34:
avg training loss vs avg validation loss

###  III-E DWMTJ STT 300k

Figure 35: innercore update error Balanced DWMTJ STT 300K numeric Figure 36:
avg training loss vs avg validation loss Figure 37: innercore update error
Balanced DWMTJ STT 300K standard Figure 38: avg training loss vs avg
validation loss Figure 39: innercore update error Balanced DWMTJ STT 300K
multi Figure 40: avg training loss vs avg validation loss

###  III-F DWMTJ STT 400k

Figure 41: innercore update error Balanced DWMTJ STT 400K numeric Figure 42:
avg training loss vs avg validation loss Figure 43: innercore update error
Balanced DWMTJ STT 400K standard Figure 44: avg training loss vs avg
validation loss Figure 45: innercore update error Balanced DWMTJ STT 400K
multi Figure 46: avg training loss vs avg validation loss

###  III-G ENODe

Figure 47: innercore update error Balanced ENODe numeric Figure 48: avg
training loss vs avg validation loss Figure 49: innercore update error
Balanced ENODe standard Figure 50: avg training loss vs avg validation loss
Figure 51: innercore update error Balanced ENODe multi Figure 52: avg training
loss vs avg validation loss

###  III-H TaOx

Figure 53: innercore update error Balanced TaOx numeric Figure 54: avg
training loss vs avg validation loss Figure 55: innercore update error
Balanced TaOx standard Figure 56: avg training loss vs avg validation loss
Figure 57: innercore update error Balanced TaOx multi Figure 58: avg training
loss vs avg validation loss

## References

  * [1] . S. S. Wong H. S., “Memory leads the way to better computing. nature nanotechnology,” vol. 10, no. 3, pp. 191–194, 2015. [Online]. Available: https://doi.org/10.1038/nnano.2015.29
  * [2] A. Z. T. . K. F. R. R. A. M. R. Zahoor F., “an overview of materials, switching mechanism, performance, multilevel cell (mlc) storage, modeling, and applications.” _Springer Series_ , 2020. [Online]. Available: https://doi.org/10.1186/s11671-020-03299-9
  * [3] T. P. Xiao, B. Feinberg, C. H. Bennett, V. Prabhakar, P. Saxena, V. Agrawal, S. Agarwal, and M. J. Marinella, “On the accuracy of analog neural network inference accelerators [feature],” _IEEE Circuits and Systems Magazine_ , vol. 22, no. 4, pp. 26–48, 2022.
  * [4] D. Dua and C. Graff, “UCI machine learning repository,” 2017. [Online]. Available: http://archive.ics.uci.edu/ml
  * [5] C. H. Bennett, D. Garland, R. B. Jacobs-Gedrim, S. Agarwal, and M. J. Marinella, “Wafer-scale taox device variability and implications for neuromorphic computing applications,” in _2019 IEEE International Reliability Physics Symposium (IRPS)_ , 2019, pp. 1–4.
  * [6] J. Han and C. Moraga, “The influence of the sigmoid function parameters on the speed of backpropagation learning,” in _From Natural to Artificial Neural Computation_ , J. Mira and F. Sandoval, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 1995, pp. 195–201.
  * [7] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _International Conference on Learning Representations_ , 12 2014.
  * [8] S. Liu, T. P. Xiao, C. Cui, J. C. Incorvia, C. H. Bennett, and M. J. Marinella, “A domain wall-magnetic tunnel junction artificial synapse with notched geometry for accurate and efficient training of deep neural networks,” _Applied Physics Letters_ , vol. 118, no. 20, 5 2021.
  * [9] Y. Li, T. P. Xiao, C. H. Bennett, E. Isele, A. Melianas, H. Tao, M. J. Marinella, A. Salleo, E. J. Fuller, and A. A. Talin, “In situ parallel training of analog neural network using electrochemical random-access memory,” _Frontiers in Neuroscience (Online)_ , vol. 15, 4 2021.


Tr\,}}(\Omega)\to{\cal H}_{\cal A}^{1}(\Omega)$ by

$\gamma(\varphi)=u,$

where $u\in{\cal H}_{\cal A}^{1}(\Omega)$ is the unique element with trace
$\varphi$.

We next wish to prove that $\gamma$ maps bounded functions in ${\mathop{\rm
Tr\,}}(\Omega)$ into bounded functions. We need a lemma.

######  Lemma 5.4.

Let $\varphi\in{\mathop{\rm Tr\,}}(\Omega)\cap L_{\infty}(\Gamma)$. Then there
exist $u_{1},u_{2},\ldots\in H^{1}(\Omega)\cap C(\overline{\Omega})$ and $u\in
H^{1}(\Omega)$ such that $\lim u_{n}=u$ in $H^{1}(\Omega)$, $\lim
u_{n}|_{\Gamma}=\varphi$ in $L_{2}(\Gamma)$ and
$\|u_{n}\|_{L_{\infty}(\Omega)}\leq 2\|\varphi\|_{L_{\infty}(\Gamma)}$ for all
$n\in\mathds{N}$.

###### Proof.

First suppose that $\varphi$ is real valued. There are $u_{1},u_{2},\ldots\in
H^{1}(\Omega)\cap C(\overline{\Omega})$ and $u\in H^{1}(\Omega)$ such that
$\lim u_{n}=u$ in $H^{1}(\Omega)$ and $\lim u_{n}|_{\Gamma}=\varphi$ in
$L_{2}(\Gamma)$. Taking the real part we may assume that $u_{n}$ and $u$ are
real valued for all $n\in\mathds{N}$. Define
$v_{n}=(-\|\varphi\|_{L_{\infty}(\Gamma)})\vee
u_{n}\wedge\|\varphi\|_{L_{\infty}(\Gamma)}$ for all $n\in\mathds{N}$. Then
$v_{n}\in H^{1}(\Omega)\cap C(\overline{\Omega})$ for all $n\in\mathds{N}$.
Moreover, $\lim v_{n}=(-\|\varphi\|_{L_{\infty}(\Gamma)})\vee
u\wedge\|\varphi\|_{L_{\infty}(\Gamma)}\in H^{1}(\Omega)$ in $H^{1}(\Omega)$
and $\lim v_{n}|_{\Gamma}=\varphi$ in $L_{2}(\Gamma)$. Clearly
$\|v_{n}\|_{L_{\infty}(\Omega)}\leq\|\varphi\|_{L_{\infty}(\Gamma)}$ for all
$n\in\mathds{N}$.

If $\varphi$ is general, then write $\varphi=\mathop{\rm
Re}\varphi+i\mathop{\rm Im}\varphi$ and apply the above to $\mathop{\rm
Re}\varphi$ and $\mathop{\rm Im}\varphi$. ∎

######  Proposition 5.5.

Suppose that $V(\Omega)=H^{1}_{0}(\Omega)$. Let $\varphi\in{\mathop{\rm
Tr\,}}(\Omega)\cap L_{\infty}(\Gamma)$. Then $\gamma(\varphi)\in
L_{\infty}(\Omega)$ and $\|\gamma(\varphi)\|_{L_{\infty}(\Omega)}\leq
2c\,\|\varphi\|_{L_{\infty}(\Gamma)}$, where $c=\|T\|_{C(\Gamma)\to
C_{b}(\Omega)}$.

###### Proof.

By Lemma 5.4 there are $w_{1},w_{2},\ldots\in H^{1}(\Omega)\cap
C(\overline{\Omega})$ and $w\in H^{1}(\Omega)$ such that $\lim w_{n}=w$ in
$H^{1}(\Omega)$, $\lim w_{n}|_{\Gamma}=\varphi$ in $L_{2}(\Gamma)$ and
$\|w_{n}\|_{L_{\infty}(\Omega)}\leq 2\|\varphi\|_{L_{\infty}(\Gamma)}$ for all
$n\in\mathds{N}$. For all $n\in\mathds{N}$ write $w_{n}=v_{n}+u_{n}$ and
$w=v+u$, where $v_{n},v\in H^{1}_{0}(\Omega)$ and $u_{n},u\in{\cal H}_{\cal
A}^{1}(\Omega)$. Since $H^{1}_{0}(\Omega)$ and ${\cal H}_{\cal A}^{1}(\Omega)$
are closed in $H^{1}(\Omega)$, it follows from the closed graph theorem that
$\lim u_{n}=u$ in $H^{1}(\Omega)$.

We know that $\gamma(\varphi)=u$. Then $u_{n}=T(w_{n}|_{\Gamma})$ for all
$w\in\mathds{N}$ by Theorem 1.1. So

$\|u_{n}\|_{L_{\infty}(\Omega)}\leq
c\,\|w_{n}|_{\Gamma}\|_{L_{\infty}(\Gamma)}\leq
2c\,\|\varphi\|_{L_{\infty}(\Gamma)}$

for all $n\in\mathds{N}$. Now $\lim u_{n}=u$ in $H^{1}(\Omega)$ and hence also
in $L_{2}(\Omega)$. Passing to a subsequence if necessary we may assume that
$\lim u_{n}=u$ almost everywhere. Hence
$\|\gamma(\varphi)\|_{L_{\infty}(\Omega)}=\|u\|_{L_{\infty}(\Omega)}\leq
2c\,\|\varphi\|_{L_{\infty}(\Gamma)}$ as required. ∎

######  Theorem 5.6.

Suppose that $V(\Omega)=H^{1}_{0}(\Omega)$. Then $T\varphi=\gamma(\varphi)$
for all $\varphi\in C(\Gamma)\cap{\mathop{\rm Tr\,}}(\Omega)$.

###### Proof.

Clearly $T(\Phi|_{\Gamma})=\gamma(\Phi|_{\Gamma})$ for all $\Phi\in
H^{1}(\Omega)\cap C(\overline{\Omega})$ by Theorem 1.1.

Let $\varphi\in C(\Gamma)\cap{\mathop{\rm Tr\,}}(\Omega)$. By Stone–Weierstraß
there exist $\Phi_{1},\Phi_{2},\ldots\in H^{1}(\Omega)\cap
C(\overline{\Omega})$ such that $\lim\Phi_{n}|_{\Gamma}=\varphi$ in
$C(\Gamma)$. Then $T\varphi=\lim T(\Phi_{n}|_{\Gamma})$ in $C_{b}(\Omega)$ and
in particular pointwise on $\Omega$. If $n\in\mathds{N}$, then

$\|\gamma(\varphi)-\gamma(\Phi_{n}|_{\Gamma})\|_{L_{\infty}(\Omega)}\leq
2c\,\|\varphi-\Phi_{n}|_{\Gamma}\|_{C(\Gamma)}$

by Proposition 5.5, where $c=\|T\|_{C(\Gamma)\to C_{b}(\Omega)}$. Hence
$\lim\gamma(\Phi_{n}|_{\Gamma})=\gamma(\varphi)$ almost everywhere on
$\Omega$. So $\lim T(\Phi_{n}|_{\Gamma})=\gamma(\varphi)$ almost everywhere on
$\Omega$. Therefore $T\varphi=\gamma(\varphi)$ almost everywhere. ∎

One would expect that if $\varphi\in C(\Gamma)$ and $T\varphi\in
H^{1}(\Omega)$, then $\varphi$ is a trace of $T\varphi$. The proof relies on
Theorem 4.1, whose proof was quite involved and depended on capacity methods.

######  Theorem 5.7.

Let $\varphi\in C(\Gamma)$. Suppose that $T\varphi\in H^{1}(\Omega)$. Then
$\varphi$ is a trace of $T\varphi$.

###### Proof.

It follows from Corollary 3.12 and the choice $u=T\varphi$ in Theorem 4.1 that
for all $n\in\mathds{N}$ there exists a $\Phi_{n}\in C(\overline{\Omega})\cap
H^{1}_{\rm loc}(\Omega)$ such that $\Phi_{n}|_{\Gamma}=\varphi$,
$T\varphi-\Phi_{n}\in H^{1}_{0}(\Omega)$ and in addition
$\|T\varphi-\Phi_{n}\|_{H^{1}_{0}(\Omega)}\leq\frac{1}{n}$ for all
$n\in\mathds{N}$. Since $T\varphi\in H^{1}(\Omega)$ also $\Phi_{n}\in
H^{1}(\Omega)$ for all $n\in\mathds{N}$. So $\Phi_{n}\in
C(\overline{\Omega})\cap H^{1}(\Omega)$ for all $n\in\mathds{N}$. Hence
$\lim\Phi_{n}=T\varphi$ in $H^{1}(\Omega)$ and obviously
$\lim\Phi_{n}|_{\Gamma}=\varphi$ in $L_{2}(\Gamma)$. Therefore $\varphi$ is a
trace of $T\varphi$. ∎

Theorems 5.6 and 5.7 allow us to give another characterisation of those
$\varphi\in C(\Gamma)$ for which the Perron solution $T\varphi$ is an element
of $H^{1}(\Omega)$. See also Corollary 4.4.

######  Theorem 5.8.

Suppose that $V(\Omega)=H^{1}_{0}(\Omega)$. Let $\varphi\in C(\Gamma)$. Then
the following are equivalent.

  * (i)

$T\varphi\in H^{1}(\Omega)$.

  * (ii)

$\varphi\in{\mathop{\rm Tr\,}}(\Omega)$.

###### Proof.

‘(i)$\Rightarrow$(ii)’. It follows from Theorem 5.7 that $\varphi$ is a trace
of $T\varphi$. Hence $\varphi\in{\mathop{\rm Tr\,}}(\Omega)$.

‘(ii)$\Rightarrow$(i)’. By assumption $\varphi\in C(\Gamma)\cap{\mathop{\rm
Tr\,}}(\Omega)$. Hence $T\varphi=\gamma(\varphi)\in H^{1}(\Omega)$ by Theorem
5.6. ∎

The next example shows that the condition $V(\Omega)=H^{1}_{0}(\Omega)$ in
Theorem 5.8 cannot be omitted.

######  Example 5.9.

Let $\Omega$ be the Wiener regular bounded open set as in Example 5.3. We
shall show that there exists a $\varphi\in C(\Gamma)$ such that
$\varphi|_{\partial B(0,2)}=0$ and $T\varphi\not\in H^{1}(\Omega)$. Hence
$\varphi=0$ almost everywhere on $\Gamma$, so $\varphi\in
C(\Gamma)\cap{\mathop{\rm Tr\,}}(\Omega)$.

Set $d_{0}=\frac{\log 2}{\log 3}$. Recall that $S={\cal C}\times\\{0\\}$. Then
$S$ is a $d_{0}$-set in the sense of Jonsson–Wallin by [JW] Example II.2. Fix
$s\in(0,\frac{d_{0}}{2})$. Since $s<1\wedge d_{0}$ it follows from [CC]
Theorem 3.2 that there exists a Weierstraß-type function $\psi\in C(S)$ such
that $\dim_{H}G(\psi)=d_{0}+1-s$, where $\dim_{H}$ denotes the Hausdorff
dimension and $G(\psi)$ is the graph of $\psi$. Define $\varphi\in C(\Gamma)$
by

$\varphi(z)=\left\\{\begin{array}[]{ll}\psi(z)&\mbox{if }z\in S,\\\\[5.0pt]
0&\mbox{if }z\in\partial B(0,2).\end{array}\right.$

Since $\Omega$ is Wiener regular, it follows from Corollary 3.4 that the
Dirichlet problem $(D_{\varphi})$ has a (unique) classical solution $u\in
C(\overline{\Omega})\cap H^{1}_{\rm loc}(\Omega)$. Then $T\varphi=u|_{\Omega}$
and obviously $\varphi\in C(\Gamma)\cap{\mathop{\rm Tr\,}}(\Omega)$.

Now suppose that $T\varphi\in H^{1}(\Omega)$. Then $u\in
C(\overline{\Omega})\cap H^{1}(\Omega)\subset H^{1}(B(0,2))$ by [BW2]
Proposition 3.6, since $\sigma(S)<\infty$. Because every element in
$H^{1}(B(0,2))$ can be extended to an element of
$H^{1}(\mathds{R}^{2})=B^{22}_{1}(\mathds{R}^{2})$, a Besov space, it follows
from [JW] Theorem VII.1 that $\psi=u|_{S}\in B^{22}_{d_{0}/2}(S)$. Then [CC]
Theorem 3.3 implies that $\dim_{H}G(\psi)\leq d_{0}+1-\frac{d_{0}}{2}$. But
$\dim_{H}G(\psi)=d_{0}+1-s$. So $\frac{d_{0}}{2}\leq s$, which is a
contradiction. Hence $T\varphi\not\in H^{1}(\Omega)$.

If $\Omega$ is Wiener regular, then it follows from Biegert–Warma [BW1]
Theorem 3.2(c)$\Rightarrow$(b) and the Wiener condition (12) that each $u\in
H^{1}_{0}(\Omega)\cap C(\overline{\Omega})$ vanishes (pointwise) on $\Gamma$.
This is no longer true if $u\in V(\Omega)\cap C(\overline{\Omega})$. In fact,
in Example 5.3 the set $\Omega$ is Wiener regular, but nevertheless there
exists a $u\in V(\Omega)\cap C(\overline{\Omega})$ such that $u|_{\Gamma}\neq
0$ in $C(\Gamma)$. Next we show that any such $u$ vanishes $\sigma$-a.e. on
$\Gamma$.

######  Proposition 5.10.

Let $u\in V(\Omega)\cap C(\overline{\Omega})$. Then $u|_{\Gamma}=0$
$\sigma$-a.e.

###### Proof.

Write $\varphi=u|_{\Gamma}\in C(\Gamma)$. Recall that $\Gamma_{r}$ and
$\Gamma_{s}$ are Borel measurable subsets of $\Gamma$. Then
$\varphi|_{\Gamma_{r}}=0$ $\sigma$-a.e. by construction/definition. Define

$Z=\\{z\in\Gamma_{s}:\limsup_{r\downarrow 0}\frac{|\Omega\cap
B(z,r)|}{r^{d}}>0\\}.$

Then $\sigma(Z)=0$ by [Sau2] Theorem 4.15. We shall show that $\varphi(z)=0$
for all $z\in\Gamma_{s}\setminus Z$.

Let $z\in\Gamma_{s}\setminus Z$ and let $\delta>0$. We first prove that
$\sigma(\Gamma_{r}\cap B(z,\delta))>0$. Indeed, suppose that
$\sigma(\Gamma_{r}\cap B(z,\delta))=0$. By [AFP] (3.43) there exists a
$\gamma>0$ such that

$\Big{(}|\Omega\cap B(z,\delta)|\wedge|\Omega^{\rm c}\cap
B(z,\delta)|\Big{)}^{(d-1)/d}\leq\gamma\,P(\Omega,B(z,\delta)),$ (25)

where $P(\Omega,B(z,\delta))$ denotes the perimeter of $\Omega$ in
$B(z,\delta)$. Now

$P(\Omega,B(z,\delta))=\sigma((\partial^{*}\Omega)\cap B(z,\delta))$

by [Fed] Theorem 4.5.11 and [AFP] (3.62), where $\partial^{*}\Omega$ is the
measure theoretic boundary of $\Omega$. Note that

$\mathds{R}^{d}\setminus\partial^{*}\Omega=\\{p\in\mathds{R}^{d}:\lim_{r\downarrow
0}\frac{|\Omega\cap B(p,r)|}{r^{d}}=0\mbox{ or}\lim_{r\downarrow
0}\frac{|\Omega^{\rm c}\cap B(p,r)|}{r^{d}}=0\\}.$

Clearly $\partial^{*}\Omega\subset\Gamma$. Moreover, $\Gamma_{s}\setminus
Z\subset\Gamma\setminus\partial^{*}\Omega$. Hence
$\partial^{*}\Omega\subset\Gamma_{r}\cup Z$. Then

$P(\Omega,B(z,\delta))=\sigma((\partial^{*}\Omega)\cap
B(z,\delta))\leq\sigma(\Gamma_{r}\cap B(z,\delta))+\sigma(Z)=0.$

On the other hand, $|\Omega\cap B(z,\delta)|>0$ since $z\in\partial\Omega$ and
$\Omega$ is open. Also $\limsup_{r\downarrow 0}\frac{|\Omega\cap
B(z,r)|}{r^{d}}=0$ since $z\in\Gamma_{s}\setminus Z$. Then $\lim_{r\downarrow
0}\frac{|\Omega\cap B(z,r)|}{r^{d}}=0$ and consequently $\lim_{r\downarrow
0}\frac{|\Omega^{\rm c}\cap B(z,r)|}{|B(z,r)|}=1$. So $|\Omega^{\rm c}\cap
B(z,\delta)|>0$. This contradicts (25).

Let $z\in\Gamma_{s}\setminus Z$. We proved that $\sigma(\Gamma_{r}\cap
B(z,\delta))>0$ for all $\delta>0$. Since $\varphi|_{\Gamma_{r}}=0$
$\sigma$-a.e. and $\varphi$ is continuous, one deduces that $\varphi(z)=0$. ∎

We conclude with a sufficient condition which implies for a Perron solution to
vanish.

######  Proposition 5.11.

Suppose that $\sigma(\Gamma\cap B(z,\delta))>0$ for all $z\in\Gamma$ and
$\delta>0$. Let $\varphi\in C(\Gamma)$ and suppose that $T\varphi\in
V(\Omega)$. Then $\varphi=0$ and $T\varphi=0$.

###### Proof.

It follows from Corollary 3.12 and Theorem 4.1 that there exists a $\Phi\in
C(\overline{\Omega})\cap H^{1}(\Omega)$ such that $\Phi|_{\Gamma}=\varphi$ and
$T\varphi-\Phi\in H^{1}_{0}(\Omega)$. Then $\Phi\in V(\Omega)$. By Proposition
5.10 there exists a measurable set $N\subset\Gamma$ such that $\sigma(N)=0$
and $\Phi(z)=0$ for all $z\in\Gamma\setminus N$. Let $z\in\Gamma$. If
$n\in\mathds{N}$, then $\sigma((\Gamma\setminus N)\cap B(z,\frac{1}{n}))>0$,
hence there exists a $z_{n}\in(\Gamma\setminus N)\cap B(z,\frac{1}{n})$. Then
$\varphi(z)=\lim\varphi(z_{n})=0$. ∎

By the next lemma, the first condition in the previous proposition is
satisfied if $\Omega$ is regular in topology, that is if
$\overline{\Omega}\raisebox{6.10834pt}{\makebox[0.0pt]{$\scriptstyle\circ$}}=\Omega$.
On the other hand, it is not sufficient to suppose that $\Omega$ is regular in
capacity by Example 5.3.

######  Lemma 5.12.

Suppose $\Omega$ is regular in topology. Then $\sigma(\Gamma\cap
B(z,\delta))>0$ for all $z\in\Gamma$ and $\delta>0$.

###### Proof.

Let $z\in\Gamma$ and $\delta>0$. Then $|\Omega\cap B(z,\delta)|>0$ and
$|\Omega^{c}\cap B(z,\delta)|>0$. Hence by (25) one deduces that
$P(\Omega,B(z,\delta))>0$. Arguing as in the proof of Proposition 5.10 one
obtains

$0<P(\Omega,B(z,\delta))=\sigma((\partial^{*}\Omega)\cap
B(z,\delta))\leq\sigma(\Gamma\cap B(z,\delta))$

as required. ∎

######  Corollary 5.13.

If $\Omega$ is regular in topology, then

$C(\overline{\Omega})\cap{\cal H}^{1}_{\cal A}(\Omega)\cap V(\Omega)=\\{0\\}.$

###### Proof.

Let $u\in C(\overline{\Omega})\cap{\cal H}^{1}_{\cal A}(\Omega)\cap
V(\Omega)$. Then $u=T(u|_{\Gamma})$ by Proposition 2.16. Now use Lemma 5.12
and Proposition 5.11. ∎

Applying the arguments in [Sau1] Example 7.55 to the domain in Example 5.3,
one can construct a topologically and Wiener regular bounded set $\Omega$ with
$\sigma(\Gamma)<\infty$ such that $H^{1}_{0}(\Omega)\subsetneqq V(\Omega)$. By
Theorem 5.2 the variational solution of the Dirichlet problem is not unique on
this set $\Omega$. Still, for every $\varphi\in C(\Gamma)$ with $T\varphi\in
H^{1}(\Omega)$ there exists a unique $u\in C(\overline{\Omega})\cap{\cal
H}^{1}_{\cal A}(\Omega)$ with trace $\varphi$ by Theorem 5.7 and Corollary
5.13.

### Acknowledgements

The second-named author is most grateful for the hospitality extended to him
during a fruitful stay at the University of Ulm. He wishes to thank the
University of Ulm for financial support. The third-named author had a
wonderful time during his visit at the University of Auckland in early 2020.
He is most grateful for the hospitality, the productive collaboration and the
financial support. This work is supported by the Marsden Fund Council from
Government funding, administered by the Royal Society of New Zealand.

## References

  * [AH] Adams, D. R. and Hedberg, L. I., Function spaces and potential theory.  Grundlehren der mathematische Wissenschaften 314. Springer, Berlin, 1996\.
  * [AFP] Ambrosio, L., Fusco, N. and Pallara, D., Functions of bounded variation and free discontinuity problems.  Oxford Mathematical Monographs. The Clarendon Press, Oxford University Press, New York, 2000.
  * [AD1] Arendt, W. and Daners, D., The Dirichlet problem by variational methods.  Bull. London Math. Soc. 40 (2008), 51–56.
  * [AD2] , Varying domains: stability of the Dirichlet and the Poisson problem.  Discrete and Continuous Dynamical Systems - Series A 21 (2008), 21–39.
  * [AE1] Arendt, W. and Elst, A. F. M. ter, Sectorial forms and degenerate differential operators.  J. Operator Theory 67 (2012), 33–72.
  * [AE2] , The Dirichlet problem without the maximum principle.  Annales de l’Institut Fourier 69 (2019), 763–782.
  * [AEG] Arendt, W., Elst, A. F. M. ter and Glück, J., The maximum principle via irreducibility.  Adv. Nonlinear Stud. 20 (2020), 633–650.
  * [AEKS] Arendt, W., Elst, A. F. M. ter, Kennedy, J. B. and Sauter, M., The Dirichlet-to-Neumann operator via hidden compactness.  J. Funct. Anal. 266 (2014), 1757–1786.
  * [AU] Arendt, W. and Urban, K., Partial differential equations. An introduction to analytical and numerical methods.  Springer, Berlin, 2022.  To appear.
  * [BW1] Biegert, M. and Warma, M., Regularity in capacity and the Dirichlet Laplacian.  Potential Anal. 25 (2006), 289–305.
  * [BW2] Biegert, M. and Warma, M., Removable singularities for a Sobolev space.  J. Math. Anal. Appl. 313 (2006), 49–63.
  * [Bre] Brelot, M., Familles de Perron et problème de Dirichlet.  Acta Litt. Sci. Szeged 9 (1939), 133–153.
  * [CC] Carvalho, A. and Caetano, A., On the Hausdorff dimension of continuous functions belonging to Hölder and Besov spaces on fractal $d$-sets.  J. Fourier Anal. Appl. 18 (2012), 386–409.
  * [Chi] Chicco, M., Un’osservazione sulla regolarità alla frontiera delle soluzioni de equazioni ellittiche di tipo variazionale.  Boll. Un. Mat. Ital. (4) 3 (1970), 379–383.
  * [Dan1] Daners, D., Robin boundary value problems on arbitrary domains.  Trans. Amer. Math. Soc. 352 (2000), 4207–4236.
  * [Dan2] , Dirichlet problems on varying domains.  J. Differential Equations 188 (2003), 591–624.
  * [EE] Edmunds, D. E. and Evans, W. D., Spectral theory and differential operators.  Oxford Mathematical Monographs. Oxford University Press, Oxford etc., 1987\.
  * [ER] Elst, A. F. M. ter and Rehberg, J., Hölder estimates for second-order operators on domains with rough boundary.  Adv. Diff. Equ. 20 (2015), 299–360.
  * [Fed] Federer, H., Geometric measure theory.  Grundlehren der mathematischen Wissenschaften 153. Springer-Verlag, Berlin, 1969.
  * [FOT] Fukushima, M., Oshima, Y. and Takeda, M., Dirichlet forms and symmetric Markov processes, vol. 19 of de Gruyter Studies in Mathematics.  Walter de Gruyter & Co., Berlin, 1994.
  * [GM] Giaquinta, M. and Martinazzi, L., An introduction to the regularity theory for elliptic systems, harmonic maps and minimal graphs.  Edizioni Della Normale, Pisa, 2005.
  * [GT] Gilbarg, D. and Trudinger, N. S., Elliptic partial differential equations of second order.  Second edition, Grundlehren der mathematischen Wissenschaften 224. Springer-Verlag, Berlin etc., 1983.
  * [Her] Hervé, R.-M., Un principe du maximum pour les sous-solutions locales d’une équation uniformément elliptique de la forme.  Ann. Inst. Fourier (Grenoble) 14 (1964), 493–508.
  * [JW] Jonsson, A. and Wallin, H., Function spaces on subsets of ${\bf R}^{n}$.  Math. Rep. 2, No. 1 (1984).
  * [Kel1] Keldyš, M. V., On the Dirichlet problem.  C. R. (Dokl.) Acad. Sci. URSS 32 (1941), 308–309.
  * [Kel2] , On the solvability and stability of the Dirichlet problem.  Am. Math. Soc., Transl., II. Ser. 51 (1966), 1–73.  Translation from Usp. Mat. Nauk 8, 171–231 (1941).
  * [Kre] Kreuter, M., Vector-valued elliptic boundary value problems on rough domains.  PhD thesis, Ulm University, Ulm, Germany, 2018.  OpenAccessRepositorium der Universitiät Ulm, https://doi.org/10.18725/OPARU-11852.
  * [LSW] Littman, W., Stampacchia, G. and Weinberger, H. F., Regular points for elliptic equations with discontinuous coefficients.  Ann. Sc. Norm. Super. Pisa, Sci. Fis. Mat., III. Ser. 17 (1963), 43–77.
  * [Maz] Maz’ya, V. G., Sobolev spaces with applications to elliptic partial differential equations.  Second edition, Grundlehren der mathematischen Wissenschaften 342. Springer-Verlag, Berlin etc., 2011.
  * [MS] Maz’ya, V. and Shaposhnikova, T., Jacques Hadamard, a universal mathematician.  History of Mathematics 14. London Mathematical Society, London, 1998.
  * [Net] Netuka, I., The Dirichlet problem for harmonic functions.  Amer. Math. Monthly 87 (1980), 621–628.
  * [Ouh] Ouhabaz, E. M., Analysis of heat equations on domains.  London Mathematical Society Monographs Series 31. Princeton University Press, Princeton, NJ, 2005.
  * [Per] Perron, O., Eine neue Behandlung der ersten Randwertaufgabe für $\Delta u=0$.  Math. Z. 18 (1923), 42–54.
  * [Sau1] Sauter, M. S., Degenerate elliptic operators with boundary conditions via form methods.  PhD thesis, The University of Auckland, Auckland, New Zealand, 2013.  http://hdl.handle.net/2292/21318.
  * [Sau2] Sauter, M., Uniqueness of the approximative trace.  Indiana Univ. Math. J. 69 (2020), 139–172.
  * [Sta] Stampacchia, G., Le problème de Dirichlet pour les équations elliptiques du second ordre à coefficients discontinus.  Ann. Inst. Fourier, Grenoble 15 (1965), 189–258.
  * [Tsu] Tsuji, M., On the capacity of general Cantor sets.  J. Math. Soc. Japan 5 (1953), 235–252.
  * [Wie1] Wiener, N., Certain notions in potential theory.  J. Math. Phys. 3 (1924), 24–51.
  * [Wie2] , The Dirichlet problem.  J. Math. Phys. 3 (1924), 127–146.
  * [Wie3] , Note on a paper of O. Perron.  J. Math. Phys 4 (1925), 21–32.