# Multiplication-Avoiding Variant of Power Iteration with Applications

###### Abstract

Power iteration is a fundamental algorithm in data analysis. It extracts the
eigenvector corresponding to the largest eigenvalue of a given matrix.
Applications include ranking algorithms, principal component analysis (PCA),
among many others. Certain use cases may benefit from alternate, non-linear
power methods with low complexity. In this paper, we introduce multiplication-
avoiding power iteration (MAPI). MAPI replaces the standard $\ell_{2}$ inner
products that appear at the regular power iteration (RPI) with multiplication-
free vector products, which are Mercer-type kernels that induce the $\ell_{1}$
norm. For an $n\times n$ matrix, MAPI requires $n$ multiplications, while RPI
needs $n^{2}$ multiplications per iteration. Therefore, MAPI provides a
significant reduction of the number of multiplication operations, which are
known to be costly in terms of energy consumption. We provide applications of
MAPI to PCA-based image reconstruction as well as to graph-based ranking
algorithms. When compared to RPI, MAPI not only typically converges much
faster, but also provides superior performance.

Index Terms—  Power iteration, multiplication-free algorithms, principal
component analysis, PageRank algorithm.

##  1 Introduction

Let $\mathbf{A}$ be a diagonalizable matrix and $\mathbf{b}$ be some initial
vector. Power iteration is described by the update equation or recurrence
relation

$\displaystyle\mathbf{b}\leftarrow\frac{\mathbf{A}\mathbf{b}}{\|\mathbf{A}\mathbf{b}\|},$
(1)

where $\|\cdot\|$ represents the Euclidean norm. It is well-known that (1)
converges to the eigenvector of $\mathbf{A}$ corresponding to the dominant
eigenvalue. Note that the normalization can be omitted, i.e.,
$\mathbf{b}\leftarrow\mathbf{A}\mathbf{b}$ results in the same direction as
the dominant eigenvector.

A primary application of the power iteration (1) is conventional (or, $L^{2}$)
principal component analysis (PCA): Suppose that we collect members of a zero-
mean $D$-dimensional dataset
$\\{\mathbf{x}_{1},\ldots,\mathbf{x}_{N}\\}\subset\mathbb{R}^{D}$ to a
$D\times N$ matrix $\mathbf{X}=[\mathbf{x}_{1}\ \mathbf{x}_{2}\ ...\
\mathbf{x}_{N}]\in\mathbb{R}^{D\times N}$. Consider the corresponding sample
covariance matrix

$\displaystyle\mathbf{C}=\frac{1}{N}\mathbf{X}\mathbf{X}^{T}.$ (2)

The first principal vector, say $\mathbf{p}_{1}$, is then the dominant
eigenvector of $\mathbf{C}$. The eigenvector $\mathbf{p}_{1}$ can be extracted
via the power iteration in (1) applied with the substitution
$\mathbf{A}=\mathbf{C}$. The $i$th principal vector can then be extracted via
power iteration on
$(\mathbf{I}-\sum_{k=1}^{i-1}\mathbf{p}_{k}\mathbf{p}_{k}^{T})\mathbf{C}$.

In the process of computing the PCA of $\mathbf{C}$ through (1), one
effectively computes the Euclidean inner product of the candidate principal
vector $\mathbf{b}$ with the dataset elements
$\mathbf{x}_{1},\ldots,\mathbf{x}_{N}$ at each iteration. The fundamental
underlying operation for each such Euclidean inner product is the
multiplication of the components of $\mathbf{b}$ with the components of
$\mathbf{x}_{i}$. On the other hand, it is well-known that multiplication
operation will overamplify the effects of outliers or noise in the dataset. To
increase robustness, our idea is to replace the Euclidean inner products that
appear in (1) with Multiplication Avoiding Vector Products (MAVPs), which were
originally developed in the context of neural networks [1, 2, 3, 4, 5, 6]. Two
of the MAVPs satisfy the Mercer-type kernel requirement [7, 8]. The resulting
multiplication-avoiding power iteration (MAPI) becomes a power iteration in
the Reproducing Kernel Hilbert Space (RKHS) defined by the kernel. Our MAVPs
utilize the sign information of vector components to preserve the correlative
property of multiplication; see [9] for another work that utilizes the related
idea of sign regression.

Energy efficiency and improved computational complexity provides another major
motivation for utilizing MAPIs in lieu of ordinary power iteration. In fact,
MAVPs rely only on minimum operations and sign changes, and avoid the energy
consuming multiplication operation. Therefore, compared to an Euclidean inner
product, an MAVP can be executed in an energy efficient manner in many
processors. The same benefits transfer to MAPI, which utilize MAVPs.

We reemphasize that RPI and MAPI will result in different vectors upon
convergence; i.e. they are not equivalent. We expect MAPI to be more robust
than RPI as it utilizes MAVPs that induce the $\ell_{1}$-norm. We also expect
the MAPI to be more energy- and computationally-efficient as it avoids the
multiplication operation.

Kernel-PCA was introduced by Scholkopf et al. [8, 10]. It has been applied to
many signal and image processing problems [11, 12, 13, 14]. Related work
includes recursive $\ell_{1}$-PCA and PCA methods using the similarity
measures related with the $\ell_{1}$-norm [15, 16, 17, 18, 7]. However all of
the above mentioned methods are computationally costly in large covariance
matrices because they either require the eigen-decomposition of the covariance
matrix or the solution of a complex optimization problem. To the best of our
knowledge, this paper is the first paper describing power iterations in the
kernel domain. This is probably due to the fact that other kernel PCA methods
require costly kernel computations compared to the regular vector dot product
[14, 19, 20, 21, 22]. On the other hand, MAPI kernel operations are more
energy efficient than the dot product. In the context of PCA, MAPI can not
only improve robustness but provide significant improvements in computational
complexity. MAPI provides similar benefits in other applications in which the
power iteration is used. Examples will be provided throughout the paper.

The rest of the paper is organized as follows: In Section 2, we formally
introduce the MAPI. In Section 3, we describe applications of MAPI and report
corresponding results. Finally, in Section 4, we draw our main conclusions.

##  2 Methodology

In this section, we provide an overview of the MAVPs, and formally introduce
the MAPI method that utilizes MAVPs.

###  2.1 Multiplication-Avoiding Vector Products (MAVPs)

MAVPs were first studied in [1, 23] to develop a robust region covariance
matrix. They were used in computationally efficient neural networks. In this
work, we will utilize the MAVP as

$\displaystyle\mathbf{w}^{T}\odot\mathbf{x}$
$\displaystyle\triangleq\sum_{i=1}^{n}\mathrm{sign}(w_{i}x_{i})\min(|w_{i}|,|x_{i}|).$
(3)

We define another related dot-product as follows:

$\displaystyle\mathbf{w}^{T}\odot_{m}\mathbf{x}\triangleq\sum_{i=1}^{n}\mathbf{1}\left(\mathrm{sign}(w_{i})=\mathrm{sign}(x_{i})\right)\min(|w_{i}|,|x_{i}|)$
(4)

where $\mathbf{1}(\cdot)$ is the indicator function. We call (3) as min1
operation and (4) as min2 operation. In the following sessions, we will denote
the two min-operations as $\oplus=\\{\odot,\odot_{m}\\}$. Note that unlike an
ordinary Euclidean inner product, (3) and (4) do not contain any
multiplication operations. Energy efficiency of $\oplus$ operation varies from
processor to processor. For example, multiplication-based regular dot-product
operation consumes about 4 times more energy compared to the multiplication
avoiding dot product operations defined in (3) and (4) in compute-in-memory
(CIM) implementation at 1 GHz operating frequency [6]. The MAVPs induce the
$\ell_{1}$-norm as
$\mathbf{x}^{T}\oplus\mathbf{x}=\sum_{i=1}^{n}\min(|x_{i}|,|x_{i}|)=\|\mathbf{x}\|_{1}$.

The MAVPs can be extended to matrix multiplications as follows: Let
$\mathbf{W}=[\mathbf{w}_{1}\cdots\mathbf{w}_{m}]\in\mathbb{R}^{n\times m}$ and
$\mathbf{X}=[\mathbf{x}_{1}\cdots\mathbf{x}_{p}]\in\mathbb{R}^{n\times p}$ be
arbitrary matrices. We define $\mathbf{W}^{T}\oplus\mathbf{X}$ as an $n\times
p$ matrix whose entry in the $i$th row, $j$th column is
$\mathbf{w}_{i}^{T}\oplus\mathbf{x}_{p}$. Thus, the definition is similar to
matrix multiplication $\mathbf{W}^{T}\mathbf{X}$ by only changing the element-
wise product to element-wise MAVP.

Recall from Section 1 that given a dataset $\mathbf{X}=[\mathbf{x}_{1}\
\mathbf{x}_{2}\ ...\ \mathbf{x}_{N}]\in\mathbb{R}^{D\times N}$, the
eigendecomposition of $\mathbf{C}=\frac{1}{N}\mathbf{X}\mathbf{X}^{T}$ yields
the ordinary PCA. Alternatively, one can construct the sample covariance
matrix through MAVPs. In particular, in [7], we considered the
eigendecomposition of the “min-covariance matrix”

$\mathbf{A}=\tfrac{1}{N}\mathbf{X}\oplus\mathbf{X}^{T}$ (5)

We have shown in [7] that the resulting “Min-PCA” provides better resilience
against impulsive noise than regular PCA in image reconstruction experiments.

###  2.2 Multiplication-Avoiding Power Iteration (MAPI)

We are now ready to introduce the MAPI. We replace the standard products in
(1) with a MAVP. To further reduce the computational complexity, we replace
the normalization by $\ell_{2}$-norm by a normalization by $\ell_{1}$-norm.
Note that the calculation of the $\ell_{2}$-norm requires multiplications
while $\ell_{1}$-norm can be calculated via additions only. Our revisions
yield the iteration

$\mathbf{w}\leftarrow\frac{\mathbf{A}\oplus\mathbf{w}}{||\mathbf{A}\oplus\mathbf{w}||_{1}}$
(6)

We have observed that such a change of normalization does not effect the final
performance greatly; also see [24, 25] for studies that utilize an $\ell_{1}$
normalization. The final MAPI algorithm is shown in Algorithm 1. For
applications to PCA, we can normalize the final $\mathbf{w}_{t}$ by its
$l_{2}$-norm for extraction of subsequent principal vectors.

Algorithm 1 MAPI

0: $\mathbf{A}\in\mathbb{R}^{N\times N}$, iteration times $T$.

1: Initialize $\mathbf{w}_{0}$ as a random vector with $||\mathbf{w}_{0}||=1$;

2: for $t=0,1,...,T-1$ do

3: $\mathbf{w}_{t+1}=\mathbf{A}\oplus\mathbf{w}_{t}$;

4: $\mathbf{w}_{t+1}\leftarrow\mathbf{w}_{t+1}/||\mathbf{w}_{t+1}||_{1}$;

5: end for

6: (optional) $\mathbf{w}_{T}=\mathbf{w}_{T}/||\mathbf{w}_{T}||_{2}$;

7: return dominant pseudo-eigenvector $\mathbf{w}_{T}$.

Each step of the MAPI defined in (6) corresponds to a transformation in the
RKHS. As a result the convergence of MAPI depends on the matrix ${\bf A}$.
Since we normalize the iterations in (8) at each step, the iterates are
bounded and they satisfy $||{\bf w}_{t}||_{1}=1$. We have observed that the
MAPI in Algorithm 1 converges in all experiments that we have tried and the
resulting vector can be used in practical applications. For certain MF
operators, convergence can even be proved analytically. In particular, let

$\displaystyle\mathbf{w}^{T}\lozenge\mathbf{x}=\sum_{i=1}^{n}\left(\mathrm{sign}(x_{i})y_{i}+x_{i}\mathrm{sign}(y_{i})\right)$
(7)

The operator $\lozenge$ has proved very useful for neural network applications
[2]. The convergence behavior of the MAPI is particularly simple, at least in
the case of positive entries. Let
$\mathbf{A}=[\mathbf{a}_{1}^{T}\cdots\mathbf{a}_{n}^{T}]$.

######  Proposition 1.

For any initial vector $\mathbf{w}$ and matrix $\mathbf{A}$ with positive
entries, the iterations
$\mathbf{w}\\!\leftarrow\\!\frac{\mathbf{A}\lozenge\mathbf{w}}{||\mathbf{A}\lozenge\mathbf{w}||_{1}}$
converges to the vector

$\displaystyle\frac{[1+\|\mathbf{a}_{1}\|_{1}\cdots
1+\|\mathbf{a}_{n}\|_{1}]^{T}}{\sum_{i=1}^{n}(1+\|\mathbf{a}_{i}\|_{1})}$ (8)

in at most two iterations.

###### Proof.

The proof is by calculation. After the first iteration, one obtains a vector
$\mathbf{c}=\mathbf{A}\lozenge\mathbf{w}=[c_{1}\cdots c_{N}]^{T}$ whose $i$th
component is

$\displaystyle
c_{i}\triangleq\frac{\mathbf{a}_{1}^{T}\lozenge\mathbf{b}}{\sum_{j=1}^{n}(\mathbf{a}_{j}^{T}\lozenge\mathbf{b})}=\frac{\|\mathbf{a}_{i}\|_{1}+\|\mathbf{b}\|_{1}}{\sum_{j=1}(\|\mathbf{a}_{j}\|_{1}+\|\mathbf{b}\|_{1})}.$
(9)

Note that $\|\mathbf{c}\|_{1}=1$. Therefore, after the second iteration, we
obtain a vector whose $i$th component is

$\displaystyle\frac{\|\mathbf{a}_{i}\|_{1}+\|\mathbf{c}\|_{1}}{\sum_{j=1}(\|\mathbf{a}_{j}\|_{1}+\|\mathbf{c}\|_{1})}$
(10)

The result follows since $\|\mathbf{c}\|_{1}=1$. ∎

The general case for different operators and arbitrary entries for matrices is
left as future work.

Note that MAPI only requires $N$ divisions per iteration as opposed to the
RPI, which requires $N^{2}$ multiplications and $N$ divisions. Therefore, we
expect MAPI to consume significantly less energy compared to RPI in most
processors.

##  3 Applications and Numerical Results

###  3.1 Image Reconstruction Example

We consider image reconstruction example studied in [15]. Our experiment
follows the same structure in [15, 7]. We use regular power iteration (RPI)
and MAPI to compare the image reconstruction results of the MAPI. In this
experiment, the image size $D\times D=128\times 128$, and the pixel values are
in the range of $[0,1]$. Suppose that we want to reconstruct a gray-scale
image (Fig. LABEL:fig:_Image_Reconstruction_original_3b) from its $N=10$
occluded versions. As Fig. LABEL:fig:_Image_Reconstruction_occluded_3b shows,
the corrupted images are created by partitioning the original image into
sixteen tiles of size $32\times 32$ and replacing three arbitrarily selected
tiles by $32\times 32$ gray-scale noise patches. The pixel values of the noise
patches are independent and identically distributed random variables that are
uniform on $[0,1]$. The reconstruction algorithm is described in Algorithm 2.
In particular, in Step 5 and Step 6, we obtain the first two dominant
generalized eigenvectors of the min-covariance matrix of images via MAPI. We
then reconstruct the image using these two generalized eigenvectors in Line 8.

Algorithm 2 Image Reconstruction via MAPI

0: $N$ corrupted versions of an image:
$\mathbf{I}_{1},\mathbf{I}_{2},...,\mathbf{I}_{N}\in\mathbb{R}^{D\times D}$,
pixels are in range of $[0,1]$.

1: Reshape $\mathbf{I}_{i}$ to vector
$\mathbf{v}_{i}\in\mathbb{R}^{D^{2}\times 1}$ for $i=1,\ldots,N$;

2: $\mathbf{V}=[\mathbf{v}_{1}\ \mathbf{v}_{2}\ ...\
\mathbf{v}_{N}]\in\mathbb{R}^{D^{2}\times N}$;

3: Reduce mean:
$\mathbf{m}=\text{mean}(\mathbf{V},2)\\!\in\\!\mathbb{R}^{D^{2}\times
1},\mathbf{V}\\!=\\!\mathbf{V}-\mathbf{m}$;

4: Construct the min-covariance matrix
$\mathbf{C}=\frac{1}{N-1}\mathbf{V}\oplus\mathbf{V}^{T}\in\mathbb{R}^{D^{2}\times
D^{2}}$ of $\mathbf{V}$;

5: Perform MAPI on $\mathbf{C}$ to obtain the dominant eigenvector
$\mathbf{w}_{1}\in\mathbb{R}^{D^{2}\times 1}$;

6: Perform MAPI on
$\mathbf{C}-(\mathbf{w}_{1}\oplus\mathbf{w}_{1}^{T})\mathbf{C}$ to obtain the
second dominant eigenvector $\mathbf{w}_{2}\in\mathbb{R}^{D^{2}\times 1}$;

7: $\mathbf{w}=[\mathbf{w}_{1}\ \mathbf{w}_{2}]\in\mathbb{R}^{D^{2}\times 2}$

8: Reconstruct image:
$\mathbf{\hat{v}}=(\mathbf{w}\oplus\mathbf{w}^{T})(\mathbf{v}_{i}-\mathbf{m})+\mathbf{m}$;

9: Reshape $\mathbf{\hat{v}}$ back to the matrix form $\mathbf{\hat{I}}$;

10: return reconstructed image $\mathbf{\hat{I}}$.

In Table 1, we compare the peak signal-to-noise ratio (PSNR) performances
provided by different algorithms. As Fig.
LABEL:fig:_Image_Reconstruction_l2_3b and
LABEL:fig:_Image_Reconstruction_min_3b show, MAPI provides higher PSNRs than
RPI, globally. The average PSNR of min2-PI, which represents MAPI via the min2
operator in (4), is 1.62 dB higher than RPI. The MAPI based reconstruction
(24.8 dB) is also superior to recursive $\ell_{1}$-PCA (24.2 dB). Thus, based
on the experimental PSNR results in Table 1, we can conclude that MAPI is as
robust as $\ell_{1}$-PCA based restoration. We also expect the computational
cost MAPI to be much lower than $\ell_{1}$-PCA methods. In fact, for an
$N\times N$ covariance matrix, RPI needs $O(N^{2})$ multiplications per
iteration while MAPI avoids a quadratic number of multiplications and needs
$O(N^{2})$ additions only. Note that $\ell_{1}$-PCA utilizes regular power
iteration updates and need $O(N^{2})$ multiplications also. We also
experimentally validated in Matlab that to compute the first principal vector,
the recursive $\ell_{1}$-PCA and MAPI methods need 4.54 and 3.24 seconds,
respectively.

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 1: Image reconstruction examples: (a) original images; (b) occluded images; (c) recursive $\ell_{1}$-PCA [15] results; (d) PCA using RPI results; (e) min1-PI results; (f) min2-PI results. MAPI is as robust as $\ell_{1}$-PCA based restoration. Table 1: Image Reconstruction PSNRs (dB).  Image |  Occlu- ded |  Recursive $\ell_{1}$-PCA |  RPI |  Min1-PI |  Min2-PI
---|---|---|---|---|---
Statue 1 |  17.67 |  26.86 |  26.67 |  27.12 |  27.08
Statue 2 |  16.87 |  25.02 |  24.61 |  25.45 |  25.34
Earth |  14.91 |  21.74 |  22.13 |  20.78 |  25.03
Pikachu |  15.29 |  22.71 |  18.86 |  22.70 |  22.58
Flower |  16.40 |  24.41 |  21.22 |  24.59 |  24.44
Orange |  15.77 |  23.69 |  23.59 |  22.50 |  26.27
Cat |  16.91 |  24.82 |  24.80 |  24.28 |  24.52
Lenna |  16.77 |  24.71 |  22.86 |  25.59 |  24.73
Food |  15.94 |  23.85 |  22.72 |  23.68 |  23.70
Car |  15.42 |  23.37 |  23.15 |  23.43 |  23.41
Cobra |  16.81 |  25.10 |  22.31 |  25.02 |  25.57
River |  17.27 |  25.23 |  24.50 |  25.07 |  25.88
Butterfly |  16.66 |  24.91 |  24.40 |  23.93 |  23.92
Bridge |  15.66 |  22.91 |  22.23 |  22.55 |  24.34
Average |  16.31 |  24.24 |  23.15 |  24.05 |  24.77

###  3.2 Comparison with Stochastic PCA Method

We now compare the MAPI with the stochastic power iteration method on a
synthetic dataset as in [26]. The synthetic dataset
$\smash{\mathbf{X}\in\mathbb{R}^{10^{6}\times 10}}$ is generated using
singular value decomposition. In detail, for a diagonal matrix
$\mathbf{\Sigma}=\text{diag}\\{1,\sqrt{0.9},...,\sqrt{0.9}\\}\in\mathbb{R}^{10\times
10}$, and random orthogonal matrices $\mathbf{U}\in\mathbb{R}^{10^{6}\times
10},\,\mathbf{V}\in\mathbb{R}^{10\times 10}$, it is guaranteed that the matrix
$\mathbf{C}=\mathbf{X}^{T}\mathbf{X}$ has an eigen-gap of $0.1$, where
$\mathbf{X}=\mathbf{U\Sigma V}^{T}$.

MAPI method can be also implemented into the mini-batch power method with
momentum algorithm [26, Algorithm 1]. We apply MAPI on
$\mathbf{A}=\mathbf{X}^{T}\odot\mathbf{X}$ with the same mini-batch strategy,
as Algorithm 3 shows. The difference between Algorithm 3 in this paper and
Algorithm 1 in [26] is that we change all regular multiplication operations
with our min1 operation (3) and normalize the iterates using the
$\ell_{1}$-norm. At the end of iterations we normalize the final vector using
the $\ell_{2}$-norm of the vector. In this experiment, we cannot use min2-PI
operation because the min2 operation (4) cannot return negative entries. The
vector $\mathbf{u}_{1}$ can have negative entries.

(a)

(b)

Fig. 2: RPI versus MAPI on a synthetic dataset
$\mathbf{X}\in\mathbb{R}^{10^{6}\times 10}$ where the covariance matrix has
eigen-gap $\Delta=0.1$.

In Fig. 2, we compute
$(1-(\mathbf{w}_{t}^{T}\mathbf{u}_{1}/||\mathbf{w}_{t}||)^{2})$ at each
iteration, where $\mathbf{u}_{1}$ is the dominant eigenvector obtained from
eigendecomposition as in [26]. The MAPI iteration converges to a vector very
close to the actual eigenvector because
$(1-(\mathbf{w}_{t}^{T}\mathbf{u}_{1}/||\mathbf{w}_{t}||)^{2})$ is very close
to zero as shown in Fig. LABEL:fig:_stocastic_a for $t$ greater than 20. Such
preservation of the eigenvector information can be considered as a direct
analogue of Bussgang’s theorem [27, 28]. However, $\mathbf{w}_{T}$ obtained
using the MAPI method is not exactly the same as the eigenvector
$\mathbf{v}_{1}$ as we see from Fig. LABEL:fig:_stocastic_b whose vertical
axis has a different range from Fig. LABEL:fig:_stocastic_a. This is expected
because we perform the iterations in the RKHS domain. Nevertheless, based on
our observation, ranks of the entries of $\mathbf{w}_{T}$ obtained from the
min1-PI are the same as the ranks obtained from $\mathbf{w}_{T}$ of the RPI.
After 100 iterations, $\mathbf{w}_{T}=[$-0.143, 0.417, -0.117, 0.386, 0.129,
-0.132, -0.433, -0.210, -0.577, -0.211] from the RPI and
$\mathbf{w}_{T}=[$-0.165, 0.417, -0.137, 0.389, 0.148, -0.151, -0.431, -0.236,
-0.536, -0.237] from the min1-PI. When we order from the largest to the
smallest, both vectors produce the same ranks {2, 4, 5, 3, 6, 1, 8, 10, 7, 9}.
Hence, min1-PI converges significantly faster than RPI and both algorithms
produce the same final ranks. In Section 3.3, we will show how these
properties allow us to employ MAPI in graph-based ranking algorithms such as
Google PageRank [29]. Moreover, if we need
$(1-(\mathbf{w}_{t}^{T}\mathbf{u}_{1}/||\mathbf{w}_{t}||)^{2})$ to reach
“absolute” $0$, we can optimize the vector via the MAPI first and then switch
to the RPI for further convergence.

Algorithm 3 Mini-batch MAPI with Momentum

0: Data $\mathbf{X}\in\mathbb{R}^{N\times D}$, iteration times $T$, batch size
$s$, momentum parameter $\beta$.

1: Initialize $\mathbf{w_{0}}$ as a random vector with $||\mathbf{w_{0}}||=1$;

2: for $t=0,1,\ldots,T-1$ do

3: Generate a mini batch of independent and identically distributed samples
$B=\\{\tilde{\mathbf{A}}_{t_{1}},...,\tilde{\mathbf{A}}_{t_{s}}\\}$

4:
$\mathbf{w}_{t+1}=(\frac{1}{s}\sum_{i=1}^{s}\tilde{\mathbf{A}}_{t_{i}})\odot\mathbf{w}_{t}-\beta\mathbf{w}_{t-1}$;

5:
$\mathbf{w}_{t}=\mathbf{w}_{t}/||\mathbf{w}_{t+1}||_{1},\mathbf{w}_{t+1}=\mathbf{w}_{t+1}/||\mathbf{w}_{t+1}||_{1}$;

6: end for

7: (optional) $\mathbf{w}_{T}=\mathbf{w}_{T}/||\mathbf{w}_{T}||$;

8: return dominant pseudo-eigenvector $\mathbf{w}_{T}$.

###  3.3 PageRank Algorithm Using MAPI

PageRank algorithm uses the hyperlink structure of the web to view inlinks
into a page as a recommendation of that page from the author of the inlinking
page [30]. Specifically, it relies on applying power iteration to the matrix

$\mathbf{G}=\alpha\mathbf{H}+\frac{1-\alpha}{N}\mathbf{1}$ (11)

where $\mathbf{H}$ is the network adjacent matrix, $\mathbf{1}$ is an all-ones
matrix, and $\alpha=0.85$ is known as a damping factor.

We first execute the PageRank algorithm with RPI or MAPI on the network graph
in Fig. LABEL:fig:_Network_Graph. We compute
$||\mathbf{w}_{t}-\mathbf{w}_{t-1}||$ after each iteration to compare the
convergence.

(a)

(b)

Fig. 3: A PageRank example.

We apply one $\ell_{2}$-normalization after the final iteration for easier
comparison. After 20 iterations, RPI yields $w_{T}=[$0.634, 0.345, 0.434,
0.535], and MAPI provides $w_{T}=[$0.568, 0.349, 0.520, 0.535]. Consequently,
although the weights are different, both power iteration methods provide the
same index ranks {1, 4, 3, 2}.

We further use the PageRank algorithm with both RPI and MAPI on Gnutella08
[31] and Gnutella09 [32] peer-to-peer network datasets. Gnutella08 contains
6,301 nodes with 20,777 edges, and Gnutella09 contains 8,114 nodes with 26,013
edges. The convergence curves are shown in Fig. 4. MAPI still converges
remarkably faster than RPI for both networks. However, because the size of the
networks are very large, the ranks provided by the two methods are not the
same. After 10 iterations, from Gnutella08, the top-10 ranks of the indices
from the RPI are {367, 249, 145, 264, 266, 123, 127, 122, 1317, 5}, while the
MAPI yields {266, 123, 367, 127, 424, 249, 145, 264, 427, 251}. On the other
hand, from Gnutella09, the top-10 ranks of the indices from the RPI are {351,
563, 822, 534, 565, 825, 1389, 1126, 356, 530}, while the MAPI returns {351,
822, 51, 1389, 563, 565, 530, 825, 356, 1074}. Therefore, there are 7 common
top-10 ranks of indices {367, 249, 145, 265, 226, 123, 127} from the
6,301-node dataset Gnutella08 and 8 common top-10 ranks of indices {351, 563,
822, 565, 825, 1389, 356, 530} from the 8,114-node dataset Gnutella09.
Therefore, if we implement the MAPI in the page-rank-based web search system,
the links displayed on the first page is very close to the conventional page-
rank based system. The search time can also be reduced significantly.

(a)

(b)

Fig. 4: PageRank convergence curves on Gnutella.

##  4 Conclusion

We proposed two types of multiplication-avoiding power iteration (MAPI)
algorithms which replace the standard vector product operations in regular
power iteration (RPI) with multiplication-free vector products. The MAPI is
energy efficient because it significantly reduces the number of multiplication
operations, which are known to be costly in terms of energy consumption in
many processors. MAPI is as robust as $\ell_{1}$ principal component analysis
as its kernels are related with the $\ell_{1}$-norm. According to our graph-
based ranking experiment, the MAPI is also superior to the RPI in terms of
convergence speed and energy efficiency. Though the final ranking obtained
from the MAPI is not always exactly identical to the ranking obtained from the
RPI but they are very close to each other.

## References

  * [1] H. Tuna, I. Onaran, and A. E. Cetin, “Image description using a multiplier-less operator,” _IEEE Signal Processing Letters_ , vol. 16, no. 9, pp. 751–753, 2009.
  * [2] A. Afrasiyabi, B. Nasir, O. Yildiz, F. T. Y. Vural, and A. E. Cetin, “An energy efficient additive neural network,” in _SIU_ , 2017.
  * [3] A. Afrasiyabi, D. Badawi, B. Nasir, O. Yildi, F. T. Y. Vural, and A. E. Çetin, “Non-euclidean vector product for neural networks,” in _ICASSP_ , 2018.
  * [4] H. Pan, D. Badawi, X. Zhang, and A. E. Cetin, “Additive neural network for forest fire detection,” _Signal, Image and Video Processing_ , pp. 1–8, 2019\.
  * [5] T. Ergen, A. H. Mirza, and S. S. Kozat, “Energy-efficient lstm networks for online learning,” _IEEE Trans. Neural Nets. Learning Sys._ , vol. 31, no. 8, pp. 3114–3126, 2019.
  * [6] S. Nasrin, D. Badawi, A. E. Cetin, W. Gomes, and A. R. Trivedi, “Mf-net: Compute-in-memory sram for multibit precision inference using memory-immersed data conversion and multiplication-free operators,” _IEEE Trans. Circuits and Systems_ , 2021.
  * [7] H. Pan, D. Badawi, E. Koyuncu, and A. E. Cetin, “Robust principal component analysis using a novel kernel related with the l1-norm,” in _EUSIPCO_ , 2021\.
  * [8] B. Schölkopf, A. Smola, and K. Müller, “Kernel principal component analysis,” in _Artificial Neural Networks — ICANN’97_ , ser. Lecture Notes in Computer Science, vol 1327, Heidelberg, Berlin, Germany, 1997.
  * [9] E. Eweda, “Comparison of rls, lms, and sign algorithms for tracking randomly time-varying channels,” _IEEE Transactions on Signal Processing_ , vol. 42, no. 11, pp. 2937–2944, 1994.
  * [10] B. Schölkopf, A. Smola, and K.-R. Müller, “Nonlinear component analysis as a kernel eigenvalue problem,” _Neural computation_ , vol. 10, no. 5, pp. 1299–1319, 1998.
  * [11] H. Goldberg, H. Kwon, and N. M. Nasrabadi, “Kernel eigenspace separation transform for subspace anomaly detection in hyperspectral imagery,” _IEEE Geoscience and Remote Sensing Letters_ , vol. 4, no. 4, pp. 581–585, 2007.
  * [12] N. M. Nasrabadi, “Hyperspectral target detection: An overview of current and future challenges,” _IEEE Signal Processing Magazine_ , vol. 31, no. 1, pp. 34–44, 2013.
  * [13] H. Van Nguyen, V. M. Patel, N. M. Nasrabadi, and R. Chellappa, “Kernel dictionary learning,” in _2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2012, pp. 2021–2024.
  * [14] H. Hoffmann, “Kernel pca for novelty detection,” _Pattern recognition_ , vol. 40, no. 3, pp. 863–874, 2007.
  * [15] P. P. Markopoulos, G. N. Karystinos, and D. A. Pados, “Optimal algorithms for $l_{1}$-subspace signal processing,” _IEEE Trans. Signal Process._ , vol. 62, no. 19, pp. 5046–5058, 2014.
  * [16] P. P. Markopoulos, S. Kundu, S. Chamadia, and D. A. Pados, “Efficient l1-norm principal-component analysis via bit flipping,” _IEEE Trans. Signal Process._ , vol. 65, no. 16, pp. 4252–4264, 2017.
  * [17] P. P. Markopoulos and F. Ahmad, “Indoor human motion classification by l1-norm subspaces of micro-doppler signatures,” in _2017 IEEE Radar Conference (RadarConf)_. IEEE, 2017, pp. 1807–1810.
  * [18] P. P. Markopoulos, M. Dhanaraj, and A. Savakis, “Adaptive l1-norm principal-component analysis with online outlier rejection,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 6, pp. 1131–1143, 2018.
  * [19] Y. Xiao, H. Wang, and W. Xu, “Model selection of gaussian kernel pca for novelty detection,” _Chemometrics and Intelligent Laboratory Systems_ , vol. 136, pp. 164–172, 2014.
  * [20] C. Kim and D. Klabjan, “L1-norm kernel pca,” _arXiv preprint arXiv: 1709.10152_ , 2017.
  * [21] C. Varon, C. Alzate, and J. A. Suykens, “Noise level estimation for model selection in kernel pca denoising,” _IEEE TNNLS_ , vol. 26, no. 11, pp. 2650–2663, 2015.
  * [22] S. Battaglino and E. Koyuncu, “A generalization of principal component analysis,” in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 3607–3611.
  * [23] C. E. Akbas, A. Bozkurt, M. T. A. Tunc, H. Aslanoglu, and A. E. Cetin, “L1 norm based multiplication-free cosine similarity measures for big data analysis,” in _IWCIM_ , 2014.
  * [24] F. Lin and W. W. Cohen, “Power iteration clustering,” in _ICML_ , 2010.
  * [25] O. Arikan, A. E. Cetin, and E. Erzin, “Adaptive filtering for non-gaussian stable processes,” _IEEE Signal Processing Letters_ , vol. 1, no. 11, pp. 163–165, 1994.
  * [26] P. Xu, B. He, C. De Sa, I. Mitliagkas, and C. Re, “Accelerated stochastic power iteration,” in _International Conference on Artificial Intelligence and Statistics_. PMLR, 2018, pp. 58–67.
  * [27] G. Giunta, G. Jacovitti, and G. Scarano, “Bussgang gaussianity test for stationary series,” in _Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics_. IEEE, 1997, pp. 434–437.
  * [28] G. Giunta, G. Jacovitti, and G. Scavano, “Bussgang test: A powerful non-gaussianity test,” in _9th European Signal Processing Conference (EUSIPCO 1998)_. IEEE, 1998, pp. 1–4.
  * [29] L. Page, S. Brin, R. Motwani, and T. Winograd, “The pagerank citation ranking: Bringing order to the web.” Stanford InfoLab, Tech. Rep., 1999.
  * [30] C. Benincasa, A. Calden, E. Hanlon, M. Kindzerske, K. Law, E. Lam, J. Rhoades, I. Roy, M. Satz, E. Valentine _et al._ , “Page rank algorithm,” _Department of Mathematics and Statics, University of Massachusetts, Amherst, Research_ , 2006.
  * [31] “Gnutella peer-to-peer network,” https://snap.stanford.edu/data/p2p-Gnutella08.html Accessed: 2021-10-03.
  * [32] “Gnutella peer-to-peer network,” https://snap.stanford.edu/data/p2p-Gnutella09.html Accessed: 2021-10-03.


# Beyond Borders: Investigating Cross-Jurisdiction Transfer in
Legal Case Summarization

Santosh T.Y.S.S1,Vatsal Venkatkrishna2, Saptarshi Ghosh2, Matthias Grabmair1
1Technical University of Munich, Germany
2Indian Institute of Technology, Kharagpur, India
{santosh.tokala<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Legal professionals face the challenge of managing an overwhelming volume of
lengthy judgments, making automated legal case summarization crucial. However,
prior approaches mainly focused on training and evaluating these models within
the same jurisdiction. In this study, we explore the cross-jurisdictional
generalizability of legal case summarization models. Specifically, we explore
how to effectively summarize legal cases of a target jurisdiction where
reference summaries are not available. In particular, we investigate whether
supplementing models with unlabeled target jurisdiction corpus and extractive
silver summaries obtained from unsupervised algorithms on target data enhances
transfer performance. Our comprehensive study on three datasets from different
jurisdictions highlights the role of pre-training in improving transfer
performance. We shed light on the pivotal influence of jurisdictional
similarity in selecting optimal source datasets for effective transfer.
Furthermore, our findings underscore that incorporating unlabeled target data
yields improvements in general pre-trained models, with additional gains when
silver summaries are introduced. This augmentation is especially valuable when
dealing with extractive datasets and scenarios featuring limited alignment
between source and target jurisdictions. Our study provides key insights for
developing adaptable legal case summarization systems, transcending
jurisdictional boundaries.

##  1 Introdution

Legal professionals, including lawyers, judges, and paralegals, are often
inundated with an overwhelming amount of case judgements containing intricate
textual content encompassing multiple issues and arguments, making it time-
consuming to read and comprehend every document in detail manually. To address
this, many legal information systems offer case summaries authored by legal
experts. There has been a lot of academic research over the years to
automatically produce the summaries of legal case judgements, which can help
reduce the human cost effectively Bhattacharya et al. (2021); Shukla et al.
(2022); Agarwal et al. (2022).

Traditional legal case judgement summarization methods have primarily focused
on extractive summarization, where crucial sentences are selected from source
documents. Initially, unsupervised approaches were prominent, tailored to
capture legal discourse nuances without requiring training data. More
recently, supervised extractive summarizers have emerged, relying on expert-
annotated summaries for training. Recognizing the limitations of extractive
methods in providing comprehensive context and coherence, researchers have
shifted towards abstractive summarization, leveraging recent advances in
unsupervised pre-training (e.g., Devlin et al. 2019; Lewis et al. 2020), legal
domain-specific continued pre-training (e.g., Paul et al. 2022; Chalkidis et
al. 2020; Gururangan et al. 2020), and summarization task-specific pre-
training (e.g., Zhang et al. 2020).

The task of summarizing legal case judgements presents distinctive challenges
due to several factors, such as the legal domain vocabulary being slightly
different in different jurisdictions (e.g., country / court), as well as the
complex sentence and discourse structures influenced by various writing styles
that are in use in different jurisdictions. The current strategy to design a
legal case summarization system for a _new_ jurisdiction (which we call
_target jurisdiction_) either involves unsupervised methods or collecting
expert annotated datasets to fine-tune supervised summarization models. Though
these supervised models usually show improved performance compared to
unsupervised methods, this often demands a large number of expert summaries,
making it costly and less adaptable to new jurisdictions. This prompts us to
ask how we can build an effective summarization system for a target
jurisdiction without data annotation efforts. In this regard, we seek to
investigate whether models trained on some other jurisdiction for which
training data exists (which we call as _source jurisdiction_) can generate
better summaries than unsupervised methods for the target jurisdiction. Thus,
we evaluate the cross-jurisdiction generalizability of different legal case
summarization systems and further propose an adversarial training-based setup
to improve the cross-jurisdiction transfer performance, paving the way for the
development of summarization systems that are generalizable in real-world
legal scenarios. We summarize our three primary research questions:

RQ1: In situations where reference summaries are unavailable to train
supervised models for a specific target jurisdiction, can supervised
summarization models, trained using data from a different jurisdiction (source
jurisdiction) outperform unsupervised methods in generating more effective
summaries? What criteria should guide the selection of the most suitable
source jurisdiction for a specific target jurisdiction? Which models exhibit
better cross-jurisdiction summarization capabilities?

RQ2: Can we leverage unlabeled judgment data (only documents, not reference
summaries) from the target jurisdiction to improve transfer performance during
training supervised summarization models on source jurisdiction? This approach
is akin to unsupervised domain adaptation, where unlabelled target inputs are
used when fine-tuning on a labelled source corpus. We employ an adversarial
domain discriminator method Ganin et al. (2016) to learn the jurisdiction-
agnostic feature representations to facilitate cross-jurisdictional transfer.

RQ3: Can extractive silver summaries on the target jurisdiction data, obtained
using unsupervised summarization algorithms without manual human annotation,
further improve transfer performance in addition to unlabelled target
jurisdiction data? These silver summaries aid the decoder to grasp semantic
nuances in target jurisdiction, complementing adversarial domain
discriminator.

To answer these questions, we conducted a comprehensive experimental analysis
on three datasets from different jurisdictions. We find that supervised models
trained on a different source jurisdiction can yield better summarization in
the target jurisdiction, compared to unsupervised models. Further, we observe
that the choice of source dataset is highly influenced by jurisdictional
similarity rather than the dataset size or abstractiveness (RQ1). We witness
an improvement in transfer performance using domain discriminator-based
adversarial training. However, this improvement is contingent on the choice of
backbone model. For instance, we observe a decline in transfer performance
when applied to Legal pre-trained Pegasus in contrast to improvement in
general pre-trained BART model (RQ2). Finally, we notice an improvement when
adding silver summaries in scenarios when the dataset is extractive and there
is less similarity between source-target jurisdictions (RQ3).

##  2 Related Work

Legal Case Summarization: Traditional works in this area have primarily
employed extractive methods known for their faithfulness to the input case
document Bhattacharya et al. (2019). These encompass both unsupervised methods
Bhattacharya et al. (2021); Polsley et al. (2016); Farzindar (2004); Saravanan
et al. (2006); Mandal et al. (2021) and supervised approaches Agarwal et al.
(2022); Liu and Chen (2019); Zhong et al. (2019); Xu et al. (2021); Mandal et
al. (2021), employing diverse strategies such as knowledge engineering of
domain-specific features, redundancy handling using MMR, joint multi-task
learning with Rhetorical Role Labeling, leveraging argument structures,
integrating document-specific catchphrases.

While extractive approaches excel in faithfully representing source document
content, they face inherent challenges, including incomplete or incorrect
discourse, coreference resolution issues, and a lack of context, impacting the
readability of generated summaries Zhang et al. (2022). Hence, there is a
growing interest in exploring abstractive summarization methods for legal case
summarization Shukla et al. (2022); Ray et al. (2020); Schraagen et al.
(2022). Shukla et al. 2022 applied transformer-based pre-trained abstractive
models like BART, Legal-LED, and Legal-Pegasus to legal case summarization. To
address long legal documents, Moro and Ragazzi 2022 proposed chunking input
documents into semantically coherent segments, enabling these pre-trained
transformer-based models to summarize lengthy documents without truncation.
Recent efforts also aim to enhance factual accuracy in generated summaries by
employing entailment modules to select faithful candidates Feijo and Moreira
(2023). Additionally, there are efforts to understand the argumentative
structure in the legal documents to improve the performance of legal case
summarization models Xu and Ashley (2022); Elaraby and Litman (2022); Elaraby
et al. (2023). In contrast to the approaches mentioned, which train and
evaluate summarization models within specific jurisdictions, our work focuses
on evaluating the cross-jurisdiction generalizability of legal case
summarization systems.

Cross Domain/Dataset Generalization: This assessment in NLP has been
undertaken across various tasks, including constituent parsing Fried et al.
(2019), reading comprehension Talmor and Berant (2019), named entity
recognition Fu et al. (2020), summarization Chen et al. (2020), sentiment
analysis Yu and Jiang (2016) and relation extraction Bassignana and Plank
(2022), among others. Recently, Thakur et al. Thakur et al. (2021) created a
robust benchmark called BEIR (Benchmarking IR), for evaluation of retrieval
model generalization capabilities across various domains.

In the legal domain, Kumar et al. Kumar et al. (2022) explored the cross-
domain transferability of text generation models across four legal domains,
including congressional bills, treaties, legal contracts, and privacy
policies. However, the tasks they investigated within each domain exhibited
significant variations in terms of generated text, such as titles in the
privacy policy domain and actual summaries in congressional bills. In
contrast, our work specifically focuses solely on summarizing legal case
judgments. We consider each jurisdiction as a distinct domain characterized by
its unique vocabulary and discourse writing style. Furthermore, we explore
strategies to enhance cross-jurisdictional transfer performance through
adversarial learning-based domain adaptation.

Domain Adaptation: Domain adaptation mitigates covariate shifts between source
and target data distributions Ruder (2019). Unsupervised domain adaptation
(UDA) leverages labeled source data and unlabeled target data, extensively
studied in NLP and computer vision Daumé III (2009); Sun et al. (2016); Wang
et al. (2019); Shah et al. (2018); Ganin et al. (2016); Shen et al. (2018).
Recently, DA has extended to information retrieval Xin et al. (2022); Wang et
al. (2022); Yu et al. (2022), aided by benchmarks like BEIR. Domain adaptation
often employs domain discriminator-based adversarial training Ganin et al.
(2016) to align source and target distributions within an encoder, reducing
distribution disparities. In the legal domain, this technique has been applied
to case outcome classification, treating legal articles as separate domains to
assess transfer performance across violation predictions for different
articles Tyss et al. (2023).

We employ DA to create jurisdiction-invariant representations in legal case
summarization. However, relying solely on a jurisdiction-agnostic encoder
representation, as commonly used in classification tasks, is insufficient as
in the context of summarization, the decoder must also capture semantic
nuances to generate domain-specific summaries. Therefore, we explore the use
of silver summaries derived from unsupervised methods for target domain data.

##  3 Datasets

There are very few publicly available English legal case summary corpora
available. We use the following three datasets from prior works.

(i) UK-Abstractive dataset (abbreviated as ‘UK-Abs’) Shukla et al. (2022):
This dataset includes 793 case documents and their summaries from the UK
Supreme Court, spanning from 2009 onwards. The abstractive summaries are
sourced from the official press-released version available on the court’s
website. We use the provided split, with 693 documents for training and 100
for testing.

(ii) Indian-Abstractive dataset (IN-Abs) Shukla et al. (2022): This dataset
comprises case-summary pairs from Indian Supreme Court judgements and is
obtained from the website of the Legal Information Institute of India. These
abstractive summaries, also known as headnotes, are available for 7130 cases,
out of which 7030 are used for training and the remaining 100 for testing.

(iii) BVA-Extractive (BVA-Ext) Zhong et al. (2019): It comprises case-summary
pairs focusing on single-issue Post-Traumatic Stress Disorder decisions from
the US Board of Veterans’ Appeals (BVA). It encompasses a total of 112
decisions, each accompanied by an expert-annotated gold-standard extractive
summary. Within this dataset, 92 cases within the training set are associated
with one annotated extractive summary each. Additionally, the test set
comprises 20 cases, for which four distinct extractive summaries are provided
by multiple annotators. In all our experiments, for each document in the test
set, we report the averaged metric score across these four reference
summaries.

Dataset characteristics: To gain deeper insights into model performance, we
measure the following dataset characteristics. Compression Ratio Grusky et al.
(2018) indicates the word ratio between the summary and the case document.
Coverage Grusky et al. (2018) indicates the percentage of words in the summary
that are part of an extractive fragment with the input, and quantifies the
extent to which a summary is derivative of an input text. Density Grusky et
al. (2018) quantifies how well the word sequence of a summary can be described
as a series of extractions. It is derived from the average length of the
extractive fragment to which each word in the summary belongs. Copy Length
Chen et al. (2020) denotes the average length of segments in summary copied
from the source document. Repetition See et al. (2017) indicates the
proportion of n-grams repeated in the summary itself. Novelty See et al.
(2017) denotes the proportion of n-grams present in the summary that are not
in the source. High values of coverage, density, and copy length suggest a
more extractive dataset, while novelty reflects the level of abstractiveness
in the reference summaries. The lower the compression ratio, the more precise
capture of critical aspects from the article text is required, which presents
a greater challenge in summarization.

Table 1: Statistics of the case summarization datasets. Sum.:Summary;
Doc.:Document

| UK-Abs | IN-Abs | BVA-Ext
---|---|---|---
Train size | 693 | 7030 | 92
Test Size | 100 | 100 | 20
Compression Ratio | 0.129 | 0.224 | 0.152
Avg. #Tokens in Sum. | 1268 | 1067 | 259
Avg. #Tokens in Doc. | 14599 | 5408 | 2548
Copy Length | 2.958 | 3.725 | 39.86
Coverage | 0.968 | 0.943 | 1.00
Density | 8.129 | 10.314 | 53.91
Novelty (3-gram) | 0.44 | 0.375 | 0.00
Repetition (2-gram) | 0.019 | 0.021 | 0.013

Table 2: Perplexity score of the datasets using GPT2

Dataset | Perplexity
---|---
UK-Abs | 16.91
IN-Abs | 17.81
BVA-Ext | 14.74

Table 3: Vocabulary overlap (%) between datasets.

| UK-Abs | IN-Abs | BVA-Ext
---|---|---|---
UK-Abs | 100 | 49.7 | 30.7
IN-Abs | 49.7 | 100 | 25.9
BVA-Ext | 30.7 | 25.9 | 100

Table 3 reports the characteristics of the three datasets. Barring the
completely extractive BVA-Ext, UK-Abs tend to have slightly higher coverage
than IN-Abs. By contrast, IN-Abs has more copy length and density, suggesting
a higher presence of verbatim copy segments when compared to UK-Abs.
Conversely, UK-Abs is characterized by a higher 3-gram novelty score, which,
coupled with its low compression ratio, indicates its challenging nature.
Additionally, IN-Abs tends to include more 2-gram repeated phrases within the
summaries.

Similarity between the datasets from different jurisdictions: We seek to
quantify the similarity between jurisdictions on both lexical and semantic
levels. On the lexical level, we calculate the vocabulary overlap (%) between
the jurisdictions from both the documents and summaries. The vocabulary for
each domain is created by considering the top 1K most frequent words
(excluding stopwords) Gururangan et al. (2020) from the documents and the
summaries. At the semantic level, we employ a pre-trained GPT-2 model Radford
et al. (2019) to compute perplexity scores using a strided sliding window
approach with a stride of 512. We gauge the similarity between jurisdictions
based on the order of perplexity scores among the jurisdictions, which
provides insight into their semantic resemblance.

From Table 3 and Table 3, it becomes evident that IN-Abs holds more
resemblance to UK-Abs compared to BVA-Ext. This observation can be attributed
to two key factors. Firstly, India’s historical ties to the UK have led to a
substantial influence on its legal system, stemming from British colonial
rule. Secondly, both UK-Abs and IN-Abs datasets originate from Supreme Courts
dealing with a broad spectrum of legal matters due to their comprehensive
jurisdiction. In contrast, the BVA corpus, focusing specifically on single-
issue PTSD decisions, might exhibit more narrowly focused language. With
respect to BVA-Ext, UK-Abs tend to be more similar than IN-Abs on both
similarities.

##  4 Models & Evaluation Metrics

We use the following representative models from different legal case
summarization methods in Shukla et al. 2022 for our study. We briefly describe
each method and refer the reader to the aforementioned work for implementation
details.

Unsupervised Extractive Methods: (i) Domain-agnostic methods which employ
graph-based salient sentence identification, namely LexRank Erkan and Radev
(2004) and Reduction Mihalcea and Tarau (2004), matrix factorization-based LSA
Steinberger et al. (2004) and TF-IDF based Luhn Luhn (1958). (ii) Legal
Domain-specific methods include LetSum Farzindar (2004), CaseSummarizer
Polsley et al. (2016), which ranks sentences based on their TF-IDF weights
coupled with legal-specific features, Maximal Marginal Relevance (MMR) Zhong
et al. (2019) uses iterative selection of predictive sentences and finally
uses MMR to select the final summary sentences.

Supervised Extractive Methods: SummaRuNNer Nallapati et al. (2017) uses binary
classification for sentence selection to determine whether it should be part
of the summary. To build the training data for the classifier from a source
document $d$ and its abstractive reference summary $s$, each sentence in $s$
is matched with the top sentences from $d$, chosen based on the highest
average ROUGE-1, 2 and L scores. The extractive pseudo-reference summary is
created by combining all selected sentences and are then used for supervised
training of the model.

Abstractive Methods: We use BART Lewis et al. (2020) pre-trained on general
English corpora, and the ‘Legal Pegasus’
model111https://huggingface.co/nsi319/legal-pegasus that is further trained on
documents from litigations of U.S. Securities and Exchange Commission (SEC)
after its pre-training phase. We assess their performance with and without
fine-tuning. To accommodate the 1024-token input limit, we chunk the input
document, generate summaries for each chunk, and merge them, similar to what
was done in Shukla et al. 2022. To create reference summaries for each chunk
for fine-tuning, we map each summary sentence to its most similar counterpart
in the document using cosine similarity between mean token-level Sentence-BERT
Reimers and Gurevych (2019) embeddings and concatenating them as references
for each document chunk. We also use the Legal-Longformer Encoder-Decoder
(Legal-LED) model222 https://huggingface.co/nsi319/legal-led-base-16384 which
is trained on SEC legal corpus and is tailored specifically for long documents
with 16,384 tokens through sparse attention mechanism Beltagy et al. (2020).

Hybrid Extractive-Abstractive Methods: We also use both vanilla and fine-tuned
models of BERT-BART Bajaj et al. (2021), wherein the document length is
reduced by selecting salient sentences using a BERT-based extractive
summarization model followed by a BART model to generate the final summary.

Evaluation metrics: For evaluating the quality of generated and extracted
summaries, we employ ROUGE-L F-score (R-L) Lin (2004) and BERTScore (BS) Zhang
et al. (2019) and finally report the average scores across all the test
instances of the target jurisdiction.

Table 4: ROUGE-L F scores (R-L) and BERT Score (BS) of all methods across
three datasets. Each test dataset is represented by a column, while training
dataset used for supervision is indicated in Train column, if applicable. Best
results in each group and overall are bolded in black and magenta
respectively.

|  Test → | UK-Abs | IN-Abs | BVA-Ext
---|---|---|---|---
Model |  Train ↓ |  R-L/BS |  R-L/BS |  R-L/BS
Unsupervised, Extractive Models
LexRank | - | 0.427/0.836 | 0.439/0.850 | 0.356/0.862
LSA | - | 0.361/0.823 | 0.410/0.841 | 0.308/0.854
Luhn | - | 0.435/0.828 | 0.414/0.843 | 0.353/0.860
Reduction | - | 0.424/0.829 | 0.423/0.847 | 0.364/0.865
CaseSummarizer | - | 0.443/0.835 | 0.476/0.836 |  0.386/0.829
LetSum | - | 0.361/0.772 | 0.362/0.797 | 0.298/0.807
MMR | - |  0.467/0.834 |  0.485/0.847 | 0.384/0.825
Supervised, Extractive Models
SummaRuNNer | UK-Abs | - |  0.410/0.832 |  0.405/0.861
IN-Abs |  0.430/0.834 | - | 0.327/0.846
BVA-Ext | 0.378/0.782 | 0.374/0.839 | -
Abstractive & Hybrid Models without fine-tuning
BART | - |  0.517/0.833 | 0.486/0.839 | 0.324/0.841
Legal-Pegasus | - |  0.517/0.831 | 0.522/0.841 |  0.386/0.830
Legal-LED | - | 0.240/0.821 | 0.292/0.812 | 0.232/0.832
BERT-BART | - | 0.501/0.836 | 0.482/0.838 | 0.329/0.844
Abstractive & Hybrid Models with fine-tuning
BART | UK-Abs | - |  0.522/0.845 |  0.341/0.842
IN-Abs |  0.532/0.837 | - | 0.306/0.834
BVA-Ext | 0.519/0.835 | 0.503/0.840 | -
Legal-Pegasus | UK-Abs | - |  0.532/ 0.847 |  0.451/0.861
IN-Abs |  0.533/ 0.838 | - | 0.434/0.854
BVA-Ext | 0.522/0.834 | 0.522/0.842 | -
Legal-LED | UK-Abs | - |  0.418/0.838 | 0.321/0.834
IN-Abs |  0.479/0.832 | - |  0.339/0.831
BVA-Ext | 0.233/0.817 | 0.289/0.812 | -
BERT-BART | UK-Abs | - | 0.513/0.846 | 0.342/0.842
IN-Abs | 0.515/0.837 | - | 0.308/0.836
BVA-Ext | 0.508/0.836 | 0.501/0.839 | -

##  5 RQ1: Cross-Jurisdiction Generalizability

Unlike previous studies Bhattacharya et al. (2019); Shukla et al. (2022) that
primarily evaluated these summarization algorithms in in-domain settings,
where fine-tuning and evaluation occur on the same dataset, we focus on
assessing their generalization capabilities across domains/jurisdictions.
Unsupervised models can be applied directly to summarize documents of any
dataset (jurisdiction). For supervised models, we train the model on a
specific dataset (source jurisdiction) and evaluate its performance on other
datasets (target jurisdictions) in a blind zero-shot manner.

Results (Table 4): Among unsupervised methods (Block 1 in Tab.4), domain-
specific algorithms CaseSummarizer and MMR consistently performed the best,
especially in terms of Rouge-L score. Supervised extractive model SummaRuNNer
did not consistently yield better cross-jurisdiction performance compared to
unsupervised models, owing to its lack of pre-training except in case of BVA-
Ext target using UK-Abs as source (Block 2 in Tab. 4).

Abstractive & Hybrid models, due to their pre-training, demonstrated better
performance compared to unsupervised methods even without any fine-tuning, as
reflected over UK-Abs and IN-Abs primarily due to their abstractive nature of
datasets and pre-training objective (Block 3 in Tab.4). LED is semi-pretrained
as it has been initialized by repeatedly copying BART without undergoing end-
to-end general pre-training and then it has been fine-tuned on SEC legal
corpus to create Legal-LED, making it overfit to that specific domain, as
reflected in its lower zero-shot performance.

Finally, abstractive & hybrid models fine-tuned on datasets from source
jurisdictions generally performed well compared to the unsupervised methods,
emphasizing the importance of legal adaptation during fine-tuning in addition
to their pre-training either generally as with BERT/BART-BERT or legal-
specific as with Legal-Pegasus (Block 4 in Table 4). They all displayed
greater generalization stability across different source datasets irrespective
of the relation between the source and the target. Overall, Legal-Pegasus
stood out as the most robust model across all the datasets attributed to its
legal pre-training.

Choice of Source Jurisdiction: For both UK-Abs and IN-Abs, training on each
other’s data yielded better results compared to training on BVA-Ext (e.g.,
BART applied on UK-Abs test set, achieves 0.532/0.837 when trained on IN-Abs,
compared to 0.519/0.835 when trained on BVA-Ext) due to their larger size,
abstractive nature, and jurisdictional similarity between In-Abs and UK-Abs,
as stated earlier in Section 3. Interestingly, for BVA-Ext, UK-Abs proved to
be a better source domain than IN-Abs (for most supervised methods), despite
the latter’s larger size and more extractive nature. This transferability is
mainly attributed to the similarity between jurisdictions, as observed in
Section 3. This result shows that, while choosing the source jurisdiction
(whose data will be used to train supervised summarization models), blindly
choosing the jurisdiction with the largest dataset size is not an optimal
approach. Rather, the source jurisdiction should be chosen based on its
lexical and semantic similarity with the target jurisdiction. To this end, the
metrics we used in Section 3 can be useful to quantify the similarity between
jurisdictions.

Main Takeaways: Our findings reveal that supervised models trained on a
different source jurisdiction can yield better summarization in the target
jurisdiction, compared to unsupervised models. Specifically, pre-trained
models exhibit superior cross-jurisdictional generalizability due to their
pre-training. Legal-oriented training further enhances their generalization
capabilities. The choice of the source dataset significantly influences the
development of an effective system for a specific target jurisdiction, with
jurisdictional similarity playing a critical role than dataset size and
abstractiveness.

##  6 RQ2: Leveraging Unlabelled Target Jurisdiction Corpus

In this section, we explore whether we can improve the cross-domain
performance of supervised abstractive summarization models using the unlabeled
judgement documents from the target jurisdiction, while training the models on
source jurisidiction judgement-summary pairs. To this end, we adopt domain-
adversarial training from unsupervised domain adaptation literature Ganin et
al. (2016). Note that, we still do not use any reference summary from the
target jurisdiction.

In this technique, we additionally introduce an auxiliary jurisdiction
classifier that tries to discriminate the source jurisdiction embeddings from
the target ones. The encoder of the abstractive model is updated not only to
facilitate the decoder in producing summaries for source jurisdiction
documents, but is also adversarially trained to confuse the jurisdiction
classifier to learn jurisdiction-invariant feature representations. Put
differently, we want our models to learn how to summarize the given input
document with minimal encoding of the jurisdiction-specific information
contained in the texts, facilitating model to generalize to documents from
target jurisdictions.

Table 5: ROUGE-L F-score (R-L) and BERT Score (BS) of three abstractive
summarization models trained under different settings. Each target (test)
dataset is represented by a column and and ’S’ columns denote the source
dataset for fine-tuning. An empty configuration indicates that the model is
fine-tuned on the source dataset and evaluated on the target dataset (RQ1).
‘GRL’ indicates models fine-tuned in an adversarial fashion (RQ2). ‘Silver’
indicates models trained with silver summaries of the target dataset obtained
through an unsupervised MMR algorithm (RQ3). Entries marked with ∗ are
statistically significantly higher than the baseline (the empty configuration)
using 95% confidence interval by Wilcoxon signed rank test.

| Target → |  | UK-Abs |  | IN-Abs |  | BVA-Ext
---|---|---|---|---|---|---|---
Model Config |  S |  R-L/BS |  S |  R-L/BS |  S |  R-L/BS
BART |  |  IN | 0.532/0.837 |  UK | 0.522/0.847 |  UK | 0.341/0.842
GRL | 0.536/0.838 | 0.526/0.848 | 0.460/0.868 ∗
Silver | 0.529/0.838 | 0.507/0.832 |  0.471/0.872 ∗
|  BVA | 0.519/0.835 |  BVA | 0.503/0.840 |  IN | 0.306/0.834
GRL | 0.544/0.837 ∗ | 0.513/0.842 ∗ | 0.440/0.864 ∗
Silver |  0.546/0.839 ∗ |  0.515/0.842 ∗ |  0.471/0.871 ∗
BERT- BART |  |  IN | 0.515/0.838 |  UK | 0.513/0.846 |  UK | 0.342/0.842
GRL |  0.520/0.838 ∗ | 0.522/0.848∗ | 0.461/0.868 ∗
Silver | 0.516/0.838 | 0.508/0.838 |  0.465/0.868 ∗
|  BVA | 0.508/0.837 |  BVA | 0.501/0.839 |  IN | 0.308/0.836
GRL | 0.522/0.838 ∗ | 0.504/0.840 | 0.440/0.857 ∗
Silver |  0.527/0.838 ∗ |  0.509/0.843 ∗ |  0.465/0.869 ∗
Legal- Pegasus |  |  IN | 0.533/0.836 |  UK | 0.532/0.847 |  UK | 0.451/0.861
GRL | 0.528/0.834 | 0.524/0.838 | 0.394/0.829
Silver | 0.534/0.836 |  0.537/0.848 ∗ |  0.469/0.868 ∗
|  BVA | 0.522/0.834 |  BVA | 0.522/0.842 |  IN | 0.434/0.854
GRL | 0.517/0.828 | 0.521/0.842 | 0.326/0.818
Silver | 0.524/0.835 | 0.524/0.844 |  0.465/0.862 ∗

In our adaptation scenario, we have access to labelled judgement-summary pairs
of a source jurisdiction $\\{x_{s},y_{s}\\}$ and unlabelled judgement corpus
from target jurisdiction $\\{x_{t}\\}$. Let $\theta_{e}$ and $\theta_{d}$ be
the encoder $E$ and decoder $D$ parameters of a summarization model. We
introduce an auxiliary jurisdiction classifier $J$ with parameters
$\theta_{j}$, implemented as a linear layer, which takes the document
representation from the encoder and performs binary classification to identify
the document’s jurisdiction. We train the classifier in an adversarial fashion
to maximize the encoder’s ability to capture information required for the
summarization task while minimizing its ability to predict the jurisdiction.
Instead of the two-step adversarial min-max objective of GAN Goodfellow et al.
(2014), Ganin et al. 2016 proposed to jointly optimize all the components
using a Gradient Reversal Layer (GRL). The GRL is inserted between the encoder
and jurisdiction classifier, where it acts as the identity during the forward
pass but, during the backward pass, scales the gradients flowing through by
$-\lambda$, making the encoder receive the opposite gradients from the
jurisdiction classifier. Here, hyperparameter $\lambda$ is the adaptation rate
controlling the influence of the jurisdiction classifier on the encoder during
training. Thus, the overall objective function can be compactly represented
as:

$\begin{split}\arg\min\limits_{\theta_{e},\theta_{d},\theta_{j}}L_{1}(D(E(x_{s})),y_{s})~{}+\\\
\lambda L_{2}(J(GRL(E(x_{?}))),y_{j})\end{split}$ (1)

where the first term represents the actual loss term for the summarization
task, computed over the labelled article-summary of source jurisdiction
$\\{x_{s},y_{s}\\}$ and the second term represents the binary cross entropy
loss computed for classification of jurisdiction for both source and target
documents $x_{?}$ ($x_{?}$ can be $x_{s}$ or $x_{t}$) and it belongs to,
between the source or target jurisdiction ($y_{j}$).

Experimental Setup: We select BART, Legal-Pegasus, and BERT-BART for these
experiments, given their robust cross-domain performance in Section 5. For
each pair of source-target dataset, we apply this setup to all three
abstractive models. We adopt the hyperparameters from Shukla et al. 2022. The
jurisdiction classifier takes the mean of token-level embeddings obtained from
the encoder as the document representation. The adaptation rate is given by
$\lambda=\frac{2}{1+\exp(-\gamma p)}-1$, where $p=\frac{t}{T}$ and $t$ and $T$
represent the current training steps and total steps respectively. We fine-
tune $\gamma$ within [0.05, 0.1] using the validation set.

Results (Table 5): On comparing the models trained with the GRL (RQ2) to their
non-GRL counterparts (blind zero-shot transfer, as in RQ1), we notice an
improvement with addition of GRL for both BART and BERT-BART across the three
test sets, indicating that the models gained more robust and domain-invariant
representations through adversarial training, ultimately enhancing their
transferability. For instance, for BART trained on IN-Abs and tested on UK-
Abs, the R-L/BS scores for non-GRL variant are 0.532/0.837, and they improved
to 0.536/0.838 for the GRL variant. This improvement is greatly pronounced
when we consider BVA-Ext as the test set (e.g., 0.341/0.842 to 0.460/0.868 in
the case of UK-Abs as the train dataset for BART). However, in the case of
Legal-Pegasus, which already demonstrated strong cross-domain generalizability
(RQ1) due to its pre-training on legal texts, we notice a decline in its
performance with addition of GRL (eg., 0.533/0.836 to 0.528/0.834 in case of
IN-Abs as train, UK-Abs as test). We attribute this to what we refer to as
‘representation erasure’. It seems that the incorporation of the GRL and
jurisdiction discriminator inadvertently leads to the dilution of
generalizable knowledge that the model had acquired during its pre-training
phase. This erasure effect is most pronounced in BVA-Ext as test dataset
(e.g., 0.451/0.861 to 0.394/0.829 when UK-Abs is used as train dataset), which
aligns with Legal-Pegasus pre-training data in terms of its jurisdiction (US)
specific features compared to others.

Main Takeaway: Adversarial learning can enhance general pre-trained models
(e.g., BART, BERT-BART) making them proficient at acquiring domain-invariant
representations. But this strategy can also result in decreased performance
for legal pre-trained models like Legal-Pegasus, attributable to
representation erasure.

##  7 RQ3: Leveraging Silver Summaries of Target Jurisdiction

In this section, we explore whether incorporation of silver summaries from the
target jurisdiction, obtained via unsupervised extractive summarization
algorithms without the need for human annotations, can enhance effectiveness
of cross-domain transfer. We employ the same domain-adversarial training
framework, with a slight modification accounting for availability of judgment-
summary pairs from both the source jurisdiction (expert-annotated) and the
target jurisdiction (extractive silver summaries obtained via unsupervised
algorithms). The first term in overall objective, as described in Eqn. 1, is
now used for target judgement-summary pairs as well. In other words, the
$x_{s}$ in first term of Eqn. 1 can now be $x_{?}$ (i.e., either $x_{s}$ or
$x_{t}$). This modification also strengthens the robustness of the adversarial
learning approach. In the previous setup, the decoder’s updates were solely
influenced by source jurisdiction summaries, potentially leading to
overfitting to source jurisdiction semantics and limiting its generalizability
to new jurisdictions. The inclusion of silver summaries helps the decoder
align with the semantics of the target jurisdiction, thereby facilitating more
effective transfer performance.

Experimental Setup: Similar to RQ2, we replicate the setup using three
abstractive models BART, Legal-Pegasus and BERT-BART for each dataset pair,
designating one as the source jurisdiction with expert-annotated summaries and
the other as the target jurisdiction with silver summaries. We obtain the
silver summaries for the target jurisdiction using the MMR algorithm.333We
also experimented with CaseSummarizer to obtain the silver summaries and
observed that MMR showed better results than CS, as the trend in RQ1 Tab. 4.

Results (Table 5): With respect to BART and BERT-BART on UK-Abs (IN-Abs) as
the target dataset, augmenting models with silver summaries yield improvements
using BVA-Ext as train (eg., 0.544/0.837 to 0.546/0.839 on UK-Abs test, BVA-
Ext train for BART), while models trained on IN-Abs (UK-Abs) did not exhibit
enhancement. This can be attributed to the semantic similarity between UK-Abs
and IN-Abs, allowing effective learning without additional silver summaries
from each other. However, while using BVA-Ext as train dataset, the semantics
specific to UK-Abs (IN-Abs) necessitate the inclusion of their silver
summaries to facilitate the learning of target semantics.

With respect to Legal-Pegasus on both UK-Abs and IN-Abs as test set, we
witness improvements in all training setups. For BVA-Ext as the test set,
incorporation of silver summaries led to greater performance improvements
across all three models and both source datasets consistently (e.g.,
0.460/0.868 to 0.471/0.872 on UK-Abs train set on BART). This lends to the
nature of BVA-Ext dataset being extractive getting boost from addition of
silver summaries which are extractive.

Main Takeaways: Adding silver summaries significantly improved performance,
particularly for extractive datasets like BVA-Ext and when dealing with less
similar jurisdictions (e.g., using BVA-Ext for training and IN-Abs/UK-Abs for
testing). This, along with our RQ2 findings, highlights the efficacy of silver
summaries in mitigating representational erasure in legal pre-trained models,
as seen in the improved performance of Legal-Pegasus.

##  8 Case Study

Tables 7 and 8 (in the Appendix) illustrate some errors found in the summaries
generated by different configurations of BART models for a specific Indian
Supreme Court judgement. One notable error involves Indian jurisdiction-
specific terms such as I.P.C. (Indian Penal Code). Models with blind-zero
transfer (RQ1) and those with access to the source text only in GRL (RQ2)
often output terms like K.P.C. or K.C. (in place of I.P.C.), demonstrating a
limitation in the semantics of the decoder to understand Indian-specific
jargon. However, upon adding silver summaries (RQ3), the decoder becomes
exposed to such jargon, leading to the correct output of I.P.C. Another common
error across all models is incomplete sentences and coherence issues. In some
cases, the model abruptly shifts to a new aspect without completing the
previous one, potentially resulting in misrepresentations of the input
document. These observations highlight the challenges in long text
summarization, and the need for future improvement.

##  9 Conclusion

Our study reveals the following practical insights on building an effective
legal summarization system for a target jurisdiction without annotated data:
(i) Fine-tuning on non-target datasets outperforms unsupervised methods, but
success depends on the similarity between source and target jurisdictions.
(ii) When one has access to similar source data (e.g., IN-Abs for target UK-
Abs), using general pre-trained models like BART and BERT-BART, combined with
adversarial training, enhances transfer. (iii) When access to only non-similar
source data (e.g., BVA-Ext for UK-Abs) is available, augmenting models with
silver summaries from the target jurisdiction improves transfer. (iv) When
employing adversarial learning with legal pre-trained models like Legal-
Pegasus, it is important to be mindful of representational erasure, which can
be mitigated by incorporating silver summaries.

## Limitations

While our research contributes valuable insights into the cross-jurisdictional
generalizability of legal case summarization systems, it is crucial to
acknowledge certain limitations that affect the applicability of our findings.
Our experiments are conducted on three specific legal datasets from diverse
jurisdictions. The extent to which our observations hold true for a broader
range of legal systems remains an open question. The legal domain is vast and
varied, and different jurisdictions may exhibit unique characteristics that
impact the generalizability of summarization models. The scarcity of publicly
available legal summarization datasets limits the diversity and size of our
training and evaluation data. Hence we are constrained with use of these
three, only publicly available legal case summarization datasets in English.
This constraint could impact the robustness and generalizability of our
proposed methods and insights.

Our evaluation primarily relies on established summarization metrics such as
ROUGE and BERTScore. While these metrics have been used in many prior works on
legal document summarization, and are known to provide a quantitative measure
of summarization quality, they may not fully capture the nuanced legal
content, context, and intricacies essential for legal professionals. A
potential avenue for further research could be developing additional legal
domain-specific evaluation metrics. Another significant limitation of our
study is the absence of direct participation or validation by legal experts in
the assessment of summarization outputs, which we could not perform due to
lack of access to legal experts.

## Ethics Statement

We use publicly available datasets from prior works, which are obtained from
the web. Though the case documents are not anonymized, we do not foresee any
harm beyond their availability. We acknowledge the potential presence of
biases within the training data, which may inadvertently be reflected in the
generated summaries. To deploy these models in a production system, one must
thoroughly check for such biases by comprehensively evaluating summarization
performance across relevant groups (e.g., gender and race). Legal systems can
even carry historical biases, and training models on biased datasets may
perpetuate or exacerbate existing inequalities. As such, we urge vigilance in
scrutinizing dataset biases and committing to fair and unbiased
representation.

An inherent challenge in the development of automatic summarization models for
legal decisions lies in potential performance variations across different
partitions with in the same legal domain. For instance, in contexts like the
Board of Veterans’ Appeals (BVA) as discussed in Agarwal et al. 2022, cases
involving rarely occurring disabilities or specialized legal and military
situations may lead to suboptimal summaries due to sparsity in the training
data. This variability could disproportionately impact groups that should be
treated equally if their characteristics coincide with these less frequent
legal configurations. Engaging domain experts to curate datasets with better
representation across different types of injuries and legal phenomena can be a
proactive step. This can enhance the model’s understanding of uncommon or
group-related legal contexts, potentially mitigating disparities in
performance.

## References

  * Agarwal et al. (2022) Abhishek Agarwal, Shanshan Xu, and Matthias Grabmair. 2022.  Extractive summarization of legal decisions using multi-task learning and maximal marginal relevance.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 1857–1872.
  * Bajaj et al. (2021) Ahsaas Bajaj, Pavitra Dangati, Kalpesh Krishna, Pradhiksha Ashok Kumar, Rheeya Uppaal, Bradford Windsor, Eliot Brenner, Dominic Dotterrer, Rajarshi Das, and Andrew McCallum. 2021.  Long document summarization in a low resource setting using pretrained language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop_ , pages 71–80.
  * Bassignana and Plank (2022) Elisa Bassignana and Barbara Plank. 2022.  Crossre: A cross-domain dataset for relation extraction.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 3592–3604. Association for Computational Linguistics.
  * Beltagy et al. (2020) Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020.  Longformer: The long-document transformer.  _arXiv preprint arXiv:2004.05150_.
  * Bhattacharya et al. (2019) Paheli Bhattacharya, Kaustubh Hiware, Subham Rajgaria, Nilay Pochhi, Kripabandhu Ghosh, and Saptarshi Ghosh. 2019.  A comparative study of summarization algorithms applied to legal case judgments.  In _Advances in Information Retrieval: 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I 41_ , pages 413–428. Springer.
  * Bhattacharya et al. (2021) Paheli Bhattacharya, Soham Poddar, Koustav Rudra, Kripabandhu Ghosh, and Saptarshi Ghosh. 2021.  Incorporating domain knowledge for extractive summarization of legal case documents.  In _Proceedings of the eighteenth international conference on artificial intelligence and law_ , pages 22–31.
  * Chalkidis et al. (2020) Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020.  Legal-bert: The muppets straight out of law school.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 2898–2904.
  * Chen et al. (2020) Yiran Chen, Pengfei Liu, Ming Zhong, Zi-Yi Dou, Danqing Wang, Xipeng Qiu, and Xuan-Jing Huang. 2020.  Cdevalsumm: An empirical study of cross-dataset evaluation for neural summarization systems.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3679–3691.
  * Daumé III (2009) Hal Daumé III. 2009.  Frustratingly easy domain adaptation.  _arXiv preprint arXiv:0907.1815_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * Elaraby and Litman (2022) Mohamed Elaraby and Diane Litman. 2022.  Arglegalsumm: Improving abstractive summarization of legal documents with argument mining.  In _Proceedings of the 29th International Conference on Computational Linguistics_ , pages 6187–6194.
  * Elaraby et al. (2023) Mohamed Elaraby, Yang Zhong, and Diane Litman. 2023.  Towards argument-aware abstractive summarization of long legal opinions with summary reranking.  _arXiv preprint arXiv:2306.00672_.
  * Erkan and Radev (2004) Günes Erkan and Dragomir R Radev. 2004.  Lexrank: Graph-based lexical centrality as salience in text summarization.  _Journal of artificial intelligence research_ , 22:457–479.
  * Farzindar (2004) Atefeh Farzindar. 2004.  Atefeh farzindar and guy lapalme,’letsum, an automatic legal text summarizing system in t. gordon (ed.), legal knowledge and information systems. jurix 2004: The seventeenth annual conference. amsterdam: Ios press, 2004, pp. 11-18.  In _Legal knowledge and information systems: JURIX 2004, the seventeenth annual conference_ , volume 120, page 11. IOS Press.
  * Feijo and Moreira (2023) Diego de Vargas Feijo and Viviane P Moreira. 2023.  Improving abstractive summarization of legal rulings through textual entailment.  _Artificial intelligence and law_ , 31(1):91–113.
  * Fried et al. (2019) Daniel Fried, Nikita Kitaev, and Dan Klein. 2019.  Cross-domain generalization of neural constituency parsers.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 323–330.
  * Fu et al. (2020) Jinlan Fu, Pengfei Liu, and Qi Zhang. 2020.  Rethinking generalization of neural models: A named entity recognition case study.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 7732–7739.
  * Ganin et al. (2016) Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016.  Domain-adversarial training of neural networks.  _The journal of machine learning research_ , 17(1):2096–2030.
  * Goodfellow et al. (2014) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014.  Generative adversarial nets.  _Advances in neural information processing systems_ , 27.
  * Grusky et al. (2018) Max Grusky, Mor Naaman, and Yoav Artzi. 2018.  Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_. Association for Computational Linguistics.
  * Gururangan et al. (2020) Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith. 2020.  Don’t stop pretraining: Adapt language models to domains and tasks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8342–8360.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Kumar et al. (2022) Vinayshekhar Bannihatti Kumar, Kasturi Bhattacharjee, and Rashmi Gangadharaiah. 2022.  Towards cross-domain transferability of text generation models for legal text.  In _Proceedings of the Natural Legal Language Processing Workshop 2022_ , pages 111–118.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880.
  * Lin (2004) Chin-Yew Lin. 2004.  Looking for a few good metrics: Rouge and its evaluation.
  * Liu and Chen (2019) Chao-Lin Liu and Kuan-Chun Chen. 2019.  Extracting the gist of chinese judgments of the supreme court.  In _proceedings of the seventeenth international conference on artificial intelligence and law_ , pages 73–82.
  * Luhn (1958) Hans Peter Luhn. 1958.  The automatic creation of literature abstracts.  _IBM Journal of research and development_ , 2(2):159–165.
  * Mandal et al. (2021) A. Mandal, Paheli Bhattacharya, Sekhar Mandal, and Saptarshi Ghosh. 2021.  Improving legal case summarization using document-specific catchphrases.  In _International Conference on Legal Knowledge and Information Systems_.
  * Mihalcea and Tarau (2004) Rada Mihalcea and Paul Tarau. 2004.  Textrank: Bringing order into text.  In _Proceedings of the 2004 conference on empirical methods in natural language processing_ , pages 404–411.
  * Moro and Ragazzi (2022) Gianluca Moro and Luca Ragazzi. 2022.  Semantic self-segmentation for abstractive summarization of long documents in low-resource regimes.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, pages 11085–11093.
  * Nallapati et al. (2017) Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017.  Summarunner: A recurrent neural network based sequence model for extractive summarization of documents.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 31.
  * Paul et al. (2022) Shounak Paul, Arpan Mandal, Pawan Goyal, and Saptarshi Ghosh. 2022.  Pre-training transformers on indian legal text.  _arXiv preprint arXiv:2209.06049_.
  * Polsley et al. (2016) Seth Polsley, Pooja Jhunjhunwala, and Ruihong Huang. 2016.  Casesummarizer: a system for automated summarization of legal texts.  In _Proceedings of COLING 2016, the 26th international conference on Computational Linguistics: System Demonstrations_ , pages 258–262.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Ray et al. (2020) Oliver Ray, Amy Conroy, and Rozano Imansyah. 2020.  Summarisation with majority opinion.  In _JURIX_ , pages 247–250.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992.
  * Ruder (2019) Sebastian Ruder. 2019.  _Neural transfer learning for natural language processing_.  Ph.D. thesis, NUI Galway.
  * Saravanan et al. (2006) Murali Saravanan, Balaraman Ravindran, and Shivani Raman. 2006.  Improving legal document summarization using graphical models.  _Frontiers in Artificial Intelligence and Applications_ , 152:51.
  * Schraagen et al. (2022) Marijn Schraagen, Floris Bex, Nick Van De Luijtgaarden, and Daniël Prijs. 2022.  Abstractive summarization of dutch court verdicts using sequence-to-sequence models.  In _Proceedings of the Natural Legal Language Processing Workshop 2022_ , pages 76–87.
  * See et al. (2017) Abigail See, Peter J Liu, and Christopher D Manning. 2017.  Get to the point: Summarization with pointer-generator networks.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1073–1083.
  * Shah et al. (2018) Darsh Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, and Preslav Nakov. 2018.  Adversarial domain adaptation for duplicate question detection.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1056–1063.
  * Shen et al. (2018) Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu. 2018.  Wasserstein distance guided representation learning for domain adaptation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32.
  * Shukla et al. (2022) Abhay Shukla, Paheli Bhattacharya, Soham Poddar, Rajdeep Mukherjee, Kripabandhu Ghosh, Pawan Goyal, and Saptarshi Ghosh. 2022.  Legal case document summarization: Extractive and abstractive methods and their evaluation.  In _Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing_ , pages 1048–1064.
  * Steinberger et al. (2004) Josef Steinberger, Karel Jezek, et al. 2004.  Using latent semantic analysis in text summarization and summary evaluation.  _Proc. ISIM_ , 4(93-100):8.
  * Sun et al. (2016) Baochen Sun, Jiashi Feng, and Kate Saenko. 2016.  Return of frustratingly easy domain adaptation.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 30.
  * Talmor and Berant (2019) Alon Talmor and Jonathan Berant. 2019.  Multiqa: An empirical investigation of generalization and transfer in reading comprehension.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4911–4921.
  * Thakur et al. (2021) Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021.  Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_.
  * Tyss et al. (2023) Santosh Tyss, Oana Ichim, and Matthias Grabmair. 2023.  Zero-shot transfer of article-aware legal outcome classification for european court of human rights cases.  In _Findings of the Association for Computational Linguistics: EACL 2023_ , pages 593–605.
  * Wang et al. (2019) Huazheng Wang, Zhe Gan, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, and Hongning Wang. 2019.  Adversarial domain adaptation for machine reading comprehension.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2510–2520.
  * Wang et al. (2022) Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. 2022.  Gpl: Generative pseudo labeling for unsupervised domain adaptation of dense retrieval.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2345–2360.
  * Xin et al. (2022) Ji Xin, Chenyan Xiong, Ashwin Srinivasan, Ankita Sharma, Damien Jose, and Paul Bennett. 2022.  Zero-shot dense retrieval with momentum adversarial domain invariant representations.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 4008–4020.
  * Xu and Ashley (2022) Huihui Xu and Kevin Ashley. 2022.  Multi-granularity argument mining in legal texts.  _arXiv preprint arXiv:2210.09472_.
  * Xu et al. (2021) Huihui Xu, Jaromir Savelka, and Kevin D Ashley. 2021.  Toward summarizing case decisions via extracting argument issues, reasons, and conclusions.  In _Proceedings of the eighteenth international conference on artificial intelligence and law_ , pages 250–254.
  * Yu and Jiang (2016) Jianfei Yu and Jing Jiang. 2016.  Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification.  In _Proceedings of the 2016 conference on empirical methods in natural language processing_ , pages 236–246.
  * Yu et al. (2022) Yue Yu, Chenyan Xiong, Si Sun, Chao Zhang, and Arnold Overwijk. 2022.  Coco-dr: Combating the distribution shift in zero-shot dense retrieval with contrastive and distributionally robust learning.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 1462–1479.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  In _International Conference on Machine Learning_ , pages 11328–11339. PMLR.
  * Zhang et al. (2022) Shiyue Zhang, David Wan, and Mohit Bansal. 2022.  Extractive is not faithful: An investigation of broad unfaithfulness problems in extractive summarization.  _arXiv preprint arXiv:2209.03549_.
  * Zhang et al. (2019) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019.  Bertscore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.
  * Zhong et al. (2019) Linwu Zhong, Ziyi Zhong, Zinian Zhao, Siyuan Wang, Kevin D Ashley, and Matthias Grabmair. 2019.  Automatic summarization of legal decisions using iterative masking of predictive sentences.  In _Proceedings of the seventeenth international conference on artificial intelligence and law_ , pages 163–172.

Table 6: Segment-wise ROUGE-L Recall scores on the UK-Abs dataset. BGA, Jud., Rea. indicate ‘Background to the Appeal’, ‘Judgement’, and ‘Reasons for Judgement’ respectively. Source → | IN-Abs | BVA-Ext
---|---|---
Model Config |  BGA |  Jud. |  Rea. |  BGA |  Jud. |  Rea.
BART |  | 0.319 | 0.488 | 0.283 | 0.315 | 0.449 | 0.295
GRL | 0.377 | 0.528 | 0.340 | 0.318 | 0.452 | 0.301
Silver | 0.348 | 0.492 | 0.293 | 0.338 | 0.471 | 0.302
BART-BERT |  | 0.312 | 0.474 | 0.266 | 0.309 | 0.445 | 0.281
GRL | 0.337 | 0.493 | 0.294 | 0.313 | 0.45 | 0.286
Silver | 0.337 | 0.479 | 0.286 | 0.334 | 0.47 | 0.294

##  Appendix A Implementation Details

We follow the hyper parameters for all of the baseline models as outlined by
Shukla et al. 2022. All the implementational details and code of the
unsupervised baselines are provided in their github
repository444https://github.com/Law-AI/summarization. For BART, we employ a
learning rate of 2e-5. We train the model end-to-end for 3 epochs with a batch
size of 1. We limit maximum input and output length to 1024 and 512
respectively. For Legal Pegasus, we use a learning rate of 5e-5 and with batch
size of 1 for 2 epochs, with maximum input and output length of 512 and 256
respectively. For Legal LED, we use a learning rate of 1e-3 trained for 3
epochs with a batch size of 4, with input and output length of 16384 and 1024
respectively. All models are trained with Adam optimizer Kingma and Ba (2014).

Config | Train | Summary Snippet | Explanation for Errors
---|---|---|---
Blind transfer (RQ1) | UK-Abs |  | … under section 302 K.P.C. ……
---
and had a bath there. He The
murder was committed ………..
mere possession of a sword so
stained be not sufficient to
establish conclusively. The extra
judicial confession of the ……
| It mentions as K.P.C., but it has to
---
be I.P.C.
Incomplete sentence: It starts with “He”
and digresses to the next sentence.
Incoherent discourse: It doesn’t mention
what can’t be established. The closest
sentence in the source document is
“… not sufficient to establish conclusively
that the person who possessed it so
shortly after the murder of a person with
whom he had enmity, had committed the
murder…”
BVA-Ext |  | …. under section I.C., by
---
the Session …. cattle shed in
village Bhadurpur Ghar. With
respect to the learned Judges,
theseobservations are not very
consistent. To us, it seems that
in the middle of June when
the chari and sugar cane crop
would not have been very ……
| It mentions as I.C instead of I.P.C.
---
and misses actual section number.
This sentence in the source document
is in context to Ujagar Singh’s
extrajudicial confession, and not the
details of the crime.
Incoherent discourse: It digresses
into the details of the extrajudicial
confession without describing the
remaining evidence against the
appellant
Table 7: Examples of errors in summaries produced by BART model in zero-shot blind transfer (RQ1) for the Indian Supreme Court judgement https://indiankanoon.org/doc/1811974/, trained with different jurisdiction datasets. Config | Train | Summary Snippet | Explanation for Errors
---|---|---|---
| UK-Abs |  | … under Section 302 K.P.C.
---
…………………… murder.
There is no good reasons for
Ujagar Singh to state falsely.
If the statement made
| It mentions as K.P.C instead of I.P.C
---
Incomplete sentence: It doesn’t mention
what Ujagar Singh states falsely.
GRL (RQ2) | BVA-Ext |  |  section I.C., by the Session
---
Judge……………… the
appellant caused injuries to
Sheo Sahai, who was
sleeping in his cattle shed in
village Bhadurpur Ghar. With
respect to the learned Judges,
these observations are not very
consistent. The learned Sessions
Judge has discussed the criticism
urged ……………it seems
that in the middle of June when
the chari and sugar cane crop
would not have been very
high, it is improbable Appeal
dismissed. The evidence ……..
| It mentions as I.C instead of I.P.C.
---
and misses actual section number.
This sentence in the source document
is in context to Ujagar Singh’s
extrajudicial confession, and not the
details of the crime.
Incomplete sentence: It doesn’t
describe what is improbable and
moves to next sentence.
| UK-Abs |  | Criminal Appeal No.
---
1782 of 60 and Referred
No.125 of 1960, section 302
I.P.C……….High Court erred
in rejecting the statement
of Ujagar Singh about the
appellant ’s confessing to him
that The extra judicial ….
| Incomplete sentence, which doesn’t
---
describe what the appellant confessed.
Silver (RQ3) | BVA-Ext |  | … in Criminal Appeal
---
No.125 of 1960, section 302
I.P.C…….. for the appellant
has argued …….
| Missing complete details: The source
---
document states “…Criminal Appeal
No. 1782 of 60 and Referred No. 125
of 1960…”
Table 8: Examples of errors in summaries produced by BART model in GRL setting
(RQ2) and silver summary setting (RQ3) for the Indian Supreme Court judgement
https://indiankanoon.org/doc/1811974/, trained with different jurisdiction
datasets.

##  Appendix B Segment-wise Evaluation

Each summary in UK-Abs dataset is segmented into three segments – ‘Background
to the Appeal’, ‘Judgement’, and ‘Reasons for Judgement’ –allowing segment-
wise evaluation to assess the quality of summarization for each rhetorical
segment. In line with Shukla et al. 2022, we present the ROUGE-L recall scores
for each rhetorical segment, as available in the UK-Abs dataset (stated in
Sec. 3). These scores are computed by comparing the generated summary to the
corresponding golden summary under each segment.

Table 6 showcases the results for BART and BERT-BART models trained on IN-Abs
and BVA-Ext source datasets, considering both non-GRL, GRL, and silver
variants. Notably, when trained with BVA-Ext, the incorporation of silver
summaries yields better scores across all segments than GRL and non-GRL
variants. Conversely, for IN-Abs, the GRL variant exhibits strong performance
across both models than silver summary variants. These segment-level trends
also align with document-wise trends in Table 5, where performance of silver-
enhanced models dropped when trained on IN-Abs but improved when trained on
BVA-Ext (UK-Abs column of Tab. 5).