2024 [1,2]Sisipho Hamlomo [1]Marcellin Atemkeng 1]Department of Mathematics,
Rhodes University, PO Box 94, Makhanda, 6140, South Africa 2]Department of
Statistics, Rhodes University, PO Box 94, Makhanda, 6140, South Africa
3]Computer Vision, Institute of Cognitive Science, Osnabrück University,
Wachsbleiche 27, Osnabrück, D-49090, Germany 4]International Centre for Radio
Astronomy Research (ICRAR), Curtin University, Bentley, WA, Australia 5]ARC
Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D),
Bentley, Australia 6]Curtin Institute of Radio Astronomy, GPO Box U1987,
Perth, WA 6845, Australia *]Correspondence<EMAIL_ADDRESS><EMAIL_ADDRESS>
# Advancing Low-Rank and Local Low-Rank Matrix Approximation in Medical
Imaging: A Systematic Literature Review and Future Directions

*  Yusuf Brima  Chuneeta Nunhokee  Jeremy Baxter  [  [  [  [  [  [  [

###### Abstract

The large volume and complexity of medical imaging datasets are bottlenecks
for storage, transmission, and processing. To tackle these challenges, the
application of low-rank matrix approximation (LRMA) and its derivative, local
LRMA (LLRMA) has demonstrated potential.

This paper conducts a systematic literature review to showcase works applying
LRMA and LLRMA in medical imaging. A detailed analysis of the literature
identifies LRMA and LLRMA methods applied to various imaging modalities. This
paper addresses the challenges and limitations associated with existing LRMA
and LLRMA methods.

We note a significant shift towards a preference for LLRMA in the medical
imaging field since 2015, demonstrating its potential and effectiveness in
capturing complex structures in medical data compared to LRMA. Acknowledging
the limitations of shallow similarity methods used with LLRMA, we suggest
advanced semantic image segmentation for similarity measure, explaining in
detail how it can measure similar patches and their feasibility.

We note that LRMA and LLRMA are mainly applied to unstructured medical data,
and we propose extending their application to different medical data types,
including structured and semi-structured. This paper also discusses how LRMA
and LLRMA can be applied to regular data with missing entries and the impact
of inaccuracies in predicting missing values and their effects. We discuss the
impact of patch size and propose the use of random search (RS) to determine
the optimal patch size. To enhance feasibility, a hybrid approach using
Bayesian optimization and RS is proposed, which could improve the application
of LRMA and LLRMA in medical imaging.

###### keywords:

Medical imaging, Local low-rank matrix approximation, Semantic segmentation,
Similarity measures, Random search, Bayesian optimization

##  1 Introduction

Medical images obtained using technologies such as PET, CT scans, MRI and
ultrasound are crucial for diagnosing and monitoring health conditions
hussain2022modern ; umar2019review ; ganguly2010medical . These images are
central to treatment planning, guiding surgical procedures, and facilitating
minimally invasive procedures shah2014recent ; lieberman2006bone . They also
play a crucial role in monitoring disease progression, evaluating treatment
effectiveness and adjusting the treatment plan promptly umar2019review ;
islam2023introduction ; tavazzi2020mri ; delorme2009imaging . Rapid and
accurate diagnosis through medical imaging is essential in emergency medicine
to identify critical conditions and initiate urgent action chin2013pilot ;
tayal2003prospective . In addition to clinical applications, medical images
contribute significantly to research, education and the emerging field of
personalized medicine, where tailored treatments based on individual anatomy
and pathology are becoming increasingly important aerts2016potential ;
maceachern2021machine ; pinker2018precision . However, handling medical data
is challenging due to noise, high-dimensional data, corrupted or incomplete
data, and large amounts of data that increase memory and computational time
li2018large ; panayides2020ai ; yu2021convolutional ; bolon2016feature ;
lee2017medical . Various techniques have been proposed in the literature to
overcome these challenges. These techniques include smoothing algorithms
(i.e., Weiner filter, Gaussian filter, average filter, median filter, etc.)
used in processing and analyzing medical datasets to reduce noise
shinde2012noise ; na2018guided ; kumar2017noise ; ilango2011new ;
bhonsle2012medical ; gravel2004method ; naimi2015medical . Methods such as
sparse coding devadoss2019near ; sudha2011two ; liu2019fast ;
anandan2016medical , discrete cosine transform (DCT) and wavelet transform are
used for image compression jones1995comparative ; wu2001medical ;
kaur2015image ; singh2015multiple . In addition, techniques such as
t-distributed stochastic neighbour embedding (t-SNE) and local linear
embedding (LLE) are used to reduce the dimensions of these datasets
devassy2020dimensionality ; bibal2020explaining ; faust2018visualizing ;
jamieson2010exploring . A common limitation of the above smoothing algorithms
is their limited ability to effectively handle images corrupted by complex
noise patterns or non-Gaussian noise. These filters may have difficulty
preserving fine details and edges in different types of noise, resulting in
either over-smoothing or incomplete noise removal jin2003adaptive ;
milanfar2013symmetrizing ; doymaz2001wavelet ; rahman2003wavelet .
Furthermore, the DCT and wavelet transform techniques lead to blocking and
ringing artefacts and are also unable to capture directional information
pradhan2016comparative ; toufik2012wavelet  while sparse coding can be
computationally expensive, especially for large images xiang2011learning ;
zheng2010graph . The computational and space complexity of t-SNE grows
quadratically with the number of data pairs, limiting its application to large
datasets van2008visualizing ; gisbrecht2015parametric ; van2014accelerating .
Finding an appropriate $k-$nearest neighbors value for all input data in LLE
is challenging due to the complexity, nonlinearity, and variety of high-
dimensional input data saul2003think ; ge2008hand ; pan2009weighted .

low-rank matrix approximation (LRMA) has shown potential in overcoming the
challenges mentioned above zhou2014low ; gu2019brief ; haeffele2014structured
; ebiele2020conventional . LRMA refers to a technique used to approximate a
given matrix as the product of two lower-dimensional matrices, to reduce the
complexity of the original matrix while preserving important features. LRMA
techniques allow these large datasets to be compressed and efficiently
represented, enabling faster storage, transmission and processing lee2017deep
; muller2004review ; razzak2018deep . However, these datasets can be affected
by missing or corrupted entries due to factors such as equipment limitations
or patient movement magrabi2012using ; shellock2002magnetic ; catana2015motion
. LRMA techniques, such as image inpainting, are used to reconstruct missing
or corrupted regions of the medical images wang2021medical ; armanious2020ipa
; li2019progressive . In signal processing, such as biological signal analysis
from physiological measurements, LRMA techniques help with denoising and
feature extraction to improve diagnostic accuracy sweeney2012artifact ;
singh2006optimal . Genomics and proteomics use LRMA for data analysis by
facilitating dimensionality reduction and pattern recognition in complex
biological datasets meng2016dimension ; misra2019integrated . In
chemoinformatics, LRMA is used in the analysis of chemical and biological data
matrices in drug discovery, helping to identify relationships between
molecular structures and biological activities hao2018new ; t20165 . In multi-
modal data fusion, that is, when dealing with data from multiple sources
(e.g., imaging, clinical, genomic), LRMA is used to integrate and fuse
different modalities, providing a comprehensive view for diagnosis and
treatment planning he2022fidelity ; liu2015medical ; zhong2016predict .

While LRMA proves to be a valuable tool in medical imaging, it has
limitations. One of the primary concerns is the potential loss of image detail
and fidelity huang2014robust ; lu2016reference ; lu2018gradient . As the rank
of the approximation decreases, fine textures, intricate patterns, and subtle
color variations may be compromised, leading to a degraded image. The choice
of the appropriate rank becomes crucial, striking a balance between reducing
storage requirements and preserving visual quality oseledets2009breaking ;
hastie2015matrix . Another challenge lies in the computation time associated
with LRMA methods. Singular value decomposition (SVD), a common technique for
LRMA, has a time complexity of approximately $\mathcal{O}(mn\min\\{m,\,n\\})$
holmes2007fast ; mamat2007statistical ; brand2006fast ; roughgarden2015cs168 ;
atemkeng2023lossy ; halko2011finding , where $m$ and $n$ are the dimensions of
the image and truncating it to $r$ singular values and vectors results in a
complexity of $\mathcal{O}(mnr)$ brand2006fast ; roughgarden2015cs168 ;
burca2014fast ; atemkeng2023lossy ; halko2011finding . This can be
computationally expensive for high-dimensional images. While randomized SVD
offers a faster alternative with a time complexity of $\mathcal{O}(mn\log(r))$
for rank-$r$ approximation halko2011finding , the computation still demands
careful consideration, especially for real-time applications.

The local low-rank matrix approximation (LLRMA), which is an extension of LRMA
proposed in lee2013local  in 2013 overcomes the drawbacks mentioned above and
significantly relaxes the low-rank assumption. That is, instead of assuming
that a matrix has a low-rank globally, they assumed that it acts as a low-rank
matrix in the neighborhood of particular row-column combinations. This
approach can capture complicated spatial patterns in the data. By decomposing
the input image into low-rank components specific to local regions, this
technique captures local variations and fine-grained details that may be
missed in global LRMA lee2013local ; liu2019adaptive . This makes it
particularly suitable for image processing tasks, where objects and features
often have varying degrees of complexity and structure within different
regions guo2017patch . By exploiting the redundancy and similarity of
neighboring regions, this technique reduces the computational complexity
associated with traditional LRMA lee2013local ; dib2020local . This efficiency
can result in faster processing times and reduced memory requirements, making
it feasible to handle larger datasets and real-time applications to be
processed lee2016llorma .

However, LLRMA technique has its drawbacks. One notable limitation is the
potential for patch artifacts or inconsistencies along the boundaries of
localized regions. These artifacts can arise from abrupt transitions between
adjacent regions with different low-rank structures. Careful selection of
patch sizes and strategies to mitigate such artifacts are essential to ensure
the quality of the approximation. In addition, for certain types of data where
information from distant regions plays a critical role in the overall
structure, the LLRMA may have problems capturing global dependencies and long-
range interactions.

This review highlights works in the literature that apply LRMA and LLRMA to
medical images. To the best of our knowledge, there is no systematic
literature review that addresses LRMA and/or LLRMA with a specific focus on
medical data. Due to this gap in the literature, review questions are
formulated to guide the systematic investigation and evaluation. These
questions help structure and guide the review process to ensure that it
remains focused and addresses the most important aspects of interest.

###  1.1 Problematic

Medical imaging faces several challenges that affect the quality and
reliability of diagnostic information. Challenges such as noise, artifacts,
large amounts of data, high-dimensional data, geometric deformations, and low
resolution are common in various imaging modalities dhawan2011medical . These
challenges are particularly detrimental to image registration, a critical
process in medical imaging where information from different image modalities
(e.g. MRI, CT) is aligned and integrated. High noise levels and artifacts can
lead to inaccuracies in the alignment process, and the large volume and
complexity of medical image data make traditional registration methods
computationally intensive and error-prone. Geometric deformations further
complicate accurate registration, leading to suboptimal alignment and
potentially misleading clinical interpretations. Various solutions have been
proposed to overcome these challenges. These solutions include image denoising
saladi2017analysis ; pandey2016anatomization ; lyra2012improved ;
mcgivney2014svd ; fang2015spatio ; yang2018low ; kopriva2023low , image
compression doukas2007region ; wang1996medical ; zukoski2006novel , image
reconstruction he2022fidelity ; asslander2018low ; peng2016accelerated ;
ulas2016spatio ; xu2017dynamic , matrix completion li2020rank ;
friedland2006algorithm , and tensor decomposition liu2020low ; yi2021joint ;
cheng2014curvilinear ; jia2022nonconvex . While these methods have proven
useful in overcoming some of these challenges by reducing noise and capturing
the essential features, they are not sufficient on their own. When using the
LRMA, the local variations in medical images can be overlooked, resulting in
the loss of important anatomical details liu2019adaptive ; lee2016llorma . In
addition, LRMA has difficulty dealing with the large amount of data and high
dimensionality of medical images, which limits its effectiveness in preserving
important information kishore2017literature ; nguyen2009fast ;
achlioptas2007fast ; chu2003structured ; liberty2007randomized . The LLRMA
technique is important in the medical domain since it overcomes these
limitations by allowing for variations in rank at the local level. This is
particularly relevant in medical imaging, where subtle tissue characteristics
variations can be clinically significant. LLRMA enables a more accurate
representation of complex medical images, improving the performance of image
processing tasks like registration.

To achieve the main objective of the study, the following questions were
formulated:

  * •

To understand and critique the application of LRMA and LLRMA in medical
imaging.

  * •

Is LLRMA more effective than LRMA when applied to medical images?

  * •

Are there specific considerations or modifications required when applying
LLRMA to different modalities of medical data?

  * •

What are the limitations of LLRMA, and under what conditions does it perform
well?

  * •

Discuss and propose how LRMA or LLRMA can be applied to several medical data
types.

  * •

Has the medical community switched to applying LLRMA rather than LRMA?

  * •

What are the medical datasets that are used in the literature with LRMA and
LLRMA

  * •

Review and critique the similarity measurement methods used to measure similar
patches in medical imaging

  * •

What are the most commonly used similarity measure techniques in the context
of LLRMA, and how do they impact the overall performance of LLRMA techniques?

  * •

How does the choice of the local neighborhood or patch size impact the quality
of the approximation, and what are the trade-offs involved?

  * •

Propose future research directions and solutions

  * •

Discuss the applicability and feasibility of the proposed solutions.

This investigation fills the research gaps by comparing LRMA and LLRMA applied
in the medical imaging field in terms of their strengths, weaknesses and
limitations and suggests the way forward in this field.

###  1.2 Contributions

In this study, we conducted a comprehensive review of 47 publications. Of
these, 25 applied LRMA techniques to medical data and 32 applied LLRMA
techniques between 2006 and 2023. We discuss various datasets that have been
used in LRMA or LLRMA applications. In addition, we summarize various tasks
that have been performed on these datasets and recommend other machine-
learning tasks that could be performed. We highlight a notable shift in
preference for LLRMA in the medical field since 2015, demonstrating its
potential and effectiveness in capturing complex structures in medical data
compared to LRMA.

We acknowledge the limitations of shallow similarity methods commonly used in
this field and suggest exploring advanced deep learning models such as DeepLab
chen2017deeplab . We explain in detail how DeepLab can measure similar patches
and how feasible it is. Furthermore, our study highlights the importance of
applying LRMA to different modalities of healthcare data, including structured
and semi-structured data. We discuss the limitations of LRMA on irregular data
types and show how LRMA can also be used on regular data with missing entries.
We extend this by discussing the impact of inaccuracies in the prediction of
missing values and how these affect the application of LRMA. In addition, the
impact of patch size is addressed and we propose the use of a random search
(RS) technique to find the optimal patch size. To improve the feasibility of
this method, we propose a hybrid approach using Bayesian optimization and RS.

###  1.3 Manuscript Organization

The organization of this work is as follows: Section 2 discusses the
methodology used in this paper. Section 3 investigates the application of LRMA
to medical data and highlights various methods researchers propose to improve
the quality of medical imaging datasets, such as MRI, CT, microarray, and
infrared imaging. Section 4 reviews the literature on the application of LLRMA
to various medical imaging modalities. Section 5 presents various medical
datasets used in LRMA and LLRMA. Section 6 explores various similarity
measurement algorithms used to measure similar patches. Section 8 discusses
the results and drawbacks of applying the LRMA technique to medical data.
Section 9 suggests possible future research directions and discusses the
applicability and feasibility of the proposed solutions. This section serves
as a conclusion of the research and summarizes the main findings of the study.

##  2 Methodology

This systematic review examines LLRMA to establish a comprehensive and
rigorous approach that identifies, selects, and evaluates relevant field
studies. The methodology outlines the systematic process that will be followed
to ensure the review’s transparency, reliability, and replicability. This
process includes formulating the research question, defining inclusion and
exclusion criteria, systematically searching literature databases, reviewing
and selecting studies according to predefined criteria, extracting data from
the selected studies and assessing the quality and potential biases in the
included studies. In this study, we follow methodology based on the frameworks
proposed by uman2011systematic  for systematic review. First, we define a
strategy for conducting a comprehensive literature search. The defined search
strategy is then executed, and publications that meet the predefined inclusion
and exclusion criteria are identified. Relevant data are extracted through
thorough reading and synthesis of the selected publications. In the final
step, the extracted data is analyzed and interpreted to summarize the current
state of research and suggest new ways for further investigation.

###  2.1 Data Sources and Search Strategy

In this systematic review, we used three databases, namely, Scopus, Web of
Science and PubMed, to collect publications using the advanced search.
Keywords were identified to compose the search string to select studies to
answer the research questions. Boolean operators were used to combine and
divide these strings. Publications were first selected based on their title,
keywords, and abstracts. Table 2.2 shows the search details.

###  2.2 Selection and Data Extraction

The selection of studies for this systematic review followed predefined
inclusion and exclusion criteria to ensure relevance and quality. Following
the search mentioned in Section 2.1, all journals were imported into different
Excel sheets and sorted by keywords. Duplicates were removed using the VLOOKUP
and COUNTIF functions. The COUNTIF function counts the number of cells within
a specified range that meet a given condition or criteria. It was used in our
case to count the number of times a specific journal appeared in a sheet
(i.e., checking for duplicates within the sheet). The VLOOKUP function, on the
other hand, was used to look up a journal in a sheet and return a
corresponding journal from another sheet in the same Excel workbook, that is,
checking if a journal in Sheet1 also appears in sheet2 on the same Excel
workbook. After removing duplicates, relevant studies were chosen based on the
inclusion and exclusion criteria. Remaining studies were then evaluated
against the inclusion criteria, any differences were discussed, and an
agreement was reached. Fig. 1 depicts the study selection procedure. A
standardized form was developed for data extraction, which included key
aspects such as study details (i.e., title, authors, publication year), LLRMA
techniques (i.e., SVD, PCA), application (i.e., image denoising,
classification), main results, and conclusion. Two reviewers independently
extracted data from the included studies.

Table 1: Search strings used in different databases.

Database |  Advanced search string |  Search date |  Results
---|---|---|---
Web of Science |  ALL=((low-rank approximation OR low-rank approximation OR local low-rank approximation OR local low-rank approximation) AND (image compression OR image filtering OR medical images)) |  02 Sep, 2023 |  1097
Scopus |  TITLE-ABS-KEY(((low-rank approximation) OR (low-rank approximation) OR (local low-rank approximation) OR (local low-rank approximation)) AND ((image compression) OR (image filtering) OR (medical images))) |  02 Sep, 2023 |  909
PubMed |  ((low-rank approximation) OR (low-rank approximation) OR (local low-rank approximation) OR (local low-rank approximation)) AND ((image compression) OR (image filtering) OR (medical images)) |  02 Sep, 2023 |  6
Total |  |  |  2012

###  2.3 Inclusion, Exclusion and Quality Assessment

The study included publications that satisfied specific inclusion criteria and
underwent a quality assessment.

####  2.3.1 Inclusion and Exclusion Criteria

The publications were collected from search engines (Scopus, Web of Science,
and PubMed), and relevant studies were selected based on predefined inclusion
and exclusion criteria.

  * •

Inclusion criteria: The publications included in this systematic review met
the requirement of applying matrix approximation. In addition, they met at
least one of the following requirements:

    * i.

In the study, a technique is used to compress or denoise or reconstruct or
complete an image, etc.

    * ii.

The study discusses LLRMA.

    * iii.

The study applies LRMA or LLRMA on medical images.

  * •

Exclusion criteria: Publications were excluded from this review if they met at
least one of the following criteria:

    * i.

The publication is not written in English.

    * ii.

The publication does not discuss LRMA or LLRMA.

    * iii.

The publication is a reduced version of another retrieved publication.

    * iv.

The publication is not peer-reviewed.

Figure 1: Flow chart depicting the study selection process for the systematic
literature review.

####  2.3.2 Quality Assessment

The quality of all the selected publications was evaluated. To determine the
research’s reliability and soundness, publications were assessed using the
following checklist:

  * •

Are the research objectives well-defined? Is the statistical analysis
appropriate for the study’s research question and objectives?

  * •

Does the data support the conclusion? Are the study’s limitations and
implications discussed?

  * •

Are references relevant, up-to-date, and correctly cited? Do other authors
cite the study?

##  3 LRMA Applied to Medical Data

Researchers in medical imaging have proposed various methods to improve the
quality of MRI, CT, microarray, and infrared imaging data. lyra2012improved ,
for instance, proposed using SVD-based LRMA approximation to reconstruct and
denoise multi-dimensional MRI data. mcgivney2014svd , on the other hand, used
SVD to compress the size of the dictionary in the time domain for pattern
recognition, whereas liu2020low  proposed a proximal operator for tensor
nuclear norm approximation based on tensor-train rank-1 (TTr1) decomposition
through SVD. In kopriva2023low , the authors introduced low tensor-train rank
and low multilinear rank approximations to concurrently address despeckling
and compression in tomography images, targeting a specified compression ratio.
In addressing the calculation of optimal regularized inverse matrices subject
to rank constraints, chung2017optimal  proposed an effective rank-update
technique that decomposes the problem into smaller rank-related challenges. In
contrast, yi2021joint  proposed the use of multi-contrast Hankel tensor
completion (MC-HTC) to leverage shared information in multi-contrast datasets,
considering their highly correlated image structure, joint spatial support,
and joint coil sensitivity for joint reconstruction. Meanwhile, in
friedland2006algorithm , an enhanced fixed-rank approximation algorithm
(IFRAA) was proposed to address the limitations of the fixed-rank
approximation algorithm (FRAA) when estimating missing values in a gene
expression matrix.

peng2016accelerated  proposed a method to improve the quality of fast T2
mapping, while asslander2018low  proposed a reconstruction framework for
magnetic resonance fingerprinting. Another area of research has been on
spatiotemporal reconstruction for dynamic MRI, as seen in the works of
ulas2016spatio  and xu2017dynamic . On the other hand, fang2015spatio
proposed a spatiotemporal low-rank total variation method to recover the
arterial spin labelling MRI (ASL-MRI) data. Additionally, researchers have
explored LRMA methods for MRF dictionaries, as discussed in yang2018low .

In other areas of medical imaging, researchers have also proposed various
techniques. For instance, mahoney2006tensor  proposed an extension matrix CUR
decomposition to tensor-based datasets with distinguished mode. ding2018image
proposed an image-domain multi-material decomposition method for dual-energy
computed tomography (DECT) that aims to suppress noise, reduce cross-
contamination, and improve the accuracy of decomposed material images.
Meanwhile, li2020rank  proposed a method that factorizes an image matrix as a
sum of rank-one matrices, where the rank is automatically estimated during
convergence, eliminating the need for pre-specified rank information. On the
other hand, zhen2016descriptor  introduced a novel supervised descriptor
learning (SDL) technique for multi-output regression that can construct
discriminative and compact feature representations to improve multivariate
estimate performance in medical images. The SDL algorithm extracts
discriminative and compact feature representations using LRMA with supervised
manifold regularization. Lastly, he2022fidelity  proposed a multi-modality
medical image fusion method for computer-aided diagnosis applications. The
proposed method suppresses noise in medical images and provides detailed
information for disease diagnosis.

These works highlight the importance of LRMA and regularization techniques in
medical imaging. Table LABEL:LRSummaryTable gives a summary of the above-
discussed publications. This table summarizes the low-rank methods applied to
medical datasets with different modalities. It provides information on the
strengths and weaknesses of the papers that have used low-rank methods in
various applications. The table includes references, year of publication,
application, low-rank method used, strengths, and weaknesses of each
publication.

Table 2: Summary of low-rank method, strengths and weaknesses of papers applying LRMA to medical datasets with different modalities. References |  Year |  Application |  low-rank method |  Strengths |  Weakness
---|---|---|---|---|---
Mahoney et al. mahoney2006tensor  |  2006 |  Image compression and reconstruction |  SVD |  $\bullet$Captures the most essential underlying structures and patterns in the data for compression, reconstruction, and other tasks. |  $\bullet$May not perform well for tensors with multiple equally important modes or similar properties.
Friedland et al. friedland2006algorithm  |  2006 |  Estimating missing values |  SVD |  $\bullet$Local-global algorithm exploiting local data similarity. |  $\bullet$Computational complexity not extensively discussed; may be computationally intensive for large-scale micro-array datasets.
Lyra et al. lyra2012improved  |  2012 |  Denoising MRI images |  SVD |  $\bullet$Estimates number of singular values using AIC, not just visual inspection. |  $\bullet$Equally compresses the dataset.
Cheng et al. cheng2014curvilinear  |  2014 |  Tracking curvilinear structures in dynamic X-ray images. |  Tensor decomposition |  $\bullet$Uses tensor-based algorithm with model propagation for robust tracking. |  $\bullet$Specifically designed for tracking deformable objects in X-ray images; may not apply directly to other types of images or objects.
Mcgivney et al. mcgivney2014svd  |  2014 |  Image compression |  SVD |  $\bullet$Reduces computations without sacrificing signal-to-noise. |  $\bullet$Applies equal weight to different singular values.
Fang et al. fang2015spatio  |  2015 |  Image denoising |  SVD |  $\bullet$Uses joint spatial total variation and temporal low-rank regularization for improved SNR in ASL-MRI. |  $\bullet$Due to low signal-to-noise ratio, multiple acquisitions are required, leading to a longer scan time.
Table 2: Cont. References |  Year |  Application |  low-rank method |  Strengths |  Weakness
---|---|---|---|---|---
Peng et al. peng2016accelerated  |  2016 |  Image reconstruction |  SVD |  $\bullet$Utilizes exponential parametric model in a relaxed manner to balance the trade-off between data consistency and the exponential structure. |  $\bullet$Applies a uniform threshold to enforce Hankel low-rankness across all spatial locations.
Ulas et al. ulas2016spatio  |  2016 |  Image reconstruction |  Nuclear norm minimization |  $\bullet$The suggested approach utilizes a reconstruction model that integrates penalties for dTV sparsity and nuclear norm simultaneously. |  $\bullet$Nuclear norm minimization is sensitive to noise and outliers in the data.
Xu et al. xu2017dynamic  |  2017 |  Reconstruction of dynamic MRI data |  RPCA |  $\bullet$Employs a tighter nonconvex rank approximation, leading to improved image clarity and computational efficiency. |  $\bullet$How effective the method is in different imaging modalities is not explored.
Chung et al. chung2017optimal  |  2017 |  Optimization |  SVD |  $\bullet$Breaks down the optimization issue into smaller subproblems of varying ranks and employs gradient-based techniques capable of leveraging linearity. |  $\bullet$ Employing diverse noise levels in each problem or incorporating larger alterations in $k_{j}$ could lead to extended CPU times and/or elevated reconstruction errors in the update approach.
Assländer et al. asslander2018low  |  2018 |  Reconstruction accuracy noise propagation |  SVD |  $\bullet$The LRMA approximation simplifies the computation by reducing the need for numerous Fourier transformations in the signal evolution. |  $\bullet$The limitations and effectiveness of the proposed method in different imaging modalities are not explored.
Yang et al. yang2018low  |  2018 |  Image compression |  Randomized SVD |  $\bullet$Decreases the memory needed to produce high-resolution MRF maps and enhances the overall speed of the compression process. |  $\bullet$Randomized SVD may perform differently based on the specific properties of the MR fingerprinting matrix.
Table 2: Cont. References |  Year |  Application |  low-rank method |  Strengths |  Weakness
---|---|---|---|---|---
Ding et al. ding2018image  |  2018 |  Image-domain multi-material decomposition for dual-energy CT |  SVD |  $\bullet$Enhances image resolution and accuracy through noise suppression and reduced cross-contamination. |  $\bullet$Equal weighting to singular values in SVD.
Liu et al. liu2020low  |  2020 |  Tensor completion |  SVD |  $\bullet$Utilizes tensor train rank-1 (TTr1) decomposition, enhancing decomposition accuracy. |  $\bullet$Incorporates total variation as regularization, potentially unsuitable for datasets without significant local smoothness.
Li et al. li2020rank  |  2020 |  Matrix completion |  $l_{p}$-norm minimization |  $\bullet$Robust to outliers, no need for rank or noise information. |  $\bullet$Uniform weighting of singular values in $l_{p}$-norm minimization.
Yi et al. yi2021joint  |  2021 |  Image reconstruction |  HOSVD |  $\bullet$Enables tensor low-rankness via higher-order SVD, improving reconstruction of multiple slices. |  $\bullet$Iterative updates and higher-order SVD increase computational complexity.
He et al. he2022fidelity  |  2022 |  Reconstruction and image fusion |  SVD |  $\bullet$Utilizes rank coefficient optimization of LLRMA for multi-modality medical image reconstruction. |  $\bullet$Equal weight application to different singular values in SVD.
Kopriva et al. kopriva2023low  |  2023 |  Despeckling and compression |  Tensor SVD |  $\bullet$Simultaneously suppresses speckle and provides memory-efficient low-dimensional representations. |  $\bullet$Evaluation on a limited dataset may not fully represent OCT image diversity in various clinical scenarios.

##  4 LLRMA Applied to Medical Data

LLRMA is a technique that is applied to medical data to effectively analyze
and extract meaningful information while reducing computational complexity and
noise. The term local low-rank refers to finding low-rank sub-matrices within
localized regions or areas of the data. Medical data often exhibit spatial or
temporal correlations where neighbouring regions exhibit similar patterns or
structures. By applying LLRMA methods, we can exploit these local correlations
and represent the data in each partial region where the rank of each of the
small partial regions is smaller than that of the entire data. This enables
more efficient processing and analysis of medical data, especially in
scenarios with high-dimensional data. Applying LLRMA to medical data provides
a robust framework for processing high-dimensional, spatially or temporally
correlated data. By exploiting the inherent structure and redundancy within
localized regions, efficient processing, noise reduction, dimensionality
reduction, and feature extraction is enabled, facilitating more accurate and
insightful analysis in the medical domain. There are several types of medical
data modalities such as MRI, CT-scan, X-ray, ultra-sound, PET, multi-spectral
imaging and retinal imaging. This section reviews the literature on LLRMA
applied to the different medical image modalities.

###  4.1 LLRMA with MRI

Recently, there has been an increasing focus on developing efficient
techniques to improve the accuracy of clinical diagnosis by denoising 3D MRI
data. Several techniques to denoise 3D MRI data have been suggested in the
literature. fu20163d  proposed a novel method that fully exploits the MRI’s
local and non-local similarity using low-rank tensor approximation. The
adaptive low-rank tensor approximation effectively filters the 3D MRI patch
stacks, which the adaptive higher-order singular value thresholding (AHOSVT)
algorithm solves. While chen2021novel  proposed a novel approach for denoising
MR images by combining a non-local self-similarity technique with a LRMA
method. Another method proposed in xia2017denoising  is based on LLRMA with
regularization of weighted nuclear norm minimization (WNNM) to remove Rician
noise from MR images. Since the method proposed in fu20163d  involves tensor
matricizations, it lacks flexibility. To overcome this drawback,
khaleel2018denoising  proposed a method for denoising MR images degraded by
Rician noise based on low-rank tensor approximation. zhai2018weighted
performed a similar study as xia2017denoising ; the authors used weighted
Schatten p-norm as a minimization method.

mandava2018accelerated  proposed a reconstruction method for multi-contrast
imaging and parameter mapping based on a union of local subspace constraint.
lv2019denoising  conducted a study similar to that of fu20163d , incorporating
forward and inverse variance stabilizing transforms for the Rician
distribution. Both fu20163d  and lv2019denoising  utilized a denoising
technique based on the HOSVD approach. This approach entails calculating
transform coefficients using a hard threshold function. This constrains the
denoising performance. wang2020modified  proposed modified HOSVD (MHOSVD),
which uses a parameterized logarithmic nonconvex penalty function for
optimization to overcome this drawback. chen2021joint  proposed a denoising
method that incorporates an extended difference of Gaussian (DoG) filter and
non-local low-rank regularization to denoise an MR image. While zhang2020image
proposed a novel structure-constrained LRMA (SLR) method for MRI denoising
that exploits geometric information of local MRI image contents and non-local
self-similarity of global MRI image structures. The proposed method
incorporates two complementary regularization techniques, Kernel Wiener
filtering (KWF) and low-rank regularization. On the other hand, zhao2022joint
proposed a joint denoising method to exploit natural information redundancy in
MR diffusion‐weighted images (DWIs) through low-rank patch matrix
approximation. Finally, he2022denoising  proposed a denoising method based on
weighted tensor nuclear norm minimization and balanced non-local patch
tensors.

###  4.2 LLRMA with CT-scan

Denoising CT images is a challenging task that requires sophisticated
algorithms to reconstruct a noiseless image from a noisy one. lei2018denoising
proposed a denoising algorithm that uses low-rank sparse coding to reconstruct
CT images. This method incorporates non-local self-similarity constraint and
group-wise sparse coding noise regularization to reduce sparse coding error.
On the other hand, sagheer2019denoising  proposed another approach for CT
image denoising that takes advantage of the global spatial correlation and
local smoothness properties. The suggested approach involves the extraction of
overlapping cubic patches, followed by vectorization. Similar patches in the
noisy image are then grouped into blocks, forming a third-order tensor. Each
corresponding tensor block is subjected to denoising using tensor singular
value decomposition (t-SVD), while the preservation of edges and assurance of
local smoothness are achieved through the use of total variation (TV). In a
related work by jia2022nonconvex , a super-resolution model tailored for
single 3D medical images was introduced.

Additionally, shen2021ct  recently proposed a novel non-local low-rank
regularization and data-driven tight-frame CT image reconstruction model (NLR-
DDTF). This model integrates non-local low-rank matrix approximation (NLLRMA)
and data-driven tight frame-based regularization for CT image reconstruction.
The study combines the advantages of both NLLRMA and data-driven tight frame-
based regularization, aiming to improve the accuracy of image reconstruction.

###  4.3 LLRMA with X-Ray

cheng2014curvilinear  propose a deformable tracking method based on tensors
and model propagation. The proposed method uses tensors and model propagation
to solve the multi-dimensional assignment problem. The approach addresses the
challenge of tracking anatomical structures and devices prone to appearance
change and low visibility in X-ray images. While hariharan2019preliminary
introduced a denoising technique for 2D digital subtraction angiography (DSA),
employing a weighted local low-rank approach facilitated by an advanced
neurovascular replication system. On the other hand, hariharan2018photon  uses
photon recycling to denoise ultra-low dose X-ray sequences. The method
exploits non-local self-similarity and weighted patch-matching to align
similar image regions in current and previous frames. A noise level-based
LLRMA and weighted aggregation of the patches are used to produce denoised
images. Both methods demonstrate the potential for improving the robustness
and accuracy of medical imaging in challenging environments.

###  4.4 LLRMA with Ultra-sound

sagheer2017ultrasound  proposed two methods of recovering a despeckled 2D
image from the speckled data. Despeckling is the process of removing speckle
noise while maintaining edge preservation without sacrificing speckle
reduction. The LLRMA is the first method used to despeckle ultrasound images
(DLRA). This method converts multiplicative noise to additive noise through
homomorphic filtering before applying weighted nuclear norm minimization
(WNNM) since LLRMA methods for denoising an image are applicable to addictive
noise. In 2019, sagheer2019despeckling  proposed an extension of the method.
However, the direct extension of this approach disrupts the temporal
correlation found in 3D US images, reducing despeckling performance. As a
result, the novel method used tensor LLRMA to denoise 3D ultrasound images.

In a similar study, yang2021ultrasound  proposed a nonconvex LLRMA model based
on non-local similar patch matrices to remove the speckle noise. The proposed
method for ultrasound image restoration employs a nonconvex LLRMA model that
incorporates the WNNM and data fidelity term.

###  4.5 LLRMA with PET

In order to apply a tensor LLRMA effectively and efficiently, xie20193d
suggested a non-local tensor low-rank framework for dynamic PET reconstruction
using a t-SVD-based technique. The suggested methodology incorporates hidden
data and improves optimization efficiency by using an expectation maximization
(EM) method as a fidelity term and a TV constraint in addition to the
regularization method to extract local structure. Distributed optimization is
used to jointly solve a Poisson PET model that combines these regularizations.
Beyond spatiotemporal correlation, the Poisson model contains a novel non-
local low-rank tensor constraint that captures data correlation in multiple
dimensions in dynamic PET. This model recovers highly degraded data and
complements structures in low-active frames by leveraging temporal information
between frames and non-local self-similarities inside each frame to increase
structured sparsity for each image.

In contrast, yang2020pet  proposed a PET image denoising method based on LRMA
that exploits self-similarity and low-rank of the images. The proposed method
pre-filters the original noisy image in the first iteration to improve the
selection accuracy of similar patches. However, instead of directly grouping
patches on the pre-filter image in the first iteration, the coordinates of the
pre-filter image for the selected patches are recorded. These coordinates are
used to group the patches of the noisy image. Then similar patches are grouped
as column vectors to form a group matrix estimated using the LRMA method. All
denoised patches are aggregated to obtain an initially denoised image. The
enhanced back projection technique is applied at each iteration to improve
denoising results and retrieve information loss due to a residual image. A
final denoised image is obtained after several iterations.

###  4.6 LLRMA with multispectral imaging

he2019segmenting  proposed a learning-based approach that takes advantage of
multispectral retinal image’s spatial and spectral characteristics. The
proposed method first represents each pixel’s feature vector in each spectral
MSI slice using a 2D spatial-spectral matrix. A generalized low-rank matrice
approximation (GLRAM) framework is used to solve the feature learning problem.
The purpose of GLRAM is to construct low-dimensional, compact representations
of a sequence of spatial-spectral matrices. The proposed method creates a 2D
spatial-spectral matrix of local binary pattern features to accurately
describe the spatial attributes of each spectrum image.

###  4.7 LLRMA with retinal imaging

ren2017drusen  proposed a method for drusen segmentation in retinal images,
employing a supervised feature learning approach. The technique utilizes GLRAM
and supervised manifold regularization to extract discriminative and concise
descriptors from image patches extracted from retinal images. The learned
features are closely associated with drusen, and potentially may be free of
irrelevant information for accurate drusen identification against the
background. The support vector machine classifier is used in the final stage
of the proposed model to determine the presence of drusen in the pixels of
test images.

###  4.8 LLRMA with a Mixture of modalities

Various methods for enhancing the quality and accuracy of medical images have
been suggested in the existing literature. liu2015medical  proposed a method
that uses LLRMA with NNM constraint and block matching to fuse medical images
and overcome distortion and information loss. zhong2016predict  proposed a
kNN-regression method for predicting CT images from MRI data by finding
nearest neighbors and using supervised descriptor learning based on LRMA and
manifold regularization. In contrast, yang2018predicting  proposed a method
for predicting pseudo-CT images from T1w and T2w MRI data using learned
nonlinear local descriptors and feature matching. The method removes bias
field artifacts from MR images using a bias correction algorithm. The
intensity normalization technique then reduces the variance between MR images
from different patients.

Another method proposed in liu2019medical  uses non-local self-similar
redundancy and low-rank prior techniques to improve the resolution of low-
resolution medical images and recover high-resolution images. The proposed
method applies low-rank filtering to the group matrices to obtain pixel
estimates. A weighted averaging approach is used to handle the issue of
multiple estimates for each pixel. They performed a low-rank reconstruction
and iteratively applied a sub-sampling consistency constraint to improve the
method’s performance. These methods enhance medical image analysis and
visualization, improving the accuracy of medical diagnosis and treatment.

Table 5 shows a comprehensive summary of similarity measures, low-rank
methods, strengths and weaknesses of papers applying LLRMA technique to
medical datasets with different modalities.

Table 5: Summary of measure of similarity, low-rank method, strengths, and drawbacks of papers employing LLRMA on medical datasets with different modalities. References |  Year |  Application |  Measure of Similarity |  low-rank Method |  Strengths |  Weakness
---|---|---|---|---|---|---
Liu et al. liu2015medical  |  2015 |  Image fusion |  Weighted Euclidean distance |  SVD |  $\bullet$Addresses fusion distortion and information loss. |  $\bullet$Applies equal weights to different singular values for low-rank approximation.
Fu et al. fu20163d  |  2016 |  Denoising MRI image |  KNN |  tensor approximation |  $\bullet$Considers non-local spatial self-similarity and cross-slice correlation in 3D MR images |  $\bullet$Lacks flexibility due to tensor matricization khaleel2018denoising .
Zhong el al.zhong2016predict  |  2016 |  Prediction of CT image from MRI |  KNN |  GLRA |  $\bullet$Utilizes supervised descriptor learning based on LRMA and manifold regularization to optimize local MR image patch descriptors |  $\bullet$KNN sensitivity to data noise and outliers.
Xia et al. xia2017denoising  |  2017 |  Denoising MRI images |  Block matching |  Weighted nuclear norm minimization |  $\bullet$Employs a flexible thresholding strategy. $\bullet$Allows image attenuation without losing crucial features. |  $\bullet$Inability to fully exploit correlation among different dimensions in high-dimensional MR data.
$\bullet$Loss of high-frequency components, such as edges and fine structures.
Ren et al. ren2017drusen  |  2017 |  Binary classification |  Distance |  GLRA |  $\bullet$Diminishes image patch dimensionality while enhancing discriminative power. |  $\bullet$ Not always practical since it requires a large amount of labelled training data.
Table 5: Cont. References |  Year |  Application |  Measure of similarity |  low-rank method |  Strengths |  Weakness
---|---|---|---|---|---|---
Sagheer et al. sagheer2017ultrasound  |  2017 |  Denoising ultrasound images |  Block matching |  Weighted nuclear norm minimization |  $\bullet$Includes a preprocessing stage considering statistical properties of ultrasound images. |  $\bullet$Extending this method can compromise the temporal correlation in 3D ultrasound images, adversely affecting despeckling performance.
Lei et al. lei2018denoising  |  2018 |  Denoising CT images |  KNN |  SVD |  $\bullet$Utilizes group-wise sparse coding for noise minimization. $\bullet$Employs Bayesian interpretation for adaptive parameter learning. |  $\bullet$Robustness to different types or levels of noise is not extensively analyzed. $\bullet$Computational requirements are not thoroughly discussed, potentially affecting feasibility.
Hariharan et al. hariharan2018photon  |  2018 |  Denoising X-ray images |  Weighted patch matching |  SVD |  $\bullet$Enables parallel computing by denoising patches independently. |  $\bullet$ Denoised images may exhibit mild blurring around instrument edges.
Khaleel et al. khaleel2018denoising  |  2018 |  Denoising MRI images |  Euclidean distance |  t-SVD |  $\bullet$Demonstrates flexibility and computational efficiency compared to traditional techniques. |  $\bullet$The proposed method lacks generalizability for different medical image modalities.
Mandava et al. mandava2018accelerated  |  2018 |  Reconstruction method for multi-contrast imaging |  Subspace basis |  SVD |  $\bullet$Utilizes a union of local subspace constraints coupled with a sparsity-promoting penalty. |  $\bullet$The reconstruction time for MOCCO-LS is also longer.
Zhai et al. zhai2018weighted  |  2018 |  Denoising MRI images |  Block matching |  Weighted Schatten p-norm minimization |  $\bullet$The proposed method has a global optimum efficiently solvable by the generalized iterated shrinkage method when weights are in non-decreasing order. |  $\bullet$Involves solving a computationally expensive weighted Schatten p-norm minimization problem.
Table 5: Cont. References |  Year |  Application |  Measure of similarity |  low-rank method |  Strengths |  Weakness
---|---|---|---|---|---|---
Yang et al. yang2018predicting  |  2018 |  Prediction of CT image from MRI |  KNN |  GLRA |  $\bullet$Utilizes a combination of dense scale-invariant feature transform descriptors and normalized raw patches for MR images, enhancing bone identification capability. |  $\bullet$Lacks detailed discussion on the computational requirements, potentially impacting feasibility.
Sagheer et al. sagheer2019despeckling  |  2019 |  Denoising ultrasound images |  $l_{2}$ distance |  Tensor approximation |  $\bullet$Effectively exploits spatial and temporal correlation in data for denoising. |  $\bullet$ $l_{2}$ distance treats each pixel uniformly, neglecting contextual information and varying intensity significance across regions.
Liu et al. liu2019medical  |  2019 |  Denoising CT and MRI images |  Euclidean distance |  SVD |  $\bullet$Combines low-rank prior and nonlocal self-similar prior with iterative refinement for improved results. |  $\bullet$Euclidean distance solely considers pixel intensities, overlooking inherent uncertainty or variability due to noise.
Xie et al. xie20193d  |  2019 |  Denoising and PET reconstruction |  Euclidean distance |  t-SVD |  $\bullet$Spontaneously exploits inner temporal correlation without tracer information or model fitting. |  $\bullet$Limited effectiveness for data with involuntary but noticeable motion. Not guaranteed to perform well in 3D PET reconstruction.
Sagheer et al. sagheer2019denoising  |  2019 |  Denoising CT image |  $l_{2}$ distance |  Tensor approximation |  $\bullet$Denoises images using tensor LRMA and tensor total variation technique. |  $\bullet$Effectiveness depends on image characteristics, may not be suitable for all types of CT images.
Table 5: Cont. References |  Year |  Application |  Measure of similarity |  low-rank method |  Strengths |  Weakness
---|---|---|---|---|---|---
Hariharan et al. hariharan2019preliminary  |  2019 |  Denoising X-ray images |  |  Weighted low-rank |  $\bullet$Applies constrained low-rank approximations in both rows and columns. |  $\bullet$The choice of a fixed patch size may not adequately consider anatomic variability.
Lv et al. lv2019denoising  |  2019 |  Denoising MRI images |  KNN |  Tensor approximation |  $\bullet$ Exploits non-local and low-rank properties of the grouped image block. |  $\bullet$Denoising methods based on HOSVD constrains the denoising performance wang2020modified .
He et al. he2019segmenting  |  2019 |  Image segmentation |  feature point matching |  GLRA |  $\bullet$Enables the learning of patterns closely related to their class labels for generating more discriminative representations. |  $\bullet$The spatial-spectral features of other retinal diseases may vary from those of diabetic retinopathy, potentially affecting the efficacy of the proposed approach.
Yang et al. yang2020pet  |  2020 |  Denoising PET images |  Euclidean distance |  SVD |  $\bullet$Utilizes an enhanced back-projection step to compensate for the loss of details, improving the denoising result |  $\bullet$As Euclidean distance is not robust to noise in image data, a small amount of noise can significantly impact the distance calculation, resulting in inaccurate similarity measurements.
Wang et al. wang2020modified  |  2020 |  Denoising MRI images |  Block matching |  HOSVD |  $\bullet$Uses nonconvex logarithmic regularization and nonlocal similarity to apply an adaptive multilinear tensor rank approximation approach. |  $\bullet$The proposed method may not effectively eliminate noise in MR images with very low signal-to-noise ratios.
Zhang et al. zhang2020image  |  2020 |  Denoising MRI images |  Block matching |  SVD |  $\bullet$Uses a structure-constrained LRMA model that exploits both local and nonlocal priors. |  $\bullet$The method’s effectiveness may depend on the specific characteristics of the images to be denoised.
Table 5: Cont. References |  Year |  Application |  Measure of similarity |  low-rank method |  Strengths |  Weakness
---|---|---|---|---|---|---
Yang et al. yang2021ultrasound  |  2021 |  Ultrasound image restoration |  Euclidean distance |  SVD |  $\bullet$By incorporating the notion of fidelity, the method accounts for the unique characteristics of ultrasound images. |  $\bullet$The choice of a fixed patch size may not sufficiently account for anatomic variability.
Chen et al. chen2021joint  |  2021 |  denoising MRI images |  Euclidean distance |  SVD |  $\bullet$High-frequency components are retrieved using DoG filter. |  $\bullet$Euclidean distance is based on pixel intensities and does not consider the inherent uncertainty or variability caused by noise.
Chen et al. chen2021novel  |  2021 |  Denoising MRI images |  Euclidean distance |  SVD |  $\bullet$Produces better approximation especially when the noise level is high. |  $\bullet$The method’s sensitivity with respect to parameter settings, i.e., patch size, has not been thoroughly investigated.
Shen et al. shen2021ct  |  2021 |  reconstructing CT images |  Euclidean distance |  SVD |  $\bullet$Uses an asymmetric treatment for image reconstruction and Radon domain inpainting |  $\bullet$Euclidean distance is based on pixel intensities and does not consider the inherent uncertainty or variability caused by noise.
He et al. he2022denoising  |  2022 |  Denoising MRI images |  K-means clustering |  Tensor approximation |  $\bullet$Constructs highly correlated 3D non-local patch tensors.
$\bullet$Exploits the effectiveness of t-SVD using balanced 3D non-local patch tensors. |  $\bullet$If the estimate of $\sigma_{R}$ is not accurate, the VST conversion may introduce residual errors or distortions that affect denoising performance.
Zhao et al. zhao2022joint  |  2022 |  Denoising diffusion weighted images |  Block matching |  SVD |  $\bullet$The proposed method has application to other clinical MRI protocols besides diffusion MRI. |  $\bullet$If a large patch window size is used the performance of the method may degrade.
Jia et al. jia2022nonconvex  |  2022 |  High-resolution |  Euclidean distance |  Tensor decomposition |  $\bullet$ Avoids estimation bias caused by the traditional convex procedure. |  $\bullet$ Increased computational cost.

##  5 Medical Datasets Employed with LRMA and LLRMA

Medical image datasets are collections of digital images derived from various
imaging technologies such as PET, CT, MRI and others. These datasets serve as
valuable resources for training and developing machine-learning algorithms and
computer-aided diagnosis systems in the medical domain. In this section, we
briefly summarize several medical datasets that have been used for various
machine-learning applications.

One such dataset used in various studies fu20163d ; xia2017denoising ;
zhai2018weighted ; khaleel2018denoising ; asslander2018low ; lv2019denoising ;
liu2019medical ; wang2020modified ; he2022denoising ; jia2022nonconvex  is the
synthetic 3D MRI dataset from the BrainWeb database. This dataset comprises
T1-weighted (T1w) data without noise, along with T2-weighted (T2w) data and
proton density-weighted (PDw) data associated with the healthy brain.
Furthermore, these studies used the real 3D MRI dataset from the open access
series of imaging studies (OASIS) data to validate the reliability of the
proposed models. There are 416 individuals aged 18 to 96 years, representing a
cross-sectional sample. The participants consist of both males and females who
are all right-handed. Among the subjects, there are 100 individuals aged sixty
and older in the study who have been clinically diagnosed with very mild to
moderate Alzheimer’s disease (AD).

In addition to the above-mentioned datasets, khaleel2018denoising  and
wang2020modified  used Auckland cardiac MRI Atlas and multi-modal brain tumour
image segmentation challenge (BraTS) 2013 databases, respectively. The
Auckland cardiac MRI Atlas dataset utilized a 1.5T SIEMENS Avanto MR device to
acquire T1-weighted images, true FISP Cines, MR tagging, and contrast MRI.
T1-weighted images and true FISP Cines were obtained through a retrospectively
controlled steady-state free precession anatomic spin echo sequence, employing
a phased array surface coil and ECG R-wave triggering. In contrast, the BraTS
database offered pre-optimized MRI datasets through various image
preprocessing methods, making them suitable for further applications like CAD
segmentations in simulation scenarios. On the other hand, mahoney2006tensor
used hyperspectral image data that comprises of 59 data cubes from 59
biopsies, with each biopsy representing a different patient. Among these
biopsies, there are 20 normal samples, 19 samples of benign adenoma, and 20
samples of malignant carcinoma colon. Each data cube contains a series of 128
grayscale images captured at a magnification of 400X. The images cover a
frequency range of approximately 440 nm to 700 nm. Each image has dimensions
of 495 pixels in width and 656 pixels in height.

yousefi2021diagnostic  used a breast cancer dataset that included a total of
208 participants in the breast screening study, categorized as either healthy
(asymptomatic) or sick (symptomatic). The sick patients were diagnosed with
breast cancer or non-cancerous conditions with symptoms using clinical breast
examination (CBE) and mammography. The median age of the participants in the
study was 60 years, and they represented different ethnicities: 77 (37%) were
Caucasian, 57 (27.4%) were African, 72 (34.6%) were Pardo, 1 (0.5%) was
Mulatto, and 1 (0.5%) was indigenous. All participants underwent infrared
imaging using a FLIR thermal caeach indmera (model SC620). The acquired images
had a spatial resolution of $640\times 480$ pixels. While hariharan2018photon
used 20 clinical sequences consisting of 3 different sites, 7 patients, and
500 images. These sequences were acquired with a significantly reduced
radiation dose, called ultra-low dose, set at 50% of the standard low dose.
These clinical ultra-low-dose sequences were acquired during cardiac and renal
electrophysiological procedures (EP). Image sequences were acquired with a
matrix size of either $1024\times 1024$ or $960\times 960$ and at a rate of 3
frames per second. cheng2014curvilinear  used X-ray fluoroscopic sequences
from 17 clinical cases, with data acquired at a pixel size of $512\times 512$
and a resolution of $0.4313mm\times 0.3450mm$.

sagheer2019despeckling  used three datasets, including real 3D ultrasound
images and two distinct sets of simulated ones. The real 3D ultrasound data
comprises images captured pre and post tumor resection, whereas the simulated
data comprises phantom images with dimensions of $128\times 128\times 128$ and
images with a frequency of 20Hz sized at $512\times 512\times 3$,
respectively. shen2021ct  used a head image with smaller details, a head image
and a brain image with more details to evaluate the proposed method.
hariharan2019preliminary  used X-ray sequences obtained at three different
concentration levels of the contrast agent and four different dose levels of
the standard dose level. ren2017drusen  used a STARE and DRIVE database; the
STARE database contained 400 retinal fundus images of the retina, each with a
size of $700\times 605$. Sixty-three images were clinically found to contain
drusen, while the DRIVE database contained 40 RGB color retinal images; each
image is $768\times 584$. he2022fidelity  used the brain atlas database, which
contained sections on the normal brain, cerebrovascular disease, neoplastic
disease, degenerative disease, and infectious disease of MR and PET images.
he2019segmenting  employed MSI comprising 40 sequences indicative of unhealthy
conditions and ten sequences representing healthy conditions. These sequences
captured images of the oculus dexter (OD) or oculus sinister (OS) from a
cohort of 20 patients diagnosed with diabetic retinopathy (DR) and five
healthy subjects. The images were in DICOM format with a bit depth of 16, each
having dimensions of $2048\times 2048$.chen2021novel  and chen2021joint  used
T2 MR data obtained from a healthy volunteer with fast-field-echo sequences:
triceps surae of $1050/8.3ms$, field of view of $280\times 280mm^{2}$, flip
angle of 111∘. asslander2018low  used a single slice of MR fingerprinting data
of a healthy volunteer’s brain acquired with a 3T Skyra scanner.
kopriva2023low  employed three-dimensional optical coherence tomography (OCT)
images centered on the macula, obtained using the Topcon 3D OCT-1000 scanner.
Each 3D OCT image consisted of 64 two-dimensional scans, each with dimensions
of $480\times 512$ pixels.

Table 10: This table provides an overview of the various datasets. Each dataset is described by a brief data description, the type of data, the tasks that can be performed on the dataset, the tasks performed, the links to access the dataset, and the publications that have used the data. The x symbol indicates that the possible tasks have not been performed on that dataset, while $\checkmark$ indicates that the task has been performed on that particular dataset. Where dataset information was missing on a particular publication, the term Not provided was used for columns such as Data description, Link and Dataset while the term Not applicable was used in the Task and Perfomed task columns due to the missing information. Dataset |  Data description |  Data type |  Task |  Performed task |  Link |  Publications using this dataset
---|---|---|---|---|---|---
Synthetic 3D BrainWeb MRI |  Comprises T1w and T2w data without noise and a PDw data corresponding to the healthy brain. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Multi-modal fusion |  x
x
✓
x |  http://brainweb.bic.mni.mcgill.ca/brainweb |  Fu et al. fu20163d , Xia et al. xia2017denoising , Zhai et al. zhai2018weighted , Khaleel et al. khaleel2018denoising , Asslander et al.asslander2018low , Lv et al. lv2019denoising , Liu et al. liu2019medical , Wang et al. wang2020modified , Chen et al. chen2021joint , He et al.he2022denoising
Real 3D MRI (OASIS) |  Includes 416 cross-sectional subjects (aged 18-96) with 3-4 T1-weighted MRI scans per subject, including both right-handed men and women. It also features 100 subjects over 60 diagnosed with very mild to moderate Alzheimer’s disease and a reliability subset of 20 nondemented subjects imaged within 90 days. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Multi-modal fusion |  x
x
✓
x |  http://www.oasis-brains.org/) |  Fu et al. fu20163d , Xia et al. xia2017denoising , Zhai et al. zhai2018weighted , Khaleel et al. khaleel2018denoising , Lv et al. lv2019denoising , Wang et al. wang2020modified , He et al.he2022denoising , Jia et al. jia2022nonconvex
AMRG Cardiac MRI Atlas |  The dataset comprises a labeled collection of MRI images depicting the heart of a healthy patient. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Multi-modal fusion |  x
x
✓
x |  https://www.cardiacatlas.org/amrg-cardiac-atlas/ |  Khaleel et al. khaleel2018denoising
BraTS |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Wang et al. wang2020modified
Not provided |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Chen et al. chen2021joint
Real 3D CT |  Not provided |  images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Multi-modal fusion |  x
x
✓
x |  http://www.via.cornell.edu/databases |  Sagheer et al. sagheer2019denoising
Table 10: Cont. Dataset |  Data description |  Data type |  Task |  Performed task |  Link |  Publications using this dataset
---|---|---|---|---|---|---
Low dose CT |  Not provided |  images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Multi-modal fusion |  x
x
✓
x |  http://www.aapm.org/GrandChallenge/LowDoseCT/ |  Sagheer et al. sagheer2019denoising
Simulated brain DWI |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Zhao et al. zhao2022joint
In vivo human brain DWI |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Zhao et al. zhao2022joint
T2 MR data |  Obtained from a healthy volunteer with fast-field-echo sequences: triceps surae of $1050/8.3ms$, field of view of $280\times 280mm^{2}$, flip angle of 111∘. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Multi-modal fusion |  x
x
✓
x |  Not provided |  Chen et al. chen2021novel , Chen et al. chen2021joint
DSC-MRI brain perfusion sequence |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Ulas et al. ulas2016spatio
in-vivo breath-hold cardiac perfusion sequence |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Ulas et al. ulas2016spatio
Cardiac perfusion |  Not provided |  Images |  Not applicable |  Not applicable |  https://cai2r.net/resources/ |  Xu et al. xu2017dynamic
Cardiac cine |  Not provided |  Images |  Not applicable |  Not applicable |  https://cai2r.net/resources/ |  Xu et al. xu2017dynamic
In-vivo MRI data |  Not provided |  Images |  Not applicable |  Not applicable |  http://shorty.usc.edu/class/591/fall04/ |  Lyra et al. lyra2012improved
Multicontrast multichannel 2D human brain datasets |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Yi et al. yi2021joint
CT images |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Lei et al. lei2018denoising
CT images |  A head image with smaller details, a head image and a brain image with more details. |  Images |  Not applicable |  Not applicable |  Not provided |  Shen et al. shen2021ct
Table 10: Cont. Dataset |  Data description |  Data type |  Task |  Performed task |  Link |  Publications using this dataset
---|---|---|---|---|---|---
|  The image sequences were acquired at 3 frames per second. There were 20 clinical sequences, that is, 7 patients and 500 clinical images. |  Images/
video |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Object detection |  x
x
✓
x |  Not provided |  Hariharan et al. hariharan2018photon
X-ray images |  X-ray fluoroscopic sequences obtained from 17 clinical cases. |  Images/
video |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Tracking of deformable objects |  x
x
x
✓ |  Not provided |  Cheng et al. cheng2014curvilinear
X-ray images |  X-ray sequences are obtained at three different concentration levels of the contrast agent and four different dose levels of the standard dose level. |  Images/
video |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Dose optimization |  x
✓
x
x |  Not provided |  Hariharan et al. hariharan2019preliminary
|  A single slice MR fingerprinting data of a healthy volunteer’s brain that was acquired with a 3T Skyra scanner. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Image reconstruction |  x
x
x
✓ |  Not provided |  Asslander et al. asslander2018low
Real ultrasound data |  Not provided |  Images/
video |  Not applicable |  Not applicable |  Not provided |  Sagheer et al. sagheer2017ultrasound
Simulated ultrasound data |  Not provided |  Images/
video |  Not applicable |  Not applicable |  Not provided |  Sagheer et al. sagheer2017ultrasound
Simulated ultrasound images |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Yang et al. yang2021ultrasound
Real ultrasound images |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Yang et al. yang2021ultrasound
Digital mouse phantom |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Yang et al. yang2020pet
Human brain phantom |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Yang et al. yang2020pet
Table 10: Cont. Dataset |  Data description |  Data type |  Task |  Performed task |  Link |  Publications using this dataset
---|---|---|---|---|---|---
Real 3D ultrasound data |  Consist of three dimensional ultrasound images before and after tumor resection. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Image reconstruction |  x
x
✓
x |  http://www.bic.mni.mcgill.ca/ |  Sagheer et al. sagheer2019despeckling
Simulated 3D ultrasound data |  consist of phantom images corresponding to a size of $128\times 128\times 128$. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Image reconstruction |  x
x
✓
x |  https://in.mathworks.com/matlabcentral/ |  Sagheer et al. sagheer2019despeckling
Simulated 3D ultrasound data |  consist of simulated 3D images at a frequency of 20Hz of size $512\times 512\times 3$. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Image reconstruction |  x
x
✓
x |  http://splab.cz/en/research/zpracovani-medicinskych-signalu/databaze/artery |  Sagheer et al. sagheer2019despeckling
Zubal head phantoms |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Xie et al. xie20193d
Real Cardiac data |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Xie et al. xie20193d
CT lung data |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Liu et al. liu2019medical
CT cardiac data |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Liu et al. liu2019medical
T1, T2 weighted MRI |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Zhong et al. zhong2016predict , Yang et al. yang2018predicting , Peng et al. peng2016accelerated
CT images |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Zhong et al. zhong2016predict , Yang et al. yang2018predicting
MSI images |  Consist of 40 sequences indicative of unhealthy conditions and ten sequences representing healthy conditions. The images were in DICOM format with a bit depth of 16, each having dimensions of $2048\times 2048$. |  Images |  $\bullet$DR lesion segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Image reconstruction |  ✓
x
x
x |  Not provided |  He et al. he2019segmenting
Table 10: Cont. Dataset |  Data description |  Data type |  Task |  Performed task |  Link |  Publications using this dataset
---|---|---|---|---|---|---
STARE database |  It contains 400 retinal fundus images of the retina, each image having a size of $700\times 605$. Sixty-three images were clinically found to contain drusen. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$ Optic disc localization
$\bullet$Generic lesion detection |  x
✓
x
x |  http://cecas.clemson.edu/~ahoover/stare/ |  Ren et al. ren2017drusen
DRIVE database |  It contains 40 RGB color retinal images; each image is $768\times 584$. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$ Optic disc localization
$\bullet$Generic lesion detection |  x
✓
x
x |  https://www.isi.uu.nl/Research/Databases/ |  Ren et al. ren2017drusen
Hyperspectral image |  Comprises 59 sets of data cubes obtained from 59 biopsies, with each biopsy representing a different patient. Among these biopsies, there are 20 normal samples, 19 samples of benign adenoma, and 20 samples of malignant carcinoma colon. |  Images |  $\bullet$Target detection
$\bullet$Classification
$\bullet$ Anomaly detection
$\bullet$Image compression |  x
✓
x
✓ |  Not provided |  Mahoney et al. mahoney2006tensor
Brain atlas |  Contains the sections on normal brain, cerebrovascular disease, neoplastic disease, degenerative disease, and infectious disease of MR and PET images |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Multi-modal fusion |  x
x
x
✓ |  http://www.med.harvard.edu/AANLIB/home.html |  He et al. he2022fidelity
Infrared images |  A total of 208 individuals were included in the study for breast screening, consisting of both healthy individuals without symptoms and individuals with symptoms. |  Images |  $\bullet$Image segmentation
$\bullet$Classification
$\bullet$Image denoising
$\bullet$Lesion Detection |  x
✓
x
x |  Not provided |  Yousefi et al. yousefi2021diagnostic
Real clinical data |  Not provided |  Images |  Not applicable |  Not applicable |  Not provided |  Ji et al. ji2020medical
OCT |  OCT images of normal eyes acquired with the Topcon 3D OCT-1000 scanner |  images |  $\bullet$Image segmentation
$\bullet$image compression
$\bullet$Image despeckling
$\bullet$Classification |  x
✓
✓
x |  https://pan.baidu.com/share/init?surl=edkG7k8W3Wkjhq8vffCYng |  Kopriva et al. kopriva2023low

##  6 Similarity Measure Algorithms

We delve into various similarity measures that can be utilized for medical
data analysis and pattern recognition in general. The concept of similarity
measures play a crucial role in various fields of study, such as data mining,
ML, and information retrieval. Similarity measures are used to quantify the
degree of relatedness between two data points based on some specified
criteria. In medical data analysis, data embeddings can be used to represent
relationships between medical conditions and symptoms. For instance, an
embedding could be created to show that the symptom of chest pain is related
to the medical condition of myocardial infarction, while shortness of breath
is related to heart failure. This is in line with the medical notion of
disease categorization and conceptual hierarchies, which can aid in diagnosis,
treatment planning, and patient outcomes. By using latent representations in
medical data analysis, healthcare professionals can identify patterns, cluster
similar conditions together, and make informed decisions. Thus, deriving
similarities whether among physical concepts or abstract ones is very
important in pattern recognition. Similarity measures are also used in LLRMA
to find similar patches in a medical image when a LRMA is in effect for
denoising, dimension reduction or compression, feature extraction and etc.

This section discusses in detail five different similarity measure algorithms
that are used in the LLRMA methods applied to medical data discussed in
Section 4. These similarity measures include the K-nearest neighbor (KNN)
algorithm fix1989discriminatory , K-means clustering macqueen1967some ,
Euclidean distance deisenroth2020mathematics  and weighted Euclidean distance
zhou2003ranking , weighted patch matching barnes2010generalized , and block
matching using $l_{1}$ norm or Manhattan distance mairal2009non . We will
provide an in-depth explanation of each method, its advantages and
limitations, and its specific use cases. By understanding the characteristics
and performance of these similarity measures, we hope researchers and
practitioners can choose the most appropriate measure while using LLRMA.

###  6.1 Distance Measures

In pattern recognition and ML, a distance measure as stated in Eq. 1,
quantifies the degree of relatedness between two objects on their features or
attributes bishop2006pattern ; goodfellow2016deep ; roberts2022principles .
Distance measures are often referred to as distance functions or metrics. They
are used to compare the degree of similarity or difference between two data
points, and they are a fundamental component of many learning algorithms. A
distance function assigns a numerical value to the similarity between two
objects (we are using it in the broadest sense to refer to things that can be
represented in a mathematically measurable space), with a smaller value
indicating a higher degree of similarity and a larger value indicating a
higher degree of dissimilarity. We have shown a geometric illustration of this
notion in Fig. 2 and Fig.3.

$d:X\times Y\to\mathbb{R}_{\geq 0}.$ (1)

There are various types of distance measures (in fact there is a whole field
called Measure Theory dedicated to this and more such as volume, probability,
and Lebesgue measure), including the Euclidean distance, Manhattan distance,
and Hamming distance gray2009probability ; cohn2013measure ; halmos2013measure
; hamming1950error . Each distance measure has its own mathematical
formulation, and is appropriate for different types of data, depending on the
specific features or attributes being compared roberts2022principles . The
choice of a suitable distance measure is essential in ensuring the accuracy
and effectiveness of learning algorithms in various applications. In order for
a function to be considered a valid distance measure, it must satisfy four key
properties:

  * •

Non-negativity: If $d(\mathbf{a},\mathbf{b})$ is the distance between points
$\mathbf{a}\in X$ and $\mathbf{b}\in Y$, then $d(\mathbf{a},\mathbf{b})\geq
0$.

  * •

Identity: If $\mathbf{a}\in X$ and $\mathbf{b}\in Y$ are identical points,
then $d(\mathbf{a},\mathbf{b})=0$.

  * •

Symmetry: If $\mathbf{a}\in X$ and $\mathbf{b}\in Y$ are any two points, then
$d(\mathbf{a},\mathbf{b})=d(\mathbf{b},\mathbf{a})$.

  * •

Triangle inequality: If $\mathbf{a}\in X$, $\mathbf{b}\in Y$, and
$\mathbf{c}\in Z$ are any three points, then
$d(\mathbf{a},\mathbf{b})+d(\mathbf{b},\mathbf{c})\geq
d(\mathbf{a},\mathbf{c})$.

These four properties ensure that a true distance measures a valid
mathematical function and produces meaningful and consistent results. Distance
measures satisfying these properties are commonly used in various
applications, such as clustering, classification, and pattern recognition.

The Euclidean distance (often referred to as the $l_{2}$ norm) and cosine
similarity are two commonly used measures of similarity between vectors in
tensor analysis deisenroth2020mathematics . Euclidean distance is a measure of
the straight-line distance between two points in an $n$-dimensional space,
while cosine similarity measures the cosine of the angle between two vectors.
The relationship between these two measures can be understood through their
mathematical connection, which arises from the geometric interpretation of
vectors. In particular, given two vectors $\mathbf{a}$ and $\mathbf{b}$, the
Euclidean distance $d_{E}(\mathbf{a},\mathbf{b})$ between them is given by the
square root of the sum of the squared differences in their coordinates, i.e.,

$\displaystyle d_{E}(\mathbf{a},\mathbf{b})$
$\displaystyle=\sqrt{\sum_{i=1}^{n}(\mathbf{a}_{i}-\mathbf{b}_{i})^{2}}$ (2)
$\displaystyle=\sqrt{(\mathbf{a}-\mathbf{b})\cdot(\mathbf{a}-\mathbf{b})}.$
(3)

On the other hand, cosine similarity is given by the dot product of the two
vectors, normalized by the product of their magnitudes, i.e.,

$\cos(\theta)=\frac{\mathbf{a}\cdot\mathbf{b}}{\lVert\mathbf{a}\rVert\lVert\mathbf{b}\rVert}.$
(4)

The cosine similarity can be related to the Euclidean distance by the identity

$\displaystyle\cos(\theta)$
$\displaystyle=\frac{\mathbf{a}\cdot\mathbf{b}}{\lVert\mathbf{a}\rVert\lVert\mathbf{b}\rVert}$
(5)
$\displaystyle=\frac{\sum_{i=1}^{n}a_{i}b_{i}}{\sqrt{\sum_{i=1}^{n}a_{i}^{2}}\sqrt{\sum_{i=1}^{n}b_{i}^{2}}},$
(6)

which shows that the cosine similarity is a normalized version of the dot
product of the vectors. Therefore, while Euclidean distance measures the
absolute distance between two vectors, cosine similarity measures their
relative orientation. These measures are often used in combination with ML
algorithms such as k-means clustering, nearest neighbor classification, and
LLRMA where they play complementary roles in measuring similarity and
distance.

Figure 2: The Euclidean distance is a commonly used measure of similarity
between two points in a Euclidean space. It is defined as the square root of
the sum of the squared differences between the corresponding coordinates of
the two points as shown in this illustration where $\mathbf{a}$ and
$\mathbf{b}$ are two vectors in $\mathbb{R}^{2}$. These can be, for example,
representations of two MRI scans as depicted in this figure. However, this
notion generalizes to $\mathbb{R}^{n}$. Figure 3: The dot product between two
vectors ($\mathbf{a}$ and $\mathbf{b}$), is defined as the sum of the products
of their corresponding components. As shown in the figure, it is related to
the Euclidean distance.

###  6.2 Euclidean Distance and Weighted Euclidean Distance

The Euclidean distance can be used to measure the similarity between two
vectors with respect to their magnitude and direction. However, the weighted
Euclidean distance (WED) is a modification of Euclidean distance, where each
coordinate of the vectors is multiplied by a weight factor. These weights are
used to adjust the contribution of each coordinate to the distance measure.
This is computed as:

$d_{w}(\mathbf{a},\mathbf{b})=\sqrt{\sum_{i=1}^{n}\mathbf{w}_{i}(\mathbf{a}_{i}-\mathbf{b}_{i})^{2}},$
(7)

where $\mathbf{a}$ and $\mathbf{b}$ are two vectors of dimension $n$ and
$\mathbf{w}_{i}$ is the weight factor associated with the $i$th coordinate.

The WED is commonly used in fields such as data mining and image processing,
where the data may have a non-uniform distribution or varying importance of
the features. The weights can be used to emphasize the importance of certain
features or to reduce the impact of irrelevant or noisy features. Both
Euclidean distance and WED are useful measures of similarity between vectors
and can be applied to a variety of fields, including ML, pattern recognition,
and data analysis.

###  6.3 K-Nearest Neighbors

K-Nearest Neighbors (K-NN) is a widely used ML algorithm in data analysis for
classification and regression problems laaksonen1996classification ;
khalid2011mri ; hu2016distance ; mack1981local . The K-NN algorithm is a non-
parametric and lazy (instance-based) learning method that does not make any
assumptions about the underlying data distribution. The basic idea behind K-NN
is to find the K-nearest neighbors to a given data point (which we shall call
a query) based on a distance metric and use the labels of those neighbors to
classify or predict the label of the given data point as shown in Algorithm 1.
In the case of classification, the majority vote of the K-nearest neighbors is
taken as the predicted label of the data point, while in regression, the
average of the K-nearest neighbors’ labels is taken as the predicted value.

To apply the K-NN algorithm, we first need to choose an appropriate distance
metric. In medical data analysis, the Euclidean distance and WED are commonly
used as distance metrics.

Algorithm 1 K-Nearest Neighbors

1:Dataset
$X=\\{(\mathbf{x}_{1},y_{1}),(\mathbf{x}_{2},y_{2}),\ldots,(\mathbf{x}_{n},y_{n})\\}$,
query point $\mathbf{x}$, number of neighbors $K$, distance metric $d(x,y)$

2:Initialize distances array $D=[]$

3:for $i=1$ to $n$ do

4: Compute distance $d_{i}=d(\mathbf{x},\mathbf{x}_{i})$

5: Append distance $D.\text{append}(d_{i})$

6:end for

7:Sort $D$ in ascending order

8:Get the first $K$ distances in $D$: $D_{K}$

9:Get the indices of $D_{K}$: $I_{K}$

10:if classification then

11: $y=\text{mode}({y_{i}\mid i\in I_{K}})$

12:else if regression then

13: $y=\text{mean}\left({y_{i}\mid i\in I_{K}}\right)$

14:end ifreturn Predicted label $y$

However, in some cases, not all features in the data may be equally important.
For instance, in medical data analysis, some features may be more clinically
relevant than others. In such cases, K-NN uses a WED, which assigns different
weights to each feature based on its importance. The weights can be assigned
based on domain knowledge or learned from the data using techniques such as
principal component analysis (PCA). We have shown a visual illustration of the
K-NN algorithm in Fig. 4.

Figure 4: (Left) A 2D feature visualization of a dataset that comprises three
classes ($C_{1},C2,C_{3}$). The colors indicate the class labels. With a new
datapoint $X_{\mathrm{new}}$, the objective of the K-NN algorithm is to find
the class $C_{i}$ to which the datapoint can be classified based on its $K$
neighbors labels. (Right) After computing the distance between the query
$X_{\mathrm{new}}$ and all the samples in the dataset, it has been classified
into class $C_{3}$ which comprises a majority of the new sample neighbors.

In the K-NN algorithm, the choice of $K$ is a hyperparameter that needs to be
tuned since it can lead to overfitting or underfitting depending on the value.
Methods such as cross-validation or grid search can be used to obtain the
optimal value of $K$. The choice of distance metric, such as Euclidean
distance and WED, and the value of $K$ need to be carefully chosen based on
the specific problem and data at hand.

###  6.4 K-Means Clustering

$K$-means clustering is a commonly used technique in various domains,
including medical data analysis sarker2021machine ; bi2020computer ;
chander2023data . The goal of $K$-means clustering is to group similar data
points together based on their features. As shown in Algorithm 2, an initial
guess of $K$ centroids of the clusters is given and then each point is
iteratively assigned to its nearest centroid based on a distance metric such
as Euclidean distance, and the centroid is updated based on the mean of the
assigned data points within a given cluster.

Algorithm 2 K-Means Clustering

1:Number of clusters $K$, dataset
$(\mathbf{x}_{1},\mathbf{x}_{2},\ldots,\mathbf{x}_{n})$

2:Initialize centroids $\bm{\mu}_{1},\bm{\mu}_{2},\ldots,\bm{\mu}_{K}$

3:repeat

4: for $i=1$ to $n$ do

5: Compute distances: $d_{j}=\lVert\mathbf{x}_{i}-\bm{\mu}_{j}\rVert^{2}$

6: Assign sample: $c_{i}=\operatorname*{arg\,min}_{j}d_{j}$

7: end for

8: for $j=1$ to $K$ do

9: Update centroid:
$\bm{\mu}j=\frac{1}{n_{j}}\sum_{i=1}^{n}\mathbf{x}i\cdot\delta{c_{i},j}$

10: end for

11:until convergence

Figure 5: A visual illustration of $K$-Means clustering on a $1D$ dataset
where we have $n=14$ samples and the cluster centriods, $K=3$. In this
formulation, the initial centroids are uniformly randomly chosen as indicated
by the vector means $\large{\bm{\mu}}\sim\mathcal{U}(\mathcal{D},K)$ as
indicated in step 1. The distance $d_{E}(x_{i},\large{\bm{\mu}}_{i})$ is
computed and the sample $x_{i}$ is assigned to the cluster with the minimum
distance as shown in the figure where sample $x_{1}$ is assigned to cluster
$1$. The new cluster centroids as computed from the updated cluster assignment
such that the mean $\large{\bm{\mu}}_{k}$ of cluster $k$ is all samples that
belong to that cluster. This step is repeated until convergence. Finally, all
samples have been properly clustered as indicated by step n in the figure
where the colors indicate the clusters to which each data point belongs.

The algorithm iterates until convergence, typically determined by a stopping
criterion such as a maximum number of iterations or when the change in the
centroids falls below a certain threshold. We have illustrated this approach
to clustering a $1D$ feature space in Fig. 5. K-means clustering is a
computationally efficient algorithm and can handle large datasets. However,
its performance can be sensitive to the initial choice of centroids and the
number of clusters. Additionally, k-means clustering assumes that clusters are
spherical and equally sized, which may not always be true in practice.

###  6.5 Weighted Patch Matching

Weighted Patch Matching (WPM) is an effective technique for measuring
similarity which can be applied to medical image analysis guo2015deformable ;
cordier2015patch . It is based on matching small image patches between a
reference image $I_{R}$ and a target image $I_{T}$ using weighted feature
vectors mechrez2016patch . To do this, we first extract corresponding patches
$P_{i}^{R}$ and $P_{j}^{T}$ from the reference and target images respectively.
Each patch is represented by a feature vector $f_{i}^{R}$ and $f_{j}^{T}$
containing intensity, gradient, or other information. To provide robustness
against intensity variations, a weight vector $w_{i}^{R}$ is computed for each
reference patch based on the gradient magnitude:

$w_{i}^{R}(k)=\frac{\mbox{{}\rm\sf\leavevmode\hbox{}\hbox{}\/}\nabla
I_R(i+k)\mbox{{}\rm\sf\leavevmode\hbox{}\hbox{}}_2}{\max\limits_{k}
\mbox{{}\rm\sf\leavevmode\hbox{}\hbox{}}\nabla
I_{R}(i+k)\mbox{{}\rm\sf\leavevmode\hbox{}}{}_{2}}, $$ where $k$ indexes the
pixels in the patch $P_{i}^{R}$. The weighted feature vector is:
$$\hat{f}_{i}^{R}=w_{i}^{R}\odot f_{i}^{R}.$$ The distance between two patches
is then computed as:
$$d(P_{i}^{R},P_{j}^{T})=\mbox{{}\rm\sf\leavevmode\hbox{}\hbox{}\/}\hat{f}_{i}^{R}
- f_{j}^{T}\mbox{{}\rm\sf\leavevmode\hbox{}}{}_{2}. $$ By minimizing $d$, we
find corresponding patches that match well in terms of weighted features, even
if the raw intensity values differ. We have shown the conceptual steps for WPM
in Algorithm~{}\ref{alg:WPM}. \par WPM is well-suited for medical image
analysis tasks like registration and segmentation. It can match patches
despite differences in scanner modalities or image intensities. The gradient-
based weights emphasize edges and textures while suppressing flat regions.
This focuses the matching on perceptually relevant structures. For example,
WPM has been applied to align magnetic resonance (MR) and transrectal
ultrasound (TRUS) images for targeted prostate biopsy. It outperformed
standard patch matching due to its ability to handle intensity variations
between modalities~{}\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wu2012brain}{\@@citephrase{(}}{\@@citephrase{)}}}. WPM has
also been used to improve atlas-based segmentation of abdomen CT scans,
providing accuracy comparable to interactive
segmentation~{}\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{tong2015discriminative}{\@@citephrase{(}}{\@@citephrase{)}}}.
\par\begin{algorithm}\@@toccaption{{\lx@tag[ ]{{3}}{Weighted Patch Matching
(WPM)}}}\@@caption{{\lx@tag[ ]{{Algorithm 3}} {Weighted Patch Matching
(WPM)}}}
\par\lx@algorithmicx@beginlist@{}\par\lx@algorithmicx@@item\noindent\hskip
0.0ptExtract patches $P_{i}^{R}$ from reference image $I_{R}$ and $P_{j}^{T}$
from target image $I_{T}$ \par\lx@algorithmicx@@item\noindent\hskip
0.0pt{for}\ $i=1$ to $N_{R}$\ {do} \par\lx@algorithmicx@@item\noindent\hskip
15.00002ptCompute feature vector $f_{i}^{R}$ for patch $P_{i}^{R}$
\par\lx@algorithmicx@@item\noindent\hskip 15.00002ptCompute weight vector
$w_{i}^{R}$ based on $\nabla P_{i}^{R}$
\par\lx@algorithmicx@@item\noindent\hskip 15.00002ptCompute weighted feature
$\hat{f}_{i}^{R}=w_{i}^{R}\odot f_{i}^{R}$
\par\lx@algorithmicx@@item\noindent\hskip 15.00002pt{for}\ $j=1$ to $N_{T}$\
{do} \par\lx@algorithmicx@@item\noindent\hskip 30.00003ptCompute distance
$d(P_{i}^{R},P_{j}^{T})$ between $\hat{f}_{i}^{R}$ and $f_{j}^{T}$
\par\lx@algorithmicx@@item\noindent\hskip 15.00002pt{end}\
{for}\par\lx@algorithmicx@@item\noindent\hskip 15.00002ptFind
$k=\operatorname*{arg\,min}_{j}d(P_{i}^{R},P_{j}^{T})$
\par\lx@algorithmicx@@item\noindent\hskip 15.00002ptMatch $P_{i}^{R}$ to
$P_{k}^{T}$ \par\lx@algorithmicx@@item\noindent\hskip 0.0pt{end}\
{for}\par\lx@algorithmicx@endlist \end{algorithm} \par Overall, WPM is a
versatile technique for medical image-matching problems where robustness to
intensity variations is critical. Its continued development, including
learning-based weight optimization and integration with deep networks, will
likely extend its applicability even further.
\par\par\@@numbered@section{subsection}{toc}{Block Matching} Block Matching is
a widely used technique for estimating correspondences between images by
matching small local blocks or patches. In medical imaging, it can be highly
effective for tasks like inter-frame motion tracking and deformable image
registration. One particularly useful variant utilizes the $l_{1}$ norm as the
block similarity metric to provide robustness against
outliers~{}\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{sotiras2011deformable}{\@@citephrase{(}}{\@@citephrase{)}}}.
\par The key idea in Block Matching is to divide the reference image $I_{R}$
and target image $I_{T}$ into smaller blocks $B_{i}^{R}$ and $B_{j}^{T}$ of
pixels respectively, as shown in Algorithm~{}\ref{alg:bm} lines 1-2. The goal
is then to find correspondences between individual blocks from $I_{R}$ and
semantically similar blocks in $I_{T}$. To enable numeric comparison, each
image block is represented by a feature vector $f_{i}^{R}$ and $f_{j}^{T}$
capturing intensities, gradients, textures, or other descriptive features (see
lines 4-9, Algorithm \ref{alg:bm}). \par A distance metric between feature
vectors is then used to quantify block similarity. Using the $l_{1}$ norm,
$\lVert\cdot\lVert_{1}$ defined as: \@@amsalign d(B_{i}^{R},B_{j}^{T})&=\lVert
f_{i}^{R}-f_{j}^{T}\rVert_{1}\\\ &=\sum_{k=1}^{n}\mid
f_{i}^{R}(k)-f_{j}^{T}(k)\mid, provides a robust block comparison that is less
influenced by outlier feature differences compared to the $l_{2}$
norm~{}\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{sotiras2011deformable}{\@@citephrase{(}}{\@@citephrase{)}}}.
Here, $n$ denotes the dimensionality of the block feature vectors. By
minimizing the $l_{1}$ distance (lines 12-19), the best matching target block
$B_{j}^{T}$ can be found for each reference block
$B_{i}^{R}$~{}\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{ledesma2005motion}{\@@citephrase{(}}{\@@citephrase{)}}}.
\par Applying Block Matching with $l_{1}$ similarity to medical images is
advantageous as intensity values can vary considerably across modalities and
between scans due to noise, artifacts, and pathologies. The $l_{1}$ metric
provides a degree of invariance to such complex conditions. \par In summary,
Block Matching using $l_{1}$ block feature distance is a versatile and robust
technique for medical motion estimation and correspondence
problems~{}\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{sotiras2011deformable,ledesma2005motion}{\@@citephrase{(}}{\@@citephrase{)}}}.
Key advantages including simplicity, flexibility across modalities, and strong
performance under challenging imaging conditions make it well-suited for a
variety of clinical applications. \par\begin{algorithm}\@@toccaption{{\lx@tag[
]{{4}}{Block Matching with $l_{1}$ norm}}}\@@caption{{\lx@tag[ ]{{Algorithm
4}} {Block Matching with $l_{1}$ norm}}} \lx@algorithmicx@beginlist@{}
\par\lx@algorithmicx@@item\noindent\hskip 0.0ptDivide reference image $I_{R}$
into blocks $B_{i}^{R}$ \lx@algorithmicx@@item\noindent\hskip 0.0ptDivide
target image $I_{T}$ into blocks $B_{j}^{T}$
\par\lx@algorithmicx@@item\noindent\hskip 0.0pt{for}\ $i=1$ to $N_{R}$\ {do}
\lx@algorithmicx@@item\noindent\hskip 15.00002ptCompute feature vector
$f_{i}^{R}$ for block $B_{i}^{R}$ \lx@algorithmicx@@item\noindent\hskip
0.0pt{end}\ {for}\par\lx@algorithmicx@@item\noindent\hskip 0.0pt{for}\ $j=1$
to $N_{T}$\ {do} \lx@algorithmicx@@item\noindent\hskip 15.00002ptCompute
feature vector $f_{j}^{T}$ for block $B_{j}^{T}$
\lx@algorithmicx@@item\noindent\hskip 0.0pt{end}\
{for}\par\lx@algorithmicx@@item\noindent\hskip 0.0pt{for}\ $i=1$ to $N_{R}$\
{do} \lx@algorithmicx@@item\noindent\hskip 15.00002ptInitialize best match
distance $d_{\min}=\infty$ \lx@algorithmicx@@item\noindent\hskip
15.00002pt{for}\ $j=1$ to $N_{T}$\ {do}
\par\lx@algorithmicx@@item\noindent\hskip 30.00003ptCompute $l_{1}$ distance
$d(B_{i}^{R},B_{j}^{T})=\mbox{{}\rm\sf\leavevmode\hbox{}\hbox{}\/}f_{i}^{R} -
f_{j}^{T}\mbox{{}\rm\sf\leavevmode\hbox{}\hbox{}}1$\lx@algorithmicx@@item\noindent\hskip
30.00003pt\textbf{if}\ $d(B_{i}^{R}, B_{j}^{T}) < d_{\min}$\
\textbf{then}\lx@algorithmicx@@item\noindent\hskip 30.00003pt$d_{\min} =
d(B_{i}^{R}, B_{j}^{T})$\lx@algorithmicx@@item\noindent\hskip
30.00003pt$match(B_{i}^{R}) = B_{j}^{T}$\lx@algorithmicx@@item\noindent\hskip
30.00003pt\lx@algorithmicx@@item\noindent\hskip 15.00002pt\textbf{end}\
\textbf{for}\lx@algorithmicx@@item\noindent\hskip 0.0pt\textbf{end}\
\textbf{for}\par\lx@algorithmicx@endlist\end{algorithm}
\par\@@numbered@section{section}{toc}{low-rank method}{} This section delves
into the mathematics and applications of the low-rank techniques within the
studies reviewed in Sections~{}\ref{sec:LRMA} and \ref{sect:LLRMA}. The
reviewed papers used several types of low-rank techniques such as SVD,
weighted SVD, randomized SVD, tensor SVD, and PCA across various medical image
data modalities to facilitate tasks such as feature extraction, compression,
denoising, decomposition, and more. The low-rank method provides a solid
framework for analyzing and processing high-dimensional data by extracting the
most relevant information while reducing computational complexity and/or
noise. \par\par\@@numbered@section{subsection}{toc}{Traditional SVD}{} When an
image can be approximated sufficiently well by a low-rank matrix, SVD can be
used to find this approximation. This LRMA can be represented more compactly
than the original image. SVD decomposes a complex data matrix
$\mathbf{A}\in\mathbb{C}^{m\times n}$ into three independent matrices
\cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{wall2003singular,
stewart1993early, klema1980singular}{\@@citephrase{(}}{\@@citephrase{)}}}:
\@@amsalign{}\mathbf{A}&=\mathbf{U}\mathbf{\Sigma}\mathbf{V}^{\dagger}\\\
&=\left[\hat{\mathbf{U}}\,\,\,\hat{\mathbf{U}}^{\delta}\right]\begin{bmatrix}\hat{\mathbf{\Sigma}}&\mathbf{0}\\\
\mathbf{0}&\mathbf{0}\end{bmatrix}\mathbf{V}^{\dagger}, where
$\mathbf{U}\in\mathbb{C}^{m\times m}$ and $\mathbf{V}\in\mathbb{C}^{n\times
n}$ are unitary matrices,
$\mathbf{\Sigma}=\text{diag}(\xi_{1},\xi_{2},\cdots,\xi_{m})\in\mathbb{C}^{m\times
n}$ is a diagonal matrix of rank $m$ and ${}^{\dagger}$ denotes a conjugate
transpose. The singular values of the matrix $\mathbf{A}$, which are denoted
as $\xi_{i}$ and lie on the diagonal entries of the matrix $\mathbf{\Sigma}$,
are uniquely determined by $\mathbf{A}$. These singular values are arranged in
a descending order, that is
$\xi_{1}\geq\xi_{2}\geq\xi_{3}\cdots\geq\xi_{m}\geq 0$. It is easy to show
that $\lambda_{i}=\sqrt{\xi_{i}}$ where $\lambda_{i}$ denotes the eigenvalues
of $\mathbf{AA}^{\dagger}$ or $\mathbf{A}^{\dagger}\mathbf{A}$. The columns of
$\mathbf{U}$ and the columns of $\mathbf{V}$ are the left and right singular
vectors of $\mathbf{A}$, respectively. The vectors
$\mathbf{u}_{i}\in\mathbb{C}^{m}$ and $\mathbf{v}_{i}\in\mathbb{C}^{n}$ are
the orthonormal eigenvectors of $\mathbf{AA}^{\dagger}$ and
$\mathbf{A}^{\dagger}\mathbf{A}$ respectively. Therefore, Eq.~{}\ref{SVD1} can
be written as follows
\@@amsgather\mathbf{A}=\sum_{i=1}^{m}\xi_{i}\mathbf{u}_{i}\mathbf{v}_{i}^{\dagger}.
If there exists a matrix of rank $r$ such that $0\leq r\leq m$, then the
optimal approximation of $\mathbf{A}$ that preserves the most important
features is given by
\@@amsgather\mathbf{A}_{r}=\sum_{i=1}^{r}\xi_{i}\mathbf{u}_{i}\mathbf{v}_{i}^{\dagger}.
When $r=m$ then $\mathbf{A}=\mathbf{A}_{r}$. Each value of $\xi_{i}$
quantifies how much the corresponding component
$\xi_{i}\mathbf{u}_{i}\mathbf{v}_{i}^{\dagger}$ contributes in the
reconstruction of $\mathbf{A}$, that is, the larger the value of $\xi_{i}$ the
more $\xi_{i}\mathbf{u}_{i}\mathbf{v}_{i}^{\dagger}$ contributes to the
reconstruction of $\mathbf{A}$. The SVD technique constructs aLRMA matrix
$\mathbf{A}_{r}$ by solving the following optimization problem:
\@@amsgather\underset{\mathbf{A}_{r},\,\,\text{s.t.}\,\,\text{rank}(\mathbf{A}_{r})=r}{\mathrm{argmin}}\,\lVert\mathbf{A}-\mathbf{A}_{r}\rVert_{F}=\mathbf{U}_{r}\mathbf{\Sigma}_{r}\mathbf{V}_{r}^{\dagger},
where $\mathbf{U}_{r},\mathbf{V}_{r}$ denotes the first $r$ leading columns of
$\mathbf{U}$ and $\mathbf{V}$, $\mathbf{\Sigma}_{r}$ denotes a sub-matrix of
$\mathbf{\Sigma}$ and $\lVert\cdot\rVert_{F}$ denotes the Frobenius norm.
Fig.~{}\ref{fg:SVD} demonstrates the steps in full and truncated SVD. \par The
main strength of SVD is its ability to reduce the dimensionality of data,
which is particularly useful for tasks involving large datasets. By
approximating complex matrices with simpler matrices, SVD enables efficient
feature extractions, compression and noise reduction, improving signal quality
and/or image processing techniques. However, SVD has its drawbacks. The
computational cost of full SVD is relatively high in terms of time and memory.
Its time complexity is typically $\mathcal{O}(mn\min\\{m,n\\})$
\cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{holmes2007fast,
mamat2007statistical, brand2006fast, roughgarden2015cs168, atemkeng2023lossy,
halko2011finding}{\@@citephrase{(}}{\@@citephrase{)}}}. In terms of memory, it
requires $\mathcal{O}(mn)$ to store the three component matrices
\cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{roughgarden2015cs168,
burca2014fast, atemkeng2023lossy,
halko2011finding}{\@@citephrase{(}}{\@@citephrase{)}}}. The full SVD provides
a complete representation of the original matrix and preserves all information
that may be important for certain applications. However, it can be impractical
for large datasets due to the high computational requirements. The truncated
SVD, on the other hand, approximates the original matrix by retaining only $r$
singular values and their corresponding columns from $\mathbf{U}_{r}$ and
$\mathbf{V}^{\dagger}_{r}$. This significantly reduces the computational cost.
The time complexity of truncated SVD is approximate $\mathcal{O}(mnr)$
\cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{brand2006fast,
roughgarden2015cs168, burca2014fast, atemkeng2023lossy,
halko2011finding}{\@@citephrase{(}}{\@@citephrase{)}}}, and its memory
complexity is $\mathcal{O}(r(m+n))$ \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{aishwarya2016lossy, roughgarden2015cs168, burca2014fast,
atemkeng2023lossy}{\@@citephrase{(}}{\@@citephrase{)}}}. The truncated SVD is
well suited for dimensionality reduction and noise reduction tasks, where the
lower-dimensional approximation captures the most important features of the
data. However, some information is discarded in the process, which can lead to
a loss of accuracy in certain applications. In addition, the assumption of
orthogonal data in SVD may not be true for all datasets, limiting its
effectiveness in capturing the underlying structures.
\begin{figure}\centering\includegraphics[scale={0.45}]{figures/SVD.jpg}
\@@toccaption{{\lx@tag[ ]{{6}}{A demonstration of full and truncated SVD. The
columns of $\hat{\mathbf{U}}^{\delta}$ span a vector space that is
complementary and orthogonal to that spanned by $\hat{\mathbf{U}}$ and
$\hat{\mathbf{\Sigma}}$ is an $m\times m$ diagonal
matrix.}}}\@@caption{{\lx@tag[: ]{{\small Figure 6}}{\small A demonstration of
full and truncated SVD. The columns of $\hat{\mathbf{U}}^{\delta}$ span a
vector space that is complementary and orthogonal to that spanned by
$\hat{\mathbf{U}}$ and $\hat{\mathbf{\Sigma}}$ is an $m\times m$ diagonal
matrix.}}} \@add@centering\end{figure}
\par\@@numbered@section{subsection}{toc}{Weighted SVD} Weighted SVD is a
method for matrix factorization that extends the standard SVD method
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{van1976generalizing}{\@@citephrase{(}}{\@@citephrase{)}}}.
The weighted matrix, $\tilde{\mathbf{A}}$ with respect to the diagonal weight
matrix $\mathbf{W}$ is defined as
\@@amsalign\tilde{\mathbf{A}}=\mathbf{W}\odot\mathbf{A}, where $\odot$ denotes
the Schur product \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{paulsen1989schur}{\@@citephrase{(}}{\@@citephrase{)}}}.
The weighted SVD of $\tilde{\mathbf{A}}$ is expressed as
\@@amsalign\tilde{\mathbf{A}}=\mathbf{U}\mathbf{\Sigma}^{w}\mathbf{V}^{\dagger},
where $\mathbf{\Sigma}^{w}$ denotes the diagonal matrix of weighted singular
values, which is obtained by scaling the diagonal matrix $\mathbf{\Sigma}$
with the weights
\@@amsalign\mathbf{\Sigma}^{w}_{jj}=\mathbf{W}_{jj}\mathbf{\Sigma}_{jj}. The
decomposition solves the optimization problem:
\@@amsgather\underset{\tilde{\mathbf{A}},\,\,\text{s.t.}\,\,\text{rank}(\tilde{\mathbf{A}})=r}{\mathrm{argmin}}\,\lVert\mathbf{A}-\tilde{\mathbf{A}}\rVert_{F}=\mathbf{U}\mathbf{\Sigma}^{w}\mathbf{V}^{\dagger}
Fig.~{}\ref{fg:wSVD} illustrates how the weighted SVD works. The choice of
weights relies on the specific application. In image compression, for example,
weights are assigned based on the importance of each pixel in the image. In
other words, pixels with higher contrast may be given more weight than pixels
with lower contrast. \par Weighted SVD accurately captures the underlying
structure of the data by weighting singular values according to their
importance. This can be particularly beneficial in the case of noisy or
incomplete data, as the algorithm can give more weight to reliable
information. In addition, weighted SVD allows for better interpretability and
control over the decomposition process, as particular singular values can be
assigned a higher or lower priority based on domain knowledge. However, the
computational cost of weighted SVD is generally higher than that of the
traditional SVD due to the added complexity of handling weights. The weighted
SVD introduces additional operations to account for the weights, which can
lead to a higher time complexity depending on the specific implementation
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{chen2017weighted}{\@@citephrase{(}}{\@@citephrase{)}}}.
The memory requirement is the same as for truncated or full SVD, since
$\mathbf{W}_{jj}\mathbf{\Sigma}_{jj}$ can be stored directly.
\begin{figure}\centering\includegraphics[scale={0.45}]{figures/WeightedSVD.jpg}
\@@toccaption{{\lx@tag[ ]{{7}}{A demonstration of weighted
SVD.}}}\@@caption{{\lx@tag[: ]{{\small Figure 7}}{\small A demonstration of
weighted SVD.}}} \@add@centering\end{figure}
\par\@@numbered@section{subsection}{toc}{Randomized SVD} Randomized Singular
Value Decomposition (rSVD) refers to a technique that approximates the SVD of
a matrix using random projections \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{halko2011finding}{\@@citephrase{(}}{\@@citephrase{)}}}.
The rSVD method has gained increasing popularity in recent years owing to its
efficiency and scalability in large-scale data analysis
\cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{mahoney2011randomized,
halko2011finding, gower2015randomized}{\@@citephrase{(}}{\@@citephrase{)}}}.
In a general LRMA problem, the objective is to approximate the provided matrix
$\mathbf{A}$ using two smaller matrices, $\mathbf{B}\in\mathbb{C}^{m\times r}$
and $\mathbf{C}\in\mathbb{C}^{r\times n}$ with the conditions $r\ll n$ and
$\mathbf{A}\approx\mathbf{B}\mathbf{C}$. To achieve this approximation through
rSVD, a two-stage computation was proposed by \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{halko2011finding}{\@@citephrase{(}}{\@@citephrase{)}}}. In
the initial stage, a random projection matrix
$\mathbf{P}\in\mathbb{C}^{n\times r}$ (with entries typically drawn from a
standard normal or uniform distribution) is created to sample the column space
of $\mathbf{A}\in\mathbb{C}^{m\times n}$, represented as:
\@@amsalign\mathbf{Z}=\mathbf{A}\mathbf{P}, where
$\mathbf{Z}\in\mathbb{C}^{m\times r}$ may be smaller than $\mathbf{A}$,
especially for low-rank matrices with $r\ll n$. As a random projection matrix
$\mathbf{P}$ is unlikely to eliminate crucial components of $\mathbf{A}$,
$\mathbf{Z}$ serves as an accurate approximation of the column space of
$\mathbf{A}$ \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{brunton2022data}{\@@citephrase{(}}{\@@citephrase{)}}}.
Consequently, the low-rank QR decomposition of $\mathbf{Z}$ can be computed to
obtain an orthonormal basis for $\mathbf{A}$:
\@@amsalign\mathbf{Z}=\mathbf{Q}\mathbf{R}. In the second step, using the low-
rank basis $\mathbf{Q}$, $\mathbf{X}$ is projected into a smaller space,
expressed as: \@@amsalign\mathbf{Y}=\mathbf{Q}^{\dagger}\mathbf{A}. The SVD of
$\mathbf{Y}$ is given by:
\@@amsalign\mathbf{Y}=\hat{\mathbf{U}}\mathbf{\Sigma}\mathbf{V}^{\dagger}. As
$\mathbf{Q}$ is orthonormal and approximates the column space of $\mathbf{A}$,
the matrices $\mathbf{\Sigma}$ and $\mathbf{V}$ are the same for both
$\mathbf{Y}$ and $\mathbf{A}$. The high-dimensional left singular vectors
$\mathbf{U}$ are reconstructed using $\hat{\mathbf{U}}$ and $\mathbf{Q}$:
\@@amsalign\mathbf{U}=\mathbf{Q}\hat{\mathbf{U}}.
\begin{figure}\centering\includegraphics[scale={0.42}]{figures/rSVD.jpg}
\@@toccaption{{\lx@tag[ ]{{8}}{A demonstration of random
SVD.}}}\@@caption{{\lx@tag[: ]{{\small Figure 8}}{\small A demonstration of
random SVD.}}} \@add@centering\end{figure} The rSVD algorithm is based on the
idea that the matrix $\mathbf{Q}$ acts as a randomized projection matrix,
compressing the information in $\mathbf{A}$ to a smaller dimension, which is
then used to compute the SVD of $\mathbf{A}$. The accuracy of the
approximation depends on the number of columns, denoted as $r$, in the matrix
$\mathbf{Q}$. A larger value of $r$ leads to increased accuracy but also comes
with higher computational cost. Fig.~{}\ref{fig:rSVD} demonstrates the steps
in rSVD. \par rSVD offers a computational advantage over traditional SVD by
significantly reducing the time complexity associated with matrix
factorization, especially for large datasets. The computational cost of rSVD
is approximately given by $\mathcal{O}(mn\log(r))$
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{halko2011finding}{\@@citephrase{(}}{\@@citephrase{)}}}.
The logarithmic term ($\log(r)$) represents the number of iterations required
for the power iteration method to converge to the desired number of singular
values or components. In addition, rSVD has lower memory requirements, with a
memory complexity of about $\mathcal{O}(mn+nr)$ \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{singh2020range,
halko2011finding}{\@@citephrase{(}}{\@@citephrase{)}}}. These attributes make
rSVD highly scalable and efficient for applications with large amounts of data
\cite[cite]{\@@bibref{Authors Phrase1YearPhrase2}{mahoney2011randomized,
halko2011finding, gower2015randomized}{\@@citephrase{(}}{\@@citephrase{)}}}.
Moreover, due to the introduction of randomness, rSVD can provide higher
robustness to outliers and noise, leading to more stable results in some
instances. However, rSVD comes with certain limitations. It provides an
approximate decomposition that may not achieve the same level of accuracy as
the exact decomposition provided by traditional SVD. The need for parameter
tuning, including selecting appropriate values for parameters such as the
number of samples and the desired rank, may affect the approximation quality
and must be carefully considered. It should also be noted that rSVD is best
suited for tasks where an approximate decomposition is acceptable, such as
dimensionality reduction. It may not be suitable for applications that require
exact results. \par\par\@@numbered@section{subsection}{toc}{Tensor SVD} The
tensor SVD works by unfolding the tensor along one of its modes (i.e.,
dimensions), resulting in a tensor matrix representation
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{kilmer2021tensor}{\@@citephrase{(}}{\@@citephrase{)}}}.
The matrix SVD is then applied to this matrix representation to produce a set
of orthogonal matrices and singular values. Let
$\mathcal{A}\in\mathbb{C}^{I_{1}\times I_{2}\times\cdots\times I_{n}}$ denotes
a tensor of order $n$. The decomposition of a tensor, $\mathcal{A}$, is given
by
\@@amsalign{}\mathcal{A}=\mathcal{B}\times_{1}\mathbf{U}^{(1)}\times_{2}\mathbf{U}^{(2)}\times_{3}\cdots\times_{n}\mathbf{U}^{(n)},
where $\mathcal{B}$ is a core tensor of size $(r_{1},r_{2},\cdots r_{n})$ with
rank $r$, and $\mathbf{U}^{(1)},\mathbf{U}^{(2)},\cdots,\mathbf{U}^{(n)}$ are
orthogonal matrices with size $I_{1}\times r_{1},I_{2}\times
r_{2},\cdots,I_{n}\times r_{n}$ respectively. The weights associated with each
tensor component are stored in the core tensor $\mathcal{B}$, and the
orthogonal matrices
$\mathbf{U}^{(1)},\mathbf{U}^{(2)},\cdots,\mathbf{U}^{(n)}$ represent the
contribution of each mode to each component. The number of components is
determined by $r$, which is typically much smaller than the tensor size. The
components are recorded in descending order, with the first capturing the most
variation in the tensor. Eq.~{}\ref{tensor} can be represented using a matrix
form as
\@@amsalign\mathcal{A}=\sum_{i_{1}=1}^{r_{1}}\sum_{i_{2}=1}^{r_{2}}\cdots\sum_{i_{n}=1}^{r_{n}}b_{i_{1},i_{2},\cdots,i_{n}}\mathbf{u}_{i_{1}}^{(1)}\otimes\mathbf{u}_{i_{2}}^{(2)}\cdots\otimes\mathbf{u}_{i_{n}}^{(n)},
where $\mathbf{u}_{i_{j}}^{(n)}$ denotes an $i^{th}$ column of
$\mathbf{U}^{(n)}$ and $\otimes$ denotes the tensor product operation. If
$b_{i_{1},i_{2},\cdots,i_{n}}=0$, that is, $\mathcal{B}$ is diagonal then
\@@amsalign{}\mathcal{A}=\sum_{i_{1}=1}^{r}b_{ii\cdots
i}\mathbf{u}_{i}^{(1)}\otimes\mathbf{u}_{i}^{(2)}\cdots\otimes\mathbf{u}_{i}^{(n)}
except when $i_{1}=i_{2}=\cdots=i_{n}$. The t-SVD is defined as
\@@amsalign\mathcal{A}=\sum_{i_{1}=1}^{r}\xi_{i}\mathbf{u}_{i}^{(1)}\otimes\mathbf{u}_{i}^{(2)}\cdots\otimes\mathbf{u}_{i}^{(n)},
where $\xi_{i}$ is the $i^{th}$ singular value. Eq.~{}\ref{mtSVD} underlines
the similarity between the t-SVD and the matrix SVD, where the singular values
are replaced by tensors, and the factor matrices are replaced by vectors. \par
The t-SVD technique has both advantages and disadvantages in various
applications. One of its notable advantages is its ability to reduce the
dimensionality of multidimensional data, which is useful in compression, noise
reduction, and feature extraction. This technique proves valuable in data
analysis, as it helps uncover intricate patterns and relationships in complex
datasets. However, tensor SVD also has drawbacks. As with any dimensionality
reduction method, t-SVD can result in information loss since only the most
significant singular values and vectors are preserved. While it can reveal
patterns, their interpretation can be difficult, especially for high-
dimensional tensors. In addition, the computational cost can be significant,
especially for high-dimensional tensors, making it less practical for large
datasets. The computational complexity of t-SVD in some typical tensor
decomposing-based methods is about $\mathcal{O}(n^{2}m^{2}k+k^{3})$, where
$n$, $m$, and $k$ denote the tensor dimensions \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wang2023tensor}{\@@citephrase{(}}{\@@citephrase{)}}}.
\par\@@numbered@section{subsection}{toc}{Principal Component Analysis}
Principal component analysis (PCA) is a mathematical technique used in data
analysis and ML for dimensionality reduction and feature extraction. It
identifies the most important features, or principal components, in the data
matrix $\mathbf{A}$ that capture the most significant variance. When reducing
the number of feature variables in a dataset, reduced dimensionality trades
accuracy for simplicity. Small datasets are easier to explore, visualize, and
analyze with ML algorithms since they have few variables. Let $\mathbf{x}_{i}$
denote the $i^{th}$ column vector of the data matrix $\mathbf{X}$ (denotes a
covariance matrix of the centered variable) with $n$ feature variables. A
linear combination of $\mathbf{x}$'s is expressed as
\@@amsalign\mathbf{t}&=\sum_{i=1}^{n}w_{i}\mathbf{x}_{i}\\\
&=\mathbf{X}\mathbf{w}. The objective is to find a set of orthogonal linear
combinations of variables that capture the most variation in the data. That
is,
\@@amsalign\underset{\lVert\mathbf{w}\rVert=1}{\operatorname{argmax}}\,\,\text{var}(\mathbf{\mathbf{X}\mathbf{w}}).
To compute the first principal component, the first weight vector,
$\mathbf{w}_{(1)}$, has to satisfy
\@@amsalign{}\mathbf{w}_{(1)}&=\underset{\lVert\mathbf{w}\rVert=1}{\operatorname{argmax}}\,\,\lVert\mathbf{X}\mathbf{w}\rVert^{2}\\\
&=\underset{\lVert\mathbf{w}\rVert=1}{\operatorname{argmax}}\,\,\left(\mathbf{\mathbf{w}^{\dagger}\mathbf{X}^{\dagger}\mathbf{X}\mathbf{w}}\right).
Since $\mathbf{w}$ is a unit vector, Eq.~{}\ref{maxvar} can be rewritten as
\@@amsalign\mathbf{w}_{(1)}=\underset{}{\operatorname{argmax}}\,\,\left(\frac{\mathbf{\mathbf{w}^{\dagger}\mathbf{X}^{\dagger}\mathbf{X}\mathbf{w}}}{\mathbf{w}^{\dagger}\mathbf{w}}\right).
To compute the $k^{th}$ component, the $k-1$ principal components are
subtracted from $\mathbf{X}$, that is
\@@amsalign\tilde{\mathbf{X}}_{k}=\mathbf{X}-\sum_{j=1}^{k-1}\mathbf{X}\mathbf{w}_{(j)}\mathbf{w}_{(j)}^{\dagger}
and the weight vector that extracts maximum variance from
$\tilde{\mathbf{X}}_{k}$ is given by
\@@amsalign\mathbf{w}_{(k)}&=\underset{\lVert\mathbf{w}\rVert=1}{\operatorname{argmax}}\,\,\lVert\tilde{\mathbf{X}}_{k}\mathbf{w}\rVert^{2}\\\
&=\underset{\lVert\mathbf{w}\rVert=1}{\operatorname{argmax}}\,\,\left(\mathbf{w}^{\dagger}\tilde{\mathbf{X}}^{\dagger}_{k}\tilde{\mathbf{X}}_{k}\mathbf{w}\right).
Let $\mathbf{W}$ denote a matrix containing the $k$ selected eigenvectors as
its columns, then the full principal component decomposition can be computed
as \@@amsalign\mathbf{T}=\mathbf{X}\mathbf{W}, where $\mathbf{T}$ is the
matrix of principal components and each column represents a principal
component. The resulting matrix $\mathbf{T}$ contains the data's reduced-
dimensional representation, capturing the most significant variation. The
covariance matrix's eigenvectors and eigenvalues are critical in determining
the directions and magnitudes of the principal components, respectively.
Fig.~{}\ref{fig:pca} demonstrates the steps in PCA. \par The main advantages
of PCA include its ability to reduce the dimensionality of data while
preserving important information, noise reduction, improve model performance,
and visualize high-dimensional data. It is an unsupervised technique suitable
for exploratory analysis. However, PCA assumes a linear data distribution and
may not perform well with nonlinear relationships. It is also sensitive to
outliers, as these can disproportionately affect the calculated principal
components. In addition, interpretation of the transformed components can be
challenging for complex datasets. The computation time for PCA is mainly
influenced by the number of data points, $m$, and the number of features, $n$,
in the dataset. The most time-consuming step is usually the eigenvalue
decomposition or SVD, which has a computational complexity of
$\mathcal{O}(n^{3})$ for the covariance matrix and
$\mathcal{O}(\min\\{m^{2}n,mn^{2}\\})$ for SVD \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{van2009dimensionality,li2019tutorial}{\@@citephrase{(}}{\@@citephrase{)}}}.
The memory requirement depends on the size of the covariance matrix or the SVD
factor matrices. It is typically $\mathcal{O}(n^{2})$ for the covariance
matrix and $\mathcal{O}(\min\\{m^{2},mn\\})$ for SVD
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{van2009dimensionality,li2019tutorial}{\@@citephrase{(}}{\@@citephrase{)}}}.
\begin{figure}\centering\includegraphics[scale={0.45}]{figures/pca.jpg}
\@@toccaption{{\lx@tag[ ]{{9}}{A demonstration of PCA In this example, $k$
denotes the number of principal components to retain.}}}\@@caption{{\lx@tag[:
]{{\small Figure 9}}{\small A demonstration of PCA In this example, $k$
denotes the number of principal components to retain.}}}
\@add@centering\end{figure} \par Table~{}\ref{tab:lowrank} provides a summary
of various low-rank methods used in data analysis. It highlights their key
strengths, weaknesses, and computational costs. The table is a valuable
reference for selecting the most suitable low-rank method based on specific
data analysis requirements. \begin{landscape} \pagestyle{empty}
\begin{table}\begin{center} \@@toccaption{{\lx@tag[ ]{{15}}{This table
provides a detailed analysis of various low-rank methods, highlighting their
respective strengths, weaknesses, computational cost, and compatibility with
different data size constraints.}}}\@@caption{{\lx@tag[: ]{{\small Table
15}}{\small This table provides a detailed analysis of various low-rank
methods, highlighting their respective strengths, weaknesses, computational
cost, and compatibility with different data size
constraints.}}}\begin{tabular}[]{p{2cm} p{5cm} p{5cm} p{4.5cm}
p{2cm}}\hline\cr{low-rank method}&{Strengths}&{Weakness}&{Computational
cost}\\\ \hline\cr SVD&$\bullet$ Provides an exact decomposition of the data
matrix into orthogonal factors.\newline $\bullet$ Useful for dimensionality
reduction, noise reduction, and feature extraction.&$\bullet$ Can be
computationally expensive for large matrices.\newline $\bullet$ Sensitive to
missing data and outliers.\newline $\bullet$ May not handle high-dimensional
data well.&$\bullet$ Full SVD has time complexity of
$\mathcal{O}(mn\min\\{m,n\\})$ and memory complexity of $\mathcal{O}(mn)$.
While truncated SVD has time complexity $\mathcal{O}(mnr)$, and its memory
complexity is $\mathcal{O}(mr+nr)$.\\\ \hline\cr weighted SVD&$\bullet$
Accurately captures the underlying structure of the data by weighting singular
values according to their importance.\newline $\bullet$ Provides better
interpretability and control over the decomposition process.&$\bullet$
Computational cost is generally higher than that of the traditional SVD.&When
$\mathbf{W}_{jj}\mathbf{\Sigma}_{jj}$ is stored directly, memory requirement
is approximately $\mathcal{O}(mnr)$ or $\mathcal{O}(mn\min\\{m,n\\})$\\\
\hline\cr rSVD&$\bullet$ Faster than computing the full SVD for large
matrices.\newline $\bullet$ Requires less memory than traditional SVD.\newline
$\bullet$ Extract the most important features of a matrix while discarding
noise and less significant components.&$\bullet$ SVD might not have clear
interpretations as they would with the traditional SVD.\newline $\bullet$
Approximation might not be suitable for applications where high precision is
crucial.&$\bullet$ Has time complexity of approximately
$\mathcal{O}(mn\log(r))$ and a memory complexity of about
$\mathcal{O}(mn+nr)$.\\\ \hline\cr t-SVD&$\bullet$ Suitable for multi-
dimensional data, capturing interactions among multiple modes.\newline
$\bullet$ Reduces the dimensionality of multidimensional data, which is useful
in compression, noise reduction, and feature extraction&$\bullet$
Computational cost can be significant, especially for high-dimensional
tensors\newline $\bullet$ Can result in information loss since only the most
significant singular values and vectors are preserved.&$\bullet$ Complexity
for full tensor SVD can be approximated as
$\mathcal{O}(n^{2}m^{2}k+k^{3})$.\\\ \hline\cr PCA&$\bullet$ Has the ability
to reduce the dimensionality of data while preserving important information,
noise reduction, improve model performance, and visualize high-dimensional
data.&$\bullet$ Sensitive to scaling of features.\newline $\bullet$ It is not
well suited for capturing nonlinear relationships in data.&$\bullet$ Has a
computational complexity of $\mathcal{O}(n^{3})$ for the covariance matrix and
$\mathcal{O}(\min\\{m^{2}n,mn^{2}\\})$ for SVD. Memory complexity is typically
$\mathcal{O}(n^{2})$ for the covariance matrix and
$\mathcal{O}(\min\\{m^{2},mn\\})$ for SVD.\\\ \end{tabular} \end{center}
\end{table} \end{landscape} \par\par\@@numbered@section{section}{toc}{Findings
and Discussion}{} This section addresses the findings and drawbacks that arise
from the review. We also provide valuable insights and recommendations for
possible future directions in this area of study. Fig.~{}\ref{fg:SWork}
summarizes the findings of this study. The figure shows the low-rank method
used in the reviewed publications used while performing various tasks on the
medical data. It also highlights the different types that were used. In terms
of the LLRMA approximation, the figure shows the similarity measurement
methods used in the studied publications during the matching phase.
\par\par\@@numbered@section{subsection}{toc}{High Rate of LLRMA Applied from
2015} From the reviewed papers, LLRMA has shown the potential and ability to
capture the underlying structure of data while reducing its dimensionality
with little to no feature loss or distortion due to the locality coherence