# LHU-Net: A Light Hybrid U-Net for Cost-Efficient, High-Performance
Volumetric Medical Image Segmentation

Yousef Sadegheih† Faculty of Informatics and Data Science, University of
Regensburg, Regensburg, 93053, Germany  Afshin Bozorgpour† Faculty of
Informatics and Data Science, University of Regensburg, Regensburg, 93053,
Germany  Pratibha Kumari  Faculty of Informatics and Data Science, University
of Regensburg, Regensburg, 93053, Germany  Reza Azad  Faculty of Electrical
Engineering and Information Technology, RWTH Aachen University, 52062 Aachen,
Germany  Dorit Merhof  Corresponding author: Dorit Merhof, e-mail:
<EMAIL_ADDRESS>${\dagger}$ Indicates equal contribution Faculty of
Informatics and Data Science, University of Regensburg, Regensburg, 93053,
Germany  Fraunhofer Institute for Digital Medicine MEVIS, Bremen 28359,
Germany

###### Abstract

As a result of the rise of Transformer architectures in medical image
analysis, specifically in the domain of medical image segmentation, a
multitude of hybrid models have been created that merge the advantages of
Convolutional Neural Networks (CNNs) and Transformers. These hybrid models
have achieved notable success by significantly improving segmentation
accuracy. Yet, this progress often comes at the cost of increased model
complexity, both in terms of parameters and computational demand. Moreover,
many of these models fail to consider the crucial interplay between spatial
and channel features, which could further refine and improve segmentation
outcomes. To address this, we introduce LHU-Net, a Light Hybrid U-Net
architecture optimized for volumetric medical image segmentation. LHU-Net is
meticulously designed to prioritize spatial feature analysis in its initial
layers before shifting focus to channel-based features in its deeper layers,
ensuring a comprehensive feature extraction process. Rigorous evaluation
across five benchmark datasets — Synapse, LA, Pancreas, ACDC, and BRaTS 2018 —
underscores LHU-Net’s superior performance, showcasing its dual capacity for
efficiency and accuracy. Notably, LHU-Net sets new performance benchmarks,
such as attaining a Dice score of 92.66 on the ACDC dataset, while
simultaneously reducing parameters by $85\%$ and quartering the computational
load compared to existing state-of-the-art models. Achieved without any
reliance on pre-training, additional data, or model ensemble, LHU-Net’s
effectiveness is further evidenced by its state-of-the-art performance across
all evaluated datasets, utilizing fewer than 11 million parameters. This
achievement highlights that balancing computational efficiency with high
accuracy in medical image segmentation is feasible. Our implementation of LHU-
Net is freely accessible to the research community on GitHub.

_Keywords_ Volumetric Medical Image Segmentation, Deep Learning, Computational
Efficiency, High-Performance Segmentation, Light Hybrid Architecture

##  1 Introduction

Medical imaging technologies such as MRI, CT, or planar X-ray imaging, have
substantially transformed diagnostics by providing non-invasive visualizations
of internal anatomical structures. Segmentation of medical imaging data is
commonly performed to support diagnostics, intervention planning, or disease
progression assessment. However, manual segmentation by radiologists is not
only time-consuming, but is also highly subjective and prone to
inconsistencies. Accordingly, there is a growing need for automated
segmentation approaches.

Recently, deep learning approaches (and Convolutional Neural Networks (CNNs)
in particular) have shown great potential for medical image segmentation.
Prominent CNN-based segmentation networks include Fully Convolutional Networks
(FCNs) Long et al. (2015), U-Net Ronneberger et al. (2015), DeepLab Liang-
Chieh et al. (2015), ResNet He et al. (2016), and V-Net Milletari et al.
(2016). Each of these network designs has made significant progress by
consistently outperforming earlier state-of-the-art networks Azad et al.
(2022); Bozorgpour et al. (2023).

Although CNNs can detect local patterns, they are fundamentally unable to
comprehend the wider context or global connections present in an image. This
constraint becomes especially apparent while performing intricate image
analysis tasks that require a comprehensive understanding of the entire scene
or image. The Vision Transformer (ViT) Dosovitskiy et al. (2020) design has
been central to recent developments aimed at overcoming this constraint. The
attention mechanism is the spine of ViT’s architecture which allows the
aggregation of data over the whole input sequence. Overcoming the limited
receptive field limitations commonly seen in CNNs, this strategy enables the
network to absorb long-range contextual information Shamshad et al. (2023).
Nevertheless, here a major drawback is set by the underrepresentation of local
information hence compromising the performance Azad et al. (2023a). This is
more noticeable when it comes to detecting local textures with a fine grain,
which is an essential component of many diagnostic and prognostic activities
Azad et al. (2023b). This restriction stems from the fact that ViTs use a
novel approach to image processing. To comprehend the inter-dependencies among
the image’s patches, ViT models use self-attention mechanisms to break the
image into smaller pieces. Although ViTs have made advancements in capturing
overall contextual information, CNNs may have an advantage in extracting
specific features within a local area because of the differences in their
fundamental methods for image processing Geirhos et al. (2018).

Consequently, there has been a paradigm shift towards hybrid models that
amalgamate the strengths of both CNNs and vision transformer models Geirhos et
al. (2018). A notable strategy involves the fusion of CNN and transformer
model features, leveraging the local feature extraction capabilities of CNNs
and the global contextual understanding of transformers. This synergy aims to
overcome the individual limitations of each architecture. For instance,
TransUNet Chen et al. (2021), an innovative architecture, merges transformer
layers within the CNN bottleneck, enabling the simultaneous modeling of both
local and global dependencies. Another exemplary model, HiFormer Heidari et
al. (2023), introduces the integration of a Swin transformer module with a
CNN-based encoder. This architecture generates dual multi-scale feature
representations, which are then coalesced through a Double-Level Fusion
module, enhancing the overall segmentation accuracy. Further, architectures
like UNETR Hatamizadeh et al. (2022) and CoTr Xie et al. (2021) have been
instrumental in advancing 3D medical image segmentation. UNETR utilizes a
transformer-based encoder in tandem with a CNN decoder, optimizing the
segmentation process. CoTr, on the other hand, innovatively bridges a CNN
encoder and decoder with a transformer, specifically targeting an enhanced
performance in low-resolution stages of segmentation. Furthermore, TransBTS
Wang et al. (2021) represents another stride in this direction, employing a
similar hybrid approach to optimize segmentation results. Additionally,
architectures such as Swin-UNet Cao et al. (2022) offer unique advantages. The
Swin transformer’s shifted window-based self-attention mechanism within a
U-Net-like architecture facilitates the efficient handling of large-scale
medical images while capturing intricate details and broad structural
contexts, making it highly effective for complex segmentation tasks.

CNNs are distinguished by their hierarchical architecture, adept at discerning
spatial hierarchies in imagery via convolutional filters, benefiting from
their inherent translation invariance and localized connectivity. This design
tends to yield models characterized by a reduced parameter count due to the
sharing of weights across spatial domains and the confinement of convolution
operations to localized areas. As a result, CNNs generally demand less
computational power, particularly in applications where spatial coherence is
pivotal. Conversely, ViTs employ self-attention mechanisms, which, although
offering comprehensive analysis capabilities across an image, cause a
quadratic surge in computational costs relative to the input size. This
attribute renders ViTs more demanding in terms of computationally and
parameter volume. Hybrid models seek to amalgamate CNNs’ local processing
efficiencies with ViTs’ expansive global receptive fields. Yet, this ambition
often culminates in a marked escalation of parameters and computational
expenditure. Certain hybrid models indiscriminately increase model complexity,
adding layers or expanding attention mechanisms across the network in the hope
of boosting performance. This brute-force scaling-up strategy often results in
diminishing returns on performance gains relative to the higher computational
overhead Zhou et al. (2021); Yang et al. (2024); Chen et al. (2023); Shaker et
al. (2022). The insufficiency of current hybrid models, particularly in the
context of volumetric medical image segmentation, may well stem from their
sub-optimal self-attention configurations. Moreover, these models typically
fail to explicitly leverage the interplay between spatial and channel
features, an oversight that could otherwise improve segmentation accuracy. A
refined strategy adapts attention mechanisms to suit various model stages.
Initially, where the acquisition of spatial information is crucial,
specialized attention blocks tailored for spatial focus can effectively manage
local features. As the model delves deeper, the emphasis shifts towards
assimilating broader contextual data, necessitating attention frameworks that
prioritize channel dynamics. Such sequential customization facilitates a
harmonious balance between model intricacy and efficacy. Aligning
architectural design with the specific demands of information processing at
each stage enables enhanced performance, circumventing the disproportionate
parameter and computational load increase seen in more homogenized hybrid
models. In essence, the evolution of hybrid models offers a promising avenue
to exploit the unique strengths of both CNNs and ViTs. Nevertheless, simply
amplifying model complexity fails to serve as a panacea. Instead, a
meticulously crafted network featuring bespoke attention mechanisms attuned to
the distinct requirements of varying processing phases emerges as a more adept
and resource-efficient methodology for capitalizing on the synergistic
potential of CNNs and ViTs in visual data analysis.

In this vein, we introduce LHU-Net (Light Hybrid U-Net), a model embodying the
ideal equilibrium between complexity and performance. LHU-Net exemplifies that
achieving an optimal balance is not only feasible but pivotal, setting a new
precedent in the design of hybrid models for sophisticated image segmentation
tasks.

Figure 1: Accuracy (Dice score) vs. model complexity (FLOPs and parameters)
comparison on ACDC.

### Contributions

This work introduces several significant advancements in the realm of medical
image segmentation, pushing the boundaries of current methodologies and
setting new benchmarks for efficiency and accuracy. Below, we delineate our
key contributions, which underscore the potential of our proposed LHU-Net in
addressing complex segmentation tasks across diverse medical imaging datasets.

❶ Efficient Hybrid Attention Selection for Better Contextual Understanding:  A
cornerstone of our approach is the strategic deployment of two specialized
attention mechanisms within ViTs, meticulously engineered to calibrate the
model’s focus across varying stages of analysis. Incorporating Large Kernel
Attention with an additional deformable layer (LKAd) in conjunction with
spatial and channel attention, this deliberate synthesis empowers our model to
adeptly navigate long-range dependencies. This approach not only facilitates
the extraction of expansive global contextual insights but also addresses
texture bias, ensuring the preservation of high-frequency details without
sacrificing the integrity of feature representation and inductive biases. In
the initial phases, our architecture employs spatial attention to masterfully
discern local features and subtleties within the early layers of the network.
As the model evolves, it seamlessly transitions to channel attention, a
pivotal shift that underscores the importance of distilling and accentuating
critical features within the deeper layers. This bifurcated attention strategy
enables LHU-Net to harness the strengths of both spatial and channel
attentions effectively, facilitating a nuanced and comprehensive feature
extraction that is attuned to the unique demands of medical image
segmentation.

❷ Setting New Efficiency Benchmarks in Medical Segmentation: LHU-Net
represents a paradigm shift in achieving high-performance segmentation with
significantly reduced computational resources. In one notable dataset
(Automated Cardiac Diagnosis Challenge), our architecture achieved a reduction
in the number of parameters by approximately 85% and a quartering of FLOPS
(see Figure 1), all while maintaining a lean parameter footprint of under 11
million across all five evaluated datasets. This breakthrough demonstrates
that a well-crafted, purpose-specific architecture can achieve exceptional
performance without the traditional trade-offs between computational demand
and accuracy. It proves that balancing model complexity with computational
efficiency, far from being an elusive goal, is a tangible reality with LHU-
Net.

❸ Versatile Superiority Across Diverse Imaging Modalities and Segmentation
Challenges: LHU-Net’s design principles and execution excellence have
propelled it to the forefront of state-of-the-art methodologies across
multiple datasets, showcasing unparalleled versatility. Whether analyzing CT
scans, MR images, or datasets that incorporate up to four distinct modalities
(as in the BraTS challenge), LHU-Net consistently delivers top-tier
performance. Furthermore, its adaptability extends to handling a wide range of
segmentation tasks, from single-label segmentation to complex scenarios
involving up to eight distinct labels (e.g., in the MICCAI Multi-Atlas Abdomen
Labeling Challenge). This broad spectrum applicability underscores LHU-Net’s
robustness and its potential as a universal solution for medical image
segmentation, capable of accommodating the multifaceted nature of clinical
imaging data.

In summary, our contributions through the development of LHU-Net not only
advance the technical capabilities in medical image segmentation but also
offer a scalable, efficient pathway to integrating deep learning solutions
into clinical workflows. By addressing the critical challenges of
computational efficiency and model adaptability, LHU-Net paves the way for
more accessible and effective diagnostic tools, promising significant impacts
on patient care and medical research.

Figure 2: The proposed LHU-Net architecture. On the left, an abstract
representation of the proposed network is depicted. At the top levels, the
‘Init’ and ‘Out’ blocks manage input and output respectively, while ‘CNN Macro
Blocks’ facilitate information flow between these top-level blocks and the
‘Hybrid Macro Blocks’ at the network’s core. Thorough explanations of the
utilized attention blocks and their sub-components, along with other relevant
details, are provided on the right side of the network, offering a
comprehensive understanding of the architecture’s design. Note that ‘PE’ and
‘RB’ stand for Positional Embedding and ResBlock, respectively.

##  2 Methodology

Our approach, named LHU-Net, is designed to efficiently extract enriched
features across various network levels while optimizing parameter efficiency.
By integrating convolutional-based blocks with hybrid attention mechanisms,
LHU-Net effectively captures contextual information, addressing both strong
local features and non-local dependencies. Figure 2 illustrates the
architecture of LHU-Net, showcasing its hierarchical U-Net encoder-decoder
structure. The encoder stages, encode 3D input patches into varying levels of
representations, spanning from local to global, facilitating the transmission
of essential information via skip connections to the decoder stages,
ultimately generating the final segmentation masks. Table 1 shows the
notations used throughout this section.

Table 1: Notations used throughout Section 2. Acronym | Meaning
---|---
BN | Batch normalization layer
ViT | Vision transformer
PW-Conv | Point-wise convolution block
DW-Conv | Depth-wise convolution block
DWD-Conv | Depth-wise dilated convolution
DDW-Conv | Deformable depth-wise convolution
GELU | Gaussian error linear unit
LKAd | Large kernel attention with one additional deformable layer
A | Self-attention matrix
n | Number of tokens ($HWD$)
h | The number of heads in the ViT attention block
x | Input variable
$d$ | Hidden state dimension
Q | The query embedding inputs
K | The key embedding inputs
V | The value embedding inputs
S | The skip connection
F | The feature map
ResBlock | Residual Block
PReLU | Parametric rectified linear unit
PE | Positional embedding
CA | Channel attention module
SA | Self-attention module

Network Architecture: LHU-Net comprises three distinct stages respecting the
spatial dimensions of the feature map. The first stage includes the “Init
Block,” and “Out Block,” at the beginning and the end of the network consists
of a single block maintaining the spatial dimensions of both the input and
output of the network. This stage plays a pivotal role in preserves the
spatial shape of the input tensor throughout the network, ensuring the
accurate transformation of input data for the encoder and producing the
desired output after the decoder. At the intermediate stage, convolutional
blocks (termed “CNN Blocks”) are utilized to reduce spatial dimensions while
enhancing local features, capturing high-frequency representations
effectively. Subsequently, the output is channeled to the final stage (named
“Hybrid Blocks”), where a trio of hybrid attention mechanisms, incorporating
convolutional large kernel attention Guo et al. (2023) with one additional
deformable layer Azad et al. (2024) alongside spatial and channel-wise ViT
attention blocks, further refines features, accounting for global
representations and long-range dependencies. Throughout the encoder stages,
skip connections maintain connectivity with corresponding decoder blocks,
ensuring seamless information flow. Detailed explanations of each stage follow
in subsequent subsections.

###  2.1 Init and Out Blocks

To optimize the performance of subsequent convolutional blocks in the deeper
levels of the network and enhance the overall quality of the final output,
special emphasis is placed on the boundary regions by employing the Init and
Out stages. These stages ensure that all processing is conducted within the
same spatial space as the input image, preserving its spatial dimensions while
boosting the model’s effectiveness. The Init block begins with a point-wise
convolutional operation (PW-Conv) applied to the input data, adjusting the
channel dimensions to match those of the subsequent level.

$F_{\text{I}}=\text{BN}(\text{PReLU}(\text{PW-Conv}(I)))$ (1)

Here, $F_{\text{I}}$ is of the same spatial size as $I$ (the input image). It
represents the feature map produced directly from the input image, serving as
the input for subsequent levels and acting as the skip connection for the last
step of the CNN Decoder Block simultaneously.
Moreover, the model applies a ResBlock directly to the input data,
facilitating the extraction of high-frequency representations that encapsulate
sensitive local features. This approach enables the capture of crucial details
for accurate boundary estimation in the final segmentation output.
Subsequently, these representations are fed into the Out block (Equation 2),
contributing to the establishment of optimal guidance for the final mask
($X_{\text{Out}}$).

$X_{\text{Out}}=\text{PW-
Conv}(\text{ResBlock}(\text{ResBlock}(I)+X_{\text{CD}}))$ (2)

$X_{\text{CD}}$ denotes the output of the CNN Decoder Block.

###  2.2 CNN Blocks

An integral part of our proposed model are pure CNN blocks, renowned for their
adeptness in capturing robust local contextual features at these high- and
mid-frequency feature levels. Unlike fully ViT approaches, our method
strategically integrates ResBlocks in two levels to diminish spatial
dimensions without overwhelming feature representation, preparing the input
for the subsequent hybrid stage. This decision is based on two considerations:
Firstly, the initial spatial dimensions carry substantial computational costs
and don’t efficiently leverage spatial ViT blocks, while also failing to
achieve the comprehensive channel representation needed for effective channel-
wise ViT attention blocks. Secondly, by refraining from downsampling in the
Init block, our focus remains on optimizing parameter efficiency and
preserving local features, delegating the responsibility for more global
features to the subsequent level.
At each level of encoders ($l$), the Down Conv block encodes the input feature
$F_{l}$ into the output feature $F_{l+1}$ as defined:

$\begin{aligned} \hat{F}&=\text{BN}(\text{DW-
Conv}(\text{LeakyReLU}(\text{BN}(\text{DW-Conv}_{\text{Down}}(F_{l})))))\\\
R&=\text{BN}(\text{PW-Conv}(F_{l}))\\\
F_{l+1}&=\text{LeakyReLU}(\hat{F}+R)\end{aligned},$ (3)

where, $\text{DW-Conv}_{\text{Down}}$ denotes the convolution operation
responsible for downsampling the spatial dimensions of $F_{l}$. $\hat{F}$
represents the processed feature map obtained by applying normalization,
convolution, and activation functions to the downsampled feature map $F_{l}$.
$R$ captures the original input feature $F_{l}$ after a normalization and
convolutional operation. Finally, $F_{l+1}$ is computed by adding the
processed feature map $\hat{F}$ to the original feature map $R$, followed by
an activation function. Also, Equation 3 can be considered A ResBlock
operations when $R=F_{l}$. This process facilitates the flow of information
while efficiently managing spatial dimensions across different levels of the
encoder.

###  2.3 Hybrid Blocks (Hybrid Attention)

The strategic incorporation of the LKAd module within the network
architecture, particularly in intermediate to deep layers, is instrumental in
enhancing the network’s efficacy for semantic segmentation tasks, wherein the
intricate interplay between local details and global context is paramount for
accurate delineation of semantic boundaries and objects within the image.

In this final stage of the proposed network, comprising intermediate and deep
layers, the emphasis shifts towards employing hybrid attention techniques.
Having established a robust foundation in capturing and learning local
features in the preceding stage, our focus now broadens to encompass a diverse
array of attention mechanisms.

This includes the integration of larger kernel sizes, which enable to capture
a broader range of spatial representations, coupled with deformations to
ensure the preservation and appropriate focusing on the desired receptive
fields. Simultaneously, incorporating spatial ViT attention in the initial
layers of this stage and channel ViT attention in the final layers serves to
aggregate rich and high-level descriptions within the channels, facilitating
the extraction and incorporation of optimal features and descriptions.

This strategic arrangement not only provides efficiency by judiciously
selecting appropriate combinations across different layers, but also avoids
the need for repeated use of a fixed block Shaker et al. (2022); Rahman et al.
(2024); Liu et al. (2024) or multiple repetitions Azad et al. (2024); Wang et
al. (2023a); Hong et al. (2023); Li et al. (2023) across levels. Such an
approach makes it possible to capture nuanced and meaningful descriptions
while maintaining complexity and minimizing the number of parameters, thereby
facilitating effective learning.

####  2.3.1 Self-Adaptive Contextual Fusion Module

Incorporating the spatial attention block within the top hybrid blocks, both
residually in the encoder stage and collectively in the decoder stage,
alongside the convolutional information flow, presents a sophisticated
strategy to fortify the model’s capacity in addressing global structures. This
integration facilitates the effective capture of global contextual information
through a fusion of spatial attention maps, computed in parallel using the
outputs of LKAd and self-attention mechanisms. By strategically intertwining
these components, this block aids in mitigating information loss, especially
within the deeper layers of the encoder, while seamlessly transferring
features to the CNN Blocks in the decoder.

In the spatial attention block, we leverage the LKAd block alongside spatial
self-attention. This strategic combination enables the model to concurrently
address long-range dependencies for extracting comprehensive global contextual
representations and mitigate texture bias while ensuring the effective
extraction of features and inductive biases without compromising the high-
frequency representation. LKAd facilitates the consideration of large regions
within the input image in a flexible and adaptive conventional manner, thereby
enabling a broader receptive field by combining the information with a
deformable grid which is utilized after two convolution layers of LKAd
compared to standard attention mechanisms confined to fixed-size regions. Such
adaptability empowers the block to efficiently capture both local and global
contextual information.

As depicted in Figure 2, the final output of the Self-adaptive Contextual
Fusion block is computed as:

$F_{S}=\text{Comb}\left(F,\delta_{s}\mathbf{LKAd}(F)+\gamma_{s}\mathbf{SA}(F)\right)$
(4)

Where $F\in\mathbb{R}^{C\times H\times W\times D}$ represents the input
feature maps, and $F_{S}$ denotes the output of this block with the same
dimensions. $\delta_{s}$ and $\gamma_{s}$ represent learnable parameters per
channel that control the combination weights of the two different attention
mechanisms.

LKAd: This module orchestrates attention computation, as delineated in Azad et
al. (2024) by the following formulation:

$\displaystyle\hat{x}$ $\displaystyle=\text{GELU}(\text{PW-Conv}(x))$ (5)
$\displaystyle x_{\text{LA}}$ $\displaystyle=\text{DWD-Conv}\left(\text{DW-
Conv}(\hat{x})\right)$ $\displaystyle x_{\text{LKAd}}$
$\displaystyle=\text{PW-Conv}(\text{DDW-Conv}_{3}(x_{\text{LA}}))$
$\displaystyle x_{\text{Ad}}$ $\displaystyle=\text{PW-
Conv}(x_{\text{LKAd}}\otimes\hat{x})+x,$

where the input tensor $x\in\mathbb{R}^{C\times H\times W\times D}$ undergoes
a series of transformations aimed at enhancing its representational capacity
and contextual understanding. First, the input is subject to a pointwise
convolution operation, denoted as $\text{Conv}_{1}$, followed by an activation
function, Gaussian Error Linear Units (GELU), to induce nonlinearity and
dimensionality reduction. The transformed tensor $\hat{x}$ is then processed
through a sequence of depth-wise convolutions (DW-Conv) and depth-wise dilated
convolutions (DWD-Conv) to extract multi-scale features while preserving
spatial information. The resulting tensor $x_{\text{LA}}$ encapsulates locally
attentive representations. Notably, the distinctive aspect of the LKAd module
lies in its integration of deformable depth-wise convolutions (DDW-Conv),
denoted as $\text{DDW-Conv}_{3}$, which introduces deformable grids to enable
adaptive sampling of feature maps, thereby enhancing the model’s ability to
capture fine-grained details and long-range dependencies. Finally,
acknowledging that $\otimes$ and $+$ denote element-wise multiplication and
summation, respectively, the refined feature map $x_{\text{LKAd}}$ is
convolved with the original input $\hat{x}$ through a pointwise convolution
operation, facilitating the fusion of global contextual information with local
features. This resulting tensor, $x_{\text{Ad}}$, embodies a synergistic
fusion of local and global context, effectively enriching the representation
of the input tensor $x$.

Self-Attention: The $SA$ refers to the self-attention module. To compute the
final result $SA(x)$, the normalized tensor $x$ of shape $n,C$ is generated,
where $n=HWD$. This involves computing $Q$, $K$, and $V$ projections using 3
linear layers, resulting in dimensions $n,h,C/h$, where $h$ denotes the number
of heads, by performing $Q=W^{Q}X$, $K=W^{K}X$, and $V=W^{V}X$. Here, $W^{Q}$,
$W^{K}$, and $W^{V}$ represent the projection weights for $Q$, $K$, and $V$,
respectively. To manage complexity and learn spatial information efficiently,
a technique proposed in Shaker et al. (2022) is used, which involves two
projections for $K$ and $V$. This yields $K_{p}=W^{p}_{K}\times K$ and
$V_{p}=W^{p}_{V}\times V$, projecting the $n$ quantized spatial tokens from
$K$ and $V$ into $p$ learnable complex spatial separations in $K_{p}$ and
$V_{p}$, altering the dimensions from $n,h,C/h$ to $p,h,C/h$, where $p\ll n$.
Then, by multiplying the normalized $Q$ by the transpose of $K_{p}$ and
calibrating the resulting values of each head using the learnable parameter
$\alpha$, and applying a softmax function ($\sigma$), the similarity between
each feature and the remaining spatial features is obtained. Finally, these
computed similarities are multiplied by the projected $V$ layer to produce the
final spatial attention with dimensions $n\times C$. This spatial attention
mechanism is described by the equation:

$x_{SA}=\sigma\left(\frac{QK_{p}^{T}}{\sqrt{d}}\right)\cdot V_{p}$ (6)

Here, $Q$, $K_{p}$, and $V_{p}$ stand for queries, projected keys, and
projected spatial value layers, respectively, each of size $d$. Next, we pass
$x_{SA}$ through a linear normalization followed by a linear layer to enhance
the performance as the result of $SA$ module. Finally, through the integration
of the Comb function, as defined in Equation 7:

$\displaystyle\text{Comb}(x,A)$ $\displaystyle=C+\text{PW-Conv}(\text{DW-
Conv}_{3}(C))$ (7) $\displaystyle\text{where}\quad C$
$\displaystyle=x\otimes(1+A)$

where $\text{DW-Conv}_{3}$ is the 3×3×3 convolution block, we obtain the final
output.

####  2.3.2 OmniFocus Attention Block

Located at the deepest level of our network architecture (which has the
highest number of feature layers of all hybrid blocks), both residually in the
encoder stage and collectively in the decoder stage, this block synergizes
with the convolutional information flow to provide a crucial enhancement in
feature representation to augment the model’s ability to capture intricate
channel-wise relationships and helps the model gain the ability to selectively
emphasize relevant meaningful features while suppressing noise across spatial
locations. The integration of ViT channel attention along with LKAd in this
omni focus attention block, enables the efficient capture of interdependencies
between different channels through the use of the proposed combined attention
mechanisms, facilitating seamless feature transfer for the next decoder
layers. Within this attention block, we leverage the LKAd block, similar to
Self-Adaptive Contextual Fusion Module, alongside channel attention
mechanisms. This synergistic combination enables the model to simultaneously
address inter-channel relationships and long-range dependencies, facilitating
the extraction of comprehensive feature representations. By integrating these
mechanisms, the channel attention block empowers the model to effectively
extract and utilize rich contextual information for enhanced feature
representation and improved final performance.
As depicted in Figure 2, the final output of this attention block similar to
Equation 8 is computed as:

$F_{S}=\text{Comb}\left(F,\delta_{c}\mathbf{LKAd}(F)+\gamma_{c}\mathbf{CA}(F)\right)$
(8)

Where $F$ denotes the previously extracted input feature maps in the shape of
$C,H,W,D$ and $\theta_{c}$ and $\gamma_{c}$ represent learnable parameters per
channel layers that control the combination weights of the two different
attention mechanisms with respect to each head ($h$). The LKAd module computes
attention to the input feature $F$ following the formulation introduced in
Equation 5, while $CA$ denotes the channel attention module. This module
captures interdependencies between feature channels by employing the dot
product operation in the channel dimension between the channel value layer and
the channel attention maps. Similar to the creation of $Q$, $K$, and $V$
projections using three linear layers for self-attention, resulting in
dimensions $n,h,C/h$, we repeat this process to obtain $Q_{C}$, $K_{C}$, and
$V_{C}$ for calculating channel attention:

$x_{CA}=V_{C}\cdot\sigma\left(\frac{Q_{C}^{T}K_{C}}{\sqrt{d}}\right)$ (9)

Next, we pass $x_{CA}$ through a linear normalization followed by a linear
layer to improve performance, akin to the result of the $CA$ module. Then, as
defined in Equation 7, using the given input $x$ and the summation of weighted
attention results, we generate the final output utilizing the Comb function.

##  3 Experimental setup and analysis

In this section, we discuss datasets, experimental setup, and evaluation
matrices, used to showcase the effectiveness of our proposed framework for
medical image segmentation.

###  3.1 Datasets

MICCAI Multi-Atlas Abdomen Labeling Challenge (Synapse) Landman et al. (2015):
The Synapse dataset consists of 30 abdominal CT scans ranging in size from 512
x 512 x 85 to 512 x 512 x 198 voxels. This collection offers a wide range of
slice thicknesses (2.5 mm to 5.0 mm) and in-plane resolutions (0.54 x 0.54 mm²
to 0.98 x 0.98 mm²), making it ideal for developing and testing advanced 3D
segmentation algorithms. Following the method described in Azad et al. (2024);
Shaker et al. (2022), the training set of the dataset consists of 18 samples,
with the remaining 12 samples reserved for testing. Each scan in this dataset
is precisely labeled to identify eight major abdominal organs: the aorta,
gallbladder, spleen, left and right kidneys, liver, pancreas, and stomach. The
images were randomly cropped to 128 × 128 × 64 for model input, as described
in Zhou et al. (2021); Isensee et al. (2021). The same method as in previous
studies Zhou et al. (2021); Chen et al. (2023) was used for inference.

Automated Cardiac Diagnosis Challenge (ACDC) Bernard et al. (2018): The ACDC
dataset, specifically designed to advance 3D cardiac image segmentation,
consists of a collection of cine MRI images from a diverse group of 100
patients. This dataset is crucial for the advancement of advanced 3D
segmentation algorithms, specifically in the accurate delineation of three
major cardiac anatomical structures: the left ventricle (LV), the right
ventricle (RV), and the myocardium (Myo). Each scan in this dataset provides
intricate visual representations of these structures, allowing for extremely
precise segmentation. The MRI scans vary in spatial resolution, typically
ranging from 1.37 to 1.68 mm² per pixel. Furthermore, the dataset consists
primarily of images with a slice thickness of 5 mm, while there are some
occurrences with a slice thickness of 8 mm thickness. Occasionally, there is a
5 mm gap between slices. The variance in the thickness and the gaps between
slices present specific obstacles for segmentation algorithms. These
challenges require the use of strong approaches that can successfully handle
these differences.

In terms of preprocessing, the method adheres to the protocols established in
Chen et al. (2023); Shaker et al. (2022), involving the random cropping of
images to dimensions of 160 x 160 x 16 and using sliding windows for the
inference. The patient dataset is partitioned into separate subsets for the
purpose of model training and evaluation. Specifically, 70 patients are
assigned for training, 10 for validation, and 20 for testing. This allocation
ensures a thorough evaluation of the segmentation models that are generated
using the ACDC dataset.

Left Atrial Dataset (LA) Xiong et al. (2021): constitutes a curated set of 100
three-dimensional MR images, all of which have undergone gadolinium
enhancement. These images are uniform in resolution, each measuring 0.625 mm
across all three spatial dimensions, with two prevalent spatial
configurations: 640 × 640 × 88 and 576 × 576 × 88 pixels. The dataset has been
divided into five distinct groups, with each group consisting of 20 patients.
During the preprocessing stage, we followed the preprocessing in Luo et al.
(2021); Wang et al. (2023b) where the images are standardized to a zero mean
and unit variance to facilitate consistent model input. The images also
undergo strategic cropping, incorporating augmented margins to ensure the
focal region is accentuated. During model training, random cropping of the
volumes is performed to yield sub-volumes of 96 × 96 × 96, which serve as
inputs for the model. During the inference step, The study followed the
methods outlined by Wang et al. (2023b) utilizing the sliding window technique
provided by the MONAI frameworkCardoso et al. (2022).

NIH pancreas dataset (CT-82) Roth et al. (2015): This dataset encompasses a
collection of 82 publicly available, contrast-enhanced, annotated three-
dimensional CT scans of the abdominal region. Each scan within this dataset is
characterized by a resolution of 512 × 512 pixels across the axial plane, with
a variable number of slices spanning from 181 to 466 to accommodate the
diverse anatomical structures present in the patient population. For
structured analysis, the dataset is organized into four cohorts, with each
segment comprising 20, 20, 21, and 21 patients, respectively. In alignment
with the preprocessing methods proposed by Luo et al. (2021); Wang et al.
(2023b), this dataset underwent a standardized preprocessing protocol. The
same configurations for training and inference as established for the LA
dataset were applied.

Brain Tumor Segmentation (BraTS) Challenge 2018 Bakas et al. (2017); Menze et
al. (2014): The Brain Tumor Segmentation (BraTS) Challenge datasets from the
year 2018 play a pivotal role in advancing medical image analysis. This
dataset, constituting a diverse array of pre-operative Magnetic Resonance
Imaging (MRI) scans, are sourced from a multitude of institutions. Key
characteristics of these datasets include the rigorous alignment of the MRIs,
isotropic resampling to a resolution of 1 × 1 × 1 mm, and the removal of the
skull (skull-stripping). The MRIs present a uniform size of 240 × 240 × 155.
The data, encompassing a broad spectrum, are collected from several academic
centers using varied MRI scanning equipment. Each dataset encompasses four MRI
modalities: native T1, post-contrast T1-weighted (T1c), T2-weighted (T2), and
T2 Fluid-attenuated Inversion Recovery (FLAIR). Tumor segmentation in these
datasets is categorized into three distinct sub-regions: the Whole Tumor (WT),
the Tumor Core (TC), and the Enhancing Tumor (ET). Prior to model training and
analysis, this dataset underwent a standardized preprocessing method,
following the preprocessing guideline proposed by Dorent et al. (2019). For
the training phase, MRI scans were randomly cropped to a size of 128 × 128 ×
128 voxels to fit the network input. During the inference phase, we employed
MONAI’s Cardoso et al. (2022) sliding window technique.

The BraTS-2018 dataset includes MRIs from 285 patients, with 210 exhibiting
high-grade gliomas and 75 displaying low-grade gliomas. The data allocation
was as follows: 70% for training, 10% for validation, and 20% for testing. The
2018 BraTS datasets provide an unparalleled resource for the advancement of
automated brain tumor segmentation techniques, presenting diverse challenges
and opportunities for the development of cutting-edge image analysis methods.

###  3.2 Experimental setup

The developed neural network model is implemented using Python (version
3.10.12) and is built upon the PyTorch framework Paszke et al. (2019); Falcon
and The PyTorch Lightning team (2019) (version 2.1.0). The model was trained
on a single NVIDIA A100 graphics processing unit, equipped with 80 GB of VRAM.
The training leveraged a composite loss function, DICE loss Milletari et al.
(2016) with binary cross-entropy, expressed as:

$\mathcal{L}_{total}=0.5\cdot\mathcal{L}_{dice}+0.5\cdot\mathcal{L}_{ce}$ (10)

In terms of optimization, we have utilized the Stochastic Gradient Descent
(SGD) optimization method. The SGD configuration includes a weight decay
factor of $3\times 10^{-5}$, and a momentum of 0.99, enhanced with Nesterov
momentum and a varied learning rate for each dataset. The learning rate is
dynamically updated at each epoch following the equation:

$\text{learning rate}=\text{base
}lr\times\left(\frac{epoch}{max\\_epoch}\right)^{0.9}$ (11)

Table 2: Configuration specifics for network training across various datasets, highlighting the tailored approach to optimize performance for each unique dataset challenge. | Synapse | ACDC | BraTS 2018 | LA & Pancreas
---|---|---|---|---
Crop size | 128 x 128 x 64 | 160 x 160 x 16 | 128 x 128 x 128 | 96 x 96 x 96
Base learning rate | 0.003 | 0.003 | 0.01 | 0.01
Batch size | 2 | 4 | 2 | 2
Sliding window overlap (%) | 50 | 50 | 50 | 85
Downsample111A single integer indicates uniform downsampling across all axes. | [2, 2, 1], 2, 2, 2, 2 | [2, 2, 1], [2, 2, 1], [2, 2, 1], 2, 2 | 2, 2, 2, 2, 2 | 2, 2, 2, 2, 2

Data augmentation techniques consistent with Zhou et al. (2021); Shaker et al.
(2022) were employed. The training duration varied across datasets, with the
Synapse and ACDC datasets undergoing 1000 epochs, featuring 250 patches per
epoch, amounting to 250,000 iterations in total. Conversely, for the BraTS
datasets, a uniform 1000 epochs were implemented, whereas the LA and Pancreas
datasets were subjected to 10,000 iterations, reflecting the tailored approach
to training regimen based on dataset specificity. Table 2 encapsulates the
network configurations deployed for each dataset under study, detailing
parameters such as crop size, initial learning rate, batch size, percentage of
sliding window overlap, and downsampling strategy. These configurations
underscore the customized approach to each dataset, optimizing for both
performance and computational efficiency.

###  3.3 Evaluation metrics

Dice Similarity Coefficient (DSC) Milletari et al. (2016): It is used to
quantify the similarity between the predicted segmentation and the ground
truth in medical images, formulated as below:

$\text{DSC}=\frac{2\times|\text{GT}\cap\text{PR}|}{|\text{GT}|+|\text{PR}|}$
(12)

where GT represents the ground truth segmentation, PR is the predicted
segmentation, and $|\text{GT}\cap\text{PR}|$ is the number of pixels or voxels
common to both.

Jaccard Similarity Coefficient Taha and Hanbury (2015): This coefficient, also
referred to as Intersection over Union, measures the overlap between the
predicted segmentation and ground truth. It is calculated as:

$\text{Jaccard
Index}=\frac{|\text{GT}\cap\text{PR}|}{|\text{GT}\cup\text{PR}|}$ (13)

where $|\text{GT}\cup\text{PR}|$ represents the total number of unique pixels
or voxels in either the ground truth or the prediction.

95% Hausdorff Distance (HD95) Taha and Hanbury (2015): It is a metric employed
to assess the discrepancy between two point sets, commonly representing the
ground truth and the predicted segmentation in image segmentation tasks.
Contrary to the traditional Hausdorff Distance, which identifies the maximum
distance among the sets, HD95 concentrates on the 95th percentile of these
distances to minimize the effect of outliers. Formally, HD95 is delineated as
the maximum of two directional distances at the 95th percentile:

$\text{HD95}(\text{GT},\text{PR})=\max\left(\tilde{h}_{95}(\text{GT},\text{PR}),\tilde{h}_{95}(\text{PR},\text{GT})\right)$
(14)

where $\tilde{h}_{95}(\text{GT},\text{PR})$ signifies the 95th percentile of
all distances from points in the ground truth set to their nearest
counterparts in the prediction set, and $\tilde{h}_{95}(\text{PR},\text{GT})$
indicates the 95th percentile of distances for the inverse relation, from
points in PR to their closest points in GT. metric focuses on the closest 95%
of points between predicted and actual segmentations, excluding the most
extreme 5% to reduce outlier impact. This provides a reliable gauge of
segmentation accuracy, reflecting true model performance.

Average Surface Distance (ASD) Taha and Hanbury (2015): ASD measures the
average distance from points on the predicted segmentation surface to the
closest points on the ground truth surface, and vice versa. The formula is:

$\text{ASD}(\text{GT},\text{PR})=\frac{1}{|S_{\text{GT}}|+|S_{\text{PR}}|}\times\left(\sum_{g\in
S_{\text{GT}}}\min_{p\in S_{\text{PR}}}d(g,p)+\sum_{p\in
S_{\text{PR}}}\min_{g\in S_{\text{GT}}}d(p,g)\right)$ (15)

where $S_{\text{GT}}$ and $S_{\text{PR}}$ are the surfaces of the ground truth
and prediction, respectively, and $d(g,p)$ is the Euclidean distance between
points $g$ and $p$.

##  4 Results and Discussion

###  4.1 Quantitative Results and Discussion

Table 3: Comparative analysis of our approach versus state-of-the-art in the
benchmarks for MICCAI Multi-Atlas Abdomen Labeling Challenge (Synapse). The
highest-performing result is highlighted in blue, while the second-best result
is denoted in red. Parameter counts are reported in millions (M), and FLOPS
are expressed in billions (G). The Dice Similarity Coefficient (DSC) for
Spleen (Spl), Right Kidney (RKid), Left Kidney (LKid), Gallbladder (Gal),
Liver (Liv), Stomach (Sto), Aorta (Aor), and Pancreas (Pan) are detailed as
percentages. Furthermore, the average DSC and 95% Hausdorff Distance (HD95)
for all organs are included to provide a comprehensive evaluation of
segmentation accuracy. Two-dimensional (2D) models are indicated with *, and
three-dimensional (3D) models are italicized.

Methods | Params$\downarrow$ | FLOPS$\downarrow$ | Spl | RKid | LKid | Gal | Liv | Sto | Aor | Pan | Average
---|---|---|---|---|---|---|---|---|---|---|---
DSC $\uparrow$ | HD95 $\downarrow$
U-Net* Ronneberger et al. (2015) | - | - | 86.67 | 68.60 | 77.77 | 69.72 | 93.43 | 75.58 | 89.07 | 53.98 | 76.85 | -
TransUNet* Chen et al. (2021) | 96.07 M | 88.91 G | 85.08 | 77.02 | 81.87 | 63.16 | 94.08 | 75.62 | 87.23 | 55.86 | 77.49 | 31.69
SSTrans-Net* Fu et al. (2024) | 29.64 M | - | 92.23 | 83.33 | 86.99 | 72.87 | 94.61 | 81.37 | 86.72 | 64.98 | 82.89 | 15.55
FET* Azad et al. (2023c) | 47.01 M | - | 91.85 | 80.06 | 86.66 | 69.67 | 94.43 | 80.34 | 85.31 | 67.08 | 81.92 | 18.41
HiFormer* Heidari et al. (2023) | 25.51 M | - | 90.99 | 79.77 | 85.23 | 65.69 | 94.61 | 81.08 | 86.21 | 59.52 | 80.39 | 14.70
UNETR Hatamizadeh et al. (2022) | 92.49 M | 75.76 G | 85.00 | 84.52 | 85.6 | 56.3 | 94.57 | 70.46 | 89.8 | 60.47 | 78.35 | 18.59
Swin-UNETR Hatamizadeh et al. (2021) | 62.83 M | 384.2 G | 95.37 | 86.26 | 86.99 | 66.54 | 95.72 | 77.01 | 91.12 | 68.8 | 83.48 | 10.55
nnFormer Zhou et al. (2021) | 150.5 M | 213.4 G | 90.51 | 86.25 | 86.57 | 70.17 | 96.84 | 86.83 | 92.04 | 83.35 | 86.57 | 10.63
UNETR++ Shaker et al. (2022) | 42.96 M | 47.98 G | 95.77 | 87.18 | 87.54 | 71.25 | 96.42 | 86.01 | 92.52 | 81.1 | 87.22 | 7.53
MoAFormer Luo et al. (2022) |  $\sim$ 50 M | - | 92.21 | 85.23 | 86.72 | 71.84 | 95.92 | 81.56 | 89.73 | 75.46 | 84.80 | 16.3
TC-CoNet Chen et al. (2023) | 313.67 M | 699.58 G | 91.77 | 87.92 | 88.16 | 62.00 | 96.35 | 79.4 | 92.45 | 72.78 | 83.86 | 9.64
MS-TCNet Ao et al. (2024) |  $\sim$ 59 M |  $\sim$ 172 G | 93.70 | 86.66 | 86.88 | 66.96 | 95.82 | 82.82 | 91.60 | 68.19 | 84.08 | 8.44
MOSformer Huang et al. (2024) |  $\sim$ 77 M |  $\sim$ 100 G | 92.29 | 83.58 | 90.32 | 71.90 | 95.96 | 87.87 | 88.95 | 74.14 | 85.63 | 13.4
HCA-former Yang et al. (2024) |  $\sim$ 100 M | - | 92.46 | 86.67 | 87.21 | 70.19 | 96.52 | 85.91 | 92.65 | 82.24 | 86.92 | 7.86
LHU-Net | 10.52 M | 51.75 G | 96.01 | 87.80 | 87.57 | 72.43 | 96.68 | 85.79 | 92.42 | 81.21 | 87.49 | 4.15

---
|  |  |  |
|  |  |  |
|  |  |  |
Ground Truth | LHU-Net | nnFormer | UNETR++ | UNETR

Figure 3: Qualitative comparison of the results of our model with respect to
other state-of-the-art (SOTA) in MICCAI Multi-Atlas Abdomen Labeling Challenge
(Synapse) (images were generated using 3D Slicer Fedorov et al. (2012)).

MICCAI Multi-Atlas Abdomen Labeling Challenge (Synapse): Table 3 presents a
comprehensive comparison of the LHU-Net model against leading models in the
field on the Synapse dataset, a part of the MICCAI Multi-Atlas Abdomen
Labeling Challenge. LHU-Net distinguishes itself with an impressive overall
average DSC of 87.49%, showcasing its superior organ segmentation accuracy.
Remarkably, LHU-Net achieves this high level of accuracy while maintaining a
model size of only 10.52 million parameters and requiring 51.74 billion FLOPS,
demonstrating an exceptional balance between performance and computational
efficiency. Specifically, LHU-Net leads in spleen segmentation with a DSC of
96.01%, evidencing its capability in accurately delineating this organ. It
also achieves notable success in segmenting the right kidney, with DSCs of
87.80% illustrating its adeptness in handling organs with complex anatomical
structures. The model also performs commendably in segmenting the gallbladder
and liver, organs known for their challenging variability in shape and size,
achieving DSCs of 72.42% and 96.68%, respectively. This is particularly
significant given the gallbladder’s propensity for variable positioning and
the liver’s large spatial occupancy, further emphasizing LHU-Net’s robust
feature extraction capabilities. A pivotal aspect of LHU-Net’s success is its
achievement of the lowest HD95 across all models at 4.15 mm. This demonstrates
the model’s enhanced precision, ensuring that 95% of the points on the
predicted segmentation surface are within 4.15 mm of the actual segmentation
surface. Such precision is critical in clinical contexts, where the accuracy
of segmentation has immediate implications for treatment planning and patient
outcomes. Contrasted with other high-performance models, LHU-Net not only
surpasses them in segmentation accuracy but also in computational efficiency.
It outperforms more computationally intensive models like TC-CoNet and Swin-
UNETR, demonstrating that increased model complexity does not necessarily
correlate with better performance.

Table 4: Comparative analysis of our approach versus state-of-the-art in the
benchmarks for Automated Cardiac Diagnosis Challenge (ACDC). The highest-
performing result is highlighted in blue, while the second-best result is
denoted in red. Parameter counts are reported in millions (M), and FLOPS are
expressed in billions (G). Dice Similarity Coefficient (DSC) is provided for
the Left Ventricle (LV), Right Ventricle (RV), and Myocardium (Myo), expressed
as a percentage. Two-dimensional (2D) models are indicated with *, and three-
dimensional (3D) models are italicized. The overall average DSC across all
heart regions is also provided to reflect comprehensive model performance.

Methods | Params$\downarrow$ | FLOPS$\downarrow$ | LV | Myo | RV | Avg. DSC $\uparrow$
---|---|---|---|---|---|---
TransUNet* Chen et al. (2021) | - | - | 95.73 | 84.54 | 88.86 | 89.71
Swin-UNet* Cao et al. (2022) | - | - | 95.83 | 85.62 | 88.55 | 90.00
SSTrans-Net* Fu et al. (2024) | 29.64 M | - | 94.40 | 87.51 | 89.02 | 90.31
UNETR Hatamizadeh et al. (2022) | 92.69 M | 77.84 G | 94.02 | 86.52 | 85.29 | 88.61
nnFormer Zhou et al. (2021) | 37.16 M | 47.73 G | 95.59 | 89.53 | 90.22 | 91.78
MoAFormer Luo et al. (2022) | 46.12 M | - | 95.15 | 88.49 | 90.15 | 91.26
TC-CoNet Chen et al. (2023) | 86.79 M | 145.16 G | 95.47 | 88.98 | 90.27 | 91.58
D-LKA Net Azad et al. (2024) | 39.95 M | 42.41 G | 94.68 | 88.04 | 91.35 | 91.36
neU-Net Yang et al. (2023) | 161.28 M | 129.83 G | 95.66 | 89.91 | 90.75 | 92.11
HCA-Former Yang et al. (2024) |  $\sim$ 100 M | - | 95.8 | 89.76 | 90.77 | 92.11
MOSformer Huang et al. (2024) | 77.09 M | 100.06 G | 96.05 | 89.65 | 90.86 | 92.19
MS-TCNet Ao et al. (2024) | 59.49 M | 172.89 G | 95.77 | 89.09 | 89.43 | 91.43
D-Former Wu et al. (2023) |  $\sim$ 40 M |  $\sim$ 50 G | 95.93 | 89.60 | 91.33 | 92.29
LHU-Net | 5.07 M | 12.37 G | 96.26 | 90.56 | 91.15 | 92.66

---
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
Image | Ground Truth | LHU-Net | nnFormer | TC-CoNet | UNETR

Figure 4: Qualitative comparison of the results of our model with respect to
other SOTA in Automated Cardiac Diagnosis Challenge (ACDC) dataset.

Automated Cardiac Diagnosis Challenge (ACDC): Table 4 provides a detailed
comparative analysis of advanced deep learning models applied to the Automated
Cardiac Diagnosis Challenge (ACDC) dataset, focusing on the segmentation of
key cardiac structures: the left ventricle (LV), myocardium (Myo), and right
ventricle (RV). Utilizing the DSC as the primary metric for assessment, the
LHU-Net model distinguishes itself by attaining an average DSC of 92.66%. This
exemplary performance underscores its efficacy in cardiac structure
segmentation. The efficiency of LHU-Net is underscored by its lean parameter
count of only 5.07 million and the lowest computational footprint, denoted by
12.37 billion FLOPS. This efficiency starkly contrasts with other contenders,
where an uptick in parameters and computational complexity has yielded only
incremental performance enhancements. LHU-Net’s architecture proves that
through thoughtful design, significant leaps in performance can be achieved
without proportionally escalating computational demands. LHU-Net showcases
superior performance in Myocardium (Myo) segmentation with a DSC of 90.56%, a
pivotal achievement considering the Myo’s intricate structure and crucial role
in diagnostic evaluations. Its accomplishments in LV and RV segmentation are
equally noteworthy, with DSCs of 96.26% and 91.15%, respectively, either
surpassing or closely rivalling the top models in these domains.
Comparatively, LHU-Net’s nearest rivals, such as D-Former and MOSformer,
report impressive DSC scores but incur substantial parameter and computational
costs. D-Former, for instance, achieves an average DSC of 92.29% yet demands
significantly more resources than LHU-Net, highlighting the diminishing
performance returns as model complexity increases. This makes LHU-Net a
preferable alternative for medical image segmentation, balancing efficiency
with efficacy. The overall percentage improvement in DSC by LHU-Net, alongside
a remarkable reduction in parameters by over 85% and FLOPS by almost 70%
compared to certain models, marks a notable leap forward in segmentation
technology. This efficiency does not detract from accuracy but rather
demonstrates that judicious architectural innovations can result in superior
performance outcomes.

Table 5: Comparative analysis of our approach versus state-of-the-art in the
benchmarks for Left Atrial Dataset (LA). The top-performing result is
highlighted in blue, while the next best is marked in red. A five-fold cross-
validation approach has been employed, and the results are displayed as mean
followed by standard deviation across all folds. Parameter counts are reported
in millions (M), and FLOPS are expressed in billions (G). DSC and Jaccard
metrics are reported in percentages.

Methods | Params$\downarrow$ | FLOPS$\downarrow$ | Average
---|---|---|---
ASD $\downarrow$ | DSC $\uparrow$ | HD95 $\downarrow$ | Jaccard $\uparrow$
SwinUNETR Hatamizadeh et al. (2021) | 62.2 M | 319.38 G | 1.90 (0.34) | 91.20 (1.07) | 6.07 (1.16) | 83.92 (1.57)
SwinUNETR-V2 He et al. (2023) | 72.77 M | 345.23 G | 1.74 (0.28) | 91.48 (0.79) | 5.59 (1.03) | 84.41 (1.26)
UNETR Hatamizadeh et al. (2022) | 92.78 M | 73.51 G | 3.60 (0.47) | 88.25 (1.31) | 11.55 (1.58) | 79.32 (1.97)
UNETR++ Shaker et al. (2022) | 29.54 M | 29.74 G | 1.77 (0.30) | 91.05 (1.12) | 5.47 (1.01) | 83.62 (1.79)
nnFormer Zhou et al. (2021) | 149.25 M | 176.45 G | 2.49 (0.36) | 88.79 (1.98) | 7.09 (0.75) | 80.21 (2.72)
VNet Milletari et al. (2016) | 9.44 M | 82.76 G | 1.99 (0.31) | 89.90 (1.79) | 6.11 (1.09) | 81.91 (2.65)
LHU-Net | 8.55 M | 32.55 G | 1.55 (0.14) | 91.55 (0.95) | 4.97 (0.75) | 84.54 (1.51)

|  |  |  |  |
---|---|---|---|---|---
|  |  |  |  |
|  |  |  |  |
Ground Truth | LHU-Net | nnFormer | SwinUNETR-V2 | UNETR | UNETR++

Figure 5: Qualitative comparison of the results of our model with respect to
other SOTA in Left Atrial Dataset (images were generated using 3D Slicer
Fedorov et al. (2012)).

Left Atrial Dataset (LA): Table 5 presents a comprehensive quantitative
analysis across several methods. The LHU-Net model establishes itself as a
benchmark of excellence in the comprehensive analysis of the Left Atrial
Dataset. This model not only excels in standard metrics like the DSC and
Jaccard index but also sets new standards in ASD and HD95, providing a
holistic view of its segmentation prowess. LHU-Net, with its lean architecture
requiring merely 8.57 million parameters and 34.28 billion FLOPS, emerges as
the epitome of efficiency and efficacy. This is highlighted by its outstanding
average DSC of 91.55, which marginally outperforms SwinUNETR-V2. This
underscores LHU-Net’s superior ability to accurately delineate the complex
anatomy of the left atrium. This slight yet significant edge in DSC is
complemented by leading performance in the Jaccard index at 84.54, reinforcing
the model’s effectiveness in capturing the true extent of atrial structures.
Crucially, LHU-Net’s excellence extends to the ASD and HD95 metrics, where it
achieves the lowest scores of 1.55 and 4.97, respectively. These figures
represent the model’s unparalleled precision in approximating the true surface
of the atrium and minimizing the distance to the farthest outliers, essential
for clinical applications where every millimeter could be critical. The ASD
score, in particular, showcases LHU-Net’s ability to maintain consistent
proximity to the actual atrial boundary across all dataset variations.
Comparatively, while other models like UNETR++ and SwinUNETR-V2 exhibit
commendable performances, particularly in 3D segmentation tasks, they do not
achieve the same balance of high accuracy and computational efficiency as LHU-
Net. For instance, despite SwinUNETR-V2’s competitive DSC and Jaccard scores,
LHU-Net’s superior ASD and HD95 metrics underscore its ability to provide more
reliable and precise segmentations, which are crucial for clinical diagnostics
and interventions.

NIH pancreas dataset (CT-82): Table 6 presents the quantitative assessment of
various segmentation models on the NIH Pancreas dataset (CT-82). LHU-Net’s
proficiency is underscored not only by its exemplary performance in the
average DSC and HD95 but also in its achievements across the Jaccard index and
ASD, metrics essential for assessing the comprehensive accuracy and proximity
of predicted segmentations to ground truth. With a minimal architecture
demanding only 8.57 million parameters and 34.28 billion FLOPS, LHU-Net
achieves remarkable efficiency, evidenced by its top-ranking average DSC of
81.96 and the lowest HD95 at 6.78. This indicates an unparalleled ability to
delineate pancreatic tissues accurately, minimizing outliers and extreme
segmentation errors that are critically important in clinical diagnostics. The
model’s excellence extends to the Jaccard index, achieving a leading score of
70.12, which reflects a high degree of overlap between the segmented and
actual pancreatic regions. This metric, along with the model’s superior ASD
score of 1.31, highlights LHU-Net’s ability to closely approximate the surface
of the pancreas, ensuring that the segmented boundary closely matches the true
anatomical boundary, a testament to the model’s precision in capturing
intricate organ shapes. In comparison to its counterparts, such as UNETR++ and
SwinUNETR-V2, LHU-Net not only presents a more computationally efficient
framework but also ensures that this efficiency does not come at the cost of
segmentation accuracy. Its balanced approach in optimizing both computational
resources and segmentation fidelity positions LHU-Net as an ideal solution for
clinical deployment, where time and computational efficiency are as valued as
accuracy and reliability. In essence, LHU-Net demonstrates high accuracy
across DSC, HD95, Jaccard, and ASD metrics, along with computational
efficiency.

Table 6: Comparative analysis of our approach versus state-of-the-art in the
benchmarks for NIH pancreas dataset (CT-82). The top-performing result is
highlighted in blue, while the next best is marked in red. A four-fold cross-
validation approach has been employed, and the results are displayed as mean
followed by standard deviation across all folds. Parameter counts are reported
in millions (M), and FLOPS are expressed in billions (G). DSC and Jaccard
metrics are reported in percentages.

Methods | Params$\downarrow$ | FLOPS$\downarrow$ | Average
---|---|---|---
ASD $\downarrow$ | DSC $\uparrow$ | HD95 $\downarrow$ | Jaccard $\uparrow$
SwinUNETR Hatamizadeh et al. (2021) | 62.2 M | 319.38 G | 1.99 (0.51) | 80.52 (2.13) | 7.52 (1.52) | 68.17 (2.79)
SwinUNETR-V2 He et al. (2023) | 72.77 M | 345.23 G | 2.42 (0.97) | 81.05 (1.90) | 8.85 (2.55) | 68.87 (2.52)
UNETR Hatamizadeh et al. (2022) | 92.78 M | 73.51 G | 4.25 (0.83) | 76.18 (2.55) | 12.98 (2.50) | 62.51 (3.24)
UNETR++ Shaker et al. (2022) | 29.54 M | 29.74 G | 1.95 (0.22) | 81.49 (0.84) | 7.64 (2.01) | 69.07 (1.16)
nnFormer Zhou et al. (2021) | 149.25 M | 176.45 G | 3.28 (0.90) | 78.05 (2.29) | 10.65 (3.74) | 64.81 (2.81)
VNet Milletari et al. (2016) | 9.44 M | 82.76 G | 2.79 (0.30) | 79.31 (1.95) | 8.99 (1.96) | 66.42 (2.55)
LHU-Net | 8.55 M | 32.55 G | 1.31 (0.14) | 81.96 (1.00) | 6.78 (0.52) | 70.12 (1.29)

|  |  |  |  |
---|---|---|---|---|---
|  |  |  |  |
|  |  |  |  |
Ground Truth | LHU-Net | nnFormer | SwinUNETR-V2 | UNETR | UNETR++

Figure 6: Qualitative comparison of the results of our model with respect to
other SOTA in NIH pancreas dataset (images were generated using 3D Slicer
Fedorov et al. (2012)).

Brain Tumor Segmentation (BraTS) Challenge 2018: Table 7 provides a detailed
comparative analysis of various deep learning models, highlighting their
performance in terms of DSC and HD95 for Brain Tumor Segmentation (BraTS)
Challenge 2018 dataset. In this comparative study, the LHU-Net model stands
out as the most proficient, achieving the highest average DSC and the lowest
average HD95. This underscores its effectiveness in accurately identifying
tumor boundaries and overall tumor regions while minimizing the distance
between the predicted and actual tumor borders. LHU-Net’s superior performance
in ET and WT segmentation, as well as its exceptional ability to delineate the
TC, is further highlighted by its leading scores in these categories. Notably,
LHU-Net accomplishes this with a significantly lower parameter count of 10.5
million and a reasonable computational demand, as indicated by its FLOPS
measurement. The LHU-Net model’s performance in segmenting the TC is
particularly impressive, outperforming all other models with a DSC of 87.42.
This is pivotal considering the TC’s clinical importance, as it includes the
bulk of the tumor’s malignant tissue. In the WT category, which represents the
entire tumor area, LHU-Net again outshines the competition with a DSC of
89.57, indicating its exceptional ability to capture the full extent of the
tumor’s presence. In the realm of computational efficiency, LHU-Net
demonstrates a commendable balance, achieving high accuracy with relatively
low computational overhead. While the UNETR++ model exhibits competitive
performance, particularly in the ET segmentation, LHU-Net’s balanced
excellence across all measures of tumor segmentation signifies a robust model
that can provide comprehensive and accurate segmentations.

###  4.2 Qualitative Results and Discussion

Following the quantitative analysis, we shift our focus to a qualitative
evaluation of the same datasets. Here, we assess the performance of models
through visual examination, considering the nuances that are not captured by
numerical data alone. This section offers a narrative interpretation of how
the models’ segmentations align with the expected outcomes and the ground
truth of the medical images. Through this qualitative lens, we glean insights
into the models’ practical capabilities, which further informs their potential
for real-world clinical integration.

Table 7: Comparative analysis of our approach versus state-of-the-art in the
benchmarks for Brain Tumor Segmentation (BraTS) Challenge 2018. The top-
performing result is highlighted in blue, while the next best is marked in
red. Parameter counts are reported in millions (M), and FLOPS are expressed in
billions (G). The terms "Whole Tumor" (WT) refer to the entire tumor area,
"Tumor Core" (TC) denotes the central region of the tumor, and "Enhancing
Tumor" (ET) highlights areas with active tumor cells that are visible after
contrast enhancement.

Methods | Params | FLOPS | DSC $\uparrow$ | HD95 $\downarrow$ | Average
---|---|---|---|---|---
ET | TC | WT | ET | TC | WT | DSC $\uparrow$ | HD95 $\downarrow$
ResUNet Lee et al. (2017) | 28.44 M | 472.62 G | 78.6 | 82.74 | 88.75 | 6.519 | 9.886 | 9.547 | 83.36 | 8.651
ResUNetSE Toubal et al. (2020) | 29.31 M | 472.8 G | 77.84 | 82.55 | 88.67 | 6.699 | 8.954 | 8.066 | 83.02 | 7.906
3D-UNET Çiçek et al. (2016) | 16.47 M | 516.71 G | 78.02 | 82.99 | 88.92 | 7.103 | 9.884 | 8.228 | 83.31 | 8.405
SwinUNETR-V2 He et al. (2023) | 72.77 M | 836.26 G | 80,75 | 84.15 | 88.04 | 5.591 | 7.807 | 7.84 | 84.31 | 7.079
SwinUNETR Hatamizadeh et al. (2021) | 62.2 M | 774.98 G | 79.43 | 82.39 | 88.51 | 5.835 | 7.625 | 10.87 | 83.44 | 8.11
UNETR Hatamizadeh et al. (2022) | 102.45 M | 184.76 G | 76.97 | 78.81 | 86.51 | 9.182 | 11.48 | 13.44 | 80.76 | 11.367
UNETR++ Shaker et al. (2022) | 42.65 M | 73.59 G | 82.29 | 85.87 | 89.38 | 4.348 | 6.456 | 6.23 | 85.85 | 5.678
nnFormer Zhou et al. (2021) | 37.75 M | 106.45 G | 78.95 | 85.57 | 89.4 | 5.676 | 6.362 | 8.723 | 84.64 | 6.920
LHU-Net | 10.50 M | 81.96 G | 81.16 | 87.42 | 89.57 | 3.779 | 5.632 | 6.481 | 86.05 | 5.597

---
|  |  |  |  |  |
|  |  |  |  |  |
|  |  |  |  |  |
Image | Ground Truth | LHU-Net | nnFormer | SwinUNETR-V2 | UNETR | UNETR++

Figure 7: Qualitative comparison of the results of our model with respect to
other SOTA in Brain Tumor Segmentation (BraTS) Challenge 2018 dataset.

MICCAI Multi-Atlas Abdomen Labeling Challenge (Synapse): Figure 3 offers a
qualitative perspective on the segmentation capabilities of various models in
the MICCAI Multi-Atlas Abdomen Labeling Challenge (Synapse). For the first
patient, visualized in the top row, we observe that UNETR struggles
significantly, producing fragmented and scattered segmentations that fail to
represent the organs as cohesive entities. nnFormer and UNETR++ exhibit
irregular segmentation contours, particularly for the left kidney and aorta,
while UNETR++ additionally misses the stomach entirely. In contrast, the LHU-
Net model demonstrates a commendable segmentation performance, accurately
delineating most organs with smooth boundaries, albeit slightly overestimating
the extent of the aorta. The middle row, showcasing the second patient,
presents a more challenging scenario where all models encounter difficulties,
particularly with the segmentation of the kidneys. Here, both nnFormer and
UNETR++ erroneously detect large areas far from the ground truth, highlighting
the clinical significance of minimizing false positives. In this complex case,
LHU-Net outperforms other models by correctly segmenting other organs and only
minor inaccuracies in kidney localization. This indicates a relatively better
understanding of the organ boundaries within the abdominal cavity. For the
third patient, depicted in the bottom row, the pancreas poses significant
segmentation challenges, further complicated by the absence of the gallbladder
in the ground truth segmentation. LHU-Net adeptly delineates the organ
boundaries precisely, whereas UNETR++ and nnFormer encounter difficulties in
accurately segmenting the tail of the pancreas and sharply defining its edges.
UNETR, consistent with its performance in previous cases, struggles with
segmentation accuracy across multiple organs, including the aorta.
Additionally, all models erroneously predict the presence of the gallbladder,
highlighting a common difficulty in correctly identifying missing organs
across the evaluated models. This observation underscores the necessity for
enhanced model sensitivity to the variability and complexities inherent in
medical image segmentation tasks. In conclusion, the LHU-Net model exhibits
superior performance in organ detection and segmentation across the Synapse
dataset. Its segmentation results are not only smoother and more accurate but
also showcase robustness against false positives, a critical factor in
clinical applicability.

Automated Cardiac Diagnosis Challenge (ACDC): Figure 4 presents a qualitative,
two-dimensional evaluation of cardiac segmentation across several state-of-
the-art models within the Automated Cardiac Diagnosis Challenge (ACDC)
dataset. This visual assessment spans across three distinct patient cases,
each represented by a different row of images, to illuminate the comparative
performance of each model in segmenting the left ventricle (LV), myocardium
(Myo), and right ventricle (RV). In the first patient, depicted in the top
row, there is a noticeable challenge for the models in segmenting the RV, with
LHU-Net demonstrating a partial success in capturing some aspects of this
anatomical structure. TC-CoNet and UNETR, in particular, exhibit difficulties
in accurately delineating the Myo and LV, as evidenced by the deviations from
the ground truth. Moving to the second patient, shown in the middle row, the
absence of the RV in the slice poses a test of the models’ specificity, where
false positives are a concern. In this instance, LHU-Net shows commendable
restraint, avoiding the overprediction seen in nnFormer and TC-CoNet, which
falsely identify a substantial RV presence. Meanwhile, all models perform well
on the Myo and LV, indicating their effectiveness in these regions under less
challenging circumstances. The third patient’s imagery, the bottom row,
provides a scenario where LHU-Net’s segmentation closely aligns with the
ground truth, effectively capturing all relevant cardiac structures.
Conversely, nnFormer and TC-CoNet tend towards over-segmentation of the RV,
and UNETR presents with under-segmentation, highlighting a potential
limitation in their segmentation algorithms. In conclusion, the visualization
suggests that LHU-Net possesses superior robustness and precision in cardiac
structure segmentation, outperforming the other models under various
conditions. This robustness is particularly notable in instances of potential
false positives or challenging RV delineation, where LHU-Net maintains high
fidelity to the ground truth.

Left Atrial Dataset (LA): Figure 5 presents a qualitative, three-dimensional
visualization of various models’ performance in segmenting the Left Atrial
dataset for individual patients. Each row in the figure is dedicated to the
segmentation results from a single patient’s dataset, visually assessing each
model’s predictive capabilities. In the case of the first patient, shown in
the top row, all models except UNETR manage to generate a single entity
segmentation, highlighting their ability to integrate disparate anatomical
features into a unified model prediction. However, a common challenge across
all models is the accurate segmentation of the right corner of the atrium,
suggesting a shared area of potential improvement. For the second patient,
delineated in the second row, it is discernible that competing models struggle
to accurately segment the entirety of the ground truth, resulting in multiple
segmented instances rather than a single cohesive structure. Contrarily, LHU-
Net succeeds in rendering a single, contiguous segmentation that closely
resembles the ground truth, demonstrating its ability to produce smoother and
more coherent segmentations. The last patient, as illustrated in the bottom
row, presents a challenging case where all models exhibit a tendency for over-
segmentation at the base of the atrium. This is indicative of the intrinsic
complexity of accurately delineating this anatomical region. Nevertheless, it
is worth noting that certain models, specifically UNETR and SwinUNETR-V2,
additionally exhibit under-segmentation in other areas, further complicating
the segmentation task. In summation, the visualizations in Figure 5 suggest
that LHU-Net consistently generates more robust and smoother segmentations,
avoiding the creation of extraneous entities.

NIH pancreas dataset (CT-82): Figure 6 provides a three-dimensional
visualization for the qualitative analysis of pancreatic segmentation across
several state-of-the-art models. Each row visually represents the segmentation
results from an individual patient’s dataset, offering insight into the
predictive precision of each model. For the first patient, illustrated in the
first row, it is observable that all models, except for LHU-Net, UNETR++, and
SwinUNETR-v2, could not produce a singular, cohesive entity representing the
organ. UNETR++ generates a single entity; however, it significantly under-
segments the pancreas, failing to capture the complete anatomical structure.
In stark contrast, LHU-Net achieves a segmentation that closely approximates
the ground truth, underscoring its capability to preserve the organ’s
integrity in the segmentation process. The second patient’s data, depicted in
the second row, further accentuates the proficiency of LHU-Net in producing a
smooth and robust segmentation, particularly when compared to its
counterparts. Notably, UNETR++ exhibits a fragmentation in the segmentation,
resulting in multiple entities and suggesting a lack of robustness within this
dataset. The visualization of the third patient, in the third row, reveals the
intrinsic challenges posed by this dataset, as all models predict multiple
entities. Nonetheless, LHU-Net distinguishes itself by accurately segmenting
the predominant portion that aligns with the ground truth and generating only
a minor secondary entity. This contrasts with the other models, which produce
multiple, larger erroneous segments, thus compromising the segmentation’s
fidelity. Overall, LHU-Net demonstrates superior robustness and smoothness in
its segmentation outputs, consistently predicting the pancreas with a higher
degree of anatomical accuracy.

Brain Tumor Segmentation (BraTS) Challenge 2018: Figure 7 provides a
qualitative comparison of segmentation results on the Brain Tumor Segmentation
(BraTS) Challenge 2018 dataset, illustrating the performance of the LHU-Net
model relative to other contemporary state-of-the-art models. Each row in the
figure corresponds to a different patient’s MRI slice, showcasing the
segmentation of three distinct tumor regions: the necrotic and non-enhancing
tumor core (NCR), the peritumoral edematous/invaded tissue (ED), and the
enhancing tumor (ET). These regions are critical for evaluating the extent and
aggressiveness of brain tumors. In the analysis of the first patient,
represented in the first row, a challenge emerges for the models to
differentiate between NCR and ET accurately. While many models exhibit
proficiency in segmenting the whole tumor (WT) and tumor core (TC), discerning
the ET from the NCR proves more problematic. LHU-Net, although striving for
accuracy, reveals difficulties at the boundaries of the NCR. SwinUNETR-V2
uniquely attempts to delineate the NCR at the edges, but misplaced detections
highlight the complexity of this task. Nevertheless, LHU-Net presents more
robust and smoother segmentations overall, indicating its advanced capability
for nuanced differentiation within the tumor regions. The second patient,
depicted in the second row, presents a scenario with a large NCR and a smaller
ET region encircling it. LHU-Net distinguishes itself by accurately
delineating all regions, whereas models like UNETR struggle with the finer
details of the NCR and ET. Other models tend to overestimate the NCR size,
demonstrating the difficulty in capturing precise tumor boundaries. LHU-Net’s
performance in this context underscores its exceptional capability to segment
with precision, even in complex scenarios. For the third patient, shown in the
bottom row, the small NCR and ET amidst the irregularly shaped ED tissue
present a significant segmentation challenge. Most models, including UNETR,
produce a continuous segmentation of the ED, yet UNETR also segments an
additional entity far from the tumor’s actual location, indicating a false
positive detection. In contrast, UNETR++ and LHU-Net show a tendency to over-
segment the NCR, while SwinUNETR-V2 and nnFormer strive to provide complete
segmentation across all regions, despite the challenging nature of the task.

##  5 Ablation studies

This section is dedicated to dissecting the architectural nuances of our LHU-
Net model, specifically focusing on the variations of macro CNN and hybrid
modules, as well as the individual contributions of distinct attention
mechanisms. These ablation studies are vital to understanding the impact of
each architectural component on the model’s performance.

Influence of macro module composition: The LHU-Net model’s architecture
employs a blend of convolutional neural networks (CNN) and hybrid macro
modules, sequenced strategically for optimal feature extraction. We
investigate the model’s performance sensitivity to the depth of the network by
varying the number of downsampling stages and the composition of macro modules
at each level. In Table 8, we present a comprehensive analysis of the
influence that varying the number of macro modules exerts on the DSC within
the ACDC dataset. As established in Section 2, our configuration preserves the
sequence of initially deploying CNN modules followed by hybrid modules. Our
findings indicate that reducing the number of downsampling layers from five to
four significantly decreases the parameter count, but it adversely affects the
DSC, suggesting a trade-off between model complexity and segmentation
accuracy. Moreover, we observe that merely increasing the number of CNN macro
modules does not enhance performance, confirming that an excess of such
modules does not equate to better results. The optimal configuration for our
LHU-Net, which is also the proposed model configuration, comprises two CNN
macro modules succeeded by three hybrid macro modules. This particular
arrangement is instrumental in achieving the model’s pinnacle DSC, thus
highlighting the importance of a balanced macro-module composition for
sophisticated segmentation tasks.

Table 8: Ablation analysis on the effects of macro module composition on the ACDC dataset performance, evaluated using the average DSC. The optimal model configuration is highlighted in bold. N. CNNs | N. hybrids | Params |  Avg. DSC $\uparrow$
---|---|---|---
2 | 2 | 3.37 M | 92.43
2 | 3 | 5.07 M | 92.65
3 | 2 | 3.07 M | 92.31

Influence of attention mechanisms within hybrid macro modules: In the hybrid
macro modules of LHU-Net, we have the opportunity to deploy different forms of
attention blocks—namely, OmniFocus attention and self-adaptive contextual
fusion module. OmniFocus attention mechanisms focus on amplifying informative
features across channels, while the self-adaptive contextual fusion module
prioritizes relevant spatial information. The strategic placement of these
blocks within the hybrid macro modules is crucial to the model’s performance.
Our study evaluates the efficacy of various configurations of these attention
blocks through their impact on the average DSC on the ACDC dataset. Table 9
presents the results of this analysis, with the optimal sequence of hybrid
blocks being two self-adaptive contextual fusion modules followed by an
OmniFocus attention block. This configuration yielded the highest average DSC,
indicating a superior segmentation capability when spatial attention is
prioritized in the initial stages, potentially due to its effectiveness in
capturing the spatial distribution of salient features. The subsequent channel
attention seems to complement this by refining the feature maps across the
channels, thus solidifying the importance of a heterogeneous but strategic
arrangement of attention mechanisms.

Table 9: Ablation analysis investigating the impact of attention block sequence within hybrid macro modules on segmentation performance in the ACDC dataset. S represents self-adaptive contextual fusion module, and C represents OmniFocus attention. The most effective sequence is highlighted in bold. Sequence of attentions |  Avg. DSC $\uparrow$
---|---
SSS | 92.32
CCC | 92.24
SSC | 92.65
SCC | 92.5

The evidence from our ablation study demonstrates that the configuration of
attention blocks within the hybrid modules is not merely a matter of
preference but rather a substantive decision that significantly influences the
model’s performance.

##  6 Conclusion

In this work, we introduced LHU-Net, a novel light hybrid U-Net architecture
tailored for volumetric medical image segmentation. LHU-Net exemplifies the
synergy between minimalistic design and advanced performance, utilizing a
strategic implementation of attention mechanisms. By integrating spatial
attention for initial broad feature capture and channel attention in deeper
layers for meticulous feature refinement, LHU-Net efficiently processes
diverse datasets and tackles various segmentation challenges with fewer than
11 million parameters. Through LHU-Net, we highlight the untapped potential of
achieving high segmentation accuracy at limited model complexity, marking a
significant advancement in the development of accessible yet powerful medical
image analysis tools.

##  7 Acknowledgment

The authors gratefully acknowledge the computational and data resources
provided by the Leibniz Supercomputing Centre (www.lrz.de).

## References

  * Long et al. [2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully convolutional networks for semantic segmentation.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3431–3440, 2015.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_ , pages 234–241. Springer, 2015.
  * Liang-Chieh et al. [2015] Chen Liang-Chieh, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan Yuille.  Semantic image segmentation with deep convolutional nets and fully connected crfs.  In _International Conference on Learning Representations_ , 2015.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Milletari et al. [2016] Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi.  V-net: Fully convolutional neural networks for volumetric medical image segmentation.  In _2016 fourth international conference on 3D vision (3DV)_ , pages 565–571. Ieee, 2016.
  * Azad et al. [2022] Reza Azad, Ehsan Khodapanah Aghdam, Amelie Rauland, Yiwei Jia, Atlas Haddadi Avval, Afshin Bozorgpour, Sanaz Karimijafarbigloo, Joseph Paul Cohen, Ehsan Adeli, and Dorit Merhof.  Medical image segmentation review: The success of u-net.  _arXiv preprint arXiv:2211.14830_ , 2022.
  * Bozorgpour et al. [2023] Afshin Bozorgpour, Yousef Sadegheih, Amirhossein Kazerouni, Reza Azad, and Dorit Merhof.  Dermosegdiff: A boundary-aware segmentation diffusion model for skin lesion delineation.  In _International Workshop on PRedictive Intelligence In MEdicine_ , pages 146–158. Springer, 2023.
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Shamshad et al. [2023] Fahad Shamshad, Salman Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, and Huazhu Fu.  Transformers in medical imaging: A survey.  _Medical Image Analysis_ , page 102802, 2023.
  * Azad et al. [2023a] Reza Azad, Amirhossein Kazerouni, Moein Heidari, Ehsan Khodapanah Aghdam, Amirali Molaei, Yiwei Jia, Abin Jose, Rijo Roy, and Dorit Merhof.  Advances in medical image analysis with vision transformers: A comprehensive review.  _arXiv preprint arXiv:2301.03505_ , 2023a.
  * Azad et al. [2023b] Reza Azad, Amirhossein Kazerouni, Babak Azad, Ehsan Khodapanah Aghdam, Yury Velichko, Ulas Bagci, and Dorit Merhof.  Laplacian-former: Overcoming the limitations of vision transformers in local texture detection.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 736–746. Springer, 2023b.
  * Geirhos et al. [2018] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel.  Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness.  _arXiv preprint arXiv:1811.12231_ , 2018.
  * Chen et al. [2021] Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou.  Transunet: Transformers make strong encoders for medical image segmentation.  _arXiv preprint arXiv:2102.04306_ , 2021.
  * Heidari et al. [2023] Moein Heidari, Amirhossein Kazerouni, Milad Soltany, Reza Azad, Ehsan Khodapanah Aghdam, Julien Cohen-Adad, and Dorit Merhof.  Hiformer: Hierarchical multi-scale representations using transformers for medical image segmentation.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 6202–6212, 2023.
  * Hatamizadeh et al. [2022] Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger R Roth, and Daguang Xu.  Unetr: Transformers for 3d medical image segmentation.  In _Proceedings of the IEEE/CVF winter conference on applications of computer vision_ , pages 574–584, 2022.
  * Xie et al. [2021] Yutong Xie, Jianpeng Zhang, Chunhua Shen, and Yong Xia.  Cotr: Efficiently bridging cnn and transformer for 3d medical image segmentation.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part III 24_ , pages 171–180. Springer, 2021.
  * Wang et al. [2021] Wenxuan Wang, Chen Chen, Meng Ding, Hong Yu, Sen Zha, and Jiangyun Li.  Transbts: Multimodal brain tumor segmentation using transformer.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24_ , pages 109–119. Springer, 2021.
  * Cao et al. [2022] Hu Cao, Yueyue Wang, Joy Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang.  Swin-unet: Unet-like pure transformer for medical image segmentation.  In _European conference on computer vision_ , pages 205–218. Springer, 2022.
  * Zhou et al. [2021] Hong-Yu Zhou, Jiansen Guo, Yinghao Zhang, Lequan Yu, Liansheng Wang, and Yizhou Yu.  nnformer: Interleaved transformer for volumetric segmentation.  _arXiv preprint arXiv:2109.03201_ , 2021.
  * Yang et al. [2024] Fan Yang, Fan Wang, Pengwei Dong, and Bo Wang.  Hca-former: Hybrid convolution attention transformer for 3d medical image segmentation.  _Biomedical Signal Processing and Control_ , 90:105834, 2024.
  * Chen et al. [2023] Yong Chen, Xuesong Lu, and Qinlan Xie.  Collaborative networks of transformers and convolutional neural networks are powerful and versatile learners for accurate 3d medical image segmentation.  _Computers in Biology and Medicine_ , 164:107228, 2023.
  * Shaker et al. [2022] Abdelrahman Shaker, Muhammad Maaz, Hanoona Rasheed, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan.  Unetr++: delving into efficient and accurate 3d medical image segmentation.  _arXiv preprint arXiv:2212.04497_ , 2022.
  * Guo et al. [2023] Meng-Hao Guo, Cheng-Ze Lu, Zheng-Ning Liu, Ming-Ming Cheng, and Shi-Min Hu.  Visual attention network.  _Computational Visual Media_ , 9(4):733–752, 2023.
  * Azad et al. [2024] Reza Azad, Leon Niggemeier, Michael Hüttemann, Amirhossein Kazerouni, Ehsan Khodapanah Aghdam, Yury Velichko, Ulas Bagci, and Dorit Merhof.  Beyond self-attention: Deformable large kernel attention for medical image segmentation.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 1287–1297, 2024.
  * Rahman et al. [2024] Md Motiur Rahman, Shiva Shokouhmand, Smriti Bhatt, and Miad Faezipour.  Mist: Medical image segmentation transformer with convolutional attention mixing (cam) decoder.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 404–413, 2024.
  * Liu et al. [2024] Yajun Liu, Zenghui Zhang, Jiang Yue, and Weiwei Guo.  Scanext: Enhancing 3d medical image segmentation with dual attention network and depth-wise convolution.  _Heliyon_ , 2024.
  * Wang et al. [2023a] Jing Wang, Haiyue Zhao, Wei Liang, Shuyu Wang, and Yan Zhang.  Cross-convolutional transformer for automated multi-organs segmentation in a variety of medical images.  _Physics in Medicine & Biology_, 68(3):035008, 2023a.
  * Hong et al. [2023] Zhifang Hong, Mingzhi Chen, Weijie Hu, Shiyu Yan, Aiping Qu, Lingna Chen, and Junxi Chen.  Dual encoder network with transformer-cnn for multi-organ segmentation.  _Medical & Biological Engineering & Computing_, 61(3):661–671, 2023.
  * Li et al. [2023] Jiao Li, Jinyu Ye, Hongxia Deng, and Hong Shi.  Cpftransformer: transformer fusion context pyramid medical image segmentation network.  _Frontiers in Neuroscience_ , 17:1288366, 2023.
  * Landman et al. [2015] Bennett Landman, Zhoubing Xu, J Igelsias, Martin Styner, T Langerak, and Arno Klein.  Miccai multi-atlas labeling beyond the cranial vault–workshop and challenge.  In _Proc. MICCAI Multi-Atlas Labeling Beyond Cranial Vault—Workshop Challenge_ , volume 5, page 12, 2015.
  * Isensee et al. [2021] Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier-Hein.  nnu-net: a self-configuring method for deep learning-based biomedical image segmentation.  _Nature methods_ , 18(2):203–211, 2021.
  * Bernard et al. [2018] Olivier Bernard, Alain Lalande, Clement Zotti, Frederick Cervenansky, Xin Yang, Pheng-Ann Heng, Irem Cetin, Karim Lekadir, Oscar Camara, Miguel Angel Gonzalez Ballester, et al.  Deep learning techniques for automatic mri cardiac multi-structures segmentation and diagnosis: is the problem solved?  _IEEE transactions on medical imaging_ , 37(11):2514–2525, 2018.
  * Xiong et al. [2021] Zhaohan Xiong, Qing Xia, Zhiqiang Hu, Ning Huang, Cheng Bian, Yefeng Zheng, Sulaiman Vesal, Nishant Ravikumar, Andreas Maier, Xin Yang, et al.  A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging.  _Medical image analysis_ , 67:101832, 2021.
  * Luo et al. [2021] Xiangde Luo, Jieneng Chen, Tao Song, and Guotai Wang.  Semi-supervised medical image segmentation through dual-task consistency.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 35, pages 8801–8809, 2021.
  * Wang et al. [2023b] Yongchao Wang, Bin Xiao, Xiuli Bi, Weisheng Li, and Xinbo Gao.  Mcf: Mutual correction framework for semi-supervised medical image segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 15651–15660, 2023b.
  * Cardoso et al. [2022] M Jorge Cardoso, Wenqi Li, Richard Brown, Nic Ma, Eric Kerfoot, Yiheng Wang, Benjamin Murrey, Andriy Myronenko, Can Zhao, Dong Yang, et al.  Monai: An open-source framework for deep learning in healthcare.  _arXiv preprint arXiv:2211.02701_ , 2022.
  * Roth et al. [2015] Holger R Roth, Le Lu, Amal Farag, Hoo-Chang Shin, Jiamin Liu, Evrim B Turkbey, and Ronald M Summers.  Deeporgan: Multi-level deep convolutional networks for automated pancreas segmentation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part I 18_ , pages 556–564. Springer, 2015.
  * Bakas et al. [2017] Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin S Kirby, John B Freymann, Keyvan Farahani, and Christos Davatzikos.  Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features.  _Scientific data_ , 4(1):1–13, 2017.
  * Menze et al. [2014] Bjoern H Menze, Andras Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, Yuliya Burren, Nicole Porz, Johannes Slotboom, Roland Wiest, et al.  The multimodal brain tumor image segmentation benchmark (brats).  _IEEE transactions on medical imaging_ , 34(10):1993–2024, 2014.
  * Dorent et al. [2019] Reuben Dorent, Samuel Joutard, Marc Modat, Sébastien Ourselin, and Tom Vercauteren.  Hetero-modal variational encoder-decoder for joint modality completion and segmentation.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22_ , pages 74–82. Springer, 2019.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in neural information processing systems_ , 32, 2019.
  * Falcon and The PyTorch Lightning team [2019] William Falcon and The PyTorch Lightning team.  PyTorch Lightning, March 2019.  URL https://github.com/Lightning-AI/lightning.
  * Taha and Hanbury [2015] Abdel Aziz Taha and Allan Hanbury.  Metrics for evaluating 3d medical image segmentation: analysis, selection, and tool.  _BMC medical imaging_ , 15:1–28, 2015.
  * Fu et al. [2024] Liyao Fu, Yunzhu Chen, Wei Ji, and Feng Yang.  Sstrans-net: Smart swin transformer network for medical image segmentation.  _Biomedical Signal Processing and Control_ , 91:106071, 2024.
  * Azad et al. [2023c] Reza Azad, Amirhossein Kazerouni, Alaa Sulaiman, Afshin Bozorgpour, Ehsan Khodapanah Aghdam, Abin Jose, and Dorit Merhof.  Unlocking fine-grained details with wavelet-based high-frequency enhancement in transformers.  In _International Workshop on Machine Learning in Medical Imaging_ , pages 207–216. Springer, 2023c.
  * Hatamizadeh et al. [2021] Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger R Roth, and Daguang Xu.  Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images.  In _International MICCAI Brainlesion Workshop_ , pages 272–284. Springer, 2021.
  * Luo et al. [2022] Yixi Luo, Huayi Yin, and Xia Du.  Moaformer: Aggregating adjacent window features into local vision transformer using overlapped attention mechanism for volumetric medical segmentation.  In _Proceedings of the 2022 11th International Conference on Computing and Pattern Recognition_ , pages 121–127, 2022.
  * Ao et al. [2024] Yu Ao, Weili Shi, Bai Ji, Yu Miao, Wei He, and Zhengang Jiang.  Ms-tcnet: An effective transformer–cnn combined network using multi-scale feature learning for 3d medical image segmentation.  _Computers in Biology and Medicine_ , page 108057, 2024.
  * Huang et al. [2024] De-Xing Huang, Xiao-Hu Zhou, Xiao-Liang Xie, Shi-Qi Liu, Zhen-Qiu Feng, Mei-Jiang Gui, Hao Li, Tian-Yu Xiang, Xiu-Ling Liu, and Zeng-Guang Hou.  Mosformer: Momentum encoder-based inter-slice fusion transformer for medical image segmentation.  _arXiv preprint arXiv:2401.11856_ , 2024.
  * Fedorov et al. [2012] Andriy Fedorov, Reinhard Beichel, Jayashree Kalpathy-Cramer, Julien Finet, Jean-Christophe Fillion-Robin, Sonia Pujol, Christian Bauer, Dominique Jennings, Fiona Fennessy, Milan Sonka, John Buatti, Stephen Aylward, James Miller, Steve Pieper, and Ron Kikinis.  3D Slicer as an Image Computing Platform for the Quantitative Imaging Network.  _Magnetic Resonance Imaging_ , 30(9):1323–1341, November 2012.  doi:10.1016/j.mri.2012.05.001.
  * Yang et al. [2023] Weibin Yang, Longwei Xu, Pengwei Wang, Dehua Geng, Yusong Li, Mingyuan Xu, and Zhiqi Dong.  More complex encoder is not all you need.  _arXiv preprint arXiv:2309.11139_ , 2023.
  * Wu et al. [2023] Yixuan Wu, Kuanlun Liao, Jintai Chen, Jinhong Wang, Danny Z Chen, Honghao Gao, and Jian Wu.  D-former: A u-shaped dilated transformer for 3d medical image segmentation.  _Neural Computing and Applications_ , 35(2):1931–1944, 2023.
  * He et al. [2023] Yufan He, Vishwesh Nath, Dong Yang, Yucheng Tang, Andriy Myronenko, and Daguang Xu.  Swinunetr-v2: Stronger swin transformers with stagewise convolutions for 3d medical image segmentation.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 416–426. Springer, 2023.
  * Lee et al. [2017] Kisuk Lee, Jonathan Zung, Peter Li, Viren Jain, and H Sebastian Seung.  Superhuman accuracy on the snemi3d connectomics challenge.  _arXiv preprint arXiv:1706.00120_ , 2017.
  * Toubal et al. [2020] Imad Eddine Toubal, Ye Duan, and Deshan Yang.  Deep learning semantic segmentation for high-resolution medical volumes.  In _2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR)_ , pages 1–9. IEEE, 2020.
  * Çiçek et al. [2016] Özgün Çiçek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger.  3d u-net: learning dense volumetric segmentation from sparse annotation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19_ , pages 424–432. Springer, 2016.