# U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic
Segmentation

Bingyu Li${}^{\textbf{\textdagger}}$, Da Zhang${}^{\textbf{\textdagger}}$,
Zhiyuan Zhao, Junyu Gao, and Xuelong Li∗ † : Equal Contribution;
*Corresponding author: Xuelong Li.Bingyu Li, Zhiyuan Zhao and Xuelong Li are
with the Institute of Artificial Intelligence (TeleAI), China Telecom, P. R.
China. (E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>xuelong_li@ieee.org).Da Zhang and Junyu Gao are with the School of Artificial
Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical
University, Xi’an 710072, China and with the Institute of Artificial
Intelligence (TeleAI), China Telecom, P. R. China. (E-mail:
<EMAIL_ADDRESS>gjy3035@gmail.com).

###### Abstract

Multimodal semantic segmentation is a pivotal component of computer vision and
typically surpasses unimodal methods by utilizing rich information set from
various sources. Current models frequently adopt modality-specific frameworks
that inherently biases toward certain modalities. Although these biases might
be advantageous in specific situations, they generally limit the adaptability
of the models across different multimodal contexts, thereby potentially
impairing performance. To address this issue, we leverage the inherent
capabilities of the model itself to discover the optimal equilibrium in
multimodal fusion and introduce U3M: An Unbiased Multiscale Modal Fusion Model
for Multimodal Semantic Segmentation. Specifically, this method involves an
unbiased integration of multimodal visual data. Additionally, we employ
feature fusion at multiple scales to ensure the effective extraction and
integration of both global and local features. Experimental results
demonstrate that our approach achieves superior performance across multiple
datasets, verifing its efficacy in enhancing the robustness and versatility of
semantic segmentation in diverse settings. Our code is available at
U3M-multimodal-semantic-segmentation.

###### Index Terms:

Semantic Segmentation, Multi-Modality, Multi-Scale Fusion, Unbiased Modality
Fusion

##  I Introduction

Semantic segmentation [1, 2, 3] is a crucial task within the field of computer
vision, with applications spanning various domains such as scene understanding
[4, 5, 6, 7], autonomous driving [8, 9], etc. RGB-based semantic segmentation
(depicted Fig. 1(a)), serving as a foundational task, is suitable for the
analysis of most scenes and has been extensively explored by the research
community, yielding many impressive works and experimental results [10, 11,
12]. However, employing only RGB channels to segment certain complex and
special scenes presents challenges, particularly where RGB information is
elusive [13, 5]. In contrast to RGB cameras, which depend on visible light and
often falter in darkness, thermal infrared (TIR) sensors directly detect heat
emissions from objects, offering substantial contrast in the absence of light
[9, 7]. Additionally, other modalities such as depth [6, 14] and LiDAR [15,
16] can also provide additional visual semantic information and are
increasingly being integrated into semantic segmentation efforts, as
illustrated in Fig. 1(b).

Figure 1: The Evolution of Multimodal Semantic Segmentation Model
Architectures. (a) Training a feature extractor using only RGB images. (b)
Sharing a trainable feature extractor between RGB images and other modalities.
(c) Up: Sharing a fine-tunable pre-trained feature extractor between RGB
images and other modalities. Down: Fine-tuning adapters for different
modalities with one frozen feature extrator. (d) Up: Single-scale feature
fusion within the model. Down: Multiscale feature fusion.

In the context of multimodal semantic segmentation, multimodal feature
extraction, serving as the foundation for downstream tasks, has been
extensively explored in numerous studies [1, 10]. Although these models have
demonstrated exceptional performance in certain scenarios and datasets, they
are typically trained from scratch on specific small-scale datasets, the
limited number of images and scenes in multimodal visual training datasets
leads to poor generalization across different scenes and data, not to mention
better feature extraction capabilities. In contrast to aforementioned models
trained from scratch, recent pretrained models, mostly based on Convolutional
Neural Network (CNN) [17] and Transformer[18], have shown strong performance
on downstream tasks by building models with large-scale parameters and
pretraining on vast amounts of visual images. The rise of large-scale visual
pretraining models has led to a series of studies exploring those applications
in semantic segmentation. For instance, a large-scale pretrained convolutional
neural network[17], demonstrated excellent semantic segmentation capabilities
and performance by pretraining on ImageNet. Building on this well-established
pretrained model, [14] introduced ACNet, which refines feature extraction
through asymmetric convolution blocks, enhancing network accuracy and
robustness. Compared with CNN-based pretrained models, Transformers offer a
larger receptive field [18] and stronger global modeling capabilities. Recent
research [19, 20] using plain vision Transformers for information extraction
struggles with multiscale semantic information. To address this, Transformers
with moving and multiscale windows have been explored [21, 22]. Multiscale
information extractors have proven more practical. To balance speed and
accuracy, we use the hierarchical multiscale vision Transformer Segformer [3]
as the modal information extractor.

Although extracted by large-scale pretrained models with strong visual
information extraction capabilities, visual multimodal semantic information
still presents some domain gaps. To bridge this, novel fine-tuning strategies
using prompting techniques have been proposed [23], efficiently adapting
pretrained models to multimodal modes with minimal updates. Another approach
involves using pretrained models as modality extractors, followed by fine-
tuning with limited multimodal data, as shown in Fig. 1(c). However, this
often leads to catastrophic forgetting. Therefore, designing an additional
feature fusion module is receiving increasing attention. For example, [8]
designed an additional dual attention network to complement the pretrained
feature extractor at the only low-resolution feature map (Fig. 1(d)), which
does not affect the original pretrained model. However, using single-scale
information presents problems such as weak local spatial information and low
input resolution. To address these issues, FuseNet [6] integrates multiscale
RGB-Depth multimodal features at every encoder stage, effectively improving
performance on the SUN RGB-D benchmark. Similarly, CACFNet [7] incorporates a
cross-modal attention fusion module that extracts multiscale RGB-thermal
information. Besides, lots of extensive research [15, 24] have demonstrated
multiscale modal fusion modules (as shown in Fig. 1(d)) enhance the adaptive
integration of multimodal.

Despite the excellent results achieved by existing work, particularly [13,
24], most of the multimodal fusions discussed above, notably CMNEXT [15],
exhibit some modalities bias. Specifically, these methods typically prioritize
one modality as dominant and treat others as auxiliary (generally in the
format of RGB+Xs) [25]. However, these approaches overlook the dynamic
dominant correlation within multimodal data [25, 26], which hampers the
ability to fully utilize complementary multimodal information in complex
scenarios (as illustrated in Fig.2), thus limiting performance.

To avoid the inherent modal bias in model design and facilitate the fusion of
multi-scale modal information, we first introduce an Unbiased Multiscale Modal
Fusion Model (U3M) by treating all modalities equitably and performing
multiscale fusion, which is different from the existing technology. This
strategy enables the model to autonomously generate modal preferences
applicable to various segmentation scenarios. Secondly, based on the fusion
paradigm, we develop two multiscale fusion modules utilizing multiscale
pooling and convolution, which effectively integrate and fuse global and local
information across different scales for multimodal information. Finally, the
model’s efficacy is validated by outstanding results across multiple datasets.
Our contributions are summarized as follows:

  * •

We demonstrate the presence of modal bias in model design and develop an
unbiased modal fusion methodology. This approach leverages the inherent
properties of the model to autonomously generate modality preferences,
substantially reducing biases introduced by manual design interventions.

  * •

We design a multiscale model fusion layer that incorporates multiscale
convolution and pooling. In this way, the fusion layer enhances the capability
for multiscale modal fusion throughout various feature encoding stages.

  * •

Comprehensive experiments are conducted on two challenging multimodel semantic
segmentation benchmarks (i.e., FMB [9], Mcubes [16]), which further
demonstrates that our method achieves state-of-the-art performance.

Figure 2: The dynamic dominant correlation of multimodal data in different
scenes. Left: Under conditions of insufficient light, infrared images can
capture more intricate details than RGB images. Middle: In outdoor situations
where light is abundant, infrared images tend to lose more details, while RGB
images showcase their superiority. Right: In certain common instances, the
detailed information in infrared images and RGB images can serve as a
complement to each other.

##  II Related Works

###  II-A Semantic Segmentation

The field of semantic segmentation has seen significant advancements,
particularly with the advent of fully convolutional networks that
revolutionized pixel classification [10]. Based on this, some improvements
such as multi-scale feature extraction and fusion have proven effective [1,
27]. For instance, Chen et al. further developed DeepLab by integrating an
encoder-decoder architecture for efficient multi-scale context aggregation
[2]. Similarly, Zhao et al. introduced a pyramid pooling module to aggregate
context from varied regions at multiple scales [12].

Channel and self-attention mechanisms have evolved to capture more global
semantic information. Lin et al. developed RefineNet, employing multi-path
refinement networks with channel attention for high-resolution segmentation
[28]. Choi et al. proposed CARS, emphasizing channel-wise attention for
region-based segmentation [29]. Another paradigm, Context-based refinement
algorithms, integrates extensive background contextual information [11, 30].
Some other context-based models leverage context to refine segmentation
through adaptive feature recalibration [31], while [32] developed OCNet to
enhance semantic understanding by aggregating object context.

Edge detection techniques also serve as complementary cues for semantic
segmentation. For example, Li et al. improved edge detection using deep
learning [33], and Borse et al. proposed InverseForm to enhance edge detection
accuracy in complex images [34]. The recent adoption of vision transformers
for recognition tasks has led to the development of dense prediction
transformers specialized for semantic segmentation. Notable examples include
CSWin Transformer, which captures long-range dependencies using cross-shaped
windows [21], and HRFormer, which integrates high-resolution representations
for detailed segmentation [22]. These advancements have facilitated the
segmentation of discrete objects and amorphous regions [35, 36], with
transformers now incorporating token mixing via attention mechanisms [37],
multi-layer perceptron elements [38, 39], and pooling and convolutional blocks
[40].

Despite setting new benchmarks in image segmentation, challenges persist,
particularly under real-life conditions where RGB images are inadequate, such
as in low-light environments or when capturing fast-moving subjects.
Consequently, multimodal semantic segmentation is garnering increasing
attention.

###  II-B Multimodality Semantic Segmentation

Multimodal semantic segmentation is increasingly recognized for its ability to
integrate diverse modal data, effectively compensating for the inherent
limitations of each modality [15, 13, 16]. Zhou et al. [41] integrated edge-
aware features for enhanced semantic segmentation of RGB and thermal images,
improving object boundary delineation in multimodal scenarios. Deng et al.
[42] incorporated an attention mechanism that boosts feature representation
for real-time RGB-thermal semantic segmentation. Zhao et al. [43] utilized
deep image decomposition to fuse infrared and visible images, preserving
critical features from both modalities. Huang et al. [44] employed a recurrent
network to iteratively refine multi-modality image fusion, enhancing detail
preservation and reducing artifacts. Additionally, in RGB-depth modality
fusion, ACNet [14] employs an attention mechanism to optimize the usage of RGB
and depth data for improved semantic segmentation accuracy, especially in
scenarios with complex visibility. FuseNet [6] integrates RGB and depth data
using a dual-stream CNN, leveraging depth as an auxiliary input to enhance
segmentation accuracy. The evolution of model frameworks has progressed from
those based on CNN [1, 2] to those founded on Transformers [3, 40, 21]. This
transition facilitates a more nuanced analysis of the interplay between global
semantics and local features, enhancing feature extraction. For modal fusion,
some techniques employ attention mechanisms to integrate different modalities
[14]. For example, CACFNet [7] utilizes cross-attention mechanisms to
selectively enhance the integration of contextual information from different
modalities to improve semantic correlation and feature extraction efficiency.
Other approaches employ convolution as a feature fusion extraction module
[45]; Reza et al. [26] introduced specialized convolution layers designed to
process and merge information from multiple modalities effectively. EGFNet [5]
uses gated convolution to selectively fuse the most relevant features from
diverse modalities, improving fusion effectiveness. Apart from that, Zhang et
al. [15] innovated by adapting pooling strategies to multimodal contexts,
optimizing feature reduction and abstraction processes to better accommodate
the diverse characteristics of different data types. Despite these
advancements, many existing modal fusion models rely excessively on RGB
images, often involving specialized feature extractors for the RGB channels.
This strategy risks neglecting the variable significance of different
modalities across various scenarios. To address this, we propose the Unbiased
Multiscale Modal Fusion Model for Multimodal Semantic Segmentation.

##  III Method

Figure 3: Unbiased Multiscale Modal Fusion Model. Utilizing Segformer [3] with
frozen parameters as the feature extractor. Each modality’s information is fed
into respective feature extractors, divided into four distinct scales for
unbiased fusion of multiscale information. Each feature fusion layer comprises
two modules based on multiscale pooling and convolution, adaptively extracting
features with varied scales. In the end, the multiscale information is
concatenated and fed into a shared semantic segmentation head to generate
segmentation results.

In this section, we first present the problem definition, followed by a
detailed presentation of the proposed U3M.

###  III-A Problem Definition

Multimodal semantic segmentation seeks to assign a semantic label to each
pixel in an image from predefined categories, such as ’car’, ’tree’, or
’road’. Distinct from traditional semantic segmentation which depends on a
single modality, typically RGB images, multimodal semantic segmentation
leverages data from various sources or sensors to improve the accuracy and
robustness of segmentation.

Given a set $\mathcal{I}=\\{I_{1},I_{2},\dots,I_{M}\\}$ of images, where
$I_{m}\in\mathbb{R}^{H\times W\times C_{m}}$ represents the $m$-th modality
out of $M$ total modalities. Here, $H$, $W$, and $C_{m}$ denote the height,
width, and channel size of the image, respectively. The objective of
multimodal semantic segmentation is to compute a segmentation map
$S\in\mathcal{P}^{H\times W}$. Here, $\mathcal{P}=\\{1,2,\dots,N\\}$ denotes
the set of $N$ semantic labels.

Each encoder ($\text{Enc}_{m}:\mathbb{R}^{H\times W\times
C_{m}}\rightarrow\mathbb{R}^{H^{\prime}\times W^{\prime}\times C_{f}}$)
processes the corresponding modality $I_{m}$, yielding feature maps $F^{m}$.
These are fused by $\text{Fusion}:(\mathbb{R}^{H^{\prime}\times
W^{\prime}\times C_{f}})^{M}\rightarrow\mathbb{R}^{H^{\prime}\times
W^{\prime}\times F^{\prime}}$ to produce a combined representation $F$
($C_{f}$=$F^{\prime}$ in this paper). Finally, $S$ is obtained through a
decoder $\text{Dec}:\mathbb{R}^{H^{\prime}\times W^{\prime}\times
F^{\prime}}\rightarrow\mathcal{P}^{H\times W}$.

The effectiveness of a multimodal semantic segmentation model is typically
evaluated based on its ability to accurately segment objects and scenes under
varying conditions, making use of the additional information provided by the
various modalities to overcome the limitations of single-modality
segmentation.

###  III-B Overall model Architecture

The schema of our model’s architecture is presented in Fig. 3 accommodating
$M$ discrete modalities. The architecture’s sophistication lies in its
utilization of modality-specific encoders, which are tailored to distill
unique feature hierarchies from each modality, encapsulated by the equation:

$F_{m}=\text{Enc}_{m}(I_{m}),$ (1)

where $I_{m}\in\mathbb{R}^{H\times W\times C_{m}}$ represents the modality-
specific input imagery for $m\in\\{1,2,\ldots,M\\}$, and
$\text{Enc}_{m}(\cdot)$ is the corresponding encoder. These encoders are adept
at generating a spectrum of feature maps at diminished resolutions—precisely,
$\frac{1}{4},\frac{1}{8},\frac{1}{16},\frac{1}{32}$ of the initial
resolution—collectively represented as
$F_{m}=\\{F_{m}^{1},F_{m}^{2},F_{m}^{3},F_{m}^{4}\\}$. For the sake of
brevity, the feature map’s dimensions at the $i$-th encoding stage are denoted
as $(H^{\prime}_{i}\times W^{\prime}_{i}\times C_{fi})$, with $i$ traversing
{1, 2, 3, 4}. A quartet of fusion blocks, each allied to a corresponding
encoder stage, is orchestrated to merge the features from each encoding
cascade. The prominent features $F^{i}_{m}$ from every modality are
assimilated within the $i$-th fusion module:

$F^{i}=\text{FusionBlock}^{i}(\\{F^{i}_{m}\\}_{m}).$ (2)

This integrative process yields a synthesized feature composite
$F=\\{F^{1},F^{2},F^{3},F^{4}\\}$, with $F^{i}$ signifying the fused feature
at the $i$th stage. Culminating the process, these aggregated features $F$ are
input into an MLP decoder, as elucidated in [3], to extrapolate the
segmentation contours.

###  III-C Modality Feature Encoder

Utilizing a mix transformer encoder as referenced in [3], our system
effectively extracts hierarchical features from various input data types.
Every image $I_{m}$ undergoes a patch embedding process, getting segmented
into $4\times 4$ patches as per the method described in [3], before being
processed by the mixed transformer encoder units. Illustrated in Fig. 3 is the
mix transformer unit’s structure, with $X^{i}_{m}$ denoted the $m$-th modal
input for the $i$-th mix transformer, which has the shape of
$\mathbb{R}^{H_{i}\times W_{i}\times C_{i}}$. This input is then reshape into
a $N_{i}\times C_{i}$ matrix, where $N_{i}$ equates to $H_{i}\times W_{i}$,
for utilization as the query $Q$, the key $K$, and the value $V$. In an effort
to curtail computational demands, we employ spatial reduction as suggested by
[3], leveraging a reduction ratio $R$. The matrices $K$ and $V$ undergo an
initial transformation into $\frac{N_{i}}{R}\times C_{i}$ matrices, followed
by a remapping into $\frac{N_{i}}{R}\times C_{i}$ matrices via a linear
transformation process. Subsequently, a conventional Multi-Head Self-attention
Mechanism (MHSA) is employed to map $Q,K,V$ into intermediate representations
as delineated by:

$MHSA(Q,K,V)=\text{Concatenate}(head_{1},\ldots,head_{h})W^{O},$ (3)
$head_{j}=\text{Attention}(QW_{j}^{Q},KW_{j}^{K},VW_{j}^{V}).$ (4)

In this context, $h$ signifies the total number of attention heads, with
$W_{j}^{Q}$, $W_{j}^{K}$, $W_{j}^{V}$, and $W^{O}$ serving as the respective
projection matrices within the spaces $\mathbb{R}^{C_{i}\times d_{k}}$ and
$\mathbb{R}^{hd_{v}\times C_{i}}$, and $d_{k}$, $d_{v}$ denoting the
dimensions of $K$, and $V$. The Attention function is characterized as:

$\text{Attention}(Q,K,V)=\text{Softmax}\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V.$
(5)

Here, $Q,K$, and $V$ correspond to the input query, key, and value matrices,
respectively. This MHSA phase is succeeded by a mixing layer composed of two
MLPs and a $3\times 3$ convolutional layer, which provides the necessary
positional encoding within the transformer encoder to maximize segmentation
efficacy as noted in [3]. The computation within this layer is expressed as:

${X}_{\text{in}}=MHSA(Q,K,V),$ (6)
$X_{\text{out}}=\text{MLP}(\text{GELU}(\text{Conv}_{3\times
3}(\text{MLP}({X}_{\text{in}}))))+{X}_{\text{in}}.$ (7)

To conclude, an overlapping patch merge technique is applied to
$X_{\text{out}}$ follow [3], culminating in the synthesis of the ultimate
output.

###  III-D Pyramidal Multiscale Modal Fusion Layer

Feature fusion post-hierarchical extraction is performed via a designated
fusion block depicted in Fig. 4. This block integrates features from modality-
specific encoders across all four stages. Considering $F^{i}_{m}$ as the input
features for the $i$th block, where $F^{i}_{m}\in\mathbb{R}^{H_{i}\times
W_{i}\times C_{i}}$, sourced from $m$th modality, we first merge these along
the channel axis, obtaining $F^{\prime i}$. Subsequent reduction and
combination of channels is achieved through a linear layer, outputting
$\tilde{F}^{i}$ with a reduced channel count of $C_{i}$. This process is
mathematically formulated as:

$\tilde{F}^{i}=\text{Linear}(\big{\|}_{m=1}^{M}F^{i}_{m}).$ (8)

In this equation, $\big{\|}$ signifies the concatenation of the modal features
within the channel space, scaling down an $MC_{i}$-dimensional input to a
$C_{i}$-dimensional output.

Figure 4: Multiscale Feature Fusion Module. To enhance the extraction of
information across multiple scales, a multiscale feature extractor is
proposed.

To enhance the extraction of information across multiple scales, a multiscale
feature extractor is proposed. Specifically, we have designed two modules: the
Pyramidal Convolution-Based Multimodal Information Fusion Layer and the
Pyramidal Pooling-Based Multimodal Information Fusion Layer. They respectively
utilize convolution and pooling operations across various dimensions to
discern information of different granularities within the multimodal fusion
features.

1) Pyramidal Pooling-Based Multimodal Information Fusion Layer: The $i$th
linear fusion feature $\tilde{F}^{i}$ is refined using a multiscale feature
fusion module. This unit is structured with a pair of convolutional projection
layers, enclosing average pooling operations, to facilitate dimensional
interlacing. Employing convolutions of sizes $1\times 1$, $2\times 2$,
$3\times 3$, and $6\times 6$, the module adeptly captures and consolidates
features across scales, enhancing them with residual data. Sandwiching the
average pooling, the dual projection layers are instrumental in the
comprehensive integration of the feature landscape.

$F_{pooling}^{i}=\text{Conv}_{1\times 1}(\tilde{F}^{i}),$ (9)
$F_{pooling}^{i,k}=\text{Conv}_{1\times 1}(\text{AvgPooling}_{k\times
k}(F_{pooling}^{i})),$ (10)
$F_{pooling}^{i}=\sum_{k\in\\{1,2,5,6\\}}\text{Upsample}(F_{pooling}^{i,k}),$
(11) $F_{pooling}^{i}=\text{Conv}_{1\times 1}(F_{pooling}^{i}).$ (12)

2) Pyramidal Convolution-Based Multimodal Information Fusion Layer: Similar to
the pooling fusion layer, the enhancement of the $i$th linear fusion feature
$\tilde{F}^{i}$ is conducted through a dedicated multiscale feature fusion
apparatus. This configuration entails a sequence of convolutional projection
layers that enwrap average pooling operations, thereby facilitating
sophisticated feature interweaving. With convolutional kernels of dimensions
$3\times 3$, $5\times 5$, and $7\times 7$, analogous to those employed by
[26],

Figure 5: Multiscale pooling and convolution. Pooling and convolution at
different scales are capable of capturing local and global features across
multiple levels, thereby complementing the global attention mechanisms
integrated within the backbone architecture effectively. This approach ensures
that the resultant fused features encompass a comprehensive focus on both
local and global dimensions.

$F_{conv}^{i}=\text{Conv}_{1\times 1}(\tilde{F}^{i}),$ (13)
$F_{conv}^{i,k}=\text{Conv}_{k\times k}(F_{conv}^{i}),$ (14)
$F_{conv}^{i}=\sum_{k\in\\{3,5,7\\}}(F^{i}+F_{conv}^{i,k}),$ (15)
$F_{conv}^{i}=\text{Conv}_{1\times 1}(F_{conv}^{i}).$ (16)

The two varieties of fine-grained features extracted are fused via addition,

$F_{fusion}^{i}=F_{Pooling}^{i}+F_{conv}^{i}.$ (17)

Subsequently, $F_{fusion}^{i}$ passes through a linear layer and a Channel
Attention Mechanism (CA) [46] for feature refinement.

###  III-E Shared Segmentation Head

The fused features generated from all the 4 fusion blocks are sent to the
shared MLP decoder. We use the decoder design proposed in [3]. The
segmentation head shown in Fig. 3 can be represented as the following
equations:

$\hat{F}^{i}=\text{Linear}(F_{out}^{i}),\quad\forall i\in\\{1,2,3,4\\}$ (18)
$\hat{F}^{i}=\text{Upsample}(\hat{F}^{i}),\quad\forall i\in\\{1,2,3,4\\}$ (19)
$F=\text{Linear}(\hat{F}^{1}\parallel\ldots\parallel\hat{F}^{i}),$ (20)
$P=\text{Linear}(F).$ (21)

The first linear layers take the fused features of different shapes and
generate features having the same channel dimension. Then the features are up-
sampled to $\frac{1}{4}$ of the original input shape, concatenated along the
channel dimension and passed through another linear layer to generate the
final fused feature $F^{\prime}$. Finally, $F$ is passed through the last
linear layer to generate the predicted segmentation map $P$.

##  IV Experiments

###  IV-A Experimental Setup and Parameters

Figure 6: Visualization on FMB dataset. This figure presents a comprehensive
visualization of semantic segmentation results on the FMB dataset, juxtaposing
RGB images with their thermal counterparts and the corresponding segmentation
outcomes from two different methods: Results with RGB and Results with RGB-T.
This figure exemplifies the pivotal role of integrating multimodal data in
enhancing segmentation accuracy under varied environmental and lighting
conditions.

Experiment Configuration. All experiments were conducted on a computational
platform equipped with 4 NVIDIA GeForce RTX 4090 GPUs. To ensure the
reproducibility of results, the experiments were performed under consistent
hardware and software configurations.

Parameters. Our model utilized uniform training parameters across all
datasets. The learning rate was initially set at $6\times 10^{-5}$, with the
Adam optimizer used for adjustments. The batch size was established at 4, with
a total training epoch of 400 epochs for the Mcubes dataset and 120 for the
FMB dataset. Cross-entropy loss function was employed. Data augmentation
techniques, including random rotations, scaling, and horizontal flipping, were
applied to enhance the model’s generalization capabilities.

###  IV-B Dataset

Mcubes. [16] The MCubeS dataset contains sets of RGB, Near-Infrared (NIR),
Degree of Linear Polarization (DoLP), and Angle of Linear Polarization (AoLP)
pairs. It is designed for researching semantic material segmentation across 20
categories. The dataset comprises 302/96/102 image pairs allocated for
training/validation/testing, all sized at 1224×1024.

FMB. [9] The FMB dataset is a comprehensive multi-modality benchmark designed
for image fusion and segmentation. It contains 1,500 well-registered pairs of
infrared and visible images, annotated with 15 pixel-level categories. The
training and test set contains 1220 and 280 image pairs respectively. These
images encompass a variety of real-world conditions such as dense fog and low-
light scenarios, making them particularly suitable for autonomous driving and
semantic understanding applications. The dataset aims to improve the
generalization capabilities of fusion and segmentation models across diverse
environmental conditions.

###  IV-C Experimental Results

TABLE I: Performance comparison on Multimodal Material Segmentation (MCubeS)
dataset [16]. Here A, D, and N represent angle of linear polarization (AoLP),
degree of linear polarization (DoLP), and near-infrared (NIR) respectively.

Method | Modalities | % mIoU
---|---|---
DRConv [47] | RGB-A-D-N | 34.63
DDF [48] | RGB-A-D-N | 36.16
TransFuser [49] | RGB-A-D-N | 37.66
DeepLabv3+ [2] | RGB-A-D-N | 38.13
MMTM [50] | RGB-A-D-N | 39.71
FuseNet [6] | RGB-A-D-N | 40.58
MCubeSNet [16] | RGB-A-D-N | 42.46
CMNeXt [15] | RGB-A-D-N | 51.54
U3M (Ours) | RGB-A-D-N | 51.69

1) Results on Mcubes Dataset. Table I compares different methods based on mean
Intersection over Union (mIoU) percentage, which is a common metric for
evaluating the accuracy of segmentation models. The methods include various
modalities, specifically RGB-A-D-N, where A, D, and N represent angle of
linear polarization (AoLP), degree of linear polarization (DoLP), and near-
infrared (NIR) respectively. Our model achieves the highest mIoU of 51.69,
significantly outperforming all other listed methods. This suggests that U3M
is particularly effective at handling multimodal inputs to provide superior
segmentation accuracy. The closest competitor, CMNeXt, achieves a mIoU of
51.54, making our model’s lead relatively narrow but still notable. The
superior performance of U3M suggests its potential applicability in real-world
scenarios where precise material identification is crucial, such as in
autonomous driving environments or quality control in manufacturing.

The mIoU results of each category from the MCubeS dataset provide a nuanced
view of the segmentation performance across diverse material categories, as
shown in Table II. Our proposed model demonstrates robust performance, often
surpassing state-of-the-art models in per-class IoU metrics. Notably, our
model achieves superior results in classes such as ’Leaf’ (76.4), ’Water’
(63.8), and ’Cobblestone’ (73.5), indicating its proficiency in handling
complex textures and varied lighting conditions that these materials present.
However, some categories like ’Human’ (12.2) and ’Plastic’ (26.3) exhibit
weaker performance, suggesting potential areas for further model refinement.
This underperformance could be attributed to the challenges associated with
the high variability in human appearances and the often subtle differences in
plastic materials’ visual characteristics under different conditions.
Comparatively, our model’s performance in the ’Rubber’ (71.7) and ’Leaf’
(76.4) categories is particularly noteworthy, underscoring its effectiveness
in segmenting materials with distinct textural properties. The overall mean
IoU of 51.7 places our model competitively within the landscape of current
methods, closely following the CMNeXt model, which exhibits a slightly lower
mean IoU of 51.5.

TABLE II: Per-class % IoU comparison on MCubeS dataset. Our proposed model
shows better performance in detecting most of the classes compared to the
current state-of-the-art models. * indicates that the code and pretrained
model from the authors were used to generate the results.

Methods | Asphalt | Concrete | Metal | Road marking | Fabric | Glass | Plaster | Plastic | Rubber | Sand | Gravel | Ceramic | Cobblestone | Brick | Grass | Wood | Leaf | Water | Human | Sky | Mean
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
MCubeSNet [16] | 85.7 | 42.6 | 47.0 | 59.2 | 12.5 | 44.3 | 3.0 | 10.6 | 12.7 | 66.8 | 67.1 | 27.8 | 65.8 | 36.8 | 54.8 | 39.4 | 73.0 | 13.3 | 0.0 | 94.8 | 42.9
CMNeXt [15]* | 84.3 | 44.9 | 53.9 | 74.5 | 32.3 | 54.0 | 0.8 | 28.3 | 29.7 | 67.7 | 66.5 | 27.7 | 68.5 | 42.9 | 58.7 | 49.7 | 75.4 | 55.7 | 18.9 | 96.5 | 51.5
(Ours) | 86.2 | 44.5 | 55.1 | 68.0 | 33.4 | 53.9 | 1.2 | 26.3 | 26.9 | 68.2 | 71.7 | 25.0 | 73.5 | 44.1 | 59.7 | 47.4 | 76.4 | 63.8 | 12.2 | 96.4 | 51.7

TABLE III: Performance comparison on FBM dataset [9]. We show performance for different methods from already published works. Methods | Modalities | % mIoU
---|---|---
GMNet [51] | RGB-Infrared | 49.2
LASNet [52] | RGB-Infrared | 42.5
EGFNet [41] | RGB-Infrared | 47.3
FEANet [42] | RGB-Infrared | 46.8
DIDFuse [43] | RGB-Infrared | 50.6
ReCoNet [44] | RGB-Infrared | 50.9
U2Fusion [53] | RGB-Infrared | 47.9
TarDAL [54] | RGB-Infrared | 48.1
SegMiF [9] | RGB-Infrared | 54.8
U3M (Ours) | RGB-Infrared | 60.8
TABLE IV: Per-class % IoU comparison on FMB [9] dataset for both RGB only and
RGB-infrared modalities. T-Lamp and T-Sign stand for Traffic Lamp and Traffic
Sign respectively. Our model outperforms all the methods for all the classes
except for the truck class.

Methods | Modalities | Car | Person | Truck | T-Lamp | T-Sign | Building | Vegetation | Pole | % mIoU
---|---|---|---|---|---|---|---|---|---|---
GMNet [51] | RGB-Infrared | 79.3 | 60.1 | 22.2 | 21.6 | 69.0 | 79.1 | 83.8 | 39.8 | 49.2
LASNet [52] | RGB-Infrared | 72.6 | 48.6 | 14.8 | 2.9 | 59.0 | 75.4 | 81.6 | 36.7 | 42.5
EGFNet [41] | RGB-Infrared | 77.4 | 63.0 | 17.1 | 25.2 | 66.6 | 77.2 | 83.5 | 41.5 | 47.3
FEANet [42] | RGB-Infrared | 73.9 | 60.7 | 32.3 | 13.5 | 55.6 | 79.4 | 81.2 | 36.8 | 46.8
DIDFuse [43] | RGB-Infrared | 77.7 | 64.4 | 28.8 | 29.2 | 64.4 | 78.4 | 82.4 | 41.8 | 50.6
ReCoNet [44] | RGB-Infrared | 75.9 | 65.8 | 14.9 | 34.7 | 66.6 | 79.2 | 81.3 | 44.9 | 50.9
U2Fusion [53] | RGB-Infrared | 76.6 | 61.9 | 14.4 | 6.8 | 68.9 | 78.8 | 82.2 | 42.2 | 47.9
TarDAL [54] | RGB-Infrared | 74.2 | 56.0 | 18.3 | 7.8 | 69.0 | 79.1 | 81.7 | 41.9 | 48.1
SegMiF [9] | RGB-Infrared | 78.3 | 65.4 | 18.8 | 6.5 | 64.8 | 78.0 | 85.0 | 49.8 | 54.8
(Ours) | RGB | 83.3 | 57.6 | 41.6 | 42.7 | 78.3 | 81.7 | 85.6 | 49.5 | 60.5
(Ours) | RGB-Infrared | 82.3 | 66.0 | 41.9 | 46.2 | 81.0 | 81.3 | 86.8 | 48.8 | 60.8

TABLE V: Performance evaluation (measured in % mIoU) on the Multimodal
Material Segmentation (MCubeS) dataset [16] across various modality pairings
is presented. The modalities A, D, and N correspond to angle of linear
polarization (AoLP), degree of linear polarization (DoLP), and near-infrared
(NIR), respectively.

Modalities | MCubeSNet [16] | CMNeXt [15] | (Ours)
---|---|---|---
RGB | 33.70 | 48.16 | 49.22
RGB-A | 39.10 | 48.42 | 49.89
RGB-A-D | 42.00 | 49.48 | 50.26
RGB-A-D-N | 42.86 | 51.54 | 51.69

TABLE VI: Performace comparison on Mcubes dataset with different combinations
of proposed modules [9].

Methods | Modalities | % mIoU
---|---|---
Linear Layer Only | RGB-ADN | 49.89
Linear Layer + ChannelAttention | RGB-ADN | 50.34
Linear Layer + PSPModule | RGB-ADN | 50.62
U3M | RGB-ADN | 51.69

2) Results on FMB Dataset. The results presented in Table III for the
performance comparison on the FBM dataset highlight the capabilities of
various semantic segmentation models utilizing RGB-Infrared modalities, a
combination critical for enhancing material differentiation under varying
illumination conditions. Notably, our model, U3M, achieves an impressive mIoU
score of 60.8, which surpasses all other models listed. This superior
performance can be attributed to the effective integration of RGB and infrared
data, which allows U3M to robustly capture and utilize the complementary
information provided by these modalities. Infrared imaging, known for its
utility in low-light conditions and its ability to differentiate materials
based on thermal properties, combined with the rich detail available in RGB
images, provides a more comprehensive understanding of the scene, enhancing
segmentation accuracy. The closest competitors, SegMif and TarDAL, achieve
mIoU scores of 54.8 and 48.1, respectively, which indicates that while these
models are effective, there remains a significant gap in performance compared
to U3M. This gap suggests that U3M may employ more sophisticated or optimized
techniques for multimodal integration, perhaps through advanced feature fusion
strategies or more effective neural network architectures.

Table IV provides a detailed per-class percentage of mIoU analysis for models
tested on the FMB dataset, utilizing both RGB and RGB-infrared modalities. The
data demonstrates the superior performance of our model across the majority of
categories, with particularly high IoU scores in ’Building’ (81.3) and
’Vegetation’ (86.8), illustrating its robustness in identifying and segmenting
structural and natural elements in urban environments. However, a notable
exception is the ’Truck’ category, where our model’s performance (41.9) lags
behind other models like the HDNet (60.7) and EGNet (63.0), indicating
potential challenges in distinguishing larger vehicles possibly due to their
similar spectral signatures with other objects or insufficient training data
representing this category. Our model’s overall mIoU of 60.8 is the highest
among the listed methods, confirming its efficacy in integrating RGB and
infrared data to enhance segmentation accuracy. The integration of infrared
data is particularly beneficial in improving the model’s performance under
varying lighting conditions, as infrared provides consistent material
recognition capabilities that are less susceptible to variations in visible
light. The segmentation performance in the ’Traffic Lamp’ (46.2) and ’Traffic
Sign’ (62.2) classes, while not the highest, still represents competitive
results, suggesting that the model effectively utilizes the infrared component
to detect these objects typically characterized by their distinct material
properties not always apparent in RGB imagery.

Figure 7: Segmentation results visualization on Mcubes dataset. The figure
provides a vivid visualization of semantic segmentation results on the Mcubes
dataset, showcasing the effectiveness of different modalities: RGB, RGB-A,
RGB-A-D, and RGB-A-D-N.

###  IV-D Visulization

1) Segmentation result visualization on FMB dataset. The visualization in
Figure 6 compellingly illustrates the effectiveness of multimodal semantic
segmentation using RGB and thermal (RGB-T) data on the FMB dataset. The
comparative analysis of RGB-only versus RGB-T segmentation underscores the
limitations of relying solely on visible light data, particularly under
adverse lighting conditions. The enhanced segmentation accuracy achieved with
RGB-T highlights the thermal modality’s critical role in distinguishing and
classifying various elements within urban scenes, reinforcing the necessity
for incorporating multimodal data in segmentation tasks to tackle diverse
environmental challenges. This analysis confirms the hypothesis that thermal
data significantly boosts the segmentation capability, particularly in
detecting living entities and differentiating them from inanimate backgrounds,
thereby providing a more reliable and robust segmentation framework for real-
world applications.

Figure 8: t-SNE visualization on FMB dataset. The visualization comprises
multiple panels, each line represents a different combination of modalities
used for segmentation, labeled as RGB and RGBT. These panels help illustrate
the distribution and separation of features in a two-dimensional space, giving
insights into the discriminative power of the features extracted under
different input conditions.

2) t-SNE visualization on FMB dataset. The t-SNE visualizations in Fig. 8
underscore the significant impact of integrating thermal imagery with RGB data
on the feature separability and overall effectiveness of semantic segmentation
models. The enhanced separation and definition of clusters with RGB-T data
corroborate the hypothesis that thermal information aids in resolving
ambiguities encountered with RGB-only data, especially in complex segmentation
scenarios. This analysis not only highlights the value of multimodal
approaches in improving semantic segmentation tasks but also emphasizes the
need for models that can effectively leverage diverse data types to achieve
more accurate and robust segmentation results. Such insights are critical for
advancing the development of segmentation technologies for applications in
areas such as autonomous driving, environmental monitoring, and urban
planning, where accurate material and object differentiation are paramount.

3) Segmentation result visualization on Mcubes dataset. The segmentation
visualizations in Fig. 7 underscore the incremental benefits of integrating
multiple data modalities into semantic segmentation tasks on the Mcubes
dataset. Each added modality—AoLP, DoLP, and NIR—contributes uniquely to the
enhancement of the segmentation accuracy, improving specific challenges
encountered with the RGB-only model. The results vividly demonstrate that
while RGB data provides a foundational layer of visual information, the
incorporation of polarization and thermal data crucially enriches the feature
set available for segmentation, thereby enabling more precise and contextually
aware delineations of urban scene elements. This multimodal approach
illustrates the potential for such technologies to be applied in real-world
scenarios where diverse environmental factors and varied object interactions
complicate the accurate interpretation of urban spaces.

###  IV-E Ablation Experiment

1) Ablation experiment on different modalities. The data presented in Table V
showcases the mIoU percentages for semantic segmentation performance on the
Multimodal Material Segmentation (MCubeS) dataset. The analysis compares the
performance of various models utilizing different combinations of modalities:
RGB, RGB-A, RGB-A-D, and RGB-A-D-N. Our model demonstrates a progressive
improvement in segmentation accuracy as additional modalities are integrated.
Starting with RGB alone, our model achieves a mIoU of 49.22, which is higher
compared to MCubeSNet’s 33.70 and CMNeXt’s 48.16. This trend of superior
performance continues with the inclusion of AoLP (RGB-A), where our model
scores 49.89, slightly ahead of CMNeXt’s 48.42. The enhancement is more
pronounced in configurations involving both AoLP and DoLP (RGB-A-D), with our
model achieving a mIoU of 50.26, compared to CMNeXt’s 49.48. The most
comprehensive modality combination, incorporating RGB, AoLP, DoLP, and NIR
(RGB-A-D-N), allows our model to reach its peak performance at a mIoU of
51.69, which stands out significantly against CMNeXt’s 51.54 and MCubeSNet’s
42.86. The incremental improvements observed with each additional modality
underscore the efficacy of integrating multimodal data to capture a richer,
more comprehensive feature set for material segmentation. The integration of
NIR, in particular, appears to provide critical enhancements, likely due to
its capability to offer consistent material properties detection that is less
dependent on visible light conditions.

2) Ablation experiment on different module combinations. The data presented in
Table VI illustrates a comparative analysis of semantic segmentation
performance on the MCubes dataset using different architectural enhancements
within a given framework. These results highlight the influence of various
module integrations on the mIoU metric. Starting with a baseline configuration
that utilizes a Linear Layer only, the model achieves a mIoU of 49.39. This
setup serves as a foundational benchmark for evaluating the effectiveness of
additional modules. Upon incorporating a Channel Attention mechanism, there is
a modest increase in performance, with the mIoU improving to 50.34. Channel
Attention likely aids the model in focusing more effectively on relevant
features by re-weighting channel-specific features, thus providing a more
refined feature map for segmentation tasks. A more substantial improvement is
observed when a Pyramid Scene Parsing (PSP) module is added, resulting in the
mIoU of 50.62. The PSP module, known for its capability to aggregate context
information at different scales, evidently enhances the model’s ability to
capture and integrate multi-scale contextual information, which is crucial for
accurate segmentation. Our model, U3M, which incorporates RGB-ADN modalities,
achieves the highest performance with a mIoU of 51.69. This indicates that the
combination of RGB data along with AoLP and DoLP from the ADN modality,
effectively utilized in U3M, significantly contributes to the segmentation
accuracy. This performance underscores the model’s robustness and its enhanced
capability to discriminate between material types and conditions in a complex
dataset like MCubes.

##  V Conclusion and Limitation

1) Conclusion. The proposed U3M makes a significant leap in multimodal
semantic segmentation, featuring innovative modality integration and feature
fusion techniques. It addresses modal bias by employing an unbiased multiscale
modal fusion methodology that equitably treats all modalities, thereby
reducing manual bias. This model utilizes multiscale fusion modules that
combine convolutional and pooling strategies, effectively integrating
modalities at various scales and enhancing adaptability across diverse
environments. Extensive experiments on challenging datasets show U3M
consistently surpasses existing models in both accuracy and robustness,
proving its suitability for real-world applications such as autonomous driving
and urban planning.

2) Limitation and future work. Future work could focus on optimizing the
model’s architecture for greater efficiency. Additionally, integrating more
varied modalities and extensive real-world testing could broaden its
application scope, ensuring it meets the evolving demands of practical
implementations in areas like autonomous driving and urban planning.

## References

  * [1] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 40, no. 4, pp. 834–848, 2017.
  * [2] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 801–818.
  * [3] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” _Advances in neural information processing systems_ , vol. 34, pp. 12 077–12 090, 2021.
  * [4] Y. Liao, J. Xie, and A. Geiger, “Kitti-360: A novel dataset and benchmarks for urban scene understanding in 2d and 3d,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 45, no. 3, pp. 3292–3310, 2022.
  * [5] S. Dong, W. Zhou, C. Xu, and W. Yan, “Egfnet: Edge-aware guidance fusion network for rgb–thermal urban scene parsing,” _IEEE Transactions on Intelligent Transportation Systems_ , 2023.
  * [6] C. Hazirbas, L. Ma, C. Domokos, and D. Cremers, “Fusenet: Incorporating depth into semantic segmentation via fusion-based cnn architecture,” in _Computer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan, November 20-24, 2016, Revised Selected Papers, Part I 13_. Springer, 2017, pp. 213–228.
  * [7] W. Zhou, S. Dong, M. Fang, and L. Yu, “Cacfnet: Cross-modal attention cascaded fusion network for rgb-t urban scene parsing,” _IEEE Transactions on Intelligent Vehicles_ , 2023.
  * [8] J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 3146–3154.
  * [9] J. Liu, Z. Liu, G. Wu, L. Ma, R. Liu, W. Zhong, Z. Luo, and X. Fan, “Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2023, pp. 8115–8124.
  * [10] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 3431–3440.
  * [11] Z. Jin, T. Gong, D. Yu, Q. Chu, J. Wang, C. Wang, and J. Shao, “Mining contextual information beyond image for semantic segmentation,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2021, pp. 7231–7241.
  * [12] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 2881–2890.
  * [13] J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “Cmx: Cross-modal fusion for rgb-x semantic segmentation with transformers,” _IEEE Transactions on Intelligent Transportation Systems_ , 2023.
  * [14] X. Hu, K. Yang, L. Fei, and K. Wang, “Acnet: Attention based network to exploit complementary features for rgbd semantic segmentation,” in _2019 IEEE International conference on image processing (ICIP)_. IEEE, 2019, pp. 1440–1444.
  * [15] J. Zhang, R. Liu, H. Shi, K. Yang, S. Reiß, K. Peng, H. Fu, K. Wang, and R. Stiefelhagen, “Delivering arbitrary-modal semantic segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 1136–1147.
  * [16] Y. Liang, R. Wakaki, S. Nobuhara, and K. Nishino, “Multimodal material segmentation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 19 800–19 808.
  * [17] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [18] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [19] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , October 2021, pp. 10 012–10 022.
  * [20] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _arXiv preprint arXiv:2010.11929_ , 2020.
  * [21] X. Dong, J. Bao, D. Chen, W. Zhang, N. Yu, L. Yuan, D. Chen, and B. Guo, “Cswin transformer: A general vision transformer backbone with cross-shaped windows,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2022, pp. 12 124–12 134.
  * [22] Y. Yuan, R. Fu, L. Huang, W. Lin, C. Zhang, X. Chen, and J. Wang, “Hrformer: High-resolution transformer for dense prediction,” _arXiv preprint arXiv:2110.09408_ , 2021.
  * [23] Q. He, “Prompting multi-modal image segmentation with semantic grouping,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 38, no. 3, 2024, pp. 2094–2102.
  * [24] M. Kaykobad Reza, A. Prater-Bennette, and M. Salman Asif, “Multimodal transformer for material segmentation,” _arXiv e-prints_ , pp. arXiv–2309, 2023.
  * [25] B. Cao, J. Guo, P. Zhu, and Q. Hu, “Bi-directional adapter for multimodal tracking,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 38, no. 2, 2024, pp. 927–935.
  * [26] M. K. Reza, A. Prater-Bennette, and M. S. Asif, “Multimodal transformer for material segmentation,” _arXiv preprint arXiv:2309.04001_ , 2023.
  * [27] Q. Hou, L. Zhang, M.-M. Cheng, and J. Feng, “Strip pooling: Rethinking spatial pooling for scene parsing,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 4003–4012.
  * [28] G. Lin, A. Milan, C. Shen, and I. Reid, “Refinenet: Multi-path refinement networks for high-resolution semantic segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1925–1934.
  * [29] S. Choi, J. T. Kim, and J. Choo, “Cars can’t fly up in the sky: Improving urban-scene segmentation via height-driven attention networks,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 9373–9383.
  * [30] H. Zhang, K. Dana, J. Shi, Z. Zhang, X. Wang, A. Tyagi, and A. Agrawal, “Context encoding for semantic segmentation,” in _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , 2018, pp. 7151–7160.
  * [31] C. Yu, J. Wang, C. Gao, G. Yu, C. Shen, and N. Sang, “Context prior for scene segmentation,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 12 416–12 425.
  * [32] Y. Yuan, L. Huang, J. Guo, C. Zhang, X. Chen, and J. Wang, “Ocnet: Object context for semantic segmentation,” _International Journal of Computer Vision_ , vol. 129, no. 8, pp. 2375–2398, 2021.
  * [33] X. Li, X. Li, L. Zhang, G. Cheng, J. Shi, Z. Lin, S. Tan, and Y. Tong, “Improving semantic segmentation via decoupled body and edge supervision,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII 16_. Springer, 2020, pp. 435–452.
  * [34] S. Borse, Y. Wang, Y. Zhang, and F. Porikli, “Inverseform: A loss function for structured boundary-aware segmentation,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 5901–5911.
  * [35] B. Cheng, A. Schwing, and A. Kirillov, “Per-pixel classification is not all you need for semantic segmentation,” _Advances in neural information processing systems_ , vol. 34, pp. 17 864–17 875, 2021.
  * [36] B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, “Masked-attention mask transformer for universal image segmentation,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2022, pp. 1290–1299.
  * [37] M.-H. Guo, C.-Z. Lu, Z.-N. Liu, M.-M. Cheng, and S.-M. Hu, “Visual attention network,” _Computational Visual Media_ , vol. 9, no. 4, pp. 733–752, 2023.
  * [38] Q. Hou, Z. Jiang, L. Yuan, M.-M. Cheng, S. Yan, and J. Feng, “Vision permutator: A permutable mlp-like architecture for visual recognition,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 45, no. 1, pp. 1328–1334, 2022.
  * [39] D. Lian, Z. Yu, X. Sun, and S. Gao, “As-mlp: An axial shifted mlp architecture for vision,” _arXiv preprint arXiv:2107.08391_ , 2021.
  * [40] M.-H. Guo, C.-Z. Lu, Q. Hou, Z. Liu, M.-M. Cheng, and S.-M. Hu, “Segnext: Rethinking convolutional attention design for semantic segmentation,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 1140–1156, 2022.
  * [41] W. Zhou, S. Dong, C. Xu, and Y. Qian, “Edge-aware guidance fusion network for rgb–thermal scene parsing,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 36, no. 3, 2022, pp. 3571–3579.
  * [42] F. Deng, H. Feng, M. Liang, H. Wang, Y. Yang, Y. Gao, J. Chen, J. Hu, X. Guo, and T. L. Lam, “Feanet: Feature-enhanced attention network for rgb-thermal real-time semantic segmentation,” in _2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2021, pp. 4467–4473.
  * [43] Z. Zhao, S. Xu, C. Zhang, J. Liu, P. Li, and J. Zhang, “Didfuse: Deep image decomposition for infrared and visible image fusion,” _arXiv preprint arXiv:2003.09210_ , 2020.
  * [44] Z. Huang, J. Liu, X. Fan, R. Liu, W. Zhong, and Z. Luo, “Reconet: Recurrent correction network for fast and efficient multi-modality image fusion,” in _European conference on computer Vision_. Springer, 2022, pp. 539–555.
  * [45] P. Li, J. Chen, B. Lin, and X. Xu, “Residual spatial fusion network for rgb-thermal semantic segmentation,” _arXiv preprint arXiv:2306.10364_ , 2023.
  * [46] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [47] J. Chen, X. Wang, Z. Guo, X. Zhang, and J. Sun, “Dynamic region-aware convolution,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 8064–8073.
  * [48] J. Zhou, V. Jampani, Z. Pi, Q. Liu, and M.-H. Yang, “Decoupled dynamic filter networks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 6647–6656.
  * [49] A. Prakash, K. Chitta, and A. Geiger, “Multi-modal fusion transformer for end-to-end autonomous driving,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 7077–7087.
  * [50] H. R. V. Joze, A. Shaban, M. L. Iuzzolino, and K. Koishida, “Mmtm: Multimodal transfer module for cnn fusion,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 13 289–13 299.
  * [51] W. Zhou, J. Liu, J. Lei, L. Yu, and J.-N. Hwang, “Gmnet: Graded-feature multilabel-learning network for rgb-thermal urban scene semantic segmentation,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 7790–7802, 2021.
  * [52] G. Li, Y. Wang, Z. Liu, X. Zhang, and D. Zeng, “Rgb-t semantic segmentation with location, activation, and sharpening,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 33, no. 3, pp. 1223–1235, 2022.
  * [53] H. Xu, J. Ma, J. Jiang, X. Guo, and H. Ling, “U2fusion: A unified unsupervised image fusion network,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 1, pp. 502–518, 2020.
  * [54] J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong, and Z. Luo, “Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2022, pp. 5802–5811.


# Viscoplastic sessile drop coalescence

Vanessa R. Kern<EMAIL_ADDRESS>Torstein Sæter  Andreas Carlson
<EMAIL_ADDRESS>Department of Mathematics, Mechanics Division, University
of Oslo, Oslo 0316, Norway

###### Abstract

The evolution of the liquid bridge formed between two coalescing sessile
yield-stress drops is studied experimentally. We find that the height of the
bridge evolves similar to a viscous Newtonian fluid, $h_{0}\sim t$, before
arresting at long time prior to minimizing its liquid/gas interfacial energy.
We numerically solve for the final arrested profile shape and find it depends
on the fluid’s yield stress $\tau_{y}$ and coalescence angle $\alpha$,
represented by the Bingham number $\tau_{y}h_{drop}/\sigma$ modified by the
drop’s height-width aspect ratio. We present a scaling argument for the
bridge’s temporal evolution using the length scale found from an analysis of
the arrested shape as well as from the similarity solution derived for the
bridge’s evolution.

††preprint: APS/123-QED

Yield-stress fluids only flow when subject to a stress greater than their
yield stress $\tau_{y}$. We rely on yield stress fluids. They allow us to
squeeze toothpaste from tubes, lay mortar for bricks, plaster walls and even
paint. Much focus has been spent on yield-stress flows, with a growing
interest in capillary flows such as surface spreading for inkjet printing and
spray coating [1, 2, 3, 4, 5], viscous fingering [6], and the adhesion of
pastes [7].

We focus here on the dynamics of sessile viscoplastic drop coalescence
utilizing both bottom-view and side-view imaging as shown in Fig. 1a,b. When
two drops coalesce, between them is formed a liquid bridge. Much emphasis has
been placed on the evolution of this bridge for both free and sessile
Newtonian drops [8, 9, 10, 11]. For sessile drops, the growth of the bridge
can be characterized by its width $r_{0}$ as viewed from below and its height
$h_{0}$ as viewed from the side. The presence of a symmetry breaking substrate
adds additional stress near the growing bridge, where the coalescence angle
$\alpha$ has been shown to drastically alter its temporal evolution [12].
Different coalescence regimes have been identified, where the bridge height
for inertially dominated flows grows as $h_{0}\sim t^{0.5}$ [10] and for
viscous dominated flows as $h_{0}\sim t$ [13, 11], with corrections in the
early time as $h_{0}\sim t\ln{t}$ [13]. For viscous flows, it has been shown
$r_{0}\sim t^{0.5}$ [14] before transitioning to Tanner’s-like spreading at
long time [15, 16]. Inertial to viscous crossover depends on liquid properties
and drop size [17].

Despite relevance in many motivating applications [18, 19, 20], far less is
known about non-Newtonian coalescence. For polymeric fluids, recently the
effects of elasticity have been considered, showing polymer stretching affects
spatial self-similarity, but not temporal growth [21]. Other studies have
shown new scaling laws for temporal bridge growth [22], transitions from
inertial to visco-elastic regimes [23] and effects of approach velocity during
sessile coalescence [24]. When two yield-stress drops coalesce, their
interface arrests before minimizing their surface energy. Arrested coalescence
events have been previously observed for “jammed” interfaces in pickering
emulsions [25] as well as for anisotropic colloids [26, 27, 28].

Here we find that despite the symmetry-breaking substrate, the shear-thinning
properties of the drops considered, as well as a yield-stress fluid’s ability
to have yielded, liquid-like and non-yielded, solid-like, flow regions, the
evolution of the liquid bridge evolution shares many similarities with the
viscous Newtonian case in its growth as $h_{0}\sim t$, its temporal collapse
and its spatial self similarity, with the effect of the yield stress evident
only in its final arrested shape.

Figure 1:  Bottom (a) and side (b) view viscoplastic sessile drop coalescence
with yield stress $\tau_{y}=27.5$ Pa. High speed imagery captures the
coalescence event. Post-coalescence (a) the contact line spreads characterized
by $r_{0}$, and (b) the bridge grows characterized by its minima $h_{0}$,
before halting due to the drop’s yield stress. The initial height of the drop
$h_{drop}$ and its footprint $a$ are related through the coalescence angle
$\alpha$. Bottom and side view time sequences are presented from two different
experiments with similar initial conditions.

Experiment \- Side and bottom view sessile coalescence events were imaged
using a Photron Fastcam SA5 with a Nikon 200mm f/4 AF-D Macro lens and two
teleconverters (Nikon 2x and 1.7x) resulting in resolutions of $\sim
6\mu$m/pixel. Further processing in Matlab using partial area subpixel edge
detection techniques afforded resolutions below $6\mu$m/pixel [29, 30].
Coalescence substrates were glass microscope slides (VWR Ca. 631-1550) rinsed
with isopropanol (VWR Ca. 20922.364), plasma cleaned and kept in DI water
until use. Immediately prior to use slides were blown dry with oil free
compressed air. Coalescing drops were Carbopol 940 solutions with yield
stresses varying from $\tau_{y}=2$ Pa to $50$ Pa measured using the stress
growth test, details in the Supplementary Material [31]. Drop densities were
assumed to be close to water $\rho\approx 1000$ kg/m3 and liquid/gas surface
tensions were assumed to be $\sigma=0.053$ N/m as measured for a liquid bridge
increasing in radius [32].

For consistency of comparison to viscous coalescence, only relatively
spherical-cap shaped pre-coalescence sessile drops with aspect ratios
$a/h_{drop}>1.5$, and $\alpha<45^{\circ}$ were considered. To minimize
deviations from the spherical-cap shape, drops with footprints $a<1$ mm were
preferred [2]. When yield-stress drops spread on substrates, spreading arrests
before drops minimize their surface energy as by the Young-Dupré equation
$\sigma\cos(\alpha)=\sigma_{SV}-\sigma_{SL}$ [1]. Therefore for our
experiments, the smallest obtainable initial coalescence angles $\alpha$ we
expect are $6.5^{\circ}$ to $34^{\circ}$, for $\tau_{y}=2$ Pa and $50$ Pa,
respectively [31, 1, 2]. Slides were plasma cleaned before each set of
experiments to maintain a low substrate surface energy, thereby minimizing
$\alpha$. By knowing the initial drop size, approximate arrested angle
$\alpha$ and footprint $a$ at which spreading will cease, drops were
simultaneously dispensed at known relative distances achieving both minimal
$\alpha$ as well as contact line velocities below the initial bridge height
velocity $\mathrm{d}h_{0}/\mathrm{d}t$. Post coalescence, drops arrested
within seconds rendering negligible evaporative losses.

Results \- We frame our problem as a 2D viscous flow parallel to the bridge
justified by the small relative aspect ratio $h_{0}/r_{0}\ll 1$, and low film
Reynolds number [10, 13, 31]. As the fluid has a yield stress $\tau_{y}$, the
system can be described using viscoplastic lubrication theory [1, 33],

$\begin{split}\frac{\partial h}{\partial
t}&+\frac{1}{6\eta}\frac{\partial}{\partial
x}\left((3h-Y)Y^{2}\left|\frac{\partial P}{\partial x}\right|\right)=0\text{
and }\\\ Y&=\max{\left(0,h-\tau_{y}\left|\frac{\partial P}{\partial
x}\right|^{-1}\right)}\end{split}$ (1)

with $h$, $P$, and $Y$ all functions of $(x,t)$. Here a minimal rheological
model, the Bingham model, is used to represent Carbopol (see the Supplementary
Material), where $Y(x,t)$ represents the boundary between the yielded and
arrested regions of the drop, $h(x,t)$ the liquid/gas interface, $\eta$ the
effective viscosity of the fluid, and $P(x,t)$ the capillary pressure. For
lubrication theory to hold, one would expect that for the $\alpha$ at which
our experiment is constrained that the lubrication approximation would be
inaccurate, however this approximation has been shown to hold for $\alpha$ as
large as $\alpha=67^{\circ}$ [34].

Figure 2: (a) Post-coalescence ($t=\infty$) final arrested shapes for
increasing yield stress $\tau_{y}$, each scale bar is $500\mu$m. For similar
coalescence angles $\alpha$, increasing $\tau_{y}$ decreases the arrested
bridge height $h_{f}$. Increases in $\alpha$ serve to increase the arrested
bridge height, as shown for the $49.5$ Pa case. (b) Comparison of the
experimentally measured liquid/gas interfacial profiles $h(x,\infty)$ (open
circles) against $x/a$ with their corresponding numerical solutions (solid
lines) obtained from a scaled (2),
$\hat{h}\hat{h}_{\hat{x}\hat{x}\hat{x}}=J/(h_{drop}/a)^{3}$ with boundary
conditions $\mathrm{d}\hat{h}/\mathrm{d}\hat{x}=0\text{ at
}(\hat{x}=-1,0)\text{ and }\hat{h}(\hat{x}=-1)=1$. Notably plotting a
normalized $h(x,\infty)$ as shown collapses the numerically determined final
profile heights for all 21 different experiments ranging $J=[0.01-0.47]$ and
$h_{drop}/a=[0.15-0.58]$ onto a single curve.

Final arrested shape \- Post Newtonian drop-drop coalescence, interfacial
energies are minimized and the liquid/gas interface assumes a spherical-cap
shape. However, post yield-stress drop-drop coalescence, the drop’s bridge
arrests at some fraction of its total height $h_{f}/h_{drop}$, see Fig. 2a.

From (1) we see that as $Y\rightarrow 0$ the yielded region of the drop
vanishes. The shape of the arrested interface $h(x,\infty)$ can then be
characterized by the non-linear ordinary differential equation obtained from
(1) in the limit $Y\rightarrow 0$,

$h~{}\frac{\mathrm{d}^{3}h}{\mathrm{d}x^{3}}=\frac{\tau_{y}}{\sigma}.$ (2)

Here the pressure $P$ is approximated as the local interfacial curvature
$\sigma~{}\mathrm{d}^{2}h/\mathrm{d}x^{2}$ with the small angle approximation
and flow unidirectional towards the bridge $x=0$. We numerically solve (2) as
a system of first order ODEs in python using SciPy’s ‘solve_bvp’ function by
prescribing the experimentally determined aspect ratio $h_{drop}/a$ and
Bingham number $J=\tau_{y}h_{drop}/a$, scalings for the height
$\hat{h}=h/h_{drop}$ and length $\hat{x}=x/a$ and boundary conditions
$\mathrm{d}\hat{h}/\mathrm{d}\hat{x}=0\text{ at }(\hat{x}=-1,0)\text{ and
}\hat{h}(\hat{x}=-1)=1$, representing symmetry at $h(-a,\infty)=h_{drop}$,
symmetry at the bridge $h(0,\infty)=h_{f}$, and the height of the drop,
respectively.

Figure 3:  Bridge evolution and self-similarity. (a) Unscaled bridge $h_{0}$
evolution data. (b) Scaled bridge $h_{0}$ evolution data. Inclusion of an
effective viscosity $\eta$ in the x-axis scaling collapses the coalescence
events. Unaccounted for variation in $\eta$ from varying $\alpha$ causes splay
in the data. Here we use $\eta=[0.7,3.8,8.4,12.4]$ Pa$\cdot$s for the
corresponding $\tau=[2,14.4,27.5,49.5]$ Pa. (c) Scaled drop interface
$H=h(x,t)/h_{0}(t)$ against the similarity variable $\xi$ in the linear regime
of b. The solid line is the numerically determined similarity solution
obtained when $Y\approx h$ from (1) with scaling from (3),
$\mathcal{H}-\xi\mathcal{H}^{\prime}+1/V(\mathcal{H}^{3}\mathcal{H}^{\prime\prime\prime})^{\prime}=0$
with one unknown parameter $V$ and boundary conditions $\mathcal{H}(0)=1$,
$\mathcal{H}^{\prime}(0)=\mathcal{H}^{\prime\prime\prime}(0)=0$,
$\mathcal{H}^{\prime\prime}(\infty)=0$ and $\mathcal{H}^{\prime}(\infty)=1$
[34], $()^{\prime}=\mathrm{d}/\mathrm{d}\xi$. Excellent agreement can be seen
despite increasing $\tau_{y}$.

Fig. 2b shows the results of numerically integrating a scaled (2) compared
with experimental data. We see good height profile agreement when the pre-
coalesced drop shape is relatively spherical, with deviations a byproduct of
initial deposition. We find in general that initial deviations from a
spherical-cap shape cause deviations in the final arrested profile.
Normalizing by the minimum bridge height as shown serves to collapse the
numerically determined final profile heights for 21 different experiments
ranging $J=[0.01,0.47]$ and $h_{drop}/a=[0.15,0.58]$ onto a single curve. More
details about this numerical solution as well as a comparison of numerically
recovered versus experimentally determined final bridge heights $h_{f}$ can be
found in the Supplementary Material [31].

From the scaling of (2) we can see that a non-dimensional quantity in the form
of the Bingham number modified by the aspect ratio $J/(h_{drop}/a)^{3}$
arises, where a decrease in this quantity leads to an increase in the final
arrested bridge height, recovering the intuition of a simple pressure balance
shown in the Supplementary Material [31].

Temporal evolution \- In Fig. 3a we use the bridge height $h_{0}$ as a measure
of the liquid bridge’s temporal evolution and observe that $\alpha$,
$\tau_{y}$ and $\eta$ significantly affect the coalescence velocity and
arrested bridge height. We observe for similar yield stresses $\tau_{y}$ that
increasing $\alpha$ decreases the overall coalescence time and increases the
final arrested bridge height $h_{f}$ [31]. We also observe for similar
coalescence angle $\alpha$ increasing the yield stress $\tau_{y}$ decreases
the bridge velocity and the final arrested bridge height $h_{f}$.

Despite scanning a large parameter space, we observe the majority of the
bridge’s temporal evolution is similar to the viscous Newtonian case
$h_{0}\sim t$. Similarities also appear to the Newtonian case in the radial
direction for the drop’s advancing contact line as $r_{0}\sim t^{0.5}$ before
transitioning to a long-time relaxation as $r_{0}\sim t^{0.1}$ similar to
Tanner’s law spreading [16], shown in the Supplementary Material [31]. From
(1) we can see that as $\tau_{y}\rightarrow 0$ or $\left|\partial P/\partial
x\right|\gg\tau_{y}$, $Y\approx h$, reducing (1) to the 2D Newtonian
lubrication equation [34, 35]. This implies that the observed linear regime is
a result of a rapid change in interfacial curvature such as in the vicinity of
$h_{0}$.

Due to similarities in the temporal evolution of the liquid bridge with the
viscous Newtonian case for the majority of the dynamics, we are then tempted
to seek a similar universal re-scaling and similarity solution as can be shown
for the viscous Newtonian case [34]. Following the logic of [34] when
$Y\approx h$ we find similarity variables for (1) of the form,

$\begin{split}\mathcal{H}(\xi)&=\frac{h(x,t)}{vt}=\frac{h(x,t)}{h_{0}},\text{
with}\\\
\xi&=\frac{\tan(\alpha)~{}x}{vt}=\frac{\tan(\alpha)~{}x}{h_{0}}.\end{split}$
(3)

Here $v$ represents the coalescence velocity and $\tan({\alpha})$ is not
simplified to $\alpha$ in order to account for the larger coalescence angles.
A scaling for the coalescence time $\hat{t}$ is also found to be of the form

$\hat{t}\sim\frac{a^{3}\tau_{y}\tan^{4}(\alpha)~{}t}{\eta h_{drop}^{3}}.$ (4)

Here we scale by the characteristic length scale
$(h_{drop}/a)^{3}~{}\sigma/\tau_{y}$ resulting from the dimensionless quantity
$J/(h_{drop}/a)^{3}$ found previously, as well as a coalescence velocity
$v\sim\sigma\tan^{4}(\alpha)/\eta$. From (4) an immediate concern arises in
the handling of $\eta$, as $\eta$ is a function of the shear rate
$\dot{\gamma}$ and consequently a function of time. Without a good estimate of
the yielded region in the vicinity of $h_{0}$, the shear rate $\dot{\gamma}$
cannot easily be estimated. Instead we observe that scaling by $\hat{t}$ with
all $\eta=1$ Pa$\cdot$s, as shown in the Supplementary Material [31], groups
the data by $\tau_{y}$, with increases in $\tau_{y}$ decreasing the
coalescence velocity. From the rheological data it can be seen that increases
in $\tau_{y}$ increase the viscosity $\eta$ at every shear rate [31], thus it
can be expected that increases in $\tau_{y}$ should lead to decreases in the
coalescence velocity $v$.

Since we find that the experiments can be grouped by coalescence velocity $v$
as a function of $\tau_{y}$, and because a linear time dependence in the
growth of $h_{0}$ is still observed for the majority of each coalescence, we
are inclined to believe there is an effective viscosity $\eta$ for each
$\tau_{y}$ that can be used to scale the data. By writing $\eta$ from the
definition of the coalescence velocity as

$\eta\sim\frac{\sigma\tan^{4}(\alpha)}{v},$ (5)

we can extract effective viscosities during the linear regime $h_{0}\sim t$.
We find that for similar $\alpha$ the relationship between these viscosities
$\eta$ and the fluid’s yield stress $\tau_{y}$ is linear. From the rheological
data, we can then infer that the effective shear rate near the evolving bridge
is $O(1)s^{-1}$.

The results of rescaling the data using effective viscosities for each
$\tau_{y}$ determined at the shear rate $\dot{\gamma}=4s^{-1}$ are shown in
Fig. 3b. As not all experiments are perfectly spherical-capped shaped
initially, error inherently exists in the determination of the drop’s aspect
ratio $h_{drop}/a$ as well as coalescence angle $\alpha$. Additional error
exists in the determination of $\eta$ as the shear rate near $h_{0}$ is
unknown. However despite these uncertainties, scaling in this fashion does
group the data around a collapsed linear regime.

For times $\hat{t}$ in this collapsed region of Fig. 3b, we expect that the
interface $h(x,t)$ should also collapse onto a self-similar shape for all
coalescence angles $\alpha$ and yield stresses $\tau_{y}$. By rescaling our
data with the similarity variables of (3) we find that in this linear regime
for varying yield stress a self similar collapse of the liquid bridge
evolution does occur as presented in Fig. 3c. Comparing this collapse with the
numerically calculated similarity solution from (1) as $Y\approx h$ shows
excellent agreement, implying that for this system liquid bridge evolutions
across varying yield stresses are indeed self-similar and can be collapsed
using viscous Newtonian lubrication theory [35, 34].

Discussion \- Here we present a first look at the sessile drop coalescence of
viscoplastic drops. Surprisingly we find that the majority of the bridge’s
evolution follows viscous Newtonian scaling laws. We also find that 2D
Newtonian lubrication theory captures the self similarity in the bridge’s
evolution across multiple yield stresses $\tau_{y}$ and coalescence angles
$\alpha$. We postulate this reduction to the Newtonian case can be explained
as the viscoplastic lubrication equations (1) reduce to the Newtonian
lubrication equations for regions of rapidly changing interfacial curvature
such as in the vicinity of $h_{0}$. We find that both the yield stress
$\tau_{y}$ and coalescence angle $\alpha$ affect the fractional height
$h_{f}/h_{drop}$ at which the evolving bridge arrests and solve numerically
for the final profile shape. Here only experiments where drops were reasonably
spherical-cap shaped were considered. These conditions, however, only
represent a small set of cases for the coalescence of sessile yield-stress
drops that can easily adopt highly non-spherical cap shapes [2]. It would then
be interesting for future study to include drops with highly non-uniform
initial liquid/gas interfacial curvatures and potentially larger initial
coalescence angles $\alpha$.

Acknowledgments \- The authors gratefully acknowledge the financial support of
the Research Council of Norway through the program NANO2021, project number
301138.

## References

  * Jalaal  _et al._ [2021] M. Jalaal, B. Stoeber, and N. J. Balmforth, Spreading of viscoplastic droplets, Journal of Fluid Mechanics 914 (2021).
  * Martouzet  _et al._ [2021] G. Martouzet, L. Jørgensen, Y. Pelet, A.-L. Biance, and C. Barentin, Dynamic arrest during the spreading of a yield stress fluid drop, Physical Review Fluids 6, 044006 (2021).
  * Jalaal  _et al._ [2018] M. Jalaal, C. Seyfert, B. Stoeber, and N. Balmforth, Gel-controlled droplet spreading, Journal of Fluid Mechanics 837, 115 (2018).
  * Jalaal  _et al._ [2019] M. Jalaal, C. Seyfert, and J. H. Snoeijer, Capillary ripples in thin viscous films, Journal of fluid mechanics 880, 430 (2019).
  * De Ruiter  _et al._ [2018] R. De Ruiter, L. Royon, J. H. Snoeijer, and P. Brunet, Drop spreading and gelation of thermoresponsive polymers, Soft Matter 14, 3096 (2018).
  * Lindner  _et al._ [2000] A. Lindner, P. Coussot, and D. Bonn, Viscous fingering in a yield stress fluid, Physical Review Letters 85, 314 (2000).
  * Barral  _et al._ [2010] Q. Barral, G. Ovarlez, X. Chateau, J. Boujlel, B. Rabideau, and P. Coussot, Adhesion of yield stress fluids, Soft Matter 6, 1343 (2010).
  * Menchaca-Rocha  _et al._ [2001] A. Menchaca-Rocha, A. Martínez-Dávalos, R. Nunez, S. Popinet, and S. Zaleski, Coalescence of liquid drops by surface tension, Physical Review E 63, 046309 (2001).
  * Thoroddsen  _et al._ [2005] S. Thoroddsen, K. Takehara, and T. Etoh, The coalescence speed of a pendent and a sessile drop, Journal of Fluid Mechanics 527, 85 (2005).
  * Duchemin  _et al._ [2003] L. Duchemin, J. Eggers, and C. Josserand, Inviscid coalescence of drops, Journal of Fluid Mechanics 487, 167 (2003).
  * Aarts  _et al._ [2005] D. G. Aarts, H. N. Lekkerkerker, H. Guo, G. H. Wegdam, and D. Bonn, Hydrodynamics of droplet coalescence, Physical review letters 95, 164503 (2005).
  * Eddi  _et al._ [2013] A. Eddi, K. Winkels, and J. Snoeijer, Influence of droplet geometry on the coalescence of low viscosity drops, Physical review letters 111, 144502 (2013).
  * Eggers  _et al._ [1999] J. Eggers, J. R. Lister, and H. A. Stone, Coalescence of liquid drops, Journal of Fluid Mechanics 401, 293 (1999).
  * Ristenpart  _et al._ [2006] W. Ristenpart, P. McCalla, R. Roy, and H. A. Stone, Coalescence of spreading droplets on a wettable substrate, Physical review letters 97, 064501 (2006).
  * Bonn  _et al._ [2009] D. Bonn, J. Eggers, J. Indekeu, J. Meunier, and E. Rolley, Wetting and spreading, Reviews of modern physics 81, 739 (2009).
  * Tanner [1979] L. Tanner, The spreading of silicone oil drops on horizontal surfaces, Journal of Physics D: Applied Physics 12, 1473 (1979).
  * Paulsen  _et al._ [2011] J. D. Paulsen, J. C. Burton, and S. R. Nagel, Viscous to inertial crossover in liquid drop coalescence, Physical Review Letters 106, 114501 (2011).
  * Carrier and Colin [2003] V. Carrier and A. Colin, Coalescence in draining foams, Langmuir 19, 4535 (2003).
  * Tcholakova  _et al._ [2006] S. Tcholakova, N. D. Denkov, I. B. Ivanov, and B. Campbell, Coalescence stability of emulsions containing globular milk proteins, Advances in Colloid and interface Science 123, 259 (2006).
  * Mahdi  _et al._ [2008] K. Mahdi, R. Gheshlaghi, G. Zahedi, and A. Lohi, Characterization and modeling of a crude oil desalting plant by a statistically designed approach, Journal of Petroleum Science and Engineering 61, 116 (2008).
  * Dekker  _et al._ [2022] P. J. Dekker, M. A. Hack, W. Tewes, C. Datt, A. Bouillant, and J. H. Snoeijer, When elasticity affects drop coalescence, Physical Review Letters 128, 028004 (2022).
  * Varma  _et al._ [2020] S. C. Varma, A. Saha, S. Mukherjee, A. Bandopadhyay, A. Kumar, and S. Chakraborty, Universality in coalescence of polymeric fluids, Soft Matter 16, 10921 (2020).
  * Varma  _et al._ [2021] S. C. Varma, A. Saha, and A. Kumar, Coalescence of polymeric sessile drops on a partially wettable substrate, Physics of Fluids 33, 123101 (2021).
  * Sivasankar  _et al._ [2021] V. S. Sivasankar, S. A. Etha, D. R. Hines, and S. Das, Coalescence of microscopic polymeric drops: Effect of drop impact velocities, Langmuir 37, 13512 (2021).
  * Pawar  _et al._ [2011] A. B. Pawar, M. Caggioni, R. Ergun, R. W. Hartel, and P. T. Spicer, Arrested coalescence in pickering emulsions, Soft Matter 7, 7710 (2011).
  * Pawar  _et al._ [2012] A. B. Pawar, M. Caggioni, R. W. Hartel, and P. T. Spicer, Arrested coalescence of viscoelastic droplets with internal microstructure, Faraday discussions 158, 341 (2012).
  * Kraft  _et al._ [2009] D. J. Kraft, W. S. Vlug, C. M. van Kats, A. van Blaaderen, A. Imhof, and W. K. Kegel, Self-assembly of colloids with liquid protrusions, Journal of the American Chemical Society 131, 1182 (2009).
  * Sacanna and Pine [2011] S. Sacanna and D. J. Pine, Shape-anisotropic colloids: Building blocks for complex assemblies, Current opinion in colloid & interface science 16, 96 (2011).
  * Trujillo-Pino  _et al._ [2013] A. Trujillo-Pino, K. Krissian, M. Alemán-Flores, and D. Santana-Cedrés, Accurate subpixel edge location based on partial area effect, Image and Vision Computing 31, 72 (2013).
  * Trujillo-Pino [2019] A. Trujillo-Pino, Accurate subpixel edge location (https://www.mathworks.com/matlabcentral /fileexchange/48908-accurate-subpixel-edge-location), MATLAB Central File Exchange (2019).
  * [31] See supplemental material at [url will be inserted by publisher] for detailed rheological data., .
  * Jørgensen  _et al._ [2015] L. Jørgensen, M. Le Merrer, H. Delanoë-Ayari, and C. Barentin, Yield stress and elasticity influence on surface tension measurements, Soft Matter 11, 5111 (2015).
  * Balmforth [2019] N. J. Balmforth, Viscoplastic asymptotics and other analytical methods, in  _Lectures on visco-plastic fluid mechanics_ (Springer, 2019) pp. 41–82.
  * Hernández-Sánchez  _et al._ [2012] J. Hernández-Sánchez, L. Lubbers, A. Eddi, and J. Snoeijer, Symmetric and asymmetric coalescence of drops on a substrate, Physical review letters 109, 184502 (2012).
  * Oron  _et al._ [1997] A. Oron, S. H. Davis, and S. G. Bankoff, Long-scale evolution of thin liquid films, Reviews of modern physics 69, 931 (1997).
  * [36] _White paper: Understanding Yield Stress Measurements_, Tech. Rep. 120416 (Malvern Instruments Limited, Grovewood Road, Malvern, Worcestershire, UK. WR141XZ).
  * Żołek Tryznowska [2016] Z. Żołek Tryznowska, 6 - rheology of printing inks, in  _Printing on Polymers_, edited by J. Izdebska and S. Thomas (William Andrew Publishing, 2016) pp. 87–99.
  * Medhi  _et al._ [2020] S. Medhi, S. Chowdhury, D. K. Gupta, and A. Mazumdar, An investigation on the effects of silica and copper oxide nanoparticles on rheological and fluid loss property of drilling fluids, Journal of Petroleum Exploration and Production Technology 10, 91 (2020).
  * Shafiei  _et al._ [2018] M. Shafiei, M. Balhoff, and N. W. Hayman, Chemical and microstructural controls on viscoplasticity in carbopol hydrogel, Polymer 139, 44 (2018).
  * Balmforth  _et al._ [2000] N. J. Balmforth, A. Burbidge, R. Craster, J. Salzig, and A. Shen, Visco-plastic models of isothermal lava domes, Journal of Fluid Mechanics 403, 37 (2000).
  * Boujlel and Coussot [2013] J. Boujlel and P. Coussot, Measuring the surface tension of yield stress fluids, Soft Matter 9, 5898 (2013).

## Supplementary Material

##  I Rheology

###  I.1 Carbopol preparation procedure

Carbopol 940 (CAS. 9003-01-4) was briefly mixed in DI water between 300-1000
rpm using a Silverson L5M high shear laboratory mixer until dissolved to avoid
clumping. 1M NaOH was added to the dissolved solution in an 8:1 wt:wt% ratio
of 1M NaOH:Carbopol 940. The solution was then gently mechanically mixed using
a Kenwood Chef XL Titanium for approximately 10 days. The amount of dissolved
Carbopol was controlled to vary the yield stress, with $[0.8,1.1,1.5,2]$ g/L
Carbopol/Water resulting in fluid yield stresses of $[2.5,14,28,50]$ Pa,
respectively.

###  I.2 Rheological data

Figure 4:  Rheological data for Carbopol 940. a Measurement of yield stress
using the constant oscillatory shear rate test. b Measurement of viscosity
against shear rate.

No universal method for determining the yield stress of a material exists,
with measured values varying depending on the type of test and the condition
under which it was performed [36]. For instance, many complex fluids exhibit a
reversible thixotropy, where at a given shear rate the viscosity of the fluid
decreases over time, a behavior attributed to the breaking up of the fluid’s
internal structure [37]. Measurements of the yield-stress behaviour are
commonly carried out using a rotational rheometer, with some common and
accepted methods including direct model fitting of measured stress versus
strain data as by the Herschel-Bulkley model from (6), stress growth tests
where a low shear rate, typically $0.01s^{-1}$, is induced and the plateau in
shear stress over time is taken to be the yield stress, or the oscillation
amplitude sweep with the yield stress accepted to lie within the between the
initial drop in the loss modulus $G^{\prime}$ and the crossover in the loss
$G^{\prime}$ and storage modulus $G^{\prime\prime}$ of the material.

Carbopol 940 solutions exhibit no thixotropic behavior, and for this study we
chose the simplest and subjectively fastest of these yield stress measurement
methods, the stress growth test with an oscillatory shear rate of
$\dot{\gamma}=0.001s^{-1}$, shown in Fig. 4a. Carbopol solutions were
characterized on an Anton-Paar Rheometer with the ridged parallel plate
geometry to prevent slip and before each set of experiments in order to remove
any measurement error due to the degradation of the physical properties over
time. Before each measurement a preshear step was included to consequently
break and reheal the fluid’s microstructure [38]. It was observed that storing
the solutions in plastic containers could lead to degradation of the yield
stress by up to 30% over the course of one month, therefore care was taken to
store the prepared solutions in glass containers to avoid yield-stress
degradation.

Carbopol 940 is known as a Herschel-Bulkley fluid with a stress $\tau$ versus
shear rate $\dot{\gamma}$ dependency that can be expressed as

$\tau=\tau_{y}+K\dot{\gamma}^{n}\text{ for }\tau\geq\tau_{y},$ (6)

where $K$ and $n$ are known as the consistency index and power-law index,
respectively ([39],[40]). Flow occurs when $\tau>\tau_{y}$ with the fluid
arrested when $\tau<\tau_{y}$ [41]. Expressed in terms of viscosity,

$\eta=K\left|\dot{\gamma}\right|^{n-1}+\tau_{y}\left|\dot{\gamma}\right|^{-1}.$
(7)

We see that $n<1$ indicates a shear thinning flow and $n>1$ a shear thickening
flow. For our experiments $n<1$, and increasing shear rates $\dot{\gamma}$
decrease the effective viscosity as shown in Fig. 4b.

###  I.3 Deposition minimum $\alpha$

Figure 5:  Measured initial coalescence angles $\alpha$. Error bars represent
the deviation between the two drops’ coalescence angles. Red x’s note the
experiments considered, as they had minimal liquid/gas interfacial deviation
from the spherical cap shape and minimal deviation in coalescence angles
$\alpha$.

When yield stress drops spread on substrates, spreading arrests before drops
minimize their surface energy as by the Young-Dupré equation
$\sigma\cos(\alpha)=\sigma_{SV}-\sigma_{SL}$ [1]. The contact angle at which
spreading arrests can be predicted for our system by adapting (4) from [2] for
a 0 surface energy solid as

$\alpha\approx\cos^{-1}\left({1-0.17\frac{\tau_{y}a}{\sigma}}\right).$ (8)

A plot of the measured $\alpha$ against $\tau_{y}a$ is shown in Fig. 5.

##  II liquid bridge temporal evolution

As shown in Fig. 6 for the experiments of Fig. 1, the temporal evolution of
the coalesced drops’ width $r_{0}$ and height $h_{0}$ are found to follow
viscous scaling laws before arresting due to the drop’s yield stress.

Additionally in the linear region of bridge evolution, it can be seen in Fig.
7 that in the vicinity of $h_{0}$ the liquid bridge indeed grows self
similarly at varying times as well as yield stresses.

Figure 6:  Temporal evolution of the bridge height $h_{0}$ (circles) and
radius $r_{0}$ (x’s) scaled by the footprint $a$ for two different experiments
with $\tau_{y}=28$ Pa drops. At the early stages of coalescence $h_{0}$ is
found to evolve linearly in time, similar to the viscous-dominated Newtonian
case before arresting due to the liquid’s yield stress. Similarly, the
coalesced drops’ contact line (CL) at $r_{0}$ evolves at early times as
approximately $t^{0.5}$ before transitioning at long time to a slow
relaxation, $t^{0.1}$.

Figure 7:  Bridge self similarity at varying times and yield stresses. a
$t=0.11\rightarrow 0.28$s and $\tau_{y}=2$ Pa b $t=0.07\rightarrow 0.17$s and
$\tau_{y}=14$ Pa c $t=0.11\rightarrow 0.28$s and $\tau_{y}=28$ Pa d

$t=0.38\rightarrow 0.95$s and $\tau_{y}=50$ Pa

##  III Equilibrium shape

###  III.1 Static approach

An intuitive approach to predict the arrested drop’s liquid/gas interfacial
shape would have us assume $h_{0}$ arrests when the capillary pressure $\Delta
P$ driving the coalescence no longer overcomes the yield stress $\tau_{y}$, as

$\tau_{y}=\Delta P=\frac{1}{2}\sigma\left(\frac{1}{r}-\frac{1}{R}\right)$ (9)

where $\Delta P$ is represented by the Laplace equation with $\sigma$ the
surface tension, $1/r$ the curvature of $h(x,t)$ along $x$ and $1/R$
representing the negative curvature of the liquid/gas interface along $y$.

We can approximate the suction radius of curvature $R$ as,

$R\approx\frac{h_{f}}{1-\cos{\alpha}},$ (10)

by assuming the LG interface along the $y$-axis is a circular segment with a
CL that spreads outward with an advancing contact angle equal to $\alpha$, and
then assume $r$ has the form

$r\approx\frac{h_{0}}{\tan{\alpha}}\frac{h_{drop}}{(h_{drop}-h_{0})}+\epsilon,$
(11)

correct to some small constant $\epsilon$. Here $r$ is approximated as one
half of the gap thickness between the original interface of the drop at the
height $h_{0}$, corrected by the term $h_{drop}/(h_{drop}-h_{0})$ that goes to
$\infty$ as $h_{0}\rightarrow h_{drop}$.

Substituting (11) and (10) into (9) and rearranging we find

$\frac{h_{f}}{h_{drop}}=\frac{\tan(\alpha)+\cos(\alpha)-1}{2\tau_{y}h_{drop}/\sigma+\tan(\alpha)},$
(12)

which can be approximated when $\alpha$ is small as

$\frac{h_{f}}{h_{drop}}=\frac{1}{J/\alpha+1}\text{ with
}J=\frac{\tau_{y}h_{drop}}{\sigma}.$ (13)

Here $J$ is known as the plasto-capillary number or Bingham number. For our
experiments $\sigma$ and $h_{drop}$ remained relatively constant, therefore
from (13) we can glean the intuition that the observed final arrested shape is
a function of primarily the yield stress $\tau_{y}$ and also the coalescence
angle $\alpha$, with increasing $\alpha$ or decreasing $\tau_{y}$ increasing
$h_{f}/h_{drop}$.

###  III.2 Numeric Approach

Figure 8:  a. Numerically determined $J$ from (17) with boundary conditions
(18) against experimentally measured values. $J$ can be reasonably recovered
from (15) upon prescription of the final bridge height. b. Numerically
determined $h_{f}/h_{drop}$ from (15,17) with boundary conditions
$H0(\hat{x}=0)=1$, $H1(\hat{x}=0,1)=0$ against experimentally measured values.
The value of $h_{f}$ can be fairly well recovered for $J=[0.01,0.47]$ and
$b=[0.15,0.58]$. The inset shows a comparison with the $h_{f}$ determined when
using the small angle approximation.

The pressure gradient $P_{x}$ from (1) can be approximated as

$P_{x}=\sigma\left(\frac{h_{xx}}{(1+h_{x})^{3/2}}\right)_{x},$ (14)

where $()_{x}=\mathrm{d}/\mathrm{d}x$. Here the pressure is represented as the
curvature in the x-direction without the small angle approximation. After
applying scalings for the height $\hat{h}=h/h_{drop}$ and length
$\hat{x}=x/a$, we can solve for $\hat{h}_{\hat{x}\hat{x}\hat{x}}$ obtaining

$\hat{h}_{\hat{x}\hat{x}\hat{x}}=\frac{J}{b^{3}}\frac{\left(1+b^{2}{\hat{h}_{\hat{x}}}^{2}\right)^{3/2}}{\hat{h}}+3b^{2}\frac{h_{\hat{x}}{h_{\hat{x}\hat{x}}}^{2}}{\left(1+b^{2}{\hat{h}_{\hat{x}}}^{2}\right)},$
(15)

where here $b$ represents the aspect ratio $h_{drop}/a$. It can be seen that
when $b<<1$, only the yield stress dependent term remains

$\hat{h}\hat{h}_{\hat{x}\hat{x}\hat{x}}=J/b^{3},$ (16)

In the main text, we solve (16) to determine the shape of the final bridge
height, however a solution from this governing equation whilst capturing the
shape of the final profile can be made more accurate in its prediction of the
nominal value of $h_{f}$ by solving without any small angle approximation.

We approach a solution of (15) in two ways. First we solve for $J$ using
SciPy’s ‘solve_bvp’ to solve the following system of 3 ODEs and one free
parameter $J$

$\begin{split}H_{0}^{\prime}&=H_{1},\\\
H_{1}^{\prime}&=H_{2},~{}~{}\text{and}\\\
H_{2}^{\prime}&=\left|\frac{J(1+b^{2}H_{1}^{2})^{1.5}}{(b^{3}H_{0})}+\frac{3b^{2}H_{2}^{2}H_{1}}{((1+b^{2}H_{1}^{2}))}\right|,\end{split}$
(17)

with 4 boundary conditions

$\begin{split}H0(\hat{x}=0)&=1,\\\
H0(\hat{x}=1)&=h_{f}/h_{drop},~{}~{}\text{and}\\\
H1(\hat{x}=0,1)&=0.\end{split}$ (18)

The results of solving for $J$ are shown in Fig. 8a, where good agreement can
be seen invariant of yield stress $\tau_{y}$ or coalescence angle
$\alpha\approx b$.

Figure 9:  Temporal bridge evolutions. (a) For yield stress $\tau_{y}=50$ Pa,
increasing the coalescence angle $\alpha$ decreases the coalescence velocity
and increases the arrested bridge height, however the evolution remains
approximately $h_{0}\sim t$. The inset shows the relevant phase portrait
$\hat{Y}$ against $\hat{h}$ from (20) with $h_{drop}=1$ mm. It can be seen
that for the bulk of the evolution $Y\approx h$, thereby supporting the idea
that $h_{o}\sim t$. (b) Bridge evolutions grouped by yield stress
$\tau_{y}=50$ Pa. Increasing $\tau_{y}$ decreases the coalescence velocity.
The inset shows the relevant phase portrait $\hat{Y}$ against $\hat{h}$ with
$h_{drop}=1$ mm and $\alpha=30^{\circ}$. Here also it can be seen $Y\approx h$
for the bulk of the evolution.

We can also solve (17) for the final interface shape independent of any post-
coalescence measured quantities by prescribing $J$ and removing the boundary
condition $H_{0}(\hat{x}=1)=h_{f}/h_{drop}$. In Fig. 8b we see a plot of the
numerically calculated final arrested bridge heights $h_{f}/h_{drop}$ found by
solving these 3 ODEs. Here we see that the predictive capacity of (15) versus
(16) in determining the final arrested bridge height is increased through
inclusion of the additional terms of (14).

##  IV When is $Y\approx h$?

When $Y\approx h$, the viscoplastic lubrication equations reduce to the
Newtonian case. It would be useful to understand when during the bridge
evolution $Y\approx h$. Near $h_{0}$ we find $\left|\partial P/\partial
x\right|$ can be written in terms of the coalescence angle $\alpha$ and the
minimum bridge height $h_{0}$ as

$\left|\frac{\partial P}{\partial
x}\right|\sim\sigma\frac{\tan^{2}(\alpha)}{h^{2}_{0}}\frac{(h_{drop}-h_{0})}{h_{drop}},$
(19)

by using the approximation $P\approx\sigma/r$ as $R\rightarrow\infty$ with
$\Delta x$ taken to be $h_{0}/\tan(\alpha)$. From the viscoplastic lubrication
equations $Y$ can then be rewritten as

$\hat{Y}=\hat{h}-\frac{J}{\tan^{2}({\alpha})}\frac{\hat{h}^{2}}{(1-\hat{h})}$
(20)

where $\hat{Y}=Y/h_{drop}$, $\hat{h}=h_{0}/h_{drop}$ and $J$ is the Bingham
number $\tau_{y}h_{drop}/\sigma$. Solving for $\hat{h}$ when $Y=0$ recovers
two equilibrium points, $\hat{h}=0$ representing the beginning of the
coalescence and $\hat{h}=1/(J/\tan^{2}{\alpha}+1)$ representing the end of the
coalescence. This second point is strikingly similar to (13), however a
difference arises as we assume $R\rightarrow\infty$.

Fig. 9a shows the results of plotting $\tau_{y}=50$ Pa experiments with
varying $\alpha$. It can be seen that increasing $\alpha$ decreases the
coalescence velocity, however the bridge evolution remains approximately
$h_{0}\sim t$ during the bulk of the evolution. The inset shows the results of
plotting the phase portrait $\hat{Y}$ against $\hat{h}$ from (20). It can be
seen as $\alpha$ increases the final arrested height increases, but for the
bulk of the evolution $Y$ remains $\approx h$. Fig. 9b shows the effect of
$\tau_{y}$ on the bridge evolution. Scaling time to recover a viscosity we can
collapse the effect of $\alpha$ on the evolution and group the data by yield
stress $\tau_{y}$. Increasing $\tau_{y}$ decreases the coalescence velocity as
the effective viscosity increases, however similar to the varying $\alpha$
case we still recover a region where $h_{0}\sim t$.