#  Qontsum: On Contrasting Salient Content for Query-focused Summarization

Sajad Sotudeh<EMAIL_ADDRESS>IR Lab, Georgetown
UniveristyWashington D.C., USA and  Nazli Goharian<EMAIL_ADDRESS>IR Lab, Georgetown UniveristyWashington D.C., USA

(2023)

###### Abstract.

Query-focused summarization (QFS) is a challenging task in natural language
processing that generates summaries to address specific queries. The broader
field of Generative Information Retrieval (Gen-IR) aims to revolutionize
information extraction from vast document corpora through generative
approaches, encompassing Generative Document Retrieval (GDR) and Grounded
Answer Retrieval (GAR). This paper highlights the role of QFS in Grounded
Answer Generation (GAR), a key subdomain of Gen-IR that produces human-
readable answers in direct correspondence with queries, grounded in relevant
documents. In this study, we propose Qontsum, a novel approach for QFS that
leverages contrastive learning to help the model attend to the most relevant
regions of the input document. We evaluate our approach on a couple of
benchmark datasets for QFS and demonstrate that it either outperforms existing
state-of-the-art or exhibits a comparable performance with considerably
reduced computational cost through enhancements in the fine-tuning stage,
rather than relying on large-scale pre-training experiments, which is the
focus of current SOTA. Moreover, we conducted a human study and identified
improvements in the relevance of generated summaries to the posed queries
without compromising fluency. We further conduct an error analysis study to
understand our model’s limitations and propose avenues for future research.

††journalyear: 2023††conference: The First Workshop on Generative Information
Retrieval; July 27, 2023; Taipei, Taiwan††booktitle: The First Workshop on
Generative Information Retrieval (Gen-IR@SIGIR23), July 27, 2023, Taipei,
Taiwan††copyright: rightsretained

##  1\. Introduction

In recent years, the proliferation of digital content has led to an explosion
of data, making it increasingly challenging to extract relevant information
and insights from large amounts of text. Summarization, the process of
condensing large volumes of text into shorter, more manageable summaries, has
emerged as a promising solution to this problem. Among different types of
summarization, query-focused summarization (QFS) (Dang, 2005; Daumé and Marcu,
2006; Katragadda and Varma, 2009; Badrinath et al., 2011; Xu and Lapata,
2020a, 2022; Laskar et al., 2022; Yuan et al., 2022) has gained significant
attention due to its ability to generate summaries tailored to specific user
queries or information needs.

QFS is situated within the larger domain of Generative Information Retrieval
(Gen-IR), an area that seeks to revolutionize information extraction from vast
document corpora by employing generative techniques. In order to understand
the relationship between QFS and Gen-IR, it is essential to first explore the
divisions within Gen-IR. Generative Information Retrieval can be divided into
two main subfields: Generative Document Retrieval (GDR) and Grounded Answer
Generation (GAR). GDR is concerned with retrieving a ranked list of documents
w.r.t a given query, while GAR focuses on generating specific answers,
grounded on relevant documents 111Definitions of Gen-IR, GDR, and GAR are
taken from the Gen-IR workshop at https://coda.io/@sigir/gen-ir. In this
context, QFS can be viewed as a method associated with GAR. QFS is essential
in a wide range of natural language processing applications, such as
information retrieval (Metzler et al., 2007; tau Yih and Meek, 2007; Gu et
al., 2016; Chen et al., 2020b; Althammer et al., 2021; Park and Ko, 2022), and
document analysis (Goldstein-Stewart et al., 1999; Lin and Hovy, 2003; Gupta
et al., 2007). In information retrieval, for example, QFS can help users
quickly and effectively identify relevant information from large volumes of
search results. Similarly, in document analysis, query-focused summarization
can provide decision-makers with key insights and information necessary for
making informed decisions (Hsu and Tan, 2021).

By condensing large volumes of information into a focused summary that
directly addresses the user’s information need, QFS enhances the
informativeness of generated responses. This ability is particularly
beneficial for tasks such as question answering, chatbots, and personal
assistants, where the quality and relevance of generated responses are
critical for user satisfaction and engagement. Incorporating QFS into Gen-IR
systems can help address the issue of generating excessively long or
irrelevant responses, a common challenge in natural language response
generation. By providing users with more concise and targeted information, QFS
can improve the effectiveness of Gen-IR systems, ultimately enhancing the user
experience.

Figure 1. A sample query/question with its human-written and system-generated
responses (i.e., summaries) from the QMSum benchmark. Yellow highlights in the
summaries signify text spans that are directly relevant to the query. In
contrast, gray highlights denote spans that are not relevant to the query, as
produced by the state-of-the-art (SOTA) summarizer. Relevance is evaluated
based on the system’s ability to generate query-related responses, without
being limited to solely the information in the human-written summary.

Existing methods for query-focused summarization can be broadly categorized
into extractive, abstractive, and hybrid approaches. Extractive approaches
involve selecting and aggregating the most important information from the
input document based on various heuristics or statistical models. Abstractive
approaches, on the other hand, aim to generate a summary that captures the
essence of the input document in a more human-like manner, often by generating
novel phrases or sentences. Hybrid approaches aim to combine the strengths of
both extractive and abstractive methods. Despite significant progress in
recent years, these approaches still face several challenges, such as the
difficulty of capturing the nuances of human language and understanding the
contextual information in the input document. Moreover, ensuring the
faithfulness and relevance of generated summaries remains a critical
challenge, particularly in scenarios where the input document contains complex
information or where the query is granular. In this paper, we focus on
enhancing the relevance of the generated summary to the query, a challenge
that the state-of-the-art systems often encounter, as shown in Figure 1.

In the context of an ever-growing volume of information, long-input
summarization has become increasingly important, as it can facilitate the
efficient and effective extraction of key insights from extensive documents.
Thus, our study opts to focus on long-input QFS, rather than short-input QFS.
To address the challenges and limitations posed by current techniques in the
domain of query-focused summarization, this study introduces an innovative
method that employs contrastive learning (Chopra et al., 2005) to distinguish
between pertinent and non-pertinent content within the input document. The
proposed methodology seeks to incorporate the most salient spans of the source
document into the summarization process by juxtaposing them against negative
regions that are typically the focus of attention for the summarization
system. We posit that the incorporation of contrastive learning into query-
focused summarization can bolster the model’s capacity to discern relevant
information from input documents, thereby yielding more relevant and pertinent
summaries. Our empirical findings, based on two long-input datasets,
demonstrate either enhanced or comparable performance in relation to the
previous state-of-the-art, while simultaneously reducing computational cost.
In summary, this investigation presents a promising new avenue for augmenting
the relevance of query-focused summarization and enriching the quality of
generated responses in Generative Information Retrieval and Query-focused
Summarization. Our contributions are twofold:

  * •

We present a novel QFS system that combines contrastive learning with state-
of-the-art techniques, surpassing or achieving the performance of existing
approaches on relevant benchmark datasets.

  * •

We conduct an evaluation of our model, including automatic performance
comparisons, human assessment, and error analyses, to gain insights into the
model’s strengths, limitations, and potential future research directions.

##  2\. Related work

Query-focused summarization (QFS) is a specialized area within automatic text
summarization, where the objective is to generate summaries tailored to
address a specific query by selecting and condensing relevant information from
a given document collection. In the early stages of QFS research, extractive
methods were the predominant techniques used to generate summaries. These
methods identify salient sentences or passages from the input documents and
combine them to form a summary without altering the original text. One popular
approach is query expansion, which enriches the user query with additional
terms to better capture the user’s intent and improve the relevance of the
extracted summaries (Metzler et al., 2007; Sriram et al., 2010). Another
method is query-biased summarization in information retrieval, which ranks
sentences based on their similarity to the user query and their importance
within the document (Tombros and Sanderson, 1998; Metzler and Kanungo, 2008;
Gu et al., 2012). For instance, Graph-based methods, such as LexRank (Erkan
and Radev, 2004) and TextRank (Mihalcea and Tarau, 2004), represent documents
as graphs, where nodes correspond to sentences and edges indicate the
similarity between them. By analyzing the graph structure, these methods
identify and extract key sentences based on their centrality and relevance to
the user query.

The field of query-focused summarization (QFS) has experienced significant
advancements due to the incorporation of deep learning and neural networks.
Abstractive methods have become increasingly popular in QFS research, as they
generate flexible and coherent summaries by paraphrasing and rephrasing the
input text. Various techniques, such as sequence-to-sequence models (Sutskever
et al., 2014), attention mechanisms (Cho et al., 2014), and transformer-based
architectures (Vaswani et al., 2017), have been employed to improve QFS model
performance by assigning weights to input tokens. Pre-trained language models,
like Bart (Lewis et al., 2020), and transformer-based architectures form the
foundation of state-of-the-art (SOTA) models for QFS.

The availability of high-quality query-focused summarization datasets, such as
QMSum (Zhong et al., 2021b) and SQuALITY (Wang et al., 2022), has fueled
increased interest in QFS. Recent research has explored extract-then-generate
methods, including passage/answer retrieval (Baumel et al., 2018; Laskar et
al., 2020; Su et al., 2021; Zhong et al., 2021b; Wang et al., 2022). Other
methods involve adapting attention mechanisms to query-focused summarization
through query-utterance interactions (Liu et al., 2023), adapting SOTA
summarizers like Bart to long-input QFS through overlapping segment-based
summarization (Vig et al., 2022), and replacing full attention with block-wise
attention (Beltagy et al., 2020). Additional approaches include generating
pseudo queries for ranking evidence sentences (Xu and Lapata, 2020b),
employing data augmentation techniques (Pasunuru et al., 2021), and pre-
training language models with dialog-specific (Zhong et al., 2021a) and
question-driven (Pagnoni et al., 2022) objectives. These advancements in QFS
research have contributed to the development of more effective and coherent
summarization models, paving the way for further innovations in natural
language understanding and generation.

Different than prior work, our approach incorporates contrastive learning to
help the model perform generation from relevant regions of the input document,
with the goal of increasing the summary’s relevance to the query. To the best
of our knowledge, this is the first attempt to utilize contrastive learning in
QFS. By leveraging this technique, we aim to guide the model to focus on the
most relevant parts of the input document, thereby producing summaries that
are better aligned with the query.

Figure 2. Overview of Qontsum model. The schema inside the dashed line
represents the SegEnc baseline, which performs segment-based encoding,
concatenates the encoding into a dense embedding space, and utilizes a decoder
to attend to this embedding. Our model builds upon the SegEnc baseline: we
first segment the input documents into fixed-length overlapping segments and
then feed these segments into the Encoder network. The Segment Scorer, placed
on top of the encoder, pools the segment representations and projects them
into segment probabilities. Given the probabilities of the segments and the
gold segments (in green), we generate positive and negative samples for
contrastive learning. Both types of samples are fed into a shared-weight
decoder along with the query, enabling the summarization model to learn to
focus on relevant segments. The entire network is finally optimized using
three losses: generation, classification, and contrastive which are all
combined with a balancing $\lambda$ parameter.

##  3\. Background: Segment-based long summarization

Prior to presenting our contrastive learning approach, we present an overview
of the SOTA Segment Encoder (SegEnc) summarization model, as proposed by Vig
et al. (2022), which serves as the backbone summarization component of our
framework. In the SegEnc model, the source document is initially divided into
overlapping segments of fixed length 222We use 512 tokens, with each segment
exhibiting a 50% overlap with its adjacent segment (empirically determined).,
each of which is appended to the query and subsequently encoded using a
conventional Transformer model, such as Bart. In this context, the encoder
focuses on both query and segment tokens through its self-attention mechanism
(Vaswani et al., 2017), and generates token representations for each segment
that are attuned to the query. After processing all segments with the shared
encoder, their representations are merged into a single continuous embedding
sequence, which is then fed into the decoder to create a summary tailored to
the query (i.e., the response). Owing to the absence of cross-attention
between encoded segments, the attention mechanism’s scalability is linearly
proportional to the number of segments and, consequently, the length of the
input document. Nevertheless, the decoder is capable of attending to all
encoded segments jointly, allowing the summarizer to operate in an end-to-end
manner.

The generation loss for this network is calculated using cross-entropy, which
measures the difference between the predicted summary output and gold summary
as follows:

(1) $\mathcal{L}_{\texttt{gen}}=-\sum_{t=1}^{T}\log p(y_{t}|y_{<t},x)$

where $T$ is the length of the generated summary, $y_{t}$ is the $t$-th token
in the generated summary, $y_{<t}$ represents the previously generated tokens,
and $x$ is the input text. The generation loss measures the negative log-
likelihood of generating the ground truth summary given the input text, and
the goal during training is to minimize this loss.

##  4\. Methodology

This section introduces our proposed approach for query-focused summarization
using contrastive learning, which involves applying Generative Information
Retrieval techniques to the task of text summarization. The key aspect of our
approach is based on the concept of contrastive learning, which has been
successfully applied in various other domains (Hoffer and Ailon, 2014; Bell
and Bala, 2015; Khosla et al., 2020; Gao et al., 2021). Our approach, named
Query-focused Contrastive Summmarization (i.e., Qontsum), leverages
contrastive learning to distinguish between positive and negative instances
during training, enabling discrimination between important (positive) and
unimportant (negative) content in the summarization process. Figure 2 provides
a detailed illustration of our model. I the following sections, we provide a
systematic breakdown of our proposed approach.

###  4.1. Segment Scoring

Our architecture incorporates a feed-forward neural network as a critical
component for scoring encoded segments in an extractive framework. This
network plays a fundamental role in dynamically selecting negative samples
during the training process. Specifically, the encoded segment
representations, which are associated with the ¡s¿ representations as the
classification head (i.e., [CLS]), are fed into a feed-forward neural network
with a Sigmoid classifier.

(2) $p_{i}=\sigma(h_{i}W_{i}+b_{i})$

in which $p_{i}$ represents the extraction probability of the $i$-th segment,
$\sigma$ is the Sigmoid activation function, and $W_{i}$ and $b_{i}$ are the
learnable parameters. After obtaining the segment probabilities, we minimize
the cross-entropy function as the classification loss, given the ground-truth
labels for each segment:

(3)
$\mathcal{L}_{\texttt{cls}}=-\sum_{i=1}^{S}y_{i}\log(p_{i})+(1-y_{i})\log(1-p_{i})$

where $S$ is the number of segments, $y_{i}$ is a binary label indicating
whether the $i$-th segment is contributive in summary or not, and
$\mathcal{L}_{\texttt{cls}}$ is the classification loss.

###  4.2. Contrastive Learning Framework

In this subsection, we describe the contrastive learning framework that forms
the basis of our proposed approach to query-focused summarization. The main
objective of contrastive learning is to distinguish between similar (positive)
and dissimilar (negative) instances by learning to map them to different
regions in the embedding space. Our framework selects positive (fixed) and
negative (dynamically) instances during the training process, which allows the
model to learn effective representations that can discriminate between
important and unimportant content for summarization.

####  4.2.1. Positive and Negative instance selection

To generate positive instances, we first check if ground-truth labels are
available in the dataset. If not, we generate supervised labels, which will be
further explained in Section 5. We examine each segment $S_{i}$ to determine
if it contains any gold spans or sentences. If it does, we add the entire
segment to our positive contrastive set. For negative instances, we rely on
the segment-scoring mechanism introduced in the previous subsection.
Specifically, we consider the segments with high extraction probabilities
($p_{i}$) conditioning that they are non-gold as negative instances. We
continue selecting negative instances until the number of selected negative
segments matches the number of positive segments already selected. This
selection process is dynamic and happens during training, meaning that the
model is continually being challenged to distinguish between important and
unimportant content. By incorporating both positive and negative instances, we
ensure that our model is trained to focus on meaningful information, leading
to better performance in downstream tasks. After identifying the positive and
negative instances (i.e., segments), we feed them through a shared-weight
decoder to generate a transformed representation for loss computations.

####  4.2.2. Computing contrastive loss

To achieve our objective of maximizing the similarity between positive
instances and their corresponding query embeddings while minimizing the
similarity between negative instances and query embeddings, we opt to use the
InfoNCE loss for contrastive learning, which is a popular contrastive learning
technique introduced by van den Oord et al. (2018) and has been proven to
exhibit robust performance on QA tasks in recent studies (Liu and Zhou, 2021;
Caciularu et al., 2022; Hu et al., 2022).

A few studies have acknowledged the significance of feature transformation
employing a small neural network for mapping representations to a space where
contrastive loss is computed (Chen et al., 2020a; He et al., 2020). Drawing
inspiration from these works, and to address the challenge of decoder outputs
being optimized for the token prediction task rather than semantic similarity
within our framework, we utilize a multi-layer perceptron (MLP) coupled with
Batch Normalization (BN)(Ioffe and Szegedy, 2015) and Rectified Linear Unit
(ReLU) activation function(Agarap, 2018) to transform the decoder outputs
(i.e., logits) into a new embedding space, as described below:

(4) $\displaystyle h_{i}$
$\displaystyle=\texttt{ReLU}(\texttt{BN}(W_{i}d_{i}+b_{i}))$ $\displaystyle
s_{i}$ $\displaystyle=W_{j}h_{i}+b_{j}$

in which $d_{i}$ is the decoder outputs (i.g., logits), $W_{i}$, $W_{j}$,
$b_{i}$, and $b_{j}$ are trainable parameters.

We continue to use cosine similarity as the pairwise similarity function to
measure the similarity between contrastive and query embeddings as follows:

(5)
$\text{sim}(s_{i},q)=\frac{\textbf{q}^{\top}\textbf{s}_{i}}{|\textbf{q}||\textbf{s}_{i}|}$

where q is the query embedding (i.e., summary generated from the entire input
segments in our framework), $\textbf{s}_{i}$ is the contrastive instance
(i.e., positive/negative) embedding, and $sim$ is the cosine similarity
between the query and the contrastive instances. It is worth noting that since
the output of the decoder network for q and $\textbf{h}_{i}$ are token
sequences, we compute the token-wise cosine similarity. We then define the
InfoNCE contrastive loss as follows:

(6)
$\mathcal{L}_{cont}=-\log\left(\frac{e^{\text{sim}(s^{+},q_{i})/\tau}}{\displaystyle\sum_{s\in
S}e^{\text{sim}(s,q)/\tau}}\right)$

where $sim(\cdot)$ denotes the similarity score between the query and the
positive ($s^{+}$) and a contrastive ($s$) instances, $S$ is the set of
contrastive instances, and $\tau$ is a temperature hyperparameter that
controls the concentration of the probability distribution over the instances.

The InfoNCE loss encourages the model to learn embeddings that are more
similar for positive instances and less similar for negative instances,
effectively improving the model’s ability to discriminate between important
and unimportant content. The temperature parameter $\tau$ controls the
sharpness of the distribution, with lower values of $\tau$ leading to a more
focused distribution around the highest-scoring positive instance and higher
values resulting in a smoother distribution across all instances. This
parameter can be fine-tuned to strike a balance between focusing on the most
relevant content and being robust to a diverse set of instances.

###  4.3. Joint Training Objective

We train our model by optimizing a joint objective that combines the
generation loss, classification loss, and contrastive loss, introduced earlier
as follows:

(7)
$\mathcal{L}=\lambda_{0}\mathcal{L}_{\texttt{gen}}+\lambda_{1}\mathcal{L}_{\texttt{cls}}+\lambda_{2}\mathcal{L}_{\texttt{cont}}$

where $\lambda$ parameters balance the learning between the three tasks, and
their sum is equal to 1.

##  5\. Experimental Setup

In this section, we elaborate on the datasets, and baseline models that we
have utilized in our experiments. We also provide training and implementation
details.

###  5.1. Datasets

QMSum (Zhong et al., 2021b) is a query-focused meeting summarization dataset
consisting of 1,808 query-focused summaries extracted from 232 multi-turn
meetings 333A “multi-turn dataset” is a collection of conversational data that
involves multiple rounds of dialogue between two or more participants such as
those in the meetings.  across various domains, including product design,
academia, and political committees. Additionally, the dataset includes
annotations such as topic segmentations and highlighted text spans associated
with reference summaries. The dataset is split into 1,257, 272, and 279
instances for training, validation, and testing, respectively. The length
statistics represent the average source length of 9K tokens and summary length
of 70 tokens. Given the widespread use of QMSum in prior research, we believe
it serves as a valuable benchmark for comparison in our work.

SQuALITY (Wang et al., 2022) is a collection of question-focused abstractive
summarization data comprising 100 stories, 500 questions, and 2000 summaries.
Each question in the dataset is accompanied by 4 reference summaries, written
by trained writers who reviewed each other’s work to ensure the data is of
high quality. The dataset provides 39/25/36 (train/validation/test) splits,
which is equivalent to 195/125/180 document-question pairs. The documents and
summaries have a length of 5.2K and 237 tokens on average, respectively.

###  5.2. Baselines

We experiment with different strong and state-of-the-art summarization models,
which are outlined as follows.

  * -

Bart is a Transformer-based encoder-decoder (Vaswani et al., 2017) model
pretrained on a token infilling and sentence permutation pretraining
objectives (Lewis et al., 2020). Bart netowrk has a maximum input length of
1024 tokens, hence the documents are truncated to fit this baseline.

  * -

Pegasus is an abstractive summarizer that is pretrained using a task-specific
objective for summarization (Zhang et al., 2020). Specifically, it is
pretrained to predict the masked-out sentences as Gap Sentence Prediction
(GSP) objective. The inputs are truncated to 2048 tokens to fit this baseline
(Wang et al., 2022).

  * -

Bart+DPR is an extract-then-summarize method amongst the baselines on SQuALITY
dataset as proposed by Wang et al. (2022). Unlike Bart, it first retrieves the
sentences that are most relevant to the question and concatenates them to form
the input to the abstractive summarizer.

  * -

LED is Longformer Encoder-Decoder model, amongst the abstractive summarization
systems suited for long document summarization tasks (Beltagy et al., 2020).
This model modifies the conventional self-attention mechanism of Transformers
architecture for a more efficient and robust scale-up to long documents.

  * -

Bart-LS is an extension of Bart model for long document summarization,
adapting it for long-sequence inputs. In particular, it replaces the full
attention mechanism in Transformers with pooling-augmented block-wise
attention, and pretrains the model with a masked-span prediction task with
spans of varying lengths.

  * -

DialogLM is a pre-trained neural model for understanding and summarizing long
dialogues. It uses an encoder-decoder architecture and a window-based
denoising pre-training task to equip the model with the ability to reconstruct
noisy dialogue windows.

  * -

SegEnc is a state-of-the-art abstractive summarizer, tailored to address
query-focused long-input documents. It divides the input into fixed-length
segments, encodes them separately, and then enables the decoder to jointly
attend to these segments. Two different configurations of the SegEnc model are
employed: (1) the default checkpoint, which is the bart-large architecture;
and (2) the Wikisum Pre-Finetuned model (aka SegEnc-W), which performs pre-
finetuning of the SegEnc model on the Wikisum dataset.

  * -

Socratic is a pre-training framework that uses a question-driven objective
specifically designed for controllability in summarization tasks (Pagnoni et
al., 2022). This framework trains a model to generate and respond to relevant
questions (i.e., ask & answer) in a given document and achieves the SOTA on
QMSum and SQuALITY datasets. Socratic is finetuned on SegEnc summarization
system. We also include a version of Pegasus that is pretrained on Book3 pre-
training dataset, and fine-tuned using SegEnc approach.

  * -

TUQFS is a query-aware approach that employs joint modeling of tokens and
utterances through Token-Utterance Attention (Liu et al., 2023). This
technique integrates both token-level and utterance-level query relevance into
the generation process via an attention mechanism. In our experiments, the
TUQFS model is considered one of the state-of-the-art methods for the QMSum
dataset.

###  5.3. Training and implementation details

Our method is implemented using Huggingface Transformers. For the model’s
hyperparameters, we set a learning rate of 5e-5 with a weight decay of 0.01
and train the model for 10 epochs, with validation conducted at the end of
each epoch. We select the checkpoint that achieves the highest mean Rouge
scores for inference time, a strategy similar to SegEnc (Vig et al., 2022). To
compute the contrastive loss, we tune the $\tau$ parameter from the set of
{0.2, 0.4, 0.6, 0.8} and fix it at 0.6 and 0.8 for the QMSum and SQuALITY
datasets, respectively. We tried different values of $\lambda$ in joint
learning and fixed them on ($\lambda_{0}$=0.6, $\lambda_{1}$=0.2,
$\lambda_{2}$=0.2) for both datasets.

In the SQuALITY dataset, unlike the QMSum dataset, there are no human-
annotated ground-truth spans for each query-summary pair. To provide
supervised span labels, we adopt a method similar to that used in (Song et
al., 2022) and label the input segments based on the word bigram overlap
between the segment and summary. If a segment has six or more common bigrams
with the summary, it is labeled as positive; otherwise, it is labeled as
negative.

##  6\. Experiments

In this section, we present the automatic evaluation results, as well as the
findings from a human study conducted to compare the system-generated outputs
against each other.

###  6.1. Automatic results

The performance of various summarization models on the QMSum and SQuALITY
benchmarks was evaluated using the Rouge and BertScore metrics, as displayed
in Tables 1 and 2, respectively. Our proposed method, Qontsum, outperforms the
majority of the baselines across all metrics in the QMSum benchmark. When
compared to the Socratic Pret. (Pagnoni et al., 2022) as the strongest
baseline, our system achieves improvements of +0.36 (1%) Rouge-1, +0.52 (1.5%)
Rouge-L, and +0.09 (0.1%) BertScore points (relative improvements). Although
our method falls short by 0.24 points in Rouge-2 compared to the Socratic
Pret., it is crucial to emphasize that our approach outperforms the state-of-
the-art Socratic Pret. model.

In terms of computational cost, our proposed Qontsum model offers a distinct
advantage over the Socratic model. Specifically, our model leverages a
contrastive learning strategy that obviates the need for extensive pre-
training, which typically requires a significantly larger dataset and
consequently, leads to increased computational overhead. The Socratic Pret. 1M
model and Socratic Pret., for instance, underwent an extensive pre-training
phase involving 1 million, and 30 million instances from the Book3 collection.
In contrast, our Qontsum model only employs contrastive learning during the
fine-tuning stage, which uses a significantly smaller dataset. This difference
in approach has a direct effect on computational efficiency: the drastically
reduced dataset size in our model’s training phase reduces the computational
cost significantly. Despite this reduction in training complexity, our model
was able to achieve improved or at least comparable results across all metrics
when compared to Socratic. These results not only illustrate the efficiency of
our model but also open a new perspective on how to construct more
computationally efficient models for Query-focused Summarization tasks.

| RG-1 | RG-2 | RG-L | BS
---|---|---|---|---
Bart (Lewis et al., 2020) | 29.20 | 6.37 | 25.49 | -
Bart-LS (Xiong et al., 2022) | 37.90 | 12.10 | 33.10 | -
DialogLM (Zhong et al., 2021a) | 34.00 | 9.20 | 30.00 | -
TUQFS (Liu et al., 2023) | 37.99 | 13.66 | 33.36 | -
SegEnc (Vig et al., 2022) | 37.05 | 13.03 | 32.62 | 87.44
\+ Wikisum Pre-Finetuned (Vig et al., 2022) | 37.80 | 13.43 | 33.38 | -
\+ Socratic Pret. 1M (Pagnoni et al., 2022) | 37.46 | 13.32 | 32.79 | 87.54
\+ Socratic Pret. (Pagnoni et al., 2022) | 38.06 | 13.74 | 33.51 | 87.63
Qontsum (ours) | 38.42 | 13.50 | 34.03 | 87.72
Table 1. Performance of various summarization systems in terms of Rouge (RG) and BertScore (i.e., BS) on QMSum dataset. The top and middle boxes are reported performances from previous works (Zhong et al., 2021b; Vig et al., 2022; Pagnoni et al., 2022). While Socratic Pret. is the strongest baseline, it was not released at the time of writing this paper; hence, we reproduced SegEnc \+ Wikisum Pre-Finetuned for human evaluation purposes. | RG-1 | RG-2 | RG-L | BS
---|---|---|---|---
LED (Beltagy et al., 2020) | 27.7 | 5.9 | 17.7 | -
Pegasus (Zhang et al., 2020) | 38.2 | 9.0 | 20.2 | -
Bart (Lewis et al., 2020) | 40.2 | 10.4 | 20.8 | -
Bart + DPR (Wang et al., 2022) | 41.5 | 11.4 | 21.0 | -
SegEnc (Vig et al., 2022) | 45.68 | 14.51 | 22.47 | 85.86
\+ Pegasus Pret. (Pagnoni et al., 2022) | 45.78 | 14.43 | 22.90 | 85.94
\+ Socratic Pret. (Pagnoni et al., 2022) | 46.31 | 14.80 | 22.76 | 86.04
Qontsum (ours) | 45.76 | 14.27 | 24.14 | 86.07
Table 2. Performance of various summarization systems in terms of Rouge (i.e.,
RG) and BertScore (i.e., BS) metrics on SQuALITY benchmark. The reported
results are directly taken from the corresponding papers (Wang et al., 2022;
Pagnoni et al., 2022).

Similarly, the Rouge-L score of our proposed Qontsum model demonstrates a
significant improvement compared to the SOTA baselines on the SQuALITY dataset
(Table 2), resulting in an increase of 1.24 (5%) points (relative
improvements). However, there is a relatively slighter decrease of 0.55 points
(Rouge-1) and 0.53 (Rouge-2) when compared to the best baseline results. We
hypothesize that the relatively lower scores in Rouge-1 and Rouge-2 can be
attributed to the absence of gold spans, which were provided by humans in the
QMSum experiments, used in the training of our Qontsum model. This finding
suggests that there is room for improvement by employing more sophisticated
techniques to enhance positive segment labeling, potentially leading to better
performance in the Rouge-1 and Rouge-2 metrics.

Figure 3. The effect of temperature parameter (infoNCE loss) to the overall
performance of the model over two datasets.

The impact of the temperature parameter (infoNCE loss) on system performance
is demonstrated in Figure. 3. As depicted, the mean Rouge scores for both
QMSum and SQuALITY datasets rise with an increase in the temperature
parameter. In the case of the QMSum dataset, the score noticeably ascends from
0.4 to 0.6, then experiences a minor decline at 0.8, thereby suggesting that
the most effective parameter value could be approximately 0.6. In contrast,
the SQuALITY dataset displays a steady, though marginal, augmentation in the
score with increasing parameter values, hinting at a more linear correlation.
Generally, across all temperature parameters, the QMSum dataset exhibits a
slightly superior mean ROUGE score compared to the SQuALITY dataset.

###  6.2. Human study

Many previous studies have acknowledged the limitations of automatic
evaluation metrics and their correlation with human judgments. To gain a
better understanding of system-generated summary qualities and provide a basis
for error analysis, we aim to conduct a human study on randomly sampled
summaries. This approach will allow us to obtain insights into the strengths
and weaknesses of our proposed Qontsum model and identify areas where it can
be improved. By comparing the results of the human study with the automatic
evaluation metrics, we can gain a deeper understanding of the performance of
our model and make informed decisions on how to optimize it further. The
results of this human study will complement the experimental findings
presented in this paper and contribute to a more comprehensive evaluation of
our proposed summarization model.

In order to conduct the human study, we randomly selected a list of 50
meeting-query instances from the QMSum dataset and provided the human
annotator with summaries generated by three different methods: human-written,
SegEnc-W generated, and Qontsum generated. Due to the typically lengthy nature
of meeting transcripts, which often exceeded 9K tokens in our study, we only
provided the gold spans relevant to the given query during our evaluation
process. This allowed the annotator to focus specifically on the information
that was most important and directly related to the query, without being
overwhelmed by extraneous information. To avoid any bias, we shuffled the
order in which the summaries were presented for evaluation. We then defined
three qualitative metrics for the evaluation process, which are listed below,
and each case was scored on a Likert scale ranging from 1 (best) to 5 (worst).

  * -

Fluency:  This metric measures the extent to which the summary is fully
understandable (score of 5) or completely gibberish (score of 1);

  * -

Relevance:  This metric quantifies how well the summary (even short responses
within it) aligns with the given query, with a score of 5 for high relevance
and 1 for low 444Note that for the relevance metric, we do not consider the
information within the ground-truth summary, but rather focusing solely on
assessing the quality of the summary based on its relevance to the query at
hand.;

  * -

Faithfulness: This metric measures the extent to which the materials produced
in the summary are supported (score of 5) or not supported (score of 1) within
the meeting.

Method | Fluency | Relevance | Faithfulness
---|---|---|---
Human-written | 4.34 | 4.72 | 4.85
SegEnc-W (Vig et al., 2022) | 4.22 | 3.84 | 3.52
Qontsum (ours) | 4.23 | 3.96 | 3.42
Table 3. Results of the human study on 50 evaluation samples from the QMSum
dataset, comparing system-generated summaries with human-written ones. While
there is parity between human performance and system performances, our
proposed Qontsum method improves the relevance metric over the SegEnc-W
baseline, without sacrificing fluency.

The results of the human study presented in Table 3 show that our proposed
Qontsum method achieves a comparable level of fluency to human-written
summaries, which is justifiable considering the current state-of-the-art
abstractive summarization systems given their strong pre-training ability.
However, there is a clear gap between the performance of systems and that of
human-written summaries on the qualitative metrics. Despite this, our Qontsum
method outperforms the SegEnc-W baseline, particularly on relevance, where it
achieves an average score of 3.96 compared to 3.84 for SegEnc-W. This
demonstrates that our model is better at capturing the most important and
relevant information in the meeting transcripts and summarizing it
effectively, without sacrificing fluency over the baseline system. We also see
a gap between the relevance metric of human-written summaries and system-
generated summaries, which might be due to the inclusion of irrelevant spans
of information in the system-generated summaries compared to the human-written
summaries that have been written from the gold spans. Despite this, our
Qontsum method shows promise in improving the relevance metric. However, there
is still a need for further research to improve the relevance and faithfulness
metric, especially in domain-specific contexts like meeting transcripts. The
observed gap in faithfulness between human performance and summarization
systems, as also pointed out in previous studies (Zhong et al., 2021b; Vig et
al., 2022), might be attributed to the challenges of understanding and
representing natural language nuances in domain-specific contexts like meeting
transcripts.

Figure 4. A sample case from the error analysis study. As demonstrated, while
Qontsum was able to capture a few relevant pieces of information (highlighted
in the summary), it failed to mention critical points, such as the product’s
“technological innovation”. Additionally, the sample case exposes a common
error in our generated summaries, which is the repetition of words or phrases.
In this case, the product’s “easy-to-use” nature is referenced multiple times
in the summary, leading to a lack of fluency. On the other hand, while the
baseline method was able to capture useful information, it suffered from a
transcripts copying error in the final truncated sentence. These findings
indicate that there is still room for improvement in both our proposed method
and the baseline method. Further research could explore ways to address these
limitations and improve the quality of the generated summaries.  Figure 5. A
sample case from the error analysis study. As shown, the Qontsum-generated
summary has taken a partial phrase (i.e., two hundred) from the transcript
(golden highlight), while SegEnc-W could bring the complete phrase (blue
highlight), although the material does not appear in the Human-written
summary. The purple highlight represents instances of transcript copying,
which were observed during the evaluation process and affected the fluency of
the generated summaries. This case highlights one of the limitations of our
proposed approach and suggests that future research could explore ways to
address this issue to further improve the quality of the generated summaries

###  6.3. Error analysis

In the following, we present our findings on the types of errors observed
affecting each qualitative metric, their causes, and potential solutions to
address the detected issues.

####  6.3.1. Fluency.

When evaluating our system on the fluency metric, we found that its
performance was generally comparable to that of the human-written and baseline
systems. However, in cases where our system underperformed the other systems,
we observed a couple of common errors, particularly information repetition
(Figure. 4), incoherence, trasncripts copying (Figure. 5), and speaker mix-up,
particularly on QMSum benchmark.

We note that segmenting the input documents could have a negative impact on
fluency in summarization. Specifically, since the segmentation is performed on
fixed-length chunks, it may cause boundary sentences to be split across
segments, potentially affecting the coherence of the system summary. We
further observed that the current state-of-the-art abstractive summarization
models, including our proposed system, face significant challenges in
summarizing multi-turn meeting transcripts.

These challenges stem from the complexity of the task, which involves
capturing the nuances of human conversation, understanding the speaker’s
intent and context, and producing a coherent summary that reflects the key
takeaways from the discussion. Addressing these challenges will require
further research and the development of novel techniques that can effectively
handle the complexity of multi-turn summarization tasks such as the works done
by (Zhong et al., 2021b, a).

####  6.3.2. Relevance

When evaluating the system’s performance in terms of relevance, we found that
our proposed system outperformed the baseline but still fell short of human
parity. In investigating the common reasons contributing to the errors
observed in the underperformed cases, we identified several factors.

First, we found that the system sometimes struggled to accurately identify the
most important information relevant to the query, leading to summaries that
were less informative or missed critical details. This issue was more
prevalent on the SQuALITY dataset, where no span annotations were provided,
making it more challenging for the system to identify the relevant
information. However, on the QMSum benchmark, where high-quality human-
annotated text spans were provided, we observed fewer instances of this
problem. This suggests that the integration of enhanced explicit supervision
or utilization of more advanced labeling techniques may potentially contribute
to the system’s performance for selecting pertinent information, thereby
generating more relevant summaries.

Furthermore, We also observed that the system’s ability to capture the nuances
of the query was sometimes limited, especially in cases where the query was
too broad or too specific. For instance, open-ended queries such as “Summarize
the whole meeting” lack specificity and can be interpreted in multiple ways,
making it challenging for the system to determine which information is most
relevant to include in the summary, particularly in longer documents. On the
other hand, extremely specific queries, such as “What did Marketing think of
the incorporation of current fashion trends in the prototype when making a
simulation market evaluation of the new remote control?’’ can also pose
challenges for the system. In such cases, the system may struggle to
accurately interpret the underlying meaning of the query, leading to summaries
that may not fully address the information needs.

Finally, we also noted that the choice of input document could significantly
impact the system’s performance, with some documents being more challenging
for the system to summarize effectively. For instance, documents that contain
multi-turn dialogues or include idiomatic expressions that are often made
during meetings can pose significant challenges for the system. Similarly, the
use of figurative language or domain-specific terminology can also impact the
system’s ability to accurately interpret the underlying meaning of the input
document. To address these issues, we suggest further exploration of
techniques for improving the system’s ability to identify and select relevant
information, such as using more advanced supervision signals (e.g., improving
the semi-supervised labeling mechanism) or incorporating external knowledge
sources to learn in-domain terminologies more effectively.

####  6.3.3. Faithfulness.

Faithfulness is a crucial aspect of summarization, as it ensures that the
summary accurately reflects the key information from the input document.
However, it has been identified as a significant challenge for long-input
summarization tasks, including Query-Focused summarization. Prior works have
also reported similar challenges in achieving faithful summaries (Zhong et
al., 2021b, a; Vig et al., 2022) that accurately capture the relevant
information from the input document and address the information needs.

In our error analysis, we observed several instances where the proposed model
produced summaries that were not faithful to the input document. Specifically,
we observed a common error where the generated summary was written from
multiple segments that all did not align with the entire query. This could be
attributed to the model’s inability to effectively filter out irrelevant
information or to correctly interpret the underlying meaning of the query. For
example, the model may have included information that was only tangentially
related to the query, but not directly relevant to the user’s information
needs. For instance, a query such as “Did the group think the remote control
was easy to use when discussing evaluation criteria of the remote control?”
might be asked. However, when it comes to interpreting the query meaning, the
model might put more focus on only “being easy to use” and include segments
that discuss all items that should be easy to use without considering the
specific context and information within the query, such as the group’s
discussion on the remote control’s evaluation criteria. In such cases, the
model mixes up different items from the attended segments and generates a
summary that is not faithful to the input.

Another error we observed was the omission of critical details or information
that was relevant to the query, resulting in incomplete or inaccurate
summaries, as shown in Figure. 5. This could be due to the model’s limitations
in accurately interpreting and analyzing the contextual information in the
input document. Another error was that the model exhibited a deficiency in
tracking specific items that were frequently referenced during the meeting.
This resulted in a failure to accurately capture the final decision or outcome
that was made regarding a specific discussion. In other words, the model
demonstrated an inability to consistently comprehend and retain crucial
information regarding the frequently mentioned discussions on specific items.
This type of error can have significant consequences, especially in contexts
where the mentioned items play a critical role in decision-making.

To address these errors and improve the faithfulness of the system, we suggest
exploring techniques such as integrating fact-checking mechanisms.
Additionally, further research could be conducted to develop more advanced
natural language understanding techniques such as reinforcement learning with
faithfulness-focused rewards, which encourage the model to increase the
faithfulness of the generated summaries.

##  7\. Conclusion

This study proposes a novel approach for query-focused summarization (QFS)
that employs contrastive learning to enhance the relevance of the summary to
the given query. The proposed method utilizes the relevant segments of the
document as positive instances and high-scored non-gold segments during the
training process as negative instances for generating contrastive samples.
Specifically, after identifying contrastive segments, they are fed into the
abstractive summarizer for generating summaries for each contrastive instance,
which then contribute to the computation of the contrastive loss (i.e.,
InfoNCE loss function). The entire network is optimized using a joint loss
that combines generation, classification, and contrastive losses with
balancing hyperparameters. Experimental results indicate that the proposed
method outperforms existing state-of-the-art techniques and achieves a new
SOTA (with 1% and 1.5% improvements over Rouge-1 and Rouge-2, respectively on
QMSum; and 1.24 (5%) Rouge-L on SQuALITY benchmarks as compared to the
previous SOTA) or comparable performance with reduced computational overhead
without further large-scale pretraining. Furthermore, a human study analysis
demonstrates the effectiveness of the approach in terms of relevance, without
sacrificing fluency. The conducted error analysis further provides insights
into the current limitations and future research directions for QFS. The
study’s contribution adds to the growing body of research on natural language
generation and has the potential to advance the state-of-the-art in QFS.
Overall, the proposed method represents a step forward in addressing the
challenge of QFS in Gen-IR, and it is hoped that it will inspire further
research in this area.

## References

  * (1)
  * Agarap (2018) Abien Fred Agarap. 2018\.  Deep Learning using Rectified Linear Units (ReLU).  _ArXiv_ abs/1803.08375 (2018).
  * Althammer et al. (2021) Sophia Althammer, Arian Askari, Suzan Verberne, and Allan Hanbury. 2021.  DoSSIER@COLIEE 2021: Leveraging dense retrieval and summarization-based re-ranking for case law retrieval.  _ArXiv_ abs/2108.03937 (2021).
  * Badrinath et al. (2011) Rama Badrinath, Suresh Venkatasubramaniyan, and C. E. Veni Madhavan. 2011\.  Improving Query Focused Summarization Using Look-Ahead Strategy. In _European Conference on Information Retrieval_.
  * Baumel et al. (2018) Tal Baumel, Matan Eyal, and Michael Elhadad. 2018.  Query Focused Abstractive Summarization: Incorporating Query Relevance, Multi-Document Coverage, and Summary Length Constraints into seq2seq Models.  _ArXiv_ abs/1801.07704 (2018).
  * Bell and Bala (2015) Sean Bell and Kavita Bala. 2015.  Learning visual similarity for product design with convolutional neural networks.  _ACM Transactions on Graphics (TOG)_ 34 (2015), 1 – 10.
  * Beltagy et al. (2020) Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020\.  Longformer: The Long-Document Transformer.  _ArXiv preprint_ abs/2004.05150 (2020).  https://arxiv.org/abs/2004.05150
  * Caciularu et al. (2022) Avi Caciularu, Ido Dagan, Jacob Goldberger, and Arman Cohan. 2022\.  Long Context Question Answering via Supervised Contrastive Learning. In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. 2872–2879.
  * Chen et al. (2020a) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020a.  A Simple Framework for Contrastive Learning of Visual Representations. In _Proceedings of the 37th International Conference on Machine Learning_ _(ICML’20)_. JMLR.org, Article 149, 11 pages.
  * Chen et al. (2020b) Wei-Fan Chen, Shahbaz Syed, Benno Stein, Matthias Hagen, and Martin Potthast. 2020b.  Abstractive Snippet Generation.  _Proceedings of The Web Conference 2020_ (2020).
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014\.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In _Conference on Empirical Methods in Natural Language Processing_.
  * Chopra et al. (2005) Sumit Chopra, Raia Hadsell, and Yann LeCun. 2005\.  Learning a similarity metric discriminatively, with application to face verification.  _2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)_ 1 (2005), 539–546 vol. 1.
  * Dang (2005) Hoa Trang Dang. 2005\.  Overview of DUC 2005.
  * Daumé and Marcu (2006) Hal Daumé and Daniel Marcu. 2006.  Bayesian Query-Focused Summarization.  _ArXiv_ abs/0907.1814 (2006).
  * Erkan and Radev (2004) Günes Erkan and Dragomir R. Radev. 2004.  LexRank: Graph-based Lexical Centrality as Salience in Text Summarization.  _J. Artif. Intell. Res._ 22 (2004), 457–479.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple Contrastive Learning of Sentence Embeddings.  _ArXiv_ abs/2104.08821 (2021).
  * Goldstein-Stewart et al. (1999) Jade Goldstein-Stewart, Mark Kantrowitz, Vibhu Mittal, and Jaime G. Carbonell. 1999.  Summarizing text documents: sentence selection and evaluation metrics. In _Annual International ACM SIGIR Conference on Research and Development in Information Retrieval_.
  * Gu et al. (2012) Yanhui Gu, Zhenglu Yang, Miyuki Nakano, and Masaru Kitsuregawa. 2012\.  Towards Efficient Similar Sentences Extraction. In _Ideal_.
  * Gu et al. (2016) Yanhui Gu, Zhenglu Yang, Junsheng Zhou, Weiguang Qu, Jinmao Wei, and Xingtian Shi. 2016\.  A Fast Approach for Semantic Similar Short Texts Retrieval. In _Annual Meeting of the Association for Computational Linguistics_.
  * Gupta et al. (2007) Surabhi Gupta, Ani Nenkova, and Dan Jurafsky. 2007\.  Measuring Importance and Query Relevance in Topic-focused Multi-document Summarization. In _Annual Meeting of the Association for Computational Linguistics_.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020.  Momentum Contrast for Unsupervised Visual Representation Learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_.
  * Hoffer and Ailon (2014) Elad Hoffer and Nir Ailon. 2014.  Deep Metric Learning Using Triplet Network. In _International Workshop on Similarity-Based Pattern Recognition_.
  * Hsu and Tan (2021) Chao-Chun Hsu and Chenhao Tan. 2021.  Decision-Focused Summarization.  _ArXiv_ abs/2109.06896 (2021).
  * Hu et al. (2022) Minda Hu, Muzhi Li, Yasheng Wang, and Irwin King. 2022\.  Momentum Contrastive Pre-training for Question Answering. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 4324–4330.  https://aclanthology.org/2022.emnlp-main.291
  * Ioffe and Szegedy (2015) Sergey Ioffe and Christian Szegedy. 2015.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In _International Conference on Machine Learning_.
  * Katragadda and Varma (2009) Rahul Katragadda and Vasudeva Varma. 2009.  Query-Focused Summaries or Query-Biased Summaries?. In _Annual Meeting of the Association for Computational Linguistics_.
  * Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020.  Supervised Contrastive Learning.  _ArXiv_ abs/2004.11362 (2020).
  * Laskar et al. (2022) Md Tahmid Rahman Laskar, Enamul Hoque, and Jimmy Xiangji Huang. 2022\.  Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization.  _Computational Linguistics_ 48, 2 (06 2022), 279–320.  https://doi.org/10.1162/coli_a_00434 arXiv:https://direct.mit.edu/coli/article-pdf/48/2/279/2029099/coli_a_00434.pdf
  * Laskar et al. (2020) Md Tahmid Rahman Laskar, Enamul Hoque, and Xiangji Huang. 2020\.  Query Focused Abstractive Summarization via Incorporating Query Relevance and Transfer Learning with Transformer Models. In _Canadian Conference on AI_.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 7871–7880.  https://doi.org/10.18653/v1/2020.acl-main.703
  * Lin and Hovy (2003) Chin-Yew Lin and Eduard H. Hovy. 2003.  Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In _North American Chapter of the Association for Computational Linguistics_.
  * Liu and Zhou (2021) Hongli Liu and Yanquan Zhou. 2021.  Bridge Architecture for Medical Question Answering with Negative Sample Training. In _2021 7th International Conference on Computer and Communications (ICCC)_. IEEE, 1623–1627.
  * Liu et al. (2023) Xingxian Liu, Bin Duan, Bo Xiao, and Yajing Xu. 2023\.  Query-Utterance Attention with Joint modeling for Query-Focused Meeting Summarization. In _Proc. ICASSP_.  arXiv:2303.04487
  * Metzler et al. (2007) Donald Metzler, Susan T. Dumais, and Christopher Meek. 2007\.  Similarity Measures for Short Segments of Text. In _European Conference on Information Retrieval_.
  * Metzler and Kanungo (2008) Donald Metzler and Tapas Kanungo. 2008.  Machine Learned Sentence Selection Strategies for Query-Biased Summarization.
  * Mihalcea and Tarau (2004) Rada Mihalcea and Paul Tarau. 2004.  TextRank: Bringing Order into Text. In _Conference on Empirical Methods in Natural Language Processing_.
  * Pagnoni et al. (2022) Artidoro Pagnoni, Alexander R. Fabbri, Wojciech Kryscinski, and Chien-Sheng Wu. 2022.  Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization.  _ArXiv_ abs/2212.10449 (2022).
  * Park and Ko (2022) Choongwon Park and Youngjoong Ko. 2022.  QSG Transformer: Transformer with Query-Attentive Semantic Graph for Query-Focused Summarization. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_ (Madrid, Spain) _(SIGIR ’22)_. Association for Computing Machinery, New York, NY, USA, 2589–2594.  https://doi.org/10.1145/3477495.3531901
  * Pasunuru et al. (2021) Ramakanth Pasunuru, Asli Celikyilmaz, Michel Galley, Chenyan Xiong, Yizhe Zhang, Mohit Bansal, and Jianfeng Gao. 2021.  Data Augmentation for Abstractive Query-Focused Multi-Document Summarization. In _AAAI 2021_.  https://www.microsoft.com/en-us/research/publication/data-augmentation-for-abstractive-query-focused-multi-document-summarization/
  * Song et al. (2022) Kaiqiang Song, Chen Li, Xiaoyang Wang, Dong Yu, and Fei Liu. 2022.  Towards Abstractive Grounded Summarization of Podcast Transcripts. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics_.
  * Sriram et al. (2010) Bharathi Sriram, David Fuhry, Engin Demir, Hakan Ferhatosmanoğlu, and Murat Demirbas. 2010\.  Short text classification in twitter to improve information filtering.  _Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval_ (2010).
  * Su et al. (2021) Dan Su, Tiezheng Yu, and Pascale Fung. 2021.  Improve Query Focused Abstractive Summarization by Incorporating Answer Relevance. In _Findings_.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014\.  Sequence to Sequence Learning with Neural Networks. In _NIPS_.
  * tau Yih and Meek (2007) Wen tau Yih and Christopher Meek. 2007.  Improving Similarity Measures for Short Segments of Text. In _AAAI Conference on Artificial Intelligence_.
  * Tombros and Sanderson (1998) Anastasios Tombros and Mark Sanderson. 1998.  Advantages of query biased summaries in information retrieval. In _Annual International ACM SIGIR Conference on Research and Development in Information Retrieval_.
  * van den Oord et al. (2018) Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018\.  Representation Learning with Contrastive Predictive Coding.  _ArXiv_ abs/1807.03748 (2018).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need. In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998–6008.  https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
  * Vig et al. (2022) Jesse Vig, Alexander Fabbri, Wojciech Kryscinski, Chien-Sheng Wu, and Wenhao Liu. 2022.  Exploring Neural Models for Query-Focused Summarization. In _Findings of the Association for Computational Linguistics: NAACL 2022_. Association for Computational Linguistics, Seattle, United States, 1455–1468.  https://doi.org/10.18653/v1/2022.findings-naacl.109
  * Wang et al. (2022) Alex Wang, Richard Yuanzhe Pang, Angelica Chen, Jason Phang, and Samuel R. Bowman. 2022.  SQuALITY: Building a Long-Document Summarization Dataset the Hard Way. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, 1139–1156.  https://aclanthology.org/2022.emnlp-main.75
  * Xiong et al. (2022) Wenhan Xiong, Anchit Gupta, Shubham Toshniwal, Yashar Mehdad, and Wen-tau Yih. 2022.  Adapting Pretrained Text-to-Text Models for Long Text Sequences.  _arXiv preprint arXiv:2209.10052_ (2022).
  * Xu and Lapata (2020a) Yumo Xu and Mirella Lapata. 2020a.  Coarse-to-Fine Query Focused Multi-Document Summarization. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Online, 3632–3645.  https://doi.org/10.18653/v1/2020.emnlp-main.296
  * Xu and Lapata (2020b) Yumo Xu and Mirella Lapata. 2020b.  Generating Query Focused Summaries from Query-Free Resources. In _Annual Meeting of the Association for Computational Linguistics_.
  * Xu and Lapata (2022) Yumo Xu and Mirella Lapata. 2022.  Document Summarization with Latent Queries.  _Transactions of the Association for Computational Linguistics_ 10 (2022), 623–638.
  * Yuan et al. (2022) Ruifeng Yuan, Zili Wang, Ziqiang Cao, and Wenjie Li. 2022\.  Few-shot Query-Focused Summarization with Prefix-Merging. In _Conference on Empirical Methods in Natural Language Processing_.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020\.  PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization. In _ICML_.
  * Zhong et al. (2021a) Ming Zhong, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021a.  DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization. In _AAAI Conference on Artificial Intelligence_.
  * Zhong et al. (2021b) Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, and Dragomir Radev. 2021b.  QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization. In _North American Association for Computational Linguistics (NAACL)_.


# Boundary Condition Induced Passive Chaotic Mixing in Straight Microchannels

Habilou Ouro-Koura  Department of Mechanical Engineering, Rensselaer
Polytechnic Institute, Troy, NY 12180 USA  University of Maryland Eastern
Shore, 1, Backbone Road, Princess Anne, MD 21853 USA  Ayobami Ogunmolasuyi
Department of Mechanical Engineering, Dartmouth College, Troy, NY 12180 USA
University of Maryland Eastern Shore, 1, Backbone Road, Princess Anne, MD
21853 USA  Othman Suleiman  Jaylah Easter  University of Maryland Eastern
Shore, 1, Backbone Road, Princess Anne, MD 21853 USA  Yasmin Roye  Department
of Biomedical Engineering, Pratt School of Engineering, Duke University,
Durham, NC 27708, USA  University of Maryland Eastern Shore, 1, Backbone Road,
Princess Anne, MD 21853 USA  Othman Suleiman  University of Maryland Eastern
Shore, 1, Backbone Road, Princess Anne, MD 21853 USA  Kausik S Das
<EMAIL_ADDRESS>University of Maryland Eastern Shore, 1, Backbone Road, Princess
Anne, MD 21853 USA

###### Abstract

Mixing in low Reynolds number flow is difficult because in this laminar regime
it occurs mostly via slow molecular diffusion. This letter reports a simple
way to significantly enhance low Reynolds number passive microfluidic flow
mixing in a straight microchannel by introducing asymmetric wetting boundary
conditions on the floor of the channel. We show experimentally and numerically
that by creating carefully chosen hydrophobic slip patterns on the floor of
the channels we can introduce stretching, folding and/or recirculation in the
flowing fluid volume, the essential elements to achieve mixing in absence of
turbulence. We also show that there are two distinctive pathways to produce
homogeneous mixing in microchannels induced by the inhomogeneity of the
boundary conditions. It can be achieved either by: 1) introducing stretching,
folding and twisting of fluid volumes, i.e., via a horse-shoe type
transformation map, or 2) by creating chaotic advection, through manipulation
of the hydrophobic boundary patterns on the floor of the channels. We have
also shown that by superposing stretching and folding with chaotic advection,
mixing can be optimized by significantly reducing mixing length, thereby
opening up new design opportunities for simple yet efficient passive
microfluidic reactors.

Microfluidics deals with control and manipulation of small volumes of fluids
through channels with characteristic length scales in the micron order an it
exercises precise dynamic control over the flow to study new phenomena
occurring in fluids. Microfluidics has the potential to solve some of the
grand engineering challenges such as engineering better medicines, providing
access to clean water, solving energy problems due to its ability to use very
small controlled volume of samples and reagents, high resolution separation
and detection with great sensitivity[1, 2, 3] etc. Low manufacturing cost,
short time scales for analysis along with high throughput designs using device
miniaturization such as lab-on-a-chip[4] or a reactor-on-a-chip[5] are other
advantages. The application of microfluidics is manyfold, ranging from
microelectronics, bioanalysis[6, 7], nanoparticle synthesis, organ in a chip,
drug development, testing and controlling multiphase flow[8, 9, 10],
optofluidics, acoustofluidic patterning[11], to detection of a single cell
[12, 13], single file diffusion[14] and even a single molecule[15].

It is clear that microfluidics offers solutions to a plethora of problems,
however, growth of microfluidic technlogies to the desired level has been
stalled by some bottlenecks. One major bottleneck in developing an efficient
microfluidic device is to mix fluids in a microchannel passively, where
miscible fluid components can be mixed homogeneously and spontaneously in low
Raynolds number regime. Most of the applications of microfluidic chips
involving chemical and biological components homogeneous mixing of reactants
is essential to achieve uniform solution environments. However, it is
difficult to mix component fluids in microchannels spontaneously, thanks to
the absence of turbulence in the viscosity dominated laminar flow regimes in
the small characteristic length scales. Microfluidic mixers are thus of
pivotal importance for efficient functioning of microfluidic devices for a
range of important applications.

In a typical microchannel cross-section length scale (l) is $\sim 100$ micron
or less. With small flow rates $U\sim 0.1$m/s, Reynolds numbers
($\textrm{Re}=Ul\rho/\mu$) are typically of the order of 10 or less, where
$\mu$ is the dynamic viscosity ($\sim 10^{-3}$ Pa.S) and $\rho$ is the density
($\sim 10^{3}$ Kg/m3 for water) of the fluid. This means that not only inertia
is negligible, but also molecular diffusion is the dominant mechanism of
mixing of the component fluids in this regime. However, typical diffusivity
($D$) of species varies from approximately 10-9 m2/s for small molecules
including ions, to 10-11 m2/s for large biomolecules. As a result, mixing time
scale ($l^{2}/D>$10 s) and Péclet number ($Pe=Ul/D$), the relative measure of
diffusive time scale over advective time scale, are large (104 $-$ 106). The
length required to mix via diffusion in a flow thus ($l_{mix}=lPe$) typically
ranges between tens of centimeters to even in meters!

One of the strategies that can be employed to enhance mixing in steady laminar
diffusion dominated flow regimes comes through a Smale horseshoe map and/or a
baker’s map [16]. By effectively decreasing the striation length, $l_{\rm
Str}\rightarrow 0$, defined as the length scale over which diffusion acts to
homogenize the concentration, these maps transform the space (the flow domain)
into itself through successive stretching and folding. The effective interface
area across which diffusion takes place thus increases exponentially leading
to a positive Lyapunov exponent. In other words this is a route to chaos or a
rapid growth in mixing, when the problem is translated into the language of
dynamical systems.

The other pathway to achieve mixing in low Reynolds number flow goes through
‘chaotic advection’, where the fluid still associates itself with stretching
and folding on top of switching flows instantaneously from one streamline to
another[17]. This mechanism of mixing is observed in blinking flows and can be
understood theoretically using a twist map[18]. Although both the
transformations can be analyzed rigorously using mathematical concepts of
ergodicity and dynamical systems, a practical demonstration of these idealized
mathematical models is rather difficult to achieve. Mixing via chaotic
advection was first demonstrated in the pioneering work on the Staggered
Herringbone Mixer (SHM) [19], where flow transverse to the primary flow
direction suitable for chaotic advection was generated by placing three
dimensional patterned ridges on the floor of the channels at oblique angles
with respect to the downstream direction of the flow. Mixing length varied
linearly with ln(Pe) confirming that the stretching and folding of the volumes
grow exponentially with respect to the effective mixing length. In this paper
we take a totally new approach to generate similar transverse flow by creating
anisotropic wetting conditions on the floor of the channels experimentally and
numerically. Moreover, we show that this type of mixing is not limited to just
Harringbone type patterns, rather a range of CFD simulations reveal the
effects of many different patterns on the flow behavior and their
corresponding mixing mechanisms in simple microfluidic channels.

It is well known that surfaces play a big role in micro and nanoscales because
the boundary layer essentially spans the whole cross sections of the channels.
Since the ratio of solid-liquid interface area ($\textrm{A}\sim l^{2}$) in a
channel and the volume of fluid ($\textrm{V}\sim l^{3}$) varies inversely
proportional to the characteristic length scale ($\sim 1/l$) of the system, in
small length scales such as micro or nano channels ($l\rightarrow 0$), the
effect of boundary conditions become increasingly dominant in this asymptotic
regime. In this work we exploit this fact and report that unlike [19], where
3D miropatterning involving rather complex fabrication process was needed to
achieve chaotic advection, here we have achieved mixing in a microchannel at
Re $\leq$10 by creating hyrdophobic patterns on the floor using water-
repellant ink dots on the channel floor. To be precise, we have used simple
hydrophobic permanent marker ink[20] that contains hydrophobic polymer resin
to create these hydrophobic patterns on one of the inner walls of the straight
channel. These hydrophobic dots create a discontinuous change in the wall
boundary conditions forcing the streamlines to readjust instantaneously while
crossing the boundary between no-slip to hydrophobic regions. Obviously there
is less resistance to flow over the water-repellant regions and the liquid
flows faster over them than the no-slip regions on the floor of the channels.
Moreover, the hydrophobic patterns constrain the fluid to avoid those regions,
in turn potentially forcing the fluid to generate a velocity component in the
transverse direction. We hypothesize that because of this anisotropic and
abrupt flow resistance change due to the difference in wetting boundary
conditions on the floor, the fluid generates an average transverse flow,
resulting in twisting and stretching of the fluid volume over the cross
section of the channel, or creating counter rotating recirculation zones
leading to chaotic advection.

Figure 1: Design of the straight microchannels with asymmetric boundary
condition. Dark spots are the hydrophobic regions on the floor of the
microchannels.

The whole fabrication process of microchannels with anisotropic boundary
conditions is quite simple and can be done using soft photolithography process
with polydimethylsiloxane (PDMS) elastomer. Designs of various microchannels
are shown in Fig.1. To create slip regions on the floor of the channels we
have used an unique technique. Recently, it has been shown that many
commercially available permanent marker inks, such as Sharpie® inks create
elastic thin polymer films upon drying on a substrate and exhibit water-
repellent hydrophobic behavior [20]. Taking advantage of this property and
easy availability of school-supply permanent markers, we have created slip
dots on the floor of the channels by simply placing the dots using an
ultrafine Sharpie® marker manually. After the hydrophobic dots are created on
a glass slide (Fig.1) PDMS open channels created by standard soft
photolithography process[21] are bonded with the slide using plasma boding
techniques[22, 23]. The hydrophobic spots created manually on the floor of the
channels are not fully symmetrical as expected, and can be seen in Fig.1 c.
Several microchannels are made with width and depth in the range of 100$\mu$m
to 300 $\mu$m, with the length of the channels 5-10 mm. In the experiments
Reynolds number is always kept at 10 by controlling the flow speed in the
syringe pumps.

Figure 2: Schematic of the microfluidic mixing experiment. One of the syringe
pumps pushes a colored fluid and the other brings plain water to the
microfluidic chip. Fluid is collected at the outlets after going through the
mixer and analyzed by a UV-VIS spectrometers.

Fig.2 shows the schematic of our experiments where two miscible fluids are
pumped into the microfluidic reactor through the inlets by two syringe pumps
and collected in the cuvettes placed at the outlet. The fluids collected at
the outlet are analyzed by UV-VIS absorption spectroscopy to determine the
mixing index and the overall effectiveness of the hydrophobic spots in the
mixing of component fluids. In our experiment we use deionized (DI) distilled
water (Barnstead NANOpureDiamond water purification system, specific
resistivity 18.2 M$\Omega$-cm) in one of the syringe pumps, and DI water mixed
with Allura Red dye (D=3.23$\times 10^{-10}$m2/s) in the other. In a simple
microchannels with usual no-slip inner walls the output confirms a fully
separated flow, i.e, one of the cuvettes collects red colored fluid with same
concentration of the input colored fluid, confirmed by the UV-VIS
spectroscopic analysis, whereas the other output cuvette collects colorless
water. This fully separated flow is characterized by the mixing index 0.5. On
the other hand as the hydrophobic marker dots are introduced, two input
liquids start to mix. A fully mixed homogeneous fluid’s concentration is
analyzed by the spectrophotometer and the mixing index is scaled as 0.

Figure 3: Experimental results on the effect of periodic hydrophobic regions
on mixing in a straight 7mm microchannel. Normalized mixing index 0.5
signifies separated flow and mixing index 0 signifies homogeneous mixing.

The effect of the slip dots on the degree of mixing at the outlet of the
channels, measured experimentally using a spectrophotometer is shown in Fig.3.
A transition from laminar separated flow to a homogeneous mixing of component
fluids is observed when the number of hydrophobic dots exceeds 3 in a 7mm
channel.

To understand the details of the mixing mechanism a numerical investigation is
done using the finite element simulation software package COMSOL. Navier-
Stokes and convection diffusion equations are solved with hydrophobic spots
considered as slip regions in the boundary conditions, while imposing no slip
conditions on the rest of the inner wall regions of the channel. Steady
incompressible flow with laminar outflow and zero outlet static pressure is
employed. In the code Reynolds number is kept at 10 to begin with, to match it
with our experimental conditions.

To further benchmark the code we first created herringbone type 2D hydrophobic
patters on the floor of the channels to mimic the 3D grooved channels in SHM
previously reported[19]. It shows similar mixing profiles (Fig. 3 of [19]) in
comparison to our simulation results (Fig.6 of this paper). From the numerical
simulation we have identified two different pathways to achieve homogeneous
mixing of component fluids in the low Reynolds number laminar flow regime.
Preferential bending of the flow can be achieved by introducing periodic
asymmetries in the boundary conditions, created by asymmetric hydrophobic
patterns about the centerline of the channel shown in Fig.4a. Analogous to
Magnus effect[24], where difference between the speeds of a fluid flowing over
two opposite sides of an object gives rise to a pressure difference in the
transverse direction results in bending, here our ‘duck’ shape patterns, for
example, create a transverse pressure gradient generated through the
difference of uneven flow resistances about the centerline.

Figure 4: Effect of periodic hydrophobic regions on mixing in a straight
microchannel. a) The asymmetric duck-shape hydrophobic regions introduce
difference of resistance to flow about the centerline of the channel. b) A
zoomed version of one of the hydrophobic regions shows the possible path of
the fluid that flows over the the shapes. It is clear that this pattern forces
the fluids to bends the fluid and one component of the fluid intrudes the
other thereby introducing effective transverse rotation and stretching and
folding of the fluid volume as it flows downstream.

We argue that the floor of the channels is no-slip everywhere except the
locations where there is a hydrophobic film. So, the hydrophobic regions on
the floor applies less resistance to the fluid flow in comparison to the
corresponding no slip region on the other side of the centerline. Each of the
patterns thus may direct one component fluid (in our case could be blue) to
intrude into the other (red), eventually creating a twisting, stretching and
folding of both the components as the fluids flow downstream and leading to a
Smale horshoe type mixing pathway as seen in Fig.4.

Figure 5: Effect of periodic hydrophobic regions on mixing in a straight
microchannel through symmetric V shapes hydrophobic patches. a) The symmetric
V shaped regions introduce difference of resistance to flow along the arms. b)
A zoomed version of one of the hydrophobic regions shows the path of the fluid
that flows along the arms. An important observation is that the vertex of the
V shape divides the flow and channel them obliquely in two opposite
directions, thus creating counter rotating recirculation zones.

Another pathway of mixing is found by creating counter rotating recirculation
zones with the help of ‘V’ shaped hydrophobic patterns (Fig. 5 ). It is clear
that the stagnation point between the recirculating fluids align with the
position of the apex point of the ‘V’ shape. This feature is evident in the
herringbone pattern (Fig. 6) also, where the stagnation points can be shifted
periodically in the transverse direction by moving the apex points of the
asymmetric ‘V’ shapes. This creates different recirculation patterns as the
fluids flow downstream leading towards effective mixing through chaotic
advection. To check if this enhanced mixing is an artefact of chaos we
investigated the power-law dependence of the mixing length on the Péclet
number $Pe$. When mixing occurs only via diffusion, the channel length
required for mixing is given by $l_{mix}\sim Ut_{diff}\sim Ul^{2}/D\sim lPe$,
whereas if there is indeed chaotic root to mixing via stretching and folding
of liquid volumes over the cross section of the channel then the mixing length
would vary as $l_{mix}\sim l\textrm{ ln}(Pe)$ [25, 19, 26]. A linear profile
of $l_{mix}$ with respect to $\textrm{ln}(Pe)$ can thus be considered as a
signature of chaotic advection in the system. In Fig.7 we have indeed seen
this behavior in our irregular V shaped passive microfluidic mixers.

Figure 6: Effect of periodic hydrophobic regions on mixing in a straight
microchannel. a) The asymmetric herringbone shaped hydrophobic regions
introduce difference of resistance to flow along the arms of the asymmetric V
shapes. b) A zoomed version of one of these regions shows the path of the
fluid that flows over the the patches. One can think of the Harrinbone
structures as irregular V shapes. Here also the vertices introduce
recirculation. However, as the vertices change location depending on the
wavelength of the pattern, stagnation points of the recirculations also move
back and forth, in turn introducing chaotic advection in the flow. Figure 7:
Effect of Peclet number on mixing in a Herringbone mixer. A linear
relationship between ln(Pe) and the mixing length, where 90% of mixing occurs
indicates route to chaos[19] in a low Re environment.

We were also curious to see if we can create a superposition of the Magnus
type bending with the recirculating cells using both the pathways mentioned
above and the results are shown in Fig. 8. It shows that alternating twisting
and recirculation gives rise to excellent mixing in low Reynolds number
laminar flow in a straight microchannel.

Figure 8: Effect of periodic hydrophobic regions on mixing in a straight
microchannel through hybrid patterns. We have observed a superposition of
effects of stretching through rotation (by duck shapes) and recirculation (by
V shapes) or vice versa. a) Here the duck shape patterns were created first
close to the inlet followed by V shape patterns. It is clear that one
component of the fluid intrudes into the other and introduces stretching and
folding by the duck shape as discussed earlier. On top of that the symmetric V
shapes create recirculating flow patterns leading towards complex mixing
effect. b) An opposite effect is observed when the pattern is switched, i.e.,
V shapes followed by the duck shapes. Recirculation followed by stretching and
folding give rise to a different mixed state.

In conclusion we claim that rapid mixing in low Reynolds number flow even in a
straight microchannel can be achieved by manipulating twisting, stretching and
recirculation in the flow created by passive interventions through carefully
chosen hydrophobic patterns on the walls/floor of the microchannels. In this
case we won’t even need complex three dimensional micro/nano patterning on the
floor of the channels[19] to introduce chaos in this regime. This letter may
lead the microfluidic industry to an easy and inexpensive fabrication process
to achieve efficient homogeneous mixing with a small mixing length scale.

## References

  * Whitesides [2006] G. M. Whitesides, The origins and the future of microfluidics, Nature 442, 368 (2006).
  * Fraikin  _et al._ [2011] J.-L. Fraikin, T. Teesalu, C. M. McKenney, E. Ruoslahti, and A. N. Cleland, A high-throughput label-free nanoparticle analyser, Nature nanotechnology 6, 308 (2011).
  * Sinton [2014] D. Sinton, Energy: the microfluidic frontier, Lab on a Chip 14, 3127 (2014).
  * Mark  _et al._ [2010] D. Mark, S. Haeberle, G. Roth, F. Von Stetten, and R. Zengerle, Microfluidic lab-on-a-chip platforms: requirements, characteristics and applications, in  _Microfluidics Based Microsystems_ (Springer, 2010) pp. 305–376.
  * Srisa-Art  _et al._ [2008] M. Srisa-Art, A. J. Demello, and J. B. Edel, Fluorescence lifetime imaging of mixing dynamics in continuous-flow microdroplet reactors, Physical Review Letters 101, 014502 (2008).
  * Sackmann  _et al._ [2014] E. K. Sackmann, A. L. Fulton, and D. J. Beebe, The present and future role of microfluidics in biomedical research, Nature 507, 181 (2014).
  * Martinez  _et al._ [2010] A. W. Martinez, S. T. Phillips, G. M. Whitesides, and E. Carrilho, Diagnostics for the developing world: Microfluidic paper-based analytical devices, Analytical Chemistry 82, 3 (2010).
  * Teh  _et al._ [2008] S.-Y. Teh, R. Lin, L.-H. Hung, and A. P. Lee, Droplet microfluidics, Lab on a Chip 8, 198 (2008).
  * Ando  _et al._ [2012] K. Ando, A.-Q. Liu, and C.-D. Ohl, Homogeneous nucleation in water in microfluidic channels, Physical Review Letters 109, 044501 (2012).
  * Nordstrom  _et al._ [2010] K. N. Nordstrom, E. Verneuil, P. Arratia, A. Basu, Z. Zhang, A. G. Yodh, J. P. Gollub, and D. J. Durian, Microfluidic rheology of soft colloids above and below jamming, Physical Review Letters 105, 175701 (2010).
  * Collins  _et al._ [2018] D. J. Collins, R. O’Rorke, C. Devendran, Z. Ma, J. Han, A. Neild, and Y. Ai, Self-aligned acoustofluidic particle focusing and patterning in microfluidic channels from channel-based acoustic waveguides, Physical Review Letters 120, 074502 (2018).
  * Nagrath  _et al._ [2007] S. Nagrath, L. V. Sequist, S. Maheswaran, D. W. Bell, D. Irimia, L. Ulkus, M. R. Smith, E. L. Kwak, S. Digumarthy, A. Muzikansky, _et al._ , Isolation of rare circulating tumour cells in cancer patients by microchip technology, Nature 450, 1235 (2007).
  * Burg  _et al._ [2007] T. P. Burg, M. Godin, S. M. Knudsen, W. Shen, G. Carlson, J. S. Foster, K. Babcock, and S. R. Manalis, Weighing of biomolecules, single cells and single nanoparticles in fluid, Nature 446, 1066 (2007).
  * Locatelli  _et al._ [2016] E. Locatelli, M. Pierno, F. Baldovin, E. Orlandini, Y. Tan, and S. Pagliara, Single-file escape of colloidal particles from microfluidic channels, Physical Review Letters 117, 038001 (2016).
  * Dittrich and Manz [2005] P. S. Dittrich and A. Manz, Single-molecule fluorescence detection in microfluidic channels—the holy grail in $\mu$TAS?, Analytical and bioanalytical chemistry 382, 1771 (2005).
  * Christov  _et al._ [2011] I. C. Christov, R. M. Lueptow, and J. M. Ottino, Stretching and folding versus cutting and shuffling: An illustrated perspective on mixing and deformations of continua, American Journal of Physics 79, 359 (2011).
  * Wiggins and Ottino [2004] S. Wiggins and J. M. Ottino, Foundations of chaotic mixing, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences 362, 937 (2004).
  * Aref [1984] H. Aref, Stirring by chaotic advection, Journal of Fluid Mechanics 143, 1 (1984).
  * Stroock  _et al._ [2002] A. D. Stroock, S. K. Dertinger, A. Ajdari, I. Mezić, H. A. Stone, and G. M. Whitesides, Chaotic mixer for microchannels, Science 295, 647 (2002).
  * Khodaparast  _et al._ [2017] S. Khodaparast, F. Boulogne, C. Poulard, and H. A. Stone, Water-based peeling of thin hydrophobic films, Physical Review Letters 119, 154502 (2017).
  * Xia and Whitesides [1998] Y. Xia and G. M. Whitesides, Soft lithography, Annual review of materials science 28, 153 (1998).
  * Barnes  _et al._ [2021] B. K. Barnes, H. Ouro-Koura, J. Derickson, S. Lebarty, J. Omidokun, N. Bane, O. Suleiman, E. Omagamre, M. J. Fotouhi, A. Ogunmolasuyi, _et al._ , Plasma generation by household microwave oven for surface modification and other emerging applications, American Journal of Physics 89, 372 (2021).
  * Bhattacharya  _et al._ [2005] S. Bhattacharya, A. Datta, J. M. Berg, and S. Gangopadhyay, Studies on surface wettability of poly (dimethyl) siloxane (pdms) and glass under oxygen-plasma treatment and correlation with bond strength, Journal of Microelectromechanical Systems 14, 590 (2005).
  * Barkla and Auchterlonie [1971] H. Barkla and L. Auchterlonie, The magnus or robins effect on rotating spheres, Journal of Fluid Mechanics 47, 437 (1971).
  * Ottino [1989] J. M. Ottino,  _The kinematics of mixing: stretching, chaos, and transport_ , Vol. 3 (Cambridge university press, 1989).
  * Jones  _et al._ [1989] S. W. Jones, O. M. Thomas, and H. Aref, Chaotic advection by laminar flow in a twisted pipe, Journal of Fluid Mechanics 209, 335 (1989).