# Dynamic Q&A of Clinical Documents with Large Language Models

Ran Elgedawy
<EMAIL_ADDRESS>
University of Tennessee, Knoxville  Sudarshan Srinivasan
<EMAIL_ADDRESS>
Oak Ridge National Laboratory  Ioana Danciu
<EMAIL_ADDRESS>
Oak Ridge National Laboratory

###### Abstract

Electronic health records (EHRs) contain a wealth of valuable patient
information embedded within clinical notes. However, the ever-increasing
volume and complexity of these unstructured clinical notes makes it extremely
challenging for researchers and clinicians to manually locate and extract
relevant information in a timely manner. Intelligent systems that can
understand clinical documents and enable users to efficiently query and
retrieve key details on demand are urgently needed. In this work our aim is to
develop a natural language conversational interface powered by state-of-the-
art open source large language models (LLMs) that allows users to explore
clinical notes through dynamic question-answering. A chatbot interface was
built using Langchain, a framework for creating and managing applications
powered by large language models, paired with powerful transformer-based
language models. The system allows users to simply ask questions in natural
language and receive answers extracted from relevant parts of the clinical
notes. Extensive experiments were performed to evaluate combinations of
semantic embedding models such as the state of the art SentenceTransformers
framework, based on the Sentence-BERT embedding models [1], and advanced large
language models on their ability to encode both queries and documents for
optimal information retrieval. Wizard Vicuna demonstrated the highest accuracy
due to its 13B parameters but required heavy compute. To improve inference
latency and deployability, model optimization techniques like weight
quantization was utilized to reduce latency by approximately _48_ times. While
results are promising, several limitations remain including model
hallucinations, and lack of robust evaluation across diverse medical cases.
Addressing these gaps represents important future work to unlock the value in
clinical notes and advance clinical decision making through AI.

##  1 Introduction

Electronic health record (EHR) systems have become ubiquitous in modern
healthcare environments [2, 3, 4]. They offer an integrated digital repository
of patient health information collected from various clinical encounters
across time. EHRs may contain a range of data including demographics, medical
and family histories, laboratory test results, radiology images, computed
biosignals and crucially, clinical notes generated by physicians, nurses and
other providers. These clinical notes, documenting physician-patient
interactions, encapsulate a wealth of vital information about the patient’s
conditions, treatments, care plans and outcomes in rich natural language.
However, a significant limitation of EHRs is that critical patient information
is buried within large volumes of unstructured clinical notes that are
challenging for providers to efficiently search through and extract relevant
details on demand. For instance, a newly assigned physician may need to
rapidly comprehend a patient’s care history across multiple specialists to
make appropriate decisions. Or a researcher may want to analyze treatment
response patterns across populations. These use cases require intelligently
sifting through troves of notes to pinpoint key facts, relationships and
temporal trends - a formidable manual task. Hence, there is a profound need
for intelligent systems that can automatically understand clinical notes
written in natural language and enable users to easily query and retrieve
targeted information on the fly through conversational interactions.

Traditionally, information retrieval systems have relied on methods like
Boolean models, probabilistic models, and term frequency-inverse document
frequency (TF-IDF) for surface-level matching of keywords between queries and
documents. While these techniques have been useful for basic keyword-based
retrieval, they have their limitations. Boolean models utilize operators like
AND, OR, and NOT to retrieve documents containing specific keyword
combinations, but they lack the ability to understand the nuances and context
of natural language queries. Probabilistic models rank results based on
keyword occurrences, which can be effective for certain tasks, but they
struggle to grasp the underlying meaning of the text. TF-IDF is another widely
used method that assigns weights to keywords in queries and documents based on
their rarity. This approach helps identify important terms, but it still falls
short in capturing the full semantic context and relationships between words.
However the primary constraint of these methods is that they lack semantic
understanding and the ability to interpret context.

As a result, these traditional techniques often miss relevant information when
applied to complex natural language text. They fail to recognize synonyms,
antonyms, word variations, and other language intricacies that are essential
for accurate and comprehensive information retrieval.

To overcome these limitations and unlock deeper meaning, more advanced
techniques such as natural language methods based on large language models
(LLMs) [5] can be utilized. The rapidly advancing LLMs, like GPT-3 and its
successors, have revolutionized the field of natural language processing, with
their contextual understanding, generative capabilities, and fine-tuning
adaptability. Integrating LLMs into information retrieval systems holds the
potential to deliver more accurate, insightful, and context-aware results,
transforming the way users access information in a more personalized and
efficient manner.

In this project we explore the utility of modern natural language processing
(NLP) methods based on large language models (LLMs) to develop a
conversational agent or chatbot that allows users to explore clinical notes
through dynamic question answering. By interfacing with the chatbot in natural
language as one would with a human assistant, users can extract relevant
details buried within notes, facilitating improved clinical care and research.

Contributions: To this end, our key contributions are as follows:

  * •

We developed a conversational question answering system based large language
models that allows natural language interaction for exploring clinical notes

  * •

We discussed detailed step by step pipeline of the question answering system
based on a medical dataset

  * •

We conducted extensive experiments evaluating combinations of different
embedding models and LLMs to determine optimal configurations for accuracy and
speed

  * •

We showed how the latency and deployability of these models can be optimized
by using weights quantization

  * •

We highlighted the superiority of retrieval augmented generation over domain-
specific fine-tuning, demonstrating its advantages in our study

  * •

We highlighted key limitations around model performance, evaluation rigor, and
real-world deployment that open the doors for future research in that domain

Organization: The paper is organized as follows: We discuss related works in
section 2. Section 3 discusses the approach. Details of the dataset are
discussed in section 4. Evaluation and results are discussed in section 5. We
discuss the limitations and future work in section 6 and section 7 concludes
the paper.

##  2 Related Work

###  2.1 LLMs in information retrieval

The intersection of Large Language Models (LLMs) and Information Retrieval
(IR) has emerged as a pivotal area of research, transforming traditional
approaches and reshaping the landscape of information access.

The work by Zhu et al. [6] provides a strong foundation by offering a
comprehensive survey of the diverse applications of LLMs in different facets
of IR, including query rewriting, retrieval, re-ranking, and reading. While
their survey spans a wide array of studies, our research takes a specific
angle, honing in on the integration of LLMs in the information retrieval
process. Expanding upon this foundation, we delve into the potential of
enhancing traditional retrieval methods by incorporating text generation
capabilities. Our aim is to provide a more natural and conversational
interaction between users and the system, ensuring a seamless and user-
friendly experience.

Another influential contribution in the realm of information retrieval is the
tutorial presented by Begoli et al. [7]. This tutorial focuses on advancing
both the theoretical and practical aspects of transformer-based approaches,
particularly with an emphasis on efficiency and robustness in large-scale
information retrieval scenarios. The tutorial aims to equip the audience with
insights gained from developing and fine-tuning transformer/BERT-based models
for IR tasks. In contrast, our approach focuses on the utilization of off-the-
shelf models, enabling the direct retrieval of information from provided
resources without the need for intricate fine-tuning procedures. This
streamlined method significantly boosts usability and efficiency,
circumventing the time and computational expenses associated with fine-tuning.
Furthermore, our research demonstrates that the accuracy achieved through our
direct utilization method surpasses that attained through domain-specific
fine-tuning processes.

###  2.2 Medical question answering using LLMs

In recent years, the field of medical question answering has witnessed
remarkable advancements, driven by the integration of Large Language Models
(LLMs) into the domain. The quest for expert-level performance in this grand
challenge has seen notable contributions from various research works.

Lee et al. [8] address the unique challenges posed by biomedical text mining.
Recognizing the limitations of applying general NLP advancements to biomedical
corpora, they introduce BioBERT, a domain-specific language representation
model pre-trained on large-scale biomedical corpora. BioBERT outperforms
generic models like BERT and previous state-of-the-art models in various
biomedical text mining tasks, including biomedical named entity recognition,
biomedical relation extraction, and biomedical question answering.

Another work by Chakraborty et al. [9]emphasizes the urgency of accelerating
biomedical research in light of the SARS-CoV-2 pandemic. Introducing
BioMedBERT, a pre-trained model on the extensive BREATHE v1.0 dataset, the
authors showcase its efficacy in Question-Answering (QA) and Information
Retrieval (IR) tasks. The model achieves state-of-the-art results on BioASQ
datasets and exhibits superior performance when coupled with Elasticsearch for
IR tasks on the BioASQ dataset.

Singhal et al. [10] present Med-PaLM 2, a groundbreaking development in
medical question answering. Addressing the inherent challenges in prior
models, Med-PaLM 2 leverages a combination of improvements in base LLMs
(specifically PaLM 2), medical domain fine-tuning, and innovative prompting
strategies, including a novel ensemble refinement approach. The model achieved
an impressive score of up to 86.5% on the MedQA dataset, surpassing its
predecessor by over 19% and establishing a new state-of-the-art.

In contrast to the aforementioned works, our work diverges by placing a
specific emphasis on the integration of retrieval augmented generation for
medical question answering. While the discussed models, such as Med-PaLM 2,
BioBERT, and BioMedBERT, have made significant strides in achieving expert-
level performance through advancements in base LLMs, domain fine-tuning, and
innovative prompting strategies, my approach focuses on enhancing
deployability and mitigating computational and time constraints. The
integration of retrieval augmented generation aims to strike a balance between
performance and resource efficiency, making the model more accessible for
widespread adoption in medical settings.

###  2.3 Impact of embedding models on output of large language models

The exploration of word embeddings and their influence on the output of large
language models (LLMs) is an under-explored but crucial aspect in natural
language processing (NLP). While numerous studies focus on the applications
and performance of LLMs, the specific impact of embedding models on their
output remains relatively uncharted territory.

The only relevant work in this area is the work by Asudani et al. [11]. In
this work the authors focuses on the application of word embeddings and deep
learning models in the context of text analytics tasks. Through a systematic
literature review, the authors classify and analyze relevant articles,
highlighting the increasing interest in employing these techniques for
analysis and prediction. The study categorizes literature based on text
analytics applications, encompassing areas like text classification, sentiment
analysis, named entity recognition (NER), recommendation systems, biomedical
text mining, and topic modeling. The paper concludes that utilizing domain-
specific word embeddings and the LSTM model can significantly enhance overall
text analytics task performance. In our investigation, we direct our attention
to a distinct domain that remains unexplored in the aforementioned study —
medical question answering. Furthermore, we delve into a different suite of
embedding models and language models, areas that Asudani et al. did not
specifically address in their research.

##  3 Approach

Figure 1:  Information retrieval pipeline

Our approach primarily utilizes Retrieval-Augmented Generation (RAG) [12], a
technique that merges the power of pre-trained language models with the
benefits of external knowledge sources. RAG employs a two-step process: a
retrieval step that fetches relevant documents from a vast corpus of text, and
a generation step that crafts responses based on these retrieved documents and
the given input query.

This technique is particularly advantageous when the required response
includes information that the language model has not encountered during its
training. By combining the ability to generalize from language model
pretraining with the capacity to retrieve specific details from external
sources, RAG can generate highly detailed and accurate responses while
avoiding the overhead of fine tuning the language model on specific datasets.

As illustrated in Figure 1 our approach consists of two main phases: a)
Ingestion b) Generation.

  1. 1.

Ingestion

     * •

Content Extraction: We begin by extracting the content of the clinical
documents from the objects that encapsulate both the content and metadata (i.e
source path, file size,…etc).

     * •

Content Splitting: The loaded content is then split into chucks. This step is
crucial as it breaks down the text into smaller units, to comply with the
embedding models context size limitations. Throughout this step, we
intentionally allow for some overlap between the different chunks. This
strategy helps to maintain the contextual integrity of the data.

     * •

Text Embedding: Each chunk of text is then transformed into an embedding.
These embeddings are numerical representations that encapsulate the semantic
meaning of the text. The transformation process is carried out by a distinct
embedding model, the details of which will be discussed later.

     * •

Vector Store: The embeddings, along with some reference to the original
content the embedding was created from, are subsequently stored in a vector
database or vector store, which forms the backbone of our retrieval system.

  2. 2.

Generation

     * •

Document Retrieval: The user input query is processed using the same process
described above 1. After that, an index – a data structure designed for quick
retrieval – is used by a retriever component to scan the vector store. Using a
cosine similarity, the retriever identifies and returns the most relevant
documents to the user’s input query.

     * •

Output Generation: The re-ranked vectors, which represent the selected
documents, along with the input prompt, are passed to a language model. This
model is responsible for text summarization and final output generation, and
it is distinct from the earlier mentioned embedding model.

The embedding model and the language model are two separate but integral
components of our system. The embedding model is used to transform raw text
into meaningful numerical representations, while the language model generates
coherent and contextually appropriate text. A summary of the distinguish
between embedding and language models is shown in table 1

Model type | Description | Applications
---|---|---
Language Models (LMs) |  | Understand and generate human-like text,
---
learning patterns from large data
| •Text completion
---
•Language generation
Embedding Models |  | Encode text into vector representations called embeddings,
---
capturing semantic meaning.
•Comparison and similarity calculations

Table 1: Embedding vs Language models

To achieve the previously described pipeline we utilized a tool called
Langchain. LangChain is a powerful and versatile Python framework designed to
facilitate the development of state-of-the-art language models and their
applications. It offers a wide range of pre-configured models and utilities
that allow researchers and developers to focus on the creative aspects of
their projects, rather than spending time on repetitive and time-consuming
tasks such as model configuration, training, and deployment. With its modular
and flexible design, LangChain allows users to easily integrate various
components and adapt them to their specific needs.

In our approach, we leveraged multiple components of the LangChain framework.
First, we used the Document Store component for efficiently storing and
retrieving documents. This component allowed us to create a robust retrieval
system, which is a crucial aspect of our RAG model. The Document Store uses a
vector database to store embeddings and references to the original content,
providing a fast and scalable solution for storing and retrieving large
volumes of data.

Second, we employed LangChain’s Retriever component for document retrieval.
The retriever uses an index, a data structure designed for quick retrieval, to
scan the vector store. Using cosine similarity, it identifies and returns the
most relevant documents to the user’s input query. This component’s ability to
quickly and accurately retrieve relevant documents is essential for the
successful application of our RAG model.

We also integrated LangChain’s Memory component into our system. In the
context of language model applications, especially those involving
conversational interfaces, the ability to refer to information from earlier
interactions is crucial. This is where the Memory component comes into play.
The Memory component in LangChain offers utilities for storing information
about past interactions, acting like a conversation history that can be read
from and written to. By integrating this component into our system, we have
not only improved the quality of the responses generated but also ensured a
more natural and human-like conversational experience.

Lastly, we implemented the Chain component provided by LangChain, which was
instrumental in developing our more complex system requirements. Chains
allowed us to create coherent workflows, linking multiple components together.
For instance, we created a chain that takes user input, formats it with a
specific template, and then passes the formatted response to the language
model for generation.

##  4 Dataset

The Medical Information Mart for Intensive Care (MIMIC-IV) [13] , is a single-
center dataset encompassing information about over 40k patients admitted to
intensive care units (ICUs) at at the Beth Israel Deaconess Medical Center
(BIDMC). This dataset includes a comprehensive range of data such as
medications, laboratory measurements, observations, vital signs, care provider
notes, procedure codes, fluid balance, diagnostic codes, hospital length of
stay, imaging reports, survival data, and more. MIMIC-IV serves a vital role
in various applications, supporting industrial research, academic endeavors,
and initiatives for quality improvement.

In addition to the MIMIC-IV dataset,we also utilized the MIMIC-IV-Note [14],
which offers deidentified free-text clinical notes for patients included in
the MIMIC-IV clinical database. MIMIC-IV-Note contains over 331k discharge
summaries. It also includes over 2 million radiology reports. These notes have
undergone deidentification, ensuring the removal of protected health
information (PHI) in accordance with the Health Insurance Portability and
Accountability Act (HIPAA) Safe Harbor provision. MIMIC-IV-Note allows
researchers to study clinical natural language processing and related areas by
providing context to the clinical data within MIMIC-IV.

Just like their predecessors, MIMIC-IV and MIMIC-IV-Note are freely available,
but accessing them requires undergoing CITI training. This training provides
materials on maintaining patient privacy, handling medical records
responsibly, and importantly, it includes the necessary Health Insurance
Portability and Accountability Act (HIPAA) training, as the data originates
from the United States.

The dataset itself is substantial and comes in CSV files, along with SQL
scripts that facilitate loading the data into the RDS (Relational Database
Service). This allows for easy exploration, querying, and filtering of the
data to prepare it for the large language models. High-level statistics on the
dataset are available in Table 2.

Figure 2 depicts the number of notes taken per year, with the years projected
into the future as part of the deidentification process. Additionally, Figure
3 displays statistics of the note lengths, ranging from a minimum of 3
characters to a maximum of 60,381 characters.

Description | Count
---|---
Total Number of Patients | 46,520
Total Number of Notes | 2,653,149
Average Notes per Patient | 10.78
Number of Note types | 3
Average Number of Notes per Type | 884383
Table 2: Dataset Statistics Figure 2:  Notes taken per year Figure 3:  Notes
length

###  4.1 Data Preprocessing

The preprocessing steps undertaken for this study are outlined below:

  1. 1.

Extraction of Relevant Fields: To streamline the dataset and focus on the most
pertinent information, only the fields that are directly relevant to our
analysis were extracted. This not only helps in reducing the data’s
dimensionality but also ensures that only the most meaningful attributes are
considered for subsequent stages. Relevant fields included: Subject (patient)
ID, Gender, Race, Time when note was taken, Age when note was taken, Admission
type and note text.

  2. 2.

Handling Missing Values: For the integrity of our analyses, records containing
empty or null values were dropped. This decision was made to prevent potential
biases or inaccuracies in the analysis due to incomplete data.

##  5 Evaluation

###  5.1 Model Pairing Accuracy

In our initial evaluation phase, we aimed to assess the impact of various
combinations of embedding and language models on the precision of the results.
From the Hugging Face’s Open LLM leaderboard [15, 16], we selected a diverse
set of 10 language models and paired them with 6 distinct embedding models to
comprehensively gauge their performance. Table 3 provides an overview of the
various models employed. It’s important to note that the selection of these
models was influenced by our hardware constraints. Additionally, models with
fewer than 3B parameters exhibited reduced accuracy, indicating saturation.

Table 3: Language models overview Model name | Base model | # Parameters | Reference
---|---|---|---
Vicuna | LLaMA | 13B | [17]
Wizard Vicuna | LLaMA | 13B | [18]
RedPajama-Chat | RedPajama-Base | 7B | [19]
Aplaca v1 | LLaMA | 7B | [20]
Alpaca v2 | LLaMA | 13B | [20]
Med Alpaca | LLaMA | 7B | [21]
GPT 4 x Alpaca | LLaMA | 13B | [22]
FastChat - T5 | Flan-t5-xl | 3B | [23]
Flan T5 xl | Google T5 | 3B | [24]
LexPodLM | LLaMA | 13B | [25]

####  5.1.1 Experimental Setup

For our preliminary set of evaluations, we crafted five question-answer pairs
based on five different notes from the MIMIC dataset that we curated earlier
to serve as our evaluation metric. Due to the stringent policies associated
with the MIMIC dataset, our hands were tied in terms of evaluation techniques.
As a result, commercially acclaimed state-of-the-art models, such as OpenAI’s
GPT-4, were off the table. This limitation necessitated manual evaluation of
the outputs produced by the selected embedding and language model pairs,
ensuring adherence to dataset guidelines while striving for accuracy in our
analysis.

####  5.1.2 Results

Figure 4 illustrates the accuracy outcomes for various pairings of embedding
and language models. The figure reveals that the ”Wizard Vicuna” model, a 13
Billion-parameter derivative of the Meta LLaMA model further refined using
conversations from the ’ShareGPT’ website [26], achieved the top accuracy
(80%) when paired with the ”Sentence Transformers: Language-agnostic sentence
embedding” model.

Figure 4:  Accuracy of Embedding vs Language models

###  5.2 Single Document Evaluations

For both single and multi-document evaluations 5.3, we employed synthetic data
that closely mirrors the structure of the MIMIC dataset. This approach was
taken to enable the utilization of OpenAI’s GPT-4 in deriving the ground truth
answers. The synthetic nature of the data ensured that we comply with the
terms of use for the MIMIC dataset, while maintaining a structure similar to
the original dataset for relevance and applicability. Table 4 compares the
outputs of two sample questions asked to both our model and the OpenAI GPT-4
models. We also report the inference time of our model.

###  5.3 Multi Documents Evaluations

Next, we employed multiple documents in parallel to assess our approach in a
real-world context. Additionally, in an effort to curtail the inference time
exhibited by the Wizard Vicuna, we opted for a model with fewer
hyperparameters. Consequently, we selected the 3B Google Flan xl model for
this purpose. The inference time as well as the output of using both models
are reported in table 5.The table illustrates that Wizard Vicuna boasts an
accuracy rate of 100%, outperforming the Flan T5 model’s 60% accuracy.
However, this comes at a cost: Wizard Vicuna’s inference time is over 24 times
longer than that of Flan T5.

Table 4: Single document evaluations Question |  ChatGPT (GPT 4) |  Wizard Vicuna (13B) | Time (s)
---|---|---|---
How old was the patient when the note was taken? |  The patient was 88 years old when the note was taken. |  The patient was 88 years old when the note was taken. | 20.4
What is the history of the patient’s illness? |  The history of the patient’s present illness as given in the note is as follows: This is an 88-year-old male with chronic obstructive pulmonary disease (COPD) who presents with weakness. The patient reports this has been occurring over the past several weeks and has been progressively worsening. The patient denies any other associated symptoms. |  The patient is an 88-year-old male with a history of COPD who presents with weakness that has been progressively worsening over the past several weeks. | 33.6
Question |  Flan T5 |  Time |  Wizard Vicuna |  Time
---|---|---|---|---
Can you summarize the findings for the patient with ID: 15022933 ? |  No |  1.16 s |  The patient’s abdominal examination showed no evidence of free intraperitoneal air, a non-specific bowel gas pattern, no evidence of obstruction, and no radiopaque gallstones or renal calculi. The osseous structures of the abdomen were intact. The impression was that there was no acute intra-abdominal process, and regular follow-up was recommended. |  12 mins 20s
What is the past medical history for patient with ID: 14169918? |  Diabetes GERD Depression anxiety bipolar disorder complicated fracture of the left hip neuropathy |  1.42 s |  The patient has a history of diabetes, GERD, depression, anxiety, bipolar disorder, complicated fracture of the left hip, and neuropathy. |  1min 37s
Did the patient with ID 16509550 have any known allergies to drugs? |  Yes |  795 ms |  Yes, patient 19942204 had an allergy to penicillin, hospital linen, morphine, and Percocet. |  1 min 36s
What is the marital status of patient 18954748? |  Single |  922 ms |  The patient is widowed. |  1min 3s
What is the discharge condition for patient 10986730? |  Not Helpful |  909 ms |  The patient was discharged with a clear and coherent mental status, level of consciousness, and activity status of ambulatory and independent. |  11min 29s
Table 5: Results from the multi-document evaluation comparing Flan T5 and
Wizard Vicuna. For each model, the immediate right column indicates the
model’s inference time.

###  5.4 Model optimization

In pursuit of enhancing the inference speed of Wizard Vicuna without
compromising the impressive accuracy we’ve attained, we turned our attention
to model optimization. Specifically, we assessed the performance of the model
post-optimization through quantization [27, 28].

Large Language Models (LLMs), while being remarkably powerful, often grapple
with substantial memory and computational overhead. These constraints present
formidable challenges in storage requirements and inference speed. A viable
solution to alleviate these challenges is quantization. For instance, while a
model might initially be trained using 32-bit floating-point numbers, through
quantization, these can be downscaled to 16-bit or even 8-bit integers. Of the
two predominant quantization techniques: quantization-aware training (QAT) and
post-training quantization (PTQ), we opted for the latter. This method
involves applying quantization after the model’s training phase, paving the
way for quicker model loading and reduced inference latency.

Table 6 shows the efficiency gains achieved through quantization. Both the
quantized and unquantized models boast an impressive accuracy of 100%.The
quantized model demonstrates a marked improvement in speed, averaging around
7.552 seconds per query, it also utilizes significantly less GPU memory,
coming in at 11.93 GB out of the available 32GB. It’s worth noting that for
the sake of patient privacy and dataset guidelines, the specific questions and
outputs have been omitted from the table, as these evaluations were conducted
directly on the sensitive MIMIC data.

Question | Quantized | Unquantized
---|---|---
1 | 6.44 s | 37.1 s
2 | 6.54 s | 15min 50s
3 | 8.24 s | 42.3 s
4 | 9.95 s | 1min 52s
5 | 6.59 s | 11min 29s
Total | 37.76 s | 30 mins 16s
Average | 7.552 s | 6 mins
GPU Memory | 11.93 GB | 17.56 GB
Table 6: Comparison of inference performance between quantized and unquantized
models, highlighting time efficiency, and GPU memory usage.

###  5.5 Domain specific fine tuning

In our concluding set of evaluations, we shifted our focus to the fine-tuning
of the Wizard Vicuna 7B model. The objective was to investigate if a domain-
specific fine-tuning of a smaller model could rival the accuracy of its
larger, non-fine-tuned counterpart. To facilitate this, we utilized a dataset
comprising 1,250 question-answer pairs, all meticulously curated based on the
MIMIC dataset. This approach underscores the potential of targeted model
training, honing in on domain specificity to possibly achieve comparable, if
not superior, performance metrics.

In our domain-specific fine-tuning, we leveraged the capabilities of QLoRA
[29]—an innovative fine-tuning approach renowned for its efficiency. QLoRA
allows for memory-efficient fine-tuning, making it feasible to fine-tune
colossal models, even on a single 48GB GPU without compromising performance.

Upon deploying the fine-tuning approach, the results were notably suboptimal.
As an illustrative example, certain queries that previously received precise
and coherent responses under other models now yielded vague or even inaccurate
answers. For example for the input question ’For patient 2, did she have
dysuria or did she sob when she had nausea?’, the output from the fine-tuned
model was ’For patient 3, did she have a fever or did she have a headache?For
patient 4, did she have a cough or did she have a sore throat? For patient 5,
did she have a runny nose or did she have a sneeze?’, which is basically
referred to as model hallucination as compared with the actual output ’she
denied any vomiting, rash, rhinorrhea, dysuria, cough, sob or abdominal
discomfort’. This outcome reinforced our initial hypothesis and underscored
the efficacy of our chosen strategy. The utilization of the RAG (Retrieval-
Augmented Generation) framework has consistently demonstrated superior
performance in the realm of information retrieval with LLMs. The contrast
between the fine-tuning results and the RAG-based approach confirms the latter
as the most robust and effective method for tackling the intricate challenges
of information extraction using large language models.

##  6 Limitations and Future Work

Throughout this research, we encountered several constraints:

  * •

GPU RAM Constraints: Models, particularly those having 13 billion parameters
or more, demanded significant GPU RAM, which at times limited the extent of
our experiments.

  * •

Fine-Tuning Overheads: The associated computational and time costs of fine-
tuning large models restricted the range of experiments we could conduct.

  * •

Evaluation Challenges: Establishing an appropriate evaluation mechanism was
challenging. Defining precise metrics and benchmarks to gauge the models’
performance was not straightforward.

Looking ahead, several avenues have been identified for further exploration:

  * •

Application Containerization: Our future endeavors include packaging the
chatbot within an application container, enhancing its deployment,
scalability, and accessibility.

  * •

Evaluation Strategies: A paramount next step involves formulating a more
structured approach to testing and evaluation, ensuring that results are
consistent and repeatable across various settings.

  * •

Expanding Dataset Scope: We aim to test our models on more extensive datasets,
hoping to gain deeper insights and uncover nuances not evident in the current
dataset.

##  7 Conclusion

In the rapidly evolving landscape of Electronic Health Records (EHRs), the
vast and intricate nature of clinical notes poses significant challenges for
manual retrieval and analysis. The presented work offers a transformative
solution by harnessing the power of state-of-the-art Large Language Models
(LLMs) to develop a dynamic conversational interface for clinical note
exploration. Leveraging the capabilities of the Langchain framework and
transformer-based models, the proposed system enables intuitive natural
language querying, extracting pertinent information directly from clinical
notes. Preliminary evaluations underscore the efficacy of combining semantic
embedding models with advanced LLMs for optimized information retrieval.
Notably, the Wizard Vicuna model emerges as a front-runner in accuracy, albeit
with substantial computational demands. Through judicious model optimization,
such as weight quantization, the team achieved significant reductions in
latency, paving the way for more agile and deployable solutions. While the
initial results are encouraging, the journey ahead warrants further
exploration to address inherent model limitations and refine evaluation
metrics.

## References

  * [1] Nils Reimers and Iryna Gurevych.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019\.  URL: https://arxiv.org/abs/1908.10084.
  * [2] Jennifer King, Vaishali Patel, Eric W. Jamoom, and Michael F. Furukawa.  Clinical benefits of electronic health record use: National findings.  Health Services Research, 49(1pt2):392–404, 2014.  URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/1475-6773.12135, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/1475-6773.12135, doi:https://doi.org/10.1111/1475-6773.12135.
  * [3] Douglas Manuel, Laura Rosella, and Thérèse Stukel.  Importance of accurately identifying disease in studies using electronic health records.  BMJ (Clinical research ed.), 341:c4226, 08 2010.  doi:10.1136/bmj.c4226.
  * [4] Douglas G Manuel, Laura C Rosella, and Thérèse A Stukel.  Importance of accurately identifying disease in studies using electronic health records.  Bmj, 341, 2010.
  * [5] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2023.  arXiv:1706.03762.
  * [6] Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou, and Ji-Rong Wen.  Large language models for information retrieval: A survey, 2023.  arXiv:2308.07107.
  * [7] Edmon Begoli, Sudarshan Srinivasan, and Maria Mahbub.  Improving efficiency and robustness of transformer-based information retrieval systems.  In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’22, page 3433–3435, New York, NY, USA, 2022. Association for Computing Machinery.  doi:10.1145/3477495.3532681.
  * [8] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  Bioinformatics, 36(4):1234–1240, 09 2019.  arXiv:https://academic.oup.com/bioinformatics/article-pdf/36/4/1234/48983216/bioinformatics\\_36\\_4\\_1234.pdf, doi:10.1093/bioinformatics/btz682.
  * [9] Souradip Chakraborty, Ekaba Bisong, Shweta Bhatt, Thomas Wagner, Riley Elliott, and Francesco Mosconi.  BioMedBERT: A pre-trained biomedical language model for QA and IR.  In Donia Scott, Nuria Bel, and Chengqing Zong, editors, Proceedings of the 28th International Conference on Computational Linguistics, pages 669–679, Barcelona, Spain (Online), December 2020. International Committee on Computational Linguistics.  URL: https://aclanthology.org/2020.coling-main.59, doi:10.18653/v1/2020.coling-main.59.
  * [10] Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, Mike Schaekermann, Amy Wang, Mohamed Amin, Sami Lachgar, Philip Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, Nenad Tomasev, Yun Liu, Renee Wong, Christopher Semturs, S. Sara Mahdavi, Joelle Barral, Dale Webster, Greg S. Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, and Vivek Natarajan.  Towards expert-level medical question answering with large language models, 2023.  arXiv:2305.09617.
  * [11] Deepak Suresh Asudani, Naresh Kumar Nagwani, and Pradeep Singh.  Impact of word embedding models on text analytics in deep learning environment: A review.  Artif. Intell. Rev., 56(9):10345–10425, feb 2023.  doi:10.1007/s10462-023-10419-1.
  * [12] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela.  Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021\.  arXiv:2005.11401.
  * [13] Alistair Johnson, Luca Bulgarelli, Tom Pollard, Steven Horng, Leo A. Celi, and Roger Mark.  MIMIC-IV (version 2.2).  PhysioNet, 2023.  URL: https://doi.org/10.13026/6mm1-ek67.
  * [14] Alistair Johnson, Tom Pollard, Steven Horng, Leo A. Celi, and Roger Mark.  MIMIC-IV-Note: Deidentified free-text clinical notes.  PhysioNet, 2023.  URL: https://doi.org/10.13026/1n74-ne17.
  * [15] Hugging Face.  Language models leaderboard, 2023.  Accessed: 07-2023.  URL: https://huggingface.co/spaces/mteb/leaderboard.
  * [16] Hugging Face.  Embedding models leaderboard, 2023.  Accessed: 07-2023.  URL: https://huggingface.co/spaces/mteb/leaderboard.
  * [17] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.  URL: https://lmsys.org/blog/2023-03-30-vicuna/.
  * [18] Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang.  Wizardlm: Empowering large language models to follow complex instructions, 2023.  arXiv:2304.12244.
  * [19] Together Computer.  Redpajama: An open source recipe to reproduce llama training dataset, 2023\.  URL: https://github.com/togethercomputer/RedPajama-Data.
  * [20] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto.  Stanford alpaca: An instruction-following llama model.  https://github.com/tatsu-lab/stanford_alpaca, 2023.
  * [21] Tianyu Han, Lisa C. Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K. Bressem.  Medalpaca – an open-source collection of medical conversational ai models and training data, 2023.  arXiv:2304.08247.
  * [22] Chavinlo.  Gpt-4 x alpaca, 2023.  URL: https://huggingface.co/chavinlo/gpt4-x-alpaca.
  * [23] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica.  Judging llm-as-a-judge with mt-bench and chatbot arena, 2023.  arXiv:2306.05685.
  * [24] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei.  Scaling instruction-finetuned language models, 2022.  URL: https://arxiv.org/abs/2210.11416, doi:10.48550/ARXIV.2210.11416.
  * [25] 64bits.  Lexpodlm-13b, 2023.  URL: https://huggingface.co/64bits/LexPodLM-13B.
  * [26] Sharegpt, 2022.  URL: https://sharegpt.com/.
  * [27] Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han.  SmoothQuant: Accurate and efficient post-training quantization for large language models.  In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 38087–38099. PMLR, 23–29 Jul 2023.  URL: https://proceedings.mlr.press/v202/xiao23c.html.
  * [28] Zhewei Yao, Xiaoxia Wu, Cheng Li, Stephen Youn, and Yuxiong He.  Zeroquant-v2: Exploring post-training quantization in llms from comprehensive study to low rank compensation, 2023.  arXiv:2303.08302.
  * [29] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer.  Qlora: Efficient finetuning of quantized llms, 2023.  arXiv:2305.14314.


# Pixel detector hybridization and integration with anisotropic conductive
adhesives

Alexander Volker,11footnotetext: Corresponding author. Janis Viktor Schmidt
Dominik Dannheim  Peter Svihra  Mateus Vicente Barreto Pinto  Rui de Oliveira
Justus Braach  Xiao Yang  Marie Ruat  Débora Magalhaes  Matteo Centis Vignali
Giovanni Calderini  Helge Kristiansen

###### Abstract

A reliable and cost-effective interconnect technology is required for the
development of hybrid pixel detectors. The interconnect technology needs to be
adapted for the pitch and die sizes of the respective applications. For small-
scale applications and during the ASIC and sensor development phase,
interconnect technologies must also be suitable for the assembly of single-
dies typically available from Multi-Project-Wafer submissions. Within the CERN
EP R&D program and the AIDAinnova collaboration, innovative and scalable
hybridization concepts are under development for pixel-detector applications
in future colliders. This contribution presents recent results of a newly
developed in-house single-die interconnection process based on Anisotropic
Conductive Adhesives (ACA). The ACA interconnect technology replaces solder
bumps with conductive micro-particles embedded in an epoxy layer applied as
either film or paste. The electro-mechanical connection between the sensor and
ASIC is achieved via thermocompression of the ACA using a flip-chip device
bonder. The ACA technology can also be used for ASIC-PCB/FPC integration,
replacing wire bonding or large-pitch solder bumping techniques. A specific
pixel-pad topology is required to enable the connection via micro-particles
and create cavities into which excess epoxy can flow. This pixel-pad topology
is achieved with an in-house Electroless Nickel Immersion Gold process that is
also under development within the project. The ENIG and ACA processes are
qualified with a variety of different ASICs, sensors, and dedicated
interconnect test structures, with pad diameters ranging from 12 $\mu$m to 140
$\mu$m and pitches between 20 $\mu$m and 1.3 mm. The produced assemblies are
characterized electrically, with radioactive-source exposures, and in tests
with high-momentum particle beams. This contribution introduces the developed
interconnect and plating processes and showcases different hybrid assemblies
produced and tested with the above-mentioned methods. A focus is placed on
recent optimization of the plating and interconnect processes, resulting in an
improved plating uniformity and interconnect yield.

##  1 Introduction

Interconnect technologies from fine pitch hybridization to large pitch module
packaging are important for a wide range of applications in the field of high-
energy physics and other silicon pixel-detector applications. In many cases
state of the art is solder bump bonding in which the electrical connection
between metallized pads is created by solder balls under reflow conditions.
This requires many steps of preparation including a process called Under Bump
Metallization (UBM) which are done on a wafer level prior to dicing. For multi
project wafers as they are common in detector R&D these wafer level processes
are either not possible or too costly for the scale of the project. Out of
this issue arises a need for cost effective single die processing with a fast
turn-around time. An innovative approach for in-house single die bonding is
discussed in this contribution. The electrical interconnection is created
using Anisotropic Conductive Adhesives (ACA) in the form of conductive
particles embedded in an epoxy paste or film and by thermo-compressing the
adhesive. Commercial use of these types of adhesives can be found in display
panel integration[1] and also increasingly in chip-to-film and chip-to-flex
bonding[3, 4]. The required surface properties of the devices bonded with ACA
are achieved with an in-house single-die plating process.

##  2 Electroless Plating

For the topology requirement needed to bond with the ACA, Electroless Nickel
Immersion Gold (ENIG) plating is utilized, a method typically used in PCB
production[5]. It is a chemical plating process that grows nickel on
metallized pads without the need for an external electrical current, making it
thereby use able for technologies operating on semiconductor electronics. In
the case of ASICs and sensors, the pad metallization widely used is an
aluminium alloy which first requires a pre-treatment of the pads in order for
the ENIG plating to create a catalytic surface on which the reaction can start
reliably. The first step after mounting the samples on a holder is plasma
cleaning them. This removes part of the thin oxide layer and increases the
wettability for further chemical processing. In the next step the oxide layer
is chemically removed and more roughened surface for plating to adhere to is
created. The next step is generally referred to as zincation. In this step a
thin seed layer of Zinc is chemically deposited on the aluminium surface. The
cycling of this step twice or thrice can increase the quality of the zinc seed
layer. The sample can now be submerged in the Electroless Nickel bath
containing hypophosphite and Ni-Ions to start the plating process as shown in
Figure 1. Due to the auto catalytic nature of the process the nickel will grow
radially outwards from all points of deposition, thereby creating a mushroom-
like shape with increasing plating height.

$\displaystyle\text{H}_{2}\text{PO}_{2}^{-}+\text{Ni}^{2+}+2\text{H}_{2}\text{O}\rightarrow\text{H}_{2}\text{PO}_{3}^{-}+\text{H}_{2}+2\text{H}^{+}+\text{Ni}^{0}$
(2.1)

Figure 1: The graphic shows the simplified chemical process in the Electroless
Plating bath when plating nickel on aluminium pads partially covered by a
passivation layer.

The plating height is controlled by the plating time while defects in the
plating growth process limit the height at which uniform growth on all pads
can be achieved. Defects observed are single pads not plated or skipped, pads
near edges of the chip un-plated or considerably lower plating height.
Observations show a plate out effect showing nucleation of nickel plating on
surfaces where plating is undesired (see Figure 2 right). This occurs more
frequently when the plating gets high enough for the space between pads to
shrink considerably. It starts a chain reaction creating increasingly more
nucleation spots that each grow with further plating time. A lead-based
stabilizer is used as catalyst poison in order to inhibit the plate out.
Another challenge to achieve an even plating topology are pads with different
metal structures and shorts. These pads can show different plating speeds
compared to the rest of the pixel matrix leading to uneven topologies (see
Figure 2 left).

Figure 2: Electroless Nickel defects on small pitch (55 $\mu$m) devices. The
left picture shows a electrically connected row of pads in the bottom of the
pixel matrix. This row shows frequently different plating behaviour compared
to the rest of the pixel matrix. The right picture shows plate out happening
between pads. Small dots that increase in number and grow leading to shorts.

After the Electroless Nickel process an Immersion Gold process is used to
create a thin gold layer on the previously plated nickel in order to protect
them from corrosive conditions. The gold atoms replace the nickel atoms at the
surface, slowly growing the gold layer. The chemical equation for this process
is shown in 2.2.

$\displaystyle\text{Ni}+2\text{Au}^{+}\rightarrow\text{Ni}^{2+}+2\text{Au}$
(2.2)

Figure 3: Graph showing the calculated increase of Ni-pad surface with
increased plating height assuming uniform radial growth. The initial Al-pad is
14$\mu$m in diameter and 2$\mu$m in height below the passivation layer.

Agitation of the Electroless Nickel bath is an important parameter to control
as it strongly affects the diffusion zone in which the chemical processes of
the plating take place[6]. A stirring hotplate setup is used where the
stirring speed is adjusted according to the surface of the growing nickel
bumps $\omega\sim r(r+1)$ . Due to the mentioned mushroom- like shape the
surface area will grow quickly for small pad devices as shown in Figure 3. All
measures of plating height are stated with reference to the passivation
covering the edges of the pads. Adjusting the stirring results in more uniform
topologies needed for higher interconnection yields.

##  3 Bonding with Anisotropic Conductive Adhesives

###  3.1 ACA materials

While the Anisotropic Conductive Films (ACF) are commercial products with
detailed bonding parameters (see Table 1), adjustments based on material and
pixel pad layout still have to be done. This can be the case for large chips
where it is required to squeeze out as much of the epoxy as possible prior to
curing the epoxy. For the Anisotropic Conductive Paste (ACP), commercial
epoxies with different conductive particles embedded are available for use
based on material characteristics such as viscocity and radiation hardness
needed for the device and bonding process. Mixing the conductive particles
into the paste is also done in-house. In use are a range of particles in sizes
and coating materials as shown in Table 1.

Table 1: (left) Table of commercially available ACF with highlighted number 1
and 2 due to their most common useage. (right) Table of conductive particles
available for inhouse ACP mixing.

###  3.2 Flip-chip bonding

ACF is applied to one side of the parts to be connected in a lamination
process defined by the manufacturer, while ACP is dispensed by a syringe
dispensing system. The dispensing force and the speed of the syringe tip while
dispensing is set to control the adhesive quantity. Reducing the quantity of
adhesive dispensed without creating uncovered pads due to skipping. For the
ACF the quantity of adhesive is controlled by the thickness of the film. A too
thick film of ACF will lead to an excess of high-viscosity adhesive resulting
in areas of unconnected pixels. For most applications, in particular fine
pitch ones, thinner films are therefore preferred. Newest developments in the
ACF industry show film thicknesses down to 10 $\mu$m with embedded metal
coated polymer particles of 3 $\mu$m diameter. Larger particles are required
when connecting uneven structures as the particle itself can be deformed to
less than half its diameter, therefore evening out the topological
differences. Smaller particles in the 3-4 $\mu$m diameter range however are
still in favor for most of fine pitch applications, as reliable connections
require a high particle density that can not be realised efficiently with
larger particles. For the commercially available ACF like ACF3 or ACF5,
simulations of the particle distribution show pads of devices with fine pitch
not having a high enough particle yield. An example of a simulation of an ACP
particle distribution for a larger pitch device is shown in Figure 4. The
particle placement is picked randomly and then in case of overlap pushed away
radially. This is done to emulate the conditions of particles that overlap in
2D top-down view but not 3D view to behave as if thermo-compressed.

A cross-section analysis shows particles being pinned between nonparallel pad
surfaces at an angle of 20° (see Figure 5). This allows for particles being
pinned beyond the nominal pad size as the ENIG plating creates round edges
around the pad.

Figure 4: Histogram of a simulation result showing the distribution of
particles pinned per pad. The pads are 50 $\mu$m in radius while the particles
are 10 $\mu$m in diameter. In this realisation a particle volume fraction of
0.10 is assumed. The total pad count is 6724.  Figure 5: Microscopy cross-
section picture of an ACP bonded ALTIROC2 ASIC (top) and an LGAD sensor
(bottom). Two conductive particles are pinned between the ENIG plated pads,
therefore electrically connecting the two pads. The right particle is pinned
between two parallel planes while the left on is pinned between two planes at
an angle of about 20°.

For the flip-chip process, a device bonder that can apply up to 100 kilograms
of force and temperatures up to 400°C is used. The temperature used is
dependent on the curing temperature of the epoxy used as adhesive for the ACA.
For ACF1 and ACF2 150°C is used as curing temperature while bonding. The
bonding pressure used can be as high as 90 MPa for the pad connection area
with some devices requiring a lower pressure on the sensor material. For ACP
the temperature and bonding pressure applied is generally lower and highly
dependent on the materials used. The bonding force is applied till after the
cooling of the sample back to room temperature.

##  4 Results

###  4.1 ENIG plating

Consistent plating results for large pitch (>1000$\mu$m) substrates with large
pads(>50 $\mu$m in diameter)have been achieved, showing uniform plating
heights as desired for the applications. Plating heights for those substrates
up to 20 $\mu$m have been achieved. Other types of substrates show defects
limiting the plating to certain ranges depending on the specific layout. Small
pitch (<50$\mu$m) pixel matrices tend to plate out once growing pads decrease
the gap between the pads. For small pads(<20$\mu$m diameter) the plating is
perturbed and therefore strongly influenced by bath parameters such as
temperature, agitation, and concentration of bath constituents. This results
in large-pad substrates with small pixel pitches showing plating resilient to
perturbations, but early onset of plate out. The plating height for such
substrates is therefore limited by the distance between the pixel pad edges.
Substrates with large pitches and small pads are dominated by perturbations.
Substrates with small pitches and small pads pose the biggest challenge for
the current ENIG plating process. ENIG plating on chips with a pixel pitch of
50 $\mu$m and 7 $\mu$m pad radius has been achieved up to a plating height of
8 $\mu$m by carefully controlling stabiliser concentrations and stirring
speeds while processing.

###  4.2 Test-Structure Bonding

Figure 6: Schematic of a daisy chain device for testing of ACA bonded
interconnections.

For a systematic characterisation an optimisation of the plating and
interconnect process, test devices with a daisy chain connection (see Figure
6) have been designed and produced at FBK (Trento, Italy). These devices have
a pixel-pad layout similar to common hybridisation and module interconnect use
cases. Three daisy chain structures were produced and bonded that are shown in
Figure 7. They emulate an ASIC with a peripheral pad layout on a 20 mm by 20
mm glass substrate. In total 536 connection pads measuring 88 $\mu$m by 88
$\mu$m with a 140 $\mu$m pixel pitch. The ACA used for these tests were ACF1
and ACF2 and one ACP1 with 4 $\mu$m diameter silver coated conductive
particles embedded in Araldite2011. All pads that are covered by the adhesive
and have intact read out lines for measuring the resistance show a connection.
The electrical resistance is most strongly impacted by the wire length to the
readout/measurement pads near the bottom of the device. In the graph the
resistances are shown as a function of wire length for the readout connection.
The value of electrical resistance per connection chain is calculated to be
below 5 Ohms, after deducting the line resistivity. The ACP sample shows a
larger statistical spread than the ACF samples, due to the lower particle
density of 5% volume fraction used.

Figure 7: Pictures of the three different daisy chain devices and their
corresponding electrical resistance measurements. On the left is the ACF1
applied as sheet while the ACF used in the device in the middle is made with
ACF2 applied in four separate stripes. For the assembly shown on the right the
ACP was dispensed in four lines to get a good coverage without risking air
entrapment.

###  4.3 LGAD Hybridisation

Developing and testing interconnection of large pitch modules has been done as
an alternative for solder bump bonding where additional mechanical strength is
required. The ALTIROC2 ASIC[7] as well as the Low Gain Avalanche Detector
(LGAD)[8] sensors produced by USTC (Hefei, China), have a pixel matrix of
15x15 pixels. The pixel pitch is 1.3 mm and the connecting area per pad is
around 8000 $\mu$m². The adhesive used for bonding is ACP3 with Ag coated 10
$\mu$m diameter particles. Mechanical samples for cross section microscopy and
functional samples have been produced this way. The results of the cross-
sections are shown in Figure 8 and Figure 9. The distances between pads in the
cross-section measurements are close enough to pin particles in all observed
measurements. The functional assembly shows a connection yield of above 98%.

Figure 8: Microscopy picture of the cross-section for an ALTIROC2/LGAD device.
The connection shown is from row 10 and pad number 7. It shows clearly two
particles of 10 $\mu$m diameter being strongly squished between two ENIG
coated pads.  Figure 9: 2D map of the cross-section measurements done on the
ALTIROC2/LGAD sample. The color coded bar on the right shows the distance in
$\mu$m between the ENIG pads of the ASIC and sensor. Connections are confirmed
for all analysed pads. Row 1 and 17 correspond to pads on the ASIC that
connect to the guard ring on the sensor.

###  4.4 SPHIRD Hybridisation

Pixel detector developments for the SPHIRD[9] project of X-ray detectors at
the European Synchrotron Radiation Facility (ESRF)(Grenoble, France) use
high-Z sensor materials. The hybridization of sensors made of such materials
like CZT has proven to be challenging due to the required single-chip
processing as well as temperature and pressure sensitivity of the sensors. The
ASIC and sensors used in this study contain a pixel matrix of 32x64 pixels
with a pitch of 50 $\mu$m. The ASIC is silicon based while various sensor
materials (Si, CdTe, CZT)) are used. First preliminary beam test results for
an ACF bonded ASIC to silicon sensor made with ACF2 have been achieved. The
results of the test-beam study show that 85% of the pixels show a response
expected for a good connection while about 7% show a weak response. 7% show no
response. Mechanical sample testing has been performed and analysed via
microscopy. The mechanical sample tests use an ENIG plated silicon sensor (as
ASIC substitution) and an unplated CZT sensor to test bonding on a sample with
plating only on the ASIC side. The cross-section pictures in Figure 10 show
connections between the pads either via particles or direct pad-to-pad
connections.

Figure 10: Microscope picture of a cross-section performed on a mechanical
assembly of a silicon and a CZT SPHIRD sensor, where only the silicon sensor
has been ENIG plated. Particles can bee seen in the small gap between CZT-
sensor and ENIG pad.

###  4.5 Timepix3 Hybridisation

Bonding of Timepix3[10] ASICs to planar silicon sensors of 300 $\mu$m
thickness is performed with ACF. The Timepix3 ASIC has a pixel matrix of
256x256 with one additional row of guard ring pads. The pixel pitch is 55
$\mu$m and the pad diameter is 12 $\mu$m. For devices like the Timepix3 the
distance from the center of the pixel matrix to the edge of the chip is too
large too efficiently squeeze the epoxy out of the matrix. Achieving a
sufficient plating height needed to displace the excessive adhesive in gaps
between the ENIG pads is challenging. ASIC-side plating of 7-8 $\mu$m height
above the passivation has been achieved with no defects. The bonding has been
done with ACF1 and an extended pre bonding step. For this the temperature was
kept at 80°C for 600 seconds while the maximum bonding force of 100 kgf was
applied, to give the adhesive time to squeeze out of the connection. This was
done at reduced temperatures to ensure that the epoxy does not cure
prematurely. Afterwards the temperature is raised to 150°C for 5s bonding and
full curing.

Figure 11: Pixel efficiency maps of a Timepix3 assembly.Obtained from a test-
beam measurement with minimum ionizing particles and a reference beam
telescope. The left map shows the efficiency close to baseline while the right
map shows the efficiency about 4000 $e^{-}$ from baseline.

The produced device was analysed in a test beam measurement. Figure 11 shows
that moderate increase in detection threshold leads to reduced efficiency in
the centre regions of the device, as expected from weakly coupled connections.
This is assumed to be caused by too large quantities of adhesive in the
centre, which could not be squeezed out of the matrix.

##  5 Conclusions

Electrical measurements on dedicated chain devices confirmed that a low-ohmic
connection with high interconnect yield can be achieved for large pitch and
large pad areas for both ACF and ACP. The results presented for various ASICs
and sensors, show promising ACA applications in a wide range of use cases. For
large pitch hybridisation it offers an alternative to bump bonding with
enhanced mechanical stability provided by the adhesives good interconnect
yield. The processes under development have proven beneficial for the R&D
efforts of the participating detector projects, due to the fast turnaround and
low cost. The results of the SPHIRD project show a not yet sufficient yield
for such small-scale and fine pitch devices. ACA shows benefits in the bonding
of the CZT sensors, where standard solder bump bonding is not use able.
Simulation results of particle distributions show that a higher particle
density is required for a sufficient connection yield. For larger matrices
such as Timepix3 the effects relating to adhesive quantity and maximum bonding
force of the equipment are to be considered. Currently plating heights
sufficient and uniform required to displace the amount of adhesive of the ACFs
in use are achieved.

##  6 Outlook

A higher ENIG plating and/or thinner ACF or a reduced ACF coverage is needed
to a achieve a better connection yield for fine pitch assemblies. More daisy
chain test structures are being produced for further bonding tests as well as
thermal and mechanical testing in a dedicated reliability study. Tests with
larger particles for ACP are planned for large pitch applications.

## Acknowledgments

This project has received funding from the European Union’s Horizon 2020
Research and Innovation programme under GA no 101004761.

## References

  * [1] MJ Yim and KW Paik, _Design and understanding of anisotropic conductive films (ACF’s) for LCD packaging_ , _IEEE Transactions on Components, Packaging, and Manufacturing Technology: Part A_ , 1998, 21, 2, 226-234.
  * [2] SM Chang et al., _Characteristic study of anisotropic-conductive film for chip-on-film packaging_ , _Microelectronics Reliability_ , 2011, 41, 12, 2001-2009.
  * [3] P. Svihra et al., _Development of novel single-die hybridisation processes for small-pitch pixel detectors_ , _JINST_ , 2023, 18, C03008.
  * [4] J.V. Schmidt et al., _Pixel detector hybridisation with Anisotropic Conductive Films_ , _JINST_ , 2023, 18, C01040.
  * [5] C. M. Chan et al., _Development of novel immersion gold for electroless nickel immersion gold process (ENIG) in PCB applications_ , _2010 5th International Microsystems Packaging Assembly and Circuits Technology Conference_ , 2010, 1-4.
  * [6] X. Yin et al., _Role of a Pb2+ Stabilizer in the Electroless Nickel Plating System: A Theoretical Exploration_ , _J. Phys. Chem. B_ , 2004, 108, 10919-10929
  * [7] ATLAS-HGTD collaboration, M. Morenas, _Performance of ALTIROC2 readout ASIC with LGADs for ATLAS HGTD picosecond MIP timing detector_ _JINST_ , 2023, 18, C01070.
  * [8] C.H. Li et al., _Performance of LGAD sensors with carbon enriched gain layer produced by USTC_ , _Nuclear Instruments and Methods in Physics Research - section A_ , 2022, 1039, 167008\.
  * [9] P. Grybos et al., _SPHIRD–Single Photon Counting Pixel Readout ASIC With Pulse Pile-Up Compensation Methods_ , _IEEE Transactions on Circuits and Systems II: Express Briefs_ , 2023, 70, 3248-3252.
  * [10] T. Poikela et al., _Timepix3: a 65K channel hybrid pixel readout chip with simultaneous ToA/ToT and sparse readout_ _JINST_ 2014 9, C05013.