[VOLUME # TBD] 2024 Machine Learning for Healthcare

# Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in
Radiology with General-Domain Large Language Model

Seonhee Cho<EMAIL_ADDRESS>
Kim Jaechul Graduate School of AI
KAIST
Seongnam  Gyeonggi  Korea  Choonghan Kim<EMAIL_ADDRESS>
Moon Soul Graduate School of Future Strategy
KAIST
Daejeon  Korea  Jiho Lee
Department of Computer Science and Engineering
Ewha Womans University
Seoul  Korea  Chetan Chilkunda
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh  PA  USA  Sujin Choi
Division of Environmental Science and Engineering
POSTECH
Pohang  Gyeongbuk  Korea  Joo Heung Yoon<EMAIL_ADDRESS>
Division of Pulmonary Corresponding author Allergy  Critical Care  and Sleep
Medicine
Department of Medicine
University of Pittsburgh
Pittsburgh  PA  USA

###### Abstract

Recent advancements in Large Multimodal Models (LMMs) have attracted interest
in their generalization capability with only a few samples in the prompt. This
progress is particularly relevant to the medical domain, where the quality and
sensitivity of data pose unique challenges for model training and application.
However, the dependency on high-quality data for effective in-context learning
raises questions about the feasibility of these models when encountering with
the inevitable variations and errors inherent in real-world medical data. In
this paper, we introduce MID-M, a novel framework that leverages the in-
context learning capabilities of a general-domain Large Language Model (LLM)
to process multimodal data via image descriptions. MID-M achieves a comparable
or superior performance to task-specific fine-tuned LMMs and other general-
domain ones, without the extensive domain-specific training or pre-training on
multimodal data, with significantly fewer parameters. This highlights the
potential of leveraging general-domain LLMs for domain-specific tasks and
offers a sustainable and cost-effective alternative to traditional LMM
developments. Moreover, the robustness of MID-M against data quality issues
demonstrates its practical utility in real-world medical domain applications.

##  1 Introduction

Recently, large multimodal models (LMMs) have made significant progress,
becoming capable of interpreting and integrating information across different
data types, such as text and images (Achiam et al., 2023; Driess et al.,
2023). This progress closely aligns to advancements in Large Language Models
(LLMs), since many LMMs are built upon the structural foundation of pre-
trained LLMs (Touvron et al., 2023; Chiang et al., 2023). Consequently,
attempts to use LMM’s in-context learning capability without fine-tuning on
specific tasks are increasing (Wei et al., 2022; Zhang et al., 2023b). This
tendency also appears in the medical domain, where the data collection is
challenging due to the concerns on using sensitive clinical information
containing re-identifiable patient data (Benitez and Malin, 2010) and the
expensive labeling process which requires expert knowledge.

However, the performance of in-context learning approach is highly susceptible
to the data quality (Liu et al., 2023). One primary cause of degrading data
quality is data loss occurring during the data collection and curation
process, even when guided by expert input. For instance, the analysis of
radiographic images has reported an error rate of 3 to 5% (Robinson et al.,
1999) and the problem appears challenging to resolve until now (Brady, 2017).
Similarly, a basic chart review showed electronic health record (EHR) errors
could reach 9 to 10% (Feng et al., 2020). Additionally, variations in data
interpretation by medical professionals are quite common (Whiting et al.,
2015) and this can further exacerbate these issues by adding another layer of
complexity to ensuring the quality and consistency of data.

In this paper, we introduce MID-M, a Multimodal framework with Image
Description for Medical domain. It is an in-context learning framework that
demonstrates robust performance even with low-quality data. Our framework uses
general-domain LLM and leverages multimodal data by converting images to
textual descriptions. Notably, it achieves comparable performance to other
general-domain and fine-tuned LMMs without pre-training on multimodality and
extensive fine-tuning for the medical domain. In addition, by processing the
image as a text description, it has the advantage of representing the image in
an accessible and interpretable way, compared to traditional embedded vector
representations. Our framework is illustrated in Figure 1.

Figure 1: Overview of MID-M framework.

To evaluate the model’s performance in the scenario with low-quality data, we
systematically mask the texts in a medical dataset and make the new
adversarial evaluation data. We compare the performance of general domain pre-
trained and medically fine-tuned LMM models to our framework in a few-shot
setting, focusing on their accuracy and semantic comprehensibility. Through
this experiment, we aim to assess the effectiveness of LMMs in generalizing to
domain-specific tasks and managing incomplete source data.

### Generalizable Insights about Machine Learning in the Context of Healthcare

In this study, we analyze the generalization capabilities of LLMs and LMMs
across various dimensions. First, we investigate the ability of models trained
in general domains to effectively generalize to medical texts through few-shot
learning. This aligns with the recent trend towards adopting eco-friendly
approaches in AI model training by reducing the need for massive pretraining
and extensive fine-tuning. It also contributes to the validation of model’s
potential to generalize well across any domain or task, as aimed for in
general domain research. Second, we verify the models’ ability to generalize
in situations where data quality deteriorates. The challenge that in-context
learning models are easily influenced by the number of few-shot samples and
the quality of trained data remains unresolved. Therefore, concrete evaluation
in this regard is crucial for the advancement of models.

Furthermore, our research reveals that text-only models can efficiently handle
multimodal healthcare tasks with less computational demand. We demonstrate
that smaller language models can achieve results comparable to larger models,
indicating a path for wider adoption of AI in healthcare, especially in
resource-limited settings in computation or a lower level of healthcare
environment. This goes in line with a move towards developing globally
accessible AI solutions by the National Institutes of
Health111https://commonfund.nih.gov/bridge2ai, highlighting the potential
utility of proposed research in expanding AI’s reach.

##  2 Background

In this section, we review the relevant literature on large language or
multimodal models in the general domain as well as those applied specifically
to the tasks in healthcare.

###  2.1 Large Language Models for Vision and Language Tasks

#### Multimodal pretraining

Large multimodal models have often leveraged language models as the backbone
architecture and incorporate methods to connect the image encoder with the
language model (Awadalla et al., 2023; Alayrac et al., 2022; Li et al., 2023a,
b; Xu et al., 2024; Laurençon et al., 2024; Liu et al., 2024; Dai et al.,
2024). Notably, LLaVA (Liu et al., 2024) employs a direct projection layer,
BLIP-2 (Li et al., 2023b) uses a lightweight Q-Former, and Flamingo (Alayrac
et al., 2022) uses gated cross-attention, to name a few examples.

However, this paradigm suffers from two critical drawbacks. First, pretraining
requires a high computational burden. Jointly aligning vision backbones and
large language models requires large computational resources, despite
considerably fewer trainable parameters compared to the total parameters. For
example, Flamingo takes two weeks with 1,536 TPUs for the largest run, and
BLIP-2 takes around 10 days with 16 A100 GPUs. Hence, it becomes prohibitively
expensive to switch to a different large language model. The second limitation
is the lack of modularity and flexibility. Even though they integrate
pretrained image encoder and frozen LLM, updating this integrated system
demands training the network with substantial data instances and time
(Laurençon et al., 2024), hindering iterative research and development.

#### Multimodal Reasoning with Language

There have also been approaches that attempt to tackle multimodal tasks using
language models alone (Yang et al., 2022; Guo et al., 2022; Hu et al., 2022;
Xie et al., 2022; Surís et al., 2023; Berrios et al., 2023). Among these, LENS
(Berrios et al., 2023) is particularly relevant to our work, as this framework
extracts generic information such as tags, attributes, and captions with three
different vision modules, and provides them to the language model. While this
method can be beneficial for general domain tasks, such an approach does not
scale well to tasks that require expert knowledge. In contrast, our framework
facilitates LLM’s multimodal reasoning within the medical domain. We introduce
a more streamlined framework with a single ‘vision module’ and tailor a
general domain LLM specifically for medical tasks, by leveraging the
adaptability of the LLM in this knowledge-intensive domain.

###  2.2 Large Language Models for Healthcare Domain

Several efforts have been made to specialize these general-purpose foundation
models through fine-tuning on healthcare domain-specific data (Li et al.,
2024; Moor et al., 2023; Zhang et al., 2023a; Wu et al., ; Tu et al., 2024;
Sun et al., 2023). However, task-specific fine-tuning is often inefficient,
requiring large amounts of high-quality medical data and extensive computing
resources.

Consequently, efficient adaptation methods are also explored, which primarily
follow two categories: lightweight fine-tuning techniques (Hu et al., 2021;
Houlsby et al., 2019), and leveraging the models’ in-context learning
capabilities in a few-shot setting. While fine-tuned models generally
outperform their counterparts in few-shot setting(Van Veen et al., 2023b, a;
Lewis et al., 2020), there is a growing interest in exploring domain
adaptation techniques, mainly in-context learning, which do not require
additional training (Agrawal et al., 2022; Liévin et al., 2023; Byra et al.,
2023; Yan et al., 2023; Nori et al., 2023; Van Veen et al., 2023a). In-context
learning has many potential advantages for the clinical domain because there
is often a limited set of labeled data due to the high level of expertise
needed for annotation. Our work also aligns with this in-context learning
approach. By leveraging the capabilities of general-purpose LLMs without
additional fine-tuning, our approach aims to overcome the limitations of task-
specific fine-tuning while maintaining the models’ broad knowledge and
generalization abilities.

##  3 Methods

This section introduces MID-M, an efficient in-context learning framework that
performs multimodal tasks using only small-sized language models. MID-M
consists of two key components: a domain-specific image converter and a
general-domain language model.

#### Image Conversion with Domain-Specific Image Classifier

To reformulate vision-language tasks to text-to-text task, we first extract
key information from images and convert it to text. Taking inspiration from
Classification-via-Description (Menon and Vondrick, 2022) which showed good
generalization performance, we employ a Description-via-Classification
strategy. Classification-via-description involves generating textual features
representing each class with a language model, then analyzing whether the
input exhibits those features to make the final classification decision.
Conversely, we use the classification result to generate an image description.
Note that using a general-domain image module trained only on generic data, as
in Berrios et al. (2023), cannot yield meaningful classification results since
chest X-rays require domain knowledge. Therefore, we utilize a SOTA model
(Chong et al., 2023) for CheXpert, a publicly available large chest X-ray
classification dataset. This classifer computes a probability of 14 major
diseases for chest radiology images. For diseases that the classifier predicts
to have a probability of presence greater than 20%, we use the sentence “It
seems there is {disease} in the image.” Conversely, for diseases predicted as
absent, we use “It seems there is no {disease} in the image.” These sentences
describing the presence or absence of each disease are all concatenated and
used as an image description. This method can be extended to other domains
without domain expertise, by simply replacing the classifier with a domain-
specific one. We provide an example of our prompt in Appendix A.

#### General-domain LLM

We utilize a general-domain language model for multimodal reasoning. While
chains-of-thought (Wei et al., 2022) are generally advantageous for domain
adaptation and complex reasoning, their benefits are not consistent across
generation tasks (Kim et al., 2023). Furthermore, it requires domain knowledge
to generate rationales. Thus, we utilize strategies known to be helpful for
domain adaptation (Van Veen et al., 2023a) instead. Specifically, we prepend a
sentence assigning the role of an AI assistant as a domain expert: “You are an
expert medical professional.” This is followed by a detailed task description
crafted for the radiology domain. For the few-shot examples, we provide the
image description, corresponding finding, and impression in sequence. During
inference, we omit the impression from the examples to be completed by the
model. We use a BM25 retriever, known for its capability to compute token
overlap quickly and retrieve relevant examples effectively.

##  4 Dataset

We utilized the MIMIC-CXR dataset (Johnson et al., 2019), a de-identified and
publicly available chest radiograph database collected from 2011 to 2016. It
has a total number of 227,835 images and 128,032 paired reports from 65,379
patients. Paired reports include finding and impression. A finding is a
detailed observation made from the X-ray images written by radiologists, and
an impression is a concise summary of the finding which becomes the primary
mode of communication between medical professionals.

#### Preliminary Analysis

Our examination of the MIMIC-CXR dataset revealed 3,146 samples (2.46% of the
total) that were unsuitable for research due to one or more of the following
reasons: the finding label contained fewer than three words, was shorter than
the impression label, or had more than three words masked. This level of
incompleteness aligns with error rates reported in previous studies (Robinson
et al., 1999).

#### Pre-preprocessing

To prepare the data for our study, we first merged the original training,
validation, and test sets. We then removed samples with the findings at both
extremes of length, removing the shortest and longest 25% of findings to
ensure a meaningful level of detail remained after applying masking
techniques. This process resulted in a dataset of 64,613 images and reports,
which we then randomly split into training, validation, and test sets
containing 62,613, 1,000, and 1,000 samples, respectively.

#### Masking

To further evaluate the robustness of LLMs against incomplete data, we
employed a masking technique akin to that used by BERT (Devlin et al., 2018),
randomly deleting words or phrases at subword levels. We conducted masking at
three different rates to the original test set and yielded four distinct test
sets: original (full), corrupted at 10% (corrupted 0.1), corrupted at 30%
(corrupted 0.3), and corrupted at 50% (corrupted 0.5).

We evaluated the effectiveness of our corruption strategy using the F1CheXbert
(Zhang et al., 2020) metric. CheXbert (Smit et al., 2020) is designed to
identify 14 medical observations in chest X-ray images. The F1CheXbert score
is calculated by comparing CheXbert’s predictions on the generated text
against the corresponding reference text. We compare the findings from full
against the findings from each corrupted test set. We discovered a strong
correlation between the extent of text corruption and the precision of disease
mentions. The micro-average F1CheXbert scores were 91.2% for the 10% corrupted
set, 69.3% for the 30% corrupted set, and 51.1% for the 50% corrupted set.
Examples of corrupted text are illustrated in Table 1.

Table 1: Example of MIMIC-CXR findings with different masking rate. Full |  Corrupted 0.1 |  Corrupted 0.3 |  Corrupted 0.5
---|---|---|---
PA and lateral views of the chest provided. AICD projects over the left chest wall with lead tip extending to the region of the right ventricle. The heart is mildly enlarged. There is no evidence of pneumonia or CHF. No effusion or pneumothorax seen. Bony structures are intact. |  PA and lateral views _ the chest provided. AICD projects over the _ chest wall with lead tip extending to the region of the right ventric_. _ heart is mildly enlarged. there is no _ of pneumonia or CHF. _ effusion or pneumothorax seen. Bony structures are intact. |  PA and lateral _ of _ chest provided. _CD projects over the left _ wall with lead tip _ to the region of the _ ventric _ the heart is _. _ of pneumonia or CH_. no effusion or pneum_rax seen. _ intact. |  PA and lateral views of _ provided. AI _ the _ chest _ with _ tip _ to the _ vent _ heart is _ enlarged. there _ no _ F _ no _ usion or pneum _rax _ structures are intact _

##  5 Experimental Setup

###  5.1 Task

In this study, the task requires the model to generate an impression based on
an X-ray image and its corresponding findings. For all experiments, we
employed a 2-shot learning approach, utilizing the BM25 algorithm to retrieve
two training samples with findings similar to the findings of the test sample.
When experimenting with corrupted test sets, the samples are retrieved based
on the corrupted findings. This task is designed to comprehensively evaluate
the model’s ability to understand medical images and clinical data, as well as
to identify salient information even when incomplete data is given.

###  5.2 Baselines

We have two groups of baselines in our experiments: LMMs that are pre-trained
in the general domain and those further fine-tuned with medical domain data.
However, considering the constraints of a few-shot learning setting, we found
a limited number of LMMs are capable of processing multiple independent images
in a single prompt. Notably, Flamingo (Alayrac et al., 2022), is trained on
interleaved text and image pairs and supports multiple images. We used its
open-sourced version, OpenFlamingo (Awadalla et al., 2023). This functionality
is also applied to the models that are based on Flamingo, IDEFICS (Laurençon
et al., 2024) and OTTER (Li et al., 2023a). For models trained on data from
the medical domain, we selected MedFlamingo (Moor et al., 2023) and RadFM (Wu
et al., 2023). MedFlamingo is trained with medical publications and textbooks
and is based on OpenFlamingo. RadFM is a specialized model for the radiology
domain and pre-trained with large-scale, high-quality multimodal dataset named
MedMD. It’s important to note that although MedMD includes MIMIC-CXR, we still
chose to include it in our baseline set. Despite data leakage concerns in our
full test setting, we believe that evaluating the model’s performance on
corrupted versions of MIMIC-CXR would provide valuable insights into its
robustness and generalization capabilities.

A detailed comparison is presented in Table 2. Note that all the baseline
models have 9 billion parameters or more, whereas our backbone model,
Flan-T5-xl, has only 3 billion parameters. For the baselines, we adhered to
the example code in their public GitHub repositories when available;
otherwise, we utilized a consistent prompt with the example code from other
baselines.

Table 2: Descriptions of baseline models. The first four models are general-domain LMMs that can handle multiple images in one input. The two models in the middle are LMMs that are fine-tuned with medical domain data. Model | Size | Domain | Image Encoder | Language Model
---|---|---|---|---
Otter${}_{\text{M}}$ | 9B | General | ViT-L/14 | MPT-7b
Otter${}_{\text{L}}$ | 9B | General | ViT-L/14 | LLaMA-7b
OpenFlamingo | 9B | General | ViT-L/14 | MPT-7b
IDEFICS | 9B | General | ViT-H/14 | LLaMA-7b
MedFlamingo | 9B | Medical | ViT-L/14 | LLaMA-7b
RadFM | 14B | Radiology | ViT-B/32 | MedLLaMA-13b
MID-M | 3B | General | Classifier(7M) | Flan-T5-xl

###  5.3 Evaluation

The performance of the models are compared using two different LLM evaluation
methods, ROUGE-L (Lin, 2004) and F1RadGraph (Delbrouck et al., 2022). ROUGE,
the Recall-Oriented Understudy for Gisting Evaluation, is a metric widely used
in the text summarization task. This score evaluates the similarity between
the generated text and the reference text based on token overlap.
Specifically, we used ROUGE-L which measures the longest common subsequence
(LCS) to assess similarity at the sentence level. F1RadGraph is a metric that
evaluates medical entities and relations within the generated reports. It
utilizes a PubMedBERT (Gu et al., 2021) fine-tuned on the RadGraph (Jain et
al., 2021), a dataset with chest X-ray radiology reports annotated with
medical entities and their relations. This fine-tuned PubMedBERT model
identifies the entities and relations from the generated reports and the
reference reports. When the generated report correctly matches the entities
and their relation to that in the reference, they receive a reward of 1, and
otherwise, 0. The final F1 score is computed based on this reward score. Both
metrics are implemented using the Python Package Index
(PyPI)222https://pypi.org/project/radgraph/

##  6 Experiment

###  6.1 Result

Table 3: Model performances in each test set with regard to the accuracy. The highest score is bolded and the second highest is underlined. Method | Full | Corrupted 0.1 | Corrupted 0.3 | Corrupted 0.5
---|---|---|---|---
ROUGE
Otter${}_{\text{M}}$ | 0.3140 | 0.2258 | 0.2116 | 0.1894
Otter${}_{\text{L}}$ | 0.2655 | 0.2089 | 0.1876 | 0.1461
OpenFlamingo | 0.4310 | 0.3458 | 0.3420 | 0.2888
IDEFICS | 0.3951 | 0.3776 | 0.3321 | 0.2807
MedFlamingo | 0.3804 | 0.3622 | 0.3270 | 0.2784
RadFM | 0.5222 | 0.3952 | 0.3299 | 0.2534
MID-M(ours) | 0.4064 | 0.3719 | 0.3438 | 0.2977
F1RadGraph
Otter${}_{\text{M}}$ | 0.2801 | 0.2114 | 0.1704 | 0.1327
Otter${}_{\text{L}}$ | 0.2587 | 0.1994 | 0.1720 | 0.1237
OpenFlamingo | 0.4030 | 0.3427 | 0.3151 | 0.2541
IDEFICS | 0.4089 | 0.3688 | 0.3064 | 0.2543
MedFlamingo | 0.3973 | 0.3510 | 0.3046 | 0.2566
RadFM | 0.5054 | 0.3839 | 0.3094 | 0.2363
MID-M(ours) | 0.3732 | 0.3379 | 0.2973 | 0.2622

The experimental results are presented in Table 3. MID-M achieves performance
comparable to that of other baseline models in all settings, despite using
only a third of the parameters. It even surpasses other models in experiments
with corrupted data, highlighting its exceptional generalization capabilities.
RadFM, which previously trained on MIMIC-CXR, achieves the highest score on
the full test set, which is not a surprising result. IDEFICS and OpenFlamingo
demonstrate strong generalization ability, achieving scores around 0.4 across
both ROUGE and F1RadGraph metrics.

However, we observed that some models are highly sensitive to even a minor
level of data corruption (i.e. 0.1). For instance, RadFM suffers from a
significant performance drop in corrupted settings and shows performance
similar to or even worse than large-scale general domain models. This
sensitivity to text corruption is likely due to its training on only high-
quality radiology images and associated texts. It is critical problem in
clinical applications where even small decreases in performance can
significantly impact patient care. Otter and OpenFlamingo also shows
considerable performance declines in corrupted settings. Conversely, IDEFICS,
MedFlamingo and our framework maintain robust performance on low-quality data.
As the text masking probability increases (i.e. 0.3, 0.5), our model begins to
demonstrate its full potential. It achieves the highest scores under the ROUGE
metric and exhibits either comparable or superior performance on the
F1RadGraph metric.

It is also surprising that MedFlamingo, which is further fine-tuned from
OpenFlamingo with medical domain text, does not outperform OpenFlamingo. This
may be because MedFlamingo is primarily fine-tuned for medical visual
question-answering tasks rather than various medical generation tasks. These
experimental results shows the importance of task setting during pre-training
and fine-tuning.

###  6.2 Ablation Studies

To comprehend the the effect of each component in our framework to the overall
performance, we systematically remove each element - textual input (findings),
image input (descriptions), and both. In scenarios where both text and images
are excluded, the model must generate impressions based solely on two
impressions retrieved based on the similarity of findings. The results of
these ablation studies are presented in Table 4.

The results indicate that the multimodal framework exhibits the strongest
generalization capability by leveraging all available information. Since the
impressions are primarily derived from findings, the text component alone
proves to be most effective when the text data encompasses sufficient
information. Still, using both image and text information achieves comparable
scores and outperforms in the corrupted test sets. When text corruption
exceeds a 10% probability, incorporating image descriptions improves the
model’s accuracy compared to using only the findings or only the image
descriptions in most cases. One interesting point is, at the highest level of
text corruption, the corrupted text acts as noise and severely impairs the
model’s performance, leading to worse outcomes than when both text and images
were excluded. This trend remains consistent across evaluation metrics of
token overlap, measured by ROUGE, and semantic coherence, measured by
F1RadGraph.

Table 4: Model performances in different input settings. Method | Full | Corrupted 0.1 | Corrupted 0.3 | Corrupted 0.5
---|---|---|---|---
ROUGE
MID-M | 0.4064 | 0.3719 | 0.3438 | 0.2977
w/o text | 0.3794 | 0.3563 | 0.3440 | 0.2942
w/o image | 0.4119 | 0.3683 | 0.3218 | 0.2550
w/o text and image | 0.3649 | 0.3309 | 0.3129 | 0.2670
F1RadGraph
MID-M | 0.3732 | 0.3379 | 0.2973 | 0.2622
w/o text | 0.3354 | 0.3185 | 0.2948 | 0.2575
w/o image | 0.3918 | 0.3323 | 0.2784 | 0.2247
w/o text and image | 0.3298 | 0.2976 | 0.2783 | 0.2378

###  6.3 Disease Identification Ability

In this section, we further analyze the model’s ability to identify the major
diseases in X-rays and corresponding findings. The evaluation in Section 6.1
is based on a broad comparison of impressions and predictions at the word and
semantic level. Additionally in this section, we use the F1CheXbert to assess
whether the model identifies all the key diseases noted in the gold
impressions in its predictions as well. We report both the micro-average F1
score and the individual F1 scores for five main diseases, as shown in Table
5.

Table 5: Comparison of MID-M’s performance on the identification of major diseases under different levels of data corruption. Abbreviations: CMG, cardiomegaly; Consol, consolidation; Atelect, atelectasis; PE, pleural effusion. | CMG | Edema | Consol | Atelect | PE | Micro Avg.
---|---|---|---|---|---|---
Full | 0.4200 | 0.5714 | 0.4444 | 0.4916 | 0.6335 | 0.5302
Corrupted 0.1 | 0.3204 | 0.4923 | 0.1538 | 0.4067 | 0.5543 | 0.4379
Corrupted 0.3 | 0.3010 | 0.3884 | 0.1176 | 0.3972 | 0.4250 | 0.3724
Corrupted 0.5 | 0.2631 | 0.3606 | 0.1538 | 0.2704 | 0.4467 | 0.3327
Support | 87 | 148 | 25 | 151 | 138 | 549

We found that the model’s performance declines with increased data corruption,
but the rate of decline varies across diseases. For instance, the model’s
ability to identify pleural effusion (PE) remains relatively robust against
corruption, even under conditions of high data corruption, compared to other
diseases. In contrast, our model experiences a significant drop in
consolidation (Consol) even at the lowest level of corruption (0.1). This
suggests that the model manages to recognize some diseases in a manner less
susceptible to corruption. This might be attributed to the feasibility of
retrieving similar samples. When there are many samples with a certain disease
in the training set, it becomes easier to retrieve relevant samples for the
diseases and vice versa. In addition, a significant drop in consolidation
might be attributed to a unique linguistic challenge. When some of the
subwords from consolidation are obscured, the remaining segments (such as
’solid’ or ’-tion’) could be mistakenly associated with other terms frequently
used in radiology reports, complicating its identification.

###  6.4 In-Context Learning Ability

In this section, we evaluate the model’s in-context learning capabilities by
varying the number of examples (shots) included in the prompt. Previous
experiments, as discussed in Section 6.1, uniformly used a two-shot setting.
In addition to this, we explore the model’s abilities in zero-shot and one-
shot settings. The results are presented in Table 6.

Table 6: Model performances in different number of shots in the input prompt.  Method | Full | Corrupted 0.1 | Corrupted 0.3 | Corrupted 0.5
---|---|---|---|---
ROUGE
0-shot | 0.1673 | 0.1270 | 0.1127 | 0.1091
1-shot | 0.3818 | 0.3438 | 0.3172 | 0.2180
OURS (2-shot) | 0.4064 | 0.3719 | 0.3438 | 0.2977
F1RadGraph
0-shot | 0.1573 | 0.0994 | 0.072 | 0.0439
1-shot | 0.3536 | 0.3091 | 0.2797 | 0.2386
OURS (2-shot) | 0.3732 | 0.3379 | 0.2973 | 0.2622

We observe a decline in performance in both zero-shot and one-shot settings
compared to the two-shot scenario. However, the disparity in performance
across these settings is noteworthy. In the zero-shot setting, the model
struggles significantly, achieving a ROUGE score of only about 0.1673 on the
full test set and with the F1RadGraph score approaching zero across all
corrupted test sets. In contrast, introducing just one relevant example
enhances the model’s performance substantially, achieving results that are
comparatively close to those in the two-shot setting in many cases. This
underscores the impressive in-context learning capabilities of pretrained
large models, highlighting their potential for easy adaptation to different
domains without the need for fine-tuning.

##  7 Discussion

In this paper, we introduce MID-M, a novel multimodal framework designed for
the medical domain, which leverages the in-context learning capabilities of a
general-domain LLM. By transforming images into textual descriptions, MID-M
facilitates an interpretable representation of medical images, enabling
complex information to be more accessible to engineers and, potentially, to
practitioners. Notably, MID-M demonstrates superior generalization capability,
outperforming models with substantially more parameters that are extensively
trained for multimodality and medical applications. This emphasizes the
potential of leveraging general-domain models for specialized tasks, offering
a sustainable and cost-effective approach without massive pretraining and
extensive fine-tuning. This approach aligns with the ongoing shift towards
developing powerful AI tools in healthcare. Furthermore, it highlights the
importance of robustness and generalization in systems, particularly in
healthcare settings where data quality can vary significantly. The framework’s
ability to maintain robust performance even with degraded data quality,
presents a compelling case for its application in real-world medical scenarios
even with limited resources. We hope our work can contribute to the
development of globally accessible AI systems.

#### Limitations

Despite the contributions of our research, there are a few limitations to our
work. First, while MID-M demonstrates superior in-context learning capability,
it still underperforms compared to task-specific fine-tuned models. This is
observed from the performance of RadFM on the full test set. However, it is
important to note that models like RadFM demand high computational resources
for their training and show a significant drop in generalization ability when
faced with incomplete data. In future works, we could focus on developing in-
context learning approaches that can achieve performance comparable to that of
task-specific models, even under ideal conditions. The second limitation is
the artificial nature of our masking approach. In real-world applications,
masking is likely to occur at the word level and targets sensitive
information. Our method, however, employed subword-level random masking.
Despite this fundamental difference, we believe our test set includes samples
with realistically plausible masking since sub-word masking cumulatively
approximates word-level masking, and sensitive information can also be
obscured by randomly chosen. Our statistical analysis, which shows main
diseases are masked in proportion to the corruption rate, also supports our
hypothesis. Moreover, this approach allows for the systematic masking with
which we produced four different test sets. We believe the insights derived
from our experiments can provide valuable guidance for future research.

This work was supported by Institute of Information & communications
Technology Planning & Evaluation (IITP) grant funded by the Korea
government(MSIT) (RS-2022-00143911,AI Excellence Global Innovative Leader
Education Program). Yoon JH has received grant support from NIH K23GM138984.

## References

  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ , 2023.
  * Agrawal et al. (2022) Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David Sontag.  Large language models are few-shot clinical information extractors.  _arXiv preprint arXiv:2205.12689_ , 2022.
  * Alayrac et al. (2022) Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al.  Flamingo: a visual language model for few-shot learning.  _Advances in neural information processing systems_ , 35:23716–23736, 2022.
  * Awadalla et al. (2023) Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al.  Openflamingo: An open-source framework for training large autoregressive vision-language models.  _arXiv preprint arXiv:2308.01390_ , 2023.
  * Benitez and Malin (2010) Kathleen Benitez and Bradley Malin.  Evaluating re-identification risks with respect to the hipaa privacy rule.  _Journal of the American Medical Informatics Association_ , 17(2):169–177, 2010.
  * Berrios et al. (2023) William Berrios, Gautam Mittal, Tristan Thrush, Douwe Kiela, and Amanpreet Singh.  Towards language models that can see: Computer vision through the lens of natural language.  _arXiv preprint arXiv:2306.16410_ , 2023.
  * Brady (2017) Adrian P Brady.  Error and discrepancy in radiology: inevitable or avoidable?  _Insights into Imaging_ , 8(1):171–182, 2017.  10.1007/s13244-016-0534-1.
  * Byra et al. (2023) Michal Byra, Muhammad Febrian Rachmadi, and Henrik Skibbe.  Few-shot medical image classification with simple shape and texture text descriptors using vision-language models.  _arXiv preprint arXiv:2308.04005_ , 2023.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.  _See https://vicuna. lmsys. org (accessed 14 April 2023)_ , 2(3):6, 2023.
  * Chong et al. (2023) Chak Fong Chong, Xu Yang, Tenglong Wang, Wei Ke, and Yapeng Wang.  Category-wise fine-tuning for image multi-label classification with partial labels.  In _International Conference on Neural Information Processing_ , pages 332–345. Springer, 2023.
  * Dai et al. (2024) Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi.  Instructblip: Towards general-purpose vision-language models with instruction tuning.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Delbrouck et al. (2022) Jean-Benoit Delbrouck, Pierre Chambon, Christian Bluethgen, Emily Tsai, Omar Almusa, and Curtis P Langlotz.  Improving the factual correctness of radiology report generation with semantic rewards.  _arXiv preprint arXiv:2210.12186_ , 2022.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Driess et al. (2023) Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al.  Palm-e: An embodied multimodal language model.  _arXiv preprint arXiv:2303.03378_ , 2023.
  * Feng et al. (2020) James E Feng, Afshin A Anoushiravani, Paul J Tesoriero, et al.  Transcription error rates in retrospective chart reviews.  _Orthopedics_ , 43(5):e404–e408, 2020.
  * Gu et al. (2021) Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon.  Domain-specific language model pretraining for biomedical natural language processing.  _ACM Transactions on Computing for Healthcare (HEALTH)_ , 3(1):1–23, 2021.
  * Guo et al. (2022) Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven CH Hoi.  From images to textual prompts: Zero-shot vqa with frozen large language models.  _arXiv preprint arXiv:2212.10846_ , 2022.
  * Houlsby et al. (2019) Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.  Parameter-efficient transfer learning for nlp.  In _International conference on machine learning_ , pages 2790–2799. PMLR, 2019.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_ , 2021.
  * Hu et al. (2022) Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A Smith, and Jiebo Luo.  Promptcap: Prompt-guided task-aware image captioning.  _arXiv preprint arXiv:2211.09699_ , 2022.
  * Jain et al. (2021) Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven QH Truong, Du Nguyen Duong, Tan Bui, Pierre Chambon, Yuhao Zhang, Matthew P Lungren, Andrew Y Ng, et al.  Radgraph: Extracting clinical entities and relations from radiology reports.  _arXiv preprint arXiv:2106.14463_ , 2021.
  * Johnson et al. (2019) Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng.  Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports.  _Scientific data_ , 6(1):317, 2019.
  * Kim et al. (2023) Seungone Kim, Se June Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo.  The cot collection: Improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning.  _arXiv preprint arXiv:2305.14045_ , 2023.
  * Laurençon et al. (2024) Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al.  Obelics: An open web-scale filtered dataset of interleaved image-text documents.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Lewis et al. (2020) Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov.  Pretrained language models for biomedical and clinical tasks: Understanding and extending the state-of-the-art.  In Anna Rumshisky, Kirk Roberts, Steven Bethard, and Tristan Naumann, editors, _Proceedings of the 3rd Clinical Natural Language Processing Workshop_ , pages 146–157, Online, November 2020. Association for Computational Linguistics.  10.18653/v1/2020.clinicalnlp-1.17.  URL https://aclanthology.org/2020.clinicalnlp-1.17.
  * Li et al. (2023a) Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu.  Mimic-it: Multi-modal in-context instruction tuning.  _arXiv preprint arXiv:2306.05425_ , 2023a.
  * Li et al. (2024) Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao.  Llava-med: Training a large language-and-vision assistant for biomedicine in one day.  _Advances in Neural Information Processing Systems_ , 36, 2024.
  * Li et al. (2023b) Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  In _International conference on machine learning_ , pages 19730–19742. PMLR, 2023b.
  * Liévin et al. (2023) Valentin Liévin, Christoffer Egeberg Hother, Andreas Geert Motzfeldt, and Ole Winther.  Can large language models reason about medical questions?  _Patterns_ , 2023.
  * Lin (2004) Chin-Yew Lin.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , pages 74–81, 2004.
  * Liu et al. (2024) Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee.  Visual instruction tuning.  _Advances in neural information processing systems_ , 36, 2024.
  * Liu et al. (2023) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _ACM Computing Surveys_ , 55(9):1–35, 2023.
  * Menon and Vondrick (2022) Sachit Menon and Carl Vondrick.  Visual classification via description from large language models.  _arXiv preprint arXiv:2210.07183_ , 2022.
  * Moor et al. (2023) Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar.  Med-flamingo: a multimodal medical few-shot learner.  In _Machine Learning for Health (ML4H)_ , pages 353–367. PMLR, 2023.
  * Nori et al. (2023) Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, et al.  Can generalist foundation models outcompete special-purpose tuning? case study in medicine.  _arXiv preprint arXiv:2311.16452_ , 2023.
  * Robinson et al. (1999) PJ Robinson, D Wilson, A Coral, A Murphy, and P Verow.  Variation between experienced observers in the interpretation of accident and emergency radiographs.  _British Journal of Radiology_ , 72(856):323–330, Apr 1999.  10.1259/bjr.72.856.10474490.
  * Smit et al. (2020) Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, Andrew Y Ng, and Matthew P Lungren.  Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert.  _arXiv preprint arXiv:2004.09167_ , 2020.
  * Sun et al. (2023) Yuxuan Sun, Chenglu Zhu, Sunyi Zheng, Kai Zhang, Zhongyi Shui, Xiaoxuan Yu, Yizhi Zhao, Honglin Li, Yunlong Zhang, Ruojia Zhao, et al.  Pathasst: Redefining pathology through generative foundation ai assistant for pathology.  _arXiv preprint arXiv:2305.15072_ , 2023.
  * Surís et al. (2023) Dídac Surís, Sachit Menon, and Carl Vondrick.  Vipergpt: Visual inference via python execution for reasoning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 11888–11898, 2023.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023.
  * Tu et al. (2024) Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaekermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Charles Lau, Ryutaro Tanno, Ira Ktena, et al.  Towards generalist biomedical ai.  _NEJM AI_ , 1(3):AIoa2300138, 2024.
  * Van Veen et al. (2023a) Dave Van Veen, Cara Van Uden, Maayane Attias, Anuj Pareek, Christian Bluethgen, Malgorzata Polacin, Wah Chiu, Jean-Benoit Delbrouck, Juan Manuel Zambrano Chaves, Curtis P Langlotz, et al.  Radadapt: Radiology report summarization via lightweight domain adaptation of large language models.  _arXiv preprint arXiv:2305.01146_ , 2023a.
  * Van Veen et al. (2023b) Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean-Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pareek, Malgorzata Polacin, Eduardo Pontes Reis, Anna Seehofnerova, et al.  Clinical text summarization: Adapting large language models can outperform human experts.  _Research Square_ , 2023b.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in neural information processing systems_ , 35:24824–24837, 2022.
  * Whiting et al. (2015) Penny F Whiting, Clare Davenport, Catherine Jameson, Margaret Burke, Jonathan A C Sterne, Chris Hyde, and Yoav Ben-Shlomo.  How well do health professionals interpret diagnostic information? a systematic review.  _BMJ Open_ , 5(7), 2015.  ISSN 2044-6055.  10.1136/bmjopen-2015-008155.  URL https://bmjopen.bmj.com/content/5/7/e008155.
  * (46) C Wu, X Zhang, Y Zhang, Y Wang, and W Xie.  Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data. arxiv [preprint]. 2023 [accessed on october 2, 2023].
  * Wu et al. (2023) Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie.  Towards generalist foundation model for radiology.  _arXiv preprint arXiv:2308.02463_ , 2023.
  * Xie et al. (2022) Yujia Xie, Luowei Zhou, Xiyang Dai, Lu Yuan, Nguyen Bach, Ce Liu, and Michael Zeng.  Visual clues: Bridging vision and language foundations for image paragraph captioning.  _Advances in Neural Information Processing Systems_ , 35:17287–17300, 2022.
  * Xu et al. (2024) Ruyi Xu, Yuan Yao, Zonghao Guo, Junbo Cui, Zanlin Ni, Chunjiang Ge, Tat-Seng Chua, Zhiyuan Liu, Maosong Sun, and Gao Huang.  Llava-uhd: an lmm perceiving any aspect ratio and high-resolution images.  _arXiv preprint arXiv:2403.11703_ , 2024.
  * Yan et al. (2023) An Yan, Yu Wang, Yiwu Zhong, Zexue He, Petros Karypis, Zihan Wang, Chengyu Dong, Amilcare Gentili, Chun-Nan Hsu, Jingbo Shang, et al.  Robust and interpretable medical image classifiers via concept bottleneck models.  _arXiv preprint arXiv:2310.03182_ , 2023.
  * Yang et al. (2022) Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, and Lijuan Wang.  An empirical study of gpt-3 for few-shot knowledge-based vqa.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, pages 3081–3089, 2022.
  * Zhang et al. (2023a) Kai Zhang, Jun Yu, Zhiling Yan, Yixin Liu, Eashan Adhikarla, Sunyang Fu, Xun Chen, Chen Chen, Yuyin Zhou, Xiang Li, et al.  Biomedgpt: A unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks.  _arXiv preprint arXiv:2305.17100_ , 2023a.
  * Zhang et al. (2020) Yuhao Zhang, Derek Merck, Emily Tsai, Christopher D. Manning, and Curtis Langlotz.  Optimizing the factual correctness of a summary: A study of summarizing radiology reports.  In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5108–5120, Online, July 2020. Association for Computational Linguistics.  10.18653/v1/2020.acl-main.458.  URL https://aclanthology.org/2020.acl-main.458.
  * Zhang et al. (2023b) Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola.  Multimodal chain-of-thought reasoning in language models.  _arXiv preprint arXiv:2302.00923_ , 2023b.

## Appendix A. Example of Prompt

Our prompt is composed of the role assignment, task description, two retrieved
examples, and the test sample. Here we provide one example of the prompt that
we used for our framework.

⬇

You are an expert medical professional. Write a concise summary of the
following chest X-ray report. The text description of the X-ray images and the
full report, named "Finding" will be provided. Focus on the key findings and
diagnoses noted primarily in the image description, while also incorporating
relevant details from the full report. The summary, named "Impression", should
be concise and presented in correct English sentences.

Image description: There is Atelectasis in the image in 24.20 probability.

There is Cardiomegaly in the image in 1.82 probability.

There is Consolidation in the image in 3.99 probability.

There is Edema in the image in 0.80 probability.

There is Enlarged Cardiomediastinum in the image in 8.94 probability.

There is Fracture in the image in 10.79 probability.

There is Lung Lesion in the image in 7.47 probability.

There is Lung Opacity in the image in 18.30 probability.

There is No Finding in the image in 31.68 probability.

There is Pleural Effusion in the image in 3.73 probability.

There is Pleural Other in the image in 1.05 probability.

There is Pneumonia in the image in 4.88 probability.

There is Pneumothorax in the image in 6.48 probability.

There is Support Devices in the image in 10.13 probability.

Finding: The heart size is normal. The hilar and mediastinal contours are
unremarkable. The lungs are well expanded and clear. There is no evidence of a
pneumothorax or pleural effusion. The visualized osseous structures are
unremarkable.

Impression: No acute cardiopulmonary processes. Specifically, no evidence of
an infiltrative process suggestive of pneumonia.

Image description: There is Atelectasis in the image in 2.38 probability.

There is Cardiomegaly in the image in 0.39 probability.

There is Consolidation in the image in 0.76 probability.

There is Edema in the image in 0.10 probability.

There is Enlarged Cardiomediastinum in the image in 3.32 probability.

There is Fracture in the image in 7.94 probability.

There is Lung Lesion in the image in 1.50 probability.

There is Lung Opacity in the image in 4.21 probability.

There is No Finding in the image in 78.47 probability.

There is Pleural Effusion in the image in 0.62 probability.

There is Pleural Other in the image in 0.93 probability.

There is Pneumonia in the image in 0.78 probability.

There is Pneumothorax in the image in 1.96 probability.

There is Support Devices in the image in 9.13 probability.

Finding: The heart size is normal. The hilar and mediastinal contours are
unremarkable. The lungs are well expanded and clear. There is no evidence of a
pneumothorax or pleural effusion. The visualized osseous structures are
unremarkable.

Impression: Normal chest x-ray. Specifically, no pulmonary evidence of TB.

Image description: There is Atelectasis in the image in 4.65 probability.

There is Cardiomegaly in the image in 0.89 probability.

There is Consolidation in the image in 1.28 probability.

There is Edema in the image in 0.21 probability.

There is Enlarged Cardiomediastinum in the image in 4.43 probability.

There is Fracture in the image in 14.87 probability.

There is Lung Lesion in the image in 4.14 probability.

There is Lung Opacity in the image in 5.80 probability.

There is No Finding in the image in 45.93 probability.

There is Pleural Effusion in the image in 3.02 probability.

There is Pleural Other in the image in 3.44 probability.

There is Pneumonia in the image in 1.32 probability.

There is Pneumothorax in the image in 2.53 probability.

There is Support Devices in the image in 5.05 probability.

Finding: The heart size is normal. The hilar and mediastinal contours are
unremarkable. The lungs are slightly hyperinflated, however appear to be
clear. There is no evidence of pneumothorax or pleural effusions. The
visualized osseous structures are unremarkable.

Impression: