# Bound on efficiency of heat engine from uncertainty relation viewpoint

Pritam Chattopadhyay<EMAIL_ADDRESS>Cryptology and Security Research
Unit, R.C. Bose Center for Cryptology and Security,
Indian Statistical Institute, Kolkata 700108, India  Ayan Mitra
<EMAIL_ADDRESS>Department of Mechanical and Aerospace Engineering,
School of Engineering and Digital Sciences, Nazarbayev University,
Kabanbay Batyr Ave. 53, Nur-Sultan 010000, Kazakhstan  Goutam Paul
<EMAIL_ADDRESS>Cryptology and Security Research Unit, R.C. Bose
Center for Cryptology and Security,
Indian Statistical Institute, Kolkata 700108, India  Vasilios Zarikas
<EMAIL_ADDRESS>Department of Mechanical and Aerospace
Engineering, School of Engineering and Digital Sciences, Nazarbayev
University,
Kabanbay Batyr Ave. 53, Nur-Sultan 010000, Kazakhstan

###### Abstract

Quantum cycles in established heat engines can be modeled with various quantum
systems as working substances. For example, a heat engine can be modeled with
an infinite potential well as the working substance to determine the
efficiency and work done. However, in this method, the relationship between
the quantum observables and the physically measurable parameters—i.e., the
efficiency and work done—is not well understood from the quantum mechanics
approach. A detailed analysis is needed to link the thermodynamic variables
(on which the efficiency and work done depends) with the uncertainty principle
for better understanding. Here, we present the connection of the sum
uncertainty relation of position and momentum operators with thermodynamic
variables in the quantum heat engine model. We are able to determine the upper
and lower bounds on the efficiency of the heat engine through the uncertainty
relation.

##  I Introduction

Thermodynamics is a prominent theory in evaluating the performance of the
engines. It stands as a pillar of theoretical physics and even contributes to
our understanding of modern theories, e.g., black hole entropy and temperature
hawk , gravity thanu ; gurs . Though it is successful in the classical regime,
the application of thermodynamics needs to be reinvestigated in a quantum
system, as the energy is discrete instead of continuous. So, we can expect new
thermodynamic effects to come up in the quantum world. However, while dealing
with thermodynamic laws in a quantum regime, a striking similarity between the
quantum–thermodynamic system and the macroscopic system (which are described
by classical thermodynamics) can be found. For example, in the case of thermal
baths, the Carnot efficiency of the engines is equally relevant for the
quantum system (even with a single particle) ra . This raises a question: can
all the thermodynamic effects of heat engines of small quantum systems be
analyzed and predicted by the known classical thermodynamics? Various works
have been performed on the analysis of generic thermodynamic effects and
dynamical behavior, which are purely quantum in their nature, with no
classical counterpart involved uzd .

Quantum thermodynamics explores thermodynamic quantities like temperature,
entropy, heat, etc. for the microscopic system. It can even analyze the above
parameters for a single particle model. The study of quantum thermodynamics
comprises of the analysis of quantum thermal machines in the microscopic
regime kos ; rezek ; rkos1 ; skr ; mkol ; htq ; jro ; lac ; rdo  and also in
the thermalization mechanism arie . All of the various methods specified so
far do not exploit quantum effects in thermodynamics, i.e., there exists some
classical analog in these methods.

A framework for quantum heat engine realization and its experimental setup has
been proposed adec ; oab ; kzh ; ama ; wh . Heat engines can be discrete or
continuous in nature. Two-stroke and four-stroke engines belong to the
discrete group whereas turbines belong to the continuous engine. The Szilard
engine was a seminal work lsz  to solve the violation of the 2nd law of
thermodynamics by Maxwell’s demon. The quantum version of this engine was
proposed by Kim et al. kim . This is an example of the quantum version of a
four-stroke engine. During the insertion and deletion of the barrier in the
quantum Szilard engine, a certain amount of work and heat exchange occurs in
the system that does not occur in the classical system. Different models and
methods to explain the working principle of the Szilard engine have been
explored in various works li ; kh ; cy ; lia ; jb . Various models and working
mediums for the analysis of thermal machines has been recently explored in
various works rev1 ; rev2 ; rev3 ; rev4 ; rev5 ; rev6 ; rev6a ; rev7 ; rev8 ;
rev9 ; rev10 ; rev11 .

In this work, we study the engines from a more fundamental concept of quantum
mechanics and try to connect the efficiency of engines with the fundamental
uncertainty relation of two incompatible operators. We consider the one-
dimensional potential well as the working substance for the quantum heat
engine. Here, we consider a specific model for our analysis, though it is
applicable globally to all the engines. We develop an effective method to
analyze the useful work using the uncertainty relation of the position and the
momentum of a particle in a box without performing any measurement, but by
applying two reservoirs of different temperatures. The thorough analysis
performed in this work is done in the nonrelativistic limit. A parallel
analysis in the relativistic limit is analyzed in our work pc23 . During the
evolution of quantum information, the essence and importance of uncertainty
relation in technology got enriched. It has various applications in quantum
technology like quantum cryptography koas ; koas1 ; caf , entanglement
detection hofm ; ost ; mart ; guh , and even in quantum metrology giov  and
quantum speed limit deba ; marv ; deff ; pires . In recent times, the work
xiao ; maw ; baek  authenticated the uncertainty relation experimentally. The
thermal uncertainty relation that we will be applying here is a special case
of the general quantum uncertainty relations. The uncertainty relation of two
incompatible observables is given by

${\Delta}a{\Delta}b{\sim}\frac{\hbar}{2},$ (1)

where $a$ and $b$ are any two canonical variable pairs. No better lower bound
was known to us until it was explored in the work mondal . They have not only
given a better lower bound than the previously known Pati–Maccone uncertainty
relation (PMUR), but also developed an upper bound for the uncertainty
relation. It is popularly known as the reverse uncertainty relation. Using
this principle, we will similarly develop the bound of efficiency and work of
the heat engine in terms of uncertainty relation.

We have organized the paper as follows. In Section II, we develop the thermal
uncertainty relation for a one-dimensional potential well of length $2L$. In
Section III, the bound on the sum of variance from the thermal standpoint as
well as the traditional one is established. We have devoted Section IV to
develop the correlation between the thermodynamic variables and the sum of
variance of the position and momentum operator of one-dimensional potential.
Section V is dedicated to discussing the Stirling cycle and establishing the
work done and efficiency in terms of the thermal uncertainty relation. In this
section, we illustrate the bound on the work done and efficiency of the
quantum engine. The paper is concluded in Section VI with some discussions and
future prospects of this work in the field of quantum thermodynamics.

##  II Thermal Uncertainty Relation

In the first phase of our analysis, we evaluate the thermal uncertainty
relation (which is one of the special cases of the general uncertainty
relation) for a particle in one-dimensional potential well. To do so, let us
first revisit our textbook problem of the one-dimensional potential well. The
one-dimensional potential well is a well-known problem in quantum mechanics
schiff ; dj . Here, we consider a particle of mass $m$ inside a one-
dimensional potential box of length $2L$. The wave function of this system for
the $n$-th level is

$|\psi_{n}\rangle=\sqrt{\frac{1}{L}}sin(\frac{n\pi x}{2L}).$ (2)

So, when the wave function of the model under study is known, we can calculate
the eigenvalue of the system. Eigenvalues of the 1-D potential well are

$E_{n}=\frac{n^{2}\pi^{2}\hslash^{2}}{2m(2L)^{2}},$ (3)

where $\hslash$ is Planck’s constant.

With the wave function of the model in hand, we are all set to derive the
uncertainty relation of the position and the momentum for this system. The
uncertainty relation for our model is described as schiff ; dj

$\displaystyle\Delta x\Delta p$ $\displaystyle=$
$\displaystyle\frac{\hbar}{2}\sqrt{\Big{(}\frac{(n\pi)^{2}}{3}-2\Big{)}}\quad\geq\frac{\hbar}{2},$
(4)

where $\Delta x^{2}=\langle x^{2}\rangle-\langle x\rangle^{2}$ and $\Delta
p^{2}=\langle p^{2}\rangle-\langle p\rangle^{2}$ and we have $\langle
p\rangle=0$ for all eigenstates. The expectation values of $\langle x\rangle$,
$\langle x^{2}\rangle$ and $\langle p^{2}\rangle$ can be easily evaluated by
considering the defined wave function of the considered system.

We formulate the uncertainty relation of the system at a certain temperature
$T$ from a thermodynamics viewpoint. The formulation of the thermal
uncertainty relation is performed by the analysis of the partition function of
the system. The partition function reif1 , $Z$, for the 1-D potential well is
expressed as

$Z=\sum_{n=1}^{\infty}e^{-\beta
E_{n}}\approx\frac{1}{2}\sqrt{\frac{\pi}{\beta\alpha}}\,,$ (5)

where $\beta=\frac{1}{k_{B}T}$, $k_{B}$ is Boltzmann’s constant and
$\alpha=\frac{\pi^{2}\hbar^{2}}{2m(2L)^{2}}$. The expression of $Z$ converges
to the form mentioned, as the product of $\beta$ and $\alpha$ is a small
quantity. We use the Gaussian integral as the approximation considering that
the error in the integration is negligible as the product of $\beta$ and
$\alpha$ is a small quantity. The mean energy for this system evolves to
$\langle E\rangle=-\partial lnZ\big{/}\partial\beta=\frac{1}{2\beta}$. The
average of the quantum number for the system under study can be conveyed as

$\bar{n}=\frac{\sum_{n}ne^{-\beta E_{n}}}{\sum_{n}e^{-\beta
E_{n}}}\approx\frac{1}{\sqrt{\pi\alpha\beta}}\,.$ (6)

Having the mathematical form of the partition function in hand, we have all
the resources to develop the thermal uncertainty relation. Now, we focus on
the development of the dispersion relation of the position and the momentum
operator at a certain temperature. The dispersion in position can be expressed
as

$\displaystyle(\Delta X)^{2}_{T}$ $\displaystyle=$
$\displaystyle\langle(\Delta X)^{2}\rangle_{T}=\langle
X^{2}\rangle_{T}-\langle X\rangle^{2}_{T}$ (7) $\displaystyle=$
$\displaystyle\frac{L^{2}}{3}-\frac{2L^{2}}{\pi^{2}}\times\frac{e^{-\alpha\beta}-\sqrt{\pi\alpha\beta}\times
erfc(\sqrt{\alpha\beta})}{\frac{1}{2}\sqrt{\frac{\pi}{\alpha\beta}}}$
$\displaystyle=$
$\displaystyle\frac{L^{2}}{3}-\frac{4L^{2}\sqrt{\alpha\beta}}{\pi^{5/2}}\times(e^{-\alpha\beta}-\sqrt{\pi\alpha\beta}),$

$erfc$ is the complementary error function, which appears while solving
$\langle x^{2}\rangle$. The dispersion relation of the momentum operator can
be analyzed similarly. It is expressed as

$\displaystyle(\Delta P)^{2}_{T}$ $\displaystyle=$
$\displaystyle\langle(\Delta P)^{2}\rangle_{T}=\langle
P^{2}\rangle_{T}-\langle P\rangle^{2}_{T}$ (8) $\displaystyle=$
$\displaystyle\frac{\pi^{3}\hbar^{2}\bar{n}^{2}}{8L^{2}}.$

So, the thermal uncertainty relation for the system at temperature $T$ can be
evaluated from Equations (7) and (8) as

$\displaystyle(\Delta X)_{T}(\Delta P)_{T}$ $\displaystyle=$
$\displaystyle\frac{\hbar\bar{n}\pi^{3/2}}{2\sqrt{2}}{\Big{[}\frac{1}{3}-\frac{4}{\bar{n}\pi^{3}}(e^{-\frac{1}{\pi\bar{n}^{2}}}-\frac{1}{\bar{n}})\Big{]}}^{\frac{1}{2}}$
(9) $\displaystyle\geq$ $\displaystyle\frac{\hbar}{2}.$

The product uncertainty relation loses its importance when the system under
consideration is an eigenstate of the observable under study. The sum of
uncertainty macc  was introduced to capture uncertainty in the observables
when the system happens to be an eigenstate of the observables. The sum of
uncertainty for this system at a particular temperature $T$ is expressed as

$\displaystyle(\Delta X)_{T}+(\Delta P)_{T}$ $\displaystyle=$ $\displaystyle
L{\Big{[}\frac{1}{3}-\frac{4\sqrt{\alpha\beta}}{\pi^{5/2}}(e^{-\alpha\beta}-\sqrt{\pi\alpha\beta})\Big{]}}^{\frac{1}{2}}$
(10) $\displaystyle+$ $\displaystyle\frac{\hbar\bar{n}\pi^{3/2}}{2\sqrt{2}L}.$

The parameters that are considered for the analysis are expressed in table :

Parameter | Values
---|---
$\bar{n}$ | 1, 2
Length ($L$) | 0-5 nm
Hot bath ($T_{1}$) | 320K
Cold bath ($T_{2}$) | 80K

In Figure 1, the variation of sum uncertainty relation (Equation (10)) with
respect to different temperatures is shown. The thermal uncertainty relation
that we have developed (Equation (10)) for the considered system encounters a
negligible amount of variation when the length of the potential well is small,
whereas the difference is large for higher values of $L$ (length is considered
in nanometers).

Figure 1: The variation of sum uncertainty relation (Equation (10)) for
different temperatures. The dotted line is for lower and the solid line is for
higher temperature.

The variation of uncertainty relation for different levels is shown in Figure
2. Similar to the case of temperature analysis, the variation is negligible
for lower values of $L$ and is large for higher values.

Figure 2: Similarly, this shows variation for different values of $n$.

##  III Bound on Sum Uncertainty for the One-Dimension Potential Well

The product of variances is sometimes unable to capture the uncertainty for
two incompatible observables. If the state of the system is an eigenstate of
one of the observables, then the product of the uncertainties vanishes pc ;
mondal . To overcome this, the sum of variances is introduced to capture the
uncertainty of two incompatible observables. For any quantum model, the sum of
variance of two incompatible observables that results in the lower bound is
defined as

$\Delta A^{2}+\Delta
B^{2}\geq\frac{1}{2}\sum_{n}\Big{(}\Big{|}\langle\psi_{n}|\bar{A}|\psi\rangle\Big{|}+\Big{|}\langle\psi_{n}|\bar{B}|\psi\rangle\Big{|}\Big{)}^{2}.$
(11)

For our system, we calculate the lower bound of sum uncertainty for the
position and momentum operator. So, we replace $A=X$ and $B=P$, which yields
to

$\Delta X^{2}+\Delta
P^{2}\geq\frac{1}{2}\sum_{n}\Big{(}\Big{|}\langle\psi_{n}|\bar{X}|\psi\rangle\Big{|}+\Big{|}\langle\psi_{n}|\bar{P}|\psi\rangle\Big{|}\Big{)}^{2}.$
(12)

The computation of the reverse uncertainty relation of two observables results
to the upper bound of uncertainty relation. To develop the upper bound, we
have to utilize the definition of the Dunkl–Williams inequality dn1 . The
mathematical form of this inequality is

$\Delta A+\Delta
B\leq\frac{\sqrt{2}\Delta(A-B)}{\sqrt{1-\frac{Cov(A,B)}{\Delta A.\Delta
B}}}\,.$ (13)

Squaring both sides of the Equation (13), we get the upper bound of the sum of
variance for two variables as

$\Delta A^{2}+\Delta
B^{2}\leq\frac{2\Delta(A-B)^{2}}{1-\frac{\textrm{Cov}(A,B)}{\Delta A\Delta
B}}-2\Delta A\Delta B\,,$ (14)

where $\textrm{Cov}(A,B)$ is defined as
$Cov(A,B)=\frac{1}{2}\langle\\{A,B\\}\rangle-\langle A\rangle\langle
B\rangle,$ and
$\Delta(A-B)^{2}=\langle(A-B)^{2}\rangle-\langle(A-B)\rangle^{2}$.
$\Delta(A-B)^{2}$ is the variance of the difference of the two incompatible
observables.

For our one-dimensional potential well system, which we have considered as a
working substance, we calculate the upper bound of the sum of variance for the
position and momentum operator. So, we have to replace $A=X$ and $B=P$ in
Equation (14) and it results to

$\displaystyle\Delta X^{2}+\Delta P^{2}$ $\displaystyle\leq$
$\displaystyle\frac{2\Delta(X-P)^{2}}{1-\frac{Cov(X,P)}{\Delta X\Delta
P}}-2\Delta X\Delta P$ (15) $\displaystyle\leq$
$\displaystyle\frac{L^{2}}{3}-\frac{2L^{2}}{(n\pi)^{2}}+\frac{\pi^{2}\hbar^{2}n^{2}}{4L^{2}}\,.$

In the above equation, i.e., Equation (15), the upper bound of the system from
the standard method is developed using the definition described in Equation
(14). Now, we develop the sum of variance of the same incompatible observables
from the thermodynamic standpoint. The expression for the sum of variance of
the system at a particular temperature evolves as

$\displaystyle\Delta X^{2}+\Delta P^{2}$ $\displaystyle\leq$
$\displaystyle\frac{4L^{2}}{3}-\frac{8L^{2}\sqrt{\alpha\beta}}{\pi^{5/2}}\times(e^{-\alpha\beta}-\sqrt{\pi\alpha\beta})$
(16) $\displaystyle+$
$\displaystyle\frac{\hbar^{2}\bar{n}^{2}\pi^{3}}{4L^{2}}\,.$

The bounds of sum uncertainty relation (from the thermodynamic perspective
developed using Equation (12) for the lower bound and Equation (16) for the
upper bound for the considered system) with a particular temperature for
different levels are shown in Figure 3. The upper part of the plot is for
$n=1$ and the lower part is for $n=2$. From Figure 3, we can infer that the
effect of the bounds of uncertainty relation are prominent for higher values
of the length of the potential well. The bounds are less prominent for lower
values of $L$.

Figure 3: The bound of uncertainty relation (Equation (12) and (16)) for a
particular temperature for different values of $n$ from a thermodynamic
standpoint.

##  IV Connection of Thermodynamic Quantities with Uncertainty

In the next phase of our analysis, we want to establish a bridge between the
thermodynamic quantities with the uncertainty relation. We consider the sum of
variance to overcome the flaw that will appear if we consider the product form
of uncertainty and if the system is an eigenstate of the observables. We will
first demonstrate a connection of partition function with our uncertainty
relation. The mathematical form of this is given by

$\displaystyle Z$ $\displaystyle=$
$\displaystyle\frac{\pi\bar{n}}{2}=\frac{L\sqrt{2}}{\hbar\sqrt{\pi}}\Big{(}\Delta
X_{T}+\Delta P_{T}+C_{T}\Big{)},$ (17)

where
$C_{T}=-\frac{L}{3}+\frac{2L}{\pi^{5/2}\sqrt{\alpha\beta_{T}}}[\alpha\beta_{T}-\sqrt{\pi}(\alpha\beta_{T})^{3/2}-1]$
is a constant for a specific temperature, which is derived by expanding
Equation 10, and neglecting the higher order terms as the products of $\alpha$
and $\beta$ are small.

Since we are to able to bridge a relationship between the uncertainty relation
and the partition function, we are set to explain all the thermodynamic
variables in terms of uncertainty relations. We develop the Helmholtz free
energy reif1 , $F$, from the uncertainty viewpoint which takes the form of

$\displaystyle F$ $\displaystyle=$ $\displaystyle\frac{-1}{\beta}lnZ$ (18)
$\displaystyle=$
$\displaystyle\frac{-1}{\beta}\,ln\Big{[}\frac{L\sqrt{2}}{\hbar\sqrt{\pi}}\Big{(}\Delta
X_{T}+\Delta P_{T}+C_{T}\Big{)}\Big{]}.$

Entropy is expressed in terms of Helmholtz free energy. So, we uncover the
relationship between the variance of position and momentum with entropy. The
mathematical form for entropy from the uncertainty viewpoint is

$\displaystyle S$ $\displaystyle=$ $\displaystyle-\frac{\partial F}{\partial
T}$ $\displaystyle=$ $\displaystyle
k_{B}\,\,ln\Big{[}\frac{L\sqrt{2}}{\hbar\sqrt{\pi}}\Big{(}\Delta X_{T}+\Delta
P_{T}+C_{T}\Big{)}\Big{]}$ $\displaystyle+$
$\displaystyle\frac{\hbar\sqrt{\pi}k_{B}(\nu+\gamma)}{\sqrt{2}L\beta(\Delta
X_{T}+\Delta P_{T}+C_{T})}\,,$

where
$\nu=\frac{\frac{L^{2}\sqrt{\alpha}}{\sqrt{\beta}\pi^{5/2}}\Big{(}e^{-\alpha\beta}-\sqrt{\pi\alpha\beta}\Big{)}-\frac{2L^{2}\sqrt{\alpha\beta}}{\pi^{5/2}}\Big{(}\alpha
e^{-\alpha\beta}-1/2\sqrt{\frac{\pi\alpha}{\beta}}\Big{)}}{\Big{[}\frac{L^{2}}{3}-\frac{4L^{2}\sqrt{\alpha\beta}}{\pi^{5/2}}\Big{(}e^{-\alpha\beta}-\sqrt{\pi\alpha\beta}\Big{)}\Big{]}^{\frac{1}{2}}}$
and $\gamma$ is expressed as
$\gamma=-\frac{L}{\pi^{5/2}\sqrt{\alpha}\beta^{3/2}}(\alpha\beta-\sqrt{\pi}(\alpha\beta)^{3/2}-1)+\frac{2L}{\pi^{5/2}\sqrt{\alpha\beta}}(\alpha-\sqrt{\pi\beta}\alpha^{3/2})$.

In Figure 4, the variation of entropy in terms of the uncertainty relation is
shown. We can observe an increase in the entropy with an increase in the
uncertainty for different temperatures. With an increase in uncertainty, the
disorder in the system increases, causing an increase in the entropy.

We know that entropy is a measure of entanglement. So, from this relation, we
are able to bridge a connection between the uncertainty relation with
entanglement. So, we can measure the entanglement property of the system from
the uncertainty relation if we are able to model our system with any well-
known quantum systems.

Knowing the Helmholtz free energy saham ; reif1  $F$, for a given system, all
the relevant thermodynamic quantities can be computed from it. Here, the
correlation of $F$ with the uncertainty in the measure of the position and
momentum is established. Hence, this replaces the explicit requirement of
computing the internal energy of the system for deriving the thermodynamic
quantities. In addition, it also raises the question of whether phase
transition (Landau theory and its multimode coupling) can be analyzed from an
uncertainty perspective.

Figure 4: The variation of entropy (Equation (IV)) for different values of
temperature. The scattered plot is for higher temperature and solid line is
for lower temperature.

##  V Stirling Cycle and Bound on Efficiency

A Stirling cycle say ; agar ; hua ; blick  is composed of four stages, two
isothermal processes, and two isochoric processes. During the first stage, we
insert a barrier isothermally in the middle of the well. While this quasi-
static insertion process is being done, the working medium stays at an
equilibrium condition with a hot bath at a temperature $T_{1}$. During the
second stage, we perceive an isochoric heat extraction of the working medium
by connecting it with a bath at a lower temperature of $T_{2}$. In the next
stage of the cycle, there is an isothermal removal of the barrier where we
retain the engine in equilibrium at temperature $T_{2}$. In the final stage,
we bridge the engine to the hot bath at temperature $T_{1}$ and this give rise
to isochoric heat absorption. It is represented pictorially in Figure 5.

Similar to thomas , we calculate the work done and the efficiency but in terms
of uncertainty relation. To determine the work done of the engine, a one-
dimensional well of length $2L$ is considered with a particle of mass $m$ at a
temperature $T_{1}$. The energy of this system is
$E_{n}=\frac{n^{2}\pi^{2}\hslash^{2}}{2m(2L)^{2}}$. The partition function
$Z_{A}$ for the system is $Z\approx\frac{1}{2}\sqrt{\frac{\pi}{\beta\alpha}}$.
Then, a wall is inserted isothermally that converts the potential well into an
infinite double well potential. Due to this insertion of the wall, the energy
level for even values of $n$ remain unchanged but the odd ones shift and
overlap with their nearest neighboring energy level. So, the energy of the
newly formed partitioned one-dimensional potential box is

$E_{2n}=\frac{(2n)^{2}\pi^{2}\hslash^{2}}{2m(2{}{L})^{2}}.$ (20)

So, the new partition function stands as

$\displaystyle Z_{B}=\sum_{n}2e^{-\beta_{1}E_{2n}}.$ (21)

The internal energies for the system are calculated from the partition
function. The internal energy $U_{A}$ and $U_{B}$ is defined as
$U_{i}=-\partial lnZ_{i}\big{/}\partial\beta_{1}$, where $i=A,B$ and
$\beta_{1}=\frac{1}{k_{B}T_{1}}$. This results to

$U_{A}=U_{B}=\frac{1}{2\beta_{1}}.$ (22)

The heat exchanged in this isothermal process can be expressed as

$Q_{AB}=U_{B}-U_{A}+k_{B}T_{1}lnZ_{B}-k_{B}T_{1}lnZ_{A}.$ (23)

Then, the system is connected to a heat bath at a lower temperature $T_{2}$.
The partition function for this lower temperature, where the energy remains
the same, is defined as

$Z_{C}=\sum_{n}2e^{-\beta_{2}E_{2n}}.$ (24)

The heat exchanged for this stage of the cycle is the difference of the
average energies of the initial and the final states, i.e.,

$Q_{CB}=U_{C}-U_{B},$ (25)

where $U_{C}=-\partial lnZ_{C}\big{/}\partial\beta_{2}$ and
$\beta_{2}=\frac{1}{k_{B}T_{2}}$. While the system is connected to the heat
bath at temperature $T_{2}$, we remove the wall isothermally, which we call
stage 3. The energy is now of the form
$E_{n}=\frac{n^{2}\pi^{2}\hslash^{2}}{2m(2L)^{2}}$. The corresponding
partition function is given by

$Z_{D}=\sum_{n}e^{-\beta E_{n}}.$ (26)

We can calculate the internal energy $U_{D}$ similarly as $U_{C}$. The heat
exchanged during this process is given by

$Q_{CD}=U_{D}-U_{C}+k_{B}T_{2}lnZ_{D}-k_{B}T_{2}lnZ_{C}.$ (27)

In the fourth stage of the cycle, the system is connected back to the heat
bath at temperature $T_{1}$. The corresponding energy exchange for this stage
can be expressed as

$Q_{DA}=U_{A}-U_{D}.$ (28)

So, the total work done for the process in terms of variance of the position
and the momentum operator evolves to

$\displaystyle W$ $\displaystyle=$ $\displaystyle Q_{AB}+Q_{BC}+Q_{CD}+Q_{DA}$
(29) $\displaystyle=$
$\displaystyle\frac{8L^{2}\alpha}{\hbar^{2}\pi^{2}}\Big{[}D\,ln\Big{(}\frac{Z_{B}}{Z_{A}}\Big{)}+E\,ln\Big{(}\frac{Z_{D}}{Z_{C}}\Big{)}\Big{]}.$

The efficiency of the Stirling cycle engine stands as

$\displaystyle\eta$ $\displaystyle=$ $\displaystyle
1+\frac{Q_{BC}+Q_{CD}}{Q_{DA}+Q_{AB}}$ (30) $\displaystyle=$
$\displaystyle\frac{\Big{(}\bar{n}_{T_{2}}^{2}\,ln\Big{(}\frac{Z_{D}}{Z_{C}}\Big{)}+\bar{n}_{T_{1}}^{2}\,ln\Big{(}\frac{Z_{B}}{Z_{A}}\Big{)}\Big{)}}{\Big{(}-\bar{n}_{T_{2}}^{2}/2+\bar{n}_{T_{1}}^{2}\Big{(}ln\Big{(}\frac{Z_{B}}{Z_{A}}\Big{)}+1/2\Big{)}\Big{)}}$
$\displaystyle=$
$\displaystyle\frac{\Big{[}D\,ln\Big{(}\frac{Z_{B}}{Z_{A}}\Big{)}+E\,ln\Big{(}\frac{Z_{D}}{Z_{C}}\Big{)}\Big{]}}{\Big{[}-E/2+D\Big{(}\,ln\Big{(}\frac{Z_{B}}{Z_{A}}\Big{)}+1/2\Big{)}\Big{]}}\,,$

where $D=\frac{8L^{2}}{\pi^{3}\hbar^{2}}(\Delta X_{T_{1}}+\Delta
P_{T_{1}}+C_{T_{1}})^{2}$ and $E=\frac{8L^{2}}{\pi^{3}\hbar^{2}}(\Delta
X_{T_{2}}+\Delta P_{T_{2}}+C_{T_{2}})^{2}$.

Figure 5:  The figure shows the four stages (two isothermal and two isochoric
processes) of the Stirling cycle modeled using the potential well.

In Equation (30), the upper and lower bound of the efficiency are evaluated in
terms of the bound that is being analyzed for the thermal uncertainty relation
of the position and the momentum operator. Here, the expression of $D$ and $E$
(for the working model considered for the analysis of the heat engine) gives
the required uncertainty relation for the illustration of the bound of the
efficiency.

In this paper, we are able to bridge a connection between the efficiency of
the heat engine with the variance of the position and the momentum operator.
The upper bound of the efficiency for the heat engine is near-constant when
the uncertainty is high, whereas it dips a little when uncertainty is low. As
shown in Figure 6, the lower bound of the efficiency is high when the
uncertainty in measurement is less and dips gradually with the increase in
uncertainty. Thus, with an increase in uncertainty, we can visualize that the
lower bound of the efficiency decreases. From Figure 6, one can infer that the
lower and the upper bound of the efficiency are near about the same when the
uncertainty in the position and the momentum operator is quite small. The
decrease in the efficiency with the uncertainty is due to the fact that the
conversion ratio of the thermal machine decrease with the increase in the
thermal energy of the system, which has a relation with the uncertainty of the
working medium. The asymptotic behavior for the higher values of the
uncertainty is due to the fact that the conversion rate gets saturated.
Computing the error margins would help us understand this behavior better,
which we intend to do in a future paper.

With the increase in uncertainty, the conversion ratio of the heat engine
decreases as the thermal energy of the system is directly proportional to the
uncertainty of the system. In the case of the upper bound of the efficiency,
which is depicted in terms of the uncertainty relation defined in Equation 22,
the decrease in the efficiency is more prominent due to the presence of the
exponential component, which causes exponential growth in the thermal energy
of the engine and the dissipated heat over the work output.

The Carnot efficiency for the low temperature limit is expressed as
$\Big{(}1-\frac{T_{2}}{T_{1}}\Big{)}$, where $T_{2}$ and $T_{1}$ are the
temperature of the cold and hot bath, respectively. The upper bound of the
efficiency from an uncertainty viewpoint is consistent with the bound given by
Carnot cycle. So, we can infer that the position and the momentum of the
particle has a direct link with the thermodynamic variables. The work nied
suggests that the efficiency of engines that are powered by nonthermal baths
can be higher than the usual convention. This can be testified from an
uncertainty viewpoint.

In the quantum regime, after measurement, the system collapses to one of its
eigenstates. So, to describe and analyze the efficiency of the engine for
different conditions, we must have multiple copies of the system, whereas if
we know the uncertainty relation of the model considered for analysis, we can
describe and analyze different conditions without any measurement being done
on the system. So, multiple copies of the system are not required for our
analysis. This also reduces the measurement cost for analysis of the system.

Figure 6: The bounds on the efficiency by heat engine in terms of the
uncertainty relation. The dotted line represents the upper bound and the solid
line represents the lower bound of the efficiency.

##  VI Discussion

The quantum heat engine has a predominant role in better understanding of the
quantum engines, information, and quantum thermodynamics. This work develops a
relationship between the thermodynamic variables with the position and
momentum of the particle in the system. We give the analytic formulation of
the work and efficiency of the engine in terms of the thermal uncertainty
relation. Though we have considered a specific model for our analysis, this
analysis can cater to a spectrum of global effects, i.e., it can be used to
explain the efficiency of the various engines with different quantum models as
the working substance. Based on these formulations, the physical properties of
the heat engine and the thermodynamic variables that we have encountered are
as follows:

  1. (a)

Every quantum thermodynamic variable has a direct connection with the
uncertainty relation. Helmholtz free energy shows the dependence of the
internal energy of the thermodynamic system with the uncertainty relation of
the incompatible observables. The detailed analysis of entropy with the
uncertainty relation shows that entropy increases when the uncertainty of any
one of the observables increases for a definite temperature. The rate of
increase is different for different temperatures (Figure 4).

  2. (b)

The total work and the efficiency depends on the position and momentum of the
particle. The change in the uncertainty of the position and the momentum has a
direct impact on the efficiency rate and the work of the engine. The lower
bound of the efficiency of the engine drops gradually when the uncertainty of
the observable increases (Figure 6). The upper bound of the efficiency (Figure
6) shows a small variation for higher uncertainty relation, which conveys that
the conversion rate of work input to output is near-constant for higher
uncertainty.

  3. (c)

The uncertainty relation, which is the fundamental principle of quantum
mechanics, is able to predict the efficiency and the total work of the engine
without performing any measurement. So, the measurement cost for the system
gets reduced if we are able to model the system under study with a quantum
model for which we can develop the uncertainty relation.

##  VII Conclusions

The bridge of the uncertainty relation with thermodynamic variables raises a
question of whether we can analyze the phase transition (Landau theory) from
an uncertainty perspective.

Most of the known methods for the measurement of entanglement converge to the
analysis of entropy plen . If we can model the system that is being analyzed
with a quantum model, we can construct the entanglement from the uncertainty
relation of the system. This would be a standard method to measure the
entanglement property of the system, which might be a solution to the open
problem of entanglement measure.

The 1-D problem in the nonrelativistic case is a standard problem. However, in
the relativistic case it is not a standard problem. The study of the heat
engine with a relativistic particle can be analyzed. The mapping of the
entropy with uncertainty to explain the entanglement property for the
relativistic system albe  is an open area to explore. Even the holographic
interpretation of entanglement entropy of anti-de Sitter (ADS)/conformal field
theory (CFT) ryu  can be mapped with uncertainty relation.

This work can be extended to the development of quantum engines in deformed
space structures kempf ; ques ; melj  through the correlation of the
generalized uncertainty relation and thermodynamic variables. In the paper
rezek , the authors mentioned the noncommutativity of kinetic and potential
energy of the quantum harmonic heat engine. Therefore, the possibility of a
connection between the deformed space structures pc  and the heat engines can
be further explored in the future. One can even study the anharmonic
oscillator models through the uncertainty standpoint.

The study of other thermodynamic cycles and procuring the bound for different
thermodynamic parameters is a wide open area to explore. As entropy can be
mapped with the uncertainty relation, this leaves us with questions for future
study as to whether all thermodynamic analyses can be mapped with the
uncertainty of the observables.

## References

  * (1) Hawking, S.W. Black holes and thermodynamics. _Phys. Rev. D_ 1976, _13_ , 191.
  * (2) Padmanabhan, T. Gravity and the thermodynamics of horizons. _Phys. Rep._ 2005, _406_ , 49–125.
  * (3) Gürsoy,U.; Kiritsis, E.; Mazzanti, L.; Nitti, F. Holography and thermodynamics of 5D dilaton-gravity. _J. High Energy Phys._ 2009, _2009_ , 033.
  * (4) Alicki, R. The Quantum Open System as a Model of the Heat Engine. _J. Phys. A_ 1979, _12_ , L103.
  * (5) Uzdin, R.; Levy, A.; Kosloff, R. Equivalence of quantum heat machines, and quantum-thermodynamic signatures. _Phys. Rev. X_ 2015, _5_ , 031044.
  * (6) Kosloff, R. A Quantum Mechanical Open System as a Model of a Heat Engine. _J. Chem. Phys._ 1984, _80_ , 1625.
  * (7) Kosloff, R.; Levy, A. Quantum Heat Engines and Refrigerators: Continuous Devices. _Annu. Rev. Phys. Chem._ 2014, _65_ , 365.
  * (8) Skrzypczyk, P.; Short, A.J.; Popescu, S. Work Extraction and Thermodynamics for Individual Quantum Systems. _Nat. Commun._ 2014, 5, 4185.
  * (9) Kolář, M.; Gelbwaser-Klimovsky, D.; Alicki, R.; Kurizki, G. Quantum Bath Refrigeration Towards Absolute Zero: Challenging the Unattainability Principle. _Phys. Rev. Lett._ 2012, _109_ , 090601.
  * (10) Quan, H.T.; Liu, Y.X.; Sun, C.P.; Nori, F. Quantum Thermodynamic Cycles and Quantum Heat Engines. _Phys. Rev. E_ 2007, _76_ , 031105.
  * (11) Roßnagel, J.; Abah, O.; Schmidt-Kaler, F.; Singer, K.; Lutz, E. Nanoscale Heat Engine Beyond the Carnot Limit. _Phys. Rev. Lett._ 2014, _112_ , 030602.
  * (12) Correa, L.A.; Palao, J.P.; Alonso, D.; Adesso, G. Quantum-Enhanced Absorption Refrigerators. _Sci. Rep._ 2014, _4_ , 3949.
  * (13) Dorner, R.; Clark, S.R.; Heaney, L.; Fazio, R.; Goold, J.; Vedral, V. Extracting Quantum Work Statistics and Fluctuation Theorems by Single-Qubit Interferometry. _Phys. Rev. Lett._ 2013, _110_ , 230601.
  * (14) Riera, A.; Gogolin, C.; Eisert, J. Thermalization in Nature and on a Quantum Computer. _Phys. Rev. Lett._ 2012, _108_ , 080402.
  * (15) Abah, O.; Roßnagel, J.; Jacob, G.; Deffner, S.; Schmidt Kaler, F.; Singer, K.; Lutz, E. Single-Ion Heat Engine at Maximum Power. _Phys. Rev. Lett._ 2012, _109_ , 203006.
  * (16) Dechant, A.; Kiesel, N.; Lutz, E. All-Optical Nano-mechanical Heat Engine. _Phys. Rev. Lett._ 2015, _114_ , 183602.
  * (17) Zhang, K.; Bariani, F.; Meystre, P. Quantum Opto-mechanical Heat Engine. _Phys. Rev. Lett._ 2014, _112_ , 150602.
  * (18) Mari, A.; Eisert, J. Cooling by Heating: Very Hot Thermal Light Can Significantly Cool Quantum Systems. _Phys. Rev. Lett._ 2012, _108_ , 120602.
  * (19) Zurek, W.H. Maxwell’s demon, Szilard’s engine and quantum measurements. _arXiv_ 2003, arXiv:quant-ph/0301076.
  * (20) Szilard, L. Über die Entropieverminderung in einem thermodynamischen System bei Eingriffen intelligenter Wesen. _Zeitschrift für Physik_ 1929, _53_ , 840–856.
  * (21) Kim, S.W.; Sagawa, T.; De Liberato, S.; Ueda, M. Quantum Szilard engine. _Phys. Rev. Lett._ 2011, _106_ , 070401.
  * (22) Li, H.; Zou, J.; Li, J.-G.; Shao, B.; Wu, L.-A. Revisiting the quantum Szilard engine with fully quantum considerations. _Ann. Phys._ 2012, _327_ , 2955.
  * (23) Kim, K.H.; Kim, S.W. Szilard’s information heat engines in the deep quantum regime. _J. Korean Phys. Soc._ 2012, _61_ , 1187.
  * (24) Cai, C.Y.; Dong, H.; Sun, C.P. Multiparticle quantum Szilard engine with optimal cycles assisted by a Maxwell’s demon. _Phys. Rev. E_ 2012, _85_ , 031114.
  * (25) Zhuang, Z.; Liang, S.D. Quantum Szilard engines with arbitrary spin. _Phys. Rev. E_ 2014, _90_ , 052117.
  * (26) Bengtsson, J.; Nilsson Tengstrand, M.; Wacker, A.; Samuelsson, P.; Ueda, M.; Linke, H.; Reimann, S.M. Supremacy of the quantum many-body Szilard engine with attractive bosons. _arXiv_ 2017, arXiv:1701.08138.
  * (27) Kosloff, R.; Rezek, Y. The quantum harmonic Otto cycle. _Entropy_ 2017, _19_ , 136.
  * (28) Deffner, S. Efficiency of harmonic quantum Otto engines at maximal power. _Entropy_ 2018, _20_ , 875.
  * (29) Bustos-Marún, R.A.; Calvo, H.L. Thermodynamics and steady state of quantum motors and pumps far from equilibrium. _Entropy_ 2019, _21_ , 824.
  * (30) Meng, Z.; Chen, L.; Wu, F. Optimal power and efficiency of multi-stage endoreversible quantum Carnot heat engine with harmonic oscillators at the classical limit. _Entropy_ 2020, _22_ , 457.
  * (31) Insinga, A.R. The quantum friction and optimal finite-time performance of the quantum Otto cycle. _Entropy_ 2020, _22_ , 1060.
  * (32) Dann, R.; Kosloff, R.; Salamon, P. Quantum finite-time thermodynamics: Insight from a single qubit engine. _Entropy_ 2020, _22_ , 1255.
  * (33) Kosloff, R. Quantum thermodynamics and open-systems modeling. _J. Chem. Phys._ 2019, _150_ , 204105.
  * (34) Chen, L.; Liu, X.; Wu, F.; Xia, S.; Feng, H. Exergy-based ecological optimization of an irreversible quantum Carnot heat pump with harmonic oscillators. _Phys. A Stat. Mech. Appl._ 2020, _537_ , 122597.
  * (35) Diaz, G.A.; Forero, J.D.; Garcia, J.; Rincon, A.; Fontalvo, A.; Bula, A.; Padilla, R.V. Maximum power from fluid flow by applying the first and second laws of thermodynamics. _J. Energy Resour. Technol._ 2017, _139_ , 3.
  * (36) Del Campo, A.; Boshier, M.G. Shortcuts to adiabaticity in a time-dependent box. _Sci. Rep._ 2012, _2_ , 1–6.
  * (37) Stefanatos, D. Optimal shortcuts to adiabaticity for a quantum piston. _Automatica_ 2013, _49_ , 3079–3083.
  * (38) Stefanatos, D. Exponential bound in the quest for absolute zero. _Phys. Rev. E_ 2017, _96_ , 042103.
  * (39) Chattopadhyay, P.; Paul, G. Relativistic quantum heat engine from uncertainty relation standpoint. _Sci. Rep._ 2019, _9_ , 16967.
  * (40) Fuchs, C.A.; Peres, A. Quantum-state disturbance versus information gain: Uncertainty relations for quantum information. _Phys. Rev. A_ 1996, _53_ , 2038.
  * (41) Koashi, M. Unconditional security of quantum key distribution and the uncertainty principle. _J. Phys. Conf. Ser._ 2006, _36_ , 16.
  * (42) Koashi, M. Simple security proof of quantum key distribution via uncertainty principle. _arXiv_ 2005, arXiv:0505108.
  * (43) Hofmann, H.F.; Takeuchi, S. Violation of local uncertainty relations as a signature of entanglement. _Phys. Rev. A_ 2003, _68_ , 032103.
  * (44) Osterloh, A. Entanglement and its facets in condensed matter systems. _arXiv_ 2008, arXiv:0810.1240.
  * (45) Marty, O.; Epping, M.; Kampermann, H.; Bruß, D.; Plenio, M.B.; Cramer, M. Quantifying entanglement with scattering experiments. _Phys. Rev. B_ 2014, _89_ , 125117.
  * (46) Gühne, O. Characterizing entanglement via uncertainty relations. _Phys. Rev. Lett._ 2004, _9_ , 117903.
  * (47) Giovannetti, V.; Lloyd, S.; Maccone, L. Advances in quantum metrology. _Nat. Photonics_ 2011, _5_ , 222.
  * (48) Mondal, D.; Datta, C.; Sazim, S.K. Quantum coherence sets the quantum speed limit for mixed states. _Phys. Lett. A_ 2016, _380_ , 689–695.
  * (49) Marvian, I.; Spekkens, R.W.; Zanardi, P. Quantum speed limits, coherence, and asymmetry. _Phys. Rev. A_ 2016, 93, 052331.
  * (50) Deffner, S.; Campbell, S. Quantum speed limits: from Heisenberg’s uncertainty principle to optimal quantum control. _J. Phys. A_ 2017, _50_ , 453001.
  * (51) Pires, D.P.; Cianciaruso, M.; Céleri, L.C.; Adesso, G.; Soares-Pinto, D.O. Generalized geometric quantum speed limits. _Phys. Rev. X_ 2016, _6_ , 021031,
  * (52) Xiao, L. Experimental test of uncertainty relations for general unitary operators. _Opt. Express_ 2017, _25_ , 17904–17910.
  * (53) Ma, W. Experimental test of Heisenberg’s measurement uncertainty relation based on statistical distances. _Phys. Rev. Lett._ 2016, _116_ , 160405.
  * (54) Baek, S.Y.; ; Kaneda, F.; Ozawa, ; M.; Edamatsu, K. Experimental violation and reformulation of the Heisenberg’s error disturbance uncertainty relation. _Sci. Rep._ 2013, _3_ , 2221.
  * (55) Mondal, D.; Bagchi, S.; Pati, A.K. Tighter uncertainty and reverse uncertainty relations. _Phys. Rev. A_ 2017, _95_ , 052117.
  * (56) Schiff, L.  Quantum Mechanics, International series in pure and applied physics; McGraw-Hill: New York, NY, USA, 1955\.
  * (57) Griffiths, D. J. Introduction to Quantum Mechanics; Pearson Prentice Hall: Upper Saddle River, NJ, USA, 2005\.
  * (58) Reif, F.  Fundamentals of Statistical and Thermal Physics; Waveland Press: Long Grove, IL, USA, 2009\.
  * (59) Maccone, L.; Pati, A.K. Stronger uncertainty relations for all incompatible observables. _Phys. Rev. Lett._ 2014, _113_ , 260401.
  * (60) Chattopadhyay, P.; Mitra, A.; Paul, G. Probing Uncertainty Relations in Non-Commutative Space. _Int. J. Theor. Phys._ 2019, _58_ , 2619–2631.
  * (61) Cerone, P.; Dragomir, S.S. Mathematical Inequalities; Chapman and Hall/CRC: London, UK, 2011; pp. 241–313.
  * (62) Saha, M.; and Srivastava, B. N. Treatise on Heat; Longman, Rees, Orme, Brown, Green & Longman: New York, NY, USA, 1935.
  * (63) Saygin, H.; Şişman, A. Quantum degeneracy effect on the work output from a Stirling cycle. _J. Appl. Phys._ 2001, _90_ , 3086–3089.
  * (64) Agarwal, G.S.; Chaturvedi, S. Quantum dynamical framework for Brownian heat engines. _Phys. Rev. E_ 2013, _88_ , 012130.
  * (65) Huang, X.L.; Niu, X.Y.; Xiu, X.M.; Yi, X.X. Quantum Stirling heat engine and refrigerator with single and coupled spin systems. _Eur. Phys. J. D_ 2014, _68_ , 32.
  * (66) Blickle, V.; Bechinger, C. Realization of a micrometre-sized stochastic heat engine. _Nat. Phys._ 2011, _8_ , 143.
  * (67) Thomas, G.; Das, D.; Ghosh, S. Quantum heat engine based on level degeneracy. _Phys. Rev. E_ 2019, _100_ , 012123.
  * (68) Niedenzu, W.; Mukherjee, V.; Ghosh, A.; Kofman, A.G.; Kurizki, G. Quantum engine efficiency bound beyond the second law of thermodynamics. _Nat. Commun._ 2018, _9_ , 165.
  * (69) Plenio, M.B.; Virmani, S. An introduction to entanglement measures. In _Quantum Information and Coherence_ ; Springer: Cham, Switzerland, 2014; pp. 173–209
  * (70) Alberto, P.; Fiolhais, C.; Gil, V.M.S. Relativistic particle in a box. _Eur. J. Phys._ 1996, _17_ , 19.
  * (71) Ryu, S.; Takayanagi, T. Holographic derivation of entanglement entropy from the anti–de sitter space/conformal field theory correspondence. _Phys. Rev. Lett._ 2006, _96_ , 181602.
  * (72) Kempf, A.; Mangano, G.; Mann, R.B. Hilbert space representation of the minimal length uncertainty relation. _Phys. Rev. D_ 1995, _52_ , 1108.
  * (73) Quesne, C.; Tkachuk, V.M. Composite system in deformed space with minimal length. _Phys. Rev. A_ 2010, _81_ , 012106.
  * (74) Meljanac, S.; Krešić-Jurić, S. Noncommutative differential forms on the kappa-deformed space. _J. Phys. A_ 2009, _42_ , 365204.
  * (75) Rezek, Y.; Kosloff, R. Irreversible performance of a quantum harmonic heat engine. _New J. Phys._ 2006, _8_ , 83.


* Fong et al. (2019) Ruth Fong, Mandela Patrick, and Andrea Vedaldi. 2019\.  Understanding Deep Networks via Extremal Perturbations and Smooth Masks. In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_. 2950–2958.
  * Gal and Ghahramani (2016) Yarin Gal and Zoubin Ghahramani. 2016.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In _Proceedings of the International Conference on Machine Learning (ICML)_. 1050–1059.
  * Gale et al. (2019) William Gale, Luke Oakden-Rayner, Gustavo Carneiro, Andrew P Bradley, and Lyle J Palmer. 2019\.  Producing Radiologist-Quality Reports for Interpretable Deep Learning. In _Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI)_. 1275–1279.
  * Ghorbani et al. (2019a) Amirata Ghorbani, Abubakar Abid, and James Zou. 2019a.  Interpretation of Neural Networks is Fragile. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 3681–3688.
  * Ghorbani et al. (2019b) Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. 2019b.  Towards Automatic Concept-Based Explanations. In _Advances in Neural Information Processing Systems_ , Vol. 32.
  * Goodman and Flaxman (2017) Bryce Goodman and Seth Flaxman. 2017.  European Union Regulations on Algorithmic Decision-Making and a “Right to Explanation”.  _AI Magazine_ 38, 3 (2017), 50–57.
  * Gour and Jain (2022) Mahesh Gour and Sweta Jain. 2022.  Uncertainty-aware convolutional neural network for COVID-19 X-ray images classification.  _Computers in Biology and Medicine_ 140 (2022), 105047.
  * Graziani et al. (2020) Mara Graziani, Vincent Andrearczyk, Stéphane Marchand-Maillet, and Henning Müller. 2020.  Concept attribution: Explaining CNN decisions to physicians.  _Computers in Biology and Medicine_ 123 (2020), 103865.
  * Groh et al. (2021) Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. 2021\.  Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 1820–1828.
  * Gulum et al. (2021) Mehmet A Gulum, Christopher M Trombley, and Mehmed Kantardzic. 2021.  A Review of Explainable Deep Learning Cancer Detection Models in Medical Imaging.  _Applied Sciences_ 11, 10 (2021), 4573.
  * Gunning and Aha (2019) David Gunning and David Aha. 2019.  DARPA’s Explainable Artificial Intelligence (XAI) Program.  _AI Magazine_ 40, 2 (2019), 44–58.
  * Gupta et al. (2021) Tarun Gupta, Libin Kutty, Ritu Gahir, Nnamdi Ukwu, Sayantan Polley, and Marcus Thiel. 2021\.  IRTEX: Image Retrieval with Textual Explanations. In _Proceedings of the IEEE International Conference on Human-Machine Systems (ICHMS)_. 1–4.
  * Hansell et al. (2008) David M. Hansell, Alexander A. Bankier, Heber MacMahon, Theresa C. McLoud, Nestor L. Müller, and Jacques Remy. 2008.  Fleischner Society: Glossary of Terms for Thoracic Imaging.  _Radiology_ 246, 3 (2008), 697–722.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising Diffusion Probabilistic Models.  _Advances in Neural Information Processing Systems_ 33 (2020), 6840–6851.
  * Hoffmann et al. (2021) Adrian Hoffmann, Claudio Fanconi, Rahul Rade, and Jonas Kohler. 2021.  This Looks Like That… Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks. In _ICML Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI_.
  * Hooker et al. (2019) Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. 2019.  A Benchmark for Interpretability Methods in Deep Neural Networks. In _Advances in Neural Information Processing Systems_ , Vol. 32. 9737–9748.
  * Hu et al. (2022) Brian Hu, Bhavan Vasu, and Anthony Hoogs. 2022.  X-MIR: EXplainable Medical Image Retrieval. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_. 440–450.
  * Irvin et al. (2019) Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. 2019\.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 33. 590–597.
  * Jing et al. (2018) Baoyu Jing, Pengtao Xie, and Eric Xing. 2018.  On the Automatic Generation of Medical Imaging Reports. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL)_.
  * Johnson et al. (2019) Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. 2019\.  MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports.  _Scientific Data_ 6, 1 (2019), 1–8.
  * Kawahara et al. (2019) Jeremy Kawahara, Sara Daneshvar, Giuseppe Argenziano, and Ghassan Hamarneh. 2019.  Seven-Point Checklist and Skin Lesion Classification Using Multitask Multimodal Neural Nets.  _IEEE Journal of Biomedical and Health Informatics_ 23, 2 (2019), 538–546.
  * Kazerouni et al. (2022) Amirhossein Kazerouni, Ehsan Khodapanah Aghdam, Moein Heidari, Reza Azad, Mohsen Fayyaz, Ilker Hacihaliloglu, and Dorit Merhof. 2022\.  Diffusion models for medical image analysis: A comprehensive survey.  _arXiv preprint arXiv:2211.07804_ (2022).
  * Keenan et al. (2020) Oisin J. F. Keenan, George Holland, Julian F. Maempel, John F. Keating, and Chloe E. H. Scott. 2020\.  Correlations between radiological classification systems and confirmed cartilage loss in severe knee osteoarthritis.  _The Bone & Joint Journal_ 102-B, 3 (2020), 301–309.
  * Kim et al. (2018) Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. 2018\.  Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In _Proceedings of the International Conference on Machine Learning (ICML)_. 2668–2677.
  * Kim et al. (2021b) Eunji Kim, Siwon Kim, Minji Seo, and Sungroh Yoon. 2021b.  XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanations. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 15719–15728.
  * Kim et al. (2021a) Junho Kim, Minsu Kim, and Yong Man Ro. 2021a.  Interpretation of Lesional Detection via Counterfactual Generation. In _Proceedings of the IEEE International Conference on Image Processing (ICIP)_. 96–100.
  * Kindermans et al. (2018) Pieter-Jan Kindermans, Kristof T Schütt, Maximilian Alber, Klaus-Robert Müller, Dumitru Erhan, Been Kim, and Sven Dähne. 2018\.  Learning how to explain neural networks: PatternNet and PatternAttribution. In _International Conference on Learning Representations (ICML)_.
  * Koh et al. (2020) Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020.  Concept Bottleneck Models. In _Proceedings of the International Conference on Machine Learning (ICML)_. 5338–5348.
  * Kokhlikyanet (2019) Kokhlikyanet. 2019\.  Pytorch Captum.  [Online]. Accessed January, 21 2021. Available: https://github.com/pytorch/captum.
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012\.  ImageNet Classification with Deep Convolutional Neural Networks.  _Advances in Neural Information Processing Systems (NIPS)_ 25 (2012).
  * Kumar et al. (2009) Neeraj Kumar, Alexander C. Berg, Peter N. Belhumeur, and Shree K. Nayar. 2009.  Attribute and Simile Classifiers for Face Verification. In _Proceedings of the IEEE International Conference on Computer Vision (CVPR)_. 365–372.
  * Lampert et al. (2009) Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. 2009\.  Learning to Detect Unseen Object Classes by Between-class Attribute Transfer. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 951–958.
  * Lamy et al. (2019) Jean-Baptiste Lamy, Boomadevi Sekar, Gilles Guezennec, Jacques Bouaud, and Brigitte Séroussi. 2019.  Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach.  _Artificial Intelligence in Medicine_ 94 (2019), 42–53.
  * Lapuschkin et al. (2016) Sebastian Lapuschkin, Alexander Binder, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2016.  The LRP Toolbox for Artificial Neural Networks.  _Journal of Machine Learning Research_ 17, 114 (2016), 1–5.
  * Lavie and Agarwal (2007) Alon Lavie and Abhaya Agarwal. 2007.  METEOR: An Automatic Metric for MT Evaluation with High Levels of Correlation with Human Judgments. In _Proceedings of the Workshop on Statistical Machine Translation_. 228–231.
  * Lee et al. (2019) Hyebin Lee, Seong Tae Kim, and Yong Man Ro. 2019.  Generation of Multimodal Justification Using Visual Word Constraint Model for Explainable Computer-Aided Diagnosis.  In _Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support_. 21–29.
  * Lee et al. (2017) Rebecca Sawyer Lee, Francisco Gimenez, Assaf Hoogi, Kanae Kawai Miyake, Mia Gorovoy, and Daniel L Rubin. 2017.  A curated mammography data set for use in computer-aided detection and diagnosis research.  _Scientific Data_ 4, 1 (2017), 1–9.
  * Li et al. (2018) Christy Y Li, Xiaodan Liang, Zhiting Hu, and Eric P Xing. 2018\.  Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation. In _Proceedings of the International Conference on Neural Information Processing Systems (NIPS)_. 1537–1547.
  * Li et al. (2023) Mingjie Li, Rui Liu, Fuyu Wang, Xiaojun Chang, and Xiaodan Liang. 2023.  Auxiliary signal-guided knowledge encoder-decoder for medical report generation.  _World Wide Web_ 26, 1 (2023), 253–270.
  * Lin (2004) Chin-Yew Lin. 2004\.  ROUGE: A Package for Automatic Evaluation of Summaries. In _Text Summarization Branches Out_. 74–81.
  * Lin and Lee (2020) Tsung-Chieh Lin and Hsi-Chieh Lee. 2020.  Covid-19 Chest Radiography Images Analysis Based on Integration of Image Preprocess, Guided Grad-CAM, Machine Learning and Risk Management. In _Proceedings of the International Conference on Medical and Health Informatics (ICMHI)_. 281–288.
  * Lipton (2017) Zachary C Lipton. 2017\.  The Doctor Just Won’t Accept That!  _arXiv preprint arXiv:1711.08037_ (2017).
  * Litjens et al. (2018) Geert Litjens, Peter Bandi, Babak Ehteshami Bejnordi, Oscar Geessink, Maschenka Balkenhol, et al. 2018\.  1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset.  _GigaScience_ 7, 6 (2018).
  * Litjens et al. (2017) Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. 2017.  A Survey on Deep Learning in Medical Image Analysis.  _Medical Image Analysis_ 42 (2017), 60–88.
  * Liu et al. (2021a) Fenglin Liu, Xian Wu, Shen Ge, Wei Fan, and Yuexian Zou. 2021a.  Exploring and Distilling Posterior and Prior Knowledge for Radiology Report Generation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 13753–13762.
  * Liu et al. (2021b) Fenglin Liu, Changchang Yin, Xian Wu, Shen Ge, Ping Zhang, and Xu Sun. 2021b.  Contrastive Attention for Automatic Chest X-ray Report Generation. In _Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_. 269–280.
  * Liu et al. (2019) Guanxiong Liu, Tzu-Ming Harry Hsu, Matthew McDermott, Willie Boag, Wei-Hung Weng, Peter Szolovits, and Marzyeh Ghassemi. 2019\.  Clinically Accurate Chest X-Ray Report Generation. In _Machine Learning for Healthcare Conference_. 249–269.
  * Lopatina et al. (2020) Alina Lopatina, Stefan Ropele, Renat Sibgatulin, Jürgen R Reichenbach, and Daniel Güllmar. 2020\.  Investigation of Deep-Learning-Driven Identification of Multiple Sclerosis Patients Based on Susceptibility-Weighted Images Using Relevance Analysis.  _Frontiers in N euroscience_ (2020), 1356.
  * Lucieri et al. (2022) Adriano Lucieri, Muhammad Naseer Bajwa, Stephan Alexander Braun, Muhammad Imran Malik, Andreas Dengel, and Sheraz Ahmed. 2022\.  ExAID: A Multimodal Explanation Framework for Computer-Aided Diagnosis of Skin Lesions.  _Computer Methods and Programs in Biomedicine_ (2022), 106620.
  * Lucieri et al. (2020) Adriano Lucieri, Muhammad Naseer Bajwa, Andreas Dengel, and Sheraz Ahmed. 2020.  Explaining AI-based Decision Support Systems using Concept Localization Maps. In _Proceedings of the International Conference on Neural Information Processing_. 185–193.
  * Lundberg et al. (2018) Scott M Lundberg, Gabriel G Erion, and Su-In Lee. 2018\.  Consistent Individualized Feature Attribution for Tree Ensembles.  _arXiv preprint arXiv:1802.03888_ (2018).
  * Lundberg and Lee (2017) Scott M Lundberg and Su-In Lee. 2017.  A Unified Approach to Interpreting Model Predictions. In _Proceedings of the International Conference on Neural Information Processing Systems (NIPS)_. 4768–4777.
  * Madry et al. (2018) Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018\.  Towards Deep Learning Models Resistant to Adversarial Attacks. In _Proceedings of the International Conference on Learning Representations (ICLR)_.
  * Magesh et al. (2020) Pavan Rajkumar Magesh, Richard Delwin Myloth, and Rijo Jackson Tom. 2020.  An Explainable Machine Learning Model for Early Detection of Parkinson’s Disease using LIME on DaTscan Imagery.  _Computers in Biology and Medicine_ 126 (2020), 104041.
  * Majkowska et al. (2020) Anna Majkowska, Sid Mittal, David F Steiner, Joshua J Reicher, Scott Mayer McKinney, Gavin E Duggan, Krish Eswaran, Po-Hsuan Cameron Chen, Yun Liu, Sreenivasa Raju Kalidindi, et al. 2020\.  Chest Radiograph Interpretation with Deep Learning Models: Assessment with Radiologist-adjudicated Reference Standards and Population-adjusted Evaluation.  _Radiology_ 294, 2 (2020), 421–431.
  * Malhi et al. (2019) Avleen Malhi, Timotheus Kampik, Husanbir Pannu, Manik Madhikermi, and Kary Främling. 2019\.  Explaining Machine Learning-Based Classifications of In-Vivo Gastral Images. In _Digital Image Computing: Techniques and Applications (DICTA)_. 1–7.
  * Margeloiu et al. (2020) Andrei Margeloiu, Nikola Simidjievski, Mateja Jamnik, and Adrian Weller. 2020.  Improving Interpretability in Medical Imaging Diagnosis using Adversarial Training. In _Medical Imaging Meets NeurIPS Workshop_.
  * Matsoukas et al. (2021) Christos Matsoukas, Johan Fredin Haslum, Magnus Söderberg, and Kevin Smith. 2021\.  Is it Time to Replace CNNs with Transformers for Medical Images?. In _ICCV 2021: Workshop on Computer Vision for Automated Medical Diagnosis (CVAMD)_.
  * Mendonça et al. (2013) Teresa Mendonça, Pedro Ferreira, Jorge Marques, André Marcal, and Jorge Rozeira. 2013\.  PH 2-A dermoscopic image database for research and benchmarking. In _Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS)_. 5437–5440.
  * Messina et al. (2022) Pablo Messina, Pablo Pino, Denis Parra, Alvaro Soto, Cecilia Besa, Sergio Uribe, Marcelo Andía, Cristian Tejos, Claudia Prieto, and Daniel Capurro. 2022\.  A survey on deep learning and explainability for automatic report generation from medical images.  _ACM Computing Surveys (CSUR)_ 54, 10s (2022), 1–40.
  * Molnar (2022) Christoph Molnar. 2022\.  Interpretable Machine Learning: A Guide for Making Black Box Models Explainable.  Available: https://christophm.github.io/interpretable-ml-book.
  * Montavon et al. (2017) Grégoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, and Klaus-Robert Müller. 2017.  Explaining NonLinear Classification Decisions with Deep Taylor Decomposition.  _Pattern Recognition_ 65 (2017), 211–222.
  * Moore and Swartout (1988) Johanna D Moore and William R Swartout. 1988.  Explanation in Expert Systems: A Survey.  _University of Southern California_ (1988).
  * Moreira et al. (2012) Inês C Moreira, Igor Amaral, Inês Domingues, António Cardoso, Maria Joao Cardoso, and Jaime S Cardoso. 2012.  INbreast: Toward a Full-field Digital Mammographic Database.  _Academic Radiology_ 19, 2 (2012), 236–248.
  * Nevitt et al. (2006) M Nevitt, D Felson, and Gayle Lester. 2006.  The Osteoarthritis Initiative.  _Protocol for the Cohort Study_ 1 (2006).
  * Nguyen et al. (2020) Ha Nguyen, Khanh Lam, Linh Le, Hieu Pham, Dat Tran, et al. 2020\.  VinDr-CXR: An Open Dataset of Chest X-rays with Radiologist’s Annotations.  _arXiv preprint arXiv:2012.15029_ (2020).
  * Nguyen et al. (2021) Hieu T. Nguyen, Hieu H. Pham, Nghia T. Nguyen, Ha Q. Nguyen, Thang Q. Huynh, et al. 2021\.  VinDr-SpineXR: A Deep Learning Framework for Spinal Lesions Detection and Classification from Radiographs. In _Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)_. 291–301.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002\.  BLEU: a Method for Automatic Evaluation of Machine Translation. In _Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)_. 311–318.
  * Pascanu et al. (2013) Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013\.  On the difficulty of training Recurrent Neural Networks. In _Proceedings of the International Conference on Machine Learning (ICML)_. 1310–1318.
  * Patrício et al. (2023) Cristiano Patrício, João C. Neves, and Luis F. Teixeira. 2023\.  Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_. 3799–3808.
  * Pelka et al. (2018) Obioma Pelka, Sven Koitka, Johannes Rückert, Felix Nensa, et al. 2018\.  Radiology Objects in COntext (ROCO): A Multimodal Image Dataset. In _Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis_. 180–189.
  * Petsiuk et al. (2018) Vitali Petsiuk, Abir Das, and Kate Saenko. 2018.  RISE: Randomized Input Sampling for Explanation of Black-box Models. In _British Machine Vision Conference (BMVC)_.
  * Pocevičiūtė et al. (2020) Milda Pocevičiūtė, Gabriel Eilertsen, and Claes Lundström. 2020.  Survey of XAI in Digital Pathology.  In _Artificial Intelligence and Machine Learning for Digital Pathology: State-of-the-Art and Future Challenges_. 56–88.
  * Punn and Agarwal (2021) Narinder Singh Punn and Sonali Agarwal. 2021.  Automated diagnosis of COVID-19 with limited posteroanterior chest X-ray images using fine-tuned deep neural networks.  _Applied Intelligence_ 51, 5 (2021), 2689–2702.
  * Rajpurkar et al. (2018) Pranav Rajpurkar, Jeremy Irvin, Robyn L Ball, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, et al. 2018\.  Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists.  _PLoS medicine_ 15, 11 (2018).
  * Rehman et al. (2021) Aasia Rehman, Muheet Ahmed Butt, and Majid Zaman. 2021.  A Survey of Medical Image Analysis Using Deep Learning Approaches. In _Proceedings of the IEEE International Conference on Computing Methodologies and Communication (ICCMC)_. 1334–1342.
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016\.  ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In _Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. 1135–1144.
  * Richter et al. (2020) Lorenz Richter, Ayman Boustati, Nikolas Nüsken, Francisco Ruiz, and Omer Deniz Akyildiz. 2020\.  VarGrad: A Low-Variance Gradient Estimator for Variational Inference.  _Advances in Neural Information Processing Systems_ 33 (2020), 13481–13492.
  * Rio-Torto et al. (2020) Isabel Rio-Torto, Kelwin Fernandes, and Luís Teixeira. 2020\.  Understanding the decisions of CNNs: An in-model approach.  _Pattern Recognition Letters_ 133 (2020), 373–380.
  * Robertson (2004) Stephen Robertson. 2004\.  Understanding Inverse Document Frequency: on Theoretical Arguments for IDF.  _Journal of Documentation_ 60 (2004), 503–520.
  * Rotemberg et al. (2021) Veronica Rotemberg, Nicholas Kurtansky, Brigid Betz-Stablein, Liam Caffery, Emmanouil Chousakos, Noel Codella, Marc Combalia, Stephen Dusza, Pascale Guitera, David Gutman, et al. 2021\.  A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Context.  _Scientific Data_ 8, 1 (2021), 1–8.
  * Rudin (2019) Cynthia Rudin. 2019\.  Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead.  _Nature Machine Intelligence_ 1, 5 (2019), 206–215.
  * Sadeghi et al. (2018) Mahya Sadeghi, Parmit K Chilana, and M Stella Atkins. 2018\.  How Users Perceive Content-based Image Retrieval for Identifying Skin Images.  In _Understanding and Interpreting Machine Learning in Medical Image Computing Applications_. 141–148.
  * Salahuddin et al. (2022) Zohaib Salahuddin, Henry C. Woodruff, Avishek Chatterjee, and Philippe Lambin. 2022.  Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods.  _Computers in Biology and Medicine_ 140 (2022), 105111.
  * Samek et al. (2016) Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. 2016\.  Evaluating the Visualization of What a Deep Neural Network Has Learned.  _IEEE Transactions on Neural Networks and Learning Systems_ 28, 11 (2016), 2660–2673.
  * Sanchez et al. (2022) Pedro Sanchez, Antanas Kascenas, Xiao Liu, Alison Q O’Neil, and Sotirios A Tsaftaris. 2022\.  What is healthy? generative counterfactual diffusion for lesion localization. In _Deep Generative Models: Second MICCAI Workshop, DGM4MICCAI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 22, 2022, Proceedings_. 34–44.
  * Sarkar et al. (2022) Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, and Vineeth N Balasubramanian. 2022\.  A Framework for Learning Ante-hoc Explainable Models via Concepts. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 10286–10295.
  * Sayres et al. (2019) Rory Sayres, Ankur Taly, Ehsan Rahimy, Katy Blumer, David Coz, Naama Hammel, Jonathan Krause, Arunachalam Narayanaswamy, Zahra Rastegar, Derek Wu, et al. 2019\.  Using a Deep Learning Algorithm and Integrated Gradients Explanation to Assist Grading for Diabetic Retinopathy.  _Ophthalmology_ 126, 4 (2019), 552–564.
  * Schlegl et al. (2015) Thomas Schlegl, Sebastian M Waldstein, Wolf-Dieter Vogl, Ursula Schmidt-Erfurth, and Georg Langs. 2015.  Predicting Semantic Descriptions from Medical Images with Convolutional Neural Networks. In _Proceedings of the International Conference on Information Processing in Medical Imaging (IPMI)_. 437–448.
  * Schutte et al. (2021) Kathryn Schutte, Olivier Moindrot, Paul Hérent, Jean-Baptiste Schiratti, and Simon Jégou. 2021\.  Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Images. In _Medical Imaging Meets NeurIPS Workshop_.
  * Selivanov et al. (2023) Alexander Selivanov, Oleg Y Rogov, Daniil Chesakov, Artem Shelmanov, Irina Fedulova, and Dmitry V Dylov. 2023.  Medical image captioning via generative pretrained transformers.  _Scientific Reports_ 13, 1 (2023), 4171.
  * Selvaraju et al. (2017) Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. In _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_. 618–626.
  * Shamshad et al. (2023) Fahad Shamshad, Salman Khan, Syed Waqas Zamir, Muhammad Haris Khan, Munawar Hayat, Fahad Shahbaz Khan, and Huazhu Fu. 2023.  Transformers in Medical Imaging: A Survey.  _Medical Image Analysis_ (2023), 102802.
  * Shapley (2016) L. S. Shapley. 2016\.  A Value for n-Person Games.  In _Contributions to the Theory of Games (AM-28)_ , Harold William Kuhn and Albert William Tucker (Eds.). Vol. II. 307–318.
  * Shrikumar et al. (2017) Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017\.  Learning Important Features Through Propagating Activation Differences. In _Proceedings of the International Conference on Machine Learning (ICML)_ , Vol. 70. 3145–3153.
  * Silva et al. (2020) Wilson Silva, Alexander Poellinger, Jaime S Cardoso, and Mauricio Reyes. 2020.  Interpretability-guided Content-based Medical Image Retrieval. In _Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)_. 305–314.
  * Simonyan et al. (2014) Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014\.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. In _Workshop at International Conference on Learning Representations (ICLR)_.
  * Singh et al. (2020) Amitojdeep Singh, Sourya Sengupta, and Vasudevan Lakshminarayanan. 2020\.  Explainable Deep Learning Models in Medical Image Analysis.  _Journal of Imaging_ 6, 6 (2020), 52.
  * Singh and Yow (2021a) Gurmail Singh and Kin-Choong Yow. 2021a.  An Interpretable Deep Learning Model For Covid-19 Detection With Chest X-ray Images.  _IEEE Access_ 9 (2021), 85198–85208.
  * Singh and Yow (2021b) Gurmail Singh and Kin-Choong Yow. 2021b.  These do not Look Like Those: An Interpretable Deep Learning Model for Image Recognition.  _IEEE Access_ 9 (2021), 41482–41493.
  * Singh et al. (2019) Sonit Singh, Sarvnaz Karimi, Kevin Ho-Shon, and Len Hamey. 2019\.  From Chest X-Rays to Radiology Reports: A Multimodal Machine Learning Approach. In _Digital Image Computing: Techniques and Applications (DICTA)_. 1–8.
  * Singla et al. (2023) Sumedha Singla, Motahhare Eslami, Brian Pollack, Stephen Wallace, and Kayhan Batmanghelich. 2023\.  Explaining the Black-box Smoothly - A Counterfactual Approach.  _Medical Image Analysis_ 84 (2023), 102721.
  * Smilkov et al. (2017) Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. 2017.  SmoothGrad: Removing Noise by Adding Noise. In _ICML Workshop on Visualization for Deep Learning_.
  * Society (2019) Asia Pacific Tele-Ophthalmology Society. 2019\.  APTOS Diabetic Retinopathy Dataset.  [Online]. Accessed November, 23 2021. Available: https://www.kaggle.com/c/aptos2019-blindness-detection/data.
  * Spanhol et al. (2015) Fabio A Spanhol, Luiz S Oliveira, Caroline Petitjean, and Laurent Heutte. 2015.  A Dataset for Breast Cancer Histopathological Image Classification.  _IEEE Transactions on Biomedical Engineering_ 63, 7 (2015), 1455–1462.
  * Springenberg et al. (2014) Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. 2014.  Striving for Simplicity: The All Convolutional Net.  _arXiv preprint arXiv:1412.6806_ (2014).
  * Sun et al. (2019) Li Sun, Weipeng Wang, Jiyun Li, and Jingsheng Lin. 2019\.  Study on Medical Image Report Generation Based on Improved Encoding-Decoding Method. In _Intelligent Computing Theories and Application_. 686–696.
  * Sundararajan et al. (2017) Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017.  Axiomatic Attribution for Deep Networks. In _Proceedings of the International Conference on Machine Learning (ICML)_. 3319–3328.
  * Thiagarajan et al. (2021) Ponkrshnan Thiagarajan, Pushkar Khairnar, and Susanta Ghosh. 2021\.  Explanation and Use of Uncertainty Obtained by Bayesian Neural Network Classifiers for Breast Histopathology Images.  _IEEE Transactions on Medical Imaging_ 41, 4 (2021), 815–825.
  * Tjoa and Guan (2020) Erico Tjoa and Cuntai Guan. 2020.  A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI.  _IEEE Transactions on Neural Networks and Learning Systems_ 32, 11 (2020), 4793–4813.
  * Tonekaboni et al. (2019) Sana Tonekaboni, Shalmali Joshi, Melissa D McCradden, and Anna Goldenberg. 2019.  What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use. In _Proceedings of the Machine Learning for Healthcare Conference_. 359–380.
  * Tschandl et al. (2019) Philipp Tschandl, Giuseppe Argenziano, Majid Razmara, and Jordan Yap. 2019.  Diagnostic Accuracy of Content Based Dermatoscopic Image Retrieval with Deep Classification Features.  _British Journal of Dermatology_ 181, 1 (2019), 155–165.
  * Tschandl et al. (2018) Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. 2018\.  The HAM10000 Dataset: A Large Collection of Multi-Source Dermatoscopic Images of Common Pigmented Skin Lesions.  _Scientific Data_ 5, 1 (2018), 1–9.
  * Tsiknakis et al. (2020) Nikos Tsiknakis, Eleftherios Trivizakis, Evangelia E Vassalou, Georgios Z Papadakis, Demetrios A Spandidos, Aristidis Tsatsakis, Jose Sánchez-García, Rafael López-González, Nikolaos Papanikolaou, Apostolos H Karantanas, et al. 2020\.  Interpretable artificial intelligence framework for COVID-19 screening on chest X-rays.  _Experimental and Therapeutic Medicine_ 20, 2 (2020), 727–735.
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton. 2008.  Visualizing Data using t-SNE.  _Journal of Machine Learning Research_ 9, 11 (2008).
  * van der Velden et al. (2022) Bas H.M. van der Velden, Hugo J. Kuijf, Kenneth G.A. Gilhuijs, and Max A. Viergever. 2022.  Explainable Artificial Intelligence (XAI) in Deep Learning-based Medical Image Analysis.  _Medical Image Analysis_ 79 (2022), 102470.
  * Van Lent et al. (2004) Michael Van Lent, William Fisher, and Michael Mancuso. 2004\.  An Explainable Artificial Intelligence System for Small-unit Tactical Behavior. In _Proceedings of the National Conference on Artificial Intelligence (AAAI)_. 900–907.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention Is All You Need. In _Advances in Neural Information Processing Systems_. 5998–6008.
  * Vedantam et al. (2015) Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015\.  CIDEr: Consensus-based Image Description Evaluation. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 4566–4575.
  * Wang et al. (2020a) Linda Wang, Zhong Qiu Lin, and Alexander Wong. 2020a.  COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images.  _Scientific Reports_ 10, 1 (2020), 1–12.
  * Wang et al. (2020b) Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Darrin Eide, Kathryn Funk, Rodney Kinney, Ziyang Liu, William Merrill, et al. 2020b.  CORD-19: The COVID-19 Open Research Dataset.  _ArXiv_ (2020).
  * Wang et al. (2021) Sutong Wang, Yunqiang Yin, Dujuan Wang, Yanzhang Wang, and Yaochu Jin. 2021.  Interpretability-Based Multimodal Convolutional Neural Networks for Skin Lesion Diagnosis.  _IEEE Transactions on Cybernetics_ (2021).
  * Wang et al. (2017) Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald Summers. 2017.  ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 2097–2106.
  * Wang et al. (2018) Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, and Ronald Summers. 2018.  TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rays. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 9049–9058.
  * Wang et al. (2023) Zhanyu Wang, Lingqiao Liu, Lei Wang, and Luping Zhou. 2023\.  METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokens. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 11558–11567.
  * Wang et al. (2022) Zhanyu Wang, Mingkang Tang, Lei Wang, Xiu Li, and Luping Zhou. 2022.  A Medical Semantic-Assisted Transformer for Radiographic Report Generation. In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part III_. 655–664.
  * Wickramanayake et al. (2021a) Sandareka Wickramanayake, Wynne Hsu, and Mong Li Lee. 2021a.  Comprehensible Convolutional Neural Networks via Guided Concept Learning. In _Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN)_. 1–8.
  * Wickramanayake et al. (2021b) Sandareka Wickramanayake, Wynne Hsu, and Mong Li Lee. 2021b.  Explanation-based Data Augmentation for Image Classification. In _Advances in Neural Information Processing Systems_ , Vol. 34.
  * Windisch et al. (2020) Paul Windisch, Pascal Weber, Christoph Fürweger, Felix Ehret, Markus Kufeld, Daniel Zwahlen, and Alexander Muacevic. 2020.  Implementation of model explainability for a basic brain tumor detection using convolutional neural networks on MRI slices.  _Neuroradiology_ 62, 11 (2020), 1515–1518.
  * Xu et al. (2015) Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015\.  Show, attend and tell: Neural image caption generation with visual attention. In _International Conference on Machine Learning (ICML)_. 2048–2057.
  * Xu et al. (2021) Yesheng Xu, Ming Kong, Wenjia Xie, Runping Duan, Zhengqing Fang, Yuxiao Lin, Qiang Zhu, Siliang Tang, Fei Wu, and Yu-Feng Yao. 2021\.  Deep Sequential Feature Learning in Clinical Image Classification of Infectious Keratitis.  _Engineering_ 7, 7 (2021), 1002–1010.
  * Yan et al. (2023) Siyuan Yan, Zhen Yu, Xuelin Zhang, Dwarikanath Mahapatra, Shekhar S Chandra, Monika Janda, Peter Soyer, and Zongyuan Ge. 2023\.  Towards Trustable Skin Cancer Diagnosis via Rewriting Model’s Decision. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 11568–11577.
  * Yang et al. (2023) Shuxin Yang, Xian Wu, Shen Ge, Zhuozhao Zheng, S Kevin Zhou, and Li Xiao. 2023\.  Radiology report generation with a learned knowledge base and multi-modal alignment.  _Medical Image Analysis_ (2023), 102798.
  * Yin et al. (2019) Changchang Yin, Buyue Qian, Jishang Wei, Xiaoyu Li, Xianli Zhang, Yang Li, and Qinghua Zheng. 2019.  Automatic Generation of Medical Imaging Diagnostic Report with Hierarchical Recurrent Neural Network. In _Proceedings of the IEEE International Conference on Data Mining (ICDM)_. 728–737.
  * Young et al. (2019) Kyle Young, Gareth Booth, Becks Simpson, Reuben Dutton, and Sally Shrapnel. 2019.  Deep Neural Network or Dermatologist?  In _Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support_. 48–55.
  * Yu et al. (2021) Hang Yu, Laurence T Yang, Qingchen Zhang, David Armstrong, and M Jamal Deen. 2021.  Convolutional Neural Networks for Medical Image Analysis.  _Neurocomputing_ 444 (2021), 92–110.
  * Yuksekgonul et al. (2022) Mert Yuksekgonul, Maggie Wang, and James Zou. 2022.  Post-hoc concept bottleneck models.  _arXiv preprint arXiv:2205.15480_ (2022).
  * Zeiler and Fergus (2014) Matthew D Zeiler and Rob Fergus. 2014.  Visualizing and Understanding Convolutional Networks. In _Proceedings of the European Conference on Computer Vision (ECCV)_. 818–833.
  * Zhang et al. (2023) Junsan Zhang, Xiuxuan Shen, Shaohua Wan, Sotirios K Goudos, Jie Wu, Ming Cheng, and Weishan Zhang. 2023.  A Novel Deep Learning Model for Medical Report Generation by Inter-Intra Information Calibration.  _IEEE Journal of Biomedical and Health Informatics_ (2023).
  * Zhang et al. (2017) Zizhao Zhang, Yuanpu Xie, Fuyong Xing, Mason McGough, and Lin Yang. 2017.  MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 6428–6436.
  * Zhou et al. (2016) Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016.  Learning Deep Features for Discriminative Localization. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 2921–2929.

##  Appendix A Appendix

###  A.1. Intepretability Frameworks

The increasing interest in the interpretability of machine learning fostered
the development of frameworks implementing classical XAI methods. The LRP
Toolbox (70) was launched in 2016 and provides the implementation of the LRP
(10) algorithm for artificial neural networks supporting Matlab and Python.
Additionally, the LRP toolbox released an extension to be compatible with the
Caffe Deep Learning framework. DeepExplain (7) is a framework that implements
perturbation and gradient-based attribution methods, including Saliency Maps
(133), Gradient * Input (131), Integrated Gradients (IG) (144), DeepLIFT
(131), $\epsilon$-LRP (10), and DeConvNet (174). It also supports Tensorflow
as well as Keras with Tensorflow backend. Alternatively, iNNvestigate (5) is a
more complete toolbox that provides implementations for SmoothGrad (139),
DeConvNet (174), Guided-BackProp (142), PatternNet (63), Input * Gradients
(131), DeepTaylor (98), PatternAttribution (63), LRP (10) and Integrated
Gradients (IG) (144). It also supports Tensorflow and Keras.

With regard to PyTorch frameworks, TorchRay (37) implements several
visualization methods, namely Gradient (133), Guided-BackProp (142), Grad-CAM
(128) and RISE (108). TorchRay is considered research-oriented, since it
provides code for reproducing results that appear in several papers. Captum
(65) is a PyTorch library that provides state-of-the-art algorithms for model
interpretability and understanding. It contains general purpose
implementations of Integrated Gradients (144), SmoothGrad (139), VarGrad (114)
and others for PyTorch models. Table 4 summarizes the aforementioned
frameworks alongside the supported XAI methods. Additionally, we refer the
reader to the work of Darias et al. (30) in which some other model-agnostic
XAI libraries were approached.

Table 4. Publicly available interpretability frameworks.

Framework | Year | Methods |  Supported DL Libraries
---|---|---|---
LRP Toolbox (70) | 2016 | LRP | Caffe
DeepExplain (7) | 2017 |  Saliency maps, Grad * Input, $\epsilon$-LRP, DeepLIFT, DeConvNet | Tensorflow, Keras
iNNvestigate (5) | 2019 |  SmoothGrad, DeConvNet, Guided-BackProp, PatternNet, LRP Input * Gradients, DeepTaylor, PatternAttribution, IG | Tensorflow, Keras
TorchRay (37) | 2019 |  Gradient, Guided-BackProp, Grad-CAM, RISE | PyTorch
Captum (65) | 2019 |  SmoothGrad, DeConvNet, Guided-BackProp, PatternNet, LRP Input * Gradients, DeepLIFT, DeepTaylor, LIME, Kernel SHAP, IG | PyTorch

###  A.2. Methods

Table 5. Summary of the XAI methods categorized by interpretability method
employed, image modality and dataset. The spaces marked with a ”-” mean that
explanation is only provided through text sentences.

Method | Year | Interpretability Method | Modality | Dataset
---|---|---|---|---
Zhang et al. (176) | 2017 | Attention-based | Microscopic Images | BCIDR
Jing et al. (55) | 2017 | Attention-based | X-ray | IU Chest X-ray, PEIR Gross
Rajpurkar et al. (111) | 2018 | CAM | X-ray | ChestX-ray8
Wang et al. (160) | 2018 | Saliency maps | X-ray | IU Chest X-ray,ChestX-ray14
Gale et al. (39) | 2018 | SmoothGrad | X-ray | Pelvic X-ray
Li et al. (74) | 2018 | Text-based | X-ray | IU Chest X-ray, CX-CHR (private)
Malhi et al. (92) | 2019 | LIME | Endoscopy | Red Lesion Endoscopy
Young et al. (171) | 2019 | KernelSHAP | Dermoscopy | HAM10000
Eitel et al. (35) | 2019 | Occlusion | MRI | ADNI Initiative
Sayres et al. (124) | 2019 | Integrated Gradients | Fundus Images | Private
Barata et al. (12) | 2019 | CAM | Dermoscopy | ISIC 2017, ISIC 2018
Tschandl et al. (148) | 2019 | CBIR | Dermoscopy | EDRA, ISIC 2017, Private Dataset
Sun et al. (143) | 2019 | Text-based | Mammography | Inbreast
Lee et al. (72) | 2019 | Saliency maps | Mammography | CBIS-DDSM
Lamy et al. (69) | 2019 | CBR | Mammography |  BCW, Mammographic Mass, Breast Cancer
Singh et al. (137) | 2019 | Stacked LSTM | X-ray | IU Chest X-ray
Yin et al. (170) | 2019 | t-SNE | X-ray | IU Chest X-ray
Liu et al. (83) | 2019 | Attention maps | X-ray | IU Chest X-ray, MIMIC-CXR
Windish et al. (165) | 2020 | Grad-CAM | MRI | IXI, Gliobastoma
Magesh et al. (90) | 2020 | LIME | DaTscan | PPMI
Lin et al. (77) | 2020 | Guided Grad-CAM | X-ray | COVIDx
Lopatina et al. (84) | 2020 | DeepLIFT | MRI | Private
Graziani et al. (44) | 2020 | TCAV | Microscopic Images | Camelyon16, Camelyon17
Margeloiu et al. (93) | 2020 | Adversarial Training | Dermoscopy | HAM10000
Rio-Torto et al. (115) | 2020 | In-model | Microscopic Images | NIH-NCI Cervical Cancer
Fang et al. (36) | 2020 | Concept-based | Microscopic Images | Infectious Keratitis Dataset
Chen et al. (25) | 2020 | Multi-Head Attention | X-ray | IU Chest X-ray, MIMIC-CXR
Silva et al. (132) | 2020 | CBIR, Saliency Map | X-ray | CheXpert
Punn et al. (110) | 2021 | LIME | X-ray | COVID-19 Dataset
Wang et al. (158) | 2021 | SHAP | Dermoscopy | HAM10000
Billah and Javed (16) | 2021 | BCNN | Microscopy Images | ALL-IDB
Barata et al. (13) | 2021 | CBIR | Dermoscopy | ISIC 2018
Thiagarajan et al. (145) | 2021 | t-SNE | Microscopic Images | Breast Histopathology
Barnett et al. (14) | 2021 | Prototype Activation Map | Mammography | Mammography Private Dataset
Kim et al. (62) | 2021 | Counterfactual Examples | X-ray | Chest X-ray 14, VinDr-CXR
Schutte et al. (126) | 2021 | Grad-CAM | X-ray/Microscopy Images | Osteoarthritis X-ray, Camelyon16
Kim et al. (61) | 2021 | Saliency maps | X-ray | NIH chest X-ray14
Singh et al. (135) | 2021 | Prototype Activation Maps | X-ray | CORD-19
Liu et al. (81) | 2021 | Text-based | X-ray | IU Chest X-ray, MIMIC-CXR
Liu et al. (82) | 2021 | Text-based | X-ray | IU Chest X-ray, MIMIC-CXR
Lucieri et al. (85) | 2022 | TCAV | Dermoscopy | ISIC 2019, Derm7pt, PH2, SkinL2
Hu et al. (53) | 2022 | CBIR | X-ray/Dermoscopy | COVIDx, ISIC 2019
Gour and Jain (43) | 2022 | Uncertainty-based | X-ray | COVID19CXr
Yuksekgonul et al. (173) | 2022 | CBM | Dermoscopy | HAM 10000, ISIC 2020
Wang et al. (162) | 2022 | Text-based | X-ray | MIMIC-CXR
Singla et al. (138) | 2023 | Counterfactual Examples | X-ray | MIMIC-CXR
Yan et al. (168) | 2023 | CBM | Dermoscopy | ISIC 2016-2020
Selivanov et al. (127) | 2023 | Text-based | X-ray | IU Chest X-ray, MIMIC-CXR
Zhang et al. (175) | 2023 | Text-based | X-ray | IU Chest X-ray, MIMIC-CXR, COV-CTR
Wang et al. (161) | 2023 | Text-based | X-ray | IU Chest X-ray, MIMIC-CXR
Chen et al. (23) | 2023 | Text-based | X-ray | IU Chest X-ray, MIMIC-CXR
Yang et al. (169) | 2023 | Text-based | X-ray | IU Chest X-ray, MIMIC-CXR
Patrício et al. (106) | 2023 | CBM | Dermoscopy | PH2, Derm7pt