# Structural Influence on Exciton Formation and the Critical Role of Dark
Excitons in Polymeric Carbon Nitrides

Changbin Im  Institute of Electrochemistry, Ulm University, Albert-Einstein-
Allee 47, 89081 Ulm, Germany.  Radim Beranek  Institute of Electrochemistry,
Ulm University, Albert-Einstein-Allee 47, 89081 Ulm, Germany.  Timo Jacob
Institute of Electrochemistry, Ulm University, Albert-Einstein-Allee 47, 89081
Ulm, Germany.  Helmholtz-Institute-Ulm, Helmholtzstr. 11, 89081, Ulm, Germany
Karlsruhe Institute of Technology (KIT), P.O. Box 3640, 76021, Karlsruhe,
Germany

###### Abstract

Polymeric carbon nitrides (PCNs) exhibit intriguing optical properties and
exceptional performance in (photo)catalysis, optoelectronics, and energy
storage. Nevertheless, the intricate phenomena involving light absorption,
formation of long-lived excitons, photo-charging, and photochemical processes
observed in PCNs remain poorly understood. Our theoretical investigation
elucidates the origin of distinct dark and bright excitons, their stability
and lifetimes, and their correlation with the microstructural attributes of
PCNs. Furthermore, we identify the decisive role of dark excitons in catalytic
reactivity, which underlies the differences in the photocatalytic performance
of different PCN derivatives.

††preprint: APS

Introduction.—Polymeric carbon nitrides (PCNs), commonly referred to as
graphitic carbon nitrides or melon, along with their (semi)crystalline
derivatives such as poly(heptazine imide) (PHI) have emerged as compelling
materials for photocatalysis, optoelectronics, and energy storage
applications.[1, 2, 3, 4, 5] This can be attributed to their straightforward
synthesis, robust chemical stability, and optical properties that span from
the UV to visible light range.[5] Research on PCNs has demonstrated their
efficacy in photocatalytic applications such as hydrogen production under
near-visible light, water splitting, CO2 reduction, and H2O2 production.[6]
Highly crystalline forms of carbon nitrides are typically obtained using
molten salt methods, resulting in highly ordered 2D structures.[7]
Particularly, these derivatives, containing cations, are of significant
interest due to enhanced photocatalytic activity and the photo-charging
behavior that can be utilized in time-delayed photocatalysis or boosted
photocurrent response.[8, 9, 10] Despite the plethora of useful and intriguing
properties, the knowledge of the relationship between the structural and
optical properties in PCN materials is still largely underdeveloped.[5]
Merschjann et al. reported that charge transport in PCN structures occurs
between interplanar layers.[11] Godin et al. discussed the charge trapping
phenomena in conventional PCNs, and Yang et al. reported that the excess
electron accumulation in ionic PCNs leads to lower quantum yields due to
accelerated electron–hole ($e$–$h$) recombination.[12, 13] However, any clear
conclusions from experimental studies of exciton dynamics in PCNs using time-
resolved spectroscopy remain limited due to the lack of detailed understanding
of the structure-induced excited state properties.[12, 13, 14] For example,
the frequently observed trapped excitons in PCNs are typically attributed to
structural defects. Nonetheless, there is a notable absence of detailed
discussion on the nature of these structural defects or their impact on
photocatalytic activity. Utilizing many-body perturbation theory based on
Green’s function formalism can provide accurate descriptions of excited state
properties such as accurate bandgap, exciton energy, and absorption spectra.
Our previous studies reported strong exciton binding energy,[15] and have
examined the effects of PCN microstructures, such as corrugation, degree of
condensation, and stacking, focusing on their thermodynamic stability,
electronic structure, and optical properties.[16] We particularly noted that
the interlayer interactions in PCNs play a significant role in both
thermodynamic and electronic properties.[17] Herein, we perform first-
principles calculations based on GW and the Bethe–Salpeter equation (GW-
BSE),[18] to elucidate the crucial link between the various structural motifs
in PCNs and the corresponding optical properties, formation of bright and dark
excitons, exciton stability and lifetime, as the resulting factors governing
the photocatalytic activity. We demonstrate that the strongly localized
electrons in PCNs induce active interlayer transitions due to the stacked
structure of PCNs. Furthermore, we show that the interactions between
heptazine units vary with the microstructure of PCNs, affecting the formation
and the lifetimes of the dark excitons more significantly than of the bright
excitons. Our findings thus establish the pronounced structure-related
variations of the dark excitons as a key factor determining the experimentally
encountered optical properties and photocatalytic performance of different
types of PCNs.

Method.—All ab initio calculations were performed using VASP (6.4.1), which
uses the projector augmented wave potentials (PAWs).[19] We considered PCN
structures optimized by the local density approximation (LDA) and generalized
gradient approximation (GGA) functionals, respectively, to compare the
description of the highly-localized $\pi$-electrons within the heptazine
units.[17] The Perdew–Zunger parametrization of Ceperley–Alder Monte-Carlo
correlation (CA) functional[20, 21] with the many-body dispersion energy
method (MBD@rsSCS)[22, 23] is used within LDA, while PBE-D3 is adopted for the
GGA calculations.[24, 25] To describe the single-particle excitations of the
PCN structure, we conduct a single-shot GW calculation to avoid bandgap
overestimation.[26, 27] In brief, the GW approximation neglects the vertex
functions in Hedin’s formalisms, and the self-energy ($\Sigma$) is given as
the product of the Green function ($G$) with the screened self interactions
($W$). Then, the quasi-particle energy is given by:

$E^{\mathrm{QP}}_{n\textbf{k}}=E_{n\textbf{k}}+Z_{n\textbf{k}}\langle\psi_{n\textbf{k}}|\Sigma-
V_{\mathrm{xc}}|\psi_{n\mathrm{\textbf{k}}}\rangle,$ (1)

where k and $n$ are the wave vector and band index, $Z_{n\textbf{k}}$ is the
renormalization function obtained from the self-energy with DFT
eigenvalues.[28] In the $G_{0}W_{0}$ calculations, we include 256 bands for
the 2D PCN-structure and 512 bands for 3D PCN-structure, respectively. For the
2D and 3D PCN structures, $\Gamma$-centered $2\times 2\times 1$ and $2\times
2\times 2$ k-space meshes are used, respectively. The quasi-particle energy is
updated by using BSE to include e–h interactions within the Tamm–Dancoff
approximation (TDA)[29], given by:

$(E^{\mathrm{QP}}_{c\textbf{k}}-E^{\mathrm{QP}}_{v\textbf{k}})A_{vc\textbf{k}}+\sum_{v^{\prime}c^{\prime}\textbf{k}^{\prime}}\langle
vc\textbf{k}|K_{e-h}|v^{\prime}c^{\prime}\textbf{k}^{\prime}\rangle
A_{v^{\prime}c^{\prime}\textbf{k}^{\prime}}=\Omega A_{vc\textbf{k}},$ (2)

where $c$ and $v$ are indices of conduction and valance bands, respectively,
$A_{vc\textbf{k}}$ are $e$–$h$ coupling coefficients, $\Omega$ are the BSE
eigenvalues, and $K_{e-h}$ is the $e$–$h$ interaction kernel.[30, 31] For the
BSE calculations we considered 32 eigenvalues. Then, the independent quasi-
particle (IP) energy considering electron–electron (e–e) interactions is
obtained as GW@IP, and the BSE is used to include e–h interactions, finally
denoted as GW@BSE. The real and the imaginary parts of the macroscopic
dielectric functions are computed including local field effect.[32] From the
calculated oscillator strengths, the bright excitons are selected having the
5$\%$ of the highest intensity, and the rest are classified as the dark
states. We modified the VASP source code to obtain the transitional dipole
momentum kernel ($\mu_{\mathrm{S}}$) to calculate the radiative decay rate of
the exciton state $\mathrm{S}$ ($\gamma_{\mathrm{S}}$) at wavevector
$\textbf{Q}=0$ using the equation, given by:[33]

$\gamma_{\mathrm{S}}(0)=\tau_{\mathrm{S}}(0)^{-1}=\frac{8\pi
e^{2}E_{\mathrm{S}}(0)}{\hbar^{2}c}\frac{\mu^{2}_{\mathrm{S}}}{A},$ (3)

where $E_{\mathrm{S}}$ is the exciton energy at Q, $A$ is the area of the unit
cells, $c$ is the speed of light, and $\tau_{\mathrm{S}}$ is the exciton
lifetime. Furthermore, we calculate the effective radiative lifetime
$\langle\tau_{\mathrm{eff}}\rangle$ at temperature ($T$) assuming a
neglectable exciton momentum in PCN structures at high symmetry points.[34]
Therefore, the effective radiative lifetime using the Boltzmann average of
$\tau_{\mathrm{S}}$ is given by:

$\langle\tau_{\mathrm{eff}}\rangle^{-1}=\frac{\sum_{\mathrm{S}}\tau_{\mathrm{S}}^{-1}e^{{-E_{\mathrm{S}}(0)}\mathbin{/}{k_{\mathrm{B}}T}}}{\sum_{\mathrm{S}}e^{{-E_{\mathrm{S}}(0)}\mathbin{/}{k_{\mathrm{B}}T}}}$
(4)

Figure 1: Structures of polymeric carbon nitrides.

Models.—Figure 1 illustrates the different PCN structures considered in this
work. These models encompass variations in the degree of poly-condensation of
heptazine units, corrugations, and stacking configurations.[16] Increased
poly-condensation leads to higher carbon and nitrogen contents, but a reduced
hydrogen ratio. The planar-gCN (p-gCN) is known to be energetically less
stable compared to corrugated-gCN (c-gCN), which represents a local minimum in
the energy landscape.[35] PCN structures, composed of these heptazine units,
typically exhibit highly-localized $\pi$ electrons. Thus, we primarily present
results for optimized geometries using CA (LDA) functionals. These functionals
provide an efficient description of structures with highly-localized
electrons, considered as key properties. To address the limitations of the
MBD@rsSCS dispersion correction, which depends on the exchange–correlation
energy from the LDA functionals, we also included results obtained with the
PBE-D3 xc-functional. This approach offers a better description of interlayer
properties. Results of the optimized geometry with PBE-D3 are available in the
supplementary information (SI) for comparison (Figs. S9–S20). The lattice
parameters of the optimized geometries obtained with both functionals are
summarized in Table S1.

Figure 2: Imaginary part of dielectric functions. The solid and the dashed
lines are obtained by the GW@BSE and the GW@IP respectively.

Optical spectra.—The sole use of density functional theory (DFT) to accurately
describe optical properties has numerous limitations.[26, 32] To address these
challenges, we use the BSE to address the energy originating from e–h
interactions on top of the quasi-particle energy derived from $GW$
calculations. This $GW$-based quasi-particle energy incorporates precisely
screened e–e interactions, thereby enhancing the accuracy of single excitation
property predictions. Notably, the quasi-particle energy adjusted by the BSE
correction typically exhibits a red shift compared to those predicted by the
IP approximation, which omits e–h interactions. By analyzing the imaginary
part of the macroscopic dielectric functions from both GW@BSE and GW@IP
calculations, we can estimate the exciton binding energy in 2D and 3D PCN
structures, as illustrated in Figure 2. Spatially-projected absorption
coefficients reveal distinct patterns: 2D PCN structures show strong exciton
binding energy (> 2 eV).[15] However, this significant exciton binding energy
diminishes to less than 1 eV in 3D stacked structures. Moreover, the
transition from a 2D monolayer to a stacked 3D configuration alters the
absorption edges and reduces the optical anisotropy observed in PHI-2D and
c-gCN-2D structures. Since the optical anisotropy is attributed to the in-
plane joint patterns of the heptazine units formed during poly-condensation,
the reduced optical anisotropy implies that the fractions of the 2D PCN micro-
structure can indeed interact with each other throughout the 3D assembly. In
brief, this means that diverse forms of PCN fragments within the PCN structure
interact electronically and optically through their stacking configurations,
regardless of size or shape. Our findings provide clear evidence of strong
interlayer interactions, resulting in dramatically decreased exciton binding
energy and changes in absorption patterns including altered optical anisotropy
and absorption edges.

Figure 3: Illustration of the calculated absorption spectra for 2D and 3D PCN
models along averaged spatial directions. The oscillator strengths in relation
to wavelength are presented in red.

To better elucidate the origins of the absorption features near the absorption
edge, in Figure 3 we depict the oscillator strengths and averaged absorption
spectra along spatial directions for PCN 2D and 3D structures obtained from
GW@BSE calculations. Firstly, we observed that each 2D structure of PCNs
exhibits distinct absorption spectra patterns. When these structures are
stacked to form a 3D structure, additional peaks arise due to interlayer
interactions. These emergent peaks are more pronounced in flat structures
(melon and p-gCN), where interlayer interactions result in more absorption
patterns. In contrast, in buckled structures (PHI and c-gCN), small absorption
peaks appear due to interlayer interactions with the red-shifted 2D peaks. The
strong intensity of absorption peaks near 200 nm in 3D PCNs is due to the
stacking of infinite 2D models, indicating that finite-sized ribbon structures
exhibit a reasonable decrease in the intensity of these absorption peaks[17]
The peaks appearing beyond 400 nm originate from the interactions between
layers and the in-plane $\pi$–$\pi^{*}$ transitions, which will be discussed
below. The absorption edges of the different PCN structures are obtained as
2.93 eV for melon-3D, 2.60 eV for PHI-3D, 2.62 eV for c-gCN-3D, and 1.69 eV
for p-gCN-3D. These PCN structures and bandgaps are completely in line with
the experimentally observed bandgaps of PCNs, which range from 2.6 to 2.8
eV.[5, 36, 37, 38] Especially, the frequently observed long absorption tails
can not only be attributed to thermal vibrational motion, but can have a
distinct structural origin, primarily from the graphitic (i.e. fully
condensed) domains.[39]

Figure 4: Representaton of eigenvalues (BSE and IP) and coupling coefficients
(radius of circles) of bright states for both 2D and 3D PCN structures,
corresponding to the absorption edges (top) and interlayer interactions
(bottom).

Exciton formation.—Figure 3 demonstrates the oscillator strengths that,
according to the Franck–Condon principle, represent the probability of
electronic transitions at each energy level. Cases of high oscillator
strengths with relatively low absorption peaks occur in the absorption
spectrum due to consideration of the dipole momentum and selection rules.
Therefore, by comparing absorption spectra with oscillator strengths, we can
predict the types of electronic transitions. Notably, most electronic
transitions near the absorption edge in PCN 3D structures are combined with
the dark states. Our results from the BSE calculations provide the coupling
coefficients corresponding to transitions between energy levels at each high
symmetry point, as shown in Figure 4. The coupling constants of the other
lowest 32 excited states are presented in the Supplemental Material. The
coupling coefficients (radius of circles) at the band edges for each PCN 3D
structure are displayed at the top line of the figure, and the coupling
coefficients that are most pronounced for interlayer interactions are
displayed at the bottom line. At the absorption edges, stable excitons are
related to transitions between the band edges in melon, c-gCN, and p-gCN
structures while relatively more degenerate excitons are formed in the PHI
structure. The coupling constants at the bottom line of Figure 4 represent the
exciton formation at the $Z$-point in each structure, indicating a direction
toward the other layer from the $\Gamma$-point. These interlayer transitions
are identified as the electron transitions from the sub-levels of the valence
band to near the conduction band edge, indicating that these transitions
involve different $\pi$ electrons than those associated with band edge
transitions. Our previous study found that absorption peaks of PCN structure
can be sorted as in-plane $\pi$–$\pi^{*}$ transitions and $\pi$–$\pi^{*}$
interlayer transitions.[17] When comparing the relative stability of excitons
formed in the PCN structures at the interlayer, the c-gCN structure exhibits
the most stable exciton with an energy gap of 2.96 eV. Melon and PHI also show
stabilization of interlayer excitons at 3.11 eV and 2.97 eV, respectively,
each showing roughly a 0.3 eV difference from their absorption edges. The
differences in exciton stability at the interlayer are determined by the
degree of overlap between orbitals of different layers, specifically being
regulated by the magnitude of the transition momentum off-diagonal components.
These off-diagonal components turn out to be the degree of interaction of the
nitrogen non-bonding ($n$) electrons, which impact the $\pi$ electrons at the
interlayers. This explains how the interlayer distances are related to the
configurations of PCN structures. This is more clearly understood when
comparing the coupling constant plots of LDA-optimized geometries with
different interlayer distances (see Table S1) to those of GGA-optimized
geometry results (Figs. S11, S13–S20). Furthermore, our previous study found
that the thermodynamic contributions from vibrational motion due to the
flexible layered structure of PCNs are non-negligible.[16] Thus, we expect
that enhanced interlayer interactions may be evident due to interlayer
vibration modes.

Figure 5: Calculated radiative exciton lifetimes of both 2D and 3D PCN
structures. Bright states are denoted in blue, while dark states are
represented in red. Their respective effective lifetimes at 300 K are
indicated by dashed lines.

Prolonged exciton lifetime and photocatalytic activity.—We have calculated the
exciton lifetimes and effective lifetimes at 300 K based on the energy
required for each excited state, transition probability, and transition dipole
momentum. The obtained behavior is shown in Figure 5 and summarized in Table
1. Here, we define the energy states with the top 5$\%$ oscillator strengths
as bright states (blue) and the rest as dark states (red) (see Method
section). Firstly, the effective lifetimes of bright excitons in 3D PCN
structures are slightly increased compared to those in 2D structures. This
suggests that most bright excitons in PCN structures are formed by the direct
in-plane transitions of strongly localized $\pi$ electrons and the interlayer
$\pi$ transitions. Therefore, the stacking configuration helps stabilize these
bright excitons. In contrast, the dark excitons formed in 3D structures are
destabilized by interlayer interactions. Most dark excitons are related to in-
plane $n$–$\pi^{*}$ or distorted $\pi$–$\pi^{*}$ transitions and interlayer
interactions cannot sufficiently stabilize these states due to the lack of
interaction. Notably, the effective lifetime of dark excitons increases
dramatically in graphitic structures (c-gCN and p-gCN). This indicates that a
new form of excitons, different from those in melon and PHI structures, exists
in gCN. We propose that these long-lived excitons are closely related to the
trapped excitons reported in the literature[12]. Furthermore, we suggest that
the calculated lifetimes of dark excitons shed light on the photo-charging
phenomenon observed in ionic-PHI.[8, 9, 10, 13, 14, 40] Notably, the
photocharging phenomenon persists in the proton-exchanged structure (H-PHI),
indicating that the phenomenon is more attributed to the high symmetry
crystalline regions rather than the influence of existing cations. However,
the principle and the nature of electron accumulation in PHIs have not been
elucidated so far. Our results demonstrate that the extended lifetime of
excitons is observed in both 3D stacked and 2D graphitic structures (c-gCN and
p-gCN), indicating that these long lifetimes are primarily due to structural
features rather than interlayer interactions. In contrast, the PHI structures
exhibit the shortest dark exciton lifetime even shorter than those of melon
structures. This might explain the general experimental observation that the
photo(electro)catalytic activity and photoinduced electron accumulation in
ionic (PHI-type) PCNs are enabled only in the presence of effective electron
donors, such as alcohols, that can quickly extract the hole from the short-
lived excitons in PHI-type PCNs.[8, 9, 10, 13, 14, 41]

Figure 6: (left) Electron density of electron hole and the excited electrons
and (right) the proposed mechanisms of photoexcitation process for (a) PHI and
(b) c-gCN-3D structures, where $\pi_{\mathrm{dist}}$ and
$\pi^{*}_{\mathrm{dist}}$ are the energy levels of distorted $\pi$ electrons,
$\pi_{\mathrm{inter}}$ is the $\pi^{*}$ state at the other layer, and
$\mathrm{E_{b}}$ is the exciton binding energy, respectively.

In order to further elucidate the nature of these dark states, Figure 6 shows
the electronic density corresponding to the electron and the electron hole for
the dark excitons of the longest lifetime in the PHI and the c-gCN structure.
It clearly shows that these dark excitons are different in PHI and c-gCN
structures. This suggests that the repulsive force between heptazine units
causes a distorted orientation of the $n$ or $\pi$ orbitals. The varying
degrees of this distortion control the selection rules, resulting in allowed
transitions in PHI and forbidden transitions in c-gCN for dark excitons. This
forbidden in-plane transition further hinders the interlayer interaction due
to the lack of orbital overlaps. Moreover, this distortion also affects the
energy levels of $\pi$ states, resulting in a larger energy gap between
$\pi^{*}$ and $\pi^{*}_{\mathrm{dist}}$ in c-gCN than in the case of PHI.
Therefore, the larger distortion of the orbitals can exhibit prolonged exciton
lifetimes and lower energy gaps for the dark states. (See Table 1) In the case
of melon, the presence of hydrogen bonding slightly alters one of the $\pi$
electrons, resulting in dark states with lifetimes similar to but slightly
longer than those in PHI.[17] In p-gCN, the strongly forced forbidden
$n$–$\pi^{*}$ transitions due to the high symmetry lead to the detrimental
effect of dark states, similar to those in c-gCN.

Table 1: Bandgaps of 2D and 3D PCN structures (optimized using CA functionals) were calculated via independent particle approximations ($G_{0}W_{0}$@IP) and BSE ($G_{0}W_{0}$@BSE) methods, alongside the effective exciton lifetime at 300 K. (D) and (I) denote the direct and indirect bandgaps, respectively. | Structure |  $G_{0}W_{0}$@IP (eV) |  $G_{0}W_{0}$@BSE (eV) |  | Exiton lifetime
---
<$\tau_{\mathrm{300K}}$>
bright | dark
2D | melon | 5.50 (D) | 3.31 | 1.96 ps | 8.70 ns
PHI | 5.83 (I) | 3.48 | 0.78 ps | 1.65 ns
c-gCN | 4.48 (D) | 2.62 | 8.36 ps |  141.00 $\mu$s
p-gCN | 4.26 (I) | 1.95 | 0.20 ps | 28.73 ms
3D | melon | 3.46 (I) | 2.93 | 6.96 ps | 2.72 ns
PHI | 3.67 (D) | 2.60 | 8.77 ps | 277.34 ps
c-gCN | 3.12 (D) | 2.62 | 8.36 ps | 141.00 $\mu$s
p-gCN | 2.15 (I) | 1.69 | 26.04 ps | 1.92 ms

Furthermore, these findings shed light on the experimentally observed
detrimental effects of deep charge trap states on photocatalytic activity in
both conventional and ionic (PHI) carbon nitrides.[12, 13] The significant
distortion of $\pi$ or $n$ orbitals can create deeply trapped dark excitons
compared to the bright states (Fig. 6b). Due to their longer exciton
lifetimes, these trapped excitons are less affected by hole-quenching rates
and can effectively accumulate electrons. However, these electrons, being
spatially distorted and trapped, can not directly contribute to the interlayer
interactions for the carrier transport, resulting in significantly reduced
quantum yields due to their strongly diminished reductive potential, as
reported in the literature.[12] Moreover, the presence of these trapped
electrons might shield interlayer charge transport of the bright states, which
could further promote the recombination of bright excitons. The presence of
these deeply trapped electrons also suggests that they can be used as a
descriptor to indicate whether conventional or ionic (PHI) carbon nitrides
obtained from the synthesis process contain graphitic domains. On the other
hand, in PHI structures (Fig. 6a), the dark states of the less affected $\pi$
electrons exhibit similar energy levels to the bright states, forming shallow
trapped states. Due to this similarity, these dark states can have lifetimes
as short as the bright states. Although a fast electron donor rate is required
for hole-quenching due to these short lifetimes, once the hole of the dark
exciton is filled, the nearly undistorted orientation of the $\pi$ electrons
enables efficient interlayer carrier transport, leading to effective surface
catalytic reactions. We assume that these dark-state-induced phenomena
(enhanced photocatalytic activity or time-delayed photocatalysis) in PHI can
be more pronounced in the presence of cations, as the stabilization of these
excess electrons is also a key factor in preventing recombination.[8, 9, 40]
We note that, although the model structures used in our simulations cannot
fully account for the complex and still not fully resolved structure of real-
life PCN materials, our results are in excellent agreement with experimental
observations, and elucidate the crucial role of dark states in photocatalytic
activity of PCN materials.

Conclusion.—Our study elucidates the principles underlying the exciton
formation, transitions, and lifetimes within PCN structures. The results
establish a valuable link between the structural and optical properties of PCN
materials, which is a key prerequisite for rational tuning of their electronic
structures for optimal performance. Utilizing GW@BSE calculations, we find the
decisive role of dark excitons for effective photocatalytic performance. The
structure-dependent mechanisms of dark exciton formation elucidate the
electron accumulation phenomena and subsequent surface reactions in different
types of carbon nitrides. Our study thus establishes a valuable theoretical
framework for further spectroscopic studies, and paves the way for the
knowledge-driven design of PCN-based materials with tuned properties.

This work was funded by the Deutsche Forschungsgemeinschaft (DFG – German
Research Foundation) through TRR-234 CataLight (project no. 364549901) as well
as JA 1072/27-1 and BE 5102/5-1 (project no. 428764269). R.B. acknowledges
funding from the European Union’s Horizon Europe programme for research and
innovation under grant agreement No. 101122061 (SUNGATE). The authors
acknowledge support by the state of Baden-Württemberg through bwHPC and the
German Research Foundation (DFG) through grant no INST 40/575-1 FUGG (JUSTUS 2
cluster). C.I. acknowledges the German Academic Exchange Service (DAAD, Ref.
No. 91676720).

## References

  * Kessler  _et al._ [2017] F. K. Kessler, Y. Zheng, D. Schwarz, C. Merschjann, W. Schnick, X. Wang, and M. J. Bojdys, Functional carbon nitride materials—design strategies for electrochemical devices, Nat. Rev. Mater. 2, 1 (2017).
  * Podjaski and Lotsch [2021] F. Podjaski and B. V. Lotsch, Optoelectronics meets optoionics: light storing carbon nitrides and beyond, Adv. Energy Mater. 11, 2003049 (2021).
  * Gouder  _et al._ [2023] A. Gouder, F. Podjaski, A. Jiménez-Solano, J. Kröger, Y. Wang, and B. V. Lotsch, An integrated solar battery based on a charge storing 2d carbon nitride, Energy Environ. Sci. 16, 1520 (2023).
  * Liu  _et al._ [2015] J. Liu, Y. Liu, N. Liu, Y. Han, X. Zhang, H. Huang, Y. Lifshitz, S.-T. Lee, J. Zhong, and Z. Kang, Metal-free efficient photocatalyst for stable visible water splitting via a two-electron pathway, Science 347, 970 (2015).
  * Cao  _et al._ [2015] S. Cao, J. Low, J. Yu, and M. Jaroniec, Polymeric photocatalysts based on graphitic carbon nitride, Adv. Mater. 27, 2150 (2015).
  * Cheng  _et al._ [2021] L. Cheng, H. Zhang, X. Li, J. Fan, and Q. Xiang, Carbon–graphitic carbon nitride hybrids for heterogeneous photocatalysis, Small 17, 2005231 (2021).
  * Chen  _et al._ [2017] Z. Chen, A. Savateev, S. Pronkin, V. Papaefthimiou, C. Wolff, M. G. Willinger, E. Willinger, D. Neher, M. Antonietti, and D. Dontsova, “the easier the better” preparation of efficient photocatalysts—metastable poly (heptazine imide) salts, Adv. Mater. 29, 1700555 (2017).
  * Lau  _et al._ [2017] V. W.-h. Lau, D. Klose, H. Kasap, F. Podjaski, M.-C. Pignié, E. Reisner, G. Jeschke, and B. V. Lotsch, Dark photocatalysis: storage of solar energy in carbon nitride for time-delayed hydrogen generation, Angew. Chem., Int. Ed. 129, 525 (2017).
  * Adler  _et al._ [2021] C. Adler, S. Selim, I. Krivtsov, C. Li, D. Mitoraj, B. Dietzek, J. R. Durrant, and R. Beranek, Photodoping and fast charge extraction in ionic carbon nitride photoanodes, Adv. Funct. Mater. 31, 2105369 (2021).
  * Kasap  _et al._ [2016] H. Kasap, C. A. Caputo, B. C. Martindale, R. Godin, V. W.-h. Lau, B. V. Lotsch, J. R. Durrant, and E. Reisner, Solar-driven reduction of aqueous protons coupled to selective alcohol oxidation with a carbon nitride–molecular ni catalyst system, J. Am. Chem. Soc. 138, 9183 (2016).
  * Merschjann  _et al._ [2015] C. Merschjann, S. Tschierlei, T. Tyborski, K. Kailasam, S. Orthmann, D. Hollmann, T. Schedel-Niedrig, A. Thomas, and S. Lochbrunner, Complementing graphenes: 1d interplanar charge transport in polymeric graphitic carbon nitrides., Adv. Mater. 27, 7993 (2015).
  * Godin  _et al._ [2017] R. Godin, Y. Wang, M. A. Zwijnenburg, J. Tang, and J. R. Durrant, Time-resolved spectroscopic investigation of charge trapping in carbon nitrides photocatalysts for hydrogen generation, J. Am. Chem. Soc. 139, 5216 (2017).
  * Yang  _et al._ [2019] W. Yang, R. Godin, H. Kasap, B. Moss, Y. Dong, S. A. Hillman, L. Steier, E. Reisner, and J. R. Durrant, Electron accumulation induces efficiency bottleneck for hydrogen production in carbon nitride photocatalysts, J. Am. Chem. Soc. 141, 11219 (2019).
  * Li  _et al._ [2021] C. Li, C. Adler, I. Krivtsov, D. Mitoraj, R. Leiter, U. Kaiser, R. Beranek, and B. Dietzek, Ultrafast anisotropic exciton dynamics in a water-soluble ionic carbon nitride photocatalyst, Chem. Commun. 57, 10739 (2021).
  * Wei and Jacob [2013] W. Wei and T. Jacob, Strong excitonic effects in the optical properties of graphitic carbon nitride g-C3N4 from first principles, Phys. Rev. B 87, 1 (2013).
  * Im  _et al._ [2023] C. Im, B. Kirchhoff, I. Krivtsov, D. Mitoraj, R. Beranek, and T. Jacob, Structure and optical properties of polymeric carbon nitrides from atomistic simulations, Chem. Mater. 35, 1547 (2023).
  * Im  _et al._ [2024] C. Im, B. Kirchhoff, D. Mitoraj, I. Krivtsov, A. Farkas, R. Beránek, and T. Jacob, Unraveling the optical signatures of polymeric carbon nitrides: Insights into stacking-induced excitonic transitions, arXiv preprint arXiv:2403.13685 (2024).
  * Rohlfing and Louie [1998] M. Rohlfing and S. G. Louie, Electron-hole excitations in semiconductors and insulators, Phys. Rev. Lett. 81, 2312 (1998).
  * Hafner [2008] J. Hafner, Ab-initio simulations of materials using vasp: Density-functional theory and beyond, J. Comput. Chem. 29, 2044 (2008).
  * Ceperley and Alder [1980] D. M. Ceperley and B. J. Alder, Ground state of the electron gas by a stochastic method, Phys. Rev. Lett. 45, 566 (1980).
  * Perdew and Zunger [1981] J. P. Perdew and A. Zunger, Self-interaction correction to density-functional approximations for many-electron systems, Phys. Rev. B 23, 5048 (1981).
  * Tkatchenko  _et al._ [2012] A. Tkatchenko, R. A. DiStasio, R. Car, and M. Scheffler, Accurate and efficient method for many-body van der waals interactions, Phys. Rev. Lett. 108, 236402 (2012).
  * Ambrosetti  _et al._ [2014] A. Ambrosetti, A. M. Reilly, J. DiStasio, Robert A., and A. Tkatchenko, Long-range correlation energy calculated from coupled atomic response functions, J. Chem. Phys. 140, 18A508 (2014).
  * Perdew  _et al._ [1996] J. P. Perdew, K. Burke, and M. Ernzerhof, Generalized gradient approximation made simple, Phys. Rev. Lett. 77, 3865 (1996).
  * Grimme  _et al._ [2010] S. Grimme, J. Antony, S. Ehrlich, and H. Krieg, A consistent and accurate ab initio parametrization of density functional dispersion correction (dft-d) for the 94 elements h-pu, J. Chem. Phys. 132 (2010).
  * Hedin [1965] L. Hedin, New method for calculating the one-particle green’s function with application to the electron-gas problem, Phys. Rev. 139, A796 (1965).
  * van Schilfgaarde  _et al._ [2006] M. van Schilfgaarde, T. Kotani, and S. Faleev, Quasiparticle self-consistent $gw$ theory, Phys. Rev. Lett. 96, 226402 (2006).
  * Ljungberg  _et al._ [2015] M. P. Ljungberg, P. Koval, F. Ferrari, D. Foerster, and D. Sanchez-Portal, Cubic-scaling iterative solution of the bethe-salpeter equation for finite systems, Phys. Rev. B 92, 075422 (2015).
  * Sander  _et al._ [2015] T. Sander, E. Maggio, and G. Kresse, Beyond the tamm-dancoff approximation for extended systems using exact diagonalization, Phys. Rev. B 92, 045209 (2015).
  * Leng  _et al._ [2016] X. Leng, F. Jin, M. Wei, and Y. Ma, Gw method and bethe–salpeter equation for calculating electronic excitations, Wiley Interdiscip. Rev. Comput. Mol. Sci. 6, 532 (2016).
  * Rohlfing and Louie [2000] M. Rohlfing and S. G. Louie, Electron-hole excitations and optical spectra from first principles, Phys. Rev. B 62, 4927 (2000).
  * Albrecht  _et al._ [1998] S. Albrecht, L. Reining, R. Del Sole, and G. Onida, Ab initio calculation of excitonic effects in the optical spectra of semiconductors, Phys. Rev. Lett. 80, 4510 (1998).
  * Palummo  _et al._ [2015] M. Palummo, M. Bernardi, and J. C. Grossman, Exciton radiative lifetimes in two-dimensional transition metal dichalcogenides, Nano Lett. 15, 2794 (2015).
  * Perebeinos  _et al._ [2005] V. Perebeinos, J. Tersoff, and P. Avouris, Radiative lifetime of excitons in carbon nanotubes, Nano Lett. 5, 2495 (2005).
  * Gracia and Kroll [2009] J. Gracia and P. Kroll, Corrugated layered heptazine-based carbon nitride: the lowest energy modifications of c 3 n 4 ground state, J. Mater. Chem. 19, 3013 (2009).
  * Butchosa  _et al._ [2014] C. Butchosa, P. Guiglion, and M. A. Zwijnenburg, Carbon nitride photocatalysts for water splitting: a computational perspective, J. Phys. Chem. C 118, 24833 (2014).
  * Zhang  _et al._ [2017] G. Zhang, G. Li, Z.-A. Lan, L. Lin, A. Savateev, T. Heil, S. Zafeiratos, X. Wang, and M. Antonietti, Optimizing optical absorption, exciton dissociation, and charge transfer of a polymeric carbon nitride with ultrahigh solar hydrogen production activity, Angew. Chem., Int. Ed. 129, 13630 (2017).
  * Wang  _et al._ [2019] Y. Wang, P. Du, H. Pan, L. Fu, Y. Zhang, J. Chen, Y. Du, N. Tang, and G. Liu, Increasing solar absorption of atomically thin 2d carbon nitride sheets for enhanced visible-light photocatalysis, Adv. Mater. 31, 1807540 (2019).
  * Wang  _et al._ [2012] X. Wang, S. Blechert, and M. Antonietti, Polymeric graphitic carbon nitride for heterogeneous photocatalysis, ACS Catal. 2, 1596 (2012).
  * Schlomberg  _et al._ [2019] H. Schlomberg, J. Kröger, G. Savasci, M. W. Terban, S. Bette, I. Moudrakovski, V. Duppel, F. Podjaski, R. Siegel, J. Senker, _et al._ , Structural insights into poly (heptazine imides) a light-storing carbon nitride material for dark photocatalysis, Chem. Mater. 31, 7478 (2019).
  * Pulignani  _et al._ [2022] C. Pulignani, C. A. Mesa, S. A. Hillman, T. Uekert, S. Giménez, J. R. Durrant, and E. Reisner, Rational design of carbon nitride photoelectrodes with high activity toward organic oxidations, Angew. Chem., Int. Ed. 134, e202211587 (2022).

##  I Supplementary data

Table S1: The lattice parameters of the PCN structures obtained by the CAPZ and PBE functionals. |  |  | Lattice parameters
---
(CA) (Å)
|  | Lattice parameters
---
(PBE) (Å)
a | b | c |  | a | b | c
2D | melon | 16.45 | 12.57 | 12 |  | 16.42 | 12.62 | 12
PHI | 22.29 | 12.82 | 12 |  | 21.58 | 12.46 | 12
c-gCN | 13.28 | 11.8 | 12 |  | 13.35 | 11.88 | 12
p-gCN | 14.15 | 14.15 | 12 |  | 14.23 | 14.23 | 12
3D | melon | 16.29 | 12.45 | 5.91 |  | 16.42 | 12.62 | 6.53
PHI | 21.71 | 12.55 | 5.6 |  | 21.58 | 12.46 | 6.51
c-gCN | 13.14 | 11.81 | 6.5 |  | 13.29 | 11.84 | 7.35
p-gCN | 14.13 | 14.13 | 6.04 |  | 14.23 | 14.23 | 6.66
Table S2: Bandgaps of 2D and 3D PCN structures (optimized using PBE functionals) were calculated via independent particle approximations ($G_{0}W_{0}$@IP) and BSE ($G_{0}W_{0}$@BSE) methods, alongside the effective exciton lifetime at 300 K. The (D) and (I) denote the direct and the indirect bandgap, respectively. | Structure |  $G_{0}W_{0}$@IP (eV) |  $G_{0}W_{0}$@BSE (eV) |  | Exiton lifetime
---
<$\tau_{\mathrm{300K}}$>
bright | dark
2D | melon | 5.44 (D) | 3.24 | 1.87 ps |  196.36 ps
PHI | 5.26 (I) | 2.03 | 5.35 ps | 633.69 ps
c-gCN | 4.47 (D) | 2.63 | 5.18 ps | 63.82 ns
p-gCN | 4.27 (D) | 1.97 | 0.16 ps | 747.08 $\mu$s
3D | melon | 3.82 (D) | 3.22 | 4.61 ps | 23.29 ns
PHI | 4.25 (I) | 2.53 | 20.34 ps | 289.93 ps
c-gCN | 3.39 (D) | 2.67 | 3.32 ps | 2.88 ns
p-gCN | 2.87 (I) | 2.00 | 8.24 ps | 31.49 ms

Figure S1: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of melon-2D structure optimized CA (LDA) functionals.
Figure S2: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of melon-3D structure optimized CA (LDA) functionals.
Figure S3: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of PHI-2D structure optimized CA (LDA) functionals.
Figure S4: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of PHI-3D structure optimized CA (LDA) functionals.
Figure S5: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of c-gCN-2D structure optimized CA (LDA) functionals.
Figure S6: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of c-gCN-3D structure optimized CA (LDA) functionals.
Figure S7: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of p-gCN-2D structure optimized CA (LDA) functionals.
Figure S8: Illustration of the $e$–$h$ coupling constants corresponding to the
32 excitation energy of p-gCN-3D structure optimized CA (LDA) functionals.

Figure S9: Imaginary part of dielectric functions of the PCN structure
optimized with PBE-D3 functionals. The solid and the dashed lines are obtained
by the GW@BSE and the GW@IP respectively.

Figure S10: The illustration of the calculated absorption spectra for 2D and
3D PCN models, optimized PBE functionals, along averaged spatial directions.
The oscillator strengths in relation to wavelength are presented in red.

Figure S11: Representaton of eigenvalues (BSE and IP) and coupling
coefficients (radius of circles) of bright states for both 2D and 3D PCN
structures, optimized by PBE-D3 functionals, corresponding to the absorption
edges (top) and interlayer interactions (bottom).

Figure S12: Calculated radiative exciton lifetime of both 2D and 3D PCN
structures optimized by PBE-D3 functionals. Bright states are denoted in blue,
while dark states are represented in red. Their respective effective lifetime
at 300 K are indicated by dashed lines.

Figure S13: Illustration of the $e$–$h$ coupling constants corresponding to
the 32 excitation energy of melon-2D structure optimized PBE (GGA)
functionals.

Figure S14: Illustration of the $e$–$h$ coupling constants corresponding to
the 32 excitation energy of melon-3D structure optimized PBE (GGA)
functionals.

Figure S15: Illustration of the $e$–$h$ coupling constants corresponding to
the 32 excitation energy of PHI-2D structure optimized PBE (GGA) functionals.

Figure S16: Illustration of the $e$–$h$ coupling constants corresponding to
the 32 excitation energy of PHI-3D structure optimized PBE (GGA) functionals.

Figure S17: Illustration of the $e$–$h$ coupling constants corresponding to
the 32 excitation energy of c-gCN-2D structure optimized PBE (GGA)
functionals.

Figure S18: Illustration of the $e$–$h$ coupling constants corresponding to
the 32 excitation energy of c-gCN-3D structure optimized PBE (GGA)
functionals. Figure S19: Illustration of the $e$–$h$ coupling constants
corresponding to the 32 excitation energy of p-gCN-2D structure optimized PBE
(GGA) functionals. Figure S20: Illustration of the $e$–$h$ coupling constants
corresponding to the 32 excitation energy of p-gCN-3D structure optimized PBE
(GGA) functionals.


# Small Black String Thermodynamics

Jyotish Kumar  Sudhaker Upadhyay111Corresponding author222Visiting Associate,
IUCAA Pune, Maharashtra-411007, India and Himanshu Kumar Sudhanshu

###### Abstract

We consider a cylindrically symmetric solution for the field equations of
Einstein-Hilbert action with a negative cosmological constant in four
dimensions. The small statistical fluctuation in the equilibrium
thermodynamics of the black string solution is investigated. The small black
string under the influence of small statistical fluctuation also follows the
first law of thermodynamics. The behaviour of equation of states for black
string changes significantly due to the fluctuation. The fluctuation causes
instability to the small-sized black string only. Assuming the black string is
fluid, the compressibility of the black string is inversely proportional to
the fluctuation parameter.

##  1 Literature survey and motivation

The radiation and evaporation of the black holes [1, 2] merge three areas of
physics: quantum theory, general relativity and thermodynamics. Even though
black holes cannot be seen directly, there is no doubt regarding their
existence. It is found that the entropy of black holes depends on the
horizon’s area. Nowadays, it is confirmed that the entropy of large black
holes is directly proportional to the horizon’s area [3, 4]. However, for the
entropy of small black holes, the small statistical fluctuations around the
equilibrium play an essential role [5, 6]. This is well-known that the
leading-order correction to the entropy is always logarithmic due to small
statistical fluctuations around the equilibrium. Recently, thermodynamics of
various small black holes under the influence of small statistical
fluctuations have been investigated extensively [7, 8, 9, 10, 11, 12].

In general, the black hole solutions are axisymmetric solutions characterised
by four parameters: mass, angular momentum, charge and the cosmological
constant. There are two possibilities for such axisymmetric solutions. One is
a solution with spherical symmetry like the Schwarzschild solution, which has
been studied extensively. The other is the cylindrical symmetric solution
(black or cosmic string), which is not studied extensively. However, the
cylindrical symmetry has been found very crucial in the various contexts of
general relativity like the static solutions of Levi-Civita [13, 14] and
Chazy-Curzon [15, 16], and the stationary solutions of Lewis [17]. The
cylindrical symmetry is used to study cosmic strings in astrophysics [18]. The
general cylindrically symmetric static solution of Einstein’s equation with
cosmological constant was found by Linet in Ref. [19], which was not a black
hole solution. Lamos found the first black hole solution with cylindrical
symmetry in Ref. [20]. In general, black string and cylindrical black hole are
different, but they mimic each other in $4$ dimensions.

The spherically symmetric black holes were explored initially. The study of
prolate gravitational collapse of cylindrical and other similar objects
discovered later during the hoop conjecture formulation, which postulates that
horizon formation is possible if the circumference of compressed mass is less
than its Schwarzschild circumference in every direction. This excluded the
possibility of the formation of a cylindrical black hole. In fact, the hoop
conjecture is valid only in the absence of a cosmological constant. However,
cylindrically symmetric black holes exist in a negative cosmological constant.
Recently, the black strings have been constructed for 4-dimensional Einstein
gravity with a negative cosmological constant [20]. The thermodynamic
properties of a small black string, where a change in entropy due to thermal
fluctuation playing a significant role is not studied yet. This provides us
with a motivation for the present investigation which will bridge the gap.

In this paper, we mainly consider a system of a black string with a negative
cosmological constant and revisit the thermodynamics of the black string where
small statistical fluctuation for the entropy is not relevant, and eventually,
entropy follows the area law. The small statistical fluctuation plays a vital
role in the thermodynamics of the small black strings, so we have considered
the respective modification to the entropy of the black string. Eventually,
the black string’s whole thermodynamics gets corrected due to modified
entropy. We further investigate the change in the behaviour of the thermal
equation of states due to thermal fluctuations. Graphically, we observe that
the additional terms of entropy of the black string are significant for small
horizon black string. On the other hand, the fluctuation decreases the
internal energy for the black string, which is significant for larger-sized
black strings. The corrected Helmholtz free energy matches its uncorrected
counterpart for a more extensive horizon radius. Finally, the small
statistical fluctuation causes the small black string radius instability.
However, the black string remains stable without considering thermal
fluctuation.

The paper is organised in the following manner. In section 2, we recapitulate
the considered model describing black string and the cosmological constant.
Following the black string mechanics, we discuss the introductory
thermodynamics of the system. In section, 3, we investigate the impacts of
small statistical fluctuation on the equilibrium thermodynamics of black
string. The stability and fluid compressibility of the black string is
discussed in section 4. Here, the system of the black string is considered to
follow fluid dynamics. Finally, we conclude the discussion in section 5.

##  2 Black string and its thermodynamics

The Einstein-Hilbert action for the four-dimensional uncharged black string in
the presence of cosmological constant (setting gravitational constant, $G=1$)
is given by [21]

$I=\frac{1}{16\pi}\int d^{4}x\sqrt{-g}(R+6\alpha^{2}),$ (2.1)

where $g$ is the determinant of the metric $g_{\mu\nu}$, $R$ is the Ricci
scalar curvature, and $\alpha^{2}=-\frac{\Lambda}{3}>0$ denotes the negative
cosmological constant.

Varying the Einstein-Hilbert action (2.1) with the metric yields the field
equation as

$R_{\mu\nu}-\frac{1}{2}g_{\mu\nu}R=3\alpha^{2}g_{\mu\nu},$ (2.2)

where $R_{\mu\nu}$ is the Ricci tensor.

The general metric solution for the Einstein equation (2.1) in the
cylindrically symmetric and time-independent spacetime $(t,r,\phi,z)$ is
written by [20, 21]

$ds^{2}=-f(r)dt^{2}+g(r)dr^{2}+r^{2}d\phi^{2}+\alpha^{2}r^{2}dz^{2},$ (2.3)

here, $-\infty<t,0\leq r<+\infty,0\leq\phi\leq 2\pi$ and $z<+\infty$. The
metric function has the following form [21]:

$f(r)=g^{-1}(r)=\left(\alpha^{2}r^{2}-\frac{4M}{\alpha r}\right),$ (2.4)

where $M$ is the integration constant related to the black string’s Arnowitt-
Deser-Misner (ADM) mass density. The singularity at $r=0$ is the cosmological
one, and for a moment, we are not interested in such singularity conditions.

The event horizon of the black string can be obtained from the root of the
horizon condition, $f(r_{+})=0$. For positive $M$, it has a horizon at
$r_{+}=\frac{1}{\alpha}\left(4M\right)^{\frac{1}{3}}$ in the positive $r$
direction, it has a naked singularity in the negative $r$ direction. The
situation is reversed for negative $M$. This property is a new feature of such
a black string.

To study thermodynamics of the black string solutions, we employ the Euclidean
method [22, 23]. The requirement of the absence of the conical singularity in
the Euclidean spacetime (2.4) causes the Euclidean time must have a period
$T^{-1}$, which satisfies

$T=\left.\frac{f^{\prime}(r)}{4\pi}\right|_{r=r_{+}}=\frac{3\alpha^{2}r_{+}}{4\pi}.$
(2.5)

which is just the Hawking temperature of the black plane solutions (strings)
calculated from surface gravity. Here, prime (′) denotes the derivative
concerning $r$. In terms of the mass of the uncharged black string, the
Hawking temperature identifies to $T=(3\alpha/2\pi)\left(M/2\right)^{1/3}$,
varies as $M^{1/3}$, which is different from the regular Schwarzschild black
holes. This implies that the black string has a different configuration from
black holes due to changes in topological structures and, hence, it has a
different thermodynamical configuration.

Since the black string mimics a thermodynamical system and, therefore, follows
the first law of the thermodynamics [24, 25]

$dM=TdS_{0},$ (2.6)

where $S_{0}$ is the equilibrium entropy of the black string. Here, we note
that for the first law of black string thermodynamics, the work term (pressure
and volume) is absent [24, 25]. However, one can have such a work term in
extended phase space [24]. Using Eqs. (2.5) and (2.6), the first law of
thermodynamics leads to

$S_{0}=\int\frac{dM}{T}=\frac{\pi\alpha r_{+}^{2}}{2}.$ (2.7)

In terms of the mass of the black string, entropy,
$S_{0}=(\pi/2\alpha)(4M)^{2/3}$, varies as $M^{2/3}$ and hence different
configuration from the black hole.

From the Area-law, $S_{0}=\sigma/4$ suggests that the area of the event
horizon of the black string, $\sigma$, must have the following expression:

$\sigma=2\pi\alpha
r_{+}^{2}=\frac{\pi}{2\alpha}\left(4M\right)^{\frac{2}{3}}.$ (2.8)

The thermodynamic volume of the black string can be calculated from the area
(entropy) relation as

$V=\int\sigma dr_{+}=\frac{2}{3}\pi\alpha r_{+}^{3}.$ (2.9)

We can now estimate the differences in the equilibrium thermodynamics due to
the small statistical disturbances that become significant for quantum black
bodies.

##  3 Non-equilibrium thermodynamics of black string

It is well-known that the thermodynamics of the black bodies gets logarithmic
perturbation which plays a crucial role for the small black holes. Therefore,
on this basis, we anticipate that the thermodynamics of black string will also
follow the same trend. The nature of the logarithmic correction to the entropy
due to statistical fluctuations at the equilibrium is given by [26, 27]

$S=S_{0}-\beta\ln(S_{0}T^{2}),$ (3.1)

where $\beta$ is a label parameter which is equal to $1/2$ when fluctuation
exists and vanishes for the system at equilibrium.

For the given values of Hawking temperature (2.5) and equilibrium entropy
(2.7), the entropy for the uncharged black string under thermal fluctuation
reads

$S=\frac{\pi\alpha
r_{+}^{2}}{2}-\beta\ln\left(\frac{9\alpha^{5}r_{+}^{4}}{32\pi}\right).$ (3.2)

To visualise the effects of correction on the nature of entropy, we plot the
equilibrium entropy and the corrected entropy of the black string due to
thermal fluctuations in Fig. 1.

Figure 1: The plot of entropy $(S)$ with horizon radius $(r_{+})$ of the black
string.

The correction term does not show significant change for the larger radial
circumference of the horizon of the black string. Still, for the smaller
horizon radius, the corrected entropy of the black string is rather
substantial and makes the entropy asymptotically prominent as the horizon
ceases to point. The plot shows that entropy takes only a positive value in
both cases without and with thermal corrections.

Once we have Hawking temperature (2.5) and the corrected entropy (3.2), the
corrected internal energy of the black string, $E$, due to the thermal
fluctuation at equilibrium is computed as

$E=\int
TdS=\frac{\alpha^{3}r_{+}^{3}}{4}-\beta\left(\frac{3\alpha^{2}r_{+}}{\pi}\right).$
(3.3)

Now, the behaviour of the above expression is depicted in Fig. 2, which
studies the effects of fluctuation on the equilibrium internal energy.

Figure 2: The plot of internal energy $(E)$ with horizon radius $(r_{+})$ of
the black string.

The plot shows that the internal energy is an increasing function of the
horizon radius. Interestingly, the fluctuation decreases the value of internal
energy for the black string, which is not rather significant for small black
strings. The internal energy of a small black string becomes negatively valued
if we introduce thermal fluctuation to the system, which is quite an
interesting result. This suggests that the thermal fluctuation is taking
energy from the system and dissipating away as the system is not doing any
work.

The Helmholtz free energy a for non-equilibrium black string can be obtained
from the relation, $F=-\int SdT$. Utilizing the Hawking temperature (2.5) and
the entropy (3.2), this leads to

$F=-\frac{\alpha^{3}r_{+}^{3}}{8}+\frac{\beta\alpha^{2}r_{+}}{4\pi}\left[\ln\left(\frac{9\alpha^{5}r_{+}^{4}}{32\pi}\right)^{3}-6\right].$
(3.4)

Now, the behaviour of expression (3.4) is depicted in Fig. 3.

Figure 3: The plot of Helmholtz free energy $(F)$ with horizon radius
$(r_{+})$ of the black string.

Here, we observe that Helmholtz’s free energy of the small-sized black string
shows inverse power-law behaviour. However, it matches the equilibrium
counterpart for a more extensive horizon radius. A negative value of
Helmholtz’s free energy may indicate that the final energy of the black string
is smaller than its original energy. This result is also in agreement with the
internal energy.

For the given Helmholtz free energy (3.4) and volume (2.9), the thermodynamic
pressure of the black string is calculated in the extended phase space (by
identifying the negative cosmological constant with the equilibrium pressure)
with the help of relation, $P=-\left(\frac{\partial F}{\partial V}\right)$, as

$P=\frac{3\alpha^{2}}{16\pi}+\frac{3\beta\alpha}{8\pi^{2}r_{+}^{2}}\ln\left(\frac{32\pi}{9\alpha^{5}r_{+}^{4}}\right).$
(3.5)

Now, we plot the pressure (3.5) concerning the horizon radius in Fig. 4.

Figure 4: The plot of thermodynamic pressure $(P)$ with horizon radius
$(r_{+})$ of the black string.

The pressure due to thermal fluctuation is asymptotically large for small
black string. This means that thermal fluctuation for the system of small
black string induces more stress on the system. However, the system has
constant pressure for large black string for both the equilibrium and non-
equilibrium conditions.

The Gibbs free energy is also a substantial quantity for the thermodynamical
system. The definition of Gibbs free energy is given by $G=F+PV$. From the
volume (2.9), the Helmholtz free energy (3.4) and pressure (3.5), the Gibbs
free energy for the non-equilibrium system is calculated as

$G=\frac{\beta\alpha^{2}r_{+}}{2\pi}\left[\ln\left(\frac{9\alpha^{5}r_{+}^{4}}{32\pi}\right)^{3}-6\right].$
(3.6)

The comparison of the equilibrium and non-equilibrium black string is depicted
in Fig. 5.

Figure 5: The plot of Gibbs free energy $(G)$ with horizon radius $(r_{+})$ of
the black string.

Here, we see a significantly different behaviour of Gibbs free energy for the
black string in equilibrium and non-equilibrium states. The Gibbs free energy
without including thermal correction is zero. This means that no more work can
be done. In contrast to the equilibrium case, the Gibbs free energy under
thermal fluctuation depends on the horizon radius and becomes more negatively
valued for small black strings. This suggests that the thermal fluctuation
makes the reaction random or spontaneous.

##  4 Stability and compressibility

With the help of specific heat, the stability and the phase transition during
Hawking evaporation process can be analysed. To study the stability of the
black string, we follow the signature of (specific) heat capacity. The
positive sign of heat capacity indicates the black string is in a stable
phase; however, the negative sign of heat capacity shows an unstable phase
transition. Here, we calculate the heat capacity from the following
definition: $C=-\left(\frac{\partial E}{\partial T}\right)$. For the given
expressions (2.5) and (3.3), the heat capacity of black string specifies to

$C=-\left(\frac{\partial E}{\partial T}\right)=\pi\alpha r_{+}^{2}-4\beta.$
(4.1)

Here, we can see that the heat capacity takes a positive value ($C>0$) except
$r_{+}^{2}<4\beta/\pi\alpha$. However, the heat capacity is always positive
for the vanishing value of $\beta$ (no thermal correction) and shows a
parabolic curve. The black string’s phase transition from an unstable to a
stable state occurs when the horizon radius increases.

To specify the region of stability and instability, we plot the heat capacity
concerning $r_{+}$ as depicted in Fig. 6.

Figure 6: The plot of specific heat $(C)$ with horizon radius $(r_{+})$ of the
black string.

The figure shows that the fluctuation causes instability for a small radius of
the black string in the region $r_{+}^{2}<4\beta/\pi\alpha$. However, no
discontinuity is found in the black string’s heat capacity.

We considered the black string as a fluid system, so it is essential to derive
the compressibility, $K$, of the black string. This can be calculated with the
help of equations (2.9) and (3.5) as

$K=-\frac{1}{V}\left(\frac{\partial V}{\partial
P}\right)=\frac{4\pi^{2}r_{+}^{2}}{\beta\alpha}\left[2-\ln\left(\frac{9\alpha^{5}r_{+}^{4}}{32\pi}\right)\right]^{-1}.$
(4.2)

Here, we see that the compressibility of the black string depends on the
fluctuation parameters inversely.

##  5 Final remarks

It is possible to extend the spherical black hole solution to a cylindrical
black hole (black string) solution [20]. Considering the black string
solutions in asymptotically AdS space, the CFT dual to such solutions can also
be analysed [28, 29]. We have considered a system of a black string with a
negative cosmological constant. Assuming the system is in equilibrium, we have
discussed the thermodynamics of black string.

The small statistical fluctuation may play an essential role in the
thermodynamics of the cylindrical black hole solutions with negative
cosmological constant (black strings), so we have considered the respective
modification to the entropy of the black string. The modified entropy
eventually attributed to the thermodynamics of black string. We studied the
change in the behaviour of the thermal equations of states due to thermal
fluctuations. Here, we have found that the correction term of entropy of the
black string becomes significant for small horizon black string. The thermal
fluctuation introduces more disorderedness to the system of small black
strings. On the other hand, the fluctuation decreases the internal energy for
the black string, which is significant for larger-sized black strings.
However, the Helmholtz free energy becomes negative due to thermal fluctuation
for a small black string. The pressure is also calculated for the black string
under thermal fluctuation in the extended phase space. The pressure of the
small black string becomes very high due to thermal fluctuation. We have
calculated the Gibbs free energy also for the system of black string under the
influence of thermal fluctuation shows a matching behaviour with the
equilibrium counterpart for the sufficiently large horizon radius.

Finally, we have calculated the specific heat for the black string and found
that the small statistical fluctuation causes instability for a small radius
of the black string. However, excluding thermal fluctuation, the black string
remains stable. The compressibility of the black string depends on the
fluctuation parameters inversely. Since cylindrical black hole (black string)
in $4D$ has a rich structure. Therefore, the present investigation can be used
as a theoretical laboratory which may provide hints about the underlying
feature of the interaction between geometry and quantum physics. Investigating
the behaviour of $P-v$ criticality and quasi-normal modes for the black string
solutions under the influence of small statistical fluctuation will also be
interesting. This is the subject of future investigations.

## References

  * [1] S. W. Hawking, Commun. Math. Phys. 43 (1975) 199.
  * [2] S. W. Hawking, Nature 248 (1989) 30.
  * [3] A. Strominger and C. Vafa, Phys. Lett. B 379, 99 (1996).
  * [4] A. Ashtekar, J. Baez, A. Corichi and K. Krasnov, Phys. Rev. Lett. 80, 904 (1998).
  * [5] S. Carlip, Class. Quant. Grav. 17, 4175 (2000).
  * [6] R. B. Mann and S. N. Solodukhin, Nucl. Phys. B 523, 293 (1998).
  * [7] A. Pourdarvish, J. Sadeghi, H. Farahani, and B. Pourhassan, Int. J. Theor. Phys. 52, 3560 (2013).
  * [8] S. Upadhyay, Phys. Lett. B 775, 130 (2017).
  * [9] S. Upadhyay, B. Pourhassan and H. Farahani, Phys. Rev. D 95, 106014 (2017).
  * [10] D. Birmingham and S. Sen, Phys. Rev. D63, 047501 (2001).
  * [11] B. Pourhassan, S. Upadhyay, H. Saadat and H. Farahani, Nucl. Phys. B 928, 415 (2018).
  * [12] S. Upadhyay, Gen. Rel. Grav. 50, 128 (2018).
  * [13] T. Levi-Civita, Rend. Acc. Lincei, 26, 317 (1917).
  * [14] H. Weyl, Annalen Phys., 54, 117 (1917).
  * [15] J. Chazy, Bull. Soc. Math. France, 52, 17 (1924).
  * [16] H. E. J. Curzon, Procc. R. Soc. London, 23, 477 (1924).
  * [17] T. Lewis, Proc. R. Soc. London, A 136, 176 (1932).
  * [18] A. Vilenkin, Phys. Rep, 121, 263 (1985).
  * [19] B. Linet, J. Math. Phys., 27, 1817 (1986).
  * [20] J. P. S. Lemos, Phys. Lett. B 353, 46 (1995).
  * [21] R. G. Cai and Y. Zhang, Phys. Rev. D 54 (1996) 4891.
  * [22] G. W. Gibbons and S. W. Hawking, Phys. Rev. D 15, 2752 (1977).
  * [23] H. W. Barden, J. D. Brown, B. F. Whiting, and J. W. York, Jr., Phys. Rev. D 42, 3376 (1990).
  * [24] S. H. Hendi, H. Zarei, M. Faizal, B. Pourhassan and Z. Armanfard, Nucl.Phys.B 965, 115362 (2021).
  * [25] S. G. Ghosh, R. Kumar, L. Tannukij and P. Wongjun, Phys. Rev. D 101, 104042 (2020).
  * [26] S. Das, P. Majumdar and R. K. Bhaduri, Class. Quant. Grav. 19, 2355 (2002).
  * [27] H. K. Sudhanshu, S. Upadhyay, D. V. Singh and S. Kumar, Int. J. Theor. Phys. 61, 248 (2022).
  * [28] E. Greenwood, E. Halstead and P. Hao, JHEP 02, 044 (2010).
  * [29] H. Culetu, Int. J. Mod. Phys. D 10, 791 (2001).


11institutetext: Fudan University 22institutetext: China Pharmaceutical
University

# CLIP-Lung: Textual Knowledge-Guided Lung Nodule Malignancy Prediction

Yiming Lei 11 Zilong Li 1 1  Yan Shen 22 Junping Zhang 11 Hongming Shan 11

###### Abstract

Lung nodule malignancy prediction has been enhanced by advanced deep-learning
techniques and effective tricks. Nevertheless, current methods are mainly
trained with cross-entropy loss using one-hot categorical labels, which
results in difficulty in distinguishing those nodules with closer progression
labels. Interestingly, we observe that clinical text information annotated by
radiologists provides us with discriminative knowledge to identify challenging
samples. Drawing on the capability of the contrastive language-image pre-
training (CLIP) model to learn generalized visual representations from text
annotations, in this paper, we propose CLIP-Lung, a textual knowledge-guided
framework for lung nodule malignancy prediction. First, CLIP-Lung introduces
both class and attribute annotations into the training of the lung nodule
classifier without any additional overheads in inference. Second, we designed
a channel-wise conditional prompt (CCP) module to establish consistent
relationships between learnable context prompts and specific feature maps.
Third, we align image features with both class and attribute features via
contrastive learning, rectifying false positives and false negatives in latent
space. The experimental results on the benchmark LIDC-IDRI dataset have
demonstrated the superiority of CLIP-Lung, both in classification performance
and interpretability of attention maps.

###### Keywords:

Lung nodule classification vision-language model prompt learning.

##  1 Introduction

Lung cancer is one of the most fatal diseases worldwide, and early diagnosis
of the pulmonary nodule has been identified as an effective measure to prevent
lung cancer. Deep learning-based methods for lung nodule classification have
been widely studied in recent years [9, 12, 10]. Usually, the malignancy
prediction is often formulated as benign-malignant binary classification [9,
19], and the higher classification performance and explainable attention maps
are impressive. Most previous works employ a learning paradigm that utilizes
cross-entropy loss between predicted probability distributions and ground-
truth one-hot labels. Furthermore, inspired by ordered labels of nodule
progression, researchers have turned their attention to ordinal regression
methods to evaluate the benign-unsure-malignant classification task [18, 11,
2, 21, 13], where the training set additionally includes nodules with
uncertain labels. Indeed, the ordinal regression-based methods are able to
learn ordered manifolds and to further enhance the prediction accuracy.

However, the aforementioned methods still face challenges in distinguishing
visually similar samples with adjacent rank labels. For example in Fig. 1 (a),
since we conduct unimodal contrastive learning and map the samples onto a
spherical space, the false positive nodule with a malignancy score of $2.75$
has a closer distance to that with a score of $4.75$, and the false negative
one should not be closer to that of score $2.5$. To address this issue, we
found that the text attributes, such as “subtlety”, “sphericity”, “margin”,
and “lobulation”, annotated by radiologists can exhibit the differences
between these hard samples. Therefore, we propose leveraging the text
annotations to guide the learning of visual features. In reality, this also
aligns with the fact that the annotated text information represents the direct
justification for identifying lesion regions in the clinic.

Figure 1: Motivation of CLIP-Lung. (a) Unimodal contrastive learning. (b)
Proposed textual knowledge-guided contrastive learning. Yellow values are
annotated malignancy scores. Dashed boxes contain pairs of textual attributes
and annotated values.

To integrate text annotations into the image-domain learning process, an
effective text encoder providing precise text features is required.
Fortunately, recent advancements in vision-language models, such as
contrastive language-image pre-training (CLIP) [16], provide us with a
powerful pre-trained text encoder learned from text-based supervisions and
have shown impressive results in downstream vision tasks. Nevertheless, it is
ineffective to directly transfer CLIP to medical tasks due to the data
covariate shift. Therefore, in this paper, we propose CLIP-Lung, a framework
to classify lung nodules using image-text pairs. Specifically, CLIP-Lung
constructs learnable text descriptions for each nodule, including class- and
attribute-level. Inspired by CoCoOp [20], we proposed a channel-wise
conditional prompt (CCP) module to allow nodule descriptions to guide the
generation of informative feature maps. Different from CoCoOp, CCP constructs
specific learnable context prompts conditioned on grouped feature maps and
triggers more explainable attention maps such as Grad-CAM [17], whereas CoCoOp
provides only the common condition for all the prompt tokens. Then, we design
a textual knowledge-guided contrastive learning based on obtained image
features and textual features involving classes and attributes. Experimental
results on LIDC-IDRI [1] dataset demonstrate the effectiveness of learning
with textual knowledge for improving lung nodule malignancy prediction.

The contributions of this paper are summarized as follows.

  1. 1)

We proposed CLIP-Lung for lung nodule malignancy prediction which leverages
clinical textual knowledge to enhance the image encoder and classifier.

  2. 2)

We designed a channel-wise conditional prompt module to establish consistent
relationships among the correlated text tokens and feature maps.

  3. 3)

We align the image features with class and attribute features through
contrastive learning while generating more explainable attention maps
simultaneously.

Figure 2: Illustration of the proposed CLIP-Lung.

##  2 Methodology

###  2.1 Overview

Problem formulation In this paper, we arrange the lung nodule classification
dataset as $\\{\mathcal{I},\mathcal{Y},\mathcal{C},\mathcal{A}\\}$, where
$\mathcal{I}=\\{\bm{I}_{i}\\}_{i=1}^{N}$ is an image set contained $N$ lung
nodule images. $\mathcal{Y}=\\{y_{i}\\}_{i=1}^{N}$ is the corresponding class
label set and $y_{i}\in\\{1,2,\ldots,K\\}$, and $K$ is the number of classes.
$\mathcal{C}=\\{\vec{c}_{k}\\}_{k=1}^{K}$ is a set of text embeddings of
classes. Finally, $\mathcal{A}=\\{\vec{a}_{m}\\}_{m=1}^{M}$ is the set of
attribute embeddings, where each element $\vec{a}_{m}\in\mathbb{R}^{d\times
1}$ is a vector representing the embedding of an attribute word such as
“spiculation”. Then, for a given sample $\\{\bm{I}_{i},y_{i}\\}$, our aim is
to learn a mapping $f_{\bm{\theta}}:\bm{I}_{i}\mapsto y_{i}$, where $f$ is a
deep neural network parameterized by $\bm{\theta}$.

CLIP-Lung In Fig. 2 (a), the training framework contains an image encoder
$f_{\bm{\theta}}$ and a text encoder $g_{\bm{\phi}}$. First, the input image
$\bm{I}_{i}$ is fed into $f_{\bm{\theta}}$ and then generates the feature
maps. Then according to Fig. 2 (b), the feature maps are converted to channel-
wise feature vectors $f_{\bm{\theta}}(\bm{I}_{i})=\bm{F}_{t,:}$ and then to
learnable tokens $\bm{l}^{\prime}_{t}$. Second, we initialize the context
tokens $\bm{l}_{t}$ and add them with $\bm{l}^{\prime}_{t}$ to construct the
learnable prompts, where $T$ is the number of context words. Next, the
concatenation of the class token and $\bm{l}_{t}+\bm{l}^{\prime}_{t}$ is used
as input of text encoder yielding the class features
$g_{\bm{\phi}}(\bm{c}_{k})=\bm{C}_{k,:}$, note that $\bm{C}_{k,:}$ is
conditioned on channel-wise feature vectors $\bm{F}_{t,:}$. Finally, the
attribute tokens $\bm{a}_{m}$ are also fed into the text encoder to yield
corresponding attribute features $g_{\bm{\phi}}(\bm{a}_{m})=\bm{A}_{m,:}$.
Note that the vectors $\bm{F}_{t,:}$, $\bm{l}_{t,:}$, $\bm{l}^{\prime}_{t,:}$,
and $\bm{C}_{k,:}$ are with the same dimension $d=512$ in this paper.
Consequently, we have image feature $\bm{F}\in\mathbb{R}^{T\times d}$, class
feature $\bm{C}\in\mathbb{R}^{K\times d}$, and attribute feature
$\bm{A}\in\mathbb{R}^{M\times d}$ to conduct the textual knowledge-guided
contrastive learning.

###  2.2 Instance-Specific Attribute Weighting

For the attribute annotations, all the lung nodules in the LIDC-IDRI dataset
are annotated with the _same_ eight attributes: “subtlety”, “internal
structure”, “calcification”, “sphericity”, “margin”, “lobulation”,
“spiculation”, and “texture” [4, 8], and the annotated value for each
attribute is ranged from $1$ to $5$ except for “calcification” that is ranged
from $1$ to $6$. In this paper, we fix the parameters of a pre-trained text
encoder so that the generated eight text feature vectors are the same for all
the nodules. Therefore, we propose an instance-specific attribute weighting
scheme to distinguish different nodules. For the $i$-th sample, the weight for
each $\bm{a}_{m}$ is calculated through normalizing the annotated values:

$\displaystyle
w_{m}=\frac{\text{exp}(v_{m})}{\sum_{m=1}^{M}\text{exp}(v_{m})},$ (1)

where $v_{m}$ denotes the annotated value of $\bm{a}_{m}$. Then the weight
vector of the $i$-th sample is represented as
$\vec{w}_{i}=(w_{1},w_{2},\ldots,w_{M})^{\top}\in\mathbb{R}^{M\times 1}$.
Hence, the element-wise multiplication $\vec{w}_{i}\cdot\bm{A}_{i}$ is unique
to $\bm{I}_{i}$.

###  2.3 Channel-wise Conditional Prompt

CoCoOp [20] firstly proposed to learn context prompts for vision-language
models conditioned on visual features. However, it is inferior to align
context words with partial regions of the lesion. Therefore, we propose a
channel-wise conditional prompt (CCP) module, in Fig. 2 (b), to split latent
feature maps into $T$ groups and then flatten them into vectors
$\bm{F}_{t,:}$. Next, we denote $h(\cdot)$ as a context net that is composed
of a multi-layer perceptron (MLP) with one hidden layer, and each learnable
context token is now obtained by $\bm{l}^{\prime}_{t}=h(\bm{F}_{t,:})$. Hence,
the conditional prompt for the $t$-th token is
$\bm{l}_{t}+\bm{l}^{\prime}_{t}$. In addition, CCP also outputs the
$\bm{F}_{t,:}$ for image-class and image-attribute contrastive learning.

###  2.4 Textual Knowledge-Guided Contrastive Learning

Contrastive learning can effectively shorten the distances between positive
pairs and increase the distances between negative ones [5, 3, 7], and vision-
language models also applied contrastive learning using cross-modal image-text
pairs and achieved generalized image and text encoders [16]. In CLIP-Lung, our
aim is to align $\bm{F}\in\mathbb{R}^{T\times d}$ with
$\bm{C}\in\mathbb{R}^{K\times d}$ and $\bm{A}\in\mathbb{R}^{M\times d}$ as
illustrated in Fig. 2, _i.e._ , using class and attribute knowledge to
regularize the feature maps.

Image-class alignment First, the same to CLIP, we align the image and class
information by minimizing the cross-entropy (CE) loss based on the prediction
probability $p_{\text{i}}$:

$\displaystyle\mathcal{L}_{\text{IC}}=-\sum_{t=1}^{T}\sum_{k=1}^{K}y_{i}\text{log}(p_{\text{i}}),\quad
p_{\text{i}}=\frac{\text{exp}(\sigma(\bm{F}_{t,:},\bm{C}_{k,:})/\tau)}{\sum_{k^{\prime}=1}^{K}\text{exp}(\sigma(\bm{F}_{t,:},\bm{C}_{k^{\prime},:})/\tau)},$
(2)

where
$\bm{C}_{k,:}=g_{\bm{\phi}}(\bm{c}_{k}\bigoplus(\bm{l}_{1}+\bm{l}^{\prime}_{1},\bm{l}_{2}+\bm{l}^{\prime}_{2},\ldots,\bm{l}_{T}+\bm{l}^{\prime}_{T}))\in\mathbb{R}^{d\times
1}$ and “$\bigoplus$” denotes concatenation, _i.e._ , $\bm{C}_{k,:}$ is
conditioned on learnable prompts $\bm{l}_{t}+\bm{l}^{\prime}_{t}$.
$\sigma(\cdot,\cdot)$ calculates cosine similarity and $\tau$ is temperature
term. Therefore, $\mathcal{L}_{\text{IC}}$ implements the contrastive learning
between channel-wise features and corresponding class features, _i.e._ , the
ensemble of grouped image-class alignment results.

Image-attribute alignment In addition to image-class alignment, we further
expect the image features to be correlated with specific attributes. So we
conduct image-attribute alignment by minimizing the InfoNCE loss [5, 16]:

$\displaystyle\mathcal{L}_{\text{IA}}=-\sum_{t=1}^{T}\sum_{m=1}^{M}\text{log}\frac{\text{exp}(\sigma(\bm{F}_{t,:},\bm{w}_{m,:}\cdot\bm{A}_{m,:})/\tau)}{\sum_{m^{\prime}=1}^{M}\text{exp}(\sigma(\bm{F}_{t,:},\bm{w}_{m^{\prime},:}\cdot\bm{A}_{m^{\prime},:})/\tau)}.$
(3)

Due to each vector $\bm{F}_{t,:}$ is mapped from the $t$-th group of feature
maps through context net $h(\cdot)$, then $\mathcal{L}_{\text{IA}}$ indicates
which attribute the $\bm{F}_{t,:}$ is closest to. Therefore, certain feature
maps can be guided by specific annotated attributes.

Class-attribute alignment Although the image features have been aligned with
classes and attributes, the class embeddings obtained by the pre-trained CLIP
encoder may shift in latent space. This will result in inconsistent class
space and attribute space, _i.e._ , annotated attributes do not match the
corresponding classes, which is contradictory to the actual clinical
diagnosis. To avoid this weakness, we further align the class and attribute
features:

$\displaystyle\mathcal{L}_{\text{CA}}=-\sum_{k=1}^{K}\sum_{m=1}^{M}\text{log}\frac{\text{exp}(\sigma(\bm{C}_{k,:},\bm{w}_{m,:}\cdot\bm{A}_{m,:})/\tau)}{\sum_{m^{\prime}=1}^{M}\text{exp}(\sigma(\bm{C}_{k,:},\bm{w}_{m^{\prime},:}\cdot\bm{A}_{m^{\prime},:})/\tau)},$
(4)

and this loss implies semantic consistency between classes and attributes.

Finally, the total loss function is defined as follows:

$\displaystyle\mathcal{L}=\mathbb{E}_{\bm{I}_{i}\in\mathcal{I}}\big{[}\mathcal{L}_{\text{CE}}+\mathcal{L}_{\text{IC}}+\alpha\cdot\mathcal{L}_{\text{IA}}+\beta\cdot\mathcal{L}_{\text{CA}}\big{]},$
(5)

where $\alpha$ and $\beta$ are hyperparameters for adjusting the losses and
are set as $1$ and $0.5$, respectively. $\mathcal{L}_{\text{CE}}$ denotes the
cross-entropy loss between predicted probabilities obtained by the classifier
and the ground-truth labels. Note that during the inference phase, test images
are only fed into the trained image encoder and classifier, therefore, CLIP-
Lung does not introduce any additional computational overhead in inference.

##  3 Experiments

###  3.1 Dataset and Implementation Details

Dataset LIDC-IDRI
111https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=1966254
is a dataset for pulmonary nodule classification or detection based on low-
dose CT, which involves 1,010 patients. All the nodules were labeled with
scores from $1$ to $5$, indicating the malignancy progression. We cropped all
the nodules with a square shape of a doubled equivalent diameter at the
annotated center, then resized samples to the volume of $32\times 32\times
32$. Following [9, 11], we modified the first layer of the image encoder to be
with $32$ channels. According to existing works [11, 18], we regard a nodule
with an average score between $2.5$ and $3.5$ as unsure nodules, benign and
malignant categories are those with scores lower than $2.5$ and larger than
$3.5$, respectively. In this paper, we construct three sub-datasets: LIDC-A
contains three classes of nodules both in training and test sets; according to
[11], we construct the LIDC-B, which contains three classes of nodules _only_
in the training set, and the test set contains benign and malignant nodules;
LIDC-C includes benign and malignant nodules both in training and test sets.

Experimental settings In this paper, we apply the CLIP pre-trained text
encoder ViT-B/16 as the text encoder for CLIP-Lung, and the image encoder we
used is ResNet-18 [6] due to the relatively smaller scale of training data.
The image encoder is initialized randomly. Note that for the text branch, we
froze the parameters of the text encoder, and update the learnable tokens
$\vec{l}$ and $\vec{l}^{\prime}$ during training. The learning rate is $0.001$
following the cosine decay, the optimizer is stochastic gradient descent with
momentum $0.9$ and weight decay $0.00005$. The temperature $\tau$ is
initialized as $0.07$ and updated during training. All of our experiments are
implemented with PyTorch [15] and trained with NVIDIA A100 GPUs. The
experimental results are reported with average values through five randomly
independent split folds. For different classes, we report the recall and
F1-score values, and “$\pm$” indicates standard deviation.

###  3.2 Experimental Results and Analysis

Performance comparisons In Table 1, we compare the classification performances
on the LIDC-A dataset, where we regard the benign-unsure-malignant as an
ordinal relationship. Compared with ordinal classification methods such as
Poisson, NSB, UDM, and CORF, CLIP-Lung achieves the highest accuracy and
F1-scores for the three classes, which demonstrates the effectiveness of
textual knowledge-guided learning. CLIP and CoCoOp also outperform ordinal
classification methods and show the superiority of large-scale pre-trained
text encoders. Furthermore, CLIP-Lung obtained higher recalls than CLIP and
CoCoOp _w.r.t._ benign and malignant classes, however, the recall of unsure is
lower than theirs, we argue that this is due to the indistinguishable textual
annotations such as similar attributes of different nodules.

In Table 2, we compare the performances on LIDC-B and LIDC-C datasets. CLIP-
Lung obtains higher evaluation values other than recalls of benign class. We
conjecture the reason is that most of the benign nodules are with similar
appearances and subtle differences in text attributes, therefore, aligning
these two types of features is difficult and the text features will be biased
to those of malignant nodules.

Table 1: Classification results on the test set of LIDC-A. Method | Accuracy | Benign | Malignant | Unsure
---|---|---|---|---
Recall | F1 | Recall | F1 | Recall | F1
CE Loss | 54.2$\pm 0.6$ | 72.2 | 62.0 | 64.4 | 61.3 | 29.0 | 36.6
Poisson [2] | 52.7$\pm 0.7$ | 60.5 | 56.8 | 58.4 | 58.7 | 41.0 | 44.1
NSB [13] | 53.4$\pm 0.7$ | 80.7 | 63.0 | 67.3 | 63.8 | 16.0 | 24.2
UDM [18] | 54.6$\pm 0.4$ | 76.7 | 64.3 | 49.5 | 53.5 | 32.5 | 39.5
CORF [21] | 56.8$\pm 0.4$ | 71.3 | 63.3 | 61.3 | 62.3 | 38.5 | 44.3
CLIP [16] | 56.6$\pm 0.3$ | 59.5 | 59.2 | 55.2 | 60.0 | 53.9 | 52.2
CoCoOp [20] | 56.8$\pm 0.6$ | 59.0 | 59.2 | 55.2 | 60.0 | 55.1 | 52.8
CLIP-Lung | 60.9$\pm 0.4$ | 67.5 | 64.4 | 60.9 | 66.3 | 53.4 | 54.1
Table 2: Classification results on test sets of LIDC-B and LIDC-C. | LIDC-B | LIDC-C |
---|---|---|---
Method | Accuracy | Benign | Malignant | Accuracy | Benign | Malignant
Recall | F1 | Recall | F1 | Recall | F1 | Recall | F1
CE Loss | 83.3$\pm 0.6$ | 92.4 | 88.4 | 63.4 | 70.3 | 85.5$\pm 0.5$ | 91.5 | 89.7 | 72.3 | 75.6
Poisson [2] | 81.8$\pm 0.4$ | 94.2 | 87.7 | 54.5 | 65.1 | 84.0$\pm 0.3$ | 87.9 | 88.3 | 75.2 | 74.5
NSB [13] | 78.1$\pm 0.5$ | 90.6 | 85.8 | 50.5 | 60.7 | 84.9$\pm 0.7$ | 91.0 | 89.2 | 71.3 | 74.6
UDM [18] | 79.3$\pm 0.4$ | 87.0 | 86.2 | 62.4 | 67.7 | 84.6$\pm 0.5$ | 88.8 | 88.8 | 75.2 | 75.2
CORF [21] | 81.5$\pm 0.3$ | 95.9 | 87.8 | 49.5 | 62.8 | 83.0$\pm 0.2$ | 87.9 | 87.7 | 72.3 | 72.6
CLIP [16] | 83.6$\pm 0.6$ | 92.0 | 88.7 | 64.4 | 70.4 | 87.5$\pm 0.3$ | 92.0 | 91.0 | 77.0 | 78.8
CoCoOp [20] | 86.8$\pm 0.7$ | 94.5 | 90.9 | 69.0 | 75.9 | 88.2$\pm 0.6$ | 95.0 | 91.8 | 72.4 | 78.8
CLIP-Lung | 87.5$\pm 0.3$ | 94.5 | 91.7 | 72.3 | 79.0 | 89.5$\pm 0.4$ | 94.0 | 92.8 | 80.5 | 82.8

Figure 3: The t-SNE (Left) and Grad-CAM (Right) results.

Visual features and attention maps To illustrate the influence of
incorporating class and attribute knowledge, we provide the t-SNE [14] Grad-
CAM [17] results obtained by CLIP, CoCoOp, and CLIP-Lung. In Fig. 3, we can
see that CLIP yields a non-compact latent space for two kinds of nodules.
CoCoOp and CLIP-Lung alleviate this phenomenon, which demonstrates that the
learnable prompts guided by nodule classes are more effective than fixed
prompt engineering. Further, compared with CLIP-Lung, CoCoOp could not
consider the attribute information to learn the prompts, therefore, it results
in more false negatives in latent space. From the attention maps we can
observe that CLIP cannot precisely capture spiculation and lobulation regions
that are highly correlated with malignancy. Simultaneously, CLIP-Lung performs
better than CoCoOp, which demonstrates the guidance from textual descriptions
such as “spiculation”.

Ablation studies In Fig. 3, we verify the effectiveness of different loss
components on the three constructed datasets. Based on
$\mathcal{L}_{\text{IC}}$, $\mathcal{L}_{\text{IA}}$ and
$\mathcal{L}_{\text{CA}}$ improve the performances on LIDC-A, indicating the
effectiveness of capturing fine-grained features of ordinal ranks using class
and attribute texts. However, they perform relatively worse on LIDC-B and
LIDC-C, especially the $\mathcal{L}_{\text{IC}}+\mathcal{L}_{\text{CA}}$. That
is to say, $\mathcal{L}_{\text{IA}}$ is more important in latent space
rectification, _i.e._ , image-attribute consistency. In addition, we observe
that $\mathcal{L}_{\text{IC}}+\mathcal{L}_{\text{IA}}$ performs better than
$\mathcal{L}_{\text{IA}}+\mathcal{L}_{\text{CA}}$, which is attributed to that
$\mathcal{L}_{\text{CA}}$ regularizes the image features indirectly.

Table 3: Ablation study on different losses. We report classification accuracies.  $\mathcal{L}_{\text{IC}}$ | $\mathcal{L}_{\text{IA}}$ | $\mathcal{L}_{\text{CA}}$ | LIDC-A | LIDC-B | LIDC-C
---|---|---|---|---|---
✓ |  |  | 56.8$\pm 0.6$ | 86.8$\pm 0.7$ | 88.2$\pm 0.6$
✓ | ✓ |  | 59.4$\pm 0.4$ | 86.8$\pm 0.6$ | 86.7$\pm 0.4$
| ✓ | ✓ | 58.1$\pm 0.2$ | 85.7$\pm 0.6$ | 87.5$\pm 0.5$
✓ |  | ✓ | 56.9$\pm 0.3$ | 84.7$\pm 0.4$ | 84.0$\pm 0.7$
✓ | ✓ | ✓ | 60.9$\pm 0.4$ | 87.5$\pm 0.5$ | 89.5$\pm 0.4$

##  4 Conclusion

In this paper, we proposed a textual knowledge-guided framework for pulmonary
classification, named CLIP-Lung. We explored the utilization of clinical
textual annotations based on large-scale pre-trained text encoders. CLIP-Lung
aligned the different modalities of features generated from nodule classes,
attributes, and images through contrastive learning. Most importantly, CLIP-
Lung establishes correlations between learnable prompt tokens and feature maps
using the proposed CCP module, and this guarantees explainable attention maps
localizing fine-grained clinical features. Finally, CLIP-Lung outperforms
compared methods, including CLIP on LIDC-IDRI benchmark. Future work can
concentrate on extending CLIP-Lung with more diverse textual knowledge.

## References

  * [1] Armato III, S.G., McLennan, G., Bidaut, L., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans. Medical Physics 38(2), 915–931 (2011)
  * [2] Beckham, C., Pal, C.: Unimodal probability distributions for deep ordinal classification. In: Proceedings of the International Conference on Machine Learning (ICML). pp. 411–419 (2017)
  * [3] Chen, X., He, K.: Exploring simple siamese representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 15750–15758 (2021)
  * [4] Hancock, M.C., Magnan, J.F.: Predictive capabilities of statistical learning methods for lung nodule malignancy classification using diagnostic image features: an investigation using the lung image database consortium dataset. In: Medical Imaging 2017: Computer-Aided Diagnosis. vol. 10134, pp. 558–569 (2017)
  * [5] He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9729–9738 (2020)
  * [6] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016)
  * [7] Henaff, O.: Data-efficient image recognition with contrastive predictive coding. In: International conference on machine learning. pp. 4182–4192. PMLR (2020)
  * [8] Joshi, A., Sivaswamy, J., Joshi, G.D.: Lung nodule malignancy classification with weakly supervised explanation generation. Journal of Medical Imaging 8(4), 044502–044502 (2021)
  * [9] Lei, Y., Tian, Y., Shan, H., Zhang, J., Wang, G., Kalra, M.K.: Shape and margin-aware lung nodule classification in low-dose CT images via soft activation mapping. Medical Image Analysis 60, 101628 (2020)
  * [10] Lei, Y., Zhang, J., Shan, H.: Strided self-supervised low-dose ct denoising for lung nodule classification. Phenomics 1, 257–268 (2021)
  * [11] Lei, Y., Zhu, H., Zhang, J., Shan, H.: Meta ordinal regression forest for medical image classification with ordinal labels. IEEE/CAA Journal of Automatica Sinica 9(7), 1233–1247 (2022)
  * [12] Liao, Z., Xie, Y., Hu, S., Xia, Y.: Learning from ambiguous labels for lung nodule malignancy prediction. IEEE Transactions on Medical Imaging 41(7), 1874–1884 (2022)
  * [13] Liu, X., Zou, Y., Song, Y., Yang, C., You, J., K Vijaya Kumar, B.: Ordinal regression with neuron stick-breaking for medical diagnosis. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 335–344 (2018)
  * [14] Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. Journal of Machine Learning Research 9(11) (2008)
  * [15] Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., Lerer, A.: Automatic differentiation in pytorch (2017)
  * [16] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763 (2021)
  * [17] Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision. pp. 618–626 (2017)
  * [18] Wu, B., Sun, X., Hu, L., Wang, Y.: Learning with unsure data for medical image diagnosis. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). pp. 10590–10599 (2019)
  * [19] Xie, Y., Zhang, J., Xia, Y.: Semi-supervised adversarial model for benign–malignant lung nodule classification on chest ct. Medical image analysis 57, 237–248 (2019)
  * [20] Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Conditional prompt learning for vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16816–16825 (2022)
  * [21] Zhu, H., Shan, H., Zhang, Y., Che, L., Xu, X., Zhang, J., Shi, J., Wang, F.Y.: Convolutional ordinal regression forest for image ordinal estimation. IEEE Transactions on Neural Networks and Learning Systems 33(8), 4084–4095 (2022)