# Special Alternating knots are Band Prime

Joe Boninger and Joshua Evan Greene  Department of Mathematics, Boston
College, Chestnut Hill, MA<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

We prove that a special alternating knot does not decompose as a non-trivial
band sum. This restricts concordances from special alternating knots, and we
conjecture that special alternating knots are ribbon concordance minimal. We
verify our conjecture in many cases. This work is motivated by another
conjecture of Owens and the second author, which posits that the set of
alternating knots is downward closed under ribbon concordance.

JB was supported by the National Science Foundation under Award No. 2202704.
JEG was supported by the National Science Foundation under Award No.
DMS-2304856.

##  1\. Introduction

A diagram $D\subset S^{2}$ of a link $K\subset S^{3}$ is alternating if a
traveller along $D$ alternates between passing over and under crossing points.
It is special if Seifert’s algorithm returns one of the chessboard surfaces of
$D$ or, equivalently, if one of the chessboard surfaces of $D$ is orientable.

A link $K$ is alternating if it admits an alternating diagram, and it is
special alternating if it admits a diagram which is both special and
alternating. Special alternating links and their canonical Seifert surfaces
are building blocks for the set of all alternating links under the operation
of Murasugi sum. They include $(2,n)$-torus links, and they can be obtained
more generally from any bipartite planar graph by inverting the Tait graph
construction.

Suppose knots $K_{0},K_{1}\subset S^{3}$ are separated from each other by a
2-sphere $S^{2}$, and let $b:I\times I\to S^{3}$ be an embedded band such that
$b^{-1}(K_{i})=I\times\\{i\\}$, $i\in\\{0,1\\}$. The band sum
$K=K_{0}\\#_{b}K_{1}\subset S^{3}$ is the knot $(K_{0}\cup
K_{1}-b(I\times\partial I))\cup b(\partial I\times I)$. This band sum is
trivial if $b^{-1}(S^{2})=I\times\\{1/2\\}$, so that $K_{0}\\#_{b}K_{1}$ is
equivalent to the connected sum $K_{0}\\#K_{1}$. A knot $K$ is band prime if
it cannot be written as a non-trivial band sum.

The main result of this note is the following:

######  Theorem 1.1.

Special alternating knots are band prime.

Our motivation for studying band sums is their relationship to ribbon
concordance, which was introduced by Gordon in an influential paper [7]. Two
knots $K_{0}$, $K_{1}$ are (smoothly) concordant if there exists a smooth,
properly embedded annulus $C\subset S^{3}\times I$ such that $\partial
C\cap(S^{3}\times\\{i\\})=K_{i}\times\\{i\\}$. The annulus $C$ is called a
ribbon concordance from $K_{1}$ to $K_{0}$, and we write $K_{0}\leq K_{1}$, if
the restriction $h|_{C}$ of the height function $h:S^{3}\times I\to I$ is
Morse and has no critical points of index two. Ribbon concordance generalizes
the notion of a ribbon knot, since a knot is ribbon if and only if it is
ribbon concordant to the unknot.

Theorem 1.1 implies:

######  Corollary 1.2.

Suppose that $K_{1}$ is a special alternating knot, and $K_{0}\leq K_{1}$ by a
ribbon concordance $C\subset S^{3}\times I$ such that $h|_{C}$ has two
critical points. Then $K_{0}\simeq K_{1}$, and $C$ is isotopic to the trivial
concordance $K_{1}\times I$.

###### Proof.

Morse theory implies $h|_{C}$ has one critical point of index $0$—a birth—and
one of index $1$—a saddle. Equivalently, the knot $K_{1}$ is a band sum of
$K_{0}$ and an unknot, and the result follows from Theorem 1.1. ∎

Agol [1] proved ribbon concordance forms a partial ordering on the set of all
knots in $S^{3}$, resolving a long open conjecture of Gordon [7, Conjecture
1.1]. This ordering has important properties: for instance, if $K_{0}\leq
K_{1}$, then the Seifert genus of $K_{0}$ is less than or equal to that of
$K_{1}$ [22]; the Alexander polynomial of $K_{0}$ divides that of $K_{1}$ [6,
4]; and if $K_{1}$ is fibered, then so is $K_{0}$ [18, 22]. In this vein, the
second author and Owens conjectured that if $K_{0}\leq K_{1}$ and $K_{1}$ is
alternating, then so is $K_{0}$ [10, Conjecture 9].

Motivated by this conjecture and by 1.2, we propose:

######  Conjecture 1.3.

Special alternating knots are ribbon concordance minimal.

That is, if $K_{1}$ is special alternating, and $K_{0}\leq K_{1}$, then
$K_{0}\simeq K_{1}$. Note that a positive answer to [10, Conjecture 9] does
not directly imply one to 1.3. On the other hand, 1.3 implies 1.1, because
Miyazaki proved that ribbon concordance minimal knots are band prime: for a
prime knot, this is [17, Theorem 3], and for a composite knot, this
additionally requires [19, Theorem 1].

We report on some more evidence in support of 1.3. One piece of evidence
follows quickly from work of Zemke [22]:

######  Proposition 1.4.

Suppose $K$ is a special alternating knot, and $K^{\prime}\leq K$. Then

$\widehat{HFK}(K^{\prime})\cong\widehat{HFK}(K).$

In particular, the two knots have the same genus, Alexander polynomial, and
determinant.

Here $\widehat{HFK}(K)$ denotes the hat flavor of the knot Floer homology of
$K$ with $\mathbb{F}_{2}$ coefficients.

More evidence draws on the notions of $\mathbb{Q}$-anisotropy and transfinite
nilpotence, which we recall in Section 3. Gordon proved that any knot which is
$\mathbb{Q}$-anisotropic and transfinitely nilpotent is ribbon concordance
minimal, and he asked whether alternating knots are transfinitely nilpotent
[7, Theorem 1.3 and Question 5.2]. An elementary argument shows that a special
alternating knot is $\mathbb{Q}$-anisotropic (Lemma 3.1 below). Thus, a
positive answer to Gordon’s question would imply a positive answer to 1.3.
Transfinite nilpotence is known in certain circumstances, which leads to the
following result:

######  Proposition 1.5.

Suppose $K$ is a special alternating knot such that

  1. (1)

$K$ is fibered,

  2. (2)

the leading coefficient of its Alexander polynomial is a prime power, or

  3. (3)

$K$ is a two-bridge knot.

Then $K$ is ribbon concordance minimal.

We close the introduction with a conjecture strengthening 1.1 and 1.3. A knot
$K\subset S^{3}$ is positive if it admits an oriented diagram in which every
crossing is positive. If we orient a special alternating diagram, then every
crossing has the same sign, positive or negative. Thus, up to mirroring, every
special alternating knot is a positive knot. Nakamura proved that special
alternating knots are the knots which are both alternating and positive up to
mirroring [20, Theorem 1]. Extending 1.3, we posit:

######  Conjecture 1.6.

Positive knots are ribbon concordance minimal.

Other examples of positive knots include torus knots, which Gordon showed are
ribbon concordance minimal [7, Remark before Theorem 1.3].

### Outline

In Section 2, we prove Theorem 1.1. In Section 3, we prove Propositions 1.4
and 1.5.

##  2\. Proof of the Main Theorem

We begin by reducing the proof of 1.1 to the case of prime special alternating
knots; this is possible thanks to results of Eudave-Muñoz [3] and Menasco
[15].

######  Lemma 2.1.

If a composite knot is not band prime, then one of its factors is not band
prime.

###### Proof.

Suppose a composite knot $K$ can be written as a non-trivial band sum
$K_{0}\\#_{b}K_{1}$. Eudave-Muñoz showed that there exists a decomposing
sphere $F\subset S^{3}$ for $K$ (as a composite knot) which is disjoint from
the band $b$ [3, Theorem 2]. This implies

$2=|F\cap K|=|F\cap(K_{0}\sqcup K_{1})|,$

and therefore $F$ is a (possibly trivial) decomposing sphere for either
$K_{0}$ or $K_{1}$. We assume the latter without loss of generality, and let
$K_{1}=K_{A}\\#K_{B}$ be the decomposition of $K_{1}$ determined by $F$. Since
$F$ is disjoint from $b$, $b$ is contained on one side of $F$, which we assume
is the side containing $K_{A}$. Equivalently, we have

$K=(K_{0}\\#_{b}K_{A})\\#K_{B}.$

If the band sum $K_{0}\\#_{b}K_{A}$ is trivial, then

$K=(K_{0}\\#K_{A})\\#K_{B}=K_{0}\\#(K_{A}\\#K_{B})=K_{0}\\#K_{1}.$

By an observation of Miyazaki following Eudave-Muñoz’s work, this implies the
band sum $K_{0}\\#_{b}K_{1}$ is trivial [19, Theorem 1]. This contradiction
shows the band sum $K_{0}\\#_{b}K_{A}$ is nontrivial, so $K$ admits a factor
which is not band prime. ∎

######  Lemma 2.2.

If a special alternating knot $K$ decomposes as a connected sum
$K_{0}\\#K_{1}$, then each of $K_{0}$ and $K_{1}$ are special alternating.

###### Proof.

Let $D$ be an alternating diagram of $K$. A theorem of Menasco implies that
$K_{i}$ has an alternating diagram $D_{i}$ for $i=0,1$ such that
$D=D_{0}\\#D_{1}$ [15, Theorem 1]. It is straightforward to check that each of
the $D_{i}$ is special if $D$ is. ∎

######  Corollary 2.3.

If there exists a special alternating knot which is not band prime, then there
exists a prime special alternating knot which is not band prime. ∎

###### Proof of Theorem 1.1.

Let $K\subset S^{3}$ be a special alternating knot, and suppose toward a
contradiction that $K$ decomposes as a non-trivial band sum
$K_{0}\\#_{b}K_{1}$. By 2.3, we may assume $K$ is prime.

Gabai showed that there exists a minimal genus Seifert surface $S$ for $K$ and
a properly embedded arc $\alpha\subset S$, coinciding with a co-core of $b$,
such that $S^{\prime}=S-\eta(\alpha)$ is a Seifert surface for the split link
$K_{0}\sqcup K_{1}$ [5, Proof of Theorem 1]. Here $\eta$ denotes a regular
open neighborhood.

Hirasawa-Sakuma proved any minimal genus surface of a special alternating knot
is a chessboard surface of an alternating diagram of that knot [11, Theorem
1.1]. Accordingly, we choose an alternating diagram $D\subset S^{2}$ of $K$
admitting $S$ as a chessboard surface. Let $G\subset S^{2}$ be the
corresponding Tait graph, and let $\lambda_{S}$ denote the symmetrized Seifert
pairing on $H_{1}(S)$. Since $S$ is a chessboard surface and $D$ is
alternating, Gordon and Litherland proved in this case that
$(H_{1}(S),\lambda_{S})$ is isometric, up to an overall sign, to the lattice
$\mathcal{F}(G)$ of integer-valued flows on $G$ [8, Theorem 1]. In particular,
$\lambda_{S}$ is definite.

A Mayer-Vietoris argument shows the inclusion-induced map
$H_{1}(S^{\prime})\to H_{1}(S)$ injects, so it follows that
$\lambda_{S^{\prime}}$ is definite as well. Hence both $S$ and $S^{\prime}$
are incompressible, by [9, Corollary 3.2].

Let $F\subset S^{3}$ be a 2-sphere separating $K_{0}$ and $K_{1}$, chosen so
$F$ and $S^{\prime}$ are in general position and $F\cap S^{\prime}$ has the
minimal number of components. A standard innermost curve argument, using the
incompressibility of $S^{\prime}$, shows $F\cap S^{\prime}=\varnothing$. Thus
$S^{\prime}$ has two components which are separated by the sphere $F$; we
denote these by $S_{0}$ and $S_{1}$, so that $\partial S_{i}=K_{i}$. Viewing
$S$ as $S_{0}\cup_{\alpha}S_{1}$, we conclude that the inclusion map
$S_{0}\sqcup S_{1}\hookrightarrow S$ induces a lattice isomorphism
$(H_{1}(S_{0}),\lambda_{S_{0}})\oplus(H_{1}(S_{1}),\lambda_{S_{1}})\cong(H_{1}(S),\lambda_{S})$.

We claim each of $H_{1}(S_{0})$ and $H_{1}(S_{1})$ are non-zero. Suppose not;
then $H_{1}(S_{1})$, say, has rank $0$, so $S_{1}$ is a disk. In this case the
boundary of a regular neighborhood of $S_{1}$ in $S^{3}-K_{0}$ is a sphere
which separates $K_{0}$ and $K_{1}$ and meets $S$ in a single arc parallel to
$\alpha$, implying $b$ is trivial. This contradiction proves the claim, and it
follows that $(H_{1}(S),\lambda_{S})$ admits a non-trivial decomposition as an
orthogonal direct sum.

We have thereby shown that $\mathcal{F}(G)\cong(H_{1}(S),\pm\lambda_{S})$
admits a non-trivial decomposition as an orthogonal direct sum. By a result of
Bacher, de la Harpe, and Nagnibeda, it follows that $G$ is separable: there
exist subgraphs $G_{0},G_{1}\subset G$ with
$\mathcal{F}(G_{i})\cong(H_{1}(S_{i}),\lambda_{S_{i}})$, such that $G_{0}\cup
G_{1}=G$ and $G_{0}\cap G_{1}$ consists of a single vertex [2, Proposition 4].
Equivalently, $D$ admits a connected sum decomposition $D=D_{0}\\#D_{1}$ with
$G_{i}$ a Tait graph of $D_{i}$. Let $K_{i}\subset S^{3}$ be the knot
represented by $D_{i}$, $i=0,1$. Each form $\lambda_{S_{i}}$ is non-zero and
definite, so $\sigma(K_{i})=\sigma(\lambda_{S_{i}})\neq 0$, implying each
$K_{i}$ is non-trivial. We conclude that $K$ is not prime, a contradiction
which proves the theorem. ∎

######  Remark 2.4.

The proof of 1.1 shows, in fact, that all non-split special alternating links
are band prime. While the cited results of Eudave-Muñoz [3], Gabai [5], and
Miyazaki [19] are stated only for knots, the remark in [3] following the proof
of Theorem 2 can be used in place of [3, Theorem 2], and [21, Proposition 8.9]
can be used in place of Gabai’s work. Similarly, the proof of Miyazaki’s
result extends to non-split links. On the other hand, it would require more
effort to extend the result to split special alternating links: one would like
to know that the split union of band prime links is again band prime. We
decided not to pursue this issue, besides to sense that it is non-trivial.

######  Remark 2.5.

We had originally attempted to prove [10, Conjecture 9] using the
characterization of alternating knots in terms of definite spanning surfaces
[9]. However, this approach proved more challenging to implement than
expected. The use of a definite chessboard surface in the proof of 1.1 is the
surviving remnant of this idea.

##  3\. Ribbon Concordance

In light of Corollary 1.2, Theorem 1.1 provides evidence that special
alternating knots are ribbon concordance minimal. In this section we prove
Propositions 1.4 and 1.5, which provide more support for this conjecture.

###### Proof of 1.4.

The relation $K^{\prime}\leq K$ implies $\sigma(K^{\prime})=\sigma(K)$. Zemke
showed that for any ribbon concordance from $K$ to $K^{\prime}$, the induced
map $\widehat{HFK}(K^{\prime})\to\widehat{HFK}(K)$ is injective [22, Theorem
1.1]. Since knot Floer homology detects genus, it follows that
$g(K^{\prime})\leq g(K)$ [22, Theorem 1.3]. As in the proof of 1.1, since $K$
is special alternating, it satisfies $|\sigma(K)|=2g(K)$. Thus

$|\sigma(K)|=|\sigma(K^{\prime})|\leq 2g(K^{\prime})\leq 2g(K)=|\sigma(K)|,$

and $g(K)=g(K^{\prime})$. Additionally, because $K$ is alternating, $K$—and
therefore $K^{\prime}$—are homologically thin. Letting $\Delta$ denote the
Alexander polynomial, we obtain

$\text{span}(\Delta_{K^{\prime}})=2g(K^{\prime})=2g(K)=\text{span}(\Delta_{K}).$

Next, we use the fact that $K^{\prime}\leq K$ implies $\Delta_{K^{\prime}}$
divides $\Delta_{K}$ [6, 4]. Since
$\text{span}(\Delta_{K^{\prime}})=\text{span}(\Delta_{K})$, we have
$m\Delta_{K^{\prime}}=\Delta_{K}$ for some $m\in\mathbb{Z}$. Since
$\Delta_{K}(1)=\Delta_{K^{\prime}}(1)=1$, we have $m=1$ and
$\Delta_{K^{\prime}}=\Delta_{K}$. Finally, because $K^{\prime}$ and $K$ are
homologically thin and supported in the same $\delta$-grading, their knot
Floer homology groups are determined by their Alexander polynomials. Thus,

$\widehat{HFK}(K^{\prime})\cong\widehat{HFK}(K).\qed$

Before proving 1.5, we review some terminology. Given a knot $K$, we let $C$
denote the commutator subgroup of the knot group $\pi_{1}(S^{3}-K)$. The lower
central series $\\{\gamma_{i}(C)\\}_{i\geq 1}$ of $C$ is defined recursively
by

$\displaystyle\gamma_{1}(C)$ $\displaystyle=C,$ $\displaystyle\gamma_{n}(C)$
$\displaystyle=[G,\gamma_{n-1}(G)],$

and for a limit ordinal $\omega$ we define
$\gamma_{\omega}(G):=\bigcap_{\alpha<\omega}\gamma_{\alpha}(G)$. We say $C$ is
transfinitely nilpotent if its lower central series is eventually trivial, and
we say $K$ is transfinitely nilpotent if $C$ is. Free groups give the simplest
examples of groups which are transfinitely nilpotent but not nilpotent [13,
Chapter 5].

If $X$ denotes the universal abelian cover of $K$, then Milnor defined a
quadratic form $\mu$ on $H^{1}(X;\mathbb{Q})$ with rank
rk$(\mu)=\text{span}(\Delta_{K})$ and signature $\sigma(\mu)=\sigma(K)$ [16].
Following Gordon, we say $K$ is $\mathbb{Q}$-anisotropic if
$H^{1}(X;\mathbb{Q})$ contains no non-trivial subspace which is annhilated by
$\mu$ and invariant under deck transformations [7, Proposition 4.5].

######  Lemma 3.1.

Special alternating knots are $\mathbb{Q}$-anisotropic.

###### Proof.

If $K$ is a special alternating knot, then we have

$|\sigma(\mu)|=|\sigma(K)|=\text{span}(\Delta_{K})=\text{rk}(\mu),$

so $\mu$ is a definite form. It follows that there is no non-trivial subspace
of $H^{1}(X;\mathbb{Q})$ annhilated by $\mu$, let alone one which is invariant
under deck transformations. ∎

######  Proposition 3.2.

If $K$ is a special alternating knot and $K$ is transfinitely nilpotent, then
$K$ is ribbon concordance minimal.

###### Proof.

Gordon proved that any knot which is $\mathbb{Q}$-anisotropic and
transfinitely nilpotent is ribbon concordance minimal [7, Theorem 1.3]. Hence
the result follows from Lemma 3.1.

Alternatively, Gordon proved that if $K^{\prime}\leq K$, $K$ is transfinitely
nilpotent, and span$(\Delta_{K})=\text{span}(\Delta_{K^{\prime}})$, then
$K^{\prime}\simeq K$ [7, Lemma 3.4]. Hence the result follows from 1.4. ∎

###### Proof of 1.5.

If $K$ satisfies any of the listed hypotheses, then $K$ is transfinitely
nilpotent:

  1. (1)

follows because commutator subgroups of fibered knots are free,

  2. (2)

is a classical result of Mayland and Murasugi [14, Theorem A], and

  3. (3)

was proven more recently by Johnson [12, Corollary 1.3 and Remark 1.20].

Hence the result follows from 3.2. ∎

## References

  * [1] Ian Agol, _Ribbon concordance of knots is a partial ordering_ , Comm. Amer. Math. Soc. 2 (2022), 374–379.
  * [2] Roland Bacher, Pierre de la Harpe, and Tatiana Nagnibeda, _The lattice of integral flows and the lattice of integral cuts on a finite graph_ , Bull. Soc. Math. France 125 (1997), no. 2, 167–198.
  * [3] Mario Eudave-Muñoz, _Band sums of links which yield composite links. the cabling conjecture for strongly invertible knots_ , Trans. Amer. Math. Soc. 330 (1992), no. 2, 463–501.
  * [4] Stefan Friedl and Mark Powell, _Homotopy ribbon concordance and Alexander polynomials_ , Arch. Math. (Basel) 115 (2020), no. 6, 717–725.
  * [5] David Gabai, _Genus is superadditive under band connected sum_ , Topology 26 (1987), no. 2, 209–210.
  * [6] Patrick M. Gilmer, _Ribbon concordance and a partial order on $S$-equivalence classes_, Topology Appl. 18 (1984), no. 2-3, 313–324.
  * [7] C. McA. Gordon, _Ribbon concordance of knots in the $3$-sphere_, Math. Ann. 257 (1981), no. 2, 157–170.
  * [8] C. McA. Gordon and R. A. Litherland, _On the signature of a link_ , Invent. Math. 47 (1978), no. 1, 53–69.
  * [9] Joshua Evan Greene, _Alternating links and definite surfaces_ , Duke Math. J. 166 (2017), no. 11, 2133–2151, With an appendix by András Juhász and Marc Lackenby.
  * [10] Joshua Evan Greene and Brendan Owens, _Alternating links, rational balls, and cube tilings_ , 2023.
  * [11] Mikami Hirasawa and Makoto Sakuma, _Minimal genus Seifert surfaces for alternating links_ , KNOTS ’96 (Tokyo), World Sci. Publ., River Edge, NJ, 1997, pp. 383–394.
  * [12] Jonathan Johnson, _Residual torsion-free nilpotence, bi-orderability and two-bridge links_ , 2021.
  * [13] Wilhelm Magnus, Abraham Karrass, and Donald Solitar, _Combinatorial group theory_ , second ed., Dover Publications, Inc., Mineola, NY, 2004.
  * [14] E. J. Mayland, Jr. and Kunio Murasugi, _On a structural property of the groups of alternating links_ , Canadian J. Math. 28 (1976), no. 3, 568–588.
  * [15] W. Menasco, _Closed incompressible surfaces in alternating knot and link complements_ , Topology 23 (1984), no. 1, 37–44.
  * [16] John W. Milnor, _Infinite cyclic coverings_ , Conference on the Topology of Manifolds (Michigan State Univ., E. Lansing, Mich., 1967), Prindle, Weber & Schmidt, Boston, Mass., 1968, pp. 115–133.
  * [17] Katura Miyazaki, _Band-sums are ribbon concordant to the connected sum_ , Proc. Amer. Math. Soc. 126 (1998), no. 11, 3401–3406.
  * [18] Katura Miyazaki, _A note on genera of band sums that are fibered_ , J. Knot Theory Ramifications 27 (2018), no. 12, 1871002, 3.
  * [19] by same author, _When is a band-connected sum equal to the connected sum?_ , Topology Appl. 272 (2020), 107071, 5.
  * [20] Takuji Nakamura, _Positive alternating links are positively alternating_ , J. Knot Theory Ramifications 9 (2000), no. 1, 107–112.
  * [21] Martin Scharlemann, _Sutured manifolds and generalized Thurston norms_ , J. Differential Geom. 29 (1989), no. 3, 557–614.
  * [22] Ian Zemke, _Knot Floer homology obstructs ribbon concordance_ , Ann. of Math. (2) 190 (2019), no. 3, 931–947.


11institutetext: 1Shenzhen Key Lab of Computer Vision and Pattern Recognition,
SIAT-SenseTime Joint Lab, Shenzhen Institutes of Advanced Technology, Chinese
Academy of Sciences
2 University of Chinese Academy of Sciences 3 University of Macau
4 Shanghai AI Laboratory, Shanghai, China
11email: {xn.liu, jf.hu1<EMAIL_ADDRESS><EMAIL_ADDRESS>
# UDC-UNet: Under-Display Camera Image Restoration via U-shape Dynamic Network

Xina Liu∗ 1122 Jinfan Hu∗ 1122 Xiangyu Chen 113344 Chao Dong† 1144

###### Abstract

Under-Display Camera (UDC) has been widely exploited to help smartphones
realize full-screen displays. However, as the screen could inevitably affect
the light propagation process, the images captured by the UDC system usually
contain flare, haze, blur, and noise. Particularly, flare and blur in UDC
images could severely deteriorate the user experience in high dynamic range
(HDR) scenes. In this paper, we propose a new deep model, namely UDC-UNet, to
address the UDC image restoration problem with an estimated PSF in HDR scenes.
Our network consists of three parts, including a U-shape base network to
utilize multi-scale information, a condition branch to perform spatially
variant modulation, and a kernel branch to leverage the prior knowledge of the
PSF. According to the characteristics of HDR data, we additionally design a
tone mapping loss to stabilize network optimization and achieve better visual
quality. Experimental results show that the proposed UDC-UNet outperforms the
state-of-the-art methods in quantitative and qualitative comparisons. Our
approach won second place in the UDC image restoration track of the MIPI
challenge. Codes and models are available at
https://github.com/J-FHu/UDCUNet.††* Equal contributions, $\dagger$
Corresponding author

##  1 Introduction

As a new design for smartphones, Under-Display Camera (UDC) systems can
provide a bezel-less and notch-free viewing experience, and attracts much
attention from the industry. However, it is difficult to preserve the full
functionality of the imaging sensor under a display. As the propagation of
light in the imaging process is inevitably affected, various forms of optical
diffraction and interference would be generated. The image captured by UDC
often suffers from noise, flare, haze, and blurry artifacts. Besides, UDC
images are often captured under a high dynamic range in real scenes.
Therefore, severe over-saturation could occur in highlight regions, as shown
in Fig. 1.

Image restoration task aim to restore the clean image from its degraded
version, such as denoising [51, 52, 16], deraining [30, 56, 29], deblurring
[23, 27], super-resolution [7, 54, 4], and HDR reconstruction [10, 3]. Similar
to these tasks, UDC image restoration aims at reconstructing the degraded
image generated by the UDC system. To model the complicated degradation
process of the UDC imaging system, existing works [55, 11] propose to utilize
a particular diffraction blur kernel, i.e. Point Spread Function (PSF). Then
UDC image restoration can be regarded as an inversion problem for a measured
PSF. Though existing methods have made significant progress in this task,
their performance is still limited.

Figure 1: Visual results of under-display camera image restoration. Affected
by the UDC imaging mechanism, artifacts including flare, blur, and haze will
appear in UDC images. Our method generates visually pleasing results by
alleviating the artifacts.

In this work, we propose UDC-UNet to restore images captured by UDC in HDR
scenes. The framework consists of a base network, a condition branch, and a
kernel branch. First, we exploit the commonly used U-shape structure to build
the base network for utilizing the multi-scale information hierarchically.
Then, to achieve the spatially variant modulation for different regions with
different exposures, we adopt several spatial feature transform (SFT) layers
[46] to build the condition branch as the design in [3]. Additionally, the
prior knowledge of an accurately measure PSF has been proven to be useful for
the restoration process [11]. Hence, we also add a kernel branch that uses an
estimated PSF to refine the intermediate features. Besides, considering the
data characteristic of HDR images, we design a new tone mapping loss to
normalize image values into [0,1]. It can not only balance the effects of
pixels with different intensities but also stabilize the training process.
Experimental results show that our method surpasses the state-of-the-art
methods in both quantitative performance and visual quality. Our approach can
generate visually pleasing results without artifacts even in the over-
saturated regions, as shown in Fig. 1. The proposed UDC-UNet won second place
in the MIPI UDC image restoration challenge.

In summary, our contributions are three-fold:

1) We propose a new deep network, UDC-UNet, to address the UDC image
restoration problem. 2) We design a tone mapping loss to address the UDC image
restoration problem in HDR scenes. 3) Experiments demonstrate the superiority
of our method in quantitative performance and visual quality.

##  2 Related Work

###  2.1 UDC Restoration

Under-display camera is a new imaging system that mounts a display screen on
top of a traditional digital camera lens. While providing a better user
experience of viewing and interaction, the UDC system may sacrifice the
quality of photography. Due to low light transmission rate and diffraction
effects in the UDC system, significant quality degradations may appear in UDC
images, including flare, haze, blur, and noise. Some works [39, 45] have
analyzed the imaging principle of the UDC system and attempted to restore UDC
images with deconvolutional-based methods like Wiener Filter [14]. Qin et al.
[38] demonstrated the issue of see-through image blurring of transparent
organic light-emitting diodes display. Kwon et al. [24] paid attention to the
analysis of transparent displays. Zhou et al. [55] recovered the original
signal from the point-spread-function-convoluted image. Feng et al. [11]
treated the UDC image restoration as a non-blind inverse problem for an
accurately measured PSF.

Until now, there is only few works to directly solve the problem of UDC image
restoration. MCIS [55] modeled the image formation pipeline of UDC and
proposed to address the image restoration of UDC using a Deconvolution-based
Pipeline (DeP) and data-driven learning-based methods. Although this variant
of UNet solved many problems of UDC restoration, it lacks consideration of HDR
in data generation and PSF measurement. A newly proposed structure named
DISCNet [11] considered high dynamic range (HDR) and measured the real-world
PSF of the UDC system. Besides, DISCNet regarded PSF as useful domain
knowledge and added a separate branch to fuse its information. Experiments
have shown the effectiveness of doing so in removing diffraction image
artifacts.

However, in HDR scenes, existing methods still could not perform well in both
blurring of unsaturated regions and flare in over-saturated areas. UDC images
in this scenario have different exposures and brightness. Previous networks
apply the same filter weights across all regions, whose capability are no
longer powerful enough in this task. Spatial feature transform (SFT) layer is
a excellent design provided in SFTGAN [46] that can achieve affine
transformation to the intermediate features and perform a spatially variant
modulation. A lot of works [25, 42, 15] have proven the effectiveness of the
SFT layer, and especially, HDRUNet [3] demonstrated the effectiveness of the
module for HDR reconstruction. Thus, we adopt exploit SFT in our network to
process the various patterns across the image. Additionally, since Feng et al.
[11] presented that the dynamic convolutions generated by the measured PSF
could also benefit the UDC image restoration task in HDR scenes, we utilize
this mechanism in our design to further refine the intermediate features.

###  2.2 Image Restoration

Image restoration aims to reconstruct a high-quality image from its degraded
version. Prior to the deep-learning era, many traditional image processing
methods [1, 17, 43, 44, 19, 36] have been proposed for image restoration.
Since the first deep learning method was successfully applied in low-level
vision tasks [8], a flurry of approaches have been developed for various image
restoration problems, including super-resolution [9, 54, 22, 4], denoising
[51, 52, 16], deraining [30, 56], deblurring [23, 27], and HDR reconstruction
[10, 3].

As one of the most important task in the field of image restoration, image
super-resolution attracts the attention of numerous researchers, and thus a
series of methods [9, 54, 47, 32, 31, 4] have been proposed and made great
progress. Denoising is a fundamental task for image restoration, and some good
methods have been designed to deal with this problem [51, 52, 16, 26, 33].
Deblurring, deraining and HDR reconstruction are also practical problems in
the image restoration field. Deblurring methods aim to address the image blur
caused by camera shake, object motion, and out-of-focus [23, 27, 13, 53, 37,
6]. Deraining aims at removing the rain streaks from the degraded images,
which is also a widely studied problem in image restoration [30, 56, 12, 29,
40]. Reconstructing LDR images to their HDR version can greatly improve the
visual quality of images and viewing experience for users. There are also many
works focusing on this task [10, 3, 34, 20, 5].

Since UNet [41] first proposed the U-shape network for biomedical image
segmentation, this design is widely used in image restoration task. For
example, Cho et al [6] introduced a U-shape network to deal with image
deblurring. Chen et al [10, 3] also proposed UNet-style networks for single
image HDR reconstruction. As U-shape network can better utilize multi-scale
information and save computations by deploying multiple down-sampling and up-
sampling operations, we also exploit this structure for UDC image restoration
task. With the development of large models and the pre-training technique,
many works propose to address image restoration problems using Transformer
[48, 32, 49] and try to deal with multiple image restoration task
simultaneously [2, 28]. However, there are still few methods for UDC image
restoration.

##  3 Methodology

###  3.1 Problem Formulation

As mentioned in [11], a real-world UDC image formation model with multiple
types of degradations can be formulated as

$\hat{y}=\phi[C(x*k+n)],$ (1)

where $x$ represents the real HDR scene irradiance. $k$ is the known
convolution kernel, commonly referred to as the PSF for this task. “$*$”
denotes the convolution operation and $n$ represents the camera noise.
$C(\cdot)$ denotes the clipping operation for the digital sensor r with
limited dynamic range and $\phi(\cdot)$ represents the non-linear tone-mapping
function.

Following [11], we treat this task as a non-blind image restoration problem,
where the degraded image $\hat{y}$ and the estimated PSF $k$ are given to
restore the clean image. Hence, our proposed network $f$ can be defined as

$x^{\prime}=f(\hat{y},k;\theta),$ (2)

where $x^{\prime}$ denotes the predicted image and $\theta$ represents the
network parameters.

###  3.2 Network Structure

The overall architecture of the proposed network consists of three components
$–$ a base network, a condition branch, and a kernel branch as shown in Fig.
2,

Figure 2: The overall architecture of UDC-UNet. It consists of a U-shape base
network, a condition branch, and a kernel branch.

####  3.2.1 Base Network.

A U-shape structure is adopted to build the base network. Its effectiveness
has been proven in many image restoration methods [10, 3, 50, 48]. This kind
of structure helps the network make full use of the hierarchical multi-scale
information from low-level features to high-level features. The shallow layers
in the network can gradually extract the features with a growing receptive
field and map the input image to high-dimensional representation. For the deep
layers, features can be reconstructed step by step from the encoded
representation. Benefiting from the skip connections, shallow features, and
deep features can be combined organically. For the basic blocks in the base
network, we simply exploit the common residual block with two $3\times 3$
convolutions.

####  3.2.2 Condition Branch.

The key to UDC image restoration in HDR scenes is to deal with the blurring of
unsaturated regions and address the flare in over-saturated areas. Traditional
convolution filters apply the same filter weights across the whole images,
which are inefficient to perform such a spatially variant mapping. HDR-UNet
[3] proposes to leverage the SFT layer [46] to achieve the spatially variant
manipulation for HDR reconstruction with denoising and dequantization. In our
approach, we also adopt this module to build the condition branch for
adaptively modulating the intermediate features in the base network.

As demonstrated in Fig. 2, we use the input image to generate conditional maps
with different resolutions. Then, these conditions are applied to the features
in the base network through the SFT layers. The operation of the SFT layer can
be written as

$SFT(x)=\alpha\odot x+\beta,$ (3)

where “$\odot$” denotes the element-wise multiplication.
$x\in\mathbb{R}^{C\times H\times W}$ is the modulated intermediate features in
the base network. $\alpha\in\mathbb{R}^{C\times H\times W}$ and
$\beta\in\mathbb{R}^{C\times H\times W}$ are two modulation coefficient maps
generated by the condition branch. By using such a modulation mechanism, the
proposed network can easily perform spatially variant mapping for different
regions. Experimental results in Sec. 4.2 shows the effectiveness of this
branch for the UDC image restoration task.

####  3.2.3 Kernel Branch.

Feng et al. [11] have demonstrated that the PSF can provide useful prior
knowledge for UDC image restoration. Thus, we exploit a similar approach to
leverage the PSF to further refine the intermediate features in our base
network. Following [15, 11], we first project the PSF onto a $b$-dimensional
vector as the kernel code. Then, we stretch the kernel code into $H\times
W\times b$ features and concatenate it with the degraded image of size
$H\times W\times C$ to obtain the conditional input of size $H\times
W\times(b+C)$. Through the kernel branch, the conditional inputs are mapped
into scale-specific kernel features. For the modulated features of $h\times
w\times c$, the generated kernel feature is of size $h\times w\times ck^{2}$.
Then, the dynamic convolutions are performed on each pixel as

$F_{i,j,c}=K_{i,j,c}\cdot M_{i,j,c},$ (4)

where $K_{i,j,c}$ represents the $k\times k$ filter reshaped from the kernel
feature at position $(i,j,c)$ with the size of $1\times 1\times k^{2}$.
$M_{i,j,c}$ and $F_{i,j,c}$ denote the $k\times k$ patch centered at $(i,j,c)$
of the modulated features and the element at $(i,j,c)$ of the output feature.
“ $\cdot$ ” means the inner product operation. Note that we directly use the
measured PSF provided by [11] in this paper.

###  3.3 Loss Function

Traditional loss functions for image restoration task, such as $\ell_{1}$ and
$\ell_{2}$ loss, assign the same weight to all pixels regardless of intensity.
But for images captured in HDR scenes, such loss function will make the
network focus on those pixels with large intensity, resulting in an unstable
training process and poor performance. A common practice is to design a
specific loss function according to the tone mapping function [3, 35].
Inspired by these works, we propose a ${Mapping}_{-}\ell_{1}$ loss to optimize
the network, which is formulated as

${Mapping}_{-}\ell_{1}(Y,X)=|{Mapping}(Y)-{Mapping}(X)|,$ (5)

where $Y$ represents the predicted result and $X$ means the corresponding
ground truth. For the tone mapping function, ${Mapping}(I)=I/(I+0.25)$.

##  4 Experiments

###  4.1 Experimental Setup

####  4.1.1 Dataset.

UDC images and their corresponding ground-truth images in real scenes are
currently difficult to obtain. Therefore, Feng et al. in [11] propose to use
the PSF to simulate the degraded UDC images from the clean images. In the MIPI
UDC image restoration
challenge111https://codalab.lisn.upsaclay.fr/competitions/4874#participate,
132 HDR panorama images with the spatial size of 8196 $\times$ 4096 from the
HDRI Haven dataset222https://hdrihaven.com/hdris/ are exploited to generate
the synthetic data. This dataset contains images captured in various indoor
and outdoor, day and night, nature and urban scenes. The images are first
projected to the regular perspective and then cropped into 800 $\times$ 800
image patches. Each of these patches is degraded by using Eqn. (1) to obtain
the simulated UDC images. As the result, 2016 image pairs are available for
training. There are also 40 image pairs left as validation set and testing
set.

####  4.1.2 Implementation Details.

During the training phase, 256 $\times$ 256 patches randomly cropped from the
original images are fed into the network. The mini-batch size is set to 32 and
the whole network is trained for $6\times 10^{5}$ iterations. Both the number
of residual blocks in the condition branch and kernel branch are set to 2,
while the number of residual SFT blocks in the base network is set to [2, 2,
2, 8, 2, 2, 2], respectively. We use the exponential moving average strategy
during the training. The number of channels $C$ is set to 32 in our UDC-UNet.
We also provide a small version of our method, UDC-UNetS, by reducing the
channel number $C$ to 20. The initialization strategy [18] and Adam optimizer
[21] are adapted for training. The learning rate is initialized as $2\times
10^{-4}$ and decayed with a cosine annealing schedule, where
$\eta_{min}=1\times 10^{-7}$ and $\eta_{max}=2\times 10^{-4}$. Furthermore,
the learning rate is restarted at $[5\times 10^{4},1.5\times 10^{5},3\times
10^{5},4.5\times 10^{5}]$ iterations. For the evaluation metrics, PSNR, SSIM,
and LPIPS are calculated in the tone-mapped images through the tone mapping
function ($\operatorname{\textit{Mapping}}(I)=I/(I+0.25)$) provided by the
challenge.

###  4.2 Ablation Study

In this part, we first conduct experiments to explore the effectiveness of the
key modules. Then we investigate the influence of different conditional inputs
on the kernel branch. Finally, we study the effects of different loss
functions. Note that for fast validation, we use the UDC-UNet and set the
batch size as 6 for all experiments in this section.

Table 1: Ablation study of the key modules in UDC-UNet. Models | (a) | (b) | (c) | (d) | (e) (Ours)
---|---|---|---|---|---
Skip Connections | ✘ | ✔ | ✔ | ✔ | ✔
Condition Branch | ✘ | ✘ | ✘ | ✔ | ✔
Kernel Branch | ✘ | ✘ | ✔ | ✘ | ✔
PSNR(dB) | 42.19 | 44.50 | 44.58 | 45.23 | 45.37
SSIM | 0.9884 | 0.9897 | 0.9893 | 0.9897 | 0.9898
LPIPS | 0.0164 | 0.0155 | 0.0157 | 0.0166 | 0.0162
|  |  |  |  |

####  4.2.1 Effectiveness of Key Modules.

As our approach consists of three components – the base network, the condition
branch, and the kernel branch, we conduct ablation experiments to demonstrate
the effectiveness of these modules. In addition, we emphasize that the skip
connections (all the dotted lines in the base network shown in Fig. 2) in the
base network have significant positive effects on performance. Our base
network without any skip connections and extra branches can obtain the
quantitative performance of more than 42dB as shown in Tab. 1, demonstrating
the effectiveness of the U-shape structure for this task. By comparing models
(a) and (b), we can observe that the skip connections bring more than 2dB
performance gain. It presents the significance of the skip connection in the
base network. The models with the condition branch (d,e) outperform the models
without this branch (b,c) by more than 0.7dB, illustrating the necessity of
the condition branch. The kernel branch also brings a considerable performance
gain by comparing the models (c,e) and models (b,d). Combining these key
modules, our method achieves the highest PSNR and SSIM.

Table 2: Quantitative results of different conditional inputs for the kernel branch. Conditional Inputs | PSNR(dB) | SSIM | LPIPS
---|---|---|---
(a) None | 45.23 | 0.9897 | 0.0166
(b) Image | 45.17 | 0.9896 | 0.0162
(c) PSF | 45.26 | 0.9895 | 0.0166
(d) Image+PSF | 45.37 | 0.9898 | 0.0162
|  |  |

####  4.2.2 Different Conditional Inputs of the Kernel Branch.

An intuitive method for introducing the PSF prior is to directly use the
estimated PSF to generate the dynamic filters in the kernel branch.
Nonetheless, experiments in [11] show that utilizing the combination of the
PSF and the degraded image to generate filters can bring more performance
improvement. Thus, we also conduct experiments to investigate the different
conditional inputs of the kernel branch. As shown in Tab. 2, using only the
degraded image or the PSF does not bring obvious performance gain and even
causes a little performance drop. However, when using the image and the PSF
simultaneously as the conditional input, the model obtains a nonnegligible
performance improvement.

Table 3: Quantitative results of our method using different loss functions. Losses | PSNR(dB) | SSIM | LPIPS
---|---|---|---
$\ell_{1}$ | 41.30 | 0.9812 | 0.0301
$\operatorname{\textit{Mapping}}_{-}\ell_{2}$ | 40.19 | 0.9838 | 0.0238
$\operatorname{\textit{Mapping}}_{-}\ell_{1}$ | 45.37 | 0.9898 | 0.0162
|  |  |

####  4.2.3 Effects of Different Loss Functions.

We also conduct experiments to show the effects of different loss functions,
including the traditional $\ell_{1}$ loss, the proposed
$\operatorname{\textit{Mapping}}_{-}\ell_{1}$ loss, and $\ell_{2}$ loss
calculated after the same tone mapping function, denoting as the
$\operatorname{\textit{Mapping}}_{-}\ell_{2}$ loss. As depicted in Tab. 3, we
can find that our $\operatorname{\textit{Mapping}}_{-}\ell_{1}$ brings a large
quantitative performance improvement compared to the traditional $\ell_{1}$
loss and also outperforms the $\operatorname{\textit{Mapping}}_{-}\ell_{2}$
loss by a large margin. The results show that the loss function significantly
affects the optimization of the network for this task.

We also provide the visual comparisons of models using different loss
functions, as shown in Fig. 3. We can observe that $\ell_{1}$ loss and
$\operatorname{\textit{Mapping}}_{-}\ell_{2}$ loss make the network focus more
on the over-saturated regions, resulting in severe artifacts at these areas.
We visualize the tone mapping function in Fig. 4 to further illustrate the
effect of our proposed loss. Since the pixel intensities of the used HDR data
are in the range [0,500], the large values could significantly affect network
optimization and easily make the network focus on the over-saturated regions.
After the tone mapping function, the pixel values can be normalized to the
range of [0,1). An intuitive manifestation is that values greater than 1 will
be compressed to (0.8,1] through the tone mapping function. Hence, the
proposed $\operatorname{\textit{Mapping}}_{-}\ell_{1}$ loss can balance the
influence of different regions with different exposures for network
optimization.

Input

PSNR/SSIM

$\ell_{1}$

32.09dB/0.9594

$\operatorname{\textit{Mapping}}_{-}\ell_{2}$

36.47dB/0.9778

$\operatorname{\textit{Mapping}}_{-}\ell_{1}$

38.18dB/0.9831

Figure 3: The visual comparisons of models using different loss functions. The
bottom row presents the residual maps, E =
$\operatorname{\textit{Mapping}}(|Y-X|)$, where $Y$ means the generated output
and $X$ indicates the ground truth.

$I\in[0,1]\mapsto\operatorname{\textit{Mapping}}(I)\in[0,0.8]$,
$I\in(1,500]\mapsto\operatorname{\textit{Mapping}}(I)\in(0.8,1)$

Figure 4: Visualization of the tone mapping function, i.e.,
$\operatorname{\textit{Mapping}}(I)=I/(I+0.25)$.

###  4.3 Comparison with State-of-the-art Methods

To evaluate the performance of our UDC-UNet, we conduct experiments on the
simulated data and make comparisons with several state-of-the-art methods.
Since the task of UDC image restoration is a newly-proposed task, there are
not many related works available to compare. Therefore, we select three
representative methods for comparison, including DISCNet [11] for UDC image
restoration, Uformer [48] for general image restoration tasks, and HDRUNet [3]
for HDR image reconstruction. Specifically, we retrain the compared methods
using the official codes on the same data with the same
$\operatorname{\textit{Mapping}}_{-}\ell_{1}$ loss. Note that we use the small
version of Uformer with channel number 32 to save computations. To further
demonstrate the effectiveness of our method, we also provide the results of
the small version of our method, UDC-UNetS, by reducing the channel number $C$
to 20.

Table 4: Quantitative comparisons of our method with state-of-the-art approaches. The best values are in bold, and the second best values are underlined. Methods | PSNR(dB) | SSIM | LPIPS | Params | GMACs
---|---|---|---|---|---
Uformer [48] | 37.97 | 0.9784 | 0.0285 | 20.0M | 490
HDRUNet [3] | 40.23 | 0.9832 | 0.0240 | 1.7M | 229
DISCNet [11] | 39.89 | 0.9864 | 0.0152 | 3.8M | 367
UDC-UNetS (Ours) | 45.98 | 0.9913 | 0.0128 | 5.7M | 169
UDC-UNet (Ours) | 47.18 | 0.9927 | 0.0100 | 14.0M | 402
|  |  |  |  |

####  4.3.1 Quantitative Results.

The quantitative results are provided in Tab. 4. We can observe that UDC-UNet
achieves the best performance and significantly outperforms the compared
state-of-the-art methods by more than 7 dB. Considering the computations, we
also provide the results of a smaller variant UDC-UNetS. We can observe that
UDC-NetS also obtains considerable performance with relatively fewer
parameters and the least computation. The GMACs of all the methods are
calculated on the input size of 800 $\times$ 800\. Note that due to more
downsampling operations, our methods have much fewer GMACs than the compared
approaches.

Figure 5: Visual comparisons of our method with state-of-the-art approaches on
the validation set.

####  4.3.2 Visual Comparison.

We provide the visual results of our method and state-of-the-art approaches in
Fig. 5. Compared to other methods, our UDC-UNet obtains visually pleasing
results with clean textures and little artifacts, especially on the over-
saturated regions. Besides, the results generated by UDC-UNetS also have
relatively fewer artifacts compared to other methods. To further demonstrate
the superiority of UDC-UNet, we also provide visual comparisons with current
models on the testing data. It can be seen that our method obtains better
visual quality than the several compared approaches.

Figure 6: Visual comparisons of our method with state-of-the-art approaches on
the testing set.

##  5 Results of the MIPI UDC Image Restoration Challenge

We participate in the mobile intelligent photography and imaging (MIPI)
challenge333http://mipi-challenge.org/ and won second place in the UDC image
restoration track. The top 5 results are depicted in Tab. 5.

Table 5: Top 5 results of the MIPI UDC image restoration challenge. Teams | PSNR(dB) | SSIM | LPIPS
---|---|---|---
USTC_WXYZ | 48.4776 | 0.9934 | 0.0093
XPixel Group (ours) | 47.7821 | 0.9913 | 0.0122
SRC-B | 46.9227 | 0.9929 | 0.0098
MIALGO | 46.1177 | 0.9892 | 0.0159
LVGroup_HFUT | 45.8722 | 0.9920 | 0.0109
|  |  |

##  6 Conclusions

In this paper, we propose UDC-UNet to address UDC image restoration task in
HDR scenes. The designed network consists of a U-shape base network to utilize
multi-scale information, a condition branch to perform spatially variant
modulation, and a kernel branch to leverage the prior knowledge of the
measured PSF. In addition, we design a tone mapping loss to balance the
effects of pixels with different intensities. Experiments show that our method
obtains the best quantitative performance and visual quality compared to
state-of-the-art approaches. We participated in the MIPI challenge and won
second place in the UDC image restoration track.

####  6.0.1 Acknowledgements.

This work is partially supported by National Natural Science Foundation of
China (61906184, U1913210), and the Shanghai Committee of Science and
Technology, China (Grant No. 21DZ1100100).

## References

  * [1] Calvetti, D., Reichel, L., Zhang, Q.: Iterative solution methods for large linear discrete ill-posed problems. In: Applied and computational control, signals, and circuits, pp. 313–367. Springer (1999)
  * [2] Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., Gao, W.: Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 12299–12310 (2021)
  * [3] Chen, X., Liu, Y., Zhang, Z., Qiao, Y., Dong, C.: Hdrunet: Single image hdr reconstruction with denoising and dequantization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 354–363 (2021)
  * [4] Chen, X., Wang, X., Zhou, J., Dong, C.: Activating more pixels in image super-resolution transformer. arXiv preprint arXiv:2205.04437 (2022)
  * [5] Chen, X., Zhang, Z., Ren, J.S., Tian, L., Qiao, Y., Dong, C.: A new journey from sdrtv to hdrtv. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 4500–4509 (2021)
  * [6] Cho, S.J., Ji, S.W., Hong, J.P., Jung, S.W., Ko, S.J.: Rethinking coarse-to-fine approach in single image deblurring. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4641–4650 (2021)
  * [7] Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. pp. 184–199. Springer (2014)
  * [8] Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. pp. 184–199. Springer (2014)
  * [9] Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. pp. 391–407. Springer (2016)
  * [10] Eilertsen, G., Kronander, J., Denes, G., Mantiuk, R.K., Unger, J.: Hdr image reconstruction from a single exposure using deep cnns. ACM Transactions on Graphics 36(6), 1–15 (2017)
  * [11] Feng, R., Li, C., Chen, H., Li, S., Loy, C.C., Gu, J.: Removing diffraction image artifacts in under-display camera via dynamic skip connection network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 662–671 (2021)
  * [12] Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 3855–3863 (2017)
  * [13] Gao, H., Tao, X., Shen, X., Jia, J.: Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3848–3856 (2019)
  * [14] Goldstein, J.S., Reed, I.S., Scharf, L.L.: A multistage representation of the wiener filter based on orthogonal projections. IEEE Transactions on Information Theory 44(7), 2943–2959 (1998)
  * [15] Gu, J., Lu, H., Zuo, W., Dong, C.: Blind super-resolution with iterative kernel correction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1604–1613 (2019)
  * [16] Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 1712–1722 (2019)
  * [17] Hansen, P.C.: Regularization tools: A matlab package for analysis and solution of discrete ill-posed problems. Numerical algorithms 6(1), 1–35 (1994)
  * [18] He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international Conference on Computer Vision. pp. 1026–1034 (2015)
  * [19] Jauch, W.: The maximum-entropy method in charge-density studies. ii. general aspects of reliability. Acta Crystallographica Section A: Foundations of Crystallography 50(5), 650–652 (1994)
  * [20] Kim, S.Y., Oh, J., Kim, M.: Deep sr-itm: Joint learning of super-resolution and inverse tone-mapping for 4k uhd hdr applications. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 3116–3125 (2019)
  * [21] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [22] Kong, X., Liu, X., Gu, J., Qiao, Y., Dong, C.: Reflash dropout in image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6002–6012 (2022)
  * [23] Kupyn, O., Martyniuk, T., Wu, J., Wang, Z.: Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8878–8887 (2019)
  * [24] Kwon, H.J., Yang, C.M., Kim, M.C., Kim, C.W., Ahn, J.Y., Kim, P.R.: Modeling of luminance transition curve of transparent plastics on transparent oled displays. Electronic Imaging 2016(20), 1–4 (2016)
  * [25] Lee, C.H., Liu, Z., Wu, L., Luo, P.: Maskgan: Towards diverse and interactive facial image manipulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5549–5558 (2020)
  * [26] Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M., Aila, T.: Noise2noise: Learning image restoration without clean data. arXiv preprint arXiv:1803.04189 (2018)
  * [27] Li, L., Pan, J., Lai, W.S., Gao, C., Sang, N., Yang, M.H.: Blind image deblurring via deep discriminative priors. International journal of computer vision 127(8), 1025–1043 (2019)
  * [28] Li, W., Lu, X., Lu, J., Zhang, X., Jia, J.: On efficient transformer and image pre-training for low-level vision. arXiv preprint arXiv:2112.10175 (2021)
  * [29] Li, X., Wu, J., Lin, Z., Liu, H., Zha, H.: Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European conference on computer vision (ECCV). pp. 254–269 (2018)
  * [30] Li, Y., Tan, R.T., Guo, X., Lu, J., Brown, M.S.: Rain streak removal using layer priors. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2736–2744 (2016)
  * [31] Li, Z., Liu, Y., Chen, X., Cai, H., Gu, J., Qiao, Y., Dong, C.: Blueprint separable residual network for efficient image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 833–843 (2022)
  * [32] Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: Swinir: Image restoration using swin transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1833–1844 (2021)
  * [33] Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-cnn for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops. pp. 773–782 (2018)
  * [34] Liu, Y.L., Lai, W.S., Chen, Y.S., Kao, Y.L., Yang, M.H., Chuang, Y.Y., Huang, J.B.: Single-image hdr reconstruction by learning to reverse the camera pipeline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1651–1660 (2020)
  * [35] Liu, Z., Lin, W., Li, X., Rao, Q., Jiang, T., Han, M., Fan, H., Sun, J., Liu, S.: Adnet: Attention-guided deformable convolutional network for high dynamic range imaging. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 463–470 (2021)
  * [36] O’Sullivan, J.A., Blahut, R.E., Snyder, D.L.: Information-theoretic image formation. IEEE Transactions on Information Theory 44(6), 2094–2123 (1998)
  * [37] Park, D., Kang, D.U., Kim, J., Chun, S.Y.: Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: European Conference on Computer Vision. pp. 327–343. Springer (2020)
  * [38] Qin, Z., Tsai, Y.H., Yeh, Y.W., Huang, Y.P., Shieh, H.P.D.: See-through image blurring of transparent organic light-emitting diodes display: calculation method based on diffraction and analysis of pixel structures. Journal of Display Technology 12(11), 1242–1249 (2016)
  * [39] Qin, Z., Xie, J., Lin, F.C., Huang, Y.P., Shieh, H.P.D.: Evaluation of a transparent display’s pixel structure regarding subjective quality of diffracted see-through images. IEEE Photonics Journal 9(4), 1–14 (2017)
  * [40] Ren, D., Zuo, W., Hu, Q., Zhu, P., Meng, D.: Progressive image deraining networks: A better and simpler baseline. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3937–3946 (2019)
  * [41] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
  * [42] Shao, Y., Li, L., Ren, W., Gao, C., Sang, N.: Domain adaptation for image dehazing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 2808–2817 (2020)
  * [43] Starck, J.L., Murtagh, F.: Astronomical image and data analysis (2007)
  * [44] Starck, J.L., Pantin, E., Murtagh, F.: Deconvolution in astronomy: A review. Publications of the Astronomical Society of the Pacific 114(800), 1051 (2002)
  * [45] Tang, Q., Jiang, H., Mei, X., Hou, S., Liu, G., Li, Z.: 28-2: Study of the image blur through ffs lcd panel caused by diffraction for camera under panel. In: SID Symposium Digest of Technical Papers. vol. 51, pp. 406–409. Wiley Online Library (2020)
  * [46] Wang, X., Yu, K., Dong, C., Loy, C.C.: Recovering realistic texture in image super-resolution by deep spatial feature transform. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pp. 606–615 (2018)
  * [47] Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., Change Loy, C.: Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV) workshops. pp. 0–0 (2018)
  * [48] Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., Li, H.: Uformer: A general u-shaped transformer for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 17683–17693 (2022)
  * [49] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Restormer: Efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5728–5739 (2022)
  * [50] Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H.: Restormer: Efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 5728–5739 (2022)
  * [51] Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing 26(7), 3142–3155 (2017)
  * [52] Zhang, K., Zuo, W., Zhang, L.: Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Transactions on Image Processing 27(9), 4608–4622 (2018)
  * [53] Zhang, K., Luo, W., Zhong, Y., Ma, L., Stenger, B., Liu, W., Li, H.: Deblurring by realistic blurring. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 2737–2746 (2020)
  * [54] Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV). pp. 286–301 (2018)
  * [55] Zhou, Y., Ren, D., Emerton, N., Lim, S., Large, T.: Image restoration for under-display camera. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9179–9188 (2021)
  * [56] Zhu, L., Fu, C.W., Lischinski, D., Heng, P.A.: Joint bi-layer optimization for single-image rain streak removal. In: Proceedings of the IEEE international conference on computer vision. pp. 2526–2534 (2017)