# Localized modes in $\chi^{(2)}$ media with ${\cal PT}$-symmetric localized
potential

F. C. Moreira1,2, F. Kh. Abdullaev3, V. V. Konotop2 and A. V. Yulin2
1Universidade Federal de Alagoas, Campus A. C. Simões - Av. Lourival Melo
Mota, s/n, Cidade Universitária, Maceió - AL 57072-900, Brazil
2Centro de Física Teórica e Computacional and Departamento de Física,
Faculdade de Ciências, Universidade de Lisboa, Avenida Professor Gama Pinto 2,
Lisboa 1649-003, Portugal
3Instituto de Fisica Teórica, Universidade Estadual Paulista, R. Dr. Bento
Teobaldo Ferraz, 271 Barra Funda, São Paulo, CEP 01140-070, Brasil

###### Abstract

We study the existence and stability of solitons in the quadratic nonlinear
media with spatially localized ${\cal PT}$-symmetric modulation of the linear
refractive index. Families of stable one and two hump solitons are found. The
properties of nonlinear modes bifurcating from a linear limit of small
fundamental harmonic field are investigated. It is shown that the fundamental
branch, bifurcating from the linear mode of the fundamental harmonic is
limited in power. The power maximum decreases with the strength of the
imaginary part of the refractive index. The modes bifurcating from the linear
mode of the second harmonic can exist even above ${\cal PT}$ symmetry breaking
threshold. We found that the fundamental branch bifurcating from the linear
limit can undergo a secondary bifurcation colliding with a branch of two-hump
soliton solutions. The stability intervals for different values of the
propagation constant and gain/loss gradient are obtained. The examples of
dynamics and excitations of solitons obtained by numerical simulations are
also given.

###### pacs:

42.65.Tg, 42.65.Sf

##  I Introduction

Non-Hermitian Hamiltonians satisfying the parity-time (${\cal PT}$) symmetry
can have real eigenvalues Bender1 . While originally these ideas were
developed in the quantum-mechanical context, it became soon clear that new
broad applications they can find in optics. The first suggestions of the
optical analogues of the ${\cal PT}$ symmetric Hamiltonians was proposed in
Muga  and was based on a linear planar waveguide structure. ${\cal PT}$
symmetry effect predictions are confirmed in experiments with light
propagation in couplers with gain and loss Exp1 . Later on it was suggested to
explore nonlinear optical media obeying ${\cal PT}$-symmetry, and in
particular it was shown the possibility of soliton propagation in such media
Christodoulides1 . The ${\cal PT}$-symmetry was modeled by the refractive
index having symmetric real part and anti-symmetric imaginary part. A
particularly interesting realization of a ${\cal PT}$ symmetric modulation of
the refractive index is when losses and gain are localized in space, giving
rise to a ${\cal PT}$ symmetric localized impurity. Such impurities allow for
existence of localized (defect) modes. In the linear theory such modes were
studied for exactly integrable models in Znojil ; Ahmed , and their nonlinear
extension was reported in Christodoulides1 . Solitons supported by other
${\cal PT}$ symmetric defects were also reported for focusing Gaussian  and
defocusing defoc  media. Linear scattering by a ${\cal PT}$-symmetric
inhomogeneity and emerging of the related spectral singularities was described
in Mostafa_scattering . The effect of two and various randomly distributed
${\cal PT}$ symmetric impurities on the lattice dynamics was addressed in
Kottos . Switching of solitons in a unidirectional coupler using ${\cal
PT}$-symmetric defects was suggested in AKOS . Nonlinear modes in even more
sophisticated, double well ${\cal PT}$-symmetric potentials were studied
recently double_well .

All of the works mentioned above and devoted to nonlinear modes dealt with the
PT symmetric media possessing Kerr nonlinearity (see the list of recent works
on solitons in PT_sol ). It is a natural further step to address a possibility
of existence of defect modes and their stability in another class of widely
used optical media, which is are the $\chi^{(2)}$ materials. Solitons in
quadratic nonlinear media with conservative defects were investigated in
Clausen1 ; Clausen2 , where it was shown that solutions are dynamically stable
in the case of attractive impurities. In the present paper we study the
existence of solitons in the media with quadratic nonlinearity and localized
${\cal PT}$-symmetric potentials.

The paper is organized as follows. In Section II the model and statement of
the problem are formulated. The properties of localized modes for different
ratios between fundamental and second harmonics are studied in Sections III-V.
The stability and dynamics of localized solutions are considered in Section
VI.

##  II Statement of the problem

We consider the system

$\displaystyle i\frac{\partial
u_{1}}{\partial\zeta}+\frac{\partial^{2}u_{1}}{\partial\xi^{2}}+V\left(\frac{1}{\cosh^{2}\xi}+i\alpha\frac{\sinh\xi}{\cosh^{2}\xi}\right)u_{1}+2\overline{u}_{1}u_{2}=0,$
(1a) $\displaystyle i\frac{\partial
u_{2}}{\partial\zeta}+\frac{1}{2}\frac{\partial^{2}u_{2}}{\partial\xi^{2}}+2\left(\frac{V}{\cosh^{2}\xi}+q\right)u_{2}+u_{1}^{2}=0$
(1b)

describing spatial second-harmonic generation in a $\chi^{(2)}$ material with
localized modulation of the refractive index, $u_{1}$ and $u_{2}$ being the
dimensionless fields of the first and second harmonics, $\xi$ and $\zeta$ are
the dimensionless transverse and propagation coordinates, scaled to the
characteristic size of the modulation of the refractive index which is
characterized by the amplitude $V$. The mismatch in the propagation constants
of field components is described by $q$. We notice that experimentally the
introduced model can describe a medium with active dopants, typically having
rather narrow spectral resonances, i.e. affecting only a limited range of
frequencies. In particular, such impurities can induce gain and dissipation,
whose strengths is characterized by $\alpha$, only for one of the field
component, which in our case is the FF.

Before into the detail study of the system (1) we note, that in the standard
way solitonic solutions can be found in the analytical form in the limit of
large mismatch parameter $-q\gg 1$, when $u_{2}\approx-u_{1}^{2}/(2q)$ and the
system (1) reduces to the NLS equation with PT-symmetric potential for the
fundamental harmonic $u_{1}$

$i\frac{\partial
u_{1}}{\partial\zeta}+\frac{\partial^{2}u_{1}}{\partial\xi^{2}}+V(\frac{1}{\cosh^{2}\xi}+i\alpha\frac{\sinh\xi}{\cosh^{2}\xi})-\frac{1}{q}|u_{1}|^{2}u_{1}=0.$
(2)

For $q<0$ the bright soliton solution has the form Christodoulides1

$\displaystyle u_{1}=\sqrt{|q|A}\mbox{sech}(\xi)\exp[i\frac{\alpha
V}{3}\tan^{-1}(\sinh(\xi))+i\zeta],$ $\displaystyle
A=(2-V+\frac{\alpha^{2}V^{2}}{9})$ (3)

We are interested in the localized solutions

$u_{n}\left(\xi,\zeta\right)=w_{n}\left(\xi\right)e^{inb\zeta},\quad n=1,2,$
(4)

where $b$ is the propagation constant of the first harmonic and $w_{1,2}$
solves the system

$\frac{d^{2}w_{1}}{d\xi^{2}}+\left[V\left(\frac{1}{\cosh^{2}\xi}+i\alpha\frac{\sinh\xi}{\cosh^{2}\xi}\right)-b\right]w_{1}+2\overline{w}_{1}w_{2}=0,$
(5a)
$\frac{1}{2}\frac{d^{2}w_{2}}{d\xi^{2}}+2\left(V\frac{1}{\cosh^{2}\xi}+q-b\right)w_{2}+w_{1}^{2}=0.$
(5b)

subject to the zero boundary conditions $w_{1,2}\left(\xi\right)\to 0$ as
$|\xi|\to\infty$.

We restrict our consideration mainly to solutions bifurcating from the linear
limit, which is understood as a limit where at least one of the harmonics
vanishes. It follows from (5), that so defined linear limit does not
necessarily implies that both amplitudes $w_{1}$ and $w_{2}$ are infinitely
small. Indeed it is sufficient to require that only the amplitude of the
fundamental harmonic $w_{1}$ is infinitely small to consider the equations (5)
in the linear limit. That is why one can distinguish three different
bifurcations of the fundamental soliton solution from the linear limit:

(i) The amplitude of the second harmonic is of the order of the squared
amplitude of the first harmonic, i.e. is negligible compared to the amplitude
of the first harmonic

$\displaystyle w_{2}=O(w_{1}^{2})\qquad w_{1}\to 0$ (6)

(ii) The second harmonic is finite in the limit of negligible first harmonic

$\displaystyle w_{2}=O(1)\qquad w_{1}\to 0$ (7)

(iii) Both harmonics are of the same order

$\displaystyle w_{2}=O(w_{1})\qquad w_{1}\to 0$ (8)

As it follows from (6) and (8) the maximal intensities of both $w_{1}$ and
$w_{2}$ go to zero at the bifurcation point for cases (i) and (iii). However
in case (ii) the maximum of the absolute value of field $w_{2}$ goes to a
finite value when the solution approaches the bifurcation point. Which of the
cases is realized, depends on the parameters of the system, and in particular
on the mismatch $q$. Below we consider these three cases separately.

##  III Modes with negligible second harmonic in the linear limit.

Let us start with the conditions necessary for (6) to occur. In this limit the
nonlinear term in (5a) can be neglected and in the leading order we have the
eigenvalue problem

$L_{1,\alpha}w_{1l}=bw_{1l},$ (9a)
$L_{1,\alpha}=\frac{d^{2}}{d\xi^{2}}+V\left(\frac{1}{\cosh^{2}\xi}+i\alpha\frac{\sinh\xi}{\cosh^{2}\xi}\right).$
(9b)

Eq. (9b) has been studied before. Therefore below we only briefly outline the
features necessary for our analysis, referring to Znojil ; Ahmed  for more
details.

Figure 1: (Color online) Families of the solutions bifurcating from
$b_{1,0}=\eta_{1}^{2}$ in the case (i) for several values of $\alpha$. Thick
(thin) lines represent stable (unstable) solutions. Insertion shows the
branches for larger values of $\alpha$. The black circles represent solutions
studied in text. The parameters of the structure are $V=1/2$ and $q=0$ and
$\alpha_{cr}=1.5$.

For $V>0$ Eq. (9b) possesses localized solutions. When $\alpha$ is below the
${\cal PT}$-symmetry breaking threshold

$\alpha<\alpha_{cr}=1+\frac{1}{4V},$ (10)

the spectrum of Eq. (9b) has discrete real eigenvalues given by Midya

$b_{1,n}=(n-\eta_{1})^{2},\quad n=0,1,...<\eta_{1},$ (11)

where

$\eta_{1}=\frac{1}{2}\left(\sqrt{V(\alpha_{cr}-\alpha)}+\sqrt{V(\alpha+\alpha_{cr})}-1\right).$
(12)

Above the symmetry breaking point ($\alpha>\alpha_{cr}$) the eigenvalues of
the bound states are complex valued. We notice here that no fundamental branch
satisfying condition (6) with $\alpha>\alpha_{cr}$ was found.

Figure 2: (Color online) Families of the solutions bifurcating from
$b_{1,0}=\eta_{1}^{2}$ and corresponding to the case (i) for several values of
$q$. Thick (thin) lines represent stable (unstable) solutions. The parameters
of the structure are $V=1/2$ and $\alpha=1.44$.

Therefore from now on we concentrate only on the results for ${\cal
PT}$-symmetry preserving case (10). Moreover, our consideration will be
limited to nonlinear modes that bifurcate from the ground state of defect
potential in Eq. (9b), i.e. $n=0$. The respective eigenstate of the linear
problem (9b) reads Znojil ; Ahmed

$w_{1l}(\xi)=W_{1}\text{sech}^{\eta_{1}}(\xi)\exp\left[\frac{i}{2}\Theta\tan^{-1}(\sinh\xi)\right]$
(13)

where $W_{1}$ is a constant and

$\Theta=\sqrt{V(\alpha_{cr}-\alpha)}-\sqrt{V(\alpha+\alpha_{cr})}.$ (14)

Passing to (5b), in the small amplitude limit one can look for a solution with
$w_{1}\approx w_{1l}$, which plays the role of the inhomogeneous term in the
linear equation for $w_{2}$. However, to obtain the complete families of
solutions one has to consider both Eqs. (5). We did this using relaxation
Newton-Raphson method using the described linear solutions as the initial
ansatz.

Fig. 1 shows the dependence of the total power

$P=P_{1}+P_{2},\qquad P_{n}=n\int_{-\infty}^{\infty}|w_{n}|^{2}d\xi$ (15)

on the propagation constant with several values of $\alpha$. We observe that
the fundamental branches shown in Fig. 1 have a maximal in power $P_{M}$,
$P\leq P_{M}$. The value of $P_{M}$ is decreasing as $\alpha$ increases. We
also observed that $P_{M}\to 0$ as $\alpha\to\alpha_{cr}$, i.e. the
fundamental branch disappears.

Figure 3: (Color online) Spatial distributions of the intensities
$|w_{n}|^{2}$ (upper panels) and the currents $j_{n}$ (lower panels). The left
panels corresponds to a stable solution with $b=0.024$, as marked by a black
circle in Fig. 1, pertaining to the fundamental branch that bifurcates from
$b_{1,0}=0.034$, where $V=1/2$, $q=0$ and $\alpha=1.452$. The right panels
corresponds to a stable solution with $b=0.82$, pertaining to the fundamental
branch that bifurcates from $b_{1,0}=0.92$, where $V=2$, $q=0$ and
$\alpha=0.5$. Shadowed domains show the localized impurity (darker areas
represent higher values of its real part, $V\text{sech}^{2}\xi$.

For a given $\alpha$, the position of $P_{M}$ in respect to $b$ approaches
$b=0$ as $q$ decreases, as one can see in Fig. 2. The position of $P_{M}$
moves to the right in the case $q>0$. One can also see in Fig. 2 that in this
case the branch ends in $b=-q$ because the mismatch shifts the position of the
continuum spectrum of the linear part of (5b) and $w_{2}$ becomes delocalized.
There is no low amplitude linear limit for $w_{1}$ in this case.

In Fig. 3 we show the typical distributions of the fields $w_{n}$ and the
real-valued currents $j_{n}$ defined as ($n=1,2$)

$j_{n}(\xi)=|w_{n}|^{2}\frac{d\theta_{n}}{d\xi},\qquad\theta_{n}(\xi)=\text{arg}w_{n}(\xi).$
(16)

By construction $|w_{n}|^{2}$ and $j_{n}$ are even functions. The effective
width of the intensities of modes may significantly exceed the size of the
impurity, particularly in the modes closer to the edge of the continuous
spectrum (the left panels of Fig. 3).

We note that in the left panels of Fig. 3 the solution has a relatively small
amplitude. This is a peculiarity of the chosen strength of the potential (it
was $V=1/2$, solitons with larger amplitudes were found to be unstable). While
this potential (ensuring the existence of only one linear defect level in the
localized potential) is used below along the text, in the right panels of Fig.
3 we show a higher amplitude soliton for the potential well having the width
$V=2$ (and the parameters $\alpha=0.5$ and $q=0$).

##  IV Nonlinear modes without linear limit.

In this section we investigate the case when the second harmonic remains
finite at $w_{1}\rightarrow 0$. Then one can neglect the nonlinear term
$w_{1}^{2}$ in (5b) reducing it to the well known linear eigenvalue problem
(see e.g. Landau )

$L_{2}w_{2l}=2bw_{2l},$ (17a)
$L_{2}=\frac{1}{2}\frac{d^{2}}{d\xi^{2}}+2\left(V\frac{1}{\cosh^{2}\xi}+q\right),$
(17b)

whose eigenvalues are

$b_{2,n}=\frac{\left(n-\eta_{2}\right)^{2}}{4}+q,\quad
n<0,1,...,\eta_{2}=\sqrt{\frac{1}{4}+4V}-\frac{1}{2}$ (18)

Here we again consider the case when there is only one localized mode. This
leads to the requirement that $\eta_{2}\leq 1$ and consequently $V\leq 1/2$.
The corresponding eigenfunction of $b_{2,0}$ reads

$w_{2,0}(\xi)=W_{2}\text{sech}^{\eta_{2}}\xi.$ (19)

where $W_{2}$ is some constant which must be determined.

Figure 4: (Color online) Left panel: The eigenvalues $W_{2}$ of (20b). The
parameters of the structure are $V=1/2$, $\alpha=1.4$ and $q=0$. Insertion
shows the first few eigenvalues in detail. Right panel: The lowest
$\left|W_{2}\right|$ of (20b) as a function of the mismatch $q$ of the lowest
$P$ branch. The insertion shows how the minimum value of $W_{2}$ reaches zero
at $q=q_{0}$.

This can be done from the condition that the propagation constants in the
equations (17b) and (5a) are the same. In the vicinity of the bifurcation
point one can approximate $w_{2}\approx w_{2l}$, i.e., Eq. (5a) can be
approximated by the following linear system

$L\left(\begin{array}[c]{c}\bar{w}_{1}\\\
w_{1}\end{array}\right)=W_{2}\left(\begin{array}[c]{c}\bar{w}_{1}\\\
w_{1}\end{array}\right),$ (20a)
$L=-\frac{1}{2}\cosh^{\eta_{2}}(\xi)\left(\begin{array}[c]{cc}0&L_{1}-b_{2,0}\\\
\bar{L}_{1}-b_{2,0}&0\end{array}\right).$ (20b)

Figure 5: (Color online) The power diagrams of the fundamental branches
bifurcating from the linear mode $b_{2,0}=0.25$ for several values of
$\alpha$. Thick (thin) lines represent stable (unstable) solutions. Insertion
shows in detail that $P_{1}$ goes to zero in the vicinity of $b_{2,0}$ while
$P_{2}$ remains finite. Filled circles represents two stable solutions shown
below in Fig. 6. The parameters of the structure are $V=1/2$ and $q=0$.

Let us note that (20a) is an eigenvalue equation and so the allowed values of
$W_{2}$ are simply the eigenvalues. Next we define a bra– and ket– vectors:
$\langle\psi|\equiv(\psi(\xi),\bar{\psi}(\xi))$ and
$|\psi\rangle\equiv(\bar{\psi}(\xi),\psi(\xi))^{T}$ ($T$ stays for the
transpose matrix), where $\psi(\xi)\to 0$ at $\xi\to\pm\infty$, and verify
that the operator $L$ is Hermitian with respect to the weighted inner product:

$\langle\psi_{1}|\psi_{2}\rangle=\int_{-\infty}^{\infty}\text{sech}^{\eta_{2}}(\xi)\left[{\psi_{1}}(\xi)\bar{\psi}_{2}(\xi)+\bar{\psi}_{1}(\xi)\psi_{2}(\xi)\right]d\xi.$
(21)

The Hermiticity of $L$ ensures the reality of the admissible $W_{2}$. We also
note that if $\left(\bar{w}_{1}(\xi),w_{1}(\xi)\right)^{T}$ is a solution of
(20b) with eigenvalue $W_{2}$, then $-W_{2}$ is also an eigenvalue with
eigenfunction $\left(-i\bar{w}_{1}(\xi),iw_{1}(\xi)\right)^{T}$. This allows
us to restrict the consideration to $W_{2}>0$.

We investigated the system (20b) numerically and found that there is an
infinite number of discrete eigenvalues $W_{2}$. The ones having the smalest
absolute values are shown in Fig. 4.

Figure 6: (Color online) An example of a stable solution of case (ii) with
$b=0.6$ and $\alpha=2>\alpha_{cr}$, marked with the filled circle in Fig. 5.
Shadowed domains show the localized impurity (darker areas representing higher
values of the real part of the localized potential). The parameters of the
structure are $V=1/2$ and $q=0$.

The amplitude of the second harmonic $W_{2}$ depends on $b=b_{2,0}$ (See
Eq.(20a)). In Fig. 4, we show the dependence of $W_{2}$ on $q$ corresponding
to the lowest $|W_{2}|$ branch. The special case, when for a certain value of
$q$ the amplitude of the second harmonic $W_{2}\to 0$, happens when
$b_{1,0}=b_{2,0}$. This leads to

$\displaystyle q=q_{0},\qquad q_{0}=\eta_{1}^{2}-\frac{1}{4}\eta_{2}^{2},$
(22)

which is precisely the case however (8); we consider it in the next section.

We numerically studied the existence of bifurcations satisfying (7) in Fig. 5.
It is possible to see in the insertion of Fig. 5 that at the bifurcation point
the branches satisfy $P_{1}(b_{2,0})=0$ and consequently
$P(b_{2,0})=P_{2}(b_{2,0})$. A simple integration in (15) after the
substitution $w_{2}=w_{2l}$, reveals that for each $\alpha$ when $V=1/2$ and
$\eta_{2}=1$ (the case of Fig. 5) one has $P(b_{2,0})=4W_{2}^{2}$.

It can be seen in Fig. 5 for the power diagrams of the fundamental branches,
where stable solutions exist above the ${\cal PT}$\- symmetry breaking point.
The existence of stable nonlinear modes even when the spectrum of the linear
system is not purely real has been earlier reported in Zezyulin  (see also
recent work Tsoy .)

In Fig. 6 there is an example of a mode in the case (ii).

##  V Bifurcation of the nonlinear modes from the linear spectrum.

Now we consider the case (iii) for which the relation (8) holds.

Figure 7: (Color online) Several fundamental branches for the case (iii) with
different values of $\alpha$. Note that the branch disappear when
$\alpha\to\alpha_{cr}=3/2$. Stable (unstable) solutions are represented by
thick(thin) lines. Insertion shows the regions of bifurcations from $b_{1,0}$
(FF) and $b_{2,0}$ separated by a vertical dashed line. The parameters of the
structure are $V=1/2$ and $q=q_{0}$($\alpha_{cr}=1.5$).

Previously we have shown that if the bifurcation point is at the same time an
eigenvalue of (9b) and (17b), i.e., $b_{1,0}=b_{2,0}$, then the mismatch must
have the special value $q=q_{0}$. We also have seen that in this case
$W_{2}=0$. As a direct consequence, (20b) reduces to (9b) and not only
$w_{2}\to w_{2l}$ but also the FF satisfy $w_{1}\to w_{1l}$ at the bifurcation
point.

The Fig. 7 shows power diagrams of solutions satisfying (8) for several values
of $\alpha$.

Figure 8: (Color online) Left panel: Shows the fundamental branch (line) of
case (iii) with $\alpha=1.3$ near $b=0$ and the merged two-peaked branch.
Thick lines (lines) are stable (unstable) solutions. Right panel: Shows a
fundamental branch and a two-peaked branch with $\alpha=1.44$ near $b=0$.
Black circles are solutions represented in Fig. 9 and in Fig.14. The
parameters of the structure are $V=1/2$ and $q=q_{0}$.

Figure 9: (Color online) Stable double peaked solution with $b=0.0057$
corresponding to the black circle in left panel of Fig. 8. Left panel shows
the intensities $|w_{n}|^{2}$, right panel shows the currents $j_{n}$.
Shadowed domain show the localized impurity $V\text{sech}^{2}(\xi)$ (darker
areas represent higher values of the real part of the localized potential).
The parameters of the structure are $V=1/2$, $\alpha=1.3$ and $q=q_{0}$.

It is possible to see that two bifurcations occur at $b=b_{1,0}=b_{2,0}$ (See
the dashed line in the insertion of Fig. 7). The branch that goes to the right
is a bifurcation of $b_{2,0}$ and the branch that goes to the left is a
bifurcation of $b_{1,0}$. Both branches have a behaviour very similar to
branches of case (i) and (ii).

Figure 10: The intensity profiles $|w_{1}|^{2}$ of solutions, pertaining to
the fundamental branch of case (iii) bifurcating from $b_{1,0}=0.063$, at
different $b$ illustrating the transition of a single-peaked profile into a
double-peaked one. The local minimum occurs exactly at $\xi=0$. The parameters
of the structure are $V=1/2$, $\alpha=1.3$ and $q=q_{0}$.

We observed in the numerical simulations, that there may be a collision of the
fundamental branch with a non fundamental branch with two peaked solutions.
The Fig. 8 shows the corresponding bifurcation diagrams, whereas Fig. 9
illustrates the distribution of the intensities and the currents in a two-hump
soliton solution. The intensities and the current $j_{2}$ are largely
distributed far from the center of the potential, while the current $j_{1}$ is
localized at the defect.

One can see in Fig. 10 that as $b$ decreases, $|w_{1}(0)|^{2}$ decreases at
the same time that two emergent peaks become increasingly separated. The
intensity $|w_{2}(0)|^{2}$ (not shown) decreases as well.

In respect to phase, we found all stable solutions that bifurcate from
$b_{1,0}$ to satisfy $w_{n}(\xi)=\overline{w_{n}(-\xi)}$. This means that the
peaks of the two-hump solution shown in Fig. 10 are out of phase.

##  VI Stability and dynamics of localized solutions

The stability was studied by direct numerical simulations of the system (1)
and within the framework of eigenvalue evaluation of the eigenvalue problem,
obtained from perturbations of the form

Figure 11: (Color online) The evolution of a stable solution with $b=0.076$
(left upper panel) and an unstable solution with $b=0.09$ (right upper panel)
of the fundamental branch of case (i). The corresponding eigenvalues of the
linear stability analysis are given in the lower panels. The parameters of the
structure are $V=1/2$, $q=0$ and $\alpha=0.9$.

$u_{n}(\xi,\zeta)=\left(w_{n}(\xi)+p_{n+}(\xi)e^{-i\lambda\zeta}+\overline{p}_{n-}(\xi)e^{i\overline{\lambda}\zeta}\right)e^{inb\zeta},$
(23)

with $p_{n+}(\xi)$ and $p_{n-}(\xi)$ being small perturbations. The resulting
eigenvalue problem is given by

$\left(\begin{array}[c]{cccc}L_{2}&2w_{1}&0&0\\\
2\bar{w}_{1}&L_{1,\alpha}&0&2w_{2}\\\ 0&0&-\bar{L}_{2}&-2\bar{w}_{1}\\\
0&-2\bar{w}_{2}&-2w_{1}&-\bar{L}_{1,\alpha}\end{array}\right)\left(\begin{array}[c]{c}p_{2+}\\\
p_{1+}\\\ p_{2-}\\\
p_{1-}\end{array}\right)=\lambda\left(\begin{array}[c]{c}p_{2+}\\\ p_{1+}\\\
p_{2-}\\\ p_{1-}\end{array}\right).$ (24)

Whenever an eigenvalue $\lambda$ with $\text{Im}(\lambda)>0$ occurs the
solution is unstable.

Figure 12: (Color online) Panel shows the maximal value of $\alpha$ for which
a fundamental branch solution is stable as a function of $b$. The branch
bifurcates from $b_{2,0}$. The parameters of the structure are $V=1/2$ and
$q=0$.

Let us start the stability analysis with case (i). Then the branches have two
stable regions, one close to $b_{1,0}$ and the other close to $b=0$ as is
shown in Fig. 1. It was found numerically that only low amplitude solutions
are stable. The instability is produced by pairs of purely imaginary
eigenvalues $\lambda$ , see Fig. 11 showing the eigenvalues and the typical
evolution of the soliton resulting in its rapid decay.

For case (ii), we investigate the stability and found that all the solutions
are stable if $\alpha\lessapprox 0.9$, for $\alpha\gtrapprox 0.9$ some parts
of the bifurcation curve become unstable, see Fig. 12. In Fig. 12 one can see
that the unstable part of the bifurcation curve becomes larger when $\alpha$
increases, but the stable solutions survive even for $\alpha\gg\alpha_{cr}$.
The linear stability analysis shows that the instability arises from quartets
of complex eigenvalues (See Fig. 13).

Figure 13: (Color online) The evolution of a stable solution with $b=0.6$
(left upper panel) and an unstable solution with $b=0.27$ (right upper panel)
of the fundamental branch of case (ii) that bifurcates from $b_{2,0}$. Both
solutions were perturbed by 10% of amplitude random noise. The corresponding
eigenvalues of the linear stability matrix are given in the lower panels. Both
solutions are marked by black circles in Fig. 5. The parameters of the
structure are $V=1/2$, $q=0$ and $\alpha=2$.

Finally we discuss the case (iii). We found that in respect to stability, the
behaviour is similar to case (i) and (ii). For values $b>b_{1,0}$, i.e.,
bifurcations of $b_{2,0}$, the stability behaves like in case (ii), with the
appearance of instability intervals that increase in length as $\alpha$
increases. The region $b<b_{1,0}$ has solutions that bifurcates from
$b_{1,0}$. There is always a stable region adjacent to $b_{1,0}$ and also
another stable region close to $b=0$. The instability, when observed, was due
to a quartet of complex eigenvalues of the stability matrix contained in the
region $b>b_{1,0}$ and two purely imaginary eigenvalues in the region
contained in $b<b_{1,0}$ (See the middle panel of the Fig. 14). We observed
that there are stable solutions in the region where the fundamental branch
bifurcating from $b_{1,0}$ merges with a two-peaked branch.

Figure 14: (Color online) Left panels: Propagation of 10% of amplitude
perturbations of solutions marked by black circles in lower panel of Fig. 8
corresponding to case (iii). The upper left panel has $b=0.0172$, the midle
left panel has $b=0.0072$ and lower left panel has $b=0.0017$. Right panels
are the respective eigenvalues of the linear problem. Parameters of the
structure are $V=1/2$, $\alpha=1.45$ and $q=q_{0}$.

##  VII Conclusion

In summary, we show the existence of solitons in quadratic nonlinear media
with localized ${\cal PT}$\- symmetric modulations of linear refractive index.
The families of stable one and two-hump solitons are found. The properties of
nonlinear modes bifurcating from a linear limit of small fundamental harmonic
field are investigated. It is shown that the fundamental branch have a maximum
in a power. This maximum is decreasing with the strength of of imaginary part
of the refractive index $\alpha$. For the case when both harmonics are of the
same order, the scenarios of bufurcations of different branches of solution on
the propagation constant $b$ are investigated.It was shown that modes
bifurcating from linear mode of the second harmonic can exist, even above
${\cal PT}$ symmetry breaking threshold. We found that the fundamental branch
bifurcating from the linear limit can undergo a secondary bifurcation
colliding with a branch of two-hump soliton solutions.

For nonlinear modes no having linear limit i.e. $|u_{2}|\sim O(1)$, different
branches of solutions in dependence on the parameters $b$ and the phase
mismatch $q$ has been investigated. The stability intervals for different
values of parameters $b$ and $\alpha$ are obtained. The examples of dynamics
and excitations of solitons by numerical simulations of full $\chi^{(2)}$
system of equations with ${\cal PT}$ symmetric potential are confirm
theoretical predictions.

###### Acknowledgements.

FCM akwnoledge the support of Alban. VVK and AVY acknowledge support of the
FCT (Portugal) under the grants PTDC/FIS/112624/2009, and PEst-
OE/FIS/UI0618/2011. FKA acknowledge support of the FAPESP(Brasil).

## References

  * (1) C. M. Bender and S. Boettcher, Phys.Rev.Lett. 80, 5243 (1998);
  * (2) A. Ruschaupt, F. Delgado and J. G. Muga, J. Phys. A 38 L171 (2005).
  * (3) A. Guo, G. J. Salamo, D.Duchesne,R. Morandotti, M. Volatier-Ravat, V. Aimez, G.A. Siviloglou, D.N. Christodoulides, Phys. Rev. Lett., 103, 093902 (2009); C. E. Rüter, K.G.Makris, R. El-Ganainy,D.N. Christodoulides, M. Segev, and D. Kip, Nat. Phys. 6, 192 (2010);
  * (4) Z. H. Musslimani, K. G. Makris, R. El-Ganainy, and D. N. Christodoulides Phys. Rev. Lett. 100, 030402, (2008).
  * (5) M. Znojil, J.Phys. A Math. Gen. 33, L61 (2000).
  * (6) Z. Ahmed, Phys. Lett. A 282, 343-348 (2001).
  * (7) Sumei Hu, Xuekai Ma, Daquan Lu, Zhenjun Yang, Yizhou Zheng, and Wei Hu Phys. Rev. A, 84, 043818 (2011).
  * (8) Zhiwei Shi, Xiujuan Jiang, Xing Zhu, and Huagang Li, Phys. Rev. A 84, , 053855 (2011)
  * (9) A. Mostafazadeh, Phys. Rev. A, 80 032711 (2009).
  * (10) O. Bendix, R. Fleischmann, T. Kottos, and B. Shapiro, Phys. Rev. Lett. 103, (2009)
  * (11) F. K. Abdullaev, V. V. Konotop, M. Ögren, and M. P. Sørensen, Opt. Lett., 36, 4566 (2011).
  * (12) H. Cartarius and G. Wunner arXiv:1203.1885v1 (2012)
  * (13) F. Kh. Abdullaev, V.V. Konotop, M. Salerno, and A. V. Yulin, Phys. Rev. E 82, 056606 (2010); F. Kh. Abdullaev, Y. V. Kartashov, V.V. Konotop, and D. A. Zezyulin, Phys. Rev. A 83, 041805(R) (2011); Y. He , X. Zhu, D. Mihalache, J. Liu and Z. Chen, Phys. Rev. A 85, 013831 (2012); S. Nixon, L. Ge, and J.Yang, Phys. Rev A 85, 023822 (2012); V. Achilleos, P. G. Kevrekidis, D. J. Frantzeskakis, and R. Carretero-Gonzales, Phys. Rev. A 86 , 013808 (2012); J. Zeng and Y. Lan, Phys.Rev. E 85, 047601 (2012); S.V. Suchkov, S.V. Dmitriev, B. A. Malomed, and Yu. S. Kivshar, Phys. Rev. A 85, 033825 (2012).
  * (14) C.B. Clausen, J.P. Torres, and L. Torner, Phys. Lett. A 249, 455 (1998).
  * (15) C.B. Clausen and L. Torner, Phys. Rev. Lett. 81, 790 (1998).
  * (16) B. Midya, B. Roy, R. Roychoudhury, Phys. Lett. A 374 2605-2607 (2010).
  * (17) D. Landau, E. M. Lifshitz, Quantum Mechanics: Non-Relativistic Theory, Volume 3, (Elsevier Science, MA, 1958)
  * (18) D. A. Zezyulin and V.V. Konotop, Phys. Rev. Lett. 108, 213906 (2012).
  * (19) E.N. Tsoy, S. Tadjimuratov, and F.Kh. Abdullaev, Opt. Commun. 285, 3441 (2012).
  * (20) I.S. Gradshteyn, I.M. Ryzhik, Table of Integrals, Series and Products, 4th ed., Academic Press, New York, 1980, p. 838, (7.375.2).


11institutetext: The Hong Kong University of Science and Technology
22institutetext: Fudan University

# QMix: Quality-aware Learning with Mixed Noise for Robust Retinal Disease
Diagnosis

Junlin Hou 11 Jilan Xu 22 Rui Feng 22 Hao Chen 11

###### Abstract

Due to the complexity of medical image acquisition and the difficulty of
annotation, medical image datasets inevitably contain noise. Noisy data with
wrong labels affects the robustness and generalization ability of deep neural
networks. Previous noise learning methods mainly considered noise arising from
images being mislabeled, _i.e._ label noise, assuming that all mislabeled
images are of high image quality. However, medical images are prone to
suffering extreme quality issues, _i.e._ data noise, where discriminative
visual features are missing for disease diagnosis. In this paper, we propose a
noise learning framework, termed as QMix, that learns a robust disease
diagnosis model under mixed noise. QMix alternates between sample separation
and quality-aware semi-supervised training in each training epoch. In the
sample separation phase, we design a joint uncertainty-loss criterion to
effectively separate (1) correctly labeled images; (2) mislabeled images with
high quality and (3) mislabeled images with low quality. In the semi-
supervised training phase, we train a disease diagnosis model to learn robust
feature representation from the separated samples. Specifically, we devise a
sample-reweighing loss to mitigate the effect of mislabeled images with low
quality during training. Meanwhile, a contrastive enhancement loss is proposed
to further distinguish mislabeled images with low quality from correctly
labeled images. QMix achieved state-of-the-art disease diagnosis performance
on five public retinal image datasets and exhibited substantial improvement on
robustness against mixed noise.

###### Keywords:

Label noise Image quality Retinal disease diagnosis

##  1 Introduction

Figure 1: Left: An illustration of correctly labeled data, mislabeled data w/
high quality (Mis-H), and mislabeled data w/ low quality (Mis-L) about two
types of retinal images. Right: Performance degradation of current LNL methods
in the presence of mixed noise.

Deep learning has achieved significant advancements in medical image analysis.
One of the main reasons is the growing availability of increasingly large
datasets. In practice, given the complexity of the data collection and
labeling process, medical datasets inevitably contain noise [21]. For
instance, doctors’ labeling experience and inter-annotator variability could
potentially lead to images being incorrectly labeled. Deep neural networks
trained on such datasets are prone to over-fitting noisy labels, resulting in
limited generalization ability [41]. Learning from noisy labels (LNL) has been
a long-standing challenge in image recognition [35, 1]. Most existing LNL
methods filter correctly labeled samples from the entire noisy dataset [17,
13, 23]. The remaining noisy samples are either assigned with smaller weights
in the loss [17, 12] or treated as unlabeled data and re-labeled for semi-
supervised training [23, 20].

Despite the generalization performance, existing works in the general domain
simply consider samples with label noise in the dataset, _i.e._ , the image is
being mislabeled. Nevertheless, in medical domain, the source of noise turns
out much more complicated. For example, due to variations in shooting
environments, differences in equipment, or improper cooperation from patients,
some collected images might have poor image quality. As illustrated in Fig. 1
left, samples with correct disease category labels (Given = True) are valuable
for network training, and we simply denote them as Correct. Mislabeled samples
have incorrect data labels (Given $\neq$ True), but some still hold comparably
high image quality (Mis-H, Top-left). Mislabeled samples with low quality
(Mis-L), however, fail to contain sufficient information for annotators to
perform reliable labeling (Bottom-left). In Fig. 1 right, we empirically show
that current LNL approaches [23, 20] manage to handle the case where only
Correct and Mis-H occur in the dataset (blue histograms), but their diagnostic
performance drops significantly when Mis-L occur simultaneously (orange
histograms). In contrast, our method exhibits robust diagnostic performance
under mixed noise.

In this paper, we propose a novel LNL framework, termed as QMix, that learns a
robust disease classification model under both label noise and image data
noise simultaneously. Following the separate-then-learn pipeline, we specially
design a novel sample separation strategy to distinguish among Correct, Mis-H
and Mis-L, followed by a semi-supervised training process with two newly
devised loss functions to learn a robust disease diagnosis model. These two
processes alternate in each training epoch. The motivation of our novel sample
separation strategy arises from two aspects: (1) We empirically show that DNNs
tend to learn correct samples in early training iterations by producing
smaller losses, then fit Mis-L, and finally memorize Mis-H; (2) Low-quality
images lack enough discriminative visual clues, leading to higher uncertainty
in network predictions. Therefore, we devise a joint uncertainty-loss
criterion for sample separation. Given separated samples, we adopt a semi-
supervised training framework to learn robust feature representations. The
network is an dual architecture with shared parameters, where the labeling
branch takes the weakly-augmented Correct, Mis-H, Mis-L as input and generates
their pseudo disease labels. The online branch inputs the strongly-augmented
counterparts and is supervised by the pseudo labels. Furthermore, we introduce
two novel losses, _i.e._ , sample-reweighing loss and contrastive enhancement
loss. The former dynamically mitigates the potential negative impact of Mis-L
during semi-supervised training. The contrastive enhancement loss further
distinguishes the Mis-L from correctly labeled samples, which benefits the
sample separation in the next round. We evaluate our proposed QMix on five
public retinal disease diagnosis benchmarks, _i.e._ , DDR [24], DRTiD [16],
DeepDRiD [25], EyeQ [10] and DRAC [31]. Experimental results show our method
achieved state-of-the-art performance on all the datasets. Especially when the
noise ratio is extremely high, _e.g._ 80% Mis-H and 15% Mis-L data, the
baseline method suffered noise while QMix exhibited the robustness (15.9% vs.
75.9% Quadratic Weighted Kappa). The main contributions of our method are
listed below:

  * •

We propose QMix, a novel LNL framework that learns a robust disease diagnosis
model under mixed noise, _i.e._ , label noise and data noise.

  * •

We develop a novel sample separation strategy to separate correctly labeled
samples, mislabeled images w/ high quality, and mislabeled images w/ low
quality by modeling the joint distribution of uncertainty and loss.

  * •

We devise two loss functions, _i.e._ , a sample-reweighing loss for improving
quality-aware training and a contrastive enhancement loss for encouraging
discrimination between correctly labeled samples and low-quality samples.

  * •

Extensive experiments on both synthetic and real-world noise from five retinal
disease datasets show that QMix significantly outperformed other LNL methods
and revealed strong robustness even at a high noise ratio.

##  2 Related Work

###  2.1 Learning with Noisy Labels

Existing methods of learning with noisy labels mainly fall into two
categories. The first category of works [30, 15, 5, 6] aims to estimate a
noise transition matrix, which represents the transition probabilities from
truth labels to noisy ones, for label correction or loss calibration. However,
it is challenging to calculate the transition matrix with the increasing
number of classes in real-world applications.

The second category, namely sample selection, is to filter the correctly
labeled instances from a noisy dataset and use them to train DNNs. Several
works [17, 12] discard or reweigh the noisy samples in the objective. However,
discarding the noisy samples fails to leverage the visual information in noisy
samples. Another line of works rejects the original labels by re-labeling all
samples using network predictions [36] or learned label distributions [39].
Malach _et al_. [27] and Han _et al_. [13] adopted a co-training strategy of
two networks’ collaboration to alleviate error accumulation caused by
incorrect selections. Recently, a group of approaches [23, 28] leveraged semi-
supervised learning (SSL) by regarding noisy samples as unlabeled data, thus
rejecting their labels while exploiting the image content. Combinations of SSL
and contrastive learning [29, 20, 26] are further developed to combat the
memorization of noisy labels. Inspired by these works, we particularly devise
a novel sample separation strategy based on uncertainty-loss criterion to
effectively distinguish among correctly labeled, mislabeled images w/ high
quality and mislabeled images w/ low-quality.

###  2.2 Medical Image Analysis with Noisy Labels

There have been several works attempting to learn robust medical image
representation with noisy labels [9, 22, 38, 18, 42]. For instance, Dgani _et
al_. [9] leveraged a noise adaptation layer [11] to estimate the transition
matrix for mammography classification. Based on sample reweighing [32], Le _et
al_. [22] trained the model on a large noisy corpus of patches, incorporating
weights from a small clean set for pancreatic cancer detection. Similarly, Xue
_et al_. [38] adopted a reweighing strategy to remove samples with high loss
values for skin lesion diagnosis. For retinal disease classification, Ju _et
al_. [18] used dual-uncertainty estimation to tackle disagreement and single-
target label noise. Zhou _et al_. [42] employed consistency regularization and
disentangled distribution learning. However, most works directly adapt
existing LNL methods from general domain to medical domain, without developing
specialized methods to meet the characteristics of medical noisy labels. In
this work, we dive into the mislabeled images and further consider the image
quality issue that frequently occurs in medical domain and propose effective
sample separation and respective LNL training strategies.

##  3 Methodology

Figure 2:  An overall framework of the proposed QMix for quality-aware
learning with mixed noise. (a) QMix consists of sample separation and semi-
supervised learning that alternates in each epoch, in which a co-training
scheme is adopted to avoid error accumulation. (b) Sample separation separates
Correct/Mis-H/Mis-L by modeling the joint distribution of uncertainty and
loss. (c) Quality-aware semi-supervised learning with semi-supervised loss
($\mathcal{L}_{SSL}$) and contrastive enhancement loss ($\mathcal{L}_{con}$).

In this section, we introduce our proposed LNL framework, termed as QMix,
which is comprised of sample separation and quality-aware semi-supervised
training. As illustrated in Fig. 2(a), these two processes _execute
alternatively_ in each training epoch. Sample separation (Sec. 3.1) aims to
distinguish correctly labeled, mislabeled samples w/ high quality and
mislabeled samples w/ low quality from the input data via a newly designed
uncertainty-loss criterion. Subsequently, the semi-supervised learning process
(Sec. 3.2) trains the network to learn robust feature representation with a
novel sample-reweighing loss and a contrastive enhancement loss. To avoid
error accumulation in self-training, QMix co-trains two networks, where the
training data separated by each network is used to train the other network.

###  3.1 Sample Separation

Figure 3: Memorization effect under (a) label noise only and (c) mixed noise;
(b) Previous small-loss separation; (d) Our joint uncertainty-loss criterion.

Given the training data $\mathbbm{D}=(X,Y)=\\{(x_{i},y_{i})\\}_{i=1}^{N}$
where $x_{i}$ is an image and $y_{i}\in\\{0,1\\}^{C}$ is the one-hot label
over $C$ disease classes. Sample separation aims to identify the possible
noisy samples from the correctly labeled data by assigning a noise indicator
$n_{i}\in\\{0,1,2\\}$, where 0, 1, and 2 refer to correctly labeled image
(Correct), mislabeled image w/ high quality (Mis-H), and mislabeled image w/
low quality (Mis-L), respectively. Notably, the separation is conducted in an
unsupervised manner, _i.e._ , the ground-truth label of samples being
Correct/Mis-H/Mis-L is not available in practice. We start by reviewing the
basic memorization effect, which has been widely used to guide the separation
between Correct and Mis-H. Then, we introduce our proposed solution to handle
sample separation where both Mis-H and Mis-L are involved.

####  3.1.1 Memorization effect on mixed label noise.

Memorization effect [3] refers to the behavior exhibited by deep neural
networks trained on noisy samples. As shown in Fig. 3(a), during training, the
network first learns simple patterns from Correct in early training
iterations. As the training process continues, the network gradually memorizes
Mis-H by over-fitting the wrong labels. In this way, prior works [17, 2, 13,
40, 23] treated samples with smaller losses as Correct and separated them from
Mis-H at the early stage of training, as shown in Fig. 3(b). Despite good
performance, these works only distinguish Correct from Mis-H, _i.e._
$n_{i}\in\\{0,1\\}$, without considering Mis-L ($n_{i}=2$) in the dataset. We
further investigate the memorization effect on mixed noise, where both Mis-H
and Mis-L are included. As can be observed in Fig. 3(c), the network tends to
learn Correct in early training iterations, then fit Mis-L, and finally
memorize Mis-H. Nevertheless, the small-loss criterion initially designed to
separate Correct from Mis-H fails to handle mixed noise, as the loss
distribution of Mis-L may have overlap with Correct and Mis-H as shown in Fig.
3(d) top.

####  3.1.2 Uncertainty-loss criterion for sample separation.

Generally, both Correct and Mis-H provide clear visual patterns for the
network to make correct predictions in the early training stage. Instead,
Mis-L lacks discriminative visual information and lead to higher uncertainty
in network predictions. This motivates us to conduct sample separation based
on the joint distribution of per-sample uncertainty and loss. Regarding the
joint distribution depicted in Fig. 3(d), samples exhibiting small loss and
low uncertainty are classified as Correct, and Mis-L (moderate loss and high
uncertainty) and Mis-H (high loss and low uncertainty) can also be separated
accordingly.

Formally, given training samples $\\{(x_{i},y_{i})\\}_{i=1}^{N}$, we train a
disease classification model $F(\phi)$ parameterized by $\phi$. We adopt the
standard cross-entropy loss $l(\phi)$ to reflect how well the model fits the
training samples and entropy $e(\phi)$ to measure the uncertainty of the
model’s predictions:

$(l(\phi),e(\phi))=\\{(l_{i},e_{i})\\}_{i=1}^{N}=\\{(-y_{i}\log\mathrm{p}(x_{i};\phi),-\mathrm{p}(x_{i};\phi)\log\mathrm{p}(x_{i};\phi))\\}_{i=1}^{N},$
(1)

where $\mathrm{p}(x_{i};\phi)$ denotes the model’s output softmax probability.

To separate Correct, Mis-H and Mis-L, we fit a three-component Gaussian
Mixture Model (GMM) [33] to the joint distribution of normalized
$(l(\phi),e(\phi))$. In specific, we define the model for each sample
$s_{i}=(l_{i},e_{i})$ as a probability mixture model of three 2D Gaussian
distributions $\mathcal{N}(\mu^{1},\mu^{2},\sigma^{1},\sigma^{2},\rho)$:

$\mathrm{p}(s_{i}|\theta)=\sum\nolimits_{k=0}^{2}\lambda_{k}\mathcal{N}(s_{i}|\theta_{k}),$
(2)

where
$\theta_{k}=(\mu^{1}_{k},\mu^{2}_{k},\sigma^{1}_{k},\sigma^{2}_{k},\rho_{k})$
denotes the parameter of the $k$-th 2D gaussian distribution;
$\theta=\\{\theta_{k},\lambda_{k}\\}_{k=0,1,2}$. We adopt the Expectation-
Maximization algorithm [7] to obtain the posterior probability $w_{i}^{k}$ of
sample $s_{i}$ belonging to the $k$-th Gaussian component $z_{k}$:

$w^{k}_{i}=\mathrm{p}(z_{k}|s_{i})=\frac{\lambda_{k}\mathcal{N}(s_{i}|\theta_{k})}{\sum_{j=0}^{2}\lambda_{j}\mathcal{N}(s_{i}|\theta_{j})}$
(3)

In this way, the predicted noise indicator $n_{i}$ is computed by indexing the
maximum probability over three Gaussian components:

$n_{i}=\text{argmax}_{k\in\\{0,1,2\\}}w^{k}_{i}.$ (4)

Network Co-training. The accuracy of sample separation affects the subsequent
semi-supervised training process. For example, if Mis-H is wrongly classified
as Correct, it would keep having lower loss due to the model overfitting to
its label. We adopt a co-training strategy [27] to avoid error accumulation,
as illustrated in Fig. 2 (a). In particular, two networks $F(\phi_{A})$ and
$F(\phi_{B})$ are trained simultaneously in each epoch. The samples
$\\{(x_{i},y_{i},n_{i}^{A})\\}_{i=1}^{N}$ with noise indicator $n_{i}^{A}$
produced by $F(\phi_{A})$ are used to guide the semi-supervised training for
the other network $F(\phi_{B})$, and vice versa. This co-training scheme makes
two networks keep diverging from each other and robust to noise, due to
different initialization and data division [23]. Notably, the co-training
scheme is only adopted in the training phase. In the test phase, either
$F(\phi_{A})$ or $F(\phi_{B})$ can be used to perform robust disease
diagnosis.

###  3.2 Quality-aware Semi-Supervised Training

In this phase, we aim to train the network to learn robust feature
representations from these separated samples via a novel semi-supervised
learning (SSL) pipeline. As shown in Fig. 2 (c), we introduce a dual
architecture with shared model parameters, where the labeling branch takes the
weakly-augmented Correct, Mis-H and Mis-L as input and generates the pseudo
disease classification labels to supervise the online branch. The online
branch inputs the strongly-augmented samples, and it is trained by optimizing
the proposed combined loss. Note that, only the online branch is trained, and
the labeling branch always maintains the model parameters of the online
branch. Given training samples
$\mathbbm{D}=\\{(x_{i},y_{i},w_{i},n_{i})\\}_{i=1}^{N}$, where
$w_{i}=\\{w_{i}^{k}\\}_{k=0}^{2}$ refers to the probability of the sample
being Correct/Mis-H/Mis-L produced by GMM and $n_{i}$ denotes its predicted
noise indicator. The correctly labeled samples
$\mathbbm{D}_{correct}=\\{(x_{i},y_{i},w_{i},n_{i})|n_{i}=0\\}_{i=1}^{N_{c}}$
and mislabeled samples w/ low quality
$\mathbbm{D}_{mis-l}=\\{(x_{i},y_{i},w_{i},n_{i})|n_{i}=2\\}_{i=1}^{N_{l}}$
are regarded as labeled samples, while mislabeled samples w/ high quality
$\mathbbm{D}_{mis-h}=\\{(x_{i},w_{i},n_{i})|n_{i}=1\\}_{i=1}^{N_{h}}$ are
treated as unlabeled samples by discarding their original class labels.

####  3.2.1 Labeling branch.

For each labeled sample $x\in\\{\mathbbm{D}_{correct},\mathbbm{D}_{mis-l}\\}$,
we refine classification labels via a co-refinement strategy. To be specific,
we linearly combine (1) the original label $y$ and (2) the average predicted
class probability of $M$ weakly-augmented samples:

$\bar{y}=(w^{0}+w^{2})\cdot
y+(1-w^{0}-w^{2})\cdot\frac{1}{M}\sum\nolimits_{m=1}^{M}\mathrm{p}(g_{weak}(x)|\phi),$
(5)

where $w^{0}$ and $w^{2}$ represent the probability of being Correct and Mis-L
generated by the other network; $g_{weak}$ refers to a weak-augment function.
For an unlabeled sample $x\in\mathbbm{D}_{mis-h}$, we perform co-guessing to
predict the pseudo label by integrating the predictions of $2M$ weakly-
augmented samples from both networks:

$\bar{y}=\frac{1}{2M}\sum\nolimits_{m=1}^{2M}p(g_{weak}(x)|\phi).$ (6)

####  3.2.2 Online branch training.

Next, we conduct semi-supervised training based on the strongly-augmented
samples. Given the samples with refined labels
$\bar{\mathbbm{D}}=\\{x_{i},\bar{y}_{i}\\}_{i=1}^{N}$, we adopt different loss
functions for Correct, Mis-H, and Mis-L, respectively. Following [23], we
adopt the standard cross-entropy (CE) loss $\mathcal{L}_{correct}$ on Correct
and the mean square error (MSE) $\mathcal{L}_{mis-h}$ on Mis-H.

In particular, for low-quality samples $\bar{\mathbbm{D}}_{mis-l}$, we propose
a sample-reweighing loss to mitigate their potential negative impact on
network training:

$\mathcal{L}_{mis-l}=-\frac{\Omega(t)}{|\mathbbm{D}_{mis-l}|}\left(\sum\nolimits_{x\in\mathbbm{D}_{mis-l}}\bar{y}\log(\mathrm{p}(g_{strong}(x)|\phi))\right),$
(7)

where $g_{strong}(x)$ indicates strong augmentation; $\Omega(t)$ denotes the
weight over Mis-L at epoch $t$. As Mis-L contains less distinguishable visual
information, an intuitive idea is to discard the sample by setting
$\Omega(t)=0$ so that it does not contribute to the loss propagation. However,
Correct might be incorrectly classified as Mis-L at the early stages of
network training, due to the insufficient model capability. To mitigate the
potential information loss, we further improve our sample-reweighing loss by
introducing a linear weight decay strategy as:

$\Omega(t)=\frac{\Omega(0)}{T_{w}-T}\cdot(t-T),$ (8)

where $T$ and $T_{w}$ denote the total number of training epochs and warm-up
epochs, respectively. This strategy involves initially assigning a small
weight $\Omega(0)$ to Mis-L. As the training progresses, the network’s
accuracy in classifying Correct/Mis-H/Mis-L improves steadily. Concurrently,
the weight gradually decays to zero, minimizing the adverse influence of Mis-L
on network training.

####  3.2.3 Contrastive enhancement on low-quality samples.

Generally, the ground-truth labels of Mis-L samples should not correspond to
any of the $C$ disease classification categories but rather belong to a
distinct category referred to as “ungradable”. Therefore, we incorporate a
contrastive enhancement loss into the semi-supervised training process, aiming
to distinguish Mis-L from Correct. Specifically, for each strongly-augmented
sample, it is encoded and mapped to a low-dimensional vector
$z_{i}=P(F(g_{strong}(x_{i})|\phi))\in\mathbb{R}^{d_{p}}$, where $P(\cdot)$ is
the projection network. We define any two samples $i$ and $j$ from the same
category as positive, _i.e.,_ $\bar{y}_{i}=\bar{y}_{j}$, otherwise they are
negative. Note that the total number of categories is $C+1$ ($C$ disease
classes with an additional ungradable class). The contrastive enhancement loss
can thus be defined as:

$\mathcal{L}_{con}=\frac{1}{N}\sum_{i=1}^{N}\frac{-1}{N_{\bar{y}_{i}}-1}\sum_{j=1}^{N}\mathbbm{1}_{i\neq
j}\cdot\mathbbm{1}_{\bar{y}_{i}=\bar{y}_{j}}\cdot\log\frac{\exp(z_{i}^{T}\cdot
z_{j}/\tau)}{\sum_{k=1}^{2N}\mathbbm{1}_{i\neq k}\cdot\exp(z_{i}^{T}\cdot
z_{k}/\tau)},$ (9)

where $\mathbbm{1}\in\\{0,1\\}$ is the indicator function; $\tau>0$ denotes
the temperature; $N_{\bar{y}_{i}}$ is the number of samples sharing the same
label with $\bar{y}_{i}$ in each mini-batch. Eq. 9 enforces Mis-L to be
grouped together and kept separated from Correct, while the distance between
different disease classes remains. Discrimination between Mis-L and Correct
encourages the sample separation process in the next round.

####  3.2.4 Loss function.

The total loss is composed of CE loss on Correct, sample-reweighing loss on
Mis-L, MSE loss on Mis-H, and contrastive enhancement loss with balancing
weights $\lambda_{m},\lambda_{c}$:

$\mathcal{L}=\underbrace{\mathcal{L}_{correct}+\mathcal{L}_{mis-l}+\lambda_{m}\mathcal{L}_{mis-h}}_{\mathcal{L}_{SSL}}+\lambda_{c}\mathcal{L}_{con}.$
(10)

##  4 Experiments

In this section, we evaluate our QMix in different label noise scenarios,
including (1) synthetic noisy label dataset (_i.e._ , DDR) and (2) real-world
noisy label datasets (_i.e._ , DRTiD, DeepDRiD, EyeQ and DRAC). We introduce
these datasets (Sec. 4.1) and implementation details (Sec. 4.2.1), followed by
the comparison results (Sec. 4.3 and Sec. 4.4) and ablation studies (Sec.
4.5).

###  4.1 Datasets

We validate our method on five datasets for diabetic retinopathy (DR) grading.
The severity of DR can be divided into five stages, including no DR, Mild
Nonproliferative DR (NPDR), Moderate NPDR, Severe NPDR, and Proliferative DR
(PDR) [37]. The five grades can also be merged into ternary classification,
_i.e._ , no DR, NPDR and PDR. DRTiD [16] and DeepDRiD [25] provide two-field
color fundus photography (CFP), which leverage images from two views (_i.e._ ,
macula-centric and optic disc-centric) for a more comprehensive DR screening.
EyeQ [10] is a large-scale CFP dataset for five-level DR grading, and DRAC
[31] is an optical coherence tomography angiography (OCTA) dataset for ternary
DR diagnosis. The statistical details of these datasets can be found in Table
1. Due to variations in shooting environments, differences in equipment, and
doctors’ experience, real-world retinal image datasets are prone to containing
both mislabeled images with high and low quality.

Table 1: Details of noisy label datasets, including numbers of images, numbers(ratio) of Mis-H or Mis-L training images, retinal image types, and DR grading tasks. Dataset | #Train/Test | #Mis-H | #Mis-L | Type | DR grading
---|---|---|---|---|---
Synthetic noisy label dataset |  |
DDR [24] | 6,260/2,503 | synthetic | 1,151+synthetic | CFP | five-class
Real-world noisy label dataset |  |
DRTiD [16] | 2,698/1,100 | 850(31.5%) | 698(25.8%) | CFP | five-class
DeepDRiD [25] | 1,200/400 | $\checkmark$ | 624(52.0%) | CFP | five-class
EyeQ [10] |  12,553/13,029 | $\checkmark$ | 2,330(18.5%) | CFP | five-class
DRAC [31] | 611/386 | $\checkmark$ | 50(7.5%) | OCTA | ternary

###  4.2 Experimental Setups

####  4.2.1 Implementation details.

Our model consists of a ResNet-50 [14] encoder pre-trained on ImageNet [8],
the projection network is a two-layer MLP. The input images are resized to
$384\times 384$. During sample separation, data augmentation is not applied to
the input images. In the semi-supervised learning process, the weak
augmentation includes random horizontal flip, vertical flip and rotation, and
the strong augmentation includes random horizontal flip, vertical flip, color
jittering, random cropping, affine transformation, grayscaling, etc. The
network is trained for 100 epochs with a batch size of 16 and a learning rate
of 1e-3. To make the sample separation reliable, the network first undergoes
warm-up training for 10 epochs using the standard CE loss. After that, we
alternate the sample separation and semi-supervised training in each epoch. We
adopt Quadratic Weighted Kappa [19] as the evaluation metric. Please refer to
the supplementary file for the settings of hyperparameters under different
noise ratios.

####  4.2.2 Compared methods.

The baseline method is a DR grading model trained with the standard cross-
entropy loss, without considering the noisy images in the datasets. We compare
with other state-of-the-art LNL methods. DivideMix [23], MOIT [29], AugDMix
[28], UNICON [20] are sample selection based methods with SSL training. SEAL-
IDN [4], CD-CCR [6], MEIDTM [5] are particularly proposed for instance-
dependent label noise, where mislabeling is dependent on the specific visual
appearance of each image, _e.g._ , visual ambiguity. Images of low-quality can
also be viewed as a source of instance-dependent label noise [5].

Table 2: Results (%) on the DDR dataset with symmetric and asymmetric noise. The top-1 and top-2 results are marked in bold and underline. |  | Symmetric | Asymmetric
---|---|---|---
Mis-L ratio |  | 15% | 15% | 15% | 45% | 75% | 15% | 15% | 35% | 55%
Mis-H ratio |  | 20% | 50% | 80% | 20% | 20% | 30% | 50% | 20% | 20%
Standard CE |  Best |  88.3 |  81.4 |  72.4 |  86.7 |  87.1 |  85.3 |  79.3 |  85.9 |  85.6
Last | 76.1 | 39.7 | 15.9 | 69.4 | 60.7 | 66.2 | 52.0 | 75.6 | 70.4
DivideMix [23] | Best | 88.7 | 83.6 | 71.6 | 86.5 | 86.2 | 85.3 | 79.4 | 87.3 | 86.3
Last | 87.2 | 71.1 | 42.9 | 84.8 | 82.5 | 84.0 | 67.8 | 84.6 | 84.3
MOIT [29] | Best | 88.4 | 83.5 | 73.2 | 87.3 | 87.7 | 85.4 | 82.0 | 86.7 | 85.6
Last | 84.8 | 83.5 | 65.2 | 77.3 | 70.9 | 83.4 | 79.4 | 84.7 | 82.7
AugDMix [28] | Best | 88.7 | 83.1 | 79.3 | 87.5 | 83.5 | 86.7 | 81.8 | 86.1 | 85.0
Last | 88.4 | 75.2 | 66.8 | 84.9 | 78.8 | 85.6 | 66.8 | 83.5 | 82.4
UNICON [20] | Best | 87.9 | 83.5 | 66.5 | 86.7 | 86.8 | 86.6 | 84.1 | 87.1 | 86.7
Last | 86.8 | 70.9 | 39.8 | 84.1 | 83.1 | 83.0 | 79.4 | 83.0 | 78.6
SEAL-IDN [4] | Best | 87.3 | 83.3 | 75.3 | 86.7 | 87.3 | 84.1 | 78.2 | 85.8 | 84.9
Last | 86.3 | 81.1 | 69.7 | 86.2 | 85.5 | 81.6 | 72.8 | 83.4 | 82.9
CD-CCR [6] | Best | 87.3 | 82.6 | 69.6 | 86.8 | 86.7 | 86.4 | 78.6 | 86.8 | 86.4
Last | 84.2 | 61.3 | 50.0 | 80.7 | 74.7 | 76.0 | 58.8 | 78.3 | 76.5
MEIDTM [5] | Best | 86.6 | 83.0 | 72.8 | 86.6 | 85.5 | 84.9 | 81.3 | 85.8 | 85.6
Last | 80.7 | 54.4 | 38.3 | 74.3 | 63.7 | 72.2 | 58.1 | 79.0 | 77.2
QMix (ours) | Best | 90.1 | 87.9 | 81.2 | 89.2 | 88.5 | 89.7 | 87.4 | 89.1 | 88.3
Last | 88.2 | 86.3 | 75.9 | 87.1 | 87.2 | 88.9 | 85.3 | 87.4 | 86.7

###  4.3 Results on Synthetic Noisy Label Dataset

First, we evaluate our method on the synthetic noisy label dataset DDR, which
contains 6,260 correctly labeled samples and 1,151 mislabeled samples with low
quality, respectively. Mislabeled samples with high quality are generated by
changing the labels of partial correct samples in a symmetric or asymmetric
manner. In specific, symmetric noise indicates the wrong labels are generated
under a uniform distribution of all five classes. Asymmetric noise refers to
assigning wrong labels based on the categorical distribution of the dataset,
which means the label distribution of mislabeled samples could be imbalanced,
posing additional challenges. We experiment with different noise ratios by
changing the number of Mis-H/Mis-L for comprehensive comparison. Notably, in
addition to the provided Mis-L, we adopt an off-the-shelf low-quality retinal
image generation algorithm [34] to generate more Mis-L for higher rates. Table
2 shows the results on the DDR dataset with different levels of symmetric and
asymmetric label noise. To show the robustness of LNL methods, we report both
the model checkpoint with the best validation results across all epochs (Best)
and the average of the last 10 epochs (Last). If the model suffers noisy
samples, the Last checkpoint is expected to be significantly lower than Best
checkpoint.

In the case of symmetric noise, as the noise rate increases, QMix consistently
outperforms all the methods except for the Last model under the low noise
rate, _i.e._ , 15% Mis-L ratio and 20% Mis-H ratio. For high noise rates,
_e.g._ , 80% Mis-H ratio, the Last checkpoint of the baseline CE method drops
significantly (72.4% $\rightarrow$ 15.9%), indicating that the noisy labels
over-fitted by the baseline model greatly affect the its DR grading
performance. Other LNL approaches, _e.g._ , DivideMix, UNICON, and MEIDTM,
also suffer the high noise ratio, exhibiting $\sim$30% performance drop.
Similar results can be observed under a high Mis-L noise ratio of 75%. In
comparison, QMix achieves significantly better performance improvements over
these state-of-the-arts. In the case of asymmetric noise, both correct and
mislabeled samples share an imbalanced categorical distribution. This makes
the selection of correct samples more challenging. However, the improvement of
our QMix is substantial under asymmetric noise across all noise rates. For
instance, under 55% Mis-L noise ratio and 20% Mis-H noise ratio, QMix achieves
88.3% and 86.7% for the Best checkpoint and Last checkpoint, respectively.
This indicates our model is robust to these mislabeled images during the whole
training process.

Table 3: Results (%) on four real-world noisy labeled retinal image datasets. Method | DRTiD | DeepDRiD | EyeQ | DRAC
---|---|---|---|---
| Best | Last | Best | Last | Best | Last | Best | Last
Standard CE |  72.0 |  51.3↓20.7 |  91.7 |  87.8↓3.9 |  78.4 |  75.7↓2.7 |  82.5 |  74.1↓8.4
DivideMix [23] | 78.3 | 74.9${}_{\color[rgb]{0.0600000000000001,0.46,1}\downarrow~{}3.4}$ | 93.0 |  91.7↓1.3 | 75.8 | 73.4↓2.4 | 83.1 | 81.8↓1.3
MOIT [29] | 74.1 | 63.1↓11.0 | 94.1 | 90.0↓4.1 | 80.7 | 77.2↓3.5 | 83.5 | 79.6↓3.9
AugDMix [28] | 77.6 |  75.4${}_{\color[rgb]{0.0600000000000001,0.46,1}\downarrow~{}2.2}$ | 92.6 | 88.2↓4.4 | 76.6 | 69.0↓7.6 | 81.2 | 78.3↓2.9
UNICON [20] | 77.9 | 75.1${}_{\color[rgb]{0.0600000000000001,0.46,1}\downarrow~{}2.8}$ | 92.3 | 88.3↓4.0 | 78.1 | 68.4↓9.7 | 82.7 | 76.0↓6.7
SEAL-IDN [4] | 75.4 | 70.2${}_{\color[rgb]{0.0600000000000001,0.46,1}\downarrow~{}5.2}$ | 93.6 | 88.1↓5.5 | 78.8 | 76.3↓2.5 | 83.3 | 79.6↓3.7
CD-CCR [6] | 73.4 | 55.9↓17.5 | 91.5 | 89.0↓2.5 | 79.2 |  78.1↓1.1 | 83.4 | 81.5↓1.9
MEIDTM [5] | 74.7 | 60.9↓13.8 | 91.2 | 88.2↓3.0 | 78.9 | 77.1↓1.2 | 82.4 | 78.1↓4.3
QMix (ours) | 80.8 | 78.4${}_{\color[rgb]{0.0600000000000001,0.46,1}\downarrow~{}2.4}$ | 94.1 | 92.2↓1.9 | 80.4 | 79.1↓1.3 | 84.1 |  81.7↓2.4

###  4.4 Results on Real-World Noise Datasets

Table 3 illustrates the superior performance of our proposed QMix when
training in the presence of real-world label noise on four retinal image
datasets.

DRTiD dataset. When trained by standard CE loss, the network shows a more than
20% performance decrease from the Best checkpoint to the Last checkpoint.
Similarly, the performance of CD-CCR and MEIDTM drops by a large margin (17.5%
and 13.8%), revealing that they are less robust when dealing with mixed noise
(31.5% Mis-H and 25.8% Mis-L on DRTiD). We achieve 2.5% and 3.0% performance
improvement on the results of Best and Last checkpoints over the second-best
methods, _i.e._ , DivideMix and AugDMix, respectively.

DeepDRiD dataset. Results on DeepDRiD further confirm that our model achieves
substantial improvement for retinal disease diagnosis with two-field fundus
images. Despite a high Mis-L ratio (52%) on DeepDRiD, our QMix retains the
performance and shows robustness to over-fitting on noisy samples when the
training set is comparably small, only containing 1,200 images.

EyeQ dataset. Despite the EyeQ dataset containing a low proportion (18.5%) of
Mis-L, our QMix still achieves an improvement about 2% on both Best and Last
checkpoints compared to the baseline Standard CE. We achieve a comparable Best
result with the best-performing MOIT, while obtaining a higher performance on
the result of Last checkpoint (79.1% vs. 77.2%). Our method demonstrates
robustness on the EyeQ dataset.

DRAC dataset. On this real-world OCTA image dataset, the baseline Standard CE
also suffers the over-fitting problem (82.5%$\rightarrow$74.1%). Despite the
challenge, QMix achieves the SOTA performance with a Kappa of 84.1%, while
maintaining a competitive result to DivideMix in terms of the Last checkpoint.

###  4.5 Ablation Study and Discussion

We conduct ablation studies on the DDR dataset and make discussions on (1)
sample separation performance; (2) joint uncertainty-loss criterion; (3)
sample-reweighing loss and contrastive enhancement loss in quality-aware
learning.

Sample separation performance. Generally, the overall diagnostic performance
is highly dependent on the precision of correct sample selection. We compare
ROC-AUC scores between our selection mechanism and other methods [23, 28, 20].
As illustrated in Fig. 4 left, DivideMix and UniCon have a difficulty in
distinguishing Correct in the presence of mixed noise, as they solely rely on
the small-loss criterion to differentiate Mis-H from Correct. In comparison,
QMix retains a high AUC of separating Correct during the whole training
process, and thus benefiting the quality-aware SSL training. Fig. 4 right
shows the joint uncertainty-loss distribution of samples. After 10 epochs of
warm-up training, Correct is initially mixed with Mis-H and Mis-L. Our QMix
manages to separate three types of samples at the final epoch.

Figure 4: Sample separation performance under symmetric noise. Left:
Comparison of correct sample AUC. Right: Visualization of the joint
uncertainty-loss distribution.

Effect of joint uncertainty-loss criterion. We compare our joint uncertainty-
loss criterion with the basic small-loss criterion that only leverages the
loss value for sample separation. As observed in Table 4, “QMix w/o
uncertainty” leads to a substantial performance drop on the Last checkpoint,
especially for high noise rates, _e.g._ , 80% Mis-H ratio. In comparison, our
QMix demonstrates superior and robust diagnostic results under both symmetric
and asymmetric scenarios, due to the improved sample separation.

Effect of sample-reweighing loss. We evaluate the effectiveness of the sample-
reweighing loss $\mathcal{L}_{mis-l}$ on mislabeled images with low quality.
“QMix w/o $\mathcal{L}_{mis-l}$” is achieved by setting $\Omega(t)$ as a
constant zero value, discarding the selected Mis-L images during the entire
training process. As revealed in the second row of Table 4, the diagnostic
performance on both the Best and Last checkpoints demonstrates a considerable
decrease, especially at a high noise ratio. Due to the insufficient model
capability in sample separation, the selected Mis-L set might contains partial
correct samples in early training epochs, and discarding $\mathcal{L}_{mis-l}$
results in the loss of useful correct samples. In contrast, our sample-
reweighing loss with a linear weight decay strategy begins with a small weight
and gradually decays to zeros. It enhances the learning process and boosts
diagnostic accuracy, particularly in the case of challenging asymmetric noise.

Effect of contrastive enhancement loss. We show the effect of the contrastive
enhancement loss $\mathcal{L}_{con}$ in the last two rows from Table 4. As
observed, incorporating $\mathcal{L}_{con}$ is proven to be effective for
diagnostic performance improvement over all the noise scenarios.
$\mathcal{L}_{con}$ pulls the representation of _any disease category_ in
correct samples far from that of Mis-L in the latent space, while maintaining
the distance between different diseases. This allows the model to focus on
disease-relevant features while minimizing the interference of mislabeled
samples with low quality on training. By discriminating Mis-L and Correct, the
sample separation process in the next round is expected to be more accurate.

Table 4: Ablation study under different types and levels of noise on DDR.  |  | Symmetric | Asymmetric
---|---|---|---
Mis-L ratio |  | 15% | 15% | 15% | 45% | 75% | 15% | 15% | 35% | 55%
Mis-H ratio |  | 20% | 50% | 80% | 20% | 20% | 30% | 50% | 20% | 20%
QMix w/o uncertainty | Best | 89.2 | 85.3 | 74.6 | 88.4 | 88.0 | 89.1 | 87.0 | 88.9 | 87.9
Last | 87.9 | 80.6 | 47.3 | 86.4 | 84.9 | 83.7 | 80.2 | 85.9 | 85.7
QMix w/o $\mathcal{L}_{mis-l}$ | Best | 89.9 | 85.2 | 74.1 | 88.7 | 88.5 | 87.2 | 86.7 | 87.0 | 86.8
Last | 87.2 | 80.4 | 69.9 | 85.2 | 87.2 | 84.8 | 82.8 | 85.2 | 85.1
QMix w/o $\mathcal{L}_{con}$ |  Best |  90.1 |  85.4 |  78.2 |  89.0 |  88.3 |  89.1 |  85.5 |  88.1 | 87.7
Last | 88.2 | 78.9 | 69.5 | 86.8 | 83.7 | 87.2 | 83.6 | 86.6 | 86.3
QMix | Best | 90.1 | 87.9 | 81.2 | 89.2 | 88.5 | 89.7 | 87.4 | 89.1 | 88.3
Last | 88.2 | 86.3 | 75.9 | 87.1 | 87.2 | 88.9 | 85.3 | 87.4 | 86.7

##  5 Conclusion

In this paper, we consider the problem of learning a robust disease diagnosis
model under mixed noise, where label noise and data noise occur in the dataset
simultaneously. We propose a novel LNL framework, termed as QMix, that
alternates sample separation and quality-aware semi-supervised learning in
each epoch. In particular, we make comprehensive studies on the memorization
effect of modern DNNs in the case of mixed noise, and design a joint
uncertainty-loss criterion to effectively separate Correct/Mis-H/Mis-L images.
In the semi-supervised training phase, we propose two novel losses, _i.e._ ,
sample re-weighing loss and contrastive enhancement loss, to mitigate the
negative impact of Mis-L on network training and enhance disease-specific
feature learning. Extensive experiments on five public retinal image datasets
show the superiority and robustness of QMix over state-of-the-art methods. In
future work, we aim to expand our method to effectively handle mixed noise in
diverse real-world medical imaging datasets, including chest X-ray and
dermoscopy.

## References

  * [1] Algan, G., Ulusoy, I.: Image classification with deep learning in the presence of noisy labels: A survey. Knowledge-Based Systems 215, 106771 (2021)
  * [2] Arazo, E., Ortego, D., Albert, P., O’Connor, N., McGuinness, K.: Unsupervised label noise modeling and loss correction. In: International conference on machine learning. pp. 312–321. PMLR (2019)
  * [3] Arpit, D., Jastrzebski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M.S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al.: A closer look at memorization in deep networks. In: International conference on machine learning. pp. 233–242. PMLR (2017)
  * [4] Chen, P., Ye, J., Chen, G., Zhao, J., Heng, P.A.: Beyond class-conditional assumption: A primary attempt to combat instance-dependent label noise. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 11442–11450 (2021)
  * [5] Cheng, D., Liu, T., Ning, Y., Wang, N., Han, B., Niu, G., Gao, X., Sugiyama, M.: Instance-dependent label-noise learning with manifold-regularized transition matrix estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16630–16639 (2022)
  * [6] Cheng, D., Ning, Y., Wang, N., Gao, X., Yang, H., Du, Y., Han, B., Liu, T.: Class-dependent label-noise learning with cycle-consistency regularization. Advances in Neural Information Processing Systems 35, 11104–11116 (2022)
  * [7] Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society: series B (methodological) 39(1), 1–22 (1977)
  * [8] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)
  * [9] Dgani, Y., Greenspan, H., Goldberger, J.: Training a neural network based on unreliable human annotation of medical images. In: 2018 IEEE 15th International symposium on biomedical imaging (ISBI 2018). pp. 39–42. IEEE (2018)
  * [10] Fu, H., Wang, B., Shen, J., Cui, S., Xu, Y., Liu, J., Shao, L.: Evaluation of retinal image quality assessment networks in different color-spaces. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 48–56. Springer (2019)
  * [11] Goldberger, J., Ben-Reuven, E.: Training deep neural-networks using a noise adaptation layer. In: International conference on learning representations (2016)
  * [12] Guo, S., Huang, W., Zhang, H., Zhuang, C., Dong, D., Scott, M.R., Huang, D.: Curriculumnet: Weakly supervised learning from large-scale web images. In: Proceedings of the European conference on computer vision (ECCV). pp. 135–150 (2018)
  * [13] Han, B., Yao, Q., Yu, X., Niu, G., Xu, M., Hu, W., Tsang, I., Sugiyama, M.: Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems 31 (2018)
  * [14] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)
  * [15] Hendrycks, D., Mazeika, M., Wilson, D., Gimpel, K.: Using trusted data to train deep networks on labels corrupted by severe noise. Advances in neural information processing systems 31 (2018)
  * [16] Hou, J., Xu, J., Xiao, F., Zhao, R.W., Zhang, Y., Zou, H., Lu, L., Xue, W., Feng, R.: Cross-field transformer for diabetic retinopathy grading on two-field fundus images. In: 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 985–990. IEEE (2022)
  * [17] Jiang, L., Zhou, Z., Leung, T., Li, L.J., Fei-Fei, L.: Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International conference on machine learning. pp. 2304–2313. PMLR (2018)
  * [18] Ju, L., Wang, X., Wang, L., Mahapatra, D., Zhao, X., Zhou, Q., Liu, T., Ge, Z.: Improving medical images classification with label noise using dual-uncertainty estimation. IEEE transactions on medical imaging 41(6), 1533–1546 (2022)
  * [19] Kaggle: Kaggle diabetic retinopathy detection competition. https://www.kaggle.com/c/diabetic-retinopathy-detection.
  * [20] Karim, N., Rizve, M.N., Rahnavard, N., Mian, A., Shah, M.: Unicon: Combating label noise through uniform selection and contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9676–9686 (2022)
  * [21] Karimi, D., Dou, H., Warfield, S.K., Gholipour, A.: Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis. Medical image analysis 65, 101759 (2020)
  * [22] Le, H., Samaras, D., Kurc, T., Gupta, R., Shroyer, K., Saltz, J.: Pancreatic cancer detection in whole slide images using noisy label annotations. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part I 22. pp. 541–549. Springer (2019)
  * [23] Li, J., Socher, R., Hoi, S.C.: Dividemix: Learning with noisy labels as semi-supervised learning. arXiv preprint arXiv:2002.07394 (2020)
  * [24] Li, T., Gao, Y., Wang, K., et al.: Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Information Sciences 501, 511–522 (2019)
  * [25] Liu, R., Wang, X., Wu, Q., Dai, L., Fang, X., Yan, T., et al.: Deepdrid: Diabetic retinopathy—grading and image quality estimation challenge. Patterns p. 100512 (2022)
  * [26] Liu, X., Zhou, B., Cheng, C.: Plremix: Combating noisy labels with pseudo-label relaxed contrastive representation learning. arXiv preprint arXiv:2402.17589 (2024)
  * [27] Malach, E., Shalev-Shwartz, S.: Decoupling" when to update" from" how to update". Advances in neural information processing systems 30 (2017)
  * [28] Nishi, K., Ding, Y., Rich, A., Hollerer, T.: Augmentation strategies for learning with noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 8022–8031 (2021)
  * [29] Ortego, D., Arazo, E., Albert, P., O’Connor, N.E., McGuinness, K.: Multi-objective interpolation training for robustness to label noise. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6606–6615 (2021)
  * [30] Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., Qu, L.: Making deep neural networks robust to label noise: A loss correction approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1944–1952 (2017)
  * [31] Qian, B., Chen, H., Wang, X., Guan, Z., Li, T., Jin, Y., Wu, Y., Wen, Y., Che, H., Kwon, G., et al.: Drac 2022: A public benchmark for diabetic retinopathy analysis on ultra-wide optical coherence tomography angiography images. Patterns (2024)
  * [32] Ren, M., Zeng, W., Yang, B., Urtasun, R.: Learning to reweight examples for robust deep learning. In: International conference on machine learning. pp. 4334–4343. PMLR (2018)
  * [33] Richardson, S., Green, P.J.: On bayesian analysis of mixtures with an unknown number of components (with discussion). Journal of the Royal Statistical Society Series B: Statistical Methodology 59(4), 731–792 (1997)
  * [34] Shen, Z., Fu, H., Shen, J., Shao, L.: Modeling and enhancing low-quality retinal fundus images. IEEE Transactions on Medical Imaging 40(3), 996–1006 (2020)
  * [35] Song, H., Kim, M., Park, D., Shin, Y., Lee, J.G.: Learning from noisy labels with deep neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems (2022)
  * [36] Tanaka, D., Ikami, D., Yamasaki, T., Aizawa, K.: Joint optimization framework for learning with noisy labels. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 5552–5560 (2018)
  * [37] Wilkinson, C., Ferris III, F.L., Klein, R.E., Lee, P.P., Agardh, C.D., Davis, M., et al.: Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales. Ophthalmology 110(9), 1677–1682 (2003)
  * [38] Xue, C., Dou, Q., Shi, X., Chen, H., Heng, P.A.: Robust learning at noisy labeled medical images: Applied to skin lesion classification. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). pp. 1280–1283. IEEE (2019)
  * [39] Yi, K., Wu, J.: Probabilistic end-to-end noise correction for learning with noisy labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7017–7025 (2019)
  * [40] Yu, X., Han, B., Yao, J., Niu, G., Tsang, I., Sugiyama, M.: How does disagreement help generalization against label corruption? In: International Conference on Machine Learning. pp. 7164–7173 (2019)
  * [41] Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning (still) requires rethinking generalization. Communications of the ACM 64(3), 107–115 (2021)
  * [42] Zhou, Y., Huang, L., Zhou, T., Sun, H.: Combating medical noisy labels by disentangled distribution learning and consistency regularization. Future Generation Computer Systems 141, 567–576 (2023)