# Non-equilibrium spatial distribution of Rashba spin torque in ferromagnetic
metal layer

N. L. Chung<EMAIL_ADDRESS>Computational Nanoelectronics and Nano-device
Laboratory, Electrical and Computer Engineering Department National University
of Singapore, 4 Engineering Drive 3, Singapore 117576.  M. B. A. Jalil
Computational Nanoelectronics and Nano-device Laboratory, Electrical and
Computer Engineering Department National University of Singapore, 4
Engineering Drive 3, Singapore 117576.  Information Storage Materials
Laboratory, Electrical and Computer Engineering Department National University
of Singapore, 4 Engineering Drive 3, Singapore 117576.  S. G. Tan
Computational Nanoelectronics and Nano-device Laboratory, Electrical and
Computer Engineering Department National University of Singapore, 4
Engineering Drive 3, Singapore 117576.  Data Storage Institute, A*STAR (Agency
for Science, Technology and Research), DSI Building, 5 Engineering Drive 1,
Singapore 117608.

###### Abstract

We study the spatial distribution of spin torque induced by a strong Rashba
spin-orbit coupling (RSOC) in a ferromagnetic (FM) metal layer, using the
Keldysh non-equilibrium Green’s function method. In the presence of the
$s$-$d$ interaction between the non-equilibrium conduction electrons and the
local magnetic moments, the RSOC effect induces a torque on the moments, which
we term the Rashba spin torque.

A correlation between the Rashba spin torque and the spatial spin current is
presented in this work, clearly mapping the spatial distribution of Rashba
spin torque in a nano-sized ferromagnetic device. When local magnetism is
turned on, the out-of-plane ($S_{z}$) Spin Hall effect (SHE) is disrupted, but
rather unexpectedly an in-plane ($S_{y}$) SHE is detected. We also study the
effect of Rashba strength ($\alpha_{R}$) and splitting exchange ($\Delta$) on
the non-equilibrium Rashba spin torque averaged over the device. Rashba spin
torque allows an efficient transfer of spin momentum such that a typical
switching field of 20 mT can be attained with a low current density of less
than $10^{7}$A/$\mathrm{cm}^{2}$.

###### pacs:

72.25.-b,72.25.Ba,74.78.Na,75.75.-c

##  I Introduction

Ever since the theoretical prediction of the spin transfer torque (STT)
slonczewski:jmmm159 ; berger:9353 , there has been much research effort in
utilizing the STT phenomenon to induce magnetization switching and precession
in ferromagnetic (FM) nanostructures without the need for an externally
applied magnetic field. Devices which rely on the STT effect for magnetization
switching offer the advantages of lower power consumption and reduced device
dimension, which are crucial factors for nanoscale and high-density spintronic
applications. The STT effect has been studied in conventional magnetic
nanostructures such as spin valves katine:prl3149  and magnetic tunneling
junctions huai:apl84 . For STT to occur in these magnetic multilayers, one
requires a pair of FM layers, i.e., a reference spin layer to generate a spin-
polarized current for injection into the second free (switchable) layer. The
two layers are magnetized in a noncollinear configuration so as to induce the
transfer of the transverse spin momentum from the reference to the free layer,
which is mediated by conduction electrons flowing between the two layers. In
the above process, the role of the spin-orbit coupling (SOC) effect is
neglected. However, it is well-established that SOC can generate a
nonequilibrium spin accumulation under the passage of current. Thus, it is
conceivable that, in the presence of strong SOC effect, one can induce a STT
without the need for an additional reference FM layer. This is corroborated by
previous theoretical work which showed that the presence of Rashba spin-orbit
coupling (RSOC) whose strength is denoted by $\alpha_{R}$, and exchange
interaction $\Delta$ between conduction electrons and local spins, can give
rise to domain wall motion via spin momentum transfer obata:prb77 . The same
spin transfer mechanism can also occur in a FM layer with a large $\alpha_{R}$
and $\Delta$ values tan:annal326 ; manchon:prb78 . The predicted RSOC-induced
spin momentum transfer was experimentally demonstrated in a nanowire array
mihai:natm9 ; pi:apl97 . The above findings suggest that by utilizing Rashba-
induced STT, one can achieve magnetization switching within a single FM layer,
without an additional non-collinear FM layer. Such _single layer switching_
holds several potential advantages over conventional STT devices, such as a
more symmetric current switching profile and the reduced influence of spin
depolarization at the interfaces.

A key element which determines the feasibility of Rashba STT is the presence
of a strong Rashba SOC in the FM metal layer. Initial studies on the Rashba
effect were focused on semiconductor (SC) materials miller:prl90 ; sato:jap89
; giglberger:prb75 ; larionov:prb78 ; akabori:prb77 , especially in two-
dimensional electron gas (2DEG) heterojunction structures, which consist of
two SC layers with different energy bandgaps. The conduction electrons in the
2DEG experience a strong RSOC effect due to the large potential gradient, as a
result of the band-bending at the heterojunction interface. However, utilizing
the Rashba-induced STT in SC materials is not an attractive proposition as SCs
are intrinsically non-magnetic. Even if ferromagnetic behavior can be induced
in them via doping (e.g. in dilute magnetic semiconductors or DMS), the
resulting Curie temperature lies well below room temperature. Recent studies
have shown, however, that a strong RSOC effect can also be induced in metallic
nanostructures, both of the FM and non-FM types lashell:prl77 ; ast:prl98 ;
cercellier:prb73 ; krupin:prb71 . It is known that the Rashba SOC requires a
structural inversion asymmetry (SIA), which gives rise to an internal electric
field. In a metallic FM layer, the SIA can be enhanced by adjacent layers of
heavy metals and oxides, which create the requisite band structure mismatch
and large potential gradient at the interfaces premper:prb76 ; christian:prb77
; abdelouahed:prb82 ; dil.prl101 . By engineering the interfaces of the
metallic FM layer, one can control the strength of the RSOC effect within the
layer. The ability to enhance the RSOC coupling via interfacial effects has
led to the experimental demonstration of the effect of Rashba-induced STT, as
mentioned previously mihai:natm9 ; pi:apl97 . However, to effectively harness
this effect in future magnetic memory applications, it is essential to have an
understanding of the microscopic spin transport in the presence of the RSOC
effect, and the resulting non-equilibrium spatial distribution of the Rashba-
induced STT.

Thus, in this paper, we apply the Keldysh nonequilibrium Green’s function
(NEGF) technique to study the spin torque generated by the Rashba SOC on the
local magnetization in a metallic FM layer. The NEGF method is suitable for
the study of the Rashba STT, which is essentially driven by nonequilibrium
spin accumulation generated by the passage of current in the presence of RSOC.
In addition, the NEGF method can systematically incorporate the effects of the
leads, and interactions (RSOC and exchange coupling) as self-energy terms. In
Section II of the paper, we introduce the system Hamiltonian, consisting of
the Rashba term $H_{SO}=\alpha_{R}(\hat{\bm{z}}\times\hat{\bm{p}})$, where
$\hat{\bm{z}}$ is a unit vector parallel to the internal electric field
${\bm{E}}$, which acts perpendicular to the FM layer, $\hat{\bm{p}}$ is the
electron wavevector and $\alpha_{R}$ is the RSOC strength. The Hamiltonian
also includes the s-d interaction characterized by the exchange energy
$\Delta$, which couples the nonequilibrium spin density due to RSOC effect to
the local moments. Based on the second-quantized form of the Hamiltonian, we
apply the tight-binding NEGF formalism, and calculate various microscopic
transport quantities in the system, such as the local spin current and spin
density, and the overall spin torque generated. In Section III, we numerically
investigate (i) the spin torque efficiency as a function of the strengths of
the RSOC effect $(\alpha_{R})$, and the $s$-$d$ exchange interaction
($\Delta$), (ii) the relationship between the spin torque distribution and the
local spin currents, and (iii) the in-plane spin Hall effect arising from
RSOC. Finally, the summary of results and conclusion are presented in Section
IV.

##  II Theory and Model

Figure 1: Schematic diagram of a ferromagnetic (FM) layer sandwiched between
two dissimilar materials (oxides or heavy elements) to increase the vertical
electric field $E_{z}$ and thus enhance the Rashba SOC effect. Current
$\bm{j}_{e}$ flows in the in-plane $x$-direction. The magnetization of the FM
layer $\bm{M}$ is oriented in the vertical $z$-direction.

The structure under consideration is depicted in Fig. 1. It consists of a
metallic FM layer, sandwiched between two dissimilar materials (oxides or
heavy elements) to enhance the RSOC interaction at the interfaces and within
the FM layer. The local magnetization $\bm{M}$ is oriented along the vertical
$z$-direction. A charge current $\bm{\hat{j}}_{e}$ is injected in the
$x$-direction, which generates a field $H_{\mathrm{eff}}$ along the
$\bm{\hat{y}}=(\bm{\hat{z}}\times\bm{\hat{j}}_{e}$) direction. The Hamiltonian
for the system can be expressed as:

$\displaystyle\hat{H}$ $\displaystyle=\hat{H}_{0}+\hat{H}_{so},$ (1)
$\displaystyle\hat{H}_{0}$
$\displaystyle=\frac{\hat{\bm{p}}^{2}}{2m}-\Delta({\bm{M}}\cdot\bm{\hat{S}}),$
(2) $\displaystyle\hat{H}_{so}$
$\displaystyle=\frac{\alpha_{R}}{\hbar}(\hat{\bm{p}}\times\hat{\bm{z}})\cdot\hat{\bm{S}},$
(3)

where $m$ is the free electron mass, and $\hbar$ is the reduced Planck’s
constant. Here, $H_{0}$ denotes the kinetic energy of the conduction electrons
in the FM layer, ${\bm{M}}$ is the magnetization direction, $\Delta$ is the
exchange coupling between the free electron spin and the local moments, and
$\hat{\bm{S}}=\\{\hat{S}_{j}\\}$ (where $j=\\{x,y,z\\}$) is the vector of
Pauli spin matrices. $\hat{H}_{so}$ denotes the Rashba interaction which
couples the electron spin with its momentum, with $\hat{\bm{p}}$ being the
electron momentum, and the potential gradient inducing the RSOC effect being
assumed to be in the direction $\bm{z}$ normal to the FM layer. The potential
gradient may arise from a variety of sources such as impurities, host atoms,
and structural confinement sih.nat1 ; kim.apl012504 ; szunyogh.prl96 ;
castro.prl103 ; matos.prb81 . In order to apply the many-body NEGF formalism,
the above Hamiltonian has to be recast into the second quantized form:

$\displaystyle\hat{H}_{0}$
$\displaystyle=-t_{0}\sum_{\bm{r},\sigma}\sum_{\pm}c^{{\dagger}}_{\bm{r}\sigma}(c_{\bm{r}\pm\bm{a}\sigma}+c_{\bm{r}\pm\bm{b}\sigma})+\sum_{\bm{r},\sigma}\varepsilon_{\bm{r}\sigma}c^{{\dagger}}_{\bm{r}\sigma}c_{\bm{r}\sigma}$
(4) $\displaystyle\hat{H}_{so}$
$\displaystyle=-it_{SO}\sum_{\bm{r},\sigma,\sigma^{\prime}}\sum_{\pm}c^{{\dagger}}_{\bm{r}\sigma}(\pm(\hat{S}_{x})_{\sigma\sigma^{\prime}}c_{\bm{r}\pm\bm{b}\sigma^{\prime}}\mp(\hat{S}_{y})_{\sigma\sigma^{\prime}}c_{\bm{r}\pm\bm{a}\sigma^{\prime}}).$
(5)

where $c_{\bm{r}\sigma}$($c_{\bm{r}\sigma}^{{\dagger}}$) is the fermionic
annihilation(creation) operator of an electron with spin
$\sigma=\uparrow,\downarrow$ at position $\bm{r}$. Here, $a$ is the lattice
spacing representation on a square lattice in the tight-binding NEGF
formulation , $\bm{a}=a\bm{e}_{x}$ and $\bm{b}=a\bm{e}_{y}$ are the unit
lattice vectors. $t_{0}$ represents the hopping energy between lattice points,
and is obtained by $t_{0}=\hbar/2ma^{2}$. The terms
$\varepsilon_{\bm{r}\uparrow}=4t_{0}+\Delta/2$ and
$\varepsilon_{\bm{r}\downarrow}=4t_{0}-\Delta/2$ represent the on-site energy
at the lattice site, and $t_{SO}=\alpha_{R}/2a$ is the SO coupling energy due
to the Rashba interaction.

In order to perform numerical analysis through the NEGF, the retarded
($G^{r}$) and lesser ($G^{<}$) Green’s functions are required. These are
defined as

$\displaystyle
G^{r}_{\bm{r}\sigma,\bm{r}^{\prime}\sigma^{\prime}}(t,t^{\prime})$
$\displaystyle=i\langle\\{c_{\bm{r}\sigma}(t),c^{{\dagger}}_{\bm{r^{\prime}}\sigma^{\prime}}(t^{\prime})\\}\rangle\theta(t-t^{\prime}),$
(6) $\displaystyle
G^{<}_{\bm{r}\sigma,\bm{r}^{\prime}\sigma^{\prime}}(t,t^{\prime})$
$\displaystyle=i\langle
c^{{\dagger}}_{\bm{r^{\prime}}\sigma^{\prime}}(t^{\prime})c_{\bm{r}\sigma}(t)\rangle$
(7)

After Fourier transformation, the expression for $G^{r}$ in energy space is
given by

$\displaystyle G^{r}(\epsilon)$
$\displaystyle=[\epsilon-\hat{H}-\Sigma^{r}(\epsilon)]^{-1}.$ (8)

In the above, $\Sigma^{r}=\sum_{\alpha}\Sigma_{\alpha}^{r}$ is the retarded
self-energy incurred by the lead $\alpha$, where $\alpha=L(R)$ represents the
left (right) lead. $\Sigma_{\alpha}^{r}$ can be determined by
$\Sigma_{\alpha}^{r}=V_{\alpha}g_{\alpha}^{r}V^{\dagger}_{\alpha}$, where
$V_{\alpha}$ is the coupling matrix between the lead $\alpha$ and the FM
layer, and $g_{\alpha}^{r}$ is the retarded Green’s function of the lead
$\alpha$ and can be calculated numerically by the renormalization method
nikolic:prb73 . $G^{<}$ can be calculated from the relation

$\displaystyle G^{<}(\epsilon)$
$\displaystyle=G^{r}(\epsilon)\Sigma^{<}(\epsilon)G^{a}(\epsilon).$ (9)

where $G^{a}=(G^{r})^{{\dagger}}$.
$\Sigma^{<}=\sum_{\alpha}\Sigma_{\alpha}^{<}$, where
$\Sigma_{\alpha}^{<}=if_{\alpha}\Gamma_{\alpha}$ is the lesser self-energy due
to lead $\alpha$, $f_{\alpha}$ is the Fermi function in lead $\alpha$, and
$\Gamma_{\alpha}=-2$Im$\Sigma^{r}_{\alpha}$ is the linewidth function
representing the coupling between the lead $\alpha$ and the central FM region.

Various transport properties can be evaluated once the different Green’s
functions ($G^{r}$, $G^{a}$, and $G^{<}$) have been solved via Eqs. (8) and
(9). The charge current through the system can be expressed in terms of the
different Green’s functions, as follows

$\displaystyle I_{\alpha}=$
$\displaystyle\frac{e}{h}\int^{+\infty}_{-\infty}d\epsilon\medspace\mathrm{Tr}\left\\{\left[\Sigma^{<}_{\alpha}(\epsilon)A(\epsilon)\right]-\left[\Gamma(\epsilon)_{\alpha}G^{<}(\epsilon)\right]\right\\},$
(10)

where $A(\epsilon)=i[G^{r}(\epsilon)-G^{a}(\epsilon)]$ is spectral function.
Likewise, the current-driven local spin density in the central region is
related to $G^{<}$ as follows

$\displaystyle\langle s_{i}\rangle_{m}$
$\displaystyle=\frac{\hbar}{2}\sum_{\sigma\sigma^{\prime}}(\hat{S}_{i})_{\sigma\sigma^{\prime}}\langle\hat{c}^{\dagger}_{m\sigma}\hat{c}_{m\sigma^{\prime}}\rangle$
$\displaystyle=\frac{\hbar}{4\pi
i}\int^{\infty}_{-\infty}d\epsilon\sum_{\sigma\sigma^{\prime}}(\hat{S}_{i})_{\sigma\sigma^{\prime}}G^{<}_{mm,\sigma\sigma^{\prime}}(\epsilon),$
$\displaystyle=\frac{\hbar}{4\pi
i}\int^{\infty}_{-\infty}d\epsilon\medspace\mathrm{Tr}[\hat{S}_{i}G^{<}_{mm}(\epsilon)],$
(11)

where the subscript $m$ refers to the site index.

The spin torque exerted on the local magnetization can be defined as the
difference between spin current going into and coiming out of the lattice
point. We express the spin torque as the divergence of spin
currentsalahuddin:arxiv :

$\displaystyle{\bm{\tau}}$ $\displaystyle=\mu_{B}\int
dV\nabla\cdot\bm{j}^{S_{i}},$ (12)

where $V$ is the volume, $\mu_{B}$ is the Bohr magneton, and $\bm{j}^{S_{i}}$
is the spin current density between lattice points. The spin torque
${\bm{\tau}}$ can be also be defined as
$\bm{\tau}=-\gamma\bm{M}\times{\bm{H}}$, where $\gamma$ is the gyromagnetic
ratio. We focus on the effective field $H_{\mathrm{eff}}$ induced by Rashba
SOC which acts on the local moments along the
$\bm{\hat{y}}=(\bm{\hat{z}}\times\bm{\hat{j}}_{e}$) direction. Thus, the
effective field due to RSOC is

$\displaystyle H_{\mathrm{eff}}$ $\displaystyle=H_{y}=\frac{\tau_{x}}{\gamma
M_{s}V},$ (13)

where $M_{s}$ is the saturation magnetization, and $\tau_{x}$ is obtained from
Eq. (12). The torque efficiency is then given by ratio of
$H_{\mathrm{eff}}/{j}_{e}$.

Under steady-state condition and in the absence of dissipative processes, the
spin torque ${\bm{\tau}}$, as defined according to Eq. (12), is related to the
divergence of the spin current. By considering the Heisenberg equation of
motion, the local spin bond current between sites $m$ and $m^{\prime}$ can be
expressed in terms of $G^{<}$ nikolic:prb73 ; hattori:JPSJ78 , i.e.

$\displaystyle\langle\hat{j}^{s_{i}}_{mm^{\prime}}\rangle$
$\displaystyle=\langle\hat{j}^{s_{i}(kin)}_{mm^{\prime}}\rangle+\langle\hat{j}^{s_{i}(SO)}_{mm^{\prime}}\rangle,$
$\displaystyle\langle\hat{j}^{s_{i(kin)}}_{mm^{\prime}}\rangle$
$\displaystyle=\frac{et_{0}}{2\hbar}\int^{\infty}_{-\infty}\frac{d\epsilon}{2\pi}\medspace\mathrm{Tr}[\hat{S}_{i}(G^{<}_{m^{\prime}m}(\epsilon)-G^{<}_{mm^{\prime}}(\epsilon))]$
$\displaystyle\langle\hat{j}^{s_{i(SO)}}_{mm^{\prime}}\rangle$
$\displaystyle=[\bm{e}_{i}\times(\bm{m^{\prime}}-\bm{m})]_{z}\frac{et_{SO}}{2\hbar}\int^{\infty}_{-\infty}\frac{d\epsilon}{2\pi
i}\medspace\mathrm{Tr}[(G^{<}_{m^{\prime}m}(\epsilon)+G^{<}_{mm^{\prime}}(\epsilon))],$
(14)

where $(\bm{m^{\prime}}-\bm{m})$ represents the unit vector between
neighbouring sites on the $x$-$y$ plane and $\bm{e}_{i}$ represents the unit
vector of spin $\langle S_{i}\rangle$. The above expression for the bond spin
current comprises of two terms, i.e., the kinetic and SO coupling terms,
arising from the corresponding terms in the Hamiltonian of Eqs. (2) and (3).
By considering Eqs. (12) and (14) together, the spin torque is then given by
the divergence of the spin bond current $(\nabla\cdot\bm{j}^{s})$, which in
the discretized tight-binding model is approximated as:nikolic:prb73 ;
hattori:JPSJ78

$\displaystyle\tau_{x(\bm{m})}$
$\displaystyle=-\mu_{B}\left(\langle\hat{j}^{s_{z}}_{\bm{m},\bm{m}+\bm{e}_{x}}\rangle+\langle\hat{j}^{s_{z}}_{\bm{m}-\bm{e}_{x},\bm{m}}\rangle\right)/L_{SO},$
(15) $\displaystyle\tau_{y(\bm{m})}$
$\displaystyle=-\mu_{B}\left(\langle\hat{j}^{s_{z}}_{\bm{m},\bm{m}+\bm{e}_{y}}\rangle+\langle\hat{j}^{s_{z}}_{\bm{m}-\bm{e}_{y},\bm{m}}\rangle\right)/L_{SO},$
(16) $\displaystyle\tau_{z(\bm{m})}$
$\displaystyle=\mu_{B}\left(\langle\hat{j}^{s_{x}}_{\bm{m},\bm{m}+\bm{e}_{x}}\rangle+\langle\hat{j}^{s_{x}}_{\bm{m}-\bm{e}_{x},\bm{m}}\rangle+\langle\hat{j}^{s_{y}}_{\bm{m},\bm{m}+\bm{e}_{y}}\rangle+\langle\hat{j}^{s_{y}}_{\bm{m}-\bm{e}_{y},\bm{m}}\rangle\right)/L_{SO},$
(17)

where $L_{SO}$ is the spin precession length (over which spin precesses by 1
radian), and can be expressed as $L_{SO}=\frac{\pi at_{0}}{2t_{SO}}$. The
above constitutes to the spin torque expression of Eq. (12).

##  III Results and Discussion

Based on the tight-binding NEGF formulation presented in the above section, we
performed numerical calculations of transport parameters such as the local
spin density, bond spin current, and the effective field $H_{\mathrm{eff}}$ in
order to analyze the effect of RSOC induced non-equilibrium spatial spin
torque on the FM layer structure. In our calculations, the following parameter
values are assumed, unless otherwise stated:
$\alpha_{R}=10^{-11}~{}\mathrm{eVm}$, $m=9.1\times 10^{-31}$ kg,
$a=0.05~{}\mathrm{nm}$, $E_{F}=7.83~{}\mathrm{eV}$, $M_{s}=1.09\times
10^{6}~{}\mathrm{Am^{-1}}$, $\Delta=1.6~{}\mathrm{eV}$ Miranda.surf117 , and
room temperature $T=300$ K. The Fermi energy $E_{F}$ and saturation
magnetization $M_{s}$ assume exemplary values corresponding to that of Co.

Figure 2: The dependence of the effective current induced field
($H_{\mathrm{eff}}$) due to the Rashba spin torque is plotted as a function of
charge current density (${j}_{e}$) for (a) varying Rashba strength
$\alpha_{R}$ with a fixed exchange coupling $\Delta=1.6$ eV, and (b) varying
exchange coupling $\Delta$ with a fixed $\alpha_{R}=10^{-10}$ eVm. In (c), the
spin torque efficiency ($H_{\mathrm{eff}}/j_{e}$) is plotted as a function of
both $\Delta$ and $\alpha_{R}$. In the calculations, we assume the dimension
of the sample to be $50a\times 50a$, where $a=0.05$ nm.

We first analyze the role of two key parameters $\alpha_{R}$ and $\Delta$ in
determining the strength of the effective field $H_{\mathrm{eff}}$ and the
torque efficiency of the system. Figs. 2 and 2 show that, with a fixed
$\alpha_{R}$ and $\Delta$ respectively, $H_{\mathrm{eff}}$ increases linearly
with $j_{e}$. This trend is consistent with the prediction that

$\displaystyle
H_{\mathrm{eff}}=\frac{\alpha_{R}P}{\mu_{0}\mu_{B}}(\hat{\bm{z}}\times\bm{j}_{e}),$
(18)

derived from either gauge formulation tan:annal326  or from semiclassical
(Boltzmann) transport equation manchon:prb78  in the strong coupling limit.
Eq. (18) is a global expression of spin torque under linear response. In the
gauge formulation, the factor $P$ assumes a value of $\frac{1}{2}$ in the
adiabatic limit, while in the Boltzmann model, it refers to the spin
polarization of current. We now consider the torque efficiency, which is given
by the gradient of $H_{\mathrm{eff}}$ with respect to $j_{e}$. As can be seen
from Figs. 2 and 2, the torque efficiency is generally enhanced with increase
in either $\alpha_{R}$ and $\Delta$. However, in our non-equilibrium spatial
treatment, it is clear from the plot in Fig. 2 that the torque efficiency does
not vary linearly with $\alpha_{R}$, unlike the prediction of Eq. (18). The
difference can be accounted for by noting that the global expression of Eq.
(18) is derived in the limit of large coupling $\Delta$, i.e., up to only the
linear order in $\frac{\alpha_{R}}{\Delta}$. In our model, as can be seen from
Fig. 2, the torque efficiency shows a slight oscillatory dependence
superimposed upon a general increase with respect to $\alpha_{R}$, especially
at the region of $\alpha_{R}<10^{-10}$ eVm. However, at the region where
$\alpha_{R}\geq 10^{-10}$ eVm, its behavior is similar to the prediction
derived from the Boltzmann semiclassical model for arbitrary coupling strength
manchon:prb78 . From the effective field $H_{\mathrm{eff}}$, one can estimate
the critical current density required for magnetization switching. In Figs. 2
and 2, we consider RSOC strengths ranging from $10^{-11}$ to $10^{-10}$ eVm,
which roughly corresponds to the practical values observed at the interfaces
with heavy metal or oxide layers. Assuming an exemplary spin polarization of
$P=\frac{\Delta}{E_{F}}\approx 0.5$, RSOC strength of $\alpha_{R}=10^{-10}$
eVm, and a switching field of $H_{s}\approx 0.02$ T applicable for Co nanowire
structures mihai:natm9 , we find that the critical current density for
switching is approximately $10^{6}$ A/$\mathrm{cm}^{2}$ [see Fig. 2]. This is
significantly lower than the critical current density of the order of $10^{7}$
A/$\mathrm{cm}^{2}$ for the case of the conventional Slonczewski spin torque
in spin valve structures jiang:prl92 ; sukegawa:apl96 .

Figure 3: The spatial distribution of the (a) Rashba effect spin torque
$\tau_{x}$ and its correlation with the local spin current $\langle
j^{s_{z}}_{mm^{\prime}}\rangle$ by setting $\alpha_{R}$ to $0.5\times
10^{-10}$ eVm, (b) $\tau_{x}$ and its correlation with $\langle
j^{s_{z}}_{mm^{\prime}}\rangle$ by setting $\alpha_{R}$ to $1.5\times
10^{-10}$ eVm. The spin torque density is expressed in units of
$\mu_{B}/L_{SO}$. The sample has a lateral size of $50a\times 50a$.

Next, we examine the relationship between the Rashba-induced torque $\tau$ and
the spatial distribution of the spin currents. In Fig. 3, we plot the spin
torque component $\tau_{x}$ based on the torque definition of Eq. (15), which
relates it to the divergence of the local spin bond current
$j^{s_{z}}_{mm^{\prime}}$. For comparison, we plot the spatial distribution of
the spin bond current $j^{s_{z}}_{mm^{\prime}}$ in Fig. 3. We observe a close
correlation between the spatial distribution of $\tau_{x}$ and the flow of the
$z$-polarized spin current $j^{s_{z}}_{mm^{\prime}}$. The presence of RSOC
causes a vortex-like flow of the bond spin current $j^{s_{z}}_{mm^{\prime}}$
as shown in Fig. 3. Regions where $j^{s_{z}}_{mm^{\prime}}$ is flowing in the
$+x$ ($-x$) direction corresponds to a large positve (negative) $\tau_{x}$.
Conversely, in regions where the positive and negative spin current fluxes
meet and cancel each other, the spin torque $\tau_{x}$ becomes small. When the
Rashba coupling strength $\alpha_{R}$ is increased, the magnitude of
$\tau_{x}$ is generally larger since it scales with $\alpha_{R}$, as shown in
Fig. 2. In addition, the vortices associated with the spin current become
spatially smaller. This may be attributed to the increase in the rate of spin
precession of the conduction electrons with $\alpha_{R}$. The increased
density of the vortices result in some cancelation of the bond spin currents
near the center of the FM layer, so that more of the bond spin current flows
at the boundaries, as shown in Fig. 3.

(a) $\Delta=0$, $\alpha_{R}=10^{-10}$ eVm, $\langle s_{z}\rangle_{m}$. SHE
recovered.

(b) $\Delta=0$, $\alpha_{R}=10^{-10}$ eVm, $\langle s_{y}\rangle_{m}$. Absence
of SHE.

(c) $\Delta=1.6$ eV, $\alpha_{R}=5\times 10^{-11}$ eVm, $\langle
s_{z}\rangle_{m}$. SHE disrupted

(d) $\Delta=1.6$ eV, $\alpha_{R}=5\times 10^{-11}$ eVm, $\langle
s_{y}\rangle_{m}$. SHE detecteded.

(e) $\Delta=1.6$ eV, $\alpha_{R}=1.5\times 10^{-10}$ eVm, $\langle
s_{y}\rangle_{m}$. SHE disrupted.

Figure 4: The spatial distribution of the spin density (a) $\langle
s_{z}\rangle_{m}$, (b) $\langle s_{y}\rangle_{m}$, both with $\Delta=0$ eV,
$\alpha_{R}=1\times 10^{-10}$ eVm, (c) $\langle s_{z}\rangle_{m}$, (d)
$\langle s_{y}\rangle_{m}$, both with $\Delta=1.6$ eV, $\alpha_{R}=1.5\times
10^{-10}$ eVm. In (e) $\langle s_{y}\rangle_{m}$ is plotted with a larger
$\alpha_{R}=1.5\times 10^{-10}$ eVm, and $\Delta=1.6$ eV. The sample has a
lateral size of $50a\times 50a$.

Finally, we analyze the spin density distribution and its dependence on the
exchange strength $\Delta$. Figs. 4(a) and 4(b) plot the spin density of
$\langle s_{z}\rangle_{m}$ in the absence and presence of $\Delta$,
respectively. In the absence of exchange coupling ($\Delta=0$), the
distribution profile of $\langle s_{z}\rangle_{m}$ clearly indicates a
transverse separation of the $z$-spins, i.e. an out-of-plane spin Hall effect.
This agrees with previous calculations based on the multimode scattering
matrix method which predict a spin-Hall like separation of the out-of-plane
spin component in the presence of Rashba effect brusheim:prb74 . However, the
clear out-of-plane spin Hall separation disappears when a sizable exchange
$\Delta$ is present, as shown in Fig. 4(c). It is found that the magnitude of
$\langle s_{z}\rangle_{m}$ assumes a much larger value throughout the FM
layer. This increase may be attributed to the alignment of the electron spin
to the local moments oriented along the $z$-direction. We also analyze the in-
plane spin density $\langle s_{y}\rangle_{m}$ distribution, as shown in Figs.
4(b), 4(d), and 4(e). There is no transverse separation of the in-plane spin
density in the absence of $\Delta$ [Fig. 4(b)]. This is in line with
theoretical prediction where the spin Hall effect induced by RSOC applies only
to out-of-plane spins. However, in the presence of strong exchange coupling
$\Delta$, an “in-plane” spin Hall effect is present [Fig. 4(d)]. This in-plane
spin Hall effect is destroyed in the presence of a strong Rashba strength,
i.e. when $\alpha_{R}$ is increased to $1.5\times 10^{-10}$ eVm [Fig. 4(e)].
This may be explained by noting that a large RSOC strength increases the rate
of spin precession. Thus, the in-plane spin density $\langle s_{y}\rangle_{m}$
oscillates and changes signs along the direction of electron propagation
($x$-direction), as can be seen in Fig. 4(d).

##  IV Conclusion

In summary, we have studied the non-equilibrium spatial intrinsic spin torque
induced by Rashba spin orbit coupling in a ferromagnetic metal layer. Unlike
the conventional Slonczewski spin torque, the Rashba induced torque is
generated within a single layer, i.e. it does not require spin injection from
an another ferromagnetic reference layer. We analyze the effect of two crucial
parameters determining the strength of the Rashba spin torque: (i) the
strength $\alpha_{R}$ of the RSOC effect which is responsible for polarizing
the injected charge current, and (ii) the exchange splitting $\Delta$ which
couples the conduction electron to the local FM moments, thus allowing the
transfer of spin momentum to the latter. The spin transport through the system
is modeled via the tight-binding non-equilibrum Green’s function (NEGF)
formalism. The NEGF theory systematically incorporates many-body effects
including interactions with the leads as self-energy terms, and enables
current and spin density to be evaluated spatially under nonequilibrium (bias-
driven) conditions. Based on the NEGF theory, we numerically evaluate various
transport parameters of the system, such as the effective field
$H_{\mathrm{eff}}$ due to the spin torque, and the spatial distribution of the
non-equilibrium spin current and spin accumulation. We found that
$H_{\mathrm{eff}}$ generally increases with both the RSOC strength
$\alpha_{R}$ and the exchange coupling $\Delta$. However, the dependence of
$H_{\mathrm{eff}}$ on both parameters is not totally linear, unlike previous
predictions based on gauge formulation or semiclassical Boltzmann which are
global and only partially non-equilibrium (linear response), and in the strong
coupling limits. For practical values of $\Delta$ and $\alpha_{R}$, the
calculated critical current density corresponding to a typical switching field
of 200 mT is calculated to be lower than $10^{7}$ A/$\mathrm{cm}^{2}$,
comparable to that obtained via the conventional Slonczewski spin torque. For
the structure under consideration where net current is in the $x$-direction
and the local moments are aligned in the vertical $z$-direction, the net
effective field (spin torque) is in the $y$ ($x$)-direction. We plot the
spatial profile of the $x$-component of the spin torque $\tau_{x}$, which
bears a close correlation to that of the $z$-polarized bond spin current. It
is also observed that the Rashba torque $\tau_{x}$ is concentrated near the
boundaries of the FM layer. We also found that the combined presence of RSOC
effect and exchange coupling $\Delta$ induces a Hall separation of in-plane
spins, whereas the spin Hall effect for out-of-plane spins disappear with the
introduction of $\Delta$. Our calculations predict an effective field
$H_{\mathrm{eff}}$ of the order of 1 Tesla for a current density of $10^{7}$
$\mathrm{A/cm}^{2}$, thus indicating the feasibility of utilizing the Rashba
induced spin torque to achieve magnetization switching in spintronic
applications.

## References

  * (1) J. C. Slonczewski,  J. Magn. Magn. Mater. 159, L1 (1996).
  * (2) L. Berger,  Phys. Rev. B 54, 9353 (1996).
  * (3) J. A. Katine, F. J. Albert, R. A. Buhrman, E. B. Myers, and D. C. Ralph,  Phys. Rev. Lett. 84, 3149 (2000).
  * (4) Y. Huai, F. Albert, P. Nguyen, M. Pakala, and T. Valet,  Appl. Phys. Lett. 84, 3118 (2004).
  * (5) K. Obata and G. Tatara,  Phys. Rev. B 77, 214429 (2008).
  * (6) S. G. Tan, M. B. A. Jalil, T. Fujita, and X.-J. Liu,  Ann. Phys. 326, 207 (2011), (S. G. Tan and M. B. A. Jalil and X-J Liu[arXiv:0705.3502v1]).
  * (7) A. Manchon and S. Zhang,  Phys. Rev. B 78, 212405 (2008).
  * (8) I. Mihai Miron et al.,  Nat. Mater. 9, 230 (2010).
  * (9) U. H. Pi et al.,  Appl. Phys. Lett. 97, 162507 (2010).
  * (10) J. B. Miller et al.,  Phys. Rev. Lett. 90, 076807 (2003).
  * (11) Y. Sato, T. Kita, S. Gozu, and S. Yamada,  J. Appl. Phys. 89, 8017 (2001).
  * (12) S. Giglberger et al.,  Phys. Rev. B 75, 035327 (2007).
  * (13) A. V. Larionov and L. E. Golub,  Phys. Rev. B 78, 033302 (2008).
  * (14) M. Akabori et al.,  Phys. Rev. B 77, 205320 (2008).
  * (15) S. LaShell, B. A. McDougall, and E. Jensen,  Phys. Rev. Lett. 77, 3419 (1996).
  * (16) C. R. Ast et al.,  Phys. Rev. Lett. 98, 186807 (2007).
  * (17) H. Cercellier et al.,  Phys. Rev. B 73, 195413 (2006).
  * (18) O. Krupin et al.,  Phys. Rev. B 71, 201403 (2005).
  * (19) J. Premper, M. Trautmann, J. Henk, and P. Bruno,  Phys. Rev. B 76, 073310 (2007).
  * (20) C. R. Ast et al.,  Phys. Rev. B 77, 081407 (2008).
  * (21) S. Abdelouahed and J. Henk,  Phys. Rev. B 82, 193411 (2010).
  * (22) J. H. Dil et al.,  Phys. Rev. Lett. 101, 266802 (2008).
  * (23) V. Sih et al.,  Nat. Phys. 1, 31 (2005).
  * (24) K.-H. Kim, H. jun Kim, H. C. Koo, J. Chang, and S.-H. Han,  Appl. Phys. Lett. 97, 012504 (2010).
  * (25) L. Szunyogh, G. Zaránd, S. Gallego, M. C. Muñoz, and B. L. Györffy,  Phys. Rev. Lett. 96, 067204 (2006).
  * (26) A. H. Castro Neto and F. Guinea,  Phys. Rev. Lett. 103, 026804 (2009).
  * (27) A. Matos-Abiague,  Phys. Rev. B 81, 165309 (2010).
  * (28) B. K. Nikolić, L. P. Zârbo, and S. Souma,  Phys. Rev. B 73, 075303 (2006).
  * (29) S. Salahuddin, D. Datta, and S. Datta,  (2008), arXiv:0811.3472v1.
  * (30) K. Hattori,  J. Phys. Soc. Jpn 78, 084703 (2009).
  * (31) R. Miranda, F. Yndurán, D. Chandesris, J. Lecante, and Y. Petroff,  Surface Science 117, 319 (1982).
  * (32) Y. Jiang et al.,  Phys. Rev. Lett. 92, 167204 (2004).
  * (33) H. Sukegawa, S. Kasai, T. Furubayashi, S. Mitani, and K. Inomata,  Appl. Phys. Lett. 96, 042508 (2010).
  * (34) P. Brusheim and H. Q. Xu,  Phys. Rev. B 74, 205307 (2006).


# Concept Bottleneck Model with Additional Unsupervised Concepts

Yoshihide Sawada
Tokyo Research Center, Aisin
<EMAIL_ADDRESS>
Keigo Namamura
Tokyo Research Center, Aisin
<EMAIL_ADDRESS>

###### Abstract

With the increasing demands for accountability, interpretability is becoming
an essential capability for real-world AI applications. However, most methods
utilize post-hoc approaches rather than training the interpretable model. In
this article, we propose a novel interpretable model based on the concept
bottleneck model (CBM). CBM uses concept labels to train an intermediate layer
as the additional visible layer. However, because the number of concept labels
restricts the dimension of this layer, it is difficult to obtain high accuracy
with a small number of labels. To address this issue, we integrate supervised
concepts with unsupervised ones trained with self-explaining neural networks
(SENNs). By seamlessly training these two types of concepts while reducing the
amount of computation, we can obtain both supervised and unsupervised concepts
simultaneously, even for large-sized images. We refer to the proposed model as
the concept bottleneck model with additional unsupervised concepts (CBM-AUC).
We experimentally confirmed that the proposed model outperformed CBM and SENN.
We also visualized the saliency map of each concept and confirmed that it was
consistent with the semantic meanings.

##  1 Introduction

The lack of explainability of black-box neural networks is a serious barrier
to the practical application of AI technologies. Especially when
accountability is required, such as in medical and self-driving systems,
black-box models may be expected to cause severe problems. To deploy more AI
into the real world, neural networks are required to have greater
interpretability.

Various methods have been developed to improve interpretability, and these can
be categorized into two approaches, including (1) post-hoc and (2) ante-hoc.
The first approach visualizes the features that are important for the trained
network to determine its output. Although these methods are model-agnostic,
they can be unreliable because they do not always contain the desired
information [1]. In contrast, the second approach embeds human-understandable
domain knowledge into the models. By incorporating this knowledge during
training, models can prevent distortions in interpretation.

One interpretable model is called the concept bottleneck model (CBM) [2]. CBM
has an intermediate layer representing human-understandable concepts as the
additional visible layer (we call it concept layer). CBM outputs through this
layer and trains it by supervised learning. After training, each unit in the
concept layer outputs a single concept necessary for solving the target task.
Thanks to this concept layer, CBM has high interpretability. However, CBM
involves a constraint that the number of concepts limits the dimension of the
concept layer. This constraint makes it difficult for CBM to achieve high
accuracy with relatively small number of concept labels.

To solve this problem, we propose a novel interpretable model inspired by
human judgement and decision-making. It is well known that humans do not
simply make decisions based on the concepts (reasons) that we verbalize in our
explanations. In other words, we use implicit knowledge (know-how) in
combination with explicit knowledge [3]. However, the concept layer in CBM
uses only explicit knowledge (supervised concepts). From this perspective, we
extend this layer to enable the model to leverage explicit and implicit
knowledge to perform task prediction, similar to humans.

We assume that implicit knowledge corresponds to the unsupervised concepts in
the sense that they are not labeled (not verbalized). Namely, in order to
perform the above extension, supervised and unsupervised concepts must be
learned simultaneously.

In this study, we adopt the self-explaining neural network (SENN) to obtain
the unsupervised concepts. SENN is composed of an encoder-decoder architecture
and a parametrizer, which estimates the weights of each concept [4]. This
method has been shown to be effective on the MNIST [5] and CIFAR-10 [6]
datasets. However, it is known that SENN is difficult to apply to the datasets
of large images [2]. Therefore, simply integrating SENN and CBM yields a model
that is difficult to apply to complex problems such as self-driving systems.
To address this issue, we adopt a weight-sharing technique and a
discriminator, as an alternative to the decoder. By using these techniques,
not only SENN but also our integrated model can be trained on datasets with
large images. We call this integrated model the concept bottleneck model with
additional unsupervised concepts (CBM-AUC). Experimental results using large
image datasets (CUB-200-2011 [7] and BDD-OIA [8]) show that our model was
effective. We also visualized the saliency maps of each concept and found that
the concepts can be consistent with their semantic meanings. This result
indicates that the statement by Margeloiu et al. [9] is not necessarily valid.

The main contributions of this study are summarized as follows:

  * •

We propose the concept bottleneck model with additional unsupervised concepts
(CBM-AUC) based on the CBM and SENN.

  * •

We apply a discriminator and weight-sharing to perform training efficiently.

  * •

Experimental results demonstrate the effectiveness of the proposed model.

(A)

(B)

Figure 1: Network overview of (A) a simple integration of CBM and SENN, and
(B) our proposed CBM-AUC. $\mbox{\boldmath$c$}^{ex}$ and
$\mbox{\boldmath$c$}^{im}$ represent explicit (supervised) and implicit
(unsupervised) concepts. $\mbox{\boldmath$g$}(.)$ in (A) represents the
decorder, and $d(.)$ in (B) represents the discriminator.
$\mbox{\boldmath$c$}(.)$ and $\mbox{\boldmath$\theta$}(.)$ represent the
encoder and parametrizer, which estimates the weights of each concept. In CBM-
AUC, we set
$\mbox{\boldmath$c$}(.)=\mbox{\boldmath$e$}^{1}(\mbox{\boldmath$h$}(.))$ and
$\mbox{\boldmath$\theta$}(.)=\mbox{\boldmath$e$}^{2}(\mbox{\boldmath$h$}(.))$.

##  2 Related Work

Research on interpretable concepts can be mainly divided into post-hoc and
ante-hoc methods.

For the post-hoc approach, Goyal et al. [10] used a variational autoencoder to
measure the causal concept effect of a trained model. Zhou et al. [11]
decomposed saliency maps generated by CAM [12] or grad-CAM [13] to each
concept, and Kim et al. [14] and Ghorbani et al. [15] computed concept
sensitivity using directional derivatives. Also, Bau et al. [16] used pixel-
wise segmentation to estimate the concepts. These methods focused on the
concept visualization of state-of-the-art models. However, there are no
guarantees that these models use the desired concepts.

For ante-hoc approaches, Koh et al. [2] proposed a CBM that predicted target
tasks through the concept layer. Bahadori and Heckerman [17] extended this
model to denoise the concepts, and Belém et al. [18] applied a CBM to perform
weakly supervised learning. Alvarez and Jakkola [4] proposed a SENN to
generalize the interpretable linear model and combines the encoder-decoder
architectures as in [19], and Marcinkevics and Vogt [20] extended SENN for
sequential data.

Aside from these models, Losch et al. [21] added a concept bottleneck layer to
the trained segmentation network, and Kim et al. [22] proposed a network
architecture for visual concepts to capture the global context. Chen et al.
[23] proposed concept whitening as a normalization technique, and Li et al.
[19] and Chen et al. [24] introduced an intermediate layer, which was not easy
to train [25] but which was designed to describe prototypes. Similarly, Cao et
al. [26] proposed a meta-learning method that learned a mapping of concepts
into semi-structured metric spaces and combined the outputs of each concept
learner. These concept whitening and prototyping approaches require one
additional patch image per concept.

Multi-task or auxiliary learning methods also use concepts to improve the
target performance [27, 8]. However, in these methods, the concepts may not
affect the targets tasks [2].

##  3 Method

In this section, we first summarize existing two models, CBM and SENN. Then,
we explain CBM-AUC in detail after describing two techniques to enable SENN to
handle large-sized images.

###  3.1 Concept Bottleneck Model (CBM)

CBM predicts target labels $y\in\mathbb{R}$ based on concepts
$\mbox{\boldmath$c$}\in\mathbb{R}^{D_{\rm{ex}}}$, where $D_{\rm{ex}}$ is the
number of supervised concepts [2]. They proposed three training approaches:
independent, sequential, and joint training. The independent approach first
trains $\mbox{\boldmath$x$}\rightarrow\mbox{\boldmath$c$}$, then trains
$\mbox{\boldmath$c$}\rightarrow y$ by using the ground truth concepts. The
sequential approach also trains
$\mbox{\boldmath$x$}\rightarrow\mbox{\boldmath$c$}$ at first, then trains
$\mbox{\boldmath$c$}\rightarrow y$ by using the predicted concepts, not the
ground truth concepts. In contrast, the joint approach trains
$\mbox{\boldmath$x$}\rightarrow\mbox{\boldmath$c$}\rightarrow y$
simultaneously. We focus on joint training due to higher accuracy than other
approaches [2].

Given $\\{\mbox{\boldmath$x$}_{i},\mbox{\boldmath$c$}_{i},y_{i}\\}_{i=1}^{N}$,
CBM is minimized as follows.

$\mathcal{L}_{\rm{cbm}}=\sum_{i}^{N}\Bigl{(}\mathcal{L}_{y}(f(\mbox{\boldmath$x$}_{i}),y_{i})+\lambda\sum_{j}^{D_{\rm{ex}}}\mathcal{L}_{j}(c_{j}^{\rm{ex}}(\mbox{\boldmath$x$}_{i}),c_{i,j})\Bigr{)},$
(1)

where $\mathcal{L}_{y}$ represents the classification loss, $\mathcal{L}_{j}$
represents the $j$-th concept loss, $f(\mbox{\boldmath$x$})$ and
$c_{j}^{\rm{ex}}(\mbox{\boldmath$x$})$ are the predictions of targets and
$j$-th concept, $c_{i,j}$ is the $j$-th ground truth concept of
$\mbox{\boldmath$x$}_{i}$,
$\mbox{\boldmath$c$}_{i}=[c_{i,1},c_{i,2},\cdots,c_{i,D_{\rm{ex}}}]^{\top}$,
and $\lambda$ is a hyperparameter. These losses measure the discrepancy by the
standard loss functions, e.g., cross-entropy and mean squared error.

###  3.2 Self-Explaining Neural Network (SENN)

SENN is an unsupervised concept learning method that uses an interpretable
linear model [4]. The prediction of SENN is described as

$\displaystyle
f(\mbox{\boldmath$x$})=\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})^{\top}\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$}),$
(2)

where $\mbox{\boldmath$\theta$}(.)$ is a parametrizer which estimates weights
of each concept, and $\mbox{\boldmath$c$}^{\rm{im}}(.)$ is the encoder to
output unsupervised concepts.
$\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$}),\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})\in\mathbb{R}^{D_{\rm{im}}}$,
and $D_{\rm{im}}$ represents the number of dimensions of unsupervised
concepts. SENN represents $\mbox{\boldmath$c$}^{\rm{im}}(.)$ and
$\mbox{\boldmath$\theta$}(.)$ by neural networks, respectively. The main
difference between SENN and CBM is that SENN trains networks without using
concept labels (i.e., $\\{\mbox{\boldmath$x$}_{i},y_{i}\\}_{i=1}^{N}$). Given
the training dataset $\\{\mbox{\boldmath$x$}_{i},y_{i}\\}_{i=1}^{N}$, SENN
minimizes the following loss function.

$\mathcal{L}_{\rm{senn}}=\sum_{i}^{N}\mathcal{L}_{y}(f(\mbox{\boldmath$x$}_{i}),y_{i})+\alpha\mathcal{L}_{x}(\mbox{\boldmath$x$}_{i},\mbox{\boldmath$g$}(\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$}_{i})))+\beta\mathcal{L}_{\theta}(f(\mbox{\boldmath$x$}_{i})),$
(3)

where $\mathcal{L}_{y}$ represents the classification loss, and
$\mathcal{L}_{x}$ represents the reconstruction error.
$\mbox{\boldmath$g$}(\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$}_{i}))$
is the reconstructed image using the decoder $\mbox{\boldmath$g$}(.)$, and
$\alpha$ and $\beta$ are the hyperparameters. $\mathcal{L}_{\theta}$ is the
regularization term representing the stability of
$\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})$ by the gradient.

$\mathcal{L}_{\theta}(f(\mbox{\boldmath$x$}))=||\mbox{\boldmath$\nabla$}_{x}f(\mbox{\boldmath$x$})-(\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})^{\top}\mbox{\boldmath$J$}^{c^{\rm{im}}}_{x})^{\top}||\approx
0,$ (4)

where $\mbox{\boldmath$\nabla$}_{x}f(\mbox{\boldmath$x$})$ is the derivative
of $f(\mbox{\boldmath$x$})$ and $\mbox{\boldmath$J$}^{c^{\rm{im}}}_{x}$ is the
Jacobian of the concept $\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$})$
with respect to $x$. This regularization term makes
$\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})$ robust to small changes in
concepts [4].

###  3.3 Adapting SENN to Large Images

In most autoencoder networks, including SENN, the same network architecture is
used for the encoder and decoder. Therefore, the reconstruction error requires
a large number of parameters [28]. In addition, SENN must compute the Jacobian
$\mbox{\boldmath$J$}^{c^{\rm{im}}}_{x}\in\mathbb{R}^{D_{\rm{im}}\times D}$
with respect to each input $\mbox{\boldmath$x$}\in\mathbb{R}^{D}$. Therefore,
as the input size increases, the size of the Jacobian to be computed also
increases.

We adopt two techniques to address this problem. First, we use the
discriminator proposed by Hejelm et al. [29] instead of the decoder (see Sec.
4.2 for details of the network architecture). To train this discriminator
$d(.)$, we adopt a simple least squared error as follows.

$\mathcal{L}_{\rm{dis}}(d(\mbox{\boldmath$z$}),d(\mbox{\boldmath$z$}^{\prime}))=||d(\mbox{\boldmath$z$})-a||^{2}+||d(\mbox{\boldmath$z$}^{\prime})-b||^{2},$
(5)

where
$\mbox{\boldmath$z$}=[\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$})^{\top},\mbox{\boldmath$h$}(\mbox{\boldmath$x$})^{\top}]^{\top}$,
$\mbox{\boldmath$z$}^{\prime}=[\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$})^{\top},\mbox{\boldmath$h$}(\mbox{\boldmath$x$}^{\prime})^{\top}]^{\top}$,
$\mbox{\boldmath$x$}^{\prime}$ is the input differed from $x$, and
$\mbox{\boldmath$h$}(.)$ is the intermediate output before the concept layer.
$z$ and $\mbox{\boldmath$z$}^{\prime}$ represent real and fake [29], and $a$
and $b$ represent the labels of them. In this study, we set $a=1$ and $b=0$.

Second, we share the intermediate network
$\mbox{\boldmath$h$}(\mbox{\boldmath$x$})\in\mathbb{R}^{D_{\rm{h}}}$ with
$\mbox{\boldmath$c$}^{\rm{im}}(\mbox{\boldmath$x$})$ and
$\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})$. By sharing
$\mbox{\boldmath$h$}(\mbox{\boldmath$x$})$, $\mbox{\boldmath$\nabla$}_{x}f$
and $\mbox{\boldmath$J$}^{c^{\rm{im}}}_{x}$ can be transformed by the chain
rule as follows.

$\displaystyle\mbox{\boldmath$\nabla$}_{x}f(\mbox{\boldmath$x$})$
$\displaystyle=$
$\displaystyle(\mbox{\boldmath$\nabla$}_{h}f(\mbox{\boldmath$x$})^{\top}\mbox{\boldmath$J$}^{h}_{x})^{\top},$
(6) $\displaystyle\mbox{\boldmath$J$}^{c^{\rm{im}}}_{x}$ $\displaystyle=$
$\displaystyle\mbox{\boldmath$J$}^{c^{\rm{im}}}_{h}\mbox{\boldmath$J$}^{h}_{x}.$
(7)

By substituting the these equations to Eq.(4), we obtain

$\displaystyle\tilde{\mathcal{L}}_{\theta}(f(\mbox{\boldmath$x$}))=||\mbox{\boldmath$\nabla$}_{h}f(\mbox{\boldmath$x$})-(\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})^{\top}\mbox{\boldmath$J$}^{c^{\rm{im}}}_{h})^{\top}||\approx
0.$ (8)

By using this equation, we need only compute
$\mbox{\boldmath$J$}^{c^{\rm{im}}}_{h}$, the size of $D_{\rm{im}}\times
D_{\rm{h}}$. Therefore, the computational burden is reduced under
$D>D_{\rm{h}}$. Furthermore, we can use sophisticated pretrained models (e.g.,
Inception-v3 [30]) as the intermediate layer $\mbox{\boldmath$h$}(.)$
regardless of the decoder.

From these techniques, the modified loss function of SENN is as follows.

$\tilde{\mathcal{L}}_{\rm{senn}}=\sum_{i}^{N}\mathcal{L}_{y}(f(\mbox{\boldmath$x$}_{i}),y_{i})+\alpha\mathcal{L}_{\rm{dis}}(d(\mbox{\boldmath$z$}_{i}),d(\mbox{\boldmath$z$}_{i}^{\prime}))+\beta\tilde{\mathcal{L}}_{\theta}(f(\mbox{\boldmath$x$}_{i})).$
(9)

We use this loss function because the original SENN could not learn the
dataset used in our experiments. We call this model the modified SENN
(M-SENN).

###  3.4 Concept Bottleneck Model with Additional Unsupervised Concepts (CBM-
AUC)

As described in Sec. 1, because the prediction of the CBM has a close link to
explicit knowledge, the model becomes a highly interpretable DNN model.
However, CBM has a constraint that the number of concepts limits the concept
layer’s dimension. This constraint leads to low classification performance. We
address this problem by adding unsupervised concepts (implicit knowledge),
i.e., combining the CBM and M-SENN as follows.

$\displaystyle\mathcal{L}_{\rm{cbmauc}}=\sum_{i}^{N}\Bigl{(}\mathcal{L}_{y}(f(\mbox{\boldmath$x$}_{i}),y_{i})+\alpha\mathcal{L}_{\rm
dis}(d(\mbox{\boldmath$z$}_{i}),d(\mbox{\boldmath$z$}_{i}^{\prime}))$
$\displaystyle+\beta\tilde{\mathcal{L}}_{\theta}(f(\mbox{\boldmath$x$}_{i}))+\lambda\sum_{j}^{D_{\rm{ex}}}\mathcal{L}_{j}(c_{j}^{\rm{ex}}(\mbox{\boldmath$x$}_{i}),c_{i,j})\Bigr{)}.$
(10)

The first term corresponds to classification loss, and the second and third
terms correspond to the M-SENN’s losses. The last term corresponds to the
CBM’s regularization for the supervised concepts. Note that we compute
$\tilde{\mathcal{L}}_{\theta}$, $\mathcal{L}_{\rm dis}$, and $\mathcal{L}_{y}$
over all concepts
$\mbox{\boldmath$c$}(\mbox{\boldmath$x$})=[(\mbox{\boldmath$c$}^{{\rm{ex}}}(\mbox{\boldmath$x$}))^{\top},(\mbox{\boldmath$c$}^{{\rm{im}}}(\mbox{\boldmath$x$}))^{\top}]^{\top}\in\mathbb{R}^{D_{\rm{ex}}+D_{\rm{im}}}$,
therefore, $f(\mbox{\boldmath$x$})$ becomes as follows.

$\displaystyle
f(\mbox{\boldmath$x$})=\mbox{\boldmath$\theta$}(\mbox{\boldmath$x$})^{\top}\mbox{\boldmath$c$}(\mbox{\boldmath$x$})=\mbox{\boldmath$e$}^{2}(\mbox{\boldmath$h$}(\mbox{\boldmath$x$}))^{\top}\mbox{\boldmath$e$}^{1}(\mbox{\boldmath$h$}(\mbox{\boldmath$x$})),$
(11)

where $\mbox{\boldmath$e$}^{1}$ and $\mbox{\boldmath$e$}^{2}$ are the
remainder of networks. An overview of the proposed network is shown in Fig. 1.
We also show a network integrating the original SENN and CBM for ease of
comparison.

##  4 Experimental Setup

In this study, we evaluated our method on the CUB-200-2011 [7] and BDD-OIA [8]
datasets. For these experiments, we used one GPU, an NVIDIA Tesla v100, with
32GB.

###  4.1 Datasets

CUB-200-2011 (Caltech-UCSD Birds-200-2011) is dataset of 11788 images of birds
(training: 4796; validation: 1198; test: 6794) which have 200 species and 312
concepts. In Koh et al. [2], the authors used the 112 denoising binary bird
concepts (e.g., “Wing color”, “Beak shape”), and we followed their
configuration.

BDD-OIA consists of 22924 videos (training: 16802; validation: 2270; test:
4572), which have four actions (“Forward”, “Stop”, “Left”, “Right”) and 21
concepts (e.g., “Traffic light is green”, “Load is clear”). In this study, we
used only the last frame of each video as in the original paper [8]. Note that
each image has multiple actions, differing from CUB-200-2011.

###  4.2 Network Architecture

For the CUB-200-2011, we used the Inception-v3 network [30] pretrained by the
ImageNet as the intermediate network $h$, as in Koh et al. [2]. Following $h$,
we set $c$ and $\theta$. As $c$, we used a fully connected (FC) layer to
output $D_{\rm{ex}}+D_{\rm{im}}$ concepts. As $\theta$, we used three FC
layers ($2048\times 1024$, $1024\times 512$,
$512\times(D_{\rm{ex}}+D_{\rm{im}})$) because $\theta$ required more complex
network than $c$ [4]. For the first two layers of $\theta$, we used batch
normalization before the activation function Mish [31]. As the discriminator
$d$, we set the three FC layers ($(D_{\rm{ex}}+D_{\rm{im}})\times 512$,
$512\times 512$, and $512\times 1$, the same as global deep infomax
architecture in [29]), and the activation of the first two layers is Mish.

For the BDD-OIA, we used Faster RCNN network [32] as $h$, as in Xu et al. [8].
This network was pretrained by COCO [33], fine-tuned with BDD100k [34], and
was frozen during training. Following Faster RCNN, we computed the input
features for $c$ and $\theta$ by referring to the global module [8]. The
difference is that we flattened the output of the module and executed one FC
layer with ReLU to match the input of $c$ and $\theta$ described above. Note
that the network architectures of $c$, $\theta$, and $d$ are the same as those
used on the CUB-200-2011 experiment.

It should be noted that the selection of $\mbox{\boldmath$h$}(.)$ is arbitrary
as long as $\mbox{\boldmath$h$}(.)$ is a good representation of the input due
to the desiderata of SENN [4]. Assuming that the well-known pre-trained models
are satisfied with this condition, we selected the pre-trained Inception-v3
and faster RCNN for CUB200-2011 and BDD-OIA, respectively, as in [2] and [8].

###  4.3 Hyperparameters

According to Alvarez and Jakkola [4], sparsity is the essential property for
unsupervised concepts. In this study, we used the $k$-WTA [35] activation for
the unsupervised concepts. $k$-WTA retained the $k$ largest values
($k=0,1,2,\cdots,D_{\rm{im}}$) and set all others to be zero. This is a
natural extension of Winner-Take-All, which has been widely studied in the
field of the spiking neural networks [36]. By using this activation, it is
unnecessary to add the sparsity regularization (e.g., L1 norm [4]) such that
the hyperparameter is real-valued. We set $k$ so that 50% of unknown concepts
were zero for CUB-200-2011 and $k=7$ for BDD-OIA, while maintaining a balance
between accuracy and sparsity.

Followings are the other hyperparameters. For the CUB-200-2011, we used the
cross-entropy and MSE losses for $\mathcal{L}_{y}$ and $\mathcal{L}_{j}$,
respectively. We set $D_{\rm{ex}}+D_{\rm{im}}=256$, and the batch size was
$64$. As the implementation of Koh et al. [2, 37], each model was trained
three times over random seeds and computed the mean $\pm$ 2.0 standard
deviation. For the BDD-OIA, we used the BCE loss for $\mathcal{L}_{y}$ and
$\mathcal{L}_{j}$. In addition, we set $D_{\rm{ex}}+D_{\rm{im}}=30$, and the
batch size was $16$. In all experiments, the epoch was set to 50. Note that
these hyperparameters, including comparison methods, are properly determined
beforehand by the experiments.

We further examined the optimal hyperparameters for both datasets as follows.
Optimizer: {SGD, Adam}, learning rate: $\\{0.001,0.01\\}$, $\lambda$:
$\\{0.1,0.5,1.0,2.0,3.0\\}$, $\alpha$: $\\{0.1,0.5,1.0\\}$, and $\beta$:
$\\{0.001,0.01\\}$. For the other hyperparameters, we used the default values
of PyTorch [38].

##  5 Experimental Results

###  5.1 Effectiveness of Network Sharing and Discriminator

Dataset | $D_{h}/D$ | Parameter reduction
---|---|---
CUB-200-2011 | 0.014 | 0.404
BDD-OIA | $7.407\times 10^{-4}$ | 0.377
Table 1: Reduction effects of network sharing and discriminator. Parameter
reduction was obtained by dividing the number of network parameters of simple
integration of CBM and original SENN.

We first show the effectiveness of the network sharing and discriminator.
Table 1 shows the result of integration with the case where the simple
combination was conducted. Here, $D_{h}/D$ and parameter reduction represent
the reduction rate of the size of the Jacobian and the network parameters,
respectively.

As shown in this table, we were able to significantly reduce the size of the
Jacobian ($\mathcal{O}(10^{-2})$ on CUB-200-2011, $\mathcal{O}(10^{-4})$ on
BDD-OIA). In addition, the number of parameters was reduced to about 40%. As
described in Sec. 3, original SENN and simple integration could not run on
CUB-200-2011 and BDD-OIA. In addition, the other variations could not run
either. In contrast, M-SENN and our CBM-AUC were able to function in our
environment. Therefore, we only show the results of the M-SENN and our CBM-AUC
in the following experimental results.

###  5.2 Experimental Results on CUB-200-2011

In this section, we show the experimental results on CUB-200-2011. Following
Koh et al. [2], we evaluated the performance by an average 0-1 task error and
a root mean squared error (RMSE) of supervised concepts. We also used a linear
probe [14, 2] to compute the concept’s RMSE of SENN because there was no one-
to-one correspondence between the unsupervised concept and the unit.

####  5.2.1 Performance Comparison

Model | Task | Concept
---|---|---
CBM | 0.199 $\pm$ 0.006 | 0.031 $\pm$ 0.000
M-SENN | 0.191 $\pm$ 0.003 | 0.053 $\pm$ 0.004
CBM-AUC | 0.177 $\pm$ 0.009 | 0.028 $\pm$ 0.001
Table 2: Comparison results for CUB-200-2011.

We compared CBM-AUC with CBM [2] and M-SENN. For a fair comparison, we used
256 unsupervised concepts of M-SENN so that the total number matched CBM-AUC.
In contrast, CBM used only 112 supervised concepts because it cannot be
increased beyond $D_{\rm{ex}}$. Note that this is clearly a situation in line
with our motivation (see Sec. 1). See Fig. 2 for the result when the total
number of concepts is equal to the CBM.

As shown in table 2, CBM-AUC outperformed CBM for task performance because CBM
can only use supervised concepts. In addition, CBM-AUC also outperformed the
concept accuracy of CBM and the task and concept accuracies of M-SENN. We
consider that the higher accuracy of the concept than CBM was due to the
disentanglement of explicit and implicit knowledge by the increased
dimensionality of the concept layer. Therefore, using supervised and
unsupervised concepts is necessary to obtain high performance.

####  5.2.2 Performance when changing the number of concepts

Model | $D_{\rm{ex}}+D_{\rm{im}}$ | Task | Concept
---|---|---|---
CBM-AUC | 128 | 0.183 $\pm$ 0.003 | 0.028 $\pm$ 0.001
CBM-AUC | 256 | 0.177 $\pm$ 0.009 | 0.028 $\pm$ 0.001

Table 3: CBM-AUC’s performance comparison between
$D_{\rm{ex}}+D_{\rm{im}}=256$ and $D_{\rm{ex}}+D_{\rm{im}}=128$.

(A)

(B)

Figure 2: Performance when limited supervised concepts. (A): change of the
task performance, (B): change of the concept performance.

This section investigates the performance when the number of concepts
$D_{\rm{ex}}$ and $D_{\rm{im}}$ changed.

Table 3 shows the CBM-AUC’s performances of $D_{\rm{ex}}+D_{\rm{im}}=256$ and
$D_{\rm{ex}}+D_{\rm{im}}=128$. Note that the number of supervised concepts
$D_{\rm{ex}}$ was fixed ($D_{\rm{ex}}=112$). As shown in this table, the task
performance decreased when $D_{\rm{im}}$ was small. In addition, CBM-AUC
performed better than CBM (Table 2) when $D_{\rm{im}}=16$
($D_{\rm{ex}}+D_{\rm{im}}=128$). This result indicates that the unsupervised
concepts are important even when $D_{\rm{im}}$ is small.

We also investigated the performance when the number of supervised concepts
was limited. We randomly removed the supervised concepts from the
$D_{\rm{ex}}=112$ and added the same number of unsupervised concepts, keeping
the number of total concepts $D_{\rm{ex}}+D_{\rm{im}}=112$ fixed.

When the number of supervised concepts was reduced, RMSE could not accurately
evaluate the concept accuracy. Therefore, we evaluated the concept
correlation, defined as follows.

$\displaystyle\bar{r}^{2}=\frac{1}{D_{\rm{ex}}}\sum_{j}\max_{i}r^{2}(c_{j},c_{i}(\mbox{\boldmath$x$})),$
(12)

where $\bar{r}^{2}$ is the average coefficient, $r$ is the coefficient between
two concepts, $c_{j}$ is the ground truth of supervised concept, and
$c_{i}(\mbox{\boldmath$x$})(i=1,2,\cdots,D_{\rm{ex}}+D_{\rm{im}})$ is the
$i$-th predicted concept. Namely, $\bar{r}^{2}$ represents how the predicted
concepts are close to the ground truth concepts.

Figure 2 shows the task error and correlation of CBM and CBM-AUC. The red and
green lines represent CBM-AUC and CBM, respectively. Note that $D_{\rm{ex}}=0$
and $D_{\rm{ex}}=112$ in CBM-AUC are equivalent to M-SENN and CBM. Comparing
these results, the task performance of CBM decreased significantly when the
concept correlation decreased, whereas the task performance of CBM-AUC changed
only slightly. Meanwhile, as shown in Fig. 2 (B), CBM-AUC’s correlation
decreased similarly to CBM’s one. If the additional unsupervised concepts
acquire concepts similar to the supervised ones, the correlation should also
be constant in CBM-AUC. This decreasing tendency implies that there was low
correlation between supervised and unsupervised concepts. In future works, we
intend to obtain the explicit concepts in an unsupervised manner.

###  5.3 Experimental Results on BDD-OIA

In this section, we show the experimental results on BDD-OIA. Following Xu et
al. [8], we evaluated the performance by two F1 scores, $F1_{all}$ and $mF1$.
$F1_{all}$ averages the F1 scores over all the predictions, and $mF1$ computes
the mean of the F1 score for each action. We also computed the F1 scores of
concept in the same manner.

####  5.3.1 Performance Comparison

Model | F | S | R | L | $mF1$ | $F1_{all}$ | $mF1_{cpt}$ | $F1_{cpt,all}$
---|---|---|---|---|---|---|---|---
CBM | 0.795 | 0.732 | 0.431 | 0.483 | 0.610 | 0.661 | 0.292 | 0.412
M-SENN | 0.705 | 0.727 | 0.339 | 0.385 | 0.539 | 0.612 | 0.098 | 0.216
CBM-AUC | 0.803 | 0.751 | 0.525 | 0.551 | 0.658 | 0.704 | 0.342 | 0.522

Table 4: Performance of the action and reason prediction for BDD-OIA. F/S/R/L
represent the F1 scores of actions of “Forward”, “Stop”, “Left”, and “Right”,
respectively. $mF1$ and $F1_{all}$ represent the F1 scores of actions, and
$mF1_{cpt}$ and $F1_{cpt,all}$ represent the F1 scores of supervised concepts.

(A)

(B)

Figure 3: The examples of the output of CBM-AUC. In these situations, our
model can predict actions and concepts correctly.

(A)

(B)

(C)

(D)

(E)

Figure 4: Saliency maps by grad-CAM: (A) are input images, (B) and (C) are the
saliency maps for supervised concepts, and (D) and (E) are the unsupervised
concept ones. Concepts of (B) and (C) are “Obstacle: car”, and “Traffic light
is red”. Note that the green signal turns on in the left-hand side image.

Table 4 shows the experimental results of CBM, M-SENN, and CBM-AUC. As shown
in this table, the performances of CBM-AUC are better than CBM and M-SENN.
Figs. 3 and 4 show an example of predictions and saliency maps of some units
in the concept layer. As shown in Fig. 3, our model can predict actions and
concepts correctly. In addition, from Fig. 4, our model can pay attention to
areas corresponding to each supervised concept. Furthermore, the unsupervised
concepts provide valuable concepts for the action classification. These
results indicate the effectiveness of our combination of supervised and
unsupervised concepts, as in Sec. 5.2.1.

####  5.3.2 Performance for the Different Intermediate Network

$h$ | $mF1$ | $F1_{all}$ | $mF1_{cpt}$ | $F1_{cpt,all}$
---|---|---|---|---
Inception-v3 | 0.485 | 0.592 | 0.16 | 0.31
Faster RCNN | 0.658 | 0.704 | 0.342 | 0.522

Table 5: Performance comparison with respect to changing backbone network.

(A)

(B)

(C)

Figure 5: Comparison of saliency maps when changing $\mbox{\boldmath$h$}(.)$.
Left: Faster RCNN, right: Inception-v3. Each ground truth concept is (A):
“Traffic light is green”, (B): “Obstacle: car” and (C): “Obstacle: person".
The same image as the left-hand side of Fig. 4 (A) is used.

To investigate the influence of $\mbox{\boldmath$h$}(.)$, we replaced
$\mbox{\boldmath$h$}(.)$ from Faster RCNN to Inception-v3. Table 5 shows the
performance comparison. Note that we present results for only four important
metrics (i.e., $mF1$, $F1_{all}$, $mF1_{cpt}$, and $F1_{cpt,all}$). As shown
in this table, the performance when using Inception-v3 was worse than when
using the Faster RCNN. Figure 5 shows the saliency maps for some supervised
concepts. As shown in this figure, saliency maps using Inception-v3 were not
consistent with the semantic meanings. From these results, we consider that
the Inception-based model is not suitable for BDD-OIA.

####  5.3.3 Performance Comparison with Multi-Task Models

Model | $mF1$ | $F1_{all}$ | $mF1_{cpt}$ | $F1_{cpt,all}$
---|---|---|---|---
Global [8] | 0.392 | 0.601 | 0.180 | 0.331
Local [39] | 0.699 | 0.711 | 0.196 | 0.406
Global+Local [8] | 0.718 | 0.734 | 0.208 | 0.422
CBM-AUC | 0.658 | 0.704 | 0.342 | 0.522

Table 6: Performance comparison for different models.

We also compared the different model architecture. Table 6 shows the
comparison with the multi-task models performed in [8] and [39]. “Global” is
the accuracy when using the simplest global module we also use, “Local” uses
the local features produced by the RPN and ROI head layers of Fater RCNN, and
“Global+Local” combines global and local features. Although we cannot compare
these results directly since the probability is different111The models in [39,
8] compute
$\mbox{\boldmath$x$}\rightarrow(\mbox{\boldmath$c$},\mbox{\boldmath$y$})$
while our model computes
$\mbox{\boldmath$x$}\rightarrow\mbox{\boldmath$c$}\rightarrow y$, where
$\mbox{\boldmath$x$},\mbox{\boldmath$c$}$ and $y$ correspond to input,
concepts and task, respectively., concept accuracy of our model was greater
than the multi-task models. However, the multi-task models, especially using
local features, exhibited better action accuracies than our model. There is a
potential to improve the task accuracy of our model by modifying
$\mbox{\boldmath$h$}(.)$, such as utilizing local information; however, this
topic remains as our future works.

##  6 Discussion

(A)

(B)

Figure 6: Examples of ambiguous labels. (A): Ground truth concept is
“Obstacle:car”, and the ground truth actions are “Stop”, “Left”, and “Right”.
(B): Ground truth concepts are “Obstacle:car” and “Solid line on the left”,
and the ground truth actions are “Stop” and “Right”.

It is essential to “visualize” where the model is looking at for a reliable
prediction. A recent paper [9] showed that the CBM does not look at the region
where the concept indicates. However, we found that our model for BDD-OIA
seems to look at reasonable areas, as shown in Fig. 4. We also suggested that
these different results come from the intermediate network
$\mbox{\boldmath$h$}(.)$ shown in Fig. 5.

It is also interesting to observe the saliency maps for unsupervised concepts.
As shown in Figs. 4 (D) and (E), the unsupervised concept is an entangled set
of multiple concepts that humans can understand. For example, the right-hand
side of Fig. 4 (D) pays attention to the traffic sign, signals, and front
cars. Ideally, these concepts should be able to be extracted separately. How
to disentangle human-understandable concepts through unsupervised learning is
one of the open problems.

Addressing ambiguous labels is also an important future step. Figure 6 shows
an example. Figures 6 (A) and 3 (B) have the same ground truth concept
“Obstacle:car”, whereas, the ground truth action is “Stop” for Fig. 3 (B) and
are “Stop”, “Left”, and “Right” for Fig. 6 (B). In both images, “Stop” seems
reasonable. However, “Left” and “Right” are difficult to determine from the
image in Fig. 6 (A) alone. In fact, our model outputs “Stop” and
“Obstacle:car”. In contrast, the ground truth concepts and actions of Fig. 6
(B) are “Obstacle:car” and “Solid line on the left”, and “Stop” and “Right”.
Our model can output these actions and concepts correctly. However, it may
also output a concept that does not seem to be wrong but does not contain the
ground truth concepts (“No lane on the left”). Using video as an input may
solve this problem. However, many challenges still remain to output
appropriate concepts in time series (e.g., the drastic increase in annotation
cost).

##  7 Conclusion

In this study, we have proposed Concept Bottleneck Model with Additional
Unsupervised Concepts motivated by actual human concepts. To realize this, we
have combined CBM and modified SENN to represent explicit and implicit
knowledge. We have examined the effectiveness of our model with two datasets,
CUB-200-2011 and BDD-OIA, and we found that our model outperformed CBM and
modified SENN for all datasets. We also showed that the saliency maps of each
concept were consistent with the semantic meaning for BDD-OIA.

In the future, we plan to adapt the proposed model to video input, improve our
model by changing the intermediate layer, and output human-understandable
concepts by unsupervised learning. Although opportunities for further research
remain, we believe that this approach paves the way from System1 to System 2
[40].

## References

  * [1] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian J. Goodfellow, Moritz Hardt, and Been Kim.  Sanity checks for saliency maps.  CoRR, abs/1810.03292, 2018.
  * [2] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang.  Concept bottleneck models.  arXiv preprint arXiv:2007.04612, 2020.
  * [3] Michael Polanyi.  Personal Knowledge: Towards a Post-Critical Philosophy.  University of Chicago Press, 1958.
  * [4] David Alvarez Melis and Tommi Jaakkola.  Towards robust interpretability with self-explaining neural networks.  Advances in Neural Information Processing Systems, 31:7775–7784, 2018.
  * [5] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  In Proceedings of the IEEE, number 11 in 86, pages 2278–2324, 1998.
  * [6] Alex Krizhevsky and Geoffrey Hinton.  Learning multiple layers of features from tiny images. master’s thesis, university of tronto, 2009.
  * [7] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie.  The caltech-ucsd birds-200-2011 dataset.  Technical Report CNS-TR-2011-001, California Institute of Technology, 2011\.
  * [8] Yiran Xu, Xiaoyin Yang, Lihang Gong, Hsuan-Chu Lin, Tz-Ying Wu, Yunsheng Li, and Nuno Vasconcelos.  Explainable object-induced action decision for autonomous vehicles.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9523–9532, 2020.
  * [9] Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller.  Do Concept Bottleneck Models Learn as Intended?  arXiv e-prints, page arXiv:2105.04289, May 2021.
  * [10] Yash Goyal, Amir Feder, Uri Shalit, and Been Kim.  Explaining classifiers with causal concept effect (cace).  arXiv preprint arXiv:1907.07165, 2019.
  * [11] Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba.  Interpretable basis decomposition for visual explanation.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 119–134, 2018.
  * [12] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.  Learning deep features for discriminative localization.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016.
  * [13] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In Proceedings of the IEEE international conference on computer vision, pages 618–626, 2017.
  * [14] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al.  Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav).  In International conference on machine learning, pages 2668–2677. PMLR, 2018.
  * [15] Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim.  Towards automatic concept-based explanations.  In Advances in Neural Information Processing Systems, pages 9277–9286, 2019.
  * [16] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Network dissection: Quantifying interpretability of deep visual representations.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6541–6549, 2017.
  * [17] Mohammad Taha Bahadori and David E Heckerman.  Debiasing concept bottleneck models with instrumental variables.  arXiv preprint arXiv:2007.11500, 2020.
  * [18] Catarina Belém, Vladimir Balayan, Pedro Saleiro, and Pedro Bizarro.  Weakly supervised multi-task learning for concept-based explainability.  CoRR, abs/2104.12459, 2021.
  * [19] Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin.  Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions.  arXiv preprint arXiv:1710.04806, 2017.
  * [20] Ricards Marcinkevics and Julia E. Vogt.  Interpretable models for granger causality using self-explaining neural networks.  CoRR, abs/2101.07600, 2021.
  * [21] Max Losch, Mario Fritz, and Bernt Schiele.  Interpretability beyond classification output: Semantic bottleneck networks.  arXiv preprint arXiv:1907.10882, 2019.
  * [22] Taesup Kim, Sungwoong Kim, and Yoshua Bengio.  Visual concept reasoning networks.  arXiv preprint arXiv:2008.11783, 2020.
  * [23] Zhi Chen, Yijie Bei, and Cynthia Rudin.  Concept whitening for interpretable image recognition.  Nature Machine Intelligence, 2(12):772–782, 2020.
  * [24] Chaofan Chen, Oscar Li, Alina Barnett, Jonathan Su, and Cynthia Rudin.  This looks like that: deep learning for interpretable image recognition.  CoRR, abs/1806.10574, 2018.
  * [25] Cynthia Rudin.  Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead.  arXiv e-prints, page arXiv:1811.10154, November 2018.
  * [26] Kaidi Cao, Maria Brbic, and Jure Leskovec.  Concept learners for generalizable few-shot learning.  CoRR, abs/2007.07375, 2020.
  * [27] Shaoli Huang, Zhe Xu, Dacheng Tao, and Ya Zhang.  Part-stacked cnn for fine-grained visual categorization.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1173–1182, 2016.
  * [28] Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi.  Dream to control: Learning behaviors by latent imagination.  In International Conference on Learning Representations, 2019.
  * [29] R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio.  Learning deep representations by mutual information estimation and maximization.  In International Conference on Learning Representations, 2018.
  * [30] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016.
  * [31] Diganta Misra.  Mish: A self regularized non-monotonic neural activation function.  arXiv preprint arXiv:1908.08681, 2019.
  * [32] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun.  Faster R-CNN: towards real-time object detection with region proposal networks.  CoRR, abs/1506.01497, 2015.
  * [33] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick.  Microsoft COCO: common objects in context.  CoRR, abs/1405.0312, 2014.
  * [34] Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell.  Bdd100k: A diverse driving dataset for heterogeneous multitask learning.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2636–2645, 2020.
  * [35] Chang Xiao, Peilin Zhong, and Changxi Zheng.  Resisting adversarial attacks by k-winners-take-all.  CoRR, abs/1905.10510, 2019.
  * [36] Peter U Diehl and Matthew Cook.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity.  Frontiers in computational neuroscience, 9:99, 2015.
  * [37] https://github.com/yewsiang/ConceptBottleneck.
  * [38] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  Advances in neural information processing systems, 32:8026–8037, 2019.
  * [39] Dequan Wang, Coline Devin, Qi-Zhi Cai, Fisher Yu, and Trevor Darrell.  Deep object-centric policies for autonomous driving.  In 2019 International Conference on Robotics and Automation (ICRA), pages 8853–8859. IEEE, 2019.
  * [40] Daniel Kahneman.  Thinking, fast and slow.  Macmillan, 2011.