# Quantum description of angles in the plane

Roberto Beneducia, Emmanuel Frionb and Jean-Pierre Gazeauc A Dipartimento di
Fisica, Università della Calabria
and Istituto Nationale di Fisica Nucleare, Gruppo c. Cosenza, 87036 Arcavacata
di Rende (Cs), Italy B Helsinki Institute of Physics, P. O. Box 64,
FIN-00014 University of Helsinki, Finland C Université de Paris,
CNRS, Astroparticule et Cosmologie 75013 Paris, France
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

The real plane with its set of orientations or angles in $[0,\pi)$ is the
simplest non trivial example of a (projective) Hilbert space and provides nice
illustrations of quantum formalism. We present some of them, namely covariant
integral quantization, linear polarisation of light as a quantum measurement,
interpretation of entanglement leading to the violation of Bell inequalities,
and spin one-half coherent states viewed as two entangled angles.

###### Contents

  1. 1 Introduction
  2. 2 Background
    1. 2.1 Definition of POVMs
    2. 2.2 Integral quantization
  3. 3 Euclidean plane as Hilbert space of quantum states
    1. 3.1 Mixed states As Density Matrices
    2. 3.2 Describing Non-commutativity and Finding Naimark Extensions through Rotations
    3. 3.3 Linear Polarization of Light as a Quantum Phenomenon
  4. 4 Entanglement and isomorphisms
    1. 4.1 Bell States and Quantum Correlations
    2. 4.2 Bell Inequality and Its Violation
    3. 4.3 Entanglement of two angles
  5. 5 Conclusions

##  1\. Introduction

The formulation of quantum mechanics in a real Hilbert space has been analyzed
by Stueckelberg in 1960 [1] in order to show that the need for a complex
Hilbert space is connected to the uncertainty principle. Later, Sol r [2]
showed that the lattice of elementary propositions is isomorphic to the
lattice of closed subspaces of a separable Hilbert space (over the reals, the
complex numbers or the quaternions). In other words, the lattice structure of
propositions in quantum physics does not suggest the Hilbert space to be
complex. More recently, Moretti and Oppio [3] gave stronger motivation for the
Hilbert space to be complex which rests on the symmetries of elementary
relativistic systems.

In this contribution, we do not address the question of the physical validity
of the real Hilbert space formulation of quantum mechanics but limit ourselves
to use the real 2-dimensional case, i.e. the Euclidean plane, as a toy model
for illustrating some aspects of the quantum formalism, as quantization,
entanglement and quantum measurement. The latter is nicely represented by the
linear polarization of light. This real $2$-dimensional case relies on the
manipulation of the two real Pauli matrices

(1.1) $\displaystyle\sigma_{1}=\begin{pmatrix}0&1\\\
1&0\end{pmatrix}\,,\quad\sigma_{3}=\begin{pmatrix}1&0\\\ 0&-1\end{pmatrix}\;,$

and their tensor products, with no mention of the third, complex matrix
$\sigma_{2}=\begin{pmatrix}0&-\mathsf{i}\\\ \mathsf{i}&0\end{pmatrix}$. As a
matter of fact, many examples aimed to illustrate tools and concepts of
quantum information, quantum measurement, quantum foundations, … (e.g., Peres
[4]) are illustrated with manipulations of these matrices.

In [5], it was shown that the set of pure states in the plane is represented
by half of the unit circle and the set of mixed states by half the unit disk,
and also that rotations in the plane rule time evolution through Majorana-like
equations, all of this using only real quantities for both closed and open
systems.

This paper is a direct extension of our previous paper [6], and for this
reason we start the discussion by recalling some key elements of the
mathematical formalism.

##  2\. Background

###  2.1. Definition of POVMs

We start with the definition of a normalized Positive-Operator Valued measure
(POVM) [7]. It is defined as a map
$F:\mathcal{B}(\Omega)\to\mathcal{L}^{+}_{s}(\mathcal{H})$ from the Borel
$\sigma$-algebra of a topological space $\Omega$ to the space of linear
positive self-adjoint operators on a Hilbert space $\mathcal{H}$ such that

(2.1) $\displaystyle
F\left(\bigcup_{n=1}^{\infty}\Delta_{n}\right)=\sum_{n=1}^{\infty}F(\Delta_{n})\,\quad
F(\Omega)=\mathbbm{1}\;.$

In this definition, $\\{\Delta_{n}\\}$ is a countable family of disjoint sets
in $\mathcal{B}(\Omega)$ and the series converges in the weak operator
topology. If $\Omega=\mathbb{R}$, we have a real POVM. If $F(\Delta)$ is a
projection operator for every $\Delta\in\mathcal{B}(\Omega)$, we recover the
usual projection-valued measure (PVM).

A quantum state is defined as a non-negative, bounded self-adjoint operator
with trace $1$. The space of states is a convex space and is denoted by
$S(\mathcal{H})$. A quantum measurement corresponds to an affine map
$S(\mathcal{H})\mapsto\mathcal{M}_{+}(\Omega)$ from quantum states to
probability measures, $\rho\mapsto\mu_{\rho}$. There is [8] a one-to-one
correspondence between POVMs
$F:\mathcal{B}(\Omega)\to\mathcal{L}_{s}^{+}(\mathcal{H})$ and affine maps
$S(\mathcal{H})\mapsto\mathcal{M}_{+}(\Omega)$ given by
$\mu_{\rho}(\Delta)=\mathrm{Tr}(\rho F(\Delta))$,
$\Delta\in\mathcal{B}(\Omega)$.

###  2.2. Integral quantization

Quantum mechanics is usually taught in terms of projection operators and PVM,
but measurements usually give a statistical distribution around a mean value,
incompatible with the theory. We recall here a generalization of a
quantization procedure, the integral quantization, based on POVMs instead of
PVM. The basic requirements of this programme are the following: the
quantization of a classical function defined on a set $X$ must respect

  1. (1)

Linearity. Quantization is a linear map $f\mapsto A_{f}$:

(2.2)
$\displaystyle\mathfrak{Q}:\mathcal{C}(X)\mapsto\mathcal{A}({\mathcal{H}})\,,\qquad\mathfrak{Q}(f)=A_{f}\,,$

where

     * •

$\mathcal{C}(X)$ is a vector space of complex or real-valued functions $f(x)$
on a set $X$, i.e. a “classical” mathematical model,

     * •

$\mathcal{A}({\mathcal{H}})$ is a vector space of linear operators in some
real or complex Hilbert space ${\mathcal{H}}$, i.e., a “quantum” mathematical
model, notwithstanding the question of common domains in the case of unbounded
operators.

  2. (2)

Unity. The map (2.2) is such that the function $f=1$ is mapped to the identity
operator $\mathbbm{1}$ on ${\mathcal{H}}$.

  3. (3)

Reality. A real function $f$ is mapped to a self-adjoint operator $A_{f}$ in
${\mathcal{H}}$ or, at least, a symmetric operator (in the infinite-
dimensional case).

  4. (4)

Covariance. Defining the action of a symmetry group G on X by
$\left(g,x\right)\in G\times X$ such as $\left(g,x\right)\mapsto g\cdot x\in
X$, there is a unitary representation $U$ of $G$ such that
$A_{T(g)f}=U(g)A_{f}U(g^{-1})$, with $(T(g)f)(x)=f\left(g^{-1}\cdot x\right)$.

Performing the integral quantization [9] of a function $f(x)$ on a measure
space $(X,\nu)$ boils down to the linear map:

(2.3) $\displaystyle f\mapsto A_{f}=\int_{X}\,{\sf
M}(x)\,f(x)\,\mathrm{d}\nu(x)\,,$

where we introduce a family of operators ${\sf M}(x)$ solving the identity.
More precisely, we have

(2.4) $\displaystyle X\ni x\mapsto\mathsf{M}(x)\,,\quad\int_{X}\,{\sf
M}(x)\,\mathrm{d}\nu(x)=\mathbbm{1}\,.$

If the ${\sf M}(x)$ are non-negative, they provide a POVM. Indeed, the
quantization of the characteristic function on the Borel set $\Delta$,
$A(\chi_{\Delta})$,

(2.5) $\displaystyle F(\Delta):=A(\chi_{\Delta})=\int_{\Delta}\,{\sf
M}(x)\,\mathrm{d}\nu(x)\;.$

is a POVM which provides a quantization procedure

$f\mapsto A_{f}=\int_{X}\,f(x)\,dF(x).$

##  3\. Euclidean plane as Hilbert space of quantum states

###  3.1. Mixed states As Density Matrices

Density matrices act as a family of operators which can be used to perform
covariant integral quantization. In the context of the Euclidean plane and its
rotational symmetry, one associates the polar angle $\phi\in[0,2\pi)$ with the
unit vector $\widehat{\mathbf{u}}_{\phi}$ to define the pure state
$|\phi\rangle:=\left|\widehat{\mathbf{u}}_{\phi}\right\rangle$.

As shown in Figure 1, two orthogonal pure states
$\widehat{\boldsymbol{\imath}}=|0\rangle$ and
$\widehat{\boldsymbol{\jmath}}=\left|\dfrac{\pi}{2}\right\rangle$ are readily
identified with the unit vectors spanning the plane. In this configuration,
the pure state $|\phi\rangle$ is defined by an anticlockwise rotation of angle
$\phi$ of the pure state $|0\rangle$. Denoting the orthogonal projectors on
$\widehat{\boldsymbol{\imath}}$ and $\widehat{\boldsymbol{\jmath}}$ by
$|0\rangle\langle 0|$ and
$\left|\frac{\pi}{2}\right\rangle\left\langle\frac{\pi}{2}\right|$
respectively, we visualize the resolution of the identity as follows

(3.1) $\begin{split}\mathbbm{1}=|0\rangle\langle
0|&+\left|\frac{\pi}{2}\right\rangle\left\langle\frac{\pi}{2}\right|\\\
&\Updownarrow\\\ \left(\begin{array}[]{cc}1&0\\\
0&1\end{array}\right)&=\left(\begin{array}[]{cc}1&0\\\
0&0\end{array}\right)+\left(\begin{array}[]{cc}0&0\\\
0&1\end{array}\right)\;.\end{split}$

$\widehat{\boldsymbol{\imath}}=|0\rangle\equiv\begin{pmatrix}1\\\
0\end{pmatrix}$$|\phi\rangle=\begin{pmatrix}\cos\phi\\\
\sin\phi\end{pmatrix}\leftrightarrow
E_{\phi}=|\phi\rangle\langle\phi|$$O$$\langle
0|0\rangle=1=\left\langle\frac{\pi}{2}\right|\left.\frac{\pi}{2}\right\rangle\,,\quad\langle
0\left|\frac{\pi}{2}\right\rangle=0$$\widehat{\boldsymbol{\jmath}}=|\frac{\pi}{2}\rangle\equiv\begin{pmatrix}0\\\
1\end{pmatrix}$$1$$\phi$ Figure 1. The Euclidean plane and its unit vectors
viewed as pure quantum states in Dirac ket notations.

Recalling that a pure state in the plane, equivalently an orientation, can be
decomposed as
$|\phi\rangle=\cos{\phi}\,|0\rangle+\sin{\phi}\,\left|\frac{\pi}{2}\right\rangle$,
with $\langle 0|\phi\rangle=\cos\phi$ and
$\left\langle\frac{\pi}{2}\right|\phi\Big{\rangle}=\sin\phi$, it is
straightforward to find the orthogonal projector corresponding to the pure
state $|\phi\rangle$,

(3.2) $\displaystyle E_{\phi}=\begin{pmatrix}\cos^{2}\phi&\cos\phi\sin\phi\\\
\cos\phi\sin\phi&\sin^{2}\phi\end{pmatrix}\;,$

from which we can construct the density matrix corresponding to all the mixed
states

(3.3)
$\displaystyle\rho=\left(\frac{1+r}{2}\right)E_{\phi}+\left(\frac{1-r}{2}\right)E_{\phi+\pi/2}\,,\quad
0\leq r\leq 1\;.$

In this expression, the parameter $r$ represents the degree of mixing. Hence
the upper half-disk $(r,\phi)$, $0\leq r\leq 1$, $0\leq\phi<\pi$ is in one-to-
one correspondence with the set of density matrices $\rho\equiv\rho_{r,\phi}$
written as

(3.4)
$\begin{split}\rho_{r,\phi}&=\frac{1}{2}\mathbbm{1}+\frac{r}{2}\mathcal{R}(\phi)\sigma_{3}\mathcal{R}(-\phi)\\\
&=\begin{pmatrix}\frac{1}{2}+\frac{r}{2}\cos 2\phi&\frac{r}{2}\sin 2\phi\\\
\frac{r}{2}\sin 2\phi&\frac{1}{2}-\frac{r}{2}\cos
2\phi\end{pmatrix}=\frac{1}{2}\left(\mathbbm{1}+r\sigma_{2\phi}\right)\,,\end{split}$

where $\mathcal{R}(\phi)=\begin{pmatrix}\cos\phi&-\sin\phi\\\
\sin\phi&\cos\phi\end{pmatrix}$ is a rotation matrix in the plane, and

$\displaystyle\sigma_{\phi}$
$\displaystyle:=\cos\phi\,\sigma_{3}+\sin\phi\,\sigma_{1}$ (3.5)
$\displaystyle\equiv\overrightarrow{\boldsymbol{\sigma}}\cdot\widehat{\mathbf{u}}_{\phi}=\begin{pmatrix}\cos\phi&\sin\phi\\\
\sin\phi&-\cos\phi\end{pmatrix}=\mathcal{R}(\phi)\,\sigma_{3}\,.$

The observable $\sigma_{\phi}$ has eigenvalues $\\{\pm 1\\}$ and eigenvectors
$\left|\frac{\phi}{2}\right\rangle$ and
$\left|\frac{\phi+\pi}{2}\right\rangle$ respectively. It plays a crucial rôle
since, as we show right after, it is at the core of both the non-commutative
character and the entanglement of two quantum states of the real space. It is
a typical observable used to illustrate quantum formalism [4].

###  3.2. Describing Non-commutativity and Finding Naimark Extensions through
Rotations

Let us apply integral quantization with the real density matrices (3.4). With
$X=\mathbb{S}^{1}$, the unit circle, equipped with the measure
$\mathrm{d}\nu(x)=\frac{\mathrm{d}\phi}{\pi}$, $\phi\in[0,2\pi)$, we obtain
the resolution of the identity for an arbitrary $\phi_{0}$,

(3.6)
$\displaystyle\int_{0}^{2\pi}\rho_{r,\phi+\phi_{0}}\,\frac{\mathrm{d}\phi}{\pi}=\mathbbm{1}\,.$

Hence, quantizing a function (or distribution) $f(\phi)$ on the circle is done
through the map

$\displaystyle f\mapsto
A_{f}=\int_{0}^{2\pi}f(\phi)\rho_{r,\phi+\phi_{0}}\,\frac{\mathrm{d}\phi}{\pi}$
$\displaystyle=\begin{pmatrix}\langle
f\rangle+\frac{r}{2}C_{c}\left(R_{\phi_{0}}f\right)&\frac{r}{2}C_{s}\left(R_{\phi_{0}}f\right)\\\
\frac{r}{2}C_{s}\left(R_{\phi_{0}}f\right)&\langle
f\rangle-\frac{r}{2}C_{c}\left(R_{\phi_{0}}f\right)\end{pmatrix}$ (3.7)
$\displaystyle=\langle
f\rangle\,\mathbbm{1}+\frac{r}{2}\left[C_{c}\left(R_{\phi_{0}}f\right)\,\sigma_{3}+C_{s}\left(R_{\phi_{0}}f\right)\,\sigma_{1}\right]\,,$

with $\langle f\rangle:=\frac{1}{2\pi}\int_{0}^{2\pi}f(\phi)\,\mathrm{d}\phi$
the average of $f$ on the unit circle and
$R_{\phi_{0}}(f)(\phi):=f(\phi-\phi_{0})$. Here we have defined cosine and
sine doubled angle Fourier coefficients of $f$

(3.10) $\displaystyle C_{\stackrel{{\scriptstyle
c}}{{s}}}(f)=\int_{0}^{2\pi}f(\phi)\left\\{\begin{array}[]{c}\cos\\\
\sin\end{array}\right.2\phi\,\frac{\mathrm{d}\phi}{\pi}\,.$

In [6], we drew three consequences from this result. The first consequence is
that, upon identification of $\mathbb{R}^{3}$ with the subspace
$V_{3}=\mathrm{Span}\Big{\\{}e_{0}(\phi):=\frac{1}{\sqrt{2}},e_{1}(\phi):=\cos
2\phi,e_{2}(\phi):=\sin 2\phi\Big{\\}}$ in
$L^{2}(\mathbb{S}^{1},\mathrm{d}\phi/\pi)$, the integral quantization map with
$\rho_{r,\phi+\phi_{0}}$ yields a non-commutative version of $\mathbb{R}^{3}$
:

$\displaystyle A_{e_{0}}$ $\displaystyle=\frac{\mathbbm{1}}{\sqrt{2}}\,,$
$\displaystyle A_{e_{1}}$ $\displaystyle=\frac{r}{2}[\cos
2\phi_{0}\,\sigma_{3}+\sin
2\phi_{0}\,\sigma_{1}]\equiv\frac{r}{2}\sigma_{2\phi_{0}}\,,$ $\displaystyle
A_{e_{2}}$ $\displaystyle=\frac{r}{2}[-\sin 2\phi_{0}\,\sigma_{3}+\cos
2\phi_{0}\,\sigma_{1}]\equiv\frac{r}{2}\sigma_{2\phi_{0}+\pi/2}\,.$

Now, the commutation rule reads

$\displaystyle\left[A_{e_{1}},A_{e_{2}}\right]=-\frac{r^{2}}{2}\tau_{2}\,,\quad\tau_{2}:=\begin{pmatrix}0&-1\\\
1&0\end{pmatrix}=-\mathsf{i}\sigma_{2}\,,$

which depends on the real version of the last Pauli matrix and on the degree
of mixing.

A second consequence, typical of quantum-mechanical ensembles, is that all
functions $f(\phi)$ in $V_{3}$ yielding density matrices through this map
imply that

(3.11)
$\displaystyle\rho_{s,\theta}=\int_{0}^{2\pi}\underset{f(\phi)}{\underbrace{\left[\frac{1}{2}+\frac{s}{r}\,\cos
2\phi\right]}}\,\rho_{r,\phi+\theta}\,\frac{\mathrm{d}\phi}{\pi}\,.$

If $r\geq 2s$, this continuous superposition of mixed states is convex.
Therefore, a mixed state is composed of an infinite number of other mixed
states. This has consequences in quantum cryptography, for example, since the
initial signal cannot be recovered from the output.

The third and last consequence we mention here concerns the Naimark extension
of a function defined on the circle. In particular, we focus on the Toeplitz
quantization of $f(\phi)$, which is a kind of integral quantization. In [6],
we used this framework to show there exist orthogonal projectors from
$L^{2}(\mathbb{S}^{1},\mathrm{d}\phi/\pi)$ to $\mathbb{R}^{2}$ such that for a
function $f(\phi)$ the multiplication operator on
$L^{2}(\mathbb{S}^{1},\mathrm{d}\phi/\pi)$, defined by

(3.12) $\displaystyle v\mapsto M_{f}v=fv\,,$

maps $M_{f}$ to $A_{f}$. They are precisely Naimark’s extensions of POVMs
represented by density matrices (see [6] for details).

###  3.3. Linear Polarization of Light as a Quantum Phenomenon

In this section, we recall that the polarization tensor of light can be
expressed as a density matrix, which allows us to relate the polarization of
light to quantum phenomena such as the Malus Law and the incompatibility
between two sequential measurements [6].

First, remember that a complex-valued electric field for a propagating quasi-
monochromatic electromagnetic wave along the $z$-axis reads as

(3.13)
$\displaystyle\overrightarrow{\mathcal{E}}(t)=\overrightarrow{\mathcal{E}_{0}}(t)\,e^{\mathsf{i}\omega
t}=\mathcal{E}_{x}\,\widehat{\boldsymbol{\imath}}+\mathcal{E}_{y}\,\widehat{\boldsymbol{\jmath}}=\left(\mathcal{E}_{\alpha}\right)\,,$

in which we have used the previous notations for the unit vectors in the
plane. The polarization is determined by
$\overrightarrow{\mathcal{E}_{0}}(t)$. It slowly varies with time, and can be
measured through Nicol prisms, or other devices, by measuring the intensity of
the light yielded by mean values
$\propto\mathcal{E}_{\alpha}\mathcal{E}_{\beta}$,
$\mathcal{E}_{\alpha}\mathcal{E}^{\ast}_{\beta}$ and conjugates. Due to
rapidly oscillating factors and a null temporal average
$\langle\cdot\rangle_{t}$, a partially polarized light is described by the
$2\times 2$ Hermitian matrix (Stokes parameters) [10, 11, 12]

$\begin{split}\frac{1}{J}\begin{pmatrix}\left\langle\mathcal{E}_{0x}\mathcal{E}^{\ast}_{0x}\right\rangle_{t}&\left\langle\mathcal{E}_{0x}\mathcal{E}^{\ast}_{0y}\right\rangle_{t}\\\
\left\langle\mathcal{E}_{0y}\mathcal{E}^{\ast}_{0x}\right\rangle_{t}&\left\langle\mathcal{E}_{0y}\mathcal{E}^{\ast}_{0y}\right\rangle_{t}\end{pmatrix}&\equiv\rho_{r,\phi}+\frac{A}{2}\sigma_{2}\\\
&=\frac{1+r}{2}E_{\phi}+\frac{1-r}{2}E_{\phi+\pi/2}+\mathsf{i}\frac{A}{2}\tau_{2}\;.\end{split}$

Here, $J$ describes the intensity of the wave. In the second line, it is clear
that the degree of mixing $r$ describes linear polarization, while the
parameter $A$ ($-1\leq A\leq 1$) is related to circular polarization. In real
space, we have $A=0$, so we effectively describe the linear polarization of
light.

$\widehat{\boldsymbol{k}}$$\widehat{\boldsymbol{\jmath}}$$\widehat{\boldsymbol{\imath}}$$\bullet$$\mathrm{Re}\left(\overrightarrow{\mathcal{E}}\right)$$y$$z$$x$

We now wish to describe the interaction between a polarizer and a partially
linear polarized light as a quantum measurement. We need to introduce two
planes and their tensor product: the first one is the Hilbert space on which
act the states $\rho^{M}_{s,\theta}$ of the polarizer viewed as an orientation
pointer. Note that the action of the generator of rotations
$\tau_{2}=-\mathsf{i}\sigma_{2}$ on these states corresponds to a $\pi/2$
rotation :

(3.14)
$\displaystyle\tau_{2}\rho^{M}_{s,\theta}\tau_{2}^{-1}=-\tau_{2}\rho^{M}_{s,\theta}\tau_{2}=\rho^{M}_{s,\theta+\pi/2}\,.$

The second plane is the Hilbert space on which act the partially linearized
polarization states $\rho^{L}_{r,\phi}$ of the plane wave crossing the
polarizer. Its spectral decomposition corresponds to the incoherent
superposition of two completely linearly polarized waves

(3.15)
$\displaystyle\rho^{L}_{r,\phi}=\frac{1+r}{2}\,E_{\phi}+\frac{1-r}{2}\,E_{\phi+\pi/2}\,.$

The pointer detects an orientation in the plane determined by the angle
$\phi$. Through the interaction pointer-system, we generate a measurement
whose time duration is the interval $I_{M}=(t_{M}-\eta,t_{M}+\eta)$ centred at
$t_{M}$. The interaction is described by the (pseudo-) Hamiltonian operator

(3.16)
$\displaystyle\widetilde{H}_{\mathrm{int}}(t)=\,g^{\eta}_{M}(t)\tau_{2}\otimes\rho^{L}_{r,\phi}\,,$

where $g^{\eta}_{M}$ is a Dirac sequence with support in $I_{M}$, i.e.,

$\lim_{\eta\to
0}\int_{-\infty}^{+\infty}\mathrm{d}t\,f(t)\,g^{\eta}_{M}(t)=f(t_{M})\,.$

The interaction (3.16) is the tensor product of an antisymmetric operator for
the pointer with an operator for the system which is symmetric (i.e.,
Hamiltonian). The operator defined for $t_{0}<t_{M}-\eta$ as

$\displaystyle U(t,t_{0})$
$\displaystyle=\exp\left[\int_{t_{0}}^{t}\mathrm{d}t^{\prime}\,g^{\eta}_{M}(t^{\prime})\,\tau_{2}\otimes\rho^{L}_{r,\phi}\right]$
(3.17)
$\displaystyle=\exp\left[G_{M}^{\eta}(t)\,\tau_{2}\otimes\rho^{L}_{r,\phi}\right]\,,$

with
$G_{M}^{\eta}(t)=\int_{t_{0}}^{t}\mathrm{d}t^{\prime}\,\,g^{\eta}_{M}(t^{\prime})$,
is a unitary evolution operator. From the formula involving an orthogonal
projector $P$,

(3.18) $\displaystyle\exp(\theta\tau_{2}\otimes P)=\mathcal{R}(\theta)\otimes
P+\mathbbm{1}\otimes(\mathbbm{1}-P)\,,$

we obtain

(3.19) $\displaystyle U(t,t_{0})=$
$\displaystyle\mathcal{R}\left(G_{M}^{\eta}(t)\,\frac{1+r}{2}\right)\otimes
E_{\phi}+\mathcal{R}\left(G_{M}^{\eta}(t)\,\frac{1-r}{2}\right)\otimes
E_{\phi+\pi/2}\,.$

For $t_{0}<t_{M}-\eta$ and $t>t_{M}+\eta$, we finally obtain

(3.20) $\displaystyle U(t,t_{0})=\mathcal{R}\left(\frac{1+r}{2}\right)\otimes
E_{\phi}+\mathcal{R}\left(\frac{1-r}{2}\right)\otimes E_{\phi+\pi/2}\,.$

Preparing the polarizer in the state $\rho^{M}_{s_{0},\theta_{0}}$, we obtain
the evolution of the initial state for $t>t_{M}+\eta$

$\displaystyle
U(t,t_{0})\,\rho^{M}_{s_{0},\theta_{0}}\otimes\rho^{L}_{r_{0},\phi_{0}}\,U(t,t_{0})^{{\dagger}}=$
$\displaystyle\rho^{M}_{s_{0},\theta_{0}+\frac{1+r}{2}}\otimes\frac{1+r_{0}\cos
2(\phi-\phi_{0})}{2}\,E_{\phi}$
$\displaystyle+\rho^{M}_{s_{0},\theta_{0}+\frac{1-r}{2}}\otimes\frac{1-r_{0}\cos
2(\phi-\phi_{0})}{2}\,E_{\phi+\pi/2}$
$\displaystyle+\frac{1}{4}\left(\mathcal{R}(r)+s_{0}\sigma_{2\theta_{0}+1}\right)\otimes
r_{0}\sin 2(\phi-\phi_{0})\,E_{\phi}\tau_{2}$ (3.21)
$\displaystyle-\frac{1}{4}\left(\mathcal{R}(-r)+s_{0}\sigma_{2\theta_{0}+1}\right)\otimes
r_{0}\sin 2(\phi-\phi_{0})\,\tau_{2}E_{\phi}\,.$

Therefore, the probability for the pointer to rotate by $\frac{1+r}{2}$,
corresponding to the polarization along the orientation $\phi$ is

(3.22)
$\displaystyle\mathrm{Tr}\left[\left(U(t,t_{0})\,\rho^{M}_{s_{0},\theta_{0}}\otimes\rho^{L}_{r_{0},\phi_{0}}\,U(t,t_{0})^{{\dagger}}\right)\left(\mathbbm{1}\otimes
E_{\phi}\right)\right]=\frac{1+r_{0}\cos 2(\phi-\phi_{0})}{2}\,,$

that for the completely linear polarization of the light, i.e. $r_{0}=1$,
becomes the familiar Malus law, $\cos^{2}(\phi-\phi_{0})$. Similarly, the
second term gives the probability for the perpendicular orientation
$\phi+\pi/2$ and the pointer rotation by $\frac{1-r}{2}$

(3.23)
$\displaystyle\mathrm{Tr}\left[\left(U(t,t_{0})\,\rho^{M}_{s_{0},\theta_{0}}\otimes\rho^{L}_{r_{0},\phi_{0}}\,U(t,t_{0})^{{\dagger}}\right)\left(\mathbbm{1}\otimes
E_{\phi+\pi/2}\right)\right]=\frac{1-r_{0}\cos 2(\phi-\phi_{0})}{2}\,,$

corresponding (in the case $r_{0}=1$) to the Malus law
$\sin^{2}(\phi-\phi_{0})$.

##  4\. Entanglement and isomorphisms

In this section, we develop our previous results further by giving an
interpretation in terms of quantum entanglement. Previously, we described the
interaction between a polarizer and a light ray as the tensor product (3.16),
which is analogous to the quantum entanglement of states, since it is a
logical consequence of the construction of tensor products of Hilbert spaces
for describing quantum states of composite system. In the present case, we are
in presence of a remarkable sequence of vector space isomorphisms due to the
fact that $2\times 2=2+2$ 111Remind that $\mathrm{dim}(V\otimes
W)=\mathrm{dim}V\mathrm{dim}W$ while $\mathrm{dim}(V\times
W)=\mathrm{dim}V+\mathrm{dim}W$ for 2 finite-dimensional vector spaces $V$ and
$W$ :

(4.1)
$\displaystyle\mathbb{R}^{2}\otimes\mathbb{R}^{2}\cong\mathbb{R}^{2}\times\mathbb{R}^{2}\cong\mathbb{R}^{2}\oplus\mathbb{R}^{2}\cong\mathbb{C}^{2}\cong\mathbb{H}\,,$

where $\mathbb{H}$ is the field of quaternions. Therefore, the description of
the entanglement in a real Hilbert space is equivalent to the description of a
single system (e.g., a spin $1/2$) in the complex Hilbert space
$\mathbb{C}^{2}$, or in $\mathbb{H}$. In Section 4.3 we develop such an
observation.

###  4.1. Bell States and Quantum Correlations

It is straightforward to transpose into the present setting the 1964 analysis
and result presented by Bell in his discussion about the EPR paper [13] and
about the subsequent Bohm’s approaches based on the assumption of hidden
variables [14]. We only need to replace the Bell spin one-half particles with
the horizontal (i.e., $+1$) and vertical (i.e., $-1$) quantum orientations in
the plane as the only possible issues of the observable $\sigma_{\phi}$ (3.1),
supposing that there exists a pointer device designed for measuring such
orientations with outcomes $\pm 1$ only.

In order to define Bell states and their quantum correlations, let us first
write the canonical, orthonormal basis of the tensor product
$\mathbb{R}_{A}^{2}\otimes\mathbb{R}_{B}^{2}$, the first factor being for
system “$A$” and the other for system “$B$”, as

(4.2)
$\begin{split}&|0\rangle_{A}\otimes|0\rangle_{B}\,,\quad\left|\frac{\pi}{2}\right\rangle_{A}\otimes\left|\frac{\pi}{2}\right\rangle_{B}\,,\quad|0\rangle_{A}\otimes\left|\frac{\pi}{2}\right\rangle_{B}\,,\quad\left|\frac{\pi}{2}\right\rangle_{A}\otimes|0\rangle_{B}\,.\end{split}$

The states $|0\rangle$ and $\left|\frac{\pi}{2}\right\rangle$ pertain to $A$
or $B$, and are named “$q$-bit” or “qubit” in the standard language of quantum
information. Since they are pure states, they can be associated to a pointer
measuring the horizontal (resp. vertical) direction or polarisation described
by the state $|0\rangle$ (resp. $\left|\frac{\pi}{2}\right\rangle$).

There are four Bell pure states in
$\mathbb{R}_{A}^{2}\otimes\mathbb{R}_{B}^{2}$, namely

(4.3) $\displaystyle|\Phi^{\pm}\rangle$
$\displaystyle=\frac{1}{\sqrt{2}}\left(|0\rangle_{A}\otimes|0\rangle_{B}\pm\left|\frac{\pi}{2}\right\rangle_{A}\otimes\left|\frac{\pi}{2}\right\rangle_{B}\right)\,,$
(4.4) $\displaystyle|\Psi^{\pm}\rangle$
$\displaystyle=\frac{1}{\sqrt{2}}\left(\pm|0\rangle_{A}\otimes\left|\frac{\pi}{2}\right\rangle_{B}+\left|\frac{\pi}{2}\right\rangle_{A}\otimes|0\rangle_{B}\right)\,.$

We say that they represent _maximally entangled_ quantum states of two qubits.
Consider for instance the state $|\Phi^{+}\rangle$. If the pointer associated
to $A$ measures its qubit in the standard basis, the outcome would be
perfectly random, with either possibility having a probability 1/2. But if the
pointer associated to $B$ then measures its qubit instead, the outcome,
although random for it alone, is the same as the one $A$ gets. There is
_quantum correlation_.

###  4.2. Bell Inequality and Its Violation

Let us consider a bipartite system in the state $\Psi^{-}$. In such a state,
if a measurement of the component
$\sigma^{A}_{\phi_{a}}:=\overrightarrow{\boldsymbol{\sigma}}^{A}\cdot\widehat{\mathbf{u}}_{\phi_{a}}$
($\widehat{\mathbf{u}}_{\phi_{a}}$ is an unit vector with polar angle
$\phi_{a}$) yields the value $+1$ (polarization along the direction
$\phi_{a}/2$), then a measurement of $\sigma^{B}_{\phi_{b}}$ when
$\phi_{b}=\phi_{a}$ must yield the value $-1$ (polarization along the
direction $\frac{\phi_{a}+\pi}{2}$), and vice-versa. From a classical
perspective, the explanation of such a correlation needs a predetermination by
means of the existence of hidden parameters $\lambda$ in some set $\Lambda$.
Assuming the two measurements to be separated by a space-like interval, the
result $\varepsilon^{A}\in\\{-1,+1\\}$ (resp. $\varepsilon^{B}\in\\{-1,+1\\}$)
of measuring $\sigma^{A}_{\phi_{a}}$ (resp. $\sigma^{B}_{\phi_{b}}$) is then
determined by $\phi_{a}$ and $\lambda$ only (locality assumption), not by
$\phi_{b}$, i.e. $\varepsilon^{A}=\varepsilon^{A}(\phi_{a},\lambda)$ (resp.
$\varepsilon^{B}=\varepsilon^{B}(\phi_{b},\lambda)$). Given a probability
distribution $\rho(\lambda)$ on $\Lambda$, the classical expectation value of
the product of the two components $\sigma^{A}_{\phi_{a}}$ and
$\sigma^{B}_{\phi_{b}}$ is given by

(4.5)
$\displaystyle\mathsf{P}(\phi_{a},\phi_{b})=\int_{\Lambda}\mathrm{d}\lambda\,\rho(\lambda)\,\varepsilon^{A}(\phi_{a},\lambda)\,\varepsilon^{B}(\phi_{b},\lambda)\,.$

Since

(4.6)
$\displaystyle\int_{\Lambda}\mathrm{d}\lambda\,\rho(\lambda)=1\,\quad\textup{and}\quad\varepsilon^{A,B}=\pm
1\;,$

we have $-1\leq\mathsf{P}(\phi_{a},\phi_{b})\leq 1$. Equivalent predictions
within the quantum setting then imposes the equality between the classical and
quantum expectation values:

$\displaystyle\mathsf{P}(\phi_{a},\phi_{b})$
$\displaystyle=\left\langle\Psi^{-}\right|\sigma^{A}_{\phi_{a}}\otimes\sigma^{B}_{\phi_{b}}\left|\Psi^{-}\right\rangle$
(4.7)
$\displaystyle=-\widehat{\mathbf{u}}_{\phi_{a}}\cdot\widehat{\mathbf{u}}_{\phi_{b}}=-\cos(\phi_{a}-\phi_{b})\,.$

In the above equation, the value $-1$ is reached at $\phi_{a}=\phi_{b}$. This
is possible for $\mathsf{P}(\phi_{a},\phi_{a})$ only if
$\varepsilon^{A}(\phi_{a},\lambda)=-\varepsilon^{B}(\phi_{a},\lambda)$. Hence,
we can write $\mathsf{P}(\phi_{a},\phi_{b})$ as

(4.8) $\displaystyle\mathsf{P}(\phi_{a},\phi_{b})$
$\displaystyle=-\int_{\Lambda}\mathrm{d}\lambda\,\rho(\lambda)\,\varepsilon(\phi_{a},\lambda)\,\varepsilon(\phi_{b},\lambda)\,,\quad\varepsilon(\phi,\lambda)\equiv\varepsilon^{A}(\phi,\lambda)=\pm
1\,.$

Let us now introduce a third unit vector $\widehat{\mathbf{u}}_{\phi_{c}}$.
Due to $\varepsilon^{2}=1$, we have

(4.9)
$\displaystyle\mathsf{P}(\phi_{a},\phi_{b})-\mathsf{P}(\phi_{a},\phi_{c})$
$\displaystyle=\int_{\Lambda}\mathrm{d}\lambda\,\rho(\lambda)\,\varepsilon(\phi_{a},\lambda)\,\varepsilon(\phi_{b},\lambda)\,\left[\varepsilon(\phi_{b},\lambda)\,\varepsilon(\phi_{c},\lambda)-1\right]\,.$

From this results the (baby) Bell inequality:

$\displaystyle|\mathsf{P}(\phi_{a},\phi_{b})-\mathsf{P}(\phi_{a},\phi_{c})|\leq\int_{\Lambda}\mathrm{d}\lambda\,\rho(\lambda)\,\left[1-\varepsilon(\phi_{b},\lambda)\,\varepsilon(\phi_{c},\lambda)\right]=1+\mathsf{P}(\phi_{b},\phi_{c})\,.$

Hence, the validity of the existence of hidden variable(s) for justifying the
quantum correlation in the singlet state $\Psi^{-}$, and which is encapsulated
by the above equation, has the following consequence on the arbitrary triple
$(\phi_{a},\phi_{b},\phi_{c})$:

$\displaystyle
1-\cos(\phi_{b}-\phi_{c})\geq\left|\cos(\phi_{b}-\phi_{a})-\cos(\phi_{c}-\phi_{a})\right|\,.$

Equivalently, in terms of the two independent angles $\zeta$ and $\eta$,

$\displaystyle\zeta=\dfrac{\phi_{a}-\phi_{b}}{2}\,,\quad\eta=\dfrac{\phi_{b}-\phi_{c}}{2}\;,$

we have

(4.10)
$\displaystyle\left|\sin^{2}\zeta-\sin^{2}(\eta+\zeta)\right|\leq\sin^{2}\eta\,.$

It is easy to find pairs $(\zeta,\eta)$ for which the inequality (4.10) does
not hold true. For instance with $\eta=\zeta\neq 0$, i.e.,

$\phi_{b}=\dfrac{\phi_{a}+\phi_{c}}{2}\;,$

we obtain

(4.11) $\displaystyle|4\sin^{2}\eta-3|\leq 1\,,$

which does not hold true for all $|\eta|<\pi/4$, i.e., for
$\left|\phi_{a}-\phi_{b}\right|=\left|\phi_{b}-\phi_{c}\right|<\pi/2$.
Actually, we did not follow here the proof given by Bell, which is a lot more
elaborate. Also, Bell considered unit vectors in $3$-space. Restricting his
proof to vectors in the plane does not make any difference, as it is actually
the case in many works devoted to the foundations of quantum mechanics.

###  4.3. Entanglement of two angles

Quantum entanglement is usually described by the complex two-dimensional
Hilbert space $\mathbb{C}^{2}$. As a complex vector space, $\mathbb{C}^{2}$,
with canonical basis ($\mathbf{e}_{1}$, $\mathbf{e}_{2}$), has a real
structure, i.e., is isomorphic to a real vector space which makes it
isomorphic to $\mathbb{R}^{4}$, itself isomorphic to
$\mathbb{R}^{2}\otimes\mathbb{R}^{2}$. A real structure is obtained by
considering the vector expansion

$\displaystyle\mathbb{C}^{2}\in\mathbf{v}$
$\displaystyle=z_{1}\mathbf{e}_{1}+z_{2}\mathbf{e}_{2}$ (4.12)
$\displaystyle=x_{1}\mathbf{e}_{1}+y_{1}\left(\mathsf{i}\mathbf{e}_{1}\right)+x_{2}\mathbf{e}_{2}+y_{2}\left(\mathsf{i}\mathbf{e}_{2}\right)\,,$

which is equivalent to writing $z_{1}=x_{1}+\mathsf{i}y_{1}$,
$z_{2}=x_{2}+\mathsf{i}y_{2}$, and considering the set of vectors

(4.13)
$\displaystyle\left\\{\mathbf{e}_{1},\mathbf{e}_{2},\left(\mathsf{i}\mathbf{e}_{1}\right),\left(\mathsf{i}\mathbf{e}_{2}\right)\right\\}$

as forming a basis of $\mathbb{R}^{4}$. Forgetting about the subscripts $A$
and $B$ in (4.2), we can map vectors in the Euclidean plane $\mathbb{R}^{2}$
to the complex “plane” $\mathbb{C}$ by

(4.14) $\displaystyle|0\rangle\mapsto
1\,,\qquad\left|\frac{\pi}{2}\right\rangle\mapsto\mathsf{i}\,,$

which allows the correspondence between bases as

(4.15)
$\displaystyle|0\rangle\otimes|0\rangle=\mathbf{e}_{1}\,,\quad\left|\frac{\pi}{2}\right\rangle\otimes\left|\frac{\pi}{2}\right\rangle=-\mathbf{e}_{2}\,,\quad|0\rangle\otimes\left|\frac{\pi}{2}\right\rangle=\left(\mathsf{i}\mathbf{e}_{1}\right)\,,\quad\left|\frac{\pi}{2}\right\rangle\otimes|0\rangle=\left(\mathsf{i}\mathbf{e}_{2}\right)\;.$

Also, the spin of a particle in a real basis, given by the “up” and “down”
states, are defined by

(4.16)
$\displaystyle\mathbf{e}_{1}\equiv|\uparrow\,\rangle\equiv\begin{pmatrix}1\\\
0\end{pmatrix}\,,\qquad\mathbf{e}_{2}\equiv|\downarrow\,\rangle\equiv\begin{pmatrix}0\\\
1\end{pmatrix}\,.$

Finally, we obtain an unitary map from the Bell basis to the basis of real
structure of $\mathbb{C}^{2}$

$\displaystyle\begin{pmatrix}|\Phi^{+}\rangle&|\Phi^{-}\rangle&|\Psi^{+}\rangle&|\Psi^{-}\rangle\end{pmatrix}=\begin{pmatrix}\mathbf{e}_{1}&\mathbf{e}_{2}&\left(\mathsf{i}\mathbf{e}_{1}\right)&\left(\mathsf{i}\mathbf{e}_{2}\right)\end{pmatrix}\frac{1}{\sqrt{2}}\begin{pmatrix}1&1&0&0\\\
-1&1&0&0\\\ 0&0&1&-1\\\ 0&0&1&1\end{pmatrix}\,.$

In terms of respective components of vectors in their respective spaces, we
have

(4.17) $\displaystyle\begin{pmatrix}x_{1}\\\ x_{2}\\\ y_{1}\\\
y_{2}\end{pmatrix}=\frac{1}{\sqrt{2}}\begin{pmatrix}1&1&0&0\\\ -1&1&0&0\\\
0&0&1&-1\\\ 0&0&1&1\end{pmatrix}\begin{pmatrix}x^{+}\\\ x^{-}\\\ y^{+}\\\
y^{-}\end{pmatrix}\,.$

In complex notations, with $z^{\pm}=x^{\pm}+\mathsf{i}y^{\pm}$, this is
equivalent to

(4.18) $\displaystyle\begin{pmatrix}z^{+}\\\
z^{-}\end{pmatrix}=\frac{1}{\sqrt{2}}\begin{pmatrix}1&-\mathsf{C}\\\
\mathsf{C}&1\end{pmatrix}\begin{pmatrix}z_{1}\\\
z_{2}\end{pmatrix}\equiv\mathcal{C}_{@}\begin{pmatrix}z_{1}\\\
z_{2}\end{pmatrix}\,,$

in which we have introduced the conjugation operator $\mathsf{C}z=\bar{z}$,
i.e., the mirror symmetry with respect to the real axis, $-\mathsf{C}$ being
the mirror symmetry with respect to the imaginary axis.

Let us now see what is the influence of having real Bell states on Schrödinger
cat states. The operator “cat” $\mathcal{C}_{@}$ can be expressed as

(4.19)
$\displaystyle\mathcal{C}_{@}=\frac{1}{\sqrt{2}}\left(\mathbbm{1}+\mathsf{F}\right)\,,\quad\mathsf{F}:=\mathsf{C}\tau_{2}=\begin{pmatrix}0&-\mathsf{C}\\\
\mathsf{C}&0\end{pmatrix}\,.$

Therefore, with the above choice of isomorphisms, Bell entanglement in
$\mathbb{R}^{2}\otimes\mathbb{R}^{2}$ is not represented by a simple linear
superposition in $\mathbb{C}^{2}$. It involves also the two mirror symmetries
$\pm\mathsf{C}$. The operator $\mathsf{F}$ is a kind of “flip” whereas the
“cat” or “beam splitter” operator $\mathcal{C}_{@}$ builds, using the up and
down basic states, the two elementary Schrödinger cats

(4.20) $\displaystyle\mathsf{F}\,|\uparrow\,\rangle$
$\displaystyle=|\downarrow\,\rangle\,,\quad\mathcal{C}_{@}\,|\uparrow\,\rangle=\frac{1}{\sqrt{2}}(|\uparrow\,\rangle+|\downarrow\,\rangle)\,,$
(4.21) $\displaystyle\mathsf{F}\,|\downarrow\,\rangle$
$\displaystyle=-|\uparrow\,\rangle\,,\quad\mathcal{C}_{@}\,|\downarrow\,\rangle=\frac{1}{\sqrt{2}}(-|\uparrow\,\rangle+|\downarrow\,\rangle)\,.$

The flip operator also appears in the construction of the spin one-half
coherent states $|\theta,\phi\rangle$, defined in terms of spherical
coordinates $(\theta,\phi)$ as the quantum counterpart of the classical state
$\hat{\mathbf{n}}(\theta,\phi)$ in the sphere $\mathbb{S}^{2}$ by

(4.22)
$\begin{split}|\theta,\phi\rangle=\left(\cos\frac{\theta}{2}\,|\uparrow\,\rangle+e^{\mathsf{i}\phi}\sin\frac{\theta}{2}\,|\downarrow\,\rangle\right)\equiv\begin{pmatrix}\cos\frac{\theta}{2}e^{\mathsf{i}\phi}\sin\frac{\theta}{2}\end{pmatrix}=\begin{pmatrix}\cos\frac{\theta}{2}&-\sin\frac{\theta}{2}e^{-\mathsf{i}\phi}\\\
\sin\frac{\theta}{2}e^{\mathsf{i}\phi}&\cos\frac{\theta}{2}\end{pmatrix}\begin{pmatrix}1\\\
0\end{pmatrix}\equiv
D^{\frac{1}{2}}\left(\xi^{-1}_{\hat{\mathbf{n}}}\right)|\uparrow\,\rangle\,.\end{split}$

Here, $\xi_{\hat{\mathbf{n}}}$ corresponds, through the homomorphism SO$(3)$
$\mapsto$ SU$(2)$, to the specific rotation $\mathcal{R}_{\hat{\mathbf{n}}}$
mapping the unit vector pointing to the north pole,
$\hat{\mbox{{{k}}}}=(0,0,1)$, to $\hat{\mathbf{n}}$. The operator
$D^{\frac{1}{2}}\left(\xi^{-1}_{\hat{\mathbf{n}}}\right)$ represents the
element $\xi^{-1}_{\hat{\mathbf{n}}}$ of SU$(2)$ in its complex two-
dimensional unitary irreducible representation. As we can see in matrix
(4.22), the second column of
$D^{\frac{1}{2}}\left(\xi^{-1}_{\hat{\mathbf{n}}}\right)$ is precisely the
flip of the first one,

(4.23) $\displaystyle
D^{\frac{1}{2}}\left(\xi^{-1}_{\hat{\mathbf{n}}}\right)=\begin{pmatrix}|\theta,\phi\rangle&\mathsf{F}|\theta,\phi\rangle\end{pmatrix}\,.$

Actually, we can learn more about the isomorphisms
$\mathbb{C}^{2}\cong\mathbb{H}\cong\mathbb{R}_{+}\times\mathrm{SU}(2)$ through
the flip and matrix representations of quaternions. In quaternionic algebra,
we have the property
$\hat{\boldsymbol{\imath}}=\hat{\boldsymbol{\jmath}}\hat{\mbox{{{k}}}}$ $+$
even permutations, and a quaternion $q$ is represented by

$\displaystyle\mathbb{H}\ni q$
$\displaystyle=q_{0}+q_{1}\hat{\boldsymbol{\imath}}+q_{2}\hat{\boldsymbol{\jmath}}+q_{3}\hat{\mbox{{{k}}}}=q_{0}+q_{3}\hat{\mbox{{{k}}}}+\hat{\boldsymbol{\jmath}}\left(q_{1}\hat{\mbox{{{k}}}}+q_{2}\right)$
(4.24) $\displaystyle\equiv\begin{pmatrix}q_{0}+\mathsf{i}q_{3}\\\
q_{2}+\mathsf{i}q_{1}\end{pmatrix}\equiv Z_{q}\in\mathbb{C}^{2}\,,$

after identifying $\hat{\mbox{{{k}}}}\equiv\mathsf{i}$ as both are roots of
$-1$. Then the flip appears naturally in the final identification
$\mathbb{H}\cong\mathbb{R}_{+}\times\mathrm{SU}(2)$ as

(4.25) $q\equiv\begin{pmatrix}q_{0}+\mathsf{i}q_{3}&-q_{2}+\mathsf{i}q_{1}\\\
q_{2}+\mathsf{i}q_{1}&q_{0}-\mathsf{i}q_{3}\end{pmatrix}=\begin{pmatrix}Z_{q}&\mathsf{F}Z_{q}\end{pmatrix}\,.$

Let us close this article with a final remark on spin-$1/2$ coherent states as
vectors in $\mathbb{R}_{A}^{2}\otimes\mathbb{R}_{B}^{2}$. The “cat states” in
$\mathbb{C}^{2}$ given by (4.22) and equivalently viewed as $4$-vectors in
$\mathbb{H}\sim\mathbb{R}^{4}$ as

(4.26)
$\displaystyle|\theta,\phi\rangle\mapsto\begin{pmatrix}\cos\frac{\theta}{2}\\\
-\sin\frac{\theta}{2}\cos\phi\\\ \sin\frac{\theta}{2}\sin\phi\\\
0\end{pmatrix}\,,$

are represented as entangled states in
$\mathbb{R}_{A}^{2}\otimes\mathbb{R}_{B}^{2}$ by

$\begin{split}|\theta,\phi\rangle=&\cos\frac{\theta}{2}|0\rangle_{A}\otimes|0\rangle_{B}\
-\sin\frac{\theta}{2}\cos\phi\left|\frac{\pi}{2}\right\rangle_{A}\otimes\left|\frac{\pi}{2}\right\rangle_{B}\\\
&+\sin\frac{\theta}{2}\sin\phi|0\rangle_{A}\otimes\left|\frac{\pi}{2}\right\rangle_{B}+0\left|\frac{\pi}{2}\right\rangle_{A}\otimes|0\rangle_{B}\;.\end{split}$

Therefore, we can say that two entangled angles in the plane can be viewed as
a point in the upper half-sphere $\mathbb{S}^{2}/\mathbb{Z}_{2}$ in
$\mathbb{R}^{3}$ shown in Figure 2.

Figure 2. Each point in the upper half-sphere is in one-to-one correspondence
with two entangled angles in the plane.

##  5\. Conclusions

Integral quantization is a quantization scheme constructed on Positive
Operator-Value Measures. When applied to a two-dimensional real space, it
allows for a description of quantum states as pointers in the real unit half-
plane. We recalled in this paper that in this case, a family of density
matrices is sufficient to perform this kind of quantization as it describes
all the mixed states in this space. Furthermore, a density matrix in a two-
dimensional real space depends on the usual observable
$\sigma_{\phi}=\begin{pmatrix}\cos\phi&\sin\phi\\\
\sin\phi&-\cos\phi\end{pmatrix}$, which captures the essence of non-
commutativity in real space. As a consequence, commutation relations are
expressed in terms of the real matrix $\tau_{2}$, which serves as the basis to
the description of quantum measurement.

We provide an illustration considering linearly-polarized light passing
through a polarizer. The pointer, associated with $\tau_{2}$, can rotate by an
angle $(1\pm r)/2$ with $r$ the degree of mixing of the density matrix, with a
probability given by the usual Malus’ laws (3.22) and (3.23). We extended the
analysis by showing that the interaction between a polarizer and a light ray
is equivalent to the quantum entanglement of two Hilbert spaces. Orientations
in the plane have only two outcomes ($\pm 1$), which are the possible issues
of $\sigma_{\phi}$. We showed that for a general bipartite system, the
classical and quantum measurement of $\sigma_{\phi}$ deny the existence of
local hidden variables, resulting in the well-known violation of Bell
inequalities, here given by (4.10). Finally, we demonstrated that the
isomorphism $\mathbb{C}^{2}\simeq\mathbb{R}^{4}$ allows to write Bell states
in real space, with the introduction of the “flip” operator (4.19). This
operator is necessary for constructing spin one-half coherent states, that we
can fully describe by a set of orientations in $\mathbb{R}^{3}$, as shown in
(4.26).

### Acknowledgments

R.B. The present work was performed under the auspices of the GNFM (Gruppo
Nazionale di Fisica Matematica). EF thanks the Helsinki Institute of Physics
(HIP) for their hospitality.

## References

  * [1] Ernst CG Stueckelberg  “Quantum theory in real Hilbert space”  In _Helv. Phys. Acta_ 33.727, 1960, pp. 458
  * [2] Maria Pia Sol r  “Characterization of Hilbert spaces by orthomodular spaces”  In _Communications in Algebra_ 23.1 Taylor & Francis, 1995, pp. 219–243
  * [3] Valter Moretti and Marco Oppio  “Quantum theory in quaternionic Hilbert space: How Poincar symmetry reduces the theory to the standard complex one”  In _Reviews in Mathematical Physics_ 31.04 World Scientific, 2019, pp. 1950013
  * [4] Asher Peres  “Neumark’s Theorem and Quantum Inseparability”  In _Foundations of Physics_ 20, 1990, pp. 1441–1453
  * [5] Herv Bergeron, Evaldo MF Curado, Jean-Pierre Gazeau and Ligia MCS Rodrigues  “Orientations in the Plane as Quantum States”  In _Brazilian Journal of Physics_ 49.3 Springer, 2019, pp. 391–401
  * [6] Roberto Beneduci, Emmanuel Frion, Jean-Pierre Gazeau and Amedeo Perri  “Real POVMS on the plane: integral quantization, Naimark theorem and linear polarization of the light”  In _arXiv preprint arXiv:2108.04086_ , 2021
  * [7] Roberto Beneduci  “Joint measurability through Naimark’s theorem”  In _arXiv preprint arXiv:1404.1477_ , 2014
  * [8] Alexander S Holevo  “Probabilistic and statistical aspects of quantum theory”  Springer Science & Business Media, 2011
  * [9] Herv Bergeron and Jean-Pierre Gazeau  “Integral quantizations with two basic examples”  In _Annals of Physics_ 344 Elsevier, 2014, pp. 43–68
  * [10] H William  “McMaster. Polarization and the Stokes parameters”  In _American Journal of Physics_ 22.6, 1954, pp. 351–362
  * [11] Beth Schaefer et al.  “Measuring the Stokes polarization parameters”  In _American Journal of Physics_ 75.2 American Association of Physics Teachers, 2007, pp. 163–168
  * [12] E.. L..  “The Classical Theory of Fields. Vol. 2 (4th ed.)”  In _Butterworth-Heinemann. ISBN 978-0-7506-2768-9._ Butterworth-Heinemann. ISBN 978-0-7506-2768-9., 1975
  * [13] John S Bell  “On the Einstein-Podolsky-Rosen paradox”  In _Physics_ 1.3 APS, 1964, pp. 195
  * [14] David Bohm  “A suggested interpretation of the quantum theory in terms of" hidden" variables. I”  In _Physical Review_ 85.2 APS, 1952, pp. 166


:

# Small-Group Learning, with Application to Neural Architecture Search

Xuefeng Du<EMAIL_ADDRESS>Pengtao Xie*<EMAIL_ADDRESS>
University of California San Diego

###### Abstract

Small-group learning is a broadly used methodology in human learning and shows
great effectiveness in improving learning outcomes: a small group of students
work together towards the same learning objective, where they express their
understanding of a topic to their peers, compare their ideas, and help each
other to trouble-shoot problems. We are interested in investigating whether
this powerful learning technique can be borrowed from humans to improve the
learning abilities of machines. We propose a novel learning approach called
small-group learning (SGL). In our approach, each learner uses its
intermediately trained model to generate a pseudo-labeled dataset and re-
trains its model using pseudo-labeled datasets generated by other learners. We
propose a multi-level optimization framework to formulate SGL which involves
three learning stages: learners train their network weights independently;
learners train their network weights collaboratively via mutual pseudo-
labeling; learners improve their architectures by minimizing validation
losses. We develop an efficient algorithm to solve the SGL problem. We apply
our approach to neural architecture search and achieve significant improvement
on CIFAR-100, CIFAR-10, and ImageNet.

##  1 Introduction

††footnotetext: ∗Corresponding author.

In human learning, an effective and widely used methodology for improving
learning outcome is small group learning (SGL). In SGL, a small group of
students work together towards the same learning objective, where they express
their understanding of a topic to their peers, compare their ideas, and help
each other to trouble-shoot problems. SGL can effectively facilitate deeper
and more meaningful learning.

Inspired by this collaboration-driven learning technique of humans, we are
interested in investigating whether this methodology is helpful for improving
machine learning as well. We propose a novel learning framework – small-group
learning (SGL). In this framework, there are a group of learner models. They
solve the same learning task $T$. Without loss of generality, we assume $T$ is
classification while noting that our framework can be applied to other tasks
as well. Each learner $k$ has a learnable neural architecture $A_{k}$ and two
sets of network weights $W_{k}$ and $V_{k}$. The architectures and network
weights of different learners are different. All learners share the same
training dataset and validation dataset. These learners help each other to
learn, in the following way: given an unlabeled dataset, each learner uses its
intermediately trained model (including architecture and network weights) to
generate pseudo-labels on this dataset and each learners leverages the pseudo-
labeled datasets generated by other peers to re-train its model. In this
framework, there are three learning stages. In the first stage, each learner
$k$ trains its network weights $V_{k}$ on the training dataset, with its
architecture $A_{k}$ fixed. In the second stage, each learner $k$ uses its
optimally trained $V^{*}_{k}$ in the first stage to make predictions on the
unlabeled dataset and generates a pseudo-labeled dataset; then each learner
$k$ trains its second set of network weights $V_{k}$ using the pseudo-labeled
datasets generated by other learners and the human-labeled training dataset.
In the third stage, each learner $k$ updates its architecture $A_{k}$ by
minimizing the prediction losses on the validation dataset. The three stages
are performed jointly end-to-end in a multi-level optimization framework,
where different stages influence each other. We apply our method for neural
architecture search in image classification tasks on CIFAR-100, CIFAR-10, and
ImageNet (Deng et al., 2009). Our method achieves significantly better
accuracy than state-of-the-art baselines.

The major contributions of this paper is as follows:

  * •

Inspired by the small-group learning technique of humans, we propose a novel
machine learning approach called small-group learning (SGL). In our approach,
each learner uses its intermediately trained model to generate a pseudo-
labeled dataset and re-trains its model using pseudo-labeled datasets
generated by other learners.

  * •

We propose a multi-level optimization framework to formulate SGL which
involves three learning stages: learners learn independently; learners learn
collaboratively; learners validate themselves.

  * •

We develop an efficient algorithm to solve the SGL problem.

  * •

We apply our approach for neural architecture search on CIFAR-100, CIFAR-10,
and ImageNet. The results demonstrate the effectiveness of our method.

The rest of the paper is organized as follows. Section 2 and 3 present the
method and experiments respectively. Section 4 reviews related works. Section
5 concludes the paper.

##  2 Methods

In this section, we propose a framework for small-group learning (SGL) and
develop an optimization for solving the SGL problem.

###  2.1 Small-Group Learning

In our framework, there are a set of $K$ learners, all of which learn to solve
the same target task. Without loss of generality, we assume the task is
classification. Each learner $k$ has a learnable architecture $A_{k}$ and two
sets of learnable network weights $V_{k}$ and $W_{k}$. All learners share the
same training dataset $D^{(\textrm{tr})}$, the same validation dataset
$D^{(\textrm{val})}$, and an unlabeled dataset $D^{(\textrm{u})}$. The $K$
learners perform learning in three stages. In the first stage, each learner
$k$ trains its weights $V_{k}$, with its architecture $A_{k}$ fixed:

$V^{*}_{k}(A_{k})=\textrm{min}_{V_{k}}\;L(V_{k},A_{k},D^{(\textrm{tr})}).$ (1)

The architecture $A_{k}$ is used to define the training loss. But it is not
updated at this stage. If $A_{k}$ is learned by minimizing this training loss,
a trivial solution will be yielded where $A_{k}$ is very large and complex
that it can perfectly overfit the training data but will generalize poorly on
unseen data. Note that the optimally trained weights $V^{*}_{k}(A_{k})$ is a
function of $A_{k}$ since $V^{*}_{k}(A_{k})$ is a function of
$L(V_{k},A_{k},D^{(\textrm{tr})})$ and $L(V_{k},A_{k},D^{(\textrm{tr})})$ is a
function of $A_{k}$.

In the second stage, each learner $k$ uses its optimal model
$V^{*}_{k}(A_{k})$ learned in the first stage to make predictions on the
unlabeled dataset $D^{(\textrm{u})}=\\{x_{i}\\}_{i=1}^{N}$ and generates a
pseudo-labeled dataset
$D^{(\textrm{pl})}_{k}(D^{(\textrm{u})},V^{*}_{k}(A_{k}))=\\{(x_{i},f(x_{i};V^{*}_{k}(A_{k})))\\}_{i=1}^{N}$,
where $f(\cdot;V^{*}_{k}(A_{k}))$ is the network parameterized by
$V^{*}_{k}(A_{k})$. $f(x_{i};V^{*}_{k}(A_{k}))$ is a $J$-dimensional vector
where $J$ is the number of classes and the $j$-th element of
$f(x_{i};V^{*}_{k}(A_{k}))$ indicates the probability that $x_{i}$ belongs to
the $j$-th class. The sum of all elements in $f(x_{i};V^{*}_{k}(A_{k}))$ is
one. Meanwhile, each learner $k$ uses the pseudo-labeled datasets
$\\{D^{(\textrm{pl})}_{j}\\}_{j\neq k}^{K}$ produced by other learners as well
as the human-labeled dataset $D^{(\textrm{tr})}$ to train its network weights
$W_{k}$

$\begin{array}[]{l}W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq
k}^{K})=\underset{W_{k}}{\textrm{min}}\;L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum\limits_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{*}_{j}(A_{j})))\end{array}$
(2)

where the first loss term $L(W_{k},A_{k},D^{(\textrm{tr})})$ in the objective
is defined on the human-labeled training dataset and the second loss term is
defined on the pseudo-labeled datasets produced by other learners. Both losses
are cross-entropy losses. $\lambda$ is a tradeoff parameter. Note that
$W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K})$ is a function of
$A_{k}$ and $\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K}$ since
$W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K})$ is a function of
$L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{*}_{j}(A_{j})))$
which is a function of $A_{k}$ and $\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K}$. In
the third stage, each learner validates its network weights
$W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K})$ on the validation set
$D^{(\textrm{val})}$. The learners optimize their architectures by minimizing
the validation losses:

$\underset{\\{A_{k}\\}_{k=1}^{K}}{\textrm{min}}\;\sum\limits_{k=1}^{K}L(W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq
k}^{K}),A_{k},D^{(\textrm{val})}).$ (3)

The three stages are mutually dependent: $\\{V^{*}_{k}(A_{k})\\}_{k=1}^{K}$
learned in the first stage are used to define the objective function in the
second stage; $\\{W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq
k}^{K})\\}_{k=1}^{K}$ learned in the second stage are used to define the
objective in the third stage; the updated $\\{A_{k}\\}_{k=1}^{K}$ in the third
stage in turn change the objective functions in the first stage, which
subsequently render $\\{V^{*}_{k}(A_{k})\\}_{k=1}^{K}$ to be changed.

Putting these pieces together, we have the following SGL framework, which is a
three-level optimization problem:

$\begin{array}[]{ll}\underset{\\{A_{k}\\}_{k=1}^{K}}{\textrm{min}}&\sum\limits_{k=1}^{K}L(W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq
k}^{K}),A_{k},D^{(\textrm{val})})\\\
s.t.&\\{W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq
k}^{K})\\}_{k=1}^{K}=\underset{\\{W_{k}\\}_{k=1}^{K}}{\textrm{min}}\;\sum\limits_{k=1}^{K}L(W_{k},A_{k},D^{(\textrm{tr})})\\\
&\qquad\qquad\qquad\qquad\quad\;+\lambda\sum\limits_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{*}_{j}(A_{j})))\\\
&\\{V^{*}_{k}(A_{k})\\}_{k=1}^{K}=\underset{\\{V_{k}\\}_{k=1}^{K}}{\textrm{min}}\;\sum\limits_{k=1}^{K}L(V_{k},A_{k},D^{(\textrm{tr})})\end{array}$
(4)

This formulation nests three optimization problems. On the constraints of the
outer optimization problem are two inner optimization problems corresponding
to the first and second learning stage respectively. The objective function of
the outer optimization problem corresponds to the third learning stage.

Figure 1: Illustration of small-group learning. The solid arrows denote the process of making predictions and calculating losses. The dotted arrows denote the process of updating learnable parameters by minimizing corresponding losses. For simplicity, we assume there are two learners in the group. Extension to multiple learners is straightforward. Notation | Meaning
---|---
$A_{k}$ | Architecture of learner $k$
$V_{k}$ | The first set of weights of learner $k$
$W_{k}$ | The second set of weights of learner $k$
$D^{(\textrm{tr})}$ | Training dataset
$D^{(\textrm{val})}$ | Validation dataset
$D^{(\textrm{u})}$ | Unlabeled dataset
Table 1: Notations in small-group learning

Similar to (Liu et al., 2019), we represent the architecture $A$ of the
learner in a differentiable way. The search space of $A$ is composed of a
large number of building blocks. The output of each block is associated with a
variable $a$ indicating how important this block is. After learning, blocks
whose $a$ is among the largest are retained to form the final architecture. In
this end, architecture search amounts to optimizing the set of architecture
variables $A=\\{a\\}$.

###  2.2 Optimization Algorithm

In this section, we derive an optimization algorithm to solve the SGL problem
defined in Eq.(4). Inspired by (Liu et al., 2019), we approximate
$V^{*}_{k}(A_{k})$ using one-step gradient descent update of $V_{k}$ with
respect to $L(V_{k},A_{k},D^{(\textrm{tr})})$. We plug the approximation
$\\{V^{\prime}_{j}\\}_{j=1}^{K}$ of $\\{V^{*}_{j}(A_{j})\\}_{j=1}^{K}$ into
$\sum_{k=1}^{K}(L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{*}_{j}(A_{j}))))$
and obtain an approximated objective $\sum_{k=1}^{K}O^{w}_{k}$. Then we
approximate $W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K})$ using one-
step gradient descent update of $W_{k}$ with respect to $O^{w}_{k}$. Finally,
we plug the approximation $\\{W^{\prime}_{k}\\}_{k=1}^{K}$ of
$\\{W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K})\\}_{k=1}^{K}$ into
$\sum_{k=1}^{K}L(W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq
k}^{K}),A_{k},D^{(\textrm{val})})$ and perform gradient-descent update of
$\\{A_{k}\\}_{k=1}^{K}$ with respect to this approximated objective. In the
sequel, we use $\nabla^{2}_{Y,X}f(X,Y)$ to denote $\frac{\partial
f(X,Y)}{\partial X\partial Y}$.

We approximate $V_{k}^{*}(A_{k})$ using

$V^{\prime}_{k}=V_{k}-\xi_{v}\nabla_{V_{k}}L(V_{k},A_{k},D^{(\mathrm{tr})})$
(5)

where $\xi_{v}$ is a learning rate. Plugging $\\{V^{\prime}_{j}\\}_{j=1}^{K}$
into $\sum_{k=1}^{K}L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},$
$D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{*}_{j}(A_{j})))$, we obtain an
approximated objective $\sum_{k=1}^{K}O^{w}_{k}$ where
$O^{w}_{k}=L(W_{k},A_{k},$
$D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j}))$.
Then we approximate $W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq k}^{K})$
using one-step gradient descent update of $W_{k}$ with respect to $O^{w}_{k}$:

$\begin{array}[]{l}W^{\prime}_{k}=W_{k}-\xi_{w}\nabla_{W_{k}}(L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j}))).\end{array}$
(6)

Finally, we plug $\\{W^{\prime}_{k}\\}_{k=1}^{K}$ into
$\sum_{k=1}^{K}L(W^{*}_{k}(A_{k},\\{V^{*}_{j}(A_{j})\\}_{j\neq
k}^{K}),A_{k},D^{(\textrm{val})})$ and get $\sum_{k=1}^{K}L(W^{\prime}_{k},$
$A_{k},D^{(\textrm{val})})$. We can update the architecture $A_{k}$ of learner
$k$ by descending the gradient of
$\sum_{j=1}^{K}L(W^{\prime}_{j},A_{j},D^{(\textrm{val})})$ w.r.t $A_{k}$:

$\begin{array}[]{l}A_{k}\leftarrow
A_{k}-\eta_{A}(\nabla_{A_{k}}L(W^{\prime}_{k},A_{k},D^{(\textrm{val})})+\sum\limits_{j\neq
k}^{K}\nabla_{A_{k}}L(W^{\prime}_{j},A_{j},D^{(\textrm{val})})\end{array}$ (7)

where

$\begin{array}[]{l}\nabla_{A_{k}}L(W^{\prime}_{k},A_{k},D^{(\textrm{val})})=\\\
\nabla_{A_{k}}L(W_{k}-\xi_{w}\nabla_{W_{k}}(L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j}))),A_{k},D^{(\textrm{val})})=\\\
\nabla_{A_{k}}L(W^{\prime}_{k},A_{k},D^{(\textrm{val})})-\xi_{w}\nabla^{2}_{A_{k},W_{k}}(L(W_{k},A_{k},D^{(\textrm{tr})})+\\\
\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j})))\nabla_{W^{\prime}_{k}}L(W^{\prime}_{k},A_{k},D^{(\textrm{val})})\\\
\end{array}$ (8)

The matrix-vector product
$\nabla^{2}_{A_{k},W_{k}}(L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j})))$
is computationally expensive. We can reduce its computational complexity by a
finite difference approximation:

$\begin{array}[]{ll}\nabla^{2}_{A_{k},W_{k}}(L(W_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j})))\nabla_{W^{\prime}_{k}}L(W^{\prime}_{k},A_{k},D^{(\textrm{val})})\approx\\\
\frac{1}{2\alpha}(\nabla_{A_{k}}(L(W^{+}_{k},A_{k},D^{(\textrm{tr})})+\lambda\sum_{j\neq
k}^{K}L(W^{+}_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j})))-\nabla_{A_{k}}(L(W^{-}_{k},A_{k},D^{(\textrm{tr})})\\\
+\lambda\sum_{j\neq
k}^{K}L(W^{-}_{k},A_{k},D^{(\textrm{pl})}_{j}(D^{(\textrm{u})},V^{\prime}_{j})))),\end{array}$
(9)

where
$W^{\pm}_{k}=W_{k}\pm\alpha\nabla_{W^{\prime}_{k}}L(W^{\prime}_{k},A_{k},D^{(\textrm{val})})$
and $\alpha$ is a small scalar that equals
$0.01/\|\nabla_{W^{\prime}_{k}}L(W^{\prime}_{k},$
$A_{k},D^{(\textrm{val})})\|_{2}$.

For $\nabla_{A_{k}}L(W^{\prime}_{j},A_{j},D^{(\textrm{val})})(j\neq k)$ in
Eq.(7), it can be calculated as:

$\begin{array}[]{ll}\nabla_{A_{k}}L(W^{\prime}_{j},A_{j},D^{(\textrm{val})})=\\\
\frac{\partial W^{\prime}_{j}}{\partial
A_{k}}\nabla_{W^{\prime}_{j}}L(W^{\prime}_{j},A_{j},D^{(\textrm{val})})=\\\
-\xi_{w}\lambda\par\frac{\partial\nabla_{W_{j}}L(W_{j},A_{j},D^{(\textrm{pl})}_{k}(D^{(\textrm{u})},V^{\prime}_{k}))}{\partial
A_{k}}\par\par\nabla_{W^{\prime}_{j}}L(W^{\prime}_{j},A_{j},D^{(\textrm{val})})=\\\
-\xi_{w}\lambda\par\par\frac{\partial V^{\prime}_{k}}{\partial
A_{k}}\par\par\nabla^{2}_{V^{\prime}_{k},W_{j}}L(W_{j},A_{j},D^{(\textrm{pl})}_{k}(D^{(\textrm{u})},V^{\prime}_{k}))\par\nabla_{W^{\prime}_{j}}L(W^{\prime}_{j},A_{j},D^{(\textrm{val})})\\\
\end{array}$ (10)

where

$\displaystyle\frac{\partial V^{\prime}_{k}}{\partial A_{k}}$
$\displaystyle=\frac{\partial(V_{k}-\xi_{v}\nabla_{V_{k}}L(V_{k},A_{k},D^{(\mathrm{tr})}))}{\partial
A_{k}}=-\xi_{v}\nabla^{2}_{A_{k},V_{k}}L(V_{k},A_{k},D^{(\mathrm{tr})})$ (11)

The algorithm for solving SGL is summarized in Algorithm 1.

while  _not converged_ do

1\. For learner $1,\cdots,K$, update its weights $V_{k}$ using Eq.(5)
2\. For learner $1,\cdots,K$, update its weights $W_{k}$ using Eq.(6)
3\. For learner $1,\cdots,K$, update its architecture $A_{k}$ using Eq.(7)

end while

Algorithm 1 Optimization algorithm for small-group learning

##  3 Experiments

We apply SGL for neural architecture search in image classification tasks.
Following (Liu et al., 2019), we first perform architecture search which finds
out an optimal cell, then perform architecture evaluation which composes
multiple copies of the searched cell into a large network, trains it from
scratch, and evaluates the trained model on the test set.

###  3.1 Datasets

We used three datasets in the experiments: CIFAR-10, CIFAR-100, and ImageNet
(Deng et al., 2009). The CIFAR-10 dataset contains 50K training images and 10K
testing images, from 10 classes (the number of images in each class is equal).
Following (Liu et al., 2019), we split the original 50K training set into a
new 25K training set and a 25K validation set. In the sequel, when we mention
“training set”, it always refers to the new 25K training set. During
architecture search, the training set is used as $D^{\textrm{(tr)}}$ in SGL.
The validation set is used as $D^{\textrm{(val)}}$ in SGL. During architecture
evaluation, the combination of the training data and validation data is used
to train the large network stacking multiple copies of the searched cell. The
CIFAR-100 dataset contains 50K training images and 10K testing images, from
100 classes (the number of images in each class is equal). Similar to
CIFAR-100, the 50K training images are split into a 25K training set and 25K
validation set. The usage of the new training set and validation set is the
same as that for CIFAR-10. The ImageNet dataset contains a training set of
1.2M images and a validation set of 50K images, from 1000 object classes. The
validation set is used as a test set for architecture evaluation. Following
(Liu et al., 2019), we evaluate the architectures searched using CIFAR-10 and
CIFAR-100 on ImageNet: given a cell searched using CIFAR-10 and CIFAR-100,
multiple copies of it compose a large network, which is then trained on the
1.2M training data of ImageNet and evaluated on the 50K test data. In SGL,
when searching architectures on CIFAR-10, the input images (without labels) in
CIFAR-100 are used as the unlabeled dataset; vice versa.

###  3.2 Experimental Settings

Our framework is a general one that can be used together with any
differentiable search method. Specifically, we apply our framework to the
following NAS methods: 1) DARTS (Liu et al., 2019), 2) P-DARTS (Chen et al.,
2019), 3) PC-DARTS, and 4) DARTS- (Chu et al., 2020a). The search space in
these methods are similar. The candidate operations include: $3\times 3$ and
$5\times 5$ separable convolutions, $3\times 3$ and $5\times 5$ dilated
separable convolutions, $3\times 3$ max pooling, $3\times 3$ average pooling,
identity, and zero. In SGL, the network of each learner is a stack of multiple
cells, each consisting of 7 nodes. $\lambda$ is set to 0.1.

For CIFAR-10 and CIFAR-100, during architecture search, each learner’s network
is a stack of 8 cells, with the initial channel number set to 16. The search
is performed for 50 epochs, with a batch size of 64. The hyperparameters for
each learner’s architecture and weights are set in the same way as DARTS,
P-DARTS, PC-DARTS, and DARTS-. The network weights are optimized using SGD
with a momentum of 0.9 and a weight decay of 3e-4. The initial learning rate
is set to 0.025 with a cosine decay scheduler. The architecture variables are
optimized with the Adam (Kingma and Ba, 2014) optimizer with a learning rate
of 3e-4 and a weight decay of 1e-3. During architecture evaluation, 20 copies
of the searched cell are stacked to form each learner’s network, with the
initial channel number set to 36. The network is trained for 600 epochs with a
batch size of 96 (for both CIFAR-10 and CIFAR-100). The experiments are
performed on a single Tesla v100. For ImageNet, following (Liu et al., 2019),
we take the architecture searched on CIFAR-10 and evaluate it on ImageNet. We
stack 14 cells (searched on CIFAR-10) to form a large network and set the
initial channel number as 48. The network is trained for 250 epochs with a
batch size of 1024 on 8 Tesla v100s. Each experiment on SGL is repeated for
ten times with the random seed to be from 1 to 10. We report the mean and
standard deviation of results obtained from the 10 runs.

###  3.3 Results

Method | Error(%) | Param(M) | Cost
---|---|---|---
*ResNet (He et al., 2016) | 22.10 | 1.7 | -
DenseNet (Huang et al., 2017) | 17.18 | 25.6 | -
*PNAS (Liu et al., 2018a) | 19.53 | 3.2 | 150
ENAS (Pham et al., 2018) | 19.43 | 4.6 | 0.5
AmoebaNet (Real et al., 2019) | 18.93 | 3.1 | 3150
*DARTS-2nd (Liu et al., 2019) | 20.58$\pm$0.44 | 1.8 | 1.5
GDAS (Dong and Yang, 2019) | 18.38 | 3.4 | 0.2
R-DARTS (Zela et al., 2020) | 18.01$\pm$0.26 | - | 1.6
ΔDARTS+ (Liang et al., 2019) | 17.11$\pm$0.43 | 3.8 | 0.2
DropNAS (Hong et al., 2020) | 16.39 | 4.4 | 0.7
†DARTS-1st (Liu et al., 2019) | 20.52$\pm$0.31 | 1.8 | 0.4
SGL-DARTS-1st (ours) | 18.54$\pm$0.21 | 2.2 | 1.2
*DARTS- (Chu et al., 2020a) | 17.51$\pm$0.25 | 3.3 | 0.4
†DARTS- (Chu et al., 2020a) | 18.97$\pm$0.16 | 3.1 | 0.4
SGL-DARTS- (ours) | 16.90$\pm$0.10 | 3.5 | 2.0
*P-DARTS (Chen et al., 2019) | 17.49 | 3.6 | 0.3
SGL-P-DARTS (ours) | 16.58$\pm$0.18 | 3.6 | 2.1
†PC-DARTS (Xu et al., 2020) | 17.01$\pm$0.06 | 4.0 | 0.1
SGL-PC-DARTS (ours) |  16.34$\pm$0.11 | 4.1 | 0.5
Table 2: Results on CIFAR-100, including classification error (%) on the test set, number of parameters (millions) in the searched architecture, and search cost (GPU days). SGL-DARTS-1st denotes that our method SGL is applied to the search space of DARTS. Similar meanings hold for other notations in such a format. DARTS-1st and DARTS-2nd denotes that first order and second order approximation is used in DARTS. * means the results are taken from DARTS- (Chu et al., 2020a). ${\dagger}$ means we re-ran this method for 10 times. The search cost is measured by GPU days on a Tesla v100.  Method | Error(%) | Param(M) | Cost
---|---|---|---
*DenseNet (Huang et al., 2017) | 3.46 | 25.6 | -
*HierEvol (Liu et al., 2018b) | 3.75$\pm$0.12 | 15.7 | 300
NAONet-WS (Luo et al., 2018) | 3.53 | 3.1 | 0.4
PNAS (Liu et al., 2018a) | 3.41$\pm$0.09 | 3.2 | 225
ENAS (Pham et al., 2018) | 2.89 | 4.6 | 0.5
NASNet-A (Zoph et al., 2018) | 2.65 | 3.3 | 1800
AmoebaNet-B (Real et al., 2019) | 2.55$\pm$0.05 | 2.8 | 3150
*R-DARTS (Zela et al., 2020) | 2.95$\pm$0.21 | - | 1.6
GDAS (Dong and Yang, 2019) | 2.93 | 3.4 | 0.2
SNAS (Xie et al., 2019) | 2.85 | 2.8 | 1.5
ΔDARTS+ (Liang et al., 2019) | 2.83$\pm$0.05 | 3.7 | 0.4
BayesNAS (Zhou et al., 2019) | 2.81$\pm$0.04 | 3.4 | 0.2
DARTS-2nd (Liu et al., 2019) | 2.76$\pm$0.09 | 3.3 | 1.5
MergeNAS (Wang et al., 2020) | 2.73$\pm$0.02 | 2.9 | 0.2
NoisyDARTS (Chu et al., 2020b) | 2.70$\pm$0.23 | 3.3 | 0.4
ASAP (Noy et al., 2020) | 2.68$\pm$0.11 | 2.5 | 0.2
SDARTS (Chen and Hsieh, 2020) | 2.61$\pm$0.02 | 3.3 | 1.3
DropNAS (Hong et al., 2020) | 2.58$\pm$0.14 | 4.1 | 0.6
FairDARTS (Chu et al., 2019) | 2.54 | 3.3 | 0.4
DrNAS (Chen et al., 2020) | 2.54$\pm$0.03 | 4.0 | 0.4
*DARTS-1st (Liu et al., 2019) | 3.00$\pm$0.14 | 3.3 | 0.4
SGL-DARTS-1st (ours) |  2.41$\pm$0.06 | 3.7 | 1.2
*DARTS- (Chu et al., 2020a) | 2.59$\pm$0.08 | 3.5 | 0.4
†DARTS- (Chu et al., 2020a) | 2.97$\pm$0.04 | 3.3 | 0.6
SGL-DARTS- (ours) | 2.60$\pm$0.07 | 3.1 | 2.0
*P-DARTS (Chen et al., 2019) | 2.50 | 3.4 | 0.3
SGL-P-DARTS (ours) | 2.47$\pm$0.10 | 3.6 | 2.1
*PC-DARTS (Xu et al., 2020) | 2.57$\pm$0.07 | 3.6 | 0.1
SGL-PC-DARTS (ours) | 2.60$\pm$0.12 | 3.5 | 0.5
Table 3:  Results on CIFAR-10. * means the results are taken from DARTS- (Chu et al., 2020a), NoisyDARTS (Chu et al., 2020b), and DrNAS (Chen et al., 2020). The rest notations are the same as those in Table 2.  Method | Top-1 | Top-5 | Param | Cost
---|---|---|---|---
Error (%) | Error (%) | (M) | (GPU days)
*Inception-v1 (Szegedy et al., 2015) | 30.2 | 10.1 | 6.6 | -
MobileNet (Howard et al., 2017) | 29.4 | 10.5 | 4.2 | -
ShuffleNet 2$\times$ (v1) (Zhang et al., 2018) | 26.4 | 10.2 | 5.4 | -
ShuffleNet 2$\times$ (v2) (Ma et al., 2018) | 25.1 | 7.6 | 7.4 | -
*NASNet-A (Zoph et al., 2018) | 26.0 | 8.4 | 5.3 | 1800
PNAS (Liu et al., 2018a) | 25.8 | 8.1 | 5.1 | 225
MnasNet-92 (Tan et al., 2019) | 25.2 | 8.0 | 4.4 | 1667
AmoebaNet-C (Real et al., 2019) | 24.3 | 7.6 | 6.4 | 3150
*SNAS-CIFAR10 (Xie et al., 2019) | 27.3 | 9.2 | 4.3 | 1.5
BayesNAS-CIFAR10 (Zhou et al., 2019) | 26.5 | 8.9 | 3.9 | 0.2
PARSEC-CIFAR10 (Casale et al., 2019) | 26.0 | 8.4 | 5.6 | 1.0
GDAS-CIFAR10 (Dong and Yang, 2019) | 26.0 | 8.5 | 5.3 | 0.2
DSNAS-ImageNet (Hu et al., 2020) | 25.7 | 8.1 | - | -
SDARTS-ADV-CIFAR10 (Chen and Hsieh, 2020) | 25.2 | 7.8 | 5.4 | 1.3
PC-DARTS-CIFAR10 (Xu et al., 2020) | 25.1 | 7.8 | 5.3 | 0.1
ProxylessNAS-ImageNet (Cai et al., 2019) | 24.9 | 7.5 | 7.1 | 8.3
FairDARTS-CIFAR10 (Chu et al., 2019) | 24.9 | 7.5 | 4.8 | 0.4
FairDARTS-ImageNet (Chu et al., 2019) | 24.4 | 7.4 | 4.3 | 3.0
DrNAS-ImageNet (Chen et al., 2020) | 24.2 | 7.3 | 5.2 | 3.9
DARTS+-ImageNet (Liang et al., 2019) | 23.9 | 7.4 | 5.1 | 6.8
DARTS--ImageNet (Chu et al., 2020a) | 23.8 | 7.0 | 4.9 | 4.5
DARTS+-CIFAR100 (Liang et al., 2019) | 23.7 | 7.2 | 5.1 | 0.2
*DARTS-2nd-CIFAR10 (Liu et al., 2019) | 26.7 | 8.7 | 4.7 | 4.0
SGL-DARTS-1st-CIFAR10 (ours) | 24.9 | 7.7 | 5.2 | 4.0
*P-DARTS-CIFAR10 (Chen et al., 2019) | 24.4 | 7.4 | 4.9 | 0.3
SGL-P-DARTS-CIFAR10 (ours) | 24.3 | 7.2 | 5.1 | 0.3
*P-DARTS-CIFAR100 (Chen et al., 2019) | 24.7 | 7.5 | 5.1 | 0.3
SGL-P-DARTS-CIFAR100 (ours) | 23.9 | 7.2 | 5.3 | 0.3
Table 4: Results on ImageNet, including top-1 and top-5 classification errors
on the test set, number of weight parameters (millions), and search cost (GPU
days). * means the results are taken from DARTS- (Chu et al., 2020a) and DrNAS
(Chen et al., 2020). The rest notations are the same as those in Table 2. The
first row block shows networks designed by humans manually. The second row
block shows non-gradient based search methods. The third block shows gradient-
based methods.

Table 2 shows the classification error (%), number of weight parameters
(millions), and search cost (GPU days) of different NAS methods on CIFAR-100.
From this table, we make the following observations. First, when our method
SGL is applied to different NAS baselines including DARTS-1st (first order
approximation), DARTS-, P-DARTS, and PC-DARTS, the classification errors of
these baselines can be significantly reduced. For example, applying our method
to DARTS-1st, the error reduces from 20.52% to 18.54%. Applying our method to
P-DARTS, the error reduces from 17.49% to 16.58%. This demonstrates the
effectiveness of our method in searching for a better architecture. In our
method, learners with different architectures collaboratively solve the same
task. They provide pseudo-labeled datasets to each other to help each other
train better models. In single-learner NAS, there is a risk that the searched
architecture gets stuck in a local optimal. Group-learning can significantly
reduce this risk. If one learner is stuck in a local optimal, other learners
can help it escape the local optimal via knowledge transfer based on pseudo-
labeling. Second, our method SGL-PC-DARTS achieves the best performance among
all methods, which further demonstrates the effectiveness of SGL in driving
the frontiers of neural architecture search forward. Third, the number of
weight parameters and search costs corresponding to our methods are on par
with those in differentiable NAS baselines. This shows that SGL is able to
search better-performing architectures without significantly increasing
network size and search cost. A few additional remarks: 1) On CIFAR-100,
DARTS-2nd with second-order approximation in the optimization algorithm is not
advantageous compared with DARTS-1st which uses first-order approximation; 2)
In our run of DARTS-, the performance reported in (Chu et al., 2020a) cannot
be achieved.

Table 3 shows the classification error (%), number of weight parameters
(millions), and search cost (GPU days) of different NAS methods on CIFAR-10.
As can be seen, applying our proposed SGL method to DARTS-1st, †DARTS- (our
re-run), and P-DARTS reduces the errors of these baselines. Especially on
DARTS-1st, with the usage of SGL, the error is reduced from 3.00% to 2.41%,
which is the best among all methods in this table. This further demonstrates
the efficacy of our method in searching better-performing architectures, by
letting a group of learners mutually help each other to train better models.

Table 4 shows the results on ImageNet, including top-1 and top-5
classification errors on the test set, number of weight parameters (millions),
and search costs (GPU days). Following (Liu et al., 2019), we take the
architectures searched by SGL-DARTS-1st on CIFAR-10, by SGL-P-DARTS on
CIFAR-10, by SGL-P-DARTS on CIFAR-100, and evaluate them on ImageNet. As can
be seen, applying our SGL method to DARTS, P-DARTS, and PC-DARTS, the top-1
and top-5 errors are both reduced significantly. This further demonstrates the
effectiveness of our method.

##  4 Related Works

Neural architecture search (NAS) has achieved remarkable progress recently,
which aims at searching for the optimal architecture of neural networks to
achieve the best predictive performance. In general, there are three paradigms
of methods in NAS: reinforcement learning (RL) approaches (Zoph and Le, 2017;
Pham et al., 2018; Zoph et al., 2018), evolutionary learning approaches (Liu
et al., 2018b; Real et al., 2019), and differentiable approaches (Cai et al.,
2019; Liu et al., 2019; Xie et al., 2019). In RL-based approaches, a policy is
learned to iteratively generate new architectures by maximizing a reward which
is the accuracy on the validation set. Evolutionary learning approaches
represent the architectures as individuals in a population. Individuals with
high fitness scores (validation accuracy) have the privilege to generate
offspring, which replaces individuals with low fitness scores. Differentiable
approaches adopt a network pruning strategy. On top of an over-parameterized
network, the weights of connections between nodes are learned using gradient
descent. Then weights close to zero are later pruned. There have been many
efforts devoted to improving differentiable NAS methods. In P-DARTS (Chen et
al., 2019), the depth of searched architectures is allowed to grow
progressively during the training process. Search space approximation and
regularization approaches are developed to reduce computational overheads and
improve search stability. PC-DARTS (Xu et al., 2020) reduces the redundancy in
exploring the search space by sampling a small portion of a super network.
Operation search is performed in a subset of channels with the held out part
bypassed in a shortcut. Our proposed SGL framework can be applied to any
differentiable NAS methods.

##  5 Conclusions

In this paper, we propose a new machine learning approach – small-group
learning (SGL), inspired by the small-group cooperative learning technique of
humans. In SGL, a set of learners help each other to learn better: each
learner uses its intermediately trained model to make predictions on unlabeled
data examples and generates a pseudo-labeled dataset; meanwhile, each learner
uses pseudo-labeled datasets generated by other learners to retrain and
improve its model. We propose a multi-level optimization framework to
formalize SGL which involves three learning stages: each learner trains an
initial model separately; all learners retrain their models collaboratively by
mutually performing pseudo-labeling; all learners improve their neural
architectures based on the feedback obtained during model validation. Our
framework is applied for neural architecture search and achieves significant
improvement on CIFAR-100, CIFAR-10, and ImageNet.

## References

  * Cai et al. (2019) Han Cai, Ligeng Zhu, and Song Han.  Proxylessnas: Direct neural architecture search on target task and hardware.  In _ICLR_ , 2019.
  * Casale et al. (2019) Francesco Paolo Casale, Jonathan Gordon, and Nicoló Fusi.  Probabilistic neural architecture search.  _CoRR_ , abs/1902.05116, 2019.
  * Chen and Hsieh (2020) Xiangning Chen and Cho-Jui Hsieh.  Stabilizing differentiable architecture search via perturbation-based regularization.  _CoRR_ , abs/2002.05283, 2020.
  * Chen et al. (2020) Xiangning Chen, Ruochen Wang, Minhao Cheng, Xiaocheng Tang, and Cho-Jui Hsieh.  Drnas: Dirichlet neural architecture search.  _CoRR_ , abs/2006.10355, 2020.
  * Chen et al. (2019) Xin Chen, Lingxi Xie, Jun Wu, and Qi Tian.  Progressive differentiable architecture search: Bridging the depth gap between search and evaluation.  In _ICCV_ , 2019.
  * Chu et al. (2019) Xiangxiang Chu, Tianbao Zhou, Bo Zhang, and Jixiang Li.  Fair DARTS: eliminating unfair advantages in differentiable architecture search.  _CoRR_ , abs/1911.12126, 2019.
  * Chu et al. (2020a) Xiangxiang Chu, Xiaoxing Wang, Bo Zhang, Shun Lu, Xiaolin Wei, and Junchi Yan.  DARTS-: robustly stepping out of performance collapse without indicators.  _CoRR_ , abs/2009.01027, 2020a.
  * Chu et al. (2020b) Xiangxiang Chu, Bo Zhang, and Xudong Li.  Noisy differentiable architecture search.  _CoRR_ , abs/2005.03566, 2020b.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Dong and Yang (2019) Xuanyi Dong and Yi Yang.  Searching for a robust neural architecture in four GPU hours.  In _CVPR_ , 2019.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _CVPR_ , 2016.
  * Hong et al. (2020) Weijun Hong, Guilin Li, Weinan Zhang, Ruiming Tang, Yunhe Wang, Zhenguo Li, and Yong Yu.  Dropnas: Grouped operation dropout for differentiable architecture search.  In _IJCAI_ , 2020.
  * Howard et al. (2017) Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  _CoRR_ , abs/1704.04861, 2017.
  * Hu et al. (2020) Shoukang Hu, Sirui Xie, Hehui Zheng, Chunxiao Liu, Jianping Shi, Xunying Liu, and Dahua Lin.  DSNAS: direct neural architecture search without parameter retraining.  In _CVPR_ , 2020.
  * Huang et al. (2017) Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger.  Densely connected convolutional networks.  In _CVPR_ , 2017.
  * Kingma and Ba (2014) Diederik Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _International Conference on Learning Representations_ , 12 2014.
  * Liang et al. (2019) Hanwen Liang, Shifeng Zhang, Jiacheng Sun, Xingqiu He, Weiran Huang, Kechen Zhuang, and Zhenguo Li.  DARTS+: improved differentiable architecture search with early stopping.  _CoRR_ , abs/1909.06035, 2019.
  * Liu et al. (2018a) Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan L. Yuille, Jonathan Huang, and Kevin Murphy.  Progressive neural architecture search.  In _ECCV_ , 2018a.
  * Liu et al. (2018b) Hanxiao Liu, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, and Koray Kavukcuoglu.  Hierarchical representations for efficient architecture search.  In _ICLR_ , 2018b.
  * Liu et al. (2019) Hanxiao Liu, Karen Simonyan, and Yiming Yang.  DARTS: differentiable architecture search.  In _ICLR_ , 2019.
  * Luo et al. (2018) Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu.  Neural architecture optimization.  In _NeurIPS_ , 2018.
  * Ma et al. (2018) Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun.  Shufflenet V2: practical guidelines for efficient CNN architecture design.  In _ECCV_ , 2018.
  * Noy et al. (2020) Asaf Noy, Niv Nayman, Tal Ridnik, Nadav Zamir, Sivan Doveh, Itamar Friedman, Raja Giryes, and Lihi Zelnik.  ASAP: architecture search, anneal and prune.  In _AISTATS_ , 2020.
  * Pham et al. (2018) Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc V. Le, and Jeff Dean.  Efficient neural architecture search via parameter sharing.  In _ICML_ , 2018.
  * Real et al. (2019) Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le.  Regularized evolution for image classifier architecture search.  In _Proceedings of the aaai conference on artificial intelligence_ , volume 33, pages 4780–4789, 2019.
  * Szegedy et al. (2015) Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In _CVPR_ , 2015.
  * Tan et al. (2019) Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V. Le.  Mnasnet: Platform-aware neural architecture search for mobile.  In _CVPR_ , 2019.
  * Wang et al. (2020) Xiaoxing Wang, Chao Xue, Junchi Yan, Xiaokang Yang, Yonggang Hu, and Kewei Sun.  Mergenas: Merge operations into one for differentiable architecture search.  In _IJCAI_ , 2020.
  * Xie et al. (2019) Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin.  SNAS: stochastic neural architecture search.  In _ICLR_ , 2019.
  * Xu et al. (2020) Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong.  PC-DARTS: partial channel connections for memory-efficient architecture search.  In _ICLR_ , 2020.
  * Zela et al. (2020) Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hutter.  Understanding and robustifying differentiable architecture search.  In _ICLR_ , 2020.
  * Zhang et al. (2018) Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun.  Shufflenet: An extremely efficient convolutional neural network for mobile devices.  In _CVPR_ , 2018.
  * Zhou et al. (2019) Hongpeng Zhou, Minghao Yang, Jun Wang, and Wei Pan.  Bayesnas: A bayesian approach for neural architecture search.  In _ICML_ , 2019.
  * Zoph and Le (2017) Barret Zoph and Quoc V. Le.  Neural architecture search with reinforcement learning.  In _ICLR_ , 2017.
  * Zoph et al. (2018) Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le.  Learning transferable architectures for scalable image recognition.  In _CVPR_ , 2018.