singularly umbilic at ${\bm{p}}$ of umbilic type $(\alpha,\beta,\gamma)$ and
order $n_{0}=\alpha+\beta+\gamma>6$. Then, there exists a holomorphic
transformation mapping $M$ to the convergent normal form

$v=z\bar{z}+\sum_{\scriptstyle j+k+\ell\geq n_{0}\atop{\scriptstyle j\geq 2,\
k\geq 4,\ \ell\geq 0,\hbox{\vrule
height=8.0pt,depth=4.0pt,width=0.0pt}\atop\scriptstyle{\rm or}\ j\geq 4,\
k\geq 2,\ \ell\geq
0}}\frac{1}{j!\,k!\,\ell!}V_{Z^{j}\overline{Z}{}^{k}U^{\ell}}z^{j}\bar{z}^{k}u^{\ell}$
(4.30)

with

$\displaystyle
V_{Z^{\alpha}\overline{Z}{}^{\beta}U^{\gamma}}=V_{Z^{\beta}\overline{Z}{}^{\alpha}U^{\gamma}}=\sigma,\qquad
V_{Z^{\alpha-1}\overline{Z}{}^{\beta}U^{\gamma+1}}=V_{Z^{\beta}\overline{Z}{}^{\alpha-1}U^{\gamma+1}}={\rm
Re}\>V_{Z^{\alpha}\overline{Z}{}^{\beta}U^{\gamma+1}}=0,$

where $\sigma$ is a nonzero real constant satisfying (4.29). Moreover, the
holomorphic isotropy group of $M$ at ${\bm{p}}$ is at most discrete.

######  Remark 4.14.

Although all terms of order $6\leq n<n_{0}$ in the singularly umbilic normal
form vanish, this only implies that the corresponding differential invariants,
including the Cartan curvature, vanish at the point ${\bm{p}}$, and does not
imply that they vanish in a neighborhood thereof.

###  4.6. Semi-circular points

Suppose ${\bm{p}}$ is a singularly umbilic point of umbilic type
$(\alpha,\alpha,\gamma)$ and semi-circular type $(\delta,\kappa,\lambda)$. In
this case, only the real form ${\rm Re}\>\mu_{Z}$ is normalizable through the
recurrence relations of order $n_{0}$. However, setting ${\rm
Im}\>V_{Z^{\delta}\overline{Z}{}^{\kappa}U^{\lambda}}=\tau$ for suitable
$0\neq\tau\in\mathbb{R}$, the resulting phantom recurrence formula allows us
to normalize ${\rm Im}\>\mu_{Z}$. The remainder of the analysis proceeds in
the same fashion as at a generic singularly umbilic point, and the details are
left to the reader. Here we just state the final result.

######  Theorem 4.15.

Let $M\subset\mathbb{C}^{2}$ be a nondegenerate hypersurface that is
singularly umbilic at ${\bm{p}}$ of umbilic type $(\alpha,\alpha,\gamma)$,
semi-circular type $(\delta,\kappa,\lambda)$ and order
$n_{0}=2\alpha+\gamma\geq 8$. Then, there exists a holomorphic transformation
mapping $M$ to the convergent normal form (4.30) with

$\displaystyle V_{Z^{\alpha}\overline{Z}{}^{\alpha}U^{\gamma}}=\sigma,\quad
V_{Z^{\alpha-1}\overline{Z}{}^{\beta}U^{\gamma+1}}=V_{Z^{\beta}\overline{Z}{}^{\alpha-1}U^{\gamma+1}}={\rm
Re}\>V_{Z^{\alpha}\overline{Z}{}^{\beta}U^{\gamma+1}}=0,\quad{\rm
Im}\>V_{Z^{\lambda}\overline{Z}{}^{\mu}U^{\nu}}=\tau,$ (4.31)

where $\sigma,\tau$ are nonzero real constants. Moreover, the holomorphic
isotropy group of $M$ at ${\bm{p}}$ is at most discrete.

###  4.7. Circular points

Finally, let us consider the normal form of a circular singularly umbilic
hypersurface, of umbilic type $(\alpha,\alpha,\gamma)$ with $\alpha\geq 4$,
and order $n_{0}=2\alpha+\gamma\geq 8$.

Looking ab initio at the normalizations made in the generic case, one finds
that it is possible to normalize the four Maurer–Cartan forms ${\rm
Re}\>\mu_{Z},\ {\rm Re}\>\mu_{ZU},\ \mu_{U},\ \overline{\mu}_{U}$, by setting

$V_{Z^{\alpha}\overline{Z}{}^{\alpha}U^{\gamma}}=1\qquad{\rm and}\qquad
V_{Z^{\alpha}\overline{Z}{}^{\alpha}U^{\gamma+1}}=V_{Z^{\alpha-1}\overline{Z}{}^{\alpha}U^{\gamma+1}}=V_{Z^{\alpha}\overline{Z}{}^{\alpha-1}U^{\gamma+1}}=0.$

However, the residual rotational isotropy of the circular normal form means
that the remaining Maurer–Cartan form ${\rm Im}\>\mu_{Z}$ cannot be
normalized, [32].

######  Theorem 4.16.

Let $M$ be a circular hypersurface of $\mathbb{C}^{2}$ singularly umbilic at
${\bm{p}}$ of umbilic type $(\alpha,\alpha,\gamma)$ and order
$n_{0}=2\alpha+\gamma\geq 8$. Then there exists a holomorphic transformation
mapping $M$ to the convergent normal form

$v=z\bar{z}+\frac{1}{(\alpha!)^{2}\,\gamma!}\,z^{\alpha}\bar{z}^{\alpha}u^{\gamma}+\sum_{2k+\ell\geq
n_{0}\atop{k\geq\alpha+1}}\,\frac{1}{(k!)^{2}\ell!}\,V_{Z^{k}\overline{Z}{}^{k}U^{\ell}}\,(z\bar{z})^{k}u^{\ell}$
(4.32)

with $V_{Z^{\alpha}\overline{Z}{}^{\alpha}U^{\gamma+1}}=0$. The holomorphic
isotropy group of $M$ at ${\bm{p}}$ is one-dimensional and, when mapped to the
origin, is generated by the rotational vector field $\,{\rm
i}\,z\,\partial_{z}$. Moreover, the circular normal form (4.32) is unique.

###  4.8. Symmetry and Isotropy

Before concluding this paper, let us present a classification of nondegenerate
hypersurfaces of $\mathbb{C}^{2}$ according to their (isotropy) group of
holomorphic automorphisms. This classification follows immediately from the
above considerations and can be deduced, in part, from [3].

######  Theorem 4.17.

Let $M$ be a hypersurface of $\mathbb{C}^{2}$ nondegenerate at a point
${\bm{p}}$. Let $\mathfrak{G}_{\bm{p}}$ be the holomorphic isotropy group at
${\bm{p}}$. Then one of the following possibilities will occur

  * $\bullet$

${\rm dim}\;\mathfrak{G}_{\bm{p}}=5$ then $M$ is locally holomorphically
equivalent to the Heisenberg sphere $\mathbb{H}$.

  * $\bullet$

${\rm dim}\;\mathfrak{G}_{\bm{p}}=1$ then $M$ is a circular hypersurface
singularly umbilic at ${\bm{p}}$.

  * $\bullet$

${\rm dim}\;\mathfrak{G}_{\bm{p}}=0$ then $M$ is non-umbilic or semi-circular
or generic singularly umbilic at ${\bm{p}}$.

Similarly, using the argument outlined in the introduction, we deduce the
following result on the local symmetry group of a nondegenerate hypersurface.

######  Theorem 4.18.

Let $M$ be a nondegenerate hypersurface. Then the local symmetry group of $M$
near a point ${\bm{p}}$ has dimension $8$ if and only if $M$ is locally
holomorphically equivalent to the Heisenberg sphere $\mathbb{H}$. Otherwise,
the dimension of the local symmetry group is $0,1,2,3$ depending upon whether
the number of functionally independent differential invariants on a
neighborhood of ${\bm{p}}$ is $3,2,1,0$, respectively. In the maximally
symmetric nonsingular case having a three-dimensional local symmetry group,
$M$ is locally equivalent to an orbit of a three-dimensional subgroup of the
holomorphic pseudo-group.

###### Proof.

If $M$ is totally singular, then it must be umbilic and hence locally
congruent to the Heisenberg sphere. Otherwise, the dimension of the symmetry
group cannot be $\geq 4$, since, given that the dimension of $M$ is $3$, this
would imply that every point ${\bm{p}}\in M$ would have an isotropy group of
dimension $\geq 1$, and hence be umbilic, and we are back to the previous
case. The connection between the dimension of the symmetry group and the
number of functionally independent differential invariants at a nonsingular
point follows from [8, Theorem 5.17]; see also [17, Theorem 14.26]. ∎

The eight-dimensional case corresponds to the Heisenberg (and ordinary)
sphere. Cartan, [5], classified the homogeneous hypersurfaces possessing a
three-dimensional symmetry group. According to [4, p. 656], there are six
classes; the _tubular hypersurfaces_

  * (t1)

$v=y^{\lambda}$ for $y>0$, $|\,{\lambda}\,|\geq 1$, $\lambda\neq 1,2$;

  * (t2)

$v=y\log y$ for $y>0$;

  * (t3)

$r=e^{a\,\varphi}$ for $a\geq 0$, where $y+\,{\rm i}\,v=r\,e^{\,{\rm
i}\,\varphi}$;

and the _projective hypersurfaces_

  * (p1)

$\hphantom{-}1+|\,{z}\,|^{2}+|\,{w}\,|^{2}=a\,|\,{1+z^{2}+w^{2}}\,|\hphantom{-}$
for $a>1$;

  * (p2)

$\hphantom{-}1+|\,{z}\,|^{2}-|\,{w}\,|^{2}=a\,|\,{1+z^{2}-w^{2}}\,|\hphantom{-}$
for $a>1$;

  * (p3)

$-1+|\,{z}\,|^{2}+|\,{w}\,|^{2}=a\,|\,{-1+z^{2}+w^{2}}\,|$ for
$0<|\,{a}\,|<1$.

We note that the tubular hypersurface of Type (t3) can be rewritten in the
alternative implicit form

$(y+\,{\rm i}\,v)^{1-\,{\rm i}\,a}(y-\,{\rm i}\,v)^{1+\,{\rm i}\,a}=1.$

The associated symmetry groups are as follows. The tubular hypersurfaces all
have nilpotent symmetry groups. For the first, the symmetry algebra is spanned
by

${\rm(t1)}\hskip 40.0pt\partial_{z},\qquad\partial_{w},\qquad
z\,\partial_{z}+\lambda\,w\,\partial_{w}.\hskip 100.0pt$

The second tubular hypersurface has symmetry algebra

${\rm(t2)}\hskip 40.0pt\partial_{z},\qquad\partial_{w},\qquad
z\,\partial_{z}+(z+w)\,\partial_{w}.\hskip 80.0pt$

As for the third, we have

${\rm(t3)}\hskip
40.0pt\partial_{z},\qquad\partial_{w},\qquad(a\,z+w)\,\partial_{z}+(-z+a\,w)\,\partial_{w}.\hskip
28.0pt$

The symmetry algebra of the first class of projective hypersurfaces is spanned
by

${\rm(p1)}\hskip
22.0pt-w\,\partial_{z}+z\,\partial_{w},\qquad(1+z^{2})\,\partial_{z}+z\,w\,\partial_{w},\qquad
z\,w\,\partial_{z}+(1+w^{2})\,\partial_{w}.\hskip 56.0pt$

We note that these vector fields span the complexification of the
infinitesimal generators of the projective action of the rotation group ${\rm
SO}(3)$ on the projective plane $\mathbb{RP}^{2}$. The other two projective
hypersurfaces have similar symmetry generators

${{\rm(p2)}\hskip 35.0ptw\,\partial_{z}+z\,\partial_{w},\qquad\ \
(1-z^{2})\,\partial_{z}-z\,w\,\partial_{w},\qquad\
-\,z\,w\,\partial_{z}+(1-w^{2})\,\partial_{w},\hskip 4.0pt\atop{\rm(p3)}\hskip
22.0pt-w\,\partial_{z}+z\,\partial_{w},\qquad(-1+z^{2})\,\partial_{z}+z\,w\,\partial_{w},\qquad\
\ z\,w\,\partial_{z}+(-1+w^{2})\,\partial_{w}.}\hskip 30.0pt$

Because the tubular and projective hypersurfaces admit three-dimensional
symmetry groups, the differential invariants
${\mathfrak{J}},{\mathfrak{K}},{\mathfrak{L}}$ (and all higher order
differential invariants) are all constant on them.

Turning to two-dimensional symmetry groups, away from singular points there
are, up to holomorphic equivalence, two locally transitive possibilities,
[17]. The first is the abelian translation group whose Lie algebra is spanned
by $\partial_{z},\partial_{w}$. A translationally invariant hypersurface has
the form

$v=f(y).$ (4.33)

We note that the Heisenberg sphere $v=z\bar{z}$ is locally of the form (4.33);
replacing $z\mapsto e^{\,{\rm i}\,z}$, it becomes $v=e^{\,{\rm
i}\,(z-\bar{z})}=e^{-2y}$. We claim that, for a suitably generic function $f$,
the dimension of the symmetry group is exactly $2$. To see this, excluding the
Heisenberg sphere, we need to show that a generic $f$ cannot admit a three-
dimensional symmetry algebra. A short computation shows that any Lie algebra
containing the abelian translation algebra has a third generator of one of the
two forms

$e^{\alpha\,z+\beta\,w}(\gamma\,\partial_{z}+\delta\,\partial_{w}),\qquad(\alpha\,z+\beta\,w)\,\partial_{z}+(\gamma\,z+\delta\,w)\,\partial_{w},$

for $\alpha,\beta,\gamma,\delta\in\mathbb{C}$.

In the first case, we have $(\alpha,\beta),(\gamma,\delta)\neq(0,0)$;
moreover, since the symmetry algebra of the hypersurface must be real,
$\alpha,\beta\in\mathbb{R}$. The hypersurface (4.33) admits this generator if
and only if $f(y)$ satisfies

$f^{\prime}(y)=-\,\frac{{\rm Im}\>\delta\,e^{\,{\rm
i}\,(\alpha\,y+\beta\,f(y))}}{{\rm Im}\>\gamma\,e^{\,{\rm
i}\,(\alpha\,y+\beta\,f(y))}},$

which imposes severe restrictions on the form of $f(y)$. In the second case,
we can apply an invertible linear transformation with a real191919The
transformation must be real so as not to complexify the translation
generators. coefficient matrix to $(z,w)$ to place the coefficient matrix of
the vector field into real canonical form, [10, §3.4.1], and hence there are
three subcases

$a\,z\,\partial_{z}+b\,w\,\partial_{w},\qquad
a\,z\,\partial_{z}+(z+a\,w)\,\partial_{w},\qquad(a\,z+w)\,\partial_{z}+(-z+a\,w)\,\partial_{w},$

where, by the reality of the symmetry algebra, $a,b\in\mathbb{R}$. It is
easily shown that the corresponding hypersurfaces are, respectively,
equivalent under a linear transformation in $w$ to Cartan’s three tubular
hypersurfaces (in the first two cases, there is no invariant hypersurface of
the form (4.33) when $a=0$).

Any locally transitive non-abelian two-dimensional transformation group is
equivalent to the one whose Lie algebra is spanned by
$\partial_{z},e^{z}\partial_{w}$. In this case, the hypersurface equation
takes the form

$v=-u\tan y+f(y).$ (4.34)

Using a similar argument as above, only very particular functions $f(y)$ will
admit a third symmetry generator. We conclude that, in all cases, a
hypersurface corresponding to a sufficiently generic $f(y)$ in (4.34) has only
a two-dimensional symmetry group.

As for one-dimensional symmetry groups, one can, away from singular points,
map the infinitesimal generator to $\partial_{u}$, which corresponds to
translations in $u$. The corresponding hypersurfaces are of the form
$v=f(z,\bar{z})$, where $f$ is any real analytic function that does not depend
on $u$. One can clearly choose $f$ so that the hypersurface is nondegenerate
and non-umbilic. If the hypersurface were to admit a two-dimensional symmetry
group, one could apply a holomorphic transformation to map it into the form
(4.33) or (4.34), and hence a hypersurface defined by a generic $f(z,\bar{z})$
only admits a one-dimensional symmetry group.

### Acknowledgments

The authors would like to thank Howard Jacobowitz for his helpful comments and
discussions during the preparation of this paper. We also thank Valerii
Beloshapka for introducing us to [3] and helpful discussions. We further thank
Niky Kamran for reading a draft version and sending many helpful comments. The
research of the second author was supported in part by a grant from IPM, No.
1400510415.

## References

  * [1] Ö. Arnaldsson, Involutive moving frames, Diff. Geom. Appl. 69 (2020), 101603.
  * [2] M.S. Baouendi, P. Ebenfelt, and L.R. Rothschild, CR automorphisms of real analytic manifolds in complex space, Comm. Anal. Geom. 6(2) (1998), 291–315.
  * [3] V.K. Beloshapka, On the dimension of the group of automorphisms of an analytic hypersurface, Math. USSR Izv. 14(2) (1980), 223–245.
  * [4] V.K. Beloshapka, and I.G. Kossovskiy, Classification of homogeneous CR-manifolds in dimension 4, J. Math. Anal. Appl. 374 (2011), 655–672.
  * [5] É. Cartan, Sur la géométrie pseudo-conforme des hypersurfaces de l’espace de deux variables complexes I, Ann. Mat. Pura Appl. 11 (1933), 17–90 and II, Ann. Scuola Norm. Sup. Pisa 1(4) (1932), 333–354.
  * [6] É. Cartan, _La Méthode du Repère Mobile, la Théorie des Groupes Continus, et les Espaces Généralisés_ , Exposés de Géométrie, no. 5, Hermann, Paris, 1935.
  * [7] S.S. Chern and J.K. Moser, Real hypersurfaces in complex spaces, Acta Math. 133 (1974), 219–271.
  * [8] M. Fels and P.J. Olver, Moving coframes: II. regularization and theoretical foundations, Acta Appl. Math. 55 (1999), 127–208.
  * [9] H.W. Guggenheimer, _Differential Geometry_ , McGraw–Hill, New York, 1963.
  * [10] R.A. Horn and C.R. Johnson, _Matrix Analysis_ , 2nd ed., Cambridge University Press, Cambridge, 2013.
  * [11] H. Jacobowitz, _An Introduction to CR Structures_ , Math. Surveys and Monographs, 32, AMS, Providence, 1990.
  * [12] G.R. Jensen, _Higher Order Contact of Submanifolds of Homogeneous Spaces_ , Lecture Notes in Math., vol. 610, Springer–Verlag, New York, 1977.
  * [13] I.A. Kogan and P.J. Olver, Invariant Euler-Lagrange equations and the invariant variational bicomplex, Acta Appl. Math. 76 (2003), 137–193.
  * [14] M. Kolář, I. Kossovskiy, and D. Zaitsev, Normal forms in Cauchy-Riemann geometry, Contem. Math. 681 (2017), 153–177.
  * [15] B. Kruglikov and V. Lychagin, Global Lie-Tresse theorem, Sel. Math., New Ser. 22 (2016), 1357–1411.
  * [16] J. Merker, and M. Sabzevari, Explicit expression of Cartan’s connections for Levi-nondegenerate $3$-manifolds in complex surfaces, and identification of the Heisenberg sphere, Cent. Eur. J. Math. 10(5) (2012), 1801–1835.
  * [17] P.J. Olver, _Equivalence, Invariants and Symmetry_ , Cambridge, Cambridge University Press, 1995, xvi+525 pp.
  * [18] P.J. Olver, Moving frames and singularities of prolonged group actions, Selecta Math., New Series 6 (2000), 41–77.
  * [19] P.J. Olver, Differential invariants of surfaces, Diff. Geom. Appl. 27 (2009), 230–239.
  * [20] P.J. Olver, Recursive moving frames, Results Math. 60 (2011), 423–452.
  * [21] P.J. Olver, Modern developments in the theory and applications of moving frames, London Math. Soc. Impact150 Stories 1 (2015), 14–50.
  * [22] P.J. Olver, The symmetry groupoid and weighted signature of a geometric object, J. Lie Theory 26 (2016), 235–267.
  * [23] P.J. Olver, Normal forms for submanifolds under group actions, Sym. Diff. Eq. Appl. (V. G. Kac et al. (eds.)), Springer Proc. Math. Stat. (2018), 1–25.
  * [24] P.J. Olver and J. Pohjanpelto, Maurer–Cartan forms and the structure of Lie pseudo groups, Sel. Math., New Ser. 11 (2005), 99–126.
  * [25] P.J. Olver and J. Pohjanpelto, Moving frames for Lie pseudo-groups, Canad. J. Math. 60(2) (2008), 1336–1386.
  * [26] P.J. Olver and J. Pohjanpelto, Differential invariant algebras of Lie pseudo-groups, Adv. Math. 222 (2009), 1746–1792.
  * [27] P.J. Olver and F. Valiquette, Recursive moving frames for Lie pseudo-groups, Results Math. 73 (2018), 57.
  * [28] H. Poincaré, Les fonctions analytiques de deux variables et la représentation conforme, Rend. Circ. Mat. Palermo 23 (1907), 185–220.
  * [29] M. Sabzevari, Convergent normal forms for five dimensional totally nondegenerate CR manifolds in $\mathbb{C}^{4}$, J. Geom. Anal. 31 (2021), 7900–7946.
  * [30] M. Sabzevari and J. Merker, The Cartan equivalence problem for Levi-non-degenerate real hypersurfaces $M^{3}\subset\mathbb{C}^{2}$, Izv. Math. 78(6) (2014), 1158–1194.
  * [31] F. Valiquette, Equivariant moving frame method and the local equivalence of $u_{xx}=r(x,u,v,u_{x},v_{x})$ under fiber-preserving transformations, J. Dyn. Cont. Syst. 17(4) (2011), 555–589.
  * [32] F. Valiquette, Solving local equivalence problems with the equivariant moving frame method, Sym. Int. Geom.: Meth. Appl. (SIGMA) 9 029 (2013), 43 pp.
  * [33] S.M. Webster, On the Moser normal form at a non-umbilic point, Math. Ann. 233 (1978), 97–102.


# Impossible Triangle: What’s Next for Pre-trained Language Models?

Chenguang Zhu, Michael Zeng
Microsoft Cognitive Services Research Group
{chezhu<EMAIL_ADDRESS>
###### Abstract

Recent development of large-scale pre-trained language models (PLM) have
significantly improved the capability of models in various NLP tasks, in terms
of performance after task-specific fine-tuning and zero-shot / few-shot
learning. However, many of such models come with a dauntingly huge size that
few institutions can afford to pre-train, fine-tune or even deploy, while
moderate-sized models usually lack strong generalized few-shot learning
capabilities. In this paper, we first elaborate the current obstacles of using
PLM models in terms of the Impossible Triangle: 1) moderate model size, 2)
state-of-the-art few-shot learning capability, and 3) state-of-the-art fine-
tuning capability. We argue that all existing PLM models lack one or more
properties from the Impossible Triangle. To remedy these missing properties of
PLMs, various techniques have been proposed, such as knowledge distillation,
data augmentation and prompt learning, which inevitably brings additional work
to the application of PLMs in real scenarios. We then offer insights into
future research directions of PLMs to achieve the Impossible Triangle, and
break down the task into several key phases.

##  1 Background

In recent years, large-scale pre-trained language models (PLM) have
significantly improved the performance on various NLP tasks. Starting from
BERT (Devlin et al., 2018) and GPT-2 (Radford et al., 2019), the the paradigm
of self-supervised pre-training followed by supervised fine-tuning has
achieved great success, refreshing state-of-the-art results in many NLP areas
such as semantic similarity (Jiang et al., 2019), machine reading
comprehension (Yang et al., 2019), commonsense reasoning (Xu et al., 2021),
and text summarization (Zhang et al., 2020). Moreover, the moderate size111In
this paper, we refer to models with fewer than 1B parameters as “moderate-
sized”, where a single GPU today can usually handle the model fine-tuning
process. of these PLMs allow widespread and fast model fine-tuning and
adaptation.

However, in many real, especially novel, NLP scenarios, there are extremely
limited labeled data for effective fine-tuning due to either budget or time
constraint. This stimulates the development of zero-shot and few-shot NLP
models. Starting from GPT-3 (Brown et al., 2020), super large-scale PLMs (SL-
PLM) have manifested superior performance on general NLP tasks when given only
task descriptions and possibly some manual examples (Hoffmann et al., 2022;
Rae et al., 2021; Chowdhery et al., 2022). This capability was not previously
observed in moderate-sized PLMs. Nevertheless, the unprecedented scale of
these SL-PLMs have to a large degree hindered their widespread application.
One can hardly acquire enough computational resources even to load such
models, let alone efficient deployment and possible fine-tuning.

Thus, we argue that currently there have been no instance of a light-weight
PLM that has excellent performance in both supervised learning and zero/few-
shot scenarios for general NLP tasks. This has led to a lot of additional work
in the employment of these PLMs in real scenarios, detailed in the next
section.

##  2 Impossible Triangle

In this section, we summarize the current obstacles of PLMs in the Impossible
Triangle (Fig. 1). This triangle depicts the three key properties desired in
PLMs for effective and efficient usage: P1: moderate model size, P2: state-of-
the-art few-shot learning capability, and P3: state-of-the-art supervised
learning capability. These three properties correspond to three requirements
on the practical application of PLMs: P1 is for efficient deployment using a
reasonable amount of computational resources; P2 is for scenarios with zero or
very few labeled data; P3 is for scenarios with relatively abundant labeled
data.

Figure 1: The Impossible Triangle of pre-trained language models (PLM)
consists of three desired properties for model deployment in real scenarios.
P1: moderate model size, i.e., with fewer than 1 billion parameters, P2:
state-of-the-art few-shot learning capability, and P3: state-of-the-art fine-
tuning capability. (Triangle image is from
https://commons.wikimedia.org/wiki/File:Penrose_triangle.svg)

One potential reason for the existence of the impossible triangle is that at
the current stage strong few-shot learning power only emerges when a PLM has
reached an immense size with enough model capacity. Although there are work
such as iPET (Schick and Schütze, 2020) which designs moderate-sized PLMs to
achieve better few-shot learning performances than GPT-3 (Brown et al., 2020),
they have been outperformed by later SL-PLMs such as PaLM (Chowdhery et al.,
2022). And it is observed that there is discontinuous improvement of zero/few-
shot performance with model scale (Chowdhery et al., 2022). For instance,
compared to model variants with 8B and 62B parameters, PaLM with 540B
parameters demonstrates a drastic jump in accuracy on many tasks. Thus, it
remains a big challenge to develop a moderate-sized model with SoTA zero/few-
shot learning performances while maintaining superb supervised learning
capability.

Although no PLMs have achieved all three properties in the Impossible
Triangle, many of them have acquired one or two of these capabilities:

Moderate-size PLMs (With P1 + P3). These language models have a moderate model
size, i.e., fewer than 1 billion parameters, which enables efficient model
tuning and deployment. They have set state-of-the-art results in general NLP
tasks such as GLUE benchmark (Raffel et al., 2019), text summarization (Zhang
et al., 2020), open-domain question answering (Izacard and Grave, 2020) and
commonsense reasoning (Xu et al., 2021). Nevertheless, these models usually
have relatively weak zero/few-shot capability, meaning that using such models
depends on sufficient labeled data in the target domain.

Super large-scale PLMs (With P2). These language models have immense model
size (1 to 1,000 billion parameters), and were pre-trained on very large-scale
data. For instance, PaLM (Chowdhery et al., 2022) with 540 billion parameters
was pre-trained on a diverse text corpus with 780 billion tokens. They have
achieved SoTA performance in general zero/few-shot NLP tasks, when prompted
with only the task descriptions and possibly a few example input-output pairs.
However, in general, i) the zero/few-shot performance of SL-PLMs is below that
of supervised trained models, and ii) after fine-tuning, many SL-PLMs still
underperform the best fine-tuned moderate-size PLMs (Hu et al., 2021),
possibly due to their immense model sizes222The most recent SL-PLM model PaLM
has shown much better results on some NLP tasks such as summarization after
fine-tuning..

##  3 Current Remedies

Because of the Impossible Triangle, many measures have been taken to address
the missing capability(ies) of employed PLMs in practice. We summarize them as
follows:

Immense model size (Lack of P1). This happens when a SL-PLM shows superb few-
shot learning capability and also strong performance after fine-tuning. To
obtain a moderate-size model with a performance similar to that of the SL-PLM,
a common practice is knowledge distillation (KD) (Gou et al., 2021). In KD,
the larger model acts a teacher and the smaller model is the student, learning
from the predicted distribution and/or parameters from the teacher. Knowledge
distillation has been effective in creating much more efficient models with a
little sacrifice in performance.

However, there remain two problems. First, the learned student can hardly
attain the same performance of the teacher. Second, the immense size of SL-
PLMs hinders efficient inference, making them inconvenient as teacher models.

Inferior zero/few-shot performance (Lack of P2). This is most common for
moderate-sized PLMs which achieve SoTA performance after fine-tuning, but have
relatively low zero/few-shot learning capability. In many scenarios, one wants
to deploy such models when there is a lack of sufficient labeled data. Thus,
one way to remedy this problem is data augmentation (Feng et al., 2021). By
generating pseudo labels and pseudo data instances from other models (Wang et
al., 2021) or noise injection (Feng et al., 2021), the model can leverage
these additional data for effective supervised training. Nevertheless, the
variation in pseudo data quality and the diversity of data types in different
tasks pose a challenge to a generally applicable solution.

Inferior supervised training performance (Lack of P3). This is typical when
fine-tuning an SL-PLM, where the computational resource is limited or the
amount of training data is insufficient for fine-tuning a super large model. A
typical solution is prompt learning (Liu et al., 2021b). One can leverage
either a hard prompt, i.e., discrete textual template, or a soft prompt, i.e.,
continuous parameter embeddings, so that only the words and in-context
examples in hard prompt or parameters in the soft prompt are updated during
fine-tuning (Liu et al., 2021a; Li and Liang, 2021). This has been shown to be
quite effective to improve the performance of an SL-PLM given labeled data.
However, the performance could be very sensitive to prompt selection and
training data (Zhao et al., 2021) and be still inferior to that of a moderate-
size PLM with supervised learning.

##  4 Future

While the Impossible Triangle currently exists for NLP models, we argue that
it may be solved in a multi-phase approach.

Phase 1. PLMs are developed with the goal of achieving some desired properties
in the triangle while improving on the other missing properties. For instance,
a moderate-sized model with SoTA supervised learning capability could be
improved on its few-shot learning performance; or a SL-PLM with SoTA few-shot
learning capability is compressed into smaller models with better supervised
learning performance.

Phase 2. PLMs achieving all three desired properties are developed for one of
a few NLP tasks, such as NER or text summarization. To achieve this, one can
leverage the unique characteristics of the target task, e.g., less dependency
of the performance on the training data scale, smaller gap between zero/few-
shot and supervised learning performances, etc.

Phase 3. PLMs achieving all three desired properties on general NLP tasks are
developed, built upon the progress in Phase 1 and Phase 2. Potential methods
include pre-training a moderate-sized model with much larger data, better
knowledge distillation, generalized data augmentation methods, etc.

Once a PLM is equipped with all three properties of the Impossible Triangle on
general NLP tasks, it will transform the entire landscape of NLP research and
applications, facilitating fast, efficient and high-quality model development
and deployment.

##  5 Conclusion

To adapt to various practical scenarios, a pre-trained language model (PLM)
needs to have a reasonable size and acquire superb zero/few-shot and
supervised learning capabilities. However, currently there exists no models
that can achieve all these three desired properties, which we frame as the
Impossible Triangle. To remedy the missing properties from Impossible
Triangle, various technologies have been employed in practice, such as
knowledge distillation, data augmentation and prompt learning. We then propose
future research directions of pre-trained language models to achieve the
Impossible Triangle in three progressive phases.

## References

  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Feng et al. (2021) Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, and Eduard Hovy. 2021.  A survey of data augmentation approaches for nlp.  _arXiv preprint arXiv:2105.03075_.
  * Gou et al. (2021) Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021.  Knowledge distillation: A survey.  _International Journal of Computer Vision_ , 129(6):1789–1819.
  * Hoffmann et al. (2022) Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022.  Training compute-optimal large language models.  _arXiv preprint arXiv:2203.15556_.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_.
  * Izacard and Grave (2020) Gautier Izacard and Edouard Grave. 2020.  Distilling knowledge from reader to retriever for question answering.  _arXiv preprint arXiv:2012.04584_.
  * Jiang et al. (2019) Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. 2019.  Smart: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization.  _arXiv preprint arXiv:1911.03437_.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  _arXiv preprint arXiv:2101.00190_.
  * Liu et al. (2021a) Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021a.  What makes good in-context examples for gpt-$3$?  _arXiv preprint arXiv:2101.06804_.
  * Liu et al. (2021b) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2021b.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  _arXiv preprint arXiv:2107.13586_.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Rae et al. (2021) Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021.  Scaling language models: Methods, analysis & insights from training gopher.  _arXiv preprint arXiv:2112.11446_.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_.
  * Schick and Schütze (2020) Timo Schick and Hinrich Schütze. 2020.  It’s not just size that matters: Small language models are also few-shot learners.  _arXiv preprint arXiv:2009.07118_.
  * Wang et al. (2021) Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021.  Want to reduce labeling cost? gpt-3 can help.  _arXiv preprint arXiv:2108.13487_.
  * Xu et al. (2021) Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, and Xuedong Huang. 2021.  Human parity on commonsenseqa: Augmenting self-attention with external attention.  _arXiv preprint arXiv:2112.03254_.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020.  Pegasus: Pre-training with extracted gap-sentences for abstractive summarization.  In _International Conference on Machine Learning_ , pages 11328–11339. PMLR.
  * Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.  Calibrate before use: Improving few-shot performance of language models.  In _International Conference on Machine Learning_ , pages 12697–12706. PMLR.


# Learning Control of Quantum Systems

Daoyi Dong D. Dong is with the School of Engineering and Information
Technology, University of New South Wales, Canberra, ACT 2600, Australia
<EMAIL_ADDRESS>
###### Abstract

This paper provides a brief introduction to learning control of quantum
systems. In particular, the following aspects are outlined, including
gradient-based learning for optimal control of quantum systems, evolutionary
computation for learning control of quantum systems, learning-based quantum
robust control, and reinforcement learning for quantum control.

##  I Introduction

Controlling quantum systems has become a central task in the development of
quantum technologies, and quantum control has witnessed rapid progress in the
last two decades; for an overview, see, e.g., the survey papers [1, 2, 3, 4,
5] or the monographs [6, 7]. The general goal of quantum control is to
actively manipulate and control the dynamics of quantum systems for achieving
given objectives [8, 9] (e.g., rapid state transfer, high-fidelity gate
operation). Two of fundamental issues in quantum control include investigating
controllability of quantum systems and designing control laws to achieve
expected control systems performance. Controllability is concerned with what
control targets can be achieved and the controllability of finite-dimensional
closed systems has been well addressed [7]. A few results on the
controllability of open quantum systems have also been presented. For control
law design, optimal control theory [1], Lyapunov control approaches [10],
learning control algorithms [2] and robust control methods [11] have been
developed in manipulating quantum systems for achieving various control
objectives.

Among various control design approaches, learning control is recognized as a
powerful method for many complex quantum control tasks and has achieved great
success in laser control of molecules and other applications since the
approach was presented in the seminal paper [12]. Many quantum control tasks
may be formulated as an optimization problem and a learning algorithm can be
employed to search for an optimal control field satisfying a desired
performance condition. Gradient algorithms have been demonstrated to be an
excellent candidate for numerically finding an optimal field and have achieved
successful applications in nuclear magnetic resonance (NMR) systems due to
their high efficiency [13]. In many other optimal control problems, the
gradient information may not be easy to obtain and some complex problems may
have local optima. For these situations, stochastic search algorithms usually
have improved performance to find a good control field. The genetic algorithm
(GA) and differential evolution (DE) [11] have been widely used in the area of
quantum control of molecular systems and achieved great success [2]. Another
task in quantum control is to achieve robustness performance in quantum
systems. Gradient-based learning algorithms and stochastic search algorithms
may be smartly modified to search for robust control fields. Also, some other
machine learning algorithms such as reinforcement learning have found
successful applications in various tasks (e.g., quantum error correction
[14]).

##  II Gradient-based learning for optimal control of quantum systems

Consider a finite-dimensional quantum control system where its state
$|\psi(t)\rangle$ (using the Dirac notation) is described by the following
Schrödinger equation (setting $\hbar=1$):

$\frac{d}{dt}|{\psi}(t)\rangle=-i[H_{0}+\sum_{m=1}^{M}u_{m}(t)H_{m}]|\psi(t)\rangle,\quad
t\in[0,T],$ (1)

where $H_{0}$ is the free Hamiltonian of the system and
$H_{c}(t)=\sum_{m=1}^{M}u_{m}(t)H_{m}$ is the control Hamiltonian at time $t$
that represents the interaction of the system with the external fields
$u_{m}(t)$. The $H_{m}$ are Hermitian operators through which the controls
couple to the system. The objective of quantum optimal control is to find
control fields $u_{m}(t)$ for maximizing a performance functional $\Phi$.
$\Phi$ may be a given functional of the state $|\psi\rangle$ and control
defined according to practical requirement. For example, the fidelity
$\Phi=|\langle\psi(T)|\psi_{f}\rangle|^{2}$ between the final state
$|\psi(T)\rangle$ and target state $|\psi_{f}\rangle$ or the expectation
$\Phi=|\langle\psi(T)|\hat{O}|\psi(T)\rangle|^{2}$ of an operator $\hat{O}$
may be defined as a performance index for state transfer task. In order to
maximize the performance $\Phi$, we may employ the GRAPE (gradient ascent
pulse engineering) algorithm [13] or the Krotov method [15] to search for the
control field. For simplicity, we may discretize time $T$ in $N$ equal steps
and during each step let the control fields $u_{m}$ be constant. The basic
idea in the GRAPE algorithm is that the control fields are iteratively updated
following the gradient direction of $\frac{\delta\Phi}{\delta u_{m}(k)}$ with
a learning rate $\eta$, i.e.,

$u_{m}(k+1)=u_{m}(k)+\eta\frac{\delta\Phi}{\delta u_{m}(k)}.$ (2)

The gradient-based learning method can also be extended to the optimal control
problem of unitary transformations (e.g., quantum gates) and open quantum
systems. For example, for a unitary transformation $U$, its evolution is
described by the following equation

$\dot{U}(t)=-i[H_{0}+\sum_{m=1}^{M}u_{m}(t)H_{m}]U(t),\ \ \ \ U(0)=I.$ (3)

Now the objective is to design the controls $u_{m}(t)$ to steer the unitary
$U(t)$ from $U(0)=I$ to a desired target $U_{F}$ with high fidelity. We may
define the performance function as $\Phi=|{\langle
U_{F}|e^{i\varphi}U(T)\rangle}|^{2}$ for an arbitrary phase factor $\varphi$.
Then we can calculate the gradient $\delta\Phi/\delta u_{m}(k)$, and the
optimal control field can be searched for by following the gradient [16]. When
we consider the optimal control problem of an open quantum system, its state
should be represented by a density matrix $\rho$ and its dynamics should be
described by a master equation. The dynamics of a Markovian open quantum
system can be described using the following master equation in the Lindblad
form as [6]

$\dot{\rho}(t)=-i[H_{0}+\sum_{m=1}^{M}u_{m}(t)H_{m},\rho(t)]+\sum_{k}\gamma_{k}\mathcal{D}[L_{k}]\rho(t),$
(4)

where the non-negative coefficients $\gamma_{k}$ specify the relevant
relaxation rates, $L_{k}$ are appropriate Lindblad operators and
$\mathcal{D}[L_{k}]\rho=(L_{k}\rho
L_{k}^{\dagger}-\frac{1}{2}L_{k}^{\dagger}L_{k}\rho-\frac{1}{2}\rho
L_{k}^{\dagger}L_{k}).$ The open GRAPE algorithm has also been developed to
calculate the gradient based on the master equation (see [17]).

Using the basic idea of gradient-based learning control, some variants have
been developed for various requirements in quantum optimal control. For
example, a data-driven gradient optimization algorithm (d-GRAPE) has been
proposed to correct deterministic gate errors in high-precision quantum
control by jointly learning from a design model and the experimental data from
quantum tomography [18]. A gradient-based frequency-domain optimization
algorithm has been developed to solve the optimal control problem with
constraints in the frequency domain [19]. Existing results show that gradient-
based learning methods can usually achieve excellent performance for solving
optimal control problems when the system model is known and the dynamics can
be equivalently (or approximately) described using a closed quantum system.
This is also analyzed using quantum control landscape theory [20].

##  III Evolutionary computation for learning control of quantum systems

Gradient algorithms have shown powerful capability for numerically finding
optimal controls due to their excellent performance [13]. In many practical
applications, it may be difficult to obtain the gradient information or there
exist local optima in complex quantum control problems. For these situations,
a natural idea is to employ stochastic search algorithms to seek good
controls. Evolutionary computation including GA and DE has been widely used in
the area of quantum control. In these evolutionary computation methods,
crossover, mutation and selection operations are iteratively implemented to
search for good solutions (optimal controls) in a parameter space. For
example, a subspace-selective self-adaptive differential evolution (SUSSADE)
algorithm has been proposed to achieve a high-fidelity single-shot Toffoli
gate and single-shot three-qubit gates [21], [22]. Existing results showed
that DE with equally-mixed strategies can achieve improved performance for
quantum control problems [23]. Several promising evolution algorithms have
been investigated comparatively in [24] and it was found that DE usually
outperformed GA and particle swarm optimization for hard quantum control
problems.

The above introduction of gradient-based learning and evolutionary computation
mainly involves open-loop control strategies. Evolutionary computation has
demonstrated extremely powerful capability when it is integrated into closed-
loop control design. Closed-loop learning control, where each cycle of the
closed-loop is executed on a new sample, has achieved great successes in the
laser control of laboratory chemical reactions [2, 12]. A closed-loop learning
control procedure generally involves three components [1, 2]: (i) a trial
laser control input, (ii) the laboratory generation of the control that is
applied to the sample and subsequently observed for its impact, and (iii) a
learning algorithm to suggest the form of the next control input by
considering the prior experiments. The initial trial control input may be a
random input field or a well-designed laser pulse. A feature of a good closed-
loop learning control design is its insensitivity to the initial trials. A key
task is to develop a good learning algorithm for ensuring that the learning
process converges to achieve a predetermined objective. GA, DE and several
rapid convergence algorithms have been developed for this task [4, 12]. The
optimal control problem is usually formulated as solving an optimization
problem by maximizing a functional which is related to some variables such as
the control inputs, quantum states and control time but may have no analytical
form. In the learning process, the optimization problem is solved iteratively.
First, a trial input is applied to a sample to be controlled and the result is
observed. Second, a learning algorithm suggests a better control input based
on the prior experiments. Third, the “better” control input is applied to a
new sample. This process continues until the control objective is achieved or
the maximum permitted iteration number is reached. It is often feasible to
produce many identical-state samples for laboratory chemical molecules. If the
control objective is well selected, there is a capability to apply specified
control inputs to the samples, and the learning algorithm is sufficiently
smart for searching for good control inputs, this process will converge and an
optimal control pulse can be found [2].

##  IV Learning-based quantum robust control

The robust control of quantum systems has been recognized as a key task in
developing practical quantum technology since the existence of noise and
uncertainties is unavoidable. Learning control is an effective candidate for
achieving robust performance in some quantum control problems [11]. We first
consider the control problem of inhomogeneous quantum ensembles. An
inhomogeneous quantum ensemble consists of many individual quantum systems
(e.g., atoms, molecules or spin systems) and the parameters describing the
system dynamics of these individual systems could have variations [25, 26]. An
example is that a spin ensemble in NMR may encounter large dispersion in the
strength of the applied radio frequency field and there also exist variations
in the natural frequencies of these spins [25]. Inhomogeneous quantum
ensembles have wide applications in many fields ranging from quantum memory to
magnetic-resonance imaging. Hence, it is highly desirable to design control
laws for an inhomogeneous ensemble to employ the same control inputs to steer
individual systems with different dynamics from a given initial state to a
target state.

A sampling-based learning control (SLC) method has been developed to achieve
high fidelity control of inhomogeneous quantum ensembles [26]. Consider an
inhomogeneous ensemble in which the Hamiltonian of each individual system has
the following form

$H_{\omega,\theta}(t)=\omega H_{0}+\sum_{m=1}^{M}\theta u_{m}(t)H_{m}.$ (5)

We assume that the parameters $\omega\in[1-\Omega,1+\Omega]$ and
$\theta\in[1-\Theta,1+\Theta]$, and the constants $\Omega\in[0,1]$ and
$\Theta\in[0,1]$ represent the bounds of the parameter dispersion. The
objective is to design the controls $\\{u_{m}(t)\\}$ to simultaneously
stabilize the individual systems (with different $\omega$ and $\theta$) of the
quantum ensemble from an initial state $|\psi_{0}\rangle$ to the same target
state $|\psi_{f}\rangle$ with high fidelity. This task can be achieved using
the SLC method including two steps of “training” and “testing and evaluation”
[26]. In the training step, we select $N$ samples from the quantum ensemble
regarding the distribution (e.g., uniform distribution) of the inhomogeneity
parameters and then construct a generalized system as follows

$\frac{d}{dt}\left(\begin{array}[]{c}|{\psi}_{\omega_{1},\theta_{1}}(t)\rangle\\\
|{\psi}_{\omega_{2},\theta_{2}}(t)\rangle\\\ \vdots\\\
|{\psi}_{\omega_{N},\theta_{N}}(t)\rangle\\\
\end{array}\right)=-i\left(\begin{array}[]{c}H_{\omega_{1},\theta_{1}}(t)|\psi_{\omega_{1},\theta_{1}}(t)\rangle\\\
H_{\omega_{2},\theta_{2}}(t)|\psi_{\omega_{2},\theta_{2}}(t)\rangle\\\
\vdots\\\
H_{\omega_{N},\theta_{N}}(t)|\psi_{\omega_{N},\theta_{N}}(t)\rangle\\\
\end{array}\right)$ (6)

where
$H_{\omega_{n},\theta_{n}}=\omega_{n}H_{0}+\sum_{m}\theta_{n}u_{m}(t)H_{m}$
with $n=1,2,\dots,N$. The cost function for the generalized system is defined
by

$\Phi_{N}(u):=\frac{1}{N}\sum_{n=1}^{N}\Phi_{\omega_{n},\theta_{n}}(u).$ (7)

The task of the training step is to find a control strategy $u^{*}$ to
maximize the cost functional $\Phi_{N}(u)$. A gradient-based learning
algorithm (s-GRAPE) can be developed to complete this task. In the process of
testing and evaluation, a number of sampling individual systems are randomly
selected to evaluate the control performance. Results show that the SLC method
is potential for control design of various inhomogeneous quantum ensembles
(including inhomogeneous open quantum ensembles).

Besides inhomogeneous quantum ensembles, the SLC method is useful for robust
control of single quantum systems with various uncertainties. For example, Eq.
(5) can also correspond to the Hamiltonian of a quantum system with inaccurate
model parameter $\omega$ and uncertain multiplicative noise $\theta$. In order
to achieve robust control for such a quantum system, we may employ the SLC
method to search for robust control pulses [27, 28, 29]. The performance of
SLC approach can be further improved by exploring the richness and diversity
of samples. Inspired by deep learning, a batch-based gradient algorithm
(b-GRAPE) has been presented for efficiently seeking robust quantum controls,
and numerical results showed that b-GRAPE can achieve improved performance
over the SLC method for remarkably enhancing the control robustness while
maintaining high fidelity [30]. In other applications where we need to enhance
the robustness in closed-loop learning control, we may either use the Hessian
matrix information [31] or integrate the idea of SLC into the learning
algorithm in searching for robust control fields. For example, an improved DE
algorithm (called as _msMS_ _DE) has been proposed to search for robust
femtosecond laser pulses to control fragmentation of the molecule
$\text{CH}_{2}\text{BrI}$ [11]. In _msMS_ _DE, multiple samples are used for
fitness evaluation and a mixed strategy is employed for the mutation
operation.

##  V Reinforcement learning for quantum control

Reinforcement learning (RL) [32] is another important machine learning
approach and it addresses the problem of how an active agent can learn to
approximate an optimal strategy while interacting with its environment. It is
a model-free feedback-based approach and works well even when the system model
is unknown or with uncertainties. RL has been used for learning control of
quantum systems. For example, a fidelity-based probabilistic Q-learning
approach has been presented to naturally solve the balance problem between
exploration and exploitation and was applied to learning control of quantum
systems [33]. The authors in [34] showed that the performance of RL is
comparable to optimal control approaches in the task of finding a short and
high-fidelity protocol, controlling from an initial to a given target state in
nonintegrable many-body quantum systems of interacting qubits. RL can also
help identify variational protocols with nearly optimal fidelity even in the
glassy phase. In [35], deep reinforcement learning is employed to
simultaneously optimize the speed and fidelity of quantum computation against
both leakage and stochastic control errors. A universal quantum control
framework was presented to improve the control robustness by adding control
noise into training environments for RL agents trained with trusted-region-
policy-optimization. By utilizing two-stage learning with teacher and student
networks and a reward quantifying the capability to recover the quantum
information stored in a quantum system, the authors in [14] showed how a
network-based “agent” in RL can discover good quantum-error-correction
strategies to protect qubits against noise.

##  VI Conclusions

Machine learning has shown powerful capability in discovering high quality
controls to achieve optimal control and enhance robust performance for quantum
systems. On one hand, it is necessary to further develop or improve existing
machine learning algorithms to effectively solve complex quantum control
problems emerged from new quantum technologies. On the other hand, various
cutting-edge machine learning techniques should be able to find new
application opportunities in the area of quantum control.

## References

  * [1] D. Dong, and I. R. Petersen, “Quantum control theory and applications: a survey,” IET Control Theory & Applications, vol. 4, no. 12, pp. 2651-2671, 2010.
  * [2] H. Rabitz, R. De Vivie-Riedle, M. Motzkus, and K. Kompa, “Whither the future of controlling quantum phenomena?” Science, vol. 288, no. 5467, pp. 824-828, 2000.
  * [3] S. J. Glaser, U. Boscain, T. Calarco, C. P. Koch, W. Köckenberger, R. Kosloff, I. Kuprov, B. Luy, S. Schirmer, T. Schulte-Herbrüggen, D. Sugny and F. K. Wilhelm, “Training Schrödinger’s cat: quantum optimal control,” The European Physical Journal D, vol. 69, p. 279, 2015.
  * [4] C. Brif, R. Chakrabarti and H. Rabitz, “Control of quantum phenomena: past, present and future,” New Journal of Physics, Vol.12, p.075008, 2010.
  * [5] C. Altafini and F. Ticozzi, “Modeling and control of quantum systems: an introduction,” IEEE Transactions on Automatic Control, vol.57, no.8, pp.1898-1917, 2012.
  * [6] H. M. Wiseman, and G. J. Milburn, “Quantum Measurement and Control,” Cambridge, England: Cambridge University Press, 2010\.
  * [7] D. D’Alessandro, Introduction to Quantum Control and Dynamics, Chapman & Hall/CRC, 2007.
  * [8] A. Acín, I. Bloch, H. Buhrman, T. Calarco, C. Eichler, J. Eisert, _et al._ , “The quantum technologies roadmap: a European community view”, New Journal of Physics, vol. 20, p. 080201, 2018.
  * [9] Y. Guo, C. C. Shu , D. Dong, and F. Nori, “Vanishing and revival of resonance Raman scattering”, Physical Review Letters, vol. 123, p. 223202, 2019.
  * [10] S. Kuang, D. Dong, and I. R. Petersen, “Rapid Lyapunov control of finite-dimensional quantum systems” Automatica, vol. 81, pp.164-175, 2017.
  * [11] D. Dong, X. Xing, H. Ma, C. Chen, Z. Liu and H. Rabitz, “Learning-based quantum robust control: algorithm, applications and experiments,” IEEE Transactions on Cybernetics, vol.50, pp.3581- 3593, 2020.
  * [12] R. S. Judson, and H. Rabitz, “Teaching lasers to control molecules,” Physical Review Letters, vol. 68, pp. 1500-1503, 1992.
  * [13] N. Khaneja, T. Reiss, C Kehlet, T. Schulte-Herbrüggen, and S. J. Glaser, “Optimal control of coupled spin dynamics: design of NMR pulse sequences by gradient ascent algorithms,” Journal of Magnetic Resonance, vol. 172, no. 2, pp. 296-305, 2005.
  * [14] T. Fösel, P. Tighineanu, T. Weiss, and F. Marquardt, “Reinforcement learning with neural networks for quantum feedback”, Physical Review X, vol. 8, p. 031084, 2018.
  * [15] G. Jäger, D. M. Reich, M. H. Goerz, C. P. Koch, and U. Hohenester, “Optimal quantum control of Bose-Einstein condensates in magnetic microtraps: Comparison of gradient-ascent-pulse-engineering and Krotov optimization schemes,” Physical Review A, vol. 90, p. 033628, 2014.
  * [16] D. Dong, C. Wu, C. Chen, B. Qi, I. R. Petersen and F. Nori, “Learning robust pulses for generating universal quantum gates,” Scientific Reports, vol. 6, p. 36090, 2016.
  * [17] T. Schulte-Herbrüggen, A. Spörl, N. Khaneja, & S. J. Glaser. “Optimal control for generating quantum gates in open dissipative systems”. J. Phys. B: At. Mol. Opt. Phys. vol. 44, p. 154013, 2011.
  * [18] R. B. Wu, B. Chu, D.H. Owens, H. Rabitz, “Data-driven gradient algorithm for high-precision quantum control,” Physical Review A, vol. 97, p. 042122, 2018.
  * [19] C. C. Shu, T. S. Ho, X. Xing and H. Rabitz, “Frequency domain quantum optimal control under multiple constraints,” Physical Review A, vol. 93, p. 033417, 2016.
  * [20] R. Chakrabarti and H. Rabitz, “Quantum control landscapes,” International Reviews in Physical Chemistry, vol. 26, no. 4, pp. 671-735, 2007.
  * [21] E. Zahedinejad, J. Ghosh, and B. C. Sanders, “High-fidelity single-shot Toffoli gate via quantum control,” Physical Review Letters, vol. 114, no. 20, p. 200502, 2015.
  * [22] E. Zahedinejad, J. Ghosh, and B. C. Sanders, “Desgining high-fidelity single-shot three-qubit gates: A machine-learning approach,” Physical Review Applied, vol. 6, p. 054005, 2016.
  * [23] H. Ma, D. Dong, C.-C. Shu, Z. Zhu, and C. Chen, “Differential evolution with equally-mixed strategies for robust control of open quantum systems,” Control Theory and Technology, vol. 15, pp. 226-241, 2017.
  * [24] E. Zahedinejad, S. Schirmer, and B. C. Sanders, “Evolutionary algorithms for hard quantum control,” Physical Review A, vol. 90, no. 3, p. 032310, 2014.
  * [25] J. S. Li, and N. Khaneja, “Control of inhomogeneous quantum ensembles,” Physical Review A, vol. 73, no. 3, p. 030302, 2006.
  * [26] C. Chen, D. Dong, R Long, I. R. Petersen and H. A. Rabitz, “Sampling-based learning control of inhomogeneous quantum ensembles,” Physical Review A, vol. 89, no. 2, p. 023402, 2014.
  * [27] D. Dong, M. A. Mabrok, I. R. Petersen, B. Qi, C. Chen, and H. Rabitz, “Sampling-based learning control for quantum systems with uncertainties,” IEEE Transactions on Control Systems Technology, vol. 23, pp. 2155-2166, 2015.
  * [28] D. Dong, C. Chen, B. Qi, I. R. Petersen and F. Nori, “Robust manipulation of superconducting qubits in the presence of fluctuations,” Scientific Reports, vol. 5, p. 7873, 2015.
  * [29] C. Wu, B. Qi, C. Chen, and D. Dong, “Robust learning control design for quantum unitary transformations,” IEEE Transactions on Cybernetics, vo. 47, pp. 4405-4417, 2017.
  * [30] R. Wu, H. Ding, D. Dong and X. Wang, “Learning robust and high-precision quantum controls,” Physical Review A, vol. 99, p. 042327, 2019.
  * [31] X. Xing, R. Rey-de-Castro and H. Rabitz, “Assessment of optimal control mechanism complexity by experimental landscape Hessian analysis: fragmentation of CH2BrI,” New Journal of Physics, vol. 16, p. 125004, 2014.
  * [32] R. Sutton and A. G. Barto, _Reinforcement Learning: An Introduction_. Cambridge, MA: MIT Press, 1998.
  * [33] C. Chen, D. Dong, H.X. Li, J. Chu and T.J. Tarn, “Fidelity-based probabilistic Q-learning for control of quantum systems”, _IEEE Transactions on Neural Networks and Learning Systems_ , Vol. 25, pp.920-933, 2014.
  * [34] M. Bukov, A. G. R. Day, D. Sels, P. Weinberg, A. Polkovnikov, and P. Mehta, “Reinforcement learning in different phases of quantum control”, Physical Review X, vol. 8, p. 031086, 2018.
  * [35] M. Y. Niu, S. Boixo, V. N. Smelyanskiy, and H. Neven, “Universal quantum control through deep reinforcement learning”, npj Quantum Information, vol. 5, p. 33, 2019.