open question is whether our framework can be used in such cases to learn
relaxed symmetries, as described by Wang et al. (2022). In other words, we may
hope to find elements in ${\mathcal{F}}$ that are nearly symmetric, and to
bound the degree of asymmetry based on the quantities derived from the
fundamental operators, such as their norms. Our proposed convex penalization
methods are likely affected by the choice of $G$ and the discretization of the
fundamental operators. A natural question is whether a large candidate group
of symmetries can be used without biasing the model towards undesirable or
nonphysical symmetries. Additionally, the choice of inner products associated
with the discretization of the fundamental operators could affect the results
of nuclear norm penalization. Our reliance on the Lie algebra to study
continuous symmetries also limits the ability of our proposed methods to
account for partial symmetries, such as the invariance in classifying the
characters “Z” and “N” to rotations by small angles, but not large angles.

In a follow-up paper we will apply the proposed methods to a wide range of
examples, explaining in detail how to implement the methods in practice. A
main goal will be to study to what extent the nuclear norm relaxation can
recover underlying symmetry groups and reduce the amount of data required to
train accurate machine learning models. Additionally, we will examine how the
proposed techniques perform in the presence of noisy data, with the goal of
understanding the empirical effects of problem dimension, noise level, and the
candidate symmetry group. The follow-up will also explore different choices of
inner products in the discretization of the fundamental operators and the
effect this has on the performance of the proposed symmetry-promoting
regularization methods.

Other important avenues of future work include investigating computationally
efficient approaches to discretize the fundamental operators and use them to
enforce, discover, and promote symmetry within our framework. This could
involve leveraging sparse structure of the discretized operators in certain
bases to enable the use of efficient Krylov subspace algorithms. It will also
be useful to identify efficient optimization algorithms for training symmetry-
constrained or symmetry-regularized machine learning models. Promising
candidates include projected gradient descent, proximal splitting algorithms,
and the Iteratively Reweighted Least Squares (IRLS) algorithms described by
Mohan and Fazel (2012). Using IRLS could enable symmetry-promoting penalties
to be based on non-convex Schatten $p$-norms with $0<p<1$, potentially
improving the recovery of underlying symmetry groups compared to the nuclear
norm where $p=1$.

There are also several avenues we plan to explore in future theoretical work.
These include extending the techniques presented here via jet bundle
prolongation to study symmetries in machine learning for Partial Differential
Equations (PDEs). Combining analogues of our proposed methods in this setting
with techniques using the weak formulation proposed by Messenger and Bortz
(2021); Reinbold et al. (2020) could provide robust ways to identify symmetric
PDEs in the presence of high noise and limited training data. We also aim to
study the perturbative effects of noisy data in algorithms to discover and
promote symmetry with the goal of understanding the effects of problem
dimension, noise level, and number of data points on recovery of symmetry
groups. Another important direction of theoretical study will be to build on
the work of Peitz et al. (2023); Steyert (2022) by studying symmetry in the
setting of Koopman operators for dynamical systems. The connection between the
fundamental operators in this paper and Koopman operators may be strengthened
by establishing that the fundamental operators $\\{{\mathcal{K}}_{g}\\}_{g\in
G}$ form a strongly continuous group in an $L^{2}$ space of sections of the
appropriate vector bundle, and that the corresponding infinitesimal generators
are given by closures of the Lie derivatives. One could then show that the
Koopman operators of symmetric systems commute with these generators, and use
this information to constrain the spectral projections of the Koopman
operators. This might be used to discover underlying symmetries based on
Koopman operators, as well as to enhance algorithms to approximate Koopman
operators by enforcing and promoting symmetry. To do this, one might follow
the program set forth by Colbrook (2023), where the measure preserving
property of certain dynamical systems is exploited to enhance the Extended
Dynamic Mode Decomposition (EDMD) algorithm of Williams et al. (2015).

## Acknowledgements

The authors acknowledge support from the National Science Foundation AI
Institute in Dynamic Systems (grant number 2112085). SLB acknowledges support
from the Army Research Office (ARO W911NF-19-1-0045). The authors would also
like to acknowledge valuable discussions with Tess Smidt and Matthew Colbrook.

## References

  * Abraham et al. (1988) R. Abraham, J. E. Marsden, and T. Ratiu.  _Manifolds, Tensor Analysis, and Applications_ , volume 75 of _Applied Mathematical Sciences_.  Springer-Verlag, 1988.
  * Abraham and Marsden (2008) Ralph Abraham and Jerrold E Marsden.  _Foundations of mechanics_.  AMS Chelsea Publishing, 2 edition, 2008.
  * Ahmadi and Khadir (2020) Amir Ali Ahmadi and Bachir El Khadir.  Learning dynamical systems with side information.  In _Proceedings of the 2nd Conference on Learning for Dynamics and Control_ , volume 120 of _Proceedings of Machine Learning Research_ , pages 718–727. PMLR, 10–11 Jun 2020.  URL https://proceedings.mlr.press/v120/ahmadi20a.html.
  * Baddoo et al. (2023) Peter J Baddoo, Benjamin Herrmann, Beverley J McKeon, J Nathan Kutz, and Steven L Brunton.  Physics-informed dynamic mode decomposition.  _Proceedings of the Royal Society A_ , 479(2271):20220576, 2023.
  * Batzner et al. (2022) Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Kornbluth, Nicola Molinari, Tess E Smidt, and Boris Kozinsky.  E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials.  _Nature communications_ , 13(1):2453, 2022.
  * Benton et al. (2020) Gregory Benton, Marc Finzi, Pavel Izmailov, and Andrew G Wilson.  Learning invariances in neural networks from training data.  _Advances in neural information processing systems_ , 33:17605–17616, 2020.
  * Berry and Giannakis (2020) Tyrus Berry and Dimitrios Giannakis.  Spectral exterior calculus.  _Communications on Pure and Applied Mathematics_ , 73(4):689–770, 2020.
  * Bouwmans et al. (2018) Thierry Bouwmans, Sajid Javed, Hongyang Zhang, Zhouchen Lin, and Ricardo Otazo.  On the applications of robust PCA in image and video processing.  _Proceedings of the IEEE_ , 106(8):1427–1457, 2018.
  * Brunton and Kutz (2022) S. L. Brunton and J. N. Kutz.  _Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control_.  Cambridge University Press, 2nd edition, 2022.
  * Brunton et al. (2016) Steven L Brunton, Joshua L Proctor, and J Nathan Kutz.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems.  _Proceedings of the national academy of sciences_ , 113(15):3932–3937, 2016.
  * Brunton et al. (2020) Steven L. Brunton, Bernd R. Noack, and Petros Koumoutsakos.  Machine learning for fluid mechanics.  _Annual Review of Fluid Mechanics_ , 52:477–508, 2020.
  * Brunton et al. (2022) Steven L Brunton, Marko Budišić, Eurika Kaiser, and J Nathan Kutz.  Modern Koopman theory for dynamical systems.  _SIAM Review_ , 64(2):229–340, 2022.
  * Cahill et al. (2023) Jameson Cahill, Dustin G Mixon, and Hans Parshall.  Lie PCA: Density estimation for symmetric manifolds.  _Applied and Computational Harmonic Analysis_ , 2023.
  * Callaham et al. (2022) Jared L Callaham, Georgios Rigas, Jean-Christophe Loiseau, and Steven L Brunton.  An empirical mean-field model of symmetry-breaking in a turbulent wake.  _Science Advances_ , 8(eabm4786), 2022.
  * Candès et al. (2011) Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright.  Robust principal component analysis?  _Journal of the ACM (JACM)_ , 58(3):1–37, 2011\.
  * Caron and Traynor (2005) Richard Caron and Tim Traynor.  The zero set of a polynomial.  _WSMR Report 05-02_ , 2005.  URL https://www.researchgate.net/profile/Richard-Caron-3/publication/281285245_The_Zero_Set_of_a_Polynomial/links/55df56b608aecb1a7cc1a043/The-Zero-Set-of-a-Polynomial.pdf.
  * Champion et al. (2020) Kathleen Champion, Peng Zheng, Aleksandr Y Aravkin, Steven L Brunton, and J Nathan Kutz.  A unified sparse optimization framework to learn parsimonious physics-informed models from data.  _IEEE Access_ , 8:169259–169271, 2020.
  * Chen et al. (2020) Shuxiao Chen, Edgar Dobriban, and Jane H. Lee.  A group-theoretic framework for data augmentation.  _J. Mach. Learn. Res._ , 21(1), jan 2020.  ISSN 1532-4435.
  * Cohen and Welling (2016) Taco Cohen and Max Welling.  Group equivariant convolutional networks.  In Maria Florina Balcan and Kilian Q. Weinberger, editors, _Proceedings of The 33rd International Conference on Machine Learning_ , volume 48 of _Proceedings of Machine Learning Research_ , pages 2990–2999, New York, New York, USA, 20–22 Jun 2016. PMLR.  URL https://proceedings.mlr.press/v48/cohenc16.html.
  * Cohen et al. (2018) Taco S Cohen, Mario Geiger, Jonas Köhler, and Max Welling.  Spherical CNNs.  _arXiv preprint arXiv:1801.10130_ , 2018.
  * Cohen et al. (2019) Taco S Cohen, Mario Geiger, and Maurice Weiler.  A general theory of equivariant CNNs on homogeneous spaces.  _Advances in neural information processing systems_ , 32, 2019.
  * Colbrook (2023) Matthew J Colbrook.  The mpEDMD algorithm for data-driven computations of measure-preserving dynamical systems.  _SIAM Journal on Numerical Analysis_ , 61(3):1585–1608, 2023.
  * Cubuk et al. (2019) Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le.  Autoaugment: Learning augmentation strategies from data.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 113–123, 2019.
  * Desai et al. (2022) Krish Desai, Benjamin Nachman, and Jesse Thaler.  Symmetry discovery with deep learning.  _Physical Review D_ , 105(9):096031, 2022.
  * Esteves et al. (2018) Carlos Esteves, Christine Allen-Blanchette, Ameesh Makadia, and Kostas Daniilidis.  Learning SO(3) equivariant representations with spherical CNNs.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 52–68, 2018.
  * Finzi et al. (2020) Marc Finzi, Samuel Stanton, Pavel Izmailov, and Andrew Gordon Wilson.  Generalizing convolutional neural networks for equivariance to Lie groups on arbitrary continuous data.  In _International Conference on Machine Learning_ , pages 3165–3176. PMLR, 2020.
  * Finzi et al. (2021) Marc Finzi, Max Welling, and Andrew Gordon Wilson.  A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups.  In _International Conference on Machine Learning_ , pages 3318–3328. PMLR, 2021.
  * Fukushima (1980) Kunihiko Fukushima.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position.  _Biological cybernetics_ , 36(4):193–202, 1980\.
  * Gotô (1950) Morikuni Gotô.  Faithful representations of Lie groups II.  _Nagoya mathematical journal_ , 1:91–107, 1950.
  * Gross (2011) David Gross.  Recovering low-rank matrices from few coefficients in any basis.  _IEEE Transactions on Information Theory_ , 57(3):1548–1566, 2011.
  * Gross (1996) David J Gross.  The role of symmetry in fundamental physics.  _Proceedings of the National Academy of Sciences_ , 93(25):14256–14259, 1996.
  * Gruver et al. (2022) Nate Gruver, Marc Anton Finzi, Micah Goldblum, and Andrew Gordon Wilson.  The Lie derivative for measuring learned equivariance.  In _The Eleventh International Conference on Learning Representations_ , 2022.
  * Guan et al. (2021) Yifei Guan, Steven L Brunton, and Igor Novosselov.  Sparse nonlinear models of chaotic electroconvection.  _Royal Society Open Science_ , 8(8):202367, 2021\.
  * Hall (2015) Brian C. Hall.  _Lie Groups, Lie Algebras, and Representations: An Elementary Introduction_.  Springer, 2015.
  * Hataya et al. (2020) Ryuichiro Hataya, Jan Zdenek, Kazuki Yoshizoe, and Hideki Nakayama.  Faster autoaugment: Learning augmentation strategies using backpropagation.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16_ , pages 1–16. Springer, 2020.
  * Holmes et al. (2012) P. J. Holmes, J. L. Lumley, G. Berkooz, and C. W. Rowley.  _Turbulence, coherent structures, dynamical systems and symmetry_.  Cambridge Monographs in Mechanics. Cambridge University Press, Cambridge, England, 2nd edition, 2012.
  * Horn and Johnson (2013) Roger A. Horn and Charles R. Johnson.  _Matrix Analysis_.  Cambridge University Press, 2 edition, 2013.
  * Kaiser et al. (2018) Eurika Kaiser, J Nathan Kutz, and Steven L Brunton.  Discovering conservation laws from data for control.  In _2018 IEEE Conference on Decision and Control (CDC)_ , pages 6415–6421. IEEE, 2018.
  * Kaiser et al. (2021) Eurika Kaiser, J Nathan Kutz, and Steven L Brunton.  Data-driven discovery of koopman eigenfunctions for control.  _Machine Learning: Science and Technology_ , 2(3):035023, 2021.
  * Kolář et al. (1993) Ivan Kolář, Peter W. Michor, and Jan Slovák.  _Natural Operations in Differential Geometry_.  Springer-Verlag, 1993.
  * Kondor and Trivedi (2018) Risi Kondor and Shubhendu Trivedi.  On the generalization of equivariance and convolution in neural networks to the action of compact groups.  In _International Conference on Machine Learning_ , pages 2747–2755. PMLR, 2018.
  * Koopman (1931) B. O. Koopman.  Hamiltonian systems and transformations in Hilbert space.  _Proceedings of the National Academy of Sciences_ , 17:315–318, 1931.
  * Koralov and Sinai (2012) Leonid B. Koralov and Yakov G. Sinai.  _Theory of Probability and Random Processes_.  Springer, 2 edition, 2012.
  * LeCun et al. (1989) Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel.  Backpropagation applied to handwritten zip code recognition.  _Neural computation_ , 1(4):541–551, 1989.
  * Lee (2013) John M. Lee.  _Introduction to Smooth Manifolds: Second Edition_.  Springer, 2013.
  * Lezcano-Casado and Martınez-Rubio (2019) Mario Lezcano-Casado and David Martınez-Rubio.  Cheap orthogonal constraints in neural networks: A simple parametrization of the orthogonal and unitary group.  In _International Conference on Machine Learning_ , pages 3794–3803. PMLR, 2019.
  * Liu and Tegmark (2022) Ziming Liu and Max Tegmark.  Machine learning hidden symmetries.  _Phys. Rev. Lett._ , 128:180201, May 2022.  doi: 10.1103/PhysRevLett.128.180201.  URL https://link.aps.org/doi/10.1103/PhysRevLett.128.180201.
  * Loiseau and Brunton (2018) J.-C. Loiseau and S. L. Brunton.  Constrained sparse Galerkin regression.  _Journal of Fluid Mechanics_ , 838:42–67, 2018.
  * Maron et al. (2018) Haggai Maron, Heli Ben-Hamu, Nadav Shamir, and Yaron Lipman.  Invariant and equivariant graph networks.  _arXiv preprint arXiv:1812.09902_ , 2018.
  * Marsden and Ratiu (1999) J. E. Marsden and T. S. Ratiu.  _Introduction to mechanics and symmetry_.  Springer-Verlag, 2nd edition, 1999.
  * Mauroy et al. (2020) Alexandre Mauroy, Y Susuki, and I Mezić.  _Koopman operator in systems and control_.  Springer, 2020.
  * Messenger and Bortz (2021) Daniel A Messenger and David M Bortz.  Weak SINDy: Galerkin-based data-driven model selection.  _Multiscale Modeling & Simulation_, 19(3):1474–1497, 2021.
  * Meyer (2000) Carl D Meyer.  _Matrix analysis and applied linear algebra_ , volume 71.  Siam, 2000.
  * Mezić (2005) Igor Mezić.  Spectral properties of dynamical systems, model reduction and decompositions.  _Nonlinear Dynamics_ , 41:309–325, 2005.
  * Miao and Rao (2007) Xu Miao and Rajesh PN Rao.  Learning the Lie groups of visual invariance.  _Neural computation_ , 19(10):2665–2693, 2007\.
  * Mohan and Fazel (2012) Karthik Mohan and Maryam Fazel.  Iterative reweighted algorithms for matrix rank minimization.  _Journal of Machine Learning Research_ , 13(1):3441–3473, 2012.
  * Moskalev et al. (2022) Artem Moskalev, Anna Sepliarskaia, Ivan Sosnovik, and Arnold Smeulders.  LieGG: Studying learned Lie group generators.  _Advances in Neural Information Processing Systems_ , 35:25212–25223, 2022.
  * Noether (1918) E Noether.  Invariante variationsprobleme nachr. d. könig. gesellsch. d. wiss. zu göttingen, math-phys. klasse 1918: 235-257.  _English Reprint: physics/0503066, http://dx. doi. org/10.1080/00411457108231446_ , page 57, 1918.
  * Otto and Rowley (2021) Samuel E Otto and Clarence W Rowley.  Koopman operators for estimation and control of dynamical systems.  _Annual Review of Control, Robotics, and Autonomous Systems_ , 4:59–87, 2021.
  * Peitz et al. (2023) Sebastian Peitz, Hans Harder, Feliks Nüske, Friedrich Philipp, Manuel Schaller, and Karl Worthmann.  Partial observations, coarse graining and equivariance in Koopman operator theory for large-scale dynamical systems.  _arXiv preprint arXiv:2307.15325_ , 2023.
  * Raissi et al. (2019) Maziar Raissi, Paris Perdikaris, and George Em Karniadakis.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.  _Journal of Computational physics_ , 378:686–707, 2019\.
  * Rao and Ruderman (1999) Rajesh Rao and Daniel Ruderman.  Learning Lie groups for invariant visual perception.  _Advances in neural information processing systems_ , 11, 1999.
  * Recht et al. (2010) Benjamin Recht, Maryam Fazel, and Pablo A. Parrilo.  Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization.  _SIAM Review_ , 52(3):471–501, 2010.
  * Reinbold et al. (2020) Patrick AK Reinbold, Daniel R Gurevich, and Roman O Grigoriev.  Using noisy or incomplete data to discover models of spatiotemporal dynamics.  _Physical Review E_ , 101(010203), 2020.
  * Rowley et al. (2003) Clarence W. Rowley, Ioannis G. Kevrekidis, Jerrold E. Marsden, and Kurt Lust.  Reduction and reconstruction for self-similar dynamical systems.  _Nonlinearity_ , 16(4):1257, 2003.
  * Shorten and Khoshgoftaar (2019) Connor Shorten and Taghi M Khoshgoftaar.  A survey on image data augmentation for deep learning.  _Journal of big data_ , 6(1):1–48, 2019.
  * Steyert (2022) Vivian T Steyert.  _Uncovering Structure with Data-driven Reduced-Order Modeling_.  PhD thesis, Princeton University, 2022.
  * Tao (2011) Terence Tao.  Two small facts about Lie groups.  https://terrytao.wordpress.com/2011/06/25/two-small-facts-about-lie-groups/, 6 2011.
  * Van Dyk and Meng (2001) David A Van Dyk and Xiao-Li Meng.  The art of data augmentation.  _Journal of Computational and Graphical Statistics_ , 10(1):1–50, 2001.
  * Wang et al. (2022) Rui Wang, Robin Walters, and Rose Yu.  Approximately equivariant networks for imperfectly symmetric dynamics.  In _International Conference on Machine Learning_ , pages 23078–23091. PMLR, 2022.
  * Weiler and Cesa (2019) Maurice Weiler and Gabriele Cesa.  General E(2)-equivariant steerable CNNs.  _Advances in neural information processing systems_ , 32, 2019.
  * Weiler et al. (2018) Maurice Weiler, Mario Geiger, Max Welling, Wouter Boomsma, and Taco S Cohen.  3D steerable CNNs: Learning rotationally equivariant features in volumetric data.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Williams et al. (2015) Matthew O Williams, Ioannis G Kevrekidis, and Clarence W Rowley.  A data-driven approximation of the Koopman operator: extending dynamic mode decomposition.  _Journal of Nonlinear Science_ , 6:1307–1346, 2015.
  * Yang et al. (2023) Jianke Yang, Robin Walters, Nima Dehmamy, and Rose Yu.  Generative adversarial symmetry discovery.  _arXiv preprint arXiv:2302.00236_ , 2023.
  * Yuan and Lin (2006) Ming Yuan and Yi Lin.  Model selection and estimation in regression with grouped variables.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 68(1):49–67, 2006.

##  Appendix A Proofs of minor results

Proof [Proposition 6] Obviously, if ${\mathcal{K}}_{g}K=K$ then
${\mathcal{K}}_{g}^{({\mathcal{W}})}\circ{\mathcal{T}}_{K}\circ{\mathcal{K}}_{g^{-1}}^{({\mathcal{V}})}={\mathcal{T}}_{K}$.
On the other hand, suppose that ${\mathcal{K}}_{g}K(x_{0},y_{0})\neq
K(x_{0},y_{0})$ for some $(x_{0},y_{0})\in\mathbb{R}^{n}\times\mathbb{R}^{m}$.
Hence, there are vectors $v\in{\mathcal{V}}$ and $w\in{\mathcal{W}}^{*}$ such
that $\langle w,\ {\mathcal{K}}_{g}K(x_{0},y_{0})v-K(x_{0},y_{0})v\rangle>0$.
This remains true for all $y$ in a neighborhood ${\mathcal{U}}$ of $y_{0}$ by
continuity of $K$ and ${\mathcal{K}}_{g}K$. Letting $F(x)=v\varphi(x)$ where
$\varphi$ is a smooth, nonnegative, function with $\varphi(y_{0})>0$ and
support in ${\mathcal{U}}$, we obtain

$\big{\langle}w,\
{\mathcal{K}}_{g}^{({\mathcal{W}})}\circ{\mathcal{T}}_{K}\circ{\mathcal{K}}_{g^{-1}}^{({\mathcal{V}})}F(x)-{\mathcal{T}}_{K}F(x)\big{\rangle}=\int_{\mathbb{R}^{m}}\big{\langle}w,\
{\mathcal{K}}_{g}K(x,y)v-K(x,y)v\big{\rangle}\varphi(y)\operatorname{\mathrm{d}}y>0,$
(129)

meaning
${\mathcal{K}}_{g}^{({\mathcal{W}})}\circ{\mathcal{T}}_{K}\circ{\mathcal{K}}_{g^{-1}}^{({\mathcal{V}})}\neq{\mathcal{T}}_{K}$.
Therefore, ${\mathcal{K}}_{g}K=K$ if and only if
${\mathcal{K}}_{g}^{({\mathcal{W}})}\circ{\mathcal{T}}_{K}\circ{\mathcal{K}}_{g^{-1}}^{({\mathcal{V}})}={\mathcal{T}}_{K}$.

Proof [Proposition 15] As an intersection of closed subgroups,
$H:=\bigcap_{l=1}^{L}\operatorname{Sym}_{G}\big{(}F^{(l)}\big{)}$ is a closed
subgroup of $G$. By the closed subgroup theorem (see Theorem 20.12 in Lee
(2013)), $H$ is an embedded Lie subgroup, whose Lie subalgebra we denote by
$\mathfrak{h}$. If $\xi\in\mathfrak{h}$ then
$\exp(t\xi)\in\operatorname{Sym}_{G}\big{(}F^{(l)}\big{)}$ for all
$t\in\mathbb{R}$ and every $l=1,\ldots,L$. Differentiating
${\mathcal{K}}_{\exp(t\xi)}F^{(l)}=F^{(l)}$ at $t=0$ proves that
${\mathcal{L}}_{\xi}F^{(l)}=0$, i.e.,
$\xi\in\operatorname{\mathfrak{sym}}_{G}(F^{(l)})$ by Theorem 4. Conversely,
if ${\mathcal{L}}_{\xi}F^{(l)}=0$ for every $l=1,\ldots,L$, then by Theorem 4,
$\exp(t\xi)\in H$. Since $H$ is a Lie subgroup, differentiating $\exp(t\xi)$
at $t=0$ proves that $\xi\in\mathfrak{h}$.

Proof [Proposition 16] Let $f_{1},\ldots,f_{N}$ be a basis for
${\mathcal{F}}^{\prime}$. Consider the sequence of Gram matrices
${\boldsymbol{G}}_{M}$ with entries

$[{\boldsymbol{G}}_{M}]_{i,j}=\left\langle f_{i},\
f_{j}\right\rangle_{L^{2}(\mu_{M})}.$ (130)

It suffices to show that ${\boldsymbol{G}}_{M}$ is positive-definite for
sufficiently large $M$. Since the $L^{2}(\mu)$ inner product is positive-
definite on ${\mathcal{F}}^{\prime}$, it follows that the Gram matrix
${\boldsymbol{G}}$ with entries

$[{\boldsymbol{G}}]_{i,j}=\left\langle f_{i},\
f_{j}\right\rangle_{L^{2}(\mu)}$ (131)

is positive-definite. Hence, its smallest eigenvalue
$\lambda_{\text{min}}({\boldsymbol{G}})$ is positive. Since the ordered
eigenvalues of symmetric matrices are continuous with respect to their entries
(see Corollary 4.3.15 in Horn and Johnson (2013)) and
$[{\boldsymbol{G}}_{M}]_{i,j}\to[{\boldsymbol{G}}]_{i,j}$ for all $1\leq
i,j\leq N$ by assumption, we have
$\lambda_{\text{min}}({\boldsymbol{G}}_{M})\to\lambda_{\text{min}}({\boldsymbol{G}})$
as $M\to\infty$. Therefore, there is an $M_{0}$ so that for every $M\geq
M_{0}$ we have $\lambda_{\text{min}}({\boldsymbol{G}}_{M})>0$, i.e.,
${\boldsymbol{G}}_{M}$ is positive-definite.

Proof [Proposition 29] Consider a leaf ${\mathcal{M}}$ of an $m$-dimensional
foliation on the $n$-dimensional manifold ${\mathcal{N}}$ and let
$\gamma:[a,b]\to{\mathcal{N}}$ be a smooth curve satisfying
$\gamma((a,b))\subset{\mathcal{M}}$. First, it is clear that ${\mathcal{M}}$
is a weakly embedded submanifold of ${\mathcal{N}}$ since ${\mathcal{M}}$ is
an integral manifold of an involutive distribution (Lee (2013, Proposition
19.19)) and the local structure theorem for integral manifolds (Lee (2013,
Proposition 19.16)) shows that they are weakly embedded.

By continuity of $\gamma$, any neighborhood of $\gamma(b)$ in ${\mathcal{N}}$
must have nonempty intersection with ${\mathcal{M}}$. By definition of a
foliation (see Lee (2013)), there is a coordinate chart
$({\mathcal{U}},{\boldsymbol{x}})$ for ${\mathcal{N}}$ with
$\gamma(b)\in{\mathcal{U}}$ such that ${\boldsymbol{x}}({\mathcal{U}})$ is a
coordinate-aligned cube in $\mathbb{R}^{n}$ and
${\mathcal{M}}\cap{\mathcal{U}}$ consists of countably many slices of the form
$x^{m+1}=c^{m+1},\ldots,x^{n}=c^{n}$ for constants $c^{m+1},\ldots,c^{n}$.
Since $\gamma$ is continuous, there is a $\delta>0$ so that
$\gamma((b-\delta,b])\subset{\mathcal{U}}$, and in particular,
$\gamma((b-\delta,b))\subset{\mathcal{M}}\cap{\mathcal{U}}$. By continuity of
$\gamma$, there are constants $c^{m+1},\ldots,c^{n}$ such that
$x^{i}(\gamma(t))=c^{i}$ for every $i=m+1,\ldots,n$ and $t\in(b-\delta,b)$.
Hence, we have

$x^{i}(\gamma(b))=\lim_{t\to b}x^{i}(\gamma(t))=c^{i},\qquad i=m+1,\ldots,n,$
(132)

meaning that $\gamma(b)\in{\mathcal{M}}$. An analogous argument shows that
$\gamma(a)\in{\mathcal{M}}$, completing the proof that ${\mathcal{M}}$ is
arcwise-closed.

##  Appendix B Proof of Proposition 17

Our proof relies on the following lemma:

###### Lemma 36

Let ${\mathcal{P}}$ denote a finite-dimensional vector space of polynomials
$\mathbb{R}^{m}\to\mathbb{R}$. If $M\geq\dim({\mathcal{P}})$ then the
evaluation map $T:{\mathcal{P}}\to\mathbb{R}^{M}$ defined by

$T_{(x_{1},\ldots,x_{M})}:P\mapsto(P(x_{1}),\ldots,P(x_{M}))$ (133)

is injective for almost every $(x_{1},\ldots,x_{M})\in(\mathbb{R}^{m})^{M}$
with respect to Lebesgue measure.

Proof  Letting $M_{0}=\dim({\mathcal{P}})$ and choosing a basis
$P_{1},\ldots,P_{M_{0}}$ for ${\mathcal{P}}$, injectivity of
$T_{(x_{1},\ldots,x_{M})}$ is equivalent to injectivity of the $M\times M_{0}$
matrix

${\boldsymbol{T}}_{(x_{1},\ldots,x_{M})}=\begin{bmatrix}P_{1}(x_{1})&\cdots&P_{M_{0}}(x_{1})\\\
\vdots&\ddots&\vdots\\\ P_{1}(x_{M})&\cdots&P_{M_{0}}(x_{M})\end{bmatrix}.$
(134)

Finally, this is equivalent to

$\phi(x_{1},\ldots,x_{M})=\det\big{(}({\boldsymbol{T}}_{(x_{1},\ldots,x_{M})})^{T}{\boldsymbol{T}}_{(x_{1},\ldots,x_{M})}\big{)}$
(135)

taking a nonzero value. We observe that $\phi$ is a polynomial on the
Euclidean space $(\mathbb{R}^{m})^{M}$.

Suppose there exists a set of points
$(\bar{x}_{1},\ldots,\bar{x}_{M})\in(\mathbb{R}^{m})^{M}$ such that
$T_{(\bar{x}_{1},\ldots,\bar{x}_{M})}$ is injective. Then for this set
$\phi(\bar{x}_{1},\ldots,\bar{x}_{M})\neq 0$. Obviously, $\phi(0,\ldots,0)=0$,
meaning that $\phi$ cannot be constant. Thanks to the main result in Caron and
Traynor (2005), this means that each level set of $\phi$ has zero Lebesgue
measure in $(\mathbb{R}^{m})^{M}$. In particular, the level set
$\phi^{-1}(0)$, consisting of those $x_{1},\ldots,x_{M}$ for which
$T_{(x_{1},\ldots,x_{M})}$ fails to be injective, has zero Lebesgue measure.
Therefore, it suffices to prove that there exists
$(\bar{x}_{1},\ldots,\bar{x}_{M})\in(\mathbb{R}^{m})^{M}$ such that
$T_{(\bar{x}_{1},\ldots,\bar{x}_{M})}$ is injective. We do this by induction.

It is clear that there exists $\bar{x}_{1}$ so that the $1\times 1$ matrix

${\boldsymbol{T}}_{1}=\begin{bmatrix}P_{1}(\bar{x}_{1})\end{bmatrix}$ (136)

has full rank since $P_{1}$ cannot be the zero polynomial. Proceeding by
induction, we assume that there exists $\bar{x}_{1},\ldots,\bar{x}_{s}$ so
that

${\boldsymbol{T}}_{s}=\begin{bmatrix}P_{1}(\bar{x}_{1})&\cdots&P_{s}(\bar{x}_{1})\\\
\vdots&\ddots&\vdots\\\
P_{1}(\bar{x}_{s})&\cdots&P_{s}(\bar{x}_{s})\end{bmatrix}$ (137)

has full rank. Suppose that the matrix

${\boldsymbol{\tilde{T}}}_{s+1}(x)=\begin{bmatrix}P_{1}(\bar{x}_{1})&\cdots&P_{s}(\bar{x}_{1})&P_{s+1}(\bar{x}_{1})\\\
\vdots&\ddots&\vdots&\vdots\\\
P_{1}(\bar{x}_{s})&\cdots&P_{s}(\bar{x}_{s})&P_{s+1}(\bar{x}_{s})\\\
P_{1}(x)&\cdots&P_{s}(x)&P_{s+1}(x)\end{bmatrix}$ (138)

has rank $<s+1$ for every $x\in\mathbb{R}^{m}$. Since the upper left $s\times
s$ block of ${\boldsymbol{\tilde{T}}}_{s+1}(x)$ is ${\boldsymbol{T}}_{s}$, we
must always have $\operatorname{rank}({\boldsymbol{\tilde{T}}}_{s+1}(x))=s$.
The nullspace of ${\boldsymbol{\tilde{T}}}_{s+1}(x)$ is contained in the
nullspace of the upper $s\times(s+1)$ block of
${\boldsymbol{\tilde{T}}}_{s+1}(x)$. Since both nullspaces are one-
dimensional, they are equal. The upper $s\times(s+1)$ block of
${\boldsymbol{\tilde{T}}}_{s+1}(x)$ does not depend on $x$, so there is a
fixed nonzero vector ${\boldsymbol{v}}\in\mathbb{R}^{s+1}$ so that
${\boldsymbol{\tilde{T}}}_{s+1}(x){\boldsymbol{v}}={\boldsymbol{0}}$ for every
$x\in\mathbb{R}^{m}$. The last row of this expression reads

$v_{1}P_{1}(x)+\cdots+v_{s+1}P_{s+1}(x)=0\qquad\forall x\in\mathbb{R}^{m},$
(139)

contradicting the linear independence of $P_{1},\ldots,P_{s+1}$. Therefore
there exists $\bar{x}_{s+1}$ so that
${\boldsymbol{T}}_{s+1}={\boldsymbol{\tilde{T}}}_{s+1}(\bar{x}_{s+1})$ has
full rank. It follows by induction on $s$ that there exists
$\bar{x}_{1},\ldots\bar{x}_{M_{0}}\in\mathbb{R}^{m}$ so that
${\boldsymbol{T}}_{(\bar{x}_{1},\ldots\bar{x}_{M_{0}})}={\boldsymbol{T}}_{M_{0}}$
has full rank. Choosing any $M-M_{0}$ additional vectors yields an injective
${\boldsymbol{T}}_{(\bar{x}_{1},\ldots\bar{x}_{M})}$, which completes the
proof.

Proof [Proposition 17] The sum in Eq. 75 clearly defines a symmetric,
positive-semidefinite bilinear form on ${\mathcal{F}}^{\prime}$. It remains to
show that this bilinear form is positive-definite. Suppose that there is a
function $f\in{\mathcal{F}}^{\prime}$ such that $\langle
f,f\rangle_{L^{2}(\mu_{M})}=0$. Thanks to Lemma 36, our assumption that
$M\geq\dim(\pi_{i}({\mathcal{F}}^{\prime}))$ means that the evaluation
operator $T_{(x_{1},\ldots,x_{M})}$ is injective on
$\pi_{i}({\mathcal{F}}^{\prime})$ for almost every
$(x_{1},\ldots,x_{M})\in(\mathbb{R}^{m})^{M}$ with respect to Lebesgue
measure. Since a countable (in this case finite) intersection of sets of
measure zero has measure zero, it follows that for almost every
$(x_{1},\ldots,x_{M})\in(\mathbb{R}^{m})^{M}$ with respect to Lebesgue
measure, $T_{(x_{1},\ldots,x_{M})}$ is injective on every
$\pi_{i}({\mathcal{F}}^{\prime})$, $i=1,\ldots,n$. Defining the positive
diagonal matrix

${\boldsymbol{D}}=\frac{1}{\sqrt{N}}\begin{bmatrix}\sqrt{w_{1}}&&\\\
&\ddots&\\\ &&\sqrt{w_{M}}\end{bmatrix},$ (140)

and using Eq. 75 yields

$0=\langle
f,f\rangle_{L^{2}(\mu_{M})}=\sum_{j=1}^{n}\big{(}{\boldsymbol{D}}T_{(x_{1},\ldots,x_{M})}\pi_{j}f\big{)}^{T}{\boldsymbol{D}}T_{(x_{1},\ldots,x_{M})}\pi_{j}f.$
(141)

This implies that $T_{(x_{1},\ldots,x_{M})}\pi_{j}f={\boldsymbol{0}}$ for
$j=1,\ldots,n$. Since $T_{(x_{1},\ldots,x_{M})}$ is injective on each
$\pi_{j}({\mathcal{F}}^{\prime})$ it follows that each $\pi_{j}f=0$, meaning
that $f=0$. This completes the proof.

##  Appendix C Proof of Proposition 20

We begin by proving

$\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{K}}_{\exp(t\xi)}F={\mathcal{L}}_{\xi}{\mathcal{K}}_{\exp(t\xi)}F={\mathcal{K}}_{\exp(t\xi)}{\mathcal{L}}_{\xi}F.$
(142)

The first equality follows from the composition law for the operators
${\mathcal{K}}_{g}$ and the definition of the Lie derivative in Eq. 83,
yielding

$\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=t_{0}}{\mathcal{K}}_{\exp(t\xi)}F=\lim_{t\to
0}\frac{1}{t}\left[{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t_{0}\xi)}F-{\mathcal{K}}_{\exp(t_{0}\xi)}F\right]={\mathcal{L}}_{\xi}{\mathcal{K}}_{\exp(t_{0}\xi)}F.$
(143)

To prove the second equality, we choose $p\in{\mathcal{M}}$, let
$p^{\prime}=\theta_{\exp(t_{0}\xi)}(p)$, and compute

$\displaystyle\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=t_{0}}\big{(}{\mathcal{K}}_{\exp(t\xi)}F\big{)}(p)$
$\displaystyle=\lim_{t\to
0}\frac{1}{t}\left[\big{(}{\mathcal{K}}_{\exp(t_{0}\xi)}{\mathcal{K}}_{\exp(t\xi)}F\big{)}(p)-\big{(}{\mathcal{K}}_{\exp(t_{0}\xi)}F\big{)}(p)\right]$
(144) $\displaystyle=\lim_{t\to
0}\frac{1}{t}\Theta_{\exp(-t_{0}\xi)}\circ\left({\mathcal{K}}_{\exp(t\xi)}F-F\right)\circ\theta_{\exp(t_{0}\xi)}(p)$
$\displaystyle=\Theta_{\exp(-t_{0}\xi)}\left(\lim_{t\to
0}\frac{1}{t}\left[\big{(}{\mathcal{K}}_{\exp(t\xi)}F\big{)}(p^{\prime})-F(p^{\prime})\right]\right)$
$\displaystyle=\Theta_{\exp(-t_{0}\xi)}\left({\mathcal{L}}_{\xi}F(p^{\prime})\right)=\big{(}{\mathcal{K}}_{\exp(t_{0}\xi)}{\mathcal{L}}_{\xi}F\big{)}(p).$

In the third equality, we used the fact that $\Theta_{\exp(-t_{0}\xi)}$ is a
continuous vector bundle homomorphism.

Next, we prove

${\mathcal{L}}_{\alpha\xi+\beta\eta}=\alpha{\mathcal{L}}_{\xi}+\beta{\mathcal{L}}_{\eta}.$
(145)

To do this, we choose $F\in\operatorname{\Sigma}(E)$, $p\in{\mathcal{M}}$, and
define the map $h:G\to E_{p}$ by

$h:g\mapsto{\mathcal{K}}_{g}F(p)=\Theta\big{(}F(\theta(p,g)),g^{-1}\big{)}.$
(146)

As a composition of smooth maps, $h$ is smooth, and its derivative at the
identity is

$\operatorname{\mathrm{d}}h(e)\xi_{e}=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}h(\exp(t\xi))={\mathcal{L}}_{\xi}F(p)$
(147)

for every $\xi_{e}\in T_{e}G\cong\operatorname{Lie}(G)$. Since the derivative
is linear, it follows that $\xi\mapsto{\mathcal{L}}_{\xi}F(p)$ is linear.

Finally, we prove that

${\mathcal{L}}_{[\xi,\eta]}=\frac{1}{2}\left.\frac{\operatorname{\mathrm{d}}^{2}}{\operatorname{\mathrm{d}}t^{2}}\right|_{t=0}{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}={\mathcal{L}}_{\xi}{\mathcal{L}}_{\eta}-{\mathcal{L}}_{\eta}{\mathcal{L}}_{\xi}.$
(148)

Recall that $\operatorname{Fl}_{\xi}^{t}:g\mapsto g\cdot\exp(t\xi)$ gives the
flow of the left-invariant vector field $\xi\in\operatorname{Lie}(G)$ (see
Theorem 4.18(3) in Kolář et al. (1993)). By Theorem 3.16 in Kolář et al.
(1993) the curve $\gamma:\mathbb{R}\to G$ given by

$\gamma(t)=\operatorname{Fl}_{-\eta}^{t}\circ\operatorname{Fl}_{-\xi}^{t}\circ\operatorname{Fl}_{\eta}^{t}\circ\operatorname{Fl}_{\xi}^{t}(e)=\exp(t\xi)\exp(t\eta)\exp(-t\xi)\exp(-t\eta).$
(149)

satisfies $\gamma(0)=e$, $\gamma^{\prime}(0)=0$, and

$\frac{1}{2}\gamma^{\prime\prime}(0)=[\xi,\eta]_{e}\in T_{e}G$ (150)

in the sense that
$\gamma^{\prime\prime}(0):f\mapsto(f\circ\gamma)^{\prime\prime}(0)$ is a
derivation on $C^{\infty}(G)$. Composing with the map in Eq. 146 yields

$0=\operatorname{\mathrm{d}}h(e)\gamma^{\prime}(0)=(h\circ\gamma)^{\prime}(0)=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}{\mathcal{K}}_{\gamma(t)}F(p).$
(151)

Combining Eq. 150 and Eq. 147 (noting the definition of the tangent map
$\operatorname{\mathrm{d}}h(e)$ acting on derivations, as in Kolář et al.
(1993), Lee (2013)) gives

${\mathcal{L}}_{[\xi,\eta]}F(p)=\frac{1}{2}\operatorname{\mathrm{d}}h(e)\gamma^{\prime\prime}(0)=\frac{1}{2}(h\circ\gamma)^{\prime\prime}(0)=\frac{1}{2}\left.\frac{\operatorname{\mathrm{d}}^{2}}{\operatorname{\mathrm{d}}t^{2}}\right|_{t=0}{\mathcal{K}}_{\gamma(t)}F(p).$
(152)

This proves the first equality in Eq. 148 thanks to the composition law

${\mathcal{K}}_{\gamma(t)}={\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}.$
(153)

To differentiate the above expression, we use the following observations. If
$F_{t}\in\operatorname{\Sigma}(E)$ is such that $(t,p)\mapsto F_{t}(p)$ is
smooth for $(t,p)\in\mathbb{R}\times{\mathcal{M}}$, then obviously
$\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}F_{t}\in\operatorname{\Sigma}(E)$
with the usual identification $TE_{p}\cong E_{p}$. Moreover, we have

$\displaystyle\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{K}}_{g}F_{t}(p)$
$\displaystyle=\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\Theta_{g^{-1}}\big{(}F_{t}(\theta_{g}(p))\big{)}$
(154)
$\displaystyle=\Theta_{g^{-1}}\Big{(}\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}F_{t}(\theta_{g}(p))\Big{)}={\mathcal{K}}_{g}\Big{(}\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}F_{t}\Big{)}(p)$

because $F_{t}(\theta_{g}(p))\in E_{\theta_{g}(p)}$ for all $t\in\mathbb{R}$
and $\Theta_{g^{-1}}$ is linear on $E_{\theta_{g}(p)}$. Using this, we obtain

$\displaystyle\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{L}}_{\xi}F_{t}(p)$
$\displaystyle=\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\left.\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}\tau}\right|_{\tau=0}{\mathcal{K}}_{\exp(\tau\xi)}F_{t}(p)$
(155)
$\displaystyle=\left.\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}\tau}\right|_{\tau=0}\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{K}}_{\exp(\tau\xi)}F_{t}(p)$
$\displaystyle=\left.\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}\tau}\right|_{\tau=0}{\mathcal{K}}_{\exp(\tau\xi)}\Big{(}\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}F_{t}\Big{)}(p)={\mathcal{L}}_{\xi}\Big{(}\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}F_{t}\Big{)}(p)$

because $(t,\tau)\mapsto{\mathcal{K}}_{\exp(\tau\xi)}F_{t}(p)$ lies in the
vector space $E_{p}$, allowing us to exchanged the order of differentiation.
Since
$(t_{1},t_{2},t_{3},t_{4})\mapsto{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}F(p)$
lies in the vector space $E_{p}$ for all
$(t_{1},t_{2},t_{3},t_{4})\in\mathbb{R}^{4}$, we can apply the chain rule and
Eq. 154 to obtain

$\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{K}}_{\gamma(t)}=\left.\frac{\partial}{\partial
t_{1}}\right|_{t_{1}=t}{\mathcal{K}}_{\exp(t_{1}\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}\\\
+{\mathcal{K}}_{\exp(t\xi)}\left.\frac{\partial}{\partial
t_{2}}\right|_{t_{2}=t}{\mathcal{K}}_{\exp(t_{2}\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}\\\
+{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}\left.\frac{\partial}{\partial
t_{3}}\right|_{t_{3}=t}{\mathcal{K}}_{\exp(-t_{3}\xi)}{\mathcal{K}}_{\exp(-t\eta)}\\\
+{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}\left.\frac{\partial}{\partial
t_{4}}\right|_{t_{4}=t}{\mathcal{K}}_{\exp(-t_{4}\eta)}.$ (156)

Using Eq. 142 gives

$\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{K}}_{\gamma(t)}={\mathcal{L}}_{\xi}\overbrace{{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}}^{{\mathcal{K}}_{\gamma(t)}}\\\
+{\mathcal{K}}_{\exp(t\xi)}{\mathcal{L}}_{\eta}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}\\\
+{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{L}}_{-\xi}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}\\\
+\underbrace{{\mathcal{K}}_{\exp(t\xi)}{\mathcal{K}}_{\exp(t\eta)}{\mathcal{K}}_{\exp(-t\xi)}{\mathcal{K}}_{\exp(-t\eta)}}_{{\mathcal{K}}_{\gamma(t)}}{\mathcal{L}}_{-\eta}.$
(157)

Applying the same technique to differentiate a second time and using the
linearity in Eq. 145 to cancel terms yields

$\left.\frac{\operatorname{\mathrm{d}}^{2}}{\operatorname{\mathrm{d}}t^{2}}\right|_{t=0}{\mathcal{K}}_{\gamma(t)}={\mathcal{L}}_{\xi}\underbrace{\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}{\mathcal{K}}_{\gamma(t)}}_{0}+\underbrace{{\mathcal{L}}_{\xi}{\mathcal{L}}_{\eta}+{\mathcal{L}}_{\eta}{\mathcal{L}}_{-\xi}+{\mathcal{L}}_{\eta}{\mathcal{L}}_{-\xi}+{\mathcal{L}}_{-\xi}{\mathcal{L}}_{-\eta}}_{2\big{(}{\mathcal{L}}_{\xi}{\mathcal{L}}_{\eta}-{\mathcal{L}}_{\eta}{\mathcal{L}}_{\xi}\big{)}}+\underbrace{\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}{\mathcal{K}}_{\gamma(t)}}_{0}{\mathcal{L}}_{-\eta},$
(158)

which completes the proof. $\blacksquare$

##  Appendix D Proof of Theorem 21

If $F\in\operatorname{\Sigma}(E)$ is $G_{0}$-equivariant, then
${\mathcal{K}}_{\exp(t\xi)}F=F$ for all $\xi\in\operatorname{Lie}(G)$ and
$t\in\mathbb{R}$. Differentiating with respect to $t$ at $t=0$ gives
${\mathcal{L}}_{\xi}F=0$. Conversely, suppose that ${\mathcal{L}}_{\xi}F=0$
for all $\xi\in\operatorname{Lie}(G)$. Choosing $\xi\in\operatorname{Lie}(G)$
and $p\in{\mathcal{M}}$, the smooth curve $\gamma:\mathbb{R}\to E_{p}$ defined
by $\gamma(t)={\mathcal{K}}_{\exp(t\xi)}F(p)$ satisfies $\gamma(0)=F(p)$ and

$\gamma^{\prime}(t)=\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{K}}_{\exp(t\xi)}F(p)={\mathcal{K}}_{\exp(t\xi)}{\mathcal{L}}_{\xi}F(p)=0\in
E_{p}$ (159)

thanks to Eq. 84 of Proposition 20. Therefore, $\gamma(t)=F(p)$ for all
$t\in\mathbb{R}$, and in particular $\gamma(1)=F(p)$, meaning that

${\mathcal{K}}_{\exp(\xi)}F=F,\qquad\forall\xi\in\operatorname{Lie}(G).$ (160)

We show by induction that ${\mathcal{K}}_{g}F=F$ for every finite product
$g=h_{m},\ldots,h_{1}$ of elements $h_{i}=\exp(\xi_{i})$. Supposing this holds
for any product $g$ of $m$ elements in the range of the exponential map and
choosing $h_{m+1}=\exp(\xi_{m+1})$ we have

${\mathcal{K}}_{h_{m+1}g}F={\mathcal{K}}_{h_{m+1}}{\mathcal{K}}_{g}F={\mathcal{K}}_{h_{m+1}}F=F,$
(161)

proving the claim. The conclusion of the theorem now follows immediately from
Lemma 37, below, since $G$ is the union of its connected components.
$\blacksquare$

###### Lemma 37

Let $G_{0}$ be the identity component of a Lie group $G$. Then every element
$g\in G_{0}$ can be expressed as a finite product $g=h_{m}\cdots h_{1}$ of
elements $h_{i}=\exp(\xi_{i})$ for $\xi_{i}\in\operatorname{Lie}(G)$. Let
$G_{i}$ be a connected component of $G$ and let $g_{i}\in G_{i}$. Then every
element $g\in G_{i}$ can be expressed as $g=g_{0}g_{i}$ for some $g_{0}\in
G_{0}$.

Proof  By the inverse function theorem (more specifically by Proposition
20.8(f) in Lee (2013)), the range of the exponential map contains an open,
connected neighborhood ${\mathcal{U}}$ of the identity element $e\in G$. The
inverses of the elements in ${\mathcal{U}}$ also belong to the range of the
exponential map thanks to Proposition 20.8(c) in Lee (2013). By Proposition
7.14(b) and Proposition 7.15 in Lee (2013), it follows that ${\mathcal{U}}$
generates the identity component $G_{0}$ of $G$. That is, any element $g\in
G_{0}$ can be written as a finite product of elements in ${\mathcal{U}}$ and
their inverses, which proves the first claim.

By Proposition 7.15 in Lee (2013), $G_{0}$ is a normal subgroup of $G$ and
every connected component of $G$ is diffeomorphic to $G_{0}$. In fact in the
proof of this result it is shown that every connected component of $G$ is a
coset of $G_{0}$. Therefore, if $G_{i}$ is a non-identity connected component
of $G$ and $g_{i}\in G_{i}$ then $G_{i}=G_{0}\cdot g_{i}$, which proves the
second claim.

##  Appendix E Proof of Theorem 22

We begin by showing that $\operatorname{Sym}_{G}(F)$ is a closed subgroup of
$G$. It is obviously a subgroup, for if
$g_{1},g_{2}\in\operatorname{Sym}_{G}(F)$ then

${\mathcal{K}}_{g_{1}g_{2}}F={\mathcal{K}}_{g_{1}}{\mathcal{K}}_{g_{2}}F={\mathcal{K}}_{g_{1}}F=F,$
(162)

meaning that $g_{1}g_{2}\in\operatorname{Sym}_{G}(F)$. To show that
$\operatorname{Sym}_{G}(F)$ is closed, we observe that for each
$p\in{\mathcal{M}}$, the map $h_{p}:G\to E$ defined by

$h_{p}:g\mapsto{\mathcal{K}}_{g}F(p)=\Theta\big{(}F(\theta(p,g)),g^{-1}\big{)}$
(163)

is smooth, as it is a composition of smooth maps. As $F(p)$ is a single point
in $E$, the preimage set $h_{p}^{-1}\big{(}\\{F(p)\\}\big{)}$ is closed in
$G$. Since $\operatorname{Sym}_{G}(F)$ is an intersection,

$\operatorname{Sym}_{G}(F)=\bigcap_{p\in{\mathcal{M}}}h_{p}^{-1}\big{(}\\{F(p)\\}\big{)},$
(164)

of closed sets, it follows that $\operatorname{Sym}_{G}(F)$ is closed in $G$.
By the closed subgroup theorem (Theorem 20.12 in Lee (2013)) it follows that
$\operatorname{Sym}_{G}(F)$ is an embedded Lie subgroup of $G$.

Let ${\mathfrak{h}}=\operatorname{Lie}(\operatorname{Sym}_{G}(F))$ be the Lie
algebra of $\operatorname{Sym}_{G}(F)$. By Theorem 21, (or simply by
differentiating ${\mathcal{K}}_{\exp(t\xi)}F=F$ with respect to $t$) we have
${\mathcal{L}}_{\xi}F=0$ for every $\xi\in{\mathfrak{h}}$. This means that
${\mathfrak{h}}\subset\operatorname{\mathfrak{sym}}_{G}(F)$, as defined by Eq.
88. To show the reverse containment, choose
$\xi\in\operatorname{\mathfrak{sym}}_{G}(F)$, and observe that Eq. 84 in
Proposition 20 implies that

$\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}{\mathcal{K}}_{\exp(t\xi)}F={\mathcal{K}}_{\exp(t\xi)}{\mathcal{L}}_{\xi}F=0\qquad\forall
t\in\mathbb{R}.$ (165)

It follows that ${\mathcal{K}}_{\exp(t\xi)}F=F$, that is,
$\exp(t\xi)\in\operatorname{Sym}_{G}(F)$ for all $t\in\mathbb{R}$.
Differentiating at $t=0$ proves that $\xi\in{\mathfrak{h}}$. Therefore,
${\mathfrak{h}}=\operatorname{\mathfrak{sym}}_{G}(F)$, completing the proof.
$\blacksquare$

##  Appendix F Proof of Theorem 30

Our proof of the theorem relies on the following technical lemma concerning
the integral curves of vector fields tangent to weakly embedded, arcwise-
closed submanifolds.

###### Lemma 38

Let ${\mathcal{M}}$ be an arcwise-closed weakly embedded submanifold of a
manifold ${\mathcal{N}}$. Let $V\in\mathfrak{X}({\mathcal{N}})$ be a vector
field tangent to ${\mathcal{M}}$, that is

$V_{p}\in T_{p}{\mathcal{M}}\qquad\forall p\in{\mathcal{M}}.$ (166)

If $\gamma:I\to{\mathcal{N}}$ is a maximal integral curve of $V$ that
intersects ${\mathcal{M}}$, then $\gamma$ lies in ${\mathcal{M}}$.

Proof  By the translation lemma (Lemma 9.4 in Lee (2013)), we can assume
without loss of generality that $0\in I$ and
$p_{0}=\gamma(0)\in{\mathcal{M}}$. Let
$\imath_{{\mathcal{M}}}:{\mathcal{M}}\hookrightarrow{\mathcal{N}}$ denote the
inclusion map. Since ${\mathcal{M}}$ is an immersed submanifold of
${\mathcal{N}}$ and $V$ is tangent to ${\mathcal{M}}$, there is a unique
smooth vector field $V|_{{\mathcal{M}}}\in\mathfrak{X}({\mathcal{M}})$ that is
$\imath_{{\mathcal{M}}}$-related to $V$ thanks to Proposition 8.23 in Lee
(2013). Let $\tilde{\gamma}:\tilde{I}\to{\mathcal{M}}$ be the maximal integral
curve of $V|_{{\mathcal{M}}}$ with $\tilde{\gamma}(0)=p_{0}$. By the
naturality of integral curves (Proposition 9.6 in Lee (2013))
$\imath_{{\mathcal{M}}}\circ\tilde{\gamma}$ is an integral curve of $V$ with
$\imath_{{\mathcal{M}}}\circ\tilde{\gamma}(0)=p_{0}$. Since integral curves of
smooth vector fields starting at the same point are unique (Theorem 9.12, part
(a) in Lee (2013)) we have $\tilde{I}\subset I$ and

$\imath_{{\mathcal{M}}}\circ\tilde{\gamma}(t)=\gamma(t)\qquad\forall
t\in\tilde{I}.$ (167)

Therefore, it remains to show that $\tilde{I}=I$.

By the local existence of integral curves (Proposition 9.2 in Lee (2013)), the
domains $I$ and $\tilde{I}$ of the maximal integral curves $\gamma$ and
$\tilde{\gamma}$ are open intervals in $\mathbb{R}$. Suppose, for the sake of
producing a contradiction, that there exists $t\in I$ with $t>\tilde{I}$. Then
it follows that the least upper bound $b=\sup\tilde{I}$ is an element of $I$.
By Eq. 167 and continuity of $\gamma$ we have

$q_{0}=\gamma(b)=\lim_{t\to b}\imath_{{\mathcal{M}}}\circ\tilde{\gamma}(t).$
(168)

Since ${\mathcal{M}}$ is arcwise-closed, it follows that
$q_{0}\in{\mathcal{M}}$.

To complete the proof, we use the local existence of an integral curve for
$V|_{{\mathcal{M}}}$ starting at $q_{0}$ to contradict the maximality of
$\tilde{\gamma}$. By the local existence of integral curves (Proposition 9.2
in Lee (2013)) and the translation lemma (Lemma 9.4 in Lee (2013)), there is
an $\varepsilon>0$ and an integral curve
$\hat{\gamma}:(b-\varepsilon,b+\varepsilon)\to{\mathcal{M}}$ of
$V|_{{\mathcal{M}}}$ such that $\hat{\gamma}(b)=q_{0}=\gamma(b)$. Shrinking
the interval, we take $0<\varepsilon<b-a$. Again, by nauturality and
uniqueness of integral curves we must have
$\imath_{{\mathcal{M}}}\circ\hat{\gamma}(t)=\gamma(t)$ for all
$t\in(b-\varepsilon,b+\varepsilon)$. Hence, by Eq. 167 and injectivity of
$\imath_{{\mathcal{M}}}$ it follows that $\hat{\gamma}(t)=\tilde{\gamma}(t)$
for all $t\in(b-\varepsilon,b)$. Applying the gluing lemma (Corollary 2.8 in
Lee (2013)) to $\tilde{\gamma}$ and $\hat{\gamma}$ yields an extension of
$\tilde{\gamma}$ to the larger open interval
$\tilde{I}\cup(b-\varepsilon,b+\varepsilon)$. Since this contradicts the
maximality of $\tilde{\gamma}$, there is no $t\in I$ with $t>\tilde{I}$. The
same argument shows that there is no $t\in I$ with $t<\tilde{I}$, and so we
must have $\tilde{I}=I$.

Proof [Theorem 30] First, suppose that ${\mathcal{M}}$ is $G_{0}$-invariant.
In particular, this means that for every $p\in{\mathcal{M}}$ and
$\xi\in\operatorname{Lie}(G)$, the smooth curve
$\gamma_{\xi}^{(p)}:\mathbb{R}\to{\mathcal{N}}$ defined by

$\gamma_{\xi}^{(p)}(t)=p\cdot\exp(t\xi)$ (169)

lies in ${\mathcal{M}}$. Since ${\mathcal{M}}$ is weakly embedded in
${\mathcal{N}}$, $\gamma_{\xi}^{(p)}$ is also smooth as a map into
${\mathcal{M}}$. Specifically, there is a smooth curve
$\tilde{\gamma}_{\xi}^{(p)}:\mathbb{R}\to{\mathcal{M}}$ so that
$\gamma_{\xi}^{(p)}=\imath_{{\mathcal{M}}}\circ\tilde{\gamma}_{\xi}^{(p)}$
where $\imath_{{\mathcal{M}}}:{\mathcal{M}}\hookrightarrow{\mathcal{N}}$ is
the inclusion map. Differentiating at $t=0$ yields

$\hat{\theta}(\xi)_{p}=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\gamma_{\xi}^{(p)}(t)\right|_{t=0}=\operatorname{\mathrm{d}}\imath_{{\mathcal{M}}}(p)\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\tilde{\gamma}_{\xi}^{(p)}(t)\right|_{t=0},$
(170)

which lies in
$T_{p}{\mathcal{M}}=\operatorname{Range}\left(\operatorname{\mathrm{d}}\imath_{{\mathcal{M}}}(p)\right)$.

Conversely, suppose that the tangency condition expressed in Eq. 123 holds. By
Lemma 20.14 in Lee (2013) the vector fields $\xi\in\operatorname{Lie}(G)$ and
$\hat{\xi}=\hat{\theta}(\xi)$ are $\theta^{(p)}$-related for every
$p\in{\mathcal{N}}$. By the naturality of integral curves (Proposition 9.6 in
Lee (2013)) it follows that $\gamma_{\xi}^{(p)}:\mathbb{R}\to{\mathcal{N}}$
defined by

$\gamma_{\xi}^{(p)}(t)=p\cdot\exp(t\xi)$ (171)

is the unique maximal integral curve of $\hat{\xi}$ passing through $p$ at
$t=0$. When $p\in{\mathcal{M}}$, this integral curve lies in ${\mathcal{M}}$
thanks to Lemma 38. This means that ${\mathcal{M}}$ is invariant under the
action of any group element in the range of the exponential map on $G$.

Proceeding by induction, suppose that $G$ is invariant under the action of any
product of $m$ group elements in the range of the exponential map. If
$g=h_{1}\cdots h_{m}\cdot h_{m+1}$ is a product of $m+1$ elements
$h_{i}\in\operatorname{Range}(\exp)\subset G$, then it follows from
associativity and the induction hypothesis that

${\mathcal{M}}\cdot(h_{1}\cdots h_{m}\cdot h_{m+1})=({\mathcal{M}}\cdot
h_{1}\cdots h_{m})\cdot h_{m+1}\subset{\mathcal{M}}\cdot
h_{m+1}\subset{\mathcal{M}}.$ (172)

Therefore, ${\mathcal{M}}$ is invariant under the action of any finite product
of group elements in the range of the exponential map by induction on $m$.
Thanks to Lemma 37, it follows that ${\mathcal{M}}$ is $G_{0}$-invariant,
proving the first claim of the theorem.

Now suppose in addition that for each connected component $G_{i}$ of $G$ there
is an element $g_{i}\in G_{i}$ satisfying ${\mathcal{M}}\cdot
g_{i}\subset{\mathcal{M}}$. By Lemma 37, we have $G_{i}=G_{0}\cdot g_{i}$.
Therefore,

${\mathcal{M}}\cdot G_{i}={\mathcal{M}}\cdot G_{0}\cdot
g_{i}\subset{\mathcal{M}}\cdot g_{i}\subset{\mathcal{M}},$ (173)

proving that ${\mathcal{M}}$ is invariant under $G_{i}$. Since $G$ is the
union of its connected components it follows that ${\mathcal{M}}$ is
$G$-invariant.

##  Appendix G Proof of Theorem 31

The set $\operatorname{\mathfrak{sym}}_{G}({\mathcal{M}})$ is a subspace of
$\operatorname{Lie}(G)$, for if
$\xi_{1},\xi_{2}\in\operatorname{\mathfrak{sym}}_{G}({\mathcal{M}})$ and
$a_{1},a_{2}\in\mathbb{R}$ then

$\hat{\theta}(a_{1}\xi_{1}+a_{2}\xi_{2})_{p}=a_{1}\underbrace{\hat{\theta}(\xi_{1})}_{\in
T_{p}{\mathcal{M}}}+a_{2}\underbrace{\hat{\theta}(\xi_{2})}_{\in
T_{p}{\mathcal{M}}}\in T_{p}{\mathcal{M}}$ (174)

thanks to linearity of the infinitesimal generator $\hat{\theta}$. To show
that $\operatorname{\mathfrak{sym}}_{G}({\mathcal{M}})$ is a Lie subalgrebra,
we must show that it is also closed under the Lie bracket. Recall that
$\hat{\theta}$ is a Lie algebra homomorphism (see Theorem 20.15 in Lee
(2013)), and so
$\hat{\theta}([\xi_{1},\xi_{2}])=[\hat{\theta}(\xi_{1}),\hat{\theta}(\xi_{1})]$.
Since the Lie bracket of two vector fields tangent to an immersed submanifold
is also tangent to the submanifold (see Corollary 8.32 in Lee (2013)), it
follows that $[\hat{\theta}(\xi_{1}),\hat{\theta}(\xi_{1})]$ is tangent to
${\mathcal{M}}$. Hence, $\operatorname{\mathfrak{sym}}_{G}({\mathcal{M}})$ is
closed under the Lie bracket and is therefore a Lie subalgebra of
$\operatorname{Lie}(G)$. By Theorem 19.26 in Lee (2013), there is a unique
connected Lie subgroup of $\operatorname{Sym}_{G}({\mathcal{M}})_{0}\subset G$
whose Lie subalgebra is $\operatorname{\mathfrak{sym}}_{G}({\mathcal{M}})$.

Now suppose that ${\mathcal{M}}$ is weakly embedded and arcwise-closed in
${\mathcal{N}}$. Since $\operatorname{Sym}_{G}({\mathcal{M}})_{0}$ is
connected, it is equal to its identity component. Applying Theorem 30 to the
induced action of $\operatorname{Sym}_{G}({\mathcal{M}})_{0}$ on
${\mathcal{M}}$, we immediately obtain
${\mathcal{M}}\cdot\operatorname{Sym}_{G}({\mathcal{M}})_{0}\subset{\mathcal{M}}$.
Suppose that $H$ is another connected Lie subgroup of $G$ such that
${\mathcal{M}}\cdot H\subset{\mathcal{M}}$. Choosing any $p\in{\mathcal{M}}$
and $\xi\in\operatorname{Lie}(H)$, we have

$p\cdot\exp(t\xi)\in{\mathcal{M}}\qquad\forall t\in\mathbb{R}.$ (175)

Since ${\mathcal{M}}$ is weakly embedded in ${\mathcal{N}}$, this defines a
smooth curve $\gamma:\mathbb{R}\to{\mathcal{M}}$ such that
$\imath_{{\mathcal{M}}}\circ\gamma(t)=p\cdot\exp(t\xi)$. Differentiating and
using the definition of the infinitesimal generator gives

$\hat{\theta}(\xi)_{p}=\left.\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}\right|_{t=0}p\cdot\exp(t\xi)=\operatorname{\mathrm{d}}\imath_{{\mathcal{M}}}(p)\left.\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}\right|_{t=0}\gamma(t)\in
T_{p}{\mathcal{M}}.$ (176)

Therefore,
$\operatorname{Lie}(H)\subset\operatorname{\mathfrak{sym}}_{G}({\mathcal{M}})$
which implies that $H\subset\operatorname{Sym}_{G}({\mathcal{M}})_{0}$ by
Theorem 19.26 in Lee (2013).

Now suppose that ${\mathcal{M}}$ is properly embedded in ${\mathcal{N}}$ and
denote

$\operatorname{Sym}_{G}({\mathcal{M}})=\\{g\in G\ :{\mathcal{M}}\cdot
g\subset{\mathcal{M}}\\}=\bigcap_{p\in{\mathcal{M}}}\big{(}\theta^{(p)}\big{)}^{-1}({\mathcal{M}}).$
(177)

The equality of these expressions is a simple matter of unwinding their
definitions. It is clear that $\operatorname{Sym}_{G}({\mathcal{M}})$ is a
subgroup of $G$, for if $g_{1},g_{2}\in\operatorname{Sym}_{G}({\mathcal{M}})$
then the composition law for the group action gives
${\mathcal{M}}\cdot(g_{1}\cdot g_{2})=({\mathcal{M}}\cdot g_{1})\cdot
g_{2}\subset{\mathcal{M}}\cdot g_{1}\subset{\mathcal{M}}$. Since
${\mathcal{M}}$ is properly embedded, it is closed in ${\mathcal{M}}$ (see Lee
(2013, Proposition 5.5)), meaning that each preimge set
$\big{(}\theta^{(p)}\big{)}^{-1}({\mathcal{M}})$ is closed in $G$ by
continuity of $\theta^{(p)}$. As an intersection of closed subsets, it follows
that $\operatorname{Sym}_{G}({\mathcal{M}})$ is closed in $G$. By the closed
subgroup theorem (Lee (2013, Theorem 20.12)),
$\operatorname{Sym}_{G}({\mathcal{M}})$ is a properly embedded Lie subgroup of
$G$. The same holds for the identity component $H_{0}$ of
$\operatorname{Sym}_{G}({\mathcal{M}})$ since $H_{0}$ is closed in
$\operatorname{Sym}_{G}({\mathcal{M}})$, which implies that $H_{0}$ is closed
in $G$ thanks to the closure of $\operatorname{Sym}_{G}({\mathcal{M}})$ in
$G$.

Next we show that $\operatorname{Sym}_{G}({\mathcal{M}})_{0}=H_{0}$ is the
identity component of $\operatorname{Sym}_{G}({\mathcal{M}})$. First, we
observe that $\operatorname{Sym}_{G}({\mathcal{M}})_{0}\subset H_{0}$ because
$\operatorname{Sym}_{G}({\mathcal{M}})_{0}$ is connected and contained in
$\operatorname{Sym}_{G}({\mathcal{M}})$. The reverse containment follows from
the fact that $H_{0}$ is a connected Lie subgroup satisfying
${\mathcal{M}}\cdot H_{0}\subset{\mathcal{M}}$, which by our earlier result
implies that $H_{0}\subset\operatorname{Sym}_{G}({\mathcal{M}})_{0}$.
$\blacksquare$

##  Appendix H Proof of Theorem 34

Proof [Lemma 33] The map $\imath_{F}$ defined in a local trivialization $\Phi$
by Eq. 126 is injective. It is a vector bundle homomorphism because
$\operatorname{\mathrm{d}}\Phi\circ\imath_{F}\circ\Phi^{-1}$, $\Phi$, and
$\operatorname{\mathrm{d}}\Phi$ are vector bundle homomorphisms and $\Phi$ and
$\operatorname{\mathrm{d}}\Phi$ are invertible. It remains to show that the
definition of $\imath_{F}$ does not depend on the choice of local
trivialization. Given two local trivializations $\Phi$ and $\tilde{\Phi}$
defined on $\pi^{-1}({\mathcal{U}})\subset E$ where ${\mathcal{U}}$ is an open
subset of ${\mathcal{M}}$, it suffices to show that the following diagram
commutes:

${{{\mathcal{U}}\times\mathbb{R}^{k}}}$${{\pi^{-1}({\mathcal{U}})\subset
E}}$${{{\mathcal{U}}\times\mathbb{R}^{k}}}$${{T({\mathcal{U}}\times\mathbb{R}^{k})}}$${{T(\pi^{-1}({\mathcal{U}}))\subset
TE}}$${{T({\mathcal{U}}\times\mathbb{R}^{k})}}$$\scriptstyle{\imath_{F}}$$\scriptstyle{\imath_{\Phi\circ
F}}$$\scriptstyle{\imath_{\tilde{\Phi}\circ
F}}$$\scriptstyle{\Phi}$$\scriptstyle{\operatorname{\mathrm{d}}\Phi}$$\scriptstyle{\tilde{\Phi}}$$\scriptstyle{\operatorname{\mathrm{d}}\tilde{\Phi}}$
(178)

Since $\tilde{\Phi}\circ\Phi^{-1}$ is a bundle homomorphism descending to the
identity, it can be written as

$\tilde{\Phi}\circ\Phi^{-1}:(p,{\boldsymbol{v}})\mapsto(p,{\boldsymbol{T}}(p){\boldsymbol{v}})$
(179)

for a matrix-valued function
${\boldsymbol{T}}:{\mathcal{U}}\to\mathbb{R}^{k\times k}$. Moreover, the
matrices are invertible because the local trivializations are bundle
isomorphisms. Differentiating, we obtain

$\operatorname{\mathrm{d}}\tilde{\Phi}\circ\operatorname{\mathrm{d}}\Phi^{-1}:(w_{p},{\boldsymbol{w}})_{(p,{\boldsymbol{v}})}\mapsto\big{(}w_{p},\operatorname{\mathrm{d}}{\boldsymbol{T}}(p)w_{p}+{\boldsymbol{T}}(p){\boldsymbol{w}}\big{)}_{\tilde{\Phi}\circ\Phi^{-1}(p,{\boldsymbol{v}})},$
(180)

where $w_{p}\in T_{p}{\mathcal{U}}$. Composing this with $\imath_{\Phi\circ
F}:(p,{\boldsymbol{v}})\mapsto(0,{\boldsymbol{v}})_{\Phi(F(p))}$, we obtain

$\operatorname{\mathrm{d}}\tilde{\Phi}\circ\operatorname{\mathrm{d}}\Phi^{-1}\circ\imath_{\Phi\circ
F}(p,{\boldsymbol{v}})=(0,{\boldsymbol{T}}(p){\boldsymbol{v}})_{\tilde{\Phi}(F(p))}=\imath_{\tilde{\Phi}\circ
F}(p,{\boldsymbol{T}}(p){\boldsymbol{v}})=\imath_{\tilde{\Phi}\circ
F}\circ\tilde{\Phi}\circ\Phi^{-1}(p,{\boldsymbol{v}}),$ (181)

proving that the diagram commutes.

Proof [Theorem 34] We observe that $F\circ\pi:E\to E$ is a smooth idempotent
map whose image is $\operatorname{im}(F)\subset E$. By differentiating the
expression $(F\circ\pi)\circ(F\circ\pi)=F\circ\pi$ at a point
$F(p)\in\operatorname{im}(F)$, we obtain

$\operatorname{\mathrm{d}}(F\circ\pi)(F(p))\operatorname{\mathrm{d}}(F\circ\pi)(F(p))=\operatorname{\mathrm{d}}(F\circ\pi)(F(p)),$
(182)

meaning that $\operatorname{\mathrm{d}}(F\circ\pi)(F(p)):T_{F(p)}E\to
T_{F(p)}E$ is a linear projection. Since

$\operatorname{\mathrm{d}}(F\circ\pi)(F(p))=\operatorname{\mathrm{d}}F(p)\operatorname{\mathrm{d}}\pi(F(p)),$
(183)

we have
$\operatorname{Range}\big{(}\operatorname{\mathrm{d}}(F\circ\pi)(F(p))\big{)}\subset\operatorname{Range}(\operatorname{\mathrm{d}}F(p))=T_{F(p)}\operatorname{im}(F)$.
Differentiating $F=(F\circ\pi)\circ F$ yields

$\operatorname{\mathrm{d}}F(p)=\operatorname{\mathrm{d}}(F\circ\pi)(F(p))\operatorname{\mathrm{d}}F(p),$
(184)

meaning that
$T_{F(p)}\operatorname{im}(F)\subset\operatorname{Range}\big{(}\operatorname{\mathrm{d}}(F\circ\pi)(F(p))\big{)}$.
Since
$\operatorname{Range}\big{(}\operatorname{\mathrm{d}}(F\circ\pi)(F(p))\big{)}=T_{F(p)}\operatorname{im}(F)$
it follows that $\operatorname{\mathrm{d}}(F\circ\pi)(F(p))$ is a linear
projection onto $T_{F(p)}\operatorname{im}(F)$.

We observe that the generalized Lie derivative in Eq. 83 can be expressed as

$\displaystyle({\mathcal{L}}_{\xi}F)(p)$ $\displaystyle=\lim_{t\to
0}\frac{1}{t}\left[\Theta_{\exp(-t\xi)}(F(\theta_{\exp(t\xi)}(p)))-F(p)\right]$
(185) $\displaystyle=\lim_{t\to 0}\Theta\left(\exp(-t\xi),\
\frac{1}{t}\left[F(\theta_{\exp(t\xi)}(p))-\Theta_{\exp(t\xi)}(F(p))\right]\right)$
$\displaystyle=\lim_{t\to
0}\frac{1}{t}\left[F(\theta_{\exp(t\xi)}(p))-\Theta_{\exp(t\xi)}(F(p))\right].$

The first equality follows because $\Theta_{g^{-1}}$ is a vector bundle
homomorphism, meaning that the restricted map $\Theta_{g^{-1}}|_{E_{p\cdot
g}}:E_{p\cdot g}\to E_{p}$ is linear; here $g=\exp(t\xi)$. The second equality
follows because $\Theta:E\times G\to E$ is continuous. Note that in the first
expression the limit is taken in the vector space $E_{p}$, whereas in the last
expression the limit must be taken in $E$.

We proceed by expressing everything in a local trivialization
$\Phi:\pi^{-1}({\mathcal{U}})\to{\mathcal{U}}\times\mathbb{R}^{k}$ of an open
neighborhood ${\mathcal{U}}\subset{\mathcal{M}}$ of $p\in{\mathcal{M}}$. Since
the maps $\Theta_{g}$, $\Phi$, and $\Phi^{-1}$ are vector bundle
homomorphisms, there is a matrix-valued function
${\boldsymbol{T}}_{g}:{\mathcal{U}}\to\mathbb{R}^{k\times k}$ such that

$\tilde{\Theta}_{g}=\Phi\circ\Theta_{g}\circ\Phi^{-1}:(p,{\boldsymbol{v}})\mapsto(\theta_{g}(p),\
{\boldsymbol{T}}_{g}(p){\boldsymbol{v}}).$ (186)

Differentiating $\tilde{\Theta}_{\exp(t\xi)}(p,{\boldsymbol{v}})$ with respect
to $t$ yields the generator

$\hat{\tilde{\Theta}}(\xi)_{(p,{\boldsymbol{v}})}=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}\tilde{\Theta}_{\exp(t\xi)}(p,{\boldsymbol{v}})=\left(\hat{\theta}(\xi)_{p},\
{\boldsymbol{\hat{T}}}(\xi)_{p}{\boldsymbol{v}}\right)_{(p,{\boldsymbol{v}})},$
(187)

where
${\boldsymbol{\hat{T}}}(\xi)_{p}=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}{\boldsymbol{T}}_{\exp(t\xi)}(p)$.
We define the function
${\boldsymbol{\tilde{F}}}:{\mathcal{U}}\to\mathbb{R}^{k}$ by

$(p,{\boldsymbol{\tilde{F}}}(p))=\Phi\circ F(p)\qquad\forall
p\in{\mathcal{U}}.$ (188)

Using the above definitions, we can express the generalized Lie derivative in
the local trivialization:

$\displaystyle\Phi\circ({\mathcal{L}}_{\xi}F)(p)$ $\displaystyle=\lim_{t\to
0}\left(\theta_{\exp(t\xi)}(p),\
\frac{1}{t}\left[{\boldsymbol{\tilde{F}}}(\theta_{\exp(t\xi)}(p))-{\boldsymbol{T}}_{\exp(t\xi)}{\boldsymbol{\tilde{F}}}(p)\right]\right)$
(189) $\displaystyle=\left(p,\
\operatorname{\mathrm{d}}{\boldsymbol{\tilde{F}}}(p)\hat{\theta}(\xi)_{p}-{\boldsymbol{\hat{T}}}(\xi)_{p}{\boldsymbol{\tilde{F}}}(p)\right).$

Applying Lemma 33 allows us to express the left-hand-side of Eq. 127 as

$\operatorname{\mathrm{d}}\Phi\left[\imath_{F}\circ({\mathcal{L}}_{\xi}F)(p)\right]=\left(0,\
\operatorname{\mathrm{d}}{\boldsymbol{\tilde{F}}}(p)\hat{\theta}(\xi)_{p}-{\boldsymbol{\hat{T}}}(\xi)_{p}{\boldsymbol{\tilde{F}}}(p)\right)_{\Phi(F(p))}.$
(190)

We can also express the quantities on the right-hand-side of Eq. 127 in the
local trivialization. To do this, we compute

$\displaystyle\operatorname{\mathrm{d}}\Phi(F(p))\operatorname{\mathrm{d}}(F\circ\pi)(F(p))\hat{{\Theta}}(\xi)_{F(p)}$
$\displaystyle=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}\Phi\circ
F\circ\pi\circ\Theta_{\exp(t\xi)}(F(p))$ (191)
$\displaystyle=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}\Phi\circ
F(\theta_{\exp(t\xi)}(p))$ $\displaystyle=\left(\hat{\theta}(\xi)_{p},\
\operatorname{\mathrm{d}}{\boldsymbol{\tilde{F}}}(p)\hat{\theta}(\xi)_{p}\right)_{\Phi(F(p))}$

and

$\displaystyle\operatorname{\mathrm{d}}\Phi(F(p))\hat{\Theta}(\xi)_{F(p)}$
$\displaystyle=\left.\operatorname{\frac{\operatorname{\mathrm{d}}}{\operatorname{\mathrm{d}}t}}\right|_{t=0}\Phi\circ\Theta_{\exp(t\xi)}\circ\Phi^{-1}\circ\Phi\circ
F(p)$ (192)
$\displaystyle=\hat{\tilde{\Theta}}(\xi)_{\Phi(F(p))}=\left(\hat{\theta}(\xi)_{p},\
{\boldsymbol{\hat{T}}}(\xi)_{p}{\boldsymbol{\tilde{F}}}(p)\right)_{\Phi(F(p))}.$

Subtracting these yields

$\operatorname{\mathrm{d}}\Phi\left[\left(\operatorname{\mathrm{d}}(F\circ\pi)(F(p))-\operatorname{Id}_{T_{F(p)}E}\right)\hat{\Theta}(\xi)_{F(p)}\right]=\left(0,\
\operatorname{\mathrm{d}}{\boldsymbol{\tilde{F}}}(p)\hat{\theta}(\xi)_{p}-{\boldsymbol{\hat{T}}}(\xi)_{p}{\boldsymbol{\tilde{F}}}(p)\right)_{\Phi(F(p))},$
(193)

which, upon comparison with Eq. 190 completes the proof.