# Towards OOD Detection in Graph Classification
from Uncertainty Estimation Perspective

Gleb Bazhenov  Sergei Ivanov  Maxim Panov  Alexey Zaytsev  Evgeny Burnaev

###### Abstract

The problem of out-of-distribution detection for graph classification is far
from being solved. The existing models tend to be overconfident about OOD
examples or completely ignore the detection task. In this work, we consider
this problem from the uncertainty estimation perspective and perform the
comparison of several recently proposed methods. In our experiments, we find
that there is no universal approach for OOD detection, and it is important to
consider both graph representations and predictive categorical distribution.

out-of-distribution detection, graph classification, uncertainty estimation,
deep learning

##  1 Introduction

Out-of-distribution (OOD) detection requires us to distinguish between the in-
distribution (ID) observations that can be processed by model and OOD samples
that have to be rejected as inconsistent with the training data. This problem
is out there for a long time for all kinds of modalities including the graph-
structured data.

One way to deal with this problem is to provide an approach that assigns point
uncertainties for a particular model. In general, there are several approaches
to uncertainty estimation in classification task that employ the ensembling
techniques (Gal & Ghahramani, 2016; Lakshminarayanan et al., 2017), the
probabilistic frameworks based on Dirichlet distribution (Malinin & Gales,
2018; Sensoy et al., 2018; Charpentier et al., 2020), and other methods (Liu
et al., 2020; van Amersfoort et al., 2020, 2021; Mukhoti et al., 2021). While
there were previous attempts to explore the OOD detection for image
classification (Sinhamahapatra et al., 2022), their direct application to
graph classification requires further research due to a complex network
structure and additional attributes associated with the nodes.

Recent works (Zhao et al., 2020; Stadler et al., 2021) consider the OOD
detection in node classification problem using uncertainty estimation;
however, the node-level detection is different from the graph-level one due to
the dependent relationships between nodes, which does not occur for graphs.
Likewise, some works (Wu et al., 2021; Ding et al., 2021; Li et al., 2022)
explored the question of OOD generalisation on graph-structured data, where
robust models have been proposed to deal with the distribution shifts;
however, such works do not consider the detection of OOD samples, which is the
focus of our paper.

The problem of OOD detection for graph classification is not solved yet. The
existing models tend to be overconfident about OOD examples or completely
ignore the detection task. In our work, we consider this problem from the
uncertainty estimation perspective and perform the comparison of several
recently proposed methods. In particular, we find that there is no universal
approach for OOD detection across different graph datasets, and it is
important to consider not only the entropy of predictive distribution, but
also the geometry of the latent space produced by graph encoder. In addition,
we observe that sometimes the classification task has a default ID class for
most OOD samples.

##  2 Methods

In general, we construct the model that for a given graph $x$ and the
associated classifier $c(x)$ can provide the uncertainty estimate $u(x)$.
Since these estimates are used for ranking the test samples, we require that
such model predicts higher uncertainty for OOD examples.

In this work, we consider two common distinct classes of methods for OOD
detection through uncertainty estimation:

  * •

entropy-based methods, which use the entropy of predictive categorical
distribution as the measure of uncertainty, treating the predictions with high
entropy as less certain;

  * •

density-based methods, which perform probability density estimation using the
provided representations in latent space and assign high uncertainty to the
points with low density.

The entropy-based methods expect that we have a particular classification
model, while the density-based focus on a density estimate $\mathtt{p}(x)$
that can be independent from our classifier. However, in practice, they can
share the representations with the main classification model.

Most methods that we describe here can distinguish between two sources of
uncertainty (Gal, 2016) — data uncertainty $u_{\text{data}}$, which reflects
the internal noise in data due to class overlap or data labeling errors, and
knowledge uncertainty $u_{\text{know}}$, which refers to the lack of knowledge
about unseen data. In our experiments, we use both types of uncertainty and
investigate which one is more consistent with the OOD detection problem.

###  2.1 Entropy-Based

Figure 1: OOD detection performance per class of all the considered methods
using
knowledge uncertainty KU $u_{\text{know}}$ on ENZYMES (left) and IMDB-MULTI
(right) datasets. Figure 2: t-SNE graph embeddings produced by single for
classes with 1 (left) and 5 (right) triangles
from TRIANGLES dataset. It reflects how the joint distribution of graph
representations
changes as the OOD class moves from the boundary of ID space to its interior.

The entropy-based methods that we use in our experiments can be described in
terms of the probabilistic framework which implies some unknown distribution
$\mathtt{p}(\theta|\mathcal{D})$ of model parameters $\theta$ given the
training data $\mathcal{D}$. In our classification task, the predictive
distribution is expressed as follows:

$\mathtt{P}(y|x,\mathcal{D})=\mathbb{E}_{\mathtt{p}(\theta|\mathcal{D})}\mathtt{P}(y|x,\theta),$

where $\mathtt{P}(y|x,\theta)$ is defined by a graph classification model
$f_{\theta}(x)$. Each method presented below provides a particular form of
distribution $\mathtt{p}(\theta|\mathcal{D})$ and the uncertainty measure that
is used for OOD detection in graph classification problem.

####  2.1.1 Single GNN Model

This approach employs the most simple form of distribution
$\mathtt{p}(\theta|\mathcal{D})$ which is expressed through a delta function
around the MLE of model parameters $\theta$:

$\mathtt{p}(\theta|\mathcal{D})=\delta(\theta-\theta_{\text{MLE}}).$

In such case, the predictive distribution has the form

$\mathtt{P}(y|x,\mathcal{D})=\mathtt{P}(y|x,\theta_{\text{MLE}}).$

The model consists of several graph convolution layers and global graph
pooling that provides a representation $z(x)$, which is then processed by a
single-layer fully-connected network in order to obtain the predictive
distribution.

In this approach, we do not distinguish between data and knowledge uncertainty
and base the total uncertainty measure of this model on the entropy of
predictive distribution:

$u_{\text{total}}=\mathbb{H}\big{[}\mathtt{P}(y|x,\theta_{\text{MLE}})\big{]}.$

Further in the text, we refer to this method as single.

####  2.1.2 Ensemble of GNN Models

We also consider two ensembling techniques for constructing more robust
predictive models — Monte Carlo Dropout (Gal & Ghahramani, 2016) and Deep
Ensembles (Lakshminarayanan et al., 2017). To formally define these
approaches, we consider the approximation of distribution
$\mathtt{p}(\theta|\mathcal{D})$ via a set of models, which either represent
the dropout versions of the same estimator described in Section 2.1.1 (MC), or
independently trained instances of such model (DE):

$\theta\sim\mathtt{q}(\theta|\mathcal{D}),$

where the distribution $\mathtt{q}(\theta|\mathcal{D})$ is discrete uniform
over $n$ model instances $\theta_{k}$. Given this, the predictive distribution
can be expressed as a discrete average instead of the original integral:

$\mathtt{P}(y|x,\mathcal{D})=\frac{1}{n}\sum\limits_{k=1}^{n}\mathtt{P}(y|x,\theta_{k}).$

It enables us to separate data and knowledge uncertainty in the same way as it
was done in (Malinin & Gales, 2018):

$\begin{split}\begin{gathered}u_{\text{data}}=\mathbb{E}_{\mathtt{q}(\theta|\mathcal{D})}\mathbb{H}\big{[}\mathtt{P}(y|x,\theta)\big{]},\\\
u_{\text{know}}=\mathbb{H}\big{[}\mathbb{E}_{\mathtt{q}(\theta|\mathcal{D})}\mathtt{P}(y|x,\theta)\big{]}-u_{\text{data}}.\end{gathered}\end{split}$

For the sake of brevity, we refer to Deep Ensemble as DE and Monte Carlo
Dropout as MC.

###  2.2 Density-Based

These methods are rather more flexible and diverse in terms of formulation,
but their main idea is to estimate the density function of probability
distribution in the latent space of graph representations $z(x)$ which are
associated with a particular classification task. The density-based methods
all suffer from challenging density estimation in high-dimensional embedding
space. However, existing approaches in various ways try to mitigate this issue
for large-dimension spaces.

####  2.2.1 NUQ

The first density-based method for uncertainty estimation that we consider is
Non-Parametric Uncertainty Quantifaction (Kotelevskii et al., 2022), which can
be derived from the problem of statistical risk minimisation and leads to the
natural decomposition of total uncertainty into the sum of data uncertainty

$\eta_{k}(x)=\mathtt{P}(y=k|x)\Longrightarrow
u_{\text{data}}=\min_{k}\eta_{k}(x),$

where $\mathtt{P}(y=k|x)$ is defined by the Nadaraya-Watson estimator
(Nadaraya, 1964), and knowledge uncertainty

$u_{\text{know}}\sim\max\limits_{k}\sigma_{k}^{2}(z)/\mathtt{p}(z),$

where $\sigma_{k}^{2}(z)=\eta_{k}(x)\big{[}1-\eta_{k}(x)\big{]}$ and the
probability density function $\mathtt{p}(z)$ is approximated using the kernel
density estimator (KDE) in graph representation space, which in turn is
induced by the already trained graph encoder described in Section 2.1.1. We
use its short notation NUQ for convenience.

####  2.2.2 NatPostNet

Another view on the uncertainty estimation is given by Natural Posterior
Network (Charpentier et al., 2021), which predicts the parameters of Dirichlet
distribution instead of directly modeling the posterior categorical
distribution.

Using the parameters of Dirichlet distribution $\mathtt{p}(\mu|x,\theta)$, the
predictive categorical distribution can be defined as

$\mathtt{P}(y|x,\mathcal{D})=\mathbb{E}_{\mathtt{p}(\mu|x,\theta)}\mathtt{P}(y|\mu),$

where the distribution $\mathtt{p}(\mu|x,\theta)$ is learned using the same
model $f_{\theta}(x)$ as described in Section 2.1.1 — the only difference is
that such GNN predicts the Dirichlet parameters instead of categorical ones.
In this framework, data uncertainty is defined as the entropy of predictive
distribution:

$u_{\text{data}}=\mathbb{H}\big{[}\mathtt{P}(y|x,\mathcal{D})\big{]}.$

As for knowledge uncertainty, it can be expressed through the density estimate
provided by a single normalising flow $g_{\psi}(z)$ (Kobyzev et al., 2020)
over graph representations $z$:

$u_{\text{know}}\sim-g_{\psi}(z).$

This method is denoted by NatPostNet.

Figure 3: OOD confusion (left) and pair-wise distance (right) matrices
produced by NatPostNet on REDDIT-MULTI-12K dataset.

##  3 Experiments

Because of the space limitation, we report the graph dataset statistics and
their class names in Appendix B. Moreover, we describe the evaluation
procedure and OOD detection metrics for all datasets in Appendices A and C,
while here we discuss only the key insights that we have managed to obtain
during our experiments. In particular, we formulate and answer the following
questions:

Are there methods that provide the optimal OOD detection performance on both
chemical and social graph datasets?

Based on Figure 1, it is natural to conclude that chemical and social datasets
vary in the difficulty of OOD detection for the same graph encoder
architecture, since the performance of DE and both considered density-based
methods on IMDB-MULTI is much better than on ENZYMES. It means that particular
applications such as chemistry might require a special design of encoders that
model the graph manifold more efficiently.

How the OOD detection performance is influenced by the structure of the graph
dataset and the associated latent space induced by the model?

As it can be seen from the last row in Figure 4, the OOD detection performance
of the entropy-based methods on TRIANGLES dataset drops significantly as the
OOD class moves from the interior of ID region to its boundary (i.e., from 5
to 1 or 10). We can say that these methods become overconfident for the
boundary classes.

However, looking on the distribution of graph embeddings (see Figure 2), one
can notice that for the boundary class (1 triangle) the OOD embeddings seem to
be better distinguishable from ID ones than for the classes in the middle of
the range (5 triangles). It means that the usage of methods based solely on
the entropy of predictive distribution is not enough, and one also may need to
take into account the geometry of embeddings.

What the predictive categorical distribution over OOD samples can say about
the ID classes?

We recall that the classes that are available to classifier are referred to as
ID and those that it has not seen are treated as OOD. By retraining the
classifiers on a different set of ID classes we can obtain different
predictions for OOD classes. In Figure 3 (left), one can notice that
regardless of a particular OOD class, the model often predicts it to be the
WorldNews class, which can be treated as a default ID class for any OOD
sample.

Can we confirm that the most distant classes in the latent space appear to be
the most simple for OOD detection?

Comparing the OOD detection performance using knowledge uncertainty on REDDIT-
MULTI-12K dataset in the corresponding row of Figure 4 and pair-wise distances
between its classes in Figure 3 (right), one can understand that the most
distant classes such as TrollXChromosomes tend to be easier for OOD detection.

What type of uncertainty is more consistent with the OOD detection problem for
graph classification?

In general, the estimates based on knowledge uncertainty appear to be more
appropriate for OOD detection than using data uncertainty, as can be seen in
Figure 4.

##  4 Conclusion

We investigated the OOD detection problem in graph classification task by
comparing several uncertainty estimation methods and discussing some key
insights about the dependence of OOD detection performance on the structure of
graph representations and the interpretation of predictive categorical
distribution in context of OOD detection.

## References

  * Charpentier et al. (2020) Charpentier, B., Zügner, D., and Günnemann, S.  Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts.  _Advances in Neural Information Processing Systems_ , 33:1356–1367, 2020.
  * Charpentier et al. (2021) Charpentier, B., Borchert, O., Zügner, D., Geisler, S., and Günnemann, S.  Natural posterior network: Deep bayesian predictive uncertainty for exponential family distributions.  _arXiv preprint arXiv:2105.04471_ , 2021.
  * Ding et al. (2021) Ding, M., Kong, K., Chen, J., Kirchenbauer, J., Goldblum, M., Wipf, D., Huang, F., and Goldstein, T.  A closer look at distribution shifts and out-of-distribution generalization on graphs.  In _NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications_ , 2021.
  * Gal (2016) Gal, Y.  Uncertainty in deep learning.  _University of Cambridge, Cambridge_ , 2016.
  * Gal & Ghahramani (2016) Gal, Y. and Ghahramani, Z.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _international conference on machine learning_ , pp. 1050–1059. PMLR, 2016.
  * Kobyzev et al. (2020) Kobyzev, I., Prince, S. J., and Brubaker, M. A.  Normalizing flows: An introduction and review of current methods.  _IEEE transactions on pattern analysis and machine intelligence_ , 43(11):3964–3979, 2020.
  * Kotelevskii et al. (2022) Kotelevskii, N., Artemenkov, A., Fedyanin, K., Noskov, F., Fishkov, A., Petiushko, A., and Panov, M.  Nuq: Nonparametric uncertainty quantification for deterministic neural networks.  _arXiv preprint arXiv:2202.03101_ , 2022.
  * Lakshminarayanan et al. (2017) Lakshminarayanan, B., Pritzel, A., and Blundell, C.  Simple and scalable predictive uncertainty estimation using deep ensembles.  _Advances in neural information processing systems_ , 30, 2017.
  * Li et al. (2022) Li, H., Wang, X., Zhang, Z., and Zhu, W.  Out-of-distribution generalization on graphs: A survey.  _arXiv preprint arXiv:2202.07987_ , 2022.
  * Liu et al. (2020) Liu, J., Lin, Z., Padhy, S., Tran, D., Bedrax Weiss, T., and Lakshminarayanan, B.  Simple and principled uncertainty estimation with deterministic deep learning via distance awareness.  _Advances in Neural Information Processing Systems_ , 33:7498–7512, 2020.
  * Malinin & Gales (2018) Malinin, A. and Gales, M.  Predictive uncertainty estimation via prior networks.  _Advances in neural information processing systems_ , 31, 2018.
  * Mukhoti et al. (2021) Mukhoti, J., Kirsch, A., van Amersfoort, J., Torr, P. H., and Gal, Y.  Deterministic neural networks with appropriate inductive biases capture epistemic and aleatoric uncertainty.  _arXiv e-prints_ , pp. arXiv–2102, 2021.
  * Nadaraya (1964) Nadaraya, E. A.  On estimating regression.  _Theory of Probability & Its Applications_, 9(1):141–142, 1964.
  * Sensoy et al. (2018) Sensoy, M., Kaplan, L., and Kandemir, M.  Evidential deep learning to quantify classification uncertainty.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Sinhamahapatra et al. (2022) Sinhamahapatra, P., Koner, R., Roscher, K., and Günnemann, S.  Is it all a cluster game?–exploring out-of-distribution detection based on clustering in the embedding space.  _arXiv preprint arXiv:2203.08549_ , 2022.
  * Stadler et al. (2021) Stadler, M., Charpentier, B., Geisler, S., Zügner, D., and Günnemann, S.  Graph posterior network: Bayesian predictive uncertainty for node classification.  _Advances in Neural Information Processing Systems_ , 34, 2021.
  * van Amersfoort et al. (2020) van Amersfoort, J., Smith, L., Teh, Y. W., and Gal, Y.  Simple and scalable epistemic uncertainty estimation using a single deep deterministic neural network.  2020\.
  * van Amersfoort et al. (2021) van Amersfoort, J., Smith, L., Jesson, A., Key, O., and Gal, Y.  Improving deterministic uncertainty estimation in deep learning for classification and regression.  _arXiv e-prints_ , pp. arXiv–2102, 2021.
  * Wu et al. (2021) Wu, Q., Zhang, H., Yan, J., and Wipf, D.  Handling distribution shifts on graphs: An invariance perspective.  In _International Conference on Learning Representations_ , 2021.
  * Zhao et al. (2020) Zhao, X., Chen, F., Hu, S., and Cho, J.-H.  Uncertainty aware semi-supervised learning on graph data.  _Advances in Neural Information Processing Systems_ , 33:12827–12836, 2020.

##  Appendix A Experimental Setup

In this work, we consider a particular OOD detection setup when one of the
originally available $C$ classes in graph dataset is removed and marked as
OOD, while the graph classification model is trained on the remaining set of
classes in stratified validation mode. After that, we predict the uncertainty
measures for both ID and OOD samples separated previously. Based on this, we
evaluate the ability of the considered methods to rank observations and detect
the OOD samples — we compute the AUROC metric on the united set of predictions
for ID validation samples and OOD samples, treating the OOD detection problem
as formal binary classification task. In other words, we solve one distinct ID
classification problem for each of $C$ OOD classes. For the sake of a
consistent evaluation, we perform 5 random splits of the ID set into train and
validation parts, and then average the obtained metrics.

Along with the OOD detection metrics, we also provide the related materials
which might be helpful for interpretation. Appendix D contains the figures of
graph embeddings on TRIANGLES dataset that represent the structure of latent
space in the corresponfing OOD detection problem. Moreover, Appendix E
presents the OOD confusion matrices for all the considered datasets, where
rows correspond to particular OOD classes, while columns — to the predicted ID
classes. In Appendix F, we place the pair-wise distance matrices between
classes in standard ID classification, which reveals the correlation between
the distance to other classes and the difficulty of OOD detection.

##  Appendix B Datasets Details

###  B.1 Basic Statistics

In Table 1, we report basic graph dataset statistics, such as number of graphs
and avergae number of nodes and edges.

Table 1: Dataset statistics. Note that only ENZYMES dataset has natural
features
in nodes, so we assign the same artifiial feature 1 to nodes in other datasets. Name | # Graphs | # Classes | # Nodes | # Edges | # Features
---|---|---|---|---|---
ENZYMES | 600 | 6 | 32.63 | 62.14 | 3
IMDB-MULTI | 1500 | 3 | 13.00 | 65.94 | 1
REDDIT-MULTI-5K | 4999 | 5 | 508.52 | 594.87 | 1
REDDIT-MULTI-12K | 11929 | 11 | 391.41 | 456.89 | 1
TRIANGLES | 45000 | 10 | 20.85 | 32.74 | 1

###  B.2 Class Names

In Table 2, you can find the list of class names for each considered graph
dataset.

Table 2: List of class names for every considered graph dataset. Name | Class Names
---|---
ENZYMES | Oxidoreductases Transferases Hydrolases
| Lyases Isomerases Ligases
IMDB-MULTI | Comedy Romance Sci-Fi
REDDIT-MULTI-5K | WorldNews Videos AdviceAnimals
| Aww MildlyInteresting
REDDIT-MULTI-12K | AskReddit AdviceAnimals Atheism Aww IAmA
| MildlyInteresting ShowerThoughts Videos
| TodayILearned WorldNews TrollXChromosomes
TRIANGLES | 1 2 3 4 5 6 7 8 9 10

##  Appendix C Out-of-Distribution Detection Performance

The OOD detection metrics presented here are partially discussed in Section 3
— they enable us to compare the performance of various methods and conclude
that there is no universal approach for OOD detection in graph classification.

Figure 4: OOD detection performance per class of all the mentioned methods
using
data uncertainty DU (left column) and knowledge uncertainty KU (right column)
on all the considered datasets.

##  Appendix D t-SNE Graph Embeddings

Here you can see how the joint distribution of graph representations changes
depending on the particular OOD class.

Figure 5: t-SNE graph embeddings produced by single for all classes from the
TRIANGLES dataset.
Red points represent OOD graphs, blue points — graphs from the nearest ID
class,
green — graphs from the most distant ID class, and grey — all other graphs.

##  Appendix E OOD Confusion Matrices

These OOD confusion matrices are used to interpret the predictive categorical
distribution over OOD samples and find that sometimes the classification task
has a default ID class for most OOD samples, as discussed in Section 3.

Figure 6: OOD confusion matrices produced by single, DE and NatPostNet for all
the considered datasets.
Rows correspond to the OOD classes, while columns — to the predicted ID
classes.

##  Appendix F Distance Matrices Between Classes

Comparing these matrices and the OOD detection performance on the
corresponding dataset, we can confirm that the most distant classes in latent
space appear to be the most simple for OOD detection.

Figure 7: Distance matrices produced by single and NatPostNet for all the
considered datasets.
The distance between classes is defined as the Euclidean distance between the
corresponding centroids of graph embeddings.


# A Rogers–Brascamp–Lieb–Luttinger inequality in the space of matrices

J. Haddad

###### Abstract.

We consider convex bodies in $\operatorname{M}_{n,m}(\mathbb{R})$, the space
of matrices of $n$-rows and $m$-columns. A special case of fiber-
symmetrization in $\operatorname{M}_{n,m}(\mathbb{R})$ was recently introduced
in [5, 6]. We prove a Rogers–Brascamp–Lieb–Luttinger type inequality with
respect to this symmetrization, for quasi-concave functions and provide some
applications.

##  1\. Introduction

Let $K\subseteq\mathbb{R}^{n}$ be a Lebesgue-measurable set, and $v$ be a unit
vector in $\mathbb{R}^{n}$. The Steiner symmetrical of $K$ with respect to $v$
is

$S_{v}K=\\{x+tv\in\mathbb{R}^{n}:t\in\mathbb{R},x\perp
v,|t|\leq\ell_{v}(x)/2\\}$

where $\ell_{v}(x)$ is the one-dimensional measure of the set obtained
intersecting $K$ with the line parallel to $v$ passing through $x$.

Steiner symmetrization has a huge number of applications in geometry ranging
from isoperimetric inequalities to functional analysis and it’s at the core of
the solution of many problems in Convex Geometry. For an overview of the many
applications we refer to the books [1, 11] and the references therein. The
operator $S_{v}$ enjoys two important properties. First, it preserves the
Lebesgue measure, and second, any convex body (compact, convex with non-empty
interior) can be transformed by an infinite sequence of successive
symmetrizations, converging to a Euclidean ball.

Let $f:\mathbb{R}^{n}\to\mathbb{R}$ be a non-negative measurable function and
$t\geq 0$. Denote $\\{f\geq t\\}:=\\{x\in\mathbb{R}^{n}:f(x)\geq t\\}$, and
similarly with $\leq$. We say that $f$ is quasi-concave if its super-level
sets $\\{f\geq t\\}$ are convex for every $t\in\mathbb{R}$. A function $f$ is
quasi-convex if its sub-level sets $\\{f\leq t\\}$ are convex for every
$t\in\mathbb{R}$.

The Steiner symmetrization of a quasi-concave function $f$ with respect to $v$
is defined by

$f^{(v)}(x)=\int_{0}^{\infty}\operatorname{1}_{S_{v}\\{f\geq t\\}}(x)dt$

where $\operatorname{1}_{K}$ is the characteristic function of the set $K$.
The function $f^{(v)}$ can be characterized by being the unique function whose
super-level sets are the Steiner symmetricals of the super-level sets of $f$,
in the direction $v$.

Rogers [9] and Brascamp, Lieb and Luttinger [3] proved independently a general
integral inequality, which generalizes the celebrated Riesz convolution
inequality:

######  Theorem 1.1.

Let $f_{i}:\mathbb{R}^{n}\to\mathbb{R},i=1,\ldots,k$ be non-negative and
measurable functions, and let $a_{j}^{(i)}$ be real numbers, then

(1)
$\int_{\mathbb{R}^{n}}\cdots\int_{\mathbb{R}^{n}}\prod_{i=1}^{k}f_{i}\left(\sum_{j=1}^{d}a_{j}^{(i)}x_{j}\right)dx_{1}\ldots
dx_{d}\\\
\leq\int_{\mathbb{R}^{n}}\cdots\int_{\mathbb{R}^{n}}\prod_{i=1}^{k}f^{(v)}_{i}\left(\sum_{j=1}^{d}a_{j}^{(i)}x_{j}\right)dx_{1}\ldots
dx_{d}.$

Notice that if the matrix with entries $a_{j}^{(i)}$ has rank less than $d$,
the integrals at both sides are either $0$ (if all $f_{i}$ are zero a.e.) or
infinite. If $k=d$ and the matrix is invertible, there is equality by change
of variables, and the fact that symmetrization preserves the integral. This
inequality can be applied iteratively for different directions $v\in S^{n-1}$,
and by taking limits, we may replace $f^{(v)}$ in the right-hand side of (1)
by the symmetric decreasing rearrangement $f^{*}$ (see [3]).

An interesting generalization of Theorem 1.1 was used to study randomized
isoperimetric inequalities by Paouris and Pivovarov [8] (Theorem 1.3 below).
This inequality is attributed to Christ [4]. The following definition is given
in [8].

######  Definition 1.2.

A function $f:(\mathbb{R}^{n})^{d}\to\mathbb{R}$ is called Steiner concave
(resp. Steiner convex) if for every $v\in S^{n-1}$ and $y_{1},\ldots,y_{d}\in
v^{\perp}$, the function $G:\mathbb{R}^{d}\to\mathbb{R}$ defined by

$G(t_{1},\ldots,t_{d})=f(y_{1}+t_{1}v,\ldots,y_{d}+t_{d}v)$

is even and quasi-concave.

The evenness in this definition is of absolute central importance, as we shall
see in the course of the paper.

######  Theorem 1.3 ([8, Theorem 3.8]).

Let $f_{1},\ldots,f_{k_{1}}$ be non-negative integrable functions on
$\mathbb{R}^{n}$, $a_{j}^{(i)}$ real numbers with
$j=1,\ldots,d,i=1,\ldots,k_{1}$. Let
$F^{(i)}:(\mathbb{R}^{n})^{d}\to\mathbb{R}$ be non-negative Steiner concave
functions $1\leq i\leq k_{2}$, and let $\mu$ be a non-negative measure with a
rotationally invariant quasi-concave density in $\mathbb{R}^{n}$. Then,

(2)
$\int_{\mathbb{R}^{n}}\cdots\int_{\mathbb{R}^{n}}\prod_{i=1}^{k_{2}}F^{(i)}(x_{1},\ldots,x_{d})\prod_{i=1}^{k_{1}}f_{i}\left(\sum_{j=1}^{m}a_{j}^{(i)}x_{j}\right)d\mu(x_{1})\ldots
d\mu(x_{d})\\\
\leq\int_{\mathbb{R}^{n}}\cdots\int_{\mathbb{R}^{n}}\prod_{i=1}^{k_{2}}F^{(i)}(x_{1},\ldots,x_{d})\prod_{i=1}^{k_{1}}f^{(v)}_{i}\left(\sum_{j=1}^{m}a_{j}^{(i)}x_{j}\right)d\mu(x_{1})\ldots
d\mu(x_{d}).\\\ $

We shall see that this theorem can be generalized in an elegant way if one
considers a symmetrization procedure adapted to the product structure of
$(\mathbb{R}^{n})^{d}$. Both sets of functions $F^{(j)},f_{i}$ together with
the measure $\mu$ will be seen as objects of the same nature under this
perspective. Here each $F^{(i)}$ is a function that is already symmetric in
some sense.

Let $m\geq 1$. It is natural to identify $(\mathbb{R}^{n})^{m}$ with the set
of matrices of $n$ rows and $m$ columns, $\operatorname{M}_{n,m}(\mathbb{R})$.
For $(x_{1},\ldots,x_{m})\in(\mathbb{R}^{n})^{m}$ we write
$x\in\operatorname{M}_{n,m}(\mathbb{R})$ for the matrix whose columns are the
vectors $x_{i}$. The expression $\sum a_{j}^{(i)}x_{j}$ in inequality (1) can
be written as $xa^{(i)}$, where $a^{(i)}$ is the column vector whose entries
are $a_{j}^{(i)}$.

In [7] McMullen defined the fiber combination of convex sets, and later in [2]
Bianchi, Gardner and Gronchi defined the fiber-symmetrization with respect to
a subspace of $\mathbb{R}^{d}$. We will use this symmetrization procedure in
$\operatorname{M}_{n,m}(\mathbb{R})$, with respect to a specific set of
subspaces. This particular case of the fiber symmetrization, which is well
adapted to the product structure of
$\operatorname{M}_{n,m}(\mathbb{R})=(\mathbb{R}^{n})^{m}$, has found
applications only recently, see [5] and [6].

######  Definition 1.4.

For $v\in S^{n-1}$, consider the $m$-dimensional space

$v^{m}:=\\{vt:t\in\operatorname{M}_{1,m}(\mathbb{R})\\}\subseteq\operatorname{M}_{n,1}(\mathbb{R})$

and let

$v^{\perp
m}=\\{x\in\operatorname{M}_{n,m}(\mathbb{R}):v^{t}x=0\in\operatorname{M}_{m,1}(\mathbb{R})\\}.$

The subspace $v^{m}$ consists of matrices whose columns are all multiples of
$v$, while $v^{\perp m}$ consists of matrices, whose columns are all
perpendicular to $v$. Of course, we have
$\operatorname{M}_{n,m}(\mathbb{R})=v^{m}\oplus v^{\perp m}$.

For any set $K\subseteq\mathbb{R}^{m}$, its difference body is the set
$DK=\\{x-y\in\mathbb{R}^{m}:x,y\in K\\}$.

######  Definition 1.5.

Let $K\subseteq\operatorname{M}_{n,m}(\mathbb{R})$ be a measurable set. We
define the $m$-th higher-order Steiner symmetrical of $K$ with respect to $v$
by

$\bar{S}_{v}K=\bigcup_{y\in v^{\perp
m}}\left(y+\frac{1}{2}D(K\cap(y+v^{m}))\right),$

Furthermore, we have

(3)
$\bar{S}_{v}K=\left\\{y+v\frac{t-s}{2}\in\operatorname{M}_{n,m}(\mathbb{R})\colon
y\in v^{\perp m},\text{ and }y+vt,y+vs\in K\right\\}.$

Notice that the notion of Steiner-concavity of a quasi-concave function
corresponds to the fact that its super-level sets are invariant by
$\bar{S}_{v}$ for every $v\in S^{n-1}$. In particular, the characteristic
function $\operatorname{1}_{K}$ of a convex set
$K\subseteq\operatorname{M}_{n,m}(\mathbb{R})$, is Steiner concave if and only
if $\bar{S}_{v}K=K$ for every $v\in S^{n-1}$.

######  Remark 1.6.

Notice that for $m=1$, $v^{m}$ is just a line and, if $K$ convex, the
intersection $K\cap(y+v^{m})$ is an interval, while
$y+\frac{1}{2}D(K\cap(y+v^{m}))$ is the translated interval centered at $y$.
This is, for $m=1$ and $K$ convex, the operator $\bar{S}_{v}$ is the usual
Steiner Symmetrization operator $S_{v}$. However, if $K$ is not convex this is
no longer true, as the difference body of an arbitrary set might not be an
interval.

######  Definition 1.7.

Let $f:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}$ be non-negative and
quasi-concave. Define the (higher-order) Steiner symmetrical of $f$ as

$f^{(v)}(x)=\int_{0}^{\infty}\operatorname{1}_{\bar{S}_{v}\\{f\geq
t\\}}(x)dt.$

######  Definition 1.8.

If $f:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}$ is a non-negative
quasi-convex function we define

$f_{(v)}(x)=\int_{0}^{\infty}(1-\operatorname{1}_{\bar{S}_{v}\\{f<t\\}}(x))dt.$

The operator $\bar{S}_{v}$ was recently used in [5, 6] to prove the higher-
order Petty projection inequality. In this note we study the operator
$\bar{S}_{v}$ acting on sets and functions, and prove a
Rogers–Brascamp–Lieb–Luttinger type inequality. Some applications will be
given. Our main theorem reads as follows:

######  Theorem 1.9.

Let $f_{i}:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R},i=1,\ldots,k$ be
non-negative and quasi-concave, and let
$L_{i}\in\operatorname{M}_{d,m}(\mathbb{R}),i=1,\ldots,k$ be real matrices of
rank $m$, then for $v\in S^{n-1}$,

$\int_{\operatorname{M}_{n,d}(\mathbb{R})}\prod_{i=1}^{k}f_{i}(xL_{i})dx\leq\int_{\operatorname{M}_{n,d}(\mathbb{R})}\prod_{i=1}^{k}f_{i}^{(v)}(xL_{i})dx.$

Inequality (1) in the quasi-concave case corresponds to the case where $m=1$
so all matrices $L_{i}$ are non-zero column vectors. Theorem 1.3 is a
particular case of Theorem 1.9 where $m=d$,
$L_{i}=\operatorname{Id}_{d},i=1,\ldots,k_{1}$, and the first $k_{1}$
functions satisfy $f_{i}^{(v)}=f_{i},\forall v\in S^{n-1}$, while the rest of
the functions ($f_{i}$ and $\mu(x_{i})$) depend on only one variable.

We insist here that Theorem 1.9 extends the aforementioned results, only for
quasi-concave functions. If the functions $f_{i}$ are general measurable
functions, there is the possibility that $f_{i}^{(v)}$ is no longer
measurable. This owes to the fact that the difference body of a Lebesgue
measurable set may be non-measurable. This problem can be overcome by using a
measure-theoretic definition of $D$ and $\bar{S}_{v}$, but we shall not pursue
this line of research here. For general measurable non-negative functions the
inequality still holds, since Theorem 1.9 is proved using only the
Brunn–Minkowski inequality, which is valid also in this context. But we obtain
a weaker result, due to the observation in Remark 1.6. For this reason, and
for a clearer exposition, we deal here only with convex sets and quasi-concave
or quasi-convex functions.

The rest of the paper is organized as follows: In Section 2 we recall basic
facts about convexity. In Section 3 we prove basic properties satisfied by the
operator $\bar{S}_{v}$ and its analogue on functions. In Section 4 we
establish symmetrization inequalities and prove Theorem 1.9. In Section 5 we
give some applications, and finally in Section 6 we study the sets that remain
invariant under $\bar{S}_{v}$.

##  2\. Notation and Preliminaries

We consider the vector space $\operatorname{M}_{n,m}(\mathbb{R})$ of real
matrices of $n$ rows and $m$ columns, with the usual euclidean structure given
by

(4) $\langle A,B\rangle=\sum_{i=1}^{n}\sum_{j=1}^{m}A_{i,j}B_{i,j},\ \ \
\|A\|_{2}=\sqrt{\sum_{i=1}^{n}\sum_{j=1}^{m}A_{i,j}^{2}}$

for $A,B\in\operatorname{M}_{n,m}(\mathbb{R})$. Here $A_{i,j}$ denote
$(i,j)$-th entry of $A\in\operatorname{M}_{n,m}(\mathbb{R})$.

The Lebesgue measure in $\operatorname{M}_{n,m}(\mathbb{R})$ is inherited from
the natural identification between $\operatorname{M}_{n,m}(\mathbb{R})$ and
$\mathbb{R}^{nm}$. We denote $\left|\cdot\right|_{n}$ and
$\left|\cdot\right|_{nm}$ for the volume in $\mathbb{R}^{n}$ and
$\operatorname{M}_{n,m}(\mathbb{R})$ respectively.

For notational convenience we identify $\mathbb{R}^{n}$ with
$\operatorname{M}_{n,1}(\mathbb{R})$, and $\mathbb{R}^{m}$ with
$\operatorname{M}_{1,m}(\mathbb{R})$, unless stated otherwise. We denote the
unit Euclidean spheres
$S^{n-1}\subseteq\mathbb{R}^{n}=\operatorname{M}_{n,1}(\mathbb{R})$, and
$\mathbb{S}^{nm-1}\subseteq\operatorname{M}_{n,m}(\mathbb{R})$. Also, we
identify
$\operatorname{M}_{n,a+b}(\mathbb{R})=\operatorname{M}_{n,a}(\mathbb{R})\times\operatorname{M}_{n,b}(\mathbb{R})$
in the natural way.

For a set $U\subseteq\operatorname{M}_{n,m}(\mathbb{R})$,
$A\in\operatorname{M}_{k,n}(\mathbb{R}),$ and
$B\in\operatorname{M}_{m,l}(\mathbb{R})$, we write

$AU=\\{Ax\in\operatorname{M}_{k,m}(\mathbb{R}):x\in U\\}\ \ \mathrm{and}\ \ \
UB=\\{xB\in\operatorname{M}_{n,l}(\mathbb{R}):x\in U\\}.$

We denote the set of convex bodies in $\operatorname{M}_{n,m}(\mathbb{R})$ by
$\mathcal{K}^{n,m}$. The support function of $K\in\mathcal{K}^{n,m}$ is given
by

$h_{K}(x)=\sup\\{\langle x,y\rangle:y\in K\\}$

for $x\in\operatorname{M}_{n,m}(\mathbb{R})$, where the inner product is the
one given in (4).

######  Definition 2.1.

Let $A\in\operatorname{M}_{d,m}(\mathbb{R})$ then $A$ induces a linear map

$\bar{A}:\operatorname{M}_{n,d}(\mathbb{R})\to\operatorname{M}_{n,m}(\mathbb{R})$

by right-multiplication, $\bar{A}(x)=xA$.

Observe that

(5) $\overline{AB}=\bar{B}\circ\bar{A},\ \overline{I_{m}}=I_{nm}.$

The general Brunn-Minkowski inequality states that if
$K,L\subseteq\mathbb{R}^{n}$ are convex bodies, then

(6)
$\left|K+L\right|_{n}^{1/n}\geq\left|K\right|_{n}^{1/n}+\left|L\right|_{n}^{1/n}$

where $K+L=\\{x+y:x\in K,y\in L\\}$ is the Minkowski sum of sets. Equality
holds in (6) if and only if $K$ and $L$ are homothetic.

A convex body $K\in\mathcal{K}^{n,m}$ is origin-symmetric if $K=-K$, while it
is symmetric with respect to a point $c\in\operatorname{M}_{n,m}(\mathbb{R})$
if $K-c$ is origin-symmetric. The difference body of $K$ is defined by
$DK=K+(-K)$. One can see analyzing the support functions of $K$ and
$\frac{1}{2}DK$, that $K=\frac{1}{2}DK$ if and only if $K$ is origin
symmetric. If $K$ is symmetric with respect to a point, then $\frac{1}{2}DK$
is just a translate of $K$. By the Brunn-Minkowski inequality,
$\left|\frac{1}{2}DK\right|_{n}\geq\left|K\right|_{n}$ with equality if and
only if $K$ is symmetric with respect to some point.

The mean width of $K$ is $w(K)=2\int_{S^{nm-1}}h_{K}(x)d\sigma(x)$ where
$\sigma$ is the rotation invariant probability measure

The Hausdorff distance between two convex bodies $K,L$ is defined by
$\|h_{K}-h_{L}\|_{\infty}$, where $\|\cdot\|_{\infty}$ denotes supremum norm.

For $v\in S^{n-1}$ we denote by $R_{v}\in\operatorname{M}_{n,n}(\mathbb{R})$
the matrix of the reflection in $\mathbb{R}^{n}$ with respect to $v^{\perp}$.
This is, $R_{v}(w+\lambda v)=w-\lambda v$ for every $w\in v^{\perp}$ and
$\lambda\in\mathbb{R}$. Notice that also $R_{v}(x+vt)=x-vt$ for every $x\in
v^{\perp m}$ and $t\in\operatorname{M}_{1,m}(\mathbb{R})$.

##  3\. Symmetrization

In this section we establish basic properties of the fiber symmetrization in
convex sets and quasi-concave functions. Some of these results appear already
in [5, 6].

######  Proposition 3.1.

The operator $\bar{S}_{v}$ satisfies the following properties:

  1. 3.1-1

$\left|\bar{S}_{v}K\right|_{nm}\geq\left|K\right|_{nm}$ with equality if and
only if $K\cap(y+v^{m})$ is symmetric (with respect to a point possibly
depending on $y$) for almost every $y\in v^{\perp m}$.

  2. 3.1-2

If $K\subseteq\operatorname{M}_{n,1}(\mathbb{R})$ is convex then
$\bar{S}_{v}K=S_{v}K$ is the usual Steiner symmetrization. In particular it
preserves $n$-dimensional volume.

  3. 3.1-3

It $K\subseteq\operatorname{M}_{1,m}(\mathbb{R})$ and $v\in
S^{0}=\\{-1,+1\\}$, then $\bar{S}_{\pm 1}K=\frac{1}{2}DK$.

  4. 3.1-4

If $K\subseteq L$ then

$\bar{S}_{v}K\subseteq\bar{S}_{v}L$

  5. 3.1-5

If $K$ is a convex body, then so is $\bar{S}_{v}K$.

  6. 3.1-6

Let $A\in\operatorname{M}_{n,n}(\mathbb{R})$ be an orthogonal matrix and
$K\in\mathcal{K}^{n,m},v\in S^{n-1}$, then

$\bar{S}_{Av}(AK)=A.\bar{S}_{v}K$

  7. 3.1-7

If $K\in\mathcal{K}^{n,d}$ and $A\in\operatorname{M}_{d,m}(\mathbb{R})$ is any
matrix, we have

$(\bar{S}_{v}K)A\subseteq\bar{S}_{v}(KA).$

Moreover, if the rank of $A$ is $d$ (we implicitly assume $m\geq d$) then
there is equality.

  8. 3.1-8

If $K_{i}\in\mathcal{K}^{n,m_{i}}$ with $m_{i}\in\mathbb{N}$,
$m_{1}+\cdots+m_{r}=m$, and $K_{1}\times\cdots\times
K_{r}\subseteq\operatorname{M}_{n,m}(\mathbb{R})$,

(7) $\bar{S}_{v}(K_{1}\times\cdots\times K_{r})=S_{v}K_{1}\times\cdots\times
S_{v}K_{r}.$

  9. 3.1-9

If $K\in\mathcal{K}^{n,m}$ and $A\in\operatorname{M}_{d,m}(\mathbb{R})$ is a
rank $m$ matrix (we implicitly assume $d\geq m$), then

$\bar{A}^{-1}(\bar{S}_{v}K)=\bar{S}_{v}\bar{A}^{-1}(K).$

###### Proof.

3.1-1 If $\left|L\right|_{m}<\infty$, by Brunn-Minkowski,
$\left|\frac{1}{2}DL\right|_{m}\geq\left|L\right|_{m}$ with equality if and
only if $L$ is symmetric with respect to some point. Then by Fubini,
$\left|\bar{S}_{v}K\right|_{nm}\geq\left|K\right|_{nm}$ with equality if and
only if $K\cap(y+v^{m})$ is symmetric (with respect to a point possibly
depending on $y$) for almost every $y\in v^{\perp m}$.

3.1-2 If $m=1$ and $K\subseteq\mathbb{R}^{n}$ is convex, then $K\cap(x+v^{m})$
is a one dimensional interval, and $\frac{1}{2}D\left(K\cap(x+v^{m})\right)$
is the same interval centered at the origin.

3.1-3 This is clear from the definition.

3.1-4 This follows from the fact that $D$ is a monotone operator.

3.1-5 This fact was proven in [7, Theorem 2.3].

3.1-6 This is clear from the definition.

3.1-7 Let $z\in(\bar{S}_{v}K)A$. By formula (3),

$z=(x+v\frac{t-s}{2})A,\text{ with }x+vt,x+vs\in K,\text{ and }v^{t}x=0.$

Clearly $xA+vtA,xA+vsA\in KA$ and $v^{t}xA=0$, which implies that

$z=xA+v\frac{tA-sA}{2}\in\bar{S}_{v}(KA).$

Conversely, let $z\in\bar{S}_{v}(KA)$, then

$z=x+v\frac{t-s}{2}$

with $t,s\in\operatorname{M}_{1,m}(\mathbb{R})$,
$x\in\operatorname{M}_{n,m}(\mathbb{R})$, $x+vt,x+vs\in KA$ and $v^{t}x=0$.

We can write

(8) $\displaystyle x+vt$ $\displaystyle=\tilde{x}_{1}A+v\tilde{t}A$ (9)
$\displaystyle x+vs$ $\displaystyle=\tilde{x}_{2}A+v\tilde{s}A$

with $\tilde{t},\tilde{s}\in\operatorname{M}_{1,d}(\mathbb{R})$,
$\tilde{x}_{1},\tilde{x}_{2}\in\operatorname{M}_{n,d}(\mathbb{R})$,
$\tilde{x}_{1}+v\tilde{t},\tilde{x}_{2}+v\tilde{s}\in K$ and
$v^{t}\tilde{x}_{1}=v^{t}\tilde{x}_{2}=0$. But since
$v^{t}\tilde{x}_{1}A=v^{t}\tilde{x}_{2}A=0$, by uniqueness of the
decomposition $\operatorname{M}_{n,m}(\mathbb{R})=v^{m}\oplus v^{\perp m}$, we
must have $t=\tilde{t}A,s=\tilde{s}A$ and $\tilde{x}_{1}A=x=\tilde{x}_{2}A$.
Now using that $A$ has rank $d$, from $\tilde{x}_{1}A=\tilde{x}_{2}A$ we get
$\tilde{x}_{1}=\tilde{x}_{2}$.

Finally, we obtain

$z=(\tilde{x}_{1}+v\frac{\tilde{t}-\tilde{s}}{2})A$

with $\tilde{x}_{1}+v\tilde{t},\tilde{x}_{1}+v\tilde{s}\in
K,v^{t}\tilde{x}_{1}=0$ and we conclude that $z\in(\bar{S}_{v}K)A$.

3.1-8 By formula (3),

(10) $\displaystyle\bar{S}_{v}K$
$\displaystyle=\left\\{(x_{1},\ldots,x_{r})+v\frac{(s_{1},\ldots,s_{r})-(t_{1},\ldots,t_{r})}{2}:x_{i}+vt_{i}\in
K_{i}\right\\}$ (11)
$\displaystyle=\left\\{(x_{i}+v\frac{t_{i}-s_{i}}{2})_{i}:x_{i}+vt_{i}\in
K_{i}\right\\}$ (12) $\displaystyle=K_{1}\times\cdots\times K_{r}.$

3.1-9 Consider the $d\times m$ matrix

$T_{d,m}=\left(\begin{array}[]{c}\operatorname{Id}_{m}\\\
0_{d-m,m}\end{array}\right)$

so that $\overline{T_{d,m}}(x_{1},\ldots,x_{d})=(x_{1},\ldots,x_{m})$ for
$x_{i}\in\mathbb{R}^{n}$. The proposition is true replacing $A$ by $T_{d,m}$,
by property 3.1-8 with $r=2,m_{1}=m,m_{2}=m-d,K_{1}=K,K_{2}=\mathbb{R}^{d-m}$.

If $d=m$, the matrix $A$ must be invertible and the proposition is also true
by property 3.1-7.

For general $A$, we use that $A$ can be decomposed as $A=\varphi T_{d,m}\psi$
where $\varphi\in GL_{m},\psi\in GL_{d}$. We conclude by equation (5) and
properties 3.1-8 and 3.1-7 again. ∎

The following properties of the symmetrization on functions are obtained
immediately.

######  Proposition 3.2.

  1. 3.2-1

A quasi-concave function $f:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}$
is Steiner concave if, and only if, $f^{(v)}=f$ for every $v\in S^{n-1}$.

  2. 3.2-2

If $f(x_{1},\cdots,x_{m})=g(x_{i})$ then

$f^{(v)}(x_{1},\cdots,x_{m})=g^{(v)}(x_{i}).$

  3. 3.2-3

If $K\subseteq\mathbb{R}^{m}=\operatorname{M}_{1,m}(\mathbb{R})$, then
$S^{0}=\\{-1,+1\\}$, and

$f^{(\pm 1)}(x)=\int_{0}^{\infty}\operatorname{1}_{\frac{1}{2}D\\{f\geq
t\\}}(x)dt.$

  4. 3.2-4

If $A\in\operatorname{M}_{d,m}(\mathbb{R})$ is a rank $m$ matrix, then

$(f\circ\bar{A})^{(v)}=f^{(v)}\circ\bar{A}.$

The measure-preserving property of the usual Steiner symmetrization is a
useful property that is lost for the operator $\bar{S}_{v}$. Nevertheless,
$\bar{S}_{v}$ still preserves some weaker measures on convex bodies like the
mean width and the volumes of some projections.

######  Proposition 3.3.

Let $K\subseteq\operatorname{M}_{n,m}(\mathbb{R})$ be a convex body. For any
$A\in GL_{m}$ and $p\geq 1$,

$\int_{S^{n-1}}h_{(\bar{S}_{v}K)A}(v)^{p}dv\leq\int_{S^{n-1}}h_{KA}(v)^{p}dv.$

If $p>1$ there is equality if and only if $R_{v}K=K$.

###### Proof.

By property 3.1-7 it suffices to consider the case $A=I_{m}$.

Recall that $R_{v}\in\operatorname{M}_{n,n}(\mathbb{R})$ is the matrix of the
reflection in $\mathbb{R}^{n}$ with respect to $v^{\perp}$.

Let $x\in v^{\perp m}$, $x+vt,x+vs\in K$ and
$z\in\operatorname{M}_{n,m}(\mathbb{R})$. Then

$\langle x+v\frac{t-s}{2},z\rangle=\frac{1}{2}(\langle x+vt,z\rangle+\langle
x-vs,z\rangle)\leq\frac{1}{2}(h_{K}(z)+h_{K}(R_{v}z))$

Taking supremum for all $x+vt,x+vs\in K$ and using (3), we get
$h_{\bar{S}_{v}K}(z)\leq\frac{1}{2}(h_{K}(z)+h_{R_{v}K}(z))$. Since
$R_{v}S^{nm-1}=S^{nm-1}$, we obtain

$\|h_{\bar{S}_{v}K}\|_{p}\leq\frac{1}{2}(\|h_{K}\|_{p}+\|h_{R_{v}K}\|_{p})=\|h_{K}\|_{p}.$

For the equality case, notice that if
$\|h_{\bar{S}_{v}K}\|_{p}=\|h_{K}\|_{p}$,

$\frac{1}{2}\|h_{K}+h_{R_{v}K}\|_{p}\geq\|h_{\bar{S}_{v}K}\|_{p}=\|h_{K}\|_{p}=\frac{1}{2}(\|h_{K}\|_{p}+\|h_{R_{v}K}\|_{p}),$

which for $p>1$ implies that that there exists $\lambda\in\mathbb{R}$ with
$h_{K}(w)=\lambda h_{R_{v}K}(w)$ for almost every $w$. Integrating on $w\in
S^{nm-1}$ on both sides of this inequality we get $w(K)=\lambda w(R_{v}K)$
which implies that $\lambda=1$.

Thus we obtain $h_{K}(w)=h_{R_{v}K}(w)$ for almost every $w\in S^{nm-1}$. By
continuity, this occurs for every $w\in S^{nm-1}$ and we get $K=R_{v}K$. ∎

For $w\in\operatorname{M}_{m,1}(\mathbb{R})\setminus\\{0\\}$ the function
$\bar{w}:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}^{n}$ is a linear
projection onto an $n$-dimensional space. In particular,
$\overline{e_{i}}=\pi_{i}:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}^{n}$
is the projection onto the $i$-th column, this is, $\pi_{i}(x)$ is the $i$-th
column of $x$.

######  Proposition 3.4.

For $K\subseteq\operatorname{M}_{n,m}(\mathbb{R})$ and
$w\in\operatorname{M}_{m,1}(\mathbb{R})$,

$\left|\bar{w}(\bar{S}_{v}K)\right|_{n}\leq\left|\bar{w}(K)\right|_{n}.$

In particular,

$\left|\pi_{i}(\bar{S}_{v}K)\right|_{n}\leq\left|\pi_{i}(K)\right|_{n}.$

###### Proof.

By the properties 3.1-7, 3.1-2, we have

$\left|\bar{w}(\bar{S}_{v}K)\right|_{n}=\left|(\bar{S}_{v}K)w\right|_{n}\leq\left|S_{v}(Kw)\right|_{n}=\left|Kw\right|_{n}=\left|\bar{w}(K)\right|_{n}$

∎

##  4\. Rogers–Brascamp–Lieb–Luttinger inequalities

We start with an inequality of convex sets which is the geometric core of
Theorem 1.9.

######  Theorem 4.1.

Let
$K_{i}\subseteq\operatorname{M}_{1,m}(\mathbb{R})=\mathbb{R}^{m},i=1,\ldots,k$
be convex sets, (not necessarily convex bodies, possibly with infinite
volume), then

$\left|\bigcap_{i=1}^{k}K_{i}\right|_{m}\leq\left|\bigcap_{i=1}^{k}\frac{1}{2}DK_{i}\right|_{m}.$

If the left-hand side has infinite volume, then so has the right-hand side.

Moreover, if $K$ is symmetric and has finite volume,

$\left|K\setminus L\right|_{m}\geq\left|K\setminus\frac{1}{2}DL\right|_{m}.$

###### Proof.

Let $x\in\frac{1}{2}D(\bigcap_{i=1}^{k}K_{i})$, then $x=\frac{a-b}{2}$ with
$a,b\in\bigcap_{i=1}^{k}K_{i}$. In particular, $x\in\frac{1}{2}DK_{i}$ for
every $i$. This proves that

$\frac{1}{2}D\left(\bigcap_{i=1}^{k}K_{i}\right)\subseteq\bigcap_{i=1}^{k}\frac{1}{2}DK_{i}.$

By Brunn-Minkowski inequality,

$\left|\bigcap_{i=1}^{k}K_{i}\right|_{m}\leq\left|\frac{1}{2}D\bigcap_{i=1}^{k}K_{i}\right|_{m}\leq\left|\bigcap_{i=1}^{k}\frac{1}{2}DK_{i}\right|_{m}$

and the first part of the theorem follows.

For the second part,

(13) $\displaystyle\left|K\setminus L\right|_{m}$
$\displaystyle=\left|K\right|_{m}-\left|K\cap L\right|_{m}$ (14)
$\displaystyle\geq\left|K\right|_{m}-\left|\frac{1}{2}DK\cap\frac{1}{2}DL\right|_{m}$
(15) $\displaystyle\geq\left|K\right|_{m}-\left|K\cap\frac{1}{2}DL\right|_{m}$
(16) $\displaystyle=\left|K\setminus\frac{1}{2}DL\right|_{m}.$

∎

We remark here that Theorem 4.1 is neither stronger nor weaker than the
inequality
$\left|\bigcap_{i=1}^{k}K_{i}\right|_{m}\leq\max_{i}\\{\left|K_{i}\right|_{m}\\}$
which is obtained applying inequality (1) with the symmetric decreasing
rearrangement. For example, if $K_{1},K_{2}$ are two symmetric convex bodies
of volume $1$ with very small intersection, then Theorem 4.1 gives an
equality, while the inequality $\left|K_{1}\cap K_{2}\right|_{m}<1$ is strict.

######  Theorem 4.2.

Let $K_{i}\subseteq\operatorname{M}_{n,m}(\mathbb{R}),i=1,\ldots,k$ be convex
sets (not necessarily convex bodies), and $v\in S^{n-1}$, then

$\left|\bigcap_{i=1}^{k}K_{i}\right|_{nm}\leq\left|\bigcap_{i=1}^{k}\bar{S}_{v}K_{i}\right|_{nm}.$

If the left-hand side has infinite volume, then so has the right-hand side.
Moreover, if $K$ satisfies $\bar{S}_{v}K=K$ has finite volume,

$\left|K\setminus L\right|_{nm}\geq\left|K\setminus\bar{S}_{v}L\right|_{nm}.$

###### Proof.

By Fubini and Theorem 4.1,

(18) $\displaystyle\left|\bigcap_{i=1}^{k}K_{i}\right|_{nm}$
$\displaystyle=\int_{v^{\perp
m}}\left|(x+v^{m})\cap\bigcap_{i=1}^{k}K_{i}\right|_{m}dx$ (19)
$\displaystyle=\int_{v^{\perp m}}\left|\bigcap_{i=1}^{k}\left((x+v^{m})\cap
K_{i}\right)\right|_{m}dx$ (20) $\displaystyle\leq\int_{v^{\perp
m}}\left|\bigcap_{i=1}^{k}\frac{1}{2}D\left((x+v^{m})\cap
K_{i}\right)\right|_{m}dx$ (21) $\displaystyle=\int_{v^{\perp
m}}\left|\bigcap_{i=1}^{k}(x+v^{m})\cap\bar{S}_{v}K_{i}\right|_{m}dx$ (22)
$\displaystyle=\int_{v^{\perp
m}}\left|(x+v^{m})\cap\bigcap_{i=1}^{k}\bar{S}_{v}K_{i}\right|_{m}dx=\left|\bigcap_{i=1}^{k}\bar{S}_{v}K_{i}\right|_{nm}$

The second statement follows as in the second part of the proof of Theorem
4.1. ∎

######  Theorem 4.3.

Let $f_{i}:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R},i=1,\ldots,k$ be
non-negative and quasi-concave. Then

$\int\prod_{i=1}^{k}f_{i}(x)dx\leq\int\prod_{i=1}^{k}f_{i}^{(v)}(x)dx$

###### Proof.

By the layer-cake formula,

(23) $\displaystyle\int\prod_{i=1}^{k}f_{i}(x)dx$
$\displaystyle=\int_{0}^{\infty}\cdots\int_{0}^{\infty}\left|\bigcap_{i=1}^{k}\\{f_{i}\geq
t_{i}\\}\right|_{nm}dt_{1}\cdots dt_{k}$ (24)
$\displaystyle\leq\int_{0}^{\infty}\cdots\int_{0}^{\infty}\left|\bigcap_{i=1}^{k}\bar{S}_{v}\\{f_{i}\geq
t_{i}\\}\right|_{nm}dt_{1}\cdots dt_{k}$ (25)
$\displaystyle=\int_{0}^{\infty}\cdots\int_{0}^{\infty}\left|\bigcap_{i=1}^{k}\\{f_{i}^{(v)}\geq
t_{i}\\}\right|_{nm}dt_{1}\cdots dt_{k}$ (26)
$\displaystyle=\int\prod_{i=1}^{k}f_{i}^{(v)}(x)dx$

∎

Finally, we can deduce the Rogers–Brascamp–Lieb–Luttinger inequality in its
full generality.

###### Proof of Theorem 1.9.

Apply Theorem 4.3 and use property 3.2-4. ∎

For convex functions we obtain the following:

######  Theorem 4.4.

Let $f:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}$ be a non-negative and
quasi-concave and Steiner-concave function, and let
$g:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}$ be a non-negative and
quasi-convex, then

$\int f(x)g(x)dx\geq\int f(x)g_{(v)}(x)dx$

###### Proof.

By the layer-cake formula, Definition 1.8, and the second part of Theorem 4.2,

(27) $\displaystyle\int f(x)g(x)dx$
$\displaystyle=\int_{0}^{\infty}\int_{0}^{\infty}\left|\\{f\geq
t\\}\cap\\{g\geq t_{2}\\}\right|_{nm}dt_{1}dt_{2}$ (28)
$\displaystyle=\int_{0}^{\infty}\int_{0}^{\infty}\left|\\{f\geq
t\\}\setminus\\{g<t_{2}\\}\right|_{nm}dt_{1}dt_{2}$ (29)
$\displaystyle\geq\int_{0}^{\infty}\int_{0}^{\infty}\left|\bar{S}_{v}\\{f\geq
t\\}\setminus\bar{S}_{v}\\{g<t_{2}\\}\right|_{nm}dt_{1}dt_{2}$ (30)
$\displaystyle=\int_{0}^{\infty}\int_{0}^{\infty}\left|\\{f^{(v)}\geq
t\\}\setminus\\{g^{(v)}<t_{2}\\}\right|_{nm}dt_{1}dt_{2}$ (31)
$\displaystyle=\int_{0}^{\infty}\int_{0}^{\infty}\left|\\{f^{(v)}\geq
t\\}\cap\\{g^{(v)}\geq t_{2}\\}\right|_{nm}dt_{1}dt_{2}=\int
f(x)g_{(v)}(x)dx.$

∎

##  5\. Some Consequences

###  5.1. Particular cases of Theorem 1.9

As mentioned in the introduction, we see that Theorem 1.3 is a particular case
of Theorem 1.9.

A particular case of Theorem 4.3 (which is Theorem 1.9 with $m=d$ and
$L_{i}=\operatorname{Id}$) is the following

######  Corollary 5.1.

Let $f:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}$ be Steiner-concave,
and $K\in\mathcal{K}^{n,m}$, then

$\int_{K}f(x)dx\leq\int_{\bar{S}_{v}K}f(x)dx.$

For example we have

######  Corollary 5.2.

Let $K\subseteq\operatorname{M}_{n,m}(\mathbb{R})$ a measurable set and
$\gamma:(0,\infty)\to(0,\infty)$ is a decreasing function, then

$\int_{K}\gamma(\left|\operatorname{conv}(\\{x_{1},\ldots,x_{m}\\})\right|_{n})dx\leq\int_{\bar{S}_{v}K}\gamma(\left|\operatorname{conv}(\\{x_{1},\ldots,x_{m}\\})\right|_{n})dx$

For $n=1$ we obtain

######  Corollary 5.3.

Let $f_{i}:\mathbb{R}^{m}\to\mathbb{R},i=1,\ldots k$, then

$\int_{\mathbb{R}^{m}}\prod_{i=1}^{k}f_{i}(x)dx\leq\int_{\mathbb{R}^{m}}\prod_{i=1}^{k}f_{i}^{(1)}(x)dx.$

where $f_{i}^{(1)}$ is defined as in property (3.2-3).

Notice that, as in the remark after Theorem 4.1, this inequality cannot be
proven directly with inequality (1).

###  5.2. Operator norms

For any convex bodies $K\subseteq\mathbb{R}^{m},L\subseteq\mathbb{R}^{n}$
consider the set

$B_{K,L}=\\{x\in\operatorname{M}_{n,m}(\mathbb{R}):xw\in L,\ \ \forall w\in
K\\}.$

Where we identify vectors in $\mathbb{R}^{m}$ and $\mathbb{R}^{n}$ with column
vectors in $\operatorname{M}_{m,1}(\mathbb{R})$ and
$\operatorname{M}_{n,1}(\mathbb{R})$ respectively.

If $K,L$ are symmetric, the set
$B_{K,L}\subseteq\operatorname{M}_{n,m}(\mathbb{R})$ is the unit ball in the
operator norm, of the set of linear maps between Banach spaces
$(\mathbb{R}^{m},K)\to(\mathbb{R}^{n},L)$. Each one of these map induces a
linear map of the dual spaces

$(\mathbb{R}^{m},L)^{*}\to(\mathbb{R}^{n},K)^{*}$

by transposition. In general (for non symmetric convex bodies $K,L$) this
reads as

(32) $B_{K,L}^{t}=B_{L^{\circ},K^{\circ}}.$

In [6] the following result is obtained as a limit case of the higher-order
$L^{p}$ Petty-projection inequality. We give here a shorter and clearer proof.

######  Theorem 5.4.

For any convex bodies $K\subseteq\mathbb{R}^{m},L\subseteq\mathbb{R}^{n}$,

$\bar{S}_{v}B_{K,L}\subseteq B_{K,S_{v}L}$

###### Proof.

Take any $w\in K\subseteq\operatorname{M}_{m,1}(\mathbb{R})$, so that
$B_{K,L}w\subseteq L$. By (3.1-7) and (3.1-4),

$(\bar{S}_{v}B_{K,L})w\subseteq\bar{S}_{v}(B_{K,L}w)\subseteq\bar{S}_{v}L.$

This implies that $\bar{S}_{v}B_{K,L}\subseteq B_{K,S_{v}L}$. ∎

The theorem obviously implies that
$\left|B_{K,L}\right|_{nm}\leq\left|B_{K,B_{L}}\right|_{nm}$ where $B_{L}$ is
the centered euclidean ball of same volume as $L$. Some more can be said:

######  Theorem 5.5.

Let $F:\operatorname{M}_{n,m}(\mathbb{R})\to\mathbb{R}$ be non-negative and
Steiner concave, then

$\int_{B_{K,L}}F(x)dx\leq\int_{B_{K,B_{L}}}F(x)dx.$

Let $F:\operatorname{M}_{m,n}(\mathbb{R})\to\mathbb{R}$ be non-negative and
Steiner concave, then

$\int_{B_{K,L}}F(x^{t})dx\leq\int_{B_{B_{K^{\circ}},L}}F(x^{t})dx.$

For example, for $n=m$ and $\beta\in(-1,0)$,

(33) $\displaystyle\int_{B_{K,L}}|\det(x)|^{\beta}dx$
$\displaystyle\leq\left(\frac{\left|L\right|_{n}\left|K^{\circ}\right|_{n}}{\left|B_{2}^{n}\right|_{n}^{2}}\right)^{n+\beta}\int_{B_{B,B}}|\det(x)|^{\beta}dx$
(34)
$\displaystyle\leq\left(\frac{\left|L\right|_{n}}{\left|K\right|_{n}}\right)^{n+\beta}\int_{B_{B,B}}|\det(x)|^{\beta}dx=\int_{B_{B_{K},B_{L}}}|\det(x)|^{\beta}dx$

###### Proof.

The first inequality is a direct consequence of Theorem 4.3. The second one,
comes from the first inequality, formula (32) and the change of variables
$x\mapsto x^{t}$.

For the third inequality apply both inequalities and the fact that
$x\mapsto|\det(x)|^{\beta}$ is Steiner concave and invariant by transposition,
to obtain

(36) $\displaystyle\int_{B_{K,L}}|\det(x)|^{\beta}dx$
$\displaystyle\leq\int_{B_{K,B_{L}}}|\det(x)|^{\beta}dx$ (37)
$\displaystyle\leq\int_{B_{B_{L}^{\circ},K^{\circ}}}|\det(y)|^{\beta}dy$ (38)
$\displaystyle\leq\int_{B_{B_{L}^{\circ},B_{K^{\circ}}}}|\det(y)|^{\beta}dy$
(39)
$\displaystyle\leq\int_{B_{(B_{K^{\circ}})^{\circ},B_{L}}}|\det(x)|^{\beta}dx$
(40)
$\displaystyle\leq(\frac{\left|L\right|_{n}\left|K^{\circ}\right|_{n}}{\left|B_{2}^{n}\right|_{n}^{2}})^{n+\beta}\int_{B_{2,2}}|\det(x)|^{\beta}dx.$

The last inequality follows from the Blaschke-Santaló inequality.

∎

###  5.3. Schneider’s Difference body

For $K\subseteq\mathbb{R}^{n}$ and $m=1$, Schneider [10] defined the higher-
order difference body as

$D^{m}K=\left\\{(x_{1},\ldots,x_{m}):K\cap(K+x_{1})\cap\cdots\cap(K+x_{m})\neq\emptyset\right\\}.$

Here we prove that Schneider’s operator intertwines with the higher-order
symmetrization.

######  Theorem 5.6.

If $K\in\mathcal{K}^{n,m}$, $\bar{S}_{v}(D^{m}K)\supseteq D^{m}(S_{v}K)$.

###### Proof.

Define the matrix $P_{m}\in\operatorname{M}_{m+1,m}(\mathbb{R})$ by

$P_{m}=\left(\begin{array}[]{cccc}1&1&\cdots&1\\\ -1&0&\cdots&0\\\
0&-1&\cdots&0\\\ \cdots&\cdots&\cdots&\cdots\\\ 0&0&\cdots&-1\\\
\end{array}\right).$

It is easy to see that $D^{m}K=K^{m+1}P_{m}$. By properties (3.1-7) and
(3.1-8),

$\bar{S}_{v}(D^{m}K)\supseteq\bar{S}_{v}(K^{m+1})P_{m}=(S_{v}K)^{m+1}P_{m}=D^{m}(S_{v}K).$

∎

In [10] Schneider conjectured that the volume of $D^{m}K$ is minimized among
convex sets if $K$ is an ellipsoid. Unfortunately Theorem 5.6 is not
appropriate to study this conjecture, since the operator $\bar{S}_{v}$ is
volume-increasing.

However, combining Theorems 5.6 and Proposition 3.3 we get a (much weaker)
result in this direction.

######  Theorem 5.7.

Among all convex bodies $K\subseteq\mathbb{R}^{n}$ of a fixed volume, the
Euclidean balls minimize the mean width of $D^{m}K$.

Some more can be said:

######  Theorem 5.8.

Let $L\in\mathcal{K}^{n,m}$ satisfy $R_{v}L=L$ for every $v\in S^{n-1}$ (see
Section 6 for examples of invariant sets). Among all convex bodies
$K\subseteq\mathbb{R}^{n}$ of a fixed volume, the Euclidean balls minimize the
integral

$\int_{L}h_{D^{m}K}(x)dx.$

###### Proof.

As in the proof of Proposition 3.3, for every $E\in\mathcal{K}^{n,m}$,

$\int_{L}h_{\bar{S}_{v}E}(x)dx\leq\int_{L}h_{E}(x)dx.$

For $E=D^{m}K$, by Theorem 5.6,

$\int_{L}h_{D^{m}S_{v}K}\leq\int_{L}h_{\bar{S}_{v}D^{m}K}\leq\int_{L}h_{D^{m}K}.$

∎

##  6\. Invariant sets

In this section we study the sets $K$ that remain invariant under
$\bar{S}_{v}$.

######  Definition 6.1.

A convex body $K\in\mathcal{K}^{n,m}$ is said to be $O_{n}$ invariant if
$RK=K$ for every $n$-dimensional rotation $R\in O_{n}$. Notice that this does
not necessarily imply that $K$ is a ball, since $R$ is an $n\times n$ matrix,
not $nm\times nm$.

Contrary to the classical case $m=1$ where we only have the centered Euclidean
balls of different radius, many convex bodies in
$\operatorname{M}_{n,m}(\mathbb{R})$ are $O_{n}$ invariant. As examples we
have the unit ball $B_{2}^{nm}$, the product of balls $(B_{2}^{n})^{m}$, the
unit ball of any operator norm of the type $B_{K,B_{2}^{n}}$ and the ball of
any unitary invariant norm (in particular any Schatten class).

Here we show that $O_{n}$ invariant convex bodies characterize the sets
invariant by the operator $\bar{S}_{v}$.

######  Proposition 6.2.

Let $K\in\mathcal{K}^{n,m}$. The following statements are equivalent:

  1. (1)

For every $v\in S^{n-1}$, $\bar{S}_{v}K=K$.

  2. (2)

For every $v\in S^{n-1}$, $R_{v}K=K$.

  3. (3)

$K$ is $O_{n}$ invariant.

If $K$ is also symmetric, it is the unit ball of a matrix norm
$\|\cdot\|_{K}$. Then $O_{n}$ invariance just means left-orthogonal invariance
for the norm.

###### Proof.

Since the reflections $R_{v},v\in S^{n-1}$ generate $O_{n}$,
$(2)\Leftrightarrow(3)$ are clearly equivalent. To see $(1)\Leftrightarrow(2)$
take $v\in S^{n-1}$. The property $\bar{S}_{v}K=K$ holds if and only if for
any $y\in v^{\perp m}$, the set
$K_{y}=\\{t\in\operatorname{M}_{1,m}(\mathbb{R}):y+vt\in K\\}$ is equal to
half its difference body, and this is equivalent to $K_{y}=-K_{y}$. This
happens when, for any such $y$, $x=y+vt$ belongs to $K$ if and only if
$R_{v}x=y-vt$ does.

∎

For functions we obtain the following characterization:

######  Proposition 6.3.

A quasi-concave function $f$ is Steiner-concave if and only if $f(Rx)=f(x)$
for every $R\in O_{n}$.

Finally, as in the classical case, every convex body can be transformed into
an extremal set.

######  Theorem 6.4.

Let $K\subseteq\operatorname{M}_{n,m}(\mathbb{R})$ be a convex body, then
there exists an $O_{n}$ invariant convex body
$L\subseteq\operatorname{M}_{n,m}(\mathbb{R})$ such that for every
$\varepsilon>0$ there are vectors $v_{1},\ldots,v_{k}$ with

$d_{H}(\bar{S}_{v_{m}}\circ\cdots\circ\bar{S}_{v_{1}}K,L)<\varepsilon,$

where $d_{H}$ is the Hausdorff distance.

###### Proof.

For a $k$-tuple of vectors $\bar{v}=(v_{1},\ldots,v_{k})$ we denote
$\bar{S}_{\bar{v}}=\bar{S}_{v_{m}}\circ\cdots\circ\bar{S}_{v_{1}}$ the
composition of the symmetrizations $\bar{S}_{v_{i}}$.

Let $R>r>0$ such that $B(0,R)\supseteq K\supseteq B(0,r)$. Recall that by
property (3.1-4) we always have $B(0,R)\supseteq\bar{S}_{\bar{v}}K\supseteq
B(0,r)$. Fix any $p>1$ and consider the infimum

$w_{p,\text{min}}=\inf\\{\|h_{\bar{S}_{\bar{v}}K}\|_{p}:k\geq
1,\bar{v}=(v_{1},\ldots,v_{k}),v_{i}\in S^{n-1}\\}$

which in view of $\bar{S}_{\bar{v}}K\supseteq B(0,r)$, has to be positive.
Take a sequence of tuples of vectors $\bar{v}^{(i)}$ such that
$\|h_{\bar{S}_{\bar{v}^{(i)}}K}\|_{p}\to w_{p,\text{min}}$. Since
$B(0,R)\supseteq\bar{S}_{\bar{v}^{(i)}}K$ we may apply Blaschke ’s selection
theorem to obtain a subsequence (again denoted by $\bar{v}^{(i)}$) such that
$S_{\bar{v}^{(i)}}K$ converges in the Hausdorff distance to a convex body $L$.
By continuity, $\|h_{L}\|_{p}=w_{p,\text{min}}$.

Take any $v\in S^{n-1}$. By continuity and Proposition 3.3,

(41) $\displaystyle w_{p,\text{min}}$ $\displaystyle=\|h_{L}\|_{p}$ (42)
$\displaystyle\geq\|h_{\bar{S}_{v}L}\|_{p}$ (43)
$\displaystyle=\lim_{i\to\infty}\|h_{\bar{S}_{v}\bar{S}_{\bar{v}^{(i)}}}K\|_{p}$
(44) $\displaystyle\geq w_{p,\text{min}}$

and we get $\|h_{L}\|_{p}=\|h_{\bar{S}_{v}L}\|_{p}$. The equality case of
Proposition 3.3 (notice that we chose $p>1$) implies $L=R_{v}L$.

Since $v\in S^{n-1}$ was arbitrary, $L$ must be $O_{n}$ invariant by
Proposition 6.2.

∎

The results in this last section point to the fact that $\bar{S}_{v}$ makes a
matrix norm $\|\cdot\|_{K}$ more and more left-invariant. Of course one can
also consider the transformation $K\mapsto(\bar{S}_{v}K^{t})^{t}$ which makes
the norm more right-invariant. If the norm is transformed by a sequence of
left and right symmetrizations into a limiting norm $\|\cdot\|_{L}$, then it
will be unitarily invariant and, by singular value decomposition, $L$ will be
uniquely determined by the set of singular values of the matrices inside $L$.
Unfortunately it appears that the set of singular values may vary from $K$ to
$\bar{S}_{v}K$ for arbitrary $K$, so it is not clear how (if possible) to
determine $L$ from $K$.

## References

  * [1] Artstein-Avidan, S., Giannopoulos, A., and Milman, V. D. (2015). Asymptotic Geometric Analysis, Part I. (Mathematical surveys and monographs; Vol. volume 202). American Mathematical Society. https://doi.org/10.1090/surv/202
  * [2] Bianchi, G., Gardner, R. J., and Gronchi, P. Symmetrization in geometry. Adv. Math. 306 (2017),
  * [3] Brascamp H. J., Lieb E. H. and Luttinger J. M. A general rearrangement inequality for multiple integrals. Journal of functional analysis 17.2 (1974): 227-237.
  * [4] M. Christ, Estimates for the k-plane transform, Indiana Univ. Math. J. 33 (1984), no. 6, 891–910.
  * [5] Haddad, J., Langharst, D., Putterman, E., Roysdon, M. and Ye, D., 2023. Affine isoperimetric inequalities for higher-order projection and centroid bodies. arXiv preprint arXiv:2304.07859.
  * [6] Haddad, J., Langharst, D., Putterman, E., Roysdon, M. and Ye, D., 2023. General Higher Order $L^{p}$ Isoperimetric and Sobolev Inequalities. arXiv preprint arXiv:2305.17468.
  * [7] McMullen, P. New combinations of convex sets. Geom. Dedicata 78, 1 (1999), 1–19.
  * [8] Paouris, G. and Pivovarov, P., 2017. Randomized isoperimetric inequalities. In Convexity and concentration (pp. 391-425). Springer New York.
  * [9] C. A. Rogers, A single integral inequality, J. London Math. Soc. 32 (1957), 102–108.
  * [10] Schneider, R., Eine Verallgemeinerung des Differenzenkörpers. Monatsh. Math. 74 (1970), 258–272. 51–88.
  * [11] Schneider, R. Convex Bodies: the Brunn-Minkowski Theory, 2nd expanded ed., vol. 151 of Encyclopedia of Mathematics and its Applications. Cambridge University