# A Comparative Analysis of the Ensemble Methods for Drug Design

Rifkat Davronov<EMAIL_ADDRESS>Fatima Adilova
<EMAIL_ADDRESS>Institute of Mathematics Academy of Sciences, Tashkent,
Uzbekistan

###### Abstract

Quantitative structure-activity relationship (QSAR) is a computer modeling
technique for identifying relationships between the structural properties of
chemical compounds and biological activity. QSAR modeling is necessary for
drug discovery, but it has many limitations. Ensemble-based machine learning
approaches have been used to overcome limitations and generate reliable
predictions. Ensemble learning creates a set of diverse models and combines
them. In our comparative analysis, each ensemble algorithm was paired with
each of the basic algorithms, but the basic algorithms were also investigated
separately. In this configuration, 57 algorithms were developed and compared
on 4 different datasets. Thus, a technique for complex ensemble method is
proposed that builds diversified models and integrates them. The proposed
individual models did not show impressive results as a unified model, but it
was considered the most important predictor when combined. We assessed whether
ensembles always give better results than individual algorithms. The Python
code written to get experimental results in this article has been uploaded to
Github (https://github.com/rifqat/Comparative-Analysis).

##  I Introduction

Quantitative structure-activity relationship (QSAR) is a computer or
mathematical modeling method for identifying the relationship between
biological activity and the structural properties of chemical compounds. The
underlying principle is that variations in structural properties cause
different biological activities [1]. Structural properties refer to
physicochemical properties, and biological activity corresponds to various
pharmacokinetic properties, such as absorption, distribution, metabolism,
excretion and toxicity.

QSAR simulations help rank the hit line of a large number of chemicals in
terms of their desired biological activity and greatly reduces the number of
candidates for testing. QSAR modeling has become a common process in
pharmacology, but with all the progress of QSAR, even after the well-known
article by a group of co-authors [2] , there are many limitations [3, 4].

For example: the data may include more than hundreds of thousands of
compounds, or, on the contrary, a very small sample; each compound can be
represented by multiple descriptors; some features are highly correlated; it
is assumed that the dataset contains some errors as relationships are
estimated through in-situ experiments. Due to these and other limitations for
predicting a QSAR based model, it is difficult to achieve a reliable
prediction result.

In forecasting based on QSAR, machine learning approaches were applied: linear
regression models [5] and Bayesian neural networks [6-8] were used. The random
forest (RF) [9,10] deserves special mention - it is the most frequently used
algorithm with a high level of predictability, simplicity, and reliability. RF
is a kind of ensemble method based on sets of decision trees that can prevent
overfitting. RF is considered the gold standard in this area, so new QSAR
forecasting methods are often compared in performance with this algorithm.

The well-known Merck Kaggle competition in 2012 drew people’s attention to
neural networks. The winning team used multitasking neural networks (MTNN)
[11]. The fundamental learning structure is based on simple feedforward neural
networks; it avoids overfitting by studying multiple biological analyzes at
the same time. The team achieved results that consistently outperformed the
random forest algorithm. Despite achieving high performance with a
multitasking neural network, this team ended up using an ensemble combining
different methods. RF, and many of the algorithms in the famous Kaggle
competition used ensemble learning, a technique that creates a set of training
models and combines multiple models to produce final predictions. It has been
shown theoretically and empirically that the predictive power of ensemble
learning is superior to the predictive power of an individual algorithm even
if the last are accurate and diverse [12-15]. Ensemble learning manages the
strengths and weaknesses of learning individual algorithms, similar to
consensus decision making in critical situations.

Thus, it is popular to use algorithm ensembles by using several algorithms and
combining their results. In ensembles, the base algorithms generate partially
dependent or independent results on the same or a different part of a dataset,
and then results are combined in several ways. The success of an ensemble
depends on two main properties: the first is the individual success of the
base algorithms of the ensemble, and the second one is the independence of
base algorithms’ results from each other (low error, high diversity)[16].

Ensemble methods, including an ensemble of neural networks based on bootstrap
sampling in QSAR (data sampling ensemble) [17]; ensemble versus different
training methods for drug interactions [18], Bayesian ensemble model with
various QSAR instruments (ensemble method) [8], ensemble training based on
qualitative and quantitative SAR models [19], hybrid QSAR prediction model
with various training methods [20 ], ensembles with different boosting methods
[21], hybrid feature selection and training in QSAR simulations [22], and
ensemble against various chemicals to predict carcinogenicity (representative
ensembles) [23] have been widely used in drug-like studies.

In contrast to the work in which the results of a comparative analysis of
ensemble algorithms are presented [24],this study aims at overcoming the
difficulties of QSAR modeling by using ensembles. Our experiments focus on
regression ensembles because this type of models is simpler and easier to
understand for medical chemists. The performance of ensemble algorithms is
investigated with respect to ensemble algorithms themselves, and the base
algorithms used within the ensemble algorithms.

The article consists of the following sections: in section 2 we described
ensemble and base regression algorithms, dimension reduction process, dataset
collection. In section 3 presented the results of simulation running and their
discussion. In conclude part (section 4) we showed the previous works in order
to detailed the success of our results in comparing with achievements from
different studies.

##  II  Materials and Methods

In this section, the base and ensemble algorithms used in our study are
briefly described. For the evaluation of the algorithms, the scikit-learn
library was used [25]. Each ensemble algorithm was used with each of the base
algorithms. The base algorithms were also used alone. With this configuration
(2 ensemble + 1 single) x (19 base) = 57 different algorithms were obtained
and used.

We ran our experiments on Windows 10 (Intel(R) Core(TM) i7-9700 CPU © 3.00GHz
3.00 GHz). We used the Scikit-learn library package (version 0.23.2) for
conventional machine learning methods.

###  II.1  Ensemble algorithms

Bagging/bootstrapping (BG): Bagging generates N new equal-sized datasets from
the original dataset by selecting samples with a replacement [26]. The base
algorithms are trained with the datasets. The independence of the individual
results is confirmed in the experiments to some degree. N was chosen as 10 in
our experiments. The results of the base algorithms are simply averaged to
produce the ensemble result.

Additive regression (AR): This is the adaptation of the AdaBoost algorithm to
regression types of problems [27]. At each iteration, the samples having big
errors at the previous iteration are considered. The iteration number was
chosen as 10 in our study. The ensemble result is the weighted mean of the
base algorithms. The weights are inversely proportional to the errors of the
base algorithms.

###  II.2  Base regression algorithms

In our study, 19 regression algorithms were used as base learners in the
ensembles. They are as follows:

Lasso: The Lasso is a linear model that estimates sparse coefficients. It is
useful in some contexts due to its tendency to prefer solutions with fewer
non-zero coefficients, effectively reducing the number of features upon which
the given solution is dependent. For this reason Lasso and its variants are
fundamental to the field of compressed sensing. Under certain conditions, it
can recover the exact set of non-zero coefficients [26].

Ridge: This model solves a regression model where the loss function is the
linear least squares function and regularization is given by the l2-norm. Also
known as Ridge Regression or Tikhonov regularization. This estimator has
built-in support for multi-variate regression[27].

ElasticNet: ElasticNet is a linear regression model trained with both $\ell$1
and $\ell$2-norm regularization of the coefficients. This combination allows
for learning a sparse model where few of the weights are non-zero like Lasso,
while still maintaining the regularization properties of Ridge[28].

Orthogonal Matching Pursuit (OMP): OMP is based on a greedy algorithm that
includes at each step the atom most highly correlated with the current
residual. It is similar to the simpler matching pursuit (MP) method, but
better in that at each iteration, the residual is recomputed using an
orthogonal projection on the space of the previously chosen dictionary
elements[29].

Bayesian Regression: Bayesian regression techniques can be used to include
regularization parameters in the estimation procedure: the regularization
parameter is not set in a hard sense but tuned to the data at hand[30].

Automatic Relevance Determination: Fit the weights of a regression model,
using an ARD prior. The weights of the regression model are assumed to be in
Gaussian distributions. Also estimate the parameters lambda (precisions of the
distributions of the weights) and alpha (precision of the distribution of the
noise). The estimation is done by an iterative procedures (Evidence
Maximization)[31].

Passive Aggressive Algorithms: The passive-aggressive algorithms are a family
of algorithms for large-scale learning. They are similar to the Perceptron in
that they do not require a learning rate. However, contrary to the Perceptron,
they include a regularization parameter C [32].

Theil-Sen estimator: TheilSenRegressor is comparable to the Ordinary Least
Squares (OLS) in terms of asymptotic efficiency and as an unbiased estimator.
In contrast to OLS, Theil-Sen is a non-parametric method which means it makes
no assumption about the underlying distribution of the data. Since Theil-Sen
is a median-based estimator, it is more robust against corrupted data aka
outliers. In univariate setting, Theil-Sen has a breakdown point of about
29.3% in case of a simple linear regression which means that it can tolerate
arbitrary corrupted data of up to 29.3% [33].

Huber Regression: This makes sure that the loss function is not heavily
influenced by the outliers while not completely ignoring their effect[34].

Kernel ridge regression (KRR): Kernel ridge regression combines ridge
regression (linear least squares with l2-norm regularization) with the kernel
trick. It thus learns a linear function in the space induced by the respective
kernel and the data. For non-linear kernels, this corresponds to a non-linear
function in the original space [35].

Support Vector Regression (SVR): The model produced by support vector
classification (as described above) depends only on a subset of the training
data, because the cost function for building the model does not care about
training points that lie beyond the margin. Analogously, the model produced by
Support Vector Regression depends only on a subset of the training data,
because the cost function ignores samples whose prediction is close to their
target [36].

Decision Tree Regressor (DTR): The goal is to create a model that predicts the
value of a target variable by learning simple decision rules inferred from the
data features [37].

Random Forest Regressora: Random forest is a meta estimator that fits a number
of classifying decision trees on various sub-samples of the dataset and uses
averaging to improve the predictive accuracy and control over-fitting[38].

Extra-Trees Regressor: This class implements a meta estimator that fits a
number of randomized decision trees various sub-samples of the dataset and
uses averaging to improve the predictive accuracy and control over-
fitting[39].

Nearest Neighbors Regression: Neighbors-based regression can be used in cases
where the data labels are continuous rather than discrete variables. The label
assigned to a query point is computed based on the mean of the labels of its
nearest neighbors [40].

Multi-layer Perceptron regressor(MLPR): MLPRegressor trains iteratively since
at each time step the partial derivatives of the loss function with respect to
the model parameters are computed to update the parameters. It can also have a
regularization term added to the loss function that shrinks model parameters
to prevent overfitting. This implementation works with data represented as
dense and sparse numpy arrays of floating point values [41].

###  II.3  Dimension reduction process

Drug design datasets generally have a very large number of features. In our
study, the original datasets and their dimensionally reduced versions are
used. By doing so, the effects of the feature selection process on the
accuracies of the algorithms are investigated. The accuracies over the
original and dimensionally reduced datasets are compared. The Random Forest
Importances method is used for feature selection [42].

###  II.4  Dataset collection

Our drug data collection consists of 4 drug datasets obtained from several
studies. The datasets are shown in Table 1. The datasets with 2075 descriptors
were formed using the Dragon [43]. The molecules and outputs were obtained
from the original studies.

Table 1:  The 4 original datasets used in study  Dataset ID | Dataset name | Number of samples | Original number of descriptors | Number of selected features | Reference
---|---|---|---|---|---
1 | polymer_133 | 133 | 836 | 10 | [44]
2 | alkaloid_53 | 53 | 2221 | 10 | [45]
3 | alkaloid_103 | 103 | 355 | 10 | [46]
4 | Polymer_150 | 150 | 474 | 10 | [47]

##  III Results and Discussion

Nineteen base regressors were used together with each ensemble algorithm on 4
regression-type drug design problems. Before simulation we have posed the same
questions as in the article [24], since these are the most important
characteristics of ensemble methods:

  * •

Do the algorithm ensembles generate more successful results than a single
algorithm?

  * •

What is the most successful ensemble algorithm?

  * •

What is the base algorithm-ensemble pair with the best results?

  * •

Which algorithm performs well with the ensembles?

  * •

What is the most successful single algorithm?

  * •

How are the algorithms and datasets grouped according to their performances?

  * •

How does the dimension reduction process affect the results?

To answer these questions, 57 algorithms ((2 ensemble + 1 single) x (19 base
algorithms) = 57) were employed on the 4 drug design datasets described in
Table 1 and their dimensionally reduced versions. A cross validation was used
and the RMSE results were averaged.

The RMSE is defined as:

$RMSE_{alg10.name}=\sqrt{\frac{1}{N}\displaystyle\sum_{i-1}^{N}y_{alg10.name}^{i}-y_{actual}^{i}}$

where $y_{alg10.name}^{i}$ is the prediction of alg.name for the ith test
sample, $y_{actual}^{i}$ is the actual output value of the ith test sample,
and N is the number of test samples.

Our base and ensemble algorithms have some hyperparameters to optimize. We
used the default hyperparameters.

In the cross-validation methodology, the dataset is randomly divided after
shuffling into 2 halves. One half is used in the training and the other is
used in the testing. This validation is repeated 5 times. In the results of
this validation, 5 estimates of testing the RMSE were obtained for each
algorithm and each dataset. In some experiments, very high RMSE results were
obtained, especially with the simple linear regression algorithm disturbing
the overall averages. Because of this, the performance comparisons of the
algorithms were done with the algorithms’ success ranking instead of the
averaged RMSEs. In each experiment, the averaged cross-validation RMSEs were
sorted in ascending order. The algorithm with the lowest RMSE got the 1st
ranking. The worst got the 57th ranking. These success rankings are given in
Tables 2 and 3. In Table 2. the results with the original datasets are shown.
In Table 3, the results with the dimensionally reduced datasets are shown. The
4 datasets are ordered along the columns of the tables. The algorithms are
ordered along the rows of the tables. The average success rate and standard
deviation of each algorithm are shown in the last 2 columns.

In Tables 4 and 5, the summaries of Tables 2 and 3 are given, respectively.
Each cell is the averaged success ranking of the experiments with the base
algorithm in the cell’s row and the ensemble algorithm in the cell’s column.
The average success rankings of the single algorithms used are given in the
‘Single’ column. In the Avg. column, the averaged success rankings of the
experiments with respect to the base algorithms are given. In the ‘Avg.’ row,
the averaged success rankings of the experiments with respect to the ensemble
algorithms are given.

When Tables 2, 3,4,5 are investigated, the following conclusions are reached.
For the experiments with the original datasets (Tables 2 and 4): –The best
ranking performance (5.75) is obtained with the Extra Trees Regressor
algorithm.
–The best performed ensemble algorithms are additive regression (AR).
–The best performed base algorithm is Support Vector Machine.
–All of the ensemble algorithms generally increased the performance of each
base algorithm. The exceptions are Bayesian Ridge, Support Vector Machine and
K Neighbors Regressor.
–The Decision Tree and Support Vector Machine base algorithms had their best
performances with BG. The Decision Tree and Automatic Relevance Determination
algorithms with AR, achieved their best performances.

Table 2:  The success ranking of 57 algorithms on 4 original datasets (see.Table 1) (best to worst, 0 to 56). No. | Algorithm | Dataset’s ID
---|---|---
|  | 1 | 2 | 3 | 4 | Avg.
0 | Lasso Regression | 25 | 2 | 25 | 44 | 24
1 | BG-Lasso Regression | 18 | 6 | 18 | 43 | 21,25
2 | AR-Lasso Regression | 31 | 4 | 31 | 40 | 26,5
3 | Ridge Regression | 52 | 55 | 52 | 27 | 46,5
4 | BG-Ridge Regression | 47 | 39 | 47 | 28 | 40,25
5 | AR-Ridge Regression | 43 | 29 | 43 | 32 | 36,75
6 | Elastic Net | 28 | 1 | 28 | 37 | 23,5
7 | BG-Elastic Net | 21 | 3 | 21 | 38 | 20,75
8 | AR-Elastic Net | 29 | 0 | 29 | 36 | 23,5
9 | Lasso Least Angle Regression | 34 | 44 | 34 | 45 | 39,25
10 | BG-Lasso Least Angle Regression | 32 | 43 | 32 | 46 | 38,25
11 | AR-Lasso Least Angle Regression | 36 | 42 | 36 | 47 | 40,25
12 | Orthogonal Matching Pursuit | 48 | 48 | 48 | 0 | 36
13 | BG-Orthogonal Matching Pursuit | 30 | 24 | 30 | 6 | 22,5
14 | AR-Orthogonal Matching Pursuit | 24 | 19 | 24 | 1 | 17
15 | Bayesian Ridge | 26 | 5 | 26 | 29 | 21,5
16 | BG-Bayesian Ridge | 49 | 35 | 49 | 23 | 39
17 | AR-Bayesian Ridge | 41 | 32 | 41 | 30 | 36
18 | Automatic Relevance Determination | 45 | 50 | 45 | 4 | 36
19 | BG-Automatic Relevance Determination | 38 | 17 | 38 | 3 | 24
20 | AR-Automatic Relevance Determination | 20 | 13 | 20 | 5 | 14,5
21 | Passive Aggressive Regressor | 37 | 40 | 37 | 51 | 41,25
22 | BG-Passive Aggressive Regressor | 35 | 30 | 35 | 52 | 38
23 | AR-Passive Aggressive Regressor | 40 | 14 | 40 | 53 | 36,75
24 | TheilSen Regressor | 55 | 56 | 55 | 2 | 42
25 | BG-TheilSen Regressor | 53 | 53 | 53 | 12 | 42,75
26 | AR-TheilSen Regressor | 44 | 34 | 44 | 24 | 36,5
27 | Huber Regressor | 23 | 52 | 23 | 48 | 36,5
28 | BG-Huber Regressor | 27 | 49 | 27 | 49 | 38
29 | AR-Huber Regressor | 33 | 46 | 33 | 50 | 40,5
30 | Kernel Ridge | 51 | 54 | 51 | 25 | 45,25
31 | BG-Kernel Ridge | 46 | 47 | 46 | 26 | 41,25
32 | AR-Kernel Ridge | 42 | 31 | 42 | 31 | 36,5
33 | Support Vector Machine | 4 | 16 | 4 | 41 | 16,25
34 | BG-Support Vector Machine | 1 | 23 | 1 | 39 | 16
35 | AR-Support Vector Machine | 14 | 26 | 14 | 42 | 24
36 | K Neighbors Regressor | 19 | 41 | 19 | 33 | 28
37 | BG-K Neighbors Regressor | 16 | 37 | 16 | 34 | 25,75
38 | AR-K Neighbors Regressor | 22 | 38 | 22 | 35 | 29,25
39 | Decision Tree | 39 | 8 | 39 | 19 | 26,25
40 | BG-Decision Tree | 15 | 10 | 15 | 20 | 15
41 | AR-Decision Tree | 0 | 18 | 0 | 15 | 8,25
42 | Random Forest | 12 | 12 | 12 | 18 | 13,5
43 | BG-Random Forest | 8 | 15 | 8 | 21 | 13
44 | AR-Random Forest | 13 | 28 | 13 | 11 | 16,25
45 | Extra Trees Regressor | 3 | 7 | 3 | 10 | 5,75
46 | BG-Extra Trees Regressor | 6 | 9 | 6 | 16 | 9,25
47 | AR-Extra Trees Regressor | 11 | 20 | 11 | 14 | 14
48 | AdaBoost Regressor | 9 | 25 | 9 | 17 | 15
49 | BG-AdaBoost Regressor | 2 | 21 | 2 | 22 | 11,75
50 | AR-AdaBoost Regressor | 5 | 33 | 5 | 13 | 14
51 | Gradient Boosting Regressor | 17 | 22 | 17 | 8 | 16
52 | BG-Gradient Boosting Regressor | 10 | 11 | 10 | 9 | 10
53 | AR-Gradient Boosting Regressor | 7 | 27 | 7 | 7 | 12
54 | Multi Level Perceptron | 56 | 51 | 56 | 56 | 54,75
55 | BG-Multi Level Perceptron | 50 | 36 | 50 | 55 | 47,75
56 | AR-Multi Level Perceptron | 54 | 45 | 54 | 54 | 51,75
Table 3:  The success ranking of 57 algorithms on the 4 dimensionally reduced datasets (best to worst, 0 to 56). No. | Algorithm | Dataset’s ID
---|---|---
|  | 1 | 2 | 3 | 4 | Avg.
0 | Lasso Regression | 41 | 36 | 41 | 46 | 41
1 | BG-Lasso Regression | 38 | 32 | 38 | 48 | 39
2 | AR-Lasso Regression | 37 | 31 | 37 | 45 | 37,5
3 | Ridge Regression | 9 | 7 | 9 | 26 | 12,75
4 | BG-Ridge Regression | 12 | 3 | 12 | 23 | 12,5
5 | AR-Ridge Regression | 19 | 15 | 19 | 17 | 17,5
6 | Elastic Net | 33 | 34 | 33 | 39 | 34,75
7 | BG-Elastic Net | 31 | 29 | 31 | 40 | 32,75
8 | AR-Elastic Net | 34 | 28 | 34 | 38 | 33,5
9 | Lasso Least Angle Regression | 42 | 35 | 42 | 47 | 41,5
10 | BG-Lasso Least Angle Regression | 39 | 33 | 39 | 49 | 40
11 | AR-Lasso Least Angle Regression | 45 | 30 | 45 | 50 | 42,5
12 | Orthogonal Matching Pursuit | 25 | 26 | 25 | 37 | 28,25
13 | BG-Orthogonal Matching Pursuit | 44 | 12 | 44 | 36 | 34
14 | AR-Orthogonal Matching Pursuit | 43 | 5 | 43 | 35 | 31,5
15 | Bayesian Ridge | 20 | 21 | 20 | 9 | 17,5
16 | BG-Bayesian Ridge | 35 | 27 | 35 | 18 | 28,75
17 | AR-Bayesian Ridge | 30 | 53 | 30 | 5 | 29,5
18 | Automatic Relevance Determination | 28 | 45 | 28 | 8 | 27,25
19 | BG-Automatic Relevance Determination | 47 | 43 | 47 | 11 | 37
20 | AR-Automatic Relevance Determination | 48 | 52 | 48 | 4 | 38
21 | Passive Aggressive Regressor | 55 | 56 | 55 | 53 | 54,75
22 | BG-Passive Aggressive Regressor | 27 | 41 | 27 | 52 | 36,75
23 | AR-Passive Aggressive Regressor | 29 | 54 | 29 | 51 | 40,75
24 | TheilSen Regressor | 52 | 37 | 52 | 6 | 36,75
25 | BG-TheilSen Regressor | 53 | 40 | 53 | 3 | 37,25
26 | AR-TheilSen Regressor | 54 | 44 | 54 | 7 | 39,75
27 | Huber Regressor | 40 | 55 | 40 | 25 | 40
28 | BG-Huber Regressor | 49 | 46 | 49 | 16 | 40
29 | AR-Huber Regressor | 23 | 50 | 23 | 31 | 31,75
30 | Kernel Ridge | 11 | 49 | 11 | 27 | 24,5
31 | BG-Kernel Ridge | 17 | 48 | 17 | 22 | 26
32 | AR-Kernel Ridge | 16 | 51 | 16 | 29 | 28
33 | Support Vector Machine | 15 | 8 | 15 | 43 | 20,25
34 | BG-Support Vector Machine | 14 | 4 | 14 | 41 | 18,25
35 | AR-Support Vector Machine | 7 | 10 | 7 | 42 | 16,5
36 | K Neighbors Regressor | 36 | 13 | 36 | 33 | 29,5
37 | BG-K Neighbors Regressor | 32 | 11 | 32 | 32 | 26,75
38 | AR-K Neighbors Regressor | 50 | 23 | 50 | 44 | 41,75
39 | Decision Tree | 51 | 38 | 51 | 21 | 40,25
40 | BG-Decision Tree | 46 | 24 | 46 | 24 | 35
41 | AR-Decision Tree | 22 | 17 | 22 | 12 | 18,25
42 | Random Forest | 24 | 18 | 24 | 10 | 19
43 | BG-Random Forest | 26 | 6 | 26 | 15 | 18,25
44 | AR-Random Forest | 8 | 9 | 8 | 20 | 11,25
45 | Extra Trees Regressor | 0 | 2 | 0 | 0 | 0,5
46 | BG-Extra Trees Regressor | 2 | 0 | 2 | 1 | 1,25
47 | AR-Extra Trees Regressor | 1 | 1 | 1 | 2 | 1,25
48 | AdaBoost Regressor | 13 | 22 | 13 | 34 | 20,5
49 | BG-AdaBoost Regressor | 10 | 14 | 10 | 28 | 15,5
50 | AR-AdaBoost Regressor | 4 | 16 | 4 | 30 | 13,5
51 | Gradient Boosting Regressor | 3 | 25 | 3 | 19 | 12,5
52 | BG-Gradient Boosting Regressor | 18 | 20 | 18 | 13 | 17,25
53 | AR-Gradient Boosting Regressor | 6 | 19 | 6 | 14 | 11,25
54 | Multi Level Perceptron | 56 | 47 | 56 | 56 | 53,75
55 | BG-Multi Level Perceptron | 21 | 42 | 21 | 55 | 34,75
56 | AR-Multi Level Perceptron | 5 | 39 | 5 | 54 | 25,75
Table 4:  The averaged success rankings of the algorithms on the original datasets (best to worst, 0 to 56). Algorithm | BG | AG | Single | Avg.
---|---|---|---|---
Lasso Regression | 21,25 | 26,5 | 24 | 23,92
Ridge Regression | 40,25 | 36,75 | 46,5 | 41,17
Elastic Net | 20,75 | 23,5 | 23,5 | 22,58
Lasso Least Angle Regression | 38,25 | 40,25 | 39,25 | 39,25
Orthogonal Matching Pursuit | 22,5 | 17 | 36 | 25,17
Bayesian Ridge | 39 | 36 | 21,5 | 32,17
Automatic Relevance Determination | 24 | 14,5 | 36 | 24,83
Passive Aggressive Regressor | 38 | 36,75 | 41,25 | 38,67
TheilSen Regressor | 42,75 | 36,5 | 42 | 40,42
Huber Regressor | 38 | 40,5 | 36,5 | 38,33
Kernel Ridge | 41,25 | 36,5 | 45,25 | 41,00
Support Vector Machine | 16 | 24 | 16,25 | 18,75
K Neighbors Regressor | 25,75 | 29,25 | 28 | 27,67
Decision Tree | 15 | 8,25 | 26,25 | 16,50
Random Forest | 13 | 16,25 | 13,5 | 14,25
Extra Trees Regressor | 9,25 | 14 | 5,75 | 9,67
AdaBoost Regressor | 11,75 | 14 | 15 | 13,58
Gradient Boosting Regressor | 10 | 12 | 16 | 12,67
Multi Level Perceptron | 47,75 | 51,75 | 54,75 | 51,42
Avg. | 27,08 | 27,07 | 29,86 |
Table 5:  The averaged success rankings of the algorithms on the dimensionally reduced datasets (best to worst, 0 to 56). Algorithm | BG | AG | Single | Avg.
---|---|---|---|---
Lasso Regression | 39 | 37,5 | 41 | 39,17
Ridge Regression | 12,5 | 17,5 | 12,75 | 14,25
Elastic Net | 32,75 | 33,5 | 34,75 | 33,67
Lasso Least Angle Regression | 40 | 42,5 | 41,5 | 41,33
Orthogonal Matching Pursuit | 34 | 31,5 | 28,25 | 31,25
Bayesian Ridge | 28,75 | 29,5 | 17,5 | 25,25
Automatic Relevance Determination | 37 | 38 | 27,25 | 34,08
Passive Aggressive Regressor | 36,75 | 40,75 | 54,75 | 44,08
TheilSen Regressor | 37,25 | 39,75 | 36,75 | 37,92
Huber Regressor | 40 | 31,75 | 40 | 37,25
Kernel Ridge | 26 | 28 | 24,5 | 26,17
Support Vector Machine | 18,25 | 16,5 | 20,25 | 18,33
K Neighbors Regressor | 26,75 | 41,75 | 29,5 | 32,67
Decision Tree | 35 | 18,25 | 40,25 | 31,17
Random Forest | 18,25 | 11,25 | 19 | 16,17
Extra Trees Regressor | 1,25 | 1,25 | 0,5 | 1,00
AdaBoost Regressor | 15,5 | 13,5 | 20,5 | 16,50
Gradient Boosting Regressor | 17,25 | 11,25 | 12,5 | 13,67
Multi Level Perceptron | 34,75 | 25,75 | 53,75 | 38,08
Avg. | 27,95 | 26,83 | 29,22 |

For the experiments with the dimensionally reduced datasets (Tables 3 and 5):
–The best ranking performance (0.5) is obtained with the Extra Trees Regressor
algorithm.
–The best performed ensemble algorithms are additive regression (AR) and
bagging (BG).
–The best performed base algorithm is Ridge Regression.
–All of the ensemble algorithms generally increased the performance of each
base algorithm. The exceptions are Orthogonal Matching Pursuit, Bayesian
Ridge, Automatic Relevance Determination, TheilSen Regressor and Kernel Ridge.
–The Ridge Regression and Support Vector Machine base algorithms had their
best performances with BG. The Ridge Regression, Support Vector Machine and
Decision Tree algorithms with AR, achieved their best performances.

The average successes of the algorithms were investigated above. Next, the
best performing algorithm will be investigated over each individual dataset.
In Table 6, the dataset name, and the error and the name of the best
performing algorithm are shown for the original and dimensionally reduced
datasets.

Table 6:  The averaged success rankings of the algorithms on the dimensionally reduced datasets (best to worst, 0 to 56). | With all of the features | With the selected features
---|---|---
Dataset name | Best performing algorithm | RMSE | Best performing algorithm | RMSE
polymer_133 | AR-Automatic Relevance Determination | 0,01 | BG-Gradient Boosting Regressor | 0,01
alkaloid_53 | AR-Elastic Net | 0,29 | BG-Extra Trees Regressor | 0,31
alkaloid_103 | AR-Decision Tree | 0,57 | Extra Trees Regressor | 0,65
Polymer_150 | Orthogonal Matching Pursuit | 0,00 | Orthogonal Matching Pursuit | 0,02

When Table 6 is investigated, the following conclusions are reached:
–The best performing algorithms are generally ensemble algorithms. This is in
agreement with the average success of the algorithms.
–experiments with dimensional reduced data sets do not have better results
than the original data sets, except for 1 data set (polymer_133).

When the algorithms are clustered, the algorithms are represented by points
having 4 (the number of datasets) features (dimensions). When the datasets are
clustered, the datasets are represented by points having 57 (the number of
algorithms) features (dimensions).

According to Figure 1, the following conclusions are reached:
–In both figures, the ensemble-algorithm pairs are generally clustered with
their base single algorithms.
–The feature selection process does not affect the similarities of the
algorithms dramatically.

According to Figure 2, the following conclusions are reached:
–On the left side of Figure 2, there is no obvious pattern between the
clusters and the number of features/samples.
–To the right of Figure 2, the polymers and alkaloids are clustered
separately.

Figure 1: The hierarchical clusters of the algorithms according to their RMSE
values on the original (left) and dimensionally reduced (right) 4 datasets.
Figure 2: The hierarchical clusters of the original (left) and dimensionally
reduced (right) 4 datasets according to their RMSE values obtained with 57
algorithms. In the figures, the dataset names, the number of features, and the
samples are given.

##  IV Previous works

The selected previous studies in this area for both classification and
regression are shown comparatively in Table 7.

According to Table 7, together with our experiments, the following conclusions
are reached:
–The number of drug design / chemical data sets used in our experiments is
greater than in previous studies, except for [24].
–The number of base machine learning methods used in our experiments is
greater than in previous studies.
–The superior success of ensemble algorithms over single algorithms is
confirmed.

Table 7:  Previous works. Reference | Compared methods in the study | Datasets | Results
---|---|---|---
[48] | ctree, rtree, cforest, rforest, gbm, fnn, | 1 regression-type | RandomForest showed
| earth, glmnet, ridge, lm, pcr, plsr, rsm, | (chemical data). | good results.
| rvm, ksvm, ksvmfp, nnet, nneth2o |  |
[24] | (4 ensemble + 1 single) * (7 base) = 35 | 15 regression-type | Ensemble methods showed
|  | (chemical data). | good results.
[49] | AdaBoostM1+Bagging (Ada_Bag), |  | Bagging (Ada_Bag)
| AdaBoostM1+Jrip (Ada_Jrip), |  | and Random Forest
| AdaBoostM1+J48 (Ada_J48), |  | (Ada_RF) algorithms
| AdaBoostM1+PART (Ada_PART), |  | showed good
| AdaBoostM1+RandomForest (Ada_RF), | MDDR database. | results.
| and |  |
| AdaBoostM1+REPTree (Ada_RT). |  |
[50] | EnsemDT, EnsemKRR and | 2 classification datasets | EnsemDT and EnsemKRR than
| other single methods | (chemical data) | showed better results
|  |  | other single methods.

##  V Conclusion

In machine learning, committee algorithms (ensembles), especially those with
classification applications, are highly popular because they have better
performances than single algorithms.

In this study, the comparative performances of algorithm ensembles with drug
design datasets in regression applications were investigated. A drug design
dataset collection with 4 regression-type datasets was used for this purpose.
We obtained the performances of the single algorithms and the algorithm
ensembles on those datasets. The combinations of 19 base algorithms and 2
ensemble algorithms were investigated.

###### Acknowledgements.

The authors would like to express their sincere gratitude to the Ministry of
innovative development of the Republic of Uzbekistan (Grant FA-tech-3018-4).
The authors are is also thankful to several colleagues in both laboratories,
who were engaged in many scientific discussions that helped the authors
formulate the best current practices modeling discussed herein.


N^{-\frac{d_{\overline{X}}}{2}}\varepsilon\|\widehat{\nu}\|_{L^{1}}+o_{f,\gamma_{0},\varepsilon}(N^{-\frac{d_{\overline{X}}}{2}})$

This implies

$\limsup_{N\to+\infty}\sup_{g,h\in\mathcal{D}_{N}(\delta_{0})}N^{\frac{d_{\overline{X}}}{2}}|\Theta^{N}_{p(g),p(h)}(f)-(D_{\sqrt{N}}\nu*NX)_{p(g),p(h)}(f)|\leq\varepsilon\|\widehat{\nu}\|_{L^{1}}$

As $\varepsilon$ is arbitrary, the limit must be zero, which is exactly
Theorem 6.1. ∎

To conclude the proof of Theorem 1.9, it is now sufficient to check that when
exactly one of the deviations is non moderate, the terms
$\Theta^{N}_{p(g),p(h)}(f)$ and $(D_{\sqrt{N}}\nu*N\chi)_{p(g),p(h)}(f)$ are
both negligible compared to $N^{-d_{\overline{X}}/2}$, and this holds
uniformly in $g,h$. This is the content of the next lemma.

###### Proof of Theorem 6.1.

Let $K\subseteq\mathfrak{g}$ be a compact set containing the support of $f$.
We may replace $f$ without loss of generality by $\mathbbm{1}_{K}$. For any
$g,h\in\widetilde{\mathfrak{g}}$, one has

$\displaystyle\Theta^{N}_{p(g),p(h)}(K)=\Upsilon^{N}_{g,h}(p^{-1}(K))=\Upsilon^{N}((-g)*p^{-1}(K)*(-h))$

Let $\delta_{0}>\delta_{1}>0$ and assume $p(g)\notin
p(\mathcal{D}_{N}(\delta_{0}))$, $h\in\mathcal{D}_{N}(\delta_{1})$. If
$\delta_{1}\lll\delta_{0}$, then for large $N$ (depending on $K$,
$\delta_{0}$), we have

$K*p(\mathcal{D}_{N}(\delta_{1}))*p(\mathcal{D}_{N}(\delta_{1}))\subseteq
p(\mathcal{D}_{N}(\delta_{0}))$

and this implies333Indeed, we would otherwise have $k^{\prime}*(-h)*x=g$ where
$k^{\prime}\in p^{-1}(K)$, $x\in\mathcal{D}_{N}(\delta_{1})$, hence
$K*p(\mathcal{D}_{N}(\delta_{1}))^{*2}\ni p(g)$, which contradicts our
assumption on $g$.

$(-g)*p^{-1}(K)*(-h)\subseteq\widetilde{\mathfrak{g}}\smallsetminus\mathcal{D}_{N}(\delta_{1})$

We deduce that for $N$ large, any $g,h$ as above,

$\Theta^{N}_{p(g),p(h)}(K)\leq\Upsilon^{N}(\widetilde{\mathfrak{g}}\smallsetminus\mathcal{D}_{N}(\delta_{1}))$

By the moment estimate Proposition 4.3, it follows that
$\Theta^{N}_{p(g),p(h)}(K)$ must be bounded by
$o_{K,\delta_{0},\delta_{1}}(N^{-\frac{d_{\overline{X}}}{2}})$. The other
cases are similar.

∎

##  7 Local limit theorem for ratios

In this section, we give an asymptotic for $(\mu^{*N}*\delta_{g_{n}})(f)$
where $g_{n}$ is a suitable recentering parameter and $f$ is a continuous
function with compact support.

We keep the notations
$(\mathfrak{g},*,\widetilde{\mathfrak{g}},\overline{X},d_{\overline{X}},(\mathfrak{m}^{(b)})_{b\leq
2s-1},X,\chi,*^{\prime})$ from Section 2. We assume that $\mu$ is an aperiodic
probability measure on $\mathfrak{g}$ with finite $(d_{\overline{X}}+2)$-th
moment for the weight filtration induced by $\overline{X}$, and expectation
$\mathbb{E}(\mu_{ab})=\overline{X}$. In particular $X=\mathbb{E}(\mu^{(1)})$.
We recall that $\nu=v(x)dx$ denotes the limit density in the central limit
theorem for the $\mu$-walk on $(\mathfrak{g},*)$ established in Theorem 5.2.

Our argument relies on the uniform local limit theorem established in Section
6 and a good control of the limit density $\nu$. Accordingly, all the
estimates of this section are still valid if we only assume $\mu$ to have a
finite moment of order strictly greater than $2$ and replace $\mu^{*N}$ by its
truncated approximation $\Theta^{N}$, see Section 6. We leave this
verification to the reader.

###  7.1 Comparison with Lebesgue

We prove Theorem 1.10. It follows from the following more general result.

######  Proposition 7.1 (Asymptotic with variable recentering).

For every $f\in C_{c}(\mathfrak{g})$, $C>0$ $N\geq 1$, $g,h\in
D_{\sqrt{N}}(B_{\widetilde{\mathfrak{g}}}(0,C))$ we have

$(\mu^{*N}*-NX)_{p(g),p(h)}(f)=v(x_{N,g,h})N^{-d_{\overline{X}}/2}\int_{\mathfrak{g}}fdx\,\,+\,\,o_{f,C}(N^{-d_{\overline{X}}/2})$

where $v$ is the density of the limit measure $\nu=v(x)dx$, and

$x_{N,g,h}=p\left(D_{\frac{1}{\sqrt{N}}}(g)\right)^{-1}*p\left(D_{\frac{1}{\sqrt{N}}}(h)^{-1}\right)$

for $p:\widetilde{\mathfrak{g}}\rightarrow\mathfrak{g}$ the projection
$x\oplus t\chi\mapsto x+tX$.

_Remark_. Recall from Section 3.3 that $v(0)$ may vanish. In particular, the
sequence $(\mu^{*N}*-NX)(f)$ may be $o_{f,C}(N^{-d_{\overline{X}}/2})$. This
never happens in the centered case (i.e. $\overline{X}=0$) for which $v>0$
(see 7.2).

###### Proof.

By Theorem 6.1,

$\displaystyle(\mu^{*N}*-NX)_{p(g),p(h)}(f)=(D_{\sqrt{N}}\nu)_{p(g),p(h)}(f)+o_{f,C}(N^{-d_{\overline{X}}/2})$

Let $\gamma_{0}<1/2$. Set $e_{\xi,N,g,h}=e^{-2i\pi\xi\circ
p(g^{(\chi)}+h^{(\chi)})}$ and $g_{N}=D_{\frac{1}{\sqrt{N}}}g$,
$h_{N}=D_{\frac{1}{\sqrt{N}}}h$. Applying Fourier inversion formula, 6.8 and
6.7, we obtain

$\displaystyle(D_{\sqrt{N}}\nu)_{p(g),p(h)}(f)$
$\displaystyle=\int_{\widehat{\mathfrak{g}}}\widehat{f}(\xi)\widehat{(D_{\sqrt{N}}\nu)_{p(g),p(h)}}(\xi)d\xi$
$\displaystyle=\int_{\widehat{\mathfrak{g}}}\widehat{f}(\xi)\widehat{(D_{\sqrt{N}}\nu)_{g,h}}(\xi)e_{\xi,N,g,h}d\xi$
$\displaystyle=\int_{\mathcal{U}_{N}(\gamma_{0})}\widehat{f}(\xi)\widehat{(D_{\sqrt{N}}\nu)_{g,h}}(\xi)e_{\xi,N,g,h}d\xi+\|f\|_{L^{1}}O_{\gamma_{0}}(N^{-\frac{d_{\overline{X}}+1}{2}})$
$\displaystyle=\int_{\mathcal{U}_{N}(\gamma_{0})}\widehat{f}(\xi)\widehat{(g*^{\prime}D_{\sqrt{N}}\nu*^{\prime}h)}(\xi)e_{\xi,N,g,h}d\xi+\|f\|_{L^{1}}O_{\gamma_{0}}(N^{-\frac{d_{\overline{X}}+1}{2}+O(\gamma_{0})})$
$\displaystyle=\int_{\mathcal{U}_{N}(\gamma_{0})}\widehat{f}(\xi)\widehat{(D_{\sqrt{N}}(g_{N}*^{\prime}\nu*^{\prime}h_{N}))}(\xi)e_{\xi,N,g,h}d\xi+\|f\|_{L^{1}}O_{\gamma_{0}}(N^{-\frac{d_{\overline{X}}+1}{2}+O(\gamma_{0})})$
$\displaystyle=\int_{\mathcal{U}_{N}(\gamma_{0})}\widehat{f}(\xi)\widehat{(D_{\sqrt{N}}(g_{N}*\nu*h_{N}))}(\xi)e_{\xi,N,g,h}d\xi+\|f\|_{L^{1}}O_{\gamma_{0}}(N^{-\frac{d_{\overline{X}}+1}{2}+O(\gamma_{0})})$
$\displaystyle=[D_{\sqrt{N}}(p(g_{N})*\nu*p(h_{N}))](f)+\|f\|_{L^{1}}O_{\gamma_{0}}(N^{-\frac{d_{\overline{X}}+1}{2}+O(\gamma_{0})})$

Pursuing the computation

$\displaystyle[D_{\sqrt{N}}(p(g_{N})*\nu*p(h_{N}))](f)$
$\displaystyle=\int_{\mathfrak{g}}f\left(D_{\sqrt{N}}(p(g_{N})*x*p(h_{N}))\right)v(x)dx$
$\displaystyle=N^{-d_{\overline{X}}/2}\int_{\mathfrak{g}}f(x)\,v\left(p(g_{N})^{-1}*D_{\frac{1}{\sqrt{N}}}x*p(h_{N})^{-1}\right)dx$
$\displaystyle=v\left(p(g_{N})^{-1}*p(h_{N})^{-1}\right)N^{-d_{\overline{X}}/2}\int_{\mathfrak{g}}f(x)dx+r$

where the difference $r$ is bounded by

$\displaystyle r\leq
N^{-d_{\overline{X}}/2}\int_{\mathfrak{g}}\big{|}v\left(p(g_{N})^{-1}*D_{\frac{1}{\sqrt{N}}}x*p(h_{N})^{-1}\right)-v\left(p(g_{N})^{-1}*p(h_{N})^{-1}\right)\big{|}\,\,|f(x)|dx$

whence is $o_{C}(N^{-d_{\overline{X}}/2})$ by dominated convergence. This
concludes the proof. ∎

###  7.2 Comparison with $\nu$

The goal of the section is to show Theorem 1.11. We assume that $\mu$ is
centered, aperiodic, with finite moment of order $m_{\mu}>d_{\mathfrak{g}}+2$
for the central filtration, and we show that the asymptotic
$\mu^{*N}(f)\sim(D_{\sqrt{N}}\nu)(f)$ implied by Theorem 1.10, actually holds
uniformly for a certain range of deviations, namely those in

$W_{N}(c):=\\{x\in\mathfrak{g},\,|x^{(i)}|\leq c\left(N\log N\right)^{i/2}\\}$

for $c>0$ small.

The assumption that $\mu$ is centered is used to guarantee that the limit
density $\nu=v(x)dx$ has a Gaussian lower bound. Indeed, we record the
following fact [VSCC92].

######  Fact 7.2.

Assume $\overline{X}=0$. Let $|.|_{\mathfrak{g}}$ denote the distance to $0$
for a left-invariant distance on $(\mathfrak{g},*)$. There exists $A>1$ such
that for all $x\in\mathfrak{g}$

$A^{-1}e^{-A|x|_{\mathfrak{g}}^{2}}\leq v(x)\leq
Ae^{-A^{-1}|x|_{\mathfrak{g}}^{2}}$

Using the lower bound, we show

######  Proposition 7.3.

Assume $\overline{X}=0$. Let $\delta>0$ and $c\lll_{\delta}1$. Let $f\in
C^{\infty}(\mathfrak{g})$ be a non-negative integrable function with compactly
supported Fourier transform, such that $f\neq 0$. For all $N\geq 1$ large
enough, $g,h\in W_{N}(c)$

$(D_{\sqrt{N}}\nu)_{g,h}(f)\geq N^{-\frac{d_{\mathfrak{g}}}{2}-\delta}$

Combining with Theorem 6.1, we get

######  Corollary 7.4.

Assume $\overline{X}=0$, $m_{\mu}>d_{\mathfrak{g}}+2$ and $c\lll 1$. Let $f\in
C^{\infty}(\mathfrak{g})$ be a non-negative integrable function with compactly
supported Fourier transform, such that $f\neq 0$. For $N\geq 1$, $g,h\in
W_{N}(c)$, we have

$\frac{\mu^{*N}_{g,h}(f)}{(D_{\sqrt{N}}\nu)_{g,h}(f)}=1+o_{f}(1)$

We will then deduce Theorem 1.11 by applying a refinement of the approximation
technique used in the proof of the uniform local limit theorem.

###### Proof of Proposition 7.3.

. We can write $(D_{\sqrt{N}}\nu)_{g,h}=N^{-d_{\mathfrak{g}}/2}w_{N,g,h}(x)dx$
where

$w_{N,g,h}(x)=v\circ
r_{N,g,h}(x)\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,r_{N,g,h}(x)=D_{\frac{1}{\sqrt{N}}}\left(g^{-1}*x*h^{-1}\right)$

We need to show that for $c\lll_{\delta}1$, $N\geq 1$ large enough (possibly
depending on $f$), $g,h\in W_{N}(c)$,

$\int_{\mathfrak{g}}f(x)w_{N,g,h}(x)dx\geq N^{-\delta}$

As $f$ is non-negative non-identically zero, it is enough to show that for
$c\lll_{\delta}1$, any compact subset of $\mathfrak{g}$ is included in the set
$\\{w_{N,g,h}\geq N^{-\frac{\delta}{2}}\\}$ for $N$ large enough and $g,h\in
W_{N}(c)$. To analyse this set, we use the lower bound in 7.2

Consider $x$ such that $w_{N,g,h}(x)<N^{-\frac{\delta}{2}}$. Using 7.2, we
infer

$\displaystyle
A^{-1}e^{-A|r_{N,g,h}(x)|_{\mathfrak{g}}^{2}}<N^{-\frac{\delta}{2}}$

which can be rewritten

$\frac{\delta}{2A}\log N-\frac{\log
A}{A}\leq|r_{N,g,h}(x)|_{\mathfrak{g}}^{2}$

so for $N\ggg_{A,\delta}1$,

$\sqrt{\frac{\delta\log N}{3A}}\leq|r_{N,g,h}(x)|_{\mathfrak{g}}$

This discussion justifies the inclusion

$\\{w_{N,g,h}\geq
N^{-\frac{\delta}{2}}\\}\supseteq\left\\{x\in\mathfrak{g}\,:\,|r_{N,g,h}(x)|_{\mathfrak{g}}<\sqrt{\frac{\delta\log
N}{3A}}\right\\}$

for any $N\ggg_{A,\delta}1$. However, by the ball-box principle,

$|r_{N,g,h}(x)|_{\mathfrak{g}}\ll\max_{i\leq
s}\|(r_{N,g,h}(x))^{(i)}\|^{1/i}=N^{-1/2}\,\max_{i\leq
s}\|(g^{-1}*x*h)^{(i)}\|^{1/i}$

so

$\displaystyle\\{w_{N,g,h}\geq N^{-\frac{\delta}{2}}\\}$
$\displaystyle\supseteq\left\\{x\in\mathfrak{g}\,:\,\forall i\leq
s,\,\|(g^{-1}*x*h^{-1})^{(i)}\|\lll_{A,\delta}(N\log N)^{i/2}\right\\}$

Noticing that for any $\delta>0$, we have
$W_{N}(\delta)*W_{N}(\delta)\subseteq W_{N}(O(\delta))$, we conclude that for
$c\lll_{A,\delta}1$, $N\ggg_{A,\delta}1$, $g,h\in W_{N}(c)$,

$\displaystyle\\{w_{N,g,h}\geq N^{-1/4}\\}$ $\displaystyle\supseteq W_{N}(c)$

As $A$ can be fixed as a function of the initial data, our conventions allow
to remove it from the subscript, see Section 2.11.

∎

We can now conclude the proof of Theorem 1.11.

###### Proof of Theorem 1.11.

Fix $\delta\in(0,1/2]$ such that $\mu$ has a finite moment of order
$d_{\mathfrak{g}}+2+2\delta$. We choose $\delta$ is a function of $\mu$, so it
does not appear in the subscripts of our Landau and Vinogradov notations (see
Section 2.11). Fix $c\lll 1$ as in Proposition 7.3 and Corollary 7.4. Let
$\varepsilon>0$ and fix $f^{-}\leq f\leq f^{+}$ an $\varepsilon$-approximation
of $f$ by functions with compactly supported Fourier transforms, as in
Proposition 6.9. As $f$ is non negative and non identically zero, we have for
$N\geq 1$, $g,h\in W_{N}(c)$

$\displaystyle\frac{\mu^{*N}_{g,h}(f)}{(D_{\sqrt{N}}\nu)_{g,h}(f)}$
$\displaystyle\geq\frac{\mu^{*N}_{g,h}(f^{-})}{(D_{\sqrt{N}}\nu)_{g,h}(f^{+})}$
$\displaystyle=\frac{(D_{\sqrt{N}}\nu)_{g,h}(f^{-})+o_{f^{-}}(N^{-\frac{d_{\mathfrak{g}}}{2}-\delta})}{(D_{\sqrt{N}}\nu)_{g,h}(f^{+})}$
$\displaystyle=\frac{(D_{\sqrt{N}}\nu)_{g,h}(f^{-})}{(D_{\sqrt{N}}\nu)_{g,h}(f^{+})}+o_{f^{\pm}}(1)$

where the second line uses the local limit theorem with power saving from
Theorem 6.1, and the last line uses Proposition 7.3.

Moreover,

$\displaystyle\frac{\mu^{*N}_{g,h}(f)}{(D_{\sqrt{N}}\nu)_{g,h}(f)}$
$\displaystyle\leq\frac{\mu^{*N}_{g,h}(f^{+})}{(D_{\sqrt{N}}\nu)_{g,h}(f)}$
$\displaystyle=\frac{\mu^{*N}_{g,h}(f^{+})}{(D_{\sqrt{N}}\nu)_{g,h}(f^{+})}\frac{(D_{\sqrt{N}}\nu)_{g,h}(f^{+})}{(D_{\sqrt{N}}\nu)_{g,h}(f)}$
$\displaystyle=(1+o_{f^{+}}(1))\frac{(D_{\sqrt{N}}\nu)_{g,h}(f^{+})}{(D_{\sqrt{N}}\nu)_{g,h}(f)}$

where the last line uses Corollary 7.4. To conclude, it is enough to prove
that

$\frac{(D_{\sqrt{N}}\nu)_{g,h}(f^{-})}{(D_{\sqrt{N}}\nu)_{g,h}(f^{+})}=1+O_{f}(\varepsilon)+o_{f^{\pm}}(1)$

Only the lower bound needs a proof. We can reformulate it as

$\displaystyle(D_{\sqrt{N}}\nu)_{g,h}(f^{+}-f^{-})\leq\left(O_{f}(\varepsilon)+o_{f^{\pm}}(1)\right)(D_{\sqrt{N}}\nu)_{g,h}(f^{+})$
(40)

To prove (40), we write as above
$(D_{\sqrt{N}}\nu)_{g,h}=N^{-d_{\mathfrak{g}}/2}w_{N,g,h}(x)dx$ where

$w_{N,g,h}(x)=u\circ
r_{N,g,h}(x)\,\,\,\,\,\,\,\,\,\,\,\,r_{N,g,h}(x)=D_{\frac{1}{\sqrt{N}}}(g^{-1}*x*h^{-1})$

We also introduce the domain

$\Omega_{N}:=\\{x\in\mathfrak{g}\,:\,\|x\|\leq N^{\frac{1}{4s}}\\}$

######  Lemma 7.5.

For $c\lll 1$, there exists $V>1$ such that for all $N\geq 1$, $g,h\in
W_{N}(c)$,

$\frac{\sup_{\Omega_{N}}w_{N,g,h}}{\inf_{\Omega_{N}}w_{N,g,h}}\leq V$

Let us see how the lemma allows to conclude. Write $\psi=f^{+}-f^{-}$.

$\displaystyle(D_{\sqrt{N}}\nu)_{g,h}(f^{+}-f^{-})$
$\displaystyle=N^{-d_{\mathfrak{g}}/2}\int_{\mathfrak{g}}\psi(x)w_{N,g,h}(x)dx$
$\displaystyle=\underbrace{N^{-d_{\mathfrak{g}}/2}\int_{\Omega_{N}}\psi(x)w_{N,g,h}(x)dx}_{T_{N}}\,+\,\underbrace{N^{-d_{\mathfrak{g}}/2}\int_{\mathfrak{g}\smallsetminus\Omega_{N}}\psi(x)w_{N,g,h}(x)dx}_{S_{N}}$

Asking that $c$ also satisfies 7.5, and taking $N$ large enough so that
$\int_{\Omega_{N}}f\geq\frac{\|f\|_{L^{1}}}{2}$, we can control the first
term:

$\displaystyle T_{N}$ $\displaystyle\leq
N^{-d_{\mathfrak{g}}/2}\int_{\Omega_{N}}\psi(x)\sup_{\Omega_{N}}w_{N,g,h}\ dx$
$\displaystyle\leq
N^{-d_{\mathfrak{g}}/2}\|\psi\|_{L^{1}}V\inf_{\Omega_{N}}w_{N,g,h}$
$\displaystyle\leq
N^{-d_{\mathfrak{g}}/2}\|\psi\|_{L^{1}}V\frac{2}{\|f\|_{L^{1}}}\int_{\Omega_{N}}f(x)w_{N,g,h}(x)dx$
$\displaystyle\leq\|\psi\|_{L^{1}}V\frac{2}{\|f\|_{L^{1}}}(D_{\sqrt{N}}\nu)_{g,h}(f^{+})$

As $\|\psi\|_{L^{1}}\leq\varepsilon$, we finally have for $N\geq 1$, $g,h\in
W_{N}(c)$,

$\displaystyle T_{N}=O_{f}(\varepsilon)\,(D_{\sqrt{N}}\nu)_{g,h}(f^{+})$

Let us now deal with $S_{N}$. Using that $w_{N,g,h}$ is bounded by $\sup v$,
and that $f^{+},f^{-}$ are rapidly decreasing, we obtain for every $Q>0$

$\displaystyle S_{N}$ $\displaystyle\ll
N^{-d_{\mathfrak{g}}/2}\int_{\mathfrak{g}\smallsetminus\Omega_{N}}\psi(x)dx$
$\displaystyle=O_{f^{+},f^{-},Q}(N^{-Q})$

Choosing $Q=\frac{d_{\mathfrak{g}}}{2}+1$ and using the lower bound
$(D_{\sqrt{N}}\nu)_{g,h}(f^{+})\geq N^{-\frac{d_{\mathfrak{g}}}{2}-\delta}$
from Proposition 7.3 and our choice for $c$, we deduce that

$\displaystyle S_{N}=o_{f^{\pm}}(1)(D_{\sqrt{N}}\nu)_{g,h}(f^{+})$

This concludes the proof of Theorem 1.11, up to showing 7.5 that we now take
care of. ∎

###### Proof of 7.5.

We need to show that for $c\lll 1$, the maximum variation

$\sup_{x,y\in\Omega_{N}}\big{|}\log(w_{N,g,h}(x))-\log(w_{N,g,h}(y))\big{|}$

is bounded by some constant $V^{\prime}>0$ independent of $N\geq 1$, $g,h\in
W_{N}(c)$. Enumerate as $e_{1},\dots,e_{d}$ the basis $(e^{(i)}_{j})$ of
$\mathfrak{g}$ fixed in Section 2 and write $\partial_{i}$ for the derivation
in the variable $e_{i}$ (in the classical sense). Fixing
$i\in\\{1,\dots,d\\}$, it is sufficient to show that for $c\lll 1$,
$x\in\mathfrak{g}$

$|(\partial_{i}\log w_{N,g,h})(x)|\leq|P_{N,g,h}(x)|$

where $P_{N,g,h}$ is a polynomial on $\mathfrak{g}$ of degree at most $s$, and
whose coefficients are $O(\frac{(c\log N)^{O(s)}}{N^{1/3}})$ uniformly in
$N\geq 2$, $g,h\in W_{N}(c)$.

Let us analyse this derivative. We can rewrite

$(\partial_{i}\log
w_{N,g,h})(x)=(\partial_{i}r_{N,g,h})(x)\,\frac{\partial_{i}v}{v}(r_{N,g,h}(x))$

We study each term in the product.

For the first term $\partial_{i}r_{N,g,h}$, recall that
$r_{N,g,h}(x)=D_{\frac{1}{\sqrt{N}}}(g^{-1}*x*h^{-1})$. As in the proof of
Theorem 6.1 (Section 6.3), we can write $r_{N,g,h}$ as a sum

$r_{N,g,h}(x)=Q_{N}(D_{\frac{1}{\sqrt{N}}}g,D_{\frac{1}{\sqrt{N}}}x,D_{\frac{1}{\sqrt{N}}}h)$

where $Q_{N}$ is a polynomial on $\mathfrak{g}^{3}$ of degree at most $s$,
only444$Q_{N}$ also depends on the initial data $\mathfrak{g}$, $\mu$, etc.
fixed once and for all in Section 2. depending on $N$, and with coefficients
$O(1)$.

As $\|D_{\frac{1}{\sqrt{N}}}g\|,\|D_{\frac{1}{\sqrt{N}}}h\|=O(c(\log
N)^{s/2})$, and as any term in $Q_{N}$ with non-zero $\partial_{i}$-derivative
in $x$ must involve $D_{\frac{1}{\sqrt{N}}}x$, we infer that
$\partial_{i}r_{N,g,h}(x)$ is a polynomial whose coefficients are
$O(\frac{(c\log N)^{O(s)}}{\sqrt{N}})$ uniformly in $N\geq 2$, $g,h\in
W_{N}(c)$.

For the second term $\frac{\partial_{i}v}{v}(r_{N,g,h}(x))$, we bound the
numerator by a constant and use the lower bound on $v$ from 7.2 to get

$|\frac{\partial_{i}v}{v}(r_{N,g,h}(x))|\ll
Ae^{A|r_{N,g,h}(x)|_{\mathfrak{g}}^{2}}$

However, we have for all $N\geq 1$, $x\in\Omega_{N}$, $g,h\in W_{N}(c)$ that

$|r_{N,g,h}(x)|_{\mathfrak{g}}\leq O(c)\sqrt{\log N}$

It follows that

$|\frac{\partial_{i}v}{v}(r_{N,g,h}(x))|\ll AN^{AO(c)^{2}}\ll N^{\frac{1}{6}}$

up to taking $c\lll_{A}1$. As $A$ can be fixed as a function of the initial
data, our conventions allow to remove it from the subscript, see Section 2.11.

The result follows by combining the discussions for each term.

∎

##  8 Consequences for Choquet-Deny and equidistribution.

###  8.1 Bounded harmonic functions on nilpotent groups

In this paragraph, we prove Corollary 1.12, namely the Choquet-Deny property.
While doing so, we also establish the proposition below, which is an important
consequence of our truncated uniform local limit theorem 6.1, and will be
crucial in the next section.

######  Proposition 8.1.

Let $(\mathfrak{g},*)$ be a simply connected nilpotent Lie group. Let $\mu$ be
an aperiodic probability measure on $\mathfrak{g}$ with a finite moment of
order strictly greater than $2$ for the weight filtration induced by
$\overline{X}_{\mu}=\mathbb{E}(\mu_{ab})$. Let $\nu$ be the limit measure
arising in the central limit theorem 5.2 for some choice of weight
decomposition $\mathfrak{g}=\oplus\mathfrak{m}^{(b)}$. Fix a right invariant
distance on $(\mathfrak{g},*)$ and let

$\mathscr{L}=\\{F:\mathfrak{g}\rightarrow\mathbb{R}\,\,|\,\,F\,\,1\emph{-lipschitz},\,\|F\|_{\infty}\leq
1\\}$

Then

$\sup_{F\in\mathscr{L}}\,|\mu^{*N}(F)-(D_{\sqrt{N}}\nu*NX_{\mu})(F)|\underset{N\to+\infty}{\longrightarrow}0$

_Remarks_. 1) Note that $F$ may not decay at infinity. The idea of the proof
is to pixelize $F$, and then apply the uniform local limit theorem to each
pixel.

2) If we replace the right-invariant metric by a left invariant metric, then
the statement still holds, following the same line of argument. We will need
the metric to be right-invariant below in order to regularize bounded harmonic
functions for the right-random walk using convolution on the left.

###### Proof.

We fix $0<\gamma_{0}\lll 1$ so that the truncation $\Theta^{N}$ defined in
Section 6 satisfies the uniform local limit theorem 6.1. As $\mu$ has in
particular finite second moment, 4.2 yields that

$\|\mu^{*N}-\Theta^{N}\|\to 0$

in total variation. Hence, we may replace $\mu^{*N}$ by $\Theta^{N}$ in order
to prove Proposition 8.1.

Let $\varepsilon>0$. We fix a “bump function”
$\chi_{\varepsilon}:\mathfrak{g}\rightarrow\mathbb{R}^{+}$, i.e. a continuous
function supported on the ball $B_{\mathfrak{g}}(0,\varepsilon)$ with total
sum $1$. We introduce the convolution $F_{\varepsilon}:=\chi_{\varepsilon}*F$,
given by

$F_{\varepsilon}(x)=\int_{\mathfrak{g}}\chi_{\varepsilon}(u^{-1})F(ux)du=\int_{\mathfrak{g}}\chi_{\varepsilon}(xu^{-1})F(u)du$

where, here and below, we write $xy=x*y$ and $x^{-1}=-x$ to lighten notations.
As $F$ is $1$-Lispchitz we have
$\|F-F_{\varepsilon}\|_{\infty}\leq\varepsilon$, so for all $N\geq 1$,

$|\Theta^{N}(F)-(D_{\sqrt{N}}\nu*NX_{\mu})(F)|\leq|\Theta^{N}(F_{\varepsilon})-(D_{\sqrt{N}}\nu*NX_{\mu})(F_{\varepsilon})|+2\varepsilon$
(41)

On the other hand

$\displaystyle\Theta^{N}(F_{\varepsilon})$
$\displaystyle=\int_{\mathfrak{g}}(\Theta^{N}*\delta_{u^{-1}})(\chi_{\varepsilon})\,F(u)\,du$
(42)

The uniform local limit theorem 6.1 with arbitrary deviations by right
multiplication implies that uniformly in $u$

$\displaystyle\Theta^{N}*\delta_{u^{-1}}(\chi_{\varepsilon})=D_{\sqrt{N}}\nu*NX_{\mu}*\delta_{u^{-1}}(\chi_{\varepsilon})+o_{\gamma_{0},\chi_{\varepsilon}}(N^{-\frac{d_{\overline{X}}}{2}})$
(43)

where the $o_{\gamma_{0},\chi_{\varepsilon}}(.)$ does not depend on $u$.
However, we may not replace directly $\Theta^{N}$ by
$D_{\sqrt{N}}\nu*NX_{\mu}$ in the integral because the error term is a priori
not integrable. To remedy this, we truncate the domain of integration in a
part on which the error integrates to $o_{\gamma_{0},\chi_{\varepsilon}}(1)$,
and a part that is negligible thanks to the central limit theorem. More
precisely, for $C>0$ we set

$Q_{C,N}=\\{u\in\mathfrak{g}\,:\,\forall i,\,\,\|u^{(i)}\|\leq CN^{i/2}\\}$

By the central limit theorem, Theorem 5.2 above, and the observation that
$Q_{C,N}=D_{\sqrt{N}}(Q_{C,1})$, we may choose $C=C(\mu,\varepsilon)$ large
enough so that for $N\geq 1$,

$(\Theta^{N}*-NX_{\mu})(Q^{c}_{C,N})<\varepsilon\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,D_{\sqrt{N}}\nu(Q_{C,N})<\varepsilon$

where $Q^{c}_{C,N}:=\mathfrak{g}\smallsetminus Q^{c}_{C,N}$. Then we split the
integral in (42) into two parts corresponding to the integration domains
$Q_{2C,N}*NX_{\mu}$ and $Q^{c}_{2C,N}*NX_{\mu}$. Since $\|F\|_{\infty}\leq 1$,
the second part is bounded above by
$(\Theta^{N}*-NX_{\mu})(B_{\mathfrak{g}}(0,\varepsilon)*Q^{c}_{2C,N})$, which
is itself at most $(\Theta^{N}*-NX_{\mu})(Q^{c}_{C,N})<\varepsilon$. The same
applies to $D_{\sqrt{N}}\nu$ in place of $\Theta^{N}*-NX_{\mu}$, so in the end
in view of (43), we get:

$\displaystyle|\Theta^{N}(F_{\varepsilon})-D_{\sqrt{N}}\nu*NX_{\mu}(F_{\varepsilon})|\leq
2\varepsilon+o_{\gamma_{0},\chi_{\varepsilon}}(1)$

Finally, for large $N$ (depending on $\mu$, $\gamma_{0}$, $\varepsilon$,
$\chi_{\varepsilon}$ but not on $F\in\mathscr{L}$), the left-hand side in (41)
is at most $5\varepsilon$, and the final result follows letting $\varepsilon$
go to $0$. ∎

Finally, we show Corollary 1.12.

###### Proof Corollary 1.12.

We identify $G$ with its Lie algebra $\mathfrak{g}$ via the exponential map,
and denote by $*$ the induced group structure. Up to replacing $\mu$ by
$\frac{1}{2}\mu+\frac{1}{2}\delta_{0}$, which has the same harmonic functions,
we may assume that $\mu$ is aperiodic.

We endow $\mathfrak{g}$ with a $*$-right invariant Riemannian metric. Without
loss of generality we may further restrict attention to harmonic functions $F$
that are Lipschitz. Indeed, we may replace $F$ by the convolution $\chi*F$,
where $\chi$ is any smooth bump function with integral $1$. This new function
remains harmonic and bounded. It is furthermore Lipschitz. Finally if $\chi*F$
is a.e. constant for all such $\chi$, then $F$ too is a.e. constant.

Arguing by induction on the dimension of $\mathfrak{g}$ (applying the result
to $\mathfrak{g}/\mathfrak{z}(\mathfrak{g})$), it is enough to prove that $F$
is invariant under translation by the center $\mathfrak{z}(\mathfrak{g})$ of
$\mathfrak{g}$.

So, assume that $F$ is Lipschitz, pick $x_{0}\in\mathfrak{g}$ and
$z\in\mathfrak{z}(\mathfrak{g})$. The rest of the proof is very similar to
Nelson’s proof of Liouville’s theorem [Nel61]. The left translate
$F_{x_{0}}(x)=F(x_{0}x)$ is again bounded, harmonic and it remains
Lipschitz555The Lipschitz constant of $F_{x_{0}}$ depends on $x_{0}$ but this
is not an issue.. Thus by Proposition 8.1 we have

$F(x_{0})=\mu^{*N}(F_{x_{0}})=(D_{\sqrt{N}}\nu*NX_{\mu})(F_{x_{0}})+o_{F_{x_{0}}}(1)$
(44)

and the same holds with $x_{0}z$ in place of $x_{0}$. But since $z$ lies in
the center $\mathfrak{z}(\mathfrak{g})$ we have:

$(D_{\sqrt{N}}\nu*NX_{\mu})(F_{x_{0}z})=\int_{\mathfrak{g}}F(x_{0}zx*NX_{\mu})\,dD_{\sqrt{N}}\nu(x)=\int_{\mathfrak{g}}F(x_{0}x*NX_{\mu})\,d(D_{\sqrt{N}}\nu*\delta_{z})(x)$

In view of (44), we can conclude that $F(x_{0})=F(x_{0}z)$ provided we show
that

$\int_{\mathfrak{g}}F(x_{0}x*NX_{\mu})\,d(D_{\sqrt{N}}\nu*\delta_{z})-D_{\sqrt{N}}\nu)(x)\longrightarrow
0.$

But this is indeed the case, since setting $D_{\sqrt{N}}\nu=u_{N}(x)dx$ where
$v_{N}=N^{-\frac{d_{\overline{X}}}{2}}v\circ D_{\frac{1}{\sqrt{N}}}$, the
above integral is bounded above by:

$\displaystyle\|F\|_{\infty}\int_{\mathfrak{g}}|v_{N}(x+z)-v_{N}(x)|dx=\|F\|_{\infty}\int_{\mathfrak{g}}|v(x+D_{\frac{1}{\sqrt{N}}}z)-v(x)|dx$

which tends to $0$ as $N\to+\infty$.

∎

###  8.2 Equidistribution in homogeneous spaces

In this section we prove Theorem 1.13 regarding equidistribution of unipotent
random walks on homogeneous spaces of finite volume. In a nutshell, we will
use our LLT (twice!) in the guise of Lemma 8.1 above in order to reduce to a
version of Ratner’s equidistribution theorem proved by Shah, Theorem 8.4
below. A similar strategy was employed in [Bre05b].

We first reduce to the case of aperiodic walks thanks to the following general
lemma.

######  Lemma 8.2 (Reduction to aperiodic case).

Let $\mu$ be a probability measure on a group $G$ and
$\mu^{\prime}=\frac{1}{2}\mu+\frac{1}{2}\delta_{1}$. Then for all $N>1$,

$\|\frac{1}{2N}\sum_{k=1}^{2N}\mu^{\prime
k}-\frac{1}{N}\sum_{k=1}^{N}\mu^{k}\|\leq C\frac{\log N}{\sqrt{N}},$

where $\|.\|$ is the total variation norm and $C$ an absolute constant.

###### Proof.

Expanding the power $\mu^{\prime k}$, and setting $\binom{k}{i}=0$ if $i>k$,
we may write

$\displaystyle\sum_{k=1}^{2N}\mu^{\prime
k}=\sum_{i=1}^{2N}\sum_{k=1}^{2N}\frac{1}{2^{k}}\binom{k}{i}\mu^{i}=\sum_{i=1}^{2N}\mathbb{E}(N_{2N}(i))\mu^{i}$

where $N_{p}(i)$ is the occupation time $N_{p}(i):=|\\{k\leq p,S_{k}=i\\}|$
and $(S_{k})_{k\geq 0}$ is the Markov chain on $\mathbb{N}_{\geq 0}$ starting
at $0$ with transition probabilities $p(i,i)=p(i,i+1)=1/2$, for $i\geq 0$. We
can then write:

$\|\frac{1}{2N}\sum_{k=1}^{2N}\mu^{\prime
k}-\frac{1}{N}\sum_{k=1}^{N}\mu^{k}\|\leq\frac{1}{N}\sum_{i=1}^{2N}|\frac{1}{2}\mathbb{E}(N_{2N}(i))-\mathbbm{1}_{i\leq
N}|$ (45)

We split the right hand side in three terms $A_{1}+A_{2}+A_{3}$ according to
the position of $i$ with respect to $p_{N}:=\lfloor N-\sqrt{N}\log N\rfloor$
and $q_{N}=\lfloor N+\sqrt{N}\log N\rfloor$.

Recall, either from the general theory of discrete Markov chains or in this
case by a direct calculation, that $\mathbb{E}(N_{\infty}(i))=2$ for all $i$.
In particular, if $i\in[p_{N},q_{N}]$, each term in the sum $A_{2}$ is bounded
by $2$, so $A_{2}\leq 4\log N/\sqrt{N}$. Next:

$\displaystyle A_{3}$ $\displaystyle=\frac{1}{N}\sum_{i\geq
q_{N}}\mathbb{E}(N_{2N}(i))=\frac{1}{N}\sum_{k\leq 2N}\sum_{i\geq
q_{N}}\mathbb{P}(S_{k}=i)=\frac{1}{N}\sum_{k\leq 2N}\mathbb{P}(S_{k}\geq
q_{N})$ $\displaystyle\leq 2\mathbb{P}(S_{2N}\geq q_{N}).$

However $S_{2N}$ is a binomial random variable and by classical tail bounds
$\mathbb{P}(S_{2N}\geq q_{N})$ decays faster than any inverse power of $N$. So
$A_{3}\leq\log N/\sqrt{N}$ for $N$ large enough.

Finally if $i\leq p_{N}$,

$\displaystyle|\frac{1}{2}\mathbb{E}(N_{2N}(i))-1|=\frac{1}{2}|\mathbb{E}(N_{2N}(i))-\mathbb{E}(N_{\infty}(i))|\leq\frac{1}{2}\sum_{k>2N}\mathbb{P}(S_{k}=i)$

Summing over $i\leq p_{N}$, we get:

$A_{1}\leq\frac{1}{2}\sum_{k>2N}\mathbb{P}(S_{k}\leq
p_{N})\leq\frac{1}{2}\sum_{k>2N}\mathbb{P}(S_{k}\leq
k/2-\sqrt{k/2}\log(k/2)).$

Again each term of the latter sum decays faster than any inverse power of $k$,
hence so does the sum as a function of $N$. This ends the proof. ∎

We keep the notation of Theorem 1.13 and consider the Alexandrov
compactification $\Omega\cup\\{\infty\\}$ of the homogeneous space
$\Omega=H/\Lambda$. The next proposition ensures that the sequence of measures
has no escape of mass and that any limit measure is absolutely continuous with
respect to the homogeneous measure $m_{\omega}$ given by Ratner’s theorem:

######  Proposition 8.3 (Domination by $m_{\omega}$).

There exists $C>0$ such that every weak limit $m$ of the sequence of
probability measures
$(\frac{1}{N}\sum_{k=1}^{N}\mu^{k}*\delta_{\omega})_{n\geq 1}$ on
$\Omega\cup\\{\infty\\}$ satisfies $m\leq Cm_{\omega}$. The result holds
without Cesàro if $\mu$ is centered.

Before giving the proof, we recall the Ratner-Shah equidistribution theorem.
Let $\mathfrak{u}$ be the Lie algebra of the connected $Ad$-unipotent subgroup
$U\leq H$, so $(\mathfrak{u},*)$ is the simply connected covering of $U$. We
say that a vector space basis $(e_{1},\dots,e_{d})$ of $\mathfrak{u}$ is
_triangular_ if it satisfies
$[e_{i},e_{j}]\subseteq\text{Vect}\\{e_{k},\,k>\max(i,j)\\}$ for all $i,j$. A
basis is _regular_ if it can be obtained by permuting the elements in a
triangular basis.

######  Theorem 8.4 (Ratner-Shah).

Let $(e_{1},\dots,e_{d})$ be a _regular_ basis of $\mathfrak{u}$. For
$\underline{t}=(t_{1},\dots,t_{d})\in\mathbb{R}^{d}$, set
$R_{\underline{t}}:=[0,t_{1}]e_{1}*\dots*[0,t_{d}]e_{d}$. Then for
$\omega\in\Omega$,

$\frac{\text{leb}_{|R_{\underline{t}}}}{\text{leb}(R_{\underline{t}})}*\delta_{\omega}\underset{\inf
t_{i}\to+\infty}{\longrightarrow}m_{\omega}$

where leb stands for the Lebesgue measure on $\mathfrak{u}$.

This theorem is due to Ratner when $U$ is one-dimensional. It was subsequently
generalized by Shah [Sha94, Corollary 1.3] in the present form. Note that Shah
states it for simply connected $U$, but his proof does not require this
assumption.

Applying this result to a finite union of rectangles, it clearly holds with
$R_{\underline{t}}$ replaced by
$B_{\underline{t}}:=[-t_{1},t_{1}]e_{1}*\dots*[-t_{d},t_{d}]e_{d}$, as $\inf
t_{i}\to+\infty$.

We now assume that $e_{1},\dots,e_{d}$ enumerates the basis $(e^{(i)}_{j})$
introduced in Section 2. It is indeed a regular basis (thanks to the relation
$[\mathfrak{g}^{(i_{1})},\mathfrak{g}^{(i_{2})}]\subseteq\mathfrak{g}^{(i_{1}+i_{2})}$).
In the non-centred case, we also assume $e_{d}=X_{\mu}$.

###### Proof of Proposition 8.3.

First we observe that Proposition 8.1 allows us to replace $\mu^{k}$ by the
measure $D_{\sqrt{k}}\nu*kX_{\mu}$ without loss of generality. Indeed, endow
$H$ with a right invariant Riemannian metric, and $\Omega$ with the quotient
metric. For every Lipschitz function $f$ on $\Omega$, we set
$F:\mathfrak{u}\to\mathbb{R}$, $F(u)=f(\exp(u)\omega)$. Then $F$ is Lipschitz
on $\mathfrak{u}$ for say a right-invariant Riemannian metric on
$(\mathfrak{u},*)$, and Proposition 8.1 guarantees that

$(\mu^{k}-D_{\sqrt{k}}\nu*kX_{\mu})(F)\underset{k\to+\infty}{\longrightarrow}0.$

Hence, the set of weak limits of
$\frac{1}{N}\sum_{k=1}^{N}\mu^{k}*\delta_{\omega}$ (or
$\mu^{N}*\delta_{\omega}$ in the centered case) does not change if we consider
$D_{\sqrt{k}}\nu*kX_{\mu}$ in place of $\mu^{k}$.

Proposition 8.3 now follows from the following claim.

######  Claim.

There is $C>0$, such that for all $N,\varepsilon>0$ there are positive
measures $m_{N}$ and $\sigma_{N}$ on $\Omega$, with $\|m_{N}\|=1$ and
$\|\sigma_{N}\|\leq\varepsilon$, such that

$\frac{1}{N}\sum_{k=1}^{N}D_{\sqrt{k}}\nu*kX_{\mu}\leq Cm_{N}+\sigma_{N}$

and $m_{N}\to_{N\to+\infty}m_{\omega}$. The statement also holds without
Cesàro average if $X_{\mu}=0$.

Let $K_{1}$ be a compact neighbourhood of the origin in $\mathfrak{u}$ and let
$K_{r}=D_{\sqrt{r}}K_{1}$. Since $\nu$ has a continuous density, there is
$C_{1}>0$ such that $\nu\leq
C_{1}\text{leb}_{|K_{r}}+\nu_{\mathfrak{u}\smallsetminus K_{r}}$ for all
$r\geq 1$. For all $N\geq 1$,

$\displaystyle D_{\sqrt{N}}\nu$ $\displaystyle\leq
D_{\sqrt{N}}C_{1}\text{leb}_{K_{r}}+D_{\sqrt{N}}\nu_{\mathfrak{u}\smallsetminus
K_{r}}=C_{1}N^{-d_{\overline{X}}/2}\text{leb}_{D_{\sqrt{rN}}K_{1}}+D_{\sqrt{N}}\nu_{\mathfrak{u}\smallsetminus
K_{r}}$ $\displaystyle\leq
C_{1}M\frac{\text{leb}_{B_{\underline{t}_{Nr}}}}{\text{leb}(B_{\underline{t}_{Nr}})}+D_{\sqrt{N}}\nu_{\mathfrak{u}\smallsetminus
K_{r}}=Cm_{N}+\sigma_{N}$

setting $C=C_{1}M$, where the constant $M$ and the box
$B_{\underline{t}_{rN}}\supset D_{\sqrt{rN}}K_{1}$ are given by 8.5 below
applied to $K_{1}$. Choosing $r$ such that $\|\nu_{\mathfrak{u}\smallsetminus
K_{r}}\|<\varepsilon$, the result follows in case $\mu$ is centered. In the
general case when $X_{\mu}\neq 0$, we translate by $kX_{\mu}$ on the right and
average over $k$:

$\frac{1}{N}\sum_{k=1}^{N}D_{\sqrt{k}}\nu*kX_{\mu}\leq
C\frac{1}{N}\sum_{k=1}^{N}\frac{\text{leb}_{B_{\underline{t}_{rN}}*kX_{\mu}}}{\text{leb}(B_{\underline{t}_{rN}})}+\sigma_{N}$

where
$\sigma_{N}=\frac{1}{N}\sum_{k=1}^{N}(D_{\sqrt{k}}\nu_{\mathfrak{u}\smallsetminus
K_{r}})*kX_{\mu}$ has mass less than $\varepsilon$. As we took care of
choosing $e_{d}=X_{\mu}$, the diffeomorphism
$t_{1}e_{1}*\dots*t_{d}e_{d}\mapsto(t_{1},\dots,t_{d})$ sends the Lebesgue
measure $\text{leb}_{B_{\underline{t}_{rN}}*kX_{\mu}}$ to the product measure

$\text{leb}_{[-\underline{t}_{rN,1},\underline{t}_{rN,1}]}\otimes\dots\otimes\text{leb}_{[-\underline{t}_{rN,d-1},\underline{t}_{rN,d-1}]}\otimes\text{leb}_{[-\underline{t}_{rN,d},\underline{t}_{rN,d}]+k}$

By construction of $\underline{t}_{rN}$, we have
$\underline{t}_{rN,d}=M\sqrt{rN}$, so for $N$ large enough

$\frac{1}{N}\sum_{k=1}^{N}\text{leb}_{[-\underline{t}_{rN,d},\underline{t}_{rN,d}]+k}\leq\frac{2M\sqrt{Nr}+1}{N}\text{leb}_{[-2N,2N]}$

Hence

$\frac{1}{N}\sum_{k=1}^{N}D_{\sqrt{k}}\nu*kX_{\mu}\leq
C(2M+1)\frac{\sqrt{r}\text{leb}_{B_{\underline{t}^{\prime}_{rN}}}}{\sqrt{N}\text{leb}(B_{\underline{t}_{rN}})}+\sigma_{N}$

where $\underline{t}^{\prime}_{rN}$ is obtained from $\underline{t}_{rN}$ by
replacing the coordinate $\underline{t}_{rN,d}$ par $2N$. As
$\text{leb}(B_{\underline{t}^{\prime}_{rN}})$ is comparable to
$\frac{2N}{\sqrt{rN}}\text{leb}(B_{\underline{t}_{rN}})$ the claim follows by
Theorem 8.4. ∎

In the proof we used the following well-known comparison principle between
Malcev coordinates of the first and second kind [CG90, 1.2].

######  Lemma 8.5 (comparison principle).

Let $K\subseteq\mathfrak{u}$ be a compact set. There exists $M>0$ such that
for all $r>1$, there exists $\underline{t}_{r}\in\mathbb{R}^{d}$ satisfying

$D_{\sqrt{r}}K\subseteq
B_{\underline{t}_{r}}\,\,\,\,\,\,\,\,\,\,et\,\,\,\,\,\,\,\,\,\,\text{leb}(B_{\underline{t}_{r}})\leq
Mr^{\frac{d_{\overline{X}}}{2}}$

###### Proof.

It is sufficient to check that for all $C>0$, there exists $M>0$ such that for
all $r>1$,

$\displaystyle\left\\{\sum_{i,j}s^{(i)}_{j}e^{(i)}_{j}\,:\,\forall
i,j,\,|s^{(i)}_{j}|\leq C\right\\}\subseteq
D_{\frac{1}{\sqrt{r}}}\left\\{\prod_{i,j}t^{(i)}_{j}e^{(i)}_{j}\,:\,\forall
i,j,\,|t^{(i)}_{j}|\leq Mr^{i/2}\right\\}$ (46)

where the product follows the order prescribed the enumeration
$e_{1},\dots,e_{d}$. But we can write
$D_{\frac{1}{\sqrt{r}}}\prod_{i,j}t^{(i)}_{j}e^{(i)}_{j}=\prod^{*^{\prime}}_{i,j}r^{-i/2}t^{(i)}_{j}e^{(i)}_{j}+P_{r}((r^{-i/2}t^{(i)}_{j})_{i,j})$
where $*^{\prime}$ is the graded product coming from $*$ for the weight
filtration, and $P_{r}$ is obtained by dividing by a (non zero) power of
$\sqrt{r}$ each coefficent of a polynomial function $P$ with $d$ variables.
Then, we choose $r_{0}>0$ such that for $r>r_{0}$,

$\|P_{r}((r^{-i/2}t^{(i)}_{j})_{i,j})\|\leq 1$

and $M>0$ large enough so that

$\left\\{\sum_{i,j}s^{(i)}_{j}e^{(i)}_{j}\,:\,\forall i,j,\,|s^{(i)}_{j}|\leq
C+1\right\\}\subseteq\left\\{{\prod_{i,j}}^{*^{\prime}}t^{(i)}_{j}e^{(i)}_{j}\,:\,\forall
i,j,\,|t^{(i)}_{j}|\leq M\right\\}$

This proves (46) for $r>r_{0}$. Up to increasing the value of $M$ even more,
we get the result for all $r>1$. ∎

We can finally conclude the proof of Corollary 1.13.

###### Proof of Corollary 1.13.

Proposition 8.3 guarantees that every weak limit of the sequence
$(\frac{1}{N}\sum_{k=1}^{N}\mu^{N}*\delta_{\omega})_{N\geq 1}$ is a
probability measure on $\Omega$ that is absolutely continuous with respect to
$m_{\omega}$. It is also $\mu$-stationary, so it must be $U$-invariant by
Corollary 1.12 and hence $m_{\omega}$ itself, because $m_{\omega}$ is
$U$-ergodic by Ratner’s theorem [Rat91].

When $\mu$ is centered, we may apply the same argument to the sequence
$(\mu^{N}*\delta_{\omega})_{N\geq 1}$, but we need to justify why limit
measures are $\mu$-stationary. It is sufficient to check that for every
compactly supported smooth function $f$ on $\Omega$, we have
$|\mu^{N+1}*\delta_{\omega}(f)-\mu^{N}*\delta_{\omega}(f)|\rightarrow 0$ as
$N\to+\infty$. Setting $F(u)=f(\exp(u)\omega)$ we obtain a bounded Lipschitz
function on $\mathfrak{u}$ and by Proposition 8.1, it boils down to checking
that $|D_{\sqrt{N+1}}\nu(F)-D_{\sqrt{N}}\nu(F)|\rightarrow 0$. To see this, we
observe that for the norm in total variation,

$\|D_{\sqrt{N+1}}\,\nu-D_{\sqrt{N}}\nu\|=\|\nu-
D_{\sqrt{\frac{N}{N+1}}}\,\nu\|\longrightarrow 0$

whence the result.

∎

## References

  * [AS04] Andrei A. Agrachev and Yuri L. Sachkov.  Control theory from the geometric viewpoint, volume 87 of Encyclopaedia of Mathematical Sciences.  Springer-Verlag, Berlin, 2004.  Control Theory and Optimization, II.
  * [Ale02a] Georgios Alexopoulos.  Centered densities on Lie groups of polynomial volume growth.  Probab. Theory Related Fields, 124(1):112–150, 2002.
  * [Ale02b] Georgios Alexopoulos.  Random walks on discrete groups of polynomial volume growth.  Ann. Probab., 30(2):723–801, 2002.
  * [Ale02c] Georgios Alexopoulos.  Sub-Laplacians with drift on Lie groups of polynomial volume growth.  Mem. Amer. Math. Soc., 155(739):x+101, 2002.
  * [AP17] Jürgen Angst and Guillaume Poly.  A weak Cramér condition and application to Edgeworth expansions.  Electron. J. Probab., 22:Paper No. 59, 24, 2017.
  * [AK63] Vladimir Arnold and A Krylov.  Uniform distribution of points on a sphere and certain ergodic properties of solutions of linear ordinary differential equations in a complex domain.  Dokl. Akad. Nauk SSSR, 148:9–12, 1963.
  * [Bel96] André Bellaïche.  The tangent space in sub-Riemannian geometry.  In Sub-Riemannian geometry, volume 144 of Progr. Math., pages 1–78. Birkhäuser, Basel, 1996.
  * [Bénar] Timothée Bénard.  Equidistribution of mass for random processes on finite-volume spaces.  Israel J. of Math., pages 333–379, to appear.
  * [BdS16] Yves Benoist and Nicolas de Saxcé.  A spectral gap theorem in simple Lie groups.  Invent. Math., 205(2):337–361, 2016.
  * [BQ11] Yves Benoist and Jean-François Quint.  Mesures stationnaires et fermés invariants des espaces homogènes.  Ann. of Math. (2), 174(2):1111–1162, 2011.
  * [BQ16] Yves Benoist and Jean-François Quint.  Random walks on reductive groups, volume 62 of Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics].  Springer, Cham, 2016.
  * [BP96] Vidmantas Bentkus and Gyula Pap.  The accuracy of Gaussian approximations in nilpotent Lie groups.  J. Theoret. Probab., 9(4):995–1017, 1996.
  * [Bla55] David Blackwell.  On transient Markov processes with a countable number of states and stationary transition probabilities.  Ann. Math. Statist., 26:654–658, 1955.
  * [Bou81] Philippe Bougerol.  Théorème central limite local sur certains groupes de Lie.  Ann. Sci. École Norm. Sup. (4), 14(4):403–432 (1982), 1981\.
  * [BG12] Jean Bourgain and Alexander Gamburd.  A spectral gap theorem in ${\rm SU}(d)$.  J. Eur. Math. Soc. (JEMS), 14(5):1455–1511, 2012.
  * [Bre92] Leo Breiman.  Probability, volume 7 of Classics in Applied Mathematics.  Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1992.  Corrected reprint of the 1968 original.
  * [Bre04] Emmanuel Breuillard.  Equidistribution of random walks on nilpotent Lie groups and homogeneous spaces.  ProQuest LLC, Ann Arbor, MI, 2004.  Thesis (Ph.D.)–Yale University.
  * [Bre05a] Emmanuel Breuillard.  Distributions diophantiennes et théorème limite local sur $\mathbb{R}^{d}$.  Probab. Theory Related Fields, 132(1):39–73, 2005.
  * [Bre05b] Emmanuel Breuillard.  Local limit theorems and equidistribution of random walks on the Heisenberg group.  Geom. Funct. Anal., 15(1):35–82, 2005.
  * [Bre14] Emmanuel Breuillard.  Geometry of locally compact groups of polynomial growth and shape of large balls.  Groups Geom. Dyn., 8(3):669–732, 2014.
  * [BFH09] Emmanuel Breuillard, Peter Friz, and Martin Huesmann.  From random walks to rough paths.  Proc. Amer. Math. Soc., 137(10):3487–3496, 2009.
  * [Cha06] Sourav Chatterjee.  A generalization of the Lindeberg principle.  Ann. Probab., 34(6):2061–2076, 2006.
  * [Che77] Kuo Tsai Chen.  Iterated path integrals.  Bull. Amer. Math. Soc., 83(5):831–879, 1977.
  * [CD60] Gustave Choquet and Jacques Deny.  Sur l’équation de convolution $\mu=\mu\ast\sigma$.  C. R. Acad. Sci. Paris, 250:799–801, 1960.
  * [CG90] Lawrence Corwin and Frederick Greenleaf.  Representations of nilpotent Lie groups and their applications. Part I, volume 18 of Cambridge Studies in Advanced Mathematics.  Cambridge University Press, Cambridge, 1990.  Basic theory and examples.
  * [CR78] Pierre Crépel and Albert Raugi.  Théorème central limite sur les groupes nilpotents.  Ann. Inst. H. Poincaré Sect. B (N.S.), 14(2):145–164, 1978\.
  * [DH21] Persi Diaconis and Robert Hough.  Random walk on unipotent matrix groups.  Ann. Sci. Éc. Norm. Supér. (4), 54(3):587–625, 2021.
  * [DL57] Jacques Dixmier and W. G. Lister.  Derivations of nilpotent Lie algebras.  Proc. Amer. Math. Soc., 8:155–158, 1957.
  * [Dye70] Joan Dyer.  A nilpotent Lie algebra with nilpotent automorphism group.  Bull. Amer. Math. Soc., 76:52–56, 1970.
  * [Dyn47] Eugene Dynkin.  Calculation of the coefficients in the Campbell-Hausdorff formula.  Doklady Akad. Nauk SSSR (N.S.), 57:323–326, 1947.
  * [DM61] Eugene Dynkin and Mikhail Maljutov.  Random walk on groups with a finite number of generators.  Dokl. Akad. Nauk SSSR, 137:1042–1045, 1961.
  * [FHTVF19] Joshua Frisch, Yair Hartman, Omer Tamuz, and Pooya Vahidi Ferdowsi.  Choquet-Deny groups and the infinite conjugacy class property.  Ann. of Math. (2), 190(1):307–320, 2019.
  * [FH14] Peter Friz and Martin Hairer.  A course on rough paths.  Universitext. Springer, Cham, 2014.  With an introduction to regularity structures.
  * [FV10] Peter Friz and Nicolas Victoir.  Multidimensional stochastic processes as rough paths, volume 120 of Cambridge Studies in Advanced Mathematics.  Cambridge University Press, Cambridge, 2010.  Theory and applications.
  * [Gav77] Bernard Gaveau.  Principe de moindre action, propagation de la chaleur et estimées sous elliptiques sur certains groupes nilpotents.  Acta Math., 139(1-2):95–153, 1977.
  * [GK68] B. V. Gnedenko and A. N. Kolmogorov.  Limit distributions for sums of independent random variables.  Addison-Wesley Publishing Co., Reading, Mass.-London-Don Mills, Ont., revised edition, 1968.  Translated from the Russian, annotated, and revised by K. L. Chung, With appendices by J. L. Doob and P. L. Hsu.
  * [Gre63] Ulf Grenander.  Probabilities on algebraic structures.  John Wiley & Sons, Inc., New York-London; Almqvist & Wiksell, Stockholm-Göteborg-Uppsala, 1963.
  * [Gro96] Mikhael Gromov.  Carnot-Carathéodory spaces seen from within.  In Sub-Riemannian geometry, volume 144 of Progr. Math., pages 79–323. Birkhäuser, Basel, 1996.
  * [Gui73] Yves Guivarc’h.  Croissance polynomiale et périodes des fonctions harmoniques.  Bull. Soc. Math. France, 101:333–379, 1973.
  * [Gui76] Yves Guivarc’h.  Equirépartition dans les espaces homogènes.  In Théorie ergodique (Actes Journées Ergodiques, Rennes, 1973/1974), Lecture Notes in Math., Vol. 532, pages 131–142. Springer, Berlin, 1976.
  * [HS01] Wilfried Hazod and Eberhard Siebert.  Stable probability measures on Euclidean spaces and on locally compact groups, volume 531 of Mathematics and its Applications.  Kluwer Academic Publishers, Dordrecht, 2001.  Structural properties and limit theorems.
  * [Heb89] Waldemar Hebisch.  Sharp pointwise estimate for the kernels of the semigroup generated by sums of even powers of vector fields on homogeneous groups.  Studia Math., 95(1):93–106, 1989.
  * [Hör67] Lars Hörmander.  Hypoelliptic second order differential equations.  Acta Math., 119:147–171, 1967.
  * [Hou19] Robert Hough.  The local limit theorem on nilpotent Lie groups.  Probab. Theory Relat. Fields, 174(3-4):761–786, 2019.
  * [Hun56] G. A. Hunt.  Semi-groups of measures on Lie groups.  Trans. Amer. Math. Soc., 81:264–293, 1956.
  * [Ish03] Satoshi Ishiwata.  A central limit theorem on a covering graph with a transformation group of polynomial growth.  J. Math. Soc. Japan, 55(3):837–853, 2003.
  * [IKN20] Satoshi Ishiwata, Hiroshi Kawabi, and Ryuya Namba.  Central limit theorems for non-symmetric random walks on nilpotent covering graphs: Part I.  Electron. J. Probab., 25:Paper No. 86, 46, 2020.
  * [IKN21] Satoshi Ishiwata, Hiroshi Kawabi, and Ryuya Namba.  Central limit theorems for non-symmetric random walks on nilpotent covering graphs: Part II.  Potential Anal., 55(1):127–166, 2021.
  * [JR07] Wojciech Jaworski and Robinson Edward Raja.  The Choquet-Deny theorem and distal properties of totally disconnected locally compact groups of polynomial growth.  New York J. Math., 13:159–174, 2007.
  * [Jør75] Palle Jørgensen.  Representations of differential operators on a Lie group.  J. Functional Analysis, 20(2):105–135, 1975.
  * [KI40] Yukiyosi Kawada and Kiyosi Itô.  On the probability distribution on a compact group. I.  Proc. Phys.-Math. Soc. Japan (3), 22:977–998, 1940.
  * [Kaz65] David Kazhdan.  Uniform distribution on a plane.  Trudy Moskov. Mat. Obšč., 14:299–305, 1965.
  * [Kis79] Jan Kisyński.  On semigroups generated by differential operators on Lie groups.  J. Functional Analysis, 31(2):234–244, 1979.
  * [Kog22] Constantin Kogler.  Local limit theorem for random walks on symmetric spaces.  arXiv preprint arXiv:2211.11128, page 45p., 2022.
  * [KP16] Alessia E. Kogoj and Sergio Polidoro.  Harnack inequality for hypoelliptic second order partial differential operators.  Potential Anal., 45(3):545–555, 2016.
  * [Lév51] Paul Lévy.  Wiener’s random function, and other Laplacian random functions.  In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, 1950, pages 171–187. University of California Press, Berkeley-Los Angeles, Calif., 1951.
  * [Lia04] Ming Liao.  Lévy processes in Lie groups, volume 162 of Cambridge Tracts in Mathematics.  Cambridge University Press, Cambridge, 2004.
  * [LV16] Elon Lindenstrauss and Péter Varjú.  Random walks in the group of Euclidean isometries and self-similar measures.  Duke Math. J., 165(6):1061–1127, 2016.
  * [LQ02] Terry Lyons and Zhongmin Qian.  System control and rough paths.  Oxford Mathematical Monographs. Oxford University Press, Oxford, 2002\.  Oxford Science Publications.
  * [Mor05] Dave Witte Morris.  Ratner’s theorems on unipotent flows.  Chicago Lectures in Mathematics. University of Chicago Press, Chicago, IL, 2005.
  * [MOO10] Elchanan Mossel, Ryan O’Donnell, and Krzysztof Oleszkiewicz.  Noise stability of functions with low influences: invariance and optimality.  Ann. of Math. (2), 171(1):295–341, 2010.
  * [Nel61] Edward Nelson.  A proof of Liouville’s theorem.  Proc. Amer. Math. Soc., 12:995, 1961.
  * [Pap91] Gyula Pap.  Rate of convergence in CLT on stratified groups.  J. Multivariate Anal., 38(2):333–365, 1991.
  * [RR62] R. Ranga Rao.  Relations between weak and uniform convergence of measures with applications.  Ann. Math. Statist., 33:659–680, 1962.
  * [Rat91] Marina Ratner.  On Raghunathan’s measure conjecture.  Ann. of Math. (2), 134(3):545–607, 1991.
  * [Rau78] Albert Raugi.  Théorème de la limite centrale sur les groupes nilpotents.  Z. Wahrsch. Verw. Gebiete, 43(2):149–172, 1978.
  * [Rau04] Albert Raugi.  A general Choquet-Deny theorem for nilpotent groups.  Ann. Inst. H. Poincaré Probab. Statist., 40(6):677–683, 2004\.
  * [Reu93] Christophe Reutenauer.  Free Lie algebras, volume 7 of London Mathematical Society Monographs. New Series.  The Clarendon Press, Oxford University Press, New York, 1993.  Oxford Science Publications.
  * [RY99] Daniel Revuz and Marc Yor.  Continuous martingales and Brownian motion, volume 293 of Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences].  Springer-Verlag, Berlin, third edition, 1999.
  * [Rot75] V. I. Rotar.  Limit theorems for multilinear forms and quasipolynomial functions.  Teor. Verojatnost. i Primenen., 20(3):527–546, 1975.
  * [Sac07] Yuri L. Sachkov.  Control theory on Lie groups.  Sovrem. Mat. Fundam. Napravl., 27:5–59, 2007.
  * [SC01] Laurent Saloff-Coste.  Probability on groups: random walks and invariant diffusions.  Notices Amer. Math. Soc., 48(9):968–977, 2001.
  * [ST66] Vyacheslav Sazonov and Valeri Tutubalin.  Probability distributions on topological groups.  Teor. Verojatnost. i Primenen., 11:3–55, 1966.
  * [Ser06] Jean-Pierre Serre.  Lie algebras and Lie groups, volume 1500 of Lecture Notes in Mathematics.  Springer-Verlag, Berlin, 2006.  1964 lectures given at Harvard University, Corrected fifth printing of the second (1992) edition.
  * [Sha94] Nimish Shah.  Limit distributions of polynomial trajectories on homogeneous spaces.  Duke Math. J., 75(3):711–732, 1994.
  * [Sie82] Eberhard Siebert.  Absolute continuity, singularity, and supports of Gauss semigroups on a Lie group.  Monatsh. Math., 93(3):239–253, 1982.
  * [Sie84] Eberhard Siebert.  Densities and differentiability properties of Gauss semigroups on a Lie group.  Proc. Amer. Math. Soc., 91(2):298–305, 1984.
  * [Str87] Robert S. Strichartz.  The Campbell-Baker-Hausdorff-Dynkin formula and solutions of differential equations.  J. Funct. Anal., 72(2):320–345, 1987.
  * [SV72] Daniel W. Stroock and S. R. S. Varadhan.  On the support of diffusion processes with applications to the strong maximum principle.  Proc. 6th Berkeley Sympos. math. Statist. Probab., Univ. Calif. 1970, 3, 333-359 (1972)., 1972.
  * [SV73] Daniel W. Stroock and S. R. S. Varadhan.  Limit theorems for random walks on Lie groups.  Sankhyā Ser. A, 35(3):277–294, 1973.
  * [Sus87] Hector Sussmann.  A general theorem on local controllability.  SIAM J. Control Optim., 25(1):158–194, 1987.
  * [Tut64] Valeri N. Tutubalin.  Compositions of measures on the simplest nilpotent group.  Teor. Verojatnost. i Primenen, 9:531–539, 1964.
  * [Var15] Péter Varjú.  Random walks in Euclidean space.  Ann. of Math. (2), 181(1):243–301, 2015.
  * [Var88] Nicholas Varopoulos.  Analysis on Lie groups.  J. Funct. Anal., 76(2):346–410, 1988.
  * [VSCC92] Nicholas Varopoulos, Laurent. Saloff-Coste, and Thierry Coulhon.  Analysis and geometry on groups, volume 100 of Cambridge Tracts in Mathematics.  Cambridge University Press, Cambridge, 1992.
  * [Vir74] Alexander D. Virtser.  Limit theorems for compositions of distributions on certain nilpotent Lie groups.  Teor. Verojatnost. i Primenen., 19:84–103, 1974.
  * [Weh62] Donald Wehn.  Probabilities on Lie groups.  Proc. Nat. Acad. Sci. U.S.A., 48:791–795, 1962.

Centre for Mathematical Sciences, Wilberforce Road, Cambridge CB3 0WB, United
Kingdom

Email address<EMAIL_ADDRESS>

Mathematical Institute, University of Oxford, Woodstock Rd Oxford OX2 6GG,
United Kingdom

Email address<EMAIL_ADDRESS>