# An Efficient Online Prediction of Host Workloads Using Pruned GRU Neural
Nets

Amin Setayesh  University of Tehran
<EMAIL_ADDRESS>Hamid Hadian  University of Virginia Tech
<EMAIL_ADDRESS>Radu Prodan  University of Klagenfurt
<EMAIL_ADDRESS>
###### Abstract

Host load prediction is essential for dynamic resource scaling and job
scheduling in a cloud computing environment. In this context, workload
prediction is challenging because of several issues. First, it must be
accurate to enable precise scheduling decisions. Second, it must be fast to
schedule at the right time. Third, a model must be able to account for new
patterns of workloads so it can perform well on the latest and old patterns.
Not being able to make an accurate and fast prediction or the inability to
predict new usage patterns can result in severe outcomes such as service level
agreement (SLA) misses. Our research trains a fast model with the ability of
online adaptation based on the gated recurrent unit (GRU) to mitigate the
mentioned issues. We use a multivariate approach using several features, such
as memory usage, CPU usage, disk I/O usage, and disk space, to perform the
predictions accurately. Moreover, we predict multiple steps ahead, which is
essential for making scheduling decisions in advance. Furthermore, we use two
pruning methods: L1 norm and random, to produce a sparse model for faster
forecasts. Finally, online learning is used to create a model that can adapt
over time to new workload patterns.

###### Index Terms:

Cloud Computing, Workload Prediction, Gated Recurrent Units (GRU), Pruning,
Online Learning

##  I Introduction

Cloud computing has gained extreme usage and popularity in recent years, and
the global market size reached an estimation of 482 billion dollars in 2022
[1]. Many end-users and IT companies use cloud computing services such as
storage, hosting, computing, and software services due to their financial
efficiency gained by the elastic resource provisioning and pay-as-you-go plans
[2]. Cloud environments and clusters provide on-demand access over a shared
pool of computing resources for for IoT applications and scientific
applications and host a vast number of applications with various resource
demands and optimization objectives[3][4]. The dynamic workload of
applications and the ever-changing need of users make it hard to estimate
resource utilization accurately, which may result in failure or a reduction in
Quality of service (QoS) due to inaccurate resource allocation decisions.

Optimal allocation of resources ensures satisfying users’ needs and minimizing
cost. Keeping all the computational resources active results in over-
provisioning and is not efficient. On the other hand, provisioning
insufficient resources causes violations of the SLA [5], [6], [7]. The
approach towards resource provisioning could be either reactive or proactive.
In the reactive method, the application constantly monitors some metrics, like
(CPU usage and waiting time in the queue) and allocates resources accordingly.
Still, the main problem is that the reactive method may perform late
provisions because of identifying insufficient resources at the exact time
they happen. However, the proactive approach tries to anticipate the required
resources several time steps ahead to deal with lacking resources in advance
[6], [8]. Acquiring proper insight into future proactive resource utilization
in cloud environments can pave the way to have a better scheduling plan to
spread tasks across virtual machines. in addition, an accurate estimation
helps applications to exploit auto-scaling and migration techniques before
encountering a heavy workload.

Resource prediction mathematically can be modeled by time series. In time
series prediction, previous trends of data are used to predict new events in
the future [9]. Several statistical methods, like regressive integrated moving
average (ARIMA) [10] and Markov models [11], [12] are commonly used to
forecast future workloads. As workload trends are non-linear, AI-based methods
like neural networks, support vector machines (SVM), and reinforcement
learning (RL) for workload prediction are also utilized by researchers to
minimize prediction error [9], [13]. Due to the ability of recurrent neural
networks (RNNs) to capture long-range patterns of workloads, these neural
networks have recently been widely used for host load prediction [3], [9].
Because of not being able to carry learning gradients well in deep traditional
RNNs, several new variants, such as long short-term memory (LSTM) [14], and
gated recurrent unit (GRU) [15], are introduced to perform better on deep
neural networks.

For the prediction of resource utilization, various parameters affect each
other. For example, the amount of used memory or the number of I/O requests
can directly impact CPU utilization. Multivariate models consider the effect
of different features on each other and make a more accurate model to predict
future trends [16]. In addition, being only one step ahead of time while
predicting resource utilization does not guarantee precise provisioning of
resources. Moreover, The workload in cloud environments is quite variant and a
trained model cannot fully exhibit the trends. Hence, using adaptive models
enables parameter adjustments at run time [5], [17]. Another vital challenge
is that neural network models require considerable computation resources like
memory and CPU, so pruning aims to drop some of the weights to reduce
prediction time and resource consumption. Specifically, making a real-time
prediction for critical systems is crucial to providing resources as soon as
possible.

In this regard, the main contributions of the paper would be:

  1. 1.

_Analysis_ of different GRU network hyperparameters in addition to comparison
with baseline models.

  2. 2.

_Development_ of a multivariate multistep GRU network being able to predict
host workloads accurately.

  3. 3.

_Comparison_ of two different Sparse GRU methods: random structured and L1
norm approaches, developed to reduce prediction time with slight accuracy
decay.

  4. 4.

_Examination_ of online adaptation using two optimization methods: GD and
L-BFGS, used to let our model improve over time by observing new patterns.

The rest of the paper is organized as follows: First, the related work is
introduced in Section II. Next, the methodology being used is described
thoroughly in Section III. Then, our experiments are presented in Section IV.
Finally, our conclusions and future work are discussed in Section V.

##  II Related Work

Various studies have investigated predicting the workload of cloud hosts and
applied different methods to solve the workload prediction problem, improving
models’ accuracy. These methods can be classified into four broad categories:
statistical, machine learning, neural network, and hybrid. We emphasize the
neural network approaches more as our study falls in this category.

Using Regression methods, Gupta et al. [18] provide an ensemble approach,
which uses multilinear regression in combination with the moving average (MV),
weighted average, and neural networks. The Ensemble Workload Prediction
(EnWoP) proposed in this study uses a new prediction evaluation parameter
called C-Rate to predict Google Cluster trace workloads.

Analytical models such as autocorrelation function (ACF), autoregressive
integrated moving average (ARIMA) [19], and seasonal integrated ARMA (SARIMA)
[20] are widely used for time series prediction. Many works utilize these
approaches to predict host workloads. In [19], the authors used the ARIMA
model to predict workload. Their results show an average accuracy of up to
91%. Additionally, Kumar et al. [20] compared several ARMA models, such as
ARIMA, seasonal integrated ARMA (SARIMA), and fractionally integrated ARMA
(ARFIMA). They evaluated the models using the Wikimedia CPU, RAM, and Network
traces; the researchers found ARIMA the model providing the best forecast
results.

Using machine learning schemes, researchers mostly used clustering, SVM,
decision trees, and random forest approaches. Zhong et al. [21] and Nikravesh
et al. [22] used SVM-based techniques to predict host loads. Zhong et al. [21]
use an SVM in combination with a wavelet transform (WT) to create the WWSVM
prediction model, which they state is superior to the previous naive SVM model
and ARIMA model in prediction accuracy. Nikravesh et al. [22] use SVMs and
MLP-based algorithms to achieve better prediction accuracy using different
algorithms for different workload patterns. Furthermore, in [23] and [24]
decision trees were used. In addition, Cetinski et al. [25] used a random
forest classifier to improve prediction accuracy. They also provided
confidence factors for the results for more reliability.

Neural network-based methods have attracted significant attention in recent
years for forecasting trends in time series. Primarily, recurrent neural
networks and the derived models, such as Long Short-Term Memory (LSTM), are
widely used to predict host loads. Kumar et al. [26] exploits the LSTM-RNN
method to predict the workload of different servers. They compared the LSTM-
RNN results with the previously used approaches and showed this model
outperforms the other methods. In [27], the authors used a convolutional
neural network (CNN) in combination with LSTMs to achieve higher accuracy.
They evaluated the model on the Bit-brains distributed data center and showed
it has a lower mean absolute error (MAE) than the ARIMA-LSTM, VAR-GRU, and
VAR-MLP methods. Furthermore, Dang-Quang et al. [28] proposed an autoscaling
framework based on the bi-directional LSTM (Bi-LSTM) method. They used
Bitbrains and Materna data centers’ traces to evaluate the results. The
evaluation showed that the multivariate Bi-LSTM model could outperform the
univariate Bi-LSTM model in addition to LSTM and CNN-LSTM models. Cheng et al.
[16] proposed a method using a combination of GRU and the exponential
smoothing method. They experimented with different multi-step CPU utilization
forecasting on the Google cluster traces. They found that the error increases
with step count. However, experimental results also suggested an improvement
in errors compared to previous state-of-the-art models.

Trying to create a model which improves through time and can adapt itself with
variations, Kumar et al. [29] and Amiri et al. [30] use an online learning
approach. The online learning models try to capture new patterns in workloads
through time, which is essential in cloud environments. Furthermore, in [31],
the authors describe an online learning method to estimate ARIMA models, which
can be used by the workload prediction models using analytical approaches.

In summary, current methods in workload prediction use regression,
statistical, machine learning, and specially RNN-based methods to improve
model accuracy. In addition, some research has been done on faster prediction
and online adaptation of cloud workloads, which is essential for cloud
environments. However, to the authors’ knowledge, limited work has been done
to develop a fast, accurate, and capable of online adaptation model for cloud
workloads at the same time.
To deal with the previous limitations, we use a multivariate GRU network to
improve multistep predictions. We also utilize online learning and pruning
methods to improve prediction accuracy over time and enhance prediction time
to create a fast and adaptable model in addition to being accurate.

##  III Methodology

In this section, we describe our approach to predicting host workloads and
introduce the improvements made to develop a faster and more accurate model
with the ability of online learning over time. First, we define the cloud host
load prediction as a time series problem. Next, we introduce our approach,
which uses a GRU neural network. Then, our pruning methods are introduced,
enabling us to create a model predicting loads in a shorter time, which is
essential in high-performance computing environments. Finally, we present our
online learning method, which helps model adaptation over time.

###  III-A Workload Prediction Problem

The host load prediction problem can be modeled by a time series prediction
problem. The host loads from the past are available to this model, and the
goal is to predict the future workloads based on the previous observations
accurately. The simplest form of this prediction is just to predict one step
ahead using the previous trends; however, it might be needed to know multi-
steps ahead, and the model must also enable these kinds of predictions.

We define $F_{t}$ as the feature vector, representing the features available
at time $t$. $F_{t}$ can be shown as
$(F_{t}^{1},F_{t}^{2},F_{t}^{3},...,F_{t}^{N})$, where $F_{t}^{n}$ shows the
value of feature $n$ at time $t$. Furthermore, we define the complete
observations as $F_{1},F_{2},F_{3},...,F_{T}$.

It should be noted that the features are normalized in a range between zero
and one. Let us define function $f$ as the normalizing function and the
$f^{-1}$ as the reverse normalizing function. The function $f$ is applied to
the feature vector $F_{t}$ to create the normalized feature vector, which we
define as $F^{\prime}_{t}$.

We define the single-step workload prediction problem as forecasting
$F^{\prime i}_{t}$ from the previous $k$ workload observations, which can be
shown as $F^{\prime}_{t-k},F^{\prime}_{t-k+1},...,F^{\prime}_{t-1}$. We define
this predicted normalized workload as $\hat{F^{\prime i}_{t}}$, and the non-
normalized predicted workload $\hat{F^{i}_{t}}$ is calculated by applying the
reverse normalizing function $f^{\prime}$. Similarly, the multi-step ahead
prediction is formulated as forecasting $F^{\prime i}_{t},F^{\prime
i}_{t+1},...,F^{\prime i}_{t+m}$ from the previous $k$ workload observations,
which can be shown as
$F^{\prime}_{t-k},F^{\prime}_{t-k+1},...,F^{\prime}_{t-1}$. In most usages,
the single-step prediction is not enough, and the multi-step forecast is used
for better knowledge of future workloads.

###  III-B GRU

Gated recurrent neural networks (GRU) were introduced by [32] as a variation
of the vanilla RNNs. LSTMs were first developed to help models capture long-
term dependencies, which vanilla RNNs could not previously detect due to the
gradient vanishing problem. Following the LSTM method, the GRU approach was
introduced, and the comparison in [15] showed the GRU method could outperform
the LSTM approach in terms of convergence and generalization on some datasets.

Due to the similarity of the GRU cell and the LSTM cell, we first explain the
LSTM cell and then go on to explain the GRU cell. The LSTM cell uses a notion
of memory, which enables the neural network to capture long-term dependencies
by using the memory. LSTM unit can choose to either use the memory output or
the current output. The LSTM cell is depicted in Figure 1.

Figure 1: The architecture of an LSTM cell

$f_{t}$ is the extent to which the memory is forgotten, and it’s defined as:

$f_{t}=\sigma(W_{f}x_{t}+U_{f}h_{t-1}+V_{f}c_{t-1})$ (1)

$i_{t}$ is the extent to which the new memory is added to the previous memory
values, and it’s defined as:

$i_{t}=\sigma(W_{i}x_{t}+U_{i}h_{t-1}+V_{i}c_{t-1})$ (2)

$V_{f}$ and $V_{i}$ are diagonal matrices [15].

The memory is then updated using $f_{t}$ and $i_{t}$ as the rate the memory is
going to be forgotten and the rate the new memory is learned, respectively.
final memory $c_{t}$ is calculated as:

$c_{t}=f_{t}c_{t-1}+i_{t}\tilde{c_{t}}$ (3)

Where $\tilde{c_{t}}$ is the new memory content, and it’s defined as:

$\tilde{c_{t}}=tanh(W_{c}x_{t}+U_{c}h_{t-1})$ (4)

$h_{t}$ is the activation, and it’s calculated as:

$h_{t}=o_{t}tanh(c_{t})$ (5)

where $o_{t}$ is the output gate.

Unlike the LSTM cell, the GRU cell doesn’t have a separate memory cell. Figure
2 shows the illustration of a GRU cell.

Figure 2: The architecture of a GRU cell

$h_{t}$ shows the activation, calculated as the combination of the previous
activations and the candidate activate function, which we introduce later.
$h_{t}$ is defined as:

$h_{t}=(1-z_{t})h_{t-1}+z_{t}\tilde{h_{t}}$ (6)

where $z_{t}$ is the update gate, which shows how much the activation will be
updated. In addition, $\tilde{h_{t}}$ is the candidate activation function.
The update gate is defined as:

$z_{t}=sigma(W_{z}x_{t}+U_{z}h_{t-1})$ (7)

The candidate activation function $\tilde{h_{t}}$ is defined as:

$\tilde{h_{t}}=tanh(Wx_{t}+U(r_{t}\odot h_{t-1}))$ (8)

where $r_{t}$ is the reset gate and determines how much info will be
forgotten. The reset gate is calculated as:

$r_{t}=\sigma(W_{r}x_{t}+U_{r}h_{t-1})$ (9)

###  III-C Model Pruning

Most neural network models keep numerous weights as their parameters after the
training process. However, this can increase the model prediction time because
of more computations involved. Model pruning is the process of removing some
model parameters to achieve better model speed without a notable drawback to
the model’s accuracy. It is essential to mention that cloud computing
workloads are often generated with a high ratio, and a faster model can help
make better scheduling decisions in advance. Therefore, we used pruning
algorithms to enhance the model’s speed with minimum accuracy drop.

Structured and unstructured methods are the main approaches used to achieve
model pruning. Unstructured pruning is the process of setting zero values for
the individual model’s weights. On the other hand, structured pruning removes
entire layers or neurons to achieve a lighter model. Most current deep
learning frameworks just set zero values for model weights while performing
unstructured pruning. Because of this, unstructured pruning does not result in
notable speed changes since the zero values are multiplied as before, and the
total computations remain the same to some degree. We used two structured
pruning methods to obtain better speed-ups: random and L1 norm structured
approaches. The random structured method starts to choose model parameters
randomly and then removes them iteratively. However, the L1 norm structured
method removes model parameters by calculating the L1 norms and iteratively
removing parameters with the lowest magnitudes. L1 norm of a vector $V$ is
defined as the following:

$L1(V)=\sum_{i=1}^{N}v_{i}$ (10)

Where $N$ is the number of elements in the vector, and $v_{i}$ shows the $i$th
element of the vector $V$.

###  III-D Online Learning

Traditional workload prediction methods mostly perform initial training on the
data, and no approach is suggested to improve the trained model while new
workloads are observed. Consequently, the model may become inaccurate over
time because of the inability to utilize new patterns generated in the
workloads’ data. Although the inaccuracy can be avoided by retraining the
model on the newly developed data, the model forecasts inaccurate predictions
in the interval before retraining. We use an online learning approach to
overcome the limitations of the mentioned approaches.

Online learning is the practice of extracting useful information from the
newly observed data and utilizing the data to develop a more accurate model.
Figure 3 depicts an abstract overview of online learning. This method is
beneficial since the model can get better over time. Moreover, This approach
is especially helpful while forecasting server workloads due to the fast
generation of new workloads, which may contain different usage patterns.

Figure 3: Overview of the online learning approach

We utilized two optimization methods: gradient descent (GD) and Limited-memory
BFGS (L-BFGS). GD is a first-order optimization approach widely used in
machine learning optimization problems. GD works by updating the weights using
the partial derivative of the current model error to the weight being updated.
Equation 11 shows the update performed in the GD method, where $\alpha$ is the
learning rate and $\partial J(\Theta)$ shows the cost function calculated from
the model parameters $\Theta$.

$weight_{updated}=weight_{old}-\alpha\frac{\partial J(\Theta)}{\partial
weight_{old}}$ (11)

L-BFGS is a quasi-Newton family algorithm that utilizes the objective
function’s second-order derivatives (Hessians) while using limited memory. The
following update equation is used iteratively in the BFGS algorithm to
optimize the objective function:

$\begin{split}weight_{updated}=weight_{old}-H(weight_{old})^{-1}\frac{\partial
J(\Theta)}{\partial weight_{old}}\end{split}$ (12)

The L-BFGS approach approximates the calculation of inverse Hessian
$H(weight_{old})^{-1}$, which is computationally expensive.

##  IV Experimental Results

In this section, we provide the experimental results and their related
discussions. First, we give an explanation of the dataset used in our study.
Then, we provide our initial GRU model prediction results and compare the
errors with other models. Next, the pruning results and the effect on the
model’s prediction time and error rate are shown. Finally, the results are
shown after applying our online learning method.

###  IV-A Dataset and Preprocessing

We used the Google cluster traces dataset [33] for our experiments. The
dataset provides detailed information about one Borg cell, which consists of
12,583 machines. The data is collected for the cluster from May 1, 2011, and
29 days of information is recorded. Each Borg cell consists of different
machines, and the management system allocates work to them. Work arrives in
the form of jobs, consisting of several tasks. A task is the smallest form of
work, and its resource requirements are determined. The Google traces dataset
contains information about machines, jobs, and tasks, such as job events,
machine attributes, machine events, task constraints, task events, and
resource usages. We emphasize these resource usages due to their relevance to
our experiments. The resource usages are reported every five minutes in the
dataset and are sampled approximately every one second. Then, the mean of the
samples is calculated to show the average resource usage in the five-minute
interval. The dataset contains data about several resources, such as average
CPU usage rate, canonical memory usage, mean disk I/O time, and mean local
disk space used. The average CPU usage rate is measured in units of CPU-core
seconds per second. In addition, the canonical memory usage calculates the
number of user-accessible pages, including page cache and excluding stale
pages [33]. Moreover, the mean disk I/O time is measured in units of disk-time
seconds per second and shows the sum across all disks in the machine. An
overview of the dataset is provided in Table I.

TABLE I: Overview of the Google cluster traces dataset Metric | Result
---|---
Unique jobs count | 672,074
Unique machines count | 12,583
Average CPU usage rate | 0.0205
Average canonical memory usage | 0.0201
Average disk I/O time | 0.002
Average local disk space used | 8.982e-05

As discussed earlier, Google traces dataset provides resource usage
information for each task over time. On the other hand, our method is centered
on predicting machine resource usage; to create the dataset needed for this
purpose, we aggregated task usage in every period, calculating the total
machine resource usage in the period. Figure 4 depicts the first 350 total
aggregated CPU usages over time for the chosen machine. 18745 CPU usages are
available in the dataset.

Figure 4: The first 350 CPU usages in the dataset

To prepare the data for prediction, we analyzed several factors: First, we
examined the features for missing values. No missing values were found in CPU
usage rate, canonical memory usage, and local disk space; however, disk I/O
time included missing values, so we replaced these values with a linear
interpolation using existing data points. Second, feature values are
normalized between a range of zero and one to overcome model fitting problems.
We utilized a MinMax scaler for normalizing, defined in Equation 13, where $x$
is the feature being scaled.

$x_{scaled}=\frac{x-x_{min}}{x_{max}-x_{min}}$ (13)

###  IV-B Evaluation Metrics

We used mean absolute error (MAE) and root mean squared error (RMSE) to
evaluate our results and compare our model with other models. These errors are
defined as:

$MAE=\frac{1}{n}\sum_{i=1}^{n}|y_{i}-y^{\prime}_{i}|$ (14)
$RMSE=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_{i}-y^{\prime}_{i})^{2}}$ (15)

where $y_{i}$ and $y^{\prime}_{i}$ show the actual and the predicted output,
respectively.

###  IV-C GRU Training and Prediction

We used the Darts [34], PyTorch Lightning [35], and PyTorch [36] frameworks to
train our forecasting model. To develop the GRU model, we split the complete
workloads data into training subsets with 80% and 20% of the data. Next, we
used the GridSearch algorithm to find the best hyperparameters for the model.
GridSearch finds the most accurate model by comparing different combinations
of the passed parameters and the respective error rates. The size of the GRU’s
hidden layer neurons, the number of RNN layers, and the number of past steps
fed into the model were analyzed with GridSearch to develop the most accurate
model. Subsequently, we analyzed 18 models with different values for the
mentioned metrics. The size of the hidden layer neurons is selected from the
set 32, 64, the number of RNN layers is chosen from the set 1, 3, 5, and we
used 4, 8, 12 as the number of past steps used in the training process. Figure
5 depicts the error rates of the four best models after the GridSearch
process. Moreover, Table II shows the hyperparameters chosen for these models.
Model 1, which has 3 RNN layers with 64 hidden neurons in each layer,
outperforms the other models in RMSE and MAE error rates. Model 2, having 1
RNN layer with 32 hidden neurons and a previous step size of 12, similar to
Model 1, is the next best model, according to the results. Model 3 and Model 4
are the next most accurate models and have near RMSE and MAE error rates. All
these models use a three-step prediction, resulting in a 15-minute early
forecast due to the 5-minute gap between data points in the Google cluster
traces dataset.

Figure 5: The best GRU models trained with different hyperparameters TABLE II: The best GRU models’ hyperparameters Model | Hidden layer size | GRU layers | Previous steps
---|---|---|---
Model 1 | 64 | 3 | 12
Model 2 | 32 | 1 | 12
Model 3 | 32 | 1 | 4
Model 4 | 32 | 1 | 8

Due to the error rates, we used Model 1, the best RMSE and MAE model, as our
base GRU model in our research. In addition, Figure 6 shows the forecasted
workloads and the actual observations plot. According to the forecasts, the
true CPU usage rate observations are accurately predicted using the model
trained with our method.

Figure 6: The CPU usage rate forecasts using the trained GRU model with 64
cells in each hidden layer, 3 GRU layers, and a previous step of 12

To compare the GRU model’s accuracy, we trained other models using different
methods, such as random forest, ARIMA, vanilla RNN, and LSTM; these models’
RMSE and MAE error rates are provided in Figure 7. According to the results,
our model outperforms the other methods in MAE and RMSE rates. The random
forest model has the highest error rate in the results and performs worse than
other models. Moreover, the ARIMA model has a higher error rate compared with
the neural network-based models such as vanilla RNN and LSTM. The vanilla RNN
and LSTM models performed similarly regarding RMSE error rates due to their
network structure similarity. However, the RNN model had fewer MAE errors than
the LSTM model, which we think is because of the different hyperparameters
when training.

Figure 7: The RMSE and MAE errors of our model compared with the baseline
models

###  IV-D Pruning Results

We analyze different pruning approaches to increase the model speed with a
slight decay in accuracy. We compared two main methods: L1 norm structured and
random structured pruning. We started with the initial model trained with the
hyperparameters chosen from the GRU training process, and we increased the
pruning amount to observe the effect on the RMSE and MAE error rates. In
addition, we analyzed the impact of the pruning amount on the prediction
speed.

Figure 9 depicts the MAE and RMSE error rates for different pruning amounts
and methods. As seen in the results, the L1 norm structured pruning is
performing better regarding MAE and RMSE error rates. Observing the RMSE error
rate plot, we find that the 0.05 pruning amount did not affect the RMSE error
rate of the L1 norm much, and the RMSE rate is very similar to the initial
unpruned model. However, the increase has a more negative effect when the
pruning is done randomly. Moreover, The plot shows that the rise of pruning
amount to 0.1 and 0.2 has a significant error rate increase for both models.
On the other hand, speaking about the MAE errors plot, we do not see a
considerable MAE error rate increase after raising the pruning amount to 0.1
since the error is relatively the same for the 0.05 and 0.1 pruning ratios.
Similar to the RMSE results, increasing the pruning amount to 0.2 impacts the
MAE error considerably for both L1 norm and random methods.

Figure 8: The prediction time results using L1 norm and random structured
pruning methods

Figure 8 shows the prediction speed of both methods with different pruning
amounts. According to the prediction time results, a lower prediction time is
obtained by increasing the amount of pruning. However, the L1 norm and random
methods differ regarding prediction speed; the initial L1 norm pruning method
has a higher prediction time compared with the random pruning method.
Increasing the pruning amount to 0.05, we observe that the L1 norm prediction
time notably decreases, even lower than the time the random pruning method
takes at the 0.05 pruning point. By raising the pruning amount more, the
random method significantly decreases the prediction time, which is not seen
in the L1 norm results; The L1 norm method’s prediction speed remains
relatively the same after increasing the pruning amount past the 0.05 value.

Considering the error rate results and the speed-up measurements, we see that
the L1 norm method has better results regarding accuracy; however, it falls
back from the random method in terms of speed in higher pruning levels. We
find that a pruning amount of 0.05 for the L1 norm method is the best choice
since it has a good RMSE and MAE error rate, and no increased speed-up is seen
after this value according to the prediction time results. In addition, the
random structured pruning works better using a pruning amount of 0.1, which
increases the model speed significantly. Generally, the choice of pruning
amount is a trade-off between the model’s accuracy decay and speed-up gains;
Therefore, one must consider the requirements to choose the appropriate
pruning method and amount.

Figure 9: The RMSE and MAE error rate of L1 norm and random structured pruning
methods compared after pruning amount changes

###  IV-E Online Learning Results

This section describes our online learning method’s results. We used GD and
L-BFGS optimization mechanisms to analyze different optimization approaches
for adapting the model. Moreover, the online learning batch size’s effect on
the RMSE and MAE errors is analyzed. The online learning batch size is the
number of observations the model waits to be available to perform online
adaptation. Figure 11 depicts the results of the two optimization approaches
with different online learning batch sizes. Taking into account the RMSE error
results, we find that both model accuracies improve after performing online
adaptation compared with the initial model with no online learning method;
This is because online learning considers the new patterns available in the
observations, which the initial model did not. The GD method performed better
regarding error rates. Moreover, the L-BFGS approach took more time to
converge and performed poorly. Furthermore, the model error rates improve when
the online learning batch size decreases, enabling the model to learn new
patterns faster and utilize them on the following observed workloads. Both
models show the same pattern of decreases in errors with smaller batch sizes;
however, the GD method’s error rate does not change much when we decrease the
batch size from 128 to 64.

We see a similar pattern in the MAE results. GD performs better regarding the
MAE error rate. Like the RMSE results, both models’ error rates improve by
decreasing the online learning batch sizes, and the online learning models
outperform the initial model with no online learning mechanism. Again, the GD
model’s MAE error does not notably change by decreasing the batch size from
128 to 64. Even though the L-BFGS error rates are worse compared with the GD
method, the decrease in the error rate is faster when we decrease the batch
size. A smaller batch size is significantly better for the L-BFGS method and
helps it perform better.

In addition, Figure 10 shows the plot of forecasts and actual observations for
batch number 10 when online adaptation is made using a batch size of 128
observations. As we can see, the forecasts after online prediction match the
actual trends in the workloads and predict them well.

Figure 10: The plot of CPU usage forecasts for batch number 10 after online
adaptation using a batch size of 128 observations Figure 11: The RMSE and MAE
error rates of GD and L-BFGS optimization methods when online learning batch
size is changed

##  V Conclusion

Our research developed a multivariate multistep GRU neural network to forecast
workloads accurately. First, the best performing GRU model is created on the
Google traces dataset using GridSearch in the hyperparameters space. We find
that a GRU model with 3 GRU layers, 64 neurons, and 12 observations performs
best. Next, we compared the trained GRU model with several baseline models. We
observe that the trained GRU model outperforms the other baseline models.
Furthermore, we used two pruning methods: L1 norm structured and random
structured, to create a sparser model to predict workloads faster. We notice
that a higher pruning amount results in slight error rate increases and better
prediction times, which is expected. We note that the L1 norm structured
method performs better regarding prediction accuracies, and the random method
predicts faster in higher pruning amounts. Finally, we analyzed the online
adaptation of the created model using two optimization techniques: GD and
L-BFGS. After changing the online learning batch size, we compared the RMSE
and MAE results. The results show better performances after decreasing the
batch size. Also, we observe that GD performs better in both RMSE and MAE
error rates. In conclusion, our approach enhances the accuracy and speed of
prediction using a multivariate multistep GRU model in addition to pruning and
online adaptation.

For future work, the following can be done:

  * •

Analysis of the effect of the prediction horizon on the workload forecast
accuracy and scheduling decisions

  * •

Analyses of the pruning speed achievements in a real-time cloud system

  * •

The comparison of online adaptation frameworks in a real-time workload system
regarding accuracy enhancement

## References

  * [1] “Fact.mr – cloud computing market analysis by deployment (public, private, hybrid cloud computing), by service (infrastructure as a service (iaas), platform as a service (paas), software as a service (saas)), by enterprise size, by end use & regional forecast 2022-2032.” [Online]. Available: https://bit.ly/3Tk58Oz
  * [2] L. M. Vaquero, L. Rodero-Merino, J. Caceres, and M. Lindner, “A break in the clouds: towards a cloud definition,” pp. 50–55, 2008.
  * [3] B. Song, Y. Yu, Y. Zhou, Z. Wang, and S. Du, “Host load prediction with long short-term memory in cloud computing,” _The Journal of Supercomputing_ , vol. 74, no. 12, pp. 6554–6568, 2018.
  * [4] A. Validi, V. Kashansky, J. Khiari, H. Hadian, R. Prodan, J. Li, F.-Y. Wang, and C. Olaverri-Monreal, “Hybrid on/off blockchain approach for vehicle data management, processing and visualization exemplified by the adapt platform,” in _2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)_. IEEE, 2022, pp. 3152–3158.
  * [5] S. Gupta and D. A. Dinesh, “Online adaptation models for resource usage prediction in cloud network,” in _2017 Twenty-third National Conference on Communications (NCC)_. IEEE, 2017, pp. 1–6.
  * [6] T. Kaur and I. Chana, “Energy efficiency techniques in cloud computing: A survey and taxonomy,” _ACM computing surveys (CSUR)_ , vol. 48, no. 2, pp. 1–46, 2015.
  * [7] M. Farrokh, H. Hadian, M. Sharifi, and A. Jafari, “Sp-ant: An ant colony optimization based operator scheduler for high performance distributed stream processing on heterogeneous clusters,” _Expert Systems with Applications_ , vol. 191, p. 116322, 2022.
  * [8] V. R. Messias, J. C. Estrella, R. Ehlers, M. J. Santana, R. C. Santana, and S. Reiff-Marganiec, “Combining time series prediction models using genetic algorithm to autoscaling web applications hosted in the cloud infrastructure,” _Neural Computing and Applications_ , vol. 27, no. 8, pp. 2383–2406, 2016.
  * [9] S. Gupta and D. A. Dinesh, “Resource usage prediction of cloud workloads using deep bidirectional long short term memory networks,” in _2017 IEEE international conference on advanced networks and telecommunications systems (ANTS)_. IEEE, 2017, pp. 1–6.
  * [10] C. Sudhakar, A. R. Kumar, N. Siddartha, and S. V. Reddy, “Workload prediction using arima statistical model and long short-term memory recurrent neural networks,” in _2018 International Conference on Computing, Power and Communication Technologies (GUCON)_. IEEE, 2018, pp. 600–604.
  * [11] K. Surya and V. Rajam, “Prediction of resource contention in cloud using second order markov model,” _Computing_ , vol. 103, no. 10, pp. 2339–2360, 2021.
  * [12] H. Hadian, M. Farrokh, M. Sharifi, and A. Jafari, “An elastic and traffic-aware scheduler for distributed data stream processing in heterogeneous clusters,” _The Journal of Supercomputing_ , vol. 79, no. 1, pp. 461–498, 2023.
  * [13] M. Xu, C. Song, H. Wu, S. S. Gill, K. Ye, and C. Xu, “esdnn: Deep neural network based multivariate workload prediction in cloud computing environments,” _ACM Transactions on Internet Technology (TOIT)_ , 2022.
  * [14] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” _Neural Computation_ , vol. 9, no. 8, pp. 1735–1780, 11 1997. [Online]. Available: https://doi.org/10.1162/neco.1997.9.8.1735
  * [15] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” _arXiv preprint arXiv:1412.3555_ , 2014.
  * [16] Y. Cheng, C. Wang, H. Yu, Y. Hu, and X. Zhou, “Gru-es: Resource usage prediction of cloud workloads using a novel hybrid method,” in _2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS)_. IEEE, 2019, pp. 1249–1256.
  * [17] S. Gupta, A. D. Dileep, and T. A. Gonsalves, “Online sparse blstm models for resource usage prediction in cloud datacentres,” _IEEE Transactions on Network and Service Management_ , vol. 17, no. 4, pp. 2335–2349, 2020.
  * [18] B. Liu, Y. Lin, and Y. Chen, “Quantitative workload analysis and prediction using google cluster traces,” in _2016 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)_. IEEE, 2016, pp. 935–940.
  * [19] R. N. Calheiros, E. Masoumi, R. Ranjan, and R. Buyya, “Workload prediction using arima model and its impact on cloud applications’ qos,” _IEEE transactions on cloud computing_ , vol. 3, no. 4, pp. 449–458, 2014.
  * [20] A. S. Kumar and S. Mazumdar, “Forecasting hpc workload using arma models and ssa,” in _2016 International Conference on Information Technology (ICIT)_. IEEE, 2016, pp. 294–297.
  * [21] W. Zhong, Y. Zhuang, J. Sun, and J. Gu, “A load prediction model for cloud computing using pso-based weighted wavelet support vector machine,” _Applied Intelligence_ , vol. 48, no. 11, pp. 4072–4083, 2018.
  * [22] A. Y. Nikravesh, S. A. Ajila, and C.-H. Lung, “Towards an autonomic auto-scaling prediction system for cloud resource provisioning,” in _2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems_. IEEE, 2015, pp. 35–45.
  * [23] R. Marcus and O. Papaemmanouil, “Workload management for cloud databases via machine learning,” in _2016 IEEE 32nd International Conference on Data Engineering Workshops (ICDEW)_. IEEE, 2016, pp. 27–30.
  * [24] R. Yang, X. Ouyang, Y. Chen, P. Townend, and J. Xu, “Intelligent resource scheduling at scale: a machine learning perspective,” in _2018 IEEE symposium on service-oriented system engineering (SOSE)_. IEEE, 2018, pp. 132–141.
  * [25] K. Cetinski and M. B. Juric, “Ame-wpc: Advanced model for efficient workload prediction in the cloud,” _Journal of Network and Computer Applications_ , vol. 55, pp. 191–201, 2015.
  * [26] J. Kumar, R. Goomer, and A. K. Singh, “Long short term memory recurrent neural network (lstm-rnn) based workload forecasting model for cloud datacenters,” _Procedia Computer Science_ , vol. 125, pp. 676–682, 2018, the 6th International Conference on Smart Computing and Communications. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1877050917328557
  * [27] S. Ouhame, Y. Hadi, and A. Ullah, “An efficient forecasting approach for resource utilization in cloud data center using cnn-lstm model,” _Neural Computing and Applications_ , vol. 33, no. 16, pp. 10 043–10 055, 2021.
  * [28] N.-M. Dang-Quang and M. Yoo, “An efficient multivariate autoscaling framework using bi-lstm for cloud computing,” _Applied Sciences_ , vol. 12, no. 7, p. 3523, 2022.
  * [29] J. Kumar, A. K. Singh, and R. Buyya, “Self directed learning based workload forecasting model for cloud resource management,” _Information Sciences_ , vol. 543, pp. 345–366, 2021.
  * [30] M. Amiri, L. Mohammad-Khanli, and R. Mirandola, “An online learning model based on episode mining for workload prediction in cloud,” _Future Generation Computer Systems_ , vol. 87, pp. 83–101, 2018.
  * [31] C. Liu, S. C. Hoi, P. Zhao, and J. Sun, “Online arima algorithms for time series prediction,” in _Thirtieth AAAI conference on artificial intelligence_ , 2016.
  * [32] K. Cho, B. Van Merriënboer, D. Bahdanau, and Y. Bengio, “On the properties of neural machine translation: Encoder-decoder approaches,” _arXiv preprint arXiv:1409.1259_ , 2014.
  * [33] C. Reiss, J. Wilkes, and J. L. Hellerstein, “Google cluster-usage traces: format+ schema,” _Google Inc., White Paper_ , vol. 1, 2011.
  * [34] J. Herzen, F. LÃ¤ssig, S. G. Piazzetta, T. Neuer, L. Tafti, G. Raille, T. V. Pottelbergh, M. Pasieka, A. Skrodzki, N. Huguenin, M. Dumonal, J. KoÅcisz, D. Bader, F. Gusset, M. Benheddi, C. Williamson, M. Kosinski, M. Petrik, and G. Grosch, “Darts: User-friendly modern machine learning for time series,” _Journal of Machine Learning Research_ , vol. 23, no. 124, pp. 1–6, 2022\. [Online]. Available: http://jmlr.org/papers/v23/21-1177.html
  * [35] W. Falcon, J. Borovec, A. Wälchli, N. Eggert, J. Schock, J. Jordan, N. Skafte, V. Bereznyuk, E. Harris, T. Murrell _et al._ , “Pytorchlightning/pytorch-lightning: 0.7. 6 release,” _Zenodo: Geneva, Switzerland_ , 2020.
  * [36] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in _Advances in Neural Information Processing Systems 32_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds. Curran Associates, Inc., 2019, pp. 8024–8035. [Online]. Available: http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf


# A further generalisation of sums of higher derivatives of the Riemann Zeta
Function

Andrew Pearce-Crump  Department of Mathematics, University of York, York, YO10
5DD, United Kingdom<EMAIL_ADDRESS>
###### Abstract.

We prove an asymptotic for the sum of $\zeta^{(n)}(\rho)X^{\rho}$ where
$\zeta^{(n)}(s)$ denotes the $n$th derivative of the Riemann zeta function,
$X$ is a positive real and $\rho$ denotes a non-trivial zero of the Riemann
zeta function. The sum is over the zeros with imaginary parts up to a height
$T$, as $T\rightarrow\infty$.

We also specify what the asymptotic formula becomes when $X$ is a positive
integer, highlighting the differences in the asymptotic expansions as $X$
changes its arithmetic nature.

##  1\. Introduction

Let $\zeta(s)$ denote the Riemann zeta function and let $\rho=\beta+i\gamma$
be a non-trivial zero of $\zeta(s)$. The starting point for the motivation of
this paper is Landau’s theorem [14] which states that for any $X>1$ we have

$\sum_{0<\gamma\leq T}X^{\rho}=-\frac{T}{2\pi}\Lambda(X)+O(\log T).$ (1.1)

Here and throughout this article, $\Lambda(X)$ denotes the von Mangoldt
function given by

$\Lambda(X)=\begin{cases}\log p&\text{ if $X=p^{k}$ for some prime $p$ and
some integer $k\geq 1$}\\\ 0&\text{ otherwise}.\end{cases}$ (1.2)

Formula (1.1) is proven by estimating the integral

$\frac{1}{2\pi i}\int_{R}\frac{\zeta^{\prime}}{\zeta}(s)X^{s}\ ds$

for a suitably chosen rectangle enclosing those zeros $\rho$ for which
$0<\gamma\leq T$.

Gonek [8] has generalised Landau’s formula to be uniform in both variables. He
showed that for $X,T>1$,

$\displaystyle\sum_{0<\gamma\leq T}X^{\rho}=$
$\displaystyle-\frac{T}{2\pi}\Lambda(X)+O(X\log(2XT)\log\log(3X))$
$\displaystyle+O\left(\log X\min\left(T,\frac{X}{\langle
X\rangle}\right)\right)+O\left(\log T\min\left(T,\frac{1}{\log
X}\right)\right)$ (1.3)

where $\langle X\rangle$ denotes the distance from $X$ to the nearest prime
power other than $X$ itself. Note then that (1.1) follows from (1.3) provided
that $X$ is fixed and $T\rightarrow\infty$.

The large number of error terms are explained in Gonek [8] which are all due
to the different behaviour of the sum in different $X$ ranges. He then also
notes that when $X=n\in\mathbb{N}$ and $T\gg n$, then the last two error terms
of (1.3) are absorbed by the first error term and so (1.3) becomes

$\sum_{0<\gamma\leq
T}n^{\rho}=-\frac{T}{2\pi}\Lambda(n)+O(n\log(2nT)\log\log(3n)).$ (1.4)

The differences in formula (1.3) and formula (1.4) highlights the differences
in behaviour of sums of this form when $X$ is no longer an arbitrary positive
real but is instead a positive integer, a theme that will reoccur several
times in this paper.

Fujii has generalised Gonek’s result (assuming RH) in [3] and [4] to find the
sub-leading and sub-sub-leading behaviour. This is given by

$\sum_{0<\gamma\leq
T}X^{\frac{1}{2}+i\gamma}=-\frac{T}{2\pi}\Lambda(X)+\frac{X^{\frac{1}{2}+iT}\log\left(\frac{T}{2\pi}\right)}{2\pi
i\log
X}+X^{\frac{1}{2}+iT}\frac{1}{\pi}\arg\zeta\left(\frac{1}{2}+iT\right)+O\left(\frac{\log
T}{(\log\log T)^{2}}\right).$ (1.5)

Fujii has also shown in [5] that

$\sum_{0<\gamma\leq
T}\zeta^{\prime}(\rho)=\frac{T}{4\pi}\log^{2}\left(\frac{T}{2\pi}\right)+(-1+C_{0})\frac{T}{2\pi}\log\left(\frac{T}{2\pi}\right)+(1-C_{0}-C_{0}^{2}+3C_{1})\frac{T}{2\pi}+E(T)$
(1.6)

where $E(T)$ is given explicitly both unconditionally and assuming RH and
where $C_{0}$ and $C_{1}$ are the constants in the expansion

$\zeta(s)=\frac{1}{s-1}+C_{0}+C_{1}(s-1)+....$

Fujii forms a hybrid of the sums (1.1) and (1.6) in his paper [6]. He gives
the following explicit formula for a fixed positive real number $X$,

$\displaystyle\sum_{0<\gamma\leq T}\zeta^{\prime}(\rho)X^{\rho}$
$\displaystyle=-\Delta(X)\frac{T}{2\pi}\Bigg{(}\log
X\left(\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)-\sum_{mr=X}\Lambda(r)\log m\Bigg{)}$
$\displaystyle+X\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi
imX}\log^{2}m+\frac{1}{2}X\log X\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}\log
m$ $\displaystyle-\left(\frac{1}{2}X\log^{2}X-\frac{\pi i}{4}X\log
X\right)\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}-X\sum_{mr\leq\frac{T}{2\pi
X}}e^{2\pi imrX}\Lambda(r)\log m$ $\displaystyle+O(\sqrt{T}\log^{3}T)$ (1.7)

where $\Delta(X)$ is defined by

$\Delta(X)=\begin{cases}1&\text{ if $n\in\mathbb{Z}$}\\\ 0&\text{
otherwise}.\\\ \end{cases}$ (1.8)

throughout this article.

Fujii then narrows down $X$ to be a positive integer and gives an asymptotic
formula for this, again exhibiting the behaviour that Gonek noted in [8],
about how the arithmetic nature of $X$ can affect the asymptotic expansion.
Specifically, he finds for $X\geq 1$, $X\in\mathbb{Z}$ that

$\displaystyle\sum_{0<\gamma\leq T}\zeta^{\prime}(\rho)X^{\rho}$
$\displaystyle=\frac{T}{4\pi}\log^{2}\left(\frac{T}{2\pi}\right)+(-1+C_{0}-\log
X)\frac{T}{2\pi}\log\left(\frac{T}{2\pi}\right)$
$\displaystyle+\frac{T}{2\pi}\left(1-C_{0}-C_{0}^{2}+3C_{1}+\sum_{X=mn}\Lambda(n)\log
m-\left(C_{0}-1+\frac{1}{2}\log X\right)\log X\right)$
$\displaystyle+O(Te^{-C\sqrt{\log T}})$ (1.9)

where $C_{0},C_{1}$ are given above and $C$ is a positive constant. Clearly
setting $X=1$ then gives (1.6).

A generalisation of Fujii’s result has been given in Jakhlouti and Mazhouda
[12] to give an analogue of (1.7) for Dirichlet $L$-functions. This extension
is taken further to an asymptotic formula at the $L$-function’s $a$-points,
for any fixed complex number $a$. We state the analogue to (1.7) which is
their Lemma 3 with $a=0$,

$\sum_{0<\gamma_{\chi}\leq T}L^{\prime}(\rho_{\chi},\chi)X^{\rho_{\chi}}=\\\
-\Delta(X)\chi[\Delta(X)X]\frac{T}{2\pi}\Bigg{(}\log
X\left(\frac{1}{2}\log\left(\frac{qT}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)-\sum_{mr=X}\Lambda(r)\log m\Bigg{)}\\\
+\frac{X}{\sqrt{q}}\sum_{m\leq\frac{qt}{2\pi X}}e^{2\pi
im\frac{X}{q}}\overline{\chi}(m)\log^{2}m+\frac{1}{2\sqrt{q}}X\log
X\sum_{m\leq\frac{qt}{2\pi X}}e^{2\pi im\frac{X}{q}}\overline{\chi}(m)\log
m\\\ -\left(\frac{1}{2\sqrt{q}}X\log^{2}X-\frac{\pi i}{4\sqrt{q}}X\log
X\right)\sum_{m\leq\frac{qt}{2\pi X}}e^{2\pi
im\frac{X}{q}}\overline{\chi}(m)\\\
-\frac{X}{\sqrt{q}}\sum_{mr\leq\frac{qt}{2\pi X}}e^{2\pi
imr\frac{X}{q}}\Lambda(r)\overline{\chi}(r)\overline{\chi}(m)\log
m+O(\sqrt{T}\log^{3}T)$

as $T\rightarrow\infty$ for a fixed positive number $X$ and where $\chi$ is a
primitive character mod $q$. Fixing $q=1$ gives (1.7) and setting $q=1$ and
$X=1$ gives (1.6).

Other ideas have grown from (1.1). For example, Ford and Zaharescu [2] start
with Landau’s theorem (1.1) and investigate the distribution of the fractional
parts of $\alpha\gamma$, where $\alpha$ is a fixed non-zero real number. This
idea is then expanded upon by Ford, Soundararajan and Zaharescu in [1].
Further examples of results that start with (1.1) are found in papers [9] and
[13].

##  2\. Statement of the Results

Let $X$ be a fixed positive number. Let $\zeta^{({n})}(s)$ denote the $n$th
derivative of the Riemann zeta function $\zeta(s)$ and let
$\rho=\beta+i\gamma$ be a non-trivial zero of $\zeta(s)$. Write $s=\sigma+it$
with $\sigma,t\in\mathbb{R}$. We suppose that $T>T_{0}$ and that $T$ is not
the imaginary part of the zeros of $\zeta(s)$. We further assume that
$|T-\gamma|\gg\frac{1}{\log T}$. This restriction has no effect on the final
result. Then we have the following result which is an analogue of Fujii’s
(1.7).

######  Theorem 1.

For $X$ a fixed positive real number, we have

$\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}=\\\
(-1)^{n}\left\\{\Delta(X)\frac{T}{2\pi}\left(\log^{n}X\left(\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)-\sum_{mr=X}\Lambda(r)\log^{n}m\right)\right.\\\
+X\log^{n}X\left(\frac{1}{2}\log X-\frac{\pi
i}{4}\right)\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi
imX}+\frac{X}{2}\log^{n}X\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}\log m\\\
-X\sum_{mr\leq\frac{T}{2\pi X}}e^{2\pi
imrX}\Lambda(r)\log^{n}(rX)\Bigg{\\}}+O(\sqrt{T}\log^{n+2}T),$

where

$\Delta(X)=\begin{cases}1&\text{ if $n\in\mathbb{Z}$}\\\ 0&\text{
otherwise}.\\\ \end{cases}$

######  Remark.

Setting $n=1$ in the theorem recovers Fujii’s result in (1.7).

When we restrict $X$ to being a positive integer we obtain a special case of
the above results. It is evident from the statement of the following corollary
that we see the changing behaviour of our asymptotic expansions depending on
whether $X>0$ is any real number and when $X\geq 1$ is an integer.

######  Corollary 1.

If $X\geq 1$, $X\in\mathbb{Z}$, then

$\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}=\\\
(-1)^{n+1}\frac{T}{2\pi}\log^{n}X\Bigg{\\{}\sum_{k=0}^{n}\sum_{u=0}^{k+1}{n\choose
k}{{k+1}\choose
u}(-1)^{u}\frac{1}{k+1}\log^{k+1-u}\left(\frac{T}{2\pi}\right)\log^{u-k}X\\\
+\sum_{k=0}^{n}\sum_{l=0}^{k}\sum_{u=0}^{k-l}{n\choose k}{k\choose
l}{{k-l}\choose
u}(-1)^{l}l!\left(-1+\sum_{j=0}^{l}(-1)^{j}C_{j}\right)\log^{k-l-u}\left(\frac{T}{2\pi}\right)\log^{u-k}X\\\
+\sum_{k=0}^{n}{n\choose
k}(-1)^{k+1}k!A_{k}\log^{-k}X-\left(\log\left(\frac{T}{2\pi}\right)-1-\sum_{mr=X}\Lambda(r)\log^{n}m\right)\Bigg{\\}}+E_{n}(T).$

where

$E_{n}(T)=O\left(T\mathrm{e}^{-C\sqrt{\log T}}\right)$

with $C$ is a positive constant. If we assume the Riemann Hypothesis, then

$E_{n}(T)=O\left(T^{\frac{1}{2}}\log^{n+2}T\right).$

Further, the $C_{j}$ are the coefficients in the Laurent expansion for
$\zeta(s)$ about $s=1$, given by

$\zeta(s)=\frac{1}{s-1}+\sum_{j=0}^{\infty}C_{j}(s-1)^{j}$

and the $A_{j}$ are the coefficients in the Laurent expansion for
$\frac{\zeta^{\prime}(s)}{\zeta(s)}$ about $s=1$, given by

$\frac{\zeta^{\prime}(s)}{\zeta(s)}=-\frac{1}{s-1}+\sum_{j=0}^{\infty}A_{j}(s-1)^{j}.$

######  Remark.

Note that the $A_{j}$ are related to the $C_{j}$ by the following recursive
formula, as shown in Israilov [11], given by

$A_{j}=\begin{cases}C_{0}&\text{ if $j=0$}\\\
(j+1)C_{j}-\sum_{k=0}^{j-1}A_{k}C_{j-1-k}&\text{ if $j\geq 1$}.\\\
\end{cases}$

######  Remark.

Setting $n=1$ in the corollary recovers Fujii’s result in (1.9). Setting $n=1$
and $X=1$ recovers Fujii’s result in (1.6). Setting $X=1$ recovers our result
from [10] for general $n$, given later on in this paper as Theorem 2.

##  3\. Outline of the Paper

So far we have described the motivation for studying asymptotic expansions of
the sums given in Section 1. We have stated the main results in Section 2 that
we will prove in the following sections.

In Section 4 we will recall some basic facts about the Riemann zeta function
$\zeta(s)$ and the Riemann xi function $\xi(s)$. One of the main results that
we will need to state is the Theorem from Hughes and Pearce-Crump [10] that
will be essential in proving Corollary 1.

In Section 5 we will use the tools from Section 4 to prove Theorem 1. The
integral we use to prove this theorem is given by

$I=\frac{1}{2\pi i}\int_{R}\frac{\xi^{\prime}}{\xi}(s)\zeta^{(n)}(s)X^{s}\ ds$
(3.1)

where $\xi(s)$ is the Riemann xi function, $\zeta^{(n)}(s)$ denotes the $n$th
derivative of $\zeta(s)$ and R denotes the rectangular positively oriented
contour with the vertices are $c+i,c+iT,1-c+it,1-c+i$ connected in this order
and $c=1+\frac{1}{\log T}$. The non-trivial zeros of $\zeta(s)$ up to a height
$T$ are contained within $R$ and so by Cauchy’s Theorem the integral
represents the summation

$I=\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}.$ (3.2)

We split this section up into several subsections, each corresponding to
different parts of the contour that we will be integrating over, to show that
most of the contribution comes from the left-hand side of the contour, while
the other sides mostly only contribute to the error term.

Finally in Section 6 we will prove Corollary 1 which will highlight the
differences between the general case proved in Section 5 for any positive
number $X$ and the case when $X$ is a positive integer. As described above we
will use a result from [10] to do most of the work here. This will again
highlight the observation of Gonek’s in [8] that the asymptotic formulae tend
to change quite dramatically depending on the arithmetic nature of $X$.

##  4\. Preliminary Lemmas

In this section we recall some basic information about $\zeta(s)$ and
$\xi(s)$, as well as recalling some results from other papers that will be
useful in our proof. Any facts that are not explicitly referenced in this
section can be found in any good text about the Riemann zeta function, for
example they can be found in Titchmarsh [16].

Firstly recall that the functional equation for $\zeta(s)$ is given by

$\zeta(s)=\chi(s)\zeta(1-s)$

where

$\chi(s)=2^{s}\pi^{s-1}\sin\left(\frac{\pi s}{2}\right)\Gamma(1-s)$

and $\Gamma(s)$ is the Gamma function throughout this paper. We state a more
general functional equation for $\zeta^{(n)}(s)$ that is proved using the
functional equation for $\zeta(s)$ and the Leibniz product rule.

######  Lemma 1.

The general functional equation for $\zeta^{(n)}(s)$ is given by the following
formula

$\zeta^{(n)}(s)=\frac{1}{\chi(1-s)}\sum_{k=0}^{n}\binom{n}{k}(-1)^{k}\frac{\chi^{(n-k)}(s)}{\chi(s)}\zeta^{(k)}(1-s).$
(4.1)

We now recall that for $\sigma>1$ we may write both $\zeta^{(n)}(s)$ and
$\frac{\zeta^{\prime}}{\zeta}(s)$ in terms of their Dirichlet series which are
given by

$\zeta^{(n)}(s)=(-1)^{n}\sum_{m=1}^{\infty}\frac{\log^{n}m}{m^{s}}$ (4.2)

and

$\frac{\zeta^{\prime}}{\zeta}(s)=-\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{s}},$
(4.3)

where $\Lambda(r)$ is the von Mangoldt function defined in (1.2).

We will also need the following result that we proved in [10, Sect.5] for the
integral along the top and the bottom of our contour.

######  Lemma 2.

For $c=1+\frac{1}{\log T}$, we have

$\int_{1-c}^{c}|\zeta^{(n)}(\sigma+iT)|\ d\sigma\ll T^{\frac{1}{2}}\log^{n}T.$

For the Riemann xi function we write

$\xi(s)=\frac{1}{2}s(s-1)\pi^{-\frac{s}{2}}\Gamma\left(\frac{s}{2}\right)\zeta(s)$
(4.4)

so the functional equation for $\xi(s)$ is given by

$\xi(s)=\xi(1-s).$

We now observe that

$\frac{\xi^{\prime}}{\xi}(s)=\frac{2s-1}{s(s-1)}-\frac{1}{2}\log\pi+\frac{1}{2}\psi\left(\frac{s}{2}\right)+\frac{\zeta^{\prime}}{\zeta}(s)$
(4.5)

where we have written

$\psi(s)=\frac{\Gamma^{\prime}}{\Gamma}(s)$

and for $|\arg s|<\pi-\delta$ with arbitrarily fixed positive $\delta$ and for
$|s|\geq\frac{1}{2}$ we have

$\psi(s)=\log s+O\left(\frac{1}{|s|}\right)=\log t+\frac{\pi
i}{2}+O\left(\frac{1}{t}\right)$

as $t\rightarrow\infty$.

Combining these two observations, we have the following Lemma.

######  Lemma 3.

With the conditions written above, we have

$\frac{\xi^{\prime}}{\xi}(s)=\frac{1}{2}\log\left(\frac{t}{2\pi}\right)+\frac{\pi
i}{4}+\frac{\zeta^{\prime}}{\zeta}(s)+O\left(\frac{1}{t}\right).$ (4.6)

Finally to prove Corollary 1 we will need the main result from [10] which we
state in fullness here for ease of reference.

######  Theorem 2.

With the setup as stated in our main results Section 2 and with the $C_{j}$
and $A_{j}$ coefficients defined in Corollary 1, we have

$\displaystyle\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)$
$\displaystyle=(-1)^{n+1}\frac{1}{n+1}\frac{T}{2\pi}\log^{n+1}\left(\frac{T}{2\pi}\right)$
$\displaystyle+(-1)^{n+1}\sum_{k=0}^{n}\binom{n}{k}(-1)^{k}k!\left(-1+\sum_{j=0}^{k}(-1)^{j}C_{j}\right)\frac{T}{2\pi}\log^{n-k}\left(\frac{T}{2\pi}\right)$
$\displaystyle+n!A_{n}\frac{T}{2\pi}+E_{n}(T)$

where unconditionally,

$E_{n}(T)=O\left(T\mathrm{e}^{-C\sqrt{\log T}}\right)$

with $C$ is a positive constant. If we assume the Riemann Hypothesis, then

$E_{n}(T)=O\left(T^{\frac{1}{2}}\log^{n+2}T\right).$

##  5\. Proof of Theorem 1

Let $X$ be a fixed positive real. We write $s=\sigma+it$ with
$\sigma,t\in\mathbb{R}$ and $\rho=\beta+i\gamma$ for a non-trivial zero of the
Riemann zeta function $\zeta(s)$. Suppose $T>T_{0}$ and $T$ is not the
imaginary part of the zeros of $\zeta(s)$ and further that
$|T-\gamma|\gg\frac{1}{\log T}$ where $\gamma$ is the imaginary part of any
zero $\rho$. This restriction on $T$ is harmless within our remainder terms.

Set $c=1+\frac{1}{\log T}$ and consider the integral

$I=\frac{1}{2\pi i}\int_{R}\frac{\xi^{\prime}}{\xi}(s)\zeta^{(n)}(s)X^{s}\ ds$
(5.1)

where $\xi(s)$ is the Riemann xi function, $\zeta^{(n)}(s)$ denotes the $n$th
derivative of $\zeta(s)$ and R denotes the rectangular positively oriented
contour with vertices given by $c+i,c+iT,1-c+it,1-c+i$, connected in this
order.

By Cauchy’s Theorem,

$I=\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}.$ (5.2)

We now need to evaluate $I$ in another way to obtain our asymptotic expansion.
Decomposing the integral (5.1) along the sides of the contour as

$\displaystyle I$ $\displaystyle=\frac{1}{2\pi
i}\left(\int_{1-c+i}^{c+i}+\int_{c+i}^{c+iT}+\int_{c+iT}^{1-c+iT}+\int_{1-c+iT}^{1-c+i}\right)\frac{\xi^{\prime}}{\xi}(s)\zeta^{(n)}(s)X^{s}\
ds$ $\displaystyle=I_{B}+I_{R}+I_{T}+I_{L}.$

###  5.1. Bounding $I_{B}$ and $I_{T}$

Notice that by our choice of $T$ we may bound $I_{B}$ and $I_{T}$ trivially
within our error. To do this, recall that for $-1\leq\sigma\leq 2$ and with
our general assumptions we have from Gonek [7, Sect.2, p.127] that

$\frac{\xi^{\prime}}{\xi}(s)\ll\log^{2}T.$

By Lemma 2, we have

$I_{B}+I_{T}\ll\log^{2}T\int_{1-c}^{c}|\zeta^{(n)}(\sigma+it)X^{\sigma+it}|\
d\sigma\ll\sqrt{T}\log^{n+2}T.$ (5.3)

###  5.2. Evaluating $I_{R}$

Writing $s=c+it$ and using (4.5) we may write

$\displaystyle I_{R}$
$\displaystyle=\frac{1}{2\pi}\int_{1}^{T}\frac{\xi^{\prime}}{\xi}(c+it)\zeta^{(n)}(c+it)X^{c+it}\
dt$
$\displaystyle=\frac{1}{2\pi}\int_{1}^{T}\left(\frac{2(c+it)-1}{(c+it)(c+it-1)}-\frac{1}{2}\log\pi+\psi\left(\frac{c+it}{2}\right)+\frac{\zeta^{\prime}}{\zeta}(c+it)\right)\zeta^{(n)}(c+it)X^{c+it}\
dt$

Using the Dirichlet series (4.2),(4.3) and Lemma 3 we may rewrite this as

$\displaystyle I_{R}$
$\displaystyle=\frac{(-1)^{n}}{2\pi}\int_{1}^{T}\left(\frac{1}{2}\log\left(\frac{t}{2\pi}\right)+\frac{\pi
i}{4}-\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{c+it}}+O\left(\frac{1}{t}\right)\right)\sum_{m=1}^{\infty}\frac{\log^{n}m}{m^{c+it}}X^{c+it}\
dt$
$\displaystyle=\frac{(-1)^{n}}{2\pi}X^{c}\sum_{m=1}^{\infty}\frac{\log^{n}m}{m^{c}}\int_{1}^{T}\left(\frac{1}{2}\log\left(\frac{t}{2\pi}\right)+\frac{\pi
i}{4}+O\left(\frac{1}{t}\right)\right)\left(\frac{X}{m}\right)^{it}\ dt$
$\displaystyle\qquad+\frac{(-1)^{n+1}}{2\pi}X^{c}\sum_{m=1}^{\infty}\frac{\log^{n}m}{m^{c}}\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{c}}\int_{1}^{T}\left(\frac{X}{mr}\right)^{it}\
dt$ $\displaystyle=I_{R,1}+I_{R,2}.$

Consider $I_{R,1}$ first. We have (with $\Delta(X)$ defined as in (1.8)),

$\displaystyle I_{R,1}$
$\displaystyle=\frac{(-1)^{n}}{2\pi}X^{c}\sum_{m=1}^{\infty}\frac{\log^{n}m}{m^{c}}\int_{1}^{T}\left(\frac{1}{2}\log\left(\frac{t}{2\pi}\right)\
+\frac{\pi
i}{4}+O\left(\frac{1}{t}\right)\right)\left(\frac{X}{m}\right)^{it}\ dt$
$\displaystyle=\frac{(-1)^{n}}{2\pi}\Delta(X)\log^{n}X\int_{1}^{T}\left(\frac{1}{2}\log\left(\frac{t}{2\pi}\right)\
+\frac{\pi i}{4}\right)\ dt$
$\displaystyle\quad+\frac{(-1)^{n}}{2\pi}X^{c}\sum_{\begin{subarray}{c}m=1\\\
m\neq
X\end{subarray}}^{\infty}\frac{\log^{n}m}{m^{c}}\int_{1}^{T}\left(\frac{1}{2}\log\left(\frac{t}{2\pi}\right)\
+\frac{\pi i}{4}\right)\left(\frac{X}{m}\right)^{it}\ dt$
$\displaystyle\qquad+O(\log^{n+2}T)$
$\displaystyle=I_{R,1,1}+I_{R,1,2}+O(\log^{n+2}T).$

Firstly,

$I_{R,1,1}=(-1)^{n}\Delta(X)\log^{n}X\frac{T}{2\pi}\left(\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)+O(1).$

Next, integrating by parts and summing we obtain

$\displaystyle I_{R,1,2}$ $\displaystyle\ll
X^{c}\sum_{\begin{subarray}{c}m=1\\\ m\neq
X\end{subarray}}^{\infty}\frac{\log^{n}m}{m^{c}}\frac{\log
T}{\log\left|\frac{X}{m}\right|}\ll\log^{n+2}T$

If $0<X<1$ we can do slightly better than this error term. Recombining, we
have

$I_{R,1}=(-1)^{n}\Delta(X)\log^{n}X\frac{T}{2\pi}\left(\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)+O(\log^{n+2}T).$ (5.4)

Now consider $I_{R,2}$. We have

$\displaystyle I_{R,2}$
$\displaystyle=\frac{(-1)^{n+1}}{2\pi}X^{c}\sum_{m=1}^{\infty}\frac{\log^{n}m}{m^{c}}\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{c}}\int_{1}^{T}\left(\frac{X}{mr}\right)^{it}\
dt$
$\displaystyle=\frac{(-1)^{n+1}}{2\pi}X^{c}\sum_{k=1}^{\infty}\frac{1}{k^{c}}\sum_{k=mr}\log^{n}m\
\Lambda(r)\int_{1}^{T}\left(\frac{X}{k}\right)^{it}\ dt$
$\displaystyle=\frac{(-1)^{n+1}}{2\pi}\Delta(X)\sum_{X=mr}\log^{n}m\
\Lambda(r)\int_{1}^{T}1\ dt$
$\displaystyle\quad+\frac{(-1)^{n+1}}{2\pi}X^{c}\sum_{\begin{subarray}{c}k=1\\\
k\neq X\end{subarray}}^{\infty}\frac{1}{k^{c}}\sum_{k=mr}\log^{n}m\
\Lambda(r)\int_{1}^{T}\left(\frac{X}{k}\right)^{it}\ dt$
$\displaystyle=I_{R,2,1}+I_{R,2,2}.$

Then

$I_{R,2,1}=(-1)^{n+1}\Delta(X)\frac{T}{2\pi}\sum_{X=mr}\log^{n}m\
\Lambda(r)+O(1)$

and as with the case for $I_{R,1,2}$ above, we have

$\displaystyle I_{R,2,2}$ $\displaystyle\ll
X^{c}\sum_{\begin{subarray}{c}k=1\\\ k\neq
X\end{subarray}}^{\infty}\frac{1}{k^{c}}\sum_{k=mr}\log^{n}m\
\Lambda(r)\frac{1}{\log|X/k|}\ll\log^{n+2}T$

where again the error can be improved slightly for $0<X<1$. Combining, we have

$I_{R,2}=(-1)^{n+1}\Delta(X)\frac{T}{2\pi}\sum_{X=mr}\log^{n}m\
\Lambda(r)+O(\log^{n+2}T).$ (5.5)

Finally, we may combine (5.4) and (5.5) to obtain $I_{R}$, given by

$I_{R}=(-1)^{n}\Delta(X)\frac{T}{2\pi}\left(\log^{n}X\left\\{\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right\\}-\sum_{X=mr}\log^{n}m\ \Lambda(r)\right)+O(\log^{n+2}T).$ (5.6)

###  5.3. Evaluating $I_{L}$

Finally we evaluate $I_{L}$, which is where most of the contribution to the
asymptotic expansion comes from. Using Lemma 1 we have

$\displaystyle I_{L}$ $\displaystyle=-\frac{1}{2\pi
i}\int_{1-c+i}^{1-c+iT}\frac{\xi^{\prime}}{\xi}(s)\zeta^{(n)}(s)X^{s}\ ds$
$\displaystyle=-\frac{1}{2\pi
i}\int_{1-c+i}^{1-c+iT}\left(-\frac{\xi^{\prime}}{\xi}(1-s)\right)\left(\frac{1}{\chi(1-s)}\sum_{k=0}^{n}{n\choose
k}(-1)^{k}\frac{\chi^{(n-k)}(s)}{\chi(s)}\zeta^{(k)}(1-s)\right)X^{s}\ ds$
$\displaystyle=-\frac{1}{2\pi}\int_{1}^{T}\left(-\frac{\xi^{\prime}}{\xi}(c-it)\right)\left(\frac{1}{\chi(c-it)}\sum_{k=0}^{n}{n\choose
k}(-1)^{k}\frac{\chi^{(n-k)}(1-c-it)}{\chi(1-c-it)}\zeta^{(k)}(c-it)\right)X^{1-c+it}\
dt.$

By complex conjugation we get

$\displaystyle\overline{I_{L}}$
$\displaystyle=\frac{X^{1-c}}{2\pi}\int_{1}^{T}\frac{\xi^{\prime}}{\xi}(c+it)\left(\frac{1}{\chi(c+it)}\sum_{k=0}^{n}{n\choose
k}(-1)^{k}\frac{\chi^{(n-k)}(1-c+it)}{\chi(1-c+it)}\zeta^{(k)}(c+it)\right)X^{-it}\
dt$
$\displaystyle=\frac{X^{1-c}}{2\pi}\int_{1}^{T}\left\\{\frac{1}{2}\log\left(\frac{t}{2\pi}\right)+\frac{\pi
i}{4}-\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{c+it}}+O\left(\frac{1}{t}\right)\right\\}$
$\displaystyle\qquad\left(\chi(1-c-it)\sum_{k=0}^{n}{n\choose
k}(-1)^{k}\left((-1)^{n-k}\log^{n-k}\left(\frac{t}{2\pi}\right)+O\left(\frac{1}{t}\right)\right)(-1)^{k}\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c+it}}\right)X^{-it}\
dt$
$\displaystyle=\frac{X^{1-c}}{2\pi}\int_{1}^{T}\left\\{\frac{1}{2}\log\left(\frac{t}{2\pi}\right)+\frac{\pi
i}{4}-\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{c+it}}\right\\}$
$\displaystyle\qquad\left(\chi(1-c-it)\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\log^{n-k}\left(\frac{t}{2\pi}\right)\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c+it}}\right)e^{-it\log
X}\ dt+O(\log^{n+2}T)$

where the second line follows from Lemma 3.

We now split this integral and evaluate each part separately.

$\displaystyle\overline{I_{L}}$
$\displaystyle=\frac{X^{1-c}}{2\pi}\frac{1}{2}\int_{1}^{T}\chi(1-c-it)\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\log^{n-k+1}\left(\frac{t}{2\pi}\right)\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c+it}}e^{-it\log
X}\ dt$ $\displaystyle+\frac{X^{1-c}}{2\pi}\frac{\pi
i}{4}\int_{1}^{T}\chi(1-c-it)\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\log^{n-k}\left(\frac{t}{2\pi}\right)\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c+it}}e^{-it\log
X}\ dt$
$\displaystyle+\frac{X^{1-c}}{2\pi}\int_{1}^{T}\chi(1-c-it)\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{c+it}}\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k+1}\log^{n-k}\left(\frac{t}{2\pi}\right)\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c+it}}e^{-it\log
X}\ dt$ $\displaystyle+O(\log^{n+2}T)$
$\displaystyle=J_{1}+J_{2}+J_{3}+O(\log^{n+2}T).$

The key component to this section of the proof here is the method of
stationary phase. Applications of this method to these types of problems can
be in Gonek [7, Sect.4, p.131], in Levinson [15] and in Jakhlouti and Mazhouda
[12, Sect.2, p.13], amongst other places.

In an entirely analogous way to the proof that Gonek writes in [7], we are
able to prove the following result which we need to evaluate the integrals
$J_{1},J_{2},J_{3}$.

######  Lemma 4.

Let $X$ be a fixed positive real. Let $\\{b_{m}\\}_{m=1}^{\infty}$ be a
sequence of complex numbers such that for any $\epsilon>0,b_{m}\ll
m^{\epsilon}$. Let $c>1$ and let $k\geq 0$ be an integer. Then for $T$
sufficiently large, we have

$\displaystyle\frac{1}{2\pi}$
$\displaystyle\int_{1}^{T}\chi(1-c-it)\left(\sum_{m=1}^{\infty}b_{m}m^{-c-it}\right)\log^{k}\left(\frac{t}{2\pi}\right)e^{-it\log
X}\ dt$ $\displaystyle=X^{c}\sum_{1\leq m\leq\frac{T}{2\pi
X}}b_{m}\log^{k}(mX)\ e^{-2\pi imX}+O\left(T^{c-\frac{1}{2}}\log^{k}T\right).$

Applying Lemma 4 to each of the summations $J_{k}$, $k=1,2,3$, we have

$\displaystyle J_{1}$
$\displaystyle=\frac{X^{1-c}}{2\pi}\frac{1}{2}\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c}}\int_{1}^{T}\chi(1-c-it)\log^{n-k+1}\left(\frac{t}{2\pi}\right)e^{-it\log(mX)}\
dt$ $\displaystyle=\frac{X}{2}\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\sum_{m\leq\frac{T}{2\pi X}}\log^{k}(m)\log^{n-k+1}(mX)\ e^{-2\pi
imX}+O(\sqrt{T}\log^{n+2}T)$
$\displaystyle=\frac{X}{2}\sum_{m\leq\frac{T}{2\pi X}}e^{-2\pi
imX}\left[\sum_{k=0}^{n}{n\choose
k}(-1)^{n-k}\log^{k}(m)\log^{n-k}(mX)\right](\log(mX))+O(\sqrt{T}\log^{n+2}T)$
$\displaystyle=\frac{X}{2}\sum_{m\leq\frac{T}{2\pi X}}e^{-2\pi
imX}\log(mX)\left[-\log(mX)+\log m\right]^{n}+O(\sqrt{T}\log^{n+2}T)$
$\displaystyle=(-1)^{n}\frac{X}{2}\sum_{m\leq\frac{T}{2\pi X}}e^{-2\pi
imX}\log(mX)\log^{n}X+O(\sqrt{T}\log^{n+2}T)$
$\displaystyle=(-1)^{n}\frac{X}{2}\log^{n+1}X\sum_{m\leq\frac{T}{2\pi
X}}e^{-2\pi imX}+(-1)^{n}\frac{X}{2}\log^{n}X\sum_{m\leq\frac{T}{2\pi
X}}e^{-2\pi imX}\log m+O(\sqrt{T}\log^{n+2}T).$

Similarly,

$\displaystyle J_{2}$ $\displaystyle=\frac{X^{1-c}}{2\pi}\frac{\pi
i}{4}\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c}}\int_{1}^{T}\chi(1-c-it)\log^{n-k}\left(\frac{t}{2\pi}\right)e^{-it\log(mX)}\
dt$ $\displaystyle=\frac{\pi i}{4}X\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\sum_{m\leq\frac{T}{2\pi X}}\log^{k}(m)\log^{n-k}(mX)\ e^{-2\pi
imX}+O(\sqrt{T}\log^{n+1}T)$ $\displaystyle=\frac{\pi
i}{4}X\sum_{m\leq\frac{T}{2\pi X}}e^{-2\pi imX}\left[\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\log^{k}(m)\log^{n-k}(mX)\right]+O(\sqrt{T}\log^{n+1}T)$
$\displaystyle=\frac{\pi i}{4}X\sum_{m\leq\frac{T}{2\pi X}}e^{-2\pi
imX}\left[\log(m)-\log(mX)\right]^{n}+O(\sqrt{T}\log^{n+1}T)$
$\displaystyle=(-1)^{n}\frac{\pi i}{4}X\log^{n}X\sum_{m\leq\frac{T}{2\pi
X}}e^{-2\pi imX}+O(\sqrt{T}\log^{n+1}T).$

Finally,

$\displaystyle J_{3}$
$\displaystyle=\frac{X^{1-c}}{2\pi}\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k+1}\sum_{r=1}^{\infty}\frac{\Lambda(r)}{r^{c}}\sum_{m=1}^{\infty}\frac{\log^{k}m}{m^{c}}\int_{1}^{T}\chi(1-c-it)\log^{n-k}\left(\frac{t}{2\pi}\right)e^{-it\log(mrX)}\
dt$ $\displaystyle=X\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k+1}\sum_{mr\leq\frac{T}{2\pi
X}}\Lambda(r)\log^{k}(m)\log^{n-k}(mrX)\ e^{-2\pi
imrX}+O(\sqrt{T}\log^{n+2}T)$ $\displaystyle=-X\sum_{mr\leq\frac{T}{2\pi
X}}e^{-2\pi imrX}\Lambda(r)\left[\sum_{k=0}^{n}{n\choose
k}(-1)^{n+k}\log^{k}(m)\log^{n-k}(mrX)\right]+O(\sqrt{T}\log^{n+2}T)$
$\displaystyle=-X\sum_{mr\leq\frac{T}{2\pi X}}e^{-2\pi
imrX}\Lambda(r)\left[-\log(mrX)+\log m\right]^{n}+O(\sqrt{T}\log^{n+2}T)$
$\displaystyle=(-1)^{n+1}X\sum_{mr\leq\frac{T}{2\pi X}}e^{-2\pi
imrX}\Lambda(r)\log^{n}(rX)+O(\sqrt{T}\log^{n+2}T).$

Recombining these, we have

$\displaystyle\overline{I_{L}}$
$\displaystyle=(-1)^{n}\left\\{X\log^{n}X\left[\frac{1}{2}\log X+\frac{\pi
i}{4}\right]\sum_{m\leq\frac{T}{2\pi X}}e^{-2\pi
imX}+\frac{1}{2}X\log^{n}X\sum_{m\leq\frac{T}{2\pi X}}e^{-2\pi imX}\log
m\right.$ $\displaystyle\qquad\qquad-X\sum_{mr\leq\frac{T}{2\pi X}}e^{-2\pi
imrX}\Lambda(r)\log^{n}(rX)\Bigg{\\}}+O(\sqrt{T}\log^{n+2}T).$

Taking complex conjugates,

$\displaystyle I_{L}$
$\displaystyle=(-1)^{n}\left\\{X\log^{n}X\left[\frac{1}{2}\log X-\frac{\pi
i}{4}\right]\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi
imX}+\frac{X}{2}\log^{n}X\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}\log
m\right.$ $\displaystyle\qquad\qquad-X\sum_{mr\leq\frac{T}{2\pi X}}e^{2\pi
imrX}\Lambda(r)\log^{n}(rX)\Bigg{\\}}+O(\sqrt{T}\log^{n+2}T).$ (5.7)

###  5.4. Finalising the Proof of Theorem 1

Combining (5.3), (5.6), and (5.7) gives $I$ in the second way that we are
looking for. Specifically, we have

$\displaystyle I=(-1)^{n}$
$\displaystyle\left\\{\Delta(X)\frac{T}{2\pi}\left(\log^{n}X\left(\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)-\sum_{mr=X}\Lambda(r)\log^{n}m\right)\right.$
$\displaystyle+X\log^{n}X\left(\frac{1}{2}\log X-\frac{\pi
i}{4}\right)\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}$
$\displaystyle+\frac{X}{2}\log^{n}X\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi
imX}\log m$ $\displaystyle-X\sum_{mr\leq\frac{T}{2\pi X}}e^{2\pi
imrX}\Lambda(r)\log^{n}(rX)\Bigg{\\}}$ $\displaystyle+O(\sqrt{T}\log^{n+2}T),$

Combining this asymptotic expansion with our observation that
$I=\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}$ in (5.2) completes the
proof of Theorem 1.

##  6\. Proof of Corollary 1

We may rewrite Theorem 1 in a slightly different way as follows.

######  Corollary 2.

For $X$ a fixed positive real number, we have

$\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}=\\\
(-1)^{n}\left\\{\Delta(X)\frac{T}{2\pi}\left(\log^{n}X\left(\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)-\sum_{mr=X}\Lambda(r)\log^{n}m\right)\right.\\\
+X\log^{n}X\left(\frac{1}{2}\log X-\frac{\pi
i}{4}\right)\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi
imX}+\frac{X}{2}\log^{n}X\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}\log m\\\
-X\sum_{k=0}^{n}{n\choose k}\log^{n-k}X\left(\sum_{mr\leq\frac{T}{2\pi
X}}e^{2\pi imrX}\Lambda(r)\log^{k}r\right)\Bigg{\\}}+O(\sqrt{T}\log^{n+2}T),$

where $\Delta(X)$ is given in Theorem 1.

###### Proof.

This follows from Theorem 1 by using the binomial expansion on $\log^{n}(rX)$
in the last summation in the braces in Theorem 1. ∎

######  Remark.

The advantage to rewriting Theorem 1 in the form of Corollary 2 is that none
of the summations involving exponentials have any reliance on powers of $\log
X$.

When $X\geq 1$ and $X\in\mathbb{Z}$, notice that

$\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}=\frac{T}{2\pi X}+O(1),$
$\sum_{m\leq\frac{T}{2\pi X}}e^{2\pi imX}\log m=\frac{T}{2\pi
X}\log\left(\frac{T}{2\pi X}\right)-\frac{T}{2\pi X}+O(\log T)$
$\sum_{mr\leq\frac{T}{2\pi X}}e^{2\pi
imrX}\Lambda(r)\log^{k}r=\sum_{mr\leq\frac{T}{2\pi X}}\Lambda(r)\log^{k}r.$

Define

$S=(-1)^{k+1}\sum_{mr\leq\frac{T}{2\pi X}}\Lambda(r)\log^{k}r$ (6.1)

Then in an entirely analogous way to that done in Hughes and Pearce-Crump
[10], we have

$\displaystyle S$ $\displaystyle=(-1)^{k+1}\frac{1}{k+1}\frac{T}{2\pi
X}\log^{k+1}\left(\frac{T}{2\pi X}\right)$
$\displaystyle+(-1)^{k+1}\sum_{l=0}^{k}{k\choose
l}(-1)^{l}l!\left(-1+\sum_{j=0}^{l}(-1)^{j}C_{j}\right)\frac{T}{2\pi
X}\log^{k-l}\left(\frac{T}{2\pi X}\right)+k!A_{k}\frac{T}{2\pi X}+E_{k}(T)$

where

$E_{k}(T)=\begin{cases}O\left(Te^{-C\sqrt{\log T}}\right)&\text{
unconditionally}\\\ O\left(\sqrt{T}\log^{k+2}T\right)&\text{ under RH}\\\
\end{cases}$

and $C>0$ is a constant. Multiplying $S$ by $(-1)^{k+1}$ gives the summation
that we were originally looking for.

Substituting these expressions into the asymptotic expansion from Corollary 2
gives

$\displaystyle\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}=(-1)^{n}$
$\displaystyle\left\\{\frac{T}{2\pi}\left(\log^{n}X\left(\frac{1}{2}\log\left(\frac{T}{2\pi}\right)-\frac{1}{2}+\frac{\pi
i}{4}\right)-\sum_{mr=X}\Lambda(r)\log^{n}m\right)\right.$
$\displaystyle+\log^{n}X\left(\frac{1}{2}\log X-\frac{\pi
i}{4}\right)\frac{T}{2\pi}$
$\displaystyle+\frac{1}{2}\log^{n}X\frac{T}{2\pi}\log\left(\frac{T}{2\pi
X}\right)-\frac{T}{2\pi}$ $\displaystyle-\sum_{k=0}^{n}{n\choose
k}\log^{n-k}X\left(\frac{1}{k+1}\frac{T}{2\pi}\log^{k+1}\left(\frac{T}{2\pi
X}\right)\right.$ $\displaystyle+\sum_{l=0}^{k}{k\choose
l}(-1)^{l}l!\left(-1+\sum_{j=0}^{l}(-1)^{j}C_{j}\right)\frac{T}{2\pi}\log^{k-l}\left(\frac{T}{2\pi
X}\right)$
$\displaystyle+(-1)^{k+1}k!A_{k}\frac{T}{2\pi}\Bigg{)}\Bigg{\\}}+E_{n}(T),$

where $E_{n}(T)$, $C_{j}$ and $A_{k}$ are defined in the statement of
Corollary 1.

We want to simplify this so that the $\log\left(\frac{T}{2\pi X}\right)$ terms
can be written separately in terms of $\log\left(\frac{T}{2\pi}\right)$ and
$\log X$. To do this, we begin by writing this previous expansion in a
slightly different form and do some simplifying.

$\displaystyle\sum_{0<\gamma\leq T}\zeta^{(n)}(\rho)X^{\rho}$
$\displaystyle=(-1)^{n}\frac{T}{2\pi}\log^{n}X\left(\log\left(\frac{T}{2\pi}\right)-1-\sum_{mr=X}\Lambda(r)\log^{n}m\right)$
$\displaystyle+(-1)^{n+1}\sum_{k=0}^{n}{n\choose
k}\log^{n-k}X\Bigg{\\{}\frac{1}{k+1}\frac{T}{2\pi
X}\left(\log\left(\frac{T}{2\pi}\right)-\log X\right)^{k+1}$
$\displaystyle\qquad+\sum_{l=0}^{k}{k\choose
l}(-1)^{l}l!\left(-1+\sum_{j=0}^{l}(-1)^{j}C_{j}\right)\frac{T}{2\pi}\left(\log\left(\frac{T}{2\pi}\right)-\log
X\right)^{k-l}$
$\displaystyle\qquad+(-1)^{k+1}k!A_{k}\frac{T}{2\pi}\Bigg{\\}}+E_{n}(T).$

Applying the binomial theorem, we have

$\displaystyle\sum_{0<\gamma\leq T}$
$\displaystyle\zeta^{(n)}(\rho)X^{\rho}=(-1)^{n}\frac{T}{2\pi}\log^{n}X\left(\log\left(\frac{T}{2\pi}\right)-1-\sum_{mr=X}\Lambda(r)\log^{n}m\right)$
$\displaystyle+(-1)^{n+1}\frac{T}{2\pi}\Bigg{\\{}\sum_{k=0}^{n}\sum_{u=0}^{k+1}{n\choose
k}{{k+1}\choose
u}(-1)^{u}\frac{1}{k+1}\log^{k+1-u}\left(\frac{T}{2\pi}\right)\log^{n-k+u}X$
$\displaystyle+\sum_{k=0}^{n}\sum_{l=0}^{k}\sum_{u=0}^{k-l}{n\choose
k}{k\choose l}{{k-l}\choose
u}(-1)^{l}l!\left(-1+\sum_{j=0}^{l}(-1)^{j}C_{j}\right)\log^{k-l-u}\left(\frac{T}{2\pi}\right)\log^{n-k+u}X$
$\displaystyle+\sum_{k=0}^{n}{n\choose
k}(-1)^{k+1}k!A_{k}\log^{n-k}X\Bigg{\\}}+E_{n}(T).$

Finally, we may rearrange slightly to complete the proof of Corollary 1 to
give

$\displaystyle\sum_{0<\gamma\leq T}$
$\displaystyle\zeta^{(n)}(\rho)X^{\rho}=(-1)^{n+1}\frac{T}{2\pi}\log^{n}X\Bigg{\\{}\sum_{k=0}^{n}\sum_{u=0}^{k+1}{n\choose
k}{{k+1}\choose
u}(-1)^{u}\frac{1}{k+1}\log^{k+1-u}\left(\frac{T}{2\pi}\right)\log^{u-k}X$
$\displaystyle+\sum_{k=0}^{n}\sum_{l=0}^{k}\sum_{u=0}^{k-l}{n\choose
k}{k\choose l}{{k-l}\choose
u}(-1)^{l}l!\left(-1+\sum_{j=0}^{l}(-1)^{j}C_{j}\right)\log^{k-l-u}\left(\frac{T}{2\pi}\right)\log^{u-k}X$
$\displaystyle+\sum_{k=0}^{n}{n\choose
k}(-1)^{k+1}k!A_{k}\log^{-k}X-\left(\log\left(\frac{T}{2\pi}\right)-1-\sum_{mr=X}\Lambda(r)\log^{n}m\right)\Bigg{\\}}+E_{n}(T).$

## Acknowledgements

The author is grateful to his supervisor, Dr. Christopher Hughes, for all of
his help and guidance when writing this paper, as well as introducing him to
the original paper that inspired this generalisation.

######  Remark.

This paper will form part of the author’s PhD thesis at the University of
York.

## References

  * [1] K. Ford, K. Soundararajan, and A. Zaharescu.  On the distribution of imaginary parts of zeros of the Riemann zeta function, II.  Math. Annalen, 343:487–505, 2009.
  * [2] K. Ford and A. Zaharescu.  On the distribution of imaginary parts of zeros of the Riemann zeta function.  J. reine angew. Math., 479:145–158, 2005.
  * [3] A. Fujii.  On a Theorem of Landau.  Proc. Japan Acad., 65(2):51–54, 1989.
  * [4] A. Fujii.  On a Theorem of Landau. II.  Proc. Japan Acad., 66(9):291–296, 1990.
  * [5] A. Fujii.  On a Conjecture of Shanks.  Proc. Japan Acad., 70(4):109–114, 1994.
  * [6] A. Fujii.  On the Distribution of Values of the Derivative of the Riemann Zeta Function at Its Zeros. I.  Proceedings of the Steklov Institute of Mathematics, 276:51–76, 2012.
  * [7] S. M. Gonek.  Mean values of the Riemann zeta-function and its derivatives.  Invent. math., 75:123–141, 1984.
  * [8] S. M. Gonek.  A Formula of Landau and Mean Values of $\zeta(s)$.  In Topics in Analytic Number Theory. University of Texas Press, Austin, 1985.
  * [9] S. M. Gonek.  An explicit formula of Landau and its applications to the theory of the zeta-function.  In A Tribute to Emil Grosswalk: Number Theory and Related Analysis, volume 143, pages 395–413. Contemporary Mathematics, 2011.
  * [10] C. Hughes and A. Pearce-Crump.  A Discrete Mean-Value Theorem for the Higher Derivatives of the Riemann Zeta Function, 2021.  Submitted, arXiv:2106.03005.
  * [11] M. I. Israilov.  The Laurent expansion of the Riemann zeta function.  Trudy Mat. Inst. Steklov., 158:98–104, 1981.
  * [12] M. Jakhlouti and K. Mazhouda.  Distribution of the values of the derivative of the Dirichlet ${L}$-functions at its $a$-points.  Bulletin of the Korean Mathematical Society, 54(4):1141–1158, 2017.
  * [13] J. Kaczorowski, A. Languasco, and A. Perelli.  A Note on Landau’s Formula.  Funct. Approx. Comment. Math., 28:173–186, 2000.
  * [14] E. Landau.  Über die Nullstellen der Zetafunktion.  Math. Annalen, 71:548–568, 1911.
  * [15] N. Levinson.  More than One Third of Zeros of Riemann’s Zeta-Function are on $\sigma=\frac{1}{2}$.  Advances in Mathematics, 13:383–436, 1974.
  * [16] E. C. Titchmarsh.  The Theory of the Riemann Zeta Function.  Oxford Science Publications, 1951.