aainstitutetext: Engineering Physics, McMaster University, Hamilton, ON,
Canada, L8S 4M1bbinstitutetext: AI Endurance Inc, Hamilton, ON, Canada, L8P
0A1

# Extremal learning: extremizing the output of a neural network in regression
problems

Zakaria Patel b and Markus Rummel<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Neural networks allow us to model complex relationships between variables. We
show how to efficiently find extrema of a trained neural network in regression
problems. Finding the extremizing input of an approximated model is formulated
as the training of an additional neural network with a loss function that
minimizes when the extremizing input is achieved. We further show how to
incorporate additional constraints on the input vector such as limiting the
extrapolation of the extremizing input vector from the original training data
set. An instructional example of this approach using TensorFlow is included.

##  1 Introduction

Neural networks (NNs) McCulloch:1943  allow us to model complex relationships
between certain input $(x_{n})$ and output data sets $(y_{n})$ when the
underlying functional dependence is unknown. Advances in hardware, the advent
of big data, and computational methods e.g. Hinton1985 ; Rumelhart:1986 ;
LeCun1989 ; LeCun1998 ; bengio2003neural ; srivastava2014dropout ; LeCun2015
have made it possible to apply (deep) NNs to a plethora of regression problems
Goodfellow-et-al-2016 . While it is extremely useful to obtain a prediction
function $y=f(\theta;x)$ with parameters $\theta$ in the first place, one is
often interested in properties such as minima or maxima of this function $f$.

For example, take radiotherapy cancer treatment. In a first step, one would
like to find out how the output $y$, the tumor size, depends on the input $x$,
the kinds and dosages of certain radiotherapies. Once the NN is trained via
fitting the parameter values $\theta$ and can predict the outcome of a certain
treatment to the desired accuracy, one would like to know: what is the ideal
treatment for the patient’s cancer? In this case, we want to find the input
vector $x_{\rm min}$ that minimizes the tumor size. Were $f$ a simple analytic
function, one would simply proceed by finding extrema via solving for zeros of
the derivative function w.r.t. $x$. However, due to the NN’s complicated
nonlinear functional structure, generally high dimensionality and potential
recursive input dependencies lstm  this easily becomes a highly complicated
system of equations that is extremely difficult to solve.111For a simple
example, consider a single layered neural network with activation
$a(z)=\tanh(z)$. Then the derivative of $a(x)=\tanh(W\cdot x+b)$ with weights
$W$ and biases $b$ allows us to find the optimal vector $x_{\rm ext}$ for the
network. The strength of NNs also becomes a weakness in a sense: while NNs can
reveal the elusive black box functions presented by certain processes, it can
only do so in a complicated web of weights and biases that make an analytical
analysis of the NN challenging.

In this paper, we present an alternative solution to this problem via a
process that we term extremal learning: Finding the extremizing input
$\hat{x}$ is formulated as the training of an NN itself. The parameters
$\theta$ of the NN are frozen while the input vector $x$ is promoted to a
trainable variable itself. A loss function $\mathcal{L}$ is defined such that
its minimization, e.g. via gradient descent Rumelhart:1986 ;
lecun2012efficient , is equivalent to achieving an extremal value for $f$.
Hence, custom machine learning frameworks such as TensorFlow
abadi2016tensorflow  can be used to calculate extrema of NNs via this method
of extremal learning. For an implementation of extremal learning in
TensorFlow, we provide example code at: https://github.com/ZakariaPZ/Extremal-
Learning.

The loss function $\mathcal{L}$ may also include further constraints on the
problem at hand. A common constraint in regression problems is that $\hat{x}$
should not stray too far from the input data set $(x_{n})$. Respecting this
constraint avoids extrapolating into a no-data regime where the predictions of
the NN are becoming unreliable. This can be achieved by limiting the
extrapolation of the input vector to within some set number of standard
deviations. For instance, consider fitting a car’s fuel expenditure as a
function of its speed $v$ using data from low speeds. The data reflects a
situation in which the rolling resistance force $F_{r}$ that does not depend
on speed is dominating the fuel expenditure. For higher speeds, the force
dominating the fuel expenditure is the drag force $F_{d}\propto v^{2}$.
Clearly, extrapolating to higher speeds with a model trained on low speed data
will yield far too low a fuel expenditure. In a similar spirit, we risk making
misleading predictions if we look for extremizing inputs far from the data set
the model was trained on. Our initial data may not confidently reflect the
behaviour of our system at points lying far from our observations. We will
demonstrate how to include these or other constraints into extremal learning
via the definition of custom loss functions as also discussed in Trinh2017gi ;
2020arXiv200613554M .

#### Relationship to GANs

Extremal learning has some similarities with Generative Adversarial Networks
(GANs) Goodfellow2014gi ; 8253599 ; Hong2019gi . In a typical example of
extremal learning, we have a trained model for image classification and now
want to find an image that maximizes the output for a given class. A GAN
operates with a similar goal, yet executes it differently. Given a random
noise input $z$, the generator in a GAN is trained to minimize the loss

$\mathcal{L}=\log\left\\{1-D\left[G(z)\right]\right\\}\,,$ (1)

where $G(z)$ is the output of a generator network and $D$ is a discriminator
network. The goal of the discriminator is to identify which inputs are real
and which inputs are counterfeits of the generator, while the generator
attempts to fool the discriminator into classifying its output as a true
input. By minimizing the above loss, the generator should learn a generative
model capable of transforming a random noise sample into an output $G(z)$
which closely resembles the nature of a true instance $y$. Then, if we passed
$G(z)$ into a discriminator network, it should classify the input similarly to
that of the true instance $y$. Succinctly, a GAN works to minimize the
dissimilarity between an artificially produced data distribution and a true
distribution. For extremal learning, instead of a generator-discriminator
pair, we consider a lone discriminator whose loss function we try to minimize.
An important distinction here is that extremal learning directly changes the
input vector $x$, rather than using other optimization methods such as Markov
Chains to create a generative model.

#### Relationship to adversarial training

We can also draw parallels to adversarial training 10.1145/2046684.2046692 ;
Goodfellow2014gi ; 2016arXiv161101236K . In image classification, adversarial
attacks involve injecting an adversarial vector $x_{\rm adv}$ into the input
vector $x$, where $x_{\rm adv}$ acts as a perturbation, imperceptible to the
human eye yet causing misclassification in a machine learning model.
Adversarial training aims to achieve the generation of such noise vectors
$x_{\rm adv}$, known as adversarial examples. This involves training an input
vector $x$. One technique to generate adversarial examples is the fast method
Kurakin2017gi .

This method aims to maximize a linearized loss function $\mathcal{L}$ that is
approximated to first order as

$\mathcal{L}(\tilde{x};\theta)\approx\mathcal{L}(x;\theta)+(\tilde{x}-x)^{T}\nabla_{x}\mathcal{L}(x;\theta)\,,$
(2)

where $\tilde{x}$ is the perturbed vector, i.e. the adversarial example in
training. The above loss is maximized subject to the following max-norm
constraint on $\tilde{x}$ 2014arXiv1412.6572G :

$||\tilde{x}-x||_{\infty}\leq\epsilon\,,$ (3)

i.e. each pixel can only be changed by a value $\epsilon$. The adversarial
example is then generated by training the input as

$\tilde{x}=x+\epsilon\,{\rm sign}[\nabla_{x}\mathcal{L}(x;\theta)]\,.$ (4)

The extremal learning technique introduced in this paper also seeks to train
inputs, but instead hopes to minimize a nonlinearized loss function via
backpropagation. The input perturbations are not limited by a max-norm
constraint, but the learning rate of the backpropagation.

### A road map

This paper is structured as follows: in Section 2, we describe the general
formalism of extremal learning for regression problems. In Section 3, we
present a toy example of this approach by finding a maximizing input vector
using an implementation of extremal learning in TensorFlow. We conclude in
Section 4.

##  2 Finding extrema

We first recap supervised learning in regression problems: given a set of
labeled observations $(x_{n})$ and $(y_{n})$ with an NN function $f(\theta;x)$
with parameters $\theta$, for supervised learning, the goal is to find the
best-fit parameters

$\hat{\theta}=\arg\min_{\theta}\left\\{\sum_{n}\mathcal{L}_{t}\left[f(\theta;x_{n}),y_{n}\right]\right\\}.$
(5)

Here, $\mathcal{L}_{t}(y^{\prime},y)$ is the training loss function that
measures the distance between the predicted output
$y^{\prime}=f(\hat{\theta};x)$ is from the true output $y$. Common choices in
regression problems include mean squared error (MSE), mean squared logarithmic
error or mean absolute error Goodfellow-et-al-2016 . Once a model has been
trained, i.e. once we have a set of parameters $\hat{\theta}$, predictions can
be made. Given an input vector $x$, evaluating the function
$f(\hat{\theta};x)$ is called inference - the rules dictating how
$f(\hat{\theta};x)$ behaves are now approximately encoded within the NN’s
parameters.

###  2.1 Extremal learning

In this paper, we want to find an extremizing input $\hat{x}$ because we are
interested in maximizing or minimizing the output of $f$ that can represent a
quantity that we would like to optimize as for instance a return of
investment, effectiveness of a treatment, amount spent etc. We formulate this
extremization as yet another optimization problem: given a trained model
$f(\hat{\theta};x)$, we want to find the input(s) $\hat{x}$ that maximize the
output of the model, i.e.

$\hat{x}=\begin{cases}\arg\max_{x}f(\hat{\theta};x)&\text{for
maximization}\,,\\\ \arg\min_{x}f(\hat{\theta};x)&\text{for
minimization}\,.\end{cases}$ (6)

We call this task extremal training, because we are looking for an input
vector that is extremizing the output. We can implement this optimization as
fitting $f$ via the following steps:

  * •

Freeze the parameters $\theta$ of the model to the value $\hat{\theta}$ from
the first training iteration, i.e. make them not trainable anymore.

  * •

Promote the input vector $x$ to a trainable variable. From the fitting
procedure point of view, there is no input data anymore as
$\theta=\hat{\theta}$ is fixed and $x$ is now effectively a parameter. We now
have a different NN, albeit with the same architecture as the original NN.

  * •

Define a loss function $\hat{\mathcal{L}}$ such that its minimization is
equivalent to the optimization problem at hand, i.e. finding the extremizing
input $\hat{x}$. Reasonable, MSE inspired, loss functions are

$\hat{\mathcal{L}}=\begin{cases}\left[f(\hat{\theta};x)^{2}+\kappa\right]^{-1}&\text{for
maximization}\,,\\\ f(\hat{\theta};x)^{2}&\text{for
minimization}\,,\end{cases}$ (7)

with constant $\kappa>0$ to avoid the zero divergence.

  * •

Minimize the loss $\hat{\mathcal{L}}$ via common fitting procedures, such as
gradient descent and backpropagation Rumelhart:1986 ; lecun2012efficient . The
gradient descent rule is applied to update the input vector $x$ using the
partial derivatives of the loss with respect to the input $x$ Yan1988 .

$x\rightarrow x-\alpha\nabla_{x}\hat{\mathcal{L}}\,,$ (8)

where $\alpha$ is the learning rate. One has to provide an initial vector
$x_{\rm init}$ to start the gradient descent. This may be chosen at random or
from the training data and may be chosen with caution in case of non-convexity
LeCun2015 .

###  2.2 Constraints via additional loss functions

We may also introduce a variety of $k$ additional loss functions
$\mathcal{L}_{i}$ to penalize undesirable inputs $x$ or outputs $y$

$\mathcal{L}=\hat{\mathcal{L}}+\sum_{i=1}^{k}\mathcal{L}_{i}\,.$ (9)

In practice, one may introduce as many $\mathcal{L}_{i}$ as desired to
constrain the search for $\hat{x}$ to a space that is desirable for the
problem at hand for both the inputs and outputs.

A common constraint is that the global or local extrema of $f$ may not be the
most feasible solution to the optimization problem. Optimizing an input
demands that we appreciate that there exists some boundary beyond which the
input may become unrealistic. This is the extrapolation problem: we do not
want to extrapolate too far from the original data set $(x_{n})$ as the
predictions $f(\hat{\theta};x)$ may become arbitrarily unrealistic. Using
extremal learning in conjunction with custom loss functions $\mathcal{L}_{i}$
facilitates convergence towards an optimum within an error and constraint
window that is often more useful in real-life applications than global or
local extrema of the function $f$.

One way to define a loss function that penalizes extrapolation is as follows:
if $x$ deviates more than $c$-times the standard deviation $\sigma$ of the
data set $(x_{n})$ from the mean of the data set, those input vectors are
penalized:

$\mathcal{L}_{1}=\kappa_{1}\sum_{i=0}^{m}\begin{cases}(x_{i}-\mu_{i}+c\sigma_{i})^{2}&x_{i}<\mu_{i}-c\sigma_{i}\,,\\\
0&\mu_{i}-c\sigma_{i}\leq x_{i}\leq\mu_{i}+c\sigma_{i}\,,\\\
(x_{i}-\mu_{i}-c\sigma_{i})^{2}&x_{i}>\mu_{i}+c\sigma_{i}\,,\end{cases}$ (10)

where $x_{i}$ are the individual components of the $m$-dimensional input
vector $x$ and $\kappa_{1}$ is a normalization constant. To avoid problems
during gradient descent, one may choose continuous functions for the
additional loss functions $\mathcal{L}_{i}$ and ensure their relative
weighting through normalization constants $\kappa_{i}$ is inline with how
strict the different constraints should be enforced relative to each other
Trinh2017gi ; 2020arXiv200613554M .

One may also combine additional constraints with the extremal loss function
$\hat{\mathcal{L}}$. For instance, let us consider a situation in which
negative outputs are ill defined as is the case for example if $y$ represents
a temperature that we want to maximize. In this case, we can define the loss
function as

$\mathcal{L}_{2}=\begin{cases}-\kappa_{2}y+\hat{\kappa}&y<0\,,\\\
\left(y^{2}+\hat{\kappa}^{-1}\right)^{-1}&y\geq 0\,,\end{cases}$ (11)

where the first term ensures that the output is indeed positive. To give this
physical constraint priority over maximization we would choose
$\kappa_{2}\gg\hat{\kappa}$. For a visualization of the maximization and
extrapolation loss, see Figure 1.

If (some of) the components of the input vector are required to be positive
this can be enforced via a loss function similar to (11):

$\mathcal{L}_{3}=\kappa_{3}\sum_{i=0}^{m}\begin{cases}-x_{i}&x_{i}<0\,,\\\
0&x_{i}\geq 0\,,\end{cases}$ (12)

Figure 1: (a) Plot of maximization loss defined in (11) with $\kappa_{2}=10$
and $\hat{\kappa}=1$. (b) Plot of extrapolation loss defined in (10) with
$\kappa_{1}=1$.

##  3 Example and case study

We now consider a simple toy example to demonstrate extremal learning. We
first generate fake data that is used to fit a feed forward NN. After fitting,
we apply the extremal learning approach described above to find a maximizing
input. We provide the source code to this example using TensorFlow at:
https://github.com/ZakariaPZ/Extremal-Learning.

Consider a simple problem in which $x\in\mathbb{R}^{4}$ represents a specific
intake of food classes to be optimized with respect to some arbitrary measure
of “goodness of health” $y$. The classes are as follows:

  * •

$x_{0}$: carbohydrate intake

  * •

$x_{1}$: protein intake

  * •

$x_{2}$: cake intake

  * •

$x_{3}$: candy intake

All inputs are chosen to be in the range $x_{i}\in[-1,1]$. We want to know how
health depends on the different intakes and, in the next step, determine which
intake composition is the most optimal for health.

###  3.1 Data generation

To generate a data set, we come up with a function that measures "goodness of
health" as a function of the four different inputs in arbitrary units:

$g_{\rm true}(x)=1-|x_{0}|-x_{1}^{2}-x_{2}-e^{x_{3}}+\epsilon\,.$ (13)

In moderation, $x_{0}$ and $x_{1}$, representing protein and carbs
respectively, are good to intake, but as with anything, excess is deleterious
to our health. $x_{2}$ and $x_{3}$ contribute negatively - the more intake of
these foods, the worse your health becomes. The $\epsilon$ term contributes
Gaussian noise with mean $\mu=0$ and standard deviation $\sigma=0.05$. Note
that the function has no biological foundation - it is purely for data
generation in this toy example.

The data set $(x_{n})$ and $(y_{n})$ with $y_{n}=g_{\rm true}(x_{n})$ is
created by sampling (13) over $n=1000$ inputs sampled from a uniform
distribution where $-1\leq x_{n,i}\leq 1$. We plot the data set as health vs
the various inputs in each dimension in Figure 2.

Figure 2:  Each input $x_{i}$ is plotted against the output health $y$. We
want to maximize the output with respect to some constraints. Accordingly, we
see that both $x_{0}$ and $x_{1}$ maximize $y$ at $x_{0}=x_{1}=0$, while
$x_{2}$ and $x_{3}$ formally maximize $g_{\rm true}$ at $x_{2}=x_{3}=-\infty$
(though of course, our extrapolation constraint impose limits on the domain
within which we will search for a solution).

###  3.2 Finding a maximizing input

Ideally, we would like the network to learn the underlying functional
dependence (13). To this extend, we define a feed forward NN $f(\theta;x)$
that is trained via gradient descent on the data $(x_{n})$, $(y_{n})$.
Subsequently, we have a set of parameters $\hat{\theta}$ containing the
weights and biases of the NN which are conducive to a good approximation of
$g_{\rm true}$. We show the fitted NN in Figure 3.

Figure 3:  The trained NN is superimposed onto the training data for each
intake $x_{i}$. The NN describes the underlying functional dependence $g_{\rm
true}$ of the data to sufficient accuracy.

Following 2.1, we can now apply extremal learning. First we freeze the
parameters $\theta$ to $\hat{\theta}$, i.e. they are not trainable parameters
anymore. Secondly, we promote the input $x$ to trainable parameters. We now
have a second NN that has identical architecture to the original NN with the
only difference being what is considered a parameter and what is considered an
input. The extremal learning NN has no input, just trainable parameters $x$
that were the input of the previous NN.

When training the extremal learning NN, one has to provide a starting value
$x_{\rm init}$. We randomly initialize the gradient descent with an input
vector $x_{\rm init}$ from a normal distribution such that $x_{{\rm
init},i}\in[-1,1]$.

Next, we set constraints using additional loss functions defined in Section
2.2 tailored to this specific problem. We use the extrapolation loss function
(10) and penalize in the search for the optimal input if it is more than two
standard deviations away from the mean. For the maximizing loss we use (11)
with $\kappa_{2}=10$ and $\hat{\kappa}=1$. Hence, the total loss is

$\mathcal{L}=\mathcal{L}_{1}+\mathcal{L}_{2}\,,$ (14)

with

$\mathcal{L}_{1}=\frac{1}{2}\sum_{i=0}^{3}\begin{cases}(x_{i}-\mu_{i}+2\sigma_{i})^{2}&x_{i}<\mu_{i}-2\sigma_{i}\,,\\\
0&\mu_{i}-2\sigma_{i}\leq x_{i}\leq\mu_{i}+2\sigma_{i}\,,\\\
(x_{i}-\mu_{i}-2\sigma_{i})^{2}&x_{i}>\mu_{i}+2\sigma_{i}\,,\end{cases}$ (15)

and

$\mathcal{L}_{2}=\begin{cases}-10\,y+1&y<0\,,\\\
\left(y^{2}+1\right)^{-1}&y\geq 0\,,\end{cases}$ (16)

The mean and standard deviation of the data set generated in Section 3.1
numerically evaluate to 222In the limit $n\rightarrow\infty$, we expect
$\mu=[0,0,0,0]$ as the data is sampled from a uniform distribution with mean
zero. However, we see some noise remnants due to finite $n=1000$. The standard
deviation of a uniform distribution sampled over the interval $a\leq x\leq b$
is approximately $\sigma=\frac{(b-a)}{\sqrt{12}}[1,1,1,1]$, which evaluates to
$\sigma\simeq[0.577,0.577,0.577,0.577]$ in this case.

$\displaystyle\begin{aligned} \mu&\simeq[0.007,-0.028,0.005,0.006]\,,\\\
\sigma&\simeq[0.555,0.577,0.577,0.567]\,.\end{aligned}$ (17)

From here, we perform gradient descent to train the extremal learning NN via
subsequently applying (8) until we converge on a maximizing input $\hat{x}$.
We expect that the extremal NN’s output should approach a maximum which is not
the analytical maximum since we are using a numerical technique with
additional constraints arising from the loss functions. However, considering
the form of $g_{\rm true}$, there is no true maximum as decreasing $x_{1}$ and
$x_{2}$ towards $-\infty$ will continuously increase the value of $g_{\rm
true}(x)$. We should instead see that the new output of the input-optimized
model converges towards a value limited by the extrapolation loss. Indeed, we
numerically find the maximizing input to be

$\hat{x}\simeq[-0.167,-0.0861,-1.193,-1.153]\,,$ (18)

which is reasonably close to the true optimal value under the limits of the
extrapolation loss we have chosen:

$\hat{x}_{\rm
true}=[0,0,-2\sigma_{2},-2\sigma_{3}]\simeq[0,0,-1.154,-1.134]\,.$ (19)

Similarly, the optimal output value calculated via extremal learning
$\hat{y}\simeq 1.702$ is reasonably close to the true optimal value of the
underlying function in this optimization problem $\hat{y}_{\rm
true}=1+2\sigma_{2}-e^{-2\sigma_{3}}\simeq 1.832$

##  4 Conclusions

In this paper, we have introduced extremal learning which allows to calculate
extremizing inputs of a trained NN in regression problems. This is a pressing
issue in many applications of modern machine learning, where one is not just
interested in inference, i.e. making predictions from a trained NN, but also
finding the input vector that extremizes a certain output. Examples are
finding the optimal cancer treatment after an NN has been trained that
represents how the patient’s tumor size responds to different cancer
treatments, or the toy example discussed in Section 3: what is the healthiest
diet for an individual based on an NN that evaluates health as a function of
the intake of various food classes. There are many other real-world examples
where solving extremization questions of this kind are very valuable.
Analytically solving for extrema is generally not a feasible option as one is
dealing with a coupled system of nonlinear equations of high dimensionality,
with potentially recursive structure lstm , due to the generally complicated
underlying functional dependence defining NNs.

Extremal learning relies on the same NN infrastructure created to train the NN
network in the first place to perform the optimization task of extremization.
This way we can take advantage of how machine learning overcomes the curse of
dimensionality which also plagues the task of extremization. The basic
components of extremal learning are freezing the parameters such as weights
and biases of the original NN and promoting the input vector $x$ to a
trainable variable. Via choosing an appropriate loss function, common machine
learning optimization techniques such as gradient descent and backpropagation
Rumelhart:1986 ; lecun2012efficient  can then be used to compute the
extremizing input. Common machine learning frameworks such as TensorFlow have
sufficient flexibility to perform this task with minimal configuration effort,
see https://github.com/ZakariaPZ/Extremal-Learning. While there are certain
parallels with GANs Goodfellow2014gi ; 8253599 ; Hong2019gi  and adversarial
training 10.1145/2046684.2046692 ; Goodfellow2014gi ; 2016arXiv161101236K ,
extremal learning presents an efficient way to find extrema of NNs in
regression problems.

We also demonstrate how to incorporate further constraints on the input and/or
output vector via additional loss functions, see Section 2.2. A common feature
of many extremization tasks is to constrain the input vector not to measure
too far a distance from the data set the NN was originally trained on. The
distance from the original data set can be penalized via such an additional
loss function and, in practice, prevent the NN to look for an extremizing
input where the predictions may become unrealistic. We further demonstrate
that it is straightforward to further constrain the input or output to one’s
liking via additional loss functions.

In the future, we would like to study the numerical performance of extremal
learning in a variety of regression examples. An interesting question remains
what extremal learning can add in the context of convolutional neural
networks, in particular if it can be combined with GANs and/or adversarial
training to solve input optimization problems.

###### Acknowledgements.

We thank Sergei Bobrovskyi, Patrick González, and Sebastian Wetzel and in
particular Malte Nuhn for helpful discussions.

## References

  * (1) W. S. McCullloch and W. Pitts, _A logical calculus of the ideas immanent in nervous activity_ , _The bulletin of mathematical biophysics_ 5 (1943) 115.
  * (2) D. H. Ackley, G. E. Hinton and T. J. Sejnowski, _A learning algorithm for boltzmann machines_ , _Cognitive Science_ 9 (1985) 147 .
  * (3) D. E. Rumelhart, G. E. Hinton and R. J. Williams, _Learning representations by back-propagating errors_ , _Nature_ 323 (1986) 533.
  * (4) Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard et al., _Backpropagation applied to handwritten zip code recognition_ , _Neural Computation_ 1 (1989) 541 [https://doi.org/10.1162/neco.1989.1.4.541].
  * (5) Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, _Gradient-based learning applied to document recognition_ , _Proceedings of the IEEE_ 86 (1998) 2278.
  * (6) Y. Bengio, R. Ducharme, P. Vincent and C. Janvin, _A neural probabilistic language model_ , _The journal of machine learning research_ 3 (2003) 1137.
  * (7) N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov, _Dropout: a simple way to prevent neural networks from overfitting_ , _The journal of machine learning research_ 15 (2014) 1929.
  * (8) Y. LeCun, Y. Bengio and G. Hinton, _Deep learning_ , _Nature_ 521 (2015) 436.
  * (9) I. Goodfellow, Y. Bengio and A. Courville, _Deep Learning_. MIT Press, 2016\.
  * (10) S. Hochreiter and J. Schmidhuber, _Long short-term memory_ , _Neural Computation_ 9 (1997) 1735.
  * (11) Y. A. LeCun, L. Bottou, G. B. Orr and K.-R. Müller, _Efficient backprop_ , in _Neural networks: Tricks of the trade_ , pp. 9–48, Springer, (2012).
  * (12) M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean et al., _Tensorflow: A system for large-scale machine learning_ , in _12th Symposium on Operating Systems Design and Implementation 16)_ , pp. 265–283, 2016\.
  * (13) T. H. Trinh, A. M. Dai, M.-T. Luong and Q. V. Le, _Learning Longer-term Dependencies in RNNs with Auxiliary Losses_ , _arXiv e-prints_ (2018) arXiv:1803.00144 [1803.00144].
  * (14) X. Ma, H. Huang, Y. Wang, S. Romano, S. Erfani and J. Bailey, _Normalized Loss Functions for Deep Learning with Noisy Labels_ , _arXiv e-prints_ (2020) arXiv:2006.13554 [2006.13554].
  * (15) I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair et al., _Generative Adversarial Networks_ , _arXiv e-prints_ (2014) arXiv:1406.2661 [1406.2661].
  * (16) A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta and A. A. Bharath, _Generative adversarial networks: An overview_ , _IEEE Signal Processing Magazine_ 35 (2018) 53.
  * (17) Y. Hong, U. Hwang, J. Yoo and S. Yoon, _How Generative Adversarial Networks and Their Variants Work: An Overview_ , _arXiv e-prints_ (2017) arXiv:1711.05914 [1711.05914].
  * (18) L. Huang, A. D. Joseph, B. Nelson, B. I. Rubinstein and J. D. Tygar, _Adversarial machine learning_ , _Association for Computing Machinery_ (2011) 43–58.
  * (19) A. Kurakin, I. Goodfellow and S. Bengio, _Adversarial Machine Learning at Scale_ , _arXiv e-prints_ (2016) arXiv:1611.01236 [1611.01236].
  * (20) A. Kurakin, I. Goodfellow and S. Bengio, _Adversarial examples in the physical world_ , _arXiv e-prints_ (2016) arXiv:1607.02533 [1607.02533].
  * (21) I. J. Goodfellow, J. Shlens and C. Szegedy, _Explaining and Harnessing Adversarial Examples_ , _arXiv e-prints_ (2014) arXiv:1412.6572 [1412.6572].
  * (22) Y. LeCun, D. Touresky, G. Hinton and T. Sejnowski, _A theoretical framework for back-propagation_ , in _Proceedings of the 1988 connectionist models summer school_ , vol. 1, pp. 21–28, 1988.


# On particle dynamics near the Schwarzschild singularity

A. Radosz  Faculty of Basic Problems of Technology (Wroclaw),Wroclaw
University of Science and Technology, 50-370Wroclaw, Poland
<EMAIL_ADDRESS>A. V. Toporensky  Sternberg Astronomical Institute,
Lomonosov Moscow State University and Kazan Federal University, Kremlevskaya
18, Kazan 420008, Russia<EMAIL_ADDRESS>O. B. Zaslavskii  Department of
Physics and Technology, Kharkov V.N. Karazin National University, 4 Svoboda
Square, Kharkov 61022, Ukraine<EMAIL_ADDRESS>
###### Abstract

The problem of the speed of the objects inside the Schwarzschild black hole is
considered. The general result is that the value of the relative speed of the
objects following their non-zero angular momentum trajectories, both of
geodesic and non-geodesic character, when approaching the ultimate
singularity, tends to the value of speed of light. There is only one exception
when both objects move in the same plane and have parallel angular momenta.
This outcome appears to have a deeper sense: it reflects the anisotropic
character of the dynamics of interior of this particular black hole. The
result in question means that near the singularity, collisions of two
particles lead to an indefinitely large energy in the center of mass frame.
Aforementioned properties have their counterpart in the phenomenon of an
indefinitely large blueshift near the singularity.

black hole, horizon, peculiar velocities, flow

###### pacs:

04.20.-q; 04.20.Cv; 04.70.Bw

##  I Introduction.

The studies of the properties of the strong gravitational fields and in
particular the properties of the black holes (BH) have got in a recent decade
a significant theoretical but also experimental impact. The development of the
theoretical interest has been mainly related to the BSW-like effect: two-
particle collisions undergoing in the vicinity of the BH’s horizon would lead
to the unbounded energy release ban . The first ever picture of the BH namely,
supermassive M-87 BH 87 , gravitational waves emission following two BHs
merger merg  and the intriguing temporarily varying radiation emission of the
accretion disk of the sources of the strong gravitational field ed  \- jac
are the most important recent experimental aspects of the presence of the
strong gravitational fields.

In this broad landscape of the BHs associated phenomena, there has been the
steadily growing interest in the studies of the BH’s interior. The progress in
the understanding (mostly kinematical) phenomena undergoing inside the
Schwarzschild’s black hole has been based on two features. One is a particular
property of the black hole’s horizon: the speed of a test particle following
geodesic and approaching the horizon tends to that of light, $V\rightarrow 1$
in the static frame. The other one is an anisotropic character BH’s interior:
exterior of the Schwarzschild black hole is a static and isotropic but its
interior, also called T-”sphere” rub  is anisotropic and dynamic, whose
spatial-like section is a hypercylinder of $R^{1}\times S^{2}$ symmetry. It is
expanding longitudinally, along homogeneity direction $R$ and contracting
transversely, perpendicularly to this direction in the angular coordinates of
the sphere $S^{2}$ (see e.g. rub , dor ). Interplay of these two features has
allowed to find an interpretation of variety of seemingly contradicting
outcomes. One can invoke the case of a test object radially falling towards
the horizon, whose speed increases to $1$ with respect to the static frame,
but after crossing the horizon the speed turns out to be decreasing to zero
speed , as if the test particle being hampered inside horizon.

It is also worth mentioning that this non-monotonic behavior of a 3-velocity
is a specific property of the observers, static outside and resting inside
horizon speed  and does not appear in the frame connected with Lemaître
coordinates. Indeed, it was shown that the 3-velocity with respect to the
Lemaître frame at a horizon can take any value from $0$ to $1$, and this
velocity decreases in a monotonic way (if it is not equal to zero identically)
in the Schwarzschild black hole reaching $0$ at a singularity (if the angular
momentum is zero) Zero . Thus some kinematical properties of particles falling
into a black hole can be more easily interpreted in a coordinate system,
different from the one offered by Schwarzschild coordinates (after mutual
exchange of a spatial and temporal coordinates). Another nice feature of the
frame in question is that hypersurfaces of a constant Lemaître time are flat,
so, for example, the proper distance between two points at the same radius is
simply the difference in their static coordinates $r$ which is useful for
visualizing the properties of a free fall. The price paid for it consists in
loosing some symmetry properties of the space-time under study, since the
metric in the Lemaître frame depends upon both spatial and temporal
coordinates, turning the equations of motion into a rather cumbersome form. In
the present paper we will deal mostly with the Schwarzschild coordinate frame,
however, noticing the difference between that one and the Lemaître frame in
some aspects.

Another apparently self-contradicting example is the observation that the
speed of a uniformly accelerated test particle (that initially increases)
eventually decreases to zero value (see e.g. rad19 ). The cause in both cases
is the longitudinal expansion of this particular space-time, extremely violent
in its final stage, that brings eventually all of the objects (moving along
homogeneity axis) to the state of relative rest.

This expansion is also responsible for the for the indefinitely large Doppler
redshift near the singularity. Namely, the Doppler frequency shift $d$ is
defined as the the ratio of the frequencies, $d=\frac{\omega_{r}}{\omega_{s}}$
recorded by a receiver, $\omega_{r}$ and by a sender $\omega_{s}$. An object
falling radially from the outer space initial position’s $r_{0\text{
}}$records the radiation emitted at $r_{0}$ as redshifted and the redshift $d$
monotonically decreases, reaching value 1/2 on the horizon and then tends to
zero, when approaching to the ultimate singularity. This may be interpreted as
a conclusion about the growing darkness inside Schwarzschild black hole.

This, however, appears to be incorrect if we consider massless particles with
non-zero angular momentum. It leads to a bright ring around a singularity (see
page B-25 of TW ). A careful analytical study of red/blue shift for such
massless particles have been recently done in gr20 . Namely, it was shown that
if an object following a non-zero angular momentum geodesic perceives the
zero-angular momentum light ray incoming from the outer space sources it turns
out to be not red- but blueshifted. Moreover, an object following zero-angular
momentum trajectory records a non-zero angular momentum light ray as
blueshifted; in both cases the blueshift is indefinite near the singularity.
Therefore, the Doppler redshift described for the radially infalling objects
recording radial radiation turns out to be an exception but not the rule.
Thus, the interior of the Schwarzschild black hole does not turn to be darker,
as it has been sometimes believed (see e.g. ham , gg )) but turns out to be
getting brighter when approaching the singularity. All these results can be
rather easily explained using the properties of peculiar velocities inside a
horizon, as it was shown in flows .

Motivated by those findings, we are going to make a revision of the seemingly
well-understood properties of the time-like geodesics inside the Schwarzschild
BH. The aim of this paper is to reconsider the question of the relative speed
of the particles inside the horizon in a general case of non-zero angular
momentum trajectories of one or both of the particles. One finds that such a
relative speed may tend to the speed of light in some circumstances; then the
question of unbounded energy collision will be revised with a rather
challenging outcome. Finally, we will present an argument that the effects
described in this paper and those discussed in the context of the blueshift
are both direct outcomes of the anisotropic character of the dynamics of the
interior of Schwarzschild BH. It is also argued that although anisotropic
dynamics is a common property for all of the BHs interior, the critical
contraction, as specified below is a particular feature of the Schwarzschild
BH that causes the indefinite blueshift and, at the same time, leads to the
unbounded energy collisions. Therefore, we are going to discuss in detail the
problem: how the critical contraction affects the dynamics of the test objects
in the vicinity of the ultimate singularity (although their final fate is
beyond the scope of our considerations). We shall consider the problem of the
relative speed of the massive test objects. We examine separately two cases:
geodesic motion and motion under the action of some finite force. In doing so,
we assume that, in general, particles have non-zero angular momenta.

The paper is organized as follows. In Sec. II we define the line element and
the tetrad for resting observer inside Schwarzschild BH and we apply them
(Sec. III) for description of the kinematics of a freely falling test
particle. In Sec. IV the anisotropy of the this space-time is described.
Particle collisions are analyzed in the following three sections: V - general
setup, VI - in-plane collisions, VII - different planes collisions. In Secs.
VIII-IX the effects of action of external force are described. In Sec. X we
consider the effect of tidal forces. Discussion and final remarks are
presented in the final section.

##  II Metric, tetrad

Let us consider the black hole metric

$ds^{2}=-fdt^{2}+\frac{dr^{2}}{f}+r^{2}d\omega^{2}.$ (1)

Here, $f(r_{+})=0$, where $r_{+}$ is the radius of the event horizon. Our main
concern is the Schwarzschild metric for which $f=1-\frac{r_{+}}{r}$. We will
also discuss very briefly the case of the Reissner-Nordström metric. Inside
the horizon, the mutual role of temporal, $t$ and spatial, $r$, coordinates
interchanges. We can choose $T=-r$, $y=t$, where $-r_{+}\leq T\leq 0$,
$-\infty<y<\infty$ nov61 . Then

$ds^{2}=-\frac{dT^{2}}{g}+gdy^{2}+T^{2}d\omega^{2}\text{,}$ (2)

where $g=-f$.

In what follows, it is convenient to use the tetrad attached to a resting
observer with constant spatial, $y,\theta,\phi$ coordinates. Such an observer
follows a geodesic that has no analogue in the outer part of space-time dor .
Namely, in the coordinates ($T,y,\theta,\phi$)

$\hat{h}_{(0)\mu}=-\frac{1}{\sqrt{g}}(1,0,0,0),$ (3)
$\hat{h}_{(1)\mu}=(0,\sqrt{g},0,0),$ (4)
$\hat{h}_{(2)\mu}=(0,0,\left|T\right|,0)$

$\hat{h}_{(3)\mu}=(0,0,0,\left|T\right|\sin\theta).$ (5)

##  III Motion of a free particle

Outside the horizon there exists the time-like Killing vector that corresponds
to the time translation and it leads to the conservation of a particle’s
energy. Inside the horizon it becomes a space-like one and this leads to the
conservation of the $y$-component of momentum. The angular momentum is
conserved everywhere.

Then, the four-velocity $u^{\mu}$ of a particle that moves within the plane
$\theta=\frac{\pi}{2}$ has the form in the coordinate system (2)

$u^{\mu}=(P\text{,}-\frac{p}{g},0,\frac{L}{T^{2}})\text{,}$ (6)

where

$P=\sqrt{p^{2}+g(\frac{L^{2}}{T^{2}}+1)},$ (7)

$p=-u_{y}$ is the specific conserved momentum (the sign minus is chosen to
keep the maximum similarity with the region outside the horizon), $L=u_{\phi}$
is the specific, conserved angular momentum.

The corresponding tetrad components read

$u^{(a)}=u^{\mu}h_{\mu}^{(a)}=(\frac{P}{\sqrt{g}},-\frac{p}{\sqrt{g}},0,\frac{L}{\left|T\right|})\text{.}$
(8)

Then, one can obtain (see Sec. 8 in flows ) that the three-velocity has
components

$V^{(1)}=-\frac{p}{P}\text{,}$ (9)
$V^{(3)}=\frac{L\sqrt{g}}{\left|T\right|P}.$ (10)

The absolute value of the velocity
$V=\sqrt{\left(V^{(1)}\right)^{2}+\left(V^{(3)}\right)^{2}}$,

$V=\sqrt{1-\frac{g}{P^{2}}}.$ (11)

The Lorentz gamma factor

$\gamma=\frac{1}{\sqrt{1-V^{2}}}=\frac{P}{\sqrt{g}}.$ (12)

For given $p$ and $L$, the velocity under discussion obeys the condition

$\frac{V^{2}}{1-V^{2}}=\frac{p^{2}}{g}+\frac{L^{2}}{T^{2}}.$ (13)

If the singularity is being approached, $T\rightarrow 0$,
$g\rightarrow\infty$.

Then, if $L\neq 0$, we have

$\left|V\right|\approx 1-\frac{1}{2}\left(\frac{T}{L}\right)^{2}.$ (14)

In doing so, $V^{(1)}\rightarrow 0$, $V^{(3)}\rightarrow\pm 1$.

If $L=0$, $V^{(3)}=0$. In the limit under discussion $V^{(1)}\rightarrow 0$.

Thus, in any case $V^{(1)}\rightarrow 0$.

##  IV Geometry and dynamics

The above result can be given the following geometric interpretation. The
space-time described by the line element (2) may be referred to as a
$T$-”sphere” rub . It has got some particular properties: it is non-static,
homogeneous, finite in time extent. It has a hypercylindrical a space-like
section $V^{3}=R^{1}\times S^{2}$ with no symmetry center, open
($-\infty<y<\infty$) in radial, homogeneity direction $R^{1}$. This may be
regarded as an anisotropic cosmological model, expanding longitudinally and
contracting transversely in a two-sphere $S^{2}$ of radius $\left|T\right|$
(see also dor  \- rad19 ). Expansion along $y$ axis is finally getting
extremely violent: all of the objects are carried away in such a manner that
their ”own” speeds are getting negligible - they are finally in a relative
rest. And that is the meaning of the first of the results of the former
section: the resting (in $y$ axis) observer measures the speed of the test
object travelling along this axis as diminishing to zero, $V\rightarrow 0$, as
is seen from (13) when $L=0$ and the singularity is approached, so
$g\rightarrow\infty$.

If the velocity vector of a test object has got a transverse (to $y$ axis)
component, i.e. its angular momentum is non-zero, $L\neq 0$, it is also
carried away transversely due to the transverse contraction. This transverse
contraction is of critical character: hypercylinder $V^{3}$ collapses to the
line as the radius of the two-sphere tends to zero. All of the massive and
massless particles are carried away in the following manner. The value of the
speed of the massive test objects as measured by resting observers goes to
that of light, $V\rightarrow 1,$ and the light rays (massless test objects)
are perceived by the resting observer as indefinitely blueshifted gr20 .

Then, the following interesting question arises: what is relative speed of the
two observers depending on their angular momenta? If particles collide,
whether their energy in the center of mass frame remains finite or grows
indefinitely? In particular, it concerns particles travelling with (a)
parallel, (b) antiparallel angular momenta. These questions are considered
below.

##  V Particle collisions: general setup

Now, we consider collisions of two particles of masses $m_{1}$ and $m_{2}$ and
briefly analyze the behavior of the energy $E_{c.m.}$ in the center of mass at
the point of collision. By definition,

$E_{c.m.}^{2}=-P_{\mu}P^{\mu},$ (15)

where $P^{\mu}=m_{1}u_{1}^{\mu}+m_{2}u_{2}^{\mu}$ is the total four-momentum.
Then,

$E_{c.m.}^{2}=m_{1}^{2}+m_{2}^{2}+2m_{1}m_{2}\gamma_{12},$ (16)

where $w$ has the meaning of the relative speed, the Lorentz factor of
relative motion

$\gamma_{12}=-u_{1\mu}u^{2\mu}=\frac{1}{\sqrt{1-w^{2}}}$ (17)

should not be confused with the individual gamma factor of each particle (12).

Below, we discuss two cases separately.

##  VI Particles move in the same plane

Then, it follows from (6) and (17) that

$\gamma_{12}=\frac{P_{1}P_{2}-p_{1}p_{2}}{g}-\frac{L_{1}L_{2}}{T^{2}},$ (18)

It is instructive to describe collisions in terms of kinematics
characteristics. One can define the angle $\psi$ between particles 1 and 2
according to

$\cos\psi=\frac{\vec{V}_{1}\vec{V}_{2}}{V_{1}V_{2}}\text{,}$ (19)

where $\vec{V}_{1}\vec{V}_{2}=V_{1}^{(1)}V_{2}^{(1)}+V_{1}^{(3)}V_{2}^{(3)}$.
Then, it follows from (9), (10) that

$\cos\psi=\frac{1}{\sqrt{p_{1}^{2}+g\frac{L_{1}^{2}}{T^{2}}}\sqrt{p_{2}^{2}+g\frac{L_{2}^{2}}{T^{2}}}}(p_{1}p_{2}+\frac{L_{1}L_{2}g}{T^{2}})\text{,}$
(20) $\gamma_{12}=\gamma_{1}\gamma_{2}(1-\cos\psi)\text{.}$ (21)

Our main concern is the behavior of $\gamma_{12}$ near the singularity. For
fixed $\,L_{1},$ $L_{2}$, the absolute velocity of each particle in the limit
when the singularity is approached, can take only two values: either $V=0$ or
$V=1$ flows , rad . Below, we enumerate different sub-cases separately
depending on the angular momentum of each particle.

###  VI.1 A $L_{1}=0=L_{2}$.

Then,

$P=\sqrt{p^{2}+g}.$ (22)

For $g\rightarrow\infty$ we have

$\gamma_{12}\approx 1+\frac{w^{2}}{2},$ (23)

where

$w\approx\frac{\left|p_{1}-p_{2}\right|}{\sqrt{g}}\rightarrow 0.$ (24)

Also,

$V_{1}\rightarrow 0,V_{2}\rightarrow 0,$ (25) $\cos\psi\rightarrow
sign(p_{1}p_{2})\text{,}$ (26)

If a particle entered the interior of the horizon from its exterior, $p>0$. If
it entered from the left (mirror) region, $p<0$. Thus in a physically relevant
case when both particles came from infinity, $\psi\rightarrow 0$.

###  VI.2 B $L_{1}=0$, $L_{2}=L\neq 0$

$\gamma_{12}\approx\left|\frac{L}{T}\right|\rightarrow\infty,$ (27)
$w^{2}\approx 1-\frac{T^{2}}{L^{2}}\rightarrow 1\text{,}$ (28)
$V_{1}\rightarrow 0\text{, }V_{2}\rightarrow 1\text{,}$ (29)
$\cos\psi\rightarrow 0\text{.}$ (30)

###  VI.3 C $L_{1}L_{2}>0$

$\gamma_{12}\rightarrow\frac{L_{1}^{2}+L_{2}^{2}}{L_{1}L_{2}}\text{,}$ (31)

$w\rightarrow\frac{\left|L_{1}^{2}-L_{2}^{2}\right|}{L_{1}^{2}+L_{2}^{2}}<1,$
(32) $V_{1}\rightarrow 1\text{, }V_{2}\rightarrow 1\text{,}$ (33)
$\cos\psi\approx
1-\frac{T^{2}}{g}\frac{(p_{1}L_{2}-p_{2}L_{1})^{2}}{L_{1}^{2}L_{2}^{2}}\text{.}$
(34)

Particles move almost parallel to each other near the singularity.

###  VI.4 D $L_{1}L_{2}<0$

$\gamma_{12}\approx 2\frac{\left|L_{1}L_{2}\right|}{T^{2}}\rightarrow\infty,$
(35) $w\approx 1-\frac{T^{4}}{4L_{1}^{2}L_{2}^{2}}\rightarrow 1,$ (36)
$V_{1}\rightarrow 1\text{, }V_{2}\rightarrow 1\text{,}$ (37)
$\cos\psi\rightarrow-1\text{.}$ (38)

This means that head-on collision occurs, $\psi\rightarrow\pi$.

Now, we can summarize the results of the present section.

| $L_{1}$ | $L_{2}$ | $w$ | $\psi$
---|---|---|---|---
A | $0$ | $0$ | $0$ | $0$
B | $0$ | $\neq 0$ | $1$ | $\frac{\pi}{2}$
C | $\neq 0$ |  $\neq 0$, $L_{2}$ parallel to $L_{1}$ | separated from $1$ | $0$
D | $\neq 0$ |  $\neq 0$, $L_{2}$ antiparallel to $L_{1}$ | $1$ | $\pi$

Table 1. Types of particles collisions near the singularity

##  VII Particles move within different planes

As is well-known, in the case of conserved angular momentum a particle moves
within a plane. According to above consideration, we can choose this plane to
be $\ \theta=\frac{\pi}{2}$ for, say, particle 1. However, in general, this is
not the case for particle 2, the variable $\theta$ will be varying in time.
Will it significantly affect the results for the relative velocity and Lorentz
factor $\gamma_{12}$ near the singularity? To answer this question, we
generalize the results of the previous section. Omitting the details of
derivation, we give the corresponding formulas below. Now,

$u^{\mu}=(P\text{,}-\frac{p}{g},\frac{\sigma
Q}{T^{2}},\frac{L}{T^{2}\sin^{2}\theta})\text{,}$ (39)

where $\sigma=\pm 1$,

$Q=\sqrt{L_{tot}^{2}-\frac{L^{2}}{\sin^{2}\theta}}\text{,}$ (40)
$P=\sqrt{p^{2}+g(1+\frac{L_{tot}^{2}}{T^{2}})},$ (41)

it is implied that

$L_{tot}\geq\frac{\left|L\right|}{\sin\theta}\text{.}$ (42)

Here, the integral of motion $L_{tot}$ has the meaning of the total angular
momentum of a particle, while $L$ is its component corresponding to a variable
$\phi$. Then, for $V^{(a)}=(V^{(1)},$ $V^{(2)},$ $V^{(3)})$ one finds

$V^{((a)}=(-\frac{p}{P}\text{, }\frac{\sigma
Q\sqrt{g}}{\left|T\right|P}\text{,
}\frac{L\sqrt{g}}{\left|T\right|P\sin\theta})\text{,}$ (43)

eq. (12) is still valid but now with (41). Obviously,

$V_{\perp}=\sqrt{\left(V^{(2)}\right)^{2}+\left(V^{(3)}\right)^{2}}=\frac{\sqrt{g}}{\left|T\right|P}L_{tot}\text{.}$
(44)

We assume that for particle 1 $\theta=\frac{\pi}{2}$, $Q_{1}=0,$
$L_{1tot}=\left|L_{1}\right|$. Then, in the point of collision both particles
have the same coordinates, so $\theta=\frac{\pi}{2}$ for particle 2 as well.
It is convenient to introduce an angle $\alpha$ for particle 2. so that
$L_{2}=L_{tot}\cos\alpha$, where $\cos\alpha$ can have any sign. Then, in the
point of collision we have for particle 2

$u^{\mu}=(P\text{,}-\frac{p}{g},\frac{L_{2tot}\sin\alpha}{T^{2}},\frac{L_{2tot}\cos\alpha}{T^{2}})\text{.}$
(45)

Eqs. (18), (21) are also valid but in $P$ the quantity $L_{tot}$ appears
instead of $L$.

Now,

$\cos\psi=\frac{1}{\sqrt{p_{1}^{2}+g\frac{L_{1}^{2}}{T^{2}}}}\frac{1}{\sqrt{p_{2}^{2}+g\frac{L_{2tot}^{2}}{T^{2}}}}(p_{1}p_{2}+\frac{L_{1}L_{2}g}{T^{2}})\text{,}$
(46)
$\gamma_{12}=\frac{P_{1}P_{2}-p_{1}p_{2}}{g}-\frac{L_{1}L_{2}}{T^{2}}\text{.}$
(47)

When a singularity is approached, $V^{(1)}\rightarrow 0$ as before, while
$V_{\perp}\rightarrow 1$, so $V\rightarrow 1$ as well. Let us denote the cases
A-D depend on the $L_{1}$, $L_{2}$ in the manner similar to that in the former
section. Then, one can see that cases A and B coincide with those from Table
1. Indeed, if one of angular momenta is zero, one can choose the equatorial
plane for another particle to be $\theta=\frac{\pi}{2}$, so nothing new
happens. Obviously, case D is similar to that from Table 1. It remains to
check what happens in case C. Then,

$\cos\psi\rightarrow\frac{L_{2}}{L_{2tot}}=\cos\alpha\text{,}$ (48)

$\psi=\alpha$. Taking into account that $V_{1}\rightarrow 1$ and
$V_{2}\rightarrow 1$, we see that according to (21), in case C a new
possibility arises :

$\gamma_{12}\approx\frac{\left|L_{1}\right|(L_{2tot}-L_{2})}{T^{2}},$ (49)

so $\gamma_{12}\rightarrow\infty$ in spite of $L_{1}L_{2}>0$. Such a
possibility was absent when both particles had been moving within the same
plane (see Table 1 above). The similar phenomenon for massless particles was
discussed in h .

##  VIII Motion under the action of force

Let now some force act on a particle. Then, the equations of motion formally
retain their form but the quantities $p$ and $L$ cease to be integrals of
motion and become the functions of time. If there is an acceleration
$a^{\mu}$, one finds its tetrad components using (3) - (5) that (assuming
$\theta=\frac{\pi}{2}$)

$a^{(3)}=\left|T\right|a^{\phi}=\frac{a_{\phi}}{\left|T\right|},$ (50)
$a^{(y)}=\sqrt{g}a^{y}=\frac{a_{y}}{\sqrt{g}},$ (51)
$a^{(\hat{t})}=-\frac{a^{T}}{\sqrt{g}}=a_{T}\sqrt{g}.$ (52)

If $\xi^{\mu}$ is the Killing vector, it is easy to notice that

$\frac{d}{d\tau}(\xi^{\mu}u_{\mu})=\xi^{\mu}a_{\mu}\text{.}$ (53)

Then,

$\frac{dp}{d\tau}=a_{y}=\sqrt{g}a^{(y)}\text{,}$ (54)
$\frac{dL}{d\tau}=a_{\phi}=\left|T\right|a^{(3)}\text{,}$ (55)

where we used the same definitions $p=-u_{y}$ and $L=u_{\phi}$ as for free
particles. Now,

$\frac{p^{2}}{g}-\frac{\left(u^{T}\right)^{2}}{g}+\frac{L^{2}}{T^{2}}=-1,$
(56)

where

$u^{T}=\frac{dT}{d\tau}=\sqrt{p^{2}+g(1+\frac{L^{2}}{T^{2}})}\text{.}$ (57)

It follows from equations of motion that

$\frac{dp}{dT}=-\frac{dp}{dr}=\frac{\sqrt{g}a^{(y)}}{\sqrt{p^{2}+g(1+\frac{L^{2}}{T^{2}})}},$
(58)
$\frac{dL}{dT}=-\frac{dL}{dr}=\frac{\left|T\right|a^{(3)}}{\sqrt{p^{2}+g(1+\frac{L^{2}}{T^{2}})}}.$
(59)

It is clear from (58), (59) that $p$ and $L$ remain finite, if $a^{(y)\text{
}}$and $a^{(3)\text{ }}$are finite.

This has important consequences for the properties of velocities. In
particular, in the tetrad (3) - (5) $V^{(1)}\rightarrow 0$ and
$V^{(3)}\rightarrow\pm 1$, if $L\neq 0$ ($V^{(3)}=0$ for $L=0$). These
conclusions are valid for any finite $E$, $L$rad , so they apply to the case
under discussion as well.

##  IX When particle velocity can approach the speed of light

It follows from the above consideration that eq. (13) indeed retains its
validity, if the constants of motion $p$ and $L$ are replaced by their
momentary values $p(T)$ and $L(T).$ In turn, this has an important
consequence. For a finite acceleration, the velocity can reach the limiting
value $V=1$ only in two cases: when approaching the horizon and/or
singularity. In the first case, the right hand side of (13) diverges due to
the first term where $g\rightarrow 0$. In the second one it does so due to the
second term where $T\rightarrow 0$.

All these conclusions are obtained with the assumption that $a_{(i)}$ are
finite and hence $p$ and $L$ are finite as well. If we relax the requirement
of finiteness of $a_{(i)}$, an additional possibility opens that $V\rightarrow
1$ due to unbounded acceleration and, correspondingly, unbounded $p$ and $L$.
Thus there are three possibilities for getting $V\rightarrow 1$: (i) horizon,
(ii) singularity, (iii) infinite acceleration.

This result is valid for the velocities with respect to the Lemaître frame as
well. Moreover, it is valid with respect to a general radially free falling
system formed by particles with the specific energy $e_{0}$. It is known that
in such a general case the radial component of velocity of a particle with
specific energy $e$ is given by (see rad )

$V^{(1)}=\frac{P_{0}e-Pe_{0}}{e_{0}e-PP_{0}}$ (60)

and the angular component is

$V^{(3)}=\frac{\mathcal{L}f}{r(e_{0}e-PP_{0})}\text{,}$ (61)

where

$P_{0}=\sqrt{e_{0}^{2}-f}.$ (62)

Formulae of the present paper for the components of velocity of an individual
particle can be thought of as a particular case $e_{0}=0$ (corresponding to a
resting observer) of these general formulae. Assuming that $e$ is finite, we
see that $V^{(3)}<1$ as it should be and $V^{(3)}\rightarrow 1$ at
singularity. As for the radial component, substituting $P$ and $P_{0}$ into
the condition $V^{(1)}=1$ and considering finite $e$ we get, after a simple
algebra, that $f=0$. This means that $V^{(1)}$ can take the value $1$ at the
horizon only, provided $e$ is finite. It is worth noting that this result is
valid for any spherically symmetric static space-times since we do not specify
the function $f$. The coordinate system $e_{0}=0$ considered here becomes
singular at the horizon itself. However, discussion contained in Sec. VI
explains, why $V^{(1)}=1$ for more general systems is safe from physical point
of view (note, that unlike a singularity we have equality, not a limit here!).

##  X Motion with respect to a frame versus motion of nearby particles

Apart from the behavior of velocity with respect to a fixed frame, another
interesting question is mutual movement of nearby points. The properties of
such a motion can be very different from the motion with respect to a fixed
frame, even if one of the points considered is at rest with respect to the
coordinate system in question. The reason is that the velocity with respect to
a frame is a local entity, while a distance between two points is a space-like
variable. Indeed, consider the radial motion. The velocity along the leg of a
hypercylinder in metric (2) is known to decrease and vanish in a singularity
flows , rad . As for the distance to a nearby point, it increases as it can be
clearly seen from the form of the metrics (1), (2). At the singularity the
proper distance diverges. The picture is similar to the Big Rip cosmological
singularity, apart from the fact that the Big Rip is isotropic.

In popular books, when describing influence of tidal forces to an unhappy
observer, falling freely into a black hole, authors usually illustrate the
text by emotional pictures of an observer ”spaghettified” in the direction
toward a singularity. This has no sense in coordinates like $T$, $y$ since (i)
they are homogeneous inside a horizon, and (ii) a singularity, being space-
like and in absolute future for an observer, is not present in any of an
observer’s $T=const$ slices. By itself, ”spaghettization” does occur but has
another meaning: if we make a series of snapshots of cross-sections $T=const$
for different $T$, the object extends more and more when $T$ grows approaching
$T=0$.

Another picture arises in the Lemaître coordinates. Singularity is present in
the sections of constant Lemaître time, so the direction towards a singularity
makes sense. Since for two radially separated particles singularity occur at
different moments of the proper time $\tau$ along the trajectory, the
separation between two points reaches its finite maximum when the ”inner”
particle hits a singularity (in this context the word ”hits” is conditional
since the singularity is space-like, we use it for brevity only).

It is easy to estimate this maximum for a pure radial motion. Suppose two
particles, being at rest with respect to the Lemaître system are separated by
some distance. Let a particle move with $E=m$, so it would start its motion
from the rest at infinity. Then, it is known (see, e.g. eq. 2.3.12 of fn )
that if a particle moves from some $r$ to $r_{f}<r$, the proper time is equal
to

$\tau(r,r_{f})=(2/3)r_{g}^{-1/2}(r^{3/2}-r_{f}^{3/2}).$ (63)

In particular, the proper time between a given position $r$ and the
singularity $r=0$ is obtained from (63) if we put there $r=0$, so

$\tau(r,0)=(2/3)r_{g}^{-1/2}r^{3/2}.$ (64)

It is also worth mentioning that for such a particle the Lemaître time
coincides with the proper one (see, e.g. eq. 14 of flows ). Also, the proper
distance in this case is equal to the difference of the coordinate values of
$r$.

Let we have two such particles initially separated by the coordinate distance
$l$. We want to find the location $r_{f}$ of the ”outer” particle initially
located at $r+l$, at the moment $\tau$ when the ”inner” particle hits the
singularity. Equating $\tau(r+l,r_{f})=\tau(r,0$), assuming small $l$ and
expanding the right hand side with respect to $l/r$ we get

$r_{f}=[(3/2)l]^{2/3}r^{1/3},$ (65)

which gives for the ratio

$r_{f}/l=(3/2)^{2/3}(r/l)^{1/3}.$ (66)

Thus small absolute displacements remain small. However, relative displacement
may be arbitrary large.

As a trilling example we can consider the following situation: suppose that
different parts of human body ($l\sim 1m$) start to move geodesically after
tidal acceleration $g_{t}$ exceeds the free fall acceleration at the surface
of Earth ($g_{E}\sim 10m/s^{2}\sim 10^{-16}m^{-1}$ in natural units $c=1$).
For small $l$, $g_{t}\approx r_{g}l/r^{3}$ (see, e.g. page B-20 of TW ). Thus
free fall begins at $r=(r_{g}l/g_{E})^{1/3}$. Using these data we can estimate

$r_{f}/l\sim 60r_{g}^{1/9},$ (67)

where $r_{g}$ is expressed in meters. This indeed indicates ”spagettization”
\- the size in $r$-direction enlarges from about 100 times for a stellar mass
black hole to about 1000 times for a supermassive ($10^{9}$ solar masses) one
(note, that the dependence upon black hole mass is rather weak).

For the motion in the angular direction the situation is quite opposite.
Suppose we have a particle with a zero angular momentum, so it falls along
$\phi=0$ line, and a nearby particle does so with some small but non-zero $L$.
We know that $V^{(3)}$ of the second particle tends to $1$ when the
singularity is approached. Does this mean that the proper distance between
these two particles increase rapidly? The answer is ”no” as the direct
dependence $\phi(r)$ in the Schwarzschild metric shows (Fig.1). In this
picture we plot $V^{(3)}$ of a particle with $L=m^{2}$ inside a horizon. It
tends to $1$ near a singularity. In the same plot we show the distance from
the line $\phi=0$ to this particle (we assume that this particle crosses the
line $\phi=0$ at the horizon) which is equal to $r\phi$. This distance first
increases due to non-zero $L$ (as it would be in a flat space also), then it
starts to decrease despite growing velocity $V^{(3)}$. The contraction in the
angular direction overcomes, and the distance in the $\phi$ direction appears
to be always smaller than it would be without gravity.

This picture is qualitatively the same in the Lemaître coordinates as well.
The only difference is that $V^{(3)}$ in static coordinate always vanishes at
a horizon (this is a counterpart of the statement that radial velocity is
always $1$ at a horizon), while the analog of this value with respect to the
Lemaître system can take any value from $0$ to $1$.

As for the angle $\phi$ itself, it reaches a finite value at singularity. This
value grows with growing $L$, tending to $\pi$ for $L\rightarrow\infty$ (see
eq.21of rad19 ).

Figure 1: The angular component $V^{(3)}$ of velocity of a particle with
$L=m^{2}$ inside a horizon (green) and distance to the particle in angular
direction from the radius $\phi=0$ crossed by this particle at a horizon
(blue). The unit for $V^{(3)}$ is $c$, the unit for the distance is $r_{g}$.

##  XI Discussion and Conclusions

In this paper we have considered dynamical phenomena in the vicinity of the
singularity of the Schwarzschild space-time. In this case one can regard
horizon’s interior as an anisotropic, dynamical $V^{4}$ space-time with a
hypercylinder $V^{3}=R^{1}\times S^{2}$ space-like sections. There is a
longitudinal, $R^{1}$-expansion and transversal, $S^{2}$-contraction. Due to
extremely violent $R^{1}$-expansion in its final stage one could expect the
asymptotic state of mutual rest of all the particles moving along
$y$-direction (see e.g rad19 ). This picture has been completed by a Doppler’s
blueshift, for the case of transverse component of trajectories: a light-like,
non-zero angular momentum geodesics have been recorded blueshifted gr20 . We
have verified here the kinematics of the test particles following time-like,
non-zero angular momentum trajectories of both geodesic and non-geodesic
character. If a test particle moves along an arbitrary non-zero angular
momentum trajectory, then its speed as measured by resting observers, those
with constant spatial coordinates, approaches that of light, $w\rightarrow 1$
as $T\rightarrow 0.$ Previously, it was found that this is valid for geodesic
trajectories flows , rad . Now, we showed that this is valid for an arbitrary
finite force.

If, instead of one particle, we take the two particles following non-zero
angular momenta trajectories, their relative velocity $w\rightarrow 1$ with
only one exceptional case. It occurs if both particles move in the same plane
and have parallel angular momenta; then the value of their relative speed $w$
is smaller than that of light, $w<1$. This also happens if both particles have
zero angular momenta. Otherwise, non-zero angular momentum of a test particle
is a necessary and sufficient condition for $w\rightarrow 1.$

It should be pointed out that there exists the reason, common for both the
indefinite blueshift for the class of non-zero angular momentum light-like
geodesics and indefinite tendency of the relative speed of the particles
following their non-zero angular momentum trajectories to the speed of light
when approaching the ultimate singularity $T\rightarrow 0$ of Schwarzschild
BH’s interior. This effect is caused by a contraction in the course of highly
anisotropic dynamics of space-time. Indeed, when approaching $T\rightarrow 0,$
rub  the hypercylinder is critically contracting, i.e. the radius
$\left|T\right|$ of the two-sphere, diminishes to the zero value,
$T\rightarrow 0.$ This critical contraction carries all of the objects,
massive and massless, in such a way that the light recorded by a resting or
moving along $y$-axis observer turns out to be indefinitely blueshifted and
the speed of a test particle as measured by resting or moving along $y$ axis
observer tends indefinitely to the speed of light, $w\rightarrow 1$. When two
colliding massive particles follow their non-zero angular momenta
trajectories, then in general they experience head-on collision and their
relative speed approaches that of light. (For motion within the same plane and
the same directions of the angular momenta the effect is moderate: the
relative speed of the colliding particles is found to be smaller than the
speed of light, $w<1.$ This is an analogy of the finding in gr20  where for
motion within the same plane and the same directions of the angular momenta of
the observer and the light a finite blueshift is found).

The result $w\rightarrow 1$ may be regarded as a center of mass energy
collision tending to infinity. This interpretation provides a particular
perspective. All of the variety of the BSW effect, unbounded energy collisions
in the vicinity of the black hole horizons, outer or inner, have lead to the
conclusion about arbitrary large limit which, however, is not reached in any
particular collision, so an infinite limit cannot be realized. This is called
a principle of kinematic censorship cens . Meanwhile, in the case under
discussion this principle is violated when $T\rightarrow 0$ ($r\rightarrow
0$). This is probably quite natural since in the singularity itself all known
laws of physics can be violated and geometry as such ceases to exist.

Since exact vanishing of angular momentum and exact coinciding of planes of
motion for two particles represent zero-measure set of initial conditions and
cannot be exactly satisfied in any realistic physical situation, we can
conclude that tending $w$ to $1$ at a singularity is unavoidable.
Correspondingly, indefinite growth of $E_{c.m.}$ is general feature for
particle collisions near the singularity.

It is also shown for spherically symmetric space-times of a quite general form
that a particle velocity can approach the speed of light only in three cases:
(i) on the horizon, (ii) in the singularity, (iii) when a proper acceleration
diverges.

##  XII Acknowledgement

The work of AT is supported by the Program of Competitive Growth of Kazan
Federal University and by the Interdisciplinary Scientific and Educational
School of Moscow University in Fundamental and Applied Space Research. O. Z.
thanks H. V. Ovcharenko for useful discussion.

## References

  * (1) M. Bañados, J. Silk and S.M. West, Kerr black holes as particle accelerators to arbitrarily high energy, Phys. Rev. Lett. 103, 111102 (2009). arXiv:0909.0169
  * (2) The Event Horizon Telescope Collaboration: First M87 Event Horizon Telescope Results. I. The shadow of the supermassive black hole. 2019 ApJL 875 L1.
  * (3) B. P. Abbott, R. Abbott, T. D. Abbott, M. R. Abernathy, F. Acernese, K. Ackley, C. Adams et al. Observation of gravitational waves from a binary black hole merger. Phys. Review Lett. 116, 061102 (2016). arXiv:1602.03837
  * (4) M. Brightman et al., Breaking the limit: Super-Eddington accretion onto black holes and neutron stars, Bull. Am. Astron. Soc. 51, 352 (2019). arXiv:1903.06844
  * (5) P.C. Fragile, S.M. Etheridge, P. Anninos, B. Mishra and W. Kluźniak, Relativistic viscous radiation hydrodynamic simulations of geometrically thin disks. Part I. Thermal and other instabilities, Astrophys. J. 857, 1 (2018). arXiv:1803.06423.
  * (6) D. Farrah et al., Stellar and black hole assembly in z $<$ 0.3 infrared-luminous mergers: intermittent starbursts versus super-Eddington accretion, Mon. Not. Roy. Astron. Soc. 513, 4770 (2022) arXiv:2205.00037
  * (7) J.E. Jacak, Quantum contribution to luminosity of quasars, JCAP10, 092 (2022). arXiv:2110.13651
  * (8) V.A. Ruban, Spherically Symmetric T-models in the general theory of relativity, Gen. Rel. and Grav., 33, 375 (2001)
  * (9) R. Doran, F.S. Lobo, P. Crawford, Interior of a Schwarzschild black hole revisited. Found. Phys. 38, 160 (2008). arXiv:gr-qc/0609042
  * (10) A. T. Augousti, P. Gusin, B. Kuśmierz, J. Masajada, A. Radosz. On the speed of a test particle inside the Schwarzschild event horizon and other kinds of black holes, Gen. Relat. Grav. 50, 131 (2018).
  * (11) A. V. Toporensky, O. B. Zaslavskii, Zero-momentum trajectories inside a black hole and high energy particle collisions, JCAP 12, 063 (2019). arXiv:1808.05254
  * (12) A. Radosz, P. Gusin, A. T. Augousti and F. Formalik. Inside spherically symmetric black holes or how a uniformly accelerated particle may slow down. Eur. Phys. J.C 79, 876 (2019)
  * (13) E.Taylor and J.Wheeler, Exploring Black Holes: Introduction to General Relativity (Addison Wesley Longman, 2000).
  * (14) O. B. Zaslavskii, Redshift/blueshift inside the Schwarzschild black hole, Gen. Relat. Grav. 52, 37 (2020). arXiv:1910.00669.
  * (15) A. J. S. Hamilton, G. Polhemus, Stereoscopic visualization in curved spacetime: seeing deep inside a black hole, New J. Phys. 12, 123027 (2010). arXiv:1012.4043
  * (16) L. E. Gurevich and E. B. Gliner, General relativity after Einstein. Moscow, 1972 (In Russian).
  * (17) I. D. Novikov, Sov. Astron. — AJ 5, 423 (1961) (Astron. Zh. 38, 564 (1961)).
  * (18) A. V. Toporensky and O. B. Zaslavskii, Flow and peculiar velocities for generic motion in spherically symmetric black holes, Gravit. and Cosmol. 27, 126 (2021). arXiv:2011.08048.
  * (19) H. V. Ovcharenko and and O. B. Zaslavskii, In preparation.
  * (20) A. V. Toporensky and O. B. Zaslavskii, General radially moving references frames in the black hole background, arXiv:2210.03670.
  * (21) Yu. V. Pavlov and O. B. Zaslavskii, Kinematic censorship as a constraint on allowed scenarios of high energy particle collisions, Grav. Cosmol. 25, 390 (2019). [arXiv:1805.07649].
  * (22) V.P. Frolov and I.D. Novikov, Physics of black holes (Kluwer Academic, Dordrecht, 1998)