#  Deep Graph Convolutional Networks for Wind Speed Prediction

Tomasz Stańczyk a<EMAIL_ADDRESS>Siamak Mehrkanoon
a<EMAIL_ADDRESS>aDepartment of Data Science and
Knowledge Engineering, Maastricht University, The Netherlands

###### Abstract

Wind speed prediction and forecasting is important for various business and
management sectors. In this paper, we introduce new models for wind speed
prediction based on graph convolutional networks (GCNs). Given hourly data of
several weather variables acquired from multiple weather stations, wind speed
values are predicted for multiple time steps ahead. In particular, the weather
stations are treated as nodes of a graph whose associated adjacency matrix is
learnable. In this way, the network learns the graph spatial structure and
determines the strength of relations between the weather stations based on the
historical weather data. We add a self-loop connection to the learnt adjacency
matrix and normalize the adjacency matrix. We examine two scenarios with the
self-loop connection setting (two separate models). In the first scenario, the
self-loop connection is imposed as a constant additive. In the second scenario
a learnable parameter is included to enable the network to decide about the
self-loop connection strength. Furthermore, we incorporate data from multiple
time steps with temporal convolution, which together with spatial graph
convolution constitutes spatio-temporal graph convolution. We perform
experiments on real datasets collected from weather stations located in cities
in Denmark and the Netherlands. The numerical experiments show that our
proposed models outperform previously developed baseline models on the
referenced datasets. We provide additional insights by visualizing learnt
adjacency matrices from each layer of our models.

###### keywords:

Wind speed prediction, deep learning, graph convolutional networks, spatio-
temporal convolution, adjacency matrix visualization

††journal: arXiv

##  1 Introduction

Accurate weather forecasting is important for agriculture, transportation and
management as it influences decision making processes and optimal settings in
these sectors. Weather prediction can also be used to predict natural
disasters, e.g. heatwaves or hurricanes and therefore contribute to saving
human lives. Moreover, accurate weather elements prediction can shed light in
analysis regarding the climate change [1].

Previous approaches involve using numerical weather prediction (NWP) [2]. NWP
uses physical assumptions about the weather elements and mathematically models
the atmosphere as a fluid. Weather variables are then predicted by means of
simulation of partial differential equations [3]. However, this approach
requires high computing power and processing the data might take up to several
hours [4]. Furthermore, due to the weather assumptions mentioned above as well
as the noise present in the data, the accuracy of NWP can degrade over time
[5]. Machine Learning data driven based models on the other hand, do not make
any assumptions about the atmosphere. Instead, these data driven based models
take historical weather data as an input and train a model aiming to predict
the target values for some time step ahead as an output.

In recent years literature has witnessed the success of deep machine learning
models in many application domains [6, 7, 8, 9, 10, 11, 12, 13, 14]. In
particular, deep Learning based models learn their own features during the
training, so that the features are the most optimal for network and they do
not require to be hand-crafted [6]. Deep Learning has already been used for
predicting climate data [15, 16, 17]. Further, deep Learning methodologies
based on Convolutional Neural Networks (CNNs) have already been successfully
applied to the weather forecasting problem [18, 19]. However, CNN-based
approaches do not incorporate spatial relations between the weather stations.
The authors in [18], casted the historical data in a tensorial format (weather
stations, weather variables, time steps) which was then passed to the model
and convolution operation was performed over the data volume. In this way, the
neighborhood relation between the weather stations is only based on their
order in the dataset.

Graph Convolution Networks (GCNs) can generalize CNNs to work on graphs rather
than on regular grids [20]. In particular, it enables incorporating the
neighbor relation information, e.g. through an adjacency matrix of a graph.
GCNs have already been applied to various domains, e.g. computer vision [21,
22], natural language processing [23, 24] or natural sciences [25, 26]. The
application of GCNs to weather prediction has not been yet extensively
discovered.

In this work, we treat weather stations and their corresponding weather
variable values from different time steps as a spatio-temporal graph, as
presented in Fig 2(b). Here, we develop our own novel models from ST-GCN [27]
and 2s-AGCN [28] architectures which were successful on skeleton-based action
recognition tasks. Our models take as an input tensor data containing values
of different weather variables (e.g. temperature, wind speed, air pressure
among the others) for several cities and historical time steps and return
values wind speed for selected cities and for selected time step ahead.
Furthermore, the adjacency matrix containing the values of relations between
the cities is optimized together with other parameters during the training
phase. In this way, the network is able to learn and decide about the city
relations based on the given training data.

The proposed models are trained on two datasets containing the weather data
from Danish and Dutch datasets. These data have been previously introduced in
[18] and [19]. We evaluate our models using Mean Absolute Error and Mean
Squared Error, which are the original metrics used for evaluation of the
baseline models [18, 19]. The obtained results show that the performances of
our models surpass those of the baseline models. Prediction plots and error
values are provided. Furthermore, since the learnable adjacency matrix is the
only element deciding about the relations between the cities, we visualize
what network has learnt and provide some interpretation insights.

This paper is organized as follows. Related work is provided in Section 2.
Section 3 presents the proposed models and relevant details. Section 4
describes the conducted experiments with corresponding results and Section 5
includes discussion. Finally, conclusions are drawn in Section 6.

##  2 Related Work

Weather forecasting has already been approached by deep learning based
methodologies. Although Long Short-Term Memory (LSTM) recurrent networks [29],
work well for time series prediction [30], they do not include spatial
relations within the data. Authors in [31] combined LSTM with convolutions to
create ConvLSTM, which was used for precipitation tasks, while also capturing
the data spatial structure. The authors in [18] introduced models based on
1D-CNNs for multi-source 1D data, 2D- and 3D-CNNs to process the tensor-form
3D data. Spatial-temporal relations were extracted with 2D and 3D CNNs. The
authors in [19] proposed using depthwise-separable convolutions to process
different dimensions of the input tensor and then concatenating resulting
tensor along single dimension. However, all the CNN-based models above discard
the spatial relations between the cities (weather stations). In other words,
these approaches process input tensor, where neighborhood of the cities is
determined only by the order of the cities in the tensor or the dataset.

GCN based approaches on the other hand can process non-grid data [20] and
therefore include spatial relations between weather stations forming a graph.
The authors in [32] introduced weighted graph convolutional LSTM architecture,
which combines LSTM with matrix multiplications replaced with graph
convolutions with a single (one for the whole model), learnable adjacency
matrix. In [32] temperature and wind speed values were predicted. In [33], the
authors proposed an architecture which combined latent representation of a
graph coming from the encoder with output from an LSTM structure and
temperature values were predicted. In [34], the authors created a graph based
on the wind farms and for each node of the graph, temporal features were
extracted with LSTMs. Their Graph Convolutional architecture was used for
predicting wind speed.

In [35] the authors introduced spatio-temporal graph convolutional networks
(STGCN) to perform traffic forecasting. GCNs were used to extract spatial
features and gated CNNs were used to extract temporal features. In [27], the
authors introduced spatial temporal graph convolutional networks (ST-GCN, not
to be confused with already mentioned STGCN) for skeleton-based action
recognition. Adopted version of graph convolution from [36] was used as
spatial convolution and standard 2D convolution as temporal convolution. The
authors in [27] used fixed graph with importance weighting. Further, authors
in [28] expanded the idea of ST-GCN and created a two-stream architecture with
adaptive graph convolutional networks (2s-AGCN). Similar approaches as for ST-
GCN were applied, yet with enabling the adjacency matrix to be partially fixed
and partially learnable, and with no transformations included from [36]. The
discussed approaches, using graph convolutional methodologies, reached
remarkable performance in their target domains [35, 27, 28].

Based on the methodologies discussed above, we develop new novel models and
apply them to the task of weather forecasting to predict wind speed in the
selected cities (weather stations) in Denmark and The Netherlands. The
proposed models are presented in Section 3.

##  3 Methods

In this work we experiment on two Dutch and Danish datasets.111The datasets
used within the scope of this work are described more in detail in Section 4.
The data consists of historical observations (time steps) for several Dutch
and Danish cities and several weather variables. The aim is to predict the
wind speed values for selected cities some time step ahead. At a single time
step, we treat the input cities as a graph where each city is a node in the
graph. Node attributes are then weather variables such as temperature, wind
speed and air pressure (for the first layer of the network) and features
encoded by the network (in the next layers). For the cities shown in Fig. 1(a)
, at a single time step, the corresponding spatial graph could be perceived as
in Fig. 2(a).

(a)  (b)

Figure 1: Map with the weather stations including the data measured for the
processed datasets: (a) Danish cities, (b) Dutch cities.

(a)  (b)

Figure 2: Possible: (a) spatial graph; (b) spatial-temporal graph; built from
Danish weather stations.

In practice, time series of historical data involve multiple time steps.
Therefore, we include this information by expanding the previous graph into a
spatial-temporal graph as shown in Fig. 2(b). Each node in the spatial graph
has now its corresponding temporal neighbors, in other words, feature values
for the given city at previous and/or next time step. In this way, information
from multiple time steps is incorporated in the graph.

Two types of convolutions are included in our proposed models, i.e. graph
convolution (spatial convolution) and temporal convolution. In what follows,
we describe the two involved convolutional operations.

###  3.1 Spatial Convolution

Spatial convolution aggregates the information from the spatial neighbors of
the graph. The graph is represented by an adjacency matrix. When binarized,
the adjacency matrix determines whether there is a connection between the
particular nodes in the graph. If the elements of Adjacency matrix are not
binarized, then in this case they denote strength of the relation between the
nodes. Following the idea from [32] and the design from [28], we make the
adjacency matrix learnable. During the optimization, the network learns the
graph spatial connections between the weather stations. It should be noted
that since all the entries are learnt in an end-to-end fashion, the adjacency
matrix is not symmetric. The learnable adjacency matrix is further transformed
during the training. Operations similar to those from GCNs of [37] and [36],
involving adding the self-loop connection and normalization with degree matrix
are applied. To the best of our knowledge, it is the first time to apply such
transformations over the adjacency matrix which is learnable.

The new, transformed matrix is created as follows. Self-loop connection in the
form of an identity matrix is added to the learnable adjacency matrix:

$\hat{A}=A+I.$ (1)

The resulting matrix $\hat{\text{A}}$ is then scaled as follows:

$\hat{A}=\frac{\hat{A}-\hat{A}_{min}}{\hat{A}_{max}-\hat{A}_{min}}.$ (2)

A diagonal node degree matrix $\hat{\text{D}}$ is computed based on the
normalized matrix:

$\hat{D}_{ii}=\sum_{j}\hat{A}_{ij}.$ (3)

Next, we apply the symmetric normalization [37, 36] with the degree matrix:

$\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}.$ (4)

The resulting transformed matrix is used for the graph convolution operation.
Input data tensor $\mathcal{X}_{in}$ with the shape of $C\times T\times V$,
where $C=\\#channels$ (features), $T=\\#timesteps$ and $V=\\#graph\ vertices$
(cities) is reshaped into a matrix $X_{in}$ with dimension of $CT\times V$.
The graph convolution is initially performed by multiplying the reshaped input
matrix with the transformed adjacency matrix:

$X_{out}=X_{in}(\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}).$ (5)

The adjacency matrix is the only element controlling the incorporation of the
information from multiple vertices. Briefly, the learned and transformed
values in the adjacency matrix entries decide how much of information coming
from one weather station has to be included in the processed information of
another weather station. Next, the output matrix $X_{out}$ is reshaped back
into a tensor $\mathcal{X}_{out}$ with the shape of $C\times T\times V$.
Finally, simple $1\times 1$ 2D convolution is performed in order to add linear
combinations of features channel-wise and to change the number of channels.

Additionally, we create a separate alternative model with a learnable scalar
parameter, which is multiplied with the identity matrix in the transformation
process:

$\hat{A}=A+\gamma I.$ (6)

In this manner, we let the network decide about the need and importance of the
self-loop connection, which is originally imposed in Eq. 1. The other elements
in the alternative model remain the same as for the first proposed model.

###  3.2 Temporal Convolution

Temporal convolution aggregates the information from the temporal neighbors in
the graph. For each node and its features, information from the next and/or
previous time step is included. The temporal convolution is implemented as a
regular 2D convolution with filter of size $k\times 1$. The value of $k$ is
set to 3, as this value was experimentally found to be optimal value for our
models regarding their performance on the used datasets. The temporal
convolution includes information from only one node at a time and for three
($k=3$) time steps and all the features (node attributes).

###  3.3 Spatio-Temporal Blocks

Using spatial convolution and temporal convolution operations, we create a
spatio-temporal block (ST-block). As presented in Fig. 3, both spatial and
temporal convolution are followed by a batch normalization (BN) layer and a
ReLU activation function. Spatial convolution is performed before temporal
convolution. In addition, a residual connection is added over the whole block.
Each block contains its own, separate, learnable adjacency matrix within
spatial convolution, as described above.

Figure 3: Spatio-temporal block (ST-block) used in our models. Figure inspired
by [28].

###  3.4 Proposed Models

We stack three spatio-temporal blocks with the following number of output
channels: 16, 32 and 64. The last block is followed by $1\times 1$ 2D
convolution reducing the number of channels to 4 before flattening. A fully
connected (FC) layer is added at the end of the network to obtain wind speed
predictions for for selected output cities. The model architecture is
presented in Fig. 4. We call our first proposed model WeatherGCNet. As stated
in Section 3.1, we also create an alternative model with an additional
learnable parameter $\gamma$ inside the spatial convolution block. This model
is refered to as WeatherGCNet with $\gamma$.

Figure 4: Overview of the proposed model(s). ST-block stands for the spatio-
temporal block. The numbers above the blocks indicate the number of input and
output channels respectively. The number of output neurons is specific for the
Danish cities dataset (3 output cities).

The following setup is applied for the training procedure. Batch size is set
to 64. Adam optimizer is used with the default value of learning rate set to
0.001. The number of input timesteps $T$ processed by our models is set to 30.
The details of the used datasets as well as the evaluation are described in
the next section. The implementation of our proposed architecture and the
trained models are available on
Github.222https://github.com/tstanczyk95/WeatherGCNet

##  4 Experiments

We consider two datasets for the task of wind speed prediction. The data comes
from weather stations located in Denmark and the Netherlands.

The Danish dataset contains hourly measurements with the values of the
following weather variables: temperature, air pressure, wind speed and wind
direction. Each measurement includes five cities in Denmark, as presented in
Fig. 1(a). The data covers time span of years 2000-2010. The data from
2000-2009 are used for the training and validation set and the data from 2010
is used for the test set. The dataset is normalized using min-max
normalization based on values coming from the training set. One sample of the
dataset contains values of all aforementioned weather variables, for all
cities and for user-defined-number of historical time steps. In this study,
the target is set to wind speed in three cities: Esbjerg, Odense, Roskilde.
The input is the tensor with the shape of $C\times T\times V$, where
$C=\\#channels$ (weather variables), $T=\\#timesteps$, $V=\\#cities$ (treated
as vertices in the graph). This dataset is available
online.333https://sites.google.com/view/siamak-mehrkanoon/code-data

The Dutch dataset contains hourly measurements of the following weather
variables: wind speed, wind direction, temperature, dew point, air pressure
and rain amount. Each measurement include seven cities in the Netherlands as
presented in Fig. 1(b). The data covers the time period between January 1,
2011 and March 29, 2020. Samples from January 2011 to December 2018 are used
for the training and validation set, and samples from January 2019 to March
2020 are used for the test set. The data is normalized using min-max
normalization before it is fed to the network. The input data arrangement is
the same as for the Danish weather stations, with the shape of $C\times
T\times V$. The target is wind speed for all weather stations. This dataset is
available online.444https://github.com/HansBambel/multidim_conv

We evaluate our models together with the other models presented in [19]. The
referenced models are trained and evaluated on the same datasets using the
same data splits as used for our models. Analogously as in [19], we report
mean absolute error (MAE) and mean squared error (MSE) for each model on the
test set. The reported MAEs and MSEs are obtained by taking the average over
all output cities. The MAE and MSE are defined as follows:

$MAE=\frac{\sum_{i=1}^{n}|y_{i}-\hat{y_{i}}|}{n},$ (7)
$MSE=\frac{\sum_{i=1}^{n}(y_{i}-\hat{y_{i}})^{2}}{n}.$ (8)

Here, $n$ denotes the number of samples in the test set. $y_{i}$ and
$\hat{y_{i}}$ denote ground truth and predicted value respectively. For the
Danish dataset, we train the models for predicting wind speed over 6, 12, 18
and 24 hours ahead. The obtained MAE and MSE on the test set for wind speed
are tabulated in Table 1. The best MAE and MSE are underlined for each
prediction time. The scores of the baseline models are taken from the original
paper [19].

Table 1: The average MAEs and MSEs of evaluated models for wind speed prediction over the Danish cities dataset. | MAE | MSE
---|---|---
Model | 6h ahead | 12h ahead | 18h ahead | 24h ahead | 6h ahead | 12h ahead | 18h ahead | 24h ahead
2D | 1.304 | 1.746 | 1.930 | 2.004 | 2.824 | 5.088 | 6.120 | 6.610
2D + Attention | 1.313 | 1.715 | 1.905 | 1.950 | 2.885 | 4.896 | 5.933 | 6.201
2D + Upscaling | 1.307 | 1.723 | 1.858 | 1.985 | 2.826 | 4.931 | 5.639 | 6.474
3D | 1.311 | 1.677 | 1.908 | 1.957 | 2.855 | 4.595 | 5.958 | 6.238
Multidimensional | 1.302 | 1.706 | 1.873 | 1.925 | 2.804 | 4.779 | 5.773 | 6.066
WeatherGCNet | 1.279 | 1.638 | 1.777 | 1.869 | 2.698 | 4.407 | 5.148 | 5.641
WeatherGCNet with $\gamma$ | 1.267 | 1.616 | 1.767 | 1.853 | 2.684 | 4.285 | 5.096 | 5.566
|  |  |  |  |  |  |  |
Table 2: The average MAEs and MSEs of evaluated models for wind speed prediction over the Dutch cities dataset. | MAE | MSE
---|---|---
Model | 2h ahead | 4h ahead | 6h ahead | 8h ahead | 10h ahead | 2h ahead | 4h ahead | 6h ahead | 8h ahead | 10h ahead
2D | 8.18 | 10.08 | 12.03 | 13.15 | 14.51 | 118.35 | 179.38 | 253.60 | 303.28 | 369.16
2D + Attention | 8.10 | 10.09 | 11.83 | 13.10 | 14.13 | 116.09 | 180.69 | 247.22 | 300.41 | 351.26
2D + Upscaling | 8.24 | 10.22 | 11.83 | 13.74 | 14.80 | 120.65 | 183.87 | 248.85 | 332.87 | 387.92
3D | 8.05 | 10.15 | 11.93 | 13.01 | 14.24 | 115.17 | 183.55 | 251.11 | 294.39 | 355.44
Multidimensional | 8.10 | 10.03 | 11.46 | 12.79 | 13.81 | 115.92 | 178.39 | 228.97 | 283.13 | 336.29
WeatherGCNet | 7.96 | 9.97 | 11.16 | 12.30 | 13.33 | 111.83 | 174.55 | 219.45 | 265.71 | 309.82
WeatherGCNet with $\gamma$ | 7.97 | 9.74 | 10.99 | 12.44 | 13.55 | 113.32 | 168.08 | 212.30 | 272.92 | 319.05
|  |  |  |  |  |  |  |  |  |

It can be observed that the proposed models, WeatherGCNet and WeatherGCNet
with $\gamma$, outperform all the other baseline models in both MAE and MSE
scores. This is due to the fact that our models learn and incorporate the
spatial neighbor relations between the weather stations based on a graph form
rather than tensorial form.

For the Dutch dataset, we train the models for predicting wind speed over 2,
4, 6, 8 and 10 hours ahead. The obtained MAE and MSE scores for the prediction
of wind speed are tabulated in Table 2. The relevant scores of the baseline
models are initially unavailable. We use the authors’ publicly available code
implementation to perform appropriate training and collect the corresponding
MAE and MSE scores.

Similarly as for the Danish dataset, it can be seen that both, WeatherGCNet
and WeatherGCNet with $\gamma$, outperform the other baselines for the Dutch
dataset.

Fig. 5 shows the performance of the baseline models and our proposed models
per weather station. The scores (MAE errors) are averaged over all considered
prediction time steps. From Fig. 5, one can observe that our proposed models,
WeatherGCNet and WeatherGCNet with $\gamma$ perform better than the baselines
for each city.

(a)  (b)

Figure 5: Test MAE of the models per city averaged over all considered
prediction time steps.

Fig. 6(a) and 6(b) show plots of wind speed prediction obtained by the
proposed model for the selected Danish City Roskilde. Fig 7(a) and 7(b) show
wind speed prediction for selected Dutch city Maastricht. For all the above-
mentioned figures, WeatherGCNet (without $\gamma$) is used. It should be noted
that a subset of test data is used for plotting whereas MAE scores included in
the figures are reported for the whole test set. The MAE scores indicate
average absolute differences between model predictions and corresponding
ground truth values in the included units for expressing wind speed values
(see the vertical axes of Fig. 6 and 7). E.g. the MAE score of $7.8546$
presented in 7(a) means that on the average (all samples in the test set)
absolute difference between the model predictions and ground truth values is
$7.8546$ of $0.1m/s$ and thus it is equivalent to $0.78546m/s$.

(a)  (b)

Figure 6: WeatherGCNet (without $\gamma$) wind speed predictions for Roskilde
(DK) for: (a) 6h ahead, (b) 24h ahead.

(a)  (b)

Figure 7: WeatherGCNet (without $\gamma$) wind speed predictions for
Maastricht (DK) for: (a) 2h ahead, (b) 10h ahead.

##  5 Discussion

The proposed graph convolutional architectures outperform the referred
baselines for the studied datasets in all cases examined. The obtained results
show that by treating the weather stations as the nodes in a graph, the
proposed models better learn the underlying spatial relations between the
weather stations which result in improving the prediction accuracy. In the
case of referenced baseline models [19], the convolution operations are
performed over the input tensor. Neighborhood and relation between the weather
stations are determined by the order of the items in the dataset. On the other
hand, with the proposed graph based approach, one can use an adjacency matrix
to define the relations between the cities, the number of neighbors and the
strength of each relation. In our proposed models we make the adjacency matrix
learnable, so that the network can decide about the strength of the relations
on its own based on the historical weather data observations. From Tables 1-2,
one can observe that the approach with learnable adjacency matrix indicates
the superiority of the proposed models over the previous baseline models.

We also visualize the learnt (and transformed as described in Section 3)
adjacency matrices of the proposed models for the 2h ahead wind speed
prediction for the Dutch cities dataset. In particular, Fig. 8(a,c,e) show the
visualization of the adjacency matrix in first, second and third spatio-
temporal layer of WeatherGCNet respectively. Fig. 8(b,d,f) show analogous
visualization, but for WeatherGCNet with $\gamma$. Fig. 8(a,c,e) correspond to
the adjacency matrix when we are enforcing the self-connection. Fig. 8(b,d,f)
correspond to the adjacency matrix when we allow the network to actually
decide about the self-connection strength. Given the possibility to decide via
the additional parameter $\gamma$, the network prefers the self-loop
connections to be relatively small (compared to the other entries), as visible
in Fig. 8(b,d,f).

(a)  (b)

(c)  (d)

(e)  (f)

Figure 8: Adjacency matrix visualization for wind speed 2h ahead prediction on
Dutch cities dataset. (a,c,e): The visualizations for WeatherGCNet. (b,d,f):
The visualizations for WeatherGCNet with $\gamma$. Here, layers 1, 2 and 3 of
the models are presented respectively.

The multiplication with adjacency matrix (Section 3.1, Eq. 5) is the only
mechanism in the whole model incorporating the information from multiple (in
fact from all) graph nodes. All the other operations, e.g. temporal
convolution, residual connection process the data independently for each node.
Each entry of adjacency matrix controls how much of information coming from
one node is to be included in the information constituting another node,
considering all node attributes. Therefore, it can be perceived that the
adjacency matrix determines relation between the nodes of the graph (the
weather stations) based on the stored attributes - combinations of all
features at all considered time steps. We denote the learnt and transformed
adjacency matrix as $A_{t}$. E.g, in Fig.8(a), presenting the adjacency matrix
for wind speed prediction for the Dutch weather stations 2h ahead
(WeatherGCNet, layer 1), $A_{t}(7,5)$ shows graphically how much (relatively
to other relations) overall information coming from Maastricht is to be
included in the resulting information of Rotterdam in layer 1. In Fig.8(a), it
can be observed that relatively much information coming from Eelde is to be
included in resulting information of Schiphol $A_{t}(4,1)$ and relatively
little information coming from Leeuwarden is to be included in resulting
information of Eelde $A_{t}(3,4)$, etc.

##  6 Conclusion

In this paper, new models based on GCN architecture are proposed for wind
speed prediction using historical weather data. A spatial-temporal graph is
designed whose nodes are weather stations and the node attributes are weather
variables over time. Thanks to the applied spatial-temporal convolutional
operations on the built graph, the network learns the underlying relations
between weather stations through a learnable adjacency matrix of the graph.
The self-loop connection of the adjacency matrix is enforced with and without
learnable scalar parameter enabling the network to decide about the strength
of the self connectivity in the graph. The obtained results of the two
approaches, with and without learnable scalar parameter, are comparable. In
the case that the learnable scalar parameter is involved, the network sets the
self-loop connection entries to relatively low values compared with other
entries of the learnt adjacency matrix. Comparing with previously introduced
baselines, our proposed models provide more accurate predictions in all cases
examined.

## Acknowledgment

Simulations were performed with computing resources granted by RWTH Aachen
University.

## References

  * [1] B. O’Neill, M. Oppenheimer, R. Warren, S. Hallegatte, R. Kopp, H.-O. Pörtner, R. Scholes, J. Birkmann, W. Foden, R. Licker, K. Mach, P. Marbaix, M. Mastrandrea, J. Price, K. Takahashi, J.-P. van Ypersele, G. Yohe, Ipcc reasons for concern regarding climate change risks, Nature Climate Change 7 (2017) 28–37.  doi:10.1038/nclimate3179.
  * [2] A. C. Lorenc, Analysis methods for numerical weather prediction, Quarterly Journal of the Royal Meteorological Society 112 (474) (1986) 1177–1194.  doi:10.1002/qj.49711247414.
  * [3] L. F. Richardson, P. Lynch, Frontmatter, 2nd Edition, Cambridge Mathematical Library, Cambridge University Press, 2007, pp. i–iv.
  * [4] P. Bauer, A. Thorpe, G. Brunet, The quiet revolution of numerical weather prediction, Nature 525 (2015) 47–55.  doi:10.1038/nature14956.
  * [5] M. Tolstykh, A. Frolov, Some current problems in numerical weather prediction, Izvestiya Atmospheric and Oceanic Physics 41 (2005) 285–295.
  * [6] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (2015) 436–44.  doi:10.1038/nature14539.
  * [7] A. Krizhevsky, I. Sutskever, G. Hinton, Imagenet classification with deep convolutional neural networks, Neural Information Processing Systems 25 (01 2012).  doi:10.1145/3065386.
  * [8] Y. Bengio, A. Courville, P. Vincent, Representation learning: A review and new perspectives, IEEE transactions on pattern analysis and machine intelligence 35 (2013) 1798–1828.  doi:10.1109/TPAMI.2013.50.
  * [9] S. Webb, Deep learning for biology 554 (7693) (2018) 555–557.  doi:10.1038/d41586-018-02174-z.
  * [10] S. Mehrkanoon, J. Suykens, Deep hybrid neural-kernel networks using random fourier features, Neurocomputing 298 (02 2018).  doi:10.1016/j.neucom.2017.12.065.
  * [11] S. Mehrkanoon, Deep neural-kernel blocks, Neural Networks 116 (2019) 46–55.  doi:10.1016/j.neunet.2019.03.011.
  * [12] S. Mehrkanoon, Cross-domain neural-kernel networks, Pattern Recognition Letters 125 (2019) 474–480.
  * [13] S. Mehrkanoon, A. Zell, J. A. Suykens, Scalable hybrid deep neural kernel networks., in: ESANN, 2017.
  * [14] S. Mehrkanoon, J. A. Suykens, Regularized semipaired kernel cca for domain adaptation, IEEE transactions on neural networks and learning systems 29 (7) (2017) 3199–3213.
  * [15] S. Scher, Toward data-driven weather and climate forecasting: Approximating a simple general circulation model with deep learning, Geophysical Research Letters 45 (11 2018).  doi:10.1029/2018GL080704.
  * [16] A. G. Salman, B. Kanigoro, Y. Heryadi, Weather forecasting using deep learning techniques, 2015 International Conference on Advanced Computer Science and Information Systems (ICACSIS) (2015) 281–285.
  * [17] J. G. Fernández, I. A. Abdellaoui, S. Mehrkanoon, Deep coastal sea elements forecasting using u-net based models, arXiv preprint arXiv:2011.03303 (2020).
  * [18] S. Mehrkanoon, Deep shared representation learning for weather elements forecasting, Knowledge-Based Systems 179 (2019) 120–128.
  * [19] K. Trebing, S. Mehrkanoon, Wind speed prediction using multidimensional convolutional neural networks, arXiv preprint arXiv:2007.12567 (2020).
  * [20] S. Zhang, H. Tong, J. Xu, R. Maciejewski, Graph convolutional networks: a comprehensive review, Computational Social Networks (2019).
  * [21] J. Johnson, A. Gupta, L. Fei-Fei, Image generation from scene graphs, CoRR abs/1804.01622 (2018).  arXiv:1804.01622.
URL http://arxiv.org/abs/1804.01622

  * [22] O. Litany, A. M. Bronstein, M. M. Bronstein, A. Makadia, Deformable shape completion with graph convolutional autoencoders, CoRR abs/1712.00268 (2017).  arXiv:1712.00268.
URL http://arxiv.org/abs/1712.00268

  * [23] H. Peng, J. Li, Y. He, Y. Liu, M. Bao, L. Wang, Y. Song, Q. Yang, Large-scale hierarchical text classification with recursively regularized deep graph-cnn, Proceedings of the 2018 World Wide Web Conference (2018).
  * [24] N. Zhang, S. Deng, Z. Sun, G. Wang, X. Chen, W. Zhang, H. Chen, Long-tail relation extraction via knowledge graph embeddings and graph convolution networks, CoRR abs/1903.01306 (2019).  arXiv:1903.01306.
URL http://arxiv.org/abs/1903.01306

  * [25] A. Fout, J. Byrd, B. Shariat, A. Ben-Hur, Protein interface prediction using graph convolutional networks, in: I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Advances in Neural Information Processing Systems, Vol. 30, Curran Associates, Inc., 2017, pp. 6530–6539.
URL
https://proceedings.neurips.cc/paper/2017/file/f507783927f2ec2737ba40afbd17efb5-Paper.pdf

  * [26] T. Xie, J. C. Grossman, Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties, Phys. Rev. Lett. 120 (2018) 145301.  doi:10.1103/PhysRevLett.120.145301.
URL https://link.aps.org/doi/10.1103/PhysRevLett.120.145301

  * [27] S. Yan, Y. Xiong, D. Lin, Spatial temporal graph convolutional networks for skeleton-based action recognition, CoRR (2018).
  * [28] L. Shi, Y. Zhang, J. Cheng, H. Lu, Adaptive spectral graph convolutional networks for skeleton-based action recognition, CoRR (2018).
  * [29] S. Hochreiter, J. Schmidhuber, Long short-term memory, Neural Computation 9 (8) (1997) 1735–1780.
  * [30] S. Siami Namini, N. Tavakoli, A. Siami Namin, A comparison of arima and lstm in forecasting time series, 2018, pp. 1394–1401.  doi:10.1109/ICMLA.2018.00227.
  * [31] X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-k. Wong, W.-c. Woo, Convolutional lstm network: A machine learning approach for precipitation nowcasting, NIPS’15, MIT Press, 2015, p. 802–810.
  * [32] T. Wilson, P. Tan, L. Luo, A low rank weighted graph convolutional approach to weather prediction, in: 2018 IEEE International Conference on Data Mining (ICDM), 2018, pp. 627–636.  doi:10.1109/ICDM.2018.00078.
  * [33] S. Seo, A. Mohegh, G. Ban-Weiss, Y. Liu, Graph convolutional autoencoder with recurrent neural networks for spatiotemporal forecasting, International Workshop on Climate Informatics, 2017.
  * [34] M. Khodayar, J. Wang, Spatio-temporal graph deep neural network for short-term wind speed forecasting, IEEE Transactions on Sustainable Energy 10 (2) (2019) 670–681.  doi:10.1109/TSTE.2018.2844102.
  * [35] B. Yu, H. Yin, Z. Zhu, Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting, Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (Jul 2018).  doi:10.24963/ijcai.2018/505.
URL http://dx.doi.org/10.24963/ijcai.2018/505

  * [36] T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, CoRR abs/1609.02907 (2016).
URL http://arxiv.org/abs/1609.02907

  * [37] U. von Luxburg, A tutorial on spectral clustering, CoRR abs/0711.0189 (2007).
URL http://arxiv.org/abs/0711.0189


# Exploring Heterogeneous Metadata for Video Recommendation with Two-tower
Model

Jianling Wang  Texas A&M University<EMAIL_ADDRESS>,  Ainur Yessenalina
Amazon Inc<EMAIL_ADDRESS>and  Alireza Roshan-Ghias  Amazon Inc.
<EMAIL_ADDRESS>
(2016)

###### Abstract.

Online video services acquire new content on a daily basis to increase
engagement, and improve the user experience. Traditional recommender systems
solely rely on watch history, delaying the recommendation of newly added
titles to the right customer. However, one can use the metadata information of
a cold-start title to bootstrap the personalization. In this work, we propose
to adopt a two-tower model, in which one tower is to learn the user
representation based on their watch history, and the other tower is to learn
the effective representations for titles using metadata. The contribution of
this work can be summarized as: (1) we show the feasibility of using two-tower
model for recommendations and conduct a series of offline experiments to show
its performance for cold-start titles; (2) we explore different types of
metadata (categorical features, text description, cover-art image) and an
attention layer to fuse them; (3) with our Amazon proprietary data, we show
that the attention layer can assign weights adaptively to different metadata
with improved recommendation for warm- and cold-start items.

††copyright: rightsretained††doi: 10.475/123_4††isbn:
123-4567-24-567/08/06††conference: ACM Woodstock conference; July 1997; El
Paso, Texas USA††journalyear: 1997††article: 4††price: 15.00

##  1\. Introduction

Online video services like Amazon Prime Video add cold-start contents, i.e.
titles that do not have any watch history, on a regular basis to their
catalog. Newly added titles are typically the most interesting for users due
to the novelty factor. In fact, many users scroll the page endlessly to find
new titles that pique their interest. Thus, it is important to match these
highly desired titles to the right users as soon as possible.

Figure 1. The detail page for a video in Amazon Prime Video. There are
different types of metadata for each video, including categorical features,
synopsis and cover art.

Recommender systems based on watch history such as neural network-based
architectures (Covington et al., 2016; Rybakov et al., 2018) or matrix
factorization techniques (He et al., 2017; Koren et al., 2009) can only
generate recommendations for a set of items seen during the model training.
This hinders the recommendation to work effectively typically for many days
until enough watches have happened. Cold-start titles typically come with
different types of metadata, including categorical features (i.e., genre,
cast, release time, etc), synopses and cover art images. As in Figure 1,
metadata always shows up on the content page to characterize the title and it
helps users decide whether they are interested in a title or not. In this
work, we postulate that we can utilize the metadata of the cold-start titles
and match the right title to the right user, based on their previous watch
history. Meanwhile, the metadata can work as supplementary information and
help to infer users’ preferences on the warm-start titles.

In this work, we adopt the two-tower architecture for recommendation of videos
with heterogeneous metadata. The item and user representations are learned
from two different neural towers, which can be used to infer the user-item
preference with dot product operation. To support heterogeneous metadata, we
develop an attention fusion layer to combine metadata from different
modalities. In this work, we explore the use of three different metadata in
Amazon Prime Video: categorical features (i.e., genres, actors and directors
and release time), synopsis features, and cover art features. With the
experiments, we show how different metadata contribute to the title
representation learning and also contribute to both warm- and cold-start item
recommendation.

##  2\. Related Work

Context-aware recommender systems improve recommendations by incorporating
contextual information of the user’s decision into the recommendation process
(Gong et al., 2020; Adomavicius et al., 2005; Wang et al., 2020a). Previous
research in video recommendation benefits a lot from different types of
contextual information. In an early work, (Davidson et al., 2010) tried to
obtain the candidate videos based on co-visitation counts and ranks them
utilizing the rule-based signals. To obtain informative user representations
for candidate generation, the authors of (Covington et al., 2016) concatenated
heterogeneous user features and input them into a deep neural network. To
enable the recommendation for video with limited historic feedback, different
content-based video recommendation framework is proposed recently. For
example, (Lee and Abu-El-Haija, 2017) embedded all the videos relying on their
raw video and audio content, and recommended the similar videos located
closely on an embedding space. They concatenated heterogeneous user features
and generate user embedding with a deep neural network. In our work, we are
exploring the impacts of heterogeneous metadata with the two-tower model to
conduct efficient user modeling and video metadata modeling at the same time.

Besides modeling user preferences on warm-start videos, we also aim to explore
the feasibility of recommending the newly-available cold-start videos. In
fact, cold-start item recommendation has been explored via mapping between
metadata and the well-trained embedding (Gantner et al., 2010) or training
strategies like Dropout (Volkovs et al., 2017) and meta-learning (Lee et al.,
2019). These methods usually work on one particular type of metadata. In this
work, we are focusing on exploring the feasibility of utilizing heterogeneous
metadata in characterizing videos for cold-start recommendation.

The two-tower structure we have adopted in this work has been applied in the
industry to predict the relevance between a query and the candidate items
(e.g., news, apps and documents) (Huang et al., 2013; Elkahky et al., 2015;
Wang et al., 2019; Rybakov et al., 2018), in which one tower is used to model
the query and the other one is for the candidate item. These works show that
the two-tower model can be easily modified and extended for multi-view
learning, for cross-domain transfer learning or for supporting different
negative sampling strategies. There are also previous works utilizing the two-
tower model for user and item representation learning in recommendation
systems (Yang et al., 2020). However, those works usually rely on only one
specific type of metadata or directly concatenate various types of metadata.
We adopt the two-tower model for user-video preference prediction task and
explore the design of the item tower with different item metadata. Ultimately,
the model can generate predictions on both warm and cold-start items.

##  3\. Two-tower Model

###  3.1. Architecture

As the name implies, the model has two components: user and item towers, each
producing the corresponding embeddings, culminating in a dot product between
the two and passing through a sigmoid activation function.

User Tower. Users are represented by their watch histories in the training
time period and some additional user-level features such as their country. We
are adopting a user encoder architecture previously tested in production. Due
to the focus on item encoder in this work, we did not experiment with
different types of user encoder and have its architecture fixed. With the user
input, the watch history is firstly encoded with a position-aware attention
layer to extract the sequential patterns. The user-level features are directly
embedded with an MLP layer. Then the encoder concatenates watch history
embeddings and user feature embeddings, passing them through the three
residual blocks (He et al., 2016), which is introduced to assist in training a
deeper architecture and thus help to achieve better performance.

Item Tower. For each item appearing in the training period, we can use so-
called ID feature that is in essence the one-hot encoding for all known items.
The embedding layer of the ID feature is uniquely linked to the ID of the
video and is updated during the training process. Meanwhile, each item also
has other metadata available (genres, actors, directors, etc; synopsis, cover
art image), which can be used to generate a dense item representation. More
details are provided in the following subsections.

Preference Prediction. Given a user-item pair $(u,i)$, we can generate the
user embedding u and item embedding i with the corresponding towers. Then
following the idea of matrix factorization (Koren et al., 2009), we can use
the dot product $\textbf{u}\cdot\textbf{i}$ to approximate $u$’s preference on
item $i$. Note that we also apply the Sigmoid function $\sigma(\cdot)$ on the
dot product $\textbf{u}\cdot\textbf{i}$ as activation. In the training
process, we use cross-entropy to calculate the loss. In the prediction
process, we will predict users’ preference scores on the set of videos and
recommend top scoring titles.

Figure 2. The proposed two-tower model with heterogeneous metadata.

###  3.2. Metadata

In the following section, we will elaborate on three types of metadata used in
the item tower: categorical features, synopsis and cover art.

Categorical features is a subset of metadata listed on the product page to
characterize the videos. They include categorical features represented with a
high-dimensional binary vector.

  * •

Genre is used to indicate the topic of a title as displayed on the detail
page, which covers 27 tags and 227 subgenre tags indicating more fine-grained
topics. We use a binary vector with 227+ 27 dimensions to represent this
feature.

  * •

Actors/Directors are important factors indicating the relevance of the title
to a user. To avoid the long-tail issue, only the top-2000 actors and
directors are taken into consideration.

  * •

Maturity Rating containing 17 different levels, indicating the maturity level
of a title.

  * •

Country of Origin containing 159 different values, indicating the
country/region title is created in.

  * •

Release Year indicating the time the title was released.

  * •

Acquisition Date indicating when the title became available in Prime Video. We
set the granularity to be 1-month to convert the release dates into
categorical values.

  * •

Popularity is based on the number of views in a certain period. We use two
different values, one is based on the 2-year watching history and the other
one is based on the watching history of the most recent 60 days. We use the
total number of views to normalize the values and use the log function to
smooth the values. We apply uniform discretization transformation on the
resulting values to covert them into categorical type of input.

By concatenating the vectors generated from each of the features above, we
obtain a high dimensional vector for each of the title.

Synopsis is the short text (usually a few sentences) displayed on the detail
page which summarizes a title. It can help users to make a decision on whether
to watch a title or not. We encode the synopsis using TF-IDF-weighted sum of
the word2vec (Mikolov et al., 2013b; Mikolov et al., 2013a) embeddings of each
word.

Cover Art is the image displayed on the detail page to give a visual for the
movie, which can also hint on the content and style of the title. In order to
obtain an embedding for the cover art, we use a pre-trained 34-layer ResNet
trained on ImageNet data (He et al., 2016), and extract the activations before
the last layer.

ID is used to represent titles present during the training. Note that we do
not have IDs for cold-start users during training. During the evaluation
process, we set the ID embedding to zero for cold-start titles.

###  3.3. Attention Layer for Fusion

After obtaining the ID, categorical, synopsis and cover art embeddings, we
concatenate them all together. However, different components can have
different influences on the title representations. For example, some titles
have informative cover arts but the others may not. Thus we need to fuse the
metadata from different modalities considering their importance. We adopt the
attention mechanism (Vaswani et al., 2017; Wang et al., 2020b) to calculate
the weights for each component. Let $\alpha_{t}^{m}$ denote the attention
weight for metadata type $m$ of video $t$ and M includes all the types of
metadata that the model considering as input for the item tower. Then we will
have the pre-score $O_{t}^{m}$:

$\alpha_{t}^{m}=\frac{\exp O_{t}^{m}}{\sum_{k\in M}\exp O_{t}^{k}},\quad
O_{t}^{\textit{m}}=\textbf{z}^{T}\cdot\tanh(\textbf{P}\textbf{h}_{t}^{\textit{m}}+\textbf{b})$

where $\textbf{h}_{t}^{\textit{m}}$ represents the embedding for metadata type
$m$ of title $t$, P is the weight matrix, z is a transform vector and b is the
bias vector. By applying softmax operation on the pre-scores, we can obtain
the attention weights of different components which can sum up to be one.
After we obtaining the weights, they will be multiplied with the metadata
before the concatenation. In the end, a 1-layer perceptron is used to convert
the fused embedding to have the same size of the user representation. Given
that cold-start items haven’t shown up in the training, the ID embedding will
be a zero vector. Thus the item representation is purely based on its
metadata.

##  4\. Experiment

In this section, we will first elaborate the experiment setup and then explore
the feasibility of utilizing heterogeous metadata for both warm- and cold-
start items in video recommendation with experiments on Amazon Prime Video
streaming history.

Figure 3. Illustration of training and scoring data splitting.

###  4.1. Evaluation methodology and metrics

To evaluate the proposed model, we conduct a series of offline experiments on
real-world Amazon Prime Video data. As shown in Figure 3, during the training
process, we use the watch history of users in a two-year time period ($X$) to
predict their watch behaviors in the following two-week period ($Y$). Then, in
the scoring (testing) mode, given the watch history in the two-year period
$X^{\prime}$, it can calculate the preference scores and predict users’ watch
behavior for the following one-week time period $Y^{\prime}$. To avoid data
leakage, there is no overlap between $Y^{\prime}$ and ($X$ $\cup$ $Y$).

For each user, we calculate preference scores for a list of candidate movies
or TV shows. We rank these titles based on the predicted scores and select the
Top-K titles for different categories (i.e.,movies and series). We pick $K=6$
to simulate the use cases in Amazon Prime Video. With the top-k titles and the
ground-truth, we adopt 4 different metrics to evaluate the recommendation
performance. (1) Precision@K represents the ratio of the actual watched items
among the Top-K Recommendation; (2) Recall@K is the ratio of the actual
watched items which were uncovered by the recommendations; (3) Coverage@K
shows the number of unique items that were recommended for all users, and (4)
Converted Coverage@K to indicate the number of unique items recommended to all
users that were actually watched.

Data. We collect and split the data according to the scheme explained in
Figure 3. Specifically, the training data with density111density =
$\frac{\sharp watches}{\sharp users*\sharp items}$ of 0.00672% has the length
of 2-year in total (i.e., $X+Y=2$ years), in which the data for the last
14-day is treated as labels. For the test set, we collect user streaming
history in 2-year as input features to predict what user is going to watch in
the next 7-day (Time Period $Y^{\prime}$). To evaluate how the proposed model
works for cold-start title recommendation we did a series of offline
experiments. Here we have a set of items that have no watch history in time
period $X$, $Y$ or $X^{\prime}$ but have at least one watch in time period
$Y^{\prime}$. There are 360 movies and 75 TV series in total in this category.
In the following experiments, during the scoring/evaluation, we only calculate
preference scores for this set of cold-start titles and rank among them.

| P@6 (%) | R@6 (%) | Cov@6 | ConCov@6
---|---|---|---|---
Model | Movie | TV | Movie | TV | Movie | TV | Movie | TV
Synopsis | (B) | (B) | (B) | (B) | (B) | (B) | (B) | (B)
Cover art | 0.61 | 0.77 | 2.07 | 1.41 | -30.87 | -26.65 | -25.60 | -23.59
Categorical | 11.38 | -25.38 | 10.27 | -11.76 | 19.24 | -55.50 | 30.13 | -46.38
NCF-extention | 15.38 | 7.69 | 12.75 | 8.24 | 44.05 | 4.52 | 70.72 | 15.82
Con (w/o ID) | 13.53 | 1.54 | 12.76 | 1.88 | 33.83 | 0.31 | 36.78 | 4.29
Att (w/o ID) | 17.23 | 15.38 | 14.94 | 16.24 | 65.06 | 9..03 | 79.49 | 18.49
Con (w/ ID) | 17.85 | 18.46 | 14.95 | 19.05 | 79.92 | 25.14 | 99.15 | 39.14
Att (w/ ID) | 26.76 | 25.38 | 21.88 | 24.71 | 167.57 | 50.63 | 176.23 | 81.76

Table 1. Comparison on warm-start video recommendation. For fair comparison,
NCF-extention extends NCF (He et al., 2017) by concatenating both the watch
history embedding and user-level features embedding with the user ID
embedding. (B) indicates the basline for percentage calculation. All numbers
are reported in percentage (%) lift w.r.t. the baseline. Synopsis, Cover art
and Categorical denote the models use only categorical features, synopsis
features or cover art features; Con concatenate all types of metadata
directly, and feeds the concatenation into the 1-NN; Att means to fuse
different types of metadata using attention.

Parameter Setup. We use positive samples from a user’s watch history, and
negative samples from randomly sampling a set of items the user hasn’t watched
before. We tried different negative sampling rates ranging from 1 to 30. We
found that more negative samples can lead to better performance. Empirically,
the recommendation performance becomes stable when the negative sampling rate
is larger than 20. The embedding size before the dot product is also set to be
512. We used Adam optimizer with the learning rate of 0.001.

###  4.2. Results

####  4.2.1. Warm-start Video Recommendation Task.

To fully understand how different metadata and the proposed attention layer
work for video recommendation, we evaluate the recommendations for the warm-
start videos. The results are summarized in Table 1. First, we compare the
models using only one type of metadata. We can see that the categorical
features can result in the best offline performance under different evaluation
metrics, indicating that the categorical features are more informative in
characterizing the warm-start videos compared with other metadata. When we
fuse the aforementioned metadata, the proposed attention layer can bring in
significant improvement compared to the models using one of the metadata,
while the simple concatenation only works slightly better than those models.
This observation illustrates the effectiveness of the attention layer.
Furthermore, if we take the ID embedding into consideration, under the warm-
start setup, the models can perform better than all the models without ID
embedding. The reason is that for items with abundant watch history, the ID
embedding layer can learn an informative representation, which hints at the
necessity of ID embedding in the warm-start setup.

| P@6 (%) | R@6 (%) | Cov@6 | ConCov@6
---|---|---|---|---
Model | Movie | TV | Movie | TV | Movie | TV | Movie | TV
Random | (B) | (B) | (B) | (B) | (B) | (B) | (B) | (B)
Att (w/o ID) | 2556.52 | 385.42 | 2976.51 | 400.11 | -26.46 | -9.33 | 354.54 | 3.85
Att (w/ ID) | 1221.73 | 368.75 | 1277.35 | 329.92 | -2.78 | 0.00 | 486.36 | 57.69

Table 2. Comparison on cold-start video recommendation. (B) indicates the
baseline for percentage calculation. All numbers are reported in percentage
(%) lift w.r.t. the baseline. Random is the random recommendation strategy;
Att w/o ID is the two-tower model with Attention layer fusing all types of
metadata; Att with ID is the two-tower model considering all types of metadata
and ID embeddings, using Attention layer to fuse those metadata and ID
embeddings.

####  4.2.2. Cold-start Item Recommendation Task.

We compare the proposed two-tower model with the random baseline in which we
just randomly select 6 cold-start titles to each user. For cold-start titles,
during evaluation/scoring phase, we set their ID embedding to zero. To examine
how the ID embedding will influence the cold-start recommendation, we also
compare the two-tower model with all the metadata and the version with
metadata and ID embedding. The results are summarized in Table 2. We can see
that, both the versions with or without ID embedding can beat the random
method significantly, explaining the effectiveness of the two-tower model with
metadata. Further, the model with ID embedding falls behind the model without
ID embedding in terms of precision@6 and recall@6. The reason is that higher
weights are usually assigned to the ID embedding by the attention layer in the
training phases, and leading to a weak representation for the metadata
components. Thus if we remove the ID embedding, we can see the improvements
for the cold-start item prediction task. Note, the convertedCoverage@6 and
Coverage@6 are slightly higher for the two-tower model with ID embedding. We
need to carefully consider this tradeoff scenario while designing an
appropriate model for cold-start video recommendation.

###  4.3. Visualization

To examine how attention layer works in controlling the fusion process, in
Figure 4, we summarize the resulted weights for different types of metadata in
TV shows and movies. The results for movie and TV shows are shown
individually. We find that for both title categories, cover art feature gets
the lowest weights, and categorical features get the highest weights. This is
intuitive, since categorical data is rich with important features, whereas the
cover art is only represented by a pre-trained embedding, and not fine-tuned
for this task. When comparing TV shows and movies, we find that synopsis
features are more important for TV shows compared to movies.

##  5\. Conclusion

We proposed a two-tower model for cold- and warm-start item recommendation
with heterogeneous metadata. We also explored different types of metadata,
including categorical features, cover-art images and synopsis, With offline
experiments, we show that the proposed framework can produce best
recommendations by fusing different types of metadata using attention. By
introducing the metadata with the well-designed attention layer, the two-tower
model enables the recommendation for cold-start videos and also improve the
recommendation for warm-start videos.

Figure 4. The distribution of attention weights.

## References

  * (1)
  * Adomavicius et al. (2005) Gediminas Adomavicius, Ramesh Sankaranarayanan, Shahana Sen, and Alexander Tuzhilin. 2005.  Incorporating contextual information in recommender systems using a multidimensional approach.  ACM Transactions on Information systems (TOIS) 23, 1 (2005), 103–145.
  * Covington et al. (2016) Paul Covington, Jay Adams, and Emre Sargin. 2016\.  Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191–198.
  * Davidson et al. (2010) James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, et al. 2010\.  The YouTube video recommendation system. In Proceedings of the fourth ACM conference on Recommender systems. 293–296.
  * Elkahky et al. (2015) Ali Mamdouh Elkahky, Yang Song, and Xiaodong He. 2015.  A multi-view deep learning approach for cross domain user modeling in recommendation systems. In Proceedings of the 24th international conference on world wide web. 278–288.
  * Gantner et al. (2010) Zeno Gantner, Lucas Drumond, Christoph Freudenthaler, Steffen Rendle, and Lars Schmidt-Thieme. 2010\.  Learning attribute-to-feature mappings for cold-start recommendations. In 2010 IEEE International Conference on Data Mining. IEEE, 176–185.
  * Gong et al. (2020) Boning Gong, Mesut Kaya, and Nava Tintarev. 2020.  Contextual Personalized Re-Ranking of Music Recommendations through Audio Features. In CARS Workshop at ACM RecSys.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016\.  Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173–182.
  * Huang et al. (2013) Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013\.  Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management. 2333–2338.
  * Koren et al. (2009) Yehuda Koren, Robert Bell, and Chris Volinsky. 2009\.  Matrix factorization techniques for recommender systems.  Computer 42, 8 (2009), 30–37.
  * Lee et al. (2019) Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. 2019.  Melu: Meta-learned user preference estimator for cold-start recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1073–1082.
  * Lee and Abu-El-Haija (2017) Joonseok Lee and Sami Abu-El-Haija. 2017.  Large-scale content-only video recommendation. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 987–995.
  * Mikolov et al. (2013a) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781 (2013).
  * Mikolov et al. (2013b) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b.  Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111–3119.
  * Rybakov et al. (2018) O. Rybakov, V. Mohan, A. Misra, S. LeGrand, R. Joseph, Kiuk Chung, Siddharth Singh, Q. You, Eric T. Nalisnick, Leo Dirac, and Runfei Luo. 2018\.  The Effectiveness of a two-Layer Neural Network for Recommendations. In ICLR Workshop.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In Advances in neural information processing systems. 5998–6008.
  * Volkovs et al. (2017) Maksims Volkovs, Guang Wei Yu, and Tomi Poutanen. 2017.  DropoutNet: Addressing Cold Start in Recommender Systems.. In NIPS. 4957–4966.
  * Wang et al. (2020a) Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. 2020a.  Next-item recommendation with sequential hypergraphs. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 1101–1110.
  * Wang et al. (2020b) Jianling Wang, Ziwei Zhu, and James Caverlee. 2020b.  User Recommendation in Content Curation Platforms. In Proceedings of the 13th International Conference on Web Search and Data Mining. 627–635.
  * Wang et al. (2019) Ruoxi Wang, Zhe Zhao, Xinyang Yi, Ji Yang, Derek Zhiyuan Cheng, Lichan Hong, Steve Tjoa, Jieqi Kang, Evan Ettinger, and H Chi. 2019\.  Improving Relevance Prediction with Transfer Learning in Large-scale Retrieval Systems. In Proceedings of the 1st Adaptive & Multitask Learning Workshop.
  * Yang et al. (2020) Ji Yang, Xinyang Yi, Derek Zhiyuan Cheng, Lichan Hong, Yang Li, Simon Xiaoming Wang, Taibai Xu, and Ed H Chi. 2020\.  Mixed negative sampling for learning two-tower neural networks in recommendations. In Companion Proceedings of the Web Conference 2020. 441–447.


11institutetext: Institute for High Energy Physics, Austrian Academy of
Sciences, Nikolsdorfergasse 18,
A-1050 Vienna, Austria

# Bethe–Salpeter Bound-State Solutions:
Examining Semirelativistic Approaches

Wolfgang Lucha 11<EMAIL_ADDRESS>
###### Abstract

Within the formalism of relativistic quantum field theory an adequate
framework for the description of two-particle bound states, such as, for
instance, all conventional (i.e., non-exotic) mesons, is provided by the
Poincaré-covariant homogeneous Bethe–Salpeter equation. In applications,
however, this approach usually proves to be rather involved, whence it is not
always quite easy to extract the predictions sought. In view of this, a coarse
idea of the bound-state spectrum to be expected might be gained by adhering to
some simplifying approximations – which constitutes an entirely legitimate
first step. The reliability of the insights inferred from the arising simpler
bound-state equation may be straightforwardly examined by taking into account
a couple of rigorous constraints on the obtained discrete spectrum.
Application of these tools is illustrated for popular potentials.

##  1 Bound States of Spinless Constituents: Semirelativistic Approach

A main issue in both relativistic quantum physics and quantum field theories,
such as quantum electrodynamics and quantum chromodynamics, is to devise
appropriate (and, if manageable, convenient) approaches to the bound states
expected within specific settings. In this context, a very useful tool is
provided by the spinless Salpeter equation, i.e., the eigenvalue equation of a
Hamiltonian operator composed of the relativistic kinetic energies of the
involved bound-state constituents and some static interaction potential $V$
assumed to reflect the bound-state relevant aspects of any underlying quantum
(field) theory. If $V$ meets these expectations, this approach should return a
rough idea of the bound-state spectrum. For two bound-state constituents with
relative coordinates $\bm{x}$ and $\bm{p}$ in configuration and momentum
space, this Hamiltonian becomes

$H=\sqrt{\bm{p}^{2}+m_{1}^{2}}+\sqrt{\bm{p}^{2}+m_{2}^{2}}+V(\bm{x})$ (1)

(where $m_{1}$ and $m_{2}$ denote both interacting particles’ masses) and, for
$m_{1}=m_{2}=m$, reduces to

$\widetilde{H}=2\,\sqrt{\bm{p}^{2}+m^{2}}+V(\bm{x})\ .$ (2)

Conceptually, the spinless Salpeter equation can be formulated along two
opposite directions:

  * •

On the one hand, the spinless Salpeter equation can be understood as an
improvement of the nonrelativistic Schrödinger equation that is achieved upon
allowing, in the Hamiltonian, the kinetic part to retain its proper
relativistic form instead of sticking to its nonrelativistic limit.

  * •

On the other hand, upon starting from the Poincaré-covariant _Bethe–Salpeter
formalism_ BS , representing an appropriate framework for the description of
bound states within the realms of relativistic quantum field theories, the
spinless Salpeter equation can be easily derived by applying several
simplifying approximations to the homogeneous Bethe–Salpeter equation. The
detailed recipe reads: discard all timelike coordinates (which leads to the
_instantaneous_ Bethe–Salpeter formalism WLe ), require free propagation of
bound-state constituents (which produces the Salpeter equation SE ), drop all
negative-energy contributions (which results in what is called the _reduced_
Salpeter equation) and, finally, ignore the spin degree of freedom of all
bound-state constituents in order to eventually arrive at the _spinless_
Salpeter equation.

##  2 Analysis of Semirelativistic Hamiltonian Operators: Some Insights

The nonlocality of the Hamiltonian (1), induced by the square-root operators
of the relativistic kinetic energies, impedes an entirely analytic treatment
of any emerging bound-state problem. Nonetheless, by utilization of
instruments supplied by functional analysis a qualitative or even
semiquantitive picture of expectable solutions of the spinless Salpeter
equation may be drawn. In particular, establishing the mere existence of
discrete bound states by proving boundedness from below of the spectrum of
Hamiltonian operators is, beyond doubt, of utmost importance.

###  2.1 Spectrum of Semirelativistic Hamiltonian Operator: Boundedness from
Below

Quite generally, well-definedness of a Hamiltonian requires, among others,
this operator to be bounded from below. Noting the manifest positivity of the
relativistic kinetic-energy operator, for semirelativistic Hamiltonians – such
as those presented in Eqs. (1) or (2) – the crucial issue is the _possibly
singular_ behaviour of the interaction potentials showing up in the
Hamiltonian. In such a context, the (in some sense) worst-case scenario
originates in the Coulomb potential:

$V_{\rm C}(\bm{x})=V_{\rm C}(|\bm{x}|)\equiv-\frac{\alpha}{|\bm{x}|}\
,\qquad\alpha>0\ .$ (3)

For the spinless relativistic Coulomb problem, in turn, the spectral theory
has been thoroughly worked out IWH1 ; IWH2 : A semirelativistic Hamiltonian
(2) with a Coulomb interaction potential (3) is bounded from below if and only
if the involved coupling parameter $\alpha$ satisfies the constraint

$\alpha<\frac{4}{\pi}=1.273239\dots\ .$ (4)

Only for these strengths $\alpha$, the lower bound to the spectrum
$\sigma(\widetilde{H})$ of the Hamiltonian (2) reads

$\sigma(\widetilde{H})\geq
2\,m\,\sqrt{1-\left(\frac{\pi\,\alpha}{4}\right)^{\\!2}}\ .$ (5)

For a more refined range, $\alpha\leq 1$, a somewhat increased lower bound on
$\sigma(\widetilde{H})$ can be found MR :

$\sigma(\widetilde{H})\geq 2\,m\,\sqrt{\frac{1+\sqrt{1-\alpha^{2}}}{2}}\ .$
(6)

###  2.2 Spinless-Salpeter Hamiltonian Operators: Number of Discrete
Eigenstates ICD

Rigorous upper limits on the total number of bound states described by some
spinless Salpeter equation may be formulated ICD  in terms of (Lebesgue)
spaces of measurable functions, $L^{p}$, on the Euclidean space
${\mathbb{R}}^{3}$, for $p=3/2,3$. For a Hamiltonian operator acting on the
Hilbert space $L^{2}({\mathbb{R}}^{3})$ and a potential that is a negative
smooth function of compact support, hence, subject to

$V(\bm{x})\in C^{\infty}_{0}({\mathbb{R}}^{3})\ ,\qquad V(\bm{x})\leq 0\ ,$
(7)

in equal-mass two-particle problems (2) the total number of bound states, $N$,
is constrained by

$N\leq\frac{C}{48\,\pi^{2}}\int{\rm
d}^{3}x\left[|V(\bm{x})|\left(|V(\bm{x})|+4\,m\right)\right]^{3/2}\ ,\quad
C=\left\\{\begin{array}[]{rl}14.107590867&\quad\mbox{if}\ \ \ m>0\
,\\\\[2.15277pt] 6.074898097&\quad\mbox{if}\ \ \ m=0\ ,\end{array}\right.$ (8)

with numerical results for $C$ taken from Ref. WL:K . Finiteness forces the
potential $V(\bm{x})$ to satisfy

$V(\bm{x})\in L^{3/2}({\mathbb{R}}^{3})\cap L^{3}({\mathbb{R}}^{3})\ .$ (9)

###  2.3 Semirelativistic Hamiltonian Operators: Upper Limits on Discrete
Eigenvalues

Upon having succeeded to establish – in the delicate instance of singular
interaction potentials by rather shameless abuse of the findings gained for
the spinless relativistic Coulomb problem IWH1 ; IWH2  (reviewed in Subsect.
2.1) – the rigorous boundedness from below of the semirelativistic Hamiltonian
in one’s focus of interest, that is, of that operator’s spectrum, most likely
one will be tempted to try, as a next step, to acquire (at least, limited)
information on the actual location of its lowest-lying discrete eigenvalues.
Such goal can be achieved by, for instance, narrowing down the conceivable
range of any of these eigenvalues by finding an upper bound to its range. The
theoretical foundation of any attempt of this kind is the minimum–maximum
theorem RS :

  * •

Consider some self-adjoint operator, $H$, bounded from below, with its
(ordered) eigenvalues

$E_{0}\leq E_{1}\leq E_{2}\leq\cdots\ .$ (10)

  * •

Define the operator $\widehat{H}$ by _restricting_ $H$ to some $d$-dimensional
subspace of the domain of $H$.

  * •

Let $\widehat{E}_{0},\widehat{E}_{1},\dots,\widehat{E}_{d-1}$ be the $d$
eigenvalues of the restriction $\widehat{H}$, likewise ordered according to

$\widehat{E}_{0}\leq\widehat{E}_{1}\leq\widehat{E}_{2}\leq\cdots\leq\widehat{E}_{d-1}\
;$ (11)

  * •

these form upper bounds to the first $d$ eigenvalues of $H$ below its
essential spectrum’s onset:

$E_{k}\leq\widehat{E}_{k}\qquad\forall\quad k=0,1,2,\dots,d-1\ .$ (12)

Below, in Sect. 3, we will show slight predilection for interactions governed
by spherically symmetric potentials. If dealing with central potentials, mere
convenience dictates to span the $d$-dimensional subspace required by the
minimum–maximum theorem by a basis the elements of which are functions,
$\psi_{k\ell}(\bm{x})$, that are products of a spherical harmonic ${\cal
Y}_{\ell}$ for orbital angular momentum $\ell$ and a radial factor involving
the generalized-Laguerre polynomials $L_{k}^{(\gamma)}$ WL:T1 ; WL:T2 :

$\displaystyle\psi_{k\ell}(\bm{x})\propto|\bm{x}|^{\ell+\beta-1}\exp(-\mu\,|\bm{x}|)\,L_{k}^{(2\ell+2\beta)}(2\,\mu\,|\bm{x}|)\,{\cal
Y}_{\ell}(\Omega_{\bm{x}})\ ,\qquad k\in{\mathbb{N}}_{0}\
,\qquad\ell\in{\mathbb{N}}_{0}\ ,$ (13) $\displaystyle
L_{k}^{(\gamma)}(x)\equiv\sum_{t=0}^{k}\,\binom{k+\gamma}{k-t}\,\frac{(-x)^{t}}{t!}\
,\qquad\mu>0\ ,\qquad\beta>-\frac{1}{2}\ ;$

herein, $\Omega_{\bm{x}}$ indicates the $\bm{x}$-space solid angle, and $\mu$
and $\beta$ represent two variational parameters.

###  2.4 Spinless-Salpeter Hamiltonian Operator: Quality Assurance of its
Eigenstates

Irrespective of the actual origin of approximate eigenstates $|\chi\rangle$ of
a Hamiltonian operator, their accuracy WL:Q1 ; WL:Q2  can be judged (or even
quantified) by their degree of fulfilment of the _master_ or _relativistic_
virial theorem WL:RVT1 ; WL:RVT2  relating the expectation values of the
radial derivatives of all kinetic terms and of the potential. This relation
comprises, of course, the well-known virial theorem of nonrelativistic quantum
theory as a special case. For the operators (1), it reduces to

$\left\langle\chi\left|\frac{\bm{p}^{2}}{\sqrt{\bm{p}^{2}+m_{1}^{2}}}+\frac{\bm{p}^{2}}{\sqrt{\bm{p}^{2}+m_{2}^{2}}}\right|\chi\right\rangle=\left\langle\chi\left|\,\bm{x}\cdot\frac{\partial\,V}{\partial\bm{x}}(\bm{x})\,\right|\chi\right\rangle.$
(14)

###  2.5 Trivial Upper Bounds: Nonrelativistic Limit of Spinless-Salpeter
Hamiltonians

The (undeniable) concavity of the square-root operator of the proper
relativistic expression for the kinetic energy regarded as a function of
$\bm{p}^{2}$ implies that its nonrelativistic limit is tangent to its
relativistic precursor at the point of tangency $\bm{p}^{2}=0$. Thus, the
nonrelativistic Hamiltonian

$H_{\rm NR}=m_{1}+m_{2}+\frac{\bm{p}^{2}}{2\,\hat{m}}+V(\bm{x})\
,\qquad\hat{m}\equiv\frac{m_{1}\,m_{2}}{m_{1}+m_{2}}\ ,$ (15)

forms an upper bound to its originator (1), with expectable impact on their
sets of eigenvalues:

$H\leq H_{\rm NR}\qquad\Longrightarrow\qquad\ E_{k}\leq E_{{\rm NR},k}\
,\qquad k\in{\mathbb{N}}_{0}\ .$ (16)

With respect to the total number $N$ of bound states of the spinless Salpeter
equation, the above observation implies that this number can never be less
than its nonrelativistic counterpart WL:NR .

##  3 Informative Application to Two Spinless Semirelativistic Problems

In the past, the _effortlessness_ of applying all the findings collected in
Sect. 2 to semirelativistic bound-state problems has been demonstrated WL:WS ;
WL:Q@W ; WL:H ; WL:Y ; WL:K ; WL:Hm ; WL:R  for a number of frequently
employed, rather popular interaction potentials. Therefore, let’s take a brief
look at two of these analyses, where simplicity motivated us to confine
ourselves to spherically symmetric central potentials

$V(\bm{x})=V(r)\ ,\qquad r\equiv|\bm{x}|\ .$ (17)

###  3.1 Short-Range Singular Example: Spinless Relativistic Hulthén Problem
WL:Q@W ; WL:H

The Hulthén potential $V_{\rm S}(r)$, originally introduced in nuclear physics
but since then in physics widely used, is defined by two parameters, the
coupling strength $\eta$ and an exponential range $b$:

$V_{\rm S}(r)\equiv-\frac{\eta}{\exp(b\,r)-1}\ ,\qquad b>0\ ,\qquad\eta\geq 0\
.$ (18)

As shown in Fig. 1, the potential $V_{\rm S}(r)$ approaches, for short
distances, from above a Coulomb potential $V_{\rm C}(r)$ with strength
$\eta/b$ (i.e., exhibits Coulombic behaviour) and, for large distances, from
below a negative exponential potential $V_{\rm E}(r)$, or increases
exponentially damped to zero:

$V_{\rm S}(r)\xrightarrow[r\to 0]{}V_{\rm C}(r)\equiv-\frac{\eta}{b\,r}\
,\qquad V_{\rm S}(r)\xrightarrow[r\to\infty]{}V_{\rm
E}(r)\equiv-\eta\exp(-b\,r)\ .$ (19)

Conversely, the Hulthén potential (18) is bounded from below by the Coulomb
potential (3) iff

$\alpha\geq\frac{\eta}{b}\ .$ (20)

Consequently, the Hulthén potential (18) inevitably exhibits a Coulomb-like
singularity at spatial origin $r=0$. Nevertheless, according to Subsect. 2.1
the assigned Hamiltonian (1) and bound-state spectrum are bounded from below
if the two involved potential parameters satisfy

$\frac{\eta}{b}<\frac{4}{\pi}\ .$ (21)

Figure 1: Hulthén potential $V_{\rm S}(r)$ (solid black line), Eq. (18), and
asymptotes (19): negative exponential potential $V_{\rm E}(r)$ (dotted blue
line) and Coulomb potential $V_{\rm C}(r)$ (dashed magenta line), for strength
$\eta=1$.

For the _nonrelativistic_ limit (15) of the Hamiltonian (1) with Hulthén
potential (18) and all states of orbital angular momentum $\ell=0$, the
eigenvalues $E_{\rm NR}$ can be found analytically SF :

$E_{{\rm
NR},n}=m_{1}+m_{2}-\frac{\left(2\,\hat{m}\,\eta-n^{2}\,b^{2}\right)^{2}}{8\,\hat{m}\,n^{2}\,b^{2}}\
,\qquad\hat{m}\equiv\frac{m_{1}\,m_{2}}{m_{1}+m_{2}}\ ,\qquad
n\in{\mathbb{N}}\ ,$ (22)

where the _radial_ quantum number $n$ is constrained by the two potential
characteristics $\eta$ and $b$,

$n\leq\frac{\sqrt{2\,\hat{m}\,\eta}}{b}\ .$ (23)

In the spirit of Subsect. 2.5, the expressions (22) may serve to provide, at
least, a vague idea of the actual location of the discrete eigenstates of any
spinless semirelativistic Hulthén problem.

###  3.2 Category of Spinless Semirelativistic Generalized-Hellmann Problems
WL:Hm ; WL:R

Generalized Hellmann potentials WL:Hm ; WL:R  establish a class of central
interaction potentials that encompasses the shape introduced, some time ago H1
; H2 , for applications in atomic-physics theory. Each member of this set is
defined to be a linear combination of an attractive Coulomb contribution
$V_{\rm C}(r)$ and an optionally _either attractive or repulsive_ Yukawa
contribution $V_{\rm Y}(r)$,

$V_{\rm H}(r)\equiv V_{\rm C}(r)+V_{\rm
Y}(r)=-\frac{\kappa}{r}-\upsilon\,\frac{\exp(-b\,r)}{r}\ ,$ (24)

Coulomb coupling $\kappa$, Yukawa coupling $\upsilon$, and exponential-range
parameter $b$ hence satisfying

$\kappa\geq 0\ ,\qquad\upsilon\gtreqqless 0\ ,\qquad b>0\ .$ (25)

(Note that, in the notation (24), the interaction potential originally
proposed by Hellmann H1  corresponds to assuming for the Yukawa coupling
parameter $\upsilon$ a strictly negative value $\upsilon<0$.) At large
distances $r$, all generalized Hellmann potentials approach their Coulomb
component,

$V_{\rm H}(r)\xrightarrow[r\to\infty]{}V_{\rm C}(r)\ .$ (26)

This guarantees that any discrete eigenvalue $E_{k}$ of $H$ is nonpositive,
thus bounded from above:

$E_{k}\leq 0\qquad\forall\quad k\in{\mathbb{N}}_{0}\ .$ (27)

Due to the asymptotic approach (26) to Coulomb-type behaviour, every
generalized Hellmann potential (24) fails to meet all requirements imposed in
Subsect. 2.2, particularly, because WL:Hm

$V_{\rm H}(r)\notin L^{3/2}({\mathbb{R}}^{3})\cap L^{3}({\mathbb{R}}^{3})\ ,$
(28)

whence no finite upper bound (8) on the total number of its discrete bound
states can be found.

_Nonsingular_ generalized Hellmann potentials, in the classification of Table
1 identified by $\kappa+\upsilon\leq 0$, are bounded from below, as likewise
(because of $\sqrt{\bm{p}^{2}+m^{2}}\geq 0$) their Hamiltonians:

$\displaystyle H\geq V_{\rm H}(r)$ $\displaystyle\geq\min_{0\leq
r<\infty}V_{\rm H}(r)>-\infty$ for $\displaystyle\upsilon<-\kappa\ ,$ (29)
$\displaystyle H\geq V_{\rm H}(r)$ $\displaystyle\geq V_{\rm
H}(0)=\upsilon\,b$ for $\displaystyle\upsilon=-\kappa\ .$ (30)

_Singular_ generalized Hellmann potentials, in their systematics of Table 1
characterized by $\kappa+\upsilon>0$, develop _negative singularities_ at
$r=0$ bounded from below by a Coulomb potential:

$V_{\rm
H}(r)\geq-\frac{\alpha}{r}\qquad\mbox{with}\qquad\left\\{\begin{array}[]{ll}\alpha=\kappa+\upsilon&\qquad\mbox{for}\qquad\upsilon>0\
,\\\\[2.15277pt] \alpha=\kappa&\qquad\mbox{for}\qquad\upsilon\leq 0\
.\end{array}\right.$ (31)

As consequence of this, _mutatis mutandis_ the relevant insights collected in
Subsect. 2.1 apply. Most importantly, spinless relativistic generalized-
Hellmann problems are well defined WL:Hm  if

$\kappa+\upsilon\leq\frac{4}{\pi}\ .$ (32)

Table 1: Characterization of the seven types of generalized Hellmann potentials $V_{\rm H}(r)$ distinguished by comparing the Yukawa coupling $\upsilon$ with the (by assumption nonnegative) Coulomb coupling $\kappa\gneqq 0$ WL:Hm . Boundedness | Characteristic | Behaviour near | Sign of sum | Relation between
---|---|---|---|---
from below | of potential | the origin $r=0$ | of couplings | couplings $\upsilon$ and $\kappa$
unbounded | “singular” | $V_{\rm H}(r)\xrightarrow[r\to 0]{}-\infty$ | $\kappa+\upsilon>0$ | $\upsilon>\kappa$
|  |  |  | $\upsilon=\kappa$
|  |  |  | $0<\upsilon<\kappa$
| pure Coulomb |  |  | $\upsilon=0$
|  |  |  | $-\kappa<\upsilon<0$
bounded | finite at origin | $V_{\rm H}(r)\xrightarrow[r\to 0]{}\upsilon\,b$ | $\kappa+\upsilon=0$ | $\upsilon=-\kappa$
| repulsive core | $V_{\rm H}(r)\xrightarrow[r\to 0]{}+\infty$ | $\kappa+\upsilon<0$ | $\upsilon<-\kappa$

Merely for illustrative purposes, Fig. 2 depicts a typical representative of
four of our seven classes of generalized Hellmann potentials (24) identified
in Table 1. There is no need to show the trivial case of the pure Coulomb
potential arising for vanishing Yukawa coupling strength.

In order to exemplify the calculation of upper bounds on energies along the
lines sketched in Subsect. 2.3, Table 2 provides, for few low-lying
eigenstates of three generalized-Hellmann problems and a fixed choice of the
numerical values of the parameters $\mu$ and $\beta$ in the basis (2.3) of
some $d$-dimensional subspace, upper bounds on the associated binding
energies, defined by

$B_{k}\equiv E_{k}-2\,m\ ,\qquad k\in{\mathbb{N}}_{0}\ .$ (33)

The upper bounds in Table 2 can straightforwardly be optimized variationally,
in two respects: definitely by increasing the dimension $d$ of the adopted
subspace, and potentially by trying out different values for (i.e., varying)
the parameters $\mu$ and/or $\beta$ defining the subspace’s basis (2.3).

|
---|---
(a) | (b)
|
(c) | (d)

Figure 2: Generic representatives (solid black lines) of four of the only
seven distinguishable categories in our systematic classification (Table 1) of
generalized Hellmann potentials $V_{\rm H}(r)$ defined in Eq. (24) by linear
combinations of Coulombic (dashed magenta lines) and Yukawa (dotted blue
lines) contributions, choosing $\kappa=1$ and (a) $\upsilon=10$, (b)
$\upsilon=1$, (c) $\upsilon=-1$, or (d) $\upsilon=-2$ for the involved
coupling strengths.

Table 2: Bounds on binding energies, for states of radial excitation $n_{\rm r}$ and orbital angular momentum $\ell$, for exemplary spinless relativistic generalized-Hellmann problems WL:Hm ; WL:R  ($b=\mu=m$, $\beta=1$, $d\gtrapprox 29$). |  | Upper bounds on $B_{n_{\rm r}\ell}\;[m]$
---|---|---
Bound state | $\kappa=\upsilon=\frac{1}{2}$ |  $\kappa=1$, $\upsilon=-1$ |  $\kappa=1$, $\upsilon=-2$
$n_{\rm r}$ | $\ell$ | [case Fig. 2(b)] | [case Fig. 2(c)] | [case Fig. 2(d)]
0 | 0 | $\quad\\!\\!-0.11673$ | $\quad-0.17951$ | $\quad-0.14410$
0 | 1 | $\quad\\!\\!-0.01579$ | $\quad-0.06294$ | $\quad-0.06157$
0 | 2 | $\quad\\!\\!-0.00616$ | $\quad-0.02813$ | $\quad-0.02812$
1 | 0 | $\quad\\!\\!-0.02107$ | $\quad-0.05464$ | $\quad-0.04786$
1 | 1 | $\quad\\!\\!-0.00509$ | $\quad-0.02810$ | $\quad-0.02762$
2 | 0 | $\quad\\!\\!-0.00688$ | $\quad-0.02566$ | $\quad-0.02338$
Lower bound on $B_{n_{\rm r}\ell}\;[m]$ | $\quad\\!\\!-0.58578\dots$ | $\quad-1$ | $\quad-0.37336\dots$

## References

  * (1) E.E. Salpeter, H.A. Bethe, Phys. Rev. 84, 1232 (1951)
  * (2) W. Lucha, F.F. Schöberl, J. Phys. G: Nucl. Part. Phys. 31, 1133 (2005)
  * (3) E.E. Salpeter, Phys. Rev. 87, 328 (1952)
  * (4) I.W. Herbst, Commun. Math. Phys. 53, 285 (1977)
  * (5) I.W. Herbst, Commun. Math. Phys. 55, 316 (addendum) (1977)
  * (6) A. Martin, S.M. Roy, Phys. Lett. B 233, 407 (1989)
  * (7) I. Daubechies, Commun. Math. Phys. 90, 511 (1983)
  * (8) W. Lucha, F.F. Schöberl, Int. J. Mod. Phys. A 30, 1550062 (2015)
  * (9) M. Reed, B. Simon, _Methods of Modern Mathematical Physics IV: Analysis of Operators_ (Academic Press, New York, 1978)
  * (10) W. Lucha, F.F. Schöberl, Int. J. Mod. Phys. A 14, 2309 (1999)
  * (11) W. Lucha, F.F. Schöberl, Recent Res. Dev. Phys. 5, 1423 (2004)
  * (12) W. Lucha, F.F. Schöberl, Phys. Rev. A 60, 5091 (1999)
  * (13) W. Lucha, F.F. Schöberl, Int. J. Mod. Phys. A 15, 3221 (2000)
  * (14) W. Lucha, F.F. Schöberl, Phys. Rev. Lett. 64, 2733 (1990)
  * (15) W. Lucha, Mod. Phys. Lett. A 5, 2473 (1990)
  * (16) W. Lucha, F.F. Schöberl, Phys. Rev. A 54, 3790 (1996)
  * (17) W. Lucha, F.F. Schöberl, Int. J. Mod. Phys. A 29, 1450057 (2014)
  * (18) W. Lucha, F.F. Schöberl, EPJ Web Conf. 80, 00049 (2014)
  * (19) W. Lucha, F.F. Schöberl, Int. J. Mod. Phys. A 29, 1450181 (2014)
  * (20) W. Lucha, F.F. Schöberl, Int. J. Mod. Phys. A 29, 1450195 (2014)
  * (21) W. Lucha, F.F. Schöberl, Int. J. Mod. Phys. A 34, 1950028 (2019)
  * (22) W. Lucha, PoS (EPS-HEP2019), 537 (2020)
  * (23) S. Flügge, _Practical Quantum Mechanics, Vol. I_ (Springer, Berlin, 1994)
  * (24) H. Hellmann, J. Chem. Phys. 3, 61 (1935)
  * (25) H. Hellmann, W. Kassatotschkin, J. Chem. Phys. 4, 324 (1936)