# A Support Vector Model of Pruning Trees Evaluation Based on OTSU Algorithm
††thanks: This research is funded by Natural Science Foundation of Zhejiang
Province (Grant No. LQ20G010002), and the National Science Foundation of China
(71571162, 71702164). This research is supported by the project of China
(Hangzhou) cross-border electricity business school, the Philosophy and Social
Science Foundation of Zhejiang Province (21NDJC083YB). .

Yuefei Chen  Rutgers University
Piscataway, NJ, USA
<EMAIL_ADDRESS>Xinli Zheng  Zhejiang Tianyanweizhen
network Polytron Technologies
Hanzhou, Zhejiang, China
email address or ORCID  Chunhua Ju  Contemporary Business
and Trade Research Center
Zhejiang Gongshang University
Hangzhou, Zhejiang, China
<EMAIL_ADDRESS>Fuguang Bao  School of Management Science and Engineering
Zhejiang Gongshang University
Hangzhou, Zhejiang, China
<EMAIL_ADDRESS>
###### Abstract

The tree pruning process is the key to promoting fruits’ growth and improving
their productions due to effects on the photosynthesis efficiency of fruits
and nutrition transportation in branches. Currently, pruning is still highly
dependent on human labor. The workers’ experience will strongly affect the
robustness of the performance of the tree pruning. Thus, it is a challenge for
workers and farmers to evaluate the pruning performance. Intended for a better
solution to the problem, this paper presents a novel pruning classification
strategy model called “OTSU-SVM” to evaluate the pruning performance based on
the shadows of branches and leaves. This model considers not only the
available illuminated area of the tree but also the uniformity of the
illuminated area of the tree. More importantly, our group implements OTSU
algorithm into the model, which highly reinforces robustness of the evaluation
of this model. In addition, the data from the pear trees in the Yuhang
District, Hangzhou is also used in the experiment. In this experiment, we
prove that the OTSU-SVM has good accuracy with 80% and high performance in the
evaluation of the pruning for the pear trees. It can provide more successful
pruning if applied into the orchard. A successful pruning can broaden the
illuminated area of individual fruit, and increase nutrition transportation
from the target branch, dramatically elevating the weights and production of
the fruits.

###### Index Terms:

pruning, classification strategy, support vector machine model, OTSU Algorithm

##  I Introduction

With the rapid development of modern agriculture, an increasing number of
farmers tend to grow crops technically and scientifically. As a significant
role in the growth of the fruit tree, the pruning process is essential for
improving the production of the fruit. Proper training through correct pruning
is important for a healthy, strong fruit tree [1]. Generally, pruning will be
done during the summer. It is the best way to remove vigorous, upright water
sprouts so that overly-vigorous trees can be brought into balance again.

Considering the relationship between pruning and production, three factors of
pruning will affect the production of fruit. At first, pruning can leave
enough space for fruit growth, in case that a lot of small fruits crowd in a
small space and compete for resources like air, water transportation. A
crowded space will restrain respiration and transpiration, which can speed up
fruits decaying and influence the quality. More importantly, it will not
stimulate the new branch growth which is not desired [2]. A new branch
represents a new competitor for the nutrition and organism. The branch with
fruit will obtain fewer resources than what is obtained in the situation
without a new branch. Thus, pruning can ensure each branch enough
transportation of nutrition and organism. Additionally, pruning can expand the
overall light area and accelerate transmittance throughout the canopy of the
tree. It can increase the amount of light by approximately 25$\%$ in the lower
part of the canopy compared to the group without pruning controlled [3]. With
the purpose to increase the production of fruit, light intensity and luminance
are ignorable issues to be concentrated on. These factors will affect the
absorbance of lights and the photosynthesis efficiency of fruits. Yuan Gao and
his team has presented the light distribution of the tree is highly correlated
with the growth of fruit. High light can promote photosynthetic ability
thereby effectively decreasing fruit abscission in fruit trees [4].

Currently, there are two kinds of methods for pruning fruit trees – manual
pruning and automation technologies. The majority of fruit tree pruning
systems are highly dependent on human labor. Experienced laborers are in great
demand for orchard pruning, who can make a highly successful rate of pruning
based on their experience. What they do can promote fruit production. However,
with the increasing cost of labor and limited labor availability, it is highly
desirable for growers to seek mechanical or robotic solutions for performing
these production operations [5, 6]. Meanwhile, the accuracy of pruning becomes
an unignorable factor for researchers when they are designing automated
pruning systems because only successful pruning can promote the increment of
production. In addition, mechanical pruning had a significantly negative
impact on the photosynthetic rate [7]. Thus, researchers have to evaluate the
performance of fruit tree pruning and make sure of the robustness of this
accuracy in the automated pruning systems.

Therefore, in this paper, we propose a novel pruning evaluation model based on
the shadow of the branches and leaves. To our knowledge, there is little
research aiming at evaluating pruning performance by using the shadows of the
fruit trees. We explore this aspect and develop the model for pruning. The key
contributions of this paper are as follows.

  * •

The paper conducts a comprehensive investigation to study the current models
which predicts the pruning process and related pruning strategies, and we also
go to the orchard to take the images and discuss with the agricultural
experts. Our investigation drives us to create the dataset including the
information about the fruit trees and their pruning performance. Based on this
dataset, we analyze those two features of the pruning shadows, which
significantly affect the pruning performance and production of the fruit
trees.

  * •

Additionally, the paper proposes a new simple model called OTSU-SVM to
evaluate the pruning performance. In contrast to the model which uses deep
learning like back propagation neural network [8], this model does not need a
great scale of data. The OTSU-SVM will select and extract the features of the
shadows of fruit trees and preprocess them. These results will be used to
train a supported vector machine (SVM) model to classify the different
performances of pruning.

  * •

More importantly, the paper extensively implements and evaluates OTSU-SVM
based on the real-world data in Hangzhou including the shadows of pear trees
in the orchard and pruning performance. The result shows that the accuracy of
OTSU-SVM can go to 80$\%$ and give most of the shadows of the fruit trees a
proper pruning evaluation.

The rest of the paper is organized as follows. Section II introduces the
related work. In Section III, we depict the details about the methods and
materials about the OTSU-SVM model. Section IV presents the process of our
experiment in the orchard and Section V illustrates the result of the
experiment of our OTSU-SVM model and comparisons with another basic model.
Section VI displays a discussion about the result and interprets the findings,
and then we conclude this paper and discuss the future work.

##  II Related Work

###  II-A Automation Pruning System

Pruning is highly dependent on human labor. However, skilled labor is in short
supply, and the increasing cost of labor is becoming a big issue for the tree
industry. Meanwhile, work safety is another issue in the manual pruning [6].
To address these challenging pruning issues, lots of researchers have
concentrated on the alternate aspect about automation pruning system. Some of
them tend to develop an application with knowledge of complex skills like tree
training and pruning to teach growers how to evaluate it [9]. Additionally,
many scholars use detection and 3d reconstructions to analyze the tree
construction and make decisions [5, 6, 8]. Most of them apply computer vision
and 3d modeling technology to reconstruct the branches of trees. However, they
have not considered the effect of leaves of trees on the fruits, which results
in the factor from the light and photosynthesis efficiency of fruits is
ignored so that the strategy of pruning will be involved. What motivates us to
explore a new mechanism considering the factors of leaves and the light area
of fruits.

###  II-B Pruning Decision Model

The pruning decision model becomes a popular research topic in recent years
with the development of robotic and automatic pruning. Barbosa and his team
focus on the pruning strategies on pear trees with phenological evaluation.
They propose to analyze the variation of climate in a year and fruit habits in
different cultivars and then find how to affect the determination of pruning
strategies [10]. Dr. Schupp and his team proposed a severity level pruning
strategy for tall spindle apple trees to provide guidelines for determining
the cutting threshold for robotic pruning [11].

###  II-C Summary

Technically, in contrast to the other pruning model, the key advantage of the
method in this paper is that authors consider the effects of leaves and light
area of fruits, which is also a significant factor of the production of the
fruit, besides the effect of branches. More importantly, the model does not
need a large scale of data for training driving and is not complex as the
model that needs 3-d reconstruction [8] but still has a high performance of
evaluation on fruit tree pruning.

##  III Method

In this section, the pruning data is firstly introduced. After that, an
overview of the model “OTSU-SVM” is given and illustrated into two parts of
the model in detail.

###  III-A Data Descriptions

Figure 1: The metadata of pruning tree dataset

Pear trees are traditional and prevailing trees growing in China. In Hangzhou,
there are thousands of hectares of pear trees. It is convenient for
researchers to collect the data and store it in the pruning dataset. In the
dataset, the metadata of the dataset is shown in Fig. 1. Every tree has its id
information and individual photo id. Then, they have all labeled the pruning
performance which is evaluated by experts. $1$ represents performs good and
$0$ represents perform not good. Shadows of the pear trees are collected in
the form of images. The process of collection is shown in Fig. 2.

Figure 2: collection of shadows under the pear trees

The collection time is at noon and the sun light is nearly vertical to the
ground. The white board is used to get shadow of the leaves and branches. The
distance between whiteboard and branches are $0.5$ meter. With the expansion
of the branches of trees, ten points will be randomly selected under the tree
to take images. When they are stored in the database, it will be transformed
into a 3-dimensions of images matrix as shown in Fig. 3. It is the $3\times
n\times m$ matrix where n and m are the length and width of the image. The
other dimension means three channels of one pixel of RGB image, where
$Image[x][y]=(r,g,b)$ indicates that the pixel at the position $(x,y)$ has a
color $(r,g,b)$. In this pixel, $r$ represents the level of red, $g$
represents green, and $b$ represents blue. The number of the matrix as values
between $0$ and $255$ represents the intensity of the corresponding channel.

Figure 3: The metadata of pruning tree dataset

Figure 4: Three layers of feature recognition architecture

###  III-B Overviews of OTSU-SVM

The OTSU-SVM model is designed to analyze the image data from the fruit tree.
It comprises three parts. As shown in Fig. 4, a three-layer feature
recognition architecture called OTSU-SVM for fruit trees is presented. The
bottom layer is the image data layer. It collects and stores data from the
orchard and lays the data foundation for feature selection and model
evaluation. The second layer is the data wrangling and feature selection
layer. This layer preprocesses the image data and normalizes the image matrix.
Additionally, the features are also selected and analyzed, which is the input
for the next layer. In the top layer, i.e., Support Vector Machine (SVM)
model. We use the features obtained from the data wrangling and feature
selection layer to classify the points and train the model. After training the
model, it can evaluate the pruning performance of the fruit trees and classify
which points of fruit trees perform well.

####  III-B1 Data Layer

The data in the dataset is the image data of each selected fruit tree. Its
details are given in the data description section. These data including the
shadows distribution of the fruit trees.

####  III-B2 Data Wrangling and feature selection Layer

Figure 5: The layer of preprocessing

In this layer, there are two parts of the preprocessing. The details are shown
in Fig. 5. The first part is data wrangling and contains four steps. In these
steps, the first step is to transform the 3 channels RGB image into 1 channel
grayscale image and store it into an image matrix. The second step is the
significant part in the data wrangling. It uses OTSU algorithm to set
threshold of two classes, dark and white pixels, and separate gray scale
pixels into these two classes. The OTSU algorithm is an effective threshold
selection method and seeks the threshold which can lead to a high accuracy of
analysis. Additionally, the next step is sampling. Sampling is optional in
this research. That is because it aims to reduce the complexity of the matrix
and lessen the burden of the computer. if the research group has high
performance computing capability, this step is recommended to skip because
skipping this step can help obtain more accurate result. At last, the data
needs to be normalized into the range from $0$ to $1$. The purpose is to
decrease the deviation and it is prepared for the feature selection.

The second part is feature selection. In this part, uniformity and black
pixels rate are selected as the feature of image matrices.

Figure 6: The 3 channels image conversion into 1 channel grayscale image

#####  III-B2a Data wrangling

The paper explores data from the bottom layer to extract the features of the
image of shadow. In order to fulfill the purpose, the 3 channels RGB image
matrix is transformed into 1 channel grayscale image. A classic RGB image to
gray conversion formula is given by

$Gray(r,g,b)=0.21r+0.72g+0.07b$ (1)

In this equation, the resulting value $Gray(r,g,b)$ is between 0 and 255 and
$r,g,b$ represents the levels of red, green, and blue. The Fig. 6 shows that
process of the conversion.

Moreover, the grayscale image also has too much-complicated data to process.
The purpose of paper is to distinguish the part of the shadow and the area of
lights. Thus, only two values of the data is in demand and the shadow pixel is
set to be $0$, which represents the color of the pixel is black, and the light
pix as $255$ and white. To separate these two parts of the image, thresholding
is an optimizing way to solve it. The threshold selection method called OTSU
algorithm [12] with our statements of $0$ and $255$ is used to set a threshold
and divide the pixels of the image into two parts. The OTSU algorithm is used
to perform automatic image thresholding in computer vision and image
processing [13]. In this paper, the algorithm returns a intensity threshold
that separate pixels into two classes, shadows and lights. This threshold is
determined by minimizing intra-class intensity variance, or equivalently, by
maximizing inter-class variance [12]. The intra-class variance is defined as a
weight sum of variance of separated two classes,

$\sigma^{2}_{w}(t)=w_{0}(t)\sigma^{2}_{0}(t)+w_{1}(t)\sigma^{2}_{1}(t)$ (2)

In this equation, weights $w_{0}$ and $w_{1}$ are the probabilities of the two
classes separated by a threshold $t$, and $\sigma^{2}_{0}$ and
$\sigma^{2}_{1}$ are variance of these two classes. The class probability
$w_{0}(t)$ and $w_{1}(t)$ can be expressed as,

$\begin{split}w_{0}(t)=\sum^{t-1}_{i=0}p(i),\\\\[2.84526pt]
w_{1}(t)=\sum^{L-1}_{i=t}p(i)\end{split}$ (3)

In this equation, the range is between $0$ and $L-1$. In addition, the inter-
class variance is defined as the variance between two separated classes. The
equation is

$\begin{split}\sigma^{2}_{b}(t)=\sigma^{2}-\sigma^{2}_{w}(t)&=w_{0}(\mu_{0}-\mu_{T})^{2}+w_{1}(\mu_{1}-\mu_{T})^{2}\vspace{1ex}\\\
&=w_{0}(t)w_{1}(t)[\mu_{0}(t)-\mu_{1}(t)]^{2}\end{split}$ (4)

In this equation, $\mu$ is the class mean of these classes. The class means
$\mu_{0}(t)$, $\mu_{1}(t)$ and $\mu_{T}$ are expresses as,

$\begin{split}\mu_{0}=\frac{\sum^{t-1}_{i=0}ip(i)}{w_{0}(t)}\vspace{1ex},\\\\[2.84526pt]
\mu_{1}=\frac{\sum^{L-1}_{i=t}ip(i)}{w_{1}(t)}\vspace{1ex},\\\\[2.84526pt]
\mu_{T}=\sum^{L-1}_{i=t}ip(i)\end{split}$ (5)

According the equation (4), the relationship between the inter-class variance
and the inter-class variance can be verified that minimizing intra-class
variance is equivalent to maximizing inter-class variance.The algorithm 1 with
our statement is as follows.

Algorithm 1 OTSU thresholding algorithm with statements

0: Image Matrix $S$ with length $m$ and width $n$

0: Thresholding Matrix $S_{1}$

Define $min\ variance=\infty$

Define $threshold=-1$

Define $p=0$

while $p<256$ do

Define $List_{A},List_{B}$ as empty lists

Define $i,j=0$

while $i<m$ do

while $j<n$ do

if $S[i][j]\geq p:$ then

$List_{A}.append(S[i][j])$

else

$List_{B}.append(S[i][j])$

end if

end while

end while

$S_{A}=var(A)$

$S_{B}=var(B)$

$I_{A}=sum(A)$

$I_{B}=sum(B)$

$Intra\ class\ variance=S_{A}\times I_{A}+S_{B}\times I_{B}$

if $Intra\ class\ variance<min\ variance$ then

$minvariance=Intraclassvariance$

$threshold=p$

end if

end while

Define $S_{1}$ as empty array with length $m$ and width $n$

Define $k,l=0$

while $k<m$ do

while $l<n$ do

if $S[k][l]\geq threshold$ then

$S_{1}[i][j]=255$

else

$S_{1}[i][j]=0$

end if

end while

end while

return $S_{1}$

Additionally, as an image matrix, its width and length are generally over than
2000 pixels. In other words, mostly image matrices contain more than four
million pixels. It will leave a heavy burden to the program and the CPU
computing. The algorithm is designed to sample the image matrix by summarizing
the colors of pixels in patches of the matrix. The algorithm is as follows.

Algorithm 2 Sample image matrices algorithm

0: Image Matrix $S$ with length $m$ and width $n$, shrinking times $p$

0: Adjusted Matrix $S_{1}$

Define $S_{1}$ as empty array with length $m$ and width $n$

Define $i,j=0$

while $i<m/p$ do

while $j<n/p$ do

$Sum=sum(S[p*i][p*j]+…+S[p*(i-1)+1][p*j]+…+S[p*(i-1)+1][p*(j-1)+1])$

if $Sum<5$ then

$S_{1}[i][j]=0$

else

$S_{1}[i][j]=255$

end if

end while

end while

return $S_{1}$

In details, this process is similar with the pooling layer [14] in the deep
learning and convolutional neural network (CNNs) [15, 16]. The algorithm aims
to find the representative value in a $p\times p$ grids as its value and
summarize the image matrix. It is to progressively reduce the spatial size of
the representation to reduce the amount of computation in the following
normalization and model. In this experiment, the p is set to be 3. The Fig. 7
shows that process of selecting the represented values of the image. In other
words, the size of the new image matrices is diminished into $1/9$ of the
original image matrices, which can improve computation efficiency and does not
lose a lot of information about the image.

Figure 7: The selection process of the represented values of the image

#####  III-B2b features selection

More importantly, we are based on the preprocessed data to extract features.
We consider the lights area of the fruit and the effects of the distribution
of the leaves and branches. Therefore, the shadow of the fruit tree is a good
choice to analyze these factors. We find two features from the shadow. The one
is black pixels rate. The other one is uniformity. The black pixels rate means
the rate of black pixels in the whole image. The equation is

$\begin{split}black\ pixels\
rate=\frac{count(blackpixels)}{count(allpixels)}\end{split}$ (6)

It reveals the light transmittance and the light area of the fruit tree. If
the black pixels rate is close to $1$, the transmittance of sun light is close
to $0$. The uniformity is an indicator that can determine if the distribution
of the branch and leaves is uniform. The approach to getting the uniformity is
as follows. In this experiment, the size of a grid is set to be $100\times
100$. In detail, the number of white pixels are counted in every grid of the
matrix and store into the list. The Fig. 8 shows the process of the grid
moving and counting the number of white pixels.

Figure 8: A grid moving and counting the number of white pixels in the grid

After that, the variance of the values is calculated in the list to get the
standard deviation of these grids. Standard deviation in these grids
represents the extent of deviation between these grids. When these grids
deviation are large, it means they are not uniform. If it is close to 0, the
matrix is uniform. Thus, the standard deviation is equivalent to the
uniformity we defined. The equation is as follows.

$\begin{split}uniformity=\sigma=\sqrt{\frac{\sum{(x_{i}-\mu)^{2}}}{N}}\end{split}$
(7)

In this equation, $N$ is the number of grids. $x_{i}$ is the number of white
pixels in each grid. $\mu$ is the mean of $x_{i}$ in all grids. Using this
equation, we build the matrix of uniformity in this algorithm 3.

Algorithm 3 calculate the uniformity

0: Processed Image Matrix $S$ with length $m$ and width $n$, grid $G$ with
size of $p\times p$

0: uniformity

Define empty list $L=[]$

Define $i,j=0$

while $i<m/p$ do

while $j<n/p$ do

$Sum=sum(S[p*i][p*j]+…+S[p*(i-1)+1][p*j]+…+S[p*(i-1)+1][p*(j-1)+1])$

$L.append(Sum)$

end while

end while

$uniformity=\sqrt{(variance(L))}$

return $uniformity$

In addition, the ranges of the two features are different. In other words,
different ranges will have unstable effects on the weights of two variables in
the SVM model. Therefore, it is necessary for the model to normalize two
features into the data with a range from 0 to 1. The normalization is
frequently applied in image recognition [17, 18, 19, 20, 21]. The goal of the
normalization is to transform features to be on a similar scale and improves
the performance and training stability of the model. There are four common
normalization techniques. That is scaling to a range, clipping, log scaling,
and z-score. In this method, scaling is used to convert floating-point feature
values from their natural range into a standard range ($0$ to $1$). The
equation is as follows.

$\begin{split}x’=\frac{x–x_{min}}{x_{max}–x_{min}}\end{split}$ (8)

In this equation, the $x$ is the original value. $x_{max}$ and $x_{min}$ are
maximum and minimum values of the feature values. $x’$ is the new value of the
feature matrix. When the data is preprocessed and two features are extracted.
The result will be used in the SVM model.

####  III-B3 Support Vector Machine Model Layer

This layer is based on the support vector machine (SVM) technique to generate
a classification model. The SVM model is a supervised machine learning model
that uses classification algorithms for two-group classification problems
[22]. In other words, it analyzes data for regression analysis and classifies
the samples into two labels. Given a set of training samples, which are marked
as in one of two categories, an SVM method builds a classification model that
classifies testing samples into predicted categories and makes a non-
probabilistic binary linear classifier. SVM method is also popular in image
recognition and classification [23, 24]. Its application is broad, just like
in agricultural field [25], and the medical field [26]. In our experiment, we
use linear SVM to classify the label. We transform our dataset into the
following form.

$\begin{split}(\mathbf{x}_{1},y_{1}),…,(\mathbf{x}_{n},y_{n}),\end{split}$ (9)

where the $y_{i}$ only has two values $0$ or $1$, indicating the class to
which the point $\mathbf{x}_{i}$ belongs. It represents the pruning
performance of the fruit tree. Each $\mathbf{x}_{i}$ is a vector and each
dimension is one of the features. SVM model needs to find the maximum margin
hyperplane that divides the samples of $\mathbf{x}_{i}$ into two groups based
on the value of $y_{i}$. The hyperplane is defined as the maximized distance
between the hyperplane and the nearest point $\mathbf{x}_{i}$ from either
group. It can be written as the expression,

$\begin{split}\mathbf{w}^{T}\mathbf{x}–b=0,\end{split}$ (10)

where the $\mathbf{w}^{T}$ is the normal vector to the hyperplane. Then we
select two parallel hyperplanes to separate these two classes of data.
Additionally, the distance between two hyperplanes is as large as possible.
When the distance is maximized, the region bounded by these two hyperplanes is
called margin, and the maximum margin hyperplane lies between them.
Eventually, the dataset will be taken apart by the maximum margin hyperplane.
The one side belongs to the points predicted as good pruning and points on the
other side are predicted as not good.

##  IV Evaluation

In this section, the real-world datasets are introduced for evaluation and the
process of the experiment that we collect data. Then the application with the
dataset into the SVM model and training the model are illustrated in detail.

A real-world dataset from the entire dataset introduced in Section III is
leveraged to validate the OTSU-SVM algorithm. This dataset includes pear trees
shadows and over $100$ images of pear trees’ shadows and shapes. The dataset
is collected in mid-June 2020, at 11:00 am $-$ 12:00 noon. The location is in
Hangzhou, Zhejiang Province, China, whose coordinates are 30.2741° N,
120.1551° E. We use a $50\ cm\times 50\ cm$ whiteboard to collect the shadows.
In the experiment, we put the whiteboard down to the pear trees with $50\ cm$
between the board and the branches and leaves. Limited to get the whole shadow
of the tree, we randomly select several points under a pear tree to collect
the shadow on the whiteboard and take images. Then we extract features of
these image data. The mean values of these features are the features of the
pear trees. The figure 6 is our process to collect shadows under the pear
trees. When the result is obtained, we take images of the collected pear trees
to ask experts in the orchard if the pear trees are pruned and the performance
of pruning is good. If the performance and production are good, the
performance label of the pear tree will be $1$ and set $-1$ when the
performance is not good. Eventually, the data of pear trees are preprocessed
and used to analyze in the support vector machine model.

Our experiment design avoids too many issues in the fruit trees. The
collection time we select is at noon in June. It is summer in Hangzhou and can
make sure the solar zenith angle is close to $90\ degrees$, which can maximize
the light area of the fruit tree displayed on the whiteboard and reduce the
effect of the shadow of the branch and leaves to the light area. It is shown
in Fig. 9. This figure displays the variation of shadows under the different
degrees of sunlight. Smaller degrees of sunlight will lead the shadow to
lengthen and features will change out of disproportion. Especially, the
shadows of leaves and branches will be stretched and cover the area which is
for the transmitted area. Maximizing degrees of sunlight can limit this
effect. More importantly, collecting data and images in the same location and
same kind of fruit trees can exclude other variables and avoid the effect of
other factors which may influence the growth of fruit trees.

Figure 9: Tree shadows under different degrees of sunlight

##  V Experiment

Figure 10: Pear trees in the orchard

Figure 11: Pear trees in the orchard

Figure 12: Sample points of pruning not good pear tree

Figure 13: Sample points of pruning good pear tree

In this section, the process of experiment of this research is introduced in
details. The data is collected in mid-June 2020, at 11:00 am 12:00 noon. The
orchard is in Hangzhou, Zhejiang Province, China. In this experiment, 100
samples are selected and taken images. These samples are stored into the
dataset for pruning trees. Fig. 10 and Fig. 11 are the pear tree selected
which are labeled by experts as pruning good or pruning not good. Fig. 12 and
Fig. 13 are the randomly selected points under the trees for shadows. These
samples are transformed into matrices and stored into the dataset for pruning
trees. According to the materials and methods section, the method and
algorithm of preprocessing and feature selection is followed to implement this
classification model and data process in python programing language, which
will be upload on the Github website and open source soon. The Numpy library
is used to build all matrices of data. It is useful and popular in data
processing. Support Vector Machine model is building by sci-kit-learn package,
which provides APIs for building the SVM model and generating maximum margin
hyperplane to classify the points. Additionally, with purpose to get a clear
and obvious result. The Matplotlib package is used to display our result. It
is a robust package for computer visualization and data analysis. The input of
the model includes the matrix of features called $X$ and the 1-dimension
matrix label called $Y$. The table of our data frame is as follows.

TABLE I: The sample table of data frame including X, Y, and orders SUM | X | Y
---|---|---
1 | 0.7531091077884632 | 0.30455985384616746 | 1
2 | 0.6507489189014342 | 0.43863512963846113 | 1
$\cdots$ | $\cdots$ | $\cdots$ | $\cdots$
9 | 0.6826601194540411 | 0.17369677676678635 | -1
10 | 0.764769075618435 | 0.2990156489318126 | -1

In this experiment, two methods are used to build the model and predict
results. The baseline method is nonlinear SVM with Rbf kernel, which is most
common kernel in SVM. The other method is SVM with linear kernel. At first,
the dataset are split into two kinds of dataset. The one is training set, the
other one is testing set. The proportion of training set and testing set is
$0.6$ to $0.4$. The training set is used to build the model and the testing
set is used to test the performance of the model. The experiment result is as
follows.

Figure 14: The result of the linear support vector machine model

Figure 15: The result of the nonlinear support vector machine model

Eventually, the output of the model is generated in the following figure. In
this figure, the horizon axis is the value of uniformity and the vertical axis
means the value of the black pixels rate of the image. These blue points
represent the label is 1. In other words, they belong to good pruning fruit
trees. The yellow points represent the trees that are pruned not good and
label is -1. The light blue line is the hyperplane and it classifies the
points into two kinds of pruning performance. The orange dashed lines are the
maximum margin of the hyperplane. Additionally, the support vectors are marked
and circled. In the They are the points that are the closest to maximum margin
hyperplane.

In the following chart, the accuracy of these two models are shown in these
two bars. The nonlinear SVM with Rbf kernel has 47.5$\%$ accuracy and linear
kernel has 80.0$\%$. Therefore, in contrast to the baseline method with rbf
kernel. Linear kernel SVM has higher accuracy and better performance in the
comparison of models.

Figure 16: Comparison of SVM Models Accuracy

##  VI Conclusion

In this paper, the paper designs a support vector machine classification model
called OTSU-SVM to evaluate the performance of pruning. It comprehensively
measures the features of the fruit trees from two dimensions – light
transmittance dimension and uniformity dimension, and realizes pruning
evaluation by integrating these two features and the label of performance.

More importantly, the process of data preprocessing and feature selection
plays a significant role before the OTSU-SVM model analysis. We propose three
algorithms to preprocess data and measure the feature. The one is to use
threshold selection method (OTSU algorithm) to set threshold and determine
pixels of the image belong to white or black pixels. The second one is to
shrink dimensions and reduce redundancy of the images in order to increase our
model efficiency. The last algorithm is to calculate the uniformity of matrix,
which is one of features we select. In our method, we define how to measure
the black pixels rate and uniformity of shadows. Additionally, we normalize
the matrix of features. Normalization is to improve the stability and
robustness of the weights of the pruning model.

The supervised model of machine learning, support vector machine, is used to
make a maximum margin hyperplane to classify the good and not good pruning
points. Comparing with other pruning decision models, our model has high
accuracy and considers more comprehensive factors of the fruit trees. With
purpose to prove our model performance, we went to the orchard in Hangzhou,
collected the data and took images from pear trees. A dataset is built based
on these data and verifies the high accuracy of our model.

In the future work, we expect these systems can be applied into more areas of
agriculture and increase the production of fruits.

##  VII Author Contribution

Conceptualization, ; methodology, ; software, ; validation,; formal analysis,
; investigation, ; data curation, ; writing—original draft preparation, ;
writing—review and editing, ; visualization, ; supervision, ; project
administration, ; funding acquisition, . All authors have read and agreed to
the published version of the manuscript.

##  VIII Acknowledgment

This research is funded by Natural Science Foundation of Zhejiang Province
(Grant No. LQ20G010002), and the National Science Foundation of China
(71571162, 71702164). This research is supported by the project of China
(Hangzhou) cross-border electricity business school, the Philosophy and Social
Science Foundation of Zhejiang Province (21NDJC083YB). The authors also
gratefully acknowledge Qifeng Yang, which offers us an access to the orchard
in Lia Hangzhou, Zhejiang Province, China for collecting data and doing the
experiment. He gives us a lot of help on knowledge about the pear trees and
pruning.

## References

  * [1] J. R. Feucht, “Training and pruning fruit trees,” Ph.D. dissertation, Colorado State University. Libraries, 2000.
  * [2] T. Maughan, B. Black, and T. Roper, “Training and pruning apple trees,” 2017.
  * [3] N. Bhusal, S.-G. Han, and T.-M. Yoon, “Summer pruning and reflective film enhance fruit quality in excessively tall spindle apple trees,” _Horticulture, Environment, and Biotechnology_ , vol. 58, no. 6, pp. 560–567, 2017.
  * [4] Y. Gao, S. Gao, L. Jia, T. Dai, X. Wei, J. Duan, S. Liu, and X. Weng, “Canopy characteristics and light distribution in sapindus mukorossi gaertn. are influenced by crown architecture manipulation in the hilly terrain of southeast china,” _Scientia Horticulturae_ , vol. 240, pp. 11–22, 2018.
  * [5] A. Zahid, M. S. Mahmud, L. He, D. Choi, P. Heinemann, and J. Schupp, “Development of an integrated 3r end-effector with a cartesian manipulator for pruning apple trees,” _Computers and Electronics in Agriculture_ , vol. 179, p. 105837, 2020.
  * [6] L. He and J. Schupp, “Sensing and automation in pruning of apple trees: A review,” _Agronomy_ , vol. 8, no. 10, p. 211, 2018.
  * [7] K. Laužikė, V. Sirgedaitė-Šėžienė, N. Uselis, and G. Samuolienė, “the impact of stress caused by light penetration and agrotechnological tools on photosynthetic behavior of apple trees,” _Scientific reports_ , vol. 10, no. 1, pp. 1–9, 2020.
  * [8] L. Xinxing, L. Buwen, and L. Shiyang, “Pruning decision system for apple tree based on local point cloud and bp neural network,” _Transactions of the Chinese Society of Agricultural Engineering_ , vol. 37, no. 2, 2021.
  * [9] S. Kolmanič, S. Tojnko, T. Unuk, and Š. Kohek, “The computer-aided teaching of apple tree pruning and training,” _Computer applications in engineering education_ , vol. 25, no. 4, pp. 568–577, 2017.
  * [10] C. M. de Alcântara Barbosa, R. Pio, F. B. M. de Souza, R. B. Bisi, J. E. B. Neto, and D. da Hora Farias, “Phenological evaluation for determination of pruning strategies on pear trees in the tropics,” _Scientia Horticulturae_ , vol. 240, pp. 326–332, 2018.
  * [11] J. R. Schupp, H. E. Winzeler, T. M. Kon, R. P. Marini, T. A. Baugher, L. F. Kime, and M. A. Schupp, “A method for quantifying whole-tree pruning severity in mature tall spindle apple plantings,” _HortScience_ , vol. 52, no. 9, pp. 1233–1240, 2017.
  * [12] N. Otsu, “A threshold selection method from gray-level histograms,” _IEEE transactions on systems, man, and cybernetics_ , vol. 9, no. 1, pp. 62–66, 1979.
  * [13] M. Sezgin and B. Sankur, “Survey over image thresholding techniques and quantitative performance evaluation,” _Journal of Electronic imaging_ , vol. 13, no. 1, pp. 146–165, 2004.
  * [14] D. Yu, H. Wang, P. Chen, and Z. Wei, “Mixed pooling for convolutional neural networks,” in _International conference on rough sets and knowledge technology_. Springer, 2014, pp. 364–375.
  * [15] S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in _2017 International Conference on Engineering and Technology (ICET)_. Ieee, 2017, pp. 1–6.
  * [16] P. Kim, “Convolutional neural network,” in _MATLAB deep learning_. Springer, 2017, pp. 121–147.
  * [17] S.-C. Pei and C.-N. Lin, “Image normalization for pattern recognition,” _Image and Vision computing_ , vol. 13, no. 10, pp. 711–723, 1995.
  * [18] M. Alghoniemy and A. H. Tewfik, “Geometric distortion correction through image normalization,” in _2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No. 00TH8532)_ , vol. 3. IEEE, 2000, pp. 1291–1294.
  * [19] Y. S. Abu-Mostafa and D. Psaltis, “Image normalization by complex moments,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , no. 1, pp. 46–55, 1985.
  * [20] K.-M. Koo and E.-Y. Cha, “Image recognition performance enhancements using image normalization,” _Human-centric Computing and Information Sciences_ , vol. 7, no. 1, pp. 1–11, 2017.
  * [21] C. P. Loizou, M. Pantziaris, I. Seimenis, and C. S. Pattichis, “Brain mr image normalization in texture analysis of multiple sclerosis,” in _2009 9th International Conference on Information Technology and Applications in Biomedicine_. IEEE, 2009, pp. 1–5.
  * [22] C. Cortes and V. Vapnik, “Support vector machine,” _Machine learning_ , vol. 20, no. 3, pp. 273–297, 1995.
  * [23] O. Okwuashi and C. E. Ndehedehe, “Deep support vector machine for hyperspectral image classification,” _Pattern Recognition_ , vol. 103, p. 107298, 2020.
  * [24] A. F. Agarap, “An architecture combining convolutional neural network (cnn) and support vector machine (svm) for image classification,” _arXiv preprint arXiv:1712.03541_ , 2017.
  * [25] M. Islam, A. Dinh, K. Wahid, and P. Bhowmik, “Detection of potato diseases using image segmentation and multiclass support vector machine,” in _2017 IEEE 30th canadian conference on electrical and computer engineering (CCECE)_. IEEE, 2017, pp. 1–4.
  * [26] G. J. Chowdary _et al._ , “Machine learning and deep learning methods for building intelligent systems in medicine and drug discovery: A comprehensive survey,” _arXiv preprint arXiv:2107.14037_ , 2021.


Optimizing Serially Concatenated Neural Codes with  Classical Decoders

Jannis Clausius, Marvin Geiselhart and Stephan ten Brink

Institute of Telecommunications, University of Stuttgart, Pfaffenwaldring 47, 70659 Stuttgart, Germany

This work is supported by the German Federal Ministry of Education and Research (BMBF) within the project Open6GHub under grant 16KISK019 and the project FunKI  under grant 16KIS1187.

For improving short-length codes, we demonstrate that classic decoders can also be used with real-valued, neural encoders, i.e., deep-learning based “codeword” sequence generators. Here, the classical decoder can be a valuable tool to gain insights into these neural codes and shed light on weaknesses. Specifically, the turbo-autoencoder is a recently developed channel coding scheme where both encoder and decoder are replaced by neural networks. We first show that the limited receptive field of CNN-based codes enables the application of the BCJR algorithm to optimally decode them with feasible computational complexity. These MAP component decoders then are used to form classical (iterative) turbo decoders for parallel or serially concatenated CNN encoders, offering a close-to-ML decoding of the learned codes. To the best of our knowledge, this is the first time that a classical decoding algorithm is applied to a non-trivial, real-valued neural code.  Furthermore, as the BCJR algorithm is fully differentiable, it is possible to train, or fine-tune, the neural encoder in an end-to-end fashion.

§ INTRODUCTION

Over the past decades, powerful and efficient coding schemes for long block lengths such as turbo codes <cit.> and LDPC codes <cit.> have been developed which perform near the Shannon capacity of the AWGN channel.
However, good encoder-decoder pairs for short block lengths (i.e., in the order of a hundred bits) which are required for URLLC scenarios are yet to be found.

Meanwhile, deep learning based methods made their way into communication systems and channel coding in particular.
Early works <cit.> <cit.> demonstrated that general NN are able to learn the decoding of classically (i.e., polar and convolutionally) coded sequences.
Models working on the Tanner graph of a code were shown to even outperform classical decoding algorithms for short codes <cit.> <cit.>.

Soon, the question arose whether also the encoder (i.e., the code itself) can be learned as a NN. The use of an autoencoder to jointly optimize encoder and decoder was proposed in <cit.>. The main difference to previous methods is that the autoencoder is trained end-to-end in an unsupervised fashion.

However, naïve implementations of autoencoders do not scale to practical block lengths and, thus, do not exhibit competitive coding gains.
The authors of <cit.> propose the turbo-autoencoder to solve these issues.
The main idea is to use two or more concatenated CNN-based encoder networks together with iterative decoding networks — the same principle as used in turbo codes.

Recent research of the turbo-autoencoder involved speeding up training using Gaussian pre-training <cit.>. Others interpreted turbo autoencoder and classically decoded it based on an equivalent binary, non-linear generator polynomial <cit.>. In contrast, we focus on real-valued neural codes.

Moreover, <cit.> demonstrated that the serial turbo-autoencoder can be extended to also perform joint detection, synchronization and equalization, outperforming state-of-the-art classical schemes.
Another related concept of learning encoder and decoder, using the archetype of polar codes (rather than turbo codes), was proposed in <cit.>.

Selected related work on classical and deep-learning based channel coding schemes

Classical Decoding     Deep Learning based Decoding

Classical      Turbo codes <cit.>LDPC codes <cit.>     Dense Neueral Networks <cit.>RNN-based conv. decoding <cit.>Weighted belief propagation <cit.>Graph Neural Networks <cit.>

Deep Neural Encoder     This work     Autoencoder <cit.>Turbo Autoencoder <cit.> K.O. codes <cit.>

Table <ref> arranges these prior works in two dimensions: Whether classical (i.e., engineered) or deep NN are used for the encoder and decoder, respectively. The evolution outlined above moved from classical systems to first making the decoder trainable, and then learning the encoder as well. A natural question to ask is whether it is also possible to use classical decoders with neural encoders.
As classical decoders are better understood, this can serve as a valuable tool to analyze and fine-tune neural codes.

The main contributions of this paper are as follows:

* We derive a low-complexity method for analyzing the effective memory of the CNN encoders and show that it is shorter than what their receptive field would allow.

* We present how the BCJR algorithm can be applied to decode learned, real-valued CNN codes with quasi-MAP performance.
The BCJR-based turbo decoder is then applied to decode trained SCNC.

* We show that using the BCJR-based decoder, neural encoders can be fine-tuned or even trained from scratch to obtain improved performance.

§ PRELIMINARIES

§.§ Convolutional Neural Networks

A commonly used class of feed-forward neural networks are CNN.
Each layer can be mathematically described by a convolution operation of the input and trainable filter kernels, followed by a non-linear activation function.
The CNN used in the turbo autoencoder use one-dimensional (1D) convolutions over the positional axis.
Each layer can have a depth of multiple such one-dimensional feature maps and has independent trainable filter kernels of size $\kappa$ for each combination of input and output feature maps.
As illustrated in Fig. <ref>, the output of any layer only depends on $\kappa$ elements of the previous layer. Hence, after $L$ convolutional layers with kernel size $\kappa$, the number of input symbols affecting  each output symbol is limited to $R=L\cdot(\kappa-1)+1$. This number is called the receptive field of the CNN.

start chain = 1 going below,    node distance = 0pt,
cell/.style=draw, fill=white, minimum width=2em, minimum height=2em,
outer sep=0pt, on chain,

[cell] (u0) at (0,4) ;
in 1,...,10
[cell] (u) ;

[cell, right=3.4cm of u0, yshift=.4cm] (vg0) ;
in 1,...,10
[cell] (vg) ;

[cell, right=3.2cm of u0, yshift=.2cm] (vf0) ;
in 1,...,10
[cell] (vf) ;

[cell, right=3cm of u0] (v0) ;
in 1,...,10
[cell] (v) ;

[cell, right=2.4cm of v0, yshift=.4cm] (xg0) ;
in 1,...,10
[cell] (xg) ;

[cell, right=2.2cm of v0, yshift=.2cm] (xf0) ;
in 1,...,10
[cell] (xf) ;

[cell, right=2cm of v0] (x0) ;
in 1,...,10
[cell] (x) ;

[dashed] (x5.north west) to (v3.north east);
[dashed] (x5.south west) to (v7.south east);

[dashed]  (v3.north west) to ($(v3.north west)!0.30!(u1.north east)$);
[loosely dotted]  ($(v3.north west)!0.30!(u1.north east)$) to ($(v3.north west)!0.70!(u1.north east)$);
[dashed] ($(v3.north west)!0.70!(u1.north east)$) to (u1.north east);

[dashed]  (v7.south west) to ($(v7.south west)!0.30!(u9.south east)$);
[loosely dotted]  ($(v7.south west)!0.30!(u9.south east)$) to ($(v7.south west)!0.70!(u9.south east)$);
[dashed] ($(v7.south west)!0.70!(u9.south east)$) to (u9.south east);

[] at (x5) $i$;
[xshift=-1cm] at (v3) $i-\frac{\kappa-1}{2}$;
[xshift=-1cm] at (v7) $i+\frac{\kappa-1}{2}$;

[] at ($(u5)!0.5!(v5)$) $\cdots$;

[above=0.3cm of u0, text width=3cm, text centered] Input Layer
[above=0.3cm of v0, text width=3cm, text centered] Layer $L-1$
[above=0.3cm of x0, text width=3cm, text centered] Layer $L$

[<->] ([shift=(-0.7cm,0)] u9.south) – ([shift=(-0.7cm,0)] u1.north)  node [midway,sloped, yshift=0.3cm] receptive field $R=L(\kappa-1)+1$;

[<-] ([shift=(0.4cm,-0.3)] x10.south) – ([shift=(.9cm,0.2)] xg10.south)  node [midway,sloped, yshift=-0.3cm] depth;

[->] ([shift=(.9cm,0.2)] xg10.south) – ([shift=(.9cm,2.0)] xg10.south)  node [midway,sloped, yshift=-0.3cm] position;

Visualization of the receptive field of a CNN with kernel size . The positional axis is arranged vertically, the different feature maps are indicated in the depth dimension.

§.§ Serial Turbo Autoencoder based System

General-purpose DNN or CNN are not able to learn codes that are comparable to state-of-the-art (binary) channel codes due to the curse of dimensionality <cit.>. To overcome this problem and learn competitive, neural channel codes for messages $\uv$ longer than $k>16$ bits, the serial turbo-autoencoder <cit.> has been proposed.
The serial turbo-autoencoder is an autoencoder that employs the structure of serially concatenated codes with an iterative decoder.
The architecture is displayed in Fig. <ref> for the encoder and in Fig. <ref> for the decoder.
While the overall structure of this end-to-end trainable system is carefully chosen, the constituent components are general-purpose CNN. In addition to the proposed  turbo-autoencoders in <cit.>, we employ the advancements of circular convolutions from <cit.> and the linear interleaver from <cit.>. Note, that the circular padding in the encoder can be interpreted as tail-biting encoding.

Fig. <ref> shows the encoder of the serial turbo-autoencoder.
It consists of a cascade of an outer encoder $f_\mathrm{O}(\cdot)$, an interleaver and an inner encoder $f_\mathrm{I}(\cdot)$.
The hyperparameters that are used for all CNN are shown in Tab. <ref>.
The CNN of the outer encoder sees the uncoded message $\uv \in \left\{-1,1 \right\}^{k}$ and encodes it into a real-valued sequence $\cv_{\RR}^{k \times F_{\cv}}$, where $F_{\cv}$ is a hyperparameter denoting the depth of the outputs (illustrated  by  double  arrows  in  Fig. <ref>).
Subsequently, the real-valued, coded sequence $\cv_{\RR}$ is binarized, so  $\cv = \operatorname{sign}\left(\cv_{\RR}\right)$. Note, as the derivative $\frac{\partial \operatorname{sign}(x)}{\partial x}=0$ almost everywhere, we replace the derivative for back-propagation with a linear function with saturation effects

\begin{equation}
\frac{\partial \operatorname{sign}(x)}{\partial x} = \begin{cases}
1 &\text{for $|x|<1$}\\
0 &\text{else.}
\end{cases}
\end{equation}

This is known as the saturated STE <cit.>.
In a next step, the coded sequence $\cv$ with $k$ positional entries and $F_\mathrm{\cv}$ depth entries is interleaved over the positional dimension.
Finally, the inner encoder (Enc I), which acts also as modulator, computes the $n$ symbols $\xv^{k\times F_\xv}$ which then are normalized and transmitted.
Note, that the hyperparameter $F_\xv$ defines the rate $r=\nicefrac{k}{n}=\nicefrac{1}{F_\xv}$ of the SCNC.

The iterative structure for decoding serially concatenated codes is shown in Fig. <ref>.
The inner decoder (Dec I) sees the channel observations $\yv =\xv +\nv$ with $\nv\sim\mathcal{N}(0,\sigma^2)$ and the a priori information from the outer decoder (Dec O).
In the first iteration the a priori information is initialized as  $\Lm_{\cv,\pi}^{'\mathrm{E}}=\mathbf{0}$, where the subscript, i.e., $\cv$, denotes the bits which the information is about.
Then the extrinsic information  of the inner decoder is interleaved and forwarded to the outer decoder. The outer decoder outputs an estimate $\Lm_{\uv}^\mathrm{T}$ of the uncoded bits $\uv$ and the extrinsic information $\Lm_{\cv}^{'\mathrm{E}}$, which is the a priori information for the inner decoder.

dots=[line width=1pt , line cap=round,dash pattern=on 0pt off 4pt,black]

arrow = [thick,->, rounded corners=5pt]

loosely dotted=          [dash pattern=on off 8pt]

NN =[rectangle, rounded corners,   align=flush center,minimum width=3cm,minimum height=1.0cm,text centered,
draw=red, fill=red!0 , thick , label=[red]:NN];

box =[rectangle,   align=flush center,minimum width=1.0cm, minimum height=1.1cm,text centered,
draw=black, fill=red!0 , thick, text=black];
box1 =[rectangle,  dashed,  align=flush center,minimum width=4.5cm, minimum height=4.5cm,text centered,
draw=blue, fill=red!0 , thick , label=[black]:];
boxText =[rectangle,  align=flush center,minimum width=15.74776cm,
minimum height=15.74776cm,text centered,
draw=black!50, fill=red!0 , thick ];
interleaver =[rectangle,   align=flush center,minimum width=1.0cm, minimum height=1.0cm,text centered,
draw=black, fill=red!0 , thick, text=black];
plus =[circle,draw=black, fill=white, inner sep=2pt,minimum size=5pt];

[box1, draw=apfelgruen, minimum width=2.7cm, minimum height=1.8cm, label=[black]:Outer encoder $f_\mathrm{O}(\cdot)$] at (0.7,4.1) ;

[box1, draw=apfelgruen, minimum width=2.75cm, minimum height=1.8cm, label=[black]:Inner encoder $f_\mathrm{I}(\cdot)$] at (6.11,4.1) ;

[] (start) at (-1.75,4) ;
[box,label=[black]:] (enc1) at (0,4) Enc O
[interleaver] (sign) at (1.5,4) Bina-
[interleaver] (int) at (3.85,4) $\vec{\pi}$;
[box,label=[black]:] (enc2) at (5.35,4) Enc I
[box,label=[black]:] (norm) at (6.85,4) Norm;

[arrow] (start)– node[midway, above]$\uv \in $node[midway, below]$ \mathbb{F}_2^{k } $ (enc1);
[arrow,double] (enc1) – node[midway, above]$\cv_{\RR}$node[midway, below] (sign);
[arrow,double] (sign) – node[midway, above]$\cv \in $  node[midway, below]$\{ \pm 1 \}^{k \times F_{\cv}}$ (int);
[arrow,double] (int) – node[midway, above]$\cv_{\pi}$ (enc2);
[arrow] (enc2) – node[midway, above] (norm);
[arrow] (norm) – (8.,4)  node[midway, above]$\xv$ node[midway, below, xshift=.2cm]$\in \mathbb{R}^{n} $;

Serially concatenated CNN-based encoder structure of the transmitter learning a real-valued SCNC.

dots=[line width=1pt , line cap=round,dash pattern=on 0pt off 4pt,black]

arrow = [thick,->]

loosely dotted=          [dash pattern=on off 8pt]

NN =[rectangle, rounded corners,   align=flush center,minimum width=3cm,minimum height=1.0cm,text centered,
draw=red, fill=red!0 , thick , label=[red]:NN];

box =[rectangle,   align=flush center,minimum width=1.0cm, minimum height=1.1cm,text centered,
draw=black, fill=red!0 , thick, text=black];
box1 =[rectangle, dashed,  align=flush center,minimum width=4.5cm, minimum height=4.5cm,text centered,
draw=blue, fill=red!0 , thick , label=[black]:];
boxText =[rectangle,  align=flush center,minimum width=15.74776cm,
minimum height=15.74776cm,text centered,
draw=black!50, fill=red!0 , thick ];
interleaver =[rectangle,  align=flush center,minimum width=1.0cm, minimum height=1.0cm,text centered,
draw=black, fill=red!0 , thick, text=black];
plus =[circle,draw=black, fill=white, inner sep=2pt,minimum size=5pt];

[box1, draw=mittelblau, minimum width=5.33cm, minimum height=3.5cm, label=[black]:Decoder] at (2.02,3.4) ;

[] (start) at (-1.7,4) ;
[box,label=[black]:Inner] (dec1) at (0,4) Dec I
[box,label=[black]:Outer] (dec2) at (4,4) Dec O
[interleaver] (deint) at (2,4) $\vec{\pi}^{-1}$;
[interleaver] (int) at (2,2.5) $\vec{\pi}$;

[arrow] (start)– node[pos=0.5, above]$\vec{y}$ (dec1);
[arrow,double] (dec1) – node[pos=0.5, below]$\Lm_{\cv,\pi}^\mathrm{E}$ (deint);
[arrow,double] (deint) – node[pos=0.5, below]$\Lm_{\cv}^{\mathrm{E}}$ (dec2);

[arrow] (dec2) –node[pos=0.6, above]$\Lm_{\uv}^\mathrm{T}$node[pos=0.6, below]$\in \RR^{k} $ (5.5,4);
[arrow,double] (dec2) |-node[pos=0.65, above]$\Lm_{\cv}^{'\mathrm{E}}$node[pos=0.65, below]$\in \RR^{k \times F} $ (int);
[arrow,double] (int) -| node[pos=0.5, near start, below]$\Lm_{\cv,\pi}^{'\mathrm{E}}$ (dec1);

Serial CNN- or BCJR-based turbo decoder structure.

The serial turbo-autoencoder can be trained in an end-to-end fashion as all components are differentiable.
We employ the same training methods as in <cit.>.
The hyperparameters for the training are shown in Tab. <ref>.
Note, we use a random interleaver for training  and a linear interleaver during inference.
As loss function we chose the BCE function.
It is worth mentioning that the learning schedule for the encoder and decoder is alternating. That means, in a loop we calculate $T_\mathrm{TX}$ SGD updates for the whole encoder while the decoder weights are fixed, and in a next step, we calculate $T_\mathrm{RX}$ SGD updates for the whole decoder while the encoder weights are fixed.

Similar to <cit.> and <cit.> we observe a significant amount of single-bit errors after decoding.
This implies that the bit errors in the decoder are loosely correlated.
As a result, the BLER can be drastically improved by
an outer single-parity-check code
and flipping the bit with the least reliability in case of a not matching parity.
We consider the associated rate loss in form of a rate shift of $10\log_{10}(\nicefrac{k-1}{k})$ dB.

Hyperparameters for architecture and training the serial turbo-autoencoder

Parameter    Value

CNN filters     100

CNN kernel     5

Layers per CNN     5

Receiver iters.     6

$F_c$, $F_\xv$     10, 2

$(k,n)$     $(64,128)$

Padding     Circular

Parameter    Value

Loss     BCE

$T_{\mathrm{TX}}$, $T_{\mathrm{RX}}$      $100$, $500$

Batch size     $500-4000$

Optimizer     ADAM

Learning rate     $10^{-4}-10^{-6}$

Encoder SNR     $4.0~\mathrm{dB}$

Decoder SNR     $0.5-4.0~\mathrm{dB}$

§.§ Overview of the BCJR Algorithm

This subsection shortly introduces the log-domain BCJR algorithm <cit.> for decoding in a serial turbo decoder.
The BCJR algorithm is an optimal decoder in the sense of the bit-wise MAP criterion.
This decoding criterion is formulated by $\arg \max_{u_i} p(u_i |\yv)$, where the bit $u_i$ is estimated based on the noisy sequence $\yv$.
While the decoding of every component code is optimal, the turbo decoder needs a sufficient amount of iterations until it converges close to the optimal performance of the concatenated code.
The algorithm is based on the following equations:

\begin{align}
\gamma_i(s',s) &= \log( p(u_i)p(y_i|u_i)) \label{eq:gamma}\\
\alpha_i(s) &= \underset{s'}{\max}^* \left[\alpha_{l-1}(s')+\gamma_i(s',s) \right]\label{eq:alpha}\\
\beta_{i-1}(s') &= \underset{s}{\max}^* \left[\beta_{i}(s)+\gamma_i(s',s) \right]\label{eq:beta}\\
\mathrm{max}^*(x,y) &\triangleq \log(\mathrm{e}^x+\mathrm{e}^y) \approx \max(x,y)\label{eq:max}
\end{align}

Equation (<ref>) is a branch metric that needs to be calculated for every state transition from $s'$ to $s$ where $u_i$ and $y_i$ are the $i$th bit and observation, respectively, with $i=0,\dots,N-1$.
The state transitions $(s',s)$ are defined by the encoder.
The recursive equations (<ref>) and (<ref>) are path metrics that traverse the trellis in a forward and a backward manner, respectively.
Whether we use equation (<ref>) or the given approximation coincides with the labeling log-MAP and max-log respectively.
The performance loss due to the approximation is usually omittable, if a suitable damping factor is applied after every BCJR decoder.
Once the branch and path metrics are calculated, a final soft decision can be made according to

\begin{multline}
L_{\uv,i}= \underset{U^1}{\max}^* \left[ \alpha_{l-1}(s') +\gamma_i(s',s) + \beta_{i}(s) \right] \\
- \underset{U^{-1}}{\max}^* \left[ \alpha_{l-1}(s') +\gamma_i(s',s) + \beta_{i}(s) \right]
\end{multline}

where $U^{1}$ and $U^{-1}$ are the sets that contain every transition that is associated with $u_i=1$ and $u_i=-1$, respectively.
Note that the choice of $U^{1}$ and $U^{-1}$ determines whether the output is regarding the coded or the uncoded bits.

In the case of serial turbo decoding as depicted in Fig. <ref>, the inner decoder sees the channel observation $\yv$ and the a priori information  $L_{\cv,i}'^{\mathrm{E}} $ of the outer decoder. Therefore, (<ref>) is calculated as

\begin{equation}\label{eq:gamma_inner}
\gamma_i(s',s) = \frac{1}{2} L_{\cv,i}'^{\mathrm{E}} \cdot c_{s',s} - \frac{|y_i-x_{s',s}|^2}{2\sigma^2 }
\end{equation}

where $c_{s',s}$ is the true coded bit for the transition $(s',s)$ and $\sigma^2$ is the noise power.
In contrast, the outer decoder only receives the a priori information from the inner decoder, and thus

\begin{equation}\label{eq:gamma_out}
\gamma_i(s',s) = \frac{1}{2} L_{\cv,i}^{\mathrm{E}} \cdot c_{s',s}.
\end{equation}

As all presented equations are differentiable we can calculate a gradient though the BCJR decoders and, therefore, also the (unrolled) turbo decoder.
This enables the BCJR algorithm to be used in the turbo decoder structure not only for decoding, but also training  in an end-to-end fashion.

A last comment needs to be made in this section regarding the tail-biting (circular padding) structure of the encoder.
The structure determines that the state at the first position $i=0$ equals the state after the last position $i=N$.
Thus, we need to set the initial path metrics $\alpha$ and $\beta$  accordingly.
However, the initial state is unknown.
One way to circumvent this problem is to consider the wrap-around trellis <cit.>.
The initialization of the path metrics begins $w$ positions before the first position at index $i_\mathrm{start}=-w \mod N$ and $i_\mathrm{end}=N+w \mod N $ for $\alpha$ and $\beta$, respectively.
In other words, the sequence $\yv'=\left[ y_{-w\mod N},y_{-w+1\mod N},...,y_{N+w\mod N} \right]$ is decoded instead of $\yv=\left[ y_{0},y_{1},...,y_{N-1} \right]$.

§ BCJR DECODING OF NEURAL CODES

In this section we derive how to decode a learned SCNC with a classical decoder, namely the BCJR decoder. Note that for this purpose, we assume that the SCNC has been already trained in a turbo-autoencoder, as shown, e.g., in <cit.>.

§.§ Estimating the Effective Memory of the Neural Codes

This subsection shows how to estimate the effective memory of a CNN from the gradient and that the gradient is, indeed, a useful indicator.
We define that the effective memory of the CNN consists of those input positions inside the receptive field of each output that significantly contribute to the output value.
Whether decoding of CNN-based codes is feasible or not using the BCJR algorithm depends on the memory size of the CNN which exponentially relates to the number of states in the trellis.
As each BCJR decoder computes the likelihood of every possible symbol (inner decoder) or bit (outer decoder) of the respective encoder at every position, we need to pre-calculate the possible symbols and bits for the state transitions.
The number of possible symbols for an encoder CNN is $M=\left| \mathcal{F} \right|^m$, where $\left| \mathcal{F} \right|$ is the number of elements of the set of the input to the CNN and $m$ is the effective size of the memory of the CNN.
Note that the filters in the CNN are not causal and, thus, the memory does not only include past inputs but also future inputs.
The memory of a CNN is upper bounded by the receptive field that is determined by the CNN structure.
In this case the receptive field is $R=21$ which could result in up to $M=\left| \mathbb{F}_2 \right|^{R}\approx2\cdot 10^7$ symbols for the binary field $\mathbb{F}_2$.
The usage of the binarizer ensures that all inputs to the CNN in the encoder are binary.

As BCJR decoding with $2^{R-1}$ states is not feasible, positions with low contribution to the output can be pruned. We propose an easy way to measure the contribution by calculating the average energy of the gradient

\begin{align}\label{eq:grad}
e_{\partial, \vv}(j) &=  \operatorname{E}\left[ \left| \frac{\partial f(\vv)}{\partial v_j} \right|^2 \right]\\
&\approx \frac{1}{N_\mathrm{B}}\sum_{i=0}^{N_\mathrm{B}} \left| \frac{\partial f(\vv_i)}{\partial v_{i,j}} \right|^2
\end{align}

where $\vv_i$ is the $i$th sample that is fed to the neural network $f(\cdot)$ from a batch of size $N_\mathrm{B}$.
We interpret the expected gradient (change of the output by changing the input) as the exhibited expected error of omitting an input position.
The gradient is usually a by-product of any deep learning framework, and therefore Eq. (<ref>) is readily available.
In our case it is sufficient to consider a single output, as the weight-sharing property of the CNN and the circular padding ensure that each output exhibits the same dependencies to its relative input positions.
In a next step, we compare the energy of the gradients for each encoder to a threshold $t$ to obtain the effective size $m$ of the memory. If the energy is larger than the threshold, then we consider this position to contribute significantly to an output and, thus, is part of the memory. We denote the set of positions from the first to the last contributing position by $\mathcal{J}$ with $|\mathcal{J}|=m+1$ and all positions outside $\mathcal{J}$ by $\bar{\mathcal{J}}$.

Since the gradient only describes the behavior around the inputs $\pm1$, we need to verify that a bit flip from $\pm1$ to $\mp1$, respectively, exhibits a similar behavior.
For this, we calculate the expected energy of the difference quotient per bit position for a bit flip <cit.> as

\begin{align}\label{eq:diff}
e_{\Delta,\vv}(j) &= \operatorname{E}\left[  \frac{\left|f(\vv)-f(\vv\odot(1-2\ev_j))\right|^2}{|| 2\ev_j ||^2} \right]\\ %
&= \frac{1}{4\cdot2^{R}}\sum_{\vv\in\{\pm1\}^R} \left|f(\vv)-f(\vv\odot(1-2\ev_j))\right|^2
\end{align}

where $\ev_j$ denotes the unit vector which is 1 exactly in position $j$ and $\odot$ denotes element-wise multiplication.
Note, Eq. (<ref>) corresponds to the actual expected error of omitting an input position.
In the following analysis of the encoder CNN we empirically verify that Eq. (<ref>) is, indeed, an approximation of Eq. (<ref>) up to a scaling factor.

§.§ Outer Neural Encoder with Binary Output

Equivalent encoder block diagram. The outer encoder learns two output streams. Splitting the inputs to the inner encoder reduces the size of the memory $m_\xv^{(0)}$ and $m_\xv^{(1)}$, respectively, to allow BCJR decoding at feasible complexity.

Estimating the effective memory through the normalized expected error ($e_{\partial}$ approximation, $e_{\Delta}$ verification) of omitting input $u_j$ in relation to each output for the outer encoders $f_\mathrm{O}(\cdot)$. Note, the linear interpolation between points is only shown for visualization purposes.

The outer encoder implements the mapping $f_{\mathrm{O}}:\mathbb{F}_2^k \to \mathbb{F}_2^{k\times F_c} $.
This means in general the CNN can learn an encoding with a coding rate as low as $\nicefrac{1}{F_c}$, here $F_c=10$.
We observed, however, that only two out of the $F_c$ depth dimensions of the output vary dependent on the input, while the remaining eight depth dimensions stay fixed at either $1$ or $-1$ for all possible inputs. Therefore, a rate of $\nicefrac{1}{2}$ is learned which corresponds to the overall code rate of the concatenated neural encoders.
We label the two dimensions with varying output $\cv^{(0)} = f_{\mathrm{O}}^{(0)}(\uv)$ and $\cv^{(1)} = f_{\mathrm{O}}^{(1)}(\uv)$.
This is illustrated in Fig. <ref>.
It is remarkable to realize that the CNN learns this on its own without any injection of expert knowledge.
Moreover, this observation aligns with the derivation from <cit.> that an outer code with a rate larger than the overall rate of a serial concatenated code is sub-optimal for iterative decoding.

Note, Eq. (<ref>) captures the effects of the binarizer, while Eq. (<ref>) does not, due to the STE.
The obtained plot for $\cv_0$ and $\cv_1$ is shown in Fig. <ref>.
First of all, we want to mention that we expect a deviation of the approximation Eq. (<ref>) as it does not capture the influence of the binarizer.
Indeed, we observe a difference, albeit the general behavior matches.
More interesting are the obtained values for $e_{\Delta}$ which are either $1$ or $0$.
This means that a bit flip in the respective input position either leads to a bit flip in all cases or in none.
As a consequence, the outer encoder is equivalent to the linear generator polynomials $G_0(Z)=Z^{-2}+Z^0+Z^{1}$ and $G_1(Z)=Z^{-1}+Z^0+Z^{3}$.
This coincides with the reasoning from <cit.> that the outer codes of serially concatenated turbo codes may be non-recursive and linear, while the inner codes have to be recursive or non-linear.
The implication for decoding is that we can either run two BCJR decoders with memories $m_{\cv}^{(0)} = 3$, $m_{\cv}^{(1)} = 4$ and aggregate the bit estimates or run one BCJR decoder with memory $m_{\cv}=5$.
The first option can be chosen to save complexity.
With this knowledge, we can pre-compute the output bits $c_{s',s}$ for each of the $2^{m_{\cv}+1}=32$ state transitions $(\mathrm{s',s})$.
Finally, we can plug $c_\mathrm{s',s}$ into the Eq. (<ref>) and decode the outer neural code.

§.§ Inner Neural Encoder with Real-valued Output

Estimating the effective memory through the normalized expected error ($e_{\partial}$ approximation, $e_{\Delta}$ verification) of omitting input $c_j$ in relation to each output for the inner encoders $f_\mathrm{I}(\cdot)$. Positions with an error exceeding the threshold belong to the set of contributing positions $\mathcal{J}$.

As we established in the last section that the coded bits $\cv$ have essentially dimensions $k\times 2$, the inner encoder learns the mapping $\mathbb{F}_2^{k\times 2}\to \mathbb{R}^{k\times 2} $.
A problem is that the second depth dimension of the input roughly doubles the memory.
To circumvent this problem, we first encode $\cv$ with $\cv^{(1)}=\mathbf{1}$ and then encode $\cv$ with $\cv^{(0)}=\mathbf{1}$.
This is illustrated in Fig. <ref>.
Subsequently, we concatenate the first depth dimension of the first output with the second depth dimension of the second output to form output symbols $\xv =\left[\xv^{(0)},\xv^{(1)} \right]$.
Effectively, the inner encoder consists of two functions $\xv^{(0)} = f_{\mathrm{I}}^{(0)}(\cv_{\pi}^{(0)})$ and $\xv^{(1)} = f_{\mathrm{I}}^{(1)}(\cv_{\pi}^{(1)})$.
Fig. <ref> shows Eq. (<ref>) and Eq. (<ref>) for both encoding functions.
We observe that the approximation by the gradient closely follows the actual difference.
Next we can set a threshold $t$ to obtain the memory.
We found that a threshold of $t=2\cdot10^{-2}$ does not degrade the final BLER.
This leads to memory size $m_{\xv}^{(0)}=7$ for $f_{\mathrm{I}}^{(0)}(\cdot)$ and $m_{\xv}^{(1)}=2$ for $f_{\mathrm{I}}^{(1)}(\cdot)$.
However, the verification error $e_{\Delta}$ of $f_{\mathrm{I}}^{(1)}$  indicates  that actually an encoding without significant memory is learned. Thus, the learned encoding function $f_{\mathrm{I}}^{(1)}$ can be interpreted as a scaled BPSK.

For the calculation of $x_{s',s}$ in Eq. (<ref>), we average $x=f(\cv)$ with $\cv = \left[ c_{-\nicefrac{R}{2}},...,c_{\nicefrac{R}{2}} \right]$ over all possible values of $c_j$ in positions $j\in\bar{\mathcal{J}}$ and $c_j$ fixed for all $j\in\mathcal{J}$. Note that each transition $(s',s)$ corresponds to exactly one value of the sub-vector $\cv_{\mathcal{J}}$ that contains $c_j \forall j \in \mathcal{J}$. This is done for both $\cv^{(0)}$ and $\cv^{(1)}$.

§ RESULTS

§.§ Decoding Performance

As performance metric we choose the BLER over the BER, as it is more relevant in systems with short block lengths.
In our transmission model we consider an AWGN channel.
Further, we aim to transmit $k=64$ bits over $n=128$ channel uses, thus $r=0.5$.
The threshold for memory estimation is set to $t=2\cdot 10^{-2}$ and we extend the wrap-around BCJR by $w=16$ positions in both directions.
Fig. <ref> shows the performance of the log-MAP BCJR-based turbo decoder for the SCNC with 6 iterations (<ref>) and 32 iterations(<ref>)  respectively.
Additionally, a curve without a parity bit and bit flip (<ref>) is displayed, as described in section <ref>.
Recall that the number of states for the outer BCJR is set to $2^{m_\cv} = 32$ and for the inner BCJR to $2^{m_{\xv}^{(0)}}=128$ and $2^{m_{\xv}^{(1)}}=2$.
For comparison, the same SCNC  is shown with the matching CNN-based turbo decoder (<ref>).
Additionally, several baselines from <cit.> are shown and an additional LTE  turbo code (<ref>) <cit.> with 8 iterations of damped max-log decoding.
We observe that the BCJR with 6 decoder iterations (<ref>) shows slightly worse performance than the CNN decoder (<ref>).
However, 32 BCJR iterations (<ref>) yield a significant gain for lower SNR, but a omittable gain for higher SNR.
Thus, we conclude that the NN-based decoder operates close to its optimal performance for a high SNR.

Simulated BLER over an AWGN channel ($k=64$, $n=128$) for the SCNC with a BCJR-based and a NN-based turbo decoder, respectively.

§.§ Insights about the Turbo-Autoencoder

From the decoding of the SCNC with a classical decoder, we obtained insights regarding the turbo-autoencoder.
To summarize the observations so far,
Section <ref> showed that the outer encoder learned a linear encoding and one of the inner encoder exhibits systematic behavior.
The decoding results show that the learned NN-based decoder slightly outperforms the log-MAP BCJR-based decoder.
Thus, the NN-based decoder displays close to optimal performance for the SCNC.
This sets the spotlight on the encoder for potential improvements in performance.
For both decoders we noticed a significant amount of single erroneous bits in a block error.
For a SNR of $4.5$ dB the percentage of block errors with a single erroneous bit to all block errors is $71.5\%$ and $66.0\%$ for the BCJR-based and NN-based turbo decoder respectively.
Conclusively, the minimal distance between code words seems to be small.
Thus, the BLER is sub-optimal without an outer error-correcting code, which improves the minimum distance.
A major factor for this behavior might be the BCE loss function, as it is known to minimize the BER and not the BLER <cit.>.

§.§ Training with BCJR decoders

As all equations for the BCJR-based turbo decoder are differentiable, we can train a neural encoder with a classical decoder.
Here, we want to showcase  this possibility for a reduction of the size of the memory $m_\xv^{(0)}=7\to 3$ and the number of decoder iterations $6\to 2$.
The performance for this case for the not fine-tuned SCNC (<ref>) is displayed in Fig. <ref>.
The fine-tuned SCNC with the same decoder (<ref>) shows significant gain.
Also it is possible to learn an encoder from scratch, i.e., the inner encoder (<ref>).
For reference, we plot the LTE turbo code with 2 iterations of damped max-log decoding (<ref>).
The hyperparameters for fine-tuning are listed in Tab. <ref>.
We found that a high batch size and a small learning rate are essential.
Furthermore, the log-MAP BCJR lead to a smoother training with better performance than the max-log approximation.

Hyperparameters for training and fine-tuning

Parameter    Value

Batch size     1000

Learning Rate     $10^{-5}$

SNR     4

Parameter    Value

Loss     BCE

Receiver iters.     2

SGD updates     1000

Simulated BLER over an AWGN channel ($k=64$, $n=64$). Showcasing the training of SCNC for a BCJR-based decoder with reduced memory $m_\xv^{(0)}=7\to 3$ and reduced iterations $6\to2$.

§ CONCLUSION

In this paper, we demonstrated that the BCJR decoding algorithm can be applied to CNN-based encoders. To mitigate computational complexity, we showed that only a fraction of the potentially learned memory in the encoder is required to decode the sequences, limiting the number of states in the decoder trellis. The resulting BCJR-based decoders can decode and fine-tune codes that have been learned as a turbo-autoencoder or even learn neural codes from scratch.


# On the experimental verification of the uncertainty principle of
position and momentum

Thomas Schürmann  Jülich Supercomputing Centre, Jülich Research Centre,
D-52425 Jülich, Germany<EMAIL_ADDRESS>Ingo Hoffmann  Winfrid
Görlich  Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany

###### Abstract

Historically, Kennard was the first to choose the standard deviation as a
quantitative measure of uncertainty, and neither he nor Heisenberg explicitly
explained why this choice should be appropriate from the experimental physical
point of view. If a particle is prepared by a single slit of spatial width
$\Delta x$, it has been shown that a finite standard deviation
$\sigma_{p}<\infty$ can only be ensured if the wave-function is zero at the
edge of $\Delta x$, otherwise it does not exist TS09 . Under these
circumstances the corresponding sharp inequality is $\sigma_{p}\Delta
x\geq\pi\hbar$. This bound will be reconsidered from the mathematical point of
view in terms of a variational problem in Hilbert space and will furthermore
be tested in a 4f-single slit diffraction experiment of a laser beam. Our
results will be compared with a laser-experiment recently given by Fernández-
Guasti (2022) G22 .

##  I 1\. Introduction

The diffraction of particles by a single slit has often been discussed as an
illustration of Heisenberg’s uncertainty relations and their role in the
process of measurement. In the case of particles passing through a slit of
width $\Delta x$ in a diaphragm of some experimental arrangement, the
diffraction by the slit of the wave implies a spread in the momentum of the
particles, which is greater the narrower the slit. This phenomenon is an
example of the famous Heisenberg principle H27 H30 . The most familiar
formalization of the uncertainty principle is in terms of standard deviations
K27

$\displaystyle\sigma_{x}\sigma_{p}\geq\hbar/2$ (1)

with the reduced Planck constant $\hbar=h/2\pi$.

However, for particles passing through a slit of width $\Delta x$, the
diffraction of the incoming wave function $\psi$ has to be considered as a
preparation corresponding to the ordinary von Neumann-Lüders projection. This
approach is often applied in the actual experimental design of the uncertainty
relation S69 Le69 Z03 . The advantage is that the localization of the
particles is simply given by the width $\Delta x$ of the slit. Unfortunately
this approach cannot be considered as a rigorous experimental test of
Kennard’s expression (1) because $\Delta x$ and $\sigma_{x}$ are quite
different measures of localization. An obvious interrelation between $\Delta
x$ and $\sigma_{x}$ is given by the Popoviciu inequality for variances Po

$\displaystyle\sigma_{x}\leq\frac{\Delta x}{2}$ (2)

while equality holds precisely when half of the probability is concentrated at
each of the two boundary points of $\Delta x$. From (1) and (2), the following
inequality is immediately implied

$\displaystyle\sigma_{p}\Delta x>\hbar$ (3)

However, the bound $\hbar$ on the right-hand side can never be reached. This
is because a finite value of $\sigma_{p}<\infty$ can only be ensured if the
wave-function is zero at the edge of $\Delta x$, otherwise it does not exist.
According to this condition the following sharp inequality has been introduced
TS09

$\displaystyle\sigma_{p}\Delta x\geq\pi\hbar.$ (4)

Since the standard deviation $\sigma_{p}$ is (by definition) corresponding
only to the ”one-sided” deviation of the probability density it is sometimes
obvious to measure the momentum uncertainty $\Delta p$ according to the
definition

$\displaystyle\Delta p=2\sigma_{p}.$ (5)

With this definition, the inequalities (3) and (4) can equivalently be
rewritten as

$\displaystyle\Delta x\Delta p>2\hbar$ (6)

and

$\displaystyle\Delta x\Delta p\geq 2\pi\hbar.$ (7)

Actually, the sharp inequality (7) can be considered as a mathematically
rigorous version of the heuristic expression $\Delta x\Delta p\gtrsim h$
originally introduced by the measurement process of Heisenberg in H27 H30 ,
before Kennard came with the alternative measurement approach corresponding to
inequality (1).

Recently it has been mentioned that there is a quantum optical realization of
the position and momentum uncertainty relation in the quantum limit G22 .
Interference of two non-collinear photon modes with different frequencies are
space and time resolved and the detection is performed in the same space-time
region. It is argued that evaluation of the photon momentum from the position
versus time interferograms makes this procedure similar to the mechanical
momentum construct. The measured uncertainty $\Delta p_{x}$ in G22  is applied
to approximate twice as much of the standard deviation $\sigma_{p}$ in order
to verify the inequalities (6) and (7). The experimental factors corresponding
to the right-hand side of (6) and (7) are given by $1.128\hbar$, $2.464\hbar$
and $2.723\hbar$. That is, inequality (6) is experimentally falsified for the
first measurement and expression (7) for all three cases. This conclusion
seems at least questionable, such that a closer look at the measurement in G22
seems appropriate at this point. To do so, let us consider the experimental
determination of $\Delta p_{x}$ given in G22  by

$\displaystyle\Delta
p_{x}=\hbar\,\frac{\delta\omega}{2}\cdot\frac{\dot{x}_{\text{max}}-\dot{x}_{\text{min}}}{\dot{x}_{\text{max}}\,\dot{x}_{\text{min}}}.$
(8)

The notation $\dot{x}=dx/dt$ is the velocity by which the fringes of the
interferograms are displaced in time and $\delta\omega$ the frequency shift of
the incoming modes expressed in Figs. 3, 4 and 5 of G22 . In the experimental
approach $\Delta p_{x}$ is considered to be proportional to the difference of
nearby maximum and minimum slopes. Actually, the slopes extrema were drawn by
hand prior to any calculation to avoid statistical bias. An arbitrary dark
fringe was chosen and parallel slopes to the median were drawn (the distance
between them was roughly half the distance between fringe maxima). The slopes
extrema have been obtained from the diagonals of the corresponding
parallelogram.

As far as can be seen it is neither obvious nor proven in G22  that expression
(8) is an appropriate approximation of (5). The importance of this aspect is
because the momentum uncertainties relevant in this context are exclusively
based on the notion of statistical standard deviations (or twice of them).
Actually, there is no remark in G22  why the difference of nearby maximum and
minimum slopes justifies a proper approximation of $2\sigma_{p}$. Only
proportionality of (8) and $\sigma_{p}$ has been mentioned, but this cannot be
sufficient for the conclusion given in G22 .

In order to classify the measure of momentum uncertainty (8) and the
corresponding measurement results, let us first refer to a corollary of the
measurement in Fernández-Guasti’s work G22  (p. 5):

”[…] The outcome of our experiments rule out this possibility $[\,\Delta
x\Delta p_{x}\geq\pi\hbar\,]$, provided that our uncertainty estimates are
equal or larger than the effective (overall) width of the distribution
functions U85 , since all reported values in units of $\hbar$ are smaller than
$\pi$.”

The position uncertainty $\Delta x$ satisfies the ”overall” width criterion
mentioned in this corollary (for $N\to 1$ in U85 , Eq.(13)).

However, according to the above corollary one also has to check if the
momentum uncertainty estimate $\Delta p_{x}$ of the measurement in Ref. G22
is such that the probability

$\displaystyle P(\Delta p_{x})=\int\limits_{-\Delta p_{x}/2}^{\Delta
p_{x}/2}|\varphi(p)|^{2}\,dp$ (9)

of the corresponding momentum density $|\varphi(p)|^{2}$ is sufficient close
to 1 (for instance $\geq 70\%$). Otherwise, according to the criteria in Ref.
U85 , the momentum uncertainty can not be considered to be well-defined.

In order to check this criterion (which has not been discussed for the
momentum uncertainty in Ref. G22 ), we refer to a general least upper bound
$\lambda_{0}$ of the probability expression (9), which is given by

$\displaystyle\text{P}(\Delta p_{x})\leq\lambda_{0}.$ (10)

This bound follows from the work of Landau and Pollak (LP61 ,Theorem 2),
Lenard (L72 , Proposition 11) or more recently in the context of quantum
physics from Lathi L86  and Busch et al. BHL07 . The bound is a continuous and
monotonically increasing function, $\lambda_{0}(\xi)$, of a single parameter
given by

$\displaystyle\xi=\frac{\Delta x\Delta p_{x}}{h},$ (11)

with $\lambda_{0}(0)=0$ and $\lambda_{0}(\xi)\to 1$, for $\xi\to\infty$ (for
details, see Fig. 2 in LP61 , or Theorem 1 in S08 ).

Now, all measurement results given in Ref. G22 , (p. 5), can be summarized as

$\displaystyle\Delta x\Delta p_{x}=a_{i}\hbar$ (12)

for $i=1,2,3$ and

$\displaystyle a_{1}$ $\displaystyle=$ $\displaystyle 1.128,$ (13)
$\displaystyle a_{2}$ $\displaystyle=$ $\displaystyle 2.464,$ (14)
$\displaystyle a_{3}$ $\displaystyle=$ $\displaystyle 2.723.$ (15)

After equating expression (11) and (12), the $\xi$-values in (11) are given by
$\xi_{i}=a_{i}/2\pi$, that is

$\displaystyle\xi_{1}$ $\displaystyle=$ $\displaystyle 0.179,$ (16)
$\displaystyle\xi_{2}$ $\displaystyle=$ $\displaystyle 0.392,$ (17)
$\displaystyle\xi_{3}$ $\displaystyle=$ $\displaystyle 0.433.$ (18)

The numerical values of the corresponding upper bounds are given by

$\displaystyle\lambda_{0}(\xi_{1})$ $\displaystyle=$ $\displaystyle 0.178,$
(19) $\displaystyle\lambda_{0}(\xi_{2})$ $\displaystyle=$ $\displaystyle
0.376,$ (20) $\displaystyle\lambda_{0}(\xi_{3})$ $\displaystyle=$
$\displaystyle 0.412.$ (21)

In all cases, the probability weight of the momentum density (9) is far below
$50\%$ and therefore $\Delta p_{x}$ in Ref. G22  cannot be considered as a
reasonable measure of uncertainty with respect to criteria in U85 . It follows
that the conditions for the applicability of the measurement corollary are not
satisfied.

In contrast, a reasonable upper bound $\lambda_{0}(\xi)=78\%$ is obtained for
$\xi=1$ or equivalently $a=2\pi$, such that instead of the expression $"\Delta
x\Delta p_{x}\sim 2\hbar"$ mentioned in the abstract of Ref. G22 , equation
(12) implies

$\displaystyle\Delta x\Delta p_{x}=2\pi\hbar.$ (22)

This expression is corresponding to our inequality (7) mentioned above.
Finally, a correction on the right-hand side in the definition (8) by a
prefactor of $\pi$ might be appropriate.

In the following section a mathematical proof of our main inequality (4) is
given in terms of a variational problem in Hilbert space and the state of
minimum uncertainty is derived. In Sec. 3 an experimental setup for the
verification of inequality (4) is presented. An approximated state of minimal
uncertainty is prepared in a 4f-laser experiment with the help of a Lanczos-
window to cut off higher spatial frequencies. From the data of the
interference pattern, the momentum distribution and the corresponding momentum
standard deviation $\sigma_{p}$ is determined. A discussion and conclusion is
given in Sec. 4.

##  II 2\. Variation problem in Hilbert space

Let us consider particles in one spatial dimension described by a wave
function $\psi$ which is an element of the Hilbert space ${\cal
H}=L^{2}(\mathbb{R})$, the space of square integrable functions on
$\mathbb{R}$. The scalar product in Hilbert space will be denoted by angular
brackets, that is to write $\langle\phi|\psi\rangle$ for the scalar product of
two state vectors $\phi,\psi\in{\cal H}$. Accordingly, the norm of $\psi$ is
given by $||\psi||\equiv\sqrt{\langle\psi|\psi\rangle}$. The point-spectrum of
the momentum eigenvalue equation

$\displaystyle\hat{p}|\varphi_{n}\rangle=p_{n}|\varphi_{n}\rangle$ (23)

on the interval $[-\frac{\Delta x}{2},\frac{\Delta x}{2}]$ of length $\Delta
x$ is given by

$\displaystyle\varphi_{n}(x)$ $\displaystyle=$
$\displaystyle\frac{1}{\sqrt{\Delta x}}\,e^{\frac{i}{\hbar}p_{n}x}$ (24)
$\displaystyle p_{n}$ $\displaystyle=$ $\displaystyle\hbar\,k_{n}$ (25)
$\displaystyle k_{n}$ $\displaystyle=$ $\displaystyle\frac{2\pi n}{\Delta x}$
(26)

for $n=0,\pm 1,\pm 2,...$ This complete set of solutions satisfies the
orthonormality relations

$\displaystyle\langle\varphi_{n}|\varphi_{m}\rangle=\delta_{nm}.$ (27)

Thus, the decomposition

$\displaystyle|\psi\rangle=\sum_{n=-\infty}^{\infty}c_{n}\,|\varphi_{n}\rangle$
(28)

with Fourier coefficients

$\displaystyle c_{n}=\langle\varphi_{n}|\psi\rangle\in\mathbb{C}$ (29)

satisfies Parseval’s theorem $||\psi||^{2}=\sum_{n}|c_{n}|^{2}$. Accordingly,
the first and second moments of the momentum can be expressed by

$\displaystyle\langle\hat{p}\rangle$ $\displaystyle=$
$\displaystyle\frac{2\pi\hbar}{\Delta x}\,\sum_{n}n|c_{n}|^{2}$ (30)
$\displaystyle\langle\hat{p}^{2}\rangle$ $\displaystyle=$
$\displaystyle\left(\frac{2\pi\hbar}{\Delta
x}\right)^{2}\sum_{n}n^{2}|c_{n}|^{2}$ (31)

such that $\sigma_{p}^{2}=\langle\hat{p}^{2}\rangle-\langle\hat{p}\rangle^{2}$
can be written as

$\displaystyle\sigma_{p}^{2}=\left(\frac{2\pi\hbar}{\Delta
x}\right)^{2}\left[\sum_{n}n^{2}|c_{n}|^{2}-\left(\sum_{n}n|c_{n}|^{2}\right)^{2}\right].$
(32)

The associated variational problem with $||\psi||=1$ and the necessary
condition $\psi(\pm\frac{\Delta x}{2})=0$ is given by

$\displaystyle\sigma_{p}^{2}\to\min$ (33) $\displaystyle\sum_{n}|c_{n}|^{2}=1$
(34) $\displaystyle\sum_{n}(-1)^{n}c_{n}^{*}=0.$ (35)

This can be solved by the definition of a Lagrange function with two
integrating factors $\alpha$ and $\beta$ of the following form

$\displaystyle
L=\sigma_{p}^{2}-\beta\sum_{n}|c_{n}|^{2}-\alpha\sum_{n}(-1)^{n}c_{n}^{*}.$
(36)

The variation of $c_{n}$ and $c_{n}^{*}$ is independent of each other and the
condition $dL=0$ is performed with respect to $c_{n}^{*}$. After a few
algebraic steps the resulting condition is given by

$\displaystyle\left[\left(\frac{2\pi\hbar}{\Delta
x}\right)^{2}n^{2}-\frac{4\pi\hbar}{\Delta x}\langle\hat{p}\rangle
n-\beta\right]c_{n}=(-1)^{n}\alpha$ (37)

for $n=0,\pm 1,\pm 2,...$ This infinite set of conditions for the
determination of $c_{n}$ is non-linear because of the contribution of
$\langle\hat{p}\rangle$ in the second term of the left-hand side. Up to a
constant phase, the solution of this equation is given by

$\displaystyle c_{n}=\frac{\sqrt{8}\,\,}{\pi}\frac{(-1)^{n}}{1-4\,n^{2}}.$
(38)

By substitution it can be verified that the restrictions (34) and (35) are
satisfied. Moreover, the symmetry $c_{n}=c_{-n}$ of the solution implies that
$\langle\hat{p}\rangle=0$. This confirms the assumption made by the authors in
TS09 . The momentum distribution $|c_{n}|^{2}$ of (38) corresponds to the
equal sign in (4) and (7). The associated position representation of the wave-
function is obtained by Fourier transformation (28) and given by

$\displaystyle\psi(x)=\sqrt{\frac{2}{\Delta x}}\cos\left(\frac{\pi x}{\Delta
x}\right)$ (39)

for all $x\in[-\frac{\Delta x}{2},\frac{\Delta x}{2}]$. From the coefficients
in (38) the corresponding continuous function is given by substitution of
$k_{n}=2\pi n/\Delta x$. In this limit the normalized $k$-state function is

$\displaystyle\tilde{\psi}(k)=2\sqrt{\pi\Delta
x}\,\,\frac{\cos\left(\frac{\Delta x}{2}k\right)}{\pi^{2}-\Delta x^{2}k^{2}}$
(40)

for every $k\in\mathbb{R}$. The standard deviation $\sigma_{k}$ of the wave-
numbers $k$ can be obtained by ordinary integration and is given by
$\sigma_{k}=\pi/\Delta x$. After multiplication with $\hbar$ this reproduces
the equal sign of the main relation given in (4). Both (39) and (40) are the
state of minimum uncertainty corresponding to $\sigma_{p}\Delta x=\pi\hbar$.

##  III 3\. Experimental verification

For an experimental verification of the lower bound in inequality (4), a
preparation of the state of minimal uncertainty (39) would be necessary. Since
a cosine-shaped wave function on a finite domain is a superposition of
infinitely many partial waves, it is not obvious how this can experimentally
be realized. However, an approximated state of minimal uncertainty can be
prepared by apodization techniques in terms of a Lanczos-window to cut off
higher spatial frequencies. The corresponding normalized sinc-function is
given by

$\displaystyle\phi(x)=\sqrt{\frac{\pi}{\text{Si}(2\pi)\Delta
x}}\,\,\text{sinc}\left(\frac{2\pi}{\Delta x}\,x\right)$ (41)

with sinc$(x)=\frac{\sin x}{x}$ and Si$(x)=\int_{0}^{\infty}\sin(t)/t\,dt$.
This function can be straightforwardly prepared by plane-wave diffraction, see
Fig. 1.

Figure 1: State (39) of minimum uncertainty (blue) and its Lanczos-window
approximation (orange) given in (41).

The exact value of the corresponding momentum standard deviation $\sigma_{p}$
is given by

$\displaystyle\sigma_{p}\Delta x=\gamma\,\pi\hbar$ (42)

where the constant $\gamma$ is

$\displaystyle\gamma$ $\displaystyle=$
$\displaystyle\frac{2}{\sqrt{3}\,}\,\sqrt{1-\frac{1}{\pi\,\text{Si}(2\pi)}}$
(43) $\displaystyle=$ $\displaystyle 1.0168880...$ (44)

From the experimental point of view this numeric value is sufficiently close
to 1, such that (41) can be considered as an appropriate approximation of the
true state of minimum uncertainty in (39). The corresponding normalized
$k$-state function can be computed by Fourier integration and its density is
given by

$\displaystyle|\tilde{\phi}(k)|^{2}=\frac{\Delta
x}{8\pi^{2}\text{Si}(2\pi)}\left[\text{Si}\left(\frac{\Delta
x}{2}k+\pi\right)-\text{Si}\left(\frac{\Delta x}{2}k-\pi\right)\right]^{2}$

The corresponding diffraction pattern $I(y)$ at the screen of focal distance
$f$ can be obtained in the ordinary Fresnel approach with $y/f=k/k_{0}$, where
$y$ is the position on the screen. With (III), the intensity can be written as

$\displaystyle
I(y)=\frac{k_{0}}{f}\,\left|\tilde{\phi}\left(\frac{k_{0}}{f}y\right)\right|^{2}$
(46)

where $k_{0}=2\pi/\lambda_{0}$.

Figure 2: Measured (blue) versus theoretical (orange) density in semi-
logarithmic representation. The signal-to-noise ratio outside the range of
$y_{c}=\pm 1$ mm is almost zero and therefore of minor relevance for the
computation of expectation values under consideration.

In order to measure this intensity pattern, we use a HeNe monochromatic laser
beam of wavelength $\lambda_{0}=\text{632.82\ n}$m (power: 0.5 mW). The
intensity of the beam is controlled by a polarization filter and subsequently
collimated by the first beam expander seen in Fig. 4. After diffraction of the
resulting plane-wave by the slit 1 of fixed width $\Delta x_{0}$ the sinc-
function amplitude is obtained immediately afterwards the beam has passed the
beam expander 2 at focal distance $f$, where a variable slit 2 is used to
select the correct width $\Delta x$ of the Lanczos-window. Here, a careful
adjustment of the slit width $\Delta x$ is made to ensure that the main peak
is truncated precisely at the first two zeroes. This truncation is the crucial
preparation of the beam (not the measurement). The intensity of the final
diffraction pattern behind the beam expansion 3 is measured by a CCD-Linear-
Sensor (Eureca TCD1304 16 Bit A/D). The measurement range and its resolution
is given by $N=3648$ pixels with pixel-size $\delta y=8\,\mu$m. The
measurement was taken over the entire range of the detector ($N\times\delta
y=29.184$ mm), Fig. 2 shows the relevant domain of the interference pattern.
Due to the extremely fast decay of the momentum density (III), the signal-to-
noise ratio outside the range of $y_{c}=\pm 1$ mm is expected to be almost
zero, such that the tails are only of very limited usefulness for the
determination of the relevant quantities. However, the situation is not as
simple as it seems, because the very fast decrease of the density corresponds
on the other hand with zeros which show up symmetrically near the origin. To
get an impression for the convergence of the relevant quantity, the numerical
expectation value of $\gamma$ was determined step-by-step starting from the
center of the CCD accumulatively according to

$\displaystyle\hat{\gamma}_{n}=\frac{2\Delta
x}{\lambda_{0}f}\,\left[\sum_{i=\frac{N}{2}-n+1}^{\frac{N}{2}+n}\delta
y\,\,y_{i}^{2}\,\hat{I}_{i}\right]^{\frac{1}{2}}$ (47)

for $n=1,2,...,N/2$, and $\hat{I}_{i}$ is the measured (normalized) intensity
contribution given by the data at position $y_{i}$ of the screen. The
convergence of the estimator $\hat{\gamma}_{n}$ is shown in Fig. 3. The slight
overestimation of the theoretical prediction in the range of $0.7$ mm to $2.5$
mm is due to the zeros in the true density, which cannot be adequately
represented by the finite resolvability of the CCD. Above $2.5$ mm, the non-
zero signals are so rare that this overestimation is no longer important.
Therefore, we assume that beyond $2.5$ mm only symmetrical noise plays a role
which should not fundamentally question the results achieved up to that point.
Nevertheless, no statistical significance test seems to be needed to confirm
the hypothesis that $\gamma$ is greater than 1. An extension of the
integration range would only lead to a summation of symmetric Gaussian noise,
which, as expected, would not lead to any additional improvement.

Figure 3: Stochastic convergence of the $\gamma_{n}$-estimator (47) (blue) and
the exact value $\gamma=1.0168$ (green) according to (44). The theoretical
value of the incremental expectation value according to (III) is monotonically
approaching to $\gamma$ (orange).

##  IV 4\. Summary and conclusion

In this work, the importance of carefully choosing observables in
experimentally testing Heisenberg’s uncertainty principle has been emphasized.
It has been discussed that a careless or often pragmatic approximation of
standard deviations can lead to misinterpretations of experimental
measurements and their implications. Since the popular uncertainty relations
are mostly based on the notion of standard deviations, a closer look at their
experimental determination is inevitable.

The exact determination of a standard deviation requires in most cases the
integration over an infinite domain of a density function. In our 4f-setup, an
experimental preparation in terms of a Lanczos-window has been proposed such
that the associated momentum density is sufficiently narrow to allow a
reasonable experimental verification of our main inequality (4).

Actually, this conclusion is not very surprising if one considers that the
rigorous mathematical derivation of inequality (4) in Sec. 2. or in TS09  is
based on the quantum mechanical foundations in Hilbert space and is by no
means only a heuristic argument. Finally, it should be mentioned that the
concept leading to the derivation of inequality (4) has been generalized to
3-dimensional manifolds with constant (nonzero) curvature in TS18 TS20 .

Figure 4: Schematic diagram of the experimental 4f-preparation. The expanded
beam beyond the first lens (plane-wave) is modulated by two slits that single
out the main-peak of a sinc-function by $\Delta x=477\,\mu$m (preparation).
The intensity pattern corresponding to the minimum-uncertainty momentum
density is detected by the CCD (measurement).

## References

  * (1) Heisenberg W., Z. Phys. 43, 172 (1927).
  * (2) Heisenberg W., _The Physical Principles of the Quantum Theory_ , (University of Chicago Press, Chicago, 1930) [Reprinted by Dover, New York (1949, 1967)].
  * (3) Kennard E. H., Z. Phys. 44, 326 (1927).
  * (4) Shull C. G., Phys. Rev. 179, 752 (1969).
  * (5) Leavit J. A., Bills F. A. , Am. J. Phys. 37 (9), 905 (1969).
  * (6) Nairz O., Arndt M. and Zeilinger A., Phys. Rev. A 65, 032109 (2002).
  * (7) Popovicu T., Mathematica 9, 129 (1935).
  * (8) Schürmann T. and Hoffmann I., Found. Phys. 39, 958 (2009).
  * (9) Fernández-Guasti M., Phys. Lett. A 448, 128332 (2022).
  * (10) Uffink J. B. M. and Hilgevoord J., Found. Phys., 15, 925–944, (1985).
  * (11) Landau H.J. and Pollak H.O., Bell Syst. Tech. J. 40, 65-84 (1961).
  * (12) Lenard A., J. Functional Analysis 10, 410–423 (1972).
  * (13) Lahti P., Rep. Math. Phys. 23, 289–297 (1986).
  * (14) Busch P., Heinonen T., and Lahti P., Phys. Rep. 452 (6), 155–176 (2007).
  * (15) Schürmann T., Acta Phys. Pol. B 39, 587-597 (2008).
  * (16) Schürmann T., Found. Phys. 48, 716 (2018).
  * (17) Schürmann T., Eur. Phys. J. C80, 141 (2020).