# Indian Licence Plate Dataset in the wild

Sanchit Tanwar  Email<EMAIL_ADDRESS>Ayush Tiwari  Email:
<EMAIL_ADDRESS>Ritesh Chowdhry  Email<EMAIL_ADDRESS>
###### Abstract

Indian Number (Licence) Plate Detection is a problem that has not been
explored much at an open-source level. There are proprietary solutions
available for it, but there’s no big enough open-source dataset that can be
used to perform experiments and test different approaches. Most of the large
datasets available are for countries like China[1], Brazil[2], but the model
trained on these datasets do not perform well on Indian plates because the
font styles and plate designs used vary significantly from country to country.
This paper introduces an Indian Number (license) plate dataset with 16,192
images and 21,683 plate plates annotated with 4 points for each plate and each
character in the corresponding plate. We propose a two-stage approach in which
the first stage is for localizing the plate, and the second stage is to read
the text in cropped plate image. We present a model that uses semantic
segmentation to solve number plate detection. We tested object detection and
semantic segmentation model; for the second stage, we used lprnet[3] based
OCR. Data, codes and demo are available here:
https://github.com/sanchit2843/Indian_LPR

##  1 Introduction

A critical feature to training a robust deep learning model is to ensure the
dataset includes a wide variety of environments. Our dataset is developed with
images captured in an unconstrained environment with varied camera types and
viewpoints. As a result, a model trained with this dataset will be useful for
real-world applications.

License plates vary from country to country in most cases. There are many
unique number plate formats (as shown in fig 1) throughout the world. Due to
this, a model trained on one country’s license plates does not perform well on
recognizing text of another country’s plates where this format is different.

Figure 1: Different types of Licence Plates throughout the world [4]

Many open-source datasets already exist for Automatic Number Plate Recognition
from China[1] and Brazil[2], but to the best of our knowledge, there is no
sizeable open-source dataset for Indian scenarios. To address the lack of
data, we introduce an open-source dataset with 16192 images captured from road
scenes and annotated with the location and characters of the number plates. We
have described the dataset in detail in section 3 along with some examples,
annotation details, and numbers.

To solve the number plate detection problem, the most common approach is to
use object detection for the detection step, which essentially outputs a
bounding box around a plate in an image. However, there are some problems with
this approach which are discussed in section 4. To mitigate those problems, we
propose our benchmark model for this dataset, in which we have used semantic
segmentation to do the detection step. The rest of the paper is divided into
four sections: Related Work, Dataset, Model, Results, Conclusion, and Future
Work.

##  2 Related Work

This section will briefly review some of the approaches used for solving
automatic number plate recognition systems and other datasets available for
number plate recognition. The problem of number plate recognition is usually
solved in multiple stages, where the first stage is used to locate license
plates and the second stage to recognize the characters of the plate from the
cropped plate images found by stage 1. [5] uses a similar approach in which
yolov2[6] is used for detecting vehicles and number plates, for reading
characters [5] uses two separate networks where one is used for character
segmentation and the second model is used for character recognition, where
character segmentation model gives the bounding box proposals for the
characters, these bounding boxes are used to crop the characters and fed into
character recognition model. [2] uses a similar two-stage approach, for the
second stage, the model uses seven fully connected layers as output, giving
probabilities for the character at each location; such an approach can be used
in settings with fixed-length output, but in Indian settings, the number of
characters in plate varies. Some other methodologies use a single-stage
approach and thus are end to end, [1] proposes a large dataset and a single-
stage approach (RPNet) for solving number plate recognition. RPnet consists of
one module for bounding box detection; these detections are then used to crop
features from the feature maps using ROI Pooling[7], where these features are
used with several classifiers to predict the license plate number. Thus this
approach is unified and end-to-end trainable. Such an approach can also reduce
the inference time as features are shared between detection and character
recognition models.

To the best of our knowledge, all the approaches use object detection based
methods for stage 1, the object detection poses challenges in cases of tilted
number plates, where the cropped plates will have a lot of irrelevant
information. [3] uses spatial transformer network[8] to preprocess the cropped
license plate image which helps in improving the results. One other solution
can be to detect four separate points of plates, and use these 4 points with
perspective transform to finally get a warped image. We have exploited this in
our methodology by using semantic segmentation model for detecting polygon
around plate and using perspective transform to get aligned plate as output,
more details can be seen in section 4.

Most datasets for License Plate Recognition usually collect images from
traffic monitoring systems, highway toll station or parking lots. Caltech[9]
and Zemris[10] collected less than 700 images from high-resolution cameras on
the road or freeways. CCPD[1] provides over 250k unique LP images with
detailed annotations. All images are taken manually by workers of a roadside
parking management company. The resolution of each image is 720 × 1,160. Each
image in CCPD is labelled for license plate bounding box, four vertices
location, horizontal and vertical tilt and other relevant information like the
LP area. But each images in CCPD contains only a single license plate (LP).
UFPR-ALPR[5] dataset contains 150 videos and 4,500 frames captured when both
camera and vehicles are moving and also contains different types of vehicles
(i.e., cars, motorcycles, buses and trucks). Resolution of each image in UFPR-
ALPR is 1,920 × 1,080. Every image has the following annotations available in
a text file: the camera with which the image was taken, the vehicle’s position
and information such as type (car or motorcycle), manufacturer, model and
year; the identification and position of the LP, as well as the position of
its characters.

##  3 Dataset

The dataset contains a total of 16,192 images and 21,683 plates. The left
table in the tables below (table I) shows the number of images of back and
front plates, and the table on the right shows the number of types of vehicles
in the images. There are images from different types of vehicles standard on
Indian roads, which is essential because each vehicle type has a different
plate placement, and in some cases, a different design and format can also be
seen.

Back | 17286
---|---
Front | 4397

Car | 10404
---|---
Bike | 5393
Truck | 3284
3-Wheeler | 1835
Bus | 767

TABLE I: The table on the left shows the number of images of the back and front of vehicles, and the table on the right shows the number of images according to the type of vehicle. Dataset | Year | Images | Resolution | Country
---|---|---|---|---
Caltech Cars | 1999 | 126 | 896 ×592 | American
EnglishLP | 2003 | 509 | 640 ×480 | European
UCSD-Stills | 2005 | 291 | 640 ×480 | American
ChineseLP | 2012 | 411 | Various | Chinese
AOLP | 2013 | 2,049 | Various | Taiwanese
OpenALPR-EU | 2016 | 108 | Various | European
SSIG-SegPlate | 2016 | 2,000 | 1920 × 1080 | Brazilian
UFPR-ALPR | 2018 | 4,500 | 1920 × 1080 | Brazilian
CCPD | 2018 | 250k |  | Chinese
Indian-LPR | 2021 | 16k | 1920 x 1080 | Indian
TABLE II: This table shows the number of images in a few of the popular open
source datasets in different countries along with the resolution of the images
in the datasets (the last row is of our dataset).

There are images from a total of 10 states, they are filtered out from videos
captured with different settings, devices, orientations and field of view.
Some of the videos were captured from a dash cam placed inside a moving
vehicle, some were captured with a person holding a camera on the road side,
with the camera being static. Due to these differences in the video capturing,
an unconstrained environment is created, meaning that the dataset has a huge
variety in terms of type of camera being used, position of camera, viewpoint
and thus it can be used in real-world applications. In the case of Indian
Licence Plates there are many different non-standard formats that are widely
used throughout India (refer fig 2) which creates issues with the character
recognition, leading to inaccurate plate recognition. Although there’s no way
to cover all these images in a dataset, our dataset includes many images of
this kind that are from vehicles on road in early 2021.

Figure 2: Some non-standard plate formats found in India

###  3.1 Annotation

The number plates observed by a camera on the road can be at a certain angle
and orientation; in the case of a roadside camera, they are always at a
certain angle, this leads to problems in identifying the plate in an image and
performing OCR also becomes challenging. To tackle this problem and improve
the robustness of OCR in the two-stage models, we provide annotation with 4
points around number plates (refer to figure 3) which provides orientation
information as well, and thus we can use techniques like semantic
segmentation, oriented bounding box detection for number plate detection. We
can use the orientation information provided by the 4 points to straighten the
number plate, eliminating any orientation issues for the model.

Figure 3: The benefit of using 4 point annotation, the upper image is the
original image of the car, the bottom left is the 2 point annotation and the
bottom right is the 4 point annotation

For each annotated plate we also provide information of the vehicle to which
the plate belongs to and position of plate on vehicle (back or front), we also
provide the characters of each plate for the character recognition part. The
number plates which are completely readable are annotated, the unreadable or
partially readable plates are not annotated.

##  4 Model

We have created a baseline model with 2 stage approach, the first stage being
the detection of number plates in an image, followed by a second stage to
recognize characters in cropped plates. We tested two different models for the
detection stage, first using object detection that has been well explored for
the task of number plate detection in the past and then, a semantic
segmentation-based model for number plate detection. Although, semantic
segmentation is not a well-suited alternative for many object detection
problems, because it does not work on the instance level but, for this problem
it performs equally well since the individual instances (plates in this case)
are well separated from one another and thus can be individually identified
using simple post-processing steps. Although, the primary benefit of object
detection over semantic segmentation is annotation time, but since for this
task to annotate semantic segmentation polygons only 4 points are required
(unlike the usual semantic masks which requires multiple vertices in the
polygon), it is not much overhead over the basic rectangle-based object
detection annotations. We crop the plates detected by any detection method and
recognize them using LPRNet[3] trained on our dataset. These models are
individually described below.

Although we have provided the dataset with a division of the plates in 10
classes in total (car back, car front, bike around, bike front, and more), in
this work, we trained the first stage model with a single class for the sake
of simplicity.

###  4.1 Object Detection

We used a FCOS[11] with Hrnet[12] backbone. FCOS is a single-stage object
detection approach and thus works well in real-time applications such as ours.
We used FCOS as the baseline model because FCOS is an anchor-free approach and
requires minimal hyper-parameter tuning. The model was trained for 50 epochs
with a batch size of 8 using ranger[13] optimizer. We used pixel-level
augmentation techniques with random changes in brightness and contrast of the
image. No spatial data augmentation was used in the baseline experimentation.
The model was trained with a fixed resolution of 1920*1080 as downsampling the
image can make the number plates unreadable.

###  4.2 Semantic segmentation

We used similar Hrnet[12] backbone for semantic segmentation model. This
backbone is followed by a convolution layer with output channels equal to the
number of classes, i.e., 2 (the background and number plate). We used cross-
entropy loss for training, with class weightage for background equal to 0.05
and 0.95 for number plate class because of the high level of class imbalance.
We used Ranger[13] optimizer with 0.01 as the initial learning rate, and the
learning rate was decayed using polynomial learning rate decay with degree
0.9. We trained the model for 50 epochs with a batch size of 8. Like object
detection, only pixel-level augmentation techniques were applied, which
randomly changed the image’s brightness and contrast. Image resolution for
semantic segmentation was fixed at 1920*1080, and no cropping or downsampling
was performed.

The output of semantic segmentation is polygonized. The polygon is then
converted into a 4 point box; we use these 4 points to warp the plate as shown
in Figure 3.

###  4.3 LPRNET

We used LPRNet[3] for character recognition because it is a lightweight
Convolutional Neural Network with high accuracy. We used LPRNet basic, which
is based on Inception blocks [14] followed by spatial convolutions and trained
with CTC loss. The model was trained for 100 epochs with Adam optimizer using
a batch size of 32 with an initial learning rate of 0.001 and a gradient noise
scale of 0.001. Data augmentation used were random affine
transformations,e.g., rotation, scaling, and shift.

###  4.4 Advantages of using Semantic segmentation

Semantic segmentation has multiple advantages over object detection. The mask
generated by semantic segmentation can be post-processed to generate 4 points
for the plate, which has advantages over the 2 point method. It helped us crop
out the plates irrespective of their orientation in the image, making the
model usable for cases like roadside cameras.

###  4.5 Failure mode of semantic segmentation

The semantic segmentation-based model also has a few limitations in this
particular application; we found them frequently occurring during our testing.
The examples of these failure modes are shown in figure 4.

Figure 4: In the images, the green bounding box is ground truth; blue is
object detection model output; red is semantic segmentation model output. The
left column images show the incomplete bounding box failure mode of semantic
segmentation. The middle column images show the bounding box split into
multiple boxes failure modes. The third column images show the blurred plates
being detected.

Here is a brief description of the failure modes which are shown in fig 4
above.

  1. 1.

The semantic segmentation model detected an incomplete bounding box small in
size than the original plate and thus missing few characters, as shown in the
two images in the first column.

  2. 2.

The middle column images show examples in which a single plate is split into
multiple smaller blobs, which usually happens with the plates in a 2-row
format; this problem can be solved using convex hull-based post-processing or
using dice loss instead of cross entropy might improve results in such
scenarios.

  3. 3.

The third column images show the blurred plates which are detected by the
semantic segmentation model but are not detected by the object detection
model.

##  5 Results

In the tables below, we present the results of the model. We tested on the
test split of the dataset, which has 3400 images; the fps was evaluated on an
Nvidia 2070 super GPU at an image resolution of 1920*1080. The second table in
table III shows different cropping mechanisms that were used for the stage 2
model, where warp means taking perspective transform of the image using 4
points received from the semantic segmentation model, and thus the warped
image will have pixels inside the 4 points of the original image. The
rectangular crop means the crop with the rectangular bounding box received
from the object detection model, this crop will have some extra information in
the cropped image because of tilt that occurs naturally in the real world
images.

Model | AP50 | FPS
---|---|---
Object Detection | 0.8310 | 12
Semantic segmentation | 0.8313 | 14

Cropping Method | Character Accuracy
---|---
Warp | 75.6
Rectangular Crop | 66.3

TABLE III: Table on the left are the results of stage 1 model on test split,
table on the right shows the character accuracy of LPRnet with different
cropping methods.

##  6 Conclusion

To conclude, semantic segmentation model performs faster in comparison to
object detection model and still achieves similar mAP. The advantage of using
semantic segmentation model can be clearly observed in the results of stage
two, where warping shows clear advantage of semantic segmentation based
approach. Nevertheless, more work can be done on both types of model to
achieve better results. The goal of this paper is to introduce a non-existing
open source dataset for Indian licence plates, with the intention of promoting
more experimentation.

##  7 Future Work

A semantic segmentation model can be tuned to improve the results; further,
dice loss can be used to improve the results, especially in this problem
because of the ability of dice loss to inherently handle class imbalance and
being more susceptible to shapes. The character accuracy of the model in this
work is 75% which is not suitable for real-world scenarios; the primary reason
for low accuracy is two-line plates which occur a lot in a real-world system.
LPRnet fails to decode characters for two-row plates. A straightforward
solution is to annotate the characters with rectangles around each character
and using an object detection-based model in stage 2. But this requires a
significant amount of annotation work, and thus there is a need for an OCR
model that works well in two-row scenarios.

## References

  * [1] Z. Xu, W. Yang, A. Meng, N. Lu, and H. Huang, “Towards end-to-end license plate detection and recognition: A large dataset and baseline,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 255–271.
  * [2] G. R. Gonçalves, M. A. Diniz, R. Laroca, D. Menotti, and W. R. Schwartz, “Real-time automatic license plate recognition through deep multi-task networks,” in _2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI)_. IEEE, 2018, pp. 110–117.
  * [3] S. Zherzdev and A. Gruzdev, “Lprnet: License plate recognition via deep neural networks,” _arXiv preprint arXiv:1806.10447_ , 2018.
  * [4] C. Henry, S. Ahn, and S.-W. Lee, “Multinational license plate recognition using generalized character sequence detection,” _IEEE Access_ , vol. PP, pp. 1–1, 02 2020.
  * [5] R. Laroca, E. Severo, L. A. Zanlorensi, L. S. Oliveira, G. R. Gonçalves, W. R. Schwartz, and D. Menotti, “A robust real-time automatic license plate recognition based on the YOLO detector,” in _International Joint Conference on Neural Networks (IJCNN)_ , July 2018, pp. 1–10.
  * [6] J. Redmon and A. Farhadi, “Yolo9000: better, faster, stronger,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 7263–7271.
  * [7] R. Girshick, “Fast r-cnn,” in _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 1440–1448.
  * [8] M. Jaderberg, K. Simonyan, A. Zisserman _et al._ , “Spatial transformer networks,” _Advances in neural information processing systems_ , vol. 28, pp. 2017–2025, 2015.
  * [9] C.-N. E. Anagnostopoulos, I. E. Anagnostopoulos, I. D. Psoroulas, V. Loumos, and E. Kayafas, “License plate recognition from still images and video sequences: A survey,” _IEEE Transactions on intelligent transportation systems_ , vol. 9, no. 3, pp. 377–391, 2008.
  * [10] G.-S. Hsu, J.-C. Chen, and Y.-Z. Chung, “Application-oriented license plate recognition,” _IEEE transactions on vehicular technology_ , vol. 62, no. 2, pp. 552–561, 2012.
  * [11] Z. Tian, C. Shen, H. Chen, and T. He, “Fcos: Fully convolutional one-stage object detection,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 9627–9636.
  * [12] J. Wang, K. Sun, T. Cheng, B. Jiang, C. Deng, Y. Zhao, D. Liu, Y. Mu, M. Tan, X. Wang _et al._ , “Deep high-resolution representation learning for visual recognition,” _IEEE transactions on pattern analysis and machine intelligence_ , 2020.
  * [13] L. Wright, “Ranger - a synergistic optimizer.” https://github.com/lessw2020/Ranger-Deep-Learning-Optimizer, 2019.
  * [14] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 2818–2826.


]https://aovgun.weebly.com/

# Weak deflection angle by Casimir wormhole using Gauss-bonnet theorem and its
shadow

Wajiha Javed<EMAIL_ADDRESS><EMAIL_ADDRESS>Division of
Science and Technology, University of Education, Township-Lahore, Pakistan
Ali Hamza<EMAIL_ADDRESS>Division of Science and Technology,
University of Education, Township-Lahore, Pakistan  Ali Övgün
<EMAIL_ADDRESS>[ Physics Department, Eastern Mediterranean University,
Famagusta, North Cyprus via Mersin 10, Turkey.

###### Abstract

In this paper, we calculate the weak deflection angle by Casimir wormhole and
its shadow. To do so, we derive the Gaussian optical curvature and use the
Gauss-Bonnet theorem. Then we find the deflection angle by Casimir wormhole in
weak field limits. Moreover, we obtain the weak deflection angle in the
presence of plasma medium and see the effect of the plasma medium on the weak
deflection angle. Moreover, we study a shadow of Casimir wormhole and we plot
and discuss them. We show the shadow of Casimir wormhole’s behaviour when
changing the value of $a$.

Relativity and gravitation; Wormhole; Gravitational lensing; Weak deflection
angle; Gauss-Bonnet theorem; plasma medium; Shadow

###### pacs:

95.30.Sf, 98.62.Sb, 97.60.Lf

##  I Introduction

Idea of wormhole was first suggested by Einstein and Rosen in 1935. This is
also known as Einstein-Rosen bridges A1 1 . A wormhole is a shortcut of the
spacetime which makes alternate routes between two particular spacetimes. The
idea of ”wormhole” was developed firstly in 1957 inside the seminal papers of
Misner and Wheeler 4 1  and Wheeler 4 2 . Afterward, Wheeler demonstrated that
wormholes would be unsteady and non-traversable for even a photon A1 2 . The
metric of Ellis wormholes was first discussed in MPLA1  and then investigation
of Ellis wormholes in General Relativity (GR) went back to the basic works of
Morris and Thorne in 1988 4 3  where they presented a traversable wormhole
geometry. Furthermore, some scientists briefly have examined the gravitational
lensing of Ellis wormhole in (MPLA2 -MPLA9 ). Then the weak and strong
gravitational lensing of compact objects are studied in (MPLA10 -MPLA26 ) and
the properties of celestial bodies during gravitation lensing and also test
theory of gravity are studied in (MPLA27 -MPLA36 ). Later on motivated by the
Morris-Thorne paper, numerous scientists study wormhole spacetimes (A2 4 -RH4
). Then, Matt Visser proposed a thinshell wormhole in a 1989 A1 4 . Wormholes
are also using to explain quantum entanglement A1 5 . Exotic matter causes
trouble for creating stable wormholes. The Casimir effect which is found by
Casimir shows that there is a finite energy between the plates and also
calculate the finite force between the plates in his famous paper 1 . Casimir
energy is indicated as a potential source and it is also used to prove the
existence of negative energy which can be built in the laboratory CW . Casimir
wormhole is a type of traversable wormhole that can be generate by using
Casimir energy.
In this paper, we calculate the deflection angle of light (DAL) from Casimir
wormhole in weak field limits (WFL). In 2008, Gibbons and Werner (GW)
suggested a technique to calculate the DAL by different black holes in the WFL
R 35 . For calculating deflection angle, first we calculate the Gaussian
optical metric using optical geometry of the Casimir wormhole and then we
apply the Gauss-Bonnet theorem (GBT) R 35 :

$\alpha=-\int\int_{D_{\infty}}\mathcal{K}dS.$ (1)

Different authors use this technique to calculate DAL by various black holes
and wormholes spacetime (R 36 -R 45 ). Crisnejo and Gallo analyzed the DAL in
the presence of plasma medium M1 19 . Some other works can be found in Refs
Ono:2017pie ; Jusufi:2017hed ; Ishihara:2016sfv ; Arakida:2017hrm ;
Ovgun:2018tua ; Ovgun:2018fte ; Ovgun:2018prw ; Jusufi:2018jof ; Ovgun:2018ran
; Jusufi:2017uhh ; Ovgun:2018fnk ; Ono:2018ybw ; Ono:2018jrv ;
Crisnejo:2018ppm ; aoli ; Li:2019mqw ; Li:2019qyb ; Li:2019vhp ; Ovgun:2018oxk
; Javed:2019qyg ; 19 ; Javed:2019kon ; Ovgun:2019wej ; Javed:2019rrg ;
Jusufi:2017vew ; Crisnejo:2019xtp ; Javed:2019ynm ; deLeon:2019qnp ;
Kumaran:2019qqp ; km2 ; Ovgun:2019qzc ; km3 ; Ahmedov:2019dja ;
Turimov:2018ttf ; Abdujabbarov:2017pfw ; Schee:2017hof .

It is generally accepted that majority of the galaxies have super-massive
black holes at their centers S1 43 ; S1 44 , for example Milky Way and Messier
87 having super-massive black hole named as Sgr A and M87. The Event Horizon
Telescope imaged the shadow of M87* with measured diameter of 42
microarcsecond izr4 . Recently, attempts have been made to directly visualize
the shadow of Sagittarius A* izr4 , the supermassive black hole in the
Galactic Center, and likely observational results will soon be revealed.
Shadow of Schwarzschild black hole is first studied by Synge S1 45  and it is
additionally investigated by Luminet S1 46 . Bardeen S1 47 ; S1 48  analyzed
shadow cast of Kerr black hole and more work was made by Falcke S1 49 . Some
other works can be found in Refs Kumar:2020hgm ; Konoplya:2019xmn ;
Kumar:2019pjp ; Konoplya:2019goy ; Allahyari:2019jqz ; Amir:2018pcu ; Kho1 ;
Vag1 ; Vag2 ; Konoplya:2019sns ; Younsi:2016azx ; Cunha:2019hzj ;
Ovgun:2020gjz .
Main purpose of the paper is to calculate the deflection angle of Casimir
wormhole in non-plasma and plasma medium and find shadow cast by Casimir
wormhole. The paper is organized as follows: In Sect. II, we calculate the
Gaussian optical metric of Casimir wormhole and use the GBT method to
calculate its deflection angle in Sect. III. In Sect. IV, we observe the
graphical behavior of deflection angle in non-plasma. In Sect. V, we calculate
the DAL by Casimir wormhole in plasma medium. In Sect. VI, we analyzed the
plots of Casimir wormhole in plasma medium, next we find the null geodesic in
Sect. VII and in Sect. VIII we find the shadow of Casimir wormhole and finally
in Sect. IX, we conclude our result.

##  II Optical metric of casimir wormhole

The metric of a Casimir traversable wormhole in a spherically symmetric
spacetime is define as CW ;

$ds^{2}=-A(r)dt^{2}+\frac{dr^{2}}{B(r)}+r^{2}d\Omega^{2}.$ (2)

Where $A(r)$ and $B(r)$ are defined as

$A(r)=\left({\frac{3r}{3r+a}}\right)^{2}~{},~{}B(r)={1-\frac{2a}{3r}-\frac{{a}^{2}}{3{r^{2}}}}~{}and~{}d\Omega^{2}=d\theta^{2}+\sin^{2}\theta
d\phi^{2}.\\\ $ (3)

Here $a$ is a constant and $r$ is a radial coordinate such that
$r\in[a,\infty]$. After expanding $A(r)$ as a series and take only second
order of $a$, we can write A(r) as :

$A(r)=1-\frac{2a}{3r}+\frac{{a}^{2}}{3{r^{2}}}+\mathcal{O}(\frac{a^{3}}{r^{3}}).\\\
$ (4)

We can assume $(\theta=\frac{\pi}{2})$, and find the optical metric by
$ds^{2}=0$,

$dt^{2}=\frac{dr^{2}}{A(r)B(r)}+\frac{r^{2}d\phi^{2}}{A(r)}.$ (5)

Now by using Eq. (5), the non-zero christofell symbols are defined as

$\Gamma^{0}_{00}=-\frac{B^{\prime}(r)}{2B(r)}-\frac{A^{\prime}(r)}{2A(r)}~{}~{}~{}~{}~{}~{}~{}~{},~{}~{}~{}~{}~{}~{}~{}\Gamma^{1}_{01}=\frac{1}{r}-\frac{A^{\prime}(r)}{2A(r)}$
(6)

and

$\Gamma^{0}_{11}=-rB(r)+\frac{r^{2}A^{\prime}(r)B(r)}{2A(r)},$ (7)

and Ricci scalar related to the optical metric is calculated as:

$\displaystyle\mathcal{R}$ $\displaystyle=$
$\displaystyle-\frac{A(r)B^{\prime}(r)}{r}+\frac{A^{\prime}(r)B^{\prime}(r)}{2}+\frac{A^{\prime}(r)B(r)}{r}+{A^{\prime\prime}(r)}{B(r)}$
(8) $\displaystyle-$ $\displaystyle\frac{(A^{\prime}(r))^{2}B(r)}{2A(r)}.$

The Gaussian optical curvature that is calculated as follows:

$\mathcal{K}=\frac{RicciScalar}{2}.$ (9)

After simplifying, Gaussian optical curvature is given as:

$\mathcal{K}\approx-{\frac{2a}{{3r}^{3}}}+\,{\frac{2{a}^{2}}{3{r}^{4}}}+\mathcal{O}(\frac{a^{3}}{r^{3}}).$
(10)

##  III Deflection angle of Casimir wormhole

Now by using the GBT we can obtain the DAL of Casimir wormhole. We use the GBT
to the area of $\mathcal{E}_{S}$, given as R 35

$\int\int_{\mathcal{E}_{S}}\mathcal{K}dS+\oint_{\partial\mathcal{E}_{S}}kdt+\sum_{j}\epsilon_{j}=2\pi\mathcal{Y}(\mathcal{E}_{S}),$
(11)

and Gaussian curvature is indicated by $\mathcal{K}$ and geodesic curvature is
indicated by $k$, declared as
$k=\bar{g}(\nabla_{\dot{\alpha}}\dot{\alpha},\ddot{\alpha})$ in this way as
$\bar{g}(\dot{\alpha},\dot{\alpha})=1$, $\ddot{\alpha}$ represents unit
acceleration vector and the $\epsilon_{j}$ represents the exterior angle at
the jth vertex. As $S\rightarrow\infty$, both the jump angles reduce to
$\pi/2$ and we get $\theta_{O}+\theta_{S}\rightarrow\pi$. The Euler
characteristic is $\mathcal{Y}(\mathcal{E}_{S})=1$, as $\mathcal{E}_{S}$ is
non singular. So,

$\int\int_{\mathcal{E}_{S}}\mathcal{K}dS+\oint_{\partial\mathcal{E}_{S}}kdt+\epsilon_{j}=2\pi\mathcal{Y}(\mathcal{E}_{S}),$
(12)

here, $\epsilon_{j}=\pi$ shows that both $\alpha_{\bar{g}}$ and the total jump
angle is a geodesic, since the Euler characteristic number expressed by
$\mathcal{Y}$ is $1$. As $S\rightarrow\infty$, the only interesting part to be
determined is $k(D_{S})=\mid\nabla_{\dot{D}_{S}}\dot{D}_{S}\mid$. Since,
radial component of geodesic curvature is defined as R 35

$(\nabla_{\dot{D}_{S}}\dot{D}_{S})^{r}=\dot{D}^{\phi}_{S}\partial_{\phi}\dot{D}^{r}_{S}+\Gamma^{0}_{11}(\dot{D}^{\phi}_{S})^{2}.$
(13)

For large $S$, $D_{S}:=r(\phi)=S=const$. Hence, the form of the Eq. (13)
becomes $(\dot{D}^{\phi}_{S})^{2}=\frac{A^{2}(r)B(r)}{r^{2}}$. As
$\Gamma^{0}_{11}=-rB(r)+\frac{r^{2}A^{\prime}(r)B(r)}{2A(r)}$, so it becomes

$(\nabla_{\dot{D}^{r}_{S}}\dot{D}^{r}_{S})^{r}\rightarrow\frac{1}{S}.$ (14)

As topological defect is not present in geodesic curvature so,
$k(D_{S})\rightarrow S^{-1}$. But with the help of optical metric Eq. (10), we
can write as $dt=Sd\phi$. Hence, we get this;

$k(D_{S})dt=d\phi.$ (15)

With the help of all above results, we obtained

$\int\int_{\mathcal{E}_{S}}\mathcal{K}ds+\oint_{\partial\mathcal{E}_{S}}kdt=^{S\rightarrow\infty}\int\int_{T_{\infty}}\mathcal{K}dS+\int^{\pi+\Theta}_{0}d\phi.$
(16)

Light ray at 0th order in week field limit is defined as $r=b/\sin\phi$, where
’b’ is known as impact parameter which is perpendicular distance between path
of light and wormhole). Now, using (11) and (17), the DAL defined as R 35

$\Theta=-\int^{\pi}_{0}\int^{\infty}_{b/\sin\phi}\mathcal{K}\sqrt{det\bar{g}}~{}drd\phi.$
(17)

After putting the leading order terms of Gaussian curvature Eq. (10) into Eq.
(17), the deflection angle is calculated as follows:

$\displaystyle\Theta$ $\displaystyle\thickapprox$
$\displaystyle\,{\frac{4a}{3b}}-\,{\frac{\pi\,{a}^{2}}{6{b}^{2}}}+\mathcal{O}(\frac{a^{3}}{r^{3}}).$
(18)

Note that for $a=3M$, first order of term of weak deflection angle of Casimir
wormhole reduces to the deflection angle of Schwarzschild black hole
($\frac{4M}{b}$).

##  IV Graphical Analysis for non-plasma medium

Here, in this section we discuss the graphical behavior of DAL. We observe the
relation of deflection angle $\Theta$ with impact parameter $b$ with the
variation of $a$. Both ’a’ and ’b’ have the mass dimension of length so ’L’ is
the dimension of ’a’ and ’b’. In the left graph we change the value of $a$
from $1L$ to $5L$ and in right graph, we vary the values of a from $22L$ to
$30L$ by fixing the value of $b$ from $1L$ to $20L$.

###  IV.1 Deflection angle $\Theta$ w.r.t Impact parameter $b$

Figure 1: Relation between $\Theta$ and $b$.

  * •

Figure 1 display the relation of $\Theta$ w.r.t $b$ for different values of
$a$.

    1. 1.

In left plot, we see that by increasing the values of $a$ the deflection angle
first increase toward positive and then gradually decrease from positive to
negative that indicates the balanced behavior.

    2. 2.

In right figure, we observe that for choosing large values of $a$ deflection
angle is continuously decreasing from positive to negative.

##  V Effect of plasma on gravitational lensing

This section is based on the investigation of DAL by Casimir wormhole in the
presence of plasma medium. For Casimir wormhole the refractive index $n(r)$ M1
19 , is obtain as,

$n^{2}\left[r,\omega(r)\right]=1-\frac{\omega_{e}^{2}(r)}{\omega_{\infty}^{2}(r)}.$
(19)

Refractive index is defined as;

$n(r)=\sqrt{{1-\frac{\omega_{e}^{2}}{\omega_{\infty}^{2}}\left(A(r)\right)}}.$
(20)

Where $\omega_{e}$ is electron plasma frequency and $\omega_{\infty}$ is
photon frequency evaluated by an observer at infinity and so the corresponding
optical metric defined as M1 19

$dt^{2}=g^{opt}_{lm}dx^{l}dx^{m}=n^{2}\left[\frac{dr^{2}}{A(r)B(r)}+\frac{r^{2}d\phi^{2}}{A(r)}\right].$
(21)

The Gaussian curvature in terms of curvature tensor can be determined as

$\mathcal{K}=\frac{R_{r\phi r\phi}(g^{opt})}{det(g^{opt})},$ (22)

with the help of Eq. (22) Gaussian curvature is written as

$\displaystyle\mathcal{K}$ $\displaystyle\approx$
$\displaystyle-{\frac{{\omega_{e}}^{2}a}{{\omega_{\infty}}^{2}{r}^{3}}}-2/3\,{\frac{a}{{r}^{3}}}+7/3\,{\frac{{a}^{2}{\omega_{e}}^{2}}{{r}^{4}{\omega_{\infty}}^{2}}}+2/3\,{\frac{{a}^{2}}{{r}^{4}}}.$
(23)

By using GBT, we will calculate the DAL in order to compare it with non-plasma
case. In this way, for finding angle in the weak field limits, as the light
follows a straight line so we utilize a case of $r=\frac{b}{sin\phi}$ at 0th
order.

$\Theta=-\lim_{R\rightarrow
0}\int_{0}^{\pi}\int_{\frac{b}{\sin\phi}}^{R}\mathcal{K}dS.$ (24)

With the help of Eq. (16), the DAL in plasma medium is found as follows:

$\displaystyle\Theta$ $\displaystyle\thickapprox$
$\displaystyle\,{\frac{4a}{3b}}-\,{\frac{\pi\,{a}^{2}}{6{b}^{2}}}-{\frac{7\,\pi\,{a}^{2}{\omega_{e}}^{2}}{12\,{b}^{2}{\omega_{\infty}}^{2}}}+2\,{\frac{{\omega_{e}}^{2}a}{b{\omega_{\infty}}^{2}}}$
(25)

The above results tells us that photon rays are moving into medium of
homogeneous plasma. We see that Eq. $(\ref{P2})$ reduced into Eq. $(\ref{P1})$
if plasma effect can be removed.

##  VI Graphical Analysis for plasma medium

In this section, we discuss the effects of plasma medium on the deflection
angle of Casimir wormhole by plotting them. We take
$\frac{\omega_{e}}{\omega_{\infty}}$=$10^{-1}$ and observe behavior of DAL by
changing different values of $a$. Here, we also pick the value of $a$ from
$1L$ to $5L$ in the left graph and in the right graph, we varry the value of
$a$ from $20L$ to $60L$ by fixing the value of $b$ from $1L$ to $20L$.

###  VI.1 Deflection angle w.r.t Impact parameter

Figure 2: Relation between $\Theta$ and $b$.

  * •

Figure 2 demonstrates the relation of $\Theta$ with $b$ for different values
of $a$.

    1. 1.

In 1st plot we show that for small values of $a$ the DAL first increase from
negative to positive and then gradually decrease toward negative as like in
Figure 1.

    2. 2.

In 2nd plot we observe that DAL gradually decrease for large values of $a$ as
like in Figure 1.

##  VII Null Geodesic in a Casimir wormhole

The Lagrangian of the Casimir wormhole Eq. (2) is given by

$2\mathcal{L}=-\left(1-\frac{2a}{3r}-\frac{a^{2}}{3{r^{2}}}\right)\dot{t}^{2}+\left(\frac{1}{1-\frac{2a}{3r}-\frac{a^{2}}{3{r^{2}}}}\right)\dot{r}^{2}+r^{2}\dot{\theta}^{2}+r^{2}\sin^{2}\theta\dot{\phi}^{2},$
(26)

where an overdot means derivative w.r.t the affine parameter $\lambda$. As the
Lagrangian is independent of $\phi$ and t so there are only two constants of
motion named as the energy $E$ and the angular momentum $L$. Where $E$ and $L$
are defined as:

$p_{t}=\frac{\partial
L}{\partial\dot{t}}=-(1-\frac{2a}{3r}-\frac{a^{2}}{3{r^{2}}})\dot{t}=-E$ (27)

and

$p_{\phi}=\frac{\partial
L}{\partial\dot{\phi}}=r^{2}\sin^{2}\theta\dot{\phi}=L.$ (28)

Now we can determine the geodesic conditions by utilizing these preserved
measures

$\frac{dt}{d\lambda}=\dot{t}=\frac{E}{1-\frac{2a}{3r}-\frac{a^{2}}{3{r^{2}}}}~{}~{}~{},~{}~{}~{}\frac{d\phi}{d\lambda}=\dot{\phi}=\frac{L}{r^{2}\sin^{2}\theta}.$
(29)

The $r$-part and $\theta$-part of the momentum are defined as

$p_{r}=\frac{\partial
L}{\partial\dot{r}}=\frac{\dot{r}}{1-\frac{2a}{3r}-\frac{a^{2}}{3r^{2}}}~{}~{}and~{}~{}p_{\theta}=\frac{\partial
L}{\partial\dot{\phi}}=r^{2}\dot{\theta}.$ (30)

The $r$-part and $\theta$-part of the geodesic equations can be calculated by
working on the Hamilton-Jacobi equation

$\frac{\partial S}{\partial\lambda}=-\frac{1}{2}g^{\mu\nu}\frac{\partial
S}{\partial x^{\mu}}\frac{\partial S}{\partial x^{\nu}},$ (31)

and for photons $(m_{0}=0)$, Eq. (31) is used by the ansatz

$S=-Et+L\phi+S_{r}(r)+S_{\theta}(\theta),$ (32)

here $S_{r}$ is a function or $r$ and $S_{\theta}$ is the function of
$\theta$. By putting Eq. (32) into Eq. (31) and also putting the value of
contravariant metric, i.e., $g^{\mu\nu}$, and differentiate the terms of
variables $r$ and $\theta$ equal to the Carter constant $(\pm\mathcal{K})$
SA92 , we get

$\displaystyle\frac{1}{\sqrt{1-\frac{2a}{3r}-\frac{a^{2}}{3r^{2}}}}\frac{dr}{d\lambda}=^{+}_{-}\sqrt{R}(r),$
$\displaystyle r^{2}\frac{d\theta}{d\lambda}=^{+}_{-}\sqrt{T}(\theta),$ (33)

here $R$ and $\theta$ can be written as

$\displaystyle
R(r)=\frac{E^{2}}{1-\frac{2a}{3r}-\frac{a^{2}}{3{r^{2}}}}-\frac{\mathcal{K}}{r^{2}},$
$\displaystyle T(\theta)=\mathcal{K}-\frac{L^{2}}{\sin^{2}\theta}.$ (34)

As we have geodesic equations now our main concern is to find the radial
motion of photons across the throat of Casimir wormhole, that can be showed by
the computation of the effective potential

$(\frac{dr}{d\lambda})^{2}+V_{eff}=0,$ (35)

with

$V_{eff}=-\left({1-\frac{2a}{3r}-\frac{a^{2}}{3r^{2}}}\right)R(r).$ (36)

From the expression we can say that effective potential depend on constant
$a$, radial coordinate $r$ and on energy $E$. It is advantageous to decrease
the quantity of parameters by characterizing the impact parameters such as
$\xi=\frac{L}{E}$ and $\eta=\frac{\mathcal{K}}{E^{2}}$. Photon geodesics can
be communicated as far as these impact parameters ($\xi$,$\eta$ ). Now we
write $\mathcal{R}$, in the form of impact parameters as pursues

$R=E^{2}[\frac{1}{1-\frac{2a}{3r}-\frac{a^{2}}{3{r^{2}}}}-\frac{\eta}{r^{2}}].$
(37)

The investigation of effective potential expose the presence of un-stable
circular orbits of fixed radius about the throat of wormholes. Those circles
are significant from the perspective of optical perceptions.

##  VIII Shadow of Casimir Wormhole

Here, we explore the shadow cast of Casimir wormhole. First we write
conditions to check unstable circular photon orbits:

$R=0,~{}~{}~{}~{}~{}~{}~{}~{}R^{\prime}=0.$ (38)

Here prime $(\prime)$ denote derivative w.r.t $r$. Putting (37) into (38), we
can easily get

$\eta=\frac{r^{2}}{1-\frac{2a}{3r}-\frac{a^{2}}{3{r^{2}}}},~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}r=\frac{3a^{+}_{-}\sqrt{33a^{2}}}{6}$
(39)

Here, impact parameter $\eta$ depend on radial coordinate $r$ and on constant
$a$. Eq. (39) defines the shadow’s boundary. Therefore, we make the celestial
coordinates ($\alpha$,$\beta$) and associate them with impact parameters
($\xi$,$\eta$). The apparent shape of a shadow are found by using the
celestial coordinates (SA47 ,SA48 ) as

$\displaystyle\alpha=^{\lim}_{r_{0}\rightarrow\infty}(r_{0}^{2}\sin\theta_{0})\frac{d\phi}{dr},$
$\displaystyle\beta=^{\lim}_{r_{0}\rightarrow\infty}~{}r_{0}^{2}\frac{d\theta}{dr},$
(40)

here distance between wormhole and observer is denoted by $r_{0}$ and
$\theta_{0}$ is observer’s angular coordinate or we can say it ”inclination
angle”. Now putting the equations of four-velocities into Eq. (40), and doing
some simple direct calculations we get this type of celestial coordinates

$\alpha=-\frac{\xi}{\sin\theta_{0}}~{}~{}~{}~{}~{}~{}~{}and~{}~{}~{}~{}~{}~{}~{}\beta=\sqrt{\eta-\frac{\xi^{2}}{\sin\theta_{0}^{2}}}.$
(41)

Knowing the equations of celestial coordinates and impact parameters, we now
make the shadow of Casimir wormhole. So as to construct the shape of shadow,
we plot $\alpha$ versus $\beta$ which gives the shadow’s boundary. The plots
of the shadow for the Casimir wormholes can be saw from Fig 3.

Figure 3: Graphs displaying the shadow of Casimir wormhole.

By changing the values of constant $(a)$ and radial coordinate $(r)$, we check
the behavior of shadow of Casimir wormhole in the equatorial plan by putting
$\theta_{0}=\frac{\pi}{2}$. We can see from the plots that shadow’s shape is a
perfect square and it is changed by changing values of $(a)$ and $(r)$ as
shown in Fig. 3. It is seen from the graphs that shadow of Casimir wormhole
reduces by increasing the values of $(a)$ and $(r)$.

##  IX Conclusion

In this paper, first of all we have used the GBT to find the DAL by Casimir
wormhole. For this purpose, we have obtained the optical Gaussian curvature
and have applied the GBT on optical Gaussian curvature to calculate the DAL.
The deflection angle has found as follows:

$\displaystyle\Theta$ $\displaystyle\thickapprox$
$\displaystyle\,{\frac{4a}{3b}}-\,{\frac{\pi\,{a}^{2}}{6{b}^{2}}}.$ (42)

This shows that deflection angle only depends on the impact parameter and the
value of $a$. Next we also check the graphical behavior of this deflection
angle. We have analyzed the plots that for small values of $a$ deflection
angle first decreasing and then goes to positive increase and for large values
of $(a)$ deflection angle continuing to decrease from positive to negative
side. We have also discussed the deflection angle of Casimir wormhole in the
presence of plasma medium. Deflection angle of Casimir wormhole in the
presence of plasma medium has obtained as follows:

$\displaystyle\Theta$ $\displaystyle\thickapprox$
$\displaystyle\,{\frac{4a}{3b}}-\,{\frac{\pi\,{a}^{2}}{6{b}^{2}}}-{\frac{7\,\pi\,{a}^{2}{\omega_{e}}^{2}}{12\,{b}^{2}{\omega_{\infty}}^{2}}}+2\,{\frac{{\omega_{e}}^{2}a}{b{\omega_{\infty}}^{2}}}.$
(43)

Now if we neglect the plasma effect
$(\frac{\omega_{e}}{\omega_{\infty}}\rightarrow 0)$, then this deflection
angle Eq. $(\ref{P2})$ reduces into this angle Eq. $(\ref{P1})$. We have also
observed the graphical behavior in the presence of plasma medium. In these
plots we have observed that the deflection angle in plasma medium is increased
as compared with non-plasma medium.

Then, we have calculated the shadow of Casimir wormhole. To do so, we have
first obtained the null geodesic of the Casimir wormhole and then we have
found the apparent shape of a shadow by using the celestial coordinates. It
has been seen in plots that its shadow is a perfect circle and this circle
change by changing values of $r$ and $a$. Hence, the graphs of the shadows
show that the radius of the shadow decreases when we increase the value of $r$
and $a$.

## References

  * (1) A. Einstein and N. Rosen, Phys. Rev. 48, 73-77 (1935).
  * (2) C. W. Misner and J. A. Wheeler, Ann. Phys. 2, 525 (1957); C. W. Misner, Phys. Rev. 118, 1110 (1959).
  * (3) J. A. Wheeler, Ann. Phys. 2, 604 (1957); J. A. Wheeler, Geometrodynamics (Academic, New York, 1962).
  * (4) J. A. Wheeler, Phys. Rev. 97, 511 (1955); R. W. Fuller and J. A. Wheeler, Phys. Rev. 128, 919 (1962).
  * (5) H. G. Ellis, Journal of Mathematical Physics, 14, 1973 pp. 104-118; 1974 Errata: 15, p. 520.
  * (6) M. S. Morris and K. S. Thorne, Am. J. Phys. 56, 395 (1988); M. S. Morris, K. S. Thorne and U. Yurtsever, Phys. Rev. Lett. 61, 1446 (1988).
  * (7) F. Abe, Astrophys. J. 725, 787 (2010).
  * (8) Y. Toki, et al. Astrophys. J. 740, 121 (2011).
  * (9) K. Nakajima, et al. Phys. Rev. D 85, 107501 (2012).
  * (10) T. Kitamura, et al., Phys. Rev. D 87, 027501 (2013).
  * (11) K. Izumi, et al., Phys. Rev. D 88, 024049 (2013).
  * (12) T. Kitamura, et al., Phys. Rev. D 89, 084020 (2014).
  * (13) K. Nakajima, et al., Phys. Rev. D 90, 084026 (2014).
  * (14) N. Tsukamoto, et al., Phys. Rev. D 97, 084051 (2018).
  * (15) C.R. Keeton, et al., Phys. Rev. D 72, 104006 (2005).
  * (16) C.R. Keeton, et al., Phys. Rev. D 73, 044024 (2006).
  * (17) S. Sahu, et al., Phys. Rev. D 88, 103002 (2013)
  * (18) J. Man, et al., J. Cosmol. Astropart. Phys. 11, 025 (2014).
  * (19) S.S. Zhao, et al., J. Cosmol. Astropart. Phys. 07, 007 (2016).
  * (20) X. Lu, et al., Eur. Phys. J. C 76, 357 (2016).
  * (21) S.S. Zhao, et al., Eur. Phys. J. C 77, 272 (2017).
  * (22) S.S. Zhao, et al., Phys. Lett. B 774, 357 (2017).
  * (23) W.G. Cao, et al., Eur. Phys. J. C 78, 191 (2018).
  * (24) R.N. Izmailov, et al., Eur. Phys. J. C 79, 105 (2019).
  * (25) C.Y. Wang, et al., J. Cosmol. Astropart. Phys. 04, 022 (2019).
  * (26) X. Pang, et al., Class. Quantum Gravity 36, 065012 (2019).
  * (27) F.Y. Liu, et al., Phys. Lett. B 795, 475 (2019).
  * (28) X. Lu, et al., Eur. Phys. J. C 79(12), 1016 (2019).
  * (29) R.N. Izmailov, et al., Ann. Phys. 413, 168069 (2020).
  * (30) X. Lu, et al., Eur. Phys. J. C 80, 625 (2020).
  * (31) X.Y. Zhu, et al., Eur. Phys. J. C 80, 444 (2020).
  * (32) X.M. Deng, et al., Phys. Rev. D 86, 044007 (2012).
  * (33) Y. Xie, et al., Monthly Notices of the Royal Astronomical Society 433, 3584 (2013).
  * (34) X.M. Deng, Int. J. Mod. Phys. D24, 1550056 (2015).
  * (35) X.M. Deng, et al., Eur. Phys. J. C 75, 539 (2105).
  * (36) X.M. Deng, et al., Modern Physics Letters A 31, 1650021 (2016).
  * (37) X.M. Deng, Int. J. Mod. Phys. D25, 1650082 (2016).
  * (38) X.M. Deng, European Physical Journal Plus 132, 85 (2017)
  * (39) G. Li, et al., Ann. Phys. 382, 136 (2017).
  * (40) T.E. Collett, et al., Science 360, 1342 (2018).
  * (41) Y. Xie, Phys. Rev. D 98, 021501 (2018).
  * (42) T. Damour and S. N. Solodukhin, Phys. Rev., D76:024016, 2007.
  * (43) P. Bueno, P. A. Cano, F. Goelen, T. Hertog, and B. Vercnocke, Phys. Rev., D97(2):024040, 2018.
  * (44) M. Visser, Phys. Rev., D39:3182-3184, 1989.
  * (45) F. S. N. Lobo, Phys. Rev., D71:124022, 2005.
  * (46) J. P. S. Lemos, F. S. N. Lobo, and S. Q. de Oliveira, Phys. Rev., D68:064004, 2003.
  * (47) J. Maldacena and L. Susskind, Fortsch. Phys., 61:781-811, 2013.
  * (48) S. W. Hawking, Phys. Rev., D37:904-910, 1988.
  * (49) S. V. Sushkov, Phys. Rev., D71:043520, 2005.
  * (50) K. A. Bronnikov and S.-W. Kim, Phys. Rev., D67:064027, 2003.
  * (51) V. P. Frolov and I. D. Novikov, Phys. Rev., D42:1057-1065, 1990.
  * (52) M. S. R. Delgaty and R. B. Mann, Int. J. Mod. Phys., D4:231-246, 1995.
  * (53) G. P. Perry and Robert B. Mann, Gen. Rel. Grav., 24:305-321, 1992.
  * (54) J. G. Cramer, R. L. Forward, M. S. Morris, M. Visser, G. Benford, and G. A. Landis, Phys. Rev., D51:3117-3120, 1995.
  * (55) J. Oliva, D. Tempo, and R. Troncoso, JHEP, 07:011, 2009.
  * (56) J. M. Maldacena and L. Maoz, JHEP, 02:053, 2004.
  * (57) G. Clement, Gen. Rel. Grav., 16:131, 1984.
  * (58) G. Clement, Phys. Rev., D51:6803-6809, 1995.
  * (59) G. Clement, J. Math. Phys., 38:5807-5819, 1997.
  * (60) E. I. Guendelman, Gen. Rel. Grav., 23:1415-1419, 1991.
  * (61) E. I Guendelman, A. Kaganovich, E. Nissimov, and S. Pacheva, Phys. Lett., B681:457-462, 2009.
  * (62) E. I. Guendelman, A. Kaganovich, E. Nissimov, and S. Pacheva, Int. J. Mod. Phys., A25:1405-1428, 2010.
  * (63) E. Guendelman, A. Kaganovich, E. Nissimov, and S.Pacheva, Phys. Lett., B673:288-292, 2009.
  * (64) I. Sakalli and A. Ovgun, Eur. Phys. J. Plus, 130(6):110, 2015.
  * (65) I. Sakalli and A. Ovgun, Astrophys. Space Sci., 359(1):32, 2015.
  * (66) M. G. Richarte, I. G. Salako, J. P. Morais Graca, H. Moradpour, and A. Övgün, Phys. Rev., D96(8):084022,2017.
  * (67) A. Övgün, K. Jusufi, and I. Sakalli, Phys. Rev. D 99, no. 2, 024042 (2019).
  * (68) A. Övgün, I. Sakalli, J. Saavedra and C. Leiva, Mod. Phys. Lett. A 35, no.20, 2050163 (2020).
  * (69) A. Övgün and K. Jusufi, Eur. Phys. J. Plus, 132(12):543, 2017.
  * (70) A. Övgün, Eur. Phys. J. Plus, 131(11):389, 2016.
  * (71) A. Övgün and M. Halilsoy, Astrophys.Space Sci., 361(7):214, 2016.
  * (72) K.Jusufi, A. Övgün, A. Banerjee, and I. Sakalli, Eur. Phys. J. Plus 134, no. 9, 428 (2019).
  * (73) K. Jusufi, A. Övgün, and A. Banerjee, Phys. Rev., D96(8):084036, 2017.
  * (74) K. Jusufi and A. Övgün, Phys. Rev., D97(2):024042, 2018.
  * (75) M. Halilsoy, A. Ovgun, and S. Habib Mazharimousavi, Eur. Phys. J., C74:2796, 2014.
  * (76) M. Azreg-Ainou and G. Clement, Gen. Rel. Grav., 25:881-891, 1993.
  * (77) G. W. Gibbons and M. S. Volkov, Phys. Rev., D96(2):024053, 2017.
  * (78) W. T. Kim, J. J. Oh, and M. S. Yoon, Phys. Rev., D70:044006, 2004.
  * (79) K. N. Singh, A. Banerjee, F. Rahaman and M. Jasim, Phys. Rev. D 101, no.8, 084012 (2020)
  * (80) T. Manna, F. Rahaman, S. Molla and A. Ali, Mod. Phys. Lett. A 34, no.32, 1950264 (2019)
  * (81) F. Rahaman, S. Sarkar, K. N. Singh and N. Pant, Mod. Phys. Lett. A 34, no.01, 1950010 (2019)
  * (82) F. Rahaman, N. Paul, A. Banerjee, S. De, S. Ray and A. Usmani, Eur. Phys. J. C 76, no.5, 246 (2016)
  * (83) M. Visser, Lorentzian Wormholes: From Einstein to Hawking (American Institute of Physics, New York, 1995).
  * (84) Donald Marolf and Joseph Polchinski, Phys.Rev.Lett. 111, 171301 (2013) .
  * (85) H. Casimir, Proc. Kon. Ned. Akad. Wetenschap 51, 793 (1948).
  * (86) Garattini, R. Eur. Phys. J. C 79, 951 (2019).
  * (87) G.W. Gibbons and M. C.Werner, Class. Quant. Grav. 25, 235009(2008).
  * (88) G.W. Gibbons and C. M.Warnick, Phys. Rev. D 79, 064031 (2009).
  * (89) G.W. Gibbons, C. A. R. Herdeiro, C. M.Warnick and M. C.Werner, Phys. Rev. D 79, 044022 (2009).
  * (90) G.W. Gibbons and M. Vyska, Class. Quant. Grav. 29, 065016 (2012).
  * (91) C. Bloomer, arXiv:1111.4998 [math-ph].
  * (92) M. C.Werner, Gen. Rel. Grav. 44, 3047 (2012).
  * (93) G.W. Gibbons, Class. Quant. Grav. 33, no. 2, 025004 (2016).
  * (94) A. Ishihara, Y. Suzuki, T. Ono, T. Kitamura and H. Asada, Phys. Rev. D 94, no. 8, 084015 (2016).
  * (95) P. Das, R. Sk and S. Ghosh, Eur. Phys. J. C 77, no. 11, 735 (2017).
  * (96) I. Sakalli and A. Ovgun, EPL 118, no. 6, 60006 (2017).
  * (97) K. Jusufi, M. C.Werner, A. Banerjee and A. Övgün, Phys. Rev. D 95, no. 10, 104012 (2017).
  * (98) G. Crisnejo, E. Gallo, Phys. Rev. D 97(12), 124016 (2018).
  * (99) T. Ono, A. Ishihara and H. Asada, Phys. Rev. D 96, no. 10, 104037 (2017).
  * (100) K. Jusufi, I. Sakalli and A. Övgün, Phys. Rev. D 96, no. 2, 024040 (2017).
  * (101) A. Ishihara, Y. Suzuki, T. Ono and H. Asada, Phys. Rev. D 95, no. 4, 044017 (2017).
  * (102) H. Arakida, Gen. Rel. Grav. 50, no. 5, 48 (2018).
  * (103) A. Övgün, I. Sakalli and J. Saavedra, JCAP 1810, no. 10, 041 (2018)
  * (104) A. Övgün, I. Sakalli and J. Saavedra, Annals Phys. 411, 167978 (2019).
  * (105) A. Övgün, G. Gyulchev and K. Jusufi, Annals Phys. 406, 152 (2019).
  * (106) K. Jusufi, A. Övgün, J. Saavedra, P. A. Gonzalez and Y. Vasquez, Phys. Rev. D 97, 124024 (2018).
  * (107) A. Övgün, K. Jusufi and I. Sakalli, Annals Phys. 399, 193 (2018).
  * (108) K. Jusufi and A. Övgün, Phys. Rev. D 97, no. 6, 064030 (2018).
  * (109) A. Övgün, Phys. Rev. D 98, no. 4, 044033 (2018)
  * (110) T. Ono, A. Ishihara and H. Asada, Phys. Rev. D 98, no. 4, 044047 (2018)
  * (111) T. Ono, A. Ishihara and H. Asada, Phys. Rev. D 99, no. 12, 124030 (2019).
  * (112) G. Crisnejo, E. Gallo and A. Rogers, Phys. Rev. D 99, no. 12, 124001 (2019).
  * (113) Z. Li and A. Övgün, arXiv:2001.02074 [gr-qc].
  * (114) Z. Li and T. Zhou, Phys. Rev. D 101, no. 4, 044043 (2020).
  * (115) Z. Li and J. Jia, Eur. Phys. J. C 80, no. 2, 157 (2020).
  * (116) Z. Li, G. He and T. Zhou, Phys. Rev. D 101, no. 4, 044001 (2020).
  * (117) A. Övgün, Universe 5, no. 5, 115 (2019).
  * (118) W. Javed, R. Babar, A. Ovgun, Phys. Rev. D 99, no. 8, 084012 (2019).
  * (119) W. Javed, A. Hazma and A. Övgün, Preprints 2019, 2019110142 (doi:10.20944/preprints201911.0142.v1)
  * (120) W. Javed, J. Abbas, and A. Övgün, Eur. Phys. J. C 79, no. 8, 694 (2019).
  * (121) A. Övgün, Phys. Rev. D 99, no. 10, 104075 (2019).
  * (122) W. Javed, j. Abbas and A. Övgün, Phys. Rev. D 100, no. 4, 044052 (2019).
  * (123) K. Jusufi and A. Övgün, Int. J. Geom. Meth. Mod. Phys. 16, no. 08, 1950116 (2019).
  * (124) G. Crisnejo, E. Gallo, and J. R. Villanueva, Phys. Rev. D 100, no. 4, 044006 (2019).
  * (125) W. Javed, R. Babar, and A. Övgün, Phys. Rev. D 100, no. 10, 104032 (2019)
  * (126) K. de Leon and I. Vega, Phys. Rev. D 99, no. 12, 124007 (2019)
  * (127) Y. Kumaran and A. Övgün, Chin. Phys. C 44, no. 2, 025101 (2020).
  * (128) W. Javed, J. Abbas, Y. Kumaran and A. Övgün, Preprints 2019, 2019110210 (doi: 10.20944/preprints201911.0210.v1).
  * (129) A. Övgün, I. Sakalli and J. Saavedra, Chin. Phys. C 44, no.12, 125105 (2020).
  * (130) Z. Li, G. Zhang and A. Övgün, Phys. Rev. D 101, no.12, 124058 (2020).
  * (131) B. Ahmedov, B. Turimov, Z. Stuchlík and A. Tursunov, Int. J. Mod. Phys. Conf. Ser. 49, 1960018 (2019).
  * (132) B. Turimov, B. Ahmedov, A. Abdujabbarov and C. Bambi, Int. J. Mod. Phys. D 28, no.16, 2040013 (2019).
  * (133) A. Abdujabbarov, B. Ahmedov, N. Dadhich and F. Atamurotov, Phys. Rev. D 96, no. 8, 084017 (2017).
  * (134) J. Schee, Z. Stuchlík, B. Ahmedov, A. Abdujabbarov and B. Toshmatov, Int. J. Mod. Phys. D 26, no. 5, 1741011 (2017).
  * (135) M.J. Rees, Ann. Rev. Astron. Astrophys. 22, 471 (1984).
  * (136) J. Kormendy and D. Richstone, Ann. Rev. Astron. Astrophys. 33, 581 (1995).
  * (137) Event Horizon Telescope Collaboration, Astrophys. J. Lett. 875, L1 (2019).
  * (138) J.L. Synge, Mon. Not. R. Astron. Soc. 131, 463 (1966).
  * (139) J.P. Luminet Astron. Astrophys. 75, 228 (1979).
  * (140) J.M. Bardeen, in Black holes, in Proceeding of the Les Houches Summer School, Session 215239, edited by C. De Witt and B.S. De Witt and B.S. De Witt (Gordon and Breach, New York, 1973).
  * (141) S. Chandrasekhar, The Mathematical Theory of Black Holes (Oxford University Press, New York, 1992).
  * (142) H. Falcke, F. Melia and E. Agol, Astrophys. J. 528, L13 (2000).
  * (143) R. Kumar, S. G. Ghosh and A. Wang, Phys. Rev. D 101, no.10, 104001 (2020).
  * (144) R. A. Konoplya, Phys. Let. B 804, 135363 (2020).
  * (145) R. Kumar, S. G. Ghosh and A. Wang, Phys. Rev. D 100, no. 12, 124024 (2019)
  * (146) R. A. Konoplya and A. Zhidenko, Phys. Rev. D 100, no. 4, 044015 (2019)
  * (147) A. Allahyari, M. Khodadi, S. Vagnozzi and D. F. Mota, JCAP 2002, no. 02, 003 (2020).
  * (148) M. Amir, K. Jusufi, A. Banerjee and S. Hansraj, Class. Quant. Grav. 36, no.21, 215007 (2019)
  * (149) M. Khodadi, A. Allahyari, S. Vagnozzi and D. F. Mota, JCAP 09, 026 (2020).
  * (150) S. Vagnozzi, C. Bambi and L. Visinelli, Class. Quant. Grav. 37, no.8, 087001 (2020)
  * (151) S. Vagnozzi and L. Visinelli, Phys. Rev. D 100, no.2, 024020 (2019)
  * (152) R. A. Konoplya, Phys. Lett. B 795, 1 (2019)
  * (153) Z. Younsi, A. Zhidenko, L. Rezzolla, R. Konoplya and Y. Mizuno, Phys. Rev. D 94, no. 8, 084025 (2016)
  * (154) P. V. P. Cunha, N. A. Eiro, C. A. R. Herdeiro and J. P. S. Lemos, JCAP 03, 035 (2020).
  * (155) A. Övgün and I. Sakalli, Class. Quant. Grav. 37, no.22, 225003 (2020).
  * (156) B. Carter, Phys. Rev. 174, 1559 (1968).
  * (157) J.M. Bardeen, in Black holes, in Proceeding of the Les Houches Summer School, Session 215239, edited by C. De Witt and B.S. De Witt and B.S. De Witt (Gordon and Breach, New York, 1973).
  * (158) S. Chandrasekhar, The Mathematical Theory of Black Holes (Oxford University Press, New York, 1992).


# Amplitude/Phase Retrieval for Terahertz Holography with Supervised and
Unsupervised Physics-Informed Deep Learning

Mingjun Xiang  Frankfurt Institute for Advanced Studies (FIAS), 60438
Frankfurt am Main, Germany  Xidian-FIAS International Joint Research Center,
60438 Frankfurt am Main, Germany  Physikalisches Institut, Goethe-Universität
Frankfurt am Main, 60438 Frankfurt am Main, Germany  Xidian University, 710071
Xi’an, China  Hui Yuan  Physikalisches Institut, Goethe-Universität Frankfurt
am Main, 60438 Frankfurt am Main, Germany  Corresponding author:
<EMAIL_ADDRESS><EMAIL_ADDRESS>Lingxiao Wang
Frankfurt Institute for Advanced Studies (FIAS), 60438 Frankfurt am Main,
Germany  Xidian-FIAS International Joint Research Center, 60438 Frankfurt am
Main, Germany  Kai Zhou  Frankfurt Institute for Advanced Studies (FIAS),
60438 Frankfurt am Main, Germany  Xidian-FIAS International Joint Research
Center, 60438 Frankfurt am Main, Germany  Corresponding author:
<EMAIL_ADDRESS><EMAIL_ADDRESS>Hartmut G. Roskos
Physikalisches Institut, Goethe-Universität Frankfurt am Main, 60438 Frankfurt
am Main, Germany

###### Abstract

Recently, digital holographic imaging techniques (including methods with
heterodyne detection) have found increased attention in the terahertz (THz)
frequency range. However, holographic techniques rely on the use of a
reference beam in order to obtain phase information. This letter investigates
the potential of reference-free THz holographic imaging and proposes novel
supervised and unsupervised deep learning (DL) methods for amplitude and phase
recovery. The calculations incorporate Fresnel diffraction as prior knowledge.
We first show that our unsupervised dual network can predict amplitude and
phase simultaneously, thus overcoming the limitation of previous studies which
could only predict phase objects. This is demonstrated with synthetic 2D image
data as well as with measured 2D THz diffraction images. The advantage of
unsupervised DL is that it can be used directly without labeling by human
experts. We then address supervised DL – a concept of general applicability.
We introduce training with a database set of 2D images taken in the visible
spectra range and modified by us numerically to emulate THz images. With this
approach, we avoid the prohibitively time-consuming collection of a large
number of THz-frequency images. The results obtained with both approaches
represent the first steps towards fast holographic THz imaging with reference-
beam-free low-cost power detection.

††journal: optica

Imaging at THz frequencies (0.1–10 THz, wavelengths of 3 mm–30 $\mu$m) has
received considerable attention in recent years. Application areas for THz
imaging explored at present include non-destructive testing [1], quality
monitoring [2], security screening [3, 4], biomedical imaging [5], and sensing
for robotics and vehicle control [6]. The applied imaging modalities
increasingly include coherent holographic approaches [7], variously because
they allow to capture of comparatively large scenes with good spatial
resolution, offer the possibility of 3D scene reconstruction, may have a
relaxed demand on the optical imaging optics, or lend themselves for advanced
numerical image processing, e.g., exploiting sparsity effects. However, at
lower THz frequencies, and there especially in the sub-1-THz frequency range,
which is most relevant for many applications, one is forced to perform
coherent detection as a serial rather than a parallel process [8, 9, 10]. The
main reason is that power availability at these THz frequencies is a critical
issue [11]. While detector arrays (with pixel numbers limited to hundreds or
at most a few thousand [7]) in principle are available, the limited power
makes it difficult to provide a reference beam for multi-pixel interferometric
or heterodyne phase measurements. The need for serial data recording
renderscoherent imaging time-consuming and it is detrimental for image
reconstruction, since phase distortions and noise introduced during signal
recording affect the data quality strongly, especially in the case of weak
signals [12, 13]. A powerful way to substantially reduce the power
requirements, to drastically simplify the measurement system, and to
accelerate the data acquisition process would open up, if the rebuilding of
the phase information could be done computationally from the amplitude or
intensity diffraction pattern obtained with the object-beam alone. Then it
would become feasible to revert to array-based, reference-beam-free power
detection for THz holography.

Conventional amplitude-phase recovery algorithms rely on iterative processing
of diffraction patterns (DPs) recorded at different distances to improve the
convergence and reliability of the reconstruction [14, 15, 16]. The cost is
the acquisition of large amounts of data and the manual searching for the
convergence condition. Recently, rapidly evolving DL techniques offer a novel
and efficient way to reconstruct images precisely [17, 18, 19], including
supervised DL methods which require plenty of labeled experimental data [20,
21, 22], and unsupervised DL methods that are suitable for unlabeled data
(such as PhysenNet [23]). However, the expensive acquisition of THz images
hinders naive supervised training with experimental data.

This letter proposes two novel physics-informed DL methods – one supervised
and the other unsupervised – for phase retrieval in THz holographic imaging,
using only visible-light MNIST datasets [24] for training, and incorporating
Fresnel diffraction as prior physical knowledge. Physics-informed DL provides
a way to integrate mathematical physics models and data into the learning
algorithms synergistically,ch that the calculations respect the physics laws
symmetries of the modeled system [25, 26] and has been proven powerful in
tackling inverse problems in physics [27, 28, 29]. The proposed supervised DL
method requires longer training, which is however a one-off effort, leads to a
high prediction speed after training, and makes the predictions fairly robust
against noise in experimentally generated images (see later in Fig. 4(f)). The
advantage of the unsupervised method is that it can be used directly both
without labeling by human experts and training, and that it achieves
simultaneous reconstruction of amplitude and phase, thus largely expanding the
working range of previous unsupervised DL methods.

Figure 1: Illustration of the supervised phase reconstruction model. (a)
Random original image from the MNIST data set, (b) modified amplitude and
phase image, (c) amplitude diffraction pattern in the THz domain calculated by
the physical model. (d) Scheme of the convolutional neural network (CNN) of
the supervised DL method. A measured diffraction pattern is the input of the
CNN, and the output is a phase image. The mean squared error (MSE) between
this phase reconstruction and the ground truth phase (i.e., the modified phase
image in (b)) is taken as the loss function to optimize the CNN (see
Supplementary for further details).

Figure 1 displays the flow chart of the proposed supervised phase
reconstruction, illustrated for the use with handwritten digits. Lacking a
sufficiently large data set recorded with THz radiation, we start from the
public MNIST data set which was generated from thousands of photographs taken
in the visible spectral range. The images from MNIST are then preprocessed to
transpose them into effective THz images, as shown in Fig. 1(a)-(c). The pre-
processing includes (i) superimposing the Gaussian field-amplitude profile of
a measured THz beam (recorded in the experimental setup discussed below) onto
the images to obtain the modified amplitude, and (ii) determining the phase
contour according to the object’s material and thickness to obtain the
modified phase. We then employ the physical model $H$ (see below) to calculate
the field-amplitude diffraction pattern of the effective THz image according
to the Huygens–Fresnel principle [30]. This pattern is fed as input into the
convolutional neural network (CNN), as shown in Fig. 1(d). Its architecture is
inspired by U-Net [31] and consists of a down-sampling path as well as a
symmetric up-sampling path (see Supplementary for a detailed description). The
output of the calculations are reconstructed phase images. In a subsequent
step, the same network is applied for amplitude retrieval from the diffraction
patterns.

The physical model, $H$, simulates the experimental THz imaging process. If a
planar object is illuminated by a beam, the complex-valued field amplitude
immediately behind the object can be written as

$E_{0}(x,y,z=0)=A_{0}(x,y,0)e^{i\phi_{0}(x,y;0)},$ (1)

where $A_{0}$ and $\phi_{0}$ are the amplitude and the phase at the object
plane. Over a distance $d$, diffraction reshapes the field to [30]

$E_{d}(x,y,z=d)=\iint\hat{E}_{0}(f_{x},f_{y})\,G\,e^{i2\pi(f_{x}x+f_{y}y)}df_{x}\,df_{y},$
(2)

where $G=e^{ikd\sqrt{1-\lambda^{2}f_{x}^{2}-\lambda^{2}f_{y}^{2}}}$ is the
wave propagation function, $\lambda$ the wavelength, $\hat{E}_{0}$ the spatial
Fourier transform of $E_{0}$ with $f_{x}=x/\lambda d$ and $f_{y}=y/\lambda d$
as the spatial frequencies in the $x$ and $y$ directions. The diffraction
pattern is the field’s absolute value

$A(x,y,z=d)=\left|E_{d}(x,y,z=d)\right|=H(\phi_{0},A_{0}),$ (3)

where $H(\cdot)$ represents the mapping function relating the object to the
diffraction pattern $A$. The challenge is now to achieve an inverse mapping,
${H^{-1}(\cdot)}$, such that

$\phi_{0}(x,y,0)=H^{-1}(A(x,y,z=d)).$ (4)

The supervised method of Fig. 1 utilizes a parameterized network function
$R_{\theta}$ ($\theta$ denoting the network weights and bias parameters) to
approach the desired inverse mapping ${H^{-1}(\cdot)}$ via learning based upon
the labeled training set $S_{T}={(A_{k},\phi_{k}),k=1,2,...,K}$, thus solving
the optimization problem of

$R_{\theta^{*}}=\mathop{\arg\min}_{\theta}\|R_{\theta}(A_{k})-\phi_{k}\|^{2}\qquad\forall(\phi_{k},A_{k})\in
S_{T}.$ (5)

A corresponding inverse mapping reconstructs the amplitude information of the
object, $A_{0}$.

Figure 2: Schematic illustration of the pipeline of unsupervised DL. The input
to the NN is a measured field-amplitude diffraction pattern, the outputs are
estimated phase and amplitude maps, which are then numerically propagated via
the model $H$ to simulate the diffraction and measurement processes, yielding
an estimated diffraction pattern. The MSE between the true and estimated
patterns provides the loss function used to optimize the NN parameters (see
Supplementary).

A different strategy is applied with the proposed unsupervised DL method shown
in Fig. 2. Without the need for pre-training on large labeled datasets, it
works directly on the amplitude $A(x,y,z=d)$, i.e., the propagated diffraction
pattern of the object. Unlike PhysenNet, which was developed for visible-range
images and only phase objects [23], the model proposed here is suitable for
all holographic systems and has no object restrictions. The field-amplitude
diffraction pattern is the only input to the CNN, which is designed to
generate estimated phase and amplitude maps simultaneously (see Supplementary
for a detailed description). The two output paths share the CNN’s front layers
in order to ensure that the same object information is analyzed, the path
splits in two only at the backend. The model $H$ is then applied to convert
the network outputs to an estimated diffraction pattern. The MSE between the
input and the estimated diffraction pattern is fed back to the network to
optimize the weights and bias values via gradient descent. Thus, the retrieval
of the phase is formulated as

$R_{\theta^{*}}=\mathop{\arg\min}_{\theta}\|H(R_{\theta}(A))-A)\|^{2}\,.$ (6)

This will force the calculated diffraction pattern to converge to the measured
pattern $I$, as the iterative process proceeds.

The pre-training and blind tests were performed on a PC with a sixteen-core
3.50-GHz CPU and 64 GB of RAM, using an Nvidia GeForce RTX 3080 GPU. On
average, the pre-training of the supervised CNN took $\sim$4 h for 60,000
pairs of training data and 10,000 pairs of validation data during $\sim$100
training epochs. The inference time of the trained network for a hologram of
$72\times 72$ pixels was $<0.1$ s. The training (optimization process) of the
unsupervised NN (with phase and amplitude channel) took $\sim$1 min during
$\sim$500 epochs.

We demonstrate the performance of the proposed supervised and unsupervised
methods with data derived from simulation (THz-emulated MNIST) and experiment.
Fig. 3 shows results obtained with the two methods for a randomly chosen MNIST
number (“0”). Note that the diffraction pattern as shown in Fig. 3(a) is the
only input to both methods. In the case of the supervised approach, the
networks are first pre-trained to represent the mapping function between the
amplitude diffraction pattern and the objects’ phases and amplitudes. The
corresponding learning curves in Fig. 3(f1) and f(2) show convergence to low
validation loss values after $\sim$50 epochs, indicating that the networks
have not become overfitted by the training data set. We obtain a validation
loss (MSE) of 0.00018 and 0.00048 after 100 epochs for amplitude and phase,
respectively. The reconstructed amplitude and phase of the test object are
shown in Fig. 3(d) and (e), with the ground truth for comparison in (b) and
(c). The MSE between the ground truth and the reconstruction is 0.00016 and
0.00011 for phase and amplitude, respectively.

As stated above, the proposed unsupervised DL method works directly on
diffraction data without prior training. The optimization process can be
monitored by the loss curve (MSE between the input diffraction pattern and the
one calculated from the CNN-estimated phase and amplitude data) which is
displayed in Fig. 3(g). As the optimization goes on, the MSE drops to
$10^{-6}$ after 500 epochs, yielding the concurrently updated diffraction,
amplitude and phase patterns shown in Fig. 3(h)-(j). It is found that the MSE
between the ground-truth phase and the predicted phase is 0.06, while the MSE
between the ground-truth amplitude and predicted amplitude is 0.00029.
Compared to the supervised DL method, the phase prediction performance is
slightly worse, while the amplitude prediction is comparably good. This trend
is confirmed with other test objects. In more ambiguous cases, supervised DL
is found to be clearly superior to unsupervised DL (see Supplementary). Table
1 summarizes the findings.

Table 1: Comparison of the two methods method | accuracy | reliability | pre-training
---|---|---|---
supervised DL | higher | higher | yes
unsupervised DL | lower | lower | no
Figure 3: Results obtained with the testing data set for supervised and
unsupervised DL. (a) Diffraction pattern of a digit "0" from the testing data
set. (b), (c) Ground truth of the object’s amplitude and phase. (d)-(f)
Supervised DL: Evolution of the MSE for amplitude (f1) and phase (f2) during
pre-training (blue lines) with 60,000 images and testing with 10,000 ones
(orange lines), and resulting predictions for the test "0" (d), (e). (g)-(j)
Unsupervised DL: Optimization (training) curve (g) and evolution of the
estimated diffraction pattern (h), amplitude (i), and phase (j) during
optimization.

To validate the proposed methods by experiments, we performed coherent
heterodyne THz measurements at 300 GHz with two frequency-locked electrical
multiplier chains (see also Supplementary). The objects consisted of thin
metal screens with cut-outs in the form of digits. The collimated THz beam
illuminated the object, the transmitted diffracted wave was detected
coherently 70 mm behind the object by a raster-scanned single-pixel TeraFET
detector [32]. The detector and the second emitter, which provided the local-
oscillator signal, were co-located on a 2D translation stage. From the
measured complex-valued diffraction pattern ($E_{d}(z=d)$), the object’s
amplitude ($A_{0}$) and phase ($\phi_{0}$) were reconstructed by back-
propagation via the inverse operation of Eq. 2. $A_{0}$, $\phi_{0}$ represent
the ground truth for the DL evaluation of the image represented by $A(z=d)$.

The measured $A(z=d)$ diffraction pattern and the ground truth together with
predictions from the DL methods are shown in Fig. 4. We reconstructed the
amplitude and phase in three different ways, by unsupervised and supervised DL
(Fig. 4(d)-(g)) and by a combination of both in a sequential process where the
prediction obtained by supervised DL served as input to the following
unsupervised process, yielding the final reconstruction shown in Fig. 4(h),
(i). The MSE values (referenced to the ground truth) for the unsupervised
method are 0.006 (0.05) for the reconstructed amplitude (phase). The
corresponding values for supervised DL are 0.0084 (0.0249). The combined
method yields the best MSE values: 0.0035 (0.0230) for amplitude (phase). The
image exhibits sharper edges and better contrast, and is closer to the ground
truth. We attribute the reason for this considerable improvement to a
suppression of the influence of noise, which is present in the measured
images, by the supervised DL step. This denoising effect invites further
studies in the future.

Figure 4: Comparison of experimental and reconstruction results. (a) Amplitude
diffraction pattern measured at a distance of 70 mm. (b), (c) Ground-truth
amplitude and phase. (d)-(g) Reconstructed amplitude and phase maps, obtained
by unsupervised DL ((d), (e)), respectively supervised DL ((f), (g)). (h), (i)
Amplitude and phase images reconstructed by the sequential combination of the
two methods.

In conclusion, we have proposed supervised and unsupervised deep-learning
methods, and a combination thereof, for the reconstruction of 2D terahertz
images, starting from amplitude diffraction patterns and deriving the
amplitude and phase in the object plane. As it is next to impossible to
generate by coherent measurements a THz image database of sufficient size for
pre-training of the neural networks, a large set of publicly accessible
visible-range images was converted numerically into emulated complex-valued
THz images. The neural network trained with these images achieves fast and
reliable image reconstruction with both emulated and experimentally measured
THz amplitude images. Regarding the proposed unsupervised method, a dual
output layer, integrating a physical model of the wave propagation, has been
introduced to simultaneously predict the amplitude and phase information. The
best reconstruction performance with measured image data is achieved by the
sequential application of both methods: The first step with the supervised
method eliminates much of the noise of the raw images and provides an already
good image estimate for the subsequent unsupervised method. At least for
sufficiently simple scenes, our findings confirm the potential of phase
retrieval from amplitude images of THz diffraction patterns.

Funding The work was supported by XF-IJRC (M. Xiang), the German Research
Foundation DFG with grant RO 770/48-1 (H. Yuan), the BMBF under ErUM-Data (K.
Zhou), the AI grant of SAMSON AG, Frankfurt (K. Zhou, L. Wang). Disclosures
The authors declare no conflicts of interest. Data Availability Statement Raw
data will be made available upon reasonable request. Supplemental document
Supporting content provided.

## References

  * [1] I. Amenabar, F. Lopez, and A. Mendikute, Journal of Infrared, Millimeter, and Terahertz Waves 34, 152 (2013).
  * [2] F. Ellrich, M. Bauer, N. Schreiner, A. Keil, T. Pfeiffer, J. Klier, S. Weber, J. Jonuscheit, F. Friederich, and D. Molter, Journal of Infrared, Millimeter, and Terahertz Waves 41, 470 (2020).
  * [3] J. F. Federici, B. Schulkin, F. Huang, D. Gary, R. Barat, F. Oliveira, and D. Zimdars, Semiconductor Science and Technology 20, S266 (2005).
  * [4] F. Friederich, W. von Spiegel, M. Bauer, F. Z. Meng, M. D. Thomson, S. Boppel, A. Lisauskas, B. Hils, V. Krozer, A. Keil, T. Löffler, R. Henneberger, A. K. Huhn, G. Spickermann, P. H. Bolívar, and H. G. Roskos, IEEE Transactions on Terahertz Science and Technology 1, 183 (2011).
  * [5] X. Yang, X. Zhao, K. Yang, Y. Liu, Y. Liu, W. Fu, and Y. Luo, Trends in Biotechnology 34, 810 (2016).
  * [6] D. Jasteh, M. Gashinova, E. Hoare, T.-Y. Tran, N. Clarke, and M. Cherniakov, “Low-thz imaging radar for outdoor applications,” in _2015 16th International Radar Symposium (IRS),_ (IEEE, 2015), pp. 203–208.
  * [7] G. Valušis, A. Lisauskas, H. Yuan, W. Knap, and H. G. Roskos, Sensors 21, 4092 (2021).
  * [8] A. Nahata, D. H. Auston, T. F. Heinz, and C. Wu, Applied Physics Letters 68, 150 (1996).
  * [9] H. Yuan, D. Voß, A. Lisauskas, D. Mundy, and H. G. Roskos, APL Photonics 4, 106108 (2019).
  * [10] H. Yuan, A. Lisauskas, M. D. Thomson, and H. G. Roskos, IEEE Transactions on Terahertz Science and Technology p. submitted (2022).
  * [11] S. Verghese, K. McIntosh, and E. Brown, Applied Physics Letters 71, 2743 (1997).
  * [12] M. Wan, J. J. Healy, and J. T. Sheridan, Optics & Laser Technology 122, 105859 (2020).
  * [13] H. Yuan, M. Wan, A. Lisauskas, J. T. Sheridan, and H. G. Roskos, “300-GHz holography with heterodyne detection,” in _Digital Holography and Three-Dimensional Imaging 2019,_ (Optica Publishing Group, 2019), p. Th3A.21.
  * [14] R. W. Gerchberg and W. O. Saxton, Optik 35, 237 (1972).
  * [15] J. R. Fienup, Applied Optics 21, 2758 (1982).
  * [16] K. Jaganathan, Y. C. Eldar, and B. Hassibi, Optical Compressive Imaging pp. 279–312 (2016).
  * [17] A. Goy, K. Arthur, S. Li, and G. Barbastathis, Physical Review Letters 121, 243902 (2018).
  * [18] G. Zhang, T. Guan, Z. Shen, X. Wang, T. Hu, D. Wang, Y. He, and N. Xie, Optics Express 26, 19388 (2018).
  * [19] P. Hand, O. Leong, and V. Voroninski, Advances in Neural Information Processing Systems 31 (2018).
  * [20] L. Boominathan, M. Maniparambil, H. Gupta, R. Baburajan, and K. Mitra, arXiv:1805.03593 (2018).
  * [21] M. Deng, S. Li, A. Goy, I. Kang, and G. Barbastathis, Light: Science & Applications 9, 1 (2020).
  * [22] G. Ju, X. Qi, H. Ma, and C. Yan, Optics Express 26, 31767 (2018).
  * [23] W. Fei, Y. Bian, H. Wang, L. Meng, and G. Situ, Light: Science & Applications 9, 77 (2020).
  * [24] Modified National Institute of Standards and Technology database of handwritten 2D digits, http://yann.lecun.com/exdb/mnist/.
  * [25] M. Raissi, P. Perdikaris, and G. E. Karniadakis, Journal of Computational Physics 378, 686 (2019).
  * [26] G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, Nature Reviews Physics 3, 422 (2021).
  * [27] L.-G. Pang, K. Zhou, N. Su, H. Petersen, H. Stöcker, and X.-N. Wang, Nature Commun. 9, 210 (2018).
  * [28] S. Shi, K. Zhou, J. Zhao, S. Mukherjee, and P. Zhuang, Phys. Rev. D 105, 014017 (2022).
  * [29] L. Wang, S. Shi, and K. Zhou, Phys. Rev. D 106, L051502 (2022).
  * [30] J. W. Goodman, Publishers, Englewood, Colorado (2005).
  * [31] L. Alzubaidi, J. Zhang, A. J. Humaidi, A. Al-Dujaili, Y. Duan, O. Al-Shamma, J. Santamaría, M. A. Fadhel, M. Al-Amidie, and L. Farhan, Journal of Big Data 8, 1 (2021).
  * [32] K. Ikamas, D. Čibiraitė, A. Lisauskas, M. Bauer, V. Krozer, and H. G. Roskos, IEEE Electron Device Letters 39, 1413 (2018).

ref