2022

[1]Ayan Dutta

1]School of Computing, University of North Florida, Jacksonville, 32224, FL,
USA

2]Department of Computer Science, Kent State University, Kent, 44242, OH, USA

# Deep Recurrent Q-learning for Energy-constrained
Coverage with a Mobile Robot

Aaron Zellner<EMAIL_ADDRESS><EMAIL_ADDRESS>Iliya Kulbaka
<EMAIL_ADDRESS>Gokarna Sharma<EMAIL_ADDRESS>[  [

###### Abstract

In this paper, we study the problem of coverage of an environment with an
energy-constrained robot in the presence of multiple charging stations. As the
robot’s on-board power supply is limited, it might not have enough energy to
cover all the points in the environment with a single charge. Instead, it will
need to stop at one or more charging stations to recharge its battery
intermittently. The robot cannot violate the energy constraint, i.e., visit a
location with negative available energy. To solve this problem, we propose a
deep Q-learning framework that produces a policy to maximize the coverage and
minimize the budget violations. Our proposed framework also leverages the
memory of a recurrent neural network (RNN) to better suit this multi-objective
optimization problem. We have tested the presented framework within a
$16\times 16$ grid environment having charging stations and various obstacle
configurations. Results show that our proposed method finds feasible solutions
and outperforms a comparable existing technique.

###### keywords:

Mobile Robot, Coverage, Energy Constraint, Deep Q-learning, Recurrent Neural
Networks

##  1 Introduction

Path planning has been in the center of attention in the mobile robotics
research domain for the past several decades. In a shortest path planning
(SPP) problem, given a map of the environment, the goal of the robot is to
plan a path from the start to the goal location that minimizes a pre-defined
cost function dijkstra1959note ; hart1968formal ; madkour2017survey . This has
applications in route planning in automated taxi services, among others. On
the other hand, in a coverage path planning (CPP) setting, the goal of the
robot is to go through all the locations in the environment at least once. In
this case, although there is a start point, there is no specific goal location
given cabreira2019survey ; choset2001coverage ; galceran2013survey .
Applications of CPP include automated wall painting and vacuum cleaning, among
others. In this paper, we study CPP and not SPP.

Only recently, researchers have started looking at a more practical variant of
these classic path planning problems, i.e., where the robot is cordless
meaning that it is battery operated and has a limited battery budget
sharma2019optimal ; shnaps2016online ; sotolongo2021shortest ;
sundar2013algorithms . Most modern day ground mobile robots cannot work for
more than an hour or two before requiring to recharge its battery. Therefore,
it is imperative that we consider the energy constraint put on by the robot’s
on-board battery. Let us take a running example of CPP under the energy
constraint. Let us assume that an iRobot Roomba is vacuum cleaning a house and
there are multiple charging stations located in it. To clean the house
completely, the objective of the Roomba robot would be to go to one of the
charging stations before running out of battery while making sure that all the
points on the floor are cleaned. As the user would have to manually put the
robot on to a charging station if it runs out of battery anywhere else, we
would have to minimize the number of such energy constraint violations occur.
CPP is related to a neighborhood variant of the famous Travelling Salesman
Problem (TSP), where the objective is to visit the neighborhoods of the cities
arkin1994approximation . Due to the NP-hardness of TSP, the complexity of this
problem also increases drastically with higher dimensions. A specific type of
CPP, called the lawn-mowing pattern planning for grass cutting, has been
proved to be NP-hard even in an obstacle-free environment
arkin2000approximation . To avoid such complexities, vacuum cleaning robots,
such as Roombas, randomly move in the environment for long periods
conjecturing that all the dirt will be collected palacin2005measuring .

Figure 1: An illustration of a typical rectangular environment. The green
color indicates the charging stations and gray indicates obstacles. The robot
always starts from a charging station as shown here. Four maximum possible
actions from any cell are shown with red dashed lines with arrows.

To this end, we propose a deep Q-learning framework, which neither requires
careful algorithm planning to address the complexity bottleneck, nor does it
require any prior domain knowledge. Instead, it learns over time to maximize
the coverage, i.e., the number of unique locations visited by the robot, and
to minimize the number of energy constraint violations – the number of non-
charging locations the robot reaches with no battery energy remaining. The
spatial information is extracted using two convolutional neural layers. As the
robot’s past coverage helps the robot plan a more informed path in the future,
it is imperative to take past decisions into account. To do this, we use a
Long Short Term Memory (LSTM) network – a type of recurrent neural network
(RNN), which allows information from the past experiences to persist. Similar
to a standard vacuum cleaning robot, we assume our robot to have an on-board
obstacle detection sensor (e.g., infrared) to detect and avoid obstacles. This
comes in handy if the obstacle locations in the environment are not known a
priori. We have implemented the proposed framework in Python and tested it
within a $16\times 16$ four-connected grid environment with different obstacle
configurations. We have also tested with varied battery budget amounts.
Results show that our proposed technique learns paths for the robot to cover
the entire environment without violating the budget constraint.

In summary, the primary contributions of our paper are as follows.

  * •

To the best of our knowledge, this is the first study that formulates the
energy-constrained CPP problem as a deep Q-learning problem.

  * •

We have used a recurrent layer along with convolutional layers for the robot
to learn from the history of spatial navigation.

  * •

Our proposed learning model has been empirically shown to learn effective
coverage paths for the robot to completely explore environments with various
obstacle configurations and battery budget amounts.

##  2 Related Work

CPP is one of the most well-known research problems in mobile robotics. This
has applications in painting sheng2000automated ; zhou2022building ,
agriculture dutta2021multi ; pham2017aerial ; valente2013aerial , mapping
almadhoun2019survey ; matignon2018multi ; paull2018probabilistic , among
others. Surveys of CPP techniques in robotics can be found in
cabreira2019survey ; choset2001coverage ; galceran2013survey . CPP is similar
to the covering salesman problem (CSP) current1989covering , which is a
variant of the well-known NP-hard travelling salesman problem (TSP). Covering
an environment even without any obstacle has been proved to be NP-hard
arkin2000approximation . In this paper, we are interested in grid-based
coverage, where the environment is divided into uniform cells and binary
values embedded into these cells indicate the presence or absence of
obstacles. Wavefront planners can be used for covering such discrete
environments zelinsky1993planning . Other techniques involve spanning tree
gabriely2002spiral  and neural network yang2004neural -based solutions.
However, most of these exiting solutions in the literature assume the robot to
have unlimited on-board power supply. Only recently, the researchers have
started considering the energy constraint of the robot dogru2022eco ;
sharma2019optimal ; shnaps2016online ; WeiI18 . In this setting, due to the
limited energy budget, the robot will need to stop at one or more charging
stations to cover the entire environment. Understandably, this adds a new
challenge to the already complex CPP with a mobile robot. In shnaps2016online
, the authors have shown that in an online setting, where the obstacle
patterns and locations are unknown to the robot, the optimal coverage can only
provide $O(\log B)$ approximation. Sharma et al. sharma2019optimal  have
presented the first such online algorithm. For the offline version, where the
environment is known (similar to our model in this paper), Wei and Isler
presented $O(\log B)$-approximation WeiI18  and constant-factor approximation
WeiICRA18  algorithms.

On the other hand, due to the advancement in the field of deep learning and
reinforcement learning, some recent studies have employed deep reinforcement
learning (DRL) techniques for CPP with a mobile robot. The benefit of such an
approach is that no domain knowledge and/or careful designing of combinatorial
algorithm is needed. Examples include apuroop2021reinforcement ;
fevgas2022coverage ; kyaw2020coverage ; lakshmanan2020complete ;
zhu2019complete . Among these, the study by Zhu et al. zhu2019complete
consider CPP with an underwater robot. Similar to our paper, Lakshmanan et al.
lakshmanan2020complete  and Apuroop et al. apuroop2021reinforcement  have used
convolutional layers along with a recurrent module for complete area coverage.
Fevgas et al. fevgas2022coverage  have proposed an energy-efficient coverage
technique for multi-robot CPP. Kyaw et al. kyaw2020coverage  have presented an
attention-based actor-critic neural network architecture for CPP. Unlike our
studied problem in this paper, none of these DRL-based solutions from the
literature consider the energy constraint of the robot and the presence of
charging stations in the environment.

##  3 Model

Environment.  The environment $P$ is a planar polygon containing a set of
charging stations $S=s_{1},s_{2},\cdots,s_{k}$ inside it. $P$ may possibly
contain polygonal obstacles. The obstacles are assumed to be static meaning
that they do not move during the coverage process. See Fig. 1 for an
illustration of $P$ with two polygonal obstacles. $P$ is discretized into
cells forming a 4-connected grid.

Robot.  We consider the robot $r$ to be initially positioned at a charging
station $s_{s}$ inside $P$. $r$ has size $L\times L$ that fits within a grid-
cell in $P$. The robot $r$ can move to any of the four orthogonal neighbor
cells (if the cell is not occupied by an obstacle) from its current cell. We
also assume that $r$ has the knowledge of the global coordinate system through
a compass on-board, that means it knows left (West), right (East), up (North),
and down (South) cells consistently from its current cell. $r$ is equipped
with a position sensor (e.g., GPS) and an obstacle-detection sensor (e.g.,
laser rangefinder). We assume that with the laser rangefinder, the robot can
detect obstacles in any of its neighbor cells. Let $p_{crnt}$ and $p_{nxt}$
denote $r$’s current and next locations in $P$. $neigh(p)$ represents the
maximum of $4$ neighbor cells of any cell $p\in P$ and $p_{nxt}\in
neigh(p_{crnt})$. The robot has sufficient on-board memory to store
information necessary to facilitate the coverage process. The energy
consumption of the robot is proportional to the distance travelled, i.e., the
energy budget of $B$ allows the robot to move $B$ units distance.

Objective Function.  A path ${\cal Q}$ is a list of grid-cells that $r$ visits
starting with $s_{s}$. Let $br(p)$ denote the remaining battery budget at cell
$p\in P$. Notice that if there are some obstacles within $P$ located in such a
way that they divide $P$ into two sub-polygons $P_{1}$ and $P_{2}$ with
$P_{1}$ and $P_{2}$ sharing no common boundary, then $r$ cannot fully cover
$P$. Therefore, it is considered that there is no such cell $p\in P$, i.e.,
there is (at least) a path from some $s_{i}\in S$ to any obstacle-free cell of
$P$. We call a cell free if it is not occupied by an obstacle.

######  Definition 1 (Reachable Cell).

Any cell $p\in P$ is called reachable by the robot $r$, if and only if (a) it
is a free cell, (b) it is within distance $\lfloor B/2\rfloor$ from a charging
station $s_{i}\in S$, and (c) there must be at least a path of consecutive
free cells from $s_{i}$ to $p$.

The goal of the coverage algorithm is to find a path ${\cal Q}$ for the robot
such that

  * •

Condition (a): $br(p_{q})\geq 0$ for each cell $p_{q}\in{\cal Q}$.

  * •

Condition (b): The cells in ${\cal Q}$ collectively cover the whole
environment $P$, i.e., $\cup_{|{\cal Q}|}{\cal Q}=P$,

and the following performance metric is optimized:

  * •

Performance metric: The total length of the path in ${\cal Q}$, denoted as
$l({\cal Q})$, is minimized.

Note that we do not consider the charging time of the battery since this delay
does not impact the performance metrics we consider. The coverage path
planning problem is formally defined as follows.

######  Definition 2.

Given a planar polygonal environment $P$ possibly containing obstacles, and a
mobile robot $r$ having battery budget of $B$ initially positioned at a
charging station $s_{s}$ inside $P$, the objective of $r$ is to visit all the
reachable cells in $P$ through a path such that

  * •

Conditions (a) and (b) are satisfied, and

  * •

The performance metric is minimized.

##  4 Recurrent Q-learning for Energy-constrained CPP

$1$$1$$1$$1$$1$$1$$1$$1$$1$$1$$1$$1$$1$$1$$1$ObstaclesCharging stationsCurrent
locationPast locations Figure 2: Inputs to our proposed neural network model
are shown (following the illustrated environment in Fig. 1) – the obstacle and
charging station matrices do not change over time. However, the current
location matrix is updated after every action is taken by the robot and a new
cell is reached. The empty cells in the matrices are filled with 0’s (not
shown here for brevity).

###  4.1 State, Action, and Reward

State Space.  A state $x\in\mathcal{S}$ of the robot is represented by a
$4\times N\times N$ tensor (simply a 4D tensor), where $N$ is the length of
one side of the environment.

Obstacle channel.  The obstacles in the environment are represented with
$1$’s, i.e., the cells that are covered with obstacles contain $1$’s and the
free cells contain $0$’s.

Charging station channel.  The locations of the charging stations $S$ are
represented with $1$’s and the grid cells that are not charging stations
contain $0$’s.

Current location channel.  The next layer represents the robot’s current grid
cell location – it is denoted by a $1$ whereas the remaining cells contain
$0$’s.

Covered locations channel.  The state also represent the grid cell locations
that the robot has visited in the past. These visited cells are denoted by
$1$’s and the non-visited cells are denoted by $0$’s. Note that even if a cell
is visited more than once, it is still represented by a $1$.

An example of the state space is shown in Fig. 2. The idea of using multi-
dimensional binary tensors as input states is popular in the literature
heydari2021reinforcement ; theile2020uav .

Action Space.  The action space, $\mathcal{A}$, of the robot contains four
perpendicular actions: {Up, Down, Left, Right}.

Reward Function.  The reinforcement learning is driven by the underlying
reward structure. A reward function
$R:\mathcal{S}\times\mathcal{A}\longrightarrow\mathbb{R}$ maps a state-action
pair $(s,a)$ to a real number. In this paper, we model the reward such that
our multi-objective optimization criteria is accounted for. Let $r^{p}$ and
$r^{s}$ denote the rewards for our two optimization criteria respectively: 1)
maximize the number of visited cells, and 2) minimize the number of energy
constraint violations. These individual reward functions are defined as
follows where $p_{i}\in P$ is the cell where $r$ reaches after executing
$a\in\mathcal{A}$ in state $x\in\mathcal{S}$.

$r^{p}(p_{i})=\begin{cases}2,&\text{if }\begin{aligned} \text{$p_{i}$ is not
visited before}\end{aligned}\\\ -1,&\text{otherwise}\\\ \end{cases}$ (1)

$r^{s}(p_{i})=\begin{cases}0.1,&\text{if }\begin{aligned} br(p_{i})\geq
0\end{aligned}\\\ -3,&\text{otherwise}\\\ \end{cases}$ (2)

To combine these two sub-reward components, we employ a weighted linear
combination technique as follows.

$R=\frac{r^{p}(p_{i})+r^{s}(p_{i})}{2}$ (3)

Following Eq. 3, the robot will try to visit the unvisited grid cells without
violating the budget than to repeatedly visit the already visited cells. A
high negative reward of $-3$ will also lead to a policy where the number of
energy constraint violations reduce. If the robot completes covering the whole
environment without any violation, for the last action it takes, we give it a
termination reward of $R=200$.

###  4.2 Background on Reinforcement Learning

In reinforcement learning, the robot interacts with an environment in discrete
time steps $T$. First it takes an action $a\in\mathcal{A}$ in state $x$,
receives a corresponding reward $R$, and the state of the robot changes to
$x^{\prime}\in\mathcal{S}$ because of the action execution. The process
continues until a terminal state is reached. Next, the environment is reset
and the above-mentioned steps are repeated. The expected value of taking
action $a$ in state $x$ is given by its Q-value:
$Q(x,a)=\sum\limits_{k=0}^{\infty}\gamma R_{t+k}$. $\gamma\in[0,1]$ is the
discount factor that balances the importance of present and future rewards in
the Q-value. An optimal policy can be found in this one-step setting by taking
the highest Q-value actions in each state: $Q^{*}(x,a)=\max Q(x,a)$. Model-
free Q-learning is an example of one type of reinforcement learning to learn
the optimal action Q-values in each state sutton1998introduction ;
watkins1989learning . As it is infeasible to learn the optimal Q-values for
all state-action pairs, the Q-function is recursively updated as the
following.

$Q(x,a)=(1-\alpha)Q(x,a)+\\\
\alpha(R_{t}+\gamma\max_{a^{\prime}\in\mathcal{A}}Q(x^{\prime},a^{\prime}))$
(4)

In the recent years with the advancements in deep learning, the Q-values are
approximated using a neural network mnih2015human ; van2016deep . The
parameters in this network are denoted by $\phi$ and the network is
represented by $Q_{\phi}$. The goal of this network is to output the Q-values
of the state-action pairs. We adjust $\phi$ for $Q_{\phi}$ to output optimal
Q-values.

$+$Budget Figure 3: Deep recurrent Q-learning architecture for energy-
constrained CPP.

###  4.3 Neural Network Architecture and Algorithm

The proposed Q-network architecture is shown in Fig. 3. The input to the
network is a four channel state information, $x\in\mathcal{S}$. Note that in
case of a no-obstacle environment, the last channel input will not be
required. These inputs are then passed through two convolutional layers along
with batch-normalization and rectified linear unit (ReLu) activation. We have
used convolutional layers to extract the spatial information from the input
states. Both the convolutional layers have $16$ output channels. The filter
size and stride are set to $5$ and $1$ respectively in our experiments. The
output of the second convolutional layer is concatenated with the remaining
budget of the robot. Next, this is passed to an recurrent neural network (RNN)
module hausknecht2015deep . We have used Long Short Term Memory (LSTM) as our
RNN model hochreiter1997long . LSTM networks are useful for remembering long
history of information such as time-series data. LSTM maintains a cell state
that regularizes data addition or deletion using a gate. The hidden cell size
of the LSTM module is set to $128$. The output of the LSTM is then given as an
input to a linear layer of neurons, which has an output size of
$|\mathcal{A}|$. These outputs are the Q-values of the $|\mathcal{A}|$ actions
that the robot can take. If any action is infeasible, e.g., if it leads to a
collision with an obstacle, we mask the corresponding Q-value with $-\infty$.

1Initialize a priority replay memory $\mathcal{D}$, policy and target networks
$Q_{\phi}\text{ and }Q_{\phi^{-}}$ respectively.

2 for  _each training episode $e$_ do

3 for  _each step $t$ in $e$_ do

4 Sample action $a\in\mathcal{A}$ using $\epsilon$-greedy strategy;

5 Execute action $a$, receive reward $R$, and transition to state
$x^{\prime}$;

6 Store transition information $\langle x,a,R,x^{\prime}\rangle$ in replay
buffer $\mathcal{D}$;

7 Sample a minibatch of $m$ experiences from $\mathcal{D}$;

8 Calculate the target value $Y_{Q}$ (Eq. 6);

9 Regress the $Q$-network towards this target value following Eq. 5;

10 Update the target network parameters $\phi^{-}$ every $\tau$ episodes;

Algorithm 1 Deep Recurrent Q-learning for Energy-constrained CPP

We present the pseudo-code of the proposed method in Algorithm 1. Similar to
Double DQN (DDQN) van2016deep , we maintain two copies of the neural network:
one for the policy ($Q_{\phi}$) and the other for the target ($Q_{\phi^{-}}$)
for better stabilization and reducing overestimation in action values
mnih2015human ; van2016deep . The training happens in episodes. In each
episode, the robot starts from a starting cell $S$ and moves for $T$ steps. If
the environment is fully explored before $T$ steps, the episode is terminated
and the environment is reset. In each such step within an episode, $r$ first
samples an action $a\in\mathcal{A}$ using the $\epsilon$-greedy strategy,
i.e., it chooses the action with the maximum Q-value with probability
$\epsilon$, otherwise $a$ is random with probability $(1-\epsilon)$. Next, it
executes $a$ and receives a corresponding reward $R$ following Eq. 3. Due to
the action execution, the current state of the robot changes from $x$ to
$x^{\prime}$. The experience tuple $\langle x,a,R,x^{\prime}\rangle$ is stored
into a memory replay buffer $\mathcal{D}$. We have employed a priority memory
buffer fedus2020revisiting . In every step, a mini-batch of experience
memories $\mathcal{B}$ is selected from $\mathcal{D}$ and the network
parameters are regressed toward their target values by finding the gradient
descent of the following temporal loss function

$\mathcal{L}({\phi})=\mathbb{E}[(Y_{Q}(x,a,x^{\prime})-Q_{\phi}(x,a))^{2}]$
(5) $Y_{Q}(x,a,x^{\prime})=R+\gamma Q_{\phi^{-}}(x^{\prime},\text{arg}\max
Q_{\phi}(x^{\prime},a^{\prime}))$ (6)

Following DDQN, the best action $a$ is chosen using $Q_{\phi}$ but the
expected Q-value of that action is estimated using $Q_{\phi^{-}}$ van2016deep
. The optimization is performed using the Adam optimizer after every action.
On the other hand, the parameters of the target network (${\phi}^{-}$) are
updated after every $\tau$ episodes. This continues until the pre-defined
maximum number of episodes is reached. As the environment is assumed to be
known, we keep track of the best coverage path found by the robot in training
and use it as the final solution.

##  5 Experiments

###  5.1 Settings

We have implemented the proposed algorithm in Python and used PyTorch library
for deep reinforcement learning functions. The experiments are run on desktop
computers with NVIDIA GPU and CUDA. The main parameters used in the
experiments are listed in Table 1. We have designed three environment
configurations with charging stations and obstacles. These are named ‘Maps’
and are numbered from 1 through 3 as shown in Fig. 5. The environment size is
$16\times 16$, i.e., $N=16$. The robot always starts from the $(0,0)$ corner
and the budget of it is set to $5N$ unless mentioned otherwise. We have used a
prioritized memory replay to store the experience transitions SchaulQAS15 .

Baseline. As mentioned earlier, this is the first work that studies mobile
robot coverage under energy constraint in the presence of multiple charging
stations. Therefore we could not compare against any existing baseline.
Instead, we have used an existing CNN-based architecture as a baseline neural
network model theile2020uav . This said CNN model has been developed for CPP
with a unmanned aerial vehicle (UAV), where the UAV learns to cover an area
with a limited energy budget. However, no charging station is available in the
environment. Similar to our model, the map of the environment is passed to the
CNN through multiple channels in this existing model theile2020uav . We call
this baseline ‘CNN’ in the remainder of the paper.

Parameters |  Values
---|---
State |  $4\times 16\times 16$ matrix
Action |  Up, Down, Left, Right
Number of training episodes |  $10,000$
Episode length |  $10\times 16\times 16$
Priority replay memory size |  $50,000$
Mini-batch size |  $64$
Discount factor |  $0.90$
Learning rate |  $0.001$
Target network update frequency |  $20$
Epsilon decay type |  Exponential (Fig. 4)
Epsilon decay rate |  $2100$
Loss function |  Mean Square Error
Optimizer |  Adam
Table 1: List of parameters used in our experiments. Figure 4: $\epsilon$ values used through the training episodes. |  |
---|---|---
Map 1 | Map 2 | Map 3
Figure 5: Three different maps used in our experiments. Red cells indicate the charging stations and the black cells indicate the obstacles. |  |
---|---|---
Coverage: Map 1 | Coverage: Map 2 | Coverage: Map 3
Figure 6: Percentages of coverage in three training maps. |  |
---|---|---
Violations: Map 1 | Violations: Map 2 | Violations: Map 3
Figure 7: The number of violations in three training maps. |  |
---|---|---
Reward: Map 1 | Reward: Map 2 | Reward: Map 3
Figure 8: Rewards in three training maps.

###  5.2 Results

First, we are interested in investigating three metrics during training:
coverage percentage (what percentage of cells in the environment have been
covered by the robot in an episode), number of violations (the number of cells
visited by the robot with negative budget in an episode), and rewards per
episode. The results for these metrics for different maps are presented in
Figs. 6–10. The faint lines in these plots indicate the raw numbers and the
bold lines indicate the 100-episode moving averages.

Results show that our proposed technique achieves higher coverage percentage
than the CNN-based technique (Fig. 6). Out of 7000 training episodes, our
model covered the full environment 744, 1081, and 1091 times, whereas the CNN-
based did this only 34, 62, and 34 times on the three tested maps
respectively. On the other hand, for Maps 1 and 3, CNN yielded more episodes
where $r$ did not violate the energy constraint. See Table 2 for reference.
Overall, we are interested in finding a solution where the coverage is $100\%$
and the number of violations is 0. The episodes in which $r$ achieves both
these objectives, we call them the ‘best’ episodes. In Table 3, we summarize
details about the best episodes yielded by the two techniques with $B=5N$. We
notice that our model always yields a valid solution, whereas the CNN-based
approach could never find such a solution. This highlights the advantage of
using our presented model over the comparable CNN-based model. As the presence
of obstacles lead the robot to take convoluted paths to cover the environment,
it not only needs careful planning to cover while avoiding the obstacles, but
also to recharge itself among these detours. This leads to less number of
episodes with no energy constraint violation. However, due to the smaller
number of free cells to cover, the robot achieves complete coverage in more
episodes. These conclusions are evident from Tables 2, 4, and 5, where we see
that there are numerous episodes in which $r$ achieves completes coverage, but
it can do so without any constraint violation in a relatively small number of
episodes. Note that the ‘Max. Reward’ row (or, column) in the tables indicate
the highest reward earned by $r$ in the ‘best’ episodes. As the episodes in
which one of the objectives is not fulfilled are not candidates for the final
solution, therefore, we do not report the highest rewards for them.

Map | Model |  # Full Cov. Episodes |  # No Vio. Episodes
---|---|---|---
Map 1 | Our |  744 |  90
CNN theile2020uav  |  34 |  416
Map 2 | Our |  1081 |  108
CNN theile2020uav  |  62 |  76
Map 3 | Our |  1091 |  90
CNN theile2020uav  |  34 |  191
Table 2: Summary of performances of two individual optimization metrics. Map | Model | # Best Episodes | Max. Reward
---|---|---|---
Map 1 | Our | 9 | -339
CNN theile2020uav  | 0 | NA
Map 2 | Our | 7 | -215
CNN theile2020uav  | 0 | NA
Map 3 | Our | 16 | -259
CNN theile2020uav  | 0 | NA
Table 3: Summary of overall performances of our approach and the baseline.

We have also tested with budget values changed to $B=4N$ and $6N$. The results
are presented in Figs. 9, 10 and in Tables 4, 5. Note that these experiments
are run only on Map 1. When the budget is lower, the robot needs to be more
innovative in planning its path that does not lead to a constraint violation.
As expected from the earlier results, the CNN-based model could not find a
valid solution during the training process. On the other hand, when the budget
is increased to $6N$, it could find one such valid solution. However, the
highest reward yielded by the final solution using our proposed technique is
$5.70$ times higher than that of the CNN-based model (Table 5).

|  |
---|---|---
Figure 9: Coverage, violation count, and reward in Map 1 with $B=4N$. Metric | Our | CNN theile2020uav
---|---|---
# Full Cov. Episodes | 1097 | 28
# No Vio. Episodes | 11 | 104
# Best Episodes | 1 | 0
# Max. Reward | -152 | NA
Table 4: Summary of overall performances of our approach and the baseline (Map 1, $B=4N$). |  |
---|---|---
Figure 10: Coverage, violation count, and reward in Map 1 with $B=6N$. Metric | Our | CNN theile2020uav
---|---|---
# Full Cov. Episodes | 1154 | 25
# No Vio. Episodes | 247 | 1568
# Best Episodes | 55 | 1
# Max. Reward | -63 | -358
Table 5: Summary of overall performances of our approach and the baseline (Map
1, $B=6N$).

One of the main takeaways from the results is that our proposed use of a
recurrent layer leads not only to higher reward yield, but also to finding
valid solutions, i.e., where the robot covers the entire environment without
violating the energy constraint. Furthermore, we observe that finding such a
valid solution become even more challenging with lower budget, whereas with
more budget, the robot can cover the environment without budget violation
relatively easily.

##  6 Conclusion and Future Work

In this paper, we have proposed a deep reinforcement learning based solution
for coverage path planning (CPP) with an energy-constrained mobile robot.
Although combinatorial algorithms have been proposed for such CPP problems in
the presence of a single charging station, to the best of our knowledge, these
exists no solution for this problem in the presence of more than one charging
station. The environment is discretized into square grid cells and the robot
is allowed to move to any of the four orthogonal neighbor cells at any given
step. Our proposed solution in this paper uses a deep Q-network based
technique that employs two convolutional neural layer along with a recurrent
neural module, called Long Short Term Memory (LSTM). The proposed neural
network architecture takes state inputs via four channels and outputs Q-values
for the possible actions. We have compared our proposed technique against a
comparable solution that does not use any recurrent neural component. We
trained the model on three environments with different obstacle configurations
and with various budgets. Results show that our proposed technique outperforms
the baseline in all the used environment configurations.

Given that this is the first study on CPP in the presence of multiple charging
stations, our assumption was that the environment configuration, i.e., the
locations of the obstacles are known a priori, and therefore, the spatial map
of these obstacles can be passed as part of the state input to the neural
network. In the future, we plan to extend this work to incorporate unknown
obstacles. One potential direction would be to use the robot’s on-board
sensors such as a laser range finder to detect the nearby obstacles and build
the obstacle map on-the-fly. This partial information can later be fed to the
neural network. Using a recurrent module such as an LSTM, the robot will learn
to avoid the obstacles while covering the entire environment without violating
the energy constraint. Another future direction of investigation would be to
consider dynamic obstacles unlike the static ones that have usually been
considered in related studies.

## Declarations

The datasets generated during and/or analysed during the current study are
available from the corresponding author on reasonable request.

## References

  *   * (1) Dijkstra, E.W.: A note on two problems in connexion with graphs. Numerische mathematik 1(1), 269–271 (1959)
  * (2) Hart, P.E., Nilsson, N.J., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics 4(2), 100–107 (1968)
  * (3) Madkour, A., Aref, W.G., Rehman, F.U., Rahman, M.A., Basalamah, S.: A survey of shortest-path algorithms. arXiv preprint arXiv:1705.02044 (2017)
  * (4) Cabreira, T.M., Brisolara, L.B., Ferreira Jr, P.R.: Survey on coverage path planning with unmanned aerial vehicles. Drones 3(1), 4 (2019)
  * (5) Choset, H.: Coverage for robotics–a survey of recent results. Annals of mathematics and artificial intelligence 31(1), 113–126 (2001)
  * (6) Galceran, E., Carreras, M.: A survey on coverage path planning for robotics. Robotics and Autonomous systems 61(12), 1258–1276 (2013)
  * (7) Sharma, G., Dutta, A., Kim, J.-H.: Optimal online coverage path planning with energy constraints. In: Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, pp. 1189–1197 (2019). International Foundation for Autonomous Agents and Multiagent Systems
  * (8) Shnaps, I., Rimon, E.: Online coverage of planar environments by a battery powered autonomous mobile robot. IEEE Transactions on Automation Science and Engineering 13(2), 425–436 (2016)
  * (9) Sotolongo, B., Dutta, A., Sisley, S., Sharma, G.: Shortest path planning with an energy-constrained robot. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 745–750 (2021). IEEE
  * (10) Sundar, K., Rathinam, S.: Algorithms for routing an unmanned aerial vehicle in the presence of refueling depots. IEEE Transactions on Automation Science and Engineering 11(1), 287–294 (2013)
  * (11) Arkin, E.M., Hassin, R.: Approximation algorithms for the geometric covering salesman problem. Discrete Applied Mathematics 55(3), 197–218 (1994)
  * (12) Arkin, E.M., Fekete, S.P., Mitchell, J.S.: Approximation algorithms for lawn mowing and milling. Computational Geometry 17(1-2), 25–50 (2000)
  * (13) Palacin, J., Palleja, T., Valganón, I., Pernia, R., Roca, J.: Measuring coverage performances of a floor cleaning mobile robot using a vision system. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 4236–4241 (2005). IEEE
  * (14) Sheng, W., Xi, N., Song, M., Chen, Y., MacNeille, P.: Automated cad-guided robot path planning for spray painting of compound surfaces. In: Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000)(Cat. No. 00CH37113), vol. 3, pp. 1918–1923 (2000). IEEE
  * (15) Zhou, Y., Li, P., Ye, Z., Yue, L., Gui, L., Jiang, X., Li, X., Liu, Y.-h.: Building information modeling-based 3d reconstruction and coverage planning enabled automatic painting of interior walls using a novel painting robot in construction. Journal of Field Robotics (2022)
  * (16) Dutta, A., Roy, S., Kreidl, O.P., Bölöni, L.: Multi-robot information gathering for precision agriculture: Current state, scope, and challenges. IEEE Access 9, 161416–161430 (2021)
  * (17) Pham, T.H., Bestaoui, Y., Mammar, S.: Aerial robot coverage path planning approach with concave obstacles in precision agriculture. In: 2017 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS), pp. 43–48 (2017). IEEE
  * (18) Valente, J., Del Cerro, J., Barrientos, A., Sanz, D.: Aerial coverage optimization in precision agriculture management: A musical harmony inspired approach. Computers and electronics in agriculture 99, 153–159 (2013)
  * (19) Almadhoun, R., Taha, T., Seneviratne, L., Zweiri, Y.: A survey on multi-robot coverage path planning for model reconstruction and mapping. SN Applied Sciences 1(8), 1–24 (2019)
  * (20) Matignon, L., Simonin, O.: Multi-robot simultaneous coverage and mapping of complex scene-comparison of different strategies. In: AAMAS 2018-17th International Conference on Autonomous Agents and Multiagent Systems-Robotics Track, pp. 559–567 (2018). ACM
  * (21) Paull, L., Seto, M., Leonard, J.J., Li, H.: Probabilistic cooperative mobile robot area coverage and its application to autonomous seabed mapping. The International Journal of Robotics Research 37(1), 21–45 (2018)
  * (22) Current, J.R., Schilling, D.A.: The covering salesman problem. Transportation science 23(3), 208–213 (1989)
  * (23) Zelinsky, A., Jarvis, R.A., Byrne, J., Yuta, S., et al.: Planning paths of complete coverage of an unstructured environment by a mobile robot. In: Proceedings of International Conference on Advanced Robotics, vol. 13, pp. 533–538 (1993). Citeseer
  * (24) Gabriely, Y., Rimon, E.: Spiral-stc: An on-line coverage algorithm of grid environments by a mobile robot. In: Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), vol. 1, pp. 954–960 (2002). IEEE
  * (25) Yang, S.X., Luo, C.: A neural network approach to complete coverage path planning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 34(1), 718–724 (2004)
  * (26) Dogru, S., Marques, L.: Eco-cpp: Energy constrained online coverage path planning. Robotics and Autonomous Systems, 104242 (2022)
  * (27) Wei, M., Isler, V.: A log-approximation for coverage path planning with the energy constraint. In: ICAPS, pp. 532–539 (2018)
  * (28) Wei, M., Isler, V.: Coverage path planning under the energy constraint. In: ICRA, pp. 368–373 (2018)
  * (29) Apuroop, K.G.S., Le, A.V., Elara, M.R., Sheu, B.J.: Reinforcement learning-based complete area coverage path planning for a modified htrihex robot. Sensors 21(4), 1067 (2021)
  * (30) Fevgas, G., Lagkas, T., Argyriou, V., Sarigiannidis, P.: Coverage path planning methods focusing on energy efficient and cooperative strategies for unmanned aerial vehicles. Sensors 22(3), 1235 (2022)
  * (31) Kyaw, P.T., Paing, A., Thu, T.T., Mohan, R.E., Le, A.V., Veerajagadheswar, P.: Coverage path planning for decomposition reconfigurable grid-maps using deep reinforcement learning based travelling salesman problem. IEEE Access 8, 225945–225956 (2020)
  * (32) Lakshmanan, A.K., Mohan, R.E., Ramalingam, B., Le, A.V., Veerajagadeshwar, P., Tiwari, K., Ilyas, M.: Complete coverage path planning using reinforcement learning for tetromino based cleaning and maintenance robot. Automation in Construction 112, 103078 (2020)
  * (33) Zhu, D., Tian, C., Sun, B., Luo, C.: Complete coverage path planning of autonomous underwater vehicle based on gbnn algorithm. Journal of Intelligent & Robotic Systems 94(1), 237–249 (2019)
  * (34) Heydari, J., Saha, O., Ganapathy, V.: Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition (2021)
  * (35) Theile, M., Bayerlein, H., Nai, R., Gesbert, D., Caccamo, M.: Uav coverage path planning under varying power constraints using deep reinforcement learning. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1444–1449 (2020). IEEE
  * (36) Sutton, R.S., Barto, A.G., et al.: Introduction to reinforcement learning (1998)
  * (37) Watkins, C.J.C.H.: Learning from delayed rewards (1989)
  * (38) Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. nature 518(7540), 529–533 (2015)
  * (39) Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)
  * (40) Hausknecht, M., Stone, P.: Deep recurrent q-learning for partially observable mdps. In: 2015 Aaai Fall Symposium Series (2015)
  * (41) Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997)
  * (42) Fedus, W., Ramachandran, P., Agarwal, R., Bengio, Y., Larochelle, H., Rowland, M., Dabney, W.: Revisiting fundamentals of experience replay. In: International Conference on Machine Learning, pp. 3061–3071 (2020). PMLR
  * (43) Schaul, T., Quan, J., Antonoglou, I., Silver, D.: Prioritized experience replay. In: Bengio, Y., LeCun, Y. (eds.) 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (2016). http://arxiv.org/abs/1511.05952


††thanks: These authors contributed equally to this work††thanks: These
authors contributed equally to this work

# Ultrabright Sagnac-type source of non-degenerate polarization-entangled
photon pairs using only off-the-shelf optical components

Jakub Szlachetka  Corresponding author<EMAIL_ADDRESS>Institute of Physics, Faculty of Physics, Astronomy and Informatics
Nicolaus Copernicus University, Grudziadzka 5, 87–100 Torun, Poland  Kaushik
Joarder  Institute of Physics, Faculty of Physics, Astronomy and Informatics
Nicolaus Copernicus University, Grudziadzka 5, 87–100 Torun, Poland  Piotr
Kolenderski  Institute of Physics, Faculty of Physics, Astronomy and
Informatics
Nicolaus Copernicus University, Grudziadzka 5, 87–100 Torun, Poland

###### Abstract

We develop a Sagnac-type source of ultrabright, non-degenerate, polarization-
entangled photon pair that is highly stable and compact simultaneously. We use
a $20~{}\text{mm}$ long PPLN bulk crystal which, upon pumping continuously
with $532~{}\text{nm}$ wavelength, produces polarization-entangled photon-
pairs of $785~{}\text{nm}$ signal and $1651~{}\text{nm}$ idler wavelengths via
the type-0 spontaneous parametric down-conversion (SPDC) process. All optical
components used in the setup are off-the-shelf components, readily available
commercially; hence, no custom-designed or costly multi-wavelength
polarization optics are required. At the same time, long-term phase stability
is achieved without any additional active stabilization; due to the geometry
of our Sagnac-type design. We also report one of the highest brightness of
non-degenerate polarization-entangled photon pairs available in the
literature. Even with a very low pump power of $0.034~{}\text{mW}$, we detect
a coincidence rate of $(6.96\pm 0.03)\times 10^{4}$ entangled pairs/sec/mW
(averaged over three polarization-basis measurements: H/V, D/A, and R/L
basis). The source’s brightness is calculated to be $(6.17\pm 0.04)\times
10^{6}$ entangled pairs/sec/mW for the signal line width of $0.4~{}\text{nm}$.
From the raw coincidence counts (without any background coincidence
correction), the fidelity of the entangled state is measured to be $(96.72\pm
0.01)\%$ with a concurrence of $(94.68\pm 0.20)\%$. Bell-CHSH inequality
violation is reported as $S=2.71\pm 0.06$.

##  I Introduction

Quantum entanglement Horodecki  _et al._ (2009) is a purely quantum mechanical
phenomenon where two or more entangled systems show non-classical correlations
which can be exploited to perform various tasks that are impossible in the
realm of classical mechanics, for example, quantum teleportation Bennett  _et
al._ (1993), superdense coding Bennett and Wiesner (1992), quantum key
distribution (QKD)Ekert (1991); Bennett  _et al._ (1992), etc. For the
experimental demonstration of such tasks, highly entangled photon-pair sources
are being developed using various technologies, as well as using different
degrees of freedom for entangling the photon pairs, such as polarization
Steinlechner  _et al._ (2013), temporal Marcikic  _et al._ (2004), spatial
degree of freedom Walborn  _et al._ (2010), etc. Among the different types of
entangled-photon generation processes spontaneous parametric down-conversion
(SPDC) has been widely used Anwar  _et al._ (2021), where the polarizations of
the two photons are most commonly entangled with each other.

In the context of the current demand for technical and research purposes, a
non-degenerate polarization-entangled photon source with high performance has
become a necessary tool. In this source type, entangled photon pairs are
generated in two vastly separated central wavelengths. Typically the signal
wavelength is in the range of $750~{}\text{nm}$ to $850~{}\text{nm}$, and the
idler wavelength is in the range of $1500~{}\text{nm}$ to $1700~{}\text{nm}$.
This type of source is also called a “three-color” source, as all three
operating wavelengths (pump, signal, and idler wavelengths) are significantly
different from each other. The signal wavelength range is suitable for the
highest detection efficiency in typical single-photon avalanche detectors
(SPADs) that are readily available commercially. Also, it belongs to a lower
absorption window in our atmosphere. On the other hand, the idler wavelength
range is well within the optimal transmission window of commercial-grade
optical fibers. Currently available superconducting detectors (SNSPDs) also
provide higher quantum efficiency for this idler-wavelength regime. The
advantage of using non-degenerate entangled-photon pairs is significant in the
long-distance (satellite-to-earth) quantum communication channel (QKD), where
the signal wavelength is most suited for satellite-to-ground-station
communication Yin  _et al._ (2017), and the idler wavelength is efficient for
terrestrial optical-fiber-based communication networks (intra-city and inter-
city links) Valivarthi  _et al._ (2016). Another area of application is
related to quantum memory, where the narrow-band signal photons are suitable
to store information in a quantum memory (trapped Rubidium (Rb) atoms in a
cavity König  _et al._ (2005), Nd:YSO crystal Clausen  _et al._ (2014)), and
the idler photons are ideal candidates for entanglement-distribution between
long-distant nodes connected with optical fibers.

A good source of entangled photons should ideally possess the four general
properties; 1) high performance, 2) low resource requirement, 3) stability and
4) adaptability. High performance can be quantified by measuring various
parameters like entangled-pair-generation rate or brightness of the source,
detected coincidence rate, the fidelity of entanglement, concurrence, purity,
polarization-visibility, etc. Low resource requirement implies a low cost of
production as well as less weight and power consumption. For this criteria, a
compact design of the source setup with a minimal number of optical components
is desirable. Custom-designed optics requiring a cutting-edge technical
solution is costly; hence all the optical components are desired to be off-
the-shelf components easily accessible in a standard optics laboratory. Recent
impetus on satellite-based communication also encourages the design of the
miniaturized version of the source compatible with a nano-satellite (CubeSat),
which puts more constraints on the weight and power consumption. The source
also should be stable for a longer duration, meaning that the performance
should not degrade significantly over time. External factors like temperature,
humidity, and mechanical stress, as well as internal parameters like
robustness and type of interferometer, may affect the stability. To counter
this, an active stabilization mechanism can be implemented within the source
setup. However, this introduces additional components and complexity in the
design. Hence, setup design with in-built stability (for example, Sagnac
interferometric setup Kim  _et al._ (2006); Fedrizzi  _et al._ (2007)) is
highly desirable. The source must be adaptable for various purposes with
minimal modifications. For example, the source should perform in different
pumping conditions (both continuous and pulsed pumping), for various non-
linear mediums (bulk crystals, fibers, waveguides, etc.), and for all type-0,
1, and 2 pair-generation processes. Frequencies (or wavelength) of the
generated entangled pairs should be tunable for a broad range of values.

There have been a few experimental demonstrations of non-degenerate
polarization-entangled photon sources. These sources can be broadly
categorized into three general types, namely, 1) single crystal-single pass,
2) double crystal-single pass, and 3) single crystal-double pass. In the first
category de Chatellus  _et al._ (2006), the pump beam of one polarization
direction (extraordinary or $e$) is passed through a single non-linear medium,
while the specially manufactured crystal allows producing two simultaneous
parametric down-conversion processes
($\left|e\right\rangle_{p}\rightarrow\left|e\right\rangle_{s}\left|e\right\rangle_{i}$
and
$\left|e\right\rangle_{p}\rightarrow\left|o\right\rangle_{s}\left|o\right\rangle_{i}$)
at the same time. A coherent combination of the two processes generates the
entangled state
$(\left|e\right\rangle_{s}\left|e\right\rangle_{i}+\left|o\right\rangle_{s}\left|o\right\rangle_{i})/\sqrt{2}$.
Although this source is simpler, compact, and less resource intensive, it
requires active phase stabilization and specially designed crystal which is
costly and not readily available. Also, the non-collinear pair production
reduces the coupling efficiency resulting in lower performance. The second
type of source uses two separate but nearly-identical crystals to produce
$\left|i\right\rangle_{s}\left|j\right\rangle_{i}$ and
$\left|i^{\perp}\right\rangle_{s}\left|j^{\perp}\right\rangle_{i}$ states in a
coherent way. Here, $\left|i^{\perp}\right\rangle$ represents the orthogonal
polarization state to $\left|i\right\rangle$, and the state
$\left|j\right\rangle$ is either identical or orthogonal to the polarization
state $\left|i\right\rangle$ ($\left|j\right\rangle=\left|i\right\rangle$ or
$\left|j\right\rangle=\left|i^{\perp}\right\rangle$). These two crystals can
be put side by side while the second crystal is rotated $90^{\circ}$ with
respect to the first crystal Pelton  _et al._ (2004); Hübel  _et al._ (2007)
or the two crystals can be put in parallel Clausen  _et al._ (2014). The main
issue with this type of source is to manufacture two identical crystals such
that the generated photon pairs from both crystals are indistinguishable in
all degrees of freedom (spatial, temporal, spectral). Also, the use of two
crystals increases the cost of production. In addition, for the side-by-side
configuration, the crystal’s length is limited to maintain temporal overlap of
the generated photon wave packets, reducing the source’s brightness. On the
other hand, in the parallel configuration, a Mach-Zehnder (MZ) type
interferometer is required for a coherent combination of the two output states
from the two crystals, introducing additional complexity of active phase
stabilization for achieving long-term phase stability. The third type of
source requires only one non-linear crystal, whereas the pump beam excites the
crystal twice to produce the coherent states
$\left|i\right\rangle_{s}\left|j\right\rangle_{i}$ and
$\left|i^{\perp}\right\rangle_{s}\left|j^{\perp}\right\rangle_{i}$. Two
parallel pump beams can be passed simultaneously through the crystal
Fiorentino and Beausoleil (2008), or two pump beams can hit the crystal from
opposite sides along the same line König  _et al._ (2005); Sauge  _et al._
(2008); Lee  _et al._ (2021); Hentschel  _et al._ (2009). In the first
category Fiorentino and Beausoleil (2008), a beam-displacer (BD) based MZ
interferometer has been used for the coherent combination of the two states,
which is very compact in design as well as highly stable as compared to the
conventional MZ interferometer. However, one drawback is the necessary use of
the custom-designed miniaturized waveplates in one of the arms, as the two
parallel pump beams must be very closely separated to pass through a single
crystal with limited width. In the second category, conventional MZ-type
interferometers are used König  _et al._ (2005); Sauge  _et al._ (2008) that
requires active phase stabilization, or a Sagnac-type interferometer is used
Lee  _et al._ (2021); Hentschel  _et al._ (2009) which does not require any
additional stabilization mechanism due to its geometry providing in-built
phase compensation.

To our understanding, a Sagnac-type setup Lee  _et al._ (2021); Hentschel  _et
al._ (2009) is the most promising way to generate non-degenerate entangled
photons. However, a few issues still need to be addressed to achieve all four
properties of a good source, as discussed earlier. In a Sagnac design of the
source, one half-waveplate must be placed in one arm of the interferometer,
which converts horizontal polarization to vertical (and vice versa). If one
places this waveplate inside the Sagnac loop, then the half-waveplate should
work for all three operating wavelengths (pump, signal, and idler
wavelengths), which can only be custom-designed and with limited performance
as per the currently available technology. One may also put the waveplate
outside the Sagnac loop, which then necessitates the use of a miniaturized
half-waveplate. The authors in ref. Lee  _et al._ (2021) bypassed this issue
altogether by using a fiber (PMF) medium instead of a $\chi^{2}$ non-linear
crystal. One end of the fiber was rotated $90^{\circ}$ with respect to the
other end, which enables the use of the half-waveplate completely unnecessary.
However, the use of the four-wave-mixing process in PMF instead of the SPDC
process in a non-linear crystal has its drawbacks, for example, Raman
scattering and high thermal sensitivity of the PMF. Also, the PMF’s high loss
and low coupling efficiency limit the source’s brightness.

In our source design, we use a periodically poled lithium-niobate (PPLN)
crystal which produces entangled photon pairs from the SPDC process, while a
similar Sagnac-type configuration Lee  _et al._ (2021) is implemented. To
counter the half-waveplate issue, we prepare two optical periscopes inside the
Sagnac design, which act as half-waveplates. The periscope is constructed from
two protected silver mirrors that can rotate the input polarization by
$90^{\circ}$; for a broad range of wavelengths ($450~{}\text{nm}$ to
$20~{}\mu\text{m}$). The usage of periscopes instead of half-waveplates inside
the Sagnac setup was introduced earlier in ref.Hentschel  _et al._ (2009).
However, their Sagnac design required the usage of a custom-designed Glan-
Thompson polarizer for compensating angular dispersion. In contrast, our
entangled-photon source uses all off-the-shelf components readily available in
an optical laboratory. No custom-designed or costly optical components are
used. At the same time, our setup design provides long-term phase stability
without any additional active stabilization just by exploiting the inherent
stability of the Sagnac design. We use PPLN crystal for the SPDC-based photon-
pair generation, which can achieve higher brightness at lower pump power, as
compared to other conventional non-linear crystals (such as PPKTP, BBO), due
to its higher $\chi^{2}$ non-linearity coefficient. PPLN allows tuning the
wavelengths of the generated photon pairs for a broad range; by only
controlling the temperature of the crystal. Also, the spectral bandwidth of
the photons is smaller (order of hundreds of GHz) due to the non-degenerate
down-conversion process Steinlechner  _et al._ (2012), which can be modified
to be suited for cascading with atomic systems.

##  II Design considerations

The basic design of our source is represented in Figure 1, which is
conceptually similar to a beam displacer (BD) based MZ-interferometer
Fiorentino and Beausoleil (2008), except for the fact that the non-linear
crystal (C) here is pumped continuously from both directions. Two periscopes
(P1 and P2) act as half-waveplates for all three operating wavelengths.
Periscope P1 works as a half-waveplate whose optic axis is at $45^{\circ}$
angle with respect to the horizontal axis; hence it rotates input horizontal
polarization to vertical and vice versa. On the other side, Periscope P2 works
as a half-waveplate with its optic axis at $0^{\circ}$ angle, such that it
preserves the input horizontal or vertical polarization. The stability of this
design comes from the two Sagnac-type loops where all the pump, signal, and
idler photons are always in contact with the same optical components (except
for the two BDs). So, any external drifts and fluctuations affect the pump,
signal, and idler equally and get compensated by the geometry of the Sagnac
itself. In fact, one may fold back the two Sagnac loops into one single loop
by replacing BD2 with BD1; increasing the stability and reducing the resource
cost while compromising the freedom of independent alignment or adjustment of
the two BDs.

Figure 1:  Conceptual design of the source. BD1, BD2: beam displacers; P1:
periscope 1, which simultaneously converts the horizontally polarized pump
beam to vertically polarized ($H_{p}\rightarrow V_{p}$) and vertically
polarized signal and idler photon pairs to horizontally polarized
($\left|V\right\rangle_{s}\left|V\right\rangle_{i}\rightarrow\left|H\right\rangle_{s}\left|H\right\rangle_{i}$);
P2: periscope 2 which maintains the input polarization of both the pump beam
and the single-photon pairs; C: PPLN crystal for type-0 SPDC process
($\left|V\right\rangle_{p}\rightarrow\left|V\right\rangle_{s}\left|V\right\rangle_{i}$)
in bidirectional continuous pumping condition; F: filter for separating the
signal photons from the idler photons. $\Delta d_{s(i)}$ is the path
separation, and $\Delta\tau_{s(i)}$ is the time delay between the two signal
(idler) photon wave packets coming from bidirectional pumping.

The pump beam (black line in Figure 1) has $+45^{\circ}$ polarization that
gets divided into two parallel beams of equal intensities. The lower beam,
containing horizontal polarization ($H_{p}$), is slightly delayed or advanced
in time as compared to the upper beam containing vertical polarization
($V_{p}$), as the beams traverse different path-lengths and with different
group velocities inside BD1 due to the birefringence property of the beam
displacing material. However, if this time delay is negligibly small compared
to the coherence time of the pumping laser, it only introduces a phase
difference between the two pump beams which can be easily adjusted by
controlling the phase of the input pump polarization. Our PPLN crystal (C)
generates vertically polarized single photon pairs via type-0 SPDC process
only for vertically polarized pump photons,
$\left|V\right\rangle_{p}\rightarrow\left|V\right\rangle_{s}\left|V\right\rangle_{i}$.
Hence, P1 is placed at the lower arm to convert the pump photon polarization
state $\left|H\right\rangle_{p}$ to $\left|V\right\rangle_{p}$. The same
periscope P1 also converts the output signal-idler pair state
$\left|V\right\rangle_{s}\left|V\right\rangle_{i}$ generated from the upper
pump beam into $\left|H\right\rangle_{s}\left|H\right\rangle_{i}$ state. The
path of the signal (idler) photon is represented as the blue (red) dashed
line. After passing P1 and P2, the upper path now has
$\left|V\right\rangle_{s}\left|V\right\rangle_{i}$ two-photon state, and the
lower path has $\left|H\right\rangle_{s}\left|H\right\rangle_{i}$ state, which
are combined again after passing BD2. An interference filter (F) separates
signal photons from the idler photons. In general, the two signal (blue) and
the two idler (red) paths may not converge after BD2, as shown in Fig. 1. This
is because the angular dispersions experience by the signal and idler photons
are different from the pump photon, as the three wavelengths are significantly
separated from each other. This translates to the path separation $\Delta
d_{s(i)}$ and the time-delay $\Delta\tau_{s(i)}$ between the two signal
(idler) wavepackets. Interestingly, one may choose the appropriate lengths and
suitable birefringent materials for BD1 and BD2, such that $\Delta d_{s(i)}$
and $\Delta\tau_{s(i)}$ are negligibly small as compared to the respective
coherence length and coherence time of the signal (idler) wavepacket. In that
scenario, both $\left|V\right\rangle_{s}\left|V\right\rangle_{i}$ and
$\left|H\right\rangle_{s}\left|H\right\rangle_{i}$ states become
indistinguishable in all the degrees of freedom, leading to the entangled
superposition state,
$1/\sqrt{2}(\left|H\right\rangle_{s}\left|H\right\rangle_{i}+\left|V\right\rangle_{s}\left|V\right\rangle_{i})$.
Similarly, we can generate the state
$1/\sqrt{2}(\left|H\right\rangle_{s}\left|H\right\rangle_{i}-\left|V\right\rangle_{s}\left|V\right\rangle_{i})$
by changing the input pump polarization from $+45^{\circ}$ to $-45^{\circ}$.

We use two calcite beam displacers of equal length $30~{}\text{mm}$ as BD1 and
BD2 in our setup, entirely due to their availability at the time of the
experiment. This leads to a $3.3~{}\text{mm}$ parallel separation of the pump
beams after BD1. The time delay between the upper and lower pump beam is
calculated to be $0.91~{}\text{ps}$, much smaller than our pump laser’s
coherence time ($\approx 23~{}ps$). Hence, it can be safely assumed that this
delay only introduces a phase difference between the two bidirectional pumping
beams. Also, the orientation of BD2 allows partial compensation for the time
delay experienced by the pump beams. Here, the pump beam traversing the
extraordinary (ordinary) path in BD1 creates photon pairs that traverse the
ordinary (extraordinary) path in BD2, hence, partially canceling the time
delay. However, due to the significant difference between the pump and the
signal, idler wavelengths, the time delay between the two signal (idler) wave
packets still exists and is calculated to be $\Delta\tau_{s}=0.07~{}\text{ps}$
and $\Delta\tau_{i}=0.17~{}\text{ps}$. Interestingly, for the continuously
pumped SPDC process, photon-pair generation occurs randomly in time due to the
probabilistic nature of the parametric down-conversion. This means that the
difference $\left|\Delta\tau_{s}-\Delta\tau_{i}\right|$ is the determining
factor for qualifying temporal distinguishability rather than
$\Delta\tau_{s(i)}$ themselves. From the calculation,
$\left|\Delta\tau_{s}-\Delta\tau_{i}\right|=0.1~{}\text{ps}$, which is one
order less than the coherence time of our single photons ($\approx
5~{}\text{ps}$), leading to a $99.8\%$ temporal overlap. Hence, the setup does
not require any additional birefringent crystal for temporal wavepacket
matching. The path separation values for the signal and the idler wavepackets
are respectively $\Delta d_{s}=0.12~{}\text{mm}$ and $\Delta
d_{i}=0.33~{}\text{mm}$. Considering the Gaussian beam diameter of
$1.4~{}\text{mm}$ for the signal photon coupling arm and $2.1~{}\text{mm}$ for
the idler photon coupling arm, the spatial mode overlap is calculated to be
$99\%$ and $98\%$ for the signal and idler wavepackets. All the calculated
values shown above are based on the theoretical descriptions shown in Ref. Lee
_et al._ (2021).

Figure 2:  Schematic of the source. PBS: polarizing beamsplitter; HWP: half-
waveplate; QWP: quarter-waveplate; BD1, BD2: calcite beam displacers; F1:
long-pass filter (reflects $532$ nm pump beam, transmits $785$ nm signal and
$1651$ nm idler photons); PM: knife-edge right-angle prism mirror; P1:
periscope that rotates the input horizontal polarization to vertical and vice
versa; P2: periscope that maintains the input polarization; L1, L2: focusing
lens of focal length $f=50~{}\text{mm}$; PPLN: periodically poled lithium
niobate crystal; F2: long pass dichroic mirror (transmits $1651$ nm idler
photons and reflects $785$ nm signal photons); M1, M2: setup for polarization
basis measurements comprising of one QWP, one HWP, and one PBS (see insert
figure); F3: long pass filter ($1500~{}\text{nm}$ cut-on wavelength); F4: band
pass filter ($785~{}\text{nm}$ central wavelength, $3$ nm FWHM band); FC1:
fiber coupler for coupling idler photons; FC2: fiber coupler for coupling
signal photons.

##  III Experimental setup

In our experimental setup (Figure 2), a mode-locked laser (Laser Quantum opus
532) continuously pumps the non-linear crystal (PPLN) at 532 nm wavelength.
The pump beam is coupled to a single-mode fiber and then collimated; to filter
only the fundamental ($\text{TEM}_{0,0}$) Hermite-Gaussian spatial mode. A
polarizing beamsplitter (PBS1) is placed to pass only the horizontally
polarized pump beam. A half-waveplate (HWP1) and a quarter-waveplate (QWP1)
convert the pump polarization into the anti-diagonally ($-45^{\circ}$)
polarized beam. In principle, we can generate a variety of polarization states
by rotating both HWP1 and QWP1. The general polarization state of a pump
photon can be written as
$\alpha\left|H\right\rangle+e^{i\phi}\beta\left|V\right\rangle$, where ideally
$\alpha=\beta=1/\sqrt{2}$, and the phase value is chosen to be
$\phi=180^{\circ}$. However, to compensate for the additional phase and any
asymmetric loss introduced by the Sagnac interferometer (discussed in section
II), we also adjust the value of $\alpha$, $\beta$, and $\phi$, as desired.
The pump beam then passes through the first calcite beam-displacer (BD1) of
$30~{}\text{mm}$ length, where the input beam is split into two separate
parallel beams based on their polarization. The vertically polarized beam
maintains the original beam path, whereas the horizontally polarized beam
shifts parallelly $3.3~{}\text{mm}$ away from the original path. Both the pump
beams are reflected by a long-pass filter, F1 (Thorlabs FELH0750; reflects
$532~{}\text{nm}$ pump beam, transmits $785~{}\text{nm}$ signal and
$1651~{}\text{nm}$ idler photons), and then enter the Sagnac setup. Two pump
beams first get separated by a knife-edge right-angle prism mirror (PM) and
then pass through two separate optical periscopes (P1, P2) constructed from
two protected silver mirrors; before hitting the periodically-poled lithium
niobate (PPLN) crystal. Here, one periscope (P1) rotates the polarization of
the input pump beam by $90^{\circ}$ (horizontal polarization becomes vertical
polarization and vice versa), and the other one (P2) does not change the
polarization but is used only for changing the height of the beam such that
both pump beams have equal heights when they hit the PPLN crystal.

The PPLN crystal is quasi-phase-matched to produce single photon pairs via the
type-0 SPDC process. Our crystal has a dimension of $(0.5~{}\text{mm}\times
0.5~{}\text{mm}\times 20~{}\text{mm})$ and a poling period of
$7.71~{}\mu\text{m}$. To achieve optimal quasi-phase matching, the crystal is
put inside an oven whose temperature is stabilized at $62^{\circ}C$ using a
PID controller. In this condition, a vertically polarized pump photon (532 nm
wavelength) can generate both vertically-polarized signal (785 nm) and idler
(1651 nm) photons in a collinear configuration. To optimize the photon pair
generation rate, both the pump beams are equally focused onto the center of
the PPLN crystal and then collimated again using two identical lenses (L1,
L2).

The generated photon pairs from the horizontally (vertically) polarized pump
beam follow the same path as the vertically (horizontally) polarized input
pump beam inside the Sagnac setup. When they reach the long-pass filter (F1),
the pump beams get reflected again, but the photon pairs pass through it, and
both the photon paths merge into a single path after the second beam-displacer
(BD2) of $30~{}\text{mm}$ length. A long pass dichroic mirror, F2 (Thorlabs
DLMP1180; transmits $1651~{}\text{nm}$ idler photons and reflects
$785~{}\text{nm}$ signal photons) is used to separate the signal photon from
the idler photon and send them to two separate fiber couplers (FC1, FC2)
connected to single-photon detectors. For the signal photons, we use a silicon
avalanche detector (Perkin Elmer SPCM-AQRH-14-FC) with a detection efficiency
of $50\%$, the dark count rate of $100~{}\text{cps}$ and $32~{}\text{ns}$
deadtime. For the idler photon, a superconducting nanowire detector (Scontel
SNSPD) is used, whose efficiency is around $45\%$ for the $1651~{}\text{nm}$
wavelength. Both detectors are connected to a time-tagging module (quTAG)
which performs the time-correlated single photon counting (TCSPC) and
coincidence measurement for a given coincidence window of $3~{}\text{ns}$.

To understand how the entangled two-photon state,
$\left|\Psi^{-}\right\rangle=(\left|HH\right\rangle-\left|VV\right\rangle)/\sqrt{2}$
is generated inside the Sagnac setup; we have to focus on the individual pump
beams separately; first, the horizontally polarized pump beam (H beam) and
then the vertically polarized beam (V beam). The H beam passes through the
periscope P1 and converts to the vertically polarized beam, which upon
interacting with the PPLN crystal, creates both vertically polarized signal
and idler photons
($\left|V\right\rangle_{pump}\rightarrow\left|V\right\rangle_{signal}\left|V\right\rangle_{idler}$).
These photon pairs pass through periscope P2, which maintains their
polarization and only changes the path’s height. These photon pairs maintain
their original path while passing through the beam-displacer BD2. On the other
hand, the V beam passes through the periscope P2, which maintains the
polarization but lowers the beam height. The V beam interacts with the PPLN
crystal and produces pair of single photons with vertical polarization
($\left|V\right\rangle_{pump}\rightarrow\left|V\right\rangle_{signal}\left|V\right\rangle_{idler}$).
Both vertically polarized signal and idler photons are then converted to
horizontally polarized photons
($\left|V\right\rangle_{signal}\left|V\right\rangle_{idler}\rightarrow\left|H\right\rangle_{signal}\left|H\right\rangle_{idler}$)
after passing the periscope P1. These photons follow the exact path of the H
pump beam in a reverse direction. After passing BD2, the path of the photon
pairs merged with the path of the photon pairs from the H pump beam. As the
SPDC process is a probabilistic process with a small probability of photon-
pair generation ($\approx 10^{-10}$ ), it has a negligibly small probability
for both pump beams to create photon pairs simultaneously. Hence, at any time
instance, there is either an
$\left|H\right\rangle_{signal}\left|H\right\rangle_{idler}$ state or
$\left|V\right\rangle_{signal}\left|V\right\rangle_{idler}$ state after BD2.
If the two states are entirely indistinguishable in all other degrees of
freedom (spatial, temporal, spectral, etc.), according to the quantum
mechanical superposition principle, we can write the state as
$(\left|H\right\rangle\left|H\right\rangle\pm\left|V\right\rangle\left|V\right\rangle)/\sqrt{2}$.
The phase of the input polarization state of the pump beam determines the plus
or minus sign.

##  IV Results

We measure the signal and the idler count rate along with the raw coincidence
rate for different pump power values in our Sagnac setup. For this purpose,
six different pump power is chosen; $34~{}\mu\text{W}$, $90~{}\mu\text{W}$,
$200~{}\mu\text{W}$, $520~{}\mu\text{W}$, $1~{}\text{mW}$, and
$2~{}\text{mW}$. The quoted power is the total power sent to the PPLN crystal,
so the pump power on each side of the crystal (H beam or V beam in Figure 2)
is roughly half of the quoted power. All the measurements are performed for
the optimal collection efficiency of the fiber couplers. Figure 3 shows all
the experimental data points along with the numerical fitting to the data.

Our Sagnac setup is creating the entangled state
$(\left|H\right\rangle\left|H\right\rangle-\left|V\right\rangle\left|V\right\rangle)/\sqrt{2}$,
which also can be represented as
$(\left|D\right\rangle\left|A\right\rangle+\left|A\right\rangle\left|D\right\rangle)/\sqrt{2}$
and
$(\left|R\right\rangle\left|R\right\rangle+\left|L\right\rangle\left|L\right\rangle)/\sqrt{2}$
state. Here,
$\left|D(A)\right\rangle=\left|H\right\rangle\pm\left|V\right\rangle$, and
$\left|R(L)\right\rangle=\left|H\right\rangle\pm i\left|V\right\rangle$. If
one considers only one output arm, say the signal arm (or idler arm), the
state can be written as a maximally mixed state
$(\left|H\right\rangle\left\langle H\right|+\left|V\right\rangle\left\langle
V\right|)/2=(\left|D\right\rangle\left\langle
D\right|+\left|A\right\rangle\left\langle
A\right|)/2=(\left|R\right\rangle\left\langle
R\right|+\left|L\right\rangle\left\langle L\right|)/2$. Hence, we measure the
sum of the count rate for both projections $\left|H\right\rangle\left\langle
H\right|$ and $\left|V\right\rangle\left\langle V\right|$ in the signal
(idler) arm. Similarly, we also measure the sum of the count rate for both
$\left|D\right\rangle\left\langle D\right|$ and
$\left|A\right\rangle\left\langle A\right|$ projection, and again for
$\left|R\right\rangle\left\langle R\right|$ and
$\left|L\right\rangle\left\langle L\right|$ projections. The quoted count rate
value in Figure 3-a for both signal and idler is the average value of these
three count rates. Similarly, for the coincidence count rate shown in Figure
3-a, we take an average of the three coincidence rates; the sum of coincidence
rates for both $\left|H\right\rangle\left|H\right\rangle\left\langle
H\right|\left\langle H\right|$ and
$\left|V\right\rangle\left|V\right\rangle\left\langle V\right|\left\langle
V\right|$ projections, the sum of coincidence rates for both
$\left|D\right\rangle\left|A\right\rangle\left\langle D\right|\left\langle
A\right|$ and $\left|A\right\rangle\left|D\right\rangle\left\langle
A\right|\left\langle D\right|$ projections, and the sum of coincidence rates
for both $\left|R\right\rangle\left|R\right\rangle\left\langle
R\right|\left\langle R\right|$ and
$\left|L\right\rangle\left|L\right\rangle\left\langle L\right|\left\langle
L\right|$ projections. Also, the quoted coincidence rate in Figure 3-a is
corrected for the accidental coincidence rate, which is calculated as the
(signal count rate $\times$ idler count rate $\times$ coincidence window).
This accidental coincidence rate is subtracted from the raw coincidence rate
for each data point. Here, we consider a coincidence window of $3~{}\text{ns}$
for every measurement.

Figure 3:  Performance of the source. a) plot of entangled photon-pair
coincidence rate (black circle) and signal (blue circle), idler (red circle)
count rates for different values of pump power. Each data point is the average
value over 10 seconds of collection time and over 3 polarization-basis
measurement settings; rectilinear ($H/V$), diagonal ($D/A$), and circular
($R/L$). The error bar represents the standard deviation of each data point
with the assumption of Poissonian statistics. Plot of b) entangled pair-
generation rate, c) heralding efficiencies for signal and idler arms, and d)
visibility of the polarization-entangled photon pairs against the pump power
value. The blue circle in d represents the measured visibility in the
rectilinear ($H/V$) basis, and the red circle represents the average
visibility measured over 3 separate polarization bases ($H/V$, $D/A$, and
$R/L$). We show second-order polynomial fitting for every dataset in Figures
a, b, and c, whereas exponential fitting is applied for d. To cover the broad
range of the pump power values, logarithmic visualization is applied for both
the x and y-axis of the plot.

We also calculate the heralding efficiencies (Figure 3-c) and the entangled
pair-generation rate (Figure 3-b) from the experimental data. Heralding
efficiency for signal (idler) is calculated using the formula
$H_{s(i)}=C/S_{s(i)}$, where $C$ is the coincidence rate and $S_{s(i)}$ is the
signal (idler) count rate. Entangled pair-generation rate ($PGR$) is
calculated as $PGR=(S_{s}\times S_{i})/C$. The error bar for each data point
shown in Figure 3 is calculated as $\pm\sigma$, where $\sigma=\sqrt{N}$
considering the Poissonian distribution for the photon source. We employ a
second-order polynomial fitting for all the datasets in Figure 3-a, 3-b, 3-c,
which closely matches our experimental data. We observe a slight saturation
effect in the single count rates and the coincidence rate for high pump power
($\approx 2mW$), which is due to the multi-photon pair generation as well as
the saturation effect of the single-photon detectors Steinlechner  _et al._
(2012); according to our speculation.

We measure the polarization-visibility of our entangled photon pairs for
different pump power values. For the measurement of the visibility in ($H/V$)
basis, we measure all four projection values;
$\left|H\right\rangle\left|H\right\rangle\left\langle H\right|\left\langle
H\right|$, $\left|H\right\rangle\left|V\right\rangle\left\langle
H\right|\left\langle V\right|$,
$\left|V\right\rangle\left|H\right\rangle\left\langle V\right|\left\langle
H\right|$, and $\left|V\right\rangle\left|V\right\rangle\left\langle
V\right|\left\langle V\right|$. Polarization-visibility ($PV$) in the ($H/V$)
basis is then calculated as,

$PV_{(H/V)}$=$\frac{C_{\left|H\right\rangle\left|H\right\rangle\left\langle
H\right|\left\langle
H\right|}-C_{\left|H\right\rangle\left|V\right\rangle\left\langle
H\right|\left\langle
V\right|}-C_{\left|V\right\rangle\left|H\right\rangle\left\langle
V\right|\left\langle
H\right|}+C_{\left|V\right\rangle\left|V\right\rangle\left\langle
V\right|\left\langle
V\right|}}{C_{\left|H\right\rangle\left|H\right\rangle\left\langle
H\right|\left\langle
H\right|}+C_{\left|H\right\rangle\left|V\right\rangle\left\langle
H\right|\left\langle
V\right|}+C_{\left|V\right\rangle\left|H\right\rangle\left\langle
V\right|\left\langle
H\right|}+C_{\left|V\right\rangle\left|V\right\rangle\left\langle
V\right|\left\langle V\right|}}$

Here, $C_{\left|i\right\rangle\left|j\right\rangle\left\langle
i\right|\left\langle j\right|}$ is the coincidence count rate for the
projection measurement $\left|i\right\rangle\left|j\right\rangle\left\langle
i\right|\left\langle j\right|$. We plot the behavior of $PV_{(H/V)}$ with
increasing pump power in Figure 3-d (blue-colored data points). As one can
observe, the visibility value slowly drops down with higher pump power. We
correlate this behavior to the higher multi-photon generation probability at
high pumping power, also observed in Schneeloch  _et al._ (2019); Steinlechner
_et al._ (2012). In addition to the pump power, the maximum achievable
visibility is also limited by the size of the coincidence window and other
experimental imperfections, such as the polarization extinction ratios of the
polarization analyzers, the alignment of the Sagnac setup and its phase
stability, coupling efficiency of the fiber couplers, etc. A lower value of
the coincidence window enhances the visibility value. However, the selection
of a suitable coincidence window size is also limited by the timing jitter of
the single-photon detectors.

We also plot the average visibility value in Figure 3-d (red colored data
points), which is the average of the three visibility values measured in three
conjugate bases; $PV_{H/V}$, $PV_{D/A}$, and $PV_{R/L}$. We observe a slight
lowering of the average visibility as compared to $PV_{H/V}$. For example, at
$0.034~{}\mu\text{W}$ pump power, experimentally measured $PV_{H/V}$ value is
$97.29\%$ while the average visibility is $95.88\%$. This lowering in the
average visibility also qualifies the degree of entanglement between the
photon pairs. For an ideal experimental condition, maximal entanglement should
produce equal visibility on any orthogonal polarization basis. However, the
high value of the average visibility from our source signifies a good quality
of entanglement.

As one may observe, the brightness of our non-degenerate entangled photon
source is very high. This is partly due to the usage of PPLN crystal as the
non-linear medium, which has a high pair-generation rate due to its large non-
linear ($\chi^{2}$) coefficient. We detect entangled photon pairs of high
brightness even at a very low pump power of $34~{}\mu\text{W}$. At this pump
power value, we measure the average coincidence rate of $(2.37\pm 0.05)\times
10^{3}$ cps. Hence, we report our average detected coincidence rate as
$(6.96\pm 0.03)\times 10^{4}$ cps/mW. The entangled pair generation rate (PGR)
is calculated to be $(6.17\pm 0.04)\times 10^{6}$ cps/mW, using the formula,
$PGR=(S_{s}\times S_{i})/C$, as mentioned before. We measure the polarization
correlation between the signal and idler photons at this low pump power value.
For this purpose, we place a polarization basis measurement setup consisting
of a half-waveplate and a polarizing beamsplitter (PBS), in the two output
arms of the source (see Figure 2). By rotating the half-waveplate angles from
$0^{\circ}$ to $180^{\circ}$, we can project the whole $360^{\circ}$
polarization direction in the transmitting arm of the PBS. In our polarization
correlation measurement, we fix the projection on the signal photons to four
different directions; horizontal (H, $0^{\circ}$), vertical (V, $90^{\circ}$),
diagonal (D, $45^{\circ}$), and anti-diagonal (A, $135^{\circ}$). For each
fixed projection measurement on the signal photons, the projection angle on
the idler photons is continuously rotated from $0^{\circ}$ to $360^{\circ}$
with the interval of $12^{\circ}$, and the average raw coincidence count rate
is measured for 10 seconds of data collection. We observe a sinusoidal pattern
in the raw coincidence count rate for all four fixed projection angles on the
signal photons, with high visibility, $V=(\text{Max coincidence}-\text{Min
coincidence})/(\text{Max coincidence}+\text{Min coincidence})$. We fit the
experimental data with the sinusoidal curve. From the fitted curves, we
calculate the visibility as $96.49\pm 00.76\%$ (H), $95.26\pm 00.89\%$ (V),
$96.01\pm 00.81\%$ (D), and $96.27\pm 00.79\%$ (A). We also measure the Bell-
CHSH inequality value, $S=\left\langle
AB^{\phantom{{}^{\prime}}}\right\rangle-\left\langle
AB^{{}^{\prime}}\right\rangle-\left\langle
A^{{}^{\prime}}B\right\rangle-\left\langle
A^{{}^{\prime}}B^{{}^{\prime}}\right\rangle$, where $A$ and $A^{{}^{\prime}}$
are the two projection basis measurement on the signal photons with
polarization angle $0^{\circ}$ and $45^{\circ}$ respectively, and $B$,
$B^{{}^{\prime}}$ are the two projection basis measurement on the idler
photons with polarization angle $22.5^{\circ}$ and $67.5^{\circ}$ respectively
Clauser  _et al._ (1969). We observe a Bell-CHSH value of $S=2.71\pm 0.06$,
which violates the local-real bound of $S\leq 2$, by $11.8$ standard
deviation.

Figure 4: The degree of polarization-entanglement is shown by plotting the raw
coincidence counts while the signal polarization basis is fixed and the idler
polarization analyzer is rotated by a full $2\pi$ rotation. The polarization
measurement basis of the signal photon is fixed for 4 directions; horizontal
(H, blue circle), vertical (V, red circle), diagonal (D, green circle), and
anti-diagonal (A, pink circle). The colored circles are the experimental raw
coincidence count rates (without any background correction) averaged over 10
seconds of data collection. Error bars are the standard deviations calculated
with the assumption of Poissonian statistics. Sinusoidal fitting is applied on
the 4 datasets; for the lowest chi-squared value.

We measure the tomography of the generated photon-pair state; to quantify the
degree of entanglement as well as the closeness (fidelity) of the output state
to the desired entangled state,
$\left|\Phi^{-}\right\rangle=(\left|H\right\rangle\left|H\right\rangle-\left|V\right\rangle\left|V\right\rangle)/\sqrt{2}$.
For this purpose, we implement the tomography setup in the two output arms,
shown as projective measurements M1 and M2 in Figure 2. We measure the
coincidence rate for 16 projective measurements; $H_{s}H_{i}$, $H_{s}V_{i}$,
$V_{s}V_{i}$, $V_{s}H_{i}$, $L_{s}H_{i}$, $L_{s}V_{i}$, $D_{s}V_{i}$,
$D_{s}H_{i}$, $D_{s}L_{i}$, $D_{s}D_{i}$, $L_{s}D_{i}$, $H_{s}D_{i}$,
$V_{s}D_{i}$, $V_{s}R_{i}$, $L_{s}D_{i}$, $H_{s}R_{i}$ projections. Here,
$X_{s}Y_{i}$ represents the projective operation
$\left|X\right\rangle_{s}\left|Y\right\rangle_{i}\left\langle
X\right|_{s}\left\langle Y\right|_{i}$. For each projective measurement, the
coincidence rate is measured as an average of data collected over 10 seconds,
while the pump power is only $34~{}\mu W$. From these datasets, the density
matrix of the photon-pair state is reconstructed using the maximum likelihood
estimation (MLE) process James  _et al._ (2001). We find that for our
reconstructed density matrix, the concurrence is $94.68\pm 0.20$, purity of
the state is $94.40\pm 0.25$, and the fidelity is $96.72\pm 0.01$ concerning
the $\left|\Phi^{-}\right\rangle$ state. Fig. 5 shows the real and the
imaginary part of the reconstructed two-photon density matrix.

Figure 5:  The real part (left) and the imaginary part (right) of the
reconstructed density matrix, as obtained from the maximum likelihood
estimation (MLE) process.

##  V Conclusion

We developed a Sagnac-type source for non-degenerate polarization-entangled
photon pairs. The signal photon wavelength ($785~{}\text{nm}$) is vastly
different from the idler photon wavelength ($1651~{}\text{nm}$). Our source
satisfies all the required conditions of high performance, low resource
requirement, stability, and adaptability. We achieved a significantly high
brightness of $(6.17\pm 0.04)\times 10^{6}$ entangled pairs/sec/mW, whereas
the fidelity is $(96.72\pm 0.01)\%$ with respect to the desired two-photon
entangled state,
$\left|\Psi^{-}\right\rangle=(\left|HH\right\rangle-\left|VV\right\rangle)/\sqrt{2}$.
We used only off-the-shelf optical components, and the source design is highly
compact, which reduces the total cost and weight. The Sagnac-type design helps
attain long-term phase stability without using any active phase stabilization
method, further improving the resource requirements. Although we demonstrated
the performance of our source with continuous wave (CW) pumping, the source
can perform equally with pulsed pumping conditions. In that scenario, one has
to include two compensating birefringent crystals, one before and one after
the Sagnac loop, to compensate for temporal mismatch. Our source design can be
adapted for type-1 and type-2 phase-matching conditions with minimal
modification (only one additional half-waveplate or another beam displacer is
required Horn and Jennewein (2019)). One may also miniaturize the whole setup
and fit it inside a smaller dimension, further increasing the stability. For
this purpose, one may remove the two focusing lenses (L1, L2) and place them
outside the Sagnac loop.

Our non-degenerate polarization-entangled photon-pair source with high
brightness and fidelity is ideal for application in satellite-to-ground-based
QKD links. Also, using PPLN crystal for the non-degenerate SPDC process
reduces the line width of the generated photon pairs, as compared to the
conventional BBO crystal. With additional cavity-based spectral filtering, the
line width of the signal photons can be reduced even further to match the
atomic transition line width. In that scenario, our source can be used for a
quantum-memory-based communication channel. Ultrabright photon pair sources
such as the one described here can perform even with tight spectral filtering.
The performance of the source can be further modified with some additional
changes. One may use $\alpha$-BBO beam displacing crystals instead of calcite
to reduce further the temporal and spatial distinguishability between the two
wavepackets coming from the bidirectional pumping. Also, different lengths of
beam displacing crystals (BD1, BD2) can further reduce $\Delta d_{s(i)}$ and
$\Delta\tau_{s(i)}$ values, resulting in higher polarization visibility.

Funding Information. The authors acknowledge financial support by the
Foundation for Polish Science (FNP) (project First Team co-financed by the
European Union under the European Regional Development Fund, grant no. First
Team/2017-3/20).

Disclosures. The authors declare no conflicts of interest.

## References

  * Horodecki  _et al._ (2009) R. Horodecki, P. Horodecki, M. Horodecki, and K. Horodecki, Reviews of modern physics 81, 865 (2009).
  * Bennett  _et al._ (1993) C. H. Bennett, G. Brassard, C. Crépeau, R. Jozsa, A. Peres, and W. K. Wootters, Phys. Rev. Lett. 70, 1895 (1993).
  * Bennett and Wiesner (1992) C. H. Bennett and S. J. Wiesner, Phys. Rev. Lett. 69, 2881 (1992).
  * Ekert (1991) A. K. Ekert, Phys. Rev. Lett. 67, 661 (1991).
  * Bennett  _et al._ (1992) C. H. Bennett, G. Brassard, and N. D. Mermin, Phys. Rev. Lett. 68, 557 (1992).
  * Steinlechner  _et al._ (2013) F. Steinlechner, S. Ramelow, M. Jofre, M. Gilaberte, T. Jennewein, J. P. Torres, M. W. Mitchell, and V. Pruneri, Optics express 21, 11943 (2013).
  * Marcikic  _et al._ (2004) I. Marcikic, H. De Riedmatten, W. Tittel, H. Zbinden, M. Legré, and N. Gisin, Physical Review Letters 93, 180502 (2004).
  * Walborn  _et al._ (2010) S. P. Walborn, C. Monken, S. Pádua, and P. S. Ribeiro, Physics Reports 495, 87 (2010).
  * Anwar  _et al._ (2021) A. Anwar, C. Perumangatt, F. Steinlechner, T. Jennewein, and A. Ling, Review of Scientific Instruments 92, 041101 (2021), https://doi.org/10.1063/5.0023103 .
  * Yin  _et al._ (2017) J. Yin, Y. Cao, Y.-H. Li, S.-K. Liao, L. Zhang, J.-G. Ren, W.-Q. Cai, W.-Y. Liu, B. Li, H. Dai, _et al._ , Science 356, 1140 (2017).
  * Valivarthi  _et al._ (2016) R. Valivarthi, Q. Zhou, G. H. Aguilar, V. B. Verma, F. Marsili, M. D. Shaw, S. W. Nam, D. Oblak, W. Tittel, _et al._ , Nature Photonics 10, 676 (2016).
  * König  _et al._ (2005) F. König, E. J. Mason, F. N. C. Wong, and M. A. Albota, Phys. Rev. A 71, 033805 (2005).
  * Clausen  _et al._ (2014) C. Clausen, F. Bussières, A. Tiranov, H. Herrmann, C. Silberhorn, W. Sohler, M. Afzelius, and N. Gisin, New Journal of Physics 16, 093058 (2014).
  * Kim  _et al._ (2006) T. Kim, M. Fiorentino, and F. N. C. Wong, Phys. Rev. A 73, 012316 (2006).
  * Fedrizzi  _et al._ (2007) A. Fedrizzi, T. Herbst, A. Poppe, T. Jennewein, and A. Zeilinger, Optics Express 15, 15377 (2007).
  * de Chatellus  _et al._ (2006) H. G. de Chatellus, A. V. Sergienko, B. E. A. Saleh, M. C. Teich, and G. D. Giuseppe, Opt. Express 14, 10060 (2006).
  * Pelton  _et al._ (2004) M. Pelton, P. Marsden, D. Ljunggren, M. Tengner, A. Karlsson, A. Fragemann, C. Canalias, and F. Laurell, Opt. Express 12, 3573 (2004).
  * Hübel  _et al._ (2007) H. Hübel, M. R. Vanner, T. Lederer, B. Blauensteiner, T. Lorünser, A. Poppe, and A. Zeilinger, Opt. Express 15, 7853 (2007).
  * Fiorentino and Beausoleil (2008) M. Fiorentino and R. G. Beausoleil, Opt. Express 16, 20149 (2008).
  * Sauge  _et al._ (2008) S. Sauge, M. Swillo, M. Tengner, and A. Karlsson, Opt. Express 16, 9701 (2008).
  * Lee  _et al._ (2021) Y. S. Lee, M. Xie, R. Tannous, and T. Jennewein, Quantum Science and Technology 6, 025004 (2021).
  * Hentschel  _et al._ (2009) M. Hentschel, H. Hübel, A. Poppe, and A. Zeilinger, Opt. Express 17, 23153 (2009).
  * Steinlechner  _et al._ (2012) F. Steinlechner, P. Trojek, M. Jofre, H. Weier, D. Perez, T. Jennewein, R. Ursin, J. Rarity, M. W. Mitchell, J. P. Torres, _et al._ , Optics express 20, 9640 (2012).
  * Schneeloch  _et al._ (2019) J. Schneeloch, S. H. Knarr, D. F. Bogorin, M. L. Levangie, C. C. Tison, R. Frank, G. A. Howland, M. L. Fanto, and P. M. Alsing, Journal of Optics 21, 043501 (2019).
  * Clauser  _et al._ (1969) J. F. Clauser, M. A. Horne, A. Shimony, and R. A. Holt, Physical review letters 23, 880 (1969).
  * James  _et al._ (2001) D. F. James, P. G. Kwiat, W. J. Munro, and A. G. White, Physical Review A 64, 052312 (2001).
  * Horn and Jennewein (2019) R. Horn and T. Jennewein, Opt. Express 27, 17369 (2019).