#  Differentiable Particle Filters:
End-to-End Learning with Algorithmic Priors

Rico Jonschkowski, Divyam Rastogi, and Oliver Brock  Robotics and Biology
Laboratory, Technische Universität Berlin, Germany

###### Abstract

We present differentiable particle filters (DPFs): a differentiable
implementation of the particle filter algorithm with learnable motion and
measurement models. Since DPFs are end-to-end differentiable, we can
efficiently train their models by optimizing end-to-end state estimation
performance, rather than proxy objectives such as model accuracy. DPFs encode
the structure of recursive state estimation with prediction and measurement
update that operate on a probability distribution over states. This structure
represents an algorithmic prior that improves learning performance in state
estimation problems while enabling explainability of the learned model. Our
experiments on simulated and real data show substantial benefits from end-to-
end learning with algorithmic priors, e.g. reducing error rates by $\sim$80%.
Our experiments also show that, unlike long short-term memory networks, DPFs
learn localization in a policy-agnostic way and thus greatly improve
generalization. Source code is available at https://github.com/tu-
rbo/differentiable-particle-filters.

##  I Introduction

End-to-end learning tunes all parts of a learnable system for end-to-end
performance—which is what we ultimately care about—instead of optimizing each
part individually. End-to-end learning excels when the right objectives for
individual parts are not known; it therefore has significant potential in the
context of complex robotic systems.

Compared to learning each part of a system individually, end-to-end learning
puts fewer constraints on the individual parts, which can improve performance
but can also lead to overfitting. We must therefore balance end-to-end
learning with regularization by incorporating appropriate priors. Priors can
be encoded in the form of differentiable network architectures. By defining
the network architecture and its learnable parameters, we restrict the
hypothesis space and thus regularize learning. At the same time, the
differentiability of the network allows all of its parts to adapt to each
other and to optimize their parameters for end-to-end performance.

This approach has been very successful in computer vision. Highly engineered
vision pipelines are outperformed by convolutional networks trained end-to-end
[8]. But it only works because convolutional networks [15] encode priors in
the network architecture that are suitable for computer vision—a hierarchy of
local filters shared across the image. Problems in robotics possess additional
structure, for example in physical interactions with the environment. Only by
exploiting all available structure will we be able to realize the full
potential of end-to-end learning in robotics.

_But how can we find more architectures like the convolutional network for
robotics?_ Roboticists have captured problem structure in the form of
algorithms, often combined with models of the specific task. By making these
algorithms differentiable and their models learnable, we can turn robotic
algorithms into network architectures. This approach enables end-to-end
learning while also encoding prior knowledge from algorithms, which we call
_algorithmic priors_.

Here, we apply _end-to-end learning with algorithmic priors_ to state
estimation in robotics. In this problem, a robot needs to infer the latent
state from its observations and actions. Since a single observation can be
insufficient to estimate the state, the robot needs to integrate uncertain
information over time.

Given the standard assumptions for this problem, _Bayes filters_ provide the
provably optimal algorithmic structure for solving it [21], recursively
updating a probability distribution over states with prediction and
measurement update using task-specific motion and measurement models. The
_differentiable particle filter_ (DPF) is an end-to-end differentiable
implementation of the particle filter—a Bayes filter that represents
probability distributions with samples—with learnable motion and measurement
models (see Fig. 1).

Figure 1: Differentiable particle filters. Models can be learned end-to-end by
backpropagation through the algorithm.

Since DPFs are differentiable, we can learn their models end-to-end to
optimize state estimation performance. Our experiments show that end-to-end
learning improves performance compared to using models optimized for accuracy.
Interestingly, end-to-end learning in DPFs re-discovers what roboticists found
out via trial and error: that overestimating uncertainty is beneficial for
filtering performance [21, p. 118].

Since DPFs use the Bayes filter algorithm as a prior, they have a number of
advantages. First, even with end-to-end learning, DPFs remain explainable—we
can examine the learned models and their interaction. Second, the algorithmic
prior regularizes learning, which greatly improves performance in state
estimation. Compared to generic long short-term memory networks (LSTMs) [9],
DPFs reduce the error rate by $\sim$80% or require 87% less training data for
the same error rate. And finally, the algorithmic prior improves
generalization: while LSTMs fail when tested with a different policy than used
for training, DPFs are robust to changing the policy.

##  II Related Work

There is a surge of recent work that combines algorithmic priors and end-to-
end learning for planning and state estimation with histogram-based and
Gaussian belief representations.

##### Planning with known state

Tamar et al. [20] introduced value iteration networks, a differentiable
planning algorithm with models that can be optimized for value iteration.
Their key insight is that value iteration in a grid based state space can be
represented by convolutional neural networks. Silver et al. [18] proposed the
predictron, a differentiable embedding of the TD($\lambda$) algorithm in a
learned state space. Okada and Aoshima [16] proposed path integral networks,
which encode an optimal control algorithm to learn continuous tasks.

##### State estimation (and planning) with histograms

Jonschkowski and Brock [10] introduced the end-to-end learnable histogram
filter, a differentiable Bayes filter that represents the belief with a
histogram. Shankar et al. [17] and Karkus et al. [11] combined histogram
filters and QMDP planners in a differentiable network for planning in
partially observable environments. Gupta et al. [6] combined differentiable
mapping and planning in a network architecture for navigation in novel
environments. All of these approaches use convolution to operate on a grid
based state space.

##### State estimation with Gaussians

Harnooja et al. [7] presented a differentiable Kalman filter with a Gaussian
belief and an end-to-end learnable measurement model from visual input. Watter
et al. [22] and Karl et al. [12] learn a latent state space that facilitates
prediction. These approaches use (locally) linear dynamics models and Gaussian
beliefs.

Related work has established how to operate on _histogram-based_ belief
representations using convolution and how to work with _Gaussian_ beliefs
using linear operations. We build on this work and extend its scope to include
_sample-based_ algorithms, such as particle filters. Sample-based
representations can be advantageous because they can represent multi-modal
distributions (unlike Gaussians) while focusing the computational effort on
states of high probability (unlike histograms). But sample-based
representations introduce new challenges for differentiable implementations,
e.g. generating samples from networks, performing density estimation to
compute gradients, and handling non-differentiable resampling. These are the
challenges that we tackle in this paper.

##  III Background: Bayes Filters and Their Particle-Based Approximation

We consider the problem of estimating a latent _state_ $\boldsymbol{s}$ from a
history of _observations_ $\boldsymbol{o}$ and _actions_ $\boldsymbol{a}$,
e.g. a robot’s pose from camera images and odometry. To handle uncertainty, we
estimate a probability distribution over the current state
$\boldsymbol{s}_{t}$ conditioned on the history of observations
$\boldsymbol{o}_{1:t}$ and actions $\boldsymbol{a}_{1:t}$, which is called
_belief_ ,
$\text{bel}(\boldsymbol{s}_{t})=p(\boldsymbol{s}_{t}|\boldsymbol{a}_{1:t},\boldsymbol{o}_{1:t}).$

###  III-A Bayes Filters

Figure 2: Graphical model for state estimation

If we assume that our problem factorizes as shown in Fig. 2, the _Bayes
filter_ algorithm solves it optimally [21] by making use of the Markov
property of the state and the conditional independence of observations and
actions. From the Markov property follows that the last belief
$\text{bel}(\boldsymbol{s}_{t-1})$ summarizes all information contained in the
history of observations $\boldsymbol{o}_{1:t-1}$ and actions
$\boldsymbol{a}_{1:t-1}$ that is relevant for predicting the future.
Accordingly, the Bayes filter computes $\text{bel}(\boldsymbol{s}_{t})$
recursively from $\text{bel}(\boldsymbol{s}_{t-1})$ by incorporating the new
information contained in $\boldsymbol{a}_{t}$ and $\boldsymbol{o}_{t}$. From
assuming conditional independence between actions and observations given the
state follows that Bayes filters update the belief in two steps: 1)
_prediction_ using action $\boldsymbol{a}_{t}$ and 2) _measurement update_
using observation $\boldsymbol{o}_{t}$.

1) The _prediction_ step is based on the _motion model_
$p(\boldsymbol{s}_{t}\mid\boldsymbol{s}_{t-1},\boldsymbol{a}_{t})$, which
defines how likely the robot enters state $\boldsymbol{s}_{t}$ if it performs
action $\boldsymbol{a}_{t}$ in $\boldsymbol{s}_{t-1}$. Using the motion model,
this step computes the _predicted belief_
$\overline{\text{bel}}(\boldsymbol{s}_{t})$ by summing over all
$\boldsymbol{s}_{t-1}$ from which $\boldsymbol{a}_{t}$ could have led to
$\boldsymbol{s}_{t}$.

$\displaystyle\overline{\text{bel}}(\boldsymbol{s}_{t})$ $\displaystyle=\int
p(\boldsymbol{s}_{t}\mid\boldsymbol{s}_{t-1},\boldsymbol{a}_{t})\,\text{bel}(\boldsymbol{s}_{t-1})\,d\boldsymbol{s}_{t-1}.$
(1)

2) The _measurement update_ uses the _measurement model_
$p(\boldsymbol{o}_{t}\mid\boldsymbol{s}_{t})$, which defines the likelihood of
an observation $\boldsymbol{o}_{t}$ given a state $\boldsymbol{s}_{t}$. Using
this model and observation $o_{t}$, this step updates the belief using Bayes’
rule (with normalization $\eta$),

$\displaystyle\text{bel}(\boldsymbol{s}_{t})=\eta\,p(\boldsymbol{o}_{t}\mid\boldsymbol{s}_{t})\,\overline{\text{bel}}(\boldsymbol{s}_{t}).$
(2)

Any implementation of the Bayes filter algorithm for a continuous state space
must represent a continuous belief–and thereby approximate it. Different
approximations correspond to different Bayes filter implementations, for
example histogram filters, which represent the belief by a histogram, Kalman
filters, which represent it by a Gaussian, or particle filters, which
represent the belief by a set of particles [21].

(a) Prediction and measurement update; boxes represent models, colored boxes
are learned  (b) Computing the gradient for end-to-end learning requires
density estimation from the predicted particles (gray circles, darkness
corresponds to particle weight). After converting the particles into a mixture
of Gaussians (blue), we can compute the belief at the true state (orange bar
at red x) and maximize it.

Figure 3: DPF overview. Models in (a) can be learned end-to-end by maximizing
the belief of the true state (b).

###  III-B Particle Filters

Particle filters approximate the belief with particles (or samples)
$\mathcal{S}_{t}=\boldsymbol{s}^{[1]}_{t},\boldsymbol{s}^{[2]}_{t},\dots,\boldsymbol{s}^{[n]}_{t}$
with weights $w^{[1]}_{t},w^{[2]}_{t},\dots,w^{[n]}_{t}$. The particle filter
updates this distribution by moving particles, changing their weights, and
resampling them, which duplicates or removes particles proportionally to their
weight. Resampling makes this Bayes filter implementation efficient by
focusing the belief approximation on probable states.

The particle filter implements the prediction step (Eq. 1) by moving each
particle stochastically, which is achieved by sampling from a generative
motion model,

$\displaystyle\forall_{i}:\;\;\boldsymbol{s}^{[i]}_{t}\sim
p(\boldsymbol{s}_{t}\mid\boldsymbol{a}_{t},\boldsymbol{s}^{[i]}_{t-1}).$ (3)

The particle filter implements the measurement update (Eq. 2) by setting the
weight of each particle to the observation likelihood—the probability of the
current observation conditioned on the state represented by the particle,

$\displaystyle\forall_{i}:\;\;w^{[i]}_{t}=p(\boldsymbol{o}_{t}\mid\boldsymbol{s}^{[i]}_{t}).$
(4)

The particle set is then resampled by randomly drawing particles
$\boldsymbol{s}^{[i]}_{t}$ proportionally to their weight $w^{[i]}_{t}$ before
the filter performs the next iteration of prediction and update.

##  IV Differentiable Particle Filters

Differentiable particle filters (DPFs) are a differentiable implementation of
the particle filter algorithm with end-to-end learnable models. We can also
view DPFs as a new recurrent network architecture that encodes the algorithmic
prior from particle filters in the network structure (see Fig. 3a).

With end-to-end learning, we do not mean that every part of a system is
learned but that the objective for the learnable parts is end-to-end
performance. For efficient end-to-end learning in particle filters, we need
learnable models and the ability to backpropagate the gradient through the
particle filter algorithm—not to change the algorithm but to compute how to
change the models to improve the algorithm’s output.

This section describes our DPF implementation. Our source code based on
TensorFlow [1] and Sonnet [4] is available at https://github.com/tu-
rbo/differentiable-particle-filters.

###  IV-A Belief

DPFs represent the belief at time $t$ by a set of weighted particles,
$\text{bel}(\boldsymbol{s}_{t})=(S_{t},\boldsymbol{w}_{t})$, where
$S\in\mathbb{R}^{n\times d}$ describes $n$ particles in $d$-dimensional state
space with weights $\boldsymbol{w}\in\mathbb{R}^{n}$. At every time step, DPFs
update the previous belief $\text{bel}(\boldsymbol{s}_{t-1})$ with action
$\boldsymbol{a}_{t}$ and observation $\boldsymbol{o}_{t}$ to get
$\text{bel}(\boldsymbol{s}_{t})$ (see Fig. 3a).

###  IV-B Prediction

The prediction step moves each particle by sampling from a probabilistic
motion model (Eq. 3). Motion models often assume deterministic environments;
they account for uncertainty by generating noisy versions of the commanded or
measured action such that a different version of the action is applied to each
particle [21, chap. 5]. We follow the same approach by splitting the motion
model into an _action sampler_ $f$, which creates a noisy action
$\hat{\boldsymbol{a}}^{[i]}$ per particle, and a _dynamics model_ $g$, which
moves each particle $i$ according to $\hat{\boldsymbol{a}}^{[i]}$.

$\displaystyle\hat{\boldsymbol{a}}^{[i]}_{t}$
$\displaystyle=\boldsymbol{a}_{t}+f_{\boldsymbol{\theta}}(\boldsymbol{a}_{t},\boldsymbol{\epsilon}^{[i]}\sim\mathcal{N}),$
(5) $\displaystyle\boldsymbol{s}^{[i]}_{t}$
$\displaystyle=\boldsymbol{s}^{[i]}_{t-1}+g(\boldsymbol{s}^{[i]}_{t-1},\hat{\boldsymbol{a}}^{[i]}_{t}),$
(6)

where $f_{\boldsymbol{\theta}}$ is a feedforward network (see Table I),
$\boldsymbol{\theta}$ are all parameters of the DPF, and
$\boldsymbol{\epsilon}^{[i]}\in\mathbb{R}^{d}$ is a noise vector drawn from a
standard normal distribution. Using the noise vector as input for a learnable
generative model is known as the reparameterization trick [14]. Here, this
trick enables $f_{\boldsymbol{\theta}}$ to learn to sample from action-
dependent motion noise. The resulting noisy actions are fed into $g$, which
simulates how these actions change the state. Since we often know the
underlying dynamics model, we can implement its equations in $g$.
Alternatively, we can replace $g$ by a feedforward network
$g_{\boldsymbol{\theta}}$ and learn the dynamics from data (tested in Section
V-A3).

###  IV-C Measurement Update

The measurement update uses the observation to compute particle weights (Eq.
4). DPFs implement this update and additionally use the observation to propose
new particles (see Fig. 3a). The DPF measurement model consists of three
components: a shared _observation encoder_ $h$, which encodes an observation
$\boldsymbol{o}_{t}$ into a vector $\boldsymbol{e}_{t}$, a _particle proposer_
$k$, which generates new particles, and an _observation likelihood estimator_
$l$, which weights each particle based on the observation.

$\displaystyle\boldsymbol{e}_{t}$
$\displaystyle=h_{\boldsymbol{\theta}}(\boldsymbol{o}_{t}),$ (7)
$\displaystyle\boldsymbol{s}^{[i]}_{t}$
$\displaystyle=k_{\boldsymbol{\theta}}(\boldsymbol{e}_{t},\boldsymbol{\delta}^{[i]}\sim
B),$ (8) $\displaystyle w^{[i]}_{t}$
$\displaystyle=l_{\boldsymbol{\theta}}(\boldsymbol{e}_{t},\boldsymbol{s}^{[i]}_{t}),$
(9)

where $h_{\boldsymbol{\theta}}$, $k_{\boldsymbol{\theta}}$, and
$l_{\boldsymbol{\theta}}$ are feedforward networks based on parameters
$\boldsymbol{\theta}$; the input $\boldsymbol{\delta}^{[i]}$ is a dropout
vector sampled from a Bernoulli distribution. Here, dropout is not used for
regularization but as a source of randomness for sampling different particles
from the same encoding $\boldsymbol{e}_{t}$ (see Table I).

###  IV-D Particle Proposal and Resampling

We do _not_ initialize DPFs by uniformly sampling the state space—this would
produce too few initial particles near the true state. Instead, we initialize
DPFs by proposing particles from the current observation (as described above)
for the first steps. During filtering, DPFs move gradually from particle
proposal, which generates hypotheses, to resampling, which tracks and weeds
out these hypotheses. The ratio of proposed to resampled particles follows an
exponential function $\gamma^{t-1}$, where $\gamma$ is a hyperparameter set to
$0.7$ in our experiments. We use 1000 particles for testing and 100 particles
for training (to speed up the training process). DPFs implement resampling by
stochastic universal sampling [2], which is not differentiable and leads to
limitations discussed in Section IV-F.

###  IV-E Supervised Learning

DPF models can be learned from sequences of supervised data
$\boldsymbol{o}_{1:T}$, $\boldsymbol{a}_{1:T}$, $\boldsymbol{s}^{*}_{1:T}$
using maximum likelihood estimation by maximizing the belief at the _true
state_ $\boldsymbol{s}^{*}_{t}$. To estimate
$\text{bel}(\boldsymbol{s}^{*}_{t})$ from a set of particles, we treat each
particle as a Gaussian in a mixture model with weights $\boldsymbol{w}_{t}$
(see Fig. 3b). For a sensible metric across state dimensions, we scale each
dimension by dividing by the average step size
$\text{E}_{t}[\text{abs}(\boldsymbol{s}^{*}_{t}-\boldsymbol{s}^{*}_{t-1})]$.
This density estimation enables individual and end-to-end learning.

####  IV-E1 Individual learning of the motion model

We optimize the motion model individually to match the observed motion noise
by sampling states $\boldsymbol{s}^{[i]}_{t}$ from $\boldsymbol{s}^{*}_{t-1}$
and $\boldsymbol{a}_{t}$ using Eq. 5-6 and maximizing the data likelihood as
described above,
$\boldsymbol{\theta}^{*}_{f}=\text{argmin}_{\boldsymbol{\theta}_{f}}-\log
p(\boldsymbol{s}^{*}_{t}\mid\boldsymbol{s}^{*}_{t-1},\boldsymbol{a}_{t};\boldsymbol{\theta}_{f})$.
If the dynamics model $g$ is unknown, we train $g_{\boldsymbol{\theta}}$ by
minimizing mean squared error between
$g(\boldsymbol{s}^{*}_{t-1},\boldsymbol{a}_{t})$ and
$\boldsymbol{s}^{*}_{t}-\boldsymbol{s}^{*}_{t-1}$.

####  IV-E2 Individual learning of the measurement model

The particle proposer $k_{\boldsymbol{\theta}}$ is trained by sampling
$\boldsymbol{s}^{[i]}_{t}$ from $\boldsymbol{o}_{t}$ using Eq. 7-8 and
maximizing the Gaussian mixture at $\boldsymbol{s}^{*}_{t}$.

We train the observation likelihood estimator $l_{\boldsymbol{\theta}}$ (and
$h_{\boldsymbol{\theta}}$) by maximizing the likelihood of observations in
their state and minimizing their likelihood in other states,
$\boldsymbol{\theta}^{*}_{h,l}=\text{argmin}_{\boldsymbol{\theta}_{h,l}}$
$-\log(\text{E}_{t}[l_{\boldsymbol{\theta}}(h_{\boldsymbol{\theta}}(\boldsymbol{o}_{t}),\boldsymbol{s}^{*}_{t})])-\log(1-\text{E}_{t_{1},t_{2}}[l_{\boldsymbol{\theta}}(h_{\boldsymbol{\theta}}(\boldsymbol{o}_{t_{1}}),\boldsymbol{s}^{*}_{t_{2}})]).$

####  IV-E3 End-to-end learning

For end-to-end learning, we apply DPFs on overlapping subsequences and
maximize the belief at all true states along the sequence as described above,

$\boldsymbol{\theta}^{*}=\text{argmin}_{\boldsymbol{\theta}}-\log\text{E}_{t}[\text{bel}(\boldsymbol{s}^{*}_{t};\boldsymbol{\theta})].$

###  IV-F Limitations and Future Work

We compute the end-to-end gradient by backpropagation from the DPF output
through the filtering loop. _Since resampling is not differentiable, it stops
the gradient computation after a single loop iteration._ Therefore, the
gradient neglects the effects of previous prediction and update steps on the
current belief. This limits the scope of our implementation to supervised
learning, where predicting the Markov state at each time step is a useful
objective that facilitates future predictions. Differentiable resampling could
still improve supervised learning, e.g. by encouraging beliefs to overestimate
uncertainty, which reduces performance at the current step but can potentially
increase robustness of future state estimates.

Since it is difficult to generate training data that include the true state
$\boldsymbol{s}^{*}_{t}$ outside of simulation, we must work towards
unsupervised learning, which will require backpropagation through multiple
time steps because observations are generally non-Markov. Here are two
possible implementations of differentiable resampling that could be the
starting point of future work: a) Partial resampling: sample only $m$
particles in each step; keep $n-m$ particles from the previous time step; the
gradient can flow backwards through those. b) Proxy gradients: define a proxy
gradient for the weight of a resampled particle that is tied to the particle
it was sampled from; the particle pose is already connected to the pose of the
particle it was sampled from; the gradient can flow through these connections.

##  V Experiments

TABLE I: Feedforward networks for learnable DPF models

$f_{\boldsymbol{\theta}}$: |  2 x fc(32, relu), fc(3) + mean centering across particles
---|---
$g_{\boldsymbol{\theta}}$: |  3 x fc(128, relu), fc(3) + scaled by $\text{E}_{t}[\text{abs}(\boldsymbol{s}_{t}-\boldsymbol{s}_{t-1})]$
$h_{\boldsymbol{\theta}}$: |  conv(3x3, 16, stride 2, relu), conv(3x3, 32, stride 2, relu), conv(3x3, 64, stride 2, relu), dropout(keep 0.3), fc(128, relu)
$k_{\boldsymbol{\theta}}$: |  fc(128, relu), dropout*(keep 0.15), 3 x fc(128, relu), fc(4, tanh)
$l_{\boldsymbol{\theta}}$: |  2 x fc(128, relu), fc(1, sigmoid scaled to range [0.004, 1.0])

fc: fully connected, conv: convolution, *: applied at training and test time

We evaluated DPFs in two state estimation problems in robotics: _global
localization_ and _visual odometry_. We tested global localization in
simulated 3D mazes based on vision and odometry. We focused on this task
because it requires simultaneously considering multiple hypotheses, which is
the main advantage of particle filters over Kalman filters. Here, we
evaluated: a) the effect of end-to-end learning compared to individual
learning and b) the influence of algorithmic priors encoded in DPFs by
comparing to generic LSTMs. To show the versatility of DPFs and to compare to
published results with backprop Kalman filters (BKFs) [7], we also apply DPFs
to the KITTI visual odometry task [5]. The goal is to track the pose of a
driving car based on a first-person-view video. In both tasks, DPFs use the
known dynamics model $g$ but do not assume any knowledge about the map of the
environment and learn the measurement model entirely from data.

Our global localization results show that 1) algorithmic priors enable
explainability, 2) end-to-end learning improves performance but sequencing
individual and end-to-end learning is even more powerful, 3) algorithmic
priors in DPFs improve performance compared to LSTMs reducing the error by
$\sim$80%, and 4) algorithmic priors lead to policy invariance: While the LSTM
baseline learns localization in a way that stops working when the robot
behaves differently ($\sim$84% error rate), localization with the DPF remains
useful with different policies ($\sim$15% error rate).

In the visual odometry task, DPFs outperform BKFs even though the task exactly
fits the capabilities and limitations of Kalman filters—tracking a unimodal
belief from a known initial state. This result demonstrates the applicability
of DPFs to tasks with different properties: higher frequency, longer
sequences, a 5D state instead of a 3D state, and latent actions. The result
also shows that DPFs work on real data and are able to learn measurement
models that work for visually diverse observations based on less than 40
minutes of video.

###  V-A Global Localization Task

(a) Maze 1 (10x5)

(b) Maze 2 (15x9)

(c) Maze 3 (20x13)

(d) Maze 1 observations

(e) Maze 2 observations

(f) Maze 3 observations

Figure 4: Three maze environments. Red lines show example trajectories of
length 100. Blue circles show the first five steps, of which the observations
are depicted below.

The global localization task is about estimating the pose of a robot based on
visual and odometry input. All experiments are performed in modified versions
of the navigation environments from DeepMind Lab [3], where all objects and
unique wall textures were removed to ensure partial observability. Data was
collected by letting the simulated robot wander through the mazes (see Fig.
4). The robot followed a hand-coded policy that moves in directions with high
depth values from RGB-D input and performs 10% random actions. For each maze,
we collected 1000 trajectories of 100 steps with one step per second for
training and testing. As input for localization, we only used RGB images and
odometry, both with random disturbances to make the task more realistic. For
the observations, we randomly cropped the rendered $32\times 32$ RGB images to
$24\times 24$ and added Gaussian noise ($\sigma=20$, see Fig. 4d-f). As
actions, we used odometry information that corresponds to the change in
position and orientation from the previous time step in the robot’s local
frame, corrupted with multiplicative Gaussian noise ($\sigma=0.1$). All
methods were optimized on short trajectories of length 20 with Adam [13] and
regularized using dropout [19] and early stopping. We will now look at the
results for this task.

####  V-A1 Algorithmic priors enable explainability

Due to the algorithmic priors in DPFs, the models remain explainable even
after end-to-end learning. We can therefore examine a) the motion model, b)
the measurement model, and c) their interplay during filtering. Unless
indicated otherwise, all models were first learned individually and then end-
to-end.

##### Motion Model

(a) Predictions with learned motion model

(b) Comparison of learned noise

Figure 5: Learned motion model. (a) shows predictions (cyan) of the state
(red) from the previous state (black). (b) compares prediction uncertainty in
x to true odometry noise (dotted line).

Fig. 5a shows subsequent robot poses together with predictions from the motion
model. These examples show that the model has learned to spread the particles
proportionally to the amount of movement, assigning higher uncertainty to
larger steps. But how does this behavior depend on whether the model was
learned individually or end-to-end?

Fig. 5b compares the average prediction uncertainty using models from
different learning schemes. The results show that individual learning produces
an accurate model of the odometry noise (compare red and the dotted black
lines). End-to-end learning generates models that overestimate the noise
(green and orange lines), which matches insights of experts in state
estimation who report that “many of the models that have proven most
successful in practical applications vastly overestimate the amount of
uncertainty” [21, p. 118].

(a) Obs.

(b) Particle proposer (c) Obs. likelihood estimator

(d) Obs. (e) Particle proposer (f) Obs. likelihood estimator

(g) Obs. (h) Particle proposer (i) Obs. likelihood estimator

Figure 6: Learned measurement model. Observations, corresponding model output,
and true state (red). To remove clutter, the observation likelihood only shows
above average states.

Figure 7: Global localization with DPFs. One plot per time step of a test
trajectory: true state (red), 1000 particles (proposed particles have weight
0.001). Last plot: the weighted particle mean (green) matches the true state
after the first few steps.

##### Measurement Model

Fig. 6 shows three example observations and the corresponding outputs of the
measurement model: proposed particles and weights depending on particle
position. Note how the model predicts particles and estimates high weights at
the true state and other states in locally symmetric parts of the maze. We can
also see that the data distribution shapes the learned models, e.g. by
focusing on dead ends for the second observation, which is where the robot
following the hand-coded policy will look straight at a wall before turning
around. Similar to motion models, end-to-end learned measurement models are
not accurate but effective for end-to-end state estimation, as we will see
next.

##### Filtering

Figure 7 shows filtering with learned models. The DPF starts by generating
many hypotheses (top row). Then, hypotheses form clusters and incorrect
clusters vanish when they are inconsistent with observations (second row).
Finally, the remaining cluster tracks the true state.

(a) Maze 1 (10x5)

(b) Maze 2 (15x9)

(c) Maze 3 (20x13)

(d) Maze 1 (10x5), relative to LSTM

(e) Maze 2 (15x9), relative to LSTM

(f) Maze 3 (20x13), relative to LSTM

Figure 8: Learning curves in all mazes (a-c), also relative to LSTM baseline
(d-f). ind: individual learning, e2e: end-to-end learning. Shaded areas denote
standard errors.  Figure 9: Generalization between policies in maze 2. A:
heuristic exploration policy, B: shortest path policy. Methods were trained
using 1000 trajectories from A, B, or an equal mix of A and B, and then tested
with policy A or B.

####  V-A2 End-to-end learning improves performance

To quantify the effect of end-to-end learning on state estimation performance,
we compared three different learning schemes for DPFs: individual learning of
each model (ind), end-to-end learning (e2e), and both in sequence (ind+e2e).
We evaluated performance in all three mazes and varied the amount of training
trajectories along a logarithmic scale from 32 to 1000. We measured
localization performance by _error rate_ , where we consider a prediction
erroneous if the distance to the true state, divided by
$\text{E}_{t}[\text{abs}(\boldsymbol{s}_{t}-\boldsymbol{s}_{t-1})]$, is
greater than 1.

The resulting learning curves in Fig. 8a-c show that end-to-end learned DPFs
(orange line) consistently outperform individually trained DPFs (red line)
across all mazes. Individual training is worst with few training trajectories
(less than 64) but also plateaus with more data (more than 125 trajectories).
In both cases, the problem is that the models are not optimized for state
estimation performance. With few data, training does not take into account how
unavoidable model errors affect filtering performance. With lots of data, the
models might be individually accurate but suboptimal for end-to-end filtering
performance. End-to-end learning consistently leads to improved performance
for the same reasons.

Performance improves even more when we sequence individual and end-to-end
learning (green line in Fig. 8a-c). Individual pretraining helps because it
incorporates additional information about the function of each model into the
learning process, while end-to-end learning incorporates information about how
these models affect end-to-end performance. Naturally, it is beneficial to
combine both sources of information.

####  V-A3 Algorithmic priors improve performance

To measure the effect of the algorithmic priors encoded in DPFs, we compare
them with a generic neural network baseline that replaces the filtering loop
with a two-layer long-short-term memory network (LSTM) [9]. The baseline
architecture uses the same convolutional network architecture as the DPF—it
embeds images using a convolutional network $h_{\boldsymbol{\theta}}$,
concatenates the embedding with the action vector and feeds the result into
2xlstm(512), 2xfc(256, relu), and fc(3)—and is trained end-to-end to minimize
mean squared error.

The comparison between DPF (ind+e2e) and the LSTM baseline (blue) in Fig. 8a-c
shows that the error rate of DPF (ind+e2e) is lower than for LSTM for all
mazes and all amounts of training data. Also in all mazes, DPF (ind+e2e)
achieve the final performance of LSTM already with 125 trajectories,
$\frac{1}{8}$ of the full training set.

We performed a small ablation study in maze 2 to quantify the effect the known
dynamics model on this performance. When the dynamics model is learned, the
final error rate for DPFs increases from 1.6% to 2.7% compared to 6.0% error
rate for LSTMs. This shows that knowing the dynamics model is helpful but not
essential for DPF’s performance.

To visualize the performance relative to the baseline, we divided all learning
curves by LSTM’s performance (see Fig. 8d-f). Since DPFs encode additional
prior knowledge compared to LSTMs, we might expect them to have higher bias
and lower variance. Therefore, DPF’s relative error should be lowest with
small amounts of data and highest with large amounts of data (the green curves
in Fig. 8d-f should go up steadily from left to right until they cross the
blue lines). Surprisingly, these curves show a different trend: DPFs relative
performance to LSTMs improves with more data and converges to about
$\frac{1}{10}$ to $\frac{1}{3}$. There could be a slight upwards trend in the
end, but on a logarithmic data axis it would take a tremendous amount of data
to close the gap. This result suggests that the priors from the Bayes filter
algorithm reduce variance without adding bias— _that these algorithmic priors
capture some true structure about the problem_ , which data does not help to
improve upon.

####  V-A4 Algorithmic priors lead to policy invariance

To be useful for different tasks, localization must be policy-invariant. At
the same time, the robot must follow some policy to gather training data,
which will inevitably affect the data distribution, add unwanted correlations
between states and actions, etc.

We investigated how much the different methods overfit to these correlations
by changing the policy between training and test, using two policies A and B.
Policy A refers to the heuristic exploration policy that we used for all
experiments above (see Sec. V-A). Policy B uses the true pose of the robot,
randomly generates a goal cell in the maze, computes the shortest path to the
goal, and follows this path from cell to cell using a simple controller mixed
with 10% random actions.

The results in Fig. 9 show that all methods have low error rates when tested
on their training policy (although DPFs improve over LSTMs even more on policy
B). But when we use different policies for training and test, LSTM’s error
rate jumps to over 80%, while DPF (ind+e2e) still works in most cases (5% and
26% error rate).

The LSTM baseline is not able to generalize to new policies because it does
not discriminate between actions and observations and fits to any information
that improves state estimation. If the training data includes correlations
between states and actions (e.g. because the robot moves faster in a long
hallway than in a small room), then the LSTM learns this correlation. Put
differently, the LSTM learns to infer the state from the action chosen by the
policy. The problem is that this inference fails if the policy changes. The
algorithmic priors in DPFs prevent them from overfitting to such correlations
because DPFs cannot directly infer states from actions.

DPFs generalize better from A to B than from B to A. Since generalization from
B to A is equally difficult for DPFs with individually learned models, the
error increase cannot come from overfitting to correlations in the data
through end-to-end learning but is most likely because the states visited by
policy A cover those visited by policy B but not vice versa.

The alternative approach to encoding policy invariance as a prior is to learn
it by adding this variance to the data. Our results show that if we train on
combined training data from both policies (A+B), all methods perform well in
tests with either policy. This approach in the spirit of domain randomization
and data augmentation helps DPFs because it covers the union of the visited
states and (additionally) helps LSTM by including state-action correlations
from both policies. But to make the LSTM localization truly policy invariant
such that it would work with any new policy C, the training data has to cover
the space of all policies in an unbiased way, which is difficult for any
interesting problem.

(a) Visual input (image and difference image) at time steps 100, 200, and 300
(indicated in (b) by black circles) (b) Trajectory 9; starts at (0,0)

Figure 10: Visual odometry with DPFs. Example test trajectory

###  V-B Visual Odometry Task

To validate our simulation results on real data, we applied DPFs on the KITTI
visual odometry data set, which consists of data from eleven trajectories of a
real car driving in an urban area for a total of 40 minutes. The data set
includes RGB stereo camera images as well as the ground truth position and
orientation of the car in an interval of $\sim$0.1 seconds. The challenge of
this task is to generalize in a way that works across highly diverse
observations because the method is tested on roads that are never seen during
training. Since the roads are different in each trajectory, it is not possible
to extract global information about the car’s position from the images.
Instead, we need to estimate the car’s translational and angular velocity from
the stream of images and integrate this information over time to track the
car’s position and orientation.

We tackle this problem with a DPF in a five dimensional state space, which
consists of the position, orientation, forward velocity and angular velocity.
DPFs learn to perform visual odometry from a known initial state using a
simple first-order _dynamics model_ $g$ and a learnable action sampler
$f_{\boldsymbol{\theta}}$. Since there is no information about the action of
the driver, the action sampler produces zero mean motion noise on the velocity
dimensions, which is then evaluated with the measurement model. For a fair
comparison, we used the same network architecture for the observation encoder
$h_{\boldsymbol{\theta}}$ as in the backprop Kalman filter paper [7], which
takes as input the current image and the difference image to the last frame
(see Fig. 10). Our observation likelihood estimator $l_{\boldsymbol{\theta}}$
weights particles based on their velocity dimensions and the encoding
$h_{\boldsymbol{\theta}}(\boldsymbol{o}_{t})$. Since, the initial state is
known, we do not use a particle proposer. We train the DPF individually and
end-to-end, using only the velocity dimensions for maximum likelihood
estimation.

We evaluated the performance following the same procedure as in the BKF paper.
We used eleven-fold cross validation where we picked one trajectory for
testing and used all others for training with subsequences of length 50. We
evaluated the trained model on the test trajectory by computing the average
error over all subsequences of 100 time steps and all subsequences of 100,
200, 400, and 800 time steps.

Table V-B compares our results to those published for BKFs [7]. DPFs
outperform BKFs, in particular for short sequences where they reduce the error
by $\sim$30%. Any improvement over BKFs in the this task is surprising because
Gaussian beliefs seem sufficient to capture uncertainty in this task. The
improvement could come from the ability of particles to represent long tailed
probability distributions. These results demonstrate that DPFs generalize to
different tasks and can be successfully applied to real data.

TABLE II: KITTI visual odometry results

o @ X[1.2, l] X[c] X[c] @ | Test 100 | Test 100/200/400/800
---|---|---
Translational error (m/m)
BKF* | 0.2062 | 0.1804
DPF (ind) | 0.1901 $\pm$ 0.0229 | 0.2246 $\pm$ 0.0371
DPF (e2e) | 0.1467 $\pm$ 0.0149 | 0.1748 $\pm$ 0.0468
DPF (ind+e2e) | 0.1559 $\pm$ 0.0280 | 0.1666 $\pm$ 0.0379
Rotational error (deg/m)
BKF* | 0.0801 | 0.0556
DPF (ind) | 0.1074 $\pm$ 0.0199 | 0.0806 $\pm$ 0.0153
DPF (e2e) | 0.0645 $\pm$ 0.0086 | 0.0524 $\pm$ 0.0068
DPF (ind+e2e) | 0.0499 $\pm$ 0.0082 | 0.0409 $\pm$ 0.0060
Means $\pm$ standard errors; * results from [7]

##  VI Conclusion

We introduced differentiable particle filters to demonstrate the advantages of
combining end-to-end learning with algorithmic priors. End-to-end learning
optimizes models for performance while algorithmic priors enable
explainability and regularize learning, which improves data-efficiency and
generalization. The use of algorithms as algorithmic priors will help to
realize the potential of deep learning in robotics. The components of the DPF
implementation, such as sample generation and density estimation, will be
useful for producing differentiable versions of other sampling-based
algorithms.

## Acknowledgments

We gratefully acknowledge financial support by the German Research Foundation
(DFG, project number 329426068).

## References

  * [1] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng.  TensorFlow: Large-scale machine learning on heterogeneous systems.  http://tensorflow.org/, 2015.
  * [2] James E. Baker.  Reducing Bias and Inefficiency in the Selection Algorithm.  In Proceedings of the International Conference on Genetic Algorithms (ICGA), pages 14–21, 1987.
  * [3] Charles Beattie, Joel Z. Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, and others.  Deepmind Lab.  arXiv:1612.03801, 2016.
  * [4] DeepMind.  Sonnet: TensorFlow-Based Neural Network Library.  https://github.com/deepmind/sonnet, 2017.
  * [5] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun.  Vision Meets Robotics: The KITTI Dataset.  The International Journal of Robotics Research, 32(11):1231–1237, 2013.
  * [6] Saurabh Gupta, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik.  Cognitive Mapping and Planning for Visual Navigation.  arXiv:1702.03920, 2017.
  * [7] Tuomas Haarnoja, Anurag Ajay, Sergey Levine, and Pieter Abbeel.  Backprop KF: Learning Discriminative Deterministic State Estimators.  In Advances in Neural Information Processing Systems (NIPS), pages 4376–4384, 2016.
  * [8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  arXiv:1512.03385, 2015.
  * [9] Sepp Hochreiter and Jürgen Schmidhuber.  Long Short-Term Memory.  Neural Computation, 9(8):1735–1780, 1997.
  * [10] Rico Jonschkowski and Oliver Brock.  End-To-End Learnable Histogram Filters.  In Workshop on Deep Learning for Action and Interaction at the Conference on Neural Information Processing Systems (NIPS), 2016.
  * [11] Peter Karkus, David Hsu, and Wee Sun Lee.  QMDP-Net: Deep Learning for Planning under Partial Observability.  In Advances in Neural Information Processing Systems (NIPS), pages 4697–4707, 2017.
  * [12] Maximilian Karl, Maximilian Soelch, Justin Bayer, and Patrick van der Smagt.  Deep Wariational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data.  arXiv:1605.06432, 2017.
  * [13] Diederik P. Kingma and Jimmy Ba.  Adam: A Method for Stochastic Optimization.  In Proceedings of the International Conference on Learning Representations (ICLR), 2014.
  * [14] Diederik P. Kingma and Max Welling.  Auto-Encoding Variational Bayes.  arXiv:1312.6114, 2013.
  * [15] Yann A. LeCun, Bernhard E. Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne E. Hubbard, and Lawrence D. Jackel.  Backpropagation Applied to Handwritten Zip Code Recognition.  Neural Computation, 1(4):541–551, 1989.
  * [16] Masashi Okada, Luca Rigazio, and Takenobu Aoshima.  Path Integral Networks: End-to-End Differentiable Optimal Control.  arXiv:1706.09597, 2017.
  * [17] Tanmay Shankar, Santosha K. Dwivedy, and Prithwijit Guha.  Reinforcement Learning via Recurrent Convolutional Neural Networks.  In Proceedings of the International Conference on Pattern Recognition (ICPR), pages 2592–2597, 2016.
  * [18] David Silver, Hado van Hasselt, Matteo Hessel, Tom Schaul, Arthur Guez, Tim Harley, Gabriel Dulac-Arnold, David Reichert, Neil Rabinowitz, and Andre Barreto.  The Predictron: End-to-End Learning and Planning.  In Proceedings of the International Conference on Machine Learning (ICML), pages 3191–3199, 2017.
  * [19] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: A Simple Way to Prevent Neural Networks from Overfitting.  Journal of Machine Learning Research, 15(1):1929–1958, 2014.
  * [20] Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Pieter Abbeel.  Value Iteration Networks.  In Advances in Neural Information Processing Systems (NIPS), pages 2154–2162, 2016.
  * [21] Sebastian Thrun, Wolfram Burgard, and Dieter Fox.  Probabilistic Robotics.  MIT Press, 2005.
  * [22] Manuel Watter, Jost Tobias Springberg, Joschka Boedecker, and Martin Riedmiller.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images.  In Advances in Neural Information Processing Systems (NIPS), pages 2746–2754, 2015.


# Transient two pole accretion in the polar V496 UMa

M. R. Kennedy,1,2 C. Littlefield,3,4222 and P. M. Garnavich3
1Department of Physics, University College Cork, Cork, Ireland
2Jodrell Bank Centre for Astrophysics, Department of Physics and Astronomy,
The University of Manchester, M19 9PL, UK
3Department of Physics, University of Notre Dame, Notre Dame, IN 46556 USA
4Bay Area Environmental Research Institute, Moffett Field, CA 94035 USA
E-mail<EMAIL_ADDRESS>authors contributed equally to this work.

(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

We report XMM-Newton and TESS observations of V496 UMa, an AM Herculis-type
cataclysmic variable. The XMM-Newton observation reveals that at times, two
poles on the white dwarf accrete simultaneously, but accretion onto the
secondary magnetic pole is erratic and can nearly cease in less than one
binary orbit (1.5 h). Modelling of the X-ray spectrum during the primary
maximum reveals no change in the accretion structures onto the primary pole
when accretion onto the secondary pole is disrupted, suggesting that the
disruption of accretion onto the secondary pole may be caused by mass-transfer
variations from the donor star. The TESS observation, which spanned eight
weeks at a two-minute cadence, shows a stable, double-humped orbital
modulation due to cyclotron emission from the post-shock region, while the
observed times of maximum light show a slow systematic drift that does not
correlate with the system’s overall brightness.

###### keywords:

accretion, accretion discs – novae, cataclysmic variables – stars: magnetic
field – X-rays: individual: V496 UMa

††pubyear: 2021††pagerange: Transient two pole accretion in the polar V496
UMa–A

##  1 Introduction

Cataclysmic variables (CV) are compact binary systems with white dwarf (WD)
primaries that are accreting material from a nearby companion star which fills
its Roche lobe. When a new CV is discovered, it is important that a clear
understanding the accretion structures within the binary is established, as
this gives us information regarding the central WD. Indeed, the path material
from the companion takes as it flows through the inner Lagrange point (L1)
towards the WD is dictated by the magnetic field of the WD. In systems where
the WDs surface magnetic field is large (>5 MG), material flows first as a
ballistic stream towards the WD until the point at which the magnetic pressure
exerted on the material by the WDs magnetic field overcomes the ram pressure
inside the stream. From this point onwards, material couples to the WDs
magnetic field and flows towards the WDs magnetic poles. Systems in which the
magnetic field is strong enough to produce such an accretion structure are
called AM Her stars (after the archetypal system) or polars due to the high
percentage of polarsied optical light which they produce (Tapia, 1977). They
are formed as, soon after the binaries formation, the rotational period of the
WD ($P_{\rm{s}}$) synchronises to the orbital period ($P_{\rm{O}}$) of the
system and becomes tidally locked due to the interaction between the WDs
magnetic field and the secondary stars magnetic field.

Due to tidal locking, there is a preferential magnetic pole on the WD for
material to flow to - the one which is aligned best with material in the
ballistic stream (for the remainder of this paper, we shall refer to this as
the primary pole). During the early years of polar study, accretion was
thought to only occur onto a single pole of the white dwarf. Such accretion
onto a single pole leads to large amplitude variations at optical and X-ray
wavelengths as the accreting pole rotates in and out of our field of view.
However, after the discovery of polars which underwent changes in the sign of
the circular polarisation (e.g. VV Pup: Liebert & Stockman 1979), polars which
had two optical maxima and minima per orbit (e.g. EF Eri; Watson et al. 1980),
and the variable light curve of the archetype of polars AM Her (Heise et al.,
1985), it was quickly realised that a second pole might accrete within these
systems. This idea of 2 pole accretion was soldified by spectroscopic
observations of VV Pup, in which 2 distinct sets of cyclotron features
(corresponding to magnetic both poles of the WD) were observed (Wickramasinghe
et al., 1989). Since then, secondary pole accretion has become a common
feature of many polars.

While accretion onto this secondary pole can be constant and uninterrupted
(e.g. Reimers et al. 1999, Schwarz et al. 2001, and Schwarz et al. 2002), it
can also be transient, and manifests either as a change in the optical and
X-ray light curve (as in AM Her), or as a change in the sign of the circular
polarisation of light coming from a polar (e.g. as in VV Pup and QQ Vul;
Schwope et al. 2000). In such cases, the accretion stream can be thought of as
a probe of the WDs magnetic field, as it couples on to different fields lines
at different times, helping us to build a picture of the WDs magnetic field
structure.

For systems with a transient behaviour, there are two possible explanations.
The first is that the white dwarf is spinning with a period slightly longer or
shorter than the orbital period. There are a handful of polar systems for
which this true, and $P_{\rm{s}}$ is $\sim 2\%$ smaller or larger than
$P_{\rm{O}}$. These systems are thought to have been knocked out of
synchronicity by a nova eruption on the WD, an idea developed after
observations of V1500 Cyg after a nova outburst in 1975 (Stockman et al.,
1988). They are expected to synchronise after enough time has passed,and if
this asynchronicity is the case of the transient two pole accretion, then the
phenomenon should occur over a periodic timescale (e.g. as in clearly in TESS
observations of the asynchronous polar CD Ind; Hakala et al. 2019 and
Littlefield et al. 2019).

However, there are clear cases where a polar switches between one and two pole
accretion, and is not asynchronous. One need look no further than the
archetype of polars, AM Her, to see a firm example. AM Her has been observed
in both one-pole and two-pole configurations, but the timescale for switching
between configurations is months-years. For short epochs of observations
($\sim$ months), the accretion geometry seems stable (see Schwope et al. 2020
for a thorough review on the variability seen in AM Her).

The alternate model is that the transient behaviour is caused by a change in
the mass transfer rate from the binary. When the transfer rate is high, the
ram pressure within the accretion structures is high enough such that the
penetration depth the ballistic stream achieves into the WDs magnetic field is
deep enough for material to reach field lines connected to the second pole. If
the mass transfer rate drops, the penetration depth decreases, leading to
cessation of accretion onto the secondary pole. This variable accretion model
led researchers to investigate whether the X-ray emission from the secondary
pole is not described by the typical shock model (King & Lasota 1979;Lamb &
Masters 1979), but instead may be due to “blobby” accretion (Kuijpers &
Pringle 1982; Frank et al. 1988), where individual blobs of accreting material
penetrate below the WD photosphere, manifesting as thermal radiation. Such a
model has been applied to explain the different accretion regimes within AM
Her, and predicts significantly different X-ray spectra from the primary and
secondary poles. (Hameury & King 1988;Schwope et al. 2020).

The cause of the variation in the mass transfer rate have been explained by
stellar spots on the secondary star causing a temporary change in the
accretion rate (Livio & Pringle, 1994), but the time scale for switching
between one and two pole accretion is often on a timescale of weeks to years.

Differentiating whether two-pole accretion is occurring due to asynchronicity
or a variable mass transfer rate cases requires long term monitoring to
identify any periodicity in the transitions between single and two-pole
accretion. Finally, determining whether a polar is undergoing “blobby”
accretion onto the secondary pole requires X-ray spectra of both the primary
and secondary poles.

This paper focuses on the polar V496 UMa, and on answering questions
surrounding the accretion geometry and its stability. This system has been the
subject of two dedicated studies, both of which reported time-series
photometry and optical spectroscopy. Littlefield et al. (2015) measured a
91-minute orbital period and showed that a typical orbital light curve
contains two photometric maxima, one of which peaks at $V\sim 16.5$ and the
other at $V\sim 17$. A single, low-resolution spectrum showed the H, He I, and
He II emission lines which are characteristic of a polar accreting at a high
accretion rate, along with a non-thermal continuum. Littlefield et al. (2018)
followed up with time-series spectroscopy showing that V496 UMa’s emission-
line spectrum transitions into an absorption spectrum for several minutes
during each orbit when the accretion curtain eclipses the cyclotron-emitting
region. They also established that the non-thermal continuum in the optical
spectrum is caused by smearing of the harmonics of V496 UMa’s cyclotron
spectrum. V496 UMa’s parallax from Gaia EDR3 (Gaia Collaboration et al., 2016,
2021) yields a distance of $760\pm 30$ pc using the geometric algorithm from
Bailer-Jones et al. (2021).

V496 UMa’s most distinguishing property is the intermittent nature of the
secondary maximum in its optical light curve. Littlefield et al. (2018) found
that four of the 45 secondary maxima in their photometry were either
extraordinarily weak or completely absent, with no apparent impact on the rest
of the orbital light curve (Fig. 3 in Littlefield et al., 2018). When absent,
V496 UMa could be as much as 2.5 mag fainter during the expected secondary
maxima than its normal brightness during this part of the orbit. Even more
surprisingly, a failed secondary maximum in one orbit could be followed by a
normal secondary maximum in the very next orbit, establishing that the
mechanism responsible for the failed maxima operates on timescales of less
than one orbit. Littlefield et al. (2018) speculated that the missing maxima
might arise from intermittent accretion onto the secondary magnetic pole but
lacked the observational data to test this proposal.

Motivated by the question of what is causing the missing secondary maximum,
and whether the X-ray spectrum and light curve vary in a similar manner, we
obtained X-ray data of V496 UMa using the XMM-Newton X-ray telescope. During
preparation of these data (present in Section 3), V496 UMa was also observed
by the Transiting Exoplanet Survey Satellite (TESS; Ricker et al. 2015),
allowing for a unique opportunity to probe the long term nature of the
variability of the secondary maximum. These data are discussed in Section 4.

##  2 Observations

Table 1: Details of the various observations of V496 UMa. Facility | Start Time | End Time | Cadence
---|---|---|---
XMM-Newton | 2017-12-03 08:35:31 | 2017-12-03 16:38:51 | 100s (X-ray), 10s (Optical)
SLKT | 2019-08-29 01:53:49 | 2019-08-29 03:46:07 | 33s
SLKT | 2019-09-06 02:14:52 | 2019-09-06 03:44:54 | 33s
SLKT | 2019-09-18 01:12:35 | 2019-09-18 02:53:00 | 33s
SLKT | 2017-12-03 08:14:49 | 2017-12-03 11:59:14 | 33s
AAVSO | 2017-12-02 08:18:46 | 2017-12-07 12:32:30 |
TESS | 2019-08-15 | 2019-10-06 | 120 s

###  2.1 XMM-Newton

V496 UMa was observed by XMM-Newton for 29 ks starting 2017-12-03 08:35:31
(UTC). The European Photon Imaging Camera (EPIC) -pn (Strüder et al., 2001),
-MOS1, and MOS2 (Turner et al., 2001) instruments were all operated in full
frame mode with a thin filter inserted. The Reflection Grating Spectrographs
(RGS1 and RGS2; den Herder et al. 2001) were both operated in spectroscopy
HER+SES mode. The Optical Monitor (OM; Mason et al. 2001) was operated in fast
imaging mode with a white filter inserted. Due to the OMs observing mode,
there are brief gaps in coverage every $\sim 26$ min. Initial inspection of
the RGS data suggested no appreciable signal was detected, and these data will
not be discussed further.

All data were reduced using tasks in SAS v16.1.0. All data were corrected to
the solar system barycentre using Barycen. A background light curve was
inspected to look for periods of high background which may have affected the
data, but none were found. All extracted spectra and light curves are
available through an online repository.

###  2.2 TESS

TESS observed V496 UMa in two consecutive sectors at a two-minute cadence.
Observations began in Sector 14 on 2019 Aug. 15 and continued until the end of
Sector 15 on 2019 Oct. 6. The TESS data are nearly uninterrupted, except for
three downlink gaps. Both TESS light curves were extracted with lightkurve
(Lightkurve Collaboration et al., 2018). After experimenting with different
extraction apertures, we decided to use the pipeline apertures. Due to TESS’s
24-arcsec pixels, the TESS observations of V496 UMa are blended. In spite of
this blending, V496 UMa and its photometric variability were both readily
apparent in visual inspection of the TESS images.

###  2.3 Ground-Based Optical Photometry

Additional observations of V496 UMa were carried out by members of the
American Association of Variable Star Observers (AAVSO) in the days leading up
to, during, and after the XMM-Newton observations. These data were used to
identify correlations between the X-ray behaviour and optical behaviour of the
system.

Finally, we used the 80-cm Sarah L. Krizmanich Telescope (SLKT) at the
University of Notre Dame to obtain time-series photometry of V496 UMa during
the first part of the XMM-Newton observation as well as three light curves
while TESS observations were underway. Table 1 summarizes these observations.
The observations consisted of 30-second unfiltered exposures with
approximately 3 s of overhead between images. Data were debiased and
flatfielded in the usual fashion, and differential aperture photometry used to
extract the flux of V496 UMa.

##  3 X-ray

Figure 1: Light curves around the time of the XMM-Newton observations. The top
panel shows the full 0.3–10.0 keV light curve. The middle panel shows the
optical light curves using data from the OM, the SLKT, and from members of the
AAVSO community. The second panel shows the X-ray light split into 2 bands -
soft (0.3–2 keV) and hard (2–10 keV). The bottom panel shows the ratio of
these light curves, and highlights when we see a hard X-ray excess. Orbital
phase has been calculated using the ephemeris described in the text.

###  3.1 X-ray light curves

Light curves were extracted for 3 energy ranges - the full energy range of the
detectors (0.3-10.0 keV), a soft energy range of 0.3-2.0 keV, and a hard
energy range of 2.0-10.0 keV. The Hardness ratio
(($F_{2-10}-F_{0.3-2}$)/($F_{2-10}+F_{0.3-2}$); Worpel & Schwope 2015) for the
duration of the observations was also computed. The top panel of Figure 1
shows the 0.3-10.0 keV light curve of V496 UMa phased using the ephemeris from
Section 4.2. A total of 8 X-ray maxima were detected over 5 orbits of
observations. The light curve over a single orbital period is composed of
three features - a primary maximum at $\phi=0$ which corresponds to the
optical maximum, a secondary maximum which occurs at $\phi=0.4$, and a rapid
change in the Hardness ratio from -0.5 to +0.3 at $\phi=0.75$. The secondary
maximum is only clearly detected for the first 3 orbital periods of data,
after which its strength diminishes rapidly.

###  3.2 X-ray spectra

####  3.2.1 Spectral Extraction

Spectra covering the 0.3-10.0 keV were extracted for several different phase
intervals as given by:

  * •

An X-ray spectrum constructed from all data up until the first missing
secondary maximum (T (BJD)<245810.06). This is referred to as the “half data”
set in the rest of the text.

  * •

An X-ray spectrum of the primary maximum (data with $0.85<\phi<0.2$).

  * •

An X-ray spectrum of the secondary maximum (data with $0.2<\phi<0.7$, but only
for the first three orbital cycles).

  * •

An X-ray spectrum of the first failed secondary maximum (data with
$0.2<\phi<0.7$, but only for the fourth orbital cycle).

  * •

An X-ray spectrum of the second failed secondary maximum (data with
$0.2<\phi<0.7$, but only for the fifth orbital cycle).

  * •

An X-ray spectrum of regions where the Hardness ratio was measured to be
positive (data with $0.7<\phi<0.85$). This is referred to as the absorption
dip spectrum from here onward.

In the case of each spectrum, the region for source extraction was chosen to
be a circle with a radius of 24″around the target position. For the PN
instrument, background spectra were extracted using a circular annulus
centered on 13:21:32.46 +56:11:58.45 and with a radius of 57″. For both MOS
instruments, background spectra came from a circular annulus centered on
13:21:33.25 +56:09:17.63 and with a radius of 96″. These spectra, along with
the times of the X-ray observation they correspond to, are shown in Figure 2.

Figure 2: The extracted PN (black points), MOS1 (red points), and MOS2 (purple
points) for 6 different time segments, along with the model residuals in units
of $\sigma$. Above each spectrum is the 1-10 keV light curve from the PN
instrument. The spectra in each panel were extracted using the highlighted
time ranges. The best fit spectra to each epoch of data are also plotted in
each panel as histograms, with the same colour scheme as the data. The
components which are summed together to give the best fitting model to the PN
instrument data are also shown in each panel, and consist of a blackbody
component (blue), a diffuse hot plasma (Mekal; orange) and in one instance a
6.4 keV Gaussian emission component (green)

####  3.2.2 Spectral Fitting

The spectra were analysed using Xspec v12.10.1 (Arnaud, 1996). Each of the 6
extracted spectra were fit with a black body to account for the soft (<1.0
keV) component and a single temperature plasma emission model (mekal in Xspec;
Mewe et al. 1985; Mewe et al. 1986; Liedahl et al. 1995) to account for
emission produced in the shock above the white dwarfs surface. Both components
were absorbed by an interstellar absorber (tbabs, the Tuebingen-Boulder ISM
absorption model; Wilms et al. 2000). Finally, the model was multiplied by a
constant which was set to a value of 1 for the PN instrument, and allowed to
vary for both the MOS1 and MOS2 instruments to allow for cross-instrument
calibration. Such a model is a common starting place for describing the
spectra of polars (e.g. Schmidt et al. 2005; Worpel & Schwope 2015). We also
included a Gaussian emission component at 6.4 keV to account for the common
appearance of the Fe fluorescence feature at these energies in some accreting
systems.

For the absorption dip spectrum, we added an additional partial covering
absorption component (pcfabs), and froze all other parameters to their best
fit values from modelling the spectrum of the primary maximum, under the
assumption that the only difference between the absorption dip spectrum and
the primary maximum spectrum should be additional absorption from the
accretion stream.

The best fit parameters for these models were found using the default
Levenberg-Marquardt algorithm in Xspec with a maximum number of 10000
evaluations allowed and a critical delta of $1\times 10^{-4}$ required. The
parameter space was then explored to obtain errors on the parameters by using
the Goodman-Weare algorithm (Goodman & Weare, 2010) for Markov Chain Monte
Carlo’s as implemented within Xspec. A total of 20 walkers were used, each of
which were allowed to take 500,000 steps. The corner plots from the MCMC
analysis of each of the spectra are included as an online dataset, while the
corner plot from fitting the primary maximum is included in Appendix A.

The results from fitting this model to each of the spectra are shown in Figure
2, and the best-fit parameter values are given in Table 2. We also include the
unabsorbed, 0.3-10.0 keV X-ray luminosity (assuming a source distance of
760$\pm$30 pc) for just the plasma component of the model (that is, excluding
the soft thermal emission from the white dwarf), which can be used as a stand-
in for the mass accretion rate.

Of the six data sets which this model was applied to, only two have
unacceptable an $\chi^{2}$ \- the half data and primary maximum data. In the
first instance, the poor $\chi^{2}$ can be attributed to the fact that the
spectrum is the results of emission from both accreting magnetic poles of the
WD, while the model is a single temperature plasma. As such, decomposing the
spectrum into a primary and secondary spectrum improves this.

The cause of the $\chi^{2}$ of 305 for 275 degrees of freedom when fitting the
spectrum of the primary maximum is more difficult to explain. In the above, we
have assumed the hard X-ray emission comes from a single temperature plasma,
the reality is likely more complex. The plasma should have a range of
temperatures due to the ballistic stream coupling to the magnetic field across
a range of angles, rather than at a single point. This is what likely leads to
the high $\chi^{2}_{\rm R}$ value when modelling the primary maximum. As such,
we have also fit the primary maximum data with the mekal replaced by cemekl
(Singh et al., 1996), which allows for a multi-temperature plasma. The best
fitting parameters and their errors (as estimated using the same methods as
above) are given in Table 3. The $\chi^{2}$ of 282 for 274 d.o.f is a
significant improvement on the single temperature model, but there is a very
strong anti-correlation between the index of the power-law emissivity function
versus the maximum plasma temperature (as seen in Appendix A), making it
difficult to conclude anything physical from these models.

Table 2: Model parameters from fitting each of the spectrum in Figure 2 with an absorbed black body and plasma model. Errors are given at the $1\sigma$ level, and have been calculated as described in the text. Parameters marked with $a$ were frozen when fitting. The 0.3-10 keV X-ray luminosity of the Mekal component has been calculated assuming a source distance of 760$\pm$30 pc. Data Considered | Half data | Primary Max | Secondary Max | Failed Max # 1 | Failed Max # 2 | Absorption dip
---|---|---|---|---|---|---
$n_{\rm H}$ ($\times 10^{22}\>{\rm cm}^{-2}$) | <$0.005$ | <$0.008$ | <$0.01$ | <$0.06$ | <$0.15$ | 0.008a
$n_{\rm H,pcfabs}$ ($\times 10^{22}\>{\rm cm}^{-2}$) | - | - | - | - | - | $2.1\pm 0.4$
CvrFract | - | - | - | - | - | $0.68\pm 0.01$
$kT_{\rm BB}$ (keV) | $0.078^{+0.01}_{-0.009}$ | $0.09\pm 0.02$ | $0.06^{+0.01}_{-0.009}$ | <$0.25$ | <$0.27$ | 0.09a
$norm_{\rm BB}$ ($(\times 10^{-6})$) | $3.1^{+0.7}_{-0.5}$ | $2.8^{+0.7}_{-0.4}$ | $5^{+2}_{-1}$ | <$8$ | <$8$ | 2.8a
$kT_{\rm mekal}$ (keV) | $15\pm 1$ | $13\pm 1$ | $14\pm 2$ | $7^{+2}_{-1}$ | $20^{+20}_{-10}$ | 13a
$norm_{\rm mekal}$ $(\times 10^{-3})$ | $1.42\pm 0.02$ | $1.69\pm 0.03$ | $1.22\pm 0.03$ | $0.74\pm 0.04$ | $0.31\pm 0.05$ | 1.69a
$C_{\rm MOS1}$ | $0.95\pm 0.02$ | $0.94\pm 0.02$ | $0.96\pm 0.03$ | $0.96\pm 0.07$ | $1.0\pm 0.1$ | $0.92\pm 0.05$
$C_{\rm MOS2}$ | $0.96\pm 0.02$ | $0.97\pm 0.02$ | $0.94\pm 0.03$ | $0.90\pm 0.07$ | $1.0\pm 0.1$ | $0.89\pm 0.05$
$L_{\rm MEKAL,0.3-10keV}$ (erg/s) |  | $2.5\pm 0.2\times 10^{32}$ | $1.8\pm 0.1\times 10^{32}$ | $1.1\pm 0.1\times 10^{32}$ | $0.42\pm 0.05\times 10^{32}$ |
$\chi^{2}$ (d.o.f) | 340 (294) | 305 (275) | 245 (230) | 52 (65) | 26.25 (19) | 185 (98)
Table 3: Results from applying the multi-temperature plasma model to the primary maximum data. Data Considered Primary Max |
---|---
$n_{\rm H}$ ($\times 10^{22}\>{\rm cm}^{-2}$) | <$0.005$
$kT_{\rm BB}$ (keV) | $0.071^{+0.008}_{-0.009}$
$norm_{\rm BB}$ ($(\times 10^{-6})$) | $5.0^{+2.0}_{-1.0}$
$\alpha$ | $1.1^{+0.2}_{-0.1}$
$kT_{max}$ (keV) | $41^{+13}_{-9}$
$norm_{\rm mekal}$ $(\times 10^{-3})$ | $4.2\pm 0.5$
$C_{\rm MOS1}$ | $0.94\pm 0.02$
$C_{\rm MOS2}$ | $0.97\pm 0.02$
$\chi^{2}$ (d.o.f) | 282 (274)

###  3.3 X-ray absorption dip

The spikes in the Hardness ratio occur at the same orbital phase during which
absorption lines appear in the optical spectrum of V496 UMa. The X-ray spectra
extracted during this orbital phase show that the spike in the hard-soft ratio
are due to a significant decrease in the soft X-ray flux (as opposed to being
an increase in the hard X-ray flux).

Modelling of this spectrum reveals that the decrease in the soft X-ray flux is
due to a sudden increase in the absorption column between us and V496 UMa. The
most likely cause of the sudden increase in absorption is the accretion stream
passing through our line of sight, temporarily blocking the view of the
accreting primary pole and absorbing a majority of the produced soft X-rays.
Since modelling of the other extracted spectra allow us to put strong upper
limits on the interstellar absorption column of $n_{\rm H}<0.01\times 10^{22}$
cm-2 in the direction of V496 UMa we can attribute the entirety of the
measured value of $n_{\rm H}=(2.1\pm 0.4)\times 10^{22}$ cm-2 to absorption by
the accretion column. In terms of the system geometry, this absorption dip
suggests the accretion stream is leading the companion star, as this soft
X-ray absorption dip occurs before inferior conjunction of the companion.

If the single-to-noise of the individual absorption dip spectra were high
enough, the measurement of the particle density of the column could be used to
directly measure variations in the mass-accretion rate over the timescale of a
single orbit. Unfortunately, these data do not have the sufficient S/N to do
this, but it may be possible with future, more sensitive X-ray missions.

###  3.4 Shock temperatures and magnetic field geometry

The improvement of the multi-temperature plasma model over the single
temperature model when modelling the primary maximum is in line with the
connecting region between the ballistic stream and the primary poles magnetic
field spanning a range of azimuth and radii. We are not able to derive strong
constraints on the maximum plasma temperature, likely due to the low count
rate at the highest energies of our spectrum. Further X-ray data taken at high
energies (for example, with NuSTAR) will be able to better constrain the
highest shock temperature.

The very good agreement between the model and data for the secondary maximum
suggests one of two things. Either the material which is feeding this pole
comes from a very narrow connecting region, leading to a shock which is very
close to uniform in temperature or, more likely, the spectrum does not have
sufficient signal to differentiate between a single and multi-temperature
plasma. Again, observations at a higher X-ray energy will help differentiate
the two scenarios.

With the detection of two distinct maxima in the X-ray light curve, it is very
likely that the white dwarf primary in V496 UMa is accreting onto both of its
magnetic poles, as proposed by Littlefield et al. (2018). If the magnetic
field in V496 UMa were perfectly dipolar, one would naively assume that these
maxima should be 180 degrees apart, or in other words, separated by 0.5 in
orbital phase. This is very close to the observed phase separation of the two
X-ray maxima in V496 UMa when both maxima are present and stable, suggesting
the structure of WD’s magnetic field might be reasonably approximated as
dipolar. An additional test for this can be done by measuring the magnetic
field of both poles. This is typically done by measuring the cyclotron
harmonics in the optical spectrum of both accretion regions (as was done for
e.g. V808 Aur; Worpel & Schwope 2015). However, as highlighted by Littlefield
et al. (2018), measurement of the magnetic field in V496 UMa is complicated by
significant smearing of the harmonics, hampering attempts to measure the
magnetic field of both poles.

###  3.5 Failed X-ray maximum

The detection of 2 maxima per orbital phase during the first 3 orbits of XMM-
Newton data confirm the suggestion put forward by Littlefield et al. (2018)
that accretion onto the WD in V496 UMa typically occurs via two distinct
magnetic poles. Modelling of the observed X-ray spectra during these maxima
reveal that both accretion columns have approximately the same temperatures in
the shock in the accretion column, and that both polar caps of the WD are
heated to the same degree.

The two failed secondary maxima in the optical light curve during the latter
half of the XMM-Newton observations coincide with a significant decrease in
the amplitude of the secondary maxima in the X-ray light curve. Assuming the
blackbody component of our models is coming from the WD surface, modelling of
these two failed maxima show that the temperature of the WD surface was
unchanged (to within 1$\sigma$) when compared with the derived temperature
when the secondary maximum was present. On the other hand, the shock
temperature exhibits a rapid decrease between the times when the secondary
maximum is present ($kT=17^{+4}_{-2}$ keV) and when it is not present
($kT=7^{+2}_{-1}$ keV during the first failed maximum, and completely
unconstrained during the second failed maximum).

This suggests that accretion onto the second, less preferential magnetic pole
decreases significantly but does not cease entirely. The primary maxima before
these failed secondary maxima are not significantly brighter than the primary
maxima which occur when the secondary maximum is fully present. This rules out
the case that more material gets channeled onto the primary magnetic pole
during the failed secondary maxima as, if this were the case, we would expect
the primary maxima to increase in strength.

Rather, the data suggests an overall decrease in the mass transfer rate in the
system, which leads to less material making it to the secondary magnetic pole
while maintaining the same amount of material reaching the primary maximum.
The cause of this decrease in the mass transfer rate is unclear, but may be
related to activity on the surface of the secondary star. Such a model has
been invoked to explain the transient two-pole accretion seen in QS Tel
(Schwope et al., 1995; Rosen et al., 1996) and MT Dra (Schwarz et al., 2002).

##  4 Optical Photometry

###  4.1 Light curves

The TESS light curve (which can be seen in Figure 3) is generally consistent
with previous optical observations of the system (Littlefield et al., 2015;
Littlefield et al., 2018), except that the secondary maximum does not stand
out as prominently. It has a lower amplitude, and often blends with the
primary maximum. This is probably attributable to differences in the cyclotron
continua of the two poles; time-resolved spectroscopy of a binary orbit
(Littlefield et al., 2018) shows that the continuum for the primary pole shows
more variability at the longer wavelengths which TESS is sensitive to. The
variability of the second pole’s cyclotron continuum increases at shorter
wavelengths, thereby explaining why the secondary pole is more pronounced in
optical observations than in the near-infrared TESS bandpass.

Figure 3: The full TESS light curve, phased to the orbital period. The width
of the sliding window is one-eighth of a day. The three horizontal white bands
indicate gaps due to data downlink. The central gap coincides with the
transition from Sector 15 to Sector 16, and because of the changed spacecraft
pointing, there is a brightness discontinuity at that gap.

We obtained one ground-based light curve of V496 UMa with the SLKT during each
sector of TESS observations, with the aim of ascertaining whether the
variability observed in the TESS bandpass is consistent with the variability
observed in previous optical studies. The overall shape of the light curve is
consistent across the two bandpasses as can be seen in Figure 4, and the
primary photometric maximum in the TESS light curve is the same as the primary
photometric maximum in the optical light curve. However, the relative
amplitude of the variation is reduced in the TESS bandpass, a likely
consequence of blending with nearby sources. Additionally, the rapid
flickering in the SLKT light curve is not always apparent in the TESS data,
possibly because the time resolution of the SLKT was superior by a factor of
$\sim$4.

Figure 4: Comparison of simultaneous light curves of V496 UMa obtained with
TESS and the SLKT. The SLKT data were obtained without a filter and use a
Johnson $V$ zeropoint, and the TESS data were converted to an instrumental
magnitude, with an arbitrary offset added. The TESS light curve shows a
decreased amplitude of variability, attributable to a combination of blending
and a bandpass difference.

###  4.2 Optical Ephemeris

A common method of measuring the orbital period in a polar is to measure the
recurrence interval of a well-defined feature in the light curve. At first
glance, the primary photometric maximum of V496 UMa is ideal for this purpose.
We fit third-order polynomials to each of the primary photometric maxima,
visually inspected the resulting fits to ensure their adequacy, and used each
polynomial to calculate the time of maximum flux for each peak. We used a
Monte Carlo procedure to estimate the uncertainty of each timing and
calculated a best-fit linear ephemeris of

$T_{max}[BJD]=2458722.01138(4)+0.0632329(2)$ (1)

for the TESS data. This period is very different from the period of
0.063235199(40) d reported in Littlefield et al. (2018). Inspection of the
residuals from the ephemeris (Fig. 5) suggest a possible explanation for this
discrepancy. The residuals from Eq. 1 show a systematic curvature consistent
with a gradual, aperiodic phase shift of the primary photometric maximum. On
relatively short timescales ($\lesssim 2$ weeks), a linear ephemeris can
compensate for this phase drift with a change in the apparent orbital period.
For example, the residuals in Fig. 5 are clustered into four groups (each
corresponding to one spacecraft orbit), and the best-fit periods for each of
the four groups differed from the Littlefield et al. (2018) period by up to
$\sim\pm 1$ s.

It is obviously unphysical for the binary orbit to change by such a large
amount in such a short time, but it is possible for the position of the
cyclotron-emitting region to drift across the face of the WD. Such behavior is
expected in a polar, as the location of the accretion region is not fixed to
the binary frame and depends on which field lines are channeling the infalling
matter. Variations in the mass-transfer rate could therefore cause a phase
shift of the accretion region, in which case one would also expect such a
change to produce observable luminosity variations. However, the O$-$C does
not show any significant correlation with the system’s brightness.

The detection of this oscillation is reminiscent of the aperiodic drift in the
optical maxima of the intermediate polar FO Aqr, identified by Kennedy et al.
(2017) using Kepler K2 data. Kennedy et al. (2017) demonstrated how this
effect can frustrate attempts to precisely measure the orbital period, but it
has not been previously reported in a synchronous polar. The drift observed in
V496 UMa is sufficiently small and gradual that poorly sampled ground-based
observations might struggle to distinguish between this effect and an
inaccurate measurement of the orbital period. It is unclear whether this phase
drift is a persistent feature of V496 UMa or whether it occurs in polars
generally, and but as TESS continues to observe polars, it will be possible to
search for this effect in other systems.

##  5 Discussion

The failed secondary maxima in the TESS observations can be broadly classified
into two categories: those that correlate with the system’s luminosity, and
those that do not. Littlefield et al. (2018) noted that at optical
wavelengths, the primary photometric maximum appeared to be unaffected by
nearby failed secondary maxima, and a number of the failed maxima in the TESS
light curve share this property. However, the TESS light curve shows multi-
day-long depressions near BTJD = 1730 and BTJD = 1740, and during these
episodes of reduced mass transfer, the failed secondary maxima are much more
frequent, occurring in a majority of the orbital cycles.

Although the failed maxima in Littlefield et al. (2018) created the impression
that there is a relatively clear dichotomy between normal and failed maxima,
the extensive TESS data demonstrate that this is not so in the near-infrared
TESS bandpass. On the contrary, the secondary maxima observed by TESS show
such a wide range of behaviours that it can be difficult to categorize some of
the maxima. Contamination from a nearby background star of similar brightness
further complicates matters, since it means that if V496 UMa were to become
undetectably faint, there would still be a weak signal at its position.

Figure 5: Top: The brightness of each accretion region during Sectors 15 and
16 of TESS. For both the primary and secondary maxima, we calculated the
average flux within $\pm$0.1 phase units of the expected phase of maximum
light. The secondary maxima show particularly erratic variability. Bottom:
O$-$C of the primary photometric maxima with respect to the orbital period
from Littlefield et al. (2018) and a reference time $T_{0}$ from the TESS
dataset. The O$-$C values display a slow, apparently aperiodic drift that does
not correlate with the brightness of either the primary or secondary maximum.

V496 UMa joins a growing list of polars which display dips due to obscuration
of the soft X-ray producing region by the ballistic stream. The XMM-Newton
observations strongly suggest that it is a two-pole accretor with highly
intermittent accretion onto its secondary magnetic pole, and that this
intermittent behaviour is being driven in changes in the mass transfer rate
from the donor star. The TESS light curve does not show the missing secondary
maxima as clearly as previous optical observations, and also suggest that the
ephemeris derived from short intervals of observations may be unreliable. We
speculate that this is because the cyclotron spectrum of the secondary
accretion region from Littlefield et al. (2018) is quite blue, so its relative
contribution in the TESS bandpass is relatively low.

###  5.1 Comparison with other systems

Two-pole accretion within polars is an informative phenomenon, as it allows us
to study both multiple accretion regions on the white dwarf’s surface, and
probe a larger volume of the WDs magnetosphere over the single pole case. This
is particularly powerful in systems with transient two pole accretion, where
the location within the magnetic field which the accretion stream is probing
varies. Two-pole accretion is not uncommon, and there are numerous instances
in the literature of polars that have switched between one- and two-pole
accretion, the prototype polar AM Her (Heise et al., 1985), MT Dra (Schwarz et
al., 2002), and QS Tel (Rosen et al., 1996) being excellent examples. It is
not always clear why the number of active poles changes, though Rosen et al.
(1996) and Schwarz et al. (2002) considered two hypotheses for QS Tel and MT
Dra, respectively: a change in the mass-transfer rate and asynchronous
rotation. In the former, the accretion stream’s ram pressure depends on the
mass-transfer rate, causing the stream to travel deeper into the magnetosphere
at higher mass-transfer rates. In the latter, the accretion stream would latch
onto different magnetic field lines due to the differential rotation of the
magnetosphere; these variations would occur at the beat frequency between the
binary orbital frequency and the WD’s spin frequency, which ranges from a few
days to $\sim$2 months in the known asynchronous polars.

The data presented here confirm that the transient two-pole accretion in V496
UMa is not due to asynchronous rotation of the WD. If this were the case, we
would have expected correlated changes in the primary and secondary maxima of
the X-ray light curve, and a long-term periodicity in the TESS light curve (as
observed in CD Ind; Hakala et al., 2019; Littlefield et al., 2019; Mason et
al., 2020). The absence of these effects suggest the driving force between the
variability in the secondary pole may be more akin to what is occurring in AM
Her and other synchronous polars.

Although the variable mass-transfer-rate explanation is more promising, it is
has its own shortcomings. If we use V496 UMa’s time-averaged optical
brightness as a proxy for its accretion rate, then there is no consistent
relation between its overall accretion rate and the failed secondary maxima;
in Figs. 3 and 5, the failed maxima are common during a dip near BTJD=1740,
but they also occur sporadically when V496 UMa is brightest. The lack of such
a correlation is reminiscent of the behaviour of AM Her, whose accretion
geometry does not always correlate strongly with the mass-transfer rate
(Schwope et al., 2020). Similarly, Beuermann et al. (2020) found that HY Eri
remains in a two-pole-accretion state even when the mass-transfer rate varies
by three orders of magnitude. However, BL Hyi provides a countervailing
example, as it undergoes two-pole accretion at enhanced accretion rates but
one-pole accretion in its low states (Beuermann & Schwope, 1989).

A related scenario considered by Rosen et al. (1996) to explain why QS Tel
changed between one- and two-pole accretion was that the accretion stream can
be fragmented into discrete blobs of varying densities. In such a case, the
lifetime of any individual blob depends on its density, with the densest blobs
surviving longer and traveling deeper into the WD’s magnetosphere. Based on
this picture, Rosen et al. (1996) proposed that low-density material becomes
magnetically entrained shortly after it leaves the donor star, producing a
hard X-ray-emitting region, while higher-density blobs travel to a secondary
accretion region with a softer spectrum. In this scenario, a temporary
reduction in the number of dense blobs could interrupt accretion onto the
second pole. It is worth noting that AM Her and MT Dra do conform to the Rosen
et al. (1996) scenario; when their secondary poles are active, they have
softer spectra than their primary poles (Schwope et al., 2020; Schwarz et al.,
2002).

However, if this mechanism were at play in V496 UMa we would have expected to
observe a pronounced difference in the X-ray hardness of the two poles. At
most, V496 UMa may display a slight enhancement of soft X-ray emission coming
from the secondary pole when is active and accreting; the blackbody component
may be slightly higher for the secondary maximum than for the primary
maxiumum, as given in Table 2. However, the enhancement is in no way
definitive, and higher signal-to-noise spectra are required to tell. Indeed,
the differences between the secondary and primary spectra are not nearly as
extreme as in the case of AM Her, suggesting V496 UMa is not undergoing blobby
accretion.

While V496 UMa does not offer an obvious answer as to why the secondary
maximum occasionally disappears, this system stands out because of the time
scale over which this takes place. For some of the best studied polars which
undergo mode switching, the accretion geometry appears to be relatively stable
during individual observing epochs. In contrast, the geometry in V496 UMa
changes over the course of a single orbit, as shown here, and few synchronous
polars have been observed to show such rapid optical changes in the accretion
rate onto a secondary pole. One such example is DP Leo (Beuermann et al.,
2014), which showed an intermittent secondary photometric maximum in optical
photometry. However, there is insufficient data about this phenomenon in DP
Leo to draw any robust comparisons with V496 UMa.

Whichever mechanism is altering the accretion geometry in V496 UMa (and
presumably DP Leo) varies over a short timescale. The most obvious culprit are
stellar spots on the secondary star moving across the L1 point. Verification
that this is driving a change in the mass transfer rate from the companion
star would require optical spectroscopic and photometric observations in which
the companion star dominates. This is a difficult task when accretion
structures and cyclotron harmonics are present in the system, and Littlefield
et al. (2018) were unable to detect the secondary star spectroscopically when
V496 UMa was in a high state. As such, V496 UMa should be monitored
frequenctly for the onset of a low state, at which point studies of the
secondary, and indeed measurement of the WD’s magnetic field through Zeeman
splitting of the absorption lines created in the WD photosphere, would become
possible.

## Acknowledgements

We thank the TESS mission-operations personnel, particularly George Ricker and
Roland Vanderspek, for scheduling DDT observations of V496 UMa after a last-
minute pointing change made it unexpectedly observable during Sectors 15 and
16. We thank the Krizmanich Family for their generous donation to the
University of Notre Dame that funded the Sarah L. Krizmanich Telescope.

M.R.K. acknowledges support from the ERC under the European Union’s Horizon
2020 research and innovation programme (grant agreement No. 715051; Spiders),
the Royal Society in the form of a Newton International Fellowship (NIF No.
NF171019), and the Irish Research Council in the form of a Government of
Ireland Postdoctoral Fellowship (GOIPD/2021/670: Invisible Monsters).

This work made use of Astropy (Astropy Collaboration et al., 2013, 2018),
Corner (Foreman-Mackey, 2016), and Pyxspec.

## Data availability

The raw X-ray data are available through the XMM-Newton Science Archive, while
the TESS data are available through the Barbara A. Mikulski Archive for Space
Telescopes. The exact X-ray spectra and light curves used in this paper can be
found at the following permanent repository:
https://zenodo.org/record/5746735.

## References

  * Arnaud (1996) Arnaud K. A., 1996, in Jacoby G. H., Barnes J., eds, Astronomical Society of the Pacific Conference Series Vol. 101, Astronomical Data Analysis Software and Systems V. p. 17
  * Astropy Collaboration et al. (2013) Astropy Collaboration et al., 2013, A&A, 558, A33
  * Astropy Collaboration et al. (2018) Astropy Collaboration et al., 2018, AJ, 156, 123
  * Bailer-Jones et al. (2021) Bailer-Jones C. A. L., Rybizki J., Fouesneau M., Demleitner M., Andrae R., 2021, AJ, 161, 147
  * Beuermann & Schwope (1989) Beuermann K., Schwope A. D., 1989, A&A, 223, 179
  * Beuermann et al. (2014) Beuermann K., Dreizler S., Hessman F. V., Schwope A. D., 2014, A&A, 562, A63
  * Beuermann et al. (2020) Beuermann K., Burwitz V., Reinsch K., Schwope A., Thomas H. C., 2020, A&A, 634, A91
  * Foreman-Mackey (2016) Foreman-Mackey D., 2016, The Journal of Open Source Software, 1, 24
  * Frank et al. (1988) Frank J., King A. R., Lasota J. P., 1988, A&A, 193, 113
  * Gaia Collaboration et al. (2016) Gaia Collaboration et al., 2016, A&A, 595, A1
  * Gaia Collaboration et al. (2021) Gaia Collaboration et al., 2021, A&A, 649, A1
  * Goodman & Weare (2010) Goodman J., Weare J., 2010, Communications in Applied Mathematics and Computational Science, 5, 65
  * Hakala et al. (2019) Hakala P., Ramsay G., Potter S. B., Beardmore A., Buckley D. A. H., Wynn G., 2019, MNRAS, 486, 2549
  * Hameury & King (1988) Hameury J. M., King A. R., 1988, MNRAS, 235, 433
  * Heise et al. (1985) Heise J., Brinkman A. C., Gronenschild E., Watson M., King A. R., Stella L., Kieboom K., 1985, A&A, 148, L14
  * Kennedy et al. (2017) Kennedy M. R., Garnavich P. M., Littlefield C., Callanan P., Mukai K., Aadland E., Kotze M. M., Kotze E. J., 2017, MNRAS, 469, 956
  * King & Lasota (1979) King A. R., Lasota J. P., 1979, MNRAS, 188, 653
  * Kuijpers & Pringle (1982) Kuijpers J., Pringle J. E., 1982, A&A, 114, L4
  * Lamb & Masters (1979) Lamb D. Q., Masters A. R., 1979, ApJ, 234, L117
  * Liebert & Stockman (1979) Liebert J., Stockman H. S., 1979, ApJ, 229, 652
  * Liedahl et al. (1995) Liedahl D. A., Osterheld A. L., Goldstein W. H., 1995, ApJ, 438, L115
  * Lightkurve Collaboration et al. (2018) Lightkurve Collaboration et al., 2018, Lightkurve: Kepler and TESS time series analysis in Python, Astrophysics Source Code Library (ascl:1812.013)
  * Littlefield et al. (2015) Littlefield C., Garnavich P., Magno K., Murison M., Deal S., McClelland C., Rose B., 2015, Information Bulletin on Variable Stars, 6129, 1
  * Littlefield et al. (2018) Littlefield C., Garnavich P., Hoyt T. J., Kennedy M., 2018, AJ, 155, 18
  * Littlefield et al. (2019) Littlefield C., Garnavich P., Mukai K., Mason P. A., Szkody P., Kennedy M., Myers G., Schwarz R., 2019, ApJ, 881, 141
  * Livio & Pringle (1994) Livio M., Pringle J. E., 1994, ApJ, 427, 956
  * Mason et al. (2001) Mason K. O., et al., 2001, A&A, 365, L36
  * Mason et al. (2020) Mason P. A., et al., 2020, Advances in Space Research, 66, 1123
  * Mewe et al. (1985) Mewe R., Gronenschild E. H. B. M., van den Oord G. H. J., 1985, A&AS, 62, 197
  * Mewe et al. (1986) Mewe R., Lemen J. R., van den Oord G. H. J., 1986, A&AS, 65, 511
  * Reimers et al. (1999) Reimers D., Hagen H. J., Hopp U., 1999, A&A, 343, 157
  * Ricker et al. (2015) Ricker G. R., et al., 2015, Journal of Astronomical Telescopes, Instruments, and Systems, 1, 014003
  * Rosen et al. (1996) Rosen S. R., et al., 1996, MNRAS, 280, 1121
  * Schmidt et al. (2005) Schmidt G. D., et al., 2005, ApJ, 620, 422
  * Schwarz et al. (2001) Schwarz R., Schwope A. D., Staude A., 2001, A&A, 374, 189
  * Schwarz et al. (2002) Schwarz R., Greiner J., Tovmassian G. H., Zharikov S. V., Wenzel W., 2002, A&A, 392, 505
  * Schwope et al. (1995) Schwope A. D., Thomas H. C., Beuermann K., Burwitz V., Jordan S., Haefner R., 1995, A&A, 293, 764
  * Schwope et al. (2000) Schwope A. D., Catalán M. S., Beuermann K., Metzner A., Smith R. C., Steeghs D., 2000, MNRAS, 313, 533
  * Schwope et al. (2020) Schwope A. D., Worpel H., Traulsen I., Sablowski D., 2020, A&A, 642, A134
  * Singh et al. (1996) Singh K. P., White N. E., Drake S. A., 1996, ApJ, 456, 766
  * Stockman et al. (1988) Stockman H. S., Schmidt G. D., Lamb D. Q., 1988, ApJ, 332, 282
  * Strüder et al. (2001) Strüder L., et al., 2001, A&A, 365, L18
  * Tapia (1977) Tapia S., 1977, ApJ, 212, L125
  * Turner et al. (2001) Turner M. J. L., et al., 2001, A&A, 365, L27
  * Watson et al. (1980) Watson M. G., Mayo S. K., King A. R., 1980, MNRAS, 192, 689
  * Wickramasinghe et al. (1989) Wickramasinghe D. T., Ferrario L., Bailey J., 1989, ApJ, 342, L35
  * Wilms et al. (2000) Wilms J., Allen A., McCray R., 2000, ApJ, 542, 914
  * Worpel & Schwope (2015) Worpel H., Schwope A. D., 2015, A&A, 583, A130
  * den Herder et al. (2001) den Herder J. W., et al., 2001, A&A, 365, L7

##  Appendix A Corner Plots

Figure 6: Corner plot from the MCMC analysis of the primary maximum spectrum
using the single temperature plasma and thermal black body model. Figure 7:
Corner plot from the MCMC analysis of the primary maximum spectrum using the
multi temperature plasma and thermal black body model.