# The Making and Breaking of Camouflage

Hala Lamdouar1 Weidi Xie1,2 Andrew Zisserman1
1 Visual Geometry Group, University of Oxford 2 CMIC, Shanghai Jiao Tong
University
<EMAIL_ADDRESS>
###### Abstract

Not all camouflages are equally effective, as even a partially visible contour
or a slight color difference can make the animal stand out and break its
camouflage. In this paper, we address the question of what makes a camouflage
successful, by proposing three scores for automatically assessing its
effectiveness. In particular, we show that camouflage can be measured by the
similarity between background and foreground features and boundary visibility.
We use these camouflage scores to assess and compare all available camouflage
datasets. We also incorporate the proposed camouflage score into a generative
model as an auxiliary loss and show that effective camouflage images or videos
can be synthesised in a scalable manner. The generated synthetic dataset is
used to train a transformer-based model for segmenting camouflaged animals in
videos. Experimentally, we demonstrate state-of-the-art camouflage breaking
performance on the public MoCA-Mask benchmark.

##  1 Introduction

Camouflage has long been a subject of interest and fascination for the
scientific community, especially evolutionary biologists who consider it as an
excellent example of species adaptation. In order to confuse predators or to
hide from prey, and increase their chances of survival in their natural
habitat, animals have developed numerous camouflage mechanisms, e.g.,
disruptive coloration and background matching. Some species have even evolved
to develop an adaptive camouflage, e.g., an arctic fox loses its white fur to
better match the brown grey of the new season’s landscape. Perhaps the most
dramatic camouflage adaptation is the cuttlefish; it changes its patterns
dynamically and rapidly as it moves from one spot to the other, constantly
adapting and improving its camouflage.

Figure 1: The three images depict the same animal, an Arctic fox, as it adapts
its appearance to better blend with the changing landscape of the new season.
While images $a$ and $c$ exhibit better background similarity than image $b$,
the fox boundary is more visible in image $a$ than image $c$. We assess the
effectiveness of camouflage by measuring the degree of ambiguity it creates
with respect to its background.

This search for optimal camouflage has inspired numerous works in the computer
vision community, such as [27, 12], that tackled camouflage as a problem of
optimal texture synthesis, making 3D objects non-detectable in a given scene.
Others have addressed camouflage as a highly challenging object segmentation
task in [9, 20, 23, 15]. Efforts have been made in collecting large scale
camouflage datasets [9, 20] with costly annotation. In fact, camouflaged
animals often exhibit complex shapes and thin structures that add to the
boundary ambiguity and make the annotation highly time-consuming. Fan et al.
report up to $60$min per image to provide accurate pixel-level annotations for
their dataset COD10K [9]. While another line of research turned to camouflage
breaking in sequences by taking advantage of motion cues [19, 1, 18, 40, 5],
the camouflage data scarcity is even more extreme for videos. Recently,
Sim2Real [18, 39] training has shown to be very effective for motion
segmentation. By training on the optical flows of synthetic videos, these
models can generalise to real videos without suffering from the domain gaps.

In this paper, we start by asking the question: “what are the properties of a
successful camouflage ?” To answer this question, we investigate three scoring
functions for quantifying the effectiveness of camouflage, namely,
reconstruction fidelity score ($S_{R_{f}}$), boundary score ($S_{b}$) and
intra-image Fréchet score ($d_{\mathcal{F}}$). These scores are later adopted
for two critical roles, (i) they are used to assess the relevance of existing
camouflage datasets and act as quality-indicator in collecting new camouflage
data; (ii) they can be used as a proxy loss for image and video
inpainting/generation, where we establish a synthetic camouflage data
generation pipeline with a Generative Adversarial Network (GAN), that can
simultaneously generate high-quality camouflage examples and masks of the
camouflaged animals. We further train a Transformer-based architecture on the
generated camouflage video sequences, and demonstrate a performance boost over
training on only the (small scale) real data.

In summary, we make the following contributions: First, we introduce three
scoring functions to measure the effectiveness of a given camouflage in still
images and videos. We use these camouflage scores to rank the images in
existing datasets in terms of camouflage success, and also show that the
rankings are largely in accordance with human-produced rankings. Second, we
incorporate the camouflage score into a generative model, establishing a
scalable pipeline for generating high-quality camouflage images or videos,
along with the pixelwise segmentation mask of the camouflaged animals; Third,
we show that a Transformer-based model trained on the synthetically generated
data can achieve state-of-the-art performance on the MoCA-Mask video
camouflaged animal segmentation benchmark.

##  2 Related work

Camouflage Evaluation. Although there are no previous computational works that
directly assess camouflage, as far as we know, there have been a number of
human studies. Previous works proposed methods to evaluate camouflage by
analysing the human viewers’ eye movements [22, 37] or conducting subjective
perceptual experiments [27, 12, 32]. In [27, 12], participants were asked to
indicate the camouflaged object and their answers were analysed in terms of
accuracy and time needed per image. Similarly, Skurowski et al. [32] asked
human volunteers to rate CHAMELEON [32] images from 1 to 5 in terms of
camouflage effectiveness. They produced a score after compensating for
personal bias. We validate our proposed camouflage scoring functions by
comparing our rankings to these human-based rankings.

Motion Segmentation. The goal of this task is to partition the frames of a
video sequence into background and independently moving objects. Early
approaches focused on clustering motion patterns by grouping pixels with
similar trajectories [3, 24, 17, 31, 21, 25, 26]. Another line of research
tackled the problem by compensating for the background motion [1, 2, 19], via
registering consecutive frames, or explicitly estimating camera motion. More
recently, in [40], a Transformer-like architecture is used to reconstruct the
input flows, and the segmentation masks are generated as a side product, while
[6] decomposes the flow filed into regions by fitting affine transformations.
The most closely related to ours is [18, 39], where the authors adopt a
Sim2Real strategy by training the model on optical flow computed from
synthetic videos. These models can generalise towards real videos without
fine-tuning. In this paper, we train a model on synthetic camouflage RGB
sequences.

Fréchet Inception Distance and its variants. Also known as Wasserstein-2
distance [38], Fréchet distance (FD) is a metric quantifying the difference
between two probability distributions [10]. Lately, [13] introduced the
Fréchet Inception Distance (FID) in the context of generative models. Under
the assumption that real images and generated images follow Gaussian
distributions, they compute the FD of the two distributions. Note that here,
each image is represented by a vector obtained from the last pooling layer of
InceptionV3 [35]. Following their steps, [30] adapted FID to the image re-
targeting task and introduced the Single Image Fréchet Inception Distance
(SIFID). Instead of comparing entire image datasets, they compare two images
and consider the distributions of their feature vectors obtained from earlier
layer of InceptionV3 [35]. Recently, [7] included FID in the training
objective and introduced Fréchet-GAN. Our work takes inspiration from [30, 7]
and investigates a FID of regions within the same image as an auxiliary loss.

Figure 2:  The reconstruction fidelity score $S_{R_{f}}$ evaluates the
similarities between the original foreground $I_{fg}$ and its reconstruction
from background features $\Psi_{bg}(I_{fg})$. The top example shows a case
where the animal exhibits different visual patterns (color) from its
background ($S_{R_{f}}$=0.11), while the bottom example shows a better
background matching ($S_{R_{f}}$=0.82).

Figure 3: Trimap regions and animal contours. From left to right: Input image;
Trimap partition (foreground region in yellow, background in purple, and
boundary region in green); Contours $C$ computed on the original image; and
ground truth Contours $C_{gt}$. The top example shows less contour agreement
along the boundary region ($S_{b}$=0.72) than the bottom example
($S_{b}$=0.40).

##  3 Measuring the effectiveness of camouflage

Assuming there exists a dataset,
$\mathcal{D}=\\{(I_{1},m_{1}),\dots,\allowbreak(I_{N},m_{N})\\}$, where
$I_{i}\in\mathbb{R}^{H\times W\times 3}$ refers to the images, and
$m_{i}\in\\{0,1\\}^{H\times W\times 1}$ denotes a binary segmentation mask of
the camouflaged animal. Here, we aim to design a scoring function that takes
the image and segmentation mask as input, and outputs a scalar value $S$ that
can quantify the effectiveness of camouflage, i.e., how successfully an animal
blends into its background, termed as the camouflage score,

$\displaystyle S:(I_{i},m_{i})$ $\displaystyle\longmapsto s_{i}\in[0,1]$

specifically, for $i\neq j$ and $s_{i}<s_{j}$, the animal in $I_{j}$,
indicated by the mask $m_{j}$, is more concealed than the animal in $I_{i}$,
indicated by the mask $m_{i}$. Having such a scoring function enables various
applications, first, it enables to rank the images from one dataset, in terms
of the effectiveness of camouflage, and generate camouflage-relevant
statistics for the dataset; second, it can serve as the objective for low-
level image generation tasks, for example, image inpainting, editing, etc.

In the following sections, we investigate three such scoring functions,
specifically, we exploit the animal’s mask and quantify the key aspects that
determine the camouflage success both perceptually (Sec. 3.1) and
probabilistically (Sec. 3.2). Note that, we take into account the local aspect
of camouflage, i.e., instead of processing the entire background region, we
only focus on the immediate surround of the animal. Hence, when referring to
background, we mean the local background and only consider the cropped images
centred around the object.

###  3.1 Perceptual camouflage score

In this section, we define the camouflage score through a perceptual lens,
specifically, we measure the foreground and background similarity (Sect.
3.1.1), along with the boundary visibility (Sect. 3.1.2).

####  3.1.1 Reconstruction fidelity score

We attempt to reconstruct the foreground region with patches from the
background, and propose a score to quantitatively measure the discrepancy
between the original image and its reconstruction. Intuitively, for successful
camouflage, we should be able to reconstruct the foreground perfectly by
copying patches from its close neighborhood.

Formally, for a given image ($I$) and segmentation mask ($m$), subscript has
been ignored for simplicity, we consider a trimap separation and define the
foreground and background regions using morphological erosion and dilation of
the mask,

$\displaystyle I_{\text{fg}},\text{ }I_{\text{bg}}=m_{\text{fg}}\odot I,\text{
}m_{\text{bg}}\odot I$ (1)

where $m_{\text{fg}}=\text{erode}(m)$ and $m_{\text{bg}}=1-\text{dilate}(m)$.

To reconstruct the foreground region, we replace the foreground patch with the
closest one in the background. Here a patch is a $n\times n$ pixel region
($n=7$ in our case with an overlap of 3) and the the patchwise similarity is
computed by exploiting the low-level perceptual similarity, i.e., comparing
corresponding RGB values. This reconstruction method is inspired by the
texture generation and inpainting approaches, such as [8]. In practice, we
implement it with fast approximate nearest neighbor search for efficiency.

The reconstruction fidelity score is computed by assessing the difference
value between the foreground region and its reconstruction, specifically, we
count the number of foreground pixels that have been successfully
reconstructed from the background:

$\displaystyle S_{R_{f}}(I,m)=\frac{1}{N_{\text{fg}}}\sum_{(i,j)\in
I_{\text{fg}}}R_{f}(i,j)$ (2) $\displaystyle
R_{f}(i,j)=\begin{cases}1&\mbox{if
}||I_{\text{fg}}-\Psi_{I_{\text{bg}}}(I_{\text{fg}})||_{2}<\lambda||I_{\text{fg}}||_{2}\\\
0&\mbox{otherwise}\end{cases}$ (3)

$\Psi_{I_{bg}}(.)$ denotes the reconstruction operation,
$N_{\text{fg}}=|m_{\text{fg}}|$ is the total number of pixels in the
foreground region, and $\lambda$ is a threshold ($\lambda$=0.2 in our case). A
higher $S_{R_{f}}$ score, means that the animal’s visual attributes are well
represented in the background. Conversely, a low $S_{R_{f}}$ indicates that
the animal’s appearance is composed of unique features that makes it standout
from its surrounding. In Fig. 3, we present examples of camouflage evaluation
by reconstruction fidelity.

####  3.1.2 Boundary visibility score

With the trimap representation, we can easily extract the boundary region,
$I_{\text{b}}=m_{\text{b}}\odot I$, where
$m_{\text{b}}=(1-m_{\text{bg}})-m_{\text{fg}}$. Here, we aim to measure the
animal’s boundary properties, e.g., contour visibility, as they can also
provide visually evident cues for camouflage breaking. In particular, we adopt
an off-the-shelf contour extraction method [29], and run it on the original
image and foreground mask, to generate the images’ contour ($C$) and ground
truth animal’s contour ($C_{\text{gt}}$), as shown in Fig. 3.

We express the agreement between predicted contours $C$ and ground truth
contours $C_{\text{gt}}$ over the boundary region with the F1 metric:

$\displaystyle S_{b}(I,m)=1-\text{F1}(m_{b}\odot C_{\text{gt}},\text{
}m_{b}\odot C)$ (4)

This score penalises the boundary pixels that are predicted as contour in both
$C$ and $C_{\text{gt}}$. While we do not expect a perfect contour agreement
for visible edges, we consider $S_{b}$ as a reasonable approximation, given
the shape and size of the boundary region as a thin margin centred around the
animal. If a boundary pixel is predicted as a contour in $C_{\text{gt}}$ but
not in $C$, this means that the animal’s edge is not visible in the original
image. If a pixel is predicted as a contour in $C$ but not $C_{\text{gt}}$,
this indicates the presence of distracting elements, such as complex texture
patterns on the foreground animal, which also affects the visibility of the
animal’s contour and therefore improves its camouflage. A perfectly
camouflaged animal would have little contour agreement and $S_{b}=1$.

To get the final perceptual camouflage score, we linearly combine both
reconstruction fidelity score and boundary visibility score:

$\displaystyle S_{\alpha}=(1-\alpha)S_{R_{f}}+\alpha S_{b}$ (5)

We will describe our method for setting the weighting parameter $\alpha$ in
the Experiment section (Sec. 6).

###  3.2 Probabilistic scoring function

In addition to using the low-level RGB representation, in this section, we
consider the pixelwise representation in the feature space, and propose a
differentiable metric that compares the probabilistic distribution between the
foreground and background regions, which acts as a proxy for the score
$S_{\alpha}$ described in previous section, and can be used directly as a
differentiable loss function in image generation.

We consider the Intra-Image Fréchet Inception Distance, specifically, we
compute the feature maps for the foreground and background image regions:

$\displaystyle f_{fg},\text{ }f_{bg}=\Phi_{\text{v}}(I_{fg}),\text{
}\Phi_{\text{v}}(I_{bg})$ (6)

Here $\Phi_{\text{v}}(\cdot)$ denotes a pre-trained Inception network [34,
35], truncated at early layers. We take inspiration from [30], and adapt the
single image Fréchet distance to measure the deviation between feature
distributions of different regions within the same image. We adopt the Fréchet
hypothesis with respect to our regions, i.e., the features of the foreground
and background follow multivariate Gaussian distributions:
$f_{\text{fg}}\sim\mathcal{N}(\mu_{\text{fg}},\Sigma_{\text{fg}})$ and
$f_{\text{bg}}\sim\mathcal{N}(\mu_{\text{bg}},\Sigma_{\text{bg}})$. The intra-
image Fréchet distance can be formulated as follows:

$\displaystyle{\color[rgb]{0,0,0}d_{\mathcal{F}}^{2}(I,m)=||\mu_{\text{fg}}-\mu_{\text{bg}}||_{2}^{2}+Tr(\Sigma_{\text{fg}}+\Sigma_{\text{bg}}-2(\Sigma_{\text{fg}}\Sigma_{\text{bg}})^{1/2})}$

Note that, when $(I,m)$ is the output from a generative model,
$d_{\mathcal{F}}^{2}$ is differentiable with respect to its parameters, and
can therefore be used as an auxiliary loss for optimising the image generation
procedure, i.e., generate effective camouflage examples.

Figure 4: Our framework consists of a synthetic camouflage data generation
pipeline (left), where we train a generator $G$, in a GAN setting, to create
camouflage images and masks, while encouraging camouflage effectiveness via
minimising $\mathcal{L}_{\mathcal{F}}$. The generated samples are then
transformed into synthetic video sequences (middle), following our method
presented in Sec. 4.2. The transformer-based motion segmentation architecture
(right), for camouflage breaking in videos, is trained on the synthetic video
sequences. The architecture is described in Sec. 5.

##  4 Generating camouflage video sequences

In this section, we propose a scalable pipeline for generating images with
concealed animals. In particular, we incorporate the differentiable intra-
image Fréchet distance in image generation, explicitly optimising the
camouflage score of the image. We first detail the static image generation
framework, then describe how we use the images it generates to create
camouflage video sequences.

###  4.1 Camouflage image generation

In order to generate the image with a camouflaged animal, and its
corresponding segmentation mask, we adopt a Generative Adversarial Network
(GAN), where a generator is fed with a latent vector $z\sim\mathcal{N}(0,1)$
and learns to predict the pair of realistic image and segmentation mask, such
that a discriminator cannot differentiate it from real samples.

Let $(x_{i},m_{i})$ denote an image and segmentation pair sampled from the
training dataset, a subset of COD10K [9] is used here. We adopt the
conventional label notation, i.e. $y=1$ for (real) examples sampled from the
training dataset and $y=0$ for (fake) examples from the generator, and
consider the generative adversarial loss functions [11]:

$\displaystyle\mathcal{L}_{D}$
$\displaystyle=E_{\scaleto{(x_{i},m_{i})}{0.6em}}[-\log(D(x_{i},m_{i}))]+E_{z}[-\log(1-D(G(z)))$
$\displaystyle\mathcal{L}_{G}$ $\displaystyle=E_{z}[-\log(D(G(z)))]$

To enforce coherence between images and their segmentation masks, we feed the
discriminator with additional fake pairs, consisting of real images coupled
with unpaired real masks. This can be formulated as an additional coherence
loss term that minimises the probability of assigning incorrect labels, i.e.
$y=1$, to $(x_{i},m_{j})$ sampled from the training set such that $i\neq j$:

$\displaystyle\mathcal{L}_{\rm Coh}=E_{\scaleto{(x_{i},m_{j})_{i\neq
j}}{0.6em}}[-\log(1-D(x_{i},m_{j}))]$ (7)

To increase the camouflage effectiveness in the generated examples, we adopt
the intra-image Fréchet distance in the generator loss, as an auxiliary loss
term, and train our camouflage image generator with the following loss:

$\displaystyle\mathcal{L}_{\tilde{D}}=\mathcal{L}_{D}+\mathcal{L}_{\rm
Coh}~{}~{}~{}~{}~{}~{}\mathcal{L}_{\tilde{G}}=\mathcal{L}_{G}+\beta
d^{2}_{\mathcal{F}}$ (8)

We present an overview of our data generation framework in Fig. 4.

###  4.2 Camouflage video generation

Given a camouflage image and corresponding segmentation mask, generated using
the method above, we can create video sequences by applying different motions
to the foreground and background, in a similar manner to [18]. Specifically,
we first inpaint the backgrounds with an off-the-shelf model proposed by
Suvorov et al. [33] and overlay the synthetic animal (extracted from the
original generated image by using the mask) at a random location within the
image in the first frame, then following a random translational trajectory in
the following frames. We incorporate a different translational motion for the
background and include random sub-sequences where the foreground and
background undergo the same motion to simulate momentarily static objects. As
we train our generator with intra-image Fréchet loss
($\mathcal{L}_{\mathcal{F}}$), the generated images exhibit strong
similarities between the foreground and the background, and even if the object
is placed at a different location from the original, we expect it to remain
concealed within its surrounding.

##  5 Learning to break camouflage

In this section, we train a transformer-based architecture on the synthetic
dataset, and demonstrate its effectiveness for breaking the camouflage in real
videos, for example, MoCA [19]. We build on two previous architectures,
namely, the motion segmentation from [18] and the Search Identification
Network from [9]. While the first architecture processes sequences of optical
flow, the second takes single RGB images as inputs and treats them separately.
Our proposed model, shown in Fig. 4, takes both the optical flow and RGB frame
sequences as input. The flow is computed with RAFT [36] from the RGB sequence,
and processed with a motion encoder, followed by a transformer encoder. The
RGB sequence is processed with the appearance encoder from [9], pre-trained
for framewise camouflage segmentation. Then both streams are aggregated as
inputs to a Mask2Former-like [4] transformer decoder and pixel decoder to
produce the high resolution segmentation mask from the motion stream.

Motion encoder. To encode the motion cues, we use a light-weight convNet that
takes as input a sequence of optical flows,
$\bf{I_{m}}$$=\\{I_{m_{1}},I_{m_{2}},..,I_{m_{t}}\\}\in\mathcal{R}^{t\times
c_{0}\times h\times w}$ and outputs motion features:

$\displaystyle\\{f_{m_{1}},f_{m_{2}},..,f_{m_{t}}\\}=\Phi_{\text{motion}}(\bf{I_{m}})$

where each flow frame is separately embedded.

Motion transformer encoder. A transformer encoder takes as input the motion
features, concatenated along the sequence, together with learned spatial and
temporal positional encodings ($\text{Pos}_{s}$, $\text{Pos}_{t}$), as
indicated in Fig 4. $\text{Pos}_{t}$ specifies the frame, and $\text{Pos}_{s}$
specifies the position in the spatial feature map output of the motion
encoder. The output of the transformer is a set of enriched motion features.

Appearance encoder. Here, we adopt a SINet-v2 [9] architecture, that encodes
the RGB sequence,
$\bf{I_{a}}$$=\\{I_{a_{1}},I_{a_{2}},..,I_{a_{t}}\\}\in\mathcal{R}^{t\times
c_{1}\times h\times w}$ into appearance features:

$\displaystyle\\{f_{a_{1}},f_{a_{2}},..,f_{a_{t}}\\}=\Phi_{\text{app}}(\bf{I_{a}})$

Again, each RGB frame is processed separately by SINet.

Transformer decoder. A transformer-based decoder takes the output of the
motion transformer encoder together with the appearance features and a
learnable query for the mask embedding. In a similar manner to Mask2Former
[4], the query attends to multiple resolutions of the motion features
concatenated with the appearance features and produces a mask embedding for
the moving object.

Pixel Decoder. Similarly to the pixel decoder in Mask2Former, a light-weight
convNet decoder is used with skip-connections to recover high-resolution
segmentation masks $\\{\hat{m}_{1},\hat{m}_{2},..,\hat{m}_{t}\\}$ from the
motion features and the mask embedding. This is shown as the blue box in Fig.
4.

Training objective. We train the motion segmentation model on our synthetic
video sequences using the binary cross-entropy loss $\mathcal{L}_{BCE}$.

##  6 Experiments

In this section, we start by introducing the datasets involved in this paper,
followed by the implementation details. Our experiments present a thorough
analysis of the proposed camouflage scores and demonstrate their effectiveness
in our training framework.

###  6.1 Datasets

Here we describe the publicly available camouflage datasets that we included
in our experiments, as shown in Fig. 5, as well as the synthetic camouflage
datasets that we generated.

CHAMELEON [32] is one of the first camouflage image datasets. It contains 76
images collected using Google image search with the ‘camouflaged animal’
keyword and include ground truth manual annotations.

CAMO [20]. The Camouflaged Object dataset consists of 2500 images collected
from the internet, of which 1250 images (1000 training sub-set and 250 testing
sub-set) contain at least one camouflage instance from 8 categories with
manual pixelwise annotations provided.

COD10K [9]. COD10K contains 10,000 images collected from photography websites
of which 5066 depict camouflaged animals (3040 training sub-set and 2026
testing sub-set), organised into 10 classes, and 78 sub-classes (69
camouflaged). Note that, in our camouflage evaluation experiments we only used
the camouflage subset of this dataset.

Camouflaged Animals [1]. This dataset consists of 9 video sequences of
camouflage animals from 6 categories.

MoCA [19] is the first large-scale camouflage video dataset. It contains 141
video sequences (37K frames) totalling 67 animal categories. Recently, other
works have curated this dataset by selecting only the videos with more
prominent motion (locomotion) [40], and others have provided dense pixel
annotation in MoCA-Mask [5]. We use the latter version in our experiments.

Camouflaged cuboids [27, 12]. This dataset was created for texture synthesis
and consists of multiple-view scenes, where cuboids were placed at a
predefined location then synthetically camouflaged. In our evaluation, we
consider the 4-views generated textures [12] from 36 scenes as well as the
cuboids masks.

Synthetic Camouflage Images. Using the method described in Sec. 4.1, we
generate a synthetic camouflage dataset, of $5K$ frames, by discriminating
against real camouflage image and annotation masks from COD10K.

Synthetic Camouflage Videos. We generate $1K$ sequences of $30$ frames each,
incorporating static sequences using the framework described in Sec. 4.2. We
split these into $800$ sequences for training and $200$ for testing.

Figure 5: Randomly sampled examples from the camouflage datasets included in
our work. For the video datasets, Camouflaged Animals and MoCA-Mask, we show
an example sequence. For the multiple-view dataset Camouflaged cuboids
(bottom), we show example views from a scene with the 4-view texture synthesis
method from [12].

###  6.2 Implementation details

In our experiments, for a given camouflage example, the kernels for the
morphological operations are selected from a range of values [1,10], so that
the resulting annotation mask is reduced by 20% for erosion and extended by
20% for dilation. This allows the erosion/dilation to be adapted to the size
of the camouflaged animal, while always keeping a reasonably large region for
addressing the boundary effects.

Figure 6: Left: Top-4 scored examples from COD10K for $S_{Rf}(top)$,
$S_{b}(middle)$ and $S_{\alpha}(bottom)$. Right: Lowest scored examples from
COD10K. For each example, we show the obtained score and the corresponding
ground truth mask.

For our synthetic camouflage image generation, we adopt a Style-GAN
architecture [41] and train it on the camouflage images from COD10K to
generate $256\times 256$ images and masks pairs. We use $\beta=0.1$ to weight
our intra-image Fréchet auxiliary loss. For numerical stability, we adopt the
Newton Schultz iterations when calculating the matrix square root term in
Intra-Image Fréchet Inception Distance, with $T=100$ iterations.

When generating the image sequences, we compute optical flows with RAFT [36],
and train our moving camouflaged animal segmentation model. We use batches of
size 2 and sequences of size 5. We first train on our synthetically generated
dataset with a learning rate of $5\times 10^{-4}$ for 500 iterations then
fine-tune on the training subset of MoCA-Mask.

###  6.3 Results

Datasets Data type $S_{R_{f}}\uparrow$ $S_{b}\uparrow$ $S_{\alpha}$$\uparrow$
$d^{2}_{\mathcal{F}}\downarrow$ CHAMELEON [32] Image 0.694 0.445 0.607 0.70
CAMO Train [20] Image 0.672 0.451 0.595 1.01 CAMO Test [20] Image 0.683 0.470
0.608 1.00 COD10K Train [9] Image 0.655 0.433 0.577 0.90 COD10K Test [9] Image
0.657 0.431 0.578 0.90 Camouflaged Animals [1] Video 0.674 0.536 0.626 1.60
MoCA-Mask Train [19, 5] Video 0.850 0.443 0.707 1.14 MoCA-Mask Test [19, 5]
Video 0.733 0.464 0.639 2.51 Camouflaged cuboids[27, 12] Multi-view 0.894
0.433 0.733 6.2 Syn. Camouflage w.o. $\mathcal{L}_{\mathcal{F}}$ Image 0.608
0.432 0.546 1.36 Syn. Camouflage w. $\mathcal{L}_{\mathcal{F}}$ Image 0.679
0.447 0.598 1.13 Syn. Camouflage Video Video 0.658 0.430 0.578 1.18 Table 1:
Results of the proposed scores on natural camouflage datasets (top) and
synthetically generated camouflage datasets (bottom). We report the mean
scores for the single image datasets. For the video and multiple view
datasets, we compute the mean per sequence and scene respectively. Syn.
Camouflage w. $\mathcal{L}_{\mathcal{F}}$ refers to the synthetic image
dataset that we generated while minimising $\mathcal{L}_{\mathcal{F}}$ and
Syn. Camouflage Video its sequence version.  Scores CHAMELEON [32] Camouflaged
cuboids[27, 12] $S_{R_{f}}$ 0.01 -0.07 $S_{b}$ 0.03 0.42 $S_{\alpha}$ 0.42
0.41 $d^{2}_{\mathcal{F}}$ 0.10 -0.17 Table 2: Kendall-$\tau$ metric between
rankings produced via our scores and human scoring ground truth. Model
Training Appearance Transformer Transformer mIoU$\uparrow$ $F\uparrow$
$E\uparrow$ MAE$\downarrow$ dataset Encoder Encoder Aggregation A S 14.5 20.4
57.3 9.4 B MM 15.3 20.6 57.3 5.1 C S+MM 16.0 21.8 59.8 4.3 D S+MM ✓ 20.5 22.6
59.8 3.8 E S+MM ✓ ✓ 23.0 23.5 57.0 1.6 F S+MM ✓ ✓ 22.7 23.2 58.8 3.6 G MM ✓ ✓
✓ 23.4 24.7 62.0 2.4 H S+MM ✓ ✓ ✓ 30.8 34.3 77.0 1.8 Table 3: Ablation study
on the different components of the motion segmentation architecture and
training datasets. We evaluate on MoCA-Mask for all experiments. (S: our
synthetic dataset, MM: MoCA-Mask).

This section presents qualitative and quantitative results, to demonstrate the
effectiveness of our score functions and the benefit of including
$d^{2}_{\mathcal{F}}$ in the training loop.

####  6.3.1 Evaluation on camouflage effectiveness

Ranking camouflage images in COD10K. We can rank the images based on the
effectiveness of the animal’s camouflage, with our proposed scoring functions.
In the left part of Fig. 6, we show the four best scored images from the large
scale COD10K dataset. We can make the following observations: (i) the best
examples with respect to $S_{R_{f}}$ (top), which favours the background
matching of the animals without taking into account the boundary region, all
exhibit visible boundaries, with the exception of image (a). This is
especially noticeable in the rabbit example (d) along the ears and shadow
regions; (ii) the best four examples with respect to the boundary score
$S_{b}$ are all from the caterpillar subclass, mostly the baron type. These
insects have thin, and transparent boundaries, that makes them the perfect
candidates for this score. However, with the exception of (a) and (c), the
animals still stands out as they exhibit colors and patterns that are not
present within their background; (iii) the $S_{\alpha}$ combining both scores
shows highly effective camouflage examples. In contrast, the right part of
Fig. 6 shows the lowest scored images for each approach. For $S_{R_{f}}$, we
find examples with low background matching in the first row, with two ant
examples, i.e., (e), (f). While the boundary score penalises examples with
high contrasts between the animal and its background, that results in more
visible contours. This is also the case for the lowest $S_{\alpha}$ score
examples.

Dataset level camouflage comparison. We compute the camouflage score for all
the camouflage image and video datasets and report the results in Tab. 3. For
the image datasets, the dataset level score is computed as the mean of the
image scores; for the video dataset the score is first computed per frame, and
per-video average computed, then the dataset level score is the mean of the
averages. Our experiments show that MoCA-Mask [19, 5] contains the most
successful camouflages according to our scoring. While COD10K [9] subsets are
balanced in terms of camouflage effectiveness, we find that, for CAMO [20],
the test subset yields higher scores than the training subset and therefore
contains more challenging examples.

We note that Camouflaged cuboids dataset yields higher $S_{R_{f}}$ and
$S_{\alpha}$ scores compared to our synthetic datasets. This is due to the
fact that the model used in [27, 12] is only trained to produce optimal
texture for a predefined region (cuboid) in a particular location of a given
scene. However, our model learns to output a new image with a randomly located
camouflaged object of random shape, thus offering more scalability and
diversity compared to the cuboids dataset which cannot be used to train a
model for breaking real camouflage.

Comparison to human-produced rankings. We compare our rankings to the human
scoring system based on the rating of CHAMELEON [32] and the time-to-find for
Camouflaged cuboids [12]. To search for the optimal $\alpha$ parameter in
Equation (5), we select $15$ images from CHAMELEON and compare their
$S_{\alpha}$ ranking with ground truth using kendall-$\tau$ metric [16]. For
$\alpha=0.35$, we obtain kendall-$\tau=0.51$ on this validation set that we
excluded from our test reported in Tab. 3. We can draw the following
observations: (i) we find that the boundary score $S_{b}$ produces more
agreement with the ground truth ranking than $S_{R_{f}}$, suggesting that
human observers tend to pay more attention to contour visibility than
background matching; (ii) while comparing to the ranking from Camouflaged
cuboids, we found negative correlations for $S_{R_{f}}$ and $d_{\mathcal{F}}$,
we conjecture that this may be due to the nature of the dataset in [12], i.e.,
all the textured cuboids are synthetically generated with very high background
matching; (iii) for both experiments, we obtain the strongest agreement with
the $S_{\alpha}$ score combining both background similarity and boundary
blending.

Further analysis of the synthetic datasets. We adopt the FID [13] and IS [30]
metrics in Table 4. These metrics assess: (i) the similarity between the set
of synthetic images output by the generator and the set of real images used to
train it (FID); and (ii) the clarity of the object and the diversity of images
in terms of classes (IS). While they are not intended for measuring camouflage
success, which is the focus of this work, IS could somewhat be (inversely)
linked to $S_{b}$ as the object clarity and boundary visibility are close.
Note that the object clarity (IS) is decreased with
$\mathcal{L}_{\mathcal{F}}$ which is the intended purpose of improving
camouflage, however, this effect is not maintained in the sequence generation
as the animal changes location within its background.

dataset |  FID$\downarrow$ |  IS$\uparrow$
---|---|---
Syn. Camouflage w.o. $\mathcal{L}_{\mathcal{F}}$ |  75.01 |  3.49
Syn. Camouflage w. $\mathcal{L}_{\mathcal{F}}$ |  76.34 |  3.13
Syn. Camouflage Video |  86.42 |  3.85

Table 4: Overall quality of our synthetic datasets. Figure 7: Generated
camouflage images and masks: For the examples on the right, the generator was
trained with $\mathcal{L}_{\mathcal{F}}$.

Figure 8: Qualitative results on MoCA-Mask. From top to bottom: Appearance
sequence $I_{a}$, flow sequence $I_{m}$, predicted segmentations and ground
truth segmentations.

Model | RGB | Motion | mIoU$\uparrow$ | $F\uparrow$ | $E\uparrow$ | MAE$\downarrow$
---|---|---|---|---|---|---
SINet [9] | ✓ |  | 20.2 | 23.1 | 69.9 | 2.8
SINet-V2 [9] | ✓ |  | 18.0 | 20.4 | 64.2 | 3.1
SegMaR [14] | ✓ |  | 12.2 | 22.5 | 80.3 | 4.5
ZoomNet [28] | ✓ |  | 18.8 | 28.7 | 70.8 | 2.5
SLTNet [5] | ✓ | ✓ | 27.2 | 31.1 | 75.9 | 2.7
MG [40] |  | ✓ | 12.7 | 16.8 | 56.1 | 6.7
Ours-flow |  | ✓ | 17.8 | 21.5 | 60.7 | 3.7
Ours | ✓ | ✓ | 30.8 | 34.3 | 77.0 | 1.8

Table 5: Results on MoCA-Mask. Ours-flow refers to the optical flow only
version of our architecture, excluding the appearance stream.

####  6.3.2 Model ablations

On the effectiveness of $\mathcal{L}_{\mathcal{F}}$. Fig. 7 shows the
generated samples from the GAN pipeline, trained with and without the intra-
image Fréchet Distance. Our generator is able to produce realistic object
masks of complex shapes and thin structures similar to those usually
encountered in the camouflage datasets. The produced images exhibit rich
backgrounds with realistic nature-like elements mimicking rocks and coral
structures present in the real camouflage data. Adding the
$\mathcal{L}_{\mathcal{F}}$ loss results in generating images with better
blending in the background, both qualitatively and quantitatively, as shown by
the $S_{\alpha}$ computed for both datasets generated with and without the
intra-image Fréchet Distance in Tab. 3. We refer the reader to the
supplementary material for more examples of generated samples.

On the architecture design and gain from the synthetic dataset. We present in
Tab. 3 an ablation study detailing the gain from the main components of our
architecture, as well as the benefit of first training on our synthetic
dataset then fine-tuning on MoCA-Mask (S+MM), for model H, as opposed to only
training on MoCA-Mask (MM), for G.

####  6.3.3 Results on MoCA

We train our model on the synthetically generated camouflage images and fine-
tune on the training set of MoCA-Mask. Tab. 5 presents our camouflage object
segmentation results on the test set of MoCA-Mask. Ours refers to our proposed
method with SINet-V2 as the appearance encoder and Ours-flow refers to the
optical flow only version of our architecture, excluding the appearance
stream. Our main model outperforms RGB and Motion-based methods on MoCA-Mask.
Fig. 6.3.1 presents qualitative results of our segmentation model. Note that
our model is robust to degraded optical flow and static animals.

###  6.4 Limitations

While our image generation method encourages background matching through the
$\mathcal{L}_{\mathcal{F}}$ loss term minimization, the sequence generating is
not guaranteed to maintain object concealment. A possible solution could be to
use the proposed $S_{\alpha}$ camouflage score to curate the generated
sequence dataset and filter out the most visible objects.

Our proposed camouflage scores use the ground truth annotation and analyse the
different regions it defines. We found that our scoring system penalises
specific cases of camouflage by occlusion, where elements from the background
are partially occluding the animal and helping improve its camouflage. For
instance, the grass in example (g) from the last row of Fig. 6 is not treated
as part of the animal and therefore not considered in our background
similarity assessment. One can argue that this is due to the ambiguity in the
provided annotations and for such cases, extra amodal annotation should be
also considered.

##  7 Conclusion

We present three score functions for computationally assessing the
effectiveness of camouflage in images and videos. By evaluating the similarity
with background and the boundary visibility, our combined perceptual score is
strongly correlated with human perceptual ranking systems on two different
datasets. We demonstrate that training a generative model with our
differentiable camouflage function improves the effectiveness of generated
camouflage examples and can be used to generate challenging synthetic
camouflage datasets to train models to break camouflage.

### Acknowledgement

We are grateful to Przemysław Skurowski for providing the human perception
study data on the CHAMELEON dataset and Andrew Owens, Rui Guo and Oscar de
Lima for providing the data for the Camouflaged cuboids dataset. We thank
Tengda Han for fruitful discussions. This research is supported by the UK
EPSRC funded CDT in Autonomous Intelligent Machines and Systems (AIMS), the
EPSRC Programme Grant VisualAI EP/T028572/1, a Schlumberger Studentship, and a
Royal Society Research Professorship. WX is supported by the National Key R&D
Program of China (No. 2022ZD0161400).

## References

  * [1] Pia Bideau and Erik Learned-Miller.  It’s moving! a probabilistic model for causal motion segmentation in moving camera videos.  In ECCV, pages 433–449, 2016.
  * [2] Pia Bideau, Rakesh R Menon, and Erik Learned-Miller.  Moa-net: self-supervised motion segmentation.  In Proc. ECCV Workshop, 2018.
  * [3] Thomas Brox and Jitendra Malik.  Object segmentation by long term analysis of point trajectories.  In European conference on computer vision, pages 282–295, 2010\.
  * [4] Bowen Cheng, Ishan Misra, Alexander G Schwing, Alexander Kirillov, and Rohit Girdhar.  Masked-attention mask transformer for universal image segmentation.  In CVPR, pages 1290–1299, 2022.
  * [5] Xuelian Cheng, Huan Xiong, Deng-Ping Fan, Yiran Zhong, Mehrtash Harandi, Tom Drummond, and Zongyuan Ge.  Implicit motion handling for video camouflaged object detection.  In CVPR, pages 13864–13873, 2022.
  * [6] Subhabrata Choudhury, Laurynas Karazija, Iro Laina, Andrea Vedaldi, and Christian Rupprecht.  Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motion.  In BMVC, 2022.
  * [7] Khoa D Doan, Saurav Manchanda, Fengjiao Wang, Sathiya Keerthi, Avradeep Bhowmik, and Chandan K Reddy.  Image generation via minimizing fréchet distance in discriminator feature space.  arXiv preprint arXiv:2003.11774, 2020.
  * [8] A. Efros and T. Leung.  Texture synthesis by non-parametric sampling.  In Proceedings of the 7th International Conference on Computer Vision, Kerkyra, Greece, pages 1039–1046, September 1999.
  * [9] Deng-Ping Fan, Ge-Peng Ji, Ming-Ming Cheng, and Ling Shao.  Concealed object detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021\.
  * [10] Maurice Fréchet.  Sur la distance de deux lois de probabilité.  Comptes Rendus Hebdomadaires des Seances de L’Academie des Sciences, pages 689–692, 1957.
  * [11] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In NeurIPS, pages 2672–2680, 2014.
  * [12] Rui Guo, Jasmine Collins, Oscar de Lima, and Andrew Owens.  Ganmouflage: 3d object nondetection with texture fields.  In arXiv preprint arXiv:2201.07202, 2022.
  * [13] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  In Advances in Neural Information Processing Systems, 2017.
  * [14] Qi Jia, Shuilian Yao, Yu Liu, Xin Fan, Risheng Liu, and Zhongxuan Luo.  Segment, magnify and reiterate: Detecting camouflaged objects the hard way.  In CVPR, pages 4713–4722, 2022.
  * [15] Nobukatsu Kajiura, Hong Liu, and Shin’ichi Satoh.  Improving camouflaged object detection with the uncertainty of pseudo-edge labels.  In ACM Multimedia Asia, pages 1–7. 2021.
  * [16] Maurice G Kendall.  A new measure of rank correlation.  Biometrika, 30(1/2):81–93, 1938.
  * [17] Margret Keuper, Bjoern Andres, and Thomas Brox.  Motion trajectory segmentation via minimum cost multicuts.  In ICCV, pages 3271–3279, 2015.
  * [18] Hala Lamdouar, Weidi Xie, and Andrew Zisserman.  Segmenting invisible moving objects.  In BMVC, 2021.
  * [19] Hala Lamdouar, Charig Yang, Weidi Xie, and Andrew Zisserman.  Betrayed by motion: Camouflaged object discovery via motion segmentation.  In ACCV, 2020.
  * [20] Trung-Nghia Le, Tam V Nguyen, Zhongliang Nie, Minh-Triet Tran, and Akihiro Sugimoto.  Anabranch network for camouflaged object segmentation.  Computer Vision and Image Understanding, pages 45–56, 2019.
  * [21] José Lezama, Karteek Alahari, Josef Sivic, and Ivan Laptev.  Track to the future: Spatio-temporal video segmentation with long-range motion cues.  In CVPR, pages 3369–3376, 2011.
  * [22] Chiuhsiang Joe Lin, Chi-Chan Chang, and Yung-Hui Lee.  Evaluating camouflage design using eye movement data.  Applied ergonomics, pages 714–723, 2014.
  * [23] Haiyang Mei, Ge-Peng Ji, Ziqi Wei, Xin Yang, Xiaopeng Wei, and Deng-Ping Fan.  Camouflaged object segmentation with distraction mining.  In CVPR, pages 8772–8781, 2021.
  * [24] Peter Ochs and Thomas Brox.  Object segmentation in video: a hierarchical variational approach for turning point trajectories into dense regions.  In ICCV, 2011.
  * [25] Peter Ochs and Thomas Brox.  Higher order motion models and spectral clustering.  In CVPR, pages 614–621, 2012.
  * [26] Peter Ochs, Jitendra Malik, and Thomas Brox.  Segmentation of moving objects by long term video analysis.  volume 36, pages 1187–1200, 2013.
  * [27] Andrew Owens, Connelly Barnes, Alex Flint, Hanumant Singh, and William Freeman.  Camouflaging an object from many viewpoints.  In CVPR, pages 2782–2789, 2014.
  * [28] Youwei Pang, Xiaoqi Zhao, Tian-Zhu Xiang, Lihe Zhang, and Huchuan Lu.  Zoom in and out: A mixed-scale triplet network for camouflaged object detection.  In CVPR, pages 2160–2170, 2022.
  * [29] Xavier Soria Poma, Edgar Riba, and Angel Sappa.  Dense extreme inception network: Towards a robust cnn model for edge detection.  In Proceedings of the IEEE Workshop on Applications of Computer Vision, pages 1923–1932, 2020.
  * [30] Tamar Rott Shaham, Tali Dekel, and Tomer Michaeli.  Singan: Learning a generative model from a single natural image.  In ICCV, pages 4570–4580, 2019.
  * [31] Josef Sivic, Frederik Schaffalitzky, and Andrew Zisserman.  Object level grouping for video shots.  In Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic. Springer-Verlag, 2004.
  * [32] Przemysław Skurowski, Hassan Abdulameer, J Błaszczyk, Tomasz Depta, Adam Kornacki, and P Kozieł.  Animal camouflage analysis: Chameleon database.  Unpublished manuscript, 2018.  URL: https://www.polsl.pl/rau6/chameleon-database-animal-camouflage-analysis/.
  * [33] Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, and Victor Lempitsky.  Resolution-robust large mask inpainting with fourier convolutions.  arXiv preprint arXiv:2109.07161, 2021.
  * [34] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In CVPR, 2015.
  * [35] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.  Rethinking the inception architecture for computer vision.  In CVPR, pages 2818–2826, 2016.
  * [36] Zachary Teed and Jia Deng.  Raft: Recurrent all-pairs field transforms for optical flow.  In ECCV, 2020.
  * [37] Jolyon Troscianko, John Skelhorn, and Martin Stevens.  Quantifying camouflage: how to predict detectability from appearance.  BMC evolutionary biology, 17(1):1–13, 2017.
  * [38] Cédric Villani.  Optimal transport: old and new, volume 338.  Springer, 2009.
  * [39] Junyu Xie, Weidi Xie, and Andrew Zisserman.  Segmenting moving objects via an object-centric layered representation.  In Advances in Neural Information Processing Systems, 2022.
  * [40] Charig Yang, Hala Lamdouar, Erika Lu, Andrew Zisserman, and Weidi Xie.  Self-supervised video object segmentation by motion grouping.  In ICCV, 2021.
  * [41] Bowen Zhang, Shuyang Gu, Bo Zhang, Jianmin Bao, Dong Chen, Fang Wen, Yong Wang, and Baining Guo.  Styleswin: Transformer-based gan for high-resolution image generation.  In CVPR, pages 11304–11314, 2022.


# Radiation from an accelerating charge in a family of Rindler frames

Luther Rinehart

###### Abstract

The generalization of the Larmor radiation formula in gravitational fields and
with accelerating observers was obtained by Hirayama and others. We verify a
special case of their result by explicit computation using a family of
displaced Rindler frames. We discuss the role of observer-dependence of energy
and simultaneity. We also include a discussion of conservation laws in
spacetimes equipped with a Killing vector and a time function.

##  1 Background

The relationship between acceleration (Larmor) radiation and the equivalence
principle of relativity is an old paradox in classical electromagnetism. The
main questions are of two types:

  1. 1.

A charged particle with nonzero proper acceleration may be interpreted by a
comoving observer as being at rest in a gravitational field. Does such an
observer observe the emission of radiation?

  2. 2.

A charged particle undergoing inertial motion may be interpreted by an
accelerating observer as being in free fall in a gravitational field. Does
such an observer observe the emission of radiation?

Both questions remain controversial, with many explanations and thought
experiments being proposed. See [1] for a review. Nevertheless, many regard
the authoritative resolution to have been given by Rohrlich [2], who answered
no to question 1 and yes to question 2. If we set aside issues of conceptual
interpretation, both questions may be framed as well-posed problems of
classical field theory, and answered by explicit calculation of the
electromagnetic field in the appropriate coordinate system. The result is the
solution given by Rohrlich.

In this classical field theory context, the resolution of the apparent paradox
is that the notion of radiation is observer-dependent. As will be reflected in
the following calculations, this observer-dependence arises from two sources.
The first is the observer’s notion of energy, which depends on their state of
motion in a manner well-understood even at the level of special relativity.
For non-inertial motions and coordinate systems, and if the energy is
distributed over a region of space, the observer’s instantaneous velocity is
no longer sufficient to define their notion of energy. In general spacetimes,
what is needed for this is a timelike Killing vector field, generating the
observer’s notion of time-translation symmetry.

The second source of observer-dependence of radiation is the observer’s notion
of simultaneity. Radiation, as a _rate_ of energy, also depends on how the
observer does their time-slicing, or what events they regard as simultaneous.
For inertial motions, the relativities of energy and simultaneity cancel each
other under Lorentz transformations, and that is why the rate of radiation is
Lorentz-invariant. But as shown by the calculations of Rohrlich, the
cancellation does not persist for more general motion. These two inputs,
stationarity and simultaneity, are both needed to define radiation. They are
given by the data of a Killing field and a time function. We see that the
concept of an observer cannot simply be quantified by the observer’s
worldline, but requires as input a frame of reference extending throughout a
spacetime region.

Kretzschmar and Fugmann [3, 4], as well as Hirayama [5, 6], have given the
generalization of the Larmor formula to the case where observer and charge
have arbitrary velocities and accelerations. The classic questions about the
equivalence principle and radiation in accelerated frames may be treated as
special cases of this. Suppose the particle has velocity $v^{\mu}$ and proper
acceleration $a^{\mu}$, and suppose the worldlines of the observer’s reference
frame have velocity $u^{\mu}$ and proper acceleration $g^{\mu}$. Let
$g=\sqrt{g_{\mu}g^{\mu}}$. Let

$h^{\mu}_{\ \nu}=\delta^{\mu}_{\ \nu}+v^{\mu}v_{\nu}$ (1)

be the projection orthogonal to $v^{\mu}$. Define the Hirayama acceleration
vector as

$\alpha^{\mu}=h^{\mu}_{\ \nu}(a^{\nu}-g^{\nu}-gu^{\nu})$ (2)

All these quantities are to be evaluated at the retarded time, that is, at the
point of emission. The general result is that the radiated power is
proportional to $\alpha^{\mu}\alpha_{\mu}$.

The purpose of this paper is to reproduce this result, via explicit
calculation, for the case where $v^{\mu}=u^{\mu}$ and $a^{\mu}$ and $g^{\mu}$
are collinear. At the time of performing this calculation, the author was
unaware of the work of Kretzschmar, Fugmann, and Hirayama.

Rindler coordinates describe how Minkowski spacetime looks to a uniformly
accelerated observer. The main technical tool of this paper is to use a family
of Rindler coordinate systems, parameterized by their $z$-displacement, and by
their proper acceleration. This way, the possibilities for displacement
between observer and charge, and for their two independent accelerations, are
at least partly parameterized.

##  2 The field and stress-energy of a point charge in arbitrary motion

This section consists of standard textbook material which can be found, for
example, in [7] chapter 5. We will use signature $-,+,+,+$, and employ dot
product notation for the four-dimensional dot product. That is, $A\cdot
B=A_{\mu}B^{\mu}$. Consider a particle of charge $q$ in Minkowski spacetime,
with four-velocity $v^{\mu}$ and proper four-acceleration $a^{\mu}$. Let
$R^{\mu}=x^{\mu}-x^{\mu}_{0}$ be the four-dimensional displacement vector,
where $x^{\mu}_{0}$ is the location of the particle. The Lienard-Wiechert
potential of the point charge in arbitrary motion is

$A_{\nu}=\frac{qv_{\nu}}{4\pi v\cdot R}$ (3)

where everything is evaluated at the retarded time, or to put it another way,
on the future lightcone, characterized by $R\cdot R=0$. The field tensor is

$F_{\mu\nu}=\frac{q}{4\pi(v\cdot
R)^{2}}\left[R_{\mu}(a_{\nu}-fv_{\nu})-R_{\nu}(a_{\mu}-fv_{\mu})\right]$ (4)

where

$f=\frac{1+a\cdot R}{v\cdot R}$ (5)

The stress-energy tensor is

$T_{\mu\nu}=\frac{q^{2}}{(4\pi)^{2}(v\cdot
R)^{4}}\Big{[}R_{\mu}R_{\nu}(a^{2}-f^{2})+R_{\mu}(a_{\nu}-fv_{\nu})+R_{\nu}(a_{\mu}-fv_{\mu})+\frac{1}{2}g_{\mu\nu}\Big{]}$
(6)

The acceleration of the charge need not be uniform. The field on the future
lightcone of an instant on the particle’s worldline depends only on $v^{\mu}$
and $a^{\mu}$ at that instant. We may choose coordinates for the inertial
frame so that the charge passes through the origin and is instantaneously at
rest, with its acceleration in the $z$-direction. Also in the inertial frame,
let

$x_{\bot}=\sqrt{x^{2}+y^{2}}$ (7) $r=\sqrt{x_{\bot}^{2}+z^{2}}$ (8)

With the inertial frame coordinates so chosen, we have

$\displaystyle R^{\mu}=r\hat{t}+z\hat{z}+x_{\bot}\hat{x}_{\bot}$ (9)
$\displaystyle a^{\mu}=a\hat{z}$ (10) $\displaystyle v^{\mu}=\hat{t}$ (11)
$\displaystyle v\cdot R=-r$ (12) $\displaystyle a\cdot R=az$ (13)

The parameter $a$ may be positive or negative. The idea is to fix the origin
as the point of emission we are interested in. We will look at the field on
the future light cone of the origin. The behavior of the charge at other
points of its trajectory does not matter in this context, because we are using
a massless field. The emission point is fixed at the origin, and the
observer’s coordinate system will be moved around.

##  3 A family of displaced Rindler frames

Introduce a family of Rindler frames with coordinates $\chi$ and $\tau$, and
parameters $\chi_{0}$ and $g$, taken to be positive. The coordinates are
related to the inertial coordinates by

$t=\frac{\chi}{g}\sinh(g\tau)\qquad
z+\frac{\chi_{0}}{g}=\frac{\chi}{g}\cosh(g\tau)$ (14)

and cover the wedge of spacetime where $z+\frac{\chi_{0}}{g}>|t|$. This setup
is depicted in figure 1. In these coordinates, the metric is

$g_{\mu\nu}=-\chi^{2}d\tau^{2}+\frac{d\chi^{2}}{g^{2}}+dx^{2}+dy^{2}$ (15)

representing a uniform gravitational field. Rindler time $\tau$ is a time
coordinate for which the metric is invariant. Among $\tau$-stationary
observers, those at $\chi=1$ are privileged in that their proper time
coincides with the Rindler time $\tau$. The worldlines of $\tau$-stationary
observers are in uniform acceleration, although with differing values of
proper acceleration. Privileged observers at $\chi=1$ have proper acceleration
equal to $g$. The origin of the inertial coordinate system, where the charge
is located, in the Rindler coordinates is at $\chi=\chi_{0}$. Thus $\chi_{0}$
represents the vertical displacement of the charge in the Rindler spacetime,
and also represents a redshift factor between the observer and the charge.

Figure 1: The shifted Rindler coordinate system, with an example source
trajectory (red), and the future lightcone of the origin (blue)

The $\tau$ symmetry of Rindler spacetime is associated with a Killing field

$\xi^{\mu}=g\left(z+\frac{\chi_{0}}{g}\right)\hat{t}+gt\hat{z}$ (16)

which gives us the Rindler observers’ notion of conserved energy.

The 3-dimensional geometry of a constant $\tau$ hypersurface is Euclidean. The
future lightcone of the origin intersected with such a surface turns out to be
a sphere centered at

$\chi=\chi_{0}\cosh(g\tau)$ (17)

with radius

$\rho=\frac{\chi_{0}}{g}\sinh{g\tau}$ (18)

On this sphere,

$t=r=\frac{\chi\rho}{\chi_{0}}$ (19)

The geometry of the lightcone sphere as seen in Rindler space is shown in
figure 2. We may introduce a polar angle $\theta$ on the lightcone sphere, and
express the other coordinates in terms of $\rho$ and $\theta$:

$\displaystyle x_{\bot}=\rho\sin\theta$ (20) $\displaystyle
z=\frac{g\rho}{\chi_{0}}\left(\rho+\sqrt{\rho^{2}+\frac{\chi_{0}^{2}}{g^{2}}}\cos\theta\right)$
(21) $\displaystyle
r=\frac{g\rho}{\chi_{0}}\left(\sqrt{\rho^{2}+\frac{\chi_{0}^{2}}{g^{2}}}+\rho\cos\theta\right)$
(22)

The unit normal vector to the lightcone sphere is

$\begin{split}\hat{n}&=\cos\theta\,\hat{\chi}+\sin\theta\,\hat{x}_{\bot}\\\
&=\frac{g\rho\cos\theta}{\chi_{0}}\,\hat{t}+\frac{g\cos\theta}{\chi_{0}}\sqrt{\rho^{2}+\frac{\chi_{0}^{2}}{g^{2}}}\,\hat{z}+\sin\theta\,\hat{x}_{\bot}\end{split}$
(23)

Figure 2: The future lightcone of the origin at successive times $\tau$ as
seen in Rindler space, shown in the $x$-$\chi$ plane

##  4 Evaluating the Rindler energy flux on the light-sphere

We may now calculate the flux of Rindler energy through the lightcone sphere
as

$S=-T_{\mu\nu}\xi^{\mu}\hat{n}^{\nu}$ (24)

The needed ingredients are the following dot products:

$\displaystyle R\cdot\xi=-r\chi_{0}$ (25) $\displaystyle R\cdot\hat{n}=\rho$
(26) $\displaystyle\hat{n}\cdot\xi=0$ (27)
$\displaystyle(a-fv)\cdot\hat{n}=\frac{\rho\cos\theta}{r\chi_{0}}(a\chi_{0}-g)$
(28)
$\displaystyle(a-fv)\cdot\xi=-\frac{1}{r}(gz+\chi_{0}+az\chi_{0}-agx^{2}_{\bot})$
(29)

Putting these together in the stress tensor gives

$\begin{split}S=-\frac{q^{2}}{(4\pi)^{2}r^{4}}\Big{[}&-\frac{\rho\chi_{0}}{r}(a^{2}x_{\bot}^{2}-2az-1)-\rho\cos\theta(a\chi_{0}-g)\\\
&-\frac{\rho}{r}(gz+\chi_{0}+az\chi_{0}-agx_{\bot}^{2})\Big{]}\\\
=-\frac{q^{2}\rho}{(4\pi)^{2}r^{5}}&(a\chi_{0}-g)(-ax_{\bot}^{2}+z-r\cos\theta)\\\
=\frac{q^{2}\rho}{(4\pi)^{2}r^{5}}&(a\chi_{0}-g)\left(ax_{\bot}^{2}-\frac{g\rho^{2}\sin^{2}\theta}{\chi_{0}}\right)\\\
=&\frac{q^{2}\rho^{3}\sin^{2}\theta}{(4\pi)^{2}\chi_{0}r^{5}}(a\chi_{0}-g)^{2}\end{split}$
(30)

At this point several qualitative features are apparent.

  * •

In the $g\rightarrow 0$ limit, the radiation has the same features as Larmor
radiation.

  * •

An inertial charge, with $a=0$, does radiate for accelerated observers.

  * •

A Rindler-stationary charge is one with $a\chi_{0}=g$, and does not radiate.

Recall that $a$ may be positive or negative, accounting for whether the
observer and charge accelerate in the same or opposite direction. However
$\chi_{0}$ must be positive, since otherwise the future lightcone of the
origin does not intersect the observer’s Rindler wedge.

##  5 Digression on the interpretation of the conservation equation

In arbitrary spacetimes, a conservation law is expressed locally by the
equation

$\nabla_{\mu}J^{\mu}=0$ (31)

Suppose we have a static metric of the form

$g_{\mu\nu}=-u(x)dt^{2}+g^{(3)}_{\mu\nu}(x)$ (32)

where $g^{(3)}$ is a metric on 3-dimensional space independent of $t$. The
Rindler spacetime is of this form, but also others like Schwarzschild. The
vector field $J^{\mu}$ decomposes as $(J^{t},\overrightarrow{J})$. Then the
conservation equation takes the form

$\frac{\partial}{\partial
t}J^{t}+\frac{1}{\sqrt{u}}\nabla\cdot(\sqrt{u}\overrightarrow{J})=0$ (33)

where the second term involves the 3-dimensional divergence with respect to
$g^{(3)}$. This can be rewritten

$\frac{\partial}{\partial
t}(\sqrt{u}J^{t})+\nabla\cdot(\sqrt{u}\overrightarrow{J})=0$ (34)

The interpretation is that the conserved quantity has density $\sqrt{u}J^{t}$
and flux $\sqrt{u}\overrightarrow{J}$. The volume factor $\sqrt{u}$ must be
included, even though it does not appear in $g^{(3)}$. This result is
sufficient for the purpose of the present Rindler spacetime calculation. It is
generalized in section 7 to the case where the Killing vector need not be
hypersurface-orthogonal.

##  6 Results

In Rindler spacetime, the volume factor is $\chi$, making the flux

$\chi S=\frac{q^{2}\rho^{2}\sin^{2}\theta}{(4\pi)^{2}r^{4}}(a\chi_{0}-g)^{2}$
(35)

This may now be integrated over the light-sphere to give the power

$\begin{split}P&=\int\chi S\ \rho^{2}d\Omega\\\
&=\frac{q^{2}(a\chi_{0}-g)^{2}}{(4\pi)^{2}}\int_{0}^{\pi}\frac{\rho^{4}}{r^{4}}\sin^{2}\theta\,2\pi\sin\theta\
d\theta\\\
&=\frac{q^{2}(a\chi_{0}-g)^{2}}{8\pi}\int_{0}^{\pi}\left(\sqrt{1+\left(\frac{g\rho}{\chi_{0}}\right)^{2}}+\frac{g\rho}{\chi_{0}}\cos\theta\right)^{\\!-4}\sin^{3}\theta\,d\theta\end{split}$
(36)

The $\theta$ integral is equal to $4/3$ for all values of $\rho$, so the final
result is independent of $\rho$:

$\begin{split}P&=\frac{2}{3}\ \frac{q^{2}(a\chi_{0}-g)^{2}}{4\pi}\\\
&=\frac{2}{3}\
\frac{q^{2}\chi_{0}^{2}}{4\pi}\left(a-\frac{g}{\chi_{0}}\right)^{2}\end{split}$
(37)

Note that, when integrated over the sphere, the radiation rate is independent
of the distance from the charge. This is in agreement with the results of
Rohrlich [8]. This is true even retaining all the terms in the energy flux,
and not employing any large-distance, radiation limit.

We may interpret this result in the language of Hirayama as in (2), and verify
that it is in agreement with the general formula found in [5, 6]. $a^{\mu}$
and $v^{\mu}$ are as given in equations (10) and (11). In the Rindler frame,
the $\tau$-stationary worldlines have velocity
$u^{\mu}=\frac{1}{\chi}\xi^{\mu}$, and proper acceleration

$g^{\mu}=u^{\nu}\nabla_{\nu}u^{\mu}=\frac{g^{2}}{\chi^{2}}\left(t\hat{t}+(z+\chi_{0}/g)\hat{z}\right)$
(38)

These must be evaluated at the origin, giving $u^{\mu}=\hat{t}$ and
$g^{\mu}=\frac{g}{\chi_{0}}\hat{z}$. Since $u^{\mu}=v^{\mu}$, the Hirayama
acceleration vector simplifies to

$\alpha^{\mu}=a^{\mu}-g^{\mu}=\left(a-\frac{g}{\chi_{0}}\right)\hat{z}$ (39)

and the radiation is indeed proportional to $\alpha^{\mu}\alpha_{\mu}$. The
factor of $\chi_{0}^{2}$ may be interpreted as a redshift factor between the
source, located at $\chi=\chi_{0}$, and the observer, located at $\chi=1$. The
power is doubly redshifted to account for both the redshift of the energy, and
also the time dilation.

##  7 Conservation laws in a spacetime with a Killing vector and a time
function

This section generalizes the result of section 5 in a coordinate-independent
manner, dropping the assumption that the Killing vector be hypersurface-
orthogonal. If a spacetime is equipped with a stationary frame of reference in
the form of a Killing vector and a time function, this structure allows the
equation $\nabla_{\mu}J^{\mu}=0$ to be interpreted by stationary observers as
a local conservation law for some conserved quantity, in a manner compatible
with the observers’ notions of time-translation and simultaneity. Consider a
spacetime equipped with a timelike Killing vector field $\xi^{\mu}$ and a time
function $t$ that is compatible in the sense that its Lie derivative is equal
to 1:

$\mathsterling_{\xi}(t)=1$ (40)

In particular, this means

$\mathsterling_{\xi}(\nabla_{\mu}t)=0$ (41)

so the foliation of constant-$t$ hypersurfaces is preserved under the
transformation generated by $\xi^{\mu}$. This data represents a notion of
time-translation and simultaneity. It allows us to interpret spacetime as
spatial manifold $\Sigma$ with a time evolution. Let $h_{\mu\nu}$ be the
restriction of the metric to the constant-$t$ surface $\Sigma$. $h_{\mu\nu}$
is preserved under time evolution, and as a Riemannian metric, endows $\Sigma$
with its own geometry and covariant derivatives. We seek to interpret the
conservation equation in a manner compatible with this picture.

An arbitrary vector field $J^{\mu}$ uniquely decomposes as

$J^{\mu}=J^{0}\xi^{\mu}+j^{\mu}$ (42)

where

$J^{0}=J^{\mu}\nabla_{\mu}t$ (43) $j^{\mu}\nabla_{\mu}t=0$ (44)

meaning that $j^{\mu}$ is tangent to the surfaces of constant $t$. We may then
calculate

$\nabla_{\mu}J^{\mu}=\mathsterling_{\xi}J^{0}+\nabla_{\mu}j^{\mu}$ (45)

However, the second term is not the same as the divergence of $j^{\mu}$ with
respect to the metric $h_{\mu\nu}$ on $\Sigma$. The derivative operator $D$ on
$\Sigma$ associated with $h_{\mu\nu}$ is the orthogonal projection onto
$\Sigma$ of the operator $\nabla$. Hence the divergence we are interested in
is

$D_{\mu}j^{\mu}=\nabla_{\mu}j^{\mu}+n_{\mu}n^{\nu}\nabla_{\nu}j^{\mu}$ (46)

where $n^{\mu}$ is the unit normal vector to $\Sigma$. Using the fact that
$j^{\mu}n_{\mu}=0$, this equals

$=\nabla_{\mu}j^{\mu}-j^{\mu}n^{\nu}\nabla_{\nu}n_{\mu}$ (47)

Now introduce a function $f$ defined by

$n_{\mu}=f\nabla_{\mu}t$ (48)

Now we have

$\begin{split}D_{\mu}j^{\mu}&=\nabla_{\mu}j^{\mu}-j^{\mu}n^{\nu}\nabla_{\nu}(f\nabla_{\mu}t)\\\
&=\nabla_{\mu}j^{\mu}-j^{\mu}n^{\nu}f\nabla_{\nu}\nabla_{\mu}t\\\ \end{split}$
(49)

Since $\nabla$ is torsion-free, this equals

$\begin{split}&=\nabla_{\mu}j^{\mu}-j^{\mu}n^{\nu}f\nabla_{\mu}\nabla_{\nu}t\\\
&=\nabla_{\mu}j^{\mu}-j^{\mu}n^{\nu}f\nabla_{\mu}\left(\frac{n_{\nu}}{f}\right)\end{split}$
(50)

Finally, since $n^{\nu}n_{\nu}=-1$,

$\begin{split}&=\nabla_{\mu}j^{\mu}+j^{\mu}f\nabla_{\nu}\left(1/f\right)\\\
&=f\nabla_{\mu}\left(\frac{j^{\mu}}{f}\right)\end{split}$ (51)

This result holds for arbitrary vector fields tangent to $\Sigma$. Hence all
together,

$\nabla_{\mu}J^{\mu}=\mathsterling_{\xi}J^{0}+\frac{1}{f}D_{\mu}(fj^{\mu})$
(52)

Note that

$\frac{1}{f}=n^{\mu}\nabla_{\mu}t=\sqrt{\nabla_{\mu}t\nabla^{\mu}t}$ (53)

and consequently, $\mathsterling_{\xi}(f)=0$. The conservation equation can be
rewritten

$0=\mathsterling_{\xi}(fJ^{0})+D_{\mu}(fj^{\mu})$ (54)

By using Gauss’s law in $\Sigma$, we have the interpretation that there is a
conserved quantity of density $fJ^{0}$ and flux $fj^{\mu}$.

##  8 Summary and conclusions

Applying the equivalence principle to the Larmor radiation of accelerating
charged particles leads naturally to questions of how classical acceleration
radiation looks to accelerated observers, and in gravitational fields. The
work of Rohrlich [2], in answering such questions, revealed that the concept
of radiation is observer-dependent, and opened the way for a generalization of
the Larmor formula to account for the view of non-inertial observers. Such
generalization was given by Kretzschmar and Fugmann [3, 4], and Hirayama [5,
6]. To define a total rate of radiation, an observer must be equipped with a
reference frame extending throughout a region of spacetime, in the form of a
timelike Killing vector field, and a time-slicing. The result is that the
observed radiated power is proportional to the square of the _difference_
between the proper acceleration of the source, and the proper acceleration of
the frame. In particular, an inertial charge is seen to radiate by an
accelerated observer, and an accelerating charge is not seen to radiate by a
co-accelerated observer. In this paper, we have attempted to show, by explicit
calculation, how this result comes about, and how it depends on both the
observer’s notion of energy, given by the Killing vector, and the observer’s
notion of simultaneity, given by the time-slicing.

The general formula given by Hirayama was derived in the case where the
observer accelerates uniformly in a flat background. However, its terms could
easily be interpreted in any spacetime equipped with a Killing vector and a
time function, such as the Schwarzschild spacetime. For example, a charged
particle freely orbiting a black hole would presumably radiate to stationary
observers, despite having vanishing proper acceleration. The author is not
aware of any work to rigorously justify a generalization of the Hirayama
formula to other spacetimes. Therefore such generalization, if possible, is an
opportunity for future work.

##  9 Acknowledgments

Collaborative discussion was provided by Stephen Fulling of Texas A&M
University, Gerard Kennedy of the University of Southampton, and Timothy
Bates, student at Texas A&M.

## References

  * [1] Ø. Grøn, “Electrodynamics of radiating charges,” Adv. Math. Phys. 2012, Article ID 528631 (2012).
  * [2] F. Rohrlich, “The principle of equivalence,” Ann. Phys. 22, 169 (1963).
  * [3] M. Kretzschmar and W. Fugmann, “The electromagnetic field of an accelerated charge in the proper reference frame of a noninertial observer,” Il Nuovo Cimento 103, 389 (1989).
  * [4] W. Fugmann and M. Kretzschmar, “Classical elecromagnetic radiation in noninertial reference frames,” Il Nuovo Cimento 106, 351 (1991).
  * [5] T. Hirayama, “Bound and radiation fields in the Rindler frame,” Progress of Theoretical Physics 106, 71 (2001).
  * [6] T. Hirayama, “Classical radiation formula in the Rindler frame,” Progress of Theoretical Physics 108, 679 (2002).
  * [7] F. Rohrlich, Classical Charged Particles, Addison-Wesley, Reading, MA, 1964.
  * [8] F. Rohrlich, “The definition of electromagnetic radiation,” Nuovo Cimento 21, 811 (1961).