# OHPL: One-shot Hand-eye Policy Learner

Changjae Oh, Yik Lung Pang, and Andrea Cavallaro Changjae Oh and Yik Lung Pang
equally contributed. The authors are with the Centre for Intelligent Sensing,
Queen Mary University of London, E1 4NS, London, United Kingdom. {c.oh,
y.l.pang<EMAIL_ADDRESS>
###### Abstract

The control of a robot for manipulation tasks generally relies on object
detection and pose estimation. An attractive alternative is to learn control
policies directly from raw input data. However, this approach is time-
consuming and expensive since learning the policy requires many trials with
robot actions in the physical environment. To reduce the training cost, the
policy can be learned in simulation with a large set of synthetic images. The
limit of this approach is the domain gap between the simulation and the robot
workspace. In this paper, we propose to learn a policy for robot reaching
movements from a single image captured directly in the robot workspace from a
camera placed on the end-effector (a hand-eye camera). The idea behind the
proposed policy learner is that view changes seen from the hand-eye camera
produced by actions in the robot workspace are analogous to locating a region-
of-interest in a single image by performing sequential object localisation.
This similar view change enables training of object reaching policies using
reinforcement-learning-based sequential object localisation. To facilitate the
adaptation of the policy to view changes in the robot workspace, we further
present a dynamic filter that learns to bias an input state to remove
irrelevant information for an action decision. The proposed policy learner can
be used as a powerful representation for robotic tasks, and we validate it on
static and moving object reaching tasks.

##  I Introduction

Data-driven robot learning trains an agent to acquire skills for tasks, such
as locomotion [1], grasping [2], and manipulation [1, 3]. However, learning
the control policies requires a large amount of training data and expert
knowledge. Reinforcement Learning (RL) aims at learning policies by
interacting with the environment [2, 4, 5]. The agent acquires generalised
policies while exploring the environment and maximising a reward signal. RL
with real-world training requires expensive and potentially unsafe active data
collection by robots to gather a large number of experiences from scratch.

To address the limitation of active data acquisition in traditional learning-
based approaches, simulation may be employed to learn control policies that
are then transferred to the real world. The domain gap between the simulation
and real world is addressed by domain adaptation [6, 7], domain randomisation
[8], meta-learning [9, 10] or learning from depth images [11, 12, 13]. When
the policy trained in simulation is adapted to the real-world robot workspace
with sim-to-real techniques, precise camera calibration is needed to ensure
the coherency between the geometric information from the simulation and real-
world [13, 14].

Figure 1: Sequential input states and corresponding generated actions from
(top) hand-eye robot manipulation and (bottom) RL-based object localisation.
In (top-left) the robot workspace with hand-eye UR5 robot, our method takes
(bottom-left) a single image in first-person view from the camera (green cube)
on the gripper as an RL environment for sequential object localisation. In the
RL environment, the input state for the RL agent is produced by cropping and
resizing the RoI based on the bounding box. That is, the RL agent sequentially
localises the object by arranging the position (shift) and size (zoom) of the
RoI with discrete actions. The input states for the sequential localisation
task show similar view changes to the hand-eye robot reaching task in the
robot workspace. The agent thus can be directly deployed to the robot
workspace since the proxy task is performed in the nearly same domain.

In this paper, we propose a one-shot hand-eye robot policy learner (OHPL) for
the object reaching task. OHPL does not require calibration information nor
sim-to-real adaptation. Given the first-person view provided by a camera
placed on the end-effector, we observe that the hand-eye robot view changes
for the reaching task are similar to locating the object into a region-of-
interest (RoI) from the first-person-view image. In OHPL, we first capture a
single image from the robot workspace and then use RL-based sequential object
localisation as a proxy task to learn the control policy for the object
reaching task. OHPL learns the policy to generate sequential actions to change
the position and size of the RoI (from the image) until it localises the
object. To cope with the view changes in the robot workspace, OHPL includes a
dynamic filter that generates image-conditioned parameters naturally learned
to bias an input state to remove irrelevant information for an action
decision. The learned policy can then be deployed for both the static and
moving object reaching task in the robot workspace111Code and video results
are available at: http://corsmal.eecs.qmul.ac.uk/OHPL.html.

##  II Related Work

In this section, we review learning-based robot control, the domain gap
problem, and proxy tasks to learn representations without explicit
supervision.

Control policies can be learned with RL or in a supervised fashion (e.g.,
imitation learning, IL). With IL, agents acquire skills from demonstrations,
which include kinesthetic demonstrations [15], teleoperation [16], and
demonstrations from a human [17, 18, 19]. Expert demonstrations can be used
directly to train the agent behaviour [10, 20] or be used to estimate a reward
function for RL [21]. Although IL can achieve good performance using expert
demonstrations to guide policy learning [22], it requires a large amount of
training data and policies can be biased to the specific demonstration.

RL supports learning skill policies through trial and error, interactively
within an environment by maximising cumulative reward received by the agent.
RL can be directly applied in the real world to learn end-to-end policies for
motor control [2, 4, 5], but a large number of trials are needed to generalise
a policy and manual resetting may be needed between episodes, increasing the
cost for data collection [4, 5, 23]. Policies are thus often trained first in
simulated environments, then transfer to a real-world task [6, 7, 8]. If there
is no significant domain shift between the simulation and test environments,
the learned policy can be directly used in the test environment [24]. However,
transferring skills is challenging due to both visual and dynamics differences
between the two domains [6, 7, 8].

To reduce the domain gap, a manipulation model can be learned with synthetic
objects rendered with real object textures [25]. Another approach is to learn
a policy from a modality with a relatively small domain gap, e.g. depth images
[11, 12, 13]. Visual domain adaptation has also been used for robot
manipulation, transferring knowledge from simulation to the real world. The
methods include transfer learning [6] and learning policy with generated real-
like images [7] or simplified images using generative adversarial networks
[26].

Domain randomisation varies the appearance (e.g. textures, lighting and camera
position) of sensory inputs during training, thus helping the agent learn
relevant features that are invariant across the domain gap [8]. This technique
can be applied to various neural network-based robot manipulation approaches
with learning feed-forward networks [8, 27], imitation learning [28, 29], and
reinforcement learning [26, 30, 31]. While domain randomisation provides good
performance in addressing the domain gap, the trained model still has to
address an unseen environment during testing. Domain randomisation in
reinforcement learning setting may also lead to stability issues [30]. Since
these real-to-simulation approaches do not learn the policy directly from the
robot workspace, precise calibration is also a core issue to address robot
manipulation [14, 13].

When expert supervision is unavailable or insufficient, a proxy task is an
alternative to pre-train representations to be used, after finetuning, for
specific tasks. For instance, visual representations can be learned from
images, without annotations, by solving a proxy task, such as reconstructing
input under corruptions such as noise, holes, colour, by performing denoising
[32], inpainting [33], and colourisation [34], respectively. The
representations can also be learned by classifying input with pseudo-label
such as patch orders [35], segmenting object motions [36], and clustering
images [37]. An example proxy task for the RL agent is predicting a future
state from the current state and action, which helps the agent to encode the
representations that are effective for achieving the goal [38, 39].

##  III Learning Object Localisation as a Proxy Task

(a)  Figure 2: Overview of the one-shot hand-eye policy Learner (OHPL). Given
a single image as the RL environment, $\mathcal{I}$, and the bounding box
(Bbox) at time step $t$, $b_{t}$, the agent takes an extracted and resized RoI
as an input state, $s_{t}$, passes it through the dynamic filter and then
performs element-wise multiplication ($\otimes$) with $s_{t}$. The combined
input is fed into the Q-Network to determine the action, $a_{t}$. Based on
$a_{t}$, the size or position of the next RoI, $b_{t+1}$, is changed with the
state transition, $f\left(b_{t},a_{t}\right)$ to locate the target object. For
instance, the agent generates $a_{t}$ as shift left to move $b_{t}$ (outlined
orange) towards the groundtruth bounding box (outlined yellow), $b$, in the
next step. The process is repeated until the Jaccard index between $b$ and the
current bounding box becomes higher than 0.8 or the agent moves the maximum
actions in an episode. The dotted line denotes the process for $t+1$ step. The
process in the grey box can be directly deployed to the robot workspace where
the hand-eye robot performs sequential discrete actions.

###  III-A Problem definition

We cast the reaching task in the robot workspace as a sequential object
localisation task in a first-person-view image [40, 41]. The view is changed
by a set of discrete robot actions (Fig. 1, top). These view changes are
similar to sequential object localisation with discrete image-processing
actions (Fig. 1, bottom). We hypothesise that the model trained with such a
task can, if appropriately formulated, naturally acquire skills for the agent
to perform the reaching task in the robot workspace.

We formulate the sequential object localisation task by locating the RoI as a
Markov Decision Process (MDP). We take an initial observation from the robot,
a single image $\mathcal{I}$, to form an environment for RL. In the MDP, the
RL agent learns to change the position and size of the bounding box over some
discrete time steps until the RoI image captures the object in $\mathcal{I}$.
At each time step $t$, the agent receives a state, $s_{t}$, the RoI image
resized to the network input size, and selects an action, $a_{t}$, among $N$
possible actions, according to the policy that maps $s_{t}$ to $a_{t}$. After
executing the action, a state transition function generates the next state,
$s_{t+1}$ and the agent receives a reward $r_{t}$.

###  III-B State and action

The overview of our approach is shown in Fig. 2. At time step $t$, a square
bounding box is represented by a three-dimensional vector
$b_{t}=\left[x_{t},y_{t},w_{t}\right]$ where $\left[x_{t},y_{t}\right]$ and
$w_{t}$ denote the top-left pixel and the window size of the bounding box,
respectively. The RoI image, based on the bounding box, is extracted from the
RL environment, $\mathcal{I}\in{\mathbb{R}^{360\times 360\times 3}}$, and then
resized to the network input size, which becomes the input state
$s_{t}\in{\mathbb{R}^{84\times 84\times 3}}$ to the agent. The agent consists
of convolutional neural networks that reduce dimension of the high-dimensional
input state [38, 42]. We set the initial state, $s_{1}$, to an RoI image that
includes the object but with random position and size of the bounding box.
This random spawning strategy enables the agent to learn the generalised
localisation policy in the single image environment $\mathcal{I}$.

We consider seven discrete robot actions ($N=7$): move left, move right, move
up, move down, move forward, move backward, and stop. These actions and
corresponding hand-eye view changes can be imitated in the proxy task that
changes the position and size of the RoI in $\mathcal{I}$ with the following
discrete actions: shift left, shift right, shift up, shift down, zoom in, zoom
out, and no-operation (no-op). For instance, moving forward and left of the
robot corresponds to zoom in and shift left actions by changing the window
size, $w_{t}$, of the RoI and its position.

After the action $a_{t}$ is decided from $s_{t}$, the next state $s_{t+1}$ is
obtained by the state transition function $f\left(b_{t},a_{t}\right)$ that
arranges the position and size of the current bounding box, $b_{t}$. By
arranging the position $\left(x_{t},y_{t}\right)$ with the displacement
$\left(\Delta{x_{t}},\Delta{y_{t}}\right)$, and $w_{t}$ with the RoI size
change $\Delta{w_{t}}$, we obtain the next bounding box, $b_{t+1}$ as follows:

$b_{t+1}=\left[x_{t}+\Delta{x_{t}},y_{t}+\Delta{y_{t}},w_{t}+\Delta{w_{t}}\right].$
(1)

The amount of displacement depends on the size of the RoI, which corresponds
to the view changes in the robot workspace. For example, the displacement
between the current and next state is small when the object is close, while
the displacement is large when the object is far. Therefore, we adapt the
amount of displacement based on the RoI size as follows:

$\Delta{x_{t}}={\sigma}_{t}{w_{t}},\Delta{y_{t}}={\sigma}_{t}{w_{t}},$ (2)

where ${\sigma}_{t}$ controls the ratio between the amount of displacement and
the bounding box size. Considering the PID control error that is likely to
occur in real robot control we randomly generate ${\sigma}_{t}\in[0.05,0.15]$
in every step. The agent thus can learn policies with more generalised cases
to reach the goal position. The next state $s_{t+1}$ can be finally obtained
by extracting the region based on the computed $b_{t+1}$ and resizing. For
exception handling, we set $s_{t+1}=s_{t}$ when the location of $b_{t+1}$ is
outside of $\mathcal{I}$ or $w_{t}$ is smaller or larger than 20 and 360,
respectively.

(a)

(b)

Figure 3: Sequential object localisation examples in (a) the RL environment
$\mathcal{I}$, where the bounding box is colour-coded from dark blue to yellow
to indicate the temporal evolution of the actions, and (b) corresponding
sequence of input states for the OHPL agent.

(a) Gaussian blur

(b) Gaussian noise

(c) Illumination

Figure 4: Sequential object localisation results when varying the overlap
thresholds on the test data. To evaluate the model robustness, we tested the
sequential object localisation by corrupting the original image by applying
(a) Gaussian blur, (b) Gaussian noise (c) and different location of the light
source.

###  III-C Reward function

We devise a reward function that encourages an RL agent to localise the object
by producing dense rewards. The reward function integrates intrinsic
motivations to encourage the agent to naturally move towards the goal to
achieve higher Jaccard index, alleviating the problem of sparse extrinsic
rewards. For each time step $t$, the reward, $r_{t}$, is defined as follows:

$r_{t}=\begin{cases}1&J\left(b,b_{t}\right)>0.8\\\
\alpha\left(J\left(b,b_{t}\right)-1\right)&{0.8\geq
J\left(b,b_{t}\right)>0}\\\ -$1$&J\left(b,b_{t}\right)=0,\end{cases}$ (3)

where $J\left(b,b_{t}\right)$ measures the overlap ratio between the
groundtruth bounding box, $b$, and the bounding box at $t$, $b_{t}$. We
consider that the agent has reached the goal when the overlap ratio between
$b$ and $b_{t}$ is higher than 0.8, where the agent receives a positive reward
of $+1$. Instead of giving a reward of $-1$ to all the states which fail to
achieve the goal, we introduce a small negative reward that encourages the
agent to move in order to localise the object with higher Jaccard index score.
When $b_{t}$ partially overlaps with $b$, the small negative reward can be
obtained by $\alpha\left(J\left(b,b_{t}\right)-1\right)$, with $\alpha=0.5$.
The higher the Jaccard index, the smaller the negative reward. Lastly, a
negative reward of $-1$ is given when $b_{t}$ does not overlap with $b$. This
reward setting encourages the RoI to not move away from the object during
training. The episode is terminated when the agent reaches the goal or has
reached the maximum number of steps $t=T_{max}$.

###  III-D Learning policies with Deep Q-Network

We address the learning of the proxy task using the DQN framework [42]. We
employ a state encoding network to approximate the underlying action-value
function:

$Q(s_{t},a_{t})=r_{t}+\gamma\max_{a_{t+1}}Q\left(s_{t+1},a_{t+1}\right),$ (4)

where $\gamma$ is a discount factor that indicates the value of future rewards
in the current state. The Q-value $Q\left(s_{t+1},a_{t+1}\right)$ presents how
good it is to select $a_{t+1}$ at $s_{t+1}$ to receive a high reward.
$Q\left(s_{t+1},a_{t+1}\right)=0$ when $s_{t+1}$ is the terminal state. Deep
reinforcement learning methods commonly require a large discount factor as the
agent may explore thousands of steps before reaching the terminal target. The
goal in our environment is around 7-15 steps far away, and we set a low
discount factor, $\gamma=0.85$ to ensure that the agent follows the intrinsic
motivations while optimising for long term success.

Note that discrepancies exist between inputs generated from the 2D single
image, $\mathcal{I}$, and the 3D robot workspace, due to sensor noise, blur,
illumination, and object shape changes. We propose to use a dynamic filter
[43] as soft-attention to provide effective information as an input for
localisation. The network takes $s_{t}$ as an input and generates the feature
map which has the same size as $s_{t}$. We then fuse the output feature map
and $s_{t}$ by multiplying them in an element-wise manner, where the output
becomes an input for the Q-Network. Noting that the feature map generated from
the dynamic filter is conditioned on the input, the dynamic filter naturally
learns to bias the input during the training without any loss function. This
process can be considered as estimating the soft-attention based on the input
image.

Fig. 3 shows an example of sequential object localisation. Initially, a
bounding box is set with a large RoI, and then the agent arranges the RoI
position and size to locate the object.

Figure 5: (Top) Test objects from YCB Benchmark [44] and (bottom) their first-
person view as an RL environment, $\mathcal{I}$, for training the OHPL agent.

##  IV Validation

###  IV-A Models under analysis

We evaluate the OHPL agent on the reaching task when deployed in the robot
simulation workspace, PyBullet [45], but first we analyse the proposed proxy
task, RL-based sequential object localisation in the single image environment
using two models, namely OHPL-base (vanilla DQN model [42]), and OHPL-full
(DQN model with dynamic filter). A single image for each object illuminated
from the left is used as the RL environment (see Fig.5). We use 8 synthetic
objects, i.e. baseball, cup-H, cup-J, rubiks cube, apple, mug, can, cracker
box, that are adopted from the Yale-Carnegie Mellon University-Berkeley (YCB)
benchmark [44], including objects with different shapes, textures, and
colours. As each image is employed as a separate RL environment, we generate 8
models each on OHPL-base and OHPL-full, respectively. In each episode, we
randomly set a position and size for the initial RoI, while including the
object in the RoI. We set $T_{max}=50$ for a single episode, where the
localisation can be achieved in 7-15 steps. We set the position of the ground-
truth bounding box as shown in the rightmost images in Fig. 3. We compare
OHPL-base and OHPL-full on the test environment with nine object locations,
middle (M), mid-left (ML), mid-right (MR), top (T), top-left (TL), top-right
(TR), bottom (B), bottom-left (BL), and bottom-right (BR), and three different
illuminations by setting the light source location to left, right, and above.
We perform 10 trials for each object location and illumination.

We further test the models by corrupting the test images with Gaussian noise,
Gaussian blur, and varying illumination condition, which emulates camera
sensor noise, blur, illumination changes in the robot workspace. The Gaussian
blur is applied with two kernel sizes, 7 and 15, and the Gaussian noise with
two variances, 10 and 20. Noting that the light in the training environment is
located to the left-hand side, we also change the light directions to right
and above.

We then validate the OHPL agent, trained from the proxy task, in the robot
simulation workspace for the object reaching task. We use a 6 DoF UR5 robotic
arm [46] with the Robotiq 2F-85 gripper [47]. The arm is controlled by
specifying a pose goal in the Cartesian space. Actions issued to the arm have
a fixed displacement value of 0.02m. The object is placed on a platform in
front of the gripper at about 30 steps away. $360\times 360\times 3$ images
are captured at each step. We consider the task completed when the gripper
reaches a predefined area in front of the object.

We compare the OHPL agent, without robot training, and a model with robot
training. For the latter, we train an agent from scratch (Scratch) following
the DQN framework [42]. The Scratch agent first explores randomly for 1000
steps and then follows the $\epsilon$-greedy strategy. We pre-defined the
explorable region to prevent the agent from exploring states irrelevant to the
task and to prevent the robot from colliding with the environment e.g., a grey
box to place the target object. We set the maximum steps in an episode as
$T_{max}=50$ and give a positive extrinsic reward when successful reaching is
performed. In addition to the extrinsic reward, at each step, an intrinsic
reward is given based on the distance between the gripper and the object to
encourage the robot to reach the object and converge faster.

Finally, we validate the OHPL agent by deploying the trained agents to the
moving object reaching task where the environment is unseen to both the OHPL
agent and the agent with robot training (Scratch). We initialise the object
floating without the grey box in the environment. During each step, we
randomise the velocity of the object in the x, y and z directions. The same
success criteria from the static object reaching task is used. Experiments are
performed on all 8 objects in 3 lighting conditions and 9 starting positions.
3 trials are performed for each combination so a total of 648 experiments are
performed for each model.

###  IV-B Network setting

In this section, we introduce the network configuration for OHPL agent. For
sequential object localisation, an input state is cropped from the RL
environment $\mathcal{I}$ based on the RoI. The hand-eye view is used as the
input state for object reaching task in the robot workspace. For both tasks,
the input state is then resized to $84\times 84\times 3$. We adopt the context
aggregation network [48] architecture with the rectified linear unit (ReLU) as
nonlinear activation for the dynamic filter using 4 convolution layers with 3
intermediate feature maps. We set the filter kernels size to $3\times 3$. The
last convolution layer applies a $1\times 1$ convolution followed by a sigmoid
to predict the final output, which is then combined with the RoI. The dilation
factors of the convolution layers are 1, 2, 4, and 1, respectively. As
Q-Network we use the vanilla DQN [42] with minor modification to the final
fully connected layer with a single output for each valid action. The
experience replay has a $5k$ memory volume. Note that the dynamic filter makes
0.01$\%$ parameter increase to the total number of parameters of the network.
The network is optimised by RMSprop [49], where the learning rate is set to
$0.001$ and decays exponentially. We adopt the $\epsilon$-greedy strategy
during training and exponentially anneal $\epsilon$ from 0.95 to 0.05 [42].

TABLE I: Success ratio, with standard error in brackets, for the static object reaching task in the robot workspace with and without training in the robot workspace (Robot.), and with (Arch. full) and without (Arch. base) the dynamic filter  Model | Initial position
---|---
Method | Robot. | Arch. | M | ML | MR | T | TL | TR | B | BL | BR
Scratch | ✓ | base |  100.0 (0.0) |  100.0 (0.0) |  87.5 (6.8) |  100.0 (0.0) |  91.7 (5.6) |  100.0 (0.0) |  100.0 (0.0) |  100.0 (0.0) |  95.8 (4.1)
✓ | full |  100.0 (0.0) |  100.0 (0.0) |  100.0 (0.0) |  100.0 (0.0) |  100.0 (0.0) |  87.5 (6.8) |  87.5 (6.8) |  100.0 (0.00) |  87.5 (6.8)
OHPL |  | base |  95.8 (4.1) |  95.8 (4.1) |  75.0 (8.8) |  91.7 (5.6) |  95.8 (4.1) |  70.8 (9.3) |  20.8 (8.3) |  8.3 (5.6) |  4.2 (4.1)
| full |  95.8 (4.1) |  79.2 (8.3) |  79.2 (8.3) |  79.2 (8.3) |  87.5 (6.8) |  83.3 (7.6) |  37.5 (9.9) |  54.2 (10.2) |  4.2 (4.1)
|  |  |  |  |  |  |  |  |  |  |
$0$$1$Success ratebaseballcup-Hcup-Jrubiks cube00.5k1k$0$$1$EpisodeSuccess
rateapple00.5k1kEpisodemug00.5k1kEpisodecan00.5k1kEpisodecracker box Figure 6:
Training the agent from scratch (Scratch-base) in the robot workspace. We
report the success rates (vertical axis) with a running average over past 30
episodes for each episode (horizontal axis) on 8 YCB objects.

Figure 7: Examples of hand-eye view from different initial positions, (from
left to right) bottom-left (BL), bottom-right (BR), top-left (TL), and top-
right (TR). The view from BL and BR is extremely different from the RL
environment $\mathcal{I}$ (Fig. 5), which results in low success of OHP
agents.

###  IV-C Discussion

Fig. 4 shows the results for sequential object localisation with varying the
image conditions. OHPL-full generally outperforms OHPL-base in localising the
object, and the model shows consistent performance in noisy and blurry images.
But the performance drops when Gaussian blurring is applied with a $15\times
15$ kernel. Also, the performance decreases when the light source is located
to right, demonstrating that the performance can be affected when significant
light change happens.

Table I shows the evaluation of the static object reaching task in the robot
workspace. Note that Scratch is the upper bound of OHPL as the agent of
Scratch models are trained directly in the robot workspace with robot actions.
Scratch models are hence overfitted to the environment as the models have
experiences with the real view changes and rewards in the robot workspace.
However, to learn the control policies for the object-reaching task, each
Scratch model requires more than 500 episodes of training in the robot
workspace as shown in Fig. 6. The OHPL agents, without directly learning from
a robot workspace and without robot actions, show good results when the
initial position is top, T, TL and TR, or middle, M, ML and MR, while showing
low success rates in bottom cases, B, BL, and BR. As shown in Fig. 7, the
hand-eye view from the bottom position is extremely different from the middle
view which is used as the RL environment $\mathcal{I}$ (see Fig. 5). OHPL-
full, with the additional dynamic filter, records better success rates than
OHPL-base on average, by achieving better performance in bottom cases.

For a fair comparison, we further evaluate the agents when they are deployed
to the moving object reaching task. Note that this task can be considered as
an unseen scenario to both OHPL and Scratch models. Table II shows the success
ratio of the models. Compared to the success ratio of the static object
reaching task, Scratch models show significant performance decrease when they
are deployed to the moving object reaching task. While OHPL agents trained
their control policies within a single view captured from the static object
reaching environment, they generally show good performance and are competitive
to the Scratch agents.

##  V Conclusion

We presented a new approach to learn robot control policies with a proxy task:
RL-based object localisation with a single image captured from the robot
workspace. The key idea is that view changes of the RoI in a 2D image are
similar to the changes of the hand-eye camera view in the robot workspace. We
showed that the policy learned from OHPL can be deployed to the robot
workspace directly as the control policy for the reaching task, without
finetuning. We also presented a dynamic filter that works as soft attention of
an input image. With only 0.01% parameter increase, the network has better
generalisation when the agent is deployed to perform static and moving object
reaching task.

Future work includes validating the proposed framework with a physical robot,
applying to other robotic tasks, and learning policies from different
viewpoints.

TABLE II: Success ratio, with standard error in brackets, of the models deployed to the moving object reaching task in the robot workspace with (Arch. full) and without (Arch. base) the dynamic filter.  Model | Object
---|---
Method | Arch. | ball | cup-H | cup-J | rubiks | apple | mug | can | cracker
Scratch | base |  28.4 (5.0) |  87.7 (3.7) |  77.8 (4.6) |  92.6 (2.9) |  58.0 (5.5) |  64.2 (5.3) |  85.2 (3.9) |  63.0 (5.4)
full |  29.6 (5.1) |  75.3 (4.8) |  58.0 (5.5) |  42.0 (5.5) |  71.6 (5.0) |  55.6 (5.5) |  84.0 (4.1) |  98.8 (1.2)
OHPL | base |  67.9 (5.2) |  82.7 (4.2) |  54.3 (5.5) |  90.1 (3.3) |  69.1 (5.1) |  66.7 (5.2) |  56.8 (5.5) |  66.7 (5.2)
full |  79.0 (4.5) |  66.7 (5.2) |  84.0 (4.1) |  75.3 (4.8) |  92.6 (2.9) |  91.4 (3.1) |  90.1 (3.3) |  76.5 (4.7)
|  |  |  |  |  |  |  |  |

## Acknowledgement

This work is supported in part by the CHIST-ERA program through the project
CORSMAL, under the UK EPSRC grant EP/S031715/1, and the UK EPSRC grant NCNR
EP/R02572X/1.

## References

  * [1] N. Ratliff, J. A. Bagnell, and S. S. Srinivasa, “Imitation learning for locomotion and manipulation,” in _Proceedings of the IEEE-RAS International Conference on Humanoid Robots. (ICHR)_ , 2007.
  * [2] J. Peters and S. Schaal, “Reinforcement learning of motor skills with policy gradients,” _Neural Networks_ , vol. 21, no. 4, pp. 682–697, 2008.
  * [3] P. Sharma, D. Pathak, and A. Gupta, “Third-person visual imitation learning via decoupled hierarchical controller,” in _Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)_ , 2019.
  * [4] S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” _The Journal of Machine Learning Research (JMLR)_ , vol. 17, no. 1, pp. 1334–1373, 2016.
  * [5] S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” _The International Journal of Robotics Research (IJRR)_ , vol. 37, no. 4-5, pp. 421–436, 2018.
  * [6] A. A. Rusu, M. Vecerik, T. Rothörl, N. Heess, R. Pascanu, and R. Hadsell, “Sim-to-real robot learning from pixels with progressive nets,” _arXiv preprint arXiv:1610.04286_ , 2016.
  * [7] K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige _et al._ , “Using simulation and domain adaptation to improve efficiency of deep robotic grasping,” in _Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)_ , 2018.
  * [8] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in _Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2017.
  * [9] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in _Proceedings of the International Conference on Machine Learning (ICML)_ , 2017.
  * [10] Y. Duan, M. Andrychowicz, B. Stadie, O. J. Ho, J. Schneider, I. Sutskever, P. Abbeel, and W. Zaremba, “One-shot imitation learning,” in _Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)_ , 2017.
  * [11] M. Gualtieri, A. ten Pas, K. Saenko, and R. Platt, “High precision grasp pose detection in dense clutter,” in _Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2016.
  * [12] U. Viereck, A. ten Pas, K. Saenko, and R. Platt, “Learning a visuomotor controller for real world robotic grasping using simulated depth images,” _arXiv preprint arXiv:1706.04652_ , 2017.
  * [13] M. Gualtieri and R. Platt, “Learning 6-dof grasping and pick-place using attention focus,” _arXiv preprint arXiv:1806.06134_ , 2018.
  * [14] M. Yan, I. Frosio, S. Tyree, and J. Kautz, “Sim-to-real transfer of accurate grasping with eye-in-hand observations and continuous control,” _arXiv preprint arXiv:1712.03303_ , 2017.
  * [15] S. Calinon, F. Guenter, and A. Billard, “On learning, representing, and generalizing a task in a humanoid robot,” _IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)_ , vol. 37, no. 2, pp. 286–298, 2007.
  * [16] P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal, “Learning and generalization of motor skills by learning from demonstration,” in _Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)_ , 2009.
  * [17] B. C. Stadie, P. Abbeel, and I. Sutskever, “Third-person imitation learning,” _arXiv preprint arXiv:1703.01703_ , 2017.
  * [18] P. Sermanet, C. Lynch, Y. Chebotar, J. Hsu, E. Jang, S. Schaal, and S. Levine, “Time-contrastive networks: Self-supervised learning from video,” in _Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)_ , 2018.
  * [19] S. Song, A. Zeng, J. Lee, and T. Funkhouser, “Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations,” _IEEE Robotics and Automation Letters (RA-L)_ , vol. 5, no. 3, pp. 4978–4985, 2020.
  * [20] D. A. Pomerleau, “Efficient training of artificial neural networks for autonomous navigation,” _Neural Computation_ , vol. 3, no. 1, pp. 88–97, 1991.
  * [21] P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in _Proceedings of the International Conference on Machine learning (ICML)_ , 2004.
  * [22] L. Tai, J. Zhang, M. Liu, J. Boedecker, and W. Burgard, “A survey of deep network solutions for learning control in robotics: From reinforcement to imitation,” _arXiv preprint arXiv:1612.07139_ , 2016.
  * [23] D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, and S. Levine, “Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation,” _arXiv preprint arXiv:1806.10293_ , 2018.
  * [24] E. Tzeng, C. Devin, J. Hoffman, C. Finn, P. Abbeel, S. Levine, K. Saenko, and T. Darrell, “Adapting deep visuomotor representations with weak pairwise constraints,” _Algorithmic Foundations of Robotics XII_ , vol. 13, pp. 688–703, 2016.
  * [25] A. Saxena, J. Driemeyer, and A. Y. Ng, “Robotic grasping of novel objects using vision,” _The International Journal of Robotics Research (IJRR)_ , vol. 27, no. 2, pp. 157–173, 2008.
  * [26] S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, “Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019.
  * [27] S. James, A. J. Davison, and E. Johns, “Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task,” _arXiv preprint arXiv:1707.02267_ , 2017.
  * [28] S. James, M. Bloesch, and A. J. Davison, “Task-embedded control networks for few-shot imitation learning,” _arXiv preprint arXiv:1810.03237_ , 2018.
  * [29] A. Bonardi, S. James, and A. J. Davison, “Learning one-shot imitation from humans without humans,” _IEEE Robotics and Automation Letters (RA-L)_ , vol. 5, pp. 3533–3539, 2020.
  * [30] J. Matas, S. James, and A. J. Davison, “Sim-to-real reinforcement learning for deformable object manipulation,” _arXiv preprint arXiv:1806.07851_ , 2018\.
  * [31] S. Iqbal, J. Tremblay, A. Campbell, K. Leung, T. To, J. Cheng, E. Leitch, D. McKay, and S. Birchfield, “Toward sim-to-real directional semantic grasping,” in _Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)_ , 2020.
  * [32] P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, “Extracting and composing robust features with denoising autoencoders,” in _Proceedings of the International Conference on Machine learning (ICML)_ , 2008.
  * [33] D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context encoders: Feature learning by inpainting,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016.
  * [34] R. Zhang, P. Isola, and A. A. Efros, “Colorful image colorization,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2016\.
  * [35] C. Doersch, A. Gupta, and A. A. Efros, “Unsupervised visual representation learning by context prediction,” in _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , 2015.
  * [36] D. Pathak, R. Girshick, P. Dollár, T. Darrell, and B. Hariharan, “Learning features by watching objects move,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017.
  * [37] M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep clustering for unsupervised learning of visual features,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018.
  * [38] D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self-supervised prediction,” in _Proceedings of the International Conference on Machine Learning (ICML)_ , 2017.
  * [39] T. Lesort, N. Díaz-Rodríguez, J.-F. Goudou, and D. Filliat, “State representation learning for control: An overview,” _Neural Networks_ , vol. 108, pp. 379–392, 2018.
  * [40] S. Yun, J. Choi, Y. Yoo, K. Yun, and J. Y. Choi, “Action-decision networks for visual tracking with deep reinforcement learning,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017.
  * [41] Z. Jie, X. Liang, J. Feng, X. Jin, W. Lu, and S. Yan, “Tree-structured reinforcement learning for sequential object localization,” in _Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)_ , 2016.
  * [42] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” _arXiv preprint arXiv:1312.5602_ , 2013.
  * [43] X. Jia, B. De Brabandere, T. Tuytelaars, and L. V. Gool, “Dynamic filter networks,” in _Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)_ , 2016.
  * [44] B. Calli, A. Walsman, A. Singh, S. Srinivasa, P. Abbeel, and A. M. Dollar, “Benchmarking in manipulation research: Using the yale-cmu-berkeley object and model set,” _IEEE Robotics Automation Magazine_ , vol. 22, no. 3, pp. 36–52, 2015.
  * [45] E. Coumans and Y. Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” http://pybullet.org, 2016–2019.
  * [46] “Universal robots.” http://www.universal-robots.com/, accessed: 2020-05-23.
  * [47] “Robotiq 2f-85 gripper.” http://robotiq.com/products/adaptive-robot-gripper/, accessed: 2020-05-23.
  * [48] F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,” _arXiv preprint arXiv:1511.07122_ , 2015.
  * [49] T. Tieleman and G. Hinton, “Lecture 6.5-rmsprop, coursera: Neural networks for machine learning,” _University of Toronto, Technical Report_ , 2012.


Send correspondence to H.C.G or J.T.:
H.C.G. E-mail<EMAIL_ADDRESS>
J.T. E-mail<EMAIL_ADDRESS>
# Across the soft gamma-ray regime: utilizing simultaneous detections in the
Compton Spectrometer and Imager (COSI) and the Background and Transient
Observer (BTO) to understand astrophysical transients

Hannah C. Gulick  University of California, Berkeley, Department of Astronomy,
Berkeley, CA, United States, 94720, USA  Space Sciences Laboratory, University
of California, Berkeley, CA 94720, USA  Eliza Neights  George Washington
University, USA  NASA Goddard Space Flight Center, 8800 Greenbelt Road,
Greenbelt, MD 20771, USA  Samer Al Nussirat  Space Sciences Laboratory,
University of California, Berkeley, CA 94720, USA  Claire Tianyi Chen  Space
Sciences Laboratory, University of California, Berkeley, CA 94720, USA  Kaylie
Ching  Space Sciences Laboratory, University of California, Berkeley, CA
94720, USA  Cassandra Dove  Space Sciences Laboratory, University of
California, Berkeley, CA 94720, USA  Alyson Joens  Space Sciences Laboratory,
University of California, Berkeley, CA 94720, USA  Carolyn Kierans  NASA
Goddard Space Flight Center, 8800 Greenbelt Road, Greenbelt, MD 20771, USA
Hubert Liu  Space Sciences Laboratory, University of California, Berkeley, CA
94720, USA  Israel Martinez  NASA Goddard Space Flight Center, 8800 Greenbelt
Road, Greenbelt, MD 20771, USA  University of Maryland, USA  Tomas Mician
Space Sciences Laboratory, University of California, Berkeley, CA 94720, USA
Shunsaku Nagasawa  Space Sciences Laboratory, University of California,
Berkeley, CA 94720, USA  Shreya Nandyala  Space Sciences Laboratory,
University of California, Berkeley, CA 94720, USA  Isabel Schmidtke  Space
Sciences Laboratory, University of California, Berkeley, CA 94720, USA  Derek
Shah  Space Sciences Laboratory, University of California, Berkeley, CA 94720,
USA  Andreas Zoglauer  Space Sciences Laboratory, University of California,
Berkeley, CA 94720, USA  Kazuhiro Nakasawa  KMI, Nagoya University, Nagoya,
Aichi, Japan  Tadayuki Takahashi  Kavli Institute for the Physics and
Mathematics of the Universe (WPI), University of Tokyo, Kashiwa, Chiba
277-8583, Japan  Department of Physics, University of Tokyo, Bunkyo, Tokyo
113-0033, Japan  Juan-Carlos Martinez Oliveros  Space Sciences Laboratory,
University of California, Berkeley, CA 94720, USA  John A. Tomsick  Space
Sciences Laboratory, University of California, Berkeley, CA 94720, USA

###### Abstract

The Compton Spectrometer and Imager (COSI) is a NASA funded Small Explorer
(SMEX) mission slated to launch in 2027. COSI will house a wide-field gamma-
ray telescope designed to survey the entire sky in the 0.2-5 MeV range. Using
germanium detectors, the instrument will provide imaging, spectroscopy, and
polarimetry of astrophysical sources with excellent energy resolution and
degree-scale localization capabilities. In addition to the main instrument,
COSI will fly with a student collaboration project known as the Background and
Transient Observer (BTO). BTO will extend the COSI bandpass to energies lower
than 200 keV, thus enabling spectral analysis across the shared band of 30
keV–2 MeV range. The BTO instrument will consist of two NaI scintillators and
student-designed readout electronics. Using spectral information from both the
COSI and BTO instruments, physics such as the energy peak turnover in gamma-
ray bursts, the characteristics of magnetar flares, and the event frequency of
a range of transient phenomena will be constrained. In this paper, we present
the expected science returnables from BTO and comment on the shared
returnables from the COSI and BTO missions. We include simulations of gamma-
ray bursts, magnetar giant flares, and terrestrial gamma-ray flashes using
BTO’s spectral response. Additionally, we estimate BTO’s gamma-ray burst
detection rate and find that BTO will detect $\sim$150 gamma-ray bursts per
year, with most of these events being long bursts.

###### keywords:

Gamma-ray Survey, Gamma-ray Transients, COSI, Time Domain, Multimessenger

##  1 INTRODUCTION

The transient gamma-ray sky holds insights into the extreme energetics which
power and map events such as black hole accretion (Abdo et al., 2010; Ajello
et al., 2020), electron-positron annihilation (Bouchet et al., 2010;
Knödlseder et al., 2005), compact object formation and evolution (Abbott et
al., 2017a; Campana et al., 2006; Kumar & Zhang, 2015; MacFadyen & Woosley,
1999), and the progenitors of multimessenger events (Abbott et al., 2017a, b;
Eichler et al., 1989). Transient gamma-ray events are important for
understanding broader topics such as galaxy formation and evolution, the
history of star formation in the Milky Way, and dark matter. With peak
energies ranging from $\sim$1–100 keV for gamma-ray burst (GRB) and magnetar
flares (Pe’er et al., 2006; Kaspi & Beloborodov, 2017), to 100s of MeV for
blazars (Ajello et al., 2020; De Angelis et al., 2021), the gamma-ray bandpass
is extensive. As a consequence, a single instrument is not capable of
observing across the entire gamma-ray regime.

Spectral signatures from gamma-ray transients are variable across the gamma-
ray bandpass, requiring multiple instruments to observe and constrain event
characteristics. Events such as magnetar flares or Galactic electron-positron
annihilation occupy a finite fraction of the gamma-ray bandpass, peaking at
energies of 60 keV (Kaspi & Beloborodov, 2017) and 511 keV (Knödlseder et al.,
2005; Bouchet et al., 2010), respectively. Other phenomenon such as the energy
peak turnover in GRBs exhibit energy dependent structure (Pe’er et al., 2006;
Band et al., 1993), and thus require observations across a range of energies
to observe different underlying emission mechanisms. Therefore, to properly
constrain the physics powering the most energetic phenomenon in the Universe,
simultaneous observations are needed from complimentary instruments
specialized to different energy ranges.

Historically, gamma-ray observatories have included multiple instruments on a
single spacecraft platform to enable observations across a wide range of
energies. For example, the INTErnational Gamma-Ray Astrophysics Laboratory
(INTEGRAL) which launched in 2002 included both a gamma-ray imager (Lebrun et
al., 2003, IBIS) and spectrometer (Vedrenne et al., 2003, SPI) covering
energies from $\sim$20 keV to $\sim$10 MeV. These instruments were
complimented by the Joint European X-ray Monitors (Lund et al., 2003, JEM-X 1&
2) which provided images in the 3–35 keV bandpass. The Astro-rivelatore Gamma
a Immagini Leggero (Marisaldi et al., 2010, 2015, AGILE) included two
imagers—the Gamma Ray Imaging Detector (Chen et al., 2013, GRID) sensitive in
the 30 MeV–50 GeV energy ranges as well as SuperAgile (Feroci et al., 2007,
SA) sensitive in the 18–60 keV energy range. Additionally, the Fermi satellite
includes both the Large Area Telescope (Atwood et al., 2009, LAT) providing
images in the 20 MeV to 300 GeV range as well as the Gamma-ray Burst Monitor
(Meegan et al., 2009, GBM) providing spectra in the 8 keV to 40 MeV.

The future Compton Spectrometer and Imager (Tomsick et al., 2023, COSI)
mission slated to launch in 2027, will also include multiple instruments
covering a wide energy bandpass. These instruments include the main COSI
gamma-ray telescope and a second soft gamma-ray spectrometer known as the
Background and Transient Observer (BTO). While COSI will provide high
resolution images, spectroscopy, and polarization measurements in the 200 keV
to 5 MeV range, BTO will extend the COSI bandpass to lower energies by
providing spectral information in the 30 keV to 2 MeV range. Due to their
overlapping bandpasses as well as shard FOVs, COSI and BTO will enable
spectroscopic analysis of gamma-ray transients over a large gamma-ray
bandpass. Future observations made with COSI and BTO will help to isolate GRB
photospheric and non-thermal emission components (Ryde et al., 2010; Guiriec
et al., 2011; Pe’er et al., 2006) and constrain the energy peak turnover which
typically occurs below 200 keV (Band et al., 1993). Additionally, BTO’s lower
bandpass and wider FOV is better suited to detect magnetar flares (Kaspi &
Beloborodov, 2017) and will thus improve the mission’s ability to probe the
characteristic timescales, temperatures, and brightness profiles associated
with these events. Finally, the combined COSI and BTO systems will yield
observations of multimessenger events which will be used to localize the event
progenitor and to measure the delay time between gravitational waves and their
corresponding electromagnetic counterparts (Abbott et al., 2017a, b).

In this paper, we outline the full mission overview in Section 2 and include
instrument specifics for COSI and BTO in Sections 2.1 and Section 2.2,
respectively. Section 3 reports on transient event simulations as well as the
expected GRB event rates in BTO. The scientific advantages achieved through
launching the complimentary COSI and BTO systems are also highlighted through
simulated spectra of transient events in 3.1. Conclusions are made in Section
4.

##  2 Mission Overview

The Compton Spectrometer and Imager (COSI) is a space-based gamma-ray survey
mission that will provide imaging, spectroscopy, and polarimetry of
astrophysical sources associated with the birth and death of stars, the
formation of chemical elements in the Milky Way, and the progenitors of
gravitational waves. Selected in 2021 as a NASA Small Explorer (SMEX) mission,
COSI is slated to launch to a low-Earth-orbit (LEO) in 2027. As of April 2024,
COSI passed its Key Decision Point review and was confirmed to proceed to
Phase C. The COSI instrument is discussed further in Section 2.1 and a full
instrument outline is included in (Tomsick et al., 2023).

In addition to the main instrument, the COSI SMEX proposal included a
secondary proposal for a student collaboration project which was successfully
funded. The student collaboration project aims to develop an additional gamma-
ray detector system to fly on COSI—known as the Background and Transient
Observer (BTO) (see Section 2.2). BTO will fly onboard the COSI spacecraft as
a do-no-harm payload to provide observations at lower energies (30 keV to 2
MeV) and across a larger field-of-view (FoV) ($>$ 60% of the sky) than the
main COSI instrument. BTO’s main science goals will be to detect gamma-ray
transients and to constrain the soft gamma-ray background rates at different
locations in the COSI orbit. Therefore, BTO will be highly complimentary to
the COSI mission as a whole by extending the mission’s bandpass, increasing
the on-sky coverage for transient monitoring, and improving data analysis
through improved background rate measurements.

Along with COSI, BTO has passed the Preliminary Design and Key Decision Point
reviews. BTO has a functioning engineering model currently.

###  2.1 The Compton Spectrometer and Imager

COSI is a Compton telescope capable of observing from 200 keV to 5 MeV with an
instantaneous FOV covering 25% of the sky. Figure 1 shows the COSI spacecraft
with the photon-sensitive subsystems labeled. The main telescope (labeled 1)
is composed of cryogenically-cooled germanium detectors which use a double-
sided strip structure to enable measurements of an interacting photon’s x and
y location as well as its depth (z). The instrument includes four stacks of
four detectors (16 detectors total), each with 64 strips per side and a
dimension of 8 cm $\times$ 8 cm $\times$ 1.5 cm. The detectors are surrounded
by a guard ring electrode which performs signal vetoing. An Application
Specific Integrated Circuit (ASIC) is used to read out the COSI detectors and
guard ring. With the cryogenic cooler, the detectors are cooled to 80–90 K,
allowing the instrument to achieve an excellent energy resolution of of 6 keV
at 0.511 MeV and 9 keV at 1.157 MeV (Tomsick et al., 2023). These resolutions
enable unprecedented access to the low-energy part of the gamma-ray
regime—otherwise known as the ’MeV Gap’ due to the observational challenges
associated with this bandpass (i.e. high background levels) and thus the lack
of existing observations.

---
Figure 1:  A schematic of the COSI spacecraft. An in-depth description of the
COSI instrument design is included in (Tomsick et al., 2023). For the purposes
of this paper, the following components are labeled: 1. the COSI germanium
detectors contained in a vacuum cryostat, 2. the active BGO shields, and 3.
the two BTO detectors.

The germanium detectors are surrounded on four sides and underneath by active
bismuth germanium oxide (BGO) scintillators. The BGO shields, shown in orange
in Figure 1, are readout by silicon photomultipliers (SiPMs) and act to
provide passive background attenuation as well as anti-coincidence information
on background radiation and gamma-rays that escape from the germanium
detectors. Light curve information from the BGO shields will be downlinked
from the spacecraft and used to improve background reduction in the COSI data
as well as identify transient events.

Due to COSI’s Compton telescope design, both the total energy and incident
angle of the incoming photon can be identified and used to localize and
reconstruct the source’s image on the sky. COSI will be capable of imaging
events with an angular resolution of 4.1$\degree$ and 2.1$\degree$ at 0.511
MeV and 1.809 MeV, respectively. As part of the mission requirements, COSI
will create full Galaxy images at 0.511 MeV as well as four nuclear line
energies. Additional information on COSI line sensitivities is included in
Table 1 of Tomsick et al. (2023). Additionally, since the azimuthal component
of the Compton scattering angle measured in the germanium detectors depends on
the polarization of the incoming gamma-ray, COSI will also provide
polarization measurements. The COSI detectors will be sensitive to
polarization down to a flux limit of 1.4 $\times 10^{-10}$ erg cm-2 s-1 in the
0.2-0.5 MeV range.

With the use of these three observing capabilities, COSI will address four
main science goals: 1. uncovering the origin of Galactic positrons through
studies of the 0.511 MeV emission from antimatter annihilation in the Galaxy
(Bouchet et al., 2010; Knödlseder et al., 2005), 2. revealing Galactic element
formation by mapping radioactive elements from nucleosynthesis (Eichler et
al., 1989), 3. gain insight into extreme environments through polarization
measurements of accreting black holes (Ajello et al., 2020), and 4. probing
the physics of multimessenger events through the observation of short gamma-
ray bursts (sGRB) (Abbott et al., 2017a, b).

###  2.2 The Background and Transient Observer

The BTO instrument will measure gamma-rays from transient events and monitor
the soft gamma-ray background rates in the 30 keV to 2 MeV regime. BTO will
utilize two detector modules—each composed of a rectangular NaI detector (see
Figure 2), an analog board with analog-to-digital converter (ADC), and a
digital board. The BTO detectors are shown in Figure 1 on the outer edges of
the spacecraft instrument deck—otherwise known as the Payload Interface Plate
(PIP). The BTO electronics are housed in a single, stacked electronics box
which is located on the underside of the PIP and therefore not visible in
Figure 1.

A trade study was conducted to optimize the scintillator material, size, and
shape, including an analysis of afterglow signatures in CsI versus NaI
scintillators Gulick et al. submitted. The custom BTO detectors are supplied
by Scionix and utilize components with space heritage. The detectors contain a
3.8 cm $\times$ 3.8 cm $\times$ 7.6 cm NaI crystal hermetically sealed in a
4.5 cm $\times$ 4.5 cm $\times$ 12 cm aluminum housing. The detector energy
resolution is 15–20% at 662 keV. Figure 2 shows a diagram of the detector.
Each detector includes four ArrayJ-60035-4P-PCB SiPMs for detector readout and
a proprietary built-in amplifier. The flight model detectors include a single
SMA input for $\pm$ 12 V power and two SMA outputs—one for the ’raw’ SiPM
signal and one for the amplified detector signal.

---
Figure 2:  A schematic of the Scionix NaI detector package customized for the
BTO mission.

The detectors are located on the outer edges of the COSI PIP (see Figure 1) to
optimize the BTO FOV to $>$60% of the sky. With only two detectors, BTO will
not be capable of providing precise source localization. Therefore, the
mechanical housing which affixes the detector to the PIP was designed with a
’pedestal’ to raise the crystal by 11.5 cm. The pedestal—along with the
detector placement—ensures the BTO detectors are not obscured by other
components on the COSI PIP and results in the full COSI FOV being overlapped
by part of each BTO detector’s FOV. Thus any BTO detected event which occurs
in the shared FOV will be localized to $\sim$4.1$\degree$ by the COSI
germanium detectors. Furthermore, the placement of the BTO detectors on
opposite sides of the COSI PIP allows shadowing from the germanium detectors,
BGO shields, and other components on the COSI PIP to be used to roughly
constrain the location of events outside of the COSI FOV to be of the Sun,
Earth, or astrophysical origins.

Each detector will have its own set of readout electronics, consisting of an
analog, ADC, and digital board. The analog board will process the amplified
signal from the NaI detector and convert it to a digital signal with an
analog-to-digital converter (ADC). Following the ADC, the digital board will
be equipped with two modes of operation. Due to data downlinking limits, BTO
will nominally operate in a ’binned mode’ during which all counts over a 1 ms
timescale will be saved to a binned histogram and downlinked. To preserve
timing information for true gamma-ray events, however, the digital board will
switch to an ’event-by-event mode’ once triggered. In event-by-event mode, the
timing and energy information for every event will be individually saved and
downlinked. The event-by-event mode is especially important for the analysis
of short gamma-ray events such as terrestrial gamma-ray flashes (TGFs).

Following the readout and processing system will be a power board and
interface board. The power board will be responsible for converting and
filtering the spacecraft’s 28 V input to $\pm$12 V for the detector and
analog/ADC boards and to $\pm$5 V for the digital boards. The interface board
will be responsible for relaying commands and data to and from the BTO system
to the spacecraft control panel. As a do-no-harm payload, the power and
interface boards will act as the only electrical points of contact with the
COSI spacecraft.

By observing at lower energies than COSI, BTO will probe interesting science
associated with gamma-ray bursts (GRB) and magnetar flares. The BTO bandpass
is ideal for constraining the energy peak turnover in lGRB and sGRB (Band et
al., 1993) and for isolating photospheric components in GRB spectra (Pe’er et
al., 2006; Ryde et al., 2010; Guiriec et al., 2011). BTO will also be capable
of measuring the time delay between gravitational wave events and their sGRB
electromagnetic counterparts (Abbott et al., 2017a, b). Additionally, good
timing resolution ($<$0.5 ms) is necessary to detect TGF events from
lightening on Earth with timescales $<$ 1 ms) (Smith et al., 2005;
Grefenstette et al., 2009). BTO’s event-by-event mode achieves this
resolution. Furthermore, due to the anti-coincidence requirement between the
Compton telescope and active BGO shields on COSI, BTO will be the only
instrument on the spacecraft which saves and downlinks TGF event information.

##  3 Observing Astrophysical Transients with BTO & COSI

In this section we provide an in-depth discussion of the estimated science
returnables from the BTO instrument for several transient phenomena. We use
the Medium Energy Gamma-ray Library (Zoglauer et al., 2006, MEGAlib) to
simulate a BTO detector response curve and corresponding event rates and
spectra for GRBs, magnetar GFs, and TGFs.

We also comment on the scientific advantages enabled through the compatibility
of the COSI and BTO instruments. Due to their overlapping bandpasses and FOVs,
the combined COSI and BTO instruments will provide spectral information from
30 keV to 2 MeV for gamma-ray transients which occur in their shared FOVs. The
COSI response values and simulated light curves utilized in the sections below
were created as part of a larger project which will be published in an
upcoming manuscript by Neights et al. in prep.

###  3.1 Gamma-ray Bursts

We estimate the number of GRBs detectable by BTO using the MEGAlib package
GRBSampler. This package uses a catalog of real GRB observed by the Burst and
Transient Source Experiment (BATSE) Goldstein et al. (2013) over 3323 days
(effective exposure time of 2390 days). The catalog contains 2,145 GRB, with
22% of the events being sGRB. To create a population of simulated GRB, the
GRBSampler package randomly samples probability weighted histograms of the
following parameters: GRB duration (s), GRB flux (photons/cm2/s), the Band
function lower photon index, the Band function upper photon index, the Band
function break energy, and the GRB’s polar ($\theta$) and azimuthal ($\phi$)
angle on the sky.

We simulated 5000 GRB to create a robust sample. To account for the Earth
occultating the BTO FOV, only GRB with a $\theta$ $>$112.5∘ were included in
the study. For each remaining GRB, we calculate the signal-to-noise ratio
(SNR) via:

$SNR=\frac{f_{GRB}\sqrt{t}}{\sqrt{f_{GRB}+f_{b}}}$

where $f_{GRB}$ is the GRB’s flux (photon/s), $f_{b}$ is the background flux
(photon/s), and $t$ is the exposure time (s). The background flux is
calculated for the BTO geometry using MEGAlib as outlined in Gulick et al.
submitted. The average expected background rate for a single BTO detector is
57 counts/s for an orbit with an 550 km altitude and 0$\degree$ inclination.

Figure 3 shows the detection significance versus polar angle for the 5000 GRB
simulated with the NaI detector. The 3$\sigma$, 5$\sigma$, and 10$\sigma$
levels are denoted by red dotted, dashed, and solid lines respectively. In the
legend, $N_{det}$ refers to the number of GRB detected above each SNR
interval.

---
Figure 3:  The detection significance versus polar angle on the sky for 5000
simulated GRB events. In these simulations, a single BTO NaI detector geometry
was used to calculate the expected flux from each GRB. The SNR is then
calculated for each GRB as described in Section 3.1. The red dotted, dashed,
and solid lines mark the 3, 5, and 10 $\sigma$ confidence levels. The variable
$N_{det}$ gives the number of GRB which are detected above each significance
level.

To determine the number of GRB visible in BTO over the nominal mission
duration of 2 years, we scale the simulated population with the BATSE
detection rate. The BATSE catalog effective exposure time is 2390 days with
2,145 total detections. Extrapolating outwards to determine the effective
exposure time required to detect the simulated data population, we expect
BATSE to make 5000 detections in $\sim$4600 days or $\sim$12.6 years.
Therefore, we scale the total simulated/observable population by 0.16 to
estimate the number of detections over two years. Furthermore, the simulations
were performed for a single BTO detector with an all-sky FOV so the data are
further scaled by 0.6 to represent the BTO detector’s FOV.

This corresponds to $\sim$295 GRB detections with both BTO detectors over the
full 2-year mission (148 GRB per year) or a detection rate of $\sim$0.4 GRB
per day. Comparatively, the Gamma-ray Burst Monitor on the Fermi Observatory
triggers on $\sim$250 GRB per year (Meegan et al., 2009). GBM, while using the
same NaI scintillation material, has a larger effective area ($\sim$120 cm2
for a single detector (Tsvetkova et al., 2022; Meegan et al., 2009)) than BTO
(peak $\sim$50 cm2 for both detectors) and observes from $\sim$8 keV to
$\sim$40 MeV. Therefore, from the ratio of effective areas, it is expected
that GBM has $\sim$2.4$\times$ the detection rate of BTO. Therefore, the
current GRB rate estimates for BTO might be slightly overestimated (i.e.
closer to 100 GRB per year). This value will be refined in future work as the
BTO detector response matrix is completed across the full sky.

Figure 4 shows the distribution of GRB event durations for all GRB events over
the two year mission duration in grey and the short and long GRB events with a
$>$5$\sigma$ detection level in pink and orange, respectively. The
distribution is fit with a bimodal function with the first peak representing
the sGRB population, and the second peak representing the lGRB distribution.

---
Figure 4:  The $t_{90}$ distribution for GRB events simulated with MEGAlib.
The histogram has been scaled from the initial simulated population containing
5000 events to reflect a 2-year observing period and an FOV covering $\sim$60%
of the sky (see Section 3.1). The $t_{90}$ distribution for all simulated
events occurring over the 2-year mission window is shown in grey. The sGRB and
lGRB events that are observable by BTO with a $>$5$\sigma$ detection level are
shown in pink and orange, respectively. The distribution is fit with a bimodal
function.

At a 5$\sigma$ confidence level, approximately 292 of the GRB detections over
two years will be lGRB and 3 will be sGRB. For a 3$\sigma$ confidence level,
we expect to have $\sim$8 sGRB and $\sim$340 lGRB over the two year mission.
Typically, $\sim$20% of GRB detections are sGRB, as shown by Konus-Wind and
GBM (Tsvetkova et al., 2022). The Swift-Burst Alert Telescope (Swift-BAT) had
a slightly lower rate with $\sim$9% of detections contributed by sGRB.
However, only $\sim$1% of BTO’s simulated detections are sGRB. Similar to the
results found in Lien et al. (2016) for the Swift-BAT telescope, we find that
shorter gamma-ray events are difficult for the BTO geometry to detect if they
are 1. highly off-axis or 2. both low flux and short duration. As is expected,
the BTO sensitivity increases with exposure time thus BTO is capable of
detecting high-flux sGRB, or low-flux sGRB with longer durations. Therefore,
the BTO detectors will be biased toward the longest or brightest sGRB events.

Figure 5 shows the lightcurve (top) and the corresponding BTO and COSI spectra
(bottom) for a simulated lGRB event directly above the COSI/BTO instruments
(i.e. on-axis). The parameters for the GRB are included in Table 1. The COSI
spectrum is converted from units of photons/cm-2/s/keV to units of
photons/s/keV using the instrument’s effective area over the 100 keV to 5 MeV
bandpass (Neights et al. in prep). With an effective area peaking at $\sim$30
cm2 from $\sim$200–500 keV (Tomsick et al., 2023), COSI has a better response
than the BTO detectors and achieves good statistics to higher energies. In the
case of a typical lGRB, COSI provides well constrained data out to $\sim$5
MeV. BTO, on the other hand, has a sufficient response down to 30 keV and
extends the observed bandpass to lower energies to capture the energy peak
turnover at $\sim$100 keV. BTO is limited at higher energies, providing good
statistics only up to $\sim$1 MeV. Unlike COSI, BTO will return event-by-event
data for triggered events in the 30 keV to 10 MeV range. Therefore, the BTO
spectrum can utilize finer bins, while event count rates allow.

Figure 5:  [top] The lightcurve for a simulated long GRB with parameters
listed in Table 1. [bottom] The spectral response for the simulated GRB in
units of ph/s/keV for COSI (dashed black line) and BTO (green line). The COSI
spectrum covers 100 keV to 10 MeV and includes 10 energy bins. The COSI
binning is set by computational limitations associated with calculating the
current detector response matrix. In the future, the number of bins could
increase. The BTO data covers 30 keV to 2 MeV with 200 energy bins, thus
creating a finer sampling of the lGRB’s spectral structure. The count rates
for COSI and BTO become very low past 5 MeV and 1 MeV, respectively. However,
spectral information up to 10 MeV and 2 MeV for COSI and BTO, respectively,
will be recorded and used for fitting. The errors are given by Poisson
statistics.

Figure 6 shows the lightcurve (top) and the corresponding BTO and COSI spectra
(bottom) for a simulated sGRB at 70$\degree$ off-axis from the COSI/BTO
instrument zenith. The parameters for the GRB are included in Table 1. The
COSI and BTO spectra includes 10 and 25 energy bins, respectively. Similar to
the lGRB, BTO is capable of providing well constrained data at energies lower
than 100 keV but becomes dominated by errors closer to $\sim$600–700 keV. This
is due to the sGRB having less total counts than the lGRB. COSI provides good
statistics past 1 MeV.

Figure 6:  [top] The lightcurve for a simulated short GRB with parameters listed in Table 1. [bottom] The spectral response for the simulated GRB in units of ph/s/keV for COSI (dotted black line) and BTO (green line). The binning for the COSI data is the same as in Figure 5. The BTO data uses 25 energy bins. The errors assume Poisson statistics. Table 1: Simulated GRB Parameters Property | Long GRB | Short GRB
---|---|---
Spectrum Type | Band | Comptonized
Parameters |  Ebreak = 841.8 |  Epeak = 1482.6
|  $\alpha$ = -1.02 |  $\alpha$ = -0.71
|  $\beta$ = -2.83 |  —
Duration (s) | 40.6 | 0.41
Average Flux [ph/cm2/s] | 10.72 | 35.38

###  3.2 Magnetar Flares

There are two types of gamma-ray signals originating from magnetars: soft
gamma-ray repeaters (SGRs) and giant flares (GFs). SGRs occur in young, slowly
rotating magnetars and appear as dozens of rapid, repeating soft gamma-ray
signals (Thompson & Duncan, 1996). SGRs have a known initial hard transient
phase of 0.2s (Duncan & Thompson, 1992; Thompson & Duncan, 1995). About once
every 50 years (Palmer et al., 2005), an SGR’s host magnetar will emit a GF
with an energy $>$103 times the energy of the typical SGR burst in less than
one second (Mazets et al., 1979; Fenimore et al., 1996; Hurley et al., 1999;
Mereghetti et al., 2024). This prompt emission often peaks at $>$10–100s of
keV (Mereghetti et al., 2024; Kaspi & Beloborodov, 2017; Boggs et al., 2007)
and is well constrained in the 20 keV to 1 MeV band. The prompt GF emission is
followed by a softer emission tail spanning several minutes and characterized
by a periodic modulation intrinsic to the neutron star’s rotation (Mereghetti
et al., 2024). There are nearly 30 SGR catalogued in the Milky Way Galaxy,
however very few have exhibited GFs. The few confirmed GFs include SGR 0526-66
(Mazets et al., 1979; Cline et al., 1980; Mazets et al., 1982), SGR 1900+14
(Hurley et al., 1999; Feroci et al., 1999; Mazets et al., 1999), and SGR
1806-20 (Hurley et al., 2005; Boggs et al., 2007; Palmer et al., 2005;
Frederiks et al., 2007). GFs outside of the galaxy are difficult to
distinguish from sGRB, though it is possible to identify extragalactic GF
through the event’s spectral and timing properties as well as a lack of
gravitational waves (Mereghetti et al., 2024; Minaev et al., 2024).

BTO’s bandpass and large FOV make it a great instrument for detecting rare,
extragalactic events which peak in the soft gamma-ray regime—such as magnetar
GFs. Additionally BTO’s smaller effective area (when compared to COSI or GBM)
is advantageous when it comes to magnetar GFs within the Galaxy. This is due
to a GF’s brief but incredibly energetic prompt emission being strong enough
to overwhelm gamma-ray instruments, thus temporarily blinding them. With a
smaller effective area, BTO will be blinded for less time and therefore
capable of monitoring the prompt emission decay and subsequent modulating
emission tail. Simulations of Galactic GF with BTO and COSI are ongoing.

While unable to observe the prompt emission from Galactic magnetar GFs, both
COSI and BTO will be capable of observing prompt emission from extragalactic
giant flares. Using the spectral information from Mereghetti et al. (2024) and
Minaev et al. (2024) for GRB231115A—a high probability extragalactic magnetar
GF—we simulate BTO’s spectral response for a similar GF at the instrument’s
zenith. Figure 7 shows the event’s observed spectrum from GBM (Minaev et al.,
2024) and the simulated BTO spectrum. The observed GBM spectrum was fit over
the time interval -0.02 s to 0.05 s with a comptonized model. The best fit
parameters for this model are: normalization coefficient $A$ = 6.27$\pm 0.7$
photons/cm2/s/keV, a photon index of $\alpha$ = -0.34$\pm$0.2, and a peak
energy $E_{p}$ = 637$\pm\sim 70$ keV. The GBM spectrum was converted from
photons/cm2/s/keV to photons/s/keV using the GBM effective area curve fit in
Smith (2019). The BTO spectrum was simulated with MEGAlib using the best fit
parameters from GBM in the comptonized model and a fluence of 7.25e-7 erg/cm2
in the 10–1000 keV range (Minaev et al., 2024). The simulations for COSI
magnetar detections are ongoing.

---
Figure 7:  The spectra for GRB231115A—a magnetar giant flare event in M82. The
GBM spectrum (purple) is calculated from the fit to the observed GBM spectrum
performed by Minaev et al. (2024). The BTO spectrum (green) is simulated based
off the GBM fit parameters. Poisson statistics are used to calculate the error
bars on each energy bin.

###  3.3 Terrestrial Gamma-ray Flashes

TGFs are rapid and intense bursts of gamma-rays originating in the Earth’s
atmosphere. Thunderstorms—and specifically lightning activity—are thought to
be the progenitors of these events. However, the acceleration mechanism behind
TGFs is not well understood (Briggs et al., 2010; Dwyer et al., 2012). While
largely studied from the ground (Belz et al., 2017), TGFs are also detected by
space-based gamma-ray telescopes such as the Reuven Ramaty High Energy Solar
Spectroscopic Imager (Smith et al., 2005; Grefenstette et al., 2009, RHESSI),
GBM (Briggs et al., 2010), AGILE (Marisaldi et al., 2010, 2015), and were even
discovered by BATSE in 1994 (Fishman et al., 1994).

TGF signals are very brief, with a single TGF pulse lasting less than a
millisecond. A TGF event typically consists of a single pulse with energies
extending up to 25 MeV, though events have been reported up to 38 MeV (Briggs
et al., 2010). TGF emission is well modeled by bremsstrahlung radiation
emitted by electrons accelerated in the relativistic runaway electron
avalanche (RREA) process (Gurevich et al., 1992). Due to COSI/BTO’s
$\sim$0$\degree$111The current requirement for the COSI spacecraft is an orbit
with $<$2$\degree$ inclination inclination orbit, the instruments will follow
the Earth’s equator and spend periods of time over tropical regions where
storms frequently occur thus increasing detection probabilities. However,
great timing resolution ($<$0.5 ms) is needed to detect these events. BTO’s
event-by-event mode satisfies this requirement.

We simulate the BTO instrument response to a TGF event using MEGAlib. We build
a TGF spectrum using the model from Lindanger et al. (2021):

$f(E)=\frac{1}{E}\exp{\frac{-E}{7.3\mathrm{MeV}}}$

where $E$ is the incoming photon energy in MeV. The spectrum is calculated
from 30 keV to 40 MeV. The TGF photons are then beamed upward with a Gaussian
beaming profile with a half angle of 30$\degree$ and Gaussian 1$\sigma$ value
of 18$\degree$ as determined from Appendix B of Lindanger et al. (2021). To
simulate TGF events occurring on Earth, the TGF spectrum is passed through an
atmospheric model created with the Python package pyatmo222Website:
https://github.com/jabesq-org/pyatmo. The resulting spectrum contains
information on atmospheric interactions and is saved as the “post-atmosphere”
TGF spectrum. The BTO detector response is then simulated by “observing” the
post-atmosphere TGF spectrum with the BTO detector geometry.

Figure 8 shows the normalized TGF spectrum as observed by a single BTO
detector for a TGF event with a starting altitude of 10 km and a 0.001 s
duration (Lindanger et al., 2021). The spectrum is calculated from 30 keV to 2
MeV range and separated into 25 energy bins. The TGF source flux is normalized
by the number of injected photons. With $>$20 energy bins, BTO is able to
constrain the 511 keV peak produced by the TGF’s gamma-rays inducing
electron–positron annihilation in the Earth’s atmosphere (Kochkin et al.,
2018; Wada et al., 2019). The BTO spectrum provides good statistics up to
$\sim$1 MeV for this TGF event.

---
Figure 8:  A TGF spectrum as observed by BTO. The spectrum is calculated for a
1 ms TGF at an initial altitude of 10 km. The flux is normalized by the number
of photons injected into the simulation (1e7 photons).

Nominally, the main COSI detectors will not observe TGF events due to the
anti-coincidence requirement between the COSI detectors and active shields.
Since TGFs come from Earth (i.e. below the instrument), photons from these
events will always pass through the active BGO shields before reaching the
COSI detectors, and will therefore, be removed from the final, downlinked data
products. An exception may occur when COSI points toward the Crab Nebula for
calibrations, during which the telescope will be angled in such a way that a
TGF event could enter the main COSI FOV without passing through a shield
first. While the shields will save lightcurve information, the timing
resolution in the shields (50 ms) is not suitable for detecting TGF events.

##  4 Conclusions

In this paper, we discuss initial calculations for the expected science
returnables from the Background and Transient Observer (BTO) and the
compatibility of the BTO and Compton Spectrometer and Imager (COSI)
instruments. COSI is a wide-field gamma-ray telescope that will launch as a
SMEX in 2027 and provide imaging, spectroscopy, and polarization measurements
in the 0.2 to 5 MeV range. In addition to the main instrument, COSI will house
the BTO detector system which will provide spectral information in the 0.03
MeV to 2 MeV bandpass. With overlapping bandpasses and FOVs, BTO and COSI
enable spectral analysis across a broad gamma-ray bandpass. The combination of
spectral information from these two instruments will enable:

  * •

Localization of transient gamma-ray sources within the shared COSI and BTO
FOVs to $\sim$4$\degree$ (via COSI)

  * •

Measurements of the energy peak turnover and photospheric or non-thermal
emission components in GRB

  * •

Cataloging of magnetar events including their characteristic timescales,
temperatures, and brightness profiles

  * •

Identification of thunderstorms which produce TGF events

Simulations of lGRB, sGRB, magnetar GFs, and TGFs are performed to showcase
BTO’s response to these transient events. In the case of GRBs, the expected
detection rates are also calculated using the BATSE GRB catalogue. Using the
BTO geometry over a 2 year nominal mission, we expect to detect $\sim$295 GRB
events with $>$5$\sigma$ confidence, though this number might be slightly
overestimated (by $\sim$50 GRB/year) when comparing with the GBM detection
rate. A large fraction of these events ($\sim$99%) will be long GRB, revealing
that BTO is non-optimal for detecting sGRB events. Even with BTO’s low sGRB
detection rate, we expect to detect $\sim$7 sGRB with $>$3$\sigma$ confidence
which is a sufficient sample to measure time lags between sGRB events and
their corresponding gravitational wave signals.

For magnetar GFs and TGFs, we simulate spectra from BTO using models from the
literature. In both cases, BTO has good statistics up to $\sim$1 MeV where the
data becomes error dominated. The magnetar data are compared to an observed
GBM spectrum for an extragalactic GF in M82. Further analysis on the event
rates of magnetar GFs as well as magnetar SGRs and TGFs is in progress.

###### Acknowledgements.

The authors would like to acknowledge Alex Lowell (SSL), Brent Mochizuki
(SSL), Josh Forgione (SSL), and Greg Dalton (SSL) for their support and
guidance in developing the BTO instrument. COSI and BTO are a NASA Small
Explorer mission and Student collaboration project supported under NASA
contract 80GSFC21C0059. This material is based upon work supported by the
National Science Foundation Graduate Research Fellowship under Grant No. DGE
2146752.

## References

  * Abbott et al. (2017a) Abbott, B. P., Abbott, R., Abbott, T. D., et al. 2017a, The Astrophysical Journal Letters, 848, L13
  * Abbott et al. (2017b) Abbott, B. P., Abbott, R., Abbott, T. D., et al. 2017b, Physical Review Letters, 119, 161101
  * Abdo et al. (2010) Abdo, A. A., Ackermann, M., Agudo, I., et al. 2010, The Astrophysical Journal, 716, 30
  * Ajello et al. (2020) Ajello, M., Angioni, R., Axelsson, M., et al. 2020, The Astrophysical Journal, 892, 105
  * Atwood et al. (2009) Atwood, W. B., Abdo, A. A., Ackermann, M., et al. 2009, The Astrophysical Journal, 697, 1071
  * Band et al. (1993) Band, D., Matteson, J., Ford, L., et al. 1993, The Astrophysical Journal, 413, 281
  * Belz et al. (2017) Belz, J., Abbasi, R., Krehbiel, P. R., et al. 2017,  in AGU Fall Meeting Abstracts, Vol. 2017, AE31A
  * Boggs et al. (2007) Boggs, S. E., Zoglauer, A., Bellm, E., et al. 2007, The Astrophysical Journal, 661, 458
  * Bouchet et al. (2010) Bouchet, L., Roques, J. P., & Jourdain, E. 2010, The Astrophysical Journal, 720, 1772
  * Briggs et al. (2010) Briggs, M. S., Fishman, G. J., Connaughton, V., et al. 2010, Journal of Geophysical Research (Space Physics), 115, A07323
  * Campana et al. (2006) Campana, S., Mangano, V., Blustin, A. J., et al. 2006, Nature, 442, 1008
  * Chen et al. (2013) Chen, A. W., Argan, A., Bulgarelli, A., et al. 2013, Astronomy & Astrophysics, 558, A37
  * Cline et al. (1980) Cline, T. L., Desai, U. D., Pizzichini, G., et al. 1980, The Astrophysical Journal Letters, 237, L1
  * De Angelis et al. (2021) De Angelis, A., Tatischeff, V., Argan, A., et al. 2021, Experimental Astronomy, 51, 1225
  * Duncan & Thompson (1992) Duncan, R. C., & Thompson, C. 1992, The Astrophysical Journal Letters, 392, L9
  * Dwyer et al. (2012) Dwyer, J. R., Smith, D. M., & Cummer, S. A. 2012, Space Science Reviews, 173, 133
  * Eichler et al. (1989) Eichler, D., Livio, M., Piran, T., & Schramm, D. N. 1989, Nature, 340, 126
  * Fenimore et al. (1996) Fenimore, E. E., Klebesadel, R. W., & Laros, J. G. 1996, The Astrophysical Journal, 460, 964
  * Feroci et al. (2007) Feroci, M., Costa, E., Soffitta, P., et al. 2007, Nuclear Instruments and Methods in Physics Research A, 581, 728
  * Feroci et al. (1999) Feroci, M., Frontera, F., Costa, E., et al. 1999, The Astrophysical Journal Letters, 515, L9
  * Fishman et al. (1994) Fishman, G. J., Bhat, P. N., Mallozzi, R., et al. 1994, Science, 264, 1313
  * Frederiks et al. (2007) Frederiks, D. D., Golenetskii, S. V., Palshin, V. D., et al. 2007, Astronomy Letters, 33, 1
  * Goldstein et al. (2013) Goldstein, A., Preece, R. D., Mallozzi, R. S., et al. 2013, The Astrophysical Journal Supplement Series, 208, 21
  * Grefenstette et al. (2009) Grefenstette, B. W., Smith, D. M., Hazelton, B. J., & Lopez, L. I. 2009, Journal of Geophysical Research (Space Physics), 114, A02314
  * Guiriec et al. (2011) Guiriec, S., Connaughton, V., Briggs, M. S., et al. 2011, The Astrophysical Journal Letters, 727, L33
  * Gurevich et al. (1992) Gurevich, A. V., Milikh, G. M., & Roussel-Dupre, R. 1992, Physics Letters A, 165, 463
  * Hurley et al. (2005) Hurley, K., Boggs, S. E., Smith, D. M., et al. 2005, Nature, 434, 1098
  * Hurley et al. (1999) Hurley, K., Cline, T., Mazets, E., et al. 1999, Nature, 397, 41
  * Kaspi & Beloborodov (2017) Kaspi, V. M., & Beloborodov, A. M. 2017, Annual Review of Astronomy and Astrophysics, 55, 261
  * Knödlseder et al. (2005) Knödlseder, J., Jean, P., Lonjou, V., et al. 2005, Astronomy & Astrophysics, 441, 513
  * Kochkin et al. (2018) Kochkin, P., Sarria, D., Skeie, C., et al. 2018, Journal of Geophysical Research (Atmospheres), 123, 8074
  * Kumar & Zhang (2015) Kumar, P., & Zhang, B. 2015, Physics Reports, 561, 1
  * Lebrun et al. (2003) Lebrun, F., Leray, J. P., Lavocat, P., et al. 2003, Astronomy & Astrophysics, 411, L141
  * Lien et al. (2016) Lien, A., Sakamoto, T., Barthelmy, S. D., et al. 2016, The Astrophysical Journal, 829, 7
  * Lindanger et al. (2021) Lindanger, A., Marisaldi, M., Sarria, D., et al. 2021, Journal of Geophysical Research (Atmospheres), 126, e2021JD035347
  * Lund et al. (2003) Lund, N., Budtz-Jørgensen, C., Westergaard, N. J., et al. 2003, Astronomy & Astrophysics, 411, L231
  * MacFadyen & Woosley (1999) MacFadyen, A. I., & Woosley, S. E. 1999, The Astrophysical Journal, 524, 262
  * Marisaldi et al. (2015) Marisaldi, M., Argan, A., Ursi, A., et al. 2015, Geophysical Research Letters, 42, 9481
  * Marisaldi et al. (2010) Marisaldi, M., Fuschino, F., Labanti, C., et al. 2010, Journal of Geophysical Research (Space Physics), 115, A00E13
  * Mazets et al. (1999) Mazets, E. P., Cline, T. L., Aptekar’, R. L., et al. 1999, Astronomy Letters, 25, 635
  * Mazets et al. (1982) Mazets, E. P., Golenetskii, S. V., Gurian, I. A., & Ilinskii, V. N. 1982, Astrophysics and Space Science, 84, 173
  * Mazets et al. (1979) Mazets, E. P., Golentskii, S. V., Ilinskii, V. N., Aptekar, R. L., & Guryan, I. A. 1979, Nature, 282, 587
  * Meegan et al. (2009) Meegan, C., Lichti, G., Bhat, P. N., et al. 2009, The Astrophysical Journal, 702, 791
  * Mereghetti et al. (2024) Mereghetti, S., Rigoselli, M., Salvaterra, R., et al. 2024, Nature, 629, 58
  * Minaev et al. (2024) Minaev, P. Y., Pozanenko, A. S., Grebenev, S. A., et al. 2024, Astronomy Letters, 50, 1
  * Palmer et al. (2005) Palmer, D. M., Barthelmy, S., Gehrels, N., et al. 2005, Nature, 434, 1107
  * Pe’er et al. (2006) Pe’er, A., Mészáros, P., & Rees, M. J. 2006, The Astrophysical Journal, 642, 995
  * Ryde et al. (2010) Ryde, F., Axelsson, M., Zhang, B. B., et al. 2010, The Astrophysical Journal Letters, 709, L172
  * Smith et al. (2005) Smith, D. M., Lopez, L. I., Lin, R. P., & Barrington-Leigh, C. P. 2005, Science, 307, 1085
  * Smith (2019) Smith, J., 2019,  in 36th International Cosmic Ray Conference (ICRC2019), Vol. 36, International Cosmic Ray Conference, 604
  * Thompson & Duncan (1995) Thompson, C., & Duncan, R. C. 1995, Monthly Notices of the Royal Astronomical Society, 275, 255
  * Thompson & Duncan (1996) Thompson, C., & Duncan, R. C. 1996, The Astrophysical Journal, 473, 322
  * Tomsick et al. (2023) Tomsick, J. A., Boggs, S. E., Zoglauer, A., et al. 2023, arXiv e-prints, arXiv:2308.12362
  * Tsvetkova et al. (2022) Tsvetkova, A., Svinkin, D., Karpov, S., & Frederiks, D. 2022, Universe, 8, 373
  * Vedrenne et al. (2003) Vedrenne, G., Roques, J. P., Schönfelder, V., et al. 2003, Astronomy & Astrophysics, 411, L63
  * Wada et al. (2019) Wada, Y., Enoto, T., Nakamura, Y., et al. 2019, Communications Physics, 2, 67
  * Zoglauer et al. (2006) Zoglauer, A., Andritschke, R., & Schopper, F. 2006, New Astronomy Reviews, 50, 629