# CoCAtt: A Cognitive-Conditioned Driver Attention Dataset (Supplementary
Material)

Yuan Shen, Niviru Wijayaratne*, Pranav Sriram*,
Aamir Hasan, Peter Du, and Katherine Driggs-Campbell * denotes equal
contribution.Y. Shen is with the Department of Computer Science at the
University of Illinois at Urbana-Champaign. N. Wijayaratne is with the
Department of Mechanical Engineering at the University of Illinois at Urbana-
Champaign. K. Driggs-Campbell, P. Du, P. Sriram and A. Hasan are with the
Department of Electrical and Computer Engineering at the University of
Illinois at Urbana-Champaign. Emails: {yshen47, nnw2, psriram2, aamirh2,
peterdu2<EMAIL_ADDRESS>work was supported by State Farm and the
Illinois Center for Autonomy. This work utilizes resources supported by the
National Science Foundation’s Major Research Instrumentation program, grant
#1725729, as well as the University of Illinois at Urbana-Champaign [1].

##  I Dataset

CoCAtt is available for download at this link.

###  I-A Webcam Eye-tracking Data

Compared with gaze data collected by the GP3 eye-tracker, the webcam gaze
shows noisier behavior in two perspectives. For illustration purposes, we
randomly select three manual drive sessions from our dataset and visualize
their cumulative heatmap as shown in Figure 1. First, the noisiness can be
seen from the fact that the highest density region of the cumulative attention
does not always locate around the center of the driving scene, which we refer
to as the center shift issue. The center shift issue is unlikely to be the
actual driver attention property, since the road ahead is mostly forward, and
the eye fixations should be around the scene center. Second, even if the
webcam heatmap is approximately located in the center of the driving scene,
the gaze data is distributed more randomly than the eye-tracker gaze. To
address the noisy behavior of the webcam data, in Section 5 of our paper, we
propose a simple but effective baseline architecture that uses a coarse-to-
fine calibration procedure that handles the two perspectives of noisiness
above respectively.

Aside from the noisy pattern of the data distribution itself, the webcam-based
eye-tracking API that we use is susceptible to the presence of mask and skin
color due to its dependency on facial landmark detection output. In our
dataset, we mask those frames that the facial landmark detection fails due to
people wearing masks.

Figure 1: Cumulative heatmap for raw webcam and eye-tracker gaze captures. The
three heatmaps are generated by averaging three different 10-minute raw webcam
gaze captures in manual drive mode.

##  II Baseline Architectures

###  II-A Implementation Details for the Unconditioned Baseline and Multi-
branch

We visualize our architecture for the unconditioned and multi-branch baseline
models in Figure 3. For the unconditioned baseline, i.e., BDD-A, we only keep
one attention sub-branch, since there is no driver state as inputs. As for the
multi-branch model, the number of sub-branches is the number of driver states.
We select the corresponding sub-branch for each sequence based on the driver
state. Note that we divide each sequence during training so that it only
contains the same driver state for all frames.

###  II-B Implementation Details for the Modified CondConv Architecture

For the standard convolution, all the input examples share the same kernel
weights. However, for conditional convolution layer, i.e., CondConv [2], the
kernel weights is input dependent. Condconv achieves this by keeping a linear
combination of n kernel weights. We can interpret different kernel weights as
different experts where each of these experts may perform better for one
driver condition than the other. And then, a routing function generates
weights that decide how much each expert contributes to the final convolution.
Note that the original CondConv only takes feature maps from the previous
layer as inputs. To have a driver-state dependent routing function, we
modified on top of the CondConv by including 1D driver’s state as illustrated
in Figure 2.

We follow the suggestion of the original author of CondConv to inject CondConv
at multiple layers of our network and increase the dropout rate to reduce
over-fitting from 0.5 to 0.7. The detail architecture is shown in Figure 4.
For simplicity, we only visualize one modified CondConv in Figure 4 of the
main paper.

Figure 2: The architecture of the modified CondConv. In our implementation, we
use four expert weights, i.e., $N=4$, within the routing functions.

Figure 3: The architecture details of the BDD-A baseline and the multi-branch
model. num_state is 3 for the condition type of intersection intentions, and 2
for the condition type of distraction states. All of convolution layers are
followed by batchnorm2d and ReLU piece-wise nonlinearity.

Figure 4: The architecture details of the modified condconv. num_state is 3
for the condition type of intersection intentions, and 2 for the condition
type of distraction states. All of convolution layers are followed by
batchnorm2d and ReLU piece-wise nonlinearity.

Figure 5: The architecture details of the gaze calibration network. All of
convolution layers are followed by batchnorm2d and ReLU piece-wise
nonlinearity.

###  II-C Implementation Details for Gaze Calibration Network

We visualize our architecture for the gaze calibration baseline in Figure 5.
The architecture is mostly the same as the unconditioned attention baseline,
i.e., BDD-A. However, we concatenate the webcam gaze input with the encoded
spatial feature from the upstream network.

##  III Experiments

Table I: State-of-the-art model performance on our CoCAtt dataset. |  | Autopilot Test Data | Manual Test Data
---|---|---|---
Model | Training Data | Intersection | Lane-following | Intersection | Lane-following
|  | $D_{KL}\downarrow$ | $CC\uparrow$ | $D_{KL}\downarrow$ | $CC\uparrow$ | $D_{KL}\downarrow$ | $CC\uparrow$ | $D_{KL}\downarrow$ | $CC\uparrow$
BDD-A | autopilot | 2.07 | 0.36 | 1.46 | 0.47 | 1.78 | 0.43 | 1.47 | 0.50
manual | 2.22 | 0.33 | 1.71 | 0.44 | 1.60 | 0.48 | 1.35 | 0.53
DR(eye)VE | autopilot | 2.20 | 0.40 | 1.90 | 0.45 | 1.74 | 0.50 | 1.77 | 0.47
manual | 2.29 | 0.41 | 1.77 | 0.47 | 1.79 | 0.48 | 1.48 | 0.59

###  III-A Evaluation Metrics

Prediction Entropy The prediction entropy ($H$) can characterize the
confidence of the prediction for the cognitive-conditioned attention model.
Specifically, we calculate the Prediction Entropy ($H$) as follows:

$\begin{split}H(P)=-\sum_{i}P_{i}\log P_{i}\\\
\text{s.t.}\sum_{i}P_{i}=1\end{split}$

, where P is one predicted attention map and i is the pixel index.

###  III-B On the Effects of Driving Modes

The goal for this study is to report the performance of existing driver
attention model on our dataset and reveal the learning difference between
autopilot and manual drive attention data in CoCAtt dataset.

We train and evaluate two state-of-the-art driver attention models, BDD-
Attention and DR(eye)VE, on our dataset. The BDD-A model is re-implemented in
PyTorch, and we reuse the single-stream implementation of the DR(eye)VE model
with its pre-trained C3D encoder network as in [3]. Table I shows a comparison
of the BDD-A and DR(eye)VE models trained under both autopilot and manual
modes with no intention or distraction conditioning. Each model is evaluated
on both the autopilot and manual test sets.

As shown in Table I, both models exhibit worse performance if trained and
tested on the autopilot attention data, indicating that autopilot attention
has more variability and is thus harder to model if no planned action is known
beforehand. Additionally, since existing driver attention datasets contain
only manual drive data, we explore the transfer capability of models trained
in one setting and applied in another. We observe that models trained on
autopilot data can transfer better to the manual drive setting than manual-to-
autopilot. Our intuition is that drivers in autopilot mode are more likely to
attend to as many potential driving cues as possible to account for not
knowing the planned action of the ego car in advance.

Figure 6: Qualitative results for the gaze prediction with and without
intention conditioning. The last three columns are cumulative attention
predictions for the scene shown in the leftmost column. Each intention
(labeled on left) has two rows of examples. The white numbers represent the
entropy of predicted attention maps.

###  III-C Qualitative Study On the Effects of Driver States

In Section 6.2 of our paper, our quantitative results suggest that cognitive-
conditioned learning can improve driver attention prediction. The above
findings are validated by our qualitative results as well as shown in Figure
6. We use the multi-branch baseline for the qualitative study due to its
flexibility to add attention to sub-branchs for different driver states
without re-training the backbone network. Note that the same conclusion can be
reached from the modified CondConv baseline for the qualitative study. As
human attention is noisy for individual frames [4, 3], we choose to aggregate
attention maps over a long time horizon (e.g., intervals when approaching
intersections), to compare the ground-truth gazes and both unconditioned and
conditioned gaze predictions. Each cumulative attention heatmap is generated
by aggregating gaze maps when the driver is approaching an intersection. In
Figure 6, we visualize six cumulative attention heatmaps for the intention-
conditioned case at a four-way intersection scenario with three different
intentions. Compared to the unconditioned case, the intention-conditioned
model generates more focused attention with fewer weights allocated in regions
unrelated to the current driver intention.

##  IV Application

Aside from performance improvement, our cognitive-conditioned attention model
can solve new real-world challenges. To demonstrate its practicality, we
showcase one application of our cognitive-conditioned attention model on road
safety analysis. This task aims to assess the safety standard of roads to
reduce the chance of accidents [5, 6]. Traditional road safety analysis uses
survey-based methods or historical data on a case-by-case basis [7, 5, 8, 9].
However, our cognitive-conditioned attention model offers an alternative,
data-driven solution that can be generalized to address diverse road patterns.
Our approach is built on the findings of previous cognitive distraction
studies related to the positive correlation between cognitively distracted
driving and risk of car accidents [10, 11]. To estimate the risk of car
accidents per road segment, we measure the gaze behavior difference between
distracted and attentive drivers. The more significant the difference, the
more likely the distracted driver will miss important driving cues. We
quantify gaze behavior difference with the Earth Mover’s Distance, which has
the advantage over other metrics of measuring the spatial distance cost to
migrate from distracted to attentive attention maps [12].

Figure 7: Visualization for one use case of our cognitive-conditioned
attention model on road safety analysis. The color indicates the distraction
risk, which we estimate by computing the difference in gaze behavior between
distracted and attentive drivers.

We re-trained our distraction-state-conditioned model over the autopilot data.
We then predicted both attentive and distracted driver attention maps for each
timestamp by feeding attentive and distracted cognitive states as two separate
inputs into our network. Figure 7 shows the visualization of our result after
applying a median filter over the local neighborhood. Our approach offers
location-wise distraction risk measurements. For example, our method estimates
high risk at the round-about (point A in Figure 7). This insight may suggest
that traffic warning signs here would be useful to notify drivers to look for
cars as they enter the round-about. Another finding is that road segments
approaching and exiting traffic intersections are more accident-prone (point B
and C). While most of the highlighted regions are near intersection areas, our
proposed approach can also detect risky areas of other road conditions. For
example, the road region in point D, which has a steep slope increase, has a
higher value than nearby regions. This proposed analysis is a new method for
determining driver risk across roadways and driving scenarios. Such data-
driven insights exhibit the practicality and applicability of our cognitive-
conditioned attention model for road safety analysis and other potential real-
world applications.

## References

  * [1] V. Kindratenko, D. Mu, Y. Zhan, J. Maloney, S. H. Hashemi, B. Rabe, K. Xu, R. Campbell, J. Peng, and W. Gropp, “Hal: Computer system for scalable deep learning,” in _Practice and Experience in Advanced Research Computing_ , ser. PEARC ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 41–48.
  * [2] B. Yang, G. Bender, Q. V. Le, and J. Ngiam, “Condconv: Conditionally parameterized convolutions for efficient inference,” in _Advances in Neural Information Processing Systems_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Eds., vol. 32. Curran Associates, Inc., 2019.
  * [3] A. Pal, S. Mondal, and H. I. Christensen, ““Looking at the Right Stuff” \- Guided Semantic-Gaze for Autonomous Driving,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 11 883–11 892.
  * [4] A. Palazzi, D. Abati, s. Calderara, F. Solera, and R. Cucchiara, “Predicting the driver’s focus of attention: The dr(eye)ve project,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 41, no. 7, pp. 1720–1733, 2019.
  * [5] S. Kanuganti, R. Agarwala, B. Dutta, P. N. Bhanegaonkar, A. P. Singh, and A. Sarkar, “Road safety analysis using multi criteria approach: A case study in India,” _Transportation Research Procedia_ , vol. 25, pp. 4649–4661, 2017, world Conference on Transport Research - WCTR 2016 Shanghai. 10-15 July 2016. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2352146517306063
  * [6] F. Capaldo and G. Nasti, “Analysis of road safety: Three levels of investigation,” in _Proceedings of International Conference on Traffic and Transport Engineering ICTTE_ , 11 2012.
  * [7] S. Turner, J. Khoo, and H. Trumper, “How unsafe does the road look - measuring perceived risk,” in _NZ Transport Group Conference_ , 03 2014.
  * [8] Machsus, I. Prayogo, Chomaedhi, D. W. Hayati, and A. Utanaka, “Road safety analysis on Achmad Yani frontage road Surabaya,” _IOP Conference Series: Materials Science and Engineering_ , vol. 267, p. 012027, nov 2017.
  * [9] M. Montoya-Alcaraz, A. Mungaray-Moctezuma, J. Calderón-Ramírez, L. García, and C. Martinez-Lazcano, “Road Safety Analysis of High-Risk Roads: Case Study in Baja California, México,” _Safety_ , vol. 6, no. 4, 2020. [Online]. Available: https://www.mdpi.com/2313-576X/6/4/45
  * [10] C. for Disease Control and Prevention, “Distracted Driving,” Mar 2021. [Online]. Available: https://www.cdc.gov/transportationsafety/distracted_driving/index.html
  * [11] S. Yang, J. Kuo, and M. G. Lenné, “Analysis of Gaze Behavior to Measure Cognitive Distraction in Real-World Driving,” _Proceedings of the Human Factors and Ergonomics Society Annual Meeting_ , vol. 62, no. 1, pp. 1944–1948, 2018.
  * [12] Z. Bylinskii, T. Judd, A. Oliva, A. Torralba, and F. Durand, “What Do Different Evaluation Metrics Tell Us About Saliency Models?” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 41, no. 3, pp. 740–757, 2019.


# The Internet of Responsibilities - Connecting Human Responsibilities using
Big Data and Blockchain

Xuejiao Tang1 Jiong Qiu2(✉)  Wenbin Zhang3 Ibrahim Toure4 Mingli Zhang5
Enza Messina6 Xueping Xie7 Xuebing Wang2 and Sheng Yu2
1Leibniz University Hannover  Germany 2Hangzhou Quanshi Software Co  Ltd
China
3University of Maryland  Baltimore County  USA 4Doninya Inc  USA 5McGill
University  Canada
6University of Milano-Bicocca  Italy 7Hangzhou Dianzi University  China
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>{wangxuebing1109<EMAIL_ADDRESS>
###### Abstract

Accountability in the workplace is critically important and remains a
challenging problem, especially with respect to workplace safety management.
In this paper, we introduce a novel notion, the Internet of Responsibilities,
for accountability management. Our method sorts through the list of
responsibilities with respect to hazardous positions. The positions are
interconnected using directed acyclic graphs (DAGs) indicating the hierarchy
of responsibilities in the organization. In addition, the system detects and
collects responsibilities, and represents risk areas in terms of the positions
of the responsibility nodes. Finally, an automatic reminder and assignment
system is used to enforce a strict responsibility control without human
intervention. Using blockchain technology, we further extend our system with
the capability to store, recover and encrypt responsibility data. We show that
through the application of the Internet of Responsibility network model driven
by Big Data, enterprise and government agencies can attain a highly secured
and safe workplace. Therefore, our model offers a combination of
interconnected responsibilities, accountability, monitoring, and safety which
is crucial for the protection of employees and the success of organizations.

##  I Introduction

Together with delegation and representation, accountability is one of the
cornerstones of democracy. Delegation involves endowing another party with the
discretion to act, representation is about the interests that are at stake,
and accountability ensures that the exercise of discretion is checked. In the
last few years, various patents pertaining to responsibility management have
been filed and can be classified into the following schemes: responsibility
data production (collection), responsibility process design, responsibility
data exchange, responsibility data evaluation, responsibility clarity [1].
However, patents are limited in that they cannot describe the systematic and
complete literature that encompasses the close-loop management of
responsibility. Among them, Hollender et al. expanded the risk alarm
management to reflect the responsibility of each link of the overall process
[3]. In addition, Brandsma et al. proposed accountability cube to quantify
accountability [1]. Nonetheless, despite these efforts, a complete
responsibility assignment management system is still lacking. On the other
hand, data-driven AI is increasingly being used to improve every aspect of our
lives [20, 24, 22]. In this paper, a big data driven accountability management
model, the Internet of Responsibilities model (IoR), is therefore proposed to
improve the quality and safeguard of enterprises and institutions.
Accountability in a production environment is implemented through the IoR
model which will promote the staffs’ awareness of responsibility. Our model
has been deployed by the Chinese governments and companies to establish the
accountability mechanism. The results of the initial evaluation showed an
obvious increase in responsibility scores and safety awareness.

##  II System Design

###  II-A The Internet of Responsibilities System

As presented in Figure 1, IoR model consists of four layers: perception layer,
network layer, data layer and application layer. These layers are detailed
thereafter.

Figure 1: The Internet of Responsibility model.

  1. 1.

Perception Layer. This layer collects the responsibility data. Starting from
the bottom, different types of responsibility data are collected from a list
of hazardous locations. Our model takes into account hazardous locations and
the hierarchy of the organization. The data collected is transformed into a
unified data model through the perception module, using sophisticated
algorithms, and the product module, based on hardware design. In addition, the
reliability of hardware system and relative information on the perception
layer is ensured by the responsibility chip, designed in a blockchain mode to
record authentication information like device manufacturer authentication
information, device process data, etc.

  2. 2.

Internet layer. The network layer uploads the responsibility data collected at
the perception layer to the Internet via Fiber, 4G, 5G, WiFi, NB-IoT, or LoRa.

  3. 3.

Data layer. This layer adopts semi-structured governance of the responsibility
data, i.e., data integration, data analysis for all of the hazardous
locations, data processing, accountability and evaluation (score), and
encryption of responsibility data. By using directed acyclic graph (DAG),
responsibility nodes with responsibility data uploaded to the Internet layer
from the perception layer, are sorted according to the hierarchy of the
organization and the risk level, which results in a responsibility tree for
the company. For each hazardous position, all of the relevant information will
be stored in the responsibility nodes, which is implemented on the blockchain.

  4. 4.

Application layer. The application layer allows different types of companies
such as financial institutions, government agencies; to monitor, automatically
detect abnormal events, and send alerts to the responsible parties.
Furthermore, this layer is used to manage and reorganize the responsibility
list of hazardous locations.

###  II-B Details of IoR model

This section outlines the key technologies of IoR model which enable the
implementation of various layers.

####  II-B1 Responsibility module of Roles

Multiple multivariate groups are defined to describe the responsibility lists
of the positions:

  * -

Definition 1: Role (roleid, rolename, $<$RList$>$, $<$PersonList$>$,
$<$ObjectList$>$, $<$AlarmList$>$, remarks). Role refers to the position of
each employee in the company. A position contains either one or more roles.

  * -

Definition 2: RList (rid, Rtype, Rname, $<$RdataList$>$, Rscore, Rrequirement,
Rremarks). RList refers to responsibility list which consists of id, type,
responsibility data, scoring items, basis (laws, regulations, rules) and
remarks.

  * -

Definition 3: PersonList (personid, personname, $<$DatetimeList$>$,
$<$SpaceList$>$, $<$RoleidList$>$, remarks). PersonList contains the staffs’
information with a given role, DatetimeList defines the start and end time,
SpaceList means the location distributions of each person. One person can have
multiple roles.

  * -

Definition 4: ObjectList (oid, objectname, $<$QuantityList$>$, remarks).
ObjectList contains information about goals.

  * -

Definition 5: AlarmList (aid, aname, alevel, asource, $<$AlarmRNodeList$>$,
remarks). AlarmList contains items from hazardous positions, indicating the
id, name, risk level (A, B, C class are defined to indicate risk level
decreasingly), risk source and defines AlarmRNodeList that can automatically
generate responsibility lists over time.

####  II-B2 Hierarchy of Responsibilities in Organization

To indicate the hierarchy of responsibilities in the organization among
responsibility lists, positions are interconnected using DAGs as shown in
Figure 2.

  * -

S11: Obtain, modify or add responsibility lists of relevance roles.

  * -

S12: Topologically sort the responsibility lists (including responsibility
nodes) by the directed acyclic graph: we use responsibility nodes as nodes and
the pairwise relationship as edges.

  * -

S13: Generate responsibility management mode for each position from Step S12.

Figure 2: Directed acyclic graph for responsibility lists.

####  II-B3 Automatic evaluation and management

The responsibility score is calculated by automatic evaluation of completion
score and execution score.

The completion score with a total of 1000 points is divided into two parts:
daily work score and additional work score. The default proportion is set to
750 points for daily work score and 250 points for additional work score.
These scores reflect the daily completion status of relative responsibilities.
The proportion between the daily work score and additional work score can be
reset. The scores are calculated based on the timely submission of required
materials by staff members proving the completion of the responsibilities
assigned. The system computes the execution scores of each staff member with
responsibilities using the results from materials submitted and co-workers’
evaluations.

####  II-B4 Responsibility Perception Design

Responsibility perception is designed as follows:

  * -

Step 1: Sort responsibility lists and information to responsibility sets,
which contain responsibility nodes. Each responsibility set includes one or
multiple starting nodes, process nodes, and terminal nodes.

  * -

Step 2: Correlate responsibility sets with product default status, which
results in a status table of responsibilities.

  * -

Step 3: Add the methodology describing how the status table of
responsibilities generates responsibility data (from step 2) into production.

  * -

Step 4: When production status changes, responsibility data gets generated.

  * -

Step 5: Responsibility score based on responsibility data are computed while
generating relative data.

####  II-B5 Responsibility Sealing and Accountability

Using blockchain technology, the responsibility data is sealed and cannot be
changed; therefore, the authenticity and integrity of the data are guaranteed.
In addition, accident accountability can be traced back in the weighted DAG.

##  III IoR model in practice

###  III-A Applied in Enterprises and Institutions

Figure 3 shows the deployment of the proposed IoR model in Hangzhou Quanshi
Software Co., Ltd. The corresponding tree of responsibility lists is
automatically generated through a mobile application. Also, staff can submit
relevant data through the app and the system can thereafter perform evaluation
and scoring automatically. The system computes the personal responsibility
score (rank) in the department or in the organization; as well as the personal
score (status) based on the due diligence applied towards accomplishing the
responsibility assigned. Figure 4 reflects the responsibility scores. In the
case of low scores, the system notifies/warns employees to complete their
responsibilities.

Figure 3: Generated tree by responsibility lists. Figure 4: Responsibility
score.

###  III-B Applied in Government Accountability

The IoR model has been applied in government accountability with respect to
the following aspects:

####  III-B1 Safety Evaluation

With the sealed IoR data model, safety evaluation scores are displayed on the
system.

####  III-B2 Accountability Mechanism

  * -

Record accountability data in unsafe situations.

  * -

Design the corresponding responsibility reminder alarm system, based on
responsibility score.

####  III-B3 Responsibility Ranking

Safety level is ranked according to the responsibility score among units.

####  III-B4 Supervised Risk Level

Risk level in various units is summarized by the system so that units with
decreasing responsibility scores can be immediately reminded. Risk management
maturity reflects the risk management capability of an enterprise from various
aspects [Sun_2019]. Figure 5 shows risk level (decreasingly ranked by colors
red, yellow, green, blue), allowing governments to have an oversight of their
agencies with near real-time supervision and monitoring capabilities.

Figure 5: Responsibility status map

##  IV Conclusion

To address the challenging issue of allocating responsibilities in a
production environment, this work proposed the notion of the Internet of
Responsibilities and implemented an IoR system driven by big data and
blockchain technology. The practical deployments showed an obvious increase in
the responsibility scores and safety awareness in both personnel and
organizations.

## References

  * [1] G. J. Brandsma and T. Schillemans.  The Accountability Cube: Measuring Accountability.  Journal of Public Administration Research and Theory, 23(4):953–975, 09 2012.
  * [2] T. Guyet, W. Zhang, and A. Bifet.  Incremental mining of frequent serial episodes considering multiple occurrences.  In 22nd International Conference on Computational Science, pages 460–472. Springer, 2022.
  * [3] M. Hollender.  Method and device for optimizing the alarm configuration, Aug. 21 2012\.  US Patent 8,248,228.
  * [4] Z. Liu, R. Wang, N. Japkowicz, D. Tang, W. Zhang, and J. Zhao.  Research on unsupervised feature learning for android malware detection based on restricted boltzmann machines.  Future Generation Computer Systems, 120:91–108, 2021.
  * [5] S. K. Sood and I. Mahajan.  Wearable iot sensor based healthcare system for identifying and controlling chikungunya virus.  Computers in Industry, 91:33–44, 2017.
  * [6] X. Tang et al.  A data-driven human responsibility management system.  In IEEE International Conference on Big Data (Big Data), 2020.
  * [7] X. Tang, X. Huang, et al.  Cognitive visual commonsense reasoning using dynamic working memory.  In International Conference on Big Data Analytics and Knowledge Discovery. Springer, 2021.
  * [8] X. Tang, L. Zhang, et al.  Using machine learning to automate mammogram images analysis.  In IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 757–764, 2020.
  * [9] X. Tang, W. Zhang, Y. Yu, K. Turner, T. Derr, M. Wang, and E. Ntoutsi.  Interpretable visual understanding with cognitive attention network.  In International Conference on Artificial Neural Networks, pages 555–568. Springer, 2021.
  * [10] J. Wang, Z. Huang, et al.  Wearable sensor based human posture recognition.  In IEEE International Conference on Big Data (Big Data), 2016.
  * [11] Z. Wang, G. Narasimhan, X. Yao, and W. Zhang.  Fair decision-making under uncertainty.  In 2023 IEEE International Conference on Data Mining (ICDM). IEEE, 2023.
  * [12] L. Zhang, W. Zhang, and N. Japkowicz.  Conditional-unet: A condition-aware deep model for coherent human activity recognition from wearables.  In 2020 25th International Conference on Pattern Recognition (ICPR), pages 5889–5896. IEEE, 2021.
  * [13] M. Zhang, F. Zhang, J. Zhang, A. Chaddad, F. Guo, W. Zhang, J. Zhang, and A. Evans.  Autoencoder for neuroimage.  In International conference on database and expert systems applications, pages 84–90. Springer, 2021.
  * [14] M. Zhang, X. Zhao, et al.  Deep discriminative learning for autism spectrum disorder classification.  In International Conference on Database and Expert Systems Applications, pages 435–443. Springer, 2020.
  * [15] W. Zhang.  Phd forum: Recognizing human posture from time-changing wearable sensor data streams.  In IEEE International Conference on Smart Computing (SMARTCOMP), 2017.
  * [16] W. Zhang.  Learning fairness and graph deep generation in dynamic environments.  2020\.
  * [17] W. Zhang and A. Bifet.  Feat: A fairness-enhancing and concept-adapting decision tree classifier.  In International Conference on Discovery Science, pages 175–189. Springer, 2020.
  * [18] W. Zhang, A. Bifet, X. Zhang, J. C. Weiss, and W. Nejdl.  Farf: A fair and adaptive random forests classifier.  In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pages 245–256. Springer, 2021.
  * [19] W. Zhang, T. Hernandez-Boussard, and J. Weiss.  Censored fairness through awareness.  In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 14611–14619, 2023.
  * [20] W. Zhang and E. Ntoutsi.  Faht: an adaptive fairness-aware decision tree classifier.  In International Joint Conference on Artificial Intelligence (IJCAI), pages 1480–1486, 2019.
  * [21] W. Zhang, S. Pan, S. Zhou, T. Walsh, and J. C. Weiss.  Fairness amidst non-iid graph data: Current achievements and future directions.  arXiv preprint arXiv:2202.07170, 2022.
  * [22] W. Zhang, J. Tang, and N. Wang.  Using the machine learning approach to predict patient survival from high-dimensional survival data.  In IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2016.
  * [23] W. Zhang, X. Tang, and J. Wang.  On fairness-aware learning for non-discriminative decision-making.  In International Conference on Data Mining Workshops (ICDMW), pages 1072–1079, 2019.
  * [24] W. Zhang and J. Wang.  A hybrid learning framework for imbalanced stream classification.  In IEEE International Congress on Big Data (BigData Congress), pages 480–487, 2017.
  * [25] W. Zhang and J. Wang.  Content-bootstrapped collaborative filtering for medical article recommendations.  In IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2018.
  * [26] W. Zhang, J. Wang, D. Jin, L. Oreopoulos, and Z. Zhang.  A deterministic self-organizing map approach and its application on satellite data based cloud type classification.  In IEEE International Conference on Big Data (Big Data), 2018.
  * [27] W. Zhang and J. Weiss.  Fair decision-making under uncertainty.  In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 2021.
  * [28] W. Zhang and J. C. Weiss.  Longitudinal fairness with censorship.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12235–12243, 2022.
  * [29] W. Zhang and J. C. Weiss.  Fairness with censorship and group constraints.  Knowledge and Information Systems, pages 1–24, 2023.
  * [30] W. Zhang, L. Zhang, D. Pfoser, and L. Zhao.  Disentangled dynamic graph deep generation.  In Proceedings of the SIAM International Conference on Data Mining (SDM), pages 738–746, 2021.
  * [31] W. Zhang and L. Zhao.  Online decision trees with fairness.  arXiv preprint arXiv:2010.08146, 2020.


#  A Search for Planet 9 with a Sub-Relativistic Spacecraft: three-body, post-
Newtonian and non-gravitational effects

Sahin Ulas Koprucu<EMAIL_ADDRESS>TUBITAK UZAY - Space
Technologies Research Institute, Ankara, Turkey  Bayram Tekin
<EMAIL_ADDRESS>Department of Physics, Middle East Technical University,
Ankara, Turkey

###### Abstract

A hypothetical gravitating body in the outer solar system, the so-called
Planet 9, was proposed to explain the unexpected clustering of the Kuiper Belt
Objects. As it has not been observed via telescopes, it was conjectured to be
a primordial black hole (of the size of a quince) that could be
gravitationally detected by laser-launching many 1-gram mass satellites. Here,
we study various aspects that will affect such a search for Planet 9. Our
basic observable is the angular displacement in the trajectory of a sub-
relativistic spacecraft which will be mainly affected by the gravity of Planet
9, augmented with several other 3-body, non-gravitational, and post-Newtonian
forces. First, we calculate the effect of the Sun in the framework of the
circular restricted three-body problem of the Sun-Planet 9 spacecraft for the
two particular initial conditions. Then, we study the effects of non-
gravitational perturbations such as magnetic and drag forces exerted by the
interstellar medium; and the solar radiation pressure. In addition, we
investigate the post-Newtonian general relativistic effects such as the frame-
dragging, Schwarzschild effect, and geodetic precession on the spacecraft
trajectory. We show that the leading order angular displacement is due to the
solar radiation pressure which makes the detection of Planet 9 difficult.
Among the general relativistic effects, the frame-dragging has the smallest
effect; and the Schwarzschild effect due to Sun has the largest effect.
However, none of the general relativistic effects produces a meaningful
contribution to the detection.

##  I Introduction

The possible existence of a body at a distance 500 AU, with a mass around 5-10
$M_{\oplus}$, commonly known as Planet 9, was proposed to explain the
unexpected clustering of the Kuiper Belt Objects [1],[2]. So far, detection
has not yet been achieved, telescope searches are still ongoing and there is a
lot of room in this volume of space for such a planet to hide [3]. But in the
absence of visual observation, an exciting conjecture was put forward: this
object could be a primordial black hole [4]. In that case, its size would be
around the size of a medium-sized quince and alternative ways for detection
have been suggested such as sending a cluster of small sub-relativistic
spacecraft and measuring the deflection of their trajectories due to the
gravity of Planet 9 [5]; or aiming to measure the Hawking radiation spreading
from the primordial black hole with a sub-relativistic spacecraft, although
capturing the weak signal would be challenging [6]. In addition, the Zwicky
Transient Facility public archive shows no candidate [7]. More recently, an
indirect way was suggested in [8] assuming that Planet 9 has satellites, the
tidal force of Planet 9 can heat up the putative satellites causing thermal
radio flux for observations. In another recent work, a candidate location of
Planet 9 was proposed by tracing the trajectory of an interstellar meteoroid
backward and looking at its coincidence with the maximum probability region of
Planet 9 [9]. Lastly, the localization of Planet 9 was suggested in [10] by
analyzing the trajectory of a spacecraft that will be sent for a possible
Uranus mission.

In this work, we focus on the method proposed in [5] based on probing the
gravitational field of Planet 9 with the trajectory of a spacecraft attaining
sub-relativistic speed launched with the use of advanced propulsion
technologies such as solar sail [11] or laser propulsion [12]. In [5], the
deflection was studied in terms of time delay in the signal sent from the
spacecraft to the Earth with sufficiently accurate timekeeping requirements.
To avoid this requirement, measurement of the transverse displacement was
proposed by using Very Long Baseline Interferometry (VLBI) [13]. In addition,
the effects of the drag and electromagnetic forces exerted by the interstellar
medium on the spacecraft’s trajectory were also discussed in [14], including
the parameter space of the spacecraft to distinguish the gravity of Planet 9
from other perturbations.

In [5], the deflection was studied in the context of Planet 9 and the
spacecraft as a two-body problem. We extended that discussion by including the
Newtonian gravity of the Sun; and studied the deflection in the circular
restricted three-body problem (CRTBP) of the Sun-Planet 9-spacecraft with two
particular initial conditions under some assumptions. Then, the effects of
non-gravitational perturbations such as drag force, magnetic force, and solar
radiation pressure are investigated to assess whether the effect of Planet 9
can be distinguishable from the other perturbations. In fact, the effects of
drag and magnetic forces were already studied in [14] in terms of magnitude
comparison. Here, we perform the numerical integration by considering the
directions as well, and the nominal trajectory of the spacecraft is defined
with the Sun and the spacecraft’s two-body problem. Lastly, we discuss the
general relativistic effects like the frame-dragging effect, the Schwarzschild
effect, and the geodetic precession effect on the spacecraft trajectory and
give the magnitude comparison with the Newtonian gravity of Planet 9. It is
important to note that the spacecraft initially starts in the interplanetary
region, so it is under the influence of planetary perturbations but we shall
ignore them to avoid the use of ephemeris and perform the time-independent
analysis. Also, a detailed trajectory design is not the focus of this paper.

The layout of this paper is as follows: Section II is a recap of [5] where we
give more details on Planet 9 and the satellite two-body problem as a
preparation for Section III where we study the effects of the Sun within the
context of CRTBP. In section IV we study the drag force on the spacecraft
trajectory; in section V we study the magnetic force; in section VI we study
the solar radiation pressure while section VII is devoted to the post-
Newtonian general relativistic effects. As the issue here is to detect a
quince-sized object in such a large volume of space, we wanted to make sure
that we take all the known effects into consideration and compute their
contributions to the trajectory of the tiny spacecraft. Qualitatively, one
knows before the computations that these effects will be small, but our aim
here is to carefully quantify the effects.

##  II Planet 9 and Spacecraft Two-Body Problem

As a way to prepare for the three-body problem of the next section, and define
the parameters involved in the problem, let us first revisit the results given
in [5] where the deflection in the trajectory of a sub-relativistic spacecraft
in the context of Planet 9 and the spacecraft two-body problem was studied. To
understand how strong the gravitational effect of Planet 9 is on the
spacecraft, kinetic energy per unit mass, $v^{2}/2$, and gravitational
potential energy of Planet 9, $-\mu_{P9}/\rho$, can be compared in the
vicinity of Planet 9. The current estimates on the mass and the orbit of
Planet 9 given in [15] suggest that the gravitational parameter is
$\mu_{P9}\approx 6.2\mu_{E}$ where $\mu_{E}=398600$ km3$/$s2 and semi-major
axis is $a_{P9}\approx 380$ AU. Then, the smallest impact parameter for the
half sky search with 1000 spacecraft is $\rho\approx 30$ AU from the method
given in [5]. Let the sub-relativistic speed of the spacecraft be $v=0.001c$
where $c=2.99792458\times 10^{5}$ km/s is the speed of light. This sub-
relativistic speed is chosen to make the mission duration reasonable. If the
relevant parameters are put into the equations of the kinetic and potential
energies, one can obtain that kinetic energy is eight orders of magnitude
larger than the gravitational energy. So, the gravity of Planet 9 can be
considered as a perturbative effect on a straight-line trajectory of a sub-
relativistic spacecraft. The reference frame and configuration of the two-body
problem are given in Fig. 1. The straight black line is the unperturbed
trajectory in the absence of Planet 9, and the blue trajectory is the
perturbed one. Finally, $\rho$ is the minimum distance to Planet 9 along the
unperturbed trajectory.

Figure 1: The black line is the unperturbed trajectory and the blue curve is
the perturbed trajectory due to Planet 9; also $\rho$ is the impact factor
i.e. the minimum distance to the Planet 9 along the unperturbed trajectory.

The position vector of the spacecraft in the vicinity of Planet 9 would take
the form;

$\vec{r}(t)=v_{0}t\hat{i}+\rho\hat{j}+\vec{r}_{1}(t),$ (1)

where $\vec{r}_{1}(t)=x_{1}\hat{i}+y_{1}\hat{j}$ is the deviation from the
straight line trajectory. For the unperturbed trajectory, the position at
$t=0$ corresponds to the minimum distance between Planet 9 and the spacecraft.
Then, the velocity and acceleration vectors become;

$\displaystyle\dot{\vec{r}}(t)$
$\displaystyle=\left(v_{0}+\dot{x}_{1}\right)\hat{i}+\dot{y}_{1}\hat{j},$ (2a)
$\displaystyle\ddot{\vec{r}}(t)$
$\displaystyle=\ddot{x}_{1}\hat{i}+\ddot{y}_{1}\hat{j}.$ (2b)

The relative position vector of the spacecraft with respect to Planet 9
satisfies;

$\ddot{\vec{r}}=-\frac{\mu_{P9}}{r^{3}}\vec{r},$ (3)

where $r=\left[(v_{0}t+x_{1})^{2}+(\rho+y_{1})^{2}\right]^{1/2}$. Then, the
components of the equations of motion are;

$\displaystyle\ddot{x}_{1}$
$\displaystyle=-\frac{\mu_{P9}}{\left(v_{0}^{2}t^{2}+2v_{0}tx_{1}+x_{1}^{2}+\rho^{2}+2\rho
y_{1}+y_{1}^{2}\right)^{3/2}}\left(v_{0}t+x_{1}\right),$ (4a)
$\displaystyle\ddot{y}_{1}$
$\displaystyle=-\frac{\mu_{P9}}{\left(v_{0}^{2}t^{2}+2v_{0}tx_{1}+x_{1}^{2}+\rho^{2}+2\rho
y_{1}+y_{1}^{2}\right)^{3/2}}\left(\rho+y_{1}\right).$ (4b)

Since Planet 9 has a perturbative effect, the deviations would be small so
that;

$\displaystyle\frac{x_{1}}{v_{0}t}\ll 1,\quad\frac{y_{1}}{\rho}\ll 1.$ (5)

Then, the equations of motion take the following form;

$\displaystyle\ddot{x}_{1}=-\frac{\mu_{P9}}{\left(v_{0}^{2}t^{2}+\rho^{2}\right)^{3/2}}v_{0}t,\quad\ddot{y}_{1}=-\frac{\mu_{P9}}{\left(v_{0}^{2}t^{2}+\rho^{2}\right)^{3/2}}\rho.$
(6a)

which yield the perturbative velocities:

$\displaystyle\dot{x}_{1}=\frac{\mu_{P9}}{v_{0}\sqrt{v_{0}^{2}t^{2}+\rho^{2}}}+c_{x},\quad\dot{y}_{1}=-\frac{\mu_{P9}}{\rho\sqrt{v_{0}^{2}t^{2}+\rho^{2}}}t+c_{y},$
(7a)

where $c_{x}$ and $c_{y}$ are the integration constants to be found from the
boundary conditions. As $t\rightarrow-\infty$, there should not be any
gravitational effect of Planet 9 on the spacecraft, so the velocity should be;

$\lim_{t\to-\infty}\dot{\vec{r}}(t)=v_{0}\hat{i}.$ (8)

Thus, the proper boundary condition would be;

$\lim_{t\to-\infty}\dot{\vec{r}}_{1}(t)=0.$ (9)

After applying the boundary conditions, the final forms of the perturbative
velocities become;

$\displaystyle\dot{x}_{1}=\frac{\mu_{P9}}{v_{0}\sqrt{v_{0}^{2}t^{2}+\rho^{2}}},\quad\dot{y}_{1}=-\frac{\mu_{P9}}{\rho\sqrt{v_{0}^{2}t^{2}+\rho^{2}}}t-\frac{\mu_{P9}}{\rho
v_{0}}.$ (10a)

Finally, the magnitude of the transverse velocity change can be found as;

$\Delta
v_{y}=\bigg{|}\lim_{t\to\infty}\dot{y}_{1}-\lim_{t\to-\infty}\dot{y}_{1}\bigg{|}=\bigg{|}\lim_{t\to\infty}\dot{y}_{1}\bigg{|}=\frac{2\mu_{P9}}{\rho
v_{0}}.$ (11)

Here, we focus on the measurement of transverse displacement, instead of time
delay in the signal to relax the accurate timekeeping requirement. So, the
maximum angular displacement is given as [13];

$\alpha=\frac{\Delta v_{y}}{v_{x}^{(0)}}.$ (12)

Using (11) in (12), one has

$\alpha=\frac{2\mu_{P9}}{\rho v_{0}^{2}}\approx 1\times
10^{-8}\,\text{radians}.$ (13)

In [13], it was noted that that the angular displacement could be measured
precisely with the use of VLBI with a detection threshold of order $10^{-9}$
radians for high-frequency sources. (13) is already above the threshold. To
further improve the detectability, the impact parameter may be decreased by
increasing the number of spacecraft or spacecraft may be slowed down.

##  III The Sun-Planet 9-Spacecraft system: a three-body problem

In this section, the angular displacement is studied in the context of the
Sun-Planet 9-spacecraft three-body problem to understand whether the three-
body context provides new prospects for the detection. To have an idea of the
effect of the Sun, the ratio of gravitational acceleration due to Planet 9 and
the Sun is calculated by taking the gravitational parameter of the Sun as
$\mu_{S}=132712\times 10^{6}$ km3$/$s2;

$\left(\frac{\mu_{S}}{a_{P9}^{2}}\right)/\left(\frac{\mu_{P9}}{\rho^{2}}\right)\approx
335,$ (14)

moreover, the tidal acceleration due to the Sun and Planet 9 on the spacecraft
has the ratio

$\left(\frac{\mu_{S}}{a_{P9}^{3}}\right)/\left(\frac{\mu_{P9}}{\rho^{3}}\right)\approx
26.$ (15)

So these motivate us to consider the effect of the Sun on the trajectory of
the spacecraft.

For the three-body system, we shall adopt the Circular Restricted Three-Body
Problem (CRTBP) as the underlying dynamical model. In using this model, the
following assumptions are made:

  * •

The Sun and Planet 9 are moving in a circular orbit around their barycenter.
In fact, the orbit of Planet 9 was estimated to be elliptic ($e\approx
0.2-0.5$) [2], but the simple circular model may still provide insight into
the effect of the Sun.

  * •

The orbit of Planet 9 was estimated as moderately inclined ($i\approx
15^{\circ}-25^{\circ}$) [2]. However, we use the approximation that the Sun,
Planet 9, and spacecraft are moving in the same plane to reduce the dimension
of the problem.

  * •

The mass of the spacecraft is negligible such that it does not affect the
motion of Sun and Planet 9, which is the case in our problem.

The equations of motion for the planar CRTBP of Sun-Planet 9-spacecraft are
given as follows [16];

$\displaystyle\ddot{x}-2\Omega\dot{y}-\Omega^{2}x$
$\displaystyle=-\frac{\mu_{S}}{r_{1}^{3}}\left(x+\pi_{2}r_{12}\right)-\frac{\mu_{P9}}{r_{2}^{3}}\left(x-\pi_{1}r_{12}\right),$
(16a) $\displaystyle\ddot{y}+2\Omega\dot{x}-\Omega^{2}y$
$\displaystyle=-\frac{\mu_{S}}{r_{1}^{3}}y-\frac{\mu_{P9}}{r_{2}^{3}}y,$ (16b)

where $\pi_{1}=m_{S}/(m_{S}+m_{P9})$ and $\pi_{2}=m_{P9}/(m_{S}+m_{P9})$ are
the mass ratios, $r_{12}=a_{P9}$ is the average distance between the Sun and
Planet 9, $\Omega=\sqrt{\frac{(\mu_{S}+\mu_{P9})}{r_{12}^{3}}}$ is the angular
velocity of the Sun-Planet 9 rotating frame, lastly $r_{1}$ and $r_{2}$ are
the distances of the Sun-spacecraft and Planet 9-spacecraft, respectively.

We solve the equations of motion of CRTBP with numerical integration for the
particular initial conditions. In numerical integration, we use the MATLAB
built-in ode45 solver with absolute and relative tolerances of $10^{-13}$. It
is important to note that the transverse displacement should be calculated
relative to the inertial frame, although the motion is obtained in the
rotating frame of CRTBP. Therefore, frame transformations between the rotating
and the inertial frames are applied to obtain the angular displacement.
Regarding the duration of integration, we assume that the measurements start
when the spacecraft reaches a sub-relativistic speed and it takes twice as
long to reach Planet 9. For instance, if the spacecraft reaches Planet 9 in 10
years, then the total integration duration would be 20 years. The symmetric
time interval is assumed in the calculation of transverse displacement.

In order to decide on the proper initial conditions, escape trajectories from
the solar system are examined. In [17] and [18], interstellar mission
trajectories with the help of solar sails were studied. Due to the reduction
of the solar flux and its propulsive effect, in [18] separation of the solar
sail is estimated to be at approximately 5 AU. An example of an interstellar
escape trajectory with a solar sail is given in Fig. 2. In our analysis, we
assume that the spacecraft reaches the sub-relativistic speed at the solar
sail separation distance, so the initial position for the numerical
integration is chosen by considering this assumption. The two possible initial
configurations are considered which are given in Fig. 3 and Fig. 4.

Figure 2: An example of an interstellar escape trajectory with solar sail
(redrawn along the lines of the Figure 13 in [18]). Figure 3:
$\bar{x}$-$\bar{y}$ is an arbitrary inertial frame and $x-y$ is the rotating
frame of Sun-Planet 9-spacecraft CRTBP. The spacecraft is sent horizontally
with a sub-relativistic speed $V_{0}$, and the initial position of Planet 9
makes an angle $\theta$ i.e. calculated from the desired impact parameter
$\rho$. More explicitly, the initial position vector is
$\vec{r}_{0}=5\hat{\bar{x}}-5\hat{\bar{y}}\ [AU]$ , the initial velocity
vector is $\vec{v}_{0}=V_{0}\hat{\bar{x}}$, and
$\theta=\arcsin{(r_{y}/r_{P9})}$. This initial configuration is named as “case
A”. Figure 4: $\bar{x}$-$\bar{y}$ is an arbitrary inertial frame and $x-y$ is
the rotating frame of Sun-Planet 9-spacecraft CRTBP. Spacecraft is sent
diagonally with sub-relativistic speed $V_{0}$, and Planet 9 is initially on
the $x$-axis. More explicitly, the initial position vector is
$\vec{r}_{0}=5\hat{\bar{x}}-5\hat{\bar{y}}\ [AU]$ and the direction of the
velocity vector of the spacecraft is calculated from the basic trigonometric
relations to achieve desired impact parameter $\rho$. This initial
configuration is named as “case B”.

In the presence of the Sun, the unperturbed or nominal trajectory of the
spacecraft is defined as the trajectory obtained from the Sun-spacecraft two-
body problem given as;

$\ddot{\vec{r}}=-\frac{\mu_{S}}{r^{3}}\vec{r},$ (17)

where $\vec{r}$ is the position vector of the spacecraft relative to the Sun.
Then, perturbation due to Planet 9 is calculated as the difference between the
Sun-Planet 9-spacecraft three-body trajectory obtained from (16); and the Sun-
spacecraft two-body trajectory obtained from Eqn. (17). The results of both
initial configurations are given for different velocities and impact
parameters in Fig. 5 and Fig. 6.

In the figures, two-body results correspond to the perturbation of Planet 9 on
a straight-line trajectory and are obtained from (13). The three-body results
correspond to the perturbation of Planet 9 on a trajectory that is defined by
the Sun and spacecraft two-body problem. The two-body and three-body results
are compared to assess whether considering the Sun yields a significant effect
on the detection of Planet 9. Besides, different velocities and impact
parameters are also considered since the angular displacement depends on those
parameters. Velocities are decreased until $0.0004c$ because further reduction
results in a longer mission duration; and different impact parameters are used
for the case of different number of spacecraft.

Figure 5: Angular displacement of two-body and three-body trajectories with
different velocities and impact factors for the Case A. The two-body results
are obtained from the Planet 9 and spacecraft two-body problem, and the three-
body results are obtained from the difference between Sun-Planet 9-spacecraft
three-body trajectory and Sun-spacecraft two-body trajectory.

In Fig. 5, results for the initial configuration of case A are given. The
angular displacement due to Planet 9 is larger in the two-body context for all
the impact parameters and velocities. In CRTBP, Planet 9 and the Sun are
moving in circular orbits, unlike the two-body setting in which one considers
Planet 9 as stationary. Due to the movement of Planet 9, the distance between
Planet 9 and the spacecraft is larger at the flyby for the three-body
trajectory. So, a smaller angular displacement is obtained. As the velocity
decreases, the kinetic energy becomes comparable with the potential energy due
to the Sun and Planet 9. Therefore, their effects become visible, and larger
displacement occurs. As the impact parameter increases, spacecraft flybys with
Planet 9 at a larger distance. So, its gravitational effect decreases and a
smaller displacement occurs. Lastly, all the angular displacements are above
the detection threshold.

Figure 6: Angular displacement of two-body and three-body trajectories with
different velocities and impact factors for the Case B. The two-body results
are obtained from the Planet 9 and spacecraft two-body problem, and the three-
body results are obtained from the difference between Sun-Planet 9-spacecraft
three-body trajectory and Sun-spacecraft two-body trajectory.

In Fig. 6, results for the initial configuration of case B are given. Unlike
case A, angular displacement due to Planet 9 is larger in the three-body
context for all the impact parameters and velocities. Due to the movement of
Planet 9 in the CRTBP, the distance between Planet 9 and the spacecraft is
smaller at the flyby. So, the angular displacement due to Planet 9 is larger.
The behavior for the different impact parameters and velocities is similar to
case A. Here again, all the angular displacements are above the detection
threshold.

To sum up, we found that the effect of the Sun is not directly seen in the
detection but the presence of the Sun causes the motion of Planet 9 and this
yields larger or smaller angular displacements depending on the initial
condition.

##  IV Drag Force on the Spacecraft Trajectory

Due to the interstellar gas particles and dust, the spacecraft is exerted by a
drag force [14];

$F_{drag}=1.4n_{H}m_{H}v^{2}A_{sp},$ (18)

where $n_{H}$ is the proton number density and taken as $n_{H}=1$ cm-3 which
is a standard parameter for the interstellar medium [14], $m_{H}$ is the
proton mass, $v$ is the sub-relativistic speed and $A_{sp}$ is the drag area.
Then, the equation of motion of the spacecraft with drag force is given as;

$\ddot{\vec{r}}=-\frac{\mu_{S}}{r^{3}}\vec{r}-F_{drag}\frac{\vec{v}}{|\vec{v}|}.$
(19)

The first term represents the nominal trajectory of the spacecraft that is
defined by the Sun and the spacecraft two-body problem. In calculating the
drag force, the direction is opposite to that of the velocity. The proton
number density in the interplanetary medium may be different from the
interstellar medium and it may not be constant. But here, we assume that the
drag force affects the trajectory of the spacecraft all the time during the
cruise with the same proton number density.

In analyzing the effect of drag force, the initial configuration of case B is
considered since it has the larger angular displacement as well as the
velocity of the spacecraft along the trajectory that is defined by the Sun-
spacecraft two-body problem would be high so that we can consider the larger
drag force. The perturbative effect of the drag force is investigated as the
difference between the Sun-Drag-spacecraft trajectory and the Sun-spacecraft
two-body trajectory. In calculating the drag force, we take the parameters as
in [14] for comparison. Those are $A_{sp}=0.5$ cm2 and $M_{sp}=1$ gram. The
resulting angular displacement versus velocity plot is given in Fig. 7.

Figure 7: Angular displacement due to drag force with different velocities for
a spacecraft with drag area $A_{sp}=0.5$ cm2 and mass $M_{sp}=1$ gram and for
the initial configuration of case B.

As the velocity increases, one expects to have an increasing trend in the
displacement due to a larger drag force, but just the opposite happens in Fig.
7. The reason is that the spacecraft travels the same distance for all
velocities due to the assumptions on integration duration. So, the spacecraft
with high velocity travels the same distance in a smaller amount of time which
results in a shorter interaction time with the drag force. The spacecraft with
a lower velocity is affected by the drag force for a longer time, so the drag
causes more angular displacement for the lower velocities. We did not perform
analysis for the different mass and drag areas, but their effects can be
assessed from Fig. 7. For example, if the drag area is doubled, then the
angular displacement is also doubled, etc.

In [14], only the magnitudes are considered but here we have taken into
account the direction as well and found that the angular displacement due to
the drag force is about one order smaller than the gravity of Planet 9 for the
parameters used in the analysis. Since the detection threshold is $10^{-9}$
radians, the effect of drag force may not be even detectable.

##  V Magnetic Force on the Spacecraft Trajectory

During the cruise, the spacecraft gets electrically charged due to colliding
interstellar particles and also due to photoelectric effects from solar and
interstellar photons [14]. Then, the interstellar magnetic field exerts a
magnetic force on the charged spacecraft and curves the trajectory. The
magnetic force due to interstellar medium is given as [14];

$F_{mag}=\frac{e}{c}Z_{sp}vB_{\perp},$ (20)

in Gaussian units. Here, $e$ is the elementary charge, $v$ is the sub-
relativistic speed, $B_{\perp}$ is the magnetic field component perpendicular
to the motion, $c$ is the speed of light and $Z_{sp}$ is the maximum charge of
the spacecraft that is given as [14];

$Z_{sp}=\frac{m_{e}v^{2}}{e^{2}}\left(\frac{M_{sp}}{\rho}\right)^{1/3},$ (21)

where $m_{e}$ is the mass of the electron, $M_{sp}$ is the mass of the
spacecraft and $\rho$ is the density of spacecraft. One can look at [19] for a
detailed discussion about the calculation of $Z_{sp}$. Then, the equation of
motion of the spacecraft with the magnetic force is given as;

$\ddot{\vec{r}}=-\frac{\mu_{S}}{r^{3}}\vec{r}+F_{mag}\hat{b},$ (22)

where the direction $\hat{b}$ is perpendicular to both the velocity vector and
the orbit angular momentum vector of the spacecraft so that the maximum
displacement occurs due to the magnetic force. The parameters are taken as the
same as in [14] for the comparison. Those are $M_{sp}=1$ gram, $\rho=3$ gram
cm-3 and $B_{\perp}\sim 5\mu G$. Here again, the initial configuration of case
B is considered for the same reasons mentioned in the drag force section.
Then, the perturbative effect of the magnetic force is investigated as the
difference between the Sun-Magnetic Force-spacecraft trajectory and the Sun-
spacecraft two-body trajectory. The resulting angular displacement versus
velocity plot is given in Fig. 8.

Figure 8: Angular displacement due to magnetic force with different velocities
for a spacecraft with mass $M_{sp}=1$ gram and $\rho=3$ gram cm-3 and for the
initial configuration of case B

As the velocity increases, the magnetic force also increases and curves the
trajectory. Therefore, a larger angular displacement is obtained for the high
velocities. There is a cubic dependence on velocity for the magnetic force,
and this makes the effect of velocity more dominant compared to the drag
force. The maximum charge of the spacecraft depends on the physical properties
of the spacecraft such as mass and density. Their effects on the displacement
can be assessed from Fig. 8. The resulting angular displacement due to
magnetic force is below the detection threshold and is not detectable for the
parameters used in the analysis.

##  VI Solar Radiation Pressure on the Spacecraft Trajectory

The spacecraft is exposed to the Solar Radiation Pressure (SRP) during the
cruise which perturbs the trajectory. Previously, we mentioned that solar sail
is separated at about 5 AU due to the reduction of solar sail effectiveness,
although the perturbative effect of SRP still continues since the spacecraft
is exposed to solar radiation. The acceleration due to SRP on the spacecraft
is given as [16];

$F_{srp}=\nu\frac{S}{c}C_{R}A_{S},$ (23)

where $\nu$ is the eclipse factor that is always 1 in our analysis due to the
assumption that the spacecraft is always sunlit, $c$ is the speed of light,
$A_{S}$ is the surface area of the spacecraft that is exposed to SRP, $C_{R}$
is the reflectivity coefficient of the spacecraft surface and takes a moderate
value $C_{R}=1.5$. Lastly, $S$ is the solar flux as a function of the distance
of the spacecraft that is given as [16];

$S=S_{0}\left(\frac{R_{0}}{R}\right)^{2},$ (24)

where $S_{0}=63.15\times 10^{6}$ W/m2 is the radiated power intensity of the
photosphere, $R_{0}=696000$ km is the radius of the photosphere and $R$ is the
distance between the Sun and the spacecraft. Then, the equation of motion of
the spacecraft with SRP is given as;

$\ddot{\vec{r}}=-\frac{\mu_{S}}{r^{3}}\vec{r}+F_{srp}\hat{r},$ (25)

where $\hat{r}$ is the unit vector from Sun to spacecraft. In calculating the
SRP, the SRP area $A_{S}=0.5$ cm2 is taken as the same as the drag area, and
the mass of the spacecraft is taken as $M_{sp}=1$ gram. Here again, the
initial configuration of case B is considered. The perturbative effect of SRP
is investigated as the difference between the Sun-SRP-spacecraft trajectory
and the Sun-spacecraft two-body trajectory. The resulting angular displacement
versus velocity plot is given in Fig. 9.

Figure 9: Angular displacement due to SRP with different velocities for a
spacecraft with mass $M_{sp}=1$ gram and SRP area $A_{S}=0.5$ cm2 and for the
initial configuration of case B.

As the velocity increases, the spacecraft is exposed to the SRP in a shorter
time interval and therefore the angular displacement decreases. In fact, SRP
depends on the physical properties of the spacecraft like mass, SRP area, and
the reflectivity coefficient but their effects for different values can be
assessed from the Fig. 9. We found that the order of angular displacement due
to SRP is larger than or about the same as the gravity of Planet 9 which is
bad in terms of detectability of the planet. To make the gravity of Planet 9
the leading order displacement, one can increase the spacecraft’s mass. Of
course, that will increase the cost of launching the spacecraft to high
speeds.

##  VII General Relativistic Effects on the Spacecraft Trajectory

###  VII.1 Frame-Dragging Effect

Due to the rotation of the central massive body, a frame-dragging effect
arises in General Relativity. Here, we first study the effect of frame-
dragging on the detection of Planet 9 by comparing its magnitude with that of
the gravity of Planet 9. Frame-dragging correction to the acceleration of the
spacecraft is given as [20];

$\Delta{\ddot{\vec{r}}_{FD}}=(1+\gamma)\frac{GM}{c^{2}r^{3}}\left[\frac{3}{r^{2}}\left(\vec{r}\times\dot{\vec{r}}\right)\left(\vec{r}\cdot\vec{J}\right)+\left(\dot{\vec{r}}\times\vec{J}\right)\right],$
(26)

where $\gamma$ is a post-Newtonian parameter which we will take to be 1, $GM$
is the gravitational parameter, $c$ is the speed of light, $\vec{r}$ and
$\dot{\vec{r}}$ are the position and velocity vectors of the spacecraft and
lastly $\vec{J}$ is the angular momentum vector per unit mass of the relevant
body.

We want to investigate the frame-dragging effect of our solar system on a sub-
relativistic spacecraft with $v=0.001c$ at the distance of Planet 9. So the
parameters are taken as follows; $GM$ is the gravitational parameter of the
Sun, $r$ is the average distance between the Sun and Planet 9 which is
$a_{P9}$, and lastly $\vec{J}$ is the orbit angular momentum vector per unit
mass of Jupiter since it has the major contribution to the angular momentum of
our solar system. Also, since we study the planar motion, position and
velocity vectors of the spacecraft, i.e. $\vec{r}$ and $\dot{\vec{r}}$, would
be perpendicular to the angular momentum vector $\vec{J}$. In the vicinity of
Planet 9, i.e. at a distance of $\rho$, the ratio of the Newtonian gravity of
Planet 9 ($|\ddot{\vec{r}}_{P9}|$) and the frame-dragging effect due to the
solar system ($|\Delta{\ddot{\vec{r}}_{FD,SS}}|$) is

$\frac{|\ddot{\vec{r}}_{P9}|}{|\Delta{\ddot{\vec{r}}_{FD,SS}}|}\approx
10^{6}.$ (27)

So, the frame-dragging effect due to the solar system is negligible.

Next, we look at the frame-dragging effect due to Planet 9 on the spacecraft.
The specific orbit angular momentum of Planet 9 can be calculated by
considering the orbital elements given in [2] as follows [16];

$J_{P9}=\sqrt{a_{P9}\mu_{P9}(1-e_{P9}^{2})},$ (28)

where $e_{P9}$ is the eccentricity and is chosen to be $e_{P9}=0.2$ to have a
larger angular momentum. In the vicinity of Planet 9, the frame-dragging
effect due to the orbit angular momentum of Planet 9 on the spacecraft
($|\Delta{\ddot{\vec{r}}_{FD,P9}}|$) can be calculated by considering
$GM=\mu_{P9}$, $r=\rho=30$ AU, $v=0.001c$ and $J=J_{P9}$. Its comparison with
the Newtonian gravity of Planet 9 is given as follows;

$\frac{|\ddot{\vec{r}}_{P9}|}{|\Delta{\ddot{\vec{r}}_{FD,P9}}|}\approx
10^{9}.$ (29)

So, the frame-dragging effect due to orbit angular momentum of Planet 9 is
negligible.

Lastly, we consider Planet 9 as an extremal black hole, so the maximum spin
angular momentum per unit mass of a black hole can be given as;

$J_{BH}=\frac{GM}{c},$ (30)

where $GM=\mu_{P9}$. In the vicinity of Planet 9, the frame-dragging effect
due to the spin angular momentum of a black hole on the spacecraft
($|\Delta{\ddot{\vec{r}}_{FD,BH}}|$) can be calculated by considering
$GM=\mu_{P9}$, $r=\rho=30$ AU, $v=0.001c$ and $J=J_{BH}$. Its comparison with
the Newtonian gravity of Planet 9 is given as follows;

$\frac{|\ddot{\vec{r}}_{P9}|}{|\Delta{\ddot{\vec{r}}_{FD,BH}}|}\approx
10^{16}.$ (31)

So, the frame-dragging effect due to spin angular momentum of a black hole
that equals the mass of Planet 9 is negligible. It has the smallest magnitude
among the other frame-dragging effects.

###  VII.2 Schwarzschild Effect

General Relativistic corrections to the Newtonian gravity, the so-called
Schwarzschild effect, on the spacecraft can be given as [20];

$\Delta{\ddot{\vec{r}}_{Sch}}=\frac{GM}{c^{2}r^{3}}\Biggl{\\{}\left[2(\beta+\gamma)\frac{GM}{r}-\gamma\dot{\vec{r}}\cdot\dot{\vec{r}}\right]\vec{r}+2(1+\gamma)(\vec{r}\cdot\dot{\vec{r}})\dot{\vec{r}}\Biggl{\\}},$
(32)

where $\beta$ and $\gamma$ are the post-Newtonian parameters which we set to
1, $GM$ is the gravitational parameter, $c$ is the speed of light, $\vec{r}$
and $\dot{\vec{r}}$ are the position and velocity vectors of the spacecraft.

The Schwarzschild effect due to the Sun on the spacecraft
($|\Delta{\ddot{\vec{r}}_{Sch,S}}|$) in the vicinity of Planet 9 is calculated
by considering $GM=\mu_{S}$, $r=a_{P9}$, $v=0.001c$ and by assuming $\vec{r}$
and $\dot{\vec{r}}$ are in the same direction so that
$\vec{r}\cdot\dot{\vec{r}}=|\vec{r}||\dot{\vec{r}}|$. Its comparison with the
Newtonian gravity of Planet 9 is given as follows;

$\frac{|\ddot{\vec{r}}_{P9}|}{|\Delta{\ddot{\vec{r}}_{Sch,S}}|}\approx
10^{3}.$ (33)

The Schwarzschild effect due to Sun is larger than the frame-dragging effects.
But still, the deflection on the spacecraft trajectory due to the Newtonian
gravity of Planet 9 can be distinguishable from the Schwarzschild effect of
the Sun since there are three order differences.

Next, we look for the Schwarzschild effect due to Planet 9 in its
neighborhood. The angle between $\vec{r}$ and $\dot{\vec{r}}$ may change
during the flyby with Planet 9. In fact, they would be perpendicular at the
closest point, i.e. at the periapsis, since they are defined relative to
Planet 9. However, we assume that $\vec{r}$ and $\dot{\vec{r}}$ are in the
same direction to consider the maximum effect. Then, the ratio of the
Newtonian gravity of Planet 9 and the Schwarzschild effect due to Planet 9
($|\Delta{\ddot{\vec{r}}_{Sch,P9}}|$) is

$\frac{|\ddot{\vec{r}}_{P9}|}{|\Delta{\ddot{\vec{r}}_{Sch,P9}}|}\approx
10^{5}.$ (34)

In fact, the ratio would be even bigger because of the aforementioned
assumption. It does not give a meaningful contribution to the deflection.

###  VII.3 Geodetic Precession Effect

Due to the curvature in the spacetime, a vector, for example, the spin vector
of a particle orbiting around a massive body will precess, so the direction of
the spin vector would not be the same after one period. This is called as
geodetic precession effect. This effect also causes the precession on the
eccentricity vector or the orbit angular momentum vector of the particle and
changes its orbit. This effect is given as [20];

$\Delta{\ddot{\vec{r}}_{Geo}}=(1+2\gamma)\left[\dot{\vec{R}}\times\left(\frac{-GM\vec{R}}{c^{2}R^{3}}\right)\right]\times\dot{\vec{r}}.$
(35)

Here, $\gamma$ is 1, $c$ is the speed of light and $\dot{\vec{r}}$ is the
velocity vector of the spacecraft.

In calculating the geodetic precession effect due to Planet 9, $GM$ is the
gravitational parameter of the Sun, $\vec{R}$ and $\dot{\vec{R}}$ are the
position and velocity vectors of the Planet 9 relative to Sun. Since the
planar motion is studied, the orbit angular momentum vector of Planet 9
($\vec{R}\times\dot{\vec{R}}$) is perpendicular to the velocity vector of the
spacecraft ($\dot{\vec{r}}$). In the vicinity of Planet 9, the ratio of
Newtonian gravity of Planet 9 and the geodetic precession effect due to Planet
9 ($|\Delta{\ddot{\vec{r}}_{Geo,P9}}|$) is given as follows:

$\frac{|\ddot{\vec{r}}_{P9}|}{|\Delta{\ddot{\vec{r}}_{Geo,P9}}|}\approx
10^{5}.$ (36)

So, the geodetic precession effect due to Planet 9 is negligible.

##  VIII Conclusions

We studied various effects on a sub-relativistic spacecraft that is sent to
gravitationally probe Planet 9. As a measure of the detection, we used the
angular displacement in the trajectory of the spacecraft. First, we looked for
the effect of the Sun by analyzing the motion of a spacecraft in the circular
restricted three-body problem approach for the Sun-Planet 9-spacecraft system
for the two particular initial conditions. We found that the presence of the
Sun causes the motion of Planet 9 and this yields larger or smaller angular
displacements compared to Planet 9-spacecraft two-body trajectory depending on
the initial condition. However, the angular displacements in both two-body and
three-body trajectories are above the detection threshold, so they are
detectable. Next, we studied the effects of non-gravitational perturbations
like the drag force, the magnetic force, and the solar radiation pressure by
solving the corresponding equation of motion numerically. These effects depend
on the physical properties of the spacecraft. For the parameters used in our
analysis, we found that the angular displacement due to drag force would be
hard to detect. The angular displacement due to the magnetic force is below
the detection threshold, so it is not detectable. The leading order angular
displacement is due to the solar radiation pressure. It is larger than or
about the same order as the gravity of Planet 9. This makes the detection of
Planet 9 difficult. One of the solutions would be to increase the mass of the
spacecraft so that the radiation pressure is reduced. Lastly, the General
Relativistic effects like the frame-dragging, Schwarzschild, and geodetic
precession on the spacecraft trajectory were investigated by comparing their
magnitudes with the Newtonian gravity of Planet 9. We found that the frame-
dragging effect has the smallest effect and the Schwarzschild effect due to
Sun has the largest effect. However, none of the General Relativistic effects
produces a meaningful contribution to the detection.

##  IX Acknowledgments

We would like to thank Tahsin Cagri Sisman, Cetin Senturk and Niyazi Anil
Gezer for useful discussions on the topic.

## References

  * [1] K. Batygin and M. E. Brown, “Evidence for a distant giant planet in the solar system,” _The Astronomical Journal_ , vol. 151, no. 2, 2016.
  * [2] K. Batygin, F. C. Adams, M. E. Brown and J. C. Becker, “The planet nine hypothesis,” _Physics Reports_ , vol. 805, pp. 1-53, 2019.
  * [3] S. Naess _et al._ , “The Atacama cosmology telescope: A search for Planet 9,” _The Astrophysical Journal_ , vol. 923, no. 2, 2021.
  * [4] J. Scholtz and J. Unwin, “What if Planet 9 is a primordial black hole?,” _Phys. Rev. Lett._ , vol. 125, no. 5, 2020.
  * [5] E. Witten, “Searching for a Black Hole in the Outer Solar System,” https://doi.org/10.48550/arXiv.2004.14192 (accessed Jun. 16, 2023).
  * [6] A. Arbey and J. Auffinger, “Detecting Planet 9 via Hawking Radiation,” https://doi.org/10.48550/arXiv.2006.02944 (accessed Jun. 16, 2023).
  * [7] M. E. Brown and K. Batygin, “A search for Planet Nine using the Zwicky transient facility public archive,” _The Astronomical Journal_ , vol. 163, no. 2, 2022.
  * [8] M. H. Chan, “What if Planet 9 has satellites?,” _The Astrophysical Journal_ , vol. 944, no. 2, 2023.
  * [9] H. S. Navarro, “A candidate location for Planet Nine from an interstellar meteoroid: The messenger hypothesis,” _The Astrophysical Journal_ , vol. 945, no. 1, 2023.
  * [10] J. Bucko, D. Soyuer and L. Zwick, “Prospects for localizing Planet 9 with a future Uranus mission,” _Monthly Notices of the Royal Astronomical Society: Letters_ , vol. 524, no. 1, 2023.
  * [11] M. Macdonald, C. McInnes and G. Hughes, “Technology requirements of exploration beyond Neptune by solar sail propulsion,” _Journal of Spacecraft and Rockets_ , vol. 47, no. 3, 2010.
  * [12] K. L. G. Parkin, “The Breakthrough Starshot system model,” _Acta Astronautica_ , vol. 152, pp. 370-384, 2018.
  * [13] S. Lawrence and Z. Rogoszinski, “The Brute-Force Search for Planet Nine,” https://doi.org/10.48550/arXiv.2004.14980 (accessed Jun. 16, 2023).
  * [14] T. Hoang and A. Loeb, “Can Planet Nine be detected gravitationally by a subrelativistic spacecraft?,” _The Astrophysical Journal Letters_ , vol. 895, no. 2, 2020.
  * [15] K. Batygin and M. E. Brown, “The orbit of Planet Nine,” _The Astronomical Journal_ , vol. 162, no. 5, 2021.
  * [16] H. D. Curtis, _Orbital Mechanics for Engineering Students_ , 3rd ed., Butterworth-Heinemann, 2013.
  * [17] X. Zeng, G. Vulpetti and C. Circi, “Solar sail H-reversal trajectory: A review of its advances and applications,” _Astrodynamics_ , vol. 3, no. 1, pp. 1-15, 2019.
  * [18] C. G. Sauer, “Solar sail trajectories for solar-polar and interstellar probe missions,” in _AAS/AIAA Astrodynamics Specialist Conference_ , Girdwood, Alaska, 1999.
  * [19] T. Hoang and A. Loeb, “Electromagnetic forces on a relativistic spacecraft in the interstellar medium,” _The Astrophysical Journal_ , vol.848, no.1, 2017.
  * [20] K. Sośnica _et al._ , “General relativistic effects acting on the orbits of Galileo satellites,” _Celestial Mechanics and Dynamical Astronomy_ , vol.133, no.14, 2021.