Onboard machine learning on the latest satellite hardware offers the potential for significant savings in communication and operational costs. We showcase the training of a machine learning model on a satellite constellation for scene classification using semi-supervised learning while accounting for operational constraints such as temperature and limited power budgets based on satellite processor benchmarks of the neural network. We evaluate mission scenarios employing both decentralized and federated learning approaches. All scenarios achieve convergence to high accuracy (around 91% on EuroSAT RGB dataset) within a one-day mission timeframe.

§ INTRODUCTION

A new generation of satellites is currently bringing hardware suitable for machine learning (ML) onboard spacecraft into Earth orbit.
Recent works <cit.> explored the possibility to train ML models in a distributed manner onboard satellite constellations.
Distributed onboard training brings the potential to reduce communication requirements, operational cost and time, and improve autonomy by sharing ML models, trained close to the sensors, instead of the collected data.
While previous missions have demonstrated the ability to perform inference onboard spacecraft for data processing <cit.>, training onboard presents additional challenges. Convincingly addressing operational constraints is crucial, as the computational cost of training is significantly higher, and the lack of labeled examples during the mission can often be prohibitive.

In this work, we investigate the training of an ML model onboard a satellite constellation for scene classification. We employ a semi-supervised learning approach called MSMatch <cit.>, which we successfully distribute using decentralized learning techniques. Operational constraints such as temperature, communication windows, and limited power budgets are modeled using PASEOS <cit.>, a specialized Python module. We provide detailed results on various scenarios involving decentralized and federated learning approaches.

§ METHODS

This work is built upon three core components: the semi-supervised learning method MSMatch, modeling constraints with PASEOS, and adapting MSMatch for distributed implementation.

§.§ MSMatch

One of the primary challenges for ML applications on spacecraft is the scarcity of labeled training data, particularly before launch. Often, there is an insufficient number of labeled examples for training a model on the ground, necessitating the use of semi- and self-supervised techniques in many instances. MSMatch is a semi-supervised classification method specifically designed for such scenarios <cit.>, and has been proven to achieve high accuracy even when trained with merely a few labels per class.

MSMatch employs consistency regularization and pseudo-labeling to train primarily on unlabeled images. It fundamentally relies on the consistency between the model's predictions on two differently augmented (one strongly, one weakly) versions of the same image. A pseudo-label is generated for the weakly augmented version. Additionally, a supervised loss is applied to the limited available labeled examples. With as few as five labeled samples per class, MSMatch can achieve accuracies above 90% on established benchmarks such as EuroSAT <cit.>. The method has also demonstrated effectiveness with multispectral data.

For our implementation, we built upon the existing open-source codebase available online[<https://github.com/gomezzz/MSMatch> Accessed: 2023-02-27]. We utilized the EfficientNet-lite models (efficientnet-lite0), derived from the original EfficientNets <cit.>, as the backend. Due to their small memory footprint and efficiency, these models are well-suited for embedded systems and, therefore, onboard processors.

§.§ PASEOS

Training machine learning models in space necessitates accounting for factors such as power budgets, thermal management, and communication windows, as these directly impact the viability of training <cit.>. Communication windows, in particular, are a critical factor in distributed computing scenarios <cit.>.

To model these constraints, we employ the open-source Python module PASEOS <cit.> (Version 0.1.3). PASEOS simulates spacecraft orbital dynamics and power budgets, taking into account power consumption, available solar panels, and eclipses. Thermal management is modeled using a single-node ordinary differential equation <cit.>. Packet communication is calculated based on the assumed available bandwidth and the presence of a line of sight between communication partners. PASEOS operates asynchronously to the training pipeline, thereby limiting the ability to train and exchange models. A comprehensive description of the models can be found in the article dedicated to PASEOS <cit.>.

§.§ Decentralized MSMatch

In this study, we demonstrate the capability of training MSMatch in a distributed environment by leveraging well-established techniques for merging local models, such as federated averaging <cit.>. The Message Passing Interface (MPI) is employed to enable asynchronous training of multiple models while concurrently running PASEOS simulations for each satellite.
It is assumed that labeled training data are available prior to launch and preloaded onto each satellite. Unlabeled training data, on the other hand, are randomly distributed among the satellites, with each satellite receiving a fixed number of distinct, unlabeled samples. The hyperparameters of the decentralized MSMatch are comparable to those in <cit.>, with a few exceptions: the batch size is reduced to 32 for labeled data and 96 for unlabeled data, and the learning rate is increased to 0.03. MPI facilitates communication if PASEOS simulations indicate an available window for data exchange.
A decentralized MSMatch scenario involving two satellites and a ground station is depicted in Fig. <ref>. It is important to note that while the satellites share the same labeled examples, the unlabeled data differ. The code for our work is openly accessible online[<https://github.com/gomezzz/DistMSMatch> Accessed: 2023-04-05].

Decentralized MSMatch for two satellites and a ground station.

§ RESULTS

§.§ Setup and Scenarios

To test the proposed method we rely on the EuroSAT dataset  <cit.> used in the original MSMatch paper <cit.> to enable a direct comparison.
The dataset is comprised of 27000 $64 \times 64$ pixel, 13-channel images from Sentinel-2A data classified into ten classes, such as forest or river.
In our experimental results, we utilize only the RGB channels.
The choice to employ solely the RGB channels stems from two factors: firstly, the RGB channels have already demonstrated satisfactory performance; and secondly, the inclusion of all channels would substantially prolong the training time. However, it should be noted that relying on only the RGB channels places us in a less favorable situation, and leveraging more available data would likely enhance performance further.

To demonstrate the ability to learn in a semi-supervised way, five labeled images from each class, i.e., 50 labeled images in total, are extracted and loaded onto each satellite.
That is, the satellites are loaded with the same 50 images.
The test set consists of 2700 images and the remaining 24250 images are treated as unlabeled data and are randomly split into eight partitions, i.e., 3031.25 images per partition on average.

The satellites utilize a radio frequency (RF) link of 1 Mbps when communicating with a ground station, and optical inter-satellite links (ISL)  in orbit at 100 Mbps.
To account for tracking and alignment, we assume the ISL between two satellites to exhibit a setup-time of 30 seconds before every communication attempt <cit.>.
The satellites are equipped with a 0.2772 MJ battery and solar panels that charge at 20 W.
The parameters of the thermal model assume a mass of six kilograms, an initial temperature of 26.85 degree Celsius, sun absorptance of 0.9, and an infrared absorptance of 0.5.
The sun-facing and Earth-facing area of each satellite are 0.015 m$^2$ and 0.01 m$^2$, respectively. The emissive area is 0.1 m$^2$ and the thermal capacity is 5000 J/(kg * K).
The utilized EfficientNet-lite0 network occupies 12.7 MB of storage in a compressed state and the model exchange with the ground takes 201.78 seconds whereas the model exchange via ISL takes 32.03 seconds.
Training a batch required 15.98 seconds on a Unibap iX10 satellite processor CPU.
Further, communications (to ground), communications (ISL), training, and standby are assumed to consume 10 W, 13.5 W, 30 W, and 5 W, respectively.
The PASEOS simulation is run with the default configuration of v0.1.3. The initial epoch of the simulation is 2022-Dec-17 14:42:42.

The investigated, distributed scenarios, as displayed in Fig. <ref>, involve a constellation in a Sentinel-like orbit (sun-synchronous at 786 km altitude with 98.62inclination) featuring eight satellites.
The first scenario (Ground Station) assumes model exchanges via three (linked) ground stations on Gran Canaria, Svalbard and in Matera, Italy.
From a satellite perspective, the ground stations are viewed as a single unit.
In the second (Swarm), eight satellites are assumed to communicate via ISL.
Finally, in the third (Relay), one of the European Data Relay Satellite System relays (EDRS-C) is assumed to act as a central server for federated learning. Communication delays due to the relay potentially being busy are neglected.
In the federated settings (Ground Station and Relay), the global model is updated asynchronously whenever a local model becomes available by a convex combination, with weight 0.4, of the global model and the newly received model similarly to <cit.> (with constant weighting function).
Furthermore, each of the satellites will attempt to share their local models every 1000 seconds.

Ground Station.

Visualization of the different constellations and communication setups.

§.§ Training Results

Results averaged over the eight satellites over three different runs.

| X[0.5cm] | X[0.60cm] | X[0.5cm] | X[0.5cm] | X[0.55cm] | X[0.7cm] | X[0.45cm] |
Setup     Accuracy [%]      Transmitted Data [MB]     Power Consumption [Wh]     Time Training [% of total]     Time Communicating [% of total]     time between communications [s]
Ground Station       91.51 $\pm 0.95$       185.74     447.58      54.25     1.71     5913.6
Swarm      90.96$\pm 1.34$      1168.40     449.36     53.76     3.34      1878.4
Relay      91.19$\pm 0.76$     455.61     449.37     54.45     1.32     2349.3

The results obtained from 24 hours of simulation time (equivalent to 14.34 orbital revolutions) have been averaged over three independent runs per scenario and are presented in Table <ref>. All three scenarios attain an average accuracy exceeding 91%, with the Ground Station scenario achieving the highest accuracy at 91.51%.
It can also be seen that the standard deviation is the lowest for Relay and largest for Swarm.
This is expected as the Relay and Ground Station scenarios involves sharing a global model in contrast to the Swarm scenario.

Ground Station.

Top-1 accuracy on the test set for the satellites averaged over three runs. Different colors indicate different satellites.

The Swarm scenario is the most communication-intensive, with satellites sharing an average of 1168.40 MB of data. This is because there are always neighboring satellites available to receive local models. In contrast, satellites in the Relay scenario transmit an average of 455.61 MB of data, which is considerably less than in the Swarm scenario. This difference is due to the relay satellite occasionally being obscured by Earth. The Ground Station scenario has the least data transmission, with satellites transmitting only 185.74 MB of data, as the link to the ground stations is infrequently available.

Over the simulated 24-hour period, satellites in the Ground Station, Swarm, and Relay scenarios communicate their local models an average of 14.625, 46, and 35.875 times, respectively. It is important to note, however, that this is not reflected in the relative time spent communicating, as the Relay scenario spends the least time communicating due to the Ground Station relying on RF communications. The power consumption and total training time are similar across all three scenarios.
Note, however, that the cost of operating the ground station or the relay satellite is not accounted for.

The convergence of the top-1 accuracy (averaged over three independent runs) for each satellite is depicted in Fig. <ref>. The performance among different satellites in the Swarm and Relay scenarios is more consistent due to frequent communication, which prevents satellites from deviating significantly. The Swarm scenario reaches a local optimum after approximately two orbital revolutions, resulting in an 88.5% top-1 accuracy, while the Relay scenario attains a less favorable local optimum in the same time frame, with an 86.7% top-1 accuracy. Interestingly, satellites in both scenarios escape their local optima after eight orbital revolutions and find a more favorable optimum after 12 revolutions. Although the Ground Station scenario is not as consistent as the other two scenarios, it exhibits similar behavior.

As previously mentioned, PASEOS enables accounting for constraints such as power and temperature. Figure <ref> illustrates the average temperature and power consumption. In our numerical experiments, the spacecraft enters standby mode to recharge or cool down when the state of charge drops below 0.2 or the temperature exceeds 313.15 K (indicated by dashed lines in Figure <ref>). However, communication is prioritized and will always be performed.

The temperature behaves similarly across all settings, as the primary influencing factor is the training process. The temperature (represented by blue curves) can be observed to trigger standby mode after approximately 10 orbital revolutions. Subsequently, the satellites enter standby mode to cool down and initiate training as soon as the temperature no longer violates the constraint, causing the temperature to oscillate around the constraint temperature.

In contrast, the state of charge (depicted by red curves) exhibits different behavior across the three scenarios. In the Swarm scenario, spacecraft consistently communicate their models. However, in the Ground Station and Relay scenarios, satellites do not always have a communication link available to share local models, resulting in less predictable battery consumption among the satellites.
Since training consumes the most power and the orbit allows the spacecraft to charge for most of the time, the more frequent standby mode triggered by temperature enables the spacecraft to recharge the battery.

Ground Station.

Average temperature and state of charge of the constellation. Temperatures in blue and state of charge in red.

The results depicted in Figure <ref> are derived from 5 labeled examples per class, with the remaining data samples left unlabeled. To evaluate the influence of the labeled dataset size, we concentrate on the Swarm scenario and conduct the experiment with varying labeled dataset sizes. Figure <ref> illustrates the average top-1 accuracy (across satellites and 3 independent runs).

As anticipated, the top-1 accuracy increases as the size of the labeled dataset expands. Notably, performance approaches 90% with just 3 labeled examples per class, and with 100 labels per class, the top-1 accuracy attains 96.2%. For comparison purposes, we include the performance of the centralized implementation of MSMatch <cit.>. It is important to recognize that this comparison may not be entirely fair, as the centralized version undergoes training for a substantially longer duration and employs a larger EfficientNet model. Nonetheless, the distributed version of MSMatch presented herein proves to be competitive and achieves comparable performance.

Swarm top-1 test accuracy vs number of labeled examples per class. The points are averaged over satellites and three independent runs.

§ CONCLUSION

In this study, we illustrate the feasibility of training a state-of-the-art neural network in a semi-supervised manner, distributed across a satellite constellation using current satellite processors. Depending on the orbital configuration and assets, the constellation learns to classify the EuroSAT dataset with up to 91.51% accuracy after a simulated training duration of 24 hours. Moving forward, the incorporation of more intricate scenarios, communication schemes, and a refined satellite architecture will enable further optimizations and increased fidelity.

§ ACKNOWLEDGEMENT

The authors would like to thank Unibap AB for providing the iX-10 100 device that was used for our experiments. The work of Johan Östman was funded by Vinnova under grant 2020-04825 and the work of Vinutha Magal Shreenath under Vinnova grant 2021-03643 and under Swedish National Space Agency grant 2022-00013.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

journaltitlearXiv preprint arXiv:2206.00307
titleFederated Learning in Satellite Constellations

journaltitleIEEE Transactions on Geoscience and Remote Sensing
titleThe $\Phi$-Sat-1 Mission: The First On-Board Deep Neural Network Demonstrator for Satellite Earth Observation

journaltitleIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
titleMSMatch: Semisupervised Multispectral Scene Classification With Few Labels

journaltitlearXiv:2302.02659 [cs.DC]
titlePAseos Simulates the Environment for Operating multiple Spacecraft

booktitlethe IEEE International Geoscience and Remote Sensing Symposium (IGARSS)
titleIntroducing Eurosat: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification

booktitleInternational conference on machine learning
titleEfficientnet: Rethinking model scaling for convolutional neural networks

booktitleICC 2022-IEEE International Conference on Communications
titleOn-board federated learning for dense LEO constellations

journaltitlearXiv:2301.05285 [cs.NI]
titleLaser Inter-Satellite Link Setup Delay: Quantification, Impact, and Tolerable Value

journaltitlearXiv:1903.03934 [cs.DC]
titleAsynchronous federated optimization


# SPRINT: A fast, new software tool for reconstructing the evolutionary past
of polyploid datasets

Liam J. Maher, Taoyang Wu and Katharina T. Huber  School of Computing
Sciences, University of East Anglia, Norwich, UK<EMAIL_ADDRESS>School of
Computing Sciences, University of East Anglia, Norwich, UK
<EMAIL_ADDRESS>School of Computing Sciences, University of East Anglia,
Norwich, UK<EMAIL_ADDRESS>
###### Abstract.

Polyploidization is an important evolutionary process which affects organisms
ranging from plants to fish and fungi. The signal left behind by it is in the
form of a species’ ploidy level (number of complete chromosome sets found in a
cell) which is inherently non-treelike. Currently available tools for
reconstructing the evolutionary past of a polyploid dataset generally start
with a multi-labelled tree obtained for a dataset of interest and then derive
a (phylogenetic) network from that tree in some way that reflects that past by
interpreting the networks’s vertices of indegree at least two as
polyploidization events. Since obtaining such a tree can be computationally
expensive it is paramount to have alternative approaches available that allow
one to shed light into the reticulate evolutionary past of a polyploid
dataset. SPRINT aims to reconstruct the evolutionary past of a polyploid
dataset in terms of a binary network which realises the dataset’s ploidy
profile (vector of ploidy levels of the dataset’s taxa) and requires the
fewest number of polyploidization events. It does this by representing the
ploidy level of a species $x$ in terms of the number of directed paths from
the root of the network to the leaf of the network labelled by $x$. SPRINT is
distributed on GitHub: https://github.com/lmaher1/SPRINT.

###### Key words and phrases:

Keywords: Phylogenetic network, hybrid number

###### Key words and phrases:

Phylogenetic network and ploidy profile and multiplicity vector and hybrid
number and simplification sequence and prime factor decomposition

###### 1991 Mathematics Subject Classification:

1991 Mathematics Subject Classification. 05C05; 92D15

##  1\. Introduction

Polyploidization is an important evolutionary driver which affects organisms
ranging from plants to fish and fungi. Existing tools for reconstructing the
evolutionary past of polyploid datasets include those described in [4, 5].
Their main drawback is their reliance on the availability of a multi-labelled
tree for the species of interest (see e.g. [4]) because such a tree might not
be readily available. Tools that circumvent this problem are currently largely
lacking thereby hindering the exploitation of polyploidy information by
evolutionary biologists. Here, we present the Species Ploidy Realisation of
Integers with Networks Tool (SPRINT) software tool which solely relies on a
dataset’s ploidy levels for shedding light into its evolutionary past.

##  2\. SPRINT

Given a polyploid dataset SPRINT uses a heuristic approach to reconstruct the
evolutionary past of a polypoid dataset in terms of a network that realizes
its ploidy profile. In view of [2], the number of polyploidization events
required by the inferred network is minimal under certain conditions (see
below) and an upper bound in general. The generated network may be viewed
within SPRINT’s Graphical User Interface (GUI) or directly exported in DOT
format to a graph drawing tool of choice (e.g. [6]).

The algorithm underlying SPRINT is best described as a two-step process (see
[2, 3] for details and Section 3 for an example). Starting with a ploidy
profile $\vec{m}_{1}=\vec{m}$, the algorithm first constructs a simplification
sequence $\sigma(\vec{m})$ for $\vec{m}$. This is a sequence of ploidy
profiles $m_{i}$, $i\geq 1$, where $\vec{m}_{i+1}$ is obtained from
$\vec{m}_{i}$ by repeatedly taking the difference $\alpha=\mu_{1}-\mu_{2}$
between the largest ploidy level $\mu_{1}$ in $\vec{m_{i}}$ and the second
largest ploidy level $\mu_{2}$ in $\vec{m}_{i}$ until a ploidy profile of the
form $(k)$ or $(k,1,\ldots,1)$, $k$ a positive integer, is obtained. This
difference retains key properties of $\vec{m}$ and the following three cases
are distinguished : A) $\alpha=0$, B) $\alpha>\mu_{2}$, or C)
$\alpha\leq\mu_{2}$. The second step is a traceback of $\sigma(\vec{m})$ which
applies at each step a certain network operation that in some sense undoes the
corresponding case in the construction of $\sigma(\vec{m})$. By construction,
the resulting network realizes the corresponding ploidy profile in
$\sigma(\vec{m})$ at each traceback step. These operations are related to the
so called cherry reduction operations for networks introduced in [7] (see also
[3]).

SPRINT is guaranteed to find an optimal phylogenetic network that realizes a
plody profile $\vec{m}$ if the following two rules are not violated: (i) the
traceback is initialized with an optimal network for the terminal element of
$\sigma(\vec{m})$ (see [1] for an example of a non-optimal initialisation for
the ploidy profile (47)); (ii) the traceback step that corresponds to
operation B must not be used twice in a row. If it has been applied twice in a
row, SPRINT terminates and outputs the message ”no optimal network found”.

Two methods are implemented in SPRINT to initialize the construction of the
aimed for optimal network, namely the Binary representation and Prime factor
decomposition methods [2]. By default, the network that requires the minimum
number of polyploidization events is chosen (prime factor decomposition is
chosen in the case of a tie). However, a choice between these two methods is
permitted by SPRINT, as well as the ability to add a custom initialisation
network should it be favoured e.g. in the case of the ploidy profile (47) –
see above.

##  3\. Example

Consider the ploidy profile $\vec{m}=(18,14,14,8,8,8,$ $4,4,4,4,4,2)$ where
the components give the ploidy level of the species in the (ordered) Viola
data set $\\{La,Gr,Tr,$ $Bl,p933,p721,Ma,Gl,Vi,Ve,Re,Ru\\}$ considered in [2].
So, for example, species $La$ indexes the first component of $\vec{m}$ and so
has ploidy level 18. Then a screenshot for how to input this dataset into
SPRINT’s GUI is presented in Fig. 1. A screenshot of SPTINT’s output GUI that
shows the network generated for $\vec{m}$ as well as $\sigma(\vec{m})$ and the
required number of polyploidization events is presented in Fig. 2. It should
be noted that the network is optimal in this case (see [2]).

Figure 1. SPRINT’s input GUI (including guidance) for the ploidy profile
$\vec{m}$ in Section 3.

##  4\. Implementation and Testing

SPRINT is an open source Python software tool which works on any platform
where Python 3.9 or above and the NetworkX package is installed. SPRINT comes
with a GUI for ease of data entry and presentation of output. Data entry via a
.txt file using SPRINT’s file selection option is also offered. The color and
size of vertices in the generated network are adjustable and all graphics can
be saved directly in .png format for use in publications. Although the authors
suggest using pygraphviz for the DOT file and visualisation of output
(default), an alternative is provided in the form of SPRINT-nxspring.py. In
addition to SPRINT, a manual, and test files may be found on SPRINT’s GitHub
site. SPRINT was rigorously tested with ploidy profiles containing over 50
randomly chosen ploidy levels ranging from 1 to 30 which it completed in under
5 seconds each time.

Figure 2. SPRINT’s output GUI for the ploidy profile $\vec{m}$ from Section 3.
From top to bottom, the generated netwok, the ploidy profiles in
$\sigma(\vec{m})$ as well as the total number of polyploidization events
required by the network.

##  5\. Discussion

We have developed the network reconstruction tool SPRINT which is written in
Python 3.9. SPRINT uses a heuristic approach and aims to compute a network
that requires the minimum number of polyploidization events to explain a
dataset’s polyploid profile. SPRINT’s output can either be directly cut and
paste into a publication or can be input into other network drawing programs
(e.g. [6]). Future work might include combining SPRINT with existing programs
such as those in [4, 5] to help, for example, inform the generation of multi-
labelled trees or measure how different networks are that realize the same
ploidy profile.

## Acknowledgements and Funding

Conflict of interest: none declared. Funding: not applicable.

## References

  * [1] www.uea.ac.uk/web/groups-and-centres/computational-biology/software/sprint.
  * [2] K. T. Huber and L. J. Maher.  The hybrid number of a ploidy profile.  Journal of Mathematical Biology, 85, 2022.
  * [3] K.T. Huber and L.J. Maher.  Autopolyploidy, allopolyploidy, and phylogenetic networks with horizontal arcs.  submitted, 2022.
  * [4] K.T. Huber, B. Oxelman, M. Lott, and V. Moulton.  Reconstructing the evolutionary history of polyploids from multilabeled trees.  Molecular Biology and Evolution, 23:1784–1791, 2006.
  * [5] M. Lott, A. Spillner, K.T. Huber, A. Petri, B. Oxelman, and V. Moulton.  Inferring polyploid phylogenies from multiply-labeled gene trees.  BMC Evolutionary Biology, 9:216, 2009.
  * [6] P. Shannon, A Markiel, O Ozier, N.S. Baliga, J. T. Wang, D. Ramage, N. Amin, B. Schwikowski, and T. Ideker.  Cytoscape: a software environment for integrated models of biomolecular interaction networks.  Genome Research, 13:2498–504, 2003.
  * [7] L. van Iersel, R. Janssen, M. Jones, Y. Murakami, and N. Zeh.  A unifying characterization of tree-based networks and orchard networks using cherry covers.  Advances in Applied Mathematics, 129:102222, 2021.