11institutetext: College of Computer Science and Technology, Nanjing
University of Aeronautics and Astronautics, Nanjing, China
11email<EMAIL_ADDRESS>22institutetext: School of Computing and
Mathematical Sciences, University of Leicester, Leicester LE1 7RH, UK

# Attention Guided Network for Salient Object Detection in Optical Remote
Sensing Images††thanks: This work is supported in part by the Fundamental
Research Funds for the Central Universities of China under Grant NZ2019009.

Yuhan Lin 11 Han Sun 11 Ningzhong Liu 11 Yetong Bian 11 Jun Cen 11 Huiyu Zhou
22

###### Abstract

Due to the extreme complexity of scale and shape as well as the uncertainty of
the predicted location, salient object detection in optical remote sensing
images (RSI-SOD) is a very difficult task. The existing SOD methods can
satisfy the detection performance for natural scene images, but they are not
well adapted to RSI-SOD due to the above-mentioned image characteristics in
remote sensing images. In this paper, we propose a novel Attention Guided
Network (AGNet) for SOD in optical RSIs, including position enhancement stage
and detail refinement stage. Specifically, the position enhancement stage
consists of a semantic attention module and a contextual attention module to
accurately describe the approximate location of salient objects. The detail
refinement stage uses the proposed self-refinement module to progressively
refine the predicted results under the guidance of attention and reverse
attention. In addition, the hybrid loss is applied to supervise the training
of the network, which can improve the performance of the model from three
perspectives of pixel, region and statistics. Extensive experiments on two
popular benchmarks demonstrate that AGNet achieves competitive performance
compared to other state-of-the-art methods. The code will be available at
https://github.com/NuaaYH/AGNet.

###### Keywords:

Salient object detection Optical remote sensing images Position enhancement
stage Detail refinement stage.

##  1 Introduction

Salient object detection (SOD) is used to simulate the human visual attention
mechanism, which helps machine automatically search for the most attractive
areas or objects in an image.

In the early days, the traditional methods for SOD are based on hand-crafted
features. Such methods can meet the performance requirements of SOD tasks in
specific scenarios, which also have the advantages of fast speed and
convenient deployment. However, when application environment expands to a wide
variety of scenarios, traditional methods do not have satisfactory
generalization. In 2015, the SOD model was combined with deep learning
technique and convolutional neural network (CNN) for the first time, which
aroused the interest of researchers in the topic of CNN-based SOD methods.

At the present time, there has been a lot of excellent work on SOD tasks in
natural scene images (NSIs) such as GateNet [20], F3Net [14], LDF [15],
GCPANet [1]. However, as an emerging topic of sailency detection, the
researches on SOD in optical remote sensing images (RSIs) is still relatively
rare. Compared with NSIs, remote sensing images bring us more challenges.
Specifically, RSIs are usually taken from a high altitude with a bird’s-eye
view through equipments such as aerial cameras or satellites, thus the
location of salient objects may appear in various places in the frame.
Meanwhile, various scales, complex shapes and indistinct boundaries are the
unique features of objects in RSIs, which also hinder the accurate prediction
of object details. Since the RSI-SOD datasets were constructed and opened,
there have been some promising attempts to solve SOD tasks on remote sensing
images, e.g, DAFNet [19], EMFINet [22], SARNet [5]. But none of them can solve
the above-mentioned problems well, which includes the optimization of object
positioning and object details.

As another huge vision task topic, attention mechanism and salient object
detection have many similarities. Over the years, the attention mechanism has
been widely used in SOD models and proved to be an extremely effective
solution. In this paper, we aim at designing an attention guided network for
accurate salient object detection in optical RSIs, called AGNet. To obtain
accurate location prediction for salient objects, we propose a position
enhancement stage including a semantic attention module and a contextual
attention module, which provide the network with attention-guiding information
from global semantic and local context perspectives, respectively. In order to
obtain a more refined and complete saliency map, a self-refinement module is
designed to build the detail refinement stage. The self-refinement module
employs attention and reverse attention mechanisms to mine object details in
high-confidence and low-confidence regions, thereby gradually improving the
prediction results. At last, we evaluate the effectiveness of the proposed
network.

##  2 Related Work

###  2.1 Attention Mechanism for SOD

In recent years, the attention mechanism is utilized to promote the
performance of SOD models to a higher level. Zhao et al. [20] proposes a
simple gate network to filter the noise in the encoder feature map and
consider the difference in contributions from different encoder blocks. Wu et
al. [16] designs a holistic attention module to expand the coverage of the
initial saliency map, which helps to segment the entire salient object and
optimize the boundaries. Yang et al. [18] designs a branch-wise attention
module to adaptively aggregate multi-scale features so that salient objects
can be efficiently localized and detected. Tang et al. [10] proposes a novel
non-local cross-level attention, which can capture long-range feature
dependencies to enhance the discriminative ability of complete salient
objects. And Wang et al. [13] presents a pyramid attention structure to enable
the network to pay more attention to saliency regions while exploiting multi-
scale saliency information.

###  2.2 Salient Object Detection for RSIs

As we know, the RSI-SOD methods based on deep learning have become the main
research idea since 2019. Zhang et al. [19] proposes a dense attention flow
network, in which attention information flows from shallow layers to deep
layers to guide the generation of high-level feature maps. Cong et al. [2]
presents a graph structure-based reasoning module and a parallel multi-scale
attention module, which greatly improve the accuracy and completeness of the
saliency map. Zhou et al. [22] uses three encoders with different input scales
to extract and fuse multi-scale features, and then introduces the guidance of
edge information to obtain high-quality saliency maps. Huang et al. [5]
integrates multiple high-level features to locate the object position and feed
back the high-level features into the stage of shallow feature fusion. And in
[11], a joint learning scheme based on bidirectional feature transformation is
proposed to simultaneously optimize the boundaries and regions of salient
objects.

In general, since the above deep learning-based RSI-SOD methods have made
great progress, these solutions still show unsatisfactory performance when
dealing with some difficult samples in Fig. 6. In our AGNet, we design an
intuitively more natural method according to the image features of RSIs, which
is implemented by position enhancement stage and detail refinement stage.

##  3 Approach

In this section, we will specify the details of the proposed network. The
overall framework of AGNet is given in Fig. 1. From the figure, it is clear
that the inference process of the proposed network consists of three main
parts: feature encoding backbone, position enhancement stage and detail
refinement stage. In the following, the above three components will be
discussed in turn. Finally, we will also present the design of the used hybrid
loss.

Figure 1: The architecture of AGNet. CBR refers to the combination of
convolutional layer, batch nomalization operation, and activation function.
SAM means semantic attention module, CAM denotes contextual attention module,
and SRM refers to self-refinement module.

###  3.1 Feature Encoding Backbone

In this paper, we adopt the widely used Res2Net-50 [3] as the feature encoding
backbone of AGNet. To reduce the computational complexity and the overall
number of parameters in our network, a ${1\times 1}$ convolutional layer is
used to perform channel downscaling on the five convolutional block outputs of
Res2Net. At last, all the five groups of feature maps are converted to 128
channels and ready for the next processing step. This process can be
formulated as:

$F_{i}=relu(bn(conv_{1\times 1}(f_{i}))),i=1...5$ (1)

in which $conv_{1\times 1}$ denotes the ${1\times 1}$ convolution operation,
$bn$ refers to batch normalization operation, and $relu$ denotes the ReLU
activation function. These series of operations will be represented by $CBR$
in this paper.

###  3.2 Position Enhancement Stage

Inspired by the coarse-to-fine strategy, we propose a position enhancement
stage to locate potentially salient object position in the deep feature map.
Specifically, the semantic attention module (SAM) is proposed to model
semantic relations between different channels on the fourth layer feature map
and the contextual attention module (CAM) is designed to estimate the accurate
location information of salient objects on the last layer.

####  3.2.1 Semantic Attention Module.

SENet [4] is the most widely used method for calculating channel attention.
However, we think that the dimensionality reduction of channels in SENet will
have the side effect of making some semantic information lost. Also, capturing
the dependencies between all channels is inefficient and unnecessary [12].
Therefore, a novel semantic attention module without dimensionality reduction
is designed in Fig. 2, which can be efficiently implemented via one-
dimensional group convolution.

Figure 2: The architecture of SAM.

Considering that the fourth layer feature map already has a large enough
receptive field to describe the semantic information, we use this feature map
as input to SAM. First, global average pooling and global maximum pooling are
used on the feature map to obtain a global semantic representation of the
features, denoted as $v_{a}$ and $v_{m}$. Then, we design a unique method for
computing channel attention based on group convolution. Specifically, the
above two channel vectors are used to compute local semantic correlations by
two one-dimensional group convolutional layers. Each one-dimensional
convolution kernel is computed with adjacent channel features, and thus the
convolution results can reflect local cross-channel semantic interactions. In
this paper, SAM is able to provide the maximum contribution to the network
when we set the number of groups to 8. The semantic attention can be
formulated as:

$att=\sigma(gconv_{1\times 1}(avgpool(f_{4}))+gconv_{1\times
1}(maxpool(f_{4})))$ (2)

where $\sigma$ denotes the sigmoid activation function, $gconv_{1\times 1}$
denotes the two ${1\times 1}$ group convolutional layers, $avepool$ and
$maxpool$ are the average-pooling and max-pooling in spatial dimension,
respectively.

####  3.2.2 Contextual Attention Module.

In RSIs, the location of the salient object is more randomly distributed than
in NSIs, such as the center of the view or different corners of the image.
Therefore, the contextual attention module is proposed to guide the network to
provide the most accurate object position in the deepest layer.

As shown in Fig. 3, unlike previous spatial attention, our CAM calculates the
saliency of the location from the channel dimension and the local spatial
dimension, respectively. Specifically, one branch performs maximum pooling and
average pooling on the input in the channel dimension, and then cascades the
two single-channel feature maps to generate a spatial attention map by
convolutional fusion. The other branch computes an attention map directly on
the input using ${3\times 3}$ convolution, which reflects the influence of
neighboring pixels on the saliency of a location in the local spatial
dimension. The contextual attention can be formulated as:

$att1=\sigma(conv_{3\times 3}(cat(max(f_{5}),avg(f_{5}))))$ (3)
$att2=\sigma(conv_{3\times 3}(f_{5}))$ (4)

in which $avg$ and $max$ are the average-pooling and max-pooling in channel
dimension, $cat$ denotes feature concatenation along channel axis.

Figure 3: The architecture of CAM.

###  3.3 Detail Refinement Stage

As shown in most researches, the shallow features of the network often tend to
provide some additional information in terms of local details to the final
prediction results. In this section, we will introduce the most critical
component of the detail refinement stage, the self-refinement module (SRM), as
shown in Fig. 4.

Specifically, the input to each SRM are the fused feature map, which are
obtained from the deep feature of the previous layer and shallow feature of
the corresponding layer. Subsequently, the convolution operation is used to
compute the attention map reflecting the salient regions. At the same time, we
invert this attention map to obtain the reverse attention map, which reflects
some low-confidence regions ignored by the network. These two attention maps
are multiplied with the fused feature map to guide the network to further mine
the saliency information from the high-confidence salient regions and the low-
confidence background regions. Finally, the two results are summed and fused
by the convolutional layer to obtain the output of the SRM. This process can
be formulated as:

$att=\sigma(conv_{1\times 1}(f)),att\\_r=1-att$ (5)
$F=cbr(cbr(att*f)+cbr(att\\_r*f))$ (6)

At last, to fully utilize the effect of SRM, the results of the three SRM
modules are integrated to obtain the final prediction map of the network by
concatenation and convolution operations, as shown in Fig. 1.

Figure 4: The architecture of SRM.

###  3.4 Hybrid Loss of AGNet

In this paper, the widely used cross-entropy loss, IoU loss and recent F-value
loss are introduced into hybrid loss to supervise the training of the network
from the perspectives of pixels, regions, statistics. Furthermore, edge
information is also added to the loss function. In the end, the total loss of
the network can be described by the following equation:

$loss=loss_{bce}+\lambda\cdot loss_{iou}+loss_{f}$ (7)
$Loss=loss(P,G_{p})+\mu\cdot loss(E,G_{e})$ (8)

where $\lambda$ and $\mu$ are hyperparameters that balance the contributions
of the three losses and the contributions of the two supervised objects.
Empirically, we set $\lambda=0.6$ and $\mu=0.5$.

##  4 Experiment

###  4.1 Implementation Details

For the EORSSD [19] dataset and ORSSD [6] dataset, we expand their training
sets to 11200 and 4800 samples by random flipping and random rotating. We
train the proposed AGNet using the adam optimizer with initial learning rate
of 1e-4, weight decay of 5e-4, and batch size of 8. Cosine annealing decay
strategy is adopted to adjust the learning rate and the minimum learning rate
is set to 1e-5. Our network is totally trained for 60 epochs and the input
size is $224\times 224$. For a fair comparison with other methods, four
commonly used metrics including the mean absolute error (MAE), mean F-measure
(mF), S-measure (Sm) and mean E-measure (mE) are adopted to evaluate the model
performance as suggested in [5].

Figure 5: Comparison of predictions among different networks. (a) Image; (b)
Ground truth; (c) Baseline; (d) Baseline+PES; (e) Baseline+DRS; (f)
Baseline+PES+DRS.

###  4.2 Ablation Studies

To demonstrate the effectiveness of the proposed method with convincing data,
a series of ablation experiments are presented in this section. The same
experimental parameter settings are used throughout the experiment. The
qualitative comparison and quantitative comparison are shown in Fig. 5 and
Table 1. Specifically, Baseline denotes the U-shaped model with Res2Net-50 as
the backbone, PES means the position enhancement stage and DRS refers to the
detail refinement stage.

####  4.2.1 Effectiveness of PES.

From Table 1, it can be found that the addition of PES makes the performance
of the network have obvious improvements in three main metrics, such as a gain
of 0.83% in mF. The effectiveness of PES can be more thoroughly demonstrated
in Fig. 5. The fourth column of each row is the saliency map predicted by the
model combined with PES on the basis of the baseline. For example, the
pictures in the first and second rows in Fig. 5 are representative of
challenging scenes in the EORSSD dataset. The salient object in the image of
the first row is accompanied by interference information such as shadows,
while in the image of the second row, the color and texture features of the
river are extremely similar to its surroundings, which can easily guide the
network to output wrong results. Thanks to our proposed PES, the model has the
ability to remove the background interference and accurately locate the object
area, as shown in the third and fourth columns in Fig. 5.

####  4.2.2 Effectiveness of DRS.

To prove the effectiveness of our DRS, we additionally train with a
combination of baseline and DRS. The trained model shows the contribution of
DRS to the network in Table 1, such as a 0.5% improvement on mF. The third and
fourth rows of images in Fig. 5 are used to illustrate the effectiveness of
DRS. As can be seen from the fifth column of Fig. 5, the gaps between closely
spaced vehicles are more apparent compared to the baseline, while the shapes
of the vehicles are more fully highlighted. This change is because DRS
improves the prediction quality of the object details by the network. Our
method can refine and supplement the details of salient objects, which is
helpful for the application of RSI-SOD in earth observation, object counting,
etc.

Table 1: Quantitative evaluation of ablation studies on the EORSSD dataset. Methods | EORSSD
---|---
mF$\uparrow$ | mE$\uparrow$ | Sm$\uparrow$
Baseline | 0.8613 | 0.9554 | 0.9229
Baseline+PES | 0.8696 | 0.9599 | 0.9248
Baseline+DRS | 0.8663 | 0.9578 | 0.9247
Baseline+PES+DRS | 0.8736 | 0.9614 | 0.9284

###  4.3 Comparison with State-of-the-arts

To demonstrate the validity of our proposed network, we compare the proposed
method against other fourteen current state-of-the-art (SOTA) methods,
including eight SOD methods for NSIs (i.e., ITSD [21], LDF [15], MINet [9],
GCPANet [1], GateNet [20], F3Net [14], PA-KRN [17] and SUCA [8]), and six
recent SOD methods for optical RSIs (i.e., CorrNet [7], DAFNet [19], EMFINet
[22], LVNet [6], MJRBM [11] and SARNet [5]). All NSI-SOD methods are retrained
using the original code projects published by their authors. For RSI-SOD
methods, we use the saliency maps provided on the corresponding methods on
github. Table 2 and Fig. 6 show the comparison of the different methods on the
four metrics and visualization results, respectively.

Figure 6: Visual comparisons of different methods Table 2: Quantitative evaluation. The best three results are highlighted in red, blue, green. Methods | type | EORSSD | ORSSD
---|---|---|---
mF$\uparrow$ | MAE$\downarrow$ | Sm$\uparrow$ | mE$\uparrow$ | mF$\uparrow$ | MAE$\downarrow$ | Sm$\uparrow$ | mE$\uparrow$
ITSD | NSI | 0.8313 | 0.0109 | 0.9043 | 0.9265 | 0.8590 | 0.0172 | 0.9056 | 0.9410
LDF | NSI | 0.8317 | 0.0087 | 0.9078 | 0.9464 | 0.8779 | 0.0137 | 0.9143 | 0.9518
MINet | NSI | 0.8233 | 0.0097 | 0.9049 | 0.9190 | 0.8622 | 0.0153 | 0.9040 | 0.9355
GCPANet | NSI | 0.7968 | 0.0104 | 0.8855 | 0.8993 | 0.8488 | 0.0177 | 0.9005 | 0.9253
GateNet | NSI | 0.8314 | 0.0099 | 0.9090 | 0.9238 | 0.8769 | 0.0144 | 0.9192 | 0.9472
PA-KRN | NSI | 0.8434 | 0.0109 | 0.9194 | 0.9416 | 0.8824 | 0.0145 | 0.9243 | 0.9566
F3Net | NSI | 0.8164 | 0.0088 | 0.9030 | 0.9401 | 0.8677 | 0.0155 | 0.9160 | 0.9538
SUCA | NSI | 0.8040 | 0.0102 | 0.8971 | 0.9131 | 0.8347 | 0.0150 | 0.8992 | 0.9349
CorrNet | RSI | 0.8690 | 0.0087 | 0.9297 | 0.9512 | 0.9077 | 0.0107 | 0.9405 | 0.9687
DAFNet | RSI | 0.8135 | 0.0060 | 0.9175 | 0.9295 | 0.8578 | 0.0111 | 0.9186 | 0.9496
EMFINet | RSI | 0.8601 | 0.0079 | 0.9307 | 0.9445 | 0.9083 | 0.0104 | 0.9435 | 0.9662
LVNet | RSI | 0.7488 | 0.0146 | 0.8639 | 0.8729 | 0.8107 | 0.0211 | 0.8807 | 0.9204
MJRBM | RSI | 0.8162 | 0.0105 | 0.9068 | 0.9087 | 0.8679 | 0.0151 | 0.9190 | 0.9338
SARNet | RSI | 0.8625 | 0.0091 | 0.9284 | 0.9482 | 0.8942 | 0.0105 | 0.9361 | 0.9596
AGNet | RSI | 0.8736 | 0.0069 | 0.9284 | 0.9614 | 0.9109 | 0.0093 | 0.9392 | 0.9707

####  4.3.1 Quantitative comparison.

In Table 2, we report the mF, mE, Sm, MAE of our method and fourteen other
methods on two RSI-SOD datasets. Compared with other RSI-SOD methods, our
method shows impressive performance. Specifically, on the EORSSD dataset,
AGNet is much higher than the second-ranked method in mF, mE, e.g., mF: 0.8736
(AGNet) v.s. 0.8690 (CorrNet), and mE: 0.9614 (AGNet) v.s. 0.9512(CorrNet).
Meanwhile, on MAE, our method is only slightly weaker than DAFNet by a gap of
0.0009. While on the ORSSD dataset, our method ranks first on mF, mE and MAE.
Although our method fails to achieve the best score on both datasets on Sm,
our method outperforms by achieving more impressive performance with smaller
parameters, e.g., param: 26.60M (AGNet) vs 107.26M (EMFINet). And our AGNet
also beats SARNet with the same scale of parameters, the former has about 26M
and the latter has 40M. In competition with all NSI-SOD methods, our model and
most RSI-SOD methods are a lot ahead of them, which further demonstrates the
necessity of specially designing the model for the SOD of optical RSIs.

####  4.3.2 Qualitative comparison.

In Fig. 6, we show the saliency maps for the more difficult test samples. From
the visual analysis of the saliency maps in the first to third rows, AGNet
outperforms other methods when dealing with multi-scale objects, which is
exactly the benefit of a powerful multi-scale feature extractor and multi-
layer feature fusion strategy. The images in the fourth to sixth rows involve
multiple small salient objects, which is a challenging problem in the RSI-SOD
task. Our method can detect all salient objects and get refined segmentation
results. For example, there are seven circular buildings in the image in the
sixth row, all of which are clearly marked by our method with the help of PES.
But other methods can only detect a part of the buildings, or make objects
blend together, such as GateNet and DAFNet. In the last two rows in Fig. 6, we
present the performance of all methods when encountering slender or complex-
structured objects. For the ring-shaped buildings in the seventh row, our
method provides the most complete and clear detection results due to the
effect of DRS. For the two lands surrounded by the river in the eighth row,
other methods either conflate them with the river or only show a hole in the
saliency map, while our method successfully recognizes the two lands as the
background, which further confirms the importance of self-refinement for the
RSI-SOD task.

##  5 Conclusion

In this paper, a novel attention-guided network is proposed to solve the RSI-
SOD task. To enhance the detection quality of salient object position, a
semantic attention module and a contextual attention module are jointly used
to guide the network to detect coarse but accurate object locations. In order
to make the details of the object more refined and complete, a self-refinement
module is designed to refine the feature map from high-confidence and low-
confidence regions. Finally, an efficient hybrid loss is employed in the
training of the network to boost performance. Experimental results demonstrate
the robustness of our method in dealing with multi-scale objects, multiple
small objects, and objects with complex structures. In the future, we will try
to design a high-precision and lightweight network structure to further
promote the application of the RSI-SOD model in real life.

## References

  * [1] Chen, Z., Xu, Q., Cong, R., Huang, Q.: Global context-aware progressive aggregation network for salient object detection. In: Proceedings of the AAAI conference on artificial intelligence. vol. 34, pp. 10599–10606 (2020)
  * [2] Cong, R., Zhang, Y., Fang, L., Li, J., Zhang, C., Zhao, Y., Kwong, S.: Rrnet: Relational reasoning network with parallel multi-scale attention for salient object detection in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing (2021)
  * [3] Gao, S.H., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.: Res2net: A new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence 43(2), 652–662 (2019)
  * [4] Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7132–7141 (2018)
  * [5] Huang, Z., Chen, H., Liu, B., Wang, Z.: Semantic-guided attention refinement network for salient object detection in optical remote sensing images. Remote Sensing 13(11), 2163 (2021)
  * [6] Li, C., Cong, R., Hou, J., Zhang, S., Qian, Y., Kwong, S.: Nested network with two-stream pyramid for salient object detection in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing 57(11), 9156–9166 (2019)
  * [7] Li, G., Liu, Z., Bai, Z., Lin, W., Ling, H.: Lightweight salient object detection in optical remote sensing images via feature correlation. IEEE Transactions on Geoscience and Remote Sensing (2022)
  * [8] Li, J., Pan, Z., Liu, Q., Wang, Z.: Stacked u-shape network with channel-wise attention for salient object detection. IEEE Transactions on Multimedia 23, 1397–1409 (2020)
  * [9] Pang, Y., Zhao, X., Zhang, L., Lu, H.: Multi-scale interactive network for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 9413–9422 (2020)
  * [10] Tang, L., Li, B.: Class: Cross-level attention and supervision for salient objects detection. In: Proceedings of the Asian Conference on Computer Vision (2020)
  * [11] Tu, Z., Wang, C., Li, C., Fan, M., Zhao, H., Luo, B.: Orsi salient object detection via multiscale joint region and boundary model. IEEE Transactions on Geoscience and Remote Sensing (2021)
  * [12] Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q.: Eca-net: Efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 11531–11539 (2020)
  * [13] Wang, W., Zhao, S., Shen, J., Hoi, S.C., Borji, A.: Salient object detection with pyramid attention and salient edges. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1448–1457 (2019)
  * [14] Wei, J., Wang, S., Huang, Q.: F3net: fusion, feedback and focus for salient object detection. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 34, pp. 12321–12328 (2020)
  * [15] Wei, J., Wang, S., Wu, Z., Su, C., Huang, Q., Tian, Q.: Label decoupling framework for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 13025–13034 (2020)
  * [16] Wu, Z., Su, L., Huang, Q.: Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 3907–3916 (2019)
  * [17] Xu, B., Liang, H., Liang, R., Chen, P.: Locate globally, segment locally: A progressive architecture with knowledge review network for salient object detection. In: Proceedings. of the AAAI Conference On Artificial Intelligence. pp. 1–9 (2021)
  * [18] Yang, S., Lin, W., Lin, G., Jiang, Q., Liu, Z.: Progressive self-guided loss for salient object detection. IEEE Transactions on Image Processing 30, 8426–8438 (2021)
  * [19] Zhang, Q., Cong, R., Li, C., Cheng, M.M., Fang, Y., Cao, X., Zhao, Y., Kwong, S.: Dense attention fluid network for salient object detection in optical remote sensing images. IEEE Transactions on Image Processing 30, 1305–1317 (2020)
  * [20] Zhao, X., Pang, Y., Zhang, L., Lu, H., Zhang, L.: Suppress and balance: A simple gated network for salient object detection. In: European conference on computer vision. pp. 35–51. Springer (2020)
  * [21] Zhou, H., Xie, X., Lai, J.H., Chen, Z., Yang, L.: Interactive two-stream decoder for accurate and fast saliency detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9141–9150 (2020)
  * [22] Zhou, X., Shen, K., Liu, Z., Gong, C., Zhang, J., Yan, C.: Edge-aware multiscale feature integration network for salient object detection in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing (2021)


# FedToken: Tokenized Incentives for Data Contribution in Federated Learning

Shashi Raj Pandey1, Lam Duc Nguyen2, and Petar Popovski1
1 Department of Electronic Systems, Aalborg University, Aalborg 9220, Denmark.
2 Software Systems Research Group, CSIRO Data61 Eveleigh, NSW 2016
{srp<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Incentives that compensate for the involved costs in the decentralized
training of a Federated Learning (FL) model act as a key stimulus for clients’
long-term participation. However, it is challenging to convince clients for
quality participation in FL due to the absence of: (i) full information on the
client’s data quality and properties; (ii) the value of client’s data
contributions; and (iii) the trusted mechanism for monetary incentive offers.
This often leads to poor efficiency in training and communication. While
several works focus on strategic incentive designs and client selection to
overcome this problem, there is a major knowledge gap in terms of an overall
design tailored to the foreseen digital economy, including Web 3.0, while
simultaneously meeting the learning objectives. To address this gap, we
propose a contribution-based tokenized incentive scheme, namely FedToken,
backed by blockchain technology that ensures fair allocation of tokens amongst
the clients that corresponds to the valuation of their data during model
training. Leveraging the engineered Shapley-based scheme, we first approximate
the contribution of local models during model aggregation, then strategically
schedule clients lowering the communication rounds for convergence and anchor
ways to allocate _affordable_ tokens under a constrained monetary budget.
Extensive simulations demonstrate the efficacy of our proposed method.

##  1 Introduction

###  1.1 Background and Motivation

The next generation of internet architecture is said to be backed by Web 3.0,
a step further toward a fully decentralized online ecosystem powered by
blockchain technologies [1, 2, 3, 4]. This novel connectivity paradigm, though
still in its infancy, opens up unique ways to monetize decentralized
participation and allows clients of heterogeneous profiles to engage in the
secure exchange of data, assets, and trust. Recent paradigm of privacy-
preserving decentralized training of machine learning models, such as
Federated Learning (FL) [5, 6, 7], is another complementary enabler that is
already offering various applications on top of the existing network
architecture via distributed computing collaboration on private, untapped
data. Therein, it is important to facilitate appropriate participation for
distributed training of FL models to support machine learning applications of
interest, for instance, in healthcare, automation, logistics and much more [6,
8, 9]. An efficient incentive design is thus a fundamental stimulus for
participation in a collaborative operation over a decentralized network
infrastructure.

On that pretext, we begin with a natural question on incentive design to
enable FL: How to ensure quality participation with contribution-based
incentives that meet the objective of communication efficiency simultaneously?
This leads to investigating: _(i)_ ways to quantify the expected contribution
of clients’ shared model parameters in improving the model performance during
aggregation, given constraints on the offered monetary budget; and _(ii)_
transparent distribution of incentives as per the contribution, both, in terms
of participation and the contribution made in the FL training. Related works
(cf. Section-2) do not cover these issues jointly. In this work, we propose an
overall design tailored to the foreseen digital economy that simultaneously
meets the learning objectives in terms of training as well as communication
efficiency. We propose a _contribution-based tokenized incentive scheme_
backed by blockchain technology. It ensures a fair allocation of tokens
amongst the clients as per the value of contribution with their shared local
model parameters during model training.

Our main contributions are as follows:

  * •

Tokenized Incentives: We develop tokenized incentives
FedToken111https://github.com/lamnd09/FedToken. for clients to train the FL
learning models under budget constraints, reflected with _quota_ , at the
aggregator. We propose a contribution accounting mechanism based on
blockchain. The token distribution algorithm is designed to attribute the
value of participation in the model aggregation period.

  * •

Robust System Design and Improved Performance: The token allocation scheme
accounts for the objective of communication efficiency, achieved as a direct
consequence of the strategic selection of client’s updates during model
aggregation in FL. The rewards for participation are shared fairly: we use
temporal discounting amongst the clients that transmitted low-quality model
updates. This discourages _free-riders_. We incentivize quality updates in a
way that is robust to any choice of the local learning algorithm.

  * •

Implementation Features and Findings: The activities regarding the exchange
and allocation of tokens are immutably recorded into the Blockchain. The
contributors can query the relevant information, such as account balance,
receiving tokens, or missing contributions. This is the trusted platform that
ensures quality participation in federated learning.

##  2 Related Work

Incentive Mechanism for FL. Incentive designs to enable efficient and
effective participation of clients in training FL models have been a key focus
in recent works [10, 11, 12, 2].In line to this, one fundamental research
question on incentive design for FL is the value of _data contribution_. Most
incentive mechanism design for FL was investigated dominantly in the wireless
regime so as to enable participation while optimizing network resources [11,
10, 13, 12], rather than accounting for the impact of individual data
contributions.

The authors in [14] showed a contribution-index based on an approximation to
the Shapley value [15, 16] to evaluate data value in FL. The authors in [17]
presented a novel multidimensional incentive framework for FL. The authors in
[18] proposed a game-based incentive mechanism for an FL platform that
combines distributed deep learning and crowdsensing technique for big data
analytics on mobile clients. In [19], the authors propose a novel incentive
governor for an efficient FL in highly heterogeneous and dynamic scenarios
considering contribution and reputation. However, these works primarily have
two issues. First, the scoring mechanism is too subjective and lacks quality
evaluation schemes. Second, each participant only has a score and can be
easily affected by malicious raters. Therefore, in the absence of a trusted
mechanism for recording the offered incentives, the solution is not viable for
practical usage. Furthermore, despite a plethora of incentive designs to
enable FL, it has not been formalized well in the literature, _what exactly
are these incentives, anyway_?

Incentive Mechanism in Blockchained FL. Feasibility study on the integration
of FL with blockchain is explored in several literature [20, 21, 22, 23]. One
key benefit is blockchain provides a trustworthy platform to realize
_tokenization_ , which is a process of creating and managing tokens by
recording the verifiable title transfer from one user to another. _Tokens_ ,
in principle, then could quantify offered incentives in FL. Native tokens on a
blockchain (e.g., BTC[24] on Bitcoin, ETC on Ethereum [25]) can be used to
formulate a value system where the tokens represent monetary value or physical
assets offered for quality participation in FL training. However, most of the
mentioned works are about latency analysis and scalability issues rather than
contribution assessment and token allocation for improving learning objectives
in FL.

The authors in [26] present a blockchain-based _reputation_ system to train FL
models. In [27], the (mining) rewards are allocated as per the local data size
and compute dominance, which doesn’t reflect a practical scenario. Moreover,
budget constraints are overlooked, let alone the learning performance. In line
with this, the authors in [28] provide a comprehensive survey on blockchain-
based FL, where the use of blockchain in FL is reviewed for security breaches
and honest participation of clients. The work [23], which is close to our
research, introduces a tokenized incentive mechanism where tokens are used as
a means of paying for the services that provide participants and the training
infrastructure. However, the approach of computing individual contributions is
different and for a separate context. In addition, it is intuitively to argue
that participation only cannot/shouldn’t ensure tokens. We address this and
consider _contributions_ in participation and allocate tokens without
compromising the learning objectives. We show that irrespective of
participation frequency, only quality updates are considered for the tokenized
incentives, and strategic allocation of tokens jointly improves training and
communication efficiency in FL.

##  3 Overall System Design

Consider a set of clients $[N]=\\{1,2,\ldots,N\\}$ and the training examples
held by each client $n\in[N]$ as a collection of $D_{n}$ input-output pairs
$\\{x_{i},y_{i}\\}_{i=1}^{D_{n}}$, where, respectively,
$x_{i}\in\mathbb{R}^{d}$ is an input and $y_{i}\in\mathbb{R}$. Each client $n$
exchanges updates on computations of their local data set $\mathcal{D}_{n}$
with the shared edge aggregator to collaboratively train a learning model. In
FL, this translates to finding an optimal global model parameter
$w\in\mathbb{R}^{d}$ that minimizes the empirical risk on all of the
distributed training data samples, i.e., the clients collaborate to solve the
following optimization problem

$\underset{w\in\mathbb{R}^{\textrm{d}}}{\text{min}}F(w)+\lambda g(w)\ \
\text{where}\ \ F(w):=\sum\nolimits_{n=1}^{N}\frac{D_{n}}{D}f_{n}(w),$ (1)

where $f_{n}(w)$ captures the cost of prediction with the model parameter $w$;
$\lambda$ is a regularizer; and $g(w)$ is a regularization function to control
model complexity and offer _generalization_. In practice, the training
examples $(x_{i},y_{i})$ are drawn from $\mathcal{D}_{n}$, and as such, the
total available training samples is $D=\sum\nolimits_{n=1}^{N}D_{n}$. Then, we
formally define
$f_{n}(w)=\mathbb{E}_{(x_{i},y_{i})\sim\mathcal{D}_{n}}[l(w;(x_{i},y_{i}))]$
as the expected loss. Here, $l(w;(x_{i},y_{i}))$ is the measuring loss
function for the prediction [7, 5] over data samples $(x_{i},y_{i}),\forall
i\in\mathcal{D}_{n}$.

######  Assumption 1 (General Assumptions).

The loss function $f_{n}:\mathbb{R}^{\textrm{d}}\rightarrow\mathbb{R}$ is
twice continuously differentiable, $L-$smooth and $\mu-$strongly convex, that
means, we have

$\mu I\preceq\nabla^{2}f_{n}(w)\preceq
LI,\forall{w}\in\mathbb{R}^{\textrm{d}},$

where $I\in\mathbb{R}^{\textrm{d}\times\textrm{d}}$ is the identity matrix and
$\nabla^{2}f_{n}(w)$ is the Hessian of $f_{n}$ in $w$, respectively.

Following Assumption 1, we solve the learning problem (1) in its dual form as
a particular case of Fenchel duality [29, 10]. The separation of the global
problem (1) as primal-dual updates, on the one hand, allows clients to control
their local computations in improving the quality of local updates and, on the
other hand, facilitates the aggregator to evaluate contributions made by the
clients to perform efficient incentives allocation during model training.
Then, (1) in its dual form considering the availability of overall $D$ data
samples, using $\alpha\in\mathbb{R}^{D}$ as the dual variable mapping to the
primal candidate vector and $\phi(\alpha)=\frac{1}{\lambda D}X\alpha$, is
defined as

$\underset{\alpha\in\mathbb{R}^{D}}{\text{max}}\boldsymbol{\mathcal{F}}(\alpha):=\frac{1}{D}\sum\nolimits_{n=1}^{D}-f_{n}^{*}(-\alpha_{n})-\lambda
g^{*}(\phi(\alpha)),$ (2)

where $f_{n}^{*}$ and $g^{*}$ are the convex conjugates of $f_{n}$ and $g$,
respectively, and $X\in\mathbb{R}^{d\times D}$ is a matrix with each column
having data points. Then, we get the mapping between the optimal value of dual
variable $\alpha^{*}$ in (2), and the optimal solution of (1) following the
first-order optimality conditions [29] as $w(\alpha^{*})=\nabla
g^{*}(\phi(\alpha^{*}))$. Using a shorthand notation for
$\phi\in\mathbb{R}^{d}$ as $\phi{(\alpha)}$, in the following, we outline the
primal-dual solution abusing distributed computation of clients in a
systematic manner.

###  3.1 Primal-dual solution approach

We resort solving (2) in a distributed manner, as shown in [29, 30, 31], with
the iterative exchange of local dual variables $\alpha_{[n]}$, $\varrho_{[n]}$
obtained at each client $n\in[N]$. Note that both $\alpha_{[n]}$,
$\varrho_{[n]}$ are vectors in $\mathbb{R}^{D}$ with non-zero elements for the
residing local data points, i.e., for instance,
$(\alpha_{[n]})_{i}=\alpha_{i}$ if $i\in\mathcal{D}_{n}$ and $0$ otherwise.
The selected clients iterates over their local training examples to output
$\alpha_{[n]}$, $\varrho_{[n]}$. Formally, in one single global communication
round $t\in\\{1,2,\ldots,\tau\\}$, i.e., the interaction between the edge
aggregator and the clients, each selected client $n$ solves the following dual
problem in the distributed setting:

$\underset{\varrho_{[n]}\in\mathbb{R}^{D}}{\text{max}}\boldsymbol{\mathcal{F}}_{n}(\varrho_{[n]};\phi,\alpha_{[n]}),$
(3)

where the weight vector $\varrho_{[n]}\in\mathbb{R}^{D}$ has elements zero for
the unavailable data points and reports value only for the coordinates
corresponding to local variables $\alpha_{[n]}$, and $0$ otherwise;
$\boldsymbol{\mathcal{F}}_{n}(\varrho_{[n]};\phi,\alpha_{[n]})=-\frac{1}{N}g^{*}(\phi(\alpha))-\langle\frac{1}{D}X^{T}_{[n]}\nabla
g^{*}(\phi(\alpha)),\varrho_{[n]}\rangle-\frac{\lambda}{2}\|\frac{1}{\lambda
D}X_{[n]}\varrho_{[n]}\|^{2}-\frac{1}{D}\sum_{i\in\mathcal{D}_{n}}{f^{*}_{i}(-(\alpha_{[n]})_{i}-(\varrho_{[n]})_{i})}$
is a corresponding quadratic approximation of $\boldsymbol{\mathcal{F}}$ at
$(\alpha_{[n]}+\varrho_{[n]})$ with $X_{[n]}$ as a matrix with $i-$th column
having data points for $i\in\mathcal{D}_{n}$, i.e., $(X_{[n]})_{i}=(X)_{i}$,
and zero-padded otherwise. As mentioned, the selected devices invest local
computations in running stochastic gradient descent (SGD) on their local
training samples and later share the output, i.e., the obtained local
parameters $\Delta\phi^{t}_{[n]}:=\frac{1}{\lambda
D}X_{[n]}\varrho^{t}_{[n]}$, with the edge aggregator for model aggregation.
Meanwhile, the local variable gets updated in each global communication as
$\alpha_{[n]}^{t+1}=\alpha_{[n]}^{t}+\nu\varrho^{t}_{[n]}$, where
$\nu\in[0,1]$ is the tuneable local aggregation parameter often set between
$\frac{1}{N}$ and $1$.

###  3.2 Contribution-based Aggregation

Once getting the local updates from $N$ clients, a plain model aggregation
scheme is Federated Averaging (FedAvg) [5], i.e., we compute the average of
received local updates as

$\phi^{t+1}:=\phi^{t}+\frac{1}{N}\sum\nolimits_{n=1}^{N}\Delta\phi^{t}_{[n]},$
(4)

and pass $\phi^{t+1}$ back to the $[M]\subseteq[N]$ random clients to compute
(3), and iteratively as such, thereafter.

We, however, resort to the engineered, cost-efficient Shapley-based scheme
[32, 31] that allows approximation of the contribution of local models during
model aggregation to lower the overall communication overhead and design
tokenize incentives based on that. The edge aggregator performs $\delta$
random permutations over the received dual variables, where the average
marginal contribution is evaluated as

$u_{n}=\frac{1}{|M|!}\sum\nolimits_{p\in\mathcal{P}}\bigg{[}V(S_{n,p}\cup\\{n\\})-V(S_{n,p})\bigg{]}.$
(5)

Here, $p\in\mathcal{P}$ is the permutation of clients updates, i.e.,
$\Delta\phi^{t}_{[n]},\forall n\in[M]$, with a probability of
$\frac{1}{|M|!}$; $V(\cdot)$ is monotone quantifying performance score
evaluated on the test dataset with the permuted subset of client updates,
which translates to the improvement in global model accuracy and is measured
by $\epsilon$; $S_{n,p}$ is a set of preceding clients before $n$ for a
permutation $p$ selected for model aggregation – then, all possible orders of
clients contribute to the average contribution (5). Furthermore, we assume the
availability of test dataset at the aggregation server; having it as such is a
common assumption in FL [6, 33], particularly, for achieving model robustness
and a fair level of security guarantees. Then, formally, for a test dataset
$(x_{i},y_{i})\sim\mathcal{D}$,
$V(\mathcal{D})=V_{\textrm{ref}}-\frac{1}{|\mathcal{D}|}\sum\nolimits_{i=1}^{|\mathcal{D}|}l(w(\alpha);(x_{i},y_{i}))$
such that $\mathcal{F}(\alpha^{*})\leq l(;)\leq V_{\textrm{ref}}$ with
$\mathbb{E}\left[\boldsymbol{\mathcal{F}}(\alpha)-\boldsymbol{\mathcal{F}}(\alpha^{*})\right]<\epsilon$.
The method of evaluating the marginal contribution of the local models during
the aggregation period in (5) primarily ignores the constrained monetary
budget at the aggregator. Said the monetary budget can be quantified as
_tokens_ (see Definition 1), then, in principle, response to all $M$ queries
made by the aggregator, i.e., the local variables received from $M$
participating clients, cannot be incentivized with tokens except with a
compensation for their participation as participation reward.

Next, while keeping the learning objectives intact, this raises the following
two research questions : (i) how many updates can the aggregator afford with
the given budget, and (ii) how to fairly allocate the available budget given
only a subset of received local updates contribute in improving the model
performance? This also challenges finding comparable contributions,
discouraging free-riders, and handling possible attacks during model
aggregation. Therein, the tokens allocation scheme has to be _justified_ as
per the accounted marginal contribution of the local updates and the available
monetary constraints at the edge aggregator. Once done, the records of the
contribution accounting mechanism are kept on the Blockchain, hence, allowing
contributors to make queries about the relevant information during FL
training.

We formalize this with the following definition on the constrained monetary
budget at the aggregator.

######  Definition 1 (Quota).

We define quota $[\mathcal{Q}]\subseteq[M]$ as the affordable number of
enrolled model updates with the fixed monetary budget $\mathbb{B}$ that
satisfies the pre-defined global model accuracy $0\leq\epsilon\leq 1$ (i.e.,
$\mathbb{E}\left[\boldsymbol{\mathcal{F}}(\alpha)-\boldsymbol{\mathcal{F}}(\alpha^{*})\right]<\epsilon$)
upon the exchange of dual updates $\alpha$. Then, given $\mathbb{B}$ that
translates into tokens, we allocate tokens as per the proportion of
contribution amongst the chosen $[\mathcal{Q}]$. For simplicity, we assume
$\mathbb{B}$ equals to the number possible tokens.

Note that the dynamic adjustment of $|\mathcal{Q}|$ in relation to changing
the value of available budget $\mathbb{B}$ is left as a future work; here, we
focus our analysis on a fixed $\mathbb{B}$ and constrained quota. Then, with
Definition 1, we have the following proposition.

######  Proposition 1.

For a fixed $\mathbb{B}$, assuming the absence of any malicious updates, a
larger size of $[\mathcal{Q}]\subseteq[M]$ undervalues the client’s
contribution as it leads to a lower average marginal valuation of the client
local updates, and hence, affects the number of allocated tokens.

###### Proof Sketch.

The proof follows the properties of standard Shapley value [34] derived from
the concept of cooperative games while evaluating the marginal contributions
in (5). Take a specific permutation instance $p\in\mathcal{P}$, where the
score (or the payoffs) with predecessor set of client $n$, i.e.,
$S_{n,p}=\\{m\in[M]|p(m)<p(n)\\}$ is $V(S_{n,p})$. Then, following _Linearity_
and _Symmetry_ properties of $V(S_{n,p})$, we can generalize
$V(S_{n,p}\cup\\{n\\})\leq V(S_{n,p}\cup\\{n\\}\cup\\{m\\}),\forall
n,m\in[\mathcal{Q}]$. That means the average marginal contribution gets
lowered, whereas the equality holds only if the local updates from $\\{m\\}$
are invaluable (or malicious). ∎

In addition, with the increase in the rounds of permutation $\delta$ and fixed
an affordable number of available quotas $|\mathcal{Q}|$, the range of
$V(\mathcal{Q})$ changes; hence, affecting the quantity of allocated tokens.
Given the contributions are only _ranked_ in order,
$\Theta(\log|\mathcal{Q}|)$, is the best proportional fairness, and enjoys a
permutation complexity of $\mathcal{O}(M\log M)$ [32]. The details of the
overall operation are provided as a pseudo-code in Algorithm 1.

Algorithm 1 FedToken: Contribution-based Tokenized Incentives in FL

1: Initialize: Initial global model $\phi^{1}$, $[\mathcal{Q}]=\emptyset$,
$|\mathcal{Q}|$, $\delta$, learning parameters.

2: for all global iteration $t\in\\{1,2,\ldots,\tau\\}$ do

3: Select[M]: Select $[M]$ clients at random;

4: for all $m\in[M]$ do

5: Get: Dual variables $\Delta\phi^{t}_{[m]},\forall m$ solving (3);

6: end for

7: SetCounter: $c=0$;

8: while $c<\delta$ do

9: Set: $v^{c}_{0}=V(\emptyset)$;

10: for all $m\in[M]$ do

11: if $|V(\phi^{c})-v^{c}_{m-1}|<\epsilon$ then

12: Update: $v^{c}_{m}=v^{c}_{m-1}$;

13: else

14: Set: $v^{c}_{m}\leftarrow V(\\{p^{c}[1],\ldots,p^{c}[m]\\});$

15: end if

16: Update:
$u_{p^{c}[m]}\leftarrow[\frac{c-1}{c}]u_{p^{c-1}[m]}+\frac{1}{c}[v^{c}_{m}-v^{c}_{m-1}]$;

17: end for

18: UpdateCounter: $c=c+1$;

19: end while

20: Sort[M]: Sort clients in the descending order of $u_{p^{c}[m]}$;

21: Aggr[$\mathcal{Q}$]: Select $|\mathcal{Q}|$ clients updates for
aggregation;

22: for all $q\in[\mathcal{Q}]$ do

23: TokenAllocation;

24: end for

25: Update: global variable as $\phi^{t}$ (4);

26: end for

###  3.3 Tokenized Incentives for Data Contributions

With the primal-dual formalization of the global learning problem, the
participating clients can employ any arbitrary algorithm to solve the local
learning problem and trade the output dual variables with the edge aggregator.
Following the contribution-based aggregation scheme, the edge aggregator
efficiently allocates tokens as per its available budget opting for the
following simple token distribution schemes: (i) Proportional Fair (PF)
allocation, where the tokens are distributed as per the proportion of
contribution in improving the global model performance, (ii) Equal Pay (EP),
where all participating clients are allocated equal shares of available
tokens. Note that the proposed incentive design offers _set-aside_ tokens for
the initial participation. For $t\in\\{2,3,\ldots\tau\\}$, to eliminate the
free-riders problem, we multiply the initial allocated slack tokens with a
discount factor $\zeta^{t}<1$; hence, disincentivizing poor local updates.

##  4 Performance Evaluation

Dataset and Model. To validate the effectiveness of our FL design with
tokenomic mechanism, we conduct experiments with well-known CIFAR-10 datasets
[35], which covers $60000$ data samples, having $32\times 32$ colored images
in $10$ classes equally. There are 50000 training images and 10000 test
images. We leverage a convolutional neural network (CNN) model that is
maintained in a distributed manner through blockchain technology. The designed
network model possesses hidden layers for feature extraction and fully
connected layers for classification.

Implementation details. We follow [36] to replicate the non-i.i.d.
characteristics of dataset across clients. We set $N=100$ clients on a single
server with 26 core Intel Xeon 2.6 GHz, 256 GB RAM, 4 TB, Nvidia V100 GPU,
Ubuntu OS and assume that the clients and server are synchronized, and
evaluate the model performance by considering the impact of available quotas
as a selection strategy during the model aggregation. For training, each
client model is trained for $10$ epochs by SGD with a momentum 0.9, batch size
of $10$, and the learning rate $0.01$. Unless specified, we set the value of
$M=[0.1\times N]$, which exhibits a good balance between computational
efficiency and convergence rate, as demonstrated experimentally in [5]. The
default value of discounting factor $\zeta^{t}$ is set to 0.7.

Baseline. The overall system design considers strategic model aggregation for
improving the learning objective and distribution of tokens amongst clients
for training FL. In particular, the FedToken is built up based on the
contribution-based aggregation to enhance the quality of updates from FL
clients while reducing the communication overhead. Therefore, we evaluate the
performance of our proposed FedToken with comparable and competitive client
selection, and aggregation schemes FedAvg [5], and FedFomo [36], and further
present an analysis of two tokenized incentive schemes, as PF and EP. We
tokenized the contributions and reward the participating clients based on the
fixed monetary budget $\mathbb{B}$ and corresponding affordable quota
$|\mathcal{Q}|$ per training round.

(a) Impact of $|\mathcal{Q}|/|M|$ Ratio.

(b) Token rewards.

(c) Impact of $\delta$.

(d) Overhead.

(e) Varying $|\mathcal{Q}|$ for a fixed $\mathbb{B}$.

Figure 1: Performance Analysis of FedToken: a) impact of $|\mathcal{Q}|$
value, b) token Rewards, c) impact of $\delta=\\{1,2,3,4\\}$ on training
performance, d) communication overhead (in MB), and e) varying
$|\mathcal{Q}|$.

Results. Fig. 1(a) shows the impact of available quotas $|\mathcal{Q}|$ on the
training efficiency. We see the contribution-based model aggregation strategy
strengthens the performance of FedToken, as compared to FedAvg, and FedFomo.
Moreover, given a low availability of $|\mathcal{Q}|$, i.e., the monetary
constraints or the market for participation, the significance of including
quality local updates from FL clients is prominent and is observed to improve
the test accuracy up to $12.5\%$ of FL. In specific, when the ratio of
$|\mathcal{Q}|/|M|<0.8$, the FedToken achieves better accuracy and converges
faster than the baselines.

Next, in Fig.1(b), we provide an evaluation of tokenized incentives per round,
where we observe clients are encouraged to join and contribute quality local
updates in the training process for higher token offers – leading to
competitiveness in individual contributions. The figure compares the amount of
tokens received by a client under different scenarios (EP and PF). First, to
discourage free-riders and stale updates, as participation tokens are reserved
and granted, a discount factor $\zeta^{t}=0.7,\forall t$, is multiplied with
the allocated tokens. Second, FedToken can effectively detect data poisoning
attacks resulting in undesirable local updates, thereby not allocating any
participation tokens and saving the token budget for the rest. Fig. 1(c)
demonstrates the impact of random permutation in evaluating the value of local
updates in model aggregation. We observe a fundamental limit on imposing
larger local computations versus the number of permutations at the aggregator
for improving model performance. This means the aggregator can compensate the
costs of local computation with added $\delta$, revealing the scope for
incentive management.

In Fig. 1(d), we analyze the communication overhead of FedToken as compared
with the discussed baselines. The proposed approach is communication-efficient
and offers better convergence with a low overhead of approx. $72.76\%$ and
$29.47\%$ on average in comparison with FedAvg and FedFomo, respectively, due
to the strategic selection of local updates as the contribution abiding
available quotas. Finally, Fig. 1(e) evaluates the impact of budget constraint
$\mathbb{B}=1000$ tokens with the affordable quota $|\mathcal{Q}|$ on the
learning performance. To begin with, each participant is allocated with a
token in their balance and later discounted as per their contribution if not
selected during aggregation. We observe a relation between test accuracy and
the rounds of communication to exhaust the available tokens for different
values of quotas. In principle, strategic selection of clients update allows
optimal use of available tokens, as first argued in Fig. 1(a). This further
opens up several research opportunities, for instance, to design a
collaborative data trading market relating tokens and purchasing quotas.

##  5 Conclusion

In this work, we proposed tokenized incentives for clients contributing
quality updates to train an FL model in a communication-efficient manner. We
developed a holistic approach where token allocation under budget constraints,
formalized as quota, is based on the value of contribution during the model
aggregation phase, which later favors quality participation by virtue of an
efficient selection of local model parameters. The proposed tokenized
incentive design enables robust integration of clients with heterogeneous
profiles for collaborative FL training while discouraging poor updates and
attacks in the foreseen decentralized network architecture as Web 3.0.
Finally, we evaluated and analyzed the performance of the proposed approach
via extensive simulations.

##  6 Acknowledgements and Disclosure of Funding.

This work was supported by the Villum Investigator Grant “WATER” from the
Velux Foundation, Denmark.

## References

  * [1] T. Aste, P. Tasca, and T. Di Matteo, “Blockchain technologies: The foreseeable impact on society and industry,” 2017.
  * [2] L. D. Nguyen, S. R. Pandey, B. Soret, A. Broering, and P. Popovski, “A marketplace for trading ai models based on blockchain and incentives for iot data,” arXiv preprint arXiv:2112.02870, 2021.
  * [3] M. Javaid, A. Haleem, R. P. Singh, S. Khan, and R. Suman, “Blockchain technology applications for industry 4.0: A literature-based review,” Blockchain: Research and Applications, p. 100027, 2021.
  * [4] D. L. Nguyen, “Blockchain for internet of things: Data markets, learning, and sustainability,” 2022.
  * [5] B. McMahan et al., “Communication-efficient learning of deep networks from decentralized data,” in International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, USA, April 2017.
  * [6] P. Kairouz et al., “Advances and open problems in federated learning,” Foundations and Trends in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
  * [7] J. Konečnỳ, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527, 2016.
  * [8] D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, and H. V. Poor, “Federated learning for internet of things: A comprehensive survey,” IEEE Communications Surveys & Tutorials, vol. 23, no. 3, pp. 1622–1658, 2021\.
  * [9] J. Xu, B. S. Glicksberg, C. Su, P. Walker, J. Bian, and F. Wang, “Federated learning for healthcare informatics,” Journal of Healthcare Informatics Research, vol. 5, no. 1, pp. 1–19, 2021.
  * [10] S. R. Pandey et al., “A crowdsourcing framework for on-device federated learning,” IEEE Transactions on Wireless Communications, vol. 19, no. 5, pp. 3241–3256, Feb. 2020.
  * [11] T. H. T. Le, N. H. Tran, Y. K. Tun, M. N. Nguyen, S. R. Pandey, Z. Han, and C. S. Hong, “An incentive mechanism for federated learning in wireless cellular networks: An auction approach,” IEEE Transactions on Wireless Communications, vol. 20, no. 8, pp. 4874–4887, 2021.
  * [12] Y. Zhan, P. Li, Z. Qu, D. Zeng, and S. Guo, “A learning-based incentive mechanism for federated learning,” IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6360–6368, 2020.
  * [13] J. Kang, Z. Xiong, D. Niyato, H. Yu, Y.-C. Liang, and D. I. Kim, “Incentive design for efficient federated learning in mobile networks: A contract theory approach,” in 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS), pp. 1–5, IEEE, 2019.
  * [14] T. Song, Y. Tong, and S. Wei, “Profit allocation for federated learning,” in 2019 IEEE International Conference on Big Data (Big Data), pp. 2577–2586, IEEE, 2019.
  * [15] S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” Advances in neural information processing systems, vol. 30, 2017.
  * [16] A. F. Shorrocks et al., “Decomposition procedures for distributional analysis: a unified framework based on the shapley value,” Journal of Economic Inequality, vol. 11, no. 1, pp. 99–126, 2013s.
  * [17] R. Zeng, S. Zhang, J. Wang, and X. Chu, “Fmore: An incentive scheme of multi-dimensional auction for federated learning in mec,” in 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), pp. 278–288, IEEE, 2020.
  * [18] Y. Zhan, P. Li, K. Wang, S. Guo, and Y. Xia, “Big data analytics by crowdlearning: Architecture and mechanism design,” IEEE Network, vol. 34, no. 3, pp. 143–147, 2020.
  * [19] L. Gao, L. Li, Y. Chen, C. Xu, and M. Xu, “Fgfl: A blockchain-based fair incentive governor for federated learning,” Journal of Parallel and Distributed Computing, vol. 163, pp. 283–299, 2022.
  * [20] H. Kim, J. Park, M. Bennis, and S.-L. Kim, “On-device federated learning via blockchain and its latency analysis,” arXiv preprint arXiv:1808.03949, 2018\.
  * [21] H. Kim, J. Park, M. Bennis, and S.-L. Kim, “Blockchained on-device federated learning,” IEEE Communications Letters, vol. 24, no. 6, pp. 1279–1283, 2019.
  * [22] C. Ma, J. Li, L. Shi, M. Ding, T. Wang, Z. Han, and H. V. Poor, “When federated learning meets blockchain: A new distributed learning paradigm,” IEEE Computational Intelligence Magazine, vol. 17, no. 3, pp. 26–33, 2022\.
  * [23] J. Han, A. F. Khan, S. Zawad, A. Anwar, N. B. Angel, Y. Zhou, F. Yan, and A. R. Butt, “Tokenized incentive for federated learning,” in Proceedings of the Federated Learning Workshop at the Association for the Advancement of Artificial Intelligence (AAAI) Conference, 2022.
  * [24] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” Decentralized Business Review, p. 21260, 2008.
  * [25] G. Wood et al., “Ethereum: A secure decentralised generalised transaction ledger,” Ethereum project yellow paper, vol. 151, no. 2014, pp. 1–32, 2014.
  * [26] Y. Zhao, J. Zhao, L. Jiang, R. Tan, D. Niyato, Z. Li, L. Lyu, and Y. Liu, “Privacy-preserving blockchain-based federated learning for iot devices,” IEEE Internet of Things Journal, vol. 8, no. 3, pp. 1817–1829, 2020.
  * [27] X. Bao, C. Su, Y. Xiong, W. Huang, and Y. Hu, “Flchain: A blockchain for auditable federated learning with trust and incentive,” in 2019 5th International Conference on Big Data Computing and Communications (BIGCOM), pp. 151–159, IEEE, 2019.
  * [28] Z. Wang and Q. Hu, “Blockchain-based federated learning: A comprehensive survey,” arXiv preprint arXiv:2110.02182, 2021.
  * [29] S. Boyd, S. P. Boyd, and L. Vandenberghe, Convex optimization.  Cambridge university press, March 2004.
  * [30] S. Shalev-Shwartz and T. Zhang, “Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization,” in International conference on machine learning, pp. 64–72, PMLR, 2014.
  * [31] S. R. Pandey, L. D. Nguyen, and P. Popovski, “A contribution-based device selection scheme in federated learning,” IEEE Communications Letters, 2022\.
  * [32] R. Jia, D. Dao, B. Wang, F. A. Hubis, N. Hynes, N. M. Gürel, B. Li, C. Zhang, D. Song, and C. J. Spanos, “Towards efficient data valuation based on the shapley value,” in The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1167–1176, PMLR, 2019.
  * [33] X. Fang and M. Ye, “Robust federated learning with noisy and heterogeneous clients,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10072–10081, 2022.
  * [34] L. S. Shapley, “A value for n-person games,” Classics in game theory, vol. 69, 1997.
  * [35] “Cifar-10 and cifar-100 datasets.” https://www.cs.toronto.edu/~kriz/cifar.html.  (Accessed on 09/14/2022).
  * [36] M. Zhang et al., “Personalized federated learning with first order model optimization,” in International Conference on Learning Representations (ICLR), Vienna, Austria, May 2021.