# An Element-Wise Weights Aggregation Method for Federated Learning

1st Yi Hu  Department of Computer Science
Swansea University
Swansea, United Kingdom
<EMAIL_ADDRESS>2nd Hanchi Ren$\ast$ Department of Computer Science
Swansea University
Swansea, United Kingdom
<EMAIL_ADDRESS>3th Chen Hu  Department of Computer Science
Swansea University
Swansea, United Kingdom
<EMAIL_ADDRESS>4rd Jingjing Deng  Department of Computer Science
Durham University
Durham, United Kingdom
<EMAIL_ADDRESS>5th Xianghua Xie$\ast$ Department of Computer
Science
Swansea University
Swansea, United Kingdom
<EMAIL_ADDRESS>$\ast$ The corresponding authors are Hanchi Ren and
Xianghua Xie.

###### Abstract

Federated learning (FL) is a powerful Machine Learning (ML) paradigm that
enables distributed clients to collaboratively learn a shared global model
while keeping the data on the original device, thereby preserving privacy. A
central challenge in FL is the effective aggregation of local model weights
from disparate and potentially unbalanced participating clients. Existing
methods often treat each client indiscriminately, applying a single proportion
to the entire local model. However, it is empirically advantageous for each
weight to be assigned a specific proportion. This paper introduces an
innovative Element-Wise Weights Aggregation Method for Federated Learning
(EWWA-FL) aimed at optimizing learning performance and accelerating
convergence speed. Unlike traditional FL approaches, EWWA-FL aggregates local
weights to the global model at the level of individual elements, thereby
allowing each participating client to make element-wise contributions to the
learning process. By taking into account the unique dataset characteristics of
each client, EWWA-FL enhances the robustness of the global model to different
datasets while also achieving rapid convergence. The method is flexible enough
to employ various weighting strategies. Through comprehensive experiments, we
demonstrate the advanced capabilities of EWWA-FL, showing significant
improvements in both accuracy and convergence speed across a range of
backbones and benchmarks.

###### Index Terms:

Federated Learning, Weights Aggregation, Adaptive Learning

##  I Introduction

Figure 1: Illustration for our proposed EWWA-FL

As the digital world continues to expand at an unprecedented rate, the world
is inundated with a massive amount of data, distributed across various
devices, sensors, and platforms. With the growing adoption of Machine Learning
(ML) algorithms, the demand for efficient, secure, and decentralized learning
processes has become increasingly critical. FL [1, 2, 3, 4] has emerged as a
promising solution to address these challenges. It enables the deployment of
learning algorithms on decentralized data sources while safeguarding data
privacy. FL focuses on training ML models across a multitude of dispersed
devices or clients, each holding their own local datasets, eliminating the
need for data exchange. This approach effectively addresses privacy and
security concerns, as it obviates the need to transfer potentially sensitive
data to a centralized location. However, a key challenge in FL lies in the
aggregation of model weights. The process of combining model weights from
multiple, disparate clients is inherently complex due to the heterogeneous
nature of their data distributions [5, 6, 7]. In an FL network, each client
utilizes its local data to train an independent model. Consequently, these
local models may capture different data patterns, posing a challenge to the
creation of a well-generalized global model. Various strategies, such as
Federated Averaging (FedAvg), have been developed to mitigate these issues.
Nonetheless, devising an efficient and robust aggregation mechanism remains a
significant challenge in the field of FL.

There has been a surge in recent research focused on adaptive weight
aggregation. Reddi _et al._ [8] proposed a method called FedOpt. They provide
a theoretical analysis of the model’s convergence on heterogeneous data for
non-convex optimization problems, as well as the relationship between dataset
heterogeneity and communication efficiency. Three specific optimization
methods, termed FedAdam, FedAdagrad, and FedYogi, are employed by the authors.
These methods modify the global update rule of FedAvg from one-step SGD to
one-step adaptive gradient optimization. Conversely, the work presented in [9]
aims to address the challenge of high communication cost in FL. The authors
propose a novel communication-efficient adaptive FL method called FedCAMS and
also provide a theoretical analysis to guarantee model convergence. They first
improve upon FedAdam by incorporating AMSGrad [10] with max stabilization.
Both FedOpt and FedCAMS aggregate local updates and obtain an averaged
gradient, upon which global model aggregation is conducted. In other words,
both methods treat each client equally when generating the global updates.
However, the underlying philosophy of adaptive optimization generally favors
treating each individual weight independently. While FedOpt and FedCAMS both
use adaptive techniques for global model aggregation, their averaging
processes do not account for the varied contributions of local models trained
on different datasets. This is noteworthy because different datasets result in
different levels of convergence [11, 12].

In the work of Federated Boosting (FedBoosting) [11], the authors proposed an
adaptive gradient aggregation method based on the boosting algorithm. They
discovered that the generalization ability of the global model on Non-
Independent and Identically Distributed (non-IID) data is unsatisfactory due
to the presence of weight divergence, particularly when employing the FedAvg
strategy. Consequently, each client participating in the training receives a
unique aggregation percentage. Similarly, Wu _et al._ [12] found that in FL,
the path that minimizes the local objective does not necessarily align with
the path of global minimization. This implies that each client’s contribution
to global aggregation will differ. Based on this observation, Wu _et al._
proposed Federated Adaptive Weighting (FedAdp), a method that measures the
contributions of participating clients based on the correlation between local
and global gradients. All of the above studies have one thing in common: they
treat all model parameters equally when aggregating the global model.
Specifically, both FedOpt and FedCAMS perform a simple averaging of local
model weights prior to subsequent computations. Although FedBoosting and
FedAdp assign different proportions to each local model, they still allocate
the same proportion to each parameter within these models. This approach may
not be the most intuitive or efficient way to handle local models.

Key to the FL process is the merging of model weights from different clients,
which is inherently intricate and poses several challenges. The main reason
for this complexity is the heterogeneity of the data distribution of the
participating entities, cause each client’s local dataset has different
statistical properties. For example, one client’s dataset may contain one or
more specific classes, while another client does not. This heterogeneity can
result in non-IID data, which poses a significant challenge in aggregating
local updates in a way that is representative and conducive to global model
performance and generalization. To mitigate the effects of different data
distributions and to ensure robust model aggregation, many sophisticated
algorithms and techniques are proposed [1, 8, 11, 12]. Building a balanced and
harmonious model requires not only rigorous mathematical or algorithmic
knowledge, but also a comprehensive understanding of the differences and
nuances inherent in the data landscape of different clients. Based on the
findings from Wu _et al._ [12], we would like to expand the idea to a more
grained level that the elements in each local model have their personalized
path to minimize the local objective. It is conceivable that each client
should have a different weight, and likewise, each parameter within the local
model should also have a unique weight. In the context of convex optimization
for learning, the goal is to update the model’s weights to achieve
convergence. A model comprises various parameters, the values of which
fluctuate depending on the feature space of the local dataset. Within the
framework of FL, individual local models, trained on distinct datasets,
display unique patterns and directions of convergence. As a result, the same
parameter across these local models may have vastly different values and may
not align closely with each other. Additionally, each parameter may follow a
unique trend and orientation toward convergence. As a result, using a uniform
proportion to aggregate all parameters into a global model may not be the most
suitable approach. Based on this understanding, we introduce Element-Wise
Weights Aggregation Method for  (EWWA-FL), which assigns a different
aggregation proportion to each parameter in the local model. Experimental
results show that our method outperforms FedAvg, FedCAMS, and FedOpt across
various neural networks, benchmark datasets, and experimental settings. The
contributions of this work are fourfold:

  * •

We introduce a new perspective on element-wise weight combination for FL. This
approach assigns a specific proportion to each parameter in the local model,
aiming to improve aggregation. Experimental results confirm the novelty of our
proposed EWWA-FL.

  * •

A comprehensive evaluation is conducted. We test the model’s generalization
ability using various neural networks on different benchmark datasets,
employing both Independent and Identically Distributed (IID) and non-IID
strategies.

  * •

The adaptive element-wise aggregation paradigm demonstrates faster convergence
compared to other recent works.

  * •

We disclose the implementation details of the proposed algorithm to ensure its
reproducibility.

The paper is organized as follows: Section II reviews previous studies related
to adaptive weight aggregation in FL. Preliminaries on vanilla FL and the Adam
optimization algorithm are then discussed. Our proposed approach is elaborated
upon in Section III. Section IV provides insights into the experiments, offers
in-depth discussions, and suggests potential mitigation methods. Finally,
concluding remarks are presented in Section V.

##  II Related Work

A fundamental challenge in FL is the efficient aggregation of model weights
from diverse and potentially non-IID data sources to produce a globally
consistent model. Adaptive weight aggregation addresses this challenge by
assigning different proportions to local model weights based on their quality
or relevance, as opposed to treating them equally. This approach recognizes
the inherent heterogeneity present in real-world FL environments. It optimizes
the performance of the global model by leveraging the more informative weights
from local models and potentially mitigates the negative impact of less
reliable participants.

In the work [8], the authors provide a comprehensive discussion on adaptive
weight aggregation for FL and propose a flexible framework called FedOpt. This
framework is capable of incorporating multiple optimization algorithms. The
authors specialize FedOpt into FedAdam, FedAdagrad, and FedYogi by employing
three example optimization algorithms: Adam [13], Adagrad [14], and YOGI [15].
This approach closely parallels the FedAvg process, diverging only in the
final stage of weight aggregation. After obtaining the averaged local
gradients, denoted as $\hat{g}$, the first-order momentum matrices $m$ are
computed for FedAdam, FedAdagrad, and FedYogi, as detailed in (1). However,
the computation of the second-order variance matrices $v$ varies depending on
the algorithm. Specifically, FedAdam employs (2), while FedAdagrad and FedYogi
utilize (3) and (4), respectively, to derive their second-order matrices.

$\displaystyle m_{r}$
$\displaystyle=\beta_{1}m_{r-1}+(1-\beta_{1})\hat{g}_{r}$ (1) $\displaystyle
v_{r}$ $\displaystyle=\beta_{2}v_{r-1}+(1-\beta_{2})\hat{g}_{r}^{2}$ (2)
$\displaystyle v_{r}$ $\displaystyle=v_{r-1}+\hat{g}_{r}^{2}$ (3)
$\displaystyle v_{r}$
$\displaystyle=\beta_{2}v_{r-1}+(1-\beta_{2})\cdot\hat{g}_{r}^{2}\cdot
sign(v_{r-1}-\hat{g}_{r}^{2})$ (4)

where, $r$ is the training round, $\beta_{1}$ and $\beta_{2}$ are two momentum
parameters, $sign()$ is the symbolic functions. In the end, all those three
methods employ Eq.5 for weights aggregation.

$\displaystyle\omega_{r}=\omega_{r-1}+\eta_{r}\frac{m_{r}}{\sqrt{v_{r}}+\epsilon}$
(5)

where, $\eta_{r}$ is the adaptive learning rate, calculated by:

$\displaystyle\eta_{r}=\eta_{0}\frac{\sqrt{1-\beta_{2}^{r}}}{1-\beta_{1}^{r}}$
(6)

where $\eta_{0}$ denotes the initial learning rate, while $\beta_{1}^{r}$ and
$\beta_{2}^{r}$ represent the $r$-th powers of the parameters $\beta_{1}$ and
$\beta_{2}$, respectively. The authors provide a theoretical analysis to
demonstrate the superiority of the proposed FedOpt in comparison to other
methods. The primary distinction between FedOpt and our proposed method, EWWA-
FL, lies in the location of the optimization algorithm. Specifically, FedOpt
employs the optimization algorithm after averaging the local models, whereas
EWWA-FL performs the optimization after each local training. As a result,
FedOpt treats each client equally and assigns the same aggregation proportion
to each local model through averaging. In contrast, our method treats each
parameter in every local model differently. Building upon FedOpt, Wang _et
al._ [9] introduced FedCAMS with the objective of reducing communication
costs. The optimization algorithm in FedCAMS occupies the same position as in
FedOpt, thereby ensuring that all local weights are aggregated equally.

Unlike FedOpt and FedCAMS, FedBoosting [11] and FedAdp [12] assign different
proportions to each local model to perform adaptive weight aggregation.
FedBoosting computes the aggregation proportion based on the results of local
training $T^{i}_{r}$ and cross-validation $V^{i,j}_{r}$. The authors first sum
all the validation results for a local model across all other local model
validation datasets. Then, they calculate the weight of this sum of validation
results. Finally, a _Softmax_ function is applied to derive the final
proportion for each local model. Equations.7, 8, and 9 provide the local
weight aggregation proportion $p^{i}_{r}$ for the $i$-th local model in
training round $r$:

$\displaystyle p^{(i)}_{r}$
$\displaystyle=softmax(softmax(T^{(i)}_{r})\cdot\sum_{j\neq
i}^{N}V^{(i,j)}_{r})$ (7) $\displaystyle s$ $\displaystyle
oftmax(T^{(i)}_{r})=\frac{exp(T^{(i)}_{r})}{\sum_{j=1}^{N}{exp(T^{(j)}_{r})}}$
(8) $\displaystyle V^{(i,j)}_{r}$
$\displaystyle=\begin{pmatrix}V^{(1,1)}_{r}&V^{(1,2)}_{r}&\cdots&V^{(1,j)}_{r}\\\
V^{(2,1)}_{r}&V^{(2,2)}_{r}&\cdots&V^{(2,j)}_{r}\\\
\vdots&\vdots&\ddots&\vdots\\\
V^{(i,1)}_{r}&V^{(i,2)}_{r}&\cdots&V^{(i,j)}_{r}\end{pmatrix}$ (9)

On the other hand, FedAdp focuses on the angle of convergence between the
updated local weight and the global weight. In particular, they quantify the
contribution of each client in each round of global observations according to
the angle $\theta^{i}$:

$\displaystyle\theta^{(i)}=arccos(\frac{<G,g^{(i)}>}{||G||\cdot||g^{(i)}||})$
(10)

where $G$ is the global gradient, $<\cdot>$ is the inner product operation and
$||\cdot||$ denotes the L2 normalization. To suppress instability caused by
instantaneous angular randomness, the angle $\theta^{i}_{r}$ is then averaged
over previous training rounds $r$:

$\displaystyle\hat{\theta}^{(i)}_{r}=\left\\{\begin{array}[]{ccl}\theta^{(i)}_{r}&\mbox{if}&r=1\\\
\frac{r-1}{r}\hat{\theta}^{(i)}_{r-1}+\frac{1}{r}\theta^{(i)}_{r}&\mbox{if}&r>1\end{array}\right.$
(12)

The authors then designed a non-linear mapping function that quantifies each
client’s contribution based on angular information. Inspired by the _Sigmoid_
function, they use a variant of _Gompertz_ function [16]:

$\displaystyle\mathcal{F}(\hat{\theta})=\alpha(1-\frac{1}{exp(exp(\alpha(1-\hat{\theta})))})$
(13)

where $\alpha$ is a hyper-parameter. The final proportions for each local
model are calculated by giving each client’s contribution value into the
_Softmax_ function.

In comparison to FedOpt and FedCAMS, although FedBoosting and FedAdp provide
different aggregation proportions for local clients, their aggregation
proportions are still at the model level. In contrast, our proposed EWWA-FL
makes progress in this regard by providing a more fine-grained, element-wise
aggregation level. This feature enhances the adaptability and convergence of
the global model, especially considering that the local models come from
different datasets.

##  III Methodology

In this section, we first present a preliminary discussion on FedAvg, as it is
the most commonly used method in FL applications. Subsequently, we briefly
explain the Adam optimization algorithm, highlighting that Adam provides an
adaptive learning rate for each parameter, in contrast to the Stochastic
Gradient Descent (SGD) algorithm. Finally, we introduce the EWWA-FL algorithm,
which enables element-wise global weight aggregation in FL.

###  III-A FedAvg

It is the most basic method behind all the recent proposed FL methods.
Assuming we have many clients $\mathcal{C}$ and their local datasets
$\mathcal{D}$. The task is formulated as $\mathcal{F}$ with weights $\omega$.
So the local gradient $g$ is:

$\displaystyle
g_{i}=\frac{1}{||d_{i}||}\nabla_{\omega}\sum_{j}\mathcal{L}(\mathcal{F}(x^{(j)};\omega),y^{(j)});\forall
i\in||\mathcal{C}||$ (14)

Where $||\cdot||$ denotes the L2 normalization of a vector, $x$ and $y$ are
the samples and their relevant labels in the $i$-th local dataset $d_{i}$. The
server gathers all the local gradients and conduct the averaging process to
generate the next round of global model $\omega_{r}$. We assume that
$||d_{i}||=||d_{k}||;\forall d_{i},d_{k}\in\mathcal{D}$.

$\displaystyle\omega_{r}=\omega_{r-1}-\frac{1}{\mathcal{C}}\sum_{i=1}^{\mathcal{C}}g_{i}$
(15)

The FedAvg algorithm [17] aims to create a unified model by averaging
gradients from various local clients. While this method is effective for
centralizing distributed learning, it is not without shortcomings.
Specifically, inherent differences in data distributions among clients lead to
diverse convergence directions for local model weights. This diversity arises
from the incoherent feature spaces of the data, posing challenges for FedAvg.
When local datasets differ significantly in their data distributions, this can
induce a considerable bias in local model weights. Consequently, the
simplistic averaging mechanism employed by FedAvg may not yield optimal
results, particularly in the presence of significant data biases or extreme
outliers. Recognizing these limitations, we have started exploring more
nuanced aggregation strategies, such as weighted averages or other adaptive
mechanisms. These approaches gauge the contribution of each local model based
on factors like data distribution or its relevance to the overall learning
objective [18, 19, 20]. Employing these refined techniques produces a global
model that is both resilient and accurate, skillfully navigating the
complexities of differing data distributions and overcoming some limitations
inherent to traditional FedAvg methods.

###  III-B Element-Wise Aggregation for FL

Prior to detail our proposed method, we would like to briefly introduce the
Adam algorithm [13] firstly. It is a SGD optimization method based on the
momentum idea. Before each iteration, the first-order and second-order moments
of the gradient are computed and the sliding average is computed to update the
current parameters. This idea combines the ability of the Adagrad [14]
algorithm to handle sparse data with the properties of the RMSProp [21]
algorithm to deal with non-smooth data. Finally, it achieves very good test
performance on both traditional convex optimization problems and deep learning
optimization problems. More details of Adam is shown in Algorithm 1.

Algorithm 1 Adam

0: initial learning rate $\alpha$

0: exponential decay rates $\beta_{1},\beta_{2}\in[0,1]$

0: maximum iteration number $I$

1: initial weights $\omega_{0}$

2: initial $1^{st}$-order moment vector $m_{0}\leftarrow 0$

3: initial $2^{nd}$-order moment vector $v_{0}\leftarrow 0$

4: for each iteration $i=1,2,3,...,I$ do

5: $g_{i}\leftarrow\nabla_{\omega}\mathcal{L}(\mathcal{F}(\omega_{i-1}))$
$\lx@algorithmic@hfill\blacktriangleright$ get gradients at iteration $i$

6: $m_{i}\leftarrow\beta_{1}\cdot m_{i-1}+(1-\beta_{1})\cdot g_{t}$
$\lx@algorithmic@hfill\blacktriangleright$ update biased $1^{st}$-order moment
estimate

7: $v_{i}\leftarrow\beta_{2}\cdot m_{i-1}+(1-\beta_{2})\cdot(g_{t}\odot
g_{t})$ $\lx@algorithmic@hfill\blacktriangleright$ update biased
$2^{nd}$-order moment estimate

8: $\hat{m_{i}}\leftarrow\frac{m_{i}}{1-\beta_{1}^{i}}$
$\lx@algorithmic@hfill\blacktriangleright$ compute biased-corrected
$1^{st}$-order moment estimate

9: $\hat{v_{i}}\leftarrow\frac{v_{i}}{1-\beta_{2}^{i}}$
$\lx@algorithmic@hfill\blacktriangleright$ compute biased-corrected
$2^{nd}$-order moment estimate

10:
$\omega_{i}\leftarrow\omega_{i-1}-\frac{\alpha}{\sqrt{\hat{v_{i}}}+\epsilon}\cdot\hat{m_{i}}$
$\lx@algorithmic@hfill\blacktriangleright$ update weights

11: end for

12: return $\omega_{I}$

Figure 2: Diagram of EWWA-FL. The left part is the client that performs the
local training. The right side is the server calculating the local
contributions and aggregating a new global model based on the local
contributions and gradients. All calculations on the server end are done on an
element-by-element basis.

All of the above works [17, 8, 9, 11, 12] either average the local models or
assign dynamic proportions to the entire local model for global model
aggregation. The learning process for deep learning model can be viewed as a
convex optimization problem where the weights in the model are trained to
reach a minimum point. The basic components of the model are a number of
parameters whose values can vary greatly with the different feature spaces of
different local datasets. In the FL scenario, local models trained on
different local datasets obtain different degrees and directions of
convergence. In these local models, the values of the same parameters may be
completely different or even not close to each other, leading to convergence
heterogeneity in the local models. Therefore, giving one proportion for all
the parameters in the local model is not the best way to aggregate the global
model. From another perspective, in contrast to SGD, the adaptive learning
methods calculate the learning rate from the point of view of the elements.
That is to say, in common model training scenarios, Adam provides an element-
wise adaptive learning rate. We would like to follow Adam’s idea and introduce
element-level adaptive aggregation to FL. As shown in Figure 2, the left part
is the client that performs the local training. The right side is the server,
which calculates the local contributions and aggregates a new global model
based on the local contributions and gradients. All computations on the server
end are performed on an element-by-element basis. Specifically, Algorithm 2 is
the pseudo-code of our proposed EWWA-FL. In the first round of global
training, the global weights $\omega_{0}$, first-order moment vectors $m_{0}$
and second-order moment vectors $v_{0}$ are initialized. Then, the server end
assigns weights to each local model for local training. Once the local
training is complete, The server collects the gradients of all local models to
update the biased first-order moments and second-order moments $m_{r}$ and
$v_{r}$ and compute the unbiased estimates $\hat{m}_{r}$ and $\hat{v}_{r}$.
Then obtain the contribution $b_{r}$ of the local model to the new global
model. Finally, the aggregation proportion for each local model is computed
using _Softmax_. All the computations related to the first-order moment,
second-order moment, local contribution and final aggregation proportion are
element-wise, which means each parameter in each local model will receive a
specific aggregation proportion in each round, rather than one proportion for
all parameters in the local model.

Algorithm 2 EWWA-FL

0: initial local learning rate $\alpha$

0: exponential decay rates $\beta_{1},\beta_{2}\in[0,1]$

0: global training round $R$

1: initial weights $\omega_{0}$

2: initial $1^{st}$-order moment vector $m_{0}\leftarrow 0$

3: initial $2^{nd}$-order moment vector $v_{0}\leftarrow 0$

4: for each round $r=1,2,...,R$ do

5: for each client $c\in C$ do

6:
$g_{r}^{(c)}\leftarrow\nabla_{\omega}\mathcal{L}(\mathcal{F}(\omega_{r-1}))$

7: $m_{r}^{(c)}\leftarrow\beta_{1}\cdot m_{r-1}+(1-\beta_{1})\cdot
g_{r}^{(c)}$

8: $v_{r}^{(c)}\leftarrow\beta_{2}\cdot
m_{r-1}+(1-\beta_{2})\cdot(g_{r}^{(c)}\odot g_{r}^{(c)})$

9: $\hat{m}_{r}^{(c)}\leftarrow\frac{m_{r}^{(c)}}{1-\beta_{1}^{r}}$

10: $\hat{v}_{r}^{(c)}\leftarrow\frac{v_{r}^{(c)}}{1-\beta_{2}^{r}}$

11:
$b_{r}^{(c)}\leftarrow\frac{\alpha}{\sqrt{\hat{v_{r}^{(c)}}}+\epsilon}\cdot\hat{m}_{r}^{(c)}$

12: end for

13:
$p_{r}^{(c)}\leftarrow\frac{exp(b_{r}^{(c)})}{\sum_{i=c}^{C}exp(b_{r}^{(i)})};\forall
c\in C$

14: $G_{r}\leftarrow\sum_{i=c}^{C}p_{r}^{(i)}\cdot g_{r}^{(i)}$

15: end for

##  IV Experiments

In this section, we first describe the settings of all experiments. Then, we
introduce the backbone neural networks and the datasets for benchmark
evaluation. After that, we present multiple sets of experiments to access the
performance of our proposed EWWA-FL against other state-of-the-art methods.

TABLE I: Top-1 classification accuracy (%) across different methods, backbone neural networks and benchmark datasets with IID distribution on local clients. “C-10”, “C-100” and “IC-12” stand for CIFAR-10, CIFAR-100 and ILSVRC2012, respectively. FedAMS is derived from FedCAMS and has no efficient communication settings. The red ones are the highest accuracy and the blue ones are the next highest. Crossed symbols indicate experimental failure, _i.e._ no global model converged in five trials. Model |  _LeNet_ (32*32) |  _ResNet-20_ (32*32) |  _ResNet-32_ (32*32) |  _ResNet-18_ (224*224) |  _ResNet-34_ (224*224)
---|---|---|---|---|---
Dataset | MNIST | C-10 | C-100 | C-10 | C-100 | C-10 | C-100 | IC-12 | C-10 | C-100 | IC-12
FedAvg | 98.14 | 91.20 | 58.58 | 91.37 | 61.91 | 89.50 | 68.59 | 65.50 | 89.27 | 68.30 | 67.91
FedAMS | 98.77 | 86.57 | 54.58 | 87.21 | 54.23 | 91.59 | 66.07 | $\times$ | 91.91 | 65.41 | $\times$
FedCAMS | $\times$ | 76.77 | 41.65 | 84.7 | 41.77 | 91.35 | 66.62 | $\times$ | 91.44 | 66.41 | $\times$
FedOpt | Adam | $\times$ | 73.59 | 22.31 | 74.84 | $\times$ | 78.07 | $\times$ | $\times$ | 81.71 | $\times$ | $\times$
Adagrad | $\times$ | 64.63 | $\times$ | 67.36 | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
YOGI | $\times$ | 65.93 | $\times$ | 71.90 | $\times$ | 73.88 | $\times$ | $\times$ | 71.8 | $\times$ | $\times$
EWWA-FL | Adam | 98.00 | 89.73 | 64.14 | 90.17 | 65.63 | 90.77 | 70.98 | 65.64 | 91.23 | 70.15 | 68.23
Adagrad | 97.99 | 89.74 | 64.16 | 90.17 | 65.84 | 91.17 | 70.34 | 65.64 | 91.13 | 69.77 | 68.33
YOGI | 98.00 | 89.43 | 64.10 | 90.10 | 65.38 | 90.88 | 70.78 | 65.54 | 91.13 | 70.34 | 68.27
TABLE II: Percentage (%) of top-1 classification accuracy across methods, backbone neural networks, and benchmark datasets in non-IID conditions on local clients. The values highlighted in red demonstrate the highest performance. The global aggregation optimization algorithm for FedOpt and EWWA-FL is Adam. Model |  _LeNet_ (32*32) |  _ResNet-20_ (32*32) |  _ResNet-32_ (32*32) |  _ResNet-18_ (224*224) |  _ResNet-34_ (224*224)
---|---|---|---|---|---
Dataset | MNIST | C-10 | C-100 | C-10 | C-100 | C-10 | C-100 | IC-12 | C-10 | C-100 | IC-12
FedAvg | 96.30 | 72.57 | 55.49 | 75.76 | 57.11 | 82.55 | 66.26 | 50.68 | 82.64 | 65.02 | 46.13
FedAMS | 96.78 | 54.78 | 37.33 | 54.46 | 39.71 | 55.71 | 33.71 | 29.26 | 44.13 | 32.12 | 24.09
FedCAMS | $\times$ | 47.76 | $\times$ | 46.47 | 26.87 | 40.03 | 25.57 | $\times$ | 41.27 | 24.27 | $\times$
FedOpt | $\times$ | 61.16 | $\times$ | 46.75 | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
EWWA-FL | 96.88 | 76.04 | 56.07 | 78.50 | 57.56 | 83.86 | 66.74 | 51.67 | 84.36 | 65.97 | 52.50

(a) ResNet-18, CIFAR-10, IID

(b) ResNet-18, CIFAR-10, non-IID

(c) ResNet-18, CIFAR-100, IID

(d) ResNet-18, CIFAR-100, non-IID

(e) ResNet-32, CIFAR-10, IID

(f) ResNet-32, CIFAR-10, non-IID

Figure 3: Visualization of the average training loss across all local clients
is shown. The horizontal axis represents the number of global aggregation
rounds, while the vertical axis indicates the average loss for the current
round. The blue line represents our proposed EWWA-FL, the orange line is from
FedCAMS, and the green line represents FedOpt.

###  IV-A Settings, Backbones and Datasets

We utilized the PyTorch framework [22] to implement all neural network models.
For anyone interested in replicating our results, the source code is open to
the public and can be accessed here111https://github.com/Rand2AI/EWWA-FL. For
our proposed EWWA-FL method, the global aggregation learning rate for Adam,
Adagrad, and Yogi was set at $1.0$ based on the FedOpt reference [8]. We chose
the two momentum parameters of $0.9$ and $0.999$. Local training was conducted
using the SGD optimization algorithm, accompanied by a consistent learning
rate of $0.01$ and a momentum of $0.9$. The local batch size was set at $64$.
The ILSVRC2012 dataset was given a training round number of $100$, while the
other datasets were subjected to $500$ rounds. The code we used to reproduce
the FedOpt and FedCAMS results was taken from FedOpt’s official GitHub
repository222https://github.com/yujiaw98/FedCAMS.

The neural networks of choice include _LeNet_ [23], _ResNet-18_ , _ResNet-20_
, _ResNet-32_ , and _ResNet-34_ [24]. _LeNet_ is comprised of three
convolutional layers with each succeeded by a _Sigmoid_ activation function.
The output layer is a fully-connected layer. Its input dimension stands at
$32*32*3$. On the other hand, _ResNet-18_ and _ResNet-34_ are derivatives from
PyTorch’s official offerings, having an input dimension of $224*224*3$. As for
_ResNet-20_ and _ResNet-32_
333https://github.com/akamaster/pytorch_resnet_cifar10, the developers are
tailored specifically for the CIFAR-10 and CIFAR-100 datasets [25]. The input
size for them is $32*32*3$.

Specifically, we employed the MNIST dataset [26] for _LeNet_. The CIFAR-10 and
CIFAR-100 datasets underwent experimentation using _ResNet-18_ , _ResNet-20_ ,
_ResNet-32_ , and _ResNet-34_. It is worth noting that for _ResNet-18_ and
_ResNet-34_ , the sample dimensions were upsampling to $224*224*3$. The
ILSVRC2012 dataset [27] was exclusively tested using _ResNet-18_ and
_ResNet-34_. The datasets were partitioned in a $9:1$ ratio for training and
testing. Subsequently, the training data is distributed to three local
clients, following either an IID or non-IID distribution. The test samples are
retained on the server end to assess the performance of the current round of
the global model.

###  IV-B Accuracy on IID data

Table I presents a comprehensive comparison of top-1 classification accuracies
for various FL methods, employing different backbone neural networks, and
tested on multiple benchmark datasets where clients are assumed to have an IID
distribution. The table employs color-coding to highlight significant results;
values highlighted in red indicate the highest performance for each dataset,
while those in blue signify the second highest performance.

Performance on Large-Class Datasets: One of the most noteworthy observations
is that our proposed EWWA-FL algorithm exhibits exceptional performance on
datasets that have a large number of classes. Specifically, it outshines the
competition on the CIFAR-100 and ILSVRC2012 datasets, both of which have a
large number of classes, 100 and 1000 respectively. For instance, when
employing the _ResNet-20_ architecture on the CIFAR-100 dataset, our EWWA-FL
model, when optimized using the Adagrad optimizer, achieved an outstanding
accuracy of $64.16\%$. This is considerably better than the next best
performing method, FedAvg, which achieved an accuracy of $58.58\%$. The
improvement margin in this case is $9.53\%$, a significant leap in
performance. Similarly, when using _ResNet-32_ as the backbone architecture on
CIFAR-100, EWWA-FL notched an accuracy of $65.84\%$, surpassing FedAvg’s
$61.91\%$ by a margin of $6.35\%$. Though the performance gain is not as high
as observed on CIFAR-100, EWWA-FL continues to outperform other methods on
ILSVRC2012 dataset as well. For the _ResNet-18_ architecture, it reached an
accuracy of $65.64\%$, and for _ResNet-34_ , it achieved $68.33\%$. Both of
these figures are the highest among the tested methods for their respective
architectures on this dataset.

Competitive Results on Smaller-Class Datasets: Although EWWA-FL does not
achieve the highest accuracy on datasets like MNIST and CIFAR-10, it is
important to note that the algorithm is highly competitive. For the MNIST
dataset, when using the _LeNet_ architecture, the highest accuracy was
achieved by FedAMS with $98.77\%$. However, EWWA-FL was closely behind with an
accuracy of $98.00\%$, making the difference a mere $0.79\%$. On the CIFAR-10
dataset, the performance gaps are also quite narrow. For instance, the
differences in accuracy rates when comparing EWWA-FL to the best-performing
methods are $1.63\%$, $1.33\%$, $0.46\%$, and $0.75\%$ for architectures
_ResNet-20_ , _ResNet-32_ , _ResNet-18_ , and _ResNet-34_ , respectively.

In conclusion, our proposed EWWA-FL method exhibits convincing performance,
especially in challenging scenarios involving large classes of datasets.
Although it is not necessarily the absolute best in all cases, it maintains a
competitive edge in various benchmarks.

Compared with FedOpt, that is also capable of employing various global
aggregation optimization algorithms, our proposed EWWA-FL significantly
outperforms in terms of stability and convergence. In our experiments, the
average standard variance for EWWA-FL was remarkably low, at only $0.1094\%$.
Additionally, the minimum and maximum variances were confined to a tight
range, specifically between $0.0047\%$ and $0.2673\%$, respectively. This
suggests that EWWA-FL offers a highly consistent and reliable performance
across different scenarios. In contrast, FedOpt showed a much higher
variability. The average standard variance for FedOpt was $3.5200\%$, more
than thirty times higher than that of EWWA-FL. Furthermore, the minimum and
maximum variances for FedOpt were $2.0950\%$ and $4.9550\%$, respectively,
indicating a less stable performance. It is worth noting that FedOpt
encountered significant issues during our testing phase. Despite conducting at
least five separate attempts, none of the FedOpt trials converged as expected.
This suggests that FedOpt may have fundamental limitations when it comes to
achieving reliable convergence. We trialed code from the official FedCAMS
GitHub repository as well as our own deployed code. Similarly, FedAMS and
FedCAMS demonstrated a lack of robustness in our experiments. Specifically,
FedAMS failed to converge on the ILSVRC2012 dataset, while FedCAMS failed on
both the MNIST and ILSVRC2012 datasets. These failures further underscore the
superiority of EWWA-FL in achieving stable and consistent results across
various benchmark datasets.

###  IV-C Accuracy on non-IID data

The circumstance on non-IID data exhibits distinct challenges compared to
those on IID data. As the results presented in Table II, our proposed EWWA-FL
model consistently outperformed other methods across various benchmarks,
including MNIST and CIFAR-10 datasets. For these experiments, We employed Adam
as the global optimization algorithm for both FedOpt and EWWA-FL for a fair
comparison. Focusing on the MNIST dataset, EWWA-FL achieved an accuracy of
$96.88\%$. This figure is marginally but importantly higher by $0.1033\%$ when
compared to the $96.78\%$ reported for FedAMS. The small but consistent
improvement serves to highlight the efficacy of EWWA-FL in dealing with non-
IID data. On the CIFAR-10 dataset, when using _ResNet-20_ , EWWA-FL
significantly outshone its closest competitor, FedAvg, by achieving an
accuracy of $76.04\%$. This was a notable $4.7816\%$ improvement over FedAvg’s
$72.57\%$. The performance gains extended to more challenging datasets as
well. For example, on the ILSVRC2012 dataset, EWWA-FL reached an accuracy of
$52.50\%$, which was $13.8088\%$ higher than the $46.13\%$ managed by FedAvg.
This suggests that EWWA-FL is not only effective for simpler datasets but also
scales well to more complex and larger datasets. In the course of our
experimental evaluation, it became evident that certain algorithms like
FedAMS, FedCAMS, and FedOpt encountered significant difficulties in reaching
convergence. This was consistent with their performance on IID data.
Specifically, FedAMS and FedCAMS yielded unsatisfactory results in numerous
tests, such as achieving only $37.33\%$ accuracy using _ResNet-20_ on the
CIFAR-100 dataset and $40.03\%$ accuracy using _ResNet-18_ on the CIFAR-10
dataset. Moreover, FedAMS scored as low as $24.09\%$ when tested using
_ResNet-34_ on the ILSVRC2012 dataset. Similarly, FedOpt struggled in several
experiments, underscoring the limitations of current global optimization
techniques when dealing with non-IID data distributions. As elaborated in
Section III, the model divergence owing to the heterogeneity of local feature
spaces is substantial when only a single aggregation proportion is provided
for the entire local model. Given that each parameter in the local model can
have its own unique direction and level of convergence, the one-size-fits-all
approach falls short. In contrast, an element-wise global model aggregation
strategy, as implemented in EWWA-FL, offers the adaptability and flexibility
needed to facilitate better convergence across a range of neural networks and
benchmark datasets.

###  IV-D Convergence Speed

Convergence speed is also an important metric for evaluating methods in terms
of adaptive aggregation. In Figures 3, we visualize the average training
losses across all local clients for both IID and non-IID data. The figures in
the left column correspond to experiments conducted on IID data using various
backbone neural networks and datasets. Overall, our proposed EWWA-FL method
exhibits the fastest convergence when compared with FedCAMS and FedOpt. When
using a shallow neural network (_e.g. ResNet-18_), the convergence speeds of
EWWA-FL and FedCAMS are similar. However, the performance gap widens when
training on deeper neural network (_e.g. ResNet-32_). On the other hand, in
experiments conducted on non-IID data, shown in the right column, the
convergence speed of EWWA-FL remains fast. FedCAMS experiences a significant
slowdown when using _ResNet-18_ on both CIFAR-10 and CIFAR-100 datasets.
Nonetheless, FedCAMS achieves good convergence speed when using _ResNet-32_ on
the CIFAR-10 dataset. We empirically believe that this is because deeper
neural networks contain more weights, allowing for better fitting on small-
scale datasets. Lastly, FedOpt performs the worst among all the methods in our
experiments. The loss values produced by FedOpt are so large that they are
difficult to visualize; therefore, we have normalized them to fall within a
range of 0 to 1. The green lines indicate that FedOpt struggles to converge,
even after 500 rounds of global aggregation. In some experiments, as mentioned
in Section IV-B, FedOpt failed to converge. So there is no green lines in
Figures 3 (b), (c) and (d).

##  V Conclusion

In this paper, we propose an adaptive element-wise global weights aggregation
method for FL, specifically EWWA-FL. This method demonstrates better and
faster convergence compared to other recent works. Comprehensive experiments
are conducted using various neural networks and datasets to showcase the
superiority of our approach. We also provide a brief theoretical analysis
based on the Adam optimization algorithm. In future work, we plan to focus on
the theoretical proof to further validate our method from a mathematical
perspective. The implementation of our method is publicly available to ensure
its reproducibility.

## References

  * [1] H. B. McMahan, E. Moore, D. Ramage, S. Hampson _et al._ , “Communication-efficient learning of deep networks from decentralized data,” _arXiv_ , 2016.
  * [2] J. Konečnỳ, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” _arXiv_ , 2016.
  * [3] J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” _arXiv_ , 2016.
  * [4] B. McMahan and D. Ramage, “Federated learning: Collaborative machine learning without centralized training data,” _Google Research Blog_ , vol. 3, 2017\.
  * [5] T. Nishio and R. Yonetani, “Client selection for federated learning with heterogeneous resources in mobile edge,” in _ICC 2019-2019 IEEE international conference on communications (ICC)_. IEEE, 2019, pp. 1–7.
  * [6] T. Sery, N. Shlezinger, K. Cohen, and Y. C. Eldar, “Over-the-air federated learning from heterogeneous data,” _IEEE Transactions on Signal Processing_ , vol. 69, pp. 3796–3811, 2021.
  * [7] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings _et al._ , “Advances and open problems in federated learning,” _Foundations and Trends® in Machine Learning_ , vol. 14, no. 1–2, pp. 1–210, 2021\.
  * [8] S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konečnỳ, S. Kumar, and H. B. McMahan, “Adaptive federated optimization,” in _International Conference on Learning Representations_ , 2020.
  * [9] Y. Wang, L. Lin, and J. Chen, “Communication-efficient adaptive federated learning,” in _International Conference on Machine Learning_. PMLR, 2022, pp. 22 802–22 838.
  * [10] S. J. Reddi, S. Kale, and S. Kumar, “On the convergence of adam and beyond,” _arXiv_ , 2019.
  * [11] H. Ren, J. Deng, X. Xie, X. Ma, and Y. Wang, “Fedboosting: Federated learning with gradient protected boosting for text recognition,” _arXiv_ , pp. arXiv–2007, 2020.
  * [12] H. Wu and P. Wang, “Fast-convergent federated learning with adaptive weighting,” _IEEE Transactions on Cognitive Communications and Networking_ , vol. 7, no. 4, pp. 1078–1088, 2021.
  * [13] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv_ , 2014.
  * [14] J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic optimization.” _Journal of machine learning research_ , vol. 12, no. 7, 2011.
  * [15] M. Zaheer, S. Reddi, D. Sachan, S. Kale, and S. Kumar, “Adaptive methods for nonconvex optimization,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [16] M. N. Gibbs and D. J. MacKay, “Variational gaussian process classifiers,” _IEEE Transactions on Neural Networks_ , vol. 11, no. 6, pp. 1458–1464, 2000\.
  * [17] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in _Artificial Intelligence and Statistics_. PMLR, 2017, pp. 1273–1282.
  * [18] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated learning with non-iid data,” _arXiv_ , 2018.
  * [19] X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” _arXiv_ , 2019.
  * [20] C. Xu, Z. Hong, M. Huang, and T. Jiang, “Acceleration of federated learning with alleviated forgetting in local training,” _arXiv_ , 2022.
  * [21] G. Hinton, N. Srivastava, and K. Swersky, “Neural networks for machine learning lecture 6a overview of mini-batch gradient descent,” 2012.
  * [22] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, and L. Antiga, “Pytorch: An imperative style, high-performance deep learning library,” _Advances in neural information processing systems_ , vol. 32, 2019.
  * [23] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” _Proceedings of the IEEE_ , vol. 86, no. 11, pp. 2278–2324, 1998.
  * [24] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 770–778.
  * [25] A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” 2009.
  * [26] Y. LeCun, “The mnist database of handwritten digits,” _http://yann. lecun. com/exdb/mnist/_ , 1998.
  * [27] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE Conference on Computer Vision and Pattern Recognition_. Ieee, 2009, pp. 248–255.

  *[ML]: Machine Learning
  *[FL]: Federated Learning
  *[FedAvg]: Federated Averaging
  *[FedBoosting]: Federated Boosting
  *[non-IID]: Non-Independent and Identically Distributed
  *[FedAdp]: Federated Adaptive Weighting
  *[]:
  *[EWWA-FL]: Element-Wise Weights Aggregation Method for   ()
  *[IID]: Independent and Identically Distributed
  *[SGD]: Stochastic Gradient Descent


∎

11institutetext: Corresponding Author, E-mail<EMAIL_ADDRESS>
E-mail<EMAIL_ADDRESS>
E-mail<EMAIL_ADDRESS>
E-mail<EMAIL_ADDRESS>
# Optimal Design of Helical Springs of Power Law Materials

Dongming Wei  Marios Fyrillas
Adilet Otemissov  Rustam Bekishev

Nazarbayev University
53 Kabanbay Batyr Ave  Astana  010000  Kazakhstan

###### Abstract

In this paper the geometric dimensions of a compressive helical spring made of
power law materials are optimized to reduce the amount of material. The
mechanical constraints are derived to form the geometric programming problem.
Both the prime and the dual problem are examined and solved semi-analytically
for a range of spring index. A numerical example is provided to validate the
solutions.

###### Keywords:

Helical Spring Power Law Materials Geometric ProgrammingOptimal Design

Nomenclature
---
$C$=spring index $=\frac{D}{d}$
$d$=spring wire diameter, $m$
$D$=mean coil diameter, $m$
$N$=number of turns in the spring
$\delta$= tip deflection of the spring, $m$
$K$=bulk modulus, $MPa$
$n$=the power law index
$\rho$=the density of the material, $kg/m^{3}$

##  1 Introduction

Helical springs are the basic structure elements used in many mechanical
devices. In many applications, it is important to optimize the geometric
dimensions of the springs to reduce the amount of material used while
maintaining the ability to support the required loads. Optimal design of
helical springs based on geometric programing is well-known for materials
which obey Hooke’s law, see, e.g., AG , SK , MP . However, materials subject
to nonlinear stress-strain constitutive laws in this context have not been
well-studied. One of the simplest nonlinear material constitutive law is the
following power law

$\sigma=K|\epsilon|^{n-1}\epsilon$ (1)

where $\sigma$ is the axial stress, $\epsilon$ the axial strain, $K$ the
material constant-called the bulk modulus, and $n$ the power law index.
Materials which obey (1) are often called Hollomon or Ludwick materials in
literature, see, e.g. WSE  and WL . High strength alloy metals such as heat
treated metals, stainless steels, Titanium alloys, and the super plastic-
polyimide are the common examples of power law materials. See, e.g., KSK  and
SC  for a list of common metals with numerical values of the bulk modulus and
the power law index.

Helical springs are mechanical devices made of a wire coiled in the form of a
helix and considered to be a major element of shock absorber, return
mechanisms, fuel flow controller used in engineering, automotive, medical and
agricultural machinery. They are widely used for compressive loads. Because
even for small strains there is no obvious yield of the stress-strain curve
for the power law materials before ultimate yield point, the linear theory or
the traditional reduced modulus theory is not applicable or cannot be
accurately applied to calculate stress distributions.

In this work, we first derive the maximum mechanical stress or loads and the
corresponding tip-deflection under a compressive load, and then formulate the
corresponding geometric programming problem minimizing the amount of material
needed for the given loads. The optimal solutions of the geometric programming
problem is studied by considering KKTC conditions. The corresponding dual
problem for the primal problem is constructed and examined for the solution as
well. A numerical example is provided for both the prime and the dual problem.

##  2 The Mechanical Constraints for the Spring

Let $x$ denote the distance along a circular shaft from the fixed end under a
uniform torque. We assume that the rotation at $x$, denoted by $\phi(x)$, is
proportional to $x$, i.e., $\phi(x)=\alpha x$, where $\alpha$ is the rate of
twist. Further, we assume that
$\epsilon_{xx}=\epsilon_{rr}=\epsilon_{xr}=\epsilon_{r\theta}=0$, and
$\epsilon_{x\theta}=\frac{r\alpha}{2}$, where $\epsilon_{ij},i,j=x,r,\theta$
are the strains in polar coordinates. For the power law materials, we have the
shear strain due to torsion
$\tau_{x\theta}=2G|\epsilon_{x\theta}|^{n-1}\epsilon_{x\theta}=2G(\frac{r\alpha}{2})^{n}$,
where $G=\frac{K}{1+\nu}$ is the shear modulus and $\nu$ the Poison’s ratio.
Let $A$ denote the cross-section of the shaft, the total torque at $x$ is
given by $T=\int_{A}\tau_{x\theta}rdA=2G(\frac{\alpha}{2})^{n}I_{n}$, where
$I_{n}=\int_{A}r^{n+1}drd\theta=\frac{\pi d^{n+3}}{(n+3)2^{n+2}}$ is the
generalized area moment. Therefore, we have
$\alpha=2(\frac{T}{2GI_{n}})^{1/n}$, $\tau_{x\theta}=\frac{Tr^{n}}{I_{n}}$.
Assume that the angle between loading force at the tip of the spring and the
plane containing the cross-section $A$ of the wire is negligible, then the
torque acting on the wire $T=P\frac{D}{2}$ and the average shear stress in the
wire due to a vertical constant load, denoted by $P$ is
$\tau_{av}=\frac{Force}{Area}=\frac{P}{\pi d^{2}/4}=\frac{4P}{\pi d^{2}}$.
Therefore, the total stress at $r$ is approximately

$\tau=\frac{Tr^{n}}{I_{n}}+\frac{4P}{\pi d^{2}}=\frac{2^{n+1}(n+3)PDr^{n}}{\pi
d^{3+n}}+\frac{4P}{\pi d^{2}}$ (2)

We now derive the deflection of the tip by extending the standard textbook
method ( see, e.g., Sp  and Sh  )for $n=1$ to the case of any value of $n$.

Figure 1: A Compressive Helical Spring Diagram

We first have

$GI_{n}\frac{d\phi}{ds}=\tau
ds=P\left(\frac{D}{2}\right)^{2}\frac{d\beta}{ds}$

which gives $\frac{d\phi}{ds}=\frac{PD^{2}}{4GI_{n}}\frac{d\beta}{ds}$. Then,
we have

$\frac{d\delta}{ds}=l\sin{\gamma}\frac{d\phi}{ds}=l\frac{D}{2l}\frac{PD^{2}}{4GI_{n}}\frac{d\beta}{ds}$
$=\frac{(n+3)2^{n+2}}{8}\frac{D^{3}P}{G\pi d^{n+3}}\frac{d\beta}{ds}$

Therefore, we have the tip deflection formula

$\delta=\int_{0}^{2\pi N}\frac{(n+3)2^{n+2}D^{3}P}{8G\pi
d^{n+3}}d\beta=\frac{(n+3)2^{n+2}D^{3}PN}{4Gd^{n+3}}$ (3)

##  3 Formulation of the Geometric Programming Problem

The objective function of our optimization problem is the weight of a helical
spring under axial load $P$. As for the constraints, we consider only upper
bounds on the shear stress in the cross-sections of the wire as described in
the previous section and the tip deflection of the spring. First, define the
design vector to be

$\mathbf{X}=\left\\{\begin{array}[]{cc}x_{1}\\\
x_{2}\end{array}\right\\}=\left\\{\begin{array}[]{cc}D\\\
d\end{array}\right\\}$ (4)

where $D$ is the mean diameter of the coil and $d$ is the diameter of the wire
as defined in the nomenclature. Then, the objective function (mass) of the
helical spring can be expressed as

$f(\mathbf{X})=\frac{\pi d^{2}}{4}(\pi D)\rho N$ (5)

where $N$ is the number of turns in the spring and $\rho$ is the density of
the spring material. From (2) with $r=d/2$, the maximum stress for the power
law helical spring is given by

$\tau_{max}=\frac{2(n+3)PD}{\pi d^{3}}+\frac{4P}{\pi d^{2}}$
$\tau_{max}=\frac{2(n+3)PD}{\pi d^{3}}K_{s}$ (6)

where $K_{s}=1+\frac{2}{(n+3)C}$, $C=\frac{D}{d}$ is called Spring index. By
(3), the expression for the maximum tip deflection of the helical spring is
now

$\delta_{max}=\frac{(n+3)2^{n+3}}{8}\frac{x_{1}^{3}PN}{Gx_{2}^{n+3}}$ (7)

where

$I_{n}=\frac{2\sqrt{\pi}d^{n+3}}{n+3}\frac{\Gamma(1+\pi/2)}{\Gamma(3/2+\pi/2)}=\frac{\pi
d^{n+3}}{(n+3)2^{n+2}}$ (8)

We now have the following nonlinear geometric programming problem for the
parameters $\rho$, $P$, $\tau_{max}$, $\delta_{max}$, $K$, $n$, $N$:

$\begin{array}[]{ll@{}ll}Minimize&f(x_{1},x_{2})=\frac{\pi^{2}}{4}\rho
Nx_{2}^{2}x_{1}\\\ \\\ subject\;to&\frac{2(n+3)Px_{1}}{\pi
x_{2}^{3}}[1+\frac{2x_{2}}{(n+3){x_{1}}}]\leq\tau_{max}\\\ &\hskip
30.0pt\frac{(n+3)2^{n+3}}{8}\frac{x_{1}^{3}PN}{Gx_{2}^{n+3}}\leq\delta_{max}\\\
&\hskip 88.0ptx_{2}\leq x_{1}\end{array}$ (9)

An examination of the KKTC conditions of (9) results in no solution. This is
because the third constraint must be active i.e. $x_{1}=x_{2}$, which is
impossible in the context of the problem. Therefore, a third constraint, which
is $kx_{2}\leq x_{1}$, where $k>1$ is added to complete the nonlinear program
model. This is motivated by using the spring index equation
$C=\frac{x_{1}}{x_{2}}$ in literature. With the new formulation of the problem
our goal is to find optimal values of $k,x_{1},x_{2}$.We have

$\begin{array}[]{ll@{}ll}\text{minimize}&f=cx_{1}x_{2}^{2}\\\ \\\
\text{subject to}&g_{1}=c_{11}x_{1}x_{2}^{-3}+c_{12}x_{2}^{-2}-1\leq 0\\\
&g_{2}=c_{21}x_{1}^{3}x_{2}^{-n-3}-1\leq 0\\\ &g_{3}=kx_{2}-x_{1}\leq 0\\\
&x_{1},x_{2}\geq 0\\\ \end{array}$ (10)

where $c=\frac{\pi^{2}\rho N}{4}$, $c_{11}=\frac{2(n+3)P}{\pi\tau_{max}}$,
$c_{12}=\frac{4P}{\pi\tau_{max}}$,
$c_{21}=\frac{(n+3)2^{n}PN}{G\delta_{max}}$. Notice that all coefficients are
positive.

##  4 Computing Optimal Solution from KKTC Conditions

To find optimal values of $k,x_{1},x_{2}$ for the prime problem (10), we
examine the KKTC conditions as listed below:

$\begin{array}[]{ll@{}ll}cx_{2}^{2}+c_{11}\lambda_{1}x_{2}^{-3}+3c_{21}\lambda_{2}x_{1}^{2}x_{2}^{-n-3}-\lambda_{3}&=0\\\
2cx_{1}x_{2}-3c_{11}\lambda_{1}x_{1}x_{2}^{-4}-2\lambda_{1}c_{12}x_{2}^{-3}-&\\\
-\lambda_{2}(n+3)c_{21}x_{1}^{3}x_{2}^{-n-4}+k\lambda_{3}&=0\\\
(c_{11}x_{1}x_{2}^{-3}+c_{12}x_{2}^{-2}-1)\lambda_{1}&=0\\\
(c_{21}x_{1}^{3}x_{2}^{-n-3}-1)\lambda_{2}&=0\\\
(kx_{2}-x_{1})\lambda_{3}&=0\\\ \lambda_{i}\geq 0\;\;\text{for}\;\;i=1,2,3&\\\
g_{i}\leq 0\;\;\text{for}\;\;i=1,2,3&\\\ \end{array}$

If $\lambda_{3}=0$, then from the first equation it follows that $\lambda_{1}$
or/and $\lambda_{2}$ are less than $0$. Hence, $\lambda_{3}$ must be positive.

Case 1: $\lambda_{3}>0$, $\lambda_{1}=0$ and $\lambda_{2}=0$

KKTC conditions give

$\begin{array}[]{ll@{}ll}cx_{2}^{2}-\lambda_{3}&=0\\\
2cx_{1}x_{2}+k\lambda_{3}&=0\\\ kx_{2}-x_{1}&=0\\\ \end{array}$

from which $x_{2}=0$ is obtained. However, $x_{2}$ cannot be zero; therefore,
this case results in no solution.

Case 2: $\lambda_{3}>0$, $\lambda_{1}>0$ and $\lambda_{2}>0$

From KKTC conditions we have

$\begin{array}[]{ll@{}ll}c_{11}x_{1}x_{2}^{-3}+c_{12}x_{2}^{-2}-1&=0\\\
c_{21}x_{1}^{3}x_{2}^{-n-3}-1&=0\\\ kx_{2}-x_{1}&=0\\\ \end{array}$

This system, in general, has no solution since there are three equations and
two unknowns. This case gives no solution, too.

Case 3: $\lambda_{3}>0$, $\lambda_{1}>0$ and $\lambda_{2}=0$

We have the following system of equations

$\begin{array}[]{ll@{}ll}cx_{2}^{2}+c_{11}\lambda_{1}x_{2}^{-3}-\lambda_{3}&=0\\\
2cx_{1}x_{2}-3c_{11}\lambda_{1}x_{1}x_{2}^{-4}-2\lambda_{1}c_{12}x_{2}^{-3}+k\lambda_{3}&=0\\\
c_{11}x_{1}x_{2}^{-3}+c_{12}x_{2}^{-2}-1&=0\\\ kx_{2}-x_{1}&=0\\\ \end{array}$

This gives

$x_{1}=k\sqrt{c_{11}k+c_{12}},\;\;x_{2}=\sqrt{c_{11}k+c_{12}}$
$\lambda_{1}=\frac{3ckx_{2}^{5}}{2c_{11}k+2c_{12}},\;\;\lambda_{3}=\frac{5c_{11}k+2c_{12}}{2c_{11}k+2c_{12}}cx_{2}^{2}$

$\lambda$’s and $x$’s are positive and satisfy non-negativity constraints. In
this case, inequality $g_{2}\leq 0$ must also hold. That is,

${\displaystyle c_{21}x_{1}^{3}x_{2}^{-n-3}-1\leq
0\;\Rightarrow\;c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}\leq 0}$ (11)

We must choose such constant $k$ that satisfies the above inequality and
minimizes the objective function which becomes

$f_{1}=cx_{1}x_{2}^{2}=ck(c_{11}k+c_{12})^{3/2}$

Notice that minimization of the objective function is equivalent to
minimization of $k$. Thus, for this case we choose minimum $k>1$ that
satisfies (11).

Case 4: $\lambda_{3}>0$, $\lambda_{1}=0$ and $\lambda_{2}>0$

In this case KKTC conditions result in the following system of equations

$\begin{array}[]{ll@{}ll}cx_{2}^{2}+3c_{21}\lambda_{2}x_{1}^{2}x_{2}^{-n-3}-\lambda_{3}&=0\\\
2cx_{1}x_{2}-\lambda_{2}(n+3)c_{21}x_{1}^{3}x_{2}^{-n-4}+k\lambda_{3}&=0\\\
c_{21}x_{1}^{3}x_{2}^{-n-3}-1&=0\\\ kx_{2}-x_{1}&=0\\\ \end{array}$

from which we obtain

$x_{1}=c_{21}^{1/n}k^{1+3/n},\;\;x_{2}=c_{21}^{1/n}k^{3/n}$
$\lambda_{2}=\frac{3cx_{2}^{n+3}}{c_{21}k^{2}n},\;\;\lambda_{3}=c\left(1+\frac{9}{n}\right)x_{2}^{2}$

$\lambda$’s and $x$’s satisfy non-negativity constraints. Next, consider
inequality $g_{1}\leq 0$

${\displaystyle c_{11}x_{1}x_{2}^{-3}+c_{12}x_{2}^{-2}-1\leq 0\Rightarrow
c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}\geq 0}$ (12)

Notice that LHS of (11) and (12) are the same.

In this case, the objective function is

$f_{2}=cx_{1}x_{2}^{2}=cc_{21}^{3/n}k^{1+9/n}$

Again, minimization of the objective function is equivalent to minimization it
relative to $k$. Now, we prove that if the following equation

${\displaystyle c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}=0}$ (13)

has a root $k^{*}>1$ then KKTC conditions provide a desired solution. Suppose
that $k^{*}$ is a minimal root of (13) that is greater than $1$. Assume that
interval $(1,k^{*}]$ satisfies inequality (11), then there is no minimal value
of $k$ that satisfies the following system of equations

$\left\\{\begin{array}[]{rr}c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}\leq 0&\\\
k>1&\\\ \end{array}\right.$

Therefore, case 3 does not give any solution. However, case 4 provides a
solution because there exists a minimal value of $k$ of the following system
of equations

$\left\\{\begin{array}[]{rr}c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}\leq 0&\\\
k>1&\\\ \end{array}\right.$

and that minimal value of $k$ is $k^{*}$. Similarly, if the interval
$(1,k^{*}]$ satisfies inequality (12), then case 3 provides a solution.

We now show that solutions in both cases are the same. In other words, $x_{1}$
and $x_{2}$ in case 3 are equal to $x_{1}$ and $x_{2}$ in case 4,
respectively. It is enough to show that $x_{2}$’s are the same.

$\sqrt{c_{11}k^{*}+c_{12}}=c_{21}^{1/n}(k^{*})^{3/n}\;\Leftrightarrow\;c_{11}k^{*}+c_{12}=c_{21}^{2/n}(k^{*})^{6/n}$
$\Leftrightarrow\;c_{21}^{2/n}(k^{*})^{6/n}-c_{11}k^{*}-c_{12}=0$

Since $x_{2}$’s are equal, $x_{1}$’s and the values of objective functions in
two cases are the same. This means that any solution of $x_{1}$ and $x_{2}$
can be chosen from case 3 or 4. We denote the function
$c_{21}^{2/n}(k^{*})^{6/n}-c_{11}k^{*}-c_{12}$ by $g(k)$. The algorithm of
finding optimal diameters and mass of helical spring can be stated as the
following steps: Step 1:

Solve numerically the inequality

$g(k)=c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}\leq 0$

and choose a value of $k$ in the solution interval of the inequality that is
larger than 1 ( See Figure 2),
Step 2:

Calculate $x_{1}$, $x_{2}$ and $f$

$x_{1}=k\sqrt{c_{11}k+c_{12}},\;\;\;x_{2}=\sqrt{c_{11}k+c_{12}},\;\;\;f=cx_{1}x_{2}^{2}$

It can be shown ( see also Figure 2) that the equation $g(k)=0$ has one
negative root and one positive root that is greater than $1$. This gives
optimal design variables for and spring index $C=k$ in the range $(1,k^{*}]$,
where $k^{*}$ denotes the positive root. Notice that we can again optimize the
objective function relative to the spring index $C$ by choosing the design
variables corresponding to the minimal weight.

##  5 Computing the Optimal Solution of the Dual Problem

The algorithm of solving geometric programming problem by constructing its
dual is well described in RAO . We will follow that algorithm and show that
the solution of the dual coincides with the solution of the primal problem.

First, rewrite the problem in the following way

$\begin{array}[]{ll@{}ll}Minimize&f(x_{1},x_{2})=cx_{1}x_{2}^{2}\\\ \\\
subject\;to&c_{11}x_{1}x_{2}^{-3}+c_{12}x_{2}^{-2}\leq 1\\\ &\hskip
25.0ptc_{21}x_{1}^{3}x_{2}^{-(n+3)}\leq 1\\\ &\hskip
50.0ptkx_{1}^{-1}x_{2}\leq 1\end{array}$ (14)

We now form the dual of the primal problem. Remember that the maximum of the
dual problem is equal to the minimum of the prime. Hence, we have

$\begin{array}[]{ll@{}ll}Maximize&v=\left(\frac{c}{\lambda_{01}}\lambda_{01}\right)^{\lambda_{01}}\left(\frac{c_{11}}{\lambda_{11}}(\lambda_{11}+\lambda_{12})\right)^{\lambda_{11}}\\\
&\hskip
20.0pt\times\left(\frac{c_{12}}{\lambda_{12}}(\lambda_{11}+\lambda_{12})\right)^{\lambda_{12}}\left(\frac{c_{21}}{\lambda_{21}}\lambda_{21}\right)^{\lambda_{21}}\\\
&\hskip
20.0pt\times\left(\frac{k}{\lambda_{31}}\lambda_{31}\right)^{\lambda_{31}}\\\
\\\ subject\;to&\lambda_{01}=1\\\
&\lambda_{01}+\lambda_{11}+3\lambda_{21}-\lambda_{31}=0\\\
&2\lambda_{01}-3\lambda_{11}-2\lambda_{12}-(n+3)\lambda_{21}+\lambda_{31}=0\\\
&\lambda_{11}+\lambda_{12}\geq 0\\\ &\lambda_{21}\geq 0\\\ &\lambda_{31}\geq
0\\\ \end{array}$ (15)

From the equations

$\lambda_{01}=1$
$\lambda_{01}+\lambda_{11}+3\lambda_{21}-\lambda_{31}=0$
$2\lambda_{01}-3\lambda_{11}-2\lambda_{12}-(n+3)\lambda_{21}+\lambda_{31}=0$

we obtain

$\begin{array}[]{ll@{}ll}\lambda_{21}=\frac{3-2\lambda_{11}-2\lambda_{12}}{n}\\\
\lambda_{31}=1+\frac{9}{n}+(1-\frac{6}{n})\lambda_{11}-\frac{6}{n}\lambda_{12}\\\
\end{array}$ (16)

Then, the objective function becomes

$\begin{array}[]{ll@{}ll}v&\displaystyle=(c)^{\lambda_{01}}\left(\frac{c_{11}}{\lambda_{11}}(\lambda_{11}+\lambda_{12})\right)^{\lambda_{11}}\left(\frac{c_{12}}{\lambda_{12}}(\lambda_{11}+\lambda_{12})\right)^{\lambda_{12}}\\\
&\times(c_{21})^{\frac{3-2\lambda_{11}-2\lambda_{12}}{n}}(k)^{1+\frac{9}{n}+(1-\frac{6}{n})\lambda_{11}-\frac{6}{n}\lambda_{12}}\end{array}$
(17)

In order to find maximum of $v$ we differentiate $\ln v$ with repsect to
$\lambda_{11}$ and $\lambda_{12}$ and solve the system of two equations.

$\frac{\partial\ln
v}{\partial\lambda_{11}}=\ln(c_{11})+\ln(\lambda_{11}+\lambda_{12})-\ln\lambda_{11}-\ln
c_{21}^{2/n}+\ln k^{1-6/n}$ $\frac{\partial\ln
v}{\partial\lambda_{12}}=\ln(c_{12})+\ln(\lambda_{11}+\lambda_{12})-\ln\lambda_{12}-\ln
c_{21}^{2/n}-\ln k^{6/n}$

From

$\frac{\partial\ln
v}{\partial\lambda_{11}}=0\;\;\text{and}\;\;\frac{\partial\ln
v}{\partial\lambda_{12}}=0$

we obtain

$\begin{array}[]{ll@{}ll}\displaystyle\frac{\lambda_{11}+\lambda_{12}}{\lambda_{11}}&=c_{11}^{-1}c_{21}^{2/n}k^{6/n-1}\\\
\\\
\displaystyle\frac{\lambda_{11}+\lambda_{12}}{\lambda_{12}}&=c_{12}^{-1}c_{21}^{2/n}k^{6/n}\\\
\end{array}$ (18)

This gives

$c_{11}^{-1}c_{21}^{2/n}k^{6/n-1}-1=\frac{1}{c_{12}^{-1}c_{21}^{2/n}k^{6/n}-1}\Rightarrow$
$\Rightarrow c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}\leq 0$

Notice that the above equation and (13) are the same.

With (18) the objective function (17) becomes

$v^{*}=cc_{21}^{3/n}k^{1+9/n}$ (19)

Now, in order to find $x_{i}$’s we need to solve the following system of
equations

$\begin{array}[]{rl@{}rl}\lambda_{01}&\displaystyle=\frac{cx_{1}x_{2}^{2}}{v^{*}}\\\
\frac{\lambda_{11}}{\lambda_{11}+\lambda_{12}}&=c_{11}x_{1}x_{2}^{-3}\\\
\frac{\lambda_{12}}{\lambda_{11}+\lambda_{12}}&=c_{12}x_{2}^{-2}\\\
\frac{\lambda_{21}}{\lambda_{21}}&=c_{21}x_{1}^{3}x_{2}^{-n-3}\\\
\frac{\lambda_{31}}{\lambda_{31}}&=kx_{1}^{-1}x_{2}\\\ \end{array}$ (20)

With (18) and (19) the above system becomes

$\begin{array}[]{rl@{}rl}1&\displaystyle=\frac{cx_{1}x_{2}^{2}}{cc_{21}^{3/n}k^{1+9/n}}\\\
c_{11}^{-1}c_{21}^{2/n}k^{6/n-1}&=c_{11}x_{1}x_{2}^{-3}\\\
c_{12}^{-1}c_{21}^{2/n}k^{6/n}&=c_{12}x_{2}^{-2}\\\
1&=c_{21}x_{1}^{3}x_{2}^{-n-3}\\\ 1&=kx_{1}^{-1}x_{2}\\\ \end{array}$

and as a solution we get

$x_{1}=c_{21}^{1/n}k^{1+3/n}\\\ x_{2}=c_{21}^{1/n}k^{3/n}$ (21)

We also need to check whether inequality constraints in (15) are satisfied.
That is,

$\begin{array}[]{rl@{}rl}\lambda_{11}+\lambda_{12}&\geq 0\\\ \lambda_{21}&\geq
0\\\ \lambda_{31}&\geq 0\\\ \end{array}$

From the first equality in (16) it follows

$\lambda_{21}=\frac{3-2\lambda_{11}-2\lambda_{12}}{n}\geq
0\Leftrightarrow\lambda_{11}+\lambda_{12}\leq\frac{3}{2}$ (22)

Then, from (18) we obtain

$\lambda_{11}+\lambda_{12}=c_{11}^{-1}c_{21}^{2/n}k^{6/n-1}\lambda_{11}\leq\frac{3}{2}$

Therefore,

$\lambda_{11}\leq\frac{3}{2}c_{11}c_{21}^{-2/n}k^{1-6/n}$ (23)

Combining the second equality in (16) and (22) we have

$\lambda_{31}=1+\frac{9}{n}+\lambda_{11}-\frac{6}{n}(\lambda_{11}+\lambda_{12})\geq
1+\lambda_{11}\geq\lambda_{11}$ (24)

The values of $\lambda$’s cannot be obtained from (18) and they do not affect
the solution. Hence, we are free to set the values for $\lambda$’s as long as
they satisfy the constraints.

For $\lambda_{11}$ choose any positive number that satisifies (23). Then, (18)
gives

$\lambda_{12}=(c_{11}^{-1}c_{21}^{2/n}k^{6/n-1}-1)\lambda_{11}$

From (22) and (24) it follows that $\lambda_{21}$ and $\lambda_{31}$ are
positive. The solution (21) and the objective function (19) coincide with the
solution in case 3 of the primal problem. Remember that the solutions obtained
from case 3 and 4 are the same. The coefficient $k$ is again found by solving
the equation $c_{21}^{2/n}k^{6/n}-c_{11}k-c_{12}=0$. From (19) one should not
think that because we maximize $v$ we need to maximize $k$. In the dual and
primal problems $k$ is a coefficient and the value of which are free to set.
From the solution of the dual we know that $k$ must satisfy (13). However, in
order to choose the right $k$ one needs to look at the primal problem. In the
context of the primal problem $k$ should be minimized i.e. must be the minimum
positive root of (13) which is greater than 1.

##  6 Numerical Experiments

Problem: Formulate the problem of minimum weight design if a helical spring
under axial load that is made of stainless steel. Consider constraints on the
shear stress and the deflection of the spring. Number of active turns $N=10$,
the density $\rho=7700kg/m^{3}$, $n=0.1$ for $\nu=0.275$, $K=960MPa$, for the
axial load, take $P=10.0N$. The maximum deflection of the spring
$\delta_{max}=0.03m$ and maximum shear stress is $\tau_{max}=200MPa$. First,
we numerically solve for minimum root of (13) and then obtain the optimal
solution $x_{1},x_{2}$ from the solution obtained in the previous two
sections. After computing on Mathematica, we have
$c=189989.8847,c_{11}=9.8676\cdot 10^{-8},c_{12}=6.3662\cdot
10^{-8},c_{21}=1.4709\cdot 10^{-5}$. The solution of the inequality(13) is the
interval$\quad(-0.645162,33.0756].$

Figure 2: A Compressive Helical Spring Diagram

We now form the prime problem with numerical coefficients for this example:

$\begin{array}[]{rl}\textnormal{Minimize}&\displaystyle
f(x_{1},x_{2})=189989.8847x_{1}x_{2}^{2}\\\ \textnormal{Subject
to}&\displaystyle\frac{9.8676\cdot 10^{-8}x_{1}+6.3662\cdot
10^{-8}x_{2}}{x_{2}^{3}}\leq 1\\\ &\displaystyle\frac{1.4709\cdot
10^{-5}x_{1}^{3}}{x_{2}^{3.1}}\leq 1.\\\ &\displaystyle kx_{2}\leq x_{1}\\\
&\displaystyle x_{2},x_{1}\geq 0\end{array}$ (25)

where the value of $k$ can be chosen from the interval $(1,33.0756]$. By
solving this prime problem directly and by the semi-analytic solution provided
in the steps in section 5, the numerical solutions match with little error.
For example, by taking $k=10$, we have the solution $x_{1}=0.010249m$,
$x_{2}=0.0010249m$ and the objective function value is $f=0.00306809kg$. This
numerical experiment indicates that the solution obtained using the formula in
Section 5 agrees with the solutions computed by the Mathematica.

##  7 Conclusion

A nonlinear geometric programing problem is formulated for compression helical
springs made of power law materials. Both the prime and the dual problems are
shown to have the same solution by examining the KKTC conditions. A semi-
analytic solution is derived which provides solutions of the helical spring
for a range of spring index. A numerical example is also provided to
illustrate and validate the semi-analytic solution with the solution computed
by solving the prime problem numerically using Mathematica.

## References

  * (1) D. Wei, A. Sarria, and M. Elgindi, Critical buckling loads of the perfect Hollomon’s power-law columns, Mechanics Research Communications, Vol. 47, (2013) pp.69-76.
  * (2) D. Wei and Y. Liu, Analytic and finite element solutions of the power-law Euler Bernoulli beams, Finite Elements in Analysis & Design, 52, (2012) pp. 31-40.
  * (3) S. Kalpakjian, S. R. Schmid, and C-W Kok, Manufacturing processes for engineering materials, Pearson-Prentice Hall,2008.
  * (4) J. F. Shackelford and M. P. Clode, Introduction to materials science for engineers, Pearson-Prentice Hall,2000.
  * (5) S. Rao, Engineering Optimization-Theory and Practice, 4th Ed., John Wiley & Sons, Inc, 2009.
  * (6) G.K. Agrawal,TECHNICAL NOTE: Helical torsion springs for minimum weight by geometric programing, J. Optim. Theory Appl., Vol. 25, No.2, pp.307- 310 (1978).
  * (7) M. K. Suri and B. Kishor, Application of Geometric Programming for Spring Design, Def.. Sci. J., Vol. 29, (1976) pp.1-6.
  * (8) L.J. Mancini and R.L. Piziali, Optimal Design on Helical Spring by Geometric Programming, Engineering Optimization, Vol.2, Issue 1, (1976) pp.73-81.
  * (9) M.F Spotts, Design of Machine Elements, Prentice Hall India Pvt. Limited, 6th Ed., 1991
  * (10) J.E Shigley and C.R Mischke, Mechanical Engineering Design , McGraw Hill Publication, 5th Ed., 1989.