In the final part, we test the performance of -type algorithms on problems with linear constraints.
We consider all previously tested algorithms and two specifically designed simplicial partitions based  and  [Paulavičius and Žilinskas, 2016].
The main advantage of using simplices is that they can cover a feasible region defined by linear constraints.
Thus any infeasible areas are not involved in the search.
Moreover, for most problems from , the solution is located at the intersection of linear constraints.
Therefore  finds it in the early or even in the first iteration.
The performance profiles (see <ref>) reveal the overall effectiveness of the  algorithm for such problems with linear constraints.
Nevertheless, the efficiency of simplicial partition-based algorithms suffers from the problem dimension.
Three greater dimensionality problems with linear constraints were unsolved by the  algorithm (see the bottom part of <ref>, `Performance of -type algorithms devoted for problems with linear constraints only').
Moreover, simplicial partition-based implementations are pretty slow.
For example, on average,  is approximately $307$ times faster than .
Moreover,  solved all test problems with linear constraints.

group style=

group size=1 by 2,

x descriptions at=edge bottom,
y descriptions at=edge left,

vertical sep=20pt,

title  = Based on function evalutions,
ytick distance=0.2,
xtick distance=10,

[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [circle,draw=darkblue,fill=darkblue,inner sep=1.25pt] ;,decorate,,darkblue,line width=0.75pt] table[x=T,y=NAS] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=4,draw=blueryb,fill=blueryb,inner sep=1.25pt] ;,decorate,,blueryb,line width=0.75pt] table[x=T,y=BAR] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [draw=DarkRed,fill=DarkRed,inner sep=1.25pt] ;,decorate,,DarkRed,line width=0.75pt] table[x=T,y=SU2] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [regular polygon,regular polygon sides=5,draw=arsenic,fill=arsenic,inner sep=1.25pt] ;,decorate,,arsenic,line width=0.75pt] table[x=T,y=SU3] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [diamond,draw=armygreen,fill=armygreen,inner sep=1.25pt] ;,decorate,,armygreen,line width=0.75pt] table[x=T,y=SU5] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=9,fill=antiquebrass,draw=antiquebrass,inner sep=1.25pt] ;,decorate,,antiquebrass,line width=0.75pt] table[x=T,y=H] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [circle,draw=saffron,inner sep=1.25pt] ;,decorate,,saffron,line width=0.75pt] table[x=T,y=PL1] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [diamond,draw=schoolbusyellow,inner sep=1.25pt] ;,decorate,,schoolbusyellow,line width=0.75pt] table[x=T,y=PL2] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [regular polygon,regular polygon sides=5,draw=skyblue,inner sep=1.25pt] ;,decorate,,skyblue,line width=0.75pt] table[x=T,y=PL3] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [draw=forestgreen,inner sep=1.25pt] ;,decorate,,forestgreen,line width=0.75pt] table[x=T,y=C] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=7,draw=ufogreen,inner sep=1.25pt] ;,decorate,,ufogreen,line width=0.75pt] table[x=T,y=CE] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [regular polygon,regular polygon sides=9,draw=ufogreen,inner sep=1.25pt] ;,decorate,,ufogreen,line width=0.75pt] table[x=T,y=DC] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=9,draw=ufogreen,inner sep=1.25pt] ;,decorate,,ufogreen,line width=0.75pt] table[x=T,y=DV] figure_data/Linear.txt;

legend style=draw=none,
legend columns=2,
legend style=at=(1,-0.3),font=,
title  = Based on execution time,
ytick distance=0.2,
xtick distance=10,
ylabel style=xshift=5.5em,
ylabel = Proportion of solved problems,
xlabel style=yshift=0.5em,
xlabel = $\beta$
legend image code/.code=[circle,draw=darkblue,fill=darkblue,inner sep=1.25pt] ;

legend image code/.code=[star,star points=4,draw=blueryb,fill=blueryb,inner sep=1.25pt] ;

legend image code/.code=[draw=DarkRed,fill=DarkRed,inner sep=1.25pt] ;
legend image code/.code=[regular polygon,regular polygon sides=5,draw=red,fill=red,inner sep=1.25pt] ;
legend image code/.code=[diamond,draw=armygreen,fill=armygreen,inner sep=1.25pt] ;
legend image code/.code=[star,star points=9,fill=antiquebrass,draw=antiquebrass,inner sep=1.25pt] ;

legend image code/.code=[circle,draw=saffron,inner sep=1.25pt] ;
legend image code/.code=[diamond,draw=schoolbusyellow,inner sep=1.25pt] ;
legend image code/.code=[regular polygon,regular polygon sides=5,draw=skyblue,inner sep=1.25pt] ;
legend image code/.code=[draw=forestgreen,inner sep=1.25pt] ;

legend image code/.code=[star,star points=7,draw=ufogreen,inner sep=1.25pt] ;

legend image code/.code=[regular polygon,regular polygon sides=9,draw=ufogreen,inner sep=1.25pt] ;

legend image code/.code=[star,star points=9,draw=ufogreen,inner sep=1.25pt] ;

[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [circle,draw=darkblue,fill=darkblue,inner sep=1.25pt] ;,decorate,,darkblue,line width=0.75pt] table[x=T,y=NASt] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=4,draw=blueryb,fill=blueryb,inner sep=1.25pt] ;,decorate,,blueryb,line width=0.75pt] table[x=T,y=BARt] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [draw=DarkRed,fill=DarkRed,inner sep=1.25pt] ;,decorate,,DarkRed,line width=0.75pt] table[x=T,y=SU2t] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [regular polygon,regular polygon sides=5,draw=arsenic,fill=arsenic,inner sep=1.25pt] ;,decorate,,arsenic,line width=0.75pt] table[x=T,y=SU3t] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [diamond,draw=armygreen,fill=armygreen,inner sep=1.25pt] ;,decorate,,armygreen,line width=0.75pt] table[x=T,y=SU5t] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=9,fill=antiquebrass,draw=antiquebrass,inner sep=1.25pt] ;,decorate,,antiquebrass,line width=0.75pt] table[x=T,y=H] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [circle,draw=saffron,inner sep=1.25pt] ;,decorate,,saffron,line width=0.75pt] table[x=T,y=PL1t] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [diamond,draw=schoolbusyellow,inner sep=1.25pt] ;,decorate,,schoolbusyellow,line width=0.75pt] table[x=T,y=PL2t] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [regular polygon,regular polygon sides=5,draw=skyblue,inner sep=1.25pt] ;,decorate,,skyblue,line width=0.75pt] table[x=T,y=PL3t] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [draw=forestgreen,inner sep=1.25pt] ;,decorate,,forestgreen,line width=0.75pt] table[x=T,y=Ct] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=7,draw=ufogreen,inner sep=1.25pt] ;,decorate,,ufogreen,line width=0.75pt] table[x=T,y=CEt] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [regular polygon,regular polygon sides=9,draw=ufogreen,inner sep=1.25pt] ;,decorate,,ufogreen,line width=0.75pt] table[x=T,y=DCt] figure_data/Linear.txt;
[postaction=decoration=markings,mark=between positions 0 and 1 step 0.15 with [star,star points=9,draw=ufogreen,inner sep=1.25pt] ;,decorate,,ufogreen,line width=0.75pt] table[x=T,y=DVt] figure_data/Linear.txt;

Performance profiles on $[1, 200]$ of -type algorithms for problems with general and hidden constraints on the subset of constrained optimization test problems containing linear constraints.

§  PERFORMANCE ON ENGINEERING PROBLEMS

In this section, the algorithms from  were tested on eleven engineering design problems: tension/compression spring, three-bar truss, NASA speed reducer, pressure vessel, welded beam, and six different versions of the general nonlinear regression problem.
The general nonlinear regression problem is box-constrained, while the others involve different types of constraints.
Only the most promising algorithms (based on the results from <ref>) were considered.
As the global minimums are known for all these engineering problems, we used the same stopping rule in the previous experiments. A detailed description of all engineering problems and mathematical formulations is given in <ref>.

§.§ Tension/compression spring design problem

Here, we consider the tension-compression string design problem.
This problem aims to minimize the string weight under the constraints on deflection, shear stress, surge frequency, and limits on the outside diameter. A detailed description of the practical problem can be found in [Kazemi et al., 2011], while in <ref>
we give a short description and mathematical formulation.

A comparison of found solutions and performance metrics by the most promising algorithms from  is shown in <ref>.
Four considered algorithms were able to solve this problem.
Based on the number of function evaluation criteria, a slightly unexpectedly  was significantly better than other algorithms.
The surprise comes from the fact that the algorithm does not use any information about the constraint functions.

However, the  algorithm was approximate $4$ times slower than the second-best method .

Performance of the most promising -type algorithms from  on a tension/compression design problem
Algorithm (input)       Iterations     $f_{eval}$     Time (s)     $f_{min}$

$297$        $17,659$ 	     $77.46$        $0.012680$

$40,478$     $2,000,000$     $2,465.81$     $0.012867$

($sub = 5$)  	    $15,626$     $2,000,000$     $722.79$       $0.012867$

$700$        $2,000,000$     $214.81$       $0.012683$

$675$        $423,209$ 	     $47.53$        $0.012680$

$624$        $178,115$       $20.45$        $0.012680$

$624$        $178,115$       $20.70$        $0.012680$

($p = 10^3$)       $33,571$     $2,000,000$     $3,041.00$     $0.012867$

§.§ Three-bar truss design problem

Here, we consider the three-bar truss design problem.
The goal is to minimize the volume subject to stress constraints.
The detailed description of the problem is given in [Ray and Liew, 2003], while in <ref>, we provide a brief description and mathematical formulation.

A comparison of found solutions and performance metrics is shown in <ref>.
Here, hybridized  was the most efficient optimizer.
However, none of the algorithms had any difficulty solving this problem, and they found the solution in less than one-second time.

Performance of the most promising -type algorithms from  on a three-bar truss design problem
Algorithm (input)       Iterations     $f_{eval}$     Time (s)     $f_{min}$

$29$     $339$       $0.05$     $263.915790$

$13$     $125$       $0.02$     $263.915790$

($sub = 5$)  	    $14$     $161$       $0.02$     $263.915790$

$12$     $231$       $0.03$     $263.915790$

$17$     $727$       $0.08$     $263.911750$

$33$     $1,055$     $0.13$     $263.915790$

$6$      $93$        $0.03$     $263.895850$

($p = 10^3$)       $17$     $179$       $0.03$     $263.915790$

§.§ NASA speed reducer design problem

Here we consider the NASA speed reducer design problem. The goal is to minimize the overall weight subject to constraints on the gear teeth' bending stress, surface stress, transverse deflection of the shaft, and stresses in the shafts.
A detailed description of the problem can be found in [Ray and Liew, 2003], while in <ref>, we provide a short description and mathematical formulation.

A comparison of the found solutions and performance metrics is shown in <ref>.
Only two algorithms ( and ) were able to tackle this problem.
Again, the hybridized  algorithm showed the best performance.
Note that the found solution with  is better than the best know value $f_{min}$.
However, the reported solution point is outside the feasible region and violates a few constraints.

Performance of the most promising -type algorithms from  on a NASA speed reducer design problem
Algorithm (input)       Iterations     $f_{eval}$     Time (s)     $f_{min}$

$6,719$      $325,691$ 	     $43,145.14$     $3006.874789$

$32,031$     $2,000,000$     $2,625.88$      $3006.838136$

($sub = 5$)  	    $11,112$     $2,000,000$     $1,234.33$      $3006.838136$

$528$        $2,000,000$     $169.74$        $3003.135167$

$1623$       $2,000,000$     $373.40$        $3002.869474$

$254$        $123,175$ 	     $9.70$  	       $2996.572800$

$55$         $10,229$ 	     $0.93$  	       $2996.348212$

($p = 10^3$)       $661$        $26,667$        $3.01$ 	       $2995.382568^a$

5la – result is outside the feasible region

§.§ Pressure vessel design problem

In this subsection, we consider a pressure vessel design problem, and the goal is to minimize the total cost of the material, forming, and and to weld a cylindrical vessel.
A detailed description of the problem can be found in [Kazemi et al., 2011], while in <ref>, we provide a short description and mathematical formulation.

A comparison of the found solutions and performance metrics is shown in <ref>.
Four algorithms solved this problem: , , , and , and hybridized  was the most efficient optimizer again.
is the best performing and outperformed the second-best by approximately $1.8$ times, among traditional -type algorithms.
However, the  algorithm was about $26$ times slower than the second-best method ().
As in the previous case, the  returned a better than the best know value $f_{min}$, but the solution point lies outside the feasible region again.

Performance of the most promising -type algorithms from  on a pressure vessel design problem
Algorithm (input)       Iterations     $f_{eval}$     Time (s)     $f_{min}$

$273$        $31,081$ 	     $126.64$        $7164.437307$

$26,379$     $2,000,000$     $1,600.150$     $7234.041903$

($sub = 5$)  	    $15,626$     $2,000,000$     $1,328.98$      $7234.041903$

$252$        $55,837$ 	     $4.80$  	       $7164.437300$

$2,358$      $2,000,000$     $433.87$        $7224.704257$

$322$        $88,585$ 	     $8.52$  	       $7164.437301$

$5$  	       $91$ 		     $0.05$  	       $7163.739611$

($p = 10^3$)       $87$         $2,295$ 	     $0.23$  	       $7152.303079^a$

5la – result is outside the feasible region

§.§ Welded beam design problem

The fifth engineering problem is the welded beam design.
Here, the goal is to minimize a welded beam for a minimum cost, subject to seven constraints.
The detailed description is presented in [Mirjalili and Lewis, 2016, Mirjalili et al., 2014], while in <ref>, we provide a short description and mathematical formulation.

A comparison of the algorithms is shown in <ref>.
In total, five algorithms were able to solve the problem, and once again, the  was the most efficient one.
Again, the  algorithm showed the best performance (based on the total number of function evaluations) among traditional -type algorithms, but significantly suffered based on the execution time.

Performance of the most promising -type algorithms from  on a welded beam design problem
Algorithm (input)       Iterations     $f_{eval}$     Time (s)     $f_{min}$

$698$         $86,863$ 		    $4692.00$      $1.724970$

$22,934$ 	    $2,000,000$ 	    $1,363.24$     $1.728488$

($sub = 5$)  	  	    $10,954$ 	    $2,000,000$ 	    $860.06$       $1.728037$

$189$  	    $158,747$    	    $11.70$        $1.724970$

$211$ 	    $108,683$   	    $9.19$         $1.724970$

$366$  	    $104,191$   	    $9.80$         $1.724970$

$3$  		    $163$       	    $0.06$         $1.724884$

($p = 10^3$)   	    $20,767$      $2,000,000$       $1,679.76$     $1.728488$

§.§ General nonlinear regression problem

In the final part, a general nonlinear regression design problem is considered in the form of fitting a sum of damped sinusoids to a series of observations.
The detailed description of the problem can be found in [Gillard and Kvasov, 2017, Paulavičius et al., 2020, Sergeyev et al., 2016], while in <ref>,
we provide a short description and mathematical formulation.
The problem is multimodal and is considered challenging, especially with the increase in the number of samples ($T$).
The higher number of sinusoids ($s$) leads to a more accurate but at the same time and more challenging optimization problem.

Performance of the most promising -type algorithms from  on nonlinear regression design problem
1c     4c$s=1, T=10$      4c$s=2, T=10$      4c$s=3, T=10$

Algorithm     Iteration     $f_{eval}$     Time(s)     $f_{min}$     Iteration     $f_{eval}$     Time(s)     $f_{min}$     Iteration     $f_{eval}$     Time(s)     $f_{min}$

$31$      $5,061$      $0.61$      $0.000087$     $311$       $2 \times 10^6$     $227.87$       $0.005848$     $213$        $2 \times 10^6$     $201.36$       $0.099088$

$27$      $1,617$      $0.12$      $0.000021$     $184$       $62,965$ 		    $2.80$ 	     $0.000099$     $652$ 	     $543,207$ 	       $35.01$ 	    $0.000086$

$32$      $501$        $0.24$      $0.000021$     $9,328$     $2 \times 10^6$     $1,125.17$     $0.001228$     $3,186$      $2 \times 10^6$     $553.10$       $0.007295$

$35$      $305$ 	     $0.04$      $0.000021$     $203,119$     $2 \times 10^6$     $7,540.74$     $0.018320$     $93,457$     $2 \times 10^6$     $2,590.26$     $0.009475$

$32$      $501$        $0.24$      $0.000021$     $2,805$     $1,978,935$         $762.10$       $0.000097$     $1,415$      $2 \times 10^6$     $453.64$       $0.021190$

$58$      $616$        $0.43$      $0.000094$     $8,729$     $471,930$ 	       $438.66$       $0.000096$     $36,981$     $2 \times 10^6$     $7,780.35$     $0.004731$

$5$       $101$        $0.03$      $1.04 \times 10^{-13}$     $15$     $316$     $0.03$     $3.82 \times 10^{-7}$     $85,458$     $722,243$     $5,915.73$     $8.40 \times 10^{-9}$

$151$     $3,024$      $3.19$      $0.000086$     $178,562$     $43,215.10$     $0.014843$     $0.014843$     $1$     $2 \times 10^6$     $43,000.00$     N/A

4c$s=1, T=100$      4c$s=2, T=100$      4c$s=3, T=100$

$36$        $6,921$      $1.54$      $0.000091$     $311$        $2 \times 10^6$     $278.63$       $0.006979$     $213$        $2 \times 10^6$     $280.41$       $0.117012$

$29$        $1,829$      $0.19$      $0.000025$     $188$        $65,645$            $3.63$ 	      $0.000097$     $251$ 	      $158,989$ 		    $9.60$ 	     $0.000099$

$89$        $1,923$      $0.84$      $0.000025$     $9,191$      $2 \times 10^6$     $1,178.59$     $0.001696$     $3,722$      $2 \times 10^6$     $745.02$       $0.034532$

$6,811$     $45,423$     $10.90$     $0.000025$     $203,159$     $2 \times 10^6$     $8,823.94$     $0.020558$     $91,574$     $2 \times 10^6$     $2,617.19$     $0.041690$

$89$        $1,923$      $0.82$      $0.000025$     $2,602$      $1,713,841$         $674.43$       $0.000096$     $1,273$      $2 \times 10^6$     $504.28$       $0.033823$

$86$        $1,042$      $0.86$      $0.000096$     $13,028$     $675,332$           $792.08$       $0.000097$     $36,473$     $2 \times 10^6$     $7,482.06$     $0.019857$

$5$         $105$        $0.02$      $1.15 \times 10^{-13}$     $15$     $292$     $0.06$     $2.23 \times 10^{-7}$     $89,360$     $730,887$     $7,692.91$     $2.30 \times 10^{-9}$

$180$       $2,344$      $2.04$      $0.000086$     $6,840$     $185,135$     $43,000.00$     $0.018025$     $1$     $2 \times 10^6$     $43,000.00$     N/A

5lN/A – not available

Our experiments have used three different values for $s = 1, 2,$ and $3$ (correspond to $3, 6,$ and $9$-dimensional problems) and two different values, $T = 10$ and $T = 100$ for each dimension $n$ as was done in [Paulavičius et al., 2020].

The obtained results are summarized in <ref>.
Solving the lowest dimension $(n=3)$ cases (corresponding to $s=1, T=10$ and $s=1, T=100$) all algorithms located the solutions correctly.
Excluding the hybridized , the  algorithm was the most efficient one.
However, in solving bigger conditionality problems $(n=6)$, half of the algorithms failed to find the correct solution.
Finally, only  two tested algorithms solved all nonlinear regression design problems:  and .

§ CONCLUSION

This paper has introduced a new open-source -type  toolbox () for derivative-free global optimization.
The new toolbox combines various state-of-the-art -type algorithms for the global solution of box-constrained, generally-constrained, and optimization problems with hidden constraints.
All algorithms were implemented using two different data structures: static and dynamic.
Additionally, several parallel schemes were adopted to promising algorithms.
Furthermore, an online test library , containing 119 global optimization test and engineering problems, has been presented.

The performance of various algorithms within  has been investigated via a detailed numerical study using the test problems from .
A further 11 examples of the use of the  for engineering design optimization have been investigated.
The results demonstrate the promising performance of  in tackling these challenging problems.
We also gave guidance on which algorithms to use for certain types of optimization problems.

Motivated by the promising performance, we plan to extend this work to facilitate the broader adoption of .

We plan to include newly appearing promising -type algorithms within this toolbox continuously.
Another direction is extending the developed algorithms using a hybrid CPU-GPU scheme.
Finally, we will consider advanced data structures for better data organization and reduced communication overhead.

§ SOURCE CODE STATEMENT

All implemented -type algorithms ( toolbox) are available at the GitHub repository: <https://github.com/blockchain-group/DGO>, and can be used under GPL version 3.

§ DATA STATEMENT

Data underlying this article () can be accessed on Zenodo at <https://dx.doi.org/10.5281/zenodo.1218980>, and used under the Creative Commons Attribution license.

[Baker et al., 2000]
authorBaker, C.A., authorWatson, L.T.,
authorGrossman, B., authorMason, W.H.,
authorHaftka, R.T., year2000.
titleParallel global aircraft configuration design space
exploration, in: editorTentner, A. (Ed.),
booktitleHigh Performance Computing Symposium 2000,
organizationSoc. for Computer Simulation Internat. pp.

[Bartholomew-Biggs et al., 2002]
authorBartholomew-Biggs, M.C., authorParkhurst, S.C.,
authorWilson, S.P., year2002.
titleUsing DIRECT to solve an aircraft routing problem.
journalComputational Optimization and Applications
volume21, pages311–323.

[Basudhar et al., 2012]
authorBasudhar, A., authorDribusch, C.,
authorLacaze, S., authorMissoum, S.,
titleConstrained efficient global optimization with
support vector machines.
journalStructural and Multidisciplinary Optimization
volume46, pages201–221.

[Birgin et al., 2010]
authorBirgin, E.G., authorFloudas, C.A.,
authorMartínez, J.M., year2010.
titleGlobal minimization using an augmented lagrangian
method with variable lower-level constraints.
journalMathematical Programming volume125,

[Björkman and Holmström, 1999]
authorBjörkman, M., authorHolmström, K.,
titleGlobal optimization using the DIRECT algorithm in
journalAdvanced Modeling and Optimization
volume1, pages17–37.

[Cagnina et al., 2008]
authorCagnina, L.C., authorEsquivel, S.C.,
authorCoello, C.A.C., year2008.
titleSolving engineering optimization problems with the
simple constrained particle swarm optimizer.
journalInformatica (Ljubljana) volume32,

[Carter et al., 2001]
authorCarter, R.G., authorGablonsky, J.M.,
authorPatrick, A., authorKelley, C.T.,
authorEslinger, O.J., year2001.
titleAlgorithms for noisy problems in gas transmission
pipeline optimization.
journalOptimization and Engineering volume2,

[Characklis et al., 2006]
authorCharacklis, G.W., authorKirsch, B.R.,
authorRamsey, J., authorDillard, K.E.,
authorKelley, C.T., year2006.
titleDeveloping portfolios of water supply transfers.
journalWater Resources Research

[Chen and Kelley, 2016]
authorChen, X., authorKelley, C.T.,
titleOptimization with hidden constraints and embedded
Monte Carlo computations.
journalOptimization and Engineering volume17,

[Choi et al., 2000]
authorChoi, T.D., authorEslinger, O.J.,
authorKelley, C.T., authorDavid, J.W.,
authorEtheridge, M., year2000.
titleOptimization of Automotive Valve Train Components
with Implicit Filtering.
journalOptimization and Engineering volume1,

[Clerc, 1999]
authorClerc, M., year1999.
titleThe swarm and the queen: Towards a deterministic and
adaptive particle swarm optimization, in: booktitleProceedings
of the 1999 Congress on Evolutionary Computation, CEC 1999, pp.

[Costa et al., 2018]
authorCosta, M.F.P., authorRocha, A.M.A.C.,
authorFernandes, E.M.G.P., year2018.
titleFilter-based direct method for constrained global
journalJournal of Global Optimization
volume71, pages517–536.

[Cox et al., 2001]
authorCox, S.E., authorHaftka, R.T.,
authorBaker, C.A., authorGrossman, B.,
authorMason, W.H., authorWatson, L.T.,
titleA comparison of global optimization methods for the
design of a high-speed civil transport.
journalJournal of Global Optimization
volume21, pages415–432.

[David et al., 1996]
authorDavid, J.W., authorKelley, C.T.,
authorCheng, C., year1996.
titleUse of an implicit filtering algorithm for mechanical
system parameter identification.
typeTechnical Report. SAE Technical Paper.

[Di Serafino et al., 2011]
authorDi Serafino, D., authorLiuzzi, G.,
authorPiccialli, V., authorRiccio, F.,
authorToraldo, G., year2011.
titleA modified DIviding RECTangles algorithm for a
problem in astrophysics.
journalJournal of Optimization Theory and Applications
volume151, pages175–190.

[Dixon and Szegö, 1978]
authorDixon, L., authorSzegö, C.,
titleThe global optimisation problem: An introduction,
in: editorDixon, L., editorSzegö, G. (Eds.),
booktitleTowards Global Optimization.
publisherNorth-Holland Publishing Company.
volume volume2, pp. pages1–15.

[Dolan and Moré, 2002]
authorDolan, E.D., authorMoré, J.J.,
titleBenchmarking optimization software with performance
journalMathematical Programming volume91,

[Finkel, 2004]
authorFinkel, D.E., year2004.
titleMATLAB source code for DIRECT.
howpublished<http://www4.ncsu.edu/ ctk/Finkel_Direct/>.
noteOnline; accessed: 2017-03-22.

[Finkel, 2005]
authorFinkel, D.E., year2005.
titleGlobal Optimization with the Direct
Ph.D. thesis. North Carolina State University.

[Finkel and Kelley, 2004]
authorFinkel, D.E., authorKelley, C.T.,
titleAn adaptive restart implementation of direct.
journalTechnical report CRSC-TR04-30, Center for Research
in Scientific Computation, North Carolina State University, Raleigh .

[Finkel and Kelley, 2006]
authorFinkel, D.E., authorKelley, C.T.,
titleAdditive scaling and the DIRECT algorithm.
journalJournal of Global Optimization
volume36, pages597–608.

[Gablonsky, 2001]
authorGablonsky, J.M., year2001.
titleModifications of the DIRECT Algorithm.
Ph.D. thesis. North Carolina State University.

[Gablonsky and Kelley, 2001]
authorGablonsky, J.M., authorKelley, C.T.,
titleA locally-biased form of the DIRECT algorithm.
journalJournal of Global Optimization
volume21, pages27–37.

[Gillard and Kvasov, 2017]
authorGillard, J.W., authorKvasov, D.E.,
titleLipschitz optimization methods for fitting a sum of
damped sinusoids to a series of observations.
journalStatistics and its Interface

[Grbić et al., 2013]
authorGrbić, R., authorNyarko, E.K.,
authorScitovski, R., year2013.
titleA modification of the direct method for
Lipschitz global optimizatio n for a symmetric function.
journalJournal of Global Optimization
volume57, pages1193–1212.

[He et al., 2009]
authorHe, J., authorVerstak, A.,
authorSosonkina, M., authorWatson, L.T.,
titlePerformance modeling and analysis of a massively
parallel DIRECT–Part 2.
journalThe International Journal of High Performance
Computing Applications volume23, pages29–41.

[He et al., 2009]
authorHe, J., authorVerstak, A.,
authorWatson, L.T., authorSosonkina, M.,
titlePerformance modeling and analysis of a massively
parallel DIRECT–part 1.
journalThe International Journal of High Performance
Computing Applications volume23, pages14–28.

[He et al., 2002]
authorHe, J., authorWatson, L.T.,
authorRamakrishnan, N., authorShaffer, C.A.,
authorVerstak, A., authorJiang, J.,
authorBae, K., authorTranter, W.H.,
titleDynamic data structures for a DIRECT search
journalComputational Optimization and Applications
volume23, pages5–25.

[He et al., 2010]
authorHe, J., authorWatson, L.T.,
authorSosonkina, M., year2010.
titleAlgorithm 897: VTDIRECT95: Serial and Parallel Codes
for the Global Optimization Algorithm DIRECT.
journalACM Transactions on Mathematical Software

[Hedar, 2005]
authorHedar, A., year2005.
titleTest functions for unconstrained global
noteOnline; accessed: 2017-03-22.

[Horst et al., 1995]
authorHorst, R., authorPardalos, P.M.,
authorThoai, N.V., year1995.
titleIntroduction to Global Optimization.
Nonconvex Optimization and Its Application,
publisherKluwer Academic Publishers.

[John, 1975]
authorJohn, H., year1975.
titleAdaptation in natural and artificial systems.
journalThe University of Michigan Press, Ann Arbor .

[Jones, 2001]
authorJones, D.R., year2001.
titleThe Direct global optimization algorithm,
in: editorFloudas, C.A., editorPardalos, P.M. (Eds.),
booktitleThe Encyclopedia of Optimization.
publisherKluwer Academic Publishers, addressDordrect,
pp. pages431–440.

[Jones and Martins, 2021]
authorJones, D.R., authorMartins, J.R.R.A.,
titleThe DIRECT algorithm: 25 years later.
journalJournal of Global Optimization
volume79, pages521–566.

[Jones et al., 1993]
authorJones, D.R., authorPerttunen, C.D.,
authorStuckman, B.E., year1993.
titleLipschitzian optimization without the Lipschitz
journalJournal of Optimization Theory and Application
volume79, pages157–181.

[Kazemi et al., 2011]
authorKazemi, M., authorWang, G.G.,
authorRahnamayan, S., authorGupta, K.,
titleMetamodel-based optimization for problems with
expensive objective and constraint functions.
journalJournal of Mechanical Design
volume133, pages014505.

[Kennedy and Eberhart, 1995]
authorKennedy, J., authorEberhart, R.,
titleParticle swarm optimization.
journalIn Proceedings of the IEEE international conference
on neural networks IV ,

[Kirkpatrick et al., 1983]
authorKirkpatrick, S., authorGelatt, C.D.,
authorVecchi, M.P., year1983.
titleOptimization by simulated annealing.

[Kirsch et al., 2009]
authorKirsch, B.R., authorCharacklis, G.W.,
authorDillard, K.E., authorKelley, C.T.,
titleMore efficient optimization of long-term water
supply portfolios.
journalWater Resources Research

[Liu et al., 2017]
authorLiu, H., authorXu, S., authorChen,
X., authorWang, X., authorMa, Q.,
titleConstrained global optimization via a direct-type
constraint-handling technique and an adaptive metamodeling strategy.
journalStructural and Multidisciplinary Optimization
volume55, pages155–177.

[Liu et al., 2015]
authorLiu, H., authorXu, S., authorWang,
X., authorWu, X., authorSong, Y.,
titleA global optimization algorithm for simulation-based
problems via the extended direct scheme.
journalEngineering Optimization volume47,

[Liu, 2013]
authorLiu, Q., year2013.
titleLinear scaling and the direct algorithm.
journalJournal of Global Optimization
volume56, pages1233–1245.

[Liu and Cheng, 2014]
authorLiu, Q., authorCheng, W., year2014.
titleA modified DIRECT algorithm with bilevel
journalJournal of Global Optimization
volume60, pages483–499.

[Liu et al., 2017]
authorLiu, Q., authorYang, G., authorZhang,
Z., authorZeng, J., year2017b.
titleImproving the convergence rate of the DIRECT global
optimization algorithm.
journalJournal of Global Optimization
volume67, pages851–872.

[Liu et al., 2015]
authorLiu, Q., authorZeng, J., authorYang,
G., year2015b.
titleMrDIRECT: a multilevel robust DIRECT algorithm for
global optimization problems.
journalJournal of Global Optimization
volume62, pages205–227.

[Liuzzi et al., 2010]
authorLiuzzi, G., authorLucidi, S.,
authorPiccialli, V., year2010a.
titleA DIRECT-based approach exploiting local
minimizations for the solution of large-scale global optimization problems.
journalComputational Optimization and Applications
volume45, pages353–375.

[Liuzzi et al., 2010]
authorLiuzzi, G., authorLucidi, S.,
authorPiccialli, V., year2010b.
titleA partition-based global optimization algorithm.
journalJournal of Global Optimization
volume48, pages113–128.

[Luszczek, 2009]
authorLuszczek, P., year2009.
titleParallel programming in MATLAB.
journalInternational Journal of High Performance Computing
Applications volume23, pages277–283.

[Matlab, 2020]
authorMatlab, year2020.
titleParallel Computing Toolbox ™ User ' s Guide.
journalBook , pages1–729.

[Mirjalili and Lewis, 2016]
authorMirjalili, S., authorLewis, A.,
titleThe Whale Optimization Algorithm.
journalAdvances in Engineering Software

[Mirjalili et al., 2014]
authorMirjalili, S., authorMirjalili, S.M.,
authorLewis, A., year2014.
titleGrey Wolf Optimizer.
journalAdvances in Engineering Software

[Mockus et al., 2017]
authorMockus, J., authorPaulavičius, R.,
authorRusakevičius, D., authorŠešok,
D., authorŽilinskas, J., year2017.
titleApplication of Reduced-set Pareto-Lipschitzian
Optimization to truss optimization.
journalJournal of Global Optimization
volume67, pages425–450.

[Na et al., 2017]
authorNa, J., authorLim, Y., authorHan, C.,
titleA modified DIRECT algorithm for hidden constraints
in an LNG process optimization.
journalEnergy ,

[Paulavičius et al., 2018]
authorPaulavičius, R., authorChiter, L.,
authorŽilinskas, J., year2018.
titleGlobal optimization based on bisection of
rectangles, function values at diagonals, and a set of Lipschitz constants.
journalJournal of Global Optimization
volume71, pages5–20.

[Paulavičius et al., 2014]
authorPaulavičius, R., authorSergeyev, Y.D.,
authorKvasov, D.E., authorŽilinskas, J.,
titleGlobally-biased DISIMPL algorithm for expensive
global optimization.
journalJournal of Global Optimization
volume59, pages545–567.

[Paulavičius et al., 2020]
authorPaulavičius, R., authorSergeyev, Y.D.,
authorKvasov, D.E., authorŽilinskas, J.,
titleGlobally-biased BIRECT algorithm with local
accelerators for expensive global optimization.
journalExpert Systems with Applications
volume144, pages11305.

[Paulavičius and Žilinskas, 2006]
authorPaulavičius, R., authorŽilinskas, J.,
titleAnalysis of different norms and corresponding
Lipschitz constants for global optimization.
journalTechnological and Economic Development of Economy
volume36, pages383–387.

[Paulavičius and Žilinskas, 2007]
authorPaulavičius, R., authorŽilinskas, J.,
titleAnalysis of different norms and corresponding
Lipschitz constants for global optimization in multidimensional case.
journalInformation Technology and Control
volume36, pages383–387.

[Paulavičius and Žilinskas, 2013]
authorPaulavičius, R., authorŽilinskas, J.,
titleSimplicial Lipschitz optimization without the
Lipschitz constant.
journalJournal of Global Optimization
volume59, pages23–40.

[Paulavičius and Žilinskas, 2014]
authorPaulavičius, R., authorŽilinskas, J.,
titleSimplicial Global Optimization.
SpringerBriefs in Optimization, publisherSpringer New
York, addressNew York, NY.

[Paulavičius and Žilinskas, 2016]
authorPaulavičius, R., authorŽilinskas, J.,
titleAdvantages of simplicial partitioning for Lipschitz
optimization problems with linear constraints.
journalOptimization Letters volume10,

[Pillo et al., 2016]
authorPillo, G.D., authorLiuzzi, G.,
authorLucidi, S., authorPiccialli, V.,
authorRinaldi, F., year2016.
titleA DIRECT-type approach for derivative-free
constrained global optimization.
journalComputational Optimization and Applications
volume65, pages361–397.

[Pillo et al., 2010]
authorPillo, G.D., authorLucidi, S.,
authorRinaldi, F., year2010.
titleAn approach to constrained global optimization based
on exact penalty functions.
journalJournal of Optimization Theory and Applications
volume54, pages251–260.

[Pintér, 1996]
authorPintér, J.D., year1996.
titleGlobal optimization in action: continuous and
Lipschitz optimization: algorithms, implementations and applications.
volume volume6 of seriesNonconvex Optimization
and Its Applications.
publisherSpringer US.

[Piyavskii, 1967]
authorPiyavskii, S.A., year1967.
titleAn algorithm for finding the absolute minimum of a
journalTheory of Optimal Solutions volume2,
10.1016/0041-5553(72)90115-2. notein

[Preparata and Shamos, 1985]
authorPreparata, F.P., authorShamos, M.,
titleComputational Geometry.
Monographs in Computer Science, publisherSpringer-Verlag
New York.

[Ray and Liew, 2003]
authorRay, T., authorLiew, K.M., year2003.
titleSociety and civilization: An optimization algorithm
based on the simulation of social behavior.
journalIEEE Transactions on Evolutionary Computation
volume7, pages386–396.

[Rios and Sahinidis, 2007]
authorRios, L.M., authorSahinidis, N.V.,
titleDerivative-free optimization: a review of algorithms
and comparison of software implementations.
journalJournal of Global Optimization
volume56, pages1247–1293.

[Sergeyev and Kvasov, 2006]
authorSergeyev, Y.D., authorKvasov, D.E.,
titleGlobal search based on diagonal partitions and a set
of Lipschitz constants.
journalSIAM Journal on Optimization volume16,

[Sergeyev and Kvasov, 2008]
authorSergeyev, Y.D., authorKvasov, D.E.,
titleDiagonal Global Optimization Methods.
publisherFizMatLit, addressMoscow.
noteIn Russian.

[Sergeyev and Kvasov, 2011]
authorSergeyev, Y.D., authorKvasov, D.E.,
titleLipschitz global optimization, in:
editorCochran, J.J., editorCox, L.A.,
editorKeskinocak, P., editorKharoufeh, J.P.,
editorSmith, J.C. (Eds.), booktitleWiley Encyclopedia
of Operations Research and Management Science (in 8 volumes).
publisherJohn Wiley & Sons, addressNew York.
volume volume4, pp. pages2812–2828.

[Sergeyev and Kvasov, 2017]
authorSergeyev, Y.D., authorKvasov, D.E.,
titleDeterministic Global Optimization: An Introduction to
the Diagonal Approach.
SpringerBriefs in Optimization, publisherSpringer.

[Sergeyev et al., 2016]
authorSergeyev, Y.D., authorKvasov, D.E.,
authorMukhametzhanov, M.S., year2016.
titleOn the least-squares fitting of data by sinusoids.
journalSpringer Optimization and Its Applications

[Shubert, 1972]
authorShubert, B.O., year1972.
titleA sequential method seeking the global maximum of a
journalSIAM Journal on Numerical Analysis
volume9, pages379–388.

[Stoneking et al., 1992]
authorStoneking, D.E., authorBilbro, G.L.,
authorGilmore, P.A., authorTrew, R.J.,
authorKelley, C.T., year1992.
titleYield optimization using a gaas process simulator
coupled to a physical device model.
journalIEEE Transactions on Microwave Theory and
Techniques volume40, pages1353–1363.

[Stripinis and Paulavičius, 2021]
authorStripinis, L., authorPaulavičius, R.,
titleDGO: A new DIRECT-type MATLAB toolbox for
derivative-free global optimization.

[Stripinis and Paulavičius, 2021]
authorStripinis, L., authorPaulavičius, R.,
titleA new  algorithm for global optimization
with hidden constraints.
journalOptimization Letters ,

[Stripinis et al., 2018]
authorStripinis, L., authorPaulavičius, R.,
authorŽilinskas, J., year2018.
titleImproved scheme for selection of potentially optimal
hyper-rectangles in DIRECT.
journalOptimization Letters volume12,

[Stripinis et al., 2019]
authorStripinis, L., authorPaulavičius, R.,
authorŽilinskas, J., year2019.
titlePenalty functions and two-step selection procedure
based -type algorithm for constrained global optimization.
journalStructural and Multidisciplinary Optimization
volume59, pages2155–2175.

[Stripinis and Paulavičius, 2020]
authorStripinis, L., authorPaulavičius, R.,
titleDIRECTLib – a library of global optimization
problems for DIRECT-type methods, v1.2.

[Stripinis et al., 2020]
authorStripinis, L., authorŽilinskas, J.,
authorCasado, L.G., authorPaulavičius, R.,
titleOn  experience in accelerating
algorithm for constrained global optimization through dynamic data structures
and parallelization.
journalApplied Mathematics and Computation ,

[Strongin and Sergeyev, 2000]
authorStrongin, R.G., authorSergeyev, Y.D.,
titleGlobal Optimization with Non-Convex Constraints:
Sequential and Parallel Algorithms.
publisherKluwer Academic Publishers,

[Suganthan et al., 2005]
authorSuganthan, P.N., authorHansen, N.,
authorLiang, J.J., authorDeb, K.,
authorY.P.Chen, authorAuger, A.,
authorTiwari, S., year2005.
titleProblem definitions and evaluation criteria for the
cec 2006 special session on constrained real-parameter optimization.
journalKanGAL , pages251–256.

[Surjanovic and Bingham, 2013]
authorSurjanovic, S., authorBingham, D.,
titleVirtual library of simulation experiments: Test
functions and datasets.
howpublished<http://www.sfu.ca/ ssurjano/index.html>.
noteOnline; accessed: 2017-03-22.

[Vaz and Vicente, 2009]
authorVaz, A., authorVicente, L., year2009.
titlePswarm: a hybrid solver for linearly constrained
global derivative-free optimization.
journalOptimization Methods and Software
volume24, pages669–685.

[Watson and Baker, 2001]
authorWatson, L.T., authorBaker, C.A.,
titleA fully-distributed parallel global search algorithm.
engineering computations.
journalEngineering Computations volume18,

§  LIBRARY

A summary of all optimization problems in  and their properties is given in <ref>.
The first column denotes the problem type, and the second contains the problem name.
The third column contains the source of the problem.
The fourth through eighth columns ($n$,  $\mathbf{g}$,  $\mathbf{h}$, $a$, $D$) has the dimension ($n$), number of inequality ($\mathbf{g}$), and equality ($\mathbf{h}$) constraints, number of active constraints ($a$), and optimization domain ($D$), respectively.
Finally, the last column contains the best known optimal solution ($f^*$).

Key characteristics of the  [Stripinis and Paulavičius, 2020] test problems for box-constrained global optimization

2cProblem     2*Source    	5cProblem properties	     2*$ f^* $

0-1 4-8
Type     Name          $n$     $ \mathbf{g} $     $ \mathbf{h} $     $ a $     $ D $

Table  Continued from previous page

2cProblem     2*Source    	5cProblem properties	     2*$ f^* $

0-1 4-8
Type     Name          $n$     $ \mathbf{g} $     $ \mathbf{h} $     $ a $     $ D $

9cContinued on next page

BC     Ackley     	 	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-15, 30]_i, i = 1...n$     	   	     $0.0000$

Alpine     	 	    [Clerc, 1999] 		  	 			    $n$      $0$      $0$      $0$      $[0, 10]_i, i = 1...n$      	         $-2.8081^n$

Beale      	 	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-4.5, 4.5]_i, i = 1...n$   	   	     $0.0000$

Bohachevsky $1$ 	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-100, 110]_i, i = 1...n$   	   	     $0.0000$

Bohachevsky $2$ 	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-100, 110]_i, i = 1...n$   	   	     $0.0000$

Bohachevsky $3$ 	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-100, 110]_i, i = 1...n$   	   	     $0.0000$

Booth          	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-10, 10]_i, i = 1...n$  	 	     $0.0000$

Branin          	    [Hedar, 2005, Dixon and Szegö, 1978, Surjanovic and Bingham, 2013] 	    $2$      $0$      $0$      $0$      $[-5, 10]_1,[10,15]_2$  			     $0.3978$

Bukin           	    [Surjanovic and Bingham, 2013] 		  	 			    $2$      $0$      $0$      $0$      $[-15, 5]_1,[-3,3]_2$   			     $0.0000$

Csendes     	 	    [Clerc, 1999] 		  	 			    $n$      $0$      $0$      $0$      $[-10, 20]_i, i = 1...n$       	     $0.0000$

Colville        	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $4$      $0$      $0$      $0$      $[-10, 10]_i, i = 1...n$               $0.0000$

Cross-in-Tray   	    [Surjanovic and Bingham, 2013] 		  	 			    $2$      $0$      $0$      $0$      $[-10, 10]_i, i = 1...n$               $-2.0626$

Dixon-Price   	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-10, -10]_i, i = 1...n$              $0.0000$

Drop-Wave   	 	    [Surjanovic and Bingham, 2013] 		  	 			    $2$      $0$      $0$      $0$      $[-5.12, -6.12]_i, i = 1...n$          $-1.0000$

Easom           	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-100, 100]_i, i = 1...n$             $-1.0000$

Eggholder       	    [Surjanovic and Bingham, 2013] 		  	 			    $2$      $0$      $0$      $0$      $[-512, 512]_i, i = 1...n$             $-959.6406$

Goldstein & Price     [Hedar, 2005, Dixon and Szegö, 1978, Surjanovic and Bingham, 2013]     $2$     $0$      $0$      $0$      $[-2, 2]_i, i = 1...n$           	     $3.0000$

Griewank             [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-600, 700]_i, i = 1...n$             $0.0000$

Hartman $3$          [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $3$      $0$      $0$      $0$      $[0, 1]_i, i = 1...n$                  $-3.8627$

Hartman $6$          [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $6$      $0$      $0$      $0$      $[0, 1]_i, i = 1...n$            	     $-3.3223$

Holder Table         [Surjanovic and Bingham, 2013] 			 			    $2$      $0$      $0$      $0$      $[-10, 10]_i, i = 1...n$               $-19.2085$

Hump                 [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-5, 5]_i, i = 1...n$           	     $-1.0316$

Langermann           [Surjanovic and Bingham, 2013] 			 			    $2$      $0$      $0$      $0$      $[0, 10]_i, i = 1...n$           	     $-4.1558$

Levy                 [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-5, 5]_i, i = 1...n$                 $0.0000$

Matyas          	    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-10, 10]_i, i = 1...n$  		     $0.0000$

McCormick            [Surjanovic and Bingham, 2013] 	 		 			    $2$      $0$      $0$      $0$      $[-1.5, 4]_1,[-3,4]_2$  			     $-1.9132$

Michalewicz          [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[0, \pi]_i, i = 1...n$                $-1.8013$

Michalewicz          [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $5$      $0$      $0$      $0$      $[0, \pi]_i, i = 1...n$          	     $-4.6876$

Michalewicz          [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $10$     $0$      $0$      $0$      $[0, \pi]_i, i = 1...n$         	     $-9.6601$

Perm($\beta=0.5$)    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $5$      $0$      $0$      $0$      $[-n, n]_i, i = 1...n$ 			     $0.0000$

Perm($\beta=10$)     [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $8$      $0$      $0$      $0$      $[-n, n]_i, i = 1...n$                 $0.0000$

Powell $4$           [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $4$      $0$      $0$      $0$      $[-4, 5]_i, i = 1...n$                 $0.0000$

Power Sum            [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $4$      $0$      $0$      $0$      $[0, 4]_i, i = 1...n$                  $0.0000$

Qing     	        [Clerc, 1999] 		  	 			    $n$      $0$      $0$      $0$      $[-500, 500]_i, i = 1...n$      	     $0.0000$

Rastrigin            [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-6.12, 5.12]_i, i = 1...n$           $0.0000$

Rosenbrock           [Hedar, 2005, Dixon and Szegö, 1978, Surjanovic and Bingham, 2013]      $n$      $0$      $0$      $0$      $[-5, 10]_i, i = 1...n$      		     $0.0000$

Rotated Ellipsoid     [Surjanovic and Bingham, 2013] 	 				    $n$      $0$      $0$      $0$      $[-65.536, 66.536]_i, i = 1...n$       $0.0000$

Schwefel             [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-500, 500]_i, i = 1...n$             $0.0000$

Shekel($ m = 5$)     [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $4$      $0$      $0$      $0$      $[0, 10]_i, i = 1...n$                 $-10.1531$

Shekel($ m = 7$)     [Hedar, 2005, Surjanovic and Bingham, 2013]	 		    $4$      $0$      $0$      $0$      $[0, 10]_i, i = 1...n$                 $-10.4029$

Shekel($ m = 10$)    [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $4$      $0$      $0$      $0$      $[0, 10]_i, i = 1...n$                 $-10.5364$

Shubert              [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $2$      $0$      $0$      $0$      $[-10, 10]_i, i = 1...n$               $-186.7309$

Sphere               [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-5.12 ,6.12]_i, i = 1...n$     	     $0.0000$

Styblinski-Tang      [Clerc, 1999] 			 			    $n$      $0$      $0$      $0$      $[-5, 5]_i, i = 1...n$       	         $-39.1661n$

Sum of Powers 	    [Surjanovic and Bingham, 2013]		 				    $n$      $0$      $0$      $0$      $[-10 ,15]_i, i = 1...n$      	     $0.0000$

Trid                 [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $6$      $0$      $0$      $0$      $[-36, 36]_i, i = 1...n$               $-50.0000$

Trid                 [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $10$     $0$      $0$      $0$      $[-100, 100]_i, i = 1...n$             $-210.0000$

Zakharov             [Hedar, 2005, Surjanovic and Bingham, 2013] 			    $n$      $0$      $0$      $0$      $[-5, 11]_i, i = 1...n$                $0.0000$

LC	    Bunnag $1$  		    [Vaz and Vicente, 2009]       	   				    $4$      $1$      $0$      $1$ 	    $[0, 3]_i, i = 1...n$ 	   		     $0.1117$

Bunnag $2$  		    [Vaz and Vicente, 2009]       					    $4$      $2$      $0$      $2$      $[0, 4]_i, i = 1...n$ 						     $-6.4049$

Bunnag $3$  		    [Vaz and Vicente, 2009]       					    $5$      $3$      $0$      $1$      $[0, 3]_1,[0, 2]_{2,5},[0, 4]_{3,4}$ 			     $-16.3657$

Bunnag $4$  		    [Vaz and Vicente, 2009]       					    $6$      $2$      $0$      $1$      $[0, 1]_{1,2,3,4,5},[0, 20]_6$ 				     $-213.0470$

Bunnag $5$  		    [Vaz and Vicente, 2009]       					    $6$      $5$      $0$      $1$      $[0, 2]_{1,3,6},[0, 8]_2,[0, 1]_{4,5}$ 		     $-11.0000$

Bunnag $6$  		    [Vaz and Vicente, 2009]       					    $10$     $11$     $0$      $3$      $[0, 1]_i, i = 1...n$ 						     $-268.0146$

Bunnag $7$  		    [Vaz and Vicente, 2009]       					    $10$     $5$      $0$      $0$      $[0, 1]_i, i = 1...n$		 					     $-39.0000$

G01 				    [Suganthan et al., 2005] 	   				    $13$     $9$      $0$      $6$      $[0, 10]_i,[0, 100]_{10}, i = 1...9,11$ 		     $-15.0000$

Genocop $9$   	    [Vaz and Vicente, 2009] 	   					    $3$      $5$      $0$      $2$      $[0, 10]_i, i = 1...n$ 					 	     $-2.4714$

Genocop $10$  	    [Vaz and Vicente, 2009] 	   					    $4$      $5$      $0$      $0$      $[0, 3]_1,[0, 10]_{2,3},[0, 1]_4$	 			     $-4.5280$

Genocop $11$  	    [Vaz and Vicente, 2009] 	   					    $6$      $5$      $0$      $0$      $[0, 5]_{1,3},[0, 8]_2,[0, 1]_{4,5},[0, 2]_6$      $-11.0000$

Horst $1$   		    [Horst et al., 1995] 				    $2$      $3$      $0$      $1$      $[0, 3]_1,[0, 2]_2$ 		 				 	     $-1.0625$

Horst $2$   		    [Horst et al., 1995] 				    $2$      $3$      $0$      $2$      $[0, 2.5]_1,[0, 2]_2$ 			 			     $-6.8995$

Horst $3$ 	 	    [Horst et al., 1995]					    $2$      $3$      $0$      $0$      $[0, 1]_1,[0, 1.5]_2$ 						     $-0.4444$

Horst $4$  		    [Horst et al., 1995] 				    $3$      $4$      $0$      $2$      $[0.5, 2]_1,[0, 3]_2,[0, 2.8]_3$ 				     $-6.0858$

Horst $5$  		    [Horst et al., 1995] 				    $3$      $4$      $0$      $0$      $[0, 1.2]_{1,2},[0, 1.7]_3$ 			 		     $-3.7220$

Horst $6$  		    [Horst et al., 1995] 				    $3$      $7$      $0$      $2$      $[0, 6]_1,[0, 5.0279]_2,[0, 2.6]_3$ 			     $-32.5784$

Horst $7$   		    [Horst et al., 1995] 				    $3$      $4$      $0$      $2$      $[0, 6]_1,[0, 3]_{2,3}$ 				 		     $-52.8769$

hs021 		  	    [Vaz and Vicente, 2009] 						    $2$      $1$      $0$      $1$      $[2, 50]_1,[-50, 10]_2$ 			 			     $-99.9599$

hs021mod  		    [Vaz and Vicente, 2009] 						    $7$      $3$      $0$      $1$      $[2, 50]_1,[-50, 50]_2,[0, 50]_3,[2, 10]_4,[-10, 10]_5,[-10, 0]_6,[0, 10]_7$     $4.0400$

hs024 		  	    [Vaz and Vicente, 2009]     					    $2$      $3$      $0$      $2$      $[0, 5]_i, i = 1...n$ 						     $-1.0000$

hs035 		  	    [Vaz and Vicente, 2009]     					    $3$      $1$      $0$      $1$      $[0, 3]_i, i = 1...n$ 						     $0.1111$

hs036 	 	 	    [Vaz and Vicente, 2009]     					    $3$      $1$      $0$      $1$      $[0, 20]_1,[0, 11]_2,[0, 15]_3$ 				     $-3300.0000$

hs037 	 	 	    [Vaz and Vicente, 2009]     					    $3$      $2$      $0$      $1$      $[0, 42]_i, i = 1...n$ 						     $-3456.0000$

hs038 		 	    [Vaz and Vicente, 2009]     					    $4$      $2$      $0$      $0$      $[-10, 10]_i, i = 1...n$ 						     $0.0000$

hs044 		        [Vaz and Vicente, 2009]     					    $4$      $6$      $0$      $2$      $[0, 5]_i, i = 1...n$ 						     $-15.0000$

hs076 	 	        [Vaz and Vicente, 2009]     					    $4$      $3$      $0$      $1$      $[0, 1]_{1,3,4},[0, 3]_2$ 					     $-4.6818$

P14				    [Birgin et al., 2010]   	      			    $4$      $4$      $0$      $2$      $[10^{-5}, 3]_1,[10^{-5}, 4]_2,[0, 2]_3,[0, 1]_4$     $-4.51420$

s224 	 	 	    [Vaz and Vicente, 2009] 	    				    $2$      $4$      $0$      $1$      $[0, 6]_1,[0, 11]_2$ 											    $-304.0000$

s231 	 	 	    [Vaz and Vicente, 2009] 	    				    $2$      $2$      $0$      $0$      $[-10, 10]_i, i = 1...n$ 			 					    $0.0000$

s232 	 	 	    [Vaz and Vicente, 2009] 	    				    $2$      $3$      $0$      $2$      $[0, 100]_i, i = 1...n$ 				 	    $-1.0000$

s250  		 	    [Vaz and Vicente, 2009] 	    				    $3$      $2$      $0$      $1$      $[0, 20]_1,[0, 11]_2,[0, 42]_3$ 					 	    $-3300.0000$

s251 	  		    [Vaz and Vicente, 2009] 						    $3$      $1$      $0$      $1$      $[0, 42]_i, i = 1...n$ 								 	    $-3456.0000$

zecevic2 	 	    [Vaz and Vicente, 2009] 						    $3$      $2$      $0$      $1$     $[0, 10]_i, i = 1...n$ 												    $-4.1249$

zecevic3 	 	    [Vaz and Vicente, 2009] 						    $2$      $2$      $0$      $1$     $[0, 10]_i, i = 1...n$ 												    $97.3094$

zecevic4 	 	    [Vaz and Vicente, 2009] 						    $4$      $2$      $0$      $1$     $[0, 10]_i, i = 1...n$ 												    $7.5575$

GC	    circle 		        [Vaz and Vicente, 2009]       					    $3$      $10$     $0$      $3$      $[0, 10]_i, i = 1...n$ 										 	    $4.5742$

G02 		 		    [Suganthan et al., 2005]       			    $20$     $2$      $0$      $1$      $[0, 10]_i, i = 1...n$ 										 	    $-0.8036$

G03 				    [Suganthan et al., 2005] 	   				    $10$     $0$      $1$      $1$      $[0, 10]_i, i = 1...n$ 										 	    $-1.0005$

G04 	 			    [Suganthan et al., 2005]       			    $5$      $6$      $0$      $2$      $[78, 102]_1,[33, 45]_2,$ $[27, 45]_{3,4,5}$ 				 		    $-30665.5386$

G05 				    [Suganthan et al., 2005] 	   				    $4$      $2$      $3$      $3$      $[10, 1,200]_{1,2},$ $[-0.55, 0.55]_{3,4}$ 					 	    $5126.4967$

G06 		 		    [Suganthan et al., 2005]       			    $2$      $2$      $0$      $2$      $[13, 100]_1,[0, 100]_2$ 									 		    $-6961.8138$

G07 		 		    [Suganthan et al., 2005]       			    $10$     $8$      $0$      $6$      $[-10, 10]_i, i = 1...n$ 									 		    $24.3062$

G08 		 		    [Suganthan et al., 2005]       			    $2$      $2$      $0$      $0$      $[0, 10]_i, i = 1...n$ 										 	    $-0.0958$

G09 		 		    [Suganthan et al., 2005]       			    $7$      $4$      $0$      $2$      $[-10, 10]_i, i = 1...n$ 									 		    $680.6300$

G10 		 		    [Suganthan et al., 2005]       			    $8$      $6$      $0$      $6$      $[100, 10,000]_1,[1,000, 10,000]_{2,3},$ $[10, 1,000]_i, i=4...8$     $7049.2480$

G11 				    [Suganthan et al., 2005] 	   				    $2$      $0$      $1$      $1$      $[-1, 1]_i, i = 1...n$ 										 	    $0.7499$

G12 	 			    [Suganthan et al., 2005]       			    $3$      $1$      $0$      $0$      $[0.2, 10]_i, i = 1...n$ 									 		    $-1.0000$

G13 				    [Suganthan et al., 2005] 	   				    $5$      $0$      $3$      $3$      $[-2.3, 2.3]_{1,2},[-3.2, 3.2]_{3,4,5}$ 							    $0.0539$

G16 		 		    [Suganthan et al., 2005]     $5$      $38$      $0$     $4$     $[704.4148, 906.3855]_1,[68.6, 288.88]_2,$ $[0, 134.75]_3,[193, 287.0966]_4,[25, 84.1988]_5$     $-1.9051$

G18 		 		    [Suganthan et al., 2005] 					    $9$      $13$     $0$      $6$      $[0, 10]_i, i = 1...n$ 								 			    $-0.8660$

G19 	 			    [Suganthan et al., 2005] 					    $15$     $5$      $0$      $0$      $[0, 10]_i, i = 1...n$ 										 	    $32.6555$

G24 	 			    [Suganthan et al., 2005] 					    $2$      $2$      $0$      $2$      $[0, 3]_1,[0, 4]_2$ 												    $-5.5080$

Goldstein & Price*   	    [Na et al., 2017] 	   					    $2$      $2$      $0$      $1$      $[-2, 2]_i, i = 1...n$ 		 									    $3.5389$

Gomez		 	    [Birgin et al., 2010]    					    $2$      $1$      $0$      $1$      $[-1, 1]_i, i = 1...n$ 						 					    $-0.9711$

Himmelblau 		    [Cagnina et al., 2008]   					    $5$      $5$      $0$      $2$      $[78, 102]_1,[33, 45]_2,$ $[27, 45]_{3,4,5}$ 				 		    $-31025.5602$

P01 				    [Birgin et al., 2010]   					    $5$      $0$      $3$      $3$      $[-5, 5]_i, i = 1...n$ 						    $0.0293$

P02a				    [Birgin et al., 2010]   					    $9$      $10$     $0$      $5$      $[0, 100]_1,[0, 500]_i, 2 = 1...n$ 			    $-400.0000$

P02b				    [Birgin et al., 2010]   					    $9$      $10$     $0$      $5$      $[0, 600]_1,[0, 500]_i, 2 = 1...n$ 			    $-600.0000$

P02c				    [Birgin et al., 2010]   					    $9$      $10$     $0$      $4$      $[0, 100]_1,[0, 500]_i, 2 = 1...n$ 			    $-750.0000$

P02d				    [Birgin et al., 2010]   					    $10$     $12$     $0$      $5$      $[0, 300]_{1,2,6},[0, 100]_{3,5,7},[0, 200]_{4,8},$ $[0, 3]_9$     $-600.0000$

P03a				    [Birgin et al., 2010]   					    $6$      $1$      $4$      $5$      $[0, 1]_{1,2,3,4},[10^{-5}, 16]_{5,6}$ 		    $0.3888$

P03b 	 	 	    [Birgin et al., 2010] 					    $2$      $1$      $0$      $1$      $[10^{-5}, 16]_i, i = 1...n$ 				    $0.3888$

P04		  	 	    [Birgin et al., 2010] 					    $2$      $1$      $0$      $1$      $[0, 6]_1,[0, 4]_2$ 						    $-6.6666$

P05				    [Birgin et al., 2010]   					    $3$      $2$      $0$      $2$      $[0, 9.422]_1,[0, 5.903]_2,[0, 267.42]_3$ 	    $201.1600$

P06		  	 	    [Birgin et al., 2010] 					    $2$      $1$      $0$      $1$      $[0, 115.8]_1,[10^{-5}, 30]_2$ 				    $376.2900$

P07		  	 	    [Birgin et al., 2010] 					    $2$      $4$      $0$      $1$      $[-2, 2]_i, i = 1...n$ 						    $-2.8284$

P08		  	 	    [Birgin et al., 2010] 					    $2$      $2$      $0$      $1$      $[-8, 10]_1,[0, 10]_2$ 						    $-118.7000$

P09				    [Birgin et al., 2010]   					    $6$      $9$      $0$      $2$      $[10^{-5}, 3]_1,[10^{-5}, 4]_{2,3},[0, 2]_{4,5},[0, 6]_6$     $-13.4020$

P10		  	 	    [Birgin et al., 2010] 					    $2$      $2$      $0$      $2$      $[0, 1]_i, i = 1...n$ 						    $0.7417$

P11		  	 	    [Birgin et al., 2010] 					    $2$      $1$      $0$      $1$      $[0, 1]_i, i = 1...n$ 						    $-0.5000$

P12				    [Birgin et al., 2010]   					    $2$      $2$      $0$      $0$      $[0, 2]_1,[0, 3]_2$     $-16.7390$

P13				    [Birgin et al., 2010]   					    $3$      $0$      $2$      $2$      $[10^{-5}, 34]_1,[10^{-5}, 17]_2,[100, 300]_3$     $189.3500$

P15				    [Birgin et al., 2010]   					    $3$      $0$      $3$      $3$      $[10^{-5}, 12.5]_1,[10^{-5}, 37.5]_2,[0, 50]_3$     $0.0000$

P16				    [Birgin et al., 2010]   					    $5$      $6$      $0$      $0$      $[0, 1.5834]_1,[0, 3.625]_2,[0, 1]_3,$ $[0, 3]_4,[0, 4]_5$     $0.7049$

s365mod 	  	        [Vaz and Vicente, 2009] 						    $7$      $9$      $0$      $5$      $[0, 19]_i, i = 1...n$ 				 	    $52.1399$

T1       		    [Finkel, 2005] 					    $n$      $1$      $0$      $1$      $[-4, 4]_i, i = 1...n$ 	 			 	    $-n$

zy2 				    [Vaz and Vicente, 2009] 						    $2$      $3$      $0$      $1$      $[0, 10]_i, i = 1...n$ 				 		    $2.0000$

§ THE MATHEMATICAL FORMULATION OF ENGINEERING PROBLEMS

§.§ Tension/compression spring design problem

The design variables of the tension/compression spring design problem [Kazemi et al., 2011] are the number of the wire diameter $x_1$, the winding diameter $x_2$, and active coils of the spring $x_3$.
The objective function and the mechanical constraints are given by:

\begin{equation*}
\label{eq:tension}
\begin{aligned}
\min 	& \; f(\mathbf{x})=x_1^2x_2(x_3+2)\\
\st \;  & g_1(\mathbf{x})= 1- \frac{x_2^3x_3}{71875x_1^4} \leq \mathbf{0}, \\
& g_2(\mathbf{x})= \frac{x_2(4x_2-x_1)}{12566x_1^3(x_2-x_1)}+ \frac{2.46}{12566x_1^2}-1 \leq \mathbf{0}, \\
& g_3(\mathbf{x})= 1- \frac{140.54x_1}{x_3x_2^2} \leq \mathbf{0}, \\
& g_4(\mathbf{x})= \frac{x_1+x_2}{1.5}-1 \leq \mathbf{0}
\end{aligned}
\end{equation*}

where $0.05 \leq x_1 \leq 0.2,~0.25 \leq x_2 \leq 1.3,~2 \leq x_3 \leq 15$.
The best known solution $\mathbf{x}^* =$ ($0.05170517$, $0.35710042$, $11.28120672$), where $f(\mathbf{x}^*) = 0.01267931$.
Two of the constraint functions are active ($g_1$ and $g_2$).

§.§ Three-bar truss design problem

The three-bar truss design problem [Ray and Liew, 2003] has two design variables and three constraints.
The optimization problem is formulated as follows:

\begin{equation*}
\label{eq:three-bar}
\begin{aligned}
\min 	& \; f(\mathbf{x})=100(2 \sqrt{2} x_1+x_2)\\
\st \;  & g_1(\mathbf{x})= \frac{\sqrt{2} x_1+x_2}{\sqrt{2} x_1^2+2x_1x_2}2-2 \leq \mathbf{0}, \\
& g_2(\mathbf{x})= \frac{x_2}{\sqrt{2} x_1^2+2x_1x_2}2-2 \leq \mathbf{0}, \\
& g_3(\mathbf{x})= \frac{1}{x_1+\sqrt{2} x_2}2-2 \leq \mathbf{0}
\end{aligned}
\end{equation*}

where $0 \leq x_1 \leq 1,~0 \leq x_2 \leq 1$.
The best known solution $\mathbf{x}^* =$ ($0.78867512$, $0.40824832$), where $f(\mathbf{x}^*) = 263.89584535$.
One of the constraint functions is active ($g_1$).

§.§ NASA speed reducer design problem

The design variables of the NASA speed reducer design problem [Ray and Liew, 2003] are the face width $x_1$, the module of teeth $x_2$, the number of teeth on the pinion $x_3$, the length of the first shaft between the bearings $x_4$, the distance of the second shaft between the bearings $x_5$, the diameter of the first shaft $x_6$, and, finally, the width of the second shaft $x_7$.
The optimization problem is formulated as follows:

\begin{equation*}
\label{eq:nasa}
\begin{aligned}
\min 	& \; f(\mathbf{x}) = 0.7854x_1x_2^2(3.3333x_3^2+14.9334x_3-43.0934) \\
& -1.508x_1(x_6^2+x_7^2)+7.4777(x_6^3+x_7^3) \\
& +0.7854(x_4x_6^2+x_5x_7^2) \\
\st \;  & g_1(\mathbf{x}) = \frac{27}{x_1x_2^2x_3}-1 \leq \mathbf{0},~
g_2(\mathbf{x})=\frac{397.5}{x_1x_2^2x_3^2} - 1 \leq \mathbf{0}, \\
& g_3(\mathbf{x})=\frac{1.93x_4^3}{x_2x_3x_6^4} - 1 \leq \mathbf{0},~
g_4(\mathbf{x})=\frac{1.93x_5^3}{x_2x_3x_7^4} - 1 \leq \mathbf{0},\\
& g_5(\mathbf{x})=  \frac{((\frac{745x_4}{x_2x_3})^2+16.9 \times 10^6)^{0.5}}{110x_6^3}  - 1 \leq \mathbf{0}, \\
& g_6(\mathbf{x})=\frac{((\frac{745x_5}{x_2x_3})^2+157.5 \times 10^6)^{0.5}}{85x_7^3}  - 1 \leq \mathbf{0}, \\
& g_7(\mathbf{x})=\frac{x_2x_3}{40} - 1 \leq \mathbf{0},~
g_8(\mathbf{x})=  \frac{5x_2}{x_1} - 1 \leq \mathbf{0}, \\
& g_9(\mathbf{x})=\frac{x_1}{12x_2} - 1 \leq \mathbf{0},~
g_{10}(\mathbf{x})=\frac{1.5x_6+1.9}{x_4} - 1 \leq \mathbf{0}, \\
& g_{11}(\mathbf{x})=\frac{1.1x_7+1.9}{x_5} - 1 \leq \mathbf{0}
\end{aligned}
\end{equation*}

where $2.6 \leq x_1 \leq 3.6,~0.7 \leq x_2 \leq 0.8,~17 \leq x_3 \leq 28,~7.3 \leq x_4 \leq 8.3,~7.8 \leq x_5 \leq 8.3,~2.9 \leq x_6 \leq 3.9,~5 \leq x_7 \leq 5.5 $.
The best known solution $\mathbf{x}^* =$ ($3.5$, $0.7$, $17$, $7.3$, $7.8$, $3.35021468$, $5.28668323$), where $f(\mathbf{x}^*) = 2996.34817613$.
Three constraints are active ($g_5, g_6$ and $g_8$).

§.§ Pressure vessel design problem

There are four design variables in the pressure vessel design problem [Kazemi et al., 2011](in inches): the thickness of the pressure vessel $x_1$, the thickness of the head $x_2$, the inner radius of the vessel $x_3$, and the length of the cylindrical component $x_4$.
The optimization problem is formulated as follows:

\begin{equation*}
\label{eq:perssure}
\begin{aligned}
\min 	& \; f(\mathbf{x})=0.6224x_1x_3x_4+1.7781x_2x_3^2+3.1661x_1^2x_4 \\
& +19.84x_1^2x_3\\
\st \;  & g_1(\mathbf{x})= -x_1+0.0193x_3  \leq \mathbf{0}, \\
& g_2(\mathbf{x})= -x_2+0.00954x_3 \leq \mathbf{0}, \\
& g_3(\mathbf{x})=- \pi x_3^2x_4- \frac{4}{3} \pi x_3^3+1296000 \leq \mathbf{0}, \\
& g_4(\mathbf{x})= x_4-240 \leq \mathbf{0},~
g_5(\mathbf{x})= 1.1-x_1 \leq \mathbf{0}, \\
& g_6(\mathbf{x})= 0.6-x_2  \leq \mathbf{0}
\end{aligned}
\end{equation*}

where $1 \leq x_1 \leq 1.375,~0.625 \leq x_2 \leq 1,~25 \leq x_3 \leq 150,~25 \leq x_4 \leq 240$.
The best known solution $\mathbf{x}^* =$ ($1.1$, $0.625$, $56.99481866$, $51.00125165$), where $f(\mathbf{x}^*) = 7163.73957163$.
Three constraints are active ($g_1, g_3$ and $g_5$).

§.§ Welded beam design problem

The welded beam design problem [Mirjalili and Lewis, 2016, Mirjalili et al., 2014] is to design a welded beam at minimum cost, subject to some constraints [Mirjalili and Lewis, 2016, Mirjalili et al., 2014].
The objective is to find a minimum fabrication cost.
Considering the four design variables and constraints of shear stress $\tau$, bending stress in the beam $\sigma$, buckling load on the bar $P_c$, and end deflection on the beam $\delta$.
The optimization model is summarized in the following equation:

\begin{equation*}
\label{eq:welder}
\begin{aligned}
\min 	& \; f(\mathbf{x}) = 1.10471 x_1^2 x_2+0.04811 x_3 x_4 (14+x_2)\\
\st \;  & g_1(\mathbf{x}) = \tau (\mathbf{x}) - 13600 \leq \mathbf{0}, \\
& g_2(\mathbf{x}) = \sigma (\mathbf{x}) - 3\times 10^4 \leq \mathbf{0}, \\
& g_3(\mathbf{x}) = x_1 - x_4 \leq \mathbf{0}, \\
& g_4(\mathbf{x}) = 0.10471 x_1^2 + 0.04811x_3 x_4 (14 + x_2) - 5 \leq \mathbf{0}, \\
& g_5(\mathbf{x}) = \delta (\mathbf{x}) - 0.25 \leq \mathbf{0}, \\
& g_6(\mathbf{x}) = P - P_c (\mathbf{x}) \leq \mathbf{0}, \\
& g_7(\mathbf{x}) = 0.125 - x_1 \leq \mathbf{0}, \\
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
& \tau (\mathbf{x}) = \sqrt{(\tau^1)^2+(\tau^1) (\tau^2) x_2/R + (\tau^2)^2}, \\
& \tau^1 = \frac{P}{\sqrt{2} x_1 x_"}, \tau^2 = \frac{M R}{J}, M = P \lgroup L+ \frac{x_2}{2} \rgroup, \\
& R = \sqrt{\frac{x_2^2)}{4} + (\frac{x_1+x_3}{2})^2}, \sigma (\mathbf{x}) = \frac{6PL}{x_4 x_3^2}, \\
& J = 2(\sqrt{2}x_1 x_2 (\frac{x_2^2}{12} + \frac{1}{4}(x_1 x_3)^2)), \delta (\mathbf{x}) = \frac{4PL^3}{Ex_4 x_3^3}, \\
& P_c =   \frac{4.013 E \sqrt{x_3^2 x_4"6 / 36} }{L^2}(1 - \frac{x_3}{2L} \sqrt{\frac{E}{4G}}) , \\
& P = 6000, L = 14, E = 3 \times 10^7, G = 12 \times 10^6,
\end{aligned}
\end{equation*}

where $0.1 \leq x_1 \leq 2, ~0.1 \leq x_2 \leq 10, ~0.1 \leq x_3 \leq 10, ~0.1 \leq x_3 \leq 2$.
The best known solution $\mathbf{x}^* =$ ($0.20572551$, $3.47062057$, $9.03666456$, $0.20573141$), where $f(\mathbf{x}^*) = 1.72488430$.
One of the constraint functions is active ($g_3$).

§.§ General nonlinear regression problem

Parameter estimation in the general nonlinear regression model [Gillard and Kvasov, 2017, Paulavičius et al., 2020, Sergeyev et al., 2016] can be reduced to solving the minimization problem:

\begin{equation*}
\label{eq:sinusoid}
\begin{aligned}
\min 	& \; f(\mathbf{x})= \sum_{t=1}^{T} (\kappa(t) - \phi(\mathbf{x},t))^2 \\
\end{aligned}
\end{equation*}

\begin{equation*}
\begin{aligned}
&  \sum_{q=1}^{s} \kappa(t) = e^{t d_q} \sin(2 \pi t \omega_q + \theta_q), \\
&  \sum_{q=1}^{s} \phi(\mathbf{x},t) = e^{(x_{3(q - 1) + 1} t)} \sin(2 \pi t x_{3(q - 1) + 2} + x_{3(q - 1) + 3}) \\
\end{aligned}
\end{equation*}

where $-1 \leq x_{3(q - 1) + 1} \leq 0,~0 \leq x_{3(q - 1) + 2, 3(q - 1) + 2} \leq 1, q = 1...s$.
$\mathbf{d}$ is non-positive damping coefficients, $\omega$ is frequencies, and $\theta$ is phases of the sinusoids (hereafter, $d_q \in [-1, 0],~\omega_q \in [0, 1],~\theta_q \in [0, 1],~ q = 1...s$).
For signal approximation, the parameter $\mathbf{x}$ of the problem is determined to fit best the real-valued signal values observed in the uniformly distributed time moments $t = 1, 2,..., T$.
The best known solution $f(\mathbf{x}^*) = 0$, where $\mathbf{x}^* =$ ($-0.2$, $0.4$, $0.3$), for $n = 3$, $\mathbf{x}^* =$ ($-0.2$, $0.4$, $0.3$, $-0.3$, $0.3$, $0.1$), for $n = 6$, and $\mathbf{x}^* =$ ($-0.4$, $0.6$, $0.2$, $-0.3$, $0.3$, $0.1$, $-0.2$, $0.4$, $0.3$), for $n = 9$.