11institutetext: University of Johannesburg, Academy of Computer Science and
Software Engineering, Johannesburg, SA,
11email<EMAIL_ADDRESS>22institutetext: University of
Johannesburg, Institute for Intelligent Systems,
Johannesburg, SA
22email<EMAIL_ADDRESS>33institutetext: University of Johannesburg,
Faculty of Engineering and the Built Environment,
Johannesburg, SA
33email<EMAIL_ADDRESS>
# Exploring the effectiveness of surrogate-assisted evolutionary algorithms on
the batch processing problem

Mohamed Z. Variawa 11 0000-0002-3345-2754 Terence L. Van Zyl 22
0000-0003-4281-630X Matthew Woolway 33 0000-0002-4902-851X

###### Abstract

Real-world optimisation problems typically have objective functions which
cannot be expressed analytically. These optimisation problems are evaluated
through expensive physical experiments or simulations. Cheap approximations of
the objective function can reduce the computational requirements for solving
these expensive optimisation problems. These cheap approximations may be
machine learning or statistical models and are known as surrogate models. This
paper introduces a simulation of a well-known batch processing problem in the
literature. Evolutionary algorithms such as Genetic Algorithm (GA),
Differential Evolution (DE) are used to find the optimal schedule for the
simulation. We then compare the quality of solutions obtained by the
surrogate-assisted versions of the algorithms against the baseline algorithms.
Surrogate-assistance is achieved through Probablistic Surrogate-Assisted
Framework (PSAF). The results highlight the potential for improving baseline
evolutionary algorithms through surrogates. For different time horizons, the
solutions are evaluated with respect to several quality indicators. It is
shown that the PSAF assisted GA (PSAF-GA) and PSAF-assisted DE (PSAF-DE)
provided improvement in some time horizons. In others, they either maintained
the solutions or showed some deterioration. The results also highlight the
need to tune the hyper-parameters used by the surrogate-assisted framework, as
the surrogate, in some instances, shows some deterioration over the baseline
algorithm.

###### Keywords:

single-objective optimisation, machine learning, evolutionary
algorithms,surrogate models

##  1 Introduction

Single-objective optimisation problems (SOP) are a common occurrence in
numerous fields. Most optimisation problems encountered in practice have
objective functions that cannot be assessed analytically and call for time-
consuming physical experiments or simulations [10, 14, 23].

Search heuristics called evolutionary algorithms (EA) produce solutions to
optimisation and search issues. Evolutionary algorithms employ inheritance,
mutation, selection, and crossover methods that are modelled after natural
evolution. The computing restrictions of evolutionary algorithms are identical
to those of physical experiments or simulations, as often, it may take tens of
thousands of fitness assessments to find workable solutions [10].

It is possible to use computationally affordable approximations of the goal
functions to get around these restrictions while maintaining the quality of
the solutions. These low-cost estimates are known as surrogate models or
metamodels (these terms may be used interchangeably) [5, 9, 12, 6, 7, 19, 11,
18, 17, 16]. These stand-ins could be statistical models (like the Gaussian
Process) or machine learning models (e.g. artificial neural networks). These
surrogates can be trained using historical data or simulation runs that have
been carefully chosen.

Previously, [18] employed surrogate-assisted techniques for the optimisation
of design parameters of a chemical process. The authors compared a simulation-
only approach to a surrogate-assisted model [18]. The authors note that the
surrogate-assisted achieves a max value faster and is equally able to achieve
a better max revenue than the simulation-only model [18]. Further, the authors
demonstrate that surrogate-assisted Genetic Algorithms can scale into
increasingly complex systems with parallel and feedback components, with
significant speedups and robust results [17].

In this paper, a simulation of the (flowshop) batch-processing problem [8] is
designed, which accepts a set of instructions indicating which processes
should run at which time (herewith referred to as a schedule). The
metaheuristic optimisation algorithms Genetic Algorithm (GA) and Differential
Evolution (DE) obtain optimal schedules for the batch-processing problem, the
baseline algorithms. The objective function used by GA and DE is the
simulation of the problem. Then, surrogate-assisted versions of the GA and DE
derive optimal schedules for the batch-processing problem. Quality indicators
such as Success Rate (SR), Average Evaluations to a Solution (AESR) and
Average Generations to a Solution (AGSR) compare the performance of the
surrogate-assisted algorithms to the baseline algorithms.

Specifically, the aims of this paper are:

  1. 1.

to show that it is viable to create a simulation to model a well known
(flowshop) batch processing problem using the SimPy package,

  2. 2.

to illustrate the viability of using the simulation as the objective function
of an optimisation problem, and

  3. 3.

to investigate the improvements, if any, of using surrogate-assisted
evolutionary algorithms as opposed to the sole use of evolutionary algorithms.

###  1.1 Single-Objective Optimisation

The mathematical formulation of a (maximisation) single-objective optimisation
problem, is given by:

—s— f(x) g_j(x)≤0,j=0,…,J h_k(x)=0,k=0,…,K x^L ≤x ≤x^U

where $f_{\textbf{x}}$ is the objective function, $\textbf{x}\epsilon R^{n}$
is the decision vector, $\textbf{x}^{L}$ and $\textbf{x}^{U}$ are the lower
and upper bounds of the decision vector, n is the number of decision
variables, $g_{j}(\textbf{x})$ are the inequality constraints, and
$h_{k}(\textbf{x})$ are the equality constraints, J, K are the number of
inequality and equality constraints, respectively [10]. If a solution x
satisfies all constraints, it is called a feasible solution, and the solutions
that achieve the maximum value are called the optimal solutions [10].

###  1.2 Probabilistic Surrogate-Assisted Framework (PSAF)

The probabilistic surrogate-assisted framework (PSAF) employed in this
research was introduced by [1] and also forms part of the authors’ pysamoo
package, which is explored in this research [3]. PSAF may only be applied to
single-objective, unconstrained optimisation problems [1] (PSAF has been
extended for multi-objective, constrained problems in a framework known as
GPSAF [2]). The baseline algorithm (i.e. the meta-heuristic that the surrogate
will enhance) used in PSAF may be one of many evolutionary algorithms, such as
Genetic Algorithm (GA), Particle Swarm Optimisation (PSO), or Covariance
matrix adaptation evolution strategy (CMA-ES) [1].

PSAF uses the whole search pattern (i.e. using all the solutions and their
offspring) to optimise the surrogate, as opposed to other frameworks which
only use the final solution(s). The exploration-exploitation balance is
discovered by using the search pattern and accounting for the surrogate’s
accuracy. PSAF consists of two phases to enable even more adaptable use of the
surrogate. The first phase, known as $\alpha$-phase, derives a solution set
influenced by the surrogate [1]. The second phase, $\beta$-phase, introduces
bias by optimising the surrogate for a few iterations [1].

The $\alpha$-phase incorporates a popular concept among evolutionary
algorithms known as tournament selection. A population member must win a
tournament to participate in the mating process. The quantity of competitors
($\alpha$) balances how greedy the selection process will be. On the one hand,
a higher value of $\alpha$ restricts mating to elitist solutions, whereas a
lower value lessens the selection pressure [1]. The most commonly used
tournament mode for genetic algorithms is the binary tournament ($\alpha$=2),
which compares a pair of solutions regarding one or multiple metrics. A binary
tournament declares the least infeasible solution (i.e. the solution whose
objective value is closest to the objective function) as the winner if one or
both solutions are infeasible. If both solutions are feasible, the solution
with the smaller function value is the winner. In PSAF, tournament selection
compares solutions evaluated on the surrogate; this introduces surrogate bias
while generating new infill solutions [1].

Although tournament selection effectively incorporates the surrogate’s
approximation, it is limited by only looking at one iteration into the future.
During the $\beta$-phase, the baseline algorithm is run for extra consecutive
$\beta$ iterations on the surrogate’s approximation, which increases the
surrogates’ impact. The surrogate’s optimum will continuously be reached if
$\beta$ is incorrect and will completely discard the baseline algorithm’s
default infill method. An incorrectly chosen $\beta$ also reduces the infill
options’ diversity and does not consider the approximation inaccuracy of the
surrogate [1].

The optimisation of the batch-processing problem can be written as, [15]: —s—
Σ_s price(s^p)qs(s^p, p), ∀p=P, s^p ϵS^p tu(s_in,j^’,p)≥tp(s_inj,p),∀j ϵJ, p
ϵP, s_inj ϵS^sp_inJ^’, s_inj^’ ϵS^sc_inJ q_s(s,p)≤QS^U∀s ϵS, p ϵP

###  1.3 Batch processing problem

$\mathbf{s_{1}}$ Mixing $\mathbf{s_{2}}$ Reaction $\mathbf{s_{3}}$
Purification $\mathbf{s_{4}}$

Figure 1: State Task Network diagram. Table 1: System constraints data. Unit | Capacity | Suitability | Time | Price
---|---|---|---|---
Unit 1 | 100 | Mixing | 4.5 | 0
Unit 2 | 75 | Reaction | 3.0 | 0
Unit 3 | 50 | Purification | 1.5 | 0
State | Storage | Initial |  | Price
State 1 | Unlimited | Unlimited |  | 0
State 2 | 100 | 0.0 |  | 0
State 3 | 100 | 0.0 |  | 0
State 4 | Unlimited | 0.0 |  | 0

This paper’s example used as a case study is a batch processing problem
described by [8, 20, 21]. The problem studies the production of a single
product via three processes: mixing, reaction, and purification. Table 1 shows
the relevant constraints on the system, and the STN in Figure 1 illustrates
the flow of the product through the system [21]. Both fixed and variable
processing time variants of this problem have been solved using evolutionary
algorithms [20, 21, 4]. The Makespan minimisation problem has also been solved
using evolutionary algorithms [22]. The global solutions (i.e. the optimal
schedules for various time horizons) employed in this paper have been reported
in the literature [20, 21, 22]. The fixed times are shown in Table 1 while the
variable processing times are batch dependent (i.e. the time depends on the
amount of product) and not considered in this paper. Table 1 shows the storage
constraints of each storage unit in the system as well as the capacity
constraints (i.e. how much material each process can take in). A simulation of
the process in the STN diagram 1 is created and used as the objective function
in this research, described further in Section 2.1. The simulation output is
the amount of product produced by a solution for a given time horizon. The
solutions take the form of binary instruction vectors which tell the
simulation which process to run at which time. The decision variables are
binary digits which indicate whether a process should be run ($1$) or not
($0$). The instruction vectors and decision variables are discussed in more
detail in Section 2.1.

##  2 Methodology

###  2.1 Setting up the simulation

SimPy is a Discrete-Event Simulator which allows for the creation of
simulations based on (discrete) real-world processes [13]. A simulation in
SimPy is created using the parameters described in Section 1.3 and the State
Task Network (STN) diagram shown in Figure 1. The units and states in Table 1
are Containers in the simulation. The three processes (mixing, reaction, and
purification) are Python functions which control the contents of the
Containers.

The objective of the case study in Section 1.3 is to maximise the amount of
product produced in a given time horizon. As such, the simulation takes as
input a schedule (herewith referred to as the instruction vector) instructing
it at which times the processes should be started. The indices of the
instruction vector represent the 0.5H time steps from 0, e.g.
$0,0.5,1,1.5,2,\dots$. Each element in the instruction vector is a binary
vector of size 3. Each index of this binary vector represents one of the
processes (0 for purification, 1 for reaction, and 2 for mixing). Consider an
instruction vector x = [[1,0,0], [0,0,0]], then at time 0, the instructions
are [1,0,0] indicating that mixing should occur at time 0, while reaction and
purification should not. At time 0.5, none of the processes would occur;
however, since mixing started at time 0, it will run for its allotted time. If
a process starts to run at time $t$, it will not be allowed to run again until
its’ scheduled time is complete. For a time horizon $n$, the instruction
vector would have a length of $(n\times 2)\times 3$. For example, a 12H time
horizon would have an instruction vector of length 72. An example instruction
vector for the 12H time horizon is show in 2.

The simulation in this instance is not necessarily cheaper than the objective
function, however for more complex scenarios (e.g. the multi-purpose batch
processing problem [20, 21, 8], the simulation would be cheaper and the
purpose of this study is to investigate if a cheap simulation can approximate
an objective function well enough for future work. The simulation is an
approximation because instruction vectors don’t always produce the optimal
objective value, only the optimal instruction vectors do.

Adjusting the systems’ parameters creates a variation of the motivating
example (called the primary example) with a stricter bottleneck (i.e.
increasing the complexity of the optimisation problem). Reducing the storage
capacities of State 2 and State 3 in Table 1 from 100 to 50 allows for more
stringent bottleneck behaviour.

1 0 0 …0 1 0 …0 0 1 …0 0 0 $\left[\vbox{\hrule
height=6.7pt,depth=6.7pt,width=0.0pt}\right.$$\left.\vbox{\hrule
height=6.7pt,depth=6.7pt,width=0.0pt}\right]$time 0time 4.5time 7.5time 12run
mixermixerpurificatorno process to run Figure 2: Instruction vector example
for 12H time horizon.

###  2.2 Optimising the schedule of the multi-purposed batch-processing
problem

The batch-processing problem is a constrained maximisation optimisation
problem with the number of variables for a given time horizon equal to the
length of the instruction vector for that time horizon. The objective function
to maximise is the output of the simulation described in Section 2.1 (i.e. the
amount of material produced in a given time horizon). The simulation is coded
in such a way that if a solution is infeasible (i.e. a process is scheduled to
run but violates the system’s constraints), then it accepts the solution but
won’t run any processes that violate the constraints of the system. Since the
instruction vectors are binary, the lower and upper limits for the variables
are 0 and 1, respectively.

The initial population is a randomly generated instruction vector with at most
half the values set to 1 (with the rest being 0). Crossover and mutation are
applied to generate the offspring population.

Generate Initial Population Fitness Evaluation Convergence check Selection
Crossover Mutation Stop yesno

Figure 3: GA flowchart.

The evolutionary algorithms studied in this paper are GA and DE. The
algorithmic processes for GA and DE are shown in Figures 3 and 4,
respectively. The PSAF framework, described in Section 1.2, provides surrogate
assistance to the evolutionary algorithms. Various quality indicators allow
for comparing the quality of the solutions generated by the evolutionary
algorithms and their surrogate-assisted counterparts. The quality indicators
employed in this research are Success Rate (SR), Average Evaluations to a
Solution (AESR) and Average Generations to a Solution (AGSR). SR measures the
percentage of trials in which the best value returned by the algorithm was
within a specific percentage of the optimal objective value. AESR measures the
average number of evaluations required to reach a certain percentage of the
optimal objective value. AGSR is the average number of generations to reach a
certain percentage of the optimal objective value. For the primary example,
95% and 99.5% are chosen. For the variation of the problem, 90% and 95% are
chosen. For robustness, each experiment is run 30 times, and the average value
of each quality indicator is presented.

The various algorithms use the same parameters: the initial population size is
30, the number of generations is 20, and the number of offspring is 10. For
both the primary example and the variation, the parameters for PSAF are;
$\alpha$ is 5, $\beta$ is 5, and the number of infills is 10. It should be
noted that the parameters have not been tuned, which could improve results.
For the 168H time horizon, the number of generations is reduced to 15. Table 4
provides the parameters used in the GA and DE algorithms for all the time
horizons.

Generate Initial Population Fitness Evaluation Stopping criteria met? Mutation
Crossover Selection Stop yesno

Figure 4: DE flowchart.

##  3 Results

Table 2: Primary Example Quality Indicators.

Algorithm |  Time Horizon |  Objective Value |  SR @95 |  SR @99.5 |  AESR @95 |  AESR @99.5 |  AGSR @95 |  AGSR @99.5
---|---|---|---|---|---|---|---|---
GA | 12H | 100 | 100 | 100 | 30 | 30 | 1 | 1
24H | 350 | 100 | 100 | 30 | 30 | 1 | 1
36H | 625 | 100.00 | 100.00 | 44.67 | 56.67 | 2.83 | 3.83
48H | 900 | 100.00 | 73.33 | 34.00 | 107.72 | 1.47 | 8.46
60H | 1150 | 100.00 | 60.00 | 30.00 | 142.22 | 1.00 | 12.22
72H | 1425 | 100.00 | 36.67 | 34.00 | 140.00 | 1.40 | 12.00
168H | 3550 | 100 | 0 | 30 | 0 | 1 | 0
PSAF-GA | 12H | 100 | $100.00^{=}$ | $100.00^{=}$ | $31.33^{-}$ | $31.33^{-}$ | $1.13^{-}$ | $1.13^{-}$
24H | 350 | $100^{=}$ | $100^{=}$ | $30^{=}$ | $30^{=}$ | $1^{=}$ | $1^{=}$
36H | 625 | $100.00^{=}$ | $100.00^{=}$ | $32^{+}$ | $48.67^{+}$ | $1.2^{+}$ | $2.87^{+}$
48H | 900 | $100.00^{=}$ | $53.33^{-}$ | $40.67^{-}$ | $76.25^{+}$ | $2.07^{-}$ | $5.63^{+}$
60H | 1150 | $100.00^{=}$ | $33.33^{-}$ | $36.67^{-}$ | $82.00^{+}$ | $1.67^{-}$ | $6.20^{+}$
72H | 1425 | $100.00^{=}$ | $20.00^{-}$ | $31.33^{+}$ | $106.67^{+}$ | $1.13^{+}$ | $8.67^{+}$
168H | 3550 | $100.00^{=}$ | $0.00^{=}$ | $34.44^{-}$ | $0.00^{=}$ | $1.44^{-}$ | $0.00^{=}$
DE | 12H | 100 | 100.00 | 100.00 | 30.33 | 30.33 | 1.03 | 1.03
24H | 350 | 100 | 100 | 30 | 30 | 1 | 1
36H | 625 | 100.00 | 73.33 | 30.00 | 30.00 | 1.00 | 1.00
48H | 900 | 100.00 | 80.00 | 32.00 | 123.33 | 1.20 | 10.33
60H | 1150 | 100.00 | 33.33 | 30.00 | 135.00 | 1.00 | 11.50
72H | 1425 | 100.00 | 0.00 | 30.67 | 0.00 | 1.07 | 0 .00
168H | 3550 | 100 | 0 | 30 | 0 | 1 | 0
PSAF-DE | 12H | 100 | $100.00^{=}$ | $100.00^{=}$ | $30.67^{-}$ | $30.67^{-}$ | $1.07^{-}$ | $1.07^{-}$
24H | 350 | $100.00^{=}$ | $100.00^{=}$ | $31.33^{=}$ | $31.33^{-}$ | $1.13^{-}$ | $1.13^{-}$
36H | 625 | $100.00^{=}$ | $100.00^{+}$ | $30.00^{=}$ | $52.67^{-}$ | $1.00^{=}$ | $3.27^{-}$
48H | 900 | $100.00^{=}$ | $100.00^{+}$ | $44.00^{-}$ | $83.33^{+}$ | $2.40^{-}$ | $6.33^{+}$
60H | 1150 | $100.00^{=}$ | $26.67^{-}$ | $30.00^{=}$ | $132.50^{+}$ | $1.00^{=}$ | $11.25^{+}$
72H | 1425 | $100.00^{=}$ | $6.67^{+}$ | $34.67^{-}$ | $140.00^{+}$ | $1.47^{-}$ | $12.00^{+}$
168H | 3550 | $100^{=}$ | $100^{=}$ | $30^{=}$ | $30^{+}$ | $1^{=}$ | $1^{+}$
+Surrogate improvement. |  |  |  |  |
-Surrogate deterioration. |  |  |  |  |
=Surrogate maintained. |  |  |  |  |

Table 3: Variant Example Quality Indicators.

Algorithm |  Time Horizon |  Objective Value |  SR @90 |  SR @95 |  AESR @90 |  AESR @95 |  AGSR @90 |  AGSR @95
---|---|---|---|---|---|---|---|---
GA | 12H | 100 | 100 | 100 | 30 | 30 | 1 | 1
24H | 325 | 100.00 | 93.33 | 53.33 | 87.14 | 1 | 4.21
36H | 575 | 100.00 | 80.00 | 61.33 | 155.00 | 1.80 | 10.58
48H | 800 | 100.00 | 33.33 | 56.67 | 742.00 | 1.33 | 25.40
60H | 1000 | 100 | 100 | 30 | 50 | 1 | 3
72H | 1250 | 100.00 | 93.33 | 32.00 | 104.28 | 1.20 | 8.43
168H | 2825 | 100 | 100 | 30 | 30 | 1 | 1
PSAF-GA | 12H | 100 | $100^{=}$ | $100^{=}$ | $30^{=}$ | $30^{=}$ | $1^{=}$ | $1^{=}$
24H | 325 | $100.00^{=}$ | $100.00^{+}$ | $30.00^{+}$ | $65.33^{+}$ | $1.00^{=}$ | $4.53^{-}$
36H | 575 | $93.33^{-}$ | $40.00^{-}$ | $40.00^{+}$ | $58.33^{+}$ | $2.00^{-}$ | $3.83^{+}$
48H | 800 | $100.00^{=}$ | $0.00^{-}$ | $35.33^{+}$ | $0.00^{-}$ | $1.53^{-}$ | $0.00^{-}$
60H | 1000 | $100.00^{=}$ | $86.67^{-}$ | $30.00^{=}$ | $67.69^{-}$ | $1.00^{-}$ | $4.77^{-}$
72H | 1250 | $100.00^{=}$ | $53.33^{-}$ | $30.00^{+}$ | $90.00^{+}$ | $1.00^{+}$ | $7.00^{+}$
168H | 2825 | $100^{=}$ | $100^{=}$ | $30^{=}$ | $30^{=}$ | $1^{=}$ | $1^{=}$
DE | 12H | 100 | 100 | 100 | 30 | 30 | 1 | 1
24H | 325 | 100.00 | 80.00 | 30.00 | 32.5 | 1 | 1.25
| 36H | 575 | 100 | 80.00 | 38.00 | 78.33 | 1.80 | 5.83
| 48H | 800 | 100.00 | 0.00 | 50.67 | 0.00 | 3.07 | 0.00
| 60H | 1000 | 100 | 100 | 30 | 57.33 | 1.00 | 3.73
| 72H | 1250 | 100.00 | 46.67 | 35.33 | 104.29 | 1.53 | 8.43
| 168H | 2825 | 100 | 100 | 30 | 30 | 1 | 1
PSAF-DE | 12H | 100 | $100^{=}$ | $100^{=}$ | $30^{=}$ | $30^{=}$ | $1^{=}$ | $1^{=}$
24H | 325 | $100.00^{=}$ | $100.00^{=}$ | $30.00^{=}$ | $52.00^{-}$ | $1.00^{=}$ | $3.20^{-}$
36H | 575 | $100.00^{=}$ | $40.00^{-}$ | $48.00^{-}$ | $53.33^{+}$ | $2.8^{-}$ | $3.33^{+}$
48H | 800 | $100.0^{=}$ | $0.0^{=}$ | $32.0^{+}$ | $0.0^{=}$ | $1.2^{+}$ | $0.0^{=}$
60H | 1000 | $100.00^{=}$ | $100.00^{=}$ | $30.00^{=}$ | $40.67^{+}$ | $1.00^{+}$ | $2.07^{+}$
72H | 1250 | $100.00^{=}$ | $73.33^{+}$ | $31.33^{+}$ | $124.55^{-}$ | $1.13^{+}$ | $10.45^{-}$
168H | 2825 | $100^{=}$ | $100^{=}$ | $30^{=}$ | $30^{=}$ | $1^{=}$ | $1^{=}$
+Surrogate improvement. |  |  |  |  |
-Surrogate deterioration. |  |  |  |  |
=Surrogate maintained. |  |  |  |  |

Table 4: Experimental Setup (Primary and Variation Examples).

Time Horizon | Algorithm | Generations | Population Size | Offspring
---|---|---|---|---
12H | GA | 20 | 30 | 10
DE | 20 | 30 | 10
24H | GA | 20 | 30 | 10
DE | 20 | 30 | 10
36H | GA | 20 | 30 | 10
DE | 20 | 30 | 10
48H | GA | 20 | 30 | 10
DE | 20 | 30 | 10
60H | GA | 20 | 30 | 10
DE | 20 | 30 | 10
72H | GA | 20 | 30 | 10
DE | 20 | 30 | 10
168H | GA | 15 | 30 | 10
DE | 15 | 30 | 10

Tables 2 and 3 show the quality indicators for the primary and variant
examples, respectively, for each time horizon. The SR@95 and<EMAIL_ADDRESS>values
show the percentage of trials wherein the highest objective value was within
95% and 99.5% of the optimal objective value, respectively. The AESR@95 and
<EMAIL_ADDRESS>values show the average number of evaluations required to reach 95%
and 99.5% of the optimal objective value, respectively. The AGSR@95 and
<EMAIL_ADDRESS>values show the average number of generations required to reach 95%
and 99.5% of the optimal objective value, respectively. Studying the
surrogate-assisted results, we expect to see the SR values increase (i.e. the
success rates should increase). We also expect the AESR and AGSR values to
decrease (i.e. it should take the surrogate-assisted algorithms fewer
evaluations and generations to get close to the optimal objective value).

Table 2 shows that PSAF-GA provides general improvement for the<EMAIL_ADDRESS>and
<EMAIL_ADDRESS>indicators. The<EMAIL_ADDRESS>shows that PSAF-GA had much lower success
rates than the GA. The AESR@95 shows mixed results wherein there is an
improvement for the 36H, 72H and 168H time horizons, whereas there is
deterioration and maintenance for the others. The AGSR@95 also shows an
improvement for some time horizons and deterioration or maintenance for
others. Interestingly, under the specified parameters, the GA and PSAF-GA
could not reach 99.5% of the optimal objective value for the 168H time
horizon.

$0$$2$$4$$6$$8$$10$$12$$14$$16$$3{,}400$$3{,}410$$3{,}420$$3{,}430$GenerationObjective
ValueGADEPSAF-GAPSAF-DE

Figure 5: (Primary Example) 168H Generations vs Objective Value.

Comparing the DE and PSAF-DE shows a general improvement in the<EMAIL_ADDRESS>metric. However, Table 2 shows mixed results in the other metrics where the
surrogate-assisted PSAF-DE showed improvement for some time horizons and
deterioration or maintenance for others. DE could not reach 99.5% of the
optimal objective value for the 72H and 168H time horizons, whereas PSAF-DE
could. Figure 5 highlights that for the 168H time horizon, PSAF-DE and PSAF-GA
can get closer to the optimal objective value quicker than the baseline GA and
DE algorithms (i.e. it tends towards the optima in fewer generations). This
result highlights the value of enhancing evolutionary algorithms through
surrogate assistance.

Studying the results for the variation of the problem, Table 3 shows that in
some cases, PSAF-GA provided an improvement over the baseline GA, whereas, in
others, there was either maintenance or deterioration. Studying the results of
PSAF-DE and the baseline DE paints a similar picture, wherein some of the test
cases show an improvement by PSAF-DE, although that improvement is not visible
in all test cases. Figure 5 shows that for the 168H time horizon in the
primary example, PSAF-DE and PSAF-GA can converge towards the optimal
objective value in fewer generations than DE and GA, respectively. These
results showcase the potential for using a surrogate-assisted framework but
also highlight the importance of tuning the surrogates’ hyper-parameters for
achieving consistent, improved performance. Further, the results presented
here warrant the exploration of using PSAF for solving the more general multi-
purpose batch processing problem which is a multi-objective optimisation
problem.

##  4 Conclusion

This paper studied the application of the evolutionary algorithms GA, DE, and
their surrogate-assisted counterparts PSAF-GA and PSAF-DE to the batch-
processing problem [8, 20, 21].

The task is to maximise the product produced by the simulation for a given
time horizon. The solutions generated by the evolutionary algorithms represent
a schedule for the simulation to follow. This paper also considers a problem
variation in which the systems’ parameters are modified to enforce stricter
bottleneck behaviour, thereby increasing the complexity of the optimisation
task.

Table 2 shows the potential of a surrogate-assisted framework like PSAF for
solving optimisation problems. In some cases, in the primary example, the
surrogate improved over the baseline algorithm or maintained the quality of
solutions (e.g. PSAF-DE showed improvement on<EMAIL_ADDRESS>and AGSR@99.5). In
contrast, the surrogate deteriorated over the baseline algorithm in some cases
(e.g. PSAF-DE showed deterioration on AESR@90 and AGSR@90). Table 3 shows
similar behaviour in the variant of the variant example. Figure 5 shows that
for the 168H time horizon in the primary example, PSAF-GA and PSAF-DE converge
in fewer generations than GA and DE. These results show that it is feasible to
build a simulation of the problem which can be used as the objective function
in the optimisation problem.

Finally, these findings show that surrogate-assisted frameworks have the
potential to improve baseline evolutionary algorithms. In future work, tuning
the parameters of the PSAF framework, $\alpha$ and $\beta$, could provide
further improvement by the surrogate. This paper considered the single-
objective batch-processing problem, to explore the effectiveness of using a
surrogate-assisted framework like PSAF to solve optimisation problems using a
simulation as the objective function. The results presented here warrant the
exploration of using PSAF for solving the more general multi-purpose batch
processing problem which is a multi-objective optimisation problem.

## References

  * Blank and Deb [2021] Blank, J., Deb, K.: Psaf: a probabilistic surrogate-assisted framework for single-objective optimization. GECCO ’21: Proceedings of the Genetic and Evolutionary Computation Conference pp. 652–659 (2021)
  * Blank and Deb [2022a] Blank, J., Deb, K.: Gpsaf: A generalized probabilistic surrogate-assisted framework for constrained single- and multi-objective optimization. arXiv (2022a), doi:10.48550/ARXIV.2204.04054
  * Blank and Deb [2022b] Blank, J., Deb, K.: pysamoo: Surrogate-assisted multi-objective optimization in python. arXiv (2022b)
  * Bowditch et al. [2019] Bowditch, Z., Woolway, M., van Zyl, T.: Comparative metaheuristic performance for the scheduling of multipurpose batch plants. In: 2019 6th international conference on soft computing & machine intelligence (ISCMI), pp. 121–125, IEEE (2019)
  * Chen et al. [2022] Chen, Q., Ma, X., Yu, Y., Sun, Y., Zhu, Z.: Multi-objective evolutionary multi-tasking algorithm using cross-dimensional and prediction-based knowledge transfer. Information Sciences 586 (2022), doi:10.1016/j.ins.2021.12.014
  * Guo et al. [2022] Guo, D., Wang, X., Gao, K., Jin, Y., Ding, J., Chai, T.: Evolutionary optimization of high-dimensional multiobjective and many-objective expensive problems assisted by a dropout neural network. IEEE Journals 52(4.0) (2022), doi:10.1109/TSMC.2020.3044418
  * Horii [2021] Horii, H.: Advancement of vehicle occupant restraint system design by integration of artificial intelligence technologies. International Journal of Transport Development and Integration 5(3) (2021), doi:10.2495/TDI-V5-N3-242-253
  * Ierapetritou and Floudas [1998] Ierapetritou, M.G., Floudas, C.A.: Effective continuous-time formulation for short-term scheduling. 1. multipurpose batch processes. Industrial and Engineering Chemistry Research 37(11), 4341–4359 (1998)
  * Jimenez et al. [2022] Jimenez, F., Sánchez, G., Palma, J., Sciavicco, G.: Three-objective constrained evolutionary instance selection for classification: Wrapper and filter approaches. Engineering Applications of Artificial Intelligence 107 (2022), doi:10.1016/j.engappai.2021.104531
  * Jin et al. [2021] Jin, Y., Wang, H., Sun, C.: Data-driven Evolutionary Optimization. Springer (2021)
  * Li et al. [2021] Li, H., Liu, Z., Zhu, P.: An improved multi-objective optimization algorithm with mixed variables for automobile engine hood lightweight design. Journal of Mechanical Science and Technology 35(5) (2021), doi:10.1007/s12206-021-0423-5
  * Ma et al. [2022] Ma, X., Zhao, X., Zhang, Y., Liu, K., Yang, H., Li, J., Akhlaghi, Y.G., Liu, H., Han, Z., Liu, Z.: Combined rankine cycle and dew point cooler for energy efficient power generation of the power plants - a review and perspective study. Energy 238 (2022), doi:10.1016/j.energy.2021.121688
  * Matloff [2008] Matloff, N.: Introduction to discrete-event simulation and the simpy language. Davis, CA. Dept of Computer Science. University of California at Davis. 2(2009), 1–33 (2008)
  * Perumal and van Zyl [2020] Perumal, R., van Zyl, T.L.: Surrogate assisted methods for the parameterisation of agent-based models. In: 2020 7th International Conference on Soft Computing & Machine Intelligence (ISCMI) (2020)
  * Seid and Majozi [2012] Seid, R., Majozi, T.: A robust mathematical formulation for multipurpose batch plants. Chemical Engineering Science pp. 36–53 (2012)
  * Stander et al. [2022] Stander, L., Woolway, M., Van Zyl, T.L.: Surrogate-assisted evolutionary multi-objective optimisation applied to a pressure swing adsorption system. Neural Computing and Applications pp. 1–17 (2022)
  * Stander et al. [2020a] Stander, L., Woolway, M., van Zyl, T.: Extended surrogate assisted continuous process optimisation. In: 2020 7th International Conference on Soft Computing & Machine Intelligence (ISCMI), pp. 275–279, IEEE (2020a)
  * Stander et al. [2020b] Stander, L., Woolway, M., van Zyl, T.L.: Data-driven evolutionary optimisation for the design parameters of a chemical process: A case study. In: 2020 IEEE 23rd International Conference on Information Fusion (FUSION), pp. 1–8, IEEE (2020b)
  * Tong et al. [2021] Tong, J., Li, Y., Liu, J., Cheng, R., Guan, J., Wang, S., Liu, S., Hu, S., Guo, T.: Experiment analysis and computational optimization of the atkinson cycle gasoline engine through nsga ii algorithm using machine learning. Energy Conversion and Management 238 (2021), doi:10.1016/j.enconman.2021.113871
  * Woolway and Majozi [2018] Woolway, M., Majozi, T.: A novel metaheuristic framework for the scheduling of multipurpose batch plants. Chemical Engineering Science 192 (08 2018), doi:10.1016/j.ces.2018.08.031
  * Woolway and Majozi [2019] Woolway, M., Majozi, T.: On the application of a metaheuristic suite with parallel implementations for the scheduling of multipurpose batch plants. Computers and Chemical Engineering 126(2019), 371–390 (2019), doi:10.1109/TEVC.2021.3098257
  * van Zyl and Woolway [2020] van Zyl, T., Woolway, M.: Makespan minimisation for multipurpose batch plants using metaheuristic approaches. In: 2020 7th International Conference on Soft Computing & Machine Intelligence (ISCMI), pp. 56–60 (2020), doi:10.1109/ISCMI51676.2020.9311596
  * van Zyl et al. [2021] van Zyl, T.L., Woolway, M., Paskaramoorthy, A.: Parden: Surrogate assisted hyper-parameter optimisation for portfolio selection. In: 2021 8th international conference on soft computing & machine intelligence (ISCMI), pp. 101–107, IEEE (2021)


# Opportunities and Challenges to Integrate
Artificial Intelligence into Manufacturing Systems:
Thoughts from a Panel Discussion

Ilya Kovalenko, Kira Barton, James Moyne, and Dawn M. Tilbury I. Kovalenko is
with Department of Mechanical Engineering and Industrial & Manufacturing
Engineering, Pennsylvania State University, University Park, PA.
<EMAIL_ADDRESS>K. Barton and D. M. Tilbury are with the Robotics
Department and the Department of Mechanical Engineering, University of
Michigan, Ann Arbor, MI, USA. {tilbury<EMAIL_ADDRESS>J. Moyne is with
the Department of Mechanical Engineering, University of Michigan, Ann Arbor,
MI, USA<EMAIL_ADDRESS>
##  I Introduction

Rapid advances in artificial intelligence (AI)111Caveat: The panel did not
attempt to disentangle Artificial Intelligence from Machine Learning, and used
the two terms loosely interchangeably during the discussion. have the
potential to significantly increase the productivity, quality, and
profitability in future manufacturing systems. Traditional mass-production
will give way to personalized production, with each item made to order, at the
low cost and high quality consumers have come to expect. Manufacturing systems
will have the intelligence to be resilient to multiple disruptions, from
small-scale machine breakdowns, to large-scale natural disasters. Products
will be made with higher precision and lower variability. While gains have
been made towards the development of these factories of the future, many
challenges remain to fully realize the vision shown in Figure 1.

To consider the challenges and opportunities associated with this topic, a
panel of experts from Industry, Academia, and Government was invited to
participate in an active discussion at the 2022 Modeling, Estimation and
Control Conference (MECC) held in Jersey City, New Jersey from October 3-5,
2022 [1]. This panel discussion started with an overview presentation given by
Professor Ilya Kovalenko (Pennsylvania State University) on the topic of
automated learning for manufacturing systems [2]. Following the overview, the
panelists introduced themselves, their organizations, and provided preliminary
thoughts on the integration of AI into manufacturing systems. Panelists
included Dr. Meiling He (Rockwell Automation), Dr. Daewon Lee (Samsung AI
Center NY), Dr. James Moyne (Applied Materials and University of Michigan),
Dr. Robert Landers (University of Notre Dame), and Dr. Jordan Berg (National
Science Foundation). The panel discussion focused on the challenges and
opportunities to more fully integrate AI into manufacturing systems, and was
moderated by Professors Kira Barton (University of Michigan) and Dawn Tilbury
(University of Michigan).

Three overarching themes emerged from the panel discussion. First, to be
successful, AI will need to work seamlessly, and in an integrated manner with
humans (and vice versa). Second, significant gaps in the infrastructure needed
to enable the full potential of AI into the manufacturing ecosystem, including
sufficient data availability, storage, and analysis, must be addressed. And
finally, improved coordination between universities, industry, and government
agencies can facilitate greater opportunities to push the field forward. The
rest of this article briefly summarizes these three themes, and concludes with
a discussion of promising directions.

##  II Humans and AI working together

Figure 1: Vision for integrating AI Systems into the manufacturing
environment.

Although many large-scale manufacturing processes are automated to some
degree, there are always humans involved at multiple levels. The humans may be
physically interacting with the system (e.g., loading parts into a machine) or
interacting through a computer (e.g., defining the production schedule).
Different Subject Matter Experts (SMEs) have different expertise, some may
have a deep technical knowledge of a specific process such as maintenance of a
specific machine, while others may have a broader understanding of a system.
Humans are expected to be adaptable, and able to react to both small
disturbances (slightly out-of-order product flow) and large changes (new
product arriving).

Most AI deployed in manufacturing plants today is relatively fixed, i.e., it
does not have the ability to learn beyond what it has been trained to do, nor
can it ask questions when it does not understand what to do. The panel felt
that there was a significant opportunity to develop AI that can work in a more
integrated fashion with SMEs, at different levels, perhaps answering questions
posed by the SME and even posing questions when something becomes uncertain.
While some AI tools that take input from a human have been developed for data-
science applications (e.g., Google VisionAI), these capabilities need to be
extended to the manufacturing environment. In this way, the knowledge held by
the SMEs can be leveraged with the knowledge in the AI, and no human knowledge
will be left behind. This integration of SME and AI can also lead to better
trust of the AI by the SME, so that the AI is seen as a tool to help get the
job done well, empowering the SME rather than threatening to eliminate the
job.

Many currently-deployed AI systems in manufacturing detect anomalies that
occur, things that are out of the ordinary, which can lead to preventing poor
quality or machine breakdown. However, in many cases, the anomaly detection
just alerts an operator, who is called over to investigate the situation, and
make a decision about the appropriate response (ignore, shutdown, repair,
etc). Most of these solutions are developed in a specific domain, using SMEs
knowledgeable about the process. Generalizing these solutions to new domains
is difficult, and generally requires new SMEs to be engaged. Could an AI be
developed that would facilitate this type of generalization? What would it
need to learn and where would it get the information?

In many manufacturing plants today, although robots and humans may work
together to accomplish a job, they do not work in close proximity nor do they
request knowledge from each other. They are separated due to safety
constraints, with the robots sequestered in cages. Newer collaborative robots
can operate safely around humans, although they are generally restricted to
move slower than their caged counterparts. Despite closer proximity,
collaborative robots do not request additional information from a human when
new or unfamiliar challenges arise. As perception technologies improve, there
are opportunities for AI to help robots and humans work together on physical
tasks such as assembly. This could help address the challenge of personalized
production.

As we move into the era of human-centric, sustainable manufacturing [3],
questions of cooperation between humans and AI become increasingly more
important. One question that was raised was whether it was essential to
differentiate between human and artificial intelligence. From a manufacturing
system perspective, would it be possible to start looking at any intelligent
system (humans, robots, computers, etc.) as an intelligent agent with specific
capabilities? By blurring the line between human and artificial intelligence,
we can start to develop an evolutionary process to support learning resulting
from a continuum of human and AI knowledge.

##  III Infrastructure for AI in Manufacturing

Manufacturing systems are incredibly complex. From the nonlinear physics of
low-level processes (welding, 3D printing) to the material handling within a
plant, scheduling, and supply chain, everything needs to come together for a
high-quality part to be produced and delivered to a consumer. There are
hundreds of thousands of variables, of different types, and measured at
different sample rates, that interact in diverse ways to describe what is
happening. Many manufacturing plants operate multiple shifts per day, or even
24/7, leading to huge volumes of data. There are thousands or even millions of
different things that can happen. AI technology needs data to learn, and
although there is a lot of data to be found in manufacturing systems, it is
not always AI-ready. Or, even though there is a lot of data, it may not be
sufficiently diverse and capture enough samples of important events to develop
effective learning and intelligence applications. In short, most data
management systems were not developed or put in place with AI in mind (e.g.,
maintenance management databases), and so they are not AI ready in terms of
many data quality dimensions (e.g., accuracy, freshness, granularity,
volume/archive length, variety, and context richness).

Data is collected through sensors, and the panelists spent some time
discussing the limitations of current sensing in manufacturing. In many cases,
the sensors do not measure the variable of interest, limiting the ability for
AI to learn a model of the process. For additive manufacturing or welding
processes, the sensor can only measure what is on the surface and not the
important material characteristics underneath. For some manipulation
processes, only vision is available without any haptic or tactile feedback.
Some sensors, such as thermal imaging cameras, can return huge amounts of
data, which can be a challenge to process.

AI problems require massive amounts of data, and the process of obtaining that
data is expensive. Although manufacturing companies have a lot of data, they
typically do not have the amount of data required for many AI approaches.
Further, manufacturing companies are reluctant to share their data, due to its
proprietary nature, reducing the potential for supplementing the data through
additional resources. The panel discussed the advantages of industry-
university partnerships to share information (e.g., data, pre-competitive
model forms). Efforts to standardize the anonymization of information to
preserve the underlying characteristics for different types of AI and learning
would also be useful.

As discussed in Section II, there is a big need to develop techniques that
allow humans and AI technology to collaboratively work on tasks in the
manufacturing system. However, the panelists also identified that there is a
significant infrastructure challenge that needs to be addressed before humans
and AI can effectively work together. Examples were provided where SMEs are
expected to be both/either at the front end of the AI technology (e.g.,
setting up the bounds for anomaly detection) and/or the back end of the AI
technology (e.g., analyzing the information provided by the AI tool). However,
the interfaces may have been developed for a specific type of worker/SME. How
can manufacturers start to develop generalizable interfaces that enable a
comfortable working environment for the human? Can interfaces be developed
that allow the human and AI to work on the tasks that each one is good at? How
can AI be enhanced to the point of asking for information when necessary and
accepting information asynchronously?

##  IV Developing Industry-University Collaborations

The panelists have experience in industry, academia, and government. The panel
recognized that most of the identified challenges, from improving the AI
infrastructure to developing new technology, cannot be solved by a single
institution and must include collaborations between industry, academia, and
the government. Companies will often have the technology and can identify
real-world manufacturing problems that would benefit from the integration of
AI. Universities provide the education infrastructure to develop the
manufacturing workforce and the research capabilities that allow researchers
to address various challenges. The government can foster these collaborations
through programs that identify the need and provide funding for for AI in
manufacturing. The need to incorporate the workforce (e.g., operators and
SMEs) into the research discussions was also recognized. As AI technology is
developed for the manufacturing sector, training tools for the workforce must
be developed to improve the integration of new manufacturing AI technology. It
will also be important to motivate the people working in the manufacturing
environment to adopt and use AI technology for their work.

While AI has been recognized as a pivotal technology for growth and innovation
for manufacturing [4], industry has been slow to adopt AI technology within
the manufacturing environment. Some of the factors that contribute to this
slow adoption of AI technology include issues with the economy of scale. A lot
of solutions to research problems are specific and cannot be scaled to the
rest of the manufacturing enterprise due to cost and time constraints. As
mentioned above, to advance AI solutions, there is a need for manufacturers to
share data. Partnerships between universities and industry, with government
support, can help with data and information sharing, such as developing
improved data anonymization techniques and changing the cultural perspective
of data sharing in manufacturing.

##  V Promising Directions

The panel discussed ideas about how to push the manufacturing field toward a
more fully integrated paradigm of AI technology in manufacturing systems. The
advancement of manufacturing system technology (e.g, Industrial Internet of
Things, collaborative robots) and the rise of customer expectations for
customization and production capabilities provides an opportunity for the
integration of AI in the manufacturing system industry. While a number of
challenges were identified for the full integration of AI in manufacturing
systems, there are a number of promising directions for this growing area.
Industry, academia, and government agencies recognize the need for developing
collaborations and programs that can facilitate improved adoption of AI in the
manufacturing environment. Manufacturing scheduling and planning was one of
the identified high-value, target research areas that can be targeted with AI
technology. The panel believed that the scheduling and planning process can be
improved through AI technology due to the significant amount of data available
and the large opportunity for productivity improvement, especially when moving
towards personalized production. The need for improving the reusability and
extensibility of AI solutions via digital twin technology was also mentioned
as a topic of interest [5]. Another key research area identified by all
members of the panel was the need to integrate subject-matter expertise into
AI technologies deployed in the manufacturing domain at all levels. Both human
and artificial intelligence are needed to realize the promise of high-quality,
high-volume, low-cost personalized production systems. The discussion in this
panel aligned with the research directions and opportunities that have been
identified in existing literature. For example, the need to leverage
technological advancements to help create human-centric and eco-centric
manufacturing systems has recently been identified as a topic of interest by a
number of other researchers in the area of smart manufacturing [6, 7, 3, 8].
Note that while the panel discussion described in this manuscript covered a
wide array of topics, there were a number of important research areas, such as
the use of digitization and virtual manufacturing [9] and the role of robots
and cobots [10], that were only briefly mentioned and could benefit from
future discussions on challenges and opportunities.

## Acknowledgments

We would like to thank the MECC organizers [1] for the opportunity to present
the special session. We would also like to thank the panelists and audience
for their thoughtful comments and participation in the session.

## References

  * [1] “Modeling, estimation and control conference.” [Online]. Available: https://mecc2022.a2c2.org/
  * [2] I. Kovalenko, J. Moyne, M. Bi, E. C. Balta, W. Ma, Y. Qamsane, X. Zhu, Z. M. Mao, D. M. Tilbury, and K. Barton, “Toward an Automated Learning Control Architecture for Cyber-Physical Manufacturing Systems,” _IEEE Access_ , vol. 10, pp. 38 755–38 773, 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9751037/
  * [3] X. Xu, Y. Lu, B. Vogel-Heuser, and L. Wang, “Industry 4.0 and industry 5.0—inception, conception and perception,” _Journal of Manufacturing Systems_ , vol. 61, pp. 530–535, 2021.
  * [4] Deloitte, “AI Enablement on the Way to Smart Manufacturing: Deloitte Survey on AI Adoption in Manufacturing,” Tech. Rep., 2020\.
  * [5] J. Moyne, Y. Qamsane, E. C. Balta, I. Kovalenko, J. Faris, K. Barton, and D. M. Tilbury, “A requirements driven digital twin framework: Specification and opportunities,” _IEEE Access_ , vol. 8, pp. 107 781–107 801, 2020.
  * [6] J. Leng, W. Sha, B. Wang, P. Zheng, C. Zhuang, Q. Liu, T. Wuest, D. Mourtzis, and L. Wang, “Industry 5.0: Prospect and retrospect,” _Journal of Manufacturing Systems_ , vol. 65, pp. 279–295, Oct. 2022. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S0278612522001662
  * [7] G. Q. Huang, B. Vogel-Heuser, M. Zhou, and P. Dario, “Digital Technologies and Automation: The Human and Eco-Centered Foundations for the Factory of the Future [TC Spotlight],” _IEEE Robotics & Automation Magazine_, vol. 28, no. 3, pp. 174–179, Sep. 2021. [Online]. Available: https://ieeexplore.ieee.org/document/9535352/
  * [8] M. Zhou, “Editorial: Evolution from AI, IoT and Big Data Analytics to Metaverse,” _IEEE/CAA Journal of Automatica Sinica_ , vol. 9, no. 12, pp. 2041–2042, Dec. 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9970439/
  * [9] J. Yang, X. Wang, and Y. Zhao, “Parallel Manufacturing for Industrial Metaverses: A New Paradigm in Smart Manufacturing,” _IEEE/CAA Journal of Automatica Sinica_ , vol. 9, no. 12, pp. 2063–2070, Dec. 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9970436/
  * [10] L. D. Evjemo, T. Gjerstad, E. I. Grøtli, and G. Sziebig, “Trends in Smart Manufacturing: Role of Humans and Industrial Robots in Smart Factories,” _Current Robotics Reports_ , vol. 1, no. 2, pp. 35–41, Jun. 2020. [Online]. Available: https://link.springer.com/10.1007/s43154-020-00006-5