# A Benchmark Arabic Dataset for Commonsense Explanation

Saja AL-Tawalbeh, Mohammad AL-Smadi
Computer Science Department
Jordan University of Science and Technology
P.O.Box: 3030 Irbid 22110, Jordan.
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Language comprehension and commonsense knowledge validation by machines are
challenging tasks that are still under researched and evaluated for Arabic
text. In this paper, we present a benchmark Arabic dataset for commonsense
explanation. The dataset consists of Arabic sentences that does not make sense
along with three choices to select among them the one that explains why the
sentence is false. Furthermore, this paper presents baseline results to assist
and encourage the future evaluation of research in this field. The dataset is
distributed under the Creative Commons CC-BY-SA 4.0 license and can be found
on GitHub111Arabic dataset for commonsense explanation,
https://github.com/msmadi/.

Index terms— Natural Language Processing, Commonsense knowledge, Language
Model, Explanation, Arabic dataset

##  1 Introduction

Introducing the Commonsense to natural language understanding considered as a
qualitative change to emphasize the importance of how the machine able to
resolve the Commonsense problems. Various datasets have been provided aims to
support research in the field of Commonsense knowledge, such as, Choice of
Plausible Alternatives (COPA) [Roemmele et al., 2011] presented question
dataset related to events and consequences, aims to determine the suitable
reason of the premise. Moreover, JHU Ordinal Commonsense Inference (JOCI)
[Zhang et al., 2017] dataset presented data of five labels beginning from 5
(very likely) to 1 (impossible), which head to determine the reason for humans
response after a particular situation. Similarity, Situations with Adversarial
Generations (SWAG) [Zellers et al., 2018] which is considered as a large-scale
adversarial dataset presented for a sensible commonsense inference that used
to determine the expected response after certain action for a particular
situation. [Ostermann et al., 2018a] presented a dataset for a narrative text,
that aims to study various questions with their two candidate answers.

The recent datasets in the field of commonsense knowledge have been provided
by [Tawalbeh and AL-Smadi, 2020] proposed Arabic dataset for commonsense
validation. Moreover, SemEval-2020 [Wang et al., 2019, Wang et al., 2020] has
been collected by external reading materials have used to increase the
inspiration, for instance, Open Mind Common Sense [Havasi et al., 2010]
project and take advantage of existing commonsense reasoning questions, for
instance, The Winograd schema challenge [Levesque et al., 2012], Choice of
plausible alternatives [Roemmele et al., 2011], and SQUABU [Davis, 2016]. On
the other hand, Question Answering (QA) dataset assist the factual commonsense
knowledge, for instance, SQUABU [Davis, 2016] presented scientific questions
dataset. Moreover, SemEval-2018 [Ostermann et al., 2018b] presented dataset
for machine comprehension using questions commonsense knowledge to determine
the correct answer. However, COSMOS QA [Huang et al., 2019] presented
multiple-choice questions dataset. DCN [Xu et al., 2019] proposed deep comatch
network for multiple-choice reading comprehension based on BERT pre-trained
model. The evaluation done by RACE dataset with significant enhancement. [Si
et al., 2019] presented a study to investigate the information that BERT can
learn from MCRC datasets which are multiple-choice reading comprehension.
[Saeedi et al., 2020] used the recent dataset by SemEval to evaluate the
presented language model RoBERTa to solve Commonsense validation and
explanation task, their work have been conducted by express classification
task to multiple-choice task aims to increase the system performance.

The lack of the available resources in Arabic domain has encouraged the
researchers to assist this domain by providing Arabic datasets. To the best of
our knowledge, there is no publicly available dataset regarding commonsense
research field. This paper fosters the domain of Arabic Commonsense
Explanation (CSE) and provides a benchmark Arabic dataset as well. CSE
consists of sentences against the common facts and there true explanation.
Furthermore, this paper presented baseline results using the language model
methods (i.e., BERT, USE, and ULMFit) to assist and encourage the future
evaluation of research in this field. The major idea of the proposed dataset
is providing a wrong natural sentence against the real fact with three
explanation sentences, however, the system tends to determine the best
explanation.

File | Train | Validation | Test
---|---|---|---
Total | 10000 | 1000 | 1000
Table 1: The Dataset distribution

ID FalseSent OptionA OptionB OptionC label 1089 وضعت جدها في المهد (She put
her grandfather in the cot) الأجداد عادة ما يكونون كبار السن بينما يكون
الأطفال صغارًا جدًا (Grandfathers are usually very old while babies are
usually very young) لا يستطيع الجد أن يلد طفلاً (A grandfather cannot give
birth to a baby) البالغ أكبر بكثير من المهد (An adult is much bigger than a
cot) C 340 دب كان يقود سيارة في كندا (A bear was driving a car in Canada)
الدببة عادة ما تكون بنية اللون (bears are usually brown) الدب ليس لديه رخصة
سيارة وغير قادر على القيادة (Bear don’t have car license and are unable to
drive) يمكن لبعض الناس رؤية الدببة تقود سيارة في كندا (Some people can see
bears driving a car in Canada) B 1785 وضع يخت في السرير (He put a yacht in
bed) اليخت أكبر بكثير من السرير (A yacht is much larger than a bed) اليخوت
عادة ما تكون بيضاء بينما الأسرة بيج (Yachts are usually white while the beds
are beige) هناك سرير على اليخت (there is a bed on the yacht) A

Table 2:  Selected examples from the provided dataset

##  2 Dataset Collection and Translation

The commonsense explanation problem has natural sentence against the fact with
three sentences one of them explains why that sentence does not make sense.
Each sentence has labeled with the suitable explanation. The original dataset
for English language provides by SemEval-2020 [Wang et al., 2020] 222
https://github.com/wangcunxiang/SemEval2020-Task4-Commonsense-Validation-and-
Explanation Commonsense Validation and Explanation (ComVE) task which inspired
by [Wang et al., 2019]. This paper focused on the explanation task for Arabic
dataset. Each example in the provided dataset is composed of four sentences:
{s1, o1, o2, o3} . S1 is a natural sentence against the fact and does not make
sense, where o1, o2, and o3 are three options that explain why that sentence
does not make sense and the task is to select the most correct out of them.

To the best of our knowledge, there is no publicly available dataset to be
used in the research of the commonsense explanation for Arabic language. Based
on that, we are providing a benchmark Arabic dataset for the Commonsense
explanation problem (why a statement against the natural fact does not make
sense). The dataset is provided with 12k rows divided as following: train with
10k , validation with 1k, and test file with 1k (see Table 1). Each file
consists of four columns, the first column contains the false sentence,
whereas, the other three columns contain three options to explain why that
sentence is wrong. Table 2 provides examples of the presented Arabic dataset
for the commonsense explanation task.

##  3 Experimentation & Results

As we aim to present an Arabic benchmark dataset for commonsense explanation,
the dataset is provided with a baseline evaluation to address the research
task and problem discussed in section 2. The baseline evaluation is based on
several state-of-art transfer based language models i.e. BERT [Devlin et al.,
2018], USE [Cer et al., 2018], and ULMFit [Howard and Ruder, 2018]. For each
false sentence in the test file, the baseline selects the explanation sentence
among the three choices provided for this purpose. In order to evaluate
baseline models, the accuracy of the approach is measured. We also provided an
evaluation code which can be downloaded with the dataset. Future researchers
can use the baseline results to evaluate the performance of their proposed
research.Table 3 presents the evaluation results for the baseline models.

Model | Accuracy
---|---
Random | 32.5 %
USE | 33 %
ULMFiT | 32.8 %
BERT | 34.20 %
Table 3:  The experimentation results for the trained baseline models

As shown in Table 3, BERT and USE have achieved the highest scores compared to
the random results and ULMFiT methods. The results go inline with related work
as BERT achieved 45.6% accuracy for the original English dataset [Wang et al.,
2019].

##  4 Conclusion

This paper presents a benchmark Arabic dataset for commonsense explanation.
The dataset is distributed under the Creative Commons CC-BY-SA 4.0 license and
can be found on GitHub333Arabic dataset for commonsense explanation,
https://github.com/msmadi/. We also trained a state-of-the-art transformer-
based language models (i.e., BERT, USE, and ULMFit) as baseline research using
the benchmark datset. Evaluation results show how challenging is the task the
where the best performing baseline model (i.e. BERT) achieved only 34.20% of
accuracy compared to 32.5% of accuracy for the random model.

## Acknowledgments

This research is partially funded by Jordan University of Science and
Technology, Research Grant Number: 20170107.

## References

  * [Cer et al., 2018]  Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al.  2018\.  Universal sentence encoder.  arXiv preprint arXiv:1803.11175.
  * [Davis, 2016]  Ernest Davis.  2016\.  How to write science questions that are easy for people and hard for computers.  AI magazine, 37(1):13–22.
  * [Devlin et al., 2018]  Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  2018\.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805.
  * [Havasi et al., 2010]  Catherine Havasi, Robert Speer, Kenneth Arnold, Henry Lieberman, Jason Alonso, and Jesse Moeller.  2010\.  Open mind common sense: Crowd-sourcing for common sense.  In Workshops at the Twenty-Fourth AAAI Conference on Artificial Intelligence.
  * [Howard and Ruder, 2018]  Jeremy Howard and Sebastian Ruder.  2018\.  Universal language model fine-tuning for text classification.  arXiv preprint arXiv:1801.06146.
  * [Huang et al., 2019]  Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi.  2019\.  Cosmos qa: Machine reading comprehension with contextual commonsense reasoning.  arXiv preprint arXiv:1909.00277.
  * [Levesque et al., 2012]  Hector Levesque, Ernest Davis, and Leora Morgenstern.  2012\.  The winograd schema challenge.  In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning.
  * [Ostermann et al., 2018a]  Simon Ostermann, Ashutosh Modi, Michael Roth, Stefan Thater, and Manfred Pinkal.  2018a.  Mcscript: A novel dataset for assessing machine comprehension using script knowledge.  arXiv preprint arXiv:1803.05223.
  * [Ostermann et al., 2018b]  Simon Ostermann, Michael Roth, Ashutosh Modi, Stefan Thater, and Manfred Pinkal.  2018b.  Semeval-2018 task 11: Machine comprehension using commonsense knowledge.  In Proceedings of the 12th International Workshop on semantic evaluation, pages 747–757.
  * [Roemmele et al., 2011]  Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S Gordon.  2011\.  Choice of plausible alternatives: An evaluation of commonsense causal reasoning.  In 2011 AAAI Spring Symposium Series.
  * [Saeedi et al., 2020]  Sirwe Saeedi, Aliakbar Panahi, Seyran Saeedi, and Alvis C Fong.  2020\.  Cs-nlp team at semeval-2020 task 4: Evaluation of state-of-the-artnlp deep learning architectures on commonsense reasoning task.  arXiv preprint arXiv:2006.01205.
  * [Si et al., 2019]  Chenglei Si, Shuohang Wang, Min-Yen Kan, and Jing Jiang.  2019\.  What does bert learn from multiple-choice reading comprehension datasets?  arXiv preprint arXiv:1910.12391.
  * [Tawalbeh and AL-Smadi, 2020]  Saja Tawalbeh and Mohammad AL-Smadi.  2020\.  Is this sentence valid? an arabic dataset for commonsense validation.  arXiv preprint arXiv:2008.10873.
  * [Wang et al., 2019]  Cunxiang Wang, Shuailong Liang, Yue Zhang, Xiaonan Li, and Tian Gao.  2019\.  Does it make sense? and why? a pilot study for sense making and explanation.  arXiv preprint arXiv:1906.00363.
  * [Wang et al., 2020]  Cunxiang Wang, Shuailong Liang, Yili Jin, Yilong Wang, Xiaodan Zhu, and Yue Zhang.  2020\.  Semeval-2020 task 4: Commonsense validation and explanation.  arXiv preprint arXiv:2007.00236.
  * [Xu et al., 2019]  K. Xu, Jing Tin, and J. Youn.  2019\.  A bert based model for multiple-choice reading comprehension.
  * [Zellers et al., 2018]  Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi.  2018\.  Swag: A large-scale adversarial dataset for grounded commonsense inference.  arXiv preprint arXiv:1808.05326.
  * [Zhang et al., 2017]  Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme.  2017\.  Ordinal common-sense inference.  Transactions of the Association for Computational Linguistics, 5:379–395.


# Learning robot motor skills with mixed reality

Eric Rosen  Brown University
<EMAIL_ADDRESS>Sreehari Rammohan  Brown University
<EMAIL_ADDRESS>Devesh Jha  MERL
<EMAIL_ADDRESS>
###### Abstract

Mixed Reality (MR) has recently shown great success as an intuitive interface
for enabling end-users to teach robots. Related works have used MR interfaces
to communicate robot intents and beliefs to a co-located human, as well as
developed algorithms for taking multi-modal human input and learning complex
motor behaviors. Even with these successes, enabling end-users to teach robots
complex motor tasks still poses a challenge because end-user communication is
highly task dependent and world knowledge is highly varied. We propose a
learning framework where end-users teach robots a) motion demonstrations, b)
task constraints, c) planning representations, and d) object information, all
of which are integrated into a single motor skill learning framework based on
Dynamic Movement Primitives (DMPs). We hypothesize that conveying this world
knowledge will be intuitive with an MR interface, and that a sample-efficient
motor skill learning framework which incorporates varied modalities of world
knowledge will enable robots to effectively solve complex tasks.

###### Index Terms:

robot learning, mixed reality, human-robot interaction

##  I Introduction

Mixed Reality (MR) has recently garnered interest as an intuitive interface
for teaching robots. MR enables bi-directional human-robot interaction by
superimposing visualizations on the shared environment to communicate robot
knowledge and beliefs to the human, and providing a means for humans to
communicate to the robot via multi-modal channels like speech, eye-gaze, and
hand-gestures. There have been many recent studies that have demonstrated the
efficacy and ease-of-use of MR interfaces for teaching robots compared to
traditional methods like kinesthetic teaching and keyboard/monitor setups.

Figure 1: A figure describing the types of world knowledge our proposed MR
interface can enable users to provide, and how we propose a single motor skill
learning framework (CDMPs) can leverage that world knowledge. More details on
how each type of world knowledge is communicated with MR and integrated into
the CDMP can be found in Section III

Human communication is varied and task dependent, making robot teaching
difficult. We identify 4 types of world knowledge a human may wish to impart:
a) motion demonstrations, b) task constraints, c) planning representations,
and d) object information. We address the research problem of incorporating
this world knowledge into a single motor skill learning framework with MR.

We propose that a suitable motor skill learning framework must be sample-
efficient (to reduce the teaching burden from a human), provide guarantees on
constraint satisfaction (for safety), and be adaptable to new goals (to
account for novel task dependent constraints). We propose Constrained Dynamic
Movement Primitives (CDMPs) as an appropriate family of motor skills for
humans to teach using MR (Figure 1). We will develop an MR interface for users
to specify all the aforementioned types of world knowledge for a peg-in-hole
insertion task, and demonstrate that all the information can effectively be
incorporated into a CDMP learning framework to learn complex motor skills.

##  II Related work

Related works within the VAM-HRI (Virtual, Augmented, and Mixed Reality for
Human-Robot Interaction) have developed interfaces and learning algorithms for
different types of world knowledge. Here we discuss both related works with MR
interfaces for teaching different types of world knowledge, and also give
background on Constrained DMPs.

###  II-A Mixed Reality for Robot Learning

MR has been used in various contexts for robot learning. Here we discuss 4
different types of world knowledge humans have taught robots: motion
demonstrations, task constraints, planning representations, and object
information.

Motion demonstrations Spatially-tracked hand controllers have shown great
success as an intuitive interface to teleoperate the pose of a robot’s end
effector to perform manipulation tasks [1]. For robot learning with MR, users
typically provide motion demonstrations by teleoperating the robot with
spatially-tracked controller (often mapped to the end-effector with Cartesian
control), and these teleoperation demonstrations are used as supervised data
within a motor skill learning framework. [2] demonstrated that commercially-
available VR systems could be used to provide demonstrations of complex
manipulation tasks to support effective imitation learning with deep neural
networks. [3] used an apprenticeship model to efficiently use a human’s time
when teleoperating in VR to teach a grasping task.

Task constraints Visualizing task constraints and enabling end-users to easily
edit them with MR has recently shown success for positively augmenting the
learning process. [4] propose an MR system for safety-aware HRI where users’
skeletons are registered and tracked to calculate safety distances between the
human and robot, which are treated as task constraints that can be visualized
to the user. Beyond visualizing constraints, [5] proposed an AR system to
teach robots motor skills using Concept-Constrained Learning from
Demonstration (CC-LfD), allowing users to see and interact with visualizations
of constraints superimposed on the workspace. [6] expanded on CC-LfD by
incorporating the ability to perform skill maintenance and skill alteration
for different contexts.

Planning representations Tasks are often solved by decomposing the goal into
subgoals, and related works have investigated creating plans comprised of
high-level action sequences with MR. [7] used an MR system to segment
demonstrations in MR with a key-frame system, and then fitted different motor
policies with each of the sub-trajectories. [8] developed an AR system for
users to specify trajectory planning information that leverages dynamics
modeling to produce smooth trajectories between initial and end points for the
robot.

Object information MR provides an intuitive way for humans to communicate
information about objects to a robot in a shared environment. [9] proposed an
MR interface for users to build a semantic map of an environment that included
information such as object pose, semantic attributes and state, and action-
related information like grasp strategies and approach vectors. [10] propose
an AR system for intent recognition that enables users to directly specify 3D
pose information about objects in the scene, which can be used to model and
infer a human’s intended goal object using Hidden Markov Models (HMM).

Figure 2: The Kuka Arm and Peg-In-Hole insertion task. Figure 3: Insertion
Trajectory for Peg in Hole Task

###  II-B Constrained Dynamic Movement Primitives (CDMPs)

Constrained DMPs (CDMPs) were recently introduced in [11] as an extension of
the original DMP formulation introduced in [12]. The idea in CDMP is to allow
constraint satisfaction for the original DMP by introducing a perturbation in
the learned forcing function. More concretely, the CDMP is represented by the
following system of equations:

$\begin{bmatrix}\dot{s}\\\ \dot{z}\\\ \dot{y}\\\
\end{bmatrix}=\frac{1}{\tau}\begin{bmatrix}-\alpha_{s}s\\\
\left(\alpha_{z}(\beta_{z}(g-y)-z)+\tilde{f}(s)\right)\\\ z\\\ \end{bmatrix}$
(1)

where,

$\tilde{f}(s)=\frac{\sum_{i=1}^{N}(w_{i}-\zeta_{i})\psi_{i}(s)}{\sum_{i=1}^{N}\psi_{i}(s)}$
(2)

where, $w_{i}$ are the weights of the forcing function learned using the
expert demonstration and $\zeta_{i}$ are the weights learnt to satisfy the
operational constraints by solving a constrained trajectory optimization
problem. Note that the forcing function presented in [12] consists only of the
weights $w_{i}$ that can be learned from the expert demonstration by solving a
locally weighted regression [12]. In the formulation presented in [11],
constraints could be represented as barrier functions that guarantee safety or
constraint satisfaction if a feasible solution could be found. These
constraints could represent collision constraints or more complex constraints
that could represent task-specific constraints. CDMP could be solved using
existing nonlinear programming solvers [13].

##  III Proposed Work

Our proposed work is to build an MR interface that is able to take in the four
types of aforementioned world knowledge, and apply it to the CMDP learning
framework for enabling end-users to teach a robot how to perform a peg-in-hole
insertion task (Figure 2). For supplying motion demonstrations, users will be
able to teleoperate the robot’s end effector by manipulating a spatially-
tracked hand controller (Figure 3). We will use these demonstrations to fit
the forcing function of the CDMP using Locally Weighted Regression (LWR),
which is both sample and computationally efficient (Figure 4). For specifying
task constraints, users will be able to specify and adjust geometric shapes
(for now, spheres and rectangular prisms) over the workspace marking zones of
space where the robot’s end effector can not enter. The CDMP will incorporate
these task constraints into the motor skill learning by automatically
constructing an inequality using signed distance functions that respect the
spatial constraints specified by the user, which provide guarantees on
constraint satisfaction. For building plannable representations, users will be
able to designate keypoints within a trajectory where skills begin and end.
For each keypoint, we will segment the trajectory and train a separate CDMP on
each subset trajectory. At inference time, these CDMPs are stitched together
to chain skills and solve the given task. For specifying object information,
users will be able to generate object models and directly overlay them on top
of their real-world counterparts, and associate different motor skills with
each of the objects. CMDPs will have their action-space defined in $SE(3)$,
and will leverage the specified object information to determine what frame of
references the action-space is in (either the global work-space or object-
centric), which will in turn define what frame of reference reparameterized
goals are with respect to.

Figure 4: Fitted DMP for Peg-in-Hole insertion task.

Once we create our propsed MR interface and connect all the components to a
CDMP implementation, we will conduct a user study to understand how easy it is
for users to communicate these types of world knowledge, and how sufficient
they are for enabling robots to learn complex motor works. We will have end-
users try our robot teaching system with a peg-in-hole insertion task. This
will give us a better understanding of whether or not there are additional
types of world knowledge that should be incorporated into the learning
framework. After, we can then compare the efficacy of our MR interface to
traditional interface methods (such as kinesthetic teaching and desktop
setups) for enabling end-users to teach robots a complex task such as peg-in-
hole insertion.

##  IV Conclusion

We have proposed an MR interface and compatible motor learning framework for
enabling end-users to teach robots manipulation skills. We hypothesize that MR
can be an effective interface for expressing four types of world knowledge:
motion demonstrations, task constraints, plannable representations, and object
information. We suggest that CDMPs are an appropriate motor skill learning
framework for learning these different types of world knowledge without
requiring large numbers of interactions. Next steps will be to build the MR
interface and connect it to the motor skill learning framework, and conduct a
user study to compare MR against traditional robot interfaces for teaching
peg-in-hole insertion.

##  V Acknowledgements

We express appreciation for the support from the Humans To Robots Laboratory
and the Intelligent Robot Lab at Brown University for supplying our research
with robotic and MR platforms.

## References

  * [1] D. Whitney, E. Rosen, E. Phillips, G. Konidaris, and S. Tellex, “Comparing robot grasping teleoperation across desktop and virtual reality with ros reality,” in _Robotics Research_. Springer, 2020, pp. 335–350.
  * [2] T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, “Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2018, pp. 5628–5635.
  * [3] J. DelPreto, J. I. Lipton, L. Sanneman, A. J. Fay, C. Fourie, C. Choi, and D. Rus, “Helping robots learn: a human-robot master-apprentice model using demonstrations via virtual reality teleoperation,” in _2020 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 10 226–10 233.
  * [4] S. H. Choi, K.-B. Park, D. H. Roh, J. Y. Lee, M. Mohammed, Y. Ghasemi, and H. Jeong, “An integrated mixed reality system for safety-aware human-robot collaboration using deep learning and digital twin generation,” _Robotics and Computer-Integrated Manufacturing_ , vol. 73, p. 102258, 2022\.
  * [5] M. B. Luebbers, C. Brooks, M. J. Kim, D. Szafir, and B. Hayes, “Augmented reality interface for constrained learning from demonstration,” in _Proceedings of the 2nd International Workshop on Virtual, Augmented and Mixed Reality for HRI (VAM-HRI)_ , 2019.
  * [6] M. B. Luebbers, C. Brooks, C. L. Mueller, D. Szafir, and B. Hayes, “Arc-lfd: Using augmented reality for interactive long-term robot skill maintenance via constrained learning from demonstration,” in _2021 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2021, pp. 3794–3800.
  * [7] S. Y. Gadre, “Teaching robots using mixed reality,” _Brown Univ. Dep. Comput. Sci.[Google Scholar]_ , 2018.
  * [8] H. Fang, S. K. Ong, and A. Y.-C. Nee, “Robot programming using augmented reality,” in _2009 International Conference on CyberWorlds_. IEEE, 2009, pp. 13–20.
  * [9] E. Rosen, N. Kumar, N. Gopalan, D. Ullman, G. Konidaris, and S. Tellex, “Building plannable representations with mixed reality,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020, pp. 11 146–11 153.
  * [10] D. Puljiz, B. Zhou, K. Ma, and B. Hein, “Hair: Head-mounted ar intention recognition,” _arXiv preprint arXiv:2102.11162_ , 2021.
  * [11] S. Shaw, D. K. Jha, A. Raghunathan, R. Corcodel, D. Romeres, G. Konidaris, and D. Nikovski, “Constrained dynamic movement primitives for safe learning of motor skills,” in _2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2022, p. under review.
  * [12] S. Schaal, “Dynamic movement primitives-a framework for motor control in humans and humanoid robotics,” in _Adaptive motion of animals and machines_. Springer, 2006, pp. 261–280.
  * [13] A. U. Raghunathan, D. K. Jha, and D. Romeres, “PYROBOCOP : Python-based robotic control & optimization package for manipulation and collision avoidance,” _CoRR_ , vol. abs/2106.03220, 2021.


* [36] H. Pollard, The behavior of gravitational systems, J. Math. Mech. 17 (1967), 601-611.
  * [37] H. Pollard, Celestial Mechanics, Carus Math. Monogr., vol.18, Mathematical Association of America, 1976.
  * [38] H. Pollard, D. Saari, Escape from a gravitational system of positive energy, Celestial Mech. 1 (1970), 347-350.
  * [39] E. Popescu, D. Pricopi, Orbits stability under the influence of Mücket-Treder potential, Astrophys Space Sci. 365 (2020), 191.
  * [40] D. Saari, Expanding gravitational systems, Trans. Amer. Math. Soc. 156 (1971), 219-240.
  * [41] H. Seeliger, Ueber das Newton’sche Gravitationsgesetz, Astron. Nachr. 137 (1895), 129-136.
  * [42] J. C. del Valle, D. J. Nader, Toward the theory of the Yukawa potential, J. Math. Phys. 59 (2018), 102103.
  * [43] A. Wintner, Galilei group and law of gravitation, Amer. J. Math. 60 (1938), 473-476.
  * [44] Q. Wang, The global solution of the N-body problem, Celestial Mech. Dynam. Astronom. 50 (1991) 73-88.
  * [45] H. von Zeipel, Sur les singularités du probléme des $n$ corps, Ark. Math. Astr. Fys. 4 (1908), 1-4.

Jiayin Liu

School of Mathematical Science, Beihang University, Changping District Shahe
Higher Education Park South Third Street No. 9, Beijing 102206, P. R. China

and

Department of Mathematics and Statistics, University of Jyväskylä, Jyväskylä,
Finland

E-mail<EMAIL_ADDRESS>

Duokui Yan

School of Mathematical Science, Beihang University, Changping District Shahe
Higher Education Park South Third Street No. 9, Beijing 102206, P. R. China

E-mail<EMAIL_ADDRESS>

Yuan Zhou

School of Mathematical Sciences, Beijing Normal University, Haidian District
Xinejikou Waidajie No.19, Beijing 100875, P. R. China

E-mail<EMAIL_ADDRESS>


# Artificial Neural Network Approach for the Identification of Clove Buds
Origin Based on Metabolites Composition

Rustam
Industrial and Financial Mathematics Research Group
Faculty of Mathematics and Natural Sciences
Institut Teknologi Bandung, Indonesia
<EMAIL_ADDRESS>
& Agus Y. Gunawan
Industrial and Financial Mathematics Research Group
Faculty of Mathematics and Natural Sciences
Institut Teknologi Bandung, Indonesia
& Made T. A. P. Kresnowati
Food and Biomass Processing Technology Research Group
Faculty of Industrial Technology
Institut Teknologi Bandung, Indonesia
alternative email<EMAIL_ADDRESS>
###### Abstract

This paper examines the use of artificial neural network approach in
identifying the origin of clove buds based on metabolites composition.
Generally, large data sets are critical for accurate identification. Machine
learning with large data sets lead to precise identification based on origins.
However, clove buds uses small data sets due to lack of metabolites
composition and their high cost of extraction. The results show that
backpropagation and resilient propagation with one and two hidden layers
identifies clove buds origin accurately. The backpropagation with one hidden
layer offers 99.91% and 99.47% for training and testing data sets,
respectively. The resilient propagation with two hidden layers offers 99.96%
and 97.89% accuracy for training and testing data sets, respectively.

_Keywords_ Artificial neural networks $\cdot$ Backpropagation $\cdot$
Resilient propagation $\cdot$ Clove buds

##  1 Introduction

There is variation in the flavor and aroma of different plantation
commodities. For example, in Indonesia, the clove buds from Java has a
prominent wooden aroma and sour flavor while those in Bali have a sweet-spicy
flavor [1]. Arabica coffee from Gayo has a lower acidity and a strong
bitterness. In contrast, coffee from Toraja has a medium browning, tobacco, or
caramel flavor, not too acidic and bitter. Furthermore, Kintamani coffee from
Bali has a fruit flavor and acidity, mixed with a fresh flavor. Contrastingly,
Coffee from Flores has a variety of flavors ranging from chocolate, spicy,
tobacco, strong, citrus, flowers and wood. Coffee from Java has a spicy aroma
while that from Wamena has a fragrant aroma and without pulp [2].

The specific flavors and aromas are attributed to the composition of
commodities’ metabolites. Generally, specific metabolite contributes is
responsible for particular flavors and aroma. For this reason, it is vital to
recognize the characteristics of each plantation commodity based on metabolite
composition. This study investigates the origin of clove buds. This helps to
maintain the flavor of a product using clove buds as a mixture. Also, the
characteristics of food products can be predicted based on the origin of clove
buds used due to differences flavour and taste between regions [3].

Metabolic profiling is a widely used approach in obtaining information related
to metabolites contained in a biological sample. This is a quantitative
measurement of metabolites from biological samples [4, 5]. To give meaning to
the metabolites data sets, chemometrics technique was developed. This is a
chemical sub-discipline that uses mathematics, statistics and formal logic to
gain knowledge about chemical systems. It provides maximum relevant
information by analysing metabolites data sets from biological samples [6].
Additionally, it is used in pattern recognition of metabolites data sets in
complex chemical systems [3]. Pattern recognition in biological samples
identifies specific metabolites or biomarkers that form particular flavor and
aroma.

Artificial neural networks have been widely used in pattern recognition [7]
and other applications in various fields as shown by some researchs [8, 9, 10,
11, 12, 13]. However, it has not been fully implemented, especially in clove
buds. The small data sets available limits the implementation of artificial
neural networks for clove buds. This is attributed to the lack of metabolite
composition in the clove buds and high cost for extracting them. Furthermore,
some clove buds have zero metabolite concentration. However, this is because
of inefficient tools in the laboratory to detect metabolites whose values are
very small. Therefore this study implements artificial neural networks as
pattern recognition in clove buds data sets. Each origin of clove buds has
specific metabolites as a biomarker.

##  2 Materials and Methods

###  2.1 Materials

This study uses clove buds data sets obtained from Kresnowati et al. [3]. It
examined clove buds from four origins in Indonesia, including Java, Bali,
Manado and Toli-Toli. Each origin has three regions, and therefore, there are
twelve regions in total. In the laboratory, eight experiments are carried out
in each region, except for Java with only six experiments. Each experiment, 47
types of metabolites were recorded. In the matrix, data sets are 94 $\times$
47\. The row and column represent the number of experiments and metabolites,
respectively.

###  2.2 Data Preprocessing

In total, the clove buds data sets have a wide range, specifically between
$10^{-4}$ and 10. Therefore, logarithmic transformations are used to obtain
reliable numerical data. Since some metabolites data have zero concentration,
logarithmic transformation cannot be applied directly. This is because their
concentrations range below the specified threshold. The metabolite data with
zero concentration are not removed because of acting as biological markers.
Therefore, they are replaced with value of one order smaller than the smallest
concentration available. In this case, the zeros are replaced with $10^{-5}$.
Before implementing artificial neural networks, one stage preprocessing clove
buds data sets from [14] are added to normalize the values of metabolites
data. Normalization ensure that each data has the same influence or
contribution to determine its origin. The following normalization formula is
used [15]

$z_{kl}=\frac{{x}_{kl}-\overline{x}}{s}.$ (1)

Here $z_{kl}$ is the result of normalization of $x_{kl}$, $\overline{x}$ is
the mean of the $k$-th experiment and $s$ is

$s=\sqrt{\sum_{k=1}^{n}\frac{{x}_{kl}-\overline{x}}{n-1}}.$ (2)

###  2.3 Artificial Neural Network

Artificial neural networks are a false representation of the human brain that
simulates the learning process [16]. Backpropagation and resilient propagation
are learning algorithms widely used in artificial neural networks [17, 18, 19,
20, 21, 22, 23, 24, 25, 26, 27]. In this study, two different network
architectures, including resilient and backpropagation are used. The first and
second architectures consist of two and one hidden layers, respectively.

####  2.3.1 Backpropagation Learning Algorithm

Backpropagation learning algorithm is based on the repeated use of chain rules
to calculate the effect of each weight in network concerning the error
function $E$ [28].

$\frac{\partial E}{\partial w_{ij}}=\frac{\partial E}{\partial
o_{i}}\frac{\partial o_{i}}{\partial net_{i}}\frac{\partial net_{i}}{\partial
w_{ij}}$ (3)

where $w_{ij}$ is the weight from $j-th$ neuron to $i-th$ neuron, $o_{i}$ is
the output, and $net_{i}$ is the weighted number of neurons input $i$. Once
the partial derivatives for each weight are known, the goal of minimizing the
error function is achieved with gradient descent [28]:

$w_{ij}^{(t+1)}=w_{ij}^{(t)}-\epsilon\frac{\partial E}{\partial w_{ij}}^{(t)}$
(4)

where $t$ is iteration and $0<\epsilon<1$ the learning rate. From the Equation
(4), choosing a large learning rate (close to 1), allows for oscillations.
This makes the error fall above the specified tolerance value and lessens the
identification accuracy. Conversely, in case, the learning rate ($\epsilon$)
is too small (close to 0), many steps are needed for convergence of the error
function $E$. To avoid these, the backpropagation learning algorithm is
expanded by adding the momentum parameter $(0<\alpha<1)$ as shown in Equation
(5). The addition of momentum parameter also accelerates the convergence of
error function [28].

$\Delta w_{ij}^{(t+1)}=-\epsilon\frac{\partial E}{\partial
w_{ij}}^{(t)}+\alpha\Delta w_{ij}^{(t-1)}$ (5)

where it measures the effect of previous step on the currently.

To activate neurons in the hidden and output layers, the sigmoid activation
function is used. Three essential properties used in backpropagation and
resilient propagation include bounded, monotonic and continuously
differentiable. This helps to convert a weighted amount of input into an
output signal for each neuron $i$ as shown by Equation (6) [29].

$O_{i}=f(I_{i})=\frac{1}{1+e^{-\sigma I_{i}}}.$ (6)

where $I_{i}$ is the input of $i$-th weighted number of neuron, $\sigma$ the
slope parameter of the sigmoid activation function and $O_{i}$ the output of
$i$-th neuron. The threshold used on the output layer for the sigmoid
activation function is

$O_{i}=\left\\{\begin{matrix}1~{}~{}if~{}~{}O_{i}\geq 0.5\\\
0~{}~{}if~{}~{}O_{i}<0.5\end{matrix}\right.$ (7)

The weighted amount input is given in the following equation [29].

$\sum_{i=1}^{n}w_{ij}O_{i}+w_{Bj}O_{B}.$ (8)

The sum of $i$ represents the input received from all neurons in the input
layer, while $B$ is the bias neuron. Weight $w_{ij}$ is associated with
connections from $i$-th neuron to $j$-th neuron, while $w_{Bj}$ weight relates
to the connections from biased to $j$-th neuron. The weighted amount obtained
in the hidden and the output layers are activated by substituting the weighted
amount from Equation (8) to be an exponent in Equation (6).

####  2.3.2 Resilient propagation Learning Algorithm

Riedmiller et al. in [28] proposed a resilient propagation learning algorithm
developed by the backpropagation algorithm. The algorithm directly adapts to
the weight value based on local gradient information. Riedmiller et al. [28]
introduced an update value $\Delta_{ij}$ for each weight determining the size
of weight update. The adaptive update value evolves during the learning
process based on its local sight on the error function $E$, according to the
following learning rule [28]:

$\Delta_{ij}^{t}=\left\\{\begin{matrix}\eta^{+}*\Delta_{ij}^{(t-1)},~{}~{}if~{}~{}\frac{\partial
E}{\partial w_{ij}}^{(t-1)}*\frac{\partial E}{\partial w_{ij}}^{(t)}>0\\\
\eta^{-}*\Delta_{ij}^{(t-1)},~{}~{}if~{}~{}\frac{\partial E}{\partial
w_{ij}}^{(t-1)}*\frac{\partial E}{\partial w_{ij}}^{(t)}<0\\\
\Delta_{ij}^{(t-1)}~{}~{}~{}~{},~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}else\\\
\end{matrix}\right.$ (9)

where ($0<\eta^{-}<1<\eta^{+}$) $\eta^{-}$ and $\eta^{+}$ represents the
decrease and increase factors, respectively. According to this adaptation
rule, every time the partial derivative of the corresponding weight $w_{ij}$
changes its sign, which indicates that the last update is too big and the
algorithm is above the local minimum, the update value $\Delta_{ij}$ is
decreased by the factor $\eta^{-}$. In case the derivative retains its sign,
the update value slightly increases to accelerate convergence in the shallow
regions [28].

Once the update value for each weight is adjusted, the update weight itself
follows rule stating that in case the derivative is positive, the weight is
decreased by its update value. If the derivative is negative, the update value
is added

$\Delta
w_{ij}^{t}=\left\\{\begin{matrix}-\Delta_{ij}^{t-1},~{}~{}if~{}~{}\frac{\partial
E}{\partial w_{ij}}^{(t)}>0\\\ +\Delta_{ij}^{t-1},~{}~{}if~{}~{}\frac{\partial
E}{\partial w_{ij}}^{(t)}<0\\\ 0~{}~{}~{}~{}~{},~{}~{}~{}~{}~{}~{}~{}else\\\
\end{matrix}\right.$ (10)

$w_{ij}^{t+1}=w_{ij}^{t}+\Delta w_{ij}^{t}$ (11)

However, in case the partial derivative sign changes, which means the previous
step was too large and the minimum missed, the previous weight update is
reverted:

$\Delta w_{ij}^{(t)}=-\Delta w_{ij}^{(t-1)},~{}~{}if~{}~{}\frac{\partial
E}{\partial w_{ij}}^{(t-1)}*\frac{\partial E}{\partial w_{ij}}^{(t)}<0$ (12)

Due to the ’backtracking’ weight step, the derivative should change its sign
once again in the next step. To avoid another problem, there should be no
adaptation of update value in the succeeding step. In practice, this can be
carried out by setting $\frac{\partial E}{\partial w_{ij}}^{(t-1)}$ = 0 in the
$\Delta_{ij}$ adaptation rule. The update values and weights are changed every
time the whole set of patterns is presented once to the network (learning by
epoch).

The following shows the process of adaptation and resilient propagation
learning process. The $\mathbf{minimum(maximum)}$ operator is expected to
provide a minimum or maximum of two numbers. The sign operator returns +1 in
the argument is positive, -1 in case the it is negative, and 0 for otherwise.

$\begin{split}For~{}each~{}weight~{}and~{}bias\\{\\\
\mathbf{if}(\frac{\partial E}{\partial w_{ij}}^{(t-1)}*\frac{\partial
E}{\partial w_{ij}}^{(t)}>0)\mathbf{\,then\\{}\\\
\Delta_{ij}^{(t)}=\mathbf{minimum}(\Delta_{ij}^{(t-1)}*\eta^{+},\Delta_{\mathit{\mathrm{max}}})\\\
\Delta w_{ij}^{(t)}=\mathbf{sign}(\frac{\partial E}{\partial
w_{ij}}^{(t)}*\Delta_{ij}^{(t)})\\\ w_{ij}^{(t+1)}=w_{ij}^{(t)}+\Delta
w_{ij}^{(t)}\\}\\\ \mathbf{else\>if}(\frac{\partial E}{\partial
w_{ij}}^{(t-1)}*\frac{\partial E}{\partial
w_{ij}}^{(t)}<0)\mathbf{\,then\\{}\\\
\Delta_{ij}^{(t)}=\mathbf{maximum}(\Delta_{ij}^{(t-1)}*\eta^{-},\Delta_{\mathit{\mathrm{min}}})\\\
w_{ij}^{(t+1)}=w_{ij}^{(t)}-\Delta w_{ij}^{(t-1)}\\\ \frac{\partial
E}{\partial w_{ij}}^{(t)}=0\\}\\\ \mathbf{if}(\frac{\partial E}{\partial
w_{ij}}^{(t-1)}*\frac{\partial E}{\partial
w_{ij}}^{(t)}=0)\mathbf{\,then\\{}\\\ \Delta
w_{ij}^{(t)}=-\mathbf{sign}(\frac{\partial E}{\partial
w_{ij}}^{(t)}*\Delta_{ij}^{(t)})\\\ w_{ij}^{(t+1)}=w_{ij}^{(t)}+\Delta
w_{ij}^{(t)}\\}\\\ \\}\end{split}$ (13)

##  3 Results and Discussions

In this study, the percentage of training and testing data sets are 80% and
20%, respectively. The metabolites data sets in matrix are 94$\times$47\. Out
of 94 rows, 75 were chosen randomly as training data sets, while the remaining
were used as testing. The selection of training data sets is carried out
randomly 30 times. Therefore, in each network architecture, there are 30
values for the percentage of identification accuracy, coefficient of
determination and the mean squared error ($MSE$). The average is chosen as a
representative of the 30 values. In each network architecture, learning rate
($\epsilon$) 0.9, momentum parameter ($\alpha$) 0.1 and maximum epoch 5000 are
used with error target $10^{-3}$. In this study, each origin is represented by
a binary code. Specifically, the binary code for the Java origin is 1000, Bali
0100, Manado 0010 and Toli-Toli 0001. The calculation of identification
accuracy and $MSE$ is shown in Equation (14) and (15).

$\%\>accuracy=\frac{a}{k}100\%$ (14)

Where $a$ is the number of origins identified correctly, while $k$ is the
total number. $MSE$ calculated by the following equation [29]

$MSE=\frac{1}{m\cdot n}\sum_{p=1}^{m}\sum_{k=1}^{n}(T_{kp}-O_{kp})^{2}.$ (15)

where $T_{kp}$ is the desired target, $O_{kp}$ the network output and $p$ the
variable corresponding to the number of origins.

The suitability between the expected target and network output was evaluated
based on the coefficient of determination $R^{2}$. It was calculated using the
following equation [21]

$R^{2}=1-\frac{\frac{1}{n}\sum_{k=1}^{n}(T_{kp}-O_{kp})^{2}}{\frac{1}{n-1}\sum_{k=1}^{n}(T_{kp}-\overline{T_{kp}})^{2}}.$
(16)

Where $\overline{T_{kp}}$ is the average desired target.

In this study, backpropagation and resilient propagation were used, each
consisting of two and one hidden layers. For one hidden layer, the number of
neurons was determined using the formula proposed by Shibata and Ikeda in 2009
[30], specifically $N_{h}=\sqrt{N_{i}\cdot N_{o}}$, where $N_{h}$, $N_{i}$,
and $N_{o}$ represents hidden, input and output neurons, respectively. In both
backpropagation and resilient propagation, the number of neurons used do not
exceed one hidden layer. Based on Shibata and Ikeda [30] formula, the number
of neurons in one hidden layer was obtained, specifically
$N_{h}=\sqrt{N_{i}\cdot N_{o}}=\sqrt{47\cdot 4}=13.71$. However, in this
study, it was rounded up to 15 neurons. Some experiments were conducted to
evaluate the identification accuracy, and whether using one hidden layer 15
neurons might lead to a better accuracy of identification than two hidden
layers. However, the number of neurons varied, setting less than 15 neurons.
For two hidden layers, experiments were conducted with the number of
consecutive neurons as follows; 3-5 (8), 4-6 (10), 5-7 (12) and 6-8 (14). The
number of neurons in the hidden layer never exceed 15 neurons.

###  3.1 Backpropagation (B-Prop) with Two Hidden Layers

In this section, backpropagation learning algorithm with two hidden layers was
used. The number of neurons in the hidden layer varied with not more than 15
neurons. There were four variations of network architecture, including
47-3-5-4, 47-4-6-4, 47-5-7-4 and 47-6-8-4. The input layer consists of 47
neurons based on the number of metabolites. The output layer consists of 4
neurons according to the number of clove buds origins.

Table 1: Backpropagation with two hidden layers Network | MSE |  Accuracy (%) | $R^{2}$
---|---|---|---
Architecture | Training | Testing | Training | Testing | Training | Testing
47-3-5-4 | 0.10346 | 0.11357 | 76.98 | 73.68 | 0.81 | 0.76
47-4-6-4 | 0.13084 | 0.13547 | 62 | 57.54 | 0.64 | 0.61
47-5-7-4 | 0.14889 | 0.15884 | 49.73 | 41.4 | 0.51 | 0.42
47-6-8-4 | 0.15388 | 0.15874 | 50.04 | 42.46 | 0.48 | 0.43

Table 1 shows the network architecture 47-3-5-4 gives the highest value for
identification accuracy and coefficient of determination in training and
testing data sets. Similar to the $MSE$, this network architecture provides
the smallest amount of both training and the testing data sets. From Table 1,
increasing the number of neurons in the backpropagation with two hidden layers
decreases network performance. This is in line with Shafi et al. in 2006 [31],
which stated that increasing the number of neurons in the hidden layer only
heightened the complexity of the network. Still, it does not increase the
accuracy of pattern recognition.

###  3.2 Backpropagation (B-Prop) with One Hidden Layer

The backpropagation learning algorithm with one hidden layer was implemented
to evaluate its result in case of a comparison using two hidden layers. The
results obtained are shown in Table 2.

Table 2: Backpropagation with one hidden layer Network | MSE |  Accuracy (%) | $R^{2}$
---|---|---|---
Architecture | Training | Testing | Training | Testing | Training | Testing
47-15-4 | 0.0721 | 0.0773 | 99.91 | 99.47 | 0.99 | 0.98

Table 2 shows that network architecture 47-15-4 identifies the clove buds
origin effectively. The identification accuracy percentage is 99.91% and
99,47% for training and testing data sets, respectively. Besides, the $MSE$
value is also smaller compared to the two hidden layers.

For the backpropagation algorithm, the results show one hidden layer is better
than two. This is in line with Villiers and Barnard [32], which stated that
network architecture with one hidden layer is on average better than two
hidden layers. They concluded that two hidden layers are more difficult to
train. Additionally, they also established that this behaviour is caused by a
local minimum problem. The networks with two hidden layers are more prone to
the local minimum problem during training.

###  3.3 Resilient Propagation (R-Prop) with Two Hidden Layers

Resilient propagation learning algorithm contains some parameters, including
the upper and lower limits, as well as the decrease and increase factors. In
this study, the range of update values is limited to the upper limit
($\Delta_{max}$) = 50, the lower limit ($\Delta_{min}$) = $10^{-6}$, and the
decrease and increase factors ($\eta^{-}$) = 0.5 and ($\eta^{+}$) = 1.2,
respectively. The reason for choosing these values is shown in [28]. Similar
to Section 3.1, the resilient propagation learning algorithm is applied to the
network architecture with two hidden layers. The number of neurons vary but do
not exceed 15 neurons. In this section, there are four variations of network
architecture, including 47-3-5-4, 47-4-6-4, 47-5-7-4 and 47-6-8-4.

Table 3: Resilient propagation with two hidden layers Network | MSE |  Accuracy (%) | $R^{2}$
---|---|---|---
Architecture | Training | Testing | Training | Testing | Training | Testing
47-3-5-4 | 0.07209 | 0.08111 | 99.73 | 97.37 | 0.98 | 0.95
47-4-6-4 | 0.07162 | 0.08316 | 99.69 | 96.49 | 0.99 | 0.94
47-5-7-4 | 0.07160 | 0.07961 | 99.96 | 97.89 | 0.99 | 0.96
47-6-8-4 | 0.07160 | 0.07978 | 99.78 | 97.72 | 0.99 | 0.96

The results in Table 3 show the network architecture 47-5-7-4 gives the
highest identification accuracy of clove buds origin. The percentage of
identification accuracy is 99.96% and 97.89% for training and testing data
sets, respectively.

###  3.4 Resilient Propagation (R-Prop) with One Hidden Layer

In this section, the resilient propagation learning algorithm is implemented
with one hidden layer. Similar to section 3.2, the number of neurons in the
hidden layer is 15 neurons, and have the network architecture 47-15-4.

Table 4: Resilient propagation with one hidden layer Network | MSE |  Accuracy (%) | $R^{2}$
---|---|---|---
Architecture | Training | Testing | Training | Testing | Training | Testing
47-15-4 | 0.07158 | 0.07932 | 99.86 | 94.74 | 0.99 | 0.92

Table 4 shows the network architecture 47-15-4 identifies the origin of clove
buds with identification accuracy of 99.86% and 94.74% on training and testing
data sets, respectively.

The network architecture of the resilient propagation algorithm, both two
hidden layers and one hidden layer, provides identification results with very
high accuracy. However, the network architecture with two hidden layers is
slightly lower accuracy.

Tables 3 and 4 show the two-layered resilient propagation with deficient
neurons performs better than the single-layer having more neurons. This is in
line with Santra et al. [24], established that the performance of two hidden
layers with 8-10 (18) neurons is better than one hidden layer with 62 neurons.

The summary of the best identification accuracy and determination coefficient
are shown in Figures 1, 2, 3 and 4, respectively. For each network
architecture, the smallest MSE in training and testing data sets are shown in
Figures 5 and 6, respectively.

Figure 1: Identification accuracy percentage of training data sets. Figure 2:
Identification accuracy percentage of testing data sets. Figure 3:
Determination coefficient of training data sets. Figure 4: Determination
coefficient of testing data sets. Figure 5: $MSE$ of training data sets.
Figure 6: $MSE$ of testing data sets.

The results of identification from the origins of clove buds have been
obtained. In small data set categories, backpropagation with one hidden layer
provides accurate identification in the training and testing data sets. It
accurately identifies the origins of clove buds obtained using the resilient
propagation algorithm with two hidden layers.

The neural networks model obtained in this paper can be a reference from a
scientific perspective. For instance, it can be used in future studies to
identify the origin of various plantation commodities with small metabolites
data sets. At the moment, the most appropriate way of determining the origin
of a plantation commodity is qualitative, relying on the services of flavorist
to evaluate flavor and taste. This is because each commodity has a specific
flavor and taste based on the origin of its region. Furthermore, the different
origins of clove buds data sets have not been reported in the literature and
thus no direct comparison can be presented in this paper.

##  4 Conclusions

This paper demonstrated the potential and ability of a neural network approach
with backpropagation and resilient propagation learning algorithms. It was
meant to identify the clove buds origin based on metabolites composition. The
work was divided into two parts, the first one being identification of the
clove buds origin using the backpropagation learning algorithm. Two network
architectures were constructed, each containing one hidden and two hidden
layers. The results showed the use of one hidden layer gives clove buds origin
identification accurately, specifically 99.91% and 99.47% in training and
testing data sets. The second step involved the identification of the clove
buds origin using resilient propagation learning algorithm. Two network
architectures were constructed, each containing one hidden and two hidden
layers. The results showed the use of two hidden layers gives accurate clove
buds origin identification, including 99.96% and 97.89% in training and
testing data sets. From these results, it was concluded that for
identification of a small of metabolites data sets from a plantation
commodity, the backpropagation algorithm with one hidden layer and the
resilient propagation algorithm with two hidden layers should be used. This
paper also confirmed the contribution of artificial neural networks to pattern
recognition of metabolites data sets obtained by metabolic profiling
technique.

## References

  * [1] L. Broto.  Derivatisasi minyak cengkeh, dalam cengkeh: Sejarah, budidaya dan industri.  2014 (in Indonesian).
  * [2] Coffeeland 2010.  Jenis kopi arabika terbaik dari berbagai daerah di indonesia (in indonesian).
  * [3] M T A P Kresnowati, R Purwadi, M Zunita, R Sudarman, and A O Putri.  Metabolite profiling of four origins indonesian clove buds using multivariate analysis.  Report Research Collaboration PT. HM Sampoerna Tbk. and Institut Teknologi Bandung (confidential report), 2018.
  * [4] Joachim Kopka, Alisdair Fernie, Wolfram Weckwerth, Yves Gibon, and Mark Stitt.  Metabolite profiling in plant biology: platforms and destinations.  Genome biology, 5(6):109, 2004.
  * [5] Sastia Prama Putri and Eiichiro Fukusaki.  Mass spectrometry-based metabolomics: a practical guide.  CRC Press, 2016.
  * [6] Desiré Luc Massart, BGM Vandeginste, SN Deming, YKAUFMAN Michotte, and L Kaufman.  Chemometrics: a textbook.  1988\.
  * [7] T Cornelius.  Leondes: Image processing and pattern recognition, 1998.
  * [8] Benbakreti Samir and Aoued Boukelif.  New approach for online arabic manuscript recognition by deep belief network.  Acta Polytechnica, 58(5), 2018.
  * [9] A Noriega Ponce, A Aguado Behar, A Ordaz Hernández, and V Rauch Sitar.  Neural networks for self-tuning control systems.  Acta Polytechnica, 44(1), 2004.
  * [10] M Chvalina.  Demand modelling in telecommunications comparison of standard statistical methods and approaches based upon artificial intelligence methods including neural networks.  Acta Polytechnica, 49(2):48–52, 2009.
  * [11] Ivo Bukovsky and Michal Kolovratnik.  A neural network model for predicting nox at the melnik 1 coal-powder power plant.  Acta Polytechnica, 52(3):17–22, 2012.
  * [12] P Kutilek and S Viteckova.  Prediction of lower extremity movement by cyclograms.  Acta Polytechnica, 52(1), 2012.
  * [13] D Novák and D Lehkỳ.  Neural network based identification of material model parameters to capture experimental load-deflection curve.  Acta Polytechnica, 44(5-6), 2004.
  * [14] Rustam, Agus Y Gunawan, and Made Tri Ari P Kresnowati.  The hard c-means algorithm for clustering indonesian plantation commodity based on metabolites composition.  In Journal of Physics: Conference Series, volume 1315, page 012085\. IOP Publishing, 2019.
  * [15] Tanja Beltramo, Michael Klocke, and Bernd Hitzmann.  Prediction of the biogas production using ga and aco input features selection method for ann model.  Information Processing in Agriculture, 2019.
  * [16] Laurene Fausett.  Fundamentals of neural networks: architectures, algorithms, and applications.  Prentice-Hall, Inc., 1994.
  * [17] Igor Aizenberg and Claudio Moraga.  Multilayer feedforward neural network based on multi-valued neurons (mlmvn) and a backpropagation learning algorithm.  Soft Computing, 11(2):169–183, 2007.
  * [18] Erik M Johansson, Farid U Dowla, and Dennis M Goodman.  Backpropagation learning for multilayer feed-forward neural networks using the conjugate gradient method.  International Journal of Neural Systems, 2(04):291–301, 1991.
  * [19] T Todd Pleune and Omesh K Chopra.  Using artificial neural networks to predict the fatigue life of carbon and low-alloy steels.  Nuclear Engineering and Design, 197(1-2):1–12, 2000.
  * [20] Klaus LE Kaiser, Stefan P Niculescu, and Gerrit Schüürmann.  Feed forward backpropagation neural networks and their use in predicting the acute toxicity of chemicals to the fathead minnow.  Water Quality Research Journal, 32(3):637–658, 1997.
  * [21] Eddy El Tabach, Leyla Adishirinli, Nicolas Gascoin, and Guillaume Fau.  Prediction of transient chemistry effect during fuel pyrolysis on the pressure drop through porous material using artificial neural networks.  Journal of analytical and applied pyrolysis, 115:143–148, 2015\.
  * [22] RA Chayjan and M ESNA-ASHARI.  Isosteric heat and entropy modeling of pistachio cultivars using neural network approach.  Journal of Food Processing and Preservation, 35(4):524–532, 2011\.
  * [23] Aristoklis D Anastasiadis, George D Magoulas, and Michael N Vrahatis.  New globally convergent training scheme based on the resilient propagation algorithm.  Neurocomputing, 64:253–270, 2005.
  * [24] Apurba Kumar Santra, Niladri Chakraborty, and Swarnendu Sen.  Prediction of heat transfer due to presence of copper–water nanofluid using resilient-propagation neural network.  International Journal of Thermal Sciences, 48(7):1311–1318, 2009\.
  * [25] Lalit M Patnaik and K Rajan.  Target detection through image processing and resilient propagation algorithms.  Neurocomputing, 35(1-4):123–135, 2000.
  * [26] Dominik Fisch and Bernhard Sick.  Training of radial basis function classifiers with resilient propagation and variational bayesian inference.  In 2009 International Joint Conference on Neural Networks, pages 838–847. IEEE, 2009.
  * [27] MD Shiblee, B Chandra, and Prem Kumar Kalra.  Learning of geometric mean neuron model using resilient propagation algorithm.  Expert Systems with Applications, 37(12):7449–7455, 2010.
  * [28] Martin Riedmiller and Heinrich Braun.  A direct adaptive method for faster backpropagation learning: The rprop algorithm.  In Proceedings of the IEEE international conference on neural networks, volume 1993, pages 586–591. San Francisco, 1993.
  * [29] Phiroz Bhagat.  Pattern recognition in industry.  Elsevier, 2005.
  * [30] Katsunari Shibata and Yusuke Ikeda.  Effect of number of hidden neurons on learning in large-scale layered neural networks.  In 2009 ICCAS-SICE, pages 5008–5013. IEEE, 2009.
  * [31] Imran Shafi, Jamil Ahmad, Syed Ismail Shah, and Faisal M Kashif.  Impact of varying neurons and hidden layers in neural network architecture for a time frequency application.  In 2006 IEEE International Multitopic Conference, pages 188–193. IEEE, 2006.
  * [32] Jacques De Villiers and Etienne Barnard.  Backpropagation neural nets with one and two hidden layers.  IEEE transactions on neural networks, 4(1):136–141, 1993.