# Design of bistable soft deployable structures via a Kirigami-inspired planar
fabrication approach

Mrunmayi Mungekar1 Leixin Ma1 L.M<EMAIL_ADDRESS>Wenzhong Yan1 Vishal
Kackar1 Shyan Shokrazadeh1 M. K. Jawed1 M.K.J<EMAIL_ADDRESS>1University of California, Los Angeles, Department of Mechanical and Aerospace
Engineering, 420 Westwood Plaza, Los Angeles, CA, USA 90024

###### Abstract

Fully soft bistable mechanisms have shown extensive applications ranging from
soft robotics, wearable devices, and medical tools, to energy harvesting.
However, the lack of design and fabrication methods that are easy and
potentially scalable limits their further adoption into mainstream
applications. Here a top-down planar approach is presented by introducing
Kirigami-inspired engineering combined with a pre-stretching process. Using
this method, Kirigami-Pre-stretched Substrate-Kirigami trilayered precursors
are created in a planar manner; upon release, the strain mismatch—due to the
pre-stretching of substrate—between layers would induce an out-of-plane
buckling to achieve targeted three dimensional (3D) bistable structures. By
combining experimental characterization, analytical modeling, and finite
element simulation, the effect of the pattern size of Kirigami layers and pre-
stretching on the geometry and stability of resulting 3D composites is
explored. In addition, methods to realize soft bistable structures with
arbitrary shapes and soft composites with multistable configurations are
investigated, which could encourage further applications. Our method is
demonstrated by using bistable soft Kirigami composites to construct two soft
machines: (i) a bistable soft gripper that can gently grasp delicate objects
with different shapes and sizes and (ii) a flytrap-inspired robot that can
autonomously detect and capture objects.

###### pacs:

Valid PACS appear here

††preprint: APS/123-QED

##  I Introduction

The rapid development of soft robotics has recently attracted increasing
attention from biology, chemistry, materials science, and engineering [1].
Fully soft machines and robots have shown the capability of outperforming
conventional rigid counterparts in terms of adaptability, robustness, and
safety [2]. Recently, bistable mechanisms—possess two stable equilibrium
states—have been introduced into soft robotics for achieving high-performance
and multi-functionalities [3]. They have demonstrated various innovative
applications in fast grasping [4, 5, 6], shape reconfiguration [7, 8],
information storage [9, 10], high-speed locomotion on ground [11, 12, 13] and
water [14, 15], adaptive sensing [16], and mechanical oscillation [17, 18],
computation [19, 20], and feedback controls [21] towards achieving autonomy
for fully soft machines. Fabrication is non-trivial for fully soft bistable
mechanisms, including both pre-shaped and pre-buckled ones [22]. Pre-shaped
bistable mechanisms are either accomplished with vacuumed-involved casting
procedures with customized molds [23, 24] or with free-form fabrication
techniques (i.e., three dimensional printing) [25] with limited scalability
and long fabrication times [26]. Likewise, pre-buckled ones often require
constrained boundary to precisely impose buckling load [27, 28], which could
increase the fabrication difficulty and restrain their applications.
Therefore, it is desired to introduce a method for fabrication and programming
of fully soft bistable mechanisms that are easy to generate and potentially
scalable, which could encourage their further adoption into mainstream
applications.

Recently, pre-stretching method on generating three dimensional (3D)
structures and mechanisms from planer bilayered precursors (structures before
3D deployment) has drawn increasing attentions thanks to their scalability and
ease for fabrication [29, 30, 31]. The planer precursors often consist of a
pre-stretched substrate with a bonded top layer; upon released, the substrate
imposes compressive load on the top layer to force the bilayered precursor
into 3D morphology [32]. However, to introduce bistability or even
multistability, a spatial variation of thickness and selected bonding pattern
for the top layer are necessary [33]. These two technologies could enable the
creation of more delicate 3D configurations while sacrificing the ease of
fabrication.

A promising alternative method is to bring in Kirigami-inspired engineering
onto the top layer (with uniform thickness); the designated pattern on the top
layer could potentially guide the resulting composite to deform into a pre-
programmed geometry without requiring neither a spatial variation of thickness
of the top layer nor specified bonding pattern [34, 35, 36, 37]. This
Kirigami-inspired pre-stretching method has enabled various applications,
including deployable 3D shapes [34], self-assembly structures [38], etc. [39].
Yet, the capability of introducing bistability within this design paradigm has
not been demonstrated. Specifically, a scalable and simple manufacturing
method that enables the usage of the uniform material and bonding could create
a pathway for broad adoption of fully soft bistable mechanisms into
intelligent soft machines and robots.

Here we propose an easy and scalable process to create fully soft bistable
mechanisms from 2D preprogrammed precursors. This approach introduces the
bistability through bonding one central pre-stretched substrate with two
identical Kirigami top and bottom layers. Once released, the trilayered
composite deforms into a 3D free-buckling configuration with two stable
equilibrium, as schematically shown in Figure 1A and B. Our method allows us
to only use commercially available soft elastomer sheet materials, which could
vastly simplify the fabrication processes. We combine physical experiment,
finite element simulation, and composite shell theory to explore and predict
the transition from 2D to 3D shapes and the properties of the resulting
bistable mechanisms. We show methods to create soft bistable structures with
arbitrary shapes and sizes by combining optimal design results from machine
learning algorithm with scaling analysis. In addition, we explore the
multistability of proposed trilayered composites through experiment and
simulation. We demonstrate this generic design and fabrication method by
creating (i) an universal, nondestructive soft gripper that allows stable
grasping of delicate objects such as a strawberry or a butter packet; and (ii)
a flytrap-inspired robot that can autonomously sense and swiftly capture
objects (see Figure 1C and D, and Movie S1). This work opens a new avenue to
generate fully soft bistable mechanism with potential applications in wearable
devices, soft robotics, and multifunctional medical devices.

Figure 1: Overview of the bistable soft structure design. (A). The combination
of two Kirigami layers and strain mismatch creates 3D soft structures that are
fully soft and bistable. (B) The force-displacement curve demonstrates that
the trilayer soft structure can snap-through from one stable configuration to
the other. (C) The bistable structures can be applied to create a flytrap-
inspired robot allowed for autonomous sensing and fast actuation to capture
objects. (D) A bistable pyramid can function as a gripper to grasp delicate
objects with different size and shape.

##  II Results and Discussion

###  II.1 Design concept

Figure 1 presents our concept of designing bistable soft composites. We start
with a substrate layer and two identical outer layers with the same Kirigami
design cutouts (Figure 1A). The Kirigami layers in this example resemble a
lotus shape. First, a biaxial stretch is induced on the substrate layer. Then,
the two unstretched Kirigami layers are aligned and bonded at the top and the
bottom of the pre-stretched substrate layer, generating symmetry to the
composite about the substrate (Figure 1). The material removal in the Kirigami
layers can help the laminated trilayered structure to achieve targeted global
shapes with smooth transition instead of wrinkling in the local regions [40].
Once released, the strain mismatch between the three layers induces out-of-
plane buckling since bending is less energetically expensive than compression
for thin shells [41].

###  II.2 Desktop-Scale Physical Experiments

Figure 2: Experimental setup. (A1-A4) CAD model of the experimental setup (B1)
Schematic representation of the system consist of two-knob stages (1) on 250
mm tracks (2) (B2) Snapshot of the system when the substrate layer (3) is
stretched (B3) Attach one Kirigami layer (4) below the substrate layer, and
the other one (5) on top of the substrate layer (B4) Release the pre-stretch
by cutting along the outline of the circular substrate.(C) Side view of the
trilayer composite structure, and the free buckling shape of the trilayer
composite structure (D) Variation of normalzied height as a function of the
pre-stretch ($L=60$ mm)

Figure 2 presents the experimental setup for fabricating trilayered bistable
structures. The experimental setup consists of four linear translation stages
(250 mm travel, Thorlabs, Inc.). The substrate and the Kirigami layers are
made of hyper-elastic materials. The material properties are listed in Table
LABEL:table:matparam. We fit the experimentally measured stress-strain curve
using the Mooney Rivlin material model. The material constants for the
Kirigami layer are $C_{1}^{kg}$ and $C_{2}^{kg}$, while the constants for the
substrate layer are denoted as $C_{1}^{s}$ and $C_{2}^{s}$. An arbitrarily
large substrate layer is fixed with its four corners attached onto the four
stages, respectively (Figure 2A1 and B1). Then, to impose the biaxial stretch
upon the substrate, we induce the substrate layer with the same amount of
stretch in both directions (Figure 2A2 and B2). Once stretching is complete,
two Kirigami layers are attached on the top and bottom of the substrate layer
directly since both sides of layers already have strong adhesive agent (Figure
2A3 and B3). The excess substrate is cut away along a defined bonding box,
which is designed according to the desired resulting shape (Figure 2A4 and
B4). Once released, this planar composite structure then morphs to either one
of the two possible stable configurations arising due to the strain mismatch
within layers.

For our experimental study, we chose cross-shaped Kirigami layers and square-
shaped substrate layers. With the appropriate value of pre-stretch $\lambda$,
these geometric designs of the layers provide a deformed shape that looks like
a pyramid. Figure 2C displays one such soft pyramid bistable structure. The
length of the 2D precursor is denoted as $L$ while the height of the buckled
3D pyramid is $H$ as shown in the insert of Figure 2C. We define the
configuration it immediately converged to after removing the boundary
condition as the “positive” configuration, while the inverted one as the
“negative” configuration. Before the measurements were performed, both
“positive” and “negative” configurations were given a sufficient period of
time to settle down. As the pre-stretch increases, the normalized height (=
$H/L$) of the pyramid increases in a nonlinear manner, as shown in Figure 2D.
The normalized height for the “positive” and “negative” configurations were
close to each other for this size of composite structure with $L=60$ mm.

###  II.3 Finite element-based numerical simulations

The finite element software, Abaqus [42], was used to model the nonlinear
large deformation of the hyper-elastic composite structures. In the
simulations, we used the same pyramid shape and simulated the hyper-elastic
materials of the substrate and the two Kirigami layers using Mooney Rivlin
material models. Solid elements (C3D6) were used to model the layers of the
composite structure. Each layer of the composite was modelled with multiple
sub-layers of elements to accurately predict the deformation of the structure
along its thickness. Each sub-layer was meshed with two sets of 4521 nodes,
with the substrate and the kirigami having 2198 and 6046-node prismatic
elements per sub-layer, respectively. The symmetry of the Kirigami was taken
into account for the mesh. Furthermore, the elements at the interface of the
material layers shared nodes constraining their deformation in a way similar
to the layers being bonded together. The center of the composite structure was
set fixed. The pre-stretch was induced by describing an initial stress
condition on the substrate layer. This initial stress was calculated as the
Cauchy stress ($\sigma$) arising in a Mooney-Rivlin hyperelastic material
(Parameters: $C\textsubscript{1}$, $C\textsubscript{2})$ due to a given
equibiaxial stretch ($\lambda$).

$\displaystyle\sigma=2C_{1}(\lambda^{2}-\frac{1}{\lambda^{4}})-2C_{2}(\frac{1}{\lambda^{2}}-\lambda^{4})$
(1)

A nonlinear quasi-static deformation process was simulated with a very small
but non-zero distributed force on the structure to force it into one of the
two possible stable configurations, which we termed as the “positive”
configuration.

One numerical simulation took about 1-3 minutes wall clock time on a desktop
computer (Ryzen 2950wx CPU @ 2.4 GHz). To understand the variation of the
final shape with pre-stretch, the simulation was run for multiple values of
pre-stretches. The height of the structure was studied by procuring the
maximum and minimum values of the out-of-plane nodal coordinates. Further
details on this study have been described in Section II.5.

We also study the other stable configuration than the one the simulation
initially provides. This is simulated by modelling a force-based actuation on
the obtained configuration. Force is applied on four symmetrically placed
nodes on the arms of the cross-shaped Kirigami layer, pushing the structure
into the other stable configuration. This actuation provides us with details
about the “negative” configuration of the structure as well as brings to light
the multiple stable modes that a structure could have as further described in
Section II.6.

###  II.4 Analytical derivation

Figure 3: (A) Cross-sectional view of the trilayer and bilayer composite
structure (B) Sketch of the side view for a pyramid with height $H$. (C)
Sketch of the side view for a spherical cap with height $H$.

Figure 3A shows the cross-sectional views of the soft Kirigami composite
structure before bulking. The thickness of the substrate and Kirigami layers
are denoted as $t_{s}$ and $t_{k}$, respectively. We assume that after the
free buckling, the initial stretching energy of the substrate layer is
transformed into the bending energy of the whole composite structure. This is
because that bending energy is less energetically expensive than compression
for thin shells [41]. Considering the hyperelastic property of the system, the
stretching-induced energy is,

$\displaystyle\textbf{E}_{s}$
$\displaystyle=[C_{1}(2\lambda^{2}+1/\lambda^{4}-3)+C_{2}(\lambda^{4}+2/\lambda^{2}-3)]At_{s}$
(2)

where $\lambda=L_{f}/L=1+\epsilon$ is the pre-stretch, which is the ratio of
the stretched substrate size $L_{f}$ over the unstretched size $L$. $A$ is the
area of the unstretched substrate. For the square-like substrate,
$A=(L/\sqrt{2}\lambda)^{2}$.

For small strain, we can approximate the stretching-induced energy for the
substrate layer as,

$\displaystyle\textbf{E}_{s}=\frac{1}{2}AC_{s}\epsilon^{2}$ (3)

where $C_{s}=E_{s}t_{s}/(1-\nu_{s}^{2})$. $E_{s}$ and $\nu$ are the Young’s
modulus and Poisson’s ratio of the substrate layer, respectively.

Meanwhile, the bending energy can be calculated as,

$\displaystyle\textbf{E}_{b}$
$\displaystyle=\frac{1}{2}AD_{\textrm{eq}}\kappa^{2}$ (4)

where $D_{\textrm{eq}}$ is the equivalent bending stiffness of the trilayer,
which is derived in the supplementary material. From the observation of the
deformed 3D shapes, we further assume that the deformed shape is close to a
pyramid shape, with the side view sketched in Figure 3B. For the pyramid like
shape, the curvature is related by the angle $\phi$, the half-length of the
cross shape $L/2$, and the height of the pyramid $H$ via the equations as
follows,

$\displaystyle\kappa=4\tan(\phi/2)/L$
$\displaystyle=\sqrt{\frac{C_{s}}{D_{\textrm{eq}}}}{\frac{\epsilon}{1+{\epsilon}}}$
(5)

$\displaystyle H=L\sin(\phi/2))/2$ (6)

Hence, equalizing the bending energy and stretching energy, and combining
equation 5 and 6 gives,

$\displaystyle\frac{H}{L}=\sin(\phi/2)=\sin[\arctan{(\kappa L/4)}]/2$ (7)

Since $\arctan{(\kappa L/4)}$ function is bounded between 0 and $\pi/2$, the
normalized height, $H/L$, is bounded between 0 and 0.5. This helps explain why
in both simulations and experiments, the normalized heights plateau at very
large amount of pre-stretches in in Figure 4. It can be found that the
simplified analytical model agrees well with the experiments and simulations,
which further verifies the concept of shape-forming trilayer structures.

For other 3D shapes close to spherical caps in Figure 3 C, the relationship
between normalized height and curvature are found to be presented in Equation
8. The detailed derivations are in the supplementary material.

$\displaystyle\frac{H}{L}=\frac{\kappa L}{8}$ (8)

###  II.5 Effect of size and pre-stretch on free buckling shapes

In this section, we focus on the cross-shaped Kirigami layers with square-
shaped substrate as shown in Figure 2. The thickness of the material is
denoted as $t$ (including $t_{s}$ and $t_{k}$ for the substrate and Kirigami
layer, respectively), the length and width of the kirigami are denoted as $L$
and $w$, respectively.

For a fixed size of kirigami composite structure, we first investigate the
effect of prestretch on the buckling shapes. A few examples of Kirigami
composites with different amount of pre-stretches are presented in Figure 4 A
and B. The geometry in simulations agrees well with the results of
experiments. The height of the composite structure increases with the amount
of pre-stretch, as shown in Figure 4C to E. This is because with the increase
in strain energy in Figure 4F, the converted shell bending energy becomes
larger. There exists a certain threshold, below which the composite structure
hardly buckles. Despite the assumptions introduced in the analytical model,
the experiment, simulation and analytical model exhibit similar trend. As pre-
stretch increases, the maximum heights of these pyramid-like 3D shapes are
bounded by a certain maximum.

We further increase the length $L$, but fix the length/width ratio, $L/w$, and
the thickness of the composite structure $t$. This means that the ratio of
length/thickness, $L/t$, is increased. Figure 4C, D and E present the
variation of normalized height at different $L/t$. At the same level of pre-
stretch, the maximum normalized height for the pyramid tends to be larger for
thinner composite structures (with larger $L/t$). Such a behavior can be
understood from the scaling relationship in Equation 7.

Figure 4:  (A) Comparison of the free-buckling shapes in (A1-A4) simulation
and (B1-B4) experiment for the cross-shaped Kirigami composite structure (L=40
mm) at different amount of pre-stretch $\lambda=1.2,1.4,1.6,1.8,2.0$. The
colorbars correspond to the distribution of stress in the deformed trilayer
structure (C-E) The variation of the normalized height of the free-buckling
shape as a function of the pre-stretch. Three examples showing different
length/thickness ratio with diameter/width ratio fixed. (C) $L/t_{s}=4000$.
(D) $L/t_{s}=6000$. (E) $L/t_{s}=8000$. (F) Variation of the energy landscape
at different pre-stretch. Below a certain threshold, the composite structure
hardly buckles.  Figure 5:  (A) Sketches of free-buckling mode shape 1 and
mode shape 2 in the “positive” and “negative” configurations. (B) The free
buckling shapes for the for the cross-shaped Kirigami composite structure
($L=80$ mm) at different amount of pre-stretch $\lambda=1.2,1.4,1.6,1.8,2.0$.
The Mode 1 and Mode 2 are also predicted in simulations. (C) The simulated
variation of normalized height of the two modes in the “positive” and
“negative” configurations as a function of pre-stretch $\lambda$.  Figure 6:
Comparison of the free buckling shape for bilayer and trilayer soft Kirigami
composite. The Kirigami patterns is fixed, and the structures can create two
target shapes (A) Unilobe structure. (B) Bilobe structure. First column:
Kirigami patterns in red and circular substrate in blue. Second column:
Bilayer structure with specific combination of pre-stretch and initial radius.
Third column: Trilayer structure with the same combination of pre-stretch and
initial radius as the bilayer counterpart. Fourth column: Trilayer structure
with adjusted pre-stretch and radius.

###  II.6 Multistability

As $L$ increases to $80$ mm, we find there could be multiple free-buckling
mode shapes in the positive and negative configurations, as shown in Figure 5
A and B. This behavior is not observed for thicker composite structures with
larger $L/t$ in Figure 4. For the kirigami composite of $L=80$ mm, a
systematic variation finds that, at a small pre-stretch, the free buckling
shape is dominated by the mode 1 defined in Figure 5A. As the amount of pre-
stretch increases, the mode 2 type emerges. The four edges of the cross-
sections start to buckle locally. In the two modes, the cross shapes buckle in
different directions. This leads to the difference in the height of the two
modes presented in Figure 5C. Considering the symmetry of the trilayered
structure, the soft structures in this case have four stable configurations.
Such a multistable behavior further functionalizes the structure to possess
more mechanical properties. This also opens the door of conducting systematic
analytical research to understand the mechanism behind the multi-stability
phenomena in the future.

###  II.7 Creating the trilayer bistable structures that achieve target 3D
shape

Based on the same design concept of combining Kirigami structures and strain
mismatch, Miha et al. and Ma et al. [36] for the first time demonstrated that
the planar bilayered composite can be deformed to 3D soft structures. Ma et
al. [37] further developed a machine learning-aided approach to explore the
optimal Kirigami cuts, pre-stretch and Kirigami size that achieves targeted 3D
shapes, such as peanuts and flowers. The previous research enables rapid
prototyping of soft Kirigami structures through a Kirigami bilayered composite
structure. If we continue to add more Kirigami layers in the system, we can
introduce bistability for these soft kirigami structures, which can be used
for wider applications, such as biomimetics [34] and soft actuators [23].

Even though the proposed machine learning approach by Ma et al. [37] reduces
the number of searches from millions of designs to order of hundreds, it still
takes about 6 hours of computational time on a desktop computer (Ryzen 2950wx
CPU @ 2.4 GHz). In this section, we want to test whether we can achieve
similar targeted 3D bistable structures just by modifiying the size and the
amount of pre-stretch while keeping the optimized Kirigami patterns the same
as the bilayered counterparts. To achieve this goal, we make use of the
scaling analysis derived from the energy conservation.

The 3D shapes we focus on in this section are the unilobe shape and the bilobe
shape (Figure 6). For these 3D shapes, we can assume the curvature is
approximately the same everywhere. In this case, the relationship between the
curvature and height is derived in the supplementary information. Then,
equalizing the bending energy and stretching energy, the normalized height
$H/L$ can be approximated by

$\displaystyle\frac{H}{L}=\frac{L}{8}\sqrt{\frac{C_{s}}{D_{\textrm{eq}}}}{\frac{\epsilon}{1+{\epsilon}}}$
(9)

For the structure of a certain radius $R$, the bending stiffness
${D_{\textrm{eq}}}$ for the trilayer is about 3 times of the bilayer
counterpart. Figure 6 shows two examples where the Kirigami patterns in the
first column can be used to create a target unilobe and a bilobe bilayer
structures in the second column [36, 37]. The material properties for the
substrate and Kirigami layers are shown in Table LABEL:table:matparam. The
material constants for the Kirigami layer leading to unilobe structures are
$C_{1}^{kg}$ and $C_{2}^{kg}$. While the constants for kirigami that creates
the bilobe shape are $C_{1}^{kw}$ and $C_{2}^{kw}$. If we apply the same
amount of pre-stretch to the trilayer structure of the same size as the
bilayer counterpart, then the free-buckling shape is still flat in the plane.
This is due to the increased stiffness in the system. To create the targeted
3D shapes of similar height ratio $H/L$, the pre-stretch and size need to be
adjusted according to Equation 9. The last column in Figure 6 shows some
examples when the pre-stretch varies near the value for the bilayer
counterpart but the radius of the structure is increased, we can still
fabricate unilobe and bilobe shapes of similar $H/L$ using these trilayer
composite structures.

There also exists a limiting condition when the scaling relationship can be
further simplified. When the thickness of the substrate is very thin, and the
bending stiffness of the substrate is negligible, the equivalent bending
stiffness of the trilayer is about 8 times of the bilayer counterpart with the
same size. Hence the ${\frac{\epsilon}{1+{\epsilon}}}$ or size $L$ needs to be
increased to approximately $2\sqrt{2}$ of the bilayer structure.

###  II.8 Applications of bistable, soft Kirigami composites

Figure 7: Demonstrating bistable soft composites with two applications. (A)
The mechanism of a soft griper composed of a bistable dome actuated by an
extending shape memory alloy(SMA) spring. (B) The detailed structure of the
gripper with labels. (C) The gripper has the ability to gently grasp delicate
objects of different shapes and sizes. (D) Venus flytrap can detect preys and
close its leaves quickly to capture preys. (E) This strategy can be mimicked
by mechanically embedding sensing and actuation into structures by using a
bistable dome. (F) Snapshots show the autonomous object detection and quick
closing of leaves without requiring external sensing, control, and actuation.
To help overcome the energy barrier of the bistable dome, we attached a
permanent magnetic bead on the bottom of the object while placing another one
underneath the platform.

To show the potential applications, we used bistable soft Kirigami composites
to realize two soft machines: (i) A bistable soft gripper that can grasp
delicate objects with different shape and size (see Figure 7A-C); (ii) A
flytrap-inspired robot that can autonomously and passively sense and capture
objects (see Figure 7D-F).

The soft gripper is mainly composed of a bistable dome ($4.2$ gram, with
cross-shaped Kirigami layers), an extending shape memory alloy (SMA) spring,
and a mechanical pusher (see Figure 7A). The center of the bistable dome is
attached on the end of a rail. One end of the SMA spring is fixed on the rail
while the other is joined with the pusher. The spring can be activated by
Joule heating through electrical power. Thus, the bistable dome stays at the
upward stable equilibrium when unactuated; the pusher can glide along the rail
when the SMA spring is powered and eventually causes the dome to snap-through
to the other stable state. To facilitate the grasping, we modified the
original Kirigami pattern by extending the top Kirigami and substrate layers
on the edges so that the added parts of the dome can form hook-shaped
structures locally as shown in Figure 7(B). This shows the potential of our
method to generate more sophisticated 3D structures to enable more
functionalities through a monolithic planar process. We show the ability of
our gripper by grasping various fragile objects, including a strawberry (15
gram), a butter packet (9.7 gram), and an UCLA plush bear (12.6 gram) as shown
in Figure 7C. We also demonstrate sturdy grasping of a cable spool (25.3 gram,
Figure 1D, see supplementary video for details) under moderate disturbance. It
is worth noting that we only need energy during the transition phase while no
energy is required to keep the gripper closed, which could save considerable
amount of energy for applications where long grasping time is necessary.

The flytrap-inspired robot consists of a bistable trilayered dome (with lobe-
shaped Kirigami layers) with triangular leaves (see Figure 7E). The transition
of the robot from the open state to the closed one happens by applying a
minimum loading force. This snap-through transition with the intrinsic energy
barrier can function as embedded sensing and fast actuation to autonomously
capture object. We demonstrate this by placing a self-made toy (made from
Plasticine) on the center of the robot; the loading causes the bistable dome
to snap through and thus quickly close leaves to trap the object. The closing
happened within 0.4 s (see Figure 7 F, see supplementary video for details)
while the toy maintains intact thanks to the softness of the trap. To
facilitate the transition, we harnessed magnetic force by attaching a magnetic
bead on the bottom of the toy with another magnetic bead underneath the robot.
Although this is out of scope of this paper, we could lower the energy barrier
by using thinner sheet materials or reduce pre-stretch to further release the
requirement for magnetic beads. This embedded sensing and actuation into soft
material could reduce the complexity and weight, and increase the adaptability
of resulting devices; without requiring external electronic control and
actuation could also make our robots applicable for cases where conventional
electronics can not survive.

##  III Concluding Remarks

In this paper, we numerically and experimentally studied the influential
parameters on how an initially planar substrate layer and two Kirigami layers
can be used to fabricate bistable soft 3D structures. The structure size and
applied pre-stretch are found to be very critical in affecting the geometry
and the stablility of the free-buckling shapes. By introducing analytical
analysis to the optimized kirigami patterns for bilayer counterparts, we found
that by carefully adjusting the pre-stretch and structural size, we are able
to directly extending the 3D shapes designed for bilayers to trilayer
structures. This combined approach provides a rapid way of designing and
fabricating soft, bistable composite structures of different targeted 3D
shapes. The proposed fabrication techniques, systematic parametric analysis,
and mathematical modelings can lay a foundation in applications in soft
robotics, and wearable devices etc. We demonstrate the advantage of using
these soft bistable structures in engineering tasks that requires fast
grasping and stable manipulation for delicate objects, and autonomous object
detection and fast object capturing.

##  IV Source code

The source code for Finite Element Simulation for all conducted simulation
tests can be found at https://github.com/StructuresComp/bistable-kirigami.

##  V Supplementary

###  V.1 Relationship between the curvature and the maximum height for
spherical cap shapes

We further assume that the deformed shape is close to a spherical cap shape
with maximum height $H$, and has approximately constant curvature everywhere,
as shown in Figure 3C. The length of the curve $L$ can be approximated as,

$\displaystyle(L/2)^{2}=\rho^{2}-(\rho-H)^{2}+H^{2}$ (10)

where $\rho$ is the radius of the hemisphere, which is related to the
curvature $\kappa$ via $\kappa=1/\rho$. Hence,

$\displaystyle\frac{H}{L}=\frac{L}{8}\kappa$ (11)

###  V.2 Equivalent bending stiffness for composite shells

Figure 3A shows the cross-sectional view of the bilayer and trilayer composite
structures. The neutral axis of the trilayer is in the center of the
substrate. Hence, the equivalent bending stiffness is,

$\displaystyle
D_{\textrm{tri}}=n\frac{2{E_{k}[{t_{k}^{3}/12+t_{k}(t_{k}/2+t_{s}/2)^{2}}]+E_{s}[{t_{s}^{3}/12}]}}{{\left({1-{\nu^{2}}}\right)}}+(1-n)\frac{E_{s}{t_{s}^{3}/12}}{{\left({1-{\nu^{2}}}\right)}}$
(12)

where $n$ is the fraction of the area covered by the kirigami layer. $t_{s}$
and $t_{k}$ is the thickness of the substrate and kirigami layer,
respectively. $E_{s}$ and $E_{k}$ is the approximated Young’s modulus of the
substrate and kirigami layer, respectively.

While the z position of the neutral axis for the bilayer is calculated as,

$\displaystyle\bar{z}=\frac{t_{k}(t_{k}/2+t_{s})+(E_{s}/E_{k})t_{s}(t_{s}/2)}{t_{k}+(E_{s}/E_{k})t_{s}}$
(13)

For the bilayer structures, the equivalent bending stiffness is,

$\displaystyle
D_{\textrm{bi}}=n\frac{{E_{k}[{t_{k}^{3}/12+t_{k}(t_{k}/2+t_{s}-\bar{z})^{2}}]+E_{s}[{t_{s}^{3}/12+t_{s}(t_{s}/2-\bar{z})^{2}}]}}{{\left({1-{\nu^{2}}}\right)}}+(1-n)\frac{E_{s}{t_{s}^{3}/12}}{{\left({1-{\nu^{2}}}\right)}}$
(14)

Table 1: Material parameters for the substrate and Kirigami layer Parameter | Value | Unit
---|---|---
Material constant $C_{1}^{\textrm{s}}$ | 22.1 | KPa
Material constant $C_{2}^{\textrm{s}}$ | 1.7 | KPa
Thickness $t^{\textrm{s}}$ | 1.1 | mm
Material constant $C_{1}^{\textrm{kg}}$ | 17.9 | KPa
Material constant $C_{2}^{\textrm{kg}}$ | 84.5 | KPa
Thickness $t^{\textrm{kg}}$ | 1.6 | mm
Material constant $C_{1}^{\textrm{kw}}$ | -2.6 | KPa
Material constant $C_{2}^{\textrm{kw}}$ | 185.8 | KPa
Thickness $t^{\textrm{kw}}$ | 1.4 | mm

Supporting Information

Supporting Information is available from the Wiley Online Library or from the
author.

Acknowledgements

We thank Shyan Shokrzadeh and Vishal Kackar for their assistance on
experiments. Following research grants are gratefully acknowledged: NSF
(CMMI-2053971) for L.M., M.M., and M.K.J.; and NSF (CAREER-2047663,
CMMI-2101751, OAC-2209782) for M.K.J.

## References

  * [1] Daniela Rus and Michael T Tolley.  Design, fabrication and control of soft robots.  Nature, 521(7553):467–475, 2015.
  * [2] George M Whitesides.  Soft robotics.  Angewandte Chemie International Edition, 57(16):4258–4273, 2018\.
  * [3] Yinding Chi, Yanbin Li, Yao Zhao, Yaoye Hong, Yichao Tang, and Jie Yin.  Bistable and multistable actuators for soft robots: Structures, materials, and functionalities.  Advanced Materials, 34(19):2110384, 2022.
  * [4] Richard Baumgartner, Alexander Kogler, Josef M Stadlbauer, Choon Chiang Foo, Rainer Kaltseis, Melanie Baumgartner, Guoyong Mao, Christoph Keplinger, Soo Jin Adrian Koh, Nikita Arnold, et al.  A lesson from plants: high-speed soft robotic actuators.  Advanced Science, 7(5):1903391, 2020.
  * [5] Elisha Lerner, Haijie Zhang, and Jianguo Zhao.  Design and experimentation of a variable stiffness bistable gripper.  In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9925–9931. IEEE, 2020.
  * [6] Jiefeng Sun, Brandon Tighe, and Jianguo Zhao.  Tuning the energy landscape of soft robots for fast and strong motion.  In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 10082–10088. IEEE, 2020.
  * [7] Yuzhen Chen, Tianzhen Liu, and Lihua Jin.  Spatiotemporally programmable surfaces via viscoelastic shell snapping.  Advanced Intelligent Systems, page 2100270, 2022.
  * [8] Jakob A Faber, Janav P Udani, Katherine S Riley, André R Studart, and Andres F Arrieta.  Dome-patterned metamaterial sheets.  Advanced Science, 7(22):2001955, 2020.
  * [9] Tian Chen, Mark Pauly, and Pedro M Reis.  A reprogrammable mechanical metamaterial with stable memory.  Nature, 589(7842):386–390, 2021.
  * [10] Hiromi Yasuda, Tomohiro Tachi, Mia Lee, and Jinkyu Yang.  Origami-based tunable truss structures for non-volatile mechanical memory operation.  Nature communications, 8(1):1–7, 2017.
  * [11] Benjamin Gorissen, David Melancon, Nikolaos Vasios, Mehdi Torbati, and Katia Bertoldi.  Inflatable soft jumper inspired by shell snapping.  Science Robotics, 5(42):eabb1967, 2020.
  * [12] Yongjin Kim, Jay van den Berg, and Alfred J Crosby.  Autonomous snapping and jumping polymer gels.  Nature Materials, 20(12):1695–1701, 2021.
  * [13] Zechen Xiong, Yufeng Su, and Hod Lipson.  Fast untethered soft robotic crawler with elastic instability.  arXiv preprint arXiv:2210.02352, 2022.
  * [14] Yinding Chi, Yaoye Hong, Yao Zhao, Yanbin Li, and Jie Yin.  Snapping for high-speed and high-efficient, butterfly swimming-like soft flapping-wing robot.  arXiv preprint arXiv:2204.05987, 2022.
  * [15] Tian Chen, Osama R Bilal, Kristina Shea, and Chiara Daraio.  Harnessing bistability for directional propulsion of soft, untethered robots.  Proceedings of the National Academy of Sciences, 115(22):5698–5702, 2018.
  * [16] Katherine S Riley, Subhadeep Koner, Juan C Osorio, Yongchao Yu, Harith Morgan, Janav P Udani, Stephen A Sarles, and Andres F Arrieta.  Neuromorphic metamaterials for mechanosensing and perceptual associative learning.  arXiv preprint arXiv:2203.10171, 2022.
  * [17] Philipp Rothemund, Alar Ainla, Lee Belding, Daniel J Preston, Sarah Kurihara, Zhigang Suo, and George M Whitesides.  A soft, bistable valve for autonomous control of soft actuators.  Science Robotics, 3(16):eaar7986, 2018.
  * [18] Wenzhong Yan and Ankur Mehta.  A cut-and-fold self-sustained compliant oscillator for autonomous actuation of origami-inspired robots.  Soft Robotics, 2021.
  * [19] Daniel J Preston, Philipp Rothemund, Haihui Joy Jiang, Markus P Nemitz, Jeff Rawson, Zhigang Suo, and George M Whitesides.  Digital logic for soft devices.  Proceedings of the National Academy of Sciences, 116(16):7750–7759, 2019.
  * [20] Wenzhong Yan, Chang Liu, and Ankur Mehta.  Origami logic gates for printable robots.  In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6084–6089. IEEE, 2021.
  * [21] Dylan Drotman, Saurabh Jadhav, David Sharp, Christian Chan, and Michael T Tolley.  Electronics-free pneumatic circuits for controlling soft-legged robots.  Science Robotics, 6(51):eaay2627, 2021.
  * [22] Aniket Pal, Vanessa Restrepo, Debkalpa Goswami, and Ramses V Martinez.  Exploiting mechanical instabilities in soft robotics: control, sensing, and actuation.  Advanced Materials, 33(19):2006939, 2021.
  * [23] Yuhang Liu, Kai Luo, Shuai Wang, Xiaodong Song, Zhijuan Zhang, Qiang Tian, and Haiyan Hu.  A soft and bistable gripper with adjustable energy barrier for fast capture in space.  Soft Robotics, 2022.
  * [24] Xing Wang, Aarjav Khara, and Chao Chen.  A soft pneumatic bistable reinforced actuator bioinspired by venus flytrap with enhanced grasping capability.  Bioinspiration & Biomimetics, 15(5):056017, 2020.
  * [25] Hong Kai Yap, Hui Yong Ng, and Chen-Hua Yeow.  High-force soft printable pneumatics for soft robotic applications.  Soft Robotics, 3(3):144–158, 2016.
  * [26] Trevor J Jones, Etienne Jambon-Puillet, Joel Marthelot, and P-T Brun.  Bubble casting soft robotics.  Nature, 599(7884):229–233, 2021.
  * [27] Thomas George Thuruthel, Syed Haider Abidi, Matteo Cianchetti, Cecilia Laschi, and Egidio Falotico.  A bistable soft gripper with mechanically embedded sensing and actuation for fast grasping.  In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pages 1049–1054. IEEE, 2020.
  * [28] Shuang Wu, Gregory Langston Baker, Jie Yin, and Yong Zhu.  Fast thermal actuators for soft robotics.  Soft Robotics, 2021.
  * [29] Sheng Xu, Zheng Yan, Kyung-In Jang, Wen Huang, Haoran Fu, Jeonghyun Kim, Zijun Wei, Matthew Flavin, Joselle McCracken, Renhan Wang, et al.  Assembly of micro/nanomaterials into complex, three-dimensional architectures by compressive buckling.  Science, 347(6218):154–159, 2015.
  * [30] Zhichao Fan, Yiyuan Yang, Fan Zhang, Zheng Xu, Hangbo Zhao, Taoyi Wang, Honglie Song, Yonggang Huang, John A Rogers, and Yihui Zhang.  Inverse design strategies for 3d surfaces formed by mechanically guided assembly.  Advanced Materials, 32(14):1908424, 2020.
  * [31] Mingchao Liu, Lucie Domino, and Dominic Vella.  Tapered elasticæ as a route for axisymmetric morphing structures.  Soft Matter, 16(33):7739–7750, 2020.
  * [32] Zheng Yan, Fan Zhang, Jiechen Wang, Fei Liu, Xuelin Guo, Kewang Nan, Qing Lin, Mingye Gao, Dongqing Xiao, Yan Shi, et al.  Controlled mechanical buckling for origami-inspired construction of 3d microstructures in advanced materials.  Advanced functional materials, 26(16):2629–2639, 2016.
  * [33] Guoquan Luo, Haoran Fu, Xu Cheng, Ke Bai, Liping Shi, Xiaodong He, John A Rogers, Yonggang Huang, and Yihui Zhang.  Mechanics of bistable cross-shaped structures through loading-path controlled 3d assembly.  Journal of the Mechanics and Physics of Solids, 129:261–277, 2019\.
  * [34] Jianxun Cui, Felipe R Poblete, and Yong Zhu.  Origami/kirigami-guided morphing of composite sheets.  Advanced Functional Materials, 28(44):1802768, 2018.
  * [35] Yunlan Zhang, Jingyi Yang, Mingchao Liu, and Dominic Vella.  Shape-morphing structures based on perforated kirigami.  Extreme Mechanics Letters, 56:101857, 2022.
  * [36] Jan Zavodnik, Yunbo Wang, Wenzhong Yan, Miha Brojan, and M. Khalid Jawed.  Soft kirigami composites for form-finding of fully flexible deployables.  arXiv preprint, 2023.
  * [37] Leixin Ma, Mrunmayi Mungekar, Vwani Roychowdhury, and Mohammad Khalid Jawed.  Rapid design of fully soft deployable structures via kirigami cuts and active learning.  arXiv preprint, 2023.
  * [38] Teunis van Manen, Shahram Janbaz, Mahya Ganjian, and Amir A Zadpoor.  Kirigami-enabled self-folding origami.  Materials Today, 32:59–67, 2020.
  * [39] Yihui Zhang, Fan Zhang, Zheng Yan, Qiang Ma, Xiuling Li, Yonggang Huang, and John A Rogers.  Printing, folding and assembly methods for forming 3d mesostructures in advanced materials.  Nature Reviews Materials, 2(4):1–17, 2017.
  * [40] Joseph D Paulsen, Evan Hohlfeld, Hunter King, Jiangshui Huang, Zhanlong Qiu, Thomas P Russell, Narayanan Menon, Dominic Vella, and Benny Davidovitch.  Curvature-induced stiffness and the spatial variation of wavelength in wrinkled sheets.  Proceedings of the National Academy of Sciences, 113(5):1144–1149, 2016.
  * [41] Matteo Pezzulla, Gabriel P Smith, Paola Nardinocchi, and Douglas P Holmes.  Geometry and mechanics of thin growing bilayers.  Soft matter, 12(19):4435–4442, 2016.
  * [42] Michael Smith.  ABAQUS/Standard User’s Manual, Version 6.9.  Dassault Systèmes Simulia Corp, United States, 2009.


11institutetext: The University of Sydney, Sydney, NSW 2006, Australia
11email: {muhammad.umair, athman.bouguettaya<EMAIL_ADDRESS>
# Detecting Changes in Crowdsourced Social Media Images

Muhammad Umair  Athman Bouguettaya  Abdallah Lakhdari

###### Abstract

We propose a novel service framework to detect changes in crowdsourced images.
We use a service-oriented approach to model and represent crowdsourced images
as image services. Non-functional attributes of an image service are leveraged
to detect changes in an image. The changes are reported in form of a version
tree. The version tree is constructed in a way that it reflects the extent of
changes introduced in different versions. Afterwards, we find semantic
differences in between different versions to determine the extent of changes
introduced in a specific version. Preliminary experimental results demonstrate
the effectiveness of the proposed approach.

###### Keywords:

Image as a service Modified images Version tree Fake images Fake news Trust
Image provenance Social media Big data

††Acknowledgment: This research was partly made possible by LE220100078 and
DP220101823 grants from the Australian Research Council. The statements made
herein are solely the responsibility of the authors.

##  1 Introduction

Social media has become a key platform to share news and information related
to public incidents [1]. There are more than 5 billion active users on social
media [2]. Social media users publish a large amount of data related to public
events [3]. These social media images may contain critical information about
public incidents i.e., road accidents, crime scenes, violent scenes, etc. The
images related to a particular incident may have different versions uploaded
on social media. Utilizing these versions can significantly facilitate the
task of scene reconstruction to explore unfolding situations which might have
led to the incident.

Existing work on scene reconstruction is based on image processing that is
usually computationally intensive [4]. A novel technique has recently been
proposed to reconstruct scenes using images’ metadata [5]. It leverages the
service paradigm to represent social media images and related posted
information as services. It abstracts social media users as social-sensors and
an image as a social-sensor service, abbreviated as SocSen. Henceforth, we use
the term ‘image service’ to refer to ‘social-sensor service’. The only
difference between an image service and a SocSen service is that an image
service comparatively contains a vast set of non-functional attributes. An
image service is defined to have functional and non-functional properties.
Functional attributes are the parameters related to the actions pertaining to
the capture of an image service. Examples of functional attributes are
switching picture/video modes, pressing on/off button, delayed/timed picture
taking, taking panoramic shots, etc. Non-functional attributes facilitate the
delivery of the purpose of taking a picture. Examples of non-functional
attributes are subject distance, camera elevation angle, resolution, location
etc. The non-functional attributes are usually available in form of different
metadata tags. An image service can also have different versions.

Most existing work on image services is related to image service selection and
composition [6, 7]. An image service composition approach has been proposed to
form a tapestry in the spatial aspect and a storyboard in the temporal aspect
[5]. The focus of most existing work on image services has hitherto been to
reconstruct a scene. A fundamental assumption in this regard has so far been
that the participating image services are intrinsically trustworthy. However,
the trust issue becomes paramount when image services are assumed to be
crowdsourced [8]. For instance, a crime scene analysis relying on crowdsourced
image services may contain untrustworthy images which may lead to wrong
conclusions. Untrustworthy image services can be avoided by analyzing
different versions of a social media image to find the most trustworthy
version for the scene reconstruction.

Detecting untrustworthy image services has traditionally been addressed using
image processing and information retrieval techniques [9]. These approaches
are usually costly and computationally expensive. A preliminary service-based
trust framework is proposed in [10, 11] which is based on users’ comments and
stances in an image service to assess its credibility. However, the
credibility of image services may not be completely assessed based only on the
user’s stance. Fake posts on social media can get supportive comments from
other users [12]. The stance of credible users may also be biased [13].
Moreover, these approaches focus on changes in an individual image service.
Whereas, multiple social media images are being forged in conjunction to
manipulate information about an incident. To address these limitations, we
propose to assess trust among image services using a more objective and
holistic framework consisting of changes and updates in different versions of
the image services. In this regard, we leverage non-functional attributes of
different versions of an image to detect changes in image services. we operate
under the assumption that the non-functional attributes, encompassing critical
technical details, can be seamlessly accessed through the image service.

We propose a novel approach to detect changes in the non-functional attributes
of an image service, as a first step towards ascertaining whether an image
service is fake. Editing an image service may make it inconsistent with its
non-functional attributes. An attempt to hide the facts in metadata may create
some discrepancies among non-functional attributes. These discrepancies are
not straightforward to identify as they are usually embedded in the non-
functional attributes. We utilize these inconsistencies to detect changes in
an image service. In this respect, we form different groups of image non-
functional attributes such that analyzing each group collectively provides
useful insights on inconsistencies. For instance, shutter speed, exposure time
and aperture size are the attributes that inform about the intake of light
while capturing an image. Analyzing them collectively may indirectly reflect
the time of day while the image service was captured. Changes in different
versions of an image service are also investigated to get more insights about
the transformation of different versions from the original image. In this
regard, we assume that the provided image service is not the only version of
an image upload on social media. We use an image-based search i.e., Reverse
Image Search (RIS) to collect all versions of an image service available on
social media. A temporal sorting is then performed to arrange them in a
sequence they were uploaded on social media. We propose a novel representation
of changes in an image service in terms of a version tree. The version tree is
constructed in a way such that the information about changes in a specific
version is implicit in its position/placement in the tree. Therefore, one of
the main contribution of this paper is to propose a framework to build the
version tree. Afterwards, a state-of-the-art semantic similarity measure is
used to find semantic differences between an image and its versions. The
proposed framework is a kind of image provenance analysis based only on the
non-functional attributes. The proposed approach is validated on 5849 images
collected from an image metadata dataset. Below, we summarize our main
contributions:

  * •

A framework is proposed to detect changes in different versions of an image
service using only the non-functional attributes.

  * •

We introduce a unique way of reporting changes in different versions of an
image service in terms of a version tree. Knowledge about changes in all
versions is implicit in the tree.

  * •

The proposed framework also provides a serendipitous image provenance analysis
using the version tree.

The proposed method effectively handles a typical set of modifications that
are reflected in the non-functional attributes. Although, the non-functional
attributes may not completely capture certain changes within the image itself,
such as alterations in shades, intensity of colors, or distortion, it remains
well-suited for a wide range of image modifications. It is worth noting that
the proposed framework’s performance may be influenced by the availability of
non-functional attributes. In cases where a limited number of such attributes
are available, an alternative approach involves obtaining meta-information
from the social media post. While this alternative approach may be less
precise due to potential questions about the accuracy of meta-information, it
still offers valuable insights.

##  2 Motivating Scenario

We consider a scene of a plane crash in New York that happened in 2009 as our
motivating scenario. Figure 1 shows evacuation of US Airways Flight 1549 as it
floats on the Hudson River. This image was falsely claimed as the lost
Malaysian aircraft MH370 in many social media posts in 2014. In those
misleading posts, the images are original but they contain a false claim. Many
state-of-the-art solutions rely on image processing to identify untrustworthy
social media images [14]. These solutions focus on the content within the
images and hence, they may fail to identify changes which are not in images.
Moreover, these solutions consider changes in a single image and provide no
knowledge about the image provenance. To address these limitations, we propose
a unique way of identifying untrustworthy images by doing image provenance
analysis using only the non-functional attributes of an image and its
different versions. Exploring the non-functional attributes of different
versions of an image may reveal a lot of inconsistencies among them. These
inconsistencies reflect the trust of an image. For instance, in Figure 1,
metadata of different versions of an image is inconsistent and is reflective
of the modifications in different versions.

Figure 1: Motivating Scenario

##  3 Related Work

Existing work on image services is related to image service selection and
composition strategies. A context and direction aware spatio-temporal
clustering is proposed in [6]. The proposed approach helps to compose the
relevant images to form a tapestry in the spatial aspect and a story in the
temporal aspect. A fundamental assumption in this work has hitherto been that
images participating in scene reconstruction are trustworthy. However, image
services in a crowdsourced environment can be untrustworthy. Traditional
approaches to identify untrustworthy images are based on image processing and
machine learning [15]. Moreover, some text classification techniques are also
available in the literature that can be employed to classify fake text with an
image [16]. The aforementioned image processing based technique has high
accuracies in determining the fakes in an image but require high computational
power.

Some recent studies claim that a subset of trust can be derived using light
weight service-oriented approaches [11, 10]. A new image services trust model
is proposed in [10]. The trustworthiness of an image service is measured based
on the users’ stance. Textual features of the image services, i.e., comments
are utilized to determine the trust of the service. Another users’ stance and
credibility-based image service’s trust model is proposed in [11]. The
proposed model considers various indicators such as the stance embedded in the
services’ comments, their meta-data, e.g., time, along with the users’
credibility. These approaches are unable to capture modifications in an image
service because the misleading content on social media may receive positive
comments from other users [12]. Moreover, comments from credible users can be
biased.

We propose a relatively more objective and holistic approach that considers
modifications and updates introduced in an image service to determine the
trust of an image service. This paper focuses on detecting the changes, as a
first step towards determining the trust. In this regard, different versions
of an image are investigated to get more insights on image provenance details.
Versions of an image are being utilized in many state-of-the-art solutions to
identify changes in an image. For instance, an image provenance analysis is
proposed in [17] using the metadata. Different versions of an image are
investigated in [18] to explore different contexts in which the image was
shared. Different versions of video clips are analyzed in [19] to detect
misinformation in videos. A framework is proposed in [20] that relies on
different versions of images to determine if they are shared out of context.
The difference between these approaches and our proposed framework is that
these approaches majorly rely on computer vision to determine fake images,
whereas, our proposed approach is completely based on analyzing metadata of
different versions to determine changes in an image. It is worth clarifying
that the proposed framework is different from object versioning. The focus of
object versioning systems is the creation and management of versions. Whereas,
the focus of the proposed framework is to find image versions based on the
changes in the non-functional attributes of an image service.

##  4 Image Service Model

We represent an image service in terms of its functional and non-functional
attributes:

$ImgServ=\\{f\\}\cup\\{nf\\}$ (1)

where f and nf represent the set of functional and non-functional attributes
respectively. Functional attributes represent the actions involved in
capturing an image. Functional attributes can be formalized as:

$f=\\{\alpha,\mu,\gamma\\}$ (2)

where $\alpha$ represents the action of capturing an image, i.e., pressing the
shutter, $\mu$ is the action to switch from one capturing mode to another
i.e., switching from picture to video and vice versa, and $\gamma$ represents
the time delay in taking a picture. Non-functional part of an image service
consists of spatio-temporal, contextual and intrinsic attributes as listed in
Table 1.

$nf=\\{\zeta,\tau,c,\iota\\}$ (3)

where $\zeta$ represents the set of spatial attributes, $\tau$ is the set of
temporal attributes, c contains contextual attributes, and $\iota$ represents
intrinsic attributes. Intrinsic attributes reflect changes inside the images
i.e., changes in the visual content.

Table 1: Description of Non-functional Attributes Categories |  Description | Example Attributes
---|---|---
Spatial Features |  Spatial metadata tags describe the location at which the image was taken. | GPS Coordinates
City, Sate, Country
Temporal Features |  Temporal metadata tags describe the date and time when the image was taken. | GPS Timezone Offset
GPS Timestamp
Contextual Features |  Contextual features define the context of an image. Contextual attributes may also contain the details of the ambiance. | Title
Caption
| Headline

Non-functional Attributes of an Image Service: We identify different non-
functional attributes of an image service that may indicate changes within an
image (refer to Table 1). We group the non-functional attributes into the
following categories:

  * •

Spatial Features: Spatial attributes represent the location where the image
was captured. Modified spatial tags may be an indication of fake background.

  * •

Temporal Features: Temporal attributes represent the date and time when the
image was captured. Forged temporal metadata tags develop a fake story.

  * •

Contextual Features: Contextual features are related to the context of an
image. Fake context may support fake spatio-temporal tags of an image.

Potential Modifications in Non-functional Attributes: We propose a
categorization of potential modifications that may exist in an image service.
The following are the possible changes in image’s non-functional attributes:

Figure 2: Potential Changes in Images’ Non-functional Attributes

  * •

Modified Date and Time: Figure 2a claims two children in 2015 Nepal
earthquake. It is actually a picture of two Vietnamese taken in 2007.

  * •

Modified Location: Figure 2b was a viral photo in 2014 claiming the picture of
the lost Malaysian MH370 plane. It turned out to be a photo of a plane crash
in New York in 2009.

  * •

Modified Context: Figure 2c claims a camel with limbs cut off used for
begging. The camel is actually resting with legs bent under itself.

##  5 Proposed Framework

This Section provides details of our novel framework (shown in Figure 3).

###  5.1 Feature Extraction

The proposed framework takes non-functional attributes of an image service as
an input and determines whether the image is modified. The first step is to
extract the meta-information available with the image which resides in the
metadata of the uploaded image and the information posted with the image. We
propose to perform image provenance analysis using the metadata of different
versions. In this regard, we assume that the provided image is not the only
version available on social media. Therefore, we utilize Reverse Image Search
to collect all versions of the provided image along with their metadata.

Figure 3: The Proposed Framework

###  5.2 Grouping the Non-functional Attributes

We propose to determine changes in an image service using only the non-
functional attributes, i.e., without using the image itself. The non-
functional attributes should ideally be completely reflective of the content
in the picture to correctly reflect on the changes in an image. However,
state-of-the-art representation of non-functional attributes is not well-
reflective of the semantics of an image. Therefore, we propose a novel
representation of these non-functional attributes in form of different groups
in Table 2. The attributes are placed in a same group if observing them
collectively can provide some useful insights about the changes. Analyzing
each group reveals useful insights about the picture. These insights are
mostly related to spatio-temporal and contextual parameters of an image
service. For instance, shutter speed and exposure time are placed in a same
group because analyzing these attributes collectively indicates the intake of
light while capturing an image. It is indirectly reflective of the time of day
when the picture was captured.

Table 2: Grouping of Non-functional Attributes Attributes |  Description
---|---
DateTimeOriginal, DateTimeDigitized, DateTime |  An image metadata contains various timestamps. By comparing these timestamps, we can see whether the image has been modified.
Make, Model, Other attributes (focal length, aperture, etc.) |  A camera’s make and model can be compared with different attributes i.e., focal length, aperture size, and resolution etc. to check whether the camera supports these attribute values.
Aperture, Shutter speed, Exposure value |  Exposure value is calculated from aperture and shutter speed. We can re-compute exposure value and compare it with the attributes.
Shutter speed, Aperture, ISO |  The three pillars of exposure are shutter speed, aperture, and ISO. These three attributes are interconnected and changing one affects the others. For example, increasing the shutter speed may require a wider aperture or higher ISO to compensate for the reduced amount of light reaching the sensor. Similarly, using a narrow aperture for a deeper depth of field may require a slower shutter speed or higher ISO to compensate for the reduced amount of light.
Aperture, Exposure Time, Shutter speed, Datetime |  Exposure time and shutter speed reflects the light intake in a picture. It can be transformed to the time of day based on the following: Noon: high f/stop, fast shutter speed; Night: low f/stop, slow shutter speed to let more light in. However, it depends on the environment (indoor/outdoor) and purpose of shooting.
GPS info, TimeZoneOffset, DateTime, Location |  We analyze these attributes to see whether the GPS is consistent with the timezone. Moreover, we check whether the datetime is consistent with the timezone.
Datetime, GPS info, ISO, white balance and other camera settings |  By using the datetime and GPS info, we can find weather conditions of the shooting location. The white balance and ISO depends on the weather.
Temperature, Humidity, Pressure, Weather |  Using GPS and timestamp, we can retrieve weather information of location of shooting using online APIs.
WaterDepth, GPS info |  Using GPS info, we can get the water depth information of the shooting location that can be compared with the given WaterDepth to see if it is consistent.

###  5.3 Normalizing the Non-functional Attributes

Attributes in a group may have different units and scales. The attributes in
each group are first normalized on a common scale. In this regard, we
transform information provided by each group shown in Table 2 in to their
normalized values. For instance, if we have three attributes i.e., $\zeta$,
$\tau$ and $c$, then these attributes can be normalized using the following
equation:

$\hat{\zeta}=\frac{\zeta_{i}-min(\zeta,\tau,c)}{max(\zeta,\tau,c)-min(\zeta,\tau,c)}$
(4)

where $\hat{\zeta}$ represents the normalized value of $\zeta$. $\tau$ and $c$
can be normalized similarly.

###  5.4 Creating a Distributed Representation of Attributes

We create a distributed representation of the normalized non-functional
attributes. Each attribute is represented on a high dimensional space. The
number of dimensions depends on the types of attributes in a group. In most
cases, spatio-temporal and contextual attributes constitute the Cartesian
space. These dimensions are further composed of multiple sub-dimensions. For
instance, the spatial axis is represented by two axes: longitude and latitude.
Similarly, a contextual attribute can be represented as a multi-dimensional
vector using either Latent Semantic Analysis or any Word-Embeddings-based
approach. Therefore, the contextual axis is further divided in to multiple
sub-axes as shown in Figure 5. Afterwards, we plot the values of each group in
this high dimensional space. We then cluster the attribute values of each
group for a specific version. The same process is repeated for each group. As
a result, we get different clusters plotted in this distributed space as shown
in Figure 5. Each cluster corresponds to a single version of an image. It is
worth mentioning that there is an additional time axis ($t_{upload}$) in the
distributed space that informs the time of upload of a specific version. Due
to this axis, clusters cannot be overlapped completely. For two clusters to
overlap, their versions should be uploaded at exactly the same time. The
clusters projected in this space provides a holistic view of non-functional
attributes of all versions. The differences in placements and shapes of
clusters reflect discrepancies among the versions. Moreover, on time axis, it
also reflects the evolution of different versions from an image. However, this
sequence of uploads may not accurately inform about the sequence of changes
introduced in an image. For instance, version corresponding to cluster C3 in
Figure 5 is uploaded after C2, however, in reality, C3 may be the predecessor
of C2. To address it, we transform the sequence of uploads to the sequence of
changes.

Figure 4: Distributed Representation of Attributes

Figure 5: Version Tree

###  5.5 Version Tree

Most viral fake social media images are usually related to a specific incident
i.e., a road accident, a natural disaster, a man-made disaster etc.
Manipulating facts about these incidents may involve sharing relevant fake
posts. We consider relevant images and versions of an image to be two
different entities. Relevant images contain relevant stuff but they are not
originated from a same image. Whereas, an image $b$ is a version of another
image $a$ if $b$ is originated from $a$. It is crucial to analyze different
versions of an image to detect changes in an image service because it provides
a holistic view of manipulations about a specific incident. We propose an
image provenance analysis based only on the image metadata. The provenance
analysis results in a tree because of the fact that an image version can only
be originated from one single image. Therefore, we propose to construct a
version tree which is reflective of the modifications introduced in each
version of an image. It also informs about the transformation of different
attributes in that image. Figure 5 shows a generic version tree.

Definition 1: An image b is assumed to be version of another image a if

  * •

Image b originated from image a.

  * •

Attributes in image a can be transformed to attributes in image b. We leverage
the concept of transformation matrix to analyze the nature of transform.
Equation 5 shows the linear transformations of one cluster to another.

$\Delta_{t}\times
I_{a}=I_{b}\>where\>\Delta_{t}=\begin{array}[]{cc}\begin{bmatrix}w&x&.\\\
y&z&.\\\
.&.&.\end{bmatrix}\end{array}\>and\>I_{a}=\begin{array}[]{c}\begin{bmatrix}attr_{1}\\\
attr_{2}\\\ .\end{bmatrix}\end{array}$ (5)

where $attr$ stands for attribute, $I_{a}$ is an attribute matrix for image a,
$I_{b}$ is an attribute matrix for image b, and $\Delta_{t}$ is a linear
transformation matrix. According to this statement, even if two images
$image_{b}$ and $image_{c}$ are originated from $image_{a}$, if
$\Delta_{t}\times I_{b}=I_{c}$ is satisfied, then $image_{c}$ will be
considered as a version of $image_{b}$.

  * •

Linear transformation can cover the following transformations: translation,
scaling, rotation, reflection, shearing, projection, orthogonal projection and
affine transformation. If the relationship between the two clusters is
inherently nonlinear, a quadratic transformation (shown in equation 6) may
better capture the underlying transformation. Therefore, we consider the
matrix form of quadratic transformations if the linear transformation of an
attribute matrix can not be found or is too complex. The criteria to select
linear or quadratic transformation depends on the complexity of the
transformation. We choose the transform with a simpler transformation matrix.

$I_{a}^{t}\times\Delta_{t}\times I_{a}=Quadratic\>form\>of\>I_{b}$ (6)

We consider the complexity of $\Delta_{t}$ to check if $\Delta_{t}\times
I_{a}=I_{b}$ is satisfied. The lower the complexity, higher will be the
likelihood for $image_{b}$ to be version of $image_{a}$. The complexity is
determined by the similarity of the transformation matrix with the identity
matrix (the matrix that, if used as a transformation matrix, returns the same
input matrix). However, to determine the complexity of quadratic
transformation, we compute the difference of the transformation matrix with
the matrix that, if used as a transformation matrix, returns quadratic form of
the input matrix. It is worth mentioning that the tree proposed in Figure 5 is
implicitly directional. In the process of constructing the version tree, it
automatically determines the origin of the image. However, in case of a linear
tree i.e., if there is a single path from root to a leaf node, then the
direction of the tree can be deduced from the order of uploads of images.

The version tree provides a holistic view of the changes introduced in
different versions of an image. More importantly, it informs about the
transformation of an image in to different versions that serves as a tool for
image provenance analysis. The depth of each node in the tree is reflective of
the extent of changes in the corresponding version. These insights can not be
reflected by reporting changes in each version individually.

###  5.6 Heuristics to Construct a Version Tree

We propose novel heuristics to construct version tree for an image. We utilize
the clustering and the transformation matrix proposed in the previous Sections
to construct the version tree. The clusters projected on the Cartesian space
are the groups of attributes of different versions. Each cluster corresponds
to a specific group of a specific version. The distance between these clusters
reflect the changes among versions. The variations in the clusters actually
represent the transformation of a version in to another. As stated earlier, we
consider the sequence of uploads of image versions as a baseline solution.
Therefore, we propose reordering of the clusters on the time axes to make it
accurate. It will result in a sequence of edits/updates in an image. The
criteria to start the swapping is to check whether attributes in a version are
being transformed from another version. To ensure it, we rely on the
transformation matrix proposed in equation 6. According to equation 6, there
will be multiple transformation matrices for each version because we have a
transformation matrix for each group of non-functional attributes. Therefore,
for each version, we have $\Delta_{1}$ to $\Delta_{n}$ matrices where n
represents the total number of groups. In the next step, we find the nearest
neighbors for each $\Delta$ using the Frobenius Distance:

$F_{a,b}=\sqrt{trace((a-b)\times(a-b)^{\prime})}$ (7)

Next, we need to decide the sequence of versions, $V_{1}$ is placed before
$V_{2}$ if $\Delta_{1}$ to $\Delta_{n}$ of $V_{2}$ can be derived using either
linear or quadratic transformation. The reordering is only possible if
equation 5 or 6 is satisfied for two consecutive versions on a time axis.
Another factor, on which we rely to decide about swapping is the similarity
between two versions. If the clusters shapes are similar for two distant
versions, we do reordering to make them closer. The following equation is used
to check this criterion:

$if\quad C_{n+1}\cap C_{n}\geq C_{n}\cap C_{n-1}$ (8)

We need a control statement to decide how many iterations are required to
reach an optimal sequence of versions. In this regard, we introduce a
threshold in equation 9 to control the reordering:

$if\quad(C_{n+1}\cap C_{n})-(C_{n}\cap C_{n-1})\geq Threshold$ (9)

The value of threshold is assigned in a way that the criterion for reordering
becomes relatively strict after every swap.

Figure 6: Trend of Threshold

Figure 7: Accuracy

As evident from Figure 7, the threshold follows an exponential trend. The
exponential trend is reflective of the fact that the reordering becomes
relatively less likely after each iteration. However, the difference/increase
in threshold is not linear. Therefore, the change in threshold must be
computed after each interval. We calculate this gap based on the similarity of
changes between two consecutive clusters/versions on the time axis. The less
similar the changes are in two versions, the more likely should be the swap,
so we can assign a higher increase in threshold. The idea about the design of
the threshold is presented in [21, 22].

$\Delta Threshold\propto(\Delta C_{n}-\Delta C_{n-1})$ (10)

This procedure constructs one branch of the version tree. To construct other
branches, the same procedure is repeated for the remaining nodes/versions.
Once a version tree is constructed, the consecutive version nodes can be
compared using any semantic similarity measure to find its semantic difference
with the previous version. Comparing different nodes in the version tree
results in $\delta\zeta$, $\delta\tau$ and $\delta c$, where $\delta$
represents a change. The proposed framework is self-configurable. Whenever, it
encounters a distinct type of version, it becomes part of its learning. The
concept of self-configurable algorithms is presented in [23].

##  6 Experimentation and Results

We use a real image metadata dataset to conduct experiments [24]. The dataset
contains images and videos along with their metadata. The dataset includes a
wide range of images covering various subjects, scenes, and visual
characteristics. We use the metadata-extractor library, which is a Java-based
library for reading metadata from image files. The extracted metadata
encompasses information such as camera make and model, image dimensions,
capture date and time, GPS coordinates, and other technical details. Although,
the dataset provides images, however, we only exploit their metadata. We use
ChatGPT to generate different versions of images contained in this metadata
dataset. ChatGPT can be effectively utilized to generate versions of image
metadata due to its language generation capabilities and understanding of
contextual information. We also leverage ChatGPT to introduce systematic
changes in the sample metadata of different image versions. Instructions are
provided to the ChatGPT to create different types of variations in between
metadata of different versions of an image. For instance, in some images,
shutter speed and exposure time have been made inconsistent. By leveraging its
language generation capabilities, ChatGPT can produce altered metadata such as
updated timestamps, modified camera settings, or edited descriptions,
providing an indication that the image has undergone changes. Figure 8 shows a
sample metadata of two versions generated by ChatGPT. It also shows
inconsistencies introduced by ChatGPT. The inconsistencies are shown in
highlighted text in Figure 8.

The experiments are completely scalable as we use an API provided by OpenAI to
execute text commands on ChatGPT. The API allows us to make requests to the
ChatGPT model hosted on OpenAI’s servers and receive responses in real-time,
enabling interactive and dynamic conversations with the language model. By
utilizing the API, we leverage OpenAI’s infrastructure to handle the
computational resources required for generating versions of an image metadata,
ensuring scalability and availability. Moreover, the API provides quick
responses, allowing for real-time interactions and dynamic conversations with
ChatGPT. We use the HTTP POST method to send a request to the API endpoint. We
structure the request payload in JSON format. The payload may contain a list
of message objects with a role (either ”system”, ”user”, or ”assistant”) and
content (the text of the message). We cluster the attributes of each version
for each group. We use the clustering approach proposed in [25]. The distance
between the clusters and the dissimilarities among them are leveraged to
determine the transformation matrix to build the version tree.

Figure 8: A Sample Metadata and Changes Introduced by ChatGPT

####  6.0.1 Effectiveness

We report the performance of the proposed approach in terms of accuracy (in
Figure 7) and run-time. The ground truth of metadata is known from the
metadata dataset. Accuracy is calculated as the percentage of correctly
classified image versions. We compare our approach with a baseline and a brute
force approach. Sequence of uploads of an image (the order in which the image
versions were originally uploaded) on social media is regarded as the baseline
solution, whereas, the brute force approach doesn’t consider the criteria
defined in equation 9 and 10. The proposed framework achieves an accuracy of
76%. Whereas, the baseline approach is correct 64% of the times. Brute force
performs better than the proposed framework but at the cost of additional
computations. We also report the run-time complexity and the time consumed (in
nano seconds) for brute force and our proposed approach as shown in Table 3.
Time complexity is computed by counting units of time. Moreover, the increase
in run-time with the increase in the number of inconsistencies is reported in
Figure 10.

Table 3: Run Time Efficiency | Baseline | Brute-force | Heuristics
---|---|---|---
Run-time Complexity | 1 |  O(N2) |  O(N1/2)
Time Consumed (ns) | 309 | 26500 | 15300

####  6.0.2 Comparison

We compare the accuracy of our proposed framework with a state-of-the-art that
uses image metadata along with image content to perform image provenance
analysis [17]. The framework proposed in [17] has three variants: a complete
image-based solution; Kruskal’s maximum spanning tree algorithm based only on
image metadata; and Cluster-SURF which utilizes both images and their
metadata. The accuracy of the approach is computed in terms of the overlap
between the original version tree and the constructed version tree. Our
proposed approach outperforms all variants as reflected in Figure 10.

Figure 9: Run-time Efficiency

Figure 10: Comparison with State-of-the-art

##  7 Conclusion

We propose a novel framework to detect changes in an image service using only
the non-functional attributes. The proposed model returns a version tree for
an image service. Theory of matrix transformation is leveraged in this paper
to model the transformation of one image version to another. The results
validate the proposed approach. This work can be further extended to
investigate the detected changes to check whether the changes are constituting
a fake. One aspect to consider is that run-time may increase with a larger
scale of images.

## References

  * [1] Ali, K., Dong, H., Bouguettaya, A., Erradi, A., Hadjidj, R.: Sentiment analysis as a service: a social media based sentiment analysis framework. In: 2017 IEEE international conference on web services (ICWS). pp. 660–667. IEEE (2017)
  * [2] Griffis, H.M., Kilaru, A.S., Werner, R.M., Asch, D.A., Hershey, J.C., Hill, S., Ha, Y.P., Sellers, A., Mahoney, K., Merchant, R.M.: Use of social media across us hospitals: descriptive analysis of adoption and utilization. Journal of medical internet research 16(11), e3758 (2014)
  * [3] Liu, X., Troncy, R., Huet, B.: Using social media to identify events. In: Proceedings of the 3rd ACM SIGMM international workshop on Social media. pp. 3–8 (2011)
  * [4] Chae, J., Thom, D., Bosch, H., Jang, Y., Maciejewski, R., Ebert, D.S., Ertl, T.: Spatiotemporal social media analytics for abnormal event detection and examination using seasonal-trend decomposition. In: 2012 IEEE Conference on Visual Analytics Science and Technology (VAST). pp. 143–152. IEEE (2012)
  * [5] Aamir, T., Dong, H., Bouguettaya, A.: Heuristics based mosaic of social-sensor services for scene reconstruction. In: International Conference on Web Information Systems Engineering. pp. 503–515. Springer (2020)
  * [6] Aamir, T., Dong, H., Bouguettaya, A.: Social-sensor composition for tapestry scenes. IEEE Transactions on Services Computing (2020)
  * [7] Mistry, S., Bouguettaya, A., Dong, H., Qin, A.K.: Metaheuristic optimization for long-term iaas service composition. IEEE Transactions on Services Computing 11(1), 131–143 (2016)
  * [8] Shen, C., Kasra, M., Pan, W., Bassett, G.A., Malloch, Y., O’Brien, J.F.: Fake images: The effects of source, intermediary, and digital media literacy on contextual assessment of image credibility online. New media & society 21(2), 438–463 (2019)
  * [9] Gupta, A., Lamba, H., Kumaraguru, P., Joshi, A.: Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy. In: Proceedings of the 22nd international conference on World Wide Web. pp. 729–736 (2013)
  * [10] Aamir et al, T.: Trust in social-sensor cloud service. In: 2018 IEEE International Conference on Web Services (ICWS). pp. 359–362. IEEE (2018)
  * [11] Aamir, T., Dong, H., Bouguettaya, A.: Stance and credibility based trust in social-sensor cloud services. In: International Conference on Web Information Systems Engineering. pp. 178–189. Springer (2018)
  * [12] Colliander, J.: “this is fake news”: Investigating the role of conformity to other users’ views when commenting on and spreading disinformation in social media. Computers in Human Behavior 97, 202–215 (2019)
  * [13] Zimmer, F., Scheibe, K., Stock, M., Stock, W.G.: Fake news in social media: Bad algorithms or biased users? Journal of Information Science Theory and Practice 7(2), 40–53 (2019)
  * [14] Shahzad, H.F., Rustam, F., Flores, E.S., Luís Vidal Mazón, J., de la Torre Diez, I., Ashraf, I.: A review of image processing techniques for deepfakes. Sensors 22(12), 4556 (2022)
  * [15] Patel, M., Padiya, J., Singh, M.: Fake news detection using machine learning and natural language processing. In: Combating Fake News with Computational Intelligence Techniques, pp. 127–148. Springer (2022)
  * [16] Umair, M., Saeed, Z., Ahmad, M., Amir, H., Akmal, B., Ahmad, N.: Multi-class classification of bi-lingual sms using naive bayes algorithm. In: 2020 IEEE 23rd International Multitopic Conference (INMIC). pp. 1–5. IEEE (2020)
  * [17] Bharati, A., Moreira, D., Brogan, J., Hale, P., Bowyer, K., Flynn, P., Rocha, A., Scheirer, W.: Beyond pixels: Image provenance analysis leveraging metadata. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). pp. 1692–1702. IEEE (2019)
  * [18] Saez-Trumper, D.: Fake tweet buster: a webtool to identify users promoting fake news ontwitter. In: Proceedings of the 25th ACM conference on Hypertext and social media. pp. 316–317 (2014)
  * [19] Ganti, D.: A novel method for detecting misinformation in videos, utilizing reverse image search, semantic analysis, and sentiment comparison of metadata. Utilizing Reverse Image Search, Semantic Analysis, and Sentiment Comparison of Metadata (June 5, 2022) (2022)
  * [20] Qian, S., Shen, C., Zhang, J.: Fighting cheapfakes: using a digital media literacy intervention to motivate reverse search of out-of-context visual misinformation. Journal of Computer-Mediated Communication 28(1), zmac024 (2023)
  * [21] Umair, M., Shah, G.A.: Energy management of smart homes. In: 2020 IEEE International Conference on Smart Computing (SMARTCOMP). pp. 247–249. IEEE (2020)
  * [22] Umair, M., Cheema, M.A., Afzal, B., Shah, G.: Energy management of smart homes over fog-based iot architecture. Sustainable Computing: Informatics and Systems 39, 100898 (2023)
  * [23] Umair, M., Afzal, B., Khan, A., Rehman, A.U., Sekercioglu, Y.A., Shah, G.A.: Self-configurable hybrid energy management system for smart buildings. In: 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV). pp. 1241–1246. IEEE (2018)
  * [24] Noakes, D.: Database of images from various digital cameras. https://github.com/drewnoakes/metadata-extractor-images (2019)
  * [25] Rizvi, S.A., Umair, M., Cheema, M.A.: Clustering of countries for covid-19 cases based on disease prevalence, health systems and environmental indicators. Chaos, Solitons & Fractals 151, 111240 (2021)