2022

[1,2]Jiarong Hong (ORCID: 0000-0001-7860-2181)

1]Mechanical Engineering, University of Minnesota – Twin Cities, 111 Church
Street SE, Minneapolis, 55455, Minnesota, USA

2]St Anthony Falls Laboratory, University of Minnesota – Twin Cities, 2 3rd
Ave SE, Minneapolis, 55414, Minnesota, USA

3]Aerospace Engineering and Mechanics, University of Minnesota – Twin Cities,
110 Union St SE, Minneapolis, 55455, Minnesota, USA

4]Civil, Environmental, and Geo- Engineering, University of Minnesota – Twin
Cities, 500 Pillsbury Drive SE, Minneapolis, 55455, Minnesota, USA

# Imaging-based 3D Particle Tracking System for Field Characterization of
Particle Dynamics in Atmospheric Flows

Nathaniel Bristow (ORCID: 0000-0001-7287-7635)  Jiaqi Li (ORCID:
0000-0002-1201-7489)  Peter Hartford  Michele Guala (ORCID:
0000-0002-9788-8119<EMAIL_ADDRESS>[  [  [  [

###### Abstract

A particle tracking velocimetry apparatus is presented that is capable of
measuring three-dimensional particle trajectories across large volumes, of the
order of several meters, during natural snowfall events. Field experiments,
aimed at understanding snow settling kinematics in atmospheric flows, were
conducted during the 2021/2022 winter season using this apparatus, from which
we show preliminary results. An overview of the methodology, wherein we use a
UAV-based calibration method, is provided, and analysis is conducted of a
select dataset to demonstrate the capabilities of the system for studying
inertial particle dynamics in atmospheric flows. A modular camera array is
used, designed specifically for handling the challenges of field deployment
during snowfall. This imaging system is calibrated using synchronized imaging
of a UAV-carried target to enable measurements centered 10 m above the ground
within approximately a 4 m$\times$4 m$\times$6 m volume. Using the measured
Lagrangian particle tracks we present data concerning 3D trajectory curvature
and acceleration statistics, as well as clustering behavior using Voronoï
analysis. The limitations, as well as potential future developments, of such a
system are discussed in the context of applications with other inertial
particles.

###### keywords:

3D particle tracking, atmospheric flows, inertial particles

##  1 Introduction

A wide variety of geophysical processes involve atmospheric transport of
inertial particles, such as wind-blown sand and dust, sea sprays, and
snowfall, wherein the details of particle kinematics can have long-ranging
implications for society in terms of the environment, climate, and weather.
Aeolian transport of sand drives major changes in landscape, not only on Earth
but also on other planets (Lapotre et al, 2016), and the related suspension of
mineral dust aerosols in the atmosphere, and their subsequent settling, may
greatly influence global climate (Kok et al, 2018). For example, sea sprays,
which can broadly also be characterized in the same manner as inertial
particles in atmospheric turbulence, play an important role in the momentum
and thermal fluxes at the air-sea interface, and thus must be modeled
appropriately for the forecasting of tropical cyclones (Emanuel, 2018). Snow
particles, which are the primary focus of the study herein, are particularly
complex inertial particles due to their intricate morphologies, causing
different tumbling behaviors and turbulence interactions that modulate their
fall-speed and thereby the resultant ground accumulation heterogeneity
(Garrett and Yuter, 2014; Li et al, 2021b). For such reasons, it is critical
to better understand the kinematics of atmospheric inertial particles and
their interaction with the surrounding flow.

Such atmospheric flow, however, is naturally characterized by a broad range of
turbulence scales, where Reynolds numbers are typically on the order of
$10^{6}$. The largest motions that drive bulk transport can be captured by
remote sensing techniques, such as time-of-flight light detection and ranging
(LiDAR) from satellites (Huang et al, 2015), for tracking aerosol movements,
or Doppler LiDAR (Lundquist et al, 2017) for wind measurements in the
atmospheric boundary layer (ABL). However, resolution of the small-scale
motions that govern the dynamics of individual particles, and thus influence
important parameters for modeling such as fall-speed, requires alternative
approaches, such as imaging, for particle-level tracking.

Imaging systems for particle tracking and velocimetry are commonplace in the
laboratory setting, implemented in both 2D and 3D, the latter with multi-
camera arrays (for an overview, see (Discetti and Coletti, 2018)). The 3D
approach is more challenging due to the need for precise calibration of the
camera system to be able to triangulate or otherwise reconstruct particle
positions accurately, but also due to the difficulty of subsequently linking
these particles across time steps to form long Lagrangian trajectories that
are needed to study inertial particle kinematics. Great advancements have been
made in both respects, such as with iterative particle reconstruction (IPR;
(Wieneke, 2012)) and predictive tracking (e.g., Shake-the-Box; (Schanz et al,
2016a; Novara et al, 2019)), both of which aid in enabling long Lagrangian
trajectories to be obtained from dense particle images. However, such methods
have generally been limited to measurement domains of $\sim$1 m (Schanz et al,
2016b; Terra et al, 2019), due to combined challenges of tracer seeding,
illumination, and calibration, and thus have seen very limited implementation
in the field due to technical challenges.

That being said, notable examples of particle imaging-based flow measurement
have been conducted in the field. These include particle image velocimetry
(PIV) or particle tracking velocimetry (PTV) in the atmospheric surface layer
(ASL) for both wind energy research (Hong et al, 2014; Abraham and Hong, 2020;
Wei et al, 2021) and boundary layer turbulence studies (Hommema and Adrian,
2003; Morris et al, 2007; Toloui et al, 2014; Heisel et al, 2018), PTV for
snow settling in the ASL (Nemes et al, 2017; Li et al, 2021a, b), as well as
water surface velocimetry (Perks et al, 2020). In the case of the ASL, both
artificial tracers (smoke, fog) and natural tracers (snow) have been used,
while in the latter case for water surface velocimetry the patterns of surface
reflections from the water itself may be used, or else artificial tracers
introduced. However, except for (Wei et al, 2021), all the cited approaches
are two-dimensional, either in a light sheet or on a surface.

For the case of snow settling, such 2D studies have provided valuable insights
into the enhancement of settling velocities due to turbulence (Nemes et al,
2017), clustering behavior influencing settling (Li et al, 2021a), and
preferential sampling of sweeping motions from vortices (Li et al, 2021b).
Nevertheless, three-dimensional measurements of snow particle motions are
necessary in order to fully resolve their kinematics. In particular, both
trajectory curvature and particle clustering may be biased on resolving only
in 2D, and therefore 3D measurements of snow particle transport are needed.

Adaptation of three-dimensional particle tracking to the field is even rarer,
compared to cases mentioned for 2D measurement, due to limitations from
calibration procedures, particle seeding, and illumination. A recent
noteworthy implementation of 3D PTV in the field, however, is given by Wei et
al (2021), who used a multi-camera system to measure the time-averaged flow
structure and vorticity in the wake of a full-scale vertical axis wind
turbine. They achieved measurements in a large volume (10 m$\times$7
m$\times$7 m), though with a low particle concentration, as they had to use
artificial snow as tracers introduced upstream. Multiple runs had to be
performed and datapoints spatially binned over the ensemble to obtain a single
time-averaged flow field, as the entire domain could not be sampled
simultaneously. Thus, turbulent fluctuations could not be captured.

Other related work focused on obtaining Lagrangian trajectories in large-scale
measurements has been conducted in the field of bio-locomotion (e.g.,
Theriault et al, 2014; Muller et al, 2020), tracking the motions of birds and
of fish. These studies generally involve sparse objects, as usually only a few
points of interest within the field of view are to be tracked, but have
involved innovative calibration approaches to enable tracking in large
volumes. The study herein builds upon these calibration approaches, as will be
discussed in 2.2.

The aim of the present work is to track complex 3D motions of snow particles
as they settle in atmospheric turbulence, and in particular to obtain a
sufficiently high number of Lagrangian trajectories to enable the estimate of
acceleration, curvature, and high order statistics. Here we seek to overcome
challenges inherent to the snow settling case, measuring 10 m above the ground
in a large volume while also dealing with high concentrations of particles in
camera images. In the following, we describe our methodology for achieving
these aims through the development of a field-scale 3D imaging system
(2.1–2.3), including preliminary validation experiments (2.4), after which we
show results from a field deployment in April 2022 (3) to demonstrate the
capabilities of this system in terms of the new data and analysis afforded by
such measurements (4). This work is conducted as part of the Grand-scale
Atmospheric Imaging Apparatus project, hereafter referred to as GAIA.

##  2 Methods

###  2.1 System description

The principal components of the field-scale 3D imaging system developed under
the GAIA project, hereafter referred to as GAIA-PTV, and the associated
workflow, are depicted in Fig. 1. In the following, we provide a brief
overview followed by a more detailed description. Firstly, GAIA-PTV involves a
modular, multi-camera imaging array that, once calibrated, enables the desired
3D particle triangulation and tracking. To obtain the calibration of the
camera array, an unmanned aerial vehicle (UAV), equipped with its own
lighting, is flown within the measurement volume while being imaged by all
cameras. Images collected during the calibration procedure as well as the
particle images taken during the experiment are then processed as will be
described in Sections 2.2 and 2.3.

Figure 1: Schematic illustrating overview of GAIA-PTV workflow.

This system was designed with a few key criteria in mind. Due to the difficult
working conditions in the field during snowfall, GAIA-PTV needed to be easy to
use and capable of rapid deployment. Furthermore, it was desirable for
deployment configurations to be flexible and enable large distances between
cameras. Such flexibility is needed in the field when snow concentration, and
thus image density, is outside our control, and instead, we may desire to
adjust the particle image density in each camera by changing the camera
separation distances. Lastly, we needed to measure snow settling dynamics in a
volume of at least several meters in each dimension, centered high enough
above the ground to avoid surface effects on the flow.

To meet these demands, GAIA-PTV implements a modular approach. We use a
dedicated single-board-computer (SBC) for each camera and wireless
communication over a local network, using a COMFAST Outdoor WIFI Antenna. This
local network can also be used for wireless camera synchronization for frame
rates $<100$ Hz. Thus, each camera is a stand-alone unit that can be moved and
set up easily. The only cables between cameras are for synchronization $>100$
Hz, wherein GPIO connectors from separate cameras would be wired together,
setting one camera as “primary” to output a trigger signal when exposure
begins causing the remaining “secondary” cameras to simultaneously begin
capture. Having dedicated SBC’s is also a cost-saving approach, removing the
need for a more expensive central processing computer capable of handling the
bandwidth from the multiple cameras, and thus enables easy future upgrades of
the system with more than 4 cameras. This was achieved using Nvidia Jetson
Nano SBC’s, each equipped with a wireless antenna for communication and a
solid-state drive (SSD) for rapid data storage such that images do not need to
be buffered on the camera and can be continuously recorded at a sufficiently
high frame rate (e.g., 200 Hz). Nvidia’s Jetson platform provides a full Linux
operating system with enough I/O for all auxiliary devices, needing only a 5
V, 10 W power supply easily provided from a battery pack.

The cameras used are FLIR Black Fly S U3-27S5C color units with the Sony
IMX429 sensor, pixels measuring 4.5 $\mu$m, capable of 95 fps continuous
recording at 2.8 MP (1464$\times$1936 pix.), or up to 200 fps at 0.7 MP, using
decimation. Each camera is paired with a 16 mm Fujinon lens, with 3.45 $\mu$m
pixel pitch and an aspherical lens design minimizing radial distortions.

As stated above, an additional requirement for the system was that it be able
to image snow particles in a measurement region high above the ground,
approximately 10 m. This particular choice of height was motivated by the goal
of avoiding the flow effects of local surface “roughness” elements (e.g.,
bushes, small topographic changes) on snow settling, ensuring a more uniform
light background for the four-camera images, avoiding contamination from re-
suspended snow particles, and lastly, to match previous experiments involving
2D tracking that were designed under similar considerations (Nemes et al,
2017; Li et al, 2021a, b). This design criterion introduced two challenges for
the system. First, our lighting system to illuminate the snow particles needed
to be strong enough for those distances, and second, the camera calibration
would need to be done far above the ground.

The sampling volume to be imaged measures typically around 4 m$\times$4
m$\times$6 m ($x$, $y$, $z$). Therefore, to illuminate the snowflakes
throughout this large of a volume centered at a height of 10 m, a 5 kW
spotlight is used, similar to that used by Hong et al (2014) and Toloui et al
(2014). The beam is expanded at 18 degree divergence half angle such that it
spread to a 4 m wide region in the measurement volume, after reflecting off of
a 45 degree polished aluminum mirror. The conical illumination region is then
able to be further modified by adjusting the curvature of the mirror such that
the volume is elongated in the desired direction (e.g., downstream).

###  2.2 Calibration

The camera calibration cannot follow traditional approaches taken in the
laboratory, and instead is implemented using a UAV equipped with two rigidly
connected lights that is flown throughout the measurement domain. While being
imaged by the camera array, the UAV samples the entire measurement volume from
edge to edge. The basis of this approach, which follows the method by
Theriault et al (2014), is to move an “object” of known dimensions through the
measurement volume while identifying image coordinates of the object keypoints
in each camera. Here, our object consists of two LEDs, one green and one red,
held at a fixed separation of 34.5 cm by a carbon fiber tube, with the object
“keypoints” being the LEDs themselves. This apparatus is attached to the UAV,
which needs to be flown carefully throughout the measurement domain.

Snow tracking deployments, including those described in 3, are conducted at
night, with artificial illumination. In such a case, the extent of the
measurement volume in the horizontal plane, and the UAV within it, is visually
apparent to the flight operator. Therefore, the UAV flight plan can be
implemented manually, keeping within this $x$–$y$ domain while monitoring the
UAV altitude to stay within the desired region. A Holybro V500 quadcopter UAV,
with 400 mm wheelbase, is used for this purpose, with a Pixhawk flight
controller operating on the open-source ArduPilot platform wherein the Mission
Planner software can be used for monitoring the UAV.

However, it should be noted that GAIA-PTV was also designed with the future
goal in mind to be capable of other types of 3D PTV measurements in daylight
as well, where no artificial lighting would be used, in which case a precise,
automated flight plan may need to be charted with GPS coordinates. Mission
Planner enables many features for flight planning that are useful in this
regard, including tools for automated scanning of regions at various
altitudes, as well as the ability to inject real-time kinematic positioning
(RTK) corrections to the GPS. The UAV is therefore equipped with an RTK-
capable Ublox M8P GPS unit that, upon integration with NTRIPP data stream
provided from a local Minnesota Department of Transportation (MNDOT) base
station, is capable of providing $\sim$cm scale accuracy to the UAV
positioning. Regular GPS units are typically only $\sim$1 m accurate, at best.

The UAV flight for calibration typically lasts about 5 minutes while the
cameras capture images at 30 fps in order to obtain several thousand positions
at which the UAV LEDs are seen by all cameras. All images are processed using
Python with in-house code to find all potential green and red LED candidates.
A color filter is used in image processing, selecting the green and red bands
in the RGB images separately, and an intensity threshold is applied. Poor
candidates are further filtered using a temporal smoothness constraint to the
LED positions in the 2D images.

To obtain a calibration mapping from the image to the object space, the
8-point algorithm is used (Hartley and Zisserman, 2003) to initially estimate
camera pose (i.e., position and orientation), based on a pinhole model. This
estimate is then refined using sparse bundle adjustment (SBA) (Lourakis and
Argyros, 2009), implemented in “easyWand” by Theriault et al (2014). The
uncertainty of the calibration results that are obtained following the SBA is
assessed using two different metrics: reconstructed LED separation in 3D
space, varying for each frame captured; and reprojection error, given as the
r.m.s. distance between original and reprojected keypoints. If the mean
reprojection error are too large, i.e. $>$0.5 pixels, data points with the
largest reprojection errors are iteratively removed and the calibration re-
calculated. Otherwise, more calibration points could be collected with further
flights in order to reduce error. Such calibration flights are typically
conducted both before and after particle image datasets are collected.

###  2.3 Particle tracking

After collecting particle images and obtaining a camera calibration,
Lagrangian trajectories are extracted using predictive 3D PTV following the
implementation by Tan et al (2020) with OpenLPT. This open-source
implementation of 3D PTV is based on the Shake-the-Box (StB) algorithm by
Schanz et al (2016a), with additional improvements made for ghost particle
rejection. StB has been shown to be a robust predictive tracking approach that
iteratively solves the problems of triangulation and matching/linking across
frames simultaneously, leveraging the expected particle position information
to handle dense particle images (e.g., 0.1 ppp). In OpenLPT, initializing
trajectories with which to predict subsequent particle positions requires at
least 4 time steps. Following initialization, at each frame a combination of
active, inactive, and exited tracks are determined. The active tracks are a
combination of short trajectories that have not yet reached the needed
initialization length and long tracks that have been successfully predicted.
Once an active track is lost, it is added to the inactive tracks list, and if
the particle departs the specified measurement domain, it is added to the list
of exited tracks.

Once trajectories of particles are obtained, their paths must be smoothed
before computing higher-order quantities such as acceleration and curvature. A
Gaussian filter kernel is convolved with the particle position vector in each
component, with an optimal kernel size determined following the approach of
(Nemes et al, 2017). If the kernel is too small, acceleration data is
contaminated by high-frequency noise, whereas if it is excessively large,
fine-scale motions may be overly attenuated. In the results presented in 4,
this filter length was 0.26 s, or $\sim 3\tau_{\eta}$, where $\tau_{\eta}$ is
the Kolmogorov timescale, which is similar to that used in prior studies
(Nemes et al, 2017; Li et al, 2021a).

###  2.4 Method validation and demonstration

Prior to the deployment of GAIA-PTV in the snow, two experiments were
undertaken to validate the approach. First, the UAV-based calibration method
was tested at scale to determine if this methodology could provide
sufficiently accurate camera pose estimation. Figure 2 shows a spatial map of
the path taken by the UAV, where markers denote the reconstructed positions of
the UAV lights colored by their deviation from the true distance between the
lights, $D$. For this case the UAV’s flight was automated to scan throughout a
volume measuring approximately 4 m$\times$4 m$\times$4 m using the equipped
RTK GPS unit. It can be seen that reconstructed errors are largest near the
edges of the calibration volume, where they are at most $\sim$1 cm, and much
less throughout the core of the volume. A separate scale is added where $D$ is
normalized with the measured distance between lights of 34.5 cm, where
relative errors are below 3%.

Figure 2: Spatial distribution of distances, $D$, between UAV light pairs
after reconstruction, plotted with calibrated camera poses indicated by each
set of axes.

A further experiment was performed under controlled conditions to test the
tracking capabilities provided by the calibration when input to the OpenLPT
software. For this experiment, standard hole-punch confetti paper particles (6
mm diameter) were dropped manually in still-air conditions in a large
warehouse environment. Though it was not feasible to densely fill an entire
measurement with such particles, they mimicked many of the conditions expected
to be encountered in the field while also enabling some ground-truthing of
their fall-speed. In particular, the disk-shaped confetti were expected to
exhibit more complex trajectories with tumbling motions similar to that can be
exhibited by snowflakes, such as those with dendrite morphologies. Cameras
were positioned in a fan array at 11 m radial distance from the center of the
measurement volume in order to reproduce a similar scale to be measured over
in the field. The calibration here was not performed with a UAV, as the
experiment was indoors, and instead the LEDs and carbon fiber rod were held by
hand and moved through the measurement volume manually. A sample confetti
image (after enhancement) and resultant trajectories are shown in Fig 3, where
it can be seen that long trajectories are able to be captured. Furthermore, it
was estimated that approximately 93% of confetti particles could be tracked,
based on the identified particles in each image (after filtering out double-
counted particles whose centroids were within 1 pixel of each other) and the
sum of the durations of all particle tracks (in number of frames), which gave
an estimate of the average particles tracked per frame.

Looking closer at individual trajectories, as in Fig. 4a, it can be seen that
the fine-scale motions of the confetti disks can be resolved, with an
oscillation in the trajectory as the disk tumbles end-over-end. The average
fall-speed of individual confetti particles was also measured in independent
experiments and compared with the distribution of particle fall-speed measured
with PTV, which show close agreement (Fig. 4b).

Figure 3: a) A sample confetti image; b) all confetti trajectories with colors
indicating time from beginning of image sequence. Figure 4: a) Sample confetti
trajectory; b) Probability density function of mean fall speeds for individual
particles, with dashed vertical line indicating separately measured average
fall speed.

##  3 Experiment

###  3.1 Field site

A series of experiments measuring natural snowfall were undertaken during the
winter season of 2021/2022. All measurements were performed at the University
of Minnesota Outreach, Research and Education (UMore) Park, in Rosemount,
Minnesota, near the EOLOS wind energy research station. The region surrounding
the deployment location of the GAIA-PTV system is depicted in Fig. 5, and
consists of relatively flat farmland, apart from the meteorological (met)
tower nearby. The met tower is equipped with SAT3 Campbell Scientific Sonic
anemometers at elevations of 10, 30, 80, and 129 m above ground, sampling at
20 Hz, and cup-and-vane anemometers at elevations of 7, 27, 52, 77, 102, and
126 m above ground. Further details concerning the met tower are provided in
(Toloui et al, 2014; Heisel et al, 2018).

Figure 5: Satellite images of field site for experiments in Rosemount,
Minnesota, indicating positions of the system deployment in April 2022 and of
the meteorological tower, as well as the mean wind direction from the
deployment.

The data analyzed herein was collected on April 17, 2022 between the hours of
20:39 and 23:00 CST. Though multiple datasets were collected during this
period, here we present an exemplary subset of this data from 20:59 to 21:00.
Wind conditions and air temperature during this period are shown in Fig. 6a–c,
where speeds of approximately 1.7 m/s were steady within $\pm$0.3 m/s,
measured from the 10 m sonic anemometer.

The morphologies of the snow particles that precipitated during the deployment
period were analyzed using a digital inline holography system similar to that
used by Nemes et al (2017). The equivalent diameters of the particles are
shown in Fig. 6d, where particles typically measured less than 5 mm in
diameter. It may be noted that the p.d.f. differs here from those shown by
Nemes et al (2017) and Li et al (2021a), wherein a distinct peak in the
distribution could be identified, whereas here the p.d.f. increases towards
zero. This is due to the increased accuracy of the particle identification
used herein that is able to identify snow particles over a larger range of
sizes using machine learning (Li et al, 2022).

Figure 6: a–c) Meteorological tower data from the 10 m sonic anemometer,
showing a) wind speed, b) wind direction, and c) temperature. d) Snow particle
equivalent diameters from the deployment period.

The field site and equipment used are depicted in Fig. 5a, in which can be
seen the layout of the spotlight illumination, met tower, camera system, UAV,
and an example of raw particle images obtained. The layout is also
schematically illustrated in the inset of Fig. 1. Cameras were mounted on
tripods in a “fan” array spanning 90 degrees of a circular arc, 5.5 m in
radius. All were tilted upward to be oriented at 58 degrees from horizontal,
resulting in the varying magnification across the field of view in Fig. 7b.
This resulted in a measurement volume of approximately $4\times 4\times 6$ m
in $x$, $y$, and $z$, the streamwise, spanwise, and wall-normal directions,
respectively. Note that the linear size of the field of view is of the same
order of magnitude of the integral scale of the flow at that distance from the
ground, $\simeq k_{v}z\simeq 4$ m, based on the von Karman constant $k_{v}$
and the mixing length assumption in turbulent boundary layers. Cameras
recorded at 200 fps in decimated mode with 732$\times$968 pixels, providing
spatial resolution of $\approx$6.3 mm per pixel at the center of the FOV. For
a given dataset, 10,000 frames were recorded in each camera, providing 50
second duration sequences.

Figure 7: a) Photo of the field site indicating relevant equipment, and b)
sample snow particle image.

###  3.2 Calibration

A measurement volume of approximately 4 m$\times$4 m$\times$6 m was calibrated
(after the particle image datasets were collected) as depicted in the inset of
Fig. 1, where red and green markers indicate the reconstructed trajectory of
the UAV as it flew through the measurement region. Cameras recorded the UAV
lights at 30 fps during this sequence and resulted in 4063 frames in which
both lights could be seen by all four cameras and successfully identified.
This resulted in mean reprojection errors of 0.275 pixels across the four
cameras, or approximately 1.73 mm in the center of the measurement domain.
This showed an improvement upon the validation experiment, likely due to the
longer calibration sequence capturing a greater number of calibration points.
Please note that 1.73 mm is of the same order as $d_{s}$, mean snow particle
diameter, and $<1\%$ of $I_{d}$, mean inter-particle distance (see 4.3).

##  4 Results

###  4.1 Particle trajectories

Having obtained this camera calibration, snow particle images could be
processed using 3D PTV. In the following, the coorindate system is defined
such that $x$ is the streamwise direction, $y$ is spanwise, and $z$ is
vertical.

The distribution of snow particle trajectory durations, obtained from 250,619
particles tracked over 10,000 frames, is shown in Fig. 8. Due to the challenge
in tracking particle motions in 3D, durations are skewed towards lower values,
and taper off at the long end by approximately 400 frames, or 2 seconds. This
does not include particle trajectories shorter than 5 frames, which are not
considered valid trajectories for studying particle kinematics as they have
not passed the initialization phase and developed sufficient duration.
Approximately 1800 “long” particle trajectories ($>4$ frames) and 7000 “short”
trajectories are obtained at each frame. These longer trajectories are
continuously tracked until they either exit the domain or become “inactive”
(i.e., lost). The number of snow particles present in the measurement volume,
on average, is $\approx 11,000$, based on the number of particles identified
in all four camera frames. Thus the measurement yield of reconstructed
particle positions, out of all particles physically in the measurement domain,
is estimated as 80% while the yield of long trajectories is be approximated as
20%. This lower estimate compared to the confetti validation experiment is
most likely due to imperfect view overlap from different cameras at the edges
of the FOV and poorer image quality from the upper region of the measurement
volume.

Figure 8: Probability density function of snow particle tracking durations for
continuous “long” trajectories.

A sample set of snow particle trajectories is shown in Fig. 9 from two
different vantage points. As can be seen in Fig. 9a, when viewed from the side
particle trajectories appear relatively linear, elongated in the streamwise
direction. This type of view is similar to what would be captured using a 2D
imaging method. Closer inspection of Fig. 9c reveals more complex kinematics
of the particles, with subtle oscillations in their curvature. This is
emphasized through the colormap applied to the trajectories. Here, the lateral
acceleration component, $a_{y}$, is chosen as it particularly emphasizes the
three-dimensionality of the particle motion that cannot be captured with
planar or 2D imaging methods. The particles along these trajectories
accelerate back and forth, suggesting non-negligible interactions with the
turbulent flow and the different scales of vortical motions (Li et al, 2021b).
Furthermore, this behavior is not observable for all neighboring particles;
some are observed to translate relatively linearly along the primary flow
direction as they settle. This could be speculated to be the result of
differences in the size and morphology of the snow particles, affecting their
inertial properties and their ability to follow different scales of the flow,
despite different particles sampling similar regions in the flow. Note that
the mean inter-particle distance is $\simeq 0.18$ m, or $\sim 2\lambda_{T}$,
where $\lambda_{T}$ is the Taylor microscale. Estimated by the nearby sonic
anemometer, $\lambda_{T}$ provides reasonable estimates of the thickness of
the shear layers and the size of the vortex cores (Heisel et al, 2021),
implying that nearby snow particles may still sample different flow
topologies.

Figure 9: Sample snow particle trajectories plotted at different viewing
angles and magnification, colored by spanwise acceleration, $a_{y}$. The
$x$-axis is the mean streamwise direction, and the coordinate system origin is
at the center of the measurement volume.

###  4.2 Acceleration and curvature

The statistics of the particle kinematics, explored qualitatively in the
previous section, are presented in Fig. 10 plotting p.d.f.s of settling
velocity, acceleration components, and curvature. Settling velocity displays
an approximately normal distribution, weakly skewed towards lower fall speeds.
In terms of acceleration, the spanwise component, $a_{y}$, is generally of
similar magnitude to $a_{x}$ and $a_{z}$, though their distribution tails
differ, and as such these spanwise motions are important to capture. All three
components display wider tails compared to a normal distribution, where
$a_{z}$ in particular is skewed towards negative accelerations. The streamwise
component, $a_{x}$, on the other hand, is more strongly skewed with positive
accelerations.

Curvature is given as

$\rho=\frac{1}{R}=\frac{\|\bf{v}\times a\|}{\|\bf{v}\|^{3}}$ (1)

where $\rho$ is curvature, $R$ is the radius of curvature, $\bf{v}$ and
$\bf{a}$ are velocity and acceleration vectors, respectively, and double
brackets indicate taking the L2-norm. The distribution of $R$, plotted in Fig.
10c, is calculated in both 2D and 3D, where in the 2D case only the $x$ and
$z$ components of $\bf{v}$ and $\bf{a}$ are used, simulating the components
that would be resolved if a planar imaging system were used that was
appropriately aligned parallel to the streamwise direction. Markedly different
distributions are shown between these two, where the 2D case is skewed towards
larger weaker curvature, with peaks in their distributions of approximately 8
m and 6 m for the 2D and 3D cases, respectively. This agrees with the observed
behavior in the sample trajectories from Fig. 9c, which showed significant
lateral components in acceleration and should be expected given the
probability density functions of acceleration components shown in Fig. 10b.
During parts of trajectories where the particle may oscillate in the
$y$-direction, a 2D tracking system would not register this out-of-plane
motion and thus return higher radii of curvature for such tracks. Please note
that particle inertial properties are known to affect the curvature and
acceleration distribution (see e.g., Bec et al, 2006), as heavier-than-fluid
particles cannot be trapped in vortical flows. Hence, it can be expected that
the peak of curvature should be larger than the expected size of vortex cores.

Figure 10: Probability density functions of a) vertical settling velocity, b)
acceleration components, and c) radii of curvature. In a) and b), the dashed
lines indicate Gaussian distributions for comparison.)

###  4.3 Clustering

Three-dimensional particle cloud reconstructions are particularly
advantageous, as compared to 2D planar imaging, because they also enable
direct quantification of particle concentration statistics and clustering
behaviors without any 2D assumptions. This can be achieved using Voronoï
diagrams, spatial tessellations of the particle cloud, wherein the volume of
individual cells produced by the tessellation are inversely proportional to
the local concentration. This method is preferable to other approaches, such
as box-counting, as it avoids biasing due to input parameter choices such as
bin size, and is also robust to particle sub-sampling (up to 50%) effects
often unavoidable in experimental data (Monchaux, 2012). The shape and extent
of a Voronoï cell are defined such that all points within the cell (except the
edges) are closer to the single particle lying within the cell than they are
to any neighboring particles outside the cell. Thus, each Voronoï cell is
associated with a given particle, and the vertices and borderlines themselves
are equidistant to the neighboring particles. An example of a single Voronoï
cell is shown in Figure 11a, showing the complex topology of the cell in 3D
space, while the p.d.f. of Voronoï cell size is shown in Fig. 11b. Here, the
length scales associated with the cell size for both the full 3D data as well
as quasi-2D data is shown. The 2D case is simulated for comparison by
computing the 2D Voronoï tessellation on particles found within sub-volumes
oriented in the $x$–$z$ plane, with their y-dimension removed. Two different
sub-volume thicknesses are compared, of 0.3 m and 0.1 m, similar to that used
in previous 2D planar snow tracking experiments (Nemes et al, 2017; Li et al,
2021a, b). In all cases, 2D or 3D the data are taken from particles within
$\pm 1$ m of the measurement domain center (in all directions) in order to
avoid any unwanted effects near the boundaries. The length scale, $L$, for the
3D case is the cube-root of the cell volume, while for the 2D case $L$ is the
square-root of the cell area. The distributions of $L$ peak at 0.18 m for the
3D, indicative of true inter-particle spacing, $I_{d}$, whereas the 2D cases
are affected by the sub-volume thickness. The thicker 2D sub-volume results in
smaller $L$, due to the fact that more particles are projected onto the
$x$–$z$ plane, compared to the thin 2D sub-volume Monchaux (2012).

Normalizing the volumes (or areas for the 2D case) by their respective means,
these data can be compared to randomly distributed particles (Fig. 11c),
which, following Ferenc and Néda (2007), can be described for the 2D and 3D
cases, respectively, as

$f_{2D}(x)=\frac{343}{15}\sqrt{\frac{7}{2\pi}}x^{5/2}e^{-7x/2}$ (2)
$f_{3D}(x)=\frac{3125}{24}x^{4}e^{-5x}$ (3)

Figure 11: a) Example of a single Voronoï cell produced by the tessellation,
where the red dot indicates the snow particle, blue dots indicate vertices,
and green lines are the ridges of the cell; b) Probability density functions
of cell length scales, $L$, comparing the 2D and 3D cases; c) Probability
density functions in log–log space of cell volumes, compared against models
from Ferenc and Néda (2007) for randomly distributed particles given by
Equations 2 and 3.

Deviations from these distributions indicate clustering of particles due to
flow and particle inertia. In all comparisons between snow data and profiles
of $f_{2D}$ and $f_{3D}$, it can be seen that the snow particles display
clustering behavior. Evidence of such clustering can be qualitatively observed
in the raw images, as indicated in the inset of Fig. 7b, though it must be
remembered that the image is a projection of all particles across the depth of
the volume and therefore the scale from the images may not be indicative of
true spacing. Furthermore, the results of the distributions suggest that 2D
measurement, of the same snow particles, would have slightly underestimated
the extent of this clustering behavior. This is shown by the disparity between
the distribution for the 3D experimental data and the profile for $f_{3D}$,
which is greater than that of the 2D data. This is true for both the “thick”
or “thin” 2D data, where it can also be noted that the normalization here
largely removes the differences observed in Fig. 11b. Our results are largely
consistent with the results shown by Monchaux (2012) concerning 2D/3D biases
in Voronoï analysis, wherein differences between 2D and 3D treatments are
minor though appreciable. It may be speculated, however, that the extent to
which these differences between 2D or 3D treatments are manifested may be
influenced by the particular type of flow (e.g., a turbulent boundary layer)
and the statistically persistent 3D vortical structures it contains.

##  5 Conclusions and Discussion

The GAIA-PTV system presented herein has been demonstrated to provide
unprecedented Lagrangian tracking of snow settling motions in 3D using a
multiview imaging array and UAV-based calibration approach. Snow particles are
tracked within a volume measuring approximately 4 m$\times$4 m$\times$6 m
centered at a height 10 m above ground providing the ability to capture long
trajectories that free from flow disturbances due to the ground. The results
presented herein provide a snapshot of the analyses that are enabled by GAIA-
PTV, and demonstrate the utility of measuring particle kinematics in 3D over
2D.

Importantly, this work enables further systematic investigations of snow
settling dynamics. With the measurement system demonstrated, experiments can
be conducted across a range of atmospheric turbulence conditions to offer
unique insights into particle kinematics and clustering. Furthermore, the
GAIA-PTV system can be extended to measuring transport of other particles in
atmospheric flows, such as droplet sprays and dispersed pollen. The system was
designed to be both robust in the field, as has been tested under the harsh
winter conditions, and scalable depending on the needs of the flow phenomena
of interest.

In future work, two key limitations of the current system could be improved
upon. Firstly, as with most 3D PTV systems, the Lagrangian tracking capability
is limited in part by the domain size and the fact that the domain is fixed.
If the current system could be made mobile, such as by using UAV-mounted
cameras instead of fixed tripod-mounted cameras on the ground, it would enable
much longer tracking times to study particle dispersion, for example. In a
similar vein as has been performed with translating laboratory imaging systems
(e.g., the “flying PIV” by Zheng and Longmire (2013)) the system could follow
along with the particles of interest, keeping them within the measurement
domain. Such advances are challenging, however, primarily due to the problem
of calibrating this type of system.

Second, the need for artificial illumination as used herein constrains the
measurement opportunities to the hours of darkness, when the particles of
interest can be illuminated against the night sky. Advances in particle
detection methods capable of dealing with natural daylight illumination could
significantly increase use-cases, as well as simplify the deployment
requirements without the need for a high-wattage light sources. The primary
challenge is in overcoming the low contrast available when imaging during
daylight, but rapid developments in the field of computer vision and object
detection may yield promising tools for improvements in this regard.

Acknowledgments

This work was supported by the National Science Foundation under grants
CBET-2018658 and AGS-1822192.

## Declarations

  * •

Ethical approval: Not applicable.

  * •

Author contributions: J.H. and M.G. oversaw the study conception and design.
N.B. wrote the manuscript text, generated figures, and was the primary
contributor to the measurement system development. P.H. contributed to the
development of the measurement system. All authors contributed to the field
experiments.

  * •

Funding: This work was supported by the National Science Foundation under
grants CBET-2018658 and AGS-1822192.

  * •

Availability of data and materials: The data that support the findings of this
study are available from the corresponding author, J.H., upon reasonable
request.

  * •

Competing interests: The authors declare no conflicts of interest.

## References

  *   * Abraham and Hong (2020) Abraham A, Hong J (2020) Dynamic wake modulation induced by utility-scale wind turbine operation. Applied Energy 257:114,003
  * Bec et al (2006) Bec J, Biferale L, Boffetta G, et al (2006) Acceleration statistics of heavy particles in turbulence. Journal of Fluid Mechanics 550:349–358
  * Discetti and Coletti (2018) Discetti S, Coletti F (2018) Volumetric velocimetry for fluid flows. Measurement Science and Technology 29(4):042,001
  * Emanuel (2018) Emanuel K (2018) 100 years of progress in tropical cyclone research. Meteorological Monographs 59:15–1
  * Ferenc and Néda (2007) Ferenc JS, Néda Z (2007) On the size distribution of poisson voronoi cells. Physica A: Statistical Mechanics and its Applications 385(2):518–526
  * Garrett and Yuter (2014) Garrett TJ, Yuter SE (2014) Observed influence of riming, temperature, and turbulence on the fallspeed of solid precipitation. Geophysical Research Letters 41(18):6515–6522
  * Hartley and Zisserman (2003) Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press
  * Heisel et al (2018) Heisel M, Dasari T, Liu Y, et al (2018) The spatial structure of the logarithmic region in very-high-reynolds-number rough wall turbulent boundary layers. Journal of Fluid Mechanics 857:704–747
  * Heisel et al (2021) Heisel M, de Silva C, Hutchins N, et al (2021) Prograde vortices, internal shear layers and the Taylor microscale in high-Reynolds-number turbulent boundary layers. Journal of Fluid Mechanics 920:715–731
  * Hommema and Adrian (2003) Hommema SE, Adrian RJ (2003) Packet structure of surface eddies in the atmospheric boundary layer. Boundary-Layer Meteorology 106(1):147–170
  * Hong et al (2014) Hong J, Toloui M, Chamorro LP, et al (2014) Natural snowfall reveals large-scale flow structures in the wake of a 2.5-MW wind turbine. Nature communications 5(1):1–9
  * Huang et al (2015) Huang J, Liu J, Chen B, et al (2015) Detection of anthropogenic dust using calipso lidar measurements. Atmospheric Chemistry and Physics 15(20):11,653–11,665
  * Kok et al (2018) Kok JF, Ward DS, Mahowald NM, et al (2018) Global and regional importance of the direct dust-climate feedback. Nature Communications 9(1):1–11
  * Lapotre et al (2016) Lapotre M, Ewing R, Lamb M, et al (2016) Large wind ripples on mars: A record of atmospheric evolution. Science 353(6294):55–58
  * Li et al (2021a) Li C, Lim K, Berk T, et al (2021a) Settling and clustering of snow particles in atmospheric turbulence. Journal of Fluid Mechanics 912
  * Li et al (2021b) Li J, Abraham A, Guala M, et al (2021b) Evidence of preferential sweeping during snow settling in atmospheric turbulence. Journal of Fluid Mechanics 928
  * Li et al (2022) Li J, Guala M, Hong J (2022) Snow particle analyzer for simultaneous measurements of snow density and morphology. arXiv preprint arXiv:220911129
  * Lourakis and Argyros (2009) Lourakis MI, Argyros AA (2009) Sba: A software package for generic sparse bundle adjustment. ACM Transactions on Mathematical Software (TOMS) 36(1):1–30
  * Lundquist et al (2017) Lundquist JK, Wilczak JM, Ashton R, et al (2017) Assessing state-of-the-art capabilities for probing the atmospheric boundary layer: the XPIA field campaign. Bulletin of the American Meteorological Society 98(2):289–314
  * Monchaux (2012) Monchaux R (2012) Measuring concentration with voronoï diagrams: the study of possible biases. New Journal of Physics 14(9):095,013
  * Morris et al (2007) Morris SC, Stolpa SR, Slaboch PE, et al (2007) Near-surface particle image velocimetry measurements in a transitionally rough-wall atmospheric boundary layer. Journal of Fluid Mechanics 580:319–338
  * Muller et al (2020) Muller K, Hemelrijk C, Westerweel J, et al (2020) Calibration of multiple cameras for large-scale experiments using a freely moving calibration target. Experiments in Fluids 61(1):1–12
  * Nemes et al (2017) Nemes A, Dasari T, Hong J, et al (2017) Snowflakes in the atmospheric surface layer: observation of particle–turbulence dynamics. Journal of Fluid Mechanics 814:592–613
  * Novara et al (2019) Novara M, Schanz D, Geisler R, et al (2019) Multi-exposed recordings for 3d lagrangian particle tracking with multi-pulse shake-the-box. Experiments in Fluids 60(3):1–19
  * Perks et al (2020) Perks MT, Dal Sasso SF, Hauet A, et al (2020) Towards harmonisation of image velocimetry techniques for river surface velocity observations. Earth System Science Data 12(3):1545–1559
  * Schanz et al (2016a) Schanz D, Gesemann S, Schröder A (2016a) Shake-The-Box: Lagrangian particle tracking at high particle image densities. Experiments in Fluids 57(5):1–27
  * Schanz et al (2016b) Schanz D, Huhn F, Gesemann S, et al (2016b) Towards high-resolution 3D flow field measurements at cubic meter scales. In: Proceedings of the 18th International Symposium on the Application of Laser and Imaging Techniques to Fluid Mechanics, Springer
  * Tan et al (2020) Tan S, Salibindla A, Masuk AUM, et al (2020) Introducing openlpt: new method of removing ghost particles and high-concentration particle shadow tracking. Experiments in Fluids 61(2):1–16
  * Terra et al (2019) Terra W, Sciacchitano A, Shah Y (2019) Aerodynamic drag determination of a full-scale cyclist mannequin from large-scale ptv measurements. Experiments in Fluids 60(2):1–11
  * Theriault et al (2014) Theriault DH, Fuller NW, Jackson BE, et al (2014) A protocol and calibration method for accurate multi-camera field videography. Journal of Experimental Biology 217(11):1843–1848
  * Toloui et al (2014) Toloui M, Riley S, Hong J, et al (2014) Measurement of atmospheric boundary layer based on super-large-scale particle image velocimetry using natural snowfall. Experiments in Fluids 55(5):1–14
  * Wei et al (2021) Wei NJ, Brownstein ID, Cardona JL, et al (2021) Near-wake structure of full-scale vertical-axis wind turbines. Journal of Fluid Mechanics 914
  * Wieneke (2012) Wieneke B (2012) Iterative reconstruction of volumetric particle distribution. Measurement Science and Technology 24(2):024,008
  * Zheng and Longmire (2013) Zheng S, Longmire EK (2013) Flying PIV investigation of vortex packet evolution in perturbed boundary layers. In: PIV13; 10th International Symposium on Particle Image Velocimetry, Delft, The Netherlands, July 1-3, 2013, Citeseer


# Demystify Adult Learning: A Social Network and Large Language Model Assisted
Approach

Fang Liu  Singapore University of Social Sciences
Singapore, 599494
<EMAIL_ADDRESS>Bosheng Ding  Nanyang Technological University
Singapore, 639798
<EMAIL_ADDRESS>Chong Guan∗ Zhang Wei  Singapore University of Social
Sciences
Singapore, 599494
<EMAIL_ADDRESS>Singapore Institute of Technology
Singapore, 138683
<EMAIL_ADDRESS>Dusit Niyato  Nanyang Technological University
Singapore, 639798
<EMAIL_ADDRESS>Justina Tan This work was supported in part by A*STAR
under its MTC Programmatic (Award M23L9b0052), MTC Individual Research Grants
(IRG) (Award M23M6c0113), the Ministry of Education, Singapore, under the
Academic Research Tier 1 Grant (Grant ID: GMS 693), and SIT’s Ignition Grant
(STEM) (Grant ID: IG (S) 2/2023 – 792).∗ Corresponding author. Email:
<EMAIL_ADDRESS>Singapore University of Social Sciences
Singapore, 599494
<EMAIL_ADDRESS>
###### Abstract

Adult learning is increasingly recognized as a crucial way for personal
development and societal progress. It however is challenging, and adult
learners face unique challenges such as balancing education with other life
responsibilities. Collecting feedback from adult learners is effective in
understanding their concerns and improving learning experiences, and social
networks provide a rich source of real-time sentiment data from adult
learners. Machine learning technologies especially large language models
(LLMs) perform well in automating sentiment analysis. However, none of such
models is specialized for adult learning with accurate sentiment
understanding. In this paper, we present A-Learn, which enhances adult
learning sentiment analysis by customizing existing general-purpose LLMs with
domain-specific datasets for adult learning. We collect adult learners’
comments from social networks and label the sentiment of each comment with an
existing LLM to form labelled datasets tailored for adult learning. The
datasets are used to customize A-Learn from several base LLMs. We conducted
experimental studies and the results reveal A-Learn’s competitive sentiment
analysis performance, achieving up to 91.3% accuracy with 20% improvement over
the base LLM. A-Learn is also employed for word cloud analysis to identify key
concerns of adult learners. The research outcome of this study highlights the
importance of applying machine learning with educational expertise for
teaching improvement and educational innovations that benefit adult learning
and adult learners.

###### Index Terms:

social network, large language model, generative artificial intelligence,
sentiment analysis, applied artificial intelligence, adult learning

##  I Introduction

Adult learning is essential for personal growth and societal advancement [1].
It involves the continuous acquisition of knowledge and skill throughout one’s
adulthood [2]. This process contributes to a more competent workforce and
better employment prospects, and positively affects the economy with better
job opportunities and increased productivity. It also promotes social
inclusion by supporting marginalized groups to participate more fully in
society [3]. Unlike traditional education, which is typically confined to
formal learning in childhood and adolescence, adult learning brings unique
challenges. Adult learners must balance their educational pursuits with other
adult responsibilities, which interfere with their regular attendance of
classes, timely completion of assignments, and overall focus on studies. Adult
learners may also struggle with adapting to new learning environments and
technologies, and face financial constraints [4].

Therefore, it is vital to gather feedback from adult learners and understand
their unique needs to enhance the adult learning experience [5].
Traditionally, educators use surveys, in-class discussions, and course
evaluation forms to collect feedback, but these methods may yield limited and
potentially biased responses due to low participation rates. To address this,
researchers are increasingly turning to social networks where a large volume
of real-time comments from the learners are available. Analyzing such data
allows for examining the social impacts of adult learning and offering
insights into learners’ real-time opinions, experiences, and sentiments [6].

However, the vast and ever-increasing volume of data from social networks
makes traditional analysis, often manual, impractical. To overcome this
challenge, researchers have studied machine learning (ML) models to automate
data processing and analysis. Within ML, natural language processing (NLP) is
particularly useful for analyzing social network data, where text is the
dominant data modality. An important application of NLP is sentiment analysis,
which accesses the emotional tone behind comments to infer sentiments such as
Positive, Neutral, or Negative. NLP excels in such tasks, and recent
advancements have led to the development of many large language models (LLMs)
that are trained on huge text datasets for enhanced performance in sentiment
analysis. Some LLMs are specifically tailored for certain tasks and platforms,
e.g., Twitter-RoBERTa [7], trained on 58 million tweets, is highly effective
for various tasks including sentiment analysis for Twitter comments.

Unfortunately, even advanced LLMs may not perform optimally across all
subjects due to the broad nature of training data which may not cover
specialized areas comprehensively. While an LLM might demonstrate high
accuracy in specific applications, there is a risk of over-fitting, where the
model cannot generalize well to new data. This leads to a significant increase
in error rates when the model is applied to unfamiliar data. Additionally, the
constant evolution of language, including the emergence of new buzzwords and
acronyms commonly used by students on social media, presents a continuous
challenge for pre-trained LLMs which may struggle to adapt to these changes.

In this paper, we propose a customized LLM tailored to adult learning which is
crucial for effective analysis in this field. Utilizing a specialized dataset
for adult learning is essential for providing domain-specific knowledge to the
models and refining the performance on the targeted topics. Accurate labelling
of this specialized dataset is important for extracting meaningful insights.
However, sourcing such accurately labeled datasets presents a significant
challenge for adult learning research. To address this, we introduce a system
called A-Learn for adult learning sentiment analysis. A-Learn utilizes social
network comments enhanced with LLM and accordingly customizes LLM models for
adult learning. We outline our main contributions as follows.

  * •

We collected an adult learning dataset from multiple social network platforms
with over 55,000 comments from current and prospective adult learners;

  * •

We generated two additional datasets: a large-scale dataset labelled by LLM
for model fine-tuning and customization, and a small-scale dataset labelled
manually by human experts for performance evaluation;

  * •

We developed A-Learn to incorporate adult learning knowledge into existing
LLMs with optimized training and produce LLMs customized for adult learning;

  * •

We conducted experimental studies and demonstrated A-Learn’s competitive
performance in sentiment analysis for adult learning, with up to 91.3%
accuracy and a 20% improvement over the baseline.

The rest of the paper is organized as follows. The system architecture of
A-Learn is presented in Section II. Section III describes our methodology in
detail and Section IV presents an experimental study of the proposed solution
as well as result discussions. Finally, we conclude the paper in Section V.

##  II System Architecture

In this section, we present the system architecture of A-Learn and an
illustration of it is available in Fig. 1.

Figure 1: The system architecture of A-Learn for adult learning sentiment
analysis. A-Learn is ML-based and processes the social network comments as
input to generate sentiment labels as output. A-Learn’s sentiment analysis
plays a crucial role in identifying key concerns and facilitating educational
innovations in adult learning.

###  II-A LLM for Sentiment Analysis

The sentiment analysis of adult learning can be conducted in different ways
and LLM is one of the effective solutions to automatic the analysis. In a
typical scenario, an LLM takes student comments (e.g., from social media and
questionnaires) as input. The data is pre-processed such as modality alignment
before the ML analysis, where sentiment analysis is treated as a
classification problem and the model’s role is to generate sentiment labels
including Positive for satisfaction, Negative for dissatisfaction, and Neutral
otherwise. The sentiment labels allow for the grouping of comments, where in-
depth investigation can be performed for each group to gain a deep
understanding of adult learners for teaching quality enhancement and
educational innovation.

###  II-B LLM Customization for Adult Learning

Despite being a promising technology, existing LLM is not specialized for
adult learning. There are significant differences between general-purpose text
analysis and adult learning sentiments, where some phrases, acronyms, etc.,
are not common in generic settings. This calls for a customization, or domain
adaptation, of existing LLM to adult learning scenarios. In A-Learn, we first
collect an adult learning dataset from various social media platforms to
capture the adult learning concerns in different aspects. Yet, no one has
indicated the sentiment of any comments. Thus, we apply an existing LLM,
ChatGPT in specific, to label the comments in the dataset. The labels may not
be fully correct from the views of human experts, but they are cheap to obtain
and very likely to be correct. Now, with the dataset specialized for adult
learning as well as associated sentiment labels, we customize the generic LLMs
to adapt to adult learning with accurate sentiment analysis. This is a typical
transfer learning concept, which has been verified in other applications [8].
We expect the transfer or customization, to be effective with our specialized
dataset and training strategies.

##  III Methodology

We present the detailed methodology of A-Learn in this section, for both data
collection and knowledge adaptation.

(a) Mixed Sentiment

(b) Negative Sentiment

Figure 2: Sample comments with different emotions/sentiments about time
management and teaching quality for adult learning.

###  III-A LLM-assisted Sentiment Label Acquisition

Adult learning as a specific topic of sentiment analysis involves unique
patterns that are not common in generic settings. Frequent phrases such as
time management, work-life balance, teaching quality, and resource allocation,
may not be well modeled in general-purpose LLMs. Besides, the sentiment can be
complicated with mixed emotions. We show one sample comment in Fig. 2(a),
where a student expresses enthusiasm for helping peers, but feels struggles
when additional demands arise and affect his/her own learning progress. Fig.
2(b) presents a sample comment with negative sentiment, featuring network
slang and emojis, which add analysis complexity.

To understand the unique patterns of adult learning, we consider social
networks that offer comprehensive information regarding student comments about
adult learning. However as mentioned above, we lack sentiment labels of the
comments and face challenges to interpret the data and extract insights. As
such, we introduce a labelling process, which shall not be fully manual
considering the labelling complexity and cost. We follow a hybrid approach
with the assistance of both LLM and human experts. We task ChatGPT to label
comments with sentiment and let human experts verify the labels, assuming
verification is much easier and faster than labelling. According to recent
studies [9, 10] for other applications, such an approach can be cost-efficient
with big time-savings.

###  III-B A-Learn with LLM Customization

It takes a huge dataset to train an LLM to achieve optimal performance, but a
huge dataset is often very costly to obtain. This is no difference for adult
learning and we shall choose the data-lite strategies. We first adopt pre-
trained LLMs with optimal performance for understanding generic text content.
Each LLM consists of many parameters and the optimal values for generic usage
are known and downloadable for these LLMs. We take such optimal values as the
starting point for customizing the parameters to adult learning. We adjust and
optimize the values as a model re-training or fine-tuning process, and our
ultimate goal is to adopt the parameter values for general-purpose text
analysis to the domain of adult learning. This process is illustrated in Fig.
3. LLM tokenizes the input text and converts them into numerical
representations, which are then fed into the transformer encoder for language
comprehension. We follow transfer learning to re-train or fine-tune the models
with the adult learning dataset. This process adjusts the weights of various
layers in the transformer architecture. Specifically, we can freeze some
layers near the input and fine-tune the rest layers near the output. The more
layers we freeze, the more information from the pre-trained model is retained.
Conversely, adjusting more layers allows the penetration of more domain-
specific information, which however demands more data to perform well. The
output from the transformer encoding is passed to a classifier, which produces
the final three-fold sentiment labels, Positive, Negative, and Neutral.

Let $\mathcal{M}(x;\theta)$ be the sentiment analysis modal which analyzes the
comment $x$ and determines its sentiment label as $y=\mathcal{M}(x;\theta)$
with model parameters $\theta$. We assume two connected stages of the whole
model as $(\mathcal{M}_{\text{f}},\mathcal{M}_{\text{t}})$ where
$\mathcal{M}_{\text{f}}$ and $\mathcal{M}_{\text{t}}$ are the sets of frozen
and trainable layers, respectively, along with the corresponding model
parameters $(\theta_{\text{f}},\theta_{\text{t}})$. In A-Learn, we initialize
the parameters as $\theta_{\text{f}}^{0}$ and $\theta_{\text{t}}^{0}$ which
are copied from the optimal parameter values from the base LLMs. The aim of
customization is to find the optimal parameter values $\theta_{\text{t}}^{*}$
for adult learning sentiment analysis as,

$\theta_{\text{t}}^{*}=\arg\min_{\theta_{\text{t}}}\mathcal{L}(\mathcal{D};\theta_{\text{f}}^{0},\theta_{\text{t}}),$
(1)

where $\mathcal{L}(\cdot)$ is the cross-entropy loss function, $\mathcal{D}$
is our labelled dataset, and $\theta_{\text{f}}$ remains unchanged as
$\theta_{\text{f}}^{0}$. The final model of A-Learn is
$\mathcal{M}(\cdot;\theta^{*})$ where
$\theta^{*}=(\theta_{\text{f}}^{0},\theta_{\text{t}}^{*})$. Specifically, re-
training means all transformer layers are trainable where
$\mathcal{M}_{\text{f}}$ is null and the optimal model is
$\mathcal{M}(\cdot;\theta^{*})$ where $\theta^{*}=\theta_{\text{t}}^{*}$. With
comments and A-Learn’s generated labels, educators can perform in-depth
analysis for improved teaching.

Figure 3: An illustration of A-Learn with social network comments as the
input. Domain knowledge is extracted from the comments and represented as
sentiment initiates, used to fine-tune different layers in LLM’s transformer
encoder. The customized model can identify the sentiment of a comment.

##  IV Experimental Study

We present our experimental study in this section to demonstrate A-Learn’s
sentiment analysis performance for adult learning and interpret the experiment
results to find insights. A-Learn is data-driven and we start with data
description.

###  IV-A Dataset and Data Pre-processing

In this part, we first introduce the data collection for our experiments,
followed by data pre-processing descriptions.

####  IV-A1 Data Collection

Social networks are popular among various groups of users. It provides rich
and extensive data that facilitates in-depth analysis of human opinions and
sentiments. In A-Learn, we scrap plenty of comments related to adult learning
from multiple social media platforms, including Reddit, Twitter, and Tumblr.
They are selected because of their diverse user demographics (e.g. various age
groups and backgrounds) and well-supported APIs. Our comment search is based
on keywords such as adult learning, part-time studies, continuous learning,
and work study. In total, we have 31,724 Reddit comments collected from 2017
to 2023, 22,270 Twitter tweets in 2023, and 1,043 Tumblr comments from 2011 to
2023.

####  IV-A2 Data Pre-processing

The raw data (comments) collected from the above stage is not ready for ML
models and necessary pre-processing is needed to facilitate the ML-based
analysis. Firstly, we remove duplicated and incomplete content. Then, we scan
the content and eliminate special text components, including HTML links and
tags, emojis, and newline characters. All contents are normalized and
converted to lowercase to ensure consistency. Finally, the contents are
processed through tokenization, lemmatization, and stop-word removal to
prepare them to be compatible with ML.

####  IV-A3 Datasets for Experiments

We present two datasets specialized for adult learning with different
labelling schemes.

##### LLM Labels for Training

We use LLM to generate a relatively large-scale labelled dataset.
Particularly, we use GPT-3.5 and get its opinion about the sentiment of each
of the 1,000 randomly selected social network comments. Some labels are
randomly selected for manual inspection by human experts, to evaluate if the
labelling quality meets our expectations. Note that we do not expect fully
correct labels in A-Learn training.

##### Expert Labels for Testing

We need fully correct labels for performance evaluation. Thus we introduce
human expert support. We randomly select comments from our collected dataset.
We let one human expert read each comment and assign a sentiment label to it,
and the label is verified by two other human experts. Only if the three
experts reach a consensus, we include the comment along with its label into
the testing dataset. We stop the labelling once we have 300 labelled comments
with one-third for each category of sentiment. This will be our testing
dataset.

###  IV-B A-Learn Performance Analysis

A-Learn considers several widely used or latest LLMs as base models and then
customizes them with our labelled adult learning dataset. We first present the
experimental setup.

####  IV-B1 Experimental Setup

We use Google Colab to implement A-Learn. For the base model, we select the
widely used BERT [11] and five of its variants, including ALBERT [12], RoBERTa
[13], DistilBERT, DistilBERT-SST2 [14], and Twitter-RoBERTa [7]. Here,
DistilBERT-SST2 and Twitter-RoBERTa are two specialized models trained with
the SST2 dataset (a benchmark for sentiment analysis) and a Twitter dataset,
respectively. Some models have different versions and we choose the default
one, normally with a suffix of -base, which is excluded in this paper for
simplicity.

In the following part, we use Base as a reference of the original LLMs and
A-Learn is the LLM customized for adult learning. A-Learn is fine-tuned with
the LLM-labelled dataset for at least ten epochs. Accuracy, in the percentage
of the number of correct predictions to the total number of predictions, is
used as the main performance metric based on the testing results on expert-
labelled dataset. All experimental procedures are executed on NVIDIA V100
GPUs.

TABLE I: The performances in accuracy (%) and training time in minute of Base models and A-Learn customized from the models. A-Learn always achieves better accuracy compared to the corresponding Base models with significant improvement and manageable time usage for training. Model Name | Base | A-Learn | Improvment | Time
---|---|---|---|---
Bert | 41.0 | 61.7 | 20.7 | 11
ALBERT | 33.0 | 35.7 | 2.7 | 10
RoBERTa | 32.0 | 56.3 | 24.3 | 12
Distilbert | 36.0 | 59.0 | 23.0 | 6
Distilbert-SST2 | 44.0 | 59.3 | 15.3 | 7
Twitter-RoBERTa | 76.3 | 91.3 | 15.0 | 12

####  IV-B2 Sentiment Analysis Performance

We evaluate A-Learn’s performance on improving pre-trained Base models for
adult learning sentiment analysis. The statistical results of our experiments
are presented in Table I. The results demonstrate a consistent and substantial
accuracy improvement of A-Learn with six different models after model
customization with domain-specific data. The accuracy improvement ranges from
2.7% to 24.3% and the average is 16.8%. This validates our hypothesis that
incorporating domain-specific knowledge into existing models can boost the
model’s performance in adult learning sentiment analysis. Notably, A-Learn
based on Twitter-RoBERTa reaches an accuracy of 91.3%, with a big increase
from 76.3% when Twitter-RoBERTa is not customized. This underscores the
importance of customizing the base models with the data of a similar format
(e.g., from Twitter) for sentiment analysis.

A-Learn is also efficient. The training time is about ten minutes for all
tested LLMs. Compared with training a general-purpose model which takes days
or even weeks, A-Learn is much faster. One reason is that A-Learn starts its
training with the optimal model parameters for generic applications that share
similarities with adult learning. This leads to reduced training difficulty
and accelerated training.

###  IV-C Sensitivity Analysis

In this section, we explore different A-Learn settings and identify the
optimal configuration of A-Learn.

####  IV-C1 Number of Trainable Layers

We have proved that customization is effective for adult learning sentiment
analysis. In this part, we further investigate how to customize the knowledge
to achieve optimal performance. We take the best-performing model, Twitter-
RoBERTa, as the Base model and we conduct sensitivity analysis. We examine
A-Learn’s performance sensitivity to the number of trainable layers, and we
show the results in Table II. This analysis aims to identify the ideal
configuration of the trainable layers to reach the best performance of adult
learning sentiment analysis.

Specifically, we investigate the transformer stage of Twitter-RoBERTa, which
comprises 12 layers. Generally, the layers closer to the model output are more
application-specific, and should be trainable to adapt to our application. In
our experiment, let $n$ be the number of trainable layers nearest to the model
output, while the remaining layers are frozen. For example, $n=0$ refers to
the Base model where no layers is trainable. In A-Learn, we explore different
settings of $n$ ranging from 1 to 12. Due to the limit of space, we report the
representative ones with the minimum $n=1$ layer, about half of the layers
with $n=7$, and all the layers with $n=12$.

The results show an increase in accuracy as the number of trainable layers $n$
increases, reaching a peak accuracy of 91.3% when all layers are trainable.
This trend suggests that limiting the number of trainable layers can
negatively impact the model’s ability to adapt to a specific task, potentially
due to insufficient adjustment to the unique patterns of our task. Notably,
the accuracy drops to 72% when $n=1$, even lower than the performance of Base.
This implies insufficient domain adaptation for the new application, while the
analysis of the original model is disrupted with model changes. In conclusion,
allowing more layers to be trainable helps A-Learn better adapt to the
sentiment analysis application of adult learning.

TABLE II: Performance sensitivity of A-Learn to different numbers of trainable layers in terms of accuracy (%). The improvement column refers to the A-Learn’s accuracy improvement in percentage over the base model. Model | No. of Trainable Layers | Accuracy | Improvement
---|---|---|---
Base | 0 (0%) | 76.3 | $-$
A-Learn | 1 (8%) | 72.0 | -5.6
7 (58%) | 83.7 | 9.7
12 (100%) | 91.3 | 19.7

####  IV-C2 Different LLM-assisted Data

In sentiment analysis, there are two primary approaches using LLMs for data
labelling: 1) assign sentiment labels to comments, and 2) generate comments
given specific labels. Let a comment be $x$. The role of LLM in the first
approach is to generate $\hat{y}=\text{LLM}(x)$ where $x$ is the input (real
data) of the LLM and $\hat{y}$ is LLM’s inferred sentiment label (not ground-
truth). We represent this approach as $x\rightarrow\hat{y}$. For the second
approach, represented by $y\rightarrow\hat{x}$, we provide the sentiment label
$y$ (without adding any comments), and let the model generate a list of
comments corresponding to the label, i.e., $\hat{x}=\text{LLM}(y)$ for each
synthetic comment $\hat{x}$. We select Twitter-RoBERTa as the Base and report
A-Learn’s performance in Table III.

Our default setting of LLM-assisted labelling is shown in the last line of
$y\rightarrow\hat{x}$, where A-Learn is 91.3% accurate. Compared with the base
model without customization, it is nearly 20% more accurate. With another
scheme $y\rightarrow\hat{x}$, the accuracy declines compared to the base
model, by 15.6% with 64.3% sentiment analysis accuracy. A potential reason is
that social network comments usually are complex and the texts are often
unstructured and informal. There are also new and non-common phrases with the
use of slang, abbreviations, and various languages or dialects. Generating
synthetic comments $\hat{x}$ purely from labels $y$ with LLM is challenging.
We notice that LLM tends to standardize the output comments with similar
language styles, length, etc., where the complexity of the real comments is
not well captured. As a result, customization with such data does not lead to
improved performance for A-Learn, and even worse, the accuracy drops. We
assume that the dataset fails to bring extra value in adult learning to
A-Learn and meanwhile disrupts the base model with optimal performance for
general-purpose applications.

In summary, our experiments demonstrate that the LLM-labelled dataset is
useful for customizing the general-purpose LLM to adult learning with
optimized configurations. This aligns with the findings from previous works
[9] that tagging unlabeled data using LLMs is an effective method compared to
traditional data collection schemes. This is especially true for our
application where the comments own unique features specialized to adult
learning.

TABLE III: The performances in accuracy (%) of Base models and A-Learn for sentiment analysis with different labelling methods. The improvement in percentage is based on the comparison with Base model. Customization Data | Accuracy | Improvement (%)
---|---|---
Base | 76.3 | -
LLM $y\rightarrow\hat{x}$ | 64.3 | -15.6
LLM $x\rightarrow\hat{y}$ (ours) | 91.3 | 19.7

###  IV-D A-Learn-enabled Insights Visualization

In this part, we apply A-Learn in its optimal configuration for analyzing the
comments collected from all three platforms, Reddit, Twitter, and Tumblr. We
apply A-Learn on the collected comments that are not used in neither the
training nor testing dataset of small-scale, meaning most of the comments
collected are used in this part. After A-Learn processing, all these comments
are assigned with a sentiment label. We pick out the comments with the same
label, specifically Positive and Negative. For each group of comments with the
same label, we conduct a word cloud analysis to visualize the most frequent
words for each sentiment of adult learning. We show the word cloud figures in
Fig. 4 where the words with limited linguistic information like stop words are
excluded.

####  IV-D1 Negative

Negative sentiment is important as it often sparks insights into teaching
improvement and educational innovation. As seen from the figure, negative
comments are largely personal with domestic challenges. Unlike traditional
learners, adult learners must balance their studies with family and work
responsibilities. This is evidenced by the highlighted words such as time job,
family vacation, family time, etc. Additionally, adult learners often lack
frequent interactions with peers and instructors and spend much of their study
time home alone. Older students may face difficulties related to their age and
physical fitness, indicated by terms such as old enough, while younger
students may experience pressure related to their first job and financial
obligations such as make money.

####  IV-D2 Positive

Positive sentiment is related to professional development and advice for
effective learning. For instance, adult learners emphasize the need for work
smarter and work hard at their jobs, to free up time for adult learning. This
highlights the continued importance of balancing work and study. Adult
learning shall be persistent, with phrases every day. Interestingly, utilizing
online resources and open-sourced tools such as khan academy seems to be
effective in enhancing adult learning. Overall, A-Learn based word clouds for
different sentiments can shed light on teaching improvement, policy
adjustment, etc., with special attention on adult learning rather than
traditional learning.

Figure 4: The word clouds with highlighted adult learning concerns for both
negative and positive sentiments. Larger words appear more frequently in the
comments and imply significant concerns of adult learners.

##  V Conclusion

In this paper, we introduced A-Learn to demystify the sentiment of adult
learners by analyzing their social network comments. A-Learn leveraged
advanced LLMs and was tailored for adult learning. It can be based on
different LLMs and several of them were investigated in this paper. A-Learn
customizes the base LLMs, often general-purposed, with adult learning dataset
extracted from social networks and labelled by an LLM. Experiment results
demonstrated that customizing existing LLMs with domain-specific data is
effective, and A-Learn can achieve high accuracy in sentiment analysis in
adult learning scenarios. Specifically, A-Learn achieves its peak performance
when Twitter-RoBERTa is used as its Base model. Its accuracy can be up to
91.3%, surpassing the non-customized base model by 19.7%. Besdies,
customization itself demands an optimized process, and we identified the
optimal configuration of A-Learn training through sensitivity studies.
Overall, this is an interdisciplinary research combining the strengths of ML
and social science. The research outcome holds substantial implications for
continuing education and insights for better-supporting adult learners.

## Acknowledgement

The authors acknowledge the contribution of the undergraduate students from
the Singapore Institute of Technology for preliminary study in integrative
team projects in the AY22/23.

## References

  * [1] M. Tennant, _Psychology and adult learning_. Routledge, 2007.
  * [2] J. Mezirow, _Transformative dimensions of adult learning._ ERIC, 1991.
  * [3] R. Desjardins and A. Ioannidou, “The political economy of adult learning systems—some institutional features that promote adult learning participation,” _Zeitschrift für Weiterbildungsforschung_ , vol. 43, no. 2, pp. 143–168, 2020.
  * [4] C. Guan, J. Mou, and Z. Jiang, “Artificial intelligence innovation in education: A twenty-year data-driven historical analysis,” _International Journal of Innovation Studies_ , vol. 4, no. 4, pp. 134–147, 2020.
  * [5] T. Shaik, X. Tao, Y. Li _et al._ , “A review of the trends and challenges in adopting natural language processing methods for education feedback analysis,” _IEEE Access_ , vol. 10, pp. 56 720–56 739, 2022.
  * [6] K. D. Philp and M. G. Gill, “Brokering youth learning opportunities in after-school settings: The role of adult social networks and network orientations.” _Translational Issues in Psychological Science_ , vol. 8, no. 2, p. 257, 2022.
  * [7] U. Sirisha and S. C. Bolem, “Aspect based sentiment & emotion analysis with roberta, lstm,” _International Journal of Advanced Computer Science and Applications_ , vol. 13, no. 11, 2022.
  * [8] F. Liu, W. Zhang, I. Atmosukarto _et al._ , “Transline: transfer learning for accurate and explainable power line anomaly detection with insufficient data,” _CCF Transactions on Pervasive Computing and Interaction_ , vol. 5, no. 3, pp. 241–254, 2023.
  * [9] B. Ding, C. Qin, L. Liu _et al._ , “Is gpt-3 a good data annotator?” in _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2023, pp. 11 173–11 195.
  * [10] M. Li, T. Shi, C. Ziems _et al._ , “Coannotating: Uncertainty-guided work allocation between human and large language models for data annotation,” in _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , 2023, pp. 1487–1505.
  * [11] J. D. M.-W. C. Kenton and L. K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of NAACL-HLT_ , 2019, pp. 4171–4186.
  * [12] Z. Lan, M. Chen, S. Goodman _et al._ , “Albert: A lite bert for self-supervised learning of language representations,” in _International Conference on Learning Representations_ , 2019.
  * [13] Y. Liu, M. Ott, N. Goyal _et al._ , “Roberta: A robustly optimized bert pretraining approach,” _arXiv preprint arXiv:1907.11692_ , 2019.
  * [14] V. Sanh, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.” in _Proceedings of Thirty-third Conference on Neural Information Processing Systems (NIPS2019)_ , 2019.