11institutetext: Saint Louis University, 1 N. Grand Blvd., St. Louis, MO 63103
USA 11email<EMAIL_ADDRESS>

# Deployment of NLP and LLM Techniques to Control Mobile Robots at the Edge: A
Case Study Using GPT-4-Turbo and LLaMA 2

Pascal Sikorski Authors with equal contribution 0009-0007-4977-3863 Leendert
Schrader ** 0009-0003-1571-0283 Kaleb Yu ** Lucy Billadeau  Jinka Meenakshi
0009-0009-9695-8099 Naveena Mutharasan  0009-0001-4832-6419 Flavio Esposito
0000-0002-7798-4584 Hadi AliAkbarpour  0000-0002-3539-2302 Madi Babaiasl
0000-0002-3652-5795

###### Abstract

This paper investigates the possibility of intuitive human-robot interaction
through the application of Natural Language Processing (NLP) and Large
Language Models (LLMs) in mobile robotics. We aim to explore the feasibility
of using these technologies for edge-based deployment, where traditional cloud
dependencies are eliminated. The study specifically contrasts the performance
of GPT-4-Turbo, which requires cloud connectivity, with an offline-capable,
quantized version of LLaMA 2 (LLaMA 2-7B.Q5 K M). Our results show that
GPT-4-Turbo delivers superior performance in interpreting and executing
complex commands accurately, whereas LLaMA 2 exhibits significant limitations
in consistency and reliability of command execution. Communication between the
control computer and the mobile robot is established via a Raspberry Pi Pico
W, which wirelessly receives commands from the computer without internet
dependency and transmits them through a wired connection to the robot’s
Arduino controller. This study highlights the potential and challenges of
implementing LLMs and NLP at the edge, providing groundwork for future
research into fully autonomous and network-independent robotic systems. For
video demonstrations and source code, please refer to:
https://tinyurl.com/RobocupSym2024.

###### Keywords:

Human-robot Interaction Wheeled Mobile Robots Edge Computing.

##  1 Introduction

###  1.1 Context and Background

In recent years, the integration of robotics and artificial intelligence (AI)
has started a new era in automation and intelligent systems [9]. Mobile
robotics, influenced by this integration, has evolved from simple navigational
tasks to complex interaction scenarios involving dynamic decision-making
capabilities [13]. On the other hand, significant strides in natural language
processing (NLP) and large language models (LLMs) have transformed how
machines understand and respond to human language [15].

NLP technologies have traditionally relied on cloud-based infrastructures due
to their substantial computational requirements [2]. However, there is a
growing shift towards deploying these technologies on edge computing platforms
[2]. This transition addresses growing concerns about privacy [10] and the
limitations posed by intermittent or unreliable internet connectivity, which
are particularly critical in rescue operations or assistive technologies [6].
By processing data locally, edge computing significantly reduces latency and
enhances data security [1], meeting the urgent needs of real-time robotic
processing.

Integrating Language models and NLP in robotics can facilitate the robot’s
ability to interact fluidly with humans, learn from verbal instructions, and
execute complex tasks [15]. This involves not only understanding spoken
commands but also contextualizing actions and outcomes within a physical
environment. Furthermore, the use of language models in an offline edge
environment addresses significant concerns regarding data security and
operational reliability in sensitive or unpredictable environments where
constant internet connectivity is unfeasible [1]. This approach also mitigates
issues of latency which are critical in scenarios where real-time robotics
control is required, such as in navigation, obstacle avoidance, and
interactive tasks.

The case study presented in this paper explores the practical deployment of
LLMs and NLP in controlling a mobile robot and investigates the feasibility of
using state-of-the-art LLMs on edge devices. Through this study, we aim to
provide insights into the complexities and solutions associated with
implementing NLP and LLMs in mobile robotics, and the feasibility of using
them in edge devices paving the way for future research in these areas.

###  1.2 Problem Statement

Enhancing human-robot interaction through LLM and NLP poses substantial
challenges, particularly in scenarios requiring robust privacy measures such
as assistive robotics and high-stake decision-making, such as rescue
operations. Currently, most advanced robotic systems rely heavily on cloud
computing for processing complex tasks [11], which introduces significant
vulnerabilities related to latency and data privacy. Furthermore, the
dependency on continuous internet connectivity can hinder operational
capabilities in disaster-stricken or remote areas. This study investigates a
first step to a solution for intuitive human-robot interaction without the
need for continuous cloud connectivity.

###  1.3 Research Objectives

The first objective of this research is to study the feasibility of deploying
LLMs and NLP for intuitive human-robot interaction within the context of
mobile robots. The second objective is to investigate the potential and
outline the limitations of utilizing such models in scenarios where cloud
connectivity is compromised or entirely unavailable moving controlling mobile
robots towards self-sufficient systems without relying on remote servers.

###  1.4 Related Work

The integration of LLMs and NLP into robotic systems has been a new and
exciting field of research that aims to enhance robotic understanding and
interaction capabilities in complex environments [15]. The rise of LLMs like
GPT (Generative Pre-trained Transformer) [7] further enhanced the potential of
NLP in robotics [12, 14]. These models, trained on vast amounts of text data,
can understand and generate human-like text, allowing for more nuanced and
complex interactions between humans and robots. Different research groups have
started to use LLMs to control robots and autonomous vehicles [8]. Liu et. al.
[5] proposed a human-robot collaboration framework based on LLM for
manipulation tasks. Sharan et. al. [8] proposed to use the common-sense
reasoning capabilities of LLMs to generate plans for self-driving vehicles.

Despite these promising research developments, a significant challenge that
persists is the substantial computing resources required by LLMs, which can
impede their integration into mobile and resource-constrained robotic
platforms. On the other hand, recent studies showed that LLMs can be pruned by
up to 40% of their original size without a noticeable reduction in performance
accuracy [4]. This suggests a potential pathway for their integration into
more compact systems and edge devices to be used in mobile robots.

Our current ongoing research aims to build on this research by exploring the
potential for implementing LLMs and NLP into edge devices to control mobile
robots. This paper marks the first step toward integrating NLP technologies
with autonomous robotic functionalities on edge platforms. Popular language
models such as GPT have provided online API access allowing developers to
integrate their LLM capabilities into customized applications. These, however,
require the user to have steady access to OpenAI’s servers. This can cause
issues regarding network reliability and privacy. Offline LLMs however, could
alleviate those concerns as we will process the outputs entirely on a local
device. Additionally in this paper, to show the feasibility of controlling
mobile robots using LLM and NLP, we demonstrate how by using an edge device,
we are able to execute robotic commands. Through the power of LLMs, we can
convert our natural speech into a format that we then process locally on the
mobile robot.

##  2 Methods

###  2.1 System Overview

Figure 1: The schematic of the integrated mobile robot system architecture
showing the workflow from voice input to robot action, including offline
speech recognition, language understanding modules, and task execution via
Raspberry Pi Pico W and Arduino.

The core architecture of our system integrates a mobile robot with language
processing capabilities, enabling it to perform tasks based on verbal
commands. We also investigated the feasibility of edge-based deployment of
LLMs, which allows for real-time processing without the need for cloud
connectivity (See figure 1). This system includes the following components:
Speech recognition module An offline speech recognition module named VOSK by
Alpha Cephei that we used to convert spoken language into text. Language
understanding module We utilized and compared OpenAI’s GPT-4-Turbo and a
quantized version of Meta’s LLaMa 2-7B called LLaMA 2-7B.Q5_K_M. We intend to
compare and contrast the differences between these models based on their
accuracy of speech interpretation. The key difference between the two models
is that LLaMa 2-7B model is running offline on a machine and does not require
network access to generate responses. This is a significant reason to use
these models as we will make the processing of the user’s speech entirely
local. These models process the transcribed text from the speech recognition
software into appropriate tasks for the robot. This goes wirelessly to the
Task execution module which translates the tasks into actionable commands that
control the robot’s hardware (motors, and sensors) to interact with its
environment.

###  2.2 Experimental Setup

Figure 2: Experimental setup. A Dell Precision 3660 Tower serves as the
computing device running Python on Ubuntu 22.04 to process speech commands
captured by a Blue Yeti X USB Microphone. The processed commands are
wirelessly sent to a Raspberry Pi Pico W, which relays them to the robot via a
Serial connection. The robot is controlled through its modified Arduino Uno
board. The whole setup demonstrates the feasibility of edge-based command
execution in mobile robotics.

Pertaining to our hardware, we utilized a low-cost wheeled mobile robot named
ELEGOO Smart Car V4.0 [3]. The reason we chose this particular robot was its
balance of affordability and a set of features conducive to our research.
Though, in attempting to establish WiFi connectivity with the Smart Car’s
proprietary Arduino Uno plus I/O Expansion Board setup, we found that direct
Wifi capabilities were highly restricted by the provided hardware, and found
it best to outsource this connectivity into a Raspberry Pi Pico W
microcontroller. Therefore, the flow of information was as follows: through
WiFi, the user controller (desktop computer running Python on Ubuntu 22.04)
sent commands to the Raspberry Pi Pico W over a WiFi server established from
the Pico W, then the Pico W transmitted said commands through a UART Serial
connection of the board and the mobile robot’s Arduino Uno Board. The Pico W
was powered by an external 6V battery pack, with a common ground being
established between the two microcontrollers by way of an available grounding
pin on the robot’s I/O Expansion Board. The computer used is a Dell Precision
3660 Tower Core i7 with 32GB DDR5 up to 4400MHz UDIMM non-ECC memory and
Nvidia RTXA4000 GPU. For audio capture, a Blue Yeti X USB Microphone is used
to ensure high-quality input for the speech-to-text conversion process. Figure
2 shows the experimental setup.

###  2.3 Mobile Robot Calibration

####  2.3.1 Motion Calibration of the Wheeled Mobile Robot ‐ Forward,
Backward, Turning, and Stopping:

Effective motion calibration is crucial for the autonomous operation of the
robot. Our calibration process addresses the four fundamental movements:
forward, backward, turning (left and right), and stopping. The wheeled robot
that we have has 4 wheels. The right wheels are paired to operate together, as
are the left wheels. The robot is equipped with a TB6612FNG motor driver that
can control two DC motors. Therefore, one channel of the motor driver controls
the right wheels and the other channel controls the left wheels (Figure 3 \-
a).

Figure 3: Diagram illustrating the fundamental motions for the wheeled mobile
robot with differential steering: Motor Driver Channels (a), Moving Forward
(b), Moving Backward (c), Turning Left (d), and Turning Right (e).

To move forward, both pairs of wheels (left and right) are driven forward,
meaning both motors are activated to move in a direction that pushes the car
forward (Figure 3 \- b). Conversely, to move backward, the motors are
activated in the opposite direction, pulling the car backward (Figure 3 \- c).
This is achieved by setting the motor driver input pins (IN1 and IN2 for one
set of wheels, IN3 and IN4 for the other set) to the appropriate logic levels
as specified in Table 1.

For executing turns with a mobile robot, the basic principle involves creating
a differential in wheel speeds or directions between the left and right sides
of the vehicle. This differential steering mechanism is what allows the robot
to navigate left or right turns. Here, we implemented left and right turns for
the robot by changing the direction in which each set of wheels moves. For the
robot to turn left, we control the wheels such that the left wheels move
backward and the right wheels move forward (Figure 3 \- d). For the robot to
turn right, the opposite action is taken meaning that the left wheels move
forward and the right wheels move backward (Figure 3 \- e). The logic levels
for the motor driver input pins are specified in Table 1.

Table 1: Motor Driver Input Logic for Directional Movement Control of the Mobile Robot Right Wheels | Left Wheels | Robot Movement | IN1 | IN2 | IN3 | IN4
---|---|---|---|---|---|---
FW | FW | Forward | 1 | 0 | 1 | 0
BW | BW | Backward | 0 | 1 | 0 | 1
BW | FW | Right Turn | 0 | 1 | 1 | 0
FW | BW | Left Turn | 1 | 0 | 0 | 1

To stop the robot, both the right wheels and left wheels should be stopped,
and this can be done by setting the logic level of the PWM (Pulse Width
Modulation) pins of the motor driver to 0.

For linear Motion Calibration (Forward and Backward), we programmed the robot
to traverse a fixed track at different PWM outputs to capture the baseline
performance data. Distances traveled over incremental time intervals were
recorded. We then fit a trend line to the distance-time data for each PWM
value and calculated the corresponding linear speed from the slope of the
line. We then find the best fit for the PWM-speed data to find the
relationship between speed and the corresponding PWM that we should write to
PWM pins to achieve the desired speed. This relationship was modeled by
fitting a second-degree polynomial to the speed and PWM data, resulting in an
equation that allows us to predict the necessary PWM value for a given speed.
The polynomial fit for the PWM as a function of speed for the forward movement
is given by:

$\text{PWM}=-0.0264(\text{Speed})^{2}+5.4266(\text{Speed})-35.889$ (1)

A similar equation can be found for the backward movement, and then the time
that the motors should stay on can be calculated as follows:

$\text{Time}=\frac{\text{Distance}}{\text{Speed}}$ (2)

Following the methodology used for linear motion calibration, we applied a
similar process to angular motion calibration (turning maneuvers). The goal
was to characterize the relationship between PWM outputs and angular speeds
(degrees per second) as the robot rotates in place either to the right or
left. For data collection, the robot was programmed to rotate in place at
different PWM outputs. The angles turned were recorded over specified time
intervals. After that, for each PWM setting, we plotted the angle turned
against time and calculated the angular speed from the slope of the trend line
for each time interval. Finally, we fitted a polynomial of order two to the
angular speed as a function of PWM. This model helps in predicting the PWM
value required to achieve a desired angular speed.

The equation predicting the amount of PWM needed for motors based on the
angular speed (deg/sec) is:

$\text{PWM}=0.001(\text{AngularSpeed})^{2}+0.095(\text{AngularSpeed})+92.3$
(3)

A similar equation can be found for the left turn, and then the time that the
motors should stay on for the precise turn can be calculated as follows:

$\text{Time}=\frac{\text{Angle}}{\text{Angular Speed}}$ (4)

####  2.3.2 Calibration and Denoising the Ultrasonic Sensor:

For obstacle detection and avoidance, we used an HC-SR04 ultrasonic sensor,
which measures distances by emitting ultrasonic waves and calculating the time
taken for the echo to return. This section details the calibration and noise
removal processes implemented to enhance the sensor’s accuracy and
reliability.

The primary objective of sensor calibration was to correlate the sensor’s raw
measurements with actual distance measurements. We performed a series of
experiments where the sensor’s distance readings were recorded against known
distances, measured manually. These data points were then used to establish a
linear regression model that predicts the actual distance ($y$) as a function
of the sensor reading ($x$). The calibration model is defined by the linear
equation $\hat{y}=mx+b$, where $m$ is the slope and $b$ is the y-intercept.
These parameters were calculated using the least squares method, aiming to
minimize the sum of squared differences between the observed distances and
those predicted by the model. Figure 4 \- Left shows the actual distance
versus the sensor reading along with the trend line. The relationship between
the actual distance and the sensor reading can be expressed by the following
equation:

$\text{Actual Distance}=1.0759\times\text{Sensor Reading}+1.0158$ (5)

To address the noise inherent in the raw ultrasonic sensor data, which can be
caused by various factors including environmental conditions and sensor
anomalies, we implemented a Simple Moving Average (SMA) filter. The SMA filter
improves the reliability of distance measurements by averaging several
consecutive readings, thereby smoothing out random fluctuations.
Mathematically, SMA can be expressed as:

${SMA}_{i}=\frac{1}{M}\sum\limits_{j=0}^{M-1}d_{i+j}$ (6)

Where, ${\text{SMA}}_{i}$ is the simple moving average at the $i^{th}$
position in the smoothed series, M is the total number of data points in the
window used to calculate the average, including the current point $d_{i}$ and
the next M-1 points, and $d_{i+j}$ represents each data point in the window,
starting from the current data point $d_{i}$ and including the next M-1 points
in the series.

We chose a window size of 5 for the SMA, as this size provided a good balance
between noise reduction and response speed. The filter operates by maintaining
a rolling window of the last five calibrated distance readings and computing
their average. This smoothed value is then used as the output distance
reading, reducing the impact of transient noise spikes. Figure 4 \- Right
shows the smoothed distance readings obtained using the 5-point SMA filter
that clearly shows the reduction in noise spikes and the preservation of the
overall trend in the data.

Figure 4: Calibration and Noise Filtering of Ultrasonic Sensor Data. The left
graph illustrates the linear relationship between the actual distance and the
ultrasonic sensor readings with a trend line, while the right graph displays
the effectiveness of a noise-filtering process on ultrasonic sensor data over
time. Note that, the distance of the object in front of the sensor was
gradually increased over time.

###  2.4 Raspberry Pi Pico W Access Point and Arduino

We set up a Raspberry Pi Pico W to serve as an access point over Wifi. The
socket receives a string which is sent to the Arduino using microPython UART
Serial communication initialization. The specific Pico W-to-Arduino
communication was carried out through a wired Serial protocol at 9600 baud
rate using the hardware Serial receiver (RX) of the Arduino tethered to the
microPython-initialized UART transmission line (TX) of the Pico W. A common
ground was established between the Pico W and Arduino via connection from the
servo GND pin on the Arduino I/O Expansion Board to the GND pin of the Pico W
associated with the power supply. Once hardware connectivity was established,
the Pico W was programmed in microPython to send all received commands over
Wifi directly over UART to the Arduino; all mobility interpretation was coded
directly in the Arduino, with the Pico W acting as a receiver-transmitter
without any internal command transformations. This line of communication
ensures a network between the client and the mobile robot.

###  2.5 LLM Integration

Our project involves a language model as the central point. We utilized the
OpenAI API to access GPT-4-Turbo. The overall responses are accurate and quick
to generate. However, this model does require network access and a payment
method. LLaMa 2 7B is utilized through a program called ”llama.cpp” which
allows users to easily download and utilize different models on their machine.
Our hardware was not compatible with official base models due to GPU VRAM
limitations, leading to our decision to utilize a quantized version of Llama 2
called LLaMA 2-7B.Q5_K_M. This quantized model allows us to run the language
model offline and on our hardware. The models were given instructions to
convert a given string of text into a command followed by distance and/or
sequences of commands. The model will be instructed to interpret the naturally
spoken language and deduce the closest set of commands that would match the
intentions of the initial input. For example, the words ”Go forward 100cm”
would be spoken into a microphone and sent to one of our Language models. The
model would output our command format which if accurate, would result in
”f,100”. These commands would then be wirelessly sent over to our Raspberry Pi
Pico W. We evaluated the output of commands generated from these models and
determined how accurate they were in controlling our smart car through the
resulting actions.

##  3 Results

Figure 5: Some illustrative representations of the autonomous robotic response
to diverse natural language commands. The sequence demonstrates the mobile
robot’s capability to interpret and execute commands ranging from basic
directional instructions to complex navigational sequences. Each panel
captures a moment in the process flow where the robot performs actions such as
moving specific distances, executing precise turns, responding to ambiguous
commands, and combining multiple maneuvers.

From user command to calibrated execution, a plethora of actions were able to
be consistently executed in line with the expectations of natural language
input. Precise directional, translational, and angular mobility was accurately
interpreted, communicated, and employed at each stage of our information flow.
The mobile robot was also uniquely prepared for indirect commands, being able
to follow complex sequences transmitted by the user’s speech. Obstacle
avoidance, indefinite motor activation, precision directional control, travel
sequencing, as well as complex combinations of these functions were seamlessly
integrated into the mobility of the robot from simple, intuitive user speech
(see Figure 5).

Table 2: Comparison of Command Execution: GPT-4-Turbo vs LLaMA 2 7B.Q5_K_M
Quantized

Command | GPT-4-Turbo | LLaMA 2 Quantized
---|---|---
No.* | Trial 1 | Trial 2 | Trial 3 | Trial 1 | Trial 2 | Trial 3
1 | PASS | PASS | PASS | FAIL | FAIL | FAIL
2 | PASS | PASS | PASS | FAIL | FAIL | FAIL
3 | PASS | PASS | PASS | FAIL | PASS | FAIL
4 | PASS | PASS | PASS | FAIL | PASS | FAIL
5 | PASS | PASS | PASS | PASS | FAIL | FAIL
6 | PASS | PASS | PASS | FAIL | PASS | FAIL
7 | PASS | PASS | PASS | PASS | FAIL | FAIL
8 | PASS | PASS | PASS | FAIL | FAIL | FAIL
9 | PASS | PASS | PASS | FAIL | FAIL | FAIL
10 | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL
11 | PASS | FAIL | PASS | FAIL | FAIL | FAIL
12 | PASS | PASS | PASS | FAIL | FAIL | FAIL
13 | PASS | PASS | PASS | FAIL | FAIL | FAIL
14 | PASS | PASS | PASS | PASS | FAIL | PASS
15 | PASS | PASS | PASS | FAIL | FAIL | FAIL
16 | PASS | PASS | PASS | FAIL | FAIL | FAIL
17 | PASS | PASS | PASS | FAIL | PASS | FAIL
18 | PASS | PASS | PASS | FAIL | FAIL | FAIL
19 | PASS | PASS | PASS | FAIL | FAIL | PASS
20 | PASS | PASS | PASS | FAIL | FAIL | FAIL
21 | PASS | PASS | FAIL | FAIL | FAIL | FAIL
22 | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL
23 | FAIL | PASS | FAIL | FAIL | FAIL | FAIL

  * •

* Command 1: Move forward 300 cm, Command 2: Move forward 2 feet, Command 3: Move back 100 cm, Command 4: Move forward, Command 5: Move forward 50 cm, Command 6: Move backward 25 cm, Command 7: Turn right, Command 8: Turn right 180 degrees, Command 9: Turn left pi radians, Command 10: Turn, Command 11: Turn either direction, Command 12: Turn around, Command 13: Twirl, Command 14: Move forward 100 then stop moving, Command 15: Go forward then go backwards, Command 16: Move forward then come back, Command 17: Go forward, turn left, turn left, go forward, then go backward, Command 18: Do a twirl, then go to the wall, Command 19: Utilize your ultrasonic sensor, Command 20: Go behind you, then come back to where you started, Command 21: Move to the left, go to the wall, then come back, Command 22: Pick a route to traverse around the room, Command 23: Traverse around the room with lots of moves.

Integrating GPT-4-Turbo with the mobile robot has shown extremely strong
results. The mobile robot can understand a mix of simple and complex
instructions to properly execute desired commands in most contexts.
GPT-4-Turbo has shown its ability to critically translate provided prompts to
fit the parameters of robot functionality, such as to ”move forward 2 feet”.
In this example, GPT-4-Turbo needs to understand and convert 2 feet into
centimeters for proper function, as it properly does with forward output
between 60 and 61 centimeters. When increasing the complexity of prompts,
GPT-4-Turbo still has impressive performance. In a more complex prompt such as
to ”Do a twirl, then go to the wall”, we still find success. GPT-4-Turbo will
still correctly translate into correct functionality, rotating 360 degrees,
then moving forward (understanding to utilize its stopping mechanism from the
onboard ultrasonic sensor) to stop at a wall.

However, in testing with the quantized model of LLaMa 2 7B, we found offline
results to be subpar. Results taken from prompts as basic as to ”Move forward
50 centimeters” often resulted in cluttered outputs, such as moving forward
450 centimeters in added sequence, or even moving forward only 10 centimeters.
Similar results can be found across all trials using the LLaMa 2 7B model.
These results show that the implementation of a more accessible, lower-
computational cost, offline model still struggles to produce reliable output
for robotic control in real-world situations.

Table 2 demonstrates the performance comparison between GPT-4-Turbo and
quantized version of LLaMA 2 across a series of command execution trials.

##  4 Conclusion and Future Work

This paper presented a comparative analysis of the efficacy of GPT-4-Turbo and
the quantized LLaMA 2 models for integrating NLP and LLMs into mobile
robotics. The research highlighted GPT-4-Turbo’s proficiency in handling
complex commands effectively when cloud connectivity is available. In
contrast, the quantized LLaMA 2 model, despite being operable offline,
exhibited limitations in reliability and output consistency. The experimental
framework, utilizing a Raspberry Pi Pico W for wireless communication from a
control computer, and wired communication to an Arduino-driven robot,
exemplifies an approach to edge-based robotic control.

Looking forward, our future work will be dedicated to specifically fine-tuning
offline language models for our robotic applications. We aim to enhance the
model’s reliability and consistency by tailoring it more closely to the needs
and constraints of robotic interaction. This process will include pruning and
optimizing the model to ensure it operates efficiently on edge devices. The
underperformance of the LLaMA 2 model observed in our experiments is largely
attributed to its generic configuration, which was not initially optimized for
our specific robotic tasks. By adjusting the model parameters and structure to
better fit the practical requirements of mobile robots, we anticipate
significant improvements in performance. These modifications will enable the
deployment of sophisticated NLP capabilities directly on robots, facilitating
more dynamic and context-aware interactions between humans and robots without
relying on cloud connectivity. Crucially, this development will pave the way
towards a fully offline operational model, eliminating dependencies on cloud
connectivity.

## References

  * [1] Ahmadvand, H., Dargahi, T., Foroutan, F., Okorie, P., Esposito, F.: Big data processing at the edge with data skew aware resource allocation. In: 2021 IEEE conference on network function virtualization and software defined networks (NFV-SDN). pp. 81–86. IEEE (2021)
  * [2] Dong, Q., Chen, X., Satyanarayanan, M.: Creating edge ai from cloud-based llms. In: Proceedings of the 25th International Workshop on Mobile Computing Systems and Applications. pp. 8–13 (2024)
  * [3] ELEGOO: Smart robot car kit v4.0 (with camera). https://www.elegoo.com/products/elegoo-smart-robot-car-kit-v-4-0 (2024), accessed: 2024-04-22
  * [4] Gromov, A., Tirumala, K., Shapourian, H., Glorioso, P., Roberts, D.A.: The unreasonable ineffectiveness of the deeper layers. arXiv preprint arXiv:2403.17887 (2024)
  * [5] Liu, H., Zhu, Y., Kato, K., Kondo, I., Aoyama, T., Hasegawa, Y.: Llm-based human-robot collaboration framework for manipulation tasks. arXiv preprint arXiv:2308.14972 (2023)
  * [6] Poirier, S., Routhier, F., Campeau-Lecours, A.: Voice control interface prototype for assistive robots for people living with upper limb disabilities. In: 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR). pp. 46–52. IEEE (2019)
  * [7] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
  * [8] Sharan, S., Pittaluga, F., Chandraker, M., et al.: Llm-assist: Enhancing closed-loop planning with language-based reasoning. arXiv preprint arXiv:2401.00125 (2023)
  * [9] Soori, M., Arezoo, B., Dastres, R.: Artificial intelligence, machine learning and deep learning in advanced robotics, a review. Cognitive Robotics (2023)
  * [10] Tallat, R., Hawbani, A., Wang, X., Al-Dubai, A., Zhao, L., Liu, Z., Min, G., Zomaya, A.Y., Alsamhi, S.H.: Navigating industry 5.0: A survey of key enabling technologies, trends, challenges, and opportunities. IEEE Communications Surveys & Tutorials (2023)
  * [11] Vermesan, O., Bahr, R., Ottella, M., Serrano, M., Karlsen, T., Wahlstrøm, T., Sand, H.E., Ashwathnarayan, M., Gamba, M.T.: Internet of robotic things intelligent connectivity and platforms. Frontiers in Robotics and AI 7, 104 (2020)
  * [12] Wang, J., Wu, Z., Li, Y., Jiang, H., Shu, P., Shi, E., Hu, H., Ma, C., Liu, Y., Wang, X., et al.: Large language models for robotics: Opportunities, challenges, and perspectives. arXiv preprint arXiv:2401.04334 (2024)
  * [13] Yépez-Ponce, D.F., Salcedo, J.V., Rosero-Montalvo, P.D., Sanchis, J.: Mobile robotics in smart farming: current trends and applications. Frontiers in Artificial Intelligence 6, 1213330 (2023)
  * [14] Zeng, F., Gan, W., Wang, Y., Liu, N., Yu, P.S.: Large language models for robotics: A survey. arXiv preprint arXiv:2311.07226 (2023)
  * [15] Zhang, C., Chen, J., Li, J., Peng, Y., Mao, Z.: Large language models for human-robot interaction: A review. Biomimetic Intelligence and Robotics p. 100131 (2023)