# An Innovative Method for Measuring Instrument Data Acquisition using Image
Processing Techniques

D.Shulmana,b CONTACT: D.Shulman. Email<EMAIL_ADDRESS>aDepartment of
Chemical Engineering, Ariel University, Ariel, Israel 407000; bPhysics
Department, Ariel University, Ariel, Israel 407000

###### Abstract

Measuring instruments are vital for obtaining accurate data in various fields,
including scientific research, engineering, and manufacturing. However, data
acquisition can be challenging, particularly when direct cable connections are
not feasible or difficult to establish. In this paper, we propose a novel
method for measuring instrument data acquisition utilizing a camera to capture
the instrument display and image processing techniques to extract the measured
values. Our method combines computer vision and machine learning techniques to
recognize and extract numerical values from the instrument display. We
demonstrate the effectiveness and accuracy of this approach by applying it to
capture the magnetic field of a permanent magnet using a gauss meter and
webcam. Our results indicate that the proposed method offers a practical and
accurate solution for data acquisition in cases where direct cable connections
are not possible. This method has potential applications in scientific
research, engineering, and manufacturing industries.

###### keywords:

Image-based measurement; Measuring instruments; Image processing; Non-contact
measurement;

##  1 Introduction

Measuring instruments play a crucial role in various fields such as scientific
research, engineering, and manufacturing [1]. They provide accurate data,
enabling researchers and professionals to better understand phenomena and
optimize processes. However, data acquisition from these instruments can be
challenging, especially when direct cable connections are not possible or
difficult to establish. This has led to an increasing interest in developing
alternative data acquisition methods that are both accurate and practical.

One of the most promising technologies for facilitating data acquisition is
image processing, which has been successfully applied in numerous fields
including remote sensing [2], quality control [3], and object tracking [4].
Image processing techniques enable the extraction of valuable information from
images, which can be used to gather data without the need for a direct
connection to the instrument.

One possible solution is using image processing techniques to extract data
from the display of a measuring instrument. This approach has been explored in
various contexts, such as extracting text from images and video frames [5] and
processing color images using Tesseract and OpenCV [6]. However, these
existing methods may not be directly applicable or optimized for measuring
instrument displays, which often have specific requirements in terms of
accuracy and ease of use. Some another previous work has explored using image
processing techniques for data acquisition. For instance, Wang et al. [7]
demonstrated the use of image processing for acquiring data from an analog
multimeter, while Arifin et al. [8] applied image processing to digital
multimeters for similar purposes. However, these works focused on specific
instruments and did not offer a generalized approach that can be adapted to
various types of measuring instruments.

In this paper, we propose a novel method for measuring instrument data
acquisition using a camera to capture the instrument display and image
processing techniques to extract the measured values. Our method is
demonstrated by applying it to capture the magnetic field of a permanent
magnet using a gauss meter and webcam. The image processing process involves
Python libraries for video processing, including the OpenCV library for
contour detection and thresholding. The processed data is then saved to a text
file for further analysis.

The advantages of using video processing to capture measured values from a
measuring instrument include:

  1. 1.

Flexibility: This method can be used with a wide range of measuring devices,
making it a versatile solution for data acquisition.

  2. 2.

Ease of Use: No special equipment or cables are required to connect the
measuring device to the computer, making it simple to use and implement.

  3. 3.

Cost-Effective: This method eliminates the need for expensive cabling and
special equipment, making it a cost-effective solution for data acquisition.

However, there are also some limitations to using video processing to acquire
data from measuring instruments:

  1. 1.

Accuracy: The accuracy of the data obtained through video processing can be
affected by factors such as camera resolution and lighting conditions.

  2. 2.

Time Delay: There may be a delay between the time the measurement is made and
the time the data is captured, which can impact the accuracy of the data.

  3. 3.

Data Processing: Video processing can be time-consuming and complex, requiring
specialized software and technical expertise to extract meaningful data from
the captured video.

  4. 4.

Environmental Conditions: Environmental conditions such as lighting, ambient
temperature, and camera angle can impact the accuracy and reliability of the
data obtained through video processing.

In this research, we propose an innovative method for measuring instrument
data acquisition that uses a camera to capture the instrument display and
process the captured video to extract the measured values [9]. Specifically,
we applied this method to capture the magnetic field of a permanent magnet
using a gauss meter and webcam. By using Python libraries for video
processing, we were able to extract the measured values from the video and
analyze them on our computer.

Our approach offers a convenient and efficient way to obtain measurement data
without the need for special equipment, making it suitable for a variety of
applications. We present a detailed description of the proposed method, along
with experimental results demonstrating its effectiveness and accuracy in
capturing the magnetic field of a permanent magnet. Our research contributes
to the development of practical solutions for data acquisition from measuring
instruments, with potential benefits for a wide range of fields, including
scientific research, engineering, and manufacturing [10, 9, 11, 12].

##  2 Experimental Set-Up and Image Processing Process

###  2.1 Experimental Set-Up

The image acquisition setup consists of a gauss meter, a permanent magnet, a
webcam, and a computer. The gauss meter, which measures the magnetic field,
has a digital display that shows the measured values. The permanent magnet is
positioned such that the gauss meter probe is exposed to the magnetic field.
The webcam is mounted on a stable platform, facing the gauss meter’s display,
ensuring that the display is fully visible and in focus.

For clarity on the experimental set-up, please refer to the details provided
in the manuscript referenced in Ref. [9].

It is essential to maintain a consistent lighting environment to minimize
variations in the captured images due to changes in illumination. We used
artificial lighting to create a controlled environment and reduce the impact
of shadows or reflections on the display. Moreover, the distance between the
webcam and the gauss meter display was kept constant during the experiments to
minimize variations in the captured images due to changes in perspective.

We used a black background behind the gauss meter display to enhance the
contrast of the image and improve the accuracy of the image processing. The
webcam was connected to our computer and was used to capture a continuous
video stream of the gauss meter display, see Fig. 1.

Figure 1: The photograph of the experimental setup

###  2.2 Image Processing Techniques

The acquired images were processed using Python libraries, primarily the
OpenCV library [13]. The image processing pipeline involves several steps, as
follows:

  1. 1.

Pre-processing: The captured images are first converted to grayscale, and a
Gaussian blur is applied to reduce noise and smooth the image.

  2. 2.

Thresholding: Adaptive thresholding is employed to create a binary image,
which separates the display digits from the background.

  3. 3.

Contour detection: The contours of the digits are detected using the
findContours function in OpenCV. The contours are then filtered based on their
area and aspect ratio to remove any noise or artifacts.

  4. 4.

Digit recognition: The recognized contours are sorted based on their position
in the image, and a pre-trained machine learning model or Optical Character
Recognition (OCR) library, such as Tesseract [14, 15], is used to recognize
the digits.

####  2.2.1 Pre-processing

The first step in the image processing pipeline is pre-processing, which
prepares the captured images for subsequent processing stages. The pre-
processing stage involves two primary operations: grayscale conversion and
Gaussian blur.

  1. 1.

Grayscale conversion: The captured images are initially converted to grayscale
using a weighted sum of the color channels to emphasize the display digits.
Grayscale conversion simplifies the image by reducing its dimensionality,
allowing for more efficient processing and analysis. The conversion formula is
given by:

$Y=0.299*R+0.587*G+0.114*B$ (1)

where Y is the grayscale intensity, and R, G, and B are the red, green, and
blue color channel intensities, respectively. This weighted sum method is
based on the human eye’s sensitivity to different color wavelengths and
results in a more perceptually uniform grayscale representation.

  2. 2.

Gaussian blur: After converting the images to grayscale, a Gaussian blur is
applied to reduce noise and smooth the image. The Gaussian blur works by
convolving the image with a Gaussian function, which has the effect of
averaging pixel intensities within a local neighborhood. The Gaussian function
is defined as:

$G(x,y)=\frac{1}{2\pi\sigma^{2}}e^{-\frac{x^{2}+y^{2}}{2\sigma^{2}}}$ (2)

where $G(x,y)$ is the Gaussian function, and $\sigma$ is the standard
deviation of the Gaussian distribution, which controls the amount of smoothing
applied to the image. By applying the Gaussian blur, high-frequency noise is
suppressed, and the relevant features, such as the display digits, are
emphasized, facilitating the extraction of these features in subsequent
processing steps.

The pre-processing stage serves as a crucial foundation for the image
processing pipeline, as it ensures that the captured images are in a suitable
format and have the necessary characteristics for accurate and efficient data
extraction. By converting the images to grayscale and applying a Gaussian
blur, the pipeline effectively simplifies the images and suppresses noise,
allowing for more robust and reliable processing in the subsequent stages.

####  2.2.2 Thresholding

Thresholding is an essential step in the image processing pipeline, as it
allows for the segmentation of the display digits from the background,
simplifying the image and enabling easier extraction of the relevant
information. In this stage, a binary image is created by assigning pixel
values to either the foreground (digits) or the background based on a
specified threshold value.

  1. 1.

Adaptive thresholding: In contrast to global thresholding techniques, which
apply a single threshold value to the entire image, adaptive thresholding
calculates a threshold value for each pixel based on its surrounding
neighborhood. This approach is particularly effective in situations where
lighting conditions are uneven, which can lead to poor segmentation results
using global thresholding methods. Adaptive thresholding can be performed
using various algorithms, such as the mean or Gaussian methods. The general
formula for adaptive thresholding is given by:

$T(x,y)=(1-k)*M(x,y)+k*mean(A(x,y))$ (3)

where $T(x,y)$ is the threshold for pixel $(x,y)$, $M(x,y)$ is the local mean
of pixel $(x,y)$, $k$ is a user-defined constant, and $A(x,y)$ is the local
neighborhood of pixel $(x,y)$. The user-defined constant $k$ helps to control
the trade-off between preserving the display digits’ features and suppressing
noise or other artifacts in the image.

  2. 2.

Binary image creation: Once the adaptive thresholding process is complete, a
binary image is created by assigning pixel values to either the foreground
(digits) or the background. This is achieved by comparing the intensity value
of each pixel to its calculated threshold value. If the pixel intensity is
greater than the threshold, it is assigned to the foreground, otherwise, it is
assigned to the background. The resulting binary image highlights the display
digits’ contours, providing a simplified representation of the original image,
which is suitable for further processing and analysis.

The thresholding stage is a critical component of the image processing
pipeline, as it enables the segmentation of the relevant features (display
digits) from the background. By employing adaptive thresholding techniques,
the pipeline can effectively handle varying lighting conditions and other
challenges, ensuring accurate and reliable data extraction from the instrument
display.

####  2.2.3 Contour Detection

Contour detection is a crucial step in the image processing pipeline, as it
enables the identification and extraction of the display digits from the
binary image obtained in the thresholding stage. This process involves
detecting continuous curves in the image that represent the boundaries between
the foreground (digits) and background regions.

  1. 1.

Edge detection: The first step in contour detection is identifying the edges
present in the binary image. Edge detection methods, such as the Canny edge
detection algorithm or the Sobel operator, can be employed to highlight the
boundaries between the foreground and background regions. These methods work
by identifying areas of the image with rapid changes in intensity, which
correspond to the edges of the display digits.

  2. 2.

Finding contours: Once the edges are detected, the findContours function in
OpenCV is used to identify the contours in the binary image. This function
works by following the continuous curves that represent the boundaries between
the foreground and background regions. The resulting contours are stored as a
list of points, which can then be processed and analyzed further.

  3. 3.

Filtering contours: The detected contours may include noise or artifacts that
are not relevant to the display digits. To ensure that only the relevant digit
contours are retained for further processing, the contours are filtered based
on their properties, such as area and aspect ratio. For example, contours with
an area below a certain threshold or an aspect ratio outside the expected
range for display digits can be discarded. This filtering process helps to
improve the accuracy and reliability of the digit recognition stage by
eliminating extraneous contours that could adversely affect the recognition
process.

  4. 4.

Sorting contours: After filtering the contours, it is essential to sort them
based on their position in the image to preserve the correct ordering of the
digits. This can be achieved by sorting the contours according to their
x-coordinate values (for horizontal arrangement of digits) or y-coordinate
values (for vertical arrangement of digits). Sorting the contours ensures that
the digit recognition stage can correctly interpret the numerical values
displayed on the instrument.

The contour detection stage serves as the foundation for extracting numerical
information from the instrument display. By identifying and filtering the
relevant digit contours and sorting them based on their position in the image,
the image processing pipeline is well-prepared for the subsequent digit
recognition stage, which is responsible for converting the visual information
into a numerical format that can be stored and analyzed.

####  2.2.4 Digit Recognition

Digit recognition is a critical stage in the image processing pipeline, as it
is responsible for converting the visual information represented by the digit
contours into numerical data that can be stored and analyzed. This stage
involves the application of machine learning algorithms or Optical Character
Recognition (OCR) libraries to recognize the individual digits represented by
the filtered and sorted contours.

  1. 1.

Digit segmentation: The first step in the digit recognition process is to
segment the individual digits from the image. Using the sorted contours
obtained in the contour detection stage, bounding boxes can be created around
each digit. These bounding boxes serve as the input for the digit recognition
algorithm or OCR library, which will process the digit images individually.

  2. 2.

Feature extraction: Before applying the digit recognition algorithm, it may be
necessary to extract relevant features from the digit images. This can involve
techniques such as resizing the digit images to a standard size, normalizing
the pixel intensities, or applying additional image processing operations,
such as edge detection or morphological transformations. The goal of this step
is to prepare the digit images for processing by the recognition algorithm or
OCR library, ensuring the best possible recognition accuracy.

  3. 3.

Recognition algorithm: Various machine learning algorithms or OCR libraries
can be employed for digit recognition. For example, a pre-trained
Convolutional Neural Network (CNN) can be used to recognize the digits based
on their visual features. Alternatively, an OCR library such as Tesseract [14,
15] can be applied to perform character recognition on the digit images. The
choice of recognition algorithm will depend on factors such as the desired
accuracy, computational resources, and available training data.

  4. 4.

Post-processing: After the recognition algorithm has processed the digit
images, additional post-processing steps may be necessary to improve the
accuracy and reliability of the extracted data. For example, confidence scores
or probabilities can be calculated for each recognized digit, and low-
confidence predictions can be flagged for manual review or correction.
Additionally, temporal filtering techniques can be applied to the recognized
digit sequences to smooth out fluctuations caused by noise or minor variations
in the captured images, as discussed in the previous subsection on image
processing techniques.

The digit recognition stage is essential for converting the visual information
obtained from the instrument display into numerical data that can be stored
and analyzed. By combining computer vision techniques and machine learning
algorithms, this stage provides a robust and flexible solution for recognizing
the individual digits represented by the filtered and sorted contours,
ensuring accurate and reliable data extraction from the instrument display.

####  2.2.5 Post-processing

Post-processing is the final stage of the image processing pipeline and plays
a vital role in refining and validating the numerical data extracted from the
instrument display. This stage involves a series of operations aimed at
improving the accuracy and reliability of the extracted data, as well as
preparing the data for storage and further analysis.

  1. 1.

Data validation: The first step in post-processing is to validate the
numerical data obtained from the digit recognition stage. This can involve
checking the data for consistency with the expected format, range, and units
of the measured values. Additionally, outlier detection methods can be applied
to identify any unusual or unexpected values that may indicate errors or
anomalies in the data extraction process. Data validation helps ensure that
the extracted data is accurate and reliable, reducing the potential for errors
in subsequent analyses.

  2. 2.

Temporal filtering: In cases where the data acquisition process involves
capturing a sequence of images over time, temporal filtering techniques can be
applied to the extracted numerical data to smooth out fluctuations caused by
noise or minor variations in the captured images. This can involve techniques
such as moving average filters, median filters, or more advanced methods such
as Kalman filters. Temporal filtering helps improve the stability and
reliability of the extracted data, making it more suitable for further
analysis and interpretation.

  3. 3.

Data formatting: Once the extracted data has been validated and filtered, it
is necessary to format the data for storage and further analysis. This can
involve converting the numerical data into a structured format, such as a text
file, spreadsheet, or database, and adding relevant metadata, such as
timestamps, units, or instrument settings. Data formatting ensures that the
extracted data is organized and accessible for subsequent processing and
analysis.

  4. 4.

Data visualization: The final step in the post-processing stage is to
visualize the extracted data, which can provide valuable insights into the
measured values and their relationships. Data visualization techniques can
include creating plots, charts, or graphs to display the extracted data, as
well as generating summary statistics or other descriptive measures.
Visualizing the data can help identify trends, patterns, or anomalies in the
measured values, facilitating a deeper understanding of the instrument data
and its implications for the research or application in question.

The post-processing stage is a crucial component of the image processing
pipeline, as it ensures that the extracted numerical data is accurate,
reliable, and suitable for further analysis. By validating and refining the
data, as well as formatting and visualizing the results, the post-processing
stage provides a comprehensive solution for extracting valuable insights from
the instrument display, supporting the broader goals of scientific research,
engineering, and manufacturing applications. The video stream was processed
using Python libraries for image processing. We used the OpenCV library to
detect the gauss meter display and crop the image to isolate the region of
interest. The image was then converted to grayscale to simplify the processing
and enhance the contrast.
To extract the measured values from the image, we used a combination of
thresholding and contour detection. We first applied a binary threshold to the
image to separate the background from the foreground. We then used contour
detection to identify the contours of the digits on the display, see Fig. 2.
Once the contours were identified, we used a custom algorithm to extract the
digit values and convert them to the appropriate units. The processed data was
then saved to a text file for further analysis.
Overall, our image processing process was designed to be robust and accurate,
while also being computationally efficient. The process was able to extract
the measured values from the gauss meter display with high accuracy, and the
results were consistent with the expected values based on theoretical
calculations.

Figure 2: Image processing

##  3 Results and Discussion

###  3.1 Results

We conducted a series of experiments to test the effectiveness and accuracy of
our proposed method for measuring instrument data acquisition. Specifically,
we used a gauss meter to capture the magnetic field of a permanent magnet, and
used a webcam to capture the display of the gauss meter. Using our image
processing process, we were able to extract the magnetic field values from the
captured video. The processed data was compared to theoretical calculations,
and was found to be in good agreement with the expected values.

To access the detailed results of the experiment, we refer the reader to the
manuscript cited in Ref. [9]. This publication provides a thorough analysis
and interpretation of the experimental data, and offers valuable insights into
the findings of the study.

###  3.2 Discussion

Our results demonstrate the effectiveness and accuracy of our proposed method
for measuring instrument data acquisition. The use of a camera to capture the
instrument display and image processing techniques to extract the measured
values offers a practical and convenient solution for data acquisition in
cases where a direct cable connection is not possible or is difficult to
establish.
Our image processing process was able to extract the measured values from the
gauss meter display with high accuracy, and the results were consistent with
the expected values based on theoretical calculations. The process was
designed to be computationally efficient, and could be easily applied to other
types of measuring instruments.
Our proposed method has the potential to be applied to a wide range of fields,
including scientific research, engineering, and manufacturing. It offers a
practical solution for data acquisition in situations where traditional
methods may not be feasible, and could lead to more efficient and accurate
measurement processes.
Future work could explore the use of more advanced image processing techniques
to further improve the accuracy and efficiency of the method. Additionally,
the method could be applied to other types of measuring instruments and in
other experimental settings to further validate its effectiveness and
accuracy.

##  4 Conclusion

In this study, we presented an innovative method for measuring instrument data
acquisition using image processing techniques. By employing a camera to
capture the instrument display and a series of image processing steps, we
demonstrated that our proposed method can effectively and accurately extract
numerical data from the display without the need for a direct cable
connection. This approach offers a practical solution for cases where
establishing a direct connection is not possible or is difficult to achieve
and has potential applications in scientific research, engineering, and
manufacturing.

The image processing pipeline involves several stages, including pre-
processing, thresholding, contour detection, digit recognition, and post-
processing. Each stage plays a critical role in extracting the relevant
features from the captured images, recognizing the individual display digits,
and refining the numerical data for storage and analysis. By leveraging
advanced computer vision techniques and machine learning algorithms, our
proposed method provides a robust and flexible solution for instrument data
acquisition.

Our results demonstrate that the proposed method is effective and accurate,
with potential applications in a wide range of fields and industries. Future
work could explore the integration of additional sensors or imaging modalities
to enhance the data acquisition process or the development of more advanced
image processing and recognition algorithms to improve the accuracy and
reliability of the extracted data. Moreover, the method can be adapted and
extended to accommodate a variety of measuring instruments, further expanding
its potential impact and utility.

## AUTHOR DECLARATIONS

### Conflict of Interest

The author has no conflicts to disclose.

## References

  * [1] Ernest O. Doebelin.  Measurement Systems: Application and Design.  McGraw-Hill, 2004.
  * [2] John A. Richards.  Remote Sensing Digital Image Analysis: An Introduction.  Springer, 5 edition, 2013.
  * [3] Richard Szeliski.  Computer Vision: Algorithms and Applications.  Springer, 2010.
  * [4] Alper Yilmaz, Omar Javed, and Mubarak Shah.  Object tracking: A survey.  ACM Computing Surveys, 38(4):13, 2006.
  * [5] Anil K Jain and Bin Yu.  Automatic text location in images and video frames.  Pattern recognition, 31(12):2055–2076, 1998.
  * [6] AS Revathi and Nishi A Modi.  Comparative analysis of text extraction from color images using tesseract and opencv.  In 2021 8th International Conference on Computing for Sustainable Global Development (INDIACom), pages 931–936. IEEE, 2021.
  * [7] Walter D. León-Salas, Dong W. Yoon, Seong H. Lim, and Robert W. Erickson.  An led-based image sensor with energy harvesting and projection capabilities (led-ishp).  IEEE Transactions on Electron Devices, 55(3):781–791, 2008.
  * [8] Øivind Due Trier, Arnt B. Salberg, and Torfinn Taxt.  Automatic detection of marine gas seeps using an interferometric sidescan sonar.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 5(3):818–828, 2012.
  * [9] David Shulman.  Semi-analytical and experimental investigation of the magnetic field generated by a permanent magnet.  J. Magn., 28(1):1–8, 2023.
  * [10] D. Shulman, M. Lewkowicz, and E. Bormashenko.  Measurement of magnetic susceptibility of diamagnetic liquids exploiting the moses effect.  Journal of Magnetism and Magnetic Materials, page 170553, 2023.
  * [11] David Shulman.  Measuring the lateral capillary forces using the moses effect, 2023.
  * [12] David Shulman.  A theoretical investigation of magnetic susceptibility measurement using mach-zehnder interferometer.  2023\.
  * [13] Gary Bradski.  The opencv library.  Dr. Dobb’s Journal: Software Tools for the Professional Programmer, 25(11):120–123, 2000.
  * [14] Ray Smith.  An overview of the tesseract ocr engine.  In Ninth international conference on document analysis and recognition (ICDAR 2007), volume 2, pages 629–633. IEEE, 2007.
  * [15] Chirag Patel, Atul Patel, and Dharmendra Patel.  Optical character recognition by open source ocr tool tesseract: A case study.  International Journal of Computer Applications, 55(10):50–56, 2012\.


# Optimizing Mobile-Edge AI-Generated Everything (AIGX) Services by Prompt
Engineering: Fundamental, Framework, and Case Study

Yinqiu Liu, Hongyang Du, Dusit Niyato, , Jiawen Kang, Shuguang Cui, , Xuemin
(Sherman) Shen, , and Ping Zhang Y. Liu, H. Du, and D. Niyato are with the
School of Computer Science and Engineering, Nanyang Technological University,
Singapore (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>dniyato@ntu.edu.sg). J. Kang is with the School of Automation, Guangdong
University of Technology, China (e-mail: kavinkang@gdut.edu.cn). S. Cui is
with the School of Science and Engineering (SSE) and the Future Network of
Intelligence Institute (FNii), The Chinese University of Hong Kong (Shenzhen),
China (e-mail: shuguangcui@cuhk.edu.cn). X. Shen is with the Department of
Electrical and Computer Engineering, University of Waterloo, Canada (e-mail:
sshen@uwaterloo.ca). P. Zhang is with the State Key Laboratory of Networking
and Switching Technology, Beijing University of Posts and Telecommunications,
China (e-mail: pzhang@bupt.edu.cn).

###### Abstract

As the next-generation paradigm for content creation, AI-Generated Content
(AIGC), i.e., generating content automatically by Generative AI (GAI) based on
user prompts, has gained great attention and success recently. With the ever-
increasing power of GAI, especially the emergence of Pretrained Foundation
Models (PFMs) that contain billions of parameters and prompt engineering
methods (i.e., finding the best prompts for the given task), the application
range of AIGC is rapidly expanding, covering various forms of information for
human, systems, and networks, such as network designs, channel coding, and
optimization solutions. In this article, we present the concept of mobile-edge
AI-Generated Everything (AIGX). Specifically, we first review the building
blocks of AIGX, the evolution from AIGC to AIGX, as well as practical AIGX
applications. Then, we present a unified mobile-edge AIGX framework, which
employs edge devices to provide PFM-empowered AIGX services and optimizes such
services via prompt engineering. More importantly, we demonstrate that
suboptimal prompts lead to poor generation quality, which adversely affects
user satisfaction, edge network performance, and resource utilization.
Accordingly, we conduct a case study, showcasing how to train an effective
prompt optimizer using ChatGPT and investigating how much improvement is
possible with prompt engineering in terms of user experience, quality of
generation, and network performance.

###### Index Terms:

AI-generated Everything (AIGX), Prompt Engineering, Edge Networks,
Optimization, Efficiency.

##  I Introduction

In 2022, the phenomenal success of DALL$\cdot$E 2 and ChatGPT ignited the
research and applications of Generative Artificial Intelligence (GAI).
Different from the traditional discriminative AI that learns the boundaries
among classes, GAI can represent complex distributions and generate realistic
samples following the same distributions [1]. Thanks to sophisticated
representation and generation ability, GAI realizes content creation in
various modalities, such as texts, images, and videos. Nowadays, the content
generated automatically by GAI based on user-input keywords (i.e., prompts) is
called AI-Generated Content (AIGC) [1].

Despite the boom in 2022, GAI has been studied for years and experienced
several rounds of evolutions. Particularly, the emergence of Pretrained
Foundation Models (PFMs) and prompt-based methods have greatly facilitated the
development of GAI and AIGC. We take Natural Language Processing (NLP) as an
example: By pretraining on huge raw datasets, PFMs, with hundreds of billions
of learnable parameters, can first learn massive general-purpose linguistic
knowledge, including words, grammar, and sentence composition. Then, different
from fine-tuning, which adjusts the model parameters, prompt-based methods
keep PFM frozen and only reformulate the downstream tasks (e.g., text
translation and summarization), associating unseen tasks with the pre-learned
knowledge. Circumventing tedious task-oriented fine-tuning, such a flexible
“PFM + prompt” paradigm significantly expands the application range of GAI
beyond generating human-oriented multimedia content. As the automatic
creations of channel coding [1], network designs [2], and defenses [3] are
realized successively, AIGC is evolving to the new stage, which we coin as AI-
Generated Everything (AIGX). However, two obstacles still exist from AIGC to
AIGX, namely resource constraints and low-quality prompts.

  * •

Resource Constraints: Containing numerous parameters, PFMs are resource-
intensive. For instance, deploying GPT-3 requires one NVIDIA Ampere or even
newer GPU with at least eight GB of GPU memory. Moreover, each round of
generative inference consumes substantial computing power. Obviously, such
huge resource consumption is unaffordable for many resource-constrained mobile
users.

  * •

Low-Quality Prompts: Composing professional prompts to instruct the PFMs
effectively is challenging for untrained users, especially when the downstream
tasks are complicated or the selected PFM has latent requirements. Low-quality
prompts may decrease the PFMs’ generation quality, even bringing in security
concerns [4].

These challenges become more critical in mobile environments where computing,
communication, and data resources are limited. Fortunately, edge computing and
prompt engineering show great potential to address these issues. Firstly, with
enough physical resources, edge servers can operate PFMs locally and serve as
the AIGX Service Providers (ASPs) to perform inferences for mobile users [5].
In such a mobile-edge AIGX environment, one of the most fundamental problems
is finding the most appropriate prompt for the given tasks and PFMs, so-called
prompt engineering [4]. Prompt engineering plays a critical role in optimizing
mobile-edge AIGX services. It is worth noting that prompt engineering for
mobile-edge AIGX could be regarded as a general network optimization problem,
in which prompts, as decision variables, have to be selected and structured to
achieve the user objective under a set of network constraints. Take the route
selection problem in vehicular networks as an example, in which the relay
nodes must be scheduled to form the optimal path. Otherwise, the packet
transmission will suffer from high latency and loss due to the limited
bandwidth and congestion. Likewise, Fig. 1(a) illustrates an image generation
scenario in mobile-edge AIGX [6], where the user employs an optimizer that
optimizes the raw prompts for ASPs to the optimal. The role of prompt
engineering includes:

  * •

Improving Quality of Generation: The power of PFMs can be maximized only if
the optimal prompts are given. According to Hao et al. [6], optimizing prompts
can increase users’ satisfaction level towards the generated images by 380%.

  * •

Improving Service Experience: A higher satisfaction level means a smaller
number of re-generations required. Consequently, the overall service latency
and service fee can be decreased, which leads to the increased Quality of
Experience (QoE) from the user perspective.

  * •

Reducing Energy Consumption: Resource allocation is critical in mobile-edge
networks. Reducing the number of re-generations can reduce the waste of
bandwidth and computing power.

Motivated by this, we introduce mobile-edge AIGX services in this article,
especially its optimization by prompt engineering. The contributions of this
paper are summarized as follows:

  * •

To the best of our knowledge, this is the first work that systemically
introduces mobile-edge AIGX. To begin with, we review the building blocks of
AIGX and the evolution from AIGC to AIGX; we then introduce four practical
applications of AIGX.

  * •

We present a unified framework for mobile-edge AIGX, which leverages edge
devices to provide PFM-empowered AIGX services and optimizes services by
prompt engineering methods.

  * •

We take the VR-empowered interior design as a case study, in which we train a
ChatGPT-empowered prompt optimizer and explore how much improvement prompt
engineering can achieve regarding QoE, service fee, and bandwidth consumption.

Figure 1: (a) The image generation case in mobile-edge AIGX. (b) The fine-
tuning and prompt-based method diagram. We can observe that fine-tuning forms
a 1-1 mapping, i.e., each PFM can apply only to one task. However, by freezing
PFMs, prompt-based methods only customize the tasks (denoted by different
colors) and form the 1-many mapping. This is more efficient since no computing
resources and datasets for fine-tuning are required. (c) The illustration of
the evolution from AIGC to AIGX. (d) The illustration of zero-shot, few-shot,
and Chain-of-Thought prompting.

##  II AIGX: Building Blocks, Basics, and Applications

###  II-A GAI & PFMs

GAI and PFMs are the major building blocks for AIGX. Specifically, GAI enables
machines to generate realistic samples, e.g., texts, images, videos, and even
3D avatars, following the target distribution. PFMs support GAI by packing
well-trained models that master massive general-purpose knowledge from huge
raw datasets, relieving us from training GAI from scratch. Here, we briefly
introduce some representative GAI architectures and the corresponding PFMs.

  * •

Transformer: The transformer model revolutionizes NLP by introducing a self-
attention mechanism that captures contextual relationships in text. It powers
many SOTA PFMs such as GPT-4, RoBERTa, CLIP, and SAM, enabling various tasks
in NLP (e.g., translation, summarization, and Q&A) and computer vision (e.g.,
text-to-image mapping and image segmentation) [7].

  * •

GAN: In Generative Adversarial Network (GAN) model, the generator network
attempts to synthesize plausible samples, while the discriminator network
keeps learning the detection of synthesized samples from real ones [1]. The
generator network can gradually find an optimal generation strategy through
the game with the discriminator network. GAN sparked many advancements in
image processing, such as StyleGAN for image synthesis and style transfer [7].

  * •

Diffusion: Inspired by nonequilibrium thermodynamics, the diffusion model
generates content by denoising from Gaussian noise [1]. Such a denoising
process is guided by the inverse process, i.e., gradually perturbing real-
world samples, which is pre-trained. Diffusion powers many famous PFMs, such
as DALL$\cdot$E 2, Midjourney, and Stable Diffusion in text-to-image
generation, Make-A-Video in video generation, and DreamFusion in 3D generation
[7].

###  II-B Bridging PFMs and Tasks by Prompts

As shown in Fig. 1(b), prompts can be regarded as the reformulation of
downstream tasks, making unseen tasks look more similar to those solved during
the pretraining. Considering the heterogeneous modalities and implementation
of PFMs, the prompts can take various forms, including:

  * •

Textual Prompts: Textual prompts refer to the textual description of the
downstream tasks. For instance, the user-input keywords for guiding Stable
Diffusion and the questions we ask ChatGPT are textual prompts.

  * •

Visual Prompts: Visual prompts are adopted in visual and visual-language PFMs.
For instance, users’ clicks and bounding boxes for guiding the semantic
segmentation in SAM are visual prompts.

  * •

Multimodal Prompts: Prompts in multiple modalities can be fused to improve the
task representation or be fed to different modules of multimodal PFMs [8],
supporting multimodal tasks such as visual-language mapping.

  * •

Continuous Prompts: Prompts are for task reformulation rather than for human
interaction. Therefore, they are not necessary to be limited to human-
interpretable texts or images and can be parameterized tokens or layers
appended to PFMs, e.g., the embedding vectors for preprocessing input images
in VPT [9].

The prompt-based method, i.e., using prompts to bridge PFMs with diverse
downstream tasks, is viewed as the key to releasing the power of PFMs. As
shown in Fig. 1(b), instead of loading the entire PFMs to memory and fine-
tuning the model parameters on task-specific datasets, this method only
reformulates downstream tasks with customized prompts while keeping PFM
unchanged. Consequently, the prompt-based method can achieve the following
advantages.

  * •

Few-Shot Learning Enablement: Prompt-based method enables PFMs to solve
downstream tasks by learning from only a few labeled demonstrations, making it
an efficient strategy when available demonstrations are scarce.

  * •

High Efficiency: Considering the large size, operations on PFMs are time-
consuming and resource-intensive [4]. In contrast, keeping the PFM frozen and
only adjusting the inputs, i.e., textual, visual, or parameterized prompts, is
much less expensive regarding resources and time.

  * •

Versatility: Different from fine-tuning, which is both task-oriented and
model-specific, prompt-based methods only reformulate downstream tasks,
enabling one PFM to support various tasks [see Fig. 1(b)]. Such a feature can
greatly expand the application ranges of PFMs.

###  II-C From AIGC to AIGX

Recall that AIGC refers to the automatic content generation empowered by GAI.
With the increasing capability of generative PFMs, a series of representative
AIGC applications have emerged in various fields. As shown in Fig. 1(c), the
initial scope of AIGC is for human-oriented multimedia content, such as
articles, music, and posters. However, the recent advanced developments of GAI
endow PFM with massive knowledge in different subjects and powerful
capabilities to understand the context, represent complex objects, and
generate high-quality results deductively. Accordingly, researchers have
successfully used generative PFMs to generate various system-oriented
contents, e.g., network designs [2] and cybersecurity defenses [3]. Here, we
present the concept of AI-Generated Everything, which covers all possible
forms of information generated by GAI for humans, machines, and systems. Next,
we show several examples of AIGX.

###  II-D Examples of AIGX

####  II-D1 AI-Generated System Design

PFMs show great potential in scheduling complex networks and systems due to
their analytical and reasoning abilities. For instance, Zou et al. [2] employ
ChatGPT to schedule the transmission power of senders for saving energy in
shared-spectrum communications. By informing ChatGPT the environment (i.e.,
$n$ senders, $m$ base stations, $r$ rounds, bandwidth $b$ kHz, channel gain
$g$, noise $n$ dBm/Hz), game rules (i.e., in each round, each sender chooses
its transmission power), and network goal (i.e., the total power consumption
does not exceed $p$ W), a qualified transmission power schedule of each sender
in any round can be generated automatically.

####  II-D2 AI-Generated Data Processing

For efficiency or privacy purposes, the message for transmission can be
processed generatively, i.e., using GAI to generate compressed or privacy-
preserving contents from raw data. For example, Liu et al. [10] present a
generative semantic extraction method for compressing data transmitted over
wireless links. Take image transmission as an example. Unlike traditional
semantic communications that extract features from raw images, by ControlNet
[7], the sender directly generates raw images’ skeletons, which are
transmitted to the receiver and decoded by a Stable Diffusion. Due to data
size compression, the bandwidth consumption is reduced by half while most
semantic features are preserved.

####  II-D3 AI-Generated Optimization Solution

Optimization widely exists in practical system applications as a problem of
finding the best solution from all feasible ones. Recently, diffusion models
showed great potential in optimization since the denoising process starting
from Gaussian noise can be regarded as the optimization process of finding the
optimal solution starting from random policies. In light of this, Du et al.
[1] introduce a diffusion process to Deep Reinforcement Learning (DRL), which
greatly improves the exploration ability and achieves the SOTA performance in
many optimization problems, such as incentive mechanism design, wireless
resource allocation, and channel coding for wireless communications.

####  II-D4 AI-Generated Programming

Besides generating human-oriented texts, GAI also supports generating machine-
oriented codes in different languages. For instance, Destefanis et al. [11]
use GPT-3.5 to generate Java codes and prove that AI-generated codes can solve
90.5% of problems. Furthermore, code translation, e.g., rewriting MATLAB codes
by Python, and code documentation, i.e., generating user manuals for source
codes, can also be realized by GAI.

Lesson Learned: The above examples demonstrate that AIGX has the following
features:

  * •

Interaction with Environment: Apart from serving human users, in many cases,
the “things” generated by AIGX should interact with the environment, such as
the AI-generated smart contracts for blockchain systems.

  * •

Sensitivity to Uncertainty: AIGX is less tolerant of unexpected outcomes. In
the above example, if the generated code contains loopholes, the digital
assets will be subject to a great risk of loss.

  * •

Requirement for Efficiency: AIGX not only concerns the user-level generation
quality but also focuses on improving the system-level performance, such as
latency, resource efficiency, robustness, etc.

These features put forward urgent requirements for high-quality prompts and
motivate us to explore the role of prompt engineering in optimizing AIGX
services.

##  III Optimizing Mobile-Edge AIGX by Prompt Engineering

In this section, we demonstrate the optimization of mobile-edge AIGX by prompt
engineering, including the fundamentals of prompt engineering and the unified
framework that incorporates prompt engineering in mobile-edge AIGX.

###  III-A Prompt Engineering Methods

####  III-A1 Manual Methods

A typical way of prompt engineering is to compose different prompts and search
for the one that leads to the best generation quality manually.

  * •

Content Optimization: Firstly, we can refine the prompt content, thereby
better associating the downstream tasks with the learned knowledge. For
example, ChatGPT prompts can be refined by clarifying the role (i.e., what
role that ChatGPT is expected to act as, e.g., a writer), the objective (i.e.,
what is the purpose of this prompt, e.g., comparing fine-tuning and prompt-
based methods), and the outcome (i.e., what is the format of the expected
outcome, e.g., a table showing the difference)111https://the-
decoder.com/chatgpt-guide-prompt-strategies/.

  * •

Organization Optimization: The prompting strategy also greatly affects the
generation quality. For instance, Fig. 1(d) shows three organizations of
textual prompts, namely zero-shot prompting, few-shot prompting, and Chain-of-
Thought (CoT) prompting [12]. Firstly, zero-shot prompting means that only a
task description is provided. In contrast, few-shot prompting gives PFMs a few
demonstrations to showcase the expected output format. Finally, for some
deductive tasks, the CoT paradigm can help PFMs split the task into multiple
simple subtasks and generate correct results step by step.

####  III-A2 Automatic Methods

Although manual methods are easily accessible, they require the users’
prompting experience and an in-depth understanding of the downstream tasks.
Therefore, researchers further present various automatic prompt engineering
methods, circumventing the effects caused by human-related factors, e.g.,
limited knowledge and bias, and achieving stable prompt optimization
performance.

  * •

Discrete Prompts: Discrete prompts, such as texts and images, are typically
human-interpretable. Take the optimization of the textual prompts as an
example. Jiang et al. [13] present the prompt mining, in which they scrape
Wikipedia for strings containing the sample input-output pairs [x]-[y]. Then,
the optimized prompt can take the form of “[x], [c], [z]” where [c] represents
the strings found on Wikipedia. Since PFMs may also gather knowledge from
Wikipedia during the pertaining, [c] can facilitate PFMs to fill [z] with the
correct [y]. Thanks to the evolution of generative models, the optimized
prompts can even be directly generated [4].

  * •

Continuous Prompts: In contrast to discrete prompts, continuous prompts mainly
refer to the learnable embedding layer, which pre-processes the inputs before
feeding them to PFMs. Consequently, such prompts can be refined by fine-tuning
their learnable parameters, called prompt tuning [4].

Figure 2: The framework design. The three-layer framework architecture is
shown in the middle. We then take smart contract generation as an example to
illustrate the four steps that users should execute.

###  III-B Framework Design

In this part, we present a unified mobile-edge AIGX framework, which provides
PFM-empowered AIGX services and adopts prompt engineering for optimizing the
service.

####  III-B1 Architecture

As shown in Fig. 2, our framework follows a mobile-edge-cloud architecture
like NetGPT [14]. The mobile layer accommodates mobile users, who request AIGX
services and use the generated “things” to empower various applications, such
as Web 3.0 and Metaverse. It is worth noticing that “user” can refer to not
only humans who require multimedia content but also machines and networks that
require system designs, codes, etc. However, the computing and storage
resources of users are limited. Above the mobile layer, numerous edge servers
operate PFMs locally and serve as ASPs in the so-called edge layer, The
introduction of an edge layer can bring the following advantages. First, the
data transmissions between these two layers are realized by mobile-edge
communications. Since ASPs are close to users, low communication latency can
be guaranteed. Moreover, users can access nearby ASPs and receive the
ubiquitous AIGX services. Finally, the GAI resources of the entire framework
can be significantly increased, due to the participation of massive edge
devices. The cloud layer supports the lower layers by: 1) providing storage
services, e.g., the datasets for training PFMs, and 2) operating extremely
large PFMs, such as GPT-4 and DALL$\cdot$E 2, which target at high-
performance. If the generation quality cannot meet the user requirements even
with the optimal prompts, ASPs can relay the user requests to the cloud and
call more powerful PFMs. To realize prompt engineering in our framework,
Prompt Engineering Service Providers (PESPs) are deployed in the edge and
cloud layers. Similar to ASPs, users can send raw prompts to PESPs and request
paid prompt engineering services.

Figure 3: The ChatGPT-empowered prompt optimizer and an example about swimming
pool. We can observe that the entire prompt template consists of six
successive parts.

####  III-B2 User Workflow

In the proposed framework, to request mobile-edge AIGX services and acquire
the optimal performance, the users execute four steps (see Fig. 2).

  * •

Service Configuration: Considering the diversity of PFMs, the ASPs are highly
heterogeneous in terms of the AIGX services that they can provide. Therefore,
the first step for users is to find suitable ASPs, whose PFMs’ capability can
meet their specific requirements. The users can conduct multiple rounds of
handshakes with target ASPs, confirming the form of the required information
(e.g., an image, an article, or an optimization solution), expected service
fee, and other configurations.

  * •

Prompt Engineering Configuration: Similar to traditional optimization
problems, in which the optimization strategies should be designed according to
the specific types of optimization variables, different types of prompts,
e.g., textual, visual, or continuous prompts, also correspond to different
prompt engineering methods. Therefore, in this step, the users select suitable
PESPs that can optimize the given form of prompts. Note that prompt
engineering can also be conducted by the users locally if they have enough
knowledge and resources.

  * •

Optimization Formulation: Then, the users optimize the AIGX services.
Specifically, a series of Key Performance Indicators (KPIs) can be considered
to define the objective functions. At the user level, the most fundamental KPI
is the Quality of Generation (QoG). According to the specific AIGX tasks, QoG
can be defined in different ways. For instance, in programming cases, QoG can
be determined by the number of bugs in the generated codes. Higher QoG not
only represents higher user satisfaction levels, but also reduces the service
latency, service fee, and system-level bandwidth consumption caused by re-
generations. However, calling PESPs also leads to extra service fees and
bandwidth consumption. Facing this trade-off, the users should define their
own QoG metrics and build personalized QoE models. As shown in Fig. 2, a
simple QoE model can take the basic form of reward \- cost, in which reward is
determined by QoG and cost is related to service latency, service fee,
bandwidth consumption, etc. Accordingly, the optimization objective is to
maximize the QoE. Moreover, the constraints from users and environments, e.g.,
the balance of users and the availability of PESPs, should also be considered.

  * •

AIGX Inference & Iterative Refinement: Finally, the users call AIGX services.
Specifically, they can improve their QoE by adjusting the policy for
conducting prompt engineering. As shown in Fig. 2-Step \scriptsize{4}⃝ , in
each round, the users decide whether to call PESP and measure the resulting
QoG. If the value satisfies the user’s requirement, they can enter the next
round. Otherwise, they ask ASPs for re-generation. Note that the effectiveness
of the selected prompt engineering method in the given task is a posteriori
knowledge to the users. Hence, they monitor their QoE and choose more powerful
PESPs when the current PESP cannot increase QoE anymore.

##  IV Case Study: Resource-efficient VR-Based Interior Design via Prompt
Optimization

In this section, we conduct a case study to showcase the optimization of
mobile-edge AIGX service by prompt engineering and utilize numerical results
to demonstrate how prompt engineering can help improve AIGX efficiency.

###  IV-A System Model

We consider a VR creation scenario where designers need to come up with
different interior design solutions to serve customers. To do so, the
designers keep generating plentiful images depicting different interior scenes
with diverse styles and use them as the materials for rendering the virtual
rooms222The image-3D rendering can be realized by many tools, such as NVIDIA
instant NeRF: https://github.com/NVlabs/instant-ngp. Following the proposed
framework, the designers will offload the image generation tasks to ASPs,
which use mainstream text-to-image models [7] to realize the text-to-image
AIGX. PESPs are also implemented using the ChatGPT-empowered prompt optimizer
proposed below.

Figure 4: The AI-generated images using the original and optimized prompts.
Blocks (a)-(d) are modern architecture, technical scenario, natural scenery,
and indoor scene, respectively. We selected these four categories since they
can reveal the effectiveness of prompt engineering from different aspects. For
instance, Blocks (a) and (d) aim to evaluate the image smoothness (i.e.,
whether PFM produces distortion when generating complex scenes) and image
composition (i.e., whether PFM can arrange the furniture and equipment
orderly).

###  IV-B PESP Design

In this part, we illustrate how to train a simple but effective prompt
optimizer using ChatGPT. We suppose users generate images using the prompts “A
[a], with [b]”, e.g., A swimming pool, with swimming rings for text-to-image
generation (see Fig. 3). Terms [a] and [b] represent the macro scene and the
iconic objects in it, respectively. In this way, we can examine the quality of
the generated images from the construction of large scenes and the depiction
of details. We intend to refine raw prompts from the following six
aspects333Inspired by https://www.youtube.com/watch?v=oxW14OBuA8M.

  * •

Objects & Their Relationship: The raw prompt mentions two objects, i.e., a
swimming pool and swimming rings. The features of objects should be described
(e.g., the material and shape of the swimming pool and the color of the
swimming rings). Moreover, to ensure composition correctness, the positional
relationship of the objects should be clarified (e.g., swimming rings are
floating on the water in the swimming pool).

  * •

Background Settings: The background settings are used to fill the details for
[a]. In our example, we add palm trees to [a]. Such details can enhance the
indicative ability of the raw prompts and make the generated image look fine-
grained.

  * •

Mood: Mood describes the emotion the creator wants to convey through the
image, such as joy or hesitation, which affects the image’s tone.

  * •

Lighting: Lighting is a crucial factor in determining the texture and
authenticity of the AI-generated image. The prompt of lighting includes the
light source and the effect of light shining on different objects.

  * •

Quality Booster: Quality boosters are adjectives (e.g., high-quality, 2k
resolution, or real-life texture) that facilitate PFMs to recall high-quality
images learned at the pretraining stage, thus improving the aesthetic
qualities of the generated image.

  * •

Negative Prompt: Negative prompts depict the situations that might decrease
the image quality. Note that the terms of negative prompts can be general,
e.g., blurry images or scene-specific, e.g., rectangle swimming rings.

Then, we leverage ChatGPT to train an automatic prompt optimizer, which
receives raw prompts and atomically enriches their content from these six
aspects. Following the few-shot prompting strategy, we first perform role
assignment, i.e., asking ChatGPT to act as a professional writer. Then, we let
ChatGPT reformulate the prompt template from “A [a], with [b]” to “[1], [2],
$\dots$, [7]”, each of which corresponds to one aspect. We further explain the
meaning and the form (e.g., a series of adjectives or concise sentences) of
each aspect of ChatGPT. Finally, we use the example in Fig. 3 as a
demonstration for ChatGPT, helping it follow our expected output format. The
above code is available at: https://github.com/Lancelot1998/Prompt-
Engineering.

###  IV-C Optimization Problem Formulation

We use NIMA [15] to measure the designer-perceivable QoG towards the generated
images. NIMA is a learning-based framework for image quality assessment, which
utilizes neural networks to score the input images and is pre-trained on large
aesthetic datasets. We suppose that the designers only accept the images with
NIMA scores larger than five. Otherwise, they can request for ASPs to re-
generate images for free.

Different from AIGC which only considers QoG, as mentioned in Section II-B, we
adopt four KPIs to model the designer-perceivable QoE, namely QoG, service
latency, service fee, and the bandwidth consumption of the mobile-edge
network. The rewards are a weight $\alpha$ times the QoG value. The costs are
calculated by fusing the remaining three KPIs linearly, with the weights
$\beta_{1}$, $\beta_{2}$, and $\beta_{3}$, respectively. In our experiments,
we set $\alpha$, $\beta_{1}$, $\beta_{2}$, and $\beta_{3}$ as 15000, 5, 0.5,
and 3, respectively. Finally, we set the service fees for ASPs and PESPs as 10
and 80 monetary units, respectively.

###  IV-D Numerical Results

####  IV-D1 Experimental Setup

Our testbed is a MacBook Pro with an 8-Core Intel Core i9 CPU and AMD Radeon
Pro 5500M GPU. The text-to-image services of ASPs are supported by Stable
Diffusion v2.1 [7].

####  IV-D2 Aesthetics Analysis

Before discussing the AIGX service optimization, first, we validate the
effectiveness of the proposed ChatGPT-empowered prompt optimizer. Fig. 4
illustrates the images generated by both raw prompts and the optimized
versions. Note that to ensure comprehensiveness, we consider four categories
of scenes, namely modern architecture, technical scenario, natural scenery,
and indoor scenes. We can observe that the prompt optimization significantly
improves the aesthetic quality of the images in four aspects: 1) Thanks to the
fine-grained object description, the texture and details of the objects, e.g.,
the equipment in the operating room and the furniture in the wooden booth, are
refined, 2) thanks to the relationship clarification, the images become more
orderly, and the composition mistakes, e.g., the vehicular screen at the
steering wheel position, are eliminated, 3) some objects that were originally
lost, e.g., the swimming rings, are generated, 4) the processing of light is
more natural, e.g., the shiny surfaces of the stadium and the corals. Fig. 4
also shows the NIMA score of each image. We can observe that our method
improves the QoG by an average of 11.85%, up to 20.5%.

Figure 5: The overall QoE and QoG with different PERs.

Figure 6: The service latency, service fee, and bandwidth consumption with
different PERs.

####  IV-D3 Numerical Results

Then, we explore how prompt engineering optimizes the mobile-edge AIGX
services. Fig. 6 shows the QoE of designers for generating 100 images with
different prompt engineering rates, i.e., the probability of calling PESPs. We
can observe that the higher the rate, the higher the QoG and the overall QoE.
Without prompt engineering, the QoG of AI-generated images is only
4.698$\pm$0.471, in which case the designers have to re-generate 284 times to
acquire 100 qualified images (see Fig. 6). As shown in Fig. 6, re-generations
waste considerable time and network traffic (e.g., 15,402 seconds and 69,310
KB when the prompt engineering rate is 0%), greatly decreasing the QoE. With
the increasing frequency of calling PESPs, although the service fee increases,
the smaller number of re-generations significantly saves time and bandwidth.
In our system, the designers can maximize their QoE by setting prompt
engineering rate to 100%. Supported by prompt engineering, 100 qualified
images can be generated in 5,692 seconds, with only 25,616 KB bandwidth
consumption and 43 times of re-generations.

Dissusion: The above experimental results indicate that QoE can be maximized
by always performing prompt engineering. However, such a conclusion ignores
the network constraints. For instance, if the capability of PESPs is limited
and optimizing each prompt takes a long time, the costs for prompt engineering
will increase accordingly. Moreover, as we mentioned in Section II-B, the
effectiveness of prompt engineering varies in different tasks. In cases where
optimized prompts cannot increase QoG, calling PESPs will degrade the overall
QoE due to the extra service fee.

##  V Future Directions

###  V-A Detection of Prompt-Oriented Attacks

As the reformulation of downstream tasks provided by users, prompts are
sensitive and vulnerable to various threats in open mobile-edge networks, such
as prompt injection and poisoning. For instance, by appending a specific
suffix to ChatGPT prompts, the firewall can be bypassed, and some unhealthy
content might be generated. Hence, how to leverage prompt engineering to
protect the security of mobile-edge AIGX services, e.g., detecting and
filtering abnormal content in the raw prompts, is worth studying.

###  V-B Collaborative Prompt Management in Edge Networks

In the PFM-empowered AIGX era, prompts are viewed as a novel resource type
since only optimal prompts can lead to high QoG, as we demonstrated in this
article. Therefore, to overcome the resource and data limitations of each
user, collaborative training and management frameworks for prompts are
urgently required. For instance, researchers have been exploring prompt-
oriented federated learning, where users contribute certain datasets and
resources to fine-tune the prompts collaboratively. Moreover, secure prompt
sharing or trading can be realized on edge blockchains.

###  V-C Multimodal Mobile-Edge AIGX

The maturity of AIGX incurs more and more multimodal tasks. However, a series
of problems are yet to be addressed. Firstly, multimodal prompts are
complicated and hard to optimize since the optimization objective should
consider the prompt performance in different modalities. Moreover, defining
the rationale QoG for multimodal mobile-edge AIGX services is also intractable
and worth research.

##  VI Conclusion

In this article, we presented the concept of mobile-edge AI-Generated
Everything (AIGX) and discussed its optimization using prompt engineering.
Specifically, we started by introducing the building blocks of AIGX, the
evolution from AIGC to AIGX, and real-world AIGX applications. Then, we
reviewed prompt engineering methods and presented a unified framework, which
incorporates prompt engineering in PFM-empowered mobile-edge AIGX services
from an optimization perspective. Finally, we performed a case study, in which
we illustrated how much improvement prompt engineering can lead to in terms of
QoE, service latency, and bandwidth usage by numerical results. As the first
article to discuss AIGX, we hope that this article could inspire researchers
to explore more applications of AIGX.

## References

  * [1] H. Du _et al._ , “Beyond deep reinforcement learning: A tutorial on generative diffusion models in network optimization,” 2023. [Online]. Available: https://arxiv.org/abs/2308.05384
  * [2] H. Zou, Q. Zhao, L. Bariah, M. Bennis, and M. Debbah, “Wireless multi-agent generative ai: From connected intelligence to collective intelligence,” 2023\. [Online]. Available: https://arxiv.org/abs/2307.02757
  * [3] M. A. Ferrag _et al._ , “Revolutionizing cyber threat detection with large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2306.14263
  * [4] P. Liu _et al._ , “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” _ACM Comput. Surv._ , vol. 55, no. 9, jan 2023.
  * [5] Y. Liu _et al._ , “Blockchain-empowered lifecycle management for ai-generated content (aigc) products in edge networks,” _IEEE Wireless Communications_ , accepted, 2023, https://arxiv.org/pdf/2303.02836.pdf.
  * [6] Y. Hao, Z. Chi, L. Dong, and F. Wei, “Optimizing prompts for text-to-image generation,” 2022. [Online]. Available: https://arxiv.org/abs/2212.09611
  * [7] The collection of mainstream pfms in different fields. 2023. [Online]. Available: https://github.com/Lancelot1998/Prompt-Engineering
  * [8] M. U. Khattak, H. Rasheed, M. Maaz, S. Khan, and F. S. Khan, “Maple: Multi-modal prompt learning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2023, pp. 19 113–19 122.
  * [9] M. Jia _et al._ , “Visual prompt tuning,” in _Computer Vision – ECCV 2022_ , 2022, pp. 709–727.
  * [10] G. Liu _et al._ , “Semantic communications for artificial intelligence generated content (aigc) toward effective content creation,” 2023. [Online]. Available: https://arxiv.org/abs/2308.04942
  * [11] G. Destefanis, S. Bartolucci, and M. Ortu, “A preliminary analysis on the code generation capabilities of gpt-3.5 and bard ai models for java functions,” 2023\. [Online]. Available: https://arxiv.org/abs/2305.09402
  * [12] T. Brown _et al._ , “Language models are few-shot learners,” in _Advances in Neural Information Processing Systems_ , vol. 33, 2020, pp. 1877–1901.
  * [13] Z. Jiang, F. F. Xu, J. Araki, and G. Neubig, “How can we know what language models know?” _Transactions of the Association for Computational Linguistics_ , vol. 8, pp. 423–438, July 2020.
  * [14] Y. Chen _et al._ , “Netgpt: A native-ai network architecture beyond provisioning personalized generative services,” 2023. [Online]. Available: https://arxiv.org/abs/2307.06148
  * [15] H. Talebi and P. Milanfar, “Nima: Neural image assessment,” _IEEE transactions on image processing_ , vol. 27, no. 8, pp. 3998–4011, 2018.