11institutetext: Centre for Learning Analytics at Monash, Faculty of
Information Technology, Monash University, Clayton, Victoria, Australia

# Practical and Ethical Challenges of Large Language Models in Education: A
Systematic Literature Review

Lixiang Yan 11 Lele Sha 11 Linxuan Zhao 11 Yuheng Li 11 Roberto Martinez-
Maldonado 11 Guanliang Chen 11 Xinyu Li 11 Yueqiao Jin 11 Dragan Gašević 11

###### Abstract

Educational technology innovations that have been developed based on large
language models (LLMs) have shown the potential to automate the laborious
process of generating and analysing textual content. While various innovations
have been developed to automate a range of educational tasks (e.g., question
generation, feedback provision, and essay grading), there are concerns
regarding the practicality and ethicality of these innovations. Such concerns
may hinder future research and the adoption of LLMs-based innovations in
authentic educational contexts. To address this, we conducted a systematic
literature review of 118 peer-reviewed papers published since 2017 to pinpoint
the current state of research on using LLMs to automate and support
educational tasks. The practical and ethical challenges of LLMs-based
innovations were also identified by assessing their technological readiness,
model performance, replicability, system transparency, privacy, equality, and
beneficence. The findings were summarised into three recommendations for
future studies, including updating existing innovations with state-of-the-art
models (e.g., GPT-3), embracing the initiative of open-sourcing
models/systems, and adopting a human-centred approach throughout the
developmental process. These recommendations could support future research to
develop practical and ethical innovations for supporting diverse educational
tasks and benefiting students, teachers, and institutions.

###### Keywords:

Large Language Models Pre-trained Language Models Educational Data Mining
Artificial Intelligence Education GPT BERT ChatGPT Natural Language Processing
.

## Practitioner notes

What is currently known about this topic

  * •

Generating and analysing text-based educational content are time-consuming and
laborious tasks.

  * •

Large language models are capable of efficiently analysing an unprecedented
amount of textual content and completing complex natural language processing
and generation tasks.

  * •

Large language models have been increasingly used to develop educational
technologies that aim to automate the generation and analysis of textual
content.

What this paper adds

  * •

A comprehensive list of different educational tasks that could potentially
benefit from LLMs-based innovations through automation.

  * •

A structured assessment of the practicality and ethicality of existing LLMs-
based innovations from seven important aspects using established frameworks.

  * •

Three recommendations that could potentially support future studies to develop
LLMs-based innovations that are practical and ethical to implement in
authentic educational contexts.

Implications for practitioners

  * •

Updating existing innovations with state-of-the-art models may further reduce
the amount of manual effort required for adapting existing models to different
educational tasks.

  * •

The reporting standards of empirical research that aims to develop educational
technologies using large language models need to be improved.

  * •

Adopting a human-centred approach throughout the developmental process could
contribute to resolving the practical and ethical challenges of large language
models in education.

##  1 Introduction

Advancements in artificial intelligence (AI) and large language models (LLMs)
have fueled the development of many educational technology innovations that
aim to automate the often time-consuming and laborious tasks of generating and
analysing textual content (e.g., generating open-ended questions and analysing
student feedback survey) [26, 64]. These LLMs, such as Bidirectional Encoder
Representations from Transformers (BERT) [14] and Generative Pre-trained
Transformer (GPT) [8], utilise deep learning and self-attention mechanisms
[62] to selectively attend to the different parts of input texts, depending on
the focus of the current tasks, allowing the model to learn complex patterns
and relationships among textual contents, such as their semantic, contextual,
and syntactic relationships [34]. As several LLMs (e.g., GPT-3 and Codex) have
been pre-trained on massive amounts of data across multiple disciplines, they
are capable of completing natural language processing tasks with little (few-
shot learning) or no additional training (zero-shot learning) [8]. This could
lower the technological barriers to LLMs-based innovations as researchers and
practitioners can develop new educational technologies by fine-tuning LLMs on
specific educational tasks without starting from scratch. The recent release
of ChatGPT, an LLMs-based generative AI chatbot that requires only natural
language prompts without additional model training or fine-tuning [38], has
further lowered the barrier for individuals without technological background
to leverage the generative powers of LLMs.

Although educational research that leverages LLMs to develop technological
innovations for automating educational tasks is yet to achieve its full
potential (i.e., most works have focused on improving model performances [29,
42]), a growing body of literature hints at how different stakeholders could
potentially benefit from such innovations. Specifically, these innovations
could potentially play a vital role in addressing teachers’ high levels of
stress and burnout by reducing their heavy workloads by automating punctual,
time-consuming tasks [10] such as question generation [29], feedback provision
[11], scoring essays [42] and short answers [69]. These innovations could also
potentially benefit both students and institutions by improving the efficiency
of often tedious administrative processes such as learning resource
recommendation, course recommendation and student feedback evaluation,
potentially [68, 64].

Despite the growing empirical evidence of LLMs’ potential in automating a wide
range of educational tasks, none of the existing work has systematically
reviewed the practical and ethical challenges of these LLMs-based innovations.
Understanding these challenges is essential for developing responsible
technologies as LLMs-based innovations (e.g., ChatGPT) could contain human-
like biases based on the existing ethical and moral norms of society, such as
inheriting biased and toxic knowledge (e.g., gender and racial biases) when
trained on unfiltered internet text data [48]. Prior systematic literature
reviews have focused on investigating these issues related to one specific
application scenario of LLMs-based innovations (e.g., question generation,
essay scoring, chatbots, or automated feedback) [29, 11, 64, 42]. The
practical and ethical challenges of LLMs in automating different types of
educational tasks remain unclear. Understanding these challenges is essential
for translating research findings into educational technologies that
stakeholders (e.g., students, teachers, and institutions) can use in authentic
teaching and learning practices [1].

The current study is the first systematic literature review that aimed to
address this gap by reviewing the current state of research on using LLMs to
automate educational tasks and identify the practical and ethical challenges
of adopting these LLMs-based innovations in authentic educational contexts. A
total of 118 peer-reviewed publications from four prominent databases were
included in this review following the Preferred Reporting Items for Systematic
reviews and Meta-Analyses (PRISMA) [39] protocol. An inductive thematic
analysis was conducted to extract details regarding the different types of
educational tasks, stakeholders, LLMs, and machine learning tasks investigated
in prior literature. The practicality of LLMs-based innovations was assessed
through the lens of technological readiness, model performance, and model
replicability. Lastly, the ethicality of these innovations was assessed by
investigating system transparency, privacy, equality, and beneficence.

The contribution of this paper to the educational technology community is
threefold: 1) we systematically summarise a comprehensive list of 53 different
educational tasks that could potentially benefit from LLMs-based innovations
through automation, 2) we present a structured assessment of the practicality
and ethicality of existing LLMs-based innovations based on seven important
aspects using established frameworks (e.g., the transparency index [12]), and
3) we propose three recommendations that could potentially support future
studies to develop LLMs-based innovations to be practically and ethically
implement in authentic educational contexts.

##  2 Background

In this section, we first establish the definitions for the key terminologies,
specifically the definitions of practicality and ethicality in the context of
educational technology. We then provided an overview of prior systematic
literature reviews on LLMs in education. Then, we present the research
questions based on the gaps identified in the existing literature.

###  2.1 Practicality

Several theoretical frameworks have been proposed regarding the practicality
of integrating technological innovations in educational settings. For example,
Ertmer’s [18] first- and second-order barriers to change focused on the
external conditions of the educational system (e.g., infrastructure readiness)
and teachers’ internal states (e.g., personal beliefs). Becker [5] further
suggested that for technological innovations to have actual benefits in
supporting pedagogical practices, these innovations should be convenient to
access, support constructivist pedagogical beliefs, be adaptable to changes in
the curriculum, and be compatible to teachers’ level of knowledge and skills.
These factors were also presented in an earlier framework of the practicality
index [16], which summarised three critical components for integrating
educational technologies, including the degree of adoption feasibility, the
cost and benefit ratio, and the alignment with existing practices and beliefs.
Based on these prior theoretical frameworks and considering the recentness of
LLMs-based innovations (which only emerged in the past five years), the
practical challenges of LLMs-based innovations in automating educational tasks
can be assessed from three primary perspectives. First, evaluating the
technological readiness of these innovations is essential for determining
whether there is empirical evidence to support successful integration and
operation in authentic educational contexts. Second, assessing the model
performance could contribute valuable insights into the cost and benefits of
adopting these innovations, such as comparing the benefits of automation with
the costs of inaccurate predictions. Finally, understanding whether these
innovations are methodologically replicable could be important for future
studies to investigate their alignment with different educational contexts and
stakeholders. We elaborated on the evaluation items for each challenge in
Section 3.2.

###  2.2 Ethicality

Ethical AI is a prevalent topic of discussion in multiple communities, such as
learning analytics, AI in education, educational data mining, and educational
technology communities [1, 40]. There are ongoing debates regarding AI ethics
in education with a mixture of focuses on algorithmic and human ethics among
educational data mining and AI in education communities [24]. As such debates
continue, it is difficult to identify an established definition of ethical AI
from these fields. Whereas, ethicality has already been thoroughly
investigated and addressed in a closed field to AI in education, namely, the
field of learning analytics [40, 50]. Drawing on the established definition of
ethicality from the field of learning analytics [40], the ethicality of LLMs-
based innovations can thus be defined as the systematisation of appropriate
and inappropriate functionalities and outcomes of these innovations, as
determined by all stakeholders (e.g., students, teachers, parents, and
institutions). For example, Khosravi et al. [27] explained that the ethicality
of AI-powered educational technology systems needs to involve the
consideration of accountability, explainability, fairness, interpretability,
and safety of these systems. These different domains of ethical AI are all
closely related and can be addressed by considering system transparency.
Transparency is a subset of ethical AI that involves making all information,
decisions, decision-making processes, and assumptions available to
stakeholders, which in turn enhances their comprehension of the AI systems and
related outputs [12]. Additionally, for LLMs-based innovations, Weidinger et
al. [63] suggested six types of ethical risks, including 1) discrimination,
exclusion, and toxicity, 2) information hazards, 3) misinformation harms, 4)
malicious uses, 5) human-computer interaction harms, and 6) automation,
access, and environmental harms. These risks can be further aggregated into
three fundamental ethical issues, such as privacy concerns regarding
educational stakeholders’ personal data, equality concerns regarding the
accessibility of stakeholders with different backgrounds, and beneficence
concerns about the potential harms and negative impacts that LLMs-based
innovations may have on stakeholders [19]. These three fundamental ethical
issues were considered in the analysis of the reviewed literature, further
details were available in Section 3.2.

###  2.3 Related Work

Prior systematic literature reviews have focused primarily on reviewing a
specific application scenario (e.g., question generation, automated feedback,
chatbots and essay scoring) of natural language processing and LLMs. For
example, Kurdi et al. [29] have systematically reviewed empirical studies that
aimed to tackle the problem of automatic question generation in educational
domains. They comprehensively summarised the different generation methods,
generation tasks, and evaluation methods presented in prior literature. In
particular, LLMs could potentially benefit the semantic-based approaches for
generating meaningful questions that are closely related to the source
contents. Likewise, Cavalcanti et al. [11] have systematically reviewed
different automated feedback systems regarding their impacts on improving
students’ learning performances and reducing teachers’ workloads. Despite half
of their reviewed studies showing no evidence of reducing teachers’ workloads,
as these automated feedback systems were mostly rule-based and required
extensive manual efforts, they identified that using natural language
generation techniques could further enhance such systems’ generalisability and
potentially reduce manual workloads. On the other hand, Wollny et al. [64]
have systematically reviewed areas of education where chatbots have already
been applied. They concluded that there is still much to be done for chatbots
to achieve their full potential, such as making them more adaptable to
different educational contexts. A systematic literature review has also
investigated the various automated essay scoring systems [42]. The findings
have revealed multiple limitations of the existing systems based on
traditional machine learning (e.g., regression and random forest) and deep
learning algorithms (e.g., LSTM and BERT). In sum, these previous systematic
literature reviews have identified room for improvement that can be
potentially addressed using state-of-the-art LLMs (e.g., GPT-3 or Codex).
However, none of the prior systematic literature reviews has investigated the
practical and ethical issues related to LLMs-based innovations in education
generally rather than particularly (e.g., limited to a specific task).

The recent hype around one of the latest LLMs-based innovations, ChatGPT, has
intensified the discussion about the practical and ethical challenges related
to using LLMs in education. For example, in a position paper, Kasneci et al.
[26] provided an overview of some existing LLMs research and proposed several
practical opportunities and challenges of LLMs from students’ and teachers’
perspectives. Likewise, Rudolph et al. [43] also provided an overview of the
potential impacts, challenges, and opportunities that ChatGPT might have on
future educational practices. Although these studies have not systematically
reviewed the existing educational literature on LLMs, their arguments
resonated with some of the pressing issues around LLMs and ethical AI, such as
data privacy, bias, and risks. On the other hand, Sallam [44] systematically
reviewed the implications and limitations of ChatGPT in healthcare education
and identified potential utility around personalisation and automation.
However, it is worth noting that most papers reviewed in Sallam’s study were
either editorials, commentaries, or preprints. This lack of peer-reviewed
empirical studies on ChatGPT is understandable as it has only been released
since late 2022 [38]. None of the existing work has systematically reviewed
the peer-reviewed literature on prior LLMs-based innovations, such
investigations could provide more reliable and empirically-based evidence
regarding the potential opportunities and challenges of LLMs on educational
practices. Thus, the current study aimed to address this gap in the literature
by conducting a systematic literature review of prior educational research on
LLMs. Specifically, the following research questions were investigated to
guide this review:

  * •

RQ1: What is the current state of research on using LLMs to automate
educational tasks, specifically through the lens of educational tasks,
stakeholders, LLMs, and machine-learning tasks111Such as classification,
prediction, clustering, etc.?

  * •

RQ2: What are the practical challenges of LLMs in automating educational
tasks, specifically through the lens of technological readiness, model
performance, and model replicability?

  * •

RQ3: What are the ethical challenges of LLMs in automating educational tasks,
specifically through the lens of system transparency, privacy, equality, and
beneficence?

##  3 Methods

###  3.1 Review Procedures

We followed the PRISMA [39] protocol to conduct the current systematic
literature review of LLMs. We searched four reputable bibliographic databases,
including Scopus, ACM Digital Library, IEEE Xplore, and Web of Science, to
find high-quality peer-reviewed publications. Additional searches were
conducted through Google Scholar and Education Resources Information Center
(ERIC) to identify peer-reviewed publications that have yet to be indexed by
these databases, either recently published or not indexed (e.g., Journal of
Educational Data Mining; prior to 2020). Our initial search query for the
title, abstract, and keywords included terms such as ”large language model”,
”pre*trained language model”, ”GPT-*”, ”BERT”, ”education”, ”student*”, and
”teacher*”. A publication year constraint was also applied to restrict the
search to studies published since 2017, specifically from 01/01/2017 to
12/31/2022, as the foundational architecture (Transformer) of LLMs was
formally released in 2017 [62].

Two researchers independently reviewed the titles and abstracts of eligible
articles based on five predetermined inclusion and exclusion criteria. First,
we included studies that used large or pre-trained language models directly or
built on top of such models, and excluded studies that used general machine-
learning or deep-learning models with unspecified usage of LLMs. Second, we
included empirical studies with detailed methodologies, such as a detailed
description of the LLMs and research procedures, and excluded review, opinion,
and scoping works. Third, we only included full-length peer-reviewed papers,
and excluded short, workshop, and poster papers that were less than six and
eight pages for double- and single-column layouts, respectively. Additionally,
we included studies that used LLMs for the purpose of automating educational
tasks (e.g., essay grading and question generation), and excluded studies that
merely used LLMs as part of the analysis without educational implications.
Finally, we only included studies that were published in English (both the
abstract and the main text) and excluded studies that were published in other
languages. Any conflicting decisions were resolved through further discussion
between the two researchers or consulting with a third researcher to achieve a
consensus.

The database search initially yielded 854 publications, with 191 duplicates
removed, resulting in 663 publications for the title and abstract screening
(see Figure 1). After the title and abstract screening, 197 articles were
included for the full-text review with an inter-rater reliability (Cohen’s
kappa) of 0.75, indicating substantial agreement between the reviewers. A
total of 118 articles were selected for data extraction after the full-text
review with an inter-rater reliability (Cohen’s kappa) of 0.73, indicating
substantial agreement between the reviewers. Out of the initial 197 articles,
79 were excluded for various reasons, including not full paper (n=41), lack of
educational automation (n=17), lack of pre-trained or LLMs (n=12), merely
using pre-trained or LLMs as part of the analysis (n=3), non-English paper
(n=2), and non-empirical paper (n=2).

Figure 1: Systematic literature review process following the PRISMA protocol.

###  3.2 Data Analysis

For the first research question (RQ1), we conducted an inductive thematic
analysis to extract information regarding the current state of research on
using LLMs to automate educational tasks. Specifically, we extracted four
primary types of contextual information from each included paper: educational
tasks, stakeholders, LLMs, and machine-learning tasks. This contextual
information would provide a holistic view of the existing research and inform
researchers and practitioners regarding the viable directions to explore with
the state-of-the-art LLMs (e.g., GPT-3.5 and Codex).

A total of seven data extraction items were developed to address the second
and third research questions. These items were developed as they are directly
related to the definition of practicality (RQ2: Item 1-3) and ethicality (RQ3:
Item 4-7), as defined in the Background section (Section 2). The following
list elaborates on the final set of items along with the corresponding guiding
questions.

  1. 1.

Technology readiness What levels of technology readiness are the LLMs-based
innovations at? We adopted the assessment tool from the Australian government,
namely the Australian Department of Defence’s Technology Readiness Levels
(TRL) [49], which has been used to assess the maturity of educational
technologies in prior SLR [66]. There are nine different technological
readiness levels: Basic Research (TRL-1), Applied Research (TRL-2), Critical
Function or Proof of Concept Established (TRL-3), Lab Testing/Validation of
Alpha Prototype Component/Process (TRL-4), Laboratory Testing of
Integrated/Semi-Integrated System (TRL-5), Prototype System Verified (TRL-6),
Integrated Pilot System Demonstrated (TRL-7), System Incorporated in
Commercial Design (TRL-8), and System Proven and Ready for Full Commercial
Deployment (TRL-9), further explained in the Result section.

  2. 2.

Performance: How accurate and reliable can the LLMs-based innovations complete
the designated educational tasks? For example, what are the model performance
scores for classification (e.g., AUC and F1 scores), generation (e.g., BLEU
score), and prediction tasks (e.g., RMSE and Pearson’s correlation)?

  3. 3.

Replicability: Can other researchers or practitioners replicate the LLMs-based
innovations without additional support from the original authors? This item
evaluates whether the paper provided sufficient details about the LLMs (e.g.,
open-sourced algorithms) and the dataset (e.g., open-source data).

  4. 4.

Transparency: What tiers of transparency index [12] are the LLMs-based
innovations at? The transparency index proposed three tiers of transparency,
including transparent to AI researchers and practitioners (Tier 1),
transparent to educational technology experts and enthusiasts (Tier 2), and
transparent to educators and parents (Tier 3). The tier of transparency
increases as educational stakeholders become fully involved in developing and
evaluating the AI system. These tiers were further elaborated on in the
Results section.

  5. 5.

Privacy: Has the paper mentioned or considered privacy issues of their
innovations? This item explores potential issues related to informed consent,
transparent data collection, individuals’ control over personal data, and
unintended surveillance [19, 61].

  6. 6.

Equality: Has the paper mentioned or considered equal access to their
innovations? This item explores potential issues related to limited access for
students from low-income backgrounds or rural areas and the linguistic
limitation of the innovations, such as their capability to analyse different
languages [19].

  7. 7.

Beneficence: Has the paper mentioned or considered potential issues that
violate the ethical principle of beneficence? Such violations may include the
risks associated with labelling and profiling students, inadequate usage of
machine-generated content for assessments, and algorithmic biases [19, 68].

##  4 Results

###  4.1 The Current State — RQ1

We identified nine different categories of educational tasks that prior
studies have attempted to automate using LLMs (as shown in Table 1). Prior
studies have used LLMs to automate the profiling and labelling of 17 types of
education-related contents and concepts (e.g., forum posts, student sentiment,
and discipline similarity), the detection of six latent constructs (e.g.,
confusion and urgency), the grading of five types of assessments (e.g., short
answer questions and essays), the development of five types of teaching
support (e.g., conversation agent and intelligent question-answering), the
prediction of five types of student-orientated metrics (e.g., dropout and
engagement), the construction of four types of knowledge representations
(e.g., knowledge graph and entity recognition), the provision of four
different forms of feedback (e.g., real-time and post-hoc feedback), the
generation of four types of content (e.g., MCQs and open-ended questions), and
the delivery of three types of recommendations (e.g., resource and course). Of
the 118 reviewed studies, 85 studies aimed to automate educational tasks
related to teachers (e.g., question grading and generation), 54 studies
targeted student-related activities (e.g., feedback and resource
recommendation), 20 studies focused on supporting institutional practices
(e.g., course recommendations and discipline planning), and 14 studies
empowered researchers with automated methods to investigate latent constructs
(e.g., student confusion) and capture verbal data (e.g., speech recognition).

We identified five categories of LLMs used in prior studies to automate
educational tasks. BERT and its variations (e.g., RoBERTa, DistilBERT,
multilingual BERT, LaBSE, EstBERT, and Sentence-BERT) were the most
predominant model used in 109 reviewed studies. However, they often required
manual effort for fine-tuning (n=90). GPT-2 and GPT-3 have been used in five
and three studies, respectively. OpenAI’s Codex has been used in two prior
studies, specifically for code generation tasks. T5 has also been used in two
prior studies for classification and generation purposes. In terms of machine-
learning tasks, 74 studies used LLMs to perform classification tasks.
Generation and prediction tasks were investigated in 24 and 23 prior studies,
respectively. In sum, LLMs-based innovations have already been used to
automate a range of educational tasks, but most of these innovations were
developed on older models, such as BERT and GPT-2. Although state-of-the-art
models, such as GPT-3, have been introduced for over two years [8], they have
yet to be widely applied to automate educational tasks. A potential reason for
this lack of adoption could be these models’ commercial and close-sourced
nature, increasing the financial burdens of developing and operating
educational technology innovations on top of such models.

Table 1: Educational Tasks in LLMs Research Categories |  Educational Tasks
---|---
Profiling and Labelling |  Forum post classification, dialogue act classification, classification of learning designs, review sentiment analysis, topic modelling, pedagogical classification of MOOCs, collaborative problem-solving modelling, paraphrase quality, speech tagging, labelling educational content with knowledge components, key sentence and keyword extraction, reflective writing analysis, multimodal representational thinking, discipline similarity, concept classification, cognitive level classification, essay arguments segmentation
Detection |  Semantic analyses, detecting off-task messages, confusion detection, urgency detection, conversational intent detection, teachers’ behaviour detection
Assessment and Grading |  Formative and summative assessment grading, short answer grading, essay grading, subjective question grading, student self-explanation
Teaching Support |  Classroom teaching, learning community support, online learning conversation agent, intelligent question-answering, teacher activity recognition
Prediction |  Student performance prediction, student dropout prediction, emotional and cognitive engagement detection, growth and development indicators for college students, at-risk student identification
Knowledge Representation |  Knowledge graph construction, knowledge entity recognition, knowledge tracing, cause-effect relation extraction
Feedback |  Real-time feedback, post-hoc feedback, aggregated feedback, feedback on feedback (peer-review comments)
Content Generation |  MCQs generation, open-ended question generation, code generation, reply (natural language) generation
Recommendation |  English reference selection and recommendation, resource recommendation, course recommendation

###  4.2 Practical Challenges — RQ2

####  4.2.1 Technology readiness

According to the Technology Readiness Level scale [49], the LLMs-based
innovations are still in the early development and testing stage. Over three-
quarters of the LLMs studies (n=89) are in the applied research stage (TRL-2),
which aims to experiment with the capability of LLMs in automating different
educational tasks by developing different models and combining LLMs with other
machine-learning and deep-learning techniques (e.g., RCNN [55]). Thirteen
studies have established a proof of concept and demonstrated the feasibility
of using LLMs-based innovations to automate certain processes of educational
tasks (TRL-3). Nine studies have developed functional prototypes and conducted
preliminary validation under controlled laboratory settings (TRL-4), often
involving stakeholders (e.g., students and teachers) to test and evaluate the
output of their innovations. Only seven studies have taken a further step and
conducted validation studies in authentic learning environments, with most
functional components integrated into the educational tasks (TRL-5), such as
an intelligent virtual standard patient for medical students training [57] and
an intelligent chatbot for university admission [37]. Yet, none of the
existing LLMs-based innovations has been verified through successful
operations (TRL-6). Together, these findings suggest although existing LLMs-
based innovations can be used to automate certain educational tasks, they have
yet to show evidence regarding improvements to teaching, learning, and
administrative processes in authentic educational practices.

####  4.2.2 Performance

The performance of LLMs-based innovations varies across different machine-
learning and educational tasks. For classification tasks, LLMs-based
innovations have shown high performance for simple educational tasks, such as
modelling the topics from a list of programming assignments (best F1 = 0.95)
[20], analysing the sentiment of student feedback (best F1 = 0.94) [59],
constructing subject knowledge graph from teaching materials (best F1 = 0.94)
[58], and classifying educational forum posts [53] (best F1 = 0.92). However,
the classification performance of LLMs-based innovations decreases for other
educational tasks. For example, the F1 scores for detecting student confusion
in the course forum [22] and students’ off-task messages in game-based
collaborative learning [9] are around 0.77 and 0.67, respectively. Likewise,
the F1 score for classifying short-answer responses varies between 0.61 to
0.82, with the lower performance on out-of-sample questions (best F1 = 0.61)
[13]. Similar performances were also observed in classifying students’
argumentative essays (best F1 = 0.66) [23].

For prediction tasks, LLMs-based innovations have demonstrated reliable
performance compared to ground truth or human raters. For example, LLMs-based
innovations have achieved high scores of quadratic weighted kappa (QWK) in
essay scoring, specifically for off-topic (QWK = 0.80), gibberish (QWK =
0.80), and paraphrased answers (QWK = 0.94), indicating substantial to almost
perfect agreements with human raters [15]. Similar performances on essay
scoring have been observed in several other studies (e.g., 0.80 QWK in [6] and
0.81 QWK in [56]). Likewise, LLMs-based innovations’ performances on automatic
short-answer grading were also highly correlated with human ratings (Pearson’s
correlation between 0.75 to 0.82) [2, 46].

Regarding generation tasks, LLMs-based innovations demonstrated high
performance across different educational tasks. For example, LLMs-based
innovations have achieved an F1 score of 0.92 for generating MCQs with single-
word answers [28]. Educational technologies developed by fine-tuning Codex
also demonstrated the capability of resolving 81% of the advanced mathematics
problems [17]. Text summaries generated using BERT had no significant
differences compared with student-generated summaries and can not be
differentiated by graduate students [33]. Similarly, BERT-generated doctor-
patient dialogues were also found to be indistinguishable from actual doctor-
patient dialogues, which can be used to create virtual standard patients for
medical students’ diagnosis practice training [57]. Additionally, for
introductory programming courses, the state-of-the-art LLMs, Codex, could
generate sensible and novel exercises for students along with an appropriate
sample solution (around three out of four times) and accurate code explanation
(67% accuracy) [45].

In sum, although the classification performance of LLMs-based innovations on
complex educational tasks is far from suitable for practical adoption, LLMs-
based innovations have already shown high performance on several relatively
simple classification tasks that could potentially be deployed to
automatically generate meaningful insights that could be useful to teachers
and institutions, such as navigating through numerous student feedback and
course review. Likewise, LLMs-based innovations’ prediction and generation
performance reveals a promising future of potentially automating the
generation of educational content and the initial grading of student
assessments. However, ethical issues must be considered for such
implementations, which we covered in the findings for RQ3.

####  4.2.3 Replicability

Most reviewed studies (n=107) have not disclosed sufficient details about
their methodologies for other researchers and practitioners to replicate their
proposed LLMs-based innovations. Among these studies, 12 studies have open-
sourced the original code for developing the innovations but failed to open-
source the data they used. In contrast, 20 studies have open-sourced the data
they used but failed to release the actual code. Around two-thirds of the
reviewed studies (n=75) have failed to release both the original code and the
data they used, leaving only 11 studies publicly available for other
researchers and practitioners to replicate without needing to conduct the
original authors. This lack of replicability could become a vital barrier to
adoption, as 87 out of the 107 non-replicable studies required fine-tuning the
LLMs to achieve the reported performance. This replication issue also limits
others from further evaluating the generalisability of the proposed LLMs-based
innovations in other datasets, constraining potential practical utilities.

###  4.3 Ethical Challenges — RQ3

####  4.3.1 Transparency

Based on the transparency index and the three tiers of transparency [12], most
of the reviewed study reached at-most Tier 1 (n=109), which is merely
considered transparent to AI researchers and practitioners. Although these
studies reported details regarding their machine learning models (e.g.,
optimisation and hyperparameters), such information is unlikely to be
interpretable and considered transparent for individuals without a strong
background in machine learning. For the remaining nine studies, they reached
at-most Tier 2 as they often involved some form of human-in-the-loop elements.
Specifically, making the LLMs innovations available for student evaluation has
been found in three studies [37, 57, 33]. Such evaluations often involved
students differentiating AI-generated from human-generated content [57, 33]
and assessing student satisfaction with AI-generated responses [37]. Likewise,
two studies have involved experts in evaluating specific features of the
content generated by the LLMs-based innovations, such as informativeness [31]
and cognitive level [36]. Surveys have been used to evaluate students’
experience with LLMs-based innovations from multiple perspectives, such as the
quality and difficulty of AI-generated questions [17, 30] and potential
learning benefits of the systems [25]. Finally, semi-structured interviews
have been conducted to understand students’ perception of the LLM system after
using the system in authentic computer-supported collaborative learning
activities [70]. Although these nine studies had some elements of human-in-
the-loop, stakeholders were often involved in a post-hoc evaluation manner
instead of throughout the development process, and thus, have limited
knowledge regarding the operating principle and potential weakness of the
systems. Consequently, none of the existing LLMs-based innovations can be
considered as being at Tier 3, which describes an AI system that is considered
transparent for educational stakeholders (e.g., students, teachers, and
parents).

####  4.3.2 Privacy

The privacy issues related to LLMs-based innovations were rarely attended to
or investigated in the reviewed studies. Specifically, for studies that have
fine-tuned LLMs with textual data collected from students, none of these
studies has explicitly explained their consenting strategies (e.g., whether
students acknowledge the collection and intended usage of their data) and data
protection measures (e.g., data anonymisation and sanitisation). This lack of
attention to privacy issues is particularly concerning as LLMs-based
innovations work with stakeholders’ natural languages that may contain
personal and sensitive information regarding their private lives and
identities [7]. It is possible that stakeholders might not be aware of their
textual data (e.g., forum posts or conversations) on digital platforms (e.g.,
MOOCs and LMS) being used in LLMs-based innovations for different purposes of
automation (e.g., automated reply and training chatbots) as the consenting
process is often embedded into the enrollment or signing up of these platforms
[60]. This process can hardly be considered informed consent. Consequently, if
stakeholders shared their personal information on these platforms in natural
language (e.g., sharing phone numbers and addresses with group members via
digital forums), such information could be used as training data for fine-
tuning LLMs. This usage could potentially expose private information as LLMs
are incapable of understanding the context and sensitivity of text, and thus,
could return stakeholders’ personal information based on semantic
relationships [7].

####  4.3.3 Equality

Although most of the studies (n=95) used LLMs that only apply to English
content, we also identified application scenarios of LLMs in automating
educational tasks in 12 other languages. Specifically, 19 studies used LLMs
that can be applied to Chinese content. Ten prior studies used LLMs for
Vietnamese (n=3), Spanish (n=3), Italian (n=2), and German (n=2) contents.
Additionally, seven studies applied LLMs to Croatian, Indonesian, Japanese,
Romanian, Russian, Swedish, and Hindi content. While the dominance of English-
based innovations remains a concerning equality issue, the availability of
innovations that support a variety of other languages, specifically in none
western, educated, industrialized, rich and democratic (WEIRD) societies
(e.g., Indonesia and Vietnam), may indicate a promising sign for LLMs-based
innovations to have potential global impacts and levels such equality issues
in the future. However, the financial burdens from adopting the state-of-the-
art models (e.g., OpenAI’s GPT-3 and Codex) could potentially exacerbate the
equality issues, making the best-performing innovations only accessible and
affordable to WEIRD societies.

####  4.3.4 Beneficence

A total of seven studies have discussed potential issues related to the
violation of the ethical principle of beneficence. For example, one study has
discussed the potential risk of adopting underperforming models, which could
negatively affect students’ learning experiences [30]. Such issues could be
minimised by deferring decisions made by such models [47] and labelling the
AI-generated content with a warning message (e.g., teachers’ manual revision
is mandatory before determining the actual correctness) [3]. Apart from issues
with adopting inaccurate models, two studies have suggested that potential
bias and discrimination issues may occur if adopting a model that is accurate
but unfair [54, 33]. This issue is particularly concerning as most existing
studies focused solely on developing an accurate model. Only nine reviewed
studies released information regarding the descriptive data of different
sample groups, such as gender and ethicality (e.g., [41]). Two studies have
proposed potential approaches that could address such fairness issues.
Specifically, using sampling strategies, such as balancing demographic
distribution, has been found as an effective approach to improve both model
fairness and accuracy [52, 51]. These approaches are essential for ensuring
that LLMs-based innovations will not perpetuate problematic and systematic
biases (e.g., gender biases), especially as the best-performing LLMs are often
black-boxed with little interpretability, traceability, and justification of
the results [65].

##  5 Discussion

###  5.1 Main Findings

The current study systematically reviewed 118 peer-reviewed empirical studies
that used LLMs to automate educational tasks. For the first research question
(RQ1), we illustrated the current state of educational research on LLMs.
Specifically, we identified 53 types of application scenarios of LLMs in
automating educational tasks, summarised into nine general categories,
including profiling and labelling, detection, assessment and grading, teaching
support, prediction, knowledge representation, feedback, content generation,
and recommendation. While some of these categories resonate with the utilities
proposed in prior positioning works (e.g., feedback, content generation, and
recommendation) [26, 43], novel directions such as using LLMs to automate the
creation of knowledge graph and entity further indicated the potential of
LLMs-based innovations in supporting institutional practices (e.g., creating
knowledge-based search engines across multiple disciplines). These identified
directions could benefit from the state-of-the-art LLMs (e.g., GPT-3 and
Codex) as most of the reviewed studies (92%) focused on using BERT-based
models, which often required manual effort for fine-tuning. Whereas, the
state-of-the-art LLMs could potentially achieve similar performance with a
zero-short approach [4]. While the majority of the reviewed studies (63%)
focused on using LLMs to automate classification tasks, there could be more
future studies that aimed to tackle the automation of prediction and
generation tasks with the more capable LLMs [44]. Likewise, although
supporting teachers are the primary focus (72%) of the existing LLMs-based
innovations, students and institutions could also benefit from such
innovations as novel utilities could continue to emerge from the educational
technology literature.

Regarding the second research question (RQ2), we identified several practical
challenges that need to be addressed for LLMs-based innovations to have actual
educational benefits. The development and educational research on LLMs-based
innovations are still in the early stages. Most of the innovations
demonstrated a low level of technology readiness, where the innovations have
yet to be fully integrated and validated in authentic educational contexts.
This finding resonates with previous systematic literature reviews on related
educational technologies, such as reviews on automated question generation
[29], feedback provision [11], essay scoring [42], and chatbot systems [64].
There is a pressing need for in-the-wild studies that provide LLMs-based
innovations directly to educational stakeholders for supporting actual
educational tasks instead of testing on different datasets or in laboratory
settings. Such authentic studies could also validate whether the existing
innovations can achieve the reported high model performance in real-life
scenarios, specifically in prediction and generation tasks, instead of being
limited to prior datasets. This validation process is vital for preventing
inadequate usage, such as adopting a subject-specific prediction model for
unintended subjects. Researchers need to carefully examine the extent of
generalisability of their innovations and inform the limitations to
stakeholders [21]. However, addressing such needs could be difficult
considering the current literature’s poor replicability, which increases the
barriers for others to adopt LLMs-based innovations in authentic educational
contexts or validate with different samples. Similar replication issues have
also been identified in other areas of educational technology research [66].

For the third research question (RQ3), we identified several ethical
challenges regarding LLMs-based innovations. In particular, most of the
existing LLMs-based innovations (92%) were only transparent to AI researchers
and practitioners (Tier 1), with only nine studies that can be considered
transparent to educational technology experts and enthusiasts (Tier 2). The
primary reason behind this low transparency can be attributed to the lack of
human-in-the-loop components in prior studies. This finding resonates with the
call for explainable and human-centred AI, which stresses the vital role of
stakeholders in developing meaningful and impactful educational technology
[27, 67]. Involving stakeholders during the development and evaluation of
LLMs-based innovations is essential for addressing both practical and ethical
issues. For example, as the current findings revealed, LLMs-based innovations
are subject to data privacy issues but were rarely mentioned or investigated
in the literature [33], which may be due to the little voice that stakeholders
had in prior research. The several concerning issues around beneficence also
demand the involvement of stakeholders as their perspectives are vital for
shaping the future directions of LLMs-based innovations, such as how
responsible decisions can be made with these AI systems [47]. Likewise, the
equality issue regarding the financial burdens that may occur when adopting
innovations that leverage commercial LLMs (e.g., GPT-3 and Codex) can also be
further studied with institutional stakeholders.

###  5.2 Implications

The current findings have several implications for education research and
practice with LLMs, which we have summarised into three recommendations that
aim to support future studies to develop practical and ethical innovations
that can have actual benefits to educational stakeholders. First, the wide
range of application scenarios of LLMs-based innovations can further benefit
from the improvements in the capability of LLMs. For example, updating
existing innovations with state-of-the-art LLMs may further reduce the amount
of manual effort required for fine-turning and achieve similar performances
[4]. However, researchers should also consider the additional financial and
resource burdens on educational stakeholders when updating their innovations
with the latest LLMs, especially the commercial ones (e.g., GPT-3 and
ChatGPT). Second, for LLMs-based innovations to achieve a high level of
technology readiness and performance, the current reporting standards must be
improved. Future studies should support the initiative of open-sourcing their
models/systems when possible and provide sufficient details about the test
datasets, which are essential for others to replicate and validate existing
innovations across different contexts, preventing the potential pitfall of
another replication crisis [32]. Finally, adopting a human-centred approach
when developing and evaluating LLMs-based innovations are essential for
ensuring these innovations remain ethical in practice, especially as ethical
principles may not guarantee ethical AI due to their top-down manners (e.g.,
developed by regulatory bodies) [35]. Future studies need to consider the
ethical issues that may arise from their specific application scenarios and
actively involve stakeholders to identify and address such issues.

###  5.3 Limitations

The current findings should be interpreted with several limitations in mind.
First, although we assessed the practicality and ethicality of LLMs-based
innovations with seven different items, there could be other aspects of these
multi-dimensional concepts that we omitted. Nevertheless, these assessment
items were chosen directly from the corresponding definitions and related to
the pressing issues in the literature [1, 63]. Second, we only included
English publications, which could have biased our findings regarding the
availability of LLMs-based innovations among different countries. Thirdly, as
we strictly followed the PRISMA protocol and only included peer-reviewed
publications, we may have omitted the emerging works published in different
open-sourced archives. These studies may contain interesting findings
regarding the latest LLMs (e.g., ChatGPT). Additionally, this review focused
on the potential of LLMs-based innovations in automating educational tasks,
and thus, other pressing issues, such as the potential threat to academic
integrity, were outside of the scope of this systematic literature review.
Finally, the transparency index that we adopted for RQ3 did not consider the
transparency to students, which could be an important direction for future
human-centred AI studies.

##  6 Conclusion

In this study, we systematically reviewed the current state of educational
research on LLMs and identified several practical and ethical challenges that
need to be addressed in order for LLMs-based innovations to become beneficial
and impactful. Based on the findings, we proposed three recommendations for
future studies, including updating existing innovations with state-of-the-art
models, embracing the initiative of open-sourcing models/systems, and adopting
a human-centred approach throughout the developmental process. These
recommendations could potentially support future studies to develop practical
and ethical innovations that can be implemented in authentic contexts to
automate a wide range of educational tasks.

## References

  * [1] Adams, C., Pente, P., Lemermeyer, G., Rockwell, G.: Artificial intelligence ethics guidelines for k-12 education: a review of the global landscape. In: Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II. pp. 24–28. Springer (2021)
  * [2] Ahmed, A., Joorabchi, A., Hayes, M.J.: On the application of sentence transformers to automatic short answer grading in blended assessment. In: 2022 33rd Irish Signals and Systems Conference (ISSC). pp. 1–6. IEEE (2022)
  * [3] Angelone, A.M., Galassi, A., Vittorini, P.: Improved automated classification of sentences in data science exercises. In: Methodologies and Intelligent Systems for Technology Enhanced Learning, 11th International Conference 11. pp. 12–21. Springer (2022)
  * [4] Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023 (2023)
  * [5] Becker, H.J.: Findings from the teaching, learning, and computing survey. Education policy analysis archives 8, 51–51 (2000)
  * [6] Beseiso, M., Alzubi, O.A., Rashaideh, H.: A novel automated essay scoring approach for reliable higher educational assessments. Journal of Computing in Higher Education 33, 727–746 (2021)
  * [7] Brown, H., Lee, K., Mireshghallah, F., Shokri, R., Tramèr, F.: What does it mean for a language model to preserve privacy? In: 2022 ACM Conference on Fairness, Accountability, and Transparency. pp. 2280–2292 (2022)
  * [8] Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems 33, 1877–1901 (2020)
  * [9] Carpenter, D., Emerson, A., Mott, B.W., Saleh, A., Glazewski, K.D., Hmelo-Silver, C.E., Lester, J.C.: Detecting off-task behavior from student dialogue in game-based collaborative learning. In: Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part I 21. pp. 55–66. Springer (2020)
  * [10] Carroll, A., Forrest, K., Sanders-O’Connor, E., Flynn, L., Bower, J.M., Fynes-Clinton, S., York, A., Ziaei, M.: Teacher stress and burnout in australia: examining the role of intrapersonal and environmental factors. Social Psychology of Education 25(2-3), 441–469 (2022)
  * [11] Cavalcanti, A.P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y.S., Gašević, D., Mello, R.F.: Automatic feedback in online learning environments: A systematic literature review. Computers and Education: Artificial Intelligence 2, 100027 (2021)
  * [12] Chaudhry, M.A., Cukurova, M., Luckin, R.: A transparency index framework for ai in education. In: Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium: 23rd International Conference, AIED 2022, Durham, UK, July 27–31, 2022, Proceedings, Part II. pp. 195–198. Springer (2022)
  * [13] Condor, A., Litster, M., Pardos, Z.: Automatic short answer grading with sbert on out-of-sample questions. International Educational Data Mining Society (2021)
  * [14] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
  * [15] Doewes, A., Pechenizkiy, M.: On the limitations of human-computer agreement in automated essay scoring. International Educational Data Mining Society (2021)
  * [16] Doyle, W., Ponder, G.A.: The practicality ethic in teacher decision-making. Interchange 8(3), 1–12 (1977)
  * [17] Drori, I., Zhang, S., Shuttleworth, R., Tang, L., Lu, A., Ke, E., Liu, K., Chen, L., Tran, S., Cheng, N., et al.: A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level. Proceedings of the National Academy of Sciences 119(32), e2123433119 (2022)
  * [18] Ertmer, P.A.: Addressing first-and second-order barriers to change: Strategies for technology integration. Educational technology research and development 47(4), 47–61 (1999)
  * [19] Ferguson, R., Hoel, T., Scheffel, M., Drachsler, H.: Guest editorial: Ethics and privacy in learning analytics. Journal of Learning Analytics 3(1), 5–15 (2016)
  * [20] Fonseca, S.C., Pereira, F.D., Oliveira, E.H., Oliveira, D.B., Carvalho, L.S., Cristea, A.I.: Automatic subject-based contextualisation of programming assignment lists. International Educational Data Mining Society (2020)
  * [21] Gašević, D., Dawson, S., Rogers, T., Gasevic, D.: Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success. The Internet and Higher Education 28, 68–84 (2016)
  * [22] Geller, S.A., Gal, K., Segal, A., Sripathi, K., Kim, H.G., Facciotti, M.T., Igo, M., Hoernle, N., Karger, D.: New methods for confusion detection in course forums: Student, teacher, and machine. IEEE Transactions on Learning Technologies 14(5), 665–679 (2021)
  * [23] Ghosh, D., Klebanov, B.B., Song, Y.: An exploratory study of argumentative writing by young students: A transformer-based approach. In: Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications. pp. 145–150 (2020)
  * [24] Holmes, W., Porayska-Pomsta, K.: The Ethics of Artificial Intelligence in education: Practices, challenges, and debates. Taylor & Francis (2022)
  * [25] Jayaraman, J., Black, J.: Effectiveness of an intelligent question answering system for teaching financial literacy: A pilot study. In: Innovations in Learning and Technology for the Workplace and Higher Education: Proceedings of ‘The Learning Ideas Conference’2021. pp. 133–140. Springer (2022)
  * [26] Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al.: Chatgpt for good? on opportunities and challenges of large language models for education. EdArXiv (2023)
  * [27] Khosravi, H., Shum, S.B., Chen, G., Conati, C., Tsai, Y.S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., Gašević, D.: Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence 3, 100074 (2022)
  * [28] Kumar, N., Mali, R., Ratnam, A., Kurpad, V., Magapu, H.: Identification and addressal of knowledge gaps in students. In: 2022 3rd International Conference for Emerging Technology (INCET). pp. 1–6. IEEE (2022)
  * [29] Kurdi, G., Leo, J., Parsia, B., Sattler, U., Al-Emari, S.: A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education 30, 121–204 (2020)
  * [30] Li, C., Xing, W.: Natural language generation using deep learning to support mooc learners. International Journal of Artificial Intelligence in Education 31, 186–214 (2021)
  * [31] Maheen, F., Asif, M., Ahmad, H., Ahmad, S., Alturise, F., Asiry, O., Ghadi, Y.Y.: Automatic computer science domain multiple-choice questions generation based on informative sentences. PeerJ Computer Science 8, e1010 (2022)
  * [32] Maxwell, S.E., Lau, M.Y., Howard, G.S.: Is psychology suffering from a replication crisis? what does “failure to replicate” really mean? American Psychologist 70(6), 487 (2015)
  * [33] Merine, R., Purkayastha, S.: Risks and benefits of ai-generated text summarization for expert level content in graduate health informatics. In: 2022 IEEE 10th International Conference on Healthcare Informatics (ICHI). pp. 567–574. IEEE (2022)
  * [34] Min, B., Ross, H., Sulem, E., Veyseh, A.P.B., Nguyen, T.H., Sainz, O., Agirre, E., Heinz, I., Roth, D.: Recent advances in natural language processing via large pre-trained language models: A survey. arXiv preprint arXiv:2111.01243 (2021)
  * [35] Mittelstadt, B.: Principles alone cannot guarantee ethical ai. Nature machine intelligence 1(11), 501–507 (2019)
  * [36] Moore, S., Nguyen, H.A., Bier, N., Domadia, T., Stamper, J.: Assessing the quality of student-generated short answer questions using gpt-3. In: Educating for a New Future: Making Sense of Technology-Enhanced Learning Adoption: 17th European Conference on Technology Enhanced Learning, EC-TEL 2022, Toulouse, France, September 12–16, 2022, Proceedings. pp. 243–257. Springer (2022)
  * [37] Nguyen, T.T., Le, A.D., Hoang, H.T., Nguyen, T.: Neu-chatbot: Chatbot for admission of national economics university. Computers and Education: Artificial Intelligence 2, 100036 (2021)
  * [38] OpenAI: Introducing chatgpt. https://openai.com/blog/chatgpt (2023), accessed: 2023-02-25
  * [39] Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., et al.: The prisma 2020 statement: an updated guideline for reporting systematic reviews. International journal of surgery 88, 105906 (2021)
  * [40] Pardo, A., Siemens, G.: Ethical and privacy principles for learning analytics. Br J Educ Technol 45(3), 438–450 (2014)
  * [41] Pugh, S.L., Subburaj, S.K., Rao, A.R., Stewart, A.E., Andrews-Todd, J., D’Mello, S.K.: Say what? automatic modeling of collaborative problem solving skills from student speech in the wild. International Educational Data Mining Society (2021)
  * [42] Ramesh, D., Sanampudi, S.K.: An automated essay scoring systems: a systematic literature review. Artificial Intelligence Review 55(3), 2495–2527 (2022)
  * [43] Rudolph, J., Tan, S., Tan, S.: Chatgpt: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching 6(1) (2023)
  * [44] Sallam, M.: The utility of chatgpt as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations. medRxiv pp. 2023–02 (2023)
  * [45] Sarsa, S., Denny, P., Hellas, A., Leinonen, J.: Automatic generation of programming exercises and code explanations using large language models. In: Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 1. pp. 27–43 (2022)
  * [46] Sawatzki, J., Schlippe, T., Benner-Wickner, M.: Deep learning techniques for automatic short answer grading: Predicting scores for english and german answers. In: Artificial Intelligence in Education: Emerging Technologies, Models and Applications: Proceedings of 2021 2nd International Conference on Artificial Intelligence in Education Technology. pp. 65–75. Springer (2022)
  * [47] Schneider, J., Richner, R., Riser, M.: Towards trustworthy autograding of short, multi-lingual, multi-type answers. International Journal of Artificial Intelligence in Education pp. 1–31 (2022)
  * [48] Schramowski, P., Turan, C., Andersen, N., Rothkopf, C.A., Kersting, K.: Large pre-trained language models contain human-like biases of what is right and wrong to do. Nature Machine Intelligence 4(3), 258–268 (2022)
  * [49] Science, D., Group, T.: Technology readiness levels definitions and descriptions. https://www.dst.defence.gov.au, accessed: 2023-01-20
  * [50] Selwyn, N.: What’s the problem with learning analytics? JLA 6(3), 11–19 (2019)
  * [51] Sha, L., Li, Y., Gasevic, D., Chen, G.: Bigger data or fairer data? augmenting bert via active sampling for educational text classification. In: Proceedings of the 29th International Conference on Computational Linguistics. pp. 1275–1285 (2022)
  * [52] Sha, L., Raković, M., Das, A., Gašević, D., Chen, G.: Leveraging class balancing techniques to alleviate algorithmic bias for predictive tasks in education. IEEE Transactions on Learning Technologies 15(4), 481–492 (2022)
  * [53] Sha, L., Raković, M., Lin, J., Guan, Q., Whitelock-Wainwright, A., Gašević, D., Chen, G.: Is the latest the greatest? a comparative study of automatic approaches for classifying educational forum posts. IEEE Transactions on Learning Technologies (2022)
  * [54] Sha, L., Rakovic, M., Whitelock-Wainwright, A., Carroll, D., Yew, V.M., Gasevic, D., Chen, G.: Assessing algorithmic fairness in automatic classifiers of educational forum posts. In: Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part I 22. pp. 381–394. Springer (2021)
  * [55] Shang, J., Huang, J., Zeng, S., Zhang, J., Wang, H.: Representation and extraction of physics knowledge based on knowledge graph and embedding-combined text classification for cooperative learning. In: 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD). pp. 1053–1058. IEEE (2022)
  * [56] Sharma, A., Kabra, A., Kapoor, R.: Feature enhanced capsule networks for robust automatic essay scoring. In: Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part V 21. pp. 365–380. Springer (2021)
  * [57] Song, W., Hou, X., Li, S., Chen, C., Gao, D., Sun, Y., Hou, J., Hao, A., et al.: An intelligent virtual standard patient for medical students training based on oral knowledge graph. IEEE Transactions on Multimedia (2022)
  * [58] Su, Y., Zhang, Y.: Automatic construction of subject knowledge graph based on educational big data. In: Proceedings of the 2020 The 3rd International Conference on Big Data and Education. pp. 30–36 (2020)
  * [59] Truong, T.L., Le, H.L., Le-Dang, T.P.: Sentiment analysis implementing bert-based pre-trained language model for vietnamese. In: 2020 7th NAFOSTED Conference on Information and Computer Science (NICS). pp. 362–367. IEEE (2020)
  * [60] Tsai, Y.S., Gasevic, D.: Learning analytics in higher education—challenges and policies: a review of eight learning analytics policies. In: Proceedings of the seventh international learning analytics & knowledge conference. pp. 233–242 (2017)
  * [61] Tsai, Y.S., Whitelock-Wainwright, A., Gašević, D.: The privacy paradox and its implications for learning analytics. In: Proceedings of the tenth international conference on learning analytics & knowledge. pp. 230–239 (2020)
  * [62] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)
  * [63] Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al.: Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359 (2021)
  * [64] Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., Drachsler, H.: Are we there yet?-a systematic literature review on chatbots in education. Frontiers in artificial intelligence 4, 654924 (2021)
  * [65] Wu, J.: Analysis and evaluation of the impact of integrating mental health education into the teaching of university civics courses in the context of artificial intelligence. Wireless Communications and Mobile Computing 2022 (2022)
  * [66] Yan, L., Zhao, L., Gasevic, D., Martinez-Maldonado, R.: Scalability, sustainability, and ethicality of multimodal learning analytics. In: LAK22: 12th International Learning Analytics and Knowledge Conference. pp. 13–23 (2022)
  * [67] Yang, S.J., Ogata, H., Matsui, T., Chen, N.S.: Human-centered artificial intelligence in education: Seeing the invisible through the visible. Computers and Education: Artificial Intelligence 2, 100008 (2021)
  * [68] Zawacki-Richter, O., Marín, V.I., Bond, M., Gouverneur, F.: Systematic review of research on artificial intelligence applications in higher education–where are the educators? International Journal of Educational Technology in Higher Education 16(1), 1–27 (2019)
  * [69] Zeng, Z., Gašević, D., Chen, G.: On the effectiveness of curriculum learning in educational text scoring. In: Proceedings of the AAAI Conference on Artificial Intelligence (2023)
  * [70] Zheng, L., Niu, J., Zhong, L.: Effects of a learning analytics-based real-time feedback approach on knowledge elaboration, knowledge convergence, interactive relationships and group performance in cscl. British Journal of Educational Technology 53(1), 130–149 (2022)