CTshort=CT, long=computational thinking CT-cubeshort=CT-cube,
long=computational thinking cube CTPshort=CTP, long=computational thinking
problem CTtshort=CTt, long=Computational Thinking Test R2T2short=R2T2,
long=Remote Rescue with Thymio II STEMshort=STEM, long=Science, Technology,
Engineering and Math

11institutetext:  G. Adorni 22institutetext: F. Mangili 33institutetext: L.
Gambardella 44institutetext: Dalle Molle Institute for Artificial Intelligence
(IDSIA), Università della Svizzera Italiana and University of Applied Sciences
and Arts of Southern Switzerland (USI-SUPSI), Lugano, Switzerland
55institutetext: A. Piatti 66institutetext: L. Negrini 77institutetext:
Department of Education and Learning (DFA), University of Applied Sciences and
Arts of Southern Switzerland (SUPSI), Locarno, Switzerland 88institutetext: E.
Bumbacher 99institutetext: Haute école pédagogique du canton de Vaud (HEP-VD),
Lausanne, Switzerland 1010institutetext: F. Mondada 1111institutetext: Mobile
Robotic Systems Group (MOBOTS), Ecole Polytechnique Fédérale de Lausanne
(EPFL), Lausanne, Switzerland 1212institutetext: D. Assaf 1313institutetext:
School of Education, University of Applied Sciences and Arts Northwestern
Switzerland (FHNW), Windisch, Switzerland

# A theoretical framework for the design and analysis of computational
thinking problems in education

Giorgia Adorni  Alberto Piatti  Engin Bumbacher  Lucio Negrini  Francesco
Mondada  Dorit Assaf  Francesca Mangili  Luca Gambardella

###### Abstract

The field of computational thinking education has grown in recent years as
researchers and educators have sought to develop and assess students’
computational thinking abilities. While much of the research in this area has
focused on defining computational thinking, the competencies it involves and
how to assess them in teaching and learning contexts, this work takes a
different approach. We provide a more situated perspective on computational
thinking, focusing on the types of problems that require computational
thinking skills to be solved and the features that support these processes.

We develop a framework for analysing existing computational thinking problems
in an educational context. We conduct a comprehensive literature review to
identify prototypical activities from areas where computational thinking is
typically pursued in education. We identify the main components and
characteristics of these activities, along with their influence on activating
computational thinking competencies. The framework provides a catalogue of
computational thinking skills that can be used to understand the relationship
between problem features and competencies activated.

This study contributes to the field of computational thinking education by
offering a tool for evaluating and revising existing problems to activate
specific skills and for assisting in designing new problems that target the
development of particular competencies. The results of this study may be of
interest to researchers and educators working in computational thinking
education.

###### keywords:

Computational thinking Competence development Educational framework Learning
contexts Situated cognition

##  1 Introduction

CT has emerged as a crucial skill for students to acquire in the 21st century.
As a result, there has been an increased effort to integrate computer science
education into K-12 classrooms (Weintrop et al., 2021). This initiative was
triggered by Jeannette Wing’s introduction of the term CT in (Wing, 2006).
Despite the impressive growth in tools, activities and curricula for teaching
CT, significant challenges remain in successfully integrating this new concept
into schools. One critical challenge is the lack of a precise, universally
accepted definition of CT, a relatively ambiguous concept (Weintrop et al.,
2021). Instead, different definitions with varying purposes, each focusing on
different aspects of CT, have been proposed (Lafuente Martínez et al., 2022).
This lack of consensus has made it difficult for the field to advance beyond
the exploratory stage. To address this issue and systematically develop and
evaluate different approaches to teaching, developing, and assessing CT, it is
necessary to have a precise and comprehensive formalisation of the concept and
to identify widespread best practices. Therefore, efforts are needed to
establish a clear and standardised definition of CT that reflects the diverse
perspectives and purposes of the field. However, there is no easy way to
define CT (Shute et al., 2017). Prevailing approaches have focused on
decomposing it into sub-dimensions and explicitly specifying each, e.g.,
Brennan and Resnick (2012); Grover and Pea (2017). These dimension-based
approaches have been used to categorise existing assessment tasks based on
analysing the underlying skills associated with a task, e.g., Lafuente
Martínez et al. (2021). However, creating tasks for developing and assessing
these sub-dimensions has been difficult. A recent study developed a reliable
and validated CT assessment for adults by combining existing items (Lafuente
Martínez et al., 2022). While experts identified several CT sub-dimensions to
be addressed by the items, statistical analyses suggested a one-dimensional
model as the best solution. A core problem is that sub-dimensions such as
decomposition, generalisation, or pattern recognition are closely interwoven
and hard to separate (Lafuente Martínez et al., 2021). Similar issues are
observed with other complex constructs, such as scientific inquiry or
practices, as noted by previous studies (Osborne, 2014; Ford, 2015).

In this article, we propose an alternative, more situated approach by focusing
on the types of problems that require CT to be solved, which we call CTP. This
approach is based on the idea that CT can be better understood by examining
the problems that elicit it and the features that support CT processes rather
than attempting to define CT itself. The idea behind incorporating a situated
approach in this context is that CT is not just a collection of skills. Still,
it is also closely tied to the specific context in which it is being applied
and allows understanding the complexity of CT. The theories of situated
learning proposed by (Roth and Jornet, 2013; Heersmink, 2013) stress that
learning is most effective within authentic and meaningful contexts. They
argue that knowledge is constructed through interactions with the environment
and the community rather than abstract instruction.

Our framework for analysing, evaluating and revising existing CTP and
designing new CTP is first based on the theory proposed by Piatti et al.
(2022). They emphasise that CT should be considered a situated activity and
contextualised and embedded in real-world problem-solving situations. They
also highlight that CT should be seen as a dynamic and adaptive problem-
solving process rather than a fixed set of competencies or skills. Their view
combines the original definition of CT from Wing (2006) – “Computational
thinking is the thought processes involved in formulating problems and their
solutions so that the solutions are represented in a form that an information-
processing agent can effectively carry out.” – which has been widely cited and
considered the foundation of the field of CT, with the situated theories of
learning (Roth and Jornet, 2013; Heersmink, 2013).

To avoid creating a novel CT competence model, we have adopted state-of-the-
art frameworks (Brennan and Resnick, 2012; Weintrop et al., 2016; Shute et
al., 2017) and reviewed relevant literature (Tikva and Tambouris, 2021;
Bocconi et al., 2016, 2022). Our approach resulted in a catalogue of commonly
recognised CT competencies widely used to develop and assess student abilities
in CT. These competencies, also called dimensions or skills, represent the
essential capacities students need to develop to solve CTP effectively. We
established a direct link between our catalogue of CT competencies and our
framework by showing how specific skills are more likely to be activated when
solving CTP with specific characteristics. Finally, the framework is applied
to a variety of prototypical problems from standard CT domains, demonstrating
that it allows for systematically analyse and evaluate CTP based on the
competencies they are intended to elicit and to assess student abilities in a
targeted and efficient manner.

Answering research questions such as “What are the characteristics of problems
requiring CT to be solved?”, “Which characteristics should a CTP have to
activate one (or more) CT competencies?” and “Which CT dimensions are
activated in a CTP with certain characteristics?”, can provide a better
understanding of how to design and assess CTP that effectively develop CT
skills. Our proposed framework offers a systematic approach for identifying
the characteristics of CTP and the corresponding CT competencies required to
solve them. This approach enables the selection or design of tasks effective
in developing and assessing the desired skills, ultimately improving the
quality of the tasks and assessment process.

This article is divided into four main sections. Section 2 presents a
theoretical framework for analysing, evaluating, revising and designing CTP.
This approach is based on a catalogue of CT skills, which are commonly used to
develop and evaluate student abilities. To determine how the characteristics
of a CTP impact the development of CT skills, we defined a mapping between
skills and characteristics, which is discussed in detail in Appendix 0.A.
Section 3 validates the framework through a qualitative analysis of various
CTP found in the literature, focusing on unplugged, robotic, and virtual
activities. It is important to note that readers are not required to read the
entire section. They can choose to read one example per category or skip this
section altogether and move directly to Sections 4 and 5. Additionally,
Appendix 0.B provides graphical templates that offer a visual representation
of how each CTP discussed in the article aligns with the CT competencies
outlined in the study. Section 4, includes an overview of the relationship
between CT competencies and different activity domains. It outlines the
strengths, weaknesses and areas for improvement of each category analysed.
Section 5, summarises the study’s key contributions, limitations and
implications for future research and practice in the field of CT competencies
development.

##  2 Methods

This section presents a theoretical framework for analysing, evaluating,
revising and designing CTP in educational contexts.

Through a thorough analysis of prototypical activities from various classic
areas of the literature where CT is typically pursued, we identify the main
characteristics and components of these problems. Relying on existing
frameworks and literature on CT competencies, we present a catalogue of skills
typically used to assess student abilities in this area. We then define a
mapping between the identified characteristics of the CTP and these
competencies to determine how these features influence the development of CT
abilities.

###  2.1 Computational thinking problems (CTPs)

The increasing emphasis on the development and assessment of students’ CT
skills has led to a need for a structured approach to analyse, evaluate,
revise, and design CTP. Although several initiatives have been launched to
introduce computer science education in classrooms, there is currently no
universally accepted definition of CT. This lack of a standardised definition
presents significant challenges in creating appropriate assessment tasks and
identifying the various sub-dimensions of CT. Therefore, it is crucial to
establish a clear and precise definition of CT that reflects the diverse
perspectives and purposes of the field to facilitate the development and
assessment of effective CT pedagogical interventions.

To address this issue, we followed the approach in Piatti et al. (2022), which
is more closely aligned with the complex, multi-faceted settings in which CT
is typically activated, such as educational environments or situations
involving multiple people and rich physical environments. They combine the
original view of Wing (Wing, 2006) with the situated theories of learning
(Roth and Jornet, 2013; Heersmink, 2013). According to Piatti et al. (2022),
CT activities are shaped by the physical and social context in which they
occur and involve external cognitive artefacts.

Figure 1: Visualisation of the CT-cube (From Piatti et al. (2022)). This model
considers the type of activity (problem setting, algorithm, assessment), the
artefactual environment (embodied, symbolic, formal), and the autonomy
(inactive role, non-autonomous active role, or autonomous active role).

Piatti et al. (2022) proposed a framework called CT-cube, illustrated in Fig.
1, for the design of CTP and the assessment of CT. This model considers the
type of CT activity being performed or required (problem setting, algorithm,
assessment), the artefactual environment in which the activities occur,
represented by the tools used (embodied, symbolic, formal), and the social
interactions and individual’s level of autonomy a priori and/or during the
task (inactive role, non-autonomous active role, or autonomous active role).
It is important to note that the steps in the activity dimension may be
iterated as needed to arrive at a satisfactory solution.

Based on the formulation of CT from Piatti et al. (2022), we define CTP by
considering the context in which the activity is being performed. According to
our framework, every CTP consists of several components: the system,
comprising the environment and the agent, the problem solver, and the task.

The environment is a physical and/or a virtual space, characterised by one or
more variables, called “descriptors”, which may change over time according to
the dynamics of this space.

The agent is a human, robotic or virtual being capable of performing “actions”
on the environment to change the value of its descriptors and therefore alter
its state. An algorithm is a finite set of instructions that an agent should
follow to perform actions in the environment to solve the task. Algorithms for
different types of agents can take various forms, such as code for a virtual
agent, behaviour for a robot, or a verbal or written set of instructions for a
human agent.

The problem solver is a human or group of people who can solve tasks that
require the use of algorithms, such as designing, implementing, or
communicating them to an agent to change the state of an environment. They
have access to reasoning tools, which are cognitive artefacts used to think
about the task, for example, whiteboards employed to organise ideas and
understand the logic of a problem or solution. Some of these tools, known as
interaction tools, also allow the problem solver to interface with the system.
An example is a programming platform used to write a program that controls a
robotic arm. In this case, the tool serves as both a reasoning tool, enabling
the problem solver to plan and design the code, and as an interaction tool,
allowing the execution of the algorithm and the observation of its effect on
the system. The set of all tools is collectively known as artefactual
environment and is described in the works of Heersmink (2013) and Piatti et
al. (2022).

The task is the activity that the problem solver performs to find one or more
solutions to a CTP. A solution is a combination of initial states, algorithms,
and final states that meets the system’s requirements for a particular
environment, with its set of states, and a given agent, with its set of
algorithms. The initial state is the starting configuration of the
environment, while the final state is the state of the environment after the
algorithm is performed. For a solution to be valid, the execution of the
algorithm on the initial state must produce the final state.

Fig. 2 conveys the framework’s components and their interactions. Each
component is depicted in a distinct colour consistently throughout the
article.

Figure 2: Visualisation of the components of a CTP. According to the
framework, a CTP includes: (1) the problem solver (in green) characterised by
the artefactual environment, i.e., the set of reasoning and interaction tools,
(2) the system, which consists of an environment with its descriptors (in
blue) and an agent with its actions (in violet), and (3) the task (in yellow)
characterised by the set of initial states, algorithms and final states.

The proposed framework, in addition to defining CTP and their components,
allows classify them according to their characteristics, which are relevant
for eliciting and assessing CT skills in educational contexts.

######  Definition 1 (Artefactual environment)

Tools and resources used by a problem solver to reason, understand a problem
or interact with a system.

The first characteristic we defined in our framework is the artefactual
environment, in line with the definition from Piatti et al. (2022) and the
model of the three worlds of mathematics by Tall (2006, 2013, 2020). In
particular, tools can be distinguished into “embodied” or ecological and
iconic representational cognitive artefacts based on embodiment and
perception, “symbolic” cognitive artefacts used to conceive and apply
procedures and rules, and “formal” cognitive artefacts used to create,
generalise and represent structures.

######  Definition 2 (Tools functionalities)

Specific features and functions provided by a tool or resource that enable a
problem solver to express a wide range of instructions and operations to the
agent.

Another aspect of CTP that is closely related to the previous one is the set
of tools functionalities. These functionalities can include defining and
manipulating “variables”, using different types of “operators”, creating
“sequences” of actions, “repeating” actions, using “conditional” statements,
defining “functions”, executing tasks in “parallel” and triggering “events”.
For example, a symbolic artefact, such as a block-based programming platform,
may have many functionalities, such as sequences, repetitions, conditionals,
etc. In contrast, the programming interface may have limited functionalities
during a robotic activity, for example, it could only consent to the use of
operators (like moving forward) or events.

######  Definition 3 (Problem domain)

The category of an activity depending on the nature of the agent and of the
environment in which the task is performed.

The domain classification of a CTP is useful for gaining insights into the
task’s context and identifying its specific characteristics, challenges, and
considerations. Three main categories of domains are commonly recognised in
cognitive task paradigms, these include: “unplugged” activities, which involve
a human agent and a physical environment, “robotic” activities, in which the
agent is a robot and the environment is physical, and “virtual” activities,
where both agent and environment are virtual, such as a simulated entity. It
is worth noting that the agent may be “embedded“ in the environment, so its
descriptors may likewise be used to describe it. Furthermore, in some cases,
the problem solver and agent may be “overlapped”, meaning they are the same
entity.

######  Definition 4 (System resettability)

The property of a system to be restored to its initial state. It can be
achieved either through the direct intervention of the problem solver on the
system (agents and environment) or indirectly through the reversibility of
actions within the system.

Resettability is another characteristic of CTP. An action is reversible if it
is possible to undo its effects, allowing the system to be restored to a
previous state. A system is considered non-resettable if it cannot be returned
to its initial state. This characteristic is crucial in educational contexts
as it enables the problem solver to experiment, make mistakes and try
different solutions without being constrained by the previous actions. Imagine
a task where the problem solver must draw a picture on a piece of paper using
a pencil. If he makes a mistake, he can easily erase it and start over. The
system is directly resettable because the problem solver can directly
intervene and reset the system to its initial state. Alternatively, the system
is not resettable if the problem solver can only use a pen. If he makes a
mistake, it can not be erased, and the problem solver must continue with the
picture in its current state. An example of indirect resettability would be if
the problem solver is drawing a picture on a digital tablet. He can use the
undo button to wipe out the previous action and return to a previous state.

######  Definition 5 (System observability)

The property of a system that pertains to the ability of the problem solver to
observe the effects of actions taken within the system and their impact on the
system’s state.

Observability is another aspect that can vary between CTP. Systems can be
classified as partially observable, in which only the aggregate effects of a
limited number of actions can be perceived, totally observable, in which every
single action and its consequences are visible, or not observable, in which
the problem solver is unable to directly see the results of the agent’s
actions and the system’s state and must infer it from other information. For
example, in a chess game, the problem solver can see the state of the board
and the pieces all the time, making the system fully observable. In a game of
poker, conversely, the problem solver can only see his own cards and the
community cards, but not those held by the other players, making the system
partially observable. In a scenario where the problem solver remotely controls
a robot to explore an underground cave, the person cannot directly see the
environment and robot’s actions and must rely on sensor data to determine its
location and progress. As a result, the system is non-observable, and
decisions are based on limited information.

######  Definition 6 (Task type)

The category of an activity influenced by both the number and type of
objectives that need to be achieved to solve the task.

The last important set of CTP features are those related to the task. Each
element that composes the task (initial state, algorithm, final states) can be
“given” or is “to be found”. The tasks are divided into six categories, which
differ depending on the number of objectives. Tasks with a single objective
are classified into the following types: (1) _find the initial state_ : given
the final state and the algorithm that produced it, the problem solver must
infer the initial state on which the algorithm was applied; (2) _find the
algorithm_ : given the initial and the final states, the problem solver must
devise and describe an algorithm, or a part of it, that the agent can execute
to transform the system from the initial to the final state; (3) _find the
final state_ : given the initial state and an algorithm, the problem solver
must derive the final state. Tasks with multiple objectives fall into the
following types: (4) _creation act_ : given an initial state, the problem
solver must determine a desired final state and an algorithm that the agent
can use to transform the system from the initial to the final state; (5)
_application act_ : given an algorithm, the problem solver must identify one
or more pairs of initial and final states on which the algorithm can be
applied successfully; (6) _project act_ : given a desired final state, the
problem solver must define an initial state and an algorithm that the agent
should use to transform the system from the initial to the final state.

######  Definition 7 (Task cardinality)

The proportion between the number of given elements to the number of elements
to be found to solve the task.

In a task, it is possible for each of the given elements and elements that
need to be found to be singular or multiple. The relationship between the
number of given elements and the number of elements to be found can be “one-
to-one”, “many-to-one” or “many-to-many”. For example, a task with a one-to-
one cardinality can be one where the problem solver is provided with a single
initial and final state and is expected to find a single algorithm that
transforms the initial state into the final state. In contrast, a task with a
many-to-one cardinality can be one where multiple initial states are given,
and the problem solver is expected to find a single algorithm that transforms
all the initial states into a single final state. Finally, a task with a many-
to-many cardinality can be one where the problem solver is provided with
multiple initial states and a single final state and is expected to find
multiple algorithms that transform all possible initial states into the
desired final state. This type of task can be traced back to many tasks with a
many-to-one cardinality.

Figure 3: Graphical template for the analysis of CTP. Template suitable for
graphically analysing the components of any CTP according to our framework.
Colours represent the CTP components and characteristics following the same
colour scheme of Fig. 2.

######  Definition 8 (Task explicitness)

The level of explicitness or implicitness in the presentation of the given
task’s elements.

The way the elements of the task are stated can be used to distinguish a CTP
by its level of explicitness. A task with “explicitly” specified elements is
directly usable in the problem-solving process, while a task with “implicitly”
expressed through constraints requires additional interpretation to be
understood. For example, in a task where a robot must turn on its lights as
soon as it finds a ball in a playground, the given elements are the initial
state (the robot lights switched off, the robot and the ball positions) and
the final state (the robot lights turned on and the robot positioned in front
of the ball), while the element to be found is the algorithm (the set of
actions the robot should perform to find the ball). The position of the ball
can be described either explicitly using coordinates, or implicitly by stating
that it is located in the playground.

######  Definition 9 (Task constraints)

The limitations or specific requirements that must be adhered to on the
elements of a task to be found for the solution to be considered as valid.

A CTP can be distinguished by the type of constraints on the elements of the
task that need to be found. In particular, the elements that have to be found
are distinguished in “unconstrained”, meaning they can be freely chosen from
all possible states and algorithms, without any limitations or specific
requirements that need to be met to consider the solution valid, and
“constrained”, meaning they must belong to a specified subset of the
respective universe set of states or algorithms. Referring to the previous
example, the algorithm to be found can be unconstrained if the robot can
perform any action to find the ball (moving in a random direction, using
sensors to detect the ball, or following a predefined path) or constrained if
the programming platform limits the commands the robot can execute (moving
only in specific directions, using only specific sensors or following a
particular set of predefined paths to find the ball).

######  Definition 10 (Algorithm representation)

The mean by which an algorithm is conveyed.

Finally, how the algorithm is represented is a significant task
characteristic. An algorithm can be “manifest” if it is directly expressed or
“latent” if it is not stated but is tacit or inferred by the problem solver.
Manifest algorithms can be further classified as “written” if it is
represented by an external and persistent representation of it, such as
through a programming language, or “not written” if it is communicated
verbally or through other non-permanent means. When the problem solver and the
agent are the same entity, the algorithm used to solve the task is often not
explicitly expressed or written down, as the problem solver is carrying out
the steps of the algorithm directly through their actions as the agent.

Fig. 3 graphically represent the components and characteristics of a CTP
according to our framework.

###  2.2 A catalogue of computational thinking (CT) skills

In this study, we have identified a set of CT competencies commonly used to
assess student abilities in CT. These competencies, also called dimensions or
skills, represent the fundamental abilities students need to develop to solve
CT problems effectively. We decided to draw from various state-of-the-art
competency models to select and define our taxonomy of CT competencies rather
than relying on a single model. Our selection process was inspired by the
literature reviews of Tikva and Tambouris (2021) and Bocconi et al. (2016,
2022), which provides a comprehensive overview of CT skills and their
potential for compulsory education. Of great importance is the framework of
Brennan and Resnick (2012). They proposed a list of CT skills divided into
three dimensions: computational concepts, practices, and perspectives. This
characterisation is often used in literature, but it was limited for our
purposes because it only considered virtual activities, while we also
investigated robotics and unplugged ones. Therefore, we extended this list by
partially following the taxonomy of CT in STEM courses proposed by Weintrop et
al. (2016). This classification consists of four main categories: data
practices, modelling and simulation practices, computational problem-solving,
and systems thinking practices. Another work we based on is that of Shute et
al. (2017). They developed a competence model based on a review of the
relevant definitions of CT in the literature, including those of Brennan and
Resnick (2012) and Weintrop et al. (2016).

Figure 4: Visualisation of our taxonomy of CT competencies. The overall
structure is based on the CT-cube (Piatti et al., 2022). The sub-skills are
derived from validated CT models (Brennan and Resnick, 2012; Weintrop et al.,
2016; Shute et al., 2017). The three skills groups are represented with the
same colour scheme used for the CT-cube dimensions in Fig. 1.

To facilitate the assessment of competencies and ensure that it is focused and
efficient, we have organised CT skills into a hierarchy with layers of
dimensions and sub-dimensions, depicted in Fig. 4. The skills taxonomy is
based on the activity dimension of the CT-cube framework by Piatti et al.
(2022), introduced in the previous section, representing the individual’s role
in the cognitive system at each moment of the task. The activity sub-
dimensions have been developed mainly using the frameworks of Brennan and
Resnick (2012); Shute et al. (2017); Weintrop et al. (2016). Competencies
related to the activity dimension involve a wide range of operations and
cognitive processes. For example, the “problem setting” competence may require
recognising and understanding various components of the framework within the
given CTP, as well as modelling the problem. The “algorithm” competence may
involve the comprehension and exploitation of different instructions with
varying difficulty levels, often influenced by the type of artifactual
environment involved. The “assessment” competence may consider determining
whether a solution is correct or its quality is satisfactory. Table 1
summarises the competencies of the first level of the hierarchy, defining all
the possible values that the activity dimension can assume.

Table 1:  Main competencies of the framework and their definition. The skills
listed are based on the values of the activity dimension of the CT-cube
framework (Piatti et al., 2022).

Competence |  Definition
---|---
Problem setting |  Recognise, understand, reformulate or model a CTP and its components so that its solution can be computed.a
Algorithm |  Conceive and represent a set of agent’s actions that should be executed by a human, artificial or virtual agent to solve the task.b
Assessment |  Evaluate the quality and validity of the solution in relation to the original task.c

  * a

See Table 2 for sub-competencies.

  * b

See Table 3 for sub-competencies.

  * c

See Table 4 for sub-competencies.

Table 2:  Problem setting sub-competencies and their definition. The skills listed are based on leading-edge competence models (Brennan and Resnick, 2012; Shute et al., 2017; Thalheim, 2000; Weintrop et al., 2016; Wing, 2011; Bocconi et al., 2016; Selby and Woollard, 2013; Angeli et al., 2016; Csizmadia et al., 2015; Selby, 2014; Barr and Stephenson, 2011). Competence |  Definition
---|---
Analysing |  Collect, examine and interpret data about the system: environment descriptors and agent actions.
[4pt/2pt] Data
collection |  Gather details about the system.
[1pt/2pt] Pattern
recognition |  Identify similarities, trends, ideas and structures within the system.
Modelling |  Restructure, clean and update knowledge about the system.
[4pt/2pt] Decomposition |  Divide the original task into sub-tasks that are easier to be solved.
[1pt/2pt] Abstraction |  Simplify the original task, focus on key concepts and omit unimportant ones.
Representing |  Illustrate or communicate information about the system and the task.

Tables 2, 3 and 4 provide a detailed breakdown of the sub-dimensions for the
three possible values of the CT-cube activity class (problem setting,
algorithm, assessment). In the tables, each row represents a specific skill.
The parent skill is distinguished from the lower-level skills by a dashed
line, while the lower-level competencies of a particular competence are
separated by dotted lines. This helps to visually organise the table to
differentiate between the different skill levels and make it easier to
understand their relationships.

Table 3:  Algorithm sub-competencies and their definition. The skills listed are based on leading-edge competence models (Brennan and Resnick, 2012; Cui and Ng, 2021; Rodríguez-Martínez et al., 2020; Bocconi et al., 2016, 2022; Shute et al., 2017). The following definitions are about the algorithmic concepts related to the skill since the skill definition varies based on the artifactual environment: in embodied contexts, it involves recognising or describing concepts through senses; in symbolic contexts, it involves applying them; and in formal contexts, it involves understanding properties to structure complex algorithms.  Competence |  Definition
---|---
Variables |  Entity that stores values about the system or intermediate data.
Operators |  Mathematical operators (such as addition ($+$), subtraction ($-$) etc.), logical symbols (such as and (&), or (|), and not (!)) or for comparison (such as equal to (==), greater than (>), and less than (<)), or even specific commands or actions (such as “turn left” or “go straight”).
Control
structures |  Statements that define the agent actions flow’s direction, such as sequential, repetitive, or conditional.
[4pt/2pt] Sequences |  Linear succession of agent actions.
[1pt/2pt] Repetitions |  Iterative agent actions.
[1pt/2pt] Conditionals |  Agent actions dependent on conditions.
Functions |  Set of reusable agent actions which produce a result for a specific sub-task.
Parallelism |  Simultaneous agent actions.
Events |  Variations in the environment descriptors that trigger the execution of agent actions.
Table 4:  Assessment sub-competencies and their definition. The skills listed are based on leading-edge competence models (Brennan and Resnick, 2012; Shute et al., 2017; Weintrop et al., 2016; Bocconi et al., 2016). Competence |  Definition
---|---
Correctness |  Assess whether the task solution is correct.
[4pt/2pt] Algorithm
debugging |  Evaluate whether the algorithm is correct, identifying errors and fixing bugs that prevent it from functioning correctly.
[1pt/2pt] System states
verification |  Evaluate whether the system is in the expected state, detecting and solving potential issues.
[1pt/2pt] Constraints
validation |  Evaluate whether the solution satisfies the constraints established for the system and the algorithm, looking for and correcting eventual problems.
Effectiveness |  Assess how effective is the task solution.
[4pt/2pt] Optimisations |  Evaluate whether the solution meets the standards in a timely and resource-efficient manner, and eventually identify ways to optimise the performance.
Generalisation |  Formulate the task solution in such a way that can be reused or applied to different situations.

It is important to note that the activation of the activity sub-dimensions,
especially the algorithmic one, is closely related to the type of artefactual
environment being considered. The task-related artefactual environment
combines the tools given and the type of problem involved. The activation of a
competence may vary depending on the context in which it is being applied or
whether the task is performed with embodied, symbolic, or formal artefacts.
When considering algorithmic sub-competencies, for example, to activate these
skills it is necessary to consider the tool used to represent the algorithm
and the type of abstraction of the reasoning required. In embodied
environments, knowledge is represented through sensory experiences, such as
seeing, hearing, or touching. In this context, these skills may be activated
simply by recognising and describing algorithmic concepts using physical
interactions. In symbolic environments, knowledge is represented using
symbols, such as words and numbers, or languages, such as natural or formal
languages. Common types of formal language include those used to code, such as
block-based and textual programming languages. In this context, reasoning
requires the problem solver to be able to apply these competencies to solve
problems and accomplish tasks. In formal environments, knowledge is
represented with abstractions, such as mathematical models, logical systems
and proofs. In this context, it is necessary to have a deeper understanding of
how these skills work and what their properties are to be able to structure
and apply them effectively, creating a complex system.

###  2.3 Map CTP characteristics to CT skills

To formalise our framework, clarifying the role of CTP characteristics in CT
skills assessment, we utilise the previously established definition of CTP to
identify the components of these problems and their characteristics and to
understand how different features may influence the assessment of CT
competencies. We establish a direct link between our catalogue of CT
competencies and the proposed framework, demonstrating how particular CT
dimensions are more likely to be activated when solving CTP with specific
characteristics. The link between the two is discussed in detail in Appendix
0.A, which explains how the various characteristics of CTP can impact the
development of CT competencies and how certain skills are more likely to be
used when solving CTP with specific characteristics. This approach enables the
assessment of student abilities in a targeted and efficient manner.

Our analysis of the relationships between characteristics and competencies
revealed four different types of connections. From the perspective of the
features, a certain characteristic can either be required to activate a
certain competence, prevent its activation, trigger its activation or be
irrelevant to its activation. Conversely, from the perspective of the
competencies, a skill is activated if it has all required features and none of
the preventing ones, can be encouraged by triggering features or is inhibited
if it has a preventing feature or only irrelevant features. The relationship
between features and skills can vary based on the type of environment in which
the problem is presented. Table 5 provides a comprehensive explanation of the
symbols used to represent the relationship between the characteristics of a
CTP and CT competencies.

Table 5:  Overview of the notation for representing the relationship between characteristics and skills. Symbol |  Meaning
---|---
✓ |  The characteristic is required for the competence activation
✗ |  The characteristic prevents the competence activation
$+$ |  The characteristic promotes the competence activation
none |  The characteristic is irrelevant for the competence activation
✓∗ |  The characteristic is one of the possible characteristics required for the competence activation
✓${}^{\text{{SF}}}$ /  ✗${}^{\text{{SF}}}$ |  The characteristic is required for / prevents the competence activation in the symbolic and formal environments
✓${}^{\text{{F}}}$ /  ✗${}^{\text{{F}}}$ |  The characteristic is required for / prevents the competence activation in the formal environment

Table 6 has been realised to offer a practical understanding of the
relationship between different features of CTP we identified and our catalogue
of CT competencies enabling us to assess student abilities in a targeted and
efficient manner. Each column is a characteristic of the CTP and every row
represents a skill. Overall, this table provides a clear visual representation
of how different features of the CTP can influence the assessment of CT
competencies and how the different dimensions of CT skills are more likely to
be activated when solving CTP with specific characteristics.

Table 6: Comprehensive overview of the relationship between different CTP characteristics and CT competencies. The table shows the relationship between the characteristics of CTP (columns) and CT competencies (rows). The CTP features considered include the tools’ functionalities, the system’s property, and the task trait. The meaning of the symbols used is provided in Table 5. The same colour schemes used for the CT-cube dimensions in Fig. 1 and for the CTP components in Fig. 2 are employed to present the skills and features, respectively.  | Tool functionalities | System | Task
---|---|---|---
|  Variables |  Operators |  Sequences |  Repetitions |  Conditionals |  Functions |  Parallelism |  Events |  System resettable |  System not resettable |  System (partially) observable |  System not observable |  Initial or final state to be found |  Algorithm to be found |  One-to-one cardinality |  Many-to-one cardinality |  Explicit elements |  Implicit elements |  Unconstrained elements |  Constrained elements |  Algorithm manifest |  Algorithm latent |  Algorithm written |  Algorithm not written
Data collection | ✓ |  |  |  |  |  |  | $+$ | $+$ | $+$ |  $+$ |  $+$ |  |  | $+$ | $+$ |  |  $+$ |  | $+$ |  $+$ |  $+$ | $+$ | $+$
Pattern recognition | $+$ |  | $+$ | ✓∗ | $+$ | ✓∗ |  |  | $+$ | $+$ |  $+$ |  $+$ |  |  | $+$ | $+$ |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  $+$ | $+$ | $+$
1-25[1pt/1pt] Decomposition | $+$ | $+$ | ✓∗ | $+$ | $+$ | ✓∗ | $+$ |  | $+$ | $+$ |  |  $+$ |  |  | $+$ | $+$ |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  $+$ | $+$ | $+$
Abstraction | ✓ |  | $+$ | $+$ | $+$ | ✓ |  |  |  | $+$ |  |  $+$ |  |  |  | $+$ |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  $+$ | $+$ | $+$
1-25[1pt/1pt] Data representation | ✓ |  | $+$ | $+$ | $+$ | $+$ |  |  |  | $+$ |  |  $+$ |  |  |  | $+$ |  |  $+$ |  | $+$ |  $+$ |  $+$ | $+$ | $+$
Variables | ✓ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ |  |  |  $+$ |  $+$ |  | ✓${}^{\text{\tiny{F}}}$ | $+$ | $+$ |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  | $+$ | $+$
Operators | $+$ | ✓ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ |  |  |  $+$ |  $+$ |  | ✓${}^{\text{\tiny{F}}}$ | $+$ |  |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  | $+$ | $+$
1-25[1pt/1pt] Sequences | $+$ | $+$ | ✓ | $+$ |  | $+$ |  |  |  |  |  $+$ |  $+$ |  | ✓${}^{\text{\tiny{F}}}$ |  |  |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  | $+$ | $+$
Repetitions | $+$ | $+$ | $+$ | ✓ |  | $+$ |  |  |  |  |  $+$ |  |  | ✓${}^{\text{\tiny{F}}}$ |  | $+$ |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  | $+$ | $+$
Conditionals | $+$ | $+$ |  |  | ✓ |  |  | $+$ |  |  |  $+$ |  $+$ |  | ✓${}^{\text{\tiny{F}}}$ | $+$ | $+$ |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  | $+$ | $+$
1-25[1pt/1pt] Functions | $+$ | $+$ | $+$ | $+$ |  | ✓ |  |  |  |  |  $+$ |  |  | ✓${}^{\text{\tiny{F}}}$ |  | $+$ |  $+$ |  $+$ | $+$ | $+$ |  $+$ |  | $+$ | $+$
Parallelism | $+$ | $+$ |  |  |  |  | ✓ |  |  |  |  $+$ |  |  | ✓${}^{\text{\tiny{F}}}$ |  | $+$ |  $+$ |  | $+$ |  |  $+$ |  | $+$ | $+$
Events | $+$ | $+$ |  |  | $+$ |  |  | ✓ |  |  |  $+$ |  |  | ✓${}^{\text{\tiny{F}}}$ |  |  |  $+$ |  | $+$ |  |  $+$ |  | $+$ | $+$
Algorithm debugging | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | ✓ | ✗ |  $+$ |  |  | ✓ |  |  |  |  $+$ |  |  |  ✓ |  ✗ | ✓${}^{\text{\tiny{F}}}$ | ✗${}^{\text{\tiny{F}}}$
System state verification |  |  |  |  |  |  |  |  | ✓ | ✗ |  $+$ |  | ✓ |  |  |  |  |  $+$ |  |  |  ✓${}^{\text{\tiny{SF}}}$ |  ✗${}^{\text{\tiny{SF}}}$ | ✓${}^{\text{\tiny{F}}}$ | ✗${}^{\text{\tiny{F}}}$
Constraints validation | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | ✓ | ✗ |  $+$ |  |  |  |  |  |  |  | ✗ | ✓ |  |  |  |
1-25[1pt/1pt] Optimisation | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | $+$ | ✓ | ✗ |  $+$ |  |  |  |  |  |  |  |  |  |  |  |  |
Generalisation | ✓ |  | $+$ | $+$ | $+$ | ✓ |  | $+$ | ✓ | ✗ |  $+$ |  |  |  |  | $+$ |  |  $+$ |  | $+$ |  |  |  |

This framework, supported by Table 6, is intended to be used as a tool to
analyse existing CTP, understand which CT competencies can be assessed by the
available features, evaluate the problem effectiveness for intended
educational purposes and eventually revise it if it is unsuitable. To use the
framework, the first step is to identify the CTP profile and list all of its
features. If the CT competencies to be measured have not been specified in the
activity, the framework can be used to determine which skills can be assessed
from the available features. On the other hand, if the CT competencies are
outlined, the framework can be used to compare the problem’s characteristics
with those associated with the specified competencies and determine the CTP
effectiveness for intended educational purposes. The effectiveness of a CTP
can be classified into five categories based on the relationship between its
features and the CT competencies.

######  Definition 11 (Minimal CTP)

A complete and optimised CTP in which its characteristics perfectly match the
features essential to activate the CT competence it is intended to elicit.

In other words, the problem is designed to activate the intended CT skill with
the minimum required features without any irrelevant or distracting elements.

######  Definition 12 (Extensive CTP)

A complete and rich CTP that, in addition to having all of the necessary
characteristics to activate a certain CT competence, also has features that
act as a stimulus to activate that skill or others.

These additional features make the problem more engaging and provide
additional opportunities for students to demonstrate their CT abilities. The
problem is considered complete and rich, as the additional features enhance
the overall learning experience and provide a more comprehensive evaluation of
the student’s skills.

######  Definition 13 (Unfocused CTP)

A CTP in which some of the features are irrelevant to activate the skill it is
intended to elicit.

These irrelevant features can create confusion and distract the students from
focusing on the essential aspects of the problem, hindering the optimisation
of the problem in activating the desired competencies and affecting the
accuracy of the assessment. An unfocused CTP should be revised to remove these
unnecessary features to improve its effectiveness.

######  Definition 14 (Adjustable CTP)

A CTP where some essential features required to activate a certain competence
are missing.

This type of problem can be modified to include the missing features,
optimising it to activate the relevant CT skills. This can be done by
modifying the problem statement, the system, or the artefactual environment.

######  Definition 15 (Unsuitable CTP)

A CTP that is not appropriate for its intended purpose, which lacks some
essential features but has some unwanted.

This type of problem makes it difficult or impossible to effectively used to
assess certain CT competencies and it would require too many changes to be
useful, and it may be better to design or choose another problem instead.

Additionally, Table 6 can be used when creating or choosing a new CTP to
assess specific CT competencies. The first step, in this case, would be to
define the skills to be evaluated and then use the table to list the necessary
features and those that are not needed. The table can also be used to find
existing problems that match the list of characteristics or to design a new
activity from scratch.

##  3 Results

To examine and validate our method in-depth, we applied the proposed framework
to a range of CT activities that are widely recognised as representatives in
educational settings. We focused on three categories of CTP, unplugged,
robotic, and virtual activities, to provide a more nuanced understanding of
each type of activity’s unique features and challenges. The selection of
activities serves as a means to demonstrate the effectiveness and practicality
of the framework and is not exhaustive. Through this analysis, we aim to
provide a comprehensive account of the framework’s applicability and identify
best practices and areas for improvement in the design and assessment of CT
activities, with the ultimate goal of contributing to the field of CT
education.

For each CTP presented in this section, we provided in Appendix 0.B the
graphical template used for their analysis, describing the component and the
characteristics of the CTP, and the table that summarises the relationship
between the characteristics of the CTP and the CT competences the that can be
activated or not.

###  3.1 Unplugged activities

In the context of CT, an unplugged activity is an activity that does not
involve the use of a computer or technology (Brackmann et al., 2017; Del Olmo-
Muñoz et al., 2020). As per our framework, unplugged activities refer to CTP
where the agent is a human rather than a virtual or robotic agent. These
activities have a common feature of involving physical and non-technological
artefactual environments, as they require manipulating physical objects rather
than using technology for manipulation. Unplugged activities are designed to
help students develop CT skills often related to problem setting, such as
pattern recognition and fundamental computer science concepts through hands-on
and tangible activities before they are introduced to more abstract,
technology-based activities (Bell et al., 2009). Examples of unplugged CTP
include puzzles, games, and other activities that involve manipulating
physical materials, such as blocks or cards.

####  3.1.1 Cross Array Task

The Cross Array Task is an unplugged activity designed by Piatti et al. (2022)
using the CT-cube to assess the development of algorithmic skills in
compulsory schools.

Figure 5: The Cross Array Task activity adapted from Piatti et al. (2022). The
task requires the problem solver to instruct the agent to reproduce a
reference schema solely through verbal communication, with the option of
supplementing instructions via gesturing on a support schema if deemed
necessary. A removable screen separates the participant and researcher to
regulate potential visual cues.

##### Components

The Cross Array Task, illustrated in Fig. 5, involves a student and a
researcher in a classroom setting, seated at a table and separated by a
removable screen. The student has to communicate an algorithm to colour a
white cross array to match a reference schema.

  * •

Problem solver: the student who has to communicate an algorithm corresponding
to the sequence of instructions to reproduce the colouring of the reference
schema. The artefactual environment comprises cognitive tools such as the
support and the colouring schemes, available to the problem solver to reason
about the task. Additionally, the problem solver can interact with the system
to communicate the algorithm. This can be achieved using a natural language
such as the voice (symbolic) or gestures (embodied) on the empty cross array.
Moreover, by removing the screen that separates the problem solver from the
agent, he can have visual feedback (embodied) of the cross array being
coloured.

  * •

Agent: the researcher, executor of the problem solver’s instructions,
responsible for filling the colouring schema according to the problem solver’s
algorithm. The agent’s actions are not resettable.

  * •

Environment: the cross array to be coloured, whose state is described by the
colour of each dot (white, yellow, blue, green, or red).

  * •

Task: find the algorithm. The system’s state is defined by the colouring cross
status, initially white and, at the end the same as the reference schema. The
algorithm is the set of agent instructions to achieve this transformation.

##### Characteristics

The characteristics of this activity have been analysed using the graphical
templates shown in LABEL:fig:CAT-features in Appendix 0.B.

  * •

Tool functionalities: voice and gestures provide various functionalities
associated with algorithmic concepts, suitable to design the algorithm,
including (i) variables can represent different colours of the cross array
dots; (ii) operators are used to change the colour of the dots performing
actions such as colouring a dot, a row, a square and so on; (iii) sequences
determine the order in which the actions should be executed to achieve the
desired outcome; (iv) repetitions allow for repeating specific sequences of
operations, such as colouring the first column in red and repeating it every
two columns; (v) functions consist of operations that perform a specific task
and can be applied to different inputs, for example, creating a pattern of
alternating red and yellow dots in a square and applying it to different
positions of the cross array; (vi) parallelism involves executing multiple
actions simultaneously and can be associated with using symmetries to describe
the pattern.

  * •

System resettability: the system is not resettable since it is impossible to
reverse the agent’s actions.

  * •

System observability: the system is partially observable since the cross array
being coloured by default is not seen until the end of the task unless the
problem solver demands otherwise.

  * •

Task cardinality: the task has a one-to-one mapping, with given one initial
and one final state, and an algorithm to be found.

  * •

Task explicitness: all elements are given explicitly.

  * •

Task constraints: the algorithm is unconstrained.

  * •

Algorithm representation: the algorithm is represented through voice commands
or gestures. It is considered manifest, because it can be seen, but not
written since it is not stored in a permanent format.

##### Enabling features for competencies development

The relationship between features and skills is summarised in Appendix 0.B in
LABEL:tab:CAT-mapping. This paragraph explores the enabling characteristics
that support the development of competencies within the task.

  * •

Problem setting: all competencies can be activated thanks to the presence of
variables, sequences, repetitions and functions in the tool functionalities.
The presence of many tool functionalities, the non-resettability of the system
and the algorithm representation positively affect and boost problem setting
skills. The system observability supports data collection and pattern
recognition. The one-to-one cardinality, in addition, stimulates
decomposition. The explicit and unconstrained definition of the task elements
also promotes pattern recognition, decomposition and abstraction.

  * •

Algorithm: all competencies associated with the algorithmic concepts enabled
by the tool functionalities, meaning variables, operators, sequences,
repetitions and functions, can be activated in all three artefactual
environments and promote one another. The form of representation of the
algorithm, the system observability, and the explicit and unconstrained
definition of the task elements further enhance these. The one-to-one
cardinality helps to enhance some of these skills as well.

  * •

Assessment: since the system is not resettable, no assessment skills can be
developed.

##### Inhibiting features for competencies development

  * •

Conditionals and events: non-activable as these functionalities are
unavailable in the platform. A way to make conditionals available in the tool
functionalities would be allowing the problem solver to change a dot colour,
for example by communicating instructions such as: “if the dot is red, then
colour it yellow”. By doing this, the problem solver engages with the concept
of conditionals and can develop their algorithmic skills. The completion of
each row in the cross array can be considered an event. The problem solver can
specify that they want to fill the cross line by line, and once a line is
complete, the researcher will move on to the next line. This allows the
problem solver to list only the sequence of colours without repeating the
instructions for where to go. The change in the environment (completing a row)
triggers the researcher to move to the next row. Using conditionals and events
can greatly enhance the complexity of the solutions that can be generated and
help develop advanced CT skills.

  * •

Assessment skills: the inability to reset the system impairs the development
of the student’s skills. One possible solution to this issue is enabling the
student to reset the colouring scheme using a voice command. This would return
the schema to its initial blank state, allowing the student to start the task
from the beginning and practice their assessment skills. To develop system
state verification, it is also essential to not reveal the initial or final
states. Moreover, constraints should be imposed on the algorithm to develop
constraint validation skills, for example, limiting the use of specific
operators or the number of times they can be used, allowing the problem solver
to develop the ability to think about the constraints and limitations in their
algorithms.

####  3.1.2 Graph Paper Programming

Graph Paper Programming is an unplugged activity from Code.org (2015), a
nonprofit organisation that aims to provide students with the opportunity to
learn computer science as part of their education, offering various activities
designed to increase diversity in computer science and reach students at their
skill level and in ways that inspire them to continue learning.

The Graph Paper Programming activity can be divided into two parts, each with
a different task. In the first part, the student is given a $4\times 4$ grid
of white and black squares and asked to write explicit instructions for
another classmate to reproduce the image without letting the other person see
the original drawing. In the second part, the same student follows the
instructions they previously wrote to reproduce the image. By dividing the
activity into these two parts, we can gain a deeper understanding of the
cognitive processes and skills involved in each task and understand the
potential for this activity to support the development of CT abilities.

Figure 6: The first part of the Graph Paper Programming (GPP) activity,
adapted from Code.org (2015). The task requires the problem solver to instruct
the agent to reproduce the reference schema with instructions written on a
steps array using a predefined set of arrow symbols.

##### Components (part 1)

The first part of the activity is illustrated in Fig. 6.

  * •

Problem solver: the student who writes the set of instructions for the agent
to follow. The artefactual environment comprises cognitive tools such as the
reference schema (embodied) and the support and the set of arrow symbols
(embodied) available to the problem solver to reason about the task.
Additionally, the problem solver can interact with the system to communicate
the algorithm, writing the arrow symbols in the steps array (symbolic). These
can be considered as a programming language and its programming platform.

  * •

Agent: the other student who executes the problem solver’s instructions by
filling the colouring schema accordingly. Its actions are not resettable.

  * •

Environment: the schema to be coloured, described by the colour of each square
(white or black).

  * •

Task: find the algorithm. The system’s state is defined by the colouring
schema status, initially white and, at the end the same as the reference
schema. The algorithm is the set of agent instructions to achieve this
transformation.

Figure 7: The second part of the Graph Paper Programming (GPP) activity,
adapted from Code.org (2015). The task requires the problem solver to fill the
empty colouring schema following the program provided. The figure illustrates
the expected final state.

##### Components (part 2)

The second part of the activity is illustrated in Fig. 7.

  * •

Problem solver and Agent: they overlap and correspond to the student who
follows the instructions to recreate the image. The only action the agent can
perform is to paint the colouring schema without the possibility of undoing
it. The artefactual environment comprises cognitive tools such as the
colouring schema (embodied) available to the problem solver to reason about
the task. As before, the arrow symbols on the steps array (symbolic) are also
used to interact with the system. Moreover, being the agent and the problem
solver overlapped, visual feedback (embodied) is always given.

  * •

Environment: the schema to be coloured, described by the colour of each square
(white or black).

  * •

Task: find the final state. The system’s state is defined by the colouring
schema status, initially white. However, the final state is not given and has
to be found using the provided algorithm.

##### Characteristics

The characteristics of this activity have been analysed using the graphical
template presented in Appendix 0.B. LABEL:fig:GPPfa-features refers to the
first part of the activity, while LABEL:fig:GPPffs-features refers to the
second.

  * •

Tool functionalities: in both parts of the activity, the tools provide various
functionalities associated with algorithmic concepts suitable to design the
algorithm, including (i) variables can represent different colours of the
schema squares; (ii) operators correspond to the arrow symbols used to
instruct the agent to move from one square to another, determining whether a
square is coloured black or white; (iii) sequences determine the order in
which the actions should be executed to achieve the desired outcome; (iv)
repetitions allow for repeating specific sequences of operations; (v)
functions can be represented by a group of instructions that perform a
specific task, such as colouring a particular shape on the grid, that can be
used multiple times.

  * •

System resettability: the system is not resettable since it is impossible to
reverse the agent’s actions.

  * •

System observability: in the first part of the activity, the system is not
observable, as there is no visual feedback about the agent’s actions; in the
second part of the activity, the visual feedback consented thanks to the
problem solver and agent overlapping, makes the system totally observable.

  * •

Task cardinality: both tasks have a one-to-one mapping, with one initial
state, final state and algorithm.

  * •

Task explicitness: the task elements are explicit.

  * •

Task constraints: the final state is unconstrained.

  * •

Algorithm representation: the algorithm is manifest and written, represented
through the arrow symbols written in the steps array.

##### Enabling features for competencies development

This paragraph explores the enabling characteristics that support the
development of competencies within this CTP. The relationship between features
and skills in the two parts of the activity are summarised in Appendix 0.B in
LABEL:tab:GPPfa-mapping and LABEL:tab:GPPffs-mapping.

  * •

Problem setting: all competencies can be activated thanks to the presence of
variables, sequences, repetitions and functions in the tool functionalities.
The presence of many tool functionalities, the non-resettability of the system
and the algorithm representation positively affect and boost problem setting
skills. In the first part of the activity, the inability to observe the
systems further supports the development of all these competencies, while in
the second part, the system observability sustains only data collection and
pattern recognition. The one-to-one cardinality in addition stimulates data
collection and pattern recognition but also decomposition. The explicit and
unconstrained definition of the task elements also promotes pattern
recognition, decomposition and abstraction.

  * •

Algorithm: the competencies associated with algorithmic concepts, including
variables, operators, sequences, repetitions, and functions, can be developed
in the first part of the activity through the use of the tool functionalities
in all artefactual environments. However, in the second part of the activity
where the algorithm is given, these competencies cannot be developed in the
formal environment but only in the embodied and symbolic environments. The
algorithm representation, the system observability, and the explicit and
unconstrained definition of the task elements further enhance all these
skills.

  * •

Assessment: since the system is not resettable, no assessment skills can be
developed.

##### Inhibiting features for competencies development

  * •

Conditionals, parallelism and events: non-activable as these functionalities
are unavailable in the platform. Activating conditionals would be allowed by
providing a new arrow symbol that determines the colour of a square based on
certain conditions, for example, the colour of the square above. Parallelism
can be enabled by operating a new arrow symbol instructing the agent to colour
two squares simultaneously. An event that could be taken into account is that
every time a cell is filled with black, the instructions in the steps array
move to the next row rather than continuing from that specific cell.

  * •

Assessment skills: the inability to reset the system impairs the development
of the student’s skills. In the first part of the activity, where the problem
solver is writing the set of instructions for the agent, to solve this issue
the system can be reset by simply starting over with a new blank graph array
and writing a new set of instructions, or simply offering the possibility to
use pencil and eraser. In the second part of the activity, where the problem
solver and the agent are the same people, the system can be reset by either
using a new colouring schema or erasing the previously produced image and
starting over.

####  3.1.3 Triangular Peg Solitaire

Triangular Peg Solitaire is a strategy game that can be played in two
variants: the classic, on a triangular board with 15 holes and pegs, and the
paper and pencil modality. The board is initially filled with pegs, except for
one hole, which is left empty (see the top of Fig. 8). This game is often used
to teach problem-solving, logic and strategy skills, requiring the player to
determine the most efficient sequence of moves to remove all the pegs on the
board except one. Research has shown that the second activity variant can
effectively promote problem-solving skills even in older students (Barbero,
2020). The game is played by following the rules, which dictate that a peg can
only be moved by jumping over a neighbouring peg on the diagonal or horizontal
lines (see the bottom of Fig. 8). By analysing the two versions of this game,
we aim to understand their impact on promoting problem-solving and critical
thinking skills, evaluating advantages and limitations and providing insights
into how they can be used for CT development.

Figure 8: The Triangular Peg Solitaire.  The game is played on a board
containing 15 spots, with 14 pegs placed on it at the start of the game (top).
The task requires the problem solver to strategically move one peg at a time
to eliminate all other pegs on the board until only one remains (bottom),
jumping a peg over a neighbouring peg on the diagonal or horizontal lines,
with the constraint that there must be a free landing spot for the jumping peg
(adapted from Berlekamp et al. (2004)).

##### Components (board variant)

The first variant of the activity, illustrated in Fig. 8, requires solving the
game on a physical board.

  * •

Problem solver and Agent: they overlap and correspond to the player who must
determine the most efficient sequence of moves to remove all the pegs on the
board. The only action the agent can perform is to move the pegs on the board
without the possibility of undoing it. The artefactual environment comprises
tools for reasoning and interacting with the system. Being the agent and the
problem solver overlapped, visual feedback (embodied) of the state of the
board is always given. Moreover, the problem solver can physically interact
with the system by moving the pegs on the board (embodied).

  * •

Environment: the wooden board, described by the number of pegs on it.

  * •

Task: find the algorithm. The system’s initial state is the board with 14
pegs. The final state is the board with one peg. The algorithm to be found
specifies the sequence of moves to remove all the pegs.

Figure 9: A Triangular Peg Solitaire solution adapted from Bell (2007, 2008)
and Barbero and Gómez-Chacón (2018). The task requires the problem solver to
solve the game using paper and pencil by meticulously documenting their entire
thought process. The solution can be presented in multiple ways, such as
graphically using a Cartesian notation (top) or by numbering the boxes
progressively and expressing the movements used (bottom).

##### Components (paper & pencil variant)

The first variant of the activity, illustrated in Fig. 9, requires solving the
game with paper and pencil by documenting the thought process and devising a
winning strategy.

  * •

Problem solver and Agent: they overlap and correspond to the player who must
determine the most efficient sequence of moves to remove all the pegs on the
board. The only action the agent can perform is to write the thinking process
and strategy to remove the pegs on the board without the possibility of
undoing it. The artefactual environment comprises tools for reasoning and
interacting with the system. Being the agent and the problem solver
overlapped, visual feedback (embodied) of the state is always given. Moreover,
the problem solver can physically interact with the system by writing the
thinking process (symbolic).

  * •

Environment: the board drawn in the thinking process, described by the number
of pegs on it.

  * •

Task: find the algorithm. The system’s initial state is the board with 14
pegs. The final state is the board with one peg. The algorithm to be found
specifies the sequence of moves to remove all the pegs.

##### Characteristics

The characteristics of this activity have been analysed using the graphical
template presented in Appendix 0.B. In particular, LABEL:fig:TPSboard-features
refers to the board version of the activity, while LABEL:fig:TPSpp-features
refers to the paper and pencil variant.

  * •

Tool functionalities: in both variants of the activity, the tools provide
various functionalities associated with algorithmic concepts suitable to
design the algorithm, including (i) variables can represent the state of the
board and in particular the number of pegs on it; (ii) operators correspond to
the moves made by the player to change the state of the board by removing or
moving pegs from one hole to another; (iii) sequences determine the order of
moves made by the player; (iv) repetitions allow for repeating certain moves
or sequences of moves; (v) conditionals refer to the possible decisions that
the plays may need to make, such as where to jump in one direction or another;
(vi) functions can be represented by a group of instructions that perform a
specific task, for example, a function to delete pegs in a row which can be
applied to several rows.

  * •

System resettability: in the board variant, the system is not resettable
meaning that once the player has made a move, it cannot be undone. However,
the system is resettable in the paper and pencil version, even though the
player’s actions are not reversible. This is because the informal setting, in
which the player documents their thought process, allows for experimentation
and exploration without fear of judgement or negative consequences. In other
words, the player can freely make mistakes, express uncertainty, and
experiment with different strategies without permanently impacting the game.

  * •

System observability: in both variants, the system is observable because the
agent and problem solver, that overlap, can see the state of the board
anytime.

  * •

Task cardinality: both activity variants have a one-to-one mapping, with one
initial state, one final state and one algorithm.

  * •

Task explicitness: the initial state of the system is given explicitly, while
the final one is given implicitly since the task instruction does not specify
which is the position of the last remaining peg.

  * •

Task constraints: the algorithm is constrained by the game rules which dictate
that a peg can only be moved by jumping over a neighbouring peg on the
diagonal or horizontal lines and that there must be a free landing spot for
the jumping peg.

  * •

Algorithm representation: in the board variant of the game, the algorithm is
latent since it is performed physically through the player’s moving the pegs,
it is not permanently recorded and cannot be revisited. On the other hand, in
the paper and pencil modality, the player writes down the algorithm, and it
becomes a permanent record that can be reviewed and used as a reference. This
allows the player to experiment freely and change their approach without
having to start over each time. The written representation of the algorithm in
the paper and pencil modality provides a clear and concrete way to represent
the player’s thought process and strategy.

##### Enabling features for competencies development

This paragraph explores the enabling characteristics that support the
development of competencies within this CTP. The relationship between features
and skills in the two activity variants are summarised in Appendix 0.B in
LABEL:tab:TPSboard-mapping and LABEL:tab:TPSpp-mapping.

  * •

Problem setting: all competencies can be activated in both variants of the
game thanks to the presence of variables, sequences, repetitions and functions
in the tool functionalities. The presence of many tool functionalities, the
non-resettability of the system (in the board version of the game), the
implicit and constrained definition of the task elements and the algorithm
representation positively affect and boost problem setting skills. The
system’s observability sustains data collection and pattern recognition
skills, while the one-to-one cardinality and the resettability of the system
(in the paper and pencil variant of the game) also stimulate decomposition.

  * •

Algorithm: the competencies associated with algorithmic concepts, including
variables, operators, sequences, repetitions, conditionals and functions, can
be developed in all artefactual environments. The system observability, the
implicit and constrained definition of the task elements, and the manifest
written representation of the algorithm (in the paper and pencil variant of
the game) can further enhance these skills.

  * •

Assessment: no assessment skills can be developed in the board variant of the
activity since the system is not resettable. In the paper and pencil version
of the activity, algorithm debugging can be activated in all artefactual
environments due to the resettability of the system and the manifest and
written representation of the algorithm; constraint validation can be
developed because there are constraints on the algorithm that can be verified
since the system is resettable; optimisation can be activated as it only
requires the resettability of the system; generalisation can be activated
through the system’s resettability and the presence of variables and
functions. Tool functionalities as well as the system observability help
develop these competencies.

##### Inhibiting features for competencies development

  * •

Parallelism and events: non-activable as these functionalities are unavailable
in the platform. It is possible to develop parallelism and events in the game
by incorporating a variant where multiple pegs can be moved simultaneously or
by introducing multiple players who can make moves simultaneously. A way to
incorporate events would be to use technology such as a computer program or
app to play the game, creating programmed events that the player’s actions
could trigger. For example, if the player removes a certain peg, the computer
could trigger an event that changes the board’s appearance or displays a
message. However, it is important to note that these changes would result in a
different game with a different set of objectives and challenges and might not
necessarily have the same educational benefits as the original Triangular Peg
Solitaire.

  * •

Assessment skills: the inability to reset the system impairs the development
of the student’s skills in the first variant of the activity. In the paper and
pencil version, only system state verification is non-activable because both
the initial and final states are provided. The task can be adjusted by
modifying the game such that only the initial state is provided and the final
state is unknown. For example, letting the player determine the specific peg
position for the last peg This would make the task more challenging and
require the player to develop their skills in system state verification.

####  3.1.4 Computational Thinking Test (CTt)

The CTt is an assessment tool designed to evaluate the CT skills of students
between the ages of 12 and 13 (Román-González, 2015; Román-González et al.,
2017b, a; Román-González et al., 2018). It aligns with the work of researchers
such as Brennan and Resnick (2012); Kalelioğlu (2015), who have identified key
computational concepts related to algorithmic skills. Additionally, the CTt is
designed to align with the standard interfaces used by organisations
universally recognised in this context, such as Code.org, which utilise visual
blocks to teach coding. The test consists of 28 multiple-choice questions.
However, we will only analyse two.

Figure 10: Item 7 of the Computational Thinking Test (CTt) adapted from Román-
González et al. (2017b). The task requires the problem solver to correct a set
of instructions that should make the agent draw a rectangle.

##### Components (item 7)

The first CTP analysed, called “item 7”, is depicted in Fig. 10 and has been
designed to evaluate the student’s ability to identify and fix errors in code,
in a script that does not involve computational nesting concepts but only the
concept of repetitions.

  * •

Problem solver: the student taking the test presented with a wrong code script
that must be fixed. The artefactual environment comprises only cognitive
tools, thus is impossible to interact with the system, which is considered
static. The reasoning tool provided includes the sketch in the problem
description (embodied) and the visual blocks interface (symbolic), which
allows the student to think about the instructions and imagine to test
different solutions.

  * •

Agent: the artist responsible for drawing the rectangle according to the
instructions provided. It is an abstract representation, not a physical entity
that can be observed or interacted with. The actions it can perform are moving
and turning, considered reversible since they must be corrected.

  * •

Environment: the place where the imaginary rectangle should be drawn,
described by the number of drawn rectangle segments, length and orientation.

  * •

Task: find the algorithm. The system’s initial state is the imaginary
rectangle not yet being drawn, while the final state is the $50\times 100$
pixels rectangle drawn. The algorithm is not valid, for this reason, the final
and correct version of it has to be found.

Figure 11: Item 14 of the Computational Thinking Test (CTt) adapted from
Román-González et al. (2017b). The task requires the problem solver to select
the correct set of instructions to make the agent cross a predefined path to
reach a desired position.

##### Components (item 14)

The second CTP analysed, called “item 14”, is depicted in Fig. 11 and has been
designed to evaluate the student’s ability to organise a set of commands in a
logical and orderly manner in a script that does not involve computational
nesting concepts but only two specific computational concepts: repetitions and
conditionals.

  * •

Problem solver: the student taking the test is given four sets of code scripts
from which he must select the appropriate set of moving instructions. The
artefactual environment comprises only cognitive tools, including the sketch
of the maze in the problem description (embodied) and the four sets of
instruction in the form of visual blocks (symbolic), thus is impossible to
interact with the system.

  * •

Agent: Pac-Man, a representation of an abstract entity that can move in the
maze to reach the Ghost following the predefined pattern marked out. Its
actions are reversible since they must be corrected.

  * •

Environment: the maze, described by the Pac-Man and the Ghost positions, and
the path to be followed.

  * •

Task: find the algorithm. The system’s initial state corresponds to the Pac-
Man and the Ghost in their starting position, while in the final state, Pac-
Man is in front of the Ghost and has crossed the predefined path. The
algorithm is not given since four sets of codes are provided, and the correct
one has to be found to reach the desired outcome.

##### Characteristics

The characteristics of this activity have been analysed using the graphical
template presented in LABEL:fig:CTt7-features and LABEL:fig:CTt14-features in
Appendix 0.B.

  * •

Tool functionalities: in both variants of the activity, the tools provide
various functionalities associated with the visual blocks interface, including
(i) variables; (ii) operators correspond to the agent actions contained in the
blocks in turquoise; (iii) sequences; (iv) repetitions represented by the loop
in the pink blocks; (v) conditionals described by the if statements in the
blue blocks (only in the second activity); (vi) functions.

  * •

System resettability: even if the problem solver cannot interact with the
system, the system is resettable in both activities since the algorithm has to
be correct or selected from a set.

  * •

System observability: in both tests, the system is not observable because the
agents in question, the artist and Pac-Man, are imaginary entities and their
actions, such as drawing or moving, are not physically visible. The problem
solver must rely on the instructions provided to understand the actions taken
by the agent, and cannot observe their actual outcome.

  * •

Task cardinality: both CTP have a one-to-one mapping, with one initial state,
final state and algorithm.

  * •

Task explicitness: all elements are given explicitly.

  * •

Task constraints: the algorithms in both tasks are constrained since the
computational concepts addressed are already determined and limited to the
specific notions presented.

  * •

Algorithm representation: the algorithm is manifest and written in both
activities.

##### Enabling features for competencies development

This paragraph explores the enabling characteristics that support the
development of competencies within this CTP. The relationship between features
and skills in the two activity variants is summarised in Appendix 0.B in
LABEL:tab:CTt7-mapping and LABEL:tab:CTt14-mapping.

  * •

Problem setting: all competencies can be activated in both activities thanks
to the presence of variables, sequences, repetitions and functions in the tool
functionalities. The presence of many tool functionalities, the system’s non-
observability, the algorithm’s constrained definition, and its written
representation promote the development of all problem setting skills. The
system resettability, the one-to-one cardinality and the explicit
representation of elements support other competencies.

  * •

Algorithm: the competencies associated with algorithmic concepts provided by
the tool functionalities can be developed in all artefactual environments. The
non-observability of the system, the explicit and constrained definition of
the task elements, and the manifest written representation of the algorithm
can further enhance these skills.

  * •

Assessment: algorithm debugging can be activated in all artefactual
environments due to the resettability of the system and the written algorithm;
since the system can be reset, also the constraints on the algorithm can be
checked and corrected, allowing for the development of constraint validation;
similarly optimisation can be activated since the resetting capability is
sufficient; generalisation can be developed thanks to the system’s
resettability and the presence of variables and functions. The tool
functionalities available further encourage the development of these
competencies.

##### Inhibiting features for competencies development

  * •

Conditionals: non-activable in the first activity since this functionality is
not present in the visual blocks provided to the students.

  * •

Parallelism and events: non-activable in both activities, as before, because
they are not available in the visual blocks provided to the students. To
activate these skills, the visual blocks must include the tools for creating
parallelism and events in the algorithm.

  * •

System state verification: non-activable because the initial and final states
are provided.

###  3.2 Robotics activities

In robotics, educational robotics and physical computing activities involve
using physical robotic hardware equipped with controllers, sensors, and
actuators. These robotic agents are programmed to perform specific behaviours
in response to the environment. To achieve this, problem solvers are typically
provided with a programming platform allowing interaction with the robot.

There are numerous commercially available programming platforms for
educational robots, each offering its own set of hardware and programming
platforms (Bravo et al., 2017). The agents can be programmed using formal
textual programming languages, such as Python (Noone and Mooney, 2018), or
symbolic visual programming languages, which are often based on blocks, like
Blockly or Scratch (Shin et al., 2014). Some platforms also allow for embodied
physical interactions, enabling users to manipulate the robot through touch
buttons or tangible symbols that are scanned and executed (Bers and Horn,
2010; Mussati et al., 2019).

In this section, we aim to analyse various activities based on different types
of agents, including the Thymio II, the Ozobot, and the Micro:bit.

####  3.2.1 Thymio Lawnmower Mission

Thymio Lawnmower Mission is an educational robotics activity designed by
Chevalier et al. (2020) to promote the development of students’ CT skills
through the Thymio II robot. The Thymio II, for instance, is a widely used
educational robot equipped with various sensors, including proximity sensors,
an accelerometer, a remote control receiver, motors, a speaker, and LEDs,
distributed throughout its body (Riedo et al., 2013; Shin et al., 2014). The
authors aimed to address the issue of students spending excessive time
programming and not enough time problem-solving, referred to as the trial-and-
error loop, by conducting an instructional intervention on two groups of
primary school students.

Figure 12: The Thymio Lawnmower Mission adapted from Chevalier et al. (2020).
A group of pupils must program the Thymio II robot to pass over all eight
green lawn squares and avoid the fence (left). A special visual programming
language platform, with graphical icons that are straightly interpretable, is
used for this task (right).

##### Components

In the Thymio Lawnmower Mission, illustrated in Fig. 12, the Thymio II robot
must systematically traverse all green lawn squares, much like a lawnmower
would mow a lawn.

  * •

Problem solver: the group of students performing the task who must program the
agent’s lawnmower behaviour. The artefactual environment comprises tools
designed for reasoning and interaction, including a graphical programming
environment called VPL (symbolic), which allows for the creation of sensor-
action relationships to determine the robot’s behaviour, the agent (embodied)
and the visual feedback (embodied).

  * •

Agent: the Thymio II, whose actions consist in moving around the playground,
by changing velocity and orientation, and using sensors to detect obstacles.
All actions are considered irreversible.

  * •

Environment: the playground, i.e., a lawn area surrounded by a fence, divided
into eight green squares and one grey square, the garage. Its state is defined
by the grass condition or the number of squares passed over by the agent.

  * •

Task: find the algorithm. The initial state is the lawn with tall grass,
meaning the robot is not passed over any of the squares that compose it. The
final state is the same lawn with the grass mowed, meaning the robot passes
over all green squares. The algorithm is the set of moving actions to reach
the system’s final state from the initial.

##### Characteristics

In the Thymio Lawnmower Mission, the students who participated in the activity
were divided into two groups, the control group and the test group, to which
different conditions were imposed. The control group was allowed to complete
the task without any constraints. In contrast, the test group was subjected to
an instructional intervention that blocked the programming interface at
certain times to overcome the trial-and-error loop. As a result, the two
activity variants have distinct characteristics, analysed using the graphical
templates shown in LABEL:fig:TLMcontrol-features and LABEL:fig:TLMtest-
features in Appendix 0.B.

  * •

Tool functionalities: the system provides a comprehensive set of tools for the
problem solver to create and control the behaviour of the agent to solve the
task, including (i) variables such as the values of sensors or the state of
the robot; (ii) operators represent the basic actions that the agent can
perform; (iii) sequences are not represented by a specific block in the VPL
interface, but can be created by arranging blocks in a specific order; (iv)
functions are not represented by blocks in the VPL but refer to the
possibility of conceptually grouping blocks of code associated with a
particular behaviour to produce outputs given inputs; (v) events are directly
represented in the graphical interface by sensor-action relationships,
allowing the robot to perform actions in response to stimuli, such as
detecting an obstacle.

  * •

System resettability: in the control group, the problem solvers can reset the
system directly by physically moving the Thymio II agent in the environment
and restarting the task by repositioning it in the garage. On the other hand,
those in the test group do not have this option as they cannot directly
interact with the agent and cannot modify the algorithm since it has to be
first written and then executed.

  * •

System observability: the platform provides real-time visual feedback, making
the system observable.

  * •

Task cardinality: the task has a one-to-one mapping, with an initial and final
state and an algorithm.

  * •

Task explicitness: the elements of the task are given explicitly, as the
student is provided with clear instructions on what the outcome should look
like.

  * •

Task constraints: the algorithm is unconstrained.

  * •

Algorithm representation: the algorithm is written in the workspace and
expressed by the set of blocks and their connections.

##### Enabling features for competencies development

This paragraph explores the enabling characteristics that support the
development of competencies within the task. The relationship between features
and skills in the two activity variants are summarised in Appendix 0.B in
LABEL:tab:TLMcontrol-mapping and LABEL:tab:TLMtest-mapping.

  * •

Problem setting: all competencies can be activated thanks to the presence of
variables, sequences and functions in the tool functionalities. The presence
of many tool functionalities positively affects and boosts problem setting
skills. The manifest and written representation of the algorithm can further
encourage the development of these skills. In the control group, being the
system resettable, data collection, pattern recognition and decomposition are
promoted, while in the test group, abstraction and data representation are
also encouraged. The one-to-one cardinality of data elements also facilitates
data collection, pattern recognition and decomposition. The system
observability also supports data collection and pattern recognition.

  * •

Algorithm: all competencies associated with the algorithmic concepts enabled
by the tool functionalities, meaning variables, operators, sequences,
functions and events, can be activated in all three artefactual environments
and promote one another. These are further enhanced by the manifest and
written algorithm representation, system observability, and the explicit and
unconstrained definition of the task elements. The one-to-one cardinality
helps to develop variables and operators further.

  * •

Assessment: regarding the control group, algorithm debugging can be activated
in all artefactual environments since the algorithm has to be found, the
system is resettable, and the algorithm is manifest and written; optimisation
can be developed thanks to the resettability of the system; generalisation can
be activated through the system’s resettability and the presence of variables
and functions. The tool functionalities available and the system observability
help develop these skills as well. On the other hand, the system is non-
resettable in the test group, and no assessment skills can be developed.

##### Inhibiting features for competencies development

The lack of specific features may hinder the development of particular skills.

  * •

Repetitions, conditionals and parallelism: non-activable in both control and
test groups, as these functionalities are unavailable in the VPL programming
platform. Therefore, to develop these skills, it is necessary to change the
programming language to a textual programming language such as ASEBA.

  * •

Algorithm debugging: non-activable in the test group since the system is not
resettable.

  * •

System state verification: non-activable, in both control and test groups,
because the initial and final states of the system are given, and in the test
group, also because the system is not resettable.

  * •

Constraint validation: non-activable, in both control and test groups, due to
the absence of constraints on the algorithm, and in the test group because the
system is not resettable.

  * •

Optimisation: non-activable in the test group because the system is not
resettable.

  * •

Generalisation: non-activable in the test group due to the non-resettability
of the system.

####  3.2.2 Remote Rescue with Thymio II (R2T2)

R2T2 is another collaborative educational robotics activity, presented by
Mondada et al. (2016) to promote STEM education in schools and encourage
students towards careers in these fields.

Figure 13: The Remote Rescue with Thymio II (R2T2) mission on Mars adapted
from Mondada et al. (2016). Sixteen worldwide teams of pupils collaborate with
16 Thymio to restart the main generator of a simulated damaged power Mars
station (left) in five phases using a visual programming language or textual
programming language programming platforms (right).

##### Components

The R2T2 activity, illustrated in Fig. 13, is a rescue operation on a Mars
station, whose goal is to assess the damage of the power plant and restart the
main generator by remotely controlling 16 Thymio II robot. The activity is
divided into five phases, each with a specific objective. In the first phase,
the robots must enter the station and push away an obstacle blocking the main
door. In the second phase, the robots must stand on control spots to activate
access to the generator. In the third phase, the robots must look into the
generator through a small window. In the fourth phase, the robots must turn on
a light when detecting the generator rotor using proximity sensors and off
when it is no longer visible, thus estimating the generator speed. In the
final phase, the generator is restarted, and the mission is completed.

  * •

Problem solver: the group of students performing the task who must program the
agents’ behaviours to restart the main generator. The artefactual environment
comprises tools designed for reasoning, such as paper and pencils and another
robot that is physically accessible. Tools available also to interact with the
system include the programming platform (symbolic) with the two available
programming environments, VPL and ASEBA, a textual programming language
(Magnenat et al., 2011), and the five webcams, installed around the
playground, provide a delayed continuous visual feedback (embodied) through
YouTube video streams.

  * •

Agent: the 16 remote-controlled Thymio II, which can move around the
playground accelerating and rotating, use proximity sensors and turn some
lights on and off. All actions are considered irreversible.

  * •

Environment: the playground, i.e., the Mars station, characterised by
different descriptors used in the different mission stages, such as the
obstruction by the obstacle, covering of the control spots and finally, the
restart of the generator.

  * •

Task: find the algorithm. In the initial state, the generator is not working,
while it has been restarted at the end. The algorithm is the set of moving
actions to reach the system’s final state from the initial.

##### Characteristics

LABEL:fig:R2T2-features in Appendix 0.B provides the graphical template used
to analyse the task components and characteristics.

  * •

Tool functionalities: the system provides a comprehensive set of tools for the
problem solver to create and control the agent’s behaviour to solve the task,
depending on the programming environment used. VPL offers the possibility to
use variables, operators, sequences, functions and events. Additionally, ASEBA
offers control flows such as repetitions and conditionals. Furthermore,
parallelism is possible since it refers to the ability to run multiple
processes simultaneously, in this case, the Thymio II robots performing the
rescue operation in parallel. The agents can execute their tasks concurrently
without waiting for each other to complete them.

  * •

System resettability: the system cannot be reset due to the irreversible
nature of the actions carried out by the robots. Once the robots take an
action, the change in the system’s status is permanent and cannot be undone.
Furthermore, the physical separation between the problem solvers and the
system means no immediate way to reset the system.

  * •

System observability: the delayed but continued system visual feedback makes
it totally observable.

  * •

Task cardinality: the task has a one-to-one mapping, with an initial and final
state and an algorithm.

  * •

Task explicitness: all elements are given explicitly.

  * •

Task constraints: the algorithm is unconstrained.

  * •

Algorithm representation: the algorithm is manifest and written in the
programming platform.

##### Enabling features for competencies development

The relationship between features and skills is summarised in Appendix 0.B in
LABEL:tab:R2T2-mapping.

  * •

Problem setting: all competencies can be activated thanks to the presence of
variables, sequences and functions in the tool functionalities. The presence
of all tool functionalities, the manifest written representation of the
algorithm and the non-resettability of the system further encourage the
development of these skills. The system observability supports data collection
and pattern recognition. The one-to-one cardinality, in addition to these
skills, stimulates decomposition. The explicit and unconstrained definition of
the task elements also promotes pattern recognition, decomposition and
abstraction.

  * •

Algorithm: competencies across all artefactual environments can be triggered
since the tool’s functionalities enable all algorithmic concepts. The system
observability, the explicit and unconstrained definition of the task elements,
and the manifest written algorithm representation further enhance these.

  * •

Assessment: since the system is not resettable, there are no assessment skills
activable.

##### Inhibiting features for competencies development

The inability to reset the system restricts the activation of all assessment
skills. The task can be adjusted by incorporating a mechanism for resetting
the system to a previous state, enabling the problem solver to correct errors
made during the implementation of the algorithm, explore different solutions,
and learn from their mistakes. Additionally, it is necessary to omit the
initial or final states to verify the system’s state. Furthermore, to develop
the constraint validation skill, it is necessary to incorporate constraints
into the algorithm.

####  3.2.3 Ozobot maze

The Ozobot Maze activity is a screenless robotics task proposed by Bryndová
and Mališů (2020) aimed at teaching primary school students in the Czech
Republic CT skills. The educational robot used in this task is the Ozobot, a
small programmable robot, used to introduce students to coding, equipped with
sensors to follow black lines and read colour patterns called Color Codes to
change speed, direction and movements.

Figure 14: The Ozobot maze adapted from Bryndová and Mališů (2020). The task
requires the pupil to instruct the Ozobot to cross a maze avoiding obstacles
and reaching the room where the red person is. Commands such as increasing the
speed, changing direction and making some cool movements (spinning like a
tornado) are given in Color Codes.

##### Components

In the Ozobot Maze activity, illustrated in Fig. 14, the robot should be
guided through a maze to reach the room where the red person is.

  * •

Problem solver: the student who creates a suitable sequence of instructions
using Color Codes to guide the Ozobot through the maze. The artefactual
environment comprises tools for reasoning and interacting with the system.
Predefined stickers or markers to fill the empty Codes with colour sequences
(embodied) are used to give the robot the correct instructions to achieve the
goal. The visual feedback (embodied) lets the problem solver observe the agent
and its movements in the playground.

  * •

Agent: the Ozobot agent, which can move around in the playground by changing
velocity and orientation. This action is not reversible.

  * •

Environment: the playground, i.e., the house map, whose state is defined by
the agent’s position relative to the red person.

  * •

Task: find the algorithm. The initial state is the empty maze with the Ozobot
positioned near the starting point. The system’s final state is the Ozobot
reaching the end of the maze, in the room with the red person, and all the
Color Codes being filled. The algorithm is the set of agent instructions,
shown by the Color Codes, to reach the system’s final state from the initial.

##### Characteristics

The characteristics of this activity have been analysed using the graphical
templates shown in LABEL:fig:Ozobot-features in Appendix 0.B.

  * •

Tool functionalities: the system provides a comprehensive set of tools for the
problem solver to create and control the behaviour of the agent to solve the
task, including (i) variables can be used to store values such as the position
of the robot in the maze; (ii) operators are basic actions that the agent can
perform, represented by Direction Codes such as moving straight, turning left
or right; (iii) sequences represent the set of instructions used to control
the behaviour of the robot in a step-by-step manner and that the Ozobot must
follow to complete the task; (iv) repetitions are a way of repeating the same
instructions multiple times and refer to the possibility of repeating the same
Color Code multiple times, for example, if the agent encounters the same type
of intersection repeatedly in the path and the same Color Code is used to
specify the direction the agent should take; (v) conditionals are used to make
decisions based on certain conditions, for example understanding what to do at
an intersection; (vi) functions can be reflected in the different types of
Color Codes that can be reused in different situations and map inputs to
outputs, such as mapping a specific type of intersection to a specific
direction.

  * •

System resettability: the system is not resettable since it is impossible to
change the Color Codes once they have been filled in.

  * •

System observability: the real-time visual feedback makes the system
observable.

  * •

Task cardinality: the task has a one-to-one mapping.

  * •

Task explicitness: the elements of the task are given explicitly, as the
student is provided with clear instructions on what the outcome should look
like.

  * •

Task constraints: the algorithm is unconstrained.

  * •

Algorithm representation: the algorithm is manifest and written, expressed by
the set of Color Codes.

##### Enabling features for competencies development

The relationship between features and skills is summarised in Appendix 0.B in
LABEL:tab:Ozobot-mapping. This paragraph explores the enabling characteristics
that support the development of competencies within the task.

  * •

Problem setting: all competencies can be activated thanks to tool
functionalities such as variables, sequences and functions. The manifest and
written algorithm representation, as well as the non-resettability of the
system, can further encourage the development of these skills. The system
observability supports data collection and pattern recognition. The one-to-one
cardinality, in addition, stimulates decomposition. The explicit and
unconstrained definition of the task elements promotes pattern recognition,
decomposition and abstraction.

  * •

Algorithm: all competencies associated with the algorithmic concepts enabled
by the tool functionalities, meaning variables, operators, sequences,
repetitions, conditionals and functions, can be activated in all three
artefactual environments and promote one another. These are further enhanced
by the manifest and written representation of the algorithm, the system
observability, and the explicit and unconstrained definition of the task
elements. The one-to-one cardinality helps to enhance some of these skills as
well.

  * •

Assessment: since the system is not resettable, there are no assessment skills
activable.

##### Inhibiting features for competencies development

  * •

Parallelism and events: non-activable since the related features are missing
in the tool functionalities. To develop these skills, it is possible to switch
to a different interaction tool, such as OzoBlockly, a visual programming
language designed to code Ozobots Evo and includes these functionalities.

  * •

Assessment skills: The non-resettable feature of the system hinders the
development of assessment abilities. In this sense, by allowing the problem
solver to reset the system to a previous state, for example by letting them
change the Color Codes stickers and move the robot back to the starting
position, the activity can be improved. This way, the problem solver can
correct any mistakes made during the implementation of the algorithm,
experiment with different solutions, and learn from their mistakes. To develop
system state verification, it is also essential to not reveal the initial or
final states. Moreover, constraints should be imposed on the algorithm to
develop constraint validation skills.

####  3.2.4 Mini-golf challenge with micro:bit

In robotics, physical computing activities involve using microcontrollers,
sensors, and other electronic components to build and program interactive
systems. To enhance the learning experience, various off-the-shelf robotic
kits have been developed that allow students to construct robots easily and
control them through a graphical user interface.

In these activities, students are often engaged in an initial phase of
actively constructing the system using recycled materials, electronic circuits
and programming the robot. These activities evaluate the students’
understanding of algorithmic concepts, problem-solving skills, knowledge of
physics and engineering, creativity, and ability to work collaboratively.

Figure 15: The BBC micro:bit (left) and its block programming interface
(right).

One such physical computing activity is the Mini-golf challenge, proposed by
Assaf et al. (2021). In this activity, students are tasked with programming a
mini-golf lane’s moving and interactive elements using the BBC micro:bit (Ball
et al., 2016; Microbit, 2016). The micro:bit, depicted in Fig. 15, is a
pocket-sized computer that can be programmed using Microsoft’s MakeCode editor
(Makecode, 2016), which provides a user-friendly interface with colour-coded
blocks similar to Scratch and the ability to switch to JavaScript to view the
text-based code.

Figure 16: The Mini-golf challenge challenge adapted from Assaf et al. (2021).
The task requires a group of pupils to define the behaviour of the mini-golf
lane movable obstacles, sounds, and lights by programming the BBC micro:bit.

##### Components

In the Mini-golf challenge, illustrated in Fig. 16, the objective is to
program a mini-golf lane’s moving and interactive elements.

  * •

Problem solver: the group of students who must program the micro:bit. The
artefactual environment disposed of paper and pencil (embodied), a cognitive
tool to support the thinking phase. Other tools are provided to interact with
the system, including the toolkit (embodied) whose components can be assembled
and disassembled at will, the visual programming language offered by the
MakeCode editor (symbolic), and the visual feedback (embodied).

  * •

Agent: the micro:bit, which can use sensors, control the movement and actions
of the mini-golf elements, and turn LEDs and speakers on and off. All actions
are considered not reversible.

  * •

Environment: the assembled toolkit, which consists of various components,
including a ball, speakers, and lights. Its state is described by the state of
its elements, including the ball position, lights illumination and speakers
ignition.

  * •

Task: creation act. The initial state is given by the toolkit assembled. The
system’s final state and the students’ algorithm are open-ended, which defines
the mini-golf station elements’ behaviour.

##### Characteristics

The characteristics of this activity have been analysed using the graphical
templates shown in LABEL:fig:minigolf-features in Appendix 0.B.

  * •

Tool functionalities: the MakeCode editor allows using all the tool
functionalities we defined.

  * •

System resettability: the system can be directly reset by physically moving
the ball back to its starting position, and resetting the state of the lights
and speakers, for example, by turning them off. Additionally, the MakeCode
editor includes a convenient button to streamline the agent’s reset process.

  * •

System observability: the real-time visual feedback makes the system
observable.

  * •

Task cardinality: the task has a one-to-one mapping.

  * •

Task explicitness: the initial state of the toolkit, including the ball’s
position, the state of the lights, and the speakers, is not specified.

  * •

Task constraints: no constraints are imposed on the two elements to be found.

  * •

Algorithm representation: the algorithm is manifest and written in the
programming platform.

##### Enabling features for competencies development

The mapping between features and skills is summarised in Appendix 0.B in
LABEL:tab:minigolf-mapping. This passage analyses the characteristics that
support the development of competencies within the task.

  * •

Problem setting: all competencies can be activated thanks to the presence of
variables, sequences and functions in the tool functionalities. The
availability of numerous tool functionalities, the written algorithm
representation and the non-resettability of the system encourage the
development of problem setting skills. The implicit description of the task
elements creates an environment of uncertainty, allowing for multiple
interpretations and solutions, stimulating problem setting skills. The
system’s observability allows the development of data collection and pattern
recognition, while the one-to-one cardinality also encourages decomposition.
The unconstrained definition of task elements fosters pattern recognition,
decomposition, and abstraction.

  * •

Algorithm: all competencies can be activated since the tool functionalities
enable all algorithmic concepts in all three artefactual environments. The
system observability, the implicit and unconstrained definition of the task
elements and the algorithm’s manifest written representation further enhance
these. The one-to-one cardinality helps to enhance some algorithmic skills as
well.

  * •

Assessment: algorithm debugging and system state verification can be developed
in all artefactual environments by the direct resettability of the system and
the written representation of the algorithm; optimisation can be activated as
the resettability of the system alone is sufficient; generalisation is enabled
through the system’s resettability and the presence of variables and
functions. Tool functionalities, system observability and the implicit
definition of the task elements further support their development.

##### Inhibiting features for competencies development

The constraint validation skill cannot be developed as the algorithm and the
final state to be found are unconstrained. To encourage the development of
this competence, the activity can be adapted by introducing defined
constraints, such as a maximum number of moves for the ball to reach the hole
or that certain elements of the mini-golf station must be activated in a
specific order. These constraints will require students to evaluate the
feasibility of their solutions within the specified limitations and assess
their adherence to the established criteria.

###  3.3 Virtual activities

The last domain of educational activities analysed includes CTP characterised
by the presence of a virtual system. These activities typically involve
programming a virtual agent to perform a specific or a set of tasks in a
virtual environment. In contrast to CTP with a physical environment, virtual
activities always provide a virtual interface, often including a comprehensive
programming platform that allows the problem solver to use different types of
programming languages, such as textual programming language and visual
programming language. In some virtual games, the problem solver can interact
directly with the agent by clicking on it. This allows for greater flexibility
in terms of how the problem solver can program the virtual agent.
Additionally, virtual activities often include debugging tools, which allow
the problem solver to identify and fix errors in their code.

####  3.3.1 Classic Maze

Figure 17: The Angry Bird hitting the Green Pig maze adapted from
Studio.code.org (2020a). The problem solver must write a program to get the
Angry Bird through the maze to hit the Green Pig (left) by selecting the
instruction blocks (middle) and assembling them in the workspace (right).

In addition to unplugged activities, Code.org is a platform that offers many
coding activities for children based on Blockly, a Google framework for block-
based programming (Lovett, 2017). The Classic Maze is part of the Hour of Code
offered by Code.org, a worldwide effort to broaden participation in the
computer science field (Studio.code.org, 2020b). Participants must use block-
based programming to guide different characters through a maze in this
activity. The creatures include ones from popular franchises such as Angry
Birds, Plants vs Zombies, and Scrat from Ice Age. In this way, they learn the
foundations of computer science and algorithmic concepts by successfully
guiding the characters through the maze. We decided to analyse two activities
of the Classic Maze. Appendix 0.B includes the graphical template used to
analyse the components and characteristics of the task, found in
LABEL:fig:ABHGP-features and LABEL:fig:PZ-features, and the mapping between
the CTP features and the competencies in LABEL:tab:ABHGP-mapping and
LABEL:tab:PZ-mapping.

##### Components

In the first activity of the Classic Maze, presented in Román-González et al.
(2018) and illustrated in Fig. 17, the Angry Bird should be guided through a
maze to reach and hit a Green Pig.

  * •

Problem solver: the student performing the task who must program the agent’s
behaviour. The artefactual environment comprises tools designed for reasoning
and interacting with the system simultaneously, including the programming
platform composed of the virtual scenario (embodied artefact), the blocks and
the workspace (symbolic artefacts). The system also furnishes various hints to
users (embodied artefact), including video tutorials, guidance on how to use
the platform, command recommendations, suggestions on the number of blocks
required to solve the task, and feedback on the problem solver’s progress
towards a solution.

  * •

Agent: the Angry Bird, programmed to navigate a maze and hit the other
character. The agent’s actions comprise moving forward, turning left, and
right. Moving forward is considered a non-reversible action, whereas the
turning is reversible, as a turn left can be easily undone by a turn right,
and vice versa.

  * •

Environment: the virtual scenario where the two creatures are located,
described by their positions.

  * •

Task: find the algorithm. The initial state corresponds to the animals’
initial positions, while in the final, the characters are in the same
position. The algorithm is the set of moving actions to reach the system’s
final state from the initial.

Figure 18: The Plants vs Zombies maze adapted from Studio.code.org (2020c).
The problem solver must write a program to get the Zombie through a maze to
eat the plant (left) by selecting the instruction blocks (middle) to be
assembled in the workspace (right).

##### Characteristics

  * •

Tool functionalities: the programming platform enables problem solvers to
reason about the task at hand and program the movements of the red bird by
employing a set of predefined blocks, each representing a specific action the
agent is authorised to perform: (i) variables, while not explicitly delivered
in the blocks of the programming platform, can be inferred from the visual
feedback provided by the system, allowing the problem solver to store values
such as the position of the characters; (ii) operators represent the basic
actions that the agent can perform, such as moving or turning and are
represented by distinct blocks in the visual programming language depicted in
cyan by the platform; (iii) sequences are a series of blocks to be executed in
a specific order and are implicitly conveyed by the collection of blocks; (iv)
functions, which are self-contained blocks of code that perform a specific
task and can be executed multiple times with different inputs (e.g., different
initial positions of characters), are a concept of relative complexity. It is
not certain that the problem solver will recognise them as such rather than
just blocks.

  * •

System resettability: the platform provides a direct means of resetting the
task through the “start over” button, even if some agent actions are
irreversible. This allows the problem solver to start over and try a different
approach if necessary.

  * •

System observability: the system provides real-time visual feedback through
animations and graphical representations of the system state and its changes,
making it observable. The problem solver can monitor the effects of the
agent’s actions on the system, allowing him to have a complete understanding
of the system’s current state and to make informed decisions in their problem-
solving process.

  * •

Task cardinality: the task has a one-to-one mapping, with only one starting
position for the animal elements, one final position for the Angry Bird to be
placed, and only one algorithm to be found.

  * •

Task explicitness: the elements of the task are given explicitly through the
depiction of the scenario that clearly shows the animals’ positions.

  * •

Task constraints: there are no constraints on the elements to be found. All
the blocks provided are available without limitations.

  * •

Algorithm representation: the algorithm is written in the workspace and
expressed by the set of blocks and their connections.

##### Enabling features for competencies development

This paragraph explores the enabling characteristics that support the
development of competencies within the task. Certain features play a crucial
role in activating skills, while others can further enhance and encourage the
development of these competencies.

  * •

Problem setting: all competencies can be activated thanks to the presence of
variables, sequences, and functions in the tool functionalities. The manifest
and written algorithm representation can further encourage the development of
these skills. The resettability of the system and the one-to-one cardinality
of data elements facilitate data collection, pattern recognition and
decomposition. Observing the system also supports data collection and pattern
recognition. Using variables boosts pattern recognition and decomposition,
while explicit and unconstrained elements, in addition to these two skills,
encourage abstraction. Sequences positively affect pattern recognition,
abstraction and data representation, which is also facilitated by functions.
Operators also encourage decomposition.

  * •

Algorithm: all competencies associated with the algorithmic concepts enabled
by the tool functionalities, meaning variables, operators, sequences, and
functions, can be activated in all three artefactual environments. Features
such as variables, operators, sequences, and functions help activate the
algorithmic skills in a task. These are further enhanced by the manifest and
written algorithm representation, system observability, and the explicit and
unconstrained definition of the task elements. The one-to-one cardinality
enhances variables and operators.

  * •

Assessment: algorithm debugging can be activated in all artefactual
environments due to the resettability of the system and the manifest and
written representation of the algorithm; optimisation can be activated as it
only requires the resettability of the system; generalisation can be activated
through the system’s resettability and the presence of variables and
functions. Features such as variables, operators, sequences, functions and
system observability help develop algorithm debugging and optimisation, while
sequences can also foster generalisation.

##### Inhibiting features for competencies development

This paragraph explores the impact of missing features on skill activation,
focusing on how adjusting the task can enable the development of certain
competencies.

  * •

Repetitions, conditionals, parallelism, and events:non-activable due to the
absence of specific tool functionalities. These competencies cannot be
triggered until these functionalities are added to the tool.

  * •

System state verification: non-activable because both the initial and final
state are provided. The task can be adjusted to activate this skill by making
one of these states to be found. For instance, by not providing the position
of the Green Pig to the problem solver, who must then determine its location,
the system’s final state becomes unknown, and the system state verification
activable.

  * •

Constraint validation: non-activable because the algorithm to be found is
unconstrained. Some constraints can be imposed on the blocks used in the
task’s programming platform to activate this skill, such as limiting the
agent’s ability to turn right.

##### Comparison of two Classic Maze activities

The activities proposed in the Classic Maze exhibit a progressive increase in
difficulty. For example, the Plants vs Zombies maze, illustrated in Fig. 18,
is a similar task requiring finding the algorithm. The main differences
between this task and the previous are the characters involved, the difficulty
of the path that the agent, in this case, the Zombie, must traverse to reach
the plant, and the set of actions available to the agent. The programming
platform for this activity also includes the possibility of using repetitions,
represented in pink, adding another layer of difficulty. The features of the
CTP remain unchanged from the previous task, except for the addition of the
repetition functionality in the tool, which enables the activation of the
related competence.

####  3.3.2 Store the Marbles

Store the Marbles is a virtual programming activity, presented by Algorea, an
online resource designated by France-IOI, for learning the basics of
programming (ALGOREA, 2020; France-IOI.org, 2004). The activity is designed to
teach students problem-solving skills and programming concepts using a visual
block-based programming language. The activity is part of a series of
progressive difficulty courses and exercises available on the France-IOI
website.

Figure 19: The Store the Marbles activity adapted from ALGOREA (2020). The
task requires the problem solver to program the robot to produce an algorithm
valid for different situations using a visual programming language.

##### Components

In this activity, illustrated in Fig. 19, the robot should pick up the marble
on his path and drop it into a hole.

  * •

Problem solver: the student who must program the agent’s behaviour. The
artefactual environment comprises tools designed for reasoning and interacting
with the system simultaneously, including the programming platform composed of
the virtual scenario (embodied artefact), the blocks and the workspace
(symbolic artefacts). The system also furnishes suggestions on the number of
blocks required to solve the task (embodied artefact), but it does not provide
additional hints beyond this information.

  * •

Agent: the virtual robot, programmed to move on the path to collect the marble
and drop them in the hole. The agent’s actions comprise moving eastward,
picking up a marble, and dropping a marble. The simple movement is considered
irreversible, whereas picking up and dropping a marble are considered