[1,1,2,1,1,3]Philipp Bitzenbauer, Sebastian Höfler, Joaquin M. Veith, Bianca
Winkler, Tim Zenger, Christoph Kulgemeyer Friedrich-Alexander-Universität
Erlangen-Nürnberg, Professur für Didaktik der Physik, Staudtstr. 7, 91058
Erlangen, Germany,Stiftungsuniversität Hildesheim, Institut für Mathematik und
Angewandte Informatik, Samelsonplatz 1, 31141 Hildesheim, Germany,University
of Bremen, Institute for Science Education, Physics Education Department,
Otto-Hahn-Allee 1, 28359 Bremen, Germany Philipp
Bitzenbauer0000-0001-5493-291X
Joaquin M. Veith0000-0002-2147-0349
Christoph Kulgemeyer0000-0001-6659-8170 Correspondence:
<EMAIL_ADDRESS>
# Exploring the relationship between surface features and explaining quality
of YouTube explanatory videos

###### Abstract

Physics education research on explanatory videos has experienced a boost in
recent years. Due to the vast number of explanatory videos available online,
e.g. on YouTube, finding videos of high explaining quality is a challenging
task for learners, teachers, and lecturers alike. Prior research on the
explaining quality of explanatory videos on classical mechanics topics has
uncovered that the surface features provided by YouTube (e.g. number of views
or likes) do not seem to be suitable indicators of the videos’ explaining
quality. Instead, the number of content-related comments was found to be
statistically significantly correlated with the explaining quality. To date,
these findings have only been observed in the context of explanatory videos on
classical mechanics topics. The question arises whether similar correlations
between the explaining quality and YouTube surface features can be found for
videos on topics that are difficult to access visually and verbally, for
example from quantum physics. Therefore, we conducted an exploratory study
analyzing the explaining quality of $N=60$ YouTube videos on quantum
entanglement and tunnelling. To this end, we made use of a category-based
measure of explanatory videos’ explaining quality from the literature. We
report correlations between the videos’ explaining quality, and the surface
features provided by YouTube. On the one hand, our results substantiate
earlier findings for mechanics topics. On other hand, partial correlations
shed new light on the relationship between YouTube’s surface features and
explaining quality of explanatory videos.

###### keywords:

explaining quality, explanatory videos, entanglement, tunnelling, quantum
physics

##  1 Introduction

Quantum mechanics is a topical theme of physics in general [[, cf.]]Acin2018,
and of physics education research in particular [[, cf.]]BitzenbauerMDPI2021.
With today’s technological advancements, students may not only come into
contact with quantum physics in formal learning settings, e.g., in
undergraduate university courses (cf. [38, 25, 50, 51, 32, 34, 13]), but also
in the informal context: For example, interested learners can access the
quantum world via multiple digital resources, such as smart-phone/tablet apps
[[, e.g.,]]Oss2015, AR/VR applications [[, e.g.,]]Suprapato2020,Dorland2019,
games [[, e.g.,]]Chiofalo-2022,Zekir2022 or explanatory videos [[,
e.g.,]]Bitzenbauer2021b.

Explanatory videos are brief videos - typically up to 10 minutes maximum -
aimed at introducing and explaining a certain topic of interest [[,
cf.]]WolfKratzer2015neu. They have increasingly been discussed in science
education research in recent years [[,
e.g.,]]Pekag2010,Schroeder2017,Kulgemeyer2022, both in the context of formal
and informal learning environments, in particular on YouTube [[,
e.g.,]]Kulgemeyer2016neu,BeautempsBresges2021neu,Pattier-2021. In the
literature, factors that seem to be conducive to the success and popularity of
explanatory YouTube videos on scientific topics have been revealed [3, 46],
e.g., regarding the structure of a video [3]. While it is desirable to reach
as many people as possible, the main goal associated with the development of
explanatory videos, of course, is to support student learning. Besides
ensuring the success of the video, creators thus have to increase the quality
of the explanations offered in their explanatory videos.

From the physics education research perspective, it is crucial to assist
learners, teachers and university lecturers in selecting videos with high
explanation quality from the plethora of (online) resources. In the case of
YouTube explanatory videos, their popularity is publicly shown by means of
different surface features, such as the number of views, the ratings of the
video (e.g., the number of likes), or via the comment section. However, it
remains open as to whether or not these surface features indeed correlate with
the explanatory video’s explaining quality, and hence, may serve as some kind
of quality indicator in this respect. In other words: Can teachers and
students rely on them?

This question has already been posed by [21]. The authors presented a measure
of explaining quality to investigate the above-mentioned question in the
context of YouTube explanatory videos on two topics from classical mechanics,
namely Newton’s third law of motion and Kepler’s laws [21]. In their
exploratory study, the number of content-related comments given by users below
a specific video turned out to be the only variable that was statistically
significantly correlated with the explaining quality of explanatory videos -
neither the number of views, nor the number of likes or dislikes showed
correlations to explaining quality that were statistically significant [21].
[21] see the need for further studies on the relationship between surface
features provided by YouTube and explaining quality, in particular, regarding
other topics. They developed a hypothesis on this relationship that requires
further evidence. Videos on topics from quantum physics seems to add a
valuable perspective here.

Quantum physics differs fundamentally from classical mechanics, especially
since its concepts are not directly visible with the naked eye. Thus,
explanations of quantum physics topics arguably require specifically varied
explanations. As a result, the question arises as to whether or not the
metrics of YouTube explanatory videos about quantum concepts show similar
correlations to an established measure of explaining quality as has previously
been revealed by [21] for explanatory videos on classical mechanics topics.
This is where this research project comes in: We investigate the explaining
quality of YouTube explanatory videos on two genuine quantum physics topics
without classical analogies, namely quantum entanglement and quantum
tunnelling. To this end, the research methods used by [21] were leveraged into
our study. The objective of the research project presented in this article is
to expand on Kulgemeyer and Peters’ results by exploring correlations between
the YouTube surface metrics (e.g., likes, dislikes, views, number of days
since release, number of relevant comments) of explanatory videos on these two
quantum topics and the explaining quality of these videos [[,
cf.]]Kulgemeyer2016neu.

##  2 Research Questions

The present study addresses the following research questions:

  1. 1.

How is the explaining quality of YouTube explanatory videos on quantum
entanglement and quantum tunnelling correlated with the videos’ metrics such
as the number of views, the number of likes, or the number of dislikes?

  2. 2.

How is the number of content-related comments correlated with the explaining
quality of YouTube explanatory videos on quantum entanglement and quantum
tunnelling?

##  3 Research Background

###  3.1 Explaining physics

Instructional explanations are “designed with the specific purpose of teaching
a student or group of students” [[]p. 90]LeinhardtSteele2005neu. Hence,
instructional explanations need to be distinguished from scientific
explanations [[]]Treagust1999. [47] uncovered four factors that lead to
effective instructional explanations; for example, they should…

  1. 1.

…“be adapted to the learner’s knowledge prerequisites” [[]p.
51]WittwerRenkl2008neu,

  2. 2.

…“focus on concepts and principles” [[]p. 53]WittwerRenkl2008neu,

  3. 3.

…“should be integrated into the learners’ ongoing cognitive activities” [[]p.
55]WittwerRenkl2008neu, and

  4. 4.

…“should not replace learners’ knowledge-construction activities” [[]p.
56]WittwerRenkl2008neu.

These factors have been expanded to a total of nine factors in a 2019 review
addressing instructional explanations in science teaching [[]p.
90]Kulgemeyer2019-Stud. An important criterion for effective instructional
explanations is the adaption to the explainee because this criterion mirrors
that explaining is to be regarded a constructivist process
[[]]Kulgemeyer2016neu.

The constructivist nature of explanations is reflected in the communication
model for explaining physics presented by [22]. This model consists of four
pillars, namely the explainer, the explanation itself, the explainee, and the
explainee’s feedback. The fact that a good explanation requires

  1. 1.

constant evaluation of the explainee’s feedback, and

  2. 2.

prompt adaptation of the explanation based on that feedback,

is at the heart of this model [[]]KulgemeyerSchecker2013neu. According to the
communication model for explaining physics, “the explainer can vary the
explanation on four levels based on this feedback, ranging from the language
code, the graphic representation form and the mathematic code, to using
examples and analogies” [[]p. 3]Kulgemeyer2016neu.

###  3.2 Design principles for explanatory videos

The Cognitive load theory [[,
e.g.,]]Sweller1988Neu,Sweller1994neu,Sweller1998Neu assumes a limited capacity
of working memory caused by a cognitive load on learners in learning
environments, which - in its modern view [[, cf.]]Sweller-2019 - is composed
by

  * •

intrinsic cognitive load which is dependent on the concrete learning task, the
students’ prior knowledge, or the teaching materials used, and

  * •

extraneous cognitive load stemming from irrelevant cognitive processes that
tie up working memory capacities and thus hinder the learning process.

According to [40], the Cognitive load theory “provides evidence-informed
principles that can be applied to the design of instructional messages or
relatively short instructional units, such as lessons, written materials
consisting of text and pictures, and educational multimedia” (p. 274).

The Cognitive Theory of Multimedia Learning [[, cf.]]Mayer-1999 builds upon
the above-mentioned Cognitive load theory. This theory is based on three
fundamental assumptions that, taken together, describe how auditory-verbal or
visual-imagery information is processed toward long-term memory:

  * •

The Dual-Channel assumption describes that “humans possess separate channels
for processing visual and auditory information” [[]p. 63]Mayer-2009.

  * •

The Limited-Capacity assumption describes that each of the above-mentioned
channels can only process a limited amount of “chunks” [[]p. 67]Mayer-2009 of
information simultaneously.

  * •

The Active-Processing Assumption describes that students’ active engangement
is necessary for students constructing knowledge [[]]Mayer-2009.

Both the Cognitive load theory and the Cognitive Theory of Multimedia Learning
have been the basis for prior research on explanatory videos aimed at
fostering student learning [[, cf.]]Noor-2014,Kruger-2016. In addition,
different studies have derived design principles that may influence the
effectiveness of explanatory videos against the backdrop of the above-
mentioned theories [[, e.g.,]]Brame2016neu,Muller-2008,Kay2014neu. For
example, it has been indicated that the integration of interactive elements
into explanatory videos [[]]Delen2014 or the use of a 1st person perspective
in explanatory videos [[]]Fiorella2017 might have a positive impact on
students’ performance. [11] reviewed and systematized studies dealing with
potential effects of explanatory videos’ design principles on student
learning, and derived guidelines for the development of explanatory videos
based on the overall picture emerging from current empirical findings.

###  3.3 Explaining quality of explanatory videos

In the previous sections, we reviewed both the current state of research on
explaining physics and on design criteria for the development of explanatory
videos. In this section, both perspectives are merged in order to shed more
light on the state of research on the explanatory quality of explanatory
videos.

[20] presented a framework for effective explanation videos. This framework is

  * •

…consistent with guidelines on the quality of explanatory videos published
elsewhere in the literature [[, e.g.,]]Findeisen2019,Brame2016neu, and

  * •

…acknowledges research on multimedia learning [[]]Kulgemeyer2020Framework,

while building upon state-of-the-art research on instructional explanations
[[, e.g.,]]Geelan2012,WittwerRenkl2008neu. In this framework, seven factors
comprising a total of 14 features are described to have an impact on the
effectiveness of explanatory videos [[]p. 2450]Kulgemeyer2020Framework.
Examples are the use of summaries (factor: structure of the video), the use of
an appropriate language-level (factor: tools for adaption), the avoidance of
digressions (factor: minimal explanation), or the adaption to prior knowledge,
misconceptions and interest (factor: adaption). An overview of the whole
framework for effective explanation videos is presented in [[]p.
2450]Kulgemeyer2020Framework.

The above-mentioned framework has been tested empirically in order to clarify
as to whether or not an explanatory video developed with respect to the
framework leads to higher student achievement compared to a video that has not
strictly been developed according to the framework
[[]]Kulgemeyer2020Framework. The results of this study revealed that students
learning with an explanation video adhering strongly to this framework showed
significantly more declarative knowledge in a post-test than students learning
with a video that has not strictly been developed according to the framework
($d=0.42$). However, no statistically significant difference in the post-test
scores regarding conceptual knowledge was observed.

###  3.4  Evaluation of explanatory videos’ explaining quality

An online test which allows for the assessment of physics explanatory skills
has been published by [2]. This test has been developed both for its usage in
teacher education and for self-assessment.

Moreover, based on the communication model for explaining physics
[[]]KulgemeyerSchecker2013neu, [[]p. 121]KulgemeyerTomczyszyn2015neu developed
a process-oriented and category-based measure for the assessment of
explanation skills. [[]]Kulgemeyer2016neu, adopted this category-based measure
for the evaluation of explanatory videos’ explaining quality. The category
system to evaluate explanatory videos’ explaining quality (cf. appendix)
consists of seven main categories (content, structure, use of language,
contexts and examples, mathematics, interrogation, non-verbal elements)
comprising a total of 31 different categories. Each of these categories is
either assigned to a certain explanatory video (= 1 point) or not (= 0
points). Four out of the 31 subcategories (1. scientific mistake, 2. ignoring
students’ comment, 3. leaving new technical term uncommented, 4. without
context) are related to a decrease of explaining quality, and hence, a
negative point (= -1 point) is allocated to the video for their occurrence.

Within the scope of evaluating the explaining quality of explanatory videos
(i.e., in the course of categorisation), each category is considered uniformly
and there is no counting of a successive occurrence of the same category,
“since repetitions of the same wording or the repeated use of a similar
explaining aid without any variation are not considered a rich and varied
explanation” [[]p. 6]Kulgemeyer2016neu. By summing up the points received on
the basis of the categories assigned, a specific number of “category points”
[[]p. 6]Kulgemeyer2016neu, referred to as CP, can be calculated for a given
explanatory video [[]p. 6]Kulgemeyer2016neu:

$\textrm{CP}=\sum X_{+}+\sum X_{-},$

where $X_{+}$ denotes the number of positive categories assigned to a video,
and $X_{-}$ stands for the number of all negative categories assigned to a
video. The category points (with the upper limit of $28$ CP) serve as a
measure of an explanatory video’s explaining quality as has been shown by
[21].

It is important to note that the category points assigned to a specific
explanatory video may neither judge the video’s overall quality (e.g., a
video’s technical design is not taken into account), nor do the CP help
finding the best explanation of a specific topic under investigation among
multiple explanatory videos. Instead, the rationale underlying this measure is
“to distinguish between rich and varied explanations on the one hand and those
with fewer variations on the other” because “those with fewer variations in
their explanations may be less suitable for a wider range of viewers as some
learners’ needs may not be considered” [[]p. 9]Kulgemeyer2016neu.

##  4 Methods

In this section, we outline the methodology applied in our exploratory study
to approach a clarification of the research questions. We aim at expanding on
Kulgemeyer and Peters’s study according to which none of the correlations
between the surface features provided for YouTube explanatory videos and their
explaining quality was statistically significant, except from the number of
content-related comments [[]]Kulgemeyer2016neu. In a further study, [16] even
conclude that the “number of views, likes, dislikes, and comments per day is
not a predictor of high-quality videos on YouTube” (p. 1267).

###  4.1 Sample

####  4.1.1 Content domain

We decided to analyze YouTube explanatory videos on two topics: (a) quantum
entanglement, and (b) quantum tunnelling. We analyzed videos addressing these
topics because neither quantum entanglement nor quantum tunnelling has any
classical analogy and the quantum physics formalism does not enable a space-
time description of these concepts [[, cf.]]Ubben2022. In this way, our study
allows best to contrast the previous findings of [21] who analyzed explanatory
videos on topics of classical mechanics.

####  4.1.2 Inclusion-exclusion criteria and search procedure

Following [21], we found the videos to be included in our sample via YouTube’s
search engine applying the search strings “quantum entanglement” and “quantum
tunnelling”, respectively. We used the following inclusion-exclusion criteria
for selecting videos appropriate for data analysis:

  * •

The video is published in the English language.

  * •

The video exclusively covers one of the two topics quantum entanglement or
quantum tunnelling, respectively. Videos that covered both topics were
excluded.

  * •

Video-recorded lectures (or excerpts thereof) were excluded, since recorded
lectures “do not share the explainers’ intentional core of publishing a
concise explanatory video” [[]p. 4]Kulgemeyer2016neu.

  * •

The video has a maximum duration of 10 minutes.

  * •

The video is scientifically sound [[, cf.]]Kulgemeyer2016neu.

The latter criterion was important because it only makes sense to compare “the
explaining quality of scientifically correct explanations” [[]p.
5]Kulgemeyer2016neu. Applying the above-mentioned search strings, we found
more than 100.000 videos on both topics. A title-caption screening of the
search results led to the exclusion of the majority of these videos since they
did not fulfill the inclusion criteria (in this stage most often due to a
duration above 10 minutes, the coverage of topics beyond the ones under
investigation, or representing recorded lectures). In a next step, we reviewed
about 200 videos on each of the topics quantum entanglement or tunnelling in
detail. Again, we excluded the videos that did not fulfill the inclusion
criteria (in this stage most often due to serious scientific errors). Lastly,
for our final sample, we (a) settled on videos with comparable run-times of
around 5 minutes as has been done in the prior study conducted
by[[]]Kulgemeyer2016neu, and (b) aimed for a sample size comparable to the one
of the earlier study in the classical mechanics context [[]]Kulgemeyer2016neu.
The final sample consists of $60$ YouTube explanatory videos that were
included for data analysis, $30$ of which address the topic of quantum
entanglement, and $30$ of which focus on quantum tunnelling.

####  4.1.3 Description of the sample

The mean duration of the selected videos is $m=4.97$ min with a standard
deviation of SD $=2.43$ min. The explanatory videos on quantum entanglement
($m=4.74$ min, SD $=2.38$ min) were of similar length as those on quantum
tunnelling ($m=5.20$ min, SD $=2.48$ min). Moreover, the videos in our sample
are of similar length as the ones included in the prior study [[,
cf.]]Kulgemeyer2016neu.

###  4.2 Data collection

The explanatory videos included in the final selection have been analyzed in
August and September 2021. For the exploration of our research questions, the
data collection comprised three aspects: In a first step, we collected each
videos’ surface features, i.e., the number of likes and dislikes, the number
of views, and the publication date to calculate the videos’ time online (in
days). Additionally, we recorded the number of subscribers to the channels by
which the videos were published. The average view duration has been a further
surface feature which was included in Kulgemeyer and Peter’s study on
explanatory videos on classical mechanics topics [[]p. 5]Kulgemeyer2016neu.
However, at the time of conducting our data collection this feature was not
publicly accessible anymore and hence, it is not included in our analysis. In
addition, the dislike statistic is not publicly available anymore since the
end of 2021 - since our data collection was conducted in August and September
2021, however, we kept the number of dislikes for each video in our dataset
and also included it in the data analysis. This allows for a more
comprehensive comparison to the earlier results published by [21] and may help
to better understand the interaction of users with explanatory videos. For a
description of all the above-mentioned YouTube metrics, we refer the reader to
the [[]]YouTube.

In a second step, we categorised the comments given below the videos in order
to receive the number of relevant comments for each video. We provide a proper
description of (a) the term relevant comment and (b) the categorisation
procedure in the data analysis section. We explored relevant comments because
they “provide by far the most intense communication channel between explainer
and addressee” [[]p. 5]Kulgemeyer2016neu.

Lastly, following the data collection method from [21], we used the category
system described above (cf. appendix) to assess the explaining quality of the
explanatory videos included in our sample. The coding was performed by two
independent raters. The inter-rater reliability expressed via Cohen’s Kappa
can be regarded substantial ($\kappa=0.79$) according to [8]. Against this
backdrop, the category system used in this study allows for an objective
assessment of the explaining quality of explanatory videos. Furthermore, the
reliability of the measure has been found to be satisfactory (Cronbach’s
$\alpha=0.58$; in the earlier study by [21], a comparable value of
$\alpha=0.69$ has been reported). Moreover, the category-system used for this
study allows for a valid measure of explanatory videos’ explaining quality as
has been justified by [21].

As a last step of data collection, we calculated the category points CP for
each explanatory video included in our sample. These category points were then
further processed to data analysis.

###  4.3 Data analysis carried out the answer research question 1

We report descriptive statistics (range, median Mdn, mean $m$, standard
deviation SD) regarding the category points of the explanatory videos on
quantum entanglement and quantum tunnelling, respectively.

We conducted a correlation analysis in order to explore relationships between
the videos’ explaining quality (in category points CP) on the one hand, and
the surface features provided by YouTube on the other hand. We report
Pearson’s correlation coefficient $r$ because the data are of metric scale. We
interpret correlation coefficients according to [8]: weak correlation for
$0.1<|r|<0.3$, moderate correlation for $0.3\leq|r|<0.5$, strong correlation
for $|r|\geq 0.5$. In addition, we report partial correlations to verify that
observed relationships are no artefact caused by

  * •

the videos’ time online, i.e. the time that has passed between the publication
of a video and the data collection, and

  * •

the number of subscribers to the channels by which the videos were published.

The latter control variable seems particularly important due to the fact that
the YouTube algorithms promote videos published by popular channels which in
turn leads to high numbers of views for these videos. This might influence the
results, and hence, deserves special attention.

###  4.4 Data analysis carried out the answer research question 2

The comments below each video included in our sample have been categorised.
For the categorisation, we used the category system presented by [[]p.
8]Kulgemeyer2016neu which consists of four categories:

  1. 1.

Comment on content: “further question or comment on notations” [[]p.
8]Kulgemeyer2016neu.

  2. 2.

Comment on explanation: “constructive criticisms and inquiries for more
videos” [[]p. 8]Kulgemeyer2016neu.

  3. 3.

Comment on explainer’s style: “comments on the style including a reason” [[]p.
8]Kulgemeyer2016neu.

  4. 4.

Comment on use: description of “the viewer’s use of the video, e.g., revising,
preparing a talk or learning for a test” [[]p. 8]Kulgemeyer2016neu.

All comments that could be assigned to at least one of these categories, were
considered as relevant comments. Comments that could not be assigned to any of
these categories, conversely, were excluded from further analysis because they
were not related specifically to the content presented in the respective video
or to the explanation offered within. For the further analysis, we refrained
from a deeper differentiation between the different categories as has been
done by [21] because research question 2 only addresses relevant comments in
general.

The categorisation of the all comments underneath $N=60$ explanatory videos
included in our sample led to a total of $1452$ relevant comments. The number
of relevant comments for each video was included in our data set as a metric
variable and was used for correlation analysis. Again, we additionally
calculated partial correlations to verify that observed relationships are no
artefact caused by the videos’ time online, or the number of subscribers to
the channels by which the videos were published.

##  5 Results

###  5.1 Descriptives

The median value of the explanatory videos’ explaining quality (measured in
CP) was Mdn $=11$ CP for our total sample, ranging from $2$ CP (one video) to
$18$ CP (two videos). In table 5.1, descriptive statistics on the category
points assigned to the videos comprised in our sample are reported separately
for the two subject areas under investigation, namely quantum entanglement and
quantum tunnelling, respectively.

Table 1: Descriptive statistics on the measure of explaining quality of the
videos included in our sample (expressed in category points CP).

|  Range |  Mdn |  $m$ |  SD
---|---|---|---|---
Explaining quality
Total sample a |  $2-18$ |  $11.00$ |  $11.02$ |  $3.28$
Explaining quality
quantum entanglement b |  $2-18$ |  $11.00$ |  $11.03$ |  $3.62$
Explaining quality
quantum tunnelling c |  $4-18$ |  $11.50$ |  $11.00$ |  $2.96$
|  |  |  |

[para,flushleft] Note. a $N=60$. b $N=30$. c $N=30$.

###  5.2 Correlation analysis

The correlation analysis results are summarized in table 2. Within the total
sample, we find statistically significant correlations between the videos’
explaining quality and the number of views ($r=0.27$, $p<0.05$), the number of
likes ($r=0.37$, $p<0.01$), and the number of dislikes ($r=0.32,p<0.05$). The
highest correlation is uncovered between the videos’ explaining quality and
the number of relevant comments ($r=0.46$, $p<0.01$), whereas the correlation
between the videos’ explaining quality and their time online does not differ
from $0$ with statistical significance.

Table 2: Pearson’s correlation coefficient $r$ between the measure of
explaining quality (in CP) and the surface features (incl. number of relevant
comments) for the total sample, the videos on quantum entanglement and the
ones on quantum tunnelling, respectively. For all correlations, we report
$95\%$ confidence intervals ($95\%$-CI).

|  Surface feature |  $r$ |  $95\%$-CI
---|---|---|---
Explaining quality |  Time online (days) |  $-0.14$ |  $[-0.38;0.12]$
Total sample a |  Views |  $0.27^{*}$ |  $[0.02;0.49]$
|  Likes |  $0.37^{**}$ |  $[0.13;0.57]$
|  Dislikes |  $0.32^{*}$ |  $[0.06;0.53]$
|  Relevant comments |  $0.46^{**}$ |  $[0.24;0.64]$
Explaining quality |  Time online (days) |  $0.00$ |  $[-0.36;0.36]$
quantum entanglement b |  Views |  $0.32$ |  $[-0.05;0.61]$
|  Likes |  $0.42^{*}$ |  $[0.08;0.68]$
|  Dislikes |  $0.37$ |  $[0.00;0.65]$
|  Relevant comments |  $0.59^{**}$ |  $[0.29;0.79]$
Explaining quality |  Time online (days) |  $-0.28$ |  $[-0.58;0.09]$
quantum tunnelling c |  Views |  $0.23$ |  $[-0.15;0.54]$
|  Likes |  $0.30$ |  $[-0.07;0.60]$
|  Dislikes |  $0.30$ |  $[-0.07;0.60]$
|  Relevant comments |  $0.31^{(*)}$ |  $[-0.06;0.60]$
|  |  |

[para,flushleft] Note. Statistical significance of the correlations is denoted
by an asterisk: (*)p ¡ .10. *p ¡ .05. **p ¡ .01. a $N=60$. b $N=30$. c $N=30$.

A striking observation is the positive correlation between the number of
dislikes and the measure of explaining quality, both in the total sample
($r=0.37$, $p<0.01$) and the two sub-samples including videos on quantum
entanglement ($r=0.37$) and quantum tunnelling ($r=0.30$). In order to better
understand the underlying principles, we decided to introduce three further
variables into our analysis:

  1. 1.

We calculated the likes-to-dislikes ratio for each video included in our
sample. This variable allows to contrast the frequency of occurrence of likes
to that of dislikes for a given video, and thus could potentially be a more
accurate measure of the quality of an explainer video. A similar approach has
already been taken by [28].

  2. 2.

We assumed that the interaction with a specific explanatory video, i.e.,
giving a like or a dislike to a video, requires the user to be cognitively
activated to some extent. We therefore introduced the variable interactions
calculated via interactions = ∑likes + ∑dislikes, to explore the relationship
between explaining quality and the number of interactions. This might provide
further insights into how users interact with explanatory videos depending on
their explaining quality.

  3. 3.

Lastly, to check as to whether the number of likes and dislikes are really
relevant variables with respect to explanatory videos’ explaining quality, we
calculated the the likes-to-interactions ratio via likesinteractions, because
for a high quality explanatory video one could expect a high number of likes
compared to the total number of interactions. Note that the dislike-to-
interactions ratio does not contain any further information, and hence, leads
to mathematically equivalent results in the correlation analysis (up to sign).

The correlations between the measure of explanatory videos’ explaining quality
and the three above-mentioned variables are shown in table 3: It becomes
apparent that neither the likes-to-dislike-ratio ($r=0.11$) nor the likes-to-
interactions ratio ($r=-0.03$) is correlated statistically significantly with
the explaining quality of the explanatory videos included in our sample. The
only variable showing moderate but statistically significant correlation to
the videos’ explaining quality is the number of interactions itself ($r=0.39$,
$p<0.01$). From these results, of course, no particular (and a fortiori no
causal) relationship between

  1. 1.

the number of likes or dislikes (and their ratio), and

  2. 2.

the videos’ explaining quality can be inferred.

Table 3: Pearson’s correlation coefficient $r$ of the measure of explaining
quality (in CP) with the variables likes-to-dislike ratio, interactions, and
likes-to-interactions ratio, respectively. For all correlations, we report
$95\%$ confidence intervals ($95\%$-CI).

|  Variable |  $r$ |  $95\%$-CI
---|---|---|---
Explaining quality |  Likes-to-dislikes ratio |  $0.11$ |  $[-0.21;0.40]$
Total sample a |  Interactions |  $0.39^{**}$ |  $[0.14;0.59]$
|  Likes-to-interactions ratio |  $-0.03$ |  $[-0.28;0.23]$
Explaining quality |  Likes-to-dislikes ratio |  $-0.01$ |  $[-0.43;0.41]$
quantum entanglement b |  Interactions |  $0.44^{*}$ |  $[0.09;0.69]$
|  Likes-to-interactions ratio |  $-0.03$ |  $[-0.40;0.34]$
Explaining quality |  Likes-to-dislikes ratio |  $0.22$ |  $[-0.26;0.61]$
quantum tunnelling c |  Interactions |  $0.31^{(*)}$ |  $[-0.06;0.61]$
|  Likes-to-interactions ratio |  $-0.03$ |  $[-0.39;0.34]$
|  |  |

[para,flushleft] Note. Statistical significance of the correlations is denoted
by an asterisk: (*)p ¡ .10. *p ¡ .05. **p ¡ .01. a $N=60$. b $N=30$. c $N=30$.

However, it seems that the total number of user interactions is correlated
significantly with the videos’ explaining quality - no matter of whether these
interactions result in a like or a dislike in the end. It is necessary to
control the correlations presented in tables 2 and 3 for the videos’ time
online (in days), and the number of subscribers to the channels by which the
videos were published in order to explore this in more detail. Therefore, we
report partial correlations in the next subsection.

###  5.3 Partial correlations

In this subsection, we report partial correlations which refer to the entire
sample. This means that we do not distinguish between the sub-samples here for
the sake of clarity.

Controlling the correlations between our explanatory videos’ explaining
quality (measured in CPs) and the YouTube surface features for the videos’
times online, we observe the following (cf. table 4): Besides significant
correlations between explaining quality and the number of views ($r=0.33$,
$p<0.05$), the number of likes ($r=0.40$, $p<0.01$), the number of dislikes
($r=0.33$, $p<0.05$), and the number of relevant comments ($r=0.55$,
$p<0.001$), only the number of interactions ($r=0.41$, $p<0.01$) shows a
significant correlation to the explaining quality of our videos. These partial
correlations uncover similar relationships between YouTube’s surface metrics
and the videos’ explaining quality as the ones presented earlier (cf. table
2).

Table 4: Partial correlations (controlled for the time online) between the
measure of explaining quality (in CP) and YouTube’s surface metrics as well as
the likes-to-dislike ratio, the number of interactions, and likes-to-
interactions ratio.

Controlled for: time online (days) |  Variable |  $r$
---|---|---
Explaining quality |  Views |  $0.33^{*}$
Total sample a |  Likes |  $0.40^{**}$
|  Dislikes |  $0.33^{*}$
|  Relevant comments |  $0.55^{***}$
|  Likes-to-dislikes ratio |  $0.08$
|  Interactions |  $0.41^{**}$
|  Likes-to-interactions ratio |  $-0.05$
|  |

[para,flushleft] Note. Statistical significance of the correlations is denoted
by an asterisk: *p ¡ .05. **p ¡ .01. ***p ¡ .001. a $N=60$.

In a next step, we controlled for the number of subscribers to the channels by
which the videos were published. The corresponding partial correlations are
shown in table 5: Only three of the correlations remain statistically
significant in this case, namely the ones between the explanatory videos’
explaining quality and the number of likes ($r=0.43$, $p<0.01$), the number of
relevant comments ($r=0.47$, $p<0.001$), and the number of interactions
($r=0.43$, $p<0.01$). In contrast, both the correlations of the videos’
explaining quality to the number of views, and the number of dislikes are not
statistically significant anymore. We will discuss these observations in the
discussion section.

Table 5: Partial correlations (controlled for the number of subscribers)
between the measure of explaining quality (in CP) and YouTube’s surface
metrics as well as the likes-to-dislike ratio, the number of interactions, and
likes-to-interactions ratio.

Controlled for:
Number of subscribers |  Variable |  $r$
---|---|---
Explaining quality |  Views |  $0.23$
Total sample a |  Likes |  $0.43^{**}$
|  Dislikes |  $0.26$
|  Relevant comments |  $0.47^{***}$
|  Likes-to-dislikes ratio |  $0.08$
|  Interactions |  $0.43^{**}$
|  Likes-to-interactions ratio |  $-0.06$
|  |

[para,flushleft] Note. Statistical significance of the correlations is denoted
by an asterisk: *p ¡ .05. **p ¡ .01. ***p ¡ .001. a $N=60$.

##  6 Discussion

In our exploratory study, we investigated as to how the explaining quality of
YouTube explanatory videos on genuine quantum topics such as quantum
entanglement and quantum tunnelling is correlated with the surface features
provided by YouTube alongside each online video. In this section, we discuss
the results of our study with regards to our research questions, and against
the backdrop of a study published earlier that explored similar questions for
explanatory videos on classical mechanics topics [[, cf.]]Kulgemeyer2016neu.

###  6.1 Discussion of research question 1

A correlation analysis revealed statistically significant correlations between
the explanatory videos’ explaining quality and the surface features provided
by YouTube (cf. table 2):

  * •

The correlation between the number of views and the explanatory videos’
explaining quality is small and statistically significant for the total sample
($r=0.27$, $p<0.05$) but not statistically significant for the videos on
quantum entanglement and quantum tunnelling.

  * •

The correlation between the number of likes and the explanatory videos’
explaining quality is moderate and statistically significant for the total
sample ($r=0.37$, $p<0.01$) and for the sub-sample including quantum
entanglement videos ($r=0.42$, $p<0.05$). For the videos on quantum
tunnelling, however, the correlation is not statistically significant.

  * •

The correlation between the number of dislikes and the explanatory videos’
explaining quality is moderate and statistically significant for the total
sample ($r=0.32$, $p<0.05$). In contrast, it is not statistically significant
for the videos on quantum entanglement and quantum tunnelling.

Our results compare well with the findings reported earlier for the mechanics
context [[, cf.]]Kulgemeyer2016neu: While the correlations presented in both
studies seem different at first glance (cf. table 6), we note that most of the
correlations reported by [21] fall within the $95\%$ confidence intervals of
our correlation coefficients (or vice versa). In additon, our results also
shed new light on the underlying relationships: In their 2016 article [21]
found no statistically significant correlation between the videos’ explaining
quality and the number of likes although the authors expected such a
correlation due to the ’illusion of understanding’: “Students do not realise
the possible inconsistencies in their understanding and feel as if they have
understood a topic” [[]p. 11]Kulgemeyer2016neu. This assumption is supported
by empirical evidence from a recently published experimental study by [19].
For the explanatory videos on quantum topics included in our sample, we indeed
uncovered a statistically significant correlation between the number of likes
and the videos’ explaining quality ($r=0.37$, $p<0.01$).

Table 6:  Pearson’s correlation coefficient $r$ between the measure of
explaining quality (in CP) and the surface features provided by YouTube. For
the correlations calculated in our study, we report $95\%$ confidence
intervals ($95\%$-CI).

|  Surface feature |  $r$ |  $95\%$-CI
---|---|---|---
Explaining quality |  Time online (days) |  $-0.14$ |  $[-0.38;0.12]$
Videos on quantum topics a |  Views |  $0.27^{*}$ |  $[0.02;0.49]$
|  Likes |  $0.37^{**}$ |  $[0.13;0.57]$
|  Dislikes |  $0.32^{*}$ |  $[0.06;0.53]$
|  Relevant comments |  $0.46^{**}$ |  $[0.24;0.64]$
Explaining quality |  Time online (days) |  $-0.05$ |  -
Videos on mechanics topics b |  Views |  $-0.26$ |  -
[[]]Kulgemeyer2016neu |  Likes |  $0.21$ |  -
|  Dislikes |  $-0.09$ |  -
|  Relevant comments |  $0.38^{**}$ |  -
|  |  |

[para,flushleft] Note. Statistical significance of the correlations is denoted
by an asterisk: *p ¡ .05. **p ¡ .01. a $N=60$, this study. b $N=51$, [[,
cf.]p. 10]Kulgemeyer2016neu.

Moreover, we find the number of dislikes ($r=0.32$, $p<0.05$) and the number
of views ($r=0.27$, $p<0.05$) to have statistically significant correlations
with the explaining quality of the videos on quantum entanglement and
tunnelling. In contrast, [21] have not found the corresponding correlations to
be statistically significant for the videos on classical mechanics topics. The
analysis of partial correlations, though, puts these differences between the
two studies into perspective: We controlled the correlations between the
videos’ explaining quality and the surface features provided by YouTube for
the number of subscribers to the channels by which the videos were published.
As a result, the correlation between explaining quality and views ($r=0.23$)
loses its statistical significance. To describe this observation, we go along
with [21] who state that “the number of views is more influenced by […] the
popularity of the YouTube channel than the explaining quality” (p. 5).
Accordingly, the correlation between explaining quality and dislikes
($r=0.26$) loses its statistical significance, though remaining moderate (cf.
table 5).

Lastly, we newly introduced the number of interactions, i.e., the sum of likes
and dislikes for a given YouTube explanatory video, into the analysis (cf.
table 3). The number of interactions correlates statistically significantly
with the explaining quality of the explanatory videos on entanglement and
tunnelling: $r=0.39$, $p<0.01$. The partial correlation - when controlling for
the number of subscribers of the channels by which the videos are published -
of $r=0.43$, $p<0.01$ was even higher.

###  6.2 Discussion of research question 2

Compared to the metrics provided by YouTube alongside each video (e.g., the
number of views), the number of relevant comments turned out to be most
strongly correlated with the explaining quality of explanatory videos
($r=0.46$, $p<0.01$ for the total sample) on (a) quantum entanglement
($r=0.59$, $p<0.01$), and (b) quantum tunnelling ($r=0.31$, $p<0.1$).
Similarly, [[]p. 10]Kulgemeyer2016neu report a correlation of $r=0.38$
($p<0.01$) between explaining quality and the number of relevant comments for
videos on Newton’s third law and Kepler’s laws, respectively.

We controlled the correlations between the videos’ explaining quality and the
number of relevant comments for the videos’ time online (in days). As a
result, the partial correlation between explaining quality and number of
relevant comments for the total sample increased ($r=0.55$, $p<0.001$). This
result is comparable to the one reported for the mechanics context, where a
partial correlation coefficient of $p=0.40$, $p<0.01$ was found [21].

The medium to high correlation between the explanatory videos’ explaining
quality and the number of relevant comments might be justified via the users’
cognitive activation: “Hence, videos that accumulate plenty of those relevant
comments are more successful in catching viewers’ attention as these videos
might use either a more stimulating explanation or the explanation delivered
is considered as a starting point for further learning progress” [[]p.
12]Kulgemeyer2016neu.

##  7 Conclusion

Our results support the findings presented earlier for YouTube explanatory
videos on mechanics [[, cf.]]Kulgemeyer2016neu, according to which

  * •

there is a statistically significant correlation between explaining quality
and the number of content-related comments ($r=0.46$, $p<0.001$ in our study,
cf. table 2), and

  * •

YouTube’s surface metrics (e.g., likes) might not be fruitful indicators for
the explaining quality of explanatory videos (cf. table 5).

However, focusing on YouTube explanatory videos addressing quantum
entanglement and tunnelling, our study contributes to extending previous
results presented by [21] in two respects:

  1. 1.

We find a statistically significant correlation between the number of likes
and the explaining quality of explanatory videos on the quantum topics
entanglement and tunnelling ($r=0.37$, $p<0.01$, cf. table 2). Although such a
correlation has already been assumed in the previous study [[,
cf.]]Kulgemeyer2016neu, it could not be found at that time in the context of
explanatory videos on topics of classical mechanics.

  2. 2.

Our study hints that the number of interactions (e.g., the sum of likes and
dislikes) might be an indicator for videos of high explaining quality
($r=0.39$, $p<0.01$, cf. table 3). We argue that this result fits well to the
number of relevant comments being statistically significantly correlated with
the explaining quality of explanatory videos (cf. table 2).

###  7.1 Limitations

It is important to note that the results presented in this article should be
interpreted with caution for the following reasons:

  1. 1.

We could only include a small number of $N=60$ videos in our sample due to the
huge amount of data and the great effort required for data analysis (e.g.,
categorization of all comments underneath each video).

  2. 2.

Classical correlations, as presented in this article, allow for the
exploration of relationships between variables, but not for the identification
of causal connections.

  3. 3.

The data analysis is largely based on the metrics provided by YouTube, which
are not fully transparent to users (cf. [21]).

  4. 4.

In this study, we only analyzed explanatory videos on the topics quantum
entanglement and tunnelling, and hence, the correlations found are not
generalizable to different topics.

###  7.2 Outlook

Despite the above-mentioned limitations, our results may serve as a valuable
starting point for future research, in particular with respect to teaching and
learning quantum concepts: While in this study only scientifically sound
explanatory videos have been included for the analysis, the internet is
crowded with scientifically misleading or mystifying explanatory videos on
quantum concepts, such as quantum entanglement and quantum tunnelling. Since
YouTube’s surface features, however, are not likely to provide reliable
quality indicators, future educational research should (a) explore widespread
misconceptions in explanatory videos on quantum concepts, and (b) make further
efforts towards the derivation of evidence-based selection criteria that
support both students and teachers/lecturers in detecting high quality content
out of the dark noise.

###  7.3 Data availability:

The data presented in this study are available on request from the
corresponding author.

###  7.4 Disclosure Statement:

No potential conflict of interest was reported by the author(s).

###  7.5 Ethical statement:

The study met the ethics requirements of the University of Erlangen at the
time the data was collected.

## References

  * [1] A. Acín et al.  In _New J Phys._ 20, 2018, pp. 080201
  * [2] H. Bartels and C. Kulgemeyer  “Explaining physics: an online test for selfassessment and instructor training”  In _Eur. J. Phys._ 40, 2019, pp. 015701  DOI: https://doi.org/10.1088/1361-6404/aaeb5e
  * [3] Jacob Beautemps and André Bresges  “What Comprises a Successful Educational Science YouTube Video? A Five-Thousand User Survey on Viewing Behaviors and Self-Perceived Importance of Various Variables Controlled by Content Creators”  In _Fron. Comm._ 5, 2021, pp. 600595  DOI: https://doi.org/10.3389/fcomm.2020.600595
  * [4] P. Bitzenbauer  “Practitioners’ views on new teaching material for introducing quantum optics in secondary schools”  In _Phys. Educ._ 56.5, 2021, pp. 055008
  * [5] P. Bitzenbauer  “Quantum Physics Education Research over the Last Two Decades: A Bibliometric Analysis”  In _Educ. Sci._ 11, 2021, pp. 699  DOI: https://doi.org/10.3390/educsci11110699
  * [6] Cynthia J. Brame  “Effective Educational Videos: Principles and Guidelines for Maximizing Student Learning from Video Content”  In _CBE life sciences education_ 15.4, 2016, pp. es6  DOI: https://doi.org/10.1187/cbe.16-03-0125
  * [7] M.L. Chiofalo et al.  “Games for Teaching/Learning Quantum Mechanics: A Pilot Study with High-School Students”  In _Educ. Sci._ 12, 2022, pp. 446  DOI: https://doi.org/10.3390/educsci12070446
  * [8] J. Cohen  “Statistical Power Analysis for the Behavioral Sciences”  New York: Lawrence Erlbaum Associates, 1988
  * [9] E. Delen, J. Liew and V. Willson  “Effects of interactivity and instructional scaffolding on learning: Self-regulation in online video-based environments”  In _Comp. and Educ._ 78, 2014, pp. 312–320  DOI: http://dx.doi.org/10.1016/j.compedu.2014.06.018
  * [10] B. Dorland et al.  “Games and Learning Alliance. GALA 2019. Lecture Notes in Computer Science, vol 11899”  Springer, Cham, 2019, pp. 7–37
  * [11] S. Findeisen, S. Horn and J. Seifried  “Lernen durch Videos – Empirische Befunde zur Gestaltung von Erklärvideos”  In _MedienPädagogik_ , 2019, pp. 16–36  URL: https://doi.org/10.21240/mpaed/00/2019.10.01.X
  * [12] L. Fiorella, T. Gog, V. Hoogerheide and R.. Mayer  “It’s All a Matter of Perspective: Viewing First-Person Video Modeling Examples Promotes Learning of an Assembly Task”  In _J. of Educ. Psych._ 109, 2017, pp. 653–665  URL: https://doi.org/10.1037/edu0000161
  * [13] E. Galvez et al.  “Interference with correlated photons: Five quantum mechanics experiments for undergraduates”  In _Am. J. Phys._ 73.2 AAPT, 2005, pp. 127–140
  * [14] D. Geelan  “Teacher explanations”  In _Second International Handbook of Science Education_ Springer, 2012, pp. 987–999
  * [15] R.. Kay  “Developing a Framework for Creating Effective Instructional Video Podcasts”  In _Int. J. Emerg. Techn. in Learn._ 9.1, 2014, pp. 22–30  DOI: 10.3991/ijet.v9i1.3335
  * [16] B.. Kocyigit and M.. Akaltun  “Does YouTube provide high quality information? Assessment of secukinumab videos”  In _Rheum. Int._ 39, 2019, pp. 1263–1268  DOI: https://doi.org/10.1007/s00296-019-04322-8
  * [17] J.-L. Kruger and S. Doherty  “Measuring cognitive load in the presence of educational video: Towards a multimodal methodology”  In _Austr. J. of Educ. Techn._ 32, 2016, pp. 19–31  DOI: http://dx.doi.org/10.14742/ajet.3084
  * [18] C. Kulgemeyer  “Towards a framework for effective instructional explanations in science teaching”  In _Stud. Sci. Educ._ 54, 2019, pp. 103–139  DOI: https://doi.org/10.1080/03057267.2018.1598054
  * [19] C. Kulgemeyer and J. Wittwer  “Misconceptions in Physics Explainer Videos and the Illusion of Understanding: an Experimental Study”  In _Int. J. of Sci. and Math. Educ._ , 2022  DOI: https://doi.org/10.1007/s10763-022-10265-7
  * [20] Christoph Kulgemeyer  “A Framework of Effective Science Explanation Videos Informed by Criteria for Instructional Explanations”  In _Res. Sci. Educ._ 50.6, 2020, pp. 2441–2462  DOI: https://doi.org/10.1007/s11165-018-9787-7
  * [21] Christoph Kulgemeyer and Cord H Peters  “Exploring the explaining quality of physics online explanatory videos”  In _Eur. Phys. Educ._ 37.6, 2016, pp. 065705  DOI: 10.1088/0143-0807/37/6/065705
  * [22] Christoph Kulgemeyer and Horst Schecker  “Students Explaining Science—Assessment of Science Communication Competence”  In _Res. Sci. Educ._ 43.6, 2013, pp. 2235–2256  DOI: https://doi.org/10.1007/s11165-013-9354-1
  * [23] Christoph Kulgemeyer and Elisabeth Tomczyszyn  “Physik erklären – Messung der Erklärensfähigkeit angehender Physiklehrkräfte in einer simulierten Unterrichtssituation”  In _ZfDN_ 21.1, 2015, pp. 111–126  DOI: 10.1007/s40573-015-0029-5
  * [24] Gaea Leinhardt and Michael D. Steele  “Seeing the Complexity of Standing to the Side: Instructional Dialogues”  In _Cogn. and Instruc._ 23.1 Taylor & Francis, Ltd., 2005, pp. 87–163
  * [25] E. Marshman and C. Singh  “Validation and administration of a conceptual survey on the formalism and postulates of quantum mechanics”  In _Phys. Rev. Phys. Educ. Res._ 15 American Physical Society, 2019, pp. 020128  DOI: 10.1103/PhysRevPhysEducRes.15.020128
  * [26] R.. Mayer  “Multimedia aids to problem-solving transfer”  In _Int. J. of Educ. Res._ 31, 1999, pp. 611–623
  * [27] R.. Mayer  “Multimedia Learning”  Cambridge: Cambridge University Press, 2009  DOI: https://doi.org/10.1017/CBO9780511811678
  * [28] M.. Meyer  “Die Bedeutung von Online-Erklärvideos für Physiklernende”  Masterthesis, Universität Bremen, 2019  URL: https://tinyurl.com/yc3c3r2m
  * [29] D. Muller  “Designing effective multimedia for physics education”  School of Physics, University of Sydney, Australia, 2008
  * [30] N.. Noor, M. Aini and N.I. Hamizan  “Video Based Learning Embedded with Cognitive Load Theory: Visual, Auditory, and Kinaesthetic Learners’ Perspectives”  In _2014 International Conference on Teaching and Learning in Computing and Engineering_ , 2014, pp. 58–63
  * [31] S. Oss and T. Rosi  “A Bit of Quantum Mechanics”  In _Phys. Teach._ 53, 2015, pp. 230  DOI: https://doi.org/10.1119/1.4914565
  * [32] G. Passante, J. Emigh and P.. Shaffer  “Student ability to distinguish between superposition states and mixed states in quantum mechanics”  In _Phys. Rev. ST Phy. Educ. Res._ 11, 2015, pp. 020135
  * [33] D. Pattier  “Science on Youtube: Successful Edutubers”  In _Techno. Review. Int. Techn., Sci. and Soc. Rev._ 10, 2021, pp. 1–15  DOI: https://doi.org/10.37467/gka-revtechno.v10.2696
  * [34] B. Pearson and D. Jackson  “A hands-on introduction to single photons and quantum mechanics for undergraduates”  In _Am. J. Phys._ 78.5 AAPT, 2010, pp. 471–484
  * [35] B. Pekdag and J.-F. Le Marechal  “Movies in chemistry education”  In _Asia-Pacific Forum on Science Learning and Teaching_ 11.1, 2010, pp. 15
  * [36] N.L. Schroeder and A.L. Traxler  “Humanizing Instructional Videos in Physics: When Less Is More”  In _J. Sci. Educ. Technol._ 26.1, 2017, pp. 269–278  URL: https://doi.org/10.1007/s10956-016-9677-6
  * [37] Z.C. Seskir et al.  “Quantum Games and Interactive Tools for Quantum Technologies Outreach and Education”  In _arXiv:2202.07756v2_ , 2022  DOI: https://doi.org/10.48550/arXiv.2202.07756
  * [38] C. Singh  “Student understanding of quantum mechanics”  In _Am. J. Phys._ 69, 2001, pp. 885–895
  * [39] N. Suprapto, W. Nandyansah and H. Mubarok  “An Evaluation of the “PicsAR” Research Project: An Augmented Reality in Physics Learning”  In _iJET_ 15, 2020, pp. 113–125  DOI: https://doi.org/10.3991/ijet.v15i10.12703
  * [40] J. Sweller, J… Merrienboer and F. Paas  “Cognitive Architecture and Instructional Design: 20 Years Later”  In _Educ. Psych. Rev._ 31, 2019, pp. 261–292
  * [41] John Sweller  “Cognitive Load During Problem Solving: Effects on Learning”  In _Cogn. Sci._ 12.2, 1988, pp. 257–285  DOI: https://doi.org/10.1207/s15516709cog1202“˙4
  * [42] John Sweller  “Cognitive load theory, learning difficulty, and instructional design”  In _Learn. and Instruc._ 4.4, 1994, pp. 295–312  DOI: https://doi.org/10.1016/0959-4752(94)90003-5
  * [43] John Sweller, Jeroen J.. Merrienboer and Fred G… Paas  “Cognitive Architecture and Instructional Design”  In _Educ. Psych. Rev._ 10.3, 1998, pp. 251–296  DOI: https://doi.org/10.1023/A:1022193728205
  * [44] D. Treagust and A. Harrison  “The genesis of effective science explanations for the classroom”  In _Researching teaching: methodologies and practices for understanding pedagogy_ Abingdon: Routledge, 1999, pp. 28–43
  * [45] M. Ubben and P. Bitzenbauer  “Two Cognitive Dimensions of Students’ Mental Models in Science: Fidelity of Gestalt and Functional Fidelity”  In _Educ. Sci._ 12, 2022, pp. 163  DOI: https://doi.org/10.3390/educsci12030163
  * [46] Dustin J. Welbourne and Will J. Grant  “Science communication on YouTube: Factors that affect channel and video popularity”  In _Public Understanding of Science_ 25.6, 2016, pp. 706–718  DOI: 10.1177/0963662515572068
  * [47] J. Wittwer and A. Renkl  “Why Instructional Explanations Often Do Not Work: A Framework for Understanding the Effectiveness of Instructional Explanations”  In _Educ. Psych._ 43.1, 2008, pp. 49–64  DOI: https://doi.org/10.1080/00461520701756420
  * [48] Karsten D. Wolf and Verena Kratzer  “Erklärstrukturen in selbsterstellten Erklärvideos von Kindern”  In _Jahrbuch Medienpädagogik 12: Kinder und Kindheit in der digitalen Kultur_ Wiesbaden: Springer Fachmedien Wiesbaden, 2015, pp. 29–44  DOI: 10.1007/978-3-658-09809-4˙3
  * [49] YouTube Analytics and Reporting APIs  “Metrics”, 2022  URL: https://developers.google.com/youtube/analytics/metrics
  * [50] G. Zhu and C. Singh  “Improving students’ understanding of quantum measurement. I. Investigation of difficulties”  In _Phys. Rev. ST Phys. Educ. Res._ 8 American Physical Society, 2012, pp. 010117  DOI: 10.1103/PhysRevSTPER.8.010117
  * [51] G. Zhu and C. Singh  “Surveying students’ understanding of quantum mechanics in one spatial dimension”  In _Am. J. Phys._ 80.3, 2012, pp. 252–259  DOI: 10.1119/1.3677653

## Appendix

Table 7: The category system to evaluate explanatory videos’ explaining
quality [[]p. 7]Kulgemeyer2016neu. The sub-categories that are related to a
decrease of explaining quality, and hence, lead to a negative point ($=-1$ CP)
are marked with ’-’. The categories used in this context are also integrated
in the framework of effective explanation videos [[, cf.]p.
2449]Kulgemeyer2020Framework. For an in-depth description of all the
categories we refer the reader to [20].

Main category |  Subcategories
---|---
Content |  1\. Scientific mistake (-)
2\. Mistake corrected
Structure |  3\. Giving an outlook
4\. Giving a review
5\. Giving a summary
6\. Ignoring students’ comment (-)
7\. Emphazising important points
8\. Open justification of the explaining approach
9\. Addressing common misconceptions
Use of language |  10\. Paraphrasing technical terms
11\. Comment technical term with everyday language
12\. Comment technical term with other technical abc terms
13\. Leaving new technical term uncommented (-)
Contexts and examples |  14\. Addressing explainee
15\. Example close to everyday life
16\. Abstract example
17\. Without context (-)
18\. Connecting at least two examples by showing abc analogies
19\. Connecting example to explained topic by showing analogies
Mathematics |  20\. Providing numerical example for formula
21\. Using formula
22\. Describing relationships by use of ‘the more… the less/more’ relations
23\. Using mathematical terms and idealisations
Interrogation |  24\. Asking further questions
Non-verbal elements |  25\. Using realistic figures (such as photos)
26\. Using analogical figures
27\. Using logical figures (such as diagrams)
28\. Using experiments
29\. Connecting non-verbal elements
30\. Using writings
31\. Draw/amend figures
|


12pt 1.1 90 202mm 170mm darkgray Late-Breaking / Demo Session Extended
Abstract, ISMIR 2022 Conference

# MIDI-Draw: Sketching to Control Melody Generation

###### Abstract

We describe a proof-of-principle implementation of a system for drawing
melodies that abstracts away from a note-level input representation via
melodic contours. The aim is to allow users to express their musical
intentions without requiring prior knowledge of how notes fit together
melodiously. Current approaches to controllable melody generation often
require users to choose parameters that are static across a whole sequence,
via buttons or sliders. In contrast, our method allows users to quickly
specify how parameters should change over time by drawing a contour.

##  1 Introduction

Recent deep learning approaches offer more variety, musicality and longer
range structure than many previous approaches to music generation [1].
However, the high expressivity and black-box nature of these models makes it
hard to predict what outputs a model will produce given certain inputs. Some
understanding of the model is necessary for musicians to be able to
iteratively tweak the model to produce outputs that better match their
intentions, which is an important part of musical composition [2]. This is
referred to as _controllability_ or _directability_.

Many musical features have been targeted for control including low-level
musical features such as key, tempo or mode [3], mid-level perceptual features
such as tonal tension [4] or information-theoretic surprise [5], or high-level
features such as artist style and lyrics [6]. This is often achieved through
conditional models, which change the probability of output sequences given
some user-chosen feature values [3], or by manipulation of a latent space via
attribute vector arithmetic [7] or attribute regularization and
disentanglement [8]. Control over these features enables users to feel more
self-efficacy and ownership over the generated music [9].

Controls for generative models are most often presented as switches or 1D
sliders [9]. 2D sliders are also common, often displayed over an image such as
a surface plot [10]. Other control methods include tracking body movements in
space, known as sound tracing [11] and conditioning on text using contrastive
language-image pre-training (CLIP) on videos with associated music [12].
Several approaches exist that use drawing such as Oramics [13], Painting Music
[14], Hyperscore [15] and JamSketch [16]. Our proposal, MIDI-Draw, adopts this
approach because drawing is a quick, easy and natural movement that does not
need specialist equipment. A quick sketch is useful as it represents a semi-
formed idea, which fixes some attributes while allowing freedom for others to
change more easily. This allows for an intermediate representation that is
intuitive to humans but that machine learning approaches are also sensitive
to.

We demonstrate the potential of the approach on melodic contours, which
capture how pitch goes up and down over time, while disregarding quantized
note and scale information. Geometric approaches appear to be the most
promising approach for symbolic melody indexing [17] and similarity [18]. Such
methods include rectilinear functions [19], cubic splines [20], Fourier
components [16], or cosine components [21]. Many of these approaches relate
pitches directly to the curve height. We adopt cosine components here as they
can extract a slow-moving average trend, which allows for more freedom and
variety in generation. For example, notes may lie directly on the curve, or
scattered around the curve while still fitting the trend overall (Fig. 1).

Figure 1: Musical phrases can be generated with the same overall trend while
allowing for variation in tightness of fit.

##  2 Methods

Figure 2: Left: Training consists of extracting cosine components from
melodies and recreating melodies conditioned on those components. Right:
Inference consists of generating melodies by sampling from a prior
distribution conditioned on cosine components extracted from a drawn curve.

To train MIDI-Draw (Fig. 2, left), we use a synthetic dataset generated by
specifying a pitch-transition matrix with uniform pitch priors. 5000 sequences
of 16 notes are sampled from this matrix. These pitch sequences are converted
into cosine component amplitudes using the orthonormal Discrete Cosine
Transform (DCT-II). The average pitch of each melody (first component) and the
higher frequency components are set to zero such that only the second to
fourth cosine components are stored. This results in a low frequency (slow
changing) pitch trend when passed back through the inverse (DCT-III)
transform.

In order to generate music from a drawn shape rather than by continuing some
musical context, we use a conditional variational autoencoder (CVAE). A VAE
has two training objectives; to reconstruct the input and to maintain the
latent space distribution close to a chosen prior distribution. The latter
allows for sampling directly in the latent space during inference, without
needing to encode some previous context. The CVAE is trained to recreate
melodies conditioned on the cosine components by concatenating them to the
start of the input and latent vectors. Bi-directional long-short term memory
(Bi-LSTM) models are used for the encoder and decoder, which allow for long
sequences and reuse of past context due to their hidden cell state. For the
decoder, an autoregressive conductor model is used as in [7].

During inference (Fig. 2, right), a drawn curve is passed through the DCT-II
transform as above, which is repeatedly concatenated to a batch of latent
vectors generated by randomly sampling from the chosen prior, a multivariate
Gaussian distribution $\epsilon\sim\mathcal{N}(0,I)$. This is then passed
through the decoder to generate a batch of melody candidates. The DCT
transform is again performed on the resulting melody candidates and mean-
squared error is used to pick the melody candidate with components closest to
the drawn curve components.

##  3 Results and Discussion

Figure 3: Web app1.

A web app111Web app available at www.tashinamgyal.com/midi-draw (Fig. 3)
provides a canvas interface for users to draw contours and a piano roll
interface to visualize, play back and download generated melodies. Preliminary
user tests showed that non-musicians often initially drew curves that
oscillated very rapidly, even though most melodies do not do this, suggesting
additional constraints or guidance would be useful. A graph was added to show
how changing the number of cosine components used would change how closely the
generated melody fits their sketch, but further work is needed to train a
model with an adjustable number of conditional components. Users also
suggested that a constant rhythm made generated melodies sound robotic.
Variation in rhythm could be added by conditioning on rhythm onsets[22],
rhythm circles [23] or by tracing deviation from a regular beat. Lastly, we
plan to work on evaluating the quality of musical sequences and how well
sequences match user intentions.

##  4 Conclusion

We described a proof-of-principle implementation of music generation from
drawing. This frees novice musicians to comparatively, rather than absolutely,
express musical intentions, while a generative model does the hard part of
selecting notes that loosely yet melodiously follow the intended trend. We
demonstrate the potential of the approach with a web app that allows users to
draw pitch contours and generates matching melodies. This opens the way to
user-centred music generation that tracks changes in musical intention over
time more naturally than current alternatives.

##  5 Acknowledgments

Tashi Namgyal is supported by the UKRI Centre for Doctoral Training in
Interactive Artificial Intelligence (EP/S022937/1).

## References

  * [1] J.-P. Briot, G. Hadjeres, and F.-D. Pachet, Deep Learning Techniques for Music Generation. Cham, Switzerland: Springer International Publishing, 2020.
  * [2] J. Garcia, T. Tsandilas, C. Agon, and W. E. Mackay, “Structured Observation with Polyphony: a Multifaceted Tool for Studying Music Composition,” in Proc. Conf. Designing Interactive Systems, Vancouver, BC, Canada, 2014.
  * [3] N. Meade, N. Barreyre, S. C. Lowe, and S. Oore, “Exploring Conditioning for Generative Music Systems with Human-Interpretable Controls,” in Proc. Int. Conf. Computational Creativity, Charlotte, NC, USA, 2019.
  * [4] R. Guo, I. Simpson, T. Magnusson, C. Kiefer, and D. Herremans, “A Variational Autoencoder for Music Generation Controlled by Tonal Tension,” in Proc. Conf. AI Music Creativity, Stockholm, Sweden, 2020.
  * [5] Y.-W. Chen, H.-S. Lee, Y.-H. Chen, and H.-M. Wang, “SurpriseNet: Melody Harmonization Conditioning on User-controlled Surprise Contours,” in Proc. Int. Soc. Music Information Retrieval Conf., Online, 2021.
  * [6] P. Dhariwal, H. Jun, C. Payne, J. W. Kim, A. Radford, and I. Sutskever, “Jukebox: A Generative Model for Music,” 2020, arXiv:2005.00341.
  * [7] A. Roberts, J. Engel, C. Raffel, C. Hawthorne, and D. Eck, “A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music,” in Proc. Int. Conf. Machine Learning, Stockholm, Sweden, 2018.
  * [8] A. Pati, S. Gururani, and A. Lerch, “dMelodies: A Music Dataset for Disentanglement Learning,” in Proc. Int. Soc. Music Information Retrieval Conf., Montréal, Canada, 2020.
  * [9] R. Louie, A. Coenen, C. Z. Huang, M. Terry, and C. J. Cai, “Novice-AI Music Co-Creation via AI-Steering Tools for Deep Generative Models,” in Proc. Conf. Human Factors in Computing Systems, New York, NY, USA, 2020.
  * [10] N. Bryan-Kinns, B. Banar, C. Ford, C. N. Reed, Y. Zhang, S. Colton, and J. Armitage, “Exploring XAI for the Arts: Explaining Latent Space in Generative Music,” in Workshop on eXplainable AI Approaches for Debugging and Diagnosis (XAI4Debugging@NeurIPS2021), Online, 2021.
  * [11] R. I. Godøy and A. R. Jensenius, “Body Movement in Music Information Retrieval,” in Proc. Int. Soc. Music Information Retrieval Conf., Kobe, Japan, 2009.
  * [12] H.-H. Wu, P. Seetharaman, K. Kumar, and J. P. Bello, “Wav2CLIP: Learning Robust Audio Representations From CLIP,” in Proc. Int. Conf. Acoustics, Speech and Signal Processing, Singapore, Singapore, 2022.
  * [13] T. Richards, “Oramics: Precedents, Technology and Influence,” Ph.D. dissertation, Goldsmiths, Univ. London, London, UK, 2018.
  * [14] A. Starkey, K. Steenhauer, and J. Caven, “Painting Music: Using Artificial Intelligence to Create Music from Live Painted Drawings,” Drawing: Research, Theory, Practice, vol. 5, no. 2, pp. 209–224, Dec. 2020.
  * [15] M. M. Farbood, E. Pasztor, and K. Jennings, “Hyperscore: a Graphical Sketchpad for Novice Composers,” IEEE Computer Graphics and Applications, vol. 24, no. 1, pp. 50–54, Jan. 2004.
  * [16] T. Kitahara, S. Giraldo, and R. Ramírez, “JamSketch: Improvisation Support System with GA-Based Melody Creation from User’s Drawing,” in Music Technology with Swing, M. Aramaki, M. E. P. Davies, R. Kronland-Martinet, and S. Ystad, Eds. Cham, Switzerland: Springer International Publishing, 2018, pp. 509–521.
  * [17] D. Parsons, The Directory Of Classical Themes. London, UK: Piatkus, 2008.
  * [18] V. Velardo, M. Vallati, and S. Jan, “Symbolic Melodic Similarity: State of the Art and Future Challenges,” Computer Music Journal, vol. 40, no. 2, pp. 70–83, Jun. 2016.
  * [19] G. Aloupis, T. Fevens, S. Langerman, T. Matsui, A. Mesa, Y. Nuñez, D. Rappaport, and G. Toussaint, “Algorithms for Computing Geometric Measures of Melodic Similarity,” Computer Music Journal, vol. 30, no. 3, pp. 67–76, 2006.
  * [20] J. Urbano, “MIREX 2013 Symbolic Melodic Similarity: A Geometric Model supported with Hybrid Sequence Alignment,” Tech. Rep., Music Information Retrieval Evaluation eXchange, 2013.
  * [21] B. Cornelissen, W. Zuidema, and J. A. Burgoyne, “Cosine Contours: A Multipurpose Representation for Melodies,” in Proc. Int. Soc. Music Information Retrieval Conf., Online, 2021.
  * [22] K. Chen, C.-I. Wang, T. Berg-Kirkpatrick, and S. Dubnov, “Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm,” in Proc. Int. Soc. Music Information Retrieval Conf., Montréal, Canada, 2020.
  * [23] E. Hein and S. Srinivasan, “The Groove Pizza,” in New Directions in Music and Human-Computer Interaction, S. Holland, T. Mudd, K. Wilkie-McKenna, A. McPherson, and M. M. Wanderley, Eds. Cham, Switzerland: Springer International Publishing, 2019, pp. 71–94.