:
1 2024 AI4Ed-AAAI

# Improving Assessment of Tutoring Practices using Retrieval-Augmented
Generation

Zifei (FeiFei) Han<EMAIL_ADDRESS>
Jionghao Lin<EMAIL_ADDRESS>
Ashish Gurung<EMAIL_ADDRESS>
Danielle R. Thomas<EMAIL_ADDRESS>
Eason Chen<EMAIL_ADDRESS>
Conrad Borchers<EMAIL_ADDRESS>
Shivang Gupta<EMAIL_ADDRESS>
Kenneth R. Koedinger<EMAIL_ADDRESS>
Human-Computer Interaction Institute
Carnegie Mellon University
5000 Forbes Ave.
Pittsburgh  PA 15213  USA

###### Abstract

One-on-one tutoring is an effective instructional method for enhancing
learning, yet its efficacy hinges on tutor competencies. Novice math tutors
often prioritize content-specific guidance, neglecting aspects such as social-
emotional learning. Social-emotional learning promotes equity and inclusion
and nurturing relationships with students, which is crucial for holistic
student development. Assessing the competencies of tutors accurately and
efficiently can drive the development of tailored tutor training programs.
However, evaluating novice tutor ability during real-time tutoring remains
challenging as it typically requires experts-in-the-loop. To address this
challenge, this preliminary study aims to harness Generative Pre-trained
Transformers (GPT), such as GPT-3.5 and GPT-4 models, to automatically assess
tutors’ ability of using social-emotional tutoring strategies. Moreover, this
study also reports on the financial dimensions and considerations of employing
these models in real-time and at scale for automated assessment. The current
study examined four prompting strategies: two basic Zero-shot prompt
strategies, Tree of Thought prompt, and Retrieval-Augmented Generator (RAG)
based prompt. The results indicate that the RAG prompt demonstrated more
accurate performance (assessed by the level of hallucination and correctness
in the generated assessment texts) and lower financial costs than the other
strategies evaluated. These findings inform the development of personalized
tutor training interventions to enhance the the educational effectiveness of
tutored learning.

###### keywords:

Large Language Model, Personalized Tutor Training, Automatic Assessment

##  1 Introduction

The efficacy of one-on-one tutoring for knowledge acquisition and retention
continues to gather strong empirical evidence (Kraft and Falken, 2021; Nickow
et al., 2020). For tutoring to reach its optimal effectiveness, instructors
need a multifaceted skill set, enabling them to provide both content-specific
and social-emotional support to students (Thomas et al., 2023; Lin et al.,
2023c, 2022b, a, 2022a). However, it is often challenging for novice math
tutors to blend both content-specific and social-emotional supports
effectively into their teaching. They tend to focus primarily on content-
specific instructional guidance, which can result in the inadvertent neglect
of social-emotional learning components in their teaching methods (Thomas et
al., 2022). Social-emotional learning includes indispensable facets such as
self-awareness, empathy, adeptness in building relationships, and critical
decision-making abilities—crucial constituents for fostering the holistic
development of students (Jelfs et al., 2009). Prior work shows that neglecting
social-emotional learning during tutoring relates to significant loss in
tutoring effectiveness (Marshall et al., 2021). Thus, assessing social-
emotional learning support in tutors can guide personalized tutor training and
subsequently improve tutoring effectiveness. However, this assessment has been
hitherto expensive and infeasible for many educational applications, as it
usually requires human experts, which are scarce (Kraft and Falken, 2021).

In light of recent promising applications of large language models (LLMs) in
education (Wang and Demszky, 2023; Dai et al., 2023; Lin et al., 2023b), this
preliminary analysis aims to harness the potential of widely-used Generative
Pre-trained Transformers (GPT), such as GPT-3.5 and GPT-4 to automatically
assess social-emotional competencies in human tutors (OpenAI, 2023; Lehman et
al., 2022). We explore four types of prompting strategies: two basic Zero-shot
prompts, the Tree of Thought prompt (Yao et al., 2023), and a Retrieval-
Augmented Generator (RAG)-based prompt (Lewis et al., 2020). Additionally,
given the significance of educational applications at scale, both in research
and industry, it is crucial to understand the cost of using GPT models which
is vital for assessing the economic feasibility of utilizing the model on our
task. Consequently, this research investigates two Research Questions: RQ1:
Can GPT models accurately assess the social-emotional learning competencies of
human tutors? RQ2: How does the performance and cost analysis of GPT-3.5
compare to that of GPT-4 in this context?

##  2 Method

###  2.1 Data

The dataset (tutoring dialogue transcripts) was collected from real-world
middle-school math tutoring sessions in the United States. These sessions were
conducted on Zoom, with novice human tutors to teach math ranging from Grade 6
to Grade 8. It should be noted that prior to the tutoring sessions, these
human tutors completed some lessons involved social-emotional learning from a
tutor training platform as described in Lin et al. (2023c). Each session,
lasting approximately 30 minutes, involved dividing a group of students into
multiple breakout rooms, assigning each student to an individual room. Tutors
were responsible for managing three to five breakout rooms, conducting one-on-
one tutoring as they circulated among them. When tutors noticed that certain
students were capable of working independently with little assistance, they
would move on to aid another student in a different room. The entire tutoring
process was recorded via Zoom, and the recordings were transcribed using the
speech-recognition tool Whisper (Radford et al., 2022). In this work-in-
progress study, we randomly selected five tutoring transcripts where a math
tutor engages in tutoring sessions with five students, guiding them through
the process of solving basic arithmetic problems.

###  2.2 Assessing Tutoring Practices

The tutoring dialogue transcripts were evaluated based on research-based
principles in social-emotional learning and relationship building proposed by
Chhabra et al. (2022). The details of these principles are presented in
Appendix B; broadly we considered 5 categories: (1) Giving Effective Praise,
(2) Supporting a Growth Mindset, (3) Reacting to Errors, (4) Responding to
Negative Self-Talk, and (5) Using Motivational Strategies. We used principles
from these five most frequently used tutoring strategies to develop a rubric
for assessing the tutoring practice.111The details of five lessons can be
found via https://www.tutors.plus/solution/training For brevity, we take the
principle of Giving Effective Praise as an example. This principle suggests
that during tutoring, the instructor should focus on the praise on student
learning effort instead of outcome. A desired praise is “You are almost there!
I am proud of how you are persevering through and striving to solve the
problem. Keep going!” while an undesired praise is “You are so smart and
almost got the problem correct.” The principles and rubric from the five
lessons further inform the design of prompt strategies to evaluate tutor’s use
of the practice.

###  2.3 Large Language Model Generated Evaluation and Feedback

To answer RQ1, we designed four types of prompt strategies. The rationale of
prompt design follows two main elements where the prompt can foster GPT model
to 1) generate score of the evaluation and 2) provide evidence from dialogue
as the detailed interpretation of the score. By doing so, we designed four
type of prompting strategies (see Appendix A) to analyze the the ability of
GPT models on evaluating tutor performance, and these prompting strategies are
detailed below:

Basic Zero-shot Prompt Type I. This zero-shot prompt was designed by using
five effective tutoring principles of social-emotional learning and
relationship building (detailed in the Appendix A). We prompt the GPT models
to assess the whole tutoring transcript based on the tutoring principles in
terms of scores and interpretation of score.

Basic Zero-shot Prompt Type II. This zero-shot prompt (detailed in the
Appendix A) was used to firstly identify the incorrect use of tutoring
practice from the transcript, and then assess the scores and provide
interpretation of the scores.

Tree of Thoughts (ToT) Prompt. As reported by Yao et al. (2023), the Tree of
Thought prompt allows information to be organized in a structured way, like
branches of a tree. This structured format helps the language model understand
the information more comprehensively compared to simple and linear prompts. As
a result, when using the Tree of Thought prompt, the language model might
generate more precise and detailed responses. Our proposed ToT prompt
(detailed in the Appendix A) is shown in fig:imagetot. The GPT models take the
tutoring transcript as input. Then, the GPT models evaluate the tutor social-
emotional learning based on the rubrics from five principles (e.g., Using
Motivation Strategies in fig:imagetot). As a result, the ToT-driven GPT models
generate scores and interpretation of the scores on assessing the tutor
social-emotional learning ability.

fig:imagetot

Figure 1: Tree of Thought Evaluation Framework

Retrieval-Augmented Generation (RAG). As reported by Lewis et al. (2020), RAG
proves more effective than zero-shot prompts due to its integration of
external knowledge sources (e.g., transcriptions and principles of tutoring),
enriching the understanding of the GPT model and enabling it to generate more
contextually relevant and accurate responses. Our study developed an RAG-based
prompt (detailed in the Appendix A) as depicted in fig:imageragengine. Within
the information database component (see fig:imageragengine), tutoring
transcriptions and principles on social-emotional learning are initially
converted into word embeddings (i.e., words represented as vectors for
semantic relationships, as described in Kusner et al. (2015)) stored within
the database. These embeddings, stored within information database, form the
basis for our RAG-based prompt, allowing the GPT model to access a broader and
more relevant set of information. In the actual evaluation process, the RAG
model’s evaluation engine (see fig:imageragengine) can selectively retrieve
and incorporate information from our prepared word embeddings, guided by the
principles of social-emotional learning. This selective retrieval ensures that
the GPT model focuses on the most relevant aspects of the tutoring
transcripts, ultimately enabling the RAG-based prompt to generate scores and
provide evidence for these scores, illustrating the potential of RAG in
enhancing GPT models.

fig:imageragengine

Figure 2: The structure of Retrieval-Augmented Generation (RAG) based prompt

###  2.4 Evaluation Metrics

In this work-in-progress paper, annotation was conducted by a single human
coder since we aim to establish a preliminary understanding of evaluating the
accuracy of GPT models’ output, which can be later expanded and refined in
subsequent phases of research. While the use of one coder suits our current
exploratory needs, we recognize its limitation in inter-rater reliability. To
address this, we plan to involve multiple coders in future stages of the
research, which will enable us to conduct a thorough inter-rater reliability
analysis and ensure the robustness of our findings. The human coder annotated
the GPT generated output based on two metrics: 1) Correctness and 2)
Hallucination, which are detailed below:

  * •

Correctness is the metric to evaluate the capability of GPT models in
accurately assessing the tutor’s use of social-emotional learning principles
within the tutoring transcript. Evaluating correctness is essential for
verifying the model’s understanding and interpretation of the given
information. Human coders review the GPT-generated assessment results, which
include feedback and scores, against the actual tutoring transcript. The
scoring categories for this metric include “-1” indicating that no information
was generated for a specific social-emotional learning principle, “0” for GPT
model incorrect assessment, and “1” for GPT model correct assessment. The
correctness is guided by a rubric based on the five principles described in
Section. 2.2.

  * •

Hallucination denotes the phenomenon where a generative model (e.g., GPT
models) produces content that deviates from factual accuracy or logical
coherence with respect to the input prompt or source content Ji et al. (2023).
Hallucinations may manifest as fabricated facts, illogical statements, or
irrelevant responses that do not align with the established context or
contradict the known data Ji et al. (2023). In our study, we noted instances
where the GPT model generated feedback that was unrelated to events in the
tutoring transcript, exemplifying the issue of hallucination. This issue poses
significant challenges for applications that rely on the veracity of generated
text, necessitating rigorous validation mechanisms to ensure the reliability
of model outputs in critical domains. To measure the hallucination of
generated text, the human coder annotated the output into “-1”, “0”, “0.5”,
and “1” where “-1” indicates that no information was generated for a specific
social-emotional learning principle; “0” indicates no hallucination; “0.5”
signifies partial hallucination (both hallucinated and non-hallucinated
information coexist in the text), and “1” represents a completely hallucinated
response.

Evaluation on Financial Cost. To answer RQ2, we recorded the cost for using
different GPT models with different prompts. We counted the input token from
prompt and transcript whereas the output tokens from GPT generated text. We
called the API of GPT-3.5 Turbo and GPT-4 Turbo. By referring the GPT API
price,222https://openai.com/pricing we calculated the cost associated with
each generated text from GPT models.

##  3 Results

fig:image

Figure 3: Analysis of GPT models’ accuracy across various prompt strategies.
Zero-shot P1 and P2 denote the basic Zero-shot prompt type I and type II,
respectively

Results on RQ1. We present the results in Figure 3, which illustrate the
accuracy of GPT models on evaluating the tutoring transcript across various
prompt strategies. The accuracy of GPT models was measured by hallucination
and correctness metrics (detailed in Method 2.4), which aimed to assess GPT
models’ ability to evaluate a tutor’s competency in social-emotional learning.
Notably, the green area in Figure 3 represents instances of no hallucination
and correct evaluation, which is the desired evaluation on the tutoring
transcript. The left side of Figure 3 showed the accuracy of GPT-3.5 model.
This demonstrates that the RAG strategy has the potential in providing no
hallucination and correct evaluation of the transcripts. In comparison, other
prompting strategies from GPT-3.5 failed to provide desired evaluation on the
tutoring transcript. It should be noted that the results from GPT-3.5 include
an evaluation stating, “The tutor did not respond to negative self-talk by
validating the student’s feelings or building their self-efficacy.” On closer
examination of the tutoring transcript, we observed that the student did not
engage in negative self-talk, hence the tutor’s lack of response. However, the
GPT-3.5 evaluation implied that the tutor intentionally did not respond to
negative self-talk. Thus, our human annotator identified as a hallucination.
In contrast, the GPT-4 model results (on the right in Figure 3) show a
generally more accurate evaluation (evidenced by more frequent green area)
compared to the GPT-3.5 model. This suggests the advanced capability of GPT-4,
potentially outperforming the GPT-3.5 model. It is important to note that both
the RAG and Zero-shot (P1) prompts in GPT-4 yielded more accurate evaluations
than other prompting strategies. The comparative analysis of GPT-3.5 and GPT-4
underscores that the RAG-based prompting strategy consistently produces the
desired output, highlighting its effectiveness across different model
versions.

Results on RQ2. In our subsequent analysis, we evaluated the financial cost
per lesson evaluation for both GPT-3.5 Turbo and GPT-4 Turbo using various
prompting strategies, as outlined in Table 1. This analysis indicated that the
financial cost associated with GPT-4 is approximately 10 times greater than
that of GPT-3.5. The Retrieval-Augmented Generation (RAG)-based prompting
strategy was identified as the most cost-effective for both the GPT-3.5 and
GPT-4 models. As discussed earlier in the Results section on Research Question
1 (RQ1), the RAG-based approach has demonstrated its capability to provide
accurate evaluations without hallucinations in tutoring transcripts for both
models. Consequently, we propose the RAG-based prompt as the most cost-
efficient strategy. These findings provide insights into the selection of
different prompting strategies and GPT models, particularly in terms of
balancing effectiveness with financial constraints.

Table 1: Comparison of costs across various prompting strategies in GPT-3.5 and GPT-4 Prompt |  GPT-3.5 Turbo |  GPT-4 Turbo
---|---|---
Zero-shot Prompt Type I |  $0.100 |  $1.035
Zero-shot Prompt Type II |  $0.014 |  $0.188
Tree of Thoughts (ToT) |  $0.013 |  $0.137
Retrieval-Augmented Generation (RAG) |  $0.008 |  $0.137

##  4 Conclusion

This preliminary study highlights the potential of Retrieval-Augmented
Generation (RAG) prompting in evaluating the quality of tutoring based on
social-emotional learning competencies. By integrating tutoring transcripts
and principles into the GPT model via word embeddings, as elaborated in Table
5 in our appendix, RAG enables more contextually relevant and precise
evaluations. The RAG-based prompt not only showcased more accurate performance
in evaluating tutoring practices and lower financial costs compared to other
prompts but also laid the groundwork for broader applications in tutor skill
assessment. Moving forward, we aim to further explore the RAG prompt’s
effectiveness in assessing additional tutoring competencies, such as building
content skills and promoting inclusion. Additionally, we plan to employ the
RAG prompt to evaluate a broader range of real-world tutoring transcripts,
assessing its efficacy across diverse tutoring interactions. Our goal is to
identify areas where tutors may lack skills and provide corresponding training
lessons to help them improve. As an extension of this study, we intend to
design a training lesson recommender system that can process the assessment
from the GPT model and offer lesson recommendations to assist tutors in
enhancing their tutoring skills. A prototype developed for the demonstration
of the lesson recommendation system is accessible at
https://tutorevaluation.vercel.app.

## References

  * Chhabra et al. (2022) Pallavi Chhabra, Danielle Chine, Adetunji Adeniran, Shivang Gupta, and Kenneth Koedinger.  An evaluation of perceptions regarding mentor competencies for technology-based personalized learning.  In _Society for Information Technology & Teacher Education International Conference_, pages 1620–1625. Association for the Advancement of Computing in Education (AACE), 2022.
  * Dai et al. (2023) Wei Dai, Jionghao Lin, Hua Jin, Tongguang Li, Yi-Shan Tsai, Dragan Gašević, and Guanliang Chen.  Can large language models provide feedback to students? a case study on chatgpt.  In _2023 IEEE International Conference on Advanced Learning Technologies (ICALT)_ , pages 323–325. IEEE, 2023.
  * Jelfs et al. (2009) Anne Jelfs, John Richardson, and Linda Price.  Student and tutor perceptions of effective tutoring in distance education.  _Distance Education_ , 30, 11 2009.  10.1080/01587910903236551.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12):1–38, 2023.
  * Kraft and Falken (2021) Matthew A Kraft and Grace T Falken.  A blueprint for scaling tutoring and mentoring across public schools.  _AERA Open_ , 7:23328584211042858, 2021.
  * Kusner et al. (2015) Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger.  From word embeddings to document distances.  In _International conference on machine learning_ , pages 957–966. PMLR, 2015.
  * Lehman et al. (2022) Joel Lehman, Jonathan Gordon, Shawn Jain, Kamal Ndousse, Cathy Yeh, and Kenneth O. Stanley.  Evolution through large models, 2022.
  * Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  _Advances in Neural Information Processing Systems_ , 33:9459–9474, 2020.
  * Lin et al. (2022a) Jionghao Lin, Mladen Rakovic, David Lang, Dragan Gasevic, and Guanliang Chen.  Exploring the politeness of instructional strategies from human-human online tutoring dialogues.  In _LAK22: 12th International Learning Analytics and Knowledge Conference_ , pages 282–293, 2022a.
  * Lin et al. (2022b) Jionghao Lin, Shaveen Singh, Lele Sha, Wei Tan, David Lang, Dragan Gašević, and Guanliang Chen.  Is it a good move? mining effective tutoring strategies from human–human tutorial dialogues.  _Future Generation Computer Systems_ , 127:194–207, 2022b.
  * Lin et al. (2023a) Jionghao Lin, Mladen Raković, Yuheng Li, Haoran Xie, David Lang, Dragan Gašević, and Guanliang Chen.  On the role of politeness in online human–human tutoring.  _British Journal of Educational Technology_ , 2023a.
  * Lin et al. (2023b) Jionghao Lin, Danielle R Thomas, Feifei Han, Shivang Gupta, Wei Tan, Ngoc Dang Nguyen, and Kenneth R Koedinger.  Using large language models to provide explanatory feedback to human tutors.  2023b.
  * Lin et al. (2023c) Jionghao Lin, Danielle R Thomas, Zifei Han, Wei Tan, Ngoc Dang Nguyen, Shivang Gupta, Erin Gatz, Cindy Tipper, and Kenneth R Koedinger.  Personalized learning squared (plus): Doubling math learning through ai-assisted tutoring.  2023c.
  * Marshall et al. (2021) Lydia Marshall, Jonah Bury, Robert Wishart, Rebekka Hammelsbeck, and Emily Roberts.  The national online tuition pilot.  _Education Endowment Foundation. https://educationendowmentfoundation. org. uk/projects-and-evaluation/projects/online-tuition-pilot_ , 2021.
  * Nickow et al. (2020) Andre Nickow, Philip Oreopoulos, and Vincent Quan.  The impressive effects of tutoring on prek-12 learning: A systematic review and meta-analysis of the experimental evidence. working paper 27476.  _National Bureau of Economic Research_ , 2020.
  * OpenAI (2023) OpenAI.  Gpt-4 technical report, 2023.
  * Radford et al. (2022) Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever.  Robust speech recognition via large-scale weak supervision, 2022.
  * Thomas et al. (2022) Danielle Thomas, Cassandra Brentley, Carmen Thomas-Browne, J. Richey, Abdulmenaf Gul, Paulo Carvalho, Lee Branstetter, and Kenneth Koedinger.  _Educational Equity Through Combined Human-AI Personalization: A Propensity Matching Evaluation_ , pages 366–377.  01 2022.  ISBN 978-3-031-11643-8.  10.1007/978-3-031-11644-5_30.
  * Thomas et al. (2023) Danielle Thomas, Xinyu Yang, Shivang Gupta, Adetunji Adeniran, Elizabeth Mclaughlin, and Kenneth Koedinger.  When the tutor becomes the student: Design and evaluation of efficient scenario-based lessons for tutors.  In _LAK23: 13th International Learning Analytics and Knowledge Conference_ , pages 250–261, 2023.
  * Wang and Demszky (2023) Rose E. Wang and Dorottya Demszky.  Is chatgpt a good teacher coach? measuring zero-shot performance for scoring and providing actionable insights on classroom instruction, 2023.
  * Yao et al. (2023) Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan.  Tree of thoughts: Deliberate problem solving with large language models.  _arXiv preprint arXiv:2305.10601_ , 2023.

##  Appendix A Prompting Strategies

Table 2: Basic Zero-shot Prompt Type I |  Prompt
---|---
Scoring |  Given a dialogue of a tutoring session, please evaluate the Tutor based on specific best teaching practice within {Principle_Name} Please return 1 if the tutor correctly used the tutoring practice. Return 0 if the tutor incorrectly used the tutoring practice. Please only return 0 or 1
Generator |  Please briefly explain why you give the score?
Table 3: Basic Zero-shot Prompt Type II |  Prompt
---|---
Incorrect Identification |  Given the following evaluation criteria {Principle_Criteria} Please identify if there is any tutor’s incorrect use of the tutoring strategy {Principle_Name}based on the criteria above.If there is incorrect response, return the incorrect response by tutor as evidence in the dialogue and list the criteria not met. If the tutor used teaching strategy correctly, please return based on the criteria above, which ones are correct and also return evidence from the dialogue.
Score Generation |  Return the score of {Principle_Name} from 0 to 5 based on the evaluation. Give one point to each criteria met.
Table 4: Tree of Thought Prompt |  Prompt
---|---
Layer_1 |  {Social_Emotional_Learning_Principles} For the following transcript between a tutor and a middle school student, score how well the tutor performed in the competency area above. Give one point for each of the following criteria or skills being met by the tutor. For example, if a tutor did not demonstrate any evidence of a given skill or criteria give a score of 0. If a tutor met all the given criteria, give a score of 5. Please only return the evaluated score from 0 to 5.
Layer_2 |  For each criteria listed, please indicate which from the current {Social_Emotional_Learning_Principles} is not met, and which criteria are met.
Layer_3 |  Given a dialogue of a tutoring session between a tutor and a middle school student, please evaluate the Tutor based on specific given criteria : {rubric}, Please return 1 if the tutor correctly used the tutoring practice. Return 0 if the tutor incorrectly used the tutoring practice. Provide your evaluation in the form of a number. Please also list evidence why you provide the evaluation.
Table 5: Retrieval-Augmented Generation Based Prompt |  Prompt
---|---
Retriever |  For each criteria and rubric above, please identify all of tutor’s correct and incorrect use of the practice above. Return the dialogues of the tutor as evidence in the format: 1. Competency 2. Each Criteria of the competency 3. Sentences that tutor said within the dialogue serves as evidence.
Generator |  Return the score of {Principle_Name} from 0 to 5 based on the evaluation. Give one point to each criteria met. Please only return the evaluated score from 0 to 5.

##  Appendix B Social-Emotional Learning Principles

Table 6: Social Emotional Learning Principles Principles |  Description
---|---
Giving Effective Praise |  Praising students for putting forth effort by giving process-focused praise instead of praising students for getting an answer correct or getting a good grade
Supporting a Growth Mindset |  Supporting a growth mindset instead of a fixed mindset by encouraging students on the learning process and not necessarily just getting the answer
Reacting to Errors |  Responding to students when students make errors or mistakes, by not directly calling attention to the error but guiding students to realize and correct the error themselves.
Responding to Negative Self-Talk |  Responding to students positively when students engage in negative self-talk, such as saying “I can’t do this” or “this is too hard for me” by validating a student’s feelings but encouraging and building their self-efficacy
Using Motivational Strategies |  Rewarding students by using intrinsic and extrinsic motivation strategies, such as rewarding students for working hard by giving them time at the end of a session to discuss their interests


# Pairwise disjoint Moebius bands in space

Olga Frolkina
Chair of General Topology and Geometry,
Faculty of Mechanics and Mathematics,
M.V. Lomonosov Moscow State University,
Leninskie Gory 1, GSP-1,
Moscow 119991, Russia
E-mail<EMAIL_ADDRESS>
###### Abstract

V.V. Grushin and V.P. Palamodov proved in 1962 that it is impossible to place
in $R^{3}$ uncountably many pairwise disjoint polyhedra each homeomorphic to
the Moebius band. We generalize this result in two directions. First, we give
a generalization of this result to tame subsets in $R^{N}$, $N\geqslant 3$.
Second, we show that in case of $R^{3}$ the theorem holds for arbitrarily
topologically embedded (not necessarily tame) Moebius bands.

Keywords: Euclidean space, Moebius band, topological embedding, wild
embedding.

MSC 2000: 57M30, 57N35, 54C25

##  1 Introduction

Theory of wild embeddings comes back to papers of L. Antoine, P.S. Urysohn,
J.W. Alexander from 1920’s. Let us mention the reviews [16], [17], [20], [22]
and the books [23], [30], [36] which contain hundreds of references on the
subject.

######  Definition 1.1.

A subset $X\subset R^{N}$ is called a polyhedron if it is a union of a finite
collection of simplices. A subset $X\subset R^{N}$ homeomorphic to a
polyhedron is called tame if there is a homeomorphism $h$ of $R^{N}$ onto
itself such that $h(X)\subset R^{N}$ is a polyhedron; otherwise, $X$ is called
wild. An embedding $F:X\to R^{N}$ is called wild if its image $F(X)$ is a wild
subset of $R^{N}$, and tame otherwise.

For cells and spheres, it is useful to compare this with the following
definition:

######  Definition 1.2.

A subset $X\subset R^{N}$ homeomorphic to $I^{k}$ or $S^{k}$ is called flat if
there exists a homeomorphism $h$ of $R^{N}$ onto itself such that $h(X)$ is a
$k$-simplex or the boundary of a $(k+1)$-simplex.

From classical results of A. Schoenflies and L. Antoine we know that each arc
and each simple closed curve in $R^{2}$ is flat [30, II.4]. Antoine
constructed first wild arcs in $R^{3}$ in [3, 54–58, p.65–70; 83, p. 97]. His
first wild arc $ab$ [3, 54–58, p.65–70] consists of an arc $ap$ with a
sequence of trefoils tied in it and convergent to the end point $p$, united
with a straight line segment $pb$ (see the picture in [36, Exercise 2.7.3] or
[23, Fig. 2.26]; this example is often attributed to R.L. Wilder because of
the footnote on page 634 of [39]; let us underline that the footnote contains
reference to Antoine’s Thesis [3]). Second wild arc [3, 83, p. 97] contains a
wild Cantor set (now called Antoine necklace). Moreover, Antoine constructs an
everywhere wild arc in $R^{3}$ [5]. Well-known Alexander horned sphere [1] and
Antoine-Alexander sphere [4, p. 285], [2], [5, 21, p.283–284] provide examples
of wild surfaces in $R^{3}$. For $N\geqslant 3$, each polyhedron $P\subset
R^{N}$ with ${\dim P\geqslant 1}$ can be wildly embedded in $R^{N}$ (see [29]
and [11]; compare [3, 83, p. 97], [4], [5]). In $R^{N}$, $N\geqslant 3$, there
exist everywhere wild cells and spheres of all dimensions $1,\ldots,N-1$ (see
[8], [23, p.86–88, p.73–74], [36, Thm. 2.6.1]). It is known that for each $N$
and each $k\neq N-2$ any tame $k$-cell or $k$-sphere in $R^{N}$ is flat, refer
e.g. to [23, Cor. 1.1.2, Thm. 1.1.3, Cor. 1.1.6]. For any $N$, each tame arc
in $R^{N}$ is flat [30, p.125–126]. For $N\geqslant 4$ there exist tame but
not locally flat $(N-2)$-cells and $(N-2)$-spheres in $R^{N}$ [23, Example
1.4.2].

“Being wild” should not be considered as a strange exception: in the sense of
Baire category, most knots in $R^{3}$ are wild [33, Thm. 1] (see also [23,
Prop. 6.3.8]); most arcs in $R^{3}$ are wild [13]; in contrast to this, most
2-spheres in $R^{3}$ are tame [13].

Classification of wild embeddings is a hard problem. By [26, Thm.5.4], the
classification problem of embeddings of the Cantor set $\mathcal{C}$ in
$R^{3}$ is at least as complicated as the classification of countable linear
orders. Using methods of descriptive theory it is shown in [31] that the
classification problem for wild knots is strictly harder than that for
countable structures.

R.H. Bing proved that it is impossible to place in $R^{3}$ uncountably many
pairwise disjoint wild closed surfaces. Scheme of proof is given in [9], and
the full proof needs results of [7] and [10]; also, proof of Bing is explained
in [16, Thm. 3.6.1]. (The assumption of wildness is essential: concentric
spheres of all positive radii is an uncountable family of pairwise disjoint
tame surfaces.) J.R. Stallings constructed [38] a family of pairwise disjoint
wild 2-disks in $R^{3}$ which has a cardinality of continuum. J. Martin showed
that all except countably many disks in such a collection must be locally tame
at their interior points [32] (this result was predicted in [10] where
important facts needed for Martin’s arguments are proved). R.B. Sher modified
Stallings’ construction so that no two disks of the family are ambiently
homeomorphic, that is, no self-homeomorphism of $R^{3}$ can map one disk onto
another [37]. For higher dimensions, Bing non-embedding result has at the
moment only partial generalizations, see [15, Thm. 1, 2], [17, Thm. 10.5],
[22, p.383, Thm. 3C.2]. The results of Stallings and Sher were generalized by
the author to the case of arbitrary $R^{N}$, $N\geqslant 3$, and arbitrary
perfect compact space embeddable in a hyperplane $R^{N-1}\subset R^{N}$ [25],
[24]. During my Conference talk [24] I was asked by Professor Scott Carter if
it is possible to place incountably many pairwise disjoint Moebius bands in
$R^{3}$. He informed that in prior years he knew the impossibility proof from
Bob Williams (University of Texas). I became very interested in this question.
I found a paper of V.V. Grushin and V.P. Palamodov containing a partial answer
[27, Thm. 3]: Let $P$ be a 2-dimensional connected polyhedron which can be
represented as a (finite) union of $2$-simplices. In $R^{3}$ one can place
uncountably many pairwise disjoint polyhedra homeomorphic to $P$ if and only
if $P$ is orientable and each point $p\in P$ has a planar neighborhood. Let us
underline that for the “$\Rightarrow$” direction, each copy of $P$ is assumed
to be a polyhedron in $R^{3}$. The proof given in [27] essentially exploits
this strong assumption. In his Zentralblatt review to Grushin-Palamodov paper,
H.G. Bothe states their result [27, Thm. 3] in a more general form, without
assumption that $P$ is a union of $2$-simplices [12].

Below, I generalize result of V.V. Grushin and V.P. Palamodov in two
directions: 1) I replace $R^{3}$ by arbitrary $R^{N}$ but assume that the
subsets are tame (Corollary 2.2); and 2) I stay in $R^{3}$ but allow the
subsets to be wild (Proposition 3.2, Corollary 3.6). Corollary 3.6 improves
Bothe’s assertion [12].

######  Remark 1.3.

In [28], results of Grushin and Palamodov are represented in a much more
stronger form than originaly given in [27]. Unfortunately, this leads to
several mistakes.
1) “Simple continua” in terminology of V.K. Ionin and Yu.G. Nikonorov need not
be tame; for example, wild arc of Antoine-Wilder [3, 54–58, p.65–70], [36,
Exercise 2.7.3] is a “simple continuum”.
2) Theorem 1 in [28] is false; wild Antoine-Wilder arc can serve as a
counterexample. (Its “spaciality” in terms of [28] can be derived from [3,
56–57, p.66–69]; and it is not hard to show that it is “thin” in terminology
of [28]. I am planning to explain this in detail in another paper.)
3) Let us analyze Theorem 2 of [28]. Suppose that $K$ is a wild $(N-1)$-sphere
in $R^{N}$, where $N\neq 4$. In terminology of [28], $K$ is “thick” by [9],
[17, Thm. 10.5], [22, p.383, Thm. 3C.2]. Assuming that [28, Thm. 2] is true,
we conclude that $K$ can not be represented as a union of a finite family of
tame cells. This hard statement is of course not contained in [27]. (An
$(N-1)$-sphere in $R^{N}$ which is the union of interiors of tame
$(N-1)$-cells is itself locally flat and hence flat by [14]. If the interiors
of the cells do not cover the sphere, the statement is proved under strong
additional conditions in [18, Thm. 6.4] using A.V. Chernavskii’s result [19,
Thm. 2], [21].)

###  1.1 Conventions and Notation

By a map we always mean a continuous map.

For any metric compact space $M$, the space $C(M,R^{N})$ of all maps $M\to
R^{N}$ endowed with the uniform distance is known to be a complete separable
metric space.

By $\operatorname{conv}(S)$ we denote the convex hull of a set $S\subset
R^{N}$.

An $N$-manifold is a separable metric space each point of which has a
neighborhood homeomorphic to $R^{N}$. An $N$-manifold-with-boundary is a
separable metric space each point of which has a neighborhood whose closure is
homeomorphic to an $N$-simplex. (So, any $N$-manifold is an $N$-manifold-with-
boundary.) For an $N$-manifold-with-boundary $M$, its boundary is denoted by
$\partial M$ and its interior $M-\partial M$ by $\operatorname{Int}M$.

By $\overline{Y}$ we denote the closure of a set $Y$ in a given ambient space.

##  2 Families of tame $(N-1)$-surfaces in $R^{N}$

In their proof of [27, Thm. 2], Grushin and Palamodov show that if two
polyhedral Moebius bands in $R^{3}$ consisting of the same number of triangles
could be “sufficiently close” to each other, than they would be orientable.
Our next Proposition is based on the same idea. We make a more careful
investigation; and only the “limit” Moebius band is assumed to be tame.

In contrast to Grushin and Palamodov who measure “nearness” of two polyhedral
Moebius bands by considering broken lines with vertices in centers of the
faces, we use properties of space of maps $C(M,R^{N})$ (inspired by the Bing
idea, see [16, Thm. 3.6.1], [9]); we thus obtain Corollary 2.2.

######  Proposition 2.1.

Let $M$ be a compact triangulable $(N-1)$-manifold-with-boundary, and let
$f_{1},f_{2},\ldots,F$ be its embeddings into $R^{N}$ such that: $F(M)$ is
tame; the sequence $\\{f_{n}\\}$ converges to $F$ in $C(M,R^{N})$; and
$f_{n}(M)\cap F(M)=\emptyset$ for each $n$. Then $M$ is orientable.

###### Proof.

Since $F(M)$ is a tame subset of $R^{N}$, there exists a homeomorphism
$h:R^{N}\cong R^{N}$ such that $h(F(M))$ is a polyhedron. Using uniform
continuity of $h$ on a large compact ball $B\supset\operatorname{Int}B\supset
F(M)$ we see that the sequence of maps $\\{h\circ f_{n}\\}$ converges to
$h\circ F$ in $C(M,R^{N})$. Therefore we will omit the homeomorphism $h$; we
will assume that $F(M)$ is already a polyhedron in $R^{N}$, and the sequence
$\\{f_{n}\\}$ converges to $F$.

Let us assume that $M$ is non-orientable. Fix any triangulation $\tau$ of the
polyhedron $F(M)$. There exists a sequence of $(N-1)$-simplices
$\Delta_{1},\ldots,\Delta_{m}$ of $\tau$ such that each pair of adjacent
simplices (and also the last and the first one) have a common $(N-2)$-face;
and this sequence does not possess coherent orientations of its simplices. We
may also assume that $\Delta_{i}\neq\Delta_{j}$ for $i\neq j$.

For each $i=1,\ldots,m$ let $A_{i}$ be the center of the simplex $\Delta_{i}$,
and $A_{i,i+1}$ be the center of the $(N-2)$-simplex
$\Delta_{i}\cap\Delta_{i+1}$. (By $\Delta_{m+1}$ we mean $\Delta_{1}$; in
particular, $A_{m+1}=A_{1}$, and $A_{m,m+1}$ is the center of the
$(N-2)$-simplex $\Delta_{m}\cap\Delta_{1}$.) Let $A_{i}A_{i,i+1}A_{i+1}$ be
the broken line defined as the union of two segments: $A_{i}A_{i,i+1}$ and
$A_{i,i+1}A_{i+1}$. Let us denote $P_{i}=F^{-1}(A_{i})$ and
$q_{i}=F^{-1}(A_{i}A_{i,i+1}A_{i+1})$ for each $i=1,\ldots,m$.

Take a positive number $\delta$ such that

$\delta<\min_{i=1,\ldots,m}\\{d(A_{i},\partial\Delta_{i});\
d(A_{i}A_{i,i+1}A_{i+1},\partial(\Delta_{i}\cup\Delta_{i+1}))\\}.$

Take an integer $K$ such that $d(f_{K},F)<\delta$.

Fix an orientation of $R^{N}$. For each $i=1,\ldots,m$ we have

$d(f_{K}(P_{i}),A_{i})=d(f_{K}(P_{i}),F(P_{i}))<\delta<d(A_{i},\partial\Delta_{i});$

moreover, $f_{K}(P_{i})\notin\Delta_{i}\subset F(M)$. Therefore the point
$f_{K}(P_{i})$ defines the direction of a normal vector for the hyperplane
containing the simplex $\Delta_{i}$. Moreover, for each $i=1,\ldots,m$ we have
$d(f_{K}(q_{i}),A_{i}A_{i,i+1}A_{i+1})=d(f_{K}(q_{i}),F(q_{i}))<\delta$ and
$f_{K}(q_{i})\cap(\Delta_{i}\cup\Delta_{i+1})\subset f_{K}(M)\cap
F(M)=\emptyset$; hence the orientations of $\Delta_{i}$ and $\Delta_{i+1}$
thus defined are coherent. This contradiction finishes the proof. ∎

######  Corollary 2.2.

It is impossible to place in $R^{N}$, $N\geqslant 3$ uncountably many pairwise
disjoint tame homeomorphic images of a compact triangulable non-orientable
${(N-1)}$-manifold-with-boundary.

###### Proof.

Suppose the contrary. Let $\mathcal{H}=\\{h_{\alpha}:M\to R^{N},\alpha\in
A\\}$ be an uncountable family of embeddings such that $h_{\alpha}(M)\cap
h_{\beta}(M)=\emptyset$ for each $\alpha\neq\beta$, where $M$ is a compact
triangulable non-orientable $(N-1)$-manifold-with-boundary.

The space $C(M,R^{N})$ is separable. Hence its uncountable subset
$\mathcal{H}$ has a converging subsequence of distinct elements
$h_{\alpha_{m}}$ (the limit map $h=\lim\limits_{m\to\infty}h_{\alpha_{m}}$
also belongs to $\mathcal{H}$). To finish the proof, apply Proposition 2.1. ∎

##  3 Families of topological Moebius bands in $R^{3}$

######  Definition 3.1.

A set $X\subset R^{N}$ is called locally tame at $x\in X$ if there is a
neighborhood $U$ of $x$ in $R^{N}$ and a homeomorphism $h$ of $\overline{U}$
into $R^{N}$ such that $h(\overline{U}\cap X)$ is a polyhedron.

By [6] or [34], closed locally tame subsets of $R^{3}$ are tame.

######  Proposition 3.2.

Let $M$ be a compact non-orientable 2-manifold-with-boundary. It is impossible
to place uncountably many pairwise disjoint homeomorphic images of $M$ in
$R^{3}$.

###### Proof.

It suffices to consider the case when $M$ is a (compact) Moebius band
$\mu=I\times I/(0,t)\sim(1,1-t)$. Let us assume that $\\{M_{\alpha},\alpha\in
A\\}$ is an uncountable family of pairwise disjoint subsets of $R^{3}$, and
$h_{\alpha}:\mu\cong M_{\alpha}$ is a homeomorphism for each $\alpha\in A$.

Let $\pi:I\times I\to\mu$ be the factorization map. Represent $\mu$ as a union
of two disks: $\mu=D_{1}\cup D_{2},$ where

$D_{1}=\pi\left(\left[0;\frac{3}{4}\right]\times I\right);\quad
D_{2}=\pi\left(\left(\left[0;\frac{1}{4}\right]\cup\left[\frac{1}{2};1\right]\right)\times
I\right).$

Apply [32, Lemma 2] to the uncountable family of 2-disks
$\\{h_{\alpha}(D_{1}),\alpha\in A\\}$. There exists an uncountable subset
$A_{1}\subset A$ such that each disk $h_{\alpha}(D_{1})$, $\alpha\in A_{1}$,
is locally tame at each $x\in\operatorname{Int}h_{\alpha}(D_{1})$. Again,
apply [32, Lemma 2] to the uncountable family of 2-disks
$\\{h_{\alpha}(D_{2}),\alpha\in A_{1}\\}$; there exists an uncountable subset
$A_{2}\subset A_{1}$ such that each $h_{\alpha}(D_{2})$, $\alpha\in A_{2}$, is
locally tame at each of its interior points.

We obtain an uncountable set $A_{2}$ such that each disk $h_{\alpha}(D_{i})$,
$\alpha\in A_{2}$, $i=1,2$, is locally tame at each of its interior points.
Hence each Moebius band $h_{\alpha}(\mu)$, $\alpha\in A_{2}$, is locally tame
at each of its interior points. Replace $\mu$ by a smaller Moebius band
$\tilde{\mu}=\pi\left(I\times\left[\frac{1}{4};\frac{3}{4}\right]\right)\subset\mu$;
we obtain an uncountable family of pairwise disjoint locally tame Moebius
bands $\\{h_{\alpha}(\tilde{\mu}),\alpha\in A_{2}\\}$. By [6] or [34], each
band $h_{\alpha}(\tilde{\mu})$, $\alpha\in A_{2}$ is tame. Application of
Corollary 2.2 finishes the proof. ∎

Recall that a topological space $X$ is called locally planar if each point
$x\in X$ has a neighborhood $U$ which can be embedded in plane.

######  Definition 3.3.

[27, p.165] Let $P$ be a locally planar 2-dimensional polyhedron (in some
$R^{N}$). Take any of its triangulations. $P$ is called orientable if one can
give coherent orientations to all its 2-simplices.

Note that orientability implies local planarity; but we prefer to write
“locally planar and orientable” in order to avoid occasional misunderstanding.

######  Remark 3.4.

For a locally planar 2-dimensional polyhedron $P$, the following are
equivalent to the above definition.

1) Let $\hat{P}$ be the union of all 2-dimensional simplices of $P$. Note that
$\hat{P}$ is a homogeneous non-branching 2-dimensional complex in the sense of
[35, p.314, p.462]. By [35, 43.24] $\hat{P}=\cup P_{i}$ where each $P_{i}$ is
a 2-dimensional pseudomanifold and $\dim(P_{i}\cap P_{j})\leqslant 0$ for each
$i\neq j$. We say that the polyhedron $P$ is orientable if each pseudomanifold
$P_{i}$ is orientable.

2) $P$ is called orientable if it does not contain a homeomorphic copy of the
Moebius band (this is taken as a definition in [12]).

Corollary 3.6 extends [27, Thm. 3] and its generalization stated in [12].
First, let us prove a lemma (compare [27, p. 167, Lemma] where the case of
polyhedra all whose maximal simplices are 2-dimensional is considered).

######  Lemma 3.5.

Let $P$ be a locally planar 2-dimensional polyhedron (in some $R^{N}$).
Moreover, suppose that $P$ is orientable. Then there exists an orientable
2-manifold-with-boundary $M$ such that $P\subset M\subset R^{N}$ and $M$ is a
polyhedron in $R^{N}$.

###### Proof.

By [27, p.167–168, Lemma] it suffices to find a locally planar orientable
2-dimensional polyhedron $\tilde{P}$ with $P\subset\tilde{P}\subset R^{N}$
such that each maximal simplex of $\tilde{P}$ is 2-dimensional.

Fix any triangulation $\tau$ of $P$. For each isolated point $A$ of $P$
(0-dimensional maximal simplex of $\tau$), replace $A$ with a small triangle
$\Delta$ such that $A$ is among its vertices and $\Delta\cap P=\\{A\\}$.
Similarly, if $\sigma$ is a maximal 1-dimensional simplex of $\tau$, replace
it with a narrow triangle $\Delta=\operatorname{conv}(\sigma,B)$, where
$B\notin\sigma$ is a point sufficiently close to the midpoint of $\sigma$ so
that $\Delta\cap P=\sigma\cap P$. It can be easily seen that the polyhedron
$\tilde{P}$ obtained after all replacements satisfies the conditions listed
above. ∎

######  Corollary 3.6.

Let $P$ be a 2-dimensional polyhedron (in some $R^{N}$). The following are
equivalent:
(a) in $R^{3}$ one can place uncountably many pairwise disjoint polyhedra
homeomorphic to $P$;
(b) in $R^{3}$ one can place uncountably many pairwise disjoint subsets
homeomorphic to $P$;
(c) $P$ is locally planar and orientable.

###### Proof.

$(a)\Rightarrow(b)$ is evident.

Let us prove $(b)\Rightarrow(c)$.
1) Suppose that a point $p\in P$ has no planar neighborhood. There exist a
2-simplex $\Delta\subset P$ and an arc $\alpha\subset P$ such that
$p\in\operatorname{Int}\Delta\cap\partial\alpha$. By [40], it is impossible to
place uncountably many disjoint homeomorphic copies of the “umbrella”
$\Delta\cup\alpha$ (hence also of $P$) in $R^{3}$. (V.V. Grushin and V.P.
Palamodov seem to be unfamiliar with [40] while writing [27], and they
included the proof of impossibility to place in $R^{3}$ uncountably many
disjoint polyhedra homeomorphic to the umbrella.)
2) Assume that $P$ is non-orientable. Then it contains a homeomorphic image of
the Moebius band $\mu$. By Proposition 3.2, it is impossible to place
uncountably many disjoint homeomorphic copies of $\mu$ (hence also of $P$) in
$R^{3}$.

Finally, $(c)\Rightarrow(a)$ can be proved similarly to [27], using Lemma 3.5.
Namely, there exists an orientable polyhedral 2-manifold-with-boundary $M$
with $P\subset M\subset R^{N}$. It is known that $M$ is homeomorphic to a
sphere with handles and holes; clearly we can place in $R^{3}$ uncountably
many pairwise disjoint polyhedra each PL-homeomorphic to $M$. The inclusion
$P\subset M$ can be interpreted as a PL embedding; thus we obtain uncountably
many disjoint homeomorphic polyhedral copies of $P$ in $R^{3}$. ∎

## Acknowledgments

I would like to thank Sergey Melikhov for useful remarks which he made while
reviewing this paper.

## References

  * [1] J.W. Alexander, An example of a simply connected surface bounding a region which is not simply connected, Proc. Nat. Acad. Sci. USA 10 (1) (1924) 8–10.
  * [2] J.W. Alexander, Remarks on a point set constructed by Antoine, Proc. Nat. Acad. Sci. USA 10 (1) (1924) 10–12.
  * [3] L. Antoine, Sur l’homéomorphie de deux figures et de leurs voisinages. Thèses françaises de l’entre-deux-guerres, vol. 28, 1921. (= L. Antoine, Sur l’homéomorphie de deux figures et de leur voisinages. (Thèses présentées à la Faculté des Sciences de Strasbourg.) Journ. de Math. (8) 4 (1921) 221-325.)
http://www.numdam.org/issue/THESE_1921__28__1_0.pdf

  * [4] L. Antoine, Sur les ensembles parfaits partout discontinus, C.R. Acad. Sci. Paris 173 (1921) 284–285.
  * [5] L. Antoine, Sur les voisinages de deux figures homéomorphes, Fund. Math. 5 (1924) 265–287.
  * [6] R.H. Bing, Locally tame sets are tame, Ann. of Math. 59 (1954) 145–158.
  * [7] R.H. Bing, Conditions under which a surface in $E^{3}$ is tame, Fund. Math. 47 (1959) 105–139.
  * [8] R.H. Bing, A wild surface each of whose arcs is tame, Duke Math. J. 28 (1961) 1–15.
  * [9] R.H. Bing, $E^{3}$ does not contain uncountably many mutually exclusive wild surfaces, Bull. Amer. Math. Soc. 63 (1957) 404. Abstract # 801t.
  * [10] R.H. Bing, A surface is tame if its complement is 1-ULC, Trans. Amer. Math. Soc. 101 (1961), 2, 294–305.
  * [11] W.A. Blankinship, Generalization of a construction of Antoine, Ann. Math. (2) 53 (1951) 276–297.
  * [12] H.G. Bothe, Über die maximale Anzahl von paarweise fremden zueinander homömorphen Figuren, die sich im dreidimensionalen Raum unterbringen lassen (Russian) Usp. Mat. Nauk 17, No.3 (105), 163–168 (1962). Zentralblatt Review: Zbl 0109.41003.
  * [13] H.G. Bothe, Die meisten Bogen sind wild, Bull. Acad. Pol. Sci., Sér. Sci. Math. Astron. Phys. 14 (1966) 309–312.
  * [14] M. Brown, Locally flat imbeddings of topological manifolds, Ann. of Math. (2) 75 (1962) 331–341.
  * [15] J.L. Bryant, Concerning uncountable families of $n$-cells in $E^{n}$, Michigan Math. J. 15 (1968) 477–479.
  * [16] C.E. Burgess, J.W. Cannon, Embeddings of surfaces in $E^{3}$, Rocky Mountain J. of Math. 1 (2) (1971) 259–344.
  * [17] C.E. Burgess, Embeddings of surfaces in Euclidean three-space, Bull. Amer. Math. Soc. 81 (1975), 5, 795–818.
  * [18] J.C. Cantrell, R.C. Lacher, Local flattening of a submanifold, Quart. J. Math. Oxford Ser. (2) 20 (1969) 1–10.
  * [19] A.V. Čhernavskiĭ, Singular points of topological imbeddings of manifolds and the union of locally flat cells, Soviet Math. Dokl. (English translation) 7 (1966) 433–436.
  * [20] A.V. Chernavskii, On the work of L.V. Keldysh and her seminar, Russian Math. Surveys, 60 (4) (2005) 589–614; transl. from: Usp. Mat. Nauk 60 (4) (2005) 11–36.
  * [21] A.V. Chernavsky, Theorem on the union of two topologically flat cells of codimension 1 in $R^{n}$, Abstr. Appl. Anal. 2006, Article ID 82602, 9 p. (2006). https://www.hindawi.com/journals/aaa/2006/082602/abs/
  * [22] R.J. Daverman, Embeddings of $(n-1)$-spheres in Euclidean $n$-space, Bull. Amer. Math. Soc. 84 (1978), 3, 377–405.
  * [23] R.J. Daverman, D.A. Venema. Embeddings in Manifolds. Graduate Studies in Mathematics 106. Providence, RI: American Mathematical Society. 2009\.
  * [24] O. Frolkina, Uncountable families of pairwise disjoint inequivalent wild $k$-disks in $R^{n}$, 4th Russian-Chinese Conference ”Knot Theory and Related Topics”, Moscow, 3-7 July, 2017, Abstracts, Bauman Moscow State Technical University Moscow, p. 11-12.
  * [25] O. Frolkina, Wild high-dimensional Cantor fences in $\mathbb{R}^{n}$, Part I, Topol. Appl. (submitted)
  * [26] P. Gartside, M. Kovan-Bakan, On the space of Cantor subsets of $R^{3}$, Topol. Appl. 160 (2013) 1088–1098.
  * [27] V.V. Grushin, V.P. Palamodov, On the maximal number of mutually disjoint, pairwise homeomorphic figures which can be packed in 3-space (in Russian), Uspekhi Mat. Nauk, 17:3(105) (1962) 163–168.
  * [28] V.K. Ionin, Yu.G. Nikonorov, An uncountable family of disjoint spatial continua in Euclidean space, Sib. Math. J. 34 (1993), 5, 848–851; transl. from Sib. Mat. Zh. 34 (1993), 5, 63–66.
  * [29] A.A. Ivanov, Isotopy of compacta in Euclidean spaces, Doctoral dissertation (Candidate of Sciences), St. Petersburg Department of V.A. Steklov Institute of Mathematics of the USSR Academy of Sciences, Moscow-Leningrad, 1950 (in Russian).
  * [30] L.V. Keldysh, Topological imbeddings in Euclidean space (English. Russian original), Proc. Steklov Inst. Math. 81 (1966), 203 p.; translation from: Tr. Mat. Inst. Steklov. 81 (1966), 184 p.
  * [31] V. Kulikov, A non-classification result for wild knots, Trans. Am. Math. Soc. 369 (2017) 8, 5829–5853.
  * [32] J. Martin, A note on the uncountably many disks, Pacific J. Math. 13 (1963) 1331–1333.
  * [33] J.W. Milnor, Most knots are wild, Fund. Math. 54 (1964) 335–338.
  * [34] E.E. Moise, Affine structures in $3$-manifolds, VIII. Invariance of the knot type; local tame imbedding, Ann. of Math. 59 (1954) 159–170.
  * [35] W. Rinow. Lehrbuch der Topologie. VEB Deutscher Verlag der Wissenschaften, Berlin 1975.
  * [36] T.B. Rushing. Topological embeddings. Pure and Applied Mathematics, Vol. 52, New York and London, Academic Press, 1973.
  * [37] R.B. Sher, A note on the example of Stallings, Proc. Amer. Math. Soc., 19 (3) (1968) 619–620.
  * [38] J.R. Stallings, Uncountably many wild disks, Annals of Math. 71 (1) (1960) 185–186.
  * [39] R.L. Wilder, A converse of the Jordan-Brouwer separation theorem in three dimensions, Trans. Amer. Math. Soc. 32 (1930) 632–657.
  * [40] G.S. Young Jr., A generalization of Moore’s theorem on simple triods, Bull. Amer. Math. Soc. 50 (1944) 714.


# Single crystal synthesis, structure, and magnetism of Pb10-xCux(PO4)6O

P. Puphal<EMAIL_ADDRESS>Max Planck Institute for Solid State Research,
Heisenbergstraße 1, D-70569 Stuttgart, Germany  M. Y. P. Akbar  Max Planck
Institute for Solid State Research, Heisenbergstraße 1, D-70569 Stuttgart,
Germany  Faculty of Mathematics and Natural Sciences, Institut Teknologi
Bandung, Jl. Ganesha 10, 40132 Bandung, Indonesia  M. Hepting  E. Goering  M.
Isobe  Max Planck Institute for Solid State Research, Heisenbergstraße 1,
D-70569 Stuttgart, Germany  A. A. Nugroho  Max Planck Institute for Solid
State Research, Heisenbergstraße 1, D-70569 Stuttgart, Germany  Faculty of
Mathematics and Natural Sciences, Institut Teknologi Bandung, Jl. Ganesha 10,
40132 Bandung, Indonesia  B. Keimer  Max Planck Institute for Solid State
Research, Heisenbergstraße 1, D-70569 Stuttgart, Germany

###### Abstract

The recent claim of superconductivity above room temperature in
Pb10-xCux(PO4)6O with 0.9 < $x$ < 1 (referred to as LK-99) has sparked
considerable interest. To minimize the influence of structural defects and
impurity phases on the physical properties, we have synthesized phase-pure
single crystals with a copper doping level of $x\sim 1$. We find that the
crystals are highly insulating and optically transparent. X-ray analysis
reveals an uneven distribution of the substituted Cu throughout the sample.
Temperature ($T$) dependent magnetic susceptibility measurements for $2\leq
T\leq 800$ K reveal the diamagnetic response characteristic of a non-magnetic
insulator, as well as a small ferromagnetic component, possibly originating
from frustrated exchange interactions in Cu-rich clusters in the
Pb10-xCux(PO4)6O structure. No anomalies indicative of phase transitions are
observed. We therefore rule out the presence of superconductivity in
Pb9Cu(PO4)6O crystals, and provide some considerations on the origin of
anomalies previously reported in experiments on polycrystalline specimen.

††preprint: AIP/123-QED

> ##  I Introduction

Room-temperature superconductivity holds the potential to disrupt various
technologies—from enhancing energy storage and grid efficiency to advancing
maglev trains, medical imaging, and electronic devices. In this context,
recent claims about the presence of superconductivity in powders of the Cu-
substituted lead phosphate Pb10-xCux(PO4)6O with 0.9 < $x$ < 1 (referred to as
LK-99) [1, 2] have stirred significant anticipation within the condensed
matter research community and beyond. In particular, the pronounced
resistivity drop and the onset of a large diamagnetic response at a
temperatures $T\sim 400$ K have been interpreted as evidence of
superconductivity in the initial studies. Yet, efforts to replicate these key
observations in polycrystalline specimens have yielded disparate results [3,
4, 5, 6, 7, 8, 9, 10, 11, 12], likely due to different levels of impurity
phases generated by the complex synthesis via sulfur based compounds. In
particular, the initial studies [1, 2] reported a significant residue of Cu2S
in their powder samples. Moreover, grain boundaries in polycrystalline
samples, whose prevalence and properties may depend on the preparation
conditions, often exhibit physical properties different from those of the
bulk. Conclusive tests of the original claims therefore require a well-defined
pure phase, based on single-crystalline samples.

Motivated by these considerations, we have synthesized phase-pure single
crystals of Pb9Cu(PO4)6O using the travelling solvent floating zone growth
(TSFZ) method, relying solely on phosphate-based compounds as precursors. None
of the purported signatures of superconductivity are confirmed in these
specimens. In particular, the crystals are highly insulating and optically
transparent. Whereas the magnetic susceptibility includes a small
ferromagnetic component (likely due to local magnetic order in the
inhomogeneous network of Cu substituents), the magnetic susceptibility
exhibits no anomalies indicative of phase transitions (including
superconductivity) in a wide range of temperatures up to 800 K. We discuss
possible origins of the anomalies reported in phase-impure powder samples. In
particular, the impurity phase Cu2S exhibits two structural phase transitions,
with the one at 398 K coinciding with an insulator-to-metal transition [13]
that may be mistakenly attributed to superconductivity.

##  II Methods

A precursor powder was prepared by mixing the corresponding stoichiometries of
9 PbO : 1 CuO and 9 NH4H2PO4. Subsequently, the powder was ball-milled for 20
minutes and the mixture was transferred in alumina crucibles to a box furnace,
followed by heating to 750∘C for 10 hours followed by grinding and another
heating of 10 hours. Cylindrically shaped feed and seed rods were prepared by
ball-milling the sintered materials, which were filled into rubber forms with
6 mm diameter. The rubber was evacuated and pressed in a stainless steel form
filled with water using a Riken type S1-120 70 kN press. All rods were heat
treated at 800∘C. For the first growth, pellets were prepared in a 4mm
diameter press form with the Riken press. The single-crystal growth was
carried out in a Crystal System Corporation optical image furnace CSC
FZ-T-10000. Four halogen lamps operating at 150 W were used as a heating
source.

Figure 1: a) Selected region of the PbO - P2O5 phase diagram from Ref. 14,
with the phase formation temperature plotted as a function of the mol% of
P2O5. The blue arrow indicates the peritectic growth region of Pb10(PO4)6O,
and vertical thick black lines highlight the three surrounding phases. b)
Photograph of the orange PbO flux pellet balancing between the seed and feed
rods. c) Photograph of the growth process, utilizing 150 W lamps at 41.6 %
power. d,e) Photographs of a Pb9Cu(PO4)6O single crystal under polarized
light. Turning the crystal over by 90° leads to a color change from purple (d)
to brownish (e), revealing a dichroism effect. f) Powder XRD pattern of a
pulverized Pb9Cu(PO4)6O single crystal (red data points). The solid black line
corresponds to the calculated intensity from the Rietveld refinement, the
solid blue line is the difference between the experimental and calculated
intensities, and the vertical green bars are the calculated Bragg peak
positions.

The as-grown crystals are characterized by optical microscope to select the
phases obtained from the growth. The structural properties of each crystals
were characterized using powder x-ray diffraction (PXRD) performed at room
temperature using a Rigaku Miniflex diffractometer with Bragg Brentano
geometry, Cu Kα radiation and a Ni filter. Rietveld refinements were conducted
with the FullProf software suite. Single crystal diffraction was performed at
room temperature using a Rigaku XtaLAB mini II with Mo Kα radiation. The data
was analyzed with CrysAlis(Pro) and the final refinement done using Olex2 with
SHELX. Energy-dispersive x-ray spectra (EDX) were recorded with a NORAN System
7 (NSS212E) detector in a Tescan Vega (TS-5130MM) SEM. Differential Thermal
Analysis studies (DTA) and Thermogravimetry (TG) was done using a Netzsch
DTA/TG. For EDX mapping, we used a Thermofischer tabletop Phenom Pharos REM
from the MPI-IS, utilizing the high pressure option to prevent charging
phenomena. Magnetic susceptibility measurements were performed using a
vibrating sample magnetometer (MPMS 3, Quantum Design). For high temperature
measurements, we used the oven option of the MPMS SQUID. Electrical transport
measurements were attempted with a Physical Property Measurement System (PPMS,
Quantum Design). For EDX mapping, we used a Thermofischer tabletop Phenom
Pharos REM, utilizing the high pressure option to prevent charging phenomena.

##  III Results

Figure 2: (a) Reconstructed XRD intensity map of the (0 k l) plane measured on
a single crystal with a size of approximately 1 mm3. The corresponding
refinement results are presented in Tab. 1. (b) XRD map of the (h k 0) plane.
(c) Schematic of the refined crystal structure in $P6_{3}/m$ in the projection
along the c-axis direction, where the colored spheres represent Pb (light and
dark grey), P (purple), O (red), and Cu (blue). (d) Crystal structure in the
projection perpendicular to the c-axis direction, with the shortest Pb-Pb
bonds highlighted by thick gray lines.

A phase formation diagram for the synthesis of Pb10(PO4)6O has already been
reported in Ref. 14, and is depicted in Fig. 1a. Using this as a guideline for
the growth of Cu-substituted Pb10-xCux(PO4)6O single crystals, we employ the
TSFZ method, where a flux pellet is first placed on the seed rod and then
balancing between the seed and feed rods during the growth (Fig. 1b). Our
initial growth utilized a pure PbO pellet (batch 1). For the subsequent
growths, we prepared pellets with molar PbO:CuO ratios of 9:1 and 1:1 for
batch 2 and 3, respectively. The growth chamber was then evacuated and
subsequently filled with Ar at 1 bar, maintaining a flow rate of 0.3 l/min. In
accordance with the phase formation diagram, we initiated the growth using the
PbO self-flux. Upon connecting the rods, a slow growth rate of 1 mm/h was
maintained. Due to the decrease of the melting temperature when progressing
through the peritectic point, it was necessary to maintain a feed rate of 1
mm/h. This resulted in a broader seed, as illustrated in Fig. 1c.

Post-growth, the boule was fractured using a mortar, yielding transparent
crystals with a purple-brown shade in all batches. A representative crystal of
batch 1 is shown in Figs. 1d,e. The change of shading upon rotation of the
polarized light propagation direction (dichroism) confirms the single-
crystalline nature of our specimens. A powder XRD analysis of a crushed
crystal from batch 1 indicated a phase-pure Pb9Cu(PO4)6O composition with the
hexagonal $P6_{3}/m$ structure (space group #176). The refinement result is
summarized in Tab.. 1 and indicates a stoichiometry of Pb8.9(1)Cu1.1(2)P6O25.

As a next step, we performed multiple single-crystal XRD measurements,
specifically aimed at re-evaluating the details of the crystal structure of
the Cu substituted variant, which were previously determined only from powder
samples. The result of a single crystal of batch 3, indicate a stoichiometry
of Pb8.8(3)Cu1.2(3)P6O25, and are summarized in Tab. 1. The reconstructed XRD
maps along with the crystal structure are shown in Fig. 2. Notably, the
refinement of our XRD data allows for a determination of the Cu site occupancy
with a higher confidence. Furthermore, the result of the refinement suggests
that the Cu substitution coincides with a reduction in the oxygen occupation,
likely attributable to the square planar coordination of Cu. Overall, the XRD
analysis indicates that the average Cu doping level in our crystals falls into
the regime where the initial studies on powder samples [1, 2] claimed the
occurrence of superconductivity. Summarizing our XRD results we have obtained
a large varying lattice constants moving in the range of the two presented in
Tab. 1. As our powder data of one crushed crystal in Fig. 1 from batch 1 show
a,b= 9.85319(6) Å, and c = 7.44154(5) Å; another refinement of powder data
that is not shown on a crushed crystal of batch 3 yields a,b= 9.84847(12) Å,
and c = 7.43811(11) Å, while single crystal XRD of batch 1 a,b= 9.8380(9) Å,
and c = 7,422(11) Å, batch 2 a,b = 9.8140(18) Å, and c = 7.4370(15) Å, batch 3
a,b = 9.778(2) Å, and c = 7.3966(15) Åand finally the presented one in Fig. 2
of batch 3 a,b= 9.798(2) Å, and c = 7.3984(15) Å. As discussed further below
Cu is inhomogenously distributed, Pb 2+ with a coordination of VIII has a
large ionic radius of 1.29, while Cu is not even reported for such a high
coordination, hence the oxygen vacancies that we find in refinement. Cu
maximally reaches a ionic radius of 0.73 as Cu2+ with a coordination of VI.
Thus we expect a lattice contraction with substitution. Thus in accordance
with expectations we see a contraction of the lattice with enhanced Cu
substiution.

Table 1: Refined atomic coordinates of Pb9Cu(PO4)6O in the hexagonal $P6_{3}/m$ (#176) structure, extracted from powder XRD (first table) and single crystal XRD measurements (second table). The obtained lattice constants are a, b = 9.85319(6) Å, and c = 7.44154(5) Å for the powder data and a, b = 9.7393(19) Å, and c = 7.3953(12) for the single crystal data. Atom | Label | $x$ | $y$ | $z$ | Uiso (Å2) | Occ.
---|---|---|---|---|---|---
Pb | Pb1 | 0.333333 | 0.666667 | 0.0003(10) | 0.0141(6) | 0.89(2)
Cu | Cu1 | 0.333333 | 0.666667 | 0.0003(10) | 0.0141(6) | 0.11(2)
Pb | Pb2 | 0.7529(3) | 0.7550(4) | 0.25 | 0.0181(5) | 0.89(2)
Cu | Cu2 | 0.7529(3) | 0.7550(4) | 0.25 | 0.0181(5) | 0.11(2)
P | P1 | 0.4093(16) | 0.3821(15) | 0.25 | 0.022(4) | 1
O | O1 | 1 | 1 | 0.5 | 0.011(4) | 1
O | O2 | 0.365(4) | 0.515(3) | 0.25 | 0.011(4) | 1
O | O3 | 0.606(4) | 0.462(3) | 0.25 | 0.011(4) | 0.86(6)
O | O4 | 0.3579(20) | 0.2580(19) | 0.074(2) | 0.011(4) | 1
Atom | Label | $x$ | $y$ | $z$ | Uiso (Å2) | Occ.
Pb | Pb1 | 0.333333 | 0.666667 | 0.0040(3) | 0.0040(9) | 0.90(6)
Cu | Cu1 | 0.333333 | 0.666667 | 0.0040(3) | 0.0040(9) | 0.10(6)
Pb | Pb2 | 0.7531(2) | 0.7546(3) | 0.25 | 0.0060(8) | 0.86(6)
Cu | Cu2 | 0.7531(2) | 0.7546(3) | 0.25 | 0.0060(8) | 0.14(6)
P | P1 | 0.4026(16) | 0.3724(14) | 0.25 | 0.001(3) | 1
O | O1 | 1 | 1 | 0.5 | 0.008(19) | 1
O | O2 | 0.336(5) | 0.490(4) | 0.25 | 0.002(8) | 0.90(6)
O | O3 | 0.587(4) | 0.474(4) | 0.25 | 0.007(8) | 0.86(6)
O | O4 | 0.351(4) | 0.266(4) | 0.084(4) | 0.023(8) | 1

Figure 3 summarizes EDX data compiled from over 20 broken pieces of crystals
across three batches. Notably, when examining the crystals in a scanning
electron microscope, we observe a charging effect from the electron beam. This
suggests that the crystals are highly insulating, which was corroborated by
transport measurements (see below). A closer look at two EDX spectra, taken
from different locations on a single crystal’s surface (Fig. 3a), reveals a
varying Cu content as best seen in the EDX map shown in Fig. 3c). This implies
that the Cu doping is not distributed homogeneously in the crystals. However
we highlight that Cu is incorporated in the matrix as it can be found
everywhere in the single crystalline boule and is observed in single crystal
XRD refinement as shown before. To shed more light on this observation, we
plot the distribution of the atomic percentage (at%) of the Cu substitution
for several EDX measurement locations in Fig. 3b, illustrating a large
variation of Cu substitution levels. Due to the strong variation among the EDX
points, the error bar for the average Cu and Pb stoichiometry of a crystal is
relatively large. The determined average stoichiometries are
Pb8(1)Cu0.4(0.4)P6O25 for batch 1 grown with PbO flux, Pb9(1)Cu0.6(0.4)P6O25
for batch 2, and Pb9(1)Cu0.6(0.7)P6O25 for batch 3. Note that our single-
crystal XRD measured in transmission provides an accurate representation of
the substitutional content averaged across the entire crystal.

Figure 3: a) Selected EDX spectra from a Cu rich (red line) and Cu deficient
(black line) region on the cleaved surface of a Pb10-xCux(PO4)6O single
crystal. b) Distribution of the Cu/Pb substitution in at%, as extracted from
EDX spectra taken on various points on the surface of a single crystal. The
measurement time for each EDX point was 30 s. c) SEM image (top) and the
corresponding EDX map (bottom), where the spatial distribution of the EDX
intensity of Cu is highlighted in yellow.

Regarding the electronic transport properties, the optical transparency of our
crystals (Fig. 1 d,e) suggests an insulating or semiconducting nature.
However, there have been some reports of superconductivity coexisting with
optical transparency either due to an exceptionally low plasma frequency [15,
16, 17] or due to electronic phase separation into insulating majority and
superconducting minority phases [18, 19]. We therefore attempted to perform
four-point DC resistivity measurements in a PPMS setup (see Methods), but
found resistance values above several M$\Omega$ at room temperature. This
behavior is also in line with the charging effects observed in the electron
microscopy investigation presented above. Together, these findings
conclusively falsify the claim of room-temperature superconductivity in LK-99.
[1, 2]

Figure 4: a) Temperature dependence of the the magnetic susceptibility in SI
units at 7 T measured in field cooled (FC) and zero field cooled (ZFC) mode of
a crystal from batch 2 combining low temperature and heater options. The inset
shows the magnetic susceptibility in SI units in a field of 0.005 T b)
Magnetization versus field measured at 300 and 800 K of the same crystal.

We next discuss the magnetic response of the crystals between 2 to 800 K. A
first observation is a remarkably low measured signal strength, and hence we
note that a small temperature-independent offset might be present in our data,
originating from a minute signal from the sample holder, which was not
subtracted. For measurements in a small applied field of 5 mT, we detect a
positive magnetic susceptibility signal at all temperatures (Fig. 4a inset),
but no indications for a clear phase transition besides a tiny kink at 490 K.
Certainly there is no transition into a superconducting phase. Figure 4a)
shows the magnetic susceptibility measured in a strong field of 7 T, revealing
a diamagnetic response of the crystal at least up to 800 K. The measurement
was carried out on a crystal from batch 2. To verify the thermal stability of
our crystals and confirm that they do not decompose at elevated temperatures,
we conducted a differential thermal analysis (DTA/TG) up to 820 K.
Importantly, no mass loss was observed during this analysis (not shown here).
In addition, we carried out powder XRD measurements both before and after
heating, where we did not observe significant changes compared to the XRD of
the Pb9Cu(PO4)6O sample shown in Figure 1f.

We tried to evaluate the real signal strength of the diamagnetism in this
compound by using the room temperature value for the largest sample mass of 63
mg that we measured in a field of 1 T. In this case we carefully measured the
background signal and subtract it from our signal. Here, we observed a moment
of $-2.5\cdot 10^{-4}$ emu and a background signal of $-1.46\cdot 10^{-4}$ emu
reulting in $-1.04\cdot 10^{-4}$ emu for a sample of the volume of 3 x 3 x 1
mm3. Thus our volume susceptibility amounts to $-\frac{1.04\cdot
10^{-4}}{1000}\cdot\frac{4\pi\cdot 10^{-6}m^{3}}{3\cdot 3\cdot 1\cdot
10^{-9}m^{3}}\approx-1.452\cdot 10^{-4}$, which is a value that is remarkably,
comparable to Bi (notably for a superconductor we expect -1).

For zero field cooling (ZFC) a Curie tail due to isolated paramagnetic
impurities is observed at low temperatures, which leads to an upturn of the
susceptibility (Fig. 4a)). Nevertheless, the overall magnetic response remains
negative (diamagnetic) down to the lowest measured temperatures of 2 K. For
field cooling (FC) from 800 K, we observe a continuous rise in the
susceptibility as the temperature decreases. The termination of the FC
measurement below 300 K is due to technical constraints of our measurement
setup (see Methods). The difference between the ZFC and FC curves persists up
to 800 K, where the curves merge due to the here employed measurement protocol
heating to a maximum of 800 K. Note that this intersection simply reflects the
temperature where the cooling sequence is started, e.g. 390 K for the sequence
shown in Figure 5a. This behavior hints at spin glass dynamics, possibly
arising from partially ordered magnetic clusters within the sample. (Note that
we cannot rigorously rule out the presence of minute amounts of ferromagnetic
Fe impurities due to contaminated starting materials, but such contamination
is below the detection limits of both EDX and XRD measurements.) Indeed, a
magnetic field sweep at selected temperatures (Fig. 4b) reveals an initial
increase of the sample magnetization for increasing fields, pointing to the
presence of ferromagnetic or canted-antiferromagnetic correlations. We ascribe
this phenomenon to Cu clusters formed due to the inhomogeneous substitution,
resulting in a cluster spin glass effect also proposed theoretically for the
material [20]. Importantly, when exposed to stronger magnetic fields, this
effect saturates, and the material exhibits a purely diamagnetic response
(Fig. 4b). Further evidence for a cluster spin glass state is derived from the
time dependence of the magnetic susceptibility signal, observed for a
measurement at 2 K in a constant field of 7 T (Fig. 5b). The glassy dynamics
of the magnetic susceptibility, with saturation on a characteristic time
scale, is typical for disordered systems. Note that for this measurement the
temperature was firmly stabilized at 2 K, hence ruling out potential
temperature fluctuations as the origin of the observed time dependence.

Figure 5 displays the magnetic susceptibility of a crystal from batch 3,
corresponding to the highest realized Cu concentrations. Notably, for this
sample the amplitude of the paramagnetic upturn at low temperature exceeds the
diamagnetic response (Fig. 5a). Moreover, we find an enhanced ferromagnetic
response for small applied fields (not shown here). Again we highlight the
small value of this ferromagnetic part. A possibile scenario is that both the
Curie tail and the ferromagnetic (or canted-antiferromagnetic) response are
due to the Cu substituents, as expected since this is the only d-electron
species in the crystal structure. According to electronic-structure
calculations, Cu is divalent and therefore indeed carries a nonzero spin
magnetic moment.[20]

Figure 5: a) Temperature dependence of the magnetic susceptibility in SI units
of a crystal of batch 3 with the highest Cu content at 7 T in field cooled
(FC) and zero field cooled (ZFC) mode only utilizing the standard MPMS 3. b)
Time dependence of the magnetic susceptibility of the same crystal measured at
a stable temperature of 2 K under a field of 7 T. c) Magnetization versus
field in milli $\mu_{B}$ at 300 K and 800 K of the data shown in Fig. 4b) with
a linear diamagnetic background subtraction.

In Figure 5c we have subtracted the linear diamagnetic contribution, thus
highlighting the ferromagnetic contribution versus the applied field. The
saturated magnetization amounts to only about 0.001 $\mu_{B}$ per Cu, and is
therefore much smaller than 1 $\mu$B, the full spin-1/2 moment of Cu2+. We
note, however, that partial Cu substitution in Pb9Cu(PO4)6O is expected to
generate a complex (and likely highly disordered) network of Cu-O-Cu bonds,
which may elicit strong antiferromagnetic exchange interactions (Fig. 2c,d).
Canting of the Cu moments due to magnetic frustration and/or local inversion
symmetry breaking can then result in canted antiferromagnetism at high
temperatures, as observed for instance in nanocrystalline CuO (see Ref. 21 and
references therein). The connectivity of this network (and hence the amplitude
of the net magnetic moment) may depend on the preparation conditions and might
be enhanced at grain boundaries in polycrystalline samples. Regardless of its
origin, the weakly ferromagnetic response, perhaps in conjunction with the
diamagnetic background, may well be at the heart of the partial levitation of
Pb9Cu(PO4)6O above a permanent magnet at room temperature [1, 2, 9, 7].

##  IV Summary

In summary, we have successfully grown single crystals of Pb10-xCux(PO4)6O
(LK-99) via the TSFZ method. The phase-pure crystals are highly insulating,
suggesting that previously reported transport anomalies in powder samples,
such as a insulator-to-metal transition that might have been mistakenly
interpreted as a superconducting transition, are likely attributable to Cu2S
impurity phases[22]. The diamagnetic response of our crystals is broadly
consistent with expectations for a non-magnetic insulator, although we also
detect weakly ferromagnetic correlations, presumably originating from an
inhomogeneous distribution of the Cu substituents. No anomalies indicative of
phase transitions are detected for temperatures up to 800 K. These results
suggest that the previously claimed occurrence of room-temperature
superconductivity in LK-99 is highly unlikely.

## References

## References

  * Lee, Kim, and Kwon [2023] S. Lee, J.-H. Kim, and Y.-W. Kwon, “The first room-temperature ambient-pressure superconductor,”  (2023), 10.48550/ARXIV.2307.12008, arXiv:2307.12008 [cond-mat.supr-con] .
  * Lee  _et al._ [2023] S. Lee, J. Kim, H.-T. Kim, S. Im, S. An, and K. H. Auh, “Superconductor pb10-xcux(po4)6o showing levitation at room temperature and atmospheric pressure and mechanism,”  (2023), 10.48550/ARXIV.2307.12037, arXiv:2307.12037 [cond-mat.supr-con] .
  * Kumar, Karn, and Awana [2023] K. Kumar, N. K. Karn, and V. P. S. Awana, “Synthesis of possible room temperature superconductor lk-99:pb9cu(po4)6o,”  (2023), 10.48550/ARXIV.2307.16402, arXiv:2307.16402 [cond-mat.supr-con] .
  * Kumar  _et al._ [2023] K. Kumar, N. K. Karn, Y. Kumar, and V. P. S. Awana, “Absence of superconductivity in lk-99 at ambient conditions,”  (2023), 10.48550/ARXIV.2308.03544, arXiv:2308.03544 [cond-mat.supr-con] .
  * Liu  _et al._ [2023] L. Liu, Z. Meng, X. Wang, H. Chen, Z. Duan, X. Zhou, H. Yan, P. Qin, and Z. Liu, “Semiconducting transport in pb10-xcux(po4)6o sintered from pb2so5 and cu3p,”  (2023), 10.48550/arXiv.2307.16802, 2307.16802 [cond-mat.supr-con] .
  * Hou  _et al._ [2023] Q. Hou, W. Wei, X. Zhou, Y. Sun, and Z. Shi, “Observation of zero resistance above 100 k in pb10-xcux(po4)6o,”  (2023), 10.48550/arXiv.2308.01192, 2308.01192 [cond-mat.supr-con] .
  * Wu  _et al._ [2023a] H. Wu, L. Yang, B. Xiao, and H. Chang, “Successful growth and room temperature ambient-pressure magnetic levitation of lk-99,”  (2023a), 10.48550/ARXIV.2308.01516, arXiv:2308.01516 [cond-mat.supr-con] .
  * Wu  _et al._ [2023b] H. Wu, L. Yang, J. Yu, G. Zhang, B. Xiao, and H. Chang, “Observation of abnormal resistance-temperature behavior along with diamagnetic transition in pb10-xcux(po4)6o-based composite,”  (2023b), 10.48550/ARXIV.2308.05001, arXiv:2308.05001 [cond-mat.supr-con] .
  * Guo, Li, and Jia [2023] K. Guo, Y. Li, and S. Jia, “Ferromagnetic half levitation of lk-99-like synthetic samples,”  (2023), 10.48550/ARXIV.2308.03110, arXiv:2308.03110 [cond-mat.supr-con] .
  * Zhu  _et al._ [2023a] S. Zhu, W. Wu, Z. Li, and J. Luo, “First order transition in pb10-xcux(po4)6o (0.9<x<1.1) containing cu2s,” (2023a), 2308.04353v1 [cond-mat.supr-con] .
  * Timokhin  _et al._ [2023] I. Timokhin, C. Chen, Q. Yang, and A. Mishchenko, “Synthesis and characterisation of lk-99,”  (2023), 10.48550/ARXIV.2308.03823, arXiv:2308.03823 [cond-mat.supr-con] .
  * Jiang  _et al._ [2023] Y. Jiang, S. B. Lee, J. Herzog-Arbeitman, J. Yu, X. Feng, H. Hu, D. Calugaru, P. S. Brodale, E. L. Gormley, M. G. Vergniory, C. Felser, S. Blanco-Canosa, C. H. Hendon, L. M. Schoop, and B. A. Bernevig, “Pb10-xcux(po4)6(oh)2: Phonon bands, localized flat band magnetism, models, and chemical analysis,”  (2023), 10.48550/arXiv.2308.05143, arXiv:2308.05143 [cond-mat.supr-con] .
  * Abdullaev  _et al._ [1968] G. B. Abdullaev, Z. A. Aliyarova, E. H. Zamanova, and G. A. Asadov, “Investigation of the electric properties of cu2s single crystals,” physica status solidi (b) 26, 65–68 (1968).
  * Merker and Wondratschek [1960] L. Merker and H. Wondratschek, “Der oxypromorphit pb10(po4)6o und der ausschnitt pb4p2o9-pb3(po4)2 des systems pbo-p2o5,” Zeitschrift für anorganische und allgemeine Chemie 306, 25–29 (1960).
  * Soma, Yoshimatsu, and Ohtomo [2020] T. Soma, K. Yoshimatsu, and A. Ohtomo, “p-type transparent superconductivity in a layered oxide,” Science Advances 6 (2020), 10.1126/sciadv.abb8570.
  * Ohsawa  _et al._ [2020] T. Ohsawa, N. Yamada, A. Kumatani, Y. Takagi, T. Suzuki, R. Shimizu, S. Shiraki, T. Nojima, and T. Hitosugi, “Origin of optical transparency in a transparent superconductor liti2o4,” ACS Applied Electronic Materials 2, 517–522 (2020).
  * Batson  _et al._ [2023] E. Batson, M. Colangelo, J. Simonaitis, E. Gebremeskel, O. Medeiros, M. Saravanapavanantham, V. Bulovic, P. D. Keathley, and K. K. Berggren, “Reduced ITO for transparent superconducting electronics,” Superconductor Science and Technology 36, 055009 (2023).
  * Charnukha  _et al._ [2012a] A. Charnukha, J. Deisenhofer, D. Pröpper, M. Schmidt, Z. Wang, Y. Goncharov, A. N. Yaresko, V. Tsurkan, B. Keimer, A. Loidl, and A. V. Boris, “Optical conductivity of superconducting rb2fe4se5 single crystals,” Physical Review B 85, 100504 (2012a).
  * Charnukha  _et al._ [2012b] A. Charnukha, A. Cvitkovic, T. Prokscha, D. Pröpper, N. Ocelic, A. Suter, Z. Salman, E. Morenzoni, J. Deisenhofer, V. Tsurkan, A. Loidl, B. Keimer, and A. V. Boris, “Nanoscale layering of antiferromagnetic and superconducting phases in rb2fe4se5 single crystals,” Physical Review Letters 109, 017003 (2012b).
  * Si and Held [2023] L. Si and K. Held, “Electronic structure of the putative room-temperature superconductor pb9cu(po4)6o,”  (2023), 10.48550/ARXIV.2308.00676, arXiv:2308.00676 [cond-mat.supr-con] .
  * Batsaikhan  _et al._ [2020] E. Batsaikhan, C.-H. Lee, H. Hsu, C.-M. Wu, J.-C. Peng, M.-H. Ma, S. Deleg, and W.-H. Li, “Largely enhanced ferromagnetism in bare CuO nanoparticles by a small size effect,” ACS Omega 5, 3849–3856 (2020).
  * Zhu  _et al._ [2023b] S. Zhu, W. Wu, Z. Li, and J. Luo, “First order transition in pb10-xcux(po4)6o ($0.9<x<1.1$) containing cu2s,”  (2023b), 10.48550/ARXIV.2308.04353, arXiv:2308.04353 [cond-mat.supr-con] .