1318 Research Leo Yu-Ho Lo and Huamin Qu are with the Hong Kong University of
Science and Technology, Hong Kong. E-mail: {yhload@cse, <EMAIL_ADDRESS>
# How Good (Or Bad) Are LLMs at Detecting Misleading Visualizations?

Leo Yu-Ho Lo0000-0002-3660-3765 and Huamin Qu0000-0002-3344-9694

###### Abstract

In this study, we address the growing issue of misleading charts, a prevalent
problem that undermines the integrity of information dissemination. Misleading
charts can distort the viewer’s perception of data, leading to
misinterpretations and decisions based on false information. The development
of effective automatic detection methods for misleading charts is an urgent
field of research. The recent advancement of multimodal Large Language Models
(LLMs) has introduced a promising direction for addressing this challenge. We
explored the capabilities of these models in analyzing complex charts and
assessing the impact of different prompting strategies on the models’
analyses. We utilized a dataset of misleading charts collected from the
internet by prior research and crafted nine distinct prompts, ranging from
simple to complex, to test the ability of four different multimodal LLMs in
detecting over 21 different chart issues. Through three experiments–from
initial exploration to detailed analysis–we progressively gained insights into
how to effectively prompt LLMs to identify misleading charts and developed
strategies to address the scalability challenges encountered as we expanded
our detection range from the initial five issues to 21 issues in the final
experiment. Our findings reveal that multimodal LLMs possess a strong
capability for chart comprehension and critical thinking in data
interpretation. There is significant potential in employing multimodal LLMs to
counter misleading information by supporting critical thinking and enhancing
visualization literacy. This study demonstrates their applicabilitythe
applicability of LLMs in addressing the pressing concern of misleading charts.

###### keywords:

Deceptive Visualization, Large Language Models, Prompt Engineering

Introduction

Misleading visualizations have consistentlylong been identified and brought
intoincluded in public discourse. In the 1950s, Huff published How to Lie with
Statistics [15], a book that surfaced the deceptive nature of poorly
constructed charts with examples from newspapers. These charts manipulated the
data’s visual representation to appear supportive of their intended claims.
Recognizing these discrepancies is crucial when harnessing the power of data
visualizations to inform rather than mislead. Education and critical scrutiny
remain the most effective tools for identifying misleading visualizations [8,
3]. Conversely, developing automated tools to detect misleading charts
represents a promising area of research.

Recent advancements in the automatic detection of misleading visualizations
have largely centered around visualization linters [7, 19, 27]. These tools
scrutinize the structural programming of charts, identifying violations of
established visualization guidelines. Designed to integrate with visualization
authoring tools, they alert creators to the potential for misleading content
in their charts before publication. Essentially, these innovations serve as a
safeguard on the producer side. However, the challenge becomes more complex
for data consumers, who frequently encounter visualizations in the form of
unstructured bitmap images, often embellished with diverse styles and
annotations. This variety poses significant challenges for automated systems,
leading to a critical gap: the lack of tools to assist or safeguard the
everyday consumers of data visualizations, mainly the general public. Bridging
this gap is crucial, and we urgently need tools to enable data consumers to
navigate and interpret data visualizations accurately.

The recent development of Large Language Models (LLMs) [1] has opened up
massive new opportunities, making it possible to tackle complex problems that
were once impossible for computer algorithms to solve. Previous studies have
already showcased the remarkable abilities of LLMs to reason logically and
interpret data [11]. While these models were originally designed for
processing and generating text, the introduction of multimodal LLMs has been a
pivotal advancement [30] for visualization research. These multimodal models
can understand different types of input, including images, significantly
broadening the ways they can be used. This leap forwardadvancement in LLM
technologyLLMs offers a promising approach to a crucial issue: detecting
misleading charts from a consumer’s point of view. The capabilities of
multimodal LLMs open up the possibility of creating tools that help people who
use data visualizations to better navigate and comprehend visual information,
filling a vital need in our digital world.

Do multimodal LLMs possess the nuanced understanding required to identify and
flag misleading elements in data visualizations? To explore this question, our
study undertook a thorough assessment of three proprietary [30, 29, 35] and
one open-source multimodal LLM [22]. Given that the effectiveness of LLMs is
significantly shaped by the textual prompts fed into them, our initial step
was to conduct an exploratory experiment. This involved crafting three sets of
prompts designed to guide the LLMs in recognizing five specific issues within
charts. Pushing the boundaries further, our investigation expanded to assess
the LLMs’ capabilities in handling an increasing complexity of problems,
conducting further experiments that presented the models with charts
containing 10 and then 21 different issues in charts.

One of the challenges we faced was the growing difficulty in scaling the
number of issues for detection by the LLMs as we broadened the range of issues
to be detected. As we expanded the scope, it was inevitablewe inevitably had
to increase the level of detail of descriptions and instructions, leading to
longer prompts and responses. This escalation resulted in an increase in the
length of both the prompts and the generated responses, presenting a
significant challenge. Despite advancements that have expanded the context
length LLMs can handle, the processing and generation of lengthy texts remains
a computationally intensive and memory-demanding task. Utilizing the insights
gained from our exploratory experiments, our ninth and final prompt was
designed to guide the LLMs in detecting 21 different issues. Instead of
designing a single prompt, our final attempt was to generate prompts
dynamically, facilitated within a multi-round conversation setup.

Our evaluation revealed the exceptional ability of multimodal LLMs to
interpret charts presented as bitmap images. Following our instructions, these
models demonstrated the capabilities of recognizing different chart elements,
exercising critical thinking in data interpretation, and detecting a wide
range of issues in misleading charts. Interestingly, LLMs consistently sought
additional context for the charts, showcasing an innate caution that proved
instrumental in uncovering issues like dubious data sources and concealed
information. Their proficiency in detecting charts with fabricated data was
particularly impressive–a challenge that goes beyond structural analysis to
require a critical evaluation of the charts’ textual content. These findings
showed multimodal LLMs’ sophisticated visual understanding and analytical
capabilities, unveiling their potential as a powerful resource in creating
effective systems for detecting misleading charts.

In a short summary, we aim to provide a glimpse of the potential of
applyingutilizing multimodal LLMs to detect misleading visualizations.
Throughout the study, we share the following results:

  1. 1.

Three experiments with a total of nine prompts across different prompting
strategies coveredcovering up to 21 charting issues.

  2. 2.

Evaluation of the prompts on four different multimodal LLMs from proprietary
companies and the open-source community.

  3. 3.

Findings on the challenges of applying LLMs onto detecting problematic charts
and the strengths and weaknesses of LLMs on similar applications.

All the materials used in the experiments are publicly available on
OSF111https://osf.io/vx526. These include (1) datasets, (2) prompts, (3)
program code, (4) conversation logs, and (5) experiment results.

##  1 Related Work

This work intersects four areas of research: (1) Misleading Visualizations,
(2) Visualization Linters, (3) Chart Analysis with Computer Vision, and (4)
Chart Question Answering with LLMs.

###  1.1 Misleading Visualizations

The discourse on misleading visualizations dates back to the pre-digital era,
notably beginning with Darrell Huff’s seminal work, How to Lie with Statistics
[15], in the 1950s. This bookIn this book, Huff exposed the manipulative
potential of data in journalism. This early examination laid the groundwork
for Tufte’s seminal contributions in the 1980s with his work, The Visual
Display of Quantitative Information. Subsequently, in the book The Visual
Display of Quantitative Information [36], Tufte introduced essential concepts
like Graphical Integrity and Lie Factors, advancing the conversationdiscussion
on how visual data representations can distort the truth. More recently,
Cairo’s How Charts Lie [6] offers a contemporary viewpoint on the issue,
exploring the intricacies of recognizing dubious data sources and the
malicious intentions behind visual designs, particularly in the digital media
landscape. These publications provide a comprehensive foundation that informs
our understanding of the complexities and ethical issues in data
visualization.

Recent academic research has significantly deepened our understanding of
misleading visualizations, especially in today’s digital era, where
misinformation can quickly proliferate online. Researchers like Pandey et
al.[32] and Correll et al.[9] have revealed the subtle ways visualizations can
be manipulated, –for example, by truncating axes, –to drastically alter data
interpretation. This expanding body of research emphasizes the urgent need for
vigilance in data presentation and perception. Further studies by Lee et
al.[18] and Lisnic et al.[20] investigate how visualizations contribute to the
spread of misinformation on social media and other platforms, underlining the
extensive consequences of these deceptive techniques. In an effort to
construct a taxonomy onof these deceptions, Lo et al.[24] compiled a wide
range of misleading visualizations from the internet, creating a detailed
taxonomy with 74 unique issues. These academic contributions build upon the
discussions initiated by early literature and highlight the continuous
challenge and significance of devising methods to counter misinformation in
data visualization. These works offer crucial insights into the mechanics of
deceit and the essential role of integrity in data’s visual representation.

Data visualization literacy and critical data thinking is an important topic
in education. Chevalier et al.have highlighted the significance of embedding
critical thinking within visualization literacy education, advocating for its
introduction as early as elementary education [8]. This sentiment is echoed
and expanded upon at the university level by Bergstrom and West. In their
semester-long course, they instructed students to critically assess the data
they encountered, especially through data visualizations. Their work has
culminated in a comprehensive book on the subject [4]. In today’s data-driven
world, the ability to critically evaluate and interpret information become an
essential skill. Data plays a crucial role in decision-making, influencing
policies, and shaping public opinion, highlighting the critical need for
individuals to possess the skills to scrutinize data with skepticism and
insight. It’s vital to embed critical data thinking within educational
curricula and to develop and sharpen tools that enable the public to
effectively engage with data visualizations. Such tools are essential in
cultivating an informed society that approaches data visualizations with a
critical eye, promoting a more nuanced and discerning consumption of
information in our data-intensive reality.

###  1.2 Visualization Linters

In the field of data visualization, significant progress has been made in
creating automated systems that are designed to help chart creators produce
visualizations that are clear and not misleading. Inspired by the concept of
linters in computer programming—which detect potential errors in code to alert
programmers about possible issues leading to incorrect or inefficient
outcomes—these visualization linters are a step forward in ensuring the
integrity of visual data representations. McNutt and Kindlmann introduced a
linter for matplotlib, a popular charting library, implementing rules derived
from the Algebraic Visualization Design (AVD) framework to improve the clarity
and effectiveness of charts [27]. Similarly, VizLinter [7] uses Answer Set
Programming (ASP) to analyze charts created with Vega-Lite, another popular
charting library, by scrutinizing chart specifications with established best
practices. Expanding the scope, GeoLinter [19] applies cartographic principles
to map visualizations in Vega-Lite, ensuring that they are not misleading from
a cartographic perspective. These tools highlightedhighlight the research
focus on the production side of data visualization, emphasizing tools that
integrate seamlessly with visualization libraries to support chart creators.

While existing research on automated detection systems has largely focused on
aiding chart creators in the production process, there is a growing emphasis
on empowering consumers to critically evaluate the accuracy and reliability of
visual data presentations. Toward this end, Fan et al.have pioneered a system
specifically for analyzing line charts in bitmap format [12]. This system
begins with reverse engineering [33] the chart to extract visualization
specifications from its bitmap image. It then compares these specifications
against established best practices, alerting users to any discrepancies or
violations directly on the chart image with overlays highlighting misleading
aspects alongside corrected versions or annotations, facilitating a more
informed interpretation of the data. Similarly, Hopkins et al.have introduced
a visual interface that flags potential issues within charts, directing users’
attention to areas that might lead to misinterpretation [13]. Lo et
al.analyzed and evaluated six different explanation strategies in explaining
common charting issues to the general public [23]. These innovations are
crucial infor improving data literacy among visual data consumers, yet their
success heavily relies on the accuracy of chart specification extraction. Even
slight errors at this stage can compromise the system’s effectiveness,
emphasizing the importance of precision in chart analysis and visualization
reverse engineering.

###  1.3 Chart Analysis with Computer Vision

Chart analysis is a fast-developing area within computer vision research
focused on extracting data and facilitating question-answering through visual
representations. An initial effort in this field is the creation of the
FigureQA dataset, which includes over 100,000 chart images and more than one
million binary yes or noyes-or-no question-answer pairs, serving as a critical
benchmark for evaluating progress in chart analysis [17]. This initiative has
encouraged further research, with projects expanding the dataset through
synthesis or compilation of real-world charts, enriching the resources for
development. Highlighting the community’s dedication to chart analysis, the
annual Competition on Harvesting Raw Tables from Infographics [10], initiated
by Davila et al., invites participants to complete seven complex chart
analysis tasks. Together, these tasks form a pipeline that starts fromwith
identifying chart types from the chart image, then detecting text, classifying
text roles, analyzing axes and legends, identifying plot elements, and finally
extracting data. The competition, with its annotated dataset of over 17,000
synthetic and 16,000 scientific journal-sourced chart images. Such efforts
continue to extend, continues to push the limits of computer vision in
visualization reverse engineering, improving tools and techniques for machines
to understand the vast information in data visualizations.

The chartChart analysis research has rapidly expanded its scope to include
sophisticated question-answering setups, moving from simple binary questions
to complex open-ended queries that requiredrequire mathematical reasoning
[14]. This shift marks a substantial advancement in the ability to interpret
nuanced data. During the progressresearch, numerous datasets have beenwere
introduced for benchmarking [26, 16, 28]. A common approach to this challenge
is treating chart question-answering similarly to table question-answering by
extracting the underlying data from charts. This strategy utilizes advanced
techniques developed for table question -answering [26].

###  1.4 Chart Question Answering with LLMs

The integration of LLMs has further propelled these advancements, with
initiatives like Matcha [21] applying LLMs on code generation to transform
chart images back into programmaticprogram code and associated data tables for
further processing. Similarly, UniChart’s [25] encoder-decoder framework
demonstrates a versatile approach to chart question-answering, potentially
bypassing the need for data table generation. These developments highlight a
significant leap towards bridging visual data representations with actionable
insights, showcasing the field’s progress towards more intuitive and effective
data extraction and interpretation methods, thereby enhancing visualization
tools for academic research and practical applications.

Effective prompting is the key to successfully integrating LLMs into chart
analysis. An advancement in this area was introduced by Marsh et al.[11]. The
authors developed a prompt constructor module that effectively bridges the gap
between the complex requirements of chart question-answering tasks and the
operational capabilities of LLMs. This module translates these tasks into
prompts that enable LLMs to generate relevant and precise outputs. With this
strategic prompting mechanism, their system has achieved state-of-the-art
results in the field.

Our study builds on this foundation, utilizing multimodal LLMs to identify
misleading elements in visual data presentations, a step forward in refining
visual data analysis methods and exploring LLMs’ potential in mitigating
misinformation spread through data visualizations.

Table 1: Experiment One Results: Accuracy (Acc.) represents the ratio of
correct answers to the total questions asked. The F1-score (F1) is the
harmonic mean of Precision (Prec.) and Recall (Rec.). Across prompts, the
models showed higher recall and lower precision, suggesting a tendency toward
false positive In Prompt #2, Copilot chose 3 on a 5-point Likert scale in 78%
of cases, with accurate scores in the remaining six cases (three misleading
and three valid). Relevance (Rel.) is the percentage of relevant responses. An
LLM’s response is considered as relevant if the chart issue is mentioned in
the response through keyword matching.

##  2 Methodology

Our experiments aim to explore the capabilities of multimodal LLMs in
identifying misleading visualizations and distinguishing thesethem from more
specialized machine learning models. LLMs are known for their impressive
ability to generalize and adaptingadapt to a wide range of tasks beyond their
initial programming through methods like zero-shot, one-shot, or few-shot
learning [2]. The ability of multimodal LLMs in chart understanding,
critically assessing the deceptiveness of these visualizations, and the
potential of applying multimodal LLMs in detecting misleading charts are
unexplored. This investigation seeks to determine if such capacities exist
and, if so, how crafting prompts alongside the chart images can effectively
direct LLMs to produce the intended analytical outcomes.

We began the investigation with an exploratory experiment, collecting various
visualizations, including misleading and valid charts. The objective of the
exploration phase is to establish a baseline for evaluating LLMs’ ability to
distinguish between these two types of charts. Building upon the insights
gained in the exploratory experiment, the second phase of the experiments
sharpens the focus on enhancing and broadening the prompts to capture a wider
range of chart issues. However, this expansion poses scalability challenges,
particularly in managing the length of the prompts and also the length of the
outputs from the LLMs, leading to degraded performance. To address these
issues, the third phase investigates strategies to overcome these limitations,
facilitating a more exhaustive examination of the 21 most common issues in
charts. Each stage progressively deepens our understanding of multimodal LLMs’
analytical capabilities in detecting visual deceptions.

Multimodal LLMs: Our experimental strategy encompassed a comprehensive
evaluation ofexperiments covered four distinct multimodal LLMs: (1) ChatGPT by
OpenAI (gpt-4-1106-vision-preview) [30], (2) Copilot by Microsoft (model name
not specified) [29], (3) Gemini by Google (gemini-pro-vision) [35], and (4)
LLaVA-NeXT (llava-v1.6-mistral-7b-hf) [22]. ChatGPT, Copilot, and Gemini are
developed by proprietary companies, whereas LLaVA-NeXT is an open-source
multimodal LLM. Notably, at the time of our experiments, LLaVA-NeXT is
holdinghas achieved state-of-the-art results across various multimodal LLM
benchmarks among open-source LLMs. Due to hardware constraints, our
experiments were conducted with the 7B parameter variant of LLaVA-NeXT,
despite the more advanced state-of-the-art 34B parameter model. Nevertheless,
including LLaVA-NeXt in our evaluation covers an important landscape of LLM
research. The LLaVA-NeXt model was run on a server with an Nvidia Titan RTX
24GB graphical processing unit. On the other hand, the proprietary models were
evaluated via their Application Programming Interfaces (APIs). We have made
the program code of the experiments publicly available on the OSF1.

Dataset: Our study compiled an evaluation dataset from previous research on
misleading charts circulated on the internet [24]. The dataset was collected
through search engines and social media. The chart images are annotated with
the issues identified from the original web page or social media post. A total
of 74 unique chart issues were identified in the study, providing a diverse
sample of misleading charts that the general public may encounter.

In the initial phase of our exploratory study, we focused on the five most
frequently identified issues: (1) Truncated Axis, (2) 3D Chart, (3) Missing
Title, (4) Dual Axis, and (5) Misrepresentation. We subsequently expanded to
include the ten most common issues in eExperiment tTwo by including (6)
Missing Axis Title, (7) Missing Legend, (8) Inconsistent Tick Intervals, (9)
Not Data, and (10) Selective Data. In the third experiment, the study was
further broadened to cover up to 21 issues due to a tie for the 20th spot,
with both the 20th and 21st issues appearing an equal number of 25 times in
the dataset. These additional issues included (11) Dubious Data, (12) Missing
Value Labels, (13) Area Encoding, (14) Overusing Colors, (15) Inappropriate
Axis Range, (16) Indistinguishable Colors, (17) Ineffective Color Scheme, (18)
Discretized Continuous Variable, (19) Missing Normalization, (20) Missing
Axis, and (21) Inconsistent Binning Size. For each issue, we selectedrandomly
sampled six images, dividing them equally into development and test sets,
resulting in sets for evaluation across five (N=30), ten (N=60), and 21 issues
(N=126).

Moreover, we included a set of valid charts to evaluate if LLMs can
distinguish between misleading and accurate representations. This valid set,
randomly sampled from a collection by Brokin et al.[5], consisted of chart
images from various sources, including news and governmental organizations.
Through stratified sampling from this pool of 2,000 chart images–excluding
infographics and charts with issues like missing titles or 3D effects–we
assembled a valid chart set of 24 images. This set was evenly sampled from
four sources, each contributing six images, and covered six different chart
types, each represented by four images. This sampling method ensured wide
coverage of sources and chart types, minimizing bias towards particular
charting styles, data types, or chart types. Unlike the misleading chart set,
which was expanded throughout the progression of our experiments, the valid
chart set remained the same throughout the study. The valid set is divided
evenly into development and test sets while maintaining its stratified
property.

For each of our experiments, we organized the datasets as follows: (1) for
Experiment One, we had a total of 54 images ($N=54$), with 27 in the
development set ($N_{dev}=27$) and 27 in the test set ($N_{test}=27$); (2) for
Experiment Two, the total was 84 images ($N=84$), split equally into 42 for
development ($N_{dev}=42$) and 42 for testing ($N_{test}=42$); and (3) for
Experiment Three, we worked with 150 images ($N=150$), divided into 75 for
development ($N_{dev}=75$) and 75 for testing ($N_{test}=75$). Unlike
traditional machine learning models, our study lacked a training phase, hence
the development set was used for creating prompts, and the test set was
strictly for evaluation purposes.

Evaluation Metrics: Throughout the various stages of our experiments, as we
gained additional insights into the behavior of the LLMs, we fine-tuned our
evaluation metrics accordingly. In the initial exploratory study, we employed
accuracy and F1-score as indicators of classification performance, alongside
keyword matching on the reported issues, to assess the relevance of the
generated outputs. From pPrompt #3 of Experiment One and after, we discovered
that the LLMs consistently followed our naming of each issue. Therefore, exact
matching is used in these experiments, and keyword matching is only used for
Prompt #1 and Prompt #2 of Experiment One. If the chart issue is mentioned in
the response, we consider the response as relevant, and vice versa.

In order to process the LLM outputs, after each prompt, we feedfed the prompt
and output as a conversation dialogue into the LLM and ask it to summarize the
dialogue in JSON format. The reported results are based on this JSON instead
of the free-formed responses.

As we progressed to Experiment Two and Experiment Three, expanding the
misleading charts set led to an imbalance between the number of misleading and
valid charts. We recognized that accuracy was inadequate as a measure of
performance. Consequently, we rely on the F1-score in Experiment Two and
Experiment Three. Additionally, beginning with the third prompt, we introduced
factual questions about the charts into our queries to the LLMs. This enabled
us to directly measure the LLMs’ capabilities in chart comprehension.
Therefore, we labeled these chart properties manually on all the 150 chart
images and included factual correctness as a metric for evaluation in
Experiment Two and Experiment Three.

Overall, we have conducted three experiments, each with a progressively larger
set of issues to cover and posed different sets of challenges. We employed
different prompting strategies in different stages to effectively instruct the
LLMs to detect misleading visualizations.

##  3 Experiment One: Exploration Phase

Figure 1: Experiment One Prompts: Prompt #1 directly asks if the chart is
misleading or truthful. Prompt #2 requests a rating on a 5-point Likert scale,
with 3 being neutral. Prompt #3 applies the Chain of Thoughts strategy for a
structured reasoning process with LLMs. The example chart has a major issue of
truncated axis and a minor issue of 3D effects on the bars. Phrases in blue
denote accurate interpretations, and phrases in red indicate incorrect
interpretations.

Our understanding of how to prompt multimodal LLMs to identify misleading
charts—a task they have not previously encountered—is limited. Thus, we
beginbegan our exploration with a straightforward prompt: “Does this chart
depict data truthfully or misleadingly?”. Through this approach, we aim to
incrementally gain insights into effectively interacting with LLMs for the
purpose of developing an automated system capable of detecting misleading
visualizations.

Table 2: Experiment Two Results: The F1-score (F1), precision (Prec.), and
recall (Rec.) assess the LLMs’ performance in an imbalanced setting
($N_{misleading}=30,N_{valid}=12$). Compared to Experiment One, the high-
recall, low-precision situation has been improved but we are still observing
large percentage of false positives. Relevance (Rel.) is the percentage of
relevant responses. An LLM’s response is considered as relevant if the chart
issue is reported as one of the pitfalls. Factual Correctness (Fact.) measures
the percentage of correct answers to the factual questions about the chart,
e.g., chart type, title, axis, scale, and encoding.

To initiate this exploration, we startstarted with a small set of misleading
($N_{misleading}=30$) and valid ($N_{valid}=24$) charts divided equally into a
development set and a test set. To analyze the results objectively, without
relying on our interpretation of the text generated by the LLMs, we re-input
the dialogues generated in response to each prompt back into the LLMs,
requesting that theythe LLMs to summarize the dialogue in JSON format. This
method allows for a structured and unbiased analysis of the LLMs’ responses.

In this preliminary experiment, we tested three types of prompts: (1) direct
inquiry, (2) Likert scale assessment, and (3) Chain of Thought (CoT) reasoning
[38]. We evaluated the outcomes based on accuracy, F1-score, and relevance of
the responses generated by the models.

Prompt #1 (Direct Ask): The initial prompt we experimented with was directly
asking: “Does this chart depict data truthfully or misleadingly?” When applied
through web user interfaces, the responses often elaborated on the
characteristics of misleading visualizations and provided a checklist rather
than offering a direct “truthful” or “misleading” answer. To guide the
responses more effectively, we appended the instruction with “Answer
‘truthful’ or ‘misleading’. Name the issues and explain your answer.”
following the initial question. After receiving each response, we formatted
the question and response into a dialogue and requested that the LLMs
summarize this dialogue in JSON format. An illustration of various prompts and
responses from different LLMs is presented in Fig. 1.

Results of Prompt #1: In the test set comprising 27 responses, most LLMs
labeled the charts as “misleading,” resulting in high recalls and low
precisions. ChatGPT showingshowed a marginally better capability by correctly
identifying three out of 12 valid charts. The issue of misrepresentation was
the most commonly cited problem, with responses noting concerns such as “lack
of scale consistency” and “disproportional representation.” Table 1 shows the
results of Experiment One.

Despite the tendency of LLMs to criticize charts excessively, the results were
encouraging in terms of their ability to identify certain issues, such as the
inappropriate use of 3D effects, dual axes, and truncated axes. On the other
hand, the absence of titles on charts did not seem to be flagged as a problem
by any of the LLM responses. While not directly misleading, the omission of a
title results in a loss of context, rendering the chart less comprehensible.
The concept of “lacking context” was, however, frequently mentioned in the
feedback. Fig. 4 shows the summary of the issues recognized by the LLMs across
different prompts.

Prompt #2 (Likert Scale): Based on the findings inof pPrompt #1, we considered
introducing more nuance to the responses using a 5-point Likert scale, where
the midpoint represents insufficient information to make a definitive
judgment. This adjustment allows the LLMs to provide answers that aren’t
strictly “truthful” or “misleading.” We grouped their responses into three
categories: scores of 1-21 or 2 as “truthful,” a score of 3 as “neither,” and
scores of 4-54 or 5 as “misleading.” In addition to accuracy and the F1-score,
it became necessary to track the proportion of responses categorized as
“neither.”

Results of Prompt #2: When utilizing this more nuanced prompt, all LLMs
exhibited a decline in performance compared to the initial prompt, with the
exception of Copilot, which achieved a perfect F1-score. Copilot’s responses
were particularly notable, with 21 out of 27 categorized as “neither,”
alongside three correctly identified as misleading and three as truthful.
Despite this, the relevance of the responses to the specific issues present in
the charts was minimal. Overall, using a Likert scale prompt did not
effectively guide LLMs in pinpointing issues within charts.

Prompt #3 (Chain of Thought): In the third prompt, we implemented the CoT
prompting strategy, which encourages LLMs to undergo a reasoning process
before delivering a definitive answer. This approach is inspired by the
methodology that guides LLMs through a step-by-step analytical process,
culminating in a reasoned conclusion [38].

The prompt was designed to inquire about critical aspects of potential
charting issues, such as the presence of a title, the use of 3D effects, and
whether the axis begins at zero, among others. Subsequently, we presented a
set of guidelines and questioned whether the chart adhered to all these
criteria.

Results of Prompt #3: Despite the F1-scorefor this prompt being similar to
those of the first prompt, there was a notable enhancement in the relevance of
the responses. By providing clear instructions, the LLMs were able to
demonstrate an understanding of the charts and accurately identify issues as
directed. This indicates that, with precise guidance, LLMs can effectively
engage in the task of chart analysis.

Discussion of Experiment One: In this exploratory experiment, we established
an experimental setup to assess LLMs’ abilities in chart comprehension,
critical thinking, and adherence to instructions. Traditional algorithmic
detection systems often yield straightforward yes or noyes-or-no answers.
However, the LLM responses were notably non-binary. For example, in response
to pPrompt #2, “it depends” was a frequent reply. This tendency aligns with
advancements in LLM training, particularly the emphasis on generating human-
like responses through reinforcement learning based on human feedback [31]. A
plausible reason for this pattern is the preference for “it depends” responses
by human annotators over unequivocal but potentially incorrect answers.

We introduced a checklist with issue names to reduce the ambiguity in parsing
responses. For pPrompts #1 and #2, keyword matching was employed to determine
if the responses addressed the identified issues within the charts. By
contrast, in pPrompt #3, we provided a checklist, and the LLMs consistently
used the specified terminology in their responses. This approach significantly
clarified the evaluation of the responses’ relevance, making the results from
pPrompt #3 much clearer than those from the earlier prompts.

For our subsequent experiment, rather than seeking a simple yes or noyes-or-no
verdict, we will instructinstructed the LLMs to categorize identified issues
by their severity, including an option for inconclusive findings.

##  4 Experiment Two: Consolidation and Expansion Phase

Figure 2: Experiment Two Prompts: Prompt #4 poses a checklist for LLMs to go
through. Prompt #5 applies the Chain of Thoughts strategy. Prompt #6 modifies
Prompt #5 to prevent LLMs from bypassing reasoning questions. The example
chart contains fictional data for parody purposes. Phrases in blue denote
accurate interpretations. Portions of the prompt and responses are omitted for
clarity.

In the first experiment, we derived three key insights regarding the efficacy
of prompting LLMs to perform the task of detecting misleading charts: (1)
posing factual questions and providing a checklist isare beneficial, (2) LLMs
tend to avoid giving definitive answers, and (3) LLMs followed our naming of
the issues.

In eExperiment oOne, we found that the third prompt, which utilized the CoT
strategy, was particularly effective in guiding LLMs to pinpoint issues in
charts. While the CoT strategy has proven effective in previous studies [38],
our construction of pPrompt #3 included reasoning steps and a checklist of
potential chart issues. We are interested in distinguishing between the
impacts of the checklist and the CoT strategy. To explore this, pPrompt #5
replicates the approach of pPrompt #3, and pPrompt #4 employs only a
checklist, omitting the CoT reasoning steps.

As the experiment progressed to include more issues, the prompts’ lengthlength
of prompts, incorporating extensive issue definitions and related factual
questions, hinted at potential scalability challenges. Particularly, pPrompt
#5 revealed that posing numerous questions within a single prompt might lead
LLMs to overlook earlier questions, focusing instead on the final checklist
question only. To address this, pPrompt #6 adopts a multi-pass conversation
approach, dividing the extensive pPrompt #5 into smaller prompts within a
continuous dialogue format. This forces the LLMs to answer all the factual
questions and reasoning steps before jumping to the checklist question to give
the final answer.

In the second experiment, we expanded the scope to include the ten most common
issues, thereby enlarging the dataset of misleading charts. The dataset for
this phase comprised 60 misleading charts with the same set of 24 valid charts
in the first experiment. Both sets are evenly divided into development and
testing sets.

Given the dataset’s imbalance between misleading and valid charts, we
determined that accuracy was inadequate and opted to rely on the F1-score
instead. In pPrompts #5 and #6, the LLMs were prompted to include the final
judgment of chart issues and the answers to the factual questions. Therefore,
for pPrompts #5 and #6, we also evaluated the accuracy of responses to factual
questions. Fig. 2 shows the prompts evaluated in Experiment Two.

Prompt #4 (Checklist Only): We instructed LLMs to go through a checklist and
categorize identified issues into three levels of concern: major, minor, and
either potential or inconclusive pitfalls.

Prompt #5 (Chain of Thought): The prompt was structured on top of the
checklist in Prompt #4 in a tiered manner, divided into three segments. The
first segment posed basic inquiries regarding the chart’s structure, including
the type of chart, presence of 3D effects, axis, legends, and data source. The
second tier involved more complex questions necessitating interpretation, such
as determining the data type represented on the axis, the range of the axis,
and whether the axis ticks were evenly spaced. Given the observed ability of
LLMs to contextualize their analysis based on chart content, we also
inquiredinvestigated whether the information providedpresented in the chart
alone was sufficient for understanding, or if additional details, like a such
as a chart title or axis titles, were necessary. The final set of questions,
addressing the data directly, was the most intricate, probing whether the data
might have been selectively chosen or fabricated.

Prompt #6 (Split Chain of Thought): During the execution of pPrompt #5, we
noted instances where the LLMs’ generated responses either reached the output
limit or the models bypassed the initial two sets of questions, opting to
respond directly to the final checklist. To address this, we segmented the
prompt into three parts, presenting them to the LLMs in successive rounds
while incorporating the preceding dialogue. This method allowed for a more
manageable and structured approach to eliciting detailed analyses from the
LLMs.

Experiment Two Results and Discussion In categorizing the responses, we
classified charts as “misleading” if they presented at least one major issue;
otherwise, they were deemed valid. The outcomes of eExperiment tTwo are
summarized in Table 2. While the three prompts yielded a similar F1-score, the
quantity and relevance of the issues identified varied across different
prompts. Specifically, LLMs reported a greater number of irrelevant issues
when responding to the checklist in pPrompt #4. In contrast, when engaging
with the factual questions in pPrompts #5 and #6, the incidence of reported
issues decreased, yet the mention of relevant issues remained fairly
consistent. Regarding these factual questions, the LLMs demonstrated a
surprising capability of understandingability to understand the charts,
accurately answering questions about recognizing different chart components,
and correctly interpreting scales and encodings. This was achieved without
specific training or one-shot or few-shot demonstrations, relying solely on
the instructions within the prompts, showcasing the LLMs’ impressive capacity
for visual interpretation. Fig. 5 shows the results of the factual questions
answered by the LLMs.

An interesting discovery from this experiment was the LLMs’ ability to
identify charts intended as parodies created with fictitious data rather than
genuine information. Fig. 2 shows an example chartschart that werewas flagged
as “Not Data,” with advisories against considering themit as factual.

During this experiment, it was noted that LLaVA-NeXT experienced issues with
repetitive outputs, echoing certain sentences until they reached the maximum
token count and terminated, leading to outputs that contained considerable
repetitions. This significantly impacted its performance. Similarly, Gemini
indicated errors suggesting it, toothat it faced issues with repetitions, too.
To address these erratic responses, we adjusted the temperature setting of
Gemini, which, as per API documentation, influences the randomness of the
response. In other cases, generating long outputs triggered the maximum token
limitation and prematurely returned incomplete responses. We simply rerunreran
the experiment with the same prompt and image for these cases.

Experiment tTwo highlighted a limitation of LLMs: as the number of issues to
be detected increases, so does the length of the prompts. For LLMs with input
length restrictions, this poses a scalability challenge. Additionally, more
comprehensive prompts lead to longer responses, potentially exceeding the
maximum response length the model can generate. For instance, the ChatGPT API
has a limit of 5000 tokens, while Gemini’s limit is 2048 tokens on the newly
generated text. The complexity of prompts also increases the likelihood of
repetitive output issues.

Addressing the prompt length constraint, we found that while the checklist in
pPrompt #4 was shorter, it also prompted LLMs to report more irrelevant
issues. Conversely, pPrompt #6 demonstrated a successful workaround, yielding
relevant results and improved factual accuracy.

##  5 Experiment Three: Tackling Scalability Issues

Figure 3: Experiment Three Prompt #7 extends Prompt #6 in applying the Chain
of Thought strategy to include additional chart issue definitions. The example
chart has a major issue of setting the y-axis range inappropriately. Phrases
in blue denote accurate interpretations. Portions of the prompt and responses
are omitted for clarity.

In Experiment Two, expanding the scope to cover ten issues revealed challenges
associated with lengthy prompts or responses, leading to premature termination
and omission of reasoning questions. Additionally, despite LLMs correctly
answering the factual questions in Experiment Two, they occasionally made
contradictory decisions in their final assessments, such as noting a missing
title but not recognizing it as an issue in their conclusive response. We
incorporated more interpretive questions within the prompts to refine this
aspect in Experiment Three.

Prompt #7 (Split Chain of Thought): The initial layer consisted of basic
factual inquiries, followed by a second layer examining data types, encoding,
and scales, akin to the approach in pPrompt #6. The third layer required LLMs
to evaluate the chart more critically, considering the necessity of explicit
titles and axes or the suitability of the axis range and color scheme. The
strategy of segmenting prompts into multiple conversational passes proved
effective in Experiment Two. Yet, as we sought to identify a broader range of
21 chart issues in Experiment Three, the length of each segmented prompt also
increased considerably. The resulting full prompt comprises approximately
6,500 characters and totaling 1,550 tokens. Fig. 3 shows an example of Prompt
#7.

Prompt #8 (Direct JSON Output): Directly instructing LLMs to go through a
checklist was deemed ineffective based on findings from Experiment Two. In
Experiment Three, we tried to condense responses while maintaining the
reasoning steps by instructing LLMs to produce a direct issue list in JSON
format.

Prompt #9 (Dynamic Chain of Thought): Another approach involved dynamically
constructing split prompts based on LLM responses, starting with basic
questions about chart properties and tailoring follow-up queries based on
these initial answers. For instance, if an LLM identified a chart as a pie
chart, subsequent questions about axis range or tick consistency were deemed
irrelevant and omitted from the follow-up prompt.

Figure 4: Issues detected by LLMs across different prompts, each issue
appeared three times in the test set. Edge numbers indicate the correct
identification percentage for each row or column. Figure 5: Percentage of
factual questions accurately answered by LLMs on chart properties, axes,
scale, and encoding. Edge numbers indicate the correct answer percentage for
each row or column.

Experiment Three Results and Discussion: Prompt #8 encountered issues similar
to those observed with pPrompt #4, with a tendency to over-report issues in
the charts. Specifically, when using pPrompt #8, despite a high F1-score,
Copilot identified major issues in 74 charts, deeming only one chart as valid,
which was inaccurately classified as a false negative. The F1-score, generally
appropriate for datasets with an imbalance between categories, was utilized
for evaluation. The test dataset for this phase included 63 misleading charts
and 12 valid charts. Other than prompt #8, both prompts #7 and #9 showed more
balanced performance, correctly identifying valid charts (true negatives) and
accurately highlighting issues in misleading charts (true positives). Table 3
shows the results of Experiment Three.

Prompt #9, designed to reduce the number of excessively reported issues by
filtering the checklist based on factual responses provided earlier in the
conversation, reported fewer issues. However, its relevance score decreased
compared to pPrompt #7, which did not implement any filtering mechanism.

LLaVA-NeXT encountered significant difficulties with pPrompts #7 and #9, as
well as with pPrompt #6 from Experiment Two, suggesting it may struggle with
multi-pass prompts. This issue might stem from its smaller model size of 7B
parameters instead of a larger 34B parameter model. Our current hardware
constraints prevent us from testing the larger model, presenting a challenge
for future evaluations of open-source models.

Table 3: Experiment Three Results: The F1-score (F1), precision (Prec.), and
recall (Rec.) assess the LLMs’ performance in an imbalanced setting
($N_{misleading}=63,N_{valid}=12$). Relevance (Rel.) is the percentage of
relevant responses. An LLM’s response is considered as relevant if the chart
issue is reported as one of the pitfalls. Factual Correctness (Fact.) measures
the percentage of correct answers to the factual questions about the chart,
e.g., chart type, title, axis, scale, and encoding. It does not apply to
Prompt #8 since it instructed LLMs to reply only to the summary question.

##  6 Discussion

LLMs demonstrated capabilities in chart comprehension. Given that the LLMs we
assessed were not specifically trained to identify issues in charts, their
performance exceeded our initial expectations. This was particularly notable
in their ability to analyze images (a task generally reserved for computer
vision systems) and their comprehension of textual content within those
images. As shown in Fig. 5, LLMs accurately answered most questions regarding
chart properties such as chart type, data type, data encoding, titles, and the
use of 3D effects. However, they encountered difficulties with questions
related to color counting and tick consistency checking, which is a known
weakness in LLMs [39]. Consequently, as illustrated in Fig. 4, LLMs performed
less effectively on chart issues related to colors or inconsistencies in the
taxonomy [24], including overusing colors, indistinguishable colors,
ineffective color schemes, and inconsistent tick intervals. On the other hand,
it was surprising to see that these models are capable of differentiating
between fictional and factual data, especially since such discernment is
challenging for standardrule-based algorithmic detection systems. This
capability was demonstrated in the identification of “not data” charts, where
LLMs had shown a robust understanding of charts and accurately reported the
clues of fictional data in the images.

Chain of Thought is the most effective prompting strategy in our experiments.
Over the course of the three experiments, we explored various prompting
strategies to optimize LLM performance. The intermediate output aided the
model in building an understanding of complex misleading elements from simple
chart properties. The concept of misrepresentation is challenging due to the
logical dissonance of graphical integrity–the chart’s geometry not reflecting
the textual information on the chart.

The CoT strategy emerged as the most effective, yet as we expanded the range
of issues for detection, the prompts’ length increased significantly. To
address this, we tested three alternative strategies: (1) dividing the prompt
into a conversational format, (2) incorporating reasoning steps without
explicitly including them in the output, and (3) dynamically generating
prompts based on previous responses. Our findings indicated that pPrompts #4
and #8, which did not include reasoning steps in the output, led to over-
reporting issues. Based on our experiments, pPrompt #7 was identified as the
most effective. However, expanding the scope to encompass more issues poses a
challenge, as it would significantly increase the prompt’s length and
complexity, potentially leading to lengthy response times and extensive
outputs.

To address the scalability issue, alternative approaches should be considered.
Leveraging the LLMs’ ability to generate [11] or retrieve prompts [34] from a
prompt pool according to the input chart image and the conversational contexts
could reduce prompt size while maintaining relevance. The prompt pool could be
built from an example dataset synthesized from previous work on taxonomy [24]
and visualization guidelines derived from different visualization linters [7,
27].

Recent developments in prompt engineering include the agentic approach. The
mixture of agents (MoA) approach–utilizing multiple LLMs (with homogeneous or
heterogeneous weights), each taking different roles in a conversation–has
demonstrated improvement over the Chain of Thought approach [37]. Our
experiments demonstrated consistent chart comprehension capabilities across
different LLMs. Applying the MoA and retrieval-based approaches may be
promising next steps.

A benchmark dataset is urgently needed. Our evaluation efforts were limited by
two main factors: (1) the absence of a benchmark dataset and (2) the need for
refined evaluation metrics. Although prior research has addressed the
detection of misleading charts, there lacks a shared dataset and metrics for
benchmarking purposes. Consequently, we compiled our dataset of misleading and
valid charts from existing studies on using visualizations on the internet.
This study serves as a foundation for subsequent research, facilitating
comparison and overcoming the obstacles identified in our three experiments.

##  7 Conclusion

In this research, we explored the application of multimodal LLMs to the
challenge of automatically detecting misleading charts. Through a series of
three experiments, which progressed from an initial set of five types of
issues to an eventual examination of 21 issues, we identified effective
strategies for prompting multimodal LLMs. This progression allowed us to
refine our approach and understand the nuances of utilizing LLMs for this
specific task. Each experiment contributed to a deeper understanding of how
LLMs interpret and analyze charts and data, revealing both the capabilities
and limitations of these models in the context of misleading visualizations.
Our findings unveil the potential and capabilities of LLMs in augmenting
traditional methods for chart analysis and identifying misleading elements in
charts. The drasticsignificant development of LLMs provides a promising
direction for further research in developing tools to support critical
thinking on data interpretation and visualizations.

## References

  * [1] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al.  Gpt-4 technical report.  arXiv preprint arXiv:2303.08774, 2023.
  * [2] I. Beltagy, A. Cohan, R. Logan IV, S. Min, and S. Singh.  Zero-and few-shot nlp with pretrained language models.  In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pp. 32–37, 2022.
  * [3] C. T. Bergstrom and J. West.  Calling bullshit in the age of big data.  Calling Bullshits. Data Reasoning in a Digital World. Available online at: http://callingbullshit.org, 2017.
  * [4] C. T. Bergstrom and J. D. West.  Calling bullshit: The art of skepticism in a data-driven world.  Random House Trade Paperbacks, 2021.
  * [5] M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva, and H. Pfister.  What makes a visualization memorable?  IEEE Transactions on Visualization and Computer Graphics, 19(12):2306–2315, 2013.
  * [6] A. Cairo.  How Charts Lie: Getting Smarter about Visual Information.  WW Norton & Company, 2019.
  * [7] Q. Chen, F. Sun, X. Xu, Z. Chen, J. Wang, and N. Cao.  Vizlinter: A linter and fixer framework for data visualization.  IEEE Transactions on Visualization and Computer Graphics, 28(1):206–216, 2021.
  * [8] F. Chevalier, N. H. Riche, B. Alper, C. Plaisant, J. Boy, and N. Elmqvist.  Observations and reflections on visualization literacy in elementary school.  IEEE Computer Graphics and Applications, 38(3):21–29, 2018.
  * [9] M. Correll, E. Bertini, and S. Franconeri.  Truncating the y-axis: Threat or menace?  In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–12, 2020.
  * [10] K. Davila, B. U. Kota, S. Setlur, V. Govindaraju, C. Tensmeyer, S. Shekhar, and R. Chaudhry.  Icdar 2019 competition on harvesting raw tables from infographics (chart-infographics).  In 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 1594–1599. IEEE, 2019.
  * [11] X. L. Do, M. Hassanpour, A. Masry, P. Kavehzadeh, E. Hoque, and S. Joty.  Do llms work on charts? designing few-shot prompts for chart question answering and summarization.  arXiv preprint arXiv:2312.10610, 2023.
  * [12] A. Fan, Y. Ma, M. Mancenido, and R. Maciejewski.  Annotating line charts for addressing deception.  In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1–12, 2022.
  * [13] A. K. Hopkins, M. Correll, and A. Satyanarayan.  Visualint: Sketchy in situ annotations of chart construction errors.  In Computer Graphics Forum, vol. 39, pp. 219–228. Wiley Online Library, 2020.
  * [14] E. Hoque, P. Kavehzadeh, and A. Masry.  Chart question answering: State of the art and future directions.  In Computer Graphics Forum, vol. 41, pp. 555–572. Wiley Online Library, 2022.
  * [15] D. Huff.  How to lie with statistics.  WW Norton & Company, 1954.
  * [16] K. Kafle, B. Price, S. Cohen, and C. Kanan.  Dvqa: Understanding data visualizations via question answering.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5648–5656, 2018.
  * [17] S. E. Kahou, V. Michalski, A. Atkinson, Á. Kádár, A. Trischler, and Y. Bengio.  Figureqa: An annotated figure dataset for visual reasoning.  arXiv preprint arXiv:1710.07300, 2017.
  * [18] C. Lee, T. Yang, G. D. Inchoco, G. M. Jones, and A. Satyanarayan.  Viral visualizations: How coronavirus skeptics use orthodox data practices to promote unorthodox science online.  In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–18, 2021.
  * [19] F. Lei, A. Fan, A. M. MacEachren, and R. Maciejewski.  Geolinter: A linting framework for choropleth maps.  IEEE Transactions on Visualization and Computer Graphics, 2023.
  * [20] M. Lisnic, C. Polychronis, A. Lex, and M. Kogan.  Misleading beyond visual tricks: How people actually lie with charts.  In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–21, 2023.
  * [21] F. Liu, F. Piccinno, S. Krichene, C. Pang, K. Lee, M. Joshi, Y. Altun, N. Collier, and J. M. Eisenschlos.  Matcha: Enhancing visual language pretraining with math reasoning and chart derendering.  arXiv preprint arXiv:2212.09662, 2022.
  * [22] H. Liu, C. Li, Y. Li, B. Li, Y. Zhang, S. Shen, and Y. J. Lee.  Llava-next: Improved reasoning, ocr, and world knowledge, January 2024.
  * [23] L. Y.-H. Lo, Y. Cao, L. Yang, and H. Qu.  Why change my design: Explaining poorly constructed visualization designs with explorable explanations.  IEEE Transactions on Visualization and Computer Graphics, 2023.
  * [24] L. Y.-H. Lo, A. Gupta, K. Shigyo, A. Wu, E. Bertini, and H. Qu.  Misinformed by visualization: What do we learn from misinformative visualizations?  In Computer Graphics Forum, vol. 41, pp. 515–525. Wiley Online Library, 2022.
  * [25] A. Masry, P. Kavehzadeh, X. L. Do, E. Hoque, and S. Joty.  Unichart: A universal vision-language pretrained model for chart comprehension and reasoning.  arXiv preprint arXiv:2305.14761, 2023.
  * [26] A. Masry, D. X. Long, J. Q. Tan, S. Joty, and E. Hoque.  Chartqa: A benchmark for question answering about charts with visual and logical reasoning.  arXiv preprint arXiv:2203.10244, 2022.
  * [27] A. McNutt and G. Kindlmann.  Linting for visualization: Towards a practical automated visualization guidance system.  In VisGuides: 2nd Workshop on the Creation, Curation, Critique and Conditioning of Principles and Guidelines in Visualization, 2018.
  * [28] N. Methani, P. Ganguly, M. M. Khapra, and P. Kumar.  Plotqa: Reasoning over scientific plots.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1527–1536, 2020.
  * [29] Microsoft.  Microsoft copilot in bing.  https://www.bing.com/chat.  Accessed: 31-3-2024.
  * [30] OpenAI.  Gpt-4v(ision) system card.  https://cdn.openai.com/papers/GPTV_System_Card.pdf.  Accessed: 31-3-2024.
  * [31] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al.  Training language models to follow instructions with human feedback.  Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
  * [32] A. V. Pandey, K. Rall, M. L. Satterthwaite, O. Nov, and E. Bertini.  How deceptive are deceptive visualizations? an empirical analysis of common distortion techniques.  In Proceedings of the 2015 CHI Conference on Human Factors in Computing Systems, pp. 1469–1478, 2015.
  * [33] J. Poco and J. Heer.  Reverse-engineering visualizations: Recovering visual encodings from chart images.  In Computer Graphics Forum, vol. 36, pp. 353–363. Wiley Online Library, 2017.
  * [34] O. Rubin, J. Herzig, and J. Berant.  Learning to retrieve prompts for in-context learning.  In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2655–2671, 2022.
  * [35] G. Team, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, et al.  Gemini: a family of highly capable multimodal models.  arXiv preprint arXiv:2312.11805, 2023.
  * [36] E. R. Tufte and P. R. Graves-Morris.  The visual display of quantitative information.  Graphics press Cheshire, CT, 1983.
  * [37] J. Wang, J. Wang, B. Athiwaratkun, C. Zhang, and J. Zou.  Mixture-of-agents enhances large language model capabilities.  arXiv preprint arXiv:2406.04692, 2024.
  * [38] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al.  Chain-of-thought prompting elicits reasoning in large language models.  Advances in neural information processing systems, 35:24824–24837, 2022.
  * [39] Z. Yang, L. Li, K. Lin, J. Wang, C.-C. Lin, Z. Liu, and L. Wang.  The dawn of lmms: Preliminary explorations with gpt-4v (ision).  arXiv preprint arXiv:2309.17421, 9(1):1, 2023.


# Coexistence of the antiferromagnetic and superconducting order and its
effect on spin dynamics in electron-doped high-$T_{c}$ cuprates

Cui-Ping Chen  National Laboratory of Solid State of Microstructure and
Department of Physics, Nanjing University, Nanjing 210093, China  Hong-Min
Jiang  National Laboratory of Solid State of Microstructure and Department of
Physics, Nanjing University, Nanjing 210093, China  Jian-Xin Li  National
Laboratory of Solid State of Microstructure and Department of Physics, Nanjing
University, Nanjing 210093, China

###### Abstract

In the framework of the slave-boson approach to the
$t-t^{\prime}-t^{\prime\prime}-J$ model, it is found that for electron-doped
high-$T_{c}$ cuprates, the staggered antiferromagnetic (AF) order coexists
with superconducting (SC) order in a wide doping level ranged from underdoped
to nearly optimal doping at the mean-field level. In the coexisting phase, it
is revealed that the spin response is commensurate in a substantial frequency
range below a crossover frequency $\omega_{c}$ for all dopings considered, and
it switches to the incommensurate structure when the frequency is higher than
$\omega_{c}$. This result is in agreement with the experimental measurements.
Comparison of the spin response between the coexisting phase and the pure SC
phase with a $d_{x^{2}-y^{2}}$-wave pairing plus a higher harmonics term
(DP+HH) suggests that the inclusion of the two-band effect is important to
consistently account for both the dispersion of the spin response and the non-
monotonic gap behavior in the electron-doped cuprates.

###### pacs:

74.20.Mn, 74.25.Ha, 75.40.Gb

##  I Introduction

The pairing symmetry of the hole-doped high-$T_{c}$ superconductors is
generally believed to have the dominant $d_{x^{2}-y^{2}}$-wave pairing.
However, the pairing symmetry of the electron-doped high-$T_{c}$
superconductors is still under debate. 1 ; 2 ; 3 ; 4 ; 5 ; 6 ; 7 ; 8  While no
consensus has been reached yet, more and more recent experimental results have
suggested that the order parameter of electron-doped cuprates is likely to
have a dominant $d_{x^{2}-y^{2}}$-wave pairing symmetry, 3 ; 4 ; 5 ; 6 ; 7 ; 8
and with an unusual non-monotonic gap function.

Although various explanations have been proposed to account for the non-
monotonic behavior, they can generally be categorized into two scenarios. 5 ;
7 ; 9 ; 10 ; 11 ; 12 ; 13 ; 14 ; 15 ; 16 ; 17 ; 18  One is to extend the
superconducting (SC) gap out of the simple $d_{x^{2}-y^{2}}$-wave via the
inclusion of higher harmonics term. 5 ; 7 ; 9 ; 10 ; 11 ; 12 ; 13  From
theoretical perspective, the non-monotonic $d_{x^{2}-y^{2}}$-wave gap appears
under the assumption that the $d_{x^{2}-y^{2}}$-wave pairing is caused by the
interaction with the continuum of overdamped antiferromagnetic (AF) spin
fluctuations. In this scenario, the non-monotonic gap behavior is described by
the combination effect of a $d_{x^{2}-y^{2}}$-wave paring plus a higher
harmonics term (DP+HH). Therefore, it is an intrinsic property of the SC state
regardless of the presence of the AF order, and a simple one-band model is
used to reproduce the non-monotonic gap behavior. The other argues that the
non-monotonic behavior is the outcome of the coexistence of the AF and the SC
orders. 14 ; 15 ; 16 ; 17 ; 18  This scenario assumes that the AF order
disguises the $d_{x^{2}-y^{2}}$-wave character of SC gap. When the AF order is
introduced, the resulting quasiparticle (QP) excitation can be gapped by both
orders and behaves to be non-monotonic, although the SC gap itself is
monotonic. The scenario gained support from the angle-resolved photoemission
spectra (ARPES) measurements, where two inequivalent Fermi pockets around
$(\pi,0)$ and $(\pi/2,\pi/2)$ have been detected. 19 ; 20  This phenomena is
well explained in terms of the $\mathbf{k}$-dependent band-folding effect
associated with an AF order which splits the band into upper and lower
branches, 14 ; 20 ; 21 ; 22  leading to the two-band and/or two-gap model.

Recently, neutron scattering experiments in electron-doped cuprates have
revealed that the spin response is commensurate in a substantial frequency
range below a crossover frequency $\omega_{c}$ ,23 ; 24 ; 25 ; 26 ; 27 ; 28
which constitutes a distinct difference from the widely studied hour-glass
dispersion in the hole-doped cuprates.29  Although, both scenarios mentioned
above can account for the non-monotonic gap behavior of the electron-doped
cuprates, a comparative study on the spin dynamics between the two scenarios
is deserved to demonstrate the possible differences and therefore serve to
select the reasonable model for electron-doped high-$T_{c}$ cuprates.

In this paper, we investigate the spin dynamics in the coexisting phase of the
AF and the $d_{x^{2}-y^{2}}$-wave SC orders. The calculation is based on a
self-consistent determination of the QP dispersion, the AF order and the SC
gap at the slave-boson mean-field level of the
$t-t^{\prime}-t^{\prime\prime}-J$ model. It is shown that the AF and SC orders
compete and coexist in a substantial doping range in the underdoped regime.
The spin response is commensurate below a crossover frequency $\omega_{c}$ for
all dopings considered, and it becomes incommensurate when the frequency is
higher than $\omega_{c}$. This result is qualitatively consistent with
experiments. 23 ; 24 ; 25 ; 26 ; 27 ; 28  While in the framework of the pure
SC state with $d_{x^{2}-y^{2}}$-wave and/or DP+HH, 30 ; 31 ; 32 , though an
extended region of a commensurate spin fluctuation also exists, it evolves
into an incommensurate spin fluctuation at low frequencies, which is not
consistent experiments. Therefore, our result suggests that the inclusion of
the two-band effect resulting from the coexisting AF and SC orders is
important to consistently account for both the spin dynamics and the non-
monotonic gap behavior in the electron-doped cuprates.

The paper is organized as follows. In Sec. II, we introduce the theoretical
model and carry out the analytical calculations. In Sec. III, we present the
numerical results with some discussions. Finally, we present the conclusion in
Sec. IV.

##  II THEORETICAL MODEL

The Hamiltonian of the two dimensional $t-t^{\prime}-t^{\prime\prime}-J$ model
on a square lattice is written in the form,

$\displaystyle H$ $\displaystyle=$
$\displaystyle-t\sum_{<ij>,\sigma}(c^{\dagger}_{i\sigma}c_{j\sigma}+H.c.)-t_{1}\sum_{<ij>_{2},\sigma}(c^{\dagger}_{i\sigma}c_{j\sigma}+H.c.)-t_{2}\sum_{<ij>_{3},\sigma}(c^{\dagger}_{i\sigma}c_{j\sigma}+H.c.)$
(1)
$\displaystyle+J\sum_{<ij>}(\mathbf{S}_{i}\cdot\mathbf{S}_{j}-\frac{1}{4}n_{i}n_{j})-\mu_{0}\sum_{<i>,\sigma}c^{\dagger}_{i\sigma}c_{i\sigma}.$

Where the summations $<ij>$, $<ij>_{2}$, $<ij>_{3}$ run over the nearest-
neighbor(n$\cdot$n), the next-n$\cdot$n, and the third-n$\cdot$n pairs
respectively, $\mathbf{S}_{i}$ is the spin on site $i$. This Hamiltonian can
be used to model both hole-doped and electron-doped systems after a particle-
hole transformation. For electron doping, one has $t$$<$$0$, $t_{1}$$>$$0$ and
$t_{2}$$<$$0$. The slave-boson mean-field theory (SBMFT) is used to decouple
the electron operators $c_{i\sigma}$ to bosons $b_{i}$ carrying the charge and
fermions $f_{i\sigma}$ representing the spin. Then, the local constraint
$b^{\dagger}_{i}b_{i}+\sum_{i\sigma}f^{\dagger}_{i\sigma}f_{i\sigma}$=$1$ is
satisfied averagely at the mean-field (MF) level. We choose the spinon pairing
order
$\Delta_{ij}$=$<f_{i\uparrow}f_{j\downarrow}-f_{i\downarrow}f_{j\uparrow}>$=$\pm\Delta$
, where $\Delta_{ij}=\Delta(-\Delta)$ for bond $<ij>$ along $x(y)$ direction,
the uniform bond order
$\chi_{ij}$=$\sum_{\sigma}<f^{\dagger}_{i\sigma}f_{j\sigma}>$=$\chi$, the AF
order
$<f_{i\uparrow}^{{\dagger}}f_{i\uparrow}-f_{i\downarrow}^{{\dagger}}f_{i\downarrow}>/2=(-1)^{i}m$,
and replace $b_{i}$ by $<b_{i}>$=$\sqrt{x}$ due to boson condensation. After
the Fourier transformation, the mean-field (MF) Hamiltonian can be written in
the Nambu representation,

$H=\sum_{\mathbf{k}}C^{{\dagger}}(\mathbf{k})\hat{A}(\mathbf{k})C(\mathbf{k})+2NJ(\chi^{2}+m^{2}+\Delta^{2}/2)-N\mu,$
(2)

here, the Nambu operator
$C^{{\dagger}}(\mathbf{k})=(f_{\mathbf{k}\uparrow}^{{\dagger}},f_{\mathbf{k+Q}\uparrow}^{{\dagger}},f_{\mathbf{-k}\downarrow},f_{\mathbf{-k-Q}\downarrow})$,
and

$\displaystyle\hat{A}(\mathbf{k})=\left(\begin{array}[]{c c c
c}{\epsilon_{\mathbf{k}}}&{-2Jm}&{-J\Delta_{\mathbf{k}}}&0\\\
{-2Jm}&{\epsilon_{\mathbf{k+Q}}}&0&{J\Delta_{\mathbf{k}}}\\\
{-J\Delta_{\mathbf{k}}}&0&{-\epsilon_{\mathbf{k}}}&{-2Jm}\\\
0&{J\Delta_{\mathbf{k}}}&{-2Jm}&{-\epsilon_{\mathbf{k+Q}}}\end{array}\right),$
(7)

where, $\epsilon_{\mathbf{k}}=(-2tx-J\chi)(\cos k_{x}+\cos k_{y})-4t_{1}x\cos
k_{x}\cos k_{y}-2t_{2}x(\cos 2k_{x}+\cos 2k_{y})-\mu$ and
$\Delta_{\mathbf{k}}=\Delta(\cos k_{x}-\cos k_{y})$. $\mu$ is the renormalized
chemical potential, $N$ is the total number of lattice sites, and
$\mathbf{Q}=(\pi,\pi)$ is the AF momentum. Note that the wave vector
$\mathbf{k}$ is restricted to the magnetic Brillouin zone (MBZ) in all
follows.

Diagonalizing of the Hamiltonian (2) by an unitary matrix
$\hat{U}(\mathbf{k})$ leads to four energy bands
$E_{1}(\mathbf{k})=E^{+}_{\mathbf{k}}$,
$E_{2}(\mathbf{k})=E^{-}_{\mathbf{k}}$,
$E_{3}(\mathbf{k})=-E^{-}_{\mathbf{k}}$,
$E_{4}(\mathbf{k})=-E^{+}_{\mathbf{k}}$, with

$E^{\pm}_{\mathbf{k}}=\sqrt{(\xi_{\mathbf{k}}^{\pm})^{2}+(J\Delta_{\mathbf{k}})^{2}},$
(8)

where
$\xi_{\mathbf{k}}^{\pm}=\epsilon^{+}_{\mathbf{k}}\pm\sqrt{(\epsilon^{-}_{\mathbf{k}})^{2}+4J^{2}m^{2}}$
with
$\epsilon^{\pm}_{\mathbf{k}}=(\epsilon_{\mathbf{k}}\pm\epsilon_{\mathbf{k+Q}})/2$.
And the free energy is written down (Boltzmann constant $k_{B}=1$),

$F=-2T\sum_{\mathbf{k},\nu=\pm}\ln[2\cosh(\frac{E_{\mathbf{k}}^{\nu}}{2T})]-\mu
N+2NJ(\chi^{2}+m^{2}+\Delta^{2}/2).$ (9)

The MF order parameters $\chi$, $\Delta$, $m$ and the chemical potential $\mu$
for different dopings $x$ can be calculated from the self-consistent equations
obtained by $\partial F/\partial\chi=0$, $\partial F/\partial\Delta=0$,
$\partial F/\partial m=0$, and $\partial F/\partial\mu=-N(1-x)$, respectively.
The magnitudes of the parameters are chosen as $t$=$-3.0J$, $t_{1}$=$1.02J$,
$t_{2}$=$-0.51J$ and $J$=$100$ meV to model the Fermi surface observed in
ARPES experiment.19 ; 20

Then, the bare spin susceptibility (transverse) is given by,

$\chi^{\pm}_{0}(\mathbf{q},\mathbf{q}^{{}^{\prime}},\tau)=\frac{1}{N}<S^{+}_{\mathbf{q}}(\tau)S^{-}_{-\mathbf{q}^{\prime}}(0)>_{(0)},$
(10)

where $<\cdots>_{(0)}$ means thermal average over the eigenstates of $H$,
$S^{+}_{\mathbf{q}}=\sum_{k}f^{+}_{\mathbf{k+q}\uparrow}f_{\mathbf{k}\downarrow}$
is the spin operator. Considering that $\mathbf{k}$ is restricted to the MBZ,
an explicit calculation shows that the spin susceptibility should be expressed
in the following matrix form,

$\displaystyle\
\hat{\chi}_{0}^{\pm}(\mathbf{q},\omega)=\left(\begin{array}[]{c
c}{\chi_{0}^{\pm}(\mathbf{q},\mathbf{q},\omega)}&{\chi_{0}^{\pm}(\mathbf{q},\mathbf{q+Q},\omega)}\\\
{\chi_{0}^{\pm}(\mathbf{q+Q},\mathbf{q},\omega)}&{\chi_{0}^{\pm}(\mathbf{q+Q},\mathbf{q+Q},\omega)}\end{array}\right),$
(13)

where, the nondiagonal correlation function $\chi_{0}^{\pm}$ with
$\mathbf{q}^{\prime}=\mathbf{q+Q}$ arises due to the umklapp process. The
matrix elements of the bare spin susceptibility, which come from the particle-
hole $(p-h)$ excitations, are given by,

$\displaystyle\chi_{0}^{\pm}(\mathbf{q},\omega)_{\eta\eta^{\prime}}$
$\displaystyle=$
$\displaystyle\frac{1}{N}\sum_{i,j=1}^{2}\sum_{m,n=1}^{2}\sum_{\mathbf{k}}[a_{1}\frac{f(E_{m}(\mathbf{k}))-f(E_{n}(\mathbf{k+q}))}{\omega+E_{n}(\mathbf{k+q})-E_{m}(\mathbf{k})+i\Gamma}+a_{2}\frac{f(E_{n}(\mathbf{k+q}))-f(E_{m}(\mathbf{k}))}{\omega-
E_{n}(\mathbf{k+q})+E_{m}(\mathbf{k})+i\Gamma}$ (14)
$\displaystyle+b_{1}\frac{1-f(E_{m}(\mathbf{k}))-f(E_{n}(\mathbf{k+q}))}{\omega+E_{n}(\mathbf{k+q})+E_{m}(\mathbf{k})+i\Gamma}+b_{2}\frac{f(E_{m}(\mathbf{k}))+f(E_{n}(\mathbf{k+q}))-1}{\omega-
E_{n}(\mathbf{k+q})-E_{m}(\mathbf{k})+i\Gamma}],$

where, $f(E_{\mathbf{\mathbf{k}}})$ is the Fermi function and

$\displaystyle a_{1}$ $\displaystyle=$ $\displaystyle
U_{in}^{*}(\mathbf{k+q})U_{(j+\eta^{\prime}-\eta)n}(\mathbf{k+q})U_{im}(\mathbf{k})U_{jm}^{*}(\mathbf{k})+U_{in}^{*}(\mathbf{k+q})U_{(j+2)n}(\mathbf{k+q})U_{im}(\mathbf{k})U_{(j+2+\eta^{\prime}-\eta)m}^{*}(\mathbf{k}),$
$\displaystyle a_{2}$ $\displaystyle=$ $\displaystyle
U_{(i+2)n}(\mathbf{k+q})U_{(j+2+\eta^{\prime}-\eta)n}^{*}(\mathbf{k+q})U_{(i+2)m}^{*}(k)U_{(j+2)m}(\mathbf{k})+U_{(i+2)n}(\mathbf{k+q})U_{jn}^{*}(\mathbf{k+q})U_{(i+2)m}^{*}(\mathbf{k})U_{(j+\eta^{\prime}-\eta)m}(\mathbf{k}),$
$\displaystyle b_{1}$ $\displaystyle=$ $\displaystyle
U_{in}^{*}(\mathbf{k+q})U_{(j+\eta^{\prime}-\eta)n}(\mathbf{k+q})U_{(i+2)m}^{*}(\mathbf{k})U_{(j+2)m}(\mathbf{k})-U_{in}^{*}(\mathbf{k+q})U_{(j+2)n}(\mathbf{k+q})U_{(i+2)m}^{*}(\mathbf{k})U_{(j+\eta^{\prime}-\eta)m}(\mathbf{k}),$
$\displaystyle b_{2}$ $\displaystyle=$ $\displaystyle
U_{(i+2)n}(\mathbf{k+q})U_{(j+2+\eta^{\prime}-\eta)n}^{*}(\mathbf{k+q})U_{im}(\mathbf{k})U_{jm}^{*}(\mathbf{k})-U_{(i+2)n}(\mathbf{k+q})U_{jn}^{*}(\mathbf{k+q})U_{im}(\mathbf{k})U_{(j+2+\eta^{\prime}-\eta)m}^{*}(\mathbf{k}).$
(15)

The renormalized spin susceptibility due to the spin fluctuations is obtained
via the random-phase approximation (RPA),

$\hat{\chi}^{\pm}(\mathbf{q},\omega)=\frac{\hat{\chi}_{0}^{\pm}(\mathbf{q},\omega)}{\hat{1}+\alpha\hat{J}_{q}\hat{\chi}_{0}^{\pm}(\mathbf{q},\omega)},$
(16)

where,

$\displaystyle\hat{J}_{q}=\left(\begin{array}[]{c c}{J(\mathbf{q})}&{0}\\\
{0}&{J(\mathbf{q+Q})}\end{array}\right)$ (19)

with $J(\mathbf{q})=J(\cos q_{x}+\cos q_{y})$. In the coexisting phase of the
AF order and SC order, $\alpha$ is taken as 1. As for the pure SC state with
DP+HH, we choose a slightly small $\alpha=0.72$, the criteria for choosing
$\alpha$ is to set the AF instability at $x=0.12$. The parameter
$\Gamma$=$0.04J$ is introduced to account for the QP damping rate which comes
from the scattering off other fluctuations that are not included here.

##  III NUMERICAL RESULTS AND DISCUSSION

In Fig. 1, we show the MF parameters $\chi$, $m$ and $\Delta$ as a function of
doping $x$. For a comparison, we also show the doping dependence of the MF SC
gap $\Delta_{1}$ obtained without considering the AF order by setting $m=0$.
It is seen that the staggered magnetization $m$ decreases with increasing
doping $x$, and goes sharply to zero at $x\approx 0.16$, which implies a phase
transition from the antiferromagnetism (AFM) phase to the paramagnetic phase.
The SC order parameter, on the other hand, increases its value initially up to
an optimal doping level, and then decreases upon further doping, forming a
generic SC dome. 33  However, the SC order parameter $\Delta_{1}$ without the
inclusion of the AF order exhibits a monotonic decrease with doping, which
deviates obviously from the experimental observations. Furthermore, the SC
order parameter $\Delta$ with an AF order shows a noticeable suppression
compared to $\Delta_{1}$, exhibits a competitive character with the AF order.
But, they also coexist in a substantial doping range. The MF phase diagram
also shows that the optimal doping is rather low compared to that deduced from
the experiments. This may be due to the fact the SBMFT includes only the MF
value of the order parameters and treats the no-double occupancy on the
average. However, the similarity of the phase diagram obtained by the SBMFT to
that of the variational quantum-cluster theory 16 ; 22  validates the SBMFT as
a low energy effective theory. Here our aim is to use the MF theory as an
effective model to study the effect of the AF order on the spin dynamics, and
then to compare the two-band and/or two-gap model with the simple one-band
model. Therefore, the relatively simple SBMFT is qualitatively enough for our
purpose. We note that a similar phase diagram has been obtained before. 15

The doping dependence of the renormalized spin susceptibility
Im$\chi(\mathbf{q},\omega)$ at a low frequency $\omega=0.04J$ in the
coexisting phase is presented in Fig. 2. In this figure, it is found that the
low-energy excitations exhibit commensurate peaks for all $x$, which consists
with the experiments well 27 . The inset shows the spin susceptibility
Im$\chi(\mathbf{q},\omega)$ at doping $x=0.15$ in the pure SC state with DP+HH
which is used to reproduce a non-monotonic SC gap behavior. One can see that
the spin response is incommensurate at low frequency without considering the
AF order.

Detailed frequency dependence of the spin response in the coexisting phase and
the pure SC phase with DP+HH at doping $x=0.15$ are shown in Figs. 3(a) and
3(b), and Figs. 3(c) and 3(d), respectively. The difference in the low
frequency regime of the two phases is more evident here. The spin fluctuation
is commensurate in a substantial frequency range below a crossover frequency
$\omega_{c}\approx 0.52J$ and down to the lowest frequency considered in the
coexisting phase, and it switches to be incommensurate when the frequency is
higher than $\omega_{c}\approx 0.52J$ [Figs. 3(a) and 3(b)]. This feature
agrees with the neutron-scattering measurements on electron-doped cuprates
that have been reported recently. 23  While for the pure SC phase with DP+HH,
the spin response is incommensurate at low frequency, then it switches to be
commensurate within the intermediate frequency range, and becomes
incommensurate again at higher frequency. 31  These results can be summarized
in the intensity plot of the imaginary part of the renormalized spin
susceptibility Im$\chi(\mathbf{q},\omega)$ as a function of frequency and
momentum along $(\pi,q_{y})$ direction, as shown in Fig. 4. In the figure, the
solid line indicating the peak position is the dispersion of spin excitations.
The commensurate spin fluctuation prevails below $\omega_{c}$ for the
coexisting system [Fig. 4(a)]. For the pure SC phase with DP+HH, the
dispersion shows a hourglass-like behavior [Fig. 4(c)], which is similar to
the hole-doped one, and does not consistent with the experiments on electron-
doped cuprates. 23

In the presence of the AF order, the energy band of quasiparticles is split
into two bands. Therefore, the particle-hole excitations that contributed to
the spin susceptibility are composed of two kinds of excitations, the intra-
band and the inter-band excitations. In Fig. 5, we present the results for the
bare spin susceptibility $\chi_{0}(\mathbf{q},\omega)$ (without the RPA
correction) coming from the intra-band and the inter-band contributions,
respectively. Figs. 5(a1) and 5(a2) denote the imaginary part of
$\chi_{0}(\mathbf{q},\omega)$, Figs. 5(b1) and 5(b2) the real part. One
obvious feature is that, the intra-band contribution is zero at the AF
momentum $\mathbf{Q}=(\pi,\pi)$, leading to the incommensurate spin response.
It results from the fact that the coherence factor in the spin susceptibility
due to the intra-band excitations,
$1-[(2Jm)^{2}-\varepsilon_{\mathbf{k+q}}\varepsilon_{\mathbf{k}}]/[{\sqrt{\varepsilon_{\mathbf{k+q}}^{2}+(2Jm)^{2}}\sqrt{\varepsilon_{\mathbf{k}}^{2}+(2Jm)^{2}}}]$
[where, $\varepsilon_{\mathbf{k}}=(-2tx-J\chi)(\cos k_{x}+\cos k_{y})$] is
zero at $\mathbf{Q}$. While, the inter-band contribution is commensurate for
all frequencies. At low frequencies, the inter-band excitations have a larger
contribution to the spin susceptibility than the intra-band excitations, so
the spin fluctuation is commensurate. However, with the increase of frequency,
the intensity of Im$\chi_{0}(\mathbf{q},\omega)$ due to the intra-band
contributions increases more rapidly than the inter-band contribution. As a
result, the spin fluctuation switches from a commensurate to an incommensurate
structure.

##  IV conclusion

In this paper, we have investigated the spin dynamics in the electron-doped
cuprates in the coexisting phase of the $d_{x^{2}-y^{2}}$-wave SC and AF
orders, and compared the results with that in the dominant
$d_{x^{2}-y^{2}}$-wave phase with a higher harmonics term. In the coexisting
phase, we found that the spin response is commensurate in a substantial
frequency range below a crossover frequency $\omega_{c}$ for all dopings
considered, and it switches to be incommensurate when the frequency is higher
than $\omega_{c}$. The theoretical calculations are shown to be in good
agreement with the experimental measurements. However, in the dominant
$d_{x^{2}-y^{2}}$-wave phase with a higher harmonics term, the dispersion is
just like that of the hole-doped one, namely exhibits a hourglass-like
dispersion, which is not consistent with experiments. Thus, our result
suggests that the inclusion of the two-band effect is important to
consistently account for both the dispersion of the spin response and the non-
monotonic gap behavior in the electron-doped cuprates.

###### Acknowledgements.

This work was supported by the National Natural Science Foundation of China
(10525415), the Ministry of Science and Technology of China (973 project
Grants Nos.2006CB601002,2006CB921800), and the China Postdoctoral Science
Foundation (Grant No. 20080441039).

Figure 1: (Color online) Mean-field phase diagram for
$t-t^{\prime}-t^{\prime\prime}-J$ model, where $\Delta_{1}$ is the
superconducting order parameter without considering the AF order. The model
parameters are taken as: $t=-3.0J$, $t^{\prime}=1.02J$,
$t^{\prime\prime}=-0.51J$. Figure 2: (Color online) Doping dependence of
Im$\chi(\mathbf{q},\omega)$ in the coexisting phase of the AF and SC order at
low frequency $\omega=0.04J$. The momentum is scanned along $(\pi,q_{y})$. The
inset shows Im$\chi(q,\omega)$ at $\omega=0.04J$ for the pure SC state with
DP+HH at doping $x=0.15$.  Figure 3: (Color online) Frequency dependence of
Im$\chi(\mathbf{q},\omega)$ at doping $x$=$0.15$. The momentum is scanned
along $(\pi,q_{y})$. (a) and (b) are in the coexistence of AF and SC state.
(c) and (d) are in the pure SC state with DP+HH. Figure 4: (Color online)
Intensity plot of Im$\chi(\mathbf{q},\omega)$ as a function of frequency
$\omega$ and momentum $\mathbf{q}$ at doping $x$=$0.15$. The momentum is
scanned along $(\pi,q_{y})$. The solid line is the peak position. (a) is in
the coexistence of AF and SC state and (b) in the pure SC state with DP+HH.
Figure 5: (Color online) Frequency dependence of the intra- and inter-band
contributions to the bare spin susceptibility $\chi_{0}(\mathbf{q},\omega)$
[(a1) and (a2) denote the imaginary part, (b1) and (b2) the real part] in the
coexistence of AF and SC state at doping $x$=$0.15$. (a1) and (b1) show the
intra-band contributions , and (a2) and (b2) the inter-band contribution.

## References

  * (1) C. C. Tsuei and J. R. Kirtley, Phys. Rev. Lett. 85, 182 (2000); A. D. Darminto, H. J. H. Smilde, V. Leca, D. H. A. Blank, H. Rogalla, and H. Hilgenkamp, Phys. Rev. Lett. 94, 167001 (2005); A. Biswas, P. Fournier, M. M. Qazilbash, V. N. Smolyaninova, H. Balci, and R. L. Greene, Phys. Rev. Lett. 88, 207004 (2002).
  * (2) L. Shan, Y. Huang, Y. L. Wang, S. Li, J. Zhao, P. Dai, Y. Z. Zhang, C. Ren, and H. H. Wen, Phys. Rev. B 77, 014526 (2008); A. Zimmers, Y. Noat, T. Cren, W. Sacks, D. Roditchev, B. Liang, and R. L. Greene, Phys. Rev. B 76, 132505 (2007); G.-Q. Zheng, T. Sato, Y. Kitaoka, M. Fujita, and K. Yamada, Phys. Rev. Lett. 90, 197005 (2003); J. A. Skinta, M.-S. Kim, T. R. Lemberger, T. Greibe and M. Naito, Phys. Rev. Lett. 88, 207005 (2002).
  * (3) T. Sato, T. Kamiyama, T. Takahashi, K. Kurahashi, and K. Yamada, Science 291, 1517 (2001).
  * (4) N. P. Armitage, D. H. Lu, D. L. Feng, C. Kim, A. Damascelli, K. M. Shen, F. Ronning, Z. X. Shen, Y. Onose, Y. Taguchi, and Y. Tokura, Phys. Rev. Lett. 86, 1126 (2001).
  * (5) H. Matsui, K. Terashima, T. Sato, T. Takahashi, M. Fujita, and K. Yamada, Phys. Rev. Lett. 95, 017003 (2005).
  * (6) S. R. Park, Y. S. Roh, Y. K. Yoon, C. S. Leem, J. H. Kim, B. J. Kim, H. Koh, H. Eisaki, N. P. Armitage, and C. Kim, Phys. Rev. B 75, 060501(R) (2007).
  * (7) G. Blumberg, A. Koitzsch, A. Gozar, B. S. Dennis, C. A. Kendziora, P. Fournier, and R. L. Greene, Phys. Rev. Lett. 88, 107002 (2002).
  * (8) Y. Dagan, R. Beck, and R. L. Greene, Phys. Rev. Lett. 99, 147004 (2007).
  * (9) P. Krotkov and A. V. Chubukov, Phys. Rev. Lett. 96, 107002 (2006).
  * (10) H. Yoshimura and D. S. Hirashima, J. Phys. Soc. Jpn. 73, 2057 (2004); 74, 712 (2005).
  * (11) V. A. Khodel, V. M. Yakovenko, M. V. Zverev, and H. Kang, Phys. Rev. B 69, 144501 (2004).
  * (12) T. Watanabe, T. Miyata, H. Yokoyama, Y. Tanaka, and J.-I. Inoue, J. Phys. Soc. Jpn. 74, 1942 (2005).
  * (13) P. Krotkov and A. V. Chubukov, Phys. Rev. B 74, 014509 (2006).
  * (14) Q. S. Yuan, F. Yuan, and C. S. Ting, Phys. Rev. B 73, 054501 (2006); Q. S. Yuan, X. Z. Yan, and C. S. Ting, $ibid$. 74, 214503 (2006).
  * (15) W. Yuan, H.-D. Lü, H.-Y. Lu, and Q.-H. Wang, Phys. Rev. B 77, 064515 (2008).
  * (16) T. Das, R. S. Markiewicz, and A. Bansil, Phys. Rev. B 74, 020506(R) (2006); M. Aichhorn, E. Arrigoni, M. Potthoff, and W. Hanke, $ibid$. 74, 024508 (2006).
  * (17) H. G. Luo and T. Xiang, Phys. Rev. Lett. 94, 027001 (2005).
  * (18) C. S. Liu, H. G. Luo, W. C. Wu, and T. Xiang, Phys. Rev. B 73, 174517 (2006).
  * (19) N. P. Armitage, F. Ronning, D. H. Lu, C. Kim, A. Damascelli, K. M. Shen, D. L. Feng, H. Eisaki, Z.-X. Shen, P. K. Mang, N. Kaneko, M. Greven, Y. Onose, Y. Taguchi, and Y. Tokura, Phys. Rev. Lett. 88, 257001 (2002).
  * (20) H. Matsui, K. Terashima, T. Sato, T. Takahashi, S.-C. Wang, H.-B. Yang, H. Ding, T. Uefuji, and K. Yamada, Phys. Rev. Lett. 94, 047005 (2005).
  * (21) C. Kusko, R. S. Markiewicz, M. Lindroos, and A. Bansil, Phys. Rev. B 66, 140513(R) (2002).
  * (22) D. S$\acute{e}$n$\acute{e}$chal, P.-L. Lavertu, M.-A. Marois, and A.-M. S. Tremblay, Phys. Rev. Lett. 94, 156404 (2005).
  * (23) S. D. Wilson, S. Li, H. Woo, P. Dai, H. A. Mook, C. D. Frost, S. Komiya, and Y. Ando, Phys. Rev. Lett. 96, 157001 (2006)
  * (24) K. Yamada, K. Kurahashi, T. Uefuji, S. Park, S.-H. Lee, and Y. Endoh, Phys. Rev. Lett. 90, 137004 (2003)
  * (25) H. J. Kang, P. Dai, H. A. Mook, D. N. Argyriou, V. Sikolenko, J. W. Lynn, Y. Kurita, S. Komiya, and Y. Ando, Phys. Rev. B 71, 214512 (2005)
  * (26) S. D. Wilson, S. Li, P. Dai, W. Bao, J. Chung, H. J. Kang, S. H. Lee, S. Komiya, Y. Ando, and Q. Si, Phys. Rev. B 74, 144514 (2006)
  * (27) M. Fujita, M. Matsuda, S.-H. Lee, M. Nakagawa, and K. Yamada, Phys. Rev. Lett. 101, 107003 (2008)
  * (28) E. M. Motoyama, G. Yu, I. M. Vishik, O. P. Vajk, P. K. Mang, and M. Greven, Nature 445, 186 (2007).
  * (29) P. Bourges, Y. Sidis, H. F. Fong, L. P. Renault, J. Bossy, A. Ivanov, and B. Keimer, Science 288, 1234 (2000); P. Dai, H. A. Mook, R. D. Hunt, and F. Dogan, Phys. Rev. B 63, 054525 (2001); S. M. Hayden, H. A. Mook, P. Dai, T. G. Perring, and F. Dogan, Nature 429, 531 (2004); J. M. Tranquada, H. Woo, T. G. Perring, H. Goka, G. D. Gu, G. Xu, M. Fujita, and K. Yamada, Nature 429, 534 (2004).
  * (30) J. X. Li, J. Zhang, and J. Luo, Phys. Rev. B 68, 224503 (2003).
  * (31) C. P. Chen, T. Zhou and J. X. Li, Physica C 469, 234 (2009).
  * (32) K. Krüger, S. D. Wilson, L. Shan, S. Li, Y. Huang, H.-H. Wen, S.-C Zhang, P. Dai, J. Zaanen, Phys. Rev. B 76, 094506 (2007).
  * (33) P. A. Lee, N. Nagaosa, and X.-G. Wen, Rev. Mod. Phys. 78, 000017 (2006).