# MathChat: Benchmarking Mathematical Reasoning and Instruction Following in
Multi-Turn Interactions

Zhenwen Liang1,2, Dian Yu2, Wenhao Yu2, Wenlin Yao2, Zhihan Zhang1,2,
Xiangliang Zhang1, Dong Yu2
1University of Notre Dame 2Tencent AI Lab, Bellevue
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Large language models (LLMs) have demonstrated impressive capabilities in
mathematical problem-solving, particularly in single-turn question-answering
formats. However, real-world scenarios often involve mathematical question-
answering that requires multi-turn or interactive information exchanges, and
the performance of LLMs on these tasks is still under-explored. This paper
introduces MathChat, a comprehensive benchmark specifically designed to
evaluate LLMs across a broader spectrum of mathematical tasks. These tasks are
structured to assess the models’ abilities in multi-turn interactions and
open-ended generation. We evaluate the performance of various state-of-the-art
LLMs on the MathChat benchmark, and we observe that while these models excel
in single-turn question answering, they significantly underperform in more
complex scenarios that require sustained reasoning and dialogue understanding.
To address the above limitations of existing LLMs when faced with multi-turn
and open-ended tasks, we develop MathChat${}_{\text{sync}}$, a synthetic
dialogue-based math dataset for LLM fine-tuning, focusing on improving models’
interaction and instruction-following capabilities in conversations.
Experimental results emphasize the need for training LLMs with diverse,
conversational instruction tuning datasets like MathChat${}_{\text{sync}}$. We
believe this work outlines one promising direction for improving the multi-
turn mathematical reasoning abilities of LLMs, thus pushing forward the
development of LLMs that are more adept at interactive mathematical problem-
solving and real-world applications.111Data and code are available at
https://github.com/Zhenwen-NLP/MathChat. Work is done during Zhenwen’s and
Zhihan’s internship at Tencent AI lab.

##  1 Introduction

Mathematical reasoning has been an essential task for computers for decades
[7]. With the explosion in Large Language Model (LLM) development [8, 1, 50,
51, 19, 48], mathematical reasoning has been widely recognized as a key
ability for assessing these models. Most math reasoning benchmarks such as
GSM8K [11], MATH [15], SVAMP [40], MAWPS [23], ASDiv [37] and MathVista [33]
feature the format of single-turn question answering (QA), where the input is
a single question and the output is the solution. Recent studies [62, 63, 13,
34, 46] have scaled up such QA data by distilling synthetic data from stronger
LLMs like GPT-4 [1] or utilizing human-annotated datasets of rationales in
diverse formats [63, 27], continually pushing the limits of math QA accuracy.
For example, on one of the most widely recognized benchmarks, GSM8K, accuracy
has increased from 10.4% with a 175B-parameter model [8] to 88.2% achieved by
a 7B-parameter model [44] in the past few years.

While math-specialized LLMs have shown promising progress on single-round QA
benchmarks, their mathematical capabilities have not been verified in more
complex scenarios. For instance, in real-world applications, such as
interactive chatbots [24, 18] and problem-solving assistants [39], math tasks
extend beyond single-round QA and require much more advanced reasoning and
instruction following abilities such as dialogue understanding, diagnostic
reasoning, educational feedback, etc. _Can the established math-specialized
LLMs perform as well on multi-round math reasoning as they do on single-round
tasks?_ This question has not been comprehensively studied, although many
recent studies have identified critical weaknesses of state-of-the-art LLM
reasoners that could happen in multi-round interactions, such as long-context
reasoning [9], self-reflection ability [17], error identification [4], and
educational content generation [20].

Therefore, in this paper, we advance the exploration of LLMs’ mathematical
reasoning abilities by introducing a new benchmark, MathChat. Figure 2 shows
the hierarchical ability taxonomy derived from the tasks in MathChat (e.g.,
those in Figure 3), which are more advanced than the capabilities tested by
single-round QA and addresses the above limitations noted in state-of-the-art
LLMs.

Figure 1: Taxonomy of MathChat. The inner ring represents the task categories
involved in MathChat. The intermediate ring lists the evaluation tasks in
MathChat. The outer ring shows the tested capabilities in our tasks beyond
simple math problem solving. See detailed descriptions in Section 2.

Figure 2: The performance comparison among various LLMs. Math LLMs (e.g.,
Deepseek-Math) have great performance on single-round QA dataset GSM8K,
achieving similar performance to GPT-3.5. However, they significantly
underperform GPT3.5 on MathChat, which requires more advanced reasoning
abilities. We average the evaluation metrics in each task and scale all values
into 0-1 for better visibility.

Based on our MathChat benchmark, we find that current state-of-the-art math-
specialized LLMs that are fine-tuned on extensive mathematical QA data
struggle to reliably answer multi-turn questions and understand instructions
that extend beyond single-round QA. Specifically, on open-ended tasks like
error analysis and problem generation in Figure 3, the fine-tuned LLMs fail
catastrophically since they can hardly understand the provided instructions.
These shortcomings are perhaps unsurprising for models like MetaMath [62],
which was trained exclusively on augmented question-answer pairs from single-
turn math datasets GSM8K and MATH. The tasks in MathChat obviously represent a
shift in distribution that challenges such models. However, even models like
WizardMath [34] that were trained on more diverse data including open-ended
dialogues and evolving instructions fail to achieve satisfactory performance
on MathChat. We have also tried to reform our multi-turn math reasoning
problem into a one-round math QA task by including all dialogue history in the
question part, no significant performance improvement is observed. These
results indicate potential over-tuning and data saturation towards the single-
turn QA data inside current math LLMs, and also highlight a crucial open
problem for the field of LLM development:

How can we empower math-focused LLMs to engage in multi-turn dialogues and
follow diverse instructions without significantly compromising their problem-
solving abilities?

To address the identified research challenge, we conduct an exploratory study
to investigate various training data mixture strategies by leveraging
extensive public math QA data, general-domain instruction tuning data,
general-domain dialogue data, and our constructed synthetic dialogue-based
math data (MathChat${}_{\text{sync}}$). The results indicate that the model
trained with MathChat${}_{\text{sync}}$ significantly outperforms the
baselines fine-tuned on other mixture datasets on open-ended tasks and
surpasses the base LLMs on problem-solving tasks (see Section 4 for more
details).

In summary, this paper makes two main contributions. First, we introduce and
release a benchmark MathChat dedicated to multi-turn math reasoning and
conversation, aimed at advancing the development of a more generalized
reasoner and assistant in mathematical contexts—a capability that existing
math-specific LLMs currently lack. Second, we demonstrate that integrating
synthetic math-dialogue dataset MathChat${}_{\text{sync}}$ with supervised
fine-tuning (SFT) markedly enhances performance on open-ended tasks within
MathChat, without compromising much accuracy on direct problem-solving tasks.
The resulting fine-tuned LLMs surpass their counterparts trained on various
combinations of existing datasets. We believe this paper offers a new
perspective on the evaluation of math-specific LLMs and advances the goal of
developing a general math reasoning assistant.

##  2 MathChat

We introduce MathChat, designed to provide a deeper and more comprehensive
examination of LLMs’ abilities in multi-turn mathematical reasoning and
instruction-following. MathChat contains four novel tasks (follow-up QA, error
correction, error analysis, and problem generation) inspired by previous
studies in the education domain that reveal the importance of following a
sequence of Initiate-Response-Follow-up [32], learning from self-made errors
[14], and posing new problems with solutions [45]. The first two tasks focus
on multi-turn mathematical problem-solving and reasoning, whereas the final
two tasks evaluate the models’ ability to follow mathematical instructions and
respond to open-ended questions. All tasks within MathChat are sourced from
the testing set of GSM8K, which we expanded using GPT-4 222We use
gpt-4-0125-preview version in this paper. to suit our specific requirements.
All data in our benchmark is also provided with a reference answer from GPT-4
with post-verification. As a result, each task category contains the same
number of samples as the GSM8K testing set—1,319. Table 1 shows some basic
statistics of our benchmark and Figure 3 shows some examples. All prompts used
to generate the task data can be found in the Appendix A.5.

Figure 3: The examples of the four tasks in our MathChat benchmark. We place
all tasks under a scenario of a dialogue between the user and the assistant.
The part with a italics font is the input to the LLMs and the highlighted
parts are generated by LLMs and to be evaluated.

##### Follow-up QA

In this task, we form a three-round dialogue between a human user and an AI
assistant. The initial round consists of a question from the original GSM8K
testing dataset, with its ground truth answer. We then use GPT-4 to generate
two additional questions that require a deeper understanding of the original
question, and they are added to the existing dialogue to initiate new rounds
of conversations. The correct answers are produced by GPT-4 and subsequently
verified and revised by two different LLMs (i.e., GPT-4 and Claude). After
several rounds of verification and revision by the LLMs themselves, we
successfully annotate $92.4\%$ of problems. For the remaining problems that do
not meet the verification criteria, we manually create the follow-up questions
and solutions to control the problem quality. After that, we randomly sample
$50$ problems from this task and find that all of them are correct and high-
quality. During the evaluation phase, we present only the questions to the
models, and the answers generated by the evaluated LLM are used in the
subsequent round as shown in Figure 3, following methodologies similar to
those used in general-domain benchmarks like MT-bench [67].

Table 1: Average lengths in MathChat benchmark. The first-round QA is essentially GSM8k testing set. We can find that our MathChat has more informative answers than GSM8k. Follow-up QA Question (First Round) | 46.25
---|---
Follow-up QA Question (Second Round) | 34.43
Follow-up QA Question (Third Round) | 41.60
Follow-up QA Answer (First Round) | 52.78
Follow-up QA Answer (Second Round) | 87.16
Follow-up QA Answer (Third Round) | 93.84
Error Correction Wrong Attempt | 54.82
Error Correction Mistake Correction | 75.27
Error Analysis Wrong Attempt | 66.17
Error Analysis Mistake Analysis | 94.69
Problem Generation New Problem | 55.37
Problem Generation New Answer | 105.13

##### Error Correction

In this task, we present a question to GPT-4 and instruct it to intentionally
generate an incorrect answer. The incorrectness of this response is easily
verified by comparing it to the correct answer in the GSM8K dataset. We use
this incorrect QA pair as the dialogue history and clearly state that the
solution is incorrect, and then we prompt the AI assistant to correct the
wrong answer. For this task, verifying the accuracy of the ground truth answer
is unnecessary because the correct final response should match the original
answer provided in the source dataset GSM8k. Thus, during evaluation, we
simply check whether the LLM’s corrected answer aligns with the original
answer. To ensure our benchmark contains a diverse set of error types, we
conduct an analysis in Appendix 7.

##### Error Analysis

LLMs have been proven to have weak error analysis abilities [17, 4]. The
initial QA pair for the error analysis task is similar to that used in the
error correction task, where the evaluated LLM is presented with an incorrect
solution to a problem. However, the tasks diverge from the second round: while
error correction focuses on rectifying the answer, error analysis further
requires the model to first recognize that an error exists, then analyze the
error and correct it. Although the two tasks share similarities in targeting
errors, they pose distinctly different challenges for LLMs, especially those
specialized in mathematics. These models are trained to solve problems
directly, aligning well with the goal of error correction. In contrast, error
analysis demands that the model not only understand the instructions but also
identify and articulate the cause of errors before correcting them. To enhance
data diversity in our benchmark, we generate a different batch of incorrect
attempts for the error analysis task, separate from those used in error
correction.

##### Problem Generation

The final task in MathChat, Problem Generation, has been a direction of
interest in both computer science and education for many years [42, 56, 69].
In this task, we provide the LLM with an original question-solution pair from
the source dataset as part of the dialogue history. We then ask the LLM to
create a new problem-solution pair that either delves deeper into the same
topic or applies the same mathematical principles in a different context. This
task is notably different from the typical mathematical QA, as it requires a
model to generate questions rather than solve them. It challenges models to
exhibit both creative and reasoning capabilities.

##  3 Evaluation of Existing LLMs on MathChat

We assess a variety of baseline LLMs using the MathChat benchmark. Detailed
experimental settings such as the descriptions of baseline models are located
in Appendix A.2.1.

###  3.1 Evaluation Metrics

For the problem-solving tasks (Follow-up QA and Error Correction), we extract
the last numerical value that appeared in the model’s response and compare it
to the ground truth number. This approach aligns with the evaluation metrics
used in most prior studies on math word problem solving. For the instruction-
following tasks (Error Analysis and Problem Generation), we utilize GPT-4 to
assign scores from $1$ to $5$ (higher is better) based on three carefully
designed multi-dimensional criteria, similar to [67, 21]. The Error Analysis
task evaluates instruction following (IF), error diagnosis (ED), and solution
accuracy (SA). The Problem Generation task assesses IF, SA, and problem
quality (PQ). A detailed description of these evaluation rubrics is available
in Appendix A.6. All these metrics are measured on a scale of 1 to 5.
Empirically, for instruction following tasks, a score of 1 to 2 indicates the
failure to understand the instructions. A score of 2 to 3 signifies a basic
understanding of the instructions, but the generated responses are often
wrong. A score of 3 to 4 means the model has a good understanding of the
instructions and can generate corresponding answers, though mistakes may still
occur sometimes. A score higher than 4 indicates a very good response, which
is usually fluent and relevant, with mistakes being rare.

###  3.2 Prompting Template

For math-specific LLMs like MetaMath and WizardMath, which are typically
trained on specific QA templates, our MathChat involves multi-turn dialogues
that do not strictly adhere to the formats of their training data. To fully
exploit their potential in evaluation, we test these models in two settings:
(i) using the chat
template333https://huggingface.co/docs/transformers/main/en/chat_templating.
of their base models, and (ii) adapting their specific QA templates to include
our dialogue history in the question part, i.e., reforming our multi-turn math
reasoning problem to a one-round math QA task. For each task, we report
results from the better-performing setting. Empirically, we find that for
tasks requiring problem-solving skills, such as Follow-up QA and Error
Correction, the second setting significantly outperforms the first. However,
performance is nearly identical across both settings for the instruction
following tasks. These experimental evaluations reveal that solving the
challenging tasks in our benchmark requires models to possess deeper
understanding and comprehension abilities. For models that cannot perform well
on our tasks, it is not merely due to their unfamiliarity with chat-template
data.

###  3.3 Result Analysis and Observations

The performance of three open-sourced general-purpose LLMs, five math-
specialized LLMs, and GPT-3.5-turbo on MathChat. All open-sourced models are
in the size of 7B. R1, R2, and R3 denote different rounds in Follow-up QA.
Evaluation metrics: Acc. (%), and others from 1 (lowest) to 5 (highest), such
as IF = Instruction Following, ED = Error Diagnosis, SA = Solution Accuracy
and PQ = Problem Quality. We bold the best performance achieved by open-
sourced models.  Follow-up QA R1* R2 R3 Error Correction Error Analysis
Problem Generation Acc. Acc. IF ED SA IF PQ SA General-Purpose 7B LLMs:
LLaMA2-chat 15.09 11.67 8.12 38.82 2.64 1.83 1.87 4.02 3.83 3.33 Mistral-
Instruct 32.06 20.40 13.70 51.20 3.50 2.82 2.77 4.44 4.30 3.80 Gemma-it 37.60
17.65 10.57 46.15 3.07 2.05 3.11 3.09 3.75 2.48 Math-specialized 7B LLMs:
MAmmoTH 66.85 32.16 19.31 54.15 2.55 1.75 1.79 2.03 1.95 2.42 MetaMath 77.18
43.98 32.16 56.30 2.51 1.26 1.34 2.28 2.32 2.35 WizardMath 83.20 44.81 36.86
68.22 2.62 1.81 1.95 1.53 1.54 1.60 DeepSeek-Math 79.40 48.19 35.70 74.34 1.87
1.38 1.47 1.95 1.96 2.08 InternLM2-Math 83.80 40.20 28.64 72.70 2.88 2.24 2.35
4.31 4.31 3.50 GPT-3.5-turbo 74.68 55.26 45.59 75.90 4.12 3.64 3.71 4.62 4.62
4.23 GPT-4-turbo 94.62 76.36 73.41 81.11 4.60 4.35 4.45 4.94 4.94 4.87 GPT-4o
95.68 77.67 73.03 83.09 4.84 4.60 4.68 4.91 4.94 4.82

  * *

The first round performance is essentially the performance on the original
GSM8K dataset.

Overall, while most math-specific LLMs (except for MAmmoTH) outperform
GPT-3.5-turbo only in the Round1 of Follow-up QA (see the first column in
Table 3.3), they fall short in all other tasks (other columns in Table 3.3).
These outcomes suggest that current math-specific models are overly tuned to
single-round QA data, and the significant performance drop in multi-round and
complex tasks further validates the rigor of our benchmark, challenging the
models’ diverse capabilities in mathematical reasoning, as illustrated in
Figure 2. We further investigate the models’ performance across each task:

##### Follow-up QA.

In Rounds 2 and 3 of the Follow-up QA tasks, models face significant
challenges in multi-round math reasoning, with accuracy reductions ranging
from 20% to 50%. This decline indicates that while math-specific LLMs
initially outperform general-purpose LLMs and even GPT-3.5-turbo in Round 1,
their performance deteriorates more significantly in subsequent rounds.
Theoretically, if a model maintains consistent accuracy across all dialogue
rounds, with a first-round accuracy of $x_{1}$, the expected second-round
accuracy would be $x_{1}^{2}$ due to error propagation. Interestingly, when
comparing the square of the first-round accuracy ($x_{1}^{2}$) with the actual
second-round accuracy ($x_{2}$), we observe a contrasting pattern:
$x_{1}^{2}>x_{2}$ for all math-specific LLMs, indicating a decline, whereas
$x_{1}^{2}<x_{2}$ for all other general-purpose models. This finding
demonstrates that while math-specific LLMs excel at solving math problems in a
single round, they show weaker progressive long-text reasoning capabilities
within dialogues, highlighting a critical gap addressed by our benchmark.

##### Error Correction.

In the Error Correction task, a clear distinction also exists between math-
specific LLMs and general-purpose LLMs. Notably, general LLMs exhibit higher
accuracy in correcting errors than in directly solving problems (i.e., the
first-round follow-up QA), whereas the reverse is true for math-specific LLMs.
This adaptive behavior is evident in general-purpose LLMs but is noticeably
lacking in math-specific LLMs, suggesting their weak ability to learn and
reason from errors due to the over-tuning on single-round QA tasks. The
difficulty of this task in our benchmark further emphasizes the need for
models to go beyond single-round accuracy and develop robust error-correction
abilities.

##### Error Analysis.

The Error Analysis task requires that models first identify errors in a given
solution before proceeding to analyze and correct them. In practice, we find
that math-specialized LLMs often misinterpret the task’s instruction about
analyzing the solution and instead simply repeat the previous answer, or just
validate the incorrect solution as correct. Conversely, only GPT-3.5-turbo
relatively performs well in verifying the solution and pinpointing errors.
This task presents a significant challenge for open-source mathematical LLMs,
indicating a common limitation: their ability to identify and analyze errors.
The high failure rate in this task also shows the challenging nature of our
benchmark.

##### Problem Generation.

The Problem Generation task, similar to Error Analysis, requires models to
understand instructions that go beyond answering the given question. This task
assesses several abilities: a model must accurately comprehend the given
instruction, understand the provided problem-solution pair, and generate a new
and relevant problem-solution pair. We observe that all general-purpose LLMs
and only one math-specific model InternLM2-Math perform well. Other math LLMs,
which are heavily optimized for problem-solving, struggle with this task.
Empirically, we find that the above four models still consistently attempt to
solve problems even when clearly instructed to create new problems. The
difficulty of adapting to problem generation highlights the rigidity of
current math-specific models, suggesting that these models are overly tuned to
solve problems and, as a result, find it challenging to adapt to other tasks.

##  4 Enhancement via Supervised Fine-Tuning

Given the above challenges highlighted by our benchmark, it is natural to seek
solutions to address these issues. In this section, we explore the performance
improvements of general-purpose models enhanced by various supervised fine-
tuning (SFT) strategies. See Appendix A.4 for case studies.

###  4.1 Baselines

We first build a series of Mistral 7B baseline models by applying supervised
fine-tuning with various existing datasets. First, Mistral-Math is developed
to specialize Mistral-Instruct in math reasoning. This is achieved via fine-
tuning the model by Arithmo [2] compilation, which includes three existing
datasets: MetaMath [62], MathInstruct [63], and Lila-OOD [38]. The used
dataset totally comprises approximately 540,000 entries. Second, Mistral-Math-
IT is then built for enhancing the instruction following ability of Mistral-
Math. We utilized the Alpaca-GPT4 dataset [41], which includes 52,000
instruction-following instances generated by GPT-4. We also use LIMA [68],
which contains 1,000 high-quality prompts and responses from human
interactions. Last, Mistral-Math-IT-Chat gains the ability to engage in
dialogue by tuning with chit-chatting dataset. We subsample the Ultra-chat200k
dataset [12] to 50,000 dialogues to minimize the training workload.
Empirically, we find that this subsampling does not significantly affect
performance on MathChat compared to using the entire Ultrachat-200k dataset.
Similarly, a series of Gemma 7B models are developed using the same SFT
setting, and named following the same format.

###  4.2 Dialogue Dataset MathChat${}_{\text{sync}}$

While the Ultra-chat200k dataset includes dialogues spanning a variety of
topics, math-related conversations should be specifically highlighted and
incorporated into the SFT process. We thus introduce and release a new dataset
MathChat${}_{\text{sync}}$, which is created by sampling QA pairs from Arithmo
as seed examples. We then tasked GPT models to engage in real-world dialogues
based on these seeds, enriching the dataset with diverse and contextually
relevant mathematical discussions. The details of the generation prompts are
provided in the Appendix A. Due to budget constraints, we generated 16,132
dialogues using GPT-4 and 131,346 dialogues using GPT-3.5-turbo, resulting in
a total of 147,478 dialogues in the MathChat${}_{\text{sync}}$ dataset. This
dataset can serve as an augmented resource during the SFT stage for future
math LLMs, enabling them to engage in dialogues without compromising their
ability to reason in single-round QA. Since MathChat${}_{\text{sync}}$ already
includes samples in forms of instruction and dialogue, Mistral and Gemma are
tuned using both Arithmo and MathChat${}_{\text{sync}}$, resulting in Mistral-
MathChat and Gemma-MathChat models, respectively.

###  4.3 Result Analysis and Observations

Performance of LLMs that are fine-tuned with different datasets. The best
performance is bold and the second best is underlined for each series.
Follow-up QA R1* R2 R3 Error Correction Error Analysis Problem Generation
(Scaled) Overall Acc. Acc. IF ED SA IF PQ SA Average Mistral 7B Series:
Mistral-Instruct 32.06 20.40 13.70 51.20 3.50 2.82 2.77 4.44 4.30 3.80 0.550
Mistral-Math 70.20 32.31 24.60 70.22 2.18 1.60 1.71 3.54 3.28 3.75 0.519
Mistral-Math-IT 70.73 40.59 27.74 69.54 2.34 1.65 1.76 4.08 3.81 4.16 0.565
Mistral-Math-IT-Chat 71.79 39.22 27.36 69.15 2.31 1.50 1.63 4.39 4.20 4.28
0.574 Mistral-MathChat (Ours) 71.02 41.02 27.97 67.96 3.40 2.89 2.67 4.70 4.58
4.43 0.661 Gemma 7B Series: Gemma-it 37.60 17.65 10.57 46.15 3.07 2.05 3.11
3.09 3.75 2.48 0.463 Gemma-Math 70.73 29.70 19.92 62.68 1.69 1.29 1.32 3.24
3.09 3.44 0.464 Gemma-Math-IT 72.02 43.36 32.57 62.60 1.76 1.40 1.46 3.34 3.32
3.61 0.508 Gemma-Math-IT-Chat 74.68 46.35 33.64 63.85 2.05 1.64 1.70 3.64 3.48
3.99 0.549 Gemma-MathChat (Ours) 72.14 47.10 32.64 61.86 3.43 2.90 2.90 3.77
3.72 3.74 0.623

Table 4.3 presents the results of two series of LLMs that have been fine-tuned
from Mistral and Gemma models. The evaluation follows the same settings on
MathChat as presented in Table 3.3. Generally, the results suggest that our
method of augmenting the training corpus enhances performance across all
tasks. Notably, incorporating general-purpose instruction tuning data from
sources such as Alpaca and UltraChat can improve performance on mathematical
tasks. This improvement may stem partly from the inclusion of mathematical
content within these datasets. The addition of high-quality instruction data
predominantly may also boost the LLMs’ natural language comprehension, thereby
enhancing their ability to solve math problems. Moreover, the model fine-tuned
with our MathChat${}_{\text{sync}}$ dataset demonstrates markedly superior
overall performance. Appendix A.1 shows how we scale and calculate the overall
score and Table 2 contains a more comprehensive comparison in terms of the
overall performance. Since MathChat${}_{\text{sync}}$ is created in a very
simple and straightforward way, we believe that scaling up the quality and
amount of such math dialogue data can bring more performance improvement,
which we leave as our future work. Detailed analysis on each task follows.

##### Follow-up QA.

When performing SFT with existing datasets, adding instruction-following,
dialogue or our MathChat${}_{\text{sync}}$ datasets generally enhances the
performance on follow-up QA tasks. Notably, we observe that performance
improvements in the second and third rounds are significantly greater compared
to the initial round of the original GSM8K QA. A likely explanation is that
these datasets contain longer-context QA pairs, which enable the model to
reason based on the dialogue history rather than focusing predominantly on
more immediate contexts.

##### Error Correction.

Fine-tuned models exhibit better accuracy than base LLMs in error correction,
yet integrating additional datasets has not markedly boosted performance. This
limited improvement suggests that essential skills such as Diagnostic
Reasoning and Solution Refinement, indicated in Figure 2, are not effectively
learned from the used datasets. Additionally, we observed that our
MathChat${}_{\text{sync}}$ data negatively affects this task. Upon examining
the error cases, we discovered that models trained with
MathChat${}_{\text{sync}}$ indeed have a better understanding of “correcting
the error”, where they try to make improvements over previous incorrect
attempts rather than simply making new attempts. This contrasts with models
trained purely on problem-solving datasets, which tend to give completely new
solutions. The lower performance of the model trained with
MathChat${}_{\text{sync}}$ may be attributed to the dataset’s lack of manual
filtering of incorrect cases. We leave the quality control problem and
analysis in our future work.

##### Error Analysis.

Similar to Error Correction, learning the ability to perform error analysis is
challenging when using SFT with math QA and general instruction tuning
datasets. Although the performance on this task is not exceptionally high, the
inclusion of math-dialogue data in SFT has proven to be a viable method for
enhancing LLMs’ capabilities in error analysis. Our analysis in Figure 6 also
reveals that the models fine-tuned with existing datasets (i.e., three
baselines) typically affirm the correctness of previous answers and terminate
their responses prematurely. In contrast, our MathChat${}_{\text{sync}}$
dataset aids LLMs in understanding how to conduct error analysis.

Figure 4: The Round3 answer quality in follow-up QA task.

Figure 5: The answer quality in Error Analysis task.

Figure 6: The answer quality in Problem Generation task.

##### Problem Generation.

On problem generation task, we observe that the base models already have
reasonable performance and SFT without MathChat${}_{\text{sync}}$ generally
hurts the performance. However, a particularly notable finding is the increase
in Solution Accuracy (SA) scores following SFT, which suggests that fine-
tuning on mathematical data helps the model recognize the importance of
solution correctness and extend this awareness to generation tasks.
Furthermore, our MathChat-enhanced SFT model records the best performance on
this task, demonstrating the versatile utility of dialogue-enhanced training
in mathematical contexts.

##  5 Analysis of Answer Qualities

To evaluate the answer qualities of various models on our MathChat benchmark,
we analyzed 500 outputs each from Mistral, InternLM2-Math (i.e., the best
math-specialized LLM in Table 3.3), Mistral-Math, and our Mistral-
MathChat${}_{\text{sync}}$ model across tasks such as Follow-up QA, Error
Analysis, and Problem Generation. We employed GPT-4 to categorize these
outputs according to a predefined set of output categories. Our analysis
revealed that the Mistral-MathChat${}_{\text{sync}}$ models excel in tasks
requiring open-ended responses, like error analysis and problem generation,
while performing comparably in problem-solving tasks. The following sections
detail these results:

LLMs + MathChat${}_{\text{sync}}$ SFT achieves state-of-the-art accuracy in
follow-up QA. As shown in Figure 6, all three math-specific models
significantly outperform the original Mistral model, with our
MathChat${}_{\text{sync}}$ model slightly surpassing the other two, showing
the strong mathematical problem solving ability is still maintained after
MathChat${}_{\text{sync}}$ fine-tuning.

LLMs + MathChat${}_{\text{sync}}$ SFT exhibits strong error identification and
correction abilities. Figure 6 shows that although the Mistral model
identifies errors in mathematical problems, it falls short in offering
corrections. InternLM2-Math and Math-SFT show reduced error detection
capabilities due to their intensive training on straightforward math QA. In
contrast, our MathChat${}_{\text{sync}}$ model demonstrates a robust capacity
for both identifying and correcting errors.

LLMs + MathChat${}_{\text{sync}}$ SFT demonstrates superior performance in
problem generation. As shown in 6, our MathChat${}_{\text{sync}}$ model excels
in problem generation tasks, while the other two math-specific models
(InternLM2-Math and Math-SFT) struggle with instruction following and basic
comprehension, highlighting the effectiveness of our
MathChat${}_{\text{sync}}$ fine-tuning approach.

##  6 Related Work

##### Mathematical Reasoning.

After the emergence of deep learning, Seq2Seq models [55, 59, 3, 29, 31, 30,
26] becomes popular in mathematical reasoning. Then LLMs have demonstrated
success in solving math word problems through techniques like Chain of Thought
(CoT) [57, 22], Program of Thought (PoT) [10], and sampling methods [54].
These studies primarily focus on improving performance via better prompting
design or inference strategies. Another line of research explores distilling
synthetic data from LLMs to train smaller models [36, 28, 34, 63].Some
researchers also attempted extensive pre-training on math-related corpora to
obtain foundational mathematical LLMs [25, 47, 5]. As for the evaluation of
mathematical reasoning, popular benchmarks include GSM8K, MAWPS [23], MATH
[15], SVAMP [40], MathVista [33], MathVerse [65], etc., and all of them are in
single-round QA format. State-of-the-art (SOTA) models such as MetaMath [62],
WizardMath, MathInstruct [63], ToRA [13], OpenMathInstruct [49] augment
extensive amount of math QA pairs from LLMs or humans as the additional
training set to boost the performance. Most recent studies apply a three-stage
training paradigm to math-related corpus including pre-training, supervised
fine-tuning, and reinforcement learning (e.g., InternLM2-Math [61] and
Deepseek-math [44]).

##### Multi-Turn Dialogues.

The advancement of dialogue capabilities in LLMs, particularly their
proficiency in multi-turn interactions, has been a key focus in LLM research
[12, 52, 67]. Collections of chat-style datasets, which include both human-to-
LLM interactions (e.g., RealChat [66]) and LLM-to-LLM exchanges (e.g., Baize
[60], Ultrachat [12]), have been fundamental and instrumental in this area.
Various benchmarks, such as MT-bench [67] and MT-bench 101 [6], have been
developed to assess these capabilities. Most benchmarks primarily concentrate
on general dialogues, while some datasets (e.g., MINT [53]) are specifically
designed to evaluate specialized skills such as tool usage and feedback usage
in reasoning tasks. In the realm of multi-turn dialogue on mathematics, many
studies have discovered models’ capability in aiding problem-solving [58],
education [35, 64], but the community still lacks a comprehensive benchmark
and a study on how to empower LLMs to perform math conversation. Our work
distinguishes itself by examining an under-explored direction of open-ended
multi-turn dialogues: the benchmarking and analysis of combined mathematical
reasoning and instruction-following on LLMs.

##  7 Conclusion

This paper introduces the MathChat benchmark as a new evaluative framework for
assessing the capabilities of large language models (LLMs) in mathematical
problem-solving and open-ended QA within multi-turn dialogue contexts. We
demonstrate that while existing math-specialized LLMs excel at single-turn
question-answering tasks, they significantly struggle with more complex, open-
ended tasks that require understanding and following multi-turn instructions.
We also collect and release a fine-tuning dataset MathChat${}_{\text{sync}}$
with math-centered dialogue interactions. LLMs trained with
MathChat${}_{\text{sync}}$ show marked improvements in handling complex tasks
in MathChat that require higher levels of comprehension and adaptability.

## Broader Impact

The research presented in this paper significantly advances automated
mathematical reasoning and instruction-following in multi-turn interactions.
By introducing the MathChat benchmark and the MathChat${}_{\text{sync}}$
dataset, this work enhances the capabilities of large language models (LLMs)
in complex, interactive problem-solving. These advancements can lead to more
effective educational tools and intelligent tutoring systems, supporting
dynamic, interactive learning experiences. The public release of MathChat and
MathChat${}_{\text{sync}}$ promotes further innovation and ensures responsible
use, contributing to the development of AI systems that excel in real-world,
context-aware applications.

## References

  * [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  arXiv preprint arXiv:2303.08774, 2023.
  * [2] akjindal53244.  Arithmo.  https://github.com/akjindal53244/Arithmo, 2023.
  * [3] Reem Alghamdi, Zhenwen Liang, and Xiangliang Zhang.  Armath: a dataset for solving arabic math word problems.  In LREC, pages 351–362, 2022.
  * [4] Anonymous Authors.  Evaluating mathematical reasoning of large language models: A focus on error identification and correction.  In ARR Feb Submission, 2024.
  * [5] Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen Marcus McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck.  Llemma: An open language model for mathematics.  In The Twelfth International Conference on Learning Representations, 2024.
  * [6] Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, et al.  Mt-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues.  arXiv preprint arXiv:2402.14762, 2024.
  * [7] DG Boblow.  Natural language input for a computer problem-solving system.  Semantic Information Processing, pages 146–226, 1968.
  * [8] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [9] Changyu Chen, Xiting Wang, Ting-En Lin, Ang Lv, Yuchuan Wu, Xin Gao, Ji-Rong Wen, Rui Yan, and Yongbin Li.  Masked thought: Simply masking partial reasoning steps can improve mathematical reasoning learning of language models.  arXiv preprint arXiv:2403.02178, 2024.
  * [10] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen.  Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.  Transactions on Machine Learning Research, 2023.
  * [11] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al.  Training verifiers to solve math word problems.  arXiv preprint arXiv:2110.14168, 2021.
  * [12] Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou.  Enhancing chat language models by scaling high-quality instructional conversations.  arXiv preprint arXiv:2305.14233, 2023.
  * [13] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al.  Tora: A tool-integrated reasoning agent for mathematical problem solving.  The Twelfth International Conference on Learning Representations, 2024.
  * [14] Tim Heemsoth and Aiso Heinze.  Secondary school students learning from reflections on the rationale behind self-made errors: A field experiment.  The Journal of Experimental Education, 84(1):98–118, 2016.
  * [15] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.  Measuring mathematical problem solving with the math dataset.  In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
  * [16] Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al.  Lora: Low-rank adaptation of large language models.  In International Conference on Learning Representations, 2021.
  * [17] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou.  Large language models cannot self-correct reasoning yet.  arXiv preprint arXiv:2310.01798, 2023.
  * [18] Antonín Jančařík, Jarmila Novotná, and Jakub Michal.  Artificial intelligence assistant for mathematics education.  In Proceedings of the 21st European Conference on e-Learning-ECEL, pages 143–148, 2022.
  * [19] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al.  Mistral 7b.  arXiv preprint arXiv:2310.06825, 2023.
  * [20] Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al.  Chatgpt for good? on opportunities and challenges of large language models for education.  Learning and individual differences, 103:102274, 2023.
  * [21] Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo.  Prometheus 2: An open source language model specialized in evaluating other language models.  arXiv preprint arXiv:2405.01535, 2024.
  * [22] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.  Large language models are zero-shot reasoners.  NeurIPS, 35:22199–22213, 2022.
  * [23] Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi.  Mawps: A math word problem repository.  In NAACL, pages 1152–1157, 2016.
  * [24] Dabae Lee and Sheunghyun Yeo.  Developing an ai-based chatbot for practicing responsive teaching in mathematics.  Computers & Education, 191:104646, 2022.
  * [25] Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al.  Solving quantitative reasoning problems with language models.  Advances in Neural Information Processing Systems, 35:3843–3857, 2022.
  * [26] Zhenwen Liang, Tianyu Yang, Jipeng Zhang, and Xiangliang Zhang.  Unimath: A foundational and multimodal mathematical reasoner.  EMNLP, 2023.
  * [27] Zhenwen Liang, Dian Yu, Xiaoman Pan, Wenlin Yao, Qingkai Zeng, Xiangliang Zhang, and Dong Yu.  Mint: Boosting generalization in mathematical reasoning via multi-view fine-tuning.  arXiv preprint arXiv:2307.07951, 2023.
  * [28] Zhenwen Liang, Wenhao Yu, Tanmay Rajpurohit, Peter Clark, Xiangliang Zhang, and Ashwin Kaylan.  Let gpt be a math tutor: Teaching math word problem solvers with customized exercise generation.  EMNLP, 2023.
  * [29] Zhenwen Liang, Jipeng Zhang, Lei Wang, Wei Qin, Yunshi Lan, Jie Shao, and Xiangliang Zhang.  MWP-BERT: Numeracy-augmented pre-training for math word problem solving.  In Findings of NAACL, pages 997–1009, 2022.
  * [30] Zhenwen Liang, Jipeng Zhang, Lei Wang, Yan Wang, Jie Shao, and Xiangliang Zhang.  Generalizing math word problem solvers via solution diversification.  In AAAI, 2023.
  * [31] Zhenwen Liang, Jipeng Zhang, and Xiangliang Zhang.  Analogical math word problems solving with enhanced problem-solution association.  EMNLP, 2022.
  * [32] Woong Lim, Ji-Eun Lee, Kersti Tyson, Hee-Jeong Kim, and Jihye Kim.  An integral part of facilitating mathematical discussions: Follow-up questioning.  International Journal of Science and Mathematics Education, 18:377–398, 2020.
  * [33] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao.  Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.  In The Twelfth International Conference on Learning Representations, 2024.
  * [34] Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang.  Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct.  arXiv preprint arXiv:2308.09583, 2023.
  * [35] Jakub Macina, Nico Daheim, Sankalan Pal Chowdhury, Tanmay Sinha, Manu Kapur, Iryna Gurevych, and Mrinmaya Sachan.  Mathdial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems.  In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
  * [36] Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn.  Teaching small language models to reason.  arXiv preprint arXiv:2212.08410, 2022.
  * [37] Shen-Yun Miao, Chao-Chun Liang, and Keh-Yih Su.  A diverse corpus for evaluating and developing english math word problem solvers.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 975–984, 2020.
  * [38] Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, et al.  Lila: A unified benchmark for mathematical reasoning.  In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5807–5832, 2022.
  * [39] Hien D Nguyen, Vuong T Pham, Dung A Tran, and Trung T Le.  Intelligent tutoring chatbot for solving mathematical problems in high-school.  In 2019 11th International Conference on Knowledge and Systems Engineering (KSE), pages 1–6. IEEE, 2019.
  * [40] Arkil Patel, Satwik Bhattamishra, and Navin Goyal.  Are nlp models really able to solve simple math word problems?  In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080–2094, 2021.
  * [41] Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao.  Instruction tuning with gpt-4.  arXiv preprint arXiv:2304.03277, 2023.
  * [42] Oleksandr Polozov, Eleanor O’Rourke, Adam M Smith, Luke Zettlemoyer, Sumit Gulwani, and Zoran Popović.  Personalized mathematical word problem generation.  In Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.
  * [43] Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He.  Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters.  In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3505–3506, 2020.
  * [44] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, Y Wu, and Daya Guo.  Deepseekmath: Pushing the limits of mathematical reasoning in open language models.  arXiv preprint arXiv:2402.03300, 2024.
  * [45] Edward A Silver.  On mathematical problem posing.  For the learning of mathematics, 14(1):19–28, 1994.
  * [46] Zhengyang Tang, Xingxing Zhang, Benyou Wan, and Furu Wei.  Mathscale: Scaling instruction tuning for mathematical reasoning.  arXiv preprint arXiv:2403.02884, 2024.
  * [47] Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic.  Galactica: A large language model for science.  arXiv preprint arXiv:2211.09085, 2022.
  * [48] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al.  Gemma: Open models based on gemini research and technology.  arXiv preprint arXiv:2403.08295, 2024.
  * [49] Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman.  Openmathinstruct-1: A 1.8 million math instruction tuning dataset.  arXiv preprint arXiv:2402.10176, 2024.
  * [50] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  arXiv preprint arXiv:2302.13971, 2023.
  * [51] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.  Llama 2: Open foundation and fine-tuned chat models.  arXiv preprint arXiv:2307.09288, 2023.
  * [52] Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al.  Zephyr: Direct distillation of lm alignment.  arXiv preprint arXiv:2310.16944, 2023.
  * [53] Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, and Heng Ji.  Mint: Evaluating llms in multi-turn interaction with tools and language feedback.  In The Twelfth International Conference on Learning Representations, 2023.
  * [54] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.  Self-consistency improves chain of thought reasoning in language models.  In The Eleventh International Conference on Learning Representations, 2022.
  * [55] Yan Wang, Xiaojiang Liu, and Shuming Shi.  Deep neural solver for math word problems.  In EMNLP, pages 845–854, 2017.
  * [56] Zichao Wang, Andrew Lan, and Richard Baraniuk.  Math word problem generation with mathematical consistency and problem context constraints.  In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5986–5999, 2021.
  * [57] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.  Chain-of-thought prompting elicits reasoning in large language models.  NeurIPS, 35:24824–24837, 2022.
  * [58] Yiran Wu, Feiran Jia, Shaokun Zhang, Qingyun Wu, Hangyu Li, Erkang Zhu, Yue Wang, Yin Tat Lee, Richard Peng, and Chi Wang.  An empirical study on challenging math problem solving with gpt-4.  arXiv preprint arXiv:2306.01337, 2023.
  * [59] Zhipeng Xie and Shichao Sun.  A goal-driven tree-structured neural model for math word problems.  In IJCAI, pages 5299–5305, 2019.
  * [60] Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley.  Baize: An open-source chat model with parameter-efficient tuning on self-chat data.  In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6268–6278, 2023.
  * [61] Huaiyuan Ying, Shuo Zhang, Linyang Li, Zhejian Zhou, Yunfan Shao, Zhaoye Fei, Yichuan Ma, Jiawei Hong, Kuikun Liu, Ziyi Wang, et al.  Internlm-math: Open math large language models toward verifiable reasoning.  arXiv preprint arXiv:2402.06332, 2024.
  * [62] Longhui Yu, Weisen Jiang, Han Shi, YU Jincheng, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu.  Metamath: Bootstrap your own mathematical questions for large language models.  In The Twelfth International Conference on Learning Representations, 2024.
  * [63] Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen.  Mammoth: Building math generalist models through hybrid instruction tuning.  In The Twelfth International Conference on Learning Representations, 2024.
  * [64] Chao Zhang, Xuechen Liu, Katherine Ziska, Soobin Jeon, Chi-Lin Yu, and Ying Xu.  Mathemyths: Leveraging large language models to teach mathematical language through child-ai co-creative storytelling.  arXiv preprint arXiv:2402.01927, 2024.
  * [65] Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, et al.  Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?  arXiv preprint arXiv:2403.14624, 2024.
  * [66] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric Xing, et al.  Realchat-1m: A large-scale real-world llm conversation dataset.  In The Twelfth International Conference on Learning Representations, 2023.
  * [67] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al.  Judging llm-as-a-judge with mt-bench and chatbot arena.  Advances in Neural Information Processing Systems, 36, 2023.
  * [68] Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, LILI YU, et al.  Lima: Less is more for alignment.  In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  * [69] Zihao Zhou, Maizhen Ning, Qiufeng Wang, Jie Yao, Wei Wang, Xiaowei Huang, and Kaizhu Huang.  Learning by analogy: Diverse questions generation in math word problem.  arXiv preprint arXiv:2306.09064, 2023.

##  Appendix A Appendix

###  A.1 Overall Results

To facilitate a more thorough and direct comparison across different models on
our MathChat benchmark, we have formulated three comprehensive metrics based
on two key aspects: problem-solving accuracy and open-ended task quality.
Initially, we normalize all sub-metrics to a 0-1 scale. For problem-solving
tasks, including follow-up QA and error correction, accuracies are normalized
by dividing each by 100. For open-ended tasks, which are graded on a 1 to 5
scale, we normalize by dividing the scores by 5. We then define three metrics:
1) Overall Average: the average score of all ten sub-metrics listed in Tables
3.3 and 4.3; 2) Task Average: the average score across the four tasks; 3)
Category Average: the average score of the two categories, i.e., problem-
solving and open-ended QA.

The results in Table 2, based on the metrics defined above, indicate that the
model with a Mistral backbone, fine-tuned with our MathChat${}_{\text{sync}}$
dataset, achieves the best performance across all three metrics. This proves
the effectiveness of our SFT dataset and suggests that there is still
potential for improvement in math-specific LLMs.

Table 2: Overall results of 7B LLMs. The best models are bold and the second best is underlined. Model | Overall Average | Task Average | Category Average
---|---|---|---
LLaMA2-chat | 0.424 | 0.418 | 0.384
Mistral-Instruct | 0.550 | 0.544 | 0.507
Gemma-it | 0.463 | 0.463 | 0.432
MAmmoTH | 0.422 | 0.442 | 0.424
MetaMath | 0.451 | 0.470 | 0.463
WizardMath | 0.454 | 0.492 | 0.476
DeepSeek-Math | 0.452 | 0.500 | 0.476
InternLM2-Math | 0.617 | 0.635 | 0.608
Gemma-Math | 0.464 | 0.491 | 0.463
Gemma-Math-IT | 0.508 | 0.528 | 0.511
Gemma-Math-IT-Chat | 0.549 | 0.564 | 0.548
Mistral-Math | 0.519 | 0.549 | 0.514
Mistral-Math-IT | 0.565 | 0.586 | 0.557
Mistral-Math-IT-Chat | 0.574 | 0.593 | 0.565
Gemma-MathChat (Ours) | 0.623 | 0.622 | 0.608
Mistral-MathChat (Ours) | 0.661 | 0.664 | 0.638

###  A.2 Experiment Details

####  A.2.1 Existing LLM Baselines

We test three general-purpose, open-source models: LLaMA2-7B-chat [51],
Mistral-7B-Instruct [19] and Gemma-7B-it [48]. Additionally, we examine five
math-specific LLMs: MAmmoTH [63] create and release MathInstruct, a math
problem-solving dataset including CoT-style and PoT-style annotations and
perform Supervised Fine-Tuning (SFT) on various base LLMs. In this paper, we
use their released MAmmoTH-Mistral-7B variant. MetaMath-Mistral-7B [62] is
trained on augmented math data based on GSM8K and MATH. WizardMath-7B-v1.1
[34] utilizes both SFT and reinforcement learning from evol-instruct Feedback
on math instructions. InternLM2-7B-Math [61] and DeepSeek-7B-Math [44]
incorporate pre-training, SFT, and preference alignment focused on a
mathematical corpus. We also present the performance of GPT-3.5-turbo,
GPT-4-turbo and the latest GPT-4o.

####  A.2.2 Supervised Fine-tuning Implementation

We utilize Mistral-7B and Gemma-7B as our backbone models and conduct fine-
tuning using Low-Rank Adaptation (LoRA) [16], with the rank set to 8 and alpha
to 16. In our training process, we do not employ any specific templates or
prefixes for the QA pairs but utilize the default chat template of the base
models for transforming dialogues. The implementation is based on Pytorch
along with the DeepSpeed [43] Library, and the models are trained on 8 NVIDIA
V100 GPUs, each with 32GB of memory. We opt for float-16 (FP16) precision to
decrease memory demands and computational requirements. The fine-tuning is
carried out over three epochs, with a batch size of 32 and a learning rate of
3e-5. The cumulative training time for integrating all three types of datasets
amounts to approximately 72 hours, and the training time for SFT with Math +
MathChat${}_{\text{sync}}$ is around 30 hours.

###  A.3 Error Type Analysis

Figure 7: The distribution of error types on error correction task.

To ensure our benchmark contains a diverse array of error types, we randomly
sampled 500 errors from our error correction task and used GPT-4 to determine
their error types. The distribution of errors are shown in Figure 7:
Calculation Errors were most frequent, accounting for 41.8% of the total.
Reasoning Errors constituted 32.6%, indicating challenges in logical thinking
and strategizing the steps required to solve problems. Conceptual Errors,
making up 9.6%, pointed to difficulties in understanding underlying
mathematical concepts. Ambiguity in solutions was noted in 13.8% of cases,
where the provided solution is ambiguous or unclear. This range of error types
highlights the broad spectrum of challenges that MathChat contains, making our
benchmark a robust tool for diagnosing and improving error correction and
analysis ability across a variety of categories.

###  A.4 Case Study

##### Follow-up QA

Figure 8 displays the responses from four LLMs on the follow-up QA task,
specifically focusing on the third round of each model’s response. The
Mistral-instruct and Mistral-Math models, despite performing well in the first
two rounds, exhibit reasoning errors in their third-round outputs. The
InternLM2-Math model demonstrates a correct reasoning chain but makes a
calculation error, resulting in an incorrect answer. These results indicate
that the three models struggle with long-context reasoning, leading to
increased errors as the number of dialogue turns rises. In contrast, our
model, trained with MathChat${}_{\text{sync}}$, consistently performs well and
successfully solves the third-round problem.

Figure 8: This figure illustrates the responses of four different LLMs in a
follow-up QA task.

##### Error Analysis

Figure 9 shows the responses from four LLMs on an error analysis task. This
base model - Gemma-it 7B correctly identified the calculation error regarding
the sheep in Toulouse and corrected the user’s response. However, the feedback
was verbose and included unnecessary details, potentially leading to
confusion. The models fine-tuned with existing datasets failed to recognize
the error in the user’s solution, incorrectly affirming the erroneous
calculation. This indicates a lack of training focus on error identification
and correction capabilities in these models. Trained with our MathChat
dataset, the model successfully identified and corrected the calculation error
in a concise and clear manner. The analysis shows the importance of targeted,
dialogue-rich training datasets like MathChat${}_{\text{sync}}$ in developing
LLMs that are capable of effective educational interaction. The superior
performance of the MathChat-trained model demonstrates its potential as a
valuable tool in educational settings, offering precise and understandable
corrections that can aid learning.

Figure 9: This figure illustrates the responses of four different LLMs in an
error analysis task. The base Gemma-it model identifies and corrects the error
but with verbosity, models fine-tuned with existing datasets fail to detect
the error, and the MathChat${}_{\text{sync}}$ model successfully identifies
and corrects the error succinctly and accurately.

##### Problem Generation

Figure 10 presents the responses from four LLMs on the problem generation
task. The Mistral-Instruct and Mistral-Math models provide incorrect solutions
when generating a new problem, while the InternLM2-Math model fails to follow
the instructions for generating solutions to the new problem. Only our model,
trained with MathChat${}_{\text{sync}}$, successfully understands the
instructions and delivers a high-quality solution.

Figure 10: This figure illustrates the responses of four different LLMs in a
problem generation task.

###  A.5 Prompt for Generating Task Data

The prompts used to generate our MathChat benchmark is provided in Figure 11,
12, 13 and 14.

Figure 11: The system prompt for generating Follow-up QA task data. Figure 12:
The system prompt for generating Error Correction task data. Figure 13: The
system prompt for generating Error Analysis task data. Figure 14: The system
prompt for generating Problem Generation task data.

###  A.6 Prompt for Evaluating Open-ended Tasks

We provide the prompts used for evaluating the results of Error Analysis and
Problem Generation in Figure 15 and 16.

Figure 15: The system prompt for evaluating Error Analysis results using
GPT-4. Figure 16: The system prompt for evaluating Problem Generation results
using GPT-4.

### Prompt for Generating MathChat${}_{\text{sync}}$

The prompt for generating MathChat${}_{\text{sync}}$ is shown in Figure 17.

Figure 17: The system prompt for generating the MathChat${}_{\text{sync}}$
dataset for supervised fine-tuning.


# Quantum battery in the Heisenberg spin chain models with Dzyaloshinskii-
Moriya interaction

Xiang-Long Zhang  School of Physics & Optoelectronic Engineering, Anhui
University, Hefei 230601, People’s Republic of China  Xue-Ke Song  School of
Physics & Optoelectronic Engineering, Anhui University, Hefei 230601, People’s
Republic of China  Dong Wang<EMAIL_ADDRESS>School of Physics &
Optoelectronic Engineering, Anhui University, Hefei 230601, People’s Republic
of China

###### Abstract

Quantum battery (QB) is an energy storage and extraction device conforming to
the principles of quantum mechanics. In this study, we consider the
characteristics of QBs for the Heisenberg spin chain models in the absence and
presence of Dzyaloshinskii-Moriya (DM) interaction. Our results show that the
DM interaction can enhance the ergotropy and power of QBs, which shows the
collective charging can outperform parallel charging regarding QB’s
performance. Besides, it turns out that first-order coherence is a crucial
quantum resource during charging, while quantum steering between the cells is
not conducive to the energy storage of QBs. Our investigations offer insight
into the properties of QBs with Heisenberg spin chain models with DM
interaction and facilitate us to acquire the performance in the framework of
realistic quantum batteries.

##  I Introduction

The rapid development of the quantum information field has brought great
changes to the world, and some devices, based on the quantum properties of
matter, have shown great advantages compared with traditional devices. Among
them, QBs have attracted wide attention because of their unique quantum
advantages z1 ; z2 ; z3 ; z4 ; z5 ; z6 ; z7 ; z8 ; z9 . Alicki and Fannes
conducted pioneering work on the proposal of QBs (QBs) as well as the study of
quantum effects to improve their performance z10 . Like traditional
electrochemical cells, QBs are used as temporary energy storage devices; they
have a limited energy capacity and power density and may also dissipate due to
unavoidably interacting with the environment z11 ; z12 ; z13 ; z14 .
Conversely, QBs can be charged (or consumed) quantum operations that create a
coherent superposition between different states z15 , and interestingly, the
existence of quantum effects allows QBs to have high charging speeds that far
exceed those of conventional batteries.

Typically, QBs have often been defined as a set of $N$ identical and
independent subsystems, upon which a temporary charging field acts to either
extract or deposit work. Alicki and Fannes, in particular, demonstrated that
the global entangling operations allows for the extraction of more work from a
QB compared to local operations z10 . This was further observed by
Hovhannisyan et al. z16 , who found that a series of ${N}$ global entangling
operations can extract the maximum work without creating any entanglement in
the QB. In this theoretical framework, the quantum battery is modeled as a
one-dimensional Heisenberg spin chain consisting of $N$ spins, inherently
facilitating interactions among the spins and offers the entanglement z17 .
Later, two types of charging schemes, ”parallel” and ”collective” schemes were
proposed z18 ; z19 . Some conclusions have emerged about collective and
parallel charging methods for QBs. For the QB’s charging process, a collective
charging scheme offers a “quantum advantage,” that is, when ${N}{\geq 2}$ ,
the charging power of the QB outperforms that of a parallel charging scheme
z20 ; z21 ; z22 ; z23 ; z24 ; z25 ; z26 ; z27 ; z28 ; z29 ; z30 ; z31 ; z32 .
Campaioli et al. demonstrated that a battery pack consisting of $N$ highly
mixed qubits can charge collectively $N$ times faster than each qubit was
charged separately z33 .

The spin system is regarded as the most popular one for constructing QBs. To
date, much attention has been paid for exploiting QBs with spin systems. For
example, the spin-spin interactions can enhance charging power compared to
noninteracting scenarios in the XXZ Heisenberg chain z34 . Dou et al. proposed
a proposal of QB in the presence of cavity Heisenberg-spin-chain with long-
range interactions, which has significance for the performance improvement of
QBs z25 . Shi et al. revealed that quantum coherence in the battery or
entanglement between the battery and charger is essential for producing
nonzero extractable work during the charging process z35 . There is an
important term in spin systems, say Dzyaloshinsky-Moriya (DM) interaction z36
; z37 , which is deemed as the anisotropic superexchange interaction, and
significantly reflects the properties of the system of interest. In this
sense, it is fundamentally required and meaningful to unveil how the DM
interaction influences the performance of QBs in practice. Motivated by this,
our work readily contributes to addressing this issue.

The remainder of this paper is structured as follows. In Sec. II, we describe
the charging models of QBs and introduce the measure to quantify QB’s
performance. In Sec. III, we examine the influence of quantum correlation in
the evolution of the Ising model, XXZ model, and XYZ model and the influence
of the variation of DM interaction strength on them. Finally, a summary is
provided in Sec. IV.

##  II charging model

We consider a two-qubit QB model consisting of two coupled two-level systems.
The battery, meanwhile, is charged by a local field coupling each cell
individually, as shown in Fig. 1. Without loss of generality z38 , the driving
Hamiltonian of the QB system can be written as (hereafter, we set $\hbar=1$)

$\displaystyle{\cal H}$ $\displaystyle={\cal H}_{\mathrm{ch}}+{\cal
H}_{\mathrm{int}},$ (1) $\displaystyle{\cal H}_{\mathrm{ch}}$
$\displaystyle=\hbar\Omega\sum_{n=1}^{2}\hat{\sigma}_{n}^{x},$ (2)

where ${\hat{\sigma}_{n}^{x}}$ is the Pauli X-matrix acting on the ${n}$-th
spin, and ${\Omega}$ is the strength of the charged magnetic field. The second
item ${\cal H}_{\mathrm{int}}$ denotes the interaction one given by the
Heisenberg Hamiltonian with DM interaction. The Hamiltonian of the Heisenberg
chain model with DM interaction in the $z$-direction of 2 sites is

$\displaystyle\begin{split}{\cal
H}_{\mathrm{int}}=&J\hbar\left[(1+\gamma)\hat{\sigma}_{1}^{x}\hat{\sigma}_{2}^{x}+(1-\gamma)\hat{\sigma}_{1}^{y}\hat{\sigma}_{2}^{y}+\Delta\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z}\right]\\\
&+D(\hat{\sigma}_{1}^{x}\hat{\sigma}_{2}^{y}-\hat{\sigma}_{1}^{y}\hat{\sigma}_{2}^{x}),\end{split}$
(3)

where ${J}$ is the nearest exchange coupling constant, ${\gamma}$ and
${\Delta}$ are the dimensionless parameters related to chain anisotropy. ${D}$
is the strength of DM interaction in the ${z}$-direction, and
${\hat{\sigma}_{i}^{\alpha}}$ (${{\alpha=x,y}}$ ) are Pauli operators of the
${i}$-th site. If $\gamma=1$ and $D=\Delta=0$ are held, the current model is
converted to the simple Ising model. If $\gamma=D=0$ and $\Delta\neq 0$, the
model becomes the so-called XXZ model. When $\Delta\neq 0$ and $\gamma\neq 0$,
the battery is converted into the XYZ model. For QBs, the free Hamiltonian
${{\cal H}_{0}}$ considered here as ${{\cal H}_{0}=\hbar
w_{0}{\textstyle\sum_{n=1}^{2}}\hat{\sigma}_{n}^{z}}$ plays a crucial role in
the ability of storing energy, with identical Larmor frequency ${w_{0}}$ for
both qubits. Here, the ground and excited states of a single spin are
represented by ${|\downarrow\rangle}$ and ${|\uparrow\rangle}$, respectively.
Thus, the state of the fully charged battery is ${|\uparrow\uparrow\rangle}$
with energy ${2\hbar w_{0}}$, and the empty state is $|\psi_{0}\rangle$=
${|\downarrow\downarrow\rangle}$ with energy ${-2\hbar w_{0}}$. The charging
process can be described by a unitary operator ${\cal U}$, namely,

$\displaystyle{\cal U}=e^{-i{\cal H}t}$ (4)

The system dynamics obey the following equation:

$\displaystyle|\psi(t)\rangle={\cal U}|\psi(0)\rangle.$ (5)

The work extracted from a quantum battery is well defined as the ergotropy z39
, and here we introduce the concept of a passive state, mentioning the state
in which no works can be extracted from them by unitary transformations.
However, for pure states, the lowest energy state of the system is its ground
state; therefore, the passive state can be well defined as its ground state
z40 . Here, the ergotropy can be determined by the difference between the
final and ground state energies. In addition to the ergotropy, the power is
another important indicator to evaluate the QBs’ performance. Specifically,
the ergotropy and power of the battery can be given by the following formula:

$\displaystyle\zeta=\mathrm{Tr}(\hat{\rho}{{\cal{H}}_{0}})-\max\mathrm{Tr}\mathrm{(}{\cal{U}}\hat{\rho}{\cal{U^{\dagger}}}{{\cal{H}}_{0}}),$
(6) $\displaystyle P\left(t\right)=\frac{{\zeta}}{\mathit{t}},$ (7)

respectively. First, we study the case where there is no interaction between
the two batteries $(J=0)$, and each battery evolves independently driven by
the charging field. From this, we can deduce that the ergotropy of the battery
during the parallel charging is

$\displaystyle\zeta_{\shortparallel}(t)$
$\displaystyle=\zeta_{\max}\sin^{2}(\Omega t).$ (8)

From the above equation, one can obtain the minimal time
$t_{\min}=\pi/(2\Omega)$, which corresponds to the maximal ergotropy of QB in
parallel charging.

Figure 1: Diagram of two-cell QB, the spin system is charged by parallel and
collective charging, respectively. The battery is initially in the ground
state (spin-down), and it will be in the excited state (spin-up) for the
maximal energy of QB as time $t$ goes by. A local electric field acts on the
cells, and the cells interact with each other along the collective charge.
When collectively charged, the system can evolve through entangled states.

##  III Steering, first-order coherence and charging power

To study the quantumness of the two-cell QB, we herein detailedly examine
quantum steering z41 ; z42 ; z43 ; z44 ; z45  and the coherence of the system
state. Conceptually, quantum steering is logically distinct from entanglement
and Bell nonlocality, which represents a strict subset of entangled states and
a strict superset of Bell nonlocal states, and quantum steering describes the
ability of a local measurement of an entangled particle acting on one of the
particles to affect the state of the other particle non-locally z46 . For a
two-qubit system, it can be expressed as z47

$\displaystyle\frac{1}{\sqrt{3}}\left|\sum_{i=1}^{3}\mathrm{Tr}(A_{i}\otimes
B_{i}\hat{\rho}_{AB})\right|\leq 1,$ (9)

where ${\mathit{A_{i}}=\mathbf{a_{\mathit{i}}\cdot{\mathbf{\hat{\sigma}}}}}$
and ${B_{\mathit{i}}=\mathbf{b_{\mathit{i}}\cdot{\mathbf{\hat{\sigma}}}}}$
with $i\in\\{1,\mathrm{2},\mathrm{3}\\}$, are Hermitian operators acting on
qubits ${A}$ and ${B}$, respectively. Here,
${\mathbf{a_{\mathit{i}},b_{\mathit{i}}}\in\mathbb{R^{\mathrm{3}}}}$ are two
unit vectors and
${\mathbf{\hat{\sigma}=\mathrm{(}\mathrm{\hat{\sigma}_{1},\hat{\sigma}_{2},\hat{\sigma}_{3})}}}$
denotes the Pauli matrices. Any violation of inequality implies that
${\hat{\rho}_{AB}}$ is steerable, and the maximal violation can be described
as z48

$\displaystyle\
S_{AB}=\underset{\left\\{A_{\mathit{i}},B_{\mathit{i}}\right\\}}{\max}\frac{1}{\sqrt{3}}\left|\sum_{i=1}^{3}\mathrm{Tr}(A_{i}\otimes
B_{i}\hat{\rho}_{AB})\right|.$ (10)

In addition, coherence results from the superposition of quantum states and
can be used to assess the interference capability of interaction fields. In
optical systems, first-order coherence is a commonly used measure of
coherence. It is crucial to introduce the concept of first-order coherence,
which plays a significant role in understanding quantum correlations z49 ; z50
; z51 ; z52 . For the single-qubit state $\hat{\rho}_{k}$, its first-order
coherence can be quantified in terms of its purity z53 , i.e.,

$\displaystyle
Q\left(\hat{\rho}_{k}\right)=\sqrt{2\mathrm{Tr}\left(\hat{\rho}^{2}_{k}\right)-1}.$
(11)

Consider a two-qubit state ${\hat{\rho}}$, and its corresponding reduced
density matrices ${\hat{\rho}_{A}=\mathrm{Tr}_{B}(\hat{\rho})}$ and
${\hat{\rho}_{B}=\mathrm{Tr}_{A}(\hat{\rho})}$ are the reduced density
matrices. The first-order coherence for the state ${\hat{\rho}}$ can be
determined using

$\displaystyle
Q(\hat{\rho})=\sqrt{\frac{Q^{2}{(\hat{\rho}_{A})}+Q^{2}{(\hat{\rho}_{B})}}{2}}.$
(12)

In all, both the coherence and steerability can perfectly reflect the quantum
characteristics of the system of interest.

###  III.1 Ising model with DM interaction

First, we start with the simplest Ising model $({\rm i.e.,}\ \gamma=1,\
\Delta=0)$ and explore the effect of DM interactions on the QB’s performance.
It is necessary to determine a suitable time window to facilitate our
research. As mentioned before, the minimum time for a quantum battery to reach
its maximum ergotropy is ${t_{\min}}$. Therefore, to compare the two charging
models, parallel and collective charging, we consider the time window of
system evolution as ${\mathbf{\tau}}=\left[0,t_{\min}\right]$. Notably, the
ergotropy and power of QBs are regarded as two crucial aspects of evaluating
the performance of a QB. Besides, since ${t_{\min}}$ is the minimum charge
time for parallel charging if one achieves the maximum charge in a smaller
period of time than ${t_{\min}}$, we then say the quantumness of the system,
i.e., quantum correlation, plays a positive role in charging, which manifests
the quantum advantage.

To probe the influence of parallel and collective charging on ergotropy and
power, we first consider the cases without DM-interaction, and Fig. 2 has
depicted the evolution of the ergotropy and power as a function of the time
$t$, for $J=D=0$ (parallel charging mode) and $D=0$ and $J\neq 0$ (collective
charging mode). In the collective charging mode, the ergotropy of the battery
oscillatorily rises and reaches the peak ${\zeta_{\max}}$, whose value is
identical to that in the parallel charging mode at the same time. Notably,
although the ergotropy is the same for the two charging during a period
interval $[0,t_{\rm min}]$, the powers between them are different in the
interval. Specifically, for $t\in[0,\pi/(4\Omega)]$, the power in collective
charging is always higher than that in the parallel one; this can be
interpreted as the quantum correlation between cells increasing the charging
power of QBs in the current scenario.

Figure 2: When $D=0$, the evolution of ergotropy ${\zeta}$ and charging power
${P}$ evolve with time ${\Omega t}$ under collective charging and parallel
charging in the Ising model. The collective charging mode (orange solid line),
and the parallel charging mode (red dashed line). Herein, ${J=\Omega}$ is held
for the coupling strength and magnetic field.

We proceed by considering the effect of the DM interaction (i.e., ${D}$ $\neq$
0) on the QB’s performance in the Ising model. Fig. 3 plotted the variation of
the battery energy and power over time for different DM interaction
intensities. One can see that the ergotropy can faster reach peak at a moment
less than ${t_{\min}}$ with the stronger DM interaction strength, as shown in
Fig. 3(a). Additionally, the maximal powers are generally increased with the
growing ${D}$, indicated as Fig. 3 (b). With these in mind, we conclude that
the DM interaction helps enhance the QB’s performance in the Ising model.
Following Fig. 3, one can realize that the ergotropy can still reach its
maximum value (white area in the figure), when the value of $D$ is much larger
than $J$. Moreover, it is found that the increase of $D$ has little impact on
the ergotropy during the charging when $D$ is relatively strong.

Figure 3: (a)-(b) The ergotropy ${\zeta}$ and charging power ${P}$ evolve with
time ${\Omega t}$ in the Ising model. Varying the intensity of DM interactions
in the spin. The interaction intensities (in units of $\Omega$) $D=0$ (the red
dotted line), $D=3$ (orange dot-dashed line), $D=6$ (blue dashed line), $D=9$
(purple solid line) are set respectively. (c) The ergotropy versus the time
${\Omega t}$ and the interaction intensity $D$. Herein, ${J=\Omega}$ is held
for the coupling strength and magnetic field.

###  III.2 XXZ Model

Next, we consider how the effect of DM interaction on QB’s performance in the
XXZ model. As mentioned before, the examined system becomes an XXZ model with
DM interaction with respect to ${\gamma=0}$ and ${\Delta\neq 0}$. From Fig.
4(a), we can see that with the progress of charging, the charging speed of the
battery with DM interaction becomes faster, and the peak of ergotropy is
further increased. Compared with the case without DM interaction ${D=0}$, the
battery with DM interaction can achieve a better charging effect in a
relatively short time. Moreover, compared with the red line, the blue line
maintains a better trend in the long-term evolution. That is because the
energy between the systems is constantly transferred between the battery and
the charger (field) PhysRevB.99.035421, causing fluctuations in the peak
energy of the battery. DM interaction can facilitate energy transfer between
systems, making the fluctuation of the maximal ergotropy relatively small.
Furthermore, as for the power, Fig. 4(b) directly indicates that the maximal
power of QB with DM interaction is stronger than that without DM interaction.
Thereby, combining these points, the DM interaction can enhance ergotropy and
power in the current XXZ model, which agrees with the result in the previous
subsection.

Figure 4: The ergotropy ${\zeta}$ and charging power ${P}$ evolve with time
${\Omega t}$ in the XXZ model. Varying the intensity of DM interactions in the
spin. The interaction intensities (in units of $\Omega$) $D=0$ (the dashed red
line), and $D=1.7$ (solid blue line). Different timescales have been used to
properly show the maximum position of the related performance. Herein,
${J=\Omega}$ is held for the coupling strength and magnetic field. And here
the anisotropy parameter ${\Delta=2}$.

In addition, to explain why the DM interaction affects the performance of QB
in the above models, we explore the system’s quantumness, i.e., quantum first-
order coherence and steering. We plotted the first-order coherence of the
system over time in Fig. 5(a) and the steerability during this charging
process in Fig. 5(b), as for different DM interactions. The figure illustrates
that larger ${D}$ values correspond to larger first-order coherence throughout
the evolution window and weaker steerability between systems. As mentioned
previously, the DM interaction can enhance ergotropy and power of QBs.
Therefore, these suggest that the first-order coherence is in favour of the
enhancement of the QB’s performance, while the steering between batteries does
not positively impact battery energy storage.

Figure 5: In the XXZ model, the change of first-order coherence with time
${\Omega t}$ under different DM interaction intensities and the corresponding
change of battery steerability with time ${\Omega t}$ under different ${D}$
values. The DM interaction intensities (in units of $\Omega$) $D=0$ (the
dashed orange line), $D=0.8$ (the dot-dashed green line), $D=1.2$ (the dotted
blue line), and $D=1.7$ (the solid purple line) are set. Herein, ${J=\Omega}$
is held for the coupling strength and magnetic field and the anisotropy
parameter ${\Delta=2}$.

###  III.3 XYZ Model

In the above sections, we found that the DM interaction can strengthen the
QB’s ergotropy and power to some extent. Next, we consider what role the DM
interaction will play in the charging process of XYZ model. First, without the
DM-interaction with ${D=0}$, Fig. 6 describes the ergotropy and power as a
function of time $\Omega t$ with respect to ${\Delta=2.5}$ and ${\Delta=3}$
respectively. From the figure, although the maximums of both ergotropy and
power can reach with less time than ${t_{\min}}=\pi/(2\Omega)$, the magnitudes
will be smaller than those of the parallel charging. Conversely, we have
plotted the dynamics of ergotropy and power with the DM interaction in Fig. 7.
It shows that the ergotropy and power of the battery are greater than those
without DM interaction (red line in the figure); this implies that the DM
interaction can also improve the performance of QBs in the current
architecture.

Figure 6: For the XYZ model, time evolution for the ergotropy ${\zeta}$ of the
two-cell QB for different values of the anisotropy parameter in the absence of
the DM interaction strength. The anisotropy parameters $\Delta=2.5$ (dashed
blue line) and $\Delta=3$ (dotted red line). Herein, ${J=\Omega}$ is set.

Figure 7: For the XYZ model, the ergotropy ${\zeta}$ and charging power ${P}$
evolution with time when anisotropy parameters ${\Delta=2.5}$ and
${\Delta=3}$. The interaction intensities (in units of $\Omega$) $D=0$ (the
red dotted line), $D=0.5$ (blue dashed line), $D=1$ (purple solid line).
${J=\Omega}$ is set.

##  IV Discussions and conclusions

In summary, we have investigated the model of quantum batteries with one-
dimensional Heisenberg spin chains with and without DM interactions.
Explicitly, we examined and compared the properties of two types of charging
models, i.e., collective charging and parallel charging, in detail, and
unveiled the influence of DM interaction on the charging process of the
quantum battery. Interestingly, we found that the DM interaction can increase
the ergotropy and charging power of the battery in the three spin-chain
models, i.e., the Ising, XXZ, and XYZ models. Thus, we claimed that it could
improve the charging performance of QBs in the current architectures,
indicating a better charging effect can be obtained during collective charging
compared with parallel charging. Meanwhile, it is found that the first-order
coherence between the systems is a useful resource for battery charging, while
the steering between the batteries is not conducive to the storage of battery
energy. Moreover, it is argued that the current investigation, in principle,
might be generalized to $N$-cells QBs. Theoretically, the number of cells
directly affects how much energy a battery can store, and generally, more
cells are beneficial to store more energy in the QBs z1 ; z54 ; z55 . Thus, we
conjecture that the DM interaction can further enhance the battery performance
as the number of cells increases and possibly induce quantum phase transitions
and critical behaviours z56 . Overall, our investigation sheds light on the
positive role of the DM interaction in quantum batteries under the spin-chain
models and is beneficial for understanding the properties of practical spin-
based quantum batteries.

###### Acknowledgements.

This work was supported by the National Natural Science Foundation of China
(Grant Nos. 12075001 and 61601002), and Anhui Provincial Key Research and
Development Plan (Grant No. 2022b13020004).

## References

  * (1) F. Campaioli, S. Gherardini, J. Q. Quach, M. Polini, G. M. Andolina, arXiv:2308.02277
  * (2) L. Wang, S.-Q. Liu, F.-L. Wu, H. Fan, S.-Y. Liu, Phys. Rev. A 2023, 108, 062402.
  * (3) W.-L. Song, H.-B. Liu, B. Zhou, W.-L. Yang, J.-H. An, Phys. Rev. Lett. 2024, 132, 090401.
  * (4) B. Çakmak, Phys. Rev. E 2020, 102, 042111.
  * (5) S. Julià-Farré, T. Salamon, A. Riera, M. Bera, M. Lewenstein, Phys. Rev. Res. 2020, 2, 023113.
  * (6) L. Fusco, M. Paternostro, G. D. Chiara, Phys. Rev. E 2016, 94, 052122.
  * (7) F. Barra, Phys. Rev. Lett. 2019, 122, 210601.
  * (8) D. Farina, G. M. Andolina, A. Mari, M. Polini, V. Giovannetti, Phys. Rev. B 2019, 99, 035421.
  * (9) J. Q. Quach, Wi. J. Munro, Phys. Rev. Appl. 2020, 14, 024092. .
  * (10) R. Alicki and M. Fannes, Phys. Rev. E 2013, 87, 042123.
  * (11) T. F. F. Santos, Y. V. D. Almeida, M. F. Santos, Phys. Rev. A 2023, 107, 032203.
  * (12) M.-L. Song, L.-J. Li, X.-K. Song, L. Ye, D. Wang, Phys. Rev. E 2022, 106, 054107.
  * (13) M.-L. Song, X.-K. Song, L. Ye, and D. Wang, Phys. Rev. E 2024, 109, 064103.
  * (14) S. Gherardini, F. Campaioli, F. Caruso, F. C. Binder, Phys. Rev. Res. 2020, 2, 013095.
  * (15) F. Binder, S. Vinjanampathy, K. Modi, J. Goold, Phys. Rev. E 2015, 91, 032119.
  * (16) K. V. Hovhannisyan, M. Perarnau-Llobet, M. Huber, A. Acín, Phys. Rev. Lett. 2013, 111, 240401.
  * (17) F. H. Kamin, F. T. Tabesh, S. Salimi, Alan C. Santos, Phys. Rev. E 2020, 102, 052109.
  * (18) F.-Q. Dou, Y.-Q. Lu, Y.-J. Wang, J.-A. Sun, Phys. Rev. B 2022, 105, 115405.
  * (19) F. C. Binder, S. Vinjanampathy, K. Modi, J. Goold, New Journal of Physics 2015, 17, 075015.
  * (20) W. J. Zhang, S. Y. Wang, C. F. Wu, G. C. Wang, Phys. Rev. E 2023, 107, 054125.
  * (21) S. S. Seidov, S. I. Mukhin, Phys. Rev. A 2024, 109, 022210.
  * (22) K. Sen, U. Sen, Phys. Rev. A 2021, 104, L030402.
  * (23) A. C. Santos, Phys. Rev. E 2021, 103, 042118.
  * (24) Y.-Y. Zhang, T.-R. Yang, L. Fu, X.G. Wang, Phys. Rev. E 2019, 99, 052106.
  * (25) F.-Q. Dou, H. Zhou, J.-A. Sun, Phys. Rev. A 2022, 106, 032212.
  * (26) L. Gao, C. Cheng, W.-B. He, R. Mondaini, X.W. Guan, H.-Q. Lin, Phys. Rev. Res. 2022, 4, 043150.
  * (27) G. M. Andolina, M. Keck, A. Mari, V. Giovannetti, M. Polini, Phys. Rev. B 2019, 99, 205437.
  * (28) W. Chang, T.-R. Yang, H. Dong, L.B. Fu, X. G. Wang, Y.-Y. Zhang, New Journal of Physics 2021, 23, 103026.
  * (29) L. Peng, W.-B. He, S. Chesi, H.-Q. Lin, X.-W. Guan, Phys. Rev. A 2021, 103, 052220.
  * (30) F. Pirmoradian, K. Mølmer, Phys. Rev. A 2019, 100, 043833.
  * (31) J.-Y. Gyhm, D. Šafránek, D. Rosa, Phys. Rev. Lett. 2022, 128, 140501.
  * (32) J. Kim, J. Murugan, J. Olle, D. Rosa, Phys. Rev. A 2022, 105, L010201.
  * (33) F. Campaioli, F. A. Pollock, F. C. Binder, L. Céleri, J. Goold, S. Vinjanampathy, K. Modi, Phys. Rev. Lett. 2017, 118, 150601.
  * (34) T. P. Le, J. Levinsen, K. Modi, M. M. Parish, F. A. Pollock, Phys. Rev. A 2018, 97, 022106.
  * (35) H.-L. Shi, S. Ding, Q.-K. Wan, X.-H. Wang, W.-L. Yang, Phys. Rev. Lett. 2022, 129, 130602.
  * (36) I. Dzyaloshinsky, Journal of Physics and Chemistry of Solids 1958, 4, 241-255.
  * (37) T. Moriya, Phys. Rev. 1960, 120, 91–98.
  * (38) A. C. Santos, A. Saguia, M. S. Sarandy, Phys. Rev. E 2020, 101, 062114.
  * (39) A. E. Allahverdyan, R. Balian, Th. M. Nieuwenhuizen, Europhysics Letters 2004, 67, 565.
  * (40) A. C. Santos, B. Çakmak, S. Campbell, N. T. Zinner, Phys. Rev. E 2019, 100, 032107.
  * (41) R. Uola, A. C. S. Costa, H. C. Nguyen, O. Gühne, Rev. Mod. Phys. 2020, 92, 015001.
  * (42) A. F. Abouraddy, M. B. Nasr, B. E. A. Saleh, A. V. Sergienko, M. C. Teich, Phys. Rev. A 2021, 63, 063803.
  * (43) P. S. Burada, G. Schmid, Phys. Rev. E 2010, 82, 051128.
  * (44) N. Brunner, D. Cavalcanti, S. Pironio, V. Scarani, S. Wehner, Rev. Mod. Phys. 2014, 86, 419–478.
  * (45) M. T. Quintino, T. Vértesi, Da. Cavalcanti, R. Augusiak, M. Demianowicz, A. Acín , N. Brunner, Phys. Rev. A 2015, 92, 032107.
  * (46) H. M. Wiseman, S. J. Jones, A. C. Doherty, Phys. Rev. Lett. 2007, 98, 140402.
  * (47) E. G. Cavalcanti, S. J. Jones, H. M. Wiseman, M. D. Reid, Phys. Rev. A 2009, 80, 032112.
  * (48) A. C. S. Costa , R. M. Angelo, Phys. Rev. A 2016, 93, 020103.
  * (49) K. H. Kagalwala, G. D. Giuseppe, A. F. Abouraddy, B. E. A. Saleh, Nature Photonics 2012, 7, 72-78.
  * (50) J. K. Kalaga, W. Leoński, J. Peřina, Phys. Rev. A 2018, 97, 042110.
  * (51) D.-D. Dong, X.-K. Song, X.-G. Fan, L. Ye, D. Wang, Phys. Rev. A 2023, 107, 052403.
  * (52) X.-G. Fan, W.-Y. Sun, Z.-Y. Ding, F. Ming, H. Yang, D. Wang, L. Ye, New Journal of Physics 2019, 21, 093053.
  * (53) L. Mandel, E. Wolf, Optical Coherence and Quantum Optics, (Cambridge University Press 1995).
  * (54) D. Ferraro, M. Campisi, G. M. Andolina, V. Pellegrini, M. Polini, Phys. Rev. Lett. 2018, 120, 117702.
  * (55) W.-X. Guo, F.-M. Yang, F.-Q. Dou, Phys. Rev. A 2024, 109, 032201.
  * (56) F.-W. Ma, S.-X. Liu, X.-M. Kong, Phys. Rev. A 2011, 84, 052109.