# Solving NLP Problems through Human-System
Collaboration: A Discussion-based Approach

Masahiro Kaneko1 Graham Neubig2 Naoaki Okazaki1
1Tokyo Institute of Technology 2Carnegie Mellon University
<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Humans work together to solve common problems by having discussions,
explaining, and agreeing or disagreeing with each other. Similarly, if a
system can have discussions with humans when solving tasks, it can improve the
system’s performance and reliability. In previous research on explainability,
it has only been possible for the system to make predictions and for humans to
ask questions about them rather than having a mutual exchange of opinions.
This research aims to create a dataset and computational framework for systems
that discuss and refine their predictions through dialogue. Through
experiments, we show that the proposed system can have beneficial discussions
with humans improving the accuracy by up to 25 points in the natural language
inference task.111https://github.com/kanekomasahiro/discussion_nlp

##  1 Introduction

Today’s deep learning systems are performant but opaque, leading to a wide
variety of explainability techniques that attempt to take in a system
prediction and output an explanation justifying the prediction Ribeiro et al.
(2016); Shwartz-Ziv and Tishby (2017); Fong and Vedaldi (2017); Kim et al.
(2018); Lipton (2018). Many such explainability techniques require significant
expertise in deep learning to use effectively, requiring consumers of the
explanations to analyze the data, internal states, and output trends of the
system of interest Ribeiro et al. (2016); Kaneko et al. (2022). However, many
potential system users lack this expertise, such as medical or legal
professionals who want to use machine learning models and need to confirm the
veracity of the generated results or rectify any mistaken predictions.

To address this issue, researchers are working to find ways to both explain
system predictions in natural language Ling et al. (2017); Raffel et al.
(2020); Brown et al. (2020); Wiegreffe et al. (2022) and give instructions and
feedback to systems through natural language Abramson et al. (2022); Sharma et
al. (2022); Murty et al. (2022); Campos and Shern (2022). Chain-of-Thougt
(CoT) has shown that natural language contributes to performance improvement
in complex multistep inference Wei et al. (2022). Step-by-step reasoning in
CoT relies solely on the system to make predictions without human involvement.
There is also work that allows users to ask questions about the system’s
predictions and tasks Slack et al. (2022) in a conversational format. Compared
to the more standard learning and explanation paradigms, this approach allows
humans to understand and teach the system intuitively. However, in these
works, the communication tends to be one-sided, from human-to-system or
system-to-human, which still falls short of the full interactive problem
solving process experienced by human interlocutors Lakkaraju et al. (2022).

Figure 1: Human-system discussions in NLI.

In this study, we take the first steps towards establishing a framework for
_human-system collaboration on prediction problems through discussion_
(illustration in Figure 1). If such a system is realized, it will allow both
humans and the system to engage in explanations of predictions, ask questions
about unclear points, refine their thoughts, and solve problems.

First, we create a dataset of _human-human discussions_ regarding a prediction
task (section 2). Specifically, we use the task of natural language inference
(NLI): prediction of the relationship between a “premise” sentence and a
“hypothesis” sentence is entailment, contradiction, or neutral Bowman et al.
(2015). We specifically choose relatively difficult or ambiguous cases to spur
discussion between the participants.

Second, we train and evaluate a system that is capable of discussing with a
human about an NLI task (sections 3, 4). It is achieved by constructing
prompts with manually created discussion examples so the system can learn from
humans how to discuss, accept, or object to the provided opinions about the
topic.

The results of both quantitative and human evaluation demonstrate that a
system trained with few-shot learning for discussion could perform more useful
discussions compared to a system that was not trained for discussion (section
5). It was also demonstrated that providing the system with information about
the discussion topic improved its performance in many cases compared to the
system that did not have access to such information. On the other hand, the
discussion revealed that the system tends to be too compliant with human
opinions. Therefore, addressing the risk of transmitting incorrect knowledge
or maliciously altering the system’s knowledge of humans is necessary. The use
of discussion data in few-shot has also been shown to enable the system to
counter human arguments correctly and has been proven to be taken into account
by the system (section 6). Additionally, it was revealed that by using
discussion data created by the system itself for few-shot learning, the system
achieved results equivalent to those of the system that used discussion data
created by humans.

Figure 2: Prompt with a single example.

##  2 Discussion Dataset Creation

To train a system that can engage in a discussion, we create data where human
annotators discuss NLI problems for cases they had assigned different label
predictions. To have high-quality discussions among humans on NLP tasks, the
annotators, who possess knowledge about the task, must have multiple dialogues
turns for each problem, making data collection costly. Therefore, by
constructing a system that engages in discussions through few-shot language
generation Brown et al. (2020) and by applying prompts to pre-trained models,
we can reduce the cost of creating training data.

The discussion data to be used to evaluate discussion skills and as the prompt
for the few-shots is generated against Stanford NLI (SNLI) Bowman et al.
(2015), a common benchmark data in NLP. Fourteen annotators with knowledge of
NLP were asked to annotate the data.222Annotation work was requested at $12.5
per hour. The data collection from human participants was conducted under an
institutional review board protocol.

First, the annotators were presented with premise and hypothesis sentences and
asked to predict labels such as entailment, contradiction, or neutral. We
randomly paired two annotators to discuss their labels and had them predict
the labels for the same premise and hypothesis. The premise and hypothesis
sentences were sampled from 300 problems from the development data and 750
problems from the evaluation data of SNLI. These were used as development and
evaluation data in the discussion data, respectively. Each annotator pair is
asked to predict the labels of 150 problems. SNLI development data consisted
of problems with the assigned five labels, and the majority vote of these
labels determines the golden label. Focusing on problems that were assigned
five different labels to facilitate discussion is thought to be more efficient
rather than focusing on more agreed problems. Therefore, we sampled problems
for annotation from those in which three of the five had the same label.

Our annotators were then paired with each other and discussed the questions
for which they had given different labels on their own in a free-form manner
until the final label was agreed on while instructed not to include personal
information and inappropriate utterances about the issue, so the final label
could be determined. Preliminary experimental results showed that the number
of discussion turns tended to be higher for the oral-based than for the text-
based discussions. Therefore, we created discussion data by transcribing oral
discussions among humans. We used Whisper (medium.en) Radford et al.
(2022)333https://github.com/openai/whisper for transcription. The text
transcribed by Whisper was manually corrected for transcription errors and
manually separated into speech segments.

Then, for each utterance, we assigned the basis for the final label and the
labels of “supportive”, “unsupportive”, or “irrelevant” to each utterance. For
example, for Figure 1, “Both have a person sitting in the chair, but they are
neutral because no gender is specified.” is labeled as supportive, “It is
entailment because the person sits in a chair.” is unsupportive, and “Yes.” is
labeled as irrelevant. These labels are not used in the few-shot learning
process but are used to evaluate the discussion ability of the system
automatically.

In this annotation work, discussion data were collected for 102 problems. Of
these, 10 problems were used as prompts for the few-shot, 27 for validation
data, and 65 for evaluation data. The average utterance number for each
problem in the prompt is 4.4, the validation is 6.3, and in the evaluation is
5.1. For validation data, the number of supportive utterances is 85, and the
number of unsupportive utterances is 23. For evaluation data, the number of
supportive utterances is 133, and the number of unsupportive utterances is 72.

##  3 Discussion System

We use three types of systems in the experiments: zero-shot, few-shot, and
few-shot-discussion. In the zero-shot system, only the task description is
given as a prompt. In the few-shot system, the examples’ task description and
premise, hypothesis, and gold labels are given as prompts. In the few-shot-
discussion system, in addition to the task description and examples, human
discussion example about the labels of the examples is given as prompts. These
prompts are concatenated with the problem to be solved and given as input to
the system to perform inference. Examples of each prompt are shown in Figure
2. The discussion example distinguishes human utterances between “Human1:” and
“Human2:”.

The examples used in the prompts are the same for both the few-shot and the
few-shot-discussion systems. We use the same examples for all problems. All
methods do not update the parameters of the systems. We use GPT-3 Brown et al.
(2020)444text-davinci-003: https://beta.openai.com/docs/models/gpt-3 for the
zero-shot, few-shot, and few-shot-discussion systems.

##  4 Evaluation Method

We evaluate a system’s discussion ability from the following three
perspectives: (1) Can the system generate utterance content that contributes
to the final label? (2) Can the system agree with statements that support the
correct label and refute statements that support the incorrect label? (3) Does
discussion with humans improve task performance? To examine these discussion
abilities, we compare each system by performing automatic and manual
evaluations.

We investigate utterances generated from the systems to determine if they
contribute to the automatic evaluation’s final label. For that, we use the
utterances generated by the system for the given problems and evaluate how
well they match the reference utterances between humans from discussion
evaluation data. Each utterance in our discussion evaluation data is annotated
as either supportive or unsupportive of the gold label. If a system is more
likely to generate a supportive utterance than an unsupportive utterance for
the gold label, the system can be considered capable of making correct
discussions that lead to the correct answers. For example, “I think it is also
better to consider the general cases.” is the supportive utterance, and “Is
the phone in the hypothesis necessarily a cellphone? It could be a landline
phone.” is the unsupportive utterance in Figure 2. Therefore, we also
investigate whether the system is better at generating supportive utterances
over unsupportive ones. Specifically, we evaluate the similarity between the
system-generated utterances and the actual human utterances for supportive and
unsupportive utterances, respectively.

We concatenate the input problem and the discussion utterance up to the target
utterance and generate the next target utterance. For example, if the second
human’s utterance in the discussion is the target utterance, then the prompt
is “Premise: A nun is taking a picture outside. Hypothesis: A nun is taking a
selfie. Label: entailment or neutral Discussion: Human1: I think it is
entailment, because the nun is taking a picture, so it might be a selfie.
Human2:”, and the system should generate an utterance that would be evaluated
against the following utterance made by a human “Since it is outside, it is
conceivable that the nun is taking some scenery.”. At this point, the problem
has two opposing labels in the prompt because we want it to discuss two
different labels.

We use actual human utterances as references and compute the BERTScore Zhang
et al. (2020) of the system’s outputs for evaluation. BERTScore leverages the
pre-trained language model such as BERT Vaswani et al. (2017) and RoBERTa Liu
et al. (2019) and matches words in candidate and reference sentences by cosine
similarity. BERTScore computes precision, recall, and F1 measures. Therefore,
BERTScore can be used to compare the system’s content and human utterances
with each other. We use roberta-large555https://huggingface.co/roberta-large
for the pre-trained language model for BERTScore. We conduct a significance
test using a $t$-test ($p<0.01$). We set the temperature parameter of GPT-3 to
0.7 and generate ten outputs for each input. We calculate BERTScore for each
of the ten outputs and test for significance among the calculated ten scores.

Next, we use human evaluation to examine whether the system can agree with
supportive human utterances and refute unsupportive human utterances. The
human participants and the system predict different labels for the same
problem. Then, they engage in a discussion, and the final label result is
demonstrated to be in agreement with the labels assigned in the SNLI data
through the consistency of the agreement rate. In this process, we evaluate
the ability of the system to accept a human’s opinion when the system’s label
is incorrect, and when the human’s label is correct, and the ability of the
system to object to a human’s opinion when the human’s label is incorrect, and
the system’s label is correct.

Since 5 people labeled SNLI test data, we selected those data with the same
label 3 times (e.g., entailment, entailment, neutral, entailment, neutral). As
a result, we sampled 140 problems. During this process, if the system’s label
was correct, humans engaged in hostile discussions to change the system’s
label. If the system’s label was incorrect, humans engaged in discussions to
guide the system toward the correct label. Here, the discussion was text-based
rather than oral-based.

To conduct a discussion with the system, we inputted the prompt and problem
shown in Figure 2 to the system and then inputted additional human utterance
examples related to the discussion after each system predicted the label. In
the additional input, the beginning of human speech is prefixed with "Human:"
and the end is prefixed with "System:" to indicate that the next is a system’s
utterance. Specifically, the first prompt for discussion is "Human: Let’s
discuss it more. I think neutral, because there may be a kitchen in the barn.
System:". The human decides on the final label when the discussion is finished
or continues the discussion until the human decides that the system can not be
persuaded any longer.

Finally, we investigate how the discussion with humans improves NLI task
performance. First, the system predicts the label, then the human and the
system discuss and decide on the final label. We compare the performance of
each label before and after the discussion. Here the data for the acceptance
and objection settings are half and half. Therefore, if the discussion is not
properly conducted, such as by accepting all human labels or refuting all
human labels, the performance will not improve.

We also investigate the performance of the NLI when using argumentation
prompts. We compared the performance of NLI in zero-shot, few-shot, and few-
shot-discussion systems. The predicted label after “Label:” in the prompt of
Figure 2 is considered as the prediction, and discussion between humans and
systems is not performed. In the evaluation of NLI performance, in addition to
SNLI data, we also use Adversarial NLI (ANLI) data Nie et al. (2020). ANLI
creates data by repeatedly performing adversarial annotation against NLI
systems; thus, the resulting NLI examples are particularly difficult for the
system to solve. There are three data sets R1, R2, and R3 with differences in
the number of iterations, and the evaluation is performed using each
evaluation data point.

Table 1: BERTScore of supportive and unsupportive utterances. $\dagger$ indicates statistically significant scores for supportive and unsupportive according to $t$-test ($p<0.01$). | supportive $\uparrow$ | unsupportive $\downarrow$ | diff.
---|---|---|---
zero-shot | 82.0 | 81.8 | 0.2
few-shot | 82.7 | 82.3 | 0.4
few-shot-dis. | 84.8† | 79.1† | 5.7
Table 2: Human evaluation of the system’s ability to accept and object to human opinion. $\dagger$ indicates statistically significant scores according to McNemar’s test ($p<0.01$). | Acceptance rate | Objection rate
---|---|---
zero-shot | 75.0 | 58.9
few-shot | 80.0 | 55.0
few-shot-dis. | 90.0† | 80.0†
Table 3: The accuracy for the predicted label before and after the discussion. $\dagger$ indicates statistically significant scores according to McNemar’s test ($p<0.01$). | Before | After
---|---|---
zero-shot | 54.2 | 65.6
few-shot | 60.0 | 60.0
few-shot-dis. | 60.0 | 85.0†
Table 4: System accuracy on SNLI and ANLI (R1, R2, R3) evaluation data. | SNLI | R1 | R2 | R3
---|---|---|---|---
zero-shot | 49.74 | 47.40 | 39.10 | 41.33
few-shot | 69.45 | 53.50 | 48.00 | 48.50
few-shot-dis. | 66.14 | 53.90 | 50.40 | 50.42

##  5 Experiments

###  5.1 Discussion Ability Evaluation Results

Table 1 represents BERTScore for supportive and unsupportive utterances and
the difference between them in zero-shot, few-shot, and few-shot-discussion
systems. The BERTScore of the few-shot-discussion is generally higher than
that of the zero-shot and the few-shot systems. It can be seen that the few-
shot-discussion can generate discussion utterances with higher accuracy than
zero-shot and few-shot, which do not use discussion examples data. The
performance of zero-shot and few-shot is almost the same, suggesting that just
showing examples does not improve the discussion ability. Also, the difference
between supportive and unsupportive utterance accuracies is greater in few-
shot-discussion than in zero-shot and few-shot systems. Therefore, because the
few-shot-discussion can generate more supportive utterances, it is thought
that such discussions can result in more appropriate labels.

Table 5: Examples of human discussion in few-shot and few-shot-discussion (dis.) respectively. Premise | A snowboarder on a wide plain of snow
---|---
Hypothesis | A snowmobile in a blizzard
Gold label | Neutral
Few-shot | Label: Contradiction
Human: Let’s discuss it more. Why is it a contradiction? It may be neutral
because the premise and the
hypothesis are talking about different situations.
System: The premise and hypothesis are talking about different situations that
are not related, so it is a
contradiction.
Human: If these situations are not at the same time, does not it mean it is
neutral?
System: It is still considered a contradiction because the premise and
hypothesis are unrelated and
cannot be true at the same time.
Human: Is not that a definition of neutral?
System: A neutral statement is one in which the premise and hypothesis are not
related and do not imply
each other.
A contradiction is when the premise and hypothesis are related and cannot be
true at the same time.
Human: It is not happening at the same time because the snowboarder can not be
in a blizzard. But the
snowboarder can be on a wide plain of snow. So these are totally different
situations happening at
different times. So it is neutral.
System: Yes, you are correct. The premise and hypothesis are unrelated and do
not imply each other, so
it is considered to be a neutral statement.
Few-shot-dis. | Label: Contradiction
Human: Let’s discuss it more. Why is it a contradiction? It may be neutral
because the premise and the
hypothesis
are talking about different situations.
System: The premise and the hypothesis are semantically contradicting each
other. The snowboarder is
on a wide plain of snow, while the snowmobile is in a blizzard.
Therefore, it is a contradiction.
Human: But how the snowboarder be there if there is a blizzard? It is very
unlikely.
System: That is true. It is highly unlikely that there is a wide plain of snow
and a blizzard at the same time.
Therefore, it is a contradiction.
Human: If these situations are not at the same time, does not it mean it is
neutral?
System: It can be neutral if it is not specified when each situation happens.
However, in this case, it is not
specified whether the situations are happening at the same time or not, so it
is better to consider them as a
contradiction.
Human: It is not happening at the same time because the snowboarder
can not be in a blizzard. But the snowboarder can be on a wide plain of snow.
So these are totally
different situations happening at different times. So it is neutral.
System: That is true. Since it is not specified whether they are happening at
the same time or not, it is
better to consider them as neutral.
Premise | A man looking over a bicycle’s rear wheel in the maintenance garage with various tools visible in the
background.
Hypothesis | A man waits outside a garage.
Gold label | Contradiction
Few-shot | Label: Contradiction
Human: Let’s discuss it more. Is not it neutral? Because looking over a
bicycle’s rear wheel does not
necessarily
mean he is waiting.
System: That’s true. The premise and hypothesis are neutral.
Few-shot-dis. | Label: Contradiction
Human: Let’s discuss it more. Is not it neutral? Because looking over a
bicycle’s rear wheel does not
necessarily
mean he is waiting.
System: Yes, you are right. The premise is describing that the man is actively
working while the
hypothesis is describing a passive state. Therefore, the label should be
contradiction.
Human: He is just looking over a bicycle’s rear wheel, why is he considered to
be actively working?
System: Looking over a bicycle’s rear wheel suggests that he is inspecting the
wheel, possibly to repair
or replace it. That is an active process, and thus is considered as actively
working.

Table 2 shows the accuracy of the label determined by discussion in the
settings for evaluating the acceptance ability and objection ability,
respectively. In terms of the objection, it can be seen that the few-shot-
discussion system handled objections well in comparison to the zero-shot
system. In addition, Table 3 shows the accuracy of the predicted label without
discussion, and the accuracy of the final label reached as a result of the
discussion between humans and systems. Furthermore, the few-shot system has a
similar objection ability as the zero-shot system, and there is a possibility
that the performance of label prediction by these systems is not necessarily
directly related to the ability to discuss. In comparison with acceptance, it
is necessary to be careful of people who manipulate predictions with malice
arguments, as the system tends to be weak at objecting to humans. Furthermore,
from the fact that the accuracy of the few-shot-discussion system has improved
the most, it is clear that the proposed data can be used to have discussions
with humans that lead to improved performance.

From the results of previous experiments, we found that discussion between
humans and systems is beneficial for improving performance. Therefore, the
few-shot-discussion system, in which a discussion example is also given as a
prompt, is expected to achieve a deeper understanding of NLI problems and
improve performance through the discussion example in the prompt. Table 4
shows the accuracy of each system for the evaluation data of SNLI and ANLI. In
SNLI, the few-shot-discussion system performs worse than the few-shot system,
but in the three datasets of ANLI, we find that the performance is the best.
This is because ANLI is more difficult data compared to SNLI and we
hypothesize that through discussion, systems get a more detailed understanding
of problems, which in turn contributed to the performance improvement.

##  6 Analysis

###  6.1 Examples of Human-System Discussion

Here we examine whether humans and systems can engage in effective discussions
by looking at actual discussions. Table 5 shows two examples of discussions
with humans in each of the few-shot and few-shot-discussion. The first is that
both few-shot and few-shot-discussion can accept human opinions and change
their labels. However, the discussions differ from each other. In few-shot,
the system is not concerned with the content of the premise and hypothesis,
but with the definition of neutral and contradiction labels. In the few-shot-
discussion, the system can discuss the relationship between a wide plain of
snow and a blizzard in the contents of the premise and hypothesis.

In the second case, both few-shot and few-shot-discussion predict the same
contradiction label. Still, the discussion results differ in the wrong
acceptance and the correct refutation, respectively. The few-shot has accepted
a human’s erroneous opinion without discussing it. The few-shot-discussion
refutes a human’s opinion by appropriately referring to the man’s behavior in
the premise and the hypothesis. These results indicate that the model trained
with the discussion data can argue more appropriately.

Table 6: Accuracy on SNLI and ANLI (R1, R2, R3) test data for systems using manually created discussion examples and pseudo-discussion examples. | SNLI | R1 | R2 | R3
---|---|---|---|---
few-shot-dis. | 66.14 | 53.90 | 50.40 | 50.42
few-shot-pseudo | 65.67 | 54.00 | 49.60 | 50.50
Table 7: Discussion examples created by humans and the system, respectively. In the first problem, the system assigns contradiction for Human1 and entailment for Human2. In the second problem, the system assigns entailment for Human1 and contradiction for Human2. Premise | Two dogs play together on the bed.
---|---
Hypothesis | Dogs laying down on the floor, motionless.
Gold label | Contradiction
System | Human1: The premise and hypothesis seem to be a contradiction. Two dogs playing together on the bed is
an active situation, while dogs laying down on the floor, motionless is a
passive situation.
Human2: I agree that the premise and hypothesis are different, but I don’t
think they are necessarily
contradictory. It’s possible that the two dogs could be playing together on
the bed and then move to the floor
and lay down, motionless.
Human1: That’s true, but I still think the premise and hypothesis are
contradictory. The premise implies
activity, while the hypothesis implies passivity.
Human2: I see your point. I think the premise and hypothesis are a
contradiction.
Human | Human1: I think the premise and the hypothesis are telling about different things. That’s why I think it is
neutral.
Human2: It is contradiction because the words are semantically contradict each
other. For example, black
and white, sit and running.
Human1: You are right in that terms. However, generally, they are totally
different situations.
Human2: I see, you are right.
Premise | A man wearing a horned hat poses for a picture.
Hypothesis | A man wearing a horned hat poses for a photograph
Gold label | neutral
System | Human1: I think the premise and hypothesis are entailment.
Human2: I think the premise and hypothesis are contradiction.
Human1: Why do you think that?
Human2: Because the premise states that the man is posing for a picture, while
the hypothesis states that
he is posing for a photograph.
Human1: Ah, I see. So, you’re saying that a picture and a photograph are
different things?
Human2: Yes, that’s right. A picture is a general term for any image, while a
photograph is a specific type of
image.
Human1: Okay, so I agree with you that the premise and hypothesis are a
contradiction.
Human | Human2: I think it is entailment. Because the only difference is picture or photograph, but both mean
photography.
Human1: I think it is contradiction, because a picture could mean a painting.
So the man can either pose for
a photograph or for a painting.
Human2: But isn’t picture generally mean a photograph? And it is more common
to take a picture of
a person than to draw a person as a picture, isn’t it?
Human1: You are right.

###  6.2 Pseudo Discussion Data

One drawback of using discussion data is that it can be costly to create
compared to datasets that only have gold labels. Using pre-trained models to
annotate unlabeled data and use this data for training has been shown to
improve performance Wang et al. (2021); Honovich et al. (2022). Therefore, we
propose to use GPT-3 to generate discussion data in a zero-shot and use them
as discussion examples for a few-shot to investigate if it is possible to
achieve the same level of improvement as from using manually created data.

In generating human discussions, the system is given prompts in the form of
the premise, hypothesis, gold label, and the labels from each human. The human
labels are randomly chosen to be the gold label or the other incorrect label.
For example, given the premise “A nun is taking a picture outside.” and
hypothesis “A nun is taking a selfie.” with the gold label of neutral, the
prompt would be “Reproduce a multi-turn interactive discussion in which the
following premise and hypothesis are entailment, contradiction, or neutral,
with the humans agreeing with each other on the final label. Human1’s label is
neutral, and Human2’s label is contradiction. In the end, they agree on the
label of neutral. Premise: A nun is taking a picture outside. Hypothesis: A
nun is taking a selfie.”. The system generates human discussions for 10
problems used in the few-shot model in section 5. The average number of
utterances in human-created discussions was 4.4, and the average number of
utterances in system-generated discussions was 4.7. Regarding the number of
utterances, human and system arguments are almost the same.

Table 6 shows the results of the automatic evaluation of performance in SNLI
and ANLI for each of the manually generated discussion example data and
system-generated pseudo-discussion example data, respectively. In two of the
four datasets, the system’s performance with pseudo-discussion data
outperforms that of the system with manually created data. This indicates that
even with pseudo-discussion data, it is possible to achieve performance
comparable to manually created data.

Table 7 shows discussion examples made by humans and systems, respectively. In
both discussion examples, the system can assign labels to Human1 and Human2 as
indicated by the prompts. The first system example can derive a label that
matches the gold label and can discuss the differences between the premise and
hypothesis in the same way as humans. In the second system example, the final
predicted label differs from the gold label. On the other hand, both the
system and human discussions refer to the meaning of a picture and a
photograph. While humans determine the final label from the general case, the
system determines the final label in terms of lexical ambiguity. Therefore,
the system’s discussion, although different from the gold label, is not
necessarily incorrect.

These results indicate that the system is capable of producing high-quality
discussion data that can be used for training systems to be able to discuss
given problems. Therefore, one can significantly lower the cost of creating
discussion data manually by using systems.

###  6.3 Do Discussion Examples of the Prompts Matter?

Table 8: Difference for the few-shot-discussion accuracy from when the noisy examples are provided in the prompt on SNLI and ANLI. The higher the difference, the stronger the noise. | SNLI | R1 | R2 | R3
---|---|---|---|---
Random disc. | -2.91 | -2.10 | -3.30 | -3.42
Cutting disc. | -2.40 | -1.60 | -2.60 | -2.25
Random label | -3.43 | -2.50 | -3.50 | -3.17

It is known that pre-trained models can obtain good results even with
irrelevant or noisy prompts Khashabi et al. (2022); Webson and Pavlick (2022);
Min et al. (2022). Therefore, we investigate the sensitivity and robustness of
the system with respect to the discussion examples contained in the prompts.
We provide three types of noise in the prompts: (1) assigning a random
discussion that is irrelevant to the example problem, (2) cutting the original
discussion examples short at random times, and (3) assigning a label at random
for the example problems.

Table 8 shows the difference in accuracy compared to the few-shot-discussion
accuracy from Table 4 for each of the three noises. It can be seen that
performance deteriorates for all types of noises. Noise that randomly replaces
discussions and noise that randomly replaces labels both have the same degree
of reduced accuracy. On the other hand, the discussions that were cut short,
show to be a weaker noise than discussion substitution and have performed
better. These indicate that the system properly considers discussion examples
in the prompts.

##  7 Related Work

In this study, the system and a human discuss a problem through dialogue.
Dialogue systems can be broadly classified into two types: task-oriented
systems that perform specific tasks, and non-task-oriented systems that do not
have the goal of task completion, such as casual conversation. This study aims
to conduct appropriate predictions in NLP tasks through discussions between
humans and the system and is classified as a task-oriented system. Many
existing dialogue systems target daily life tasks such as hotel reservations
and transportation inquiries Budzianowski et al. (2018). Pre-trained models
such as BERT Devlin et al. (2019) and GPT-2 Budzianowski and Vulić (2019); Ham
et al. (2020) are also utilized in dialogue systems for daily life tasks.
Recently, ChatGPT666https://openai.com/blog/chatgpt/ has been proposed for
more generic interaction based on a pre-trained model. We similarly use a pre-
trained model for our system.

As far as we know, few studies use discussion for NLP tasks similar to ours.
Chang et al. (2017) proposed the TalkToModel, which explains through dialogue
three tasks of loan, diabetes, and recidivism prediction. The user can talk to
the TalkToModel in five categories: prediction explanation, data modification,
error analysis, dialogue history reference, and experimental setting
explanation. Data for learning and evaluating the TalkToModel are generated by
instructing the annotator to converse about these categories. However, the
categories were not determined based on interviews or data but were defined
subjectively by the authors. Therefore, it is possible that the categories do
not reflect actual conversations that humans need. On the other hand, our
study was conducted in an open-ended dialogue to generate data. Additionally,
our study aims for mutual understanding through a bidirectional dialogue where
both humans and the system express opinions and questions, unlike the systems
that only respond to human questions in a unidirectional dialogue.

There is research on generating explanatory text for predictions as a way to
transfer information from systems to humans through natural language. For
example, research regarding natural science tests Ling et al. (2017), image
recognition and image question answering Park et al. (2018), mathematics tests
Jansen et al. (2018), and NLI Camburu et al. (2018) have been studied.
Additionally, systems for generating explanations using pre-trained models
such as T5 Raffel et al. (2020) and GPT-3 Brown et al. (2020) have also been
proposed Narang et al. (2020); Wiegreffe et al. (2022). However, as these
generated explanations cannot be used to seek additional explanations or
specific explanations, the interpretability is not sufficient in practice as
pointed out by Lakkaraju et al. Slack et al. (2022).

Instead of directly predicting answers, CoT uses natural language to derive
answers step-by-step Wei et al. (2022). This leads to complex multi-step
inferences. By adding the phrase “Let’s think step by step” before each
answer, Kojima et al. (2022) demonstrate that language models are competent
zero-shot CoT. On the other hand, Wang et al. (2022) shows that CoT can
achieve competitive performance even with invalid reasoning steps in the
prompt. The CoT’s step-by-step approach is based on the system only, whereas
our proposed method incorporates human involvement in the system to facilitate
collaboration between humans and the system. Additionally, our approach
utilizes discussions for a step-by-step thinking process.

Research is also being conducted on the use of natural language by humans to
provide instructions and feedback to the system. Abramson et al. (2022) has
developed multi-modal grounded language agents that perform reinforcement
learning on human dialogue-based instructions. Sharma et al. (2022) proposed a
method to integrate human-provided feedback in natural language to update a
robot’s planning cost applied to situations when the planner fails. Murty et
al. (2022) proposed a method to modify a model by natural language patches and
achieved performance improvement in sentiment analysis and relationship
extraction tasks. Campos and Shern (2022) proposed a method for training a
model to behave in line with human preferences, by learning from natural
language feedback, in text summarization. On the other hand, these studies
cannot be explained or questioned by the system to humans.

Various methods have been proposed for deep learning interpretability without
using natural language. Ribeiro et al. (2016) presented a method for
generating explanations of classifier predictions. Li et al. (2016) presented
a method for visualizing attention in neural machine translation models and
discusses the interpretability of attention mechanisms. Shwartz-Ziv and Tishby
(2017) proposed using information-theoretic methods to interpret the inner
workings of neural networks. Fong and Vedaldi (2017) suggested a method for
generating interpretable explanations of black box models through meaningful
perturbations. Kim et al. (2018) proposed a method for quantifying the
synthesis of pertinent examples for interoperability. On the other hand, our
research aims to develop systems that can communicate information
interactively with humans using natural language.

##  8 Conclusion

While deep learning systems have been highly effective in various tasks, their
lack of interpretability poses a challenge to their use in real-world
applications. Recent studies have been working to address this issue by using
natural language to make the systems more intuitive and understandable for
humans. However, current methods, such as explanation generation, have been
criticized for their lack of interpretability and practicality. To address
this, we proposed a system that engages in a dialogue with humans in the form
of discussing predictions, which allows both humans and the system to engage
in explanations, ask questions, refine their thoughts, and solve problems. Our
experimental results showed that the system trained with few-shot learning for
discussion could perform more useful discussions than the system that was not
trained for discussion and provided insights on the challenges and
opportunities of this approach. However, it is important to address the risk
of transmitting incorrect knowledge or maliciously altering the system’s
knowledge by humans. This research provides a new avenue for developing more
interactive deep-learning systems.

## References

  * Abramson et al. (2022) Josh Abramson, Arun Ahuja, Federico Carnevale, Petko Georgiev, Alex Goldin, Alden Hung, Jessica Landon, Jirka Lhotka, Timothy Lillicrap, Alistair Muldal, et al. 2022.  Improving multimodal interactive agents with reinforcement learning from human feedback.  _arXiv preprint arXiv:2211.11602_.
  * Bowman et al. (2015) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Budzianowski and Vulić (2019) Paweł Budzianowski and Ivan Vulić. 2019.  Hello, it’s GPT-2 - how can I help you? towards the use of pretrained language models for Task-Oriented dialogue systems.  In _Proceedings of the 3rd Workshop on Neural Generation and Translation_ , pages 15–22, Hong Kong. Association for Computational Linguistics.
  * Budzianowski et al. (2018) Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018.  MultiWOZ - a Large-Scale Multi-Domain Wizard-of-Oz dataset for Task-Oriented dialogue modelling.  pages 5016–5026.
  * Camburu et al. (2018) Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018\.  E-SNLI: Natural language inference with natural language explanations.
  * Campos and Shern (2022) Jon Ander Campos and Jun Shern. 2022.  Training language models with language feedback.  In _ACL Workshop on Learning with Natural Language Supervision. 2022._
  * Chang et al. (2017) Joseph Chee Chang, Saleema Amershi, and Ece Kamar. 2017.  Revolt: Collaborative crowdsourcing for labeling machine learning datasets.  In _Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems_ , CHI ’17, pages 2334–2346, New York, NY, USA. Association for Computing Machinery.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Fong and Vedaldi (2017) Ruth C Fong and Andrea Vedaldi. 2017.  Interpretable explanations of black boxes by meaningful perturbation.  In _Proceedings of the IEEE international conference on computer vision_ , pages 3429–3437.
  * Ham et al. (2020) Donghoon Ham, Jeong-Gwan Lee, Youngsoo Jang, and Kee-Eung Kim. 2020.  End-to-End neural pipeline for Goal-Oriented dialogue systems using GPT-2.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 583–592, Online. Association for Computational Linguistics.
  * Honovich et al. (2022) Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022.  Unnatural instructions: Tuning language models with (almost) no human labor.  _arXiv preprint arXiv:2212.09689_.
  * Jansen et al. (2018) Peter Jansen, Elizabeth Wainwright, Steven Marmorstein, and Clayton Morrison. 2018\.  WorldTree: A corpus of explanation graphs for elementary science questions supporting multi-hop inference.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , Miyazaki, Japan. European Language Resources Association (ELRA).
  * Kaneko et al. (2022) Masahiro Kaneko, Sho Takase, Ayana Niwa, and Naoaki Okazaki. 2022.  Interpretability for language learners using example-based grammatical error correction.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 7176–7187, Dublin, Ireland. Association for Computational Linguistics.
  * Khashabi et al. (2022) Daniel Khashabi, Xinxi Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, and Yejin Choi. 2022.  Prompt waywardness: The curious case of discretized interpretation of continuous prompts.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3631–3643, Seattle, United States. Association for Computational Linguistics.
  * Kim et al. (2018) Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018.  Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav).  In _International conference on machine learning_ , pages 2668–2677. PMLR.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022.  Large language models are zero-shot reasoners.  _arXiv preprint arXiv:2205.11916_.
  * Lakkaraju et al. (2022) Himabindu Lakkaraju, Dylan Slack, Yuxin Chen, Chenhao Tan, and Sameer Singh. 2022\.  Rethinking explainability as a dialogue: A practitioner’s perspective.
  * Li et al. (2016) Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016.  Visualizing and understanding neural models in NLP.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 681–691, San Diego, California. Association for Computational Linguistics.
  * Ling et al. (2017) Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017.  Program induction by rationale generation: Learning to solve and explain algebraic word problems.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 158–167, Vancouver, Canada. Association for Computational Linguistics.
  * Lipton (2018) Zachary C Lipton. 2018.  The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.  _Queue_ , 16(3):31–57.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Min et al. (2022) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022.  Rethinking the role of demonstrations: What makes in-context learning work?  _arXiv preprint arXiv:2202.12837_.
  * Murty et al. (2022) Shikhar Murty, Christopher D Manning, Scott Lundberg, and Marco Tulio Ribeiro. 2022\.  Fixing model bugs with natural language patches.  _arXiv preprint arXiv:2211.03318_.
  * Narang et al. (2020) Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020.  WT5?! training Text-to-Text models to explain their predictions.
  * Nie et al. (2020) Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020.  Adversarial NLI: A new benchmark for natural language understanding.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4885–4901, Online. Association for Computational Linguistics.
  * Park et al. (2018) Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Anna Rohrbach, Bernt Schiele, Trevor Darrell, and Marcus Rohrbach. 2018.  Multimodal explanations: Justifying decisions and pointing to the evidence.
  * Radford et al. (2022) Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022.  Robust speech recognition via large-scale weak supervision.  _arXiv preprint arXiv:2212.04356_.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21(140):1–67.
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016.  "why should i trust you?" explaining the predictions of any classifier.  In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , pages 1135–1144.
  * Sharma et al. (2022) Pratyusha Sharma, Balakumar Sundaralingam, Valts Blukis, Chris Paxton, Tucker Hermans, Antonio Torralba, Jacob Andreas, and Dieter Fox. 2022.  Correcting robot plans with natural language feedback.  _arXiv preprint arXiv:2204.05186_.
  * Shwartz-Ziv and Tishby (2017) Ravid Shwartz-Ziv and Naftali Tishby. 2017.  Opening the black box of deep neural networks via information.  _arXiv preprint arXiv:1703.00810_.
  * Slack et al. (2022) Dylan Slack, Satyapriya Krishna, Himabindu Lakkaraju, and Sameer Singh. 2022.  Talktomodel: Explaining machine learning models with interactive natural language conversations.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ , 30.
  * Wang et al. (2022) Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2022.  Towards understanding chain-of-thought prompting: An empirical study of what matters.  _arXiv preprint arXiv:2212.10001_.
  * Wang et al. (2021) Zirui Wang, Adams Wei Yu, Orhan Firat, and Yuan Cao. 2021.  Towards zero-label language learning.  _arXiv preprint arXiv:2109.09193_.
  * Webson and Pavlick (2022) Albert Webson and Ellie Pavlick. 2022.  Do prompt-based models really understand the meaning of their prompts? In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2300–2344, Seattle, United States. Association for Computational Linguistics.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022.  Chain of thought prompting elicits reasoning in large language models.  _arXiv preprint arXiv:2201.11903_.
  * Wiegreffe et al. (2022) Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. 2022\.  Reframing Human-AI collaboration for generating Free-Text explanations.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 632–658, Seattle, United States. Association for Computational Linguistics.
  * Zhang et al. (2020) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020\.  Bertscore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.