64 | 59.6/59.9 | 83.2/87.8 | 51.9/84.0 | 13.6/53.2 | 2.6/17.7 | 40.3/35.1 | 67.0/60.4 | 20.0 | 63.2/70.2 | 15.5/15.6 | 84.9/83.6 | 30.0 | 44.3/56.7
93 | 62.1/59.6 | 85.4/87.4 | 56.8/79.9 | 13.5/55.9 | 2.6/18.3 | 39.3/36.0 | 65.1/58.0 | 17.8 | 61.6/66.9 | 16.4/16.2 | 86.4/81.5 | 29.7 |  44.7/56.0

Table 18: Effect of scaling the number of training clusters on each
generalization level for OPT-IML 30B after 2000 steps of training, aggregated
by task category. Results are in the format of 0-shot/5-shot.

| Fully Held Out | Partially Supervised | Fully Supervised | Average
---|---|---|---|---
% Pre-train |  Cause Effect |  Gram. Corr. |  Stereo. Det. |  Word Ana. | Reas. | MMLU | QA | Summ. |  Toxic Det. |  Dial ogue. | QA | Summ.
0 (Baseline) | 63.5/62.5 | 86.1/87.5 | 58.9/82.3 | 17.2/57.8 | 2.6/20.4 | 41.5/37.0 | 69.3/58.9 | 18.1 | 60.0/70.0 | 16.1/15.8 | 87.6/83.5 | 31.3 |  46.0/57.6
1 | 62.3/60.8 | 86.6/88.4 | 54.4/81.7 | 13.9/59.3 | 2.6/26.5 | 39.6/36.4 | 66.6/59.8 | 21.0 | 60.2/66.9 | 16.3/15.6 | 86.4/85.2 | 31.1 | 45.1/58.1
5 | 62.2/61.9 | 86.9/88.2 | 57.8/83.5 | 20.0/59.6 | 2.6/28.3 | 39.0/37.5 | 65.9/63.8 | 20.1 | 58.3/69.9 | 16.2/17.2 | 86.4/83.7 | 30.6 | 45.5/59.4
10 | 60.8/63.2 | 86.6/88.4 | 55.5/83.4 | 23.7/57.3 | 2.8/27.3 | 39.2/38.5 | 65.8/61.1 | 19.6 | 58.8/70.4 | 15.9/15.9 | 86.3/85.2 | 30.5 | 45.5/59.1
50 | 59.5/60.3 | 87.8/88.4 | 62.8/85.1 | 21.2/54.0 | 2.9/29.4 | 37.2/34.6 | 58.5/56.4 | 21.5 | 58.0/66.7 | 15.7/15.0 | 84.7/83.6 | 28.3 | 44.8/57.4

Table 19: Effect of % of pre-training data on each generalization level for
OPT-IML 30B after 4000 steps of training, aggregated by task category. Results
are in the format of 0-shot/5-shot.

| Fully Held Out | Partially Supervised | Fully Supervised
---|---|---|---
|  Cause Effect |  Gram. Corr. |  Stereo. Det. |  Word Ana. | Reas. | MMLU | QA | Summ. |  Toxic Det. |  Dial ogue. | QA | Summ.
Baseline | 58.5/61.0 | 87.1/87.6 | 53.8/79.8 | 12.7/51.5 | 2.4/22.0 | 40.9/36.8 | 69.6/60.6 | 19.9 | 61.5/59.3 | 15.5/15.3 | 86.1/83.9 | 31.3
1% Reas. | 60.8/61.8 | 86.8/88.0 | 55.9/80.9 | 14.9/55.2 | 31.3/32.0 | 40.6/36.4 | 68.5/60.4 | 18.8 | 62.3/67.7 | 16.1/14.6 | 86.9/84.3 | 31.1
2% Reas. | 61.4/63.0 | 86.4/87.7 | 50.9/81.3 | 14.5/61.9 | 30.0/31.1 | 38.7/35.9 | 68.2/60.1 | 19.0 | 57.1/60.8 | 15.4/14.9 | 86.2/84.1 | 31.0
4% Reas. | 59.9/63.7 | 86.4/87.9 | 51.0/82.3 | 14.7/54.7 | 30.5/30.8 | 40.6/33.2 | 68.3/62.2 | 20.8 | 59.4/57.7 | 14.4/14.7 | 85.1/83.2 | 32.0

Table 20: Effect of fine-tuning with Reasoning data on each generalization
level for OPT-IML 30B after 4000 steps, aggregated by task category. Results
are in the format of 0-shot/5-shot.

| Fully Held Out | Partially Supervised | Fully Supervised | Avg.
---|---|---|---|---
EPS |  Cause Effect |  Gram. Corr. |  Stereo. Det. |  Word Ana. | Reas. | MMLU | QA | Summ. |  Toxic Det. |  Dial ogue. | QA | Summ.
Baseline | 62.1/59.5 | 85.4/87.6 | 56.8/79.8 | 13.5/55.4 | 2.6/18.3 | 39.3/36.0 | 65.1/56.6 | 17.8/15.2 | 61.6/65.7 | 16.4/16.5 | 86.4/82.4 | 29.7/19.0 | 44.7/49.3†
Zipf a=4 | 60.5/60.6 | 84.7/88.1 | 54.1/81.2 | 13.8/55.8 | 2.9/9.7 | 38.4/37.3 | 64.4/62.8 | 18.8/19.5 | 59.5/65.8 | 15.5/15.3 | 86.1/85.4 | 30.2/29.5 | 44.1/50.9
Zipf a=4 sf. | 59.8/61.5 | 85.1/87.4 | 52.9/79.4 | 12.2/52.8 | 2.7/24.6 | 41.0/38.7 | 64.3/59.6 | 18.4/20.3 | 66.3/67.3 | 15.9/15.9 | 85.9/85.0 | 29.5/26.9 | 44.5/51.6
Zipf a=2 | 61.6/61.9 | 84.2/87.7 | 48.0/80.1 | 11.0/55.2 | 2.6/15.1 | 37.9/36.4 | 63.7/61.9 | 20.2/21.5 | 65.1/75.3 | 16.1/15.3 | 85.6/85.0 | 29.8/28.1 | 43.8/52.0
Zipf a=2 sf. | 56.1/63.5 | 87.6/88.2 | 60.8/75.0 | 14.5/44.7 | 2.6/20.3 | 39.7/38.0 | 63.4/60.5 | 19.1/20.7 | 65.2/76.0 | 16.2/16.2 | 85.4/86.3 | 31.5/28.8 |  45.2/51.5

Table 21: A repeat of the MetaICL experiments reported in §4.8 using
“$\backslash$n$\backslash$n” as the example separator during inference. Under
this setting, all MetaICL models outperform the baseline model. † The 5-shot
baseline performance is not comparable with those in the other experiment
tables since we also include the 5-shot performance on summarization tasks
here.

##  C Examples of Prompts from All Benchmarks

In this section, we present several examples from all the benchmarks to get an
overview of how the prompts look like that are used for our fine-tuning. In
all the examples, ‘black’ colored text represents prompt and ‘green’ colored
text represents the output that we optimize in our loss function.

Instructions: You are provided with an arithmetic question. Your task is to
compute the solution using the given arithmetic operations. The only
arithmetic operators needed to answer the questions are ‘+’(addition) and
‘-’(subtraction). The answer should be correct to one decimal place.
Sam went to 14 football games this year. He went to 29 games lastyear. How
many football games did Sam go to in all? Output: 43.0
---
Figure 7: Zero-shot example of task745_ai2_arithmetic_questions_arithmetic
task from question answering category of Super-NaturalInstructions benchmark.
In this task, you are given two facts, and a multiple-choice question. Based
on the given facts, answer the question with index of the correct option (e.g,
“A").
Input: Fact1: plasma is formed by electrons separating from atoms in stars,
Fact2: Gas Ionization Losing electrons ionizes the atoms in a gas., Question:
You cannot have matter in a plasma state without what? (A) energy (B) Energy.
(C) voltage (D) ionization (E) nutrients (F) light (G) heat energy (H) kinetic
energy
Answer: D
---
Figure 8: Zero-shot example of task1297_qasc_question_answering task from
question answering category of Super-NaturalInstructions benchmark. In this
task you will be given a list of numbers and you should remove all duplicates
in the list. If every number is repeated in the list an empty list should be
returned. Your list should be numbers inside brackets, just like the given
list.
[0, 0, 3, 6, 5, 3, 0, 4, 1, 5] A: [6, 4, 1]
---
Figure 9: Zero-shot example of task097_conala_remove_duplicates task from
program execution category of Super-NaturalInstructions benchmark. In this
task you will be given a list of integers. You should remove all of the
integers that are divisible by 3 from the list. If every integer in the input
list is divisible by 3 then an empty list should be returned. Zero is
divisible by 3.
Input: [-71, 46, 80, -17, -57, 67, -90, -65, 93, 17, 48]
Answer: [-71, 46, 80, -17, 67, -65, 17]
---
Figure 10: Zero-shot example of task370_synthetic_remove_divisible_by_3 task
from program execution category of Super-NaturalInstructions benchmark. Task:
You are given a concept, and a list of answers. You should generate a question
about the concept that leads to the given answer(s).
Input: concept: John Steinbeck answers: [’The Grapes of Wrath’]
answer: what book did john steinbeck wrote about the people in the dust bowl?
---
Figure 11: Zero-shot example of task1602_webquestion_question_genreation task
from question generation category of Super-NaturalInstructions benchmark.
Instructions: This task is about reading the given passage and construct a
question about the information present in the passage. Construct a question in
such a way that (i) it is unambiguous, (ii) it is answerable from the passage,
(iii) the answer is unique, (iv) its answer is a continous text span from the
paragraph. Avoid creating questions that (i) can be answered correctly without
actually understanding the paragraph and (ii) uses same words or phrases given
in the passage
Czechoslovakia or Czecho-Slovakia (; Czech and , “Česko-Slovensko") was a
sovereign state in Central Europe that existed from October 1918, when it
declared its independence from the Austro-Hungarian Empire, until its peaceful
dissolution into the Czech Republic and Slovakia on 1 January 1993.

From 1939 to 1945, following its forced division and partial incorporation
into Nazi Germany, the state did not “de facto" exist but its government-in-
exile continued to operate.

From 1948 to 1990, Czechoslovakia was part of the Soviet bloc with a command
economy. Its economic status was formalized in membership of Comecon from
1949, and its defense status in the Warsaw Pact of May 1955. A period of
political liberalization in 1968, known as the Prague Spring, was forcibly
ended when the Soviet Union, assisted by several other Warsaw Pact countries,
invaded. In 1989, as Marxist–Leninist governments and communism were ending
all over Europe, Czechoslovaks peacefully deposed their government in the
Velvet Revolution; state price controls were removed after a period of
preparation. In 1993, Czechoslovakia split into the two sovereign states of
the Czech Republic and Slovakia.

The country was of generally irregular terrain. The western area was part of
the north-central European uplands. The eastern region was composed of the
northern reaches of the Carpathian Mountains and lands of the Danube River
basin.
answer: Was Czechoslovakia ever apart of the Soviet bloc?
---
Figure 12: Zero-shot example of task917_coqa_question_generation task from
question generation category of Super-NaturalInstructions benchmark. Classify
given movie review into two categories: positive, or negative based on its
content.
director jay russell stomps in hobnail boots over natalie babbitt’s gentle ,
endearing 1975 children’s novel . A: negative
---
Figure 13: Zero-shot example of task888_reviews_classification task from
sentiment analysis category of Super-NaturalInstructions benchmark.
Instructions: You are provided with an “Event" and it’s “Intent" related to
PersonX. Determine the sentiment value of the given input as either
“Positive", “Negative", and “Unknown".
Event:PersonX is PersonY’s last day. Intent: 1) eat the same thing.
output: Negative
---
Figure 14: Zero-shot example of task923_event2mind_classifier task from
sentiment analysis category of Super-NaturalInstructions benchmark. In this
task, you are given a text of news article and corresponding headline of an
article. Your task is to give label “match" if headline is correct for
article, otherwise give label “no".
Article: Mike Snoei coached side Pune FC bounced back to winning ways with a
2-0 win over Mohammedan Sporting in an I-League round 21 encounter at the
Balewadi sports complex, here on Wednesday. Headline: Pfc bounce back to
winning ways Answer: match
---
Figure 15: Zero-shot example of task1354_sent_comp_classification task from
text matching category of Super-NaturalInstructions benchmark. In this task,
you are given two sentences. Your task is to classify the given sentences as
“Yes" if they have same meaning; otherwise, classify them as “No".
Sentence-1: I’d prefer to go to the beach. $\langle$sep$\rangle$ Sentence-2: I
would like to go out for a drink .
answer: No
---
Figure 16: Zero-shot example of task566_circa_classification task from text
matching category of Super-NaturalInstructions benchmark. Read the following
article and answer the question.

So now I have no problems . In fact , everything is getting better . I got a
job at pizza hut , and it looks like the school paper hired me as their
cartoonist as well , so basically everything ’s okay now .

What may be an invalid reason I am feeling better now ?
OPTIONS:
\- I may be able to show my imagination .
\- I may write a lot of papers for my English class .
\- None of the above choices .
\- I may be able to save money . Answer: I may write a lot of papers for my
English class .
---
Figure 17: Zero-shot example of cosmos_qa task from question answering
category of FLAN benchmark. Article: On April 4, 2008, Beyoncé married Jay Z.
She publicly revealed their marriage in a video montage at the listening party
for her third studio album, I Am… Sasha Fierce, in Manhattan’s Sony Club on
October 22, 2008. I Am… Sasha Fierce was released on November 18, 2008 in the
United States. The album formally introduces Beyoncé’s alter ego Sasha Fierce,
conceived during the making of her 2003 single “Crazy in Love", selling
482,000 copies in its first week, debuting atop the Billboard 200, and giving
Beyoncé her third consecutive number-one album in the US. The album featured
the number-one song “Single Ladies (Put a Ring on It)" and the top-five songs
“If I Were a Boy" and “Halo". Achieving the accomplishment of becoming her
longest-running Hot 100 single in her career, “Halo"’s success in the US
helped Beyoncé attain more top-ten singles on the list than any other woman
during the 2000s. It also included the successful “Sweet Dreams", and singles
“Diva", “Ego", “Broken-Hearted Girl" and “Video Phone". The music video for
“Single Ladies" has been parodied and imitated around the world, spawning the
“first major dance craze" of the Internet age according to the Toronto Star.
The video has won several awards, including Best Video at the 2009 MTV Europe
Music Awards, the 2009 Scottish MOBO Awards, and the 2009 BET Awards. At the
2009 MTV Video Music Awards, the video was nominated for nine awards,
ultimately winning three including Video of the Year. Its failure to win the
Best Female Video category, which went to American country pop singer Taylor
Swift’s “You Belong with Me", led to Kanye West interrupting the ceremony and
Beyoncé improvising a re-presentation of Swift’s award during her own
acceptance speech. In March 2009, Beyoncé embarked on the I Am… World Tour,
her second headlining worldwide concert tour, consisting of 108 shows,
grossing $119.5 million.

Now answer this question: When did Beyoncé get married?
Answer: April 4, 2008
---
Figure 18: Zero-shot example of squad_v1 task from question answering category
of FLAN benchmark. the head of the united nations nuclear watchdog expressed
concern thursday about iran ’s growing nuclear capacity and his organization
’s powerlessness to monitor the program .

Write a brief summary in a sentence or less Output: iaea chief concerned about
inability to monitor iran nuclear program
---
Figure 19: Zero-shot example of gigaword task from summarization category of
FLAN benchmark. Write highlights for this article:

By . Adrian Durham . Follow @@talkSPORTDrive . So the English obsession with
Andrea Pirlo continues. It’s as if a load of FIFA-playing so-called football
fans have discovered an alien species. A player who fundamentally stays in one
place on the field, and shifts the ball about relying not on pace or physical
strength, but on spatial awareness and control of the football. These English
’Pirlovers’ think he comes from another planet. Of course the suave
appearance, the masterful beard, and the ’couldn’t give a f***’ autobiography
all help add to the mystery and attraction of this Italian God-figure. VIDEO
Scroll down to watch The best of Andreas Pirlo from Italy’s training sessions
. Back to work: Andrea Pirlo trains with the Italy squad ahead of their second
match against Costa Rica . Running the show: Pirlo was at his masterful best
against England as he inspired Italy to victory in Manaus . The tragedy in
Manaus was that the English players, and probably even Roy Hodgson decided he
was untouchable when actually we needed to get close to him, kick his
backside, nutmeg him a few times and bring him back down to earth. He’d soon
have come off with a strained calf or broken bootlace. We respected Pirlo far
too much on Saturday, when it should have been our chance to put him in his
place after that disrespectful chipped penalty in the shoot-out at Euro 2012.
Remember Stuart Pearce’s contorted face after burying that spot-kick in 1996,
6 years on from the pain of missing from the spot at Italia 90? Remember David
Beckham slamming in that penalty in 2002 against Argentina after the injustice
of the red card in St Etienne four years earlier? That was revenge, that was
redemption. Raheem Sterling running at Pirlo and nutmegging him, or Wayne
Rooney leaving a foot in to upset – not injure – the Italian, upsetting him so
much he wants to get off the field. That’s what England should have done. More
of this: England showed Pirlo too much respect and should have closed him
down, as Jack Wilshere did . Must do better: Raheem Sterling could have run at
Pirlo and Wayne Rooney should have closed him down . Instead we stood off him,
and a load of dreamy-eyes glossed over back home at this Azzuri legend. ’If
only we had him,’ the English teenagers cried. ’England have never produced a
player like that,’ said the older generation. ’He dummied the ball! Did you
see that? He actually dummied the ball!’ screamed a gaggle of World Cup
groupies who have probably never been to a game of any level in England in
their lives. The English obsession with all things foreign takes over yet
again. Oh, if only we could produce a precious Pirlo. The sad thing is English
football did produce a Pirlo. His name is Michael Carrick and after being left
out of Roy Hodgson’s squad this summer he will end his career having played in
just one game at a World Cup finals. We won it 1-0. Lone appearance: Michael
Carrick’s only World Cup appearance came against Ecuador in 2006 . Other
plans: Carrick has been holidaying this summer after he was overlooked for a
place at the World Cup . I don’t know if Carrick would have had as good an
international career as Pirlo, but I do know the job he did in a very
successful Manchester United side under Sir Alex Ferguson for many seasons.
All these England managers felt they knew better than Sir Alex. They all
failed. It’s all about opinions, and I’d probably conclude Pirlo is a better
player than Carrick. But if England had used him in their midfield instead of
messing around trying to use Lampard and Gerrard together, we may have been
more successful. Anyone scoffing at this can’t prove me wrong. If you love
Pirlo, you’ll appreciate what Carrick has done for United all these years. If
you can’t see the connection, then English football’s problems run worryingly
deep. Mainstay: Carrick has been an almost ever-present at the heart of the
Manchester United midfield .
answer: England gave Andrea Pirlo far too much respect in Manaus .
Wayne Rooney should have closed him down and put him off his game .
Raheem Sterling could have sold him a nutmeg .
England DID produce their own Pirlo - Michael Carrick .
The Manchester United midfielder has played one game at a World Cup .
---
Figure 20: Zero-shot example of cnn_dailymail task from summarization category
of FLAN benchmark. The WHIMS study found that combination hormone therapy
doubled the risk for probable dementia in women 65 and older and did not
prevent mild cognitive impairment .
One study found that combination therapy doubled the risk of probable dementia
and did not prevent less-severe mental decline .
Are these two sentences paraphrases of each other?
OPTIONS:
\- no
\- yes
output: yes
---
Figure 21: Zero-shot example of glue_mrpc task from paraphrasing category of
FLAN benchmark. Do these mean the same?
Since 2008 , Hoyer toured Canada on several occasions , including twice with
Michael Rault , once with Sean Nicholas Savage and once with The Joe .
Since 2008 , Hoyer has been touring Canada several times , once with Sean
Nicholas Savage , once with Michael Rault , and twice with The Joe .

OPTIONS:
\- no
\- yes
answer: no
---
Figure 22: Zero-shot example of paws_wiki task from paraphrasing category of
FLAN benchmark. How might one describe the sentiment of this review?
Ran over a piece of metal, I had 2 discount tires from a few months and it
happened to be one from them that got the metal stuck in, went down and under
warranty only paid $16. Can’t beat that.

OPTIONS:
\- negative
\- positive
Output: positive
---
Figure 23: Zero-shot example of yelp_polarity_reviews task from sentiment
analysis category of FLAN benchmark. it’s going to be a long day

How would the sentiment of this tweet be described?
OPTIONS:
\- negative
\- positive
Output: negative
---
Figure 24: Zero-shot example of sentiment140 task from sentiment analysis
category of FLAN benchmark. Keywords: cat, look, mouse

What is a sentence that includes all these keywords?
Answer: a cat is looking at a dead mouse he killed
---
Figure 25: Zero-shot example of common_gen task from data to text category of
FLAN benchmark. Keywords: cat, look, mouse

What is a sentence that includes all these keywords?
Answer: a cat is looking at a dead mouse he killed
---
Figure 26: Zero-shot example of common_gen task from data to text category of
FLAN benchmark. How great it’s the sound?

Answer using extracts from the following context. If you can’t find an answer,
return Unanswerable

Context:
When hubby was at work, I would often turn on a computer radio program to
listen to while I did housework. Unfortunately, he was seriously injured at
work almost 2 years ago. His sleep is erractic and pain makes the only place
sleep is possible, is his recliner. Since I never know when he will be able to
sleep, I stopped listening to the programs I had enjoyed, so it wouldn’t
bother him. My brother suggested I check out wireless headphones. He told me a
very little bit, but mostly look for the kind that were wireless and not line
of sigth, in other words the kind that could go through walls, ceilings, etc.
So, I started researching and reading. I thought I would just pick up a pair
at Best Buy or even Walmart. Well, the ones they both carried didn’t have
great write ups nor were they any cheaper than these Sennheiser RS120’s from
Amazon’s seller Electronic Expo. Obviously, cost is important with out current
situation. I’d never heard of Sennheiser. Left to buy by brand, I would have
bought something familiar like Sony, but I kept reading good things about
Sennheiser on every site I checked. The price was as good or better than the
brands I had heard before. So I did rely heavy on the comments of others -
thanks to you who posted. I am not an audiophile or techie at all.So now you
know why I bought them, on to the headphones and how they work. In one word -
excellent! I love, love, love them! I put them on and listen to the computer
radio programs as I do housework and they don’t disturb hubby from sleeping.
Finally, once again I can enjoy what I want, when I want. I have only had them
a month or so but I use them almost everyday from 1-2 hours, to all day long
because they are compfortable enough to wear for hours, if I choose. Yeah, the
ears will get a little warm but really not bad at all since they are not
cupped over them. What little sound may escape doesn’t bother hubby as I am
not setting beside him when I use them and I doubt they would do so even then,
because I don’t have anything that close to my…
Answer: Unanswerable
---
Figure 27: Zero-shot example of subjqa__electronics task from question
answering category of PromptSource benchmark. Question: How many punts were
returned for a score?
Answer based on following passage.

Coming off their Thanksgiving win over the Raiders, the Cowboys went to Giants
Stadium for the final time to face off against their NFC East rival the New
York Giants. With a scoreless first quarter the Cowboys scored a field goal
and touchdown by Tony Romo to Roy Williams. However, the Giants countered by
scoring two touchdowns in the second quarter. Nick Folk missed a 57-yard field
goal just before halftime.In the third quarter, both teams scored touchdowns
and Nick Folk missed a 42-yard field goal which would have put the Cowboys in
front. In the fourth quarter the Giants scored two touchdowns, one being a
79-yard punt return by Domenik Hixon. Dallas would score a late touchdown from
Romo to Miles Austin but the game ended with the Giants sweeping the series.

Answer: 1
---
Figure 28: Zero-shot example of drop task from question answering category of
PromptSource benchmark. Synthesize these documents into a single one:

\- Democrats on the Senate Judiciary Committee have privately requested to
view a Brett Kavanaugh-related document in possession of the panel’s top
Democrat, Dianne Feinstein, but the senior California senator has so far
refused, according to multiple sources familiar with the situation. The
specific content of the document, which is a letter from a California
constituent, is unclear, but Feinstein’s refusal to share the letter has
created tension on the committee, particularly after Feinstein largely took a
back seat to her more junior colleagues last week, as they took over
Kavanaugh’s confirmation hearings with protests around access to
documents.$[$…continued$]$

\- Sen. Dianne Feinstein, D-Calif., on Thursday threw a cryptic curveball at
Brett Kavanaugh, insinuating the Supreme Court nominee could be guilty of a
crime even as Democrats on the Senate Judiciary Committee seek to delay his
confirmation.

The vague accusation comes after the Senate Judiciary Committee already
grilled Kavanaugh and other witnesses and prepares to vote on sending his
nomination to the full Senate. The White House blasted the ambiguous charge as
a last minute gambit.

“I have received information from an individual concerning the nomination of
Brett Kavanaugh to the Supreme Court,” Feinstein said in her surprise
statement. “That individual strongly requested confidentiality, declined to
come forward or press the matter further, and I have honored that decision. I
have, however, referred the matter to federal investigative
authorities.”$[$…continued$]$

\- Senate Judiciary Committee Chairman Chuck Grassley, R-Iowa, left,
accompanied by Sen. Dianne Feinstein, D-Calif., the ranking member, center,
speaks with Sen. Patrick Leahy, D-Vt., right, during a Senate… (Associated
Press) $[$…continued$]$ Dick Durbin, says the matter has been referred to the
FBI.

Republicans…
Output: Sen. Dianne Feinstein says she’s notified federal investigators about
information she received concerning Supreme Court nominee Brett Kavanaugh, the
AP reports. The California Democrat says in a statement that she “received
information from an individual concerning the nomination." Feinstein isn’t
saying who that person is or describing the information in any way. She says
the person “strongly requested confidentiality, declined to come forward or
press the matter further, and I have honored that decision." Feinstein—the
top-ranking Democrat on the Senate Judiciary Committee, which just finished
confirmation hearings for Kavanagh—says she has “referred the matter to
federal investigative authorities." Another committee Democrat, Illinois Sen.
Dick Durbin, says the matter has been referred to the FBI. Fox News refers to
the whole thing as “cryptic." Confirmed details are scarce, but the Intercept
says the information Feinstein is referring to came to her via a letter that
was first sent to US Rep. Anna Eshoo from someone affiliated with Stanford
University. (On Wednesday night, Sen. Cory Booker released confidential
documents relating to Kavanaugh.)
---
Figure 29: Zero-shot example of multi_news task from summarization category of
PromptSource benchmark. We remove some text from the input and replace it with
‘$[$…continued$]$’ for presentation purpose. Write a scientific title for the
following abstract: OBJECTIVES Sepsis is characterised by the frequent
presence of organ failure and marked immunologic alterations. We studied the
association between the extent of organ failure and the transcriptomic
response of septic patients. METHODS Gene expression profiles in the blood of
74 surgical patients with sepsis were compared with those of 30 surgical
patients with no sepsis. Differentially expressed genes were assessed for
their correlation with the sequential organ failure (SOFA) score. RESULTS The
expression levels of a group of genes participating in the cell cycle
(HIST1H1C, CKS2, CCNA2, CDK1, CCNB2, CIT, CCNB1, AURKA, RAD51), neutrophil
protease activity (ELANE, ADORA3, MPO, MMP8, CTSG), IL-1R and IL-18R response
correlated directly with SOFA and mortality. Genes involved in T cell (LCK,
CD3G, CD3D, ZAP70, ICOS, CD3E, CD28, IL2RB, CD8B, CD8A, CD40LG, IL23A, CCL5,
SH2D1A, ITK, CD247, TBX21, GATA3, CCR7, LEF1, STAT4) and NK cell immunity
(CD244, KLRK1, KLRD1) were inversely associated with SOFA and mortality.
CONCLUSIONS The extent of organ failure in sepsis correlates directly with the
existence of imbalanced innate and adaptive responses at the transcriptomic
level. Quantification of the expression levels of the genes identified here
could contribute to the simultaneous assessment of disease severity and
immunological alterations in sepsis.
Title: Transcriptomic correlates of organ failure extent in sepsis.
---
Figure 30: Zero-shot example of cord19__metadata task from summarization
category of PromptSource benchmark. walk around left thrice and jump right
thrice

Given the commands above, what is the corresponding correct sequence of
actions (comma-separated)?
answer: turn left, walk, turn left, walk, turn left, walk, turn left, walk,
turn left, walk, turn left, walk, turn left, walk, turn left, walk, turn left,
walk, turn left, walk, turn left, walk, turn left, walk, turn right, jump,
turn right, jump, turn right, jump
---
Figure 31: Zero-shot example of scan__template_opposite_right task from
semantic parsing category of PromptSource benchmark. Given the commands below,
what is the corresponding correct sequence of actions (comma-separated)?

run opposite left twice after turn left
Answer: turn left, turn left, turn left, run, turn left, turn left, run
---
Figure 32: Zero-shot example of scan__template_jump_around_right task from
semantic parsing category of PromptSource benchmark. Given the following
review:
Really liked the case at first but in a few days after normal use, developed a
lot of scratches on the back since it is hard clear plastic. I Sent it back
and ordered the Tottallee Clear case which I like a LOT better and it was less
$
predict the associated rating from the following choices (1 being lowest and 5
being highest)
\- 1
\- 2
\- 3
\- 4
\- 5
answer: 3
---
Figure 33: Zero-shot example of amazon_us_reviews__wireless_v1_00 task from
text scoring category of PromptSource benchmark. Rate the product by the
number of stars based on the review below: (1 being the lowest and 5 the
highest)
===
Smaller than expected. Model in picture is misleading, much smaller than
expected Product category: shoes
answer: 3
---
Figure 34: Zero-shot example of amazon_reviews_multi__en task from text
scoring category of PromptSource benchmark. Classify this request into one of
the following intents:
alarm query, alarm remove, alarm set, audio volume down, audio volume mute,
audio volume other, audio volume up, calendar query, calendar remove, calendar
set, cooking query, cooking recipe, date time convert, date time query, email
add contact, email query, email query contact, email send email, general
affirm, general command stop, general confirm, general don’t care, general
explain, general greet, general joke, general negate, general praise, general
quirky, general repeat, internet of things cleaning, internet of things
coffee, internet of things hue light change, internet of things hue light dim,
internet of things hue light off, internet of things hue light on, internet of
things hue light up, internet of things wemo off, internet of things wemo on,
lists create or add, lists query, lists remove, music dislikeness, music
likeness, music query, music settings, news query, play audiobook, play game,
play music, play podcasts, play radio, question answer currency, question
answer definition, question answer factoid, question answer maths, question
answer stock, recommendation events, recommendation locations, recommendation
movies, social post, social query, takeaway order, takeaway query, transport
query, transport taxi, transport ticket, transport traffic or weather query.
play i walk the line by johnny cash Output: play music
---
Figure 35: Zero-shot example of nlu_evaluation_data task from intent
classification category of PromptSource benchmark. This search query talks
about the coronavirus and was published on 2020-09-24. In what country was it
issued?
bupa covid test
answer: United Kingdom
---
Figure 36: Zero-shot example of bing_coronavirus_query_set task from intent classification category of PromptSource benchmark. knowledge graph: virgo super-cluster: m.01825t | common.image.appears_in_topic_gallery astronomy.galactic_super_cluster astronomy.galactic_group astronomy.galactic_super_cluster.galaxy_clusters astronomy.galactic_group.galaxy_cluster astronomy.galactic_cluster.galaxy_supercluster astronomy.galactic_cluster.galaxy_groups astronomy.galactic_filament astronomy.galactic_cluster astronomy.galactic_filament.galaxy_superclusters astronomy.galactic_super_cluster.galaxy_filament
question: which galactic cluster is in the galaxy supercluster of virgo super-
cluster?
KG query: (AND astronomy.galactic_cluster (JOIN
astronomy.galactic_cluster.galaxy_supercluster m.01825t))
---
Figure 37: Zero-shot example of grailqa task from semantic parsing category of UnifiedSKG benchmark. database: | formula_1 | circuits : circuitid , circuitref , name , location , country , lat , lng , alt , url | races : raceid , year , round , circuitid , name , date , time , url | drivers : driverid , driverref , number , code , forename , surname , dob , nationality , url | status : statusid , status | seasons : year , url | constructors : constructorid , constructorref , name , nationality , url | constructorstandings : constructorstandingsid , raceid , constructorid , points , position , positiontext , wins | results : resultid , raceid , driverid , constructorid , number , grid , position , positiontext , positionorder , points , laps , time , milliseconds , fastestlap , rank , fastestlaptime , fastestlapspeed , statusid | driverstandings : driverstandingsid , raceid , driverid , points , position , positiontext , wins | constructorresults : constructorresultsid , raceid , constructorid , points , status | qualifying : qualifyid , raceid , driverid , constructorid , number , position , q1 , q2 , q3 | pitstops : raceid , driverid , stop , lap , time , duration , milliseconds | laptimes : raceid , driverid , lap , position , time , milliseconds
question sequence: How many races has there been?
SQL query: select count ( * ) from races
---
Figure 38: Zero-shot example of cosql task from semantic parsing category of UnifiedSKG benchmark. table: col : ndeg | name | position | date of birth | nationality row 1 : 1 | marcus popp | s | 23 settembre 1981 | germany row 2 : 2 | stanislav simin | c | 4 ottobre 1986 | serbia row 3 : 3 | gerald hardy-dessources | c | 9 febbraio 1983 | france row 4 : 4 | soane falafala | s | 16 aprile 1993 | france row 5 : 5 | cyril guittet | l | 13 agosto 1992 | france row 6 : 6 | david konecny | s/o | 10 ottobre 1982 | czech republic row 7 : 7 | jean-francois exiga | l | 9 marzo 1982 | france row 8 : 8 | nuno pinheiro | p | 31 dicembre 1984 | portugal row 9 : 10 | guillaume di betta | s | 8 settembre 1994 | france row 10 : 12 | maxime dillies | p | 11 aprile 1984 | france row 11 : 13 | kamil baranek | s | 2 maggio 1983 | czech republic row 12 : 14 | renaud lachaise | p | 12 maggio 1991 | france row 13 : 15 | david smith | c | 15 maggio 1985 | united states row 14 : 16 | emmanuel ragondet | s | 6 agosto 1987 | france row 15 : 17 | victor le guennec | s | 19 giugno 1994 | france row 16 : 18 | thibault szymkowiak | c | 19 settembre 1991 | france
question: what are the birth dates of soane falafala and david smith? || who
are all the player that competed in the tours vb?
answer: 16 aprile 1993, 15 maggio 1985
---
Figure 39: Zero-shot example of sqa task from question answering category of UnifiedSKG benchmark. table: col : date | venue | opponents | competition | match report row 1 : 12 july 2008 | ochilview park | stenhousemuir | f | raith rovers official site [ dead link ] row 2 : 15 july 2008 | stark’s park | hibs | f | raith rovers official site [ dead link ] row 3 : 19 july 2008 | stark’s park | dundee united | f | raith rovers official site [ dead link ] row 4 : 26 july 2008 | stark’s park | rangers | f | raith rovers official site [ dead link ] row 5 : 30 july 2008 | recreation ground | burntisland shipyard | fife cup 2007-08 | raith rovers official site [ dead link ] row 6 : 2 august 2008 | somerset park | ayr united | second division | raith rovers official site [ dead link ] row 7 : 5 august 2008 | cliftonhill | albion rovers | slc | raith rovers official site [ dead link ] row 8 : 9 august 2008 | stark’s park | queen’s park | second division | raith rovers official site [ dead link ] row 9 : 12 august 2008 | stark’s park | ross county | scc | raith rovers official site [ dead link ] row 10 : 16 august 2008 | recreation park | alloa athletic | second division | raith rovers official site [ dead link ] row 11 : 23 august 2008 | stark’s park | stranraer | second division | raith rovers official site [ dead link ] row 12 : 26 august 2008 | stark’s park | falkirk | slc | raith rovers official site [ dead link ] row 13 : 30 august 2008 | balmoor | peterhead | second division | raith rovers official site [ dead link ] row 14 : 13 september 2008 | stark’s park | east fife | second division | raith rovers official site [ dead link ] row 15 : 20 september 2008 | stark’s park | stirling albion | second division | raith rovers official site [ dead link ] row 16 : 27 september 2008 | glebe park | brechin city | second division | raith rovers official site [ dead link ] row 17 : 4 october 2008 | gayfield park | arbroath | second division | raith rovers official site [ dead link ] row 18 : 18 october 2008 | stark’s park | ayr united | second division | raith rovers official site [ dead link ] row 19 : 25 october 2008 | stair park | stranraer | second division | raith rovers official site [ dead link ] row 20 : 1 november 2008 | stark’s park | alloa athletic | second division | raith rovers official site [ dead link ] row 21 : 8 november 2008 | bayview stadium | east fife | second division | raith rovers official site [ dead link ] row 22 : 15 november 2008 | stark’s park | peterhead | second division | raith rovers official site [ dead link ] row 23 : 22 november 2008 | stark’s park | brechin city | second division | raith rovers official site [ dead link ] row 24 : 29 november 2008 | stark’s park | alloa athletic | sc | raith rovers official site [ dead link ] row 25 : 9 december 2008 | recreation park | alloa athletic | sc | raith rovers official site [ dead link ] row 26 : 13 december 2008 | hampden park | queen’s park | second division | raith rovers official site [ dead link ] row 27 : 20 december 2008 | stark’s park | arbroath | second division | raith rovers official site [ dead link ] row 28 : 27 december 2008 | balmoor | peterhead | second division | raith rovers official site [ dead link ] row 29 : 3 january 2009 | stark’s park | east fife | second division | raith rovers official site [ dead link ] row 30 : 10 january 2009 | forthbank stadium | stirling albion | second division | raith rovers official site [ dead link ] row 31 : 17 january 2009 | stark’s park | stranraer | second division | raith rovers official site [ dead link ]
question: what is the match report from the game played on 25 april 2009?
answer: raith rovers official site [ dead link ]
---
Figure 40: Zero-shot example of wikisql task from question answering category
of UnifiedSKG benchmark. SQL: SELECT LName FROM STUDENT WHERE Sex = “F" ORDER
BY Age DESC
query in natural language: Find the last name of female (sex is F) students in
the descending order of age.
---
Figure 41: Zero-shot example of sql2text task from formal-language-to-text category of UnifiedSKG benchmark. table: Caption: black swan - class sloop | name, pennant, builder, laid down, launched, commissioned
query: eq hop argmin all_rows ; laid down ; name ; chanticleer = true
query description: the chanticleer was the first sloop to be laid down in the
black swan - class sloop .
---
Figure 42: Zero-shot example of logic2text task from formal-language-to-text
category of UnifiedSKG benchmark. table fact: $\langle$page_title$\rangle$
Bruce Weber (basketball) $\langle$/page_title$\rangle$
$\langle$section_title$\rangle$ Head coaching record
$\langle$/section_title$\rangle$ $\langle$table$\rangle$
$\langle$cell$\rangle$ Illinois Fighting Illini (Big Ten Conference)
(2003–2012) $\langle$col_header$\rangle$ Season $\langle$/col_header$\rangle$
$\langle$col_header$\rangle$ Team $\langle$/col_header$\rangle$
$\langle$col_header$\rangle$ Overall $\langle$/col_header$\rangle$
$\langle$col_header$\rangle$ Conference $\langle$/col_header$\rangle$
$\langle$col_header$\rangle$ Standing $\langle$/col_header$\rangle$
$\langle$col_header$\rangle$ Postseason $\langle$/col_header$\rangle$
$\langle$/cell$\rangle$ $\langle$cell$\rangle$ 2007–08
$\langle$col_header$\rangle$ Season $\langle$/col_header$\rangle$
$\langle$/cell$\rangle$ $\langle$cell$\rangle$ 16–19
$\langle$col_header$\rangle$ Overall $\langle$/col_header$\rangle$
$\langle$/cell$\rangle$ $\langle$cell$\rangle$ 5–13
$\langle$col_header$\rangle$ Conference $\langle$/col_header$\rangle$
$\langle$/cell$\rangle$ $\langle$/table$\rangle$
textual description: Weber finished the 2007–08 Illini season with an overall
record of 16–19 and 5–13 in the Big Ten.
---
Figure 43: Zero-shot example of totto task from data to text category of
UnifiedSKG benchmark. Select the sentence without English grammar error from
the given pair. sentence 1: The took lights weren’t orange. sentence 2: The
taken lights weren’t orange. A: sentence 2
---
Figure 44: Zero-shot example of blimp__irregular_past_participle_adjectives
task from linguistic probing category of CrossFit benchmark. Select the
sentence without English grammar error from the given pair. sentence 1:
Kimberley has really ever found those actresses to compete. sentence 2:
Kimberley has not ever found those actresses to compete. A: sentence 2
---
Figure 45: Zero-shot example of
blimp__sentential_negation_npi_licensor_present task from linguistic probing
category of CrossFit benchmark. Why do some pimples hurt and swell more than
others?
Answer: Scalp pimples are the best: you’re out and about with your day and
feel an itch on your head and go to scratch it only to apply too much pressure
to the overzealous zit and it bursts in an orgiastic explosion of bliss.
---
Figure 46: Zero-shot example of eli5__eli5 task from question answering
category of CrossFit benchmark. Why do some pimples hurt and swell more than
others?
Answer: Scalp pimples are the best: you’re out and about with your day and
feel an itch on your head and go to scratch it only to apply too much pressure
to the overzealous zit and it bursts in an orgiastic explosion of bliss.
---
Figure 47: Zero-shot example of eli5__eli5 task from question answering
category of CrossFit benchmark. Predict the content that fills in the [MASK]
placeholder to complete each fact statement: Tiv people is located in [MASK] .
Output: Nigeria
---
Figure 48: Zero-shot example of lama__trex task from information extraction
category of CrossFit benchmark. Predict the content that fills in the [MASK]
placeholder to complete each fact statement: Something you might do while
helping someone is [MASK].
output: dying
---
Figure 49: Zero-shot example of lama__conceptnet task from information
extraction category of CrossFit benchmark. Generate a normalized form of the
entity denoted in the given passage with [START_ENT] and [END_ENT]. Passage:
ATHLETICS - SALAH HISSOU BREAKS 10,000 METRES WORLD RECORD . [START_ENT]
BRUSSELS [END_ENT] 1996-08-23 Morocco ’s Salah Hissou broke the men ’s 10,000
metres world record on Friday when he clocked 26 minutes 38.08 seconds at the
Brussels grand prix on Friday . The previous mark of 26:43.53 was set by
Ethiopia ’s Haile Gebreselassie in the Dutch town of Hengelo in June last year
.
Output: Brussels
---
Figure 50: Zero-shot example of kilt_ay2 task from entity linking category of
CrossFit benchmark. Generate a normalized form of the entity denoted in the
given passage with [START_ENT] and [END_ENT]. Passage: ATHLETICS - SALAH
HISSOU BREAKS 10,000 METRES WORLD RECORD . [START_ENT] BRUSSELS [END_ENT]
1996-08-23 Morocco ’s Salah Hissou broke the men ’s 10,000 metres world record
on Friday when he clocked 26 minutes 38.08 seconds at the Brussels grand prix
on Friday . The previous mark of 26:43.53 was set by Ethiopia ’s Haile
Gebreselassie in the Dutch town of Hengelo in June last year .
Output: Brussels
---
Figure 51: Zero-shot example of kilt_ay2 task from entity linking category of
CrossFit benchmark. In this sentence, replace verbs with their synonyms.

renewed efforts to fight drugs and crime will be costly

Answer: renewed efforts to fight_back drugs and crime will be costly
---
Figure 52: Zero-shot example of styleptb task from style transfer category of
ExMix benchmark. How would Shakespeare write the following sentence?

True , “ she said , “a fine little one." “No , “ I said , “a huge wit." “Right
, “ she said , “a hugely awful one." “No , “ I said , “he has a good wit."
“Exactly , “ she said , “it’s good and mild ; it doesn’t hurt anyone." “No , “
I said , “Benedick is wise." “He is certainly , “ she said , She cried a lot
at that , and said she didn’t care .

Answer: True , “ said she , “a fine little one." “No , “ said I , “a great
wit." “Right , “ says she , “a great gross one." “Nay , “ said I , “a good
wit." “Just , “ said she , “it hurts nobody." “Nay , “ said I , “the gentleman
is wise." “Certain , “ said she , “a wise gentleman." “Nay , “ said I , “he
hath the tongues." “That I believe , “ said she , “for he swore a thing to me
on Monday night , which he forswore on Tuesday morning ; there’s a double
tongue , there’s two tongues." Thus did she an hour together transshape thy
particular virtues .
---
Figure 53: Zero-shot example of shakespearizingmodernenglish task from style
transfer category of ExMix benchmark. Transform the following sentence into
funql.

what is the cost of a flight from boston to san francisco

Answer:
answer(_fare(_flight(intersection(_from_2(city_name(boston)),_to_2(city_name(san_francisco))))))
---
Figure 54: Zero-shot example of unimer task from semantic parsing category of
ExMix benchmark. Construct a semantic representation of the following
sentence.

Victoria awarded a cake to Olivia .

Answer: award . agent ( x _ 1 , Victoria ) AND award . theme ( x _ 1 , x _ 3 )
AND award . recipient ( x _ 1 , Olivia ) AND cake ( x _ 3 )
---
Figure 55: Zero-shot example of cogs task from semantic parsing category of
ExMix benchmark. Which Wikipedia entity does the marked entity correspond to?

Gmina Jaraczewo is a rural gmina ( administrative district ) in Jarocin County
, Greater Poland Voivodeship , in west-central Poland . Its seat is the
village of Jaraczewo , which lies approximately west of Jarocin and south-east
of the regional capital Poznań . The gmina covers an area of , and as of 2006
its total population is 8,281 . Gmina Jaraczewo contains the villages and
settlements of Bielejewo , Brzostów , [START_ENT] Cerekwica [END_ENT] , Gola ,
Góra , Jaraczewo , Łobez , Łobzowiec , Łowęcice , Łukaszewo , Niedźwiady ,
Nosków , Nowa Cerekwica , Panienka , Parzęczew , Poręba , Rusko , Strzyżewko ,
Suchorzewko , Wojciechowo and Zalesie . Gmina Jaraczewo is bordered by the
gminas of Borek Wielkopolski , Dolsk , Jarocin , Koźmin Wielkopolski , Książ
Wielkopolski and Nowe Miasto nad Wartą

Answer: Cerekwica, Jarocin County
---
Figure 56: Zero-shot example of wned task from knowledge-intensive language
tasks category of ExMix benchmark. Which Wikipedia entity does the marked
entity correspond to?

Afghan leader tells U.S. Congressman of peace plan . Sayed Salahuddin KABUL
1996-08-27 Afghan government military chief Ahmad Shah Masood briefed visiting
U.S. Congressman Dana Rohrabacher on Tuesday on a peace plan for his wartorn
country . A spokesman for Masood said he had told the California Republican at
a meeting in northern Kabul that President Burhanuddin Rabbani ’s government
favoured talks with all Afghan factions to set up an interim government . The
factions should agree to appoint a transitional leader , draft a new
constitution , collect heavy weapons , create a national army and hold free
elections in which the transitional leader would be barred from standing , he
added . Rohrabacher flew into Bagram military airbase north of Kabul in a Red
Cross plane on Tuesday after meeting northern opposition militia leader
General Abdul Rashid Dostum . Masood ’s spokesman Amrollah ( one name ) said
Rohrabacher had recently visited Italy , Saudi Arabia and Pakistan as part of
a mission to promote peace in [START_ENT] Afghanistan [END_ENT] . “ We are
certainly serious more than before to find a solution to the Afghan problem
and support every U.N. plan , “ Amrollah quoted Rohrabacher as saying .
However , a spokesman for Prime Minister Gulbuddin Hekmatyar , a long-time
rival of Masood , expressed concern at signs of renewed U.S. interest in
Afghanistan . “ America wants to block the establishment of a strong Islamic
government in Afghanistan and the U.S. intends to neutralise the Afghan peace
process initiated by the Afghans themselves , “ said the spokesman , Hamid
Ibrahimi . “ A great game has been started in Afghanistan as America feels
that Tehran and Moscow have got stronger in the Afghan picture – something
Washington wants to change , “ he said . Rohrabacher was expected to visit
neutral faction leaders in the eastern city of Jalalabad and meet leaders of
the rebel Islamic Taleban militia in the southern city of Kandahar . Afghan
guerrilla factions have been locked in a bloody power struggle since the fall
of the communist government in April 1992 . Hekmatyar , once Rabbani ’s main
rival , made a peace pact with him and rejoined the government as prime
minister in June .

Answer: Afghanistan
---
Figure 57: Zero-shot example of aidayago2 task from knowledge-intensive
language tasks category of ExMix benchmark. What Lakers legend visited LeBron
James less than 24 hours before the free agent became a Laker? A: Magic
Johnson
---
Figure 58: Zero-shot example of newsquizqa task from question answering
category of ExMix benchmark. What Lakers legend visited LeBron James less than
24 hours before the free agent became a Laker? A: Magic Johnson
---
Figure 59: Zero-shot example of newsquizqa task from question answering
category of ExMix benchmark. Is the following comment toxic?

“:Thanks for the comment and bringing this to my attention. This is my first
time I have voted using that format. I didn’t notice the comments at top and
bottom instead, I went directly to my comment. If you look look at the change
I made, I added two words (my original vote with a strikethrough) that I had
mistakenly deleted, instead of using the common method of altering one’s vote
by doing a strikethrough. So, I re-added the deleted words of my previous vote
with a strikethrough.

On reflection, I think there are meaningful exceptions to rules. The change
does not alter my comment or revised “"vote."" It preserves information about
the change in status of my vote. In fact, it clarifies the vote change. So, in
this case, as the intention of the request of do not edit is to keep a clear
record of the vote, I think my very minor edit actually serves that. So, I’ll
leave it as is. Thanks,

“
answer: No
---
Figure 60: Zero-shot example of wikipedia_toxicity_subtypes task from toxic
language detection category of ExMix benchmark. Is the following comment
toxic?

“:Thanks for the comment and bringing this to my attention. This is my first
time I have voted using that format. I didn’t notice the comments at top and
bottom instead, I went directly to my comment. If you look look at the change
I made, I added two words (my original vote with a strikethrough) that I had
mistakenly deleted, instead of using the common method of altering one’s vote
by doing a strikethrough. So, I re-added the deleted words of my previous vote
with a strikethrough.

On reflection, I think there are meaningful exceptions to rules. The change
does not alter my comment or revised “"vote."" It preserves information about
the change in status of my vote. In fact, it clarifies the vote change. So, in
this case, as the intention of the request of do not edit is to keep a clear
record of the vote, I think my very minor edit actually serves that. So, I’ll
leave it as is. Thanks,

“
answer: No
---
Figure 61: Zero-shot example of wikipedia_toxicity_subtypes task from toxic
language detection category of ExMix benchmark. summarize: Editor’s note:
Ruben Navarrette is a nationally syndicated columnist and a member of the
editorial board of the San Diego Union-Tribune. Read his column here. Ruben
Navarrette asks whether Obama’s cool, calm demeanor will be a plus in
negotiating with world leaders. SAN DIEGO, California (CNN) – Make no mistake,
Barack Obama is one cool customer. Now, after the last debate, it seems all
but certain that the Iceman cometh to the White House. Radio talk show hosts
and rank-and-file Republicans spent the last few weeks pleading with John
McCain to take the gloves off and take the fight to Obama. How’s that working
out, folks? In this week’s match-up, Obama snatched the gloves out of McCain’s
hands and slapped him silly with them. I suppose the hope was that Obama would
get rattled and make a mistake. But Obama doesn’t get rattled or make many
mistakes. I still have no idea what type of president Obama would make. But
he’s an extraordinary politician. In fact, he may even be better than Bill
Clinton who, while he had the IQ and EQ, also had the burden of a legendary
red-hot temper. Obviously, it takes a lot to get under Obama’s skin. McCain
sure tried. Maybe this is the guy we want negotiating with world leaders.
Maybe after eight years of George W. Bush stubbornness, on the heels of eight
years of Clinton emotiveness, we need to send out for ice. In a CNN/Opinion
Research Corp. poll, 58 percent of those who watched the debate said Obama did
the better job and 31 percent said that about McCain. That makes three skins
for Obama. In earlier polls, 54 percent of those who watched thought Obama won
the second debate, and 51 percent thought he won the first one. This week,
McCain turned in his best performance of the debates, and the first 30 minutes
– with McCain bringing up Obama’s problematic encounter with the now famous
Joe the Plumber; and the quip about how he isn’t Bush and how Obama should
have run four years ago – were near flawless for the Republican. iReport.com:
Are you Joe the Plumber? Get out your plunger and share your thoughts. McCain
put Obama on the defensive, and it showed. If McCain had been that aggressive
throughout the first two debates – firm but not necessary unlikable – we might
be looking at a different race right now. But, over the next hour, Obama
regained his stride and eventually dominated the exchange. And, in the end,
with his sarcastic crack about school vouchers – “Because there’s not enough
vouchers; therefore, we shouldn’t do it, even though it’s working. I got it."
– McCain was profoundly unlikable. So said the polls. In the CNN/Opinion
Research Corp. poll, 70 percent of debate watchers found Obama more likable.
Only 22 percent said that about McCain. McCain’s supporters wanted him to
bring up some of the allegedly shady characters from Obama’s circle of
acquaintances that give some Americans pause and lead them to question the
Democrat’s values. There are good reasons to have that conversation, and bad
ones. A friend and fellow journalist told me Obama’s Chicago posse was
important because it formed “the political womb where the fetal Obama grew
into a politician. $[$…continued$]$ But, unless the political wind changes –
and quickly – a promotion to the other end of Pennsylvania Avenue doesn’t
appear to be in the cards. The opinions expressed in this commentary are
solely those of Ruben Navarrette.
answer: Ruben Navarrette: McCain tried to get under Obama’s skin with attacks.
Obama remained cool and wound up ending strongly in the debate, he says.
Navarrette says Ayers attacks backfired even though issue is legitimate.
America may need the cool-headedness of Barack Obama, he says.
---
Figure 62: Zero-shot example of cnn_dailymail_v002 task from summarization
category of T5 benchmark. We remove some text from the input and replace it
with ‘$[$…continued$]$’ for presentation purpose. Answer the following
question by reasoning step by step.

“ Ai n’t No Sunshine ” is a song by Bill Withers from his 1971 album Just As I
Am , produced by Booker T. Jones . The record featured musicians Donald “ Duck
” Dunn on bass guitar , Al Jackson , Jr. on drums and Stephen Stills on guitar
. String arrangements were done by Booker T. Jones , and recorded in Memphis
by engineer Terry Manning . The song is in the key of A minor .
who did ain’t no sunshine when she’s gone Output: “ Ai n’t No Sunshine ” is a
song by Bill Withers from his 1971 album Just As I Am , produced by Booker T.
Jones . The answer is Bill Withers
---
Figure 63: Zero-shot example of qed task from Reasoning benchmark. Answer the
following question by reasoning step by step.

How do most people feel about a person they love?
popularity, know all, own house, care about, flu Output: we care about people
we love. The answer is care about
---
Figure 64: Zero-shot example of cose task from Reasoning benchmark.