# Visually grounded generation of entailments from premises

Somaye Jafaritazehjani†, Albert Gatt⋄, Marc Tanti⋄
†IRISA, Université de Rennes I, France
⋄Institute of Linguistics and Language Technology, University of Malta
<EMAIL_ADDRESS>
{albert.gatt<EMAIL_ADDRESS>

###### Abstract

Natural Language Inference (NLI) is the task of determining the semantic
relationship between a premise and a hypothesis. In this paper, we focus on
the generation of hypotheses from premises in a multimodal setting, to
generate a sentence (hypothesis) given an image and/or its description
(premise) as the input. The main goals of this paper are (a) to investigate
whether it is reasonable to frame NLI as a generation task; and (b) to
consider the degree to which grounding textual premises in visual information
is beneficial to generation. We compare different neural architectures,
showing through automatic and human evaluation that entailments can indeed be
generated successfully. We also show that multimodal models outperform
unimodal models in this task, albeit marginally.

##  1 Introduction

Natural Language Inference (NLI) or Recognizing Textual Entailment (RTE) is
typically formulated as a classification task: given a pair consisting of
premise(s) P and hypothesis Q, the task is to determine if Q is entailed by P,
contradicts it, or whether Q is neutral with respect to P Dagan et al. (2006).
For example, in both the left and right panels of Figure 1, the premise P in
the caption entails the hypothesis Q.

In classical (i.e. logic-based) formulations (e.g., Cooper et al., 1996), P is
taken to entail Q if Q follows from P in all ‘possible worlds’. Since the
notion of ‘possible world’ has proven hard to handle computationally, more
recent approaches have converged on a probabilistic definition of the
entailment relationship, relying on data-driven classification methods (cf.
Dagan et al., 2006, 2010). In such approaches, P is taken to entail Q if,
‘typically, a nhuman reading P would infer that Q is most likely true’ Dagan
et al. (2006). Most approaches to RTE seek to identify the semantic
relationship between P and Q based on their textual features.

With a few exceptions Xie et al. (2019); Lai (2018); Vu et al. (2018), NLI is
defined in unimodal terms, with no reference to the non-linguistic ‘world’.
This means that NLI models remain trapped in what Roy (2005) described as a
‘sensory deprivation tank’, handling symbolic meaning representations which
are ungrounded in the external world (cf. Harnad, 1990). On the other hand,
the recent surge of interest in NLP tasks at the vision-language interface
suggests ways of incorporating non-linguistic information into NLI. For
example, pairing the premises in Figure 1 with their corresponding images
could yield a more informative representation from which the semantic
relationship between premise and hypothesis could be determined. This would be
especially useful if the two modalities contained different types of
information, effectively making the textual premise and the corresponding
image complementary, rather than redundant with respect to each other.

(a) $P$: Four guys in wheelchairs on a basketball court two are trying to grab
a ball in midair
$Q$: Four guys are playing basketball.

(b) $P$: A little girl walking down a dirt road behind a group of other
walkers
$Q$: The girl is not at the head of the pack of people

Figure 1: Images with premises and hypotheses

While NLI is a classic problem in Natural Language Understanding, it also has
considerable relevance for generation. Many text generation tasks, including
summarisation Nenkova and McKeown (2011), paraphrasing Androutsopoulos and
Malakasiotis (2010), text simplification Siddharthan (2014) and question
generation Piwek and Boyer (2012), depend on the analysis and understanding of
an input text to generate an output that stands in a specific semantic
relationship to the input.

In this paper, we focus on the task of generating hypotheses from premises.
Here, the challenge is to produce a text that follows from (‘is entailed by’)
a given premise. Arguably, well-established tasks such as paraphrase
generation and text simplification are specific instances of this more general
problem. For example, in paraphrase generation, the input-output pair should
be in a relationship of meaning preservation.

In contrast to previous work on entailment generation Kolesnyk et al. (2016);
Starc and Mladenić (2017), we explore the additional benefits derived from
grounding textual premises in image data. We therefore assume that the data
consists of triples $\langle P,I,Q\rangle$, such that premises are accompanied
by their corresponding images (I) as well as the hypotheses, as in Figure 1.
We compare multimodal models to a unimodal setup, with a view to determining
to what extent image data helps in generating entailments.

The rest of this paper is structured as follows. In Section 2 we discuss
further motivations for viewing NLI and entailment generation as multimodal
tasks. Section 3 describes the dataset and architectures used; Section 4
presents experimental results, including a human evaluation. Our conclusion
(Section 5) is that it is feasible to frame NLI as a generation task and that
incorporating non-linguistic information is potentially profitable. However,
in line with recent research evaluating Vision-Language (VL) models Shekhar et
al. (2017); Wang et al. (2018); Vu et al. (2018); Tanti et al. (2019), we also
find that current architectures are unable to ground textual representations
in image data sufficiently.

##  2 NLI, Multimodal NLI and Generation

Since the work of Dagan et al. (2006), research on NLI has largely been data-
driven (see Sammons et al., 2012, for an overview). Most recent approaches
rely on neural architectures (e.g. Rocktäschel et al., 2015; Chen et al.,
2016; Wang et al., 2017), a trend which is clearly evident in the submissions
to recent RepEval challenges Nangia et al. (2017).

Data-driven NLI has received a boost from the availability of large datasets
such as SICK Marelli et al. (2014), SNLI Bowman et al. (2015) and MultiNLI
Williams et al. (2018). Some recent work has also investigated multimodal NLI,
whereby the classification of the entailment relationship is done on the basis
of image features Xie et al. (2019); Lai (2018), or a combination of image and
textual features Vu et al. (2018). In particular, Vu et al. (2018) exploited
the fact that the main portion of SNLI was created by reusing image captions
from the Flickr30k dataset Young et al. (2014) as premises, for which
entailments, contradictions and neutral hypotheses were subsequently
crowdsourced via Amazon Mechanical Turk Bowman et al. (2015). This makes it
possible to pair premises with the images for which they were originally
written as descriptive captions, thereby reformulating the NLI problem as a
Vision-Language task. There are at least two important motivations for this:

  1. 1.

The inclusion of image data is one way to bring together the classical and the
probabilistic approaches to NLI, whereby the image can be viewed as a
(partial) representation of the ‘world’ described by the premise, with the
entailment relationship being determined jointly from both. This is in line
with the suggestion by Young et al. (2014), that images be considered as akin
to the ‘possible worlds’ in which sentences (in this case, captions) receive
their denotation.

  2. 2.

A multimodal definition of NLI also serves as a challenging testbed for VL NLP
models, in which there has been increasing interest in recent years. This is
especially the case since neural approaches to fundamental computer vision
(CV) tasks have yielded significant improvements LeCun et al. (2015), while
also making it possible to use pre-trained CV models in multimodal neural
architectures, for example in tasks such as image captioning Bernardi et al.
(2016). However, recent work has cast doubt on the extent to which such models
are truly exploiting image features in a multimodal space Shekhar et al.
(2017); Wang et al. (2018); Tanti et al. (2019). Indeed, Vu et al. (2018) also
find that image data contributes less than expected to determining the
semantic relationship between premise-hypothesis pairs in the classic RTE
labelling task.

There have also been a few approaches to entailment generation, which again
rely on the SNLI dataset. Kolesnyk et al. (2016) employed a sequence-to-
sequence architecture with attention Sutskever et al. (2014); Bahdanau et al.
(2015) to generate hypotheses from premises. They extended this framework to
the generation of inference chains by recursively feeding the model with the
generated hypotheses as the input. Starc and Mladenić (2017) proposed
different generative neural network models to generate a stream of hypotheses
given $\langle$premise, Label$\rangle$ pairs as the input.

In seeking to reframe NLI as a generation task, the present paper takes
inspiration from these approaches. However, we are also interested in framing
the task as a multimodal, VL problem, in line with the two motivations noted
at the beginning of this section. Given that recent work has suggested
shortcomings in the way VL models adequately utilise visual information, a
focus on multimodal entailment generation is especially timely, since it
permits direct comparison between models utilising unimodal and multimodal
input.

As Figure 1 suggests, given the triple $\langle P,I,Q\rangle$, the hypothesis
Q could be generated from the premise $P$ only, from the image $I$, or from a
combination of $P+I$. Our question is whether we can generate better
entailments from a combination of both, compared to only one of these input
modalities.

##  3 Methodology

Figure 2: Architecture schema that is instantiated in the different models.
The encoder returns a source representation which conditions the decoder into
generating the entailed hypothesis. The encoder of multimodal models either
(1) initialises the encoder RNN with the image features (init-inject) and
returns the final state; or (2) concatenates the encoder RNN’s final state
with the image features (merge) and returns the result; or (3) returns the
image features as is without involving the encoder RNN. The unimodal model
encoder returns the encoder RNN’s final state as is. The source representation
is concatenated to the decoder RNN’s states prior to passing them to the
softmax layer.

###  3.1 Data

We focus on the subset of entailment pairs in the SNLI dataset Bowman et al.
(2015). The majority of instances in SNLI consist of premises that were
originally elicited as descriptive captions for images in Flickr30k Young et
al. (2014); ands Liwei Wang et al. (2015).111A subset of 4000 cases in SNLI
was extracted from the VisualGenome dataset Krishna et al. (2017). Like Vu et
al. (2018), we exclude these from our experiments.

In constructing the SNLI dataset, Amazon Mechanical Turk workers were shown
the captions/premises without the corresponding images, and were asked to
write a new caption that was (i) true, given the premise (entailment); (ii)
false, given the premise (contradiction); and (iii) possibly true (neutral).

Following Vu et al. (2018), we use a multimodal version of SNLI, which we
refer to as V-SNLI, which was obtained by mapping SNLI premises to their
corresponding Flickr30k images (Xie et al., 2019, use a similar approach). The
resulting V-SNLI therefore contains premises which are assumed to be true of
their corresponding image. Since we focus exclusively on the entailment
subset, we also assume, given the instructions given to annotators, that the
hypotheses are also true of the image. Figure 1 shows two examples of images
with both premises and hypotheses.

Each premise in the SNLI dataset includes several reference hypotheses by
different annotators. In the original SNLI train/dev/test split, some premises
show up in both train and test sets, albeit with different paired hypotheses.
For the present work, once premises were mapped to their corresponding
Flickr30k images, all premises corresponding to an image were grouped, and the
dataset was resplit so that the train/test partitions had no overlap,
resulting in a split of 182,167 (train), 3,291 (test) and 3,329 (dev)
entailment pairs. This also made it possible to use a multi-reference approach
to evaluation (see Section 3.3).

Prior to training, all texts were lowercased and tokenized.222Experiments
without lowercasing showed only marginal differences in the results.
Vocabulary items with frequency less than 10 were mapped to the token $\langle
UNK\rangle$. The unimodal model (see Section 3.2) was trained on textual P-Q
pairs from the original SNLI data, while the multimodal models were trained on
pairs from V-SNLI where the input consisted of P+I (for the multimodal
text+image model), or I only (for the image-to-text model), to generate
entailments.

###  3.2 Models

Figure 2 is an illustration of the architecture schema that is employed for
both multimodal and unimodal architectures. A sequence-to-sequence encoder-
decoder architecture is used to predict the entailed hypothesis from the
source. The source can be either a premise sentence, an image, or both.

Both the encoder and the decoder use a 256D GRU RNN and both embed their input
words using a 256D embedding layer. In multimodal models, the encoder either
produces a multimodal source representation that represents the sentence-image
mixture or produces just an image representation. In the unimodal model, the
encoder produces a representation of the premise sentence only.

This encoded representation (multimodal or unimodal) is then concatenated to
every state in the decoder RNN prior to sending it to the softmax layer, which
will predict the next word in the entailed hypothesis.

All multimodal models extract image features from the penultimate layer of the
pre-trained VGG-19 convolutional neural network (Simonyan and Zisserman,
2014). All models were implemented in Tensorflow, and trained using the the
Adam optimizer (Kingma and Ba, 2014) with a learning rate of .0001 and a
cross-entropy loss.

The models compared are as follows:

  1. 1.

Unimodal model: A standard sequence-to-sequence architecture developed for
neural machine translation (NMT). It uses separate RNNs for encoding the
source text and for decoding the target text. This model encodes the premise
sentence alone and generates the entailed hypothesis. In the unimodal
architecture, the final state of the encoder RNN is used as the representation
of the premise P.

  2. 2.

Multimodal, text+image input, init-inject (T+I-Init): A multimodal model in
which the image features are incorporated at the encoding stage, that is,
image features are used to initialise the RNN encoder. This architecture,
which is widely used in image captioning (Devlin et al., 2015; Liu et al.,
2016) is referred to as init-inject, following Tanti et al. (2018), on the
grounds that image features are directly injected into the RNN.

  3. 3.

Multimodal, text+image input, merge (T+I-Merge): A second multimodal model in
which the image features are concatenated with the final state of the encoder
RNN. Following Tanti et al. (2018), we refer to this as the merge model. This
too is adapted from a widely-used architecture in image captioning (Mao et
al., 2014, 2015a, 2015b; Hendricks et al., 2016). The main difference from
init-inject is that here, image features are combined with textual features
immediately prior to decoding at the softmax layer, so that the RNN does not
encode image features directly.

  4. 4.

Multimodal, image input only (IC): This model is based on a standard image
captioning setup (Bernardi et al., 2016) in which the entailed hypothesis is
based on the image features only, with no premise sentence. This architecture
leaves out the encoder RNN completely. Note that, while this is a standard
image captioning setup, it is put to a somewhat different use here, since we
do not train the model to generate captions (which correspond to the premises
in V-SNLI) but the hypotheses (which are taken to follow from the premises).

Model | BLEU-1 | METEOR | CIDEr | Perplexity
---|---|---|---|---
Unimodal | 0.695 | 0.267 | 0.938 | 7.23
T+I-Init | 0.634 | 0.239 | 0.763 | 10.73
T+I-Merge | 0.686 | 0.271 | 0.955 | 7.26
IC | 0.474 | 0.16 | 0.235 | 13.7
Table 1: Automatic evaluation results for unimodal and multimodal models

###  3.3 Evaluation metrics

We employed the evaluation metrics BLEU-1 (Papineni et al., 2002), METEOR
(Lavie and Agarwal, 2007) and CIDEr (Vedantam et al., 2015) to compare
generated entailments against the gold outputs in the model. We also compare
the model perplexity on the test set.

As noted earlier, we adopted a multi-reference approach to evaluation,
exploiting the fact that in our dataset we grouped all reference hypotheses
corresponding to each premise (and its corresponding image). Thus, evaluation
metrics are calculated by comparing the generated hypotheses to a group of
reference hypotheses. This is advantageous, since n-gram based metrics such as
BLEU and METEOR are known to yield more reliable results when multiple
reference texts are available.

##  4 Results

###  4.1 Metric-based evaluation

The results obtained by all models are shown in Table 1. The T+I-Merge
architecture outperforms all other models on CIDEr and METEOR, while the
unimodal model, relying only on textual premises, is marginally better on BLEU
and has slightly lower perplexity.

The lower perplexity of the unimodal model is is unsurprising given that in
the unimodal model, the decoder is only conditioned on textual features
without images. However, it should also be noted that the unimodal model also
fares quite well on the other two metrics, and is only marginally worse than
T+I-Merge. Its score on BLEU-1 is also competitive with that reported by
Kolesnyk et al. (2016) for their seq2seq model, which obtained a BLEU score of
0.428. Note, however, that this score was not obtained using a multi-reference
approach, that is, each generated candidate was compared to each reference
candidate, rather than the set of references together. Recomputing our BLEU-1
score for the Unimodal model in this way, we obtain a score of 0.395, which is
marginally lower than that reported by Kolesnyk et al. (2016).

The IC model, which generates entailments exclusively from images, ranks
lowest on all metrics. This is probably due to the fact that, contrary to the
usual setup for image caption generation, this model was trained on image-
entailment pairs rather than directly on image-caption pairs. Although, based
on the instructions given to annotators Bowman et al. (2015), entailments in
SNLI are presumed to be true of the image, it must be noted that the
entailments were not elicited as descriptive captions with reference to an
image, unlike the premises.

We discuss these results in more detail in Section 4.3, after reporting on a
human evaluation.

###  4.2 Human evaluation

We conducted a human evaluation experiment, designed to address the question
whether better entailments are generated from textual premises, images, or a
combination of the two. In the metric-based evaluation (Section 4.1), the best
multimodal model was T+I-Merge; hence, we use the outputs from this model,
comparing them to the outputs from the unimodal and the image-only IC model.

#### Participants

Twenty self-reported native or fluent speakers of English were recruited
through social media and the authors’ personal network.

#### Materials

A random sample of 90 instances from the V-SNLI test set was selected. Each
consisted of a premise, together with an image and three entailments generated
using the Unimodal, IC and I+T-Merge models, respectively. The 90 instances
were randomly divided into three groups. Participants in the evaluation were
similarly allocated to one of three groups. The groups were rotated through a
latin square so that each participant saw each of the 90 instances, one third
in each of the three conditions (Unimodal, IC or I+T-Merge). Equal numbers of
judgments were thus obtained for each instance in each condition, while
participants never saw a premise more than once.

Figure 3: Example of an evaluation test item, with a sentence generated from
both text and image.

#### Procedure

Participants conducted the evaluation online. Test items were administered in
blocks, by condition (text-only, image-only or text+image) and presented in a
fixed, randomised order to all participants. However, participants evaluated
items in the three conditions in different orders, due to the latin square
rotation. Each test case was presented with an input consisting of textual
premise, an image, or both. Participants were asked to judge to what extent
the generated sentence followed from the input. Answers were given on an
ordinal scale with the following qualitative responses: totally; partly; not
clear or not at all. Figure 3 shows an example with a generated entailment in
the text+image condition.

Figure 4: Proportion of responses in each category for each input condition.

#### Results

Figure 4 shows the proportion of responses in each category for each of the
three conditions. The figure suggests that human judgments were in line with
the trends observed in Section 4.1. More participants judged the generated
entailments as following totally or partly from the input, when the input
consisted of text and image together (T+I-Merge), followed closely by the case
where the input was text only (Unimodal). By contrast, in the case of the
image-only (IC) condition, the majority of cases were judged as not following
at all.

Possible reasons for these trends are discussed in the following sub-section
in light of further analysis. First, we focus on whether the advantage of
incorporating image features with text is statistically reliable. We coded
responses as a binomial variable, distinguishing those cases where
participants responded with ‘totally’ (i.e. the generated entailment
definitely follows from the input) from all others. Note that this makes the
evaluation conservative, since we focus only on the odds of receiving the most
positive judgment in a given condition.

We fitted logit mixed-effects models. All models included random intercepts
for participants and items (premise IDs) and random slopes for participants
and items by condition. Where a model did not converge, we dropped the by-
items random slope term. All models were fitted and tested using the lme4
package in R Bates et al. (2014).

The difference between the three types of input Condition significantly
affected the odds of a positive (‘totally’) response ($z=8.06,p<.001$).

We further investigated the impact of incorporating image features with or
without text through planned comparisons. A mixed-effects model comparing the
image-only to the text+image condition showed that the latter resulted in
significantly better output as judged by our participants ($z=7.80,p<.001$).
However, the text+image model did not significantly outperform the text-only
unimodal model, despite the higher percentage of positive responses in Figure
4 ($z=1.61,p>.1$). This is consistent with the CIDEr and METEOR scores, which
show that T+I-Merge outperforms the Unimodal model, but only marginally.

###  4.3 Analysis and discussion

Source | Sentence
---|---
P (Figure 1(a)) | Four guys in wheelchairs on a basketball court two are trying to grab a ball in midair.
Q (ref) | Four guys are playing basketball.
IC | A man is playing a game.
T+I-Init | Two men are jumping.
T+I-Merge | The basketball players are in the court.
Unimodal | The men are playing basketball.
P (Figure 1(b)) | A little girl walking down a dirt road behind a group of other walkers.
Q (ref) | The girl is not at the head of the pack of people.
IC | A man is wearing a hat.
T+I-Init | A young girl is walking down a path.
T+I-Merge | A group of people are walking down a road.
Unimodal | A girl is walking .
Table 2: Premises and reference captions, with output examples. These test
cases correspond to the images in Figure 1.

Two sets of examples of the outputs of different models are provided in Table
2, corresponding to the images with premises and entailments shown in Figure
1(a).

These examples, which are quite typical, suggest that the IC model is simply
generating a descriptive caption which only captures some entities in the
image. We further noted that in a number of cases, the IC model also generates
repetitive sentences that are not obviously related to the image. These are
presumably due to predictions relying extensively on the language model
itself, yielding stereotypical ‘captions’ which reflect frequent patterns in
the data. Together, these reasons may account for the poor performance of the
IC model on both the metric-based and the human evaluations.

The main difference between the Unimodal and the multimodal T+I models seems
to be that they generate texts that correspond to different, but arguably
valid, entailments of the premise. However, the fact remains that the
ungrounded Unimodal model yields good results that are close to those of
T+I-Merge. While humans judged the T+I-Merge entailments as better than the
unimodal ones overall, this is not a significant difference, at least as far
as the highest response category (‘totally’) is concerned.

These limitations of multimodal models echo previous findings. In image
captioning, for example, it has been shown that multimodal models are
surprisingly insensitive to changes in the visual input. For instance Shekhar
et al. (2017) found that VL models perform poorly on a task where they are
required to distinguish between a ‘correct’ image (e.g. one that corresponds
to a caption) and a completely unrelated foil image Shekhar et al. (2017).
Similarly, Tanti et al. (2019) find that image captioning models exhibit
decreasing sensitivity to the input visual features as more of the caption is
generated. In their experiments on multimodal NLI, Vu et al. (2018) also found
that image features contribute relatively little to the correct classification
of pairs as entailment, contradiction or neutral. These results suggest that
current VL architectures do not exploit multimodal information effectively.

Beyond architectural considerations, however, there are also properties of the
dataset which may account for why linguistic features play such an important
role in generating hypotheses. We discuss two of these in particular.

#### V-SNLI is not (quite) multimodal

SNLI hypotheses were elicited from annotators in a unimodal setting, without
reference to the images. Although we believe that our assumption concerning
the truth of entailments in relation to images holds (see Section 3.1), the
manner in which the data was collected potentially results in hypotheses which
are in large measure predictable from linguistic features, accounting for the
good performance of the unimodal model. If correct, this also offers an
explanation for the superior performance of the T+I-Merge model, compared to
T+I-Init. In the former, we train the encoder language model separately,
mixing image features at a late stage, thereby allocating more memory to
linguistic features in the RNN, compared to T+I-Init, where image features are
used to initialise the encoder. Apart from yielding lower perplexity (see
Table 1), this allows the T+I-Merge model to exploit linguistic features to a
greater extent than is possible in T+I-Init.

#### Linguistic biases

Recently, a number of authors have expressed concerns that NLI models may be
learning heuristics based on superficial syntactic features rather than
classifying entailment relationships based on a deeper ‘understanding’ of the
semantics of the input texts McCoy et al. (2019). Indeed, studies on the SNLI
dataset have shown that it contains several linguistic biases Gururangan et
al. (2018), such that the semantic relationship
(entailment/contradiction/neutral) becomes predictable from textual features
of the hypotheses alone, without reference to the premise. Gururangan et al.
(2018) identify a ‘hard’ subset of SNLI where such biases are not present.

Many of these biases are due to a high degree of similarity between a premise
and a hypothesis. For example, contradictions were often formulated by
annotators by simply including a negation, while entailments are sometimes
substrings of the premises, as in the following pair:

  * P

: A bicyclist riding down the road wearing a helmet and a black jacket.

  * Q

: A bicyclist riding down the road.

Such biases could also account for the non-significant difference in
performance between the two best models, Unimodal and T+I-Merge (which also
allocates all RNN memory to textual features, unlike T+I-Init). In either
case, it is possible that a hypothesis is largely predictable from the textual
premise, irrespective of the image, and generation boils down to ‘rewriting’
part of the premise to produce a similar string (see, e.g., the last example
in Table 2).

(a) Histogram of Dice overlap values

(b) CIDEr scores for Low vs High overlap test cases

Figure 5: Dice overlap vs CIDEr scores (T+I-Merge)

If textual similarity is indeed playing a role, then we would expect a model
to generate entailments ($Q_{gen}$) with a better CIDEr score, in those cases
where there is a high degree of overlap between the premise and the reference
hypothesis ($Q_{ref}$).

We operationalised overlap in terms of the Dice coefficient333
$Dice(P,Q_{ref})=\frac{2\times|P\cap Q_{ref}|}{|P|+|Q_{ref}|}$ , computed over
the sets of words in $P$ or $Q_{ref}$, after stop word removal.444Stopwords
were removed using the built-in English stopword list in the Python NLTK
library. As shown by the histogram in Figure 5(a), a significant proportion of
the $P$-$Q_{ref}$ pairs have a relatively high Dice coefficient ranging from
0.4 to 0.8. We divided test set instances into those with ‘low’ overlap (Dice
$\leq$ 0.3; $n=536$) and those with high overlap (Dice $>$ 0.3; $n=2755$).
Figure 5(b) displays the mean CIDEr score between entailments generated by the
T+I-Merge model, and reference entailments, as a function of whether the
reference entailment had high overlap with the hypothesis.

Clearly, for those cases where the overlap between $P$ and $Q_{ref}$ was high,
the T+I-Merge model obtains a higher CIDEr score between generated and
reference outputs. This is confirmed by Pearson correlation coefficients
between Dice coefficient and CIDEr, computed for each of the high/low overlap
subsets: On the subset with high Dice overlap, we obtain a significant
positive correlation ($r=0.20,p<.001$); on the subset with low overlap, the
correlation is far lower and does not reach significance ($r=.06,p>.1$).

##  5 Conclusion

This paper framed the NLI task as a generation task and compared the role of
visual and linguistic features in generating entailments. To our knowledge,
this was the first systematic attempt to compare unimodal and multimodal
models for entailment generation.

We find that grounding entailment generation in images is beneficial, but
linguistic features also play a crucial role. Two reasons for this may be
adduced. On the one hand, the data used was not elicited in a multimodal
setting, despite the availability of images. It also contains linguistic
biases, including relatively high degrees of similarity between premise and
hypothesis pairs, which may result in additional image information being less
important. On the other hand, it has become increasingly clear that Vision-
Language NLP models are not grounding language in vision to the fullest extent
possible. The present paper adds to this growing body of evidence.

Several avenues for future work are open, of which three are particularly
important. First, in order to properly assess the contribution of grounded
language models for entailment generation, it is necessary to design datasets
in which the textual and visual modalities are complementary, rather than
redundant with respect to each other. Second, while the present paper focused
exclusively on entailments, future work should also consider generating
contradictions. Finally, further research should investigate more
sophisticated Vision-Language architectures, especially incorporating
attention mechanisms.

## Acknowledgments

The first author worked on this project while at the University of Malta, on
the Erasmus Mundus European LCT Master Program. The third author was supported
by the Endeavour Scholarship Scheme (Malta). We thank Raffaella Bernardi,
Claudio Greco and Hoa Vutrong and our anonymous reviewers for their helpful
comments.

## References

  * Androutsopoulos and Malakasiotis (2010) Ion Androutsopoulos and Prodromos Malakasiotis. 2010.  A survey of paraphrasing and textual entailment methods.  _Journal of Artificial Intelligence Research_ , 38:135–187.
  * Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural Machine Translation By Jointly Learning To Align and Translate.  In _Proc. ICLR’15_.
  * Bates et al. (2014) Douglas Bates, Martin Maechler, and Ben Bolke. 2014.  lme4: Linear mixed-effects models using S4 classes.
  * Bernardi et al. (2016) Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erkut Erdem, Nazli Ikizler-Cinbis, Frank Keller, Adrian Muscat, and Barbara Plank. 2016.  Automatic description generation from images: A survey of models, datasets, and evaluation measures.  _Journal of Artificial Intelligence Research_ , 55(1):409–442.
  * Bowman et al. (2015) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Proc. EMNLP’15_.
  * Chen et al. (2016) Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2016\.  Enhanced LSTM for Natural Language Inference.  _arXiv_ , 1609.06038(2016).
  * Cooper et al. (1996) Robin Cooper, Dick Crouch, Jan Van Eijck, Chris Fox, Johan Van Genabith, Jan Jaspars, Hans Kamp, David Milward, Manfred Pinkal, Massimo Poesio, and Steve Pulman. 1996.  Using the framework.  Technical Report Technical Report LRE 62-051 D-16, The FraCaS Consortium.
  * Dagan et al. (2010) Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. 2010.  The fourth pascal recognizing textual entailment challenge.  _Natural Language Engineering_.
  * Dagan et al. (2006) Ido Dagan, Oren Glickman, Bernardo Magnini, and Ramat Gan. 2006.  The PASCAL Recognising Textual Entailment Challenge.  In J. Quinonero-Candela, I. Dagan, B. Magnini, and F D’Alche-Buc, editors, _Machine Learning Challenges_ , pages 177–190. Springer, Berlin and Heidelberg.
  * Devlin et al. (2015) Jacob Devlin, Hao Cheng, Hao Fang, Saurabh Gupta, Li Deng, Xiaodong He, Geoffrey Zweig, and Margaret Mitchell. 2015.  Language models for image captioning: The quirks and what works.  In _Proc. ACL’15_.
  * Gururangan et al. (2018) Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, and Noah A. Smith. 2018.  Annotation Artifacts in Natural Language Inference Data.  _ArXiv_ , 1803.02324.
  * Harnad (1990) Stevan Harnad. 1990.  The symbol grounding problem.  _Physica_ , D42:335–346.
  * Hendricks et al. (2016) Lisa Anne Hendricks, Subhashini Venugopalan, Marcus Rohrbach, Raymond Mooney, Kate Saenko, and Trevor Darrell. 2016.  Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Data.  In _Proc. CVPR’16_.
  * Kingma and Ba (2014) Diederik P. Kingma and Jimmy Lei Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv_ , abs/1412.6980.
  * Kolesnyk et al. (2016) Vladyslav Kolesnyk, Tim Rocktäschel, and Sebastian Riedel. 2016.  Generating natural language inference chains.  _arXiv_ , abs/1606.01404.
  * Krishna et al. (2017) Ranjay Krishna, Yuke Zhu, Oliver GrothJustin, Johnson Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. 2017.  Visual genome: Connecting language and vision using crowdsourced dense image annotations.  _International Journal of Computer Vision_ , 123(1):32–73.
  * Lai (2018) Alice Yingming Lai. 2018.  _Textual entailment from image caption denotation_.  Phd thesis, University of Illinois at Urbana-Champaign.
  * Lavie and Agarwal (2007) Alon Lavie and Abhaya Agarwal. 2007.  METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments.  In _Proc. ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization_ , Ann Arbor, Michigan.
  * LeCun et al. (2015) Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015.  Deep learning.  _Nature_ , 521(7553):436–444.
  * Liu et al. (2016) Siqi Liu, Zhenhai Zhu, Ning Ye, Sergio Guadarrama, and Kevin Murphy. 2016.  Optimization of image description metrics using policy gradient methods.  _ArXiv_ , 1612.00370.
  * ands Liwei Wang et al. (2015) Bryan A. Plummer ands Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazebnik. 2015.  Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models.  _arXiv_ , abs/1505.04870.
  * Mao et al. (2015a) Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan Yuille. 2015a.  Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN).  _Proc. ICLR’15_.
  * Mao et al. (2015b) Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan Yuille. 2015b.  Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images.  In _Proc. ICCV’15_.
  * Mao et al. (2014) Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, and Alan L. Yuille. 2014.  Explain images with multimodal recurrent neural networks.  _Proc. NIPS’14_.
  * Marelli et al. (2014) Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella bernardi, and Roberto Zamparelli. 2014.  A SICK cure for the evaluation of compositional distributional semantic models.  In _Proc. LREC’14_.
  * McCoy et al. (2019) R. Thomas McCoy, Ellie Pavlick, and Tal Linzen. 2019.  Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference.  In _Proc. ACL’19_.
  * Nangia et al. (2017) Nikita Nangia, Adina Williams, Angeliki Lazaridou, and Samuel R Bowman. 2017.  The RepEval 2017 Shared Task: Multi-Genre Natural Language Inference with Sentence Representations.  In _Proc. 2nd Workshop on Evaluating Vector Space Representations for NLP_.
  * Nenkova and McKeown (2011) Ani Nenkova and Kathleen R McKeown. 2011.  Automatic Summarization.  _Foundations and Trends® in Information Retrieval_ , 5(2-3):103–233.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-jing Zhu. 2002.  BLEU : a Method for Automatic Evaluation of Machine Translation.  In _Proc. ACL’02)_ , Philadelphia, PA.
  * Piwek and Boyer (2012) Paul Piwek and Kristy Elizabeth Boyer. 2012.  Varieties of question generation: Introduction to this special issue.  _Dialogue and Discourse_ , 3(2):1–9.
  * Rocktäschel et al. (2015) Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomás Kociský, and Phil Blunsom. 2015.  Reasoning about entailment with neural attention.  _arXiv_ , abs/1509.06664.
  * Roy (2005) Deb Roy. 2005.  Semiotic schemas: A framework for grounding language in action and perception.  _Artificial Intelligence_ , 167(1-2):170–205.
  * Sammons et al. (2012) Mark Sammons, Vinod Vydiswaran, and Dan Roth. 2012.  Recognising Textual Entailment.  In Daniel M. Bikel Zitouni and Imed, editors, _Multilingual Natural Language Processing_ , pages 209–281. IBM Press, Westford, MA.
  * Shekhar et al. (2017) Ravi Shekhar, Sandro Pezzelle, Yauhen Klimovich, Aurelie Herbelot, Moin Nabi, Enver Sangineto, and Raffaella Bernardi. 2017.  FOIL it! Find One mismatch between Image and Language caption.  In _Proc. ACL’17_.
  * Siddharthan (2014) Advaith Siddharthan. 2014.  A survey of research on text simplification.  _International Journal of Applied Linguistics_ , 165(2):259–298.
  * Simonyan and Zisserman (2014) Karen Simonyan and Andrew Zisserman. 2014.  Very Deep Convolutional Networks for Large-Scale Image Recognition.  _arXiv_ , 1409.1556.
  * Starc and Mladenić (2017) Janez Starc and Dunja Mladenić. 2017.  Constructing a Natural Language Inference dataset using generative neural networks.  _Computer Speech and Language_ , 46:94–112.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014.  Sequence to sequence learning with neural networks.  In _Proc. NIPS’14_.
  * Tanti et al. (2018) M Tanti, A Gatt, and K Camilleri. 2018.  Where to put the image in an image caption generator. _Natural Language Engineering_ , 24:467–489.
  * Tanti et al. (2019) M Tanti, A Gatt, and K Camilleri. 2019.  Quantifying the amount of visual information used by neural caption generators.  In _Proc. Workshop on Shortcomings in Vision and Language_ , pages 124–132.
  * Vedantam et al. (2015) Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015.  CIDEr: Consensus-based image description evaluation.  In _Proc. CVPR’15_.
  * Vu et al. (2018) Hoa Trong Vu, Claudio Greco, Aliia Erofeeva, Somayeh Jafaritazehjani, Guido Linders, Marc Tanti, Alberto Testoni, Raffaella Bernardi, and Albert Gatt. 2018\.  Grounded textual entailment.  In _Proc. COLING’18_.
  * Wang et al. (2018) Josiah Wang, Pranava Madhyastha, and Lucia Specia. 2018.  Object Counts! Bringing Explicit Detections Back into Image Captioning.  In _Proc. NAACL-HLT’18_.
  * Wang et al. (2017) Zhiguo Wang, Wael Hamza, and Radu Florian. 2017.  Bilateral Multi-Perspective Matching for Natural Language Sentences.  In _Proc. IJCAI’17_.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018.  A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference.  In _Proc. NAACL-HLT’18_.
  * Xie et al. (2019) Ning Xie, Farley Lai, Derek Doran, and Asim Kadav. 2019.  Visual Entailment: A Novel Task for Fine-Grained Image Understanding.  _ArXiv_ , 1901.06706v1.
  * Young et al. (2014) Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  _Transactions of the Association for Computational Linguistics_ , 2:67–78.


EFI 23-X

arXiv:2305.00354

SINGLE-PHOTON DECAYS

IN SYSTEMS WITH AT LEAST ONE HEAVY QUARK

Marek<EMAIL_ADDRESS>and Jonathan L.
<EMAIL_ADDRESS>
a School of Physics and Astronomy

Raymond and Beverly Sackler Faculty of Exact Sciences

Tel Aviv University, Tel Aviv 69978, Israel

b Enrico Fermi Institute and Department of Physics

University of Chicago, 5640 S. Ellis Avenue, Chicago, IL 60637, USA

ABSTRACT

> Hadrons containing at least one heavy quark (charm or bottom) frequently
> have small enough natural widths that decay modes involving a single photon
> have detectable branching fractions. Photons of typical energy greater than
> 100 MeV have been directly detected, while those of lower energy have only
> been inferred. Here we discuss prospects for observing direct sub-100 MeV
> photons in specific radiative decays of charmed and bottom vector mesons, as
> well as a spin-excited heavy baryon.

##  I Introduction

The spectroscopy of mesons and baryons containing at least one heavy quark
($c$ or $b$) has blossomed in recent years. In part this is because many newly
discovered states have had small natural widths, tens of MeV or less, allowing
single-photon emission to compete favorably with hadronic decay. The reduction
of partial widths is further enhanced if the heavy hadron contains no light
quarks ($u$ or $d$), because coupling to pions then is suppressed.

Table I lists some decays of interest here. The single photons of energy
typically greater than 100 MeV have been directly detected, while those of
energy less than 100 MeV (marked with an asterisk) mostly remain to be
observed. Many are produced in decays of vector mesons $V$ to pseudoscalar
mesons $P$ and single photons. Their low energies are due to the small
hyperfine splittings between the initial $V$ and final $P$. Table II gives the
corresponding data for the charmed-strange baryons $\Xi_{c}^{+,0}$. All data
are quoted from PDGLive [1] except where indicated. The rare cases in which a
direct photon of less than 100 MeV is observed (e.g., in
$\Upsilon(3S)\to\chi_{b1,2}(2P)\gamma$) are due to low backgrounds and good
final-state reconstruction.

Table I: Some $X\to Y\gamma$ transitions for states with at least one heavy
quark. Asterisks denote decays not yet observed directly.

$X$ | $Y$ | $E_{\gamma}$ | BR
---|---|---|---
|  | (MeV) | (%)
$D^{*0}$ | $D^{0}$ | 137 |  $35.3\pm 0.9$ (a)
$D^{*+}$ | $D^{+}$ | 136 |  $1.6\pm 0.4$ (a)
$D_{s}^{*+}$ | $D_{s}^{+}$ | 139 | $93.5\pm 0.7$
$B^{*}$ | $B$ | 45∗ | 100
$B_{s}^{*0}$ | $B_{s}^{0}$ | 48∗ | 100
$B_{c}^{*+}$ | $B_{c}^{+}$ | 54–84∗ (b) | 100
$J/\psi$ | $\eta_{c}$ | 111 | $1.7\pm 0.4$
$\Upsilon(1S)$ | $\eta_{b}$ | $61\pm 2^{*}$ | (c)
$\Upsilon(2S)$ | $\eta_{b}$ | 605 | $(5.5^{+1.1}_{-0.9})\times 10^{-4}$
$\Upsilon(2S)$ | $\chi_{b0}$ | 162 | $3.8\pm 0.4$
$\Upsilon(2S)$ | $\chi_{b1}$ | 130 | $6.9\pm 0.4$
$\Upsilon(2S)$ | $\chi_{b2}$ | 110 | $7.15\pm 0.35$
$\Upsilon(3S)$ | $\chi_{b0}(2P)$ | 122 | $5.9\pm 0.6$
$\Upsilon(3S)$ | $\chi_{b1}(2P)$ | 99 | $12.6\pm 1.2$
$\Upsilon(3S)$ | $\chi_{b2}(2P)$ | 86 | $13.1\pm 1.6$

(a) In $D^{*0}\to D^{0}\gamma$ the contributions of the charmed and light
quarks add

constructively, while they nearly cancel one another in $D^{*+}\to
D^{+}\gamma$.

(b) To be determined in this work; our present estimate of range.

(c) The branching fraction to three gluons is about 82%, while that to

each lepton pair is about 2.5% and to $\gamma gg$ is about 2.2%. Thus the

branching fraction for $\Upsilon(1S)\to\eta_{b}\gamma$, though not quoted, is
expected

to be considerably less than 1.

Table II: Single-photon emission by baryons. States with at least one heavy quark.  Initial | Final | $E\gamma$ | BR
---|---|---|---
baryon | baryon | (MeV) | (%)
$\Xi_{c}^{{}^{\prime}+}$ | $\Xi_{c}^{+}$ | 108 | 100
$\Xi_{c}^{{}^{\prime}0}$ | $\Xi_{c}^{0}$ | 106 | 100

Some predicted M1 transition rates are summarized in Table III.

Table III: M1 transition rates and branching ratios for $M^{*}\to M\gamma$
decays: predicted [Eq. (1) with $|I|^{2}=1/2$] and (where available) observed
[1]. Quark masses are taken to be $m_{u}=m_{d}=308.5$ MeV, $m_{s}=482.2$ MeV,
$m_{c}=1665$ MeV, $m_{b}=5041$ MeV.

Quarks |  |  | Rate $\Gamma$ | Branching ratio
---|---|---|---|---
$q_{1}\bar{q}_{2}$ | $M^{*}$ | $M$ | Pred (keV) | Pred | Obs
$c\bar{u}$ | 2006.8 | 1864.8 | 20.54 | (a) | (35.3$\pm$0.9)%
$c\bar{d}$ | 2010.3 | 1869.7 | 1.40 | (a) | (1.6$\pm$0.4)%
$c\bar{s}$ | 2112.2 | 1968.3 | 0.272 |  | (93.5$\pm$0.7)%
$c\bar{c}$ | 3096.9 | 2983.9 | 1.077 | 1.16% | (1.7$\pm$0.4)%
$b\bar{d}$ | 5324.7 | 5279.7 | 0.144 | – | 100%
$b\bar{u}$ | 5324.7 | 5279.3 | 0.491 | – | 100%
$b\bar{s}$ | 5415.4 | 5366.9 | 0.0787 | – | 100%
$b\bar{c}$ | 6343.5 | 6274.5 | 0.0443 | – | 100%
$b\bar{b}$ | 9460.3 | 9398.7 | 0.00503 | $9.3\times 10^{-5}$ | –

(a) See Eqs. (2,3) for prediction of hadronic and radiative $D^{*}$ widths.

We shall describe magnetic dipole (M1) radiative transitions by sums of quark
spin-flip amplitudes. The decays of vector mesons $M^{*}$ to pseudoscalar
mesons $M$ composed of quarks $q_{1}\bar{q}_{2}$ proceed at a rate [2]

$\Gamma(M^{*}\to
M\gamma)=\frac{p_{\gamma}^{3}}{3\pi}[\mu_{1}+\mu_{2}]^{2}|I|^{2}~{},$ (1)

where $p_{\gamma}$ is the photon momentum in the c.m.s., $\mu_{1}$ and
$\mu_{2}$ are the magnetic moments of the quarks, and
$\mu_{i}=eQ_{i}/(2m_{i})$, with $eQ_{i}$ denoting the charge of quark $i$. The
amplitude $I=\langle f|i\rangle$ denotes the overlap between initial and final
wave functions. For a wide range of cases, involving both light and heavy
quarks, $|I|^{2}\simeq 1/2$. (For a summary of early work on radiative decays
of vector mesons to pseudoscalar mesons see Ref. [3].)

In this paper we discuss status and prospects for observing direct photons in
$D^{*}\to D\gamma$ and $D_{s}^{*}\to D_{s}\gamma$ in Sec. II, $B^{*}\to
B\gamma$ and $B_{s}^{*}\to B_{s}\gamma$ in Sec. III,
$\Upsilon(1S)\to\eta_{b}\gamma$ in Sec. IV, $\Xi_{c}^{\prime}\to\Xi_{c}\gamma$
in Sec. V and $B_{c}^{*}\to B_{c}\gamma$ in Sec. VI, concluding in Sec. VII.
An Appendix compares amplitudes to M1 transitions.

##  II $D^{*}\to D\gamma$ and $D_{s}^{*}\to D_{s}\gamma$

###  A Nonstrange $D$ mesons

The spacing between the pseudoscalar and vector nonstrange $D$ mesons is just
large enough to accommodate single pion emission, leading to hadronic decays
which can compete with single photon emission. The pattern of $D$ decay widths
was analyzed in Ref. [4], with successful descriptions of hadronic [5] and
radiative (M1) transitions. Combining hadronic and radiative transitions, it
was predicted that

$\Gamma_{\rm tot}(D^{*+})=(79.0+2.76|I|^{2})~{}{\rm
keV}~{}~{}~{},~{}~{}~{}\Gamma_{\rm tot}(D^{*0})=(34.7+40.8|I|^{2})~{}{\rm
keV}~{},$ (2)

to be compared with experimental values [1]

$\Gamma_{\rm tot}(D^{*+})=(83.4\pm 1.8)~{}{\rm
keV}~{}~{}~{},~{}~{}~{}\Gamma_{\rm tot}(D^{*0}<2.1~{}{\rm MeV}~{}.$ (3)

Note in particular the relatively large value of the partial width for
$D^{*0}\to D^{0}\gamma$ quoted in Table III. Using the expression in Eq. (2)
for $\Gamma_{\rm tot}(D^{*0}$, and the observed branching fraction ${\cal
B}(D^{*0}\to D^{0}\gamma)=35.3\%$, one finds $|I|^{2}=0.576$ and $\Gamma_{\rm
tot}(D^{*0})=58.2$ keV.

###  B Strange $D$ mesons

The hadronic decay of $D_{s}^{*+}$ to $\pi\pi D_{s}^{+}$ is forbidden
kinematically and to $\pi^{0}D_{s}^{+}$ is forbidden by isospin. The dominant
decay is then $D_{s}^{*}\to D_{s}\gamma$, accounting for a branching fraction
of $(93.5\pm 0.07)\%$ and hence a total width of $(291\pm 8)$ eV. A
determination of this quantity in lattice QCD [12] finds a much smaller value,
$\Gamma_{\rm tot}(D_{s}^{*})=(70\pm 28)$ eV, which has been combined with an
experimental study of the purely leptonic decay $D_{s}^{*+}\to e^{+}\nu_{e}$
to obtain a first value of the $D_{s}^{*+}$ decay constant [13]. It would be
interesting to see if a similar discrepancy between predicted and observed
total widths holds for the $D^{*0}$ decays.

##  III $B^{*}\to B\gamma$ and $B_{s}^{*}\to B_{s}\gamma$

The two charge states in $B^{*}\to B\gamma$ are not separated from one another
[1], so one is missing valuable information on the masses of $B^{*0}$ and
$B^{*+}$. This in turn would add data to systematics of isospin splittings
[6].

The detection of a photon of 45 MeV in $B^{*}\to B\gamma$ or 48 MeV in
$B_{s}^{*}\to B_{s}\gamma$ poses a significant challenge. As in all processes
discussed here and below, reconstruction of the final state is essential. If
there are enough events, one may gain background suppression by asking for a
Dalitz decay in which the photon materializes as an $e^{+}e^{-}$ pair.
Furthermore, a boosted decay geometry such as is available in LHCb may
compensate for the very low photon energy in the center-of-mass (c.m.). On the
other hand, fully reconstructed final states in $e^{+}e^{-}$ reactions, such
as provided by Belle II at a c.m. energy of the $\Upsilon(4S)$, may have
backgrounds low enough to permit detection of the $\sim 45$ MeV photons.

##  IV $\Upsilon(1S)\to\eta_{b}\gamma$

The $\eta_{b}(1S)$ was not discovered through the decay
$\Upsilon(1S)\to\eta_{b}\gamma$, which requires detection of a 61 MeV photon.
It was seen with highest statistics by Belle and BaBar in the decay
$\Upsilon(2S)\to\eta_{b}\gamma$ [1] (see $\Gamma_{75}$), in which
$E_{\gamma}\simeq 605$ MeV. Adding to the difficulty is the absence of a
large, easily reconstructed $\eta_{b}$ branching fraction. The best route to
detection of the 61 MeV photon would be to study the exclusive final states
$X$ in the observed $\eta_{b}$ decays and then select them in $\Upsilon(1S)\to
X\gamma$.

##  V $\Xi_{c}^{\prime}\to\Xi_{c}\gamma$

The photon in either charge state of $\Xi_{c}^{\prime}\to\Xi_{c}\gamma$ has an
energy of $106$–$108$ MeV. These decays are marked as “seen” in the Particle
Listings [1], based primarily on 2016 Belle data [7]. Their analysis of the
decays $\Xi_{c}^{\prime}\to\Xi_{c}\gamma$ is a textbook example of the care
that must be taken in detecting a low-energy photon. The strength of the
observed signals suggests sensitivity to photon energies well below 100 MeV.

##  VI $B_{c}^{*}\to B_{c}\gamma$

The study of the $b\bar{c}$ system is hampered by ignorance of the splitting
$\Delta_{1S}\equiv M[B_{c}^{*}(^{3}S_{1})]-M[B_{c}(^{1}S_{0})]$. Some early
estimates include ones quoted in Table IV, while some recent ones include 54
MeV [14] and 68–84 MeV [15]. We thus take $\Delta_{1S}$ = 54 MeV to 84 MeV, or
$69\pm 15$ MeV. Splitting between the 1S and 2S states has been studied by
ATLAS [16], CMS [17], and LHCb [18]. Taking $\Delta_{1S}=69\pm 15$ MeV, we
estimate $M[B^{*}_{c}(1S)]$ in the last line of Table IV. A feature which
distinguishes the CMS result from the others is the observation of two narrow
peaks in the $M(B_{c}^{+}\pi^{+}\pi^{-})-M(B_{c}^{+})$ spectrum, suggesting
good final-state reconstruction. These peaks are separated by $29.1\pm 1.5\pm
0.7$ MeV, with the lower most likely corresponding to the ${}^{3}S_{1}$ states
and the higher most likely corresponding to the ${}^{1}S_{0}$ states.§§§This
assumption is motivated by theoretical quark model calculations. One can
regard the 29.1 MeV as representing the difference between the hyperfine
splittings of the 1S and 2S $B_{c}$ doublets. One uses this quantity in
estimating $M[B^{*}_{c}(2S)]$ in the last line of Table IV. Detection of the
photon in $B^{*}_{c}(1S)\to B_{c}(1S)\gamma$ thus would be a useful adjunct to
the spectroscopy of these states.

Table IV: Some early and current predictions of 1S and 2S $b\bar{c}$ masses (MeV) compared with present values ($M(B_{c})$) or estimates ($M(B_{c}^{*})$).  Reference | $M[B_{c}(1S)]$ | $M[B^{*}_{c}(1S)]$ | $M[B_{c}(2S)]$ | $M[B^{*}_{c}(2S)]$
---|---|---|---|---
Eichten94 [8] | 6264 | 6337 | 6856 | 6899
Gershtein95 [9] | 6253 | 6317 | 6867 | 6902
Fulcher99 [10] | 6286 | 6341 | 6882 | 6914
Ebert03 [11] | 6270 | 6332 | 6835 | 6881
Eichten19 [14] | 6275 | 6329 | 6867 | 6898
Present | $6274.47\pm 0.32$ | $6343\pm 15$ | $6872.1\pm 1.0$ | $6912\pm 15$

Although the CMS collaboration reports a very small (1%) reconstruction
efficiency for the soft photon in $B_{c}^{*}\to B_{c}\gamma$, it might be
worth while exploring ways to increase this efficiency. Possibilities include
(i) selecting a sample of the most highly boosted $B_{c}$ events, (ii)
replacing the photon with an $e^{+}e^{-}$ pair (Dalitz decays), and (iii)
looking for photons which convert in the material of the detector.

##  VII Conclusions

The ability to reconstruct photons of energies less than 100 MeV would greatly
enhance the versatility of detectors studying the spectroscopy of states with
one or more heavy ($c$ or $b$) quarks. We have discussed ways of improving
their sensitivity, in such decays as $B^{*}\to B\gamma$, $B_{s}^{*}\to
B_{s}\gamma$, $\Upsilon(1S)\to\eta_{b}\gamma$, and $B_{c}^{*}\to B_{c}\gamma$.
These decays are also of interest because they can shed light on the relative
strengths of weak decays. We thank Tim Gershon for emphasizing this point, and
alerting us to the suggestion of Ref. [19].

## Appendix

The individual contributions of magnetic moments of quarks $q_{i}$ or
antiquarks $\bar{q}_{i}$ are terms $eQ_{i}/(2m_{i})$ proportional to magnetic
moments which interfere with one another constructively or destructively in
Eq. (1). They are compared in Table V and Figure 1.

Table V: Amplitudes $\mu=eQ/(2m_{q})$ contributing to M1 transitions involving single-photon decays. The arrows in each box give the contributions of quark $q_{1}$, antiquark $\bar{q}_{2}$, and their sum (all in units of $10^{-6}$ (MeV)-1). Decay | Quarks | $\mu(q_{1})$ | $\mu(\bar{q}_{2})$ | $\mu(q_{1})+\mu(\bar{q}_{2})$
---|---|---|---|---
$D^{*0}\to D^{0}\gamma$ | $c\bar{u}$ |  —61 |  —327 |  —388
$D^{*+}\to D^{+}\gamma$ | $c\bar{d}$ |  —61 | $-164$ | $-103$
$D^{*}_{s}\to D_{s}\gamma$ | $c\bar{s}$ |  —61 | $-105$ |  –$-44$
$J/\psi\to\eta_{c}\gamma$ | $c\bar{c}$ |  —61 |  —-61 |  —122
$\bar{B}^{*0}\to\bar{B}^{0}\gamma$ | $b\bar{d}$ | $-10$ | $-164$ | $\,-174$
$B^{*-}\to B^{-}\gamma$ | $b\bar{u}$ | $-10$ |  —327 |  —317
$B^{*}_{s}\to B_{s}\gamma$ | $b\bar{s}$ | $-10$ | $-105$ | $\,-115$
$B^{*}_{c}\to B_{c}\gamma$ | $b\bar{c}$ | $-10$ |  —-61 |  —-51
$\Upsilon(1S)\to\eta_{b}\gamma$ | $b\bar{b}$ | $-10$ |  -$-10$ |  –$-20$
Figure 1: Graphical depiction of Table V. Relative amplitudes contributing to
magnetic dipole (M1) transitions of $q_{1}\bar{q}_{2}$ states from
$J^{P}=1^{-}$ to $0^{-}$. In each box the first arrow denotes the contribution
of $q_{1}$, the second denotes the contribution of $\bar{q}_{2}$, and the
third denotes their sum.

Several properties are notable.

(1) The largest amplitude is for $D^{*0}\to D^{0}\gamma$, benefitting from a
large radiative contribution from the $u$ quark magnetic moment and
constructive interference with a $c$ quark moment. The radiative width of
$D^{*0}\to D^{0}\gamma$ then becomes comparable to the hadronic width of
$D^{*0}\to D^{0}\pi^{0}$ [4].

(2) The $d$ quark moment’s contribution to $(D^{*+}\to
D^{+}\gamma,~{}\bar{B}^{*0}\to\bar{B}^{*0}\gamma)$ is -1/2 times the $u$ quark
moment’s contribution to ($D^{*0}\to D^{0}\gamma,~{}B^{*-}\to B^{-}\gamma$),
simply reflecting the charge of the light quark.

(3) In $D^{*+}\to D^{+}\gamma$ the $c$ and $d$ quark moments interfere
destructively. The combined effect of (2) and (3) is to give a much smaller
radiative width for $D^{*+}\to D^{+}\gamma$ than for $D^{*0}\to D^{0}\gamma$.

(4) In comparing $D_{s}^{*}\to D_{s}\gamma$ with $D^{*+}\to D^{+}\gamma$, the
percent of U-spin violation in the $s$ vs. $d$ contributions is amplified by
destructive interference with the $c$ contribution. Thus the radiative width
for $D_{s}^{*}\to D_{s}\gamma$ is quite sensitive to the exact pattern of this
interference.

(5) For example, the measurement of the branching fraction ${\cal
B}(D_{s}^{*}\to e^{+}\nu_{e})=(2.1^{+1.2}_{-0.9}\pm 0.2)\%$ [13] combined with
the Particle Data Group [1] average ${\cal B}(D_{s}^{*}\to
D_{s}\gamma)=(93.5\pm 0.7)\%$ and Eq. (1) in the BESIII paper for the partial
width $\Gamma(D_{s}^{*}\to e^{+}\nu_{e})$ gives

$\Gamma(D_{s}^{*}\to D_{s}\gamma)=\Gamma(D_{s}^{*}\to e^{+}\nu_{e})/{\cal
B}(D_{s}^{*}\to e^{+}\nu_{e})\times{\cal B}(D_{s}^{*}\to
D_{s}\gamma)\,=\,(114^{+65}_{-50})~{}{\rm eV}~{},$ (4)

to be compared with a lattice QCD [12] estimate

$\Gamma(D_{s}^{*}\to D_{s}\gamma)=66\pm 26~{}{\rm eV},$ (5)

even smaller than the value of

$\Gamma(D_{s}^{*}\to D_{s}\gamma)=272~{}{\rm eV}|I|^{2}/(1/2)$ (6)

obtained in the present work (cf. 3rd row of Table III).

(6) The magnetic dipole transition amplitude for $B^{*-}\to B^{-}\gamma$ is
appreciable, not much affected by a small destructive interference between the
large $u$ quark and small $b$ quark contributions.

(7) Some of the M1 transition rates we have been discussing, while being the
main or exclusive decay modes of the vector meson of interest, are
sufficiently low that weak decays can appear with detectable branching ratios.
This has already been seen by BESIII [13] in the case of $D_{s}^{*}\to
D_{s}\gamma$ vs. $D_{s}^{*}\to e^{+}\nu_{e}$, and may also be promising for
$B^{*-}$, $B_{s}^{*}$, and $B_{c}^{*}$ decays (see Table III).

## Acknowledgments

We thank Christine Davies and Tim Gershon for helpful comments. The work of
M.K. was supported in part by NSFC-ISF Grant No. 3423/19.

## References

  * [1] R. L. Workman et al. [Particle Data Group], “Review of Particle Physics,” PTEP 2022, 083C01 (2022).
  * [2] E. Eichten, K. Gottfried, T. Kinoshita, K. D. Lane and T. M. Yan, “Charmonium: The Model,” Phys. Rev. D 17, 3090 (1978) [erratum: Phys. Rev. D 21, 313 (1980)]
  * [3] R. Casalbuoni, A. Deandrea, N. Di Bartolomeo, R. Gatto, F. Feruglio and G. Nardulli, “Phenomenology of heavy meson chiral Lagrangians,” Phys. Rept. 281 (1997), 145-238 [arXiv:hep-ph/9605342 [hep-ph]].
  * [4] J. L. Rosner, “Hadronic and radiative $D^{*}$ widths,” Phys. Rev. D 88, 034034 (2013) [arXiv:1307.2550 [hep-ph]].
  * [5] F. Amundson, C. G. Boyd, E. E. Jenkins, M. E. Luke, A. V. Manohar, J. L. Rosner, M. J. Savage and M. B. Wise, “Radiative D* decay using heavy quark and chiral symmetry,” Phys. Lett. B 296, 415-419 (1992) [arXiv:hep-ph/9209241 [hep-ph]].
  * [6] M. Karliner and J. L. Rosner, “Status of isospin splittings in mesons and baryons,” Phys. Rev. D 100, 073006 (2019) [arXiv:1906.07799 [hep-ph]].
  * [7] J. Yelton et al. [Belle], “Study of Excited $\Xi_{c}$ States Decaying into $\Xi_{c}^{0}$ and $\Xi_{c}^{+}$ Baryons,” Phys. Rev. D 94, 052011 (2016) [arXiv:1607.07123 [hep-ex]].
  * [8] E. J. Eichten and C. Quigg, “Mesons with beauty and charm: Spectroscopy,” Phys. Rev. D 49, 5845 (1994)[arXiv:hep-ph/9402210].
  * [9] S. S. Gershtein, V. V. Kiselev, A. K. Likhoded, and A. V. Tkabladze,“B(c) spectroscopy,” Phys. Rev. D 51, 3613 (1995) [arXiv:hep-ph/9406339].
  * [10] L. P. Fulcher, “Phenomenological predictions of the properties of the $B_{c}$ system,” Phys. Rev. D 60, 074006 (1999) [arXiv:hep-ph/9806444].
  * [11] D. Ebert, R. N. Faustov, and V. O. Galkin, “Properties of heavy quarkonia and $B_{c}$ mesons in the relativistic quark model,” Phys. Rev. D 67, 014027 (2003) [arXiv:hep-ph/0210387].
  * [12] G. C. Donald, C. T. H. Davies, J. Koponen and G. P. Lepage, “Prediction of the $D_{s}^{*}$ width from a calculation of its radiative decay in full lattice QCD,” Phys. Rev. Lett. 112 (2014), 212002 [arXiv:1312.5264 [hep-lat]].
  * [13] M. Ablikim et al. [BESIII], “First Experimental Study of the Purely Leptonic Decay $D_{s}^{*+}\to e^{+}\nu_{e}$,” [arXiv:2304.12159 [hep-ex]].
  * [14] E. J. Eichten and C. Quigg, “Mesons with Beauty and Charm: New Horizons in Spectroscopy,” Phys. Rev. D 99, 054025 (2019) [arXiv:1902.09735 [hep-ph]].
  * [15] M. Karliner and J. L. Rosner,“Baryons with two heavy quarks: Masses, production, decays, and detection,” Phys. Rev. D 90, 094007 (2014) [arXiv:1408.5877 [hep-ph]].
  * [16] G. Aad et al. [ATLAS], “Observation of an Excited $B_{c}^{\pm}$ Meson State with the ATLAS Detector,” Phys. Rev. Lett. 113, 212004 (2014) [arXiv:1407.1032 [hep-ex]].
  * [17] A. M. Sirunyan et al. [CMS], “Observation of Two Excited B${}^{+}_{\mathrm{c}}$ States and Measurement of the B${}^{+}_{\mathrm{c}}$(2S) Mass in pp Collisions at $\sqrt{s}=$ 13 TeV,” Phys. Rev. Lett. 122, 132001 (2019) [arXiv:1902.00571 [hep-ex]].
  * [18] R. Aaij et al. [LHCb], “Observation of an excited $B_{c}^{+}$ state,” Phys. Rev. Lett. 122, 232001 (2019) [arXiv:1904.00081 [hep-ex]].
  * [19] B. Grinstein and J. Martin Camalich, “Weak Decays of Excited B Mesons,” Phys. Rev. Lett. 116, 141801 (2016) [arXiv:1509.05049 [hep-ph]].