# Detecting and Mitigating Hallucinations in Machine Translation:
Model Internal Workings Alone Do Well, Sentence Similarity Even Better

David Dale Elena Voita Loïc Barrault Marta R. Costa-jussà
Meta AI
<EMAIL_ADDRESS>

###### Abstract

While the problem of hallucinations in neural machine translation has long
been recognized, so far the progress on its alleviation is very little.
Indeed, recently it turned out that without artificially encouraging models to
hallucinate, previously existing methods fall short and even the standard
sequence log-probability is more informative. It means that characteristics
internal to the model can give much more information than we expect, and
before using external models and measures, we first need to ask: how far can
we go if we use nothing but the translation model itself? We propose to use a
method that evaluates the percentage of the source contribution to a generated
translation. Intuitively, hallucinations are translations “detached” from the
source, hence they can be identified by low source contribution. This method
improves detection accuracy for the most severe hallucinations by a factor of
2 and is able to alleviate hallucinations at test time on par with the
previous best approach that relies on external models. Next, if we move away
from internal model characteristics and allow external tools, we show that
using sentence similarity from cross-lingual embeddings further improves these
results. We release an efficient implementation for the considered
methods.111https://github.com/facebookresearch/stopes

##  1 Introduction

Hallucinations in machine translation (MT) are cases when the model generates
output that is partially or fully unrelated to the source sentence. While
generally this phenomenon is not that frequent and has a relatively low impact
on corpus-level automatic metrics, the impact of hallucinations on user
experience can be rather dramatic. For example, if a machine translation
system generates The staff were very friendly and helpful in response to an
input sentence about e.g. a marvelous view from the window, in future a user
is unlikely to trust this system.

While the problem of hallucinations is known and important, addressing it is
very challenging. Firstly, hallucinations are very rare. This is why previous
work mostly resorted to settings where models are encouraged to hallucinate,
e.g. artificially perturbing source sentence Lee et al. (2019); Raunak et al.
(2021), adding specific types of noise to the training data Raunak et al.
(2021), working under domain shift Wang and Sennrich (2020); Müller et al.
(2020), among others Zhou et al. (2021). Secondly, hallucinations are hard to
identify with automatic metrics. For the most part, hallucinations were
defined as translations with low quality according to some metric such as e.g.
adjusted BLEU or chrF Lee et al. (2019); Raunak et al. (2021); Müller and
Sennrich (2021) or translations satisfying some heuristic condition Berard et
al. (2019); Raunak et al. (2021). Overall, it was not clear whether proposed
methods indeed detect hallucinations and, if so, whether they transfer to more
natural settings.

Recently, when revisiting previous work in a relatively clean setting,
Guerreiro et al. (2022) found that existing methods fall short and the
standard sequence log-probability is the most informative. To show this, the
authors gathered a large dataset with professional annotations of translations
that, according to 10 previously proposed methods, are likely to be
hallucinations. This data (hallucinations along with the model that generated
them) made it possible to first, evaluate the performance of various detection
methods and second, to work on alleviating hallucinations at test time. For
the latter, the idea is “detect-then-rewrite”: after flagging a translation as
likely to be pathological, generate several alternative hypotheses and pick
the best one relying on some measure. So far, the best realization of this
general framework uses sequence log-probability – Seq-Logprob – for detection,
Monte Carlo dropout Gal and Ghahramani (2016) to generate several alternative
translation hypotheses, and COMET-QE to pick the final candidate (see
Guerreiro et al. (2022) for more details). In this work, we use the same test
bed and substantially improve the previous results.

Regarding hallucination detection, we view the observation that Seq-Logprob
outperforms previous (specifically targeted to hallucinations) methods as
follows: internal model characteristics may contain much more information than
we expect. Therefore, before developing or using external models and measures,
we ask: how far can we go if we use nothing but the translation model itself?
We propose to use a method that evaluates the percentage of the source
contribution to a generated translation. Intuitively, since hallucinations are
translations that are “detached” from the source (by definition), low source
contribution should be able to identify hallucinations. Despite the fact that
understanding hallucinations was one of the motivations behind the first
method evaluating relative source and target contributions Voita et al.
(2021), both existing methods only looked at highly artificial hallucinations
Voita et al. (2021); Ferrando et al. (2022). We propose to use ALTI+ by
Ferrando et al. (2022), the method that aggregates layer-wise tokens
attributions, for both hallucination detection and reranking in the “detect-
then-rewrite” framework. For detection of the most severe hallucinations, it
is twice more accurate than sequence log-probability. For reranking, it
performs on par with the previous best COMET-QE. All in all, we show that we
can improve the overall pipeline results by solely relying on internal model
characteristics.

When allowing external tools, previous work mostly focused on different ways
to automatically evaluate quality of a translation example, either with
string-based methods or neural quality estimation systems. This idea (the
better we estimate translation quality, the better we are at detecting
hallucinations) is natural: hallucinations are low-quality translations in the
first place. However, implementing this idea in practice is challenging: even
state-of-the-art quality estimation system substantially fails Guerreiro et
al. (2022). We hypothesize that instead of targeting quality evaluation, it
might be beneficial to use models trained with a rather different objective.
Indeed, as we show, similarity between the source and a translation estimated
via cross-lingual sentence embeddings outperforms the best internal method.
Apart from cross-lingual sentence similarity (which is expected to be
sensitive to highly incorrect translations), we find that cross-lingual
natural language inference models (less anticipated in the context of machine
translation) also perform quite well. To the best of our knowledge, we are the
first to apply these models for hallucination detection.

Overall, we show that:

  * •

by using only the model’s inner workings, we

    * $\circ$

detect the most severe type of hallucinations with twice better precision;

    * $\circ$

overwrite hallucinations at test time with results on par with the best
previous method that relies on an external model;

  * •

models focused on semantic similarity of sentences can detect all types of
hallucinations with $80\%$ better precision than the previous methods.

##  2 Background and Setting

In this section, we describe the framework and data we use for evaluation of
hallucination detection and mitigation methods. This framework was proposed by
Guerreiro et al. (2022) and consists of a large dataset of annotated
translations along with the model that produced them. To the best of our
knowledge, this is the only released data that can be used to analyze
hallucinations in a “clean” setting.

###  2.1 Model

The model is Transformer base (Vaswani et al., 2017) from fairseq Ott et al.
(2019) with the standard hyperparameters setting. It was trained on the WMT’18
German-English news translation data excluding Paracrawl (Bojar et al., 2018)
– totalling 5.8M sentence pairs. Since Guerreiro et al. (2022) used randomly
chosen 1/3 of the dataset as a held-out set for analysis, the model was
trained on the remaining 2/3 of the dataset. We use the model released by
Guerreiro et al. (2022): this is the model that generated hallucinations we
analyze.

###  2.2 Hallucination Dataset

Figure 1: Taxonomy of translation types (based on the dataset by Guerreiro et
al. (2022)).

Hallucination dataset released by Guerreiro et al. (2022) contains fine-
grained manual annotations of 3415 German-to-English translations generated by
the model above. These translations are chosen from a set of 1.8M translations
of held-out data as the ones that are likely to be pathological. The criteria
used to flag the translations include 10 methods ranging from previously
proposed heuristics Lee et al. (2019); Berard et al. (2019); Raunak et al.
(2021) to quality estimation models Rei et al. (2020b) and uncertainty
detectors Fomicheva et al. (2020); Zerva et al. (2021); Guerreiro et al.
(2022).

The taxonomy of translation pathologies in the dataset is shown in Figure 1.
Here, hallucinations are defined as severe translation errors that are
detached from the source. These can be either oscillatory (i.e. contain
erroneous repetitions of words and phrases) or largely fluent. The latter is
also split by severity of an error into fully detached (the whole content is
not supported by the source) and strongly, but not fully, detached
(significant proportion of output is not supported by the source).222Guerreiro
et al. (2022) mention that oscillatory hallucinations can also be either fully
or strongly detached, but they do not divide this category into smaller groups
because the overall number of such translations is rather small. Other than
hallucinations, the annotated data also contains translation errors that are
deemed to be not detached from the source (see Figure 1). Overall, 323
examples are judged to be hallucinations, 1044 as less severe translation
errors and the rest as correct translations.

Note that so far, there is no “canonical” hallucination taxonomy and previous
work used various, mostly overlapping, definitions (Lee et al., 2019; Raunak
et al., 2021; Zhou et al., 2021; Ji et al., 2022; Raunak et al., 2022;
Guerreiro et al., 2022). We follow the taxonomy by Guerreiro et al. (2022) for
two reasons. Firstly, for consistency with the dataset and the evaluation
framework we use. Secondly, this taxonomy is rather general, considers some
hallucination types overlooked in previous work (e.g. strongly detached
hallucinations), and was shown to be reasonable: under these definitions,
properties of hallucinations differ from those of translation errors Guerreiro
et al. (2022).

##  3 Hallucination Detection Methods

Generally, methods for handling hallucinations can be either internal, i.e.
using only information coming from the translation model itself, or external,
i.e. using auxiliary models. In addition to these, we also consider “oracles”
relying on reference translation. Note that these cannot be used in preventive
settings when references are not available; here we use them only for
analysis.

###  3.1 Reference-Based Oracles

Following previous work Müller and Sennrich (2021); Guerreiro et al. (2022),
we use:

  * •

chrF: character $n$-gram F score of the translation with respect to reference.
We use the chrF++ version that also takes into account word unigrams and
bigrams Popović (2017);

  * •

COMET: a neural quality estimation metric by Rei et al. (2020a) which was
shown to be the state-of-the-art reference-based method (Kocmi et al., 2021).

###  3.2 Internal Measures

##### Baseline: Seq-Logprob.

This is the standard length-normalized sequence log-probability. Compared to
previously introduced methods specifically targeting hallucinations, this
simple metric performs the best Guerreiro et al. (2022).

##### We use ALTI: percentage of source contribution.

As we already mentioned above, we hypothesize that the percentage of source
impact on a generated translation might be a strong signal for identifying
hallucinations. To evaluate this relative source contribution, we use recently
introduced ALTI+ Ferrando et al. (2022). At a high level, it decomposes each
transformer block into a sum of functions of individual tokens and views an
output representation as a summation of transformed input vectors. Then it
evaluates contribution of these vectors to the resulting sum. Among other
interesting observations, ALTI+ (as well as an earlier LRP-based method by
Voita et al. (2021)) was used to show that for artificially created
hallucinations, source influence is much lower than for “healthy” correct
translations. Our work is the first to test this intuition in a real setting
where hallucinations are generated naturally.333Note that of the two methods
that can evaluate relative source and target contributions we choose ALTI+ by
Ferrando et al. (2022) over LRP-based method by Voita et al. (2021) because
the latter is more computationally expensive.

Formally, for a model and its generated translation, we compute the total
source contribution as the sum of contributions of all source tokens. We do it
for each target token individually and then average across target tokens. The
scores are computed by the same model that produced the translations (Section
2.1).

###  3.3 External models

##### Baseline: COMET-QE.

For a reference-free model, we use the state-of-the-art COMET-QE (Rei et al.,
2020b) for its superior performance compared to other quality estimators
(Mathur et al., 2020; Freitag et al., 2021; Kocmi et al., 2021).

##### We use: sentence similarity.

Overall, we consider three measures based on pretrained models that evaluate
semantic similarity of two sentences:

  * •

LASER: cosine similarity of source and translation sentence embeddings from
LASER2 Heffernan et al. (2022). LASER2 improves the encoder-decoder LASER
Artetxe and Schwenk (2019) by replacing LSTM encoder with a Transformer and
using teacher-student training;

  * •

LaBSE: cosine similarity of source and translation sentence embeddings from
LaBSE Feng et al. (2022). LaBSE is a dual-encoder approach based on pretrained
transformers and fine-tuned for translation ranking with an additive margin
softmax loss;

  * •

XNLI: product of the entailment probabilities of source to translation and
translation to source. We compute entailment scores with RoBERTa Conneau et
al. (2020) fine-tuned on a combination of NLI data in 15 languages Conneau et
al. (2018).444https://huggingface.co/joeddav/xlm-roberta-large-xnli

Figure 2: Kernel density estimation of the distribution of the detection
criteria by translation pathology type.

##  4 Detection Experiments

###  4.1 Main results

| All hall. | Fully detached
---|---|---
Metric | AUC | P@R90 | AUC | P@R90
ChrF | 75.4 | 14.4 | 89.6 | 16.6
COMET | 83.4 | 19.2 | 87.7 | 12.6
Seq-Logprob | 83.0 | 13.9 | 93.5 | 31.0
ALTI | 84.9 | 12.5 | 98.7 | 67.4
COMET-QE | 70.2 | 14.2 | 66.1 | 06.0
LASER | 79.4 | 14.4 | 91.2 | 20.8
LaBSE | 91.7 | 25.9 | 98.5 | 70.3
XNLI | 90.9 | 24.1 | 98.7 | 60.4
Table 1: Hallucination detection quality. Methods: oracle, internal, external.
Changes in scores are highlighted compared to Seq-Logprob.

Overall results are shown in Table 1. We report ROC AUC and precision at
$90\%$ recall.555This is different from Guerreiro et al. (2022) who compare
recall at thresholds cutting off a specific percentage of the dataset.
Instead, we rely on two metrics: ROC AUC that does not rely on specific
thresholds and PR@R90 that covers a specific percentage of the hallucinations
(in this case, 90%) and then reports the resulting precision. We show metrics
for all hallucinations and fully detached hallucinations separately because
the latter are the most disastrous translation mistakes that are potentially
the easiest to detect.

First, let us look at internal methods. We see that while ALTI performs
comparably to Seq-Logprob for all hallucinations, for fully detached
hallucinations it has twice better precision than Seq-Logprob. A possible root
of this discrepancy is that ALTI averages the source contribution over all
generated tokens. Therefore, it is more efficient for detecting translations
where all or most of the generated tokens are hallucinated. Note also that for
fully detached hallucinations, internal ALTI performs almost on par with the
best external methods.

Among external methods, LaBSE and XNLI substantially outperform previous best
detector: for both all and fully detached hallucinations, their precision at
$90\%$ recall is roughly twice better than that of Seq-Logprob. While such a
good performance might be expected for LaBSE that evaluates cross-lingual
sentence similarity (in a way, this might be seen as a measure of translation
quality), results for XNLI are rather surprising: to the best of our
knowledge, models optimized for XNLI have not been used in the context of
machine translation. This suggests that looking at broader class of models and
training objectives might be beneficial.

Note also the large difference between LaBSE and LASER: while the former shows
big improvements compared to Seq-Lobprob, the latter noticeably lags behind.
This is not surprising when looking at training objectives of the underlying
models. In LASER2, the cross-lingual part of the objective uses cosine
similarity between sentence encodings. Differently, LaBSE is trained as a
translation ranking task and thus encourages ordering of translations by
severity of an error more explicitly.

To further understand differences between detectors, we look at the
distributions of the detection scores in Section 4.2 and the detected
pathology types in Section 4.3.

###  4.2 Analysing Distributions of the Scores

For each of the methods, Figure 2 shows distributions of the scores for fully
detached hallucinations, strongly detached hallucinations, less severe errors
and correct translations.

##### Internal methods: partial hallucinations are bimodal.

ALTI and Seq-Logprob show similar behavior: errors are distributed similarly
to correct translations, and the scores for partial (strongly detached)
hallucinations have bimodal distribution. At a high level, for the model, some
partial hallucinations “look” more like full hallucinations, and some – more
like errors. This observation can motivate future work: it would be
interesting to understand which types of these hallucinations behave one way
or another. Specifically, whether it depends on detachment or on more simple
patterns such as e.g. the proportion of hallucinated tokens.

##### COMETs: blind to error severity.

We see that COMET and COMET-QE scores do not provide separation between
hallucinations and less severe errors. This agrees with previous work noting
that since quality estimation models are mostly trained on data that lacks
negative examples, COMETs may be inadequate at evaluating poor translations in
general Takahashi et al. (2021); Sudoh et al. (2021) and hallucinations in
particular Guerreiro et al. (2022). What is also expected, is that compared to
reference-free COMET-QE, the overlap between the scores for correct and
incorrect translations is much lower for reference-based COMET. ChrF exhibits
a behavior similar to COMET.

##### LaBSE: ranks hallucination severity best.

LaBSE is the only detector with a clear order between full, partial
hallucinations, and translations without hallucinations. As we mentioned
before, this is expected because only LaBSE is trained to rank translations.
Interestingly, for LASER, modes for the three distributions are also ordered;
unfortunately, the distributions themselves overlap significantly which makes
LASER not a good hallucination detector. Both LaBSE and LASER ignore most of
the non-hallucinated translation errors.

##### XNLI: no middle ground.

Finally, we see that XNLI distributions are very peaky and concentrated around
0 and 1. This is expected: XNLI’s decision is always binary. While this
provides good separation between fully detached hallucinations and correct
translations, it is hard to estimate severity of an error.

Figure 3:  Distribution of translation types when selecting the worst 10% of
the dataset according to each metric. In the original dataset, these types
were annotated in a multilabel manner (e.g. the same translation could be
annotated both as oscillatory hallucination and as a named entity error). To
assign a single label to each translation, we choose the most severe pathology
type (with severity increasing clockwise from “Correct translations” to “Fully
detached”).

###  4.3 Detected Pathology Types

Now we come to more fine-grained categories and look at detected pathology
types. For each method, we say that a translation is “detected” if it is
contained in a fraction (e.g. $10\%$) of the hallucination dataset
corresponding to the lowest scores.666Note that we take such a large
percentage because in the hallucination dataset we use, about $10\%$ of
translations are hallucinations and about $30\%$ more are errors. Then we look
at

  * •

the distribution of pathology types contained among detected examples (Figure
3);

  * •

recall for different translation types with respect to the whole dataset
(Figure 4).

##### The three best methods are similar.

Figure 3 shows that ALTI, LaBSE and XNLI select similar pathology types. For
them, flagged examples consist mostly from fully detached and strongly
detached hallucinations, along with other errors.

##### LASER is an outlier.

LASER behaves differently and instead of focusing on pathological
translations, it flags correct translations more. This explains its poor
performance on detection mentioned before.

##### XNLI flags undergenerations.

Figure 4 shows that XNLI (and, to a lesser extent, LaBSE) flags a large
proportion of undertranslations. This makes sense: these criteria are
symmetric, and if we swap the source and the undergenerated translation, the
longer source can be seen as a hallucination.

##### Fully detached are the easiest to detect.

As expected, fully detached hallucinations are the easiest to detect: all
methods detect them entirely when taking $20\%$ of the hallucination dataset
(Figure 4), and they are the most frequent pathology type among the examples
flagged by the best performing methods (Figure 3). Overall, our findings
confirm the conclusions of Guerreiro et al. (2022) that oscillatory and
strongly detached hallucinations are more difficult to detect, and
improvements with our methods mostly come from these types of hallucinations.

Figure 4: Recalls by translation types when selecting the worst 20% of the
dataset according to each metric. Here, the types are presented in a
multilabel manner, i.e. one translation may contribute to multiple axes.
Figure 5: For all combinations of a generation strategy and a reranker,
heatmaps show scores for the final translations.

##  5 Mitigating Hallucinations at Test Time

Finally, let us come to the second part of the “detect-then-rewrite” pipeline:
after flagging a translation as likely to be hallucinated, we want to generate
several alternative translations and select one of them based on some
criterion i.e., by reranking the hypotheses Guerreiro et al. (2022). This
general framework has two degrees of freedom: (i) generation of hypotheses,
and (ii) reranking approach. We show that

  * •

for generating hypotheses, simply applying MC dropout (as done in Guerreiro et
al. (2022)) outperforms more involved methods such as diverse beam search
(Section 5.2);

  * •

for reranking, we can match COMET-QE with internal ALTI and improve the
hallucination rate by using LaBSE (Section 5.3).

###  5.1 Evaluation methodology

In this section, we explain the setup for the experiments with automatic
evaluation in Sections 5.2 and 5.3. The setup for manual annotation is
explained later in Section 5.3.2.

##### Metrics.

In our experiments, we use several metrics. First, we use quality evaluation
metrics commonly used by the community, i.e. COMET Rei et al. (2020b) and
BLEU. Additionally, we use the two best metrics for hallucination detection:
LaBSE and XNLI. We show some of the metrics in the main text and the rest in
the appendix.

##### Data.

First, we analyze the impact of our method on translations of different
quality levels. For this, we randomly sample 150 sentences from each of the
following groups of the hallucination dataset (Section 2.2): fully detached
hallucinations, strongly detached hallucinations, all other translation
pathologies, and correct translations. We apply all versions of the
hallucination mitigation algorithm to these 600 sentences.

Note that in a practical application, we would apply the mitigation techniques
only to the translations labeled by a detection algorithm as potential
hallucination. We simulate this later in Section 5.3.2 when performing manual
annotation.

###  5.2 Generation Strategies

To generate alternative hypotheses, Guerreiro et al. (2022) use Monte Carlo
dropout Gal and Ghahramani (2016). This means they leave standard beam search
inference intact and achieve variability in translations via activating model
dropout at inference. A natural question is whether using other generation
strategies can give better results. For example, if we use e.g. beam search
specifically designed to produce diverse translations, can we get better
hypotheses?

To test this, we use the following methods:

  * •

default: standard decoding without reranking, i.e. beam search with size 5,
where we pick only the top 1 candidate;

  * •

beam search: beam search with size $n$;

  * •

sampling from the predicted distribution:

    * $\circ$

sampling: from the whole distribution;

    * $\circ$

sampling p=80: from the top $p=80\%$ of the distribution, i.e. nucleus
sampling Holtzman et al. (2020);

  * •

diverse beam search:

    * $\circ$

dbs_n: method by Vijayakumar et al. (2016) with beam widths $s=1,\ 3,\ 10$;

    * $\circ$

d_dec_r: diverse decoding with diversity rates $r=1,\ 3,\ 10$ Li et al.
(2016);

  * •

Monte Carlo dropout:

    * $\circ$

mc greedy: $n$ iterations of greedy search with dropout;

    * $\circ$

mc beam: the method used in Guerreiro et al. (2022), i.e. $n$ iterations of
beam search with dropout, each with size $10$.

Unless stated otherwise, $n=10$ in all experiments.

####  5.2.1 The Impact of Generation Strategy

The results are shown in Figure 5. To disentangle the effect of generation
strategy from the subsequent reranker performance, we show the results for all
combinations. As rerankers, we considered COMET-QE used in Guerreiro et al.
(2022) and the methods proposed in Section 3.

We see that the mc beam method clearly outperforms all the other. This is
interesting for two reasons. First, MC dropout is easy to use: one has to
apply standard inference with dropout on without other changes to the
implementation. Next, differently from modifying decoding strategies, here
variability in hypotheses comes from model predictive uncertainty Gal and
Ghahramani (2016); Zerva et al. (2021); Guerreiro et al. (2022). This is one
more evidence that understanding model inner characteristics can be beneficial
in various settings.

Based on these results, in what follows we generate hypotheses with beam
search with MC dropout.

####  5.2.2 The Impact of Number of Hypotheses

We also check whether generating more than 10 hypotheses can improve the
overall results. Figure 6 shows the final COMET scores depending on the number
of hypotheses. We see that the scores increase with more hypotheses and do not
saturate at 10. This implies that in cases when the quality of a translation
is much more important than its computational cost, one can potentially
improve the quality by generating more candidate hypotheses.

Figure 6: COMET scores for each generation method and number of hypotheses.
For each group of generation strategies, we show the best representative.

###  5.3 Reranking Approaches

Apart from detecting hallucinations, the methods we propose can be applied as
rerankers in the “detect-than-rewrite” pipeline.

####  5.3.1 Automatic Evaluation

Figure 5 shows that, regardless of the generation method, LaBSE is the best
reranker and it performs notably better than the strong COMET-QE baseline.
Apart from the average results, Table 2 also shows COMET scores for each
pathology type. We can see that reranking with any method is better then no
reranking for all groups of original translations. Compared to the COMET-QE
baseline, LABSE improves the scores for hallucinations and correct
translations, but drops quality for other pathologies.

The only internal method ALTI performs better than COMET-QE for fully detached
hallucinations, but is inferior when looking at other translation types. It
means that ALTI is very sensitive to the most severe pathology, but not
capable to rank relatively good translations.

| Pathologies | Cor. | Avg.
---|---|---|---
Reranker | F. | S. | O. |  |
No reranking | -1.23 | -0.97 | -0.59 | 0.27 | -0.63
Baseline |  |  |  |  |
COMET-QE | -0.21 | -0.13 | -0.14 | 0.35 | -0.03
Ours |  |  |  |  |
ALTI | -0.17 | -0.24 | -0.39 | 0.25 | -0.14
LASER | -0.11 | -0.23 | -0.35 | 0.27 | -0.11
LaBSE | -0.07 | -0.12 | -0.26 | 0.39 | -0.01
XNLI | -0.12 | -0.18 | -0.28 | 0.30 | -0.07
Table 2: Average COMET scores after reranking MC dropout hypotheses by various
methods. Pathologies: fully detached hallucinations (F.), strongly detached
hallucinations (S.), other pathologies (O.). In the appendix, we also show
XNLI scores.

####  5.3.2 Manual annotation

##### Data.

To confirm the results of automatic evaluation, we perform manual annotation.
For each of the methods, we their translations of the same 200 source
sentences. These sentences were randomly sampled from the hallucination
dataset with the distribution of pathologies roughly mimicking outputs of the
best detectors (Figure 3). Overall, for 55$\%$ of the sentences their original
translations are labeled as hallucinations, 25$\%$ as errors and 20$\%$ as
correct translations.777We did not select these sentences with any metric,
because it might affect the results of evaluating this metric as a reranker.
We leave comparison of all combinations of detectors and rerankers to the
future work.

We compare the original translations and three reranking methods: the baseline
COMET-QE used in Guerreiro et al. (2022), the best overall reranker LaBSE, and
the only internal method ALTI.

##### Annotation.

For each source sentence, the four translations were deduplicated and shuffled
to avoid annotator bias. The resulting sentence pairs were given to 3
annotators who labeled them with three categories: Correct, Error, and
Hallucination. The labels were aggregated by majority vote; in case of ties
(20 out of the 602 sentence pairs that were left after deduplication) we
pessimistically assumed a hallucination. Further details on the annotation
guidelines and inter-annotation agreement are reported in Appendix B. We
evaluate the statistical significance of the pairwise differences in the
proportions of correct and hallucinated translations using two-sided Student
test for two related samples with 5$\%$ confidence level.

##### Results.

Annotation results are shown in Figure 7. All reranking methods reduce
hallucinatory rate by a factor of 2.5-3. Interestingly, when looking at
hallucinations, internal ALTI performs on par with COMET-QE: the differences
between these two methods are not statistically significant. COMET-QE,
however, has less errors. This is expected: it was trained to distinguish
correct translations from errors. Coming to LaBSE, we find that it produces
slightly less hallucinations that other reranking methods, and more correct
translations than ALTI; these differences are significant at 5$\%$ confidence
level. Overall, by using sentence similarity from LaBSE, we improve both
halucinations detection and alleviating hallucinations at test time.

Note that since COMET-QE is the state-of-the-art quality estimator, it is a
very strong baseline for the reranking stage where the goal is to find a
better translation. The fact that we can match COMET-QE’s hallucinatory rate
reduction by analyzing model internal workings has value from different
perspectives. For research, it can motivate future work on model
understanding; for practitioners, it means that hallucination mitigation
methods are not limited to language pairs where external models such as COMET-
QE exist: model understanding might be enough.

Figure 7: Human annotation results: percentages of translation pathologies for
different reranking methods. For hallucinations, all the differences are
significant, except the one between ALTI vs COMET-QE. For correct
translations, the difference between LaBSE and ALTI is statistically
significant.

##  6 Conclusions

We started this work by asking how far we can go at detecting and mitigating
hallucinations if we use nothing but the translation model itself. Turns out,
we can improve the results of the overall “detect-then-rewrite” pipeline by
evaluating the percentage of source contribution to generated translation:
translations with low source contributions are likely to be “detached” from
the source, i.e. hallucinations. For detecting the most severe type of
hallucinations, this method improves previous results twice; for mitigating
hallucinations at test time, it matches the hallucination reduction rate of
the previous method based on COMET-QE. We believe this can motivate future
work on model understanding and help practitioners: using nothing but model
inner working can allow mitigating hallucinations for language pairs where
models such as COMET-QE are not available.

When allowing external models, we propose to expand the methods for handling
hallucinations from models specialized for quality estimation to a broader set
of objectives, e.g. sentence similarity from cross-lingual embeddings. Apart
from showing that e.g. LaBSE improves previous results significantly, we also
find that models so far overlooked in the context of machine translation (such
as natural language inference) can be beneficial. We hope future work will
build on this idea.

## References

  * Artetxe and Schwenk (2019) Mikel Artetxe and Holger Schwenk. 2019.  Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond.  _Transactions of the Association for Computational Linguistics_ , 7:597–610.
  * Berard et al. (2019) Alexandre Berard, Ioan Calapodescu, and Claude Roux. 2019.  Naver labs Europe’s systems for the WMT19 machine translation robustness task.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)_ , pages 526–532, Florence, Italy. Association for Computational Linguistics.
  * Bojar et al. (2018) Ondřej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Philipp Koehn, and Christof Monz. 2018.  Findings of the 2018 conference on machine translation (WMT18).  In _Proceedings of the Third Conference on Machine Translation: Shared Task Papers_ , pages 272–303, Belgium, Brussels. Association for Computational Linguistics.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451, Online. Association for Computational Linguistics.
  * Conneau et al. (2018) Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018.  Xnli: Evaluating cross-lingual sentence representations.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.
  * Feng et al. (2022) Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022\.  Language-agnostic BERT sentence embedding.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 878–891, Dublin, Ireland. Association for Computational Linguistics.
  * Ferrando et al. (2022) Javier Ferrando, Gerard I. Gállego, Belen Alastruey, Carlos Escolano, and Marta R. Costa-jussà. 2022.  Towards opening the black box of neural machine translation: Source and target interpretations of the transformer.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , Online and Abu-Dhabi, UAE. Association for Computational Linguistics.
  * Fomicheva et al. (2020) Marina Fomicheva, Lucia Specia, and Francisco Guzmán. 2020.  Multi-hypothesis machine translation evaluation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1218–1232, Online. Association for Computational Linguistics.
  * Freitag et al. (2021) Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George Foster, Alon Lavie, and Ondřej Bojar. 2021.  Results of the WMT21 metrics shared task: Evaluating metrics with expert-based human evaluations on TED and news domain.  In _Proceedings of the Sixth Conference on Machine Translation_ , pages 733–774, Online. Association for Computational Linguistics.
  * Gal and Ghahramani (2016) Yarin Gal and Zoubin Ghahramani. 2016.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _Proceedings of The 33rd International Conference on Machine Learning_ , volume 48 of _Proceedings of Machine Learning Research_ , pages 1050–1059, New York, New York, USA. PMLR.
  * Guerreiro et al. (2022) Nuno M. Guerreiro, Elena Voita, and André F. T. Martins. 2022.  Looking for a needle in a haystack: A comprehensive study of hallucinations in neural machine translation.
  * Heffernan et al. (2022) Kevin Heffernan, Onur Çelebi, and Holger Schwenk. 2022.  Bitext mining using distilled sentence representations for low-resource languages.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  In _International Conference on Learning Representations_.
  * Ji et al. (2022) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022.  Survey of hallucination in natural language generation.
  * Kocmi et al. (2021) Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita, and Arul Menezes. 2021.  To ship or not to ship: An extensive evaluation of automatic metrics for machine translation.
  * Lee et al. (2019) Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. 2019.  Hallucinations in neural machine translation.
  * Li et al. (2016) Jiwei Li, Will Monroe, and Dan Jurafsky. 2016.  A simple, fast diverse decoding algorithm for neural generation.  _arXiv preprint arXiv:1611.08562_.
  * Mathur et al. (2020) Nitika Mathur, Johnny Wei, Markus Freitag, Qingsong Ma, and Ondřej Bojar. 2020\.  Results of the WMT20 metrics shared task.  In _Proceedings of the Fifth Conference on Machine Translation_ , pages 688–725, Online. Association for Computational Linguistics.
  * Müller et al. (2020) Mathias Müller, Annette Rios, and Rico Sennrich. 2020.  Domain robustness in neural machine translation.  In _Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)_ , pages 151–164, Virtual. Association for Machine Translation in the Americas.
  * Müller and Sennrich (2021) Mathias Müller and Rico Sennrich. 2021.  Understanding the properties of minimum Bayes risk decoding in neural machine translation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 259–272, Online. Association for Computational Linguistics.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A fast, extensible toolkit for sequence modeling.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pages 48–53, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Popović (2017) Maja Popović. 2017.  chrF++: words helping character n-grams.  In _Proceedings of the Second Conference on Machine Translation_ , pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics.
  * Raunak et al. (2021) Vikas Raunak, Arul Menezes, and Marcin Junczys-Dowmunt. 2021.  The curious case of hallucinations in neural machine translation.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1172–1183, Online. Association for Computational Linguistics.
  * Raunak et al. (2022) Vikas Raunak, Matt Post, and Arul Menezes. 2022.  Salted: A framework for salient long-tail translation error detection.
  * Rei et al. (2020a) Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020a.  COMET: A neural framework for MT evaluation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2685–2702, Online. Association for Computational Linguistics.
  * Rei et al. (2020b) Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020b.  Unbabel’s participation in the WMT20 metrics shared task.  In _Proceedings of the Fifth Conference on Machine Translation_ , pages 911–920, Online. Association for Computational Linguistics.
  * Sudoh et al. (2021) Katsuhito Sudoh, Kosuke Takahashi, and Satoshi Nakamura. 2021.  Is this translation error critical?: Classification-based human and automatic machine translation evaluation focusing on critical errors.  In _Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval)_ , pages 46–55, Online. Association for Computational Linguistics.
  * Takahashi et al. (2021) Kosuke Takahashi, Yoichi Ishibashi, Katsuhito Sudoh, and Satoshi Nakamura. 2021\.  Multilingual machine translation evaluation metrics fine-tuned on pseudo-negative examples for wmt 2021 metrics task.  In _Proceedings of the Sixth Conference on Machine Translation_ , pages 1049–1052, Online. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc.
  * Vijayakumar et al. (2016) Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2016.  Diverse beam search: Decoding diverse solutions from neural sequence models.  _arXiv preprint arXiv:1610.02424_.
  * Voita et al. (2021) Elena Voita, Rico Sennrich, and Ivan Titov. 2021.  Analyzing the source and target contributions to predictions in neural machine translation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1126–1140, Online. Association for Computational Linguistics.
  * Wang and Sennrich (2020) Chaojun Wang and Rico Sennrich. 2020.  On exposure bias, hallucination and domain shift in neural machine translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3544–3552, Online. Association for Computational Linguistics.
  * Zerva et al. (2021) Chrysoula Zerva, Daan van Stigt, Ricardo Rei, Ana C Farinha, Pedro Ramos, José G. C. de Souza, Taisiya Glushkova, Miguel Vera, Fabio Kepler, and André F. T. Martins. 2021.  IST-unbabel 2021 submission for the quality estimation shared task.  In _Proceedings of the Sixth Conference on Machine Translation_ , pages 961–972, Online. Association for Computational Linguistics.
  * Zhou et al. (2021) Chunting Zhou, Graham Neubig, Jiatao Gu, Mona Diab, Francisco Guzmán, Luke Zettlemoyer, and Marjan Ghazvininejad. 2021.  Detecting hallucinated content in conditional neural sequence generation.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 1393–1404, Online. Association for Computational Linguistics.

##  Appendix A Mitigating Hallucinations at Test Time

Table 3 shows XNLI scores after reranking MC dropout hypotheses by various
methods. Note that since here XNLI was used both to rerank and well as
evaluate quality, in ths experiment XNLI can be viewed as an oracle.

| Pathologies | Correct | Avg.
---|---|---|---
Reranker | F. | S. | O. |  |
No reranking | 2 | 30 | 80 | 93 | 51
Baseline |  |  |  |  |
COMET-QE | 59 | 69 | 85 | 93 | 77
Ours |  |  |  |  |
ALTI | 64 | 73 | 92 | 91 | 80
LASER | 72 | 73 | 92 | 92 | 82
LaBSE | 74 | 80 | 92 | 94 | 85
XNLI (oracle) | 75 | 83 | 98 | 97 | 88
Table 3: Average XNLI scores after reranking MC dropout hypotheses by various
methods. Pathologies: fully detached hallucinations (F.), strongly detached
hallucinations (S.), other pathologies (O.).

##  Appendix B Manual Evaluation

In this appendix we describe the manual evaluation. First, we detail the
simple guidelines that were presented to manual annotators. Second, we report
the number of annotators and inter-annotation agreement. Third, we report the
results of statisitical sigificance tests for comparing all the methods.

##### Guidelines

Annotators were provided with the guidelines shown in Table 4. For the
reporting purposes, “Partial hallucination” was grouped together with “Full
hallucination”, and “Undertranslation” with “Other”.

Each row of the data consists of the German source sentence, its reference
English translation (it is not always accurate!), and 1 to 4 machine
translation outputs. The machine translation outputs are presented in a random
order, to exclude the possibility of bias toward any specific method. For each
of the machine translations, you need to assign one of the following labels: •
OK: An acceptable translation; it conveys the main meaning correctly and does
not introduce extra meaning. Some details still may differ, and minor errors
are acceptable. • Partial hallucination: a part of the translation is
unrelated to the source, or is related very indirectly, such as via a common
topic. • Full hallucination: most or all of the translation is unrelated to
the source, or is related very indirectly. • Undertranslation: there is no
hallucinations, but a significant part of the source is not translated at all.
• Other: there are no hallucinations or undertranlsations, but there are other
translation errors that make the translation unacceptable.
---
Table 4: Human annotations Guidelines

##### Inter-annotation agreement

We evaluated inter-annotation agreement by Fleiss’ Kappa. For the three
annotators and the three aggregated labels, it equals 0.57 on the 602 sentence
pairs that were labeled (with the 5 original labels, it is 0.55). This may be
interpreted as moderate agreement.

##### The differences

The Tables 5 and 6 compare proportions of correct and hallucinated
translations for each of the manually evaluated methods. The P-values are
computed with paired two-sided Student test (scipy.stats.ttest_rel).

Method 1 | Method 2 | Rate 1 | Rate 2 | P-value
---|---|---|---|---
LABSE | COMET-QE | 0.56 | 0.54 | 0.53
LABSE | ALTI | 0.56 | 0.49 | 0.02
LABSE | Default | 0.56 | 0.20 | 0.00
COMET-QE | ALTI | 0.54 | 0.49 | 0.12
COMET-QE | Default | 0.54 | 0.20 | 0.00
ALTI | Default | 0.49 | 0.20 | 0.00
Table 5: Comparison between manually annotated rates of correct translation. Method 1 | Method 2 | Rate 1 | Rate 2 | P-value
---|---|---|---|---
LABSE | COMET-QE | 0.16 | 0.22 | 0.01
LABSE | ALTI | 0.16 | 0.22 | 0.01
LABSE | Default | 0.16 | 0.53 | 0.00
COMET-QE | ALTI | 0.22 | 0.22 | 1.00
COMET-QE | Default | 0.22 | 0.53 | 0.00
ALTI | Default | 0.22 | 0.53 | 0.00
Table 6: Comparison between manually annotated rates of hallucinated
translation.


# Frequency-domain engineering of bright squeezed vacuum for continuous-
variable quantum information

Inbar Hurvitz  1,* Aviv Karnieli  2 and Ady Arie1  1School of Electrical
Engineering, Fleischman Faculty of Engineering, Tel Aviv University, Tel Aviv
69978, Israel
2Sackler School of Physics, Faculty of Exact Sciences, Tel Aviv University,
Tel Aviv 69978, Israel<EMAIL_ADDRESS>
††journal: oe††articletype: Research Article

Multimode bright squeezed vacuum is a non-classical state of light hosting a
macroscopic photon number while offering promising capacity for encoding
quantum information in its spectral degree of freedom. Here, we employ an
accurate model for parametric down-conversion in the high-gain regime and use
nonlinear holography to design quantum correlations of bright squeezed vacuum
in the frequency domain. We propose the design of quantum correlations over
two-dimensional lattice geometries that are all-optically controlled, paving
the way toward continuous-variable cluster state generation on an ultrafast
timescale. Specifically, we investigate the generation of a square cluster
state in the frequency domain and calculate its covariance matrix and the
quantum nullifier uncertainties, that exhibit squeezing below the vacuum noise
level.

##  1 Introduction

The spectral degree of freedom of photons has become attractive thanks to its
potentially unlimited Hilbert space dimension and to its large bandwidth.
These are appealing traits for quantum information protocols [1, 2, 3], that
were also shown useful, for example, in quantum key distribution [4] and
quantum sensing [1]. The advent of quantum optical frequency combs [5] allowed
the generation and manipulation of high-dimensional photonic qudits [6, 7, 8],
and even the formation of frequency-time bin cluster states [9, 10]. In
particular, shaping the spectrum of photon pairs generated via spontaneous
parametric down-conversion (SPDC) by second-order nonlinearity in crystals
[11, 12] and waveguides [13], third-order nonlinearity in spontaneous four-
wave mixing in fibers and resonators [14] or in hot atomic vapors [15] has
drawn tremendous attention in the past few decades. Here we focus on the first
method, whereby the progress in engineered nonlinear photonic crystals (NLPC)
[16] allows spectral-temporal entanglement [17, 18].

In parallel to the advances of such discrete-variable quantum optical schemes
in the frequency domain, there was also progress in continuous-variable (CV)
schemes employing the spectral degree of freedom. The main resource for
frequency-domain CV quantum light is the multimode squeezed vacuum [19],
demonstrating quantum correlations on the one hand, while containing a large
number of spectral modes allowing high quantum information capacity [19], and
even the realization of CV cluster states on a frequency comb [20, 8]. Bright
squeezed vacuum (BSV), a squeezed vacuum state containing a macroscopic number
of photons, can be generated in strongly-pumped NLPCs, pushing SPDC to the
high-gain regime. Tailoring the spectrum of BSV is important for several
applications such as CV cluster state generation [20], broadband homodyne
detection [3] and supersensitive phase measurements [21]. While nonlinear
holography techniques together with pump shaping are well-established methods
for engineering SPDC photons, their full potential has not yet been deployed
for the frequency-domain engineering of BSV. To date, the most prevalent
schemes for spectral shaping of BSV use either SU(1,1) interferometers [19],
or employ the frequency comb of an optical parametric oscillator (OPO) [22,
23] together with spectrally-shaped pumps.

In this paper, we analyze a new scheme for frequency-domain engineering of BSV
using a combination of nonlinear holograms and shaped pump spectra. Our model
extends recent proposal by Drago [24] to regime of high nonlinear gain. We
validate our model against known results of SPDC shaping using engineered
NLPCs extended to the high-gain regime, and then propose novel designs of
high-dimensional frequency bin correlations with square lattice geometry. This
capability allows for the generation of CV cluster states in the frequency
domain, which are important for measurement-based quantum computing (MBQC) and
quantum networks [25, 26], without the necessity of an OPO frequency comb.
Using our holographic technique, we propose an alternative method to generate
such states by simultaneously using nonlinear holograms and shaped pump
spectra, which provide additional degrees of freedom. For example, we show how
the quantum correlations can be all-optically controlled on an ultrafast
timescale using shaped pump pulses interacting with the same nonlinear
holograms. Finally, we analyze CV cluster states for a simple class of designs
and demonstrate their expected quadrature correlations and squeezing.

##  2 Theoretical model

Several accurate models exist for the description of multimode squeezed light
generation, for example, phase-space methods investigating the effects of
ultrashort pump spectra [27] or predicting spatial correlations [28]. More
recent theoretical descriptions of BSV states and their correlations were
introduced [29], predicting dynamics, broadening of angular distribution, and
Schmidt ranks as the squeezing parameter is increased. Here, we extend these
models to incorporate the combined effects of high-gain, shaped pump spectra
and modulated NLPCs (see Fig.1), while employing a simulation scheme based on
a recently developed and experimentally-validated numerical simulation tool
for SPDC [30, 31]. We focus on the case of a single spatial mode with no
diffraction, for a type-II SPDC process in a $\mathrm{KTiOPO_{4}}$ (KTP)
engineered NLPC, where the idler ($i$) and signal ($s$) fields differ by
polarization. While this is an appropriate model for plane-wave pumps in on-
axis phase-matched NLPCs, it can also be applied to nonlinear waveguides.

Figure 1: Illustration of multimode squeezed light generation in the high gain
regime. (a) An intense pump pulse with a shaped spectrum enters a modulated
NLPC hosting a spectral hologram. Each photon in the pump beam (of frequency
$\omega_{p}$) down converts to two photons (with frequencies
$\omega_{s}$,$\omega_{i}$) under conservation of energy and momentum,
summarized in (b). Panel (c) shows Wigner functions of a vacuum state (left)
and a squeezed vacuum state (right). For Gaussian states such as squeezed
vacuum, the moments of the Wigner function in phase space are all prescribed
by first-order correlations of the form of Eq.4 and Eq.10.

We begin by considering the quantum Maxwell’s equations [32] for the
frequency-dependent field operators [31], under the undepleted pump and
slowly-varying envelope approximations,

$\begin{split}i\frac{\partial}{\partial z}\left(\begin{matrix}\hat{A}_{i}\\\
\hat{A}_{s}\end{matrix}\right)&(z,\omega)=-\frac{\omega^{2}\chi^{(2)}\mathcal{E}_{p}}{c^{2}k(\omega)}d_{\mathrm{NL}}(z)\int
d\omega^{\prime}\mathrm{e}^{i\Delta
k(\omega,\omega^{\prime})z}\mathcal{P}(\omega+\omega^{\prime})\left(\begin{matrix}\hat{A}_{s}^{\dagger}\\\
\hat{A}_{i}^{\dagger}\end{matrix}\right)(z,\omega^{\prime}),\end{split}$ (1)

where $\hat{A}_{i},\hat{A}_{s}$ are, respectively, the idler and signal field
slowly-varying-envelope operators [32], $z$ is the propagation coordinate,
$\chi^{(2)}$ is the value of the quadratic nonlinear coefficient,
$d_{\mathrm{NL}}(z)$ is the modulation of the NLPC, $\mathcal{E}_{p}$ is the
pump electric field, $\mathcal{P}(\omega)$ is the normalized pump spectrum,
$c$ is the speed of light, $k(\omega)$ is the frequency-dependent wavenumber
of the medium, and $\Delta
k=k_{p}(\omega+\omega^{\prime})-k_{i}(\omega)-k_{s}(\omega^{\prime})$ is the
phase mismatch parameter.

We wish to study the quantum correlations of the generated multimode BSV in
the frequency domain. It is well known that squeezed vacuum is a zero-mean
Gaussian state [33], which could be described in phase space by a Wigner
function of the form [33]:

$W(\vec{\alpha})=\frac{1}{\pi^{N}}\frac{1}{\det\mathbf{\sigma}}\exp{(-\vec{\alpha}^{\dagger}\sigma^{-1}\vec{\alpha})}$
(2)

where $N$ is the number of modes,
$\vec{\alpha}=(\alpha_{1},\alpha_{2},...,\alpha_{N},\alpha_{1}^{*},\alpha_{2}^{*},...,\alpha_{N}^{*})$
where $\alpha_{j}$ are complex numbers, and the covariance matrix $\sigma$, of
size $2N$ by $2N$, is defined by

$\sigma_{mn}=\braket{\xi_{m}\xi_{n}^{\dagger}+\xi_{m}^{\dagger}\xi_{n}}.$ (3)

where
$\vec{\xi}=(a_{1},a_{2},...,a_{N},a_{1}^{\dagger},a_{2}^{\dagger},...,a_{N}^{\dagger})$
is a vector operator of length $2N$ containing all annihilation and creation
operators. Note that $\sigma_{mn}$ may contain all terms of the form
$\braket{a_{m}a_{n}},\braket{a_{m}^{\dagger}a_{n}}$ and
$\braket{a_{m}^{\dagger}a_{n}^{\dagger}}$. Eqs.2 and 3 imply that first-order
quantum correlations (i.e., those that are quadratic in the field operators)
hold all the information on the quantum state. Therefore, in the following, we
consider the first-order quantum correlations of BSV, and derive all relevant
observables from them.

To calculate the first-order quantum correlations, we begin by observing that
the initial state is vacuum ($\ket{\psi}=\ket{0}$) for which the first-order
correlation function can be written in terms of the field operators (Eq.1)
propagated from $z=0$ to $z=L$ (where $L$ is the crystal’s length) as

$\begin{split}G_{\alpha\beta}^{(1)}(\omega,\omega^{\prime})=\bra{0}\hat{A}_{\alpha}^{\dagger}(L,\omega)\hat{A}_{\beta}(L,\omega^{\prime})\ket{0},\end{split}$
(4)

where $\alpha,\beta=i,s$. We insert the resolution of identity in the Fock
basis $I=\int
d\Omega\sum_{n,\gamma}\ket{{n_{\Omega,\gamma}}}\bra{{n_{\Omega,\gamma}}}$ into
Eq.4, where $\gamma=i,s$. As the dynamics is linear and mixes creation and
annihilation operators, the field operator can only couple the vacuum state to
a single photon state $\ket{{1_{\Omega,\gamma}}}$. Namely, the transition
amplitudes of the form
$\braket{{n_{\Omega,\gamma}}}{\hat{A}_{\alpha}(L,\omega)}{0}$ are all zero
except for $n=1$. Following Refs [30, 31, 34, 35, 36], we write the simplified
first-order correlation function in terms of these single-photon amplitudes as

$\begin{split}G_{\alpha\beta}^{(1)}(\omega,\omega^{\prime})=\int
d\Omega\sum_{\gamma}\bra{0}&\hat{A}_{\alpha}^{\dagger}(L,\omega)\ket{{1_{\Omega,\gamma}}}\bra{{1_{\Omega,\gamma}}}\hat{A}_{\beta}(L,\omega^{\prime})\ket{0}.\end{split}$
(5)

Out of the eight possible combinations for single photon amplitudes
($\alpha,\beta,\gamma=i,s$), it is readily shown that four are identically
zero for all $z$. The remaining four, non-vanishing amplitudes are defined as

$\begin{split}&A_{i,\Omega}^{\mathrm{vac}}(z,\omega)\equiv\bra{0}\hat{A}_{i}(z,\omega)\ket{{1_{\Omega,i}}}\\\
&A_{i,\Omega}^{\mathrm{out}}(z,\omega)\equiv\bra{{1_{\Omega,{s}}}}\hat{A}_{i}(z,\omega)\ket{0}\\\
&A_{s,\Omega}^{\mathrm{vac}}(z,\omega)\equiv\bra{0}\hat{A}_{s}(z,\omega)\ket{{1_{\Omega,s}}}\\\
&A_{s,\Omega}^{\mathrm{out}}(z,\omega)\equiv\bra{{1_{\Omega,{i}}}}\hat{A}_{s}(z,\omega)\ket{0},\end{split}$
(6)

where $i,s$ denote the idler and signal respectively. These four complex field
amplitudes can be interpreted as the vacuum field
($A_{\alpha,\Omega}^{\mathrm{vac}}$) and the generated output fields
($A_{\alpha,\Omega}^{\mathrm{out}}$) in the signal and idler modes,
originating from the vacuum fluctuations in frequency $\Omega$. From Eq.1 we
obtain their dynamics in terms of four integro-differential equations

$\begin{split}&i\frac{\partial}{\partial
z}\left(\begin{matrix}A_{i,\Omega}^{\mathrm{out}}\\\
A_{i,\Omega}^{\mathrm{vac}}\end{matrix}\right)(z,\omega)=-\frac{\omega^{2}\chi^{(2)}\mathcal{E}_{p}}{c^{2}k(\omega)}d_{\mathrm{NL}}(z)\int
d\omega^{\prime}\mathrm{e}^{i\Delta
k(\omega,\omega^{\prime})z}\mathcal{P}(\omega+\omega^{\prime})\left(\begin{matrix}A_{{s},\Omega}^{\mathrm{vac*}}\\\
A_{{s},\Omega}^{\mathrm{out*}}\end{matrix}\right)(z,\omega^{\prime})\\\
&i\frac{\partial}{\partial
z}\left(\begin{matrix}A_{s,\Omega}^{\mathrm{out}}\\\
A_{s,\Omega}^{\mathrm{vac}}\end{matrix}\right)(z,\omega)=-\frac{\omega^{2}\chi^{(2)}\mathcal{E}_{p}}{c^{2}k(\omega)}d_{\mathrm{NL}}(z)\int
d\omega^{\prime}\mathrm{e}^{i\Delta
k(\omega,\omega^{\prime})z}\mathcal{P}(\omega+\omega^{\prime})\left(\begin{matrix}A_{{i},\Omega}^{\mathrm{vac*}}\\\
A_{{i},\Omega}^{\mathrm{out*}}\end{matrix}\right)(z,\omega^{\prime}).\end{split}$
(7)

Whereas Eq.1 was for field operators, Eq.7 can be computed numerically by a
spit step algorithm, since it describes the evolution of the four complex
amplitudes described in Eq.6. From here onwards, we use the normalized gain
parameter $\kappa=\omega\chi^{(2)}\mathcal{E}_{p}L/cn$ to characterize the
high-gain regime, for which $\kappa\geq 1$. The boundary conditions at $z=0$
are given as $A_{\alpha,\Omega}^{\mathrm{out}}(0,\omega)=0$ and
$A_{\alpha,\Omega}^{\mathrm{vac}}(0,\omega)=\delta(\omega-\Omega)\mathcal{E}_{0,\alpha}(\omega)$,
where $\mathcal{E}_{0,\alpha}(\omega)$ is the c-number vacuum field amplitude
at frequency $\omega$. When the fields at $z=L$ are found, we compute Eq.5 by
tracing over all possible vacuum modes ${\Omega}$,

$\begin{split}G_{\alpha\beta}^{(1)}(\omega,\omega^{\prime})=\delta_{\alpha,\beta}\int
d\Omega
A_{\alpha,\Omega}^{\mathrm{out}*}(L,\omega)A_{\beta,\Omega}^{\mathrm{out}}(L,\omega^{\prime}),\end{split}$
(8)

where the Kronecker delta in Eq.8 arises due to the vanishing photon
amplitudes other than those appearing in Eq.6, implying that
$G^{(1)}_{is}=G^{(1)}_{si}=0$.

A second type of first-order quantum correlation involves terms that are
quadratic in the annihilation or creation operators, e.g. terms of the form
$\braket{a_{m}a_{n}}$ or $\braket{a_{m}^{\dagger}a_{n}^{\dagger}}$. We denote
such terms as

$Q_{\alpha\beta}(\omega,\omega^{\prime})=\bra{0}\hat{A}_{\alpha}(L,\omega)\hat{A}_{\beta}(L,\omega^{\prime})\ket{0}$
(9)

and, employing a similar procedure to the one used to derive the first-order
correlation, Eq.8, we calculate these correlations in terms of the complex
field amplitudes as

$\begin{split}Q_{\alpha\beta}(\omega,\omega^{\prime})&=(1-\delta_{\alpha,\beta})\int
d\Omega
A_{\alpha,\Omega}^{\mathrm{vac}}(\omega)A_{\beta,\Omega}^{\mathrm{out}}(\omega^{\prime}),\end{split}$
(10)

where the Kronecker delta appears for a similar reason as in Eq.8, implying
that $Q_{ii}=Q_{ss}=0$.

Equipped with all relevant first-order correlations, we can now calculate any
desired observable. The first one is the second-order correlation function,
which predicts important measurables in quantum optics experiments in the low-
gain regime of SPDC, which will be our base case for validation of our model.
Namely, the second order correlation function
$G_{\alpha\beta\beta\alpha}^{(2)}(\omega,\omega^{\prime};\omega^{\prime},\omega)=\langle\hat{A}^{\dagger}_{\alpha}(z,\omega)\hat{A}^{\dagger}_{\beta}(z,\omega^{\prime})\hat{A}_{\beta}(z,\omega^{\prime})\hat{A}_{\alpha}(z,\omega)\rangle$
[28] denotes the probability of finding one $\alpha$ photon in mode $\omega$
and another $\beta$ photon in mode $\omega^{\prime}$, and corresponds to the
coincidence probability of these two photons. For Gaussian quantum states,
such as the multimode squeezed vacuum, the second-order correlation function
can be written in terms of first-order correlations as

$\begin{split}G_{\alpha\beta\beta\alpha}^{(2)}(\omega,\omega^{\prime};\omega^{\prime},\omega)=&G_{\alpha\alpha}^{(1)}(\omega,\omega)G_{\beta\beta}^{(1)}(\omega^{\prime},\omega^{\prime})+|G_{\alpha\beta}^{(1)}(\omega,\omega^{\prime})|^{2}+|Q_{\alpha\beta}(\omega,\omega^{\prime})|^{2}.\end{split}$
(11)

In the high-gain regime, an important observable that is commonly employed to
evaluate multimode BSV is the noise reduction factor (NRF), which quantifies
twin-beam squeezing [19, 37]. In terms of the first-order correlations we
obtain from our simulation, it can be expressed as follows

$\begin{split}\mathrm{NRF}\equiv\frac{\mathrm{Var}(N_{i}-N_{s})}{\braket{N_{i}+N_{s}}}=1+\frac{\mathcal{N}^{2}_{i}+\mathcal{N}^{2}_{s}-\mathcal{Q}_{is}^{2}-\mathcal{Q}_{si}^{2}}{\mathcal{N}_{i}+\mathcal{N}_{s}},\end{split}$
(12)

where $N_{i},N_{s}$ are the number of photons of the idler and signal,
$\mathcal{N}_{j}=\int d\omega G^{(1)}_{jj}(\omega,\omega)$ and
$\mathcal{Q}_{jk}^{2}=\int d\omega\int
d\omega^{\prime}|Q_{jk}(\omega,\omega^{\prime})|^{2}$ denote observables
integrated over a given frequency window, equivalent to the spectral bandwidth
of a filter in an experiment [37]. For twin-beam squeezing, $\mathrm{NRF}<1$.

Finally, to evaluate the CV cluster states in the frequency domain, we
consider a squeezing Hamiltonian written in terms of a discrete set of
frequency bins, which takes the form

$H=i\hbar\kappa\sum_{jk}G_{jk}(a_{i,j}^{\dagger}a_{s,k}^{\dagger}-a_{i,j}a_{s,k}),$
(13)

where $a_{\alpha,j}$, $\alpha=i,s$ are the idler (signal) ladder operators of
frequency bin $j$, and $G_{jk}$ is called the adjacency matrix of the
Hamiltonian graph, connecting modes $j$ and $k$. CV cluster states are defined
by a slightly different adjacency matrix, $\mathbf{V}$, also known as the
adjacency matrix of the cluster graph. The entries of $\mathbf{V}$ correspond
to the weights of the edges of the cluster graph, whereas the modes $\alpha,j$
constitute the vertices. The Hamiltonian and cluster adjacency matrices are
mathematically connected [22, 20], such that any Hamiltonian graph
$\mathbf{G}$ can define a cluster graph $\mathbf{V}$. For a special case
wherein $\mathbf{G}=\mathbf{G}^{-1}$, the two adjacency matrices coincide and
$\mathbf{G}=\mathbf{V}$[22, 20].

The signature of CV cluster states is manifested in entanglement witnesses
called nullifiers [22, 20]. These are linear combinations of the quadrature
operators of the different modes $j=1,2,...,N$, given by

$q_{j}(\phi)=e^{i\phi}a_{j}+e^{-i\phi}a_{j}^{\dagger},$ (14)
$p_{j}(\phi)=(e^{i\phi}a_{j}-e^{-i\phi}a_{j}^{\dagger})/i,$ (15)

where $\phi$ is a phase factor determining the orientation of the quadrature
axis in phase space. Ideally (that is, within the limit of infinite squeezing)
a CV cluster state should satisfy the vector equation

$\braket{(\textbf{p}(\phi)-\mathbf{V}\textbf{q}(\phi))^{2}}\rightarrow 0,$
(16)

where $\mathbf{q}(\phi)=(q_{1},q_{2},...,q_{N})$ and
$\mathbf{p}(\phi)=(p_{1},p_{2},...,p_{N})$ are the quadrature operator
vectors. The linear combinations defining the $N$ different nullifiers are the
entries of the vector $\mathbf{p}(\phi)-\mathbf{Vq}(\phi)$. Realistically, for
finite squeezing it will be necessary to demonstrate simultaneous noise
reduction in each of the nullifiers below the vacuum noise level.

##  3 Results

We first validate our model against known results for SPDC spectral shaping in
the low-gain regime. In this case, one considers the joint spectral intensity
(JSI)[38] of the photon pairs, found via first-order perturbation theory as

$\mathrm{JSI}=|\mathcal{P}(\omega+\omega^{\prime})\Phi(\omega,\omega^{\prime})|^{2},$
(17)

where $\Phi(\omega,\omega^{\prime})=\int dz\exp{[i\Delta
k(\omega,\omega^{\prime})z]}d_{\mathrm{NL}}(z)$ is the phase-matching
function. In the low-gain regime ($\kappa\ll 1$), the second-order correlation
of BSV given by Eq.11 reduces to the JSI. Fig.2(a-d) shows the JSI alongside
our simulation results in the low and high-gain regimes, for a $46\mu m$
periodically poled KTP crystal of length $13.7mm$ pumped by a $791nm$ laser.
The BSV correlations in the low-gain regime indeed recover the JSI.

Figure 2: Comparison between the JSI and the simulated BSV second-order
correlation for 3 different cases. (a-b) low-gain, $\kappa=0.023$ and (c-d)
high-gain, $\kappa=22.961$ regimes for a periodically poled KTP crystal; (e-f)
round JSI using an apodized crystal in the high-gain regime. The different
crystal poling patterns are illustrated in the bottom left insets. (g) NRF as
a function of $\kappa$.

However, the high-gain regime shows a broader, circularly symmetric
correlation, indicating a separable state. This is consistent with the
observation that in the high-gain regime the Schmidt rank should decrease
[39]. To increase the separability of the squeezed vacuum state throughout all
the gain regimes, one may add an apodization to the poling pattern (see insets
of Fig.2(e,f)), which prevents side lobes from forming. This results in a
circularly-symmetric JSI [11] that persists also in the high-gain regime
(Fig.2(e,f)).

Since the size of the mode in the frequency domain (the range of frequency
over which we measure the photons of each mode) depends on the gain when the
latter is increased, it causes a reduction in the number of selected modes
captured by the domain of a given window, as was shown in Refs [40, 37].
Indeed, when we increase the gain parameter we see a growth of the NRF as
shown in Fig.2(g).

Figure 3: High dimensional frequency domain quantum correlations of BSV using
ultrafast shaped pump pulses and NLPC’s. Different frequency-bin graph states
are generated using the same structured NLPC (a), whereas the pump spectrum is
varied between three (b) and four (c) Gaussians. (a) Spatial modulation of
$\chi^{(2)}$ of the crystal along $z$ (gray), and the Fourier transform of the
desired spectrum (brown). Each individual Gaussian has FWHM of $5nm$.
Simulated second-order correlation of three (d,f) and four (e,g) Gaussians
pump spectrum for different gain parameters, as well as the NRF for the lobes
circled in red and the gain parameter.

Fig.3 shows simulated high-dimensional frequency-domain quantum correlations,
using the same NLPC hologram (Fig.3(a)) that is pumped with two different
spectrally-shaped pumps (Fig.3(b-c)). The two cases shown in Fig.3(d,f) and
(e,g) represent, respectively, second-order correlations obtained using a
3-lobe (Fig.3(b)) and 4-lobe (Fig.3(c)) pump spectra, optimized such that each
node contains a separable, circularly-symmetric correlation. In the low-gain
regime, the shape of these correlations corresponds exactly to the JSI of
Eq.17. For high gain values, additional lobes appear in the second-order
correlation function (Fig.3(f,g)), due to the non-negligible probability of
higher-order photon pairs. We note that these features do not imply that a
decoherence mechanism hampers the quantum state. Rather, they imply that the
second order correlations do not encapsulate all the relevant information, as
they usually do in the low-gain regime of SPDC. Indeed, a full phase-space
description is more suitable for high-gain, and we shall use it to analyze the
next example of this section. For completeness, we calculate the NRF for a
frequency window containing one node. The NRF in Figs.3(f,g) is higher than in
Figs.3(d,e), respectively, as expected due to the increased gain.

The product of the pump spectrum, $\mathcal{P}(\omega+\omega^{\prime})$, and
the hologram encoded in the phase matching function,
$\Phi(\omega,\omega^{\prime})$, determines the adjacency matrix $\mathbf{G}$
in the Hamiltonian of Eq.13. As explained in the Theory section, the matrix
$\mathbf{G}$ defines a Hamiltonian graph state, and, by extension, a cluster
state [20]. The ability to arbitrarily engineer and optically control the
adjacency matrix $\mathbf{G}$ allows for the modulation of multimode BSV
states on an ultrafast time scale, which may prove important for scaling-up CV
quantum information processing rates.

We now focus our attention on the phase-space analysis of such CV cluster
states. First, we note that the periodic frequency bin design of Fig.3 in fact
generates two separable Hamiltonian graph states, with two independent
Hamiltonian adjacency matrices $\mathbf{G}^{A}$ and $\mathbf{G}^{B}$, such
that

$H=i\hbar\kappa_{A}\sum_{jk}G_{jk}^{A}(a_{i,j}^{\dagger}a_{s,k}^{\dagger}-a_{i,j}a_{s,k})+i\hbar\kappa_{B}\sum_{jk}G_{jk}^{B}(b_{i,j}^{\dagger}b_{s,k}^{\dagger}-b_{i,j}b_{s,k})\equiv
H_{A}+H_{B}.$ (18)

Each of the two Hamiltonian adjacency matrices corresponds to an independent
set of modes, forming two non-overlapping square lattices ($A$ and $B$,
represented by blue and green circles in Fig.4(a)) in the idler-signal
wavelength plane. Each lattice site corresponds to an edge of the graph,
connecting a pair of idler-signal modes $a_{i,j},a_{s,k}$ (lattice $A$) or
$b_{i,j},b_{s,k}$ (lattice $B$), and each idler or signal mode is a vertex in
its respective graph. Since these two sets of modes do not overlap, the two
parts of the Hamiltonian of Eq.18 commute. Therefore, the multimode squeezing
operator generating the BSV factors into two separate squeezing operations,

$S=\exp(-itH/\hbar)=\exp(-itH_{A}/\hbar)\otimes\exp(-itH_{B}/\hbar)$ (19)

generating a product of two Hamiltonian graph states. The two Hamiltonian
graph states generated by Eq.18 are known to be equivalent to cluster states
via transformation of the Hamiltonian adjacency matrix into a cluster
adjacency matrix [22]. Moreover, their square lattice geometry is known to be
universal for MBQC [20]. Each of these cluster states could then be used in
parallel for quantum computation. Importantly, as the two clusters form a
separable product state, tracing out one cluster state need not hamper the
quantum coherence of the other. Below, we focus the discussion on one of the
Hamiltonian graph states and analyze the simplest case of an emergent square
CV cluster state, first demonstrated in [22].

In Fig.4(b), we plot the product of the pump spectrum and the crystal
hologram, which can be used to generate a 4-mode square Hamiltonian graph
state in one of the square lattices, marked by blue circles in Fig.4(b). In
this case, the pump spectrum is designed to have three identical equally
spaced peaks and the crystal phase matching curve is also designed to have
three identical equally spaced peaks. Remarkably, as shown in [22], in this
case, the Hamiltonian graph state is identical to its corresponding cluster
state. Fig.4(c) depicts a graphical representation of the resulting square
cluster state. In this example, there are two idler modes, $i_{0},i_{1}$, and
two signal modes $s_{0},s_{1}$, at wavelengths $1564nm$ and $1605nm$,
respectively, comprising the four vertices of the graph. The values at these
frequency bins in Fig.4(b) correspond to the weighted edges between the four
vertices of the graph in Fig.4(c). For example, vertices $i_{1}$ and $s_{1}$
in Fig.4(c) are connected by an edge with weight $+1/\sqrt{2}$ (phase $0$),
the value of which is determined by the value of the marked frequency bin at
the top right in Fig.4(b). Vertices $s_{0}$ and $i_{1}$ in Fig.4(c), however,
are connected by an edge with weight $-1/\sqrt{2}$ (phase $\pi$),
corresponding to the marked frequency bin at the top left in Fig.4(b).

Figure 4: (a) Our shaping scheme generates two separable Hamiltonian graph
states with square lattice geometry, that are shifted with respect to each
other. Spectral filtering or demultiplexing can be used to distinguish between
the two graphs. (b) Product of pump spectrum and nonlinear hologram, for the
generation of a 4-mode square cluster state (from the highlighted bins)
depicted in (c), together with the phase of its weighted edges. The signal and
idler each have 2 modes ($s_{0,1}$, $i_{0,1}$), comprising the four nodes of
the graph. (d) The graph state that is generated by the second Hamiltonian,
together with the phase of its weighted edges. The signal and idler each have
3 modes ($s_{a,b,c}$, $i_{a,b,c}$), comprising the six nodes of the graph.

For this simple cluster state, denoting
$\mathbf{q}(\phi)=(q_{i0},q_{i1},q_{s0},q_{s1})$ and
$\mathbf{p}(\phi)=(p_{i0},p_{i1},p_{s0},p_{s1})$ as the vectors of quadrature
operators for the different modes, the Hamiltonian adjacency matrix
$\mathbf{G}$ is given by

$\mathbf{G}=\frac{1}{\sqrt{2}}\begin{pmatrix}0&0&1&1\\\ 0&0&-1&1\\\
1&-1&0&0\\\ 1&1&0&0\\\ \end{pmatrix},$ (20)

and, as mentioned above, since the Hamiltonian adjacency matrix satisfies the
condition $\mathbf{G}=\mathbf{G}^{-1}$, the cluster adjacency matrix,
$\mathbf{V}$, is identical to $\mathbf{G}$.

To further explore the resulting cluster state, we calculate the uncertainty
in each of the four nullifiers,

$\begin{split}\textbf{p}(\phi)-\mathbf{V}\textbf{q}(\phi)=\begin{pmatrix}p_{i0}-(q_{s0}+q_{s1})/\sqrt{2}\\\
p_{i1}-(q_{s1}-q_{s0})/\sqrt{2}\\\ p_{s0}-(q_{i0}-q_{i1})/\sqrt{2}\\\
p_{s1}-(q_{i0}+q_{i1})/\sqrt{2}\end{pmatrix},\end{split}$ (21)

which should all be simultaneously squeezed below the vacuum noise level to
satisfy Eq.16. Fig.5(a) shows the calculated quantum nullifier uncertainties
for the square cluster state, for two different gain parameters
$\kappa=0.8,5.9$. Indeed, in both cases, all four nullifiers are
simultaneously squeezed below the vacuum noise level for a certain phase angle
$\phi$ of Eqs.14-15. Clearly, stronger gain results in a lower nullifier
uncertainty, reaching more than $-5dB$ of squeezing. These squeezing levels
are already useful for applications in sensing and spectroscopy.

For completeness, we calculate the covariance matrix (Eq.3) determining the
Wigner function (Eq.2) of our square cluster state (Fig.4(c)), as shown in
Fig.5(b,c) for two different gain parameters. For low gain, the squeezing is
weak and the covariance matrix is expected to resemble the identity matrix -
the covariance matrix of the vacuum state[33]. As the gain increases,
additional non-zero off-diagonal elements appear in the covariance matrix,
indicating correlations between states in the chosen basis. Note that, for
similar reasons used to explain Eq.8 and Eq.10, all elements of the form
$\braket{a_{i,j}a_{i,k}},\braket{a_{s,j}a_{s,k}},\braket{a_{i,j}^{\dagger}a_{s,k}}$
for $j,k=0,1$ and their conjugates, are identically zero.

Figure 5: (a) Comparison of nullifier uncertainties for two cases of squeezing
(red, blue for $\kappa=0.8,5.9$ respectively). It is evident that for both
cases, the nullifier uncertainties fall below the quantum limit (dotted black
line) for a specific angle $\phi$. Covariance matrix of the square cluster
state for $\kappa=0.8$ (b) and $\kappa=5.9$ (c).

##  4 Conclusions

In summary, we proposed design methods for spectral shaping of BSV using
tailored NLPC holograms and ultrashort, structured, and intense pump beams,
for applications in frequency-domain continuous variable quantum information
processing. We provide a series of tools to characterize the generated states,
including the signal-idler first and second-order correlations (Eqs.8,11), the
noise reduction factor (Eq.12), the covariance matrix (Eq.3) and the nullifier
uncertainties (Eq.16).

Contrary to the conventional methods for generating frequency domain CV
cluster states employing the frequency comb of optical parametric oscillators,
our proposed nonlinear holograms [16], which could also be accompanied with
quasiperiodic modulation [41], enable to arbitrarily control the phase
matching function in the frequency domain. Further, the all-optical control
allowed by our scheme enables the modulation of such CV cluster states on
ultrafast times scales. Our method can serve as a design tool for quantum
information applications that utilize the spectral degree of freedom of light.
Owing to the high degree of control over the output state - by the pump
spectrum and crystal holograms - inverse design optimization of modular
quantum correlations over frequency-bin grids can be readily employed to
discover further designs of desired quantum states, as recently done for the
spatial degree of freedom of SPDC [31, 42].

Moreover, our proposed method can also be extended to generate energy-time CV
cluster states by exploiting the interplay between the spectral and temporal
degrees of freedom of light. This can be achieved by incorporating tailored
temporal modulation of the pump beam with the NLPC, which would enable us to
engineer the joint spectral-temporal amplitude of the generated BSV. By
combining our proposed method with low-frequency temporal modulation
techniques [43], we can generate a wide variety of high-dimensional CV cluster
states for various applications in quantum information processing.

Funding Ministry of Science, Technology and Space; Israel Science Foundation
(969/22).

Acknowledgments A.K. acknowledges support by the Adams Fellowship of the
Israeli Academy of Sciences and Humanities.

Disclosures The authors declare no conflicts of interest.

Data Availability Statement Data underlying the results presented in this
paper are not publicly available at this time but may be obtained from the
authors upon reasonable request.

## References

  * [1] S. Clemmen, A. Farsi, S. Ramelow, and A. L. Gaeta, “Ramsey interference with single photons,” Phys. Rev. Lett. 117, 223601 (2016).
  * [2] A. Karnieli and A. Arie, “Frequency domain stern gerlach effect for photonic qubits and qutrits,” Optica 5, 1297–1303 (2018).
  * [3] Y. Shaked, Y. Michael, R. Z. Vered, L. Bello, M. Rosenbluh, and A. Pe’er, “Lifting the bandwidth limit of optical homodyne measurement with broadband parametric amplification,” Nature Communications 9, 609 (2018).
  * [4] M. Bloch, S. W. McLaughlin, J. M. Merolla, and F. Patois, “Frequency-coded quantum key distribution,” Opt. Lett. 32, 301–303 (2007).
  * [5] M. Kues, C. Reimer, J. M. Lukens, W. J. Munro, A. M. Weiner, D. J. Moss, and R. Morandotti, “Quantum optical microcombs,” Nature Photonics 13, 170–179 (2019).
  * [6] H. H. Lu, K. V. Myilswamy, R. S. Bennink, S. Seshadri, M. S. Alshaykh, J. Liu, T. J. Kippenberg, D. E. Leaird, A. M. Weiner, and J. M. Lukens, “Bayesian tomography of high-dimensional on-chip biphoton frequency combs with randomized measurements,” Nature Communications 13 (2022).
  * [7] M. Kues, C. Reimer, P. Roztocki, L. R. Cortés, S. Sciara, B. Wetzel, Y. Zhang, A. Cino, S. T. Chu, B. E. Little, D. J. Moss, L. Caspani, J. Azaña, and R. Morandotti, “On-chip generation of high-dimensional entangled quantum states and their coherent control,” Nature 546, 622–626 (2017).
  * [8] M. Chen, N. C. Menicucci, and O. Pfister, “Experimental realization of multipartite entanglement of 60 modes of a quantum optical frequency comb,” Phys. Rev. Lett. 112, 120505 (2014).
  * [9] C. Reimer, S. Sciara, P. Roztocki, M. Islam, L. Romero Cortés, Y. Zhang, B. Fischer, S. Loranger, R. Kashyap, A. Cino, S. T. Chu, B. E. Little, D. J. Moss, L. Caspani, W. J. Munro, J. Azaña, M. Kues, and R. Morandotti, “High-dimensional one-way quantum processing implemented on d-level cluster states,” Nature Physics 15, 148–153 (2019).
  * [10] T. Santiago-Cruz, S. D. Gennaro, O. Mitrofanov, S. Addamane, J. Reno, I. Brener, and M. V. Chekhova, “Resonant metasurfaces for generating complex quantum states,” Science 377, 991–995 (2022).
  * [11] A. Dosseva, L. Cincio, and A. M. Brańczyk, “Shaping the joint spectrum of down-converted photons through optimized custom poling,” Phys. Rev. A 93, 013801 (2016).
  * [12] F. Graffitti, D. Kundys, D. T. Reid, A. M. Brańczyk, and A. Fedrizzi, “Pure down-conversion photons through sub-coherence-length domain engineering,” Quantum Science and Technology 2 (2017).
  * [13] L. G. Helt, M. J. Steel, and J. E. Sipe, “Spontaneous parametric downconversion in waveguides: What’s loss got to do with it?” New Journal of Physics 17 (2015).
  * [14] K. Garay-Palmett, D. B. Kim, Y. Zhang, F. A. Domínguez-Serna, V. O. Lorenz, and A. B. U’Ren, “Fiber-based photon-pair generation: tutorial,” J. Opt. Soc. Am. B 40, 469–490 (2023).
  * [15] D. Zhang, C. Li, Z. Zhang, Y. Zhang, Y. Zhang, and M. Xiao, “Enhanced intensity-difference squeezing via energy-level modulations in hot atomic media,” Phys. Rev. A 96, 043847 (2017).
  * [16] R. Shiloh and A. Arie, “Spectral and temporal holograms with nonlinear optics,” Opt. Lett. 37, 3591–3593 (2012).
  * [17] C. L. Morrison, F. Graffitti, P. Barrow, A. Pickston, J. Ho, and A. Fedrizzi, “Frequency-bin entanglement from domain-engineered down-conversion,” APL Photonics 7, 066102 (2022).
  * [18] F. Graffitti, P. Barrow, M. Proietti, D. Kundys, and A. Fedrizzi, “Independent high-purity photons created in domain-engineered crystals,” Optica 5, 514–517 (2018).
  * [19] P. R. Sharapova, O. V. Tikhonova, S. Lemieux, R. W. Boyd, and M. V. Chekhova, “Bright squeezed vacuum in a nonlinear interferometer: Frequency and temporal Schmidt-mode description,” Physical Review A 97, 053827 (2018).
  * [20] O. Pfister, “Continuous-variable quantum computing in the quantum optical frequency comb,” Journal of Physics B: Atomic, Molecular and Optical Physics 53, 012001 (2019).
  * [21] M. Manceau, G. Leuchs, F. Khalili, and M. Chekhova, “Detection loss tolerant supersensitive phase measurement with an su(1,1) interferometer,” Phys. Rev. Lett. 119, 223604 (2017).
  * [22] N. C. Menicucci, S. T. Flammia, H. Zaidi, and O. Pfister, “Ultracompact generation of continuous-variable cluster states,” Phys. Rev. A 76, 010302 (2007).
  * [23] Y. Cai, J. Roslund, G. Ferrini, F. Arzani, X. Xu, C. Fabre, and N. Treps, “Multimode entanglement in reconfigurable graph states using optical frequency combs,” Nature Communications 8, 1–9 (2017).
  * [24] C. Drago and A. M. Brańczyk, “Tunable frequency-bin multimode squeezed vacuum states of light,” Phys. Rev. A 106, 043714 (2022).
  * [25] F. Arzani, C. Fabre, and N. Treps, “Versatile engineering of multimode squeezed states by optimizing the pump spectral profile in spontaneous parametric down-conversion,” Phys. Rev. A 97, 033808 (2018).
  * [26] J. Nokkala, F. Arzani, F. Galve, R. Zambrini, S. Maniscalco, J. Piilo, N. Treps, and V. Parigi, “Reconfigurable optical implementation of quantum complex networks,” New Journal of Physics 20, 053024 (2018).
  * [27] M. J. Werner, M. G. Raymer, M. Beck, and P. D. Drummond, “Ultrashort pulsed squeezing by optical parametric amplification,” Phys. Rev. A 52, 4202–4213 (1995).
  * [28] E. Brambilla, A. Gatti, M. Bache, and L. A. Lugiato, “Simultaneous near-field and far-field spatial quantum correlations in the high-gain regime of parametric down-conversion,” Phys. Rev. A 69, 023802 (2004).
  * [29] P. R. Sharapova, G. Frascella, M. Riabinin, A. M. Pérez, O. V. Tikhonova, S. Lemieux, R. W. Boyd, G. Leuchs, and M. V. Chekhova, “Properties of bright squeezed vacuum at increasing brightness,” Phys. Rev. Research 2, 013371 (2020).
  * [30] S. Trajtenberg-Mills, A. Karnieli, N. Voloch-Bloch, E. Megidish, H. S. Eisenberg, and A. Arie, “Simulating correlations of structured spontaneously down-converted photon pairs,” Laser & Photonics Reviews 14, 1900321 (2020).
  * [31] E. Rozenberg, A. Karnieli, O. Yesharim, J. Foley-Comer, S. Trajtenberg-Mills, D. Freedman, A. M. Bronstein, and A. Arie, “Inverse design of spontaneous parametric downconversion for generation of high-dimensional qudits,” Optica 9, 602–615 (2022).
  * [32] J. C. Garrison and R. Y. Chiao, _Quantum Optics_ (Oxford University, 2008).
  * [33] G. Adesso, S. Ragy, and A. R. Lee, “Continuous variable quantum information: Gaussian states and beyond,” arXiv 21, 1440001 (2014).
  * [34] N. Quesada and J. E. Sipe, “Effects of time ordering in quantum nonlinear optics,” Phys. Rev. A 90, 063840 (2014).
  * [35] N. Quesada, G. Triginer, M. D. Vidrighin, and J. E. Sipe, “Theory of high-gain twin-beam generation in waveguides: From maxwell’s equations to efficient simulation,” Phys. Rev. A 102, 033519 (2020).
  * [36] A. Christ, B. Brecht, W. Mauerer, and C. Silberhorn, “Theory of quantum frequency conversion and type-II parametric down-conversion in the high-gain regime,” New Journal of Physics 15 (2013).
  * [37] I. N. Agafonov, M. V. Chekhova, and G. Leuchs, “Two-color bright squeezed vacuum,” Phys. Rev. A 82, 011801 (2010).
  * [38] F. Graffitti, P. Barrow, A. Pickston, A. M. Brańczyk, and A. Fedrizzi, “Direct generation of tailored pulse-mode entanglement,” Phys. Rev. Lett. 124, 053603 (2020).
  * [39] P. Sharapova, A. M. Pérez, O. V. Tikhonova, and M. V. Chekhova, “Schmidt modes in the angular spectrum of bright squeezed vacuum,” Phys. Rev. A 91, 043816 (2015).
  * [40] O. Jedrkiewicz, Y.-K. Jiang, E. Brambilla, A. Gatti, M. Bache, L. A. Lugiato, and P. Di Trapani, “Detection of sub-shot-noise spatial correlation in high-gain parametric down conversion,” Phys. Rev. Lett. 93, 243601 (2004).
  * [41] R. Lifshitz, A. Arie, and A. Bahabad, “Photonic quasicrystals for nonlinear optical frequency conversion,” Phys. Rev. Lett. 95, 133901 (2005).
  * [42] E. V. Kovlakov, S. S. Straupe, and S. P. Kulik, “Quantum state engineering with twisted photons via adaptive shaping of the pump beam,” Phys. Rev. A 98, 060301 (2018).
  * [43] J. Yoshikawa, S. Yokoyama, T. Kaji, C. Sornphiphatphong, Y. Shiozawa, K. Makino, and A. Furusawa, “Invited article: Generation of one-million-mode continuous-variable cluster state by unlimited time-domain multiplexing,” APL Photonics 1, 060801 (2016).