Gaofei Shen1, Afra Alishahi1, Arianna Bisazza2, Grzegorz Chrupała1

# Wave to Syntax: Probing spoken language models for syntax

###### Abstract

Understanding which information is encoded in deep models of spoken and
written language has been the focus of much research in recent years, as it is
crucial for debugging and improving these architectures. Most previous work
has focused on probing for speaker characteristics, acoustic and phonological
information in models of spoken language, and for syntactic information in
models of written language. Here we focus on the encoding of syntax in several
self-supervised and visually grounded models of spoken language. We employ two
complementary probing methods, combined with baselines and reference
representations to quantify the degree to which syntactic structure is encoded
in the activations of the target models. We show that syntax is captured most
prominently in the middle layers of the networks, and more explicitly within
models with more parameters.111Code: https://github.com/techsword/wave-to-
syntax

Index Terms: speech recognition, syntax, computational linguistics

##  1 Introduction

State-of-the-art models of (spoken) language rely on deep learning
architectures composed of various components and based especially on the
Transformer model (e.g. [1, 2]). Evaluating the performance of these models
via standard quantitative protocols is straightforward enough; but it is not
always easy to understand the reasons for fine-grained patterns of behavior
and failure modes, and not trivial to debug and iterate on design. One tool to
aid in this process has been the analysis and interpretation of the
representations learned by the models, as encoded in the activation patterns
within the various components [3, 4].

For text-based models, numerous works have probed these activations for many
types of information, with a special interest in syntactic structure [5, 6,
7]. By contrast, the focus on speech models has been on the encoding of
acoustic information, speaker characteristics, phonetics and phonology (e.g.
[8, 9, 10]).

However, as the current crop of speech models grows in size and
sophistication, we ask whether they also learn to encode syntactic structure
to any appreciable degree. If the knowledge of syntax is useful for optimizing
a model's training objective, the expectation would be that, given enough
data, the model will learn to represent it. As a simple example, consider the
utterance The authors of the book are French: if the timesteps in the feature
space corresponding to are are masked and the model needs to reconstruct them,
then it can do it better if it encodes number agreement between subject and
verb. In the current study, we evaluate the hypothesis that syntax information
is in general represented in such models.

We use two established representation probing techniques [5, 7] in combination
with carefully designed baselines and reference representations to quantify
the encoding of syntactic structure in selected current models of spoken
language trained via self-supervision objectives. We also apply our
methodology to models trained via visual supervision (also known as visual
grounding) and to a model trained via a combination of self- and visual
supervision. We track the encoding of syntactic structure throughout the
transformer layers of these target models.

Our findings show that syntax is captured by all these models, with the
following caveats and details: Firstly, the encoding of syntax is generally
weaker than in text-based models (such as BERT [11]) . Secondly, much of the
syntactic structure that is captured may be encoded in lexical rather than
purely syntactic form. Thirdly, self-supervised and combined objectives lead
to less syntactic encoding in the final model layers, while the visually-
supervised objective does not have this effect. Finally, increased model size
is associated with the stronger encoding of syntactic information.

##  2 Related work

Within Natural Language Processing (NLP), there has been substantial interest
in understanding the representations emerging within text-based language
models: surveys of such work include [3, 4, 12]. The predominant family of
approaches relies on correlating the activation patterns in trained models to
linguistic structures that are considered necessary for correctly processing
natural language. For written language, these are often various types of word
categories, constituency structures or syntactic and semantic dependencies,
for example as proposed in [5, 6, 7].

For models of spoken language, most previous work has focused on acoustic,
phonetic and phonemic structures as well as on speaker characteristics, which
are the most plausible types of information a speech model is expected to
learn. Works analyzing the encoding of phonemes in a variety of speech models
ranging from basic CNN-based ASR models to current transformer-based self-
supervised models [8, 9, 13, 14, 15, 16] tend to find a salient encoding of
phonemes in some layers of the models analyzed. In [10] the authors check for
the encoding of acoustic, phonemic, lexical and semantic information in the
self-supervised wav2vec2 [1] using probes such as canonical correlation
analysis and mutual information, finding an autoencoder-style behavior, where
across the layers the representations first diverge from low-level input
features and at the end approximate them again. The reverse is the case for
higher-level information such as word identity and meaning. A few papers have
focused on analyzing phonology and/or semantics in visually-grounded models
[17], for example [18, 19], in general finding phonemes encoded in lower
layers and semantics in higher layers.

Much less work has looked at syntactic structures in models of spoken
language: one partial exception is [20], who probe wav2vec2 and Mockingjay
[21] for the encoding of acoustic and linguistic information, including syntax
tree depth. The results reported are quite surprising and even implausible in
that the encoding of most linguistic features in the speech models is found to
be stronger than in the text-based BERT.

In this paper, we focus exclusively on probing for syntactic structures,
aiming to examine them in several large-scale speech models, while taking care
to include two independent methods as well as all the appropriate baselines
and sanity checks in order to quantify the encoding of syntax in a reliable
way.

##  3 Methods

This study aims to reliably establish the presence of syntactic information in
models of spoken language. We, therefore, use established methods for this
type of analysis and focus on careful experimental design rather than
technical novelty. We use two separate probing techniques [5, 7], with several
target models trained and tested on two different datasets.

###  3.1 Datasets

We use two English audio datasets for the current study: LibriSpeech [22] and
SpokenCOCO [23]. LibriSpeech consists of audiobook recordings from the
LibriVox project with a total of 960 hours of audio. SpokenCOCO is a spoken
version of the image caption dataset COCO [24] with more than 600,000 spoken
utterances paired with text captions and images. We use the LibriSpeech
train-100h split and the SpokenCOCO validation split in our probing experiment
to reduce computational load. We filter out utterances longer than 52 words
for LibriSpeech and those longer than 20 words for SpokenCOCO. LABEL:tab:data
shows the details of the data used in our experiments.

Table 1: Datasets used for this study.

table]tab:data  Name #Utts. # Filtered Utts. LibriSpeech 24,766 24,592
SpokenCOCO 28,539 27,496

###  3.2 Target Models

For this study, we use the following model variants as specified in
LABEL:tab:models:

wav2vec2

[1] is pre-trained on LibriSpeech to discriminate masked time steps in feature
encoder outputs. In addition to the base pre-trained model, we also include a
fine-tuned base model and a fine-tuned large model. The large model222
https://huggingface.co/jonatasgrosman/wav2vec2-large-english is fine-tuned for
English ASR on the same dataset. Additionally, we fine-tuned the base model
for English ASR on our experimental data for 10,000 steps, which enables us to
check the effect of model size.

HuBERT

[2] is similar to the wav2vec2 architecture but pre-trained on labels created
off-line via clustering; also pre-trained on LibriSpeech.

FaST-VGS

[25] is a visually grounded model based on the wav2vec2 architecture. It loads
model weights from the pre-trained wav2vec2-base with randomly reinitialized
final four layers and is further trained on SpokenCOCO to match images with
the speech that describes them.

FaST-VGS+

[25] same as the previous model, but trained on both SpokenCOCO and
LibriSpeech with a combination of the visually-grounded loss and the wav2vec2
self-supervised loss.

BERT

[11] text-based language model included as a ceiling reference. BERT is pre-
trained on 3,300M words of books and web content.

BoW

a text-based bag-of-words representation constructed from the combined text
from both datasets with all sentences containing non-Latin characters removed.
This representation captures all the words in the utterance, but no word-order
information.

Table 2: table]tab:models Models investigated in this study. PT = Pre-Trained, FT = Fine-Tuned, SS = Self-Supervised, VS = Visually Supervised, AV = Audio-Visual. Model | Size | Train. | Loss | Mod.
---|---|---|---|---
wav2vec2 | base | PT | SS | Audio
wav2vec2 | base | FT | SS | Audio
wav2vec2 |  large333The base models have 12 transformer layers and a hidden size of 768; the large model has 24 layers and a hidden size of 1024. | FT | SS | Audio
HuBERT | base | PT | SS | Audio
FaST-VGS | base | PT | VS | AV
FaST-VGS+ | base | PT | SS+VS | AV
BERT | base | PT | SS | Text
BoW |  |  |  | Text

###  3.3 TreeDepth Probe

The objective of this probe is to predict the maximum depth of the
constituency tree of a given utterance from the activation pattern in each
transformer layer of a model when processing this utterance.

We generated hidden-state outputs and applied mean-pooling along the time axis
to generate utterance vectors for all transformer layers of each model. We
used the Stanza parser [26] to generate constituency trees for all utterances
and calculate their tree depth. We fit a ridge regression model on embeddings
generated for both LibriSpeech and SpokenCOCO. As controls, wordcount, and
bag-of-words (BoW) model representation and their combinations with the
embeddings were used in training the regression model as well. A 75:25 train-
test split was used, and the model selected via 10-fold cross-validation was
then evaluated on the test split and its score reported.

###  3.4 TreeKernel Probe

Representational Similarity Analysis (RSA) [27] is a method which correlates
the similarity structures of two representational spaces. RSAregress [7]
introduces a trainable version of RSA, where two input and output spaces are
set up in terms of vectors of similarity to a held-out set of anchor points,
and then a multivariate regression model is fit to map between them.

As in the original paper, we used cosine similarity between vectors in the
input space as metric, and tree kernels between pairs of syntax trees in the
output space. Tree kernel is a way of measuring similarities between syntactic
trees by efficiently computing the proportion of shared tree segments. After
obtaining the constituency trees, we delexicalized the trees to ensure the
tree kernel is only based on structure and not word overlap. We used the
algorithms introduced in [28, 29] for computing the normalized tree kernel; we
closely followed the specific details in [7], using parameter
$\lambda=\frac{1}{2}$ and two hundred anchor sentences. The score of the model
selected via 10-fold cross-validation is reported.

For both probes, the hyperparameter tuned was regularization strength $\alpha$
for values $\\{10^{n}~{}|~{}n\in\\{-3,-2,-1,0,1,2\\}\\}$.

##  4 Results

LABEL:fig:treedepth and LABEL:fig:treekernel show the results for the
TreeDepth and TreeKernel probing tasks, respectively. For clarity,
LABEL:fig:treedepth only shows results on Librispeech.

(a) Text vs. audio models figure]fig:text_vs_pt

(b) Pre-training vs. fine-tuning figure]fig:ptvsft_td

(c) Self vs. visual supervision figure]fig:avsv_td

Figure 1: $R^{2}$ scores for predicting TreeDepth from embeddings
(Librispeech). X-axis = transformer layer from shallow to deep, orange dashed
line = BoW reference, purple dashed line = Word count reference. See
LABEL:tab:models for panel headings. figure]fig:treedepth

(a) Text vs. audio models figure]fig:textvspt_tk

(b) Pre-training vs. fine-tuning figure]fig:ptvsft_tk

(c) Self vs. visual supervision figure]fig:avsv_tk

Figure 2: $R^{2}$ scores for embedding distances and TreeKernel. X-axis =
transformer layer from shallow to deep. Dashed lines = BoW reference. See
LABEL:tab:models for panel headings. figure]fig:treekernel

#### Syntax encoding in spoken vs. text models.

LABEL:fig:text_vs_pt shows that spoken language models feature the strongest
encoding of tree depth in the middle to deep layers. Concatenating wordcount
features to embeddings in the TreeDepth task improves the probing results in
the middle layers and outperforms the BoW reference. LABEL:fig:textvspt_tk
shows a similar trend for the TreeKernel probe.

As expected, both probing tasks show that syntactic information is encoded
more strongly and consistently in text models across all layers.444Our results
contradict those of [20]: possibly due to them report probing scores directly
on probe-training data: the description in the paper is unclear on this point.
However, TreeKernel results for the middle layers of the spoken language
models come close to the text models, with wav2vec2-large-ft having the
highest correlation score. Overall, these results suggest that spoken language
models encode syntactic structure to a moderate degree. Comparison with the
BoW reference suggests that most of the syntactic information encoded in
speech models is entangled with lexical representations, rather than being
abstract.

#### Pre-training vs. fine-tuning

LABEL:fig:ptvsft_td shows that models fine-tuned on ASR achieve higher scores
in the TreeDepth probe than the pre-trained models. The pre-trained models
have a large dip at the final layers with the scores go back to those for the
first layer. For the fine-tuned models, the final layer dip is less
pronounced. The pre-training objective is to reconstruct/discriminate audio
features, whereas the fine-tuning objective is to output well-formed
transcriptions, which requires more syntactic information to be encoded in the
activation patterns, resulting in the contrast between the final layer scores.

Comparing the panels in LABEL:fig:ptvsft_tk, we can see that while fine-tuning
increases the amount of syntactic information encoded by the model, the size
of the model also matters. The larger hidden size and deeper model
architecture prove useful in encoding extra information. Wav2vec2-large-ft was
also fine-tuned on significantly more data than wav2vec2-base-ft.

#### Self- vs. Visual supervision

As shown in LABEL:fig:avsv_td and LABEL:fig:avsv_tk, the score curve from
FaST-VGS+ has a similar shape as wav2vec2-base. In addition to visual
supervision, the plus variant also uses the same masked language modeling loss
as wav2vec2, and therefore the two models behave similarly. In comparison, the
final layer dip is absent for the FaST-VGS model, likely due to the fact that
FaST-VGS does not use the self-supervised objective, and thus not display a
decrease in syntax encoding in the final layers.

##  5 Conclusions

We use two established probing techniques to assess the amount of syntactic
information encoded by several spoken language models. The results from both
probes confirm that spoken language models encode a moderate level of
syntactic information. We see that different training objectives considerably
affect the degree of syntax encoded in each layer of the models and so does
model size, with text-based training and larger model size leading to higher
syntactic probe scores.

Our study only looks at sentence-level representations: it would also be
interesting to extend our experiments to sub-sentence level probing [6].
Additionally, while we only use English datasets in this work, future studies
could compare the ability of large-scale spoken language models to encode
syntactic structures across different languages.

##  6 Acknowledgements

This publication is part of the project InDeep: Interpreting Deep Learning
Models for Text and Sound (with project number NWA.1292.19.399) of the
National Research Agenda (NWA-ORC) program.

## References

  * [1] A. Baevski, H. Zhou, A. Mohamed, and M. Auli, ``Wav2vec 2.0: A framework for self-supervised learning of speech representations,'' in _NIPS 2020_ , 2020\.
  * [2] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, ``Hubert: Self-supervised speech representation learning by masked prediction of hidden units,'' _TASLP_ , vol. 29, pp. 3451–3460, 2021\.
  * [3] Y. Belinkov and J. R. Glass, ``Analysis methods in neural language processing: A survey,'' _TACL_ , vol. 7, pp. 49–72, 2018.
  * [4] A. Rogers, O. Kovaleva, and A. Rumshisky, ``A primer in bertology: What we know about how bert works,'' _TACL_ , vol. 8, pp. 842–866, 2020.
  * [5] A. Conneau, G. Kruszewski, G. Lample, L. Barrault, and M. Baroni, ``What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties,'' in _ACL 2018_ , vol. 1: Long Papers, Jul. 2018, pp. 2126–2136.
  * [6] J. Hewitt and C. D. Manning, ``A structural probe for finding syntax in word representations,'' in _NAACL_ , 2019.
  * [7] G. Chrupała and A. Alishahi, ``Correlating Neural and Symbolic Representations of Language,'' in _ACL 2019_. ACL, 2019, pp. 2952–2962. [Online]. Available: https://aclanthology.org/P19-1283
  * [8] A. Krug, R. Knaebel, and S. Stober, ``Neuron activation profiles for interpreting convolutional speech recognition models,'' in _NeurIPS-IRASL_ , 2018.
  * [9] G. Chrupała, B. Higy, and A. Alishahi, ``Analyzing analytical methods: The case of phonology in neural models of spoken language,'' in _ACL 2020_. ACL, 2020, pp. 4146–4156.
  * [10] A. Pasad, J.-C. Chou, and K. Livescu, ``Layer-wise analysis of a self-supervised speech representation model,'' in _IEEE ASRU 2021_ , 2021, pp. 914–921.
  * [11] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, ``BERT: Pre-training of deep bidirectional transformers for language understanding,'' in _NAACL-HLT 2019_ , vol. Long and Short Papers, 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
  * [12] H. Sajjad, N. Durrani, and F. Dalvi, ``Neuron-level interpretation of deep nlp models: A survey,'' _TACL_ , vol. 10, pp. 1285–1303, 2021.
  * [13] T. Nagamine, M. L. Seltzer, and N. Mesgarani, ``Exploring how deep neural networks form phonemic categories,'' in _Proc. Interspeech_ , 2015.
  * [14] Y. Belinkov and J. R. Glass, ``Analyzing hidden representations in end-to-end automatic speech recognition systems,'' in _NIPS_ , 2017.
  * [15] Y. Belinkov, A. Ali, and J. Glass, ``Analyzing Phonetic and Graphemic Representations in End-to-End Automatic Speech Recognition,'' in _Proc. Interspeech 2019_ , 2019, pp. 81–85.
  * [16] M. de Seyssel, M. Lavechin, Y. Adi, E. Dupoux, and G. Wisniewski, ``Probing phoneme, language and speaker information in unsupervised speech representations,'' in _Proc. Interspeech_ , 2022.
  * [17] G. Chrupała, ``Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques,'' _J. Artif. Intell. Res._ , vol. 73, pp. 673–707, 2021.
  * [18] A. Alishahi, M. Barking, and G. Chrupala, ``Encoding of phonology in a recurrent neural model of grounded speech,'' in _Proc. 21st CoNLL 2017_. ACL, 2017, pp. 368–378. [Online]. Available: https://aclanthology.org/K17-1037
  * [19] K. Khorrami and O. Räsänen, ``Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? - a computational investigation,'' _LDR_ , vol. 1, no. 1, 2021. [Online]. Available: http://ldr.lps.library.cmu.edu/article/id/434/
  * [20] Y. K. Singla, J. Shah, C. Chen, and R. R. Shah, ``What do audio transformers hear? probing their representations for language delivery & structure,'' in _ICDMW 2022_ , 2022, pp. 910–925.
  * [21] A. T. Liu, S. wen Yang, P.-H. Chi, P.-C. Hsu, and H. yi Lee, ``Mockingjay: Unsupervised speech representation learning with deep bidirectional transformer encoders,'' _ICASSP 2020_ , pp. 6419–6423, 2019.
  * [22] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, ``Librispeech: An ASR corpus based on public domain audio books,'' in _ICASSP 2015_ , 2015, pp. 5206–5210.
  * [23] W.-N. Hsu, D. Harwath, T. Miller, C. Song, and J. Glass, ``Text-Free Image-to-Speech Synthesis Using Learned Segmental Units,'' in _ACL-IJCNLP 2021_ , vol. 1: Long Papers. ACL, 2021, pp. 5284–5300. [Online]. Available: https://aclanthology.org/2021.acl-long.411
  * [24] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and L. Zitnick, ``Microsoft COCO: Common objects in context,'' in _ECCV_ , 2014. [Online]. Available: https://www.microsoft.com/en-us/research/publication/microsoft-coco-common-objects-in-context/
  * [25] P. Peng and D. Harwath, ``Self-supervised representation learning for speech using visual grounding and masked language modeling,'' in _AAAI-SAS 2022_ , 2022.
  * [26] P. Qi, Y. Zhang, Y. Zhang, J. Bolton, and C. D. Manning, ``Stanza: A Python Natural Language Processing Toolkit for Many Human Languages,'' in _ACL 2020: System Demonstrations_. ACL, 2020, pp. 101–108. [Online]. Available: https://www.aclweb.org/anthology/2020.acl-demos.14
  * [27] N. Kriegeskorte, M. Mur, and P. A. Bandettini, ``Representational similarity analysis-connecting the branches of systems neuroscience,'' _Frontiers in systems neuroscience_ , p. 4, 2008.
  * [28] A. Moschitti, ``Making Tree Kernels Practical for Natural Language Learning,'' in _EACL 2006_. ACL, 2006, pp. 113–120. [Online]. Available: https://aclanthology.org/E06-1015
  * [29] M. Collins and N. Duffy, ``Convolution Kernels for Natural Language,'' in _NIPS 2001_ , vol. 14. MIT Press, 2001.


#  $t\bar{t}H$ production in NNLO QCD

SIMONE DEVOTO 111Work done in collaboration with Stefano Catani, Massimiliano
Grazzini, Stefan Kallweit, Javier Mazzitelli and Chiara Savoini. Dipartimento
di Fisica Aldo Pontremoli, Università di Milano and INFN, Sezione di Milano,
Via Celoria 16, I-20133 Milano, Italy

###### Abstract

In this contribution we present our recent computation of the NNLO QCD
corrections to the production of a Higgs boson associated with a top-antitop
quark pair. This process is of great importance since it allows for a direct
measurement of the top-quark Yukawa coupling, and the inclusion of NNLO
corrections is crucial in order to provide theory predictions with an
uncertainty competitive with the projected accuracy of the experimental
measurements at the end of the high-luminosity phase of the LHC.

##  1 Introduction

The discovery of the Higgs boson (H) in 2012 [1, 2] confirmed one of the most
glaring predictions of the Standard Model. Since then, studying its properties
has been one of the priorities of the LHC scientific programme. In this
context, its production associated with a pair of top quarks ($t\bar{t}$)
plays a special role because of the strong coupling between the Higgs boson
and the top quarks. Indeed, this same process allows for a direct measurement
of the top quark Yukawa coupling.

The current theoretical uncertainties of the predictions for the $t\bar{t}H$
process are of ${\cal O}(10\%)$ [3], and include the computation of next-to-
leading order (NLO) QCD corrections [4, 5, 6, 7, 8, 9] and NLO EW corrections
[10, 11, 12].

Beyond these on-shell calculations, the full off-shell process including
leptonic top decays was studied at NLO QCD [13, 14] and NLO EW [15], and soft-
gluon contributions close to the partonic kinematical threshold were resummed
[16, 17, 18, 19, 20, 21]. As a first step towards next-to-next-to-leading
order (NNLO) QCD corrections, the contribution coming from off-diagonal
channels has also been computed [22].

On the experimental side, the $t\bar{t}H$ production signal has been measured
by both the ATLAS and CMS collaborations with an uncertainty of ${\cal
O}(20\%)$ [23, 24], but the projection for the end of the High-Luminosity
phase indicates an uncertainty at the ${\cal O}(2\%)$ level [25].

In this proceeding we report on our recent computation of the full NNLO QCD
corrections for on-shell $t\bar{t}H$ production [26], predictions that are
required in order to match, on the theoretical side, the extraordinary
precision expected for the future measurements. In order to estimate the
unknown two-loop amplitudes for $t\bar{t}H$ production, we developed a soft-
Higgs approximation, which will be discussed in the following.

##  2 Challenges of the computation

Our computation of the NNLO corrections to the $t\bar{t}H$ process
$c(p_{1})+{\bar{c}}(p_{2})\to t(p_{3})+{\bar{t}}(p_{4})+H(k)$
($c=q,{\bar{q}},g$), has been performed by using the $q_{T}$-subtraction
formalism [27] to handle and cancel singularities of infrared origin arising
at intermediate steps of the computation.

Within the $q_{T}$-subtraction formalism, we can write the the cross section
$d\sigma^{t\bar{t}H}$ as follows:

$d\sigma^{t\bar{t}H}_{NNLO}=\mathcal{H}^{t\bar{t}H}_{NNLO}\otimes
d\sigma_{LO}^{t\bar{t}H}+\left[d\sigma_{NLO}^{t\bar{t}H+jet}-d\sigma^{CT}_{NNLO}\right]\;.$
(1)

The term in the square brackets in Eq. (1) represents the contribution to the
cross section with $q_{T}\neq 0$, $q_{T}$ being the transverse momentum of the
final state system. As such, it is captured by the NLO cross section for the
process $t\bar{t}H+$jet and can be computed with known NLO subtraction
techniques[28]. The counterterm $d\sigma^{CT}_{NNLO}$ cancels additional
singularities of pure NNLO type associated with the limit $q_{T}\to 0$.

The contribution with exactly $q_{T}=0$ is provided by the coefficient
$\mathcal{H}^{t\bar{t}H}_{NNLO}$:

$\mathcal{H}^{t\bar{t}H}_{NNLO}=H^{(2)}_{t\bar{t}H}\delta(1-z_{1})\delta(1-z_{2})+\delta\mathcal{H}^{(2)}_{t\bar{t}H}\;,$
(2)

which contains the main challenges that needed to be overcome in order to
generalise the $q_{T}$-subtraction formalism for this process.

The factor $\delta\mathcal{H}^{(2)}_{t\bar{t}H}$ contains the known one-loop
squared contribution and the soft parton contribution. The latter has been
computed in the case of heavy-quark pair production [29] by assuming back-to-
back kinematics for the massive quarks, and allowed us to apply
$q_{T}$-subtraction to the case of top-pair production [30, 31] and bottom-
pair production [32]. In order to extend these results to the current class of
processes, such a constraint had to be lifted: the extra contributions have
been computed numerically and their on-the-fly evaluation has been implemented
in a dedicated library [33].

The coefficient $H^{(2)}_{t\bar{t}H}$ contains the genuine virtual
contribution, in the form of the two-loop amplitude ${\cal
M}^{(2)}_{t{\bar{t}}H}$:

$H_{t\bar{t}H}^{(2)}=\frac{2\,Re\big{(}\mathcal{M}_{t\bar{t}H}^{(2)}(\mu_{IR},\mu_{R})\mathcal{M}_{t\bar{t}H}^{(0)}\big{)}}{|\mathcal{M}_{t\bar{t}H}^{(0)}|^{2}}\;,$
(3)

where $\mu_{IR}$ is the scale at which the infrared poles are subtracted and
$\mu_{R}$ is the renormalisation scale. Since $\mathcal{M}_{t\bar{t}H}^{(2)}$
is not yet known, we had to estimate it via a suitable approximation.

In the limit in which the momentum of the Higgs boson $k$ is soft ($k\to 0$),
the amplitude $\mathcal{M}_{t\bar{t}H}(\\{p_{i}\\},k)$ fulfils:

$\mathcal{M}_{t\bar{t}H}(\\{p_{i}\\},k)\simeq
F(\alpha_{S}(\mu_{R});m_{t}/\mu_{R})\,\frac{m_{t}}{v}\sum_{i=3,4}\frac{m_{t}}{p_{i}\cdot
k}\,\mathcal{M}_{t\bar{t}}(\\{p_{i}\\})\,,$ (4)

where $v=(\sqrt{2}G_{F})^{-1/2}$, and $\mathcal{M}_{t\bar{t}}(\\{p_{i}\\})$ is
the amplitude in which the Higgs boson has been removed, i.e. the amplitude
for $t\bar{t}$ production, available up to two-loop level [34]. The formula in
Eq. (4) can be derived by using the eikonal approximation and low-energy
theorems[35]. The function $F(\alpha_{S}(\mu_{R});m_{t}/\mu_{R})$ is the soft
limit of the scalar form factor of the heavy-quark [36, 37].

In order to validate our approximation, we first test it at NLO, considering
the contribution of the hard coefficient $H^{(1)}_{t\bar{t}H}$ to the NLO
cross section and comparing the exact result with the approximated one. As it
is shown in Table 1, the deviation with respect to the exact computation is
about $30\%$ for the $gg$ channel and $5\%$ for the $q\bar{q}$ channel. The
better agreement for the $q\bar{q}$ channel can be explained by the presence
of additional diagrams where a Higgs boson is radiated from a virtual top,
both at LO and NLO, in the $gg$ channel. Since the approximation captures the
leading behaviour in the soft limit $k\to 0$, the effect of the emission from
highly off-shell top propagators as the ones in these families of diagrams is
not correctly reproduced.

| $\sqrt{s}=13\,\mathrm{TeV}$ | $\sqrt{s}=100\,\mathrm{TeV}$
---|---|---
| $gg$ | $q{\bar{q}}$ | $gg$ | $q{\bar{q}}$
$\Delta\sigma_{\rm NLO,H}$ [fb] | $\phantom{-0}88.62$ | $\phantom{00}7.826$ | $\phantom{-0}8205$ | $\phantom{0}217.0$
$\Delta\sigma_{\rm NLO,H}|_{\rm soft}$ [fb] | $\phantom{-0}61.98$ | $\phantom{00}7.413$ | $\phantom{-0}5612$ | $\phantom{0}206.0$
Difference | $\phantom{00}30.1\%$ | $\phantom{00}5.27\%$ | $\phantom{00}31.6\%$ | $\phantom{00}5.06\%$
Table 1: Hard contribution to the NLO cross sections in the soft
approximation.

Having checked the behaviour of the approximation at NLO, we can employ it for
the computation of the NNLO corrections. We find a small contribution from the
(approximated) NNLO hard coefficient $H^{(2)}_{t\bar{t}H}$: it corresponds to
$1\%$ of the LO cross-section in the $gg$ channel and to $2\%$ of the LO
cross-section in the $q\bar{q}$ channel. To estimate the uncertainties due to
the approximation at NNLO, we use the results in Table 1: we consider the
deviation from the exact results at NLO as a lower bound on the NNLO
uncertainty and we multiply it by a tolerance factor of 3, finally combining
linearly the uncertainties for the $q\bar{q}$ and $gg$ channel. With this
choice, the final uncertainties on the NNLO corrections amount to $\pm 15\%$,
which corresponds to $\pm 0.6\%$ on the total cross section.

In order to test if such estimation of the uncertainties is reliable, we can
compare it with the effect given by the variation of (unphysical) parameters
of the approximation. A first check that can be performed is to use different
recoil prescriptions. In order to apply the approximation, there is the need
to map the $2\to 3$ kinematics of $t\bar{t}H$ production to the $2\to 2$
kinematics of $t\bar{t}$ production. In our computation, we use the $q_{T}$
recoil prescription [38], reabsorbing the Higgs momentum equally in the
initial-state parton momenta and leaving unchanged the top and anti-top
momenta. We verified that the difference in the result obtained by using
different prescriptions, for example by reabsorbing the transverse momentum of
the Higgs boson entirely into one of the initial state momenta, is negligible
when compared with the provided uncertainties. Another possible check is to
vary the scale $\mu_{IR}$ at which the infrared singularities are subtracted,
which in our computation we fix at the virtuality of the $t\bar{t}H$ system
$M$. In an exact computation, the dependence on this parameter would cancel
between $H^{(2)}_{t\bar{t}H}$ and $\delta\mathcal{H}^{(2)}_{t\bar{t}H}$ in Eq.
(2). By repeating our calculation with $\mu_{IR}=2M$ and $\mu_{IR}=M/2$ we
verified that the numerical result for the hard virtual contribution changes
by ${}^{+164\%}_{-25\%}$ (${}^{+142\%}_{-20\%}$) in the $gg$ channel and by
${}^{+4\%}_{-0\%}$ (${}^{+3\%}_{-0\%}$) in the $q{\bar{q}}$ channel at
$\sqrt{s}=$ 13(100) TeV, an amount comparable with the uncertainties that have
been estimated.

##  3 Results

Our results for the inclusive cross section, obtained by implementing the
$t\bar{t}H$ process within the Matrix framework [39], are shown in Table 2
(see Ref. [26]), together with the customary 7-point scale variation. For the
NNLO predictions, we quote in brackets the combination of the numerical errors
and the soft-approximation uncertainty described in the previous section: it
can be observed that they turn out to be significantly smaller than remaining
perturbative uncertainties.

The inclusion of NNLO corrections leads to an increase of $+4\%$ at centre-of-
mass energy $\sqrt{s}=13$ TeV and of $+2\%$ at $\sqrt{s}=100$ TeV. In both
cases, they also significantly reduce the scale-variation bands, which are at
the few percent level.

In Figure 1 we also present some preliminary results for the transverse-
momentum distribution of the Higgs boson. In the upper plot we show our LO
(grey), NLO (red) and NNLO (blue) predictions, with their scale variation
bands, while in the lower plot the ratio to the NLO prediction is presented.
The smaller dark blue band in the NNLO predictions shows the soft-
approximation uncertainties, computed as described above and on a bin-by-bin
basis. We can observe that the NLO and NNLO uncertainty bands overlap,
providing first signs of perturbative convergence. Despite the fact that the
soft approximation is expected to be less accurate at high $p_{T,H}$ values,
the uncertainties remain of the same order over all the spectrum. This can be
understood from the fact that at high $p_{T,H}$ the role of the $gg$ channel
is reduced and the $q\bar{q}$ channel, which is under better control, plays
the major role.

$\sigma$ [pb] | $\sqrt{s}=13\,\mathrm{TeV}$ | $\sqrt{s}=100\,\mathrm{TeV}$
---|---|---
$\sigma_{\rm LO}$ | $0.3910\,^{+31.3\%}_{-22.2\%}$ | $25.38\,^{+21.1\%}_{-16.0\%}$
$\sigma_{\rm NLO}$ | $0.4875\,^{+5.6\%}_{-9.1\%}$ | $36.43\,^{+9.4\%}_{-8.7\%}$
$\sigma_{\rm NNLO}$ | $0.5070\,(31)^{+0.9\%}_{-3.0\%}$ | $37.20(25)\,^{+0.1\%}_{-2.2\%}$
Table 2:  LO, NLO and NNLO cross sections at $\sqrt{s}=13$ TeV and
$\sqrt{s}=100$ TeV. The errors stated in brackets at NNLO combine numerical
errors with the uncertainty due to the soft Higgs boson approximation.

Figure 1: Single-differential cross sections as a function of $p_{T,H}$
(preliminary).

##  4 Conclusions

In this contribution we reported on our recent computation of the NNLO QCD
corrections to $t\bar{t}H$ production at hadron colliders. Our computation is
exact, with the exception of the contribution from the two-loop amplitudes
which have been estimated by using a soft-Higgs approximation. The uncertainty
due to such approximation is estimated to be smaller than $1\%$ of the
inclusive result, and well below the remaining perturbative uncertainties. The
inclusion of NNLO QCD corrections lead to an increase of the cross section of
a few percent, and to a significant reduction of the scale-variation bands. We
have also presented preliminary results for the transverse momentum
distribution of the Higgs boson.

## References

## References

  * [1] ATLAS Collaboration.  Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC.  Phys. Lett., B716:1–29, 2013.
  * [2] CMS Collaboration.  Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC.  Phys. Lett., B716:30–61, 2013.
  * [3] D. de Florian et al.  Handbook of LHC Higgs Cross Sections: 4. Deciphering the Nature of the Higgs Sector.  2/2017, 10 2016.
  * [4] W. Beenakker, S. Dittmaier, M. Krämer, B. Plumper, M. Spira, and P.M. Zerwas.  Higgs radiation off top quarks at the Tevatron and the LHC.  Phys. Rev. Lett., 87:201805, 2001.
  * [5] W. Beenakker, S. Dittmaier, M. Krämer, B. Plumper, M. Spira, and P.M. Zerwas.  NLO QCD corrections to t anti-t H production in hadron collisions.  Nucl. Phys. B, 653:151–203, 2003.
  * [6] L. Reina and S. Dawson.  Next-to-leading order results for t anti-t h production at the Tevatron.  Phys. Rev. Lett., 87:201804, 2001.
  * [7] L. Reina, S. Dawson, and D. Wackeroth.  QCD corrections to associated t anti-t h production at the Tevatron.  Phys. Rev. D, 65:053017, 2002.
  * [8] S. Dawson, L.H. Orr, L. Reina, and D. Wackeroth.  Associated top quark Higgs boson production at the LHC.  Phys. Rev. D, 67:071503, 2003.
  * [9] S. Dawson, C. Jackson, L.H. Orr, L. Reina, and D. Wackeroth.  Associated Higgs production with top quarks at the large hadron collider: NLO QCD corrections.  Phys. Rev. D, 68:034022, 2003.
  * [10] S. Frixione, V. Hirschi, D. Pagani, H.S. Shao, and M. Zaro.  Weak corrections to Higgs hadroproduction in association with a top-quark pair.  JHEP, 09:065, 2014.
  * [11] Yu Zhang, Wen-Gan Ma, Ren-You Zhang, Chong Chen, and Lei Guo.  QCD NLO and EW NLO corrections to $t\bar{t}H$ production with top quark decays at hadron collider.  Phys. Lett. B, 738:1–5, 2014.
  * [12] S. Frixione, V. Hirschi, D. Pagani, H. S. Shao, and M. Zaro.  Electroweak and QCD corrections to top-pair hadroproduction in association with heavy bosons.  JHEP, 06:184, 2015.
  * [13] Ansgar Denner and Robert Feger.  NLO QCD corrections to off-shell top-antitop production with leptonic decays in association with a Higgs boson at the LHC.  JHEP, 11:209, 2015.
  * [14] Daniel Stremmer and Malgorzata Worek.  Production and decay of the Higgs boson in association with top quarks.  JHEP, 02:196, 2022.
  * [15] Ansgar Denner, Jean-Nicolas Lang, Mathieu Pellen, and Sandro Uccirati.  Higgs production in association with off-shell top-antitop pairs at NLO EW and QCD at the LHC.  JHEP, 02:053, 2017.
  * [16] Anna Kulesza, Leszek Motyka, Tomasz Stebel, and Vincent Theeuwes.  Soft gluon resummation for associated $t\bar{t}H$ production at the LHC.  JHEP, 03:065, 2016.
  * [17] Alessandro Broggio, Andrea Ferroglia, Ben D. Pecjak, Adrian Signer, and Li Lin Yang.  Associated production of a top pair and a Higgs boson beyond NLO.  JHEP, 03:124, 2016.
  * [18] Alessandro Broggio, Andrea Ferroglia, Ben D. Pecjak, and Li Lin Yang.  NNLL resummation for the associated production of a top pair and a Higgs boson at the LHC.  JHEP, 02:126, 2017.
  * [19] Anna Kulesza, Leszek Motyka, Tomasz Stebel, and Vincent Theeuwes.  Associated $t\bar{t}H$ production at the LHC: Theoretical predictions at NLO+NNLL accuracy.  Phys. Rev. D, 97(11):114007, 2018.
  * [20] Alessandro Broggio, Andrea Ferroglia, Rikkert Frederix, Davide Pagani, Benjamin D. Pecjak, and Ioannis Tsinikos.  Top-quark pair hadroproduction in association with a heavy boson at NLO+NNLL including EW corrections.  JHEP, 08:039, 2019.
  * [21] Anna Kulesza, Leszek Motyka, Daniel Schwartländer, Tomasz Stebel, and Vincent Theeuwes.  Associated top quark pair production with a heavy boson: differential cross sections at NLO+NNLL accuracy.  Eur. Phys. J. C, 80(5):428, 2020.
  * [22] Stefano Catani, Ignacio Fabre, Massimiliano Grazzini, and Stefan Kallweit.  ${t{{\bar{t}}}H}$ production at NNLO: the flavour off-diagonal channels.  Eur. Phys. J. C, 81(6):491, 2021.
  * [23] M. Aaboud et al.  Observation of Higgs boson production in association with a top quark pair at the LHC with the ATLAS detector.  Phys. Lett. B, 784:173–191, 2018.
  * [24] Albert M Sirunyan et al.  Observation of $\mathrm{t\overline{t}}$H production.  Phys. Rev. Lett., 120(23):231801, 2018.
  * [25] M. Cepeda et al.  Report from Working Group 2: Higgs Physics at the HL-LHC and HE-LHC.  CERN Yellow Rep. Monogr., 7:221–584, 2019.
  * [26] Stefano Catani, Simone Devoto, Massimiliano Grazzini, Stefan Kallweit, Javier Mazzitelli, and Chiara Savoini.  Higgs Boson Production in Association with a Top-Antitop Quark Pair in Next-to-Next-to-Leading Order QCD.  Phys. Rev. Lett., 130(11):111902, 2023.
  * [27] Stefano Catani and Massimiliano Grazzini.  An NNLO subtraction formalism in hadron collisions and its application to Higgs boson production at the LHC.  Phys. Rev. Lett., 98:222002, 2007.
  * [28] S. Catani and M. H. Seymour.  A General algorithm for calculating jet cross-sections in NLO QCD.  Nucl. Phys., B485:291–419, 1997.  [Erratum: Nucl. Phys. B510, 503 (1998)].
  * [29] Stefano Catani, Simone Devoto, Massimiliano Grazzini, and Javier Mazzitelli.  Soft-parton contributions to heavy-quark production at low transverse momentum.  JHEP, 04:144, 2023.
  * [30] Stefano Catani, Simone Devoto, Massimiliano Grazzini, Stefan Kallweit, Javier Mazzitelli, and Hayk Sargsyan.  Top-quark pair hadroproduction at next-to-next-to-leading order in QCD.  Phys. Rev., D99(5):051501, 2019.
  * [31] Stefano Catani, Simone Devoto, Massimiliano Grazzini, Stefan Kallweit, and Javier Mazzitelli.  Top-quark pair production at the LHC: Fully differential QCD predictions at NNLO.  JHEP, 07:100, 2019.
  * [32] Stefano Catani, Simone Devoto, Massimiliano Grazzini, Stefan Kallweit, and Javier Mazzitelli.  Bottom-quark production at hadron colliders: fully differential predictions in NNLO QCD.  JHEP, 03:029, 2021.
  * [33] Simone Devoto, Javier Mazzitelli, in preparation.
  * [34] P. Bärnreuther, M. Czakon, and P. Fiedler.  Virtual amplitudes and threshold behaviour of hadronic top-quark pair-production cross sections.  JHEP, 02:078, 2014.
  * [35] Bernd A. Kniehl and Michael Spira.  Low-energy theorems in Higgs physics.  Z. Phys. C, 69:77–88, 1995.
  * [36] W. Bernreuther, R. Bonciani, T. Gehrmann, R. Heinesch, P. Mastrolia, and E. Remiddi.  Decays of scalar and pseudoscalar Higgs bosons into fermions: Two-loop QCD corrections to the Higgs-quark-antiquark amplitude.  Phys. Rev. D, 72:096002, 2005.
  * [37] J. Ablinger, A. Behring, J. Blümlein, G. Falcioni, A. De Freitas, P. Marquard, N. Rana, and C. Schneider.  Heavy quark form factors at two loops.  Phys. Rev. D, 97(9):094022, 2018.
  * [38] Stefano Catani, Daniel de Florian, Giancarlo Ferrera, and Massimiliano Grazzini.  Vector boson production at hadron colliders: transverse-momentum resummation and leptonic decay.  JHEP, 12:047, 2015.
  * [39] Massimiliano Grazzini, Stefan Kallweit, and Marius Wiesemann.  Fully differential NNLO computations with MATRIX.  Eur. Phys. J. C, 78(7):537, 2018.


# Experimental quantum-enhanced kernels on a photonic processor

Zhenghao Yin<EMAIL_ADDRESS>University of Vienna, Faculty of
Physics, Vienna Center for Quantum Science and Technology (VCQ),
Boltzmanngasse 5, Vienna A-1090, Austria  University of Vienna, Faculty of
Physics, Vienna Doctoral School of Physics (VDSP), Boltzmanngasse 5, Vienna
A-1090, Austria  Iris Agresti<EMAIL_ADDRESS>University of Vienna,
Faculty of Physics, Vienna Center for Quantum Science and Technology (VCQ),
Boltzmanngasse 5, Vienna A-1090, Austria  Giovanni de Felice  Quantinuum, 17
Beaumont Street, Oxford OX1 2NA, UK  Douglas Brown  Quantinuum, 17 Beaumont
Street, Oxford OX1 2NA, UK  Alexis Toumi  Quantinuum, 17 Beaumont Street,
Oxford OX1 2NA, UK  Ciro Pentangelo  Dipartimento di Fisica, Politecnico di
Milano, piazza L. Da Vinci 32, 20133 Milano, Italy  Istituto di Fotonica e
Nanotecnologie, Consiglio Nazionale delle Ricerche (IFN-CNR), piazza L. Da
Vinci 32, 20133 Milano, Italy  Simone Piacentini  Istituto di Fotonica e
Nanotecnologie, Consiglio Nazionale delle Ricerche (IFN-CNR), piazza L. Da
Vinci 32, 20133 Milano, Italy  Andrea Crespi  Dipartimento di Fisica,
Politecnico di Milano, piazza L. Da Vinci 32, 20133 Milano, Italy  Istituto di
Fotonica e Nanotecnologie, Consiglio Nazionale delle Ricerche (IFN-CNR),
piazza L. Da Vinci 32, 20133 Milano, Italy  Francesco Ceccarelli  Istituto di
Fotonica e Nanotecnologie, Consiglio Nazionale delle Ricerche (IFN-CNR),
piazza L. Da Vinci 32, 20133 Milano, Italy  Roberto Osellame  Istituto di
Fotonica e Nanotecnologie, Consiglio Nazionale delle Ricerche (IFN-CNR),
piazza L. Da Vinci 32, 20133 Milano, Italy  Bob Coecke  Quantinuum, 17
Beaumont Street, Oxford OX1 2NA, UK  Philip Walther
<EMAIL_ADDRESS>University of Vienna, Faculty of Physics, Vienna
Center for Quantum Science and Technology (VCQ), Boltzmanngasse 5, Vienna
A-1090, Austria  Christian Doppler Laboratory for Photonic Quantum Computer,
Faculty of Physics, University of Vienna, 1090 Vienna, Austria

###### Abstract

Recently, machine learning had a remarkable impact, from scientific to
everyday-life applications. However, complex tasks often imply unfeasible
energy and computational power consumption. Quantum computation might lower
such requirements, although it is unclear whether enhancements are reachable
by current technologies. Here, we demonstrate a kernel method on a photonic
integrated processor to perform a binary classification. We show that our
protocol outperforms state-of-the-art kernel methods including gaussian and
neural tangent kernels, exploiting quantum interference, and brings a smaller
improvement also by single photon coherence. Our scheme does not require
entangling gates and can modify the system dimension through additional modes
and injected photons. This result opens to more efficient algorithms and to
formulating tasks where quantum effects improve standard methods.

quantum optics, quantum machine learning

## Introduction

The past decades have witnessed a swift development of technologies based on
quantum mechanical phenomena, which have opened up new perspectives in a wide
spectrum of applications. These range from the realization of a global-scale
quantum communication network, the Quantum Internet [1, 2], to the simulation
of quantum systems [3], to quantum computing [4]. In particular, the interest
towards the last field has been fueled by some milestone discoveries, such as
Shor’s factorization and Grover’s search algorithm [5, 6], which have promised
that quantum processors can outperform their classical counterparts. However,
a clear advantage of quantum computation has been experimentally demonstrated
only recently and on different computational tasks, boson sampling [7, 8, 9,
10, 11, 12] and random circuit sampling [13], which do not have clear
practical applications.

Figure 1: Photonic quantum kernel estimation. a. The photonic quantum kernel
maps each data point $x_{i}$ to be classified from a $d$-dimensional space
into a quantum state $|\Phi\rangle_{i}$, living in a Hilbert feature space. In
detail, the classical data $x_{i}$ is encoded into a unitary evolution
$U(x_{i})$ applied on a fixed input state $\ket{\psi}$. This implies
$|\Phi\rangle_{i}=U(x_{i})|\psi\rangle$. After mapping all the data points in
the dataset, from the inner pairwise products, we perform the classification
finding the hyperplane best separating the classes, i.e. through a support
vector machine (SVM), according to Eq. (1). b. Pairs of indistinguishable
photons and distinguishable photons show a different behaviour when injected
in a Mach-Zehnder interferometer (MZI). Here, input states $11$ and
$11^{\prime}$ indicate, respectively, two indistinguishable and
distinguishable photons being injected in the circuit and being detected at
the output modes. c. Estimation of the inner product of two data points
$x_{i}$ and $x_{j}$ by encoding them in two unitaries $U(x_{i})$ and
$U(x_{j})$. The inner product $\langle\phi_{j}|\phi_{i}\rangle$ amounts to
$\langle\psi|U^{\dagger}(x_{i})U(x_{j})|\psi\rangle$. This is equivalent to
projecting the evolved state $U^{\dagger}(x_{j})U(x_{i})|\psi\rangle$ onto
$|\psi\rangle$. Each box represents a programmable MZI with two free
parameters (namely a beam splitter with tunable reflectivity and phase), as
shown in b.

Given these premises, our goal is to investigate the tasks in which quantum
computing can enhance the operation of classical computers for practically
relevant tasks. Moreover, the question is whether this can be achieved for
problems that are now within the reach of state-of-art technology, where only
noisy intermediate-scale quantum computers are available [14, 15]. In this
context, a flurry of interest has been devoted to the open question of whether
the new paradigm of quantum computing can have an impact on machine learning
[16, 17, 18], which has revolutionized classical computation, granting new
possibilities and changing our everyday lives, from email filtering to
artificial intelligence. The two main directions that have been investigated
until now are, on one side, whether quantum computation could improve the
efficiency of the learning process, allowing us to find better optima with the
need of a lower number of inquiries [19, 20, 21, 22] and, on the other, how
quantum behaviours can enhance the expressivity of the input encoding,
exploiting correlations between variables that are hard to reproduce through
classical computation [23, 24].

In this context, a straightforward application of quantum computing on kernel
models has become evident. Kernel methods are widely used tools in machine
learning [25, 26], that base their functioning on the fact that patterns for
data points, which are hard to recognize in their original space, can become
easy to identify once nonlinearly mapped to a feature space. Once the suitable
mapping is performed, it is possible to identify the hyperplane which best
separates the classes of feature data points, through a support vector machine
[27] (SVM), according to the inner product of the mapped data. Let us note
that the only part of the model that is trained is the SVM, which is
efficient, once the inner products are available. Hence, an interesting
question is whether using a quantum apparatus to perform the data mapping and
evaluate the inner products can enhance the performance, benefitting from the
quantum feature maps of the evolution of quantum systems and outsourcing the
hardest part of the computation to the quantum hardware. This question was
theoretically answered in the affirmative by [28], although the implementation
of the proposed task is out of reach for state-of-the-art technologies.
Moreover, a risk that one encounters in quantum kernel estimation is that,
once the feature space is too large, points are mapped into orthogonal states,
resulting in an ineffective classification. Hence, a moderately-sized quantum
feature space can prove more suitable, to preserve the similarity among data
belonging to the same class.

Figure 2: Classification tasks for photonic kernel methods. The datasets are
randomly generated and consist in $d$\- dimensional vectors, with entries
between 0 and 1. Then, we randomly assign labels to each point as belonging to
class +1 or -1 and we test the ability of our photonic kernels, displaying and
not displaying quantum interference (respectively indicated as quantum kernel
and coherent kernel), to correctly classify the data. This is quantified by
the accuracy of our models, which we indicate as $a_{Q}$ and $a_{C}$.

In this work, we experimentally demonstrate a quantum kernel estimation, where
feature data points are evaluated through the unitary evolution of two-boson
Fock states (see Fig. 1). Such encoding, even for relatively small dimensions,
provides enough of a non-linearity to achieve high classification accuracies.
Furthermore, we show that for given tasks, this algorithm leads to an
enhancement in the performance of quantum kernels with respect to their
classical counterparts. These tasks are selected by maximizing the so-called
geometric difference, which measures the separation in performance between a
pair of kernels [29]. In particular, we separate between quantum and coherent
kernels, that is, photonic kernels that do or do not exhibit quantum
interference.

To experimentally demonstrate this method, we exploit a photonic platform
based on an integrated photonic processor [30] where we inject two-boson Fock
states to map the data to be classified (see Fig. 1a). To estimate quantum and
coherent kernels, we inject indistinguishable and distinguishable photons,
respectively. This photonic platform is particularly suitable for this task,
as it allows us to encode and manipulate our input data with high fidelity.

To benchmark our enhanced performance, we compare classification accuracies
between photonic kernels and state-of-the-art classical computational kernels.
These include the standard gaussian kernels [25, 26], as well as the recently
introduced neural tangent kernels [31], which simulate gradient descent over
infinitely-wide neural networks. Our results show that photonic kernels
outperform classical methods and that the accuracies are further enhanced in
kernels displaying quantum interference.

## Photonic quantum kernel estimation

A kernel method relies on a function that maps $N$ input data points $x_{i}$,
on which we wish to perform binary classification, from a space
$\mathcal{X}\subseteq\mathbb{R}^{d}$ into a feature space $\mathcal{H}$. Here,
$d$ is the dimension of each data point. This is done through a feature map
$\Phi:\mathcal{X}\rightarrow\mathcal{H}$. Then, a SVM can be used to produce a
_prediction_ function $f_{K}:\mathcal{X}\to\mathbb{R}$ as
$f_{K}(x)=\sum_{i}\alpha_{i}K(x,x_{i})$, where these $\alpha_{i}$ coefficients
are obtained by solving a linear optimization problem. The inputs of the
optimization are the labels $y$ and the matrix obtained by computing the
pairwise distances between data points is
$K_{i,j}=K(x_{i},x_{j})=\absolutevalue{\innerproduct{\Phi(x_{i})}{\Phi(x_{j})}}^{2}$,
the so-called _Gram matrix_ (see Supplementary Note 1 for further
information).

In this work, we implement a quantum version of the kernel method, in which
the aforementioned pairwise distances between data points, which belong to a
class $y$ taking values $+1$ or $-1$, are estimated by sampling from the
output probability distribution arising from the unitary evolution of a Fock
input state. This process is depicted in Fig. 1a. Therefore, our feature map
plugs the data that needs to be classified into the free parameters defining a
unitary evolution applied to a fixed Fock state of dimension $m$ and whose sum
of occupational numbers is $n$: $x\mapsto|\Phi(x)\rangle=U_{x}|\psi\rangle$.
Here, $\ket{\psi}$ is the encoding state which is free to choose. Then, as
shown in Fig. 1c, the pairwise inner products of the feature points are
experimentally evaluated, as
$|\langle\psi|U(x_{i})^{\dagger}U(x_{j})|\psi\rangle|^{2}$. Such unitaries can
be effectively implemented by a programmable photonic circuit consisting of an
array of Mach-Zehnder interferometers (MZIs)[32]. Hence, the dimension of the
feature Hilbert space $\mathcal{H}$ will be $\binom{n+m-1}{n}$. At this point,
the SVM finds the hyperplane separating the training data points through the
aforementioned optimization process [33, 34] and, the binary classification of
unknown points $x$ is given by the following relation:

$y=\mathrm{sign}\left(\sum_{i=1}^{N}\alpha_{i}y_{i}K(x,x_{i})\right)$ (1)

where $\alpha_{i}$ are the coefficients optimized in the training process and
$y_{i}$ is the class of the i-th point in the training. This model is defined
implicitly, as the labels are assigned by weighted inner products of the
encoded data points [35, 36, 37, 38, 39, 40, 41, 42, 43].

Figure 3: Implementation of photonic quantum kernel estimation. a.
Experimental setup consisting of two parts, the off-chip single photon source
and the programmable integrated photonic processor. The frequency degenerate
photons are generated by a type-II spontaneous parametric down-conversion
source. Afterwards, the two photons are made indistinguishable in their
polarization and arrival time. Then, we inject these photons in two modes of
an integrated photonic processor with six input/output modes [30]. Detection
is performed by superconducting nanowire single-photon detectors (SNSPDs). The
degree of indistinguishability can then be tuned through a delay line,
changing their relative temporal delay. b. Probability distribution of photon
detection events. We show two instances of the experimental photon detection
probability, compared to the theoretical calculation. The quantum and coherent
kernel measurements are obtained respectively by injecting two
indistinguishable and distinguishable photons into the third and fourth modes
of the circuits, i.e. $|0,0,1,1,0,0\rangle$. The x axis shows all the circuit
channels which output two photons simultaneously. Thus, all 15 possible photon
detection configurations are accessible.

If the Fock state contains indistinguishable bosons, they will exhibit quantum
interference, as shown in Fig. 1b. In this case, the output probability
distribution is given by the permanents of sub-matrices of the matrix
representing the unitary evolution of the input [44]. More specifically,
considering an input configuration $s$, the probability of detecting the
output configuration $t$ is given by
${|\mathrm{Per}{U}_{s,t}|^{2}}/{\Pi_{i}^{m}s_{i}!\Pi_{i}^{m}t_{i}!}$. Here,
$\mathrm{Per}(\cdot)$ denotes the permanent matrix operation, $s_{i}$ and
$t_{i}$ are the occupational numbers at the $i$-th mode and $U_{s,t}$ is the
sub-matrix obtained by selecting the rows/columns corresponding to the
occupied modes of the input/output Fock states. On the other hand, if the
bosons are distinguishable, they will not exhibit quantum interference. In
this case the probability will amount to
${\mathrm{Per}|U_{s,t}|^{2}}/{\Pi_{i}^{m}s_{i}!\Pi_{i}^{m}t_{i}!}$.

In the following, we will refer to a kernel implemented with indistinguishable
bosons as a quantum kernel,

$K_{Q}(x_{i},x_{j})=|\mathrm{Per}U_{\psi}(x_{i},x_{j})|^{2}/N^{\prime}$ (2)

and with distinguishable ones as a coherent kernel,

$K_{C}(x_{i},x_{j})=\mathrm{Per}|U_{\psi}(x_{i},x_{j})|^{2}/N^{\prime}$ (3)

Here, $U_{\psi}(x_{i},x_{j})$ is the matrix defined by data points
$x_{i},x_{j}$ and selected encoding state $\psi$. $N^{\prime}$ is the
coefficiency related to $\psi$. As long as there is at most one photon in each
mode of $\psi$, $N^{\prime}=1$.

## Classification task

To select a classification task that would benefit from the described model,
we use a quantifier called the geometric difference [29]. Given a set of data
points $\\{{x_{i}}|{x_{i}}\in\mathcal{X}\\}$ without any labels, the geometric
difference provides the binary labels $\\{y_{i}\\}$ that maximise the expected
difference in prediction error between two kernels, $K_{Q}$ and $K_{C}$, as
depicted in Fig. 2. Therefore, we obtain this optimal labelling by solving the
following minimisation problem:

$y^{\star}=\mathrm{arg}~{}\min\limits_{y\in\mathbb{R}^{d}}\left(\frac{s_{K_{Q}}(y)}{s_{K_{C}}(y)}\right)$
(4)

where $s_{K}(y)=y^{T}Ky$ is the model complexity of the pair $K$ and $y$, i.e.
the number of features that the model needs to make accurate predictions (see
the Supplementary Note 2). To saturate the following inequality [29]:

$\exists y\quad\cdot\quad s_{K_{C}}(y)\quad\leq\quad g_{CQ}^{2}\,s_{K_{Q}}(y)$
(5)

we take $y^{T}K_{C}^{-1}y=g_{CQ}^{2}y^{T}K_{Q}^{-1}y$, and obtain the relation

$g_{CQ}=\sqrt{\norm{\sqrt{K_{Q}}\left(K_{C}\right)^{-1}\sqrt{K_{Q}}}_{\infty}}$
(6)

where $\norm{\cdot}_{\infty}$ denotes the spectral norm and $g_{CQ}$ denotes
geometric difference.

We can now use Eq. (6) to generate the classification task that, given two
kernels $K_{Q},K_{C}$ and a set of data points $\\{x_{i}\\}$, produces the
labels $\\{y_{i}\\}$ that maximise the difference in prediction error bound.
This can be done through the following procedure: (i) evaluate the Gram
matrices $K_{Q}$ and $K_{C}$ over a set of non-labelled data points
$\\{x_{i}\\}$; (ii) compute the positive definite matrix
$M=\sqrt{K_{Q}}\left(K_{C}\right)^{-1}\sqrt{K_{Q}}$; (iii) compute the
eigenvalues and eigenvectors of $M$ by spectral decomposition; (iv) find the
maximum eigenvalue $g$ and its corresponding eigenvector $v$; (v) assign the
labels $y=\sqrt{K_{Q}}v$. From a practical point of view, we start with the
two aforementioned kernels, $K_{C}$ and $K_{Q}$, and then, by maximizing the
geometric difference, we find the tasks for which the latter brings an
enhanced accuracy of the classification. For more details regarding the
algorithm to define the classification task, see the Supplementary Note 4. Let
us note that the implemented tasks constitute instances of problems that can
be naturally implemented with high accuracy on our quantum platform. As such
they constitute a first stepping stone towards the identification of practical
tasks for which quantum machine learning can enhance the performance of
classical models.

Figure 4: Experimental classification accuracies. a-b. We tested our method on
datasets of different sizes (40, 60, 80, 100) and for two different input
states ($|1,1,0,0,0,0\rangle$ and $|0,0,1,1,0,0\rangle$) respectively. For
each dataset, $2/3$ of the data points were used for training the support
vector machine (SVM) and $1/3$ for test. c. The average classification
accuracies on 5 different sets for the quantum kernel (blue) and the coherent
(orange) kernel, along with the following other computational kernels:
gaussian (grey), ntk (green), polynomial (yellow) and linear (purple). The
dashed line indicates the results of numerical simulations, while the solid
ones are the experimental results. The error bar shows the standard deviation
of the classification accuracies on 5 datasets for all the kernels.

## Experiment

Our experimental setup consists of two parts, a single-photon source
generating the input states and a programmable integrated photonic processor
depicted in Fig. 3a. First, to generate the input state, we use a type II
spontaneous parametric down-conversion source, which generates frequency
degenerate single-photon pairs at 1546 nm in a periodically poled K-titanyl
phosphate crystal. The two photons are then made indistinguishable in their
polarization and arrival time, respectively, via wave retarders and a delay
line, which we also use to tune the degree of indistinguishability of the
generated photons.

For the implementation of photonic kernels, which map our input data to a
feature space, we require an apparatus able to perform arbitrary unitary
transformations on a given input state. As mentioned before, our feature map
sends each data point $x_{i}$ onto the state resulting from the evolution
$U(x_{i})$ of a fixed input Fock state $|\psi\rangle$. Then, for the
application of the SVM, which finds the best hyperplane separating the data,
we need to evaluate the inner products between all of the points $x_{i},x_{j}$
in the feature space, which amounts to
$\langle\psi|U(x_{i})^{\dagger}U(x_{j})|\psi\rangle$. This implies that, if we
take $|\psi\rangle$ as a Fock state of $n$ photons over $m$ modes, the inner
product $\innerproduct{\Phi(x_{i})}{\Phi(x_{j})}$ is given by projecting the
evolved state $U(x_{i})^{\dagger}U(x_{j})\ket{\psi}$ onto $\ket{\psi}$.

To this aim, we employ an integrated photonic processor [30] on a borosilicate
glass substrate, in which optical waveguides are inscribed through femtosecond
laser writing [45, 46, 47]. The circuit features six input/output modes [32],
as depicted in Fig. 3a, where each interferometer is equipped with two thermal
phase shifters [48], to provide tunable reflectivity and phase. Such
arrangement allows us to perform any unitary transformation on the input
photon states. Given this property, our device is also referred to as a
universal photonic processor. Design, fabrication and calibration of the
integrated photonic circuit are described in [30].

Specifically, the data were encoded in the values of the phase shifts, as
follows:
$x_{i}=(x_{i}^{1},x_{i}^{2},...,x_{i}^{30})\rightarrow\theta_{i}=(2\pi
x_{i}^{1},2\pi x_{i}^{2},...,2\pi x_{i}^{30})$, where $\theta_{i}$ are the
phase shifts introduced by the phase shifters of a universal interferometer.
Let us note that this encoding has the remarkable advantage that no extra
processing is required on the input data, as they are directly plugged into
the optical circuit parameters. Furthermore, in principle, we would need a
sequence of two of such circuits (as in the scheme of Fig. 1c), to first
implement $U^{\dagger}(x_{i})$ and then $U(x_{j})$ on our inputs. However, in
our implementation, we adopt only one universal circuit and directly implement
the unitary corresponding to the product $U(x_{i})^{\dagger}U(x_{j})$. This
reduces the experimental complexity and the circuit propagation losses.

At the output, detection is performed by superconducting nanowire single-
photon detectors (SNSPDs), where we post-select the output events to those
featuring two detector clicks (coincidence counts, CC), i.e. collision-free
events (see Supplementary Note 3). Thus, the elements of the Gram matrix of a
given kernel can be estimated from the coincidence counting
$K(x_{i},x_{j})=\mathrm{CC}^{ij}_{\psi}/\sum_{1\leq l<m\leq
6}\mathrm{CC}^{ij}_{lm}$. Here $\mathrm{CC}^{ij}_{lm}$ is the number of
registered coincidence counts between channels $l$ and $m$, when the
implemented unitary is $U^{\dagger}(x_{j})U(x_{i})$ and $\psi$ indicates the
occupied modes of input state $|\psi\rangle$. To test the role of quantum
interference in the accuracy of the classification, we tune the
indistinguishability of the two photons by changing their relative temporal
delay. An instance of the probability distribution of the same unitary is
shown in Fig. 3b. The optimal classification task is chosen for each data set
according to the algorithm explained in the previous section.

## Results

We test the performance of two photonic kernels in several different
configurations. Firstly, we consider two different inputs, $\ket{1,1,0,0,0,0}$
and $\ket{0,0,1,1,0,0}$. This amounts to either injecting the photons into the
first two modes or the central two modes. Second, we are able to tune the
indistinguishability to implement the quantum kernel and the coherent kernel
as aforementioned. During the whole measurement, the maximal achieved
indistinguishability between the photons is $0.9720\pm 0.0044$, estimated
through on-chip Hong-Ou-Mandel interference [49].

For both input states, we fix the encoding of each data point and vary
datasets with four different sizes: 40, 60, 80 and 100. We use the setup
depicted in Fig.3a to implement all pairwise products between the unitaries
$U(x_{i})^{\dagger}U(x_{j})$. Hence,
$|\langle\psi|U(x_{i})^{\dagger}U(x_{j})|\psi\rangle|^{2}$ is given by the
probability of detecting the photons on the same modes from which they were
injected. The rate of total post-selected coincidence counts amounts to 10 kHz
and the measured probability distribution was averaged over 5 s for each
unitary configuration.

For each size $N$, we perform $N(N-1)/2$ unitaries to compute the inner
products. The distance between the unitaries experimentally realized and the
target ones can be estimated as $\sum_{i}\sqrt{P_{i}^{\mathrm{theo}}\cdot
P_{i}^{\mathrm{exp}}}$, where $P_{i}^{\mathrm{exp}}$ is the experimental
detection frequency for the $i$-th output configuration, while
$P_{i}^{\mathrm{theo}}$ is the one estimated based on the theory[50]. The mean
fidelity of all datasets is 0.9816$\pm$0.0148 and 0.9934$\pm$0.0048, for the
quantum kernel and coherent kernel respectively. For each dataset, we use
$2/3$ of the data points for training of the SVM, which yields the
coefficients mentioned in 1. The remaining $1/3$ as the test dataset can be
used to predict the classification accuracy. Accuracy is defined as the
percentage of correctly classified points out of the total size of the test
set. Let us note that values lower than 0.5 indicate that the model was not
able to learn the features of the training set and generalize to unknown data.

In Fig. 4a and b, we show the test accuracies obtained by injecting two input
states for four different dataset sizes, where the quantum kernel performs
significantly better than the coherent kernel at both experiments. In Fig. 4c,
we report the average test accuracy obtained for five different datasets with
the same size, varying the dataset sizes from 40 to 100 as well. Moreover, the
results obtained with the quantum kernel (blue) and the coherent kernel
(orange) are compared with the following numerical kernels: neural tangent
kernel (green) [31], gaussian (grey), polynomial (yellow) and linear (purple)
[25, 26]. Here, the neural tangent kernel (ntk) adopts a infinite-width neural
network to classify the data optimized by gradient descent. See the
Supplementary Note 5 for more details. The dashed lines indicate the results
of numerical simulations, while the solid lines indicate experimental results.
Although the task is built only comparing the performance of the kernels based
on indistinguishable and distinguishable photons, the both obtained
accurracies are higher than other classical kernels.

## Discussion

In this work, we show the first experimental demonstration of quantum kernel
estimation, based on the unitary evolution of Fock states through an
integrated photonic processor. We map data into a feature space through the
evolution of a fixed two-photon input state over six modes. To achieve this,
we adopt an quantum processor realized by femtosecond laser writing in a
borosilicate glass substrate [30]. The sampled output distribution is then fed
into an SVM, performing the classification. Note that, although our apparatus
only features tunable phase shifters and beam splitters, such encoding
produces a sufficient non-linearity to achieve high classification accuracy of
non-linearly separable datasets. This constitutes a difference of our method
from alternative platforms, where entangling gates are typically needed [51,
52, 29]. Furthermore, in our case, it is not necessary to increase the
dimension of the feature Hilbert space to achieve a good accuracy. This
circumvents the typical difficulty of quantum kernels whereby all data points
are encoded in orthogonal states, leading to ineffective classification [53,
54]. Moreover, the fact that our model is effective for small dimensions is a
crucial feature, since we require an approximation of the full probability
distribution deriving from the evolution of our input state. Hence this study
is relevant for medium-sized problems, because reaching high dimensions would
imply the input/output combinations to grow exponentially, along with the
number of required experimental shots to reach arbitrary accuracy.

The task we implement is designed by assigning binary labels to randomly
generated data points, which we encode in the phase shifts of an optical
circuit. This is done exploiting the so-called geometric difference, that
selects the task for which the presence of quantum interference yields a
better classification accuracy compared to cases where no interference is
displayed. Although, the geometric difference compares the performance of a
pair of kernels (in our case kernels implemented with
indistinguishable/distinguishable bosons), for the selected tasks, both
photonic kernels performed significantly better than commonly used kernels,
not only the gaussian, polyonomial and linear ones, but also the neural
tangent kernel. Our results indicate that a photonic kernel estimation can
enhance the performance, even for medium-size problems, reachable by current
quantum technologies. Moreover, using distinguishable bosons to have a
(smaller) performance enhancement represents an intriguing possibility, as it
can prove convenient to reduce the impact of photon losses on the experimental
time required to collect significant statistics.

Let us highlight that, although the interference of two photons in a six-mode
unitary matrix (or more in general for medium-sized problems) can be estimated
by classical computers, this does not affect the features of our protocol.
Firstly, because the approximation of permanents through classical algorithms,
e.g. Gurvits one [55] has a slightly worse performance when compared to the
direct sampling from an optical circuit, which naturally implements the
studied kernel. In particular, the first scales as $O(n^{2}/e^{2})$, whereas
$n$ is the number of photons and $e$ is the required precision, while the
direct sampling as $O(1/e^{2})$ [56]. This is also the reason for which we do
not use our photonic platform to reproduce classical kernels, as in [57],
being given by the natural evolution of bosons through a quantum circuit.
Moreover, this protocol sheds light on alternative computational models,
exploiting optical computation. This may be of particular importance when
considering difficulties related to energy consumption, as it has been proved
that partially optical networks can reduce energy requirements with respect to
electronic ones [58].

Despite being overshadowed by deep neural networks, kernels are still widely
used in a large number of tasks, due to their simplicity, and ability to learn
from small datasets [59, 60]. Indeed, they have seen a recent revival in
classical machine learning with the introduction of neural tangent kernels
[31] and their use in the study of state-of-the-art neural network
architectures such as transformers [61]. Another recent trend consists in
merging neural networks and kernels, where notable examples are attention
modules in natural language processing, and Hopfield layers [62].

Our method can find a wide range of promising near-term applications in tasks
such as information retrieval, natural language processing and medical image
classification [63, 64, 65, 66], where kernels have been proposed as a
keystone [67]. Our experimental results also open the door to hybrid methods
where photonic processors are used to enhance the performance of standard
machine learning methods. They also bring forward investigations on the non-
linearities that can be achieved through photonic platforms [68, 69], in
particular, for neuromorphic computation models, such as reservoir computing
[70, 71]. In addition, we envisage the combination of this kind of non-
linearity with those brought by the implementation of feedback loops, as in
the case of quantum memristor [69] and the exploitation of quantum
interference in the implementation of feature maps.

##  I Acknowledgements

The authors would like to thank Anna Pearson for her active involvement in the
first stages of this project. This research was funded in whole or in part by
the Austrian Science Fund (FWF)[10.55776/ESP205] (PREQUrSOR), [10.55776/F71]
(BeyondC), [10.55776/FG5] (Research Group 5) and [10.55776/I6002] (PhoMemtor).
For open access purposes, the author has applied a CC BY public copyright
license to any author accepted manuscript version arising from this
submission. This project has received funding from the European Union’s
Horizon 2020 research and innovation programme under grant agreement no.
899368 (EPIQUS), the Marie Skłodowska-Curie grant agreement No 956071
(AppQInfo) and grant agreement no. 101017733 (QuantERA II Programme, project
Phomemtor). Views and opinions expressed are however those of the author(s)
only and do not necessarily reflect those of the European Union or the
European Research Council. Neither the European Union nor the granting
authority can be held responsible for them. The financial support by the
Austrian Federal Ministry of Labour and Economy, the National Foundation for
Research, Technology and Development and the Christian Doppler Research
Association is gratefully acknowledged. G.d.F would like to thank Konstantinos
Meichanetzidis and Tommaso Salvatori for insightful discussions on the
project. The integrated photonic processor was partially fabricated at
PoliFAB, the micro- and nanofabrication facility of Politecnico di Milano
(https://www.polifab.polimi.it/). C.P., F.C. and R.O. wish to thank the
PoliFAB staff for the valuable technical support. R.O. acknowledges financial
support by ICSC – National Research Center in High Performance Computing, Big
Data and Quantum Computing, funded by European Union– NextGenerationEU.

##  II Author contributions

Z.Y. and I.A. designed and conducted the experiment and G.d.F., D.B. and A.T.
developed the theory and algorithm. C.P., S.P., A.C. and F.C. conducted the
design, fabrication and calibration of the integrated photonic processor.
Z.Y., I.A. and G.d.F. wrote the first draft of the manuscript. R.O., B.C. and
P.W. supervised the whole project. All authors discussed the results and
reviewed the manuscript.

##  III Competing interests

F.C. and R.O. are cofounders of the company Ephos. The authors declare that
they have no other competing interests.

## References

  * [1] Kimble, H. J.  The quantum internet.  _Nature_ 453, 1023–1030 (2008).
  * [2] Wehner, S., Elkouss, D. & Hanson, R.  Quantum internet: A vision for the road ahead.  _Science_ 362, eaam9288 (2018).
  * [3] Georgescu, I. M., Ashhab, S. & Nori, F.  Quantum simulation.  _Reviews of Modern Physics_ 86, 153 (2014).
  * [4] Nielsen, M. A. & Chuang, I. L.  Quantum computation and quantum information.  _Phys. Today_ 54, 60 (2001).
  * [5] Shor, P. W.  Algorithms for quantum computation: discrete logarithms and factoring.  In _Proceedings 35th annual symposium on foundations of computer science_ , 124–134 (Ieee, 1994).
  * [6] Grover, L. K.  A fast quantum mechanical algorithm for database search.  In _Proceedings of the twenty-eighth annual ACM symposium on Theory of computing_ , 212–219 (1996).
  * [7] Aaronson, S. & Arkhipov, A.  The Computational Complexity of Linear Optics.  _Proceedings of the 43rd annual ACM symposium on Theory of computing - STOC ’11_ 9, 333 (2010).  URL http://portal.acm.org/citation.cfm?doid=1993636.1993682.  ArXiv: 1011.3245 Publisher: ACM Press Place: New York, New York, USA ISBN: 9781450306911.
  * [8] Zhong, H.-S. _et al._ Quantum computational advantage using photons.  _Science_ 370, 1460–1463 (2020).
  * [9] Madsen, L. S. _et al._ Quantum computational advantage with a programmable photonic processor.  _Nature_ 606, 75–81 (2022).
  * [10] Tillmann, M. _et al._ Experimental boson sampling.  _Nature Photonics_ 7, 540–544 (2013).  ArXiv: 1212.2240.
  * [11] Broome, M. A. _et al._ Photonic Boson Sampling in a Tunable Circuit.  _Science_ 339, 794–798 (2013).  URL https://www.science.org/doi/abs/10.1126/science.1231440.  Publisher: American Association for the Advancement of Science.
  * [12] Crespi, A. _et al._ Integrated multimode interferometers with arbitrary designs for photonic boson sampling.  _Nature Photonics_ 7, 545–549 (2013).  URL https://www.nature.com/articles/nphoton.2013.112.  Number: 7 Publisher: Nature Publishing Group.
  * [13] Arute, F. _et al._ Quantum supremacy using a programmable superconducting processor.  _Nature_ 574, 505–510 (2019).
  * [14] Preskill, J.  Quantum computing in the NISQ era and beyond.  _Quantum_ 2, 1–20 (2018).  ArXiv: 1801.00862.
  * [15] Brooks, M.  Before the quantum revolution.  _Nature_ 574, 19–21 (2019).
  * [16] Biamonte, J. _et al._ Quantum machine learning.  _Nature_ 549, 195–202 (2017).
  * [17] Wittek, P.  _Quantum machine learning: what quantum computing means to data mining_ (Academic Press, 2014).
  * [18] Dunjko, V. & Briegel, H. J.  Machine learning & artificial intelligence in the quantum domain: a review of recent progress.  _Reports on Progress in Physics_ 81, 074001 (2018).
  * [19] Neven, H., Denchev, V. S., Rose, G. & Macready, W. G.  Training a large scale classifier with the quantum adiabatic algorithm.  _arXiv preprint arXiv:0912.0779_ (2009).
  * [20] Rebentrost, P., Mohseni, M. & Lloyd, S.  Quantum support vector machine for big data classification.  _Physical review letters_ 113, 130503 (2014).
  * [21] Leifer, M. S. & Poulin, D.  Quantum graphical models and belief propagation.  _Annals of Physics_ 323, 1899–1946 (2008).
  * [22] Saggio, V. _et al._ Experimental quantum speed-up in reinforcement learning agents.  _Nature_ 591, 1–12 (2021).  URL http://dx.doi.org/10.1038/s41586-021-03242-7.  Publisher: Springer US.
  * [23] Boixo, S. _et al._ Characterizing quantum supremacy in near-term devices.  _Nature Physics_ 14, 595–600 (2018).
  * [24] Gan, B. Y., Leykam, D. & Angelakis, D. G.  Fock state-enhanced expressivity of quantum machine learning models.  _EPJ Quantum Technology_ 9, 16 (2022).
  * [25] Shawe-Taylor, J. & Cristianini, N.  _Kernel methods for pattern analysis_ (Cambridge university press, 2004).
  * [26] Hofmann, T., Schölkopf, B. & Smola, A. J.  Kernel methods in machine learning.  _The Annals of Statistics_ 36, 1171 – 1220 (2008).
  * [27] Cortes, C. & Vapnik, V.  Support-vector networks.  _Machine learning_ 20, 273–297 (1995).
  * [28] Liu, Y., Arunachalam, S. & Temme, K.  A rigorous and robust quantum speed-up in supervised machine learning.  _Nature Physics_ 17, 1013–1017 (2021).  URL http://dx.doi.org/10.1038/s41567-021-01287-z.  ArXiv: 2010.02174 Publisher: Chapman and Hall/CRC ISBN: 9780429297595.
  * [29] Huang, H. Y. _et al._ Power of data in quantum machine learning.  _Nature Communications_ 12 (2021).  URL http://dx.doi.org/10.1038/s41467-021-22539-9.  ArXiv: 2011.01938 Publisher: Springer US.
  * [30] Pentangelo, C. _et al._ High-fidelity and polarization-insensitive universal photonic processors fabricated by femtosecond laser writing.  _Nanophotonics_ (2024).  URL https://www.degruyter.com/document/doi/10.1515/nanoph-2023-0636/html.  Publisher: De Gruyter.
  * [31] Jacot, A., Gabriel, F. & Hongler, C.  Neural tangent kernel: Convergence and generalization in neural networks.  _Advances in neural information processing systems_ 31 (2018).
  * [32] Clements, W. R., Humphreys, P. C., Metcalf, B. J., Kolthammer, W. S. & Walsmley, I. A.  Optimal design for universal multiport interferometers.  _Optica_ 3, 1460 (2016).  ArXiv: 1603.08788.
  * [33] Boser, B. E., Guyon, I. M. & Vapnik, V. N.  A training algorithm for optimal margin classifiers.  In _Proceedings of the fifth annual workshop on Computational learning theory_ , 144–152 (1992).
  * [34] Vapnik, V.  _The nature of statistical learning theory_ (Springer science & business media, 1999).
  * [35] Schuld, M.  Supervised quantum machine learning models are kernel methods.  _arXiv preprint arXiv:2101.11020_ (2021).
  * [36] Lloyd, S., Schuld, M., Ijaz, A., Izaac, J. & Killoran, N.  Quantum embeddings for machine learning.  URL http://arxiv.org/abs/2001.03622.  eprint 2001.03622.
  * [37] Kübler, J. M., Buchholz, S. & Schölkopf, B.  The Inductive Bias of Quantum Kernels (2021).  URL http://arxiv.org/abs/2106.03747.  ArXiv:2106.03747 [quant-ph, stat].
  * [38] Bartkiewicz, K. _et al._ Experimental kernel-based quantum machine learning in finite feature space.  _Scientific Reports_ 10, 1–9 (2020).  URL https://doi.org/10.1038/s41598-020-68911-5.  ArXiv: 1906.04137 Publisher: Nature Publishing Group UK ISBN: 0123456789.
  * [39] Huang, H. Y., Kueng, R. & Preskill, J.  Information-Theoretic Bounds on Quantum Advantage in Machine Learning.  _Physical Review Letters_ 126, 190505 (2021).  URL https://doi.org/10.1103/PhysRevLett.126.190505.  ArXiv: 2101.02464 Publisher: American Physical Society.
  * [40] Kusumoto, T., Mitarai, K., Fujii, K., Kitagawa, M. & Negoro, M.  Experimental quantum kernel trick with nuclear spins in a solid.  _npj Quantum Information_ 7, 94 (2021).
  * [41] Haug, T., Self, C. N. & Kim, M.  Quantum machine learning of large datasets using randomized measurements.  _arXiv preprint arXiv:2108.01039_ (2021).
  * [42] Schölkopf, B. & Smola, A. J.  _Learning with kernels: support vector machines, regularization, optimization, and beyond_ (MIT press, 2002).
  * [43] Jerbi, S. _et al._ Quantum machine learning beyond kernel methods.  _Nature Communications_ 14, 517 (2023).
  * [44] Scheel, S.  Permanents in linear optical networks.  _arXiv preprint quant-ph/0406127_ (2004).
  * [45] Davis, K. M., Miura, K., Sugimoto, N. & Hirao, K.  Writing waveguides in glass with a femtosecond laser.  _Optics letters_ 21, 1729–1731 (1996).
  * [46] Osellame, R., Cerullo, G. & Ramponi, R. (eds.) _Femtosecond Laser Micromachining: Photonic and Microfluidic Devices in Transparent Materials_ , vol. 123 of _Topics in Applied Physics_ (Springer, Berlin, Heidelberg, 2012).  URL https://link.springer.com/10.1007/978-3-642-23366-1.
  * [47] Corrielli, G., Crespi, A. & Osellame, R.  Femtosecond laser micromachining for integrated quantum photonics.  _Nanophotonics_ 10, 3789–3812 (2021).  URL https://www.degruyter.com/document/doi/10.1515/nanoph-2021-0419/html?lang=en.  Publisher: De Gruyter.
  * [48] Ceccarelli, F. _et al._ Low power reconfigurability and reduced crosstalk in integrated photonic circuits fabricated by femtosecond laser micromachining.  _Laser & Photonics Reviews_ 14, 2000024 (2020).
  * [49] Hong, C.-K., Ou, Z.-Y. & Mandel, L.  Measurement of subpicosecond time intervals between two photons by interference.  _Physical review letters_ 59, 2044 (1987).
  * [50] Scheel, S.  Permanents in linear optical networks (2004).  URL http://arxiv.org/abs/quant-ph/0406127.  ArXiv: quant-ph/0406127.
  * [51] Havlíček, V. _et al._ Supervised learning with quantum-enhanced feature spaces.  _Nature_ 567, 209–212 (2019).
  * [52] Glick, J. R. _et al._ Covariant quantum kernels for data with group structure.  _Nature Physics_ (2024).  URL https://www.nature.com/articles/s41567-023-02340-9.
  * [53] Kübler, J. M., Buchholz, S. & Schölkopf, B.  The Inductive Bias of Quantum Kernels.  URL http://arxiv.org/abs/2106.03747.  eprint 2106.03747.
  * [54] Thanasilp, S., Wang, S., Cerezo, M. & Holmes, Z.  Exponential concentration in quantum kernel methods.  URL http://arxiv.org/abs/2208.11060.  eprint 2208.11060.
  * [55] Gurvits, L.  On the complexity of mixed discriminants and related problems.  In _Mathematical Foundations of Computer Science 2005: 30th International Symposium, MFCS 2005, Gdansk, Poland, August 29–September 2, 2005. Proceedings 30_ , 447–458 (Springer, 2005).
  * [56] Aaronson, S. & Hance, T.  Generalizing and derandomizing gurvits’s approximation algorithm for the permanent.  _arXiv preprint arXiv:1212.0025_ (2012).
  * [57] Bartkiewicz, K. _et al._ Experimental kernel-based quantum machine learning in finite feature space.  _Scientific Reports_ 10, 12356 (2020).
  * [58] Hamerly, R., Bernstein, L., Sludds, A., Soljačić, M. & Englund, D.  Large-scale optical neural networks based on photoelectric multiplication.  _Physical Review X_ 9, 021032 (2019).
  * [59] Lee, J. _et al._ Finite versus infinite neural networks: an empirical study.  _Advances in Neural Information Processing Systems_ 33, 15156–15172 (2020).
  * [60] Radhakrishnan, A., Ruiz Luyten, M., Prasad, N. & Uhler, C.  Transfer learning with kernel methods.  _Nature Communications_ 14, 5570 (2023).
  * [61] Tsai, Y. H., Bai, S., Yamada, M., Morency, L. & Salakhutdinov, R.  Transformer dissection: An unified understanding for transformer’s attention via the lens of kernel.  In Inui, K., Jiang, J., Ng, V. & Wan, X. (eds.) _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_ , 4343–4352 (Association for Computational Linguistics, 2019).  URL https://doi.org/10.18653/v1/D19-1443.
  * [62] Ramsauer, H. _et al._ Hopfield networks is all you need.  _arXiv preprint arXiv:2008.02217_ (2020).
  * [63] Wang, X., Du, Y., Luo, Y. & Tao, D.  Towards understanding the power of quantum kernels in the NISQ era.  _Quantum_ 5, 531 (2021).  URL https://quantum-journal.org/papers/q-2021-08-30-531/.
  * [64] Yu, C.-H., Gao, F., Wang, Q.-L. & Wen, Q.-Y.  Quantum algorithm for association rules mining.  _Physical Review A_ 94, 042311 (2016).  URL https://link.aps.org/doi/10.1103/PhysRevA.94.042311.
  * [65] Lorenz, R., Pearson, A., Meichanetzidis, K., Kartsaklis, D. & Coecke, B.  QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer.  _ArXiv e-prints_ (2021).  eprint 2102.12846.
  * [66] Landman, J. _et al._ Quantum Methods for Neural Networks and Application to Medical Image Classification.  _Quantum_ 6, 881 (2022).  URL https://quantum-journal.org/papers/q-2022-12-22-881/.
  * [67] Schuld, M. & Petruccione, F.  Quantum Models as Kernel Methods.  In Schuld, M. & Petruccione, F. (eds.) _Machine Learning with Quantum Computers_ , Quantum Science and Technology, 217–245 (Springer International Publishing, Cham, 2021).  URL https://doi.org/10.1007/978-3-030-83098-4_6.
  * [68] Denis, Z., Favero, I. & Ciuti, C.  Photonic kernel machine learning for ultrafast spectral analysis.  _Physical Review Applied_ 17, 034077 (2022).
  * [69] Spagnolo, M. _et al._ Experimental photonic quantum memristor.  _Nature Photonics_ 16, 318–323 (2022).
  * [70] Govia, L., Ribeill, G., Rowlands, G. & Ohki, T.  Nonlinear input transformations are ubiquitous in quantum reservoir computing.  _Neuromorphic Computing and Engineering_ 2, 014008 (2022).
  * [71] Innocenti, L. _et al._ Potential and limitations of quantum extreme learning machines.  _Communications Physics_ 6, 118 (2023).

##  IV Methods

The two photon input states are generated by a type-II spontaneous parametric
down-conversion source, which generates frequency degenerate single-photon
pairs at 1546 nm via a 3-cm long periodically poled K-titanyl phosphate
(ppKTP) crystal. Afterwards, the two photons are made indistinguishable in
their polarization, which is rotated through paddles, and in arrival time,
through a delay line, which we use also to tune the degree of
indistinguishability of the generated photons.

Then, we inject these photons into two selected modes of an integrated
photonic processor with six input/output modes[30]. This circuit features 27
thermal phase shifters and its architecture follows the rectangular scheme
presented in [32], to implement arbitrary unitary evolution on any input Fock
state. Compared with the original architecture, the first three external phase
shifters are omitted, as they will not effect the probability distribution. To
apply accurate phases independently, each channel is supplied by a current
source to avoid electrical crosstalk (4$\times$Qontrol q8iv, 16bit DAC). In
the end, detection is performed by superconducting nanowire single-photon
detectors (SNSPDs) housed in a 1K cryostat. We post-select the detected events
to the cases in which two detectors click simultaneously in a temporal window
of 1 ns. A time tagger with a 15.63 ps resolution is used to process the real-
time coincidence counting for all 15 post-selection patterns. The total
coincidence counting is about 10 kHz, which varies due to the pump laser and
the detection efficiency. For each unitary configuration, we integrate 5 s to
estimate the probability distribution over 15 coincidence patterns.

##  V Data availability

The data supporting this study’s findings are available from the project page
(https://github.com/dapingQ/PhoQuKs), containing detailed explanations of all
the datasets.

##  VI Code availability

The code scripts analyzing the study is available from the project page
(https://github.com/dapingQ/PhoQuKs).