# GraphConfRec: A Graph Neural Network-Based Conference Recommender System

Andreea Iana  Data and Web Science Group
University of Mannheim
Mannheim, Germany
<EMAIL_ADDRESS>Heiko Paulheim  Data and Web Science Group
University of Mannheim
Mannheim, Germany
<EMAIL_ADDRESS>
###### Abstract

In today’s academic publishing model, especially in Computer Science,
conferences commonly constitute the main platforms for releasing the latest
peer-reviewed advancements in their respective fields. However, choosing a
suitable academic venue for publishing one’s research can represent a
challenging task considering the plethora of available conferences,
particularly for those at the start of their academic careers, or for those
seeking to publish outside of their usual domain. In this paper, we propose
GraphConfRec, a conference recommender system which combines SciGraph and
graph neural networks, to infer suggestions based not only on title and
abstract, but also on co-authorship and citation relationships. GraphConfRec
achieves a recall@10 of up to 0.580 and a MAP of up to 0.336 with a graph
attention network-based recommendation model. A user study with 25 subjects
supports the positive results.

###### Index Terms:

Recommender System, Graph Neural Network, SciGraph, Scientific Publications

##  I Introduction

Bibliographic data constitutes the third largest domain in the Linked Open
Data (LOD) cloud, after geographic and social web data, as stated by the 2014
report on the State of the LOD cloud, which records 138 such datasets (13%)
[1]. LOD cloud’s most recent version reports 150 datasets in this category
(12%).111https://lod-cloud.net/ (last accessed 10/01/2020)

SciGraph222https://scigraph.springernature.com/explorer (last accessed
17/04/2020), Springer Nature’s successor of Springer’s LOD Conference Portal
[2] is a LOD bibliographic set, encompassing information about journals,
books, and book chapters published by Springer Nature since the 19th century.
In addition to publications, it also comprises metadata about corresponding
events, such as conferences and workshops, as well as information regarding
associated persons and institutions.

On the one hand, bibliographic datasets have been largely utilised for post-
hoc studies, such as scientometrics, in quantitative analyses of publications,
citations, or authors [3, 4]. On the other hand, recommender systems developed
for scientific publications either focus on recommending research papers [5,
6], or on suggesting academic venues, utilising the title or abstract of a
paper [7, 8, 9]. Only few models incorporate knowledge of co-authorship and
citation relationships through hand-crafted features, requiring domain
knowledge [10, 11].

In contrast, in this paper, we focus on the graph nature of scientific data
(e.g. citation and co-authorship graphs), and on exploiting this using recent
advancements in graph neural networks to provide users with recommendations
for future academic venues. We propose GraphConfRec, a service which
recommends conferences to submit a paper to, based not only on title and
abstract, but also on co-authorship and citation relationships. The
recommender utilizes SciGraph for information on past conferences and
publications, and WikiCfP for data on upcoming events. GraphConfRec aims to
aid researchers getting accustomed to a new domain, or those seeking to
publish results outside their usual community, to navigate the plethora of
available conferences.

The rest of the paper is structured as follows. Section II provides an
overview of related work. The dataset used and the proposed recommendation
techniques are described in Sections III and IV, respectively. Section V
introduces the experimental setup and discusses the evaluation results. We
conclude with a summary and discussion of open issues and future work in
Sections VI and VII.

##  II Related Work

Although recommender systems for scholarly data have been researched for
nearly 20 years [5, 12], the majority of these focus on recommending research
papers [13], or on detecting whether a paper is within the scope of a venue
[14]. More recently, several recommender systems have been developed to
suggest venues based generally on the textual content of a manuscript (e.g.
title, abstract, keywords). Instances of such systems include, but are not
limited to, Springer Nature’s Journal
Suggester333https://journalsuggester.springer.com, Wiley’s Journal
Finder444https://journalfinder.wiley.com/search?type=match, Elsevier’s Journal
Finder555https://journalfinder.elsevier.com[15], Research Square’s Journal
Guide666https://www.journalguide.com, IEEE Publication
Recommender777http://publication-recommender.ieee.org/home,
Jane888https://jane.biosemantics.org/ [16], or techniques proposed by [17, 8,
9]. In contrast, models proposed by Pan et al. [10] or Rollins et al. [11]
utilise explicit feature engineering to include clues about citation or
authors networks. Pradhan et al. [18] recommend venues by enhancing the
representations of a paper’s title and abstract with meta-path features
extracted from citation and bibliographic networks. Küçüktunç et al. [19] and
Boukhris and Ayachi [20] generate recommendations without considering the
content of a prospective manuscript by extending bibliographic data with
citation relationships.

Furthermore, recommender systems can be improved by leveraging LOD. In
particular, open datasets providing detailed descriptions of items can
constitute a strong asset for content-based recommenders, which rely on the
items’ attributes for computing their similarities [21, 22]. While many of
these approaches employ handcrafted features [23], graph embeddings trained on
LOD, such as TransE [24] or RDF2vec [25], have also been utilised for
recommender systems, since they aim to encode similar entities closely in a
latent vector space. Hence, items which are projected closely in the vector
space would be used as candidates for recommendation. However, such shallow
transductive embedding techniques are only applicable to static graphs and
cannot be utilised in recommender systems built on evolving graphs, as they do
not use node features and are unable to generate embeddings for nodes not seen
during training, without additional optimization [26]. The citation and co-
authorship graphs utilized for conference recommendations evolve over time, as
new publications act as unseen nodes added to the existing graph at test time.
Such dynamic graphs require an inductive embedding method, capable of
computing latent representations for new nodes.

The particularities of graphs, comprised of unordered nodes with varied-sized
neighbourhoods, lacking spatial orientation and having diverse structures,
pose additional challenges for traditional machine-learning algorithms [27,
28]. Recently, graph neural networks (GNNs) have been extensively researched
to extend the ability of deep neural networks to extract meaningful patterns
from large-scale datasets by leveraging the input’s structure to non-Euclidean
domains, such as graphs, through a rich variety of approaches. Although GNNs
have already been considered in recommendation problems [29, 30], they have
not yet been explored, to the best of our knowledge, in the context of
recommending academic venues.

##  III Data

GraphConfRec aims to recommend conferences for a prospective manuscript. To
this end, we utilise SciGraph for training the recommender system.
Additionally, data mined from WikiCfP and Google Scholar Metrics is used to
provide optional information on future conferences (e.g. submission dates,
description), and their rankings, in the prototype implemented to demonstrate
GraphConRec’s functioning.

###  III-A SciGraph

SciGraph serves as the primary data source for training the recommender
system. SciGraph’s latest version contains information on approximately 4.5M
book chapters, 270k books published by Springer Nature since 1839 and over 7M
related persons. Only a fraction of the books contained in SciGraph correspond
to conference proceedings. Moreover, the latest datasets do not encompass
conference information. Therefore, we created a mapping between the latest
2019 Q1 SciGraph release, which incorporates citation information, and the
2018 Q1 release, which includes information about conferences, but not about
referenced works. Fig. 1 illustrates the SciGraph subsets used for
recommendation from these two datasets, namely information on abstracts,
titles, citations, and authors (the main attributes are highlighted).

Figure 1: Data model of SciGraph subset used in GraphConfRec.

The final dataset contains only papers with an English abstract and
referencing conference proceedings also contained in SciGraph. For training,
publications from the years 1975-2014 were used, whereas for validation and
testing we utilised the papers published in 2015 and 2016, respectively. Table
I shows basic statistics of the three datasets. In total, the recommender
system was trained on 1,122 distinct conference series and 137,376 papers
written by 164,103 authors. Fig. 2 shows the average length of abstracts in
the data (130 words) and the mean number of 5 citations per publication
towards other conference proceedings in SciGraph999While the total number of
citations in each paper is higher, not all cited papers are contained in
SciGraph. These numbers reflect the edges in SciGraph, i.e. a paper in
SciGraph citing another paper also in SciGraph..

(a) Average abstract length.

(b) Average citation number.

Figure 2: Characteristics of abstracts and citations in the datasets. TABLE I:
Characteristics of the training, validation, and test sets.

| Training (1975-2014) | Validation (2015) | Test (2016) | Overlap (training - test)
---|---|---|---|---
Distinct conference series IDs | 1,122 | 311 | 518 | 394
Distinct author names | 164,103 | 19,994 | 30,175 | 14,639
All papers | 293,836 | 130,002 | 213,000 | –
Papers w/ English abstracts | 290,877 | 12,990 | 21,223 | –
| Papers w/ English abstracts
---
and citations in SciGraph
137,376 | 7,511 | 11,600 | –

###  III-B WikiCfP and H5-Index

Two more datasets were utilized only in the user interface, to augment the
recommended conferences with additional information, such as conference dates
or ranking. However, these datasets were not used for training the models or
generating the recommendations, which was done solely on SciGraph.

One dataset consists of calls for papers (CfPs) published on
WikiCfP101010http://www.wikicfp.com/cfp/ (last accessed 31/12/2019). We
implemented a crawler to create a dataset of conference names, acronyms,
dates, locations, submission deadlines (e.g. mandatory attributes), as well as
the conference series, categorization in WikiCfP, textual description and
links to the conference’s webpage (e.g. optional attributes). Overall, 4,902
CfPs were crawled in January 2020, out of which nearly a third have missing
mandatory information. The crawled data was linked to the SciGraph dataset
using Damerau-Levenshtein string similarity between the conference names. For
each matched SciGraph conference, we retrieved the latest corresponding CfP
with a submission deadline after January 1st, 2020. This resulted in 8.24%
SciGraph conferences being matched to WikiCfP. The low linking precision
(0.086) and recall (0.09) are determined by the low number of CfPs that met
this strict submission deadline condition and were published on WikiCfP at the
time of crawling.

The other dataset comprises of conference ratings published by Google Scholar
Metrics111111https://scholar.google.de/citations?view_op=top_venues&hl=en
(last accessed 31/12/2019), and was utilised to provide users with information
about the quality of conferences. As there is no downloadable version of the
data, a web scraper was implemented to retrieve the h5-index of ranked
conferences. Google Scholar Metrics offers these statistics in the form of
fixed-sized tables with 20 rankings, for 8 categories and 194 subcategories. A
dataset containing 4,540 publications’ ratings was extracted. This was
integrated in the SciGraph dataset using Damerau-Levenshtein string similarity
on conference names, and resulted in 2.73% of the SciGraph conferences to be
enriched with a h5-index rating, with a precision of 0.34 and recall of 0.55.
The small fraction of matches is caused by the low number of conferences
included in the Google Scholar Metrics. Terms such as ”Conference” or
”Workshop” appear in only 3.9% of the ratings, while the word ”Journal” is
explicitly mentioned in 32.8% of the rated publications.

Since the matches were scarce, we decided to check them manually, and to
provide the supplementary information from WikiCfP and Google Scholar Metrics
in the user interface only for the correct matches.

##  IV Recommendation Techniques

We developed two classes of recommendation techniques. The baseline, an
author-based model, creates content-based recommendations using the
publication history of a paper’s authors. The model aggregates, per conference
series, all papers sharing at least one author name, and recommends unique
conference series where the authors of a given paper have previously
published121212We do not distinguish authors by unique IDs here, since only a
subset of all Springer Nature authors have been disambiguated.. The results
are ranked decreasingly by the count of publications per conference series.

GNN-based models aim to project similar entities closely in a latent vector
space based on their features and relationships to other entities. Thus, the
models learn not only the topological structure of a local neighbourhood, but
also the distribution of the node features among its neighbours. To this end,
the input data is structured as a graph with the following characteristics:
nodes represent conference proceedings, node features encode their textual
content, node labels denote the conference where the paper was published,
while the edges denote citations or co-authorship links. We constructed three
graph variants based on the edge type. The _citations graph_ contains edges
representing citations among connected papers. In the _co-authorship graph_ ,
an edge is incident in two nodes if the corresponding papers share at least
two authors, if not specified otherwise. Lastly, the _heterogeneous graph_
incorporates both kinds of edges.

The recommendation problem is modelled as a _node classification_ task
consisting in the prediction of a label for a new node denoting the input
paper. More specifically, a prospective publication represents a new node that
is attached to the existing graph by means of citation or co-authorship links.
All models are inductive, meaning that they can handle unseen nodes, and
either output node embeddings (unsupervised model), or directly compute a
probability distribution over the classes (supervised model), utilizing the
feature information and position of the new node in the graph. In the
unsupervised approaches, a classifier is trained on the embeddings generated
by the GNNs and utilised to classify the nodes. In all scenarios, the system
recommends the top $N$ most probable conferences for a given paper. We
implemented several versions of each recommendation technique based on the
training method and graph type. Fig. 3 illustrates the GraphConfRec’s
pipeline.

Figure 3: Illustration of GraphConfRec’s pipeline, using a _heterogeneous
graph._

In earlier works [8], we observed that word embeddings trained on a scientific
corpus perform better on SciGraph than those trained on general purpose
corpora, such as Wikipedia, as the dataset mostly focuses on computer science
publications. Therefore, we embed the papers’ titles and abstracts using the
pre-trained transformer-based [31] language model SciBERT [32], with an
uncased vocabulary. Since SciBERT allows sequences of maximum 512 tokens, we
assume that the most relevant information is contained within the first 512
words of an abstract or title, and the remaining words are ignored. This
observation is consistent with the data, since only 31 paper abstracts from
the training and validation sets are longer than 512 tokens. For every token,
SciBERT outputs a vector of size 768, for each of its 12 hidden layers. Table
III enumerates the strategies utilised to compute a sequence’s representation
from its individual tokens. The resulting embeddings of titles and abstracts
are then concatenated and treated as node features.

####  IV-1 GraphSAGE (SAmple and aggreGatE)-based models

compute a node’s embedding by aggregating the feature information of the nodes
in its uniformly sampled, fixed-sized neighbourhood, using several learned
aggregation functions proposed by Hamilton et al. [26]. Five architectures are
used: (1) mean aggregator – computes the element-wise mean of the vectors; (2)
mean and (3) max pooling aggregators – apply an element-wise mean or max
pooling operation on every neighbour’s vector, after each of them have been
passed through a fully-connected single-layer neural network; (4) a modified,
inductive GCN aggregator [33]; and (5) an LSTM-based aggregator [34]. We
implemented different GraphSAGE-based recommendation models to examine the
importance of training type and similarity computation on recommendations.
_GraphSAGE Neighbour_ computes the papers’ similarity in the latent vector
space and recommends conferences corresponding to the input paper’s nearest
neighbours. _GraphSAGE Classifier_ utilises a classifier trained on the
embeddings generated by the unsupervised GraphSAGE to predict conferences. We
applied _GraphSAGE Classifier_ on both the citations graph and the co-
authorship graph to analyse the influence of graph type on recommendations.
Additionally, we implemented _GraphSAGE Classifier Concat_ , which
concatenates the embeddings outputted by GraphSAGE trained on the citations
and the co-authorship graphs, before feeding the resulting vector into a
classifier. Lastly, _GraphSAGE supervised_ recommenders directly compute a
probability distribution over the classes.

####  IV-2 GraphSAGE_RL-based models

sample neighbourhoods based on a non-linear regression function which learns
the importance of different candidate neighbourhoods, and thus, are able to
sample only the most significant neighbours for a particular node, in order to
avoid the high variance in training and inference, which can be caused by
unbiasedly sampling using a uniform distribution [35]. A reinforcement
learning (RL) policy is used to determine the weight of each combination of
node and neighbourhood, from the negative classification loss output of a
GraphSAGE model. We created several recommenders using Oh et al.’s [35] GNN
architecture, namely: _GraphSAGE_RL Classifier (citations graph)_ ,
_GraphSAGE_RL supervised (citations graph)_ , _GraphSAGE_RL supervised
(heterogeneous graph)_.

####  IV-3 GAT (Graph attention network)-based models

compute a node’s latent representation by applying a self-attention mechanism
over the features of its neighbours, which receive different weights based on
their contribution, as proposed in [36]. We experimented with two variants of
the GAT-based recommenders, namely _GAT (citations graph)_ , and _GAT
(heterogeneous graph)_ , to obtain a probability distribution over the labels
and to determine whether using the attention mechanism improves computational
costs and recommendations’s quality.

####  IV-4 HAN (Heterogeneous graph attention network)-based models

operate, in contrast to the approaches discussed above, on heterogeneous
graphs, and explicitly distinguish between different edge types. Node-level
attention weights meta-path-based neighbours based on their influence on a
given node, whereas semantic-level attention distinguishes between different
meta-paths given their importance. In Wang et al.’s [37] GNN architecture, the
embedding of a node is computed by hierarchically aggregating, for all
semantic-specific embeddings, the feature information of the meta-path
neighbours. A HAN-based recommendation model was implemented to analyse
whether explicitly modelling various edge types and meta-paths (e.g. paper-
author-paper, paper-citation-paper) in the GNN improves the recommender’s
performance.

####  IV-5 SciBERT-ARGA-based models

adopt an approach inspired by Jeong et al. [38], and place a higher weight on
the paper’s abstracts. The abstracts are embedded using SciBERT, as before,
while the nodes of the citation or co-authorship networks are embedded using
the adversarially regularised graph autoencoder (ARGA) framework proposed by
Pan et al. [39]. The resulting vectors are concatenated and passed through a
feed-forward neural network (FFNN), before a softmax function computes the
probability distribution over conference labels. In ARGA, firstly a graph
convolutional autoencoder reconstructs the topological graph structure from
the embeddings generated from the node feature information and original graph
structure [39]. ARGVA follows the same model architecture as ARGA, but the
convolutional graph autoencoder (GAE) is replaced with a variational graph
autoencoder (VGAE). Secondly, an adversarial training scheme is employed to
enforce the latent representations generated by the GAE to match a prior
Gaussian distribution [39]. More specifically, a generator and a discriminator
play a minimax game in which the generator learns to produce “fake samples”
which are as close as possible to the real ones, while the discriminator tries
to distinguish whether an example comes from the true data distribution or is
produced by the generator [40]. Different variants of this recommender model
are developed, using both the _citations graph_ and the _heterogeneous graph_.

##  V Evaluation

GraphConfRec’s131313Code available at
https://github.com/andreeaiana/graph_confrec performance was evaluated in two
sets of experiments: a quantitative study on SciGraph, as well as a
qualitative user study.

###  V-A Experimental Setup

The quantitative experiments were performed on a test set comprising of
publication data from 2016. On the one hand, we analyse whether the proposed
methods result in suitable conference recommendations for a prospective
manuscript, as well as the results’ ranking. For each publication in the test
set, every model aims to predict the conference at which it has been
published, and the results are compared against the truth value (i.e. the
conference where it was published). For every technique, we generate 10
recommendations141414Authors-based recommendations may create shorter lists if
all authors have published at less than 10 conferences contained in SciGraph.,
and report the recall@10 and mean average precision (MAP) at different
positions. Moreover, we experiment with different combinations of aggregating
SciBERT embeddings and a wide range of model-specific parameter settings for
each of the GNN-based recommendation techniques to determine the influence of
each model component and hyperparameter setting on the overall results. On the
other hand, we conducted a user study using an online prototype service to
examine the quality of the recommendations from the researchers’ perspective.

###  V-B Results of Quantitative Experiments

TABLE II: Approaches for computing the SciBERT embeddings of the papers.

Notation | Description
---|---
AVG_L | Mean of last hidden layer representation of all tokens
AVG_2L | Mean of second-to-last hidden layer representation of all tokens
AVG_SUM_L4 | Mean of the sum of the last four hidden layers representation of all tokens
AVG_SUM_ALL | Mean of the sum of all the hidden layers representation of all tokens
MAX_2L | Max pool over the second-to-last hidden layer representation of all tokens
CONC_AVG_MAX_2L | Concatenation of the AVG_2L and MAX_2L vectors
CONC_AVG_MAX_SUM_L4 | Concatenation of the AVG_SUM_L4 and MAX_SUM_L4 vectors
SUM_L | Sum of the last hidden layer representation of all the tokens
SUM_2L | Sum of the second-to-last hidden layer representation of all the tokens

TABLE III: Results of the best performing individual recommendation
techniques. For each model, we only report the results of the best performing
model configuration.

| Model configuration | Training statistics | Evaluation results
---|---|---|---
Model | Embedding type | Parameter settings | Avg. time (s) per epoch | Epochs trained | Recall@10 | MAP@10 | MAP@5 | MAP@3
Authors (1st baseline) | – | – | – | – | 0.458 | 0.308 | 0.302 | 0.290
GraphSAGE Neighbour (2nd baseline) | CONC_AVG_MAX_SUM_L4 | maxpool aggregator | 11,994.46 | 10 | 0.259 | 0.096 | 0.082 | 0.072
GraphSAGE Classifier (citations graph) | CONC_AVG_MAX_2L | GCN aggregator + MLR | 4,991.29 | 10 | 0.414 | 0.244 | 0.234 | 0.221
GraphSAGE Classifier (co-authorship graph) | AVG_SUM_ALL | maxpool aggregator + MLR | 9,010.57 | 10 | 0.054 | 0.019 | 0.015 | 0.014
GraphSAGE Classifier Concat | SUM_L | GCN aggregator + KNN (n=30) | 3,271.12 | 10 | 0.395 | 0.237 | 0.228 | 0.215
GraphSAGE supervised (citations graph) | AVG_2L | GCN aggregator | 29.63 | 20 | 0.417 | 0.246 | 0.236 | 0.223
GraphSAGE supervised (heterogeneous graph) | AVG_SUM_ALL | GCN aggregator | 32.48 | 20 | 0.440 | 0.258 | 0.247 | 0.234
GraphSAGE_RL Classifier (citations graph) | SUM_L | GCN aggregator + MLR | 10,699.81 | 10 | 0.414 | 0.242 | 0.231 | 0.220
GraphSAGE_RL supervised (citations graph) | AVG_L | mean-concat aggregator + last-hop reward | 36.43 | 10 | 0.531 | 0.298 | 0.284 | 0.266
GraphSAGE_RL supervised (heterogeneous graph) | SUM_L | mean-concat aggregator + all-hops reward | 56.49 | 10 | 0.546 | 0.306 | 0.292 | 0.273
GAT (citations graph) | AVG_L | 8 attention heads with 64 hidden units each | 93.53 | 367 | 0.572 | 0.327 | 0.312 | 0.295
GAT (heterogeneous graph) | SUM_2L | 8 attention heads with 64 hidden units each | 147.25 | 503 | 0.580 | 0.336 | 0.322 | 0.303
HAN | AVG_L | 8 attention heads with 128 hidden units each | 226.34 | 301 | 0.540 | 0.300 | 0.285 | 0.267
SciBERT-ARGA (citations graph) | AVG_2L | ARGVA + FFNN with 500 hidden units | 5.71 | 200 | 0.530 | 0.293 | 0.278 | 0.261
SciBERT-ARGA (heterogeneous graph) | AVG_L | ARGA + FFNN with 500 hidden units | 6.57 | 200 | 0.534 | 0.295 | 0.280 | 0.263

Table III summarizes the best results obtained with each model, as well as
their configurations. Author-based recommendations perform reasonably well in
terms of MAP, but have a low recall, as fewer than half of the recommendations
contain the correct conference in the top 10 results. Two factors influence
the results. Firstly, the number of recommendations is arbitrary and empty
results lists are returned when none of the paper’s authors overlap with those
of the publications from the training set. This happens in 12.63% of the cases
due to the training-test sets split and illustrates the cold-start problem of
the model, namely that users without publications in SciGraph cannot receive a
recommendation. Secondly, the model cannot generate accurate predictions if
none of the authors have published at a conference before, or if they only
participated in events taking place after 2014.

In comparison, GNN-based recommendations are more diverse, considering not
only the paper’s textual content, as summarised by its title and abstract, but
also the relationship with other conference proceedings, cited works, or co-
authored publications. However, not all GNN-based models outperform the
authors-based recommendations. As aforementioned, the latter model may create
shorter lists or be unable to generate any suggestions. In contrast, all other
recommendation techniques compute recommendation lists of fixed length of 10
for all inputs, even if the author has no prior publication history. Overall,
we observed that for the majority of the methods, SciBERT embeddings averaging
or summing either the last, or the last two hidden layers, achieved the best
results. This shows that generally, these final layers of the transformer
suffice to summarise and encode the meaning of a sequence.

Moreover, all GNN-based models, with the exception of the _GraphSAGE
Classifier (co-authorship graph)_ outperform the second baseline, namely the
_GraphSAGE Neighbour_ model, in terms of both recall and MAP. We explored
different aggregation functions and classifiers for the GraphSAGE and
GraphSAGE_RL-based recommenders, namely K-Nearest Neighbours (KNN), Gaussian
Naive Bayes (GNB), Multilayer Perceptron (MLP), and Multinomial Logistic
Regression (MLR), as well as two reward policies for the reinforcement
learning approach of the _GraphSAGE_RL supervised_ models. The GCN aggregator
generally achieved the best results for the GraphSAGE-based models, while the
mean-concat aggregator performed better for the GraphSAGE_RL-based
recommenders. Among the classifiers, MLR and MLP achieve the best results,
followed closely by KNN, with a faster training time, in exchange for a slight
drop in performance. The _GraphSAGE supervised_ models were trained for 20
epochs each, while the other models were trained for 10 epochs, and the
weights of the model with the lowest validation loss used for inference. All
models from these two approaches sample 25 first-order and 10 second-order
neighbours, and use hidden layers of dimensions 512. We noticed that sampling
third-order neighbours or using smaller hidden dimensions decrease the
performance.

In contrast to GraphSAGE-based methods using the _citations graph_ , the model
trained solely on the co-authorship network yields a low performance,
potentially determined by the graph’s structure. In this case, the _co-
authorship graph_ was built by connecting papers which share at least one
author. This resulted in twice the number of connected components compared to
the _citations graph_ , roughly 5 times more edges and an average node degree
of 20 (versus 4 for the _citations graph_). The large number of neighbours per
node is prone to inducing noise in the model, since it is likely that not all
papers written by an author share the same topic. Moreover, the large
connected components act as clusters, meaning that conferences corresponding
to papers outside of these groups are less likely to be recommended, even if
they are relevant. Although the _GraphSAGE supervised_ -based techniques show
a small increase in both recall and MAP, only the _GraphSAGE_RL supervised_
-based recommenders outperform the _Authors_ model in terms of recall.
Therefore, non-linearly sampling a node’s neighbours is better suited for the
citations networks, where not all the referenced papers have an equal
influence on the work citing them.

In comparison, GAT-based recommendation techniques outperform the baselines in
terms of both metrics. For these models, we experimented with different
numbers of hidden units, attention heads, several dropout values, as well as
residual connections. The models were trained using an early stopping strategy
after 100 epochs with no change in the loss. Adopting the inductive parameter
settings proposed in [26] (i.e. two hidden layers with 256 units each, 4 heads
per layer, for a total of 1024 features, followed by a layer with a single
attention head which computes 1171 features, no dropout or residual
connections) resulted in a lower performance than that of _GraphSAGE
Classifier (co-authorship graph)_. However, GraphConfRec’s best performance
(recall@10 of 0.580 and MAP@10 of 0.336) was obtained with a GAT model with
only one hidden layer with 8 units and 8 heads, for a total of 64 features,
dropout of 0.5, and residual connections, trained on a directed _heterogeneous
graph_. Increasing or decreasing the number of heads, dropout value, training
only on the _citations graph_ or an undirected graph lowered both recall and
MAP. The findings show that adding residual connections alleviates the
vanishing gradients problem which might have caused the sudden drop in
performance for the first configuration of the model. Moreover, a directed
graph better encodes the structure of a citation network, which is naturally
directed.

The heterogeneous graph attention model151515We experimented with the same
parameter settings as for the GAT model. does not improve the results further,
meaning that, in this scenario, treating edges differently based on their role
(i.e. citation or co-authorship link) does not constitute a stronger signal
for detecting related papers. Similarly, combining abstract and graph
embeddings in the _SciBERT-ARGA_ model161616We experimented with the number of
hidden units in ARGA and FFNN. could not outperform the GAT-based recommender.
In turn, this proves that encoding the papers’ titles and abstracts as node
features is enough to represents a paper’s textual content, and that placing a
higher weight on the abstracts does not provide additional significant
information for determining the most similar publications to a given paper.

Another important factor to consider are the computational costs associated
with training each class of recommenders. As illustrated in Table III, all
recommendation models using an unsupervised GNN incur a long average training
time, varying between 83 and 200 minutes per epoch. In contrast, methods with
supervised GNNs are orders of magnitude faster, with an average training time
ranging from 6 to 226 seconds per epoch. Moreover, we observed that models
using the _citations graph_ are 8.7 (for _GraphSAGE supervised_ -based
techniques) to 36 times (for _GAT_ -based models) faster during training than
those utilizing the _heterogeneous graph_. This is determined by the more
complex structure and larger size of the _heterogeneous graph_ , which is
characterised by a greater number of edges for the same number of nodes as the
_citations graph_.

###  V-C Results of User Study

The user study171717The study description was distributed in the Data and Web
Science Group at the University of Mannheim, as well as on LinkedIn and
Twitter.was conducted with a group of 25 academic researchers. More than 80%
of the participants were either PhD or Master students, with at least one
publication, while the remaining subjects were post-docs or professors.
Moreover, 16% of the participants were contained in our graph, while the
others had no publications contained in SciGraph. The respondents were asked
to test four models (i.e. _Authors_ , _GAT (citations graph)_ , _GAT
(heterogeneous graph)_ , _HAN_) with papers of their choice, and rate the
suitability of the recommendations on a scale from 1 (worst) to 5 (best), as
well as to, optionally, provide comments about the quality of the suggestions.
All participants sought recommendations for their own papers.

TABLE IV: Results of the user study. Model | Authors | HAN | GAT (cit.) | GAT (het.)
---|---|---|---|---
Raters | 16 | 21 | 25 | 22
Avg. rating | 2.75 | 3.76 | 3.68 | 3.55

The results are shown in table IV. One important observation is that not all
respondents provided feedback for all four models. This was most prevalent for
the _Authors_ model, since authors without a publication history of their own
or their co-authors contained in the graph, were given no recommendations by
the baseline recommender.

Generally, the participants considered the recommendations to be suitable, but
that the ranking of the results could be improved, since suggestions perceived
as relevant were often positioned lower in the list than those perceived as
irrelevant. One positive aspect noticed by some of the respondents is that the
_GAT (citations graph)_ and _HAN_ models were able to predict conferences not
known by the authors, but which were highly suitable for the input paper’s
content, although no obvious clues were included in the abstract. Moreover,
these models predicted fitting academic venues based sometimes solely on the
paper’s abstract and title, since the authors and citations were either not
provided, or not found in the dataset. Additionally, including co-authorship
links in the predictions did not consistently improve results, with some users
ranking the citations-based GAT model higher. Moreover, the results of the two
GAT-based models were usually nearly identical, with differences being noticed
only in the ranking of the recommendations.

Furthermore, according to the users’ feedback, the confidence scores displayed
by the models were sometimes quite low, although the recommendations were of
good quality. This indicates a mismatch between the probability scores yielded
by the models, on the one hand, and the ratings from the user study, on the
other hand. More specifically, recommendations rated positively in the user
study still got fairly low confidence scores internally.

Another aspect noticed by a few of the participants is the strong focus on
semantic-web-related conferences (e.g such conferences were recommended even
if the topic of the paper was not related to this field), suggesting a bias
towards this domain in the training data. The lack of conferences from
different publishers, determined by the publisher-dependent dataset used, was
also among the weaknesses mentioned by some of the users. Lastly, even though
the models seem helpful in discovering interesting, less-known, or niche
academic venues which are thematically-fitting, some users would prefer
excluding suggestions concerning events organised either only once (e.g.
workshops), or locally (i.e. only in a certain geographic region). This is
determined by the researchers’ desire to not only be aware of more well-known
academic venues, but to also submit their work to prestigious conferences.

###  V-D Discussion of Results

Even though the performance metrics reported in Table III do not exceed a
recall of 0.580 and a MAP@10 of 0.336, these figures need to be analysed in a
broader context. Overall, 23.9% of the conference series from the test set do
not overlap with those in the training data, meaning that for these there are
no training signals. Since training features are generated only based on
previous publications, this implies that all the approaches discussed in this
paper cannot recommend conferences not contained in the training set.
Therefore, the maximum achievable recall and MAP@10 values are 0.761.

Overall, we found that co-authorship relations constitute a meaningful signal
for recommendations, and that a heterogeneous directed graph constitutes the
best training dataset for the GNN-based models. Additionally, we observed that
an attention mechanism which weighs neighbours differently based on their
contributions to the central node (i.e. input paper) outperforms other GNN-
based approaches at sampling neighbourhoods. Recommendations based on a graph
attention network trained on a heterogeneous graph achieve a recall@10 of
0.580 and a MAP@10 of 0.336, while the other models are clearly inferior,
obtaining at most a 9% lower recall and MAP, using a reinforcement learning
policy for training a node sampler, and a GCN-based aggregation function.

One major caveat of the quantitative evaluation is that, by design, only one
recommendation is counted as correct, since a paper can only be published at
one conference. Nonetheless, multiple suggestions can be suitable (e.g.
different machine learning conferences). Moreover, a paper might have been
rejected by a conference which is considered relevant by GraphConfRec.
However, in the current setup, these would be regarded as false predictions,
and precision is underestimated. Nevertheless, the results of the user study
show that GraphConfRec infers meaningful suggestions, as most respondents
assessed the recommendations to be of good or very good quality. In contrast
to the results of the quantitative experiments, which indicate that the _GAT
(heterogeneous graph)_ recommender achieves the highest performance in terms
of both metrics, the respondents of the user study rated the _HAN_ model
higher in the majority of cases. Moreover, they also appear to consider the
recommendations generated by _GAT (citations graphs)_ to be slightly more
suitable than those of the GAT model which incorporates authors in its
predictions. The feedback of the user study shows that, while the HAN-based
recommender achieves a lower performance on the test set, in a real-life
scenario, it offers more varied, suitable, and often less-known
recommendations than the GAT-based models.

##  VI Limitations and Future Work

Although the experiments show promising results, GraphConfRec suffers from
several limitations, for which we propose potential solutions in the following
section.

###  VI-A Dataset and Publisher Independent Recommendations

GraphConfRec recommends only conferences contained in SciGraph, thus
excluding, among others, proceedings published at ACM, IEEE, AAAI, or ACL
conferences. In turn, this assumes that authors firstly choose a publisher,
which is often not the case in practice, as indicated by some of the user
study respondents. Using a publisher-independent dataset, such as the
Microsoft Academic Knowledge Graph [41], could overcome this weakness. Another
solution would be to use SciGraph only for training the recommendation models,
and to recommend CfPs from a dataset like WikiCfP, instead of using the latter
only for augmenting the conferences with external data. While this would allow
the model to recommend new conferences that are not present in the training
dataset, and cannot currently be suggested, it would lead to a new set of
challenges, for example with regards to interlinking the datasets.

###  VI-B Refining Confidence Scores and Rankings

Some user study participants considered that the model’s confidence scores
were quite low, even when the suggestions were accurate. One solution would be
to change the presentation of the scores, e.g. by re-scaling the confidences
of the top 10 recommendations. Furthermore, some users have suggested that the
ranking of results could be improved, with more suitable recommendations being
placed higher in the list. This could be achieved by re-ranking only the
recommended results. For example, the results could be clustered based on
keywords describing conferences (e.g. extracted from CfPs categorizations),
and the confidences of recommendations added per cluster. Thus, the
conferences belonging to the cluster with the highest overall probability
could be recommended before the ones belonging to clusters with lower
confidences. The evaluation of the recommendation models shows that in most
cases, the correct conference is recommended among the first 3 results.
Moreover, for some topics, there might be less than 10 fitting conferences.
Therefore, under this approach, the model will learn to cluster and recommend
first the conferences most similar to the correct one.

###  VI-C Fine-tuning Recommendations based on Additional Features

Another limitation of GraphConfRec regards the quality of external information
provided in the user interface. Continuously crawling WikiCfP for information
about future conferences could ensure that users are always provided with up-
to-date information about the recommended conferences. Moreover, providing
more comprehensive ratings of all the suggested conferences and incorporating
different rankings, such as CORE181818http://portal.core.edu.au/conf-ranks/
(last accessed 23/04/2020), could provide an objective measure of
distinguishing between numerous conferences from different tiers. However,
removing niche conferences or workshops from the recommendation list could
reduce the diversity of results. Overall, such external information would
cater the individual needs of researchers by offering the option of fine-
tuning the results based on various features, including conference ranking,
submission deadline, or geo-based filtering.

###  VI-D Exploring Other Graph Neural Networks

Exploring other GNNs which handle heterogeneous graphs could represent another
direction for future research. In contrast to HAN, which uses meta-paths
manually defined by the user, the Graph Transformer Network proposed by Yun et
al. [42], can yield node representations without domain-specific graph pre-
processing required for constructing and learning meta-paths. A different GNN
for heterogeneous graphs, which avoids the weaknesses of handcrafted meta-
paths, was developed by Hong et al. [43]. Their model automatically extracts
information from heterogeneous graphs and learns meta-paths, by firstly
projecting the transformations between nodes of different types into a low-
dimensional entity space, and then, by aggregating this multi-relational
information of the projected neighbourhood using a GNN [43].

##  VII Conclusion

In this paper, we introduced GraphConfRec, a recommender system for
conferences, based on both a paper’s textual content, and its co-authorship
and citation networks. The system is intended for researchers searching for
conferences to publish at. The recommendations are computed using SciGraph,
with submission deadlines added from WikiCfP and conference rankings from
Google Scholar Metrics. We found that a graph attention network-based
recommender trained on a heterogeneous graph achieves the best performance,
with a recall@10 of 0.580 and a MAP@10 of 0.336. The user study showed that
GraphConfRec infers meaningful recommendations, its GNN-based predictions
achieving an overall average score of 3.7 out of 5. However, users consider
recommendations based on the heterogeneous graph attention network model to be
more diverse, and often, slightly more thematically-fitting for their papers
than those of the GAT-based recommender.

Although abstract, title, citations and co-authorships provide useful signals
for recommendation, other entities, such as keywords or affiliations, could be
incorporated to augment the data. One of the weaknesses of the current
approach is considering only referenced conference proceedings found in
SciGraph. Including information about cited books and journals, both from
SciGraph and other publishers, could improve the models’ performance to a
certain extent. Furthermore, a publisher-independent dataset could further
improve GraphConfRec’s performance and extent its usability to a larger group
of researchers which seek to publish in conference proceedings from other
institutions. Lastly, improving the ranking of the results, providing fine-
tuning of recommendations based on conference features, or exploring other GNN
models constitute potential avenues for future research.

In conclusion, while additional options should be explored in future work, the
results offer a first glance at how semantic data and graph neural networks
can be utilised to build a recommender system for conferences. Hence, it opens
promising future directions, such as extending the system into a full-fledged
recommendation engine for scientific publications.

## References

  * [1] M. Schmachtenberg, C. Bizer, and H. Paulheim, “Adoption of the linked data best practices in different topical domains,” in International Semantic Web Conference, pp. 245–260, Springer, 2014.
  * [2] A. Birukou, V. Bryl, K. Eckert, A. Gromyko, and M. Kaindl, “Springer LOD Conference Portal. Demo paper.,” International Semantic Web Conference (Posters, Demos & Industry Tracks), p. 4, 2017.
  * [3] A. Dimou, L. De Vocht, G. Van Grootel, L. Van Campe, J. Latour, E. Mannens, and R. Van de Walle, “Visualizing the information of a linked open data enabled research information system,” Procedia Computer Science, vol. 33, pp. 245–252, 2014.
  * [4] Y. Hu, K. Janowicz, G. McKenzie, K. Sengupta, and P. Hitzler, “A linked-data-driven and semantically-enabled journal portal for scientometrics,” in International Semantic Web Conference, pp. 114–129, Springer, 2013.
  * [5] A. Birukou, E. Blanzieri, and P. Giorgini, “A multi-agent system that facilitates scientific publications search,” in Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems, pp. 265–272, 2006.
  * [6] S. Li, P. Brusilovsky, S. Su, and X. Cheng, “Conference Paper Recommendation for Academic Conferences,” IEEE Access, vol. 6, pp. 17153–17164, 2018.
  * [7] A. Forrester, B.-C. Björk, and C. Tenopir, “New web services that help authors choose journals,” Learned Publishing, vol. 30, no. 4, pp. 281–287, 2017.
  * [8] A. Iana, S. Jung, P. Naeser, A. Birukou, S. Hertling, and H. Paulheim, “Building a Conference Recommender System Based on SciGraph and WikiCFP,” in Semantic Systems. The Power of AI and Knowledge Graphs, Lecture Notes in Computer Science, (Cham), pp. 117–123, Springer International Publishing, 2019.
  * [9] E. Medvet, A. Bartoli, and G. Piccinin, “Publication Venue Recommendation Based on Paper Abstract,” in 2014 IEEE 26th International Conference on Tools with Artificial Intelligence, (Limassol, Cyprus), pp. 1004–1010, IEEE, Nov. 2014.
  * [10] H. Luong, T. Huynh, S. Gauch, L. Do, and K. Hoang, “Publication Venue Recommendation Using Author Network’s Publication History,” in Intelligent Information and Database Systems, vol. 7198, pp. 426–435, Berlin, Heidelberg: Springer Berlin Heidelberg, 2012.
  * [11] J. Rollins, M. McCusker, J. Carlson, and J. Stroll, “Manuscript Matcher: A Content and Bibliometrics-based Scholarly Journal Recommendation System,” in BIR 2017 Workshop on Bibliometric-enhanced Information Retrieval, pp. 18–29, 2017.
  * [12] W. C. Janssen and K. Popat, “UpLib: a universal personal digital library system,” in Proceedings of the 2003 ACM symposium on Document engineering, pp. 234–242, 2003.
  * [13] J. Beel, B. Gipp, S. Langer, and C. Breitinger, “Research-paper recommender systems: a literature survey,” International Journal on Digital Libraries, vol. 17, pp. 305–338, Nov. 2016.
  * [14] T. Ghosal, A. Raj, A. Ekbal, S. Saha, and P. Bhattacharyya, “A deep multimodal investigation to determine the appropriateness of scholarly submissions,” in 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL), pp. 227–236, IEEE, 2019.
  * [15] N. Kang, M. A. Doornenbal, and R. J. Schijvenaars, “Elsevier Journal Finder: Recommending Journals for your Paper,” in Proceedings of the 9th ACM Conference on Recommender Systems - RecSys ’15, (Vienna, Austria), pp. 261–264, ACM Press, 2015.
  * [16] M. J. Schuemie and J. A. Kors, “Jane: suggesting journals, finding experts,” Bioinformatics, vol. 24, no. 5, pp. 727–728, 2008.  Publisher: Oxford University Press.
  * [17] K. Kobs, T. Koopmann, A. Zehe, D. Fernes, P. Krop, and A. Hotho, “Where to Submit? Helping Researchers to Choose the Right Venue,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp. 878–883, 2020.
  * [18] T. Pradhan and S. Pal, “CNAVER: A content and network-based academic venue recommender system,” Knowledge-Based Systems, vol. 189, p. 105092, 2020\.  Publisher: Elsevier.
  * [19] O. Küçüktunç, E. Saule, K. Kaya, and Ü. V. Çatalyürek, “TheAdvisor: a webservice for academic recommendation,” in Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries, pp. 433–434, 2013.
  * [20] I. Boukhris and R. Ayachi, “A novel personalized academic venue hybrid recommender,” in 2014 IEEE 15th International Symposium on Computational Intelligence and Informatics (CINTI), pp. 465–470, IEEE, 2014.
  * [21] T. Di Noia, R. Mirizzi, V. C. Ostuni, D. Romito, and M. Zanker, “Linked open data to support content-based recommender systems,” in Proceedings of the 8th international conference on semantic systems, pp. 1–8, 2012.
  * [22] B. Heitmann and C. Hayes, “Using linked data to build open, collaborative recommender systems,” in 2010 AAAI Spring Symposium Series, 2010\.
  * [23] P. Ristoski, E. Loza Mencía, and H. Paulheim, “A Hybrid Multi-strategy Recommender System Using Linked Open Data,” in Semantic Web Evaluation Challenge, vol. 475, pp. 150–156, Cham: Springer International Publishing, 2014.
  * [24] A. Bordes, X. Glorot, J. Weston, and Y. Bengio, “A semantic matching energy function for learning with multi-relational data: Application to word-sense disambiguation,” Machine Learning, vol. 94, pp. 233–259, Feb. 2014.
  * [25] P. Ristoski, J. Rosati, T. Di Noia, R. De Leone, and H. Paulheim, “RDF2Vec: RDF graph embeddings and their applications,” Semantic Web, vol. 10, no. 4, pp. 721–752, 2019.
  * [26] W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Advances in neural information processing systems, pp. 1024–1034, 2017.
  * [27] Z. Zhang, P. Cui, and W. Zhu, “Deep learning on graphs: A survey,” arXiv preprint arXiv:1812.04202, 2018.
  * [28] J. Zhou, G. Cui, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, “Graph neural networks: A review of methods and applications,” arXiv preprint arXiv:1812.08434, 2018.
  * [29] R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec, “Graph convolutional neural networks for web-scale recommender systems,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 974–983, 2018.
  * [30] F. Wenqi, Y. Ma, Q. Li, Y. He, E. Zhao, J. Tang, and D. Yin, “Graph Neural Networks for Social Recommendation,” in The World Wide Web Conference, 2019.
  * [31] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems, pp. 5998–6008, 2017.
  * [32] I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A Pretrained Language Model for Scientific Text,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), (Hong Kong, China), pp. 3613–3618, Association for Computational Linguistics, 2019.
  * [33] T. N. Kipf and M. Welling, “Semi-Supervised Classification with Graph Convolutional Networks,” arXiv preprint arXiv:1609.02907, Sept. 2016\.
  * [34] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  * [35] J. Oh, K. Cho, and J. Bruna, “Advancing GraphSAGE with A Data-Driven Node Sampling,” arXiv preprint arXiv:1904.12935, 2019.
  * [36] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph Attention Networks,” arXiv preprint arXiv:1710.10903, Oct. 2017.
  * [37] X. Wang, H. Ji, C. Shi, B. Wang, Y. Ye, P. Cui, and P. S. Yu, “Heterogeneous graph attention network,” in The World Wide Web Conference, pp. 2022–2032, 2019.
  * [38] C. Jeong, S. Jang, H. Shin, E. Park, and S. Choi, “A Context-Aware Citation Recommendation Model with BERT and Graph Convolutional Networks,” arXiv preprint arXiv:1903.06464, 2019.
  * [39] S. Pan, R. Hu, G. Long, J. Jiang, L. Yao, and C. Zhang, “Adversarially Regularized Graph Autoencoder for Graph Embedding,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, (Stockholm, Sweden), pp. 2609–2615, International Joint Conferences on Artificial Intelligence Organization, July 2018\.
  * [40] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, pp. 2672–2680, 2014.
  * [41] M. Färber, “The Microsoft Academic Knowledge Graph: A Linked Data Source with 8 Billion Triples of Scholarly Data,” in International Semantic Web Conference, pp. 113–129, Springer, 2019.
  * [42] S. Yun, M. Jeong, R. Kim, J. Kang, and H. J. Kim, “Graph Transformer Networks,” in Advances in Neural Information Processing Systems, pp. 11960–11970, 2019.
  * [43] H. Hong, H. Guo, Y. Lin, X. Yang, Z. Li, and J. Ye, “An Attention-based Graph Neural Network for Heterogeneous Structural Learning,” in Proceedings of AAAI conference (AAAI’20), p. 8, 2020.


# Notes onStar Formation

Mark R. Krumholz

Ginwidth=totalheight=keepaspectratio The Open Astrophysics Bookshelf

## Chapter 0

[display]


§ A YOUNG TYPE MAXIMAL INEQUALITY

We now begin our first steps towards a Fubini type theorem for rough integration of jointly controlled paths. Our approach is akin to the case of Young integration, whereby we first establish a "maximal inequality" over the discrete two-parameter integrals and then leverage this inequality to prove existence and uniqueness of the integral as the limit of the discrete two-parameter integrals over partitions with decreasing mesh size. A Fubini type theorem then follows from similar estimates to those used in proving the existence of the integral. We are also inspired by the sewing lemma of <cit.>, as can be seen in Lemma <ref> in the appendix, where the proof of the sewing lemma is formatted to mirror the same type of argument used here.

In comparison with <cit.> which also establishes a Fubini type theorem for rough integration, we are able to generalise to geometric driving rough paths of arbitrary $p$-variation and also remove the restriction of being smoothly approximable. The result to be given here also draws more parallels with the classical Fubini's theorem by providing a third integral constructed directly from the local approximations, serving as an analogue to the integral with respect to a product measure. We will refer to the iterated rough integrals as double integrals and refer to the third type of integral as the joint rough integral.

Borrowing from <cit.>, we will use grid-like partitions of $[s,t] \times [u,v]$ to refer to partitions of the form $\Dcal \times \Dcal'$ where $\Dcal$ and $\Dcal'$ are partitions of $[s,t]$ and $[u,v]$ respectively. For a function $\Omega : [0, T]^2 \times [0, T]^2 \rightarrow E$ on a Banach space $E$ and partitions $\Dcal = \{ s_0 < \dotsc < s_{m_0}\} \subset [s,t]$ and $\Dcal' = \{ u_0 < \dotsc < u_{n_0} \} \subset [u,v]$ we write

\[
\sum_{\Dcal \times \Dcal'}
\Omega
\sum_{m=1}^{m_0}
\sum_{n=1}^{n_0}
\Omega
\binom{s_{m}, s_{m+1}}{u_{n}, u_{n+1}}
\]

which will be used to write the discrete two-parameter integral over $\Dcal \times \Dcal'$ in the context of two parameter rough integration.

We follow the same type of argument as in the two dimensional Young-Towghi maximal inequality of <cit.> as presented in the appendix of <cit.>. The main idea is as follows: given a discrete integral over a partition, we select a point to remove from the partition and then observe the change in the discrete integral. The particular point we remove is carefully chosen to keep this change small. We then repeat until we are left with the trivial partition. To keep notation succinct, we introduce the following quantities.

Let $\{Y^{(i;j,k)}\}$ be a jointly $X$-controlled two-parameter path on $\Hom(V, V^*)$. Denote by $\Omega^Y: [0,T]^2 \times [0,T]^2 \rightarrow \Real$ the local approximation of the joint integral

\[
\Omega^Y
\binom{s,t}{u,v}
\sum_{j,k=0}^N
\Big(
\Big)
\Big(
\Big)
\sum_{j,k=0}^N
\Big(
\Big)
\Big(
\Big).
\]

Now define $\Gamma^Y: [0,T]^3 \times [0,T]^2 \rightarrow \Real$ by

\begin{equation*}
\Gamma^Y
\binom{s,s',t}{u,v}
\Omega^Y
\binom{\cdot,\cdot}{u,v}
\right)
\left(
\right)
\Omega^Y
\binom{s,s'}{u,v}
\Omega^Y
\binom{s',t}{u,v}
\Omega^Y
\binom{s,t}{u,v}
\end{equation*}

and $\Theta^Y: [0,T]^3 \times [0,T]^3 \rightarrow \Real$ by

\begin{equation*}
\Theta^Y
\binom{s,s',t}{u,u',v}
\Gamma^Y
\binom{s,s',t}{\cdot,\cdot}
\right)
\left(
\right)
\Gamma^Y
\binom{s,s',t}{u,u'}
\Gamma^Y
\binom{s,s',t}{u',v}
\Gamma^Y
\binom{s,s',t}{u,v},
\end{equation*}

where we recall that for $\Xi : [0, T]^2 \rightarrow E$ that $\delta \Xi_{s,s',t} = \Xi_{s,t} - \Xi_{s,s'} - \Xi_{s',t}$ for $0 \leq s < s' < t \leq T$.

We may also similarly define $\Gamma^Y: [0,T]^2 \times [0,T]^3 \rightarrow \Real$ by instead fixing the first pair of variables,

\[
\Gamma^Y
\binom{s,t}{u,u',v}
\Omega^Y
\binom{s,t}{\cdot,\cdot}
\right)
\left(
\right)
\]

and it is easily verified that we may alternatively define $\Theta^Y$ by

\[
\Theta^Y
\binom{s,s',t}{u,u',v}
\Gamma^Y
\binom{\cdot,\cdot}{u,u',v}
\right)
\left(
\right)
\]

due to the symmetry of $\Omega^Y$.

From Lemma <ref> we show that $\Gamma^Y$ and $\Theta^Y$ can be expressed in terms of first and second order remainders of $Y$.

Let $\{Y^{(i;j,k)}\}$ be a jointly $X$-controlled two-parameter path on $\Hom(V,V^*)$. Then the following identities hold:

\begin{align}
\label{Gamma identity}
\Gamma^Y
\binom{s,s',t}{u,v}
\sum_{j,k=0}^N
\Big(
\Big)
\Big(
\Big),
\\[0.5em]
\label{Theta identity}
\Theta^Y
\binom{s,s',t}{u,u',v}
\sum_{j,k=0}^N
\Rbf^{(1;j,k)}_{s,s';u,u'}
\Big(
\Big)
\Big(
\Big)
\end{align}

By definition we have

\begin{align*}
\Gamma^Y
\binom{s,s',t}{u,v}
\sum^N_{j,k=0}
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\\[0.5em]
\sum^N_{k=0}
\left(
\sum^N_{j=0}
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\right)
\Big(
\Big).
\end{align*}

Applying Lemma <ref> we then arrive at (<ref>). Similarly when we use this identity in the definition of $\Theta$ we get

\begin{align*}
\Theta^Y
\binom{s,s',t}{u,v}
\sum^N_{j,k=0}
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\\[0.5em]
\sum^N_{j=0}
\left(
\sum^N_{k=0}
\Big(
\Big)
\Big(
\Big)
\Big(
\Big)
\right)
\Big(
\Big).
\end{align*}

We recall now that Lemma <ref> states that for fixed $s,s' \in [0,T]$ the tuples $\left( R^{(1;j,0)}_{s,s'}, \dotsc, R^{(1;j,N)}_{s,s'}\right)$ are controlled paths with remainders $\left\{ \Rbf^{(1;j,k)}_{s,s';\cdot,\cdot} \right\}_{k=0}^N$, and so by applying Lemma <ref> again we arrive at (<ref>).

Now we work towards building a series of intermediate bounds, which will prove useful in proving both the maximal inequality and the rough Fubini type theorem. The cornerstone for these bounds is the following key estimate, which we make under assumptions of finite mixed variation.

Let $\{Y^{(i;j,k)}\}$ be a jointly $X$-controlled two-parameter path on $\Hom(V, V^*)$ and suppose that Condition <ref> holds and let $\alpha \in (1/\theta_*, 1)$. Consider partitions $\Dcal = \{s_{0}< \dotsc <s_{m_0}\} \subset [s,t]$ and $\Dcal' = \{u_{0} < \dotsc < u_{n_0} \} \subset [u,v]$, and for $n=1, \dotsc, n_0 - 1$ define $\Pcal^{(1; \Dcal, \Dcal')}_{\alpha, n}$ by

\[
\Pcal^{(1; \Dcal, \Dcal')}_{\alpha, n}
\sum^{m_0 -1}_{m = 1}
\left|
\Theta^Y
\binom{s_{m-1}, s_m, s_{m+1}}{u_{n-1}, u_n, u_{n+1}}
\right|^{\alpha}
\]

Then there exists $n^*$ such that

\begin{equation}\label{variation inequality}
\Pcal^{(1;\Dcal, \Dcal')}_{\alpha,n^*}
\leq
\frac{C(p,\alpha,\theta_*)}{(n_0 -1)^{\alpha \theta_*}}
\Vbf^{(1;p)}_{[s,t],[u,v]}(X, Y),
\end{equation}

where $C(p,\alpha,\theta_*)$ is such that $C(p,\alpha,\theta_*) \leq \pfloor^{2+\alpha \theta^*}$ and

\[
\Vbf^{(1;p)}_{[s,t],[u,v]}(X, Y)
\max_{j,k = 0, \dotsc, N}
\left\{
\right)
\right)
V^{q_j, q_k}_{[s,t]\times[u,v]}\left(
\Rbf^{(1; j, k)}
\right)
\right\}.
\]

Recall that $N = \pfloor - 1$, $p_j = p/(j+1)$ and $\frac{1}{p_j} + \frac{1}{q_j} = \theta_j \geq \theta_* > 1$. Since $\alpha < 1$ we have $\left(\sum_i |a_i|\right)^{\alpha} \leq \sum_i |a_i|^{\alpha}$, which gives us

\begin{align}
\nonumber
\Pcal^{(1; \Dcal,\Dcal')}_{\alpha, n}
\sum^N_{j,k=0}
\sum^{m_0 -1}_{m=1}
\left|
\Rbf^{(1; j, k)}_{s_{m-1}, s_m; u_{n-1}, u_n}
\right|^{\alpha}
\left|
X^{j+1}_{s_m, s_{m+1}}
\right|^{\alpha}
\left|
X^{k+1}_{u_n, u_{n+1}}
\right|^{\alpha}
\\[0.5em]
\nonumber
\sum^{N}_{j,k=0}
\left(
\sum^{m_0 -1}_{m=1}
\left|
\Rbf^{(1; j, k)}_{s_{m-1}, s_m; u_{n-1}, u_n}
\right|^{\alpha \theta_j q_j}
\right)^{1/\theta_j q_j}
\left(
\sum^{m_0 -1}_{m=1}
\left|
X^{j+1}_{s_m, s_{m+1}}
\right|^{\alpha \theta_j p_j}
\right)^{1/\theta_j p_j}
\left|
X^{k+1}_{u_n, u_{n+1}}
\right|^{\alpha}
\\[0.5em]
\nonumber
\sum^{N}_{j,k=0}
\left(
\sum^{m_0 -1}_{m=1}
\left|
\Rbf^{(1; j, k)}_{s_{m-1}, s_m; u_{n-1}, u_n}
\right|^{q_j}
\right)^{\alpha/q_j}
\left(
\sum^{m_0 -1}_{m=1}
\left|
X^{j+1}_{s_m, s_{m+1}}
\right|^{p_j}
\right)^{\alpha/p_j}
\left|
X^{k+1}_{u_n, u_{n+1}}
\right|^{\alpha}
\\[0.5em]
\label{Pcal inequality}
\sum^{N}_{j,k=0}
\left(
\sum^{m_0 -1}_{m=1}
\left|
\Rbf^{(1; j, k)}_{s_{m-1}, s_m; u_{n-1}, u_n}
\right|^{q_j}
\right)^{\alpha/q_j}
\left(
\right)^{\alpha}
\left|
X^{k+1}_{u_n, u_{n+1}}
\right|^{\alpha},
\end{align}

where we use Hölder's inequality in the second line, and in the third line we use that the $\ell^{\alpha \theta_j q_j}$-norm is dominated by the $\ell^{q_j}$-norm on $\Real^{m_0 - 1}$ since $\alpha \theta_j > 1$. To shorten future notation, define

\[
\left(
\sum^{m_0 -1}_{m=1}
\left|
\Rbf^{(1; j, k)}_{s_{m-1}, s_m; u_{n-1}, u_n}
\right|^{q_j}
\right)^{1/q_j}.
\]

For $j,k = 0 \dotsc, N$ define the sets

\[
\left\{
\; : \;
\left|
\left(
\right)
X^{k+1}_{u_n, u_{n+1}}
\right|
\max_{j',k'= 0 \dotsc, N}
\left|
\left(
\right)
X^{k'+1}_{u_n, u_{n+1}}
\right|
\right\},
\]

which are (possibly empty) subsets of $\{1, \dotsc, n_0 - 1\}$ containing $n$ such that the pair $(j,k)$ maximises the summands in (<ref>). Since each element of $\{1, \dotsc, n_0 - 1\}$ belongs to $B_{j,k}$ for at least one pair $(j,k)$, we have that
\sum_{j,k = 1}^{\pfloor}
\geq
n_0 - 1
and so there must exist at least one pair $(j_0, k_0)$ such that $|B_{j_0,k_0}| \geq (n_0 - 1)/ \pfloor^2$. Let $n^* \in B_{j_0, k_0}$ be such that

\[
\left|
\left(
X^{j_0 +1}
\right)
X^{k_0 +1}_{u_{n^*}, u_{n^*+1}}
\right|
\min_{n \in B_{j_0, k_0}}
\left|
\left(
X^{j_0 +1}
\right)
X^{k_0 +1}_{u_n, u_{n+1}}
\right|.
\]

Since $n^*$ minimises the above expression over $B_{j_0, k_0}$, we obtain the following:

\begin{align*}
\Pcal^{(1;\Dcal, \Dcal')}_{\alpha, n^*}
\pfloor^2
\left|
\left(
X^{j_0 +1}
\right)
X^{k_0 +1}_{u_{n^*}, u_{n^*+1}}
\right|^{\alpha}
\\[0.5em]
\pfloor^2
\left(
\left(
X^{j_0 +1}
\right)
\right)^{\alpha}
\left(
\prod_{n \in B_{j_0, k_0}}
\left|
\right|^{\alpha}
\left|
X^{k_0 +1}_{u_n, u_{n+1}}
\right|^{\alpha}
\right)^{1/|B_{j_0, k_0}|}.
\end{align*}

We now separate the product over $B_{j_0, k_0}$ above into two parts and work on them separately. By the arithmetic-geometric mean inequality we have

\begin{align*}
\left(
\prod_{n \in B_{j_0, k_0}}
\left|
\right|^{q_{k_0}}
\right)^{\alpha/q_{k_0}|B_{j_0, k_0}|}
\left(
\frac{1}{|B_{j_0, k_0}|}
\sum_{n \in B_{j_0, k_0}}
\left|
\right|^{q_{k_0}}
\right)^{\alpha/q_{k_0}}
\\[0.5em]
& \leq
\left(
\frac{\pfloor^2}{n_0-1}
\sum_{n =1}^{n_0 - 1}
\left(
\sum^{m_0 -1}_{m=1}
\left|
\Rbf^{(1; j_0, k_0)}_{s_{m-1}, s_m; u_{n-1}, u_n}
\right|^{q_{j_0}}
\right)^{{q_{k_0}}/{q_{j_0}}}
\right)^{\alpha/q_{k_0}}
\\[0.5em]
\left(
\frac{\pfloor^2}{n_0-1}
\right)^{\alpha/q_{k_0}}
\left(
V^{q_{j_0}, q_{k_0}}_{[s,t]\times[u,v]}
\left(
\Rbf^{(1; j_0, k_0)}
\right)
\right)^{\alpha}.
\end{align*}

Similarly, we again apply the AM-GM inequality to get

\begin{align*}
\left(
\prod_{n \in B_{j_0, k_0}}
\left|
X^{k_0 +1}_{u_n, u_{n+1}}
\right|^{p_{k_0}}
\right)^{\alpha/p_{k_0}|B_{j_0, k_0}|}
& \leq
\left(
\frac{1}{|B_{j_0, k_0}|}
\sum_{n \in B_{j_0, k_0}}
\left|
X^{k_0 +1}_{u_n, u_{n+1}}
\right|^{p_{k_0}}
\right)^{\alpha/p_{k_0}}
\\[0.5em]
& \leq
\left(
\frac{\pfloor^2}{n_0-1}
\right)^{\alpha/p_{k_0}}
\left(
\left(
X^{k_0 +1}
\right)
\right)^{\alpha}
\end{align*}

Combining these inequalities, we get

\begin{equation*}
\Pcal^{(1;\Dcal, \Dcal')}_{\alpha, n^*}
\leq
\frac{
\pfloor^{2 + \alpha \theta_{k_0}}
\left(
n_0 - 1
\right)^{\alpha \theta_{k_0}}
\left|
\left(
X^{j_0 +1}
\right)
\right|^{\alpha}
\left|
\left(
X^{k_0 +1}
\right)
\right|^{\alpha}
\left(
V^{q_{j_0}, q_{k_0}}_{[s,t]\times[u,v]}
\left(
\Rbf^{(1; j_0, k_0)}
\right)
\right)^{\alpha}
\end{equation*}

from which (<ref>) follows.

We will also want to consider the quantity

\[
\Pcal^{(2;\Dcal, \Dcal')}_{\alpha, m}
\sum^{n_0 -1}_{n = 1}
\left|
\Theta^Y
\binom{s_{m-1}, s_m, s_{m+1}}{u_{n-1}, u_n, u_{n+1}}
\right|^{\alpha},
\]

for which a calculation in the same vein tells us that there exists $m^* \in \{1, \dotsc, m_0 -1\}$ such that the inequality

\begin{equation}\label{variation inequality alt}
\Pcal^{(2; \Dcal,\Dcal')}_{\alpha, m^*}
\leq
\frac{C(p, \alpha, \theta)}{(m_0 -1)^{\alpha \theta_*}}
\Vbf^{(2;p)}_{[s,t],[u,v]}(X,Y)^{\alpha}
\end{equation}

holds, where $C(p, \alpha, \theta_*)$ is the same as in (<ref>) and

\[
\Vbf^{(2;p)}_{[s,t],[u,v]}(X,Y)
\max_{j,k = 0, \dotsc, N}
\left\{
\right)
\right)
V^{q_k, q_j}_{[u,v]\times[s,t]}\left(
\Rbf^{(2; k, j)}
\right)
\right\}
\]

The proof of this bound draws inspiration from the proof of Lemma 6.4 of <cit.> (original result from <cit.>) with modifications made to be used in the context of rough paths and jointly controlled paths.

Our strategy now is to obtain the maximal inequality by successively removing well chosen points from the discrete integrals until we are left with a discrete integral over the trivial partition. In the process of removing points, the following intermediate estimates become useful; which we obtain by using the previous bound of Lemma <ref> and methods from the proof of the same lemma.

Suppose that Condition <ref> holds. For partitions $\Dcal = \{s_0< \dotsc < s_{m_0} \} \subset [s,t]$ and $\Dcal' = \{u_0 < \dotsc < u_{n_0}\}\subset [u,v]$ define the differences

Δ^(1; m)_×'
∑_∖{s_m} ×'
s_m-1, s_m, s_m+1u_n-1, u_n

Δ^(2; n)_×'
s_m-1, s_mu_n-1, u_n, u_n+1

Then for any $\alpha \in (1/\theta_*, 1)$,

\begin{align}
\label{Gamma inequality 1}
\sum_{m = 1}^{m_0 -1}
\left|
\Delta^{(1;m)}_{\Dcal \times \{u, v\}}
\right|^{\alpha}
\sum^{N}_{j,k = 0}
\left|
X^{k+1}_{u, v}
\right|^{\alpha}
\left(
\right)^{\alpha}
\left(
\right)^{\alpha}
\left(
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}
(X, Y)
\right)^{\alpha}
\\[0.5em]
\label{Gamma inequality 2}
\sum_{n = 1}^{n_0 -1}
\left|
\Delta^{(2;n)}_{\{s,t\} \times \Dcal'}
\right|^{\alpha}
\sum^{N}_{j,k = 0}
\left|
X^{j+1}_{s, t}
\right|^{\alpha}
\left(
\right)^{\alpha}
\left(
\right)^{\alpha}
\left(
\eta^{(2;p, \alpha)}_{[s,t],[u,v]}
(X, Y)
\right)^{\alpha}
\end{align}

and there exists constant $C(p,\alpha,\theta_*)$ such that

\begin{align}
\label{Delta inequality 1}
\sum^{m_0 -1}_{m = 1}
\left|
\Delta^{(1; m)}_{\Dcal \times \Dcal'}
\Delta^{(1; m)}_{\Dcal \times \{u, v\}}
\right|^{\alpha}
\zeta\left(
\alpha \theta_*
\right)
\Vbf^{(1;p)}_{[s,t],[u,v]}(X,Y)^{\alpha}
\\[0.5em]
\label{Delta inequality 2}
\sum^{n_0 -1}_{n = 1}
\left|
\Delta^{(2; n)}_{\Dcal \times \Dcal'}
\Delta^{(2; n)}_{\{s,t\} \times \Dcal'}
\right|^{\alpha}
\zeta\left(
\alpha \theta_*
\right)
\Vbf^{(2;p)}_{[s,t],[u,v]}(X,Y)^{\alpha}
\end{align}

where $\zeta$ is the Riemann zeta function.

For (<ref>) and (<ref>) we use similar calculations to those in Lemma <ref>,

\begin{align*}
\sum_{m = 1}^{m_0 -1}
\left|
\Delta^{(1;m)}_{\Dcal \times \{u, v\}}
\right|^{\alpha}
\sum_{m = 1}^{m_0 -1}
\left|
\Gamma^Y
\binom{s_{m-1}, s_{m}, s_{m+1}}{u, v}
\right|^{\alpha}
\\[0.5em]
\sum^{N}_{j,k = 0}
\sum_{m = 1}^{m_0 -1}
\left|
R^{(2;k,j)}_{u; s_{m-1}, s_{m}}
\right|^{\alpha}
\left|
X^{j+1}_{s_{m}, s_{m+1}}
\right|^{\alpha}
\left|
X^{k+1}_{u, v}
\right|^{\alpha}
\\[0.5em]
\sum^{N}_{j,k = 0}
\left|
X^{k+1}_{u, v}
\right|^{\alpha}
\left(
\sum_{m = 1}^{m_0 -1}
\left|
X^{j+1}_{s_{m}, s_{m+1}}
\right|^{p_j}
\right)^{\alpha/p_j}
\left(
\sum_{m = 1}^{m_0 -1}
\left|
R^{(2;k,j)}_{u; s_{m-1}, s_{m}}
\right|^{q_j}
\right)^{\alpha/q_j}
\\[0.5em]
\sum^{N}_{j,k = 0}
\left|
X^{k+1}_{u, v}
\right|^{\alpha}
\left(
\right)^{\alpha}
\left(
\right)^{\alpha}.
\end{align*}

The inequality (<ref>) follows similarly so we omit the calculation.

Likewise the derivation of (<ref>) also follows similarly to that of (<ref>) so we only show the latter. We first note that for $n \in \{1, \dotsc, n_0 - 1\}$ we have

\[
\Delta^{(1; m)}_{\Dcal \times \Dcal'}
\Delta^{(1; m)}_{\Dcal \times \Dcal'\setminus \{u_{n}\}}
\Theta^Y
\binom{s_{m-1}, s_{m}, s_{m+1}}{u_{n-1}, u_{n}, u_{n+1}}.
\]

By Lemma <ref> we may iteratively remove $n^*$ such that (<ref>) is satisfied for each new partition formed from these removals. Denote the sequence of $n^*$ by $(n^*_l)_{l=1}^{n_0 - 1}$ and the resulting partitions $\Dcal'_{l} := \Dcal'_{l-1}\setminus \{u_{n^*_l}\}$ with $\Dcal'_0 := \Dcal'$. We obtain a bound on the first term in the above inequality:

\begin{align*}
\sum^{m_0 -1}_{m = 1}
\left|
\Delta^{(1; m)}_{\Dcal \times \Dcal'}
\Delta^{(1; m)}_{\Dcal \times \{u, v\}}
\right|^{\alpha}
\sum^{m_0 -1}_{m = 1}
\sum^{n_0 - 1}_{l=1}
\left|
\Delta^{(1; m)}_{\Dcal \times \Dcal'_{l-1}}
\Delta^{(1; m)}_{\Dcal \times \Dcal'_{l}}
\right|^{\alpha}
\\[0.5em]
\sum^{n_0 - 1}_{l=1}
\frac{C(p,\alpha,\theta_*)}{(n_0 -l)^{\alpha \theta_*}}
\Vbf^{(1;p)}_{[s,t],[u,v]}(X,Y)^{\alpha}
\\[0.5em]
\zeta\left(
\alpha \theta_*
\right)
\Vbf^{(1;p)}_{[s,t],[u,v]}(X,Y)^{\alpha}
\end{align*}

where we used Lemma <ref> in the second line and $\zeta$ is the Riemann zeta function.

Our first use of these intermediate estimates is to place bounds on comparisons involving discrete integrals with one or both of the one-dimensional partitions in grid-like partition being trivial.

Assume that Condition <ref> holds. Then for any $\alpha \in (1/\theta_*, 1)$

\begin{align}
\label{endpoints bound 1}
\left|
\sum_{\Dcal \times \{u,v\}}
\Omega^Y
\sum_{\{s,t\} \times \{u,v\}}
\Omega^Y
\right|
\zeta\left(
\frac{1}{\alpha}
\right)
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}
(X, Y)
\\[0.5em]
\label{endpoints bound 2}
\left|
\sum_{\{s,t\} \times \Dcal'}
\Omega^Y
\sum_{\{s,t\} \times \{u,v\}}
\Omega^Y
\right|
\zeta\left(
\frac{1}{\alpha}
\right)
\eta^{(2;p, \alpha)}_{[s,t],[u,v]}
(X, Y)
\end{align}

and there exists $C'(p, \alpha, \theta_*)$ such that

\begin{equation}\label{endpoints bound}
\left|
\sum_{\Dcal \times \Dcal'}
\Omega^Y
\sum_{\Dcal \times \{u, v\}}
\Omega^Y
\sum_{\{s, t\} \times \Dcal'}
\Omega^Y
\sum_{\{s, t\} \times \{u, v\}}
\Omega^Y
\right|
\leq
C'(p, \alpha, \theta_*)
\Vbf^{(1;p)}_{[s,t], [u,v]}(X,Y).
\end{equation}

We prove these inequalities by iteratively removing points and then applying inequalities from Lemma <ref>.

Let $\Dcal_0 := \Dcal$ and for $l = 1, \dotsc, m_0 -1$, iteratively define $m_l$ to be the index minimising $\left|\Delta^{(1;m)}_{\Dcal_{l-1} \times \{u,v\}}\right|$ where we recursively define $\Dcal_{l} = \Dcal_{l-1} \setminus \{s_{m_{l}}\}$. By our choice of $m_l$ and applying the inequality (<ref>), we have

\begin{align*}
\left|
\Delta^{(1;m_l)}_{\Dcal_{l-1} \times \{u,v\}}
\right|^{\alpha}
\frac{1}{m_0 - l}
\sum_{s_{m} \in \Dcal_{l-1}}
\left|
\Delta^{(1;m)}_{\Dcal_{l-1} \times \{u,v\}}
\right|^{\alpha}
\\[0.5em]
\frac{1}{m_0 - l}
\left(
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}
(X, Y)
\right)^{\alpha}.
\end{align*}

This then gives us

\begin{align*}
\left|
\sum_{\Dcal \times \{u,v\}}
\Omega^Y
\sum_{\{s,t\} \times \{u,v\}}
\Omega^Y
\right|
\sum_{l = 1}^{m_0 - 1}
\left|
\Delta^{(1;m_{l})}_{\Dcal_{l-1} \times \{u,v\}}
\right|
\\[0.5em]
\zeta\left(
\frac{1}{\alpha}
\right)
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}
(X, Y).
\end{align*}

The inequality (<ref>) follows similarly where we instead successively remove points from $\Dcal'$ to minimise $\left|\Delta^{(2;n)}_{\{s,t\} \times \Dcal_{l-1}'}\right|$, where $\Dcal_{l}'$ are defined analogously.

For (<ref>) we again remove points from $\Dcal$, so define $\Dcal_l$ as before but now we choose $m_l$ to minimise $\left|\Delta^{(1;m)}_{\Dcal_{l-1} \times \Dcal'}- \Delta^{(1;m)}_{\Dcal_{l-1} \times \{u,v\}}\right|$. Similarly to before, we have

\begin{align*}
\left|
\Delta^{(1;m_l)}_{\Dcal_{l-1} \times \Dcal'}
\Delta^{(1;m_l)}_{\Dcal_{l-1} \times \{u,v\}}
\right|^{\alpha}
\frac{1}{m_0 - l}
\sum_{s_{m} \in \Dcal_{l-1}}
\left|
\Delta^{(1;m)}_{\Dcal_{l-1} \times \Dcal'}
\Delta^{(1;m)}_{\Dcal_{l-1} \times \{u,v\}}
\right|^{\alpha}
\\[0.5em]
\frac{C(p,\alpha,\theta_*)}{m_0 - l}
\zeta\left(
\alpha \theta_*
\right)
\left(
\Vbf^{(1;p)}_{[s,t],[u,v]}
(X, Y)
\right)^{\alpha},
\end{align*}

where we used (<ref>) in the second line. We note that

\[
\sum_{l=0}^{m_0-1}
\Delta^{(1;m_l)}_{\Dcal_l \times \Dcal'}
\Delta^{(1;m_l)}_{\Dcal_l \times \{u,v\}}
\sum_{\Dcal \times \Dcal'}
\Omega^Y
\sum_{\Dcal \times \{u, v\}}
\Omega^Y
\sum_{\{s, t\} \times \Dcal'}
\Omega^Y
\sum_{\{s, t\} \times \{u, v\}}
\Omega^Y.
\]

Combining these we then arrive at the following inequality:

\begin{align*}
\left|
\sum_{l=0}^{m_0-1}
\Delta^{(1;m_l)}_{\Dcal_l \times \Dcal'}
\Delta^{(1;m_l)}_{\Dcal_l \times \{u,v\}}
\right|
\sum_{l=0}^{m_0-1}
\left|
\Delta^{(1;m_l)}_{\Dcal_l \times \Dcal'}
\Delta^{(1;m_l)}_{\Dcal_l \times \{u,v\}}
\right|
\\[0.5em]
\zeta\left(
\frac{1}{\alpha}
\right)
\left(
\zeta(\alpha\theta_*)
\right)^{1/\alpha}
\Vbf^{(1;p)}_{[s,t], [u,v]}(X,Y)
\end{align*}

which gives us (<ref>) with $C'(p, \alpha, \theta_*) = \zeta(1/\alpha)(C(p,\alpha,\theta_*)\zeta(\alpha\theta_*))^{1/\alpha}$.

We may also obtain (<ref>) in a similar fashion by removing points from $\Dcal'$ which would then lead to the inequality

\begin{equation}\label{endpoints bound alt}
\left|
\sum_{\Dcal \times \Dcal'}
\Omega^Y
\sum_{\Dcal \times \{u, v\}}
\Omega^Y
\sum_{\{s, t\} \times \Dcal'}
\Omega^Y
\sum_{\{s, t\} \times \{u, v\}}
\Omega^Y
\right|
\leq
C'(p, \alpha, \theta_*)
\Vbf^{(2;p)}_{[s,t], [u,v]}(X,Y),
\end{equation}

with the same $C'(p, \alpha, \theta_*)$. This allows us to then write

\begin{equation}\label{endpoints bound final}
\left|
\sum_{\Dcal \times \Dcal'}
\Omega^Y
\sum_{\Dcal \times \{u, v\}}
\Omega^Y
\sum_{\{s, t\} \times \Dcal'}
\Omega^Y
\sum_{\{s, t\} \times \{u, v\}}
\Omega^Y
\right|
\leq
C'(p, \alpha, \theta_*)
\Vbf^{(p; X, Y)}_{[s,t], [u,v]},
\end{equation}

\[
\Vbf^{(p; X, Y)}_{[s,t], [u,v]}
\min \left(
\Vbf^{(1;p)}_{[s,t], [u,v]}(X,Y),
\Vbf^{(2;p)}_{[s,t], [u,v]}(X,Y)
\right)
\]

From these bounds the following maximal inequality is easily obtainable.

Suppose that Condition <ref> holds. Then for $\alpha \in (1/\theta_*, 1)$ there exists constant $C''(p, \alpha, \theta_*)$ such that for partitions $\Dcal = \{s_0 < \dotsc < s_{m_0}\} \subset [s,t]$ and $\Dcal' = \{u_0 < \dotsc < u_{n_0}\} \subset [u,v]$ we have the bound

\begin{equation}\label{maximal inequality}
\left|
\sum_{\Dcal \times \Dcal'}
\Omega^Y
\Omega^Y
\binom{s,t}{u,v}
\right|
\leq
C''(p, \alpha, \theta_*)
\left(
\Vbf^{(p; X, Y)}_{[s,t], [u,v]}
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}
(X, Y)
\eta^{(2;p, \alpha)}_{[s,t],[u,v]}
(X, Y)
\right)
\end{equation}

This follows immediately from Lemma <ref> and writing

∑_×{u, v}
∑_{s, t} ×'
∑_{s, t} ×{u, v}

∑_{s,t} ×{u,v}
∑_{s,t} ×'
∑_{s,t} ×{u,v}

To end this section, we give a lemma which provides a bound on the difference of the sum of local approximations when a point is selectively removed from one of the one-dimensional partitions. We note that it is also possible to prove the maximal inequality directly from this lemma.

Assume Condition (<ref>) holds. For $\alpha \in (1/\theta_*, 1)$ there exists $m^* \in \{1, \dotsc, m_0 -1 \}$, $n^* \in \{1, \dotsc, n_0 -1 \}$, and constant $C''' = C'''(p, \alpha, \theta_*)$ such that we have

\begin{align}
\label{removal of a point inequality}
\left|
\sum_{\Dcal \times \Dcal'}
\Omega^Y
\sum_{\Dcal\setminus\{s_{m^*}\} \times \Dcal'}
\Omega^Y
\right|
\left(
\frac{1}{m_0 - 1}
\right)^{1/\alpha}
\left(
\Vbf^{(1;p)}_{[s,t],[u,v]}(X, Y)
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}(X, Y)
\right)
\\[0.5em]
\label{removal of a point inequality alt}
\left|
\sum_{\Dcal \times \Dcal'}
\Omega^Y
\sum_{\Dcal \times \Dcal'\setminus\{u_{n^*}\}}
\Omega^Y
\right|
\left(
\frac{1}{n_0 - 1}
\right)^{1/\alpha}
\left(
\Vbf^{(2;p)}_{[s,t],[u,v]}(X, Y)
\eta^{(2;p, \alpha)}_{[s,t],[u,v]}(X, Y)
\right).
\end{align}

The proof is similar to that of Lemma <ref> and again we omit proof of the second inequality which follows similarly to the first. We choose $m^*$ to minimise $\left| \Delta^{(1;m)}_{\Dcal \times \Dcal'}\right|$ and write

\begin{align*}
\left|
\Delta^{(1;m^*)}_{\Dcal \times \Dcal'}
\right|^{\alpha}
\frac{1}{m_0 - 1}
\sum_{m=1}^{m_0 - 1}
\left|
\Delta^{(1;m)}_{\Dcal \times \Dcal'}
\right|^{\alpha}
\\[0.5em]
\frac{1}{m_0 - 1}
\sum_{m=1}^{m_0 - 1}
\left(
\left|
\Delta^{(1;m)}_{\Dcal \times \Dcal'}
\Delta^{(1;m)}_{\Dcal \times \{u,v\}}
\right|^{\alpha}
\left|
\Delta^{(1;m)}_{\Dcal \times \{u,v\}}
\right|^{\alpha}
\right)
\\[0.5em]
\frac{1}{m_0 - 1}
\left(
C(p, \alpha, \theta_*)
\zeta(\alpha \theta_*)
\Vbf^{(1;p)}_{[s,t],[u,v]}(X, Y)^{\alpha}
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}(X, Y)^{\alpha}
\right)
\\[0.5em]
\frac{2^{1-\alpha}}{m_0 - 1}
\left(
\left(
C(p, \alpha, \theta_*)
\zeta(\alpha \theta_*)
\right)^{1/\alpha}
\Vbf^{(1;p)}_{[s,t],[u,v]}(X, Y)
\eta^{(1;p, \alpha)}_{[s,t],[u,v]}(X, Y)
\right)^{\alpha},
\end{align*}

where we used Lemma <ref> in the second last line and Hölder's inequality on the second. Taking exponent $1/\alpha$ then completes the argument.