Evolution of Real-world Hypergraphs: Patterns and Models without Oracles

Yunbum Kook
Dept. of Mathematical Sciences, KAIST

Jihoon Ko
Graduate School of AI, KAIST

Kijung Shin
Graduate School of AI & School of EE, KAIST

What kind of macroscopic structural and dynamical patterns can we observe in real-world hypergraphs?
What can be underlying local dynamics on individuals, which ultimately lead to the observed patterns, beyond apparently random evolution?

Graphs, which provide effective ways to represent pairwise interactions among entities, fail to represent group interactions (e.g., collaboration of three or more researchers, etc.).
Regarded as a generalization of graphs, hypergraphs allowing for various sizes of edges prove fruitful in addressing this limitation.
The increased complexity, however, makes it challenging to understand hypergraphs as thoroughly as graphs.

In this work, we closely examine seven structural and dynamical properties of real hypergraphs from six domains.
To this end, we define new measures, extend notions of common graph properties to hypergraphs, and assess the significance of observed patterns by comparison with a null model and statistical tests.

We also propose , a stochastic model for generating realistic hypergraphs.
Its merits are three-fold:
(a) Realistic: it successfully reproduces all seven patterns, in addition to five patterns established in previous studies,
(b) Self-contained: unlike previously proposed models, it does not rely on oracles (i.e., unexplainable external information) at all, and it is parameterized by just two scalars,
and (c) Emergent: it relies on simple and interpretable mechanisms on individual entities, which do not trivially enforce but surprisingly lead to macroscopic properties.

§ INTRODUCTION

Which structural patterns do real-world hypergraphs have and how do they evolve over time?
Are there simple mechanisms on individual nodes that these patterns emerge from?

Datasets based on relationships between two objects have naturally arisen from a wide range of domains in the real world: friendships between two users in online social networks, hyperlinks from a web page to another, and citations from a publication to another, to name a few.
Graph representations give rise to an easy and extensive analysis of this type of relational datasets, and they have been used to understand such datasets in many respects.

A thorough understanding of those datasets via graph analysis has resulted in insights behind them and facilitated the development of effective algorithms building on the insights.
Some of the well-known static properties are a small-world phenomenon, also known as six degrees of separation <cit.>, and power law distributions of spectra <cit.> and degrees <cit.> of graphs in various domains.
Temporal properties prevalent in real graphs include triadic closure <cit.>, temporal locality <cit.>, densification, and shrinking diameter over time <cit.>.
These informative properties actually serve as useful tools for designing and analyzing graph algorithms <cit.>.

Previous studies on fundamental mechanisms leading to these patterns have also proceeded with the development of generative models for realistic graphs.
Various models <cit.> have been proposed to reproduce the examined patterns and lent themselves to some applications, such as simulations <cit.>, benchmarking <cit.>, and sampling <cit.>.

An example hypergraph with five nodes ($v_{1}$, $v_{2}$, $v_{3}$, $v_{4}$, and $v_{5}$) and four hyperedges ($e_{1}$, $e_{2}$, $e_{3}$, and $e_{4}$). Note that different hyperedges may contain different numbers of nodes.

generates realistic hypergraphs.
In the first row, we observe the heavy-tailed distributions of (S1) degrees, (S2) hyperedge sizes, (S3) intersection sizes, and (S4) singular values of incidence matrices; and (T1) diminishing overlaps of hyperedges, (T2) increasing edge density (more clear in the other datasets), and (T3) shrinking diameter in real-world hypergraphs.
In the second row,
successfully reproduces all the seven patterns. See Sections <ref> and <ref> for details.

4cStructural Patterns
3cDynamic Patterns

Hyperedge Sizes
Intersection Sizes
Singular Values
Intersecting Pairs
Edge Density

[origin=l]90Real Data

Not all relational networks are restricted to have only pairwise relationships, and polyadic relationships are also ubiquitous <cit.>;
for instance, publishing a paper may involve three or more authors, and an online group chat may involve tens of participants.

Hypergraphs <cit.> are a natural extension of the conventional notion of graphs by allowing for various sizes of edges.
Formally, a hypergraph consists of a set of nodes and a set of hyperedges, where each hyperedge is a non-empty subset containing any number of nodes.
They can represent high-order interactions (i.e., interactions among any number of objects), not just between two as in the graphs.

Taking into account polyadic interactions proves fruitful and indeed inevitable for challenging tasks.
According to <cit.>, the harder tasks are, the more useful leveraging information on high-order interactions is, and a simple reduction from high-order to pairwise interactions may lead to significant performance degradation.
Hence, hypergraphs, which naturally represent such complex interplays, have drawn considerable attention in many domains, including  computer vision <cit.>, recommendation <cit.>, and graph learning <cit.>.

Despite its importance, the understanding of real hypergraphs is still not as concrete as that of real graphs.
The daunting complexity due to the variability in hyperedge sizes prevents straightforward extensions of concepts and tools used for graphs to hypergraphs.
Nevertheless, there have been fruitful and insightful attempts.
In <cit.>, hypergraphs are decomposed into conventional graphs, and well-known properties of graphs on the decomposed graphs are examined.
In <cit.>, the triadic closure theory is extended to hypergraphs, and in <cit.>, repetitive patterns of hyperedges are investigated in terms of subset correlation and recency bias.

Driven by the importance of hypergraphs and of extensive studies, we scrutinize additional four structural and three dynamical patterns inherent in real-world hypergraphs at the macroscopic level.
To this end, we come up with measures to capture new aspects, revisit well-known properties of ordinary graphs, and study their hypergraph analogs.
We thoroughly validate the significance of each observed pattern by relying on qualitative and quantitative analyses.

The established structural and dynamical patterns constitute a meaningful step toward advancing the partial understanding of real-world hypergraphs.
For structural patterns, we suggest that distributions of (S1) degrees
, (S2) hyperedge sizes, (S3) intersection sizes of hyperedges, and (S4) singular values of incidence matrices
fall under the class of heavy-tailed distributions, of which the first three and the last are close to a truncated power law and log-normal distribution, respectively, among probable candidates.
For dynamical patterns, we observe that, over time, (T1) the overlapping of hyperedges becomes less frequent,
(T2) the number of hyperedges grows faster than that of nodes (i.e., densification), and (T3) the distances between nodes decrease (i.e., shrinking diameter).

Using these patterns as criteria, we propose a stochastic model (Hypergraph Forest Fire) for realistic hypergraph generation.
In , where a hypergraph grows with new nodes, each new node goes through two stages.
In the first stage, a node is randomly chosen and a `forest fire' starts there.
The forest fire is spread through existing hyperedges stochastically, and the new node forms a hyperedge with each of the burned nodes.
In the second stage, each new hyperedge expands similarly through a forest fire.

The benefits of are three-fold.
First, successfully reproduces all seven observed patterns, whereas previously proposed models focus only on a narrow scope of patterns.
We also apply the decomposition technique <cit.> to generated hypergraphs and confirm the five known properties of decomposed graphs.
Second, while the previous models rely on unexplainable external information (e.g., the number of new hyperedges along with each new node <cit.> and the size of each new hyperedge <cit.> with the number of new nodes in it <cit.>),
requires no such oracles, being parameterized simply by two scalars: burning and expanding probabilities.
Lastly, leads to a deeper understanding of complex systems, giving a simple underlying mechanism that imposes the non-trivial macroscopic patterns.

As in the exploration of real hypergraphs, we validate through quantitative and qualitative analyses.
We also explore its parameter space and suggest values of parameters for generating realistic hypergraphs.

Our contributions are summarized as follows:

* Establishment of structural and dynamical patterns in real-world hypergraphs.

* Structural: Heavy-tailed distributions of degrees, hyperedge sizes, intersection sizes, and singular values of incidence matrices.

* Dynamical: Diminishing overlaps of hyperedges, densification, and shrinking diameter.

* Stochastic model for hypergraph generation.

* Realistic: It exhibits all seven observed patterns and the five structural patterns reported in the previous study.

* Self-contained: It does not rely on oracles or external information, and it is parameterized by just two scalars.

* Emergent: Its simple and interpretable mechanisms on individual nodes non-trivially produce the examined patterns at the macroscopic level.

Reproducibility: We make the source code and datasets used in this work publicly available at <https://git.io/JUfAr>.

In Section <ref>, we survey a line of research on
properties and generative models of real-world (hyper)graphs.
In Section <ref>, we introduce some notations and concepts.
In Section <ref>, we examine the structural and dynamical patterns in real-world hypergraphs.
In Section <ref>, we propose our model and validate it.
We conclude our work in Section <ref>.

§ RELATED WORK

Below, we review previous work on patterns in real-world hypergraphs and generation of realistic (hyper)graphs.

§.§ Patterns in Real-world Hypergraphs.

There have been considerable efforts to find out fundamental properties in the structures and dynamics of real-world hypergraphs.
Structural patterns are investigated in static hypergraphs or a few snapshots of evolving hypergraphs. Dynamical patterns are investigated in time-evolving hypergraphs.

Instead of dealing directly with the complexity of hypergraphs, <cit.> reduce  hypergraphs to a set of ordinary graphs, which contains partial or all information on the hypergraphs, and then identify the properties of the graphs, using well-defined graph measures.
The most basic way is to convert a hypergraph into the graph with the same set of nodes and the set of edges induced by replacing each hyperedge with a clique <cit.>, referred to as the $2$-projected graph.
On $2$-projected graphs, <cit.> investigates simplicial closure, the extension of triadic closure to hypergraphs, and <cit.> defines sub-hypergraph centrality and extends the notion of clustering coefficients to hypergraphs.
This simple abstraction is extended to $n$-projected graphs <cit.> and $n$-level decomposed graphs <cit.>, which aim to capture the interactions between the subsets of $n$ nodes in distinct ways.
Especially for the $n$-level decomposed graphs, we elaborate on their definition in Section <ref>.
Based on these alternatives, <cit.> shows that the $n$-level decomposed graphs obtained from real hypergraphs retain five structural patterns: giant connected components, heavy-tailed degree distributions, small diameters, high clustering coefficients, and skewed singular-value distributions.
Besides these approaches,
<cit.> focuses on local structural patterns, specifically the patterns of interactions among three hyperedges; and
<cit.> focuses on dynamical patterns regarding the subset correlation, recency bias, and repeat patterns of hyperedges.
More patterns concerning diffusion, synchronization, and games are surveyed in <cit.>.
In this work, we study seven additional generic patterns in real hypergraphs at the macroscopic level.

§.§ Generation of Realistic (Hyper)graphs.

A number of generative models have been proposed for realistic graph generation.
For example, preferential attachment <cit.> and copying models <cit.> provide simple underlying principles leading to the power law in degree distributions.
The forest fire model <cit.> reproduces two dynamical patterns in graphs: densification (i.e., increase in the number of edges per node) and shrinking diameter (i.e., decrease in the the distances between nodes) over time.
The key idea behind the model is to connect a new node with a random node and the nodes burned by a `forest fire', which starts at the random node and is spread stochastically along the edges.
The Kronecker model <cit.> is a tractable model that enjoys a well-developed theory for the Kronecker product and thus provides theoretical guarantees as well as reproduce several established patterns.
Darwini <cit.> aims to generate realistic large-scale graphs whose distributions of degrees and clustering coefficients fit explicitly given distributions.
GraphRNN <cit.> takes a machine learning approach to train an autoregressive model generating graphs structurally similar to a given set of graphs.

Compared to realistic graph generation, relatively little attention has been paid to realistic hypergraph generation.
The stochastic model proposed by <cit.> takes into consideration recurrent patterns of hyperedges;
however, it requires an oracle which provides the sizes of the next hyperedges, and the number of new nodes in them.
The generative model suggested in <cit.> extends the preferential attachment model to hypergraphs. In the model called , for each new node, the group of nodes forming a hyperedge with the new node is chosen randomly, proportionally to the number of existing hyperedges that contain the group.
While generates a hypergraph whose $n$-decomposed graphs exhibit the five empirical structural patterns, it requires in advance the distribution of hyperedge sizes and the number of hyperedges related to each new node.
However, our model described in Section <ref> is able to reproduce the seven patterns examined in this paper without any oracles, and its decomposed graphs also maintain the five structural patterns.

§ PRELIMINARIES

Structural properties shown in six real datasets: Heavy-tailed distributions of four quantities.
The first three properties mostly show well-suited straight lines on the log-log scale, implying the potential for a stronger claim: they follow a power law distribution.
This tendency is more apparent in larger hypergraphs.
See Section <ref> for details.

Data      (smallest)                     (largest)

[origin=l]90$ $ $ $ $ $ Degrees

[origin=l]90$ $ $ $ Edge Sizes

[origin=l]90Intersection Sizes

[origin=l]90Singular Values

In this section, we introduce some basic notations and important concepts used throughout the paper.

A hypergraph $G=(V, E)$ consists of a set $V$ of nodes and a set $E\subseteq 2^V$ of hyperedges.
Each hyperedge $e$ is a subset of $V$, and we define the size of a hyperedge $e$ as the number $|e|$ of nodes in the hyperedge.
Note that conventional graphs are a special case of hypergraphs where the sizes of all hyperedges are two.
The degree of a node $v$, denoted by $deg(v)$, is defined as the number of hyperedges containing $v$.
The neighborhood of a node $v$, denoted by $N(v)$, is defined as the set of neighbors, each of which is contained together with $v$ in one or more hyperedges.

Incidence Matrix.

The incidence matrix $I \in \{0, 1\}^{|V|\times |E|}$ of a graph $G=(V, E)$ indicates the membership of the nodes $V$ in the hyperedges $E$.
Each $(i,j)$-th entry $I_{ij}$ of $I$ is $1$ if and only if the $j^{th}$ hyperedge in $E$ contains the $i^{th}$ node in $V$.

Effective Diameter.

A path in a hypergraph is a sequence of hyperedges in which any two immediate hyperedges have a non-empty intersection and the length of the path is the length of the sequence.
The distance between two nodes is defined as the length of a shortest path in which the first and the last hyperedges include one of the two nodes and the other, respectively.
The diameter of the hypergraph is a maximum distance between any pairs of nodes.
Since any disconnected nodes in a hypergraph makes the diameter infinite, we consider the effective diameter <cit.>, which is defined as the smallest $d$ such that the paths of length at most $d$ connect $90\%$ of all reachable pairs of nodes.[Linear interpolation is used to find such $d$.]

Heavy-tailed Distribution.

Tails of heavy-tailed distributions decay slower than exponential distributions (i.e., they are not exponentially bounded).
Power law distributions are typical examples of the heavy-tailed distributions.
For instance, a discrete power law distribution $P$ of a random variable $X$ satisfies, possibly in a limited range, the relationship $P(X) \propto 1/|X|^{\alpha}$ for some constant $\alpha > 0$.
Note that this relationship appears as a straight line on the log-log plot of the probability distribution over the range of the random variable.

Goodness of Fit.

It is difficult to argue that an empirical dataset genuinely follows a target probability distribution, since other statistical models unexamined yet may describe the dataset with a better fit.
Thus, it is more sound to rely on comparative tests which compare the goodness of fit of candidate distributions <cit.>.
We especially utilize the log likelihood-ratio test <cit.> to this end.
When a dataset $\Dcal$ with two candidate distributions $A$ and $B$ is given, the test computes $\log \left( \frac{\Lcal_A(\Dcal)}{\Lcal_B(\Dcal)} \right)$, where $\Lcal_A(\Dcal)$ stands for the likelihood of the dataset $\Dcal$ with respect to the candidate distribution $A$.
Positive ratios imply the distribution $A$ is a better description available for the dataset between the two distributions, and negative ratios imply the opposite.

Hypergraph Sequence.

Allowing for multiple hyperedges created at the same time, we use $e_t^s$ to denote the set of hyperedges created at time $t$ and $e_t$ to denote a hyperedge in $e_t^s$.
Given a sequence $\edgesseq$ of a set of time-stamped hyperedges for some $T > 0$, we define a sequence $\{G_t = (V_t, E_t)\}_{t=1}^T$ of hypergraphs evolving under the hyperedge sequence, where the nodes $V_t$ of $G_t$ is $\bigcup_{i=1}^{t}\bigcup e_i^s$ and the hyperedges $E_t$ of $G_t$ is $\bigcup_{i=1}^t e_i^s$.
While the final snapshot $G_T$ of the hypergraph sequence is of our interest in exploring structural patterns, subsequences of the hypergraph sequence are examined for the understanding of dynamical patterns.

Null Model.

We make use of a null model as a counterpart of an evolving hypergraph $\graphseq$ so as to emphasize the significance of examined patterns in real-world hypergraphs in Section <ref>.
Formally, the null model is constructed from a sequence of a set of hyperedges $\nulledgesseq$ such that there is a one-to-one correspondence between $\edges$ and $\nulledges$.
Specifically, $\nulledge$ in $\nulledges$ has the same size with its corresponding hyperedge $\edge$ in $\edges$, while $\nulledge$ consists of randomly selected $|\nulledge|$ nodes from $V_t$.
Note that this null model has the same distribution of hyperedge sizes with its corresponding dataset.

The real-world hypergraphs used in our study.

Dataset     # of Nodes     # of Hyperedges     Summary

$327$		       $172,035$  	     Social Interaction

$1,005$ 	       $235,263$  	     Email

$3,029$ 	       $271,233$  	     Q&A

$5,556$ 	       $112,919$		     Drug

$176,445$        $719,792$ 		     Q&A

$1,924,991$      $3,700,067$ 	     Coauthorship

§ STRUCTURAL AND DYNAMICAL PATTERNS

Log-likelihood ratio of two competing distributions: three heavy-tailed dists. versus exponential dist..
We consider power law (pw), truncated power law (tpw), and log-normal (logn) distributions as candidates for the heavy-tailed distributions.
We report the log-likelihood ratio normalized by its standard deviation and make the largest one among the three candidates boldfaced.
We also provide the $p$-value of the boldfaced one if it is larger than $0.05$, where the $p$-value is the significance value for the acceptance of the heavy-tailed distribution involved in the ratio.
As suggested in Figure <ref>, the first three patterns generally have the best fit to (truncated) power law distributions,
while singular values are best described by log-normal distributions.
Remark that also shows the same trend.

3cHyperedge Sizes${}^{*}$
3cIntersection Sizes
3cSingular Values

1cHeavy-tailed Dist.

$-0.612$     $\textbf{0.495}^{\dagger}$     $-0.011$
-      -       -
-      -       -
$-291$     $-205$     $\textbf{116}$

$0.013$     $\textbf{2.01}$      $1.72$
$28.6$     $\textbf{34.0}$     $32.8$
$454$     $\textbf{463}$     $461$
$-178$     $-151$     $\textbf{73.2}$

$8.60$      $\textbf{9.51}$       $9.45$
$-713$      $-111$       $\textbf{103}$
-      -       -
$-407$      $-285$       $\textbf{145}$

$3.69$     $\textbf{3.90}$     $3.83$
$29.5$     $\textbf{31.8}$     $30.3$
$39.1$     $\textbf{39.9}$     $\textbf{40.0}$
$-361$     $-250$     $\textbf{121}$

$38.0$     $\textbf{38.5}$     $38.4$
$0.786$     $\textbf{1.05}^{\ddagger}$     $1.02$
-      -       -
$-1171$     $-832$     $\textbf{445}$

$187$     $\textbf{207}$     $204$
$4.14$     $\textbf{4.15}$     $4.15$
$2.29$     $\textbf{2.36}$     $\textbf{2.36}$
$-661$     $-471$     $\textbf{268}$

$19.7$      $\textbf{27.0}$      $25.9$
$-0.737$      $\textbf{1.36}^{\mathparagraph}$       $1.29$
-      -       -
$-525$      $-364$       $\textbf{210}$

13l${}^{\ast}$ , , , and have a small range of hyperedge sizes and intersection sizes, which make some ratio not available.

13l${}^\dagger$ The $p$-value is $0.62$. ${}^\ddagger$ The $p$-value is $0.30$. ${}^{\mathparagraph}$ The $p$-value is $0.17$.

In this section, we examine characteristics of real-world hypergraphs prevalent across $6$ distinct domains and shed light on common four structural and three dynamical patterns at the macroscopic level.
We summarize some basic statistics on the datasets <cit.> in Table <ref> and briefly describe the characteristics of each dataset below.

* contact-high-school (): each node is a student, and each hyperedge is a set of individuals interacting each other as a group during an unit interval.

* email-Eu (): each node is an email address at an European research institution, and each hyperedge consists of the sender and all recipients of an e-mail.

* tags-ask-ubuntu (): <askubuntu.com> is a question-and-answer website, where one can ask a question with up to $5$ tags attached.
Each node and hyperedge correspond to a tag and the set of tags attached to a question, respectively.

* NDC-substances (): each node is a substance, and each hyperedge indicates the set of substances which a drug is made of.

* threads-math-sx (threads): <math.stackexchange.com> is a question-and-answer website.
Each node is a user on the website, and each hyperedge corresponds to the set of users participating in a thread that lasts for at most $24$ hours.

* coauth-DBLP (): each node is an author, and each hyperedge corresponds to the set of authors in a publication recorded on DBLP.

We first observe the heavy-tailed distributions of degrees, hyperedge sizes, intersection sizes, and singular values of incidence matrices as the structural patterns.
Then, we elaborate on diminishing overlaps of hyperedges, densification, and decreasing diameter as the dynamical patterns.

§.§ Structural Patterns

In this section, we study four tendencies in the final snapshots of the $6$ real hypergraph sequences and present all the results in Figure <ref>.
We demonstrate that (S1) degrees, (S2) hyperedge sizes, (S3) intersection sizes, and (S4) singular values of incidence matrices obey heavy-tailed distributions.
Moreover, out of three probable candidates - power law, truncated power law, and log-normal distributions - quantitative validation based on the log-likelihood test supports that the first three are described best by the truncated power law distribution, and the last is closest to the log-normal distribution.
These patterns are peculiar in the sense that the random null model described in Section <ref> cannot reproduce them, as illustrated in Figure <ref>.

[Size of Intersections]

[Singular Values]

Comparison with the null model generated from substance.
Its bell-shaped degree distribution and the significantly dominant singular value, which appear across all the datasets, are main differences from real hypergraphs.
Note that the range of hyperedge sizes is significantly reduced compared to that of .

Dynamical patterns shown in six real datasets: diminishing overlaps, densification, and shrinking diameter.
The slopes smaller than $1$ in the first row indicate decreasing ratios of intersecting pairs.
The slopes larger than $1$ for the edge density imply increasing average degrees of the hypergraphs.
The effective diameters eventually decrease.
For the  dataset, the effective diameter starts to shrink almost in the end.
See Section <ref> for details.

[origin=l]90Intersecting Pairs

[origin=l]90Edge Density

[origin=l]90  Diameter

S1. Heavy-tailed degree distribution.

The degree distributions of real hypergraphs generally fall under the class of .
Note that our investigation of the degree distributions differs from the previous approaches <cit.> where the degrees of nodes $2$-projected graphs are investigated.

As reported in Table <ref>, the comparative test implies that for each dataset, the three representative have a better fit than an .
Moreover, the likelihood ratio and the well-fitted straight lines on the log-log scale in Figure <ref> suggest that for the degree distributions, the (truncated) power law distribution is the best candidate among the .

In Figure <ref>, the skewed bell-shaped degree distribution of the random null model also indicates the peculiarity of the observed patterns.

S2. Heavy-tailed hyperedge size distribution.

Hyperedge sizes are also found to follow heavy-tailed distributions.
Note that we use all hyperedges, regardless of their sizes, while only small hyperedges are taken into consideration in <cit.>.
The distribution of hyperedge sizes generally shows a better fit to the than to the , and the similar reasoning based on Figure <ref> and Table <ref> implies that the (truncated) power law distribution is the best description available.
Note that we do not report the distribution of hyperedge sizes of the null model, as its definition results in the same distribution with its corresponding real hypergraph.

S3. Heavy-tailed intersection size distribution.

The intersection sizes of two hyperedges (i.e., the number of nodes commonly contained in two hyperedges), which offer insights into pairwise intersections of hyperedges, also follow .
As shown in Table <ref>, the are probable descriptions for this pattern, and the reported likelihood ratios in Table <ref> together with the straight lines in Figure <ref> back up the goodness of fit to the (truncated) power law distribution.

In comparison with the null model, whose intersection-size distribution is shown in Figure <ref>, although its overall distribution seems similar, the range of intersection sizes and the frequency of each intersection size are significantly different from those of the corresponding hypergraph.

S4. Skewed singular values.

This pattern means that the singular values of the incidence matrices of real hypergraphs are generally heavy-tailed.
As certified in Table <ref>, the singular-value distribution is best described by the log-normal distribution, which is one of the .
Apparently in Figure <ref>, the tails of the singular-value distributions decay faster than those of the three distributions in (S1-S3).

In contrast to the real distribution, we observe in Figure <ref> that the null model has a highly dominant singular value.

§.§ Dynamical Patterns

Now we move on to the investigation of three dynamical patterns in the six real hypergraphs sequences.
We devise a quantity to measure how much the hyperedges are overlapped in overall and show that (T1) overall overlaps of the hyperedges decrease over time.
Just as phenomena of densification and shrinking diameter are well known to take place in real graphs,
the two dynamical patterns are still prevalent in real hypergraphs; (T2) average degrees increase over time and (T3) effective diameters decrease over time.

T1. Diminishing overlaps.

We first define the density of interactions as follows to capture the overall overlaps of hyperedges:
for a hypergraph $G_t = (V_t, E_t)$ at time $t$,

\begin{equation}\label{equation:doi}
DoI(G_t) := \frac{|\{ \{e_i, e_j \} \vert e_i \cap e_j \neq \emptyset \textup{ for } e_i, e_j\in E_t  \}|}
{|\{ \{e_i, e_j\} \vert e_i, e_j \in E_t \}|}
\end{equation}

In words, it is simply the ratio of the number of intersecting pairs of hyperedges to the number of all possible pairs of hyperedges, which amounts to $\binom{|E_t|}2$.
Note that this quantity can range from $0$ to $1$.

Using $y(t)$ and $x(t)$ to indicate the numerator and the denominator, respectively, in Equation <ref>, we take a closer look at the log-log plot of $y(t)$ over $x(t)$.
As shown in the first row of Figure <ref>, the slopes $s$ of fitted lines on the plots implies the formula $DoI(G_t) = y(t)/x(t) = O(x(t)^{-(1-s)})$.
As the slopes $s$ are usually smaller than $1$, the density of interactions decreases over time.
Decreasing $DoI(G_t)$ indicates that the ratio of overlapping pairs of hyperedges decreases and that the intersections of hyperedges become less frequent in overall.

T2. Densification.

In the following two subsections, we confirm that dynamical phenomena in real graphs - densification and shrinking diameter established in a seminal work <cit.> - still take place in real hypergraphs.

We proceed with the same manner as in Section <ref>.
For a hypergraph $G_t = (V_t, E_t)$ at time $t$, we plot $|E_t|$ over $|V_t|$ on the log-log scale in the second row of Figure <ref> and compute the slopes $s$ of the fitted lines, which lead to the formula $|E_t| \propto |V_t|^s$.
The slopes larger than $1$ in general indicate that the average degrees of hypergraphs, formulated as $2|E_t|/|V_t|$, increase over time, and thus the densification is also valid for real hypergraphs.

T3. Shrinking diameter.

In the third row of Figure <ref>, we show how the effective diameters of hypergraphs change over time.
We observe that the effective diameters of all real hypergraphs eventually decrease over time.
In the  dataset, while the decrease is marginal, the effective diameter starts to decrease almost in the end.

It is worthy to note that although shrinking diameter and densification seem intuitively compatible with one another, one can construct counterexamples that have only one of the two dynamical patterns, which are illustrated in the second and third row in Figure <ref>.

§ PROPOSED MODEL: HYPERGRAPH FOREST FIRE

In parallel with the establishment of underlying patterns in real hypergraphs,
we propose a growth model (Hypergraph Forest Fire) for realistic hypergraph generation.
In a nutshell, for each new node, simulates a `forest fire', which is spread stochastically through existing hyperedges, to decide the nodes each of which forms a size-$2$ hyperedge with the new node. Then, simulates a forest fire again for each created hyperedge to expand it.

reproduces all seven examined patterns and additionally reproduces all five structural patterns found in previous work <cit.> without relying on oracles, i.e., unexplainable outside information.
Notably in , the mechanisms on individual nodes are simple and intuitive, while they do not directly impose but eventually lead to the examined patterns.

In Section <ref>, we describe in detail with a pseudocode and rationales behind it.
In Section <ref>, we confirm that it successfully reproduces all the seven patterns as well as five additional patterns.
Lastly in Section <ref>, we explore the parameter space of .

: the proposed model for realistic hypergraph generation.

burning probability: $p$, expanding probability: $q$,

timespan (i.e., the target number of nodes): $T$
evolving hypergraph: $\graphseq$
$p, q, T$
$G_0 \setgets$ hypergraph with $1$ node and $0$ hyperedges
Initialize $tie$ measuring how close two nodes are
time $t$ in $[1, \dots, T]$
$\Vt \setgets V_{t-1} \cup$ $\{$new node $u\}$ & $\edges \setgets \{\}$
$w \setgets $ random ambassador from $V_{t-1}$
$Burned \setgets$ ($w, p$)
$v$ in $Burned$
Increase $tie(\{u, v\})$ by $1$
$Burned \setgets$ (v, q)
$\edges \addgets$ hyperedge $Burned\cup\{u\}$

$E_t \setgets E_{t-1} \cup e_t^s$

$ $
source, prob
$Burned = \{ \}$ & $Queue \setgets $ empty queue
$Queue \addgets source $
$Queue \neq \emptyset$
$s \setgets$ node popped from $Queue$
$Burned \addgets s $
$n \sim$ geometric dist. with mean $\frac{prob}{1-prob}$
$Candidates \setgets N(s)\setminus Burned\setminus Queue$
$Queue \addgets$ $n$ neighbors in $Candidates$ in decreasing order of $tie(\{s, neighbor\})$

Comparison between a real hypergraph and those generated by with distinct parameters.
Each row for ($p, q$) with the burning probability $p$ and expanding probability $q$ shows holistic behaviors of the model.
With suitable probabilities (e.g., $p=0.51$ and $q=0.2$), successfully reproduces all examined patterns.
The overall comparison indicates larger $p$ and $q$ lead to faster densification and shrinking diameter, respectively.
See Section <ref> for details.

4cStructural Patterns
3cDynamical Patterns

Degrees     Hyperedge sizes     Intersection Sizes     Singular Values     Intersecting Pairs     Edge Density     Diameter

[origin=l]90($0.51, 0.2$)

[origin=l]90($0.51, 0.3$)

[origin=l]90($0.45, 0.2$)

[origin=l]90($0.45, 0.3$)

§.§ Description of model

We provide the pseudocode of in Algorithm <ref>.
Note that, except the target number of nodes (i.e., $T$), it needs only two parameters: burning probability $p$ and expanding probability $q$.
starts with a hypergraph with one node and no hyperedge; and it initializes tie strengths, which measure the closeness of two nodes, to $0$. Then, it repeats the following steps for each new node $u$.

(1) The new node $u$ chooses a random ambassador $w$ from the hypergraph so far and burns the ambassador.

(2) Burn $n$ neighbors of the ambassador $w$ in descending order of tie strength, where  $n$ is sampled from the geometric distribution with mean $p/(1-p)$.
In case of burning a part of neighbors with the same tie strength, proceed with random selection.

(3) Recursively apply (2) to each burned neighbor by viewing a burned neighbor as a new ambassador of the new node $u$.
Nodes cannot be burned twice until the recursion ends.

(4) For each burned node $v$,
form a hyperedge $\{u, v\}$ and increase tie strength of $\{u, v\}$ by $1$.

(5) For each hyperedge created in (4),
reset the burning history and
start the burning process at the burned node $v$ in which we use the geometric distribution with mean $q/(1-q)$, expand the hyperedge until the process ends.

Each step has a straightforward rationale, and it may be well understood through an example of a coauthorship network.
Suppose a student joins a research community, advised by a supervisor (ambassador).
When introducing colleagues to the student for coworking, the supervisor is more likely to introduce intimate peers (considering tie strength).
Those who cowork with the student recursively introduce their close peers for follow-up research (recursive burning).
When a group of researchers looks for a new researcher to work with (hyperedge expansion), a referrer in the group is likely to introduce those who the referrer has worked with many times before than a totally new researcher (considering tie strength).

Comparison with <cit.>.
Our model has several differences from the forest fire model <cit.>, which is inspired by.
Most importantly, the forest fire model generates conventional graphs instead of hypergraphs.
Moreover, it relies on the `orientation' of burning, requiring two parameters: forward and backward burning probabilities.
Without relying on the orientation, it fails to achieve both shrinking diameters and power law distributions of degrees.
On the other hand, achieves both by additionally taking tie strength (instead of the orientation) into account, without requiring an additional parameter.
As a result, successfully copes with the complexity due to hyperedges of any size, while
maintaining the number of parameters to two (i.e., one probability for neighbor selection and the other for expansion).

suggests possible local dynamics on individual nodes of hypergraphs that give rise to the macroscopic patterns examined in this paper.
Thus, predicting an order of hyperedges at the microscopic level goes out of scope.
Also, for simplicity, it does not have a particular parameter for repetition of hyperedges.
Constructing a model tackling these sides is left for future work.

Compatibility with previous work <cit.>.
and a real dataset have similar heavy-tailed degree distributions at each decomposition level, and they even have the same tendency of deviating from the fitted lines at higher decomposition levels.
Singular values of and the dataset basically lie on their fitted lines, regardless of decomposition levels.
See Section <ref> for details.

3cSingular Values

Node     Edge     Triangle
Node     Edge     Triangle

[origin=l]90  (Real)

§.§ Experimental results

We examine the proposed model thoroughly, in terms of the established patterns, as in Section <ref>.
We also examine its $n$-level decomposed graphs, as suggested in <cit.>.

Observed patterns.
We demonstrate that with suitable parameter values[We investigate a synthetic hypergraph with $10,000$ nodes and set the burning probability to $0.51$ and the expanding probability to $0.2$.] reproduces four structural and three dynamical patterns investigated in Section <ref>.

In Figure <ref>, we provide overall patterns of models with four distinct parameter settings.
The model in the first row apparently exhibits all the patterns consistent with the representative results of one representative real dataset.
Especially for the four structural patterns, including the distribution of degrees, hyperedge sizes, intersection sizes, and singular values,
we confirm in Table <ref> that all the distributions of the model have the same tendency with real hypergraphs.
The four distributions are best suited to , and among probable , the best description for each pattern is also consistent with that of real hypergraphs; the first three and the last are close to the (truncated) power law distributions and log-normal distribution, respectively.

We also check that the model achieves the three dynamical patterns - diminishing overlap, densifying graph, and decreasing diameter - in Figure <ref>.

Structural patterns in decomposed graphs.
The multi-level decomposition method <cit.> provides a way of reducing hypergraphs to $m$ ordinary graphs for the largest size $m$ of hyperedges.
Each reduced graph is referred to as an $n$-level decomposed graph,[Formally,
the $\mathit{n}$-level decomposed graph of a hypergraph $G=(V, E)$ is defined as $G_{(n)}=(V_{(n)}, E_{(n)})$ where

\begin{align*}
& V_{(n)}  := \{v_{(n)} \in 2^{V} : |v_{(n)}| = n \text{ and } \exists e\in E \text{ s.t. } v_{(n)} \subseteq e \}, \\
& {E_{(n)}  := \{\{u_{(n)},v_{(n)}\} \in \binom{V_{(n)}}{2} : \exists e\in E \text{ s.t. } u_{(n)} \cup v_{(n)} \subseteq e \}.}
\end{align*}

] which focuses on the interplays between pairs of size-$n$ subsets of nodes.
In <cit.>, it is shown that the decomposed graphs of real hypergraphs generally retain five structural patterns of real networks - giant connected components, heavy-tailed degree distributions, small diameters, high clustering coefficients, and approximately heavy-tailed singular values of adjacency matrices - unlike the random null model that we adopt in Section <ref>.
Then, the structural patterns are suggested as judging criteria for real hypergraphs.

To make the setting consistent with <cit.>, we delete all hyperedges of size larger than $5$ and consider the decomposition level up to $4$, where each level is named as the node level, the edge level, the triangle level, and the $4$clique level, respectively.
Moreover, we focus on the node level especially for effective diameter and clustering coefficient as highlighted in <cit.>, and up to the triangle level for the distributions of degree and singular values due to the lack of space.

We compare in Figure <ref> the overall tendency of the degree and singular-value distributions of the decomposed graphs of with that of real hypergraphs.
The degree distributions of decomposed graphs, regardless of whether they are real or synthetic, seem to follow a power law, while the plotted points tend to deviate from a fitted line as the decomposition level increases.
The singular-value distributions also reveal similar trends in both synthetic and real datasets.
In Table <ref>, we report the effective diameter and clustering coefficient at the node level, and the ratio of the size of the largest connected component to the total size of each decomposed graph.
Similar to the real datasets, at the node level, the decomposed graph from also attains a small effective diameter and high clustering coefficient.
The level at which the largest connected component becomes small varies across the datasets, and our model maintains a giant connected component up to the edge level.

Size of the largest connected component at each decomposition level, and effective diameter and clustering coefficient at the node level.
The red colors indicate the level at which decomposed graphs no longer retain a giant connected component.
Note that also shows a similar trend with real datasets;
its giant connected component starts to shatter after the edge level, its effective diameter is small, and it has similar clustering coefficient with the real datasets.

4cLargest Connected Component
1c2*Clus. Coeff.

Node     Edge     Triangle     $4$clique

$1.00$      $0.46$      $\red{0.02}$     $\red{0.02}$     $2.63$     $0.44$

$0.98$     $0.70$     $0.80$     $0.41$     $2.80$     $0.36$

$0.997$     $0.94$     $0.71$     $\red{0.22}$     $2.41$     $0.23$

$0.58$     $0.78$     $0.35$     $\red{0.02}$     $3.56$     $0.28$

$0.87$     $0.45$     $\red{0.03}$     $\red{0.0004}$     $3.69$     $0.036$

$0.86$     $0.53$     $\red{0.05}$     $\red{0.0006}$     $6.84$     $0.10$

$1.00$     $0.52$     $\red{0.0003}$     $\red{0.0005}$     $3.00$     $0.33$

§.§ Exploring parameter space

In Figure <ref>, we present qualitative patterns in with parameters set from all the combinations of two burning probability and two expanding probability.
We observe that the burning probability and the expanding probability are mainly concerned with how fast a hypergraph densifies and its effective diameter decreases, respectively.
Note that with low burning probabilities, the effective diameter of a hypergraph is actually increasing in spite of the densification.

§ CONCLUSIONS

Despite the omnipresence of hypergraphs, relatively little attention has been paid to structural and dynamical patterns of real-world hypergraphs.
Toward more extensive and thorough understanding of the real-world hypergraphs, we closely examine four structural and three dynamical patterns prevalent in them.
The former includes the heavy-tailed distributions of (S1) degrees, (S2) hyperedge sizes, (S3) intersection sizes, and (S4) singular values of incidence matrices.
The latter includes (T1) diminishing overlaps, (T2) densification, and (T3) shrinking diameter.
We validate the significance of these patterns by comparison with a null model and statistical tests.

We also propose a generative model , which captures all seven observed patterns and also shows results compatible with previous findings.
Especially, it has only two scalars (e.g., burning and expanding probabilities) as parameters, and it does not rely on any external information for imitating realistic patterns.
Surprisingly, is made up of simple and intuitive mechanisms on individual nodes, which non-trivially lead to all the examined macroscopic patterns.

For reproducibility, we make the source code and datasets used in this work publicly available at <https://git.io/JUfAr>.

This work was supported by National Research Foundation of Korea (NRF) grant funded by the
Korea government (MSIT) (No. NRF-2020R1C1C1008296) and Institute of Information & Communications
Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2019-0-00075, Artificial Intelligence Graduate School Program (KAIST)).


: Responsibly Reporting AI-generated Content

Jan Philip Wahle

University of Göttingen

Terry Ruas

University of Göttingen

Saif M. Mohammad

National Research Council Canada

Norman Meuschke

University of Göttingen

Bela Gipp

University of Göttingen

Given AI systems like ChatGPT can generate content that is indistinguishable from human-made work, the responsible use of this technology is a growing concern. Although understanding the benefits and harms of using AI systems requires more time, their rapid and indiscriminate adoption in practice is a reality. Currently, we lack a common framework and language to define and report the responsible use of AI for content generation. Prior work proposed guidelines for using AI in specific scenarios (e.g., robotics or medicine) which are not transferable to conducting and reporting scientific research.
Our work makes two contributions: First, we propose a three-dimensional model consisting of transparency, integrity, and accountability to define the responsible use of AI.
Second, we introduce “AI Usage Cards”, a standardized way to report the use of AI in scientific research.
Our model and cards allow users to reflect on key principles of responsible AI usage. They also help the research community trace, compare, and question various forms of AI usage and support the development of accepted community norms.
The proposed framework and reporting system aims to promote the ethical and responsible use of AI in scientific research and provide a standardized approach for reporting AI usage across different research fields.
We also provide a free service to easily generate AI Usage Cards for scientific work via a questionnaire and export them in various machine-readable formats for inclusion in different work products at <https://ai-cards.org>.

<concept_desc>Social and professional topics Computing / technology policy</concept_desc>
<concept_desc>Software and its engineering Documentation</concept_desc>
<concept_desc>Computing methodologies Artificial intelligence</concept_desc>
<concept_desc>General and reference Evaluation</concept_desc>

[500]Computing methodologies Artificial intelligence
[500]General and reference Evaluation
[500]Social and professional topics Computing / technology policy
[500]Software and its engineering Documentation

A three-dimensional model of responsible AI usage (left) and the header of the template (right).

§ INTRODUCTION

The rapid development of AI for natural language processing has led to concerns about using AI-generated content in scientific papers. While AI systems like ChatGPT <cit.> can generate texts, ideas, and source code, some outputs may be false or fail to attribute the appropriate sources. As a result, it remains unclear whether such models should be used for content generation.

Despite the novelty of AI for content generation, researchers are already using it to support software development and academic writing. However, the rules and norms to govern this new paradigm are yet to be determined. Conferences and journals have started to define rules for using language models to generate content. The largest conference in computational linguistics (ACL) differentiates between AI-based support tools in its official language model policy <cit.> and calls for transparency, i.e., acknowledging the use of AI when generating content. Conversely, one of the most significant machine learning conferences (ICML) has prohibited using all language models for submissions in 2023 <cit.>. Although these attempts are not a final and unanimous agreement on using AI-based assistants, they do signal a response to the sudden increase in AI content generation.

While the rules and norms to govern AI-generated content are still emerging, it is clear that transparency is an essential element of responsible use. By acknowledging the use of AI, users can ensure that the generated content is reliable and appropriate attribution is given to the appropriate sources. However, the responsible use of AI for content generation is context-dependent and complex, and caution must be exercised in fields where high-stakes outcomes are involved, such as in the medical and legal fields. Ultimately, while using AI for content generation is likely here to stay, we must find ways to use it responsibly and transparently to avoid adverse outcomes.

History has shown that carefully scoped legalizations can be more effective than blanket prohibitions in many scenarios.
Similar to current discussions on social issues such as recreational drug use, a formal framework for acceptable use creates transparency, helps to control the problem, and decriminalizes usage.
However, we lack a common language and concrete principles to describe what is needed to “responsibly legalize” AI usage.
Although frameworks exist to document key characteristics and ethical considerations associated with models <cit.>, datasets <cit.>, evaluations <cit.>, and AI tasks <cit.>, in the form of cards and sheets, no standardized framework exists to report the use of AI-generated content.

In this work, we introduce the three-dimensional model shown in <Ref> to decompose the dimensions of responsible AI usage for content generation.
We use the term “AI Usage” to refer to all applications of AI-based assistants in scientific work, e.g., to inspire, generate, revise, and compare content.
In addition to transparency (“Where and how was AI used to produce content?”), our model incorporates integrity (“Have humans approved the content produced by AI systems?”) and accountability (“Who is responsible for the use/dissemination of content produced by AI systems?”).

We also propose  as a standard for acknowledging AI support in the scientific process—an important scenario among the many in which content generation can be used.
We crafted  with the phases of the scientific process in mind, e.g., hypothesis definition, literature review, and manuscript writing. However,  we designed the framework flexible enough to report AI usage across domains.
Our cards seek to make the acknowledgment of AI usage the status quo.
In summary, this paper makes the following two contributions.

* We propose a model for the responsible use of AI content generation in science based on three dimensions: transparency, integrity, and accountability.

* We introduce , as a standardized tool to report using AI in scientific works. can be generated using an online questionnaire, which we provide as a free service and exports in machine-readable formats that can be incorporated directly into any scientific work.

Our proposed model and method of documenting AI usage can serve as a valuable tool for users to engage in the responsible use of content-generating systems. However, it is important to recognize that potential scenarios for using AI in scientific work can vary significantly, ranging from low-stakes tasks, such as correcting typos in a social media post, to high-stakes scenarios that have the potential to cause harm, such as misleading diagnoses or unjustified sentencing, especially in the medical and legal fields.

In these high-stakes scenarios, AI should be used with extreme caution, and careful consideration should be given to the potential consequences of AI-generated content. There may be applications that we do not want to use AI for at all. In addition, the target audience should also be considered, whether experts or laypeople, as they perceive information as more or less critical. We want to emphasize that simply legalizing AI usage is not the solution; rather, we must find ways to define where and how we should use it and make its usage responsible for avoiding negative consequences.

While people will undoubtedly continue to use AI for content generation, we must take extra care to ensure its responsible use. If used transparently and responsibly, we believe AI can do more good than harm.

§ RELATED WORK

For some time now, studies have shown that humans are increasingly unable to detect AI-generated content <cit.>.
Efforts exist to flag generated text automatically, for example, through zero-shot detection methods <cit.> or watermarks introduced in generated texts <cit.>.
However, detection techniques alone do not solve important aspects of AI usage, e.g., integrity and accountability.

Concerns regarding the use of AI are also present in the health domain for medical diagnosis <cit.>, clinical trials <cit.>, and health care research <cit.>.
When providing diagnosis, recommending treatment, and screening patients, AI models (such as ChatGPT) must have their predictive capabilities strictly constrained to avoid serious harm, as these models contain biases and unverified information <cit.>.
Moreover, clear accountability for actions taken based on recommendations from AI is required, i.e., “Who is responsible for the diagnosis and treatment?”.
Although relevant, the contributions of <cit.> are either not transferable to other domains, lack a common agreement, or miss artifacts relevant to our use case in their reports, e.g., tracking which models are used and transparency on modified textual content.
Nonetheless, these works mostly agree on the need for better reporting guidelines and more responsible AI.

is in line with contributions such as Data Cards <cit.>, Model Cards <cit.>, and Evaluation Cards <cit.>, which complement our work.
Data Cards highlight the most important facts about machine learning datasets.
The goal of a Data Card is to inform those involved (e.g., researchers) about the characteristics of manipulated datasets during their lifecycle in specific projects (e.g., content, number of records, supporting funding agency).
Model Cards <cit.> document machine learning models—mainly from computer vision and natural language processing—to provide the necessary details for reproducing the results, e.g., training data, metrics, and model parameters.
In addition to the technical details on a specific model, <cit.> advocate for transparency of the performance of considered models.
Evaluation Cards <cit.> allow researchers to document their experiments and standardize evaluations, thereby enabling the monitoring of trends in NLP research.

§ A THREE-DIMENSIONAL MODEL

To responsibly use the outputs of AI, we propose categorizing its usage in a three-dimensional model.
The model includes pillars to help acknowledge the usage of AI models (transparency); to approve the generated content (integrity); and clarify who is responsible for the content generated by the AI model (accountability).

While specific frameworks for the use of AI exist in some areas (e.g., public decision-making <cit.>, robotics <cit.>), they are still limited as they do not specify to which aspects of the scientific work AI systems contributed.
Our model is general enough to be applied to many different research domains and still categorizes the main components of any scientific work.
In the following, we detail each dimension of our conceptual model and explain how they interact.

Transparency. The first dimension of responsible practice is the foundation on which all others build.
Transparency refers to acknowledging the use of AI models in one's work.
Throughout the project, one should document every instance of using AI models, with the intent to disclose the information in a suitably summarized form eventually.
Similar practices are established for acknowledging a funding agency for supporting a project, individuals for giving feedback, an author affiliation, or a possible conflict of interest.
The primary goal of disclosing relevant information about AI usage during scientific work is to establish traceability of the scientific process steps to which AI contributed  and help authors reflect on which parts of their work have originated from AI.

Integrity. The second dimension pertains to the correctness, truthfulness, fairness, safety, and appropriateness of the content generated by AI systems.
AI uses learnable parameters to model massive amounts of data from the internet.
Thus, users of this technology should reflect on such content and assert whether it propagates biases, factual incorrectness, other's ideas as own ideas, unethical statements, prejudice, personally identifiable information, or false claims towards individuals in their output.
This verification of fairness principles has to be carried out through human assessment.
For smaller initiatives, we recommend obtaining input from a diverse set of people, with representation from relevant groups of people most impacted by system bias.
For larger projects, we recommend adopting strategies from participatory design.
Participatory design in research and systems development centers the people, especially marginalized and disadvantaged communities, such that they are not merely passive subjects but have the agency to shape the assessment process <cit.>.
All aspects of integrity assessment must be documented for eventual reporting (as appropriate).

Accountability. The last dimension addresses how researchers use, publish, and disseminate content produced with the help of AI models.
With clear accountability, those affected by serious harm can ask authors to fix overlooked integrity failures (e.g., propagating hate speech content), especially for those already marginalized.
On one side, accountability means ensuring that generated content adheres to integrity proactively before releasing AI-generated content.
On the other side, it means being open to feedback and willing to make changes as necessary, and being open about the model’s limitations and potential risks associated with its use.
Many criticize that AI systems such as ChatGPT are largely black boxes that need more transparency, not least because there has been no official research paper associated with its release.
However, transparency is often confused with accountability.
When humans perform inexplainable decisions, we can hold them accountable, but if a search engine provides racist results, we cannot.
Without accountability, knowing what was manipulated without accountability can produce harm to individuals and does not lead to liability.
Therefore, responsible AI use requires documenting corresponding individuals for the project who can be contacted about potential issues.

Ideally, all three dimensions, i.e., transparency, integrity, and accountability, must be fulfilled simultaneously for using AI models responsibly.
Analogous to a three-legged chair, each leg/dimension is crucial for using AI in the scientific process.
By acknowledging the use of models and holding authors accountable, the output can be explained, but it may still contain inaccuracies and biases since no verification steps have been taken. It is the responsibility of both the AI user and creator to provide sources for the ideas presented, allowing readers to verify them.
If we acknowledge the usage of AI and approve its content, we achieve usability but can encounter a lack of accountability if problems occur, e.g., plagiarized content or misleading information.
The content is useable “at your own risk” due to a lack of accountability.
Finally, when accountability and integrity are verified, the output is secure because it has been affirmed. Even if human flaws exist, they can be accounted for and adjusted if necessary.
Only when we acknowledge the use of the model, mitigate its inappropriate outputs, and assign someone to respond in case of problems or questions can we mitigate major issues and make AI usage responsible.

To facilitate the documentation and reporting needs for the responsible use of AI-generated content, we propose as a standardized practice to incorporate the principles of transparency, integrity, and accountability.

The presented in this paper are motivated towards using AI in scientific works; however, these cards can easily be adapted to other domains.
for scientific works follow the scientific method and reflect common headings of scientific manuscripts.
We plan to offer different card templates for different disciplines—from computer science to zoology—and stakeholders.
Companies, funding agencies, or institutions could define individual questions with varying levels of detail depending on what they deem necessary for ensuring responsible AI usage in their scenario.

are available in multiple machine-readable formats, such as , XML, JSON, and CSV, to enable easy inclusion in different work products (cf. <Ref>) and support the automated analysis of AI usage across domains and content types in the future. can be redistributed for non-commercial purposes according to the CC BY-NC 4.0 license[<https://creativecommons.org/licenses/by-nc/4.0/>]. A free service to create [The AI card reporting usage of AI for this paper can be found on the project webpage.] is available at

§.§ Structure

are divided into six major blocks: project details, ideation and review, methodology and experiments, writing and presentation, code and data, and ethics.
Our card allows researchers and practitioners to report AI usage during all phases of scientific work, from theoretical ideation to practical use.
This includes even formulating ideas, for example, in the form of questions and acquiring knowledge during a literature review.
Researchers can prompt AI to generate hypotheses, support the development of methods, and suggest newly designed experiments.
For measurements, AI can suggest data sources and process data using generated code.
Finally, when communicating results, AI can support the writing of papers or other presentation artifacts such as figures and tables.
In the following, we summarize each of these blocks:

* Project details: Meta-information about the authors, models, and project, including their names, versions, key applications, and affiliations.

* Ideation and review: Support during theoretical preparation of the project through idea formulation (e.g., outline) and examining related work (e.g., comparing literature, finding analogies).

* Methodology and experiments: Design, comparison, and generation of processes for methodological tasks during the project, optionally resulting in experiments (e.g., proposing a new process).

* Writing and presentation: Generation and paraphrasing text used in scientific reports or papers. The improvement of content, figures, tables, and any other elements are also included.

* Code and data: Manipulation, generation, refactoring, optimization, and analysis of code and data.

* Ethics: Implications of using AI and steps to mitigate their possible errors and harms.

For more details on each category see <Ref>.
Each block contains six to seven subcategories. In each subcategory, one or more of the following classifications can be assigned.

* New: Content generated based on prompts without a significant portion of prior ideas and thoughts.

Example Prompt: “Generate some ideas on how to approach the problem of memorization for large language models.”

* Revise: Content generated based on previous content that has a significant portion of own ideas and thoughts.

Example Prompt: “Rephrase the following paragraph so that it uses academic voice, is concise and short, and adds an example of a meta-analysis.”

* Compare: Content generated by providing two or more pieces of content (own or others).

Example Prompt: “Compare my definition of plagiarism to the following [...]”.

We devise a questionnaire to generate (see <Ref> for more details).
The questions can be answered through a free online form that automatically generates a card to be incorporated in any scientific report as shown in Card <ref>.
The AI card that reports usage for our paper can be found on the project webpage.

§ PRACTICAL CONSIDERATIONS

Q1. Who should create ?

A. All individuals using any AI system to assist in work are eligible to use . The use is intended for a broader audience, from students reporting their assignments at school to researchers submitting research articles and funding proposals.

Q2. When should one create ?

A. Broadly, should be created when using AI systems to support content-related activities during any project (e.g., scientific paper, blog post). This means documenting any activity in which AI has been used as a support tool and categorizing the support according to the card.

Q3. What are the benefits of ?

A. The benefits of include providing transparency, accountability, and integrity in AI usage by encouraging authors to reflect on their AI usage, giving individuals tools to acknowledge AI usage, addressing AI biases for minorities, publishing machine-readable reports for analysis, and contributing to responsible AI usage becoming the status quo.

Q4. What should the cards not be used for?

A. should not be used to justify unethical or inappropriate content dissemination. This card should force users to reflect on the generated content and its value to society.

Q5. Should be updated?

A. Yes. As technologies change and different domains have varying requirements, the cards must be revised periodically to ensure their relevance and mitigation of outdated practices. For example, the concerns raised by the use of large language models today might not be relevant ten years from now.

Q6. Should we have specific ?

A. should serve as a framework to incorporate different questions for different needs.
Given its machine-readable format, will allow for data-driven transparency on what was considered important for responsible AI usage by different groups of people throughout time. There might be different cards for different scientific works (e.g., dissertation—more details; short paper—fewer details). Our card might also find applications in other areas, such as philosophy or art, with varying requirements.

Q7. How can we incentivize the creation of ?

A. Further simplifying the creation of , e.g., by offering use-case-specific card formats, can encourage widespread adoption. Moreover, requiring a report on AI usage for any scientific submission, similar to a statutory declaration in graduation theses, can incentivize authors to internalize responsible reporting practices. allow creating such reports efficiently in a standardized and machine-processable fashion.

Q8. What role should play in the review process of conferences and journals?

A. They should act as documentation to help conference and journal organizers to understand how much support AI systems offer to their users during their work. However, an acceptable threshold for such assistance should be clarified by the organizers.

Q9. Won't the reporting of slow the research work?

A. No. can be generated in less than five minutes using a free, interactive, and easy-to-use questionnaire. They will help individuals reflect on how they use AI systems in their work. The responsible use of AI and its report is in the interest of all parties involved <cit.>.

§ DISCUSSION

In this paper, we proposed to facilitate the reporting of AI use.
We provided a free service to produce cards through a fast and interactive questionnaire.
We showed how AI use could be reported in a standardized way for all steps of scientific work.
Beyond scientific projects, our card serves as a framework to report the usage of AI models across domains.

allow to monitor AI usage and help policymakers to evaluate their decisions.
Our card can be exported in machine-readable formats (e.g., XML) for further analysis.
Compared to other efforts for reporting AI usage <cit.>, we provide a standardized way of reporting that is transferable to other domains.

are a tool to acknowledge AI usage openly, similar to how the use of computing infrastructure, external funding, and carbon footprint is acknowledged.
The community can decide whether using specific models during certain phases of a scientific project (e.g., hypotheses formulation) is reasonable when certain biases exist.
Eventually, analyzing across a field of study or area can help to reveal trends in AI usage.

The current version of allows for only a limited number of reporters, so we cannot account for many individuals in a scientific project.
In this context, focal points or group leaders should take responsibility for reporting their team's actions.
The dimensions and properties of are a high-level framework for the responsible use and reporting of AI assistance.
The questions, subcategories, and technical details of the framework should not remain static in the long run.
We invite researchers to build on top of and iterate on its concepts.
In the same way, scientific contributions are constantly challenged by the community, AI reporting should be regularly updated to guarantee its relevance and mitigation of outdated practices in the community.
We leave to future work the investigation on how to adapt to the specific needs of its users and projects.

This work was supported by the Lower Saxony Ministry of Science and Culture and the VW Foundation.

#1     #1#1#1     #1     #1     #1     #1#1          #1#1

[Brown et al(2020)]
authorpersonTom Brown, personBenjamin Mann,
personNick Ryder, personMelanie Subbiah,
personJared D Kaplan, personPrafulla Dhariwal,
personArvind Neelakantan, personPranav Shyam,
personGirish Sastry, personAmanda Askell,
personSandhini Agarwal, personAriel Herbert-Voss,
personGretchen Krueger, personTom Henighan,
personRewon Child, personAditya Ramesh,
personDaniel Ziegler, personJeffrey Wu,
personClemens Winter, personChris Hesse,
personMark Chen, personEric Sigler,
personMateusz Litwin, personScott Gray,
personBenjamin Chess, personJack Clark,
personChristopher Berner, personSam McCandlish,
personAlec Radford, personIlya Sutskever, and
personDario Amodei. year2020.
Language Models are Few-Shot Learners. In
booktitleAdvances in Neural Information Processing
Systems, editorpersonH. Larochelle,
personM. Ranzato, personR. Hadsell,
personM.F. Balcan, and personH. Lin (Eds.),
Vol. volume33. publisherCurran Associates, Inc.,

[Cruz Rivera et al(2020)]
authorpersonSamantha Cruz Rivera,
personXiaoxuan Liu, personAn-Wen Chan,
personAlastair K. Denniston, personMelanie J.
Calvert, personThe SPIRIT-AI and CONSORT-AI Working Group,
personSPIRIT-AI and CONSORT-AI Steering Group,
personAra Darzi, personChristopher Holmes,
personChristopher Yau, personDavid Moher,
personHutan Ashrafian, personJonathan J. Deeks,
personLavinia Ferrante di Ruffano, personLivia
Faes, personPearse A. Keane, personSebastian J.
Vollmer, personSPIRIT-AI and CONSORT-AI Consensus Group,
personAaron Y. Lee, personAdrian Jonas,
personAndre Esteva, personAndrew L. Beam,
personMaria Beatrice Panico, personCecilia S. Lee,
personCharlotte Haug, personChristophe J. Kelly,
personChristopher Yau, personCynthia Mulrow,
personCyrus Espinoza, personJohn Fletcher,
personDavid Moher, personDina Paltoo,
personElaine Manna, personGary Price,
personGary S. Collins, personHugh Harvey,
personJames Matcham, personJoao Monteiro,
personM. Khair ElZarrad, personLavinia Ferrante di
Ruffano, personLuke Oakden-Rayner, personMelissa
McCradden, personPearse A. Keane, personRichard
Savage, personRobert Golub, personRupa Sarkar, and
personSamuel Rowley. year2020.
Guidelines for Clinical Trial Protocols for
Interventions Involving Artificial Intelligence: The SPIRIT-AI
journalNature Medicine volume26,
number9 (dateSept. year2020),
1078-8956, 1546-170X

[de Fine Licht and de Fine Licht(2020)]
authorpersonKarl de Fine Licht and
personJenny de Fine Licht. year2020.
Artificial Intelligence, Transparency, and Public
Decision-Making: Why Explanations Are Key When Trying to Produce
Perceived Legitimacy.
journalAI & SOCIETY volume35,
number4 (dateDec. year2020),
0951-5666, 1435-5655

[Guo et al(2023)]
authorpersonBiyang Guo, personXin Zhang,
personZiyuan Wang, personMinqi Jiang,
personJinran Nie, personYuxuan Ding,
personJianwei Yue, and personYupeng Wu.
titleHow Close is ChatGPT to Human Experts? Comparison
Corpus, Evaluation, and Detection.

[Humphries et al(2020)]
authorpersonBeth Humphries, personDonna M
Mertens, and personCarole Truman.
Arguments for an ‘emancipatory’research
In booktitleResearch and inequality.
publisherRoutledge, pages3–23.

[Hupkes et al(2023)]
authorpersonDieuwke Hupkes, personMario
Giulianelli, personVerna Dankers, personMikel
Artetxe, personYanai Elazar, personTiago Pimentel,
personChristos Christodoulopoulos, personKarim Lasri,
personNaomi Saphra, personArabella Sinclair,
personDennis Ulmer, personFlorian Schottmann,
personKhuyagbaatar Batsuren, personKaiser Sun,
personKoustuv Sinha, personLeila Khalatbari,
personMaria Ryskina, personHong Technology,
personRyan Cotterell, and personZhijing Jin.
booktitleState-of-the-art generalisation research in
NLP: a taxonomy and review.
typeWorkingPaper. institutionArXiv.

We thank Adina Williams, Armand Joulin, Elia Bruni, Lucas Weber,
Robert Kirk and Sebastian Riedel for providing us feedback on various stages
of this draft, and Gary Marcus for providing detailed feedback on the final
draft of this paper. We thank Elte Hupkes for making the app that allows
searching through references, and we thank Daniel Haziza and Ece Takmaz for
other contributions to the website.

[Ibrahim et al(2021)]
authorpersonHussein Ibrahim, personXiaoxuan
Liu, and personAlastair K. Denniston.
Reporting Guidelines for Artificial Intelligence in
Healthcare Research.
journalClinical & Experimental Ophthalmology
volume49, number5 (dateJuly
year2021), pages470–476.
1442-6404, 1442-9071

authorpersonOpenAI Inc.
titleChatGPT: Optimizing Language Models for Dialogue.
[Online; accessed 24-Jan-2023].

[Jordan Boyd-Graber(2023)]
authorpersonAnna Rogers Jordan Boyd-Graber,
Naoaki Okazaki. year2023.
titleACL 2023 Policy on AI Writing Assistance.
[Online; accessed 24-Jan-2023].

[Kirchenbauer et al(2023)]
authorpersonJohn Kirchenbauer, personJonas
Geiping, personYuxin Wen, personJonathan Katz,
personIan Miers, and personTom Goldstein.
titleA Watermark for Large Language Models.

[Liu et al(2020)]
authorpersonXiaoxuan Liu, personSamantha
Cruz Rivera, personDavid Moher, personMelanie J.
Calvert, personAlastair K. Denniston, personThe
SPIRIT-AI and CONSORT-AI Working Group, personSPIRIT-AI and
CONSORT-AI Steering Group, personAn-Wen Chan,
personAra Darzi, personChristopher Holmes,
personChristopher Yau, personHutan Ashrafian,
personJonathan J. Deeks, personLavinia Ferrante di
Ruffano, personLivia Faes, personPearse A. Keane,
personSebastian J. Vollmer, personSPIRIT-AI and
CONSORT-AI Consensus Group, personAaron Y. Lee,
personAdrian Jonas, personAndre Esteva,
personAndrew L. Beam, personAn-Wen Chan,
personMaria Beatrice Panico, personCecilia S. Lee,
personCharlotte Haug, personChristopher J. Kelly,
personChristopher Yau, personCynthia Mulrow,
personCyrus Espinoza, personJohn Fletcher,
personDina Paltoo, personElaine Manna,
personGary Price, personGary S. Collins,
personHugh Harvey, personJames Matcham,
personJoao Monteiro, personM. Khair ElZarrad,
personLavinia Ferrante di Ruffano, personLuke
Oakden-Rayner, personMelissa McCradden,
personPearse A. Keane, personRichard Savage,
personRobert Golub, personRupa Sarkar, and
personSamuel Rowley. year2020.
Reporting Guidelines for Clinical Trial Reports for
Interventions Involving Artificial Intelligence: The CONSORT-AI
journalNature Medicine volume26,
number9 (dateSept. year2020),
1078-8956, 1546-170X

[Mitchell et al(2023)]
authorpersonEric Mitchell, personYoonho
Lee, personAlexander Khazatsky, personChristopher D.
Manning, and personChelsea Finn.
titleDetectGPT: Zero-Shot Machine-Generated Text Detection
using Probability Curvature.

[Mitchell et al(2019)]
authorpersonMargaret Mitchell, personSimone
Wu, personAndrew Zaldivar, personParker Barnes,
personLucy Vasserman, personBen Hutchinson,
personElena Spitzer, personInioluwa Deborah Raji,
and personTimnit Gebru. year2019.
Model Cards for Model Reporting. In
booktitleProceedings of the Conference on Fairness,
Accountability, and Transparency (Atlanta, GA, USA)
(seriesFAT* '19). publisherAssociation for
Computing Machinery, addressNew York, NY, USA,

authorpersonSaif Mohammad.
Ethics Sheets for AI Tasks. In
booktitleProceedings of the 60th Annual Meeting of the
Association for Computational Linguistics (Volume 1: Long Papers).
publisherAssociation for Computational Linguistics,
addressDublin, Ireland, pages8368–8379.

[Neri et al(2020)]
authorpersonEmanuele Neri, personFrancesca
Coppola, personVittorio Miele, personCorrado
Bibbolino, and personRoberto Grassi.
Artificial Intelligence: Who Is Responsible for
the Diagnosis?
journalLa radiologia medica volume125,
number6 (dateJune year2020),
0033-8362, 1826-6983

authorpersonLesley-Ann Noel.
Promoting an emancipatory research paradigm in
design education and practice. In booktitleProceedings of
DRS2016 International Conference, Vol. 6: Future–Focused Thinking.
addressBrighton, United Kingdom, pages27–30.

authorpersonMichael Oliver.
Emancipatory research: Realistic goal or impossible
journalDoing disability research
volume2 (year1997), pages15–31.

[on Machine Learning Program Chairs(2023)]
authorpersonInternational Conference on Machine Learning
Program Chairs. year2023.
titleClarification on Large Language Model Policy LLM.
[Online; accessed 24-Jan-2023].

[Pushkarna et al(2022)]
authorpersonMahima Pushkarna, personAndrew
Zaldivar, and personOddur Kjartansson.
Data Cards: Purposeful and Transparent Dataset
Documentation for Responsible AI. In booktitle2022 ACM
Conference on Fairness, Accountability, and Transparency (Seoul, Republic
of Korea) (seriesFAccT '22).
publisherAssociation for Computing Machinery,
addressNew York, NY, USA, pages1776–1826.

[Spinde et al(2021)]
authorpersonTimo Spinde, personManuel
Plank, personJan-David Krieger, personTerry Ruas,
personBela Gipp, and personAkiko Aizawa.
Neural Media Bias Detection Using Distant
Supervision With BABE - Bias Annotations By Experts. In
booktitleFindings of the Association for Computational
Linguistics: EMNLP 2021.

authorpersonClay Spinuzzi.
The methodology of participatory design.
journalTechnical communication
volume52, number2 (year2005),

[Wachter et al(2017)]
authorpersonSandra Wachter, personBrent
Mittelstadt, and personLuciano Floridi.
Transparent, Explainable, and Accountable AI
for Robotics.
journalScience Robotics volume2,
number6 (dateMay year2017),

[Wahle et al(2022)]
authorpersonJan Philip Wahle, personTerry
Ruas, personFrederic Kirstein, and personBela
Gipp. year2022.
How Large Language Models are Transforming
Machine-Paraphrase Plagiarism. In booktitleProceedings of
the 2022 Conference on Empirical Methods in Natural Language Processing.
publisherAssociation for Computational Linguistics,
addressAbu Dhabi, United Arab Emirates,

§ ETHICAL CONSIDERATIONS

As AI language models are trained mostly on human-produced data to generate and manipulate textual content, they are not free of the same issues we are subject to (e.g., discrimination and bias).
Thus, using these models might result in inappropriate outcomes that need to be dealt with either during their training phase (e.g., more diverse data) or post-processing (e.g., discarding wrongful text suggestions).
The propagation of unethical content (e.g., racial slurs) can lead to destructive consequences, particularly for vulnerable populations <cit.>.

cannot guarantee AI will be used to the best of everyone's interest or it will not suggest false claims about sensitive topics and individuals.
However, the human component plays a fundamental role in evaluating altered content and ideas.
It is our responsibility to “keep in check" the technologies we create and use, so we can take action when necessary.
This does not mean our process is error-free.
Science is a collective work, so our process requires honesty from all parties involved.
Therefore, if researchers using AI models do not report their actions, our card can propagate bias, unethical statements, and prejudice.

§ FOR THIS PAPER

Card <ref> reports the AI Usage of this paper. We used AI for suggestions on the name of the cards, for comparing methods on the theoretical model, and to generate a first version of the abstract, which we did not use in the final manuscript.

§ TEMPLATE

Card <ref> shows a template for with details on each subcategory. While these categories describe steps of scientific works, they can be adjusted to other domains as well.

§ DETAILS ON THE QUESTIONNAIRE

We provide a fast process to generate by asking users questions.
First, we ask which models have been used in this work by a set of questions, including the model name, date of usage, and model version (see <Ref> for an example).
This set of questions can be repeated multiple times. Next, let users choose the main categories in which AI has been used in their work (see <Ref>).
Later, we assign each model to a specific category to differentiate whether one model has been used solely during writing while another one was also used for literature review.
The main categories answer the “where” of AI usage. Next, based on the selected main categories, we provide unique views to obtain more details on the “how” of AI usage.
For example, has AI been used to generate new ideas or to revise or compare? (<Ref>).
For each subcategory, users can provide more details in the form of text in the following steps (<Ref>).
They can explain why it was necessary to use AI, how AI has improved the content, which parts of AI content were used, etc.
Next, users are presented with ethical questions (<Ref>) and need to approve the used AI-generated content (<Ref>).
Finally, a set of questions about the project are presented, including the corresponding authors accountable for the AI usage (see <Ref> for an example).
After submitting the form, the cards are automatically generated and sent by email with a tutorial on how to include them in scientific works (<Ref>).

One question of the question set about which AI models were used, when they were used, and in which versions.

The main categorization in which parts of a work AI was used. This answers the question: “Where was AI used?”.

The sub-categories for selected main categories in which AI was used. This answers the question “How was AI used in this specific aspect of the work?”

Open text details on each subcategory to mention specifics about how a model was used.

One question of the question set about ethical considerstaions

A confirmation that authors approve the used content of AI.

A question of the set for project details to mention the corresponding authors.

The automatically generated response email to the questionnaire with exports of attached and a tutorial on how to include them in scientific works.