# Infinite-Dimensional Feature Interaction

Chenhui Xu1,2 Fuxun Yu1,3 Maoliang Li4 Zihao Zheng4
Zirui Xu1,5 Jinjun Xiong2† Xiang Chen1,4†
1 George Mason University 2University at Buffalo
3Microsoft 4 Peking University 5CVS Health

###### Abstract

The past neural network design has largely focused on feature representation
space dimension and its capacity scaling (e.g., width, depth), but overlooked
the feature interaction space scaling. Recent advancements have shown shifted
focus towards element-wise multiplication to facilitate higher-dimensional
feature interaction space for better information transformation. Despite this
progress, multiplications predominantly capture low-order interactions, thus
remaining confined to a finite-dimensional interaction space. To transcend
this limitation, classic kernel methods emerge as a promising solution to
engage features in an infinite-dimensional space. We introduce InfiNet, a
model architecture that enables feature interaction within an infinite-
dimensional space created by RBF kernel. Our experiments reveal that InfiNet
achieves new state-of-the-art, owing to its capability to leverage infinite-
dimensional interactions, significantly enhancing model performance.

##  1 Introduction

In the past decade, deep neural network architecture design has experienced
several major paradigm shifts regarding the feature representation learning.
As shown in Fig. 1(a), the early stage of neural network design is dominant by
flat stream architectures in the form of weight-feature interaction (e.g.,
$W\textbf{x}$), like multi-layer perceptron (MLP), convolution neural networks
(CNN), ResNet, etc. These models usually adopt linear superposition (e.g.,
$W_{i}\textbf{x}\oplus W_{j}\textbf{x}$) in the feature representation space.
Therefore, the feature representation space scaling is limited to increase
model channel width and depth [16, 12]. Nevertheless, this scaling approach
has witnessed model development from the very small-scale MLPs or LeNet[17] to
the recent huge ConvNext V2 [37]. With the ultra-scaled parameter amounts,
computing complexity, and model size, the return of investment on model
performance by further scaling feature dimensions has largely plateaued [9].

Despite the plateau in feature representation space, recent sporadic
architecture design works [26] shed light on another potential dimension of
scaling: feature interaction space. Specifically, as shown in Fig. 1(b), these
neural network designs generally demonstrate feature-feature interaction
(e.g., $W_{i}\textbf{x}\otimes W_{j}\textbf{x}$). As a mathematical example,
the self-attention mechanism in Transformers [34] can be formulated as
$\textbf{x}^{L+1}=f_{k}(\textbf{x}^{L})*f_{q}(\textbf{x}^{L})*f_{v}(\textbf{x}^{L})$,
which is also element-wise multiplication between processed input feature
themselves. Characterized by element-wise interaction, these designs offer
complementary feature correlation capabilities in addition to simple linear
superposition. Such feature interactions have become the essential mechanisms
of mainstream state-of-the-art neural architectures. For example, it’s
implemented in SENet with squeeze and excitation [15], non-local network with
transposed multiplication [35], vision transformers with self-attention [34,
3, 10], gated aggregation [19, 31, 42], and quadratic neurons [40, 11, 41].

Although these models greatly improve the performance of state-of-the-arts, as
mentioned above, these works provide diversified explanations that neglect the
underlying shared design of element-wise feature multiplication operation
[26], and thus may fail to reveal the fundamental source of improvement. To
provide both explainability and quantifiability, in this paper, we propose a
unified theoretical perspective to rethink the feature interaction scaling,
i.e., the dimensionality of feature interaction space. For example, as shown
in Fig 1(b), by employing the $\otimes$ multiplication, an implicit
interactive quadratic space
$\mathcal{Q}=\text{span}\left(x_{1}^{2},x_{1}x_{2},\cdots,x_{n-1}x_{n},x_{n}^{2}\right)$
with degrees of freedom ${n(n+1)}/{2}$ is constructed from the original
representative vector space
$\mathcal{V}=\text{span}\left(x_{1},x_{2},\cdots,x_{n}\right)$ with degrees of
freedom $n$ [26]. Such space dimensionality scaling is the key to improving
feature representation quality and end-task, as we will show later.

From the unified perspective, a new opportunity emerges in neural architecture
design, that is to scale to infinite-dimensional feature interactions than
former methodologies (e.g., from $n(n+1)/2$ to
$\lim_{k\to\infty}\frac{(n+k-1)!}{(n-1)!k!}$). However, scaling feature
interaction space dimensionality from architectural enhancements (e.g.,
quadratic, self-attention, and recursive gates) comes with linear scaling cost
w.r.t. order $k$, which hinders the infinite dimensionality increase [19, 10].
Thus, there is an open question:

How can we efficiently extend interactions to an infinite-dimensional space?

Inspired by traditional machine learning, we propose an approach that
introduces kernel methods for feature interaction in neural networks. We
define a set of feature interactions between features $W_{a}\mathbf{x}$ and
$W_{b}\mathbf{x}$ with a kernel function
$\mathit{K}(W_{a}\mathbf{x},W_{b}\mathbf{x})$ instead of the element-wise
multiplication. As shown in Fig 1(c), the kernel method transforms the feature
to an ultra-high dimensional space by an implicit mapping $\phi(\cdot)$. From
there, the feature interaction space is defined by the inner product on the
Reproducing Kernel Hilbert Space (RKHS) [1] $\mathcal{H}$ constituent with the
kernel function $\mathit{K}(\cdot,\cdot)$. The RKHS can greatly expand the
interaction space at very little cost, enabling infinite dimensions.

Figure 1: (a) Traditional feature representation without interaction [37, 12].
(b) Recent work with finite feature interaction [31, 41]. (c) Our method:
Kernel-enabled infinite feature interaction.

We then propose InfiNet, a novel family of neural networks that generate high-
quality feature representations. Specifically, we introduce the Radial Basis
Function (RBF) kernel [29] as a replacement of the common $\oplus$ or
$\otimes$ operations. With the infinite series expansion in RBF kernel, it
enables a theoretical provable dimensionality approximation
$\text{span}_{j\in\mathbb{N},\sum_{k}{n_{k}}=j}\\{\frac{x_{1}^{n_{1}}\cdots
x_{k}^{n_{k}}}{\sqrt{n_{1}!\cdots n_{k}!}}\\}$ while with as low-overhead as
exponential operations. In this way, InfiNet enables efficient infinite-
dimensional feature interaction space scaling upon a finite set of branches in
the model architecture, thus achieving better complexity performance tradeoffs
than prior state-of-the-arts.

Contributions. We make the following contributions:

  * •

We unify the perspectives of recent feature interactive works and identify a
novel direction of neural network performance scaling: the feature interaction
space dimensionality.

  * •

We propose a method to expand the feature interaction space to an infinite
dimension with RBF kernel, that can effectively model the complex implicit
correlations of features.

  * •

We propose InfiNet, a novel series of neural networks that explore the neural
interaction from infinite-dimensional space, and achieve state-of-the-art
performance.

Extensive ablation studies verify that the shift from finite feature
interaction space to infinite one is a key factor to learn better
representations and therefore improving model performance. Meanwhile, large-
scale experiments on ImageNet classification, MS COCO detection and ADE20K
segmentation also demonstrate InfiNet design’s effectiveness, which
consistently outperforms the state-of-the-art flat stream networks [23, 12]
and finite-order interaction networks [31, 22].

##  2 Related Work

Interactions in Neural Networks. Interaction in neural networks has undergone
a long period of change. During the evolution from AlexNet [16] to ResNet
[12], the model has for long followed the principle of summing the weighted
pixels at each position in the layer-by-layer feature iteration. And then, as
the potential of the attention mechanism was realized, model development is
towards an element-wise multiplication way. This trend is punctuated by the
emergence of models such as Non-Local [35], Transformer [34], and ViT [10].
Recent insights suggest that the crux of the attention mechanism lies in high-
order information interactions, rather than the mechanism of "Attention"
itself [3, 31, 39]. This revelation has spurred the development of innovative
neural network architectures. For example, HorNet [31], QuadraNet [40] and
MogaNet [19] examines high-order models from the perspective of spatial
interactions through multiplication-integrated architecture design. However,
limited by the support of existing deep learning platforms such as Pytorch
[30], few attempts have been made to extend the model’s feature interaction to
an ultra-high dimensional situation through the kernel method for more
potential.

Kernel Method in Neural Networks. The fundamentals of kernel methods are well-
studied in traditional ML domains like support vector machines [32] but they
are less used in neural architectures. Early extensions to deep learning
through the kernelized perceptron [6] have improved the performance but mainly
in shallow neural networks. Recent advancements include the development of
Convolutional Kernel Networks (CKN) [27], which merge CNNs’ robust feature
learning with kernel stability. This approach offers a theoretical foundation
for deep learning’s application in structured data. Additionally, the
introduction of Kervolutional Neural Networks [5] replaces traditional
convolution in CNNs with kernel-based operations to enhance feature extraction
without excessive computational costs. The combination of Gaussian processes
with neural networks to create Deep Kernel Learning [36] adjusts model
complexity based on data while maintaining Bayesian inference.

Despite these innovations, a significant limitation remains: Although these
methods expand the feature representation space, they fail to scale up the
feature interaction space, limiting the network to aggregate information in a
superposition manner. As a result, these methods also fall short of potential
feature interactions and thus are incapable of handling complex data
correlations and functionalities.

##  3 From Feature Representation Space to Interaction Space

We start with considering the transformation of feature representation of a
normal isotropic 2-layer perceptron block. Given an input
$\mathbf{x}=(x_{1},x_{2},\cdots,x_{n})$ with n-dimension. We denote the first
layer transformation as $\mathbf{z}=g(\mathbf{x})=\sigma(W_{1}\mathbf{x})$ (we
omit the bias term for simplicity), and the second layer transformation as
$h(\mathbf{z})=W_{2}\mathbf{z}$. Therefore the whole block is a mapping
$h\circ g:\mathbb{R}^{n}\mapsto\mathbb{R}^{n}\mapsto\mathbb{R}^{n}$ from
feature space $\mathbb{R}^{n}$ to a medium feature representation space
$\mathbb{R}^{n}$ and eventually to a output space $\mathbb{R}^{n}$. Expanding
the width of the neural networks, for example with a coefficient 2, is going
to expand the feature representation space to $\mathbb{R}^{2n}$. The core idea
of this dimensional expansion is that implicit associations in feature in low
dimensions will be expressed explicitly when projected into a high-dimensional
space. Model architecture (i.e. convolution, multi-branch [33]) is an additive
superposition of pixels, essentially no different from the perceptron in terms
of representation space.

###  3.1 Feature Interaction Space

Although flat stream neural networks are defined by linear transformations and
activations, modern network design also introduces multiplications in the
network structure or neurons. They are called attentions from an interpretable
point of view, that is the degree of interest of one position in relation to
another; from a more abstract point of view, they are called interactions in
spatial contexts. But invariably, these are realized with element-wise
multiplication. We conduct the following definition.

Definition 1 (Feature Interaction) A Feature interaction refers to
transformations between features with the same or different positions defined
by element-wise multiplication.

For example, Star Operation [26] is a basic 2-order feature interaction,
defined as $(W_{a}\mathbf{x})*(W_{b}\mathbf{x})$, where
$W_{a},W_{b}\in\mathbb{R}^{n}$. This is a simple element-wise multiplication
fusion of two linear transformations of the ordinary input $\mathbf{x}$. We
write the expansion of such multiplication operation:

$y=(W_{a}\mathbf{x})*(W_{b}\mathbf{x})=(\sum_{i=1}^{n}w_{ai}x_{i})(\sum_{j=1}^{n}w_{bj}x_{j})=\sum_{i\leq
j}\alpha_{i,j}x_{i}x_{j}$ (1)

where $\alpha_{i,j}=w_{ai}w_{bj}+w_{aj}w_{bi},\text{if}\ i\neq j$, and
$\alpha_{i,i}=w_{ai}w_{bi}$. Then we vectorize $\alpha$ and $x_{i}x_{j}$:

$A=[\alpha_{1,1},\alpha_{1,2},\alpha_{2,2},\cdots,\alpha_{n-1,n},\alpha_{n,n}]\in\mathbb{R}^{n(n+1)/2}$
(2) $\chi=[x_{1}x_{1},x_{1}x_{2},x_{2}x_{2},\cdots,x_{n-1}x_{n},x_{n}x_{n}]$
(3)

$\chi$ can therefore define a basis of a space. Thus the output of the current
layer can be rewritten as:

$y=\sum_{i\leq j}\alpha_{i,j}x_{i}x_{j}=A\chi.$ (4)

From the independence of the pixel level, we know that each term in $\chi$ is
linearly independent, this indicates every dimension in $\chi$ is an
individual dimension. Given a set of basis vectors $\chi$, we define
$\text{span}(\chi)$ as the feature interaction space. In this way, the
generation of the next layer of features $y$ is constituted by a linear
superposition $A\chi$ on the feature interaction space $\text{span}(\chi)$
like in Eq.(4).

###  3.2 Dimension of Feature Interaction Space

Now, we consider the number of dimensions of a feature interaction space.
Given $k$-1 multiplication operations, we first define the k-order feature
interaction space as follows:

Definition 2 (Feature Interaction Space) A k-order Feature Interaction Space
$\mathcal{S}^{k}$ refers to the span of monomial basis
$\\{x_{1}^{d_{1}}x_{2}^{d_{2}}\cdots x_{n}^{d_{n}}|\sum
d_{i}=k,d\in\mathbb{N}\\}$ defined by a k-order Feature Interaction.

An element-wise multiplication generates a feature interaction space of
$n(n+1)/2$ dimension, which is the number of elements of an upper triangular
matrix. In general, considering the symmetry of the interactions and elements,
for a k-order interaction on the $n$-dimensional feature, the dimension of the
corresponding feature interaction space is:

$dim(\mathcal{S}^{k})=\frac{(n+k-1)!}{(n-1)!k!}$ (5)

The next layer of feature generation based on the feature interaction space
greatly expands the spatial dimensions to which the features are mapped
compared to the original model based only on the feature representation space,
i.e., it is possible to explore the feature’s non-linearity in high-
dimensional space. It is worth noting that this process introduces terms like
$x_{1}x_{2}$, an interaction that cannot be captured by traditional plane
networks in feature representation space.

For example, we consider the Self-Attention in the Transformers [34]. The
Self-Attention contains two element-wise multiplication, so it explores a
3-order feature interaction space. This is due to the fact that: (1) in first
stage, in the query-key dot-product attention map computation
$Att(\mathbf{x})=Q\cdot K^{T}=W_{Q}\mathbf{x}\cdot\mathbf{x}^{T}W_{K}^{T}$
explores the feature interaction space
$\mathcal{A}=\text{span}\left(x_{1}^{2},x_{1}x_{2},\cdots,x_{n-1}x_{n},x_{n}^{2}\right)$.
(2) In the second stage, the multiplication between the attention map and the
value $y=\sigma(Att(\textbf{x}))\cdot V=\sigma(Att(\textbf{x}))\cdot
W_{v}\mathbf{x}$ explores the feature interaction space
$\mathcal{S}=\mathcal{A}\otimes\mathbb{R}^{n}=\text{span}\left(x_{1}^{3},x_{1}^{2}x_{2},x_{1}x_{2}x_{3},\cdots,x_{n-1}x_{n}^{2},x_{n}^{3}\right)$,
which has $(n+2)(n+1)n/6$ dimensions in line with Eq.(5). Thereby we explain,
from the perspective of feature interaction space, why the transformer family
of models has, so far, generally outperformed recurrent neural networks and
convolutional neural networks in various domains.

##  4 Expanding Interaction Space to Infinite Dimension

This element-wise multiplication-based interaction, since the construction of
each order of the feature interaction space is based on a multiplication
operator, leads to a problem in that feature interaction space expansion is
still difficult. This is due to the fact that these interaction operators is
explicit mapping, which tends to have quadratic or higher complexity w.r.t the
input length (e.g. self-attention [34]) or lengthy recursive designs (e.g.
HorNet [31]) and linear complexity w.r.t interaction order. But the problem
with building this mapping explicitly is: (1) The complexity of mapping
itself. The computational overhead associated with defining a set of explicit
nonlinear mappings is non-negligible. A mapping from $C$ channels to
$C^{\prime}$ channels means a computation complexity of $O(CC^{\prime})$. (2)
The complexity of interaction. The complexity of the inner product used to
interact with the two sets of features increases dramatically to
$O(C^{\prime})$ when the dimension is raised. Considering $C^{\prime}>>C$,
these two complexities will largely increase the computational overhead of the
networks.

We would like to obtain a method that can expand the dimension of feature
interaction space as much as possible in $O(1)$ time. Fortunately, the machine
learning community has already given a method for increasing the dimension of
a feature defined on the inner product: kernel methods.

###  4.1 Expanding Interaction Space with Reproducing Kernel

The nature of kernel methods is that they are substitutions for inner product
operations. This requires combining element-wise multiplication and summation
to define a set of inner products within the network. For this purpose, we
rewrite the form of element-wise multiplication in Eq.(1) on two groups of
features, which is a common design in literature architecture (e.g. multi-head
self-attention [34]). It will therefore be a superposition of multiple
interaction in the format:

$y=\sum_{i=1}^{C}W_{ai}\mathbf{x}*W_{bi}\mathbf{x}=\langle\mathbf{W_{a}x,W_{b}x}\rangle$
(6)

where
$\mathbf{W_{a}x}=[W_{a1}\mathbf{x},W_{a2}\mathbf{x},\cdots,W_{aC}\mathbf{x}]$,
$C\in\mathbb{N}$ is the number of branches. Thereby we generalize the form of
the feature interaction from element-wise multiplication to inner product
which is a multi-branch paradigm. At this point, we have
$\mathbf{W_{a}x}\in\mathbb{R}^{C}$, while it is generated from a feature
representation space $\mathbb{R}^{n}$. In order to extend the interaction
space, we need to further project $\mathbf{W_{a}x}$ and $\mathbf{W_{b}x}$ to a
high-dimensional space. An obvious way to do this is to construct an implicit
mapping $\Phi(\cdot)$ to a high-dimensional space, so we can compute
$\langle\Phi(W_{a}\mathbf{x}),\Phi(W_{b}\mathbf{x})\rangle$ for interaction.

By Mercer’s Theorm [28], taking a continuous symmetric positive semi-definite
function $K(s,t)$, there is an orthonormal basis $\\{\phi_{i}(\cdot)\\}$,
$i=0,1,\cdots,\infty$, consisting of eigenfunctions of function
$K(\cdot,\cdot)$ such that the corresponding sequence of eigenvalues
$\\{\lambda_{i}\\}$ is non-negative. These means:

$K(s,t)=\sum_{i=1}^{\infty}\lambda_{i}\phi_{i}(s)\phi_{i}(t),$ (7)

where $\forall i\neq j,\forall s\ \text{and}\
t,\langle\phi_{i}(s),\phi_{j}(t)\rangle=0$ since. Then we construct a Hilbert
space $\mathcal{H}$ with the orthonarmal basis
$\\{\sqrt{\lambda_{i}}\phi_{i}(\cdot)\\}$. Consider a vector
$f=(f_{1},f_{2},\cdots)^{T}_{\mathcal{H}}$ on the space $\mathcal{H}$, then we
have:

$f=\sum_{i=1}^{\infty}f_{i}\lambda_{i}\phi_{i}(\cdot).$ (8)

Thus for a vector $K(s,\cdot)$ in space $\mathcal{H}$:

$K(s,\cdot)=\sum_{i=1}^{\infty}\lambda_{i}\phi_{i}(s)\phi_{i}(\cdot)=\sum_{i=1}^{\infty}\sqrt{\lambda_{i}}\phi_{i}(s)\sqrt{\lambda_{i}}\phi_{i}(\cdot)=(\sqrt{\lambda_{1}}\phi_{1}(s),\sqrt{\lambda_{2}}\phi_{2}(s),\cdots)^{T}_{\mathcal{H}}.$
(9)

Therefore, for the Hilbert space $\mathcal{H}$, we can define the reprodcuing
kernel by:

$\langle
K(s,\cdot),K(t,\cdot)\rangle=\sum_{i=1}^{\infty}\sqrt{\lambda_{i}}\phi_{i}(s)\sqrt{\lambda_{i}}\phi_{i}(t)=\sum_{i=1}^{\infty}\lambda_{i}\phi_{i}(s)\phi_{i}(t).$
(10)

Implicit Mapping to High-Dimensional RKHS. Let $\Phi(s)=K(s,\cdot)$, then we
have $\langle\Phi(s),\Phi(t)\rangle=K(s,t)$. The Hilbert Space $\mathcal{H}$
is known as the Reproducing Kernel Hilbert Space (RKHS) corresponding to
kernel function $K(\cdot,\cdot)$. Note that the $\Phi(\cdot)$ is therefore
defined on the RKHS $\mathcal{H}$, which can be infinite-dimensional given
specific kernel $K(\cdot,\cdot)$. The mapping $\Phi(\cdot)$ does not have to
have an explicit expression since we can get the result of
$\langle\Phi(s),\Phi(t)\rangle$ by computing $K(s,t)$. This means that we can
achieve an extension of the dimensions for the feature interaction space by
simply replacing the inner product $\langle\mathbf{W_{a}x,W_{b}x}\rangle$ used
in the interaction with a kernel $K(\mathbf{W_{a}x,W_{b}x})$.

###  4.2 Infinite-Dimensional Feature Interaction with RBF Kernel

To maximize the dimension of the feature interaction space, we consider Radial
Basis Function (RBF) Kernel
$K_{\text{rbf}}(\mathbf{s},\mathbf{t})=\exp\left(-\frac{1}{2}\left\|\mathbf{s}-\mathbf{t}\right\|_{2}^{2}\right)$with
an infinite-dimensional RKHS, given the fact that:

$\displaystyle\exp$
$\displaystyle\left(-\frac{1}{2}\left\|\mathbf{s}-\mathbf{t}\right\|_{2}^{2}\right)=\sum_{j=0}^{\infty}\frac{\left(\mathbf{s}^{\top}\mathbf{t}\right)^{j}}{j!}\exp\left(-\frac{1}{2}\|\mathbf{s}\|_{2}^{2}+\left\|\mathbf{t}\right\|_{2}^{2}\right)$
(11) $\displaystyle=$
$\displaystyle\sum_{j=0}^{\infty}\sum_{n_{1}+n_{2}+\cdots+n_{k}=j}\exp\left(-\frac{1}{2}\|\mathbf{s}\|^{2}\right)\frac{s_{1}^{n_{1}}\cdots
s_{k}^{n_{k}}}{\sqrt{n_{1}!\cdots
n_{k}!}}\exp\left(-\frac{1}{2}\left\|\mathbf{t}\right\|^{2}\right)\frac{t_{1}^{n_{1}}\cdots
t_{k}^{n_{k}}}{\sqrt{n_{1}!\cdots n_{k}!}}$ (12) $\displaystyle=$
$\displaystyle\langle\Phi_{\text{rbf}}(\mathbf{s}),\Phi_{\text{rbf}}(\mathbf{t})\rangle,$
(13)

where
$\Phi_{\text{rbf}}(\mathbf{x})=\sum_{j=0}^{\infty}\sum_{\sum_{k}n_{k}=j}\exp\left(-\frac{1}{2}\|\mathbf{x}\|^{2}\right)\frac{s_{1}^{n_{1}}\cdots
s_{k}^{n_{k}}}{\sqrt{n_{1}!\cdots
n_{k}!}}\in\text{span}_{j\in\mathbb{N},\sum_{k}{n_{k}}=j}\\{\frac{x_{1}^{n_{1}}\cdots
x_{k}^{n_{k}}}{\sqrt{n_{1}!\cdots n_{k}!}}\\}$.

Infinite-dimensional Feature Interaction Space Observing the RKHS of such a
RBF kernel,
$\text{span}_{j\in\mathbb{N},\sum_{k}{n_{k}}=j}\\{\frac{x_{1}^{n_{1}}\cdots
x_{k}^{n_{k}}}{\sqrt{n_{1}!\cdots n_{k}!}}\\}$, We note that each of its
dimensions is a j-order interaction within the feature $\mathbf{x}$, given the
fact one of the bases of this RKHS is:

$\\{1,x_{1},\cdots,x_{n},x_{1}^{2},\cdots,x_{1}x_{n},\cdots,x_{n}^{2},\cdots,x_{1}^{j},x_{1}^{j-1}x_{2},\cdots,x_{n}^{j},\cdots\\},$
(14)

which contains an all-order monomial among all elements of the feature
$\mathbf{x}$. This means that we get an infinite-dimensional Hilbert space for
the superposition of interaction information through such an RBF kernel, and
most importantly, each dimension of this space is defined by a feature
interaction. Therefore, we get an infinite-dimensional feature interaction
space.

Figure 2: Comparison of simple representation, finite interaction, and
infinite-dimensional interaction. The ? circle in DemoBlock is chosen from
element-wise Add, element-wise Mul. or RBF kernel.

###  4.3 Demo Case Performance of Models on Different Feature Space

In order to compare networks that utilize the summing superposition of
information mappings on the feature representation space, finite feature
interaction space, and infinite-dimensional interaction space, we design a
demo network with 8 DemoBlock, as shown in Fig. 2. DemoBlock has a two-group
design, the only difference among models in different spaces is the
interaction method at the end of the block (add for simple representation,
multiplication for finite interaction, and RBF kernel for infinite dimensional
interaction), the demo models are trained on CIFAR10 and Tiny-ImageNet.

As shown in Fig. 2, with the procedure of transferring from the simple feature
representation space to a finite feature interaction and eventually to an
infinite-dimensional interaction, the performance on CIFAR10 of the networks
is growing. This is done throughout the training process, implying the
superiority of feature iteration in a high-dimensional interaction space. The
right side of Fig. 2 shows the Class Activation Mapping [43] of three
different demo nets on Tiny-ImageNet. From this, we can see that the network
of feature interaction better reflects the pixel-level correlation within the
image.

##  5 Method

Figure 3: Overview of InfiNet. (a) Four-stage hierarchical InfiNet design. (b)
InfiBlock Design

###  5.1 InfiBlock: Infinite-Dimensional Spatial Feature Interaction

InfiBlock. In this section, we present InfiBlock, the basic block to build the
high-performant InfiNet architecture to achieve infinite-dimensional spatial
feature interaction. As presented in Fig. 3(b), InfiBlock starts with a
LayerNorm layer and subsequently transforms the feature into separate
representations of the two groups through two different linear layers. Then
InfiBlock utilizes a depth-width convolution with an expansion coefficient of
$r$ and a ReLU activation to obtain a feature branch vector of length $r$ on
each group. The feature branch vectors on both groups are then fed into an RBF
kernel for feature interaction. At the same time, we retain a pathway
containing only one depth-wise convolution for summing superposition over the
feature representation space to ensure that the linear connections between
features are not neglected and overfitted. This is followed by a residual
connection with the original input values after passing through a two-layer
MLP. Starting with a input $X^{l}\in\mathbb{R}^{HWC}$, Infi-Block can be
formulated as:

$\displaystyle\hat{X}^{l}=\text{LayerNorm}(X^{l}),$ (15)
$\displaystyle[\vec{Z}_{a}^{l},\vec{Z}_{b}^{l},Z_{c}^{l}]=[\sigma(\text{Conv}_{a}(\hat{X}^{l}W_{a})),\sigma(\text{Conv}_{b}(\hat{X}^{l}W_{b})),\text{Conv}_{c}(\hat{X}^{l})],$
(16) $\displaystyle
X^{l+1}=\text{MLP}(\text{LayerNorm}(K_{rbf}(\vec{Z}^{l}_{a},\vec{Z}^{l}_{b})+Z_{c}^{l}))+X^{l},$
(17)

where
$K_{rbf}(\vec{Z}^{l}_{a},\vec{Z}^{l}_{b})=exp(-\frac{\|\vec{Z}_{a}^{l}-\vec{Z}_{b}^{l}\|_{2}^{2}}{2})$,
is an RBF kernel with out any hyper-parameter, and $\|\cdot\|_{2}^{2}$ is
squared L-2 norm. $\text{LayerNorm}(\cdot)$ is layer normalization [2] and
$\text{MLP}(\cdot)$ is activated by GELU [13].

Specifically, following the ConvNeXt [23], we select a $7\times 7$ depth-wise
convolution to obtain a larger receptive field. We set the expansion
coefficient $r$ as 7 in practice. This means:

$[\vec{Z}_{a}^{l},\vec{Z}_{b}^{l}]=[[Z_{a1}^{l},Z_{a2}^{l},\cdots,Z_{a7}^{l}]^{T},[Z_{b1}^{l},Z_{b2}^{l},\cdots,Z_{b7}^{l}]^{T}],$
(18)

where $Z_{ai}^{l}=\sigma(\text{DW-Conv}_{ai}(X^{l}W_{a}))\
\in\mathbb{R}^{HWC}$. Therefore, the RBF kernel performs feature interaction
on multiple (7) branches of convolution filter and outputs a result still in
$\mathbb{R}^{HWC}$.

###  5.2 Model Architectures

As shown in Fig. 3(a), InfiNet uses the widely adopted 4-stage hierarchical
architecture as in ResNet [12], ConvNeXt [23] and Swin Transformer [22]. We
stack InfiBlocks into each stage. We set the number of channels in each stage
as [C,2C,4C,8C] following common practice. We build a family of InfiNets that
InfiNet-T/S/B/L/XL with model variants hyper-parameters:
$C=\\{64,96,128,128,192\\}$, number of blocks in each stages $=\\{2,2,18,2\\}$
for InfiNet-T/S/B and $\\{3,3,27,3\\}$ for InfiNet-L/XL. A down-sampling with
$2\times 2$ convolution with stride = 2 is used to connect each stage. STEM
[23] is used to connect the input of InfiNet to Stage 1.

##  6 Experiments

We perform a series of experiments to validate the efficacy of InfiNets. The
primary results on ImageNet [8] are showcased and contrasted with several
architectures. Furthermore, our models were evaluated on downstream ADE20K[44]
semantic segmentation and COCO [20] object detection. ImageNet-1K experiments
are conducted on 4$\times$Nvidia A100 GPUs and ImageNet-21K on 16$\times$.

###  6.1 ImageNet Classification

Setups. We conduct image classification experiments on ImageNet-1K [8], which
contains 1.28 million training samples belonging to 1000 classes and 50K
samples for validation. We train the InfiNet-T/S/B/L models for 300 epochs
with AdamW [25] optimizer. We use the cosine learning rate scheduler [24] with
20 warmup epochs and the basic learning rate is set as $4\times 10^{-3}$. The
training resolution is set as $224\times 224$. To further evaluate the
InfiNet’s scalability, we train the InfiNet-L/XL models on 14M-sample
ImageNet-22K dataset for 90 epochs and then fine-tune on ImageNet-1K at
$384\times 384$ resolution for 30 epochs following [23]. More details can be
found in Appendix A.1.

Results. We present our ImageNet experiment results and comparison with
baselines in Table 1. Our models achieve competitive performance with state-
of-the-art. It is worth noting that InfiNet has about 20% fewer FLOPs than the
baseline models with similar parameter scales, but still achieves great
performance. It demonstrates the effectiveness of our proposed generation of
features in infinite-dimensional interaction spaces. Experiments on isotropic
models can be found in Table 3(a).

Table 1: ImageNet classification results. We compare our models with state-of-
the-art models with comparable parameters, the Top-1 accuracy is reported on
the ImageNet-1K validation set.

| Interact. | Params | FLOPs | Top1
---|---|---|---|---
Model | Orders | (M) | (G) | Acc.(%)
ConvNeXt-T[23] | no | 29 | 4.5 | 82.1
SLaK-T[21] | no | 30 | 5.0 | 82.5
Conv2Former-T[14] | 2 | 27 | 4.4 | 83.1
UniFormer-S[18] | 2 | 22 | 3.6 | 82.9
CoAtNet-0[7] | 3 | 25 | 4.2 | 82.7
FocalNet-T[42] | 3 | 28 | 4.4 | 82.1
Swin-T[22] | 3 | 28 | 4.5 | 81.3
HorNet-T[31] | 2-5 | 22 | 4 | 82.8
MogaNet-S[19] | 4 | 25 | 5.0 | 83.5
InfiNet-T | $\infty$ | 23 | 3.2 | 83.4
ConvNeXt-S[23] | no | 50 | 8.7 | 83.1
SLaK-S[21] | no | 55 | 9.8 | 83.8
Conv2Former-S[14] | 2 | 50 | 8.7 | 84.1
UniFormer-B[18] | 2 | 50 | 8.3 | 83.9
CoAtNet-1[7] | 3 | 42 | 8.4 | 83.3
FocalNet-S[42] | 3 | 50 | 8.7 | 83.5
Swin-S[22] | 3 | 50 | 8.7 | 83.0
HorNet-S[31] | 2-5 | 50 | 8.8 | 84.0
MogaNet-B[19] | 4 | 44 | 9.9 | 84.3
InfiNet-S | $\infty$ | 48 | 7.2 | 84.0

| Interact. | Params | FLOPs | Top1
---|---|---|---|---
Model | Orders | (M) | (G) | Acc.(%)
ConvNeXt-B[23] | no | 89 | 15.4 | 83.8
SLaK-B[21] | no | 85 | 17.1 | 84.0
Conv2Former-B[14] | 2 | 90 | 15.9 | 84.4
CoAtNet-2[7] | 3 | 75 | 15.7 | 84.1
FocalNet-B[42] | 3 | 89 | 15.4 | 83.9
Swin-B[22] | 3 | 89 | 15.4 | 83.5
HorNet-B[31] | 2-5 | 87 | 15.6 | 84.3
MogaNet-L[19] | 4 | 83 | 15.9 | 83.5
InfiNet-B | $\infty$ | 82 | 12.8 | 84.5
InfiNet-L | $\infty$ | 116.8 | 19.1 | 84.8
ImageNet-21K Pretrained Models Fine-tuned @$384^{2}$ |
ConvNeXt-L‡[23] | no | 198 | 101 | 87.5
CoAtNet-3‡[7] | 3 | 168 | 107 | 87.6
FocalNet-L‡[42] | 3 | 197 | 101 | 87.3
Swin-L‡[22] | 3 | 197 | 104 | 87.3
HorNet-L‡[31] | 2-5 | 202 | 102 | 87.7
MogaNet-XL‡[19] | 4 | 181 | 102 | 87.8
InfiNet-L‡ | $\infty$ | 116.8 | 60 | 87.8
ConvNeXt-XL‡[23] | no | 350 | 179 | 87.8
InfiNet-XL‡ | $\infty$ | 255.8 | 126 | 88.2

###  6.2 MS COCO Detection

Setups. We evaluate our models for object detection tasks on widely used MS
COCO [20] benchmark. In the experiments, InfiNet-T/S/b/XL serves as the
backbone network within Cascade Mask RCNN [4]. We use AdamW as the optimizer
and a batch size of 16 and adhere to the 3$\times$ schedule, following
ConvNeXt [23] and Swin [22]. We resize the input so that the longer side is at
most 1333 and the short side is at most 800. We initialize the backbone model
with ImageNet-1K pre-trained weights for T/S/B models and ImageNet-22K pre-
trained weights for XL model.

Results. As shown in Table 2, InfiNets comprehensively beat the non-
interactive model ConvNeXt [23], and space-limited interactive Swin [22] and
HorNet [31] under the same cascade Mask RCNN framework in box AP and mask AP.
This means that for such dense prediction tasks, spatial interaction of
features in a high-dimension space is crucial. The InfiNet series model
obtain$0.9\sim 1.5$ box AP and $1.3\sim 2.5$ mask AP gain compared with non-
interactive ConNeXt.

Table 2: Object detection and semantic segmentation results on MS COCO and
ADE20K.

| Object Detection with Cascade Mask R-CNN 3× | Semantic Segmentation with UperNet 160K
---|---|---
Model | AP${}^{\text{box}}$ | AP${}^{\text{mask}}$ | Params | FLOPs | mIoU${}^{\text{ss}}$ | mIoU${}^{\text{ms}}$ | Params | FLOPs
ConvNeXt-T[23] | 50.4 | 43.7 | 86M | 741G | 46.0 | 46.7 | 60M | 939G
Swin-T[22] | 50.4 | 43.7 | 86M | 745G | 44.5 | 45.8 | 60M | 945G
HorNet-T[31] | 51.7 | 44.8 | 80M | 730G | 48.1 | 48.9 | 52M | 926G
InfiNet-T | 51.9 | 46.2 | 77M | 724G | 46.7 | 47.4 | 50M | 924G
ConvNeXt-S[23] | 51.9 | 45.0 | 108M | 827G | 48.7 | 49.6 | 82M | 1027G
Swin-S[22] | 51.8 | 44.7 | 107M | 838G | 47.6 | 49.5 | 81M | 1038G
HorNet-S[31] | 52.7 | 45.6 | 107M | 830G | 49.2 | 49.8 | 81M | 1030G
InfiNet-S | 52.8 | 46.4 | 98M | 802G | 49.4 | 49.9 | 78M | 1002G
ConvNeXt-B[23] | 52.7 | 45.6 | 146M | 964G | 49.1 | 49.9 | 122M | 1170G
Swin-B[22] | 51.9 | 45.0 | 145M | 982G | 48.1 | 49.7 | 121M | 1188G
HorNet-B[31] | 53.3 | 46.1 | 144M | 969G | 50.0 | 50.5 | 121M | 1174G
InfiNet-B | 53.7 | 47.3 | 126M | 906G | 50.2 | 50.9 | 105M | 1111G
ConvNeXt-L‡[23] | 54.8 | 47.6 | 255M | 1354G | 53.2 | 53.7 | 235M | 2458G
Swin-L‡[22] | 53.9 | 46.7 | 253M | 1382G | 52.1 | 53.5 | 234M | 2468G
HorNet-L‡[31] | 55.4 | 48.0 | 251M | 1363G | 54.1 | 54.5 | 232M | 2473G
InfiNet-XL‡ | 56.3 | 48.9 | 273M | 1454G | 54.6 | 55.2 | 253M | 2544G

###  6.3 ADE20 Segmentation

Setups. We evaluate our models for the semantic segmentation task on widely
used ADE20K [44] benchmark covering 150 semantic categories on 25K images, in
which 20K are used for training. We use UperNet [38] for as the basic
framework and adopt InfiNets as the backbone model. Training details follow
the Swin [22], we use AdamW optimizer with learning rate $1\times 10^{-4}$ and
batch size 16.

Results. The right half of Table 2 lists the mIoU and corresponding model size
and FLOPs for different configurations. Our models beat most of the baseline
in the segmentation task. The results show that as the model size increases,
the performance gap between InfiNet and other baselines is getting larger,
illustrating the scalability of InfiNet on segmentation.

###  6.4 Ablation Study

We use the additive operator, Hadamard product, quadratic polynomial kernel
and cubic polynomial kernel, and RBF kernel in the kernel methods section of
InfiNet, respectively, on a tiny size model, to verify the effect of the
gradual expansion of the order of the interaction space up to infinite
dimensions on the performance of the model. As in Table 3(b), we can see that
the performance of the model is gradually improving as the order of the model
interaction space increases up to infinite dimensions.

Table 3: More Results on isotropic models and different kind of Reproducing
Kernel

(a) Isotropic Models |
---|---
| Interact. | Params | FLOPs | Top1
Model | Orders | (M) | (G) | Acc.(%)
ConvNeXt-S(iso.) | no | 22 | 4.3 | 79.7
Conv2Former(iso.) | 2 | 23 | 4.3 | 81.2
DeiT-S[23] | 3 | 22 | 4.6 | 79.8
HorNet-S(iso.) | 2-5 | 22 | 4.5 | 80.6
InfiNet-S(iso.) | $\infty$ | 22 | 4.3 | 81.4

(b) Ablation Study |
---|---
| Interact. | Params | FLOPs | Top1
Model | Orders | (M) | (G) | Acc.(%)
InfiNet-$\oplus$ | no | 23 | 3.2 | 81.6
InfiNet-$\otimes$ | 2 | 23 | 3.2 | 82.1
InfiNet-2-polyno. | 4 | 23 | 3.2 | 82.3
InfiNet-3-polyno. | 6 | 23 | 3.2 | 82.5
InfiNet-T | $\infty$ | 23 | 3.2 | 83.4

##  7 Conclusion

As a conclusion, in this paper, we propose that one of the key points of
success of today’s element-wise multiplication-based models is that they
explore a high-dimensional feature interaction space through feature
interactions. And the RBF kernel can greatly expand this interaction space
into an infinite dimensional feature interaction space. Based on this
observation, we propose InfiNet, a high-performance neural network that
explores infinite-dimensional feature interactions while using a modern model
structure, which has achieved state-of-the-art results on several visual
tasks.

## References

  * [1] N. Aronszajn.  Theory of reproducing kernels.  Transactions of the American mathematical society, 68(3):337–404, 1950.
  * [2] J. L. Ba, J. R. Kiros, and G. E. Hinton.  Layer normalization.  arXiv preprint arXiv:1607.06450, 2016.
  * [3] B. Bai, J. Liang, G. Zhang, H. Li, K. Bai, and F. Wang.  Why attentions may not be interpretable?  In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 25–34, 2021.
  * [4] Z. Cai and N. Vasconcelos.  Cascade r-cnn: Delving into high quality object detection.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6154–6162, 2018.
  * [5] C. Chen, M. Seuret, M. Liwicki, J. Hennebert, and R. Ingold.  Kervolutional neural networks.  arXiv preprint arXiv:1904.03955, 2019.
  * [6] Y. Cho and L. K. Saul.  Kernel methods for deep learning.  In Advances in neural information processing systems, pages 342–350, 2009.
  * [7] Z. Dai, H. Liu, Q. V. Le, and M. Tan.  Coatnet: Marrying convolution and attention for all data sizes.  Advances in neural information processing systems, 34:3965–3977, 2021.
  * [8] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [9] E. Dohmatob, Y. Feng, P. Yang, F. Charton, and J. Kempe.  A tale of tails: Model collapse as a change of scaling laws.  International Conference on Machine Learning, 2024.
  * [10] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In International Conference on Learning Representations, 2020.
  * [11] F.-L. Fan, H.-C. Dong, Z. Wu, L. Ruan, T. Zeng, Y. Cui, and J.-X. Liao.  One neuron saved is one neuron earned: On parametric efficiency of quadratic networks.  arXiv preprint arXiv:2303.06316, 2023.
  * [12] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [13] D. Hendrycks and K. Gimpel.  Gaussian error linear units (gelus).  arXiv preprint arXiv:1606.08415, 2016.
  * [14] Q. Hou, C.-Z. Lu, M.-M. Cheng, and J. Feng.  Conv2former: A simple transformer-style convnet for visual recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
  * [15] J. Hu, L. Shen, and G. Sun.  Squeeze-and-excitation networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132–7141, 2018.
  * [16] A. Krizhevsky, I. Sutskever, and G. E. Hinton.  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25, 2012.
  * [17] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner.  Gradient-based learning applied to document recognition.  Proceedings of the IEEE, 86(11):2278–2324, 1998.
  * [18] K. Li, Y. Wang, G. Peng, G. Song, Y. Liu, H. Li, and Y. Qiao.  Uniformer: Unified transformer for efficient spatial-temporal representation learning.  In International Conference on Learning Representations, 2022.
  * [19] S. Li, Z. Wang, Z. Liu, C. Tan, H. Lin, D. Wu, Z. Chen, J. Zheng, and S. Z. Li.  Moganet: Efficient multi-order gated aggregation network.  International Conference on Learning Representation, 2024.
  * [20] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick.  Microsoft coco: Common objects in context.  In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014.
  * [21] S. Liu, T. Chen, X. Chen, X. Chen, Q. Xiao, B. Wu, T. Kärkkäinen, M. Pechenizkiy, D. C. Mocanu, and Z. Wang.  More convnets in the 2020s: Scaling up kernels beyond 51x51 using sparsity.  In The Eleventh International Conference on Learning Representations, 2023.
  * [22] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021.
  * [23] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie.  A convnet for the 2020s.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11976–11986, 2022.
  * [24] I. Loshchilov and F. Hutter.  SGDR: Stochastic gradient descent with warm restarts.  In International Conference on Learning Representations, 2017.
  * [25] I. Loshchilov and F. Hutter.  Decoupled weight decay regularization.  In International Conference on Learning Representations, 2019.
  * [26] X. Ma, X. Dai, Y. Bai, Y. Wang, and Y. Fu.  Rewrite the stars.  Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024.
  * [27] J. Mairal, P. Koniusz, Z. Harchaoui, and C. Schmid.  Convolutional kernel networks.  In Advances in neural information processing systems, pages 2627–2635, 2014.
  * [28] J. Mercer.  Xvi. functions of positive and negative type, and their connection the theory of integral equations.  Philosophical transactions of the royal society of London. Series A, containing papers of a mathematical or physical character, 209(441-458):415–446, 1909.
  * [29] M. T. Musavi, W. Ahmed, K. H. Chan, K. B. Faris, and D. M. Hummels.  On the training of radial basis function classifiers.  Neural networks, 5(4):595–603, 1992.
  * [30] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  Advances in neural information processing systems, 32, 2019.
  * [31] Y. Rao, W. Zhao, Y. Tang, J. Zhou, S. N. Lim, and J. Lu.  Hornet: Efficient high-order spatial interactions with recursive gated convolutions.  Advances in Neural Information Processing Systems, 35:10353–10366, 2022.
  * [32] B. Schölkopf, A. Smola, and K.-R. Müller.  Nonlinear component analysis as a kernel eigenvalue problem.  Neural computation, 10(5):1299–1319, 1998.
  * [33] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich.  Going deeper with convolutions.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015.
  * [34] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [35] X. Wang, R. Girshick, A. Gupta, and K. He.  Non-local neural networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7794–7803, 2018.
  * [36] A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. P. Xing.  Deep kernel learning.  In Artificial intelligence and statistics, pages 370–378, 2016.
  * [37] S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie.  Convnext v2: Co-designing and scaling convnets with masked autoencoders.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16133–16142, 2023.
  * [38] T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun.  Unified perceptual parsing for scene understanding.  In Proceedings of the European conference on computer vision (ECCV), pages 418–434, 2018.
  * [39] C. Xu, X. Wang, F. Yu, J. Xiong, and X. Chen.  Quadranet v2: Efficient and sustainable training of high-order neural networks with quadratic adaptation.  arXiv preprint arXiv:2405.03192, 2024.
  * [40] C. Xu, F. Yu, Z. Xu, C. Liu, J. Xiong, and X. Chen.  Quadranet: Improving high-order neural interaction efficiency with hardware-aware quadratic neural networks.  The 29th Asia and South Pacific Design Automation Conference, 2024.
  * [41] Z. Xu, F. Yu, J. Xiong, and X. Chen.  Quadralib: A performant quadratic neural network library for architecture optimization and design exploration.  Proceedings of Machine Learning and Systems, 4:503–514, 2022.
  * [42] J. Yang, C. Li, X. Dai, and J. Gao.  Focal modulation networks.  Advances in Neural Information Processing Systems, 35:4203–4217, 2022.
  * [43] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba.  Learning deep features for discriminative localization.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929, 2016.
  * [44] B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba.  Scene parsing through ade20k dataset.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 633–641, 2017.

##  Appendix A Appendix

###  A.1 Training Details

The training details for ImageNet experiments are shown in Table 4 and Table
5.

Table 4: Training details for ImageNet-1K experiments Configuration | InifNet-T/S/B/L
---|---
Input resolution | 2242
Epochs | 300
Batch size | 192/128/64/64
Optimizer | AdamW
AdamW $(\beta_{1},\beta_{2})$ | $0.9,0.999$
Learning rate | 0.004
Learning rate decay | Cosine
Weight decay | 0.05
Warmup epochs | 20
Label smoothing $\epsilon$ | 0.1
Stochastic Depth | Y
Rand Augment | 9/0.5
Repeated Augment | Y
Erasing prob. | 0.25
ColorJitter | N
Gradient Clipping | Y
EMA decay | Y
Table 5: Training details for ImageNet-21K experiments Configuration | IN-21K PT | IN-1K FT
---|---|---
| L | XL | L | XL
Input resolution | 2242 | 3842
Epochs | 90 | 30
Batch size | 256 | 64
Optimizer | AdamW | AdamW
AdamW $(\beta_{1},\beta_{2})$ | $0.9,0.999$ |  $0.9,0.999$
Learning rate | $4\times 10^{-3}$ |  $5\times 10^{-5}$
Learning rate decay | Cosine | Cosine
Weight decay | 0.1 | 0.00001
Warmup epochs | 5 | 0
Label smoothing $\epsilon$ | 0.2 | 0.1 | 0.2
Rand Augment | 9/0.5 | 9/0.5
Repeated Augment | N | N
Erasing prob. | 0.25 | 0.25
Gradient Clipping | 5 | 5
EMA decay | N | Y

###  A.2 More Visualization Comparison

Figure 4: Visualization Comparison of (1) Feature Representation Space model,
(2) Finite Feature Interaction Space model, (3) Infinite-Dimensional Feature
Interaction model

In Fig. 4, we provide more Class Activation Mapping to illustrate the
interpretability of interaction models. Infinite-dimensional Interaction
models capture more totality of objects within a category, illustrates the
importance of this infinite-dimensional interaction.

###  A.3 Limitations

Although we explored polynomial kernels and RBF kernels, there are still a
wide variety of kernel methods that we did not explore, including a learnable
kernel. In addition, to avoid additional hyperparameter tuning, we fixed the
$\sigma$ parameter in the RBF kernel to 1. This may have deprived us of the
possibility of exploring the optimal InfiNet, but due to the high cost of
training the model, we will leave the impact of this hyperparameter on the
InfiNet as a follow-up work.

###  A.4 Broader Social Impact

InfiNet is a state-of-the-art vision neural network architecture. The
advancements in computer vision neural network architectures hold significant
potential for positive societal impact, particularly in enhancing healthcare
diagnostics, improving security systems, and advancing autonomous
transportation. However, it is crucial to address potential negative
implications such as privacy concerns, algorithmic biases, and job
displacement. Ensuring ethical development and deployment involves
implementing strict data protection measures, promoting fairness and
inclusivity in algorithm design, and supporting workforce retraining programs.
By proactively managing these challenges, we can maximize the benefits of
computer vision technologies while minimizing their risks. Engaging with
diverse stakeholders will be essential to ensure these technologies are used
responsibly and equitably.


# Improving the Generalizability of Collaborative Dialogue
Analysis With Multi-Feature Embeddings

Ayesha Enayet
University of Central Florida
Orlando, FL USA
<EMAIL_ADDRESS>
&Gita Sukthankar
University of Central Florida
Orlando, FL USA
<EMAIL_ADDRESS>

###### Abstract

Conflict prediction in communication is integral to the design of virtual
agents that support successful teamwork by providing timely assistance. The
aim of our research is to analyze discourse to predict collaboration success.
Unfortunately, resource scarcity is a problem that teamwork researchers
commonly face since it is hard to gather a large number of training examples.
To alleviate this problem, this paper introduces a multi-feature embedding
(MFeEmb) that improves the generalizability of conflict prediction models
trained on dialogue sequences. MFeEmb leverages textual, structural, and
semantic information from the dialogues by incorporating lexical, dialogue
acts, and sentiment features. The use of dialogue acts and sentiment features
reduces performance loss from natural distribution shifts caused mainly by
changes in vocabulary.

This paper demonstrates the performance of MFeEmb on domain adaptation
problems in which the model is trained on discourse from one task domain and
applied to predict team performance in a different domain. The
generalizability of MFeEmb is quantified using the similarity measure proposed
by Bontonou et al. (2021). Our results show that MFeEmb serves as an excellent
domain-agnostic representation for meta-pretraining a few-shot model on
collaborative multiparty dialogues.

##  1 Introduction

For many natural language processing applications, the ability to learn
features that generalize well across multiple datasets is a key desideratum
Saikia et al. (2020). This paper introduces a new multi-feature embedding,
MFeEmb, that increases the generalizability of models learned from
collaborative multiparty dialogues. Dialogues are different from single-author
documents in that, along with textual information, they contain communication
patterns that may serve as indicators of social dynamics. Treating a dialogue
as a mere text collection ignores valuable information. We advocate exploiting
implicit features present in multiparty dialogues that are less vulnerable to
distribution shifts resulting from task domain changes.

This paper demonstrates the usage of MFeEmb on a communication analysis task:
conflict prediction. Teamwork research faces a challenge of resource scarcity
since the human subjects datasets are quite small (less than 100 samples), due
to the difficulty of recruiting teams and the time consuming nature of many
group tasks. A variety of social phenomena have been investigated within team
communication research including entrainment Rahimi and Litman (2020) and
emergent leadership Maese et al. (2021). Frequency of communication is not in
itself a good predictor of team performance, but a meta-analysis conducted by
Marlow et al. (2018) that drew upon data from 150 studies conducted on 9702
teams concluded that high quality communication is positively related to team
performance in many task domains. Conversely, process conflict, “disagreement
among group members about the content of the tasks being performed, including
differences in viewpoints, ideas, and opinions” Jehn (1995), is usually
negatively correlated with taskwork success.

Our aim is to be able to learn a model to classify process conflict from
multiparty dialogues that generalizes well across multiple tasks. We treat the
task of conflict prediction as a binary classification task with high conflict
and low conflict being the two classes; the ground truth used by the conflict
prediction model is measured using a post-task team process conflict survey.
This paper focuses on three collaborative problem-solving tasks: software
engineering, search and rescue, and cooperative gameplay.

Our proposed embedding, MFeEmb, leverages textual, structural, and semantic
information from the dialogues by incorporating vocabulary, dialogue acts, and
sentiment features. Lexical embeddings such as word2Vec and BERT Devlin et al.
(2018) show good performance across multiple NLP tasks on in-domain test sets
but are less robust to domain shift. Previous work identified that dialogue
acts and sentiment sequences are informative features that predict conflict
reliably even at the earliest stage of team problem-solving Enayet and
Sukthankar (2021a); however, classifiers constructed using these features
still experience lackluster transfer performance when applied to new datasets,
particularly when detecting high conflict examples Enayet and Sukthankar
(2021b).

To address this transfer problem, we propose the usage of MFeEmb, specifically
as a meta-pretraining representation to be used within a few-shot model.
MFeEmb combines the strengths of both domain-invariant and domain-specific
features. This paper compares the generalizability potential of the MFeEmb
embedding vs. standard word embeddings using inter-class and intra-class based
similarity measures, proposed by Bontonou et al. (2021). Then we evaluate the
performance of MFeEmb in a domain adaptation scenario in which the model is
trained on discourse from one task domain and used to predict conflict in a
different domain. Our results show that:

  1. 1.

MFeEmb demonstrates superior generalizability over other embeddings for
collaborative multiparty dialogues.

  2. 2.

MFeEmb is an excellent representation choice for the meta-training stage of
few-shot learning.

  3. 3.

The domain adaptation performance of MFeEmb can be easily enhanced by task
specific synonym replacement.

##  2 Related Work

Previous studies on group interaction tasks such as conflict prediction Rahimi
and Litman (2020), disruptive talk detection Park et al. (2022), group
satisfaction Lai and Murray (2018), and task performance prediction Kubasova
et al. (2019); Murray and Oertel (2018) have focused on simply improving
performance on in-domain datasets. Very little attention has been paid to the
problem of creating generalizable models for multiparty dialogue that can be
used when training data is scarce. The intelligent tutoring system community
has empirically assessed the generalizability of common natural language
representations, such as BERT and Linguistic Inquiry Word Count (LIWC), across
collaborative problem solving tasks Pugh et al. (2022), but without
investigating methods to improve generalizability.

In domain adaptation, the goal is to train a model on data from a source
domain that performs well on a test dataset drawn from a different target
distribution. Common NLP tasks (e.g., part-of-speech (POS) tagging and named
entity recognition (NER)) have been tackled using techniques including
instance weighting Jiang and Zhai (2007) or explicitly identifying feature
correspondences between the domains Blitzer et al. (2006). An alternate
approach is to learn a single representation that generalizes well across
multiple domains. This can be done using few-shot learning Wang et al. (2020),
one of the most widely used approaches to dealing with resource scarcity. The
traditional framework comprises meta-training and meta-testing phases, where
the aim of meta-training is to learn universal representations from multiple
domains.

Triantafillou et al. (2021) introduced a method that improves few-shot
generalizability by making use of multiple datasets in order to learn a
universal template. Dvornik et al. (2020) proposed Selecting from Universal
Representations (SUR), which involves learning a multi-domain representation
by training multiple feature extractors. A multi-domain feature bank is used
to select the most relevant feature during the learning phase. Rather than
seeking to learn the new representation entirely from data, our research
exploits similarities in dialogue act sequences and sentiment patterns
commonly observed during successful collaborative problem-solving.

Representation choice has been shown to place an upper bound on target domain
performance Ben-David et al. (2006). Few-shot frameworks such as Meta-
pretraining then Meta Learning (MTM) Deng et al. (2019) have assumed that word
embeddings like BERT that are trained on large datasets are the best choice
for task agnostic pre-training. Bontonou et al. (2021) introduced a method to
quantify the generalizability of a few-shot classifier under supervised,
unsupervised and semi-supervised settings. This paper uses their inter-class
and intra-class based generalizability measure to evaluate MFeEmb vs. simple
word-based embeddings under supervised classification scenarios. Our research
demonstrates that MFeEmb is superior to word embeddings as a meta- pretraining
representation.

##  3 Methodology

This section describes our approach to learning a generalizable embedding from
multi-party dialogues for conflict prediction. We discuss our datasets,
introduce our embedding, and show how our technique can be used in combination
with data augmentation and few-shot learning.

###  3.1 Datasets

Datasets collected from different collaborative problem-solving task domains
were used in our study of generalizability:

  1. 1.

Teams corpus Litman et al. (2016): this dataset consists of dialogue from 62
teams playing a cooperative board game in groups of three or four. Each team
plays the game twice together. The Teams corpus was originally created to
study entrainment, a linguistic phenomena in which teammates adopt similar
speech patterns Rahimi and Litman (2020). The Game1 dataset of Teams corpus
contains 62 dialogues, 32 low conflicts, and 30 high conflict dialogues. The
Game2 dataset of Teams corpus contains 62 dialogues, 33 low conflicts, and 29
high conflict dialogues.

  2. 2.

ASIST dataset Huang et al. (2022): This dataset consists of 67 teams of three
people participating in a simulated search and rescue task within the
Minecraft game environment. Participants completed two different missions that
involved searching a map and triaging victims. The dataset was collected by
the ASIST project to stimulate the development of proactive assistant agents
for helping human teams. The dataset contains 113 dialogues, 58 low conflicts,
and 55 high conflict dialogues.

  3. 3.

GitHub social coding dataset Enayet and Sukthankar (2020): This dataset was
mined directly from the GitHub social coding platform. It consists of data
from issue comments of teams developing open source software over a period of
months. Teams vary in size, and comments were harvested for 50 reported
issues. The dataset contains 50 dialogues, 29 low conflicts, and 21 high
conflict dialogues.

Both the Teams and ASIST datasets contain post-task process conflict survey
data for all teams, which we divide into high and low conflict groups using
their z-scores. For GitHub, process conflict was scored according to an issue
resolution rubric (described in Appendix G).

###  3.2 Multi-Feature Embedding (MFeEmb)

This paper introduces the MFeEmb embedding which is designed to capture the
dialogues’ structural, semantic, and textual information for collaborative
task success prediction. To represent the structural information, we
incorporate information from dialogue acts (DAs) of the utterances. For
semantics, the sentiment polarities of the utterances are used, although DAs
capture both semantic and structural information. Textual information is
extracted from the vocabulary of the dialogues.

For the word embedding, we use both the Distributed Bag of Words and Dynamic
Memory models of Doc2Vec Le and Mikolov (2014) to learn embeddings (see
Appendix B). Although there is only 28% vocabulary overlap between the ASIST
and Teams datasets and 35% overlap between the GitHub and Teams datasets, word
embeddings can help preserve high performance on the training dataset, while
including structural and semantic features makes the embedding more robust to
domain shifts.

For the dialogue act (DA) embedding, we first map the sequence of utterances
to a sequence of DAs using USE-DAC (Universal Sentence Encoder Dialogue Act
Classifier, described in Appendix A). The SwDA-DAMSL tagset was used to
categorize dialogue acts. The TextBlob python module was used to assign
sentiment polarities ranging from -1 to 1 to each of the utterances.

To generate the embeddings, we use the Dynamic Memory model of Doc2Vec due to
the small vocabulary size of the sequences, which is limited by the number of
DA tags and sentiment gradations. The Dynamic Memory model leverages context
when generating embeddings, thus preserving information contained in these
communication patterns. In contrast, the Distributed Bag of Words model does
not consider context when generating embeddings. For the few-shot results, we
also report results with pre-trained Word2Vec embeddings. First, we separately
learn three embeddings from the sequence of DAs, sentiments, and utterances
(text); the final MFeEmb embedding is created either by concatenating the
three embeddings or by using LSTMs to learn a concatenation ensemble model. We
have made our code available at https://github.com/ayeshaEnayet/MFeEmb.git.

###  3.3 Corpus-Based Feature Analysis

Figure 1: Sentiment polarity distribution of the high conflict vs. low
conflict classes in the Teams dataset

To understand the ramifications of our feature selections, we performed
frequency distribution analyses across the high conflict and low conflict
classes of the Teams Dataset. This analysis shows that the high conflict class
has a high frequency of negative sentiment polarities compared to the low
conflict class and a comparable frequency of positive sentiment polarities
compared to the low conflict class (Figure 1).

In the dialogue act distribution, Statement-non-opinion (sd) is the most
frequent tag in both classes. The low conflict class has a high frequency of
positive communication indicators like Appreciation (ba), Conventional-closing
(fc), and Thanking (ft) compared to the high conflict class. The high conflict
class contains a high frequency of bad communication indicators like
Uninterpretable (%), Hedge (h), Signal-non-understanding (br), and Apology
(fa). Interestingly, high conflict classes have a high frequency of all
categories of questions compared to low conflict classes (see dialogue act
distributions and n-grams in Appendix H).

Looking at the vocabulary distribution, the high conflict class contains more
profanity words than the low conflict class, and there is no overlap between
the profanity word lists of both classes. Our analysis reveals that there is
value in all three types of features (dialogue acts, sentiment polarity, and
vocabulary) but that conflict prediction remains a challenging classification
problem.

###  3.4 Synthetic Datasets

To further improve generalization, we augment our training data with synthetic
datasets generated using synonym replacement, as proposed by Wei and Zou
(2019). Our data augmentation strategies are described below:

  1. 1.

SynReplace: We augment Teams Game1 and Game2 by replacing the words with
synonyms drawn from WordNet.

  2. 2.

ASISTReplace: We augment Teams Game1 and Game2 by replacing the words with
only the synonyms present in the ASIST dataset. First, we extract the
vocabulary of the ASIST dataset. During the replacement operation, we search
for synonyms in WordNet and only replace them with the synonyms present in the
ASIST dataset’s vocabulary.

  3. 3.

GitReplace: Similar to ASISTReplace, we generate our third dataset by
replacing the words with only the synonyms present in the GitHub dataset.

Four synthetic dialogues are generated for each dialogue of the Teams dataset
after applying random replacement on 10% of the words. Our intuition is that
collaborative problem-solving domains such as software engineering may contain
a lot of task specific jargon, and even simple synonym replacement techniques
greatly facilitates generalization.

In our experiments, the basic synonym replacement did not significantly change
the intent and sentiment of the utterances. To show the robustness of dialogue
acts and sentiment sequences towards data augmentation, we utilize TextAttack
Morris et al. (2020), a python package for adversarial attack and data
augmentation, to generate a Teams Game2 synthetic dataset. Word Swap by BERT-
Masked LM transformation was employed to generate synthetic examples from the
Teams Game2 dataset. One synthetic example is generated per dialogue of the
Game2 dataset. The synthetic dataset contains $\approx 50\%$ more unique words
than the original Game2 dataset (Figure 2). Hamming distance was used to
calculate the difference between the sequences of the Game2 original and Game2
synthetic datasets. On average, the adversarial synthetic dataset only
resulted in a 11% change in DA sequences and 14% change in sentiment sequences
(Appendix J).

Figure 2: Vocabulary overlap between original Game1, original Game2, and the
Game2 adversarially generated dataset

##  4 Experimental Setup

The Teams corpus contains 124 team dialogues from 62 different teams, playing
two different collaborative board games. We use the Teams Game1 dataset with
62 total samples, divided into 32 low and 30 high conflict samples, as our
training dataset. The small training dataset ensures that the experiments
reflect the generalization performance under the resource scarcity scenario.
Our test datasets for evaluating domain adaptation are Teams Game2, GitHub,
and ASIST. Obviously the domain shift is the smallest between the Teams Game 1
and 2 datasets. We use the GitHub and ASIST datasets to check the
transferability of MFeEmb under domain shift. The model was not fine-tuned
before evaluating the performance on GitHub and ASIST.

We evaluate our proposed MFeEmb under the following three experimental setups:

  1. 1.

SVM and logistic regression classifiers to distinguish high conflict and low
conflict classes.

  2. 2.

LSTM concatenation ensemble.

  3. 3.

Few-shot learning approach.

We benchmark MFeEmb against prior work on conflict prediction, other embedding
choices, and FsText, a few-shot model proposed by Bailey and Chopra (2018).
Experiments were performed using a 300-dimensional version of MFeEmb where the
length of all the three embeddings is the same, i.e., 100. We report the mean
and standard devication of F1-Scores after 15 runs. For the SVM and logistic
regression classification experiments, to improve the readability, we only
report the best-performing classifier results measured by mean F1 score in
Section 5; for the results of both classifiers, see Appendix K. ‘*’ denotes
that logistic regression was the top performer, and ‘+’ denotes cases where
the SVM was the best.

###  4.1 SVM and Logistic Regression

After Doc2Vec is used to generate the three embeddings for each sample, the
embeddings are concatenated to create MFeEmb. We use both SVM and logistic
regression to classify the instances and report the results of both the
classifiers. For DAs and sentiment sequences, we always use the Dynamic Memory
model (DM) of Doc2Vec.

###  4.2 Few-Shot Learning (FsText)

For few-shot learning, we use the method proposed by Bailey and Chopra (2018)
and available in the FsText Python module. The training document for the meta-
training stage of few-shot learning is represented using a pre-trained word
embedding (Word2Vec). In the case of more than one training sample per class,
the proposed method works by averaging each class’s vectors to calculate the
most effective class representative. Cosine similarity is used to measure the
distance between the test sample and each class representative, and the test
sample is assigned the label of the class with the highest similarity. We
compare the generalizability of FsText (Original) with MFeEmb-based FsText, by
replacing Word2Vec embedding with MFeEmb during the meta-training stage.

###  4.3 Concatenation Ensemble

Due to the small size of the training set, we apply the synonym replacement
technique proposed by Wei and Zou (2019) to augment the training data as
described in Section 3.4. One hot encoding is used to encode DA, sentiment
polarities, and vocabulary to train the model. We train three different
Bidirectional LSTM models, one on each of DAs, sentiments, and word-based
documents, and merge them to create our MFeEmb based ensemble. Our
Bidirectional LSTM models for each feature have an embedding layer, an LSTM
layer, one dropout layer, and one deep layer.

###  4.4 Baseline Models

We compare our proposed MFeEmb’s results with several baseline models that use
the same binary classification setup for conflict prediction. First, we show
that MFeEmb performs competitively against prior work on conflict prediction
Enayet and Sukthankar (2021a) using the proposed dialogue act only and
sentiment only embeddings. Note that our results are not directly comparable
to what was reported in the previous work because we use a reduced training
set; thus, we reimplemented the embeddings. We also compare MFeEmb to the
commonly used BERT based embedding (Appendix D).

These independent baselines are compared against three implementation options
for MFeEmb: 1) MFeEmb with simple binary classifier (SVM or logistic
regression), 2) MFeEmb concatenation ensemble learned with LSTMs (Sec. 4.3)
trained on the synonym replaced augmented dataset, 3) a variation of few-shot
learning method (FsText) Bailey and Chopra (2018) in which the Word2Vec
embedding is replaced with MFeEmb during the meta-training stage. For training
and testing, we concatenate all the utterances of the dialogue into one single
document and assign it to one of the classes depending on the conflict score
of the team.

##  5 Results

Figure 3: Performance of MFeEmb vs. other embedding choices from prior work.
Figure 4: Performance of MFeEmb with and without word embedding (WE).

This section presents results on the generalizability of MFeEmb under
different experimental setups.

###  5.1 Similarity Based Evaluation

First we quantify the potential generalization of the representation using the
similarity measure proposed by Bontonou et al. (2021). The similarity measure
is given by:

$intra(c)=\frac{1}{k(k-1)}\sum_{\begin{subarray}{c}i\\\
y_{i}=c\end{subarray}}\sum_{\begin{subarray}{c}j\neq i\\\
y_{j}=c\end{subarray}}\cos{(f_{i},f_{j})}$ (1)
$inter(c,\tilde{c})=\frac{1}{k^{2}}\sum_{\begin{subarray}{c}i\\\
y_{i}=c\end{subarray}}\sum_{\begin{subarray}{c}j\neq i\\\
y_{j}=\tilde{c}\end{subarray}}\cos{(f_{i},f_{j})}$ (2)
$similarity=\frac{1}{N}\sum_{c=1}^{N}(intra(c)-\max\limits_{c\neq\tilde{c}}(inter(c,\tilde{c})))$
(3)

where $c$ is class, $N$ is the number of classes, $k$ is number of examples,
$f$ is the embedding, $intra(c)$ is cosine similarity within a class, and
$inter(c,\tilde{c})$ is cosine similarity through classes $c$ and $\tilde{c}$.
The final similarity score reflects the comparison of the $intra(c)$ and
$inter(c,\tilde{c})$. Intuitively it can be seen that the score measures how
the representation affects the data clustering within and between classes.

We compare our proposed MFeEmb vs. a standard word embedding learned using the
bag of word model of Doc2Vec. Table 1 gives the result of the similarity-based
analysis, juxtaposed with the classification results. MFeEmb has a better
similarity score and high classification performance, compared to word-based
embeddings indicating the high generalizability potential of MFeEmb.

Word_Emb | MFeEmb
---|---
Teams Game2
similarity |  F1_score |  similarity |  F1_score
-0.067 |  0.470* |  -0.016 |  0.628+
GitHub
similarity |  F1_score |  similarity |  F1_score
-0.067 |  0.463* |  -0.017 |  0.501+
ASIST
similarity |  F1_score |  similarity |  F1_score
-0.067 |  0.446+ |  -0.016 |  0.458+
Table 1: Similarity-based generalizability analysis. Word_Emb: Distributed Bag
of Words document embedding. MFeEmb: Multi-Feature Embedding generated using
the Dynamic Memory model of Doc2Vec. The similarity score of MFeEmb accurately
predicts higher classification accuracy. ‘*’ denotes the logistic regression
results, and ‘+’ denotes the SVM results.

###  5.2 MFeEmb Performance Summary

Figure 3 provides the overall comparison of MFeEmb vs. the benchmark
embeddings. In the case where minimal domain adaptation was required (testing
classifiers on Teams2 that were trained on Teams1), the simple version of
MFeEmb using a SVM classifier is the top performer and outperforms the
embeddings used in other prior work on conflict prediction Enayet and
Sukthankar (2021a). Our most consistent model, MFeEmb with FsText, had a
significantly higher F1 score on the high conflict class compared to baseline
models (see Table 2). Note that detecting the high conflict examples is more
valuable for practical implementations.

For the more complex domain adaptation scenarios (GitHub and ASIST), the best
performance was achieved using MFeEmb as a replacement for the Word2Vec
embedding during the meta-training phase of FsText on GitHub, and the
concatenation ensemble showed significantly better performance on the ASIST
dataset. The vanilla MFeEmb generally performed comparably to the
concatenation ensemble using LSTMs on out of domain datasets. The latter
showed a high standard deviation compared to the former.

To analyze the importance of incorporating word embedding in MFeEmb, we
compare the performance of all the experimental setups with and without word
embedding (WE). For SVM & Logistic regression (Basic) and FsText, we train the
model on the Teams Game1 dataset, and for concatenation ensemble, we train on
the synonym replaced dataset. One of our main objectives in incorporating the
word embedding in MFeEmb is to maintain the performance on the in-domain
dataset, and results show that MFeEmb performed better with word embedding on
the in-domain dataset. For most transfer case setups, MFeEmb with word
embedding either gave better or comparable mean F1 scores (Figure 4). The
following sections present a more in-depth evaluation of each experimental
setup.

High Conflict Class Prediction Summary
---
Method | GitHub | ASIST
BERT_SynReplace |  0.431 |  0.347
DA_only_Team1 |  0.320* |  0.311*
Senti_only_Team1 |  0.207* |  0.300*
MFeEmb_FsText_Team1 |  0.564 |  0.478
Table 2: Summary of high conflict class F1 scores. ‘*’ denotes the logistic
regression results, and ‘+’ denotes the SVM results.

###  5.3 SVM and Logistic Regression

Table 3 gives the results for the SVM and logistic regression classifiers.
This paper presents a thorough evaluation of the performance of different
embedding choices (DM, DBOW). We also evaluate the performance of different
data augmentation methods (SynReplace, ASISTReplace, and GitReplace).

Our proposed MFeEmb trained using Doc2Vec and classified using either SVM or
logistic regression performed better than the word-embedding baseline.
Leveraging synthetic datasets yielded significant performance improvements. In
our most challenging resource-scarce scenario, where we trained the model only
on the Teams Game1 dataset, incorporating word embedding showed better
performance on the Teams Game2 and GitHub datasets, while the model performed
better on the ASIST dataset without word embedding (see Figure 4).

SVM & Logistic Regression Results
---
Method |  Teams Game2
F1_score (std) |  GitHub
F1_score (std) |  ASIST
F1_score (std)
Baseline Doc2Vec_dbow |  0.465 (0.070)* |  0.489 (0.080)* |  0.425 (0.091)*
MFeEmb_Team1_dbow |  0.533 (0.068)* |  0.437 (0.025)* |  0.347 (0.002)*
MFeEmb_Team1_dm |  0.625 (0.0295)+ |  0.495 (0.012)+ |  0.473 (0.023)+
MFeEmb_SynReplace |  0.558(0.035)+ |  0.296(0.025)* |  0.318 (0.00)*
MFeEmb_GitReplace |  0.676 (0.033)+ |  0.409 (0.039)* |  0.411 (0.041)*
MFeEmb_ASISTReplace |  0.675 (0.041)+ |  0.537 (0.060)* |  0.480 (0.042)*
Concatenation Ensemble Results
Baseline_SynReplace |  0.435 (0.048) |  0.414 (0.104) |  0.397 (0.081)
MFeEmb_SynReplace |  0.453 (0.044) |  0.429 (0.122) |  0.459 (0.044)
MFeEmb_GitReplace |  0.464 (0.044) |  0.468 (0.098) |  0.491 (0.054)
MFeEmb_ASISTReplace |  0.408 (0.075) |  0.516 (0.100) |  0.455 (0.059)
Few-Shot Learning Results
FsText Baseline |  0.689 (0.0) |  0.330 (0.0) |  0.338 (0.0)
MFeEmb_Team1_doc2Vec |  0.60 (0.028) |  0.583 (0.045) |  0.451 (0.025)
MFeEmb_Team1_word2Vec |  0.597 (0.041) |  0.507 (0.063) |  0.437 (0.027)
MFeEmb_SynReplace |  0.544 (0.021) |  0.568 (0.031) |  0.435 (0.037)
MFeEmb_GitReplace |  0.684 (0.033) |  0.567 (0.041) |  0.388 (0.266)
MFeEmb_ASISTReplace |  0.664 (0.042) |  0.608 (0.034) |  0.462 (0.053)
Table 3: Detailed performance evaluation of MFeEmb. ‘*’ denotes the logistic
regression results, and ‘+’ denotes the SVM results.

###  5.4 Concatenation Ensemble Model

Table 3 gives the results for the LSTM-based concatenation ensemble model. The
model showed a better mean F1-score than the text-based LSTM model. We also
trained the LSTM using synthetic datasets generated using GitHub and ASIST
vocabularies, which showed better performance, specifically with the GitHub
vocabulary dataset. The model performed significantly better on the ASIST
dataset compared to the other experimental setups.

###  5.5 Few-Shot Model (FsText)

The FsText baseline showed the best performance on Game2, but the performance
degraded considerably on the transfer task (GitHub and ASIST). FsText with the
proposed MFeEmb exhibited significantly better performance on the GitHub and
ASIST datasets, specifically with ASIST vocabulary’s synthetic dataset. FsText
with the proposed MFeEmb embedding also gave a comparable performance on the
Teams Game2 dataset. This demonstrates that MFeEmb is an excellent
representation for meta-pretraining a few-shot model on collaborative
multiparty dialogues even when learned from a small dataset (see Table 3).

Using a synthetic dataset showed a performance improvement in all three
experimental setups. Generation of the synthetic dataset using the vocabulary
of other collaborative tasks showed comparatively better performance on the
transfer task. Even in the in-domain experiments, the Game1 Synthetic dataset,
generated using collaborative task vocabulary, showed the best and comparable
performance on Game2 in all the experimental setups.

##  6 Conclusion and Future Work

This paper introduces a multi-feature embedding (MFeEmb) to improve the
generalizability of multiparty dialogue models under resource scarcity
scenarios. We propose the use of a combination of textual (words), structural
(DAs), and semantic (sentiment, DAs) embeddings to reduce the performance loss
due to natural distribution shift. Experiments show that the multi-feature
embedding performs significantly better than sentence (BERT), dialogue act-
only, sentiment-only, and word embeddings. Our results demonstrate that MFeEmb
is a superior representation for meta-pretraining a few-shot model that works
well across different collaborative problem-solving domains.

Our proposed data augmentation strategy successfully resolved the domain shift
problem caused by task-specific vocabulary without perturbing the dialogue act
and sentiment features. Experiments with synthetic datasets show that synonym
replacement with vocabulary drawn from a collaborative task outperforms
generic synonym replacement with WordNet. It improves both the transfer
accuracy and the test accuracy on the in-domain test set. Note that we did not
fine-tune the models on the target datasets, i.e., GitHub and ASIST, and
strictly report the model learned on the Teams dataset. Only the vocabulary of
these datasets was used to boost the performance; explicit fine-tuning of the
machine learning models could further improve the results.

##  7 Limitations

This paper only reports results on the generalizability of MFeEmb on conflict
prediction tasks; MFeEmb may not perform as well on other communication
analysis tasks. However, we believe that modifying the features used in the
embedding can address this problem. In future work, we are interested in
applying our embedding to new team communication analysis tasks such as
identifying emergent leadership.

##  8 Acknowledgments

This material is based upon work supported by the Defense Advanced Research
Projects Agency (DARPA) under Contract No. W911NF-20-1-0008 and the National
Institute of Mental Health of the National Institutes of Health under Award
Number P50MH126231. Any opinions, findings and conclusions or recommendations
expressed in this material are those of the authors and do not necessarily
reflect the views of DARPA, the National Institutes of Health, and the
University of Central Florida.

## References

  * Bailey and Chopra (2018) Katherine Bailey and Sunny Chopra. 2018.  Few-shot text classification with pre-trained word embeddings and a human in the loop.  _arXiv preprint arXiv:1804.02063_.
  * Ben-David et al. (2006) Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2006.  Analysis of representations for domain adaptation.  In _Advances in Neural Information Processing Systems_ , volume 19. MIT Press.
  * Blitzer et al. (2006) John Blitzer, Ryan McDonald, and Fernando Pereira. 2006.  Domain adaptation with structural correspondence learning.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_ , pages 120–128, Sydney, Australia. Association for Computational Linguistics.
  * Bontonou et al. (2021) Myriam Bontonou, Louis Béthune, and Vincent Gripon. 2021.  Predicting the generalization ability of a few-shot classifier.  _Information_ , 12(1):29.
  * Deng et al. (2019) Shumin Deng, Ningyu Zhang, Zhanlin Sun, Jiaoyan Chen, and Huajun Chen. 2019.  When low resource nlp meets unsupervised language model: Meta-pretraining then meta-learning for few-shot text classification.  _arXiv preprint arXivi1908.08788_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Dvornik et al. (2020) Nikita Dvornik, Cordelia Schmid, and Julien Mairal. 2020.  Selecting relevant features from a multi-domain representation for few-shot classification.  In _European Conference on Computer Vision_ , pages 769–786. Springer.
  * Enayet and Sukthankar (2020) Ayesha Enayet and Gita Sukthankar. 2020.  A transfer learning approach for dialogue act classification of GitHub issue comments.  _CoRR_ , abs/2011.04867.
  * Enayet and Sukthankar (2021a) Ayesha Enayet and Gita Sukthankar. 2021a.  Analyzing team performance with embeddings from multiparty dialogues.  In _IEEE International Conference on Semantic Computing (ICSC)_ , pages 33–39.
  * Enayet and Sukthankar (2021b) Ayesha Enayet and Gita Sukthankar. 2021b.  Learning a generalizable model of team conflict from multiparty dialogues.  _International Journal of Semantic Computing_ , 15(04):441–460.
  * Huang et al. (2022) Lixiao Huang, Jared Freeman, Nancy Cooke, Samantha Dubrow, John “JCR” Colonna-Romano, Matt Wood, Verica Buchanan, Stephen Caufman, and Xiaoyun Yin. 2022\.  Artificial Social Intelligence for Successful Teams (ASIST) Study 2\.
  * Jehn (1995) K.A. Jehn. 1995.  A multimethod examination of the benefits and determinants of intragroup conflict.  _Administrative Science Quarterly_ , 40:256–282.
  * Jiang and Zhai (2007) Jing Jiang and ChengXiang Zhai. 2007.  Instance weighting for domain adaptation in NLP.  In _Proceedings of the Annual Meeting of the Association of Computational Linguistics_ , pages 264–271, Prague, Czech Republic. Association for Computational Linguistics.
  * Kubasova et al. (2019) Uliyana Kubasova, Gabriel Murray, and McKenzie Braley. 2019.  Analyzing verbal and nonverbal features for predicting group performance.  _arXiv preprint arXiv:1907.01369_.
  * Lai and Murray (2018) Catherine Lai and Gabriel Murray. 2018.  Predicting group satisfaction in meeting discussions.  In _Proceedings of the Workshop on Modeling Cognitive Processes from Multimodal Data_ , pages 1–8.
  * Le and Mikolov (2014) Quoc Le and Tomas Mikolov. 2014.  Distributed representations of sentences and documents.  In _Proceedings of the International Conference on Machine Learning_ , volume 32 of _Proceedings of Machine Learning Research_ , pages 1188–1196, Bejing, China. PMLR.
  * Litman et al. (2016) Diane Litman, Susannah Paletz, Zahra Rahimi, Stefani Allegretti, and Caitlin Rice. 2016.  The Teams corpus and entrainment in multi-party spoken dialogues.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_ , pages 1421–1431.
  * Maese et al. (2021) Ellyn Maese, Pablo Diego-Rosell, Les DeBusk-Lane, and Nathan Kress. 2021.  Development of emergent leadership measurement: Implications for human-machine teams.  In _AAAI Symposium on Computational Theory of Mind for Human-Machine Teams_.
  * Marlow et al. (2018) Shannon Marlow, Christina Lacerenza, Jensine Paoletti, C. Shawn Burke, and Eduardo Salas. 2018.  Does team communication represent a one-size-fits-all approach?: A meta-analysis of team communication and performance.  _Organization Behavior and Human Decision Processes_ , 144:145–170.
  * Morris et al. (2020) John X Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. 2020\.  TextAttack: A framework for adversarial attacks, data augmentation, and adversarial training in NLP.  _arXiv preprint arXiv:2005.05909_.
  * Murray and Oertel (2018) Gabriel Murray and Catharine Oertel. 2018.  Predicting group performance in task-based interaction.  In _Proceedings of the ACM International Conference on Multimodal Interaction_ , pages 14–20.
  * Park et al. (2022) Kyungjin Park, Hyunwoo Sohn, Wookhee Min, Bradford Mott, Krista Glazewski, C Hmelo-Silver, and James Lester. 2022.  Disruptive talk detection in multi-party dialogue within collaborative learning environments with a regularized user-aware network.  In _Proceedings of the Annual Meeting of the Special Interest Group on Discourse and Dialogue_.
  * Pugh et al. (2022) Samuel L Pugh, Arjun Rao, Angela EB Stewart, and Sidney K D’Mello. 2022.  Do speech-based collaboration analytics generalize across task contexts?  In _International Learning Analytics and Knowledge Conference_ , pages 208–218.
  * Rahimi and Litman (2020) Zahra Rahimi and Diane Litman. 2020.  Entrainment2vec: Embedding entrainment for multi-party dialogues.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 8681–8688.
  * Saikia et al. (2020) Tonmoy Saikia, Thomas Brox, and Cordelia Schmid. 2020.  Optimized generic feature learning for few-shot classification across domains.  _arXiv preprint arXiv:2001.07926_.
  * Triantafillou et al. (2021) Eleni Triantafillou, Hugo Larochelle, Richard Zemel, and Vincent Dumoulin. 2021\.  Learning a universal template for few-shot dataset generalization.  In _International Conference on Machine Learning_ , pages 10424–10433. PMLR.
  * Wang et al. (2020) Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni. 2020.  Generalizing from a few examples: A survey on few-shot learning.  _ACM Comput. Surv._ , 53(3).
  * Wei and Zou (2019) Jason Wei and Kai Zou. 2019.  EDA: Easy data augmentation techniques for boosting performance on text classification tasks.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6383–6389, Hong Kong, China. Association for Computational Linguistics.

Figure 5: Utterances are classified using the dialogue act classifier to
produce a sequence of DAs and the sentiment classifier to produce a time
series of sentiment polarities. Along with the text data, these sequences are
used to create MFeEmb using the Dynamic Memory model of Doc2Vec. The few shot
learning and data augmentation options are not shown in the figure.

##  Appendix A Dialogue Act Classification

We use a dialogue act classifier (USE-DAC) to map dialogues to the sequence of
DAs, where each DA in a sequence corresponds to the utterance of the dialogue.
Utterances are tagged according to the SwDA-DAMSL
tagset111https://web.stanford.edu/~jurafsky/ws97/manual.august1.html which
contains 42 tags, and one sequence is generated per dialogue. Our dialogue act
classifier uses the Universal Sentence Encoder (USE) module available at
TensorFlow Hub222 https://tfhub.dev/google/universal-sentence-encoder-large/2.
After extensive experiments, we identified that USE with three dense layers
performs best on transfer tasks. We selected the USE Transformer-based
Architecture model with three dense layers and a softmax activation function.
We fine-tune USE on the SwDA dataset and use the classifier to tag the
utterances of the test and training datasets. We selected the USE transformer-
based model because it is itself trained on dialogue and discussion forum
datasets. The test accuracy of the classification model is 72%.

##  Appendix B Doc2Vec

Doc2Vec Le and Mikolov (2014) is an unsupervised method to learn paragraph
vectors from text of arbitrary size. We represent each dialogue as 1) sequence
of utterances, 2) sequence of DAs, and 3) sentiment polarities. We pass these
sequences through Doc2Vec to generate representations. We use the Doc2Vec
implementation from the python Gensim library with an epoch size of 5,
negative sampling 5, window size 5, and alpha 0.065.

##  Appendix C SVM & Logistic Regression

We use the classifier implementations from the scikit-learn library. The SVM
was trained using the RBF kernel function and the default parameters. The
parameters for logistic regression were: Cs=10, class_weight=None, cv=10,
dual=False, fit_intercept=True, intercept_scaling=1.0, max_iter=1000,
multi_class=’ovr’, n_jobs=None, penalty=’l2’, random_state=5434, refit=True,
scoring=’accuracy’, solver=’lbfgs’, tol=0.001, verbose=False. Table 6 shows
the full results of both the SVM and Logistic Regression classifiers.

##  Appendix D BERT Baseline

We use the bert_en_uncased_L12_H768_A12 model available at TensorFlow
Hub333https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/4 to
develop our baseline classifier. The model contains one dense layer, one
dropout layer, sigmoid activation function, Adam optimizer. Due to the small
size of the Game1 dataset we train the model on the synonym replaced Game1
dataset. The total number of parameters in the model is: 10,948,301.

##  Appendix E FsText

The original FsText works by using the pre-trained word2Vec embedding model
word2vec-google-news-300 available through the gensim.downloader module. For
MFeEmb we generate the embedding using Doc2Vec with the same parameters
mentioned in Appendix B. The second phase uses a cosine similarity-based
classification model that does not involve machine learning.

##  Appendix F Concatenation Ensemble

Our Bidirectional LSTM models for each feature has an embedding layer, an LSTM
layer, one dropout layer, and one deep layer. The LSTM uses a sigmoid
activation function and is trained using the Adam optimizer with a learning
rate=0.01. The output shape of each individual model is (None, 100). The total
number of parameters of the concatenation ensemble is: 2,365,081.

##  Appendix G GitHub Dataset Conflict Scoring

For GitHub, process conflict was scored according to issue resolution using
the following heuristics to determine if conflicts occurred:

  1. 1.

Unsuccessful resolution of the issue.

  2. 2.

Unanswered questions in the discussion.

  3. 3.

Lack of understanding about the issue from one or more members.

  4. 4.

Lack of understanding or disagreement between the team members.

  5. 5.

Disagreement between the members about the proposed solution.

##  Appendix H Dialogue Act Frequency Distribution Analysis

Figure 6 shows the frequency distribution for dialogue acts in the low
conflict and high conflict classes of the Teams dataset. We divided the
dialogue acts into good communication indicators, bad communication indicators
and questions. Figure 7 shows the complete dialogue act frequency
distribution. Table 4 shows the n-gram frequency distribution of dialogue acts
across all datasets. Figure 8 visualizes the separation between low conflict
and high conflict classes using both the MFeEmb and word embedding. To see if
profanities were a reliable indication of conflict, we also examined profanity
vocabulary differences. The most frequent words in the high conflict dialogues
that are in profanity list are: [’hell’,
’kill’,’suck’,’sucking’,’shit’,’strip’,’stroke’,
’rectum’,’xxx’,’dick’,’screwed’,’retard’, ’ovary’,’piss’,’lube’, ’junkie’].

The most frequent words in the low conflict dialogues that are in the
profanity list are: [’booty’, ’pot’,’carpet’, ’rum’, ’breasts’, ’pedophile’,
’urine’, ’thug’, ’screw’, ’jerk’, ’weed’, ’screwing’, ’shower’, ’stupid’].

Figure 6: Dialogue act frequency distribution in high and low conflict classes for the Teams dataset. Dialogue acts were divided into good and bad communication indicators. Figure 7: Complete dialogue act frequency distribution for high and low conflict classes in the Teams dataset. Dataset |  Unigrams |  Bigrams |  Trigrams |  4grams |  5grams
---|---|---|---|---|---
Teams |  (sd),(b),(%) |  (sd,sd),(sd,b),(b,sd) |  (sd,sd,sd),
(sd,sd,b),
(sd,b,sd) |  (sd,sd,sd, sd),
(sd,sd,sd,sd),
(sd,sd,sd,b) |  (sd,sd,sd,sd, sd),
(sd,sd,sd,sd,b),
(sd, sd,sd,b,sd)
GitHub |  (sd),(sv),(ad) |  (sd,sd),(sd,sv),(sv,sd) |  (sd,sd,sd),
(sv,sd,sd),
(sd,sd,ad) |  (sd,sd,sd,sd),
(sd,sd,sd,ad),
(sv,sd,sd,sd) |  (sd,sd,sd,sd,sd),
(sd,sd,sd,sd, ad),
(sd,sv,sd,sd,sd)
ASIST |  (sd),(qy),(sv) |  (sd,sd),(sd,qy),(qy,sd) |  (sd,sd,sd),
(sd,qy,sd),
(qy,sd, sd) |  (sd,sd,sd,sd),
(sd,qy,sd,sd),
(sd,sd,qy,sd) |  (sd,sd,sd,sd,sd),
(sd,sd,sd,sd, qy),
(sd,qy,sd,sd,sd),
Table 4: N-gram frequency distribution: top three most frequent unigrams,
bigrams, trigrams, 4grams, 5grams of all the datasets. Sequences of sd
(statement-nonopinion) are common across all datasets.

##  Appendix I Synthetic Datasets

Synthetic datasets were generated using:
https://github.com/jasonwei20/eda_nlp.

##  Appendix J Results on Adversarially Generated Dataset

This section presents results on the adversarially generated dataset
(Synthetic Game 2) created using
TextAttack444https://github.com/QData/TextAttack. Word Swap by BERT-Masked LM
transformation was employed to generate synthetic examples from the Teams
Game2 dataset. One synthetic example is generated per dialogue of the Game2
dataset. The length of the synthetic Game2 dataset vocabulary is 6084, and the
length of the original Game1 dataset vocabulary is 3441. The number of words
in the synthetic dataset that are not in the original Game1 is 3904.

Figure 2 shows a high overlap between original Game1 and original Game2
compared to synthetic Game2 and original Game1, but this does not affect the
performance of MFeEmb (Basic), and MFeEmb. (Basic) gave a better performance
on the synthetic dataset. On the other hand, the performance of the BERT
baseline decreased on the synthetic Game2 test set, with a high standard
deviation in mean F1 scores.

Game2 Synthetic Dataset Results
---
Train model |  Teams Game1
F1_score (std) |  SynReplace
F1_score (std) |  GitReplace
F1_score (std) |  ASISTReplace
F1_score (std)
MFeEmb (Basic) |  0.654 (0.033)+ |  0.443 (0.046)* |  0.617 (0.035)+ |  0.624 (0.055)+
BERT |  - |  0.490 (0.061) |  0.422 (0.037) |  0.495 (0.044)
Table 5: MFeEmb results on the Game2 synthetic dataset generated using
TextAttack

##  Appendix K SVM & Logistic Regression Results

Table 6, 7, 8 provides the detailed results of both the SVM and logistic
regression classifiers under different experimental settings.

Both SVM & Logistic Regression Results
---
Method |  Teams Game2
F1_score (std) |  GitHub
F1_score (std) |  ASIST
F1_score (std)
Baseline Doc2Vec_dbow |  0.465 (0.070)*
0.369 (0.0)+ |  0.489 (0.080)*
0.425 (0.0)+ |  0.425 (0.091)*
0.348 (0.0)+
MFeEmb_Team1_dbow |  0.533 (0.068)*
0.369 (0.0)+ |  0.437 (0.025)*
0.425 (0.0)+ |  0.347 (0.002)*
0.348 (0.0)+
MFeEmb_Team1_dm |  0.625 (0.0295)+
0.569 (0.045)* |  0.495 (0.012)+
0.428 (0.0)* |  0.473 (0.023)+
0.393 (0.032)*
MFeEmb_SynReplace |  0.558(0.035)+
0.542 (0.045)* |  0.296(0.025)*
0.248 (0.0)+ |  0.318 (0.00)*
0.318 (0.00)+
MFeEmb_GitReplace |  0.676 (0.033)+
0.593 (0.056)* |  0.409 (0.039)*
0.248 (0.0)+ |  0.411 (0.041)*
0.318 (0.00)+
MFeEmb_ASISTReplace |  0.675 (0.041)+
0.643 (0.044)* |  0.537 (0.060)*
0.248 (0.0)+ |  0.480 (0.042)*
0.318 (0.00)+
Table 6: Results for both the SVM and logistic regression classifiers side by side. Word_Emb | MFeEmb
---|---
Teams Game2
similarity |  F1_score |  similarity |  F1_score
-0.067 |  0.470*
0.369+ |  -0.016 |  0.628+
0.561*
GitHub
similarity |  F1_score |  similarity |  F1_score
-0.067 |  0.463*
0.425+ |  -0.017 |  0.501+
0.439*
ASIST
similarity |  F1_score |  similarity |  F1_score
-0.067 |  0.446+
0.348* |  -0.016 |  0.458+
0.394*
Table 7: Similarity-based generalizability analysis. ’*’ denotes the logistic
regression results, and ’+’ denotes the SVM results. High Conflict Class
Prediction Summary
---
Method | GitHub | ASIST
BERT_SynReplace |  0.431 |  0.347
DA_only_Team1 |  0.320*
0.250+ |  0.311*
0.216+
Senti_only_Team1 |  0.207*
0.090+ |  0.300*
0.036+
MFeEmb_FsText_Team1 |  0.564 |  0.478
Table 8: Summary of high conflict class F1_scores.’*’ denotes the logistic
regression results, and ’+’ denotes the SVM results.  Figure 8: Comparison of
the MFeEmb and word embedding distribution on the 2D plane. Multi-feature
embedding showed better clustering, with most instances of one of the classes
occupying the lower left and the other occupying the upper right. On the other
hand, word embeddings are very intermixed. s: low conflict (successful
dialogue), u: high conflict (unsuccessful dialogue).