# A technical note on bilinear layers for interpretability

Lee Sharkey
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Conjecture

###### Abstract

The ability of neural networks to represent more features than neurons makes
interpreting them challenging. This phenomenon, known as superposition [Olah
et al., 2020, Elhage et al., 2022b], has spurred efforts to find architectures
that are more interpretable than standard multilayer perceptrons (MLPs) with
elementwise activation functions. In this note, I examine bilinear layers
[Shazeer, 2020], which are a type of MLP layer that are mathematically much
easier to analyze while simultaneously performing better than standard MLPs.
Although they are nonlinear functions of their input, I demonstrate that
bilinear layers can be expressed using only linear operations and third order
tensors. We can integrate this expression for bilinear layers into a
mathematical framework for transformer circuits [Elhage et al., 2021], which
was previously limited to attention-only transformers. These results suggest
that bilinear layers are easier to analyze mathematically than current
architectures and thus may lend themselves to deeper safety insights by
allowing us to talk more formally about circuits in neural networks.
Additionally, bilinear layers may offer an alternative path for mechanistic
interpretability through understanding the mechanisms of feature construction
instead of enumerating a (potentially exponentially) large number of features
in large models.

##  1 Introduction

Neural networks can learn to compute interesting and complicated functions. To
a first approximation, these functions appear to be structured such that
particular computational roles or representations are assigned to particular
directions in neural activation space [Olah et al., 2020]. We call these
representations features. Somewhat surprisingly, neural networks are believed
to be able to represent more features than they have neurons [Elhage et al.,
2022b, Gurnee et al., 2023]. This phenomenon is known as superposition, since
they assign features to non-orthogonal directions which ‘overlap’ in high-
dimensional space.

We are particularly interested in mechanistically understanding large language
models that use the transformer architecture [Vaswani et al., 2017]. This
architecture mostly consists of a series of alternating attention layers
(which let activations at different points in a sequence interact with each
other) and MLP layers (which, at each point in the sequence, construct useful
output features that are nonlinear transformations of the input features).
About two thirds of the parameters in these models are in the MLP layers,
which are thought to make prodigious use of superposition [Elhage et al.,
2022a, Gurnee et al., 2023].

Nonlinear elementwise activation functions (such as ReLU [Nair and Hinton,
2010] or GeLU [Hendrycks and Gimpel, 2020]) in MLP layers can remove small
amounts of interference between non-orthogonal features [Elhage et al.,
2022b], thus making it possible for layers to represent features in
superposition without increasing the loss. Unfortunately, while the activation
function is very useful for the performance of neural networks, it makes it
quite difficult to analyze MLPs mathematically because the powerful tools of
linear algebra can no longer be readily applied.

However, it turns out that another kind of MLP layer, the bilinear layer
[Shazeer, 2020, Dauphin et al., 2016, Mnih and Hinton, 2007], is much easier
to analyze than MLPs with elementwise activation functions. Even though
bilinear layers are nonlinear functions of the input vector, bilinear layers
can be described using only linear operations and third order tensors! This
nice property lets us extend ‘A Mathematical Framework for Transformer
Circuits’ [Elhage et al., 2021] to transformers with MLP layers as well as
attention, not just attention-only transformers. We hope that this simple
change will give us a firmer analytical footing to understand large models on
a deep, mechanistic level. This might eventually let us make deeper claims
about their safety, since it could permit us to describe classes of circuits
as mathematical objects with certain properties (as induction heads were in
Elhage et al. [2021]) and to analyze learning dynamics and predict the
emergence of particular kinds of circuits.

It has been hypothesized (though not yet observed) that neural networks might
represent a number of features that is exponential in the number of neurons in
a layer [Elhage et al., 2022b]. If this is true, it would not bode well for
our ability to mechanistically understand large neural networks, which in a
sense relies on our being able to enumerate all their features. However, as
discussed in the last section of this work, bilinear layers may offer a
potential alternative path to ‘enumerative safety’ [Elhage et al., 2022b].
Instead of attempting to understand each of a large number of features, with
bilinear networks we may be able to understand a smaller number of primitive
features that bilinear layers use to ‘construct’ their (potentially
exponential) larger number of features. Thus, in the same way that we might be
able to understand an exponentially large number of executed programs by
understanding their code, we might be able to understand an exponentially
large number of features by understanding the process by which features with
certain properties are constructed. Here, we make some preliminary steps
toward understanding the mechanisms of feature construction in bilinear
layers; we show that in bilinear layers, output features are constructed
through sums of pairwise interactions between input features, whereas, in
standard MLPs, output features are constructed using all-to-all interactions
between input features that appear not to be decomposable.

##  2 Bilinear layers

###  2.1 Introducing bilinear layers

A standard MLP layer consist of an input vector $x$, a weight matrix $W$ and
an elementwise nonlinear activation function, $\sigma$ such as the ReLU
function (and an optional bias term which is omitted for notational
simplicity). The input vector is linearly transformed by the weight matrix to
yield the pre-activation vector $Wx$, to which the activation function is
applied elementwise:

$MLP_{ReLU}(x)=\sigma(Wx)$

Bilinear layers are slightly different. They take the form

$MLP_{Bilinear}(x)=(W_{1}x)\odot(W_{2}x),$

where $\odot$ denotes elementwise multiplication. They have two weight
matrices, which each separately transform the input vector. They were
introduced in different forms by Dauphin et al. [2016] and Mnih and Hinton
[2007]. They were later studied by Shazeer [2020], who showed that bilinear
layers, when used as the MLP layer in transformer language models, are
surprisingly competitive111At least for the model size they explored, which
was approximately 120M parameters, a similar size to GPT2-small [Radford et
al., 2019]. To my knowledge, it remains to be determined whether bilinear
layers continue to perform competitively at larger scales.: They are at least
as performant per parameter than standard MLPs with ReLU or GELU activation
functions and only slightly less performant than state-of-the-art SwiGLU
layers222A SwiGLU layer is equivalent to a bilinear layer but where an
elementwise Swish activation function [Ramachandran et al., 2017] is applied
to $W_{1}x$..

###  2.2 Describing bilinear layers using only linear operations and third
order tensors

The lack of an elementwise activation function in bilinear layers makes them
mathematically very simple. In fact, despite being nonlinear functions of $x$,
they can be expressed using only linear operations and third order tensors.

First, we’ll define the tensor inner product (See appendix A for some examples
of tensor inner products which may help build intuitions). Unlike the inner
product between vectors, the tensor inner product needs to define the axes
along which the inner product is taken. The tensor inner product is thus
defined as

$\boldsymbol{U}^{(n)}\boldsymbol{\cdot}_{jk}\boldsymbol{V}^{(m)}=\boldsymbol{T}^{(n+m-2)}$

where

$\boldsymbol{T}_{\gamma_{1}\cdots\gamma_{j-1}\gamma_{j+1}\cdots\gamma_{n}\gamma^{\prime}_{1}\cdots\gamma^{\prime}_{k-1}\gamma^{\prime}_{k+1}\cdots\gamma^{\prime}_{m}}=\sum_{\beta}\boldsymbol{U}_{\gamma_{1}\cdots\gamma_{j-1}\beta\gamma_{j+1}\cdots\gamma_{n}}\boldsymbol{V}_{\gamma^{\prime}_{1}\cdots\gamma^{\prime}_{k-1}\beta\gamma^{\prime}_{k+1}\cdots\gamma^{\prime}_{m}}$
(1)

For the tensor inner product between $n^{\text{th}}$ order tensor
$\boldsymbol{U}$ and $m^{\text{th}}$ order $\boldsymbol{V}$ to be defined, the
dimension of axis $j$ of tensor $\boldsymbol{U}$ must be the same dimension as
axis $k$ of tensor $\boldsymbol{V}$.

Now we show how bilinear layers can be expressed using linear operations and
third order tensors. Suppose we want to find the third order tensor $B$ such
that

$(W_{1}x)\odot(W_{2}x)=x\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}x,$

if it exists. We’ll first identify the terms in the vector on the right hand
side,

$\begin{split}((W_{1}x)\odot(W_{2}x))_{i}&=(\sum_{j}W_{1(ij)}x_{j})(\sum_{k}W_{2(ik)}x_{k})\\\
&=\sum_{j}\sum_{k}W_{1(ij)}x_{j}W_{2(ik)}x_{k}\end{split}$ (2)

Now let’s express the terms of the third order tensor $B$ using tensor inner
products. We have,

$\begin{split}(x\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}x)_{i}=\sum_{j}x_{j}\sum_{k}x_{k}B_{ijk}\\\
=\sum_{k}x_{k}\sum_{j}x_{j}B_{ijk}\\\
=\sum_{j}\sum_{k}x_{j}x_{k}B_{ijk}.\end{split}$ (3)

Note that it doesn’t matter whether we take the tensor inner product between
$B$ and $x$ on the $2$nd or $3$rd axis first, which is why
$x\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}x$ is associative, i.e.
$(x\boldsymbol{\cdot}_{12}B)\boldsymbol{\cdot}_{21}x=x\boldsymbol{\cdot}_{12}(B\boldsymbol{\cdot}_{21}x)$.
We’ll use this property when extending a Mathematical Framework for
Transformer Circuits [Elhage et al., 2021] (Section 2.3).

Comparing the terms from equations 2 and 3, we can see they are equal if
$B_{ijk}=W_{1(ij)}W_{2(ik)}.$ Thus, we can construct the tensor $B$ using the
bilinear layer weights $W_{1},W_{2}\in\mathbb{R}^{m\times n}$ and a third
order tensor $Z$ such that $Z_{ijk}=1$ where $i=j=k$ and $0$ otherwise,
because $B=W_{1}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{2}$. One
helpful way to think about the $m\times n\times n$ tensor $B$ is that the
column vector $B_{:jk}$ consists of the elementwise multiplication of the
$j^{\text{th}}$ column of $W_{1}$ with the $k^{\text{th}}$ column of $W_{2}$.

###  2.3 Extending a Mathematical Framework for Transformer Circuits

When Elhage et al. [2021] analyzed the equations for 1- and 2-layer attention-
only transformers, it offered interesting insights on the structure of these
models. It helped to reveal QK- and OV-circuits, induction heads, and virtual
attention heads, which formed the basis of much interesting follow-up work in
interpretability [Olsson et al., 2022, Wang et al., 2022].

However, one of the biggest shortcomings of Elhage et al. [2021] was that the
transformers they analyzed had no MLP layers. MLPs comprise around two thirds
of all parameters in standard transformer language models and are thought to
be be necessary for a great deal of interesting behaviour [Geva et al., 2021].
The reason MLPs were excluded was that they could not be linearised, which
made their analysis intractable. But, as we’ve seen, it is possible to
describe bilinear layers using only linear operations. This means we can write
linearized expressions for transformers with both attention and MLP layers!
It’s important to stress that the MLPs we achieve this with are close to state
of the art [Shazeer, 2020]. This opens up the possibility that we may be able
to formally analyze some very capable language models. In this section, we’ll
identify the expression for a one-layer transformer with attention and
(bilinear) MLPs. The expressions for two- and N-layer transformers are left as
lengthy exercises for the reader.

We’ll update our notation in order to be consistent with Elhage et al. [2021],
with which we expect readers to be familiar. The inputs to the language model
is a sequence of tokens $t$ of length $n_{\text{context}}$. These are embedded
by the ${d_{\text{model}}\times n_{vocab}}$ embedding matrix $W_{E}$. The
token embeddings $x_{0}=W_{E}t$ (which have shape ${n_{\text{context}}\times
d_{\text{model}}}$ ) become the residual stream, which is passed through
multiple residual blocks, each consisting of a multihead attention layer and
an MLP layer, and each added back into the residual stream. Finally, the
residual stream is unembedded by the unembedding matrix $W_{U}$ to make the
token logits.

In Elhage et al. [2021], they assumed MLPs that had an elementwise GeLU
activation function, which are very difficult to analyze. Here, we’ll instead
use bilinear layers. Define the bilinear MLP layer as

$\begin{split}F(x)&=W_{O}^{m}(x\boldsymbol{\cdot}_{12}W_{I_{1}}^{m}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}\boldsymbol{\cdot}_{21}x)\end{split}$
(4)

where $W_{O}^{m}$ is the $d_{\text{model}}\times d_{\text{mlp}}$ output weight
matrix for the MLP layer and $W_{I_{1}}^{m},W_{I_{2}}^{m}$ are the two
$d_{\text{mlp}}\times d_{\text{model}}$ input weight matrices for the bilinear
layer.

Using the path expansion trick described by Elhage et al. [2021], the input to
the MLP in a one layer transformer can be described as

$\begin{split}x_{1}&=(Id+\sum_{h\in H}A^{h}\otimes W_{OV}^{h})\cdot W_{E}t\\\
&=(W_{E}+\sum_{h\in H}A^{h}\otimes W_{OV}^{h}W_{E})t\end{split}$ (5)

where $W_{OV}^{h}=W_{O}^{h}W_{V}^{h}$ and $A^{h}=\text{softmax*}(t^{T}\cdot
W_{E}^{T}W_{QK}W_{E}\cdot t)$ in which softmax* is the softmax function with
autoregressive masking and $W_{QK}=W_{Q}^{h\top}W^{h}_{K}$. Putting our
definition of $x_{1}$ into our definition of $F(\cdot)$ we get

$F(x_{1})=W_{O}^{m}(((W_{E}+\sum_{h\in H}A^{h}\otimes
W_{OV}^{h}W_{E})t)\boldsymbol{\cdot}_{12}W_{I_{1}}^{m}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}\boldsymbol{\cdot}_{21}\\\
((W_{E}+\sum_{h\in H}A^{h}\otimes W_{OV}^{h}W_{E})t))$ (6)

Note that for arbitrary matrices $M$, $M^{\prime}$, it’s true that
$M\boldsymbol{\cdot}_{12}M^{\prime}=M^{\top}M^{\prime\top}$. So we transpose
the left hand bracket and $W_{I_{1}}^{m}$ and move the weight matrix into the
brackets:

$=W_{O}^{m}((t^{\top}(W_{E}^{\top}W_{I_{1}}^{m\top}+\sum_{h\in H}A^{h}\otimes
W_{E}^{\top}W_{OV}^{h\top}W_{I_{1}}^{m\top}))\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}\boldsymbol{\cdot}_{21}\\\
((W_{E}+\sum_{h\in H}A^{h}\otimes W_{OV}^{h}W_{E})t))$ (7)

And next, noting that $M\boldsymbol{\cdot}_{21}M^{\prime}=MM^{\prime}$, we
move $W_{I_{2}}^{m}$ into the right hand brackets:

$=W_{O}^{m}((t^{\top}(W_{E}^{\top}W_{I_{1}}^{m\top}+\sum_{h\in H}A^{h}\otimes
W_{E}^{\top}W_{OV}^{h\top}W_{I_{1}}^{m\top}))\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}\\\
((W_{I_{2}}^{m}W_{E}+\sum_{h\in H}A^{h}\otimes
W_{I_{2}}^{m}W_{OV}^{h}W_{E})t))$ (8)

Next, we move the $Z$ tensor into the left hand brackets

$=W_{O}^{m}((t^{\top}(W_{E}^{\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z+\sum_{h\in
H}A^{h}\otimes
W_{E}^{\top}W_{OV}^{h\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z))\boldsymbol{\cdot}_{21}\\\
((W_{I_{2}}^{m}W_{E}+\sum_{h\in H}A^{h}\otimes
W_{I_{2}}^{m}W_{OV}^{h}W_{E})t))$ (9)

And combining both the left hand and right hand brackets, we get the
expression for a bilinear feedforward layer

$\begin{split}=&W_{O}^{m}(t^{\top}(\\\
&{\color[rgb]{1,0,0}W_{E}^{\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{E}}+\\\
&{\color[rgb]{0,0,1}\sum_{h\in
H}A^{h}\otimes(W_{E}^{\top}W_{OV}^{h\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{E})}+\\\
&{\color[rgb]{1,0,1}\sum_{h\in
H}A^{h}\otimes(W_{E}^{\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{OV}^{h\top}W_{E})}+\\\
&{\color[rgb]{.75,0,.25}\sum_{h\in H}\sum_{h^{\prime}\in
H}A^{h}A^{h^{\prime}}\otimes(W_{E}^{\top}W_{OV}^{h\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{OV}^{h^{\prime}\top}W_{E})}\\\
&)t)\\\ \end{split}$ (10)

We can analyze each of the terms in this equation. The first summand expresses
a direct path from the token embedding matrix straight to the MLP without
passing through any attention heads. The second summand expresses the
components of the token embeddings that pass through the attention head and
then pass into only the first MLP input matrix. The third summand is similar,
but the embeddings pass through the attention heads and into the second MLP
input matrix. The last summand corresponds to token embeddings that pass
through the attention heads and then into both the first and second MLP input
matrices.

With this expression for the MLP layer, we can now express the path expansion
for the full one layer transformer, which is simply the above expression for
$F(x)$ added to the token embedding-unembedding pathway (the ‘direct pathway’)
and the pathways through the attention heads:

$\begin{split}T(t)=&{\color[rgb]{1,.5,0}(Id\otimes W_{U}W_{E})t}+\\\
&{\color[rgb]{.5,0,.5}\sum_{h\in H}(A^{h}\otimes W_{U}W_{OV}^{h}W_{E})t}+\\\
&W_{O}^{m}(t^{\top}(\\\
&W_{E}^{\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{E}+\\\
&\sum_{h\in
H}A^{h}\otimes(W_{E}^{\top}W_{OV}^{h\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{E})+\\\
&\sum_{h\in
H}A^{h}\otimes(W_{E}^{\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{OV}^{h\top}W_{E})+\\\
&\sum_{h\in H}\sum_{h^{\prime}\in H}A^{h}A^{h^{\prime}}\otimes\\\
&\qquad(W_{E}^{\top}W_{OV}^{h\top}W_{I_{1}}^{m\top}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}W_{I_{2}}^{m}W_{OV}^{h^{\prime}\top}W_{E})\\\
&)t)\\\ \end{split}$ (11)

##  3 Understanding feature construction in bilinear layers

One of the problems we may face when trying to mechanistically understand
neural networks is that they may be able to represent an exponential number of
features. If this hypothesis resolves true, then enumerating all the features
in large networks may become computationally intractable. One analogy that
gives us hope is discussed by Olah [2022]: Even though the input space to a
particular computer program might be exponentially large, we can still say
that we understand that exponentially large space of executed programs if we
understand its code. In the same way, if we can understand the process by
which features with certain properties are constructed from simpler
primitives, we may be able to overcome the issue of having to understand an
exponential number of features. In this section, which is more speculative
than earlier sections, I outline why this hopeful vision seems very hard to
realise in standard MLPs, but seems quite possible in bilinear layers.

###  3.1 Feature construction in standard MLPs is non-decomposable

Suppose we have a standard MLP layer $MLP_{ReLU}(x)=\sigma(Wx)$ with a ReLU
activation $\sigma$ (where the bias term is omitted). Also suppose that the
input vector $x\in X$ consists of sparse linear combinations of input features
$x=D^{I\top}a^{I}$, where $D^{I}$ is a dictionary of input features
represented as a $n_{\text{features}}\times d_{\text{input}}$ matrix and
$a^{I}\in A^{I}$ is a sparse vector of coefficients (with values in
$[0,\infty)$ of size $n_{\text{features}}$) such that the dataset $X$ can be
reconstructed from the features and their coefficients, $X=D^{I\top}A^{I}$.
Similarly suppose there is a dictionary of output features for this layer
$D^{O}$ and that sparse linear combinations of those output features describe
the activations observed in a large representative sample from
$p_{x}(MLP_{ReLU}(x))$, i.e.

$MLP_{ReLU}(x)=\sigma(Wx)=\sigma(W(D^{I\top}a^{I}))=D^{O\top}a^{O}$ (12)

Therefore $D^{I}$ and $D^{O}$ are overcomplete bases333In linear algebra, a
basis of a vector space is a set of vectors from which every vector in that
space can be expressed as a linear combination. An overcomplete basis is a
basis where at least one element of the basis set can be removed yet the set
remains a basis. for the input space $X$ and output space $MLP_{ReLU}(X)$
respectively.

One way to view the process of feature construction is to say that output
features $D^{O}$ are all implicitly represented in superposition in the weight
matrix $W$ and that the nonlinearity, when applied elementwise to the
preactivation vector $Wx$, modifies a set of default output features in order
to select particular output features. One candidate for the default output
features are the left singular vectors of $W$, i.e. the columns of a matrix
$U$ (We’ll discuss other candidates in the next section). We can thus
introduce a modifier vector $m(x)$ that is a function of $x$ such that

$MLP_{ReLU}(x)=m(x)\odot Wx=(m(x)\odot U)\Sigma V^{\top}x=D^{O\top}a^{O}.$

Therefore we can view linear combinations of the output features (namely
$D^{O\top}a^{O}$) as consisting of linear combinations of modified default
output features (namely $(m(x)\odot U)\Sigma V^{\top}x$).

With a ReLU activation function, $m(x)$ is binary vector of ones and zero:
$m(x)_{i}=1$ where $\sigma(Wx)_{i}>0$ and $m(x)_{i}=0$ otherwise. In general,
for vanilla MLPs with any elementwise activation function $\sigma$:

$m(x)_{i}=\frac{\sigma(Wx)_{i}}{(Wx)_{i}}\quad$ (13)

It is the modifier vector that ‘selects’ from the features represented in
superposition in $W$, or, equivalently, ‘contructs’ them by modifying the
default output features. If we could understand how $m(x)$ is computed in
terms of input features $D^{I}$, then we could begin to understand why
particular output features $D^{O}$ are constructed not others. Unfortunately,
in vanilla MLPs, the only way to calculate the value of $m(x)$ in general is
Equation 13. In other words, to get the value of the modifier vector, we first
have to pass the input through the network to observe what the post-
activations (the numerator) and pre-activations are (the denominator) to get
$m(x)$. But this is circular: We would have to already understand the
nonlinear computation in the numerator in order to understand how output
features are constructed. This framing doesn’t simplify anything at all!
Feature construction in standard MLPs can thus be considered ‘non-
decomposable’.

###  3.2 Feature construction in bilinear layers

In mechanistic interpretability, one of the major assumptions that we need to
make is that we can interpret linear transformations of almost arbitrary
dimensionality. They may still be large objects, but linear transformations
are as simple as transformations get. For large linear transformations with
non-sparse coefficients, we may have to spend more time studying them or
prioritize analysis of the largest coefficients. But overall we assume that we
can understand them to a satisfying extent. If we can’t, then the whole
business of mechanistic intepretability would be doomed even for large linear
regressions, never mind deep neural networks.

Granting this assumption, if we could describe the modifier vector $m(x)$ in
the previous section as a linear function of input features (instead of a
nonlinear one), then we could begin to understand how a layer constructs
output features. Fortunately, in bilinear layers the modifier vector is a
linear function of the input!

$MLP_{Bilinear}(x)=m(x)\odot(W_{2}x)\qquad\text{where}\qquad m(x)=W_{1}x,$

We’ll say that the modifier vector modifies the default output features
represented in $W_{2}$ to construct output features.

We still need to define what the default output feature directions and the
modifier feature directions are concretely. Ultimately this choice will always
be somewhat arbitrary because linear transformations do not imply any
particular privileged basis. As before, perhaps the most obvious candidates
for the default output feature directions are the left singular vectors of
$W_{2}$. But the largest directions in the output activations may not
necessarily have a strong relationship with the weights because the output
directions depend on both the weights and the input directions. Therefore, we
may be able to do better than the left singular vectors of $W_{2}$ by
incorporating the data distribution into the choice of bases. One way might
use the right singular vectors $MLP_{Bilinear}(X)$ or $W_{2}X$. Another –
perhaps better – way is to identify default output features that are maximally
statistically independent. This may be better because statistically
independent directions tend to be activated somewhat sparsely and therefore
might be better default output features than singular vectors, since fewer
will be significantly ‘activated’ at any one time. We could achieve this by
performing linear ICA Hyvärinen and Oja [2000] on the preactivations $W_{2}X$.
This would yield a matrix $U^{(2)}$, which is the set of vectors that are
maximally statistically independent directions of the output dataset while
still being a basis of it. We can then use multiple linear regression to find
the corresponding matrix $V^{(2)\top}$ such that $W_{2}=U^{(2)}V^{(2)\top}$.
Slightly abusing terminology, we’ll call $U^{(2)}$ and $V^{(2)\top}$ the left
and right independent components of $W_{2}$ respectively. We can define the
modifier features using the same procedure, identifying the left and right
independent components of $W_{1}=U^{(1)}V^{(1)\top}$.

Armed with such features, we may be able to describe feature construction in
bilinear networks in terms of interactions between two relatively small,
relatively sparse sets of vectors (the default output features and the
modifier features). We hope we can use this approach to tell mechanistic
stories for how features with certain properties are constructed by the
network. We might be able to do this by understanding the functional
properties of the default output features and how modifier features tend to
modify them. Optimistically, mechanistic stories like these may let us
understand an exponentially large space of features. Whether or not such an
approach will work is ultimately an empirical question, which we leave for
future work. In the next section, we explore the mathematical simplicity of
feature construction in bilinear layers, which gives us some reason to suspect
that feature construction may be understandable. 555 We can make further
modifications to the modifier features and default output features that assist
either the intuitiveness or interpretability of bilinear networks. I’ll note
them here but won’t explore them further in this work. Improving
intuitiveness: If, during training, we constrain $W_{1}x$ to be low $L_{2}$
norm and add the one vector as bias, the modifier vector would always be close
to the one vector. In other words: $m(x)=W_{1}x+\textbf{1}$ where
$||W_{1}x||\approx 0$. This would mean that modifier features simply cause
slight modifications of default output features. This addition would also help
us make a analysis prioritization decisions later (see section 3.4), but
fundamentally the modification isn’t necessary. This addition also opens up an
experimental avenue (which we won’t explore here): By imposing more or less
regularization on the norm, it allows us to control the amount of
superposition a network is able to do. This would be an interesting
experimental lever to pull, since it would allow us to directly test how much
a network’s performance is due to superposition. Improving interpretability:
We could choose an $L_{1}$ penalty for the norm constraint on the modifier
vector (instead of the $L_{2}$ norm); or we could constrain $W_{1}$ to be low
rank; alternatively, we could quantize the output of $W_{1}x$ in order to put
hard limits on the amount of superposition a network can do.

###  3.3 Feature construction in bilinear layers decompose into a sum of
pairwise interactions between input features

Not all layers have the same ‘amount’ of nonlinearity. Some are more nonlinear
than others. Here we characterize the amount of nonlinearity layers can have,
which sheds light on how bilinear layers differ from standard MLPs.

Let $C(d_{i}^{I},d_{j}^{O},a^{I})$ quantify the contribution of input feature
$d_{i}^{I}\in D^{I}$ to the activation (or ‘selection’) of output feature
$d_{j}^{O}\in D^{O}$. We then have the following (non-comprehensive) set of
degrees of nonlinearity.

  * •

Linear: Fully linear layers have no nonlinearity. There are therefore no
interactions between input features during feature construction (since there
is no modifier vector). The amount that input feature $d_{i}^{I}$ contributes
to the selection of output feature $d_{j}^{O}$ is quantified simply as
$C(d_{i}^{I},d_{j}^{O},a^{I})=[Wd_{i}^{I}a^{I}_{i}]^{\top}d_{j}^{O}$, which is
just the inner product between the preactivation caused by that input feature
and the output feature.

  * •

Additively pairwise nonlinear: In this case, output features are determined by
a sum of pairwise interactions between features. For example, if input
features $d_{1}^{I},d_{2}^{I},d_{3}^{I}$ are active in the input, the
contribution of $d_{i}^{I}$ (where $i\in\\{1,2,3\\}$) to each output feature
can be described as a sum of pairwise nonlinear interactions,
$C(d_{i}^{I},d_{j}^{O},a^{I})=[f(d_{i}^{I};d_{1}^{I},a_{1}^{I})+f(d_{i}^{I};d_{2}^{I},a_{2}^{I})+f(d_{i}^{I};d_{3}^{I},a_{3}^{I})]^{\top}d_{j}^{O}$,
where $f(\cdot)$ is some nonlinear function of the two interacting features.

  * •

Fully nonlinear: The contribution an input feature makes to the selection of
an output feature depends on every other feature in a way that can’t be
decomposed into a sum. The contribution of $d_{i}^{I}$ to each output feature
can only be described as an all-to-all nonlinear interaction between input
features that cannot be broken down into linear components:
$C(d_{i}^{I},d_{j}^{O},a^{I})=g(d_{i}^{I};d_{1}^{I},d_{2}^{I},d_{3}^{I},a^{I})^{\top}d_{j}^{O}$,
where $g(\cdot)$ is some (non-additively-pairwise) nonlinear function.

The task of understanding additively pairwise nonlinearity is easier than full
nonlinearity because we can study each pairwise interaction between features
and sum them up. Understanding full nonlinearity is significantly harder
because there is no way to linearly decompose the function $g$. Sadly,
standard MLPs are fully nonlinear. However, we show that bilinear layers are
additively pairwise nonlinear, making them significantly easier to analyze.

Suppose the input to a bilinear layer $x^{\prime}$ consists of a linear
combination of two input features $d_{1}^{I}$ and $d_{2}^{I}$ , i.e.
$x^{\prime}=a_{1}d_{1}^{I}+a_{2}d_{2}^{I}$. Using the re-expression of the
bilinear layer, inputting $x^{\prime}$ into equation 2 yields

$(a_{1}d_{1}+a_{2}d_{2})\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}(a_{1}d_{1}+a_{2}d_{2})=\\\
a_{1}d_{1}\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}a_{1}d_{1}+\\\
a_{1}d_{1}\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}a_{2}d_{2}+\\\
a_{2}d_{2}\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}a_{1}d_{1}+\\\
a_{2}d_{2}\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}a_{2}d_{2}\quad\\\ $
(14)

More generally, for arbitrary linear combinations of input features:

$(W_{1}x)\odot(W_{2}x)=(\sum_{i\in
R}a_{i}d_{i})\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}(\sum_{i\in
R}a_{i}d_{i})=\sum_{i\in R}\sum_{j\in
R}a_{i}a_{j}d_{i}\boldsymbol{\cdot}_{12}B\boldsymbol{\cdot}_{21}d_{j}$ (15)

where $R$ is the set of indices of nonzero feature coefficients. Equation 15
shows that, although all features interact to determine the output features,
these interactions can be understood as a sum of pairwise interactions between
features. Hence bilinear layers are only additively pairwise nonlinear.

We hope that this simplicity can be leveraged to tell simple stories about how
particular input features (hopefully sparsely) activate particular default
output features and modifier features. Then, if we understand the functional
properties of those default output features and the kinds of functional
modifications that those modifier features make, then we may be able to
understand the properties of the output features.

###  3.4 How should we study feature construction?

At this early stage, it’s not totally clear how best to analyze the structure
of bilinear networks. What is clear is that doing so will be easier than
analyzing fully nonlinear computations, since we’re simply studying the
structure of tensors, which is a relatively well understood domain in
mathematics. In advance of empirical results, I speculate on a few non-
mutually exclusive ways to proceed in this section.

  1. 1.

Large coefficients of $B$: As discussed at the beginning of section 2, when
interpreting any linear transformation, there may be so many coefficients that
it may be necessary to prioritize our analyses by studying only the largest
coefficients. One way to leverage this is simply to study the largest
coefficients of $B$ and how they would influence interactions between commonly
observed pairs or groups of input features.

  2. 2.

Tensor decomposition: Building on (1), we could perform Higher Order Singular
Value Decomposition (HOSVD) and study the structure of the most influential
ranks of the tensor.

  3. 3.

Maximally modified default output features: Recall that one way to view the
bilinear network is that one side of the elementwise multiplication modifies
the linear transformation on the other side. This suggests an way to
prioritize the analysis of how particular features are constructed: For each
input feature, we should prioritize analysis of the most modified default
output features. Concretely, define

$U^{(2,d_{i})}:=d_{i}^{\top}W_{1}\boldsymbol{\cdot}_{12}Z\boldsymbol{\cdot}_{21}U^{(2)}.$

This is the set of output features caused by the modifications that input
feature $d_{i}$ makes to default output feature $U^{(2)}$. Then, for each
input feature $d_{i}$ we should study the top k most modified default output
features, i.e.

$\underset{l}{\text{arg top-k}}(||U^{(2)}_{:,l}-U^{(2,d_{i})}_{:,l}||)$ (16)

This would let us focus on the most significant modifications that a given
input feature makes to the default output features. But we can prioritize our
analyses further than that. The modifications that an input feature makes to
the default output features don’t matter unless the default output feature is
actually activated by that feature or some other feature that is
simultaneously present in $x$. Therefore we can identify pairs of features,
$(d_{l},d_{m})$ that are correlated (or that have often appeared at the same
time) and where $U^{(2,d_{l})}$ is both one of the default output features
that is most modified by $d_{m}$ and simultaneously one of the default output
features that is most activated by $d_{m}$.

##  4 Conclusion

The simplicity of bilinear layers makes formal analysis much easier than for
standard MLPs. One of the most important things bilinear layers give us are
analysable expressions for performant transformers with both attention heads
and MLP layers. I hope that this will eventually let us formally analyze the
structure of the representations of large language models in this class. This
might reveal interesting features and circuits in a similar way that the
mathematical framework for attention-only transformers introduced by Elhage et
al. [2021] helped to reveal reveal QK- and OV-circuits, induction heads, and
virtual attention heads. Curiosity aside, an expression for models with
bilinear layers may let us make stronger claims about safety. For instance, it
may let us more directly compare circuit structure in different models, and
enable us to make inferences about model behaviour without necessarily running
the model.

Another potential research direction is analyzing learning dynamics. Models
with bilinear layers seem like they might lend themselves to mathematical
analysis in a similar fashion to the deep linear layers studied by Saxe et al.
[2013]. Learning dynamics may be important for safety, since understanding
them may be necessary to be able to predict dangerous model behaviors before
they emerge.

Lastly, and most speculatively, bilinear layers offer the potential to
understand the mechanisms of feature construction, which may be necessary for
understanding a potentially exponentially large number of features represented
in language models. There is still much empirical work to do to evaluate
whether intuiting the mechanisms of feature construction is possible. Overall,
I hope that this note might pique the interest of the interpretability
community by highlighting an architecture that is much gentler on the
intuitions than standard MLPs.

## Acknowledgements

I thank Trenton Bricken for helpful discussions that initiated my search for
layers that could be described in terms of higher order tensors. I thank Beren
Millidge, Sid Black, and Dan Braun for helpful discussions and detailed
feedback on this work.

## References

  * Dauphin et al. [2016] Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier.  Language modeling with gated convolutional networks.  _CoRR_ , abs/1612.08083, 2016.  URL http://arxiv.org/abs/1612.08083.
  * Elhage et al. [2021] Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah.  A mathematical framework for transformer circuits.  _Transformer Circuits Thread_ , 2021.  https://transformer-circuits.pub/2021/framework/index.html.
  * Elhage et al. [2022a] Nelson Elhage, Tristan Hume, Catherine Olsson, Neel Nanda, Tom Henighan, Scott Johnston, Sheer ElShowk, Nicholas Joseph, Nova DasSarma, Ben Mann, Danny Hernandez, Amanda Askell, Kamal Ndousse, , Jones, Dawn Drain, Anna Chen, Yuntao Bai, Deep Ganguli, Liane Lovitt, Zac Hatfield-Dodds, Jackson Kernion, Tom Conerly, Shauna Kravec, Stanislav Fort, Saurav Kadavath, Josh Jacobson, Eli Tran-Johnson, Jared Kaplan, Jack Clark, Tom Brown, Sam McCandlish, Dario Amodei, and Christopher Olah.  Softmax linear units.  _Transformer Circuits Thread_ , 2022a.  https://transformer-circuits.pub/2022/solu/index.html.
  * Elhage et al. [2022b] Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah.  Toy models of superposition.  _Transformer Circuits Thread_ , 2022b.  https://transformer-circuits.pub/2022/toy_model/index.html.
  * Geva et al. [2021] Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy.  Transformer feed-forward layers are key-value memories, 2021.
  * Gurnee et al. [2023] Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas.  Finding neurons in a haystack: Case studies with sparse probing, 2023\.
  * Hendrycks and Gimpel [2020] Dan Hendrycks and Kevin Gimpel.  Gaussian error linear units (gelus), 2020.
  * Hyvärinen and Oja [2000] A. Hyvärinen and E. Oja.  Independent component analysis: algorithms and applications.  _Neural Networks_ , 13(4):411–430, 2000.  ISSN 0893-6080.  doi: https://doi.org/10.1016/S0893-6080(00)00026-5.  URL https://www.sciencedirect.com/science/article/pii/S0893608000000265.
  * Mnih and Hinton [2007] Andriy Mnih and Geoffrey Hinton.  Three new graphical models for statistical language modelling.  In _Proceedings of the 24th International Conference on Machine Learning_ , ICML ’07, page 641–648, New York, NY, USA, 2007. Association for Computing Machinery.  ISBN 9781595937933.  doi: 10.1145/1273496.1273577.  URL https://doi.org/10.1145/1273496.1273577.
  * Nair and Hinton [2010] Vinod Nair and Geoffrey E. Hinton.  Rectified linear units improve restricted boltzmann machines.  In _Proceedings of the 27th International Conference on International Conference on Machine Learning_ , ICML’10, page 807–814, Madison, WI, USA, 2010. Omnipress.  ISBN 9781605589077.
  * Olah [2022] Chris Olah.  Mechanistic interpretability, variables, and the importance of interpretable bases, 2022.  URL https://transformer-circuits.pub/2022/mech-interp-essay/index.html.
  * Olah et al. [2020] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter.  Zoom in: An introduction to circuits.  _Distill_ , 2020.  doi: 10.23915/distill.00024.001.  https://distill.pub/2020/circuits/zoom-in.
  * Olsson et al. [2022] Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah.  In-context learning and induction heads, 2022.
  * Radford et al. [2019] Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever.  Language models are unsupervised multitask learners.  2019\.
  * Ramachandran et al. [2017] Prajit Ramachandran, Barret Zoph, and Quoc V. Le.  Searching for activation functions, 2017.
  * Saxe et al. [2013] Andrew M. Saxe, James L. McClelland, and Surya Ganguli.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks, 2013.
  * Shazeer [2020] Noam Shazeer.  Glu variants improve transformer, 2020.  URL https://arxiv.org/abs/2002.05202.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _CoRR_ , abs/1706.03762, 2017.  URL http://arxiv.org/abs/1706.03762.
  * Wang et al. [2022] Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt.  Interpretability in the wild: a circuit for indirect object identification in gpt-2 small, 2022.

##  Appendix A Tensor inner product examples

The definition of tensor inner product we use is

$\boldsymbol{U}^{(n)}\boldsymbol{\cdot}_{jk}\boldsymbol{V}^{(m)}=\boldsymbol{T}^{(n+m-2)}$

where

$\boldsymbol{T}_{\gamma_{1}\cdots\gamma_{j-1}\gamma_{j+1}\cdots\gamma_{n}\gamma^{\prime}_{1}\cdots\gamma^{\prime}_{k-1}\gamma^{\prime}_{k+1}\cdots\gamma^{\prime}_{m}}=\sum_{\beta}\boldsymbol{U}_{\gamma_{1}\cdots\gamma_{j-1}\beta\gamma_{j+1}\cdots\gamma_{n}}\boldsymbol{V}_{\gamma^{\prime}_{1}\cdots\gamma^{\prime}_{k-1}\beta\gamma^{\prime}_{k+1}\cdots\gamma^{\prime}_{m}}$

Example 1:
$\mathbb{U}^{(1)}\mathbb{\cdot}_{11}\mathbb{V}^{(1)}=\mathbb{T}^{(0)}=\sum_{\beta}\mathbb{u}_{\beta}\mathbb{v}_{\beta}=\mathbb{u}^{\top}\mathbb{v}$,
which is just the standard inner product, resulting in a scalar.

Example 2:
$\mathbb{U}^{(2)}\mathbb{\cdot}_{21}\mathbb{V}^{(1)}=\mathbb{T}^{(1)}$ where
$\mathbb{T}_{i}=\sum_{\beta}\mathbb{U}_{i\beta}\mathbb{v}_{\beta}$. This is
multiplication of a matrix on the right and a vector on the left:
$\mathbb{T}=\mathbb{U}\mathbb{v}$.

Example 3:
$\mathbb{U}^{(2)}\mathbb{\cdot}_{11}\mathbb{V}^{(1)}=\mathbb{T}^{(1)}$ where
$\mathbb{T}_{i}=\sum_{\beta}\mathbb{U}_{\beta i}\mathbb{v}_{\beta}$. This is
equivalent to multiplication of a transposed matrix on the left and a vector
on the right: $\mathbb{T}=\mathbb{U}^{\top}\mathbb{v}$.

Example 4:
$\mathbb{U}^{(1)}\mathbb{\cdot}_{12}\mathbb{V}^{(2)}=\mathbb{T}^{(1)}$ where
$\mathbb{T}_{i}=\sum_{\beta}\mathbb{u}_{\beta}\mathbb{V}_{i\beta}$. This
equivalent to multiplication of transposed vector on the left and a matrix on
the right: $\mathbb{T}=\mathbb{u}^{\top}\mathbb{V}^{\top}$. Note that
$\mathbb{T}$ is a rank one tensor, so
$\mathbb{T}=\mathbb{u}^{\top}\mathbb{V}^{\top}=\mathbb{V}\mathbb{u}$ since
tensor notation disposes of the convention that vectors are column vectors or
row vectors; instead they are just rank-one tensors. We somewhat abuse
notation in this work by assuming standard vector-matrix conventions for
multiplication unless the tensors we’re dealing with are rank-three or above,
in which case we use tensor inner product notation.

Example 5:
$\mathbb{U}^{(3)}\mathbb{\cdot}_{11}\mathbb{V}^{(1)}=\mathbb{T}^{(2)}$ which
is the matrix that is a sum of matrices consisting of slices of the rank-three
tensor $\mathbb{T}=\sum_{\beta}\mathbb{U}_{\beta::}\mathbb{v}_{\beta}$. If we
imagine the rank-three tensor as a cube, this example flattens the tensor
along its height by taking the inner product between $\mathbb{v}$ and every
3-d column of $\mathbb{U}$.

Example 6:
$\mathbb{U}^{(2)}\mathbb{\cdot}_{23}\mathbb{V}^{(3)}=\mathbb{T}^{(3)}$ which
is the rank-three tensor where
$\mathbb{T}_{i::}=\sum_{\beta}\mathbb{U}_{:\beta}\mathbb{V}_{i:\beta}$. If we
imagine tensor $\mathbb{V}$ as a cube, here take each front-to-back-row and
get its inner product with the corresponding row $i$ of matrix $\mathbb{U}$.


Igor Mikolasek, Saeedeh Ghanadbashi, Nima Afraz, Fatemeh Golpayegani
11institutetext: Transport Research Centre CDV, Brno, Czech Republic,
11email: igor.mikolasek@cdv.cz,22institutetext: School of Computer Science,
University College Dublin, Ireland

# Data Sharing at the Edge of the Network: A Disturbance Resilient Multi-modal
ITS

Igor Mikolasek 11 Saeedeh Ghanadbashi 22 Nima Afraz 22 Fatemeh Golpayegani 22

###### Abstract

Mobility-as-a-Service (MaaS) is a paradigm that encourages the shift from
private cars to more sustainable alternative mobility services. MaaS provides
services that enhances and enables multiple modes of transport to operate
seamlessly and bringing Multimodal Intelligent Transport Systems (M-ITS)
closer to reality. This requires sharing and integration of data collected
from multiple sources including modes of transports, sensors, and end-users’
devices to allow a seamless and integrated services especially during
unprecedented disturbances. This paper discusses the interactions among
transportation modes, networks, potential disturbance scenarios, and
adaptation strategies to mitigate their impact on MaaS. We particularly
discuss the need to share data between the modes of transport and relevant
entities that are at the vicinity of each other, taking advantage of edge
computing technology to avoid any latency due to communication to the cloud
and privacy concerns. However, when sharing at the edge, bandwidth, storage,
and computational limitations must be considered.

###### keywords:

Shared mobility, Mobility-as-a-Service, Decentralized Systems, Data sharing,
Edge Computing

##  1 Introduction

The concept of Mobility-as-a-Service (MaaS) is gaining interest due to
increased awareness of sustainable mobility and the adoption of Intelligent
Transportation Systems (ITS) [1, 2]. MaaS aims to shift people from cars to
more sustainable modes of transportation, relying on ITS for efficient
infrastructure use, real-time data provision, and user-guided routing.
Presently, public transport (PT) systems commonly use ITS for real-time data
on arrival times and vehicle locations. Similarly, bike/scooter-sharing
services provide device location data, and traffic signals can optimize
intersection throughput based on traffic conditions. These ITS systems often
lack cooperation among different transportation modes, limiting their
effectiveness particularly at the times of disturbances. Multimodal ITS
(M-ITS) [5] can enhance MaaS, where different modes can support or substitute
one another [4, 7, 3]. This paper provides an overview of transportation
modes, networks, and their interactions, focusing on disturbances and how
M-ITS can mitigate them. M-ITS concept can incorporate edge computing paradigm
[6] to place the processing of data closer to its source. This will address
privacy concerns, reduce data processing delays, enable decentralize
computation, and eliminate costly centralized systems. Using edge computing
paradigm, decisions are made in a decentralized manner, relying on data from
the local transport network, onboard vehicle sensors, and information from
nearby vehicles and infrastructure. Identifying data-sharing needs, and the
data flow among systems and vehicles during disturbances is essential. This
paper outlines such possible disturbances and their impact on different modes
and suggests coordinated impact-mitigation strategies for the M-ITS-based MaaS
system.

##  2 Transportation Modes and Networks

To facilitate data sharing between multiple modes of transport, we first need
to understand their dependencies and how they interact with one another. This
paper focuses on transport in urban areas, although most modes can operate
beyond. Expanding M-ITS and MaaS to intercity or interstate levels would
involve considering additional modes, especially air transport that are out of
scope for this paper. To do so, first, we model the dependencies between
transportation modes and their infrastructure networks in urban areas.111 The
commonly available transportation modes provided is not exhaustive – each
category can represent multiple modes of similar nature. Transport modes can
operate on various networks, some shared and others exclusive. Networks often
share infrastructure, for example roads are shared between cars, buses, and
cyclists. They can be modeled collectively, with parameters defining mode
usage. Multimodal modes indicate possible mode transitions. Figure 1 presents
an overview of the modes noted as $M_{i}$ and the networks they use, noted as
$N_{i}$.

Figure 1: Overview of fundamental transportation modes and networks employed.
The indices $M_{i}$ and $N_{i}$ are employed for mode and network respectively
and specifying the pairings between the two in Fig. 2.

##  3 Service Disturbances to Mode-Network Combinations and M-ITS Adaptation
Strategies

Unplanned events can disrupt normal traffic operations, impacting system
performance and requiring adaptations to limit such impacts. Data sharing and
collective intelligence can inform about and address the challenges related to
such disturbances . Five main causes of disturbances are identified, impacting
various mode-network combinations (see Fig. 2). Here, only cases of possible
direct impacts (need to re-route) are considered. Secondary impacts due to
possible relocation of transport demand are considered later for individual
disturbance scenarios. In Table 1, we present a comprehensive overview of
individual disturbances, providing further insights into potential data
sources for detection, strategies for adaptation, and special cases that might
necessitate specific responses.

Figure 2: Effects of different sources of disturbance on different
combinations of modes and networks.

Below we provide some examples of disturbances and scenarios to handle them.
For example, “Planned Work Zone” along the pedestrian network (N1-M1) might
result in disturbance for users with wheelchairs. Such work zones are not
commonly recorded in a machine-readable format. This is especially true if
they are unplanned, hence this case is omitted in the schema in Fig 2.
Pedestrians and cyclists have the advantage of being very agile and can bypass
most obstacles on the network with acceptable delay and they are not connected
to any ITS service, except shared bikes. Therefore, except for planned work
zones, they are not considered primary stakeholders. These modes are
considered here mostly as transition modes and as an adaptation strategy in
case of disturbance.

Table 1: List of detection options, adaptation strategies, and specific cases for the basic identified disturbances. D |  Detection Options |  Adaptation Strategies |  Specific Cases
---|---|---|---
D1 - Road accident D4 - Another Road blockage |  • User feedback apps (Waze) • C-ITS data from vehicles (V2I) • Traffic information centers (API, C-ITS DENM, RDS TMC, …) • Integrated rescue system dispatching • Video detection (AI image processing) • Traffic flow (TF) data from sensors |  • Warn road users nearby and with routes via the disturbance location • Info + guidance at affected stops to MaaS alternatives • Busses temporarily adjust routes if taxi/CAV can serve bypassed stops, if favorable • Intelligent traffic signs, VMS and other devices adjust to changes in traffic – warnings, guidance, change signal plan |  • Other networks affected • Reserved road lanes are affected • Reaction varies based on road blockage • Treat D4 as D2 when extended
D2 – Planned Work zone |  • Digitized registry of planned work zones with temporary traffic management measures • Intelligent “first/last cone” or contractor updates* |  • Temporarily adjust networks for all affected modes and react aptly |  • Adjust duration through regular detection means if not updated dynamically
D3 – Unplanned Work zone |  • Integrated rescue system** • User feedback apps (Waze) • C-ITS data from vehicles • Intelligent “cone”* • Traffic information centers • TF data from sensors |  • Mostly the same as D1 • For long-lasting work zones, treat as D2 once details are available |  • Mostly the same as D1 • Response varies depending on expected duration
D5-D7 – Broken vehicle Rail block |  • C-ITS data from vehicles (V2I) • Railway or public transport (PT) operator dispatching • Otherwise treat as D1/D4 |  • Inform passengers about alternatives within the MaaS • Inform M-ITS about the obstacle and react accordingly (mix of D1 and D3) • For D6, D7, replace operation with busses/CAVs |  • Certain PT routes (e.g. airport line) have higher priority • Follow train/ metro safety standards
D8 - Broken Traffic signals |  • C-ITS (I2I) from the device • Device operator • TF data from sensors |  • Inform police to regulate traffic at the intersection • Warn relevant road users • Optimize nearby traffic signals for changed traffic • Adjust routing for MaaS |  • Warn trams, if affected, and react suitably
D9 - Rescue event |  • C-ITS data from vehicles • Integrated rescue system dispatching |  • Warn nearby road users • Re-route vehicles or adjust signal plans for smooth rescue |  • Consider operation size and duration
|  *C-ITS-equipped “traffic cone” or mobile app to update WZ start/end time.
---
**Police should know first, communication with M-ITS may vary among countries.

Additional causes of disturbance include major sport or cultural events, and
gatherings, that significantly increase (or decrease) traffic demand within
certain areas of the networks. Therefore, a registry of such events, including
information about the event, time, location, affected area, and possibly
expected number of visitors should be included. Some disturbances, especially
the external ones mentioned above, can be observed and identified centrally.
Then only the warnings with relevant information can be shared with relevant
individual edge devices to be evaluated individually. This includes the
processing of information about detected disturbances from the edge devices
and transport users in cases where the impacts of the disturbance exceed the
communication range of the local edge devices.

Other impact mitigation suggestions are as follows: –For damaged CAVs, inform
its users about nearby replacements and send warnings if blocking the road.
–For broken shared bikes and scooters, notify users of alternatives; other
modes are not affected. –Temporarily adjust service limits for space-limited
services when intermodal nodes near the borders are out of service. –Warn
users at affected PT stops and guide them to nearby alternatives or other MaaS
services. –Empty bike-sharing nests provide info about nearby nests with
available bikes and PT alternatives. –Notify taxis and CAVs of disturbances
and areas with increased demand. –Provide (multimodal) route change
suggestions for users with planned routes. –General control measures for
improved performance: Integrate (multimodal) navigation apps into M-ITS;
Increase C-ITS systems and OBU penetration; Establish disturbance detection
information systems; Implement VMS and variable LED signage; Use intelligent
traffic lights, including cooperation, PT prioritization, and multimodal;
Create incentives (cost evaluation, tolling, gamification); Implement
multimodal ticket schemes; Consider multimodal performance in traffic control
systems; Design new data standards for interoperability across the EU
(identify best practices for data management; define minimal operational
standards for data requirements).

##  4 Data Sharing at the Network Edge

The disturbance-mode-network model defines combinations of disturbances and
affected transportation modes/networks, but not all users or vehicles of the
affected mode-network combination must be warned. Only relevant edge devices
should be informed with specific details. Given the small size of the basic
warning data package, this can be spread quite broadly with more details given
upon request, if available. Disturbance warnings sent to edge devices should
include the following information, –Type of disturbance, –Location on the
network affected modes, and class of the network segment for each mode (e.g.,
critical, major, inferior, minor), –Expected impact severity (one or more
measures as applicable): Capacity reduction; Lanes affected; Severity index;
Displaced traffic volume, –Estimated time of duration, and any other case-
specific information.

To efficiently distribute warnings for disturbances, it is crucial to consider
various parameters like the affected area, specific modes, networks, and user
classes. The goal is to notify only relevant users and devices without
overwhelming them. This involves a combination of general heuristic rules for
area-based warnings, C-ITS traces for approaching vehicles, data from edge
devices regarding their planned routes (including non-scheduled traffic),
adaptation strategies for dealing with the disturbance, and machine learning
algorithms for predicting trajectories. There are two main aspects to
consider: (i) information about the disturbance, which is part of the warning
itself (type, location, severity, duration), and (2) information about the
recipient of the warning. For recipients, it is essential to account for their
expected or planned trajectory, any expected TF changes along that trajectory,
and whether they are part of the adaptation strategy.

Edge devices in this environment encompass a wide range of components, from
sensors and processors within vehicles, through users’ mobile phones, to
infrastructure elements like traffic signal controllers and communication
nodes. Edge computing can greatly enhance the efficiency of distributing
warnings for disturbances based on the following considerations: Efficient
Data Processing: Edge computing processes data closer to the source, enabling
quick assessment of disturbances and reducing response times. For example, a
CAV detects a fallen tree on the road and immediately sends an alert to nearby
vehicles. Localized Decision-Making: Edge devices can make localized decisions
based on the available information. They can analyze their planned routes,
expected TF changes, and other relevant factors to determine whether a
particular disturbance warning is relevant to them and address it accordingly.
Adaptation Strategies: Edge devices can autonomously implement the pre-defined
adaptation strategies and cooperate with other devices to optimize overall
performance. Reduced Communication Network Congestion: By processing and
filtering data at the edge, the system can significantly reduce congestion of
the communication network. This is crucial during disturbances when there may
be an increased volume of data traffic due to widespread alerts.

##  5 Conclusions

This paper has explored the critical role of M-ITS in enhancing MaaS
efficiency, especially during disturbances. Establishing relationships between
transportation modes, networks, and disturbances, has highlighted the
importance of data sharing on the network edge. The selective distribution of
disturbance warnings to relevant users and devices is crucial for effective
disturbance mitigation. This framework offers valuable insights for building
resilient and adaptive urban transportation systems that prioritize
sustainability and user needs.

##  6 Acknowledgements

This project has received funding from the RE-ROUTE Project, the European
Union’s Horizon Europe research and innovation program under the Marie
Skłodowska-Curie grant agreement No 101086343.

## References

  * [1] Esztergár-Kiss, D., Kerényi, T., Mátrai, T., Aba, A.:  Exploring the MaaS market with systematic analysis.  European Transport Research Review 12(1) (2020) 1–16
  * [2] Golpayegani, F., Gueriau, M., Laharotte, P.A., Ghanadbashi, S., Guo, J., Geraghty, J., Wang, S.:  Intelligent shared mobility systems: A survey on whole system design requirements, challenges and future direction.  IEEE Access 10 (2022) 35302–35320
  * [3] Jevinger, Å., Persson, J.A.:  Potentials of context-aware travel support during unplanned public transport disturbances.  Sustainability 11(6) (2019) 1649
  * [4] Mnif, S., Darmoul, S., Elkosantini, S., Ben Said, L.:  An immune multiagent system to monitor and control public bus transportation systems.  Computational Intelligence 34(4) (2018) 1245–1276
  * [5] Qureshi, K.N., Abdullah, A.H.:  A survey on intelligent transportation systems.  Middle-East Journal of Scientific Research 15(5) (2013) 629–642
  * [6] Safavifar, Z., Ghanadbashi, S., Golpayegani, F.:  Adaptive workload orchestration in pure edge computing: A reinforcement-learning model.  In: 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), IEEE (2021) 856–860
  * [7] Yang, G., Wang, J., Zhang, F., Zhang, S., Gong, C.:  A real-time timetable rescheduling method for metro system energy optimization under dwell-time disturbances.  Journal of Advanced Transportation 2019 (2019) 1–11