# Towards a Universal Continuous Knowledge Base

Gang Chen1,3,4, Maosong Sun1,3,4,5, and Yang Liu1,2,3,4,5

1Department of Computer Science and Technology, Tsinghua University
2Institute for AI Industry Research, Tsinghua University
3Institute for Artificial Intelligence, Tsinghua University
4Beijing National Research Center for Information Science and Technology
5Beijing Academy of Artificial Intelligence Yang Liu is the corresponding
author<EMAIL_ADDRESS>
###### Abstract

In artificial intelligence (AI), knowledge is the information required by an
intelligent system to accomplish tasks. While traditional knowledge bases use
discrete, symbolic representations, detecting knowledge encoded in the
continuous representations learned from data has received increasing attention
recently. In this work, we propose a method for building a continuous
knowledge base (CKB) that can store knowledge imported from multiple, diverse
neural networks. The key idea of our approach is to define an interface for
each neural network and cast knowledge transferring as a function simulation
problem. Experiments on text classification show promising results: the CKB
imports knowledge from a single model and then exports the knowledge to a new
model, achieving comparable performance with the original model. More
interesting, we import the knowledge from multiple models to the knowledge
base, from which the fused knowledge is exported back to a single model,
achieving a higher accuracy than the original model. With the CKB, it is also
easy to achieve knowledge distillation and transfer learning. Our work opens
the door to building a universal continuous knowledge base to collect, store,
and organize all continuous knowledge encoded in various neural networks
trained for different AI tasks.

##  1 Introduction

A knowledge base (KB) is a centralized repository where structured and
unstructured information is stored, organized, and shared. KBs have been
widely used to accomplish artificial intelligence (AI) tasks. While their
initial use was closely connected with expert systems (Hayes-Roth et al.,,
1983), knowledge graphs such as Freebase (Bollacker et al.,, 2008) have become
a typical example of knowledge base used in AI systems recently, showing their
effectiveness in applications such as image classification (Marino et al.,,
2017), natural language processing (Zhang et al.,, 2018), and bioinformatics
(Ernst et al.,, 2015).

While the use of discrete representations enables KBs to be easily
interpretable to humans, it is notoriously challenging to build such discrete
knowledge bases. On one hand, it is time-consuming and labor-intensive to
manually construct KBs (Lenat and Guha,, 1990). The situation aggravates when
it comes to a vertical domain for which only a limited number of annotators
qualify to have the expertise. On the other hand, although machine learning
approaches are able to mine knowledge from data automatically, they often
inevitably have limitations in the correctness, depth, and breadth of
knowledge they can acquire (Richardson and Domingos,, 2003). Therefore, how to
build large-scale, high-quality, and wide-coverage knowledge bases still
remains a grand challenge to the community.

The past two decades have witnessed the rapid progress of deep learning
(Hinton and Salakhutdinov,, 2006), which has proven effective in learning
continuous representations from data, leading to substantial improvements in a
variety of AI tasks like speech recognition (Dahl et al.,, 2011), image
classification (Krizhevsky et al.,, 2012), and machine translation (Vaswani et
al.,, 2017). More recently, there has been a significant paradigm shift from
learning continuous representations from limited labeled data in a supervised
fashion to from abundant unlabeled data in a self-supervised way (Devlin et
al.,, 2019; Brown et al.,, 2020), which further advances the development of
learning continuous representations from data.

Given the remarkable success of deep learning, an interesting question
naturally arises: Is knowledge encoded in the learned continuous
representations? A number of researchers have developed probing methods to
evaluate the extent to which continuous representations encode knowledge of
interest (Linzen et al.,, 2016; Belinkov et al.,, 2017; Blevins et al.,, 2018;
Hewitt and Manning,, 2019). For example, while Belinkov et al., (2017) conduct
a quantitive evaluation that sheds lights on the ability of neural machine
translation models to capture word structure, Hewitt and Manning, (2019)
propose a structural probe that can test whether syntax trees are consistently
embedded in a linear transformation of word representation space.

Figure 1: A universal continuous knowledge base. $\mathcal{M}$ denotes the
knowledge base. $\mathrm{NN}_{\bm{\theta}_{n}}$ ($n\in[1,6]$) denotes the
$n$-th neural network parameterized by $\bm{\theta}_{n}$. On one hand,
continuous knowledge encoded in a neural network can be imported to the
knowledge base. On the other hand, continuous knowledge stored in the
knowledge based can be exported to a neural network. Note that these neural
networks are independent of each other: they can have different model
structures and be trained for different AI tasks.

If knowledge can be defined as the information required by a system to
accomplish AI tasks, we would like to distinguish between two categories of
knowledge: discrete and continuous. While discrete knowledge uses symbolic
representations explicitly handcrafted by humans, continuous knowledge is
implicitly encoded in neural networks automatically trained on data. If each
trained neural network can be seen as a repository of continuous knowledge,
there is an important question that needs to be answered: Is it possible to
build a universal continuous knowledge base that stores knowledge imported
from diverse neural networks?

In this work, we propose a method for building a universal continuous
knowledge base (CKB). As shown in Figure 1, the CKB allows for knowledge
transferring between multiple, diverse neural networks. The knowledge encoded
in one neural network can be imported to the CKB, from which the stored
knowledge can be exported to another neural network. As a neural network can
be seen as a parameterized function, we treat the function and its parameters
as the continuous knowledge encoded in the neural network (Section 2.1). Using
a memory hierarchy to represent the knowledge base (Section 2.2), our approach
defines an interface function for each neural network and casts importing and
exporting knowledge as a function simulation problem (Sections 2.3 and 2.5).
We adopt multi-task training to import knowledge from multiple neural networks
(Section 2.4). Experiments on text classification show that our method is able
to fuse knowledge imported from multiple, diverse neural networks and obtain
better performance than single neural networks. It is also easy to use the
knowledge base to simulate other learning paradigms such as knowledge
distillation and transfer learning.

##  2 Approach

To use a continuous knowledge base to store the knowledge encoded in neural
networks, we need to answer a number of fundamental questions:

  1. 1.

What is the knowledge encoded in a neural network? (Section 2.1)

  2. 2.

How to design a continuous knowledge base? (Section 2.2)

  3. 3.

How to import the knowledge encoded in a neural network into the continuous
knowledge base? (Section 2.3)

  4. 4.

How to import the knowledge encoded in multiple neural networks to the
continuous knowledge base? (Section 2.4)

  5. 5.

How to export the knowledge stored in the continuous knowledge base to a
neural network? (Section 2.5)

###  2.1 Knowledge Encoded in a Neural Network

A neural network can be seen as a parameterized, composite function. For
example, Figure 2 shows a simple feed-forward neural network involving the
composition of two non-linear functions:

$\displaystyle\mathbf{h}$ $\displaystyle=f(\mathbf{W}_{xh}\mathbf{x})$ (1)
$\displaystyle\mathbf{y}$ $\displaystyle=g(\mathbf{W}_{hy}\mathbf{h})$ (2)

where $\mathbf{x}$ is the input layer, $\mathbf{h}$ is the hidden layer,
$\mathbf{y}$ is the output layer, $\mathbf{W}_{xh}$ is the weight matrix
between the input and hidden layers, $\mathbf{W}_{hy}$ is the weight matrix
between the hidden and output layers, and $f(\cdot)$ and $g(\cdot)$ are two
non-linear functions. For simplicity, we omit bias terms. The neural network
shown in Figure 2 can also be denoted by

$\displaystyle\mathbf{y}=\mathrm{FNN}_{\bm{\theta}}(\mathbf{x})=g_{\bm{\theta}_{2}}(f_{\bm{\theta}_{1}}(\mathbf{x}))$
(3)

where $\mathrm{FNN}_{\bm{\theta}}(\cdot)$ is a non-linear function
parameterized by $\bm{\theta}=\\{\mathbf{W}_{xh},\mathbf{W}_{hy}\\}$,
$f_{\bm{\theta}_{1}}(\cdot)$ is a non-linear function parameterized by
$\bm{\theta}_{1}=\\{\mathbf{W}_{xh}\\}$, and $g_{\bm{\theta}_{2}}(\cdot)$ is a
non-linear function parameterized by $\bm{\theta}_{2}=\\{\mathbf{W}_{hy}\\}$.

Figure 2: Example of a feed-forward neural network that can be seen as a
composite function. We treat the parameterized function as the continuous
knowledge encoded in the neural network. $\mathbf{x}$ is the input,
$\mathbf{h}$ is the hidden state, and $\mathbf{y}$ is the output.
$\mathbf{W}_{xh}$ and $\mathbf{W}_{hy}$ are parameters.

We distinguish between two functions related to neural networks:

  1. 1.

Global function: a function that maps the input of a neural network to its
output.

  2. 2.

Local function: a function that participates in the composition of a global
function.

For example, $\mathrm{FNN}_{\bm{\theta}}(\cdot)$ is a global function and
$f_{\bm{\theta}_{1}}(\cdot)$ is a local function.

Given a trained neural network that is able to accomplish an AI task, we
believe that it is the parameterized function that represents the continuous
knowledge encoded in the neural network. It is important to note that both the
function and the parameters are indispensable. On one hand, if a function has
no parameters (e.g., the max pooling function), it can be directly called and
there is no need to import it to the knowledge base. On the other hand, as
parameters are defined to be bound to a function, parameters themselves are
useless if the associated function is missing.

Figure 3: (a) A feed-forward neural network and (b) its interface to the
knowledge base. The neural network is represented as a parameterized function
$\mathrm{FNN}_{\bm{\theta}}(\cdot)$ that takes $\mathbf{x}$ as input and
outputs $\mathbf{y}$. Its interface to the knowledge base $\mathcal{M}$ is
also defined as a parameterized function
$\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\cdot)$ that shares the same
dimensions of input and output with the neural network. An interface is used
to facilitate transferring knowledge between a neural network and the
knowledge base.

###  2.2 Continuous Knowledge Base

####  2.2.1 Memory Hierarchy

To build a CKB, we propose to use two levels of real-valued matrices inspired
by the use of memory hierarchy in computer architecture (Hennessy and
Patterson,, 2011). At the high level, the CKB maintains one real-valued matrix
$\mathbf{M}^{h}$. At the low level, the CKB maintains $K$ real-valued
matrices:
$\mathbf{M}^{l}_{1},\dots,\mathbf{M}^{l}_{k},\dots,\mathbf{M}^{l}_{K}$. As a
result, the CKB consists of $K+1$ real-valued matrices:

$\displaystyle\mathcal{M}=\left\\{\mathbf{M}^{h},\mathbf{M}^{l}_{1},\dots,\mathbf{M}^{l}_{K}\right\\}$
(4)

Note that these real-valued matrices are learnable parameters of the CKB.

####  2.2.2 Implementation of an Interface

To facilitate importing and exporting knowledge between a neural network and
the CKB, we introduce an interface for the neural network. As shown in Figure
3, given a neural network $\mathrm{FNN}_{\bm{\theta}}(\cdot)$, its interface
can be defined as a function:

$\displaystyle\mathbf{y}=\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\mathbf{x},\mathcal{M})$
(5)

where $\mathbf{x}$ is the input, $\mathbf{y}$ denotes the output, and
$\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\cdot)$ is an interface
function parameterized by $\bm{\phi}$ tailored for
$\mathrm{FNN}_{\bm{\theta}}(\cdot)$. Note that the input and output of the
interface have the same dimensions with those of
$\mathrm{FNN}_{\bm{\theta}}(\cdot)$. With interfaces, we can cast importing
and exporting knowledge between the neural network
$\mathrm{FNN}_{\bm{\theta}}(\cdot)$ and the knowledge base as a function
simulation problem:
$\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\mathbf{x},\mathcal{M})$ runs
the same way $\mathrm{FNN}_{\bm{\theta}}(\mathbf{x})$ does given the same
input $\mathbf{x}$ (see Sections 2.3, 2.4, and 2.5 for details).

Figure 4: Illustration of how an interface to the continuous knowledge base
works. $\mathcal{M}$ is a continuous knowledge base, which is organized as a
memory hierarchy: low-level real-valued matrices
$\mathbf{M}^{l}_{1},\dots\mathbf{M}^{l}_{K}$ and a high-level matrix
$\mathbf{M}^{h}$. The knowledge base provides an interface for each neural
network. Given an input $\mathbf{x}$, the interface first generates two extra
matrices $\tilde{\mathbf{M}}^{l}$, $\tilde{\mathbf{M}}^{h}$. Note that the
matrices generated on the fly are denoted by dashed rectangles. Then, the
interface uses the attention function to generate hidden states
$\mathbf{h}_{1},\dots,\mathbf{h}_{K}$, which are concatenated with
$\mathbf{M}^{h}$ and $\tilde{\mathbf{M}}^{h}$ to serve as the key and value
(i.e., $\mathbf{H}$) of another attention function to generate the output
$\mathbf{y}$.

Figure 4 shows an example that illustrates how an interface works. Given an
input $\mathbf{x}$, our approach first adds two extra matrices on the fly:

$\displaystyle\tilde{\mathbf{M}}^{l}$
$\displaystyle=\mathbf{x}\mathbf{W}^{l},$ (6)
$\displaystyle\tilde{\mathbf{M}}^{h}$ $\displaystyle=\mathbf{x}\mathbf{W}^{h}$
(7)

where $\mathbf{W}^{l},\mathbf{W}^{h}\in\bm{\phi}$ are two interface
parameters.

Then, for each low-level matrix $\mathbf{M}^{l}_{k}$, we use the attention
function (Vaswani et al.,, 2017) to obtain a hidden state:

$\begin{split}\tilde{\mathbf{M}}^{l}_{k}&=\big{[}\mathbf{M}^{l}_{k};\tilde{\mathbf{M}}^{l}\big{]},k=1,\dots,K\\\
\mathbf{h}_{k}&=\mathrm{Attention}(\mathbf{x}\mathbf{W}^{\text{q}},\tilde{\mathbf{M}}^{l}_{k}\mathbf{W}^{\text{k}},\tilde{\mathbf{M}}^{l}_{k}\mathbf{W}^{\text{v}})\end{split}$
(8)

where $\mathbf{W}^{\text{q}}$, $\mathbf{W}^{\text{k}}$, and
$\mathbf{W}^{\text{v}}$ are the transformation matrices for query, key, and
value used in the attention mechanism, respectively.

Finally, the output is calculated by calling the attention function again:

$\displaystyle\mathbf{y}=\mathrm{Attention}(\mathbf{x}\mathbf{W}^{\text{q}},\mathbf{H}\mathbf{W}^{\text{k}},\mathbf{H}\mathbf{W}^{\text{v}})$
(9)

where the hidden state matrix $\mathbf{H}$ is the concatenation of the high-
level matrix, the hidden states, and the extra matrix
$\tilde{\mathbf{M}}^{h}$:

$\displaystyle\mathbf{H}=\big{[}\mathbf{M}^{h};\mathbf{h}_{1};\dots;\mathbf{h}_{K};\tilde{\mathbf{M}}^{h}\big{]}$
(10)

While every neural network has its own interface to the knowledge base and the
interface parameters are often different, the continuous knowledge base
$\mathcal{M}$ is shared among all neural networks.

####  2.2.3 Interfaces for Global and Local Functions

As the implementation of an interface is transparent to the model structure,
it is easy to define an interface for an arbitrary neural network since one
only needs to specify the parameterized function and the dimensions of its
input and output.

Often, it is convenient to directly define an interface for the global
function if there are only a small number of parameters. For example, the
global function for the feed-forward neural network shown in Figure 2 is
$\mathrm{FNN}_{\bm{\theta}}(\cdot)$, we can use Eq. (5) to define its
interface.

However, it is more efficient to define an interface for a local function if
the neural network calls it frequently. Consider a recurrent neural network
defined as a global function:

$\displaystyle\mathbf{y}=\mathrm{RNN}_{\bm{\theta}}(\mathbf{x}_{1:T})$ (11)

where $\mathbf{x}_{1:T}=\mathbf{x}_{1},\dots,\mathbf{x}_{T}$ is the input
sequence and $\mathbf{y}$ is the output.

The global function $\mathrm{RNN}_{\bm{\theta}}(\cdot)$ runs by calling the
local function $f_{\bm{\theta}}(\cdot)$ repeatedly:

$\displaystyle\mathbf{h}_{t}=f_{\bm{\theta}}(\mathbf{x}_{t},\mathbf{h}_{t-1}),\
t=1,\dots,T$ (12)

where $\mathbf{x}_{t}$ and $\mathbf{h}_{t}$ are the input and the hidden state
at time step $t$, respectively. Note that we let $\mathbf{y}=\mathbf{h}_{T}$
for simplicity.

As a result, instead of defining an interface for the global function
$\mathrm{RNN}_{\bm{\theta}}(\cdot)$, it is more suitable to define an
interface for the local function $f_{\bm{\theta}}(\cdot)$:

$\displaystyle\mathbf{h}_{t}=\mathrm{Interface}^{f}_{\bm{\phi}}(\mathbf{x}_{t},\mathbf{h}_{t-1},\mathcal{M})$
(13)

Figure 5: Importing and exporting knowledge for single neural networks. We
cast importing knowledge from a neural network to the knowledge base as a
function simulation problem: the knowledge is successfully imported only if
the interface $\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}$ runs the same
way the neural network $\mathrm{FNN}_{\bm{\theta}}$ does. This is done by
finding knowledge base and interface parameters (i.e., $\hat{\mathcal{M}}$ and
$\hat{\bm{\phi}}$) that minimize the difference between the outputs of two
functions (i.e., $\Delta(\mathbf{y}_{1},\mathbf{y}_{2})$). Note that the
parameters of the neural networks $\bm{\theta}$ are fixed during importing.
Similarly, exporting knowledge is also treated as a function simulation
problem: finding model parameters $\hat{\bm{\theta}}$ to enable the neural
network to imitate the interface while keeping $\mathcal{M}$ and $\bm{\phi}$
fixed.

###  2.3 Importing Knowledge from Single Neural Networks

As shown in Figure 5, we cast importing knowledge from a neural network to the
knowledge base as a function simulation (Jiao et al.,, 2020) problem: the
knowledge base stores the knowledge encoded in a neural network only if the
corresponding interface runs in the same way the neural network does. For
example, to import the knowledge from the feed-forward neural network shown in
Figure 2 to the knowledge base, we require that the following equation holds
for an arbitrary input:

$\displaystyle\forall\mathbf{x}\in\mathcal{X}:\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\mathbf{x},\mathcal{M})=\mathrm{FNN}_{\bm{\theta}}(\mathbf{x})$
(14)

where $\mathcal{X}$ is a set of all possible inputs.

As a result, the importing process is equivalent to an optimization problem.
For example, given a set of inputs
$\mathcal{D}=\\{\mathbf{x}^{(n)}\\}_{n=1}^{N}$, importing
$\mathrm{FNN}_{\bm{\theta}}(\cdot)$ to the knowledge base $\mathcal{M}$ is
done by

$\displaystyle\hat{\mathcal{M}},\hat{\bm{\phi}}=\mathop{\rm
argmin}_{\mathcal{M},\bm{\phi}}\Big{\\{}L_{\mathrm{import}}(\mathcal{D},\mathcal{M},\bm{\phi})\Big{\\}}$
(15)

where the loss function is defined as

$\displaystyle
L_{\mathrm{import}}(\mathcal{D},\mathcal{M},\bm{\phi})=\sum_{n=1}^{N}\Delta\Big{(}\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\mathbf{x}^{(n)},\mathcal{M}),\mathrm{FNN}_{\bm{\theta}}(\mathbf{x}^{(n)})\Big{)}$
(16)

We use $\Delta(\cdot)$ (e.g., a cosine function) to measure the difference
between the outputs of two functions.

###  2.4 Importing Knowledge from Multiple Neural Networks

To import the knowledge encoded in multiple neural networks to the knowledge
base, a natural way is to minimize the importing loss functions of these
neural networks jointly. For example, let
$\mathcal{D}_{1}=\\{\mathbf{x}^{(m)}\\}_{m=1}^{M}$ be a set of inputs for a
feed-forward neural network and
$\mathcal{D}_{2}=\\{\mathbf{x}^{(n)}\\}_{n=1}^{N}$ be a set of inputs for a
convolutional neural network. Note that the two datasets are independent: the
input to the feed-forward neural network can be an image and the input to the
convolutional neural network can be a natural language sentence.

The importing loss function of the feed-forward neural network can be defined
as

$\displaystyle
L_{\mathrm{import}}^{\mathrm{FNN}}(\mathcal{D}_{1},\mathcal{M},\bm{\phi}_{1})=\sum_{m=1}^{M}\Delta\Big{(}\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}_{1}}(\mathbf{x}^{(m)},\mathcal{M}),\mathrm{FNN}_{\bm{\theta}_{1}}(\mathbf{x}^{(m)})\Big{)}$
(17)

Similarly, the importing loss function of the convolutional neural network can
be defined as

$\displaystyle
L_{\mathrm{import}}^{\mathrm{CNN}}(\mathcal{D}_{2},\mathcal{M},\bm{\phi}_{2})=\sum\limits_{n=1}^{N}\Delta\Big{(}\mathrm{Interface}^{\mathrm{CNN}}_{\bm{\phi}_{2}}(\mathbf{x}^{(n)},\mathcal{M}),\mathrm{CNN}_{\bm{\theta}_{2}}(\mathbf{x}^{(n)})\Big{)}$
(18)

Then, importing $\mathrm{FNN}_{\bm{\theta}_{1}}(\cdot)$ and
$\mathrm{CNN}_{\bm{\theta}_{2}}(\cdot)$ to the knowledge base $\mathcal{M}$
synchronously is given by

$\displaystyle\hat{\mathcal{M}},\hat{\bm{\phi}}_{1},\hat{\bm{\phi}}_{2}=\mathop{\rm
argmin}_{\mathcal{M},\bm{\phi}_{1},\bm{\phi}_{2}}\Big{\\{}L_{\mathrm{import}}^{\mathrm{FNN}}(\mathcal{D}_{1},\mathcal{M},\bm{\phi}_{1})+L_{\mathrm{import}}^{\mathrm{CNN}}(\mathcal{D}_{2},\mathcal{M},\bm{\phi}_{2})\Big{\\}}$
(19)

It is easy to extend the above approach to more than two neural networks.

###  2.5 Exporting Knowledge to a Neural Network

We can also use the interface to export the knowledge stored in the CKB to a
neural network. Still, we treat exporting knowledge from the knowledge base to
a neural network as a function simulation problem: the knowledge is exported
to the neural network only if the the neural network runs in the same way the
interface does. For example, to export the knowledge from the CKB to the feed-
forward neural network shown in Figure 2, we require that the following
equation holds for an arbitrary input:

$\displaystyle\forall\mathbf{x}\in\mathcal{X}:\mathrm{FNN}_{\bm{\theta}}(\mathbf{x})=\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\mathbf{x},\mathcal{M})$
(20)

As a result, the importing processing is also equivalent to an optimization
problem: our goal is to modify the parameters of the neural network to
minimize the difference between the neural network and its interface. For
example, given a set of inputs $\mathcal{D}=\\{\mathbf{x}^{(n)}\\}_{n=1}^{N}$,
exporting the knowledge stored in the knowledge base $\mathcal{M}$ to
$\mathrm{FNN}_{\bm{\theta}}(\cdot)$ is done by

$\displaystyle\hat{\bm{\theta}}=\mathop{\rm
argmin}_{\bm{\theta}}\Big{\\{}L_{\mathrm{export}}(\mathcal{D},\bm{\theta})\Big{\\}}$
(21)

where the loss function is defined as

$\displaystyle
L_{\mathrm{export}}(\mathcal{D},\bm{\theta})=\sum_{n=1}^{N}\Delta\left(\mathrm{Interface}^{\mathrm{FNN}}_{\bm{\phi}}(\mathbf{x}^{(n)},\mathcal{M}),\mathrm{FNN}_{\bm{\theta}}(\mathbf{x}^{(n)})\right)$
(22)

As the knowledge base and interfaces are fixed during the exporting process,
exporting knowledge to multiple neural networks is equivalent to exporting it
to single neural networks in parallel.

##  3 Experiments

###  3.1 Setting

We evaluated our approach on text classification on two public datasets:

  1. 1.

Amazon positive-negative review dataset (Fu et al.,, 2018). The training set
contains 796,000 reviews. The test set contains 4,000 reviews. We split the
original test set into two parts: 2,000 reviews as the validation set and
2,000 reviews as the test set. On average, each review contains 19 words.

  2. 2.

Yelp polarity review dataset (Zhang et al.,, 2015). The training set contains
560,000 reviews. The test set contains 38,000 reviews. We split the original
training set into two parts: 550,000 reviews as the training set and 10,000 as
the validation set. On average, each review contains 163 words.

Configuration | #Param. | Accuracy
---|---|---
model ($\bm{\theta}$) | interface ($\bm{\phi}$) | CKB ($\mathcal{M}$) | Amazon | Yelp
RNN |  0.20M | - | - | 84.30 | 96.31
RNN $\mapsto$ CKB $\mapsto$ RNN | - |  1.05M | 81.92K | 84.40 | 96.32
CNN |  0.26M | - | - | 84.35 | 95.95
CNN $\mapsto$ CKB $\mapsto$ CNN | - |  0.79M | 81.92K | 84.15 | 95.67
ANN |  0.79M | - | - | 84.15 | 93.51
ANN $\mapsto$ CKB $\mapsto$ ANN | - |  1.31M | 81.92K | 84.15 | 93.61
BERT | 85.64M | - | - | 87.90 | 97.34
BERT $\mapsto$ CKB $\mapsto$ BERT | - | 19.09M |  1.70M | 87.60 | 96.69
GPT-2 | 85.64M | - | - | 87.80 | 97.62
GPT-2 $\mapsto$ CKB $\mapsto$ GPT-2 | - | 19.09M |  1.70M | 87.75 | 97.13

Table 1: Results of importing and exporting knowledge for single models. “RNN
$\mapsto$ CKB $\mapsto$ RNN” denotes first importing the knowledge encoded in
the RNN model that achieves an accuracy of 84.30 on the Amazon dataset to CKB,
from which the knowledge stored is exported to another RNN model with the same
model structure. Note that the number of parameters does not include the word
embedding layer and the classification layer.

We used the following five neural networks tailored for text classification in
our experiments:

  1. 1.

RNN (Liu et al.,, 2016): recurrent neural network. We used a single gate
recurrent unit (GRU) (Cho et al.,, 2014) layer as the encoder. Its hidden size
is set to 256. We defined the interface for RNN at the local level: the input
of the interface consists of the $t$-th word $\mathbf{x}_{t}$ and the
$(t-1)$-th hidden state $\mathbf{h}_{t-1}$ and the output is the $t$-th hidden
state $\mathbf{h}_{t}$.

  2. 2.

CNN (Kim,, 2014): convolution neural network. We followed Kim, (2014) to use
three filter windows (i.e., 3, 4, and 5) with 80 feature maps, respectively.
We defined its interface at the local level: the input of the interface is a
sequence of consecutive words $\mathbf{X}_{t:t+w}$ and the output is a hidden
state $\mathbf{h}_{t}$.

  3. 3.

ANN (Vaswani et al.,, 2017): attention-based neural network. We used a single
Transformer encoder layer as the encoder. Its hidden size and intermediate
size are 256 and 1024, respectively. We defined the interface for ANN at the
global level: the input of the interface is the entire sequence $\mathbf{X}$
and the output is the sequence of hidden states $\mathbf{H}$.

  4. 4.

BERT (Devlin et al.,, 2019): Bidirectional encoder representations from
Transformers. The BERT base model 111https://huggingface.co/bert-base-uncased
was fine-tuned on the two text classification datasets. We defined the
interface for BERT at the global level: the input of the interface is the
entire sequence $\mathbf{X}$ and the output is the final output of the BERT
encoder. Note that the BERT base model uses 12 attention layers. Our interface
directly predicts the output of the 12-th layer.

  5. 5.

GPT-2 (Radford et al.,, 2019): a language model based on masked self-
attention. We fine-tuned GPT-2 222https://huggingface.co/gpt2 for text
classification and defined its interface similar to that of BERT.

  6. 6.

ALBERT (Lan et al.,, 2020): a lite BERT. 333https://huggingface.co/albert-
base-v2. The interface for ALBERT is defined at the global level in a similar
way.

We removed documents that contain less than 5 words and only retained the
first 512 tokens for documents that have more than 512 tokens. For RNN, CNN,
and ANN, we used BPE (Sennrich et al.,, 2016) with 32K operations to
preprocess the datasets. For other methods, we used their built-in tokenizers
to preprocess the datasets.

We used 10 low-level $\text{30}\times\text{256}$ matrices and one high-level
$\text{20}\times\text{256}$ matrix to build the CKB for small models (i.e.,
RNN, CNN, and ANN). For big models like BERT and GPT-2, the CKB contains 20
low-level $\text{40}\times\text{2,048}$ metrices and one high-level
$\text{20}\times\text{2,048}$ matrix. We used AdamW (Loshchilov and Hutter,,
2019) to optimize parameters of the CKB, interfaces, and neural networks.
Please refer to Appendix A for more details.

Method | Configuration | Accuracy
---|---|---
Single | RNN | 84.30
CNN | 84.35
ANN | 84.15
Ours | RNN $\mapsto$ CKB $\mapsto$ CNN | 83.35
RNN $\mapsto$ CKB $\mapsto$ ANN | 83.10
CNN $\mapsto$ CKB $\mapsto$ RNN | 82.70
CNN $\mapsto$ CKB $\mapsto$ ANN | 83.85
ANN $\mapsto$ CKB $\mapsto$ RNN | 82.05
ANN $\mapsto$ CKB $\mapsto$ CNN | 83.10

Table 2: Results of knowledge transferring between different models on the
Amazon dataset.

Method | Configuration | Accuracy
---|---|---
Single | RNN | 84.30
CNN | 84.35
ANN | 84.15
BERT | 87.90
GPT-2 | 87.80
Ensemble | RNN & CNN | 85.25
CNN & ANN | 85.40
ANN & RNN | 85.00
RNN & CNN & ANN | 85.35
BERT & GPT-2 | 88.45
Ours |  $\\{$RNN, CNN$\\}\mapsto$ CKB $\mapsto$ RNN | 85.05
$\\{$RNN, CNN$\\}\mapsto$ CKB $\mapsto$ CNN | 84.05
$\\{$CNN, ANN$\\}\mapsto$ CKB $\mapsto$ CNN | 84.25
$\\{$CNN, ANN$\\}\mapsto$ CKB $\mapsto$ ANN | 84.30
$\\{$ANN, RNN$\\}\mapsto$ CKB $\mapsto$ ANN | 84.35
$\\{$ANN, RNN$\\}\mapsto$ CKB $\mapsto$ RNN | 85.20
$\\{$RNN, CNN, ANN$\\}\mapsto$ CKB $\mapsto$ RNN | 84.95
$\\{$RNN, CNN, ANN$\\}\mapsto$ CKB $\mapsto$ CNN | 83.95
$\\{$RNN, CNN, ANN$\\}\mapsto$ CKB $\mapsto$ ANN | 84.10
$\\{$BERT, GPT-2$\\}\mapsto$ CKB $\mapsto$ BERT | 88.20
$\\{$BERT, GPT-2$\\}\mapsto$ CKB $\mapsto$ GPT-2 | 88.35

Table 3: Results of importing and exporting knowledge for multiple models on
the Amazon dataset.

###  3.2 Importing and Exporting Knowledge for Single Neural Networks

Table 1 shows the results of importing and exporting knowledge for single
neural networks. This experiment aims to verify whether CKB is able to import
and export continuous knowledge. We find that our approach is capable of
retaining the expressive power of the original neural network across a variety
of architectures.

Table 1 also lists the numbers of model, interface, and CKB parameters. The
interface for CNN has the fewest parameters (i.e., 0.79M) because it only
takes a substring of the input sequence as input. As the interface for RNN
takes both the current token and the last hidden state as input, it has more
parameters (i.e., 1.05M) than that of CNN. The interface for ANN has more
parameters than that of RNN because it takes the entire sequence as input and
contains an additional feed-forward layer. Since the interfaces for BERT and
GPT-2 directly imitate the ouput of the final layer (i.e., the 12-th layer),
they have much fewer parameters than the original models.

Table 2 shows the results of importing and exporting knowledge for different
single models. We import the knowledge from one model to the CKB and then
export the knowledge to a different model, which is similar to the setting of
zero-shot learning. We find that the knowledge stored in RNN, CNN, and ANN can
be transferred to each other with only small performance degradation.

###  3.3 Importing and Exporting Knowledge for Multiple Neural Networks

Table 3 shows the results of importing and exporting knowledge for multiple
models on the Amazon dataset. We find that first importing multiple models to
the CKB and then exporting the fused knowledge to a single model can result in
improved accuracy for the single model. For example, the single model RNN
obtains an accuracy of 84.30% while “{RNN, CNN} $\mapsto$ CKB $\mapsto$RNN”
achieves 85.05%, suggesting that the knowledge stored in CKB helps to improve
RNN. Similar results were also observed for larger models such as BERT and
GPT-2. However, we also found that not all single models to which the
knowledge is exported obtain higher accuracies. For example, “{RNN, CNN}
$\mapsto$ CKB $\mapsto$CNN” obtains a lower accuracy than the original CNN. As
a result, how to ensure all participating models benefit from the integration
still needs further exploration.

Our approach significantly differs from model ensemble in two aspects. First,
while model ensemble has to maintain all participating models during
inference, CKB can export its knowledge to a single model. Second, model
ensemble requires all participating models are trained for the same task while
CKB in principle can take advantage of models trained for different tasks.

###  3.4 Effect of the Capacity of Continuous Knowledge Base

Configuration | #Param. | Accuracy
---|---|---
BERT | - | 87.90
BERT $\mapsto$ CKB $\mapsto$ BERT | 81.92K | 86.55
1.70M | 87.60
3.11M | 87.65

Table 4: Effect of the capacity of the continuous knowledge base on the Amazon
dataset.

Table 4 shows the effect of model capacity of CKB on classification accuracy.
We find that the accuracy generally rises with the increase of the number of
model parameters (i.e., $\mathcal{M}$) and the benefit becomes modest on
larger models.

###  3.5 Knowledge Distillation via Continuous Knowledge Base

Method | Hidden Size | Accuracy
---|---|---
Base | RNN | 256 | 84.30
RNN |  64 | 83.30
KD | RNN |  64 | 83.00
Ours | RNN |  64 | 83.00

Table 5: Results of mimicking knowledge distillation (KD) by CKB on the Amazon
dataset.

We can use the CKB to realize the goal of knowledge distillation (KD). As our
CKB can export the stored knowledge to a blank model, KD can be done by
importing the knowledge from the teacher model to the CKB and then exporting
the knowledge to the student model. In our experiments, we used a big RNN with
256 hidden size and a small RNN with 64 hidden size as the teacher and the
student models, respectively. As shown in Table 5, KD with our CKB achieved
the same performance as the standard KD method. The performances of two KD
methods are slightly worse than that of the small model trained on labeled
data. One possible reason is that the teacher model contains noise, affecting
the performance of the student model.

###  3.6 Transfer Learning via Continuous Knowledge Base

Method | Setting | Accuracy
---|---|---
PT & FT | ALBERT | Init $+$ FT | 79.20
ALBERT | PT $+$ FT | 84.50
Ours | CKB | Init $+$ FT | 82.10
CKB | TL $+$ FT | 84.10

Table 6: Results of Transfer Learning by CKB on the Amazon dataset. “PT”,
“FT”, and “TL” denote pre-training, fine-tuning, and transfer learning,
respectively. “Init” means model initialization.

It is easy to imitate transfer learning based on our CKB. In our experiments,
we used the CKB to mimic the transfer learning where a pre-trained language
model (i.e., ALBERT) is fine-tuned for text classification. As shown in Table
6, the ALBERT trained from scratch on the Amazon dataset obtained 79.20
accuracy scores on the test set while the pre-trained ALBERT can obtain 84.50
accuracy scores after fine-tuning. Analogously, the CKB-based model trained
from scratch with a randomly initialized CKB on the Amazon dataset achieved
82.10 accuracy scores on the test set. However, the CKB-based model, which
imported the knowledge from the pre-trained ALBERT, performed much better.
Note that in the knowledge transfer phase, we only used the unlabeled text
data from the Amazon dataset to import the knowledge from the pre-trained
ALBERT to the CKB.

##  4 Related Work

Our work draws inspiration from two lines of research: memory networks and
knowledge bases from pre-trained models.

###  4.1 Memory Networks

Memory networks (MNs) are first proposed by Weston et al., (2015). The
proposed MNs reason with inference components combined with a long-term memory
which acts as a dynamic knowledge base to store some knowledge from the input.
Sukhbaatar et al., (2015) extend MNs to the end-to-end paradigm by introducing
the attention mechanism (Bahdanau et al.,, 2015) to estimate the relevance of
each item in the memory. Kumar et al., (2016) propose dynamic memory networks
(DMNs) that use episodic memories to help generate better answers to given
questions. The episodic memory in DMNs can be updated dynamically according to
the input. Nematzadeh et al., (2020) also argue that the separation of
computation and storage is necessary and discuss the advantage of improving
memory in AI systems. Different from the existing MNs which store the input-
related knowledge, the proposed CKB is a global knowledge base that aims to
store the knowledge from different competent neural network models.

###  4.2 Knowledge Bases from Pre-trained Models

Recently, a number of works have been studied on what does the pre-trained
language model learns (Petroni et al.,, 2019; Bouraoui et al.,, 2020; Rogers
et al.,, 2020; Wang et al.,, 2020). Petroni et al., (2019) convert the fact
(i.e., subject-relation-object triple) into the cloze statement to test the
factual and commonsense knowledge in the pre-trained language model. By
transforming relational triples into masked sentences, Feldman et al., (2019)
propose to mine commonsense knowledge from pre-trained models. Bouraoui et
al., (2020) fine-tune the pre-trained BERT (Devlin et al.,, 2019) to predict
whether a given word pair is likely to be an instance of some relations. Wang
et al., (2020) state that pre-trained language models would be open knowledge
graphs and propose an unsupervised method to build knowledge graphs. Rombach
and Esser, (2020) propose a conditional invertible neural network to translate
between fixed representations from different off-the-shelf models. These
methods show that neural network models would contain knowledge while our CKB
investigates how to store and use this uninterpretable knowledge.

##  5 Conclusion and Future Work

We propose to build a universal continuous knowledge base (CKB) in this work.
Different from conventional knowledge bases using discrete symbols to
represent information, the proposed CKB stores the knowledge in multi-level
real-valued matrices. Based on the formalization where a neural network model
is a parameterized composite function that maps the input to the output, our
CKB imports the knowledge from the neural network model by learning the
mapping between the input and the output with the model-dependent interface.
Experiments on text classification show that continuous knowledge can be
imported and exported between neural networks and the CKB. Our CKB can also
mimic knowledge distillation and transfer learning in a novel paradigm. In the
future, we will extend the CKB to cross-task scenarios.

Our work has only touched the surface of building a universal continuous
knowledge base. There are a number of interesting directions awaiting further
exploration: a more sophisticated design of memory hierarchy, integrating
neural networks trained for different AI tasks, continual learning that can
import multiple neural networks asynchronously, visualization and
interpretation of the internal workings, and importing and exporting knowledge
between discrete and continuous KBs.

## Acknowledgements

We would like to thank Qun Liu, Xin Jiang, and Meng Zhang for their
constructive comments on this work. This work was supported by the National
Key R&D Program of China (No. 2017YFB0202204), National Natural Science
Foundation of China (No. 61925601, No. 61761166008) and Huawei Noah’s Ark Lab.

## References

  * Bahdanau et al., (2015) Bahdanau, D., Cho, K., and Bengio, Y. (2015).  Neural machine translation by jointly learning to align and translate.  In Proceedings of ICLR 2015.
  * Belinkov et al., (2017) Belinkov, Y., Durrani, N., Dalvi, F., Sajjad, H., and Glass, J. (2017).  What do neural machine translation models learn about morphology?  In Proceedings of ACL 2017.
  * Blevins et al., (2018) Blevins, T., Levy, O., and Zettlemoyer, L. (2018).  Deep rnns encode soft hierarchical syntax.  In Proceedings of ACL 2018.
  * Bollacker et al., (2008) Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008).  Freebase: a collaboratively created graph database for structuring human knowledge.  In Proceedings of SIGMOD 2008.
  * Bouraoui et al., (2020) Bouraoui, Z., Camacho-Collados, J., and Schockaert, S. (2020).  Inducing relational knowledge from BERT.  In Proceedings of AAAI 2020.
  * Brown et al., (2020) Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. (2020).  Language models are few-shot learners.  In Proceedings of NeurIPS 2020.
  * Cho et al., (2014) Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014).  Learning phrase representations using rnn encoder–decoder for statistical machine translation.  In Proceedings of EMNLP 2014.
  * Dahl et al., (2011) Dahl, G. E., Yu, D., Deng, L., and Acero, A. (2011).  Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition.  IEEE Transactions on Audio, Speech, and Language Processing.
  * Devlin et al., (2019) Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019).  Bert: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of NAACL-HLT 2019.
  * Ernst et al., (2015) Ernst, P., Siu, A., and Weikum, G. (2015).  Knowlife: A versatile approach for constructing a large knowledge grapha for biomedical sciences.  BMC Bioinformatics.
  * Feldman et al., (2019) Feldman, J., Davison, J., and Rush, A. (2019).  Commonsense knowledge mining from pretrained models.  In Proceedings of EMNLP-IJCNLP 2019.
  * Fu et al., (2018) Fu, Z., Tan, X., Peng, N., Zhao, D., and Yan, R. (2018).  Style transfer in text: Exploration and evaluation.  In Proceedings of AAAI 2018.
  * Hayes-Roth et al., (1983) Hayes-Roth, F., Waterman, D., and Lenat, D. (1983).  Building Expert Systems.  Addison-Wesley.
  * Hennessy and Patterson, (2011) Hennessy, J. L. and Patterson, D. A. (2011).  Computer Architecture: A Quantitative Approach.  Morgan Kaufmann.
  * Hewitt and Manning, (2019) Hewitt, J. and Manning, C. (2019).  A structural probe for finding syntax in word representations.  In Proceedings of NAACL-HLT 2019.
  * Hinton and Salakhutdinov, (2006) Hinton, G. and Salakhutdinov, R. (2006).  Reducing the dimensionality of data with neural networks.  Science.
  * Jiao et al., (2020) Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. (2020).  TinyBERT: Distilling bert for natural language understanding.  In Proceedings of EMNLP 2020: Findings.
  * Kim, (2014) Kim, Y. (2014).  Convolutional neural networks for sentence classification.  In Proceedings of EMNLP 2014.
  * Krizhevsky et al., (2012) Krizhevsky, A., Sutskever, I., and Hinton, G. (2012).  Imagenet classification with deep convolutional neural networks.  In Proceedings of NeurIPS 2012.
  * Kumar et al., (2016) Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., Zhong, V., Paulus, R., and Socher, R. (2016).  Ask me anything: Dynamic memory networks for natural language processing.  In Proceedings of ICML 2016.
  * Lan et al., (2020) Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., and Soricut, R. (2020).  ALBERT: A lite bert for self-supervised learning of language representations.  In Proceedings of ICLR 2020.
  * Lenat and Guha, (1990) Lenat, D. B. and Guha, R. V. (1990).  Building Large Knowledge-Based Systems: Representation and Inference in the Cyc Project.  Addison-Wesley.
  * Linzen et al., (2016) Linzen, T., Dupoux, E., and Goldberg, Y. (2016).  Assessing the ability of lstms to learn syntax-sensitive dependencies.  Transactions of the Association for Computational Linguistics.
  * Liu et al., (2016) Liu, P., Qiu, X., and Huang, X. (2016).  Recurrent neural network for text classification with multi-task learning.  In Proceedings of IJCAI 2016.
  * Loshchilov and Hutter, (2019) Loshchilov, I. and Hutter, F. (2019).  Decoupled weight decay regularization.  In Proceedings of ICLR 2019.
  * Marino et al., (2017) Marino, K., Salakhutdinov, R., and Gupta, A. (2017).  The more you know: Using knowledge graphs for image classification.  In Proceedings of CVPR 2017.
  * Nematzadeh et al., (2020) Nematzadeh, A., Ruder, S., and Yogatama, D. (2020).  On memory in human and artificial language processing systems.  In Proceedings of ICLR 2020 Workshop.
  * Petroni et al., (2019) Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., and Miller, A. (2019).  Language models as knowledge bases?  In Proceedings of EMNLP-IJCNLP 2019.
  * Radford et al., (2019) Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019).  Language models are unsupervised multitask learners.  OpenAI Blog.
  * Richardson and Domingos, (2003) Richardson, M. and Domingos, P. (2003).  Building large knowledge bases by mass collaboration.  In Proceedings of K-CAP 2003.
  * Rogers et al., (2020) Rogers, A., Kovaleva, O., and Rumshisky, A. (2020).  A primer in bertology: What we know about how bert works.  Transactions of the Association for Computational Linguistics.
  * Rombach and Esser, (2020) Rombach, R. and Esser, P. (2020).  Network-to-network translation with conditional invertible neural networks.  In Proceedings of NeurIPS 2020.
  * Sennrich et al., (2016) Sennrich, R., Haddow, B., and Birch, A. (2016).  Neural machine translation of rare words with subword units.  In Proceedings of ACL 2016.
  * Sukhbaatar et al., (2015) Sukhbaatar, S., Weston, J., Fergus, R., et al. (2015).  End-to-end memory networks.  In Proceedings of NeurIPS 2015.
  * Vaswani et al., (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017).  Attention is all you need.  In Proceedings of NeurIPS 2017.
  * Wang et al., (2020) Wang, C., Liu, X., and Song, D. (2020).  Language models are open knowledge graphs.  arXiv preprint arXiv:2010.11967.
  * Weston et al., (2015) Weston, J., Chopra, S., and Bordes, A. (2015).  Memeory networks.  In Proceedings of ICLR 2015.
  * Zhang et al., (2015) Zhang, X., Zhao, J., and LeCun, Y. (2015).  Character-level convolutional networks for text classification.  In Proceedings of NeurIPS 2015.
  * Zhang et al., (2018) Zhang, Y., Dai, H., Kozareva, Z., Smola, A. J., and Song, L. (2018).  Variational reasoning for question answering with knowledge graph.  In Proceedings of AAAI 2018.

##  Appendix A Experimental Details

###  A.1 Hyper-parameter Values

When importing and exporting continuous knowledge between neural networks and
the knowledge base, each mini-batch contains 8,192 tokens for the Amazon
dataset and 24,576 tokens for the Yelp dataset, respectively. We used the
AdamW optimizer (Loshchilov and Hutter,, 2019) with $\beta_{1}=0.9$,
$\beta_{2}=0.98$, $\epsilon=10^{-9}$, and L2 weight decay of $0.01$ to
optimize parameters. For knowledge importing, we set the learning rate to
2e-4. For knowledge exporting, the learning rate was set to 1e-4.

###  A.2 Model Selection

For importing and exporting knowledge for single neural networks, model
selection is done by choosing the checkpoint with the highest accuracy on the
validation set. When synchronously importing knowledge from multiple neural
networks to the continuous knowledge base (CKB), a problem is that these
models might not achieve the highest performance on the validation set at the
same time. To address this problem, we select the checkpoint as follows:

$\displaystyle\hat{c}=\mathop{\rm
argmax}_{c}\bigg{\\{}\\!\min_{i\in[1,N]}\\!\Big{\\{}\mathrm{acc}(c,\mathrm{NN}_{i})\Big{\\}}\bigg{\\}}$
(23)

where $N$ is the number of neural networks, $\mathrm{NN}_{i}$ is the $i$-th
neural network, $c$ is a checkpoint, and $\mathrm{acc(\cdot)}$ is a function
that calculates accuracy on the validation set.

###  A.3 Runtime Environment

We conducted all experiments on a server with the following environment:

  * •

Operation System: Ubuntu 18.04.2 LTS

  * •

CPU: AMD EPYC 7302 16-Core Processor

  * •

GPU: GeForce RTX 3090


# Probabilistic degenerate Dowling polynomials associated with random
variables

Taekyun Kim  Department of Mathematics, Kwangwoon University, Seoul 139-701,
Republic of Korea<EMAIL_ADDRESS>and  Dae San Kim  Department of Mathematics,
Sogang University, Seoul 121-742, Republic of Korea<EMAIL_ADDRESS>
###### Abstract.

The aim of this paper is to study probabilistic versions of the degenerate
Whitney numbers of the second kind and those of the degenerate Dowling
polynomials, namely the probabilistic degenerate Whitney numbers of the second
kind associated with $Y$ and the probabilistic degenerate Dowling polynomials
associated with $Y$. Here Y is a random variable whose moment generating
function exists in some neighborhood of the origin. We derive some properties,
explicit expressions, certain identities, recurrence relations and generating
functions for those numbers and polynomials. In addition, we investigate their
generalizations, namely the probabilistic degenerate $r$-Whitney numbers of
the second kind associated with $Y$ and the probabilistic degenerate
$r$-Dowling polynomials associated with $Y$, and get similar results to the
aforementioned numbers and polynomials.

###### Key words and phrases:

probabilistic degenerate Whitney numbers of the second kind associated with
$Y$; probabilistic degenerate Dowling polynomials associated with $Y$;
probabilistic degenerate $r$-Whitney numbers of the second kind associated
with $Y$; probabilistic degenerate $r$-Dowling polynomials associated with $Y$

###### 2010 Mathematics Subject Classification:

11B73; 11B83

##  1\. Introduction

Carlitz initiated a study of degenerate versions of special polynomials and
numbers in his work on the degenerate Bernoulli and degenerate Euler
polynomials and numbers (see [7]). It is remarkable that various degenerate
versions of many special numbers and polynomials have been explored recently
by some mathematicians not only with their number-theoretic or combinatorial
interests but also with their applications to other areas, including
probability, quantum mechanics and differential equations. In the course of
this quest, many different tools are employed, like generating functions,
combinatorial methods, $p$-adic analysis, umbral calculus, operator theory,
differential equations, special functions, probability theory and analytic
number theory (see [8,10,12-15,17,18] and the references therein).

Assume that $Y$ is a random variable such that the moment generating function
of $Y$

(1)
$E[e^{tY}]=\sum_{n=0}^{\infty}E[Y^{n}]\frac{t^{n}}{n!},\,\,(|t|<r)\,\,\,\mathrm{exists\,\,for\,\,some}\,\,r>0,$

where $E$ stands for the mathematical expectation (see [3,4,16,24]). The aim
of this paper is to study probabilistic versions of the degenerate Whitney
numbers of the second kind and those of the degenerate Dowling polynomials,
namely the probabilistic degenerate Whitney numbers of the second kind
associated with $Y$, $W_{m,\lambda}^{Y}(n,k)$ and the probabilistic degenerate
Dowling polynomials associated with $Y$, $D_{m,\lambda}^{Y}(n,x)$. In
addition, we investigate their generalizations, namely the probabilistic
degenerate $r$-Whitney numbers of the second kind associated with $Y$,
$W_{m,\lambda}^{(Y,r)}(n,k)$ and the probabilistic degenerate $r$-Dowling
polynomials associated with $Y$, $D_{m,\lambda}^{(Y,r)}(n,x)$. Then we derive
some properties, explicit expressions, certain identities, recurrence
relations and generating functions for those numbers and polynomials.

The outline of this paper is as follows. In Section 1, we recall the Whitney
numbers of the second kind, the $r$-Whitney numbers of the second kind, the
Dowling polynomials and the $r$-Dowling polynomials. We remind the reader of
the degenerate exponentials and the degenerate Stirling numbers of the second.
We recall the degenerate Whitney numbers of the second kind, the degenerate
$r$-Whitney numbers of the second kind, the degenerate Dowling polynomials and
the degenerate $r$-Dowling polynomials. We remind the reader of the partial
Bell polynomials and the complete Bell polynomials. Then, for any random
variable $Y$ satisfying the moment condition (see (1)), we recall the
definition of the probabilistic degenerate Stirling numbers of the second kind
associated with $Y$. Section 2 is the main result of this paper. Assume that
$Y$ is a random variable satisfying the moment condition in (1). Let
$(Y_{j})_{j\geq 1}$ be a sequence of mutually independent copies of the random
variable $Y$, and let $S_{k}=Y_{1}+\cdots+Y_{k},\ (k\geq 1)$, with $S_{0}=0$,
(see [1,3,4,16,25,26]). We define $W_{m,\lambda}^{Y}(n,k)$. Then three
different expressions of those numbers are obtained in terms of expectations
of various random variables in Theorems 2.1-2.3. Another expression is derived
in terms of the partial Bell polynomials in Theorem 2.15. In Theorem 2.4, we
derive a finite sum identity involving those numbers and
$E\Big{[}(mS_{k}+1)_{n,\lambda}\Big{]}$. Then we define
$D_{m,\lambda}^{Y}(n,x)$. We obtain the generating function of
$D_{m,\lambda}^{Y}(n,x)$ in Theorem 2.5 and two explicit expressions of those
polynomials in Theorems 2.6 and 2.7. We derive a recurrence relation for
$D_{m,\lambda}^{Y}(n,x)$ in Theorem 2.8. In Theorem 2.9, we get an identity
for $D_{m,\lambda}^{Y}(n,x)$, which shows that those polynomials do not
satisfy the binomial identity. We deduce a finite sum identity involving
$D_{m,\lambda}^{Y}(n,x)$ and the partial Bell polynomials in Theorem 2.11.
Moreover, higher order derivatives of $D_{m,\lambda}^{Y}(n,x)$ are obtained in
Theorem 2.14. Next, we define $W_{m,\lambda}^{(Y,r)}(n,k)$. We get an explicit
expression for those polynomials in Theorem 2.10. In Theorems 2.12 and 2.13,
we derive identities involving those polynomials and the partial Bell
polynomials. Then we define $D_{m,\lambda}^{(Y,r)}(n,x)$. Finally, we obtain
the generating function and an explicit expression, respectively in Theorem
2.16 and Theorem 2.17. In the rest of this section, we recall the facts that
are needed throughout this paper.

A finite lattice $L$ is geometric if it is a finite semimodular lattice which
is also atomic. Dowling constructed an important finite geometric lattice
$Q_{n}(G)$ out of a finite set of $n$ elements and a finite group $G$ of order
$m$, called Dowling lattice of rank $n$ over a finite group of order $m$ (see
[11,13,15]). If $L$ is the Dowling lattice $Q_{n}(G)$ of rank $n$ over a
finite group of $G$ order $m$, then the Whitney numbers of the second kind
$W_{Q_{n}(G)}(n,k)$ are denoted by $W_{m}(n,k)$ and satisfy the following
relation

(2)
$x^{n}=\sum_{k=0}^{n}W_{m}(n,k)m^{k}\bigg{(}\frac{x-1}{m}\bigg{)}_{k},\quad(n\geq
0),\quad(\mathrm{see}\ [11,13,15]),$

where $(x)_{0}=1,\ (x)_{n}=x(x-1)\cdots(x-n+1),\ (n\geq 1)$.

From (2), we have

(3) $(mx+1)^{n}=\sum_{k=0}^{n}W_{m}(n,k)m^{k}(x)_{k},\quad(n\geq
0),\quad(\mathrm{see}\ [11,13,15,18]).$

Let $r$ be a nonnegative integer. Then, as generalizations of $W_{m}(n,k)$,
the $r$-Whitney numbers of the second kind are defined by

(4) $(mx+r)^{n}=\sum_{k=0}^{n}W_{m}^{(r)}(n,k)m^{k}(x)_{k},\quad(n\geq
0),\quad(\mathrm{see}\ [13]).$

For $n\geq 0$, the Dowling polynomials $D_{m}(n,x)$ and the $r$-Dowling
polynomials $D_{m}^{(r)}(n,x)$ are respectively given by

(5) $D_{m}(n,x)=\sum_{k=0}^{n}W_{m}(n,k)x^{k},\quad(n\geq 0),$

and

$D_{m}^{(r)}(n,x)=\sum_{k=0}^{n}W_{m}^{(r)}(n,k)x^{k},\quad(n\geq
0),\quad(\mathrm{see}\ [15]).$

It is well known that the Stirling numbers of the second kind ${n\brace k}$
are defined by

(6) $x^{n}=\sum_{k=0}^{n}{n\brace k}(x)_{k},\quad(n\geq 0),\quad(\mathrm{see}\
[1-25]).$

For any nonzero $\lambda\in\mathbb{R}$, the degenerate exponentials are
defined by

(7) $e_{\lambda}^{x}(t)=(1+\lambda
t)^{\frac{x}{\lambda}}=\sum_{k=0}^{\infty}(x)_{k,\lambda}\frac{t^{k}}{k!},\quad(\mathrm{see}\
[10,12,14]),$

where

(8)
$(x)_{0,\lambda}=1,\quad(x)_{n,\lambda}=x(x-\lambda)(x-2\lambda)\cdots(x-(n-1)\lambda),\quad(n\geq
1).$

Note that $\lim_{\lambda\rightarrow 0}e_{\lambda}^{x}(t)=e^{xt}$.
In [10], the degenerate Stirling numbers of the second kind are given by

(9) $(x)_{n,\lambda}=\sum_{k=0}^{n}{n\brace k}_{\lambda}(x)_{k},\quad(n\geq
0).$

Note that $\displaystyle\lim_{\lambda\rightarrow 0}{n\brace
k}_{\lambda}={n\brace k}$.

Recently, the degenerate Whitney numbers of the second kind
$W_{m,\lambda}(n,k)$ are defined by

(10)
$(mx+1)_{n,\lambda}=\sum_{k=0}^{n}W_{m,\lambda}(n,k)m^{k}(x)_{k},\quad(n\geq
0),\quad(\mathrm{see}\ [13]).$

Note that $\lim_{\lambda\rightarrow 0}W_{m,\lambda}(n,k)=W_{m}(n,k)$.
In view of (5), the degenerate Dowling polynomials are defined by

(11) $D_{m,\lambda}(n,x)=\sum_{k=0}^{n}W_{m,\lambda}(n,k)x^{k},\quad(n\geq
0),\quad(\mathrm{see}\ [13]).$

From (10) and (11), we can derive the following equations.

(12)
$e_{\lambda}(t)\frac{1}{k!}\bigg{(}\frac{e_{\lambda}^{m}(t)-1}{m}\bigg{)}^{k}=\sum_{n=k}^{\infty}W_{m,\lambda}(n,k)\frac{t^{n}}{n!},$

and

(13)
$e_{\lambda}(t)e^{x\big{(}\frac{e_{\lambda}^{m}(t)-1}{m}\big{)}}=\sum_{n=0}^{\infty}D_{m,\lambda}(n,x)\frac{t^{n}}{n!},\quad(\mathrm{see}\
[13]).$

For $r\geq 0$, the degenerate $r$-Whitney numbers of the second kind is given
by

(14)
$(mx+r)_{n,\lambda}=\sum_{k=0}^{n}W_{m,\lambda}^{(r)}(n,k)m^{k}(x)_{k},\quad(n\geq
0),\quad(\mathrm{see}\ [13]).$

From (14), we note that

(15)
$e_{\lambda}^{r}(t)\frac{1}{k!}\bigg{(}\frac{e_{\lambda}^{m}(t)-1}{m}\bigg{)}^{k}=\sum_{n=k}^{\infty}W_{m,\lambda}^{(r)}(n,k)\frac{t^{n}}{n!},\quad(\mathrm{see}\
[13]).$

The degenerate $r$-Dowling polynomials are defined by

(16)
$e_{\lambda}^{r}(t)e^{x(\frac{e_{\lambda}^{m}(t)-1}{m})}=\sum_{n=0}^{\infty}D_{m,\lambda}^{(r)}(n,x)\frac{t^{n}}{n!},\quad(\mathrm{see}\
[15]).$

By (15) and (16), we get

(17)
$D_{m,\lambda}^{(r)}(n,x)=\sum_{k=0}^{n}W_{m,\lambda}^{(r)}(n,k)x^{k},\quad(n\geq
0),\quad(\mathrm{see}\ [11,15]).$

For any integer $k\geq 0$, the partial Bell polynomials are given by

(18)
$\frac{1}{k!}\bigg{(}\sum_{m=1}^{\infty}x_{m}\frac{t^{m}}{m!}\bigg{)}^{k}=\sum_{n=k}^{\infty}B_{n,k}(x_{1},x_{2},\dots,x_{n-k+1})\frac{t^{n}}{n!},\quad(\mathrm{see}\
[7,12]),$ (19) $\displaystyle B_{n,k}(x_{1},x_{2},\dots,x_{n-k+1})$
$\displaystyle=\sum_{\begin{subarray}{c}l_{1}+l_{2}+\cdots+l_{n-k+1}=k\\\
l_{1}+2l_{2}+\cdots+(n-k+1)l_{n-k+1}=n\end{subarray}}\frac{n!}{l_{1}!l_{2}!\cdots
l_{n-k+1}!}\bigg{(}\frac{x_{1}}{1!}\bigg{)}^{l_{1}}\bigg{(}\frac{x_{2}}{2!}\bigg{)}^{l_{2}}\cdots\bigg{(}\frac{x_{n-k+1}}{(n-k+1)!}\bigg{)}^{l_{n-k+1}}.$

The complete Bell polynomials are defined by

(20)
$\exp\bigg{(}\sum_{i=1}^{\infty}x_{i}\frac{t^{i}}{i!}\bigg{)}=\sum_{n=0}^{\infty}B_{n}(x_{1},x_{2},\dots,x_{n})\frac{t^{n}}{n!},\quad(\mathrm{see}\
[12]).$

Thus, by (19) and (20), we get

(21)
$B_{n}(x_{1},x_{2},\dots,x_{n})=\sum_{k=0}^{n}B_{n,k}(x_{1},x_{2},\dots,x_{n-k+1}),\quad(n\geq
0).$

Recently, for any random variable $Y$ satisfying (1), the probabilistic
degenerate Stirling numbers of the second kind associated with $Y$ are
introduced by

(22)
$\frac{1}{k!}\big{(}E[e_{\lambda}^{Y}(t)]-1\big{)}^{k}=\sum_{n=k}^{\infty}{n\brace
k}_{Y,\lambda}\frac{t^{n}}{n!},\quad(k\geq 0),\quad(\mathrm{see}\ [16]).$

Note that ${n\brace k}_{Y,\lambda}={n\brace k}_{\lambda}$, when $Y=1$.

##  2\. Probabilistic degenerate Dowling polynomials associated with random
variables

Throughout this section, we assume that $Y$ is a random variable such that the
moment generating function of $Y$

$E[e^{tY}]=\sum_{n=0}^{\infty}E[Y^{n}]\frac{t^{n}}{n!},\,\,(|t|<r)\,\,\,\mathrm{exists\,\,for\,\,some}\,\,r>0,\quad(\mathrm{see}\
[1,3,4,16,21,24,25,26]).$

We let $(Y_{j})_{j\geq 1}$ be a sequence of mutually independent copies of the
random variable $Y$, and let

$S_{0}=0,\quad
S_{k}=Y_{1}+Y_{2}+\cdots+Y_{k},\quad(k\in\mathbb{N}),\quad(\mathrm{see}\
[1,3,4,16,25,26]).$

In view of (12), we define the probabilistic degenerate Whitney numbers of the
second kind associated with $Y$ by

(23)
$\frac{1}{k!}\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}^{k}e_{\lambda}(t)=\sum_{n=k}^{\infty}W_{m,\lambda}^{Y}(n,k)\frac{t^{n}}{n!},\quad(k\geq
0).$

When $Y=1$, we have $W_{m,\lambda}^{Y}(n,k)=W_{m,\lambda}(n,k)$, (see (12)).
From (23), we note taht

(24) $\displaystyle\sum_{n=k}^{\infty}W_{m,\lambda}^{Y}(n,k)\frac{t^{n}}{n!}$
$\displaystyle=\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}\Big{(}E\big{[}e_{\lambda}^{mY}(t)\big{]}\Big{)}^{j}e_{\lambda}(t)$
$\displaystyle=\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}E\big{[}e_{\lambda}^{mY_{1}}(t)\big{]}\cdots
E\big{[}e_{\lambda}^{mY_{j}}(t)\big{]}e_{\lambda}(t)$
$\displaystyle=\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}E\Big{[}e_{\lambda}^{m(Y_{1}+\cdots+Y_{j})+1}(t)\Big{]}$
$\displaystyle=\sum_{n=0}^{\infty}\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}E\Big{[}(mS_{j}+1)_{n,\lambda}\Big{]}\frac{t^{n}}{n!}.$

Therefore, by comparing the coefficients on both sides of (24), we obtain the
following theorem.

######  Theorem 2.1.

For $n,k$ with $n\geq k\geq 0$, we have

$W_{m,\lambda}^{Y}(n,k)=\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}E\big{[}(mS_{j}+1)_{m,\lambda}\big{]}.$

Now, we define the operator $\triangle_{my}$ by

(25) $\displaystyle\triangle_{my}f(x)=f(x+my)-f(x),$
$\displaystyle\triangle_{my_{1},my_{2},\dots,my_{k}}f(x)=\triangle_{my_{1}}\circ\triangle_{my_{2}}\circ\cdots\circ\triangle_{my_{k}}f(x),\quad(\mathrm{see}\
[4-16]).$

From (25), we can derive the following equation

(26)
$\triangle_{my_{1},my_{2},\dots,my_{k}}e_{\lambda}^{x}(t)=\big{(}e_{\lambda}^{my_{k}}-1\big{)}\big{(}e_{\lambda}^{my_{k-1}}-1\big{)}\cdots\big{(}e_{\lambda}^{my_{1}}-1\big{)}e_{\lambda}^{x}(t).$

Thus, by (23) and (26), we get

(27) $\displaystyle\sum_{n=k}^{\infty}W_{m,\lambda}^{Y}(n,k)\frac{t^{n}}{n!}$
$\displaystyle=\frac{1}{k!m^{k}}e_{\lambda}(t)\Big{(}E[e_{\lambda}^{mY}(t)]-1\Big{)}^{k}$
$\displaystyle=\frac{1}{k!m^{k}}e_{\lambda}(t)\Big{(}E\big{[}e_{\lambda}^{mY_{1}}(t)\big{]}-1\Big{)}\Big{(}E\big{[}e_{\lambda}^{mY_{2}}(t)\big{]}-1\Big{)}\cdots\Big{(}E\big{[}e_{\lambda}^{mY_{k}}(t)\big{]}-1\Big{)}$
$\displaystyle=\frac{1}{k!m^{k}}E[\triangle_{mY_{1},mY_{2},\dots,Y_{k}}e_{\lambda}^{x}(t)]|_{x=1}=\sum_{n=0}^{\infty}\frac{1}{k!m^{k}}E[\triangle_{mY_{1},mY_{2},\dots,mY_{k}}(x)_{n,\lambda}]|_{x=1}\frac{t^{n}}{n!}.$

By comparing the coefficients on both sides of (27), we obtain the following
theorem.

######  Theorem 2.2.

For $n\geq k\geq 0$, we have

$\frac{1}{k!m^{k}}E[\triangle_{mY_{1},mY_{2},\dots,mY_{k}}(1)_{n,\lambda}]=W_{m,\lambda}^{Y}(n,k),$

where
$E[\triangle_{mY_{1},mY_{2},\dots,mY_{k}}(1)_{n,\lambda}]=E[\triangle_{mY_{1},mY_{2},\dots,mY_{k}}(x)_{n,\lambda}]|_{x=1}$,
and we use the same convention as this one in the sequel.

From the equation (45) of [16], we note that

(28)
$E\Big{[}\triangle_{mY_{1},mY_{2},\dots,mY_{k}}f(x)\Big{]}=\sum_{l=0}^{k}\binom{k}{l}(-1)^{k-l}E\Big{[}f(x+mS_{l})\Big{]}.$

Taking $f(x)=(x)_{n,\lambda}$ in (28) and then evaluating at $x=1$, we get

(29)
$E\Big{[}\triangle_{mY_{1},mY_{2},\dots,mY_{k}}(1)_{n,\lambda}\Big{]}=\sum_{l=0}^{k}\binom{k}{l}(-1)^{k-l}E\Big{[}(mS_{l}+1)_{n,\lambda}\Big{]}.$

Thus, by using (9), (29) and Theorem 2.2, we get

(30) $\displaystyle W_{m,\lambda}^{Y}(n,k)$
$\displaystyle=\frac{1}{k!m^{k}}E[\triangle_{mY_{1},mY_{2},\dots,mY_{k}}(1)_{n,\lambda}]=\frac{1}{k!m^{k}}\sum_{l=0}^{k}\binom{k}{l}(-1)^{k-l}E\Big{[}(mS_{l}+1)_{n,\lambda}\Big{]}$
$\displaystyle=\frac{1}{k!m^{k}}\sum_{l=0}^{k}\binom{k}{l}(-1)^{k-l}\sum_{j=0}^{n}{n\brace
j}_{\lambda}E[(mS_{l}+1)_{j}]$
$\displaystyle=\frac{1}{k!m^{k}}\sum_{j=0}^{n}{n\brace
j}_{\lambda}\sum_{l=0}^{k}\binom{k}{l}(-1)^{k-l}E\Big{[}(mS_{l}+1)_{j}\Big{]}.$

Therefore, by (30), we obtain the following theorem.

######  Theorem 2.3.

For $n\geq k\geq 0$, we have

$W_{m,\lambda}^{Y}(n,k)=\frac{1}{m^{k}k!}\sum_{j=0}^{n}{n\brace
j}_{\lambda}\sum_{l=0}^{k}\binom{k}{l}(-1)^{k-l}E\Big{[}(mS_{l}+1)_{j}\Big{]}.$

Before proceeding further, we recall the well-known binomial inversion given
by

(31) $a_{k}=\sum_{l=0}^{k}\binom{k}{l}b_{l}\Longleftrightarrow
b_{k}=\sum_{l=0}^{k}(-1)^{k-l}\binom{k}{l}a_{l}.$

By (29), (31) and Theorem 2.2, we get

(32)
$\displaystyle\sum_{k=0}^{N}E\Big{[}(mS_{k}+1)_{n,\lambda}\Big{]}=\sum_{k=0}^{N}\sum_{l=0}^{k}\binom{k}{l}E\Big{[}\triangle_{mY_{1},mY_{2},\dots,mY_{l}}(1)_{n,\lambda}\Big{]}$
$\displaystyle\quad=\sum_{l=0}^{N}E\Big{[}\triangle_{mY_{1},mY_{2},\dots,mY_{l}}(1)_{n,\lambda}\Big{]}\sum_{k=l}^{N}\binom{k}{l}=\sum_{l=0}^{N}\binom{N+1}{l+1}E\Big{[}\triangle_{mY_{1},mY_{2},\dots,mY_{l}}(1)_{n,\lambda}\Big{]}$
$\displaystyle\quad=\sum_{l=0}^{N}l!m^{l}W_{m,\lambda}^{Y}(n,l)\binom{N+1}{l+1}.$

Therefore, by (32), we obtain the following theorem.

######  Theorem 2.4.

For $N\geq 0$, we have

$\sum_{k=0}^{N}E\Big{[}(mS_{k}+1)_{n,\lambda}\Big{]}=\sum_{l=0}^{N}l!m^{l}\binom{N+1}{l+1}W_{m,\lambda}^{Y}(n,l).$

Now, we define the probabilistic degenerate Dowling polynomials associated
with $Y$ by

(33)
$D_{m,\lambda}^{Y}(n,x)=\sum_{k=0}^{n}W_{m,\lambda}^{Y}(n,k)x^{k},\quad(n\geq
0).$

Note that if $Y=1$, then $D_{m,\lambda}^{Y}(n,x)=D_{m,\lambda}(n,x),\ (n\geq
0)$. When $x=1$, $D_{m,\lambda}^{Y}(n)=D_{m,\lambda}^{Y}(n,1)$ are called the
probabilistic degenerate Dowling numbers associated with $Y$.
From (23) and (33), we note that

(34) $\displaystyle\sum_{n=0}^{\infty}D_{m,\lambda}^{Y}(n,x)\frac{t^{n}}{n!}$
$\displaystyle=\sum_{n=0}^{\infty}\sum_{k=0}^{n}W_{m,\lambda}^{Y}(n,k)x^{k}\frac{t^{n}}{n!}$
$\displaystyle=\sum_{k=0}^{\infty}x^{k}\sum_{n=k}^{\infty}W_{m,\lambda}^{Y}(n,k)\frac{t^{n}}{n!}=\sum_{k=0}^{\infty}x^{k}e_{\lambda}(t)\frac{1}{k!}\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}^{k}$
$\displaystyle=e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}.$

Therefore, by (34), we obtain the following theorem.

######  Theorem 2.5.

Let $Y$ be a random variable. Then we have

(35)
$e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}=\sum_{n=0}^{\infty}D_{m,\lambda}^{Y}(n,x)\frac{t^{n}}{n!}.$

By (35), we get

(36)
$\displaystyle\sum_{n=0}^{\infty}D_{m,\lambda}^{Y}(n,x)\frac{t^{n}}{n!}=e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}=e^{-\frac{x}{m}}e_{\lambda}(t)e^{\frac{x}{m}E[e_{\lambda}^{mY}(t)]}$
$\displaystyle=e^{-\frac{x}{m}}e_{\lambda}(t)\sum_{k=0}^{\infty}\frac{\big{(}\frac{x}{m}\big{)}^{k}}{k!}\Big{(}E\big{[}e_{\lambda}^{mY}(t)\big{]}\Big{)}^{k}=e^{-\frac{x}{m}}\sum_{k=0}^{\infty}\frac{\big{(}\frac{x}{m}\big{)}^{k}}{k!}E\Big{[}e_{\lambda}^{m(Y_{1}+\cdots+Y_{k})+1}(t)\Big{]}$
$\displaystyle=\sum_{n=0}^{\infty}e^{-\frac{x}{m}}\sum_{k=0}^{\infty}\frac{x^{k}}{m^{k}k!}E\Big{[}(mS_{k}+1)_{n,\lambda}\Big{]}\frac{t^{n}}{n!}.$

Therefore, by (36), we obtain the following Dobinski-like formula.

######  Theorem 2.6.

For $n\geq 0$, we have

(37)
$D_{m,\lambda}^{Y}(n,x)=e^{-\frac{x}{m}}\sum_{k=0}^{\infty}\frac{x^{k}}{m^{k}k!}E\Big{[}(mS_{k}+1)_{n,\lambda}\Big{]}.$

From (35), we note that

(38)
$\displaystyle\sum_{n=0}^{\infty}D_{m,\lambda}^{Y}(n,x)\frac{t^{n}}{n!}=e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}=e_{\lambda}(t)e^{\frac{x}{m}\sum_{j=1}^{\infty}E[(mY)_{j,\lambda}]\frac{t^{j}}{j!}}$
$\displaystyle=e_{\lambda}(t)\sum_{k=0}^{\infty}\frac{1}{k!}\bigg{(}\frac{x}{m}\sum_{j=1}^{\infty}E[(mY)_{j,\lambda}]\frac{t^{j}}{j!}\bigg{)}^{k}$
$\displaystyle=e_{\lambda}(t)\sum_{k=0}^{\infty}\sum_{n=k}^{\infty}B_{n,k}\bigg{(}\frac{x}{m}E[(mY)_{1,\lambda}],\frac{x}{m}E[(mY)_{2,\lambda}],\dots,\frac{x}{m}E[(mY)_{n-k+1,\lambda}]\bigg{)}\frac{t^{n}}{n!}$
$\displaystyle=\sum_{j=0}^{\infty}\frac{(1)_{j,\lambda}}{j!}t^{j}\sum_{l=0}^{\infty}\sum_{k=0}^{l}B_{l,k}\bigg{(}\frac{x}{m}E[(mY)_{1,\lambda}],\frac{x}{m}E[(mY)_{2,\lambda}],\dots,\frac{x}{m}E[(mY)_{l-k+1,\lambda}]\bigg{)}\frac{t^{l}}{l!}$
$\displaystyle=\sum_{n=0}^{\infty}\sum_{l=0}^{n}\binom{n}{l}(1)_{n-l,\lambda}\sum_{k=0}^{l}B_{l,k}\bigg{(}\frac{x}{m}E[(mY)_{1,\lambda}],\frac{x}{m}E[(mY)_{2,\lambda}],\dots,\frac{x}{m}E[(mY)_{l-k+1,\lambda}]\bigg{)}\frac{t^{n}}{n!}.$

Therefore, by comparing the coefficients on both sides of (38), we obtain the
following theorem.

######  Theorem 2.7.

For $n\geq 0$, we have

$\displaystyle D_{m,\lambda}^{Y}(n,x)$
$\displaystyle\quad=\sum_{l=0}^{n}\sum_{n=0}^{l}\binom{n}{l}(1)_{n-l,\lambda}B_{l,k}\bigg{(}\frac{x}{m}E[(mY)_{1,\lambda}],\frac{x}{m}E[(mY)_{2,\lambda}],\dots,\frac{x}{m}E[(mY)_{l-k+1,\lambda}]\bigg{)}.$

From (35), we have

(39)
$\displaystyle\sum_{n=0}^{\infty}D_{m,\lambda}^{Y}(n+1,x)\frac{t^{n}}{n!}=\frac{d}{dt}\sum_{n=0}^{\infty}D_{m,\lambda}^{Y}(n,x)\frac{t^{n}}{n!}=\frac{d}{dt}\bigg{(}e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}\bigg{)}$
$\displaystyle=e_{\lambda}^{1-\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}+\frac{xE[mYe_{\lambda}^{mY-\lambda}(t)]}{m}e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}$
$\displaystyle=\sum_{l=0}^{\infty}(-1)^{l}\lambda^{l}t^{l}\sum_{k=0}^{\infty}D_{m,\lambda}^{Y}(k,x)\frac{t^{k}}{k!}+\sum_{l=0}^{\infty}\frac{x}{m}E\big{[}(mY)_{l+1,\lambda}\big{]}\frac{t^{l}}{l!}\sum_{k=0}^{\infty}D_{m,\lambda}^{Y}(k,x)\frac{t^{k}}{k!}$
$\displaystyle=\sum_{n=0}^{\infty}\sum_{k=0}^{n}(-1)^{n-k}\lambda^{n-k}\frac{D_{m,\lambda}^{Y}(k,x)}{k!}t^{n}+\sum_{n=0}^{\infty}\sum_{k=0}^{n}\binom{n}{k}D_{m,\lambda}^{Y}(k,x)\frac{x}{m}E[(mY)_{n-k+1,\lambda}]\frac{t^{n}}{n!}$
$\displaystyle=\sum_{n=0}^{\infty}\bigg{(}\sum_{k=0}^{n}(-1)^{n-k}\lambda^{n-k}\frac{n!}{k!}D_{m,\lambda}^{Y}(k,x)+\frac{x}{m}\sum_{k=0}^{n}\binom{n}{k}D_{m,\lambda}^{Y}(k,x)E[(mY)_{n-k+1,\lambda}]\bigg{)}\frac{t^{n}}{n!}.$

Therefore, by comparing the coefficients on both sides of (39), we obtain the
following theorem.

######  Theorem 2.8.

For $n\geq 0$, we have

$\displaystyle D_{m,\lambda}^{Y}(n+1,x)$
$\displaystyle=\sum_{k=0}^{n}(-1)^{n-k}\lambda^{n-k}\frac{n!}{k!}D_{m,\lambda}^{Y}(k,x)+\frac{x}{m}\sum_{k=0}^{n}\binom{n}{k}D_{m,\lambda}^{Y}(k,x)E[(mY)_{n-k+1,\lambda}].$

By (35), we get

(40)
$\displaystyle\sum_{n=0}^{\infty}\sum_{k=0}^{n}\binom{n}{k}(1)_{n-k,\lambda}D_{m,\lambda}^{Y}(k,x+y)\frac{t^{n}}{n!}=e_{\lambda}(t)e_{\lambda}(t)e^{(x+y)\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}$
$\displaystyle=e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}e_{\lambda}(t)e^{y\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}$
$\displaystyle=\sum_{l=0}^{\infty}D_{m,\lambda}^{Y}(l,x)\frac{t^{l}}{l!}\sum_{k=0}^{\infty}D_{m,\lambda}^{Y}(k,y)\frac{t^{k}}{k!}=\sum_{n=0}^{\infty}\bigg{(}\sum_{k=0}^{n}\binom{n}{k}D_{m,\lambda}^{Y}(n-k,x)D_{m,\lambda}^{Y}(k,y)\bigg{)}\frac{t^{n}}{n!}.$

By (40), we obtain the following theorem which shows that
$D_{m,\lambda}^{Y}(n,x)$ does not satisfy the binomial identity.

######  Theorem 2.9.

For $n\geq 0$, we have

$\sum_{k=0}^{n}\binom{n}{k}(1)_{n-k,\lambda}D_{m,\lambda}^{Y}(k,x+y)=\sum_{k=0}^{n}\binom{n}{k}D_{m,\lambda}^{Y}(n-k,x)D_{m,\lambda}^{Y}(k,y).$

Let $r$ be a nonnegative integer. Then we consider the probabilistic
degenerate $r$-Whitney numbers of the second kind associated with $Y$ defined
by

(41)
$\frac{1}{k!}\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}^{k}e_{\lambda}^{r}(t)=\sum_{n=k}^{\infty}W_{m,\lambda}^{(Y,r)}(n,k)\frac{t^{n}}{n!},\quad(k\geq
0).$

When $Y=1,$ we have $W_{m,\lambda}^{(Y,r)}(n,k)=W_{m,\lambda}^{(r)}(n,k)$,
(see (15)).
From (41), we have

(42)
$\displaystyle\sum_{n=k}^{\infty}W_{m,\lambda}^{(Y,r)}(n,k)\frac{t^{n}}{n!}$
$\displaystyle=\frac{1}{k!m^{k}}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}\Big{(}E[e_{\lambda}^{mY}(t)]\Big{)}^{j}e_{\lambda}^{r}(t)$
$\displaystyle=\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}E\Big{[}e_{\lambda}^{mS_{j}+r}(t)\Big{]}$
$\displaystyle=\sum_{n=0}^{\infty}\bigg{(}\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}E\Big{[}(mS_{j}+r)_{n,\lambda}\Big{]}\bigg{)}\frac{t^{n}}{n!}.$

Therefore, by comparing the coefficients on both sides of (42), we obtain the
following theorem.

######  Theorem 2.10.

For $r\in\mathbb{N}\cup\\{0\\}$ and $n\geq k\geq 0$, we have

$W_{m,\lambda}^{(Y,r)}(n,k)=\frac{1}{m^{k}k!}\sum_{j=0}^{k}\binom{k}{j}(-1)^{k-j}E\Big{[}(mS_{j}+r)_{n,\lambda}\Big{]}.$

From (35), we note that

(43)
$\displaystyle\sum_{n=0}^{\infty}\sum_{k=0}^{n}\binom{n}{k}(x-1)_{n-k,\lambda}D_{m,\lambda}^{Y}(k,x)\frac{t^{n}}{n!}=\bigg{(}e_{\lambda}(t)e^{\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}\bigg{)}^{x}$
$\displaystyle=\bigg{(}e_{\lambda}(t)e^{\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}-1+1\bigg{)}^{x}=\sum_{k=0}^{\infty}\binom{x}{k}\bigg{(}e_{\lambda}(t)e^{\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}-1\bigg{)}^{k}$
$\displaystyle=\sum_{k=0}^{\infty}\binom{x}{k}k!\frac{1}{k!}\bigg{(}\sum_{j=1}^{\infty}D_{m,\lambda}^{Y}(j)\frac{t^{j}}{j!}\bigg{)}^{k}$
$\displaystyle=\sum_{k=0}^{\infty}\binom{x}{k}k!\sum_{n=k}^{\infty}B_{n,k}\Big{(}D_{m,\lambda}^{Y}(1),D_{m,\lambda}^{Y}(2),\dots,D_{m,\lambda}^{Y}(n-k+1)\Big{)}\frac{t^{n}}{n!}$
$\displaystyle=\sum_{n=0}^{\infty}\sum_{k=0}^{n}\binom{x}{k}k!B_{n,k}\Big{(}D_{m,\lambda}^{Y}(1),D_{m,\lambda}^{Y}(2),\dots,D_{m,\lambda}^{Y}(n-k+1)\Big{)}\frac{t^{n}}{n!}.$

Therefore, by comparing the coefficients on both sides of (43), we obtain the
following theorem.

######  Theorem 2.11.

For $n\geq 0$, we have

$\sum_{k=0}^{n}\binom{n}{k}(x-1)_{n-k,\lambda}D_{m,\lambda}^{Y}(k,x)=\sum_{k=0}^{n}\binom{x}{k}k!B_{n,k}\Big{(}D_{m,\lambda}^{Y}(1),D_{m,\lambda}^{Y}(2),\dots,D_{m,\lambda}^{Y}(n-k+1)\Big{)}.$

Now, we observe that

(44)
$te_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}=t\sum_{j=0}^{\infty}D_{m,\lambda}^{Y}(j,x)\frac{t^{j}}{j!}=\sum_{j=1}^{\infty}jD_{m,\lambda}^{Y}(j-1,x)\frac{t^{j}}{j!}.$

Thus, by (41) and (44), we get

(45)
$\displaystyle\bigg{(}\sum_{j=1}^{\infty}jD_{m,\lambda}^{Y}(j-1,x)\frac{t^{j}}{j!}\bigg{)}^{k}=t^{k}\bigg{(}e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}e_{\lambda}(t)\bigg{)}^{k}$
$\displaystyle=t^{k}\sum_{j=0}^{\infty}k^{j}x^{j}\frac{1}{j!}\Big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\Big{)}^{j}e_{\lambda}^{k}(t)=t^{k}\sum_{j=0}^{\infty}k^{j}x^{j}\sum_{n=j}^{\infty}W_{m,\lambda}^{(Y,k)}(n,j)\frac{t^{n}}{n!}$
$\displaystyle=\sum_{n=0}^{\infty}\sum_{j=0}^{n}k^{j}x^{j}W_{m,\lambda}^{(Y,k)}(n,j)\frac{t^{n+k}}{n!}=\sum_{n=k}^{\infty}k!\sum_{j=0}^{n-k}\binom{n}{k}k^{j}x^{j}W_{m,\lambda}^{(Y,k)}(n-k,j)\frac{t^{n}}{n!}.$

From (45), we can derive the following equation

(46)
$\displaystyle\sum_{n=k}^{\infty}\sum_{j=0}^{n-k}\binom{n}{k}k^{j}x^{j}W_{m,\lambda}^{(Y,k)}(n-k,j)\frac{t^{n}}{n!}=\frac{1}{k!}\bigg{(}\sum_{j=1}^{\infty}jD_{m,\lambda}^{Y}(j-1,x)\frac{t^{j}}{j!}\bigg{)}^{k}$
$\displaystyle=\sum_{n=k}^{\infty}B_{n,k}\Big{(}D_{m,\lambda}^{Y}(0,x),2D_{m,\lambda}^{Y}(1,x),3D_{m,\lambda}^{Y}(2,x),\dots,(n-k+1)D_{m,\lambda}^{Y}(n-k,x)\Big{)}\frac{t^{n}}{n!}.$

Therefore, by (46), we obtain the following theorem.

######  Theorem 2.12.

For $n,k$ with $n\geq k\geq 0$, we have

$\displaystyle\sum_{j=0}^{n-k}\binom{n}{k}k^{j}x^{j}W_{m,\lambda}^{(Y,k)}(n-k,j)$
$\displaystyle=B_{n,k}\Big{(}D_{m,\lambda}^{Y}(0,x),2D_{m,\lambda}^{Y}(1,x),3D_{m,\lambda}^{Y}(2,x),\dots,(n-k+1)D_{m,\lambda}^{Y}(n-k,x)\Big{)}.$

We note that

(47)
$\displaystyle\sum_{n=k}^{\infty}B_{n,k}\Big{(}D_{m,\lambda}^{Y}(1,x)-(1)_{1,\lambda},D_{m,\lambda}^{Y}(2,x)-(1)_{2,\lambda},\dots,D_{m,\lambda}^{Y}(n-k+1,x)-(1)_{n-k+1,\lambda}\Big{)}\frac{t^{n}}{n!}$
$\displaystyle=\frac{1}{k!}\bigg{(}\sum_{j=1}^{\infty}\Big{(}D_{m,\lambda}^{Y}(j,x)-(1)_{j,\lambda}\Big{)}\frac{t^{j}}{j!}\bigg{)}^{k}=\frac{1}{k!}\bigg{(}e_{\lambda}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}-e_{\lambda}(t)\bigg{)}^{k}$
$\displaystyle=e_{\lambda}^{k}(t)\frac{1}{k!}\bigg{(}e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}-1\bigg{)}^{k}=\sum_{j=k}^{\infty}{j\brace
k}x^{j}\frac{1}{j!}\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}^{j}e_{\lambda}^{k}(t)$
$\displaystyle=\sum_{j=k}^{\infty}{j\brace
k}x^{j}\sum_{n=j}^{\infty}W_{m,\lambda}^{(Y,k)}(n,j)\frac{t^{n}}{n!}=\sum_{n=k}^{\infty}\sum_{j=k}^{n}{j\brace
k}W_{m,\lambda}^{(Y,k)}(n,j)x^{j}\frac{t^{n}}{n!}.$

Therefore, by (2), we obtain the following theorem.

######  Theorem 2.13.

For $n,k$ with $n\geq k\geq 0$, we have

$\displaystyle
B_{n,k}\Big{(}D_{m,\lambda}^{Y}(1,x)-(1)_{1,\lambda},D_{m,\lambda}^{Y}(2,x)-(1)_{2,\lambda},\dots,D_{m,\lambda}^{Y}(n-k+1,x)-(1)_{n-k+1,\lambda}\Big{)}$
$\displaystyle=\sum_{j=k}^{n}{j\brace k}W_{m,\lambda}^{(Y,k)}(n,j)x^{j}.$

From (35), we have

(48)
$\displaystyle\sum_{n=0}^{\infty}\bigg{(}\frac{d}{dx}\bigg{)}^{k}D_{m,\lambda}^{Y}(n,x)\frac{t^{n}}{n!}=\bigg{(}\frac{d}{dx}\bigg{)}^{k}e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}e_{\lambda}(t)$
$\displaystyle=k!\frac{1}{k!}\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}^{k}e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}e_{\lambda}(t)$
$\displaystyle=\frac{k!}{m^{k}}\frac{1}{k!}\Big{(}E[e_{\frac{\lambda}{m}}^{Y}(mt)]-1\Big{)}^{k}e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}e_{\lambda}(t)$
$\displaystyle=\frac{k!}{m^{k}}\sum_{l=k}^{\infty}{l\brace
k}_{Y,\frac{\lambda}{m}}\frac{m^{l}t^{l}}{l!}\sum_{j=0}^{\infty}D_{m,\lambda}^{Y}(j,x)\frac{t^{j}}{j!}$
$\displaystyle=\sum_{n=k}^{\infty}k!\sum_{j=0}^{n-k}\binom{n}{j}D_{m,\lambda}^{Y}(j,x){n-j\brace
k}_{Y,\frac{\lambda}{m}}m^{n-k-j}\frac{t^{n}}{n!}.$

In particular, for $k=1$, we have

(49)
$\displaystyle\sum_{n=1}^{\infty}\frac{d}{dx}D_{m,\lambda}^{Y}(n,x)\frac{t^{n}}{n!}=\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}e_{\lambda}(t)$
$\displaystyle=\frac{1}{m}\sum_{l=1}^{\infty}E\big{[}(Y)_{l,\frac{\lambda}{m}}\big{]}\frac{m^{l}t^{l}}{l!}\sum_{j=0}^{\infty}D_{m,\lambda}^{Y}(j,x)\frac{t^{j}}{j!}$
$\displaystyle=\sum_{n=1}^{\infty}\sum_{j=0}^{n-1}\binom{n}{j}E[(Y)_{n-j,\frac{\lambda}{m}}]m^{n-j-1}D_{m,\lambda}^{Y}(j,x)\frac{t^{n}}{n!}.$

By comparing the coefficients on both sides of (48) and (49), we obtain the
following theorem.

######  Theorem 2.14.

For $n,k\in\mathbb{N}$ with $n\geq k$, we have

$\bigg{(}\frac{d}{dx}\bigg{)}^{k}D_{m,\lambda}^{Y}(n,x)=k!\sum_{j=0}^{n-k}\binom{n}{j}D_{m,\lambda}^{Y}(j,x){n-j\brace
k}_{Y,\frac{\lambda}{m}}m^{n-k-j}.$

In particular, for $k=1$, we have

$\frac{d}{dx}D_{m,\lambda}^{Y}(n,x)=\sum_{j=0}^{n-1}\binom{n}{j}E\big{[}(Y)_{n-j,\frac{\lambda}{m}}\big{]}D_{m,\lambda}^{Y}(j,x)m^{n-j-1},\quad(n\geq
1).$

We observe that

(50)
$\displaystyle\sum_{n=k}^{\infty}W_{m,\lambda}^{Y}(n,k)\frac{t^{n}}{n!}=\frac{1}{k!}\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}^{k}e_{\lambda}(t)$
$\displaystyle=\frac{1}{m^{k}}\frac{1}{k!}\Big{(}E[e_{\frac{\lambda}{m}}^{Y}(mt)]-1\Big{)}^{k}e_{\lambda}(t)=\frac{1}{m^{k}}\frac{1}{k!}\bigg{(}\sum_{j=1}^{\infty}E[(Y)_{j,\frac{\lambda}{m}}]m^{j}\frac{t^{j}}{j!}\bigg{)}^{k}e_{\lambda}(t)$
$\displaystyle=\frac{1}{m^{k}}\sum_{l=k}^{\infty}B_{l,k}\Big{(}E[(Y)_{1,\frac{\lambda}{m}}]m,E[(Y)_{2,\frac{\lambda}{m}}]m^{2},\dots,E[(Y)_{l-k+1,\frac{\lambda}{m}}]m^{l-k+1}\Big{)}\frac{t^{l}}{l!}\sum_{j=0}^{\infty}(1)_{j,\lambda}\frac{t^{j}}{j!}$
$\displaystyle=\sum_{n=k}^{\infty}\frac{1}{m^{k}}\sum_{l=k}^{n}\binom{n}{l}B_{l,k}\Big{(}E[(Y)_{1,\frac{\lambda}{m}}]m,E[(Y)_{2,\frac{\lambda}{m}}]m^{2},\dots,E[(Y)_{l-k+1,\frac{\lambda}{m}}]m^{l-k+1}\Big{)}(1)_{n-l,\lambda}\frac{t^{n}}{n!}.$

Thus, by (50), we get the next theorem.

######  Theorem 2.15.

For $n,k$ with $n\geq k\geq 0$, we have

$W_{m,\lambda}^{Y}(n,k)=\frac{1}{m^{k}}\sum_{l=k}^{n}\binom{n}{l}B_{l,k}\Big{(}E[(Y)_{1,\frac{\lambda}{m}}]m,E[(Y)_{2,\frac{\lambda}{m}}]m^{2},\dots,E[(Y)_{l-k+1,\frac{\lambda}{m}}]m^{l-k+1}\Big{)}(1)_{n-l,\lambda}.$

Now, we define the probabilistic degenerate $r$-Dowling polynomials associated
with $Y$ by

(51)
$D_{m,\lambda}^{(Y,r)}(n,x)=\sum_{k=0}^{n}W_{m,\lambda}^{(Y,r)}(n,k)x^{k},\quad(n\geq
0).$

From (51), we note that

(52)
$\displaystyle\sum_{n=0}^{\infty}D_{m,\lambda}^{(Y,r)}(n,x)\frac{t^{n}}{n!}$
$\displaystyle=\sum_{n=0}^{\infty}\sum_{k=0}^{n}W_{m,\lambda}^{(Y,r)}(n,k)x^{k}\frac{t^{n}}{n!}$
$\displaystyle=\sum_{k=0}^{\infty}x^{k}\sum_{n=k}^{\infty}W_{m,\lambda}^{(Y,r)}(n,k)\frac{t^{n}}{n!}=\sum_{k=0}^{\infty}x^{k}e_{\lambda}^{r}(t)\frac{1}{k!}\bigg{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\bigg{)}^{k}$
$\displaystyle=e_{\lambda}^{r}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}.$

By (52), we obtain the following theorem.

######  Theorem 2.16.

Let $r$ be a nonnegative integer. Then we have

(53)
$e_{\lambda}^{r}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}=\sum_{n=0}^{\infty}D_{m,\lambda}^{(Y,r)}(n,x)\frac{t^{n}}{n!}.$

By (53) , we get

(54)
$\displaystyle\sum_{n=0}^{\infty}D_{m,\lambda}^{(Y,r)}(n,x)\frac{t^{n}}{n!}$
$\displaystyle=e_{\lambda}^{r}(t)e^{x\big{(}\frac{E[e_{\lambda}^{mY}(t)]-1}{m}\big{)}}$
$\displaystyle=e^{-\frac{x}{m}}e_{\lambda}^{r}(t)\sum_{k=0}^{\infty}\frac{\big{(}\frac{x}{m}\big{)}^{k}}{k!}\Big{(}E[e_{\lambda}^{mY}(t)]\Big{)}^{k}=e^{-\frac{x}{m}}\sum_{k=0}^{\infty}\frac{\big{(}\frac{x}{m}\big{)}^{k}}{k!}E[e_{\lambda}^{mS_{k}+r}]$
$\displaystyle=\sum_{n=0}^{\infty}e^{-\frac{x}{m}}\sum_{k=0}^{\infty}\frac{x^{k}}{m^{k}k!}E\big{[}(mS_{k}+r)_{n,\lambda}\big{]}\frac{t^{n}}{n!}.$

By (54), we obtain the following Dobinski-like theorem.

######  Theorem 2.17.

For $n\geq 0$, we have

$D_{m,\lambda}^{(Y,r)}(n,x)=e^{-\frac{x}{m}}\sum_{k=0}^{\infty}\frac{x^{k}}{k!m^{k}}E\Big{[}(mS_{k}+r)_{n,\lambda}\Big{]}.$

When $Y=1$, we have
$D_{m,\lambda}^{(Y,r)}(n,x)=D_{m,\lambda}^{(r)}(n,x),\quad(n\geq 0).$

##  3\. Conclusion

In this paper, we studied by using generating functions the probabilistic
degenerate Whitney numbers of the second kind associated with $Y$ and the
probabilistic degenerate Dowling polynomials associated with $Y$. Here $Y$ is
a random variable satisfying the moment condition in (1). In addition, we
investigated generalizations of those numbers and polynomials, namely the
probabilistic degenerate $r$-Whitney numbers of the second kind associated
with $Y$ and the probabilistic degenerate $r$-Dowling polynomials associated
with $Y$. We derived some properties, explicit expressions, certain
identities, recurrence relations and generating functions for those numbers
and polynomials.

It is one of our future projects to continue to study degenerate versions,
$\lambda$-analogues and probabilistic versions of many special polynomials and
numbers and to find their applications to physics, science and engineering as
well as to mathematics.

## References

  * [1] Abbas, M.; Bouroubi, S. _On new identities for Bell’s polynomials,_ Discrete Math. 293 (2005), no. 1-3, 5-10.
  * [2] Abramowitz, M.; Stegun, I. A. _Handbook of mathematical functions with formulas, graphs, and mathematical tables,_ For sale by the Superintendent of Documents. National Bureau of Standards Applied Mathematics Series, No. 55. U. S. Government Printing Office, Washington, DC, 1964.
  * [3] Adams, C. R.; Morse, A. P. _Random sampling in the evaluation of a Lebesgue integral,_ Bull. Amer. Math. Soc. 45 (1939), no. 6, 442-447.
  * [4] Adell, Jose A. _Probabilistic Stirling numbers of the second kind and applications,_ J. Theoret. Probab. 35 (2022), no.1, 636-652.
  * [5] Araci, S.; Acikgoz, M. _A note on the Frobenius-Euler numbers and polynomials associated with Bernstein polynoomials,_ Adv. Stud. Contemp. Math. (Kyungshang) 22 (2012), no. 3, 399-406.
  * [6] Boubellouta, K.; Boussayoud, A.; Araci, S.; Kerada, M. _Some theorems on generating functions and their applications,_ Adv. Stud. Contemp. Math. (Kyungshang) 30 (2020), no. 3, 307-332.
  * [7] Carlitz, L. _Degenerate Stirling, Bernoulli and Eulerian numbers,_ Utilitas Math. 15 (1979), 51-88.
  * [8] Chung, S.-K.; Jang, G.-W.; Kwon, J.; Lee, J. _Some identities of the degenerate Changhee numbers of second kind arising from differential equations,_ Adv. Stud. Contemp. Math. (Kyungshang) 28 (2018), no. 4, 577-587.
  * [9] Comtet, L. _Advanced combinatorics,_ The art of finite and infinite expansions. Revised and enlarged edition, D. Reidel Publishing Co., Dordrecht, 1974.
  * [10] Kim, D. S.; Kim, T. _A note on a new type of degenerate Bernoulli numbers,_ Russ. J. Math. Phys. 27 (2020), no. 2, 227-235.
  * [11] Kim, D. S.; Kim, T. _Normal ordering associated with $\lambda$-Whitney numbers of the first kind in $\lambda$-shift algebra,_ Russ. J. Math. Phys. 30 (2023), no. 3, 310-319.
  * [12] Kim, D. S.; Kim, T.; Lee, S.-H.; Park, J.-W. _Some new formulas of complete and incomplete degenerate Bell polynomials,_ Adv. Difference Equ. 2021, Paper No. 326.
  * [13] Kim, T.; Kim, D. S. _Degenerate Whitney numbers of first and second kind of Dowling lattices,_ Russ. J. Math. Phys. 29 (2022), no. 3, 358-377.
  * [14] Kim, T.; Kim, D. S. _Some identities involving degenerate Stirling numbers associated with several degenerate polynomials and numbers,_ Russ. J. Math. Phys. 30 (2023), no. 1, 62-75.
  * [15] Kim, T.; Kim, D. S. _Degenerate $r$-Whitney numbers and degenerate $r$-Dowling polynomials via boson operators,_ Adv. Appl. Math. 140 (2022), Paper No. 102394.
  * [16] Kim, T.; Kim, D. S. _Probabilistic degenerate Bell polynomials associated with random variables,_ Russ. J. Math. Phys. 30 (2023), no. 4, 528-542.
  * [17] Kim, T.; Kim, D. S.; Dolgy, D. V.; Park, J.-W. _Degenerate binomial and Poisson random variables associated with degenerate Lah-Bell polynomials,_ Open Math. 19 (2021), no. 1, 1588-1597.
  * [18] Ma, Y.; Kim, T.; Lee, H.; Kim, D. S. _Some identities of fully degenerate Dowling and fully degenerate Bell polynomials arising from umbral calculus,_ Fractals 30(2022), no. 10, 2240257. https://doi.org/10.1142/S0218348X22402575
  * [19] Pyo, S.-S. _Degenerate Cauchy numbers and polynomials of the fourth kind,_ Adv. Stud. Contemp. Math. (Kyungshang) 28 (2018), no. 1, 127-138.
  * [20] Roman, S. _The umbral calculus,_ Pure and Applied Mathematics 111, Academic Press, Inc. [Harcourt Brace Jobvanovich, Publishers], New York, 1984.
  * [21] Ross, S. M. _Introduction to probability models,_ Twelfth edition of Academic Press, London, 2019.
  * [22] Simsek, Y. _Identities on the Changhee numbers and Apostol-type Daehee polynomials,_ Adv. Stud. Contemp. Math. (Kyungshang) 27 (2017), no. 2, 199-212.
  * [23] Simsek, Y. _Construction of generalized Leibnitz type numbers and their properties,_ Adv. Stud. Contemp. Math. (Kyungshang) 31 (2021), no. 3, 311-323.
  * [24] Soni, R.; Vellaisamy, P.; Pathak, A. K. _A probabilistic generalization of the Bell polynomials,_ J. Anal. (2023). https://doi.org/10.1007/s41478-023-00642-y
  * [25] Ta, B. Q. _Probabilistic approach to Appell polynomials,_ Expo. Math. 33 (2015), no. 3, 269-294.
  * [26] Teicher, H. _An inequality on Poisson probabilities,_ Ann. Math. Statist. 26 (1955), 147-149.