# Continual Learning and Private Unlearning

†Bo Liu, †Qiang Liu †,§Peter Stone,
†The University of Texas at Austin, §Sony AI
<EMAIL_ADDRESS>
###### Abstract

As intelligent agents become autonomous over longer periods of time, they may
eventually become lifelong counterparts to specific people. If so, it may be
common for a user to want the agent to master a task temporarily but later on
to forget the task due to privacy concerns. However enabling an agent to
_forget privately_ what the user specified without degrading the rest of the
learned knowledge is a challenging problem. With the aim of addressing this
challenge, this paper formalizes this continual learning and private
unlearning (CLPU) problem. The paper further introduces a straightforward but
exactly private solution, CLPU-DER++, as the first step towards solving the
CLPU problem, along with a set of carefully designed benchmark problems to
evaluate the effectiveness of the proposed solution.

##  1 Introduction

Continual learning (CL) studies how an intelligent agent can learn continually
over a sequence of tasks. In particular, when the agent is learning a new
task, it is generally assumed that it loses access to data from previous
tasks. As a result, the goal of a successful CL algorithm is to _forget_ as
little as possible about previous tasks while maximally adapting past
knowledge to help learn the new task.

As deep learning has become increasingly popular, it has become generally
known that straightforwardly applying stochastic gradient descent (SGD) on
deep architectures when learning over a sequence of tasks leads to the so-
called _catastrophic forgetting_ phenomenon (Kirkpatrick et al., 2017), i.e.,
the network forgets much of what it learned previously when learning new
knowledge. Thus, much CL research has focused on developing methods to
mitigate forgetting. However, forgetting is _not_ always bad. Besides the fact
that graceful forgetting—the process of deliberately compressing useful
knowledge or removing useless knowledge—can help abstract learned knowledge
and leave more “space” for learning new knowledge (Bjork & Bjork, 2019), we
posit that it may also become common for an agent to be required to completely
remove any trace of having learned a specific task.

For example, consider a situation in which a person is undergoing a medical
treatment plan and requests that their service robot learns to assist with the
treatment. However, after having recovered, when a friend is about to visit,
the person may not want the robot to exhibit any evidence of their previous
medical treatment. In this case, the person would like to be able to request
that the robot privately remove all knowledge of the treatment plan without
impairing other unrelated knowledge it may have acquired during (or before or
after) the time of the treatment.

Note that even though catastrophic forgetting often happens naturally with
rich parametric models such as deep neural networks, it might not be
sufficient because 1) the user may want the agent to unlearn immediately
(instead of unlearning over time) and 2) the unlearning must happen privately,
meaning that after forgetting, it must not be possible to retrieve any
information pertaining to the task, or even detect that the task has been
previously learned.

With this motivation in mind, in this work, we present a novel but general CL
problem setting where for each task, besides providing the task data, the user
additionally provides a learning instruction indicating whether they want the
agent to learn and remember the task permanently, to temporarily learn the
task such that later on it will either forget or permanently remember it, or
to forget a certain task completely and privately. We call this novel problem
_continual learning and private unlearning_ (CLPU). To the best of our
knowledge, only one previous paper discusses a similar problem setting
pertaining to selective forgetting in continual learning (Shibata et al.,
2021). However, the problem in that paper is different from CLPU as it defines
forgetting as maximally degrading the performance on a task. As discussed in
Sec. 5, this requirement is not privacy-preserving and can potentially leak
information (e.g., that the task has been previously learned).

To address CLPU, we propose a straightforward but exact method, named CLPU-
DER++, based on both the dynamic architecture approach (Rusu et al., 2016) and
the rehearsal approach (Buzzega et al., 2020) from the CL literature.
Furthermore, we design a set of benchmark tasks along with novel evaluation
metrics for evaluating any CLPU methods. To summarize, our main contributions
are:

  * •

Formulating the continual learning and private unlearning (CLPU) problem.

  * •

Presenting an initial solution, CLPU-DER++, to CLPU that achieves exact
unlearning, and demonstrating its effectiveness on a novel set of benchmarks
designed for CLPU.

##  2 Related Work

In this section, a brief review of continual learning and machine unlearning
is provided.

#### Continual Learning

Continual learning (CL) assumes a learning agent learns continually over a
sequence of tasks and in general the agent loses access to previous data when
learning new tasks. Due to its generality, CL has been applied to a variety of
areas including computer vision (Kirkpatrick et al., 2017), reinforcement
learning (Kirkpatrick et al., 2017; Riemer et al., 2018), natural language
processing (Biesialska et al., 2020), and robotics (Liu et al., 2021). There
exist three main approaches towards continual learning. 1) Dynamic
architecture approaches study how to carefully and gradually expand the
learning model to incorporate the learning of new knowledge (Rusu et al.,
2016; Yoon et al., 2017; Mallya et al., 2018; Rosenfeld & Tsotsos, 2018;
Mallya & Lazebnik, 2018; Hung et al., 2019b; a; Wu et al., 2020a). 2)
Regularization-based methods design a regularization objective that prevents
the model parameter deviating too much from the previously learned model(s)
(Kirkpatrick et al., 2017; Chaudhry et al., 2018a; Schwarz et al., 2018;
Aljundi et al., 2019). 3) Rehearsal methods save exemplar raw data, called
episodic memory, from previously learned tasks. When learning new tasks, these
methods simultaneously learn on the new task and rehearse on episodic memories
to retain past knowledge (Chaudhry et al., 2019; Lopez-Paz & Ranzato, 2017;
Chaudhry et al., 2018b; Buzzega et al., 2020). Other than saving the raw data
points, pseudo-rehearsal like training a generative model to replay past
experience is also a popular approach (Shin et al., 2017). For a comprehensive
survey of existing continual learning methods, we refer the reader to tow
survey papers (Van de Ven & Tolias, 2019; Delange et al., 2021). In contrast
to existing CL methods that focus on reducing forgetting, the CLPU problem
requires the agent to deliberately forget a particular task in a privacy-
preserving fashion upon request, while minimally influencing other knowledge.

#### Machine Unlearning

Machine unlearning (MU) studies how to remove the effect of a specific
training sample on a learning model per a user’s request (Cao & Yang, 2015).
The most straightforward approach is to retrain the model on all data except
the portion that has been removed, but this approach is in general impractical
if the entire training set is large. To this end, typical MU approaches
consider training multiple models on different shards of data so that
unlearning only requires retraining a specific model on part of the dataset
(Bourtoule et al., 2019), or storing learned model parameters and their
gradients for rapid retraining (Wu et al., 2020b). There is also research
focusing on MU with specific model or problem assumptions, such as linear
models (Guo et al., 2019), random forests (Brophy & Lowd, 2021), or
k-means(Ginart et al., 2019). Based on differential privacy, Golatkar et al.
introduced “scrubbing” that removes information from the weights of deep
networks based on the Fisher Information Matrix (Golatkar et al., 2020).
Mixed-Linear Forgetting proposes a tractable optimization problem by lineary
approximating the amount of change in weights due to the addition of any
training data (Golatkar et al., 2021). The above methods all consider the MU
problem in general where the preserved dataset (e.g., data except the removed
ones) iis available, which is not the case in continual learning. Recently, a
particularly relevant study first considers MU in the context of continual
learning (Shibata et al., 2021). However, their problem definition aims to
make the model predict as wrongly as possible on the removed data, which does
not in general protect the user’s privacy. For instance, if the agent has
learned task B that helps improve prediction on task A, which the agent is
asked to forget, then completely random prediction on task A also reveals that
the agent has learned on it before but asked to unlearn it later. In fact, as
we observe from experimental results (Sec. 5), simply decreasing the model’s
performance on removed data is not private.

##  3 Background

In this section, we present the notation, definitions, and necessary
background information to formalize CLPU.

###  3.1 Continual Learning

In continual learning (CL), an agent observes and learns $K$ tasks in a
sequence. In this work, we assume each task $k\in[K]$111$[K]$ denotes
$\\{1,2,\dots,K\\}$. is a supervised learning task with a loss function
$\ell^{k}:\mathcal{X}\times\mathcal{Y}\rightarrow\mathbb{R}$, a training
dataset
$D^{k}=\\{(x^{k,\text{train}}_{1},y^{k,\text{train}}_{1}),\dots,(x^{k,\text{train}}_{n},y^{k,\text{train}}_{n})\\}$
and a testing dataset
$D^{k}_{\text{test}}=\\{(x^{k,\text{test}}_{1},y^{k,\text{test}}_{1}),\dots,(x^{k,\text{test}}_{n^{\prime}},y^{k,\text{test}}_{n^{\prime}})\\}$.
Here, $(x,y)$ are the raw data and labels where $x\in\mathcal{X}$ and
$y\in\mathcal{Y}$. Assume the agent adopts a model $f_{\theta}$ parameterized
by $\theta\in\mathbb{R}^{d}$. For instance, for a classification task,
$\text{softmax}(f_{\theta})$ produces a probability distribution over
$\mathcal{Y}$. Then $\ell^{k}$ evaluates how well $f_{\theta}$ predicts $y$
given $x$. For instance, $\ell^{k}$ can be the standard cross-entropy loss for
classification tasks (e.g.,
$\ell^{k}(x,y,f_{\theta})=\log\sum_{y^{\prime}}\exp\big{(}f_{\theta}(x)[y^{\prime}]\big{)}-f_{\theta}(x)[y]$).

On learning task $k$, the agent loses its access to
$D^{<k}=\\{D^{1},\dots,D^{k-1}\\}$. After learning all $K$ tasks, the agent’s
objective is to achieve low loss on all test datasets
$\\{D^{k}_{\text{test}}\\}_{k\in[K]}$. Assume the agent learns with a model
$f$ that is parameterized by $\theta\in\mathbb{R}^{m}$. Denote the agent’s
model after learning task $k$ as $f_{\theta^{k}}$. Then, the overarching
objective of a CL agent is to optimize

$\min_{\theta^{K}}\frac{1}{K}\sum_{k\in[K]}\mathbb{E}_{(x,y)\sim
D^{k}_{\text{test}}}\bigg{[}\ell^{k}(x,y,f_{\theta^{K}})\bigg{]}.$ (1)

However, equation 1 is hard to directly optimize using gradient-based methods
(e.g., Stochastic Gradient Descent (SGD)) because of the dependency of
$\theta^{K}$ on all previous $\theta^{k}$ such that $k<K$. Therefore,
alternatively, from an induction point of view, the objective can be
decomposed into $K$ objectives throughout the learning process. For learning
the first task, the agent just optimizes the training loss $\ell^{1}$ on the
training dataset $D^{1}$ as in standard supervised learning. For any task
$k>1$, the agent is asked to achieve low loss on task $k$ while maintaining
its performance on previously learned tasks (Lopez-Paz & Ranzato, 2017):

$\min_{\theta\in\mathbb{R}^{m}}~{}\underbrace{\mathbb{E}_{(x,y)\sim
D^{k}}\bigg{[}\ell^{k}(x,y,f_{\theta})\bigg{]}}_{\text{performance on
task}~{}k}~{}~{}~{}\text{s.t.}~{}~{}~{}\forall\tau<k,~{}~{}\underbrace{\mathbb{E}_{(x,y)\sim
D^{\tau}}\bigg{[}\ell^{\tau}(x,y,f_{\theta})-\ell^{\tau}(x,y,f_{\theta^{(k-1)}})\bigg{]}}_{\text{forgetting
on task}~{}\tau}\leq 0.$ (2)

Note that here we also replace the testing loss by training loss as the agent
is not assumed to have access to test data during training. In practice, when
the underlying $K$ tasks share the same loss function (e.g., they are all
image classification tasks), which we assume for the rest of this paper, we
can elide the superscript $k$ in $\ell^{k}$.

The main challenge for solving CL results from losing access to $D^{<k}$.
Regularization-based methods assume all information learned from $D^{<k}$ is
in $\theta^{k-1}$. Thus they aim to minimize the training loss on $D^{k}$
while ensuring that $\theta$ stays close to $\theta^{k-1}$:

$\min_{\theta\in\mathbb{R}^{m}}~{}\mathbb{E}_{(x,y)\sim
D^{k}}\bigg{[}\ell(x,y,f_{\theta})\bigg{]}+\alpha D(\theta,\theta^{k-1}),$ (3)

where $\alpha$ is a hyperparameter determining the strength of regularization
and $D(\theta,\theta^{k-1}$) is a divergence measure that captures how similar
$\theta$ is to $\theta^{k-1}$. Another popular and empirically more effective
approach in CL is the rehearsal-based approach. These methods allow the agent
to store a small number of exemplar data points, known as the _episodic
memory_ $\\{B^{\tau}\\}_{\tau<k}$, for maintaining the agent’s learned
knowledge on previous tasks. In particular, $B^{\tau}$ stores
$b^{\tau}\ll|D^{\tau}|$ i.i.d. sampled data points from $D^{\tau}$, optionally
with $f_{\theta^{\tau}}$’s final layer output (a.k.a. the logits):

$B^{\tau}=\bigg{\\{}\big{(}x_{i},y_{i},h_{i}=f_{\theta^{\tau}}(x_{i})\big{)}~{}~{}\bigg{|}~{}~{}(x_{i},y_{i})\stackrel{{\scriptstyle\text{i.i.d}}}{{\sim}}D^{\tau}\bigg{\\}}_{1\leq
i\leq b^{\tau}}.$

As a result, the general objective of rehearsal-based methods is

$\min_{\theta\in\mathbb{R}^{m}}~{}\frac{\beta}{|D^{k}|}\sum_{(x,y)\sim
D^{k}}\ell(x,y,f_{\theta})+(1-\beta)\sum_{\tau<k}\frac{1}{|B^{\tau}|}\sum_{(x^{\prime},y^{\prime},h^{\prime})\sim
B^{\tau}}\hat{\ell}(x^{\prime},y^{\prime},h^{\prime},f_{\theta}),$ (4)

where $\beta>0$ is a hyper-parameter that weights the trade-off between
learning new and preserving old knowledge. $\hat{\ell}$ can be the standard
cross-entropy loss $\ell$, the knowledge distillation loss
$\ell_{\text{distill}}(x,h,f_{\theta})=D_{\mathrm{KL}}\big{(}{\mathrm{softmax}(f_{\theta}(x))\mid\mid
h}\big{)}$, or the mean-square-error between the predicted and saved logits
$\ell_{\text{mse}}(x,h,f_{\theta})=\frac{1}{2}||f_{\theta}(x)-h||^{2}_{2}$,
which has shown strong performance at preserving past knowledge (Buzzega et
al., 2020). Specifically, for the Dark Experience Replay++ (DER++) method
(Buzzega et al., 2020), $\hat{\ell}$ is a linear combination of the standard
cross entropy loss and mean-square-error loss:
$\hat{\ell}(x^{\prime},y^{\prime},h^{\prime},f_{\theta})=\alpha_{1}\ell(x^{\prime},y^{\prime},f_{\theta})+\alpha_{2}\ell_{\text{mse}}(x^{\prime},h^{\prime},f_{\theta})$.

###  3.2 Machine Unlearning

Unlike continual learning which studies learning over sequential tasks,
contemporary research in machine unlearning (MU) mainly focuses on single-task
learning (Cao & Yang, 2015). In particular, MU is often studied in the context
of supervised learning, though extensions to other types of learning are
relatively straightforward. Under the standard supervised learning setting,
the agent is given a training dataset
$D=\\{(x_{1},y_{1}),(x_{2},y_{2}),\dots,(x_{n},y_{n})\\}$. The agent applies a
(stochastic) learning algorithm $A$ on $D$ to learn a model $f_{\theta}$
parameterized by $\theta$, such that $f_{\theta}$ achieves low empirical loss
(e.g., $\frac{1}{n}\sum_{i=1}^{n}\ell(x_{i},y_{i},f_{\theta})$ is small).
Denote $A(D)$ as the distribution over the resulting model parameters $\theta$
when $A$ is applied on $D$.

A user can then request that the agent unlearn part of the dataset, which we
call the forget set, $D_{f}\subset D$. Denote $D_{r}=D\setminus D_{f}$ as the
retained dataset. Machine unlearning (MU) aims to find an unlearning algorithm
$R_{A}$ that returns a model $\theta\sim R_{A}(D,A(D),D_{f})$, which possesses
no information about $D_{f}$ while performing well on $D_{r}$. In general, it
is usually assumed that $|D_{f}|\ll|D|$, otherwise one can directly retrain a
model on $D_{r}$. If the unlearned model from $R_{A}(D,A(D),D_{f})$ has no
information about $D_{f}$, an adversary cannot differentiate the model after
unlearning from a model that is retrained on $D_{r}$, and we say that
$(A,R_{A})$ achieves _exact unlearning_. The formal definitions are as
follows.

######  Definition 3.1 (Exact Unlearning).

A pair of learning and unlearning algorithms $(A,R_{A})$ achieve exact
unlearning if

$\forall D,~{}D_{f}\subset
D,~{}~{}A(D_{r})=_{d}R_{A}(D,A(D),D_{f}),~{}~{}\text{where}~{}~{}D_{r}=D\setminus
D_{f}.$ (5)

Here $X=_{d}Y$ means $X$ and $Y$ share the same distribution.

The definition of exact unlearning is quite restrictive and therefore can be
hard to achieve in practice. As a relaxation, Ginart et al. (2019) proposed
the following definition of _approximate unlearning_.

######  Definition 3.2 (Approximate $\delta$-Unlearning).

$(A,R_{A})$ satisfies $\delta$-unlearning if

$\forall D,~{}D_{f}\subset
D,~{}\text{and}~{}E\subseteq\mathbb{R}^{d},~{}~{}P\big{(}R_{A}(D,A(D),D_{f})\in
E\big{)}\leq\delta^{-1}P\big{(}A(D_{r})\in E\big{)}.$ (6)

These definitions are based on the assumption that the adversary can directly
access the model parameters $\theta$ and therefore the definitions are based
on the distribution over $\theta$. A more general assumption is that the
adversary can only access the model via an output function $O(\theta,D)$,
where $O:\Theta\times\mathcal{X}\longrightarrow\mathcal{O}$ and $\mathcal{X}$
denotes the space of input data. For instance, if $O(\theta,x)=f_{\theta}(x)$,
it means the adversary only has access to the agent’s prediction on any data
point $x$. In that case, we can modify the definitions by replacing
$R_{A}(\cdot)$ by $O\circ R_{A}(\cdot)$ and $A(\cdot)$ by $O\circ A(\cdot)$.

#### Remarks

  * •

MU depends on both $A$ and $R_{A}$. As $R_{A}$ highly depends on the learning
algorithm $A$, MU focuses on the design of both. But note that purely
achieving exact unlearning without considering the model’s performance is
meaningless. For example, one can achieve exact unlearning trivially if $A$
yields a constant mapping and $R_{A}$ is an identity mapping. Therefore, the
challenge is to maintain $R_{A}(D,A(D),D_{f})$’s performance on $D_{r}$, while
unlearning exactly.

  * •

MU differs from differential privacy (DP): $\epsilon$-DP does not divide the
data into $D_{f}$ and $D_{r}$ and requires that _no_ individual data point can
significantly influence the model’s prediction. But in MU (with exact
unlearning), it is required that any data point $x\in D_{f}$ has _zero_
influence on the model’s prediction after the unlearning, with no restrictions
on the effects of data $x\in D_{r}$.

  * •

$\delta$-Unlearning is asymmetric The above definitions implicitly assume that
$P(\theta\mid A(D_{r}))=0\Longrightarrow P(\theta\mid R_{A}(D,A(D),D_{f}))=0$.
However, we do not assume the converse, meaning that it is permissible for
$R_{A}$ to not generate models that could have been generated by $A(D_{r})$.

##  4 Problem and Method

We start this section with a formal introduction to the continual learning and
private unlearning (CLPU) problem. Then we introduce a straightforward
solution to CLPU that achieves exact unlearning by saving extra models, thus
sacrificing some space complexity compared to using a fixed-sized model
throughout learning.

###  4.1 CLPU: The Continual Learning and Private Unlearning problem

In Continual Learning and Private Unlearning (CLPU), an agent receives $T$
requests from the user sequentially and is asked to learn from a pool of $K$
tasks in total. The $t$-th request $\mathcal{R}^{t}$ is a tuple
$\mathcal{R}^{t}=(I^{t},D^{t},\rho^{t})$. Here, $I^{t}\in[K]$ is the task ID,
indicating the current task of interest. $D^{t}$ is either the training
dataset $\\{(x_{i},y_{i})\\}_{i=1}^{|D^{t}|}$ or an empty set $\emptyset$
depending on what $\rho^{t}$ is.
$\rho^{t}\in\\{\mathbf{R},\mathbf{T},\mathbf{F}\\}$ is a learning instruction:

  * •

$\rho^{t}=\mathbf{R}$: the user asks the agent to learn on task $i$
permanently.

  * •

$\rho^{t}=\mathbf{T}$: the user asks the agent to temporarily learn on task
$i$, which can be forgot in the future.

  * •

$\rho^{t}=\mathbf{F}$: the user asks the agent to forget task $i$ with exact
unlearning.

The agent keeps a dictionary $\Psi^{t}$ of the learned tasks’ statuses, which,
given $\mathcal{R}^{t}$, is updated by:

$\begin{cases}\Psi^{t}[I^{t}]=(D^{t},\rho^{t})&\text{if}~{}~{}\rho^{t}\in\\{\mathbf{R},\mathbf{T}\\}\\\
\Psi^{t}\leftarrow\Psi^{(t-1)}\setminus\\{I^{t}\\}&\text{if}~{}~{}\rho^{t}=\mathbf{F}.\end{cases}$
(7)

Here, $\Psi\setminus I$ indicates the removal of the key $I$ as well as its
corresponding values $(D,\rho)$ from $\Psi$. If $I^{t}\in\Psi^{(t-1)}$ and
$\rho^{t}=\mathbf{R}$, the agent is to fully memorize a task that has
previously been temporarily learned with instruction $\mathbf{T}$. In both
this case and the case when $\rho^{t}=\mathbf{F}$, we assume $D^{t}=\emptyset$
as there is no need for the user to provide the dataset for the same task
twice.

Now we are ready to present the formulation of CLPU. Denote all requests up to
the $(t-1)$-th request as
$\mathcal{R}^{<t}=[\mathcal{R}^{1},\mathcal{R}^{2},\dots,\mathcal{R}^{(t-1)}]$.222We
use list notation $[\mathcal{R}^{1},\mathcal{R}^{2},\dots]$ to indicate that
the ordering matters. A CLPU solution consists of a continual learning
algorithm $A$ and an unlearning algorithm $R_{A}$. Let $f_{\theta}$ be the
learning model parameterized by $\theta\in\mathbb{R}^{m}$ and $\theta^{t}$
denote the model parameter after processing the $t$-th request. Then both $A$
and $R_{A}$ map the previous model parameters and the current request to
updated model parameters. In particular, we have the following recursion:

$\begin{cases}\theta^{t}\sim
A(\theta^{(t-1)},\mathcal{R}^{t}=(I^{t},D^{t},\rho^{t}))&\text{if}~{}\rho^{t}\in\\{\mathbf{R,T}\\},\\\
\theta^{t}\sim
R_{A}(\theta^{(t-1)},\mathcal{R}^{t}=(I^{t},D^{t},\rho^{t}))&\text{if}~{}\rho^{t}=\mathbf{F}.\end{cases}$
(8)

For simplicity of notation, if all requests from $\mathcal{R}^{s}$ to
$\mathcal{R}^{t}$ ($\mathcal{R}^{s:t}$ for short), have
$\rho\in\\{\mathbf{R},\mathbf{T}\\}$, then we denote $\theta^{t}\sim
A(\theta^{s-1},\mathcal{R}^{s:t})$. Additionally, denote
$[\tau\in\Psi^{(t-1)}]$ as all tasks with $\rho\in\\{\mathbf{R,T}\\}$ that
have been observed and not removed up to the $(t-1)$-th request. The objective
of a CLPU agent when processing $\mathcal{R}^{t}$ is for $A$ and $R_{A}$ to
output $\theta^{t}$ with the following properties:

$\theta^{t}\sim\begin{cases}\begin{array}[]{l}\operatorname*{arg\,min}_{\theta}\mathbb{E}_{(x,y)\sim
D^{t}}\bigg{[}\ell(x,y,f_{\theta}\bigg{]}~{}~{}~{}\text{s.t}{}{}{}\\\
\qquad~{}~{}\forall\tau\in\Psi^{(t-1)},~{}\mathbb{E}_{(x,y)\sim
D^{\tau}}\bigg{[}\ell(x,y,f_{\theta})-\ell(x,y,f_{\theta^{(t-1)}})\bigg{]}\leq
0\end{array}&\text{if}~{}\rho^{t}\in\\{\mathbf{R,T}\\},\\\
R_{A}(\theta^{(t-1)},\mathcal{R}^{t})~{}~{}\text{s.t.}~{}~{}\mathcal{D}\bigg{(}R_{A}(\theta^{(t-1)},\mathcal{R}^{t})~{}~{}\big{|}\big{|}~{}~{}A(\theta^{0},\mathcal{R}^{[\tau\in\Psi^{(t-1)}]})\bigg{)}=0&\text{if}~{}\rho^{t}=\mathbf{F}.\end{cases}$
(9)

Here, $\mathcal{D}(A~{}||~{}B)$ is a distance between distributions $A$ and
$B$. In other words, in the first case when
$\rho^{t}\in\\{\mathbf{R},\mathbf{T}\\}$, the expected loss cannot get any
worse for any previously learned (but not forgotten) task. In the second case
when $\rho^{t}=\mathbf{F}$, then the unlearned model parameters cannot be
distinguished from the model parameters learned over the sequence of non-
forgetting tasks with the divergence $\mathcal{D}$. The CLPU problem setting
is illustrated in Fig. 1.

#### How does CLPU differ from CL and MU?

1) In CLPU, in addition to exhibiting knowledge transfer as in CL, the agent
also needs to unlearn specific tasks while maintaining all knowledge unrelated
to the forgotten tasks. 2) Unlike MU where generally the agent learns on
i.i.d. samples from the entire dataset $D$, in CLPU the agent learns online
over different tasks and hence the ordering of the sequence of tasks matters.
In addition, CLPU does not in general assume the agent can keep all previous
data, which makes the unlearning (or more specifically the retention of
learned knowledge) more difficult.

Figure 1: An illustration of the Continual Learning and private unlearning
(CLPU) problem setting. After the agent has temporarily learned on task $b$
with data $D^{b}$, if the agent is later requested to unlearn task $b$, the
unlearned model parameters $\theta^{4}$ should be indistinguishable from
$\theta^{2^{\prime}}$ in distribution as if the agent has never learned on
task $b$. Except for the unlearn requests, the agent should perform continual
learning over the remaining sequence of tasks.

###  4.2 CLPU-DER++: An Initial CLPU Algorithm

In this section, we present a straightforward method to CLPU, named CLPU-DER++
as it adapts the DER++ method (Buzzega et al., 2020) to the CLPU problem. The
CLPU-DER++ method achieves exact unlearning upon request and learns
continually over a sequence of tasks otherwise.

Inspired by Sharded, Isolated, Sliced, and Aggregated training (SISA)
(Bourtoule et al., 2019), for each task with $\rho=\mathbf{T}$, the CLPU-DER++
agent creates an isolated temporary network with parameters $\hat{\theta}$.
Based on the subsequent learning instruction for the same task, the agent
either removes this isolated model ($\mathbb{F}$) or merges it with the main
model ($\mathbb{R}$).

Specifically, we assume the agent maintains a main model with parameters
$\theta_{\text{main}}$ and a set $N$ of temporary models. In other words,
$\theta=\\{\theta_{\text{main}}\\}\cup N$. Upon the $t$-th request
$\mathcal{R}^{t}=(I^{t},D^{t},\rho^{t})$, there are four possible cases. 1) If
$\rho^{t}=\mathbf{R}$ and $I^{t}\notin\Psi^{(t-1)}$, then this is the first
time the agent has observed task $I^{t}$, so the agent then performs
conventional CL using Dark Experience Replay++ (DER++) (Buzzega et al., 2020)
and updates the main model parameters to $\theta^{t}_{\text{main}}$. 2) If
$\rho^{t}=\mathbf{T}$, in order to benefit from prior learning experience, the
agent initializes an isolated model with parameters $\hat{\theta}$ copied from
$\theta^{(t-1)}_{\text{main}}$, then directly performs SGD update on
$\hat{\theta}$ using the dataset $D^{t}$ and includes the updated network to
$N$ (e.g., $N\leftarrow N\cup\hat{\theta}^{I^{t}}$). In both cases, the agent
stores episodic memory $B^{I^{t}}$ (See Sec. 3.1) for the task $I^{t}$. 3) If
$\rho^{t}=\mathbf{R}$ and $I^{t}\in\Psi^{(t-1)}$, this means the agent has
previously learned on task $I^{t}$ with a temporary network
$\hat{\theta}^{I^{t}}$. Then the agent merges the knowledge learned in
$\hat{\theta}^{I^{t}}$ into $\theta^{t}_{\text{main}}$ to reduce space and
encourage knowledge transfer. To do so, CLPU-DER++ performs knowledge
distillation on the combined episodic memories from task $I^{t}$ and the rest
of the previously fully remembered tasks in $\Psi^{t}$. 4) If
$\rho^{t}=\mathbf{F}$, we simply remove the temporary network
$\hat{\theta}^{I^{t}}$ from $N$. The details of CLPU-DER++ are presented in
Alg. 1.

Algorithm 1 Continual Learning and private unlearning - Dark Experience
Replay++ (CLPU-DER++)

1: Input: Initial main model parameters $\theta^{0}_{\text{main}}$ and
temporary networks $N=\emptyset$, initial task status dictionary
$\Psi^{0}=\emptyset$, the total number of user requests $T$, and memory sizes
$\\{b^{t}\\}_{t=1}^{T}$.

2: for $t=1:T$ do

3: Receive request $\mathcal{R}^{t}=(I^{t},D^{t},\rho^{t})$.

4: Update $\Psi^{t}$ by
$\begin{cases}\Psi^{t}[I^{t}]=(D^{t},\rho^{t})&\text{if}~{}~{}\rho^{t}\in\\{\mathbf{R},\mathbf{T}\\}\\\
\Psi^{t}\leftarrow\Psi^{(t-1)}\setminus\\{I^{t}\\}&\text{if}~{}~{}\rho^{t}=\mathbf{F}.\end{cases}$

5: Case I: $\rho^{t}=\mathbf{R}~{}\textbf{and}~{}I^{t}\notin\Psi^{(t-1)}$

6: Perform $H$ steps of SGD from $\theta_{\text{main}}^{(t-1)}$ by optimizing:
$\theta^{t}_{\text{main}}=\operatorname*{arg\,min}_{\theta\in\mathbb{R}^{m}}~{}\frac{1}{|D^{t}|}\sum_{(x,y)\sim
D^{t}}\ell(x,y,f_{\theta})+\frac{1}{|\Psi^{(t-1)}|}\sum_{i\in\Psi^{(t-1)}}\frac{1}{|B^{i}|}\sum_{(x^{\prime},h^{\prime})\sim
B^{i}}\ell_{\text{mse}}(x^{\prime},h^{\prime},f_{\theta}).$

7: Build the episodic memory $B^{I^{t}}$:
$B^{I^{t}}=\big{\\{}\big{(}x_{i},y_{i},f_{\theta^{t}_{\text{main}}}(x_{i})\big{)}~{}~{}\big{|}~{}~{}(x_{i},y_{i})\stackrel{{\scriptstyle\text{i.i.d}}}{{\sim}}D^{t}\big{\\}}_{1\leq
i\leq b^{t}}$.

8: Case II: $\rho^{t}=\mathbf{T}$

9: Initialize $\hat{\theta}^{I^{t}}$ from $\theta^{(t-1)}_{\text{main}}$.

10: Perform $H$ steps of SGD on $\hat{\theta}^{I^{t}}$ by optimizing:
$\hat{\theta}^{I^{t}}=\operatorname*{arg\,min}_{\theta\in\mathbb{R}^{m}}~{}\frac{1}{|D^{t}|}\sum_{(x,y)\sim
D^{t}}\ell(x,y,f_{\theta})+\frac{1}{|\Psi^{(t-1)}|}\sum_{i\in\Psi^{(t-1)}}\frac{1}{|B^{i}|}\sum_{(x^{\prime},h^{\prime})\sim
B^{i}}\ell_{\text{mse}}(x^{\prime},h^{\prime},f_{\theta}).$

11: Store the temporary network: $N\leftarrow
N\cup\\{\hat{\theta}^{I^{t}}\\}$.

12: Build the episodic memory $B^{I^{t}}$:
$B^{I^{t}}=\big{\\{}\big{(}x_{i},y_{i},f_{\theta^{t}_{\text{main}}}(x_{i})\big{)}~{}~{}\big{|}~{}~{}(x_{i},y_{i})\stackrel{{\scriptstyle\text{i.i.d}}}{{\sim}}D^{t}\big{\\}}_{1\leq
i\leq b^{t}}$.

13: Case III: $\rho^{t}=\mathbf{R}$ and $I^{t}\in\Psi^{(t-1)}$

14: Merge $\hat{\theta}^{I^{t}}$ back to $\theta_{\text{main}}^{t}$ by
performing $H$ step of SGD and optimize:

15:
$\theta^{t}=\operatorname*{arg\,min}_{\theta\in\mathbb{R}^{m}}~{}\frac{1}{|B^{I^{t}}|}\sum_{(x,h)\sim
B^{I^{t}}}\ell_{\text{mse}}(x,h,f_{\theta})+\frac{1}{|\Psi^{t}|}\sum_{i\in\Psi^{t}}\frac{1}{|B^{i}|}\sum_{(x^{\prime},h^{\prime})\sim
B^{i}}\ell_{\text{mse}}(x^{\prime},h^{\prime},f_{\theta}).$

16: Remove the temporary network: $N=N\setminus\\{\hat{\theta}^{I^{t}}\\}$.

17: Case IV: $\rho^{t}=\mathbf{F}$

18: Remove the temporary network: $N=N\setminus\\{\hat{\theta}^{I^{t}}\\}$.

19: end for

#### Remark

CLPU-DER++ achieves _exact_ unlearning (See Def. 3.1) by construction. For any
task $k$ that the agent has learned previously and then attempts to unlearn,
the unlearn process only involves removing the relevant temporary model from
$N$ and the corresponding episodic memory: it does not influence the main
model parameter $\theta_{\text{main}}$. On the other hand, CLPU-DER++ achieves
privacy at the expense of memory, as it stores a full extra model for each
temporary learning task, which can be particularly important for large modern
neural architectures.

##  5 Experimental Results

In this section, we first introduce the experiment setup and introduce how we
form novel benchmarks by adapting conventional CL datasets for the CLPU
problem. Then we introduce the evaluation metrics designed for measuring the
agent’s performance in terms of both continual learning and private
unlearning. In the end, we present the evaluation results by comparing CLPU-
DER++ against the following baseline methods: sequential learning (Seq),
indepdent learning (Ind), Elastic Weight Consolidation (EWC) (Kirkpatrick et
al., 2017), Learning without Forgetting (LwF) (Li & Hoiem, 2017) , Experience
Replay (ER) (Chaudhry et al., 2019), Dark Experience Replay++ (DER++) (Buzzega
et al., 2020), and Learning with Selective Forgetting (LSF) (Shibata et al.,
2021). All the above baselines except LSF are state-of-the-art CL methods, but
we adapt some of them for the CLPU setting. In particular, for sequential
learning, the agent performs SGD directly over the sequence of tasks. For
independent learning, the agent creates a new model for each new task, and
removes a model if the user requests to unlearn the corresponding task. For ER
and DER++, for an unlearning task, we remove the corresponding episodic memory
and let the agent perform normal ER and DER++ updates on the remaining
episodic memories to accelerate forgetting.

###  5.1 CLPU Experiment Setup

We consider four conventional CL benchmarks: rotation MNIST (rot-MNIST),
permutation MNIST (perm-MNIST), split CIFAR-10 and split CIFAR-100. rot-MNIST
and perm-MNIST datasets are formed by rotating the images and randomly
permuting the pixels of the images, respectively, in the MNIST dataset. Each
task is a 10-class classification task. Split CIFAR-10 and split CIFAR-100 are
formed by treating the 10 classes in CIFAR-10 as five 2-class classification
tasks, and the 100 classes in CIFAR-100 as five 20-class classification tasks.
To be consistent, we build rot-MNIST and perm-MNIST also with 5 sequential
tasks. Then, to adapt these datasets to the CLPU setting, we form the
following sequence of requests:

$\mathcal{R}^{1:8}=\big{[}(1,D^{1},\bm{R}),~{}~{}(2,D^{2},\bm{T}),~{}~{}(3,D^{3},\bm{T}),~{}~{}(4,D^{4},\bm{R}),~{}~{}(1,\emptyset,\bm{R}),~{}~{}(2,\emptyset,\bm{F}),~{}~{}(5,D^{5},\bm{T}),~{}~{}(5,\emptyset,\bm{F})\big{]}.$

The corresponding sequence of requests that involve no unlearning is therefore

$\hat{\mathcal{R}}^{1:4}=\big{[}(1,D^{1},\bm{R}),~{}~{}(3,D^{3},\bm{T}),~{}~{}(4,D^{4},\bm{R}),~{}~{}(1,\emptyset,\bm{R})\big{]}.$

For all datasets, we use the SGD optimizer without momentum with $0.0005$
weight decay. For all datasets, the learning rate is set to $0.01$ and we
perform 10 epochs of training for each task. When the agent is asked to
unlearn a task, we also perform 10 epochs of the algorithm-specific unlearn
updates. The implementations of the baseline methods are adapted from the
open-source DER implementation.333DER code from
https://github.com/aimagelab/mammoth.

###  5.2 Evaluation Metrics

To evaluate a method on CLPU, we consider metrics both for continual learning
and for private unlearning. To measure the method’s performance on continual
learning, we report the final average accuracy (ACC) of the model over all
tasks that remain in the final task status dictionary $\Phi$, as well as the
forgetting measure (FM), which is the average drop in performance on each
task, compared to the model’s performance when the agent first learned these
tasks. Note that all evaluations are done on holdout testing data
$\\{D^{k}_{\text{test}}\\}$ for each task $k$. To be specific, denote
$a^{t}_{s}$ as the agent’s prediction accuracy on task $s$’s test dataset
$D^{s}_{\text{test}}$ after processing user’s $t$-th request, then we define

$\text{ACC}=\sum_{t=1}^{T}\sum_{s\in\Phi^{t}}a^{t}_{s}~{}~{}~{}~{}\text{and}~{}~{}~{}~{}\text{FM}=\sum_{t=1}^{T}\sum_{s\in\Phi^{t}}a^{\tau(s)}_{s}-a^{t}_{s},~{}~{}~{}~{}\text{where}~{}~{}\tau(s)=\operatorname*{arg\,min}_{t}~{}~{}(I^{t}=s).$
(10)

In short, ACC measures how well the agent performs on the tasks with
$\rho\in\\{\mathbf{R},\mathbf{T}\\}$ after processing all requests. FM
measures how much the agent forgets on the same set of tasks compared to when
they were first learned.

In addition to the above two metrics for evaluating the continual learning
performance, we also compare, on all tasks with $\rho=\mathbf{F}$, the
divergence between the model’s output distribution on that task after
unlearning versus the distribution that would have resulted had the agent not
learned the task. In other words, we use the output function
$O(\theta,x)=f_{\theta}(x)$ because comparing the distributions of
$f_{\theta}(\cdot)$ with different $\theta$s is more computationally efficient
than directly comparing the distributions of different $\theta$s. Concretely,
for all requests $\mathcal{R}^{t}$ such that $\rho^{t}=\mathbf{F}$, we measure
how different $f_{\theta^{t}}$ is from $f_{\theta^{t^{\prime}}}$, where
$\theta^{t}\sim A(\theta^{0},\mathcal{R}^{\leq t})$ and
$\theta^{t^{\prime}}\sim A(\theta^{0},\mathcal{R}^{[\tau\in\Psi^{t}]})$. To
measure the difference, we train $c$ models using different random seeds on
$\mathcal{R}^{\leq t}$ to get parameters
$\\{\theta^{t}_{1},\dots,\theta^{t}_{c}\\}$, and similarly get $c$ model
parameters $\\{\theta^{t^{\prime}}_{1},\dots,\theta^{t^{\prime}}_{c}\\}$ by
training on $\mathcal{R}^{[\tau\in\Psi^{t}]}$. After that, for each pair of
models that are trained on $\mathcal{R}^{[\tau\in\Psi^{t}]}$, we calculate the
in-group Jensen-Shannon (IJSD) distance between their outputs on the testing
dataset $D^{I^{t}}_{\text{test}}$.444We use Jensen-Shannon Distance because it
is a symmetric divergence for comparing probability distributions. Similarly,
for any model trained on $\mathcal{R}^{[\tau\in\Psi^{t}]}$ and any other model
trained on $\mathcal{R}^{\leq t}$, we also calculate their output
distributions’ Jenson-Shannon distance, which we call the Across-group Jensen-
Shannon Distance (AJSD). For the entire sequence of requests, we average over
the number of unlearning tasks for both IJSD and AJSD. Therefore, in total we
have $\frac{c(c-1)}{2}$ distances for IJSD and $c^{2}$ distances for AJSD,
over the entire sequence of requests. Formally,

$\begin{split}\text{IJSD}&=\bigg{\\{}\sum_{1\leq i<j\leq
c}\frac{1}{|\sum_{t\in[T],\rho^{t}=\mathbf{F}}1|}\sum_{t\in[T],\rho^{t}=\mathbf{F}}\mathbb{E}_{(x,y)\sim
D^{I^{t}}}\bigg{[}\mathrm{JS}\bigg{(}f(x;\theta^{t^{\prime}}_{i})~{}\bigg{|}\bigg{|}~{}f(x;\theta^{t^{\prime}}_{j})\bigg{)}\bigg{]}\bigg{\\}},\\\
\text{AJSD}&=\bigg{\\{}\sum_{i,j\in[c]}\frac{1}{|\sum_{t\in[T],\rho^{t}=\mathbf{F}}1|}\sum_{t\in[T],\rho^{t}=\mathbf{F}}\mathbb{E}_{(x,y)\sim
D^{I^{t}}}\bigg{[}\mathrm{JS}\bigg{(}f(x;\theta^{t}_{i})~{}\bigg{|}\bigg{|}~{}f(x;\theta^{t^{\prime}}_{j})\bigg{)}\bigg{]}\bigg{\\}}.\\\
\end{split}$ (11)

After the IJSDs and AJSDs are calculated, we measure the ratio of the absolute
difference between the average of IJSD and AJSD over the average of IJSD,
which we call JS-ratio, and the proportion of AJSD that are smaller than the
maximum of IJSD, which we call the In-Range Rate (IRR). Formally,

$\text{JS-
ratio}=\frac{\big{|}\frac{1}{|\text{IJSD}|}\sum_{d\in\text{IJSD}}d-\frac{1}{|\text{AJSD}|}\sum_{d\in\text{AJSD}}d\big{|}}{\frac{1}{|\text{IJSD}|}\sum_{d\in\text{IJSD}}d},~{}~{}~{}~{}\text{and}~{}~{}~{}~{}\text{IRR}=\frac{\sum_{d\in\text{AJSD}}\mathds{1}\big{(}d\leq\max(\text{IJSD})\big{)}}{\big{|}\text{AJSD}\big{|}}.$
(12)

###  5.3 Results

In Tab. 1, we report the comparison of CLPU-DER++ against the previously
mentioned baseline methods on 4 benchmark datasetes. Specifically, we report
the ACC, FM, JS-ratio and IRR metrics from previous section. To provide more
information, we also report the mean and standard deviation of the sets IJSD
and AJSD.555In Tab. 1, we abuse the notations of IJSD and AJSD a bit and
directly report the mean and standard deviation under them.

From the table, we can see that CLPU-DER++ achieves the best JS-ratio and IRR
among all methods. In contrast, all baseline methods achieve high JS-ratio and
very low IRR, meaning that the unlearning indeed reveals that the model has
learned on the unlearned task previously. On the other hand, in terms of the
CL metrics, DER++ achieves the best CL performance with CLPU-DER++ finishing a
close second. The difference is due to the fact that when merging the
temporary network back into the main model, the CLPU-DER++ agent essentially
performs knowledge distillation to distill the knowledge from a temporarily
learned task back to the main model. However, it is known that knowledge
distillation often cannot fully recover the original model’s performance.
Lastly, we observe that when creating a temporary network, initializing from
the main model (line 9 of Alg. 1) results in better performance compared to
initializing from scratch (CLPU-DER++ (scratch)).

| Perm-MNIST
---|---
Method | ACC($\uparrow$) | FM($\downarrow$) | IJSD | AJSD | JS-ratio($\downarrow$) | IRR($\uparrow$)
Seq | 75.75 $\pm$ 2.44 | 19.97 $\pm$ 2.45 | 0.17 $\pm$ 0.05 | 0.92 $\pm$ 0.03 | 4.47 | 0.00
EWC | 93.67 $\pm$ 0.25 | 0.45 $\pm$ 0.18 | 0.04 $\pm$ 0.01 | 0.65 $\pm$ 0.01 | 13.73 | 0.00
ER | 91.83 $\pm$ 0.25 | 3.96 $\pm$ 0.24 | 0.11 $\pm$ 0.02 | 0.73 $\pm$ 0.01 | 5.92 | 0.00
LwF | 79.09 $\pm$ 3.19 | 17.12 $\pm$ 3.23 | 0.08 $\pm$ 0.02 | 0.83 $\pm$ 0.03 | 9.41 | 0.00
LSF | 91.18 $\pm$ 0.24 |  0.44 $\pm$ 0.08 | 0.06 $\pm$ 0.01 | 0.49 $\pm$ 0.02 | 7.49 | 0.00
DER++ |  93.88 $\pm$ 0.14 | 2.01 $\pm$ 0.14 | 0.07 $\pm$ 0.01 | 0.66 $\pm$ 0.01 | 8.82 | 0.00
CLPU-DER++ (scratch) | 93.26 $\pm$ 0.25 | 2.33 $\pm$ 0.21 | 0.10 $\pm$ 0.01 | 0.09 $\pm$ 0.02 | 0.09 | 1.00
CLPU-DER++ | 93.48 $\pm$ 0.25 | 2.25 $\pm$ 0.30 | 0.10 $\pm$ 0.01 | 0.09 $\pm$ 0.02 | 0.13 | 0.96
| Rot-MNIST
Method | ACC($\uparrow$) | FM($\downarrow$) | IJSD | AJSD | JS-ratio($\downarrow$) | IRR($\uparrow$)
Seq | 90.88 $\pm$ 0.43 | 5.66 $\pm$ 0.38 | 0.14 $\pm$ 0.02 | 0.80 $\pm$ 0.02 | 4.88 | 0.00
EWC | 94.75 $\pm$ 0.12 |  0.29 $\pm$ 0.12 | 0.09 $\pm$ 0.01 | 0.72 $\pm$ 0.01 | 7.50 | 0.00
ER | 95.12 $\pm$ 0.18 | 1.39 $\pm$ 0.17 | 0.16 $\pm$ 0.02 | 0.79 $\pm$ 0.02 | 3.87 | 0.00
LwF | 95.72 $\pm$ 0.19 | 0.87 $\pm$ 0.18 | 0.07 $\pm$ 0.01 | 0.76 $\pm$ 0.01 | 9.60 | 0.00
LSF | 92.56 $\pm$ 0.09 | 0.30 $\pm$ 0.07 | 0.08 $\pm$ 0.01 | 0.65 $\pm$ 0.02 | 6.94 | 0.00
DER++ |  95.94 $\pm$ 0.09 | 0.36 $\pm$ 0.08 | 0.12 $\pm$ 0.02 | 0.74 $\pm$ 0.02 | 5.38 | 0.00
CLPU-DER++ (scratch) | 94.69 $\pm$ 0.11 | 1.02 $\pm$ 0.10 | 0.13 $\pm$ 0.02 | 0.11 $\pm$ 0.04 | 0.14 | 1.00
CLPU-DER++ | 95.37 $\pm$ 0.12 | 0.91 $\pm$ 0.09 | 0.14 $\pm$ 0.02 | 0.11 $\pm$ 0.04 | 0.17 | 1.00
| Split-CIFAR10
Method | ACC($\uparrow$) | FM($\downarrow$) | IJSD | AJSD | JS-ratio($\downarrow$) | IRR($\uparrow$)
Seq | 76.73 $\pm$ 2.25 | 15.38 $\pm$ 2.67 | 0.09 $\pm$ 0.03 | 0.18 $\pm$ 0.02 | 0.95 | 0.04
EWC | 79.43 $\pm$ 1.51 | 11.83 $\pm$ 2.08 | 0.14 $\pm$ 0.05 | 0.20 $\pm$ 0.03 | 0.40 | 0.76
ER | 90.44 $\pm$ 0.57 | 0.83 $\pm$ 3.36 | 0.07 $\pm$ 0.02 | 0.16 $\pm$ 0.01 | 1.07 | 0.00
LwF | 88.18 $\pm$ 1.87 | 4.34 $\pm$ 2.34 | 0.06 $\pm$ 0.02 | 0.16 $\pm$ 0.03 | 1.68 | 0.00
LSF | 90.00 $\pm$ 2.30 | 2.48 $\pm$ 2.05 | 0.07 $\pm$ 0.02 | 0.14 $\pm$ 0.03 | 0.90 | 0.48
DER++ |  91.26 $\pm$ 0.63 |  0.73 $\pm$ 3.54 | 0.03 $\pm$ 0.01 | 0.07 $\pm$ 0.01 | 1.11 | 0.60
CLPU-DER++ (scratch) | 89.52 $\pm$ 1.46 | 1.14 $\pm$ 2.17 | 0.03 $\pm$ 0.01 | 0.03 $\pm$ 0.02 | 0.04 | 0.92
CLPU-DER++ | 90.12 $\pm$ 1.65 | 1.89 $\pm$ 2.20 | 0.03 $\pm$ 0.01 | 0.03 $\pm$ 0.01 | 0.00 | 0.92
| Split-CIFAR100
Method | ACC($\uparrow$) | FM($\downarrow$) | IJSD | AJSD | JS-ratio($\downarrow$) | IRR($\uparrow$)
Seq | 44.34 $\pm$ 0.84 | 24.36 $\pm$ 2.44 | 0.44 $\pm$ 0.03 | 1.09 $\pm$ 0.03 | 1.47 | 0.00
EWC | 45.39 $\pm$ 1.74 | 20.08 $\pm$ 1.42 | 0.63 $\pm$ 0.03 | 1.27 $\pm$ 0.04 | 1.02 | 0.00
ER | 61.66 $\pm$ 1.27 | 7.69 $\pm$ 1.68 | 0.51 $\pm$ 0.03 | 1.11 $\pm$ 0.03 | 1.18 | 0.00
LwF | 61.25 $\pm$ 2.73 | 8.60 $\pm$ 1.01 | 0.39 $\pm$ 0.03 | 1.06 $\pm$ 0.03 | 1.71 | 0.00
LSF | 37.92 $\pm$ 2.14 | 26.88 $\pm$ 2.09 | 0.70 $\pm$ 0.03 | 1.09 $\pm$ 0.05 | 0.54 | 0.00
DER++ |  66.66 $\pm$ 0.69 |  2.84 $\pm$ 0.59 | 0.31 $\pm$ 0.03 | 0.70 $\pm$ 0.02 | 1.24 | 0.00
CLPU-DER++ (scratch) | 61.51 $\pm$ 0.76 | 3.46 $\pm$ 1.18 | 0.21 $\pm$ 0.01 | 0.19 $\pm$ 0.03 | 0.08 | 0.96
CLPU-DER++ | 63.90 $\pm$ 0.77 | 3.90 $\pm$ 1.05 | 0.22 $\pm$ 0.01 | 0.21 $\pm$ 0.04 | 0.08 | 0.96
Table 1: Performance of CLPU-DER++ against baseline methods on four CLPU
benchmarks. We report the mean and standard deviation for each result over 5
independent runs. The best results for each metric are bolded.

##  6 Conclusion and Future Work

In this work, we propose a novel continual learning and private unlearning
(CLPU) problem and provide its formal formulation. In addition, we introduce a
straightforward but exact unlearning method to solve CLPU, as well as novel
metrics and adapted benchmark problems to evaluate any CLPU methods. An
interesting future direction is to extend the current method to approximate
the $\delta$-unlearning setting while reducing the space complexity by saving
fewer models.

## References

  * Aljundi et al. (2019) Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars.  Task-free continual learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 11254–11263, 2019.
  * Biesialska et al. (2020) Magdalena Biesialska, Katarzyna Biesialska, and Marta R Costa-jussà.  Continual lifelong learning in natural language processing: A survey.  _arXiv preprint arXiv:2012.09823_ , 2020.
  * Bjork & Bjork (2019) Robert A Bjork and Elizabeth L Bjork.  Forgetting as the friend of learning: implications for teaching and self-regulated learning.  _Advances in Physiology Education_ , 43(2):164–167, 2019.
  * Bourtoule et al. (2019) Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot.  Machine unlearning.  _arXiv preprint arXiv:1912.03817_ , 2019.
  * Brophy & Lowd (2021) Jonathan Brophy and Daniel Lowd.  Machine unlearning for random forests.  In _International Conference on Machine Learning_ , pp. 1092–1104. PMLR, 2021.
  * Buzzega et al. (2020) Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara.  Dark experience for general continual learning: a strong, simple baseline.  _Advances in neural information processing systems_ , 33:15920–15930, 2020.
  * Cao & Yang (2015) Yinzhi Cao and Junfeng Yang.  Towards making systems forget with machine unlearning.  In _2015 IEEE Symposium on Security and Privacy_ , pp. 463–480. IEEE, 2015.
  * Chaudhry et al. (2018a) Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr.  Riemannian walk for incremental learning: Understanding forgetting and intransigence.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 532–547, 2018a.
  * Chaudhry et al. (2018b) Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny.  Efficient lifelong learning with a-gem.  _arXiv preprint arXiv:1812.00420_ , 2018b.
  * Chaudhry et al. (2019) Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc’Aurelio Ranzato.  On tiny episodic memories in continual learning.  _arXiv preprint arXiv:1902.10486_ , 2019.
  * Delange et al. (2021) Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars.  A continual learning survey: Defying forgetting in classification tasks.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2021.
  * Ginart et al. (2019) Antonio Ginart, Melody Y Guan, Gregory Valiant, and James Zou.  Making ai forget you: Data deletion in machine learning.  _arXiv preprint arXiv:1907.05012_ , 2019.
  * Golatkar et al. (2020) Aditya Golatkar, Alessandro Achille, and Stefano Soatto.  Eternal sunshine of the spotless net: Selective forgetting in deep networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 9304–9312, 2020.
  * Golatkar et al. (2021) Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto.  Mixed-privacy forgetting in deep networks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 792–801, 2021.
  * Guo et al. (2019) Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten.  Certified data removal from machine learning models.  _arXiv preprint arXiv:1911.03030_ , 2019.
  * Hung et al. (2019a) Ching-Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu-Song Chen.  Compacting, picking and growing for unforgetting continual learning.  _Advances in Neural Information Processing Systems_ , 32, 2019a.
  * Hung et al. (2019b) Steven CY Hung, Jia-Hong Lee, Timmy ST Wan, Chein-Hung Chen, Yi-Ming Chan, and Chu-Song Chen.  Increasingly packing multiple facial-informatics modules in a unified deep-learning model via lifelong learning.  In _Proceedings of the 2019 on International Conference on Multimedia Retrieval_ , pp. 339–343, 2019b.
  * Kirkpatrick et al. (2017) James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al.  Overcoming catastrophic forgetting in neural networks.  _Proceedings of the national academy of sciences_ , 114(13):3521–3526, 2017.
  * Li & Hoiem (2017) Zhizhong Li and Derek Hoiem.  Learning without forgetting.  _IEEE transactions on pattern analysis and machine intelligence_ , 40(12):2935–2947, 2017.
  * Liu et al. (2021) Bo Liu, Xuesu Xiao, and Peter Stone.  A lifelong learning approach to mobile robot navigation.  _IEEE Robotics and Automation Letters_ , 6(2):1090–1096, 2021.
  * Lopez-Paz & Ranzato (2017) David Lopez-Paz and Marc’Aurelio Ranzato.  Gradient episodic memory for continual learning.  _Advances in neural information processing systems_ , 30, 2017.
  * Mallya & Lazebnik (2018) Arun Mallya and Svetlana Lazebnik.  Packnet: Adding multiple tasks to a single network by iterative pruning.  In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ , pp. 7765–7773, 2018.
  * Mallya et al. (2018) Arun Mallya, Dillon Davis, and Svetlana Lazebnik.  Piggyback: Adapting a single network to multiple tasks by learning to mask weights.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 67–82, 2018.
  * Riemer et al. (2018) Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro.  Learning to learn without forgetting by maximizing transfer and minimizing interference.  _arXiv preprint arXiv:1810.11910_ , 2018.
  * Rosenfeld & Tsotsos (2018) Amir Rosenfeld and John K Tsotsos.  Incremental learning through deep adaptation.  _IEEE transactions on pattern analysis and machine intelligence_ , 42(3):651–663, 2018.
  * Rusu et al. (2016) Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell.  Progressive neural networks.  _arXiv preprint arXiv:1606.04671_ , 2016.
  * Schwarz et al. (2018) Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell.  Progress & compress: A scalable framework for continual learning.  In _International Conference on Machine Learning_ , pp. 4528–4537. PMLR, 2018.
  * Shibata et al. (2021) Takashi Shibata, Go Irie, Daiki Ikami, and Yu Mitsuzumi.  Learning with selective forgetting.  In _IJCAI_ , volume 2, pp. 6, 2021.
  * Shin et al. (2017) Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim.  Continual learning with deep generative replay.  _Advances in neural information processing systems_ , 30, 2017.
  * Van de Ven & Tolias (2019) Gido M Van de Ven and Andreas S Tolias.  Three scenarios for continual learning.  _arXiv preprint arXiv:1904.07734_ , 2019.
  * Wu et al. (2020a) Lemeng Wu, Bo Liu, Peter Stone, and Qiang Liu.  Firefly neural architecture descent: a general approach for growing neural networks.  _Advances in Neural Information Processing Systems_ , 33:22373–22383, 2020a.
  * Wu et al. (2020b) Yinjun Wu, Edgar Dobriban, and Susan Davidson.  Deltagrad: Rapid retraining of machine learning models.  In _International Conference on Machine Learning_ , pp. 10355–10366. PMLR, 2020b.
  * Yoon et al. (2017) Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang.  Lifelong learning with dynamically expandable networks.  _arXiv preprint arXiv:1708.01547_ , 2017.


# A Novel Approach on Dielectric Barrier Discharge using Printed Circuit
Boards

Jonathan Gail, Alisa Schmidt, Markus H. Thoma All authors were with the First
Physical Institute, Justus Liebig University, 35392 Giessen, Germany, e-mail:
<EMAIL_ADDRESS>
###### Abstract

Drug resistant bacteria, prions and nosocomial infections underline the need
of more effective sterilizing technologies. The cold plasma technology is
expected to bring a benefit in this context [1]. Six different plasma sources,
based on printed circuit boards, were evaluated fourfold. This include
measurements of the power consumption, the ignition behavior by an ICCD-camera
and ozone formation by absorption spectroscopy at 254 nm. To evaluate the
biocidal effect, four bacterial test series were performed with Escherichia
coli. The entirety of the tests analyze the plasma inactivation process from
the input parameters to the desired biocidal effect.
The discharge current and time resolved ignition behaviors indicated a
simultaneous formation of filaments at the beginning of the negative half-
cycle. The dynamics of the ozone production showed a saturated exponential
growth upon a maximum value of 435 ppm. Additionally, the microbiological test
series unveiled differences between the plasma source concepts. A total
reduction rate of $\mathbf{10^{-4}}$ within a minute was achievable. An air
flow through slits within the plasma sources destabilized the plasma.
Minor changes of the electrode geometry changed all measured parameters.
Hence, to develop a pathogen inactivating plasma source, these results
recommend a comb-shaped electrode design, which is laminated on a dielectric.

###### Index Terms:

cold atmospheric plasma, power density, absorption spectroscopy, ozone, time
resolved ignition behavior, inactivation rate

##  I Introduction

The amount of nosocomial infections [2] and the rising number of resistant
bacteria due to the application of antibiotics [3], emphasize the importance
of preventive hygiene in the medical sector. Since there are pathogens which
develop a resistance to alcohol based sterilants [4] or need a longer exposure
to heat than it is used in the daily business [5], the technology has to take
a step towards more effective sterilization methods. One promising focus of
recent research is the plasma sterilization [6, 7].
Plasma consists of electrons, ions, neutral gas and several reactive
molecules. When working with the plasma afterglow of a cold atmospheric plasma
(CAP) the charged particles recombine before they can reach a surface by
diffusion. Hence, long-living reactive oxygen and nitrogen species (RONS) are
considered to cause the main inactivating effect [8, 9]. Low temperature, high
inactivation rate and absence of plasma resistant pathogens is only a
selection of the many advantages of the plasma technology. Despite that and
the current research, there is still a lack of plasma sterilization
applications especially in the medical sector. The low temperature of CAP
enables the treatment of thermolabile surfaces like endoscopes, pharmaceutical
goods or dental instruments [10, 11]. Their reprocessing is very complicated
and susceptible to failures, which lead to a high rate of contaminated devices
[12]. This emphasizes the urgent development of new applications.
In this article six new designs of CAP sources are considered to improve the
inactivation capability and manufacturability. Those are based on dielectric
barrier discharge, which uses a dielectric to limit the discharge current and
to distribute the charges over the surface. A specific setup is the surface
micro discharge, where both electrodes (one as a grid) are attached to the
dielectric and the plasma is generated within the gaps of the grid [13]. The
power consumption drops from volume-, surface- to coplanar surface-DBDs, which
have coplanar electrodes inside of the dielectric [14]. Adapting the coplanar
surface-DBD technology, the sources are based on a thin printed circuit boards
(PCBs) with comb-shaped electrodes [15]. To measure the effects of the
different plasma sources, they become fourfold characterized by their
electrical power consumption, time resolved ignition behavior, ozone
production and inactivation capability. Two of six sources use an air-flow of
$1\text{\,}\mathrm{m}^{3}\mathrm{/}\mathrm{m}\mathrm{i}\mathrm{n}$ to improve
the distribution of the RONS.

##  II Experimental setup

The plasma sources were the result of a long development process, which was
focused on a scalable and PCB-based design. As a substrate the standard Flame
Retardant FR-4 material, which is a dielectric made of an epoxy resin and a
fiberglass inlay, is used. Copper layers are laminated on both sides, which
can be etched in every desirable shape. Thus, the advantages are fast
prototyping of different electrode geometries and the absence of air between
the electrode and the dielectric. A drawback is that the FR-4 does not have a
good ozone resistance. Additionally, the dielectric has a thickness of
$0.5\text{\,}\mathrm{m}\mathrm{m}$ and a dielectric strength of
$20\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{m}\mathrm{m}$, which results
in a maximum supply voltage of $20\text{\,}\mathrm{k}\mathrm{V}$ peak-to-peak
(P2P)[16]. Due to these conditions, the shelf life of these plasma sources was
about 15 h. Yet, the improvements of the inactivation capability and the
reduction of the power consumption underline the relevance of the plasma
research.
The voltage was supplied from a high voltage amplifier (Trek 10/40A-HS), which
used the initial sine signal from a signal-generator (Rhode & Schwarz HM8150)
of $3\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$. The signal was amplified
1000-times up to a maximum value of $10\text{\,}\mathrm{k}\mathrm{V}$.
Several tests during the development process, including the evaluation of the
discharge behavior and antibacterial capability, lead to six different plasma
sources, which are displayed in figure 1.

Figure 1: Six different dielectric- and electrode geometries, violet: plasma,
orange: copper electrode (lower electrode was grounded), black: FR-4, brown:
epoxy resin, red: area of a potential breakdown, due to the slits, geometries
(3) and (4) have spaces with only a small isolating epoxy part.

Since the diffusion of the reactive species to the contaminated surface is an
inefficient process, an airflow has been proposed to have three advantages.
First, the reactive species will reach the surface faster. Second, more
reactive species will be directed towards the substrate and third, the plasma
components with a short lifetime will have a contribution to the biocidal
effect, because they reach the surface before dissociation.
The six plasma sources can be grouped in three conceptual geometry designs.
The first has displaced electrodes, like (1 & 3). The geometries (2) and (4)
have parallel electrodes, which lead to a more uniform electric field and more
homogeneous plasma. As figure 1 shows, the area of the plasma glow is
restricted to the edges of the electrodes. Combining the advantages of both
geometries, the plasma concept of geometry (5 & 6) has displaced electrodes
without a horizontal distance between the electrodes. The high voltage
electrode was wider than the grounded electrode. Hence, the electrons can
accumulate over the dielectric and the plasma glow is larger than in case of
(2 & 4).

###  II-A Electrical Power Consumption

The electrical measurements focus on the variation of the discharge current
and its phase to the voltage. The oscilloscope (Tektronix DPO 2012B) acquired
the voltage by the analog output of the high voltage amplifier and the current
with a Rogowski coil, which enables the evaluation of the filaments [17]. From
this point the analysis of the surface power density, power, reactive power,
resistivity and capacitance was possible. The root mean square data of the
voltage (RMS), current (RMS) and phase were averaged over 125 000 cycles.
Additionally, the surface temperatures of the plasma sources were measured to
verify the positive dependency on the surface power density. Based on the
findings of the electrical characterization, the operating voltage was chosen
for later measurements. The operating frequency was constantly
$3\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$. Since the high voltage amplifier
has a power limit of $400\text{\,}\mathrm{W}$ and the impedance was influenced
by $1/(\omega C)$, this frequency minimizes the need of the reactive power and
thereby, enlarges the range of possible voltages, while the generated plasma
was still homogeneously.

###  II-B Time Resolved Ignition Behavior

Analyzing the ignition behavior of the filaments makes a very high resolution
of the time necessary. This can be achieved by triggering and shifting the
start of the frames. The photons were recorded by the ICCD-camera PI-MAX 4
from Princeton Instruments with an intensifying factor of 100, which was
needed because of the weak plasma glow.
To evaluate 100 frames of one sine period with a frequency of
$3\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$, it was necessary to have a gate
width of $3333\text{\,}\mathrm{n}\mathrm{s}$. In every period one frame was
recorded. The gate delay was constantly increasing by
$3333\text{\,}\mathrm{n}\mathrm{s}$ after each frame. Hence, every frame can
be mapped to a defined position within one period, which was fully recorded
after 100 frames. This enables the analysis of filaments and their formation.

###  II-C Ozone Production

Besides the other reactive species, ozone was expected to have a major
influence on the inactivation performance [18, 19]. Therefore, the ozone
concentration was measured for every plasma source at every possible
operational voltage by absorption spectroscopy. This revealed the dependencies
on power consumption and geometry. The measurement was made within a closed
volume of
$5\text{\,}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{s}$.
The principle of the absorption spectroscopy uses UV-light with a wavelength
of $254\text{\,}\mathrm{n}\mathrm{m}$. A deuterium lamp (D-2000, Mikropack)
generates a wide UV spectrum. The intensity at the wavelength of
$254\text{\,}\mathrm{n}\mathrm{m}$ was measured by a spectrometer (OceanOptics
HR-4000). This wavelength is in the middle of the Hartley-band of ozone, which
has a high photo-absorption cross section of $\sigma_{\lambda}=1.15\cdot
10^{-21}$\text{\,}\mathrm{m}^{2}$$ and the other molecules do not interfere
there [20]. Hence, only this wavelength becomes absorbed by the ozone in the
plasma afterglow. For this purpose, the experimental setup has an absorption
path, with a length of $l=$20\text{\,}\mathrm{m}\mathrm{m}$$. This path was
placed $18\text{\,}\mathrm{m}\mathrm{m}$ above the plasma source. Within this
defined length, the UV-light is send through the ozone containing plasma
afterglow. At the end of the optical measurement setup, the intensity was
monitored before the plasma ignition, during the operation of the plasma
source and after the plasma was extinquished. This gives a typical U-shaped
sequence, which enables the comparison of the initial reference intensity
($I_{0}$) and the intensity during plasma operation ($I_{1}$).
Based on the law of Beer-Lambert, this intensity ratio ($I_{0}/I_{1}$) can be
correlated to the absorption events / ozone concentration ($c$).

$\log_{10}(I_{0}/I_{1})=\sigma_{\lambda}\cdot c\cdot l$ (1)

The plasma was ignited for minimum 15 minutes to evaluate the dependencies of
the ozone dynamics on the heating and quenching effect. The quenching effect
is the production of nitrogen species instead of ozone [18]. After
extinguishing the plasma, the ongoing measurement proofs if the reference
intensity was reached after the dissociation of the ozone.
To achieve a good signal to noise ratio, an integration time of
$400\text{\,}\mathrm{m}\mathrm{s}$ for each data point was used in every
measurement. Also an electric dark correction was applied to reduce the offset
of the sensor.

###  II-D Inactivation Capability

Based on the measurements of the electrical characterization, the operational
voltage was chosen respectively to the power surface density. The power
density of the four sources (1) to (4) was set to
$0.427\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$, because with
these settings, (2) produces the highest ozone concentration. Due to the
restricted operational voltage of $6\text{\,}\mathrm{k}\mathrm{V}$ for
geometry (6), the power density of the third plasma source concept (5 & 6) was
set to $0.288\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$. For the
evaluation of the inactivation performance, a micro-biological study on non-
pathogenic strain K-12 of Escherichia coli (E. coli) was performed. To prepare
the experiment, the bacteria were transferred from an agar plate to
$10\text{\,}\mathrm{m}\mathrm{l}$ of the nutrient medium lysogeny broth (LB)
via inoculation loop. After mixing on a test tube shaker (Genie Vortex 1), the
solution was incubated at $37\text{\,}\mathrm{\SIUnitSymbolDegree}\mathrm{C}$
for $24\text{\,}\mathrm{h}$. Then, the optical density (OD600) was set to 1 by
diluting the culture with pure LB medium. The OD is the relative optical
transmission of the bacteria culture to the pure medium at a wavelength of
$600\text{\,}\mathrm{n}\mathrm{m}$.
The resulting solution was diluted down by $10^{-4}$ for the exposition to
plasma and down by $10^{-6}$ for the control measurement. Four series of tests
were applied to every plasma source. The bacteria were spread over the agar
plates with a Drigalski cell spreader. Each test series contained of 108 agar
plates with 18 per plasma source. This number contains of 2 dilutions at 3
exposure times with 3 agar plates per case.
For the exposure, the plasma source was placed onto a frame
$2\text{\,}\mathrm{c}\mathrm{m}$ above the petri dish. The resulting volume
around the petri dish is $0.2\text{\,}\mathrm{l}$, where ozone saturation was
reached after $3\text{\,}\mathrm{s}$. Hence, the effect of an increasing
concentration during the exposition can be neglected.
Every colony forming unit (CFU) was counted manually after
$24\text{\,}\mathrm{h}$ and $48\text{\,}\mathrm{h}$ of incubation. This
enables the differentiation of fast growing colonies in the first step and
identification of slow growing colonies in the second step. However, the
relative difference of both measurements has never exceeded $10\text{\,}\%$.

##  III Results

###  III-A Electrical Power Consumption

The operational voltage was set as a P2P-value at the sine generator. With the
voltage, current and phase, the active power, capacitance and resistance were
derived. A frequency of $3\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$ enabled a
homogenous plasma glow and reduced the reactive power, which limited the
combinations of available voltages and frequencies.
Since the current was defined by the plasma source, only voltage and frequency
were the input parameters. Therefore, the power was proportional to the square
of the voltage, which can be verified for every plasma source.
The free electrons in the plasma influenced the capacity and resistance, which
were evaluated. Additionally, the ratio of the active power to the surface
gives the surface (active) power density, which can be chosen from
$0.05\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ to
$0.57\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ with respect to
the plasma source concept. The surface power density was expected to be a
predictor of the ozone concentration [18, 19].
Increasing supply voltages lead to a higher capacity and lower resistance,
because the number of free electrons increased. Except in case of plasma
source (5), which shows an increasing resist. This could be correlated to the
curved filaments, which will be mentioned in the following. Due to the
resistive heating, the temperature of the plasma sources reached up to
$64\text{\,}\mathrm{\SIUnitSymbolDegree}\mathrm{C}$ with increasing supply
voltages.
The shape of the measured discharge current reveals the dynamic of the
filaments and was associated to the frames of the time resolved ignition
analysis. Figure 2 shows the operational voltage (sine) with
$14\text{\,}\mathrm{k}\mathrm{V}$ P2P at
$3\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$ and the discharge current (signal
with spikes). Its course was representative for all plasma sources and reveals
two plasma ignitions during one period of the voltage. Increasing the voltage,
causes stronger discharges, which unveil in higher currents. The spikes
indicate the formation of filaments in the plasma [21]. Even if the negative
current has a stronger gradient and spikes, it was not possible to clarify, if
it was caused by stronger and/or more filaments. Therefore, images of the time
resolved ignition behavior with the ICCD-camera were made.

Figure 2: Operational voltage (sine signal) and discharge current (signal with
spikes) of plasma source (1) at $3\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$ and
$14\text{\,}\mathrm{k}\mathrm{V}$; averaged over 4 (left) and 32 (right)
cycles

Due to the high variation of spikes, the data has to be averaged for the
trigger. Figure 2 compares two different averages of the same object and
operational parameters. The strong smoothing effect of an average over 32
cycles was recognizable. This reveals that the spikes were statistically
distributed and can be removed by averaging. The high gradient of the negative
current emerges clearer with a stronger averaging. It was also visible, that
the discharge starts close to the zero crossings.
The feed through led to higher discharge currents, especially during the
ignition phase of the plasma. Since the air flow did not influence the
discharge current at all, differences of the power consumption were a result
of the geometries. Even though the visible inspection shows a destabilization
of the plasma, which reduces the number of filaments, when the airflow was
applied, this was not recognizable in the measured currents.

###  III-B Time Resolved Ignition Behavior

The results were 1024 $\times$ 1024 matrices with an intensity value for each
pixel. These data were plotted as false-color images. Black areas indicate no
emissions from the plasma and violett to white areas represent the detection
of photons. Those were caused by the emission of the plasma glow.
Figures 3 and 4 show the most relevant frames of the plasma sources in the
negative half-cycle, which show the filaments and their systematical
distinctions in dependency of the geometry. Other frames and results can be
requested from the authors. Due to the distance between the electrodes, the
plasma of geometry (1) forms arc-like filaments. In the left part of figure 3
several ionization events unite to one streamer towards the grounded
electrode. Another frame (not shown here) shows the opposite direction of the
electrons. The electrons accumulate above the dielectric covered high voltage
electrode.
Figure 3 shows the simultaneous discharge of the filaments directly after the
zero-crossing of the voltage. This clarifies, that the higher discharge
current in the negative half-cycle was caused by more coincident filaments.
This behavior can be verified for every plasma source. The residual charges on
the dielectric surface were responsible for that, because they increased the
electric field. Additionally, this caused filaments slightly before the zero-
crossing.
Characteristic for concept (2) were the $1\text{\,}\mathrm{m}\mathrm{m}$ wide
areas at the edges of the electrodes, where the plasma ignites. There were
only few filaments, which reach up to the dark space between the small lines
of the plasma glow.
One of the most interesting frames were displayed in the right part of figure
4 for geometry (3). The ignition behavior of this plasma source was
remarkable. At first, the plasma ignites above the dielectric surface and
higher voltages lead to a plasma formation within the slits in the dielectric.
The mentioned frames unveil, that this behavior was a result of electron
accumulation at the sides of the slits. These edges act like virtual
electrodes. After increasing the voltage, the electrons reach to the opposite
edge of the slits and form a plasma glow (displayed). The air flow through the
slits lead to a destabilization of the plasma. Figure 5 show the plasma source
(3) with and without an applied airflow, which causes a weaker plasma and
spaces without a discharge. It is proposed, that the streamer was deformed and
extinguished.

Figure 3: False color images of geometries (1) and (2) within the negative
half-cycle, ground electrode in orange, high voltage electrode in red (left),
grounded electrode is on top of the high-voltage electrode (right)

Figure 4: False color images of geometries (3) and (5) within the negative
half-cycle, ground electrode in orange, high voltage electrode in red (left)
and whole space between the ground electrodes (right)

Figure 5: Destabilizing effect at geometry (3) with (left) and without (right)
the applied air flow

Since the positions of the filaments were the same, the air flow does not
affect the ignition behavior of geometry (4), because its filaments does not
span over the slits. Based on the first two plasma sources, the electrode
concept (5 & 6) shows the desired results. Since the edges of the high voltage
and grounded electrodes do not have a horizontal distance, the plasma ignites
as homogeneously as plasma source (2). The $1.5\text{\,}\mathrm{m}\mathrm{m}$
wide dielectric covered electrodes spread the plasma uniformly over the entire
dielectric surface. The length of the filaments in the right part of figure 4
indicate, that they were longer than the half of the high voltage electrode
width. Increasing this width to $3\text{\,}\mathrm{m}\mathrm{m}$ lead to the
formation of a darker area in the middle. The electrons were not fast enough
to fill the entire surface with filaments.
Very interesting is the unique shape of these filaments. Figure 4 shows in the
right frame curved paths of the streamers above the dielectric. This was
caused by the structure of the electric field generated by these plasma
sources.

###  III-C Ozone Production

With the results of the electrical power consumption, the ozone concentration
was measured at all available operational voltages. During all measurements
the reference intensity was reached, when the plasma was shut off, which is
visible in figure 6. There the black line indicates the reference
concentration, based on the initial intensity ($I_{0}$) in the Beer-Lambert’s
law, see equation 1.

Figure 6: Characterisitic dynamics of the ozone production, Geometry (2)
$10\text{\,}\mathrm{k}\mathrm{V}$ (red); Geometry (2)
$16\text{\,}\mathrm{k}\mathrm{V}$ (blue); Geometry (5)
$12\text{\,}\mathrm{k}\mathrm{V}$ (green)

There were three different dynamics of the ozone concentration. As figure 6
shows, the concentration is constant after reaching the saturation level. But
when the plasma source heats itself up, there was a negative linear trend. In
case of the large high voltage electrodes (5 & 6) an exponential drop of the
concentration occurs, which leads to the assumption, that the formation of RNS
quenches the ozone production and therefore it could be an indication to the
RNS production rate [18, 13].
The transition from ozone to nitrogen mode was expected to be constantly
around $0.1\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ [18], the
measurements indicate, that there was a strong influence of the plasma source
concept. On the one hand, (1) and (2) can reach
$0.32\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ without a drop of
the ozone density. On the other hand, even low surface power densities of
$0.05\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ can lead to a
decrease of the concentration in the case of geometry (5). After
$15\text{\,}\mathrm{m}\mathrm{i}\mathrm{n}$ higher power densities lower the
concentration to a non-significant level for plasma source (3) and (5). This
unveils that the transition mode had no constant power surface density value
and depends on the geometric parameters of the plasma sources. Hence, the
power density is not the only predictor of the ozone concentration.
The dynamics of RNS formation is based on the assumption, that their dynamics
were the same as the dissociation rate of the ozone density. The quenching of
the ROS with higher power is in line with previous findings [18].
Since the air flow was a form of an active cooling, the thermal effects were
stronger without it and the curve is similar to the blue in figure 6. And
according to the results of the previous section, the destabilization of the
plasma by the air flow leads to a lower ozone production after few minutes,
because of the small number of filaments. To display the dynamics of the
concentration in dependency of the power density and time, figure 7 shows the
saturation concentrations and the values after
$15\text{\,}\mathrm{m}\mathrm{i}\mathrm{n}$.

Figure 7: Ozone production dynamics of all plasma sources without an air flow.
Saturation concentration (left) and concentration after
$15\text{\,}\mathrm{m}\mathrm{i}\mathrm{n}$ (right)

###  III-D Inactivation Capability

The biocidial effect is a desired capability of the RONS produced by the
plasma. To understand the implications of the electrode geometry for the
inactivation, a microbiological study was conducted to evaluate the
differences of inactivation capability between the plasma sources. This shows
which plasma source concept is the best. But a much wider theoretical
contribution can be made by the correlation of these results to the other
measurements.
Before the inactivation performance was estimated, the data were screened.
Plates with too much CFU ($\geq$300) or outliers were removed from the
statistics. Then for every test series the initial bacteria load was evaluated
with the control plates. The counted CFUs divided by the initial CFUs is the
inactivation performance, which was estimated for every agar plate.
Therefore, figure 8 shows the means of the inactivation performance for every
plasma source ((3) and (4) with the air flow) and exposure time.

Figure 8: Inactivation rates of all plasma sources, (3) and (4) with airflow

Similar to the gas composition in figure 7, figure 8 reveals differences of
the inactivation capability, even when the power consumption was constant.
Remarkable is the difference of the concepts (1), (5) to (2). Even though (2)
has a higher ozone concentration, visible in figure 7, (1) inactivates up to
$10^{4}$ CFUs within a minute, which corresponds to a D-value of
$15\text{\,}\mathrm{s}$. This capability is comparable to plasma source (5),
which uses a lower surface power density but reaches the quenching effect.
Those sources have inactivation rates of $10^{-4}$ to $10^{-5}$, which were
the best results of all geometries. The RNS of (5) were expected to cause a
higher inactivation performance [9]. Contrarily, the inactivation rate does
not improve from minute 2 to 3, when the quenching effect takes place.
The values, which represent decreasing inactivation rates with increasing
exposition times, as (3) and (6) show, were not theoretical supported and were
a result of statistical fluctuations.

###  III-E Air Flow Analysis

All data are analyzed with respect to the air flow
($1\text{\,}\mathrm{m}^{3}\mathrm{/}\mathrm{h}$). The electrical parameters
are neither changed by the slits nor by an applied air flow through those. A
total different behavior unveil the results of the time resolved ignition
behavior. The slits bring up virtual electrodes and the air flow destabilizes
the formation of filaments within the slits. This is visible in figure 5.
During the first $10\text{\,}\mathrm{s}$ after the air flow was applied, the
plasma becomes weaker and does not ignite homogeneously. The reason can be
either a deformation of the filaments or a higher recombination due to the
higher pressure by the air flow. The air flow changes the formation of
filaments and the ozone production. Therefore, the microbiological studies
reflect those results by the inactivation capability, which was reduced three
to four orders of magnitude. The suggested advantages of this (high) air flow
compared to the diffusion has to be neglected.

##  IV Conclusion

This work verifies the applicability of thin PCBs as plasma sources. The
resulting advantages are the fast prototyping compared to dielectrics like
glass or ceramics and the lower power consumption compared to volume DBDs
[14]. Additionally, PCBs have laminated electrodes, therefore it is not
possible that gases can be between the electrodes and the dielectric.
The capacity is given by the amount of charges, which can accumulate above the
high voltage electrode. Increasing voltages lead to a higher power consumption
and higher capacities, because of the higher electron density. Surface power
densities of $0.05\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ to
$0.57\text{\,}\mathrm{W}\mathrm{/}\mathrm{c}\mathrm{m}^{2}$ were achievable.
The ignition behavior depends on the distance between the electrodes. Longer
discharges lead to arc-like filaments and due to the lower electric field, to
a shorter duration of the plasma ignition. All plasma sources have in common,
that the discharge at the beginning of the negative half-cycle was the
strongest and consists of simultaneously igniting filaments. Additionally,
this analysis verifies the concept of the virtual electrodes, which lead to a
discharge above the dielectric and not in the slits at low voltages. A higher
electric field leads to accumulation of electrons at the edges of and
filaments over the slits.
Geometry (2) produces the highest ozone concentration of
$435\text{\,}\mathrm{p}\mathrm{p}\mathrm{m}$. The ozone density shows as well
as the ozone production a positive dependency on the supply voltage. The
quenching of ozone occurs with concept (5/6) and is an indication of the RNS
production. The formation of curved filaments occurs only with this concept.
The linkage of the filaments’ shape to the gas-composition will become
addressed by further research.
The microbiological study unveils inactivation rates of $10^{-4}$ to $10^{-5}$
as the best results of all geometries. Therefore, D-values of
$15\text{\,}\mathrm{s}$ were achievable. The air flow in this case
destabilizes the plasma and extinguishes the filaments. Due to that, the worst
case (3) show a drop of the inactivation performance of three to four orders
of magnitude. Therefore, it can be concluded that the plasma chemistry and the
inactivation capability is influenced by the electrode geometry and the
surface power density as well. The air flow does not improve the plasma
technology until now.
Finally, it is to state that the electrode geometry has to be defined with
respect to the scope of application. The results recommend displaced
electrodes to improve the inactivation capability. Yet, ozone formation was
increased, when there are parallel electrodes and spaces with a darker plasma
glow.
The PCB-technology is a promising process for plasma source development. Yet,
epoxy resin based dielectrics have a limited lifetime of
$15\text{\,}\mathrm{h}$. Therefore further development effort will be invested
in PCB based SMDs with other dielectric materials. This will make the plasma
inactivation the next disinfection technology of the future, which will reduce
nosocomial infections and durations of reprocessing cycles.

TABLE I: Overview of the four evaluations Geo. | Power | Filaments’ | Ozone | Log-
---|---|---|---|---
| density [W/cm2] | shape | [ppm] | reduction
(1) | 0.32 - 0.55 | arc-like | 212 | 5
(2) | 0.06 - 0.57 | short | 435 | 4
(3) | 0.43 | virtual electrodes | 376 | 1
(4) | 0.05 - 0.45 | virtual electrodes | 125 | 2
(5) | 0.07 - 0.57 | curved | 82 | 5
(6) | 0.29 | curved | 184 | 4

## References

  * [1] J. Jeon, “Surface micro-discharge (smd)-analysis of the antimicrobial effect and the plasma chemistry,” Ph.D. dissertation, Ludwig-Maximilians-Universität München, 2015.
  * [2] H. W. Boucher, M. Scheld, J. Bartlett, G. H. Talbot, J. S. Bradley, B. Spellberg, J. E. Edwards, D. Gilbert, and L. B. Rice, “Bad Bugs, No Drugs: No ESKAPE! An Update from the Infectious Diseases Society of America,” _Clinical Infectious Diseases_ , vol. 48, no. 1, pp. 1–12, 01 2009.
  * [3] A. Talebi Bezmin Abadi, A. A. Rizvanov, T. Haertlé, and N. L. Blatt, “World health organization report: Current crisis of antibiotic resistance,” _BioNanoScience_ , vol. 9, no. 4, pp. 778–788, 2019.
  * [4] S. J. Pidot, W. Gao, A. H. Buultjens, I. R. Monk, R. Guerillot, G. P. Carter, J. Y. H. Lee, M. M. C. Lam, M. L. Grayson, S. A. Ballard, A. A. Mahony, E. A. Grabsch, D. Kotsanas, T. M. Korman, G. W. Coombs, J. O. Robinson, A. Gonçalves da Silva, T. Seemann, B. P. Howden, P. D. R. Johnson, and T. P. Stinear, “Increasing tolerance of hospital enterococcus faecium to handwash alcohols,” _Science Translational Medicine_ , vol. 10, no. 452, 2018.
  * [5] J. Goldberg, “Mikrobiologische testung der sterilisation von medizinprodukten,” Ph.D. dissertation, Justus-Liebig-Universität, Otto-Behaghel-Str. 8, 35394 Gießen, 2009.
  * [6] G. E. Morfill, T. Shimizu, B. Steffes, and H.-U. Schmidt, “Nosocomial infections—a new approach towards preventive medicine using plasmas,” _New Journal of Physics_ , vol. 11, no. 11, p. 115019, nov 2009\.
  * [7] T. Maisch, T. Shimizu, Y.-F. Li, J. Heinlin, S. Karrer, G. Morfill, and J. L. Zimmermann, “Decolonisation of mrsa, s. aureus and e. coli by cold-atmospheric plasma using a porcine skin model in vitro,” _PLOS ONE_ , vol. 7, no. 4, pp. 1–9, 04 2012.
  * [8] J. Guo, K. Huang, and J. Wang, “Bactericidal effect of various non-thermal plasma agents and the influence of experimental conditions in microbial inactivation: A review,” _Food Control_ , vol. 50, pp. 482 – 490, 2015.
  * [9] M. J. Pavlovich, D. S. Clark, and D. B. Graves, “Quantification of air plasma chemistry for surface disinfection,” _Plasma Sources Science and Technology_ , vol. 23, no. 6, p. 065036, oct 2014.
  * [10] B. Denis, S. Steves, E. Semmler, N. Bibinov, W. Novak, and P. Awakowicz, “Plasma sterilization of pharmaceutical products: From basics to production,” _Plasma Processes and Polymers_ , vol. 9, no. 6, pp. 619–629, may 2012.
  * [11] V. Taran, I. Garkusha, Y. Gnidenko, V. Krasnyj, A. Lozina, A. Taran, O. Chechelnitskyi, V. Starikov, and S. Starikova, “Portable ozone sterilization device with mechanical and ultrasonic cleaning units for dentistry,” _Review of Scientific Instruments_ , vol. 91, no. 8, p. 084105, 2020.
  * [12] W. A. Rutala, H. Kanamori, E. E. Sickbert-Bennett, and D. J. Weber, “What’s new in reprocessing endoscopes: Are we going to ensure “the needs of the patient come first” by shifting from disinfection to sterilization?” _American Journal of Infection Control_ , vol. 47, pp. A62–A66, 2019, disinfection, Sterilization and Antisepsis: Principles, Practices, Current Issues, New Research and New Technologies.
  * [13] B. L. Li, Y. Yu, and M. Y. Ye, “Effects of plasma activated species produced by a surface micro-discharge device on growth inhibition of cyanobacteria,” _Plasma Research Express_ , vol. 1, no. 1, p. 015017, mar 2019.
  * [14] M. Simor, J. Rahel’, P. Vojtek, M. Cernak, and A. Brablec, “Atmospheric-pressure diffuse coplanar surface discharge for surface treatments,” _Applied Physics Letters_ , vol. 81, no. 15, pp. 2716–2718, 2002.
  * [15] M. Cernak, D. Kovacik, J. Rahel’, P. St’ahel, A. Zahoranova, J. Kubincova, A. Toth, and L. Cernakova, “Generation of a high-density highly non-equilibrium air plasma for high-speed large-area flat surface processing,” _Plasma Physics and Controlled Fusion_ , vol. 53, no. 12, p. 124031, 2011.
  * [16] _Datenblatt: Original Bungard fotobeschichtetes Basismaterial FR4_ , Bungard Elektronik GmbH & Co. KG.
  * [17] W. Rogowski and W. Steinhaus, “Die messung der magnetischen spannung,” _Archiv für Elektrotechnik_ , vol. 1, no. 4, pp. 141–150, 1912. [Online]. Available: https://doi.org/10.1007/BF01656479
  * [18] T. Shimizu, Y. Sakiyama, D. B. Graves, J. L. Zimmermann, and G. E. Morfill, “The dynamics of ozone generation and mode transition in air surface micro-discharge plasma at atmospheric pressure,” _New Journal of Physics_ , vol. 14, no. 10, p. 103028, oct 2012.
  * [19] M. J. Pavlovich, H.-W. Chang, Y. Sakiyama, D. S. Clark, and D. B. Graves, “Ozone correlates with antibacterial effects from indirect air dielectric barrier discharge treatment of water,” _Journal of Physics D: Applied Physics_ , vol. 46, no. 14, p. 145202, 2013.
  * [20] J. Burrows, A. Richter, A. Dehn, B. Deters, S. Himmelmann, S. Voigt, and J. Orphal, “Atmospheric remote-sensing reference data from gome—2. temperature-dependent absorption cross sections of o3 in the 231–794nm range,” _Journal of Quantitative Spectroscopy and Radiative Transfer_ , vol. 61, no. 4, pp. 509–517, 1999.
  * [21] P. K. Chu and X. Lu, _Low temperature plasma technology: methods and applications_. CRC Press, 2013.