# Towards Natural Machine Unlearning

Zhengbao He Tao Li Xinwen Cheng Zhehao Huang Xiaolin Huang∗
Department of Automation, Shanghai Jiao Tong University
{lstefanie, li.tao, xinwencheng, kinght_h<EMAIL_ADDRESS>

###### Abstract

Machine unlearning (MU) aims to eliminate information that has been learned
from specific training data, namely forgetting data, from a pre-trained model.
Currently, the mainstream of existing MU methods involves modifying the
forgetting data with incorrect labels and subsequently fine-tuning the model.
While learning such incorrect information can indeed remove knowledge, the
process is quite unnatural as the unlearning process undesirably reinforces
the incorrect information and leads to over-forgetting. Towards more natural
machine unlearning, we inject correct information from the remaining data to
the forgetting samples when changing their labels. Through pairing these
adjusted samples with their labels, the model will tend to use the injected
correct information and naturally suppress the information meant to be
forgotten. Albeit straightforward, such a first step towards natural machine
unlearning can significantly outperform current state-of-the-art approaches.
In particular, our method substantially reduces the over-forgetting and leads
to strong robustness to hyperparameters, making it a promising candidate for
practical machine unlearning.

##  1 Introduction

Modern machine learning models are essential in various applications [1, 2, 3,
4]. However, their heavy reliance on extensive data for training raises
significant privacy concerns. The General Data Protection Regulations (GDPR)
[5] emphasizes individuals’ rights to request the deletion of their private
data, leading to a surge of interests in machine unlearning (MU). MU aims to
remove the influence of specific data in training set from a well-trained
model. Recently, this field has garnered considerable attention not only for
its contributions to privacy protection [6, 7, 8, 9] but also for its capacity
to eliminate erroneous and sensitive data [10, 11, 12].

Current popular MU methods are primarily optimization-based, which achieve
unlearning by fine-tuning the original model with manually crafted data. For
instance, Amnesiac [13] optimizes the model using randomly labeled forgetting
samples along with other remaining data. BadTeacher [14] relabels the
forgetting samples with the predictions of a randomly initialized model as a
“bad” teacher.

The name of “bad” teacher hits the essence of the above methods: they create
incorrect information with the forgetting samples to fine-tune the model,
compelling it to forget the correct information previously learned111 In this
paper, we consider a training point as a ”sample-label” pair, also referred to
as an ”instance”. More details regarding the terminology can be found in Sec.
3.1. . However, the incorrect information could be undesirably reinforced
during the fine-tuning process. An obvious observation, illustrated in Fig. 1,
is the so-called “over-forgetting”, where after sufficient training, the
accuracy on the forgetting samples (denoted as forgetting accuracy) is
significantly lower than expected levels. This occurs because the incorrect
information in these randomly relabeled instances appears quite unnatural in
the view of remaining data [15]. The conflicts between the relabeled instances
and remaining data cause the unlearned model to remember these forgetting
samples even more firmly. To avoid over-forgetting, one can restrict the
mobility scale by identifying a parameter mask [16] or by carefully selecting
a stopping point. However, this requires meticulous hyperparameter tuning,
which is impractical in real-world applications.

Figure 1:  Accuracy comparison of different methods on forgetting samples. The
experiments are taken on CIFAR-100 using ResNet18 under 1% random-subset
unlearning setting. The dash line is the natural forgetting accuracy of
retrained model, to which a smaller gap indicates better MU. The forgetting
accuracy of other methods continuously decreases after crossing over the dash
line, while ours can converge to that of the retrained model. Hence, to obtain
a good MU performance, other methods may need to carefully stop training at a
middle point, donated as $\star$. Best viewed in color.

To address the problem of over-forgetting, we contend that the unlearning
process should be natural, minimizing the conflicts within the fine-tuning
data. The most natural MU process is to retrain the model from scratch with
the training data excluding the forgetting ones, which is the golden standard
for MU [17, 18, 19]. This method ensures that all information used is correct,
eliminating concerns about over-forgetting since the forgetting samples are
excluded from the training process. However, retraining requires significant
computational resources, making it impractical in many cases. The challenge,
therefore, lies in balancing the inclusion of forgetting samples to enhance
unlearning efficiency and the exclusion of incorrect information to maintain
the process’s naturalness.

In this paper, we introduce a novel method called “NatMU” that aims to
facilitate a more natural unlearning by injecting correct information into the
forgetting samples. Specifically, such correct information is extracted from
the remaining data and then injected into forgetting samples for creating
hybrid samples. The hybrid samples are subsequently assigned categories
consistent with the injected information. Each hybrid sample consists of two
distinct types of information: one from the forgetting sample, and the other
from the remaining sample. By learning to pair the hybrid sample with the
reassigned label, the connection between injected information and
corresponding label is reinforced. This naturally suppresses model’s response
to the former type of information which is to be forgotten, thereby achieving
effective unlearning. Since such reinforced connection inherently exists
within the remaining set, NatMU achieves a more natural machine unlearning
process.

NatMU takes the first step towards natural machine unlearning with the
injection of correct information, which significantly reduces the conflicts
within the fine-tuning data. The resulting model can maintain a natural
generalization on forgetting samples, consistent with the predictions of
retrained model. As a result, NatMU can narrow the forgetting accuracy’s gap
with the retrained model from 39.44% to 1.72% when unlearning sub-class
“Vehicle2” on CIFAR-20. The natural property eliminates the need to stop
training at unstable points for good performance. Consequently, NatMU
demonstrates considerable robustness to hyperparameters, enabling the
application of shared hyperparameters across different settings. For instance,
when altering the forgetting ratio from 10% to 1%, NatMU maintains an average
forgetting accuracy gap of only 2.03% across three datasets, while other
methods exhibit a gap over 20%.

Our contributions can be summarized as follows:

$\bullet$ We first point out the unnatural property of the previous
optimization-based MU methods, which leads to issues such as unnatural
generalization on forgetting samples and impracticality.

$\bullet$ We propose an effective and efficient MU approach towards natural
machine unlearning, named NatMU, which successfully addresses previous issues
by injecting correct information into forgetting samples to remove the
information to be forgotten.

$\bullet$ Extensive experiments on various datasets and multiple machine
unlearning scenarios demonstrate that NatMU significantly narrows the
performance gap with retrained model compared to previous approaches.
Moreover, NatMU exhibits considerable robustness to hyperparameters.

##  2 Related Work

Machine unlearning aims to update a pre-trained model to remove the influence
of a subset of training set [20]. Although retraining from scratch with the
remaining data can achieve perfect MU, its significant demand for resources
and time is unacceptable in the current deep learning context. Therefore,
numerous works have emerged that aim to approximate the retrained model,
especially in the output space.

Optimization-free unlearning focuses on removing the influence of the
forgetting data by directly modifying model weights. Influence function is
first introduced by [21], and [7] adopts it for certified data removal in
$L_{2}$-regularized linear models. The following work [22, 23, 24, 25] is
dedicated to reducing the computational burdens introduced by Hessian
inversion. [23] adopts infinitesimal jackknife with Newton methods to reduce
the number of calculations while [24] reduces the overhead of one calculation
by selecting only important parameters. FisherForgetting [26] introduces a
weight scrubbing method by injecting noise to specific parameters to clean the
information about the forgetting data with fisher information matrix [27]. SSD
[28] addresses its computational overhead and generalization decrease through
a fast but stringent parameter selection. Besides the influence function-based
methods, [13] proposes to delete the gradient of mini-batches relevant to the
forgetting data. [18] decomposes the SGD training process and performs a
gradient ascent on the forgetting data. Although these methods can remove the
influence of forgetting data, they may significantly impair the generalization
capability of the MU model, especially in the context of modern deep learning.

Optimization-based unlearning re-optimizes the original model on a carefully
crafted dataset with a proposed unlearning objective. Amnesiac [13] fine-tunes
the model with randomly labeled forgetting samples along with unchanged
remaining data. BadTeacher [14] and SCRUB [29] both achieve machine unlearning
under a teacher-student framework, where the student network selectively obeys
the teacher models for different samples. SalUn [16] adopts the same way as
Amnesiac, while constraining the model parameters’ update with a weight
saliency mask. However, these methods all face the over-forgetting problem as
they relabels the forgetting data to some extent. Recently, some studies have
explored the scenario where no remaining data are available, called "zero-
shot" machine unlearning. MU-Mis [30] reveals the link between sample’s
contribution to learning process and model’s sensitivity to it, subsequently
proposing an algorithm by minimizing input sensitivity. JiT [31] and [32]
perturbs forgetting samples with random or adversarial perturbation, and
ensures the predictions of perturbed versions match reference predictions to
maintain model performance.

##  3 Towards Natural Machine Unlearning

###  3.1 Preliminaries

Let $\mathcal{D}=\\{(\bm{x}_{i},y_{i})\\}_{i=1}^{N}$ be the training set
containing inputs sample $\bm{x}_{i}\in\mathbb{R}^{d}$ with corresponding
label $y_{i}\in\\{1,2,...,K\\}$. We denote a sample-label pair as an
“instance” in this paper. Let forgetting set
$\mathcal{D}^{f}\subset\mathcal{D}$ be the set of forgetting data and
remaining set $\mathcal{D}^{r}=\mathcal{D}\setminus\mathcal{D}^{f}$ be the set
of remaining data. The machine learning model can be represented as a
parameterized function
$f_{\theta}(\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{K}$ with parameter
$\theta$. Let $\theta_{o}$ be the parameter of original model. The objective
of approximate MU is to narrow the performance gap between the unlearned and
retrained model in output space.

Mainstream machine unlearning work primarily focuses on three unlearning
scenarios: (1)  _full-class unlearning_ , where
$\mathcal{D}^{f}=\\{(\bm{x}_{i},y_{i})\in\mathcal{D},y_{i}=k\\}$ contains all
samples with label $y_{i}=k$. (2)  _sub-class unlearning_ , where
$\mathcal{D}^{f}\subset\\{(\bm{x}_{i},y_{i})\in\mathcal{D},y_{i}=k\\}$
contains a related subset of the samples labeled as $k$. For instance, this
scenario includes unlearning the sub-class "Shark" within class "Fish" on
CIFAR-20 dataset [33]. (3)  _random-subset unlearning_ , where
$\mathcal{D}^{f}$ is a random subset of $\mathcal{D}$. The size of
$\mathcal{D}^{f}$ is determined by the forgetting ratio. For example, 1%
forgetting ratio implies that 1% of the samples from $\mathcal{D}$ are
randomly selected to form $\mathcal{D}^{f}$. Among these unlearning scenarios,
random-subset unlearning presents the greatest challenge due to the
disconnection within $\mathcal{D}^{f}$ and the intricate intertwining between
$\mathcal{D}^{f}$ and $\mathcal{D}^{r}$.

###  3.2 Revisiting Previous MU Methods

Optimization-based MU algorithms demonstrate exceptional performance across
various scenarios, especially in random-subset unlearning. These methods
require fine-tuning the original model on a carefully crafted fine-tuning
dataset, typically comprising two parts: one built from the remaining data to
maintain generalization ability and the other from the forgetting data to
facilitate unlearning. We name the instances in the latter part as unlearning
instances. In Amnesiac [13], the fine-tuning dataset
$D_{\mathrm{AMN}}=\mathcal{D}^{r}\cup\mathcal{D}^{f}_{\mathrm{RL}}$, where
$\mathcal{D}^{f}_{\mathrm{RL}}=\\{(\bm{x}_{i},y_{i}^{\mathrm{rand}}),(\bm{x}_{i},y_{i})\in\mathcal{D}^{f},y_{i}^{\mathrm{rand}}\neq
y_{i}\\}$. BadTeacher [14] utilizes the original original $f_{\theta_{o}}$ and
a randomly initialized model $f_{\theta_{\mathrm{bad}}}$ to construct softly
labeled fine-tuning dataset from $\mathcal{D}^{r}$ and $\mathcal{D}^{f}$
respectively. Its remaining part is constructed with the original model:
$\mathcal{D}^{r}_{\mathrm{ori}}=\\{(\bm{x}_{i},f_{\theta_{o}}(\bm{x}_{i})),(\bm{x}_{i},y_{i})\in\mathcal{D}^{r}\\}$.
While the forgetting part uses a randomly initialized model:
$\mathcal{D}^{f}_{\mathrm{bad}}=\\{(\bm{x}_{i},f_{\theta_{\mathrm{bad}}}(\bm{x}_{i})),(\bm{x}_{i},y_{i})\in\mathcal{D}^{f}\\}$.
SalUn [16] improves Amnesiac by freezing certain model parameters according to
a gradient-based weight saliency map. However, it still adopts the same loss
function with Amnesiac.

To approximate the retrained model in the output space, the ideal approach
would be to relabel the forgetting samples with their predictions on the
retrained model as the ground truth labels. However, in real-world unlearning
scenarios, these ground truth labels cannot be obtained. Therefore, to ensure
the unlearning process can effectively remove the learned knowledge, previous
works can only assign incorrect labels to these samples as a secondary
priority. This line of works are based on a consensus that the information in
an instance $(\bm{x}_{i},y_{i})$ is determined by both the sample and its
corresponding label. When $y_{i}$ changes, the instance’s information also
significantly changes [34]. Thus, learning such modified data that contain
incorrect information can remove the learned knowledge from them.

However, as illustrated in Fig. 1, these methods demonstrate an “over-
forgetting” issue. After sufficient training, the accuracy of classifying the
forgetting samples as their original labels is significantly lower than that
of retraining. The underlying challenge is that incorrect information, i.e.,
the forgetting samples with their incorrect labels, is undesirably reinforced
during fine-tuning. The unlearning process is quite unnatural since it compels
the unlearned model to learn incorrect information, which actually conflicts
with the remaining data. This adversely affects the model’s natural ability to
generalize these samples, which the retrained model preserves. For example, in
random-subset unlearning, most of the forgetting samples can still be
correctly classified by the retrained model, even if they are not involved in
the training. Essentially, this unlearning process alters the learned
knowledge to incorrect knowledge, instead of removing it. In addition to
forgetting accuracy, various metrics can be also employed to measure the
differences between the unlearned and retrained models, which will be
discussed in Sec. 4.1. Despite meticulous hyperparameter tuning can make one
metric close to the retrained model, it cannot ensure that other metrics will
also be close.

Therefore, it is crucial to realize a more natural machine unlearning while
ensuring its efficiency. On one hand, the forgetting samples should be
included to enhance unlearning efficiency. On the other hand, excluding
incorrect information ensures a natural machine unlearning process. In this
paper, we propose a novel method, named NatMU, towards natural machine
unlearning. The core idea of NatMU is to inject information extracted from the
remaining data into the forgetting samples to reduce the conflicts in each
modified instance.

###  3.3 Proposed Method

Overview. Our NatMU method can be decomposed into three steps. Firstly, we
randomly select $n$ distinctive instances from the remaining set for each
forgetting sample $\bm{x}^{f}$. Secondly, a injecting function $\mathcal{T}$
is employed to inject the information of remaining sample $\bm{x}^{r}$ into
$\bm{x}^{f}$ by blending them at the pixel level, generating an unlearning
instance $(\mathcal{T}(\bm{x}^{f},\bm{x}^{r}),y^{r})$. Thirdly, by merging
remaining set with all generated unlearning data, we fine-tune original model
on the resulting fine-tuning dataset.

Selection of remaining instances. For a forgetting instance
$(\bm{x}^{f},y^{f})\in\mathcal{D}^{f}$, the $n$ selected remaining instances
$\\{(\bm{x}_{j}^{r},y_{j}^{r})\\}_{j=1}^{n}$ should have distinctive
categories from $y^{f}$ to ensure the effectiveness of unlearning. These
remaining instances inherently reflect the correct information of the
remaining data’s distribution. To prevent that the model from having a
preference for mapping the forgetting sample to any reassigned category, these
selected instances should also have different categories from each other.
Moreover, to reduce the conflicts in unlearning instances
$(\mathcal{T}(\bm{x}^{f},\bm{x}^{r}_{j}),y^{r}_{j})$, the categories of
remaining instances should be relevant to the forgetting sample. To meet these
requirements, we calculate the top $n$ predicted categories for the forgetting
sample on the original model, excluding the original category $y^{f}$. Then,
we sequentially select one remaining instance for each category.

The injecting function $\mathcal{T}$. Motivated by data augmentation like
MixUp [35] and CutMix [36], we opt for a straightforward approach where two
samples are added pixel by pixel:
$\mathcal{T}(\bm{x}^{f},\bm{x}^{r})=\bm{x}^{f}+\bm{x}^{r}$. To ensure that the
blended sample numerically matches the original distribution, a weighting mask
vector $\bm{m}\in[0,1]^{d}$ is introduced. This vector controls the
contribution of two samples at each pixel as defined by
$\mathcal{T}_{\bm{m}}(\bm{x}^{f},\bm{x}^{r})=\bm{x}^{f}\circ\bm{m}+\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m})$,
where $\circ$ represents element-wise multiplication, and $\bm{1}_{d}$
represents a $d$-dimensional vectors of ones. The weighting operation offers
an additional advantage that $\bm{x}^{f}\circ\bm{m}$ can be regarded as a
segment of the complete forgetting sample $\bm{x}^{f}$. Consequently, it makes
the unlearning process more friendly since each hybrid sample forgets only a
part of $\bm{x}^{f}$. We have developed four "gradual MixUp" weighting vectors
$\\{\bm{m}_{1},\bm{m}_{2},\bm{m}_{3},\bm{m}_{4}\\}$, with symmetrical elements
that vary gradually in different directions. To adjust the proportion of
forgetting samples in the hybrid samples, a scaling factor $\delta$ is
introduced, which scales the values of weighting vectors as follows:

$\bm{m}_{j}^{\mathrm{scaled}}=\mathrm{clip}_{[0,1]}(\bm{\delta}_{d}+\bm{m}_{j}),$
(1)

where $\bm{\delta}_{d}$ denotes a $d$-dimensional vector with all elements
equal to $\delta$ and $\mathrm{clip}_{[0,1]}(\cdot)$ denotes a function which
truncates vector elements to $[0,1]$. A detailed description is provided in
Appendix A.2.

Constructing the fine-tuning dataset. For a forgetting instance
$(\bm{x}_{i}^{f},y_{i}^{f})\in\mathcal{D}^{f}$, we select four remaining
instances $\\{(\bm{x}_{j}^{r},y_{j}^{r})\in\mathcal{D}^{r}\\}_{j=1}^{n=4}$
according to the above rules. Then, we generate unlearning instances using the
following formula:

$\mathcal{D}_{i}^{f}=\\{(\mathcal{T}_{\bm{m}_{j}^{\mathrm{scaled}}}(\bm{x}_{i}^{f},\bm{x}_{j}^{r}),y_{j}^{r})\\}_{j=1}^{n}.$
(2)

After executing the aforementioned operation on all forgetting samples,
following [13, 16], we merge all the sets from these samples with the
remaining set to compile the final fine-tuning dataset:

$\mathcal{D}_{\mathrm{Nat}}=\mathcal{D}^{r}\cup\mathcal{D}^{f}_{\mathrm{forget}},\mathrm{~{}where~{}}\mathcal{D}^{f}_{\mathrm{forget}}=\mathcal{D}_{1}^{f}\cup\mathcal{D}_{2}^{f}\cup\cdots\mathcal{D}_{|\mathcal{D}^{f}|}^{f}.$
(3)

We present the full algorithm for generating the fine-tuning dataset in
Algorithm 1. Finally, we fine-tune the original model on the resulting
dataset, akin to other optimization-based methods.

###  3.4 Discussion

(a)

$\bm{x}^{f}$ |  $\bm{x}^{f}\circ\bm{m}_{j}$ |  $\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m}_{j})$ |  $\mathcal{T}_{\bm{m}_{j}}(\bm{x}^{r},\bm{x}^{r})$ |  Before |  After
---|---|---|---|---|---
|  |  |  |  |

(b)

Figure 2:  (a) Relationship between the performance gap on forgetting accuracy
and KL divergence on different datasets. A smaller KL divergence indicates
more natural MU. NatMU’s KL divergence is much smaller than other methods,
i.e., more natural, thus resulting in a smaller performance gap. (b)
Visualization of our unlearning instances and their attention maps before and
after unlearning calcualted with LRP [37]. After unlearning, the attention is
shifted to the remaining information.

Unlearning mechanism. Given an unlearning instance
$(\bm{x}^{f}\circ\bm{m}+\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m}),y^{r})\in\mathcal{D}_{\mathrm{Nat}}$,
it consists of two distinct parts of information: the information to be
forgotten, denoted as $\bm{x}^{f}\circ\bm{m}$, and the injected information,
denoted as $\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m})$. In the original pre-trained
model, the former information is connected to the forgetting label $y^{f}$,
while the latter information is connected to the reassigned label $y^{r}$.
During the fine-tuning of unlearning process, the unlearned model learns to
pair the unlearning sample with label $y^{r}$, reinforcing the latter
connection. It naturally suppresses model’s response to the forgetting
information, since such response is harmful to make accurate predictions for
unlearning instances. As a result, it effectively removes the learned
knowledge in $\bm{x}^{f}\circ\bm{m}$.

Compared to previous label-modifying work, the reinforced information in NatMU
is the connection between $\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m})$ and $y^{r}$,
which inherently exists in the remaining set. This prevents the model from
establishing undesirable new associations between $\bm{x}^{f}\circ\bm{m}$ and
$y^{r}$, thereby promoting a more natural unlearning process.

Fig. 2(b) visualizes the unlearning instances of NatMU and their attention
maps before and after unlearning. We can see that different weighting masks
capture different parts of forgetting samples in the second column. By
comparing the attention maps before and after unlearning, model’s attention is
effectively shifted to the prominent positions of the remaining samples
through unlearning. For example, in the first row, before unlearning the model
focuses on the bird’s wings in $\bm{x}^{f}\circ\bm{m}$. After unlearning, the
model shifts it attention to the butterfly below the bird, which comes from
$\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m})$. More visual results are provided in
Appendix D.1.

Natural property. The injected information is extracted from remaining data,
therefore, the unlearning instances of NatMU align with the distribution of
remaining data more closely. It significantly reduces the conflict between the
unlearning instances and the remaining data. To quantify the degree of data
conflict, we adopt the KL divergence [38] between the reassigned unlearning
labels and retrained model’s predictions on the unlearning samples. A smaller
KL divergence indicates more natural unlearning, as the unlearned data aligns
better with the retrained model’s knowledge distribution. Since no unlearning
samples are involved in retraining, we define its KL divergence as zero. In
Fig. 2(a), we demonstrate the average KL divergence of different methods over
their unlearning instances. For Amnesiac, the unlearning instances are from
$\mathcal{D}^{f}_{\mathrm{RL}}$. For BadTeacher, from
$\mathcal{D}^{f}_{\mathrm{bad}}$; and for NatMU, from
$\mathcal{D}^{f}_{\mathrm{forget}}$. We can see that NatMU has a much smaller
KL divergence compared to other methods, indicating that its unlearning is
more natural. Therefore, NatMU’s performance is closer to that of the
retrained model.

Figure 3: Accuracy curves on partial unlearning instances
$\\{(\bm{x}^{f}\circ\bm{m},y^{r})\\}$ and forgetting accuracy of different MU
models trained with/without correct information. Conducted on CIFAR-100 with a
forgetting ratio of 1% using ResNet18. PFA: accuracy of classifying partial
forgetting samples $\bm{x}^{f}\circ\bm{m}$ as random label $y^{r}$. FA:
forgetting accuracy. CI: correct information. ReT: the retrained model.

Preventing undesirable reinforcement. In previous work, incorrect information
is reinforced during unlearning, leading to the over-forgetting problem.
Specifically, the model establishes incorrect associations between the
forgetting samples and their reassigned labels. Although NatMU also modifies
the forgetting labels, the existence of correct information from $x^{r}$ can
effectively prevent these incorrect associations. To verify this, we
demonstrate the accuracy curves onf sample-label pairs
$\\{(\bm{x}^{f}\circ\bm{m},y^{r})\\}$ and forgetting accuracy curves of
different MU models trained with or without the correct information
$\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m})$ in Fig. 3. It can be seen that, without
the injected correct information, the model quickly recognizes the partial
forgetting sample $\bm{x}^{f}\circ\bm{m}$ as the reassigned class $y^{r}$. As
a result, the forgetting accuracy faces a significant decrease. In contrast,
the model trained with correct information consistently remains a low accuracy
on $\\{(\bm{x}^{f}\circ\bm{m},y^{r})\\}$, showing that injecting correct
information can effectively prevent undesirable reinforcement.

The injected correct information form remaining data helps NatMU achieve a
more natural machine unlearning, not forced unlearning as other label-
modifying methods do. The natural generalization on forgetting samples is well
preserved in NatMU. As a result, NatMU can achieve a smaller performance gap
with the retrained model on multiple metrics. Moreover, the natural property
gives NatMU greater robustness, allowing us to use nearly identical
hyperparameters across different unlearning settings, making NatMU a promising
candidate for practical MU.

##  4 Experiments

In this section, we evaluate NatMU against other baselines across different
datasets, models, and unlearning settings. In addition to comparing the
performance of various methods with optimal hyperparameters, we also compare
the performance of each method after sufficient training, which is common
without using the retrained model as a hyperparameter tuning reference.
Furthermore, we demonstrate the hyperparameter robustness of different methods
by using the optimal hyperparameters from one setting to conduct experiments
in another setting. Finally, in ablation study, we investigate the role of the
injected correct information.

###  4.1 Setup

Datasets. NatMU is evaluated against other machine unlearning methods in the
context of supervised image classification tasks using the CIFAR-10, CIFAR-20,
CIFAR-100 [33], and TinyImageNet-200 [39] datasets. It is noteworthy that
CIFAR-20 and CIFAR-100 are closely related. CIFAR-100 dataset consists of 20
superclasses, each containing 5 subclasses, resulting in a total of 100
classes. When considering only the superclasses, CIFAR-100 reduces to
CIFAR-20.

Unlearning scenarios. Following [14] and [28], we evaluate MU methods across
three unlearning scenarios: (1) _full-class unlearning_ , (2) _sub-class
unlearning_ , and (3) _random-subset unlearning_. Models of various
architectures are trained in different unlearning scenarios, including
VGG16-BN [40], ResNet18 and ResNet34 [41].

Baselines. We compare NatMU with basic MU methods and multiple popular MU
methods, including Finetune [42] which fine-tunes the original model on
$D^{r}$, gradient ascent (GA) [18] which performs gradient ascent over
$D^{f}$, Amnesiac [13], BadTeacher [14], SalUn [16], and SSD [28]. Some of
these methods are designed for specific unlearning scenarios. Under different
unlearning scenarios, we report only the methods that performed well.

Table 1: Full-class unlearning on CIFAR-100 using ResNet18 with different
forgetting classes. The results are given by $\bm{a_{\pm
b}({\color[rgb]{0,0,1}c})}$, where $\bm{a}$ donates the mean value, $\bm{b}$
donates the standard deviation, and $\bm{{\color[rgb]{0,0,1}c}}$ donates the
performance gap with the retrained model over 5 independent trails. A smaller
$\bm{{\color[rgb]{0,0,1}c}}$ means a better performance of MU methods.

Class | Metric | Retrain | Finetune [42] | GA [18] | Amnesiac [13] | BadTeacher [14] | SalUn [16] | SSD [28] | Ours
---|---|---|---|---|---|---|---|---|---
Rocket | RA | 76.44±0.33 | 76.36±0.07(0.07) | 71.93±0.00(4.51) | 76.87±0.06(0.44) | 76.88±0.09(0.45) | 76.97±0.17(0.53) | 76.53±0.03(0.10) | 76.63±0.23(0.19)
FATrain | 0.00±0.00 | 0.00±0.00(0.00) | 2.40±0.00(2.40) | 0.00±0.00(0.00) | 5.60±0.85(5.60) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
FATest | 0.00±0.00 | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.20±0.40(0.20) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
MIA | 14.64±0.77 | 6.08±0.48(8.56) | 6.60±0.00(8.04) | 8.68±0.99(5.96) | 0.00±0.00(14.64) | 7.04±0.66(7.60) | 0.88±0.10(13.76) | 11.52±1.27(3.12)
AvgGap | - | 2.16 | 3.74 | 1.60 | 5.22 | 2.03 | 3.46 | 0.83
Vehicle2 | RA | 76.43±0.27 | 76.75±0.01(0.32) | 68.57±0.00(7.86) | 77.16±0.14(0.73) | 77.83±0.13(1.40) | 76.89±0.18(0.46) | 76.60±0.08(0.17) | 76.95±0.13(0.52)
FATrain | 0.00±0.00 | 0.00±0.00(0.00) | 1.60±0.00(1.60) | 0.00±0.00(0.00) | 3.12±2.13(3.12) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
FATest | 0.00±0.00 | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 4.40±0.80(4.40) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
MIA | 11.48±0.84 | 2.20±0.63(9.28) | 13.00±0.00(1.52) | 9.08±0.74(2.40) | 0.00±0.00(11.48) | 9.48±1.28(2.00) | 2.72±0.57(8.76) | 13.00±1.41(1.52)
AvgGap | - | 2.40 | 2.74 | 0.78 | 5.10 | 0.61 |  2.23 | 0.51
Sea | RA | 76.44±0.19 | 76.54±0.28(0.09) | 73.89±0.00(2.56) | 76.18±0.28(0.26) | 77.85±0.10(1.40) | 76.89±0.17(0.45) | 74.69±0.22(1.75) | 76.83±0.06(0.39)
FATrain | 0.00±0.00 | 0.00±0.00(0.00) | 1.40±0.00(1.40) | 0.00±0.00(0.00) | 15.76±3.62(15.76) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
FATest | 0.00±0.00 | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 25.60±3.56(25.60) | 0.00±0.00(0.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
MIA | 26.84±1.56 | 9.64±0.87(17.20) | 48.60±0.00(21.76) | 5.00±1.02(21.84) | 0.00±0.00(26.84) | 4.60±0.87(22.24) | 0.44±0.20(26.40) | 30.56±0.54(3.72)
AvgGap | - | 4.32 | 6.43 | 5.53 | 17.40 | 5.67 | 7.04 | 1.03

Table 2: Sub-class unlearning on CIFAR-20 using ResNet18 with different
forgetting sub-classes. The results are given by $\bm{a_{\pm
b}({\color[rgb]{0,0,1}c})}$, sharing the same format with Tab. 1.

Class | Metric | Retrain | Finetune [42] | GA [18] | Amnesiac [13] | BadTeacher [14] | SalUn [16] | SSD [28] | NatMU
---|---|---|---|---|---|---|---|---|---
Rocket | RA | 84.94±0.14 | 84.12±0.27(0.81) | 80.60±0.00(4.34) | 84.80±0.09(0.14) | 85.25±0.05(0.32) | 84.93±0.07(0.00) | 84.69±0.12(0.24) | 84.94±0.25(0.00)
FATrain | 3.04±0.34 | 23.88±5.01(20.84) | 1.80±0.00(1.24) | 2.60±0.33(0.44) | 15.24±1.83(12.20) | 3.96±0.61(0.92) | 0.64±0.78(2.40) | 6.44±0.92(3.40)
FATest | 1.60±0.49 | 20.40±4.08(18.80) | 1.00±0.00(0.60) | 2.40±0.49(0.80) | 5.00±0.89(3.40) | 3.20±0.40(1.60) | 1.20±0.40(0.40) | 3.60±0.49(2.00)
MIA | 21.68±2.08 | 4.20±1.37(17.48) | 15.40±0.00(6.28) | 0.00±0.00(21.68) | 0.00±0.00(21.68) | 0.00±0.00(21.68) | 8.68±0.60(13.00) | 16.84±0.72(4.84)
AvgGap | - | 14.48 | 3.11 | 5.76 | 9.40 | 6.05 | 4.01 | 2.56
Vehicle2 | RA | 84.95±0.25 | 84.12±0.21(0.83) | 83.82±0.00(1.13) | 84.61±0.11(0.34) | 85.15±0.12(0.20) | 84.75±0.07(0.20) | 83.52±0.92(1.43) | 84.86±0.17(0.09)
FATrain | 45.72±2.75 | 73.24±1.20(27.52) | 63.00±0.00(17.28) | 42.84±2.04(2.88) | 53.56±2.40(7.84) | 46.80±2.75(1.08) | 31.92±33.59(13.80) | 46.16±3.77(0.44)
FATest | 46.20±3.19 | 63.60±3.38(17.40) | 46.00±0.00(0.20) | 34.40±2.15(11.80) | 45.60±0.49(0.60) | 35.00±1.10(11.20) | 24.60±22.49(21.60) | 50.20±4.26(4.00)
MIA | 25.60±3.51 | 23.24±1.49(2.36) | 27.20±0.00(1.60) | 0.08±0.10(25.52) | 0.00±0.00(25.60) | 0.20±0.00(25.40) | 10.28±12.97(15.32) | 18.08±1.77(7.52)
AvgGap | - | 12.03 | 5.05 | 10.14 | 8.56 | 9.47 |  13.04 | 3.01
Sea | RA | 84.66±0.10 | 84.17±0.16(0.49) | 83.70±0.00(0.96) | 84.42±0.08(0.24) | 85.08±0.07(0.42) | 84.53±0.09(0.13) | 83.55±0.42(1.11) | 84.91±0.15(0.25)
FATrain | 80.08±0.81 | 93.12±0.82(13.04) | 78.20±0.00(1.88) | 80.88±2.57(0.80) | 80.88±2.89(0.80) | 82.08±1.77(2.00) | 49.68±12.93(30.40) | 81.40±1.40(1.32)
FATest | 83.60±1.62 | 91.40±0.49(7.80) | 71.00±0.00(12.60) | 73.20±2.04(10.40) | 75.40±1.36(8.20) | 73.40±1.36(10.20) | 40.60±8.33(43.00) | 80.20±2.14(3.40)
MIA | 57.36±1.78 | 60.88±1.01(3.52) | 41.40±0.00(15.96) | 0.20±0.00(57.16) | 0.00±0.00(57.36) | 0.20±0.00(57.16) | 1.48±0.37(55.88) | 48.12±1.88(9.24)
AvgGap | - | 6.21 | 7.85 | 17.15 | 16.70 | 17.37 | 32.60 | 3.55

Evaluation metrics. Following [14, 16, 28, 43, 29], we evaluate MU methods
across three metrics: retain accuracy (RA), forgetting accuracy (FA) and MIA
ratio [44]. Retain accuracy calculates the accuracy on test data, which
measures the generalization ability. Forgetting accuracy evaluates the model’s
accuracy on the forgetting samples, which measures the effectiveness of MU. In
full/sub-class unlearning, we calculate accuracy on samples of the forgetting
class in training set and test set respectively, denoted as FATrain and
FATest. MIA provides another view to evaluate the effectiveness of MU
algorithms from the entropy of model outputs. MIA attempts to utilize model
outputs to determine whether a sample is in the training set. The attack ratio
means how much forgetting samples is recognized. Although the MIA used in this
paper cannot precisely infer membership, it is widely adopted in recent MU
research because it provides an additional perspective to assess the
differences between the unlearned and retrained model beyond forgetting
accuracy. More discussions about MIA are provided in Appendix C. We also
report the average performance gap with the retrained model of above metrics,
AvgGap, since our goal is not to maximize or minimize any single metric, but
rather to reduce the total performance gap with the retrain model across
different metrics.

Implement details. The only special parameter of NatMU is the scaling factor
$\delta$, which typically ranges from -0.1 to 0.1 in subsequent experiments.
Generally, more complex datasets require a larger scaling factor for effective
unlearning. For various baselines and NatMU, we search hyperparameters to
firstly ensure their forgetting accuracy is close to retraining. In this
situation, the MIA ration is usually lower than retrained model. Other details
can be found in Appendix A.

###  4.2 Results

Table 3: Random-subset unlearning on CIFAR-10 using VGG16-BN, CIFAR-100 using
ResNet18 and TinyImageNet-200 using ResNet34 under 1% forgetting ratio and 10%
forgetting ratio. The results are given by $\bm{a_{\pm
b}({\color[rgb]{0,0,1}c})}$, sharing the same format with Tab. 1.

| Dataset | Metric | Retrain | Finetune | Amnesiac [13] | BadTeacher [14] | SalUn [16] | NatMU
---|---|---|---|---|---|---|---|---
1% | C-10 | RA | 93.26±0.12 | 93.35±0.05(0.09) | 93.01±0.05(0.24) | 93.16±0.04(0.10) | 92.82±0.04(0.43) | 93.01±0.09(0.24)
FA | 92.80±0.80 | 98.72±0.48(5.92) | 92.52±0.59(0.28) | 91.60±0.68(1.20) | 93.00±0.52(0.20) | 93.08±0.41(0.28)
MIA | 80.16±0.73 | 86.88±1.26(6.72) | 45.00±0.75(35.16) | 51.52±0.85(28.64) | 57.06±0.88(23.10) | 71.32±0.83(8.84)
AvgGap | - | 4.24 | 11.89 | 9.98 | 7.91 | 3.12
C-100 | RA | 76.18±0.23 | 76.07±0.21(0.11) | 76.17±0.12(0.01) | 76.16±0.11(0.02) | 76.15±0.10(0.03) | 76.62±0.22(0.44)
FA | 75.32±0.79 | 86.28±0.81(10.96) | 75.68±0.57(0.36) | 74.20±0.70(1.12) | 74.60±1.30(0.72) | 75.08±0.81(0.24)
MIA | 55.24±0.56 | 66.24±1.51(11.00) | 7.32±0.48(47.92) | 2.36±0.27(52.88) | 22.02±0.72(33.22) | 41.72±0.93(13.52)
AvgGap | - | 7.36 | 16.10 | 18.01 | 11.32 | 4.73
Tiny | RA | 66.93±0.23 | 66.69±0.26(0.24) | 65.59±0.11(1.34) | 66.35±0.10(0.59) | 65.59±0.10(1.34) | 66.74±0.20(0.20)
FA | 68.00±1.56 | 85.80±0.57(17.80) | 67.30±0.35(0.70) | 70.56±0.71(2.56) | 69.10±0.32(1.10) | 68.68±0.72(0.68)
MIA | 49.06±1.18 | 63.68±0.77(14.62) | 0.90±0.09(48.16) | 0.00±0.00(49.06) | 10.69±0.45(38.37) | 36.76±0.55(12.30)
AvgGap | - | 10.89 | 16.73 | 17.40 | 13.60 | 4.39
10% | C-10 | RA | 93.09±0.09 | 93.34±0.10(0.25) | 92.79±0.07(0.29) | 92.91±0.06(0.18) | 92.62±0.10(0.46) | 92.16±0.07(0.93)
FA | 93.64±0.20 | 98.74±0.06(5.09) | 93.56±0.20(0.08) | 93.44±0.21(0.20) | 93.15±0.18(0.50) | 93.88±0.17(0.23)
MIA | 80.59±0.43 | 88.66±0.18(8.06) | 36.37±0.46(44.22) | 32.64±0.45(47.95) | 40.13±0.55(40.46) | 73.80±0.28(6.79)
AvgGap | - | 4.47 | 14.87 | 16.11 | 13.81 |  2.65
C-100 | RA | 75.68±0.12 | 75.82±0.18(0.14) | 72.60±0.11(3.08) | 74.34±0.10(1.34) | 72.18±0.17(3.50) | 75.18±0.21(0.50)
FA | 74.76±0.27 | 86.47±0.14(11.72) | 74.59±0.23(0.17) | 74.42±0.72(0.34) | 75.14±0.68(0.38) | 74.50±0.60(0.26)
MIA | 55.05±0.40 | 64.99±0.32(9.94) | 4.01±0.04(51.04) | 0.10±0.00(54.95) | 17.38±0.18(37.67) | 38.92±0.19(16.14)
AvgGap | - | 7.26 | 18.10 | 18.88 | 13.85 | 5.63
Tiny | RA | 65.47±0.20 | 66.60±0.11(1.13) | 61.62±0.19(3.85) | 63.54±0.08(1.93) | 60.98±0.15(4.49) | 64.87±0.15(0.60)
FA | 66.00±0.14 | 85.66±0.17(19.66) | 66.65±0.33(0.65) | 66.17±0.31(0.16) | 66.79±0.52(0.79) | 65.72±0.33(0.28)
MIA | 46.75±0.72 | 62.91±0.53(16.16) | 1.25±0.05(45.50) | 0.00±0.00(46.75) | 10.96±0.19(35.79) | 34.66±0.35(12.09)
AvgGap | - | 12.32 | 16.66 | 16.28 | 13.69 | 4.33

Full-class and sub-class unlearning. Tab. 1 and Tab. 2 show the performance of
different methods using ResNet18 model on CIFAR-100/CIFAR-20. As indicated by
the AvgGap metric, NatMU achieves the smallest average performance gap in two
class-unlearning scenarios across six different class-unlearning settings. In
full-class unlearning, all MU methods can obtain good performances while NatMU
can achieve a smaller performance gap, especially on MIA metric. Sub-class
unlearning becomes more complex than full-class unlearning. When the
forgetting sub-class varies, the behavior of the retrained model on these sub-
class data is completely different. However, our approach not only
demonstrates good performances but also exhibits remarkable stability across
various classes.

Random-subset unlearning. Tab. 3 demonstrates the performance of different
methods in random-subset unlearning. Apart from the CIFAR-10 dataset, NatMU
achieves the smallest performance gap across all other datasets and settings.
On CIFAR-10 dataset, since the forgetting accuracy and MIA ratio are very
high, although Finetune cannot guarantee the achievement of unlearning
objectives, it achieves the smallest average performance gap. NatMU can also
obtain a second smallest performance gap with a much lower gap on FA metric.
Because the compared three optimization-based methods assign incorrect labels
to the forgetting samples, the predictions after the unlearning process is
totally different from the remaining samples, resulting in an extremely low
MIA. It can lead to a “Streisand Effect”, where the forgetting samples is
actually more noticeable [26]. Moreover, compared to these three methods,
NatMU enjoys a significantly smaller RA decrease on more complex dataset and
larger forgetting ratio, since the conflicts in our fine-tuning data are much
fewer.

Performance with sufficient training. In real-world applications, without the
retrained model as a hyperparameter tuning reference, the user may need to
fine-tune the model with more epochs to ensure effective unlearning. To
demonstrate the results in real-world scenarios, we extend the training epochs
from 5 to 20 with other hyperparameters unchanged to realize a sufficient
training. As shown in Tab. 4(c), other methods face a significant decrease in
FATrain and FATest caused by the over-forgetting problem, while NatMU still
has a small performance gap across all metrics. More results are provided in
Appendix D.3.

Robustness to hyperparameters. Another solution for the absence of the
retrained model as the hyperparameter tuning reference is to identify a set of
hyperparameters under a specific experimental settings and transfer them to
other settings. Fig. 4(a) illustrates the performance of different methods
when transferring the optimal hyperparameters with 10% forgetting ratio to 1%.
Due to the existence of remaining data, RA of four methods remains almost
unchanged, thus not reported here. In case of FA and MIA, NatMU exhibits
strong hyperparameter robustness compared to other baselines, making it a
promising candidiate for pratical MU. More results are provided in Appendix
D.2.

(a)

Metric | Retrain | AMN [13] | BT [14] | SU [16] | NatMU
---|---|---|---|---|---
RA | 84.95 | 84.63(0.32) | 85.65(0.71) | 84.70(0.25) | 85.13(0.18)
FATrain | 45.72 | 0.00(45.72) | 6.28(39.44) | 0.08(45.64) | 44.00(1.72)
FATest | 46.20 | 17.20(29.00) | 35.60(10.60) | 17.60(28.60) | 50.40(4.20)
MIA | 25.60 | 7.00(18.60) | 0.00(25.60) | 2.12(23.48) | 22.92(2.68)
AvgGap | - | 23.41 | 19.09 | 24.49 |  2.20

(b)

Metric | Retrain | NatMU-RL | NatMU-NRI | NatMU
---|---|---|---|---
RA | 76.18±0.23 | 76.66±0.28(0.48) | 76.41±0.09(0.23) | 76.62±0.22(0.44)
FA | 75.32±0.79 | 1.16±0.15(74.16) | 58.00±1.04(17.32) | 75.08±0.81(0.24)
MIA | 55.24±0.56 | 0.00±0.00(55.24) | 22.72±1.34(32.52) | 41.72±0.93(13.52)
AvgGap | - | 43.29 | 16.69 | 4.73

(c)

Figure 4:  (a) Performance under a forgetting ratio of 1% on CIFAR-100, with
the optimal hyperparameters and hyperparameters from a forgeting ratio of 10%.
(b) Performance of different methods with sufficient training. The experiments
are conducted on CIFAR-20 with a forgetting sub-class of “Vehicle2”. (c)
Ablation study on correct information and weight masks on CIFAR-100 with a
forgetting ratio of 1%. The results are given by
$\bm{a({\color[rgb]{0,0,1}c})}$, sharing the same format with Tab. 1. For
simplicity, variance is not reported here. AMN: Amnesiac, BT: BadTeacher,
SU:SalUn.

###  4.3 Ablation

We study the rule of injected information by replacing
$\mathcal{T}_{\bm{m}}(\bm{x}^{f},\bm{x}^{r})$ with
$\mathcal{T}_{\bm{m}}(\bm{x}^{f},\bm{0}_{d})=\bm{x}^{f}\circ\bm{m}$, denoted
as NatMU-NRI (No Remaining Information). We also replace the weight masks
$\bm{m}$ in NatMU-NRI with $\bm{1}_{d}$ which generates unlearning instances
$\\{(\bm{x}^{f},y^{r}_{j})\\}_{j=1}^{n=4}$ denoted as NatMU-RL (Random
Labeling). It is actually a modified version of Amnesiac which generates 4
different randomly labeled samples for one forgetting sample. As shown in Tab.
4(c), NatMU-RL has an extremely low FA and MIA, since its relabeling can lead
to the unnatural property as discussed before. NatMU-NRI can reduce the
unntural property by relabeling the partial forgetting sample
$\bm{x}^{f}\circ\bm{m}$. However, without the injected correct information, it
also face a significant performance decrease compare to NatMU.

##  5 Conclusion

Revisiting current popular machine unlearning methods, we identify their
unnatural properties and resulting issues, such as unnatural generalization
and impracticality. To address these issues, we propose a straightforward but
effective machine unlearning method, NatMU, which injects correct information
from remaining data into forgetting samples to achieve unlearning. NatMU
demonstrates superior performance and robustness across different unlearning
settings. Our initial step towards natural machine unlearning opens up new
perspectives for achieving more efficient and effective MU.

Limitations and future work. Firstly, although NatMU performs well, it lacks
theoretical support, which means it is not certified. This weakness is also
shared by the compared baseline methods. Secondly, the weighting vectors are
currently designed manually; thus, there is potential for improving, which we
leave for future works. Thirdly, our algorithm needs to access the whole
remaining data. Enhancements in efficiency could be achieved through methods
such as dataset distillation [45].

## References

  * [1] Hengshuang Zhao, Jiaya Jia, and Vladlen Koltun.  Exploring self-attention for image recognition.  In CVPR, pages 10073–10082. Computer Vision Foundation / IEEE, 2020.
  * [2] Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui.  Graph neural networks in recommender systems: A survey.  ACM Comput. Surv., 55(5):97:1–97:37, 2023.
  * [3] Michael I Jordan and Tom M Mitchell.  Machine learning: Trends, perspectives, and prospects.  Science, 349(6245):255–260, 2015.
  * [4] Julia Hirschberg and Christopher D Manning.  Advances in natural language processing.  Science, 349(6245):261–266, 2015.
  * [5] Chris Jay Hoofnagle, Bart Van Der Sloot, and Frederik Zuiderveen Borgesius.  The european union general data protection regulation: what it is and what it means.  Information & Communications Technology Law, 28(1):65–98, 2019.
  * [6] Antonio Ginart, Melody Y. Guan, Gregory Valiant, and James Zou.  Making AI forget you: Data deletion in machine learning.  In NeurIPS, pages 3513–3526, 2019.
  * [7] Chuan Guo, Tom Goldstein, Awni Y. Hannun, and Laurens van der Maaten.  Certified data removal from machine learning models.  In ICML, volume 119 of Proceedings of Machine Learning Research, pages 3832–3842. PMLR, 2020.
  * [8] Enayat Ullah, Tung Mai, Anup Rao, Ryan A. Rossi, and Raman Arora.  Machine unlearning via algorithmic stability.  In COLT, volume 134 of Proceedings of Machine Learning Research, pages 4126–4142. PMLR, 2021.
  * [9] Goel Shashwat, Prabhu Ameya, Sanyal Amartya, Lim Ser-Nam, Torr Philip, and Kumaraguru Ponnurangam.  Towards adversarial evaluations for inexact machine unlearning.  CoRR, abs/2201.06640, 2022.
  * [10] Vikram S. Chundawat, Ayush K. Tarun, Murari Mandal, and Mohan S. Kankanhalli.  Zero-shot machine unlearning.  IEEE Trans. Inf. Forensics Secur., 18:2345–2354, 2023.
  * [11] Stefan Schoepf, Jack Foster, and Alexandra Brintrup.  Parameter-tuning-free data entry error unlearning with adaptive selective synaptic dampening.  CoRR, abs/2402.10098, 2024.
  * [12] Ga Wu, Masoud Hashemi, and Christopher Srinivasa.  PUMA: performance unchanged model augmentation for training data removal.  In AAAI, pages 8675–8682. AAAI Press, 2022.
  * [13] Laura Graves, Vineel Nagisetty, and Vijay Ganesh.  Amnesiac machine learning.  In AAAI, pages 11516–11524. AAAI Press, 2021.
  * [14] Vikram S. Chundawat, Ayush K. Tarun, Murari Mandal, and Mohan S. Kankanhalli.  Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher.  In AAAI, pages 7210–7217. AAAI Press, 2023.
  * [15] Pratyush Maini, Michael Curtis Mozer, Hanie Sedghi, Zachary Chase Lipton, J. Zico Kolter, and Chiyuan Zhang.  Can neural network memorization be localized?  In ICML, volume 202 of Proceedings of Machine Learning Research, pages 23536–23557. PMLR, 2023.
  * [16] Chongyu Fan, Jiancheng Liu, Yihua Zhang, Dennis Wei, Eric Wong, and Sijia Liu.  Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation.  CoRR, abs/2310.12508, 2023.
  * [17] Anvith Thudi, Hengrui Jia, Ilia Shumailov, and Nicolas Papernot.  On the necessity of auditable algorithmic definitions for machine unlearning.  In USENIX Security Symposium, pages 4007–4022. USENIX Association, 2022.
  * [18] Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot.  Unrolling SGD: understanding factors influencing machine unlearning.  In EuroS&P, pages 303–319. IEEE, 2022.
  * [19] Jie Xu, Zihan Wu, Cong Wang, and Xiaohua Jia.  Machine unlearning: Solutions and challenges.  CoRR, abs/2308.07061, 2023.
  * [20] Yinzhi Cao and Junfeng Yang.  Towards making systems forget with machine unlearning.  In IEEE Symposium on Security and Privacy, pages 463–480. IEEE Computer Society, 2015.
  * [21] Pang Wei Koh and Percy Liang.  Understanding black-box predictions via influence functions.  In ICML, volume 70 of Proceedings of Machine Learning Research, pages 1885–1894. PMLR, 2017.
  * [22] Ryutaro Tanno, Melanie F. Pradier, Aditya V. Nori, and Yingzhen Li.  Repairing neural networks by leaving the right past behind.  In NeurIPS, 2022.
  * [23] Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou.  Approximate data deletion from machine learning models.  In AISTATS, volume 130 of Proceedings of Machine Learning Research, pages 2008–2016. PMLR, 2021.
  * [24] Ronak Mehta, Sourav Pal, Vikas Singh, and Sathya N. Ravi.  Deep unlearning via randomized conditionally independent hessians.  In CVPR, pages 10412–10421. IEEE, 2022.
  * [25] Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck.  Machine unlearning of features and labels.  In NDSS. The Internet Society, 2023.
  * [26] Aditya Golatkar, Alessandro Achille, and Stefano Soatto.  Eternal sunshine of the spotless net: Selective forgetting in deep networks.  In CVPR, pages 9301–9309. Computer Vision Foundation / IEEE, 2020.
  * [27] James Martens.  New insights and perspectives on the natural gradient method.  J. Mach. Learn. Res., 21:146:1–146:76, 2020.
  * [28] Jack Foster, Stefan Schoepf, and Alexandra Brintrup.  Fast machine unlearning without retraining through selective synaptic dampening.  In AAAI, pages 12043–12051. AAAI Press, 2024.
  * [29] Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou.  Towards unbounded machine unlearning.  In NeurIPS, 2023.
  * [30] Xinwen Cheng, Zhehao Huang, and Xiaolin Huang.  Machine unlearning by suppressing sample contribution.  CoRR, abs/2402.15109, 2024.
  * [31] Jack Foster, Kyle Fogarty, Stefan Schoepf, Cengiz Öztireli, and Alexandra Brintrup.  Zero-shot machine unlearning at scale via lipschitz regularization.  CoRR, abs/2402.01401, 2024.
  * [32] Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, and Moontae Lee.  Learning to unlearn: Instance-wise unlearning for pre-trained classifiers.  In AAAI, pages 11186–11194. AAAI Press, 2024.
  * [33] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [34] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals.  Understanding deep learning requires rethinking generalization.  In ICLR. OpenReview.net, 2017.
  * [35] Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz.  mixup: Beyond empirical risk minimization.  In ICLR (Poster). OpenReview.net, 2018.
  * [36] Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Seong Joon Oh, Youngjoon Yoo, and Junsuk Choe.  Cutmix: Regularization strategy to train strong classifiers with localizable features.  In ICCV, pages 6022–6031. IEEE, 2019.
  * [37] Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojciech Samek.  Layer-wise relevance propagation for neural networks with local renormalization layers.  In ICANN (2), volume 9887 of Lecture Notes in Computer Science, pages 63–71. Springer, 2016.
  * [38] Solomon Kullback and Richard A Leibler.  On information and sufficiency.  The annals of mathematical statistics, 22(1):79–86, 1951.
  * [39] Mohammed Ali mnmoustafa.  Tiny imagenet, 2017.
  * [40] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  In ICLR, 2015.
  * [41] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, pages 770–778. IEEE Computer Society, 2016.
  * [42] Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck.  Machine unlearning of features and labels.  In NDSS. The Internet Society, 2023.
  * [43] Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu.  Model sparsity can simplify machine unlearning.  In NeurIPS, 2023.
  * [44] Liwei Song and Prateek Mittal.  Systematic evaluation of privacy risks of machine learning models.  In USENIX Security Symposium, pages 2615–2632. USENIX Association, 2021.
  * [45] Junaid Iqbal Khan.  Dataset condensation driven machine unlearning.  CoRR, abs/2402.00195, 2024.
  * [46] Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli.  Fast yet effective machine unlearning.  IEEE Transactions on Neural Networks and Learning Systems, 2023.
  * [47] Ilya Loshchilov and Frank Hutter.  SGDR: stochastic gradient descent with warm restarts.  In ICLR (Poster). OpenReview.net, 2017.
  * [48] Christopher J. Anders, David Neumann, Wojciech Samek, Klaus-Robert Müller, and Sebastian Lapuschkin.  Software for dataset-wide xai: From local explanations to global insights with Zennit, CoRelAy, and ViRelAy.  CoRR, abs/2106.13200, 2021.

## Appendix

##  Appendix A Training Details

In full-class/sub-class unlearning, both the original and the retrained models
are trained on the CIFAR-100/CIFAR-20 dataset for a total of 100 epochs with a
warm-up period of 2 epochs, using ResNet18. We use SGD as the optimizer, and a
step learning rate schedule is implemented, where the learning rate decays at
the 80th and 90th epochs with ratio 0.1. The initial learning rate is 0.1,
weight decay is configured at 0.0005, and the batch size is 128.
RandomCrop(32, padding=4) and RandomHorizonFlip(0.5) are adopted as data
augmentation.

In random-subset unlearning, we train VGG16-BN on CIFAR-10, ResNet18 on
CIFAR-100, and ResNet34 on TinyImageNet-200. The experimental configuration is
the same as full-class/sub-class unlearning and the two main difference are
that we use a batch size of 256 to train ResNet34 on TinyImangeNet-200, and
RandomCrop(64, padding=8) to replace the RandomCrop(32, padding=4) in above
settings.

Computer resources. All the experiments are conducted in docker container with
Python version 3.8.8, PyTorch version 1.12.0+cu113, a RTX4090 GPU, CPU with 40
cores and 256GB memory.

###  A.1 Details of Baselines

When optimizing the hyperparameters for compared baselines, we prioritize to
make the forgetting accuracy close to the retrained model. We all use the
cosine learning scheduler for these baselines except GA and SSD. To enhance
the algorithm stability, data augmentation is not adopted and weight decay is
set to 0, since we have observed that under some settings (for example,
random-subset unlearning with a forgetting ratio of 10%), data augmentation
can result a significant decrease on RA. We usually use AdamW as the default
optimizer. In some tasks, we also use SGD for a better performance. The batch
size is set the same with the retrained model and original model except GA and
SSD.

For GA and SSD, we use a large batch size of 500 to enhance algorithm
stability. For GA, we use a constant learning rate 0.001 and search the
training epoch to achieve a good performance. For SSD, we search the
hyperparameter $\alpha$ and set $\lambda=1$ in their paper. Although they
recommend a range $[0.1,100]$ for $\alpha$, we found for most tasks, the best
$\alpha$ in our setting is more than 100, usually in $[200,550]$.

For Finetune, we use a large learning rate to impair the model at the initial
training stage.

For Amnesiac, BadTeacher, SalUn, we all fine-tune the original model for 5
epochs, especially in a few experiments. For SalUn, we also search the mask
ratio in $[0.1,0.9]$.

In full-class and sub-class unlearning, we provide an option to initialize the
last linear layer before unlearning for Finetune and Amnesiac, following the
idea in [9]. We found it helpful for efficient unlearning with some forgetting
class.

We referenced the code available at https://github.com/OPTML-Group/Unlearn-
Saliency and https://github.com/if-loops/selective-synaptic-dampening.

###  A.2 Details of NatMU

Following the impair-repair idea in [46], we use a high learning rate in the
initial stages of training to promote model forgetting, where the learning
rate gradually decays to zero using the cosine learning rate scheduler [47].
In full-class and sub-class unlearning, we initialize the last linear layer
before unlearning, as we implement in Finetune and Amnesiac. It is helpful for
efficient unlearning.

In most unlearning settings, we use a optimizer of AdamW, a learning rate of
4e-4, a weight decay of 5e-4, and a learning rate scheduler of cosine. We
fine-tune the original model for 5 epochs as baselines do. We also adopt
RandomCrop and RandomHorizonFlip(0.5) as data augmentation for remaining data
to improve the generalization ability, and no augmentation for unlearning
samples. We use a padding of 8 for TinyImageNet-200 and a padding of 2 for
other datasets.

Weighting mask. To facilitate understanding, the weighting vector $\bm{m}$ can
be reshaped into a $H\times W$ matrix $\bm{M}$ to align with the two-
dimensional structure of images. The elements in each column share the same
value and the value of two edge columns is 0 while that of center edges is 1.
The values in other columns transition uniformly from 0 on the edges to 1 in
the center. $\bm{M}$ can be calculated with:

$\bm{M}[\cdot,i]=\left\\{\begin{aligned} &\frac{2}{W-2}(i-1),\hfill 1\leq
i\leq W/2,\\\ &\frac{2}{W-2}(W-i),\hfill W/2\leq i\leq W,\end{aligned}\right.$
(A1)

where $\bm{M}[\cdot,i]$ donates the value of the $i$-th column and
$i\in\\{1,2,\cdots,W\\}$. In order to unlearn the forgetting sample from
multiple perspectives, we can perform a rotation or take the complement of
$\bm{m}$ to obtain different weight vectors. Let $\bm{m}_{1}=\bm{m}$, we can
obtain $\bm{m}_{2}$ with complement operation
$\bm{m}_{2}=\bm{1}_{d}-\bm{m}_{1}$, and rotate them by 90 degrees to obtain
$\bm{m}_{3}$ and $\bm{m}_{4}$. The only important parameter of NatMU is the
scaling factor $\delta$, which controls the values of weighting vectors. We
search it in $[-0.5,0.5]$ and it is usually set between $[-0.1,0.1]$. Fig.A1
visualize the 4 weighting vectors to facilitate understanding.

We can see that, a weighting mask captures the entire pixels of the forgetting
samples. Moreover, different weighting masks focus on different parts of the
images. We also have tried other weighting masks, such constant weighting mask
in MixUp [35] or area weighting mask in CutMix [36]. Experiments show that
this used implementation has a better robustness across different unlearning
scenarios.

(a) $m_{1}$.

(b) $m_{2}$.

(c) $m_{3}$.

(d) $m_{4}$

Figure A1: Visualization of weighting vectors.

Algorithm. The algorithm for generating the fine-tuning dataset for NatMU is
provided in Algorithm 1. After generating the dataset, we can fine-tine the
original model with it, using the setups described in Appendix A.2.

Algorithm 1 Construct Fine-tuning Dataset

Input: remaining set $D^{r}$, forgetting set $D^{f}$, original model
$f_{\theta_{o}}$, weighting vectors $\\{\bm{m}_{j}\\}_{j=1}^{n}$, scaling
factor $\delta$.

Output: fine-tuning dataset $D_{\mathrm{Nat}}$

1:for $j\leftarrow 1$ to $n$ do

2: Generate $\bm{m}_{j}^{\mathrm{scaled}}$ according Eq.1.

3:end for

4:$D_{\mathrm{Nat}}\leftarrow D^{r}$

5:for $(\bm{x}_{i},y_{i})\in D^{f}$ do

6: Generate $\mathcal{D}^{f}_{i}$ according to Eq.2.

7: $D_{\mathrm{Nat}}\leftarrow D_{\mathrm{Nat}}\cup\mathcal{D}^{f}_{i}$

8:end for

9:return $D_{\mathrm{Nat}}$

Unlearning efficiency. Although we do not specifically address the runtime of
the algorithm in this paper, the efficiency of NatMU is comparable to other
optimization-based methods. For instance, SalUn fine-tunes the original model
by using 5% (10/200) of the epochs for retraining in the official
implementation, and NatMU similarly employs 5% (5/100) of the retraining cost.
In simple forgetting scenarios, such as full-class/sub-class unlearning or
random-subset unlearning on the CIFAR dataset, NatMU can operate with lower
computational costs, requiring only three epochs.

#### Broader impact

With the rapid development of deep learning, its demand for large amounts of
training data has raised concerns about personal privacy. Retraining models in
large-scale deep learning is impractical when removing certain private,
harmful, or sensitive data. Our work takes a step towards eliminating the
impact of these data through post-processing of models. Our method effectively
and efficiently removes specific data while approximating the retrained
model’s performance, thus maintaining good utility, making it a promising
candidate for piratical machine unlearning. However, NatMU is not a certified
machine unlearning method. Therefore, its implementation in practical
applications necessitates careful auditing.

##  Appendix B Overview of Different Methods and Takeaways

In full-class unlearning, Finetune is a good and simple MU method with the
initialization of linear classifier layer [9]. SSD is the most efficient
method among all well-performing methods, since it only accesses the whole
training set once. In contrast, GA has a inferior performance with better
efficiency. Amnesiac and SalUn are also good, while BadTeacher cannot unlearn
the forgetting class in some settings. However, BadTeacher has an increase of
RA since it distills the knowledge of original model into unlearned model with
meaningful soft labels. NatMU achieves the best performance gap, as it owns a
closer MIA, while other metrics are easier to approximate in full-class
unlearning.

In sub-class unlearning, Finetune is a poor method since it fails to forget
the sub-class. GA surprisingly has a good performance gap across multiple
metrics, particularly in these sub-class whose samples can be still correctly
classified, indicating its potential in this unlearning scenario. SSD exhibits
sensitivity to hyperparamters in our implementation, although we adopt
techniques like enlarging the batch size for an accurate gradient estimation.
Amnesiac can effectively finish the forgetting with almostly unchanged RA,
while its MIA is close to zero. SalUn improves Amnesiac on MIA, achieving a
smaller performance gap. BadTeacher’s MIA consistently remains zero for
different forgetting sub-classes. Additionally, for certain sub-classes, it
has the issue of under-forgetting, close to its performance in full-class
unlearning. NatMU also consistently maintains the smallest performance gap
across different forgetting sub-classes, but its performance is less stable in
certain cases.

In random-subset unlearning, both GA and SSD fails to realize the effective
unlearning while remaining a decent performance in our implementation. The
underlying reason is that the gradients of forgetting set and remaining set
are quite close, while these two method rely on the difference between the
gradients. Finetune still fails to forget, although its average performance
gap is the smallest on CIFAR-10. Optimization-based methods demonstrate
exceptional performance in this scenario. However, Amnesiac and SalUn has a
lower RA, since they introduce great data conflicts into the fine-tuning
dataset by randomly hard-labeling the forgetting samples. The conflicts not
only affects the generalization on forgetting samples as discussed before, but
also can affect the generalization on remaining data, leading to the decrease
of RA on complex datasets and larger forgetting ratio. Compared to them, the
knowledge distillation on remaining data helps BadTeacher remains a better
performance. Moreover, the uniformly distributed prediction given by the
randomly initialization model on forgetting samples also helps it remain a
closer FA to the retrained model. The reason is that the predictions for most
forgetting samples remain between the original one-hot predictions and uniform
predictions. In this case, their predicted labels are still the original
labels, despite having much lower confidence. This mechanism benefits it in
random-subset unlearning, but can lead to under-forgetting in full/sub-class
unlearning. All these three methods exhibit a nearly-zero MIA. Consistently,
NatMU has the best performance gap across different datasets and settings.

The relationship between FA and MIA in optimization-based methods. When the
learning rate is increased from zero, both FA and MIA decrease with MIA
decreasing more quickly. When MIA approaches the level of retraining, FA
remains significantly higher than that of retraining. We believe that, FA is
much more important than MIA in the context of “unlearning”. Therefore, in our
experiments, we select a learning rate where FA is close to retraining.
However, at this learning rate, MIA tends to be lower, particularly for
Amnesiac and BadTeacher. More discussions about MIA are provided in the next
section.

##  Appendix C Discussion about MIA.

The MIA we used in this paper is a prediction-based MIA method which judge the
membership of a samples according to the prediction of the model on it [44].
Give a model $f_{\theta}(\cdot)$, it first calculates the entropy values of
shadow training set, i.e., $\mathcal{D}^{r}$ in unlearning framework, and
entropy values of shadow test set, i.e., the test set in unlearning framework.
Then, the entropy values of shadow training set are labeled as “member” and
those of test set are labeled as “non-member” to train a binary classifier.
Finally, for a forgetting sample $\bm{x}^{f}$ to be detected, the entropy of
$f_{\theta}(\bm{x}^{f})$ is used to predict the membership with the trained
classifier.

Usually, the training samples have been fitted by the model, so they are
predicted with high confidence, corresponding to low entropy. While the test
samples usually are predicted with a lower confidence, corresponding to a high
entropy. Basically, the binary classifier is to find a decision point to
determine membership. As a result, if we randomly label a forgetting sample
and fine-tune the model on it, the entropy of the model’s prediction for this
sample would exhibit an inverted U-shaped curve as the training progresses.
Initially, the entropy is low because the model predicts the original category
with high confidence. Then, the entropy increases as the model’s prediction
for this sample transitions from the initial category to the newly assigned
category. Finally, with sufficient training, the entropy decreases again
because the model predicts the sample as the newly assigned category with high
confidence.

For BadTeacher, the soft labels of remaining samples are close to one-hot
labels, which has a low entropy. While the soft labels of forgetting samples
are uniformly distributed with a high entropy. As a result, the MIA ratio of
forgetting samples is always zero.

##  Appendix D More Experimental Results.

###  D.1 Visual Results.

Fig.A3 provides more visual results about unlearning instances and their
attentions. The attention map is calculated by LRP [37], implemented by Zennit
[48]. We choose LRP because it can generate pixel-wise attention maps, which
are more effective in handling cases where image subjects overlap. Before
normalizing the attention map to the image range $[0,255]$, we apply the ReLU
activation function to process the attention map, preventing the image
background from being elevated by some negative outliers. To generate
attention maps, a VGG16-BN is trained on a subset of ImageNet-1K consisting of
20 classes. Then, the model is fine-tuned to forget a full class. In some
unlearning samples, the main subject of $\bm{x}^{f}$ is not prominent,
resulting in attention maps that highlight only the relevant parts of
$\bm{x}^{r}$.

(a)

(b)

(c)

(a)

(b)

(c)

Figure A3: Visualization of unlearning instances and attention maps. For each
sub-figure, from left to right, each column corresponds to: $\bm{x}^{f}$,
$\bm{x}^{f}\circ\bm{m}_{j}$, $\bm{x}^{r}$,
$\bm{x}^{r}\circ(\bm{1}_{d}-\bm{m}_{j})$,
$\mathcal{T}_{\bm{m}_{j}}(\bm{x}^{f},\bm{x}^{r})$, attention maps of the
original model for $\mathcal{T}_{\bm{m}_{j}}(\bm{x}^{f},\bm{x}^{r})$,
attention maps of the unlearned model for
$\mathcal{T}_{\bm{m}_{j}}(\bm{x}^{f},\bm{x}^{r})$, attentions map of the
retrained model for $\mathcal{T}_{\bm{m}_{j}}(\bm{x}^{f},\bm{x}^{r})$. In (a)
and (b), the unlearning class is "Brambling". In (c) and (d), the unlearning
class is "Bottlecap". In (e) and (f), the unlearning class is "Spike".  Table
A1:  Performance of different methods under different unlearning settings with
same hyperparameters. The results are given by
$\bm{a({\color[rgb]{1,0,0}d})}$, where $\bm{a}$ denotes the mean value with
hyperparameters transfered from another setting, while $\bm{d}$ denotes the
performance change with that with optimal hyperparameters compared to
$\bm{a}$, not the retrained model. AvgChange means the performance change of
two sets of hyperparameters. FC/SC: transferring the hyperparameters of
class/sub-class "Rocket" to class/sub-class "Sea" under Full-Class/Sub-Class
unlearning on CIFAR-100/CIFAR-20. RS: transferring the hyperparameters of a
forgetting ratio of $10\%$ to a forgetting ratio of $1\%$ on different
datasets in random-subset unlearning.

Setting | Metric | Retrain | Amnesiac [13] | BadTeacher [14] | SalUn [16] | Ours
---|---|---|---|---|---|---
FC | RA | 76.44 | 76.99(0.81) | 76.96(0.88) | 76.87(0.02) | 76.54(0.29)
FATrain | 0.00 | 0.00(0.00) | 35.40(19.64) | 0.00(0.00) | 0.00(0.00)
FATest | 0.00 | 0.00(0.00) | 17.40(8.20) | 0.00(0.00) | 0.00(0.00)
MIA | 26.84 | 3.88(1.12) | 0.00(0.00) | 3.80(0.80) | 29.80(0.76)
AvgChange | - | 0.48 | 7.18 | 0.20 | 0.26
SC | RA | 84.66 | 84.50(0.08) | 85.00(0.08) | 84.83(0.30) | 84.83(0.08)
FATrain | 80.08 | 70.16(10.72) | 83.80(2.92) | 69.28(12.80) | 74.36(7.04)
FATest | 83.60 | 64.60(8.60) | 77.00(1.60) | 62.80(10.60) | 73.00(7.20)
MIA | 57.36 | 0.08(0.12) | 0.00(0.00) | 0.00(0.20) | 43.36(4.76)
AvgChange | - | 4.88 | 1.15 | 5.98 | 4.77
RS-C10 | RA | 93.26 | 92.96(0.05) | 93.18(0.02) | 92.88(0.05) | 93.09(0.08)
FA | 92.80 | 87.54(4.98) | 88.09(3.51) | 85.60(7.40) | 94.00(0.92)
MIA | 80.16 | 29.97(15.03) | 46.69(4.83) | 33.61(23.44) | 71.80(0.48)
AvgChange | - | 6.69 | 2.79 | 10.30 | 0.49
RS-C100 | RA | 76.18 | 75.99(0.18) | 76.59(0.44) | 75.98(0.17) | 76.67(0.06)
FA | 75.32 | 28.97(46.71) | 39.67(34.53) | 37.97(36.63) | 73.84(1.24)
MIA | 55.24 | 1.53(5.79) | 0.00(2.36) | 16.57(5.45) | 40.44(1.28)
AvgChange | - | 17.56 | 12.44 | 14.08 | 0.86
RS-Tiny | RA | 66.93 | 65.65(0.05) | 66.89(0.54) | 65.50(0.09) | 66.40(0.34)
FA | 68.00 | 23.75(43.55) | 36.63(33.93) | 29.82(39.28) | 64.74(3.94)
MIA | 49.06 | 3.52(2.62) | 0.00(0.00) | 13.11(2.42) | 35.62(1.14)
AvgChange | - | 15.41 | 11.49 | 13.93 |  1.81

###  D.2 Robustness to Hyperparameters.

Tab.A1 shows the hyperparameter robustness of different optimization-based
methods under full-class, sub-class and random-subset scenarios. It can be
seen that in multiple unlearning scenarios, the performance of NatMU remains
consistent across different hyperparameters. Note that although BadTeacher has
a smaller average change in sub-class unlearning, its MIA ration remain zero
for different sets of hyperparameters, which is much different from that of
the retraiend model.

###  D.3 Results with Sufficient Training.

Tab.A2, Tab.A3 and Tab.A4 show the results of different optimization-based
methods with a sufficient training, where the training epoch changes form 5 to
20 with other hyperparameters unchanged. Other methods all face a significant
decrease of forgetting accuracy, since they assign incorrect labels to these
forgetting samples, damaging the natural generalization on these samples. We
can also see that, in some settings the MIA ratio of Amnesiac and SalUn is
higher than that in the main page. For example, in full-class unlearning,
their MIA is over 80%. In random-subset unlearning with a forgetting ratio of
10% on TinyImageNet-200, their MIA is even closer to the retrained model than
NatMU.

As discusses in AppendixC, when the optimal learning rate is large enough, the
unlearned model of Amnesiac and SalUn can predict the forgetting samples as
the reassigned labels with high confidence, so their MIA would also be high.
However, in this situation, their forgetting accuracy is zero, i.e., the over-
forgetting problem.

Table A2: Full-class unlearning on CIFAR-100 using ResNet18 of different
optimization-based methods, with a training epoch of 20. Results are given by
$\bm{a_{\pm b}({\color[rgb]{0,0,1}c})}$, sharing the same format with Tab.1.

Class | Metric | Retrain | Amnesiac [13] | BadTeacher [14] | SalUn [16] | Ours
---|---|---|---|---|---|---
Rocket | RA | 76.44±0.33 | 75.86±0.29(0.58) | 77.28±0.11(0.84) | 76.75±0.20(0.31) | 76.67±0.27(0.24)
FATrain | 0.00±0.00 | 0.00±0.00(0.00) | 17.76±1.57(17.76) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
FATest | 0.00±0.00 | 0.00±0.00(0.00) | 4.00±0.89(4.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
MIA | 14.64±0.77 | 79.16±0.83(64.52) | 0.00±0.00(14.64) | 81.16±3.10(66.52) | 12.68±1.36(1.96)
AvgGap | - | 16.27 | 9.31 |  16.71 | 0.55
Vehicle2 | RA | 76.43±0.27 | 76.47±0.10(0.04) | 78.02±0.21(1.59) | 76.92±0.27(0.48) | 76.81±0.17(0.38)
FATrain | 0.00±0.00 | 0.00±0.00(0.00) | 1.92±1.92(1.92) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
FATest | 0.00±0.00 | 0.00±0.00(0.00) | 4.40±1.36(4.40) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
MIA | 11.48±0.84 | 77.40±2.58(65.92) | 0.00±0.00(11.48) | 82.32±2.42(70.84) | 14.44±1.03(2.96)
AvgGap | - | 16.49 | 4.85 | 17.83 | 0.83
Sea | RA | 76.44±0.19 | 74.61±0.22(1.83) | 77.76±0.15(1.32) | 75.95±0.11(0.49) | 76.72±0.27(0.27)
FATrain | 0.00±0.00 | 0.00±0.00(0.00) | 15.12±3.99(15.12) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
FATest | 0.00±0.00 | 0.00±0.00(0.00) | 23.00±3.74(23.00) | 0.00±0.00(0.00) | 0.00±0.00(0.00)
MIA | 26.84±1.56 | 80.76±2.39(53.92) | 0.00±0.00(26.84) | 90.16±1.45(63.32) | 32.72±0.57(5.88)
AvgGap | - | 13.94 | 16.57 | 15.95 | 1.54

Table A3: Sub-class unlearning on CIFAR-20 using ResNet18 of different
optimization-based methods, with a training epoch of 20. Results are given by
$\bm{a_{\pm b}({\color[rgb]{0,0,1}c})}$, sharing the same format with Tab.1.

Class | Metric | Retrain | Amnesiac [13] | BadTeacher [14] | SalUn [16] | Ours
---|---|---|---|---|---|---
Rocket | RA | 84.94±0.14 | 84.69±0.09(0.24) | 85.76±0.06(0.83) | 84.72±0.06(0.22) | 85.08±0.13(0.15)
FATrain | 3.04±0.34 | 0.00±0.00(3.04) | 12.20±3.43(9.16) | 0.00±0.00(3.04) | 4.16±0.70(1.12)
FATest | 1.60±0.49 | 1.60±0.49(0.00) | 7.80±2.71(6.20) | 1.00±0.00(0.60) | 4.00±1.41(2.40)
MIA | 21.68±2.08 | 6.68±0.68(15.00) | 0.00±0.00(21.68) | 1.20±0.46(20.48) | 22.32±1.31(0.64)
AvgGap | - | 4.57 | 9.47 |  6.08 | 1.08
Vehicle2 | RA | 84.95±0.25 | 84.63±0.08(0.32) | 85.65±0.09(0.70) | 84.70±0.04(0.25) | 85.13±0.31(0.18)
FATrain | 45.72±2.75 | 0.00±0.00(45.72) | 6.28±3.57(39.44) | 0.08±0.10(45.64) | 44.00±2.30(1.72)
FATest | 46.20±3.19 | 17.20±0.75(29.00) | 35.60±2.65(10.60) | 17.60±1.62(28.60) | 50.40±2.15(4.20)
MIA | 25.60±3.51 | 7.00±0.78(18.60) | 0.00±0.00(25.60) | 2.12±0.27(23.48) | 22.92±1.80(2.68)
AvgGap | - | 23.41 | 19.09 | 24.49 | 2.20
Sea | RA | 84.66±0.10 | 84.43±0.04(0.22) | 85.54±0.14(0.88) | 84.68±0.05(0.02) | 85.12±0.18(0.46)
FATrain | 80.08±0.81 | 0.00±0.00(80.08) | 14.00±4.22(66.08) | 13.84±1.91(66.24) | 79.60±1.95(0.48)
FATest | 83.60±1.62 | 23.60±1.74(60.00) | 54.20±0.75(29.40) | 26.60±1.36(57.00) | 80.40±1.50(3.20)
MIA | 57.36±1.78 | 11.04±1.24(46.32) | 0.00±0.00(57.36) | 0.00±0.00(57.36) | 51.72±2.29(5.64)
AvgGap | - | 46.66 | 38.43 | 45.16 | 2.44

Table A4: Random-subset unlearning on CIFAR-10 with VGG16, CIFAR-100 using
ResNet18 and TinyImageNet-200 with ResNet34 under 1% forgetting ratio and 10%
forgetting ratio, with a training epoch of 20. The results are given by
$\bm{a_{\pm b}({\color[rgb]{0,0,1}c})}$, sharing the same format with Tab.1.

| Dataset | Metric | Retrain | Amnesiac [13] | BadTeacher [14] | SalUn [16] | Ours
---|---|---|---|---|---|---|---
1% | C-10 | RA | 93.26±0.12 | 92.44±0.07(0.82) | 93.11±0.05(0.15) | 92.32±0.04(0.94) | 93.35±0.19(0.09)
FA | 92.80±0.80 | 11.04±0.34(81.76) | 31.40±5.26(61.40) | 25.36±0.88(67.44) | 91.36±0.71(1.44)
MIA | 80.16±0.73 | 10.28±0.60(69.88) | 0.00±0.00(80.16) | 8.40±0.42(71.76) | 73.32±0.77(6.84)
AvgGap | - | 50.82 | 47.24 | 46.71 | 2.79
C-100 | RA | 76.18±0.23 | 76.08±0.06(0.10) | 76.82±0.07(0.64) | 75.72±0.13(0.45) | 76.74±0.17(0.56)
FA | 75.32±0.79 | 0.00±0.00(75.32) | 50.16±1.83(25.16) | 0.36±0.08(74.96) | 71.56±0.66(3.76)
MIA | 55.24±0.56 | 8.64±0.39(46.60) | 0.00±0.00(55.24) | 9.92±0.59(45.32) | 36.04±1.48(19.20)
AvgGap | - | 40.67 | 27.01 | 40.24 | 7.84
Tiny | RA | 66.93±0.23 | 64.74±0.09(2.20) | 67.00±0.06(0.06) | 64.37±0.09(2.56) | 66.44±0.17(0.49)
FA | 68.00±1.56 | 0.98±0.13(67.02) | 56.02±0.80(11.98) | 2.22±0.12(65.78) | 61.44±0.56(6.56)
MIA | 49.06±1.18 | 17.56±0.60(31.50) | 0.00±0.00(49.06) | 17.64±0.53(31.42) | 29.54±0.67(19.52)
AvgGap | - | 33.57 |  20.37 | 33.25 | 8.86
10% | C-10 | RA | 93.09±0.09 | 87.01±0.18(6.08) | 91.88±0.09(1.21) | 86.94±0.26(6.14) | 92.51±0.04(0.58)
FA | 93.64±0.20 | 0.07±0.02(93.58) | 10.79±0.70(82.85) | 0.08±0.03(93.56) | 92.42±0.42(1.22)
MIA | 80.59±0.43 | 32.42±1.60(48.17) | 0.00±0.00(80.59) | 32.25±0.64(48.34) | 74.28±0.46(6.31)
AvgGap | - | 49.27 | 54.88 | 49.35 | 2.70
C-100 | RA | 75.68±0.12 | 71.21±0.14(4.47) | 75.91±0.14(0.23) | 66.50±0.16(9.19) | 75.07±0.14(0.62)
FA | 74.76±0.27 | 0.01±0.01(74.75) | 63.74±0.64(11.02) | 0.01±0.01(74.75) | 68.35±0.54(6.41)
MIA | 55.05±0.40 | 24.92±1.77(30.13) | 0.00±0.00(55.05) | 34.42±0.49(20.64) | 35.13±0.25(19.92)
AvgGap | - | 36.45 | 22.10 | 34.86 | 8.98
Tiny | RA | 65.47±0.20 | 57.62±0.26(7.85) | 65.09±0.13(0.38) | 53.79±0.23(11.68) | 64.68±0.12(0.79)
FA | 66.00±0.14 | 0.00±0.00(66.00) | 47.65±0.96(18.35) | 0.15±0.02(65.85) | 56.83±0.24(9.17)
MIA | 46.75±0.72 | 44.08±0.94(2.67) | 0.00±0.00(46.75) | 52.17±0.94(5.42) | 28.12±0.37(18.63)
AvgGap | - | 25.51 | 21.83 | 27.65 | 9.53

###  D.4 Ablation Study

Recall that, in NatMU, the unlearning instances of one forgetting sample
$(\bm{x}^{f}_{i},y^{f}_{i})$ is:

$\mathcal{D}_{i}^{f}=\\{(\mathcal{T}_{\bm{m}_{j}^{\mathrm{scaled}}}(\bm{x}_{i}^{f},\bm{x}_{j}^{r}),y_{j}^{r})\\}_{j=1}^{n}=\\{(\bm{x}_{i}^{f}\circ\bm{m}_{j}^{\mathrm{scaled}}+\bm{x}_{j}^{r}\circ(\bm{1}_{d}-\bm{m}_{j}^{\mathrm{scaled}}),y_{j}^{r})\\}_{j=1}^{n}.$
(A2)

To investigate the role of injected information
$\bm{x}_{j}^{r}\circ(\bm{1}_{d}-\bm{m}_{j}^{\mathrm{scaled}})$, we replace
$\bm{x}_{j}^{r}$ with $\bm{0}_{d}$ and obtain the resulting unlearning
instances:

$\mathcal{D}_{i}^{f_{1}}=\\{(\bm{x}_{i}^{f}\circ\bm{m}_{j}^{\mathrm{scaled}}+\bm{0}_{d}\circ(\bm{1}_{d}-\bm{m}_{j}^{\mathrm{scaled}}),y_{j}^{r})\\}_{j=1}^{n}=\\{(\bm{x}_{i}^{f}\circ\bm{m}_{j}^{\mathrm{scaled}},y_{j}^{r})\\}_{j=1}^{n}.$
(A3)

We denote this as NatMU-NRI. Furthermore, we raplace the weighting masks in
NatMU-NRI with $\bm{0}_{d}$, and the resulting unlearning instances are:

$\mathcal{D}_{i}^{f_{2}}=\\{(\bm{x}_{i}^{f},y_{j}^{r})\\}_{j=1}^{n},$ (A4)

denoted as NatMU-RL. In fact, it is a modified version of Amnesiac. In
Amnesiac, one forgetting sample is relabeled once. While in NatMU-RL, one
forgetting sample is relabeled for $n$ times.

Tab. A5 shows the results of NatMU-RL, NatMU-NRI, NatMU in random-subset
unlearning with different forgetting ratios. NatMU-RL faces a significant
decrease of forgettting accuracy, since the relabeling of entire forgetting
samples results in the unnatural property as discussed before. Note that it
also face a catastrophic decrease of generalization on CIFAR-10 with 10%
forgetting ratio. It may be due to instability introduced by a large number of
multiply labeled samples and data augmentation. NatMU-NRI reduces the severe
data conflict in NatMU-RL by only relabeling a part of forgetting samples.
However, without the injected correct information, it still suffers the over-
forgetting problems.

Table A5: Random-subset unlearning on CIFAR-10 with VGG16, CIFAR-100 using
ResNet18 and TinyImageNet-200 with ResNet34 under 1% forgetting ratio and 10%
forgetting ratio. The results are given by $a_{\pm b}({\color[rgb]{1,0,0}c})$,
sharing the same format with Tab.1.

| Dataset | Metric | Retrain | NatMU-RL | NatMU-NRI | NatMU
---|---|---|---|---|---|---
1% | C-10 | RA | 93.26±0.12 | 93.13±0.14(0.13) | 93.21±0.13(0.05) | 93.01±0.09(0.24)
FA | 92.80±0.80 | 27.76±3.18(65.04) | 86.64±0.29(6.16) | 93.08±0.41(0.28)
MIA | 80.16±0.73 | 0.00±0.00(80.16) | 68.80±0.38(11.36) | 71.32±0.83(8.84)
AvgGap | - | 48.44 | 5.86 | 3.12
C-100 | RA | 76.18±0.23 | 76.66±0.28(0.48) | 76.41±0.09(0.23) | 76.62±0.22(0.44)
FA | 75.32±0.79 | 1.16±0.15(74.16) | 58.00±1.04(17.32) | 75.08±0.81(0.24)
MIA | 55.24±0.56 | 0.00±0.00(55.24) | 22.72±1.34(32.52) | 41.72±0.93(13.52)
AvgGap | - | 43.29 | 16.69 | 4.73
Tiny | RA | 66.93±0.23 | 66.90±0.14(0.03) | 66.03±0.22(0.90) | 66.74±0.20(0.20)
FA | 68.00±1.56 | 4.50±0.67(63.50) | 44.34±0.67(23.66) | 68.68±0.72(0.68)
MIA | 49.06±1.18 | 0.00±0.00(49.06) | 18.40±0.68(30.66) | 36.76±0.55(12.30)
AvgGap | - | 37.53 | 18.41 | 4.39
10% | C-10 | RA | 93.09±0.09 | 9.10±0.81(83.99) | 92.86±0.17(0.22) | 92.16±0.07(0.93)
FA | 93.64±0.20 | 1.43±0.14(92.22) | 85.44±0.14(8.20) | 93.88±0.17(0.23)
MIA | 80.59±0.43 | 40.00±0.79(40.60) | 66.88±0.57(13.71) | 73.80±0.28(6.79)
AvgGap | - | 72.27 | 7.38 | 2.65
C-100 | RA | 75.68±0.12 | 71.85±0.32(3.83) | 74.04±0.36(1.64) | 75.18±0.21(0.50)
FA | 74.76±0.27 | 10.47±0.44(64.28) | 62.08±0.36(12.67) | 74.50±0.60(0.26)
MIA | 55.05±0.40 | 0.00±0.00(55.05) | 31.26±0.65(23.80) | 38.92±0.19(16.14)
AvgGap | - | 41.06 | 12.70 | 5.63
Tiny | RA | 65.47±0.20 | 59.36±0.24(6.11) | 63.41±0.09(2.06) | 64.87±0.15(0.60)
FA | 66.00±0.14 | 16.37±0.33(49.63) | 48.14±0.33(17.87) | 65.72±0.33(0.28)
MIA | 46.75±0.72 | 4.76±0.69(41.99) | 26.02±0.47(20.73) | 34.66±0.35(12.09)
AvgGap | - | 32.58 | 13.55 | 4.33