# Differentially Private Video Activity Recognition

Zelun Luo1 Yuliang Zou2 Yijin Yang3 Zane Durante1 De-An Huang4
Zhiding Yu4 Chaowei Xiao3,4 Li Fei-Fei1 Animashree Anandkumar4,5

1Stanford University 2Virginia Tech 3Arizona State University 4NVIDIA 5Caltech

###### Abstract

In recent years, differential privacy has seen significant advancements in
image classification; however, its application to video activity recognition
remains under-explored. This paper addresses the challenges of applying
differential privacy to video activity recognition, which primarily stem from:
(1) a discrepancy between the desired privacy level for entire videos and the
nature of input data processed by contemporary video architectures, which are
typically short, segmented clips; and (2) the complexity and sheer size of
video datasets relative to those in image classification, which render
traditional differential privacy methods inadequate. To tackle these issues,
we propose Multi-Clip DP-SGD, a novel framework for enforcing video-level
differential privacy through clip-based classification models. This method
samples multiple clips from each video, averages their gradients, and applies
gradient clipping in DP-SGD without incurring additional privacy loss.
Moreover, we incorporate a parameter-efficient transfer learning strategy to
make the model scalable for large-scale video datasets. Through extensive
evaluations on the UCF-101 and HMDB-51 datasets, our approach exhibits
impressive performance, achieving $81\%$ accuracy with a privacy budget of
$\epsilon=5$ on UCF-101, marking a $76\%$ improvement compared to a direct
application of DP-SGD. Furthermore, we demonstrate that our transfer learning
strategy is versatile and can enhance differentially private image
classification across an array of datasets including CheXpert, ImageNet,
CIFAR-10, and CIFAR-100.

| Method | Architecture(s) | Pre-training | MNIST | F-MNIST | CIFAR-10 | ImageNet
---|---|---|---|---|---|---|---
w/o pre-train | DP-SGD [1] | 2-layer NN | - |<EMAIL_ADDRESS>| - | - | -
DPNAS [7] | DPNASNet | - |<EMAIL_ADDRESS>|<EMAIL_ADDRESS>|<EMAIL_ADDRESS>| -
Tempered Sigmoid [50] | 6/6/9-layer CNN | - |<EMAIL_ADDRESS>|<EMAIL_ADDRESS>|<EMAIL_ADDRESS>| -
DP-ScatterNet [62] | ScatterNet | - |<EMAIL_ADDRESS>|<EMAIL_ADDRESS>|<EMAIL_ADDRESS>| -
Norm-DP-SGD [9] | LeNet-5/VGG-16 | - |<EMAIL_ADDRESS>| - |<EMAIL_ADDRESS>| -
Private-kNN [74] | 5-layer CNN | - |<EMAIL_ADDRESS>| - | - | -
AdaCliP [52] | 2-layer NN | - |<EMAIL_ADDRESS>| - | - | -
DDP-SGD [13]. | 6-layer CNN | - |<EMAIL_ADDRESS>|<EMAIL_ADDRESS>| - | -
w/ pre-train | DP-SGD [1] | 4-layer CNN | CIFAR-100 | - | - |<EMAIL_ADDRESS>| -
Private-kNN [74] | 4-layer CNN | CIFAR-100 | - | - |<EMAIL_ADDRESS>| -
Scalable [40] | ResNet-18 | ImageNet | - | - |<EMAIL_ADDRESS>| -
DP-ScatterNet [62] | ResNet-50 | ImageNet | - | - |<EMAIL_ADDRESS>| -
DP-ImageNet [33] | ResNet-18 | Places365 | - | - | - |<EMAIL_ADDRESS>

Table 1: State-of-the-Art Differential Privacy Methods. Evaluation of
differential privacy methods on diverse vision datasets using the
acc@$\epsilon$ metric, selecting results nearest to $\epsilon=3$ for each
method. The table emphasizes the limited exploration of large-scale vision
tasks and privacy-sensitive domains in current literature.

##  1 Introduction

The rising adoption of machine learning in privacy-sensitive sectors has
intensified the need for privacy-preserving machine learning [8, 68, 69].
Within these realms, videos constitute a substantial portion of the data, such
as surveillance streams used for patient monitoring [22]. This underscores the
criticality of developing video models that are robust against privacy attacks
inherent to machine learning [48, 57, 58]. Notable advancements in image
classification models have been achieved through the employment of
differential privacy [1, 10, 33, 42, 49], furnishing probabilistic privacy
assurances by minimizing the impact of data point substitutions within the
dataset. However, the field is yet to witness equivalent breakthroughs in
video classification, leaving uncertainties regarding the extent to which the
accomplishments in image classification can be extended to videos.

We pinpoint two characteristics inherent to video classification that hinder
the straightforward adaptation of differential privacy to videos. Firstly,
existing private image classification models primarily assume _per-sample_
differential privacy, offering probabilistic guarantees for each individual
sample (_i.e_. image, video clip) fed into the model during training. This
becomes an issue for state-of-the-art video classification models that usually
process multiple short clips instead of full videos. Under per-sample
differential privacy, these models only assure privacy for the short clips
used in training, not for the entire videos. This poses a challenge for state-
of-the-art video classification models [18, 39] which, in various training
iterations, sample multiple short _clips_ from the same video instead of
processing the entire video as a single sample. Under per-sample differential
privacy, privacy is assured only for these individual short clips, and not for
the whole video from which they are sampled. Secondly, as illustrated in Table
1, video datasets that are commonly used, such as UCF-101 [60], HMDB-51 [31],
and Kinetics [4], are substantially larger in scale compared to the image
classification datasets like MNIST and CIFAR-10, which are traditionally
employed in differential privacy research [49, 1]. Directly employing
differential privacy algorithms like DP-SGD on these larger video datasets is
impractical due to its detrimental impact on large-scale training.

Summary of Contributions: (1) We achieve the first substantial result on
differentially private video action classification ($+76\%@\epsilon=5$ on
UCF-101) by simultaneously addressing the two aforementioned challenges. (2)
We establish a framework for video-level differential privacy and introduce a
novel multi-clip method (Multi-Clip DP-SGD) that allows clip-based video
classification models to attain video-level differential privacy without
demanding additional privacy budgets. (3) Through an exhaustive analysis, we
pinpoint the most effective parameter-efficient transfer learning strategy for
the application of differentially private training to large-scale video
datasets.

Firstly, we recognize a disparity between the level of privacy sought
(encompassing an entire video) and the nature of inputs processed by modern
video classification architectures [18, 39] (which are short clips of frames).
Conventional training methods focus on clips, and employing differentially
private training algorithms like DP-SGD under this setup merely achieves clip-
level privacy, compromising the privacy of the full videos. To tackle this
issue, we put forward a video-based, multi-clip differential privacy training
scheme termed Multi-Clip DP-SGD (Figure 1). For each selected video, our
approach samples several clips per iteration. Rather than directly clipping
gradients for each clip, which would inflate the privacy budget in proportion
to the number of clips and thus impede the balance between privacy and
utility, we average the gradients among clips from the same video before
applying the clipping. Our method harnesses the rich information in each video
without escalating the privacy budget.

Secondly, we recognize that video classification models and datasets
considerably exceed the size of their image classification counterparts
typically examined in differential privacy research (UCF-101 [60] with
2,916,000 frames compared to MNIST/CIFAR-10 with 60,000 images). Transfer
learning has been pivotal in successfully scaling differential privacy to
large datasets like ImageNet [55] in the realm of image classification [10,
33, 42]. By publicly pre-training models on even more extensive datasets like
JFT-3B [61], the efficacy of private fine-tuning on target datasets is
significantly enhanced. Yet, the optimal fine-tuning strategy, especially for
video architectures, remains elusive. To address this, we undertake an
exhaustive analysis of transfer learning strategies that are parameter-
efficient for differential privacy, as depicted in Figure 2. Our study reveals
two key insights: (1) fine-tuning normalization layers along with the final
linear layer yields the best performance with a negligible increase in
trainable parameters; (2) incorporating additional parameters from adapters
[26] proves advantageous when there is a substantial domain disparity in pre-
training. Leveraging these insights along with our multi-clip video training,
we conduct experiments on UCF-101 and HMDB-51 and achieve unprecedented
results in video classification. In particular, the LayerNorm [3] architecture
(in MViT [17]), parameter-efficient transfer learning, and our multi-clip
differential privacy training (Multi-Clip DP-SGD) collectively account for our
substantial gains ($+76\%@\epsilon=5$ compared to training from scratch).

Finally, we extend our insights to large-scale image classification. Our
analysis reveals that contemporary architectures employing LayerNorm [3], such
as ViT-S [12] and ConvNeXt-T [38], outperform the traditionally used
ResNet-50-GroupNorm [33, 65] in privacy-utility trade-offs, despite comparable
parameter counts and ImageNet-1K [55] performance. These discoveries propel
the state-of-the-art in differential privacy methods, showing substantial
gains on standard benchmarks (e.g., $+21.7\%@\epsilon=4$ on CIFAR-10,
$+71.1\%@\epsilon=4$ on CIFAR-100) and opening avenues for applying
differential privacy to previously unexplored privacy-sensitive datasets like
CheXpert [28].

Figure 1:  Comparison of Training Methods for Videos. We compare non-private
training (left), DP-SGD (middle), and Multi-Clip DP-SGD (right). DP-SGD
utilizes gradient clipping and adds Gaussian noise to gradients per video
clip. Multi-Clip DP-SGD selects several clips per iteration and combines
gradients within a video prior to clipping. By ensuring that each video is
observed only once per training step, the performance is enhanced without
compromising privacy.

##  2 Related work

Differentially-Private SGD (DP-SGD). ($\epsilon,\delta$)-differential privacy
has become the gold standard for database and model privacy due to its
probabilistic guarantees of minimal data leakage [14, 15]. The most common
method for ensuring differential privacy for deep learning models is DP-SGD
proposed in Abadi  _et al_. [1]. This modification on standard stochastic
gradient descent (SGD) uses gradient clipping and noise additions to mitigate
the effects of individual data points [1]. There have been several follow-up
works [11, 23, 64] on DP-SGD. Some explore adaptive methods for clipping
gradients to help learning [52]. Others propose methods for loss function
smoothing to mitigate the effects of the noise added to the gradient [64]. Our
work uses the accounting method based on Rényi differential privacy [44],
which provides a tighter bound on the privacy-utility estimate than the
accounting method used in Abadi  _et al_. [1]. There have also been a series
of works exploring the effect of neural architecture choices on the
performance obtained when using DP-SGD [45, 9, 50].

Large-Scale Differential Privacy. Recent works have shown that differential
privacy is possible even on large-scale datasets like ImageNet by leveraging
transfer learning, but requires significant pre-training [10, 33]. Prior work
shows that using regular DP-SGD to train on ImageNet achieves an accuracy of
$0-1\%$ [33]. However, pre-training image models on large-scale internet data
has led to reasonable performances on ImageNet, with Kurakin  _et al_. [33]
getting $47.9\%$ image classification accuracy with $\epsilon=10$,
$\delta=8\cdot 10^{-7}$. In natural language processing (NLP), recent work has
shown that large pre-trained language models can be effective differentially
private learners [2, 35].

Transfer Learning. In transfer learning, a model is first pre-trained on an
initial dataset and then fine-tuned on a downstream dataset [47, 75]. There
are many ways to fine-tune models. The simplest way is full fine-tuning, which
trains the entire model on the downstream dataset [32]. Another way to fine-
tune is to freeze the entire model except the last layer and use it as a
feature extractor, and then train a linear probe classifier using the feature
extractor embedding space [5, 20]. There have also been ongoing research
threads using parameter efficient fine-tuning in order to save computation
costs. Prompt-tuning has recently become a common strategy in natural language
processing (NLP) for efficient fine-tuning of large language models [34, 37],
and adapter modules have also recently become a popular way to efficiently
fine-tune large models [21, 25]. In alternative approaches to parameter
efficient transfer learning, the normalization layers or weights discovered
through the lottery ticket hypothesis [19] are fine-tuned to enable efficient
transfer learning [36, 43].

Video and Privacy. As videos may contain personal information, there have been
several works on privacy-preserving video processing framework. Possible
methods include anonymizing faces in videos [54], cryptographic approach [51],
decreasing the video resolution [56], and learning anonymization [67]. While
previous research has explored the application of differential privacy to
videos, its primary emphasis lies in directly introducing differential privacy
to the entire video [63]. We propose a multi-clip differential privacy
training algorithm for video classification, and achieve the first non-trivial
result on commonly used video classification datasets.

Figure 2: Transfer Learning Paradigms. This figure compares four transfer
learning paradigms: full fine-tuning, linear probing, selective fine-tuning,
and adapters. Notably, the latter three paradigms are characterized as
parameter-efficient as they focus on training smaller subnetworks as opposed
to the entire network.

##  3 Differentially Private Video Classification

We pinpoint two challenges in differentially private video classification: (1)
a discrepancy between model inputs (clips) and privacy objectives (videos);
(2) adapting differential privacy for large-scale video datasets. In Section
3.1, we lay the groundwork with an overview of differential privacy (DP). In
Section 3.2, we delineate video-level DP, the crux of this study. In Section
3.3, we introduce our solution, Multi-Clip DP-SGD, for video-level DP. In
Section 3.4, we outline parameter-efficient transfer learning strategies for
scaling differential privacy to videos. Lastly, we examine suitable network
architectures in Section 3.5.

###  3.1 Preliminary

Differential Privacy. Differential privacy [14] provides a formal privacy
guarantee to prevent information leakage of each individual data point within
a dataset. By adding randomization to computation over a dataset (_e.g_.
training a ML model), the influence of each data point is bounded, and thus
the instance-level privacy is protected. There are two important parameters
controlling the strength of differential privacy guarantee: $\epsilon>0$ and
$\delta\in[0,1]$. And the privacy guarantee becomes stronger as both
parameters get smaller. Formally, we have the following definition, which
makes use of the concept of adjacent datasets (_i.e_. two datasets than only
differ by one data entry).

###### Definition 1

Given a random mechanism $M:D\rightarrow R$ and two adjacent datasets,
$d_{1},d_{2}\in D$, then we say that $M$ satisfies
$(\epsilon,\delta)$-differential privacy if $\forall S\subseteq R$:

$\displaystyle P[M(d_{1})\in S]\leq e^{\epsilon}P[M(d_{2})\in S]+\delta.$

Differentially Private SGD (DP-SGD). DP-SGD [1] is one of the most common
strategies to protect the privacy of training data. With slight modifications
upon the traditional SGD algorithm, DP-SGD effectively provides instance-level
privacy protection. More specifically, it has the following modifications.
First, instead of computing mini-batch level gradients at each iteration, DP-
SGD computes the per-sample gradient and clips gradient values greater than a
constant $C$. Then, these clipped gradient values are averaged over the
training mini-batch and Gaussian noises
$\mathcal{N}(0,\sigma^{2}C^{2}\mathbf{I})$ are added. DP-SGD was designed to
integrate seamlessly with deep neural networks due to its simplicity of
implementation and no additional assumptions about the data. Figure 1 compares
training with and without DP-SGD for a clip-based video classification model.

Privacy Accountant. A privacy accountant keeps track of the privacy spendings
during model training. The momentum accountant [1] was first proposed to
compute the privacy loss for DP-SGD. Later, Rényi differential privacy was
proposed to relax differential privacy by using the Rényi divergence [44]. It
provides a tighter privacy-utility bound and thus can allow for greater
accuracy with the same probabilistic guarantees as standard DP-SGD.

###  3.2 Video-Level Differential Privacy

In this work, our goal is differentially private video classification. We aim
to train a video classifier $f(\cdot)$, which takes a video $v_{i}$ as input
and outputs its class. In addition, $f(\cdot)$ should be trained with privacy
guarantees. More specifically, we are interested in training $f(\cdot)$ with
_video-level_ differential privacy. Recall that Definition 1 builds on the
concept of adjacent datasets. By video-level DP, we mean that each data entry
in the adjacent datasets is a video, and the two datasets only differ by one
video.

While our goal is to classify the entire video, modern video classification
architectures are often designed to take multiple _clips_ consisting of
several frames as input during training [17, 39]. Each video can be temporally
chunk into $N_{i}$ clips: $v_{i}=[c_{i}^{1}\dots c_{i}^{N_{i}}]$. For
simplicity, we assume that the clips are consecutive and non-overlapping, and
by concatenating all the $c_{i}^{j}$ by order we have the original video
$v_{i}$. Clip-based architecture is a popular and effective design for video
classification. However, this creates problems for applying differential
privacy to video classification.

Since the video models are trained on clips, if we directly apply privacy-
preserving algorithms in training, such as DP-SGD, then each data entry in the
adjacent datasets of Definition 1 is no longer a video, but instead a clip. In
this case, the privacy gaurantees are given for clips instead of videos. We
refer to this as _clip-level_ differential privacy. It is important to note
that clip-level differential privacy does not imply video-level differential
privacy for the same $(\epsilon,\delta)$. Video-level differential privacy
considers adjacent datasets differ by one video $v_{i}$, which are in fact
$N_{i}$ clips. This is beyond what is covered by clip-level differential
privacy (_i.e_. adjacent datasets differ by only one clip). Since a video
$v_{i}=[c_{i}^{1}\dots c_{i}^{N_{i}}]$ consists of a group of clips
$c_{i}^{j}$, one possible approach is to apply differential privacy’s property
of _group privacy_ to translate clip-level differential privacy to video-level
differential privacy [14]. However, this would simply multiply the privacy
budget by the number of clips in videos, which significantly harms the
privacy-utility tradeoff.

###  3.3 Multi-Clip DP-SGD for Videos

We have discussed that naively applying privacy-preserving training algorithms
to video model training only achieves clip-level DP, which does not imply our
desired video-level DP. Now we discuss our solution, Multi-Clip DP-SGD, which
builds on DP-SGD [1] to fully exploit information in videos during training,
while not increasing their sensitivity in DP.

The first step is to sample by videos instead of clips in training. Next, one
naive way to process the sampled video is to sample only one clip per video in
each mini-batch during the training time, since the architectures are designed
to take a clip as input. However, this severely wastes the privacy budget,
since we only utilize a small portion of information but with the price of
increased privacy loss on the whole video. Repeating this process in training
does not help with the privacy-utility tradeoff.

Inspired by the user-level privacy protection [41], we propose a new multi-
clip scheme for video-level differential privacy training. The core idea is to
increase the usage of each video, while not increasing the
sensitivity/influence of each video. Figure 1 illustrates our Multi-Clip DP-
SGD for videos. First, we sample multiple clips from a video, and compute the
per-sample gradient for each clip as in normal DP-SGD. However, instead of
clipping the gradient for each clip, we first _average_ the gradients from all
the clips before clipping the gradient (and adding Gaussian noise).

Since there is no information exchange between different videos, the
computation above satisfies video-level DP. And since we average multiple
clips from the same video before gradient clipping, we can regard that each
video is seen only once in each forward pass. And thus the privacy loss does
not increase. As we will show in the experiments, Multi-Clip DP-SGD better
leverages the information in each video without incurring additional privacy
costs and improves the privacy-utility tradeoff of video classification
models.

###  3.4 Parameter-Efficient Transfer Learning

While we have proposed Multi-Clip DP-SGD for videos, by itself it is still not
sufficient to apply differential privacy to commonly used video datasets and
models because of their scales. Even comparatively smaller video datasets like
UCF-101 [60] and HMDB-51 [31] are still much larger than CIFAR datasets used
in differential privacy literature. Recent works have shown that transfer
learning provides a significant benefit to DP-SGD in terms of privacy-utility
trade-off and scaling to large image classification tasks [62, 35]. Building
on this strategy, we identify _parameter-efficient_ transfer learning as a
promising direction for scaling differential privacy to video datasets and
conduct a comprehensive study along this direction.

Specifically, DP-SGD aims to minimize the influence of the training data
during the training process by (1) clipping the $\ell_{2}$ norm of per-sample
gradient $\mathbf{g}$ at $\mathbf{g}/\max(1,\lVert\mathbf{g}\rVert_{2}/C)$,
and (2) adding Gaussian noise sampled from
$\mathcal{N}(0,\sigma^{2}C^{2}\mathbf{I})$, where $\sigma$ and $C$ are
gradient norm bound and noise scale respectively. The above analysis indicates
that DP-SGD favors models with smaller number of trainable parameters: the
isotropic Gaussian noise applied to training gradients has a greater impact on
models with more trainable parameters because the expected norm of the noise
increases with the number of parameters [35]. On the other hand, modern deep
learning models tend to be highly overparameterized [24, 30] and researchers
have shown that increasing the number of parameters leads to improved
generalization [46]. Parameter-efficient transfer learning provides an
opportunity to use these large models in DP, while not increasing the number
of trainable parameters. Next, we discuss the (parameter-efficent) transfer
learning schemes we study in this paper.

Full Fine-Tuning. All model parameters are trained on the target data for full
fine-tuning, which makes it not parameter-efficient. We select this approach
to compare with other parameter-efficient approaches. Although full fine-
tuning is probably the most common approach in non-DP transfer learning
scenarios, it usually does not work well under differential privacy settings.
Differential privacy struggles with more trainable parameters since DP-SGD
(clipping/noise) negatively affects models with more trainable parameters.

Linear Probing. In this setting, all model parameters are fixed except the
last linear layer. The frozen part can be regarded as a feature extractor, and
we train a liner classification on top of the extracted features. Since the
last linear layer only contains a small portion of parameters (compared to the
full model), DP-SGD can be well combined with linear probing to provide a
decent privacy guarantee. However, the feature extractor is trained on the
source domain, and thus training only a linear layer may be sub-optimal due to
the existence of domain shifts.

Selective Fine-Tuning. As a trade-off between full fine-tuning and linear
probing, one can choose to train a small portion of parameters in the feature
extractor, in addition to the last linear layer. This strategy allows a model
to adapt itself to the target distribution while still maintaining a
reasonable amount of trainable parameters. And thus, DP-SGD can be effectively
applied under such a setting to provide a good privacy-utility trade-off.
Previous work has used Lottery Ticket Hypothesis [19] to select the parameters
to train [40]. In this paper, we choose to train the normalization layers in
addition to the last linear layer.

Adapter Training. Similar to selective fine-tuning, adapter training also aims
to train a small number of model parameters in addition to the last linear
layer. However, instead of updating the parameters in the original model
architecture, we incorporate additional parameters inside the current network
and train them together with the last linear layer. Similar to adapter
training in non-DP settings [21, 26, 72], we design lightweight 2-layer MLPs
with skip-connection and insert these modules inside the pre-trained networks.
These modules are initialized as identity mappings and gradually trained to
stabilize model training.

###  3.5 Architecture

Finally, we further investigate model architectures that are suitable for
training with DP. In particular, we find that architectures with different
types of normalization layers are worth further investigation. BatchNorm [27]
is empirically proved to stabilize model training and speed up convergence
[66], and thus it has been an essential component in most modern CNNs.
However, BatchNorm layers are not supported in differentially-private models.
The training time behavior of BatchNorm requires computation of mean and
variance of each input mini-batch, creating a dependency between samples,
which violates the formulation of differential privacy. There are two ways to
resolve this issue. The first one is to replace BatchNorm with other
normalization layers (_e.g_. GroupNorm [65], LayerNorm [3]). This solution
seems to be sub-optimal, since the new normalization layers are not designed
for these CNNs. The second option is to use models without BatchNorm by its
original design. We find that the recent vision transformers are good fits in
this case. We further study a new CNN architecture, ConvNeXt. In this paper,
we study two different types of network architectures: convolutional neural
networks (CNN) and vision transformers. For CNNs, we adopt ResNet [24] and
ConvNeXt [38]. For vision transformers, we adopt ViT [12] in image experiments
and MViT [17] in video experiments.

Scheme | #Params. | #Clips | $\epsilon=5$ | $\epsilon=10$
---|---|---|---|---
From scratch | 36.4M | 1 | 4.93 | 6.60
Full fine-tune | 36.4M | 1 | 29.15 | 38.58
Linear probe | 77.7K | 1 | 75.05 | 78.17
Linear probe | 77.7K | 8 | 77.96 | 80.49
Adapter | 1.8M | 8 | 79.19 | 81.73
Selective fine-tune | 110K | 8 | 80.86 | 82.81

Table 2: Experiments on UCF-101. We adopt $\delta=10^{-5}$ and report the
top-1 accuracy. We conduct experiments using an MViT-B/16$\times$4 model pre-
trained on the Kinetics-400 dataset.

Scheme | #Params. | #Clips | $\epsilon=5$ | $\epsilon=10$
---|---|---|---|---
From scratch | 36.4M | 1 | 3.62 | 4.13
Full fine-tune | 36.4M | 1 | 17.93 | 21.21
Adapter | 1.8M | 1 | 50.34 | 53.37
Selective fine-tune | 110K | 1 | 51.68 | 56.48
From scratch | 36.4M | 8 | 6.87 | 7.24
Full fine-tune | 36.4M | 8 | 29.20 | 33.75
Adapter | 1.8M | 8 | 53.62 | 56.23
Selective fine-tune | 110K | 8 | 57.49 | 60.52

Table 3: Experiments on HMDB-51. We adopt $\delta=10^{-5}$ and report the
top-1 accuracy. We conduct experiments using an MViT-B/16$\times$4 model pre-
trained on the Kinetics-400 dataset.

(a) UCF-101

(b) CIFAR-10

(c) CIFAR-100

Figure 3: Quantitative Analysis on UCF-101, CIFAR-10, and CIFAR-100. (a)
evaluates Multi-Clip DP-SGD for video classification on UCF-101 by
experimenting with varying numbers of clips ($1,2,4,8$) and reporting the
top-1 accuracy. (b) and (c) offers a comparative analysis on CIFAR-10 and
CIFAR-100 against state-of-the-art differential privacy methods, reporting
top-1 accuracy with $\delta=10^{-5}$.

Architecture | Scheme | #Trainable Params. | $\epsilon=1$ | $\epsilon=2$ | $\epsilon=4$ | $\epsilon=8$ | $\epsilon=\infty$
---|---|---|---|---|---|---|---
ResNet-50-GN | From scratch | 23.7M | 4.16 | 7.29 | 11.83 | 17.25 | -
Full fine-tune | 23.7M | 52.96 | 59.63 | 66.92 | 71.32 | 82.40
Linear probe | 204K | 62.14 | 65.89 | 68.71 | 70.28 | -
Selective fine-tune | 258K | 62.33 | 68.93 | 72.32 | 74.69 | -
ViT-S/16-224 | From scratch | 21.7M | 13.76 | 16.22 | 19.67 | 22.43 | -
Full fine-tune | 21.7M | 79.36 | 83.64 | 85.67 | 86.59 | 90.27
Linear probe | 38.5K | 71.64 | 74.81 | 76.47 | 77.03 | -
Selective fine-tune | 57.7K | 84.73 | 86.90 | 87.68 | 88.43 | -
Adapter | 928K | 82.32 | 84.35 | 85.79 | 87.03 | -
ConvNeXt-T | From scratch | 27.9M | 9.20 | 12.93 | 15.97 | 18.45 | -
Full fine-tune | 27.9M | 65.93 | 74.00 | 79.17 | 81.22 | 89.98
Linear probe | 76.9K | 76.59 | 78.82 | 80.32 | 80.91 | -
Selective fine-tune | 93.2K | 83.11 | 84.69 | 86.06 | 86.67 | -
Adapter | 1.7M | 79.37 | 83.68 | 85.11 | 86.05 | -

Table 4: Ablation Study on CIFAR-100. The models are trained for 50 epochs
with $\delta=10^{-5}$, and the top-1 accuracy is reported to validate the
effectiveness of the design choices.

##  4 Experiments

Now we evaluate our proposed Multi-Clip DP-SGD and parameter-efficient
transfer learning for video classification. In addition, we further evaluate
our parameter-efficient transfer learning on scaling image classification and
compare it with state-of-the-art approaches in DP.

###  4.1 Differentially Private Video Classification

Experimental Setup. We use UCF-101 [60] and HMDB-51 [31] for our evaluation of
differentially private video classification. UCF-101 is a widely used action
recognition dataset that contains 133,20 action videos annotated into 101
action classes. HMDB-51 is a video classification dataset of human motion,
containing 6849 clips divided into 51 action categories, each containing a
minimum of 101 clips. Unlike image classification, video classification [59,
18] of human activities typically only samples a clip of frames from each
video in a training batch. We report top-1 accuracy as the utility metric. To
the best of our knowledge, this is the first study of differentially private
video classification on a large-scale video dataset.

Parameter-Efficient Transfer Learning. The results are shown in Table 2 and
Table 3. We find that full fine-tuning significantly improves training from
scratch (+24%@$\epsilon=5$ in UCF-101 and +14%@$\epsilon=5$ in HMDB-51). In
addition, linear probing further improves full fine-tuning by limiting the
number of trainable parameters (+46%@$\epsilon=5$ in UCF-101 and
+32%@$\epsilon=5$ in HMDB-51). Among different parameter-efficient transfer
learning schemes, selective fine-tuning improves linear probing by
2.9%@$\epsilon=5$ on UCF-101 with a slight increase of trainable parameters.
Adapter’s improvement is less than selective fine-tuning in this scenario. As
we will see in later on image classification, the relative performance
differences of these parameter-efficient schemes (linear, adapter, selective)
are dataset dependent. For transfer learning from Kinetics to UCF-101 and
HMDB-51, linear probing is already effective because of the small domain gap.

Multi-Clip DP-SGD. We study the effect of our Multi-Clip DP-SGD in Figure 3,
where we compare results with varying numbers of clips. With only 1 clip, the
result is reduced to standard DP-SGD. We use selective fine-tuning for this
analysis. With the increasing number of clips in Multi-Clip DP-SGD, the utlity
continues to improve given a fixed privacy budget. Overall, our multi-clip
approach leads to 6.7%@$\epsilon=5$ compared to standard DP-SGD, which is
larger than the difference between different parameter-efficient schemes. This
shows the importance of our multi-clip approach for video classification under
DP.

Architecture | Pre-train | Batch size | Scheme | #Trainable Params. | $\epsilon=10$
---|---|---|---|---|---
ViT-B/16-224 |  CLIP [53] |  65,536 | Full fine-tune | 86.6M | 66.09
Linear probe | 769K | 79.36
Selective fine-tune | 808K | 79.86
ViT-S/16-224 |  Places365 [73] |  1,024 | Full fine-tune | 22.1M | 29.34
Linear probe | 385K | 26.53
Selective fine-tune | 404K | 40.78
Adapter | 1.3M | 40.47
ViT-S/16-224 |  Places365 [73] |  65,536 | Full fine-tune | 22.1M | 30.21
Selective fine-tune | 404K | 45.12
Adapter | 1.3M | 41.79

Table 5: Large-Scale Differential Privacy Experiments on ImageNet. We adopt
$\delta=10^{-6}$ and report the top-1 accuracy.

Architecture | Scheme | #Trainable Params. | Pre-train | $\epsilon=1$ | $\epsilon=2$ | $\epsilon=4$ | $\epsilon=8$
---|---|---|---|---|---|---|---
ViT-S/16-224 | From scratch | 21.7M | - | 74.82 | 74.93 | 75.13 | 75.60
Full fine-tune | 21.7M | IN-21k$\rightarrow$IN-1k | 82.57 | 83.50 | 84.98 | 85.78
Selective fine-tune | 21.1K | IN-21k $\rightarrow$ IN-1k | 81.76 | 83.64 | 83.71 | 84.46
Adapter | 892K | IN-21k$\rightarrow$IN-1k | 83.31 | 83.69 | 83.75 | 84.77
ConvNeXt-T | From scratch | 27.8M | - | 76.93 | 77.19 | 77.51 | 79.08
Full fine-tune | 27.8M | IN-21k $\rightarrow$ IN-1k | 85.23 | 85.37 | 86.83 | 87.21
Selective fine-tune | 20.2K | IN-21k $\rightarrow$ IN-1k | 85.06 | 85.29 | 85.92 | 86.56
Adapter | 1.7M | IN-21k $\rightarrow$ IN-1k | 86.77 | 87.06 | 87.78 | 88.05

Table 6: Large-Scale Differential Privacy Experiments on CheXpert. We adopt
$\delta=10^{-5}$ and report the AUROC. All models were trained for 10 epochs.

###  4.2 Differentially Private Image Classification

To underscore the potency of our parameter-efficient transfer learning
strategies, we put them to the test on large-scale image classification tasks
using two datasets: ImageNet and CheXpert. ImageNet is widely regarded as the
benchmark dataset, while CheXpert offers insights into the application of
differential privacy in privacy-sensitive scenarios with large domain gaps.

ImageNet. ImageNet [55] has been widely used for image classification. It
consists of 1,000 classes of 1,281,167 training images and 50,000 validation
images. We report results on ImageNet using two ViT [12] archiectures and two
different pre-training datasets. As shown in Table 5, full fine-tuning results
in worse performance than the parameter-efficient strategies. We note that
using the pre-training dataset from CLIP [53] results in substantial
performance gains, highlighting the importance of large-scale pre-training for
transfer learning in the differentially private setting. We significantly
bridge the gap with supervised performance, with the ViT-B/16 architecture
achieving 85.49% top-1 accuracy on ImageNet with public training [12].

CheXpert (Medical Image Classification). CheXpert [29] is a large-scale
dataset of chest X-rays annotated by professional radiologists. It consists of
224,316 chest radiographs of 65,240 patients. Following the standard setting
[29, 71], we report the AUC score as the utility metric on five selected
diseases, i.e., Cardiomegaly, Edema, Consolidation, Atelectasis, Pleural
Effusion. Results are in Table 6. Adapter outperforms selective fine-tuning
with the large domain gap in this case. In addition, for the ViT backbone,
when we have sufficient privacy budget, DP-SGD is sufficient to fine-tune the
full model and full fine-tuning performs the best. However, parameter-
efficient schemes still outperform with less privacy budgets ($\epsilon=1,2$).

###  4.3 Comparison with State-of-the-Art Methods

In this section, we conduct an ablation study and compare our approach with
state-of-the-art differential privacy methods on CIFAR-100, as shown in Table
4.

Parameter-Efficient Transfer Learning. We still find that transfer learning
outperforms training from scratch by a large margin on datasets commonly used
in differential privacy literature. For CNN models (ResNet, ConvNeXt), linear
probing outperforms full fine-tuning when using a small $\epsilon$. Selective
fine-tuning is consistently the best-performing method on CIFAR-100 despite
only having a small increase of trainable parameters compared to linear
probing. The improvement is more significant for LayerNorm-based architectures
(ViT and ConvNeXt).

LayerNorm-Based Architectures. We find that using recent architectures based
on LayerNorm leads to better performance gains compared to the commonly used
ResNet-50, despite having a similar amount of trainable parameters and similar
performance when publicly trained. For ResNet-50, the performances of linear
probing and selective fine-tuning are much closer. In contrast, on ViT and
ConvNeXt, selective fine-tuning leads to more significant improvement. This
suggests that selective fine-tuning might synergize better with LayerNorm
compared to GroupNorm in our experiments.

Comparison with SOTA. We compare our models with state-of-the-art results on
CIFAR-10 and CIFAR-100. Results on the more challenging CIFAR-100 are more
limited compared to CIFAR-10. As shown in Figure 3, under our parameter-
efficient transfer learning scheme based on the previous study, our method
consistently outperforms the existing approaches by a large margin, across all
measured $\epsilon$ values and across both datasets.

###  4.4 Implementation Details

We use the implementation from the Opacus library [70] to apply DP-SGD for
model training. We utilize distributed training and model checkpointing [6] to
increase the batch size. For video experiments, we use the PyTorchVideo
library [16]. All the experiments are conducted on a Google Cloud instance
with 8 Nvidia A100 GPUs. While hyper-parameters play a critical role in
differentially private training, they are notoriously difficult to tune. In
order to facilitate comparison between methods, we pre-define the training
epochs for each dataset. And we fix the clipping norm as $C=1$. We search for
the optimal learning rate on the CIFAR-100 dataset with a fixed $\epsilon=1$.
We perform a grid search over the learning rate between $[10^{-4},10^{-2}]$.

##  5 Conclusion and Limitation

We introduce the pioneering application of differential privacy to video
classification on the UCF-101 and HMDB-51 datasets. Our integrated approach,
featuring the innovative Multi-Clip DP-SGD and parameter-efficient transfer
learning tailored for modern video architectures under DP, achieves a
remarkable 76% accuracy improvement at $\epsilon=5$ compared to the direct
application of DP-SGD. One limitation of our study is the use of
($\epsilon,\delta$)-differential privacy as the sole standard for evaluating
privacy. The lack of established criteria for practical $\epsilon$ settings
necessitates future research to assess models against pragmatic privacy
metrics.

## References

  * [1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang.  Deep learning with differential privacy.  In CCS, 2016.
  * [2] Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Manurangsi.  Large-scale differentially private bert.  arXiv preprint arXiv:2108.01624, 2021.
  * [3] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton.  Layer normalization.  arXiv preprint arXiv:1607.06450, 2016.
  * [4] Joao Carreira and Andrew Zisserman.  Quo vadis, action recognition? a new model and the kinetics dataset.  In CVPR, 2017.
  * [5] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In ICML, 2020.
  * [6] Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin.  Training deep nets with sublinear memory cost.  arXiv preprint arXiv:1604.06174, 2016.
  * [7] Anda Cheng, Jiaxing Wang, Xi Sheryl Zhang, Qiang Chen, Peisong Wang, and Jian Cheng.  Dpnas: Neural architecture search for deep learning with differential privacy.  arXiv preprint arXiv:2110.08557, 2021.
  * [8] Thomas Davenport and Ravi Kalakota.  The potential for artificial intelligence in healthcare.  Future healthcare journal, 6(2):94, 2019.
  * [9] Ali Davody, David Ifeoluwa Adelani, Thomas Kleinbauer, and Dietrich Klakow.  On the effect of normalization layers on differentially private training of deep neural networks.  arXiv preprint arXiv:2006.10919, 2020.
  * [10] Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle.  Unlocking high-accuracy differentially private image classification through scale.  arXiv preprint arXiv:2204.13650, 2022.
  * [11] Friedrich Dörmann, Osvald Frisk, Lars Nørvang Andersen, and Christian Fischer Pedersen.  Not all noise is accounted equally: How differentially private learning benefits from large sampling rates.  In 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE, 2021.
  * [12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In ICLR, 2021.
  * [13] Jian Du, Song Li, Moran Feng, and Siheng Chen.  Dynamic differential-privacy preserving sgd.  arXiv preprint arXiv:2111.00173, 2021.
  * [14] Cynthia Dwork.  Differential privacy: A survey of results.  In Manindra Agrawal, Dingzhu Du, Zhenhua Duan, and Angsheng Li, editors, Theory and Applications of Models of Computation, 2008.
  * [15] Cynthia Dwork, Aaron Roth, et al.  The algorithmic foundations of differential privacy.  Found. Trends Theor. Comput. Sci., 9(3-4):211–407, 2014.
  * [16] Haoqi Fan, Tullie Murrell, Heng Wang, Kalyan Vasudev Alwala, Yanghao Li, Yilei Li, Bo Xiong, Nikhila Ravi, Meng Li, Haichuan Yang, Jitendra Malik, Ross Girshick, Matt Feiszli, Aaron Adcock, Wan-Yen Lo, and Christoph Feichtenhofer.  PyTorchVideo: A deep learning library for video understanding.  In Proceedings of the 29th ACM International Conference on Multimedia, 2021.  https://pytorchvideo.org/.
  * [17] Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer.  Multiscale vision transformers.  In ICCV, 2021.
  * [18] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He.  Slowfast networks for video recognition.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019.
  * [19] Jonathan Frankle and Michael Carbin.  The lottery ticket hypothesis: Finding sparse, trainable neural networks.  In ICLR, 2019.
  * [20] Tomer Galanti, András György, and Marcus Hutter.  On the role of neural collapse in transfer learning.  arXiv preprint arXiv:2112.15121, 2021.
  * [21] Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao.  Clip-adapter: Better vision-language models with feature adapters.  arXiv preprint arXiv:2110.04544, 2021.
  * [22] Sara Gerke, Serena Yeung, and I. Glenn Cohen.  Ethical and legal aspects of ambient intelligence in hospitals.  JAMA, 323(7):601, feb 2020.
  * [23] Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi, and Chiyuan Zhang.  Deep learning with label differential privacy.  Advances in Neural Information Processing Systems, 34:27131–27145, 2021.
  * [24] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, 2016.
  * [25] Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jia-Wei Low, Lidong Bing, and Luo Si.  On the effectiveness of adapter-based tuning for pretrained language model adaptation.  arXiv preprint arXiv:2106.03164, 2021.
  * [26] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.  Parameter-efficient transfer learning for nlp.  In ICML, 2019.
  * [27] Sergey Ioffe and Christian Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In ICML, 2015.
  * [28] Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al.  Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison.  In AAAI, 2019.
  * [29] Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Halabi, Jesse K. Sandberg, Ricky Jones, David B. Larson, Curtis P. Langlotz, Bhavik N. Patel, Matthew P. Lungren, and Andrew Y. Ng.  CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison.  In AAAI, 2019.
  * [30] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25, 2012.
  * [31] Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre.  Hmdb: a large video database for human motion recognition.  In 2011 International conference on computer vision, pages 2556–2563. IEEE, 2011.
  * [32] Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang.  Fine-tuning can distort pretrained features and underperform out-of-distribution.  arXiv preprint arXiv:2202.10054, 2022.
  * [33] Alexey Kurakin, Steve Chien, Shuang Song, Roxana Geambasu, Andreas Terzis, and Abhradeep Thakurta.  Toward training at imagenet scale with differential privacy.  arXiv preprint arXiv:2201.12328, 2022.
  * [34] Brian Lester, Rami Al-Rfou, and Noah Constant.  The power of scale for parameter-efficient prompt tuning.  arXiv preprint arXiv:2104.08691, 2021.
  * [35] Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto.  Large language models can be strong differentially private learners.  arXiv preprint arXiv:2110.05679, 2021.
  * [36] Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou.  Revisiting batch normalization for practical domain adaptation.  arXiv preprint arXiv:1603.04779, 2016.
  * [37] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.  Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.  arXiv preprint arXiv:2107.13586, 2021.
  * [38] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.  A convnet for the 2020s.  In CVPR, 2022.
  * [39] Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu.  Video swin transformer.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3202–3211, 2022.
  * [40] Zelun Luo, Daniel J Wu, Ehsan Adeli, and Li Fei-Fei.  Scalable differential privacy with sparse network finetuning.  In CVPR, 2021.
  * [41] H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang.  Learning differentially private recurrent language models.  In ICLR, 2018.
  * [42] Harsh Mehta, Abhradeep Thakurta, Alexey Kurakin, and Ashok Cutkosky.  Large scale transfer learning for differentially private image classification.  arXiv preprint arXiv:2205.02973, 2022.
  * [43] Rahul Mehta.  Sparse transfer learning via winning lottery tickets.  arXiv preprint arXiv:1905.07785, 2019.
  * [44] Ilya Mironov.  Renyi differential privacy.  CoRR, abs/1702.07476, 2017.
  * [45] Felix Morsbach, Tobias Dehling, and Ali Sunyaev.  Architecture matters: Investigating the influence of differential privacy on neural network design.  arXiv preprint arXiv:2111.14924, 2021.
  * [46] Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro.  Towards understanding the role of over-parametrization in generalization of neural networks.  arXiv preprint arXiv:1805.12076, 2018.
  * [47] Sinno Jialin Pan and Qiang Yang.  A survey on transfer learning.  IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2009.
  * [48] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami.  Practical black-box attacks against machine learning.  In CCS, apr 2017.
  * [49] Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson.  Scalable private learning with pate.  arXiv preprint arXiv:1802.08908, 2018.
  * [50] Nicolas Papernot, Abhradeep Thakurta, Shuang Song, Steve Chien, and Ulfar Erlingsson.  Tempered sigmoid activations for deep learning with differential privacy.  arXiv preprint arXiv:2007.14191, page 10, 2020.
  * [51] Sikha Pentyala, Rafael Dowsley, and Martine De Cock.  Privacy-preserving video classification with convolutional neural networks.  In International conference on machine learning, pages 8487–8499. PMLR, 2021.
  * [52] Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X Yu, Sashank J Reddi, and Sanjiv Kumar.  Adaclip: Adaptive clipping for private sgd.  arXiv preprint arXiv:1908.07643, 2019.
  * [53] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In ICML, 2021.
  * [54] Zhongzheng Ren, Yong Jae Lee, and Michael S Ryoo.  Learning to anonymize faces for privacy preserving action detection.  In Proceedings of the european conference on computer vision (ECCV), pages 620–636, 2018.
  * [55] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al.  Imagenet large scale visual recognition challenge.  IJCV, 115(3):211–252, 2015.
  * [56] Michael S Ryoo, Brandon Rothrock, Charles Fleming, and Hyun Jong Yang.  Privacy-preserving human activity recognition from extreme low resolution.  In Thirty-First AAAI Conference on Artificial Intelligence, 2017\.
  * [57] Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes.  ML-leaks: Model and data independent membership inference attacks and defenses on machine learning models.  In Network and Distributed System Security Symposium, 2019.
  * [58] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models.  In IEEE Symposium on Security and Privacy, may 2017.
  * [59] Karen Simonyan and Andrew Zisserman.  Two-stream convolutional networks for action recognition in videos.  Advances in neural information processing systems, 27, 2014.
  * [60] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah.  Ucf101: A dataset of 101 human actions classes from videos in the wild.  arXiv preprint arXiv:1212.0402, 2012.
  * [61] Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta.  Revisiting unreasonable effectiveness of data in deep learning era.  In ICCV, 2017.
  * [62] Florian Tramer and Dan Boneh.  Differentially private learning needs better features (or much more data).  arXiv preprint arXiv:2011.11660, 2020.
  * [63] Han Wang, Shangyu Xie, and Yuan Hong.  Videodp: A flexible platform for video analytics with differential privacy.  Proc. Priv. Enhancing Technol., 2020(4):277–296, 2020.
  * [64] Wenxiao Wang, Tianhao Wang, Lun Wang, Nanqing Luo, Pan Zhou, Dawn Song, and Ruoxi Jia.  Dplis: Boosting utility of differentially private deep learning via randomized smoothing.  arXiv preprint arXiv:2103.01496, 2021.
  * [65] Yuxin Wu and Kaiming He.  Group normalization.  In ECCV, 2018.
  * [66] Yuxin Wu and Justin Johnson.  Rethinking” batch” in batchnorm.  arXiv preprint arXiv:2105.07576, 2021.
  * [67] Zhenyu Wu, Haotao Wang, Zhaowen Wang, Hailin Jin, and Zhangyang Wang.  Privacy-preserving deep action recognition: An adversarial learning framework and a new dataset.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020\.
  * [68] Liang Xiao, Xiaoyue Wan, Xiaozhen Lu, Yanyong Zhang, and Di Wu.  Iot security techniques based on machine learning: How do iot devices use ai to enhance security?  IEEE Signal Processing Magazine, 35(5):41–49, 2018.
  * [69] Serena Yeung, Francesca Rinaldo, Jeffrey Jopling, Bingbin Liu, Rishab Mehra, N Lance Downing, Michelle Guo, Gabriel M Bianconi, Alexandre Alahi, Julia Lee, et al.  A computer vision system for deep learning-based detection of patient mobilization activities in the icu.  NPJ digital medicine, 2(1):1–5, 2019.
  * [70] Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, Graham Cormode, and Ilya Mironov.  Opacus: User-friendly differential privacy library in PyTorch.  arXiv preprint arXiv:2109.12298, 2021.
  * [71] Zhuoning Yuan, Yan Yan, Milan Sonka, and Tianbao Yang.  Large-scale robust deep auc maximization: A new surrogate loss and empirical studies on medical image classification.  arXiv preprint arXiv:2012.03173, 2020.
  * [72] Renrui Zhang, Rongyao Fang, Peng Gao, Wei Zhang, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li.  Tip-adapter: Training-free clip-adapter for better vision-language modeling.  arXiv preprint arXiv:2111.03930, 2021.
  * [73] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Places: A 10 million image database for scene recognition.  IEEE transactions on pattern analysis and machine intelligence, 40(6):1452–1464, 2017.
  * [74] Yuqing Zhu, Xiang Yu, Manmohan Chandraker, and Yu-Xiang Wang.  Private-knn: Practical differential privacy for computer vision.  In CVPR, 2020.
  * [75] Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He.  A comprehensive survey on transfer learning.  Proceedings of the IEEE, 109(1):43–76, 2020.


# Fourier decay for curved Frostman measures

Shival Dasu  Department of Mathematics, Indiana University, 831 East 3rd St.,
Bloomington IN 47405<EMAIL_ADDRESS>and  Ciprian Demeter  Department of
Mathematics, Indiana University, 831 East 3rd St., Bloomington IN 47405
<EMAIL_ADDRESS>
###### Abstract.

We investigate Fourier decay for Frostman measures supported on curves with
nonzero curvature. We combine decoupling with known lower bounds for
Furstenberg sets to extend the main result in [8].

###### Key words and phrases:

curvature, decoupling, Furstenberg sets

The second author is partially supported by the NSF grant DMS-2055156

##  1\. introduction

We prove the following result.

######  Theorem 1.1.

Let $\Gamma$ be the graph of a $C^{3}$ function $\gamma:[-1,1]\to{\mathbb{R}}$
satisfying the nonzero curvature condition $\min_{-1\leq x\leq
1}|\gamma^{\prime\prime}(x)|>0$. Let $0<s<1$. Let $\mu$ be a Borel measure
supported on $\Gamma$ satisfying the Frostman condition

$\mu(B(y,r))\lesssim r^{s}$ (1)

for each $y\in{\mathbb{R}}^{2}$ and each $r>0$. Then there exists $\beta>0$
(depending on $s$) such that for each ball $B_{R}\subset{\mathbb{R}}^{2}$ of
radius $R$ we have

$\|\widehat{\mu}\|_{L^{6}(B_{R})}\lesssim R^{\frac{2-2s-\beta}{6}}.$ (2)

This theorem was proved in [8] in the case $\Gamma$ is the parabola

${\mathbb{P}}^{1}=\\{(\xi,\xi^{2}):\;|\xi|\leq 1\\}.$

The location of $B_{R}$ plays no role in the argument, so we may and will
assume $B_{R}$ is centered at the origin. Theorem 1.1 has an intrinsic
interest stemming from its interplay of curvature and the Fourier transform in
the fractal setting. In addition to this, it was observed in [8] that the case
$\Gamma={\mathbb{P}}^{1}$ encodes non trivial sum-product information, as it
implies the non existence of Borel subrings of the reals with Hausdorff
dimension $s\in(0,1)$.

The argument in [8] has two major ingredients. The first one uses in a
critical way the fact that $\Gamma={\mathbb{P}}^{1}$. Following the
discretization of the measure, the $L^{6}$ statement (2) is converted into a
3-energy estimate (see Section 2.5 for a definition). In a similar spirit with
an elementary computation from [1], this is interpreted as an incidence
problem, between $1/R$-squares and $1/R$-neighborhoods of circles. As the
circles are centered on the $x$-axis, the problem can be further reformulated
in the more familiar language of incidences between squares and tubes.

This leads to the second major ingredient in [8]. More precisely, it is proved
that if the 3-energy is too “large”, it forces the existence of a certain
“small” Furstenberg set (see Section 2.7 for a definition). However, lower
bounds from the literature on Furstenberg sets show that they have to be
“large”, leading to a contradiction.

We provide an alternative argument to Theorem 1.1, that bears both
similarities to the one in [8], but also exhibits key differences and provides
new angles that could potentially make it useful for further advances on the
topic. Let us describe how it works.

We also use two main ingredients. The first one is the (regular) $L^{6}$
decoupling.

######  Theorem 1.2.

Assume $F:{\mathbb{R}}^{2}\to{\mathbb{C}}$ has the spectrum inside the
$1/R$-neighborhood of $\Gamma$. Cover this neighborhood with essentially
rectangular boxes $\theta$ with dimensions roughly $R^{-1/2}$ and $R^{-1}$.
Let $F_{\theta}$ be the Fourier restriction of $F$ to $\theta$. Then

$\|F\|_{L^{6}({\mathbb{R}}^{2})}\lesssim_{\epsilon}R^{\epsilon}(\sum_{\theta}\|F_{\theta}\|_{L^{6}({\mathbb{R}}^{2})}^{2})^{1/2}\;\;\;\;\text{(global
version)}$

and for each ball $B_{R}$ of radius $R$

$\|F\|_{L^{6}(B_{R})}\lesssim_{\epsilon}R^{\epsilon}(\sum_{\theta}\|F_{\theta}\|_{L^{6}(w_{B_{R}})}^{2})^{1/2}\;\;\;\;\text{(local
version)},$

where $w_{B_{R}}$ is a smooth approximation of $1_{B_{R}}$

This tool is efficient for handling the 3-energy of measures supported near
$\Gamma$, and it easily implies (2) without the gain $R^{-\beta}$ (to be more
precise, with arbitrarily small $R^{\epsilon}$ losses). While decoupling is
sharp in many applications, Theorem 1.1 is an instance where decoupling needs
an “$\epsilon$-boost” from incidence geometry, in order to lead to sharper
results. We break $\widehat{\mu}$ into wave packets supported on tubes. Then
we partition the spatial domain into light squares -those intersecting only
few tubes- and heavy squares. Refined decoupling, Theorem 2.5, easily takes
care of the $L^{6}$ mass of $\widehat{\mu}$ restricted to the light squares.
The heavy squares need a more delicate argument that brings to bear the second
main ingredient: an improved incidence estimate for tubes, that we derive from
the recent progress on Furstenberg sets.

A surprising feature of the $L^{6}$ argument in [8] is that it takes place on
the spectral side, where $\mu$ lives. This is possible due to the special
nature of the exponent 6, being an even number, but also due to the special
nature of the parabola. Our argument takes place on the spatial side. We start
by breaking $\widehat{\mu}$ into wave packets. The Furstenberg set we
construct consists of the heavy squares, and they live on the spatial side.
Another interesting difference is that [8] uses $(\delta,s,2s,C)$-sets, while
we use $(\delta,1-s,1,C)$-sets. See Definition 2.14. The fact that the upper-
regularity (1) of the measure $\mu$ forces the needed
$(\delta,1-s,1,C)$-discretized structure is a consequence of the fact that
tubes point in the directions determined by arcs $\theta$ on $\Gamma$, with
prescribed mass $\mu(\theta)\approx R^{-s/2}$ close to the maximum value
allowed by (1).

There are a few advantages of our argument. On the minor side, it does not
need the fine discretization at scale $1/R$, present in [8]. The pigeonholing
(P1) in Section 2.2 is a non technical description of the fact that,
essentially, the restriction $\mu_{\theta}$ of $\mu$ to $\theta$ is
concentrated on the union of $M$ arcs of size $1/R$. The additive structure of
these arcs, whose analysis occupies a large part of the proof in [8], is
efficiently taken care by our application of decoupling. Also, we do not make
any serious use of transversality, this feature is only apparent in the proof
of the very standard estimate (24).

On the more serious side, Theorem 1.1 covers all curved $\Gamma$. It is the
special nature of ${\mathbb{P}}^{1}$ that allows for the conversion of the
3-energy into the language of incidences between lines/tubes and
points/squares. Such a mechanism is not available for arbitrary curves. Also,
our proof does not depend in any way on the fact that $6$ is an even integer.
This is because decoupling also does not rely on this feature. Given this, we
believe that the argument presented here is potentially applicable for other
problems. One such example is the case of Frostman measures supported on other
manifolds, for which the correct index replacing $6$ is fractional. These
questions may be connected with novel (e.g. sum-product) phenomena in high
dimensions. We must caution however that the analogues of Furstenberg sets in
higher dimensions are poorly understood, and likely rather difficult.

This begs another question: can one prove Theorem 1.1, by avoiding the use of
Furstenberg sets? At some level, this may seem counterintuitive; the paper [6]
shows the essential equivalence between the discretized ring conjecture and
the existence of non trivial lower bounds for sets of Furstenberg-type.

However, we can give a partial answer to the question, if in addition to the
upper regularity (1) we also assume lower regularity (such a measure is called
AD-regular)

$\mu(B(y,r))\gtrsim r^{s},\;\;\text{for each }y\in\Gamma,\;r\leq 1.$ (3)

In this case, decoupling reduces (2) to proving some sensible estimates for
$\|\widehat{\mu_{\theta}}\|_{L^{6}(B_{R})}$. Such estimates became recently
available, see the discussion in Section 2.1, and (at least some of) their
proofs seem (at least on the surface) fairly disjoint from the circle of ideas
surrounding sum-product theory. This provides an alternative argument for (2)
in the AD-regular case. That being said, Frostman measures are typically not
lower-regular. An interesting related example is the construction in [5] of
additive subgroups of the reals of arbitrary Hausdorff dimension $s\in(0,1)$.
If their Frostman measures were regular, they would be forced to satisfy
improved energy estimates, contradicting the additive structure.

There are many examples showing decoupling encodes sum-product structure,
relevant to 3-energies. To point out just one of them, if $A\subset[1,2]$ is
$\delta$-separated, then we have

$|\\{(a_{1},\ldots,a_{6})\in A^{6}:\;a_{1}+a_{2}+a_{3}=a_{4}+a_{5}+a_{6}\text{
and
}a_{1}a_{2}a_{3}=a_{4}a_{5}a_{6}\;\\}|\lesssim_{\epsilon}\delta^{-\epsilon}|A|^{3}.$

This is an immediate application of Theorem 1.2 to the $\log$ function.

In our argument, we only use decoupling at two spatial scales, $R$ and
$R^{1/2}$. One question remains on whether a multi-scale decoupling could
exploit lower regularity of $\mu$ at appropriate scales, and allow for the
efficient use of the aforementioned improved energy estimates. The heuristic
for this is that $\mu$ may be (very informally) thought of as consisting of
two parts: the AD-regular part with exponent $s$ and the upper-regular part
associated with larger exponents. The latter part is ameanable to induction,
as the upper bound in (2) becomes more favorable as $s$ increases.

We close the introduction with a word about hidden constants. The application
of decoupling will be one of the sources for $R^{\epsilon}$ losses in the
argument. Other similar losses come from pigeonholing, and the fact that wave
packets are not fully spatially supported on tubes. All these losses can be
made arbitrarily small by choosing $\epsilon$ small enough. They will be
compensated by the (fixed) gain guaranteed by the incidence geometry of the
Furstenberg sets. We will be somewhat casual in keeping track of the losses.
In fact, in order to not sacrifice the clarity of the exposition, most of the
losses will be hidden under the rug.

######  Acknowledgment.

We are grateful to Tuomas Orponen for providing feedback on the manuscript.

##  2\. Proof of Theorem 1.1

###  2.1. A warm up: energies of measures and the proof in the AD-regular case

Let $\nu$ be a Borel measure on the real line. We define its $2$ and
$3$-energies at scale $r>0$ as follows

${\mathcal{E}}_{2}(\nu,r)=\int_{{\mathbb{R}}^{3}}\nu([x_{1}+x_{2}-x_{3}-r,x_{1}+x_{2}-x_{3}+r])d\nu(x_{1})d\nu(x_{2})d\nu(x_{3}),$
${\mathcal{E}}_{3}(\nu,r)=\int_{{\mathbb{R}}^{5}}\nu([x_{1}+x_{2}+x_{3}-x_{4}-x_{5}-r,x_{1}+x_{2}+x_{3}-x_{4}-x_{5}+r])d\nu(x_{1})\ldots
d\nu(x_{5}).$

It is clear that if $\nu$ is an upper-regular (i.e satisfies (1)) probability
measure, then

${\mathcal{E}}_{3}(\nu,r)\lesssim{\mathcal{E}}_{2}(\nu,r)\lesssim r^{s}.$

These estimates can be sharp. An example is when $\nu$ is supported on
$r^{-s}$ intervals $I$ of length $r$ in $[0,1]$, with centers in arithmetic
progression. On each $I$ we let $\nu$ be the normalized Lebesgue measure
$r^{s-1}dx$. It is easy to compute that
${\mathcal{E}}_{3}(\nu,r)\sim{\mathcal{E}}_{2}(\nu,r)\sim r^{s}.$

When $\nu$ is also lower-regular according to (3), we have the following
improvement. See [4] and [2].

######  Theorem 2.1.

Assume $\nu$ is some AD-regular probability measure on $[-1,1]$. Then there is
$\beta>0$ depending on $s$ such that for each $0<r\leq 1$ we have

${\mathcal{E}}_{3}(\nu,r)\lesssim{\mathcal{E}}_{2}(\nu,r)\lesssim
r^{s+\beta}.$

We can use this to give a quick proof of Theorem 1.1 if $\mu$ is AD-regular.
This property forces $\mu$ to be supported on roughly $R^{s/2}$ arcs $\theta$
of length $R^{-1/2}$. First, we apply the local version of decoupling, Theorem
1.2

$\|\widehat{\mu}\|_{L^{6}(B_{R})}\lesssim_{\epsilon}R^{\epsilon}(\sum_{\theta}\|\widehat{\mu_{\theta}}\|_{L^{6}(w_{B_{R}})}^{2})^{1/2}\lesssim
R^{\frac{s}{4}+\epsilon}\max_{\theta}\|\widehat{\mu_{\theta}}\|_{L^{6}(w_{B_{R}})}.$

Fix $\theta$. Let $\nu$ be the measure supported on the real line such that

$\nu(S)=\mu_{\theta}(\\{(\xi,\gamma(\xi)):\;\xi\in S\\}).$

We have

$\widehat{\mu_{\theta}}(x)=\int e((\xi,\gamma(\xi))\cdot x)d\nu(\xi).$

It follows that

$\|\widehat{\mu_{\theta}}\|_{L^{6}(w_{B_{R}})}^{6}=\int_{{\mathbb{R}}^{6}}\widehat{w_{B_{R}}}(\xi_{1}+\ldots-\xi_{6},\gamma(\xi_{1})+\ldots-\gamma(\xi_{6}))\;d\nu(\xi_{1})\ldots
d\nu(\xi_{6}).$

We may assume $|\widehat{w_{B_{R}}}|\lesssim R^{2}1_{B(0,R^{-1})}$. This
restricts the domain of integration to
$|\xi_{1}+\xi_{2}+\xi_{3}-\xi_{4}-\xi_{5}-\xi_{6}|\leq R^{-1}$ and leads to
the upper bound

$\|\widehat{\mu_{\theta}}\|_{L^{6}(w_{B_{R}})}^{6}\lesssim
R^{2}{\mathcal{E}}_{3}(\nu,R^{-1}).$

The rescaled version of Theorem 2.1 shows that

${\mathcal{E}}_{3}(\nu,R^{-1})\lesssim\mu(\theta)^{5}(\frac{1}{R})^{s+\beta}\sim
R^{-\frac{7s}{2}-\beta}.$

Putting all these estimates together we arrive at the desired conclusion

$\|\widehat{\mu}\|^{6}_{L^{6}(B_{R})}\lesssim_{\epsilon}R^{\epsilon}R^{\frac{3s}{2}}R^{2}R^{-\frac{7s}{2}-\beta}=R^{2-2s+\epsilon-\beta}.$

###  2.2. Pigeonholing and the fractal distribution of wave packets

Fix a smooth $\psi:{\mathbb{R}}^{2}\to{\mathbb{R}}$ satisfying $0\leq\psi\leq
1_{B(0,1)}$. Write $\psi_{\delta}(\xi)=\delta^{-2}\psi(\frac{\xi}{\delta})$
and $\mu_{\delta}=\mu*\psi_{\delta}$. Then (2) can be reformulated
equivalently as

$\|\widehat{\mu_{1/R}}\|_{L^{6}({\mathbb{R}}^{2})}^{6}\lesssim
R^{{2-2s-\beta}}.$ (4)

Note that $\mu_{1/R}$ is supported on the $1/R$-neighborhood of $\Gamma$. We
partition this neighborhood into essentially rectangular regions $\theta$ of
dimensions roughly $R^{-1/2}$ and $1/R$. We also partition the
$1/R^{1/2}$-neighborhood into sets $\tau$, with dimensions roughly $R^{-1/4}$
and $R^{-1/2}$. Each $\theta$ sits inside some $\tau$.

Let $D,M,P$ be dyadic parameters, with $D$ and $P$ integers. We decompose
$\mu$

$\mu=\sum_{D,M,P}\mu_{D,M,P},$

with each $\mu_{D,M,P}$ satisfying the following properties:

(P1) for each $\theta$, either $\mu_{D,M,P}(\theta)=\mu(\theta)\sim MR^{-s}$,
or $\mu_{D,M,P}(\theta)=0$. We call $\theta$ active if it falls into the first
category.

(P2) each $\tau$ contains either $\sim P$ or no active $\theta$. We call
$\tau$ active if it fits into the first category.

(P3) there are $\sim D$ active $\theta^{\prime}s$, or equivalently, there are
$\sim D/P$ active $\tau^{\prime}s$.

We note that these properties together with (1) force the following
inequalities

$M\lesssim R^{s/2}$ (5) $DM\lesssim R^{s}$ (6) $MP\lesssim R^{3s/4}.$ (7)

The parameter $M$ will guarantee good control over the distribution of the
active $\theta$’s, which will in turn deliver the much needed discretized
structure for our tubes.

For the rest of the argument we fix $D,M,P$ and let
$F=R^{s-2}\widehat{\mu_{D,M,P}*\psi_{1/R}}$ be the $L^{\infty}$ upper
normalized version, $\|F\|_{\infty}\lesssim 1$. Invoking the triangle
inequality, it suffices to prove that for some $\beta>0$ we have

$\|F\|_{L^{6}}^{6}\lesssim R^{4s-10-\beta}.$ (8)

This is because only $O((\log R)^{3})$ of the 3-tuples $(D,M,P)$ are relevant
to the summation. Indeed, we have $O(\log R)$ possible choices for each of $D$
and $P$. The contribution from $M=O(R^{-100})$ is negligible, as it becomes
transparent from the main argument that follows. Given the upper bound (5), we
conclude that there are only $O(\log R)$ relevant values for $M$.

Write $\mu_{\theta}$ for the restriction of $\mu$ to an active $\theta$. Let
$F_{\theta}=R^{s-2}\widehat{\mu_{\theta}*\psi_{1/R}}$ be the Fourier
restriction of $F$ to $\theta$, so that $F=\sum_{\theta}F_{\theta}$. We will
use the following spatial decomposition for $F_{\theta}$, see e.g. Chapter 2
in [3]

$F_{\theta}=\sum_{T\in{\mathcal{T}}_{\theta}}\langle F_{\theta},W_{T}\rangle
W_{T}+O(R^{-100}).$

Each $T$ is a rectangle (referred to as tube) with dimensions $R^{1/2}$ and
$R$, with the long side pointing in the direction normal to $\theta$. The term
$O(R^{-100})$ is negligible, and can be dismissed. Its role is to ensure that
all $T$ sit inside, say, $[-R,R]^{2}$. The wave packet $W_{T}$ is $L^{2}$
normalized, has spectrum inside (a slight enlargement of) $\theta$, and is
essentially concentrated spatially in (a slight enlargement) of $T$. Thus,
with $\chi_{T}$ being a smooth approximation of $1_{T}$, we have

$|W_{T}|\lesssim R^{-3/4}\chi_{T}.$

We group the tubes $T$ according to the magnitude of $|\langle
F_{\theta},W_{T}\rangle|$. Given the dyadic parameter $\lambda$ we write

${\mathcal{T}}_{\lambda,\theta}=\\{T\in{\mathcal{T}}_{\theta}:\;|\langle
F_{\theta},W_{T}\rangle|\sim\lambda\\}.$

We record two key properties. The first one shows that the largest possible
value $\lambda_{max}$ of $\lambda$ satisfies

$\lambda_{max}\sim MR^{-\frac{5}{4}}.$ (9)

######  Proposition 2.2.

For each $T\in{\mathcal{T}}_{\theta}$ we have

$|\langle F_{\theta},W_{T}\rangle|\lesssim MR^{-\frac{5}{4}}.$

###### Proof.

We have

$R^{2-s}|\langle
F_{\theta},W_{T}\rangle|=|\langle\mu_{\theta}*\psi_{1/R},\widehat{W_{T}\rangle}|\leq\|\mu_{\theta}*\psi_{1/R}\|_{1}\|\widehat{W_{T}}\|_{\infty}\lesssim\|\mu_{\theta}\|_{1}R^{3/4}\sim
MR^{\frac{3}{4}-s}.$

∎

The second property is about the fractal-like distribution of the tubes $T$
inside boxes. This is essentially Lemma 3.3 in [8]. We sketch a proof for the
convenience of the reader.

######  Proposition 2.3.

Let $\Delta\in[R^{1/2},R]$. For each rectangle $B$ with dimensions $\Delta$
and $R$, and with orientation identical to that of the tubes in
${\mathcal{T}}_{\theta}$, we have the estimate

$N_{B}=|\\{T\in{\mathcal{T}}_{\lambda,\theta}:\;T\subset
B\\}|\lesssim(\frac{\Delta}{\sqrt{R}})^{1-s}\frac{MR^{\frac{s-5}{2}}}{\lambda^{2}}.$
(10)

###### Proof.

Let $\eta_{B}$ be a smooth approximation of $1_{B}$ with compact support
inside $B(0,1/\Delta)$. Using $L^{2}$ orthogonality and ignoring again
negligible terms, we get

$N_{B}\lambda^{2}\sim\|\sum_{T\in{\mathcal{T}}_{\lambda,\theta}:\;T\subset
B}\langle F_{\theta},W_{T}\rangle
W_{T}\|_{2}^{2}\lesssim\|F_{\theta}\eta_{B}\|_{2}^{2}=R^{2s-4}\|\mu_{\theta}*\psi_{1/R}*\widehat{\eta_{B}}\|_{2}^{2}.$
(11)

Using that $|\widehat{\eta_{B}}|\lesssim|B|1_{B(0,1/\Delta)}=\Delta
R1_{B(0,\Delta^{-1})}$ and $|\psi_{1/R}|\lesssim R^{2}1_{B(0,1/R)}$, we may
write (since $\Delta^{-1}\geq 1/R$)

$\psi_{1/R}*|\widehat{\eta_{B}}|\lesssim\Delta R1_{B(0,2\Delta^{-1})}.$

Combining this with (1) at scale $r=\Delta^{-1}\leq R^{-1/2}$ we find

$\|\mu_{\theta}*\psi_{1/R}*\widehat{\eta_{B}}\|_{\infty}\lesssim\Delta
R\Delta^{-s}.$ (12)

On the other hand

$\|\mu_{\theta}*\psi_{1/R}*\widehat{\eta_{B}}\|_{1}\leq\mu(\theta)\|\psi_{1/R}\|_{1}\|\widehat{\eta_{B}}\|_{1}\lesssim\mu(\theta)\sim
MR^{-s}.$ (13)

Interpolating (12) and (13) leads to

$\|\mu_{\theta}*\psi_{1/R}*\widehat{\eta_{B}}\|_{2}^{2}\lesssim M(\Delta
R)^{1-s}.$

The conclusion follows by combining this with (11)

$N_{B}\lesssim\lambda^{-2}R^{2s-4}(\Delta
R)^{1-s}=(\frac{\Delta}{\sqrt{R}})^{1-s}\frac{MR^{\frac{s-5}{2}}}{\lambda^{2}}.$

∎

###  2.3. Regular decoupling implies no $R^{\beta}$ gain

Recall that we need to prove (8). Let us start by emphasizing the relevance of
the gain coming from $\beta>0$. To do so, we first prove the estimate without
this gain, in fact we will tolerate arbitrarily small losses

$\|F\|_{L^{6}}^{6}\lesssim_{\epsilon}R^{4s-10+\epsilon}.$ (14)

All we need from (10) to argue this is the estimate when $B$ is the square
$[-R,R]^{2}$. Recall that all $T$ in ${\mathcal{T}}_{\lambda,\theta}$ are
subsets of this square, so (10) with $\Delta\sim R$ gives

$|\cup_{\theta}{\mathcal{T}}_{\lambda,\theta}|\lesssim\frac{MD}{\lambda^{2}R^{2}}.$
(15)

Recall also the decomposition

$F=\sum_{\lambda\leq\lambda_{max}}\sum_{\theta\text{
active}}\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle W_{T}.$

Then (14) will follow once we prove that for each $\lambda\leq\lambda_{max}$
we have

$\|\sum_{\theta\text{ active}}\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle W_{T}\|_{6}^{6}\lesssim_{\epsilon}R^{4s-10+\epsilon}.$
(16)

This is indeed enough to imply (14), since the smaller values of $\lambda$
lead to negligible contributions. To prove this estimate, we first use
decoupling, Theorem 1.2 (and Hölder’s inequality combined with property (P3)),
then (15), followed by (9)

$\displaystyle\|\sum_{\theta\text{
active}}\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle W_{T}\|_{6}^{6}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}D^{2}\sum_{\theta\text{
active}}\|\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle W_{T}\|_{6}^{6}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}D^{2}R^{-9/2}\lambda^{6}\sum_{\theta\text{
active}}\|\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\chi_{T}\|_{6}^{6}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}D^{2}R^{-9/2}\lambda^{6}R^{3/2}|\cup_{\theta}{\mathcal{T}}_{\lambda,\theta}|$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}\frac{MD^{3}\lambda^{4}}{R^{5}}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}\frac{M^{5}D^{3}}{R^{10}}.$

Finally, invoking (5) and (6) we have

$M^{5}D^{3}=(MD)^{3}M^{2}\lesssim R^{4s}.$

The combination of the last two inequalities proves the desired conclusion
(16).

######  Remark 2.4.

This argument produces the gain $R^{\beta}$, if at least one of the following
three scenarios holds

(S1) we restrict the summation to values of $\lambda$ slightly smaller than
$\lambda_{max}$

(S2) $M$ is slightly smaller than $R^{s/2}$

(S3) $D$ is slightly smaller than $R^{s/2}$.

In the next sections we will deal with the remaining scenario. More precisely,
we may assume that there is some small enough (to be chosen in the proof of
Theorem 2.16) $\alpha>0$ such that

$\lambda\geq\frac{\lambda_{max}}{R^{\alpha}}=\frac{M}{R^{\alpha+\frac{5}{4}}}$
(17)

$M\geq R^{\frac{s}{2}-\alpha}$ (18) $D\geq R^{\frac{s}{2}-\alpha}.$ (19)

The choice of $\alpha$ will in the end determine the value of $\beta$ in
Theorem 1.1.

For the remainder of the paper we fix such $\lambda,M,D$. We record two
consequences, for later use

$D\lesssim R^{\frac{s}{2}+\alpha}\;\;\text{(due to \eqref{e6} and
\eqref{e22})}$ (20)

$|{\mathcal{T}}=\cup_{\theta}{\mathcal{T}}_{\lambda,\theta}|\lesssim
R^{\frac{1}{2}+4\alpha}\;\;\text{(due to \eqref{e11}, \eqref{e25}, \eqref{e22}
and \eqref{e35}}).$ (21)

To make the refined argument work, we will need to also use the parameter $P$.

###  2.4. Refined decoupling and the improved estimate for light squares

We tile the plane with squares $q$ with side length $\sqrt{R}$. We define

${\mathcal{Q}}_{heavy}=\\{q:\;q\text{ intersects at least
}R^{\frac{s}{2}-\alpha}\;\;\text{tubes }T\in{\mathcal{T}}\\}.$

Also, ${\mathcal{Q}}_{light}$ are the remaining squares. We first deal with
this latter collection, using refined decoupling.

######  Theorem 2.5 ([7]).

Assume $F$ has spectrum inside the $1/R$-neighborhood of $\Gamma$, and the
wave packet decomposition $F=\sum_{T\in{\mathbb{T}}}a_{T}W_{T}$. Let
${\mathcal{Q}}$ be a collection of pairwise disjoint squares with side length
$\sqrt{R}$. Assume (the slight enlargement of) each $q\in{\mathcal{Q}}$
intersects at most N tubes in ${\mathbb{T}}$. Then

$\|F\|_{L^{6}(\cup_{q\in{\mathcal{Q}}}q)}\lesssim_{\epsilon}R^{\epsilon}N^{\frac{1}{3}}(\sum_{\theta}\|F_{\theta}\|_{L^{6}({\mathbb{R}}^{2})}^{6})^{1/6}.$

The advantage of this over the regular decoupling (Theorem 1.2) used in the
previous section is that $N^{\frac{1}{3}}$ replaces the “Hölder loss”
$D^{\frac{1}{3}}$. While $D$ could potentially be as large as
$R^{\frac{s}{2}+\alpha}$ (see (20)), localizing to ${\mathcal{Q}}_{light}$
allows us to use the critically smaller value $N=R^{\frac{s}{2}-\alpha}$.

######  Theorem 2.6.

We have

$\|\sum_{\theta\text{ active}}\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle
W_{T}\|_{L^{6}(\cup_{q\in{\mathcal{Q}}_{light}}q)}^{6}\lesssim_{\epsilon}R^{\epsilon-2\alpha}{R^{4s-10}}.$

###### Proof.

We repeat the computation from the previous section, incorporating the gain
from $N$.

$\displaystyle\|\sum_{\theta\text{
active}}\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle
W_{T}\|_{L^{6}(\cup_{q\in{\mathcal{Q}}_{light}}q)}^{6}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}R^{s-2\alpha}\sum_{\theta\text{
active}}\|\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle W_{T}\|_{6}^{6}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}R^{s-2\alpha-9/2}\lambda^{6}\sum_{\theta\text{
active}}\|\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\chi_{T}\|_{6}^{6}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}R^{s-2\alpha-9/2}\lambda^{6}R^{3/2}|\cup_{\theta}{\mathcal{T}}_{\lambda,\theta}|$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}R^{s-2\alpha}\frac{MD\lambda^{4}}{R^{5}}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon}R^{s-2\alpha}\frac{M^{4}(MD)}{R^{10}}$
$\displaystyle\lesssim_{\epsilon}R^{\epsilon-2\alpha}{R^{4s-10}}.$

We have used (5) and (6). The exponent gain $\beta=2\alpha-\epsilon$ is
positive, if $\epsilon$ is chosen small enough.

∎

In the remainder of the paper we will deal with ${\mathcal{Q}}_{heavy}$. The
gain in this case will come once we prove that this collection has small
cardinality.

###  2.5. Discrete energy estimates, easy and hard

This section provides some intuition for the following sections. Consider a
collection ${\mathcal{P}}$ of $\delta$-separated points in ${\mathbb{R}}^{n}$.
We may define the 2 and 3-energies of ${\mathcal{P}}$ at scale $\delta$ via

${\mathbb{E}}_{2,\delta}({\mathcal{P}})=|\\{(p_{1},p_{2},p_{3},p_{4})\in{\mathcal{P}}^{4}:\;{\operatorname{dist}\,}(p_{1}+p_{2},p_{3}+p_{4})\lesssim\delta\\}|,$
${\mathbb{E}}_{3,\delta}({\mathcal{P}})=|\\{(p_{1},p_{2},p_{3},p_{4},p_{5},p_{6})\in{\mathcal{P}}^{6}:\;{\operatorname{dist}\,}(p_{1}+p_{2}+p_{3},p_{4}+p_{5}+p_{6})\lesssim\delta\\}|.$

We can also use integrals to measure essentially the same quantities

${\mathbb{E}}_{k,\delta}({\mathcal{P}})\approx\frac{1}{|B(0,\delta^{-1})|}\int_{B(0,\delta^{-1})}|\sum_{p\in{\mathcal{P}}}e(p\cdot
x)|^{2k}dx.$

It is rather immediate that

${\mathbb{E}}_{2,\delta}({\mathcal{P}})\lesssim|{\mathcal{P}}|^{3},$
${\mathbb{E}}_{3,\delta}({\mathcal{P}})\lesssim|{\mathcal{P}}|^{5}.$ (22)

If we adopt the integral formulation, it is also immediate that

${\mathbb{E}}_{3,\delta}({\mathcal{P}})\lesssim|{\mathcal{P}}|^{2}{\mathbb{E}}_{2,\delta}({\mathcal{P}}).$

These estimates are sharp e.g. when the points lie in a straight line, and are
$\delta$-close to an arithmetic progression. However, better estimates are
known if the set ${\mathcal{P}}$ is regular, as follows.

######  Definition 2.7.

Given $0<\delta,s<1$, a $\delta$-separated set of points ${\mathcal{P}}$ in
$[-1,1]$ is $(\delta,s)$-regular if

(i) for each interval $I\subset[0,1]$ with length at least $\delta$ we have
$|{\mathcal{P}}\cap I|\lesssim(\frac{|I|}{\delta})^{s}$

(ii) if in addition the interval is centered at some $p\in{\mathcal{P}}$, then
we have $|{\mathcal{P}}\cap I|\sim(\frac{|I|}{\delta})^{s}$.

We have the following discrete reformulation of Theorem 2.1.

######  Theorem 2.8.

Assume ${\mathcal{P}}$ is $(\delta,s)$-regular. There is $\eta>0$ depending on
$s$ such that

${\mathbb{E}}_{2,\delta}({\mathcal{P}})\lesssim|{\mathcal{P}}|^{3-\eta},$

and thus

${\mathbb{E}}_{3,\delta}({\mathcal{P}})\lesssim|{\mathcal{P}}|^{5-\eta}.$

###  2.6. Bush estimates

A bush is a collection of wave packets $W_{T}$, satisfying two properties. The
spectrum of each $W_{T}$ needs to be inside an active $\theta$. Also, the
underlying tubes $T$ intersect a fixed square $q$ with side length $R^{1/2}$
(we will say that ${\mathcal{B}}$ is centered at $q$). It is easy to see that
we may have at most $O(1)$ tubes for each active $\theta$.

######  Proposition 2.9.

For each bush ${\mathcal{B}}$ centered at $q$ and for all coefficients
$a_{T}\in{\mathbb{C}}$ we have

$\|\sum_{T\in{\mathcal{B}}}a_{T}W_{T}\|_{L^{6}(q)}^{6}\lesssim_{\epsilon}R^{\epsilon-7/2}D^{3}P^{2}\|a_{T}\|_{l^{\infty}}^{6}.$

###### Proof.

The function $\sum_{T\in{\mathcal{B}}}a_{T}W_{T}$ has spectrum inside the
$R^{-1}$-neighborhood of $\Gamma$, thus also inside the larger
$R^{-1/2}$-neighborhood. We may apply the local inequality in Theorem 1.2

$\|\sum_{T\in{\mathcal{B}}}a_{T}W_{T}\|_{L^{6}(q)}\lesssim_{\epsilon}R^{\epsilon}(\sum_{\tau\text{
active}}\|\sum_{T\in{\mathcal{B}}\atop{T\sim\tau}}a_{T}W_{T}\|_{L^{6}(w_{q})}^{2})^{1/2}.$

Recall that each $\tau$ is a region with diameter $\sim R^{-1/4}$. We write
$T\sim\tau$ if $T$ is associated with some $\theta$ that sits inside $\tau$.
According to (P3), there are $\sim D/P$ active $\tau$. Thus, Hölder’s
inequality leads to

$\|\sum_{T\in{\mathcal{B}}}a_{T}W_{T}\|_{L^{6}(q)}^{6}\lesssim_{\epsilon}R^{\epsilon}(D/P)^{3}\max_{\tau\text{
active}}\|\sum_{T\in{\mathcal{B}}\atop{T\sim\tau}}a_{T}W_{T}\|_{L^{6}(w_{q})}^{6}.$

We may assume each active $\theta$ inside $\tau$ contributes only one $T$, and
we write $T\sim\theta$. Pick any point $p_{\theta}\in\theta$ and call
${\mathcal{P}}$ the collection of all $p_{\theta}$. The spectrum of $W_{T}$ is
inside $B(p_{\theta},O(R^{-1/2}))$. We may assume $w_{q}$ has spectrum inside
$B(0,R^{-1/2})$. Note that if $T_{i}\sim\theta_{i}$ then the function
$W_{T_{1}}W_{T_{2}}W_{T_{3}}\overline{W_{T_{4}}}\;\overline{W_{T_{5}}}\;\overline{W_{T_{6}}}$
has spectrum inside
$B(p_{\theta_{1}}+p_{\theta_{2}}+p_{\theta_{3}}-p_{\theta_{4}}-p_{\theta_{5}}-p_{\theta_{6}},O(R^{-1/2}))$.
We find that

$\displaystyle\|\sum_{T\in{\mathcal{B}}\atop{T\sim\tau}}a_{T}W_{T}\|_{L^{6}(w_{q})}^{6}$
$\displaystyle\leq\|a_{T}\|_{l^{\infty}}^{6}\sum_{T_{1},\ldots,T_{6}}|\int{\mathcal{F}}(W_{T_{1}}W_{T_{2}}W_{T_{3}}\overline{W_{T_{4}}}\;\overline{W_{T_{5}}}\;\overline{W_{T_{6}}}){\mathcal{F}}(w_{q})|$
$\displaystyle\lesssim\|a_{T}\|_{l^{\infty}}^{6}{\mathbb{E}}_{3,R^{-1/2}}({\mathcal{P}})\frac{|q|}{|T_{1}|^{1/2}\cdot\ldots\cdot|T_{6}|^{1/2}}$
$\displaystyle\lesssim R^{-7/2}\|a_{T}\|_{l^{\infty}}^{6}P^{5}.$

∎

######  Remark 2.10.

Recall that $\tau$ is essentially a rectangle. Because of this, there is no
curvature present in the estimate for
$\|\sum_{T\in{\mathcal{B}}\atop{T\sim\tau}}a_{T}W_{T}\|_{L^{6}(w_{q})}^{6}$.
At this scale, the points $p_{\theta}$ can be thought of as lying on a line.
In the absence of additional structure for these $\theta$, the best exponent
we may use for the 3-energy is 5, cf. (22). However, if the measure $\mu$ were
AD-regular, we could use the better energy estimate in Theorem 2.8. Such an
estimate is enough to produce the gain $R^{-\beta}$ in (1).

###  2.7. Incidence estimates and Furstenberg sets

Consider the family of rectangles
${\mathcal{T}}=\cup_{\theta}{\mathcal{T}}_{\lambda,\theta}$. They point in at
most $D$ directions, and recall that $D\lesssim R^{\frac{s}{2}+\alpha}$.
Recall also that $|{\mathcal{T}}|\lesssim R^{\frac{1}{2}+4\alpha}$. We are
interested in finding a good upper bound for the number of (pairwise disjoint)
squares $q$ with side length $\sqrt{R}$ that intersect close to the maximum
possible number of tubes. More precisely, we have defined the family

${\mathcal{Q}}_{heavy}=\\{q:\;q\text{ intersects at least
}R^{\frac{s}{2}-\alpha}\;\;\text{tubes }T\in{\mathcal{T}}\\}.$

Consider the number of incidences

${\mathcal{I}}({\mathcal{T}},{\mathcal{Q}}_{heavy})=|\\{(T,q)\in{\mathcal{T}}\times{\mathcal{Q}}_{heavy}:\;T\text{
intersects }q\\}|.$

Let us first point out the following easy estimate, in the spirit of the
bilinear Kakeya inequality.

######  Theorem 2.11.

We have

$|{\mathcal{Q}}_{heavy}|\lesssim R^{1-s}R^{10\alpha+\frac{2\alpha}{s}}.$ (23)

###### Proof.

We perform a double counting argument. On the one hand we have

$|{\mathcal{Q}}_{heavy}|R^{\frac{s}{2}-\alpha}\lesssim{\mathcal{I}}({\mathcal{T}},{\mathcal{Q}}_{heavy}).$

On the other hand, let
${\mathcal{Q}}(T_{1})=\\{q\in{\mathcal{Q}}_{heavy}:\;q\text{ intersects
}T_{1}\\}$. We claim that for each $T_{1}\in{\mathcal{T}}$

${\mathcal{Q}}(T_{1})\lesssim\frac{|{\mathcal{T}}|}{R^{\frac{s}{2}-\alpha-\frac{2\alpha}{s}}}.$
(24)

Let us postpone the proof of this claim to a few lines below, and record the
immediate consequence

${\mathcal{I}}({\mathcal{T}},{\mathcal{Q}}_{heavy})\lesssim\frac{|{\mathcal{T}}|^{2}}{R^{\frac{s}{2}-\alpha-\frac{2\alpha}{s}}}\lesssim
R^{1-\frac{s}{2}+9\alpha+\frac{2\alpha}{s}}.$

The two incidence estimates combine to prove the desired bound for
${\mathcal{Q}}_{heavy}$.

Now back to the claim. Fix $T_{1}\in{\mathcal{T}}$. Pick any
$q\in{\mathcal{Q}}(T_{1})$. There must be at least $\sim
R^{\frac{s}{2}-\alpha}$ tubes $T$ intersecting $q$, call them
${\mathcal{T}}(q)$, in such a way that the angle between $T$ and $T_{1}$ is
$\gtrsim R^{-\frac{2\alpha}{s}}$. This is because the number of active
$\theta$ (so $\mu(\theta)\sim MR^{-s}$) supported by an arc $I$ of length $\ll
R^{-\frac{2\alpha}{s}}$ is

$\lesssim\frac{\mu(I)}{MR^{-s}}\ll\frac{R^{-2\alpha}}{MR^{-s}}\lesssim
R^{\frac{s}{2}-\alpha}.$

Simple geometry shows that a given $T$ can only be in
${\mathcal{T}}(q_{1})\cap{\mathcal{T}}(q_{2})$ if
${\operatorname{dist}\,}(q_{1},q_{2})\lesssim
R^{\frac{1}{2}+\frac{2\alpha}{s}}$. In other words, each $T\in{\mathcal{T}}$
may be in at most $R^{\frac{2\alpha}{s}}$ sets ${\mathcal{T}}(q)$.

With this in mind, another double counting proves the claim

$|{\mathcal{Q}}(T_{1})|R^{\frac{s}{2}-\alpha}\lesssim{\mathcal{I}}(\cup_{q\in{\mathcal{Q}}(T_{1})}{\mathcal{T}}(q),{\mathcal{Q}}(T_{1}))\lesssim|{\mathcal{T}}|R^{\frac{2\alpha}{s}}.$

∎

To improve the easy estimate, we need to invoke the theory of Furstenberg
sets. Definition 2.7 admits the following abstract formulation.

######  Definition 2.12.

Let $(X,d)$ be a metric space, let $s,\delta>0$. A set ${\mathcal{P}}\subset
X$ is called a $(\delta,s,C)$-set if for each $x\in X$ and $r\geq\delta$ we
have

$|{\mathcal{P}}\cap B(x,r)|\leq C(\frac{r}{\delta})^{s}.$

We allow $C$ to depend on $\delta$.

We define the distance between two lines in the plane as

$d(l_{1},l_{2})=\|\pi_{1}-\pi_{2}\|+|a_{1}-a_{2}|,$

where $\pi_{i}$ is the orthogonal projection onto the linear subspace
${\mathcal{L}}_{i}$ in the direction of $l_{i}$, and
$a_{i}=l_{i}\cap{\mathcal{L}}_{i}^{\perp}$. The space ${\mathcal{A}}(2,1)$ of
all lines becomes a metric space when equipped with this distance. It is easy
to see that a collection of lines ${\mathcal{L}}$ intersecting the unit square
is a $(\delta,s,C)$-set if each rectangle with dimensions $1$ and
$r\in(\delta,1]$ intersects $\lesssim C(\frac{r}{\delta})^{s}$ lines in
${\mathcal{L}}$, at full length (by that we mean that the intersection is a
line segment of length $\sim 1$).

Let ${\mathcal{L}}$ be the collection of central axes (i.e. the lines
bisecting the tubes in the long direction) of the rescaled rectangles
$R^{-1}T$ with $T\in{\mathcal{T}}$.

######  Lemma 2.13.

The collection ${\mathcal{L}}$ is an $(R^{-1/2},1,R^{4\alpha})$-set.

###### Proof.

Fix a rectangle $B$ with dimensions $1$ and $r\geq R^{-1/2}$. The statement
follows from the following observations. For each direction, that is for each
active $\theta$, the number of lines in ${\mathcal{L}}$ in this direction that
intersect $B$ at full length is at most

$(r\sqrt{R})^{1-s}\frac{MR^{\frac{s-5}{2}}}{\lambda^{2}}.$

This is just a reformulation of (10). Using (17) and (18), the expression
above is at most $R^{3\alpha}(r\sqrt{R})^{1-s}$.

The directions of the lines are $R^{-1/2}$\- separated, as they correspond to
distinct $\theta$’s. The directions of the lines intersecting $B$ at full
length have to lie inside an arc $I$ on the circle with measure $\sim r$.
Thus, due to the curvature of $\Gamma$, the corresponding $\theta$’s lie
inside a ball $B(x,10r)$. Since for each active $\theta$ we have
$\mu(\theta)\sim MR^{-s}$ and since $\mu(B(x,10r))\lesssim r^{s}$, we conclude
that there can only be $\lesssim\frac{(Rr)^{s}}{M}$ contributing $\theta$’s.
Using again (18), this is at most $R^{\alpha}(r\sqrt{R})^{s}$. We conclude
that there are at most $R^{4\alpha}r\sqrt{R}$ lines intersecting $B$ at full
length.

∎

Let us write $l(\delta)$ for the $\delta$-neighborhood of the line $l$.

######  Definition 2.14.

Let $0<s\leq 1$, $0<t\leq 2$, $\delta\in(0,1)$. A $\delta$-separated set
$F\subset{\mathbb{R}}^{2}$ is called a discretized
$(\delta,s,t,C)$-Furstenberg set if there is a $(\delta,t,C)$-set of lines
${\mathcal{L}}$ with $|{\mathcal{L}}|\geq C^{-1}\delta^{-t}$ such that $F\cap
l(\delta)$ contains a $(\delta,s,C)$-set with cardinality $\geq
C^{-1}\delta^{-s}$, for each $l\in{\mathcal{L}}$ .

Each $(\delta,s,t,C)$-Furstenberg set is also a
$(\delta,s,t,C^{\prime})$-Furstenberg set if $C^{\prime}\geq C$.

We will need the case $t=1$ of the following result, see [9].

######  Theorem 2.15.

If $0<s<1$, $s<t\leq 2$, and $\epsilon<\epsilon(s,t)$ then every discretized
$(\delta,s,t,\delta^{-\epsilon})$-Furstenberg set has cardinality
$\gtrsim\delta^{-2s-\epsilon}$.

Using this result we can now improve on the cardinality of
${\mathcal{Q}}_{heavy}$.

######  Theorem 2.16.

Assume $\alpha<\alpha(s)$ is small enough. Then

$|{\mathcal{Q}}_{heavy}|\lesssim R^{1-s-2\alpha}.$ (25)

###### Proof.

Assume for contradiction that $|{\mathcal{Q}}_{heavy}|\geq R^{1-s-2\alpha}$.
We will show that the centers of the squares in ${\mathcal{Q}}_{heavy}$ form a
(rescaled) Furstenberg set (with $\delta=R^{-1/2}$), whose cardinality will be
forced to be large by the previous theorem. So large, that in fact it will
violate the upper bound (23).

First, let us observe that

${\mathcal{I}}({\mathcal{T}},{\mathcal{Q}}_{heavy})\gtrsim|{\mathcal{Q}}_{heavy}|R^{\frac{s}{2}-\alpha}\geq
R^{1-\frac{s}{2}-3\alpha}.$

Partition ${\mathcal{T}}={\mathcal{T}}_{light}\cup{\mathcal{T}}_{heavy}$ where
$T\in{\mathcal{T}}_{light}$ if $T$ intersects at most
$R^{\frac{1-s}{2}-10\alpha}$ squares in ${\mathcal{Q}}_{heavy}$. The axes of
the tubes in ${\mathcal{T}}_{heavy}$ will serve as the (rescaled) lines in the
definition of the Furstenberg set. We need to prove that there are many such
lines.

To achieve this, we first observe that

${\mathcal{I}}({\mathcal{T}}_{light},{\mathcal{Q}}_{heavy})\lesssim|{\mathcal{T}}|R^{\frac{1-s}{2}-10\alpha}\lesssim
R^{1-\frac{s}{2}-6\alpha}.$

The two incidence estimates force
${\mathcal{I}}({\mathcal{T}}_{heavy},{\mathcal{Q}}_{heavy})\gtrsim
R^{1-\frac{s}{2}-3\alpha}.$ We may also write, using (24)

${\mathcal{I}}({\mathcal{T}}_{heavy},{\mathcal{Q}}_{heavy})\lesssim|{\mathcal{T}}_{heavy}|R^{\frac{1-s}{2}+5\alpha+\frac{2\alpha}{s}}.$

Putting things together, we find

$|{\mathcal{T}}_{heavy}|\gtrsim R^{\frac{1}{2}-8\alpha-\frac{2\alpha}{s}}.$

This lower bound together with Lemma 2.13 implies that the central axes of the
rescaled tubes $R^{-1}T$, $T\in{\mathcal{T}}_{heavy}$, call them
${\mathcal{L}}_{heavy}$, form a
$(R^{-1/2},1,R^{100\alpha+\frac{2\alpha}{s}})$-set.

Take $F$ to consist of the rescaled centers $R^{-1}c(q)$ of the squares
$q\in{\mathcal{Q}}_{heavy}$. It is not hard to see that $F$ is a discretized
$(R^{-1/2},1-s,1,R^{100\alpha+\frac{2\alpha}{s}})$ \- Furstenberg set,
relative to ${\mathcal{L}}_{heavy}$. We picked the large exponent
$100\alpha+\frac{2\alpha}{s}$ for extra safety, and of course, anything larger
would also work. The fact that $F\cap l(R^{-1/2})$ is a
$(R^{-1/2},1-s,R^{100\alpha+\frac{2\alpha}{s}})$-set for each
$l\in{\mathcal{L}}_{heavy}$ follows by an argument similar to the earlier
ones, so we omit it. The needed lower bound for $|F\cap l(R^{-1/2})|$ follows
from the definition of ${\mathcal{T}}_{heavy}$.

It follows from Theorem 2.16 that, if $\alpha$ is chosen small enough so that
$100\alpha+\frac{2\alpha}{s}<\epsilon(1-s,1)$, then

$|{\mathcal{Q}}_{heavy}|=|F|\gtrsim R^{1-s}R^{100\alpha+\frac{2\alpha}{s}}.$

This contradicts (23).

∎

###  2.8. Improved estimates for the heavy squares

We prove the remaining $L^{6}$ estimate restricted to the heavy squares,
matching the one for the light squares.

######  Theorem 2.17.

$\|\sum_{\theta\text{ active}}\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle
W_{T}\|_{L^{6}(\cup_{q\in{\mathcal{Q}}_{heavy}}q)}^{6}\lesssim_{\epsilon}R^{\epsilon-2\alpha}R^{4s-10}.$

###### Proof.

Each $q\in{\mathcal{Q}}_{heavy}$ determines the bush ${\mathcal{B}}(q)$
consisting of all $T\in{\mathcal{T}}$ that intersect (a slight enlargement of)
$q$. Due to rapid decay, we may write for each $x\in q$

$\sum_{\theta\text{ active}}\sum_{T\in{\mathcal{T}}_{\lambda,\theta}}\langle
F_{\theta},W_{T}\rangle W_{T}(x)=\sum_{T\in{\mathcal{B}}(q)}\langle
F,W_{T}\rangle W_{T}(x)+O(R^{-100}).$

Thus, it suffices to estimate

$\sum_{q\in{\mathcal{Q}}_{heavy}}\|\sum_{T\in{\mathcal{B}}(q)}\langle
F,W_{T}\rangle W_{T}\|_{L^{6}(q)}^{6}\;.$

Using Proposition 2.9, followed by (9) and (25), this is dominated by

$|{\mathcal{Q}}_{heavy}|\lambda^{6}R^{\epsilon-7/2}D^{3}P^{2}\lesssim
R^{\epsilon-2\alpha}R^{-10-s}M^{6}D^{3}P^{2}.$

The theorem now follows once we notice that
$M^{6}D^{3}P^{2}=(MD)^{3}(MP)^{2}M\lesssim R^{5s}$, by invoking (5), (6) and
(7).

∎

Combining this result with Remark 2.4 and Theorem 2.6 delivers the proof of
(8), and thus of Theorem 1.1.

## References

  * [1] J. Bourgain and C. Demeter, The proof of the $l^{2}$ decoupling conjecture Ann. of Math. (2) 182 (2015), no. 1, 351-389
  * [2] L. Cladek and T. Tao Additive energy of regular measures in one and higher dimensions, and the fractal uncertainty principle, https://arxiv.org/pdf/2012.02747.pdf
  * [3] C. Demeter, Fourier restriction, decoupling, and applications Cambridge Studies in Advanced Mathematics, 184. Cambridge University Press, Cambridge, 2020. xvi+331 pp.
  * [4] S. Dyatlov, J. Zahl, Spectral gaps, additive energy, and a fractal uncertainty principle Geom. Funct. Anal. 26 (2016), no. 4, 1011-1094
  * [5] P, Erdos and B. Volkmann Additive Gruppen mit vorgegebener Hausdorffscher Dimension. (German) J. Reine Angew. Math. 221 (1966), 203-208
  * [6] N. Katz and T. Tao Some connections between Falconer’s distance set conjecture and sets of Furstenburg type, New York J. Math. 7 (2001), 149-187.
  * [7] Guth, Larry; Iosevich, Alex; Ou, Yumeng; Wang, Hong On Falconer’s distance set problem in the plane. Invent. Math. 219 (2020), no. 3, 779-830
  * [8] T. Orponen, Additive properties of fractal sets on the parabola, arxiv.org/pdf/2205.02770.pdf
  * [9] T. Orponen and P. Shmerkin, On the Hausdorff dimension of Furstenberg sets and orthogonal projections in the plane arXiv:2106.03338