# Adversarial Fine-tune with Dynamically Regulated Adversary

Pengyue Hou  Ming Zhou  Jie Han  Petr Musilek  Xingyu Li

###### Abstract

Adversarial training is an effective method to boost model robustness to
malicious, adversarial attacks. However, such improvement in model robustness
often leads to a significant sacrifice of standard performance on clean
images. In many real-world applications such as health diagnosis and
autonomous surgical robotics, the standard performance is more valued over
model robustness against such extremely malicious attacks. This leads us to
the question: to what extent can we improve the robustness of the model
without sacrificing standard performance? This work tackles this problem and
proposes a simple yet effective transfer learning based adversarial training
strategy that disentangles the negative effects of adversarial samples on
model’s standard performance. In addition, we introduce a training-friendly
adversarial attack algorithm, which facilitates the boost of adversarial
robustness without introducing significant training complexity. Extensive
experiments show that the proposed approach outperforms previous adversarial
training algorithms with the following objective: to improve the robustness of
the model while preserving model’s standard accuracy on clean data.

##  I Introduction

Deep learning has been widely applied to real-world applications with
excellent performance, such as autonomous driving, efficient web search and
facial recognition. However, recent works have shown that most deep learning
models, including state-of-the-art deep neural networks (DNNs), are vulnerable
to adversarial attacks [1, 2, 3, 4]. These adversarial samples are carefully
designed by adding small, human-imperceptible noises to original images, but
can lead a well-trained classifier to wrong predictions. To improve model
adversarial robustness, many defense strategies, such as data augmentation,
gradient masking, adversarial example detection, adversarial training, etc.,
have been proposed with the aim of finding countermeasures to protect DNNs [4,
5, 6, 7]. Particularly, adversarial training is widely accepted as the most
effective solution. It incorporates adversarial data in model training and
helps build model robustness to adversarial attacks.

Despite its success in improving model robustness to adversarial data, state-
of-the-art (SOTA) adversarial training strategies are observed to cause model
performance degradation [7, 8, 9, 10]. For instance, SOTA adversarial training
methods such as TRADES [11] loses about 10% standard accuracy for a 50%
adversarial robustness improvement on CIFAR-10 image set. This observation has
led to a discussion of the relationship between adversarial robustness and
standard generalization (e.g., classification accuracy), with a central debate
on whether accuracy and robustness are intrinsically in conflict. Some
studies, for example, the works by Tsipras et al. [12] and Raghunathan et al.
[13], claim that the trade-off of model accuracy and adversarial robustness is
unavoidable due to the nature of DNNs. In contrast, Raghunathan et al. [14,
15] argue that the robustness-accuracy trade-off could disappear with
unlimited data. Yang et al. [16] present a theoretical analysis, as well as a
proof-of-concept example, showing that this trade-off is not inherent and
arguing that the observed accuracy-robustness trade-off is introduced by
limitations in current adversary training methods. More recently, Xie et al.
[17] leverage adversarial samples to improve model accuracy by introducing an
auxiliary batch normalization layer particularly designed for adversarial
samples. However, this study does not discuss if the model robustness is
improved or not. To summarize, although adversarial training improves model
robustness against adversarial attacks, how to achieve this goal without
trading off model accuracy on clean data in adversarial training is still an
open question and remains under explored.

It should be noted that a big loss of standard performance on clean data is
unacceptable in many applications that might cause severe consequences.
Instead, the standard performance is more valued than model robustness against
the extremely malicious adversarial attacks. Such applications include medical
diagnosis, autonomous surgical robotics, etc. This leads to the question: To
what extent can we boost model robustness without sacrificing standard
performance? Unlike prior adversarial training works that allow model
performance degradation on clean data, we investigate if it is feasible to
improve adversarial performance without any standard performance loss.
Specifically, this paper proposes a pre-train based adversarial training
strategy, where adversarial training is applied to clean-data (vanilla-
trained) models. To prevent model catastrophic forgetting in model refining,
we follow the replay-based strategy and maintain the 1:1 clean-adversarial
data ratio in model refinement. To further reduce adversarial samples’
negative impacts on standard accuracy, we incorporate novel dynamically
regulated adversarial (DRA) samples in model adversarial refinement. Unlike
most adversarial attacks that add adversarial noise to every image pixel, DRA
searches for highly stimulated adversarial features and generates adversarial
samples accordingly. Such strategy in adversarial sample generation enforces
the model refinement to learn descriptive but non-robust features. Extensive
experimentation shows that the proposed adversarial training strategy improves
model adversarial robustness with a large margin, but without standard
performance loss. The contributions of this study are summarized as follows.

  * •

We propose a simple, yet generic adversarial training strategy that improves
adversarial robustness by a large margin without sacrificing standard
accuracy. We argue that the proposed method is significant for applications
where standard performance is more valued than adversarial robustness.

  * •

We further introduce an unbounded adversarial attack method, namely DRA. It
introduces smaller image distortions and facilitates the adversarial
refinement to focus on the learning of most descriptive but non-robust
features.

  * •

We show that our adversarially trained models exhibit robustness not only to
adversarial samples; but also against naturally corrupted images, which
suggests its potential for real-world applications.

The rest of this paper is organized as follows. Section II presents a brief
review of related works. Section III formulates the target problem and
specifies our motivations. The technical details of the proposed method are
elaborated in Section IV. Section V presents extensive experiments and
discussions, then followed by our conclusions in section VI.

##  II Related Works

Adversarial attacks aim to add small, carefully-designed, human-imperceptible
noises to clean data to fool a machine learning model. Depending on the
availability of prior knowledge of the target, adversarial attacks can be
categorized into two types: white box attacks and black box attacks. Since
adversarial samples are transferable, many white box attacks follow the
paradigm that exploits a surrogate white box model to generate attack samples
[18, 6, 19]. This section briefly reviews the first-order white-box
adversarial attacks, due to their close relevance to this study.

Many white box attacks belong to first-order adversarial methods. Fast
Gradient Method (FGM) is an intuitive back-propagation based method to
generate adversarial samples [2, 20] and evolves into a better constraint
method, namely Fast Gradient Sign Method (FGSM) [3]. FGSM searches for
adversarial samples following the direction of the gradients of current
parameter values and projects them on images.

$x_{adv}=x+\epsilon\cdot sign(\nabla_{x}L(x,y;\theta)),$ (1)

where $\epsilon$ is the noise constraint for projecting the attack noises onto
$L_{\infty}$ ball, $x$ and $y$ are the clean input samples and their labels in
dataset $D$ and $L(\cdot)$ is the loss function.

Iterative Fast Gradient Sign Method (I-FGSM), also called Basic Iterative
Method (BIM) [21], is a multi-step variant of FGSM. It provides stronger
adversarial examples but with more computational cost. Projected Gradient
Descent (PGD) [4] is another iterative variant of FGSM with a random starting
point.

$x_{adv}^{t}=\Pi_{S}[x_{adv}^{t-1}+\epsilon\cdot
sign(\nabla_{x}L(x_{adv}^{t-1},y;\theta)],$ (2)

where $t$ denotes the number of iteration and $\Pi_{S}$ is the clipping
function that forces the value to reside in a predetermined range. PGD is
widely agreed to be one of the strongest attacks and often used to benchmark
model robustness.

Adversarial training incorporates adversarial samples in model optimization
and is considered a very effective method to boost model robustness.
Particularly, Madry et al. [7] formulate adversarial training as a MinMax
optimization problem,

$\mathcal{L}(\theta)=\mathbb{E}_{(x,y)\sim D}[\underset{\delta\in
S}{\mathrm{max}}L(x+\delta,y;\theta)],$ (3)

Where $\delta$ is the perturbation superimposed on the input, and $\theta$
represents network parameters. The inner maximization tries to find
adversarial samples that maximize the loss within the allowed permutation
range, while the outer minimization tries to minimize the adversarial loss.

Since Madry’s MinMax optimization is prone to over-fit adversarial samples,
many adversarial training methods mix clean data and adversarial data in
training.

$\mathcal{L}(\theta)=\hskip 1.4457pt\mathbb{E}_{(x,y)\sim
D}[L(x,y;\theta)+\underset{\delta\in S}{\mathrm{max}}L(x+\delta,y;\theta)].$
(4)

Note, in (4), adversarial examples is leveraged to regularize the vanilla
training on clean data. Recently, Raghunathan et al. [14] introduce robust
self-training (RST) to balance the vanilla and adversarial loss by a
regularization parameter $\beta>0$.

$\mathcal{L}(\theta,\beta)=\hskip 1.4457pt\mathbb{E}_{(x,y)\sim
D}[L(x,y;\theta)+\beta\underset{\delta\in
S}{\mathrm{max}}L(x+\delta,y;\theta)].$ (5)

Another regularized adversarial training strategy, TRADES [11], is proposed to
boost model robustness following the Locally-Lipschitz smoothness constraint.

$\displaystyle\mathcal{L}(\theta,\lambda)$ $\displaystyle=$
$\displaystyle\hskip 1.4457pt\mathbb{E}_{(x,y)\sim D}[L(f_{\theta}(x),y)$
$\displaystyle+$ $\displaystyle 1/\lambda\cdot\underset{\delta\in
S}{\mathrm{max}}L(f_{\theta}(x+\delta),f_{\theta}(x))],$

where $f_{\theta}$ denotes the training model parameterized by $\theta$.
Unlike RST computing a adversarial loss between the prediction
$f_{\theta}(x+\delta)$ and label $y$ as the regularization term in (5), TRADE
regularizes the training by calculating
$L(f_{\theta}(x+\delta),f_{\theta}(x))$ from a pair of clean sample $x$ and
its adversarial version $x+\delta$. The regularization parameter $\lambda$
determines the trade-off between accuracy and robustness in the overall
optimization. It is worth notice that small regularization parameter helps the
model emphasize more on accuracy over robustness which closely align to our
objective of adversarial training. However, we show later in Section V that
even very small value of $\lambda^{-1}$ cannot guarantee models to achieve
comparable accuracy to vanilla trained models.

Pre-training is a popular training framework that can help reduce training
time or improve accuracy performance for fine-tuning downstream tasks. Jeddi
et al. [22] start with a clean data pre-trained model and fine-tune with PGD
adversarial training with the aim of reducing the time cost and overfitting
issue [23]. Hendrycks et al. [24] adversarially pre-train their model on a
downsampled ImageNet and apply adversarial fine-tuning which can significantly
improve model robustness on CIFAR datasets compare with adversarial training
from scratch. Chen et al. [25] show that self-supervised pre-training such as
Selfie [26], Jigsaw [27], also lead to better robustness than traditional
adversarial training. In contrast to previous adversarial pre-training
strategies, our approach mainly focuses on maintaining accuracy and treats
adversarial robustness as an added bonus. More specifically, we incorporate a
novel adversary generating method (DRA) to facilitate our goal by reducing the
learning complexity of adversarial training.

TABLE I: Performance of adversarial training methods on MNIST and CIFAR-10. Adversarial accuracy is evaluated with PGD attacks.  | MNIST ($\epsilon=0.3$) | CIFAR-10 ($\epsilon=8/255$)
---|---|---
| $\mathcal{A}_{std}$ | $\mathcal{A}_{rob}$ | $\mathcal{A}_{std}$ | $\mathcal{A}_{rob}$
Vanilla | 99.3% | 0.3% | 93.0 % | 0.0%
Madry’s [7] | 99.2% | 95.6% | 87.3% | 47.0%
Trades($1/\lambda=1$) [11] | 99.3% | 94.1% | 86.6% | 44.3%
Trades($1/\lambda=6$) [11] | 99.3% | 96.0% | 81.2% | 53.5%
MART [28] | 99.1% | 96.2% | 83.4% | 52.8%

##  III Primitives

###  III-A Problem Formulation

In this study, we focus on improving model robustness to imperceptible, in-
distribution adversarial samples defined by Fawzi et al. [29]. Briefly, assume
that clean data follows a distributing $D$, in-distribution adversarial
samples $(x^{\prime})$ can be also roughly described within $D$.

To answer the question: to what extent can we boost model robustness without
sacrificing standard performance? We formulate the problem as

$\displaystyle\mathcal{L}(\theta)=\hskip 1.4457pt\mathbb{E}_{(x,y)\sim
D}[\underset{\delta\in S}{\mathrm{max}}L(x+\delta,y;\theta)]$ (7)
$\displaystyle\text{s.t.}\hskip 7.22743pt\mathbb{E}_{(x,y)\sim
D}[L(x,y;\theta)]\leq\mathbb{E}_{(x,y)\sim D}[L(x,y;\theta_{std})],$

where $\theta_{std}=\underset{\theta}{\mathrm{argmin}}\mathbb{E}_{(x,y)\sim
D}[L(x,y;\theta)]$ represents the model with parameter $\theta_{std}$ that
yields the minimized standard loss. Comparing the new problem in (7) with
previous adversarisal training methods in (3-II), the new regularization term
in (7) explicitly defines the behavior of the model: to improve adversarial
robustness without the loss of model’s standard performance.

###  III-B Motivation

To tackle the question in (7), we re-examine SOTA adversarial training
strategies and obtain an interesting observation. Yang et al. [16] show that
many real image sets, such as MNIST, CIFAR-10, SVHN and Restricted ImageNet,
are $r$-separated, with the smallest inter-category sample distance being no
smaller than 2$r$. Furthermore, their empirical separation distance is 3x-7x
larger than the typical adversarial perturbation constraint $\epsilon$ adopted
in prior arts, i.e. $\epsilon<r$. In theory, any $r$-separated dataset has
more than one classifier that are both accurate and robust up to perturbations
of size $r$. This $r$-separated claim aligns well with adversarial robustness
experiments on the MNIST dataset in previous adversarial training studies [7,
11, 28]. However, as summarized in Table I where the standard performance and
adversarial performance are denoted by $\mathcal{A}_{std}$ and
$\mathcal{A}_{rob}$ respectively, on CIFAR-10 which is a more complicated
dataset, previous adversarial training often leads to around 10% standard
accuracy drop for models to gain desired adversarial robustness.

From the above observation, we hypothesize that the observed trade-off between
adversarial robustness and standard accuracy is due to the limitation of model
capacity. The problem of image classification on the MNIST dataset is
relatively easy and the models adopted in previous adversarial training
studies have enough capacity to jointly benefit from standard and adversarial
samples following (3-II). However, for complex problems such as classification
on CIFAR-10 dataset, conventional adversarial training in (3-II) increases the
difficulty in model optimization due to the noisy representation of
adversarial perturbations. In another word, learning both standard features
and adversarial features (i.e. those non-robust, yet highly predictive
patterns [30]) is beyond the capacity of those models (e.g. ResNet-18,
ResNet-50, etc.).

Under the hypothesis that model capacity is not enough to simultaneously learn
standard and adversarial features, we value clean-data accuracy over
adversarial robustness. Thus, we made two modifications to existing
adversarial training strategies to tackle the problem formulated in (7).
Briefly, we propose a transfer learning based adversarial training strategy
that starts with a clean data pre-trained model which already has strong
generalizability on clean data. To further reduce the learning complexity, we
incorporate easy-to-learn adversarial samples in the adversarial fine-tuning.
Please refer to the next section for details of the proposed method.

Figure 1: A conceptual demonstration of our adversarial fine-tune method.
Classifier 1 represents the clean data pre-trained classifier which is
accurate but not robust to adversarial samples. Our adversarial fine-tune
method seeking for both accurate and robust classifier by pushing the
classifier 2 out of the yellow adversarial regions defined by $\delta\in S$.

Figure 2: Systematic diagram of the proposed adversarial transfer adversarial
training strategy, where we use various ResNets as the backbone. From left to
right: DRA, a proposed method to generate adversarial samples. It filters out
negligible adversarial noises and reduces adversarial training complexity. The
orange block and green block represent standard training and robust training,
respectively. in standard training, we train a model with $\theta_{std}$ on
clean data that yields high standard performance. Then the robust training
aims to find a better $\theta$ in the vicinity of $\theta_{std}$ to boost
adversarial robustness without standard performance loss.

##  IV Methodology

###  IV-A Transfer Adversarial Training

SOTA adversarial training methods usually train a robust model from scratch
using either adversarial data only in (3) or a combination of clean and
adversarial data following (4-II). However, none of them guarantee that the
standard performance will be preserved. Indeed, due to the highly non-convex
loss surface in model optimization, optimizing both targets simultaneously may
be at odds with each other [12].

To solve the primal conditioned optimization problem in (7), we apply the
Karush-Kuhn-Tucker (KKT) approach and obtain a dual unconstrained optimization
problem by introducing a KKT coefficient $\lambda$,

$\displaystyle\mathcal{L}(\theta,\lambda)$ $\displaystyle=$
$\displaystyle\hskip 1.4457pt\mathbb{E}_{(x,y)\sim D}[\underset{\delta\in
S}{\mathrm{max}}L(x+\delta,y;\theta)]$ $\displaystyle+$
$\displaystyle\lambda\mathbb{E}_{(x,y)\sim
D}[L(x,y;\theta)-L(x,y;\theta_{std})]].$

Therefore, the solution to the dual problem

$\displaystyle\underset{\theta}{\mathrm{min}}\underset{\lambda,\lambda>0}{\mathrm{max}}\mathcal{L}(\theta,\lambda)$
(9)

is identical to the solution of the primal problem in (7). Note that to solve
the problem in (9), we need the margin value $\mathbb{E}_{(x,y)\sim
D}[L(x,y;\theta_{std})]$ which is fixed before the adversarial training
described in (IV-A). That is, a model with $\theta_{std}$ is already obtained
for high standard performance. Therefore, instead of training a robust model
with $\theta$ from scratch, we introduce a transfer learning strategy to solve
(IV-A) and propose to search a new $\theta$ from $\theta_{std}$, as the
standard performance is unlikely to change severely in the small vicinity of
$\theta_{std}$. Fig. 1 visualize the conceptual idea of the propose
adversarial training strategy on a $r$-separated dataset. Classifier 1 is
vanilla-trained over clean data. It is accurate but nor adversarial
robustness. By pushing the classification boundary out of the yellow
adversarial regions defined by $\delta\in S$, the obtained classifier 2 has
the identical standard performance with improved adversarial robustness.

We depict the detailed systematic diagram of the proposed adversarial training
strategy in Fig. 2. Specifically, the proposed training strategy divides the
training into two phases: vanilla standard training and adversarial robust
training. In standard training, we exploits clean data to train an accurate
model, $\theta_{std}=\underset{\theta}{\mathrm{argmin}}\mathbb{E}_{(x,y)\sim
D}[L(x,y;\theta)]$. The standard training has two benefits. First, it provides
the margin value $\mathbb{E}_{(x,y)\sim D}[L(x,y;\theta_{std})]$ in (IV-A).
Second, due to inherent transfer learning property, the downstream adversarial
robust training is more cost-efficient than SOTA adversarial training
strategies that often require large training loads and long training time to
handle the complexity introduced by adversarial features.

With the clean-data pre-trained model, adversarial samples are incorporated in
the model refinement phase. To prevent model catastrophic forgetting in model
fine-tuning, we follow the replay-based strategy and let the network
iteratively update upon clean and adversarial samples. Note, unlike
conventional fine-tuning tends to unfreeze several outer layers to preserve
the knowledge learned from the source task, we argue that it is important to
update all parameters in robust training stage. Briefly, adversarial noises
propagate through each layer in the model and aggregate into large prediction
distortions. All layers of a robust model must contribute to the defend
against adversarial attacks.

###  IV-B Dynamically Regulated Adversary in Adversarial Training

Adversarial training is usually referred to as the ”MinMax” optimization game,
where the adversarial samples are significant contributors. In fact,
aggressive adversarial attacks are preferred in adversarial training, because
models trained on aggressive adversarial attacks are more resistant to weaker
adversaries. Therefore previous studies have usually adopted the PGD attack in
adversarial training.

A good adversarial attack approach for adversarial training should be
aggressive but without greatly increasing training complexity. Although PGD is
an aggressive solution for introducing noise, we argue that PDG is not the
best candidate for adversarial training in the sense that it introduces
excessive noisy information in model robustness training. More specifically,
PGD uses a Sign() clipping method to project adversarial noises onto the
$L_{\infty}$ ball. It treats all image pixels equally and applies the same
noise injection strategy to all pixels, regardless of their contribution to
the prediction results. We argue that the treatment of adding adversarial
noise to all pixels in PGD significantly increases the training complexity. We
will show in the experimentation that the proposed DRA attack is a better
candidate than PGD in adversarial training and facilitates various adversarial
training.

This work introduces a novel gradient-based attack method, namely DRA attack,
in Algorithm 1. Unlike PGD, which treats image pixels equally, DRA
distinguishes important pixels from others by aggressively modifying only
those image features that are highly predictive, but non-robust. In this way,
DRA adversaries enforce the robustness training by focusing more on these
predictive features, thus helping to improve model’s robustness against
adversarial attacks. Specifically, DRA quantifies pixel significance by the
gradient of the loss function with respect to its pixel value. A large
gradient value suggests that the pixel contributes more to the image
prediction. In this regard, DRA abandons the $Sign(\cdot)$ method so that it
can smoothly search for the optimal adversarial samples along the gradient. In
addition, when generating adversarial samples, DRA uses a more resilient
distance metric $L_{1}$ to bound the adversarial noise instead of using the
$L_{\infty}$ constraint.

Fig. 3 presents a visual comparison of the DRA and PGD adversaries. Compared
to PGD, DRA introduces stronger noise in the ”cat” region, which is the most
predictive pattern in the image. We claim that the lower noise level in the
image background makes the learning more complex. We show in the experimental
section that DRA samples help various training strategies achieve higher
adversarial robustness than PDG for the same adversarial aggressiveness.

Input : Clean-data pre-trained model $f$, loss function $L$, clean training
pairs $(x,y)$, regulation term $\epsilon$, significant pixel percentage $p$,
iteration $T$

Output : Adversarial sample $x^{*}$

1: $\alpha=\epsilon/T$;

2: $x_{0}^{*}=x$, $t=0$;

3: if  _$t <T$_ then

Input $x^{*}_{t}$ to $f$ and obtain gradient $\nabla_{x}L(x_{t}^{*},y)$;

update $x_{t+1}^{*}$ =
$x_{t}^{*}+\frac{\alpha\cdot\nabla_{x}L(x_{t}^{*},y)}{\left\lVert\nabla_{x}L(x_{t}^{*},y)\right\rVert_{1}}$

end if

4: Rank current pixel values based on $\nabla_{x}L(x_{t}^{*},y)$ and set
threshold $T_{h}$ for the $p$ most significant values;

5: if  _$x_{ij}^{*} <T_{h}$_ then

$x_{ij}^{*}=x_{ij}$;

end if

Algorithm 1 DRA attack

Figure 3: A visual comparison of PGD (left) and DRA (right) adversaries. They
are both able to fool DNNs with imperceptible noises, however, the overall
noise budget of DRA is smaller. Furthermore, DRA focuses on highly
discriminate pixels (patterns that contribute most to final prediction
outcomes), where PGD equally distributes adversarial noises over the whole
image. As we can see from the example, DRA noises align well with the salient
map of the cat.

##  V Experiments

In this section, we present extensive experiments to evaluate the accuracy-
robustness performance of the proposed adversarial training strategy on MNIST
and CIFAR-10 datasets.

###  V-A Experimental Setup

In the proposed adversarial training strategy, we first train a model on clean
data only so that the model has high standard performance. The model is then
fine-tuned with clean and DRA samples. In both training processes, we utilize
image augmentation including random cropping and random flipping. In addition,
the image batch size is set to 128, SGD with momentum to 0.9 and weight decay
to $2e-4$ for model optimization.

On the MNIST dataset, we follow the TRADES study and use a simple CNN model
with 2 convolutional layers and 2 fully connected layers. We set
$\epsilon=0.1$, $p=2/3$ and iterate 20 times to generate DRA samples for 50
epochs of robust fine-tuning.

On the CIFAR-10 dataset, we use ResNet as the backbone model and perform 60
epochs of adversarial fine-tune in the robust training phase. Adversarial
samples used for our robust training are generated in 5 iterations. The noise
constraint is linearly decayed from 2 to 0.5 in the first 50 epochs, and only
clean images are fine-tuned in the last 10 epochs.

For evaluation purposes, adversarial samples are generated by PGD under
various noise constraints (e.g. $\epsilon=0.1,0.3$ out of 1 for MNIST images
and $\epsilon=2,5,8$ out of 255 for CIFAR-10) in 20 iterations with step size
$\epsilon/20$, unless otherwise specified. We compare our DRA fine-tuned model
with TRADES [11] on both datasets using relatively small regularization
parameter ($1/\lambda$). We find that this setting is consistent with our
objective, as the small regularization parameter intuitively enforces TRADES
to primarily optimize the clean data accuracy while ”trading off” a small
amount of adversarial robustness.

###  V-B Accuracy-Robustness Performance

The numerical results on the MNIST are shown in Table II, where ”natural”
indicates the standard performance of the model on clean data and ”vanilla
trained” indicates that the model is trained with clean data only. On the
MNIST dataset, both TRADES and our method maintain high standard accuracy
while improving model’s adversarial robustness; the proposed method obtains
2%-5% higher robustness than TRADES with various settings.

TABLE II: Accuracy-Robustness performance against PGD attacks on MNIST. Note, the target is to improve adversarial robustness without sacrificing standard performance. Model | natural | PGDϵ=0.1 | PGDϵ=0.3
---|---|---|---
Vanilla trained | 99.3% | 88.3% | 18.6%
Trades($1/\lambda=1$) | 99.3% | 98.4% | 93.5%
Trades($1/\lambda=0.5$) | 99.3% | 97.9% | 92.1%
Trades($1/\lambda=0.1$) | 99.4% | 97.2% | 90.8%
Ours | 99.3% | 98.0% | 95.9%
TABLE III: Accuracy-Robustness performance against PGD attacks on CIFAR-10, with ResNet-50 as the backbone model. Model | natural | PGDϵ=2 | PGDϵ=5 | PGDϵ=8
---|---|---|---|---
Vanilla trained | 93.7% | 45.0% | 15.9% | 5.6%
Trades($1/\lambda=0.05$) | 91.4% | 53.5% | 25.8% | 8.6%
Trades($1/\lambda=0.01$) | 92.6% | 49.4% | 19.2% | 6.3%
Ours | 93.8% | 64.9% | 31.0% | 10.9%
TABLE IV: Accuracy-Robustness performance against PGD attacks on CIFAR-10, with various backbone models. Models | natural | PGDϵ=2 | PGDϵ=5 | PGDϵ=8
---|---|---|---|---
ResNet-18 (vanilla) | 93.1% | 44.1% | 14.6% | 5.8%
| (-0.7%) | (+16.3%) | (+10.9%) | (+2.7%)
ResNet-18 (ours) | 92.4% | 60.4% | 25.5% | 8.5%
ResNet-34 (vanilla) | 93.3% | 46.6% | 15.3% | 4.5%
| (-0.4%) | (+14.8%) | (+12.6%) | (+4.4%)
ResNet-34 (ours) | 92.9% | 61.4% | 27.9% | 8.9%
ResNet-50 (vanilla) | 93.7% | 45.0% | 15.9% | 5.6%
| (+0.1%) | (+19.9%) | (+15.1%) | (+5.3%)
ResNet-50 (ours) | 93.8% | 64.9% | 31.0% | +10.9%
ResNet-101 (vanilla) | 93.7% | 46.9% | 15.9% | 6.8%
| (+0.1%) | (+22.9%) | (+16.3%) | (+7.4%)
ResNet-101 (ours) | 93.8% | 69.8% | 32.3% | +14.2%

For the more complex dataset, CIFAR-10, Table III shows the results with
ResNet-50 as the backbone. Unlike the MNIST dataset, ResNet-50 with the
proposed adversarial training strategy significantly outperforms TRADES in
terms of adversarial robustness. Note that in our experiment, the maximal
$\epsilon$ of DRA in adversarial training is 2, which corresponds to
$\epsilon=4-5$ in PDG attacks. We believe that training the model with a large
DRA $\epsilon$ would further improve the robustness.

In addition, we vary the backbone models for the CIFAR-10 experiments and
report the accuracy-robustness performance in Table IV. First, for clean
CIFAR-10 image classification, vanilla-trained models and our adversarial
fine-tuned models achieve comparable performance in all examined backbone
models. In particular, ResNet-50 and ResNet-101 can even outperform their
vanilla training counterparts in terms of clean data accuracy. Second, to
defend against PGD attacks, we observe that models with larger capacity are
able to boost their adversarial robustness to a larger margin, which supports
our hypothesis discussed in section 3.

###  V-C Effect of DRA on Adversarial Training

Generation of strong adversarial samples (with higher attack success rates) is
a critical factor in adversarial training. In this experiment, we investigate
the impact of different types of adversarial samples in three different
adversarial training strategies: Madry’s [7], TRADES [11], and ours.
Concretely, we replace PGD samples with DRA data in Madry’s, and TRADES.
Similarly, we use PDG in the proposed method and compare its performance with
DRA. We note that DRA is a stronger adversary than PGD for the same $\epsilon$
conditions due to the presence of soft-bounded constraint. To make a fair
comparison, we choose different $\epsilon$ for them so that DRA-generated
samples and PGD-generated samples can have similar attack strengths.
Specifically, we set $\epsilon=1$ for DRA and $\epsilon=2.55$ for PGD, as both
settings resulted in a robust accuracy of 44.5% on vanilla-trained ResNet-50.

Table. V reports CIFAR-10 classification performance with ResNet-50 as the
backbone model. We observe that in all three settings, DRA is more beneficial
for improving model robustness. Furthermore, the clean data accuracy of DRA
trained models is higher than that of the PGD trained models using Madry’s and
our method. Since the loss function of TRADES (Eqn.II) is essentially designed
to improve robustness through trading accuracy, we believe it is reasonable to
assume that the standard accuracy of the DRA trained model is slightly lower
than that of the model trained with PGD. In short, Table. V with numerical
results validate our hypothesis that DRA is a better adversary that benefits
model’s adversarial training.

TABLE V: ResNet-50 trained with PGD Vs. DRA on CIFAR-10 Model | Clean data | PGDϵ=2 | PGDϵ=5 | PGDϵ=8
---|---|---|---|---
Madry’s + PGD | 88.8% | 66.1% | 30.9% | 11.0%
| (+0.6%) | (+0.6%) | (+0.7%) | (+1.5%)
Madry’s + DRA | 89.4% | 66.7% | 31.6% | 12.5%
TRADES + PGD | 87.9% | 65.0% | 34.3% | 12.1%
| (-0.7%) | (+3.3%) | (+4.8%) | (+1.8%)
TRADES + DRA | 87.2% | 68.3% | 39.1% | 13.9%
ours + PGD | 90.6% | 59.2% | 27.0% | 10.1%
| (+0.7%) | (+0.8%) | (+1.9%) | (-0.3%)
ours + DRA | 91.3% | 60.0% | 28.9% | 9.8%

###  V-D Ablation on DRA Hyperparameter Setting

Our DRA attack algorithm filters out unimportant pixels by a pre-fixed
percentage and applies adversarial noises only to important image features
that contribute to the final prediction. In this ablation study, we
investigate the effect of hyperparameter settings (e.g. the value of
significant feature percentage $p$ and noise budget $\epsilon$ in Algorithm 1)
on CIFAR-10 dataset. Specifically, CIFAR-10 images are in size of $3\times
32\times 32$. We treat the values of red, green and blue independently and
thus obtain $N=3072$ pixel values per image. To comprehensively study this
problem, we vary the values of $p$ and $\epsilon$ and report DRA’s attack
success rates on CIFAR-10 vanilla-trained ResNet-18, ResNet-34 and ResNet-50.

Fig. 4 reports adversarial attack success rate versus the percentage $p$ out
of the 3072 pixels. With the settings $\epsilon=1$, we note that the
successful attack rate of DRA grows linearly in the most significant 500 (i.e.
$p=1/6$) pixels and saturates in about 1000 ($p=1/3$) pixels. For
$\epsilon=2$, the successful attack rate also almost saturates at
approximately 1000 ($p=1/3$) pixels.

To further investigate the impact of DRA thresholds on the overall adversarial
training method, we train ResNet-50 with different values of $p$ and report
the performance in Table. VI. We notice that a higher threshold value $p$ does
contribute to better model robustness, however, the improvement becomes
marginal as $p>1/3$. In particular, increasing $p$ from 1/2 to 1 only leads to
less than 0.5% robustness increment for PGD attacks, but causes the downgrade
of natural accuracy by 0.8%. These results also support our claim that
introducing too much unnecessary noise in the images complicates model
optimization and tends to lead to a standard performance loss. Similar to the
results in Fig. 4, Table VI indicates that $p=1/3-1/2$ is a good setting for
generating adversarial samples in the proposed method.

(a) $\epsilon$ = 1

(b) $\epsilon$ = 2

Figure 4: Ablation on DRA hyperparameter settings: DRA’s attack success rates versus the significant feature percentage $p$ on CIFAR-10 vanilla-trained ResNet models. A marginal improvement on attack success rate is observed when $p>1/3$. TABLE VI: Ablation on DRA hyperparameter settings: adversarial training performance versus different threshold value $p$. $p$ | natural | PGDϵ=2 | PGDϵ=5 | PGDϵ=8
---|---|---|---|---
0 | 93.7% | 45.0% | 15.9% | 5.6%
1/6 | 91.8% | 58.5% | 28.0% | 9.2%
1/3 | 91.3% | 60.0% | 28.9% | 9.8%
1/2 | 90.4% | 61.4% | 29.1% | 10.0%
1 | 89.6% | 61.9% | 29.3% | 10.1%

###  V-E Evaluation on Naturally Corrupted Images

Figure 5: Corrupted image samples in Cifar-10-C [31]. TABLE VII: The accuracy of DRA trained ResNet-50 vs. Vanilla trained ResNet-50 over different corrupted types. Corrupted Type | ResNet-50 | ResNet-50
---|---|---
| (DRA Trained) | (Vanilla trained)
Snow | 85.37% | 84.10%
Frost | 84.4% | 81.41%
Zoom$\\_$blur | 84.97% | 82.13%
Motion$\\_$blur | 76.89% | 74.08%
JPEG compression | 89.51% | 84.51%
Gaussian noise | 75.85% | 50.92%

Corrupted data with naturally occurring perturbations and distributed shifts
pose challenges to model generalization. In this experiment, we explore the
potential of DRA fine-tuned models on corrupted data. In this experiment, we
train a ResNet-50 model with our adversarial fine-tuning strategy and evaluate
its performance on the corrupted CIFAR-10 dataset (CIFAR-10-C [31]). In
CIFAR-10-C, the clean CIFAR-10 data are processed to mimic various image
distortions under harsh conditions. Table VII presents classification accuracy
on the CIFAR-10-C dataset with the DRA refined model and the vanilla trained
model. Results show that the DRA fine-tuned model exhibits stronger robustness
to different types of corruption.

##  VI Conclusion

In this paper, we aim to tackle a unique problem in adversarial training:
improving the adversarial robustness of a model without sacrificing the
standard performance. We explicitly formulate the problem and propose a cost-
efficient adversarial training strategy. It decomposes adversarial training
into two phases: standard training and robust training. In addition, we
introduce a training-friendly adversary to further benefit adversarial
training. Extensive experimentation on MNIST and CIFAR-10 datasets suggests
that the proposed adversarial training strategy serves better for the target
objective.

## References

  * [1] Anh Nguyen, Jason Yosinski and Jeff Clune  “Deep neural networks are easily fooled: High confidence predictions for unrecognizable images”  In _Conference on Computer Vision and Pattern Recognition_ , 2015, pp. 427–436
  * [2] Christian Szegedy et al.  “Intriguing properties of neural networks”  In _International Conference on Learning Representations_ , 2014
  * [3] Ian J Goodfellow, Jonathon Shlens and Christian Szegedy  “Explaining and harnessing adversarial examples”  In _International Conference on Learning Representations_ , 2015
  * [4] Alexey Kurakin, Ian Goodfellow and Samy Bengio  “Adversarial machine learning at scale”  In _International Conference on Learning Representations_ , 2017
  * [5] Florian Tramèr et al.  “Ensemble adversarial training: Attacks and defenses”  In _International Conference on Learning Representations_ , 2018
  * [6] Yinpeng Dong et al.  “Boosting adversarial attacks with momentum”  In _Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 9185–9193
  * [7] Aleksander Madry et al.  “Towards deep learning models resistant to adversarial attacks”  In _arXiv preprint arXiv:1706.06083_ , 2017
  * [8] Yair Carmon et al.  “Unlabeled data improves adversarial robustness”  In _Conference on Neural Information Processing Systems_ , 2019
  * [9] Minhao Cheng et al.  “Cat: Customized adversarial training for improved robustness”  In _arXiv preprint arXiv:2002.06789_ , 2020
  * [10] Yogesh Balaji, Tom Goldstein and Judy Hoffman  “Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets”  In _arXiv preprint arXiv:1910.08051_ , 2019
  * [11] Hongyang Zhang et al.  “Theoretically principled trade-off between robustness and accuracy”  In _International Conference on Interactive Mobile Communication, Technologies and Learning_ , 2019, pp. 7472–7482  Proceedings of Machine Learning Research
  * [12] Dimitris Tsipras et al.  “Robustness may be at odds with accuracy”  In _arXiv preprint arXiv:1805.12152_ , 2018
  * [13] Aditi Raghunathan et al.  “Adversarial training can hurt generalization”  In _arXiv preprint arXiv:1906.06032_ , 2019
  * [14] Aditi Raghunathan et al.  “Understanding and Mitigating the Tradeoff Between Robustness and Accuracy”  In _Proceedings of Machine Learning Research_ , 2020, pp. 7909–7919  Proceedings of Machine Learning Research
  * [15] David Stutz, Matthias Hein and Bernt Schiele  “Disentangling Adversarial Robustness and Generalization”  In _Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 6976–6987  Conference on Computer VisionPattern Recognition
  * [16] Yao-Yuan Yang et al.  “A closer look at accuracy vs. robustness”  In _Conference on Neural Information Processing Systems_ 33, 2020
  * [17] Cihang Xie et al.  “Adversarial examples improve image recognition”  In _Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 819–828
  * [18] Yanpei Liu, Xinyun Chen, Chang Liu and Dawn Song  “Delving into transferable adversarial examples and black-box attacks”  In _International Conference on Learning Representations_ , 2017
  * [19] Sizhe Chen et al.  “Universal Adversarial Attack on Attention and the Resulting Dataset DAmageNet”  In _IEEE Trans. Pattern Analysis and Machine Intelligence_ , 2020
  * [20] Ruitong Huang, Bing Xu, Dale Schuurmans and Csaba Szepesvári  “Learning with a strong adversary”  In _arXiv preprint arXiv:1511.03034_ , 2015
  * [21] Alexey Kurakin, Ian Goodfellow and Samy Bengio  “Adversarial examples in the physical world”  In _International Conference on Learning Representations_ , 2017
  * [22] Ahmadreza Jeddi, Mohammad Javad Shafiee and Alexander Wong  “A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning”  In _arXiv preprint arXiv:2012.13628_ , 2020
  * [23] Ludwig Schmidt et al.  “Adversarially robust generalization requires more data”  In _Advances in neural information processing systems_ 31, 2018
  * [24] Dan Hendrycks, Kimin Lee and Mantas Mazeika  “Using pre-training can improve model robustness and uncertainty”  In _International Conference on Machine Learning_ , 2019, pp. 2712–2721  PMLR
  * [25] Tianlong Chen et al.  “Adversarial robustness: From self-supervised pre-training to fine-tuning”  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 699–708
  * [26] Trieu H Trinh, Minh-Thang Luong and Quoc V Le  “Selfie: Self-supervised pretraining for image embedding”  In _arXiv preprint arXiv:1906.02940_ , 2019
  * [27] Mehdi Noroozi and Paolo Favaro  “Unsupervised learning of visual representations by solving jigsaw puzzles”  In _European conference on computer vision_ , 2016, pp. 69–84  Springer
  * [28] Yisen Wang1 et al.  “IMPROVING ADVERSARIAL ROBUSTNESS REQUIRES REVISITING MISCLASSIFIED EXAMPLES”  In _International Conference on Learning RepresentationsInternational Conference on Learning Representations_ , 2020
  * [29] Alhussein Fawzi, Hamza Fawzi and Omar Fawzi  “Adversarial vulnerability for any classifier”  In _Conference on Neural Information Processing Systems_ , 2018, pp. 1186–1195  Conference on Neural Information Processing Systems
  * [30] Andrew Ilyas et al.  “Adversarial examples are not bugs, they are features”  In _arXiv preprint arXiv:1905.02175_ , 2019
  * [31] Dan Hendrycks and Thomas Dietterich  “Benchmarking neural network robustness to common corruptions and perturbations”  In _International Conference on Learning Representations_ , 2019


# Frame-like Fourier expansions for finite Borel measures on $\mathbb{R}$

Chad Berner  Iowa State Unveristy, USA<EMAIL_ADDRESS>
###### Abstract.

It is known that if a finite Borel measure $\mu$ on $[0,1)$ possesses a frame
of exponential functions for $L^{2}(\mu)$, then $\mu$ is of pure type. In this
paper, we prove the existence of a class of finite Borel measures $\mu$ on
$[0,1)$ that are not of pure type that possess frame-like Fourier expansions
for $L^{2}(\mu)$. We also show properties and classifications of certain
measures possessing this type of Fourier expansion. Additionally, we establish
a frame-like Fourier expansion for $L^{2}(\mu)$ where $\mu$ is a singular
Borel probability measure on $\mathbb{R}$. Finally, we show measures $\mu$ on
$[0,1)$ that possess these frame-like Fourier expansions for $L^{2}(\mu)$ have
all $f\in L^{2}(\mu)$ as $L^{2}(\mu)$ limits of harmonic functions with frame-
like coefficients. We also discuss when the inner products of these expansions
coincide with model spaces and subspaces of harmonic functions on the disk.

###### Key words and phrases:

Fourier expansions on the torus, frames, finite Borel measures, singular
measures, model subspaces

###### 2020 Mathematics Subject Classification:

42A16, 42A20, 42C15, 46C07

##  1\. Introduction

We are interested in describing and proving the existence of a class of finite
Borel measures $\mu$ whose $L^{2}$ spaces can have their elements expressed as
Fourier expansions whose coefficients come from inner products. Substantial
work has been provided for finding measures $\mu$ where a sequence of
exponential functions forms an orthonormal basis of $L^{2}(\mu)$, and it is
clear that Lebesgue measure on $[0,1)$ is an example of this. Additionally,
there are even examples of singular measures $\mu$ which possess a sequence of
exponential functions that form an orthonormal basis of $L^{2}(\mu)$ [10].

Furthermore, there is recent work in understanding which measures $\mu$
possess a sequence of exponential functions that form a frame for
$L^{2}(\mu)$. If a space $L^{2}(\mu)$ admits a frame consisting of exponential
functions, there is a canonical dual frame sequence in $L^{2}(\mu)$ so that
every element of $L^{2}(\mu)$ can be expressed as a Fourier expansion.
Initially, Duffin and Schaffer provided substantial work in Fourier series
from introducing frame theory in the Fourier analysis [3]. More recently
however, in the case that $\mu$ is absolutely continuous, we know exactly when
it possesses a frame of exponential functions [12]. It is also known that if
$\mu$ possesses a frame of exponential functions, then $\mu$ is of pure type,
that is singular or absolutely continuous [7]. While there are examples known
of measures $\mu$ that are singular which possess a frame of exponential
functions such as in [14], when exactly measures $\mu$ have this property is
still an open problem. For more discussion on singular measures $\mu$ that
possess a sequence of exponential functions that form a frame of $L^{2}(\mu)$,
one can also read the work of Dutkay, Han, Sun, and Weber [4].

The purpose of this paper is to prove the existence of a class of finite Borel
measures $\mu$ on $[0,1)$ that are not of pure type which possess a sequence
$\\{g_{n}\\}_{n=-\infty}^{\infty}\subseteq L^{2}(\mu)$ such that for all $f\in
L^{2}(\mu)$,

(1.1) $f=\sum_{n=-\infty}^{\infty}\langle f,g_{n}\rangle e^{2\pi inx}.$

We also want to understand characteristics of measures with this property.
Furthermore, we are specifically interested in the case where the frequencies
of the exponential functions come from $\mathbb{Z}$. The secondary purpose of
this paper is to provide another type of frame-like Fourier expansion for
$L^{2}(\mu)$ where $\mu$ is a singular Borel probability measure on
$\mathbb{R}$.

The outline of this paper is as follows: section two is some known results and
early observations of the frame-like expansions we are interested in from
equation (1.1). Section three is a few results on classifying certain measures
with the property from equation (1.1) and after this, we prove the existence
of a class of measures with this property that are not of pure type.
Additionally, in section four we prove the existence of another type of frame-
like Fourier expansion for elements of $L^{2}(\mu)$ where $\mu$ is a singular
Borel probability measure on $\mathbb{R}$. Finally, section five of this paper
is establishing some connections between Bessel sequences and subspaces of
harmonic functions on the disk. In particular, we show that if a measure has
the property from equation (1.1) where $\\{g_{n}\\}$ is Bessel, then every
$f\in L^{2}(\mu)$ is the $L^{2}(\mu)$ boundary of a harmonic function whose
coefficients come from $\\{g_{n}\\}$.

##  2\. Preliminaries

We begin by recalling some frame terminology in a Hilbert space.

######  Definition 2.1.

Let $\\{g_{n}\\}\subseteq H$ where $H$ is a Hilbert space.

  1. (1)

If there exists $A,B>0$ such that

$A||f||^{2}\leq\sum_{n}|\langle f,g_{n}\rangle|^{2}\leq B||f||^{2}$

for all $f\in H$, then $\\{g_{n}\\}$ is called a frame.

  2. (2)

If $\\{g_{n}\\}$ is a frame and $A=B=1$, then $\\{g_{n}\\}$ is called a
Parseval frame.

  3. (3)

If $\\{g_{n}\\}$ is a frame and if

$\sum c_{n}g_{n}=0\implies\\{c_{n}\\}=0,$

where $\\{c_{n}\\}\in\ell^{2}$, we say $\\{g_{n}\\}$ is a Riesz basis.

  4. (4)

If $A$ or $B$ exist, we call $\\{g_{n}\\}$ a semi-lower frame or a Bessel
sequence, respectively.

Also, for Bessel sequence $\\{g_{n}\\}$, the map $g\to\\{\langle
g,g_{n}\rangle\\}$ is the analysis operator associated with $\\{g_{n}\\}$, and
its adjoint map: $\\{c_{n}\\}\to\sum_{n}c_{n}g_{n}$ is called the synthesis
operator. Furthermore, if $\\{g_{n}\\}$ is a frame, the map
$g\to\sum_{n}\langle g,g_{n}\rangle g_{n}$ is the frame operator, and it is
positive definite and invertible.

Now we define what it means for a sequence in a Hilbert space to have a
dextrodual. This definition is similar to a definition of psudeo-duals
discussed by Li and Ogawa in [13], and it is the frame-like Fourier expansions
we are discussing in section three.

######  Definition 2.2.

We say that sequence $\\{g_{n}\\}$ is dextrodual to sequence $\\{x_{n}\\}$ in
Hilbert space $H$ if for all $x\in H$,

$x=\sum_{n}\langle x,g_{n}\rangle x_{n}$

where the convergence is in norm and may be conditional.

It is clear that if $\\{x_{n}\\}$ is a frame in Hilbert space $H$, then its
canonical dual frame sequence is dextrodual to $\\{x_{n}\\}$. However as we
will see soon, a sequence having a dextrodual sequence is strictly weaker than
the sequence being a frame.

###  2.1. Dextrodual observations

We explore some initial observations of this notion in a Hilbert space.

######  Lemma 2.3.

If a sequence $\\{g_{n}\\}$ is a Riesz basis in a Hilbert space $H$ and is
dextrodual to sequence $\\{x_{n}\\}$, then $\\{x_{n}\\}$ is the canonical dual
frame of $\\{g_{n}\\}$. In particular, $\\{x_{n}\\}$ is a Riesz basis.

###### Proof.

Notice that the analysis operator associated with $\\{g_{n}\\}$ is surjective
since the range of this operator is closed from $\\{g_{n}\\}$ being a frame,
and it has dense range from its adjoint being injective. Therefore, for any
$\\{c_{n}\\}\in\ell^{2}(\mathbb{N})$, $\sum_{n}c_{n}x_{n}$ converges so that
the synthesis operator for $\\{x_{n}\\}$ is bounded from being a point-wise
limit of bounded operators. Then by an adjoint calculation, we get for all
$f\in H$,

$f=\sum_{n}\langle f,x_{n}\rangle g_{n}.$

Let $\\{h_{n}\\}$ be the canonical dual frame of $\\{g_{n}\\}$, we have for
all $f\in H$,

$\sum_{n}\langle f,x_{n}-h_{n}\rangle g_{n}=0\implies\\{x_{n}\\}=\\{h_{n}\\}$

since $\\{g_{n}\\}$ is a Riesz basis.

Finally, note that the synthesis operator for $\\{x_{n}\\}$ is the left
inverse of the analysis operator for $\\{g_{n}\\}$. Then the synthesis
operator for $\\{x_{n}\\}$ is injective since the analysis operator for
$\\{g_{n}\\}$ is invertible, showing $\\{x_{n}\\}$ is a Riesz basis. ∎

The following Lemma is an easy observation discussed in [13].

######  Lemma 2.4.

If $\\{g_{n}\\}$ is a Bessel sequence in Hilbert space $H$ with bound $B$ that
is dextrodual to $\\{x_{n}\\}$, then $\\{x_{n}\\}$ is a semi-lower frame with
bound at least $\frac{1}{B}$.

###  2.2. Known results

Now we establish some known results that will prove useful in our discussion.

######  Theorem 2.5 (He, Xing-Gang and Lai, Chun-Kit and Lau, Ka-Sing).

If $\\{e^{2\pi i\lambda x}\\}_{\lambda\in\Lambda}$ is a frame in $L^{2}(\mu)$
for $\mu$ that is a singular Borel probability measure on $[0,1)$ that is not
discrete, then $D^{-}(\Lambda)=0$ where $D^{-}$ denotes lower Beurling
density. In particular, If $\mu$ is a singular Borel probability measure on
$[0,1)$, then $\\{e^{2\pi inx}\\}_{n=0}^{\infty}$ is not a Bessel sequence in
$L^{2}(\mu)$.

The following was proven in [9] but was inspired by the work of Kwapien and
Mycielski [11].

######  Theorem 2.6 (Herr and Weber).

If $\mu$ is a singular Borel probability measure on $[0,1)$, then there is a
Parseval frame $\\{g_{n}\\}_{n=0}^{\infty}$ that is dextrodual to $\\{e^{2\pi
inx}\\}_{n=0}^{\infty}$.

These two theorems show that having a dextrodual sequence is strictly weaker
than being a frame. We also know exactly when $\\{e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ forms a frame for $L^{2}(\mu)$ when $\mu$ is an
absolutely continuous Borel probability measure on $[0,1)$ by the following:

######  Theorem 2.7 (Lai, Chun-Kit).

If $\mu$ is an absolutely continuous Borel probability measure on $[0,1)$,
then $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ is a frame for $L^{2}(\mu)$ if
and only if the Radon-Nykodym derivative of $\mu$ is bounded above and below
on its support.

##  3\. Measures on the torus with frame-like Fourier expansions

Our goal is to completely characterize measures $\mu$ that have the property
from equation (1.1). In particular, we prove the existence of a class of
measures on the torus that are not of pure type that possess this property in
this section.

###  3.1. Properties of measures with frame-like Fourier expansions

However, we did not achieve a complete characterization of these measures.
Instead, we begin this section with presenting some properties of finite Borel
measures $\mu$ on $[0,1)$ where $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ has
a dextrodual sequence $\\{g_{n}\\}$ in $L^{2}(\mu)$. The first thing one can
notice is that if a measure has this property, then its singular and
absolutely continuous part have it as well.

######  Lemma 3.1.

If $\mu=\mu_{a}+\mu_{s}$ is a finite Borel measure on $[0,1)$ where $\mu_{a}$
is absolutely continuous with Radon-Nikodym derivative $g$ and $\mu_{s}$ is
singular such that there is sequence $\\{g_{n}\\}$ that is dextrodual to
$\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu)$, then $\\{g_{n}\\}$
is dextrodual to $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu_{s})$
and $\\{g_{n}\\}$ is dextrodual to $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$
in $L^{2}(\mu_{a})$.

Furthermore, if $\\{g_{n}\\}$ is a semi-lower frame or Bessel sequence for
$L^{2}(\mu)$, then it is also a semi-lower frame or Bessel sequence for
$L^{2}(\mu_{s})$ and $L^{2}(\mu_{a})$.

###### Proof.

Let $f\in L^{2}(\mu_{s})$ and $S$ be a Borel set of Lebesgue measure zero
where $\mu_{s}(S^{c})=0$. Define $\tilde{f}\in L^{2}(\mu)$ to be equal to $f$
in $S$ $\mu_{s}$ almost everywhere and equal to $0$ in $S^{c}$ almost
everywhere with respect to Lebesgue measure. Then

$\tilde{f}=\sum_{n}\langle\tilde{f},g_{n}\rangle_{\mu}e^{2\pi
inx}=\sum_{n}\langle f,g_{n}\rangle_{\mu_{s}}e^{2\pi inx}$

where the convergence in is $L^{2}(\mu)$, so it is also in $L^{2}(\mu_{s})$.

Showing $\\{g_{n}\\}$ has semi-lower or Bessel bounds in $L^{2}(\mu_{s})$ is
simple since if $\\{g_{n}\\}$ has semi-lower or Bessel bounds in $L^{2}(\mu)$,

$||f||_{L^{2}(\mu_{s})}^{2}=||\tilde{f}||_{L^{2}(\mu)}^{2}\approx||\\{\langle\tilde{f},g_{n}\rangle_{\mu}\\}||_{\ell^{2}}^{2}=||\\{\langle
f,g_{n}\rangle_{\mu_{s}}\\}||_{\ell^{2}}^{2}$

The absolutely continuous case is similar. ∎

Now we will see that if the absolutely continuous part of $\mu$ is equivalent
to Lebesgue measure, then there cannot be a positive singular part of $\mu$
and have $\mu$ with this dextrodual property from equation (1.1).

######  Proposition 3.2.

Let $\mu=\mu_{a}+\mu_{s}$ where $\mu_{s}$ is a non-zero finite singular Borel
measure on $[0,1)$ and $\mu_{a}$ is an absolutely continuous Borel measure on
$[0,1)$ with Radon-Nikodym derivative that is bounded above and below. For $S$
that is a Borel set of Lebesgue measure zero where $\mu_{s}(S^{c})=0$, there
is no $\\{c_{n}\\}_{n=-\infty}^{\infty}$ such that

$\sum_{n}c_{n}e^{2\pi inx}=\chi_{S}$

with convergence in $L^{2}(\mu)$.

###### Proof.

If $\chi_{S}$ could be expressed as a Fourier series in this manner, the
convergence must also be in $L^{2}([0,1))$ since convergence in
$L^{2}(\mu_{a})$ is equivalent to convergence in $L^{2}([0,1))$. Then
$\\{c_{n}\\}=0$, which is a contraction. ∎

We can now classify measures with Radon-Nikodym derivative bounded below that
have the property from equation (1.1) in the following way:

######  Theorem 3.3.

Let $\mu$ be a finite Borel measure on $[0,1)$ with Radon-Nikodym derivative
$g$ that is bounded below. Suppose that $\\{g_{n}\\}$ is a semi-lower frame
that is dextrodual to $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in
$L^{2}(\mu)$. Then $g$ is bounded, and therefore, $\mu$ is absolutely
continuous.

###### Proof.

By Lemma 3.1, $\\{g_{n}\\}$ is a semi-lower frame and dextrodual to
$\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu_{a})$ where $\mu_{a}$
is the absolutely continuous part of $\mu$. Now for all $f\in L^{2}(\mu_{a})$,

$\frac{f}{\sqrt{g}}=\sum_{n}\langle f,g_{n}\rangle\frac{1}{\sqrt{g}}e^{2\pi
inx},$

which follows since $\frac{1}{\sqrt{g}}$ is bounded. Also, it is easy to see
that $\\{\frac{e^{2\pi inx}}{\sqrt{g}}\\}_{n=-\infty}^{\infty}$ is an
orthonormal basis of $L^{2}(\mu_{a})$. Then we must have by uniqueness of
coefficients from an orthonormal basis,

$\langle f,g_{n}\rangle=\langle\frac{f}{\sqrt{g}},\frac{e^{2\pi
inx}}{\sqrt{g}}\rangle$

so that

$g_{n}=\frac{e^{2\pi inx}}{g}$

for all $n$.

Now if we assume that $g$ is unbounded, we show $\\{g_{n}\\}$ can’t be a semi-
lower frame for $L^{2}(\mu_{a})$. For each $n>0$, find Borel set $E_{n}$ of
positive Lebesgue measure where $g>n$ on $E_{n}$. We have

$||\chi_{E_{n}}||_{\mu_{a}}^{2}\geq n|E_{n}|$

while

$\sum_{k}|\langle\chi_{E_{n}},g_{k}\rangle_{\mu_{a}}|^{2}=|E_{n}|.$

There can’t be $A>0$ such that

$An\leq 1$

for all $n$, so $\\{g_{n}\\}$ can’t be a semi lower frame, and $g$ is
therefore bounded. By Proposition 3.2, $\mu$ must be absolutely continuous. ∎

We can also classify measures who have dextroduals to the exponential
functions that are Riesz bases:

######  Theorem 3.4.

Let $\mu$ be a finite Borel measure on $[0,1)$. Then $\\{e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ has a dextrodual sequence that is a Riesz basis
if and only if $\mu$ is absolutely continuous with Radon-Nikodym derivative
bounded above and below.

###### Proof.

If $\mu$ is absolutely continuous with Radon-Nikodym derivative bounded above
and below, it is easy to show $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ is a
Riesz basis for $L^{2}(\mu)$, and its canonical dual frame is a Riesz basis as
well.

If $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ has a dextrodual sequence that is
a Riesz basis, then by Lemma 2.3, $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ is
a Riesz basis in $L^{2}(\mu)$ and by Lemma 3.1, $\\{e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ is a frame for $L^{2}(\mu_{s})$ where $\mu_{s}$
is the singular part of $\mu$, which implies $\mu$ is absolutely continuous by
Theorem 2.5. Furthermore, the Radon-Nikodym derivative of $\mu$ is bounded by
Theorem 2.7.

Suppose that the Radon-Nikodym derivative $g$ of $\mu$ is zero on a set of
Lebesgue positive measures say, $S$. Then

$\chi_{S}=\sum\langle\chi_{S},e^{2\pi inx}\rangle_{L^{2}[0,1)}e^{2\pi inx}$

and this convergence is in $L^{2}(\mu)$ as well since $g$ is bounded, which
contradicts uniqueness of coefficients for a Riesz basis. Then $g>0$ on
$[0,1)$, and by Theorem 2.7, we are done. ∎

Now we close this subsection with a more general description of measures with
the property from equation (1.1).

######  Theorem 3.5.

Let $\mu$ be a finite Borel measure on $[0,1)$ with Radon-Nikodym derivative
$g$. Suppose that there is a Bessel sequence $\\{g_{n}\\}$ with bound $B$ that
is dextrodual to $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu)$,
then

$g(x)>\frac{1}{3B}$

almost everywhere on its support.

###### Proof.

By Lemma 3.1, $\\{g_{n}\\}$ is a Bessel sequence that is dextrodual to
$\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu_{a})$ where $\mu_{a}$
is the absolutely continuous part of $\mu$. Then $\\{e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ is a semi-lower frame in $L^{2}(\mu_{a})$ with
bound at least $\frac{1}{B}$ by Lemma 2.4.

Suppose that $g(x)\leq\frac{1}{3B}$ on a set of positive Lebesgue measure $S$
in its support. We have

$S=\bigcup_{n=1}^{\infty}E_{n}$

where $E_{n}=\\{x\in
S:\frac{\frac{1}{3B}}{n+1}<g(x)\leq\frac{\frac{1}{3B}}{n}\\}$. There exists a
$k\geq 1$ such that $E_{k}$ has positive Lebesgue measure. We have

$\frac{\frac{1}{3B}}{(k+1)B}|E_{k}|\leq\frac{1}{B}\int_{E_{k}}gdx\leq\sum_{n}|\langle\chi_{E_{k}},e^{2\pi
inx}\rangle_{\mu_{a}}|^{2}$
$=\int_{E_{k}}g^{2}dx\leq\frac{(\frac{1}{3B})^{2}}{k^{2}}|E_{k}|\implies\frac{1}{3B}\geq\frac{k^{2}}{(k+1)B},$

which is a contradiction. ∎

###  3.2. Classes of measures with frame-like Fourier expansions

Now we show there are finite Borel measures $\mu$ on $[0,1)$ that have Bessel
sequences $\\{g_{n}\\}_{n=-\infty}^{\infty}$ that are dextrodual to
$\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu)$, even where $\mu$ is
not of pure type.

By Theorem 2.7 and Theorem 2.6, we have the following two examples:

######  Example 3.6 (Lai, Chun-Kit).

If $\mu$ is an absolutely continuous Borel measure on $[0,1)$ with Radon-
Nikodym derivative bounded above and below on its support, then there is a
frame sequence that is dextrodual to $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$
in $L^{2}(\mu)$.

######  Example 3.7 (Herr and Weber).

If $\mu$ is a finite singular Borel measure on $[0,1)$, then there is a frame
sequence that is dextrodual to $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$ in
$L^{2}(\mu)$.

Note Example 3.7 follows from scaling a Parseval frame from Theorem 2.6 and
defining the sequence to be zero on negative indices.

Finally, we will prove the existence of a class of measures with the property
from equation (1.1) that are not of pure type. It turns out that if the
absolutely continuous part of the measure is supported strictly inside
interval $[0,1)$, there is still hope. In particular, we have an example that
can easily be expanded to a class of measures:

######  Proposition 3.8.

Let $\mu=\frac{1}{2}\delta_{\frac{1}{4}}+\mu_{a}$ where $\mu_{a}$ is a finite
absolutely continuous Borel measure on $[0,1)$ with Radon-Nikodym derivative
$\chi_{[\frac{1}{2},1)}$ and $\delta_{\frac{1}{4}}$ is the point mass singular
Borel probability measure at $\frac{1}{4}$. There is a Parseval frame
$\\{g_{n}\\}$ that is dextrodual to $\\{e^{2\pi inx}\\}_{n=-\infty}^{\infty}$
in $L^{2}(\mu)$.

###### Proof.

Let $f\in L^{2}(\mu)$ and define

$\tilde{f}=f(\frac{1}{4})\chi_{[0,\frac{1}{2})}+f\chi_{[\frac{1}{2},1)}\in
L^{2}([0,1)).$

Notice that

$||\tilde{f}||_{L^{2}([0,1))}^{2}=\frac{1}{2}|f(\frac{1}{4})|^{2}+||f||_{\mu_{a}}^{2}=||f||_{d\mu}^{2}.$

Now see that $S_{M}(\tilde{f})=\sum_{n=-M}^{n=M}\langle\tilde{f},e^{2\pi
inx}\rangle_{L^{2}([0,1))}e^{2\pi inx}$ converges to $f$ in $L^{2}(\mu)$ as
$M\to\infty$. This is because $S_{M}(\tilde{f})(\frac{1}{4})\to
f(\frac{1}{4})$ as $M\to\infty$ since $\tilde{f}$ is differentiable at
$x=\frac{1}{4}$.

Now since the map $f\to\tilde{f}$ is a linear isometry, for all $n$, there is
$g_{n}\in L^{2}(\mu)$ where

$\langle f,g_{n}\rangle_{\mu}=\langle\tilde{f},e^{2\pi
inx}\rangle_{L^{2}([0,1))},$

and $\\{g_{n}\\}$ is a Parseval frame in $L^{2}(\mu)$ since $\\{e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ is a Parseval frame in $L^{2}([0,1))$. ∎

The general statement with a similar proof follows:

######  Proposition 3.9.

Let $\mu=\sum_{k=1}^{n}a_{k}\delta_{b_{k}}+\mu_{a}$ where $a_{k}>0$,
$0<b_{k}<c$, $g$ is a Borel function bounded above and below almost
everywhere, and $\mu_{a}$ is an absolutely continuous Borel measure on $[0,1)$
with Radon-Nikodym derivative $\chi_{[c,1)}g$. There is a frame $\\{g_{n}\\}$
that is dextrodual to $\\{e^{2\pi inx}\\}$.

We can also extend this result further with the following statement:

######  Theorem 3.10.

Let $\mu$ be a finite Borel measure on $[0,1)$ with Radon-Nikodym derivative
$g$ supported in some closed interval $[c,d]$ where $c>0$, and let $g$ be
bounded above and below on its support. Suppose also there is a closed
interval $[a,b]$ disjoint from $[c,d]$ where $a>0$ such that
$\mu_{s}([a,b]^{c})=0$ where $\mu_{s}$ is the singular part of $\mu$. Then
there is $\\{g_{n}\\}$ Bessel sequence that is dextrodual to $\\{e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu).$

Furthermore, if $L^{2}(\mu_{s})$ is finite-dimensional, the constructed
$\\{g_{n}\\}$ is a frame.

###### Proof.

Let $f\in L^{2}(\mu)$. By Example 3.7, there is a frame
$\\{h_{n}\\}_{n=-\infty}^{\infty}$ such that $\sum_{n}\langle
f,h_{n}\rangle_{\mu_{s}}e^{2\pi inx}\to f$ in $L^{2}(\mu_{s})$. Now let
$I_{1}$ and $I_{2}$ be two disjoint open intervals containing $[a,b]$ and
$[c,d]$ respectively. Since $\\{h_{n}\\}$ is a Bessel sequence, there is
$\tilde{f}\in L^{2}([0,1))$ with $\langle f,h_{n}\rangle_{\mu_{s}}$ as its
$n$th Fourier coefficient. Then we have

$||\sum_{n=-M}^{M}\langle\tilde{f}\chi_{I_{1}},e^{2\pi
inx}\rangle_{L^{2}([0,1))}e^{2\pi inx}-f||_{\mu_{s}}\leq$
$||S_{M}(\tilde{f}\chi_{I_{1}})-S_{M}(\tilde{f})||_{\mu_{s}}+||S_{M}(\tilde{f})-f||_{\mu_{s}}$

so that

$||\sum_{n=-M}^{M}\langle\tilde{f}\chi_{I_{1}},e^{2\pi
inx}\rangle_{L^{2}([0,1))}e^{2\pi inx}-f||_{\mu_{s}}\to 0$

as $M\to\infty$. This follows because $||S_{M}(\tilde{f})-f||_{\mu_{s}}\to 0$
by definition of $\\{h_{n}\\}$, and
$||S_{M}(\tilde{f}\chi_{I_{1}})-S_{M}(\tilde{f})||_{\mu_{s}}\to 0$ because
$S_{M}(\tilde{f}\chi_{I_{1}^{c}})$ converges to zero on $[a,b]$ uniformly.
Also note that $S_{M}(\tilde{f}\chi_{I_{1}})$ converges to zero uniformly on
$[c,d]$ so that $S_{M}(\tilde{f}\chi_{I_{1}})$ converges to zero in
$L^{2}(\mu_{a})$ as well where $\mu_{a}$ is the absolutely continuous part of
$\mu$.

Similarly,

$\sum_{n=-M}^{M}\langle f,\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi
inx}}{g}\rangle_{\mu_{a}}e^{2\pi inx}\to f$

in $L^{2}(\mu_{a})$. Furthermore,

$\sum_{n=-M}^{M}\langle f,\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi
inx}}{g}\rangle_{\mu_{a}}e^{2\pi inx}\to 0$ uniformly on $[a,b]$ so that the
convergence is also in $L^{2}(\mu_{s}).$

Then our Bessel sequence that is dextrodual to $\\{e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ in $L^{2}(\mu)$ is

$\\{\chi_{I_{1}}A^{*}(\chi_{I_{1}}e^{2\pi
inx})+\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi inx}}{g}\\}$

where $A:L^{2}(\mu_{s})\to L^{2}([0,1))$ is the bounded operator that maps
$f\to\tilde{f}$. Note $A$ is bounded below since $\\{h_{n}\\}$ is a frame.

Now notice that by Pythagorean theorem,

$\sum_{n}|\langle f,\chi_{I_{1}}A^{*}(\chi_{I_{1}}e^{2\pi
inx})+\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi inx}}{g}\rangle_{\mu}|^{2}=$
$\sum_{n}|\langle A(f)\chi_{I_{1}},e^{2\pi inx}\rangle_{L^{2}([0,1))}+\langle
f\chi_{\\{x:g(x)>0\\}},e^{2\pi inx}\rangle_{L^{2}([0,1))}|^{2}=$
$||A(f)\chi_{I_{1}}||_{L^{2}([0,1))}^{2}+||f\chi_{\\{x:g(x)>0\\}}||_{L^{2}([0,1))}^{2}.$

Therefore, $\\{\chi_{I_{1}}A^{*}(\chi_{I_{1}}e^{2\pi
inx})+\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi inx}}{g}\\}$ is a frame if
multiplication by $\chi_{I_{1}}$ on $Im(A)$ is bounded below. Also, because
$S_{M}(A(f)\chi_{I_{1}})$ converges to $f$ in $L^{2}(\mu_{S})$, multiplication
by $\chi_{I_{1}}$ on $Im(A)$ is injective. Then
$\\{\chi_{I_{1}}A^{*}(\chi_{I_{1}}e^{2\pi
inx})+\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi inx}}{g}\\}$ is a frame if
$\chi_{I_{1}}Im(A)$ is a closed set. In particular, if $Im(A)$ is finite-
dimensional, $\\{\chi_{I_{1}}A^{*}(\chi_{I_{1}}e^{2\pi
inx})+\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi inx}}{g}\\}$ is a frame. ∎

Now conversely, the idea that the absolutely continuous part and singular part
of the measure must be separated for this dextrodual property from equation
(1.1) to hold is also illustrated with the following result:

######  Theorem 3.11.

Let $\mu=\mu_{a}+\mu_{s}$ be a finite Borel measure on $[0,1)$ such that
$\mu_{s}$ is singular and $\mu_{a}$ is absolutely continuous with Radon-
Nykodym derivative $g$ bounded above and below on its support where the
support of $g$ contains an interval $(c,d)$. Suppose there exists a Borel set
$B$ of Lebesgue measure zero and $\mu$ positive measure such that

$B\subseteq[a,b]\subseteq(c,d)\subseteq\\{x:g(x)>0\\}$

for some $a$ and $b$. Then there is no
$\\{c_{n}\\}_{n=-\infty}^{\infty}\in\ell^{2}(\mathbb{N})$ such that

$\sum_{n}c_{n}e^{2\pi inx}=\chi_{B}$

with convergence in $L^{2}(\mu)$.

###### Proof.

Suppose there is $\\{c_{n}\\}_{n=-\infty}^{\infty}\in\ell^{2}(\mathbb{N})$
such that

$\sum_{n}c_{n}e^{2\pi inx}=\chi_{B}$

with convergence in $L^{2}(\mu)$. Let

$f=\sum_{n}c_{n}e^{2\pi inx}$

where the limit is taken in $L^{2}([0,1))$.

We claim $f=0$ on $\\{x:g(x)>0\\}$. Otherwise, let $A\subseteq\\{x:g(x)>0\\}$
be a set of positive Lebesgue measure where $|f|>0$. We have
$\sum_{n=-M}^{M}c_{n}e^{2\pi inx}\to f$ point-wise almost everywhere in
$[0,1)$ by Carleson’s Theorem. Then by Egorov’s theorem, there is a set of
positive Lebesgue measure $C\subseteq A$ so that $\sum_{n=-M}^{M}c_{n}e^{2\pi
inx}\to f$ uniformly on $C$. Then

$\lim_{M\to\infty}\int_{C}|\sum_{n=-M}^{M}c_{n}e^{2\pi
inx}|^{2}dx=\int_{C}|f|^{2}dx>0.$

But this contradicts the fact that

$\lim_{M\to\infty}\int_{\\{x:g(x)>0\\}}|\sum_{n=-M}^{M}c_{n}e^{2\pi
inx}|^{2}dx=0.$

This proves the claim.

Now we have that $\sum_{n=-M}^{M}c_{n}e^{2\pi inx}$ converges to zero
uniformly on $B$ since we can assume $f=0$ on $(c,d)$. But by assumption we
have

$\int_{B}|\sum_{n=-M}^{M}c_{n}e^{2\pi
inx}-\chi_{B}|^{2}d\mu_{s}+\int_{B^{c}}|\sum_{n=-M}^{M}c_{n}e^{2\pi
inx}-\chi_{B}|^{2}d\mu_{s}\to 0$

as $M\to\infty$ so that by uniform convergence on $B$, we must have
$\mu_{s}(B)=0$, which is a contradiction. ∎

##  4\. A Fourier dextrodual example for the real line

We now prove the existence of a frame-like Fourier expansion for singular
Borel probability measures on $\mathbb{R}$ that is of different nature than
our dextrodual expansions from the last section. This expansion arises from
Kaczmarz algorithms for Hilbert spaces as well as the Rokhlin disintegration
theorem to relate measures on $\mathbb{R}$ to measures on
$[0,1)\times\mathbb{Z}$.

###  4.1. The case for the torus

To show this expansion, we first recall the techniques of the case of the
torus. In [9], Herr and Weber prove Theorem 2.6 using the Kaczmarz algorithm
for Hilbert spaces:

######  Definition 4.1.

Given a complete sequence $\\{e_{n}\\}_{n=0}^{\infty}$ in Hilbert space $H$,
define the auxiliary sequence associated with $\\{e_{n}\\}$ recursively:

$g_{0}=e_{0}$

and for $n\geq 1$

$g_{n}=e_{n}-\sum_{k=0}^{n-1}\langle e_{n},e_{k}\rangle g_{k}.$

The sequence $\\{e_{n}\\}$ is called effective if for all $f\in H$,

$\sum_{k=0}^{n}\langle f,g_{k}\rangle e_{n}\to f$

in $H$.

Furthermore, effectiveness of the sequence $\\{e_{n}\\}_{n=0}^{\infty}$ in a
Hilbert space $H$ is equivalent to $\\{g_{n}\\}_{n=0}^{\infty}$ being a
Parseval frame by a result from Haller and Szwarc [6].

The authors in [9] show the effectiveness of $\\{e^{2\pi
inx}\\}_{n=0}^{\infty}$ using the following Theorem:

######  Theorem 4.2 (Kwapień and Mycielski).

A stationary sequence of complete unit vectors in a Hilbert space is effective
if and only if its spectral measure is singular with respect to Lebesgue
measure or normalized Lebesgue measure.

Recall that given a stationary sequence of unit vectors in a Hilbert space
$H$, $\\{e_{n}\\}$, the spectral measure $\mu$ is the probability measure on
$[0,1)$ such that

$\langle e_{0},e_{k}\rangle=\int_{0}^{1}e^{-2\pi ikx}d\mu(x)$

for all $k$ from the Bochner theorem.

###  4.2. Two dimensional case

Surprisingly, these Fourier expansions on the real line are inspired from the
Fourier expansions in $[0,1)^{2}$ by Herr, Jorgenesen, and Weber in [8]. In
this two dimensional paper, the authors use the Rokhlin disintegration [16,
15] to develop a notation of slice singular, which is stronger than singular.

######  Theorem 4.3 (Rokhlin disintegration).

For a Borel probability measure $\mu$ on $A\times B$ where $A$ and $B$ are
metric spaces, there is a Borel probability measure $\mu_{b}$ on $B$ and a
family of Borel probability measures $\gamma^{b}$ on $A$, indexed by $B$ such
that

  1. (1)

For $f(a,b)\in L^{1}(\mu)$ and $\mu_{b}$ almost every $b$, $f(a,b)\in
L^{1}(\gamma^{b})$ and $\int_{A}f(a,b)d\gamma^{b}\in L^{1}(\mu_{b})$

  2. (2)

For each $f\in L^{1}(\mu)$

$\int_{A\times B}f(a,b)d\mu=\int_{B}\int_{A}f(a,b)d\gamma^{b}d\mu_{b}.$

By the Rokhlin disintegration, for a Borel probability measure $\mu$ on
$[0,1)\times S$ where $S$ is a Borel set in Euclidean space, $\mu$ has a
family of Borel probability measures on $[0,1)$, $\gamma^{s}$ indexed by $S$
and a Borel probability measure $\mu_{s}$ on $S$ called the marginal measure
such that the above properties hold. The authors in [8] define a measure $\mu$
on $[0,1)^{2}$ to be slice singular if there is a disintegration where
$\mu_{s}$ is singular and $\gamma^{s}$ is singular for $\mu_{s}$ almost every
$s$. Part of the discussion of this section however is that $\mu_{s}$ being
singular or being defined on $[0,1)$ is not always important for general
Fourier expansions.

######  Definition 4.4.

We will call Borel probability measure $\mu$ on $[0,1)\times S$ $S$-slice
singular if for $\mu_{s}$ almost every $s$, $\gamma^{s}$ is singular where
$\gamma^{s}$ and $\mu_{s}$ are defined from Theorem 4.3.

The following result they proved using a operator version of the Kaczmarz
algorithm, and it is important for our discussion that this result holds
regardless of any properties of $\mu_{s}$.

######  Theorem 4.5 (Herr, Jorgensen, and Weber).

If $\mu$ is an $S$-slice singular measure on $[0,1)\times S$, then for all
$f\in L^{2}(\mu)$,

$f(x,s)=\sum_{n=0}^{\infty}\langle
f(x,s),g_{n}^{s}(x)\rangle_{L^{2}(\gamma^{s})}e^{2\pi inx}$

where for each $s$, $g_{n}^{s}(x)$ is the auxiliary sequence associated with
$L^{2}(\gamma^{s})$ and $\\{e^{2\pi inx}\\}$, and for each $n$,

$\langle f(x,s),g_{n}^{s}(x)\rangle_{L^{2}(\gamma^{s})}\in L^{2}(\mu_{s}).$

###  4.3. Main result

Now for our discussion for singular measures on the real line, we draw on a
result from Forelli [5] that gives us a completeness result of a lattice of
frequencies of exponential functions. This result is an extension of the F.
and M. Riesz theorem for the real line.

######  Theorem 4.6 (Forelli).

Let $a,b>0$ be linearly independent over $\mathbb{Z}$. If $\mu$ is a finite
Borel measure on $\mathbb{R}$ with

$\hat{\mu}(an+bm)=\int_{\mathbb{R}}e^{-(an+bm)x}d\mu=0$

for all $n,m\in\mathbb{Z}^{+}$, then $\mu$ is absolutely continuous with
respect to Lebesgue measure.

######  Corollary 4.1.

If $\mu$ is a singular finite Borel measure on $\mathbb{R}$, then for $a,b>0$
that are linearly independent over $\mathbb{Z}$

$\overline{span}\\{e^{2\pi
i(a^{-1}nx+b^{-1}mx)}\\}:n,m\in\mathbb{N}\\}=L^{2}(\mathbb{R},\mu)$

###### Proof.

Suppose that for some $f\in L^{2}(\mu)$

$\langle f,e^{2\pi i(a^{-1}nx+b^{-1}mx)}\rangle=\int_{\mathbb{R}}e^{-2\pi
i(a^{-1}nx+b^{-1}mx)}fd\mu=0$

for all $n,m\in\mathbb{N}$, then by Theorem 4.6, $fd\mu$ is an absolutely
continuous measure so that by the Lebesgue Decomposition Theorem, $fd\mu$ is
the zero measure, which implies $f=0$ $\mu$ almost everywhere. ∎

Now in order prove our Fourier expansion result for the real line, we use the
effective sequence result of Kwapien and Mycielski.

######  Theorem 4.7.

For a Borel probability measure $\mu$ on $\mathbb{R}$ and $a>0$, $\\{e^{2\pi
ia^{-1}nx}\\}_{n=0}^{\infty}$ is effective in $\overline{span}\\{e^{2\pi
ia^{-1}nx}:n\in\mathbb{N}\\}$ if and only if the measure $v$ defined on Borel
sets of $[0,a)$ as follows:

$v(E)=\sum_{n=-\infty}^{\infty}\mu(E+an)$

is singular or normalized Lebesgue measure. In particular if $\mu$ is
singular, $v$ is singular.

###### Proof.

Note $\\{{e^{2\pi ia^{-1}kx}}\\}_{k=0}^{\infty}$ is a stationary sequence of
unit vectors. Furthermore for each $k$,

$\langle 1,e^{2\pi ia^{-1}kx}\rangle_{\mu}=\int_{0}^{a}e^{-2\pi ia^{-1}kx}dv.$

The first result follows from the Theorem 4.2.

Now if $\mu$ is singular there is Borel set $A\subseteq\mathbb{R}$ such that
$\mu(A)=0=|A^{c}|.$ See $\bigcap_{k=-\infty}^{\infty}[A\cap[ak,a(k+1))-ak]$ is
$v$ measure zero, and for each $k$,

$|[A\cap[ak,a(k+1))]-ak)|=|A\cap[ak,a(k+1))|=a-|A^{c}\cap[ak,a(k+1)))|=a.$

Then $\bigcap_{k=-\infty}^{\infty}[A\cap[ak,a(k+1))-ak]$ has Lebesgue measure
$a$, and $v$ is singular.

∎

We do have a completeness issue for the case of the real line from needing a
lattice of frequencies, but to fix this, we consider a special case where we
do have completeness of one frequency of exponential functions.

######  Corollary 4.2.

Suppose that $\mu$ is a Borel probability measure supported on $a+b\mathbb{Z}$
for some $a,b\in\mathbb{R}$. Let $c>0$ and not a rational multiple of $b$.
Then for every $f\in L^{2}(\mu)$,

$f=\sum_{n=0}^{\infty}\langle f,g_{n}\rangle e^{2\pi ic^{-1}nx}$

where $\\{g_{n}\\}$ is the auxiliary sequence associated with $\\{e^{2\pi
ic^{-1}nx}\\}$ and $L^{2}(\mu)$.

###### Proof.

Notice

$e^{2\pi ib^{-1}x}=e^{2\pi ib^{-1}a}$

$\mu$ almost everywhere. Then by Corollary 4.1,

$L^{2}(\mu)=\overline{span}\\{e^{2\pi ic^{-1}nx}:n\in\mathbb{N}\\}$

where $c$ not a rational multiple of $b$. The result follows from Theorem 4.7.
∎

With this frame-like expansion for these discrete measures, a natural question
one asks is if these expansions come from weighted Fourier frames since it is
clear that these discrete measures can’t have true Fourier frames. This next
proposition shows this is never the case.

######  Proposition 4.8.

Let $\mu$ be a finite Borel probability measure on $2\pi\mathbb{Z}$ whose
support is not bounded. For any $\\{c_{n}\\}\in\ell^{2}(\mathbb{N})$ with all
positive entries, $\\{c_{n}e^{2\pi inx}\\}_{n=0}^{\infty}$ is a complete
Bessel sequence in $L^{2}(\mu)$ and never a frame.

###### Proof.

Suppose $f\in L^{2}(\mu)$ and let the weights for the measure $\mu$ be
$\\{a_{n}\\}_{n\in\mathbb{N}}$ under the usual ordering of $\mathbb{Z}$. We
have

$\sum_{k}|\sum_{n}a_{n}f(n)c_{k}e^{-2\pi
ikn}|^{2}\leq\sum_{k}c_{k}^{2}[\sum_{n}a_{n}|f(n)|]^{2}=||\\{c_{k}\\}||_{\ell^{2}(\mathbb{N})}^{2}||f||_{L^{1}(\mu)}^{2}$
$\leq||\\{c_{k}\\}||_{\ell^{2}(\mathbb{N})}^{2}||f||_{L^{2}(\mu)}^{2}.$

This shows that $\\{c_{n}e^{2\pi inx}\\}_{n=0}^{\infty}$ is a complete Bessel
sequence in $L^{2}(\mu)$ since we already know the sequence is complete.

Now to show $\\{c_{n}e^{2\pi inx}\\}_{n=0}^{\infty}$ does not have a lower
frame bound, consider the following sequence of norm one vectors:
$\\{f_{n}\\}=\\{\frac{1}{\sqrt{a_{n}}}\chi_{\\{n\\}}\\}$. We assume $a_{n}\neq
0$ for all $n$, otherwise we take a subsequence. We have

$\sum_{k=0}^{\infty}|\langle f_{n},c_{k}e^{2\pi
ikx}\rangle|^{2}=a_{n}||\\{c_{k}\\}||_{\ell^{2}(\mathbb{N})}^{2}\to 0$

as $n\to\infty$.

∎

Now for our expansion result, we use the two dimensional result from Theorem
4.5 and expand the coefficients using Corollary 4.2 and Theorem 4.3.

######  Corollary 4.3.

Let $\mu$ be a Borel probability measure on $[0,1)\times\mathbb{Z}$ that is
$\mathbb{Z}$-slice singular. Then for all $f\in L^{2}(\mu)$,

$f(x,k)=\sum_{n=0}^{\infty}\sum_{m=0}^{\infty}\langle
f,g_{m}(k)g_{n}^{k}(x)\rangle_{\mu}e^{imk}e^{2\pi inx}$

where $g_{m}(k)$ is the auxiliary sequence associated with the marginal
measure on $\mathbb{Z}$ and $\\{e^{imk}\\}$.

This result may seem random at first, but we will now show that singular
measures on $\mathbb{R}$ are morally the same as certain $\mathbb{Z}$ slice
singular measures. What follows is the statement of existence of frame-like
expansions for singular Borel measures on the real line.

######  Theorem 4.9.

For any singular Borel probability measure $\mu$ on $\mathbb{R}$, there is a
Parseval frame $\\{c_{mn}\\}_{n,m=0}^{\infty}\subseteq L^{2}(\mu)$ such that
for any $f\in L^{2}(\mu)$,

$f(y)=\sum_{n=0}^{\infty}\sum_{m=0}^{\infty}\langle
f,c_{nm}\rangle_{\mu}e^{im\lfloor y\rfloor}e^{2\pi iny}$

where the right limit in taken in $L^{2}(\mu)$ first for each $n$, then the
left limit is taken in $L^{2}(\mu)$.

Notice that in this Theorem, $\\{c_{mn}\\}_{n,m=0}^{\infty}$ is not a exactly
dextrodual to $\\{e^{im\lfloor y\rfloor}e^{2\pi iny}\\}$, but this expansion
is Fourier frame-like as it shows $f$ has a Fourier dextrodual expansion on
each $[k,k+1)$. More precisely, there is sequence $\\{d_{kn}\\}\subseteq
L^{2}(\mu)$ where

$\sum_{k\in\mathbb{Z}}\int_{[k,k+1)}|f(y)-\sum_{n=0}^{M}\langle
f,d_{kn}\rangle_{\mu}e^{2\pi iny}|^{2}d\mu\to 0$

as $M\to\infty.$

###### Proof.

Define a $\mathbb{Z}$ slice singular Borel probability measure $\sigma$ on
$[0,1)\times\mathbb{Z}$ as follows:

$\sigma(B)=\sum_{n\in\mathbb{Z}}\mu(B_{n}+n)$

where $B_{n}$ is the $n$-slice of $B$. To see that $\sigma$ is $\mathbb{Z}$
slice singular, one can see that with the Rokhlin distintegration, the
marginal measure $\mu_{n}$ on $\mathbb{Z}$ is $\sum_{n}\mu([n,n+1))\delta_{n}$
and the slice measures on $[0,1)$ indexed by $\mathbb{Z}$ are defined by
$\gamma^{n}(B)=\frac{\mu(B+n)}{\mu([n,n+1))},$ which is defined for $\mu_{n}$
almost every $n$ and singular for $\mu_{n}$ almost every $n$.

Now let $T:L^{2}(\mu,\mathbb{R})\to L^{2}(\sigma,[0,1)\times\mathbb{Z})$ where

$T(f(y))=F(x,k)=f(x+k).$

This map is an isometry because for all $f\in L^{2}(\mu)$,

$\int_{\mathbb{Z}}\int_{[0,1)}|f(x+k)|^{2}d\gamma^{k}d\mu_{k}=\sum_{k:\mu([k,k+1))>0}\int_{[0,1)}|f(x+k)|^{2}d[\mu+k]=||f||_{L^{2}(\mu)}^{2},$

and for any finite sequence $\\{a_{nm}\\}$,

$T(\sum_{n}\sum_{m}a_{nm}e^{im\lfloor y\rfloor}e^{2\pi
iky})=\sum_{n}\sum_{m}a_{nm}e^{imk}e^{2\pi ikx}.$

Then for $f(y)\in L^{2}(\mu)$ as a result of Corollary 4.3,

$f(y)=\sum_{n=0}^{\infty}\sum_{m=0}^{\infty}\langle
f,T^{*}(g_{m}(k)g_{n}^{k}(x))\rangle_{\mu}e^{im\lfloor y\rfloor}e^{2\pi iny}.$

Now to show $\\{T^{*}(g_{m}(k)g_{n}^{k}(x))\\}$ is a Parseval frame, consider
the following for any $f\in L^{2}(\mu)$:

(4.1) $\begin{split}\sum_{n}\sum_{m}|\langle
T(f),g_{m}(k)g_{n}^{k}(x)\rangle_{\sigma}|^{2}&=\sum_{n}\sum_{m}|\langle\langle
T(f),g_{n}^{k}(x)\rangle_{\gamma^{k}},g_{m}(k)\rangle_{\mu_{k}}|^{2}\\\
&=\sum_{n}||\langle T(f),g_{n}^{k}\rangle_{\gamma^{k}}||_{\mu_{k}}^{2}\\\
&=\int_{\mathbb{Z}}\sum_{n}|\langle
T(f),g_{n}^{k}\rangle_{\gamma^{k}}|^{2}d\mu_{k}\\\
&=\int_{\mathbb{Z}}||T(f)||_{\gamma^{k}}^{2}d\mu_{k}\\\
&=||f||_{\mu}^{2}.\end{split}$

This computation follows by the Rokhlin disintegration theorem and since
$\\{g_{m}\\}$ is a Parseval frame in $L^{2}(\mu_{k})$ and $\\{g_{n}^{k}\\}$ is
a Parseval frame in $L^{2}(\gamma^{k})$ for $\mu_{k}$ almost every $k$. ∎

##  5\. Subspaces of $h^{2}(\mathbb{D})$ generated by Bessel sequences.

We continue our discussion by considering Bessel sequences and their
connection to subspaces of harmonic functions on the unit disk, including
Bessel sequences in $L^{2}(\mu)$ that are dextrodual to $\\{e^{2\pi inx}\\}$.
In particular, we close this section by showing that if $\mu$ has the property
from equation (1.1), then all $f\in L^{2}(\mu)$ are $L^{2}(\mu)$ boundaries of
harmonic functions whose coefficients come from the dextrodual sequence.

######  Definition 5.1.

Recall one can define $h^{2}(\mathbb{D})$ as the set of complex valued
harmonic functions on the disk whose power series coefficients are square
summable. Also, recall that $H^{2}(\mathbb{D})$, the set of functions in
$h^{2}(\mathbb{D})$ that are analytic, is a closed subspace of
$h^{2}(\mathbb{D})$.

Let $\\{g_{n}\\}_{n=-\infty}^{\infty}$ be a Bessel sequence in Hilbert space
$H$. Define $A_{g_{n}}:H\to h^{2}(\mathbb{D})$ where

$A_{g_{n}}(f)=\sum_{n=-\infty}^{\infty}\langle f,g_{n}\rangle r^{|n|}e^{2\pi
in\theta}$

and $H(g_{n})=Im(A_{g_{n}}).$

###  5.1. Correspondence with model spaces

We will discuss specifically connections between $H(g_{n})$ and models spaces,
and we will write $\\{g_{n}\\}_{n=0}^{\infty}$ to denote
$\\{g_{n}\\}_{n=0}^{\infty}\cup\\{0\\}_{n=-\infty}^{-1}$. We review the
normalized Cauchy transform here:

######  Definition 5.2.

The Cauchy transform of a finite Borel measure $\mu$ on $[0,1)$, we denote
$C_{\mu}$ is a map from $L^{2}(\mu)$ to analytic functions on the unit disk
given by

$C_{\mu}(f)=\int_{0}^{1}\frac{fd\mu}{1-ze^{-2\pi
ix}}=\sum_{n=0}^{\infty}\langle f,e^{2\pi inx}\rangle z^{n}$

and the normalized Cauchy transform for $\mu$, denoted $V_{\mu}$ is given by

$V_{\mu}(f)=\frac{C_{\mu}(f)}{C_{\mu}(1)}=\sum_{n=0}^{\infty}\langle
f,h_{n}\rangle z^{n}$

where $h_{n}\in L^{2}(\mu)$ are in $span\\{e^{2\pi ikx}\\}$ for each $n$.

By work of Sarason [17], when $\mu$ is singular $V_{\mu}$ is an isometry onto

$H(b)=H^{2}(\mathbb{D})\ominus b(z)H^{2}(\mathbb{D})$

where $b(z)$ is the inner function corresponding to $\mu$ via the Herglotz
theorem. Furthermore, the model spaces $H(b)$ are exactly the proper backward-
shift-invariant subspaces of $H^{2}(\mathbb{D})$ by work of Beurling [1]. Our
main result for when $H(g_{n})$ is a model space follows, which is also
discussed by Christensen in [2].

######  Proposition 5.3.

If $\\{g_{n}\\}_{n=0}^{\infty}$ is a frame and not a Riesz basis, then
$H(g_{n})=H(b)$ for some inner function $b(z)$ if and only if there exists a
bounded operator that maps $g_{n}$ to $g_{n+1}$ for all $n$.

###### Proof.

Suppose that $H(g_{n})=H(b)$ for some inner function $b(z)$. Then $H(g_{n})$
is backward-shift-invariant, meaning for all $f\in H$, there is a $U(f)\in H$
such that

$\sum_{n=0}^{\infty}\langle f,g_{n+1}\rangle(re^{2\pi
i\theta})^{n}=\sum_{n=0}^{\infty}\langle U(f),g_{n}\rangle(re^{2\pi
i\theta})^{n}.$

By linearity of the inner product and since $\\{g_{n}\\}$ is complete, $U$ is
linear. $U$ is also bounded because $\\{g_{n}\\}$ is a frame. Therefore,
$U^{*}(g_{n})=g_{n+1}$ for all $n$.

Now suppose that there exists a bounded operator $T$ such that
$T(g_{n})=g_{n+1}$ for all $n$. Then take any $f\in L^{2}(\mu)$. We have

$\sum_{n=0}^{\infty}\langle f,g_{n+1}\rangle(re^{2\pi
i\theta})^{n}=\sum_{n=0}^{\infty}\langle T^{*}(f),g_{n}\rangle(re^{2\pi
i\theta})^{n}.$

Then $H(g_{n})$ is backward shift invariant.

∎

These spaces also have a nice reproducing kernel:

######  Proposition 5.4.

Let $\\{g_{n}\\}_{n=0}^{\infty}$ be a frame in Hilbert space $H$ and $S$ be
the frame operator of $\\{g_{n}\\}$. Then the reproducing kernel of $H(g_{n})$
for each $w\in\mathbb{D}$ is

$K_{w}(z)=\sum_{k=0}^{\infty}\sum_{n=0}^{\infty}\langle
S^{-1}(g_{n}),g_{k}\rangle(\overline{w})^{n}z^{k},$

meaning for all $w\in\mathbb{D}$ and $f(z)\in H(g_{n})$

$f(w)=\langle f(z),K_{w}(z)\rangle_{H(g_{n})}$

where $z=re^{2\pi i\theta}$.

###### Proof.

Notice that $A^{*}_{g_{n}}A_{g_{n}}=S$ and $A_{g_{n}}S^{-1}A^{*}_{g_{n}}$ is
the orthogonal projection onto $H(g_{n})$. Therefore, since
$\sum_{n=0}^{\infty}(\overline{w})^{n}z^{n}$ is the reproducing kernel of
$H^{2}(\mathbb{D})$,

$A_{g_{n}}S^{-1}A^{*}_{g_{n}}\sum_{n=0}^{\infty}(\overline{w})^{n}z^{n}=\sum_{k=0}^{\infty}\sum_{n=0}^{\infty}\langle
S^{-1}(g_{n}),g_{k}\rangle(\overline{w})^{n}z^{k}$

is the reproducing kernel of $H(g_{n})$. ∎

The next result is a classification of frames using the normalized Cauchy
transform.

######  Corollary 5.1.

Suppose that $\\{g_{n}\\}_{n=0}^{\infty}$ is a frame and not a Riesz basis for
Hilbert space $H$ and there exists $B\in B(H)$ such that $B(g_{n})=g_{n+1}$
for all $n$. Then $\\{g_{n}\\}$ is equivalent to a sequence $\\{h_{n}\\}$
defined by $V_{\mu}$ where $\mu$ is singular.

###### Proof.

By Proposition 5.3, there is inner function $b(z)$ so that $H(g_{n})=H(b).$
Now let $\\{h_{n}\\}$ be the Parseval frame defined from $V_{\mu}$ where $\mu$
is the finite singular measure on $[0,1)$ associated with $b(z)$ via the
Herglotz theorem. By uniqueness of reproducing kernels, we have for every $n$
and $k$,

$\langle S^{-1}g_{n},g_{k}\rangle_{H}=\langle h_{n},h_{k}\rangle_{L^{2}(\mu)}$

since the frame operator for a Parseval frame is the identity.

It follows that since $S$ is positive definite,
$\\{S^{-\frac{1}{2}}(g_{n})\\}$ is unitarly equivalent to $\\{h_{n}\\}$. ∎

####  5.1.1. Examples

We close this subsection with a couple of examples of Parseval frames and
their connection to model spaces.

######  Example 5.5.

Let $H$ be a Hilbert space with orthonormal basis
$\\{e_{n}\\}_{n=0}^{\infty}$. We have the set

$\\{g_{n}\\}_{n=0}^{\infty}=\\{e_{0},e_{1},\frac{1}{\sqrt{2}}e_{2},\frac{1}{\sqrt{2}}e_{2},\frac{1}{\sqrt{3}}e_{3},\frac{1}{\sqrt{3}}e_{3},\frac{1}{\sqrt{3}}e_{3},\dots\\}$

is a Parseval frame for $H$ and $H(g_{n})$ is not a model space by Proposition
5.3.

######  Example 5.6.

Let $\mu$ be a singular Borel probability measure on $\mathbb{R}$ and let
$\\{g_{n}\\}$ be the auxiliary sequence associated with Hilbert space

$\overline{span}\\{e^{2\pi inx}:n\in\mathbb{N}\\}\subseteq L^{2}(\mu)$

and sequence $\\{e^{2\pi inx}\\}_{n=0}^{\infty}$, we have

$\overline{span}\\{e^{2\pi inx}:n\in\mathbb{N}\\}(g_{n})=H(b)$

where $b(z)$ is the inner function corresponding to the measure $v$ via the
Herglotz theorem where

$v(E)=\sum_{k=-\infty}^{\infty}\mu(E+k).$

This follows from the fact that sequences $\\{e^{2\pi inx}\\}_{n=0}^{\infty}$
are unitarly equivalent in $L^{2}(\mu)$ and $L^{2}(v)$. The auxiliary
sequences in each measure space are also unitarly equivalent as a result.

###  5.2. Correspondence with subspaces of $h^{2}(\mathbb{D})$

Another result from Beurling [1] is that the subspaces of $h^{2}(\mathbb{D})$
that are invariant under multiplication by $z$ and $\overline{z}$ are exactly
$f(z)h^{2}(\mathbb{D})$ where $|f|$ is zero or one almost everywhere on
$\partial\mathbb{D}$. We have a similar result from techniques from this
section:

######  Proposition 5.7.

For a Hilbert space $H$ with frame $\\{g_{n}\\}_{n=-\infty}^{\infty}$, the
following are equivalent:

  1. (1)

$H(g_{n})=f(z)h^{2}(\mathbb{D})$ where $|f|$ is zero or one almost everywhere
on $\partial\mathbb{D}$.

  2. (2)

$\\{g_{n}\\}=\\{T^{n}g_{0}\\}$ for some bounded invertible operator $T$.

  3. (3)

$\\{g_{n}\\}$ is similar to Parseval frame $\\{\chi_{E}e^{2\pi
inx}\\}_{n=-\infty}^{\infty}$ in Hilbert space $\chi_{E}L^{2}([0,1))$ for some
$E\subseteq[0,1)$ measurable.

.

####  5.2.1. Examples

Now we discuss examples of subspaces of $h^{2}(\mathbb{D})$ arising from
$H(g_{n})$ where $\\{g_{n}\\}$ is dextrodual to $\\{e^{2\pi inx}\\}$.

######  Example 5.8.

If $\mu$ is an absolutely continuous Borel measure on $[0,1)$ with Radon-
Nikodym derivative $g$ bounded above and below on its support, it is clear
that

$L^{2}(\mu)(\chi_{\\{x:g(x)>0\\}}\frac{e^{2\pi
inx}}{g})=\chi_{\\{x:g(x)>0\\}}L^{2}([0,1))=f(z)h^{2}(\mathbb{D})$

where $f(z)=\sum_{n}\langle\chi_{\\{x:g(x)>0\\}},e^{2\pi inx}\rangle
r^{|n|}e^{2\pi inx}$.

######  Example 5.9.

Let $\mu=\sum_{k=1}^{n}a_{k}\delta_{b_{k}}+\mu_{a}$ be a measure from
Proposition 3.9. It is clear that the $H(g_{n})$ from this proposition is not
invariant under multiplication by $z$ or $\overline{z}$ since the $\tilde{f}$
in Proposition 3.9 is a step function on an interval.

####  5.2.2. Boundary behavior of $A_{g_{n}}$

We conclude with our statement about our dextroduality relating to boundary
behavior of $A_{g_{n}}$.

######  Theorem 5.10.

Let $\mu$ be a finite Borel measure on $[0,1)$ such that there is Bessel
sequence $\\{g_{n}\\}_{n=-\infty}^{\infty}$ that is dextrodual to $\\{e^{2\pi
inx}\\}$. Then for all $f\in L^{2}(\mu)$,

$A_{g_{n}}(f)(re^{2\pi i\theta})\to f$

in $L^{2}(\mu)$ as $r\to 1^{-}.$

In other words, if $\mu$ has the property from equation (1.1), then for each
$f\in L^{2}(\mu)$, $A(g_{n})(f)$ is continuous in $r$ on $[0,1]$ into
$L^{2}(\mu)$.

###### Proof.

We need to show for all $f\in L^{2}(\mu)$,

$\sum_{n}\langle f,g_{n}\rangle r^{|n|}e^{2\pi in\theta}\to\sum_{n}\langle
f,g_{n}\rangle e^{2\pi in\theta}$

in $L^{2}(\mu)$ as $r\to 1^{-}.$ This follows from Abel summability in Hilbert
spaces since $\sum_{n}\langle f,g_{n}\rangle r^{|n|}e^{2\pi in\theta}$ exists
in $L^{2}(\mu)$ for all $r\in[0,1]$ by the dextroduality property and fact
that $\sum_{n}|\langle f,g_{n}\rangle|^{2}<\infty$ for all $f\in L^{2}(\mu)$.
∎

## References

  * [1] Arne Beurling  “On two problems concerning linear transformations in Hilbert space”  In _Acta Math._ 81, 1948, pp. 239–255  DOI: 10.1007/BF02395019
  * [2] Ole Christensen, Marzieh Hasannasab and Friedrich Philipp  “Frame properties of operator orbits”  In _Math. Nachr._ 293.1, 2020, pp. 52–66  DOI: 10.1002/mana.201800344
  * [3] R.. Duffin and A.. Schaeffer  “A class of nonharmonic Fourier series”  In _Trans. Amer. Math. Soc._ 72, 1952, pp. 341–366  DOI: 10.2307/1990760
  * [4] Dorin Ervin Dutkay, Deguang Han, Qiyu Sun and Eric Weber  “On the Beurling dimension of exponential frames”  In _Adv. Math._ 226.1, 2011, pp. 285–297  DOI: 10.1016/j.aim.2010.06.017
  * [5] Frank Forelli  “The F. and M. Riesz theorem”  In _Proc. Amer. Math. Soc._ 47, 1975, pp. 431  DOI: 10.2307/2039760
  * [6] Rainis Haller and Ryszard Szwarc  “Kaczmarz algorithm in Hilbert space”  In _Studia Mathematica_ 169.2, 2005, pp. 123–132  URL: http://eudml.org/doc/284674
  * [7] Xing-Gang He, Chun-Kit Lai and Ka-Sing Lau  “Exponential spectra in $L^{2}(\mu)$”  In _Appl. Comput. Harmon. Anal._ 34.3, 2013, pp. 327–338  DOI: 10.1016/j.acha.2012.05.003
  * [8] John E. Herr, Palle E.. Jorgensen and Eric S. Weber  “Fourier Series for Fractals in Two Dimensions”  In _From classical analysis to analysis on fractals_ , Appl. Numer. Harmon. Anal.  Birkhäuser/Springer, Cham, 2023, pp. 183–229  DOI: 10.1007/978-3-031-37800-3“˙9
  * [9] John E. Herr and Eric S. Weber  “Fourier Series for Singular Measures”  In _Axioms_ 6.2, 2017  DOI: 10.3390/axioms6020007
  * [10] Palle E.. Jorgensen and Steen Pedersen  “Dense analytic subspaces in fractal $L^{2}$-spaces”  In _J. Anal. Math._ 75, 1998, pp. 185–228  DOI: 10.1007/BF02788699
  * [11] Stanisław Kwapień and Jan Mycielski  “On the Kaczmarz algorithm of approximation in infinite-dimensional spaces”  In _Studia Mathematica_ 148.1, 2001, pp. 75–86  URL: http://eudml.org/doc/284583
  * [12] Chun-Kit Lai  “On Fourier frame of absolutely continuous measures”  In _J. Funct. Anal._ 261.10, 2011, pp. 2877–2889  DOI: 10.1016/j.jfa.2011.07.014
  * [13] Shidong Li and Hidemitsu Ogawa  “Pseudo-duals of frames with applications”  In _Appl. Comput. Harmon. Anal._ 11.2, 2001, pp. 289–304  DOI: 10.1006/acha.2001.0347
  * [14] Gabriel Picioroaga and Eric S. Weber  “Fourier frames for the Cantor-4 set”  In _J. Fourier Anal. Appl._ 23.2, 2017, pp. 324–343  DOI: 10.1007/s00041-016-9471-0
  * [15] V.. Rohlin  “On the decomposition of a dynamical system into transitive components”  In _Mat. Sbornik N.S._ 25(67), 1949, pp. 235–249
  * [16] V.. Rohlin  “On the fundamental ideas of measure theory”  In _Mat. Sbornik N.S._ 25/67, 1949, pp. 107–150
  * [17] Donald Sarason  “Sub-Hardy Hilbert spaces in the unit disk” A Wiley-Interscience Publication 10, University of Arkansas Lecture Notes in the Mathematical Sciences  John Wiley & Sons, Inc., New York, 1994, pp. xvi+95