# BALF: Simple and Efficient Blur Aware Local Feature Detector

Zhenjun Zhao1,2 Yu Zhai1 Ben M. Chen1 Peidong Liu3,4
1The Chinese University of Hong Kong 2Peng Cheng Laboratory
3Westlake University 4Westlake Institute for Advanced Study
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Corresponding author.

###### Abstract

Local feature detection is a key ingredient of many image processing and
computer vision applications, such as visual odometry and localization. Most
existing algorithms focus on feature detection from a sharp image. They would
thus have degraded performance once the image is blurred, which could happen
easily under low-lighting conditions. To address this issue, we propose a
simple yet both efficient and effective keypoint detection method that is able
to accurately localize the salient keypoints in a blurred image. Our method
takes advantages of a novel multi-layer perceptron (MLP) based architecture
that significantly improve the detection repeatability for a blurred image.
The network is also light-weight and able to run in real-time, which enables
its deployment for time-constrained applications. Extensive experimental
results demonstrate that our detector is able to improve the detection
repeatability with blurred images, while keeping comparable performance as
existing state-of-the-art detectors for sharp images.

##  1 Introduction

Figure 1: The performance of keypoint detectors with motion blurred images.
Our approach achieve superior detection performance in terms of the
repeatability metric and efficiency, which paves the way for robust 3D vision
under low-lighting conditions.

Being able to accurately detect and describe salient keypoints across images
is crucial in many applications such as Simultaneous Localization and Mapping
(SLAM), Structure-from-Motion (SfM), camera calibration, video compression,
tracking, image retrieval, and visual localization. Interest points should be
sparse, repeatable, and discriminable, in order to be extracted and matched
across images from different lighting conditions or viewpoints. While mangy
state-of-the-art methods have been proposed, motion blur is still a major
challenge remaining for local feature detection methods. Motion blur is one of
the most common artifacts that degrade images. It usually occurs in low-light
conditions where longer exposure times are necessary. This would affect many
feature based approaches, which struggle to detect repeatable keypoints to
build up correspondences.

To detect keypoints in blurred images, a straightforward way is to utilize a
deblurring algorithm to restore the latent sharp image and then to detect
keypoints from the restored image. Image/video deblurring methods have been
well developed over the last decades, which mainly consist of classic
gradient-descent methods [4, 69, 16, 54, 42, 55, 9, 71, 43, 22] and learning-
based methods [53, 57, 67, 40, 60, 24, 23, 70, 77, 72, 28, 20, 33]. Although
deblurring algorithms have achieved impressive performance recently, there are
still several limitations existed. For example, existing state-of-the-art
methods usually require high computational resources and are hardly to run in
real-time even with a high-end GPU. Another limitation is that current methods
still cannot perform very well and might introduce additional artifacts for
severe motion blurred image, due to the limited information preserved by a
single blurred image. We thus aim to design a novel one-stage efficient local
feature detector from a motion blurred image directly, without any
intermediate deblurring operation, to avoid those drawbacks.

In recent years, deep learning techniques have shown great success in
improving local feature extraction. Many state-of-the-art methods have been
proposed by the community [65, 51, 79, 25, 7, 61, 58]. Even though great
progress has been achieved in extracting local features, most of them focus on
improving the robustness against viewpoint changes, illumination changes etc.
Prior work that aims to extract features from motion blurred images is still
limited, which is important towards robust 3D vision at low-lighting scenarios
(e.g. augmented reality for outdoors at night). Further inspired by the recent
success of Multi-Layer-Perceptron (MLP) in many areas of computer vision [62,
32, 75, 64, 30, 8, 10, 35, 59, 63, 18, 29, 19], we propose to explore the
possibility by applying MLP-based network for local feature detection from a
motion blurred image, which has never been attempted previously.

In this paper, we introduce BALF, a simple yet both efficient and effective
motion blur aware local feature detector. Our detector network consists of two
main components: a pure MLP-based image encoder and a keypoint detection
module. The encoder takes advantage of a cascaded of multi-axis gated MLP
blocks and “Squeeze and Excitation (SE)” MLP blocks, to learn a pyramid
feature representation of the input image. The keypoint detection module then
apply a differentiable channel-wise softmax operator to detect salient
keypoints. Extensive evaluations have been conducted on both synthetic and
real datasets. Experimental results demonstrate that our method achieves
superior performance over prior methods as shown in Fig. 1. In particular, we
achieve 15% improvements over the current best performing network on motion
blurred images with the repeatability metric, which is commonly used for local
feature detection evaluations. Besides the repeatability performance, our
method is also able to run at around 35 FPS for VGA resolution image (_i.e_.
480$\times$640 pixels) on a consumer-grade Graphic Card (_i.e_. NVIDIA Geforce
2080 Ti), which is sufficient for time-constrained applications.

In summary, our contributions are as follows:

  * •

We propose a novel and efficient MLP-based network architecture for local
feature detection, which has never been attempted for this task previously.

  * •

Extensive experimental results demonstrate that our network achieves superior
detection performance over prior works on motion blurred images, while keeping
comparable performance for sharp images.

  * •

Our motion blur robust keypoint detector is able to run in real-time, which
would enable many time-constrained applications (_e.g_. robotic navigation in
low-lighting scenarios).

##  2 Related Work

We review three main areas of related work: keypoint detection, image
deblurring, and MLP related methods.

Keypoint detection. Local feature detection plays a vital role in many vision
related tasks, such as visual localization and recognition. It thus received a
continuous influx of attention in the past decades [11, 15, 50]. Existing
works can be generally categorized into classical handcrafted based methods
and modern learning based methods. Since our work belongs to learning based
approach, we pay our attention on reviewing learned feature detectors. For
more details on classical handcrafted feature detectors, interested readers
can refer to a benchmark work from Schmid _et al_. [52].

FAST [46] is one of the first attemps to apply machine learning technique for
reliable and fast corner detection. Similar strategies have also been applied
in other related extensions [47, 48, 27]. TILDE [65], one of the first deep
learning based detector, trains a piece-wise linear regressor to detect
keypoints under drastic weather and illuminstion changes, using SIFT keypoints
as supervision. DetNet [26] derives a covariant constraint to learn stable
anchors for local features. It is further extended by Zhang _et al_. [79] to
introduce two new concepts of standard patch and canonical feature for feature
detection. Savinov _et al_. [51] later proposes Quad-networks, which is
unsupervised and trains a neural network to rank points under a
transformation-invaraint manner. It then extracts keypoints from the
top/bottom quantiles of the rankings. A similar detector is proposed by Zhang
_et al_. [78], which combines the same ranking loss with a grid-wise peakness
loss to detect keypoints in texture images. Key.Net [25] resorts to using
hand-crafted filters together with learned convolutional neural network (CNN)
features by a light weight CNN network. They propose to use a spatial softmax
operator for detecting keypoints across multi-scale regions. ELF [7] proposes
to use a pre-trained CNN for image classification task to detect saliency
keypoints without requiring extra training. Recently, it has been shown that
convolutional neural network trained for descriptor can also be used for
keypoint detection, and achieves impressive performance [61]. In addition to
the above methods which are solely designed for keypoint detection, existing
works also seek the possibility to integrate both the feature detection and
description in a unified framework. For example, LIFT [74] takes a quadruplet
of patches to jointly train a detector, an orientation estimator, and a
descriptor which are supervised by the feature correspondences from a
Structure-from-Motion (SfM) pipeline. Instead of getting supervision via a SfM
pipeline, SuperPoint [12] proposes to first pre-train the detector via a
synthetic dataset, which consists of primitive geometric shapes. They then
take advantage of a homographic adaptation module to achieve self-supervised
training together with the feature descriptor network on real images. LF-Net
[41] proposes to enforce same feature response for corresponding points across
images to train the detector. The correspondence is built based on known
camera poses and depth maps.

Different from existing works, which are usually trained to detect keypoints
from a sharp image, we propose a motion blur aware keypoint detector for
robust 3D vision under low-lighting conditions.

Image deblurring. Existing motion deblurring algorithms can be mainly
categorized into classical gradient-descent based methods and learning based
methods during inference. We will only focus on several representative
learning based single image deblurring networks, which are most related to our
work. Early learning based methods [57, 17, 53, 73] mainly exploit deep
network to estimate the unknown blur kernels and then employ conventional
deconvolution methods to restore the blurred images. The performance of single
image deblurring algorithms has been further boosted by end-to-end deep neural
networks, which formulate the deblurring problem as an image translation
problem. Xu _et al_. [70] develop a deep convolutional neural netowrk to
capture the blur degradation for image deconvolution. Sun _et al_. [57] apply
the network to predict the varying motion blur kernels, which enables to image
deblurring. Nah _et al_. [40] follow a coarse-to-fine approach to train a
multi-scale CNN for blind deblurring. Kupyn _et al_. propose DeblurGAN [23]
and DeblurGAN-v2 [24] based on a adversarial loss for motion deblurring. Tao
_et al_. [60] propose a SRN-DeblurNet, which adopts a scale-recurrent
structure to realize multi-scale image deblurring. Liu _et al_. [33] recently
propose a self-supervised network for motion deblurring. Although those
methods achieve impressive performance on image deblurring, they usually
require large computational resources and are hard to run in real-time.
Instead of firstly deblurring each image and then detecting keypoints on the
restored image, we propose an efficient one-stage feature detector from motion
blurred image directly.

MLP-like architecture in computer vision. While convolutional neural networks
and Vision Transformers (ViT) [13] have been the de-facto standards for many
computer vision applications, MLP-based architectures have also achieved
state-of-the-art performance in several vision tasks recently [62, 32, 75, 64,
30, 8, 10, 35, 59, 63, 18, 29, 19]. Due to the conceptually and technically
simplicity, MLP-based architecture is getting more attention in both visual
recognition [32, 75] and dense prediction tasks [30, 8]. Recently, MAXIM [64]
adopts a multi-axis gated MLP module for low-level image processing while
SegFormer [68] unifies Transformers with MLP decoders for semantic
segmentation tasks. While MLP-based architecture has achieved impressive
performance in several vision related tasks, their applicability in local
feature detection has never been explored. We thus propose to explore this
possibility for local feature detection, and achieve state-of-the-art
performance over prior keypoint detectors with both motion blurred and natural
sharp images.

##  3 Methods

We present, to the best of our knowledge, the first MLP-based architecture (as
shown in Fig. 2) for local feature detection with a blurred image. As shown in
Fig. 2(a), our network consists of a MLP-based encoder and a MLP-based
detection module. The encoder network learns a effective feature
representation of the input image via cascaded MLP blocks. The learned feature
representation is then input to a detection module, which takes advantages of
a differentiable channel-wise softmax operator. The resulting network is
light-weight and effective. We will detail each component as follows.

(a) BALF Framework

(b) MLPCoder Block

(c) Residual MLP Attention Block (RMAB)

Figure 2: The proposed network for motion blur aware local feature detector
(BALF). Our network consists of two main modules: a multi-stage MLP-based
encoder to extract an intermediate feature representation of the input image,
and a MLP detection module to detect salient keypoints via a differentiable
softmax operator.

###  3.1 MLP-based encoder

Our MLP-based encoder contains cascaded MLPCoder blocks (as shown in Fig.
2(b)) to obtain a pyramid level representation of the input image. Each
MLPCoder block contains a channel MLP block, a multi-axis gated MLP block and
a residual MLP attention block. The channel MLP block maps each pixel to a
high-dimensional representation, such that it can be further processed by the
following blocks. After each MLPCoder block, we apply a max-pooling layer to
extract the most significant features and reduce the spatial dimension of the
feature representation. The loss of the spatial resolution caused by the
pooling layer is compensated by the increased number of channels of the
feature representation.

Multi-axis gated MLP block. For keypoint detection task, local region
relationship among the pixels is usually more important compared to long-range
relationship. However, due to motion blur, the intensity of a particular pixel
is mixed by that of several neighboring pixels. It would be beneficial to have
a larger receptive field to take account of more context information for
better localization of the salient keypoints. We thus take advantages of a
state-of-the-art MLP block, _i.e_. the multi-axis gated MLP block, from MAXIM
[64] to extract both local and global visual cues. MAXIM [64] is a MLP-based
network for low-level image processing tasks, and achieves state-of-the-art
performance on image denoising, deblurring, deraining, dehazing and
enhancement compared to prior works. Multi-axis gated MLP block presents a
principled way to apply 1D operators on 2D images in a scalable manner, and
apply them in parallel corresponds to both local and global (dilated) mixing
of spatial information respectively. For more details, interested readers are
suggested to refer to the work of Tu _et al_. [64].

Residual MLP attention block. The multi-axis gated MLP block mainly learns
spatial dependencies across the feature representations. To better capture the
channel-wise dependencies and inspired by convolutional channel attention
block in [66, 76], we exploit the squeeze-and-excitation (SE) block from SENet
[21] to build up a residual MLP attention block (RMAB). SE block is more
efficient compared to those convolutional channel attention blocks. It allows
the network to perform feature re-calibration and exploit the inter-channel
relationship of features, through which it can learn to use global information
to selectively emphasize informative features and suppress less useful ones.

The SE block first conduct a squeeze operation, which produces a channel
descriptor by aggregating feature maps across their spatial dimensions. The
function of this descriptor is to produce an embedding of the global
distribution of channel-wise feature responses, allowing information from the
global receptive field of the network to be used by all its layers. The
aggregation is followed by an excitation operation, which takes the form of a
simple self-gating mechanism that takes the embedding as input and produces a
collection of per-channel modulation weights. These weights are applied to the
input feature maps to generate the output of the SE block which can be fed
directly into subsequent layers of the network. We further improve the channel
attention concept for keypoint detection task, by applying two channel MLP
blocks to build a residual MLP attention block (as shown in Fig. 2(c))
together with the SE block [21].

Figure 3: Detection module. The learned feature representations of the input
image is processed by two channel-wise MLP blocks. The keypoints are then
detected by using channel-wise softmax operation and mapped back to the
original image domain.

###  3.2 MLP-based detection module

After the MLP-based encoder, the resulting feature map
$\mathcal{F}\in\mathbb{R}^{H^{\prime}\times W^{\prime}\times C}$, where
$H^{\prime}=\frac{H}{2^{N}}$ and $W^{\prime}=\frac{W}{2^{N}}$, are passed
through a MLP-based detection module (as shown in Fig. 3) to output a dense
probability map for each pixel as a keypoint.

The details of the detection module are illustrated in Fig. 3. It consists of
two channel MLP blocks, which transform the input feature representation from
$C$ channels to $C_{r}$ channels at the same spatial resolution, where
$C_{r}=4^{N}$. Each element along the channel direction corresponds to the
response of a particular pixel of the input full resolution image. In other
words, all the elements along the channel direction correspond to the
responses of pixels within a $2^{N}\times 2^{N}$ sized patch of the input
image. We use $N=3$ in our experiments.

Handcrafted feature detectors, such as SIFT [34], usually perform spatial non-
maxima suppression (NMS) on the response map, to select a set of sparsely
distributed salient keypoints. To achieve similar goal, we apply a
differentiable channel-wise softmax operation on the transformed feature
representation as shown in Fig. 3. The corresponding channel index of the
maximum responsed element is then mapped to the pixel index of the full
resolution input image, such that the corresponding pixel can be assigned with
the response score. We further eliminate pixels as keypoints for those with
low responses by a pre-defined threshold, such as pixels within a homogeneous
region. The purpose of using this feature detection procedure is two-fold.
First, the channel-wise softmax layer is similar to NMS but is differentiable
and enables end-to-end training. Secondly, it replaces the decoder module with
simple non-parametric operator. It neither involves any feature learning nor
introduces any additional parameters, which further reduces the computational
overhead.

###  3.3 Loss function

We formulate the keypoint detection as a regression problem. We train the
network with a ground truth response map $\mathcal{R}_{GT}$, which are
generated by detecting SIFT keypoints [34] on the corresponding sharp images
and placing Gaussian kernels at those locations. The loss function used to
train the network can be formalized described as:

$\mathcal{L}=||\mathcal{R}-\mathcal{R}_{GT}||^{2},$ (1)

where $\mathcal{R}$ is the predicted response map (as shown in Fig. 2(a)) and
$\mathcal{R}_{GT}$ is the ground truth response map.

##  4 Experiments

The main motivation of our work is to develop a motion blur aware local
feature detector via deep networks, since many state-of-the-art methods for a
sharp image have been proposed and achieved impressive performance. There is
no existing dataset to evaluate local feature detectors on motion blurred
images. We thus create a training dataset via a publicly available single
image deblurring dataset, _i.e_. the GoPro dataset from Nah _et al_. [40],
which has paired sharp and blurred images. To evaluate our method, we generate
a synthetic dataset via HPatches dataset [5], which is commonly used for local
feature evaluations. HPatches dataset is different from the GoPro dataset, and
it can thus also reflect the generalization performance of our method. We
therefore mainly use this dataset for evaluation.

Datasets. We use the GoPro dataset from Nah _et al_. [40] to generate data to
train our network. The GoPro dataset is commonly used for evaluations of
single image deblurring networks. It consists of 3,214 paired sharp and
blurred images, which are captured from 33 scenes. We follow their convention
and use 22 sequences for training and 11 sequences for testing. To generate
supervision data for each blurred image, we assume the blurred image should
have the same keypoint locations as those for its paired sharp image. We use
SIFT [34] to first detect keypoints from the sharp image, and then generate a
heatmap and place Gaussian kernels as the ground truth response map for
network training.

|  |  |
---|---|---|---
Sharp | EASY | HARD | TOUGH
Figure 4: Example synthesized blurred images from HPatches dataset [5]. The
first column shows the sharp image, and the next three columns are blurred
images with varying blur levels. Best viewed in high resolution.

To better evaluate the generalization performance of our method, we generate a
synthetic motion blurred image dataset by using HPatches dataset [5]. HPatches
dataset is commonly used for local feature evaluations, which originally
covers both illumination and viewpoint changes. It gathers images from
existing datasets, such as DTU [1] and Oxford [39] datasets. It provides a
total of 116 sequences and is further divided into 59 sequences for viewpoint
changes and 57 sequences for illumination changes. Each sequence includes a
reference image and 5 target images with varying viewpoint or illumination
changes, together with the corresponding homographies between them. We
generate a random motion blur kernel, _i.e_. point spread function, for each
image from the HPatches dataset to synthesize a motion blurred image. We
generate three varying levels of motion blur, by changing the blur kernel size
and motion irregularities (_i.e_. the non-linearity of the motion). Fig. 4
presents several example images for different blur levels. We use this dataset
purely for evaluation.

Implementation details. Our network is light-weight and end-to-end trainable.
It requires neither large-scale pre-training nor progressive training. During
training, we use a batch size of 4, and Adam optimizer with a initial learning
rate of $10^{-4}$. After the network is trained for 20 epochs, we linearly
decay the learning rate to $10^{-6}$ till the $50^{th}$ epoch. The network
takes about 3 hours to be trained on a single NVIDIA Geforce 2080Ti graphic
card. During training, we also perform data augmentation as that of [25]. In
particular, we perform a sequence of random rotation, random scaling, random
skewing and random perspective transformation on the original image. We also
apply random photometric transformation, such that the trained network is
robust against illumination changes. Then we randomly crop a $256\times 256$
pixels image for training. Detailed network architecture can be found from our
supplementary material.

Baseline methods and evaluation metrics. We compare our approach against a
number of representative detectors, which range from classical handcrafted
methods to recent learning based methods. In particular, we compare against
SIFT [34], SURF [6], Harris Laplace [38], Shi-Tomasi [56], MSER [36], KAZE
[2], AKAZE [3] and FAST [47] with their OpenCV implementations. We also
compare with learned feature detectors, such as LIFT [74], Key.Net [25],
SuperPoint [12], LF-Net [41], D2-Net [14], and R2D2 [44]. For fair comparisons
with motion blurred images, we also re-train those learning based methods on
the GoPro dataset using their publicly available source codes. For evaluations
with sharp images, we still use their official provided pretrained models.

The repeatability metric proposed in [37] measures the quality of keypoint
detection and is commonly used by the community. For a pair of images, it is
computed as the ratio between the number of corresponding keypoints observed
by both images and the smaller number of keypoints detected in one of the two
images. To identify the corresponding keypoints, we compute the overlap error,
$\epsilon_{IoU}$, between the regions of two candidate keypoints as in [79,
78]. We also fix the maximum number of extracted keypoints and allow each
keypoint to be matched only once as in [74] for fair evaluations. In our
experiments, we consider the top 1000 interest points for repeatability
computation. It is considered as a correct match if $\epsilon_{IoU}$ is
smaller than 0.4, _i.e_., the overlap between corresponding region is more
than $60\%$.

Ablation study In this section, we study the effect of the residual MLP
attention block (RMAB). We conduct experiments with three settings, _i.e_. the
network with and without RMAB, and we also replace RMAB with a residual
convolutional attention block (RCAB) from [66]. The experimental results shown
in Tab. 1 demonstrate that RMAB block is indeed effective for local feature
detection. The RCAB block requires more parameters, longer inference time and
achieves lower repeatability compared to the network with RMAB. It further
demonstrates the potential to use MLP blocks for efficient network
architectures. The experiments are conducted with the GoPro testing sequences.

Variant | Repeatability $\uparrow$ | Params $\downarrow$ | Inference time $\downarrow$
---|---|---|---
w/o RMAB | 63.59% | 326K | 19.18ms
RCAB | 73.56% | 725K | 57.29ms
RMAB (proposed) | 75.15% | 381K | 29.02ms

Table 1: Ablation study of the residual MLP attention block (RMAB). The
experimental results demonstrate that the RMAB block is indeed efficient and
effective for local feature detection.

|  | Reference: Sharp
---
Target: Sharp
Method | Viewpoint $\uparrow$ | Illumination $\uparrow$ | Total $\uparrow$
SIFT [34] | 60.29 | 60.44 | 60.36
SURF [6] | 62.67 | 64.01 | 63.33
Harris-Laplace [38] | 63.89 | 62.91 | 63.41
Shi-Tomasi [56] | 69.28 | 64.13 | 66.74
MSER [36] | 52.45 | 50.58 | 51.53
KAZE [2] | 67.30 | 65.67 | 66.50
AKAZE [3] | 66.08 | 69.07 | 67.55
FAST [47] | 66.08 | 63.65 | 64.88
LIFT [74] | 56.97 | 60.73 | 58.82
Key.Net [25] | 68.99 | 67.47 | 68.24
SuperPoint [12] | 69.53 | 68.92 | 69.23
LF-Net [41] | 68.41 | 73.61 | 70.96
D2-Net [14] | 53.99 | 62.80 | 58.32
R2D2 [44] | 61.68 | 61.93 | 61.80
| BALF (ours)
---
67.21 | 73.51 | 70.28

Table 2: Repeatability results (%) on HPatches [5] sharp image pairs. For each
method, we report the average repeatability score on the viewpoint change,
illumination change, and all image sequences.

|  | Reference: Sharp
---
Target: Blur
| Reference: Blur
---
Target: Blur
Method | EASY $\uparrow$ | HARD $\uparrow$ | TOUGH $\uparrow$ |  | EASY $\uparrow$ | HARD $\uparrow$ | TOUGH $\uparrow$
SIFT[34] | 55.92 | 56.80 | 53.49 |  | 56.99 | 53.49 | 45.94
SURF[6] | 58.88 | 56.23 | 56.24 |  | 61.08 | 58.04 | 53.60
Harris-Laplace[38] | 36.70 | 37.97 | 34.98 |  | 35.76 | 31.95 | 27.47
Shi-Tomasi[56] | 57.33 | 55.11 | 49.11 |  | 56.29 | 53.75 | 51.37
MSER[36] | 44.19 | 41.97 | 37.05 |  | 41.81 | 38.24 | 34.59
KAZE[2] | 49.90 | 46.84 | 39.98 |  | 63.29 | 58.71 | 46.90
AKAZE [3] | 54.15 | 50.51 | 45.49 |  | 65.16 | 62.20 | 51.54
FAST[47] | 61.98 | 61.77 | 51.37 |  | 57.84 | 53.35 | 51.17
LIFT[74] | 50.69 | 50.17 | 46.99 |  | 48.34 | 46.57 | 46.53
Key.Net [25] | 60.34 | 54.71 | 44.69 |  | 62.77 | 58.17 | 49.25
SuperPoint [12] | 65.64 | 62.22 | 52.84 |  | 58.60 | 50.03 | 43.28
LF-Net [41] | 63.54 | 61.19 | 56.78 |  | 60.45 | 59.07 | 57.71
D2-Net [14] | 49.71 | 47.30 | 44.32 |  | 51.80 | 51.05 | 50.53
R2D2 [44] | 57.99 | 51.73 | 40.57 |  | 57.49 | 55.31 | 46.86
BALF (ours) | 74.12 | 74.45 | 71.84 |  | 70.48 | 68.43 | 67.71

Table 3: Repeatability results (%) on Blur-HPatches datasets. Our method
achieves the best performance on both blur-to-sharp and blur-to-blur
configurations. For compactness, we only report the total repeatability.
Detailed results on the respective changes can be found in our supplementary
material.

Evaluation with the original HPatches dataset. To study the performance of our
method on sharp images, we compare it against other methods on the original
HPatches dataset [5]. For this experiment, our network is trained on the GoPro
dataset with both sharp and blurred images, while we use the official pre-
trained netwroks for other methods. Those pre-trained models are usually well
trained with a much larger dataset, such as the COCO dataset [31] or the
ImageNet dataset [49]. The experimental results are presented in Tab. 3. It
can be demonstrated that our method performs on-par with the state-of-the-art
method (_i.e_. LF-Net [41]) for local feature detection with sharp images. Our
method is only slightly worse (_i.e_. $\sim$0.7% drop with the repeatability
metric) compared to LF-Net [41], although our network is trained with a
relatively small dataset which contains both sharp and blurred images. The
experimental results further demonstrate that our detector (_i.e_. BALF) has
superior performance even it is designed and trained to be robust against
motion blur.

|  | Reference: Sharp
---
Target: Deblur
|  | Reference: Deblur
---
Target: Deblur
| SRN-DeblurNet [60] |  | DeblurGAN-v2 [24] |  | SRN-DeblurNet [60] |  | DeblurGAN-v2 [24]
Method | EASY $\uparrow$ | HARD $\uparrow$ | TOUGH $\uparrow$ |  | EASY $\uparrow$ | HARD $\uparrow$ | TOUGH $\uparrow$ |  | EASY $\uparrow$ | HARD $\uparrow$ | TOUGH $\uparrow$ |  | EASY $\uparrow$ | HARD $\uparrow$ | TOUGH $\uparrow$
SIFT [34] | 56.62 | 55.36 | 53.83 |  | 57.63 | 56.52 | 56.50 |  | 59.75 | 58.13 | 50.63 |  | 59.44 | 57.98 | 51.21
SURF [6] | 61.89 | 59.13 | 54.88 |  | 61.97 | 59.57 | 56.34 |  | 62.44 | 61.26 | 55.27 |  | 62.07 | 60.81 | 55.09
Harris-Laplace [38] | 17.15 | 16.87 | 20.54 |  | 16.60 | 16.90 | 20.24 |  | 36.98 | 35.73 | 32.23 |  | 37.09 | 35.97 | 31.54
Shi-Tomasi [56] | 60.56 | 56.87 | 48.78 |  | 61.75 | 59.10 | 51.56 |  | 63.18 | 61.03 | 50.88 |  | 63.58 | 61.89 | 53.76
MSER [36] | 46.65 | 43.23 | 37.90 |  | 47.62 | 45.14 | 40.70 |  | 47.70 | 45.40 | 37.49 |  | 47.84 | 45.56 | 38.01
KAZE [2] | 65.14 | 63.10 | 60.16 |  | 65.23 | 63.18 | 61.41 |  | 64.20 | 62.45 | 53.41 |  | 64.13 | 61.87 | 54.19
AKAZE [3] | 66.03 | 64.02 | 60.64 |  | 66.29 | 64.50 | 62.72 |  | 65.71 | 64.08 | 56.10 |  | 65.75 | 63.75 | 57.35
FAST [47] | 61.77 | 59.67 | 61.60 |  | 62.00 | 60.44 | 58.74 |  | 62.72 | 61.14 | 50.61 |  | 63.40 | 61.70 | 55.43
LIFT [74] | 54.98 | 52.64 | 46.75 |  | 56.59 | 53.54 | 49.09 |  | 55.88 | 53.64 | 45.31 |  | 56.68 | 55.31 | 50.44
Key.Net [25] | 63.28 | 58.01 | 47.10 |  | 63.99 | 59.16 | 49.35 |  | 62.86 | 60.44 | 50.74 |  | 62.73 | 60.58 | 52.96
SuperPoint [12] | 67.72 | 64.05 | 55.26 |  | 67.95 | 65.86 | 58.22 |  | 66.38 | 63.16 | 49.52 |  | 66.50 | 63.71 | 52.09
LF-Net [41] | 62.22 | 59.90 | 54.73 |  | 62.59 | 60.24 | 54.81 |  | 63.06 | 62.03 | 57.28 |  | 63.00 | 61.79 | 57.85
D2-Net [14] | 51.81 | 49.49 | 45.94 |  | 52.64 | 50.21 | 45.88 |  | 53.60 | 53.00 | 50.93 |  | 53.93 | 53.29 | 50.74
R2D2 [44] | 60.31 | 55.43 | 43.26 |  | 60.46 | 55.68 | 45.38 |  | 58.11 | 54.80 | 45.77 |  | 57.95 | 55.03 | 47.86
BALF (ours) |  |  74.12 / 74.45 / 71.84
---
(EASY / HARD / TOUGH)
|  |  70.48 / 68.43 / 67.71
---
(EASY / HARD / TOUGH)

Table 4: Repeatability results (%) on deblurred images. The experimental
results demonstrate that single image deblurring network can indeed help with
the local feature detection. However, it still cannot perform on-par with our
one-stage detection network without doing any intermediate deblurring
operation.

Evaluation with the Blur-HPatches dataset. To evaluate the performance of our
network with motion blurred images, we evaluate it against other methods on
the synthesized blurred HPatches dataset. For fair comparisons, we re-train
all learning based methods on the same GoPro dataset. For compactness, we
report the total repeatability instead of the separated results for both
viewpoint and illumination changes as in Tab. 3. Detailed results on the
respective changes can be found in our supplementary material. We also
simulate two different application scenarios, _i.e_. the evaluations with
blur-to-sharp and blur-to-blur configurations. The blur-to-sharp configuration
can be applied for visual localization task. For example, we can pre-built a
large-scale 3D map with high-quality sharp images. It might happen that we
would use blurred image to query its location against the 3D map if we walk
around with an AR device at night. The blur-to-blur configuration can be
applied for visual odometry task, from which all the captured images within a
time window are motion blurred.

The experimental results shown in Tab. 3 demonstrate that prior methods have
degraded detection performance when the images are motion blurred. The
performance degrades more as the motion blur becomes severer. However, our
method achieves impressive performance compared to prior works. The reason
might be that prior learning based methods are usually built based on simple
convolutional layers (_e.g_. SuperPoint [12]) or networks which are not
specially designed for image deblurring. Motion blurred images thus challenge
those networks on keypoint detection task. In contrary, our network is built
based on the multi-axis gated MLP block, which has been demonstrated to be
effective for low-level image processing, _e.g_. image deblurring [64].

Evaluation with the GoPro dataset. We also evaluate the performance of our
network against the other methods on the GoPro testing sequences. For fair
comparisons, all learning based methods are re-trained on the GoPro dataset.
Since there is no ground truth homographies for GoPro dataset as those of
HPatches dataset [5], we randomly warp each testing image to get paired
transformed image for repeatability evaluation. The experimental results
presented in Tab. 5 demonstrate that our network achieves superior performance
compared to other approaches.

Evaluation with the Blur-HPatches dataset preprocessed by deblurring network.
We also study if the deblurring networks can help with keypoint detection from
blurred image even they usually cannot run in real-time. In particular, we
apply two state-of-the-art deblurring networks, _i.e_. SRN-DeblurNet [60] and
DeblurGAN-v2 [24], to deblur the images from the Blur-HPatches dataset and
then evaluate all the other methods on the restored images. We use the
official pretrained models and apply them to images from the Blur-HPatches
dataset preprocessed by these two deblurring networks without any finetuning.

The experimental results shown in Tab. 4 demonstrate that the deblurring
networks could indeed help a bit for keypoint detection from blurred image.
For example, it improves SuperPoint [12] from 62.22% to 64.05% on the
repeatability metric for the hard blur-to-sharp case. However, it still cannot
outperform our network, which has 74.45% repeatability on the blurred image
directly. The reason might be that single image deblurring networks have
limitations to restore image from severe motion blurred image. It further
demonstrates that to design an one-stage keypoint detector from blurred image
directly, would be a better option compared to that of detecting keypoints
from the intermediate deblurred image.

Method |  | Reference: Sharp
---
Target: Blur
$\uparrow$ | Reference: Blur
---
Target: Blur
$\uparrow$
SIFT [34] | 60.53 | 60.03
SURF [6] | 56.49 | 60.03
Harris-Laplace [38] | 23.54 | 16.35
Shi-Tomasi [56] | 51.26 | 57.61
MSER [36] | 46.54 | 43.09
KAZE [2] | 49.35 | 48.86
AKAZE [3] | 56.34 | 50.51
FAST [47] | 51.20 | 45.04
LIFT [74] | 48.61 | 50.56
LF-Net [41] | 60.82 | 66.60
Key.Net [25] | 57.54 | 58.37
SuperPoint [12] | 53.83 | 51.38
D2-Net [14] | 53.37 | 56.96
R2D2 [44] | 50.34 | 53.94
BALF (ours) | 75.68 | 75.15

Table 5: Repeatability results (%) on GoPro testing dataset. The experimental
results demonstrate that our network also achieves the state-of-the-art
performance compared to prior works on the GoPro dataset.

Efficiency and performance with real blurred images. We also evaluate the
efficiency of our method against other methods. Tab. 6 presents the
computational time for all those keypoint detectors. The handcrafted detectors
are evaluated on a CPU (Intel i7-8700), and remaining learning based
methods111Since it is impossible to separate the detector and descriptor
networks for individual evaluations from D2-Net or R2D2, we did not measure
their time consumption. run on a NVIDIA Geforce 2080 Ti. For fair comparisons,
we remove the descriptor network for evaluation from LIFT, SuperPoint, and LF-
Net. The experimental results demonstrate that our motion blur aware detector
is able to run in real-time ($\sim$34.46 FPS) with a VGA resolution image
(_i.e_. 480$\times$640 pixels). It further demonstrates that our detector can
be applied to many time-constrained applications, such as robotic visual
navigation.

Figure 5: Qualitative evaluations on real blurred image. The experimental
results demonstrate that our network is able to detect well distributed and
localized keypoints from either sharp and blurred images for further image
matching.

To further demonstrate the performance of our network on the real blurred
images, we also present a qualitative feature matching result between a sharp
image and a blurred image in Fig. 5. The images are selected from the RealBlur
dataset [45], which are captured by real cameras. It demonstrates that our
network is able to detect well localized and repeatable keypoints from both
sharp and blurred images. More details on the qualitative results can be found
from our supplementary material.

Method |  240$\times$320 pixels $\downarrow$ |  480$\times$640 pixels $\downarrow$
---|---|---
SIFT [34] | 21.80 | 66.70
SURF [6] | 148.46 | 165.78
Harris-LapLace [38] | 110.41 | 377.13
Shi-Tomasi [56] | 5.20 | 21.69
MSER [36] | 64.19 | 221.79
KAZE [2] | 105.85 | 298.43
AKZE [3] | 18.02 | 56.93
FAST [47] | 0.89 | 1.88
LIFT [74] | 2209.03 | 4901.38
Key.Net [25] | 15.64 | 35.82
SuperPoint [12] | 2.41 | 3.98
LF-Net [41] | 282.77 | 855.77
BALF (ours) | 8.15 | 29.02

Table 6: Computational cost (in millisecond) for different keypoint detectors
for a single image. It demonstrates that our network can run in real-time for
a VGA resolution image, _i.e_. 480$\times$640 pixels, which is commonly used
for many robotic applications.

##  5 Conclusion and Future Works

We present the first pure MLP-based network for local feature detection. Our
network takes advantages of the multi-axis gated MLP block and a squeeze-and-
excitation MLP block to build a pure MLP-based image encoder. The detection
module then apply differentiable channel-wise softmax operator for keypoint
detection. Extensive evaluations have been conducted with both synthetic and
real datasets. The experimental results demonstrate that our network delivers
on-par detection performance on sharp images, and achieves superior
performance with motion blurred images compared to prior works. Our network is
also light-weight and is able to run in real-time for VGA resolution image,
which further enables its application for time-constrained applications. It is
usually required to build correspondences via feature matching for higher
level applications, such as visual localization. We thus plan to design a
network to learn motion blur robust descriptors for those detected keypoints
as our future work.

## References

  * [1] Henrik Aanæs, A. Dahl, and Kim Steenstrup Pedersen.  Interesting interest points.  IJCV, 97:18–35, 2011.
  * [2] Pablo Fernández Alcantarilla, Adrien Bartoli, and Andrew J. Davison.  Kaze features.  In ECCV, 2012.
  * [3] Pablo Fernández Alcantarilla, Jesús Nuevo, and Adrien Bartoli.  Fast explicit diffusion for accelerated features in nonlinear scale spaces.  In BMVC, 2013.
  * [4] Yuval Bahat, Netalee Efrat, and Michal Irani.  Non-uniform blind deblurring by reblurring.  In ICCV, pages 3306–3314, 2017.
  * [5] Vassileios Balntas, Karel Lenc, Andrea Vedaldi, and Krystian Mikolajczyk.  Hpatches: A benchmark and evaluation of handcrafted and learned local descriptors.  In CVPR, pages 3852–3861, 2017.
  * [6] Herbert Bay, Tinne Tuytelaars, and Luc Van Gool.  Surf: Speeded up robust features.  In ECCV, 2006.
  * [7] Assia Benbihi, Matthieu Geist, and Cédric Pradalier.  Elf: Embedded localisation of features in pre-trained cnn.  In ICCV, pages 7939–7948, 2019.
  * [8] Shoufa Chen, Enze Xie, Chongjian Ge, Ding Liang, and Ping Luo.  Cyclemlp: A mlp-like architecture for dense prediction.  In ICLR, 2022.
  * [9] Sunghyun Cho and Seungyong Lee.  Fast motion deblurring.  In ACM SIGGRAPH Asia, 2009.
  * [10] Jaesung Choe, Chunghyun Park, François Rameau, Jaesik Park, and In-So Kweon.  Pointmixer: Mlp-mixer for point cloud understanding.  In ECCV, 2022.
  * [11] Gabriela Csurka and M. Humenberger.  From handcrafted to deep local invariant features.  ArXiv, abs/1807.10254, 2018.
  * [12] Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich.  Superpoint: Self-supervised interest point detection and description.  In CVPRW, pages 337–33712, 2018.
  * [13] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In ICLR, 2021.
  * [14] Mihai Dusmanu, Ignacio Rocco, Tomás Pajdla, Marc Pollefeys, Josef Sivic, Akihiko Torii, and Torsten Sattler.  D2-net: A trainable cnn for joint description and detection of local features.  In CVPR, pages 8084–8093, 2019.
  * [15] Steffen Gauglitz, Tobias Höllerer, and Matthew A. Turk.  Evaluation of interest point detectors and feature descriptors for visual tracking.  IJCV, 94:335–360, 2011.
  * [16] Amit Goldstein and Raanan Fattal.  Blur-kernel estimation from spectral irregularities.  In ECCV, 2012.
  * [17] Dong Gong, Jie Yang, Lingqiao Liu, Yanning Zhang, Ian D. Reid, Chunhua Shen, Anton van den Hengel, and Qinfeng Shi.  From motion blur to motion flow: A deep learning solution for removing heterogeneous motion blur.  In CVPR, pages 3806–3815, 2017.
  * [18] Jianyuan Guo, Yehui Tang, Kai Han, Xinghao Chen, Han Wu, Chao Xu, Chang Xu, and Yunhe Wang.  Hire-mlp: Vision mlp via hierarchical rearrangement.  In CVPR, pages 816–826, 2022.
  * [19] Qibin Hou, Zihang Jiang, Li Yuan, Mingg-Ming Cheng, Shuicheng Yan, and Jiashi Feng.  Vision permutator: A permutable mlp-like architecture for visual recognition.  IEEE TPAMI, 2022.
  * [20] Michal Hradis, Jan Kotera, Pavel Zemcik, and Filip Sroubek.  Convolutional neural networks for direct text deblurring.  In BMVC, 2015.
  * [21] Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu.  Squeeze-and-excitation networks.  IEEE TPAMI, 42:2011–2023, 2020.
  * [22] Dilip Krishnan, Terence Tay, and Rob Fergus.  Blind deconvolution using a normalized sparsity measure.  In CVPR, pages 233–240, 2011.
  * [23] Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiri Matas.  Deblurgan: Blind motion deblurring using conditional adversarial networks.  In CVPR, pages 8183–8192, 2018.
  * [24] Orest Kupyn, T. Martyniuk, Junru Wu, and Zhangyang Wang.  Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better.  In ICCV, pages 8877–8886, 2019.
  * [25] Axel Barroso Laguna, Edgar Riba, Daniel Ponsa, and Krystian Mikolajczyk.  Key.net: Keypoint detection by handcrafted and learned cnn filters.  In ICCV, pages 5835–5843, 2019.
  * [26] Karel Lenc and Andrea Vedaldi.  Learning covariant feature detectors.  In ECCV Workshops, 2016.
  * [27] Stefan Leutenegger, Margarita Chli, and Roland Y. Siegwart.  Brisk: Binary robust invariant scalable keypoints.  In ICCV, pages 2548–2555, 2011.
  * [28] Lerenhan Li, Jinshan Pan, Wei-Sheng Lai, Changxin Gao, Nong Sang, and Ming-Hsuan Yang.  Learning a discriminative prior for blind image deblurring.  In CVPR, pages 6616–6625, 2018.
  * [29] Wenshuo Li, Hanting Chen, Jianyuan Guo, Ziyang Zhang, and Yunhe Wang.  Brain-inspired multilayer perceptron with spiking neurons.  In CVPR, pages 773–783, 2022.
  * [30] Dongze Lian, Zehao Yu, Xing Sun, and Shenghua Gao.  As-mlp: An axial shifted mlp architecture for vision.  In ICLR, 2022.
  * [31] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick.  Microsoft coco: Common objects in context.  In ECCV, 2014.
  * [32] Hanxiao Liu, Zihang Dai, David R. So, and Quoc V. Le.  Pay attention to mlps.  In NeurIPS, 2021.
  * [33] Peidong Liu, Joel Janai, Marc Pollefeys, Torsten Sattler, and Andreas Geiger.  Self-supervised linear motion deblurring.  IEEE Robotics and Automation Letters, 5:2475–2482, 2020.
  * [34] G LoweDavid.  Distinctive image features from scale-invariant keypoints.  IJCV, 2004.
  * [35] Youssef Mansour, Kang Lin, and Reinhard Heckel.  Image-to-image mlp-mixer for image reconstruction.  ArXiv, abs/2202.02018, 2022.
  * [36] Jiri Matas, Ondřej Chum, Martin Urban, and Tomás Pajdla.  Robust wide-baseline stereo from maximally stable extremal regions.  Image Vis. Comput., 22:761–767, 2004.
  * [37] Krystian Mikolajczyk and Cordelia Schmid.  A performance evaluation of local descriptors.  In CVPR, pages II–II, 2003.
  * [38] Krystian Mikolajczyk and Cordelia Schmid.  Scale & affine invariant interest point detectors.  IJCV, 60:63–86, 2004.
  * [39] Krystian Mikolajczyk, Tinne Tuytelaars, Cordelia Schmid, Andrew Zisserman, Jiri Matas, Frederik Schaffalitzky, Timor Kadir, and Luc Van Gool.  A comparison of affine region detectors.  IJCV, 65:43–72, 2005.
  * [40] Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee.  Deep multi-scale convolutional neural network for dynamic scene deblurring.  In CVPR, pages 257–265, 2017.
  * [41] Yuki Ono, Eduard Trulls, Pascal V. Fua, and Kwang Moo Yi.  Lf-net: Learning local features from images.  In NeurIPS, 2018.
  * [42] Daniel J. Perrone and Paolo Favaro.  Total variation blind deconvolution: The devil is in the details.  In CVPR, pages 2909–2916, 2014.
  * [43] Wenqi Ren, Jiawei Zhang, Lin Ma, Jinshan Pan, Xiaochun Cao, Wangmeng Zuo, W. Liu, and Ming-Hsuan Yang.  Deep non-blind deconvolution via generalized low-rank approximation.  In NeurIPS, 2018.
  * [44] Jérôme Revaud, Philippe Weinzaepfel, César Roberto de Souza, No’e Pion, Gabriela Csurka, Yohann Cabon, and M. Humenberger.  R2d2: Repeatable and reliable detector and descriptor.  In NeurIPS, 2019.
  * [45] Jaesung Rim, Hoon Sung Chwa, and Sunghyun Cho.  Real-world blur dataset for learning and benchmarking deblurring algorithms.  In ECCV, 2020.
  * [46] Edward Rosten and Tom Drummond.  Machine learning for high-speed corner detection.  In ECCV, 2006.
  * [47] Edward Rosten, Reid B. Porter, and Tom Drummond.  Faster and better: A machine learning approach to corner detection.  IEEE TPAMI, 32:105–119, 2010.
  * [48] Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary R. Bradski.  Orb: An efficient alternative to sift or surf.  In ICCV, pages 2564–2571, 2011.
  * [49] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge.  IJCV, 115(3):211–252, 2015.
  * [50] Ehab Salahat and Murad Qasaimeh.  Recent advances in features extraction and description algorithms: A comprehensive survey.  In 2017 IEEE International Conference on Industrial Technology (ICIT), pages 1059–1063, 2017.
  * [51] Nikolay Savinov, Akihito Seki, Lubor Ladicky, Torsten Sattler, and Marc Pollefeys.  Quad-networks: Unsupervised learning to rank for interest point detection.  In CVPR, pages 3929–3937, 2017.
  * [52] Cordelia Schmid, Roger Mohr, and Christian Bauckhage.  Evaluation of interest point detectors.  IJCV, 37:151–172, 2004.
  * [53] Christian J. Schuler, Michael Hirsch, Stefan Harmeling, and Bernhard Schölkopf.  Learning to deblur.  IEEE TPAMI, 38:1439–1451, 2016.
  * [54] Jin shan Pan, Zhe Hu, Zhixun Su, and Ming-Hsuan Yang.  Deblurring text images via l0-regularized intensity and gradient prior.  In CVPR, pages 2901–2908, 2014.
  * [55] Jin shan Pan, Deqing Sun, Hanspeter Pfister, and Ming-Hsuan Yang.  Blind image deblurring using dark channel prior.  In CVPR, pages 1628–1636, 2016.
  * [56] Jianbo Shi and Carlo Tomasi.  Good features to track.  In CVPR, pages 593–600, 1994.
  * [57] Jian Sun, Wenfei Cao, Zongben Xu, and Jean Ponce.  Learning a convolutional neural network for non-uniform motion blur removal.  In CVPR, pages 769–777, 2015.
  * [58] Suwichaya Suwanwimolkul, Satoshi Komorita, and Kazuyuki Tasaka.  Learning of low-level feature keypoints for accurate and robust detection.  pages 2261–2270, 2021.
  * [59] Yehui Tang, Kai Han, Jianyuan Guo, Chang Xu, Yanxi Li, Chao Xu, and Yunhe Wang.  An image patch is a wave: Phase-aware vision mlp.  In CVPR, pages 10925–10934, 2022.
  * [60] Xin Tao, Hongyun Gao, Yi Wang, Xiaoyong Shen, Jue Wang, and Jiaya Jia.  Scale-recurrent network for deep image deblurring.  In CVPR, pages 8174–8182, 2018.
  * [61] Yurun Tian, Vassileios Balntas, Tony Ng, Axel Barroso Laguna, Y. Demiris, and Krystian Mikolajczyk.  D2d: Keypoint extraction with describe to detect approach.  In ACCV, 2020.
  * [62] Ilya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy.  Mlp-mixer: An all-mlp architecture for vision.  In NeurIPS, 2021.
  * [63] Hugo Touvron, Piotr Bojanowski, Mathilde Caron, Matthieu Cord, Alaaeldin El-Nouby, Edouard Grave, Gautier Izacard, Armand Joulin, Gabriel Synnaeve, Jakob Verbeek, and Herv’e J’egou.  Resmlp: Feedforward networks for image classification with data-efficient training.  IEEE TPAMI, 2022.
  * [64] Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Conrad Bovik, and Yinxiao Li.  Maxim: Multi-axis mlp for image processing.  In CVPR, pages 5759–5770, 2022.
  * [65] Yannick Verdie, Kwang Moo Yi, Pascal V. Fua, and Vincent Lepetit.  Tilde: A temporally invariant learned detector.  In CVPR, pages 5279–5288, 2015.
  * [66] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In-So Kweon.  Cbam: Convolutional block attention module.  In ECCV, 2018.
  * [67] Lei Xiao, Jue Wang, Wolfgang Heidrich, and Michael Hirsch.  Learning high-order filters for efficient blind deconvolution of document photographs.  In ECCV, 2016.
  * [68] Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, José Manuel Álvarez, and Ping Luo.  Segformer: Simple and efficient design for semantic segmentation with transformers.  2021\.
  * [69] Li Xu and Jiaya Jia.  Two-phase kernel estimation for robust motion deblurring.  In ECCV, 2010.
  * [70] Li Xu, Jimmy S. J. Ren, Ce Liu, and Jiaya Jia.  Deep convolutional neural network for image deconvolution.  In NeurIPS, 2014.
  * [71] Li Xu, Shicheng Zheng, and Jiaya Jia.  Unnatural l0 sparse representation for natural image deblurring.  In CVPR, pages 1107–1114, 2013.
  * [72] Xiangyu Xu, Jinshan Pan, Yujin Zhang, and Ming-Hsuan Yang.  Motion blur kernel estimation via deep learning.  IEEE TIP, 27:194–205, 2018.
  * [73] Ruomei Yan and Ling Shao.  Blind image blur estimation via deep learning.  IEEE TIP, 25:1910–1921, 2016.
  * [74] Kwang Moo Yi, Eduard Trulls, Vincent Lepetit, and Pascal V. Fua.  Lift: Learned invariant feature transform.  In ECCV, 2016.
  * [75] Tan Yu, Xu Li, Yunfeng Cai, Mingming Sun, and Ping Li.  S2-mlpv2: Improved spatial-shift mlp architecture for vision.  2021\.
  * [76] Syed Waqas Zamir, Aditya Arora, Salman Hameed Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao.  Multi-stage progressive image restoration.  In CVPR, pages 14816–14826, 2021.
  * [77] Jiawei Zhang, Jinshan Pan, Jimmy S. J. Ren, Yibing Song, Linchao Bao, Rynson W. H. Lau, and Ming-Hsuan Yang.  Dynamic scene deblurring using spatially variant recurrent neural networks.  In CVPR, pages 2521–2529, 2018.
  * [78] Linguang Zhang and Szymon M. Rusinkiewicz.  Learning to detect features in texture images.  In CVPR, pages 6325–6333, 2018.
  * [79] Xu Zhang, Felix X. Yu, Svebor Karaman, and Shih-Fu Chang.  Learning discriminative and transformation covariant local feature detectors.  In CVPR, pages 4923–4931, 2017.


# On the Fatou theorem for $\overline{\partial}_{J}$-subsolutions in wedges

Alexandre Sukhov

* Univ. Lille, Laboratoire Paul Painlevé, Departement de Mathématique, 59655 Villeneuve d’Ascq, Cedex, France<EMAIL_ADDRESS>The author is partially suported by Labex CEMPI.

Institut of Mathematics with Computing Centre - Subdivision of the Ufa
Research Centre of Russian Academy of Sciences, 45008, Chernyshevsky Str. 112,
Ufa, Russia.

Abstract. We prove a version of the Fatou theorem for bounded functions with a
bounded $\overline{\partial}_{J}$ part of the differential on wedge-type
domains in an almost complex manifold.

MSC: 32H02, 53C15.

Key words: almost complex manifold, $\overline{\partial}$-operator,
pseudoholomorphic disc, wedge-type domain, totally real manifold, the Fatou
theorem.

##  1 Introduction

The present paper is a continuation of the work [14]. Our goal is to study the
boundary behavior of certain classes of functions on almost complex manifolds
with boundary. It is well-known that non-constant holomorphic functions do not
exit (even locally) on an almost complex manifold $(M,J)$ (of complex
dimension $>1$) with an almost complex structure $J$ in general position. This
makes natural to study the functions satisfying suitable assumptions on the
$\overline{\partial}_{J}$-part of their differential: indeed, in this case the
problem of existence does not arise. Various aspects of the boundary behavior
of functions in $\mathbb{C}^{n}$ whose $\overline{\partial}$ (with respect to
the standard complex structure) part of differential is of some prescribed
growth, have been explored by several authors [8, 5, 9, 10, 2]. Their results
admit important applications in Several Complex Variables.

We extend some of the well-known results on boundary values of bounded
holomorphic functions (see [1, 4, 11, 2]) of several complex variables to the
almost complex case. Note that our main results are new also in the case of
the space $\mathbb{C}^{n}$ equipped with the standard complex structure. The
main result is Theorem 3.2 establishing a Fatou type theorem for domains with
generic corners (wedges). I also mention that, despite the fact the the
obtained results concern the classes of functions much larger than the
holomorphic ones, the presence of an (almost) complex structure is crucial. In
particular, this is due to the fact that we are working with low-dimensional
submanifolds of the boundary which are transverse to an almost complex
structure (totally real manifolds). Also, pseudoholomorphic curves (introduced
in [3]) are our main technical tool. We note that the main difficulty of the
proof is that in the case of wedge type domains the Chirka-Lindelöf principle
does not assure a non-tangential convergence. This obstacle is a principal
difference with respect to the case of smooth boundaries and it considerably
complicates the proof. This is the main motivation for the present paper.

The paper is organized as follows. Section 2 is preliminary and contains a
brief presentation of the theory of almost complex manifolds and their
properties. In Section 3 we present our main result. Section 4 contains its
proof.

##  2  Preliminaries: almost complex manifolds and their maps

Here we briefly recall basic notions concerning almost complex manifolds; a
detailed presentation is contained for example in [14]. Everywhere through
this paper we assume that manifolds and almost complex structures are of class
$C^{\infty}$ (the word ”smooth” means the regularity of this class); we notice
however that the main results remain true under considerably weaker regularity
assumptions.

Let $M$ be a smooth manifold of real dimension $2n$. An almost complex
structure $J$ on $M$ is a smooth map which associates to every point $p\in M$
a linear isomorphism $J(p):T_{p}M\to T_{p}M$ of the tangent space $T_{p}M$
such that $J(p)^{2}=-Id$; here $Id$ denotes the identity map of $T_{p}M$.
Thus, every linear map $J(p)$ is a complex structure (in the usual sense of
Linear Algebra) on a real vector space $T_{p}M$ . A couple $(M,J)$ is called
an almost complex manifold of complex dimension n.

A basic example is given by the standard complex structure
$J_{st}=J_{st}^{(2)}$ on $M=\mathbb{R}^{2}$; it is represented in the
canonical coordinates of $\mathbb{R}^{2}$ by the matrix

$\displaystyle J_{st}^{(2)}=\left(\begin{array}[]{cll}0&&-1\\\
1&&0\end{array}\right)$ (3)

More generally, the standard complex structure $J_{st}$ on $\mathbb{R}^{2n}$
is represented by the block diagonal matrix
$J_{st}=diag(J_{st}^{(2)},...,J_{st}^{(2)})$ (here and below we drop the
notation of dimension). Putting $iv:=Jv$ for $v\in\mathbb{R}^{2n}$, we
identify $(\mathbb{R}^{2n},J_{st})$ with $\mathbb{C}^{n}$; we use the notation
$z=x+iy=x+Jy$ for the standard complex coordinates
$z=(z_{1},...,z_{n})\in\mathbb{C}^{n}$.

Let $(M,J)$ and $(M^{\prime},J^{\prime})$ be smooth almost complex manifolds.
A $C^{1}$-map $f:M^{\prime}\to M$ is called $(J^{\prime},J)$-complex or
$(J^{\prime},J)$-holomorphic if it satisfies the Cauchy-Riemann equations

$\displaystyle df\circ J^{\prime}=J\circ df.$ (4)

For example, a map $f:\mathbb{C}^{n}\to\mathbb{C}^{m}$ is
$(J_{st},J_{st})$-holomorphic if and only if each component of $f$ is a usual
holomorphic function. In this special case the equations (4) coincide with the
usual Cauchy-Riemann equations in their real form. Note that in general the
first order PDE elliptic system (4) does not split on independent equations
for components of $f$.

Every almost complex manifold $(M,J)$ can be viewed locally as the Euclidean
unit ball $\mathbb{B}^{n}$ (or any other domain) in $\mathbb{C}^{n}$ equipped
with a small (in any $C^{m}$-norm) almost complex deformation of $J_{st}$. The
following well-known statement is often very useful.

###### Lemma 2.1

Let $(M,J)$ be an almost complex manifold of complex dimension $n$. Then for
every point $p\in M$, every $m\geq 0$ and $\lambda_{0}>0$ there exist a
neighborhood $U$ of $p$ and a coordinate diffeomorphism
$z:U\rightarrow\mathbb{B}^{n}$ such that $z(p)=0$, $dz(p)\circ J(p)\circ
dz^{-1}(0)=J_{st}$, and the direct image $z_{*}(J):=dz\circ J\circ dz^{-1}$
satisfies
$||z_{*}(J)-J_{st}||_{C^{m}(\overline{\mathbb{B}^{n}})}\leq\lambda_{0}$.

A simple proof is contained for example in [14].

In what follows we often denote the direct image $z_{*}(J)$ of $J$ again by
$J$, viewing it as a matrix representation of $J$ in the local coordinate
system $(z)$. Of course, the coordinate map $z$ is
($J,z_{*}(J)$)-biholomorphic. However, in general $z_{*}(J)$ does not coincide
with $J_{st}$ in a neighborhood of the origin in $\mathbb{C}^{n}$. Recall that
an almost complex structure $J$ is called integrable if $(M,J)$ is locally
biholomorphic in a neighborhood of each point to an open subset of
$(\mathbb{C}^{n},J_{st})$. In the case of complex dimension 1 every almost
complex structure is integrable. In the case of complex dimension $>1$
integrable almost complex structures form a highly special subclass in the
space of all almost complex structures on $M$. An efficient criterion of
integrablity is provided by the classical theorem of Newlander - Nirenberg
[6]: the entries of $J$ must satisfy some PDE system.

In the special case where $M^{\prime}$ has the complex dimension 1, the
solutions $f$ of (4) are called $J$-complex (or $J$-holomorphic or
pseudoholomorphic ) curves. Note that we view here the curves as maps i.e. we
consider parametrized curves. We use the notation
$\mathbb{D}=\\{\zeta\in\mathbb{C}:|\zeta|<1\\}$ for the unit disc in
$\mathbb{C}$ (i.e. $\mathbb{B}^{1}$) always assuming that it is equipped with
the standard complex structure $J_{{\rm st}}$. Considering the equations (4)
with $M^{\prime}=\mathbb{D}$, we call such a map $f$ a $J$-complex disc or a
pseudoholomorphic disc or just a holomorphic disc when a structure $J$ on the
target space is fixed. If a disc $f$ is continuous up to the boundary
$b\mathbb{D}$ of $\mathbb{D}$, then the restriction of $f$ on $b\mathbb{D}$ is
called the boundary of $f$. Let $\gamma$ be a non-empty subset of
$b\mathbb{D}$ and let $K$ be a subset of $M$. If $f(\gamma)\subset K$, we say
that $f$ is attached or glued to $K$ along $\gamma$. In this paper $\gamma$
usually will be the upper half-circle.

A fundamental fact is that pseudoholomorphic discs always exist in a suitable
neighborhood of any point of $p\in M$; furthermore, one can choose such a disc
tangent to any prescribed direction $v\in T_{p}M$. These discs depend smootly
on deformation of $J$, $p$ and $v$. Furthermore, one can view them as a small
deformation of discs in usual complex lines. This is the classical Nijenhuis-
Woolf theorem (see [7]). For the proof and other applications it is convenient
to rewrite the equations (4) in local coordinates similarly to the complex
version of the usual Cauchy-Riemann equations.

Our considerations are local, so assume that we are in a neighborhood $\Omega$
of $0$ in $\mathbb{C}^{n}$ with the standard complex coordinates
$z=(z_{1},...,z_{n})$. We assume that $J$ is an almost complex structure
defined on $\Omega$ and $J(0)=J_{st}$. Let a $C^{1}$-map

$z:\mathbb{D}\to\Omega,$ $z:\zeta\mapsto z(\zeta)$

be a $J$-complex disc. The equations (4) can be rewritten in the equivalent
form

$\displaystyle
z_{\overline{\zeta}}-A(z)\overline{z}_{\overline{\zeta}}=0,\quad\zeta\in\mathbb{D}.$
(5)

where we use the notation $z_{\overline{\zeta}}=\partial
z/\partial\overline{\zeta}$. Here a smooth map $A:\Omega\to Mat(n,\mathbb{C})$
is defined by the equality $L(z)v=A\overline{v}$ for any vector
$v\in\mathbb{C}^{n}$ and $L$ is an $\mathbb{R}$-linear map defined by
$L=(J_{st}+J)^{-1}(J_{st}-J)$. It is easy to check that the condition
$J^{2}=-Id$ is equivalent to the fact that $L$ is
$\overline{\mathbb{C}}$-linear. The matrix $A(z)$ is called the complex matrix
of $J$ in the local coordinates $z$ (see [12]). Locally the correspondence
between $A$ and $J$ is one-to-one. Note that the condition $J(0)=J_{st}$ means
that $A(0)=0$.

If $t$ are other local coordinates and $A^{\prime}$ is the corresponding
complex matrix of $J$ in the coordinates $t$, then, as it is easy to check, we
have the following transformation rule:

$\displaystyle
A^{\prime}=(t_{z}A+{t}_{\overline{z}})({\overline{t}}_{\overline{z}}+{\overline{t}}_{z}A)^{-1}$
(6)

(see the proof in [12]).

For the convenience of readers, I sketch here the proof of the above
mentionned Nijenhuis-Woolf theorem because this standard construction will be
used below in the proof of the main results. Recall that for a complex
function $f$ the Cauchy-Green transform $Tf$ is defined by

$\displaystyle Tf(\zeta)=\frac{1}{2\pi
i}\int_{\mathbb{D}}\frac{f(\omega)d\omega\wedge
d\overline{\omega}}{\omega-\zeta}$ (7)

This classical integral operator has the following properties:

  * (i)

$T:C^{r}(\mathbb{D})\to C^{r+1}(\mathbb{D})$ is a bounded linear operator for
every non-integer $r>0$ ( a similar property holds in the Sobolev scale). Here
we use the usual Hölder norm on the space $C^{r}(\mathbb{D})$.

  * (ii)

$(Tf)_{\overline{\zeta}}=f$ i.e. $T$ solves the $\overline{\partial}$-equation
in the unit disc.

  * (iii)

the function $Tf$ is holomorphic on
$\mathbb{C}\setminus\overline{\mathbb{D}}$.

Now fix a real non-integer $r>1$. Let $z:\mathbb{D}\to\mathbb{C}^{n}$,
$z:\mathbb{D}\ni\zeta\mapsto z(\zeta)$ be a $J$-complex disc. Since the
operator

$\Psi_{J}:z\longrightarrow w=z-TA(z)\overline{z}_{\overline{\zeta}}$

takes the space $C^{r}(\overline{\mathbb{D}})$ into itself, we can write the
equation (5) in the form

$\Psi_{J}(z)=h$

where $h$ is an arbitrary holomorphic (with respect to $J_{st}$) vector-
function. Thus, the disc $z$ is $J$-holomorphic if and only if the disc
$h=\Psi_{J}(z):\mathbb{D}\longrightarrow\mathbb{C}^{n}$ is
$J_{st}$-holomorphic. When the norm of $A$ is small enough (which is assured
by Lemma 2.1), then the operator $\Psi_{J}$ is a small deformation of the
identity and by the implicit function theorem this operator is invertible.
Hence we obtain a one-to-one correspondence between $J$-holomorphic discs and
usual $J_{st}$-holomorphic discs. This easily implies the existence of a
$J$-holomorphic disc in a given tangent direction through a given point of $M$
(choosing a suitable complex linear disc as $h$), as well as a smooth
dependence of such a disc on a deformation of a point or a tangent vector, or
on an almost complex structure; this also establishes the interior elliptic
regularity of discs.

Now we can define the $\overline{\partial}_{J}$-operator on an almost complex
manifold $(M,J)$. Consider first the situation when $J$ be an almost complex
structure defined in a domain $\Omega\subset\mathbb{C}^{n}$; one can view this
as a local coordinate representation of $J$ in a chart on $M$.

A $C^{1}$ function $F:\Omega\to\mathbb{C}$ is $(J,J_{st})$-holomorphic if and
only if it satisfies the Cauchy-Riemann equations

$\displaystyle F_{\overline{z}}+F_{z}A(z)=0,$ (8)

where $F_{\overline{z}}=(\partial F/\partial\overline{z}_{1},...,\partial
F/\partial\overline{z}_{n})$ and $F_{z}=(\partial
F/\partial{z}_{1},...,\partial F/\partial{z}_{n})$ are viewed as row-vectors.
Generally the only solutions to (8) are constant functions unless $J$ is
integrable (then $A$ vanishes identically in suitable coordinates). Note also
that (8) is an overdetermined linear PDE system while (5) is a quasilinear PDE
for a vector function on $\mathbb{D}$.

Every $1$-differnitial form $\phi$ on $(M,J)$ admits a unique decomposition
$\phi=\phi^{1,0}+\phi^{0,1}$ with respect to $J$. In particular, if
$F:(M,J)\to\mathbb{C}$ is a $C^{1}$-complex function, we have
$dF=dF^{1,0}+dF^{0,1}$. We use the notation

$\displaystyle\partial_{J}F=dF^{1,0}\,\,\,\mbox{and}\,\,\,\overline{\partial}_{J}F=dF^{0,1}$
(9)

In order to write these operators explicitely in local coordinates, we find a
local basic in the space of (1,0) and (0,1) forms. We view
$dz=(dz_{1},...,dz_{n})^{t}$ and
$d\overline{z}=(d\overline{z}_{1},...,d\overline{z}_{n})^{t}$ as vector-
columns. Then the forms

$\displaystyle\alpha=(\alpha_{1},...,\alpha_{n})^{t}=dz-
Ad\overline{z}\,\,\,\mbox{and}\,\,\overline{\alpha}=d\overline{z}-\overline{A}dz$
(10)

form a basis in the space of (1,0) and (0,1) forms respectively. Indeed, it
suffices to note that for 1-form $\beta$ is (1,0) (resp. $(0,1)$) for if and
only if for every $J$-holomorphic disc $z$ the pull-back $z^{*}\beta$ is a
usual (1,0) (resp. $(0,1)$) form on $\mathbb{D}$. Using the equations (5) we
obtain the claim.

Now we decompose the differential
$dF=F_{z}dz+F_{\overline{z}}d\overline{z}=\partial_{J}F+\overline{\partial}_{J}F$
in the basis $\alpha$, $\overline{\alpha}$ using (10) and obtain the explicit
expression

$\displaystyle\overline{\partial}_{J}F=(F_{\overline{z}}(I-\overline{A}A)^{-1}+F_{z}(I-A\overline{A})^{-1}A)\overline{\alpha}$
(11)

It is easy to check that the holomorphy condition $\overline{\partial}_{J}F=0$
is equivalent to (8) because $(I-A\overline{A})^{-1}A(I-\overline{A}A)=A$.
Thus

$\displaystyle\overline{\partial}_{J}F=(F_{\overline{z}}+F_{z}A)(I-\overline{A}A)^{-1}\overline{\alpha}$

We note that the term $(I-A\overline{A})^{-1}$ as well as the forms $\alpha$
affect only the non-essential constants in local estimates of the
$\overline{\partial}_{J}$-operator near a boundary point which we will perfom
in the next sections. So we may assume that in local coordinates this operator
is simply given by the left hand expression of (8).

##  3 Main result

First we introduce the main class of domains for this paper.

Let $M$ be an almost complex manifold. A generic manifold $E$ of real
codimension $k$ in $M$ can be defined as

$\displaystyle E=\\{p\in M:\rho_{j}(p)=0,j=1,...,k\\}$ (12)

where $\rho_{j}:M\to\mathbb{R}$ are smooth real functions satisfying

$\displaystyle\overline{\partial}_{J}\rho_{1}\wedge...\wedge\overline{\partial}_{J}\rho_{k}\neq
0$ (13)

near $E$. Precisely as in the case of an integrable structure, this means that
the tangent space $T_{p}E$ spans $T_{p}M$ i.e. the complex hull of $T_{p}E$
coincides with $T_{p}M$. If additionly $k=n$ (the maximal possible value of
$k$ compatible with the assumption (13)) , then $E$ is totally real. It is
equivalent to the fact that for every $p$ the holomorphic tangent space
$T_{p}E\cap J(T_{p}E)$ is trivial.

A domain $W=W(E)$ of the form

$\displaystyle W(E)=\\{p\in M:\rho_{j}(p)<0,j=1,...,k\\}$ (14)

is called a wedge with the edge $E$. Of course, if $k=1$ we have the usual
smoothly bounded domains.

Let $\Omega$ be a bounded domain in an almost complex manifold $(M,J)$. We
always assume that $\Omega$ is a wedge $W(E)$ of type (14) with the edge $E$.

Fix a hermitian metric on $M$ compatible with $J$; a choice of such metric
will not affect our results because it changes only constant factors in
estimates. We measure all distances and norms with respect to the choosen
metric.

Let $p\in E$ be a point of the edge. Fix local coordinates $z$ on $M$ near $p$
such that $p=0$ in these coordinates. A cone $K\subset\Omega$ with the vertex
$p$ is defined as the set of $z\in\Omega$ such in the above local coordinates
$K$ is a usual circular cone with vertex at the origin and directed by some
ray $l\subset\Omega$.

A non-tangential approach to $E$ at $p$ can be defined as the limit along the
sets $K$. Clearly, this notion is independent of choice of local coordinates.

###### Definition 3.1

A function $F:\Omega\to\mathbb{C}$ admits a non-tangential limit $L$ at $p\in
E$ if

$\lim_{K\ni z\to p}F(z)=L$

for each cone $K\subset\Omega$ with vertex at $p$.

As above, this definition is independent of a choice of local coordinates and
metrics.

The main result of the present paper is the following version of the Fatou
theorem.

###### Theorem 3.2

Let $(M,J)$ be an almost compex manifold of complex dimension $n\geq 1$ and
$W(E)$ be a wedge with a totally real edge $E$ in $M$. Suppose that $F\in
L^{\infty}(W(E))$ is a complex function of class $C^{1}$ on $W(E)$ and
$\overline{\partial}_{J}F$ is bounded on $W(E)$. Then $F$ admits a non-
tangential limit almost everywhere on $E$.

Of course, the interesting case arises only for $n>1$. Note that this result
is new also in the case where $M=\mathbb{C}^{n}$ and $J$ coincides with the
standard complex structure $J_{st}$. Note also that in the case where the edge
$E$ is not totally real but only a generic manifold with non-zero tangent
space, the convergent regions are tangent to $E$ along the holomorphic tangent
space of $E$, as usual in this type of problems (see [1, 2, 11, 14]). The
assumption of the boundedness of $\overline{\partial}_{J}F$ also may be
weakened. We drop the technical details focusing our presentation on the key
case.

##  4 Proof of Theorem 3.2

Our approach is based on the works [1, 4, 11, 14]. The proof of Theorem
contains several steps.

###  4.1 One-dimensional case

Recall some boundary properties of subsolutions of the
$\overline{\partial}$-operator in the unit disc.

Denote by $W^{k,p}(\mathbb{D})$ the usual Sobolev classes of functions
admitting generalized partial derivatives up to the order $k$ in
$L^{p}(\mathbb{D})$ (in fact we need only the case $k=0$ and $k=1$). In
particular $W^{0,p}(\mathbb{D})=L^{p}(\mathbb{D})$. We will always assume that
$p>2$.

Denote also by $\parallel f\parallel_{\infty}=\sup_{\mathbb{D}}|f|$ the usual
$\sup$-norm on the space $L^{\infty}(\mathbb{D})$ of complex functions bounded
on $\mathbb{D}$.

###### Lemma 4.1

Let $f\in L^{\infty}(\mathbb{D})$ and $f_{\overline{\zeta}}\in
L^{p}(\mathbb{D})$ for some $p>2$. Then

  * (a)

$f$ admits a non-tangential limit at almost every point $\zeta\in
b\mathbb{D}$.

  * (b)

if $f$ admits a limit along a curve in $\mathbb{D}$ approaching $b\mathbb{D}$
non-tangentially at a boundary point $e^{i\theta}\in b\mathbb{D}$, then $f$
admits a non-tangential limit at $e^{i\theta}$.

  * (c)

for each positive $r<1$ there exists a constant $C=C(r)>0$ (independent of
$f$) such that for every $\zeta_{j}\in r\mathbb{D}$, $j=1,2$ one has

$\displaystyle|f(\zeta_{1})-f(\zeta_{2})|\leq C(\parallel
f\parallel_{\infty}+\parallel
f_{\overline{\zeta}}\parallel_{L^{p}(\mathbb{D})})|\zeta_{1}-\zeta_{2}|^{1-2/p}$
(15)

The proof is contained in [14].

Sometimes it is convenient to apply the part (c) of Lemma on the disc
$\rho\mathbb{D}$ with $\rho>0$. Let $g\in L^{\infty}(\rho\mathbb{D})$ and
$g_{\overline{\zeta}}\in L^{p}(\rho\mathbb{D})$. The function
$f(\zeta):=g(\rho\zeta)$ satisfies the assumptions of Lemma 4.1 on
$\mathbb{D}$. Let $0<\alpha<\rho$ and let $|\tau_{j}|<\alpha$, $j=1,2$. Set
$\zeta_{j}=\tau_{j}/\rho$. Then $|\zeta_{j}|<r=\alpha/\rho<1$, $j=1,2$.
Applying (c) Lemma 4.1 to $f$ we obtain:

$\displaystyle|g(\tau_{1})-g(\tau_{2})|\leq(C(r)/\rho^{1-2/p})(\parallel
g\parallel_{\infty}+\rho\parallel
g_{\overline{\zeta}}\parallel_{L^{p}(\rho\mathbb{D})})|\tau_{1}-\tau_{2}|^{1-2/p}$
(16)

Note that $C=C(r)=C(\alpha/\rho)$ depends only on the quotient
$r=\alpha/\rho<1$. If $r$ is separated from $1$, the value of $C$ is fixed.

###  4.2 Attaching discs to a totally real manifold

Our proof of Theorem 3.2 uses the properties of a family of pseudoholomorphic
discs constructed in [13]. For the sake of completeness I briefly recall this
construction; the proofs are contained in [13]. Note also that this
construction is well known in the case of the standard complex structure.

(a) First consider the model case where $M=\mathbb{C}^{n}$ with $J=J_{st}$ and
$E=i\mathbb{R}^{n}=\\{x_{j}=0,j=1,...,n\\}$. Denote by $W$ the standard wedge
$W_{0}=\\{z=x+iy:x_{j}<0,j=1,....,n\\}$.

Consider the family of complex lines in $\mathbb{C}^{n}$:

$\displaystyle l:(c,t,\zeta)\mapsto(\zeta,\zeta t+ic)\in\mathbb{C}^{n}$ (17)

Here $\zeta\in\mathbb{C}$; the variables
$c=(c_{2},...,c_{n})\in\mathbb{R}^{n-1}$ and
$t\in\mathbb{R}^{n-1}_{+}=\\{t=(t_{2},...,t_{n})\in\mathbb{R}^{n-1}:t_{j}>0\\}$)
are viewed as parameters. Hence we wite $l(c,t,\zeta)=l(c,t)(\zeta)$. Denote
by $V$ the wedge $V=\mathbb{R}^{n-1}\times\mathbb{R}^{n-1}_{+}$. Also let
$\Pi=\\{{\rm Re\,}\zeta<0\\}$ be the left half-plane; its boundary $b\Pi$
coincides with the imaginary axis $i\mathbb{R}$. The following properties of
the above family are easy to check:

  * (a1)

the images $l(c,t)(b\Pi)$ form a family of real lines in $i\mathbb{R}^{n}=E$ .
For every fixed $t\in\mathbb{R}^{n-1}_{+}$ these lines are disjoint and

$\cup_{c\in\mathbb{R}^{n-1}}l(c,t)(b\Pi)=E.$

In other words, for every $t$ this family (depending on the parameter $c$)
forms a foliation of $E$ by parallel lines.

  * (a2)

one has

$\cup_{(c,t)\in V}l(c,t)(\Pi)=W_{0}.$

  * (a3)

For every fixed $t\in\mathbb{R}^{n-1}_{+}$, one has

$\cup_{c\in\mathbb{R}^{n-1}}l(c,t)(\Pi)=E_{t}=\\{z\in\mathbb{C}^{n}:{\rm
Re\,}(z_{j}-t_{j}z_{1})=0,j=2,...,n\\}\cap W_{0}$

and the union is disjoint. Every $E_{t}$ is a real linear $(n+1)$-dimensional
half-space contained in $W_{0}$ and $bE_{t}=E$.

  * (a4)

the family $(E_{t})$, $t\in\mathbb{R}^{n-1}_{+}$ is disjoint in $W_{0}$ and
its union coincides with $W_{0}$.

In what follows we will use these properties locally in a neighborhood of the
origin. It is convenient to reparametrize the family of complex half-lines
$l(c,t)$ by complex discs. Consider the Schwarz integral:

$\displaystyle S\phi(\zeta)=\frac{1}{2\pi
i}\int_{b\mathbb{D}}\frac{\omega+\zeta}{\omega-\zeta}\phi(\omega)\frac{d\omega}{\omega}$
(18)

For a non-integer $r>1$ consider the Banach spaces $C^{r}(b\mathbb{D})$ and
$C^{r}(\mathbb{D})$ (with the usual Hölder norm). It is classical that $S$ is
a bounded linear map in these classes of functions. For a real function
$\phi\in C^{r}(b\mathbb{D})$ the Schwarz integral $S\phi$ is a function of
class $C^{r}(\mathbb{D})$ holomorphic in $\mathbb{D}$; the trace of its real
part on the boundary coincides with $\phi$ and its imaginary part vanishes at
the origin.

In order to fill $W_{0}$ by complex discs glued to $i\mathbb{R}^{n}$ along the
(closed) upper semi-circle
$b\mathbb{D}^{+}=\\{e^{i\theta}:\theta\in[0,\pi]\\}$ we have to reparametrize
the above family of complex lines. Set also
$b\mathbb{D}^{-}:=b\mathbb{D}\setminus b\mathbb{D}^{+}$. Fix a smooth real
function $\phi:b\mathbb{D}\to[-1,0]$ such that $\phi|b\mathbb{D}^{+}=0$ and
$\phi|b\mathbb{D}^{-}<0$.

Consider now a real $2n$-parametric family of holomorphic discs
$z^{0}=(z_{1}^{0},...,z_{n}^{0}):\mathbb{D}\to\mathbb{C}^{n}$ with components

$\displaystyle
z_{j}^{0}(c,t)(\zeta)=x_{j}(\zeta)+iy_{j}(\zeta)=t_{j}S\phi(\zeta)+ic_{j},j=1,...,n$
(19)

Here $t_{j}>0$ and $c_{j}\in\mathbb{R}$ are parameters

Obviously, every $z^{0}(c,t)(\mathbb{D})$ is a subset of $l(c,t)(\Pi)$ and
$z^{0}(b\mathbb{D}^{+})=l(c,t)(b\Pi)$. Thus, the family $z^{0}(c,t)$ is a
(local) biholomorphic reparametrization of the family $l(c,t)$. As a
consequence, the properties (a1)-(a5) also hold for the family $z^{0}(c,t)$.
Notice also the following obvious properties of this family:

  * (a6)

for every $j$ one has $x_{j}|b\mathbb{D}^{+}=0$ and $x_{j}(\zeta)<0$ when
$\zeta\in\mathbb{D}$ (by the maximum principle for harmonic functions).

  * (a7)

the evaluation map $Ev_{0}:(c,t,\zeta)\mapsto z^{0}(c,t)(\zeta)$ is one-to-one
from $V\times\mathbb{D}$ to $W_{0}$.

In the general case consider a totally real manifold $E$ and the wedge
$W=W(E)$ given by (14). Applying the implicit function theorem in suitable
local coordinates, one can assume that $E$ is defined by the vector equation
$x=h(y)$ where $h(0)=0$, and $dh(0)=0$. Using the Cauchy-Green operator and
the Schwarz integral, one can write a non-linear integral equation such that
its solutions of the form

$\displaystyle(c,t,\zeta)\mapsto z(c,t)(\zeta)$ (20)

are $J$-complex discs glued to $E$ along $b\mathbb{D}^{+}$. Since
$h(y)=o(|y|)$, the family $z(c,t)$ is a small deformation of the family
$z^{0}(c,t)$ in any $C^{m}$ norm, $m>1$. This follows by the impicit function
theorem solving the above mentioned integral equation (see details in [14]).
Hence, the geometric properties of obtained discs remain similar to the above
model case: indeed, the properties of linear discs (a1)-(a5) are stable under
small perturbations.

(b) For reader’s convenience we state explicitely the properties of the family
(20).

Fix $\delta>0$. The family $z(c,t):\overline{\mathbb{D}}\to W$ of
pseudoholomorphic discs smooth on $\overline{D}$ and smoothly depending on
real parameters $t=(t_{2},...,t_{n})$, $t_{j}>0$ and $c\in\mathbb{R}^{n-1}$ ,
satisfies the following properties:

  * (b1)

the images $z(c,t)(b\mathbb{D}^{+})$ form a family of real curves in $E$ . For
every fixed $t\in\mathbb{R}^{n-1}_{+}$ these curves are disjoint and

$\cup_{c\in\mathbb{R}^{n-1}}z(c,t)(b\mathbb{D}^{+})=E.$

In other words, for every $t$ this family (depending on the parameter $t$)
forms a foliation of $E$. Furthermore, every disc is contained in $W=W(E)$.

  * (b2)

one has the inclusion

$W_{\delta}=\\{z:\rho_{j}-\delta\sum_{k\neq
j}\rho_{k}<0\\}\subset\cup_{(c,t)\in V}z(c,t)(\mathbb{D}).$

  * (b3)

For every fixed $t\in\mathbb{R}^{n-1}_{+}$, the union

$E_{t}:=\cup_{c\in\mathbb{R}^{n-1}}z(c,t)(\mathbb{D})\subset W$

is a real generic $(n+1)$-dimensional manifold with boundary $bE_{t}=E$.

  * (b4)

the family $(E_{t})$, $t\in\mathbb{R}^{n-1}_{+}$ is disjoint and its union
contains $W_{\delta}$.

###  4.3 Around the Chirka-Lindelöf principle

Here we introduce an analog of the Chirka - Lindelöf principle [1] for wedges
in almost complex manifolds. This is one the main technical tools of our
proof. Note that in [1] the situation is considered in full generality (for
integrable complex structures) with minimal assumptions on regularity of
domains. It is observed their that in the case of domains with piecewise
smooth boundaries, for a bounded holomorphic function an existence a boundary
limit along a smooth curve (transverse to the boundary) does not imply an
existence of a non-tangential limit at a boundary point. This is a serious
difference with respect to smoothly bounded domains and one of the main
technical obstacles in the proof of our main result. Nevertherless, in the
non-smooth case the convergence along a curve implies an existence of the
limit along any curve with the same tangent line at a boundary point.

Let $W=W(E)$ be a wedge with the edge $E$ in an almost complex manifold
$(M,J)$ of dimension $>1$ and let $p$ be a point of $E$.

A curve $\gamma:[0,1[\to W(E)$ is called $p$-admisssible if the following
assumptions are satisfied:

  * (i)

$\gamma$ of class $C^{\infty}[0,1]$ and $p=\gamma(1)\in E$;

  * (ii)

$\gamma$ is not tangent to any face $\\{\rho_{j}=0\\}$, $j=1,...,n$ at $p$. In
particular, $\gamma$ is not tangent to $E$.

###### Proposition 4.2

Let $W(E)$ be a wedge with the edge $E$ in an almost complex manifold $(M,J)$
of dimension $>1$, and let $F$ satisfies assumptions of Theorem 3.2. If $F$
has a limit along a $p$-admissible curve $\gamma_{1}$ at $p\in E$, then $F$
has the same limit along each curve in $W(E)$ tangent to $\gamma_{1}$ at $p$.

Proof. Let $\gamma_{2}$ be another curve satisfying the assumptions (i), (ii)
and such that $\gamma_{1}$ and $\gamma_{2}$ have the same tangent line at $p$.
Without loss of generality asssume that $p=0$ (in local coordinates).

It follows by the Nijenhuis-Woolf theorem that there exists a family
$z_{t}(\zeta):\mathbb{D}\to\mathbb{C}^{n}$, of embedded $J$-holomorphic discs
near the origin in $\mathbb{C}^{n}$ satisfying the following properties:

  * (i)

the family $z_{t}$ is smooth on $\overline{\mathbb{D}}\times[0,1]$

  * (ii)

for every $t\in[0,1]$ the disc $z_{t}$ transversally intersects each curve
$\gamma_{j}$ at a unique point coresponding to some parameter value
$\zeta_{j}(t)\in\mathbb{D}$ , $t\in[0,1[$, $j=1,2$. In other words
$\gamma_{j}(t)=z_{t}(\zeta_{j}(t))$. Furthermore, $\zeta_{1}(t)=0$, i.e. this
point is the center of the disc $z_{t}$.

In the case of the standard complex structure each such a disc is simply an
open piece (suitably parametrized) of a complex line intersecting
transversally the both of curves $\gamma_{j}$. Recall that the curves are
embedded near the origin and tangent at the origin so such a family of complex
lines obviously exist. The $J$-holomorphic discs are obtained from this family
of lines by a small deformation described in the proof of the Nijenhuis-Woolf
theorem in Section 2.

Furthermore, because of the condition (i), the restrictions $F\circ z_{t}$
have $\overline{\zeta}$\- derivatives bounded on $D$ uniformly with respect to
$t$. Indeed, it follows by the Chain Rule and (5) that

$(F\circ
z)_{\overline{\zeta}}=(F_{\overline{z}}+F_{z}A){\overline{z}}_{\overline{\zeta}}$

and now we use the assumption that $\overline{\partial}_{J}F$ is bounded.

Since the curves $\gamma_{j}$ are tangent at the origin, we have

$\displaystyle|\zeta_{2}(t)|=o(1-t)$ (21)

as $t\to 1$.

The curve $\gamma_{1}$ is admissible, so we have

$dist(\gamma_{1}(t),bW)=O(1-t)$

as $t\to 1$. Hence, there exists $\rho(t)=O(1-t)$ as $t\to 1$ such that
$z_{t}(\rho(t)\mathbb{D})$ is contained in $W$. Applying (16) to the
composition $f:=F\circ z_{t}(\zeta)$ on the disc $\rho(t)\mathbb{D}$, we
obtain (fixing $r>0$)

$\displaystyle|f(0)-f(\zeta_{2}(t))|\leq(C/O(1-t)^{1-2/p})(\parallel
f\parallel_{\infty}+O(1-t)\parallel
f_{\overline{\zeta}}\parallel_{\infty})o((1-t)^{1-2/p}\to 0$ (22)

as $t\to 1$. Note that by (21) for every $t$ the point $\zeta_{2}(t)$ is
contained in $(1/2)\rho(t)\mathbb{D}$; hence, the constant $C$ is independent
of $t$ (see remark after (16) ). This concludes the proof.

###  4.4 Convergence along families of rays

Under some additional assumption one can assure a non-tangential convergence.
Fix local coordinates such that $E=i\mathbb{R}^{n}$,
$W=W(E)=\\{x_{j}<0,j=1,...,n\\}$, $J(0)=J_{st}$. Such a change of coordinates
is always possible by Lemma 2.1. Through the remaining part of this paper we
assume that such local coordinates are fixed.

Let $z(c,t)(\zeta)$ be a $J$-complex disc constructed in the subsection 4.2.
It follows by Lemma 4.1 that the composition $F\circ z(c,t)$ admits non-
tangential limits almost everywhere on $b\mathbb{D}^{+}$.

Let $p\in E$. Assume $F\circ z(c,t)$ admits a radial limit at point
$\zeta^{0}\in b\mathbb{D}^{+}$ and $z(c,t)(\zeta^{0})=p$. The image
$\gamma_{p}$ of this radial segment by disc $z(c,t)$ in general is not a
segment of some real line in $\mathbb{C}^{n}$, but only an admissible curve.
Hence it follows by Proposition 4.2 that $F$ admits the same limit along the
real ray $l_{p}\subset W$ with vertex at $p$ and tangent to $\gamma_{p}$ at
$p$. It is enough to consider the case $p=0$.

###### Lemma 4.3

Let $\Lambda\subset W$ be the set of rays with vertex at $0$. Assume that
$\Lambda$ is the uniqueness set for holomorphic (with respect to the standard
structure $J_{st}$) functions. Suppose that $F(z)$ admits a limit along any
ray from $\Lambda$ as $z\to 0$. Then $F$ admits a limit along any ray
contained in a cone $K\subset W$. If the limits are the same for all rays from
$\Lambda$, the $F$ admits a non-tangential limit at $0$.

Proof. We present the proof in three steps.

Step 1. Consider a sequence of functions $F_{k}(z)=F(z/k)$, $k=1,2,....$. We
need the following analog of the Montel compactness principle:

###### Lemma 4.4

The family $(F_{k})$ contains a subsequence converging uniformly on compacts
in any cone $K$ to a function $F^{0}$ holomorphic with respect to $J_{st}$.

This is a consequence of Lemma 4.1. Indeed, it suffices to prove the
equicontinuity. By the Nijenhuis-Woolf theorem any ball small enough is
folaited by the psedoholomorhic discs through the center (of course, this
foliation is singular at the center of the ball). Such a foliation is a small
deformation of the foliation of the ball by complex lines through its center.
Thus, we apply Lemma 4.1 to the restriction of $F_{k}$ on each disc and
conclude that the sequence $(F_{k})$ is equicontinuous on this ball. This
implies the above mentioned compactness of this family and proves Lemma 4.4

We continue the proof of Lemma 4.3. Passing to the limit we obtain that
$\overline{\partial}_{J_{s}t}F^{0}=0$ in the sense of distributions which
implies that $F^{0}$ is a usual holomorphic function (with respect to
$J_{st}$). Indeed we have

$|F_{\overline{z}}(z)+A(z)F_{z}(z)|\leq C$

Hence for $\varepsilon>0$ we obtain

$|F_{\overline{z}}(\varepsilon z)+A(\varepsilon z)F_{z}(\varepsilon z)|\leq C$

Therefore

$|\varepsilon F_{\overline{z}}(\varepsilon z)+A(\varepsilon z)\varepsilon
F_{z}(\varepsilon z)|\leq\varepsilon C$

and as a consequence

$|(F(\varepsilon z))_{\overline{z}}+A(\varepsilon z)(F(\varepsilon
z))_{z}|\leq\varepsilon C$

Since $F(\varepsilon z)\to F^{0}(z)$ converges in the sense of distributions
as $\varepsilon\to 0$ and $A(0)=0$, we obtain that $F^{0}_{\overline{z}}=0$ in
the sense of distrubutions and so $F^{0}$ is a usual folomorphic function.

Step 2. Since $F$ admits a limit along any ray from $\Lambda$, we obtain that
$F^{0}$ is constant along such a ray. Thus given $\alpha>0$ one has
$F^{0}(\alpha z)=F^{0}(z)$ for all $z\in\Lambda$. But $\Lambda$ is a
uniqueness set. Therefore, the last identity (with fixed $\alpha$) holds for
all $z\in W$ and $F^{0}$ is constant already along any ray in $W$. This
implies that $F$ admits a limit along each ray.

Step 3. If the limits are the same , say, $L$ , along all rays from $\Lambda$
and $\Lambda$ is the uniqueness set, we obtain that $F^{0}=L$ on $\Lambda$ and
hence $F^{0}\equiv L$ on $W$. This implies Lemma.

###  4.5 Limits along rays almost everywhere

Now we study the existence of limits along rays. Fix a family $l_{z}$ of rays
smoothly depending on $z\in E=i\mathbb{R}^{n}$ such that $l_{z}\subset W$.

###### Lemma 4.5

Let $K_{z}\subset W$ be a family of open cones smoothly depending on $z$, with
vertex at $z$ and directed by $l_{z}$. For almost every $z\in E$ the function
$F$ admits a limit along any ray in $K_{z}$.

Note that at this moment we do not yet claim that the limits are the same
independenty of a ray with the same vertex.

Proof. We begin with the model flat case where $J=J_{st}$. Consider the family
of flat complex discs $z(c,t)$ given by (19). Let ${\cal S}^{n-1}$ denotes the
unit sphere in $\mathbb{R}^{n}$. Denote by $\Sigma$ a countable dense set in
${\cal S}^{n-1}\cap{\mathbb{R}^{n}_{+}}$. Set
$\Sigma=\cup_{j\in\mathbb{N}}t^{j}$ and fix some $t^{j}$. By Lemma 4.1 the
restriction of $F$ on every disc $z(c,t^{j})$ admits a non-tangential limit
almost everywhere on $b\mathbb{D}^{+}$ i.e. on the full measure subset $Y_{j}$
in $b\mathbb{D}^{+}$. Note that the image of every ray with vertex in
$b\mathbb{D}^{+}$ by $z(c,t^{j})$ belongs to $\mathbb{R}^{n}+ic$ (and its
parallel translation to $0$ belongs to $\mathbb{R}^{n}$). When $c$ runs over a
neighborhood of $0$, the images $z(c,t^{j})(Y_{j})$ sweep a full measure set
$X_{j}$ in $E=i\mathbb{R}^{n}$. The intersection $X=\cap_{j}X_{j}$ of such
sets is again a full measure set in $E=i\mathbb{R}^{n}$. For every point $p\in
X$ and every $j$ the function $F$ admits a limit along the ray with vertex at
$p$ and parallel to $t^{j}$. But $\Sigma$ is dense in ${\cal
S}^{n-1}\cap{\mathbb{R}^{n}_{+}}$ so for every fixed $p\in X$ these rays form
a uniqueness set for usual holomorphic functions. Therefore, we can apply
Lemma 4.3 at $p$.

The proof in the general case of any almost complex structure $J$ follows by a
perturbation argument. Indeed, indeed, we fill $W$ by discs $z(c,v)$ of the
form (20) constructed in subsection 4.2. In general, the images of rays by
$z(c,v)$ are not rays in $W$ , but they are the admissible curves. By
Proposition 4.2 the function $F$ admits a limit along any ray tangent to such
curve at its boundary point on $E$. Then the above argument goes through
literally. This completes the proof.

###  4.6 Non-tangential limits

Here we conclude the proof of theorem establishing the following

###### Lemma 4.6

$F$ admits non-tangential limits almost everywhere on $E$.

Proof. Recall again that $E=i\mathbb{R}^{n}$. Fix a unit vector $v$ such that
$z+l$ belongs to $W$, where $z\in E$ and the ray $l$ is directed by $v$.
Consider a family of cones $K_{z}$ with the vertex $z$ and directed by the
vector $l$; we assume that $K_{z}$ smoothly depends on $z$. By Lemma 4.5 there
exists a full measure subset $\tilde{E}$ of $E$ such that the function $F$
admits a limit $F^{*}$ along the ray $z+l$ with vertex at $z\in\tilde{E}$.
Consider the sequence of functions $f_{m}(z)=\sup_{0<t\leq
1/m}|F(z+tv)-F^{*}(z)|$, where $z\in\tilde{E}$ and $m>0$, $m\in\mathbb{N}$.
This sequence converges to a function $0$ for almost every $z\in\tilde{E}$.
Applying the Egorov theorem, we conclude that for each $\delta>0$ there exists
a subset $E_{\delta}\subset\tilde{E}$ such that

  * (i)

$m(\tilde{E}\setminus E_{\delta})<\delta$ (here $m(X)$ denotes the Lebesgue
$n$-measure of the subset $X\subset E=i\mathbb{R}^{n}$).

  * (ii)

the sequence $(f_{m})$ converges uniformly to $0$ on $E\setminus E_{\delta}$.

In particular, the functions $F(z+(1/m)v)$ are continuous on $E$ and converge
to $F^{*}$ as $m\to\infty$. Hence, the function $F^{*}$ is continuous on
$\tilde{E}\setminus E_{\delta}$ as the uniform limit of a sequence of
continuous functions. Note also that (by Lemma 4.5) one can assume that the
function $F$ admits a limit along any ray ( not only $l$) with vertex at each
point $z$ of $\tilde{E}\setminus E_{\delta}$. Recall that by the Lebesgue
theorem almost every point of $\tilde{E}\setminus E_{\delta}$ is a density
point with the density equal to $1$. We claim that $F$ admits a non-tangential
limit at such a point.

Assume that $0\in\tilde{E}\setminus E_{\delta}$ is a density point. Also we
may assume that the limit of $F$ along $l$ at $0$ is equal to $0$ i.e.
$F^{*}(0)=0$. Let $(z^{k})$ be a sequence in an arbitrary ray $l_{1}$ in
$K_{0}$, $(z^{k})$ is converging to $0$. We prove that there exists a
subsequence $(z^{k_{q}})$ such that $F(z^{k_{q}})\to 0$ as $q\to\infty$. This
obviously implies the claim.

Set $r^{k}=|z^{k}|$. Then the point $\tilde{z}=z^{k}/r^{k}$ belongs to
$l_{1}$, $|\tilde{z}|=1$. Passing again to a subsequence, assume that
$\tilde{z}$ is independent of $k$. Consider the sequence of functions
$\tilde{F}_{k}(z)=F(r^{k}z)$. It follows from Lemma 4.4 that one can extract a
subsequence $\tilde{F}_{k_{q}}$ (in what follows we skip the subindex $q$)
converging uniformly on compacts of $K_{0}$ to a function $\tilde{F}$ which is
a usual holomorphic function on $K_{0}$. Since $F$ admits a limit along every
ray at the origin, the function $\tilde{F}$ is constant along every ray with
vertex at the origin (as in the proof of Lemma 4.3) . It suffices to show that
$\tilde{F}=0$. Indeed, in this case we obtain

$\lim_{k\to\infty}F(z^{k})=\lim_{k\to\infty}\tilde{F}_{k}(z^{k}/r^{k})=\tilde{F}(\tilde{z})=0.$

Consider the set $\Lambda$ formed by the real half-lines $L=p+l$,
$p\in\tilde{E}\setminus E_{\delta}$. Clearly, $\Lambda$ is contained in the
subspace spanned by $E$ and $l$. $\Lambda$ is a generic (with respect to
$J_{st}$) half-space contained in $W$, with the boundary $E$. Consider a
sequence of points points $(q^{k})$ contained in a ray $l_{2}\subset\Lambda$
(with the vertex at the origin) and converging to $0$. We choose the sequence
$(q^{k})$ such that $r^{k}q^{k}=p^{k}+w^{k}$, where
$p^{k}\in\tilde{E}\setminus E_{\delta}$ and a vector $w^{k}$ belongs to the
ray $l$. Since $0$ is a density point, the set of rays in $\Lambda$ admitting
such a sequence, is a full measure subset of $\Lambda$. The sequence
$r^{k}q^{k}$ tends to $0$, hence the sequences $p^{k}$ and $w^{k}$ tend to $0$
as well. Therefore

$\lim_{k\to\infty}\tilde{F}_{k}(q^{k})=\lim_{k\to\infty}F(r^{k}q^{k})=\lim_{k\to\infty}F(p^{k}+w^{k})$

In view of the uniform convergence of the sequence of functions $(f_{k})$
(introduced at the beginning of the proof of Lemma) on $\tilde{E}\setminus
E_{\delta}$, and the continuity of the function $F^{*}$ at $0$, the last limit
is equal to $F^{*}(0)=0$. Hence $\tilde{F}=0$ on the ray $l_{2}$ Thus the
function $\tilde{F}$ vanishes on a full measure subset of $\Lambda$ which is
the uniqueness set for holomorphic functions. Hence $\tilde{F}=0$ on $K_{0}$
and $F$ tends to $0$ along any ray. By Lemma 4.3 $F$ admits a non-tangential
limit $0$ at the origin. Hence, $F$ admits a non-tangential limit almost
everywhere on $\tilde{E}\setminus E_{\delta}$ because almost all points are
density points (with the density $1$).

Finally, consider the sets $E_{k}=E_{\delta}$ for $\delta=1/k$. The
intersection $\Sigma=\cap_{k}E_{k}$ has the measure $0$ and $F$ admits limits
almost everywhere outside $\Sigma$, We conclude that $F$ admits non-tangential
limits almost everywhere on $E$. This completes the proof.

## References

  * [1] Chirka, E. The Lindelöf and Fatou theorems in $\mathbb{C}^{n}$, Math. U.S.S.R Sb. 21(1973), 619-641.
  * [2] Forstnerič, F. Admissible boundary values of bounded holomorphic functions in wedges, Trans. Amer. Math. Soc. 332 (1992), 583-593.
  * [3] Gromov, M. Pseudoholomorphic curves in symplectic manifolds, Invent. Math. 82 (1985), 307-347.
  * [4] Y.Khurumov, On the Lindelof theorem in $\mathbb{C}^{n}$, Dokl. Akad Nauk SSSR, 273 (1983), 1325-1328.
  * [5] Nagel, A., Rudin, W. Local boundary behavior of bounded holomorphic functions, Can. J. Math. XXX(1978), 583-592.
  * [6] Newlander, A., Nirenberg, L. Complex analytic coordinates in almost complex manifolds, Ann. Math. 65(1957), 391-404.
  * [7] Nijenhuis, A., Woolf, W. Some integration problems in almost - complex and complex manifolds, Ann. Math. 77(1963), 424-489.
  * [8] Nirenberg, L., Webster, S., Yang, P. Local boundary regularity of holomorphic mappings, Commun. Pure Appl. Math. 33(1980), 305-338.
  * [9] Pinchuk, S., Khasanov, S. Asymptotically holomorphic functions and their applications, Mat. USSR Sb. 62 (1989), 541-550
  * [10] Rosay, J.-P. A propos des ”wedges” et d’ ”edges” et de prolongements holomorphes, Trans. Amer. Math. Soc. 297(1986), 63-72.
  * [11] Sadullaev, A. A boundary uniqueness theorem in $\mathbb{C}^{n}$, Matem. Sb. bf 101(1976), 568-583.
  * [12] Sukhov, A., Tumanov, A. Filling hypersurfaces by discs in almost complex manifolds of dimension 2, Indiana Univ. Math. J. 57(2008), 509-544.
  * [13] Sukhov, A, Pluripolar sets, real submanifolds and pseudoholomorphic discs, J.Australian Math. Soc. (2019), 1-19, doi: 10.1017/S1446788719000119.
  * [14] Sukhov, A. The Chirka-Lindelof and Fatou type theorems for $\overline{\partial}_{J}$-subsolutions, Revista Math. Iberoamericana, 36 (2020), 1469-1487.