# Lightweight Monocular Depth with a Novel Neural Architecture Search Method

Lam Huynh1 Phong Nguyen1 Jiri Matas2
Esa Rahtu3 Janne Heikkilä1
1Center for Machine Vision and Signal Analysis, University of Oulu
2 Center for Machine Perception, Czech Technical University, Czech Republic
3Computer Vision Group, Tampere University

###### Abstract

This paper presents a novel neural architecture search method, called LiDNAS,
for generating lightweight monocular depth estimation models. Unlike previous
neural architecture search (NAS) approaches, where finding optimized networks
is computationally highly demanding, the introduced novel Assisted Tabu Search
leads to efficient architecture exploration. Moreover, we construct the search
space on a pre-defined backbone network to balance layer diversity and search
space size. The LiDNAS method outperforms the state-of-the-art NAS approach,
proposed for disparity and depth estimation, in terms of search efficiency and
output model performance. The LiDNAS optimized models achieve result superior
to compact depth estimation state-of-the-art on NYU-Depth-v2, KITTI, and
ScanNet, while being $7\%$-$500\%$ more compact in size, i.e the number of
model parameters.

##  1 Introduction

Depth information is essential to numerous computer vision applications,
including robotics, mixed reality, and scene understanding. Traditionally,
accurate depth measurements are acquired using stereo or multi-view setups
[23, 52] or active sensors such as ToF cameras, LIDARs. However, deploying
such methods for resource-limited devices is costly or may even be infeasible
in practice. Considering this, current advances in learning-based monocular
depth estimation proffering them as viable alternatives to conventional
approaches.

Figure 1:  Absolute relative error and thresholded accuracy ($\delta_{1}$) vs.
the number of parameters for recent depth estimation methods on NYU-Depth-v2
(top) and KITTI (bottom) – the LiDNAS models outperforms the lightweight
baselines (black), while using substantially less parameters than the current
state-of-the-art methods (blue). Compared to the recent NAS-based depth
estimation method (red), LiDNAS improves in both performance and compactness.

Recent deep neural networks (DNN) show compelling results on single image
depth estimation by formulating apparent depth cues [3, 5, 13, 27, 28, 33, 35,
36, 44, 57] or estimating relative depth in unconstrained settings [4, 34,
45]. Moreover, self-supervised methods [15, 19] offered appealing solutions
for single image depth estimation. Nevertheless, most studies focus on
increasing accuracy at the expense of model complexity, making them infeasible
to devices with limited hardware capabilities.

To tackle this problem, lightweight depth estimation methods [55] were
proposed by utilizing small and straightforward architecture. Usually such
simple designs are unreliable and yield low-quality predictions. Other popular
strategies include quantizing the weights of a network into low-precision
fixed-point operations [22] or pruning by directly cut off less important
filters [58]. That being said, these methods depend on a baseline model, tend
to degrade its performance afterward and incapable of exploring new
combinations of DNN operations. Moreover, creating a resource-constrained
model is a non-trivial task requiring 1) expert knowledge to carefully balance
accuracy and resource and 2) plenty of tedious trial-and-error work.

Neural architecture search (NAS), proposed recently [61, 62], exhibits
compelling results, and more importantly, promises to relieve from the manual
tweaking of deep neural architectures. Unfortunately, NAS methods mostly
obligate thousands of training hours on hundreds of GPUs. To address this,
recent NAS studies introduced various efficiency increasing techniques, which
include weight sharing [41], and network transformation [11]. These methods
show promising results, but they are still expensive and mainly focus on
classification and detection.

This paper introduces LiDNAS, an efficient model compactness-aware NAS
framework, with the objective of searching for accurate and lightweight
monocular depth estimation architectures. The approach is based on two main
ideas. First, we observe that previous NAS methods essentially search for a
few types of cells and then repeatedly accumulate the same cells to build the
whole network. Although doing this simplifies the search process, it also
restrains layer diversity that is important for computational efficiency.
Instead, we construct a pre-defined backbone network that utilizes different
layers striving for the right balance between flexibility and search space
size. Secondly, we proposed the Assisted Tabu Search (ATS) for efficient
neural architecture search. Inspired by the recent NAS study that suggests
estimating network performance without training [40], we integrate this idea
into our multi-objective search function to swiftly evaluate our candidate
networks. This, in turn, reduces $\sim 90\%$ search time compared to state-of-
the-art NAS-based disparity and depth estimation approaches [48].

Figure 1 summarizes a comparison between our LiDNAS models and other state-of-
the-art lightweight approaches. Compared to PyD-Net [42], our method improves
the REL, RMSE, and thresholded accuracy by $13.6\%$, $8.3\%$, and $3\%$ with
similar execution time on the Google Pixel 3a phone (see Table 4). Compared to
FastDepth [55] and EDA [54], our model achieves higher accuracy with fewer
parameters. To summarize, our work makes the following contributions:

  * •

We propose a multi-objective exploration framework, LiDNAS, searching for
accurate and lightweight monocular depth estimation architectures.

  * •

We introduce a novel scheme called Assisted Tabu Search, enabling fast neural
architecture search.

  * •

We create a well-defined search space that allows computational flexibility
and layer diversity.

  * •

We achieve the state-of-the-art results compared to the lightweight baselines
on NYU-Depth-v2, KITTI, and ScanNet while using less parameters.

The implementation of LiDNAS will be made publicly available upon publication
of the paper.

Figure 2: The search space of our LiDNAS framework. Models are constructed
from a pre-defined backbone network containing encoder, decoder, refine,
downsample and upsample blocks (green). A block is formed by several identical
layers (orange) that are generated from a pool of operations and connections.
Layers within a block are the same while layers of different blocks can be
different. Figure 3: Overview of the proposed approach.

##  2 Related work

Monocular depth estimation Learning-based single image depth estimation was
first introduced by Saxena et al. [50] and Eigen et al. [9, 10]. Later studies
improved accuracy by using large network architectures [5, 27, 32] or
integrating semantic information [30] and surface normals [43]. Fu et al. [14]
formulated depth estimation as an ordinal regression problem, while [4, 34]
estimated relative instead of metric depth. Facil et al. [13] proposed to
learn camera calibration from the images for depth estimation. Recent
approaches further improve the performance by exploiting monocular priors such
as planarity constraints [35, 36, 59, 28, 33] or occlusion [44]. Gonzalez and
Kim [20] estimated depth by synthesizing stereo pairs from a single image,
while [57] and [45] applied vision-transformer for depth prediction. However,
these studies mostly focus on increasing accuracy at the cost of model
complexity that is infeasible in resource-limited settings.

Lightweight depth estimation architectures For resource-limited hardware, it
is more desirable to not only have a fast but also accurate model. One simple
alternative is employing lightweight architectures such as MobileNet [24, 25,
49, 55], GhostNet [21], and FBNet [54]. One popular approach is utilizing
network compression techniques, including quantization [22], network pruning
[58], and knowledge distillation [60, 1]. Other methods employing well-known
pyramid networks or dynamic optimization schemes. However, these tasks are
tedious, require a lot of trial-and-error, and usually lead to architectures
with low accuracy.

Neural Architecture Search There has been increasing interest in automating
network design using neural architecture search. Most of these methods focus
on searching high-performance architecture using reinforcement learning [2,
37, 41, 61, 62], evolutionary search [46], differentiable search [38], or
other learning algorithms [39]. However, these methods are usually very slow
and require huge resources for training. Other studies [8, 12, 26] also
attempt to optimize multiple objectives like model size and accuracy.
Nevertheless, their search process optimizes only on small tasks like CIFAR.
In contrast, our proposed method targets real-world data such as NYU, KITTI
and ScanNet.

##  3 LiDNAS

We propose the LiDNAS framework to search for accurate and lightweight
monocular depth estimation architectures. The overview the our approach is
presented in Figure 3. It takes in a dataset as input to search for the best
possible model. This model can be deployed for depth estimation on hardware-
limited devices. The first subsection defines the search space while the
remaining two describe our multi-objective exploration and search algorithm.

###  3.1 Search Space

Previous neural architecture search (NAS) studies demonstrated the
significance of designing a well-defined search space. A common choice of NAS
is searching for a small set of complicated cells from a smaller dataset [62,
37, 46]. These cells are later replicated to construct the entire architecture
that hindered layer diversity and suffered from domain differences [53]. On
the other hand, unlike classification tasks, dense prediction problems involve
mapping a feature representation in the encoder to predictions at larger
spatial resolution in the decoder.

Figure 4: Plots of the score at initialisation of untrained architectures
against evaluation metrics after training: (a), (b) accuracy ($\delta_{1}$);
(e) mean absolute error of the inverse depth (iMAE); and (d), (e), (f)
absolute relative error (REL). Plots from the first, second and third row are
obtained from NYU-Depth-v2, KITTI and ScanNet dataset, respectively.

To this end, we build our search space upon a pre-defined backbone that is
shown as the set of green blocks in Figure 2. The backbone is divided into
multi-scale pyramid networks operating at different spatial resolutions. Each
network scale consists of two encoder blocks, two decoder blocks, a refine
block, a downsample and a upsample block (except for scale 1). Each block is
constructed from a set of identical layers (marked as orange in Figure 2).
Inspired by [53], we search for the layer from a pool of operations and
connections, including:

Figure 5: The flowchart of our architecture search that utilizes the Assisted
Tabu Search (ATS) with mutation to search for accurate and lightweight
monocular depth estimation networks.

  * •

The number of resolution scales $S$.

  * •

The number of layer for each block $N_{i,j}$.

  * •

Convolutional operations (ConvOps): vanilla 2D convolution, depthwise
convolution, and inverted bottleneck convolution.

  * •

Convolutional kernel size (KSize): $3\times 3$, $5\times 5$.

  * •

Squeeze and excitation ratio (SER): $0$, $0.25$.

  * •

Skip connections (SOps): residual or no connection.

  * •

The number of output channels: $F_{i,j}$.

where $i$ indicates the resolution scale and $j$ is the block index at the
same resolution. Internal operations such as ConvOps, KSize, SER, SOps,
$F_{i,j}$ are utilized to construct the layer while $N_{i,j}$ determines the
number of layer that will be replicated for blocki,j. In other words, as shown
in Figure 2, layers within a block (e.g. layers 1 to N1,2 of Encoder Block 1,2
are the same) are similar while layers of different blocks (e.g. Layer 1 in
Refine Block 1,5 versus Layer 1 in Upsample Block S,7) can be different.

We also perform layer mutation to further diversifying the network structure
during the architecture search process. The mutation operations include:

  * •

Swapping operations of two random layers with compatibility check.

  * •

Modifying a layer with a new valid layer from the predefined operations.

Moreover, we also set computational constraints to balance the kernel size
with the number of output channels. Therefore, increasing the kernel size of
one layer usually results in decreasing output channels of another layer.

Assuming we have a network of $S$ scales, and each block has a sub-search
space of size $M$ then our total search space will be $M^{5+[(S-1)*7]}$.
Supposedly, a standard case with $M=192$, $S=5$ will result in a search space
of size $\sim 2\times 10^{75}$.

###  3.2 Multi-Objective Exploration

We introduce a multi-objective search paradigm seeking for both accurate and
compact architectures. For this purpose, we monitor the validation grade
$\mathcal{G}$ that formulates both accuracy and the number of parameter of a
trained model. It is defined by

$\mathcal{G}(m)=\alpha\times
A(m)+(1-\alpha)\times\bigg{[}\frac{P}{P(m)}\bigg{]}^{r}$ (1)

where $A(m)$ and $P(m)$ are validation accuracy and the number of parameters
of model $m$. $P$ is the target compactness, $\alpha$ is the balance
coefficient, and $r$ is an exponent with $r=0$ when $P(m)\leq P$ and otherwise
$r=1$. The goal is to search for an architecture $m^{*}$ where $G(m^{*})$ is
maximum.

However, computing $G$ requires training for every architecture candidate,
resulting in considerable search time. To mitigate this problem, Mellor et al.
[40] suggested to score an architecture at initialisation to predict its
performance before training. For a network $f$, the score(f) is defined as:

$score(f)=log|K_{H}|$ (2)

where $K_{H}$ is the kernel matrix. Assume the mapping of model $f$ from a
batch of data $X=\\{x_{i}\\}^{N}_{i=1}$ is $f(x_{i})$. By assigning binary
indicators to every activation units in $f$, a linear region $x_{i}$ of data
point $i$ is represented by the binary code $c_{i}$. The kernel matrix $K_{H}$
is defined as:

$K_{H}=\begin{pmatrix}N_{A}-d_{H}(c_{1},c_{1})&\dots&N_{A}-d_{H}(c_{1},c_{N})\\\
\vdots&\ddots&\vdots\\\
N_{A}-d_{H}(c_{N},c_{1})&\dots&N_{A}-d_{H}(c_{N},c_{N})\end{pmatrix}$ (3)

where $N_{A}$ is the number of activation units, and $d_{H}(c_{i},c_{j})$ is
the Hamming distance between two binary codes. Inspired by this principle, we
generate and train a set of different architectures on NYU, KITTI, and
ScanNet. We evaluate the performance of these models and visualize the results
against the score that in our case is the mapping of depth values within image
batches. Plots in Figure 4 show that models with higher score tend to yield
better results. Leveraging this observation, we 1) utilize the score in our
initial network ranking, and 2) define the mutation exploration reward
$\mathcal{R}$ as:

$\mathcal{R}(m_{i},m_{j})=\alpha\times\frac{score(m_{j})}{score(m_{i})}+(1-\alpha)\times\bigg{[}\frac{P}{P(m_{j})}\bigg{]}^{r}$
(4)

where $m_{j}$ is a child network that is mutated from $m_{i}$ architecture.

###  3.3 Search Algorithm

The flowchart of our architecture search is presented in Figure 5. We first
randomly generate $60K$ unique parent models and create the initial network
ranking based on the score in Eq. 2. We then select six architectures in which
three are the highest-ranked while the other three have the highest score of
the networks with the size closest to the target compactness.

Starting from these initial networks, we strive for the best possible model
utilizing the Assisted Tabu Search (ATS). Tabu search (TS) [18] is a high
level procedure for solving multicriteria optimization problems. It is an
iterative algorithm that starts from some initial feasible solutions and aims
to determine better solutions while being designed to avoid traps at local
minima.

We propose ATS by applying Eq. 1 and 4 to TS to speed up the searching
process. Specifically, we mutate numerous children models ($m_{1}$, $m_{2}$,
.., $m_{n}$) from the current architecture ($m_{c}$). The mutation exploration
reward $\mathcal{R}(m_{c},m_{i})$ is calculated using Eq. 4. ATS then chooses
to train the mutation with the highest rewards (e.g. architecture $m_{i}$ as
demonstrated in Figure 5). The validation grade of this model
$\mathcal{G}(m_{i})$ is calculated after the training. The performance of the
chosen model is assessed by comparing $\mathcal{G}(m_{i})$ with
$\mathcal{G}(m_{c})$. If $\mathcal{G}(m_{i})$ is larger than
$\mathcal{G}(m_{c})$, then $m_{i}$ is a good mutation, and we opt to build the
next generation upon its structure. Otherwise, we swap to use the best option
in the tabu list for the next mutation. The process stops when reaching a
maximum number of iterations or achieving a terminal condition. The network
ranking will be updated, and the search will continue for the remaining parent
architectures.

Table 1: Evaluation on the NYU-Depth-v2 dataset. Metrics with $\downarrow$ mean lower is better and $\uparrow$ mean higher is better. Type column shows the exploration method used to obtain the model. RL, ATS, and manual, refer to reinforcement learning, assisted tabu search, and manual design, respectively. Architecture | #params | Type | Search Time | REL$\downarrow$ | RMSE$\downarrow$ |  $\boldsymbol{\delta_{1}}$$\uparrow$ |  $\boldsymbol{\delta_{2}}$$\uparrow$ |  $\boldsymbol{\delta_{3}}$$\uparrow$
---|---|---|---|---|---|---|---|---
AutoDepth-BOHB-S |  Saikia et al.’19 [48] | 63.0M | RL | 42 GPU days | 0.170 | 0.599 | - | - | -
EDA |  Tu et al.’21 [54] | 5.0M | Manual | - | 0.161 | 0.557 | 0.782 | 0.945 | 0.984
FastDepth |  Wofk et al.’19 [55] | 3.9M | Manual | - | 0.155 | 0.599 | 0.778 | 0.944 | 0.981
SparseSupNet |  Yucel et al.’21 [60] | 2.6M | Manual | - | 0.153 | 0.561 | 0.792 | 0.949 | 0.985
Ef+FBNet |  Tu & Wu et al. [54, 56] | 4.7M | Manual | - | 0.149 | 0.531 | 0.803 | 0.952 | 0.987
LiDNAS-N | Ours | 2.1M | ATS | 4.3 GPU days | 0.132 | 0.487 | 0.845 | 0.965 | 0.993

##  4 Performance analysis

In this section, we evaluate the performance of the proposed method and
compare it with several baselines on the NYU-Depth-v2, KITTI, and ScanNet
datasets.

###  4.1 Datasets

We evaluate the proposed method using NYU-Depth-v2 [51], ScanNet [7] and KITTI
[16] datasets. NYU-Depth-v2 contains $\sim 120K$ RGB-D images obtained from
464 indoor scenes. From the entire dataset, we use 50K images for training and
the official test set of 654 images for evaluation. The ScanNet dataset
comprises of 2.5 million RGB-D images acquired from 1517 scenes. For this
dataset, we use the training subset of $\sim 20K$ images provided by the
Robust Vision Challenge 2018 [17] (ROB). In this paper, we report the results
on the ScanNet official test set of $5310$ images instead. KITTI is an outdoor
driving dataset, where we use the standard Eigen split [9, 10] for training
(39K images) and testing (697 images).

###  4.2 Evaluation metrics

The performance is assessed using the standard metrics provided for each
dataset. That is, for NYU-Depth-v2 and KITTI we calculate the mean absolute
relative error (REL), root mean square error (RMSE), and thresholded accuracy
($\delta_{i}$). For the ScanNet dataset, we provide the mean absolute relative
error (REL), mean square relative error (sqREL), scale-invariant mean squared
error (SI), mean absolute error (iMAE), and root mean square error (iRMSE) of
the inverse depth values.

###  4.3 Implementation Details

For searching, we directly perform our architecture exploration on the
training samples of the target dataset. We set the target compactness
parameter $P$ using the previously published compact models as a guideline. We
set the maximum number of exploration iteration to 100 and stop the
exploration procedure if a better solution cannot be found after 10
iterations. The total search time required to find optimal architecture is
$\sim 4.3$ GPU days.

For training, we use the Adam optimizer [31] with
$(\beta_{1},\beta_{2},\epsilon)=(0.9,0.999,10^{-8})$. The initial learning
rate is $7*10^{-4}$, but from epoch 10 the learning is reduced by $5\%$ per
$5$ epochs. We use batch size 256 and augment the input RGB and ground truth
depth images using random rotation ([-5.0, +5.0] degrees), horizontal flip,
rectangular window droppings, and colorization (RGB only).

###  4.4 Comparison with state-of-the-art

#### NYU-Depth-v2:

We set the target compactness $P=2M$ with the balance coefficient $\alpha=0.6$
to search for the optimized model on NYU-Depth-v2. We then select the best
performance model (LiDNAS-N) and compare its results with lightweight state-
of-the-art methods [54, 55, 56, 60] along with their numbers of parameters. As
shown in Table 1, LiDNAS-N outperforms the baseline while containing the least
amount of parameters. Comparing with the best-performing approach, the
proposed model improves the REL, RMSE, and $\theta_{1}$ by $11.4\%$, $8.2\%$,
and $6.8\%$ while compressing the model size by $55\%$. Our method produces
high-quality depth maps with sharper details as presented in Figure 9.
However, we observe that all methods still struggle in challenging cases, such
as the scene containing Lambertian surfaces as illustrated by the example in
the third column of Figure 9. Moreover, the proposed method improves REL and
RMSE by $22.3\%$ and $18.7\%$ while using only $3\%$ of the model parameters
comparing to the state-of-the-art NAS-based disparity and depth estimation
approaches [48]. In addition, our method requires $90\%$ less search time than
[48].

Table 2: Evaluation on the KITTI dataset. Metrics with $\downarrow$ mean lower
is better and $\uparrow$ mean higher is better.

Method #params REL$\downarrow$ RMSE$\downarrow$
$\boldsymbol{\delta_{1}}$$\uparrow$ $\boldsymbol{\delta_{2}}$$\uparrow$
$\boldsymbol{\delta_{3}}$$\uparrow$ FastDepth [55] 3.93M 0.156 5.628 0.801
0.930 0.971 PyD-Net [42] 1.97M 0.154 5.556 0.812 0.932 0.970 EQPyD-Net [6]
1.97M 0.135 5.505 0.821 0.933 0.970 DSNet [1] 1.91M 0.159 5.593 0.800 0.932
0.971 LiDNAS-K 1.78M 0.133 5.157 0.842 0.948 0.980

Figure 6: The progress of different searching scenarios on the NYU-Depth-v2
dataset. From left to right, charts show the accuracy, the number of
parameters, and validation grade vs. the number of searching iterations.

#### KITTI:

In the case of KITTI, we aim at the target compactness of $P=1.5M$ with
$\alpha=0.55$. We then train our candidate architectures with the same self-
supervised procedure proposed by [19] and adopted by the state-of-the-art
approaches [1, 6, 42, 55]. After the search, we pick the best architecture
(LiDNAS-K) to compare with the baselines and report the performance figures in
Table 2. The LiDNAS-K model yields competitive results with the baselines
while also being the smallest model. We observe that our proposed method
provides noticeable improvement from PyD-Net and EQPyD-Net. Examples from
Figure 7 show that the predicted depth maps from LiDNAS-K are more accurate
and contain fewer artifacts.

#### ScanNet:

For ScanNet, we set the target compactness to $4.5M$ with $\alpha=0.57$ for
searching. Despite of being compact, our best performance model (LiDNAS-S)
produces competitive results compared with state-of-the-art methods, as shown
in Table 3. More specifically, it requires only $20\%$ of the number of
parameters in comparison with the best performance baseline. We also observe
that although SARPN [5] and Hu et al. [27] models are multiple times larger
than DS-SIDENet [47] or DAV [28], the latter still yield better results,
emphasizing the importance of optimal network structure. Furthermore, our
model produces comparable depth maps as shown in Figure 10. Details of the
generated architectures are provided in the supplementary material.

Figure 7: Comparison on the Eigen split of KITTI. (a) input image, (b)
LiDNAS-K, (c) DSNet [1], (d) PyD-Net [42], and (e) FastDepth [55]. Images in
the right column presented zoom-in view for better visualization. Table 3:
Evaluation results on ScanNet [7] dataset.

Architecture #params REL sqREL SI iMAE iRMSE SARPN [5] 210.3M 0.134 0.077
0.015 0.093 0.100 Hu et al. [27] 157.0M 0.139 0.081 0.016 0.100 0.105 DS-
SIDENet [47] 49.8M 0.133 0.057 - - - DAV [28] 25.1M 0.118 0.057 0.015 0.089
0.097 LiDNAS-S 5.2M 0.117 0.059 0.015 0.090 0.097

#### Runtime Measurement:

We also compare the runtime of our models with state-of-the-art lightweight
methods on an Android device using the app from the Mobile AI benchmark
developed by Ignatov et al. [29]. To this end, we utilize the pre-trained
models provided by the authors (Tensorflow [42], PyTorch [55]) and convert
them to tflite. Unfortunately, we can only report the measurement on the
mobile CPU due to the technical issues occurring when converting PyTorch
models to TFLite GPU delegate. That being said, the results in Table 4 suggest
that the proposed approaches produce competing performance, with the potential
of running real-time on mobile devices with further optimization.

Table 4: Average runtime comparison of the proposed method and other lightweight models. Runtime values are measured using a Pixel 3a phone with input image resolution ($640\times 480$). Architecture | CPU(ms)
---|---
FastDepth [55] | 458
Ef+FBNet [54, 56] | 852
PyD-Net [42] | 226
LiDNAS-K | 205
LiDNAS-N | 262
LiDNAS-S | 380

###  4.5 Ablation studies

#### Exploration Convergence:

We experiment with various settings for the multi-objective balance
coefficient ($\alpha$) to assess its effect on the performance. For this
purpose, we perform the architecture search with $\alpha$ set to $0.0$, $0.4$,
$0.5$, $0.6$, and $1.0$ while the target compactness $P=2.0M$. Figure 6
presents the searching progress for accuracy (left), the number of parameters
(center), and validation grade (right) from one parent architecture on NYU-
Depth-v2. We observe that, scenario with $\alpha=0.0$ quickly becomes
saturated as it only gives reward to the smallest model. Searching with
$\alpha=0.4$ favors models with compact size but also with limited accuracy.
The case with $\alpha=0.5$ provides a more balanced option, but accuracy is
hindered due to fluctuation during searching. The exploration with
$\alpha=1.0$ seeks for the network with the best accuracy yet producing
significantly larger architecture while the case where $\alpha=0.6$ achieves
promising accuracy although with slightly bigger model than the target
compactness.

#### Searching Scenarios:

To further analyze the outcome of different searching scenarios, we perform
architecture searches for six parent networks in five settings with
$\alpha=0.0,0.4,0.5,0.6,1.0$ and $P=2.0M$ on NYU-Depth-v2. Results in Figure 8
show that best performance models in case $\alpha=0.5$ are more spread out,
while training instances with $\alpha=0.6$ tend to produce both accurate and
lightweight architectures. This, in turn, emphasizes the trade-off between
validation accuracy and network size.

Figure 8: Trade-off between accuracy vs. the number of parameters of best
models trained with different searching scenarios on NYU-Depth-v2 dataset.
Figure 9: Comparison on the NYU test set. (a) input image, (b) ground truth,
(c) LiDNAS-N, (d) Ef+FBNet [54, 56], and (e) FastDepth [55]. Figure 10:
Comparison on the ScanNet test set. (a) input image, (b) ground truth, (c)
LiDNAS-S, (d) DAV [28], and (e) SARPN [5].

##  5 Conclusion

This paper proposed a novel NAS framework to construct lightweight monocular
depth estimation architectures using Assisted Tabu Search and employing a
well-defined search space for balancing layer diversity and search volume. The
proposed method achieves competitive accuracy on three benchmark datasets
while running faster on mobile devices and being more compact than state-of-
the-art handcrafted and automatically generated models. Our work provides a
potential approach towards optimizing the accuracy and the network size for
dense depth estimation without the need for manual tweaking of deep neural
architectures.

## References

  * [1] Filippo Aleotti, Giulio Zaccaroni, Luca Bartolomei, Matteo Poggi, Fabio Tosi, and Stefano Mattoccia.  Real-time single image depth perception in the wild with handheld devices.  Sensors, 21(1):15, 2021.
  * [2] Bowen Baker, Otkrist Gupta, Nikhil Naik, and Ramesh Raskar.  Designing neural network architectures using reinforcement learning.  arXiv preprint arXiv:1611.02167, 2016.
  * [3] Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka.  Adabins: Depth estimation using adaptive bins.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4009–4018, 2021.
  * [4] Weifeng Chen, Zhao Fu, Dawei Yang, and Jia Deng.  Single-image depth perception in the wild.  Advances in neural information processing systems, 29:730–738, 2016\.
  * [5] Xiaotian Chen, Xuejin Chen, and Zheng-Jun Zha.  Structure-aware residual pyramid network for monocular depth estimation.  In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 694–700. AAAI Press, 2019.
  * [6] Antonio Cipolletta, Valentino Peluso, Andrea Calimera, Matteo Poggi, Fabio Tosi, Filippo Aleotti, and Stefano Mattoccia.  Energy-quality scalable monocular depth estimation on low-power cpus.  IEEE Internet of Things Journal, 2021.
  * [7] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner.  Scannet: Richly-annotated 3d reconstructions of indoor scenes.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5828–5839, 2017.
  * [8] Jin-Dong Dong, An-Chieh Cheng, Da-Cheng Juan, Wei Wei, and Min Sun.  Dpp-net: Device-aware progressive search for pareto-optimal neural architectures.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 517–531, 2018.
  * [9] David Eigen and Rob Fergus.  Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture.  In Proceedings of the IEEE International Conference on Computer Vision, pages 2650–2658, 2015.
  * [10] David Eigen, Christian Puhrsch, and Rob Fergus.  Depth map prediction from a single image using a multi-scale deep network.  In Advances in neural information processing systems, pages 2366–2374, 2014.
  * [11] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter.  Efficient multi-objective neural architecture search via lamarckian evolution.  arXiv preprint arXiv:1804.09081, 2018.
  * [12] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter.  Multi-objective architecture search for cnns.  arXiv preprint arXiv:1804.09081, 2, 2018.
  * [13] Jose M Facil, Benjamin Ummenhofer, Huizhong Zhou, Luis Montesano, Thomas Brox, and Javier Civera.  Cam-convs: camera-aware multi-scale convolutions for single-view depth.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11826–11835, 2019.
  * [14] Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao.  Deep ordinal regression network for monocular depth estimation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2002–2011, 2018.
  * [15] Ravi Garg, Vijay Kumar Bg, Gustavo Carneiro, and Ian Reid.  Unsupervised cnn for single view depth estimation: Geometry to the rescue.  In European conference on computer vision, pages 740–756. Springer, 2016.
  * [16] Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun.  Vision meets robotics: The kitti dataset.  The International Journal of Robotics Research, 32(11):1231–1237, 2013.
  * [17] Geiger, A. and Nießner, M. and Dai, A.  Robust Vision Challenge CVPR Workshop, 2018.
  * [18] Fred Glover.  Future paths for integer programming and links to artificial intelligence.  Computers & operations research, 13(5):533–549, 1986.
  * [19] Clément Godard, Oisin Mac Aodha, Michael Firman, and Gabriel J Brostow.  Digging into self-supervised monocular depth estimation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3828–3838, 2019.
  * [20] Juan Luis GonzalezBello and Munchurl Kim.  Forget about the lidar: Self-supervised depth estimators with med probability volumes.  Advances in Neural Information Processing Systems, 33, 2020.
  * [21] Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu.  Ghostnet: More features from cheap operations.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1580–1589, 2020.
  * [22] Song Han, Huizi Mao, and William J Dally.  Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.  arXiv preprint arXiv:1510.00149, 2015.
  * [23] Richard Hartley and Andrew Zisserman.  Multiple view geometry in computer vision.  Cambridge university press, 2003.
  * [24] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, et al.  Searching for mobilenetv3.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1314–1324, 2019.
  * [25] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  arXiv preprint arXiv:1704.04861, 2017.
  * [26] Chi-Hung Hsu, Shu-Huan Chang, Jhao-Hong Liang, Hsin-Ping Chou, Chun-Hao Liu, Shih-Chieh Chang, Jia-Yu Pan, Yu-Ting Chen, Wei Wei, and Da-Cheng Juan.  Monas: Multi-objective neural architecture search using reinforcement learning.  arXiv preprint arXiv:1806.10332, 2018.
  * [27] Junjie Hu, Mete Ozay, Yan Zhang, and Takayuki Okatani.  Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries.  In IEEE Winter Conf. on Applications of Computer Vision (WACV), 2019\.
  * [28] Lam Huynh, Phong Nguyen-Ha, Jiri Matas, Esa Rahtu, and Janne Heikkilä.  Guiding monocular depth estimation using depth-attention volume.  In European Conference on Computer Vision, pages 581–597. Springer, 2020.
  * [29] Andrey Ignatov, Grigory Malivenko, David Plowman, Samarth Shukla, Radu Timofte, Ziyu Zhang, Yicheng Wang, Zilong Huang, Guozhong Luo, Gang Yu, et al.  Fast and accurate single-image depth estimation on mobile devices, mobile ai 2021 challenge: Report.  arXiv preprint arXiv:2105.08630, 2021.
  * [30] Jianbo Jiao, Ying Cao, Yibing Song, and Rynson Lau.  Look deeper into depth: Monocular depth estimation with semantic booster and attention-driven loss.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 53–69, 2018.
  * [31] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [32] Iro Laina, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab.  Deeper depth prediction with fully convolutional residual networks.  In 2016 Fourth International Conference on 3D Vision (3DV), pages 239–248. IEEE, 2016.
  * [33] Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh.  From big to small: Multi-scale local planar guidance for monocular depth estimation.  arXiv preprint arXiv:1907.10326, 2019.
  * [34] Jae-Han Lee and Chang-Su Kim.  Monocular depth estimation using relative depth maps.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9729–9738, 2019.
  * [35] Chen Liu, Kihwan Kim, Jinwei Gu, Yasutaka Furukawa, and Jan Kautz.  Planercnn: 3d plane detection and reconstruction from a single image.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4450–4459, 2019.
  * [36] Chen Liu, Jimei Yang, Duygu Ceylan, Ersin Yumer, and Yasutaka Furukawa.  Planenet: Piece-wise planar reconstruction from a single rgb image.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2579–2588, 2018.
  * [37] Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy.  Progressive neural architecture search.  In Proceedings of the European conference on computer vision (ECCV), pages 19–34, 2018.
  * [38] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  Darts: Differentiable architecture search.  arXiv preprint arXiv:1806.09055, 2018.
  * [39] Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu.  Neural architecture optimization.  arXiv preprint arXiv:1808.07233, 2018.
  * [40] Joe Mellor, Jack Turner, Amos Storkey, and Elliot J Crowley.  Neural architecture search without training.  In International Conference on Machine Learning, pages 7588–7598. PMLR, 2021.
  * [41] Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean.  Efficient neural architecture search via parameters sharing.  In International Conference on Machine Learning, pages 4095–4104. PMLR, 2018.
  * [42] Matteo Poggi, Filippo Aleotti, Fabio Tosi, and Stefano Mattoccia.  Towards real-time unsupervised monocular depth estimation on cpu.  In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5848–5854. IEEE, 2018.
  * [43] Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, and Jiaya Jia.  Geonet: Geometric neural network for joint depth and surface normal estimation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 283–291, 2018.
  * [44] Michael Ramamonjisoa and Vincent Lepetit.  Sharpnet: Fast and accurate recovery of occluding contours in monocular depth estimation.  The IEEE International Conference on Computer Vision (ICCV) Workshops, 2019.
  * [45] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun.  Vision transformers for dense prediction.  ArXiv preprint, 2021.
  * [46] Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le.  Regularized evolution for image classifier architecture search.  In Proceedings of the aaai conference on artificial intelligence, volume 33, pages 4780–4789, 2019.
  * [47] Haoyu Ren, Mostafa El-khamy, and Jungwon Lee.  Deep robust single image depth estimation neural network using scene understanding.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 37–45, 2019.
  * [48] Tonmoy Saikia, Yassine Marrakchi, Arber Zela, Frank Hutter, and Thomas Brox.  Autodispnet: Improving disparity estimation with automl.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1812–1823, 2019.
  * [49] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018.
  * [50] Ashutosh Saxena, Sung H Chung, and Andrew Y Ng.  Learning depth from single monocular images.  In Advances in neural information processing systems, pages 1161–1168, 2006.
  * [51] Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus.  Indoor segmentation and support inference from rgbd images.  In European Conference on Computer Vision, pages 746–760. Springer, 2012.
  * [52] Richard Szeliski.  Structure from motion.  In Computer Vision, pages 303–334. Springer, 2011.
  * [53] Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, and Quoc V Le.  Mnasnet: Platform-aware neural architecture search for mobile.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2820–2828, 2019.
  * [54] Xiaohan Tu, Cheng Xu, Siping Liu, Renfa Li, Guoqi Xie, Jing Huang, and Laurence Tianruo Yang.  Efficient monocular depth estimation for edge devices in internet of things.  IEEE Transactions on Industrial Informatics, 17(4):2821–2832, 2020\.
  * [55] Diana Wofk, Fangchang Ma, Tien-Ju Yang, Sertac Karaman, and Vivienne Sze.  Fastdepth: Fast monocular depth estimation on embedded systems.  In 2019 International Conference on Robotics and Automation (ICRA), pages 6101–6108. IEEE, 2019.
  * [56] Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Peter Vajda, Yangqing Jia, and Kurt Keutzer.  Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10734–10742, 2019.
  * [57] Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe, and Elisa Ricci.  Transformers solve the limited receptive field for monocular depth prediction.  arXiv preprint arXiv:2103.12091, 2021.
  * [58] Tien-Ju Yang, Andrew Howard, Bo Chen, Xiao Zhang, Alec Go, Mark Sandler, Vivienne Sze, and Hartwig Adam.  Netadapt: Platform-aware neural network adaptation for mobile applications.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 285–300, 2018.
  * [59] Wei Yin, Yifan Liu, Chunhua Shen, and Youliang Yan.  Enforcing geometric constraints of virtual normal for depth prediction.  In The IEEE International Conference on Computer Vision (ICCV), 2019\.
  * [60] Mehmet Kerim Yucel, Valia Dimaridou, Anastasios Drosou, and Albert Saa-Garriga.  Real-time monocular depth estimation with sparse supervision on mobile.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2428–2437, 2021.
  * [61] Barret Zoph and Quoc V Le.  Neural architecture search with reinforcement learning.  arXiv preprint arXiv:1611.01578, 2016.
  * [62] Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le.  Learning transferable architectures for scalable image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8697–8710, 2018.


# A Study of monogenity of Binomial Composition

Anuj Jakhar  ,  Ravi Kalwaniya  and  Prabhakar yadav  Department of
Mathematics, Indian Institute of Technology (IIT) Madras  Indian Statistical
Institute, New Delhi<EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

Let $\theta$ be a root of a monic polynomial $h(x)\in\mbox{$\mathbb{Z}$}[x]$
of degree $n\geq 2$. We say $h(x)$ is monogenic if it is irreducible over
$\mathbb{Q}$ and $\\{1,\theta,\theta^{2},\ldots,\theta^{n-1}\\}$ is a basis
for the ring $\mbox{$\mathbb{Z}$}_{K}$ of integers of $K=\mathbb{Q}(\theta)$.
In this article, we study about the monogenity of number fields generated by a
root of composition of two binomials. We characterise all the primes dividing
the index of the subgroup $\mbox{$\mathbb{Z}$}[\theta]$ in
$\mbox{$\mathbb{Z}$}_{K}$ where $K=\mathbb{Q}(\theta)$ with $\theta$ having
minimal polynomial $F(x)=(x^{m}-b)^{n}-a\in\mbox{$\mathbb{Z}$}[x]$, $m\geq 1$
and $n\geq 2$. As an application, we provide a class of pairs of binomials
$f(x)=x^{n}-a$ and $g(x)=x^{m}-b$ having the property that both $f(x)$ and
$f(g(x))$ are monogenic.

###### Key words and phrases:

Rings of algebraic integers; Index of an algebraic integer; Power basis.

###### 2010 Mathematics Subject Classification:

11R04; 11R29, 11Y40.

##  1\. Introduction and statements of results

Let $K=\mathbb{Q}(\theta)$ be an algebraic number field with $\theta$ in the
ring $\mbox{$\mathbb{Z}$}_{K}$ of algebraic integers of $K$ and $h(x)$ having
degree $n$ be the minimal polynomial of $\theta$ over the field $\mathbb{Q}$
of rational numbers. Let $d_{K}$ denote the discriminant of $K$ and $D_{h}$
denote the discriminant of the polynomial $h(x)$. It is well-known that
$d_{K}$ and $D_{h}$ are related by the following formula

$D_{h}=[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]^{2}d_{K}.$

We say that $h(x)$ is monogenic if
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$, or equivalently, if
$D_{h}=d_{K}$. In this case, $\\{1,\theta,\cdots,\theta^{n-1}\\}$ will be an
integral basis of $K$ and $K$ will be a monogenic number field. A number field
$K$ is called monogenic if there exists some
$\alpha\in\mbox{$\mathbb{Z}$}_{K}$ such that
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\alpha]$. Note that the
monogenity of a polynomial $h(x)$ implies that $K=\mathbb{Q}(\theta)$ is
monogenic where $\theta$ is a root of $h(x)$, but the converse is not true in
general. For example, let $\alpha$ and $\beta$ be the roots of
$h_{1}(x)=x^{2}-5$ and $h_{2}(x)=x^{2}-x-1$ respectively, then
$\mathbb{Q}(\alpha)=\mathbb{Q}(\beta)$. Although $h_{2}(x)$ is monogenic but
$h_{1}(x)$ is not monogenic.

The determination of monogenity of an algebraic number field is one of the
classical and important problems in algebraic number theory. An arithmetic
characterisation of monogenic number fields is a problem due to Hasse (cf.
[7]). Gaál’s book provides some classification of monogenity in lower degree
number fields (cf. [4]). In 2016, Jhorar and Khanduja gave necessary and
sufficient conditions for
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$ when $\theta$ is a root
of an irreducible binomial $x^{n}-b\in\mbox{$\mathbb{Z}$}[x]$ (cf. [10,
Theorem 1.3]. In fact, they proved the following result.

######  Theorem 1.1.

Let $K=\mathbb{Q}(\theta)$ be a Kummer extension of $\mathbb{Q}$ with $\theta$
satisfying an irreducible polynomial $x^{n}-b$ over $\mathbb{Z}$. Then the
following statements are equivalent:

  * (i)

$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$.

  * (ii)

$b$ is square-free integer and whenever a prime $p$ divides $n$, then $p^{2}$
does not divide $b^{p}-b$.

Using Dedekind’s Index Criterion, in 2016-17, Jakhar, Khanduja and Sangwan [9]
extended the above result and provided necessary and sufficient conditions for
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$ when $\theta$ is a root
of an irreducible trinomial $x^{n}+ax^{m}+b\in\mbox{$\mathbb{Z}$}[x]$ having
degree $n$. As an application, they provided infinitely many monogenic
trinomials. Further in 2020, Jones and Harrington [6] investigated few pairs
of binomials $f(x)=x^{n}-a$ and $g(x)=x^{m}-b$ having the property that both
$f(x)$ and $f(g(x))$ are monogenic. In fact, they provided necessary and
sufficient conditions for the monogenicity of the polynomial $(x^{m}-b)^{n}-a$
with $n=2,3.$ In 2022, Gaál [5] described monogenity properties of a class of
binomial compositions of degree six.

Let $K=\mathbb{Q}(\theta)$ be an algebraic number field where $\theta$ is a
root of an irreducible polynomial $f(g(x))=(x^{m}-b)^{n}-a$ over $\mathbb{Q}$.
In this article, we characterise all the primes dividing the index
$[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$. As an application, we
provide necessary and sufficient conditions for
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta].$ Moreover, we also give
class of binomials $f(x)$ and $g(x)$ such that both $f(x)$, $f(g(x))$ are
monogenic.

Throughout the paper, $f(x)=x^{n}-a$ and $g(x)=x^{m}-b$ will be polynomials in
$\mbox{$\mathbb{Z}$}[x]$ having degree $n\geq 2$ and $m\geq 1$, respectively.
$D_{F}$ will stand for the discriminant of an irreducible polynomial
$F(x)=f(g(x))=(x^{m}-b)^{n}-a$. Although the formula for $|D_{F}|$ is given in
[6, Lemma 3.1], for the sake of completion we provide a quick derivation of
$D_{F}$ in Lemma 2.3 using a different approach. We prove that $D_{F}$ is
given by

$D_{F}=(-1)^{\frac{n(n-1)}{2}+m}(mn)^{mn}a^{m(n-1)}((-b)^{n}-a)^{m-1}.$ (1.1)

For an integer $z$, $\operatorname{rad}(z)$ will denote the product of
distinct primes dividing $z$.

We wish to point out here that in view of Theorem 1.1 we can consider $n\geq
2$ while studying the monogenity of the polynomial $f(g(x))$, the composition
of two polynomials $f(x)=x^{n}-a$ and $g(x)=x^{m}-b$.

Precisely stated, we prove the following result.

######  Theorem 1.2.

Let $K=\mathbb{\mathbb{Q}}(\theta)$ be an algebraic number field with $\theta$
in the ring $\mbox{$\mathbb{Z}$}_{K}$ of algebraic integers of $K$ having
minimal polynomial $F(x)=(x^{m}-b)^{n}-a$ over $\mathbb{Q}$ with $m\geq 1$ and
$n\geq 2$. A prime factor $p$ of the discriminant $D_{F}$ of $F(x)$ does not
divide $[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
$p$ satisfies one of the following conditions:

  1. (i)

when $p\mid a$, then $p^{2}\nmid a$.

  2. (ii)

when $p\nmid a$ and $p\mid b$ with $m=p^{j}s,~{}n=p^{k}s^{\prime},~{}j+k\geq
1$ and $p\nmid ss^{\prime}$, then the polynomials
$\frac{1}{p}[a^{p^{j+k}}-a-nbg(x)^{n-1}]$ and $x^{ss^{\prime}}-a$ are coprime
$modulo~{}p$.

  3. (iii)

when $p\nmid ab$ and $p|n$ with $k\geq 1$ as the highest power of $p$ dividing
$n$, then $p^{2}\nmid(a^{p^{k}}-a)$.

  4. (iv)

when $p\nmid abn$ and $p\mid m$ with $m=p^{j}s$, then $(x^{s}-b)^{n}-a$ must
be coprime to $\frac{1}{p}\left[a^{p^{j}}-a+n\sum_{i=1}^{p-1}{p^{j}\choose
ip^{j-1}}(x^{s}-b)^{np^{j}-ip^{j-1}}b^{i}+n(b^{p^{j}}-b)\right]$.

  5. (v)

when $p\nmid abmn$, then $p^{2}\nmid((-b)^{n}-a)$.

In the special case, the above theorem quickly yields the following simple
result regarding monogenic polynomials.

######  Corollary 1.3.

Let $K=\mathbb{Q}(\theta)$ and $F(x)=(x^{m}-b)^{n}-a$ be as in Theorem 1.2.
Assume that $\operatorname{rad}(mn)$ divides $\operatorname{rad}(a)$. Then
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$ if and only if both $a$
and $(-b)^{n}-a$ are square-free.

######  Remark 1.4.

Note that if $p$ is prime number such that $p\nmid a$ with
$a\in\mbox{$\mathbb{Z}$}$, then for any positive integer $s$ the exact power
of $p$ dividing $a^{p^{s}-1}-1$ will be same as the exact power of $p$
dividing $a^{p-1}-1$; this can be easily verified keeping in mind that
$p^{s}-1=(p-1)m$ with $m\equiv~{}1~{}(mod~{}p)$ and $a^{p-1}\equiv
1~{}(mod~{}p).$

Keeping in mind the above remark, the following corollary is an immediate
consequence of Theorem 1.2.

######  Corollary 1.5.

Let $K=\mathbb{Q}(\theta)$ with $\theta$ having minimal polynomial
$F(x)=(x^{m}-b)^{n}-a\in\mbox{$\mathbb{Z}$}[x]$ with $m\geq 1,n\geq 2$. Let
$\operatorname{rad}(m)$ divide $\operatorname{rad}(an)$. Then
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$ if and only if each
prime $p$ dividing $D_{F}$ satisfies one of the following:

  1. (i)

when $p\mid a$, then $p^{2}\nmid a$.

  2. (ii)

when $p\nmid a$, then $p^{2}\nmid(a^{p}-a)$.

  3. (iii)

when $p\nmid abmn$, then $p^{2}\nmid((-b)^{n}-a)$.

Using Theorem 1.1 and the above corollary, we obtain the following result
which extends [6, Theorem 1.3] as well as it provides necessary and sufficient
conditions for the monogenity of both $f(x)$ and $f(g(x))$.

######  Corollary 1.6.

Let $f(x)=x^{n}-a$ and $F(x)=(x^{m}-b)^{n}-a\in\mbox{$\mathbb{Z}$}[x]$ with
$m\geq 1,~{}n\geq 2$ and $\operatorname{rad}(m)$ divide
$\operatorname{rad}(an)$. Suppose $f(x)$ and $F(x)$ are irreducible. Then
$f(x)$ and $F(x)$ are monogenic if and only if the following hold:

  1. (i)

$a$ is squarefree.

  2. (ii)

$a^{p}\not\equiv a$ (mod $p^{2}$) for all primes $p$ dividing $n$.

  3. (iii)

For all primes $p$ dividing $((-b)^{n}-a)$ with $p\nmid abn$, we have
$p^{2}\nmid((-b)^{n}-a)$.

The following example is an application of Corollary 1.3. In this example,
$K=\mathbb{Q}(\theta)$ with $\theta$ a root of $F(x)$.

######  Example 1.7.

Let $p$ be a prime number. Take $m=n=a=p$ and $b=2p$ in Corollary 1.3. So we
have $F(x)=(x^{p}-2p)^{p}-p$. Note that
$|D_{F}|=p^{3p^{2}-p}((-2p)^{p}-p)^{p-1}$. Therefore in view of Corollary 1.3,
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$ if and only if
$(-2p)^{p}-p$ is square-free. It can be easily checked that $(-2p)^{p}-p$ is
square-free for $p<43$ except for $p=11,29$. Hence
$\mbox{$\mathbb{Z}$}_{K}=\mbox{$\mathbb{Z}$}[\theta]$ for $p<43$ except
$p=11,29$. Here, all computations for checking the factorisation of
discriminant were done using sage.

##  2\. Preliminaries

Throughout the paper, $p$ will denote a prime number and for a polynomial
$h(x)$ belonging to $\mbox{$\mathbb{Z}$}[x]$, we shall denote by $\bar{h}(x)$
the polynomial over $\mbox{$\mathbb{Z}$}/p\mbox{$\mathbb{Z}$}$ obtained on
replacing each coefficient of $h(x)$ modulo $p$.

We now state the following two results (see [1, Lemma 2.6] for Lemma 2.1 and
[1, Theorem 1.19] for Lemma 2.2) regarding Discriminant and Norm which will be
used for obtaining the formula for the discriminant of composition of two
binomials.

######  Lemma 2.1.

Let $f(x)\in\mbox{$\mathbb{Z}$}[x]$ be monic and irreducible with $\deg(f)=n.$
Let $\theta$ be a root of $f(x)$ and $K=\mathbb{Q}(\theta).$ Then

$D_{f}=(-1)^{n(n-1)/2}~{}\mathcal{N}_{K/\mathbb{Q}}(f^{\prime}(\theta)).$

######  Lemma 2.2.

Let $K/F$ be an extension of degree $n$ and $\alpha$ be an element of $K$ with
$[F(\alpha):F]=d$. Then

$\mathcal{N}_{K/F}(\alpha)=\left(\mathcal{N}_{F(\alpha)/F}(\alpha)\right)^{n/d}.$

######  Lemma 2.3.

Let $f(x)=x^{n}-a$ and $g(x)=x^{m}-b$ be polynomials in
$\mbox{$\mathbb{Z}$}[x]$. Let $F(x)=f(g(x))$ be an irreducible polynomial,
then the discriminant $D_{F}$ of $F$ is given by

$D_{F}=(-1)^{\frac{n(n-1)}{2}+m}(nm)^{nm}a^{m(n-1)}((-b)^{n}-a)^{m-1}.$ (2.1)

###### Proof.

Let $\theta$ be a root of $F(x)$. Since
$F^{\prime}(x)=nmx^{m-1}(x^{m}-b)^{n-1}$, we have

$F^{\prime}(\theta)=nm\theta^{m-1}(\theta^{m}-b)^{n-1}.$ (2.2)

We simply write $\mathcal{N}$ for the norm $\mathcal{N}_{K/\mathbb{Q}}$, where
$K=\mathbb{Q}(\theta)$. Since $\mathcal{N}(\theta)=(-1)^{mn}((-b)^{n}-a)$ and
$\mathcal{N}(mn)=(mn)^{mn}$, taking norm both sides of $\eqref{2.1}$, we
obtain

$\mathcal{N}(F^{\prime}(\theta))=(-1)^{mn}(mn)^{mn}\mathcal{N}(\theta^{m}-b)^{n-1}((-b)^{n}-a)^{m-1}.$
(2.3)

To calculate $\mathcal{N}(\theta^{m}-b)$, let $\theta^{m}-b=z$. Since
$F(\theta)=0$, we obtain

$\displaystyle F(\theta)=(\theta^{m}-b)^{n}-a=z^{n}-a=0.$

Therefore, we see that $z$ is root of the polynomial $h(x)=x^{n}-a$. We claim
that $h(x)$ is a minimal polynomial for $z$. Keeping in mind that
$\mathbb{Q}(z)=\mathbb{Q}(\theta^{m})$, for proving our claim, it is
sufficient to show that $h(x)$ is irreducible polynomial over $\mathbb{Q}.$
Suppose that $h(x)=h_{1}(x)h_{2}(x)$ is the factorisation of $h(x)$ in
$\mbox{$\mathbb{Z}$}[x]$ with $\deg h_{1}(x),\deg h_{2}(x)\geq 1$. Clearly
$h(x^{m}-b)=F(x)$. Using this we obtain
$F(x)=h_{1}(x^{m}-b)~{}h_{2}(x^{m}-b)$, which contradicts the irreducibility
of $F(x)$. Hence $h(x)$ is irreducible. This proves our claim. So we have
$\mathcal{N}_{\mathbb{Q}(z)/\mathbb{Q}}(z)=(-1)^{n}(-a)$. Using Lemma
$\eqref{extn}$, we obtain

$\mathcal{N}(z)=(-1)^{m(n+1)}a^{m}.$ (2.4)

Therefore, the theorem follows from Lemma 2.1 and Equations $\eqref{2.3}$ and
$\eqref{3.10}$. ∎

The following well known lemma will be used in the sequel. The equivalence of
assertions $(i)$ and $(ii)$ of the theorem was proved by Dedekind (cf. [2,
Theorem 6.1.4], [3]). A simple proof of the equivalence of $(ii)$ and $(iii)$
is given in [8, Lemma 2.1].

######  Lemma 2.4.

Let $f(x)\in\mbox{$\mathbb{Z}$}[x]$ be a monic irreducible polynomial having
the factorization $\bar{g}_{1}(x)^{e_{1}}\cdots\bar{g}_{t}(x)^{e_{t}}$ modulo
a prime $p$ as a product of powers of distinct irreducible polynomials over
$\mbox{$\mathbb{Z}$}/p\mbox{$\mathbb{Z}$}$ with
$g_{i}(x)\in\mbox{$\mathbb{Z}$}[x]$ monic. Let $K=\mathbb{Q}(\theta)$ with
$\theta$ a root of $f(x)$. Then the following statements are equivalent:

  * (i)

$p$ does not divide $[\mbox{$\mathbb{Z}$}_{k}:\mbox{$\mathbb{Z}$}[\theta]]$.

  * (ii)

For each $i$, we have either $e_{i}=1$ or $\overline{g}_{i}(x)$ does not
divide $\overline{M}(x)$ where $M(x)=\frac{1}{p}(f(x)-g_{1}(x)^{e_{1}}\cdots
g_{t}(x)^{e_{t}})$.

  * (iii)

$f(x)$ does not belong to the ideal $\langle p,g_{i}(x)\rangle^{2}$ in
$\mbox{$\mathbb{Z}$}[x]$ for any $i$, $1\leq i\leq t$.

##  3\. Proof of Theorem 1.2

###### Proof of Theorem 1.2.

Let $p$ be a prime number dividing $D_{F}$. In view of Lemma 2.4, $p$ does not
divide $[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
$f(x)\not\in\langle p,g(x)\rangle^{2}$ for any monic polynomial
$g(x)\in\mbox{$\mathbb{Z}$}[x]$ such that $\bar{g}(x)$ divides $\bar{f}(x)$
and $g(x)$ irreducible modulo $p$. Note that $f(x)\not\in\langle
p,g(x)\rangle^{2}$ if $\bar{g}(x)$ is not a repeated factor of
$\overline{f}(x)$. We prove the theorem case by case.

Case (i). Suppose $p\mid a$. In this case, we have either $p\mid b$ or $p\nmid
b$. Consider the first possibility when $p\mid b$, then $F(x)\equiv
x^{mn}~{}(mod~{}p)$. Clearly $F(x)\in\langle p,x\rangle^{2}$ if and only if
$p^{2}$ divides $a$, consequently
$p\nmid[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
$p^{2}\nmid a$. We now consider the second possibility when $p\nmid b$. In
this situation, ${F}(x)\equiv(x^{m}-{b})^{n}$ (mod $p$). Let $m=p^{j}s$ with
$j\geq 0$. In view of Binomial theorem, we obtain

$F(x)\equiv(x^{m}-b)^{n}\equiv(x^{s}-b)^{np^{j}}~{}(mod~{}p).$ (3.1)

Let $\bar{g}_{1}(x)\cdots\bar{g}_{t}(x)$ be the factorization of
$x^{s}-\overline{b}$ over $\mbox{$\mathbb{Z}$}/p\mbox{$\mathbb{Z}$}$, where
$g_{i}(x)\in\mbox{$\mathbb{Z}$}[x]$ are monic polynomials which are distinct
and irreducible modulo $p$. Write $x^{s}-b$ as $g_{1}(x)\cdots g_{t}(x)+pH(x)$
for some polynomial $H(x)\in\mbox{$\mathbb{Z}$}[x]$. Using Binomial theorem,
one can easily check that

$h(x^{p^{j}})=h(x)^{p^{j}}+ph(x)T(x)+b^{p^{j}}-b$

for some polynomial $T(x)\in\mbox{$\mathbb{Z}$}[x]$. Therefore, keeping in
view that $F(x)=(h(x^{p^{j}}))^{n}-a$, it follows that

$F(x)=(h(x^{p^{j}}))^{n}-a=(h(x)^{p^{j}}+ph(x)T(x)+b^{p^{j}}-b)^{n}-a.$ (3.2)

Using again Binomial theorem and the fact $p$ divides $b^{p^{j}}-b$ in
Equation $\eqref{eq;2.2}$, we see that

$F(x)=(h(x)^{p^{j}})^{n}+ph(x)M_{1}(x)+p^{2}M_{2}(x)+(b^{p^{j}}-b)^{n}-a,$
(3.3)

for some polynomials $M_{1}(x),M_{2}(x)\in\mbox{$\mathbb{Z}$}[x]$. Since
$n\geq 2$, the first four terms on the right hand side of $\eqref{eq;2.31}$
belong to $\langle p,g_{i}(x)\rangle^{2}$ for each $i$, $1\leq i\leq t$. So
$F(x)\in\langle p,g_{i}(x)\rangle^{2}$ for some $i$, $1\leq i\leq t$ if and
only if $p^{2}\mid a$. Therefore, by Lemma 2.4
$p\nmid[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
$p^{2}\nmid a$. This proves the theorem in Case (i).

Case (ii). Suppose $p\nmid a$ and $p\mid b$. Since $p\mid D_{F}$, we conclude
that $p$ divides $mn$ in view of $(\ref{eq:1.1})$. Write $n=s^{\prime}p^{k}$,
$m=sp^{j}$ where $p\nmid{ss^{\prime}},~{}j\geq 0,k\geq 0$ along with $j+k\geq
1$. In view of Binomial theorem, we obtain

$F(x)\equiv(x^{m}-b)^{n}-a\equiv(x^{ss^{\prime}}-a)^{p^{j+k}}(mod~{}p).$ (3.4)

Let $\bar{g}_{1}(x)\cdots\bar{g}_{t}(x)$ be the factorization of
$x^{ss^{\prime}}-\overline{a}$ over
$\mbox{$\mathbb{Z}$}/p\mbox{$\mathbb{Z}$}$, where
$g_{i}(x)\in\mbox{$\mathbb{Z}$}[x]$ are monic polynomials which are distinct
and irreducible modulo $p$. Denote $h(x)=x^{ss^{\prime}}-a$ and write $h(x)$
as $g_{1}(x)\cdots g_{t}(x)+pH_{1}(x)$ for some polynomial
$H_{1}(x)\in\mbox{$\mathbb{Z}$}[x]$. Using Binomial theorem for
$(x^{m}-b+b)^{n}=(h(x)+a)^{p^{j+k}}$, we see that

$(x^{m}-b)^{n}=(h(x)+a)^{p^{j+k}}-\left(\sum\limits_{i=1}^{n}\binom{n}{i}(x^{m}-b)^{n-i}b^{i}\right)$

and hence using $p\mid b$, we have

$F(x)=(x^{m}-b)^{n}-a=h(x)^{p^{j+k}}+ph(x)M_{1}(x)+p^{2}M_{2}(x)+a^{p^{j+k}}-n(x^{m}-b)^{n-1}b-a$
(3.5)

for some polynomials $M_{1}(x),M_{2}(x)\in\mbox{$\mathbb{Z}$}[x]$. The first
three summands on the right hand side of $\eqref{3.5}$ belong to $\langle
p,g_{i}(x)\rangle^{2}$ for each $i$, $1\leq i\leq t$. So $F(x)\in\langle
p,g_{i}(x)\rangle^{2}$ for some $i$, $1\leq i\leq t$, if and only if the
polynomials $\frac{1}{p}[a^{p^{j+k}}-a-ng(x)^{n-1}b]$ and $x^{ss^{\prime}}-a$
have common root. By Lemma 2.4,
$p\nmid[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
the polynomials $\frac{1}{p}[a^{p^{j+k}}-a-ng(x)^{n-1}b]$ and
$x^{ss^{\prime}}-a$ are coprime modulo $p$. This proves the theorem in Case
$(ii)$. Here, one can observe that if $p\mid n$, then
$p\nmid[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
$p^{2}\nmid(a^{p^{j+k}}-a)$.

Case (iii). Suppose $p\nmid ab$ and $p|n$. In this case, we have
$\overline{F}(x)=(x^{m}-\bar{b})^{n}-\overline{a}$ (mod $p$). Write $m=p^{j}s$
and $n=p^{k}s^{\prime}$ with $p\nmid ss^{\prime},~{}j\geq 0,k\geq 1$. Then
$\overline{F}(x)=((x^{s}-b)^{s^{\prime}}-a)^{p^{j+k}}$. Denote
$(x^{s}-b)^{s^{\prime}}-a$ by $h(x)$ so that
$F(x)\equiv(x^{p^{j}s}-b)^{p^{k}s^{\prime}}-a\equiv h(x)^{p^{j+k}}$ (mod $p$).
Let $h(x)\equiv g_{1}(x)\cdots g_{t}(x)$ (mod $p$) be the factorization of
$h(x)$ into the product of irreducible polynomials modulo $p$ with each
$g_{i}(x)\in\mbox{$\mathbb{Z}$}[x]$ monic. Keeping in mind that
$h(x)=(x^{s}-b)^{s^{\prime}}-a$ and
$h(x^{p^{j}})\equiv{h}(x)^{p^{j}}(mod~{}p)$, we see that

$\displaystyle F(x)$ $\displaystyle=(h(x^{p^{j}})+a)^{p^{k}}-a$
$\displaystyle=(h(x)^{p^{j}}+pM_{1}(x)+a)^{p^{k}}-a,$

for some $M_{1}(x)\in\mbox{$\mathbb{Z}$}[x]$. As $k$ is a positive integer, in
view of Binomial theorem we have
$F(x)=h(x)^{p^{j+k}}+ph(x)M_{2}(x)+p^{2}M_{3}(x)+a^{p^{k}}-a$ for some
$M_{2}(x),M_{3}(x)\in\mbox{$\mathbb{Z}$}[x]$. Using
$h(x)=\prod\limits_{i=1}^{t}g_{i}(x)+pH(x)$ where
$H(x)\in\mbox{$\mathbb{Z}$}[x]$, we obtain

$\displaystyle F(x)$
$\displaystyle=\left(\prod_{i=1}^{t}g_{i}(x)+pH(x)\right)^{p^{j+k}}+p\left(\prod_{i=1}^{t}g_{i}(x)+pH(x)\right)M_{2}(x)+p^{2}M_{3}(x)+a^{p^{k}}-a$
$\displaystyle=\left(\prod_{i=1}^{t}g_{i}(x)\right)^{p^{j+k}}+p\prod_{i=1}^{t}g_{i}(x)M_{4}(x)+p^{2}M_{5}(x)+a^{p^{k}}-a,$

for some polynomials $M_{4}(x),M_{5}(x)\in\mbox{$\mathbb{Z}$}[x]$. Then
$F(x)\in\langle p,g_{i}(x)\rangle^{2}$ for some $i,~{}1\leq i\leq t$ if and
only if $p^{2}\mid(a^{p^{k}}-a)$. By Lemma 2.4, $p$ does not divide
$[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
$p^{2}\nmid(a^{p^{k}}-a)$. This proves the theorem in the present case.

Case (iv). Suppose $p\nmid abn$ and $p\mid m$. In this case, we have
$\overline{F}(x)=(x^{m}-\bar{b})^{n}-\overline{a}$. Write $m=p^{j}s$ with
$p\nmid s$ and $j\geq 1$. Then ${F}(x)=((x^{s}-b)^{n}-a)^{p^{j}}$ (mod $p$).
Denote $(x^{s}-b)^{n}-a$ by $h(x)$ so that
$F(x)=h(x^{p^{j}})=(x^{p^{j}s}-b)^{n}-a$. Clearly $F(x)\equiv h(x)^{p^{j}}$
(mod $p$). Let $h(x)\equiv g_{1}(x)\cdots g_{t}(x)$ (mod $p$) be the
factorization of $h(x)$ into a product of irreducible polynomials modulo $p$
with each $g_{i}(x)\in\mbox{$\mathbb{Z}$}[x]$ monic. In view of Binomial
theorem, we have

$\displaystyle F(x)$ $\displaystyle=((x^{s}-b+b)^{p^{j}}-b)^{n}-a$
$\displaystyle=\left((x^{s}-b)^{p^{j}}+\sum_{i=1}^{p^{j}-1}{p^{j}\choose
i}(x^{s}-b)^{p^{j}-i}b^{i}+b^{p^{j}}-b\right)^{n}-a$
$\displaystyle=(x^{s}-b)^{np^{j}}+n(x^{s}-b)^{(n-1)p^{j}}\left(\sum_{i=1}^{p^{j}-1}{p^{j}\choose
i}(x^{s}-b)^{p^{j}-i}b^{i}+b^{p^{j}}-b\right)+p^{2}N_{1}(x)-a$
$\displaystyle=(x^{s}-b)^{np^{j}}+n\left(\sum_{i=1}^{p^{j}-1}{p^{j}\choose
i}(x^{s}-b)^{np^{j}-i}b^{i}+b^{p^{j}}-b\right)+p^{2}N_{1}(x)-a$
$\displaystyle=(h(x)+a)^{p^{j}}+n\left(\sum_{i=1}^{p^{j}-1}{p^{j}\choose
i}(x^{s}-b)^{np^{j}-i}b^{i}+b^{p^{j}}-b\right)+p^{2}N_{1}(x)-a$

for some polynomial $N_{1}(x)\in\mbox{$\mathbb{Z}$}[x]$. Now using
$h(x)=(x^{s}-b)^{n}-a$ and noting that $\nu_{p}({p^{j}\choose
i})=j-\nu_{p}(i)$, we obtain

$\displaystyle F(x)=$
$\displaystyle(h(x))^{p^{j}}+ph(x)N_{2}(x)+a^{p^{j}}+p^{2}N_{3}(x)$
$\displaystyle+n\left(\sum_{i=1}^{p-1}{p^{j}\choose
ip^{j-1}}(x^{s}-b)^{np^{j}-ip^{j-1}}b^{i}+b^{p^{j}}-b\right)-a$

for some $N_{2}(x),N_{3}(x)\in\mbox{$\mathbb{Z}$}[x]$. Our desired result now
follows in view of Lemma 2.4 $(ii)$.

Case (v). Now consider the last case when $p\nmid abmn$. Since $p|D_{F}$, we
have $m\geq 2$ by virtue of (1.1). Let $\beta$ be a repeated root of
$\overline{F}(x)=(x^{m}-\bar{b})^{n}-\bar{a}$ in the algebraic closure of
$\mbox{$\mathbb{Z}$}/p\mbox{$\mathbb{Z}$}$. Then

$\overline{F}(\beta)=(\beta^{m}-\bar{b})^{n}-\bar{a}=\bar{0};~{}~{}~{}\overline{F}^{\prime}(\beta)=\bar{n}(\beta^{m}-\bar{b})^{n-1}\bar{m}\beta^{m-1}=\bar{0}.$
(3.6)

Observe that $(\beta^{m}-b)^{n-1}\not\equiv 0$ (mod $p$), otherwise in view of
the first equation of (3.6) $-\bar{a}=\bar{0}$ which is not possible as
$p\nmid a$. Therefore, keeping in mind that $p\nmid mn$, it follows that
$\beta=\bar{0}$ is the unique repeated root of $\bar{F}(x)$ in
$\mbox{$\mathbb{Z}$}/p\mbox{$\mathbb{Z}$}$. Also, as $m\geq 2$, observe that
$F(x)\in\langle p,x\rangle^{2}$ if and only if $p^{2}\mid((-b)^{n}-a)$. Hence,
by Lemma 2.4, $p$ does not divide
$[\mbox{$\mathbb{Z}$}_{K}:\mbox{$\mathbb{Z}$}[\theta]]$ if and only if
$p^{2}\nmid((-b)^{n}-a)$. This completes the proof of the theorem. ∎

## References

  * [1] S. K. Khanduja, A textbook of algebraic number theory, Unitext series, Volume 135 (Springer, 2022).
  * [2] H. Cohen, A Course in Computational Algebraic Number Theory, Springer- Verlag, Berlin-Heidelberg, 1993.
  * [3] R. Dedekind, Über den Zusammenhang zwischen der Theorie der Ideale und der Theorie der höheren Kongruenzen, Götttingen Abh., 23 (1878) 1-23.
  * [4] I. Gaál, Diophantine equations and power integral bases: Theory and Algorithms, 2nd Edi., Birkhäuser/Springer, Cham (2019).
  * [5] I. Gaál,, On the monogenity of certain binomial compositions, JP J. Algebra Number Theory Appl., 57 (2022) 1-16.
  * [6] J. Harrington and L. Jones, Monogenic Binomial Composition, Taiwanese J. Math., 24 (2020) 1073-1090.
  * [7] H. Hasse, Zahlentheorie, Akademie-Verlag, Berlin, 1963.
  * [8] A. Jakhar, S. K. Khanduja, N. Sangwan, On prime divisors of the index of an algebraic integer, J. Number Theory, 166 (2016) 47-61.
  * [9] A. Jakhar, S. K. Khanduja, N. Sangwan, Characterisation of primes dividing the index of a trinomial, Int. J. Number Theory, 13(10) (2017) 2505-2514.
  * [10] B. Jhorar, S. K. Khanduja, When is $R[\theta]$ integrally closed?, J. Algebra Appl., 15(5) (2016) 1650091 (7 pages)


#  Synaptic metaplasticity in binarized neural networks

Axel Laborieux, Maxence Ernoult , Tifenn Hirtzlin1, Damien Querlioz1 Centre de
Nanosciences et de Nanotechnologies, Université Paris-SaclayMila, Université
de Montréal

## Summary

Unlike the brain, artificial neural networks, including state-of-the-art deep
neural networks for computer vision, are subject to “catastrophic forgetting”
[1]: they rapidly forget the previous task when trained on a new one.
Neuroscience suggests that biological synapses avoid this issue through the
process of synaptic consolidation and _metaplasticity_ : the plasticity itself
changes upon repeated synaptic events [2, 3]. In this work, we show that this
concept of metaplasticity can be transferred to a particular type of deep
neural networks, binarized neural networks (BNNs) [4], to reduce catastrophic
forgetting. BNNs were initially developed to allow low-energy consumption
implementation of neural networks. In these networks, synaptic weights and
activations are constrained to $\\{-1,+1\\}$ and training is performed using
_hidden_ real-valued weights which are discarded at test time. Our first
contribution is to draw a parallel between the metaplastic states of [2] and
the hidden weights inherent to BNNs. Based on this insight, we propose a
simple synaptic consolidation strategy for the hidden weight. We justify it
using a tractable binary optimization problem, and we show that our strategy
performs almost as well as mainstream machine learning approaches to mitigate
catastrophic forgetting, which minimize task-specific loss functions [5], on
the task of learning pixel-permuted versions of the MNIST digit dataset
sequentially. Moreover, unlike these techniques, our approach does not require
task boundaries, thereby allowing us to explore a new setting where the
network learns from a stream of data. When trained on data streams from
Fashion MNIST or CIFAR-10, our metaplastic BNN outperforms a standard BNN and
closely matches the accuracy of the network trained on the whole dataset.
These results suggest that BNNs are more than a low precision version of full
precision networks and highlight the benefits of the synergy between
neuroscience and deep learning [6].

## Hidden weights as metaplastic states

The problem of forgetting in artificial neural networks results from a
dilemma: synapses need to be updated in order to learn new tasks but also to
be protected against further changes in order to preserve knowledge. In a
foundational neuroscience work, Fusi et al. show than in small Hopfield
networks, catastrophic forgetting can be addressed by introducing a hidden
metaplastic state that controls the plasticity of the synapse [2]. Synapses
can assume only $+1$ or $-1$ weight, with the metaplastic state modulating the
difficulty for the synapse to switch. Therefore, in this scheme, repeated
potentiation of a positive-weight synapse will only affect its metaplastic
state and not its actual weight. Here, we remark that the way that BNNs are
trained is remarkably similar to this situation. In BNNs, synapses can also
only assume $+1$ or $-1$ weight, and they feature a hidden real weight
($W^{\rm h}$), which is updated by backpropagation. The synaptic weight
changes between $+1$ and $-1$ only when $W^{\rm h}$ changes sign, suggesting
that $W^{\rm h}$ can be seen as a metaplastic state modulating the difficulty
for the actual weight to change sign. However, standard BNNs are as prone to
catastrophic forgetting as conventional neural networks. In [2], Fusi et al.
showed that the metaplastic changes should make subsequent affect plasticity
exponentially to mitigate forgetting, whereas $W^{\rm h}$ affects weight
changes only linearly in BNNs. Therefore, in this work, we propose to adapt
the learning process of BNNs so that the larger the magnitude of a hidden
weight $W^{\rm h}$, the more difficult to switch its associated binarized
weight $W^{\rm b}=\mbox{sign}(W^{\rm h})$. Denoting $U_{W}$ the update
provided by the learning algorithm, we implement:

$\displaystyle W^{\rm h}$ $\displaystyle\leftarrow W^{\rm h}-\eta U_{W}\cdot
f_{\rm meta}(m,W^{\rm h})\quad{\rm if}\quad U_{W}W^{\rm h}>0$ $\displaystyle
W^{\rm h}$ $\displaystyle\leftarrow W^{\rm h}-\eta U_{W}\quad{\rm otherwise.}$

As in the metaplasticity model of [2] where synaptic plasticity decreases
exponentially with the metaplastic state, we choose $f_{\rm meta}(m,W^{\rm
h})={\rm tanh^{{}^{\prime}}}(m\cdot W^{\rm h})$ to produce an exponential
decay for large metaplastic states $W^{\rm h}$, where $m$ is an hyperparameter
that controls the consolidation.

## Toy problem study

To validate the interpretation of hidden weights as metaplastic states, we
first focus on a highly simplified binary optimization task that we solve in a
way analogous to the BNN training process. We want the binarized weights
$W^{\rm b}$ to minimize a quadratic loss $\mathcal{L}$, as depicted by the
color map on Fig. 1(a) in two dimensions, with $W^{*}$ as the global optimum.
We assume that $W^{\rm h}$ is updated by loss gradients computed with
binarized weights, similarly to BNNs:

$\displaystyle W^{{\rm h}}_{t+1}=W^{{\rm
h}}_{t}-\eta\frac{\partial\mathcal{L}}{\partial W}(W^{\rm b}_{t}).$

We can show that if the infinite norm of $W^{*}$ is lesser than one, some
hidden weights diverge as $t\to\infty$. This is because $W^{\rm h}$ is updated
by loss gradients computed at the corners of the square, in contrast with
conventional optimization. More importantly, if we define importance of the
binarized weight as the increase of the loss $\Delta\mathcal{L}$ when the
weight is switched to the opposite value, we can prove that the speed of
divergence of the hidden weight is directly linked to the importance of the
binarized weight. For instance, in Fig. 1(a), $W^{\rm b}_{x}$ is more
important than $W^{\rm b}_{y}$ for optimization. Finally, we plot
$\Delta\mathcal{L}$ versus $|W^{\rm h}|$ in Fig. 1(b), (c) for higher
dimensions and for a BNN trained on MNIST and observe that the correspondence
between important weights and hidden weight divergence still holds, justifying
the fact that consolidating synapses with diverging hidden weights as our
proposal does, is a promising route for mitigating catastrophic forgetting.

## Experimental results

### Continual learning benchmark.

We now apply our consolidation strategy to the permuted MNIST benchmark on two
hidden layers perceptrons of varying number of neurons. We show in Fig.
1(d),(e) the average test accuracy as a function of the number of tasks
learned so far. We observe that our technique indeed allows sequential task
learning and performs almost as well as Elastic Weight Consolidation (EWC) [5]
adapted to BNNs (the importance factor is computed with the binarized weights)
over a wide range of hidden layer sizes when learning up to 20 tasks. We
choose $m=1.35$ and $\lambda_{EWC}=5\cdot 10^{-3}$ for EWC.

### Learning from a stream of data.

By construction, our approach does not require to update the importance factor
between two consecutive tasks. Building on this asset, we explore a new
setting, which we call stream learning, and where a task is learned by
learning sub-parts of the full dataset sequentially, with all classes evenly
distributed in each subset. We choose Fashion MNIST (FMNIST) and CIFAR-10 for
our experiments. The architectures used are a perceptron with two hidden
layers of 1,024 units for FMNIST and a VGG-16 convolutional architecture for
CIFAR-10. We plot on Fig. 1(f), (g) the test accuracy reached by those
networks when metaplasticity is used (red) or not (blue). We see that our
approach comes closer to the accuracy reached when the full dataset is learned
at once (straight lines) than the non-metaplastic counterpart. Overall, these
results highlight the benefit of metaplasticity models from neuroscience when
applied to machine learning.

## Acknowledgement

This work was supported by European Research Council Starting Grant NANOINFER
(reference: 715872).

## References

  * [1] French, R. M., Trends in cogn. sci. (1999).
  * [2] Fusi et al. Neuron (2005).
  * [3] Abraham, W. C. Nat. Rev. Neurosci. (2008).
  * [4] Courbariaux et al. arXiv:1602.02830 (2016)
  * [5] Kirkpatrick et al. PNAS (2017).
  * [6] Richards et al. Nat. Neurosci. (2019).
  * [7] Zenke et al. PMLR (2017).

Figure 1: (a) Quadratic binarized optimization in two dimensions. (b-c)
Average loss increase when switching $W^{\rm b}$ versus normalized hidden
weight, for the binary quadratic problem (b) and for a BNN on MNIST (c). (d-e)
Permuted MNIST benchmark with our method (d) and EWC (e), where the x axis
labels the number of learned tasks, with one color per network size. Fashion
MNIST (f) and CIFAR-10 (g) test accuracy in the stream learning setting
allowed by our approach (red) compared to a standard BNN (blue). Horizontal
rules denote full dataset training baseline.