# Solving Constrained CASH Problems with ADMM

P. Ram<EMAIL_ADDRESS>
S. Liu<EMAIL_ADDRESS>
D. Vijaykeerthi<EMAIL_ADDRESS>
D. Wang<EMAIL_ADDRESS>
D. Bouneffouf<EMAIL_ADDRESS>
G. Bramble<EMAIL_ADDRESS>
H. Samulowitz<EMAIL_ADDRESS>
A. G. Gray<EMAIL_ADDRESS>
IBM Research

###### Abstract

The CASH problem has been widely studied in the context of automated
configurations of machine learning (ML) pipelines and various solvers and
toolkits are available. However, CASH solvers do not directly handle black-box
constraints such as fairness, robustness or other domain-specific custom
constraints. We present our recent approach (Liu et al., 2020) that leverages
the ADMM optimization framework to decompose CASH into multiple small problems
and demonstrate how ADMM facilitates incorporation of black-box constraints.

##  1 Automated ML Pipeline Configuration

Hyper-parameter optimization (HPO) for a single machine learning (ML)
algorithm is widely studied in AutoML (Snoek et al., 2012; Shahriari et al.,
2016). HPO was generalized to the Combined Algorithm Selection and HPO (CASH)
problem to configure multiple stages of a ML pipeline (transformers, feature
selectors, predictive models) automatically (Thornton et al., 2012; Feurer et
al., 2015). Since the CASH formulation, various innovative solvers have been
proposed (Hutter et al., 2011; Bergstra et al., 2011; Mohr et al., 2018;
Rakotoarison et al., 2019). CASH has two main challenges: (i) the tight
coupling between the algorithm selection & HPO; and (ii) the black-box nature
of optimization objective lacking any explicit gradients – feedback is only
available in the form of (often expensive) function evaluations. Furthermore,
the CASH formulation does not explicitly handle constraints on the individual
ML pipelines such as fairness or domain-specific constraints on individual ML
pipelines.

We view CASH as a mixed integer black-box nonlinear program and we recently
proposed a novel solution framework leveraging the alternating direction
method of multipliers (ADMM) framework (Liu et al., 2020). ADMM offers a two-
block alternating optimization procedure that splits an involved problem (with
multiple variables & constraints) into simpler (often unconstrained) sub-
problems (Boyd et al., 2011; Parikh and Boyd, 2014).

Contributions. We utilize ADMM to decompose CASH into 3 problems: (i) a black-
box optimization with a small set of only continuous variables, (ii) a closed-
form Euclidean projection onto an integer set, and (iii) a black-box integer
program. Moreover, the ADMM framework handles any black-box constraints
alongside the black-box objective – such constraints are seamlessly
incorporated while retaining very similar sub-problems.

###  1.1 Related work

Beyond grid-search and random search (Bergstra and Bengio, 2012) for HPO,
sequential model-based optimization (SMBO) is a common technique with
different ‘surrogate models’ such as Gaussian processes (Snoek et al., 2012),
random forests (Hutter et al., 2011) and tree-parzen estimators (Bergstra et
al., 2011). Successive halving (Jamieson and Talwalkar, 2016; Sabharwal et
al., 2016) and HyperBand (Li et al., 2018) utilize computationally cheap
multi-fidelity approximations of the objective based on some budget (training
samples/epochs) with bandit learning to eliminate unpromising candidates
early. These schemes essentially perform an efficient random search over
discrete or discretized continuous spaces. BOHB (Falkner et al., 2018)
combines SMBO (with TPE) and HyperBand. Meta-learning (Vanschoren, 2018; Fusi
et al., 2018; Drori et al., 2018) leverages past experiences with search space
refinements and promising starting points. SMBO with a large number of
variables along with conditional dependencies has been used to solve CASH in
the widely-used Auto-WEKA (Thornton et al., 2012; Kotthoff et al., 2017) and
Auto-sklearn (Feurer et al., 2015) toolkits. Both apply the general purpose
SMAC framework (Hutter et al., 2011) to find optimal ML pipelines of fixed
shape. Hyperopt-sklearn (Komer et al., 2014) utilizes TPE as the SMBO.

SMBO-based CASH has been improved by partially splitting CASH and taking
advantage of the structure in the algorithm selection problem. ML-Plan (Mohr
et al., 2018) uses hierarchical task networks (HTN) planning for algorithm
selection and randomized search for HPO, while MOSAIC (Rakotoarison et al.,
2019) utilizes Monte-Carlo Tree Search (MCTS) and Bayesian Optimization (BO)
respectively. The two sub-problems are coupled with a shared surrogate model.
Even though algorithm selection and HPO are solved in independent steps,
MOSAIC still requires computations (such as the acquisition function
estimation/optimization and the surrogate-model training) over the high-
dimensional joint search-space of all algorithms and hyper-parameters (HPs).
Our proposed ADMM-based solution performs an explicit primal-dual
decomposition, significantly reducing the dimensionality of each of the
aforementioned sub-problems. Moreover, our solution presents a general
framework that can incorporate HTN-Planning and MCTS based algorithm
selection.

ADMMBO (Ariafar et al., 2019) for general BO with black-box constraints
maintains a separate surrogate model for the objective and each of the black-
box constraints (a total of $M+1$ surrogate models for $M$ constraints) and
utilizes the ADMM framework to split the constrained optimization into a
sequence of unconstrained problems to minimize the objective and constraint
violations. The sub-problem dimensionality remains the same as the original
problem. Our proposed ADMM based scheme for CASH with black-box constraints
takes advantage of the problem structure to handle $M$ black-box constraints
by adding $M$ variables to one of the three sub-problems without needing any
additional surrogate models.

##  2 CASH as a Mixed Integer Nonlinear Program

We focus on CASH for a fixed pipeline shape – for $N$ functional modules with
$K_{i}$ choices in each module, let $\mathbf{z}_{i}\in\\{0,1\\}^{K_{i}}$
denote the algorithm choice in module $i$, with the constraint
$\mathbf{1}^{\top}\mathbf{z}_{i}=1$ ensuring a single choice per module. Let
$\mathbf{z}=\left\\{\mathbf{z}_{1},\ldots,\mathbf{z}_{N}\right\\}$. Assuming
that categorical HPs can be encoded as integers, let
$\boldsymbol{\theta}_{ij}$ be the HPs of algorithm $j$ in module $i$
($\boldsymbol{\theta}_{ij}^{c}\in\mathcal{C}_{ij}\subset\mathbb{R}^{m_{ij}^{c}}$
continuous and
$\boldsymbol{\theta}_{ij}^{d}\in\mathcal{D}_{ij}\subset\mathbb{Z}^{m_{ij}^{d}}$
integer). With
$\boldsymbol{\theta}^{c}=\left\\{\boldsymbol{\theta}_{ij}^{c},\forall
i\in[N],j\in[K_{i}]\right\\}$, and $\boldsymbol{\theta}^{d}$ defined
analogously, let
$f(\mathbf{z},\\{\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}\\})$
represent the loss of a ML pipeline configured with algorithm choices
$\mathbf{z}$ and the HPs
$\\{\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}\\}$. CASH can be stated
as111 The above formulation allows for easily extending the search to more
flexible pipelines (Appendix A). To extensively evaluate our proposed CASH
solver with multiple repetitions and restarts, we also propose a novel cheap-
to-evaluate black-box objective (Appendix B) that possesses the structure of
(CASH).:

$\displaystyle\min_{\mathbf{z},\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}}f(\mathbf{z},\\{\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}\\})\operatorname*{\text{subject
to}}\left\\{\begin{array}[]{l}\mathbf{z}_{i}\in\\{0,1\\}^{K_{i}},\mathbf{1}^{\top}\mathbf{z}_{i}=1,\forall
i\in[N],\\\
\boldsymbol{\theta}_{ij}^{c}\in\mathcal{C}_{ij},\boldsymbol{\theta}_{ij}^{d}\in\mathcal{D}_{ij},\forall
i\in[N],j\in[K_{i}].\end{array}\right.$ (CASH)

We may need the ML pipelines to also explicitly satisfy application specific
constraints corresponding to $M$ general black-box functions with no analytic
form in $(\mathbf{z},\\{\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}\\})$:

$g_{m}(\mathbf{z},\\{\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}\\})\leq\epsilon_{m},m\in[M].$
(1)

For example, deployment constraints may require prediction latency below a
threshold. Business constraints may require highly accurate pipelines with
explicitly bounded false positive rate – false positives may deny loans to
eligible applicants, which is lost business and could violate anti-
discriminatory requirements. Pursuing fair AI, regulators may require that any
deployed ML pipeline explicitly satisfies bias constraints (Friedler et al.,
2019).

##  3 Decomposing CASH with ADMM

Introducing a surrogate objective $\widetilde{f}$ over the continuous domain
$(\mathcal{C}_{ij}\times\widetilde{\mathcal{D}}_{ij}),i\in[N],j\in[K_{i}]$
that matches the objective $f$ over $(\mathcal{C}_{ij}\times\mathcal{D}_{ij})$
with $\widetilde{\mathcal{D}}_{ij}$ as the continuous relaxation of the
integer space $\mathcal{D}_{ij}$, we follow ADMM, detailed in Appendix C, to
decompose (CASH) into the following 3 sub-problems to be solved iteratively,
with $(t)$ representing the ADMM iteration index, $\rho$ as the penalty for
the augmented Lagrangian term and the Lagrangian multipliers
$\boldsymbol{\lambda}$ updated as
$\boldsymbol{\lambda}{}^{(t+1)}=\boldsymbol{\lambda}{}^{(t)}+\rho(\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}-\boldsymbol{\delta}{}^{(t+1)})$:

$\displaystyle\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\right\\}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\theta}^{c}_{ij},\widetilde{\boldsymbol{\theta}}^{d}_{ij}}\widetilde{f}\left(\mathbf{z}{}^{(t)},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)+\frac{\rho}{2}\left\|\widetilde{\boldsymbol{\theta}}^{d}-\mathbf{b}\right\|_{2}^{2},\mathbf{b}\mathrel{\mathop{:}}=\boldsymbol{\delta}{}^{(t)}-\frac{1}{\rho}\boldsymbol{\lambda}{}^{(t)},$
($\boldsymbol{\theta}$-min) $\displaystyle\boldsymbol{\delta}{}^{(t+1)}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\delta}_{ij}\in\mathcal{D}_{ij}}\left\|\mathbf{a}-\boldsymbol{\delta}\right\|_{2}^{2},\quad\mathbf{a}\mathrel{\mathop{:}}=\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}+(1/\rho)\boldsymbol{\lambda}{}^{(t)},$
($\boldsymbol{\delta}$-min) $\displaystyle\mathbf{z}{}^{(t+1)}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\mathbf{z}_{i}\in\\{0,1\\}^{K_{i}},\mathbf{1}^{\top}\mathbf{z}_{i}=1}\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\right\\}\right).$
($\mathbf{z}$-min)

Solving ($\boldsymbol{\theta}$-min). This is a continuous black-box
optimization problem with variables $\boldsymbol{\theta}^{c}$ and
$\widetilde{\boldsymbol{\theta}}^{d}$. Since the algorithms
$\mathbf{z}{}^{(t)}$ are fixed in ($\boldsymbol{\theta}$-min), $\widetilde{f}$
only depends on the HPs of the chosen algorithms – the active variable set
$S=\\{(\boldsymbol{\theta}_{ij}^{c},\widetilde{\boldsymbol{\theta}}_{ij}^{d}):z_{ij}{}^{(t)}=1\\}$.
This splits ($\boldsymbol{\theta}$-min) even further into (i)
$\min_{\widetilde{\boldsymbol{\theta}}_{ij}^{d}\in\widetilde{\mathcal{D}}_{ij}}\|\widetilde{\boldsymbol{\theta}}_{ij}^{d}-\mathbf{b}_{ij}\|_{2}^{2}$
with $z_{ij}=0$ (the inactive set) requiring a Euclidean projection of
$\mathbf{b}_{ij}$ onto $\widetilde{\mathcal{D}}_{ij}$, and (ii) a black-box
optimization with a small active continuous variable set222For the CASH
problems we consider in our empirical evalutations,
$|\boldsymbol{\theta}|=|\boldsymbol{\theta}_{ij}^{c}|+|\widetilde{\boldsymbol{\theta}}_{ij}^{d}|\approx
100$ while the largest possible active set $S$ is less than $15$ and typically
less than $10$.$S$. Solvers such as BO (Shahriari et al., 2016), direct search
(Larson et al., 2019), or trust-region based derivative-free optimization
(Conn et al., 2009) work fairly well.

Solving ($\boldsymbol{\delta}$-min). This requires an elementwise minimization
$\min_{\boldsymbol{\delta}_{ij}}(\boldsymbol{\delta}_{ij}-\mathbf{a}_{ij})^{2}$
and solved in closed form by projecting $\mathbf{a}_{ij}$ onto
$\widetilde{\mathcal{D}}_{ij}$ and then rounding to the nearest integer in
$\mathcal{D}_{ij}$.

Solving $\mathbf{z}$-min. The following black-box discrete optimizers can be
utilized: (i) MCTS can be used as in Rakotoarison et al. (2019), (ii) HTN-
Planning as in Mohr et al. (2018) (iii) Multi-fidelity evaluations can be used
with successive halving (Jamieson and Talwalkar, 2016; Li et al., 2018) or
incremental data allocation (Sabharwal et al., 2016), (iv) Interpreting
($\mathbf{z}$-min) as a combinatorial multi-armed bandits problem and
utilizing Thompson sampling (Durand and Gagné, 2014) as we did in Liu et al.
(2020, Appendix 4).

###  3.1 Empirical Advantages of ADMM

ADMM has been used for both convex (Boyd et al., 2011) and non-convex
optimization (Xu et al., 2016) and various ADMM enhancements have been
proposed. We demonstrate how these improve ADMM for (CASH). For initial
evaluations, we use the artificial objective (Appendix B) for cheap
evaluations, which allow us to efficiently generate statistically significant
results over multiple trials. A maximum of 100 ADMM iterations is executed
with $\rho=1$ (ADMM for CASH appears to be stable with respect to $\rho$; see
Appendix D) for a maximum runtime of $2^{10}$ seconds. We consider a search
space of 4 modules with 8 scalers, 11 transformers, 7 feature selectors and 11
estimators (total 6776 algorithm combinations with almost $100$ HPs). See Liu
et al. (2020, Appendix 7) for complete details.

(a) Fixed vs. Adaptive

(b) Split vs. Joint

(c) Custom solvers

(d) Cold vs. Warm

Figure 1: Performance of ADMM and enhancements on the artificial black-box
objective (Appendix B) with the incumbent objective (median and inter-quartile
range over 30 trials) on the vertical axis and optimization time on the
horizontal axis. Note the logscale on both axes.

Adaptive precision. ADMM progresses by iteratively solving
($\boldsymbol{\theta}$-min) and ($\mathbf{z}$-min)
(($\boldsymbol{\delta}$-min) is solved in closed form). In practice, the sub-
problems are often solved to lower precision in the initial ADMM iterations;
the precision is progressively increased through the ADMM iterations (adaptive
precision) instead of maintaining the same precision for all ADMM iterations
(fixed precision). In Figure 1(a), we use Bayesian Optimization (BO) to solve
both ($\boldsymbol{\theta}$-min) & ($\mathbf{z}$-min). The precision of the
sub-problems are controlled by modifying the number of BO iterations. For
fixed precision ADMM, we fix the BO iterations for the sub-problems to
$I=\\{16,32,64,128,256\\}$ denoted as ADMM$16$(BO,BO) and so on. For adaptive
precision ADMM, we initially solve each sub-problem with $16$ BO iterations
and progressively increase the number of BO iterations to $256$ with an
additive factor of $F=\\{8,16\\}$ in each ADMM iteration denoted by
AdADMM-$F8$(BO,BO) & AdADMM-$F16$(BO,BO) respectively. We see the expected
behavior – fixed precision ADMM with small $I$ dominate for small time scales
but saturate soon; large $I$ require significant start-up time but dominate
for larger time scales. Adaptive precision ADMM provides best anytime
performance.

Solving smaller sub-problems. In Figure 1(b), we demonstrate the advantage of
solving multiple smaller sub-problems over solving a single large joint
problem. We use BO to solve both the joint problem (JOPT(BO)) and the sub-
problems in the adaptive ADMM (AdADMM-$F8$(BO,BO) & AdADMM-$F16$(BO,BO)). ADMM
demonstrates $120\times$ speedup to reach the best objective obtained by
JOPT(BO) within the time budget, and also provides an additional $17\%$
improvement over that objective at the end of the time budget.

Custom sub-problem solvers. BO is designed for problems with (a small number
of) continuous variables and hence is well-suited for
($\boldsymbol{\theta}$-min); it is not designed for ($\mathbf{z}$-min). We
should instead use schemes customized for ($\mathbf{z}$-min) such as MCTS
(Rakotoarison et al., 2019) or Thompson sampling for combinatorial multi-armed
bandits (Liu et al., 2020, Appendix 4). In Figure 1(c), we consider BO for
($\boldsymbol{\theta}$-min) and bandits for ($\mathbf{z}$-min) –
AdADMM-$F8$(BO,Ba) & AdADMM-$F16$(BO,Ba) – which further improves over
adaptive ADMM, demonstrating $140\times$ speedup over JOPT(BO) with an
additional $29\%$ improvement in the objective value.

Warm Start ADMM. It is common in ADMM to warm-start the sub-problem
minimizations in any ADMM iteration with the solution of the same sub-problem
from previous ADMM iterations (if available) to improve empirical convergence.
In adaptive precision ADMM, we employ warm-starts for BO in
($\boldsymbol{\theta}$-min) to get BO∗ and compare it to BO with cold-starts.
Figure 1(d) indicates that while BO and BO∗ converge to the same objective at
the end of the time budget, BO∗ has significantly better anytime performance.

Evaluation on OpenML data sets. We evaluate the performance of ADMM against
JOPT(BO) for (CASH) on 8 OpenML data sets in Appendix E and see over
$10\times$ speedup in most cases and over $10\%$ improvement in the final
objective in many cases. We refer readers to Liu et al. (2020) for the
detailed comparison of ADMM to Auto-sklearn (Feurer et al., 2015) and TPOT
(Olson and Moore, 2016) across 30 classification data sets.

##  4 CASH with Black-box Constraints

We consider (CASH) in the presence of black-box constraints (1). Without loss
of generality, let $\epsilon_{m}\geq 0$ for $m\in[M]$. With scalars
$u_{m}\in[0,\epsilon_{m}]$, we reformulate the inequality constraint (1) as an
equality constraint
$g_{m}(\mathbf{z},\\{\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}\\})=\epsilon_{m}-u_{m}$
and a box constraint $u_{m}\in[0,\epsilon_{m}]$.

Introducing (i) surrogate $\widetilde{g}_{m}$ for each black-box function
$g_{m},m\in[M]$ over the continuous domain in a manner similar to
$\widetilde{f}$ in Section 3, (ii) new Lagrangian multipliers
$\mu_{m},m\in[M]$ for each of the $M$ black-box constraints, and following the
ADMM mechanics (detailed in Appendix F), we decompose constrained (CASH) into
the following unconstrained problems:

$\min_{\boldsymbol{\theta}^{c}_{ij},\widetilde{\boldsymbol{\theta}}^{d}_{ij},u_{m}}\widetilde{f}(\mathbf{z}{}^{(t)},\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\\})+\frac{\rho}{2}[\|\widetilde{\boldsymbol{\theta}}^{d}-\mathbf{b}\|_{2}^{2}+\sum_{m=1}^{M}[\widetilde{g}_{m}(\mathbf{z}{}^{(t)},\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\\})-\epsilon_{m}+u_{m}+\frac{1}{\rho}\mu_{m}{}^{(t)}]^{2}],$
(2)
$\min_{\mathbf{z}}\widetilde{f}(\mathbf{z},\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\\})+\frac{\rho}{2}\sum_{i=1}^{M}[\widetilde{g}_{m}(\mathbf{z},\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\\})-\epsilon_{m}+u_{m}{}^{(t+1)}+\frac{1}{\rho}\mu_{m}{}^{(t)}]^{2},$
(3)

with $\mathbf{b}$ defined in ($\boldsymbol{\theta}$-min),
$\boldsymbol{\delta}$ updated as per ($\boldsymbol{\delta}$-min), and
$\mu_{m},m\in[M]$ updated as
$\mu_{m}{}^{(t+1)}=\mu_{m}{}^{(t)}+\rho(\widetilde{g}_{m}(\mathbf{z}{}^{(t+1)},\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\\})-\epsilon_{m}+u_{m}{}^{(t+1)})$.

Problem (2) is black-box optimization with continuous variables similar to
($\boldsymbol{\theta}$-min) (further split into active and inactive set of
variables as per $\mathbf{z}{}^{(t)}$). The main difference from
($\boldsymbol{\theta}$-min) are the $M$ new optimization variables
$u_{m},m\in[M]$ active in every ADMM iteration. This active set optimization
can be solved in the same way as in ($\boldsymbol{\theta}$-min) (namely with
BO). Problem (3) remains a black-box integer program in $\mathbf{z}$ with an
updated objective incorporating constraint violations and can be solved with
methods discussed for ($\mathbf{z}$-min).

###  4.1 Empirical evaluation

(a) Objective vs. $t_{p}$

(b) Objective vs. $d_{I}$

(c) CST-SAT vs. $t_{p}$

(d) CST-SAT vs. $d_{I}$

Figure 2: Performance of unconstrained and constrained ADMM (aggregated over
10 trials) executed for 1 hour with varying thresholds for the 2 constraints.
Note the log-scale on the vertical axis.

We consider data from the Home Credit Default Risk Kaggle challenge with the
objective of $(1-\mbox{AUROC})$, and 2 black-box constraints: (i) (deployment)
Prediction latency $t_{p}$ enforcing real-time predictions, (ii) (fairness)
Maximum pairwise disparate impact $d_{I}$ (Calders and Verwer, 2010) across
all loan applicant age groups enforcing fairness across groups. We perform the
following experiments: (i) fixing $d_{I}=0.7$, we vary $t_{p}\in[1,20]$ (in
$\mu s$), and (ii) fixing $t_{p}=10\mu s$ and we vary $d_{I}\in[0.05,0.15]$.
Note that the constraints get less restrictive as the thresholds increase. We
apply ADMM to the unconstrained problem (UCST) and post-hoc filter constraint
satisfying pipelines to demonstrate that these constraints are not trivially
satisfied. Then we execute ADMM with these constraints (CST). Using BO for
($\boldsymbol{\theta}$-min) (and (2)) & bandits for ($\mathbf{z}$-min) (and
(3)), we get UCST(BO,Ba) & CST(BO,Ba).

Figures 2(a) & 2(b) present the best objective achieved when limited only to
constraint satisfying pipelines as the constraint on $t_{p}$ and $d_{I}$ are
respectively relaxed. As expected, the objective improves as the constraints
relax. In both cases, CST outperforms UCST, with UCST approaching CST as the
constraints relax. Figures 2(c) & 2(d) (for varying $t_{p}$ & $d_{I}$
respectively) present the constraint satisfying capability of the optimizer by
considering the fraction of constraint-satisfying pipelines found. CST again
significantly outperforms UCST, indicating that the constraints are non-
trivial to satisfy, and that ADMM is able to effectively incorporate the
constraints for improved performance.

##  5 Conclusion

In this paper, we summarized our recent ADMM based CASH solver and
demonstrated the utility of ADMM and its multiple enhancements. We also
considered CASH with black-box constraints and demonstrated how ADMM
seamlessly handles them. While we skip the multi-fidelity aspect of CASH here,
ADMM can use multi-fidelity solvers such as Hyperband and BOHB for the
($\boldsymbol{\theta}$-min) and ($\mathbf{z}$-min) sub-problems. Moreover, we
can combine ADMM with a context-free grammar for ML pipelines and AI planning
to jointly solve CASH and the pipeline shape search problem with black-box
constraints (Katz et al., 2020).

## References

  * Ariafar et al. (2019) Setareh Ariafar, Jaume Coll-Font, Dana Brooks, and Jennifer Dy.  Admmbo: Bayesian optimization with unknown constraints using admm.  _Journal of Machine Learning Research_ , 20(123):1–26, 2019.
  * Bergstra and Bengio (2012) James Bergstra and Yoshua Bengio.  Random search for hyper-parameter optimization.  _Journal of Machine Learning Research_ , 13(Feb):281–305, 2012.
  * Bergstra et al. (2011) James S Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl.  Algorithms for hyper-parameter optimization.  In _Advances in neural information processing systems_ , pages 2546–2554, 2011.
  * Bischl et al. (2017) Bernd Bischl, Giuseppe Casalicchio, Matthias Feurer, Frank Hutter, Michel Lang, Rafael G Mantovani, Jan N van Rijn, and Joaquin Vanschoren.  Openml benchmarking suites and the openml100.  _arXiv preprint arXiv:1708.03731_ , 2017.
  * Boyd et al. (2011) Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al.  Distributed optimization and statistical learning via the alternating direction method of multipliers.  _Foundations and Trends® in Machine Learning_ , 3(1):1–122, 2011.
  * Calders and Verwer (2010) Toon Calders and Sicco Verwer.  Three naive bayes approaches for discrimination-free classification.  _Data Mining and Knowledge Discovery_ , 21(2):277–292, 2010.
  * Conn et al. (2009) Andrew R Conn, Katya Scheinberg, and Luis N Vicente.  _Introduction to derivative-free optimization_ , volume 8.  Siam, 2009.
  * Drori et al. (2018) Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni Lourenço, J One, Kyunghyun Cho, Claudio Silva, and Juliana Freire.  Alphad3m: Machine learning pipeline synthesis.  In _AutoML Workshop at ICML_ , 2018.
  * Durand and Gagné (2014) A. Durand and C. Gagné.  Thompson sampling for combinatorial bandits and its application to online feature selection.  In _Workshops at the Twenty-Eighth AAAI Conference on Artificial Intelligence_ , 2014.
  * Falkner et al. (2018) Stefan Falkner, Aaron Klein, and Frank Hutter.  BOHB: Robust and efficient hyperparameter optimization at scale.  In _Proceedings of the 35th International Conference on Machine Learning_ , pages 1437–1446, 2018.
  * Feurer et al. (2015) Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter.  Efficient and robust automated machine learning.  In _Advances in Neural Information Processing Systems_ , pages 2962–2970, 2015.
  * Friedler et al. (2019) Sorelle A Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P Hamilton, and Derek Roth.  A comparative study of fairness-enhancing interventions in machine learning.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , pages 329–338. ACM, 2019.
  * Fusi et al. (2018) Nicolo Fusi, Rishit Sheth, and Melih Elibol.  Probabilistic matrix factorization for automated machine learning.  In _Advances in Neural Information Processing Systems_ , pages 3348–3357, 2018.
  * Hutter et al. (2011) Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown.  Sequential model-based optimization for general algorithm configuration.  In _Proceedings of the 5th International Conference on Learning and Intelligent Optimization_ , LION’05, pages 507–523, Berlin, Heidelberg, 2011\. Springer-Verlag.
  * Jamieson and Talwalkar (2016) Kevin Jamieson and Ameet Talwalkar.  Non-stochastic best arm identification and hyperparameter optimization.  In _Artificial Intelligence and Statistics_ , pages 240–248, 2016\.
  * Katz et al. (2020) Michael Katz, Parikshit Ram, Shirin Sohrabi, and Octavian Udrea.  Exploring context-free languages via planning: The case for automating machine learning.  In _Proceedings of the International Conference on Automated Planning and Scheduling_ , volume 30, pages 403–411, 2020.
  * Komer et al. (2014) Brent Komer, James Bergstra, and Chris Eliasmith.  Hyperopt-sklearn: automatic hyperparameter configuration for scikit-learn.  In _ICML workshop on AutoML_ , pages 2825–2830. Citeseer, 2014.
  * Kotthoff et al. (2017) Lars Kotthoff, Chris Thornton, Holger H. Hoos, Frank Hutter, and Kevin Leyton-Brown.  Auto-weka 2.0: Automatic model selection and hyperparameter optimization in weka.  _J. Mach. Learn. Res._ , 18(1):826–830, January 2017.  ISSN 1532-4435.  URL http://dl.acm.org/citation.cfm?id=3122009.3122034.
  * Larson et al. (2019) Jeffrey Larson, Matt Menickelly, and Stefan M Wild.  Derivative-free optimization methods.  _Acta Numerica_ , 28:287–404, 2019.
  * Li et al. (2018) Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar.  Hyperband: A novel bandit-based approach to hyperparameter optimization.  _Journal of Machine Learning Research_ , 18(185):1–52, 2018.
  * Liu et al. (2020) Sijia Liu, Parikshit Ram, Deepak Vijaykeerthy, Djallel Bouneffouf, Gregory Bramble, Horst Samulowitz, Dakuo Wang, Andrew Conn, and Alexander Gray.  An ADMM based framework for automl pipeline configuration.  In _Thirty-Fourth AAAI Conference on Artificial Intelligence_ , 2020\.  URL https://arxiv.org/abs/1905.00424v5.
  * Mohr et al. (2018) Felix Mohr, Marcel Wever, and Eyke Hüllermeier.  Ml-plan: Automated machine learning via hierarchical planning.  _Machine Learning_ , 107(8-10):1495–1515, 2018\.
  * Olson and Moore (2016) Randal S Olson and Jason H Moore.  Tpot: A tree-based pipeline optimization tool for automating machine learning.  In _Workshop on Automatic Machine Learning_ , pages 66–74, 2016.
  * Parikh and Boyd (2014) Neal Parikh and Stephen Boyd.  Proximal algorithms.  _Foundations and Trends® in Optimization_ , 1(3):127–239, 2014.
  * Rakotoarison et al. (2019) Herilalaina Rakotoarison, Marc Schoenauer, and Mich le Sebag.  Automated machine learning with monte-carlo tree search.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19_ , pages 3296–3303, 2019.
  * Sabharwal et al. (2016) Ashish Sabharwal, Horst Samulowitz, and Gerald Tesauro.  Selecting near-optimal learners via incremental data allocation.  In _Thirtieth AAAI Conference on Artificial Intelligence_ , 2016.
  * Shahriari et al. (2016) B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas.  Taking the human out of the loop: A review of bayesian optimization.  _Proceedings of the IEEE_ , 104(1):148–175, 2016\.
  * Snoek et al. (2012) J. Snoek, H. Larochelle, and R. P. Adams.  Practical bayesian optimization of machine learning algorithms.  In _Advances in neural information processing systems_ , 2012.
  * Thornton et al. (2012) Chris Thornton, Holger H. Hoos, Frank Hutter, and Kevin Leyton-Brown.  Auto-weka: Automated selection and hyper-parameter optimization of classification algorithms.  _arXiv_ , 2012.  URL http://arxiv.org/abs/1208.3719.
  * Vanschoren (2018) Joaquin Vanschoren.  Meta-learning: A survey.  _arXiv preprint arXiv:1810.03548_ , 2018.
  * Xu et al. (2016) Zheng Xu, Soham De, Mario Figueiredo, Christoph Studer, and Tom Goldstein.  An empirical study of admm for nonconvex problems.  _arXiv preprint arXiv:1612.03349_ , 2016.

##  A Generalization of the CASH Problem (CASH)

#### Generalization for more flexible pipelines.

We can extend the problem formulation (CASH) to enable optimization over the
ordering of the functional modules. For example, we can choose between
‘preprocessor $\to$ transformer $\to$ feature selector’ OR ‘feature selector
$\to$ preprocessor $\to$ transformer’. The ordering of $T\leq N$ modules can
be optimized by introducing $T^{2}$ Boolean variables
$\mathbf{o}=\\{o_{ik}\colon i,k\in[T]\\}$, where $o_{ik}=1$ indicates that
module $i$ is placed at position $k$. The following constraints are then
needed:

  1. (i)

$\sum_{k\in[T]}o_{ik}=1,\forall i\in[T]$, indicating that module $i$ is placed
at a single position, and

  2. (ii)

$\sum_{i\in[T]}o_{ik}=1\forall k\in[T]$ enforcing that only one module is
placed at position $k$.

These variables and constraints can be added to $\mathbf{z}$ in problem (CASH)
($\mathbf{z}=\\{\mathbf{z}_{1},\ldots,\mathbf{z}_{N},\mathbf{o}\\}$). The
resulting formulation still obeys the generic form of (CASH), which as we will
demonstrate in the following section, can be efficiently solved by an operator
splitting framework like ADMM. We can also extend the above formulation to
allow the choice of multiple algorithms from the same module. Note that we
increase the combinatorial complexity of the problem as we extend the
formulation.

##  B Artificial black-box objective

We want to devise an artificial black-box objective to study the behaviour of
the proposed schemes and baselines that matches the properties of the AutoML
problem (CASH) where

  1. 1.

The same pipeline (the same algorithm choices $\mathbf{z}$ and the same
hyperparameters $\boldsymbol{\theta}$) always gets the same value.

  2. 2.

The objective is not convex and possibly non-continuous.

  3. 3.

The objective captures the conditional dependence between $\mathbf{z}_{i}$ and
$\boldsymbol{\theta}_{ij}$ – the objective is only dependent on the hyper-
parameters $\boldsymbol{\theta}_{ij}$ if the corresponding $z_{ij}=1$.

  4. 4.

Minor changes in the hyper-parameters $\boldsymbol{\theta}_{ij}$ can cause
only small changes in the objective.

  5. 5.

The output of module $i$ is dependent on its input from module $i-1$.

#### Novel artificial black-box objective.

To this end, we propose the following novel black-box objective that emulates
the structure of (CASH):

  * •

For each $(i,j),i\in[N],j\in[K_{i}]$, we fix a weight vector $\mathbf{w}_{ij}$
(each entry is a sample from $\mathcal{N}(0,1)$) and a seed $s_{ij}$.

  * •

We set $f_{0}=0$.

    * –

For each module $i$, we generate a value

$v_{i}=\sum_{j}z_{ij}\left|\frac{\mathbf{w}_{ij}^{\top}\boldsymbol{\theta}_{ij}}{\mathbf{1}^{T}\boldsymbol{\theta}_{ij}}\right|$

which only depends on the $\boldsymbol{\theta}_{ij}$ corresponding to the
$z_{ij}=1$, and the denominator ensures that the number (or range) of the
hyper-parameters does not bias the objective towards (or away from) any
particular algorithm.

    * –

We generate $n$ samples
$\\{f_{i,1},\ldots,f_{i,n}\\}\sim\mathcal{N}(f_{i-1},v_{i})$ with the fixed
seed $s_{ij}$, ensuring that the same value will be produced for the same
pipeline.

    * –

$f_{i}=\max\limits_{m=1,\ldots,n}|f_{i,m}|$.

  * •

Output $f_{N}$

The basic idea behind this objective is that, for each operator, we create a
random (but fixed) weight vector $\mathbf{w}_{ij}$ and take a weighted
normalized sum of the hyper-parameters $\boldsymbol{\theta}_{ij}$ and use this
sum as the scale to sample from a normal distribution (with a fixed seed
$s_{ij}$) and pick the maximum absolute of $n$ (say 10) samples. For the first
module in the pipeline, the mean of the distribution is $f_{0}=0.0$. For the
subsequent modules $i$ in the pipeline, the mean $f_{i-1}$ is the output of
the previous module $i-1$. This function possesses all the aforementioned
properties of the AutoML problem (CASH).

In black-box optimization with this objective, the black-box evaluations are
very cheap in contrast to the actual AutoML problem where the black-box
evaluation requires a significant computational effort (and hence time).
However, we utilize this artificial objective to evaluate the black box
optimization schemes when the computational costs are dominated by the actual
derivative-free optimization and not by the black-box evaluations.

##  C ADMM Decomposition for (CASH)

#### Introduction of continuous surrogate loss.

We begin by proposing a surrogate loss of problem (CASH), which can be defined
over the continuous domain. With $\widetilde{\mathcal{D}}_{ij}$ as the
continuous relaxation of the integer space $\mathcal{D}_{ij}$ (if
$\mathcal{D}_{ij}$ includes integers ranging from
$\\{l,\ldots,u\\}\subset\mathbb{Z}$, then
$\widetilde{\mathcal{D}}_{ij}=[l,u]\subset\mathbb{R}$), and
$\widetilde{\boldsymbol{\theta}}^{d}$ as the continuous surrogates for
$\boldsymbol{\theta}^{d}$ with
$\widetilde{\boldsymbol{\theta}}_{ij}\in\widetilde{\mathcal{D}}_{ij}$
(corresponding to $\boldsymbol{\theta}_{ij}\in\mathcal{D}_{ij}$), we utilize a
surrogate loss function $\widetilde{f}$ for problem (CASH) defined solely over
the continuous domain with respect to $\boldsymbol{\theta}$:

$\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)\mathrel{\mathop{:}}=f\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\mathcal{P}_{\mathcal{D}}\left(\widetilde{\boldsymbol{\theta}}^{d}\right)\right\\}\right),$
(4)

where
$\mathcal{P}_{\mathcal{D}}(\widetilde{\boldsymbol{\theta}}^{d})=\\{\mathcal{P}_{\mathcal{D}_{ij}}(\widetilde{\boldsymbol{\theta}}_{ij}^{d}),\forall
i\in[N],j\in[K_{i}]\\}$ is the projection of the continuous surrogates onto
the integer set. This projection is necessary since the black-box function is
defined (hence can only be evaluated) on the integer sets $\mathcal{D}_{ij}$s,
not the relaxed continuous set $\widetilde{\mathcal{D}}_{ij}$s. Note that this
projection and has an efficient closed form. Given the above definitions, we
have the following equivalent form of the problem (CASH):

$\displaystyle\min_{\mathbf{z},\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\boldsymbol{\delta}}\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)\operatorname*{\text{subject
to}}\left\\{\begin{array}[]{l}\mathbf{z}_{i}\in\\{0,1\\}^{K_{i}},\mathbf{1}^{\top}\mathbf{z}_{i}=1,\forall
i\in[N]\\\
\boldsymbol{\theta}_{ij}^{c}\in\mathcal{C}_{ij},\widetilde{\boldsymbol{\theta}}_{ij}^{d}\in\widetilde{\mathcal{D}}_{ij},\forall
i\in[N],j\in[K_{i}]\\\ \boldsymbol{\delta}_{ij}\in\mathcal{D}_{ij},\forall
i\in[N],j\in[K_{i}]\\\
\widetilde{\boldsymbol{\theta}}_{ij}^{d}=\boldsymbol{\delta}_{ij},\forall
i\in[N],j\in[K_{i}],\end{array}\right.$ (5)

where the equivalence between problems (CASH) & (5) is established by the
equality constraint
$\widetilde{\boldsymbol{\theta}}_{ij}^{d}=\boldsymbol{\delta}_{ij}\in\mathcal{D}_{ij}$,
implying
$\mathcal{P}_{\mathcal{D}_{ij}}(\widetilde{\boldsymbol{\theta}}_{ij}^{d})=\widetilde{\boldsymbol{\theta}}_{ij}^{d}\in\mathcal{D}_{ij}$
and
$\widetilde{f}(\mathbf{z},\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\\})=f(\mathbf{z},\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\\})$,
thereby making the objective functions in problems (CASH) and (5) equal. We
highlight that the introduction of the continuous surrogate loss (4) is the
key to efficiently handling AutoML problems (CASH) by allowing us to perform
theoretically grounded operator splitting methods, e.g., ADMM, over mixed
continuous/integer hyper-parameters and integer model selection variables.

#### Operator splitting from ADMM.

Using the notation that $I_{\mathcal{X}}(\mathbf{x})=0$ if
$\mathbf{x}\in\mathcal{X}$ else $+\infty$, and defining the sets

$\displaystyle\mathcal{Z}$
$\displaystyle=\left\\{\mathbf{z}\colon\mathbf{z}=\\{\mathbf{z}_{i}\colon\mathbf{z}_{i}\in\\{0,1\\}^{K_{i}},\mathbf{1}^{\top}\mathbf{z}_{i}=1,\forall
i\in[N]\\}\right\\},$ (6) $\displaystyle\mathcal{C}$
$\displaystyle=\left\\{\boldsymbol{\theta}^{c}\colon\boldsymbol{\theta}^{c}=\\{\boldsymbol{\theta}_{ij}^{c}\in\mathcal{C}_{ij}\forall
i\in[N],j\in[K_{i}]\\}\right\\},$ (7) $\displaystyle\mathcal{D}$
$\displaystyle=\left\\{\boldsymbol{\delta}\colon\boldsymbol{\delta}=\\{\boldsymbol{\delta}\in\mathcal{D}_{ij}\forall
i\in[N],j\in[K_{i}]\\}\right\\},$ (8) $\displaystyle\widetilde{\mathcal{D}}$
$\displaystyle=\left\\{\widetilde{\boldsymbol{\theta}}^{d}\colon\widetilde{\boldsymbol{\theta}}^{d}=\\{\widetilde{\boldsymbol{\theta}}_{ij}^{d}\in\widetilde{\mathcal{D}}_{ij}\forall
i\in[N],j\in[K_{i}]\\}\right\\},$ (9)

we can re-write problem (5) as

$\displaystyle\displaystyle\min_{\mathbf{z},\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\boldsymbol{\delta}}\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)+I_{\mathcal{Z}}(\mathbf{z})+I_{\mathcal{C}}(\boldsymbol{\theta}^{c})+I_{\widetilde{\mathcal{D}}}(\widetilde{\boldsymbol{\theta}}^{d})+I_{\mathcal{D}}(\boldsymbol{\delta})\operatorname*{\text{subject
to}}\ \ \widetilde{\boldsymbol{\theta}}^{d}=\boldsymbol{\delta},$ (10)

with the corresponding augmented Lagrangian function

$\displaystyle\mathcal{L}$
$\displaystyle(\mathbf{z},\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\boldsymbol{\delta},\boldsymbol{\lambda})\mathrel{\mathop{:}}=$
(11)
$\displaystyle\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)+I_{\mathcal{Z}}(\mathbf{z})+I_{\mathcal{C}}(\boldsymbol{\theta}^{c})+I_{\widetilde{\mathcal{D}}}(\widetilde{\boldsymbol{\theta}}^{d})+I_{\mathcal{D}}(\boldsymbol{\delta})+\boldsymbol{\lambda}^{\top}\left(\widetilde{\boldsymbol{\theta}}^{d}-\boldsymbol{\delta}\right)+\frac{\rho}{2}\left\|\widetilde{\boldsymbol{\theta}}^{d}-\boldsymbol{\delta}\right\|_{2}^{2},$

where $\boldsymbol{\lambda}$ is the Lagrangian multiplier, and $\rho>0$ is a
penalty parameter for the augmented term.

ADMM (Boyd et al., 2011) alternatively minimizes the augmented Lagrangian
function (11) over two blocks of variables, leading to an efficient operator
splitting framework for nonlinear programs with nonsmooth objective function
and equality constraints. Hence ADMM solves the original problem (CASH) when
reformulated as problem (10) with a sequence of easier sub-problems.
Specifically, ADMM solves problem (CASH) by alternatively minimizing (11) over
variables
$\\{{\boldsymbol{\theta}^{c}},{\widetilde{\boldsymbol{\theta}}^{d}}\\}$, and
$\\{\boldsymbol{\delta},\mathbf{z}\\}$. This can be equivalently converted
into 3 sub-problems over variables
$\\{{\boldsymbol{\theta}^{c}},{\widetilde{\boldsymbol{\theta}}^{d}}\\}$,
$\boldsymbol{\delta}$ and $\mathbf{z}$, respectively. ADMM decomposes the
optimization variables into two blocks and alternatively minimizes the
augmented Lagrangian function (11) in the following manner at any ADMM
iteration $t$:

$\displaystyle\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\right\\}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}}\mathcal{L}\left(\mathbf{z}{}^{(t)},\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\boldsymbol{\delta}{}^{(t)},\boldsymbol{\lambda}{}^{(t)}\right)$
(12)
$\displaystyle\left\\{\boldsymbol{\delta}{}^{(t+1)},\mathbf{z}{}^{(t+1)}\right\\}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\delta},\mathbf{z}}\mathcal{L}\left(\mathbf{z},\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)},\boldsymbol{\delta},\boldsymbol{\lambda}{}^{(t)}\right)$
(13) $\displaystyle\boldsymbol{\lambda}{}^{(t+1)}$
$\displaystyle=\boldsymbol{\lambda}{}^{(t)}+\rho\left(\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}-\boldsymbol{\delta}{}^{(t+1)}\right).$
(14)

Problem (12) can be simplified by removing constant terms to get

$\displaystyle\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\right.$
$\displaystyle\left.\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\right\\}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}}\quad\widetilde{f}\left(\mathbf{z}{}^{(t)},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)+I_{\mathcal{C}}(\boldsymbol{\theta}^{c})+I_{\widetilde{\mathcal{D}}}(\widetilde{\boldsymbol{\theta}}^{d})$
(15)
$\displaystyle\quad\quad\quad\quad\quad\quad+\boldsymbol{\lambda}{}^{(t)}{}^{\top}\left(\widetilde{\boldsymbol{\theta}}^{d}-\boldsymbol{\delta}{}^{(t)}\right)+\frac{\rho}{2}\left\|\widetilde{\boldsymbol{\theta}}^{d}-\boldsymbol{\delta}{}^{(t)}\right\|_{2}^{2},$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}}\quad\widetilde{f}\left(\mathbf{z}{}^{(t)},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)+I_{\mathcal{C}}(\boldsymbol{\theta}^{c})+I_{\widetilde{\mathcal{D}}}(\widetilde{\boldsymbol{\theta}}^{d})+\frac{\rho}{2}\left\|\widetilde{\boldsymbol{\theta}}^{d}-\mathbf{b}\right\|_{2}^{2}$
(16) $\displaystyle\quad\quad\quad\quad\quad\quad\mbox{ where
}\mathbf{b}=\boldsymbol{\delta}{}^{(t)}-\frac{1}{\rho}\boldsymbol{\lambda}{}^{(t)}.$

A similar treatment to problem (13) gives us

$\displaystyle\left\\{\boldsymbol{\delta}{}^{(t+1)},\mathbf{z}{}^{(t+1)}\right\\}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\delta},\mathbf{z}}\quad\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\right\\}\right)+I_{\mathcal{Z}}(\mathbf{z})$
(17)
$\displaystyle\quad\quad\quad\quad\quad\quad+I_{\mathcal{D}}(\boldsymbol{\delta})+\boldsymbol{\lambda}{}^{(t)}{}^{\top}\left(\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}-\boldsymbol{\delta}\right)+\frac{\rho}{2}\left\|\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}-\boldsymbol{\delta}\right\|_{2}^{2},$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\delta},\mathbf{z}}\quad\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\right\\}\right)+I_{\mathcal{Z}}(\mathbf{z})$
(18)
$\displaystyle\quad\quad\quad\quad\quad+I_{\mathcal{D}}(\boldsymbol{\delta})+\frac{\rho}{2}\left\|\mathbf{a}-\boldsymbol{\delta}\right\|_{2}^{2}\mbox{
where
}\mathbf{a}=\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}+\frac{1}{\rho}\boldsymbol{\lambda}{}^{(t)}.$

This simplification exposes the independence between $\mathbf{z}$ and
$\boldsymbol{\delta}$, allowing us to solve problem (13) independently for
$\mathbf{z}$ and $\boldsymbol{\delta}$ as:

$\displaystyle\boldsymbol{\delta}{}^{(t+1)}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\delta}}\quad
I_{\mathcal{D}}(\boldsymbol{\delta})+\frac{\rho}{2}\left\|\mathbf{a}-\boldsymbol{\delta}\right\|_{2}^{2}\mbox{
where
}\mathbf{a}=\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}+\frac{1}{\rho}\boldsymbol{\lambda}{}^{(t)},$
(19) $\displaystyle\mathbf{z}{}^{(t+1)}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\mathbf{z}}\quad\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\right\\}\right)+I_{\mathcal{Z}}(\mathbf{z}).$
(20)

So we are able to decompose problem (3) into problems (16), (19) and (20)
which can be solved iteratively along with the $\boldsymbol{\lambda}^{(t)}$
updates (see Table 1).

##  D ADMM parameter sensitivity

The performance of ADMM is dependent on the choice of the penalty parameter
$\rho>0$ for the augmented term in the augmented Lagrangian (11). Figure 3
shows the dependence of adaptive precision ADMM on $\rho$ with the artificial
black-box objective (Appendix B). The results indicate that adaptive ADMM for
CASH is fairly robust to the choice of $\rho$.

(a) Run time in log scale

(b) Run time in linear scale

Figure 3: ADMM sensitivity to the penalty parameter $\rho$ with adaptive ADMM.
The performance is aggregated over 20 trials. The run time is dominated by the
black-box optimization scheme, which, in this case, is a Bayesian optimization
via Gaussian Process Regression.

##  E Empirical Evaluation of ADMM vs. JOPT on OpenML data

We compare the performance of the adaptive precision ADMM with different sub-
problem solvers to JOPT(BO) (BO on the joint problem with all the variables)
on 8 OpenML (Bischl et al., 2017) binary classification data sets where we
optimize for the area under the ROC curve on 10% of the data as the validation
set. Both adaptive precision ADMM and JOPT(BO) are executed for 1 hour with 10
repetitions, and the incumbent objective is averaged over the 10 repetitions.
All evaluations were run single-threaded on a 8 core 8GB CentOS virtual
machines. For each data set, we note the objective $f_{J}$ achieved by
JOPT(BO) at the end of 1 hour and note the time $T_{A}$ (in seconds) required
by adaptive precision ADMM to reach that objective. Then we also note the best
objective achieved by ADMM at the end of 1 hour $f_{A}$. The speedup $S$ for
ADMM is computed as $S=\frac{3600}{T_{A}}$ and the improvement (in %)
$I=100*\frac{f_{J}-f_{A}}{f_{J}}$.

We consider two versions of adaptive precision ADMM – (i) using BO for both
the ($\boldsymbol{\theta}$-min) and ($\mathbf{z}$-min) to get AdADMM(BO,BO),
and (ii) using BO for the ($\boldsymbol{\theta}$-min) and Combinatorial multi-
armed bandits (Ba) for the ($\mathbf{z}$-min) to get AdADMM(BO,Ba). The
improvements of ADMM over JOPT(BO) are summarized in Table 1, indicating
significant speedup (over $10\times$ in most cases) and further improvement
(over $10\%$ in many cases). Table 1 shows that between AdADMM(BO,BO) and
AdADMM(BO,Ba), the latter provides significantly higher speedups, but the
former provides higher additional improvement in the final objective. This
demonstrates ADMM’s flexibility, for example, allowing choice between faster
or more improved solution.

Dataset $\mathbf{S}_{\mbox{Ba}}$ $\mathbf{S}_{\mbox{BO}}$
$\mathbf{I}_{\mbox{Ba}}$ $\mathbf{I}_{\mbox{BO}}$ Bank8FM $10\times$ $2\times$
$0\%$ $5\%$ CPU small $4\times$ $5\times$ $0\%$ $5\%$ fri-c2 $153\times$
$25\times$ $56\%$ $64\%$ PC4 $42\times$ $5\times$ $8\%$ $13\%$ Pollen
$25\times$ $7\times$ $4\%$ $3\%$ Puma8NH $11\times$ $4\times$ $1\%$ $1\%$
Sylvine $9\times$ $2\times$ $9\%$ $26\%$ Wind $40\times$ $5\times$ $0\%$ $5\%$

Table 1: Comparing ADMM schemes to JOPT(BO), we list the speedup
$\mathbf{S}_{\mbox{Ba}}$ & $\mathbf{S}_{\mbox{BO}}$ achieved by AdADMM(BO,Ba)
& AdADMM(BO,BO) respectively to reach the best objective of JOPT, and the
final objective improvement $\mathbf{I}_{\mbox{Ba}}$ &
$\mathbf{I}_{\mbox{BO}}$ (respectively) over the JOPT objective. These numbers
are generated using the aggregate performance of JOPT and AdADMM over 10
trials.

##  F ADMM Decomposition for (CASH) with black-box constraints

Without loss of generality, we assume that $\epsilon_{m}\geq 0$ for $m\in[M]$.
By introducing scalars $u_{m}\in[0,\epsilon_{m}]$, we can reformulate the
inequality constraint (1) as the equality constraint together with a box
constraint

$g_{m}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\boldsymbol{\theta}^{d}\right\\}\right)-\epsilon_{m}+u_{m}=0,u_{m}\in[0,\epsilon_{m}],~{}m\in[M].$
(21)

We then introduce a continuous surrogate black-box functions
$\widetilde{g}_{m}$ for $g_{m},\forall m\in[M]$ in a similar manner to
$\widetilde{f}$ given by (4). Following the reformulation of (CASH) that lends
itself to the application of ADMM, the version with black-box constraints (21)
can be equivalently transformed into

$\displaystyle\min_{\mathbf{z},\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\boldsymbol{\delta}}\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)\operatorname*{\text{subject
to}}\left\\{\begin{array}[]{l}\mathbf{z}_{i}\in\\{0,1\\}^{K_{i}},\mathbf{1}^{\top}\mathbf{z}_{i}=1,\forall
i\in[N]\\\
\boldsymbol{\theta}_{ij}^{c}\in\mathcal{C}_{ij},\widetilde{\boldsymbol{\theta}}_{ij}^{d}\in\widetilde{\mathcal{D}}_{ij},\forall
i\in[N],j\in[K_{i}]\\\ \boldsymbol{\delta}_{ij}\in\mathcal{D}_{ij},\forall
i\in[N],j\in[K_{i}]\\\
\widetilde{\boldsymbol{\theta}}_{ij}^{d}=\boldsymbol{\delta}_{ij},\forall
i\in[N],j\in[K_{i}]\\\ u_{m}\in[0,\epsilon_{m}],\forall m\in[M]\\\
\widetilde{g}_{m}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)-\epsilon_{m}+u_{m}=0,\forall
m\in[M].\end{array}\right.$ (22)

Compared to problem (5), the introduction of auxiliary variables $\\{u_{m}\\}$
enables ADMM to incorporate black-box equality constraints as well as
elementary white-box constraints.

Defining
$\mathcal{U}=\\{\mathbf{u}\colon\mathbf{u}=\\{u_{m}\in[0,\epsilon_{m}]\forall
m\in[M]\\}\\}$, we can go through the mechanics of ADMM to get the augmented
Lagrangian with $\boldsymbol{\lambda}$ and $\mu_{m}\forall m\in[M]$ as the
Lagrangian multipliers and $\rho>0$ as the penalty parameter as follows:

$\begin{split}\mathcal{L}&\left(\mathbf{z},\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\boldsymbol{\delta},\mathbf{u},\boldsymbol{\lambda},\boldsymbol{\mu}\right)=\\\
&\widetilde{f}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)+I_{\mathcal{Z}}(\mathbf{z})+I_{\mathcal{C}}(\boldsymbol{\theta}^{c})+I_{\widetilde{\mathcal{D}}}(\widetilde{\boldsymbol{\theta}}^{d})+I_{\mathcal{D}}(\boldsymbol{\delta})+\boldsymbol{\lambda}^{\top}\left(\widetilde{\boldsymbol{\theta}}^{d}-\boldsymbol{\delta}\right)+\frac{\rho}{2}\left\|\widetilde{\boldsymbol{\theta}}^{d}-\boldsymbol{\delta}\right\|_{2}^{2}\\\
&+I_{\mathcal{U}}(\mathbf{u})+\sum_{i=1}^{M}\mu_{m}\left(\widetilde{g}_{m}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)-\epsilon_{m}+u_{m}\right)+\frac{\rho}{2}\sum_{i=1}^{M}\left(\widetilde{g}_{m}\left(\mathbf{z},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)-\epsilon_{m}+u_{m}\right)^{2}.\end{split}$

ADMM decomposes the optimization variables into two blocks for alternate
minimization of the augmented Lagrangian in the following manner at any ADMM
iteration $t$

$\displaystyle\left\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)},\mathbf{u}{}^{(t+1)}\right\\}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\mathbf{u}}\mathcal{L}\left(\mathbf{z}{}^{(t)},\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\boldsymbol{\delta}{}^{(t)},\mathbf{u},\boldsymbol{\lambda}{}^{(t)},\boldsymbol{\mu}{}^{(t)}\right)$
(23)
$\displaystyle\left\\{\boldsymbol{\delta}{}^{(t+1)},\mathbf{z}{}^{(t+1)}\right\\}$
$\displaystyle=\operatornamewithlimits{arg\,min}_{\boldsymbol{\delta},\mathbf{z}}\mathcal{L}\left(\mathbf{z},\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)},\boldsymbol{\delta},\mathbf{u}{}^{(t+1)},\boldsymbol{\lambda}{}^{(t)},\boldsymbol{\mu}{}^{(t)}\right)$
(24) $\displaystyle\boldsymbol{\lambda}{}^{(t+1)}$
$\displaystyle=\boldsymbol{\lambda}{}^{(t)}+\rho\left(\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}-\boldsymbol{\delta}{}^{(t+1)}\right)$
(25) $\displaystyle\forall m\in[M],\ \ \mu_{m}{}^{(t+1)}$
$\displaystyle=\mu_{m}{}^{(t)}+\rho\left(\widetilde{g}_{m}(\mathbf{z}{}^{(t+1)},\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\\})-\epsilon_{m}+u_{m}{}^{(t+1)}\right).$
(26)

Note that, unlike the unconstrained case, the update of the augmented
Lagrangian multiplier $\mu_{m}$ requires the evaluation of the black-box
function for the constraint $g_{m}$. Simplifying problem (23) gives us

$\begin{split}\min_{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d},\mathbf{u}}&\quad\widetilde{f}\left(\mathbf{z}{}^{(t)},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)\\\
&\quad\quad\quad+\frac{\rho}{2}\left[\left\|\widetilde{\boldsymbol{\theta}}^{d}-\mathbf{b}\right\|_{2}^{2}+\sum_{i=1}^{M}\left[\widetilde{g}_{m}\left(\mathbf{z}{}^{(t)},\left\\{\boldsymbol{\theta}^{c},\widetilde{\boldsymbol{\theta}}^{d}\right\\}\right)-\epsilon_{m}+u_{m}+\frac{1}{\rho}\mu_{m}{}^{(t)}\right]^{2}\right]\\\
&\operatorname*{\text{subject
to}}\left\\{\begin{array}[]{l}\boldsymbol{\theta}_{ij}^{c}\in\mathcal{C}_{ij}\forall
i\in[N],j\in[K_{i}],\\\
\widetilde{\boldsymbol{\theta}}_{ij}^{d}\in\widetilde{\mathcal{D}}_{ij}\forall
i\in[N],j\in[K_{i}],\\\ u_{m}\in[0,\epsilon_{m}],\end{array}\right.\mbox{
where
}\mathbf{b}=\boldsymbol{\delta}{}^{(t)}-\frac{1}{\rho}\boldsymbol{\lambda}{}^{(t)},\end{split}$
(27)

which can be further split into active and inactive set of continuous
variables based on the $\mathbf{z}{}^{(t)}$ as in the solution of problem (16)
(the $\boldsymbol{\theta}$-min problem). The main difference from the
unconstrained case in problem (16) (the $\boldsymbol{\theta}$-min problem) to
note here is that the black-box optimization with continuous variables now has
$M$ new variables $u_{m}$ ($M$ is the total number of black-box constraints)
which are active in every ADMM iteration. This problem (27) can be solved in
the same manner as problem (16) ($\boldsymbol{\theta}$-min) using SMBO or TR-
DFO techniques.

Simplifying and utilizing the independence of $\mathbf{z}$ and
$\boldsymbol{\delta}$, we can split problem (24) into the following problem
for $\boldsymbol{\delta}$

$\min_{\boldsymbol{\delta}}\quad\frac{\rho}{2}\|\boldsymbol{\delta}-{\mathbf{a}}\|_{2}^{2}\quad\operatorname*{\text{subject
to}}\boldsymbol{\delta}_{ij}\in\mathcal{D}_{ij}\forall
i\in[N],j\in[K_{i}]\mbox{ where
}\mathbf{a}=\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}+\frac{1}{\rho}\boldsymbol{\lambda}{}^{(t)},$
(28)

which remains the same as problem (19) (the $\boldsymbol{\delta}$-min problem)
in the unconstrained case, while the problem for $\mathbf{z}$ becomes

$\begin{split}\min_{\mathbf{z}}&\quad\widetilde{f}(\mathbf{z},\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\\})\\\
&\quad\quad\quad+\frac{\rho}{2}\sum_{i=1}^{M}\left[\widetilde{g}_{m}(\mathbf{z},\\{\boldsymbol{\theta}^{c}{}^{(t+1)},\widetilde{\boldsymbol{\theta}}^{d}{}^{(t+1)}\\})-\epsilon_{m}+u_{m}{}^{(t+1)}+\frac{1}{\rho}\mu_{m}{}^{(t)}\right]^{2}\\\
&\operatorname*{\text{subject
to}}\mathbf{z}_{i}\in\\{0,1\\}^{K_{i}},\mathbf{1}^{\top}\mathbf{z}_{i}=1,\forall
i\in[N].\end{split}$ (29)

The problem for $\mathbf{z}$ is still a black-box integer programming problem,
but now with an updated black-box function and can be handled with techniques
proposed for the combinatorial problem (20) in the absence of black-box
constraints (the $\mathbf{z}$-min problem).


# Quotients of Severi–Brauer surfaces

Andrey Trepalin

###### Abstract.

We show that a quotient of a non-trivial Severi–Brauer surface $S$ over
arbitrary field $\Bbbk$ of characteristic $0$ by a finite group
$G\subset\operatorname{Aut}(S)$ is $\Bbbk$-rational, if and only if $|G|$ is
divisible by $3$. Otherwise, the quotient is birationally equivalent to $S$.

Let $\Bbbk$ be an arbitrary field of characteristic zero, and
$\overline{\Bbbk}$ be its algebraic closure. A $d$-dimensional variety $X$ is
called a Severi–Brauer variety if $\overline{X}=X\otimes\overline{\Bbbk}$ is
isomorphic to $\mathbb{P}^{d}_{\overline{\Bbbk}}$. If $d=1$ then $X$ is a
conic, and if $d=2$ then $X$ is called a Severi–Brauer surface.

A Severi–Brauer variety is called non-trivial if it is not isomorphic to
$\mathbb{P}^{d}_{\Bbbk}$. It is well known that a Severi–Brauer variety is
trivial if and only if it has a $\Bbbk$-point. Moreover, if $X$ is a non-
trivial $d$-dimensional Severi–Brauer variety and $d+1$ is a prime number,
then degree of any point on $X$ is divisible by $d+1$ (see [Kol16, Theorem
53]).

Complete classification of finite subgroups of automorphism groups of non-
trivial Severi–Brauer surfaces is obtained in works of C. Shramov and V.
Vologodsky [ShV20], [Sh20a], [Sh20b], [Sh21]. Let ${\boldsymbol{\mu}}_{n}$ be
a cyclic group of order $n$. Then we have the following.

######  Theorem 1 (cf. [Sh21, Theorem 1.3(ii)]).

Let $S$ be a non-trivial Severi–Brauer surface over a field $\Bbbk$ of
characteristic zero. Then any finite subgroup of $\operatorname{Aut}(S)$ is
isomorphic to ${\boldsymbol{\mu}}_{n}$, ${\boldsymbol{\mu}}_{3n}$,
${\boldsymbol{\mu}}_{n}\rtimes{\boldsymbol{\mu}}_{3}$ or
${\boldsymbol{\mu}}_{3}\times\left({\boldsymbol{\mu}}_{n}\rtimes{\boldsymbol{\mu}}_{3}\right)$,
where $n$ is a positive integer divisible only by primes congruent to $1$
modulo $3$ (including the case $n=1$), and
${\boldsymbol{\mu}}_{n}\rtimes{\boldsymbol{\mu}}_{3}$ is a semidirect product
corresponding to an outer automorphism of ${\boldsymbol{\mu}}_{n}$ of order
$3$ acting non-trivially on each non-trivial element of
${\boldsymbol{\mu}}_{n}$.

Moreover, for any group $G$ mentioned above there exists a field $\Bbbk$ of
charectiristic zero and a non-trivial Severi–Brauer surface $S$ over $\Bbbk$,
such that $G\subset\operatorname{Aut}(S)$.

The aim of this paper is to obtain a birational classification of quotients of
non-trivial Severi–Brauer surfaces by finite subgroups of automorphism groups.
In particular, we want to answer a question, for which Severi–Brauer surfaces
$S$ and finite subgroups $G\subset\operatorname{Aut}(S)$ the quotient $S/G$ is
$\Bbbk$-rational (i.e. birationally equivalent to $\mathbb{P}^{2}_{\Bbbk}$).
Note that for an algebraically closed field $\overline{\Bbbk}$ of
characteristic zero quotients of $\overline{\Bbbk}$-rational surfaces by
finite automorphism groups are always $\overline{\Bbbk}$-rational by
Castelnuovo’s rationality criterion (see [Cast94]). Moreover, for arbitrary
field $\Bbbk$ of characteristic zero quotients of $\mathbb{P}^{2}_{\Bbbk}$ by
finite automorphism groups are always $\Bbbk$-rational (see [Tr14, Theorem
1.3]). One can find more results about rationality of quotients of
$\Bbbk$-rational surfaces by finite automorphism groups in [Tr19]. The main
result of this paper is the following theorem.

######  Theorem 2.

Let $S$ be a non-trivial Severi–Brauer surface over a field $\Bbbk$ of
characteristic zero, and $G$ be a finite subgroup of $\operatorname{Aut}(S)$.
Then the quotient $S/G$ is $\Bbbk$-rational, if and only if $|G|$ is divisible
by $3$. Otherwise, the quotient $S/G$ is birationally equivalent to $S$.

This theorem is an analogue of the following proposition.

######  Proposition 3.

Let $C$ be a conic over a field $\Bbbk$ of characteristic zero without
$\Bbbk$-points, and $G$ be a finite subgroup of $\operatorname{Aut}(C)$. Then
the quotient $C/G$ is isomorphic to $\mathbb{P}^{1}_{\Bbbk}$, if and only if
$G$ has even order.

###### Proof.

The degree of any point on $C$ is even. Assume that the order of $G$ is odd.
Then any $G$-orbit on the set of geometric points on $C$ has odd cardinality.
Thus the image of any such orbit on $C/G$ is not defined over $\Bbbk$, and the
quotient $C/G$ does not have $\Bbbk$-points and is not isomorphic to
$\mathbb{P}^{1}_{\Bbbk}$.

Now assume that the order of $G$ is even. Therefore there is an element $g\in
G$ of order $2$. The anticanonical map given by the linear system $|-K_{C}|$
gives an embedding $C\hookrightarrow\mathbb{P}^{2}_{\Bbbk}$, such that the
action of $g$ on $C$ induces an action of $g$ on $\mathbb{P}^{2}_{\Bbbk}$. One
can choose coordinates $(x:y:z)$ on $\mathbb{P}^{2}_{\Bbbk}$ such that the
action of $g$ is given by $(x:y:z)\mapsto(x:y:-z)$. Then the quotient map
$C\rightarrow C/\langle g\rangle$ is just a restriction of the projection
$\mathbb{P}^{2}_{\Bbbk}\dashrightarrow\mathbb{P}^{1}_{\Bbbk}$ given by
$(x:y:z)\mapsto(x:y)$.

Therefore some $\langle g\rangle$-orbits are defined over $\Bbbk$, and thus
some $G$-orbits are defined over $\Bbbk$ too. It means that there are
$\Bbbk$-points on the quotient $C/G$, and therefore this quotient is
isomorphic to $\mathbb{P}^{1}_{\Bbbk}$, since one has
$\mathbb{P}^{1}_{\overline{\Bbbk}}/G\cong\mathbb{P}^{1}_{\overline{\Bbbk}}$
for any finite subgroup
$G\subset\operatorname{Aut}\left(\mathbb{P}^{1}_{\overline{\Bbbk}}\right)$.

∎

The proof of Theorem 2 is more complicated, and we decompose it into several
lemmas.

######  Lemma 4.

Let $S$ be a non-trivial Severi–Brauer surface over a field $\Bbbk$ of
characteristic zero and $G$ be a finite subgroup of $\operatorname{Aut}(S)$
isomorphic to ${\boldsymbol{\mu}}_{n}$. Then the set of $G$-fixed points on
$S$ consists of three isolated geometric points. If $n>3$ then these points
are defined over extension $K/\Bbbk$ of degree $3$.

###### Proof.

Consider the action of $G$ on
$\overline{S}\cong\mathbb{P}^{2}_{\overline{\Bbbk}}$. This action can be
diagonalized in $\mathrm{PGL}_{3}\left(\overline{\Bbbk}\right)$. One can see
that the set of fixed points of $G$ either consists of an isolated fixed point
and a line, or consists of three isolated points $p_{1}$, $p_{2}$ and $p_{3}$.
In the former case the isolated fixed point is unique, and therefore defined
over $\Bbbk$, but there are no $\Bbbk$-points on $S$. Therefore $G$ has three
isolated fixed geometric points on $S$.

Isolated fixed geometric points of $G$ are transitevely permuted by the Galois
group $\operatorname{Gal}\left(\overline{\Bbbk}/\Bbbk\right)$. Therefore these
points are defined either over extension $K/\Bbbk$ of degree $3$ with
$\operatorname{Gal}\left(K/\Bbbk\right)\cong{\boldsymbol{\mu}}_{3}$, or over
extension $K/\Bbbk$ of degree $6$ with
$\operatorname{Gal}\left(K/\Bbbk\right)\cong\mathfrak{S}_{3}$, where
$\mathfrak{S}_{3}$ is a non-abelian group of order $6$. In particular, there
is an element $\gamma\in\operatorname{Gal}\left(\overline{\Bbbk}/\Bbbk\right)$
of order $3$, such that $\gamma p_{1}=p_{2}$, $\gamma p_{2}=p_{3}$, $\gamma
p_{3}=p_{1}$.

If $n>3$ then there is a subgroup ${\boldsymbol{\mu}}_{p}\subset G$ for a
prime number $p$ such that $p$ equals $1$ modulo $3$. The action of a
generator of ${\boldsymbol{\mu}}_{p}$ on
$\overline{S}\cong\mathbb{P}^{2}_{\overline{\Bbbk}}$ can be written as
$\operatorname{diag}(\xi_{p};\xi_{p}^{a};1)$, where $\xi_{p}$ is a $p$-th root
of unity. This element acts on the tangent spaces $T_{p_{1}}\overline{S}$,
$T_{p_{2}}\overline{S}$ and $T_{p_{3}}\overline{S}$ as
$\operatorname{diag}\left(\xi_{p};\xi_{p}^{a}\right)$,
$\operatorname{diag}\left(\xi_{p}^{a-1};\xi_{p}^{-1}\right)$ and
$\operatorname{diag}\left(\xi_{p}^{-a};\xi_{p}^{1-a}\right)$ respectively.

One has $\gamma p_{1}=p_{2}$. Thus $\gamma\left(\xi_{p}\right)=\xi_{p}^{a-1}$,
and

$\xi_{p}^{-1}=\gamma\left(\xi_{p}^{a}\right)=\gamma\left(\xi_{p}\right)^{a}=\xi_{p}^{a(a-1)}.$

Therefore $a^{2}-a+1=0$ modulo $p$. In particular, $a\neq 1$ and $a\neq-1$
modulo $p$.

Assume that the Galois group
$\operatorname{Gal}\left(\overline{\Bbbk}/\Bbbk\right)$ contains an element
$\delta$ such that $\delta p_{1}=p_{1}$, $\delta p_{2}=p_{3}$ and $\delta
p_{3}=p_{2}$. Then $\delta\left(\xi_{p}\right)=\xi_{p}^{a}$, and
$\delta\left(\xi_{p}^{2}\right)=\xi_{p}$. Therefore $a^{2}=1$ modulo $p$. But
$a\neq\pm 1$ modulo $p$. We have a contradiction. Hence if $n>3$ then the
points $p_{1}$, $p_{2}$ and $p_{3}$ are defined over an extension $K/\Bbbk$ of
degree $3$. ∎

######  Lemma 5.

Let $S$ be a non-trivial Severi–Brauer surface over a field $\Bbbk$ of
characteristic zero and $G$ be a finite subgroup of $\operatorname{Aut}(S)$
isomorphic to ${\boldsymbol{\mu}}_{n}$, where $n$ is a positive integer
divisible only by primes congruent to $1$ modulo $3$. Then the quotient $S/G$
is birationally equivalent to $S$.

###### Proof.

Any point on a non-trivial Severi–Brauer surface has degree divisible by $3$.
Therefore there are no $\Bbbk$-points on $S/G$, since cardinalities of
$G$-orbits are not divisible by $3$.

For $n=1$ the assertion of the lemma is trivial. Therefore assume that $n>1$.

By Lemma 4 the set of $G$-fixed points on $S$ consists of three isolated
geometric points defined over an extension $K/\Bbbk$ of degree $3$. Any cyclic
group acting on $\overline{S}\cong\mathbb{P}^{2}_{\overline{\Bbbk}}$ is a
subgroup of a torus acting on $\overline{S}$. Therefore the quotient $S/G$ is
a $\Bbbk$-form of a toric surface with three singular points, that are images
of $G$-fixed points on $S$. The Galois group
${\boldsymbol{\mu}}_{3}\cong\operatorname{Gal}\left(K/\Bbbk\right)$ acts on
the fan of the corresponding toric surface. Let us resolve the singularities
of $S/G$ and run ${\boldsymbol{\mu}}_{3}$-equivariant minimal model program.
As a result we should obtain ${\boldsymbol{\mu}}_{3}$-minimal del Pezzo
surface or conic bundle $S^{\prime}$, such that $\overline{S}^{\prime}$ is a
toric surface. Any toric del Pezzo surface or conic bundle $X$ has
$K_{X}^{2}\geqslant 6$, and one can easily check that for such surface a group
${\boldsymbol{\mu}}_{3}$ can faithfully act only on the fan of a del Pezzo
surface of degree $9$ or $6$. Moreover, del Pezzo surface of degree $6$ is not
${\boldsymbol{\mu}}_{3}$-minimal, since one can
${\boldsymbol{\mu}}_{3}$-equivariantly blow down a triple of disjoint
$(-1)$-curves. Thus $S^{\prime}$ is a del Pezzo surface of degree $9$ without
$\Bbbk$-points, i.e. non-trivial Severi–Brauer surface.

The existence of rational map $S\dashrightarrow S^{\prime}$ implies that the
class of $S^{\prime}$ in the Brauer group $\mathrm{Br}\left(\Bbbk\right)$ lies
in the cyclic group generated by the class of $S$ (see [GS18, Exercise
3.3.8(iii)]). Therefore $S^{\prime}$ is either isomorphic to
$\mathbb{P}^{2}_{\Bbbk}$, that is impossible since there are no $\Bbbk$-points
on $S^{\prime}$, or isomorphic to $S$, or isomorphic to $S^{op}$, where
$S^{op}$ is the Severi–Brauer surface corresponding to the central simple
algebra opposite to the one corresponding to $S$. The surface $S^{op}$ is
birationally equivalent to $S$ (see [Ami55]). Thus $S/G$ is birationally
equivalent to $S$.

∎

######  Remark 6.

Note that the proof of Lemma 5 can be easily generalized on the case where
$N\cong{\boldsymbol{\mu}}_{n}$ is a normal subgroup of a finite group $G$,
such that $G$ is isomorphic to ${\boldsymbol{\mu}}_{3n}$,
${\boldsymbol{\mu}}_{n}\rtimes{\boldsymbol{\mu}}_{3}$ or
${\boldsymbol{\mu}}_{3}\times\left({\boldsymbol{\mu}}_{n}\rtimes{\boldsymbol{\mu}}_{3}\right)$.
In this case the quotient $S/N$ is $G/N$-birationally equivalent to a
Severi–Brauer surface $S^{\prime}$, since the action of $G/N$ on the fan
corresponding to $S/N$ is either trivial, or coincides with the action of
${\boldsymbol{\mu}}_{3}\cong\operatorname{Gal}\left(K/\Bbbk\right)$.

Now assume that a group ${\boldsymbol{\mu}}_{3}$ acts on a non-trivial
Severi–Brauer surface $S$. Then by Lemma 4 the group ${\boldsymbol{\mu}}_{3}$
has three isolated fixed geometric points defined either over extension
$K/\Bbbk$ of degree $3$ with
$\operatorname{Gal}\left(K/\Bbbk\right)\cong{\boldsymbol{\mu}}_{3}$, or over
extension $K/\Bbbk$ of degree $6$ with
$\operatorname{Gal}\left(K/\Bbbk\right)\cong\mathfrak{S}_{3}$. We consider
these two cases separately, and show that in the both cases the quotient
$S/{\boldsymbol{\mu}}_{3}$ is $\Bbbk$-rational.

######  Lemma 7.

Let $S$ be a non-trivial Severi–Brauer surface and $G$ be a finite subgroup of
$\operatorname{Aut}(S)$ isomorphic to ${\boldsymbol{\mu}}_{3}$. Assume that
the isolated fixed points of $G$ are defined over extension $K/\Bbbk$ of
degree $3$ with
$\operatorname{Gal}\left(K/\Bbbk\right)\cong{\boldsymbol{\mu}}_{3}$. Then the
quotient $S/G$ is $\Bbbk$-rational.

###### Proof.

Let $p_{1}$, $p_{2}$ and $p_{3}$ be the isolated fixed geometric points of $G$
on $S$. One can $G$-equivariantly blow up this triple of points and get a del
Pezzo surface $\widetilde{S}$ of degree $6$. Note that each of six
$(-1)$-curves is $G$-invariant, therefore the six points of intersection of
these six curves are $G$-fixed. One can easily check that on the tangent space
of $\overline{\widetilde{S}}$ at each of these points the group $G$ acts as
$\langle\operatorname{diag}\left(\omega;\omega\right)\rangle$, where $\omega$
is a third root of unity.

The group $\operatorname{Gal}\left(K/\Bbbk\right)\cong{\boldsymbol{\mu}}_{3}$
does not preserve any $G$-fixed point on $\widetilde{S}$. Therefore one can
blow up any triple of $G$-fixed points defined over $\Bbbk$, and get a surface
$\widetilde{X}$. One has $K_{\widetilde{X}}^{2}=3$, the exceptional divisor of
the blowup $\widetilde{X}\rightarrow\widetilde{S}$ is a triple of pointwisely
$G$-fixed $(-1)$-curves $E_{1}$, $E_{2}$ and $E_{3}$, and the proper
transforms of the six $(-1)$-curves on $\widetilde{S}$ are $(-2)$-curves on
$\widetilde{X}$. One can check that $\widetilde{X}$ is a weak del Pezzo
surface, and the anticanonical map

$\varphi_{|-K_{\widetilde{X}}|}:\widetilde{X}\rightarrow X$

blows down the six $(-2)$-curves into three $A_{2}$-singularities. Moreover,
$X$ is a singular cubic surface, and the images of $E_{1}$, $E_{2}$ and
$E_{3}$ are lines passing through pairs of singular points.

The linear systems $|-K_{\widetilde{S}}|$, $|-K_{\widetilde{X}}|$ and
$|-K_{X}|$ are $G$-invariant, since the morphisms $\widetilde{S}\rightarrow
S$, $\widetilde{X}\rightarrow\widetilde{S}$ and $\widetilde{X}\rightarrow X$
are $G$-equivariant. Therefore the action of $G$ on $X$ induces an action of
$G$ on $\mathbb{P}^{3}_{\Bbbk}$. Moreover, $G$ pointwisely fixes a plane in
$\mathbb{P}^{3}_{\Bbbk}$ since it fixes three lines meeting each other.
Therefore one can choose coordinates $(x:y:z:t)$ on $\mathbb{P}^{3}_{\Bbbk}$
such that the action of $G$ is given by $(x:y:z:t)\mapsto(x:y:z:\omega t)$.
Then the quotient map $X\rightarrow X/G$ is just a restriction of the
projection $\mathbb{P}^{3}_{\Bbbk}\dashrightarrow\mathbb{P}^{2}_{\Bbbk}$ given
by

$(x:y:z:t)\mapsto(x:y:z)$

(cf. the proof of Proposition 3). Therefore $X/G\cong\mathbb{P}^{2}_{\Bbbk}$,
and $S/G$ is $\Bbbk$-rational, since $S/G$ and $X/G$ are birationally
equivalent.

∎

For the remaining case
$\operatorname{Gal}\left(K/\Bbbk\right)\cong\mathfrak{S}_{3}$ we need the
following lemma.

######  Lemma 8.

Let $S^{\prime}$ be a del Pezzo surface of degree $6$ over arbitrary field
$\Bbbk$ of characteristic $0$. If $S^{\prime}$ contains a point of degree $2$
and a point degree $3$, then $S^{\prime}$ is $\Bbbk$-rational.

###### Proof.

Let $L$ be an extension of degree $2$ of $\Bbbk$, such that there is a
$L$-point on $S^{\prime}\otimes L$. Then $S^{\prime}\otimes L$ is
$L$-rational, therefore we can find a pair of geometric points $p_{1}$ and
$p_{2}$ on $S^{\prime}$ defined over $L$ and not lying on $(-1)$-curves.

Let us consider the blowup $X\rightarrow S^{\prime}$ at $p_{1}$ and $p_{2}$.
One can check that the surface $X$ is a (possibly, weak) del Pezzo surface of
degree $4$, and there is a point of degree $3$ on $X$. The anticanonical map
given by $|-K_{X}|$ maps $X$ to $X^{\prime}\subset\mathbb{P}^{4}_{\Bbbk}$,
where $X^{\prime}$ is a (possibly, singular) del Pezzo surface of degree $4$.
The arguments of the proof of [Sh20c, Lemma 2.4] show that in this case
$X^{\prime}$ has a smooth $\Bbbk$-point. For convenience of the reader, we
give the other proof of this result.

Indeed, let $q_{1}$, $q_{2}$ and $q_{3}$ be three geometric points on
$X^{\prime}$, such that this triple is defined over $\Bbbk$. Such triple
exists since $S^{\prime}$ contains a point of degree $3$. Consider a plane
$\Pi$ defined over $\Bbbk$ and passing through $q_{1}$, $q_{2}$ and $q_{3}$.
The intersection number $\Pi\cdot X^{\prime}=4$, and the multiplicity of the
intersection is the same at the points $q_{1}$, $q_{2}$ and $q_{3}$.
Therefore, if $\Pi\cap X^{\prime}$ is zero-dimensional, then there is an other
smooth $\Bbbk$-point $q$ on $\Pi\cap X^{\prime}$.

If $\Pi\cap X^{\prime}$ is one-dimensional, then the intersection
$C=X^{\prime}\cap\Pi$ is a conic, since $X^{\prime}$ is an intersection of two
quadrics in $\mathbb{P}^{4}_{\Bbbk}$. If $C$ is smooth, then there are
infinitely many $\Bbbk$-points on $C$, since there is a point of degree $3$ on
$C$. If $C$ is a union of two lines then the points $q_{1}$, $q_{2}$ and
$q_{3}$ have to lie on one of these lines. Therefore this line is defined over
$\Bbbk$. If $C$ is a double line, then this line is defined over $\Bbbk$. In
each case we find infinitely many $\Bbbk$-points on $C$, thus there is a
smooth $\Bbbk$-point on $X^{\prime}$.

The image of any $\Bbbk$-point on $X^{\prime}$ under the map
$X^{\prime}\rightarrow S^{\prime}$ is a $\Bbbk$-point on $S^{\prime}$.
Therefore $S^{\prime}$ is $\Bbbk$-rational by [Isk96, Chapter 4].

∎

######  Lemma 9.

Let $S$ be a non-trivial Severi–Brauer surface and $G$ be a finite subgroup of
$\operatorname{Aut}(S)$ isomorphic to ${\boldsymbol{\mu}}_{3}$. Assume that
the isolated fixed points of $G$ are defined over extension $K/\Bbbk$ of
degree $6$ with $\operatorname{Gal}\left(K/\Bbbk\right)\cong\mathfrak{S}_{3}$.
Then the quotient $S/G$ is $\Bbbk$-rational.

###### Proof.

The group $G$ has three isolated fixed points on $S$, that are defined over
$K$. Therefore the quotient $S/G$ is a $\Bbbk$-form of a singular toric
surface, such that the three singular points are not defined over $\Bbbk$, and
defined over $K$. The Galois group
$\mathfrak{S}_{3}\cong\operatorname{Gal}\left(K/\Bbbk\right)$ acts on the fan
of the corresponding toric surface. Let us resolve the singularities of $S/G$
and run $\mathfrak{S}_{3}$-equivariant minimal model programm. As a result we
should obtain $\mathfrak{S}_{3}$-minimal del Pezzo surface or conic bundle
$S^{\prime}$, such that $\overline{S}^{\prime}$ is a toric surface. Any toric
del Pezzo surface or conic bundle $X$ has $K_{X}^{2}\geqslant 6$, and one can
easily check that for such surface a group $\mathfrak{S}_{3}$ can faithfully
act only on the fan of a del Pezzo surface of degree $9$ or $6$.

Let us show that $S^{\prime}$ is $\Bbbk$-rational. Note that there is a normal
subgroup ${\boldsymbol{\mu}}_{3}$ in the group
$\mathfrak{S}_{3}\cong\operatorname{Gal}\left(K/\Bbbk\right)$, therefore there
exist an extension $L=K^{{\boldsymbol{\mu}}_{3}}$ of $\Bbbk$ of degree $2$,
such that the isolated fixed points of $G$ are not defined over $L$, and
defined over the extension $K/L$ of degree $3$. Thus $S^{\prime}\otimes
L\cong\left(S\otimes L\right)/G$ is $L$-rational by Lemma 7. So there is a
point of degree $2$ on $S^{\prime}$. If $S^{\prime}$ is a del Pezzo surface of
degree $9$ then $S^{\prime}$ is $\Bbbk$-rational, since the line passing
through a point of degree $2$ is defined over $\Bbbk$.

If $S^{\prime}$ is a del Pezzo surface of degree $6$ then there is a point of
degree $3$ on $S^{\prime}$, since the image of any point of degree $3$ on $S$
under the quotient map $S/G$ is either a $\Bbbk$-point, or a point of degree
$3$. Therefore $S^{\prime}$ is $\Bbbk$-rational by Lemma 8.

∎

Now we can prove Theorem 2.

###### Proof of Theorem 2.

A finite subgroup $G$ of $\operatorname{Aut}(S)$ is isomorphic to
${\boldsymbol{\mu}}_{n}$, ${\boldsymbol{\mu}}_{3n}$,
${\boldsymbol{\mu}}_{n}\rtimes{\boldsymbol{\mu}}_{3}$ or
${\boldsymbol{\mu}}_{3}\times\left({\boldsymbol{\mu}}_{n}\rtimes{\boldsymbol{\mu}}_{3}\right)$
by Theorem 1.

If there is a subgroup $N\cong{\boldsymbol{\mu}}_{n}$ in $G$, where $n$ is a
positive integer divisible only by primes congruent to $1$ modulo $3$, then
$N$ is normal in $G$, and the quotient $S/N$ is $G/N$-birationally equivalent
to a non-trivial Severi–Brauer surface $S^{\prime}$ with the action of $G/N$
by Lemma 5 and Remark 6. In particular, if $G\cong{\boldsymbol{\mu}}_{n}$,
then $S/G$ is birationally equivalent to $S$ by Lemma 5.

The other three cases of $G$ are reduced to the case, where a group
${\boldsymbol{\mu}}_{3}$ or ${\boldsymbol{\mu}}_{3}^{2}$ acts on a non-trivial
Severi–Brauer surface, since $S/G$ and $S^{\prime}/(G/N)$ are birationally
equivalent.

The quotient of Severi–Brauer surface $S$ by ${\boldsymbol{\mu}}_{3}$ is
$\Bbbk$-rational by Lemmas 7 and 9, and the quotient
$S/{\boldsymbol{\mu}}_{3}^{2}$ is birationally equivalent to a quotient of a
toric surface $S/{\boldsymbol{\mu}}_{3}$ by a cyclic group
${\boldsymbol{\mu}}_{3}$. This quotient is $\Bbbk$-rational by [Tr14,
Proposition 4.5].

∎

Acknowledgements. The author is grateful to Costya Shramov and Sergey
Gorchinskiy for many useful discussions and comments.

## References

  * [Ami55] S. A. Amitsur,  Generic splitting fields of central simple algebras, Ann. Math., 62:2 (1955), 8–43
  * [Cast94] G. Castelnuovo,  Sulla razionalità delle involuzioni piane,  Math. Ann., 1894, 44, 125–155
  * [GS18] S. Gorchinskiy, C. Shramov,  Unramified Brauer group and its applications,  Translations of Mathematical Monographs, 246, American Mathematical Society, Providence, 2018, xvii+179 pp.
  * [Isk96] V. A. Iskovskikh,  Factorization of birational mappings of rational surfaces from the point of view of Mori theory,  Uspekhi Mat. Nauk, 1996, 51, 3–72 (in Russian); translation in Russian Math. Surveys, 1996, 51, 585–652
  * [Kol16] J. Kollár,  Severi–Brauer varieties; a geometric treatment,  preprint, arXiv:1606.04368 (2016)
  * [Sh20a] C. A. Shramov,  Birational automorphisms of Severi–Brauer surfaces,  Sb. Math., 211:3 (2020), 466–480
  * [Sh20b] C. A. Shramov,  Non-abelian groups acting on Severi–Brauer surfaces,  Mat. Zametki, 108:6 (2020), 916–917
  * [Sh20c] C. Shramov,  Automorphisms of cubic surfaces without points,  Int. J. Math., 31:11 (2020), 2050083, 15 pp.
  * [Sh21] C. Shramov,  Finite groups acting on Severi–Brauer surfaces,  Eur. J. Math., 7:2 (2021), 591–612
  * [ShV20] C. Shramov, V. Vologodsky,  Boundedness for finite subgroups of linear algebraic groups,  preprint, arXiv:2009.14485 (2020)
  * [Tr14] A. S. Trepalin,  Rationality of the quotient of $\mathbb{P}^{2}$ by finite group of automorphisms over arbitrary field of characteristic zero,  Cent. Eur. J. Math., 12:2 (2014), 229–239
  * [Tr19] A. Trepalin,  Quotients of del Pezzo surfaces,  Int. J. Math., 30:11 (2019), 1950068, 40 pp.