11institutetext: Department of Mathematics, Birla Institute of Technology and
Science-Pilani, Hyderabad Campus, Hyderabad-500078, India.
E-mail: G. Gadbail<EMAIL_ADDRESS>S. Arora:
<EMAIL_ADDRESS>P.K. Sahoo<EMAIL_ADDRESS>
# Power-law cosmology in Weyl-type $f(Q,T)$ gravity

Gaurav Gadbail  0000-0003-0684-9702 Simran Arora  0000-0003-0326-8945 P.K.
Sahoo  0000-0003-2130-8832

(Received: 27th July 2021 / Revised version: 30th Sept. 2021)

###### Abstract

Gravity is attributed to the spacetime curvature in classical General
Relativity (GR). But, other equivalent formulation or representations of GR,
such as torsion or non-metricity have altered the perception. We consider the
Weyl-type $f(Q,T)$ gravity, where $Q$ represents the non-metricity and $T$ is
the trace of energy momentum temsor, in which the vector field $\omega_{\mu}$
determines the non-metricity $Q_{\mu\nu\alpha}$ of the spacetime. In this
work, we employ the well-motivated $f(Q,T)=\alpha Q+\frac{\beta}{6k^{2}}T$,
where $\alpha$ and $\beta$ are the model parameters. Furthermore, we assume
that the universe is dominated by the pressure-free matter, i.e. the case of
dust ($p=0$). We obtain the solution of field equations similar to a power-law
in Hubble parameter $H(z)$. We investigate the cosmological implications of
the model by constraining the model parameter $\alpha$ and $\beta$ using the
recent 57 points Hubble data and 1048 points Pantheon supernovae data. To
study various dark energy models, we use statefinder analysis to address the
current cosmic acceleration. We also observe the $Om$ diagnostic describing
various phases of the universe. Finally, it is seen that the solution which
mimics the power-law fits well with the Pantheon data better than the Hubble
data.

###### pacs:

04.50.Kd and 98.80.Es98.80.Cq

##  1 Introduction

The accelerated expansion becomes a prominent theme in modern cosmology,
confirming various observable evidence such as type Ia supernovae observations
Riess/1998 ; Perlmutter/1999 ; Spergel/2007 , baryon acoustic oscillations
Eisenstein/2005 ; Cole/2005 , and large-scale structure Hawkins/2003 . The
incorporation of Riemann geometry into General Relativity (GR), the most
successful theory has provided a robust mathematical framework for describing
gravitational field properties. However, recent observational data have raised
some concerns about the classical GR absolute validity, which may still have
some limitations on large or solar system scales. The two significant
challenges confronting modern gravitational theories are the dark energy and
dark matter problems, which aid in the accelerated expansion of the universe.
Modifying the gravitational part of the Einstein’s equations is one of the
approaches for explaining the acceleration. This approach is named as the
modified theory of gravity. So far, several modified theories of gravity
beyond GR have been proposed such as the $f(R)$ gravity Buchdahl/1970 ;
Capozziello/2011 ; Nojiri/2003 , the $f(R,T)$ gravity Arora/2020 ; Harko/2011
; Moraes/2017 ; Yousaf/2018 , the $f(R,G)$ gravity Atazadeh/2014 ;
Laurentis/2015 , the $f(T,B)$ gravity Bahamonde/2018 ; Capo/2020  etc.
Within the geometrical framework, the curvature is not the only geometrical
object, torsion and non-metricity are the other two fundamental objects
related to the connection of a metric space. There are three equivalent
representations of GR. The curvature representation in which the torsion and
the non-metricity are zero. The second is the teleparallel representation
based entirely on the torsion. And there is the symmetric teleparallel
representation, in which the non-metricity is associated with gravity. Weyl
suggested a Riemannian extension after GR in an effort to combine gravity and
electromagnetism Weyl/1918 . The non-metricity of spacetime produces the
electromagnetic field. Under parallel transport, both the orientation and the
length of vectors vary. The dilatational gauge vector, often known as the Weyl
vector, is the new vector part of the connection established. There is a scale
transformation that transforms this vector to zero when it is given by the
gradient of a function. As a result, the lengths of parallel transported
vectors through closed pathways return undisturbed, resulting in an integrable
Weyl geometry Scholz/2017 ; Wheeler/2018 . Higher symmetry techniques to
gravity also use Weyl geometry. Moreover, Weizenb$\ddot{o}$ck constructed a
geometry with torsion and zero Riemann curvature, another essential
mathematical breakthrough with applications Weitzenbock/1923 . The primary
idea behind the teleparallel formulation of gravity is to use tetrad vectors
to replace the metric $g_{\mu\nu}$ of the spacetime that describes the
gravitational field. It is named as the teleparallel equivalent to GR or
$f(T)$ gravity, where $T$ is the torsion Hayashi/1979 . In recent years, the
$f(T)$ theory has yielded some captivating cosmological behaviors, which have
been studied in the literature Benetti/2021 ; Capozziello/2011a ; Mandal/2020
; Myrzakulov/2011 . The other geometrically equivalent to GR known as the
symmetric teleparallel gravity, was also introduced and further developed into
$f(Q)$ gravity Jimenez/2018 , where the non-metricity $Q$ of a Weyl geometry
represents the basic geometrical variable describing the variation of the
length of a vector in the parallel transport. Lazkoz et al. Lazkoz/2019
analysed different forms of $f(Q)$ gravity to study an accelerated expansion
of the universe with recent observations. The behavior of cosmological
solutions and growth index of matter perturbations in $f(Q)$ gravity has also
been investigated in Khyllep/2021 . Mandal et al. Mandal/2020a  also studied
cosmography in $f(Q)$ gravity. Another recent extension of $f(Q)$ gravity
known as $f(Q,T)$ gravity Xu/2019 ; 2020  includes a non-minimal coupling in
the gravitational action, in which the Lagrangian is replaced by an arbitrary
function $f$ of the non-metricity $Q$ and the trace of the energy-momentum
tensor $T$. Many studies have demonstrated that $f(Q,T)$ gravity is viable
option for explaining current cosmic acceleration and can provide a consistent
solution to the dark energy problem. Arora et al. Arora/2021  analysed the
feasibility of $f(Q,T)$ gravity by constraining an effective equation of state
explaining the dark sector of the universe. $f(Q,T)$ gravity also contributes
significantly to gravitational baryogenesis Bhattacharjee/2020 . The energy
conditions in $f(Q,T)$ gravity was studied in Arora/2021b  In the context of
proper Weyl geometry, Yixin et al. Yixin/2020b  explored $f(Q,T)$ gravity and
adopted the explicit equation for non-metricity $Q$ that follows the non-
conservation of the metric tensor divergence. Furthermore, the field equations
in the theory were derived using the vanishing scalar curvature condition,
which was then applied to the gravitational action via the Lagrange
multiplier. The Weyl type $f(Q,T)$ gravity has been found to be an alternate
and effective way of describing accelerated and decelerated phases of the
universe. Studying variety of functional forms and model parameters, Weyl
$f(Q,T)$ could provide a strong alternative to the $\Lambda$CDM, especially
giving the late-time de sitter phase generated by Weyl geometry. Yang et al.
Yang/2021  used the Weyl $f(Q,T)$ theory to derive the geodesic and the
Raychaudhuri equations . The analysis here aims to investigate if the Weyl
type $f(Q,T)$ gravity can be used to study the accelerated phases of the
universe without introducing dark energy. We assumed the case of dust matter
i.e. $p=0$ and the linear functional form $f(Q,T)=\alpha
Q+\frac{\beta}{6k^{2}}T$, where $\alpha$ and $\beta$ are model parameters.
Also, high precision cosmological data obtained observationally, such as
Hubble data and Pantheon samples, have been used to constrain the model
parameters. We studied the evolution of the universe using the two statefinder
diagnostics : the statefinder diagnostics and the Om diagnostics.
The following are the portions of the present article: We presented a broad
review of the Weyl type $f(Q,T)$ gravity theory and the gravitational action
with its field equations in section 2. In section 3, we used 57 points of the
Hubble data points and 1048 Pantheon data points to constrain the model
parameters and compared our model with $\Lambda$CDM in error bar plots. In
section 4, we observed the behavior of energy density and the statefinder
diagnostics. We also presented the geometrical $Om(z)$ diagnostic in section 5
to illustrate dark energy models. Section 6 includes the summary of our
results obtained.

##  2 Field equations of the Weyl type $f(Q,T)$ Gravity

The action in Weyl type $f(Q,T)$ gravity is given as Yixin/2020b

$S=\int
d^{4}x\,\sqrt{-g}\left[\kappa^{2}\,f(Q,T)-\frac{1}{4}W_{\,\nu\mu}\,W^{\,\nu\mu}-\frac{1}{2}m^{2}\,w_{\nu}\,w^{\nu}+\mathcal{L}_{m}\right].$
(1)

Imposing the Lagrange multiplier $\lambda$ in the gravitational action, we get

$S=\int
d^{4}x\,\sqrt{-g}\left[\kappa^{2}\,f(Q,T)-\frac{1}{4}\,W_{\,\nu\mu}\,W^{\,\nu\mu}-\frac{1}{2}m^{2}\,w_{\nu}\,w^{\nu}+\right.\\\
\left.\lambda\,\left(R+6\,\nabla_{\alpha}w^{\alpha}-6\,w_{\alpha}\,w^{\alpha}\right)+\mathcal{L}_{m}\right].$
(2)

where, $\kappa^{2}=\frac{1}{16\pi G}$, $m$ represents the mass of the particle
associated to the vector field $w_{\mu}$, $\mathcal{L}_{m}$ is the matter
Lagrangian, $f$ is an arbitrary function of the non-metricity $Q$ and the
trace of the matter-energy-momentum tensor $T$. The second term in the action
is the standard kinetic term and the third term is a mass term of the vector
field. Also $g=det(g_{\nu\mu})$ and the scalar non-metricity $Q$ is given by

$Q\equiv-g^{\nu\mu}\left(L^{\alpha}_{\,\,\beta\mu}L^{\beta}_{\,\,\mu\alpha}-L^{\alpha}_{\,\,\beta\alpha}L^{\beta}_{\,\,\nu\mu}\right),$
(3)

where, $L^{\lambda}_{\,\,\nu\mu}$ is the deformation tensor defined as

$L^{\lambda}_{\,\,\nu\mu}=-\frac{1}{2}g^{\lambda\gamma}\left(Q_{\,\nu\gamma\mu}+Q_{\,\mu\gamma\nu}-Q_{\,\gamma\nu\mu}\right).$
(4)

In the Riemannian geometry, the covariant derivative of metric tensor is zero,
i.e., $\nabla_{\alpha}g_{\nu\mu}=0$. But in Weyl geometry, the expression is
represented as Haghani/2012

$Q_{\,\alpha\nu\mu}\equiv\widetilde{\nabla}_{\alpha}\,g_{\nu\mu}=\partial_{\alpha}g_{\nu\mu}-\widetilde{\Gamma}^{\rho}_{\,\,\alpha\nu}\,g_{\rho\mu}-\widetilde{\Gamma}^{\rho}_{\,\,\alpha\mu}\,g_{\rho\nu}=2w_{\alpha}\,g_{\nu\mu},$
(5)

where,
$\widetilde{\Gamma}^{\lambda}_{\,\,\nu\mu}\equiv\Gamma^{\lambda}_{\,\,\nu\mu}+g_{\nu\mu}\,w^{\lambda}-\delta^{\lambda}_{\nu}\,w_{\mu}-\delta^{\lambda}_{\mu}\,w_{\nu}$
and $\Gamma^{\lambda}_{\,\,\nu\mu}$ is the christoffel symbol with respect to
the metric $g_{\nu\mu}$.

Putting Eq. (5) in Eq. (3), we get the relation

$Q=-6w^{2}.$ (6)

We get the generalized Proca equation explaining the field evolution by
varying the action with respect to the vector field,

$\nabla^{\,\mu}W_{\,\nu\mu}-\left(m^{2}+12\kappa^{2}f_{Q}+12\lambda\right)w_{\nu}=6\nabla_{\nu}\lambda.$
(7)

We observe that the effective dynamical mass of the vector field when compared
with the standard Proca equation read as

$m^{2}_{\,eff}=m^{2}+12\kappa^{2}f_{Q}+12\lambda.$ (8)

Variation of the gravitational action in Eq. 2 with to the metric tensor gives
us the following generalized gravitational field equation:

$\frac{1}{2}\left(T_{\,\nu\mu}+S_{\,\nu\mu}\right)-\kappa^{2}f_{T}\left(T_{\,\nu\mu}+\Theta_{\,\nu\mu}\right)=-\frac{\kappa^{2}}{2}g_{\nu\mu}\\\
-6\kappa^{2}f_{Q}w_{\nu}w_{\mu}+\lambda\left(R_{\,\nu\mu}-6\,w_{\nu}\,w_{\mu}+3\,g_{\nu\mu}\,\nabla_{\rho}\,w^{\rho}\right)\\\
+3\,g_{\nu\mu}\,w^{\rho}\,\nabla_{\rho}\,\lambda-6\,w_{(\nu}\,\nabla_{\mu)}\,\lambda+g_{\nu\mu}\,\square\lambda-\nabla_{\nu}\nabla_{\mu}\lambda.$
(9)

Here, we define,

$f_{T}\equiv\frac{\partial f(Q,T)}{\partial T},\hskip
21.68121ptf_{Q}\equiv\frac{\partial f(Q,T)}{\partial Q},$ (10)

$T_{\,\nu\mu}\equiv-\frac{2}{\sqrt{-g}}\frac{\delta(\sqrt{-g}\,L_{m})}{\delta
g^{\nu\mu}},$ (11)

respectively. Also we have defined the quantity $\Theta_{\,\nu\mu}$

$\Theta_{\,\nu\mu}=g^{\alpha\beta}\,\frac{\delta T_{\,\alpha\beta}}{\delta
g_{\mu\nu}}=g_{\nu\mu}\,L_{m}-2\,T_{\,\nu\mu}-2\,g^{\alpha\beta}\frac{\delta^{2}\,L_{m}}{\delta
g^{\nu\mu}\,\delta g^{\alpha\beta}}.$ (12)

The re-scaled energy momentum tensor $S_{\nu\mu}$ of the Proca field is
defined as

$S_{\,\nu\mu}=-\frac{1}{4}g_{\nu\mu}\,W_{\,\rho\sigma}\,W^{\,\rho\sigma}+W_{\,\nu\rho}\,W^{\,\rho}_{\,\mu}-\frac{1}{2}m^{2}\,g_{\nu\mu}\,w_{\rho}\,w^{\rho}\\\
+m^{2}\,w_{\nu}\,w_{\mu},$ (13)

where

$W_{\nu\mu}=\nabla_{\mu}w_{\nu}-\nabla_{\nu}w_{\mu}.$ (14)

We assume that the FLRW metric in the spatially flat, isotropic and
homogeneous universe, given by

$ds^{2}=-dt^{2}+a^{2}(t)\delta_{ij}dx^{i}dx^{j},$ (15)

Where, $a(t)$ is the scale factor. Because of spatial symmetry, the vector
field is chosen in the form of

$w_{\nu}=\left(\psi(t),0,0,0\right).$ (16)

Using the above equation, we get $w^{2}=w_{\nu}w^{\nu}=-\psi^{2}(t)$, with
$Q=-6w^{2}=6\psi^{2}(t)$.
So, $u^{\nu}\nabla_{\mu}=\frac{d}{dt}$ and $H=\frac{\dot{a}}{a}$. The
Lagrangian of the perfect fluid is also assumed to be $\mathcal{L}_{m}=p$.
Now we consider the energy momentum tensor for the perfect fluid is given by

$T_{\,\nu\mu}=\left(\rho+p\right)\,u_{\nu}\,u_{\mu}+p\,g_{\nu\mu},$ (17)

where $\rho$ and $p$ are the energy density and the pressure, respectively,
$u^{\nu}$ is the four velocity vector satisfying the condition
$u_{\nu}\,u^{\nu}=-1$. Hence, we have

$T^{\,\nu}_{\,\mu}=diag\left(-\rho,p,p,p\right),$ (18)

and

$\Theta^{\,\nu}_{\,\mu}=\delta^{\,\nu}_{\,\mu}p-2T^{\,\nu}_{\,\mu}=diag\left(2\rho+p,-p,-p,-p\right).$
(19)

In cosmological case, the constraint of flat space and the generalized Proca
equation are obtained as

$\dot{\psi}=\dot{H}+2H^{2}+\psi^{2}-3H\psi,$ (20)
$\dot{\lambda}=\left(-\frac{1}{6}m^{2}-2\kappa^{2}f_{Q}-2\,\lambda\right)\psi=-\frac{1}{6}m^{2}_{eff}\psi,$
(21) $\partial_{i}\lambda=0$ (22)

From Eq. (9) the generalized Friedmann equation read as

$\kappa^{2}f_{T}\,\left(\rho+p\right)+\frac{1}{2}\rho=\frac{\kappa^{2}}{2}f-\left(6\kappa^{2}f_{Q}+\frac{1}{4}m^{2}\right)\psi^{2}\\\
-3\lambda\left(\psi^{2}-H^{2}\right)-3\dot{\lambda}\left(\psi-H\right),$ (23)

$-\frac{1}{2}p=\frac{\kappa^{2}}{2}f+\frac{m^{2}\psi^{2}}{4}+\lambda\left(3\psi^{2}+3H^{2}+2\dot{H}\right)\\\
+\left(3\psi+2H\right)\dot{\lambda}+\ddot{\lambda}.$ (24)

where dot$(\cdot)$ represents the derivative with respect to time and $f_{Q}$
and $f_{T}$ represent differentiation with respect to $Q$ and $T$
respectively.

Now, we consider the functional form
$f(Q,T)=\alpha\,Q+\frac{\beta}{6\,\kappa^{2}}\,T$, where $\alpha$ and $\beta$
are model parameters. The functional form depends on three free parameters
$\alpha$, $\beta$ and $M^{2}=\frac{m^{2}}{\kappa^{2}}$, $M$ is the mass of the
Weyl field, indicating the strengths of the Weyl geometry-matter coupling. In
this case, we have assumed $M=0.95$. It is worth mentioning that $\beta=0$
corresponds to the $f(Q,T)=\alpha Q$ i.e. a case of the successful theory of
General Relativity (GR). Also, $T=0$, the case of vacuum, the theory reduces
to $f(Q)$ gravity, which is equivalent to GR, that passes all Solar System
tests, considered in the vacuum. Furthermore, Yixin et al. Xu/2019 ; 2020
also depicts that the universe experiences an accelerating expansion ending
with a de Sitter type evolution in the considered model. We study the model
for bulk viscous fluid in a non relativistic case, i.e. $p=0$. Using this in
Eq. (23) and (24) we get,

$\left(12\kappa^{2}\alpha+m^{2}+\frac{18}{\beta}\,\kappa^{2}\alpha+\frac{3}{2\beta}m^{2}\right)\psi^{2}+3\lambda\left(4\psi^{2}+2H^{2}+2\dot{H}\right)+3\dot{\lambda}\left(4\psi+H\right)+\frac{6\lambda}{\beta}\left(3\psi^{2}+3H^{2}+2\dot{H}\right)+\\\
\frac{6\dot{\lambda}}{\beta}\left(3\psi+2H\right)+3\ddot{\lambda}+\frac{6}{\beta}\ddot{\lambda}=0,$
(25)

where $\alpha$ and $\beta$ are constants and $T=3\,p-\rho$.
We try to solve the above equation by considering $\lambda=\kappa^{2}$,

$\left(12\,\kappa^{2}\alpha+m^{2}+\frac{18}{\beta}\kappa^{2}\alpha+\frac{3}{2\beta}m^{2}\right)\psi^{2}+\\\
3\kappa^{2}\left(4\psi^{2}+2H^{2}+2\dot{H}\right)+\frac{6\kappa^{2}}{\beta}\left(3\psi^{2}+3H^{2}+2\dot{H}\right)=0.$
(26)

Further simplifying the above equation,

$\left(\left(12\beta+18\right)\alpha+\left(\beta+\frac{3}{2}\right)M^{2}\right)\psi^{2}+\\\
3\beta\left(4\psi^{2}+2H^{2}+2\dot{H}\right)+6\left(3\psi^{2}+3H^{2}+2\dot{H}\right)=0$
(27)

Using the relation $\nabla_{\lambda}g_{\nu\mu}=-w_{\lambda}g_{\nu\mu}$ and Eq.
(16), we obtain $\psi=-6H$ and Eq. (27) read as

$A\,H^{2}+B\,\dot{H}=0$ (28)

where,
$A=36\left(\left(12\beta+18\right)\alpha+\left(\beta+\frac{3}{2}\right)M^{2}+\left(12\beta+18\right)\right)+\left(6\beta+18\right)$
and $B=\left(6\beta+12\,\right)$
The obtained solution of differential equation given in (28) is,

$H(t)=\frac{1}{k_{1}\,t-c_{1}}$ (29)

where,

$\displaystyle
k_{1}=\frac{A}{B}=\left[\frac{36\left(2\beta+3\right)\left(\alpha+1\right)}{\left(\beta+2\right)}+\frac{\left(6\beta+9\right)}{\left(\beta+2\,\right)}M^{2}+\frac{\left(\beta+3\right)}{\left(\beta+2\right)}\right]$

Solving eq. (29), the expression of the scale factor is obtained as

$a(t)=c_{2}\left(k_{1}t-c_{1}\right)^{\frac{1}{k_{1}}}$ (30)

We shall write all cosmological parameter in term of redshift using the
relation (taking $a(t_{0})=1$)

$a(t)=\frac{1}{1+z}$ (31)

The Hubble parameter and deceleration parameter in terms of redshift are,

$H(z)=H_{0}(1+z)^{k_{1}}$ (32) $q(z)=\left(k_{1}-1\right)$ (33)

Here, we have obtained the power-law as the solutions of the field equations.
Power-law cosmology is an intriguing solution for dealing with some unusual
challenges like flatness, horizon problem, etc. The power-law is well-
motivated in literature. Kumar Kumar/2012  used power-law with Hz and SNe Ia
data to analyse cosmological parameters. Rani et al Rani/2015  also examined
the power-law cosmology with statefinder analysis.

##  3 Data Interpretation

###  3.1 Hubble data

Numerous observations such as the cosmic microwave background (CMB) form the
Wilkinson Microwave Anisotropy Probe team Hinshaw/2013 ; Komatsu/2011 ;
Spergel/2007  and Planck team Planck/2015 ; Planck/2018 , baryonic acoustic
oscillations (BAO) Eisenstein/2005 ; Percival/2010 , Type Ia supernovae
(SNeIa) Perlmutter/1999 ; Riess/1998  have been used to constrain cosmological
parameters. Many of these models rely on values that require Hubble parameter
to be integrated along the line of sight (the luminosity distance in SNe
observations) to explore overall expansion through time. The Hubble parameter
$H$ is intimately tied to the expansion history of the universe and is defined
as $H=\frac{\dot{a}}{a}$, where $a$ signifies the cosmic scale factor and
$\dot{a}$ as the rate of change about cosmic time. The expansion rate $H(z)$
is obtained as

$H(z)=-\frac{1}{1+z}\frac{dz}{dt},$ (34)

where $z$ is the redshift. Two procedures are commonly employed to estimate
the value of the $H(z)$ at a certain redshift. One way is to extract $H(z)$
from line-of-sight BAO data, while another uses differential age methods. We
used the revised set of 57 data points, which comprises 31 points from the
differential age (DA) approach and the left 26 points measured using BAO and
other redshift range $0.07<z<2.42$. In addition for our investigation, we use
$H_{0}=67.8kms^{-1}Mpc^{-1}$. The chi-square function is defined to find the
mean values of the model parameters $\alpha$ and $\beta$.

$\chi^{2}_{Hubb}(\alpha,\beta)=\sum_{i=1}^{57}\dfrac{\left[H^{th}_{i}(\alpha,\beta,z_{i})-H^{obs}_{i}(z_{i})\right]^{2}}{\sigma^{2}(z_{i})}$
(35)

where $H^{obs}_{i}$ denotes the observed value, $H^{th}_{i}$ indicates the
Hubble’s theoretical value while the standard error in the observed value is
denoted by $\sigma(z_{i})$. We used error bars to represent 57 points of
$H(z)$ and compared our model with the well-accepted $\Lambda$CDM model in
fig. 1. We considered $H_{0}=67.8kms^{-1}Mpc^{-1}$, $\Omega_{\Lambda_{0}}=0.7$
and $\Omega_{m_{0}}=0.3$. The best fit values of $\alpha$ and $\beta$ are
obtained through data as shown in triangle plot 2 with $1-\sigma$ and
$2-\sigma$ confidence intervals. The bounds from our analysis are
$\alpha=-1.08448^{+0.00049}_{-0.00055}$ and $\beta=0.136^{+0.056}_{-0.110}$.

Figure 1: The evolution of Hubble parameter with respect to redshift z. The
blue dots represents error bars of 57 points, the red line is the curve
obtained for our model while black dashed line corresponds to $\Lambda$CDM
model.  Figure 2: The contour plot for model parameters $\alpha$ and $\beta$
with $1-\sigma$ and $2-\sigma$ confidence regions. It also mentions the best
fit values of $\alpha$ and $\beta$ using 57 Hubble data points.

###  3.2 Pantheon data

We use the most recent compilation of Supernovae pantheon samples to constrain
the model parameters $\alpha$ and $\beta$. The Pantheon sample consists of
1048 SNe Ia in the range of $0.01<z<2.26$ Camlibel/2020 ; Scolnic/2018 . The
likelihood function is determined using the MCMC approach and emcee Python’s
library to calculate the posterior distributions of the model parameters. The
pantheon data are shown in $(m,z)$ pairs, with $m$ typically to be measured.
The theoretical distance modulus is defined as

$\mu^{th}=5log_{10}\left(\frac{D_{L}H_{0}^{-1}}{Mpc}\right)+25,$ (36)

where we define

$D_{L}=(1+z)c\int_{0}^{z}\frac{d\bar{z}}{H(\bar{z})}.$ (37)

Here $H_{0}$ is the Hubble constant. The chi-square function according to our
considered model is given as

$\chi^{2}_{Pan}(\alpha,\beta)=\sum_{i=1}^{1048}\dfrac{\left[\mu^{th}_{i}(\alpha,\beta,z_{i})-\mu^{obs}_{i}(z_{i})\right]^{2}}{\sigma^{2}(z_{i})}$
(38)

where $\sigma^{2}(z_{i})$ is the standard error, $\mu^{th}_{i}=m-M$ is the
theoretical value with $m$ and $M$ are the apparent and absolute magnitudes
respectively, $\mu^{obs}_{i}$ is the observed values from data points. We used
error bars to represent 1048 points of pantheon samples and compared our model
with the well-accepted $\Lambda$CDM model in fig. 3. We considered
$H_{0}=67.8kms^{-1}Mpc^{-1}$, $\Omega_{\Lambda_{0}}=0.7$ and
$\Omega_{m_{0}}=0.3$. The best fit values of $\alpha$ and $\beta$ are obtained
through pantheon samples as shown in triangle plot 4 with $1-\sigma$ and
$2-\sigma$ confidence intervals. The bounds from our analysis are
$\alpha=-1.09519^{+0.00060}_{-0.00068}$ and $\beta=0.137^{+0.058}_{-0.100}$.

Figure 3: The plot of $\mu(z)$ with respect to redshift $z$. The blue dots
represents error bars of 1048 pantheon points, the red line is the curve
obtained for our model while black dashed line corresponds to $\Lambda$CDM
model. Figure 4: The contour plot for model parameters $\alpha$ and $\beta$
with $1-\sigma$ and $2-\sigma$ confidence regions. It also mentions the best
fit values of $\alpha$ and $\beta$ using 1048 pantheon samples.

##  4 Cosmological parameters

###  4.1 Density parameter

By solving eqs. (23) and (24), we can obtain an expression for the density
parameter $\rho$. The behavior of density parameter is shown below in fig. 5
and 6 for the obtained $\alpha$ and $\beta$ from Hubble and Pantheon datasets
respectively. It can be observed that the density parameter for both the
datasets is showing a positive behavior with redshift $z$.

Figure 5: The behavior of density parameter for $\alpha=-1.08448$ and
$\beta=0.136$ obtained from Hubble data. Figure 6: The behavior of density
parameter for $\alpha=-1.09519$ and $\beta=0.137$ obtained from Pantheon data.

###  4.2 Statefinder diagnostics

Numerous DE models can be used to describe cosmic acceleration. Another
reliable diagnostic exists to distinguish between many cosmological models
involving dark energy. Sahni et al. Sahni/2003 ; Alam/2003  proposed a new
dark energy diagnostic known as statefinder diagnostics, dependent on the
second and third derivatives of the scale factor. It is defined with the help
of well known geometrical parameters namely the Hubble parameter
$H=\frac{\dot{a}}{a}$ and the deceleration parameter
$q=-\frac{\ddot{a}}{aH^{2}}$. The statefinder parameter pair ${r-s}$ is
defined as

$r=\frac{\dddot{a}}{a\,H^{3}}$ (39) $s=\frac{r-1}{3\,(q-\frac{1}{2})}$ (40)

The plot of ${s-r}$ is shown in fig. 7. The statefinder parameter ${s-r}$ can
be an admirable diagnostic for describing significant dark energy model
characteristics. According to the trajectories in ${s-r}$ plane, the point
$(0,1)$ corresponds to the $\Lambda$CDM model, Chaplygin gas lie to the left
of the $\Lambda$CDM model whereas quintessence lie to the right of the
$\Lambda$CDM. The evolution of ${q-r}$ is shown in Fig. 8. It is observed the
point $(q,r)=(0.5,1)$ correspond to $SCDM$ (i.e. matter dominated universe),
with the de-sitter (dS) expansion pointing to $(q,r)=(-1,1)$ in the future. As
a result, the statefinder diagnostics can successfully distinguish between
various dark energy models.

Figure 7: The figure shows the behavior of $s-r$ plane with $\beta=0.136$ and
$\beta=0.137$ and varied $\alpha$. Figure 8: The figure shows the graph of
$q-r$ plane with $\beta=0.136$ and $\beta=0.137$ and varied $\alpha$.

It is worth noting that in the obtained model, the $\Lambda$CDM statefinder
pair $(0,1)$ and correspondingly the dS point $(-1,1)$ acts as an attractor.
The constraints on statefinder from Hubble data and Pantheon data are obtained
as $r=-0.039^{+0.009}_{-0.009}$, $s=0.637^{+0.0075}_{-0.0075}$ and
$r=-0.105^{+0.008}_{-0.011}$, $s=0.434^{+0.023}_{-0.012}$ respectively
Kumar/2012 ; Rani/2015 . It is observed that the model fits well with Pantheon
datasets rather than the Hubble data.

##  5 Om Diagnostics

The $Om$ diagnostic can be studied as a simplest diagnostic than the
statefinder diagnostic Sahni/2008 ; Shahalam/2015  because it uses only the
first-order time derivative of scale factor i.e involving the Hubble
parameter. It is used to clarify various dark energy (DE) models by
differentiating $\Lambda$ CDM model. For spatially flat universe, the $Om(z)$
diagnostic is defined as

$Om\left(z\right)=\frac{\left(\frac{H\left(z\right)}{H_{0}}\right)^{2}-1}{\left(1+z\right)^{3}-1}$
(41)

where, $H_{0}$ is the Hubble constant. According to the behavior of $Om(z)$,
different dark energy models can be described. Phantom type i.e. $\omega<-1$
corresponds to the positive slope of $Om(z)$, quintessence type $\omega>-1$
corresponding to negative slope of $Om(z)$. The constant behavior of $Om(z)$
depicts the $\Lambda$CDM model. In Fig. 9, the $Om(z)$ has a negative slope,
showing quintessence-like behavior indicating the accelerated expansion. As a
result, the model may not resolve the Hubble tension at present. The study in
references Vagnozzi/2020 ; Valentino/2016  reveals that a phantom like
component with effective equation of state $\omega=-1.29$ can solve the
current tension between the Planck data set and other prior in an extended
$\Lambda$CDM scenario. It is also worth noting from Valentino/2021  that the
lower tension is attributable to a change in the value of $H_{0}$ and an
increase in its uncertainty owing to degeneracy with more physics, further
confounding the picture and indicating the need for more probes. While no
single idea stands out as very plausible or superior to all other, solutions
including early or dynamical dark energy, interacting cosmologies and modified
gravity are the best alternatives until a better one emerges.

Figure 9: This figure shows the behavior of $Om$ vs redshift $z$ with
$\alpha=-1.08448$, $\beta=0.136$ and $\alpha=-1.09519$, $\beta=0.137$
constrained by Hubble and Pantheon datasets respectively.

##  6 Conclusion

In this study, we considered an extension of the third equivalent
representation of GR (the symmetric teleparallel formulation) called $f(Q,T)$
gravity, where the non-metricity $Q$ is non-minimally coupled to the trace $T$
of energy-momentum tensor. We examined the Weyl type $f(Q,T)$ gravity, in
which the product of the metric and the Weyl vector determines the covariant
divergence of the metric tensor. As a result, the Weyl vector and metric
tensor is responsible for the geometrical features of the theory. We have
considered the case of dust and obtained the solutions of the field equations.
The Hubble parameter is found to be similar to the power-law form in redshift
$z$. We used the most recent 57 Hubble data sets and 1048 Pantheon supernovae
datasets to constrain the model parameters $\alpha$ and $\beta$. The model is
also compared to $\Lambda$CDM model shown in the error bar plots. According to
the constraints values of $\alpha$ and $\beta$, the deceleration parameter $q$
is seen to be negative. The nature of cosmic evolution in the Weyl $f(Q,T)$
gravity is greatly reliant on the values of the functional form of $f$ and the
model parameters involved. As a result, we used the statefinder diagnostics
$s-r$ and $q-r$ and the $Om$ diagnostic analysis for the model to study the
nature of dark energy models. The constrained values of $r$ and $s$ are
obtained as $r=-0.039^{+0.009}_{-0.009}$, $s=0.637^{+0.0075}_{-0.0075}$ and
$r=-0.105^{+0.008}_{-0.011}$, $s=0.434^{+0.023}_{-0.012}$ for Hubble and
Pantheon data respectively. It is observed that the model fits well with
Pantheon $SNeIa$ data better that the $Hz$ data. The obtained model is proven
to be helpful in describing the acceleration of present universe in the
context of current observations of $Hz$ and $SNeIa$. However, it fails to
provide redshift transition from deceleration to acceleration due to the
constant value of the deceleration parameter. Hence, there are many other
possibilities to check the viability of Weyl $f(Q,T)$ theory, such as
considering of the scalar field to study inflation, a theoretical study in the
presence of coupling between geometry and matter, etc.

## Acknowledgments

GG RS acknowledges University Grants Commission (UGC), New Delhi, India for
awarding Junior Research Fellowship (UGC-Ref. No.: 201610122060). SA
acknowledges CSIR, Govt. of India, New Delhi, for awarding Junior Research
Fellowship. PKS acknowledges CSIR, New Delhi, India for financial support to
carry out the Research project [No.03(1454)/19/EMR-II Dt.02/08/2019]. We are
very much grateful to the honorable referee and the editor for the
illuminating suggestions that have significantly improved our work in terms of
research quality and presentation.

##  7 Data Availability Statement

There are no new data associated with this article.

##  8 Appendix

Here, Table-1 contains the $57$ points of Hubble parameter values $H(z)$ with
errors $\sigma_{H}$ from differential age ($31$ points), and BAO and other
($26$ points) approaches, along with references.

Table-1: $H(z)$ datasets consisting of 57 data points
---
DA method (31 points)
$z$ | $H(z)$ | $\sigma_{H}$ | Ref. | $z$ | $H(z)$ | $\sigma_{H}$ | Ref.
$0.070$ | $69$ | $19.6$ | h1  | $0.4783$ | $80$ | $99$ | h5
$0.90$ | $69$ | $12$ | h2  | $0.480$ | $97$ | $62$ | h1
$0.120$ | $68.6$ | $26.2$ | h1  | $0.593$ | $104$ | $13$ | h3
$0.170$ | $83$ | $8$ | h2  | $0.6797$ | $92$ | $8$ | h3
$0.1791$ | $75$ | $4$ | h3  | $0.7812$ | $105$ | $12$ | h3
$0.1993$ | $75$ | $5$ | h3  | $0.8754$ | $125$ | $17$ | h3
$0.200$ | $72.9$ | $29.6$ | h4  | $0.880$ | $90$ | $40$ | h1
$0.270$ | $77$ | $14$ | h2  | $0.900$ | $117$ | $23$ | h2
$0.280$ | $88.8$ | $36.6$ | h4  | $1.037$ | $154$ | $20$ | h3
$0.3519$ | $83$ | $14$ | h3  | $1.300$ | $168$ | $17$ | h2
$0.3802$ | $83$ | $13.5$ | h5  | $1.363$ | $160$ | $33.6$ | h7
$0.400$ | $95$ | $17$ | h2  | $1.430$ | $177$ | $18$ | h2
$0.4004$ | $77$ | $10.2$ | h5  | $1.530$ | $140$ | $14$ | h2
$0.4247$ | $87.1$ | $11.2$ | h5  | $1.750$ | $202$ | $40$ | h2
$0.4497$ | $92.8$ | $12.9$ | h5  | $1.965$ | $186.5$ | $50.4$ | h7
$0.470$ | $89$ | $34$ | h6  |  |  |  |
From BAO & other method (26 points)
$z$ | $H(z)$ | $\sigma_{H}$ | Ref. | $z$ | $H(z)$ | $\sigma_{H}$ | Ref.
$0.24$ | $79.69$ | $2.99$ | h8  | $0.52$ | $94.35$ | $2.64$ | h10
$0.30$ | $81.7$ | $6.22$ | h9  | $0.56$ | $93.34$ | $2.3$ | h10
$0.31$ | $78.18$ | $4.74$ | h10  | $0.57$ | $87.6$ | $7.8$ | h14
$0.34$ | $83.8$ | $3.66$ | h8  | $0.57$ | $96.8$ | $3.4$ | h15
$0.35$ | $82.7$ | $9.1$ | h11  | $0.59$ | $98.48$ | $3.18$ | h10
$0.36$ | $79.94$ | $3.38$ | h10  | $0.60$ | $87.9$ | $6.1$ | h13
$0.38$ | $81.5$ | $1.9$ | h12  | $0.61$ | $97.3$ | $2.1$ | h12
$0.40$ | $82.04$ | $2.03$ | h10  | $0.64$ | $98.82$ | $2.98$ | h10
$0.43$ | $86.45$ | $3.97$ | h8  | $0.73$ | $97.3$ | $7.0$ | h13
$0.44$ | $82.6$ | $7.8$ | h13  | $2.30$ | $224$ | $8.6$ | h16
$0.44$ | $84.81$ | $1.83$ | h10  | $2.33$ | $224$ | $8$ | h17
$0.48$ | $87.79$ | $2.03$ | h10  | $2.34$ | $222$ | $8.5$ | h18
$0.51$ | $90.4$ | $1.9$ | h12  | $2.36$ | $226$ | $9.3$ | h19

## References

  * (1) Riess A. G. et al., ApJ, 116, 1009, (1998)
  * (2) Perlmutter S. et al., ApJ, 517, 565, (1999)
  * (3) Spergel D.N. et al., Astrophys. J. Suppl. Ser., 170, 377, (2007)
  * (4) Eisenstein D.J. et al., Astrophys. J., 633, 560, (2005)
  * (5) Cole S. et al., MNRAS, 362, 505, (2005)
  * (6) Hawkins E. et al., MNRAS, 346, 78, (2003)
  * (7) Buchdahl H. A., MNRAS, 150, 1, (1970)
  * (8) Capozziello S., Laurentis M. De, Phys. Rept., 509, 167, (2011)
  * (9) Nojiri S., Odintsov S.D., Phys. Rev. D, 68, 123512, (2003)
  * (10) Arora S. et al.,Class. Quantum Grav., 37, 205022, (2020)
  * (11) Harko T. et al., Phys. Rev. D, 84, 024020, (2011)
  * (12) Moraes P.H.R.S., Sahoo P.K.,Phys. Rev. D, 96, 044038, (2017)
  * (13) Yousaf Z., Bhatti M. Z.-ul-Haq, Ilyas M., Euro. Phys. J. C, 78, 307, (2018)
  * (14) Atazadeh K., Darabi F., Gen. Relativ. Gravit., 46, 1664, (2014)
  * (15) Laurentis M. De, Paolella M., Capozziello S., Phys. Rev. D, 91, 083531, (2015)
  * (16) Bahamonde S., Zubair M., Abbas G., Phys. Dark Univ., 19, 78, (2018)
  * (17) Capozziello S., Capriolo M., Caso L., Eur. Phys. J. C, 80, 156, (2020)
  * (18) Weyl H., Sitzungsber. Preuss. Akad. Wiss., 465, 1, (1918)
  * (19) Scholz E., arXiv:1703.03187., (2017)
  * (20) Wheeler J. T., Gen. Relativ. Grav., 50, 80, (2018)
  * (21) Weitzenbock R., Invariantentheorie, Noordhoff, Groningen (1923)
  * (22) Hayashi K., Shirafuji T., Phys. Rev. D, 19, 3524, (1979)
  * (23) Benetti M., Capozziello S., Lambiase G., MNRAS, 500, 1795, (2021)
  * (24) Capozziello S. et al., Phys. Rev. D, 84, 043527, (2011)
  * (25) Mandal S., Sahoo P.K., Mod. Phys. Lett. A, 35, 40, (2020)
  * (26) Myrzakulov R., Eur. Phys. J. C, 71, 1752, (2011)
  * (27) Jimenez J. B., Heisenberg L., Koivisto T., Phys. Rev. D, 98, 044048, (2018)
  * (28) Lazkoz R. et al., Phys. Rev. D, 100, 104027, (2019).
  * (29) Khyllep W., Paliathanasis A., Dutta J., Phys. Rev. D, 103, 103521, (2021).
  * (30) Mandal S., Wang D., and Sahoo P.K., Phys. Rev. D, 102, 124029, (2020).
  * (31) Xu Y. et al., Eur. Phys. J. C, 79, 708, (2019)
  * (32) Xu Y. et al., Eur. Phys. J. C,  80, 449, (2020)
  * (33) Arora S. Parida A., Sahoo P.K., Eur. Phys. J. C, 81, 555, (2021)
  * (34) Bhattacharjee S., Sahoo P.K., Eur. Phys. J. C, 80, 289, (2020)
  * (35) Arora S., Santos J.R.L., Sahoo P.K., Phys. Dark Univ., 31, 100790, (2021)
  * (36) Xu Y. et al., Eur. Phys. J. C, 80, 449, (2020)
  * (37) Yang Jin-Z. et al., Eur. Phys. J. C, 81, 111, (2021)
  * (38) Haghani Z. et al., JCAP, 10, 061, (2012)
  * (39) Kumar S., MNRAS, 422, 2532, (2012)
  * (40) Rani S. et al., JCAP, 03, 031, (2015)
  * (41) Hinshaw G. et al., ApJS, 208, 19, (2013)
  * (42) Komatsu E. et al., ApJS, 192, 18, (2011)
  * (43) Ade P.A.R. et al., A & A, 594, A13, (2016)
  * (44) Aghanim N. et al., A & A, 641, A6, (2020)
  * (45) Percival Will J. et al., MNRAS, 401, 2148, (2010)
  * (46) Camlibel A. K., Semiz I., Feyizoglu M./A. Class. Quant. Grav., 37, 235001, (2020)
  * (47) Scolnic D. M. et al.,Astrophys. J., 859, 101, (2018)
  * (48) Alam U. et al., MNRAS, 344, 1057, (2003)
  * (49) Sahni V. et al., JETP Lett., 77, 201, (2003)
  * (50) Sahni V., Shafieloo A., Starobinsky A.A. Phys. Rev. D, 78, 103502, (2008)
  * (51) Shahalam M., Sami S. & Agarwal A., MNRAS, 448, 2948, (2015)
  * (52) Vagnozzi S., Phys. Rev. D, 102, 023518, (2020)
  * (53) Valentino E. Di Phys. Lett. B, 761, 242-246, (2016)
  * (54) Valentino E. Di, Class. Quantum Grav., 38, 153001, (2021)
  * (55) Stern D. et al., J. Cosmol. Astropart. Phys., 02, 008, (2010)
  * (56) Simon J., Verde L. , Jimenez R. , Phys. Rev. D, 71, 123001, (2005)
  * (57) Moresco M. et al., J. Cosmol. Astropart. Phys., 08, 006, (2012)
  * (58) Zhang C. et al., Research in Astron. and Astrop., 14, 1221, (2014)
  * (59) Moresco M. et al., J. Cosmol. Astropart. Phys., 05, 014, (2016)
  * (60) Ratsimbazafy A. L. et al., Mon. Not. Roy. Astron. Soc.,467, 3239, (2017)
  * (61) Moresco M., Mon. Not. Roy. Astron. Soc. Lett., 450, L16, (2015)
  * (62) Gaztaaga E. et al., Mon. Not. Roy. Astron. Soc., 399, 1663, (2009)
  * (63) Oka A. et al., Mon. Not. Roy. Astron. Soc., 439, 2515, (2014)
  * (64) Wang Y. et al., Mon. Not. Roy. Astron. Soc., 469, 3762, (2017)
  * (65) Chuang C. H., Wang Y., Mon. Not. Roy. Astron. Soc., 435, 255, (2013)
  * (66) Alam S. et al., Mon. Not. Roy. Astron. Soc., 470, 2617, (2017)
  * (67) Blake C. et al., Mon. Not. Roy. Astron. Soc., 425, 405, (2012)
  * (68) Chuang C. H. et al., Mon. Not. Roy. Astron. Soc., 433, 3559, (2013)
  * (69) Anderson L. et al., Mon. Not. Roy. Astron. Soc., 441, 24, (2014)
  * (70) Busca N. G. et al., Astron. Astrophys., 552, A96, (2013)
  * (71) Bautista J. E. et al., Astron. Astrophys., 603, A12, (2017)
  * (72) Delubac T. et al., Astron. Astrophys., 574, A59, (2015)
  * (73) Font-Ribera A. et al., J. Cosmol. Astropart. Phys., 05, 027, (2014)


# Dynamic Surrogate Switching: Sample-Efficient Search
for Factorization Machine Configurations in Online Recommendations

Blaž Škrlj<EMAIL_ADDRESS>1234-5678-9012 ,  Adi Schwartz
<EMAIL_ADDRESS>,  Jure Ferlež<EMAIL_ADDRESS>,  Davorin Kopič
<EMAIL_ADDRESS>and  Naama Ziporin<EMAIL_ADDRESS>Outbrain Inc.USA

(2022)

###### Abstract.

Hyperparameter optimization is the process of identifying the appropriate
hyperparameter configuration of a given machine learning model with regard to
a given learning task. For smaller data sets, an exhaustive search is
possible; However, when the data size and model complexity increase, the
number of configuration evaluations becomes the main computational bottleneck.
A promising paradigm for tackling this type of problem is surrogate-based
optimization. The main idea underlying this paradigm considers an
incrementally updated model of the relation between the hyperparameter space
and the output (target) space; the data for this model are obtained by
evaluating the main learning engine, which is, for example, a factorization
machine-based model. By learning to approximate the hyperparameter-target
relation, the surrogate (machine learning) model can be used to score large
amounts of hyperparameter configurations, exploring parts of the configuration
space beyond the reach of direct machine learning engine evaluation. Commonly,
a surrogate is selected prior to optimization initialization and remains the
same during the search. We investigated whether dynamic switching of
surrogates during the optimization itself is a sensible idea of practical
relevance for selecting the most appropriate factorization machine-based
models for large-scale online recommendation. We conducted benchmarks on data
sets containing hundreds of millions of instances against established
baselines such as Random Forest- and Gaussian process-based surrogates. The
results indicate that surrogate switching can offer good performance while
considering fewer learning engine evaluations.

online learning, data mining, hyperparameter optimization, surrogate models,
AutoML

††journalyear: 2022††copyright: rightsretained††conference: Sixteenth ACM
Conference on Recommender Systems; September 18–23, 2022; Seattle, WA,
USA††booktitle: Sixteenth ACM Conference on Recommender Systems (RecSys ’22),
September 18–23, 2022, Seattle, WA, USA††doi: 10.1145/3523227.3547384††isbn:
978-1-4503-9278-5/22/09††ccs: Computing methodologies Machine learning††ccs:
Computing methodologies Randomized search††ccs: Computing methodologies Online
learning settings

##  1\. Introduction

Online recommendation is fundamental to functioning of the modern web. From
online stores to streaming service engines and content discovery platforms, it
enables users to discover relevant content faster, and product owners to make
their products visible to a broader public. Recommender systems can be
understood as a feedback loop between the recommendation engine and the target
audience – based on provided recommendations, the interactions with the
recommendations are stored and considered to improve a given recommender
further (Batmaz et al., 2019).

Machine learning models underlie many online recommendation systems. The most
common branch of algorithms considered, both due to their performance and
lower complexity, are factorization machines (FM). These algorithms require
the modelers to provide both the space of features and their interactions
(_featurization_), as well as model hyperparameters (Rendle, 2012). The latter
is the main focus of this paper. Tuning hyperparameters can be considered as
an optimization task, formulated as follows:

$\textrm{Solution}\approx\operatorname*{arg\,min}_{\begin{subarray}{c}\Theta\in\textrm{hyperParamSpace}\end{subarray}}\mathbb{E}\big{[}\textsc{Loss}\big{(}\textrm{\emph{LearningEngine}},\Theta,\textrm{data}\big{)}\big{]},$

where _data_ corresponds to the training data of choice, $\Theta$ to a given
hyperparameter configuration, and _LearningEngine_ to the machine learning
model considered. The minimization aims to identify minimal expected _Loss_ of
choice (lower is better in this formulation). This formulation considers one
_LearningEngine_ evaluation for each configuration. As the optimization
progresses, configuration-target pairs are obtained and stored. _Surrogate
models_ exploit this data to estimate which configuration the _LearningEngine_
will consider next. The problem with larger data sets is that the amount of
engine evaluations is _limited_ , hence having data-efficient surrogates is a
desired property that can substantially shorten research cycles. The
contributions presented are multi-fold, and are stated next. First, we
describe the process of integration of surrogate-based optimization into the
in-house AutoML framework – we considered both Bayesian and non-Bayesian
(surrogate) models, reporting on their behaviour. We next present the
implemented strategy of Dynamic Surrogate Switching (DSS) that builds on the
recent ideas of automated model switching during optimization. The results of
initial evaluations indicate this is a promising strategy to speed up
hyperparameter optimization in real-life settings, where the number of
learning engine evaluations is limited due to the scale of data considered.

##  2\. Selected Related work

[Example 3D visualization of hyperparameter landscape.]Hyperparameters
visualized: two hyperparameters with corresponding RIG scores.

Figure 1. Example two-hyperparameter non-convex (interpolated) landscape.

The vast majority of machine learning models require the specification of
hyperparameters for their normal mode of operation. Early approaches mostly
considered grid-based, random, or evolution-based search through the space of
possible configurations (Friedrichs and Igel, 2005). However, recent trends
indicate that, albeit random search often represents a viable baseline,
exploiting the information obtained during the optimization can be a better
strategy. There are multiple ways these hyperparameter-target score data
points can be utilized. The simplest examples include incorporation of their
statistical properties into the search strategy, however, considering them as
input data samples for the _surrogate model_ is also often considered (Falkner
et al., 2018). The ideas of surrogate-based optimization are tightly linked
with the field of Bayesian optimization (Malu et al., 2021); effectively, if a
surrogate is a probabilistic model capable of also outputting the (epistemic)
uncertainty associated with a given prediction, both the prediction and the
associated uncertainty can be used by the _acquisition function_ – the
procedure responsible for linking the surrogate’s output(s) with parts of the
search space that should be considered in the next round of learning engine
evaluations. There exist a plethora of commonly considered acquisition
functions – examples include the probability of improvement, expected
improvement and more (Frazier, 2018). If the surrogate is only capable of
outputting the prediction, acquisition functions are simpler – for example,
viable configurations can be identified already by sorting the space of
candidate configurations according to the surrogate’s outputs. The acquisition
function is thus responsible for using a trained surrogate to _score_ parts of
the hyperparameter space – these can be obtained via random sampling or more
involved optimization schemes (such as for example, using L-BFGS (Zhu et al.,
1997) to minimize the acquisition function directly). Non-convex optimization
landscapes can emerge already when considering as few as two hyperparameters
in production environments (example shown in Figure 1). The presented
methodology also builds on the recent ideas of Online AutoML (Automated
Machine Learning) (He et al., 2021). The goal of AutoML systems is to automate
various data-related processes commonly manually performed by human modelers.
In particular, we built on some of the ideas presented recently in (Celik et
al., 2022), where dynamic ensembles were used to perform online learning. The
main finding of the aforementioned study is that _switching_ the models can
help mitigate problems related to data quantity (at the initial stages) and
concept drift (at the latter stages). Albeit the considered setting is being
tested offline, we can interpret the search itself as a dynamic process –
commonly, a single surrogate type is considered throughout the search,
however, is this the preferred strategy? We explored whether dynamic re-
configuration of surrogate models _during the optimization_ is a sensible,
sample-efficient hyperparameter optimization strategy suitable for large-scale
model configuration search.

[Workflow depicting DSS.]A cyclic workflow showing how the models are
iteratively refined/switched.

Figure 2. Conceptual overview of DSS, part of the in-house AutoML.

[Two figures showing DSS’s performance]Left image shows iterative model
switching in action, right one shows benchmark performance against strong
baselines.

(a) Visualizing Dynamic Surrogate Switching – the surrogates change as the
optimization progresses.

(b) Benchmark results – offline tests – CTR task (Relative information gain
(He et al., 2014)) with fixed stopping.

Figure 3. Visualization of DSS benchmark experiment and an ablation. The goal
was to identify whether surrogate switching is a feasible strategy to obtain
good model configurations. The benchmark results on a private data set
consisting of hundreds of millions of instances (target was click-through rate
– CTR) are shown in Figure b). Results indicate DSS is a promising strategy
candidate for sample-efficient hyperparameter optimization, requiring minimal
human involvement during configuration.

##  3\. Dynamic Surrogate Switching (DSS)

Surrogate models learn to estimate configuration quality during hyperparameter
optimization. Changing surrogates during the search itself was previously
shown to have a positive effect on the search’s efficiency (Mehmani et al.,
2018). This work builds on similar ideas, extending them beyond common
function optimization benchmark sets to operate as a part of the internal
_AutoML framework_ that requires handling of hundreds of millions of instances
during the search for suitable FM configurations. This section describes the
key components of the presented method and the initial study of their
behaviour. We refer to the proposed method as Dynamic Surrogate Switching
(DSS). A conceptual overview of the proposed method is shown in Figure 2. The
approach follows the paradigm of surrogate-based modeling extended by the idea
of surrogate switching. We continue with an overview of DSS. As the first
step, we will consider the evaluation of the learning engine (in our case,
field-aware factorization machines (FFMs) (Juan et al., 2016)111Field aware
factorization machines implementation available as
https://github.com/outbrain/fwumious_wabbit.. Initially, diverse parts of the
search space are selected and considered by the learning engine to obtain the
initial set of configuration-score tuples suitable for surrogate-based
learning. The evaluation data update step is responsible for storing the
configurations in a format suitable for learning. Further, this step also
checks for anomalies in the evaluations (e.g., too many similar/same results).
Note that the evaluation data set is constantly updated throughout the search,
with all data considered each update (to maximize utilization of prior
evaluations). The subsequent step of surrogate selection is what
differentiates DSS from conventional surrogate-based learning the most – the
DSS is based on the previous work (Mehmani et al., 2018) and the ideas of
OAML, aimed to enable dynamic surrogate re-configuration. We implemented it by
considering a collection of different model types and their initial
configurations. The considered models that can serve as a surrogate are, for
example, Random Forests (Breiman, 2001), Gaussian Processes (Rasmussen, 2003)
and Gradient Boosting Machines (Friedman, 2001) 222Implemented with components
from (Varoquaux et al., 2015).. As the evaluation of such surrogates is
inexpensive (and thus not the bottleneck of the whole search), the surrogate
selection phase considers multiple parametrizations of the mentioned model
types – overall, hundreds of surrogates are evaluated each iteration. The next
issue we tackled was how to _score_ the surrogates. We perform the selection
based on the _proportion of explained variance_. Thus, the surrogate model
scoring (including ranking) can be formulated as:
$\operatorname*{arg\,sort}_{s\in
S}\frac{\textrm{Var}(y-s(D))}{\textrm{Var}(y)},$ where $S$ is the set of
possible surrogate models and $D$ the current set of configuration-score
tuples obtained by the learning engine during search. The highest-ranked
surrogate is selected and used during the acquisition step for a given search
iteration. An important component of surrogate-based optimization is
configuration generation. Solution candidates are scored in mini-batches to
tackle the memory overhead. We further augmented the random configuration
search with a memory structure that discards parts of the space considered
previously. Once the surrogate is selected, re-trained on the current data and
the candidate space is selected, solution scoring takes place. Here, the
surrogate is used to provide a score for each generated configuration. There
are multiple possible ways of utilizing the obtained scores; e.g., in Bayesian
search, probability-of-improvement or similar heuristics can be considered. As
the considered DSS does not always offer probabilistic outputs, simpler
acquisition functions are required. We exploit the fact that the AutoML system
that implements DSS runs in multi-threaded mode, enabling us to devote some
threads to top-ranked solutions obtained by the surrogate while leaving some
threads to perform exploration based on more randomized configurations. Once
selected, the learning engine considers the configurations, and the feedback
loop continues. Example results are summarized in Figure 3. Overall, the DSS
was identified as a promising approach to tuning FFMs.

##  4\. Conclusions

The paper presents Dynamic Surrogate Switching (DSS), an approach aimed to
facilitate hyperparameter search – one of the components of the in-house
AutoML utilized daily by many data scientists. The results indicate that
dynamic surrogates indeed offer a promising alternative to existing strong
baselines (e.g., Random Forest-based surrogates), are potentially more
adaptive to particular problems considered, and require less human
intervention during configuration. Further work includes testing the idea on
different data sets and studying its behaviour when considering different
learning engines.

## AUTHOR BIO

Blaž Škrlj is a machine learning researcher active in the areas of AutoML and
representation learning. During his PhD at the Jožef Stefan International
Postgraduate School he worked on low-resource AutoML and its applications to
natural language processing, graph-based machine learning and compressibility
of latent representations. Currently, Blaž is part of Outbrain’s AutoML team,
where he is exploring the limits of AutoML for large-scale recommendation.

## References

  * (1)
  * Batmaz et al. (2019) Zeynep Batmaz, Ali Yurekli, Alper Bilge, and Cihan Kaleli. 2019.  A review on deep learning for recommender systems: challenges and remedies.  _Artificial Intelligence Review_ 52, 1 (2019), 1–37.
  * Breiman (2001) Leo Breiman. 2001\.  Random forests.  _Machine learning_ 45, 1 (2001), 5–32.
  * Celik et al. (2022) Bilge Celik, Prabhant Singh, and Joaquin Vanschoren. 2022\.  Online AutoML: An adaptive AutoML framework for online learning.  _arXiv preprint arXiv:2201.09750_ (2022).
  * Falkner et al. (2018) Stefan Falkner, Aaron Klein, and Frank Hutter. 2018\.  BOHB: Robust and efficient hyperparameter optimization at scale. In _International Conference on Machine Learning_. PMLR, 1437–1446.
  * Frazier (2018) Peter I Frazier. 2018\.  A tutorial on Bayesian optimization.  _arXiv preprint arXiv:1807.02811_ (2018).
  * Friedman (2001) Jerome H Friedman. 2001\.  Greedy function approximation: a gradient boosting machine.  _Annals of statistics_ (2001), 1189–1232.
  * Friedrichs and Igel (2005) Frauke Friedrichs and Christian Igel. 2005.  Evolutionary tuning of multiple SVM parameters.  _Neurocomputing_ 64 (2005), 107–117.
  * He et al. (2014) Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. 2014\.  Practical lessons from predicting clicks on ads at facebook. In _Proceedings of the eighth international workshop on data mining for online advertising_. 1–9.
  * He et al. (2021) Xin He, Kaiyong Zhao, and Xiaowen Chu. 2021.  AutoML: A survey of the state-of-the-art.  _Knowledge-Based Systems_ 212 (2021), 106622.
  * Juan et al. (2016) Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016\.  Field-aware factorization machines for CTR prediction. In _Proceedings of the 10th ACM conference on recommender systems_. 43–50.
  * Malu et al. (2021) Mohit Malu, Gautam Dasarathy, and Andreas Spanias. 2021\.  Bayesian Optimization in High-Dimensional Spaces: A Brief Survey. In _2021 12th International Conference on Information, Intelligence, Systems & Applications (IISA)_. IEEE, 1–8.
  * Mehmani et al. (2018) Ali Mehmani, Souma Chowdhury, Christoph Meinrenken, and Achille Messac. 2018.  Concurrent surrogate model selection (COSMOS): optimizing model type, kernel function, and hyper-parameters.  _Structural and Multidisciplinary Optimization_ 57, 3 (01 Mar 2018), 1093–1114.  https://doi.org/10.1007/s00158-017-1797-y
  * Rasmussen (2003) Carl Edward Rasmussen. 2003\.  Gaussian processes in machine learning. In _Summer school on machine learning_. Springer, 63–71.
  * Rendle (2012) Steffen Rendle. 2012\.  Factorization machines with libfm.  _ACM Transactions on Intelligent Systems and Technology (TIST)_ 3, 3 (2012), 1–22.
  * Varoquaux et al. (2015) Gaël Varoquaux, Lars Buitinck, Gilles Louppe, Olivier Grisel, Fabian Pedregosa, and Andreas Mueller. 2015\.  Scikit-learn: Machine learning without learning the machinery.  _GetMobile: Mobile Computing and Communications_ 19, 1 (2015), 29–33.
  * Zhu et al. (1997) Ciyou Zhu, Richard H Byrd, Peihuang Lu, and Jorge Nocedal. 1997\.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization.  _ACM Transactions on mathematical software (TOMS)_ 23, 4 (1997), 550–560.