# Closed-form estimators for an exponential family derived from likelihood
equations

Roberto Vila<EMAIL_ADDRESS>Department of Statistics, University of
Brasília, 70910-900, Brasília, Brazil  Eduardo Nakano<EMAIL_ADDRESS>Department of Statistics, University of Brasília, 70910-900, Brasília, Brazil
Helton Saulo<EMAIL_ADDRESS>Department of Statistics, University of
Brasília, 70910-900, Brasília, Brazil

###### Abstract

In this paper, we derive closed-form estimators for the parameters of some
probability distributions belonging to the exponential family. A bootstrap
bias-reduced version of these proposed closed-form estimators are also
derived. A Monte Carlo simulation is performed for the assessment of the
estimators. The results are seen to be quite favorable to the proposed
bootstrap bias-reduce estimators.

Keywords. Exponential family $\cdot$ Maximum likelihood method $\cdot$ Monte
Carlo simulation $\cdot$ `R` software.
Mathematics Subject Classification (2010). MSC 60E05 $\cdot$ MSC 62Exx $\cdot$
MSC 62Fxx.

##  1 Introduction

The maximum likelihood (ML) method is certainly the most popular technique for
obtaining point estimates of the parameters of a distribution. However, it is
rare for multi-parameter probability distributions to have closed-form
expressions for the ML estimators. Although numerical optimization algorithms
are used to solve ML estimation problems that have no closed-form expressions,
the availability of a closed-form estimator can avoid some difficulties that
can occur when using iterative methods. For example, the problems of
estimation in the limits of parametric space can cause the non-convergence of
the algorithm and the requirement to adopt a multi-start approach increases
computing time. In fact, the low computational cost is the main advantage of
closed-form estimators, especially in the field of real-time processing.

In this context, the aim of this work is to provide closed-form estimators for
the parameters of a parametric set of probability distributions that belongs
to the exponential family of the following form

$\displaystyle
f(x;\psi)=h(x)\exp\left(\sum_{j=1}^{2}\theta_{j}(\psi)T_{j}(x)-\log(c(\theta(\psi)))\right),$

where $x\in D\subseteqq\mathbb{R}$, $\psi=(\mu,\sigma)$, $\mu>0$, $\sigma>0$,
$\theta(\psi)=(\theta_{1}(\psi),\theta_{2}(\psi))$,
$\theta_{1}(\psi)=-\mu\sigma$, $\theta_{2}(\psi)=\mu$,
$T_{1}:D\subseteqq\mathbb{R}\to(0,\infty)$ is a real strictly monotone twice
differentiable function,

$\displaystyle h(x)={|T_{1}^{\prime}(x)|\over T_{1}(x)},\quad
T_{2}(x)=\log(T_{1}(x)),$

with $c(\theta(\psi))={\Gamma(\mu)/(\sigma^{\mu}\mu^{\mu})}$ being the
normalizing constant. In simple terms, $f(x;\psi)$ has the following simple
form

$\displaystyle
f(x;\psi)={(\mu\sigma)^{\mu}\over\Gamma(\mu)}{|T_{1}^{\prime}(x)|\over
T_{1}(x)}\exp\left\\{-\mu\sigma T_{1}(x)+\mu\log(T_{1}(x))\right\\},$ (1)

with $x\in D\subseteqq\mathbb{R}$, $\psi=(\mu,\sigma)$ and $\mu>0,\sigma>0$.
In the above, $T_{1}^{\prime}(x)$ denotes the derivative of $T_{1}(x)$ with
respect to $x$. Note that a family of distributions equivalent to (1) has
appeared in [16].

Table 1 presents some examples of generators $T_{1}(x)$ for use in (1).

Several studies have presented closed-form estimators derived from likelihood
equations. In [24] the authors derived a closed-form estimator of the
parameters of the gamma distribution from generalised gamma distribution. The
generalized gamma can be obtained by a power transformation $Y=X^{1/\gamma}$,
where $X$ has gamma distribution and $\gamma>0$ is a power parameter. Similar
ideas were adopted to develop closed-form estimators for the parameter of the
Nakagami distribution (see [18] and [25]) and the weighted Lindley
distribution [11]. Recently, in reference [12] was presented a new procedure
for obtaining a closed-form estimator for the family of distributions using an
extension of the Box-Cox transformation. This work adopts the same idea of
[24] and [5] to develop a closed-form estimator for the parameters of some
probability distributions of the exponential family (1).

The rest of the paper proceeds as follows. In Section 2, we describe briefly a
generalization of exponential family and its basic properties. The new
proposed estimation methods and some asymptotic results are described in
Sections 3 and 4, respectively. Finally, in Section 5, we perform a Monte
Carlo simulation study to assess the performance of a bootstrap bias-reduced
version of these proposed closed-form estimators.

Table 1: Some examples of generators $T_{1}(x)$ of exponential family (1), with $D=(0,\infty)$. Distribution | $\mu$ | $\sigma$ | $T_{1}(x)$
---|---|---|---
Nakagami [13] | $m$ | ${1\over\Omega}$ | $x^{2}$
Maxwell-Boltzmann [7] | ${3\over 2}$ | ${1\over 3\beta^{2}}$ | $x^{2}$
Rayleigh [19] | $1$ | ${1\over 2\beta^{2}}$ | $x^{2}$
Gamma [21] | $\alpha$ | ${1\over\alpha\beta}$ | $x$
Inverse gamma [4] | $\alpha$ | ${1\over\alpha\beta}$ | ${1\over x}$
$\delta$-gamma [17] | ${\beta\over\delta}$ | ${1\over\beta}$ | $x^{\delta}$
Weibull [9] | $1$ | ${1\over\beta^{\delta}}$ | $x^{\delta}$
Inverse Weibull (Fréchet) [10] | $1$ | ${1\over\beta^{\delta}}$ | ${1\over x^{\delta}}$
Generalized gamma [21] | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | $x^{\delta}$
Generalized inverse gamma [14] | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | ${1\over x^{\delta}}$
New log-generalized gamma | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | $[\exp(x)-1]^{\delta}$
New log-generalized inverse gamma | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | $\big{[}\exp\big{(}{1\over x}\big{)}-1\big{]}^{\delta}$
New exponentiated generalized gamma | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | $\log^{\delta}(x+1)$
New exponentiated generalized inverse gamma | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | $\log^{\delta}\big{(}{1\over x}+1\big{)}$
New modified log-generalized gamma | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | $\exp^{\delta}\big{(}x-{1\over x}\big{)}$
New extended log-generalized gamma | ${\alpha\over\delta}$ | ${\delta\over\alpha\beta^{\delta}}$ | $x^{\delta}[\exp(x)-1]^{\delta}$
Chi-squared [9] | ${\nu\over 2}$ | ${1\over\nu}$ | $x$
Scaled inverse chi-squared [2] | ${\nu\over 2}$ | $\tau^{2}$ | ${1\over x}$
Gompertz [8] | $1$ | $\alpha$ | $\exp(\delta x)-1$
Modified Weibull extension [23] | $\lambda\alpha$ | $1$ | $\exp\big{[}\left({x\over\alpha}\right)^{\beta}\big{]}-1$
Traditional Weibull [15] | $1$ | $a$ | $x^{b}[\exp(cx^{d})-1]$
Flexible Weibull [1] | $1$ | $a$ | $\exp\left(bx-{c\over x}\right)$
Burr type XII (Singh-Maddala) [3] | $1$ | $k$ | $\log(x^{c}+1)$
Dagum (Mielke Beta-Kappa) [6] | $1$ | $k$ | $\log\big{(}{1\over x^{c}}+1\big{)}$

In this table, $m\geqslant{1/2}$ and $\Omega$, $a,b,c,d,k$,
$\alpha,\delta,\lambda,\beta,\nu$ and $\tau^{2}$ are positive parameters.

##  2 A generalization of exponential family

By using the increasing transformation $Y=X^{1/p}$, $p>0$, considered in [24]
and [5], where $X$ has the probability density function (PDF) in (1), it is
clear that the PDF of $Y$ is written as

$\displaystyle
f(y;\psi,p)=p\,{(\mu\sigma)^{\mu}\over\Gamma(\mu)}\,{|T_{1}^{\prime}(y^{p})|y^{p-1}\over
T_{1}(y^{p})}\exp\left\\{-\mu\sigma
T_{1}(y^{p})+\mu\log(T_{1}(y^{p}))\right\\},$ (2)

where $y\in D\subseteqq\mathbb{R}$, $\psi=(\mu,\sigma)$ and
$\mu>0,\sigma>0,p>0$.

In what follows we state some theoretical probabilistic properties of model
(2). Since these properties are obtained by standard calculations in the
distribution theory literature, the respective proofs are omitted.

######  Proposition 2.1 (Stochastic representation).

If $Z\sim{\rm Gamma}(\mu,1/(\mu\sigma))$ then $Y$ has the stochastic
representation $Y\stackrel{{\scriptstyle d}}{{=}}[T^{-1}_{1}(Z)]^{1/p},$ with
$\stackrel{{\scriptstyle d}}{{=}}$ being equality in distribution and
$T^{-1}_{1}$ denoting the inverse function of $T_{1}$. Conversely, if $Y$ is
distributed according (2), then $Z\stackrel{{\scriptstyle
d}}{{=}}T_{1}(Y^{p})\sim{\rm Gamma}(\mu,1/(\mu\sigma))$.

######  Proposition 2.2 (Quantiles).

The $p$-quantile of $Y$ distributed according (2), denoted by $Q_{Y}(p)$,
$p\in(0,1)$, is expressed as

$\displaystyle Q_{Y}(p)=\begin{cases}[T_{1}^{-1}(Q_{Z}(p))]^{1/p},&\text{if}\
T\ \text{is increasing},\\\ \left[T_{1}^{-1}\left({1\over
Q_{Z^{*}}(p)}\right)\right]^{1/p},&\text{if}\ T\ \text{is
decreasing}.\end{cases}$

In the above, $Z\sim{\rm Gamma}(\mu,1/(\mu\sigma))$ and $Z^{*}={1/Z}\sim{\rm
Inv}$-${\rm Gamma}(\mu,{1/(\mu\sigma)})$.

The following three propositions inform about the existence of moments
corresponding to distribution (2).

######  Proposition 2.3.

If $Y$ is distributed according (2), then the positive order moments of $Y$,
denoted by $\mathbb{E}(Y^{q})$, $0<q<ps,$ exist whenever $T_{1}(x)\geqslant
Cx^{s}$, $x>0$, $T_{1}$ increasing, for some $C>0$ and $s>0$.

######  Remark 2.4.

Note that the generator $T_{1}(x)=x^{b}[\exp(cx^{d})-1]$, $x>0$, of
traditional Weibull distribution in Table 1 satisfies the condition
$T_{1}(x)\geqslant Cx^{s}$ of Proposition 2.3 with $C=c$ and $s=b+d$. In
particular, the generators of Gompertz and modified Weibull extension in Table
1 also satisfy this condition.

######  Proposition 2.5.

If $Y$ is distributed according (2) with $T_{1}(x)=\log(x^{s}+1)$, $x>0$, for
some $s\neq 0$, then the real moments of $Y$, denoted by $\mathbb{E}(Y^{q})$,
$q<\min\\{0,ps\\}$, exist.

######  Remark 2.6.

Note that Proposition 2.5 includes the generators $T_{1}(x)$ of Burr Type XII
and Dagum distributions (see Table 1).

######  Proposition 2.7.

If $Y$ is distributed according (2) with $T_{1}(x)=Cx^{-s}$, $x>0$, for some
$C>0$ and $s\neq 0$, then the real moments of $Y$ are given by

$\displaystyle\mathbb{E}(Y^{q})=\left({\mu\sigma\over C}\right)^{q\over
ps}\,\dfrac{\Gamma\left(\mu-{q\over ps}\right)}{\Gamma(\mu)},\quad\mu-{q\over
ps}>0.$

######  Remark 2.8.

Note that Proposition 2.7 includes the generators $T_{1}(x)$ of Nakagami,
Maxwell-Boltzmann, Rayleigh, gamma, inverse gamma, $\delta$-gamma, Weibull,
inverse Weibull, generalized gamma, generalized inverse gamma, chi-squared and
scaled inverse chi-squared (see Table 1).

##  3 The new estimators

Let $\\{Y_{i}:i=1,\ldots,n\\}$ be a univariate random sample of size $n$ from
$Y$. The (random) log-likelihood function for $(\psi,p)$ is written as

$\displaystyle\log(L(\psi,p))$
$\displaystyle=\mskip-3.0mun\log(p)\mskip-3.0mu+\mskip-3.0mu{n\mu}\log(\mu)\mskip-3.0mu+\mskip-3.0mu{{n\mu}\log(\sigma)\mskip-3.0mu-\mskip-3.0mun\log(\Gamma(\mu))}$
$\displaystyle+\sum_{i=1}^{n}\log(|T_{1}^{\prime}(Y_{i}^{p})|)+(p-1)\sum_{i=1}^{n}\log(Y_{i})$
$\displaystyle-\mu\sigma\sum_{i=1}^{n}T_{1}(Y^{p}_{i})+(\mu-1)\sum_{i=1}^{n}\log(T_{1}(Y^{p}_{i})).$

A simple calculus shows that the elements of the (random) score vector are
given by

$\displaystyle{\partial\log(L(\psi,p))\over\partial\mu}$
$\displaystyle=n\log(\mu)+n\log(\sigma)+n-n\psi^{(0)}(\mu)-\sigma\sum_{i=1}^{n}T_{1}(Y^{p}_{i})+\sum_{i=1}^{n}\log(T_{1}(Y^{p}_{i})),$
$\displaystyle{\partial\log(L(\psi,p))\over\partial\sigma}$
$\displaystyle={n\mu\over\sigma}-\mu\sum_{i=1}^{n}T_{1}(Y^{p}_{i}),$
$\displaystyle{\partial\log(L(\psi,p))\over\partial p}$ $\displaystyle={n\over
p}+\sum_{i=1}^{n}{T_{1}^{\prime\prime}(Y^{p}_{i})\over
T_{1}^{\prime}(Y_{i}^{p})}\,Y^{p}_{i}\log(Y_{i})+\sum_{i=1}^{n}\log(Y_{i})$
$\displaystyle-\mu\sigma\sum_{i=1}^{n}T_{1}^{\prime}(Y^{p}_{i})Y^{p}_{i}\log(Y_{i})+(\mu-1)\sum_{i=1}^{n}{T_{1}^{\prime}(Y^{p}_{i})\over
T_{1}(Y^{p}_{i})}\,Y^{p}_{i}\log(Y_{i}),$

where $\psi^{(m)}(x)=\partial^{m+1}\log(\Gamma(x))/\partial x^{m+1}$ is the
polygamma function of order $m$. Setting these equal to zero and solving the
system of equations gives the ML estimators of $(\psi,p)$. In particular, by
resolving ${\partial\log(L(\psi,p))/\partial\sigma}=0$, we can express
$\sigma$ as a function of $p$:

$\displaystyle{\sigma}(p)={n\over\sum_{i=1}^{n}T_{1}(Y_{i}^{{p}})}.$ (3)

Substitute (3) into ${\partial\log(L(\psi,p))/\partial p}=0$ to give

$\displaystyle\mskip-3.0mu{\mu}(p)\mskip-3.0mu=\mskip-3.0mu\dfrac{{n\over
p}\mskip-3.0mu+\mskip-3.0mu\sum_{i=1}^{n}\mskip-3.0mu\log(Y_{i})\mskip-3.0mu+\mskip-3.0mu\sum_{i=1}^{n}\mskip-3.0mu\left[{T_{1}^{\prime\prime}(Y^{p}_{i})\over
T_{1}^{\prime}(Y_{i}^{p})}\mskip-3.0mu-\mskip-3.0mu{T_{1}^{\prime}(Y^{p}_{i})\over
T_{1}(Y^{p}_{i})}\right]\mskip-3.0muY^{p}_{i}\log(Y_{i})}{{\sigma}(p)\sum_{i=1}^{n}T_{1}^{\prime}(Y^{p}_{i})Y^{p}_{i}\log(Y_{i})-\sum_{i=1}^{n}{T_{1}^{\prime}(Y^{p}_{i})Y^{p}_{i}\log(Y_{i})\over
T_{1}(Y^{p}_{i})}}.$ (4)

Furthermore, notice that ${p}$ is obtained solving the non-linear equation

$\displaystyle\log({\mu}(p))-\psi^{(0)}({\mu}(p))=\log\left({1\over\sigma(p)}\right)-{1\over
n}\sum_{i=1}^{n}\log(T_{1}(Y^{{p}}_{i})).$ (5)

Now, return to the distribution in (1). We take $p=1$ in (3) and (4) to obtain
the new estimators for $\mu$ and $\sigma$ as

$\displaystyle\widehat{\sigma}={1\over{1\over n}\sum_{i=1}^{n}T_{1}(Y_{i})}$
(6)

and

$\displaystyle\widehat{\mu}\mskip-3.0mu=\mskip-3.0mu\dfrac{1+{1\over
n}\mskip-3.0mu\sum_{i=1}^{n}\mskip-3.0mu\log(Y_{i})+\mskip-3.0mu{1\over
n}\mskip-3.0mu\sum_{i=1}^{n}\mskip-3.0mu\mskip-3.0mu\left[{T_{1}^{\prime\prime}(Y_{i})\over
T_{1}^{\prime}(Y_{i})}-\mskip-3.0mu{T_{1}^{\prime}(Y_{i})\over
T_{1}(Y_{i})}\right]\mskip-3.0muY_{i}\log(Y_{i})}{{\widehat{\sigma}\over
n}\sum_{i=1}^{n}T_{1}^{\prime}(Y_{i})Y_{i}\log(Y_{i})-{1\over
n}\sum_{i=1}^{n}{T_{1}^{\prime}(Y_{i})Y_{i}\log(Y_{i})\over T_{1}(Y_{i})}}.$
(7)

Note that the ML estimator of $\sigma$, denoted by $\widehat{\sigma}_{\rm
MLE}$, satisfies the identity $\widehat{\sigma}_{\rm MLE}=\widehat{\sigma}$.
Furthermore, the ML estimator of $\mu$, denoted by $\widehat{\mu}_{\rm MLE}$,
is solution of the following equation, which is obtained from (5) by taking
$p=1$,

$\displaystyle\log({\mu})-\psi^{(0)}({\mu})=\log\left({1\over\sigma}\right)-{1\over
n}\sum_{i=1}^{n}\log(T_{1}(Y_{i})).$ (8)

######  Proposition 3.1.

The equation (8) is an unbiased estimating equation for $\mu$, i.e.,

$\displaystyle\log({\mu})-\psi^{(0)}({\mu})=\mathbb{E}\left[\log\left({1\over\sigma}\right)-{1\over
n}\sum_{i=1}^{n}\log(T_{1}(Y_{i}))\right].$

###### Proof.

The proof is immediate since $T_{1}(Y_{i})\sim{\rm Gamma}(\mu,1/(\mu\sigma))$
(Proposition 2.1) and
$\mathbb{E}[\log(T_{1}(Y_{i}))]=\psi^{(0)}(\mu)-\log(\mu\sigma)$, for
$i=1,\ldots,n$. ∎

######  Proposition 3.2.

The solution in $\mu$ to equation (8) exists and is unique.

###### Proof.

Combining (8) with (6), equation (8) is written as

$\displaystyle\log({\mu})-\psi^{(0)}({\mu})=\log\left({1\over
n}\sum_{i=1}^{n}T_{1}(Y_{i})\right)-{1\over
n}\sum_{i=1}^{n}\log(T_{1}(Y_{i})).$ (9)

We denote the right-hand side of (9) by $H(Y_{1},\ldots,Y_{n})$. By using the
Binet’s second integral for the gamma function:

$\displaystyle\log(x)-\psi^{(0)}(x)={1\over
2x}+2\int_{0}^{\infty}{t\over(t^{2}+x^{2})[\exp(2\pi t)-1]}{\rm d}t,$ (10)

note that $\log({\mu})-\psi^{(0)}({\mu})$ is a positive monotone decreasing
function of $\mu$ with a finite limit ($=0$) as $\mu\to\infty$. Since
$H(Y_{1},\ldots,Y_{n})$ is non-negative (by Jensen’s inequality) and is
constant in $\mu$, it would then follow that the plots of
$\log({\mu})-\psi^{(0)}({\mu})$ and $H(Y_{1},\ldots,Y_{n})$ would intersect
exactly once, at the ML estimator of $\mu$. ∎

######  Theorem 3.3.

Let $T_{1}(x)=Cx^{-s}$, $x>0$, for some $C>0$ and $s\neq 0$. With probability
1, $\widehat{\mu}<2\widehat{\mu}_{\rm MLE}$.

###### Proof.

If $T_{1}(x)=Cx^{-s}$, for some $C>0$ and $s\neq 0$, then
$T_{1}^{-1}(x)=(x/C)^{-1/s}$, $T_{1}^{\prime}(x)=-Csx^{-s-1}$,
$T_{1}^{\prime\prime}(x)=Cs(s+1)x^{-s-2}$,
$T_{1}^{\prime\prime}(T_{1}^{-1}(x))=Cs(s+1)(x/C)^{1+2/s}$ and
$T_{1}^{\prime}(T_{1}^{-1}(x))=-Cs(x/C)^{1+1/s}$. Consequently,
$1/\widehat{\mu}$, with $\widehat{\mu}$ as given in (7), is written as

$\displaystyle{1\over\widehat{\mu}}={{1\over
n}\sum_{i=1}^{n}Y_{i}^{-s}\log(Y_{i}^{-s})\over{1\over
n}\sum_{i=1}^{n}Y_{i}^{-s}}-{1\over n}\sum_{i=1}^{n}\log(Y_{i}^{-s}).$ (11)

As $\widehat{\mu}_{\rm MLE}$ is the only solution of equation (9) (Proposition
3.2) and from (10): $\log(x)-\psi^{(0)}(x)\geqslant(2x)^{-1}$ for $x>0$, we
get

$\displaystyle\log\left({1\over n}\sum_{i=1}^{n}Y_{i}^{-s}\right)-{1\over
n}\sum_{i=1}^{n}\log(Y_{i}^{-s})\geqslant{1\over 2\widehat{\mu}_{\rm MLE}}.$
(12)

If we prove that

$\displaystyle{{1\over n}\sum_{i=1}^{n}Y_{i}^{-s}\log(Y_{i}^{-s})\over{1\over
n}\sum_{i=1}^{n}Y_{i}^{-s}}\geqslant\log\left({1\over
n}\sum_{i=1}^{n}Y_{i}^{-s}\right),$

the proof of theorem follows by combining inequalities (11) and (12). But this
is immediate by using the convexity of function $f(x)=x\log(x)$. ∎

##  4 Large sample properties

In this section, we show that the new estimators given in Section 3 are
strongly consistent.

Let $\\{Y_{i}:i=1,\ldots,n\\}$ be a random sample from $Y$ with PDF given in
(1). By applying strong law of large numbers and by using Proposition 2.1, we
have

$\displaystyle{1\over n}\sum_{i=1}^{n}T_{1}(Y_{i})\stackrel{{\scriptstyle\rm
a.s.}}{{\longrightarrow}}\mathbb{E}[T_{1}(Y)]=\mathbb{E}(Z)={1\over\sigma};\quad
Z\sim{\rm Gamma}\left(\mu,{1\over\mu\sigma}\right);$ $\displaystyle{1\over
n}\sum_{i=1}^{n}\log(Y_{i})\stackrel{{\scriptstyle\rm
a.s.}}{{\longrightarrow}}\mathbb{E}[\log(Y)];$ $\displaystyle{1\over
n}\sum_{i=1}^{n}\left[{T_{1}^{\prime\prime}(Y_{i})\over
T_{1}^{\prime}(Y_{i})}-{T_{1}^{\prime}(Y_{i})\over
T_{1}(Y_{i})}\right]Y_{i}\log(Y_{i})\stackrel{{\scriptstyle\rm
a.s.}}{{\longrightarrow}}\mathbb{E}\left\\{\left[{T_{1}^{\prime\prime}(Y)\over
T_{1}^{\prime}(Y)}-{T_{1}^{\prime}(Y)\over T_{1}(Y)}\right]Y\log(Y)\right\\};$
$\displaystyle{1\over
n}\sum_{i=1}^{n}T_{1}^{\prime}(Y_{i})Y_{i}\log(Y_{i})\stackrel{{\scriptstyle\rm
a.s.}}{{\longrightarrow}}\mathbb{E}[T_{1}^{\prime}(Y)Y\log(Y)];$
$\displaystyle{1\over n}\sum_{i=1}^{n}{T_{1}^{\prime}(Y_{i})\over
T_{1}(Y_{i})}\,Y_{i}\log(Y_{i})\stackrel{{\scriptstyle\rm
a.s.}}{{\longrightarrow}}\mathbb{E}\left[{T_{1}^{\prime}(Y)\over
T_{1}(Y)}\,Y\log(Y)\right].$

As $T_{1}(Y)\stackrel{{\scriptstyle d}}{{=}}Z\sim{\rm
Gamma}(\mu,{1/(\mu\sigma)})$ (see Proposition 2.1) and
$\mathbb{E}[T_{1}(Y)]=\mathbb{E}(Z)={1/\sigma}$, a simple application of
continuous-mapping theorem gives

$\displaystyle\widehat{\mu}\stackrel{{\scriptstyle\rm
a.s.}}{{\longrightarrow}}\dfrac{\mathbb{E}\left\\{1+[1+U_{T_{1}}(Z)]\log(T_{1}^{-1}(Z))\right\\}}{\mathbb{E}\left[\left(\sigma-{1\over
Z}\right)T_{1}^{\prime}(T_{1}^{-1}(Z))T_{1}^{-1}(Z)\log(T_{1}^{-1}(Z))\right]}\equiv
I_{T_{1}},$ (13)

with $U_{T_{1}}$ defined as
$U_{T_{1}}(z)\equiv[{T_{1}^{\prime\prime}(T_{1}^{-1}(z))/T_{1}^{\prime}(T_{1}^{-1}(z))}-{T_{1}^{\prime}(T_{1}^{-1}(z))/z}]T_{1}^{-1}(z)$,
$z>0$, and

$\displaystyle\widehat{\sigma}\stackrel{{\scriptstyle\rm
a.s.}}{{\longrightarrow}}\dfrac{1}{\mathbb{E}[T_{1}(Y)]}=\sigma,$ (14)

where $\widehat{\mu}$ and $\widehat{\sigma}$ are the estimators, given in (7)
and (6), for $\mu$ and $\sigma$, respectively. That is, regardless of the
choice of generator $T_{1}$, the estimator $\widehat{\sigma}$ for $\sigma$ is
strongly consistent.

######  Proposition 4.1.

If $U_{T_{1}}(z)=-1,\ z>0,$ then $I_{T_{1}}=\mu.$ Furthermore, from (13), the
estimator $\widehat{\mu}$ given in (7) is strongly consistent for $\mu$.

###### Proof.

Note that $U_{T_{1}}(z)=-1$ is a second-order nonlinear ordinary differential
equation which can be written as

$\displaystyle{{\rm d}^{2}\log(T_{1}(x))\over{\rm d}x^{2}}+{1\over x}\,{{\rm
d}\log(T_{1}(x))\over{\rm d}x}=0,$

with $x=T_{1}^{-1}(z)$. It is not difficult to verify that the differential
equation solution is given by $T_{1}(x)=Cx^{-s}$, for some $C>0$ and $s\neq
0$. Consequently, $T_{1}^{-1}(x)=(x/C)^{-1/s}$,
$T_{1}^{\prime}(x)=-Csx^{-s-1}$, $T_{1}^{\prime\prime}(x)=Cs(s+1)x^{-s-2}$,
$T_{1}^{\prime\prime}(T_{1}^{-1}(x))=Cs(s+1)(x/C)^{1+2/s}$ and
$T_{1}^{\prime}(T_{1}^{-1}(x))=-Cs(x/C)^{1+1/s}$. Hence, $I_{T_{1}}$ in (13)
is written as

$\displaystyle I_{T_{1}}=\dfrac{1}{\displaystyle\mathbb{E}[(\sigma
Z-1)\log(Z)]},$ (15)

where $Z\sim{\rm Gamma}\left(\mu,{1/(\mu\sigma)}\right)$. Substituting the
known identities
$\mathbb{E}[Z\log(Z)]=[\psi^{(0)}(\mu+1)-\log(\mu\sigma)]/\sigma$ and
$\mathbb{E}[\log(Z)]=\psi^{(0)}(\mu)-\log(\mu\sigma)$, into (15), it follows
that $I_{T_{1}}(Z)=\mu.$ Therefore, the proof has been completed. ∎

######  Lemma 4.2.

If $U_{T_{1}}(z)\neq-1,\ z>0,$ then the following holds:

$\displaystyle I_{T_{1}}=\dfrac{\mu}{1+\mu\,{\varepsilon_{2}\over
g_{1}\left({1\over\sigma}\right)}}+\dfrac{{g_{1}^{\prime\prime}({1\over\sigma})\over
2g_{1}\left({1\over\sigma}\right)}\,{1\over\sigma^{2}}+\mu\,{\varepsilon_{1}\over
g_{1}\left({1\over\sigma}\right)}}{1+\mu\,{\varepsilon_{2}\over
g_{1}\left({1\over\sigma}\right)}},$

where (for $z>0$)

$\displaystyle\begin{array}[]{lll}&g_{1}(z)=1+[1+U_{T_{1}}(z)]\log(T_{1}^{-1}(z)),\\\\[8.5359pt]
&g_{2}(z)=\left(\sigma-\frac{1}{z}\right)T_{1}^{\prime}(T_{1}^{-1}(z))T_{1}^{-1}(z)\log(T_{1}^{-1}(z)),\\\\[8.5359pt]
&\varepsilon_{i}=\mathbb{E}\left[\int_{1\over\sigma}^{Z}{g_{i}^{\prime\prime\prime}(\xi)\over
2}\,(Z-\xi)^{2}{\rm d}\xi\right],\quad i=1,2.\end{array}$

###### Proof.

In case $U_{T_{1}}(z)\neq-1$ we resort to the use of Taylor theorem. Indeed,
since $g_{i}$ is a function that has infinite continuous derivatives in any
neighborhood of $z=\mathbb{E}(Z)$, by using Taylor’s formula with integral
remainder, we can write

$\displaystyle
g_{i}(Z)=g_{i}(\mathbb{E}(Z))+{g_{i}^{\prime\prime}(\mathbb{E}(Z))\over
2}\,[Z-\mathbb{E}(Z)]^{2}+\int_{\mathbb{E}(Z)}^{Z}{g_{i}^{\prime\prime\prime}(\xi)\over
2}\,(Z-\xi)^{2}{\rm d}\xi,\quad i=1,2.$

Taking expectations to each member of the above identity, we obtain

$\displaystyle\mathbb{E}[g_{i}(Z)]=g_{i}\left({1\over\sigma}\right)+{g_{i}^{\prime\prime}({1\over\sigma})\over
2}\,{1\over\mu\sigma^{2}}+\varepsilon_{i},\quad i=1,2,$

because $Z\sim{\rm Gamma}(\mu,{1/(\mu\sigma)})$.

A laborious calculation shows that
$g_{2}^{\prime\prime}\left({1/\sigma}\right)=2\sigma^{2}g_{1}\left({1/\sigma}\right).$
Hence, $I_{T_{1}}$ in (13) is written as

$\displaystyle I_{T_{1}}={\mathbb{E}[g_{1}(Z)]\over\mathbb{E}[g_{2}(Z)]}$
$\displaystyle=\dfrac{g_{1}\left({1\over\sigma}\right)+{g_{1}^{\prime\prime}({1\over\sigma})\over
2}\,{1\over\mu\sigma^{2}}+\varepsilon_{1}}{g_{1}\left({1\over\sigma}\right){1\over\mu}+\varepsilon_{2}}.$

We have thus completed the proof of lemma. ∎

In the following result we use the assertion that $f(x)=o(g(x))$ as
$x\to\infty$ (read $f(x)$ is little-$o$ of $g(x)$). This assertion means that
$g(x)$ grows much faster than $f(x)$, and is equivalent to
$\lim_{x\to\infty}{f(x)/g(x)}=0.$

By combining (13) with Lemma 4.2, it gives the following theorem.

######  Theorem 4.3.

Let $\varepsilon_{i}$ and $g_{1}$ be as in Lemma 4.2. Under the conditions
(for $i=1,2$)

$\displaystyle\varepsilon_{i}={o}\left(g_{1}\left({1\over\sigma}\right)\right),\quad{g_{1}^{\prime\prime}\left({1\over\sigma}\right)}={o}\left(\sigma^{2}g_{1}\left({1\over\sigma}\right)\right),$
(16)

as $\sigma\to\infty$, the estimator $\widehat{\mu}$ given in (7) is strongly
consistent for $\mu$.

######  Example 4.1.

By considering $T_{1}(x)=\exp(x)-1$ the functions $g_{1}$ and $g_{2}$, defined
in Lemma 4.2, are written as

$\displaystyle g_{1}(z)=1+\left[1-{\log(z+1)\over z}\right]\log(\log(z+1)),$
$\displaystyle g_{2}(z)=\left(\sigma-{1\over
z}\right)(z+1)\log(z+1)\log(\log(z+1)),$

respectively. Using the graphs of the functions
$g_{1}^{\prime\prime\prime}(z)$ and $g_{2}^{\prime\prime\prime}(z)$ note that
$\sup_{\xi>0}g_{i}^{\prime\prime\prime}(\xi)<\infty$ and that it is always
possible to choose a sufficiently large $\sigma>0$ so that
$g_{i}^{\prime\prime\prime}(\xi)\geqslant
g_{i}^{\prime\prime\prime}(1/\sigma)$, $i=1,2$. So, $\varepsilon_{i}$ in Lemma
4.2 can be limited as follows:

$\displaystyle{1\over
3\mu^{2}\sigma^{3}}\,\,g_{i}^{\prime\prime\prime}\left({1\over\sigma}\right)\leqslant\varepsilon_{i}\leqslant{1\over
3\mu^{2}\sigma^{3}}\,\sup_{\xi>0}g_{i}^{\prime\prime\prime}(\xi).$

Since $g_{1}$ is a positive function, by dividing all members of the above
inequalities by $g_{1}(1/\sigma)$ and then, by applying Squeeze theorem, it
follows that $\lim_{\sigma\to\infty}{\varepsilon_{i}/g_{1}({1/\sigma})}=0,\
i=1,2.$

On the other hand, it is a simple task to check that
$\lim_{\sigma\to\infty}{g_{1}^{\prime\prime}({1/\sigma})/(\sigma^{2}g_{1}({1/\sigma}))}=0.$

Hence, the conditions in (16) are satisfied. Then, by applying Theorem 4.3,
the estimator $\widehat{\mu}$ given in (7) is strongly consistent for $\mu$.

##  5 Simulation study

In this section, we carry out a Monte Carlo simulation study for evaluating
the performance of the proposed estimators. Due to space limitations, we only
present the results for the new log-generalized gamma distribution. In the
following, we present the proposed ML estimators for the aforementioned
distribution. We then propose a bootstrap bias-reduced version of the proposed
ML estimators, as they are biased [18].

New log-generalized gamma distribution. By considering the parameters
$\mu={\alpha}$, $\sigma={1/(\alpha\beta)}$ and generator $T_{1}(x)=\exp(x)-1$
of new log-generalized gamma (with $\delta=1$), given in Table 1, from (6) and
(7) the estimators for $\beta$ and $\alpha$ are given by

$\displaystyle{\widehat{\beta}}=\dfrac{{1\over
n}\sum_{i=1}^{n}\exp(Y_{i})Y_{i}\log(Y_{i})}{1+{1\over
n}\sum_{i=1}^{n}\log(Y_{i})-{1\over
n}\sum_{i=1}^{n}{Y_{i}\log(Y_{i})\over\exp(Y_{i})-1}}-{{1\over
n}\sum_{i=1}^{n}[\exp(Y_{i})-1]{1\over
n}\sum_{i=1}^{n}{\exp(Y_{i})Y_{i}\log(Y_{i})\over\exp(Y_{i})-1}\over 1+{1\over
n}\sum_{i=1}^{n}\log(Y_{i})-{1\over
n}\sum_{i=1}^{n}{Y_{i}\log(Y_{i})\over\exp(Y_{i})-1}}$

and
$\widehat{\alpha}=(1/\widehat{\beta}){(1/n)}\sum_{i=1}^{n}[\exp(Y_{i})-1]$,
respectively.

Now, we propose bootstrap bias-reduced ML estimators as

$\displaystyle\widehat{\theta}^{*}=2\widehat{\theta}-\frac{1}{B}\sum_{b=1}^{B}\widehat{\theta}^{(b)},$

where
$\widehat{\theta}\in\\{\widehat{\Omega},\widehat{m},\widehat{\beta},\widehat{\alpha},\widehat{\tau}^{\,2},\widehat{\nu}\\}$,
$B$ is the number of bootstrap replications, and $\widehat{\theta}^{(b)}$ is
the $b$-th bootstrap replicate from the $b$-th bootstrap sample. To study the
performance of the proposed bootstrap bias-reduced estimators, we computed the
relative bias (RB) and root mean square error (RMSE), as

$\displaystyle\widehat{\textrm{RB}}(\widehat{\theta}^{*})=\left|\frac{\frac{\sum_{i=1}^{N}\widehat{\theta}^{*(i)}}{N}-\theta}{\theta}\right|,\
\widehat{\mathrm{RMSE}}(\widehat{\theta}^{*})={\sqrt{\frac{\sum_{i=1}^{N}(\widehat{\theta}^{*(i)}-\theta)^{2}}{N}}},$

where $\theta$ and $\widehat{\theta}^{*(i)}$ are the true parameter value and
its $i$-th bootstrap bias-reduced estimate, and $N$ is the number of Monte
Carlo replications; see [20] and [22]. The sample sizes and true values of the
parameters considered are: $n\in\\{20,50,100,200,400,600\\}$,
$\alpha\in\\{0.5,1,2,4,6\\}$, and $\beta=1$. The number of Monte Carlo
replications was $N=1,000$ and the number of bootstrap replications was
$B=200$. The required numerical evaluations were implemented using the `R`
software through the use of some packages already available at
http://cran.r-project.org and some new computational routines.

Figure 1 (a)-(d) shows the results of Monte Carlo simulation study to assess
the performance of the bootstrap bias-reduced ML estimators. In terms of
parameter recovery, we observe that both RBs and RMSEs approach zero as the
sample size ($n$) increases. In general terms, the results also show that the
proposed estimators have relatively low bias, even for small samples (similar
results are obtained for other models). Therefore, the proposed the bootstrap
bias-reduced ML estimators are good alternatives for estimating the parameters
associated with the distributions in Table 1.

Figure 1: Empirical RB and RMSE of the ML estimators for the new log-
generalized gamma distribution.

## References

  * [1] M. Bebbington, C. D. Lai, and R. Zitikis, “A flexible Weibull extension,” Reliab. Eng. Syst. Saf., vol. 92, pp. 719–726, June 2007.
  * [2] J. M. Bernardo and A. F. M. Smith, Bayesian Theory, Wiley, 1993.
  * [3] I. W. Burr, “Cumulative frequency functions,” Ann. Math. Stat., vol. 13, pp. 215–232, June 1942.
  * [4] J. D. Cook, Inverse gamma distribution, Online: http://www.johndcook.com/inversegamma.pdf, Technical, 2008.
  * [5] J. Cheng and N. C. Beaulieu, “Generalized moment estimators for the Nakagami fading parameter,” IEEE Commun. Lett., vol. 6, pp. 144–146, April 2002.
  * [6] C. Dagum, “A model of income distribution and the conditions of existence of moments of finite order,” Bull. Int. Stat. Inst. (Proceedings of the 40th Session of the ISI, Contributed Paper), vol. 46, pp. 199–205, 1975.
  * [7] R. C. Dunbar, “Deriving the Maxwell Distribution,” J. Chem. Educ., vol. 59, pp. 22–23, January 1982.
  * [8] B. Gompertz, “On the nature of the function expressive of the law of human mortality and on the new model of determining the value of life contingencies,” Phil. Trans. R. Soc., vol. 115, pp. 513–585, June 1825.
  * [9] N. L. Johnson, S. Kotz, and N. Balakrishnan, Continuous Univariate Distributions, New York: John Wiley & Sons, 1994.
  * [10] M. S. Khan, G. R. Pasha, and A. H. Pasha, “Theoretical analysis of inverse Weibull distribution,” WSEAS Trans. Math., vol. 7, pp. 30–38, February 2008.
  * [11] H.-M. Kim and Y.-H. Jang, “New closed-form estimators for weighted Lindley distribution,” J. Korean Statist. Soc., vol. 50, pp. 580–606, January 2021.
  * [12] H.-M. Kim, S. Kim, Y.-H. Jang, and J. Zhao, “New closed-form estimator and its properties.” J. Korean Statist. Soc., vol. 51, pp. 47–64, March 2022.
  * [13] D. Laurenson, Nakagami Distribution, Indoor Radio Channel Propagation Modeling by Ray Tracing Techniques, 1994.
  * [14] M. Lee and A. Gross, “Lifetime distributions under unknown environment,” J. Statist. Plann. Inference., vol. 29, pp. 137–143, September 1991
  * [15] S. Nadarajah and S. Kotz, “On some recent modifications of Weibull distribution,” IEEE Trans. Reliab., vol. 54, pp. 561–562, December 2005.
  * [16] A. D. C. Nascimento, M. Bourguignon, L. M. Zea, M. Santos-Neto, R. B. Silva, and G. M. Cordeiro, “The gamma extended Weibull family of distributions,” J. Stat. Theory Appl., vol. 13, pp. 1–16, March 2014.
  * [17] G. Rahman, S. Mubeen, A. Rehman, and M. Naz, “On $k$-Gamma and $k$-Beta Distributions and Moment Generating Functions,” J. Probab. Stat., Article ID 982013, 6 pages, July 2014.
  * [18] P. L. Ramos, F. Louzada, and E. Ramos, “An Efficient, Closed-Form MAP Estimator for Nakagami-m Fading Parameter,” IEEE Commun. Lett., vol. 20, pp. 2328–2331, November 2016.
  * [19] J. W. S. Rayleigh, “On the resultant of a large number of vibrations of the same pitch and of arbitrary phase,” Lond. Edinb. Phil. Mag, vol. 10, pp. 73–78, August 1880.
  * [20] H. Saulo, A. Dasilva, V. Leiva, L. Sánchez, and H. de la Fuente-Mella, “Log-symmetric quantile regression models,” Statist. Neerlandica, vol. 76, pp. 124–163, May 2022.
  * [21] E. W. Stacy, “A Generalization of the Gamma Distribution,” Ann. Math. Stat., vol. 33, pp. 1187–1192, September 1962.
  * [22] R. Vila, N. Balakrishnan, H. Saulo, and A. Protazio, “Bivariate log-symmetric models: Distributional properties, parameter estimation and an application to public spending data,” Braz. J. Probab. Stat., vol. 37, pp. 619–642, September 2023.
  * [23] M. Xie, Y. Tang, and T. N. Goh, “A modified Weibull extension with bathtub-shaped failure rate function,” Reliab. Eng. Syst. Saf., vol. 76, pp. 279–285, June 2002.
  * [24] Z-S. Ye, and N. Chen, “Closed-Form Estimators for the Gamma Distribution Derived from Likelihood Equations,” Amer. Statist., vol. 71, Issue 2, July 2017.
  * [25] J. Zhao, S. Kim, and H.-M. Kim, “Closed-form estimators and bias-corrected estimators for the Nakagami distribution,” Math. Comput. Simulation, vol. 185, pp. 308–324, July 2021.


# Benchmarking Domain Randomisation for
Visual Sim-to-Real Transfer

Raghad Alghonaim1 and Edward Johns1 1 The Robot Learning Lab at Imperial
College London {raa318<EMAIL_ADDRESS>
###### Abstract

Domain randomisation is a very popular method for visual sim-to-real transfer
in robotics, due to its simplicity and ability to achieve transfer without any
real-world images at all. Nonetheless, a number of design choices must be made
to achieve optimal transfer. In this paper, we perform a comprehensive
benchmarking study on these different choices, with two key experiments
evaluated on a real-world object pose estimation task. First, we study the
rendering quality, and find that a small number of high-quality images is
superior to a large number of low-quality images. Second, we study the type of
randomisation, and find that both distractors and textures are important for
generalisation to novel environments.

##  I Introduction

In recent years, deep learning has been successfully applied to a range of
robotics applications, with particular success in those which require visual
observations for control [1, 2, 3, 4]. Here, convolutional neural networks
(CNNs) enable learning of task-specific visual features directly from data,
avoiding the need for laborious task-specific engineering, and potentially
outperforming methods using hand-crafted visual representations. However, the
reliance of deep learning on large labelled datasets presents a significant
challenge.

One of the most promising solutions is sim-to-real transfer, where training is
performed in simulation, and a controller is transferred directly to the real
world. Of the many sim-to-real transfer methods for vision, domain
randomisation [5, 6, 3] is the most popular, since it is simple to implement,
and can achieve zero-shot transfer without any real-world data. However,
despite its popularity, there are no significant works which benchmark the
different types of domain randomisation for visual sim-to-real transfer.
Recent benchmarking work has studied sim-to-real for dynamics [7], but the
visual sim-to-real problem is a distinct problem and is typically treated
independently from dynamics.

Figure 1: An overall illustration of our experiments. Left: we study the
effect of the rendering quality on the sim-to-real transfer performance with
known environment settings. Right: we examine the impact of different
randomisation types on the transferability of models trained in simulation and
tested in challenging real-world scenarios.

In this paper, we study a range of different design choices and empirically
evaluate their effect on a real-world task. We divide the paper into two
distinct experiments, which study two modes of design choices. In the first
experiment, we consider the case when the scene content is known in advance,
and the primary role of sim-to-real is to model the effect of illumination and
image noise on the observed image. Here, we study how the quality of the
rendered images, in terms of the fidelity of graphics pipeline, affects the
sim-to-real performance. We also study the optimal trade-off between low-
quality and high-quality images, given a fixed amount of rendering time. In
the second experiment, we consider the case when the scene content is not
known in advance, and the role of sim-to-real is to achieve robustness to the
unknown, such as illumination conditions and clutter. Here, we study how
different types of randomisation, such as colours and textures, affect the
sim-to-real performance.

We evaluated both experiments with a 6D object pose estimation task, with a
manually-labelled real-world dataset. This not only evaluates sim-to-real for
a pose estimation module within a wider control pipeline, but also is a proxy
to end-to-end control methods [3] which implicitly localise important objects
within an image. Our results highlight the importance of high-quality images
for sim-to-real transfer, and the importance of randomising both distractors
and textures. Our supplementary video gives an overview of our experimental
setup, as well as visualisations of results111www.robot-
learning.uk/benchmarking-domain-randomisation.

##  II Related Work

Deep neural networks are known to be sample-inefficient, as a massive amount
of annotated images is required for the training process to succeed. Due to
the expensive data collection on real robots [8], researchers have shifted
towards using simulators to generate the training images. Nonetheless, models
trained with synthetic images only are not capable of directly transferring to
the real world [9], because of their various physical and visual differences,
a problem referred to as the reality gap. Sim-to-real transfer is an active
area of research to bridge the gap with two main approaches, Domain Adaptation
and Domain Randomisation. To close the reality gap, sim-to-real transfer
methods are extended to both visual [3, 5, 6] and dynamics [7, 10, 11, 12, 13]
solutions. The remainder of this section focuses on reviewing visual sim-to-
real approaches.

###  II-A Domain Adaptation

Domain adaptation is an approach to overcome the reality gap by learning
transferable representations across different domains, either by minimising a
distance measure between the two distributions [14, 15, 16] or by learning
pixel-level representations to make the simulated images as close as possible
to their real-world counterparts [17, 18]. While domain adaptation showed
promising results in the literature, incorporating real-world images is
necessary for successful transfers. In this work, we focus on the zero-shot
sim-to-real transfer solutions that do not require finetuning with real-world
images.

###  II-B Domain Randomisation

One of the most common forms of sim-to-real transfer is domain randomisation,
a simple-yet-promising technique to bridge the reality gap. Instead of
collecting the training data from a single simulated environment, the model is
exposed to a variety of random environments to enforce domain invariance. One
of the earliest successes of domain randomisation is the pioneering work
presented in [6], where the authors leveraged the technique to train for a
collision-free indoor flight. They showed that a policy trained solely in
simulation is capable of adapting to real-world scenarios if exposed to a
sufficient amount of randomness at training time. Several other works have
then extended on this success for a variety of robotics manipulation tasks [5,
3, 19, 20, 21, 22, 4, 23, 24].

In [3], the researchers investigated the use of domain randomisation with a
task that requires hand-eye coordination (robotic grasping task). Their end-
to-end approach succeeded in transferring to clear environments but failed to
generalise to more cluttered ones. In [9], the focus was on training object
detector using domain randomisation. Unlike other researches, they introduced
the use of a non-realistic noise in the randomised scenes. Although their
zero-shot model produced adequate results, they needed to incorporate real-
world data for better performance. The authors in [19] exploited the
randomisation of both physical and visual properties to train a dexterous
robotic hand to perform an in-hand manipulation, where they further improved
their work in [25] by proposing an iterative approach to learn the
randomisation parameters’ distributions, the so-called Automatic Domain
Randomisation [26, 27, 28].

The authors in [5] employed domain randomisation for the task of 2D object
detection. Their model achieved a pose estimation error of $1.5$ cm upon
tested in the real world. Their work, however, was tailored to regressing to
the object position relative to the world frame, but not its orientation,
which is crucial for the majority of robotics manipulation tasks.

In general, the main goal of domain randomisation is to find what type of
randomisation is effective in different scenarios. Both [3] and [5] found that
models are sensitive to the presence of distractors and textures in the
simulated scenes. Several other factors have also been investigated in the
literature, such as object textures [3, 5, 9, 29, 30], addition of distractors
[3, 5, 9, 29], lights properties [9, 29], target object shapes [31, 29, 30],
and sample size [5, 3, 30]. However, none of these works have benchmarked the
different choices that must be made to achieve the optimal transfer. In this
paper, we empirically evaluate two modes of design choices on a real-world 6D
pose estimation task, which serves as a reasonable proxy for evaluating end-
to-end control.

##  III Method

This work is composed of two distinct experiments that are tailored to
different objectives. In this section, we describe the setting shared between
both experiments.

###  III-A Problem Definition

The goal of the experiments is to train two models, $p_{O}(I_{C})$ and
$q_{O}(I_{C})$, to map an RGB image $I$ captured from a camera $C$ to the 3D
position $(x^{p}_{O},y^{p}_{O},z^{p}_{O})$ of the target object $O$ and its 4D
quaternion $(x^{q}_{O},y^{q}_{O},z^{q}_{O},w^{q}_{O})$ relative to the camera
frame. We chose to regress to quaternions due to their compact form, although
there are other possible representations for 3D rotations [32]. While we could
train a single network to regress to the full 6D pose of the target object, as
is often done in practice [33], we trained two separate networks to avoid the
laborious process of finding the optimal balancing between the position and
the orientation terms in the loss function, which would distract us from the
main point of the paper. The training images are rendered in simulation, and
depending on the experiment, they can be of different quality levels, where
distractors might also be present in the environment. At test time, the
trained model is used without any additional finetuning with real-world
images.

###  III-B Training (Synthetic) Data Collection

We used Unity3D [34] to render all of our training images. We chose to work
with Unity as it provides several render pipelines that are used to generate
images of different quality levels. Within each sample scene, we define a
plane upon which the target object is positioned. Further, we have a camera
that moves and rotates in all the three dimensions to capture the target
object in different poses, along with some directional lights to illuminate
the scene. Depending on the experiment, we randomise different parameters
while collecting the data to provide enough variability in the training
dataset. For all experiments, however, the following aspects are randomised
for each generated example:

  * •

The colour of the target object and its background

  * •

The position, orientation, field of view, and focal length of the camera

  * •

The number of lights, their positions, orientations, and specular
characteristics

Table I shows the full list of the randomisation parameters along with their
values. The additional experiment-specific parameters are discussed in their
respective sections separately.

TABLE I: Randomisation parameters values. [.5]() Parameter |  Range of values
---|---
Camera position |  Placed within a box of size ($70\times 40\times 50$) cm around the target object
Camera rotation |  |  $x$: [$None$\- $None$]
---
$y$: [ $None$, $None$]
$z$: [ $None$, $None$]
Camera field of view |  $\pm$ $None$ from the estimated real-world equivalent
Target object colour |  $\pm$ 10% from the estimated real-world equivalent
Number of lights |  [0,3] directional lights, depending on the quality level
Lights intensity |  | HDRP: [0, 2000] lux
---
Built-in renderer: [0,1]
Lights position |  Placed within a box of size ($6\times 2.8\times 6$) m around the target object
Lights rotation |  |  $x$: [$None$, $None$]
---
$y$: [$None$, $None$]
$z$: [$None$, $None$]

[4pt]Cube[4pt]Triangular Prism[4pt]Hexagon[4pt]Rectangular Prism

Figure 2: The four primitive shapes that we used throughout the experiments.

###  III-C Testing (Real-world) Data Collection

To address the research questions, we created mesh representations for four
primitive shapes, shown in Fig. 2, that we used to evaluate both experiments.
To find the accurate ground-truth values, we used ArUco [35], an accurate
marker-based pose estimation algorithm. For each test example, we placed a
marker on top of the target object and carefully validated, with an accurate
ruler, the offset from the object’s coordinate frame. The image is then
supplied to ArUco which outputs the 6D pose value of the object relative to
the calibrated camera. Another image is captured for the same scene without
the marker to be used as an input to the trained networks.

Figure 3: The end-to-end task of 6D pose estimation. For each target object
$O$, we train two separate networks, $p_{O}(I_{C})$ and $q_{O}(I_{C})$, that
map a single image to the 6D pose of $O$. Each input image is downsized to
($224\times 224$) and supplied to two separate networks to be mapped to a
tuple of $(x^{p}_{O},y^{p}_{O},z^{p}_{O})$ that represents the 3D position of
the target object and a tuple of $(x^{q}_{O},y^{q}_{O},z^{q}_{O},w^{q}_{O})$
which represents the target’s 3D orientation.

###  III-D Network Architecture

For all experiments, we used a modified version of the VGG-16 [36] network
architecture, which was proposed in [5]. As shown in Fig. 3, the network
consists of five groups of convolutional layers each with a kernel size of
$3\times 3$ and a stride of $1$, where dimensionality reduction is performed
after each group. To guarantee a fair comparison, we fixed the hyper-
parameters, the seed, and the loss function for all experiments. All models
are trained using Adam optimiser until convergence with a starting learning
rate of $1e-4$ and a scheduler to decrease it by a factor of $0.1$ every $30$
epochs. The networks weights are initialised randomly, and both the inputs and
outputs are normalised before starting the training process.

###  III-E Evaluation Metrics

To evaluate the performance of our models, we test them using the collected
real-world images and compare the actual labels to the results obtained from
the networks. The position error is computed using the Root Mean Squared Error
(or RMSE) formula. For the orientation, however, we use Equation 1 [37] to
find the angle required to rotate from one quaternion (the label, $q_{i}$) to
the other (the prediction, $\hat{q_{i}}$).

$\theta_{i}=2\times\cos^{-1}({\left|\langle\hat{q_{i}},q_{i}\rangle\right|})$
(1)

Where $\langle\hat{q_{i}},q_{i}\rangle$ is the inner product between the label
and the prediction of the $i^{th}$ sample image.

##  IV Experiment 1: Rendering Quality

This experiment aims at studying the impact of the simulator’s fidelity on the
overall model’s performance in the real world. More precisely, our focus is on
finding answers to the following three questions:

  1. 1.

How critical is the quality of the simulator for achieving successful sim-to-
real transfers?

  2. 2.

What is the effect that each simulation parameter has on the overall sim-to-
real transfer performance?

  3. 3.

What is the optimal trade-off of low-quality and high-quality images given a
fixed amount of rendering time?

TABLE II: An overall comparison between all the considered levels of
simulation.

Quality level Feature Directional Lights Shadows Anti-aliasing Dithering
effect Render Pipeline 1 No No No No Built-In 2 Yes No No No 3 Yes Low-quality
No No 4 Yes Medium-quality No No HDRP 5 Yes Medium-quality Low-quality No 6
Yes High-quality Low-quality No 7 Yes High-quality High-quality No 8 Yes High-
quality High-quality Yes

Figure 4: Sample images from each quality level. From left to right: low-
quality (level 1) to high-quality (level 8).

###  IV-A Experimental Setup

To achieve this, we use two renderers provided by Unity: the Built-in and the
High Definition Render Pipeline (HDRP hereafter) [38] to render images of
eight quality levels. As presented in Table II, these levels vary in terms of
the following aspects:

  * •

Directional lights: we enable them for all quality levels except the baseline
(level 1).

  * •

Shadows: we alter the quality of the shadows for each simulation level to
range from low to high quality.

  * •

Anti-aliasing: anti-aliasing is a technique used to overcome the staircase
effect that usually occurs in physical simulators by smoothing the object
boundaries and corners.

  * •

Dithering effect: dithering is a form of noise that is intentionally applied
to rendered images to eliminate unwanted graphical issues, such as colour
banding.

  * •

Render pipeline: the renderer used to generate the dataset.

In Fig. 4, we visually illustrate the differences between the eight simulation
quality levels. In this experiment, we assumed that the colours of both the
target object and its background are known at training time, and no
distractors are present in the environment. For each object, we collected
eight training datasets for the different quality levels we consider, each
with $30,000$ randomised synthetic images. We tested the models with $100$
real-world testing images, per object, that are collected with a variety of
poses and lighting conditions.

TABLE III: The average error across four target objects. Quality level |  $xy$-position (cm) |  $z$-position (cm) |  Orientation (degrees)
---|---|---|---
1 |  $4.37\pm 4.45$ |  $12.46\pm 8.06$ |  $13.0\pm 4.72$
2 |  $1.44\pm 1.68$ |  $3.44\pm 2.87$ |  $4.57\pm 3.38$
3 |  $1.40\pm 1.53$ |  $3.22\pm 3.05$ |  $4.14\pm 2.97$
4 |  $0.83\pm 1.10$ |  $2.73\pm 3.60$ |  $3.52\pm 2.26$
5 |  $0.70\pm 0.62$ |  $2.67\pm 2.22$ |  $3.34\pm 2.41$
6 |  $0.63\pm 0.63$ |  $2.50\pm 2.14$ |  $3.26\pm 2.40$
7 |  $0.61\pm 0.62$ |  $2.43\pm 2.07$ |  $3.20\pm 2.57$
8 |  $\mathbf{0.53\pm 0.52}$ |  $\mathbf{2.38\pm 1.93}$ |  $\mathbf{3.08\pm 2.15}$

###  IV-B Results

In Table III, we present the average error achieved by each quality level
across the four target objects. We reported the $xy$ and the $z$ position
errors separately to study the effect on the different tasks independently:
$xy$ position (across the image plane), $z$ position (perpendicular to the
image plane), and 3D orientation.

Altering the Rendering Quality We can conclude from Table III that, as we
increase the quality of the training data, the models performed consistently
better, which answers our first question.

Directional Lights Effect As we can deduce from the first two rows of Table
III, we were able to reduce the error by $\approx 68.4\%$ after enabling
directional lights, which proves their importance for achieving successful
transfer.

Shadows Effect Without shadows, as demonstrated in the second and third rows
of Table III, the models performed the worst when tested in the real world.
Moreover, as we increased the quality of the shadows (level 3 vs. level 4),
the performance of the models has significantly improved, which indicates the
importance of the quality of the shadows.

(a) $xy$-position (b) $z$-position (c) Orientation

Figure 5: The results obtained after training the models with combinations of
high-quality and low-quality training images, under three different data
collection time constraints.

Anti-aliasing Effect Recall from Table II that the only difference between
level 4 and level 5 images is the use of anti-aliasing. As we can conclude
from Table III, models trained with level 5 images succeeded in outperforming
the ones trained with no anti-aliasing enabled (i.e. level 4), which shows the
positive impact of anti-aliasing on the overall models’ performance. However,
since the anti-aliasing used with level 5 images is of low-quality, it might
fail, for some objects with complex shapes, to fix the staircase issues.

Dithering Effect The last two rows of Table III demonstrate the results before
and after enabling this effect, where we can see that dithering plays a vital
role in boosting the transfer performance.

###  IV-C Combining High-quality and Low-quality Images

In the previous set of experiments, we showed that models trained with high-
quality images have superior performance compared to the ones trained with
lower-quality levels. Nonetheless, rendering high-quality images comes with an
additional computational cost. We found with experiments that rendering one
image from level 1 takes an average of only $8.4$ ms, while generating one
high-quality image (level 8) requires more than triple the time, with an
average of $27.2$ ms. Rather than training with images of only one level of
quality, we could also train with both high-quality and low-quality images, to
combine the merits of both fast and accurate rendering.

Fig. 5 shows the results obtained after training several models under
different time constraints. For each experiment, we fixed the data collection
time and varied the percentage of each type of images. As we can deduce from
the graphs, the overall real-world test error dropped significantly after
incorporating as minimum as $25\%$ of high-quality images. Surprisingly,
however, as we increased the percentage of high-quality images, the
performance of the models exhibited no clear trend, as it sometimes
stabilised, reduced, or improved slightly. The overall conclusion is therefore
that if a significant number of images can be collected (i.e. no time
constraint), there is no harm from rendering all images with high quality.
Otherwise, if only a small number of images can be collected, including some
high-quality ones (e.g. 25%) should significantly improve the results.

##  V Experiment 2: Randomisation Type

Figure 6: Left: high-quality synthetic images collected with random colours,
textures and distractors. Right: real-world testing images labelled using
ArUco.

(a) Baseline
(colours only)

(b) Baseline
\+ ImageNet
background

(c) Baseline
\+ textures

(d) Baseline
\+ distractors

(e) Baseline
\+ textures
\+ distractors

Figure 7: A sample from each of the five datasets collected to study the
impact of the different randomisation settings on the sim-to-real transfer
performance. In these examples, the target object is the blue cube.

(a) $xy$-position (b) $z$-position (c) Orientation

Figure 8: The sensitivity of the models to distractors, textures, and number
of images. The results are averaged across the four target objects.

In this experiment, we examine the performance of models trained with high-
quality synthetic images, when tested in varied and cluttered real-world
environments. More precisely, we strive to assess the significance of
different randomisation settings to the models’ transferability to the real
world. Similar to the previous experiment, the focus here is on evaluating the
sim-to-real performance, rather than studying the different settings of 6D
pose estimation.

###  V-A Experimental Setup

Unlike the previous experiment, in this, we assumed that the models have no
prior knowledge about the background of the target object (i.e. it can be of
any texture or colour). In addition, distractors of any shape, colour, and
size are presented at test time.

A summary of the approach is provided in Fig. 6. The models were trained using
high-quality simulated images (left) and evaluated in the real-world (right).
Random geometrical shapes were used as distractors at training time. However,
a variety of never-seen distractors are presented in the real-world evaluation
images, e.g. a cup, book, and candle. To assess which types of randomisation
are most crucial for achieving successful transfer, we collected five training
datasets, shown in Fig. 7, which have the following specifications:

  1. 1.

Baseline: images with random solid colours in the background, but no textures
or distractors

  2. 2.

Baseline + random ImageNet backgrounds

  3. 3.

Baseline + random background textures

  4. 4.

Baseline + random distractors

  5. 5.

Baseline + random textures and distractors

The colour of the target object was uniformly sampled around our best estimate
of its colour in the real world. All the five datasets were collected with
high-quality anti-aliasing, high-quality shadows, and dithering effect. For
each target object, we collected $110$ real-world testing images with a
variety of backgrounds and distractors. We did not control for lighting and
shadows in the real world, meaning that some images were captured from
relatively dark scenes. For this experiment, we focused on answering the
following three questions:

(a)

(b)

(c)

Figure 9: Examples where the model succeeds in accurately estimating the 6D
pose of the target object (cube). The axes represent the cube’s local frame,
and their colours correspondences are: $x$: red, $y$: green, and $z$: blue.

(a)

(b)

(c)

Figure 10: Examples where the model fails to locate the target. The model
missed the object when a distractor of the same colour is presented in the
environment and with extremely cluttered or dark scenes. The axes are the
cube’s local frame ($x$: red, $y$: green, $z$: blue).

  1. 1.

What elements are most important to randomise while collecting the training
data?

  2. 2.

How does the performance of the models change as we increase the training
dataset size?

  3. 3.

How robust are the trained models to never-seen backgrounds and distractors?

###  V-B Results

Type of Randomisation We assessed the sensitivity of our models to both
textures and distractors. We found, as shown in Fig. 8, that the models are
sensitive to both factors for the three tasks. However, we noticed that for
position estimation, training with distractors in the environment (the green
line in Fig. 8) is of more importance than having textures, which is concluded
from the fact that models trained with textures and no distractors had larger
errors compared to the ones trained with distractors only. In general, both
the baseline and the ImageNet datasets failed with the worst performance
compared to the rest of the models. In contrast, the full dataset attained the
best performance in all cases.

Varying the Dataset Size To answer the second question, we trained several
models on different dataset sizes, ranging from $2^{7}$ to $2^{15}$ images.
The graphs in Fig. 8 show that as we increase the training dataset size, the
average real-world error gradually decreased for all the five datasets, as
would be expected.

Qualitative Results Overall, we observed that our models, when trained with
both textures and distractors, are robust to changing environments and can
generalise to cases with never-seen distractors and backgrounds. In Fig. 9, we
show some examples when the model precisely estimated the 6D pose of the
target object, in spite of the novel backgrounds and distractors. Despite the
promising results, the model was not able to accurately estimate the pose of
the target object, as shown in Fig. 10, when another distractor of the same
colour is presented in the environment. Further, extremely cluttered
environments and dark scenes posed a challenge to the model.

##  VI Conclusion

This paper investigated two different design choices in domain randomisation
for visual sim-to-real transfer: the quality of the renderer, and the type of
randomisation. For the first set of experiments, we found there to be a very
strong relationship between the quality of the renderer, and the sim-to-real
performance. We also proposed to combine low-quality images with high-quality
images, and found that for the same overall rendering time, it is more
important to have a high percentage of high-quality images, than low-quality
images. For the second set of experiments, we showed that randomising both
distractor objects and background textures are important for generalising to
novel environments. These conclusions can now be used by others in designing
their own domain randomisation datasets, with a view towards achieving optimal
sim-to-real performance for a given amount of dataset generation time.

## References

  * [1] S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” _The Journal of Machine Learning Research_ , 2016.
  * [2] E. Johns, “Coarse-to-fine imitation learning: Robot manipulation from a single demonstration,” in _International Conference on Robotics and Automation (ICRA)_ , 2021.
  * [3] S. James, A. J. Davison, and E. Johns, “Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task,” in _Conference on Robot Learning (CoRL)_ , 2017.
  * [4] K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke, “Using simulation and domain adaptation to improve efficiency of deep robotic grasping,” _CoRR_ , vol. abs/1709.07857, 2017. [Online]. Available: http://arxiv.org/abs/1709.07857
  * [5] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in _2017 IEEE/RSJ international conference on intelligent robots and systems (IROS)_. IEEE, 2017, pp. 23–30.
  * [6] F. Sadeghi and S. Levine, “Cad2rl: Real single-image flight without a single real image,” _arXiv preprint arXiv:1611.04201_ , 2016.
  * [7] E. Valassakis, Z. Ding, and E. Johns, “Crossing the gap: A deep dive into zero-shot sim-to-real transfer for dynamics,” in _International Conference on Intelligent Robots and Systems (IROS)_ , 2020.
  * [8] S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” _The International Journal of Robotics Research_ , vol. 37, no. 4-5, pp. 421–436, 2018.
  * [9] J. Tremblay, A. Prakash, D. Acuna, M. Brophy, V. Jampani, C. Anil, T. To, E. Cameracci, S. Boochoon, and S. Birchfield, “Training deep networks with synthetic data: Bridging the reality gap by domain randomization,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2018, pp. 969–977.
  * [10] Y.-Y. Tsai, H. Xu, Z. Ding, C. Zhang, E. Johns, and B. Huang, “Droid: Minimizing the reality gap using single-shot human demonstration,” _IEEE Robotics and Automation Letters_ , 2021.
  * [11] X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics randomization,” _CoRR_ , vol. abs/1710.06537, 2017. [Online]. Available: http://arxiv.org/abs/1710.06537
  * [12] A. Rajeswaran, S. Ghotra, S. Levine, and B. Ravindran, “Epopt: Learning robust neural network policies using model ensembles,” _CoRR_ , vol. abs/1610.01283, 2016. [Online]. Available: http://arxiv.org/abs/1610.01283
  * [13] W. Yu, C. K. Liu, and G. Turk, “Preparing for the unknown: Learning a universal policy with online system identification,” _CoRR_ , vol. abs/1702.02453, 2017. [Online]. Available: http://arxiv.org/abs/1702.02453
  * [14] M. Long, Y. Cao, J. Wang, and M. I. Jordan, “Learning transferable features with deep adaptation networks,” _arXiv preprint arXiv:1502.02791_ , 2015\.
  * [15] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarial discriminative domain adaptation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 7167–7176.
  * [16] B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in _European conference on computer vision_. Springer, 2016, pp. 443–450.
  * [17] K. Bousmalis, A. Irpan, P. Wohlhart, Y. Bai, M. Kelcey, M. Kalakrishnan, L. Downs, J. Ibarz, P. Pastor, K. Konolige, S. Levine, and V. Vanhoucke, “Using simulation and domain adaptation to improve efficiency of deep robotic grasping,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_ , 2018, pp. 4243–4250.
  * [18] S. James, P. Wohlhart, M. Kalakrishnan, D. Kalashnikov, A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, “Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 12 627–12 637.
  * [19] O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, _et al._ , “Learning dexterous in-hand manipulation,” _The International Journal of Robotics Research_ , vol. 39, no. 1, pp. 3–20, 2020.
  * [20] F. Sadeghi, A. Toshev, E. Jang, and S. Levine, “Sim2real view invariant visual servoing by recurrent control,” _arXiv preprint arXiv:1712.07642_ , 2017\.
  * [21] J. Matas, S. James, and A. J. Davison, “Sim-to-real reinforcement learning for deformable object manipulation,” _CoRR_ , vol. abs/1806.07851, 2018. [Online]. Available: http://arxiv.org/abs/1806.07851
  * [22] S. James, M. Bloesch, and A. J. Davison, “Task-embedded control networks for few-shot imitation learning,” in _Conference on Robot Learning_. PMLR, 2018, pp. 783–795.
  * [23] L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, “Asymmetric actor critic for image-based robot learning,” _CoRR_ , vol. abs/1710.06542, 2017. [Online]. Available: http://arxiv.org/abs/1710.06542
  * [24] Y. Chebotar, A. Handa, V. Makoviychuk, M. Macklin, J. Issac, N. D. Ratliff, and D. Fox, “Closing the sim-to-real loop: Adapting simulation randomization with real world experience,” _CoRR_ , vol. abs/1810.05687, 2018. [Online]. Available: http://arxiv.org/abs/1810.05687
  * [25] I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, _et al._ , “Solving rubik’s cube with a robot hand,” _arXiv preprint arXiv:1910.07113_ , 2019.
  * [26] C. Heindl, S. Zambal, and J. Scharinger, “Learning to predict robot keypoints using artificially generated images,” in _2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA)_ , 2019, pp. 1536–1539.
  * [27] N. Ruiz, S. Schulter, and M. Chandraker, “Learning to simulate,” _arXiv preprint arXiv:1810.02513_ , 2018.
  * [28] C. Heindl, L. Brunner, S. Zambal, and J. Scharinger, “Blendtorch: A real-time, adaptive domain randomization library,” _arXiv preprint arXiv:2010.11696_ , 2020.
  * [29] A. Loquercio, E. Kaufmann, R. Ranftl, A. Dosovitskiy, V. Koltun, and D. Scaramuzza, “Deep drone racing: From simulation to reality with domain randomization,” _IEEE Transactions on Robotics_ , vol. 36, no. 1, p. 1–14, Feb 2020. [Online]. Available: http://dx.doi.org/10.1109/TRO.2019.2942989
  * [30] A. Dehban, J. Borrego, R. Figueiredo, P. Moreno, A. Bernardino, and J. Santos-Victor, “The impact of domain randomization on object detection: A case study on parametric shapes and synthetic textures*,” in _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , 2019, pp. 2593–2600.
  * [31] J. Tobin, L. Biewald, R. Duan, M. Andrychowicz, A. Handa, V. Kumar, B. McGrew, J. Schneider, P. Welinder, W. Zaremba, and P. Abbeel, “Domain randomization and generative models for robotic grasping,” 2018.
  * [32] Y. Zhou, C. Barnes, J. Lu, J. Yang, and H. Li, “On the continuity of rotation representations in neural networks,” 2020.
  * [33] A. Kendall, M. Grimes, and R. Cipolla, “Convolutional networks for real-time 6-dof camera relocalization,” _CoRR_ , vol. abs/1505.07427, 2015. [Online]. Available: http://arxiv.org/abs/1505.07427
  * [34] U. Technologies, “Unity3d.” [Online]. Available: https://unity3d.com/
  * [35] F. J. Romero-Ramirez, R. Muñoz-Salinas, and R. Medina-Carnicer, “Speeded up detection of squared fiducial markers,” _Image and vision Computing_ , vol. 76, pp. 38–47, 2018.
  * [36] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2014.
  * [37] S. Mahendran, H. Ali, and R. Vidal, “3d pose regression using convolutional neural networks,” in _Proceedings of the IEEE International Conference on Computer Vision Workshops_ , 2017, pp. 2174–2182.
  * [38] Unity3D, “High definition render pipeline/built-in render pipeline comparison.” [Online]. Available<EMAIL_ADDRESS>