$\displaystyle\qquad{}=q^{2}(q-1)\big{(}1-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}x\biggl{(}\frac{\big{(}q^{\lambda+K+1}\xi/x,q^{K}\xi\alpha_{1},q^{K-1}\xi\alpha_{2},q^{K}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{K}\xi/x,q^{K}\xi\beta_{1},q^{K}\xi\beta_{2},q^{K}\xi\beta_{3};q\big{)}_{\infty}}$
$\displaystyle\qquad\hphantom{=q^{2}(q-1)\big{(}1-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}x\biggl{(}}-\frac{\big{(}q^{\lambda+L+2}\xi/x,q^{L+1}\xi\alpha_{1},q^{L}\xi\alpha_{2},q^{L+1}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{L+1}\xi/x,q^{L+1}\xi\beta_{1},q^{L+1}\xi\beta_{2},q^{L+1}\xi\beta_{3};q\big{)}_{\infty}}\biggr{)}.\\!\\!\\!\\!$
(B.4)

On the other hand, it follows from equations (B.1), (B.2) that the function
$\bar{g}_{1}^{\,[K,L]}(x)$ is written as

$\displaystyle\bar{g}_{1}^{\,[K,L]}(x)=\widehat{y}_{1}^{\,[K,L]}(x)-\widehat{y}_{3}^{\,[K,L]}(x)$
$\displaystyle\qquad{}=(1-q)\sum_{n=K}^{L}\frac{s(\alpha_{3}-\alpha_{1})}{(1-\alpha_{1}s)(1-\alpha_{3}s)}\frac{\big{(}q^{\lambda+1}s/x,\alpha_{1}s,\alpha_{2}s,\alpha_{3}s;q\big{)}_{\infty}}{\big{(}qs/x,\beta_{1}s,\beta_{2}s,\beta_{3}s;q\big{)}_{\infty}}\bigg{|}_{s=q^{n}\xi}$
$\displaystyle\qquad{}=(1-q)(\alpha_{3}-\alpha_{1})\sum_{n=K}^{L}c_{n},$
$\displaystyle
c_{n}=q^{n}\xi\frac{\big{(}q^{\lambda+n+1}\xi/x,q^{n+1}\xi\alpha_{1},q^{n}\xi\alpha_{2},q^{n+1}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{n+1}\xi/x,q^{n}\xi\beta_{1},q^{n}\xi\beta_{2},q^{n}\xi\beta_{3};q\big{)}_{\infty}}.$

Since $c_{n+1}/c_{n}\to q$, $n\to+\infty$, and $c_{-(n+1)}/c_{-n}\to
q^{\lambda+1}\alpha_{1}\alpha_{2}\alpha_{3}/(\beta_{1}\beta_{2}\beta_{3})=q$,
$n\to+\infty$, the function $\bar{g}_{1}^{\,[K,L]}(x)$ converges as
$K\to-\infty$ and $L\to+\infty$. Write

$\displaystyle\bar{g}_{1}(x)$
$\displaystyle=\lim_{K\to-\infty\atop{L\to+\infty}}\bar{g}_{1}^{\,[K,L]}(x)$
$\displaystyle=(1-q)(\alpha_{3}-\alpha_{1})\sum_{n=-\infty}^{+\infty}q^{n}\xi\frac{\big{(}q^{\lambda+n+1}\xi/x,q^{n+1}\xi\alpha_{1},q^{n}\xi\alpha_{2},q^{n+1}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{n+1}\xi/x,q^{n}\xi\beta_{1},q^{n}\xi\beta_{2},q^{n}\xi\beta_{3};q\big{)}_{\infty}}.$
(B.5)

Note that the functions $\widehat{y}_{1}^{\,[K,L]}(x)$ and
$\widehat{y}_{3}^{\,[K,L]}(x)$ do not converge as $K\to-\infty$.

We investigate the limit of the non-homogeneous term in equation (B.4) as
$K\to-\infty$ and $L\to+\infty$. We have

$\lim_{L\to+\infty}\frac{\big{(}q^{\lambda+L+2}\xi/x,q^{L+1}\xi\alpha_{1},q^{L}\xi\alpha_{2},q^{L+1}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{L+1}\xi/x,q^{L+1}\xi\beta_{1},q^{L+1}\xi\beta_{2},q^{L+1}\xi\beta_{3};q\big{)}_{\infty}}=1.$

It follows from the identity

$\displaystyle\frac{\big{(}q^{\lambda+K+1}\xi/x,q^{K}\xi\alpha_{1},q^{K-1}\xi\alpha_{2},q^{K}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{K}\xi/x,q^{K}\xi\beta_{1},q^{K}\xi\beta_{2},q^{K}\xi\beta_{3};q\big{)}_{\infty}}$
$\displaystyle\qquad=\frac{\vartheta_{q}\big{(}q^{\lambda+K+1}\xi/x\big{)}\vartheta_{q}\big{(}q^{K}\xi\alpha_{1}\big{)}\vartheta_{q}\big{(}q^{K-1}\xi\alpha_{2}\big{)}\vartheta_{q}\big{(}q^{K}\xi\alpha_{3}\big{)}}{\vartheta_{q}\big{(}q^{K}\xi/x\big{)}\vartheta_{q}\big{(}q^{K}\xi\beta_{1}\big{)}\vartheta_{q}\big{(}q^{K}\xi\beta_{2}\big{)}\vartheta_{q}\big{(}q^{K}\xi\beta_{3}\big{)}}$
$\displaystyle\qquad\phantom{=}\times\frac{\big{(}q^{1-K}x/\xi,q^{1-K}/(\xi\beta_{1}),q^{1-K}/(\xi\beta_{2}),q^{1-K}/(\xi\beta_{3});q\big{)}_{\infty}}{\big{(}q^{-\lambda-K}x/\xi,q^{1-K}/(\xi\alpha_{1}),q^{2-K}/(\xi\alpha_{2}),q^{1-K}/(\xi\alpha_{3});q\big{)}_{\infty}},$

which is obtained similarly to equation (4.10), and the condition
$q^{\lambda}=\beta_{1}\beta_{2}\beta_{3}/(\alpha_{1}\alpha_{2}\alpha_{3})$
that

$\displaystyle\lim_{K\to-\infty}$
$\displaystyle\frac{\big{(}q^{\lambda+K+1}\xi/x,q^{K}\xi\alpha_{1},q^{K-1}\xi\alpha_{2},q^{K}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{K}\xi/x,q^{K}\xi\beta_{1},q^{K}\xi\beta_{2},q^{K}\xi\beta_{3};q\big{)}_{\infty}}$
$\displaystyle=\frac{\vartheta_{q}\big{(}q^{\lambda+1}\xi/x\big{)}\vartheta_{q}(\xi\alpha_{1})\vartheta_{q}\big{(}q^{-1}\xi\alpha_{2}\big{)}\vartheta_{q}(\xi\alpha_{3})}{\vartheta_{q}(\xi/x)\vartheta_{q}(\xi\beta_{1})\vartheta_{q}(\xi\beta_{2})\vartheta_{q}(\xi\beta_{3})}.$

Therefore, equation (B.4) tends to

$q^{2}(q-1)\big{(}1-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}x\bigg{(}\frac{\vartheta_{q}\big{(}q^{\lambda+1}\xi/x\big{)}\vartheta_{q}(\xi\alpha_{1})\vartheta_{q}\big{(}q^{-1}\xi\alpha_{2}\big{)}\vartheta_{q}(\xi\alpha_{3})}{\vartheta_{q}(\xi/x)\vartheta_{q}(\xi\beta_{1})\vartheta_{q}(\xi\beta_{2})\vartheta_{q}(\xi\beta_{3})}-1\bigg{)}$

as $K\to-\infty$ and $L\to+\infty$. Hence we obtain the following proposition.

######  Proposition B.1.

The function $\bar{g}_{1}(x)$ in equation (B.5) satisfies

$\displaystyle\biggl{(}x-q\frac{\beta_{1}\beta_{2}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\biggl{(}x-q\frac{\beta_{2}\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\biggl{(}x-q\frac{\beta_{3}\beta_{1}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\bar{g}_{1}(x/q)$
$\displaystyle\qquad{}+q\biggl{(}x-\frac{1}{\alpha_{1}}\biggr{)}\biggl{(}x-\frac{q}{\alpha_{2}}\biggr{)}\biggl{(}x-\frac{1}{\alpha_{3}}\biggr{)}\bar{g}_{1}(qx)$
$\displaystyle\qquad{}+\biggl{\\{}-(1+q)x^{3}+\biggl{(}\frac{q}{\alpha_{1}}+\frac{q^{2}}{\alpha_{2}}+\frac{q}{\alpha_{3}}+q\frac{\beta_{1}\beta_{2}+\beta_{2}\beta_{3}+\beta_{3}\beta_{1}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}x^{2}$
$\displaystyle\qquad\hphantom{+\biggl{\\{}}-\biggl{(}q^{2}\frac{\beta_{1}+\beta_{2}+\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}+q^{2}\frac{\beta_{1}\beta_{2}\beta_{3}}{\alpha_{1}{\alpha_{2}}^{2}{\alpha_{3}}^{2}}+q^{2}\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}{\alpha_{2}}^{2}\alpha_{3}}+q\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}\alpha_{2}{\alpha_{3}}^{2}}\biggr{)}x$
$\displaystyle\qquad\hphantom{+\biggl{\\{}}+q^{2}(1+q)\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}{\alpha_{2}}^{2}{\alpha_{3}}^{2}}\biggr{\\}}\bar{g}_{1}(x)$
$\displaystyle\qquad{}+\\!q^{2}(q-\\!1)\big{(}1\\!-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}x\biggl{(}\frac{\vartheta_{q}(q^{\lambda+1}\xi/x)\vartheta_{q}(\xi\alpha_{1})\vartheta_{q}(q^{-1}\xi\alpha_{2})\vartheta_{q}(\xi\alpha_{3})}{\vartheta_{q}(\xi/x)\vartheta_{q}(\xi\beta_{1})\vartheta_{q}(\xi\beta_{2})\vartheta_{q}(\xi\beta_{3})}-1\\!\biggr{)}\\!=0,\\!\\!$
(B.6)

where
$q^{\lambda}=\beta_{1}\beta_{2}\beta_{3}/(\alpha_{1}\alpha_{2}\alpha_{3})$.

Equation (B.6) is a non-homogeneous extension of equation (4.42). We
investigate a relationship with the fourth order difference equation in
equation (4.40). By imposing the condition in equation (4.41), it follows that
the $q$-difference equation of $\bar{g}_{1}(x)$ in equation (4.40) is
factorized as

$\displaystyle\biggl{(}T_{x}-q^{2}\frac{\beta_{1}\beta_{2}\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}(T_{x}-q)\biggl{[}(q\alpha_{1}x-1)(\alpha_{2}x-1)(q\alpha_{3}x-1)T_{x}^{2}$
$\displaystyle\qquad-\biggl{\\{}q(q+1)\alpha_{1}\alpha_{2}\alpha_{3}x^{3}-q(\alpha_{1}\alpha_{2}+\alpha_{2}\alpha_{3}+q\alpha_{3}\alpha_{1}+\beta_{1}\beta_{2}+\beta_{2}\beta_{3}+\beta_{3}\beta_{1})x^{2}$
$\displaystyle\qquad\hphantom{-\biggl{\\{}}+q\biggl{(}\beta_{1}+\beta_{2}+\beta_{3}+\frac{\beta_{1}\beta_{2}\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggl{(}\alpha_{1}+\frac{\alpha_{2}}{q}+\alpha_{3}\biggr{)}\biggr{)}x-(q+1)\frac{\beta_{1}\beta_{2}\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{\\}}T_{x}$
$\displaystyle\qquad+\frac{q(\alpha_{1}\alpha_{2}\alpha_{3}x-\beta_{1}\beta_{2})(\alpha_{1}\alpha_{2}\alpha_{3}x-\beta_{2}\beta_{3})(\alpha_{1}\alpha_{2}\alpha_{3}x-\beta_{3}\beta_{1})}{\alpha_{1}^{2}\alpha_{2}^{2}\alpha_{3}^{2}}\biggr{]}\bar{g}_{1}(x)=0.$
(B.7)

Therefore, if $\bar{g}_{1}(x)$ satisfies equation (B.6), then it also
satisfies equation (4.40) with the condition in equation (4.41).

If $\xi=1/\alpha_{1}$, $\xi=1/\alpha_{2}$, $\xi=1/\alpha_{3}$ or
$\xi=q^{-\lambda}x$, then

$\displaystyle\frac{\big{(}q^{\lambda+K+1}\xi/x,q^{K}\xi\alpha_{1},q^{K-1}\xi\alpha_{2},q^{K}\xi\alpha_{3};q\big{)}_{\infty}}{\big{(}q^{K}\xi/x,q^{K}\xi\beta_{1},q^{K}\xi\beta_{2},q^{K}\xi\beta_{3};q\big{)}_{\infty}}=0$

for any negative integer $K$, and equation (B.4) tends to

$-q^{2}(q-1)\big{(}1-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}x$

as $K\to-\infty$ and $L\to+\infty$.

We substitute $\xi=1/\alpha_{1}$, $\xi=1/\alpha_{2}$, $\xi=1/\alpha_{3}$ or
$\xi=q^{-\lambda}x$ in $\bar{g}_{1}(x)$. If $\xi=1/\alpha_{1}$, then

$\displaystyle\bar{g}_{1}(x)$
$\displaystyle=(1-q)\frac{\alpha_{3}-\alpha_{1}}{\alpha_{1}}\frac{\big{(}q^{\lambda+1}/(\alpha_{1}x),\alpha_{2}/\alpha_{1},q\alpha_{3}/\alpha_{1},q;q\big{)}_{\infty}}{\big{(}q/(\alpha_{1}x),\beta_{1}/\alpha_{1},\beta_{2}/\alpha_{1},\beta_{3}/\alpha_{1};q\big{)}_{\infty}}$
$\displaystyle\phantom{=}\times{}_{4}\phi_{3}\biggl{(}\\!\\!\begin{array}[]{c}q/(\alpha_{1}x),\beta_{1}/\alpha_{1},\beta_{2}/\alpha_{1},\beta_{3}/\alpha_{1}\\\
q^{\lambda+1}/(\alpha_{1}x),\alpha_{2}/\alpha_{1},q\alpha_{3}/\alpha_{1}\end{array};q,q\biggr{)}.$
(B.10)

If $\xi=1/\alpha_{2}$, then

$\displaystyle\bar{g}_{1}(x)$
$\displaystyle=(1-q)q\frac{\alpha_{3}-\alpha_{1}}{\alpha_{2}}\frac{\big{(}q^{\lambda+2}/(\alpha_{2}x),q^{2}\alpha_{1}/\alpha_{2},q^{2}\alpha_{3}/\alpha_{2},q;q\big{)}_{\infty}}{\big{(}q^{2}/(\alpha_{2}x),q\beta_{1}/\alpha_{2},q\beta_{2}/\alpha_{2},q\beta_{3}/\alpha_{2};q\big{)}_{\infty}}$
$\displaystyle\phantom{=}\times{}_{4}\phi_{3}\biggl{(}\\!\\!\begin{array}[]{c}q^{2}/(\alpha_{2}x),q\beta_{1}/\alpha_{2},q\beta_{2}/\alpha_{2},q\beta_{3}/\alpha_{2}\\\
q^{\lambda+2}/(\alpha_{2}x),q^{2}\alpha_{1}/\alpha_{2},q^{2}\alpha_{3}/\alpha_{2}\end{array};q,q\biggr{)}.$
(B.13)

If $\xi=1/\alpha_{3}$, then

$\displaystyle\bar{g}_{1}(x)$
$\displaystyle=(1-q)\frac{\alpha_{3}-\alpha_{1}}{\alpha_{3}}\frac{\big{(}q^{\lambda+1}/(\alpha_{3}x),q\alpha_{1}/\alpha_{3},\alpha_{2}/\alpha_{3},q;q\big{)}_{\infty}}{\big{(}q/(\alpha_{3}x),\beta_{1}/\alpha_{3},\beta_{2}/\alpha_{3},\beta_{3}/\alpha_{3};q\big{)}_{\infty}}$
$\displaystyle\phantom{=}\times{}_{4}\phi_{3}\biggl{(}\\!\\!\begin{array}[]{c}q/(\alpha_{3}x),\beta_{1}/\alpha_{3},\beta_{2}/\alpha_{3},\beta_{3}/\alpha_{3}\\\
q^{\lambda+1}/(\alpha_{3}x),q\alpha_{1}/\alpha_{3},\alpha_{2}/\alpha_{3}\end{array};q,q\biggr{)}.$
(B.16)

If $\xi=q^{-\lambda}x$, then

$\displaystyle\bar{g}_{1}(x)$
$\displaystyle=(1-q)q^{-\lambda}(\alpha_{3}-\alpha_{1})x\frac{\big{(}q^{-\lambda+1}\alpha_{1}x,q^{-\lambda}\alpha_{2}x,q^{-\lambda+1}\alpha_{3}x,q;q\big{)}_{\infty}}{\big{(}q^{-\lambda}\beta_{1}x,q^{-\lambda}\beta_{2}x,q^{-\lambda}\beta_{3}x,q^{-\lambda+1};q\big{)}_{\infty}}$
$\displaystyle\phantom{=}\times{}_{4}\phi_{3}\biggl{(}\\!\\!\begin{array}[]{c}q^{-\lambda}\beta_{1}x,q^{-\lambda}\beta_{2}x,q^{-\lambda}\beta_{3}x,q^{-\lambda+1}\\\
q^{-\lambda+1}\alpha_{1}x,q^{-\lambda}\alpha_{2}x,q^{-\lambda+1}\alpha_{3}x\end{array};q,q\biggr{)}.$
(B.19)

######  Proposition B.2.

The functions in equations (B.10), (B.13), (B.16), (B.19) satisfy

$\displaystyle\biggl{(}x-q\frac{\beta_{1}\beta_{2}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\biggl{(}x-q\frac{\beta_{2}\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\biggl{(}x-q\frac{\beta_{3}\beta_{1}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\bar{g}_{1}(x/q)$
$\displaystyle\qquad+q\biggl{(}x-\frac{1}{\alpha_{1}}\biggr{)}\biggl{(}x-\frac{q}{\alpha_{2}}\biggr{)}\biggl{(}x-\frac{1}{\alpha_{3}}\biggr{)}\bar{g}_{1}(qx)$
$\displaystyle\qquad+\biggl{\\{}-(1+q)x^{3}+\biggl{(}\frac{q}{\alpha_{1}}+\frac{q^{2}}{\alpha_{2}}+\frac{q}{\alpha_{3}}+q\frac{\beta_{1}\beta_{2}+\beta_{2}\beta_{3}+\beta_{3}\beta_{1}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}x^{2}$
$\displaystyle\qquad\hphantom{+\biggl{\\{}}-\biggl{(}q^{2}\frac{\beta_{1}+\beta_{2}+\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}+q^{2}\frac{\beta_{1}\beta_{2}\beta_{3}}{\alpha_{1}{\alpha_{2}}^{2}{\alpha_{3}}^{2}}+q^{2}\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}{\alpha_{2}}^{2}\alpha_{3}}+q\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}\alpha_{2}{\alpha_{3}}^{2}}\biggr{)}x$
$\displaystyle\qquad\hphantom{+\biggl{\\{}}+q^{2}(1+q)\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}{\alpha_{2}}^{2}{\alpha_{3}}^{2}}\biggr{\\}}\bar{g}_{1}(x)-q^{2}(q-1)\big{(}1-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}x=0.$

To obtain results corresponding to the specialization $\xi=1/\beta_{1}$,
$\xi=1/\beta_{2}$, $\xi=1/\beta_{3}$ and $\xi=x$, we replace the functions
with

$\displaystyle
P_{\lambda}(x,s)=(x/s)^{\lambda}\frac{(x/s;q)_{\infty}}{(q^{-\lambda}x/s;q)_{\infty}},\qquad
y(x)=x^{\mu^{\prime}}\frac{(q/(\beta_{1}x),q/(\beta_{2}x),q/(\beta_{3}x);q)_{\infty}}{(q/(\alpha_{1}x),q/(\alpha_{2}x),q/(\alpha_{3}x);q)_{\infty}}$

with the condition
$q^{\mu^{\prime}}\alpha_{1}\alpha_{2}\alpha_{3}/(\beta_{1}\beta_{2}\beta_{3})=1$,
i.e., $q^{\mu^{\prime}}=q^{\lambda}$. Then the function $y(x)$ also satisfies
the $q$-difference equation $y(qx)=B(x)y(x)$, where $B(x)$ is given as
equation (4.36) with the condition $\mu=0$. The function
$\bar{g}_{1}^{\,[K,L]}(x)$ is written as

$\displaystyle\bar{g}_{1}^{\,[K,L]}(x)$
$\displaystyle\\!=\\!(1\\!-\\!q)\frac{\alpha_{3}\\!-\\!\alpha_{1}}{\alpha_{1}\alpha_{3}}x^{\lambda}\\!\\!\sum_{n=K}^{L}\\!(q^{n}\xi)^{-1}\frac{\big{(}xq^{-n}/\xi,q^{1-n}/(\beta_{1}\xi),q^{1-n}/(\beta_{2}\xi),q^{1-n}/(\beta_{3}\xi);q\big{)}_{\infty}}{\big{(}xq^{-n-\lambda}/\xi,q^{-n}/(\alpha_{1}\xi),q^{1-n}/(\alpha_{2}\xi),q^{-n}/(\alpha_{3}\xi\big{)};q)_{\infty}},$

and it converges as $K\to-\infty$ and $L\to+\infty$. Write the limit by
$\bar{g}_{1}(x)$. Note that the function $\bar{g}_{1}^{\,[K,L]}(x)$ satisfies
equation (B.3) where

$\displaystyle Q^{[K,L]}(x)$
$\displaystyle=P_{\lambda}\big{(}x,q^{K-1}\xi\big{)}y\big{(}q^{K}\xi\big{)}-P_{\lambda}\big{(}x,q^{L}\xi\big{)}y\big{(}q^{L+1}\xi\big{)}$
$\displaystyle=(qx)^{\lambda}\biggl{\\{}\frac{\big{(}xq^{1-K}/\xi,q^{1-K}/(\beta_{1}\xi),q^{1-K}/(\beta_{2}\xi),q^{1-K}/(\beta_{3}\xi);q\big{)}_{\infty}}{\big{(}xq^{-\lambda+1-K}/\xi,q^{1-K}/(\alpha_{1}\xi),q^{1-K}/(\alpha_{2}\xi),q^{1-K}/(\alpha_{3}\xi);q\big{)}_{\infty}}$
$\displaystyle\qquad\qquad-\frac{\big{(}xq^{-L}/\xi,q^{-L}/(\beta_{1}\xi),q^{-L}/(\beta_{2}\xi),q^{-L}/(\beta_{3}\xi);q\big{)}_{\infty}}{\big{(}xq^{-\lambda-L}/\xi,q^{-L}/(\alpha_{1}\xi),q^{-L}/(\alpha_{2}\xi),q^{-L}/(\alpha_{3}\xi\big{)};q)_{\infty}}\biggr{\\}}.$

The non-homogeneous term in equation (B.3) is written as

$\displaystyle
q(q-1)\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{3}}x\biggl{\\{}\biggl{(}x-q\frac{\beta_{1}\beta_{2}\beta_{3}}{\alpha_{1}{\alpha_{2}}^{2}\alpha_{3}}\biggr{)}Q^{[K,L]}(x/q)-\biggl{(}x-\frac{q}{\alpha_{2}}\biggr{)}Q^{[K,L]}(x)\biggr{\\}}$
$\displaystyle\qquad=q(q-1)(1-q^{\lambda})\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{3}}x^{\lambda+2}$
$\displaystyle\qquad\hphantom{=}\times\biggl{(}\frac{\big{(}xq^{1-K}/\xi,q^{1-K}/(\beta_{1}\xi),q^{1-K}/(\beta_{2}\xi),q^{1-K}/(\beta_{3}\xi);q\big{)}_{\infty}}{\big{(}xq^{-\lambda-K}/\xi,q^{1-K}/(\alpha_{1}\xi),q^{2-K}/(\alpha_{2}\xi),q^{1-K}/(\alpha_{3}\xi\big{)};q)_{\infty}}$
$\displaystyle\qquad\hphantom{=\times\biggl{(}}-\frac{\big{(}xq^{-L}/\xi,q^{-L}/(\beta_{1}\xi),q^{-L}/(\beta_{2}\xi),q^{-L}/(\beta_{3}\xi);q\big{)}_{\infty}}{\big{(}xq^{-\lambda-1-L}/\xi,q^{-L}/(\alpha_{1}\xi),q^{1-L}/(\alpha_{2}\xi),q^{-L}/(\alpha_{3}\xi);q\big{)}_{\infty}}\biggr{)}.$
(B.20)

We investigate the limit of equation (B.20) as $K\to-\infty$ and
$L\to+\infty$. We have

$\displaystyle\lim_{K\to-\infty}\frac{\big{(}xq^{1-K}/\xi,q^{1-K}/(\beta_{1}\xi),q^{1-K}/(\beta_{2}\xi),q^{1-K}/(\beta_{3}\xi);q\big{)}_{\infty}}{\big{(}xq^{-\lambda-K}/\xi,q^{1-K}/(\alpha_{1}\xi),q^{2-K}/(\alpha_{2}\xi),q^{1-K}/(\alpha_{3}\xi);q\big{)}_{\infty}}=1.$

If $\xi=1/\beta_{1}$, $\xi=1/\beta_{2}$, $\xi=1/\beta_{3}$ or $\xi=x$, then

$\displaystyle\frac{\big{(}xq^{-L}/\xi,q^{-L}/(\beta_{1}\xi),q^{-L}/(\beta_{2}\xi),q^{-L}/(\beta_{3}\xi);q\big{)}_{\infty}}{\big{(}xq^{-\lambda-1-L}/\xi,q^{-L}/(\alpha_{1}\xi),q^{1-L}/(\alpha_{2}\xi),q^{-L}/(\alpha_{3}\xi);q\big{)}_{\infty}}=0$

for any positive integer $L$, and equation (B.20) tends to

$\displaystyle
q(q-1)\big{(}1-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{3}}x^{\lambda+2}$

as $K\to-\infty$ and $L\to+\infty$.

If $\xi=1/\beta_{1}$, then

$\displaystyle\bar{g}_{1}(x)$
$\displaystyle=(1-q)\frac{\alpha_{3}-\alpha_{1}}{\alpha_{1}\alpha_{3}}\beta_{1}x^{\lambda}\frac{(\beta_{1}x,q\beta_{1}/\beta_{2},q\beta_{1}/\beta_{3},q;q)_{\infty}}{\big{(}q^{-\lambda}\beta_{1}x,\beta_{1}/\alpha_{1},q\beta_{1}/\alpha_{2},\beta_{1}/\alpha_{3};q\big{)}_{\infty}}$
$\displaystyle\phantom{=}\times{}_{4}\phi_{3}\biggl{(}\\!\\!\begin{array}[]{c}q^{-\lambda}\beta_{1}x,\beta_{1}/\alpha_{1},q\beta_{1}/\alpha_{2},\beta_{1}/\alpha_{3}\\\
\beta_{1}x,q\beta_{1}/\beta_{2},q\beta_{1}/\beta_{3}\end{array}\\!;q,q\biggr{)}.$
(B.23)

The case $\xi=1/\beta_{j}$, $j=2,3$, is obtained from the case
$\xi=1/\beta_{1}$ by replacing $\beta_{1}$ with $\beta_{j}$. If $\xi=x$, then

$\displaystyle\bar{g}_{1}(x)$
$\displaystyle=(1-q)q\frac{\alpha_{3}-\alpha_{1}}{\alpha_{1}\alpha_{3}}x^{\lambda-1}\frac{\big{(}q^{2}/(\beta_{1}x),q^{2}/(\beta_{2}x),q^{2}/(\beta_{3}x),q;q\big{)}_{\infty}}{\big{(}q/(\alpha_{1}x),q^{2}/(\alpha_{2}x),q/(\alpha_{3}x),q^{1-\lambda};q\big{)}_{\infty}}$
$\displaystyle\phantom{=}\times{}_{4}\phi_{3}\biggl{(}\\!\\!\begin{array}[]{c}q/(\alpha_{1}x),q^{2}/(\alpha_{2}x),q/(\alpha_{3}x),q^{1-\lambda}\\\
q^{2}/(\beta_{1}x),q^{2}/(\beta_{2}x),q^{2}/(\beta_{3}x)\end{array}\\!;q,q\biggr{)}.$
(B.26)

######  Proposition B.3.

The functions in equations (B.23), (B.26) satisfy

$\displaystyle\biggl{(}x-q\frac{\beta_{1}\beta_{2}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\biggl{(}x-q\frac{\beta_{2}\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\biggl{(}x-q\frac{\beta_{3}\beta_{1}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}\bar{g}_{1}(x/q)$
$\displaystyle\qquad+q\biggl{(}x-\frac{1}{\alpha_{1}}\biggr{)}\biggl{(}x-\frac{q}{\alpha_{2}}\biggr{)}\biggl{(}x-\frac{1}{\alpha_{3}}\biggr{)}\bar{g}_{1}(qx)$
$\displaystyle\qquad+\biggl{\\{}-(1+q)x^{3}+\biggl{(}\frac{q}{\alpha_{1}}+\frac{q^{2}}{\alpha_{2}}+\frac{q}{\alpha_{3}}+q\frac{\beta_{1}\beta_{2}+\beta_{2}\beta_{3}+\beta_{3}\beta_{1}}{\alpha_{1}\alpha_{2}\alpha_{3}}\biggr{)}x^{2}$
$\displaystyle\qquad\hphantom{+\biggl{\\{}}-\biggl{(}q^{2}\frac{\beta_{1}+\beta_{2}+\beta_{3}}{\alpha_{1}\alpha_{2}\alpha_{3}}+q^{2}\frac{\beta_{1}\beta_{2}\beta_{3}}{\alpha_{1}{\alpha_{2}}^{2}{\alpha_{3}}^{2}}+q^{2}\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}{\alpha_{2}}^{2}\alpha_{3}}+q\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}\alpha_{2}{\alpha_{3}}^{2}}\biggr{)}x$
$\displaystyle\qquad\hphantom{+\biggl{\\{}}+q^{2}(1+q)\frac{\beta_{1}\beta_{2}\beta_{3}}{{\alpha_{1}}^{2}{\alpha_{2}}^{2}{\alpha_{3}}^{2}}\biggr{\\}}\bar{g}_{1}(x)+q(q-1)\big{(}1-q^{\lambda}\big{)}\frac{\alpha_{1}-\alpha_{3}}{\alpha_{1}\alpha_{3}}x^{\lambda+2}=0.$
(B.27)

It follows from equation (B.7) that, if $\bar{g}_{1}(x)$ satisfies equation
(B.27), then it also satisfies equation (4.40) with the condition in equation
(4.41).

We replace the parameters by using Proposition 4.8 to fit results on
$q$-integrals with the variant of $q$-hypergeometric equation of degree three
given in equation (4.43). We now replace the parameters in Proposition B.1 by
using Proposition 4.8. Set $g^{\langle
1\rangle}(x)=x^{-\alpha}\bar{g}_{1}(x)/\\{(1-q)(\alpha_{3}-\alpha_{1})\\}$.
Then equation (B.6) is replaced by

$\displaystyle\big{(}x-q^{h_{1}+1/2}t_{1}\big{)}\big{(}x-q^{h_{2}+1/2}t_{2}\big{)}\big{(}x-q^{h_{3}+1/2}t_{3}\big{)}g^{\langle
1\rangle}(x/q)$
$\displaystyle\qquad+q^{2\alpha+1}\big{(}x-q^{l_{1}-1/2}t_{1}\big{)}\big{(}x-q^{l_{2}-1/2}t_{2}\big{)}\big{(}x-q^{l_{3}-1/2}t_{3}\big{)}g^{\langle
1\rangle}(qx)$
$\displaystyle\qquad+q^{\alpha}\bigl{[}-(q+1)x^{3}+q^{1/2}\big{\\{}\big{(}q^{h_{1}}+q^{l_{1}}\big{)}t_{1}+\big{(}q^{h_{2}}+q^{l_{2}}\big{)}t_{2}+\big{(}q^{h_{3}}+q^{l_{3}}\big{)}t_{3}\big{\\}}x^{2}$
$\displaystyle\qquad\hphantom{+q^{\alpha}\bigl{[}}-q^{(h_{1}+h_{2}+h_{3}+l_{1}+l_{2}+l_{3}+1)/2}\big{\\{}\big{(}q^{-h_{1}}+q^{-l_{1}}\big{)}t_{2}t_{3}+\big{(}q^{-h_{2}}+q^{-l_{2}}\big{)}t_{1}t_{3}$
$\displaystyle\qquad\hphantom{+q^{\alpha}\bigl{[}}+\big{(}q^{-h_{3}}+q^{-l_{3}}\big{)}t_{1}t_{2}\big{\\}}x+q^{(h_{1}+h_{2}+h_{3}+l_{1}+l_{2}+l_{3})/2}(q+1)t_{1}t_{2}t_{3}\bigr{]}g^{\langle
1\rangle}(x)$
$\displaystyle\qquad+q^{\alpha+l_{1}+l_{2}+l_{3}-1/2}t_{1}t_{2}t_{3}\big{(}1-q^{\lambda}\big{)}x^{1-\alpha}$
$\displaystyle\qquad\hphantom{+}\times\\!\biggl{(}\frac{\vartheta_{q}\big{(}q^{\lambda+1}\xi/x\big{)}\vartheta_{q}\big{(}\xi
q^{-l_{1}+1/2}/t_{1}\big{)}\vartheta_{q}\big{(}\xi
q^{-l_{2}+1/2}/t_{2}\big{)}\vartheta_{q}\big{(}\xi
q^{-l_{3}+1/2}/t_{3}\big{)}}{\vartheta_{q}(\xi/x)\vartheta_{q}\big{(}\xi
q^{\lambda-h_{1}+1/2}/t_{1}\big{)}\vartheta_{q}\big{(}\xi q^{\lambda-
h_{2}+1/2}/t_{2}\big{)}\vartheta_{q}\big{(}\xi q^{\lambda-
h_{3}+1/2}/t_{3}\big{)}}-1\\!\biggr{)}=0,\\!\\!\\!\\!$ (B.28)

where $\lambda=(h_{1}+h_{2}+h_{3}-l_{1}-l_{2}-l_{3}+1)/2$. It is a non-
homogeneous version of the variant of $q$-hypergeometric equation of degree
three. Equation (B.5) is replaced with

$\displaystyle g^{\langle
1\rangle}(x)=x^{-\alpha}\sum_{n=-\infty}^{+\infty}\\!\\!q^{n}\xi\frac{\big{(}\xi
q^{\lambda+n+1}/x,\xi q^{n-l_{1}+3/2}/t_{1},\xi q^{n-l_{2}+3/2}/t_{2},\xi
q^{n-l_{3}+3/2}/t_{3};q\big{)}_{\infty}}{\big{(}\xi q^{n+1}/x,\xi
q^{n+\lambda-h_{1}+1/2}/t_{1},\xi q^{n+\lambda-h_{2}+1/2}/t_{2},\xi
q^{n+\lambda-h_{3}+1/2}/t_{3};q\big{)}_{\infty}},$

where $\lambda=(h_{1}+h_{2}+h_{3}-l_{1}-l_{2}-l_{3}+1)/2$. Then it follows
from Proposition B.1 that the function $g^{\langle 1\rangle}(x)$ in the above
equation is a solution to equation (B.28). By rewriting Proposition B.2, we
obtain Theorem 4.9.

To show Theorem 4.10, we replace the parameters in Proposition B.3 by using
Proposition 4.8. Set $g^{\langle
2\rangle}(x)=x^{-\alpha}\bar{g}_{1}(x)\alpha_{1}\alpha_{3}/\\{q(1-q)(\alpha_{3}-\alpha_{1})\\}$.
Then we obtain Theorem 4.10 by rewriting equations (B.23), (B.26).

### Acknowledgements

The authors are grateful to the referees for the valuable comments. The second
author was supported by JSPS KAKENHI Grant Number JP22K03368.

## References

  * [1] Aomoto K., On the 3 fundamental problems concerning $q$-basic hypergeometric functions ($q$-difference equations, asymptotic behaviours and connection problem), in Proceedinds of Fifth Oka Symposium (March 18–19, 2006, Nara), Oka Mathematical Institute, Japan, 2006, 16 pages.
  * [2] Arai Y., $q$-deformation of middle convolution and its application to $q$-difference equations, Master’s Thesis, Ochanomizu University, 2023.
  * [3] Dettweiler M., Reiter S., An algorithm of Katz and its application to the inverse Galois problem, J. Symbolic Comput. 30 (2000), 761–798.
  * [4] Dettweiler M., Reiter S., Middle convolution of Fuchsian systems and the construction of rigid differential systems, J. Algebra 318 (2007), 1–24.
  * [5] Fujii T., Special solutions of $q$-Heun equation by $q$-hypergeometric integral, Master’s Thesis, Kobe University, 2022.
  * [6] Fujii T., Nobukawa T., Hypergeometric solutions for variants of the $q$-hypergeometric equation, arXiv:2207.12777.
  * [7] Gasper G., Rahman M., Basic hypergeometric series, 2nd ed., Encyclopedia Math. Appl., Vol. 96, Cambridge University Press, Cambridge, 2004.
  * [8] Haraoka Y., Linear differential equations in the complex domain, Lecture Notes in Math., Vol. 2271, Springer, Cham, 2020.
  * [9] Hatano N., Matsunawa R., Sato T., Takemura K., Variants of $q$-hypergeometric equation, Funkcial. Ekvac. 65 (2022), 159–190, arXiv:1910.12560.
  * [10] Katz N.M., Rigid local systems, Ann. of Math. Stud., Vol. 139, Princeton University Press, Princeton, NJ, 1996.
  * [11] Matsunawa R., Sato T., Takemura K., Variants of confluent $q$-hypergeometric equations, in Geometric and Harmonic Analysis on Homogeneous Spaces and Applications, Springer Proc. Math. Stat., Vol. 366, Springer, Cham, 2021, 161–180, arXiv:2005.13223.
  * [12] Mimachi K., Connection problem in holonomic $q$-difference system associated with a Jackson integral of Jordan–Pochhammer type, Nagoya Math. J. 116 (1989), 149–161.
  * [13] Sakai H., Yamaguchi M., Spectral types of linear $q$-difference equations and $q$-analog of middle convolution, Int. Math. Res. Not. 2017 (2017), 1975–2013, arXiv:1410.3674.
  * [14] Sasaki S., Takagi S., Takemura K., $q$-middle convolution and $q$-Painlevé equation, SIGMA 18 (2022), 056, 21 pages, arXiv:2201.03960.
  * [15] Whittaker E., Watson G., A course of modern analysis, 4th ed., Cambridge Math. Lib., Cambridge University Press, New York, 1962.


# DFTWS for blockchain: Deterministic, Fair and Transparent Winner Selection

Felix Hoffmann<EMAIL_ADDRESS>0000-0001-7272-8226 and
Udo Kebschull<EMAIL_ADDRESS>Goethe University
FrankfurtTheodor-W.-Adorno-Platz 6Frankfurt am MainHessenGermany60323

(2023)

###### Abstract.

This publication describes the block winner selection process that will be
used in a novel Proof-of-Useful-Work blockchain for High Energy Physics.
Instead of spamming hashing operations to mine blocks, miners will be running
Monte Carlo simulations to support a real-world HEP experiment with useful
data. The block problems will be defined by a Root Authority which is
represented by a HEP experiment like CBM. The focus in this publication is a
mechanism that allows the Root Authority to select a winner from a list of
nodes that solved a block problem. The mechanism is designed so that winner
selection is deterministic, fair and transparent. This mechanism allows every
node to verify the fairness of the winner selection process without giving the
nodes a tool to be able to improve their own winning chances.

blockchain, winner selection, proof-of-useful-work, distributed computing,
monte carlo, simulation, HEP

††copyright: acmcopyright††journalyear: 2023††doi: XXXXXXX.XXXXXXX††journal:
DLT††journalvolume: 1††journalnumber: 1††article: 1††publicationmonth: 4

##  1\. Introduction

The authors of this publication are working on a Proof-of-Useful-Work
blockchain for High Energy Physics, in which new blocks are created by running
Monte Carlo simulations that support a real-world HEP experiment while at the
same time securing the underlying blockchain instead of wasting energy by
spamming hashing operations as common in traditional Proof-of-Work that still
is being used in e.g. Bitcoin (bitcoin). In order to generate useful data,
there must be an entity that can make use of the resulting data. As a result,
HEP experiment insiders (who are able to determine which kind of Monte Carlo
simulations are needed) act as a Root Authority that defines problems that
miners need to solve. This publication focuses on the winner selection process
of such as blockchain.

Each node generates an Ed25519 (ed25519) private key of length 32 bytes. Then
the corresponding public key of length 32 is derived. The public key is then
used to derive a libp2p node ID that identifies the node. Messages sent
between nodes are signed using Ed25519 signatures on Curve25519.

In order to create a new block, the Root Authority (e.g. a member of the CBM
collaboration (cbm)) defines the parameters for a Monte Carlo simulation that
must be run in a provided HEP environment that every miner runs locally. The
RNG of the Monte Carlo simulations is derived block the previous block hash
and is part of the problem definition. As a result, the resulting ROOT (common
file format in HEP) files are identical for each node that runs the same
simulation (reproducibility). This enables a probabilistic solution
verification mechanism that is not the focus of this paper. There will never
be the exact same (sub) problems chosen for two different blocks, because the
results would be identical, and it would give an advantage to miners who
participated in the blockchain longer than others and have a memory of past
solutions.
Each problem must be solved within a defined interval of time (part of the
problem definition). If a node has solved the block problem, it uploads the
resulting data to RA servers (only RA has read access here) and publicly
broadcasts a zero knowledge proof (Keccak-256 (keccak) of solution hash) that
retrospectively shows that this node was able to solve the problem in time
without revealing the solution hash itself. This enables the following
mechanism:

  * •

A node can prove that it solved the problem in time. Otherwise, it would not
have been able to determine the correct solution hash, hash it again and then
publicly broadcast publish signed solution hash and hashed solution hash.

  * •

Nodes can not steal the solutions of other nodes because they can see
Keccak-256(Keccak-256(solution data)) of other nodes, but they need to sign
Keccak-256(solution data) which they can not derive from
Keccak-256(Keccak-256(solution data)).

  * •

When the RA has chosen a winner, a list of all nodes that solved the problem
and the correct solution hash are also included in the block. This allows each
node to verify that the message the other nodes signed was in fact
Keccak-256(solution data). As a result, nodes can control each other (correct
solution was published) and nodes can control the RA: The list of nodes that
solved the problem can be verified because with the knowledge of the correct
solution hash it can be verified that the message the other nodes signed and
broadcast in time was the hash of the correct solution. If a node signed the
correct solution but is not included in the winner list, then each node can
instantly detect that the RA can not be trusted. Furthermore, nodes are able
to determine that the most commonly broadcast solution hash has been chosen as
correct solution by the RA (this is useful in combination with the
probabilistic solution determination algorithm which is not the focus of this
paper).

It should be noted that from the list of nodes that published a correct
solution only one winner is chosen. This is done to prevent token inflation
(winner gets a block reward token that is halved in regular block intervals)
and to prevent nodes from performing a Sybil attack to calculate the solution
once and then collect the reward for multiple identities they control.

##  2\. Winner selection

The question that will be answered in this section is: How can one winner be
chosen from the list in a fair and transparent way?

###  2.1. Procedure

Before problem is broadcast:

  1. (1)

Root authority (RA) generates 10000 pseudo-random bytes. 10000 can be replaced
with any large value, as its only purpose is to prevent brute-force guessing.
This value is used so that nodes who do not know it can not in real-time
calculate who currently would be chosen as winner.

  2. (2)

RA calculates $s=Keccak256(PreviousBlockHash+randombytes)$ [”+” means
concatenation], and adds $s$ to the block problem definition (along with all
the simulation settings miners need to run the simulations). This means value
$s$ is broadcast along with the problem, but the input that led to $s$ will
only be revealed by the RA after the problem validity interval runs out.

As soon as problem validity interval ran out and no further solutions are
accepted. At this point in time, a list of all nodes (nodeID + signature) that
submitted the correct solution is known. More specifically, each node had
broadcast its Ed25519 signature of Keccak-256(solutionHash) before uploading
the solution to RA servers. All these signatures are put into a list by the RA
to create e.g. the following simplified list of two nodes that submitted the
correct solution. Note that the list is ordered by NodeID alphabetically
ascending:

NodeID | Signature
---|---
12D3aaa | abc
12D3bbb | def

The task is to choose one winner from this list:

  1. (3)

Publish random bytes that were used to calculate $s$. Now every miner can
verify $s=Keccak256(PreviousBlockHash+randombytes)$.

  2. (4)

Concatenate all signatures from the list into $c$. The list must be ordered
alphabetically ascending so that every node can reproduce this procedure.

  3. (5)

Calculate $a=KECCAK256(c+randombytes)$ and take the first 15 characters (so
that it later can be represented by a 64 bit integer). With this, there now
exists a value that every blockchain participant can reproduce, but no one
could have predicted this value before the problem solution was due: The RA
knew the random bytes, but can not possibly predict the resulting problem
solution signatures of any node as this value can only be created by a node
that knows the solution hash and its own private key. There is no way for the
RA to choose the random bytes in a specific way to influence the winner
selection outcome, otherwise Keccak-256 would need to be replaced with a
different cryptographically secure hashing algorithm. A node knows its own
private key but not any private keys of other nodes, which means that there is
no way to predict the resulting signatures of other nodes. Additionally, the
random bytes the RA used to create the known value $s$ will not be known until
the point in time where problem solution submissions are not allowed anymore
and the ordering of the resulting list is agreed to be alphabetically
ascending, so there not only is no way for a node to influence its position on
the resulting list (NodeID can not be changed due to signature mechanism) but
there also is no way to influence the resulting problem solution signature
(Ed25519 signatures are deterministic given same private key and data to
sign).hex

  4. (6)

Represent the hex string $a$ with its decimal representation and calculate
$winnerindex=a\mod len(list)$. The resulting index will be the winner index in
the list. Even if nodes were able to influence the position on this list, they
would not be able to calculate any index that is preferred over another index.
Every blockchain participant is able to calculate $winnerindex$ after the RA
publishes the required information which means that there is consensus on who
should be selected block winner of this block.

The procedure described above is deterministic (every node can follow it and
reach the same results), fair (there is no reason to prefer any list position
over another list position and the RA can not influence the winner selection
itself in some preferred direction) and transparent (the winner selection
procedure is public knowledge and can be reproduced by any blockchain
participant).
If the RA breaks protocol every node is able to notice it. The RA can not
collude with a subset of nodes (by leaking the at this point in time secret
$randombytes$) because the resulting signatures of the nodes that are not
colluding can not be known in time (if at least one node is not colluding
there is no way to determine the winnerindex beforehand). An additional
security measure is that the random bytes are combined with the previous block
hash, which even if there were a vulnerability, would limit the time for any
pre-calculations as the hash of the previous block can not be known before a
specific point in time. Every node knows that the generation of random bytes
happens at a point in time where the at a later time resulting signatures are
not known, so there is no reason to think that there exists a malicious choice
of random bytes. A node is not able to fake signatures and the list cannot
contain nodes that had broadcast invalid signatures (invalid as in the data
that was signed was not the solution hash which now is known by everyone and
which is the same for everyone).
The data that is broadcast by nodes while solution submissions are still
allowed does not in any way help to determine the solution of the problem,
which means a node has to solve the block problem to get added to the winner
selection list. Faking node identities (and therefore improving winning
chances by having multiple entries on the list) in a Sybil attack will not be
possible, because in order to join the blockchain a node must generate a
private key, derive its public key, derive its nodeID and then register its
real-world identity along with its nodeID at the Root Authority. As long as
the real-world identity registration process is transparent for all blockchain
participants, there is no reason to question the fairness of the winner
selection process described in this paper.

###  2.2. Golang code

This code demonstrates the idea described in this paper in a simplified
scenario. The following code randomly generates five libp2p node identities,
lets them sign an imaginary solution hash using Ed25519 and then shows how to
select a winner among the nodes.

⬇

1// tested with go 1.20.8

2package main

3

4import (

5 "fmt" // stdout

6 "sort" // sorting

7 "strings" // alphabetically sorting

8 "strconv" // hex -> decimal

9 "encoding/hex" // BytesToString/StringToBytes

10 "github.com/libp2p/go-libp2p/core/crypto"// ed25519

11 "golang.org/x/crypto/sha3" // keccak256

12 libp2p "github.com/libp2p/go-libp2p" // libp2p nodeID

13)

14

15type ExampleNode struct {

16 PrivKey crypto.PrivKey

17 PubKey crypto.PubKey

18 NodeID string // libp2p nodeID

19 ExampleSig string // signature

20}

21

22func main() {

23 // example solution hash

24 solHashString := "69868b59cab0f269284b96acca

25␣␣␣␣␣␣␣␣5549ab804095fcb452d64aba3c904bc82117bc"

26

27 // example previous blockhash

28 //prevBlockHash := "d60ee5d9b1a312631632d0ab

29␣␣␣␣␣␣␣␣8816ca64259093d8ab0b4d29f35db6a6151b0f8d"

30

31 // example random bytes

32 // (shorter length and constant to keep example simple)

33 randBytes := "a4896a3f93bf4bf58378e579f3cf193bb4af

34␣␣␣␣␣␣␣␣1022af7d2089f37d8bae7157b85f"

35

36 // Root Authority publishes

37 // Keccak256(prevBlockHash + randBytes) and then later

38 // reveals randBytes

39

40 // \---- End of preparations \----

41

42 // hold example nodes in slice

43 nodeList := []ExampleNode{}

44

45 // generate 5 nodes for this example

46 for c:=0; c < 5; c+=1 {

47 // generate keys

48 priv, pub, err := crypto.GenerateKeyPair(

49 crypto.Ed25519, // ed25519

50 -1, // default key len

51 )

52 if err != nil {

53 panic(err)

54 }

55

56 // get libp2p nodeID

57 h, err := libp2p.New(

58 libp2p.Identity(priv),

59 )

60 if err != nil {

61 panic(err)

62 }

63 nodeID := h.ID().String()

64

65 // get []byte representation of solHash

66 solHash, err := hex.DecodeString(solHashString)

67 if err != nil {

68 panic(err)

69 }

70

71 // sign it

72 sig, err := priv.Sign(solHash) // returns []byte

73 if err != nil {

74 panic(err)

75 }

76

77 // ensure signature is valid

78 verifiedSigSuccess, err := pub.Verify(solHash,sig)

79 if err != nil {

80 panic(err)

81 }

82 if !verifiedSigSuccess {

83 panic("Invalid␣signature!")

84 }

85

86 // get string representation of sig

87 sigString := hex.EncodeToString(sig)

88

89 // create example node object

90 n := ExampleNode {

91 PrivKey: priv,

92 PubKey: pub,

93 NodeID: nodeID,

94 ExampleSig: sigString,

95 }

96 // append this node to slice of nodes

97 nodeList = append(nodeList, n)

98

99 }

100

101 // sort list of nodes alphabetically ascending

102 // (0<9<a<z and case-insensitive) by nodeID

103 sort.Slice(nodeList, func(i, j int) bool {

104 return strings.ToLower(nodeList[i].NodeID) <

105 strings.ToLower(nodeList[j].NodeID)

106 })

107

108 // print all sorted nodeIDs and their signatures

109 for i, node := range nodeList {

110 fmt.Printf("Node␣%v:␣%v␣\nSignature:␣%v␣\n\n", i,

111 node.NodeID,

112 node.ExampleSig)

113 }

114

115 // Event: Problem not valid anymore, no further solution

116 // submissions allowed. Root Authority publishes

117 // values solHashString and randBytes. This means nodes

118 // can now verify that other nodes signed the correct

119 // solution hash and they can follow

120 // the following winner selection procedure because

121 // randBytes is now publicly known. The list of winners

122 // in this example is nodeList

123

124 // \---- Winner selection \----

125

126 // winner list is sorted already. now concatenate all

127 // signatures into c

128 c := ""

129 for _, curNode := range nodeList {

130 c += curNode.ExampleSig

131 }

132

133 // calculate a = Keccak256(c + randomBytes)

134 a := Keccak256(c + randBytes)

135

136 // take first 15 chars of a and convert to decimal number

137 // calculate winner_index = a mod len(winnerlist)

138 aDec, err := strconv.ParseInt(a[:15], 16, 64)

139 if err != nil {

140 panic(err)

141 }

142

143 // calculate winner_index = a mod len(winnerlist)

144 winnerIndex := aDec % int64(len(nodeList))

145

146 // print nodeID of winner

147 fmt.Printf("----\nWinner␣Index:␣%v␣\nBlock␣Winner:␣%v␣\n",

148 winnerIndex,

149 nodeList[winnerIndex].NodeID)

150}

151

152// Keccak256 takes string and returns its Keccak256 hash as string

153func Keccak256(input string) string {

154 // you could replace below with New256() but then

155 // it will not show same output as common

156 // online comparison tools

157 hash := sha3.NewLegacyKeccak256()

158 hash.Write([]byte(input))

159 hashSum := hash.Sum(nil)

160 hashString := hex.EncodeToString(hashSum)

161

162 return hashString

163}


# Multilingual and cross-lingual document classification:
A meta-learning approach

Niels van der Heijden♣ Helen Yannakoudakis♠ Pushkar Mishra♢ Ekaterina Shutova♣
♣ILLC, University of Amsterdam, the Netherlands
♠Dept. of Informatics, King’s College London, United Kingdom
♢Facebook AI, London, United Kingdom
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The great majority of languages in the world are considered under-resourced
for the successful application of deep learning methods. In this work, we
propose a meta-learning approach to document classification in a limited-
resource setting and demonstrate its effectiveness in two different settings:
few-shot, cross-lingual adaptation to previously unseen languages; and
multilingual joint training when limited target-language data is available
during training. We conduct a systematic comparison of several meta-learning
methods, investigate multiple settings in terms of data availability and show
that meta-learning thrives in settings with a heterogeneous task distribution.
We propose a simple, yet effective adjustment to existing meta-learning
methods which allows for better and more stable learning, and set a new state
of the art on several languages while performing on-par on others, using only
a small amount of labeled data.

##  1 Introduction

There are more than 7000 languages around the world and, of them, around 6%
account for 94% of the population.111https://www.ethnologue.com/statistics
Even for the 6% most spoken languages, very few of them possess adequate
resources for natural language research and, when they do, resources in
different domains are highly imbalanced. Additionally, human language is
dynamic in nature: new words and domains emerge continuously and hence no
model learned in a particular time will remain valid forever.

With the aim of extending the global reach of Natural Language Processing
(NLP) technology, much recent research has focused on the development of
multilingual models and methods to efficiently transfer knowledge across
languages. Among these advances are multilingual word vectors which aim to
give word-translation pairs a similar encoding in some embedding space Mikolov
et al. (2013a); Lample et al. (2017). There has also been a lot of work on
multilingual sentence and word encoders that either explicitly utilizes
corpora of bi-texts Artetxe and Schwenk (2019); Lample and Conneau (2019) or
jointly trains language models for many languages in one encoder Devlin et al.
(2018); Conneau et al. (2019). Although great progress has been made in cross-
lingual transfer learning, these methods either do not close the gap with
performance in a single high-resource language Artetxe and Schwenk (2019);
Conneau et al. (2019); van der Heijden et al. (2019), e.g., because of
cultural differences in languages which are not accounted for, or are
impractically expensive Lai et al. (2019).

Meta-learning, or learning to learn Schmidhuber (1987); Bengio et al. (1990);
Thrun and Pratt (1998), is a learning paradigm which focuses on the quick
adaption of a learner to new tasks. The idea is that by training a learner to
adapt quickly and from a few examples on a diverse set of training tasks, the
learner can also generalize to unseen tasks at test time. Meta-learning has
recently emerged as a promising technique for few-shot learning for a wide
array of tasks Finn et al. (2017); Koch et al. (2015); Ravi and Larochelle
(2017) including NLP Dou et al. (2019); Gu et al. (2018). To our best
knowledge, no previous work has been done in investigating meta-learning as a
framework for multilingual and cross-lingual few-shot learning. We propose
such a framework and demonstrate its effectiveness in document classification
tasks. The only current study on meta-learning for cross-lingual few-shot
learning is the one by Nooralahzadeh et al. (2020), focusing on natural
language inference and multilingual question answering. In their work, the
authors focus on applying meta-learning to learn to adapt a monolingually
trained classifier to new languages. In contrast to this work, we instead show
that, in many cases, it is more favourable to not initialize the meta-learning
process from a monolingually trained classifier, but rather reserve its
respective training data for meta-learning instead.

Our contributions are as follows: 1) We propose a meta-learning approach to
few-shot cross-lingual and multilingual adaptation and demonstrate its
effectiveness on document classification tasks over traditional supervised
learning; 2) We provide an extensive comparison of meta-learning methods on
multilingual and cross-lingual few-shot learning and release our code to
facilitate further research in the
field;222https://github.com/mrvoh/meta_learning_multilingual_doc_classification
3) We analyse the effectiveness of meta-learning under a number of different
parameter initializations and multiple settings in terms of data availability,
and show that meta-learning can effectively learn from few examples and
diverse data distributions; 4) We introduce a simple yet effective
modification to existing methods and empirically show that it stabilizes
training and converges faster to better local optima; 5) We set a new state of
the art on several languages and achieve on-par results on others using only a
small amount of data.

##  2 Meta-learning methods

Algorithm 1 Meta-training procedure.

0: $p(\mathcal{D})$: distribution over tasks.

0: $\alpha,\beta$: step size hyper-parameters

Initialize $\theta$

while not done do

Sample batch of tasks $\\{D^{l}\\}=\\{(S^{l},Q^{l})\\}\sim p(\mathcal{D})$

for all $(S^{l},Q^{l})$ do

Initialize $\theta_{l}^{(0)}=\theta$

for all steps k do

Compute:
$\theta_{l}^{(k+1)}=\theta_{l}^{(k)}-\alpha(\nabla_{\theta_{l}^{(k)}}\mathcal{L}_{S_{l}}(f_{\theta_{l}^{(k)}}))$

end for

end for

Update $\theta=\theta-\beta($MetaUpdate$(f_{\theta_{l}^{(K)}},Q^{l}))$

end while

Meta-learning, or learning to learn, aims to create models that can learn new
skills or adapt to new tasks rapidly from few training examples. Unlike
traditional machine learning, datasets for either training or testing, which
are referred to as meta-train and meta-test datasets, comprise of many tasks
sampled from a distribution of tasks $p(\mathcal{D})$ rather than individual
data points. Each task is associated with a dataset $\mathcal{D}$ which
contains both feature vectors and ground truth labels and is split into a
support set and a query set, $\mathcal{D}=\\{S,Q\\}$. The support set is used
for fast adaptation and the query set is used to evaluate performance and
compute a loss with respect to model parameter initialization. Generally, some
model $f_{\theta}$ parameterized by $\theta$, often referred to as the base-
learner, is considered. A cycle of fast-adaptation on a support-set followed
by updating the parameter initialization of the base-learner based on the loss
on the query-set is called an episode. In the case of classification, the
optimal parameters maximize the probability of the true labels across multiple
batches $Q\subset\mathcal{D}$

$\displaystyle\displaystyle\theta^{*}:=arg\underset{\theta}{max}\mathbb{E}_{Q\subset\mathcal{D}}[\sum_{(x,y)\in
Q}P_{\theta}(y|x)]$ (1)

In few-shot classification/fast learning, the goal is to minimize the
prediction error on data samples with unknown labels given a small support set
for learning. Meta-training (Algorithm 1) consists of updating the parameters
of the base-learner by performing many of the formerly described episodes,
until some stop criterion is reached.

Following this procedure, the extended definition of optimal parameters is
given in Eq. 2 to include fast adaptation based on the support set. The
underlined parts mark the difference between traditional supervised-learning
and meta-learning. The optimal parameters $\theta^{*}$ are obtained by solving

$\displaystyle\scriptstyle
arg\underset{\theta}{max}\underline{\mathbb{E}_{l\subset
L}[}\mathbb{E}_{\underline{S^{l}\subset\mathcal{D}},Q^{l}\subset\mathcal{D}}[\sum_{(x,y)\in
Q^{l}}P_{\theta}(y|x,\underline{S^{l}})]\underline{]}$ (2)

In this work, we focus on metric- and optimization-based meta-learning
algorithms. In the following sections, their respective characteristics and
the update methods in Algorithm 1 are introduced.

###  2.1 Prototypical Networks

Prototypical Networks Snell et al. (2017) belong to the metric-based family of
meta-learning algorithms. Typically they consist of an embedding network
$f_{\theta}$ and a distance function $d(x_{1},x_{2})$ such as Euclidean
distance. The embedding network is used to encode all samples in the support
set $S_{c}$ and compute prototypes $\mu_{c}$ per class $c\in C$ by computing
the mean of the sample encodings of that respective class

$\displaystyle\mu_{c}:=\frac{1}{|S_{c}|}\sum_{(x_{i},y_{i})\in
S_{c}}f_{\theta}(x_{i})$ (3)

Using the computed prototypes, Prototypical Networks classify a new sample as

$\displaystyle
p(y=c|x)=\frac{exp(-d(f_{\theta}(x),\mu_{c})}{\sum_{c^{^{\prime}}\in
C}exp(-d(f_{\theta}(x),\mu_{c^{^{\prime}}})}$ (4)

Wang et al. (2019) show that despite their simplicity, Prototypical Networks
can perform on par or better than other state-of-the-art meta-learning methods
when all sample encodings are centered around the overall mean of all classes
and consecutively L2-normalized. We also adopt this strategy.

###  2.2 MAML

Model-Agnostic Meta-Learning (MAML) Finn et al. (2017) is an optimization-
based method that uses the following objective function

$\displaystyle\theta^{*}:=arg\underset{\theta}{min}\sum_{D_{l}\sim
p(D)}\mathcal{L}_{l}(f_{\theta_{l}^{(k)}})$ (5)

$\mathcal{L}_{l}(f_{\theta_{l}^{(k)}})$ is the loss on the query set after
updating the base-learner for $k$ steps on the support set. Hence, MAML
directly optimizes the base-learner such that fast-adaptation of $\theta$,
often referred to as inner-loop optimization, results in task-specific
parameters $\theta_{l}^{(k)}$ which generalize well on the task. Setting $B$
as the batch size, MAML implements its MetaUpdate, which is also referred to
as outer-loop optimization, as

$\displaystyle\theta=\theta-\beta\frac{1}{B}\sum_{D_{l}\sim
p(\mathcal{D})}(\nabla_{\theta}\mathcal{L}_{l}(f_{\theta_{l}^{(k)}}))$ (6)

Such a MetaUpdate requires computing second order derivatives and, in turn,
holding $\theta_{l}^{(j)}\forall j=1,\dots,k$ in memory. A first-order
approximation of MAML (foMAML), which ignores second order derivatives, can be
used to bypass this problem:

$\displaystyle\theta=\theta-\beta\frac{1}{B}\sum_{D_{l}\sim
p(\mathcal{D})}(\nabla_{\theta_{l}^{(k)}}\mathcal{L}_{l}(f_{\theta_{l}^{(k)}}))$
(7)

Following previous work Antoniou et al. (2018), we also adopt the following
improvements in our framework for all MAML-based methods:

#### Per-step Layer Normalization weights

Layer normalization weights and biases are not updated in the inner-loop.
Sharing one set of weights and biases across inner-loop steps implicitly
assumes that the feature distribution between layers stays the same at every
step of the inner optimization.

#### Per-layer per-step learnable inner-loop learning rate

Instead of using a shared learning rate for all parameters, the authors
propose to initialize a learning rate per layer and per step and jointly learn
their values in the MetaUpdate steps.

#### Cosine annealing of outer-loop learning rate

It has shown to be crucial to model performance to anneal the learning rate
using some annealing function Loshchilov and Hutter (2016).

###  2.3 Reptile

Reptile Nichol et al. (2018) is a first-order optimization-based meta-learning
algorithm which is designed to move the weights towards a manifold of the
weighted averages of task-specific parameters $\theta_{l}^{(k)}$:

$\displaystyle\theta=\theta-\beta\frac{1}{B}\sum_{D^{l}\sim
p(\mathcal{D})}(\theta_{l}^{(k)}-\theta)$ (8)

Despite its simplicity, it has shown competitive or superior performance
against MAML, e.g., on Natural Language Understanding Dou et al. (2019).

###  2.4 ProtoMAML

Triantafillou et al. (2020) introduce ProtoMAML as a meta-learning method
which combines the complementary strengths of Prototypical Networks and MAML
by leveraging the inductive bias of the use of prototypes instead of random
initialization of the final linear layer of the network. Snell et al. (2017)
show that Prototypical Networks are equivalent to a linear model when
Euclidean distance is used. Using the definition of prototypes $\mu_{c}$ as
per Eq. 3, the weights $w_{c}$ and bias $b_{c}$ corresponding to class $c$ can
be computed as follows

$\displaystyle\mathbf{w}_{c}:=2\mu_{c}\qquad b_{c}:=-\mu_{c}^{T}\mu_{c}$ (9)

ProtoMAML is defined as the adaptation of MAML where the final linear layer is
parameterized as per Eq. 9 at the start of each episode using the support set.
Due to this initialization, it allows modeling a varying number of classes per
episode.

#### ProtoMAMLn

Inspired by Wang et al. (2019), we propose a simple, yet effective adaptation
to ProtoMAML by applying $L_{2}$ normalization to the prototypes themselves,
referred to as ProtoMAMLn, and, again, use a first-order approximation
(foProtoMAMLn). We demonstrate that doing so leads to a more stable, faster
and effective learning algorithm at only constant extra computational cost
($\mathcal{O}(1))$.

We hypothesize the normalization to be particularly beneficial in case of a
relatively high-dimensional final feature space – in case of BERT-like models
typically 768 dimensions. Let $x$ be a sample and $\hat{x}=f_{\theta}(x)$ be
the encoding of the sample in the final feature space. Since the final
activation function is the tanh activation, all entries of both $\hat{x}$ and
$\mu_{c}$ have values between -1 and 1. The pre-softmax activation for class
$c$ is computed as $\hat{x}^{T}\mu_{c}$. Due to the size of the vectors and
the scale of their respective entries, this in-product can yield a wide range
of values, which in turn results in relatively high loss values, making the
inner-loop optimization unstable.

##  3 Related work

###  3.1 Multilingual NLP

Just as the deep learning era for monolingual NLP started with the invention
of dense, low-dimensional vector representations for words Mikolov et al.
(2013b) so did cross-lingual NLP with works like those of Mikolov et al.
(2013a); Faruqui et al. (2014). More recently, multilingual and/or cross-
lingual NLP is approached by training one shared encoder for multiple
languages at once, either by explicitly aligning representations with the use
of parallel corpora Artetxe and Schwenk (2019); Lample and Conneau (2019) or
by jointly training on some monolingual language model objective, such as the
Masked Language Model (MLM) Devlin et al. (2018), in multiple languages Devlin
et al. (2018); Conneau et al. (2019).

The formerly described language models aim to create a shared embedding space
for multiple languages with the hope that fine-tuning in one language does not
degrade performance in others. Lai et al. (2019) argue that just aligning
languages is not sufficient to generalize performance to new languages due to
the phenomenon they describe as domain drift. Domain drift accounts for all
differences for the same tasks in different languages which cannot be captured
by a perfect translation system, such as differences in culture. They instead
propose a multi-step approach which utilizes a multilingual teacher trained
with Unsupervised Data Augmentation (UDA) Xie et al. (2019) to create labels
for a student model that is pretrained on large amounts of unlabeled data in
the target language and domain using the MLM objective. With their method, the
authors obtain state-of-the-art results on the MLDoc document classification
task Schwenk and Li (2018) and the Amazon Sentiment Polarity Review task
Prettenhofer and Stein (2010). A downside, however, is the high computational
cost involved. For every language and domain combination: 1) a machine
translation system has to be inferred on a large amount of unlabeled samples;
2) the UDA method needs to be applied to obtain a teacher model to generate
pseudo-labels on the unlabeled in-domain data; 3) a language model must be
finetuned, which involves forwards and backwards computation of a softmax
function over a large output space (e.g., 50k tokens for mBERT and 250k tokens
for XLM-RoBERTa). The final classifier is then obtained by 4) training the
finetuned language model on the pseudo-labels generated by the teacher.

###  3.2 Meta-learning in NLP

#### Monolingual

Bansal et al. (2019) apply meta-learning to a wide range of NLP tasks within a
monolingual setting and show superior performance for parameter initialization
over self-supervised pretraining and multi-task learning. Their method is an
adaptation of MAML where a combination of a text-encoder, BERT Devlin et al.
(2018), is coupled with a parameter generator that learns to generate task-
dependent initializations of the classification head such that meta-learning
can be performed across tasks with disjoint label spaces. Obamuyide and
Vlachos (2019b) apply meta-learning on the task of relation extraction;
Obamuyide and Vlachos (2019a) apply lifelong meta-learning for relation
extraction; Chen et al. (2019) apply meta-learning for few-shot learning on
missing link prediction in knowledge graphs.

#### Multilingual

Gu et al. (2018) apply meta-learning to Neural Machine Translation (NMT) and
show its advantage over strong baselines such as cross-lingual transfer
learning. By viewing each language pair as a task, the authors apply MAML to
obtain competitive NMT systems with as little as 600 parallel sentences. To
our best knowledge, the only application of meta-learning for cross-lingual
few-shot learning is the one by Nooralahzadeh et al. (2020). The authors study
the application of X-MAML, a MAML-based variant, to cross-lingual Natural
Language Inference (XNLI) Conneau et al. (2018) and Multilingual Question
Answering (MLQA) Lewis et al. (2019) in both a cross-domain and cross-language
setting. X-MAML works by pretraining some model $M$ on a high-resource task
$h$ to obtain initial model parameters $\theta_{mono}$. Consecutively, a set
$L$ of one or more auxiliary languages is taken, and MAML is applied to
achieve fast adaptation of $\theta_{mono}$ for $l\in L$. In their experiments,
the authors use either one or two auxiliary languages and evaluate their
method in both a zero- and few-shot setting. It should be noted that, in the
few-shot setting, the full development set (2.5k instances) is used to
finetune the model, which is not in line with other work on few-shot learning,
such as Bansal et al. (2019). Also, there is a discrepancy in the training set
used for the baselines and their proposed method. All reported baselines are
either zero-shot evaluations of $\theta_{mono}$ or of $\theta_{mono}$
finetuned on the development set of the target language, whereas their
proposed method additionally uses the development set in either one or two
auxiliary languages during meta-training.

MetaUpdate Method | Num inner-loop steps | Inner-loop lr | Class-head lr multiplier | Inner-optimizer lr
---|---|---|---|---
Reptile | 2,3,5 | 1e-5, 5e-5, 1e-4 |  1, 10 | -
foMAML | 2,3,5 |  1e-5, 1e-4, 1e-3 | 1, 10 | 3e-5, 6e-5, 1e-4
foProtoMAMLn | 2,3,5 |  1e-5, 1e-4, 1e-3 | 1, 10 | 3e-5, 6e-5, 1e-4
Table 1: Search range per hyper-parameter. We consider the number of update
steps in the inner-loop, Num inner-loop steps, the (initial) learning rate of
the inner-loop, Inner-loop lr, the factor by which the learning rate of the
classification head is multiplied, Class-head lr multiplier, and, if
applicable, the learning rate with which the inner-loop optimizer is updated,
Inner-optimizer lr. The chosen value is underlined.

##  4 Data

In this section, we give an overview of the datasets we use and the respective
classification tasks.

#### MLDoc

Schwenk and Li (2018) published an improved version of the Reuters Corpus
Volume 2 Lewis et al. (2004) with balanced class priors for all languages.
MLDoc consists of news stories in 8 languages: English, Spanish, French,
Italian, Russian, Japanese and Chinese. Each news story is manually classified
into one of four groups: Corporate/Industrial, Economics, Government/Social
and Markets. The train datasets contain 10k samples whereas the test sets
contain 4k samples.

#### Amazon Sentiment Polarity

Another widely used dataset for cross-lingual text classification is the
Amazon Sentiment Analysis dataset Prettenhofer and Stein (2010). The dataset
is a collection of product reviews in English, French, German and Japanese in
three categories: books dvds and music. Each sample consists of the original
review accompanied by meta-data such as the rating of the reviewed product
expressed as an integer on a scale from one to five. In this work, we consider
the sentiment polarity task where we distinguish between positive (rating $>$
3) and negative (rating $<$ 3) reviews. When all product categories are
concatenated, the dataset consists of 6K samples per language per dataset
(train, test). We extend this with Chinese product reviews in the cosmetics
domain from JD.com Zhang et al. (2015), a large e-commerce website in China.
The train and test sets contain 2k and 20k samples respectively.

##  5 Experiments

We use XLM-RoBERTa Conneau et al. (2019), a strong multilingual model, as the
base-learner in all models. We quantify the strengths and weaknesses of meta-
learning as opposed to traditional supervised learning in both a cross- and a
multilingual joint-training setting with limited resources.

#### Cross-lingual adaptation

Here, the available data is split into multiple subsets: the auxiliary
languages $l_{aux}$ which are used in meta-training, the validation language
$l_{dev}$ which is used to monitor performance, and the target languages
$l_{tgt}$ which are kept unseen until meta-testing. Two scenarios in terms of
amounts of available data are considered. A small sample of the available
training data of $l_{aux}$ is taken to create a limited-resource setting,
whereas all available training data of $l_{aux}$ is used in a high-resource
setting. The chosen training data per language is split evenly and stratified
over two disjoint sets from which the meta-training support and query samples
are sampled, respectively. For meta-testing, one batch (16 samples) is taken
from the training data of each target language as support set, while we test
on the whole test set per target language (i.e., the query set).

#### Multilingual joint training

We also investigate meta-learning as an approach to multilingual joint-
training in the same limited-resource setting as previously described for the
cross-lingual experiments. The difference is that instead of learning to
generalize to $l_{tgt}\neq l_{aux}$ from few examples, here $l_{tgt}=l_{aux}$.
If we can show that one can learn many similar tasks across languages from few
examples per language, using a total number of examples in the same order of
magnitude as in “traditional” supervised learning for training a monolingual
classifier, this might be an incentive to change data collection processes in
practice.

For both experimental settings above, we examine the influence of additionally
using all training data from a high-resource language $l_{src}$ during meta-
training, English.

$\mathbf{l_{src}}$ = en | Method | Limited-resource setting | High-resource setting |  |
---|---|---|---|---|---
de | fr | it | ja | ru | zh | $\Delta$ | de | fr | it | ja | ru | zh | $\Delta$
Excluded | Non-episodic | 82.0 | 86.7 | 68.3 | 71.9 | 70.9 | 81.0 | 76.8 | 95.3 | 90.9 | 80.9 | 82.9 | 74.5 | 89.6 | 85.7
ProtoNet | 90.5 | 85.0 | 76.6 | 75.0 | 69.6 | 82.0 | 79.8 | 95.5 | 91.7 | 82.0 | 82.2 | 76.6 | 87.4 | 85.9
foMAML | 89.7 | 85.5 | 74.1 | 74.1 | 74.0 | 83.2 | 80.1 | 95.0 | 91.4 | 81.4 | 82.7 | 76.9 | 87.8 | 86.1
foProtoMAMLn | 90.6 | 86.2 | 77.8 | 75.6 | 73.6 | 83.8 | 80.7 | 95.6 | 92.1 | 82.6 | 83.1 | 77.9 | 88.9 | 86.7
Reptile | 87.9 | 81.8 | 72.7 | 74.4 | 73.9 | 80.9 | 78.6 | 95.0 | 90.1 | 81.1 | 82.7 | 72.5 | 88.7 | 85.0
Included | Zero-shot | 92.4 | 92.1 | 80.3 | 81.0 | 71.7 | 89.1 | 84.4 | 92.4 | 92.1 | 80.3 | 81.0 | 71.7 | 89.1 | 84.4
Non-episodic | 93.7 | 91.3 | 81.5 | 80.6 | 71.1 | 88.4 | 84.4 | 93.7 | 92.9 | 82.4 | 82.3 | 72.1 | 90.1 | 85.6
ProtoNet | 93.4 | 91.9 | 79.1 | 81.3 | 72.2 | 87.8 | 84.5 | 95.0 | 91.7 | 81.1 | 82.7 | 72.0 | 88.0 | 85.9
foMAML | 95.1 | 91.2 | 79.5 | 79.6 | 73.3 | 89.7 | 84.6 | 94.8 | 93.2 | 79.9 | 82.4 | 75.7 | 90.6 | 86.1
foProtoMAMLn | 94.9 | 91.7 | 81.5 | 81.4 | 75.2 | 89.9 | 85.5 | 95.8 | 94.1 | 82.7 | 83.0 | 81.2 | 90.4 | 87.9
| Reptile | 92.3 | 91.4 | 79.7 | 79.5 | 71.8 | 88.1 | 83.8 | 94.8 | 91.0 | 80.2 | 82.0 | 72.7 | 89.9 | 85.1

Table 2: Average accuracy of 5 different seeds on the unseen target languages
for MLDoc. $\Delta$ corresponds to the average accuracy across test languages.

###  5.1 Specifics per dataset

#### MLDoc

As MLDoc has sufficient languages, we set $l_{src}=$ English and $l_{dev}=$
Spanish. The remaining languages are split in two groups:
$l_{aux}=\\{\textrm{German, Italian, Japanese}\\}$; and
$l_{tgt}=\\{\textrm{French, Russian, Chinese}\\}$. In the limited-resource
setting, we randomly sample 64 samples per language in $l_{aux}$ for training.
Apart from comparing low- and high-resource settings, we also quantify the
influence of augmenting the training set $l_{aux}$ with a high-resource source
language $l_{src}$, English.

#### Amazon Sentiment Polarity

The fact that the Amazon dataset (augmented with Chinese) comprises of only
five languages has some implications for our experimental design. In the
cross-lingual experiments, where $l_{aux}$, $l_{dev}$ and $l_{tgt}$ should be
disjoint, only three languages, including English, remain for meta-training.
As we consider two languages too little data for meta-training, we do not
experiment with leaving out the English data. Hence, for meta-training, the
data consists of $l_{src}=$ English, as well as two languages in $l_{aux}$. We
always keep one language unseen until meta-testing, and alter $l_{aux}$ such
that we can meta-test on every language. We set $l_{dev}=$ French in all cases
except when French is used as the target language; then, $l_{dev}=$ Chinese.
In the limited-resource setting, a total of 128 samples per language in
$l_{aux}$ is used.

For the multilingual joint-training experiments there are enough languages
available to quantify the influence of English during meta-training. When
English is excluded, it is used for meta-validation. When included, we average
results over two sets of experiments: one where $l_{dev}=$ French and one
where $l_{dev}=$ Chinese.

Method | Limited-resource setting |  | High-resource setting |
---|---|---|---|---
de | fr | ja | zh | $\Delta$ | de | fr | ja | zh | $\Delta$
Zero-shot | 91.2 | 90.7 | 87.0 | 84.6 | 88.4 | 91.2 | 90.7 | 87.0 | 84.6 | 88.4
Non-episodic | 90.9 | 90.6 | 86.1 | 86.9 | 88.6 | 91.6 | 91.0 | 85.5 | 87.9 | 89.0
ProtoNet | 89.7 | 90.2 | 86.6 | 85.2 | 87.9 | 90.7 | 92.0 | 86.7 | 84.0 | 88.4
foMAML | 88.3 | 90.5 | 86.8 | 88.1 | 88.4 | 91.4 | 92.5 | 88.0 | 90.4 | 90.6
foProtoMAMLn | 89.0 | 91.1 | 87.3 | 88.8 | 89.1 | 92.0 | 93.1 | 88.6 | 89.8 | 90.9
Reptile | 88.1 | 87.9 | 86.8 | 87.5 | 87.6 | 90.6 | 91.7 | 87.3 | 86.2 | 89.0
Table 3: Average accuracy of 5 different seeds on the unseen target languages
for Amazon. $\Delta$ corresponds to the average accuracy across test
languages.

###  5.2 Baselines

We introduce baselines trained in a standard supervised, non-episodic fashion.
Again, we use XLM-RoBERTa-base as the base-learner in all models.

#### Zero-shot

This baseline assumes sufficient training data for the task to be available in
one language $l_{src}$ (English). The base-learner is trained in a non-
episodic manner using mini-batch gradient descent with cross-entropy loss.
Performance is monitored during training on a held-out validation set in
$l_{src}$, the model with the lowest loss is selected, and then evaluated on
the same task in the target languages.

#### Non-episodic

The second baseline aims to quantify the exact impact of learning a model
through the meta-learning paradigm versus standard supervised learning. The
model learns from exactly the same data as the meta-learning algorithms, but
in a non-episodic manner: i.e., merging support and query sets in $l_{aux}$
(and $l_{src}$ when included) and training using mini-batch gradient descent
with cross-entropy loss. During testing, the trained model is independently
finetuned for 5 steps on the support set (one mini-batch) of each target
language $l_{tgt}$.

###  5.3 Training setup and hyper-parameters

We use the Ranger optimizer, an adapted version of Adam Kingma and Ba (2014)
with improved stability at the beginning of training – by accounting for the
variance in adaptive learning rates Liu et al. (2019) – and improved
robustness and convergence speed Zhang et al. (2019); Yong et al. (2020). We
use a batch size of 16 and a learning rate of 3e-5 to which we apply cosine
annealing. For meta-training, we perform 100 epochs of 100 episodes and
perform evaluation with 5 different seeds on the meta-validation set after
each epoch. One epoch consists of 100 update steps where each update step
consists of a batch of 4 episodes. Early-stopping with a patience of 3 epochs
is performed to avoid overfitting. For the non-episodic baselines, we train
for 10 epochs on the auxiliary languages while validating after each epoch.
All models are created using the PyTorch library Paszke et al. (2017) and
trained on a single 24Gb NVIDIA Titan RTX GPU.

We perform grid search on MLDoc in order to determine optimal hyperparameters
for the MetaUpdate methods. The hyper-parameters resulting in the lowest loss
on $l_{dev}=$ Spanish are used in all experiments. The number of update steps
in the inner-loop is 5; the (initial) learning rate of the inner-loop is 1e-5
for MAML and ProtoMAML and 5e-5 for Reptile; the factor by which the learning
rate of the classification head is multiplied is 10 for MAML and ProtoMAML and
1 for Reptile; when applicable, the learning rate with which the inner-loop
optimizer is updated is 6e-5. See Table 1 for the considered grid.

$\mathbf{l_{src}}$ = en | Method | Amazon | MLDoc
---|---|---|---
de | fr | ja | zh | $\Delta$ | de | fr | it | ja | ru | zh | $\Delta$
Excluded | Non-episodic | 88.4 | 88.6 | 85.7 | 88.2 | 87.7 | 92.8 | 89.1 | 81.2 | 83.2 | 84.0 | 87.4 | 86.3
ProtoNet | 86.7 | 88.0 | 86.2 | 87.3 | 87.1 | 89.7 | 87.6 | 80.5 | 82.2 | 80.6 | 85.2 | 84.3
foMAML | 88.3 | 87.5 | 84.6 | 89.1 | 86.3 | 94.1 | 89.7 | 81.5 | 84.2 | 77.6 | 87.5 | 85.8
foProtoMAMLn | 88.9 | 89.5 | 86.5 | 89.0 | 88.5 | 94.8 | 89.5 | 81.5 | 84.8 | 81.0 | 88.7 | 86.6
Reptile | 86.1 | 86.3 | 82.9 | 87.0 | 85.6 | 92.4 | 88.2 | 80.5 | 82.5 | 79.5 | 87.8 | 85.3
Included | Non-episodic | 91.0 | 91.0 | 87.3 | 89.4 | 89.8 | 94.9 | 92.1 | 84.7 | 84.8 | 83.7 | 91.4 | 88.6
ProtoNet | 90.3 | 91.3 | 87.5 | 88.7 | 89.5 | 95.5 | 91.7 | 83.4 | 85.1 | 82.8 | 88.3 | 87.8
foMAML | 90.1 | 90.7 | 87.2 | 89.5 | 89.4 | 95.1 | 92.5 | 83.1 | 84.9 | 84.3 | 90.6 | 88.4
foProtoMAMLn | 90.7 | 91.5 | 88.0 | 90.4 | 90.2 | 96.0 | 93.6 | 85.0 | 85.7 | 84.8 | 90.8 | 89.3
Reptile | 90.0 | 89.5 | 86.5 | 87.6 | 88.4 | 94.4 | 93.1 | 83.8 | 85.2 | 83.6 | 90.4 | 88.4

Table 4: Average accuracy of 5 different seeds on the target languages in the
joint-training setting for MLDoc and Amazon. $\Delta$ corresponds to the
average accuracy across test languages.

##  6 Results

#### Cross-lingual adaptation

Tables 2 and 3 show the accuracy scores on the target languages on MLDoc and
Amazon respectively. We start by noting the strong multilingual capabilities
of XLM-RoBERTa as our base-learner: Adding the full training datasets in three
extra languages (i.e., comparing the zero-shot with the non-episodic baseline
in the high-resource, ‘Included’ setting) results in a mere 1.2% points
increase in accuracy on average for MLDoc and 0.6% points for Amazon. Although
the zero-shot333The zero-shot baseline is only applicable in the ‘Included’
setting, as the English data is not available under ‘Excluded’. and non-
episodic baselines are strong, in the majority of cases, a meta-learning
approach improves performance. This holds especially for our version of
ProtoMAML (ProtoMAMLn), which achieves the highest average accuracy in all
considered settings.

The substantial improvements for Russian on MLDoc and Chinese on Amazon
indicate that meta-learning is most advantageous when the considered task
distribution is somewhat heterogeneous or, in other words, when domain drift
Lai et al. (2019) is present. For the Chinese data used for the sentiment
polarity task, the presence of domain drift is obvious as the data is
collected from a different website and concerns different products than the
other languages. For Russian in the MLDoc dataset, it holds that the non-
episodic baseline has the smallest gain in performance when adding English
data ($l_{src}$) in the limited-resource setting (0.2% absolute gain as
opposed to 5.7% on average for the remaining languages) and even a decrease of
2.4% points when adding English data in the high-resource setting. Especially
for these languages with domain drift, our version of ProtoMAML (foProtoMAMLn)
outperforms the non-episodic baselines with a relatively large margin. For
instance, in Table 2 in the high-resource setting with English included during
training, foProtoMAMLn improves over the non-episodic baseline with 9.1%
points whereas the average gain over the remaining languages is 0.9% points. A
similar trend can be seen in Table 3 where, in the limited-resource setting,
foProtoMAMLn outperforms the non-episodic baseline with 1.9% points on
Chinese, with comparatively smaller gains on average for the remaining
languages.

#### Joint training

In this setting, we achieve a new state of the art on MLDoc for German,
Italian, Japanese and Russian using our method, foProtoMAMLn (Table 4).444The
zero-shot baselines are the same as in Tables 2 and 3. The previous state of
the art for German and Russian is held by Lai et al. (2019) (95.73% and 84.65%
respectively). For Japanese and Italian, it is held by Eisenschlos et al.
(2019) (80.55% and 80.12% respectively). The state of the art for French and
Chinese is also held by Lai et al. (2019) (96.05% and 93.32% respectively). On
the Amazon dataset, foProtoMAMLn also outperforms all other methods on
average. The state of the art is held by Lai et al. (2019) with 93.3%, 94.2%
and 90.6% for French, German and Chinese respectively and, although we do not
outperform it, the differences are rather small – between 0.2% (Chinese) and
3.4% points (German) – even when grid search is based on MLDoc, while we use a
much less computationally expensive approach.

Figure 1: Validation accuracy for 3 seeds for original foProtoMAML and our new
method, foProtoMAMLn.

Again, we use Russian in MLDoc to exemplify the difference between meta-
learning and standard supervised learning. When comparing the difference in
performance between excluding and including English meta-training episodes
($l_{src}$), opposite trends are noticeable: for standard supervised, non-
episodic learning, performance drops slightly by 0.3%, whereas all meta-
learning algorithms gain between 2.2% and 6.7% in absolute accuracy. This
confirms our earlier finding that meta-learning benefits from, and usefully
exploits heterogeneity in data distributions; in contrast, this harms
performance in the standard supervised-learning case.

Dataset | de | fr | it | ja | ru | zh | Diff
---|---|---|---|---|---|---|---
Amazon | 90.4 | 90.9 | - | 87.3 | - | 88.3 | -1.7
MLDoc | 92.8 | 92.4 | 78.6 | 79.3 | 69.3 | 88.9 | -4.3

Table 5: Average accuracy of 5 different seeds on unseen target languages using the original/unnormalized foProtoMAML model. Diff is the difference in average accuracy $\Delta$ across languages against foProtoMAMLn. Method | Limited-resource setting |  | High-resource setting |
---|---|---|---|---
de | fr | ja | zh | Diff | de | fr | ja | zh | Diff
ProtoNet | 91.1 | 90.9 | 87.1 | 85.5 | +0.75 | 91.3 | 91.1 | 87.4 | 88.7 | +1.44
foMAML | 90.8 | 87.4 | 87.3 | 85.2 | -0.75 | 91.7 | 91.2 | 87.2 | 88.1 | -1.13
foProtoMAMLn | 87.7 | 87.8 | 83.9 | 84.4 | -3.1 | 90.8 | 89.8 | 86.2 | 82.3 | -3.96
Reptile | 89.3 | 90.2 | 86.7 | 85.5 | +0.35 | 90.0 | 89.3 | 87.1 | 85.7 | -1.04
Table 6: Average accuracy of 5 different seeds on unseen target languages for
Amazon when initializing from monolingual classifier in $l_{src}$. Diff:
difference in average accuracy $\Delta$ across languages compared to
initializing from the XLM-RoBERTa language model.

##  7 Ablations

#### foProtoMAMLn

Figure 1 shows the development of the validation accuracy during training for
25 epochs for the original foProtoMAML and our model, foProtoMAMLn. By
applying $L_{2}$ normalization to the prototypes, we obtain a more stable
version of foProtoMAML which empirically converges faster. We furthermore re-
run the high-resource experiments with English for both MLDoc and Amazon using
the original foProtoMAML (Table 5) and find it performs 4.3% and 1.7% accuracy
points worse on average, respectively, further demonstrating the effectiveness
of our approach.

#### Initializing from a monolingual classifier

In our experiments, we often assume the presence of a source language
(English). We now investigate (in the $l_{src}$ = en ‘Excluded’ setting)
whether it is beneficial to pre-train the base-learner in a standard
supervised way on this source language and use the obtained checkpoint
$\theta_{mono}$ as an initialization for meta-training (Table 6) rather than
initializing from the transformer checkpoint.

We observe that only ProtoNet consistently improves performance, whereas
foProtoMAMLn suffers the most with a decrease of 3.1% and 3.96% in accuracy in
the low- and high-resource setting respectively. We surmise this difference is
attributable to two factors. Intuitively, the monolingual classifier aims to
learn a transformation from the input space to the final feature space, from
which the prototypes for ProtoNet and ProtoMAML are created, in which the
learned classes are encoded in their own disjoint sub-spaces such that a
linear combination of these features can be used to correctly classify
instances. ProtoNet aims to learn a similar transformation, but uses a Nearest
Neighbours approach to classify instances instead. ProtoMAML on the other hand
benefits the most from prototypes which can be used to classify instances
after the inner-loop updates have been performed. This, in combination with
the fact that the first-order approximation of ProtoMAML cannot differentiate
through the creation of the prototypes, could explain the difference in
performance gain with respect to ProtoNet.

##  8 Conclusion

We proposed a meta-learning framework for few-shot cross- and multilingual
joint-learning for document classification tasks in different domains. We
demonstrated that it leads to consistent gains over traditional supervised
learning on a wide array of data availability and diversity settings, and
showed that it thrives in settings with a heterogenous task distribution. We
presented an effective adaptation to ProtoMAML and, among others, obtained a
new state of the art on German, Italian, Japanese and Russian in the few-shot
setting on MLDoc.

##  9 Acknowledgements

This work was supported by Deloitte Risk Advisory B.V., the Netherlands.

## References

  * Antoniou et al. (2018) Antreas Antoniou, Harrison Edwards, and Amos Storkey. 2018.  How to train your maml.  _arXiv preprint arXiv:1810.09502_.
  * Artetxe and Schwenk (2019) Mikel Artetxe and Holger Schwenk. 2019.  Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond.  _Transactions of the Association for Computational Linguistics_ , 7:597–610.
  * Bansal et al. (2019) Trapit Bansal, Rishikesh Jha, and Andrew McCallum. 2019.  Learning to few-shot learn across diverse natural language classification tasks.  _arXiv preprint arXiv:1911.03863_.
  * Bengio et al. (1990) Yoshua Bengio, Samy Bengio, and Jocelyn Cloutier. 1990.  _Learning a synaptic learning rule_.  Citeseer.
  * Chen et al. (2019) Mingyang Chen, Wen Zhang, Wei Zhang, Qiang Chen, and Huajun Chen. 2019.  Meta relational learning for few-shot link prediction in knowledge graphs.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4217–4226, Hong Kong, China. Association for Computational Linguistics.
  * Conneau et al. (2019) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Unsupervised cross-lingual representation learning at scale.  _arXiv preprint arXiv:1911.02116_.
  * Conneau et al. (2018) Alexis Conneau, Guillaume Lample, Ruty Rinott, Adina Williams, Samuel R Bowman, Holger Schwenk, and Veselin Stoyanov. 2018.  Xnli: Evaluating cross-lingual sentence representations.  _arXiv preprint arXiv:1809.05053_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Dou et al. (2019) Zi-Yi Dou, Keyi Yu, and Antonios Anastasopoulos. 2019.  Investigating meta-learning algorithms for low-resource natural language understanding tasks.  _arXiv preprint arXiv:1908.10423_.
  * Eisenschlos et al. (2019) Julian Eisenschlos, Sebastian Ruder, Piotr Czapla, Marcin Kardas, Sylvain Gugger, and Jeremy Howard. 2019.  Multifit: Efficient multi-lingual language model fine-tuning.  _arXiv preprint arXiv:1909.04761_.
  * Faruqui et al. (2014) Manaal Faruqui, Jesse Dodge, Sujay K Jauhar, Chris Dyer, Eduard Hovy, and Noah A Smith. 2014.  Retrofitting word vectors to semantic lexicons.  _arXiv preprint arXiv:1411.4166_.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017.  Model-agnostic meta-learning for fast adaptation of deep networks.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , pages 1126–1135. JMLR. org.
  * Gu et al. (2018) Jiatao Gu, Yong Wang, Yun Chen, Kyunghyun Cho, and Victor OK Li. 2018.  Meta-learning for low-resource neural machine translation.  _arXiv preprint arXiv:1808.08437_.
  * van der Heijden et al. (2019) Niels van der Heijden, Samira Abnar, and Ekaterina Shutova. 2019.  A comparison of architectures and pretraining methods for contextualized multilingual word embeddings.  _arXiv preprint arXiv:1912.10169_.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Koch et al. (2015) Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015.  Siamese neural networks for one-shot image recognition.  In _ICML deep learning workshop_ , volume 2. Lille.
  * Lai et al. (2019) Guokun Lai, Barlas Oguz, and Veselin Stoyanov. 2019.  Bridging the domain gap in cross-lingual document classification.  _arXiv preprint arXiv:1909.07009_.
  * Lample and Conneau (2019) Guillaume Lample and Alexis Conneau. 2019.  Cross-lingual language model pretraining.  _arXiv preprint arXiv:1901.07291_.
  * Lample et al. (2017) Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2017\.  Unsupervised machine translation using monolingual corpora only.  _arXiv preprint arXiv:1711.00043_.
  * Lewis et al. (2004) David D Lewis, Yiming Yang, Tony G Rose, and Fan Li. 2004.  Rcv1: A new benchmark collection for text categorization research.  _Journal of machine learning research_ , 5(Apr):361–397.
  * Lewis et al. (2019) Patrick Lewis, Barlas Oğuz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2019.  Mlqa: Evaluating cross-lingual extractive question answering.  _arXiv preprint arXiv:1910.07475_.
  * Liu et al. (2019) Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. 2019.  On the variance of the adaptive learning rate and beyond.  _arXiv preprint arXiv:1908.03265_.
  * Loshchilov and Hutter (2016) Ilya Loshchilov and Frank Hutter. 2016.  Sgdr: Stochastic gradient descent with warm restarts.  _arXiv preprint arXiv:1608.03983_.
  * Mikolov et al. (2013a) Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013a.  Exploiting similarities among languages for machine translation.  _arXiv preprint arXiv:1309.4168_.
  * Mikolov et al. (2013b) Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b.  Distributed representations of words and phrases and their compositionality.  In _Advances in neural information processing systems_ , pages 3111–3119.
  * Nichol et al. (2018) Alex Nichol, Joshua Achiam, and John Schulman. 2018.  On first-order meta-learning algorithms.  _arXiv preprint arXiv:1803.02999_.
  * Nooralahzadeh et al. (2020) Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva, and Isabelle Augenstein. 2020.  Zero-shot cross-lingual transfer with meta learning.  _arXiv preprint arXiv:2003.02739_.
  * Obamuyide and Vlachos (2019a) Abiola Obamuyide and Andreas Vlachos. 2019a.  Meta-learning improves lifelong relation extraction.  In _Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)_ , pages 224–229, Florence, Italy. Association for Computational Linguistics.
  * Obamuyide and Vlachos (2019b) Abiola Obamuyide and Andreas Vlachos. 2019b.  Model-agnostic meta-learning for relation classification with limited supervision.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5873–5879, Florence, Italy. Association for Computational Linguistics.
  * Paszke et al. (2017) Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017.  Automatic differentiation in pytorch.  In _NIPS 2017 Workshop Autodiff Submission_.
  * Prettenhofer and Stein (2010) Peter Prettenhofer and Benno Stein. 2010.  Cross-language text classification using structural correspondence learning.  In _Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics_ , pages 1118–1127, Uppsala, Sweden. Association for Computationprettenhoferal Linguistics.
  * Ravi and Larochelle (2017) Sachin Ravi and Hugo Larochelle. 2017.  Optimization as a model for few-shot learning.  In _International Conference on Learning Representations_.
  * Schmidhuber (1987) Jurgen Schmidhuber. 1987.  Evolutionary principles in self-referential learning.  _On learning how to learn: The meta-meta-… hook.) Diploma thesis, Institut f. Informatik, Tech. Univ. Munich_ , 1(2).
  * Schwenk and Li (2018) Holger Schwenk and Xian Li. 2018.  A corpus for multilingual document classification in eight languages.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , Paris, France. European Language Resources Association (ELRA).
  * Snell et al. (2017) Jake Snell, Kevin Swersky, and Richard Zemel. 2017.  Prototypical networks for few-shot learning.  In _Advances in neural information processing systems_ , pages 4077–4087.
  * Thrun and Pratt (1998) Sebastian Thrun and Lorien Pratt. 1998.  Learning to learn: Introduction and overview.  In _Learning to learn_ , pages 3–17. Springer.
  * Triantafillou et al. (2020) Eleni Triantafillou, Tyler Zhu, Vincent Dumoulin, Pascal Lamblin, Utku Evci, Kelvin Xu, Ross Goroshin, Carles Gelada, Kevin Swersky, Pierre-Antoine Manzagol, and Hugo Larochelle. 2020.  Meta-dataset: A dataset of datasets for learning to learn from few examples.  In _International Conference on Learning Representations_.
  * Wang et al. (2019) Yan Wang, Wei-Lun Chao, Kilian Q Weinberger, and Laurens van der Maaten. 2019.  Simpleshot: Revisiting nearest-neighbor classification for few-shot learning.  _arXiv preprint arXiv:1911.04623_.
  * Xie et al. (2019) Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le. 2019.  Unsupervised data augmentation for consistency training.  _arXiv preprint arXiv:1904.12848_.
  * Yong et al. (2020) Hongwei Yong, Jianqiang Huang, Xiansheng Hua, and Lei Zhang. 2020.  Gradient centralization: A new optimization technique for deep neural networks.  _arXiv preprint arXiv:2004.01461_.
  * Zhang et al. (2019) Michael Zhang, James Lucas, Jimmy Ba, and Geoffrey E Hinton. 2019.  Lookahead optimizer: k steps forward, 1 step back.  In _Advances in Neural Information Processing Systems_ , pages 9597–9608.
  * Zhang et al. (2015) Yongfeng Zhang, Min Zhang, Yi Zhang, Guokun Lai, Yiqun Liu, Honghui Zhang, and Shaoping Ma. 2015.  Daily-aware personalized recommendation based on feature-level time series analysis.  In _Proceedings of the 24th international conference on world wide web_ , pages 1373–1383.