# Fréchet differentiability via partial Fréchet differentiability

Luděk Zajíček<EMAIL_ADDRESS>Charles University, Faculty of
Mathematics and Physics, Sokolovská 83, 186 75 Praha 8-Karlín, Czech Republic

###### Abstract.

Let $X_{1},\dots,X_{n}$ be Banach spaces and $f$ a real function on
$X=X_{1}\times\dots\times X_{n}$. Let $A_{f}$ be the set of all points $x\in
X$ at which $f$ is partially Fréchet differentiable but is not Fréchet
differentiable. Our results imply that if $X_{1},\dots,X_{n-1}$ are Asplund
spaces and $f$ is continuous (resp. Lipschitz) on $X$, then $A_{f}$ is a first
category set (resp. a $\sigma$-upper porous set). We also prove that if $X$,
$Y$ are separable Banach spaces and $f:X\to Y$ is a Lipschitz mapping, then
the set of all points $x\in X$ at which $f$ is Gâteaux differentiable, is
Fréchet differentiable along a closed subspace of finite codimension but is
not Fréchet differentiable, is $\sigma$-upper porous. A number of related more
general results are also proved.

2000 Mathematics Subject Classification: Primary: 46G05 ; Secondary: 46T20

Keywords: Fréchet differentiability, partial Fréchet differentiability,
Asplund space, first category set, $\sigma$-porous set

##  1\. Introduction

If $f$ is a real function on ${\mathbb{R}}^{n}$, denote by $A_{f}$ the set of
all points $x\in{\mathbb{R}}^{n}$ at which $f$ has all (finite) partial
derivatives $f_{1}^{\prime}(x),\dots,f_{n}^{\prime}(x)$ but it is not
(Fréchet) differentiable. Of course, $A_{f}$ can be nonempty and Stepanoff’s
[16] examples show that $A_{f}$ can have positive measure for a continuous
function on ${\mathbb{R}}^{2}$ (he constructed even such a function which is
everywhere partially differentiable and also a continuous function on
${\mathbb{R}}^{2}$ which is partially differentiable almost everywhere but it
is nowhere differentiable).

However, the situation is different, if we consider “topological smallness”
instead of “measure smallness”, as the following result shows.

Theorem C Let $G\subset{\mathbb{R}}^{n}$ be an open set and
$f:G\to{\mathbb{R}}$ a continuous function. Then

$A_{f}:=\\{x\in G:\
f_{1}^{\prime}(x)\in{\mathbb{R}},\dots,f_{n}^{\prime}(x)\in{\mathbb{R}}\
\text{and}\ f^{\prime}(x)\ \text{does not exist}\\}$

is a first category (=meagre) set.

For $n=2$, this result follows from Gorlenko’s 1977 article [7]. For general
$n$, it was proved (even for $f:G\to Y$, where $Y$ is a separable Banach
space) by Ka-Sing-Lau and Weil [11] in 1978.

A related remarkable result from [15] gives that the conclusion of Theorem C
holds if $f$ is an arbitrary function which is partially differentiable
everywhere in $G$.

Bessis and Clarke [1] in 1999 proved the following “Lipschitz version” of
Theorem C.

Theorem L Let $G\subset{\mathbb{R}}^{n}$ be an open set and
$f:G\to{\mathbb{R}}$ a Lipschitz function. Then

$A_{f}:=\\{x\in G:\
f_{1}^{\prime}(x)\in{\mathbb{R}},\dots,f_{n}^{\prime}(x)\in{\mathbb{R}}\
\text{and}\ f^{\prime}(x)\ \text{does not exist}\\}$

is a $\sigma$-porous set.

In the present article, $\sigma$-porosity is “$\sigma$-upper porosity” (see
Definition 2.1 below, cf. [19]), i.e. it is considered in “Denjoy-Dolzhenko
sense”. Note that if $A\subset{\mathbb{R}}^{n}$ is $\sigma$-porous, then it is
both of the first category and Lebesgue null, but the opposite implication
does not hold. So Theorem L does not follow from Theorem C and the Rademacher
theorem.

In the present article we prove some generalizations of Theorem C and Theorem
L in the infinite-dimensional setting. Namely, let $X_{1},\dots,X_{n}$ and $Y$
be Banach spaces, $G$ an open subset of $X:=X_{1}\times\cdots\times X_{n}$
(equipped with the maximum norm), and $f:G\to Y$ a mapping. Denote by $A_{f}$
the set of all points $x\in G$ at which all partial Fréchet derivatives of $f$
exist but the Fréchet derivative $f^{\prime}(x)$ does not exist.

In Section 4 we prove (in Theorem 4.5) that $A_{f}$ is a first category set
whenever $f$ is continuous and

(1.1) $\text{the spaces of continuous linear mappings}\
{\mathcal{L}}(X_{1},Y),\dots,{\mathcal{L}}(X_{n-1},Y)\ \text{are separable}.$

We obtain this result as an immediate consequence of an easy known result from
[13] (Proposition 4.4 below) and Theorem 4.1 (which can be of an independent
interest) which says that, under some conditions, Fréchet differentiability
along a subspace $V$ generically implies strict differentiability along $V$.

In Section 5 we prove by a different (more technical) method Theorem 5.2 which
implies that, under condition (1.1), the generalization of Theorem L also
holds.

In Section 6, using the method of separable reduction, we show in Theorem 6.7
that generalizations of Theorem C and Theorem L hold also under a condition
more general than condition (1.1). In particular, we prove that
generalizations of Theorem C and Theorem L hold if $Y={\mathbb{R}}$ and
$X_{1},\dots,X_{n-1}$ are Asplund spaces (Corollary 6.8).

I do not believe that condition (1.1) can be omitted in Theorem 4.5 and/or
Theorem 5.2. Unfortunately, I was not able to find any counterexample. So it
is still possible that the Banach space generalizations of Theorem C and/or
Theorem L hold in the full generality and from this reason I do not discuss
all cases in which the validity of these generalizations follow by the methods
of the present article (cf. Remark 6.9).

In the proof of Theorem 6.7, we use a result (Proposition 3.3) on the Borel
type of the set of all points at which a partial derivative of a continuous
(or a slightly more general) function exists. This result which generalizes a
proposition on functions on ${\mathbb{R}}^{2}$ from [12] and can be of an
independent interest is proved in Section 3.

We consider (mainly in Section 5) also related problems where instead of a
product space $X:=X_{1}\times\cdots\times X_{n}$ we consider an arbitrary
Banach space $X$ and instead of Fréchet partial derivatives we consider
Fréchet derivatives along subspaces. In this direction, we obtain Propositions
4.8, Proposition 5.3, and Corollary 5.6 which is an immediate consequence of
more general Proposition 5.5, which can be of an independent interest. An
another consequence of this proposition is Corollary 5.7 which says that if
$X$, $Y$ are separable Banach spaces and $f:X\to Y$ is a Lipschitz mapping,
then the set of all points $x\in X$ at which $f$ is Gâteaux differentiable, is
Fréchet differentiable along a closed subspace of finite codimension but is
not Fréchet differentiable, is $\sigma$-porous.

##  2\. Preliminaries

In the following, we consider real notrivial (i.e. not equal to $\\{0\\}$)
Banach spaces. In any fixed Banach space, we denote the zero vector by $0$ and
the norm by $|\cdot|$. By a subspace $Y$ of a Banach space $X$, we will mean a
Banach subspace of $X$, i.e. a closed linear subspace $Y\neq\\{0\\}$. We set
$S_{X}:=\\{x\in X:|x|=1\\}$. By $\operatorname{span}M$ we denote the linear
span of $M\subset X$. The equality $X=X_{1}\oplus\cdots\oplus X_{n}$ means
that the Banach space $X$ is the topological direct sum of its subspaces
$X_{1},\dots,X_{n}$. The symbol $B(x,r)$ will denote the open ball with center
$x$ and radius $r$.

######  Definition 2.1.

Let $A$ be a subset of a Banach space $X$.

  1. (i)

We say that $A$ is porous at a point $x\in X$ if there exists $c>0$ such that
for each $\delta>0$ there exists $t\in B(x,\delta)\setminus\\{x\\}$ such that
$B(t,c|t-x|)\cap A=\emptyset$.

  2. (ii)

$A$ is called a porous set if $A$ is porous at each point $x\in A$.

  3. (iii)

$A$ is called a $\sigma$-porous set if it is a countable union of porous sets.

If $X$ and $Y$ are Banach spaces, the space of all continuous linear mappings
$\varphi:X\to Y$ (equipped with the usual norm) will be denoted by $\cal
L(X,Y)$.

The word “generically” has the usual sense; it means “at all points except a
first category set”.

Recall that a Banach space $X$ is called an Asplund space if each continuous
convex function on $X$ is generically Fréchet differentiable and that

(2.1) $X$ is Asplund if and only if $Y^{*}$ is separable for each separable
subspace $Y\subset X$.

Let $X$, $Y$ be Banach spaces, $G\subset X$ an open set, and $f:G\to Y$ a
mapping. We say that $f$ is Lipschitz at $x\in G$ if $\limsup_{y\to
x}\frac{|f(y)-f(x)|}{|y-x|}<\infty$. The directional and one-sided directional
derivatives of $f$ at $x\in G$ in the direction $v\in X$ are defined
respectively by

$f^{\prime}(x,v):=\lim_{t\to 0}\frac{f(x+tv)-f(x)}{t}\ \ \text{and}\ \
f^{\prime}_{+}(x,v):=\lim_{t\to 0+}\frac{f(x+tv)-f(x)}{t}.$

######  Definition 2.2.

Let $X$ and $Y$ be Banach spaces, $V$ a closed subspace of $X$, $G\subset X$
an open set, $a\in G$ and $f:G\to Y$ a mapping. We say that $f$ is Fréchet
differentiable at $a$ along $V$, if the mapping $g(v):=f(a+v),\ v\in
V\cap(G-a)$, is Fréchet differentiable at $0\in V$ and set
$f^{\prime}_{V}(a):=g^{\prime}(0)\in\cal L(V,Y)$.

We say that $f$ is strictly differentiable at $a$ along $V$ if
$f^{\prime}_{V}(a)$ exists and

(2.2) $\lim_{(x,\tilde{x})\to(a,a),\ 0\neq\tilde{x}-x\in V}\
\frac{|f(\tilde{x})-f(x)-f^{\prime}_{V}(a)(\tilde{x}-x)|}{|\tilde{x}-x|}=0.$

######  Remark 2.3.

  1. (i)

In the above definition, some authors write “with respect to $V$” or “in the
direction of $V$” instead of “along $V$”.

  2. (ii)

The standard strict differentiability coincides with strict differentiability
along $V:=X$. Note that some authors by “strict differentiability” means (a
weaker) “Gâteaux strict differentiability” which is stronger than Gâteaux
differentiability.

  3. (iii)

Condition (2.2) can be clearly rewritten as

(2.3) $\forall\varepsilon>0\,\exists\delta>0\,\forall v\in V:\ \
\\{x,x+v\\}\subset
B(a,\delta)\Rightarrow|f(x+v)-f(x)-f^{\prime}_{V}(a)v|\leq\varepsilon|v|.$

The notions of partial Fréchet differentiability and strict partial
differentiability are factually special cases of notions of “directional” and
“strict directional” differentiability along a subspace. If
$X_{1},\dots,X_{n}$ are Banach spaces, we consider the Banach space
$X:=X_{1}\times\cdots\times X_{n}$ (equipped with the maximum norm). In the
following definition, we consider, if $1\leq i\leq n$ is given, $X_{i}$ as a
subspace of $X$ (identifying, as usual, $x_{i}\in X_{i}$ with
$(0,\dots,0,x_{i},0,\dots,0)\in X$).

######  Definition 2.4.

Let $X_{1},\dots,X_{n}$ and $Y$ be Banach spaces, $X:=X_{1}\times\cdots\times
X_{n}$, $G\subset X$ an open set, $a\in G$ and $f:G\to Y$ a mapping. Then, for
$i=1,\dots,n$,

  1. (i)

we set $f^{\prime}_{i}(a):=f^{\prime}_{X_{i}}(a)$ and call it (if it exists)
partial Fréchet derivative of $f$ at $a$ with respect to the $i$ th variable
and

  2. (ii)

we say that $f$ is partially strictly differentiable at $a$ with respect to
the $i$ th variable if $f$ is strictly differentiable at $a$ along $X_{i}$.

##  3\. Borel type of the set of points where a partial derivative exists

If $X$, $Y$ are Banach spaces, $G\subset X$ an open set and $f:G\to Y$ an
arbitrary mapping, then (see [18, Theorem 2])

(3.1) the set $D(f)$ of all $x\in G$ at which $f$ is Fréchet differentiable is
an $F_{\sigma\delta}$ set.

This result was proved in [18] using a characterization of Fréchet
differentiability points (the proof in [10, Corollary 3.5.5] is quite
different). Applying this characterization to partial functions, we
immediately obtain a characterization (Lemma 3.2 below) of points at which $f$
has a partial Fréchet derivative. We need the following notation.

######  Definition 3.1.

Let $X_{1}$, $X_{2}$, $Y$ be Banach spaces, $G\subset X:=X_{1}\times X_{2}$ an
open set and $f:G\to Y$ a mapping.

  1. (i)

We denote by $C_{1}(f)$ the set of all points $x=(x_{1},x_{2})\in G$ at which
$f$ is continuous with respect to the first coordinate (i.e. $f(\cdot,x_{2})$
is continuous at $x_{1}$).

  2. (ii)

For $c>0$, $\varepsilon>0$ and $\delta>0$, denote by
$D_{1}(f,c,\varepsilon,\delta)$ the set of all points $x=(x_{1},x_{2})\in G$
such that

(3.2)
$\left|\frac{f(z+kv,x_{2})-f(z,x_{2})}{k}-\frac{f(z,x_{2})-f(z-hv,x_{2})}{h}\right|\leq\varepsilon$

whenever $v\in X_{1}$, $|v|=1$, $h>0$, $k>0$, $z\in X_{1}$, $z\in
B(x_{1},\delta)$, $z-hv\in B(x_{1},\delta)$, $z+kv\in B(x_{1},\delta)$ and
$\min(h,k)>c|z-x_{1}|$.

Then, by the definitions, $x=(x_{1},x_{2})\in D_{1}(f,c,\varepsilon,\delta)$
if and only if $x_{1}\in D(F,c,\varepsilon,\delta)$, where $F:=f(\cdot,x_{2})$
and $D(F,c,\varepsilon,\delta)$ is as in [18, Definition 3]. Since
$f^{\prime}_{1}(x)$ exists if and only $F^{\prime}(x_{1})$ exists, [18,
Theorem 1 and Note 2] immediately imply the following result.

######  Lemma 3.2.

Let $X_{1}$, $X_{2}$, $Y$, $G$ and $f$ be as in Definition 3.1 and $x\in G$.
Then the following conditions are equivalent.

  1. (i)

$f^{\prime}_{1}(x)$ exists.

  2. (ii)

$x\in
C_{1}(f)\cap\bigcap_{c>0}\bigcap_{\varepsilon>0}\bigcup_{\delta>0}D_{1}(f,c,\varepsilon,\delta).$

  3. (iii)

$x\in
C_{1}(f)\cap\bigcap_{n\in{\mathbb{N}}}\bigcup_{p\in{\mathbb{N}}}\,D_{1}(f,1/n,1/n,1/p).$

Using Lemma 3.2, we will prove the following proposition generalizing the
corresponding result on real functions in ${\mathbb{R}}^{2}$ which was proved
in [12] by a quite different elementary method. This proposition will be used
in the proofs of Proposition 4.7 and Theorem 6.7.

######  Proposition 3.3.

Let $X_{1}$, $X_{2}$, $Y$ be Banach spaces, $G\subset X:=X_{1}\times X_{2}$ an
open set and $f:G\to Y$ a mapping which is continuous with respect to the
second variable (i.e. all partial mappings $f(a_{1},\cdot)$, $(a_{1},a_{2})\in
G$, are continuous). Then the set $D_{1}(f):=\\{x\in X:\ f_{1}^{\prime}(x)\
\text{exists}\\}$ is an $F_{\sigma\delta\sigma}$ set. If $G=X$, then
$D_{1}(f)$ is an $F_{\sigma\delta}$ set.

###### Proof.

For each $k\in{\mathbb{N}}$, set $F_{k}:=\\{x\in G:\
\operatorname{dist}(x,X\setminus G)\geq 1/k\\}$ if $X\neq G$ and $F_{k}:=X$ if
$X=G$. Clearly each $F_{k}$ is a closed set and
$G=\bigcup_{k\in{\mathbb{N}}}F_{k}$. So by Lemma 3.2 we obtain

(3.3) $D_{1}(f)=\bigcup_{k\in{\mathbb{N}}}(F_{k}\cap
D_{1}(f))=\bigcup_{k\in{\mathbb{N}}}\left(F_{k}\cap
C_{1}(f)\cap\bigcap_{n\in{\mathbb{N}}}\bigcup_{p\in{\mathbb{N}}}\,D_{1}(f,1/n,1/n,1/p)\right).$

Consequently it is sufficient to prove that, for each $k\in{\mathbb{N}}$, both

(3.4) $F_{k}\cap C_{1}(f)\ \ \text{is an}\ \ F_{\sigma\delta}\ \ \text{set}$

and

(3.5)
$F_{k}\cap\bigcap_{n\in{\mathbb{N}}}\bigcup_{p\in{\mathbb{N}}}\,D_{1}(f,1/n,1/n,1/p)\
\ \text{is an}\ \ F_{\sigma\delta}\ \ \text{set}.$

First we will prove (3.4). To this end, fix an arbitrary $k\in{\mathbb{N}}$
and denote, for $m,j\in{\mathbb{N}}$,

$C_{m,j}:=\\{x=(x_{1},x_{2})\in G:\ |f(t,x_{2})-f(\tau,x_{2})|\leq 1/m\
\text{whenever}\ |t-x_{1}|<1/j,\ |\tau-x_{1}|<1/j\\},$

and observe that

$C_{1}(f)=\bigcap_{m=1}^{\infty}\bigcup_{j=k}^{\infty}C_{m,j},\ \ F_{k}\cap
C_{1}(f)=\bigcap_{m=1}^{\infty}\bigcup_{j=k}^{\infty}(F_{k}\cap C_{m,j}).$

So it is sufficient to prove that $F_{k}\cap C_{m,j}$ is a closed set whenever
$j\geq k$. To this end, fix arbitrary $m,j\in{\mathbb{N}}$ with $j\geq k$ and
suppose that $(x_{1}^{i},x_{2}^{i})\in F_{k}\cap C_{m,j}$, $i\in{\mathbb{N}}$,
and $(x_{1}^{i},x_{2}^{i})\to(x_{1},x_{2}),\ i\to\infty$. Then
$(x_{1},x_{2})\in F_{k}$ since $F_{k}$ is closed. To prove $(x_{1},x_{2})\in
C_{m,j}$, consider arbitrary $t,\tau\in X_{1}$ with $|t-x_{1}|<1/j$, $|\tau-
x_{1}|<1/j$. Then, for all sufficiently large $i$, we have
$|t-x_{1}^{i}|<1/j$, $|\tau-x_{1}^{i}|<1/j$, and consequently
$|f(t,x_{2}^{i})-f(\tau,x_{2}^{i})|\leq 1/m$. Since $(x_{1},x_{2})\in F_{k}$
and $j\geq k$, we obtain that $(t,x_{2})\in G$ and $(\tau,x_{2})\in G$. Since
$f$ is continuous with respect to the second variable, $f(t,x_{2}^{i})\to
f(t,x_{2})$, $f(\tau,x_{2}^{i})\to f(\tau,x_{2})$, and therefore
$|f(t,x_{2})-f(\tau,x_{2})|\leq 1/m$. So we obtain that $(x_{1},x_{2})\in
C_{m,j}$ and we are done.

To prove (3.5), fix an arbitrary $k\in{\mathbb{N}}$. Since clearly

$D_{1}(f,1/n,1/n,1/p_{1})\subset D_{1}(f,1/n,1/n,1/p_{2})\ \ \text{ whenever}\
\ p_{1}\leq p_{2},$

we have

$F_{k}\cap\bigcap_{n\in{\mathbb{N}}}\bigcup_{p\in{\mathbb{N}}}\,D_{1}(f,1/n,1/n,1/p)=\bigcap_{n=1}^{\infty}\bigcup_{p=k}^{\infty}(F_{k}\cap
D_{1}(f,1/n,1/n,1/p)).$

So it is sufficient to prove that $F_{k}\cap D_{1}(f,1/n,1/n,1/p)$ is a closed
set whenever $p\geq k$. To this end, fix arbitrary $n,p\in{\mathbb{N}}$ with
$p\geq k$ and suppose that $x^{i}=(x_{1}^{i},x_{2}^{i})\in F_{k}\cap
D_{1}(f,1/n,1/n,1/p)$, $i\in{\mathbb{N}}$, and $x^{i}=(x_{1}^{i},x_{2}^{i})\to
x=(x_{1},x_{2}),\ i\to\infty$. Then $(x_{1},x_{2})\in F_{k}$ since $F_{k}$ is
closed. To prove that $x=(x_{1},x_{2})\in D_{1}(f,1/n,1/n,1/p)$, consider
arbitrary $v\in X_{1}$ with $|v|=1$, reals $h>0$, $k>0$ and $z\in X_{1}$ such
that $z\in B(x_{1},1/p)$, $z-hv\in B(x_{1},1/p)$, $z+kv\in B(x_{1},1/p)$ and
$\min(h,k)>(1/n)|z-x_{1}|$. Since $(x_{1},x_{2})\in F_{k}$ and $p\geq k$, we
obtain that $(z,x_{2})\in G$, $(z+kv,x_{2})\in G$ and $(z-hv,x_{2})\in G$. Our
aim is to prove that

(3.6)
$\left|\frac{f(z+kv,x_{2})-f(z,x_{2})}{k}-\frac{f(z,x_{2})-f(z-hv,x_{2})}{h}\right|\leq\frac{1}{n}.$

Since $x_{1}^{i}\to x_{1}$, for all sufficiently large $i$, we have $z\in
B(x_{1}^{i},1/p)$, $z-hv\in B(x_{1}^{i},1/p)$, $z+kv\in B(x_{1}^{i},1/p)$ and
$\min(h,k)>(1/n)|z-x_{1}^{i}|$, which together with $x^{i}\in
D_{1}(f,1/n,1/n,1/p)$ implies

(3.7)
$\left|\frac{f(z+kv,x_{2}^{i})-f(z,x_{2}^{i})}{k}-\frac{f(z,x_{2}^{i})-f(z-hv,x_{2}^{i})}{h}\right|\leq\frac{1}{n}.$

Since $f$ is continuous with respect to the second variable, we have
$f(z,x_{2}^{i})\to f(z,x_{2})$, $f(z+kv,x_{2}^{i})\to f(z+kv,x_{2})$,
$f(z-hv,x_{2}^{i})\to f(z-hv,x_{2})$, and consequently (3.7) implies (3.6). So
we obtain that $x\in D_{1}(f,1/n,1/n,1/p)$ and we are done. ∎

##  4\. Case of continuous mappings

###  4.1. Strict directional differentiability via Fréchet directional
differentiability

A well-known theorem (see e.g. [18, Theorem B, p. 476]) asserts that Fréchet
differentiability at a point $x$ of an arbitrary mapping $f:X\to Y$, where
$X$, $Y$ are arbitrary Banach spaces, generically implies strict
differentiability of $f$ at $x$. The following result which is a partial
generalization of this theorem will be applied in the proof of Theorem 4.5.

######  Theorem 4.1.

Let $X$, $Y$ be Banach spaces and let $V$ be a subspace of $X$ such that the
space $\cal L(V,Y)$ is separable. Let $G\subset X$ be an open set and $f:G\to
Y$ a continuous mapping. Then the set $A$ of all $a\in G$ such that
$f^{\prime}_{V}(a)$ exists and $f$ is not strictly differentiable at $a$ along
$V$ is a first category set.

###### Proof.

Choose a countable dense subset $\Phi$ of $\cal L(V,Y)$ and consider an
arbitrary point $a\in A$. By (2.3) we can choose $n\in{\mathbb{N}}$ such that

(4.1) $\forall\,\delta>0\,\exists\,x\in X\,\exists v\in V:\ \
\\{x,x+v\\}\subset B(a,\delta),\
|f(x+v)-f(x)-f^{\prime}_{V}(a)v|>\frac{4}{n}|v|.$

Further choose $p\in{\mathbb{N}}$ such that

(4.2) $|f(a+v)-f(a)-f^{\prime}_{V}(a)v|\leq\frac{1}{n}|v|\ \ \
\text{whenever}\ \ \ v\in V,\ |v|\leq\frac{1}{p}$

and choose $\varphi\in\Phi$ such that

(4.3) $|f^{\prime}_{V}(a)-\varphi|\leq\frac{1}{n}.$

Denote, for each $n\in{\mathbb{N}}$, $p\in{\mathbb{N}}$ and $\varphi\in\Phi$,
by $A_{n,p,\varphi}$ the set of all $a\in A$ for which the conditions (4.1),
(4.2) and (4.3) hold. Then $A=\bigcup\\{A_{n,p,\varphi}:\
n,p\in{\mathbb{N}},\varphi\in\Phi\\}$ and so it is sufficient to show that all
sets $A_{n,p,\varphi}$ are nowhere dense.

To this end, suppose to the contrary that, for some fixed $n$, $p$, $\varphi$,
the set $A_{n,p,\varphi}$ is not nowhere dense. Then there exist $z\in
A_{n,p,\varphi}$ and $\omega>0$ such that $A_{n,p,\varphi}$ is dense in
$B(z,\omega)$. Now observe that, using (4.2) and (4.3), we easily obtain that

(4.4) $|f(a+v)-f(a)-\varphi(v)|\leq\frac{2}{n}|v|\ \ \text{whenever}\ \ \ a\in
A_{n,p,\varphi},\ v\in V,\ |v|\leq\frac{1}{p}.$

Applying (4.1) to $a:=z$ with $\delta:=\min(\omega,1/2p)$, we can choose $x\in
B(z,\delta)$ and $v\in V$ such that $x+v\in B(z,\delta)$ and

(4.5) $|f(x+v)-f(x)-f^{\prime}_{V}(z)v|>\frac{4}{n}|v|.$

Clearly $|v|<1/p$. By the choice of $z$, $\omega$ and $\delta$, there exist
$a_{k}\in A_{n,p,\varphi}$, $k\in{\mathbb{N}}$, such that $a_{k}\to x$, and
therefore $a_{k}+v\to x+v$. Since $|v|<1/p$, using (4.4) to $a:=a_{k}$,
$k\in{\mathbb{N}}$, and continuity of $f$, we easily obtain
$|f(x+v)-f(x)-\varphi(v)|\leq\frac{2}{n}|v|$. This inequality together with
$|f^{\prime}_{V}(z)-\varphi|\leq 1/n$ implies
$|f(x+v)-f(x)-f^{\prime}_{V}(z)v|\leq\frac{3}{n}|v|$ which contradicts (4.5).
∎

######  Remark 4.2.

Theorem 4.1 can be applied e.g. in the following cases:

  1. (i)

$V$ is finite-dimensional and $Y$ is separable.

  2. (ii)

$V$ is a separable Asplund space and $Y$ is finite-dimensional.

  3. (iii)

$V=C(K)$ for a countable compact set and $Y$ is separable with the Radon-
Nikodým property.

  4. (iv)

$V$ is a closed subspace of $c_{0}$ and $Y$ is separable with the Radon-
Nikodým property.

  5. (v)

$V=\ell_{p}$, $Y=\ell_{q}$, $1\leq q<p<\infty$.

Indeed, in all these cases, the space ${\mathcal{L}}(V,Y)$ is separable. The
most natural cases (i) and (ii) are almost obvious. For the cases (iii) and
(iv) see [10, pp. 114–115]. In the well-known case (v) Pitt’s theorem (see,
e.g., [6, Proposition 6.25]) says that ${\mathcal{L}}(V,Y)$ coincides with the
space ${\mathcal{K}}(V,Y)$ of compact operators which is separable (e.g. by
well-known [17, Fact 5.4, p. 20]).

For some other cases involving classical Banach spaces see [17, Example 5.5].

######  Remark 4.3.

I do not believe that the assumption that ${\mathcal{L}}(V,Y)$ is separable
can be omitted in Theorem 4.1, but I do not know any counterexample.

###  4.2. Fréchet differentiability of continuous functions via partial
Fréchet differentiability

The main result of the present section (Theorem 4.5 below) is an almost
immediate consequence of Theorem 4.1 and the following known result (see [13,
Proposition 2.57], where “strict differentiability” is called “circa-
differentiability”).

######  Proposition 4.4.

([13]) Let $X_{1},\dots,X_{n}$ and $Y$ be Banach spaces,
$X:=X_{1}\times\cdots\times X_{n}$, $G\subset X$ an open set, $a\in G$ and
$f:G\to Y$ a mapping. Let $f$ be partially Fréchet differentiable at $a$ with
respect to the $n$ th variable and let $f$ be strictly differentiable at $a$
with respect to the $j$ th variable for each $1\leq j\leq n-1$. Then $f$ is
Fréchet differentiable at $a$.

######  Theorem 4.5.

Let $X_{1},\dots,X_{n}$ and $Y$ be Banach spaces, $X:=X_{1}\times\cdots\times
X_{n}$, $G\subset X$ an open set, and let $f:G\to Y$ be a continuous mapping.
Suppose that the spaces $\cal L(X_{1},Y),\dots,\cal L(X_{n-1},Y)$ are
separable. Then there exists a first category set $A\subset G$ such that, for
all $x\in G\setminus A$, the following implication holds:

$f$ has all Fréchet partial derivatives at $x$ $\Rightarrow$ $f$ is Fréchet
differentiable at $x$.

###### Proof.

Let $A_{i}$, $1\leq i\leq n-1$, be the set of all $x\in G$ such that
$f^{\prime}_{X_{i}}(x)$ exists and $f$ is not strictly differentiable at $x$
along $X_{i}$ (recall that we identify $X_{i}$ with a subspace of $X$ by the
usual way). By Theorem 4.1 each $A_{i}$ is a first category set and
consequently $A:=A_{1}\cup\dots\cup A_{n-1}$ is also a first category set. If
$x\in G\setminus A$ anf $f$ has all Fréchet partial derivatives at $x$, then
$f$ is strictly differentiable at $x$ with respect to the $j$ th variable for
each $1\leq j\leq n-1$ and so $f$ is Fréchet differentiable at $x$ by
Proposition 4.4. ∎

######  Remark 4.6.

Probably the most interesting is the case of a real function $f$ (i.e.
$Y={\mathbb{R}}$). Then we assume that the dual space $(X_{i})^{*}$ is
separable (i.e. $X_{i}$ is a separable Asplund space) for each $1\leq i\leq
n-1$. Using the method of separable reduction, we will show that the result
holds also if $X_{i}$, $1\leq i\leq n-1$, are general Asplund spaces (see
Corollary 6.8 below). A number of other concrete applications of Theorem 4.5
can be easily obtained using the facts from Remark 4.2.

As an easy consequence of Theorem 4.5 (and Proposition 3.3) we obtain the
following result on generic Fréchet differentiability of functions whose all
partial functions are generically Fréchet differentiable.

######  Proposition 4.7.

Let $X_{1},\dots,X_{n}$ and $Y$ be Banach spaces, $X:=X_{1}\times\cdots\times
X_{n}$, and let $f:X\to Y$ be continuous. Suppose that all $X_{i}$, $1\leq
i\leq n$, are separable and the spaces $\cal L(X_{1},Y),\dots,\cal
L(X_{n-1},Y)$ are separable. Let each partial function

$f(x_{1},\dots,x_{i-1},\cdot,x_{i+1},\dots,x_{n}),\ \ 1\leq i\leq n,$

be generically Fréchet differentiable on $X_{i}$. Then $f$ is generically
Fréchet differentiable.

###### Proof.

For each $1\leq i\leq n$, denote

$P_{i}:=\\{x\in X:\ f^{\prime}_{i}(x)\ \ \text{does not exist}\\}.$

Identifying by the natural way $X$ with $Y_{i}\times X_{i}$, where
$Y_{i}:=X_{1}\times\cdots\times X_{i-1}\times X_{i+1}\times\cdots\times
X_{n}$, observe that, by the assumptions, the set $\\{t\in X_{i}:(y,t)\in
P_{i}\\}$ is a first category set in $X_{i}$ for each $y\in Y_{i}$. Since
$P_{i}$ is a Borel set by Proposition 3.3, it has the Baire property and
consequently the Kuratowski-Ulam theorem (see, e.g., [9, Theorem 8.41])
implies that $P_{i}$ is a first category set. Thus $f$ has all Fréchet partial
derivatives outside the first category set $P_{1}\cup\cdots\cup P_{n}$ and
consequently $f$ is generically Fréchet differentiable by Theorem 4.5. ∎

Similarly as Theorem 4.5, we obtain the following result.

######  Proposition 4.8.

Let $X$, $Y$ be Banach spaces, $G\subset X$ an open set and $f:G\to Y$ a
continuous mapping. Let $V_{1}$ be a subspace of $X$ such that the space $\cal
L(V_{1},Y)$ is separable. Then there exists a first category set $A\subset G$
such that, for each $x\in G\setminus A$, the following assertion holds:

$(*)$ If $f$ is Fréchet differentiable at $x$ along $V_{1}$ and along some
topological complement $V_{2}^{x}$ of $V_{1}$, then $f$ is Fréchet
differentiable at $x$.

###### Proof.

Let $A$ be the set of all $x\in G$ such that $f^{\prime}_{V_{1}}(x)$ exists
and $f$ is not strictly differentiable at $x$ along $V_{1}$. By Theorem 4.1,
$A$ is a first category set. Now fix an arbitrary $x\in G\setminus A$ and
suppose that $f$ is Fréchet differentiable at $x$ along $V_{1}$ and along some
topological complement $V_{2}^{x}$ of $V_{1}$. Further identify $X$ with
$\tilde{X}:=V_{1}\times V_{2}^{x}$ by the canonical isomorphism. Then $f$
(considered on $\tilde{X}$) is strictly differentiable at $x$ along $V_{1}$
(considered as a subspace of $\tilde{X}$) and it is Fréchet differentiable
along $V_{2}^{x}$. So Proposition 4.4 implies that $f$ is Fréchet
differentiable at $x$. ∎

##  5\. Lipschitz case

######  Lemma 5.1.

Let $X$, $Y$ be Banach spaces, $G\subset X$ an open set and $f:G\to Y$ an
arbitrary mapping. Let $V_{1}$ be a subspace of $X$ such that $\cal
L(V_{1},Y)$ is separable. Then there exists a $\sigma$-porous set $A\subset G$
such that the following assertion holds for each $x\in G\setminus A$:

$(*)$ Let $f$ be Lipschitz at $x$ and Fréchet differentiable at $x$ along
$V_{1}$, and let $V^{x}$ and $V_{2}^{x}$ be subspaces of $X$ such that
$V^{x}=V_{1}\oplus V_{2}^{x}$ and $f$ is Fréchet differentiable at $x$ along
$V_{2}^{x}$. Then $f$ is Fréchet differentiable at $x$ along $V^{x}$.

###### Proof.

First choose a countable dense subset $\Phi$ of $\cal L(V_{1},Y)$. Further
denote by $A$ the set of all $x\in G$, for which assertion $(*)$ does not
hold.

Then, for each $x\in A$, $f$ is Lipschitz at $x$, is Fréchet differentiable at
$x$ along $V_{1}$, and we can fix spaces $V^{x}$ and $V_{2}^{x}$ such that
$V^{x}=V_{1}\oplus V_{2}^{x}$, $f$ is Fréchet differentiable at $x$ along
$V_{2}^{x}$ and $f$ is not Fréchet differentiable at $x$ along $V^{x}$.

Now consider an arbitrary fixed $x\in A$. We can write any $v\in V^{x}$ in a
unique way as $v=v_{1}^{x}+v_{2}^{x}$ with $v_{1}^{x}\in V_{1}$ and
$v_{2}^{x}\in V_{2}^{x}$ and choose $p\in{\mathbb{N}}$ such that

(5.1) $\max(|v_{1}^{x}|,|v_{2}^{x}|)\leq p|v|\ \ \ \text{for each}\ \ \ v\in
V^{x}.$

Further, since $f$ is Lipschitz at $x$, we can choose $l\in{\mathbb{N}}$ so
big that

(5.2) $|f(y)-f(x)|\leq l|y-x|\ \ \text{whenever}\ \ |y-x|\leq 1/l.$

Define $\Psi^{x}:V^{x}\to Y$ setting
$\Psi^{x}(v)=f^{\prime}_{V_{1}}(x)v_{1}^{x}+f^{\prime}_{V_{2}^{x}}(x)v_{2}^{x},\
\ v\in V^{x}$. Then $\Psi^{x}\in\cal L(V^{x},Y)$. Since
$f^{\prime}_{V^{x}}(x)$ does not exist, we can choose $n\in{\mathbb{N}}$ such
that

(5.3) $\limsup_{v\to 0,v\in V^{x}}\
\frac{|f(x+v)-f(x)-(f^{\prime}_{V_{1}}(x)v_{1}^{x}+f^{\prime}_{V_{2}^{x}}(x)v_{2}^{x})|}{|v|}>\frac{7}{n}.$

Further choose $m\in{\mathbb{N}}$ such that

(5.4) $|f(x+h_{1})-f(x)-f^{\prime}_{V_{1}}(x)h_{1}|\leq\frac{1}{pn}|h_{1}|\ \
\text{whenever}\ \ h_{1}\in V_{1},|h_{1}|<\frac{1}{m}$

and

(5.5)
$|f(x+h_{2})-f(x)-f^{\prime}_{V_{2}^{x}}(x)h_{2}|\leq\frac{1}{pn}|h_{2}|\ \
\text{whenever}\ \ h_{2}\in V_{2}^{x},|h_{2}|<\frac{1}{m}.$

Finally choose $\varphi\in\Phi$ such that

(5.6) $|f^{\prime}_{V_{1}}(x)-\varphi|\leq\frac{1}{pn}.$

For $p,l,n,m\in{\mathbb{N}}$ and $\varphi\in\Phi$, denote by
$A_{p,l,n,m,\varphi}$ the set of all $x\in A$ for which conditions (5.1),
(5.2), (5.3), (5.4), (5.5), (5.6) hold. Then
$A=\bigcup\\{A_{p,l,n,m,\varphi}:\ p,l,n,m\in{\mathbb{N}},\ \varphi\in\Phi\\}$
and thus it is sufficient to prove that, for each fixed
$p,l,n,m\in{\mathbb{N}}$ and $\varphi\in\Phi$, the set
$A^{*}:=A_{p,l,n,m,\varphi}$ is porous.

Suppose to the contrary that $x\in A^{*}$ such that $A^{*}$ is not porous at
$x$ is given. Then, by Definition 2.1, we can choose $0<\delta<1$ such that

(5.7) $B\left(t,\frac{|t-x|}{2pln}\right)\cap A^{*}\neq\emptyset\ \
\text{whenever}\ \ t\in B(x,\delta)\setminus\\{x\\}.$

By (5.3) we can choose $v\in V^{x}$ such that

(5.8) $0<|v|<\frac{\delta}{pm}<1$

and

(5.9)
$D:=|f(x+v)-f(x)-(f^{\prime}_{V_{1}}(x)v_{1}^{x}+f^{\prime}_{V_{2}^{x}}(x)v_{2}^{x})|>\frac{7}{n}|v|.$

By (5.1) and (5.8) we obtain

(5.10) $\max(|v_{1}^{x}|,|v_{2}^{x}|)\leq
p|v|<p\cdot\frac{\delta}{pm}=\frac{\delta}{m}<\frac{1}{m}$

and so (5.5) and (5.1) imply

(5.11)
$|f(x+v_{2}^{x})-f(x)-f^{\prime}_{V_{2}}(x)v_{2}^{x}|\leq\frac{1}{pn}|v_{2}^{x}|\leq\frac{1}{n}|v|.$

By (5.10) we have $|v_{2}^{x}|<\delta$. Further we have $v_{2}^{x}\neq 0$,
since otherwise $v=v_{1}^{x}$ and so (5.4) with (5.10) imply $D\leq(1/n)|v|$
which contradicts (5.9). Thus we can apply (5.7) to $t:=x+v_{2}^{x}$ and
obtain a point $y\in A^{*}$ such that

(5.12)
$|(x+v_{2}^{x})-y|<\frac{|v_{2}^{x}|}{2pln}\leq\frac{p|v|}{2pln}=\frac{|v|}{2ln}\leq\frac{1}{l}$

(we have used also (5.1) and (5.8)). Similarly, since $0<|v|<\delta$ by (5.8),
we obtain by (5.7) a point $z\in A^{*}$ such that

(5.13) $|(x+v)-z|<\frac{|v|}{2pln}\leq\frac{1}{l}.$

Since $y\in A^{*}$ and $z\in A^{*}$, by (5.12), (5.13) and (5.2) we obtain

(5.14) $|f(x+v_{2}^{x})-f(y)|\leq l\cdot|(x+v_{2}^{x})-y|\leq
l\cdot\frac{|v|}{2ln}\leq\frac{1}{n}|v|$

and

(5.15) $|f(x+v)-f(z)|\leq l\cdot|(x+v)-z|\leq
l\cdot\frac{|v|}{2pln}\leq\frac{1}{n}|v|.$

Since $y\in A^{*}$ and $|v_{1}^{x}|<1/m$ by (5.10), using (5.4) and (5.1) we
obtain

(5.16)
$|f(y+v_{1}^{x})-f(y)-f^{\prime}_{V_{1}}(y)v_{1}^{x}|\leq\frac{1}{pn}|v_{1}^{x}|\leq\frac{1}{n}|v|.$

We have $|(x+v)-(y+v_{1}^{x})|=|(x+v_{2}^{x})-y|\leq(2ln)^{-1}|v|$ by (5.12)
and consequently

(5.17)
$|z-(y+v_{1}^{x})|\leq|(x+v)-z|+|(x+v)-(y+v_{1}^{x})|\leq\frac{|v|}{2pln}+\frac{|v|}{2ln}\leq\frac{|v|}{ln}\leq\frac{1}{l}$

by (5.13). Since $z\in A^{*}$, we obtain by (5.17) and (5.2)

$|f(z)-f(y+v_{1}^{x})|\leq l|z-(y+v_{1}^{x})|\leq
l\cdot\frac{|v|}{ln}=\frac{1}{n}|v|,$

which together with (5.15) gives

(5.18)
$|f(x+v)-f(y+v_{1}^{x})|\leq|f(x+v)-f(z)|+|f(z)-f(y+v_{1}^{x})|\leq\frac{2}{n}|v|.$

Since $y\in A^{*}$, we obtain by (5.6) (with $x:=y$)
$|f^{\prime}_{V_{1}}(y)-\varphi|\leq(pn)^{-1}$, which together with (5.6)
gives $|f^{\prime}_{V_{1}}(y)-f^{\prime}_{V_{1}}(x)|\leq 2(pn)^{-1}$. Using
also (5.1), we obtain

(5.19)
$|f^{\prime}_{V_{1}}(y)v_{1}^{x}-f^{\prime}_{V_{1}}(x)v_{1}^{x}|\leq\frac{2}{pn}\cdot|v_{1}^{x}|\leq\frac{2}{n}|v|.$

Using (5.11), we obtain the following upper estimate of $D$ (from (5.9)).

(5.20)
$D:=|f(x+v)-f(x)-(f^{\prime}_{V_{1}}(x)v_{1}^{x}+f^{\prime}_{V_{2}^{x}}(x)v_{2}^{x})|\\\
=|(f(x+v)-f(x+v_{2}^{x})-f^{\prime}_{V_{1}}(x)v_{1}^{x})+(f(x+v_{2}^{x})-f(x)-f^{\prime}_{V_{2}^{x}}(x)v_{2}^{x})|\\\
\leq|f(x+v)-f(x+v_{2}^{x})-f^{\prime}_{V_{1}}(x)v_{1}^{x}|+\frac{1}{n}|v|.$

Further, using (5.16), (5.18), (5.14) and (5.19), we obtain

$|f(x+v)-f(x+v_{2}^{x})-f^{\prime}_{V_{1}}(x)v_{1}^{x}|\\\
\leq|f(y+v_{1}^{x})-f(y)-f^{\prime}_{V_{1}}(y)v_{1}^{x}|+|f(x+v)-f(y+v_{1}^{x})|+|f(y)-f(x+v_{2}^{x})|+|f^{\prime}_{V_{1}}(y)v_{1}^{x}-f^{\prime}_{V_{1}}(x)v_{1}^{x}|\\\
\leq\frac{1}{n}|v|+\frac{2}{n}|v|+\frac{1}{n}|v|+\frac{2}{n}|v|=\frac{6}{n}|v|.$

Thus (5.20) gives $D\leq\frac{7}{n}|v|$ which contradicts (5.9). ∎

By induction, we easily infer from Lemma 5.1 the following analogon of Theorem
4.5.

######  Theorem 5.2.

Let $X_{1},\dots,X_{n}$ and $Y$ be Banach spaces, $X:=X_{1}\times\cdots\times
X_{n}$, $G\subset X$ an open set and $f:G\to Y$ an arbitrary mapping. Let the
spaces $\cal L(X_{1},Y),\dots,\cal L(X_{n-1},Y)$ be separable. Then there
exists a $\sigma$-porous set $A\subset G$ such that, for all $x\in G\setminus
A$, the following implication holds.

$\text{$(*)$\ \ \ $f$ is Lipschitz at $x$ and has all Fr\'{e}chet partial
derivatives at $x$}\\\ \Rightarrow\ \ \text{$f$ is Fr\'{e}chet differentiable
at $x$.}$

###### Proof.

We will proceed by induction on $n$. For $n=2$ the assertion immediately
follows from Lemma 5.1 used for $V_{1}:=X_{1}$ and $V_{2}^{x}:=X_{2}$.

Now suppose that $n\geq 3$ and “the theorem holds for $n:=n-1$”. Further
suppose that $X_{1},\dots,X_{n}$, $Y$, $G$ and $f$ which satisfy the
assumptions of the theorem are given. Since $\cal L(X_{n-1},Y)$ is separable,
we can use Lemma 5.1 with $V_{1}:=X_{n-1}$, and obtain a $\sigma$-porous set
$A_{1}\subset G$ such that, for each $x\in G\setminus A_{1}$, the following
assertion holds:

($\alpha_{1}$) If $f$ is Lipschitz at $x$ and Fréchet differentiable at $x$
along $X_{n-1}$ and along $X_{n}$, then $f$ is Fréchet differentiable at $x$
along the space $\tilde{X}_{n-1}:=X_{n-1}\times X_{n}$.

Now we identify $X$ with $\tilde{X}:=X_{1}\times\cdots\times
X_{n-2}\times\tilde{X}_{n-1}$ by the usual way. By the inductive assumption,
there exists a $\sigma$-porous set $A_{2}\subset G$ such that, for each $x\in
G\setminus A_{2}$, the following assertion holds:

($\alpha_{2}$) If $f$ is Lipschitz at $x$ and Fréchet differentiable at $x$
along all spaces $X_{1},\dots,X_{n-2},\tilde{X}_{n-1}$, then $f$ is Fréchet
differentiable at $x$ (in $\tilde{X}=X$).

Setting $A:=A_{1}\cup A_{2}$, and using for each $x\in G\setminus A$ the
validity of ($\alpha_{1}$) and ($\alpha_{2}$), we obtain that implication
$(*)$ holds for each $x\in G\setminus A$. ∎

As an immedite consequence of Lemma 5.1, we obtain the following analogon of
Proposition 4.8.

######  Proposition 5.3.

Let $X$, $Y$ be Banach spaces, $G\subset X$ an open set and $f:G\to Y$ an
arbitrary mapping. Let $V_{1}$ be a subspace of $X$ such that the space $\cal
L(V_{1},Y)$ is separable. Then there exists a $\sigma$-porous set $A\subset G$
such that, for each $x\in G\setminus A$, the following assertion holds:

$(*)$ If $f$ is Lipschitz at $x$, is Fréchet differentiable at $x$ along
$V_{1}$ and along some topological complement $V_{2}^{x}$ of $V_{1}$, then $f$
is Fréchet differentiable at $x$.

In connection with Propositions 4.8 and 5.3, it is natural to ask, for which
Banach spaces $X$, $Y$ the following statement holds.

$(S)$ Let $f:X\to Y$ be continuous (resp. Lipschitz). Denote by $E_{f}$ the
set of all points $x\in X$ at which there exist a subspace $V^{x}$ of $X$ and
its topological complement $W^{x}$ such that $f$ is Fréchet differentiable at
$x$ both along $V^{x}$ and $W^{x}$ but is not Fréchet differentiable at $x$.
Then $E_{f}$ is a first category set (resp. a $\sigma$-porous set).

If $\dim X<\infty$ and $Y={\mathbb{R}}$, then the “continuous part” of $(S)$
holds; it easily follows from [8]. Further, if $\dim X<\infty$, then the
“Lipschitz part” of $(S)$ easily follows from [14, Theorem 2] (and also from
Corollary 5.6 below).

I conjecture that no part of $(S)$ holds if $X$ is an infinite-dimensional
space, but I do not know any counterexample.

For the weaker version of $(S)$ (which we obtain demanding that $V^{x}$ in the
definition of $E_{f}$ is finite-dimensional), see Corollary 5.6 below. It is
an immediate consequence of Proposition 5.5 below, which can be of an
independent interest. In its proof we use substantially Lemma 5.1 and the
following result which immediately follows from [21, Corollary 3.4] (since
each $\sigma$-directionally porous set is clearly $\sigma$-porous).

######  Proposition 5.4 ([21]).

Let $X$ be a separable Banach space, $Y$ a Banach space, $G\subset X$ an open
set, and $f:G\to Y$ an arbitrary mapping. Then there exists a $\sigma$-porous
set $A\subset G$ such that, for each $x\in G\setminus A$, the following
assertion holds.

$(*)$ If $f$ is Lipschitz at $x$ and the one-sided directional derivative
$f^{\prime}_{+}(x,u)$ exists in all directions $u$ from a set $S_{x}\subset X$
whose linear span is dense in $X$, then $f$ is Gâteaux differentiable at $x$.

######  Proposition 5.5.

Let $X$, $Y$ be separable Banach spaces, $G\subset X$ an open set and $f:G\to
Y$ an arbitrary mapping. Then there exists a $\sigma$-porous set $A\subset G$
such that, for each $x\in G\setminus A$, the following assertion holds.

$(**)$ Let $f$ be Lipschitz at $x$ and Fréchet differentiable at $x$ along a
subspace $M^{x}$ of finite codimension, and let there exist a set
$S_{x}\subset X$ such that $\operatorname{span}S_{x}$ is dense in $X$ and
$f^{\prime}_{+}(x,u)$ exists for all $u\in S_{x}$. Then $f$ is Fréchet
differentiable at $x$.

###### Proof.

For the given $f$, we define $A$ as the set of all $x\in G$ for which
assertion $(**)$ does not hold; we will prove that $A$ is $\sigma$-porous.

Choose a dense countable set $D\subset X$ and, for each point $x\in A$, set

$\text{ $k_{x}:=\inf\\{k\in{\mathbb{N}}\cup\\{0\\}:\ f$\ is Fr\'{e}chet
differentiable at \ $x$}\\\ \ \text{along a subspace $M^{x}$ of codimension
$k\\}$ },$

(where we adopt the convention that $X$ has codimension $0$). Note that, if
$x\in A$, then clearly $1\leq k_{x}<\infty$. Further, for each $x\in A$,
choose a subspace $M^{x}$ of $X$ of codimension $k_{x}$ such that $f$ is
Fréchet differentiable at $x$ along $M^{x}$ and then choose a vector $0\neq
v_{x}\in D\setminus M^{x}$. For each $k\in{\mathbb{N}}$ and $v\in D$, set

$A_{k,v}:=\\{x\in A:\ k_{x}=k,\ v_{x}=v\\}.$

Then $A=\bigcup\\{A_{k,v}:\ k\in{\mathbb{N}},\ v\in D\\}$ and thus it is
sufficient to prove that, for each fixed $k\in{\mathbb{N}}$ and $v\in D$, the
set $A_{k,v}$ is $\sigma$-porous.

To this end, set $V_{1}:=\operatorname{span}\\{v\\}$ and observe that $\cal
L(V_{1},Y)$ is separable. Let $\tilde{A}\subset G$ be a $\sigma$-porous set
which corresponds to $V_{1}$ and $f$ by Lemma 5.1; i.e.

(5.21) assertion $(*)$ from Lemma 5.1 holds for each $x\in
G\setminus\tilde{A}$.

Further, let $\tilde{\tilde{A}}$ be a $\sigma$-porous set which corresponds to
$f$ by Proposition 5.4.

Now it is sufficient to prove that
$A_{k,v}\subset\tilde{A}\cup\tilde{\tilde{A}}$. To prove this inclusion,
suppose to the contrary that there exists a point $x\in
A_{k,v}\setminus(\tilde{A}\cup\tilde{\tilde{A}})$. Set $V_{2}^{x}:=M^{x}$ and
$V^{x}:=V_{1}+V_{2}^{x}$. We know that $V_{2}^{x}$ has codimension $1\leq
k_{x}<\infty$ and $V_{1}\cap V_{2}^{x}=\\{0\\}$. Consequently $V^{x}$ is
closed, $V^{x}=V_{1}\oplus V_{2}^{x}$ (see, e.g., [6, Exercise 5.27 and
Proposition 5.3]) and it is easy to see that $V^{x}$ has codimension
$k_{x}-1$. Since $x\in A$, we have that the assumptions of assertion $(**)$
hold and so also the assumptions of assertion $(*)$ of Proposition 5.4 hold.
So, since $x\in G\setminus\tilde{\tilde{A}}$, by the choice of
$\tilde{\tilde{A}}$ we obtain that $f$ is Gâteaux differentiable at $x$, and
consequently $f$ is Fréchet differentiable along $V_{1}$ at $x$. Since $f$ is
Fréchet differentiable at $x$ along the space $V_{2}^{x}=M^{x}$ and $x\in
G\setminus\tilde{A}$, we can use (5.21) and obtain that $f$ is Fréchet
differentiable along the space $V^{x}$ of codimension $k_{x}-1$ which
contradicts the definition of $k_{x}$. ∎

As immediate consequences, we obtain the following results.

######  Corollary 5.6.

Let $X$, $Y$ be separable Banach spaces, $G\subset X$ an open set and $f:G\to
Y$ an arbitrary mapping. Then there exists a $\sigma$-porous set $A\subset G$
such that the following assertion holds.

$(*)$ Let $f$ be Lipschitz at $x$ and let there exist a finite-dimensional
subspace $V^{x}$ of $X$ and its topological complement $W^{x}$ of $X$ such
that $f$ is Fréchet differentiable at $x$ along $V^{x}$ and $W^{x}$. Then $f$
is Fréchet differentiable at $x$.

######  Corollary 5.7.

Let $X$, $Y$ be separable Banach spaces, $G\subset X$ an open set and $f:G\to
Y$ a Lipschitz mapping. Then the set $A$ of all points $x\in G$ at which $f$
is Gâteaux differentiable, is Fréchet differentiable along a closed subspace
of a finite codimension but is not Fréchet differentiable, is $\sigma$-porous.

######  Remark 5.8.

The set $A$ from Corollary 5.7 can be nonempty. To show this, set
$X:=\ell_{2}$, $Y:={\mathbb{R}}$ and denote by $e_{n},\ n\in{\mathbb{N}},$ the
canonical basis vectors in $\ell_{2}$. Set

$p_{n}:=\frac{1}{n}(e_{1}+e_{n}),\ n\geq 2,\
F:=X\setminus\bigcup_{n=2}^{\infty}B(p_{n},\frac{1}{2n})\ \ \text{and}\ \
f(x):=\operatorname{dist}(x,F),\ x\in X.$

Then $f$ is Lipschitz on $G:=X$ and it is easy to check that $0\in A$, where
$A$ is as in Corollary 5.7.

##  6\. Results proved by the separable reduction method

In this section, we will use the well-known method of separable reduction.
Namely, we will first prove “the separable case” and from it we will obtain
the “nonseparable case” using some known results which say that some notions
are “separably determined in the sense of rich families”. For the following
notion of a “rich family” see e.g. [10, p. 37] or [3].

######  Definition 6.1.

Let $X$ be a Banach space. A family $\cal F$ of separable subspaces of $X$ is
called a rich family if:

  1. (R1)

If $V_{i}\in\cal F$ ($i\in{\mathbb{N}}$) and $V_{1}\subset V_{2}\subset\dots$,
then $\overline{\bigcup\\{V_{n}:n\in{\mathbb{N}}\\}}\in\cal F.$

  2. (R2)

For each separable subspace $V_{0}$ of $X$ there exists $V\in\cal F$ such that
$V_{0}\subset V$.

A basic (easy) fact (see e.g. [10, Proposition 3.6.2]) concerning rich
families is the following.

######  Lemma 6.2.

Let $X$ be a Banach space and let $\\{\cal F_{n}:n\in{\mathbb{N}}\\}$ be rich
families of separable subspaces of $X$. Then $\cal F:=\bigcap\\{\cal
F_{n}:n\in{\mathbb{N}}\\}$ is also a rich family of separable subspaces of
$X$.

We will use also the following simple fact which is a reformulation of [20,
Lemma 4.4].

######  Lemma 6.3.

Let $X_{1},\dots,X_{n}$ be Banach spaces and $X:=X_{1}\times\cdots\times
X_{n}$. Let ${\cal F}_{k}$ be a rich family of separable subspaces of $X_{k}$,
$1\leq k\leq n$. Then

${\cal F}:=\\{V_{1}\times\cdots\times V_{n}:\ V_{k}\in{\cal F}_{k},\ 1\leq
k\leq n\\}$

is a rich family of separable subspaces of $X$.

Much more difficult is the following result which says that Fréchet
differentiability at a point is “separably determined in the sense of rich
families”.

######  Theorem 6.4 ([10, Theorem 3.6.10]).

Let $X$, $Y$ be Banach spaces, $G\subset X$ an open set and $f\colon G\to Y$ a
mapping. Then there exists a rich family $\cal F$ of separable subspaces of
$X$ such that for every $V\in\cal F$, $f$ is Fréchet differentiable (with
respect to $X$) at every $x\in V\cap G$, at which its restriction to $V\cap G$
is Fréchet differentiable (with respect to $V$).

(In fact, [10, Theorem 3.6.10] is formulated for $G=X$ only, but if we apply
this formally weaker theorem to any extension $\tilde{f}$ of $f$ to $X$, we
obtain the assertion of Theorem 6.4.)

The following result on “separable determination of first category sets and
$\sigma$-upper porous sets” were first proved in [2] and [4] “in the sense of
suitable models” and then transferred to the following result in [3].

######  Theorem 6.5 ([3, Corollary 5]).

Let $X$ be a Banach space and $A\subset X$ a Souslin set. Then there exists a
rich family $\cal F$ of separable subspaces of $X$ such that for every
$V\in\cal F$ we have

$\displaystyle(i)\ \ \ \text{$A$ is of the first category in $X$}$
$\displaystyle\ \ \Longleftrightarrow\ \ \text{$A\cap V$ is of the first
category in $V$},$ $\displaystyle(ii)\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \text{$A$
is $\sigma$-porous in $X$}$ $\displaystyle\ \ \Longleftrightarrow\ \
\text{$A\cap V$ is $\sigma$-porous in $V$}.$

Recall that every Borel set in $X$ is Souslin. We will use also the following
known fact.

######  Theorem 6.6 ([20, Theorem 4.7]).

Let $X$, $Y$ be Banach spaces, $G\subset X$ an open set, and let $f:G\to Y$ be
an arbitrary mapping. Then the following conditions are equivalent.

  1. (i)

$f$ is generically Fréchet differentiable.

  2. (ii)

There exists a rich family ${\cal F}$ of separable subspaces of $X$ such that
$f|_{V\cap G}$ is generically Fréchet differentiable (with respect to $V$) on
$V\cap G$ for each $V\in{\cal F}$.

Now we will prove, using Theorem 4.5, Theorem 5.2 and the method of separable
reduction, the following result which generalizes Theorem 4.5 and partly
generalizes Theorem 5.2.

######  Theorem 6.7.

Let $X_{1},\dots,X_{n}$ be Banach spaces, $X:=X_{1}\times\cdots\times X_{n}$
and $G\subset X$ an open set. Suppose that each space
${\mathcal{L}}(\tilde{X}_{i},\tilde{Y})$ is separable whenever $\tilde{X}_{i}$
is a separable subspace of $X_{i}$, $i=1,\dots,n-1$, and $\tilde{Y}$ is a
separable subspace of $Y$. Let $f:G\to Y$ be a continuous (resp. Lipschitz)
mapping.

Then there exists a first category (resp. $\sigma$-porous) set $A\subset G$
such that, for each $x\in G\setminus A$, the following implication holds.

$f$ has all Fréchet partial derivatives at $x$ $\Rightarrow$ $f$ is Fréchet
differentiable at $x$.

###### Proof.

We will prove the “continuous part” and the “Lipschitz part” of the theorem
together.

In the first step of the proof, we will prove the theorem in the special case
when all spaces $X_{1},\dots,X_{n}$ are separable. In this case observe that
the space $\tilde{Y}:=\overline{\operatorname{span}f(G)}$ is separable and
therefore the spaces
${\mathcal{L}}(X_{1},\tilde{Y}),\dots,{\mathcal{L}}(X_{n-1},\tilde{Y})$ are
separable by the assumptions of the theorem. Further observe that $f:G\to Y$
is partially Fréchet differentiable (resp. Fréchet differentiable) at $x\in G$
if and only if $f:G\to\tilde{Y}$ has this property. So, if $f$ is continuous
(resp. Lipschitz), then the existence of a first category (resp.
$\sigma$-porous) set $A$ from the conclusion of the theorem follows from
Theorem 4.5 (resp. Theorem 5.2).

In the second step, we will prove the general case using the method of
separable reduction. Denote by $A$ the set of all $x\in G$ at which all
partial Fréchet derivatives $f^{\prime}_{i}(x),\ i=1,\dots,n,$ exist but $f$
is not Fréchet differentiable. Our aim is to prove that

(6.1) $A$ is a first category set if $f$ is continuous

and

(6.2) $A$ is a $\sigma$-porous set if $f$ is Lipschitz.

Notice that (3.1) and Proposition 3.3 give that $A$ is a Borel set (and hence
a Souslin set) in $X$. Thus Theorem 6.5 implies that there exists a rich
family $\cal F_{1}$ of separable subspaces of $X$ such that for every
$V\in\cal F_{1}$ we have that

(6.3) $A$ is of the first category in $X$ whenever $A\cap V$ is of the first
category in $V$

and

(6.4) $A$ is of $\sigma$-porous in $X$ whenever $A\cap V$ is $\sigma$-porous
in $V$.

By Lemma 6.3, the family

${\cal F}_{2}:=\\{V_{1}\times\cdots\times V_{n}:\ V_{k}\ \text{is a separable
subspace of}\ X_{k},\ 1\leq k\leq n\\}$

is a rich family of separable subspaces of $X$.

By Theorem 6.4 there exists a rich family $\cal F_{3}$ of separable subspaces
of $X$ such that for every $V\in\cal F_{3}$, $f$ is Fréchet differentiable
(with respect to $X$) at every $x\in V\cap G$ at which its restriction to
$V\cap G$ is Fréchet differentiable (with respect to $V$).

Now, by Lemma 6.2, ${\cal F}:={\cal F}_{1}\cap{\cal F}_{2}\cap\mathcal{F}_{3}$
is a rich family of separable subspaces of $X$.

Choose an arbitrary $V\in{\cal F}$; since $V\in{\cal F}_{2}$, we have
$V=V_{1}\times\cdots\times V_{n}$ where $V_{k}$ is a separable subspace of
$X_{k}$, $1\leq k\leq n$.

If $A\cap V\neq\emptyset$, set $g:=f|_{(V\cap G)}$. Using the definition of
$A$, we easily see that $g_{i}^{\prime}(x)=g^{\prime}_{V_{i}}(x)$,
$i=1,\dots,n$, exist for each $x\in A\cap V$. Since $f$ is Fréchet
nondifferentiable at each $x\in A\cap V$ and $V\in{\cal F}_{3}$, we conclude
that $g$ is Fréchet nondifferentiable at each $x\in A\cap V$. Further, by the
assumptions of the theorem, the space ${\mathcal{L}}(\tilde{X}_{i},\tilde{Y})$
is separable whenever $\tilde{X}_{i}$ is a separable subspace of $V_{i}$,
$i=1,\dots,n-1$, and $\tilde{Y}$ is a separable subspace of $Y$. So, using for
$g$ on $G\cap V\subset V=V_{1}\times\dots\times V_{n}$ the special case of the
theorem proved in the first step of the proof, we obtain that $A\cap V$ is a
first category (resp. $\sigma$-porous) set in $V$ if $f$ is continuous (resp.
Lipschitz). Therefore, since $V\in{\cal F}_{1}$, we obtain that both (6.3) and
(6.4) hold. ∎

Using (2.1), we see that Theorem 6.7 has the following interesting
consequence.

######  Corollary 6.8.

Let $X_{1},\dots,X_{n}$ be Banach spaces such that $X_{1},\dots,X_{n-1}$ are
Asplund spaces, and let $G\subset X:=X_{1}\times\cdots\times X_{n}$ be an open
set. Let $f$ be a continuous (resp. Lipschitz) real function on $G$.

Then there exists a first category (resp. $\sigma$-porous) set $A\subset G$
such that, for each $x\in G\setminus A$, the following implication holds.

$f$ has all Fréchet partial derivatives at $x$ $\Rightarrow$ $f$ is Fréchet
differentiable at $x$.

######  Remark 6.9.

  1. (i)

There are also other concrete consequences of Theorem 6.7. For example, using
facts from Remark 4.2, it is easy to show that Theorem 6.7 can be used if each
$X_{i}$, $i=1,\dots,n-1$, is a Hilbert space (resp. a subspace of
$c_{0}(\Gamma)$) and $Y=\ell_{q}(\Gamma)$, $1\leq q<2$, (resp. $Y$ has the
Radon-Nikodým property).

  2. (ii)

Theorem 6.7 can be further slightly generalized to a more complicated general
theorem (working with “rich families of $\tilde{X}_{i}$”) which has further
concrete applications.

  3. (iii)

I do not know any example excluding the possibility that the assumptions
concerning Banach spaces $X_{1}$,…,$X_{n}$, $Y$ can be omitted in Theorem 6.7.
From this reason the observations in (i) and (ii) are mentioned without any
details.

By the separable reduction method, we prove also the following result on
continuous functions whose all partial functions are DC (recall that a
function on a Banach space is called DC if it is the difference of two
continuous convex functions).

######  Proposition 6.10.

Let $X_{1},\dots,X_{n}$ be Asplund spaces and $f$ a continuous real function
on $X:=X_{1}\times\dots\times X_{n}$. Let each partial function

$f(x_{1},\dots,x_{i-1},\cdot,x_{i+1},\dots,x_{n}),\ \ \ 1\leq i\leq n,$

is DC on $X_{i}$. Then $f$ is generically Fréchet differentiable.

###### Proof.

By Lemma 6.3, the family

${\cal F}:=\\{V_{1}\times\cdots\times V_{n}:\ V_{k}\ \text{is a separable
subspace of}\ X_{k},\ 1\leq k\leq n\\}$

is a rich family of separable subspaces of $X$. Choose an arbitrary
$V=V_{1}\times\cdots\times V_{n}\in{\cal F}$ and set $g:=f|_{V}$. Then each
partial function

$g(v_{1},\dots,v_{i-1},\cdot,v_{i+1},\dots,v_{n}),\ \ \ 1\leq i\leq n,$

is DC on the Asplund space (see (2.1)) $V_{i}$ and consequently is generically
differentiable on $V_{i}$. Since the spaces $V_{1}^{*},\dots,V_{n-1}^{*}$ are
separable, we can use Proposition 4.7 and obtain that $g$ is generically
differentiable. Therefore $f$ is generically differentiable by Theorem 6.6. ∎

######  Remark 6.11.

  1. (i)

In the case of a locally Lipschitz $f$, Proposition 6.10 immediately follows
from [20, Corollary 8.1] (and also from [20, Corollary 8.4]).

  2. (ii)

The proof of Proposition 6.10 works if we weaken the assumption that each
partial function of $f$ is DC to the assumption that it is a difference of two
approximately convex functions. Thus we obtain that, in [20, Corollary 8.1],
it is possible to suppose the continuity of $f$ instead of the local
Lipschitzness of $f$.

  3. (iii)

Proposition 6.10 could be proved quite similarly as Theorem 6.7, but the
present proof based on Theorem 6.6 is shorter.

Acknowledgements. I thank Ondřej Kalenda who suggested the generalization of
Corollary 6.8 to Theorem 6.7.

## References

  * [1] Bessis D. N., Clarke F. H., Partial subdifferentials, derivates and Rademacher’s theorem, Trans. Amer. Math. Soc. 351 (1999), 2899–2926.
  * [2] Cúth M., Separable reduction theorems by the method of elementary submodels, Fundam. Math. 219 (2012), 191–222.
  * [3] Cúth M., Separable determination in Banach spaces, Fund. Math. 243 (2018), 9–27.
  * [4] Cúth M., Rmoutil M., $\sigma$-porosity is separably determined, Czechoslovak Math. J. 63(138) (2013), 219–234.
  * [5] Engelking R., General topology, 2nd ed., Sigma Series in Pure Mathematics, 6, Heldermann Verlag, Berlin, 1989.
  * [6] Fabian M., Habala P., Hájek P., Montesinos Santalucía V., Pelant J., Zizler V., Functional analysis and infinite-dimensional geometry, CMS Books in Mathematics, 8, Springer-Verlag, New York, 2001.
  * [7] Gorlenko S. V., Certain differential properties of real functions, Ukrain. Mat. Z. 29 (1977), 246–249 (Russian); translation in Ukrainian Math. J. 29 (1977), 185–187.
  * [8] Ilmuradov D. D., On differential properties of real functions, Ukrain. Mat. Z. 46 (1994), 842–848 (Russian); translation in Ukrainian Math. J. 46 (1994), 922–928.
  * [9] Kechris A. S., Classical descriptive set theory, Graduate Texts in Mathematics, 156, Springer-Verlag, New York, 1995.
  * [10] Lindenstrauss J., Preiss D., Tišer J., Fréchet differentiability of Lipschitz maps and porous sets in Banach spaces, Princeton University Press, Princeton, 2012\.
  * [11] Lau K. S., Weil C. E., Differentiability via directional derivatives, Proc. Amer. Math. Soc. 70 (1978), 11–17.
  * [12] Mykhaylyuk V., Plichko A., On a problem of Mazur from ”the Scottish Book” concerning second partial derivatives, Colloq. Math. 141 (2015), 175–182.
  * [13] Penot J.-P., Calculus without derivatives, Graduate Texts in Mathematics, 266, Springer, New York, 2013.
  * [14] Preiss D., Zajíček L., Directional derivatives of Lipschitz functions, Israel J. Math. 125 (2001), 1–27.
  * [15] Saint-Raymond J., Sur les fonctions munies de dérivées partielles, Bull. Sci. Math. (2) 103 (1979), 375–378.
  * [16] Stepanoff W., Sur les conditions de l’existence de la differentielle totale, Mat. Sb. 32 (1925), 511–526.
  * [17] Veselý L., Zajíček L., On differentiability of convex operators, J. Math. Anal. Appl. 402 (2013), 12–22.
  * [18] Zajíček L., Fréchet differentiability, strict differentiability and subdifferentiability, Czechoslovak Math. J. 41 (1991), 471–489.
  * [19] Zajíček L., On $\sigma$-porous sets in abstract spaces, Abstract Appl. Analysis 2005 (2005), 509–534.
  * [20] Zajíček L., Generic Fréchet differentiability on Asplund spaces via a.e. strict differentiability on many lines, J. Convex Analysis 19 (2012), 23–48.
  * [21] Zajíček L., Gâteaux and Hadamard differentiability via directional differentiability, J. Convex Analysis 21 (2014), 703–713.
  * [22] Zajíček L., Hadamard differentiability via Gâteaux differentiability, Proc. Amer. Math. Soc. 143 (2015), 279–288.


# Grammatical gender in Swedish is predictable
using recurrent neural networks

Edvin Listo Zec
RISE Research Institutes of Sweden
<EMAIL_ADDRESS>
&Olof Mogren
RISE Research Institutes of Sweden
<EMAIL_ADDRESS>

###### Abstract

The grammatical gender of Swedish nouns is a mystery. While there are few
rules that can indicate the gender with some certainty, it does in general not
depend on either meaning or the structure of the word.

In this paper we demonstrate the surprising fact that grammatical gender for
Swedish nouns can be predicted with high accuracy using a recurrent neural
network (RNN) working on the raw character sequence of the word, without using
any contextual information.

##  1 Introduction

There are two different grammatical genders for Swedish nouns, common (utrum)
and neuter (neutrum) Rebbe-Gullberg-Ivan (1954). Swedish used to have more
than two genders; however the masculine and the feminine later merged into the
common gender and modern Swedish makes no difference between them. The
grammatical gender affects the indefinite article (en boll – ett bord;
English: a ball – a table), as well as the definite article and the definite
suffix. The Institute for Language and Folklore, a Swedish government agency,
states that there is no unequivocal way of determining the grammatical gender
of a word för språk och folkminnen (2019). Further the Swedish Academy states
that the grammatical gender in Swedish is a lexical property which is to a
large extent not dependent on neither meaning nor the structure of the noun
Teleman et al. (1999). There are exceptions: living things are generally utrum
(the more common class), and there are a few specific suffixes that are
reasonably good predictors. However, determining the gender of a word is
generally considered a major difficulty for non-native Swedish speakers
learning the language, and it is an open question how predictable it is.

In this paper, we investigate the predictability of grammatical gender for
Swedish nouns without any context words. We train a recurrent neural network
(RNN) model to predict the property given only the raw character sequence as
input. The model quite easily learns the necessary patterns and achieves a 95%
accuracy on the test set, which should be compared to the $71\%$ majority
class baseline.

##  2 Prediction of grammatical gender using recurrent neural networks

A recurrent neural network (RNN) is a feed forward artificial neural network
that can model a sequence of arbitrary length, using weight sharing between
each position in the sequence. In the basic RNN variant, the transition
function at time $t$ is a linear transformation of the hidden state
$\mathbf{h_{t-1}}$ and the input, followed by a point-wise non-linearity:

$\mathbf{h}_{t}=\mbox{tanh}(W\mathbf{x}_{t}+U\mathbf{h}_{t-1}+\mathbf{b}),$

where $W$ and $U$ are weight matrices, $\mathbf{b}$ is a bias vector, and tanh
is the selected nonlinearity. $W$, $U$, and $\mathbf{b}$ are typically trained
using some variant of stochastic gradient descent (SGD).

Basic RNNs struggle with learning long-range dependencies and suffer from the
vanishing gradient problem. This makes them difficult to train Hochreiter
(1998); Bengio et al. (1994), and has motivated the development of the Long
short term memory (LSTM) architecture Hochreiter and Schmidhuber (1997), that
to some extent solves these shortcomings. An LSTM is gated variant of the RNN
where the cell at each step $t$ contains an internal memory vector
$\mathbf{c}_{t}$, and three gates controlling what parts of the internal
memory will be kept (the forget gate $\mathbf{f}_{t}$), what parts of the
input that will be stored in the internal memory (the input gate
$\mathbf{i}_{t}$), as well as what will be included in the output (the output
gate $\mathbf{o}_{t}$). Later, the gated recurrent network architecture (GRU)
Cho et al. (2014) was proposed, which only has two different gates.

Figure 1: The proposed character based RNN model used in this work. The output
layer is a fully connected layer with one output unit and sigmoid actication
function.

In this paper, we propose a neural network model based on recurrent neural
networks (see figure 1), that is trained to predict the grammatical gender of
Swedish nouns. The model has a fully connected output layer with one unit and
a sigmoid activation function.

##  3 Experimental setup

Three main network architectures were set up and trained on the same dataset
and then compared on the same test set. All words in the dataset were
vectorised by transforming each character of the words into a unique integer.
Then each vectorised word was padded with zeros at the end matching the length
of the longest word, in our case 19.

We train, evaluate and compare three different models: a feed-forward network,
an LSTM network and a GRU network. The first model consists of an embedding
layer which takes as input each character sequence describing a word, and
outputs an embedding of dimension 60. It is then followed by two dense layers
with 128 and 1 unit(s) respectively for each layer, with a $\tanh$ activation
in the hidden layer and a sigmoid in the output layer. The second model is
composed of the same embedding layer, followed by an RNN with a GRU with 64
units and a $\tanh$ activation. The RNN outputs a whole sequence into a dense
layer with one unit and the sigmoid as activation. The third model is the same
as the second, with the difference being that it uses LSTM units instead of
GRUs.

###  3.1 Hyperparameters

All models were trained with the Adam optimiser with the same hyperparameter
settings as that of the paper Kingma and Ba (2014), where they used a learning
rate $\alpha=0.001$, decay rates $\beta_{1}=0.9$ and $\beta_{2}=0.999$, and
$\epsilon=10^{-8}$.

###  3.2 Dataset

The dataset consists of 88480 Swedish nouns acquired from SALDO Borin et al.
(2013)111Språkbanken (the Swedish Language Bank):
https://spraakbanken.gu.se/swe/resurs/saldo, labelled with grammatical gender.
According to the Swedish Academy about two thirds of Swedish nouns are utrum
and one third neutrum Teleman et al. (1999). In our dataset, $71\%$ of the
words belong to the utrum class, and $29\%$ to the neutrum. Further, the
Swedish Academy notes that there are some common suffixes that strongly
correlate with the grammatical gender. For example, words ending with ing,
tion, het and ist are mostly associated with utrum and words ending with eri,
skop and gram are mostly associated with neutrum. In table 1 suffixes
correlated with a certain class a listed.

As it is well known that the definite suffix indicates the grammatical gender,
and the signal can be picked up from plural suffixes too, only singular
indefinite nouns are included in the dataset.

Suffix | Number of occurrences | Fraction utrum
---|---|---
het | 8180 | 0.9999
tion | 1194 | 0.9992
ist | 509 | 0.9980
ing | 9384 | 0.9960
are | 3062 | 0.9941
skop | 35 | 0.1429
eri | 416 | 0.0385
gram | 109 | 0.0183
ande | 8931 | 0.0076
Table 1: Occurrences of words in the dataset and the fraction of the utrum
class.

Correctly predicted word | Correct label
---|---
aortaruptur | U
utrikesfråga | U
elektroakustik | U
överhopande | N
aschle | N
utvecklingsstöd | N

Incorrectly predicted word | Correct label
---|---
hut | U
lyster | U
nukleon | U
drogsug | N
zeugma | N
mattesnille | N

Table 2: Randomly selected words from the test set that the proposed model
predicted correctly (left), and incorrectly (right), respectively. U=utrum,
N=neutrum.

###  3.3 Training

A batch size of 32 was used in our experiment for all models. Furthermore, all
models were trained until no change was observed in the validation loss for 50
epochs.

###  3.4 Evaluation

The models were evaluated using prediction accuracy. The dataset was split up
into $60\%$ training, $20\%$ validation and $20\%$ test. A second set was
created from the test set, where we removed all words with the common ing,
tion, het, ist and eri suffixes. We can thus see how well a model learns
different words and how it performs without common suffixes to its disposal.
In the evaluation, the prediction accuracy is reported for the test set.

##  4 Results

In table 3 the results from the three different neural networks are presented,
with the best performing network in bold. The best model was the LSTM, with a
classification accuracy of $95.15\%$ on the test set and $92.68\%$ on the test
set with removed words. In table 4 precision, recall and F1-score for the LSTM
model are shown.

In table 2, we present a random selection of words for which the proposed
model correctly predicts the gender, and a random selection of words where it
fails.

Figure 2 depicts a t-SNE visualisation of the output from the LSTM layer on
the two different test datasets.

| Test set | Test set with removed words
---|---|---
Dense | 0.8548 | 0.8577
GRU | 0.9492 | 0.9255
LSTM | 0.9515 | 0.9268
CNN | 0.9551 | 0.9324
Table 3: The accuracy for different test sets. | Precision | Recall | F1-score
---|---|---|---
Neutrum | 0.90 (0.85) | 0.93 (0.89) | 0.92 (0.87)
Utrum | 0.97 (0.96) | 0.96 (0.94) | 0.97 (0.95)
Table 4: Precision, recall and F1-score for the LSTM model on the test set.
Test set with removed suffixes in parenthesis.

(a) t-SNE visualisation of the test set.

(b) t-SNE visualisation of the test set with removed suffixes.

Figure 2: t-SNE on the output from the LSTM layer with true labels and
incorrect model predictions.

##  5 Related work

Character-based recurrent neural networks have demonstrated effective for
identifying word forms and inflectional patterns. In morphological relational
reasoning, they can be used to generate a word in a form as demonstrated by
another inflected word Mogren and Johansson (2017). They are also the state-
of-the-art in morphological inflection and reinflection Cotterell et al.
(2016); Kann and Schütze (2016). In the current work, we instead focus on a
single property that we want to infer from the raw character sequence of a
given word. This means that we have a smaller model and a shorter path from
the learning objective to backpropagate through. The patterns that we do learn
can however be similar to those in the works mentioned above.

Nastase and Popescu (2009) trained a kernel ridge regression (KRR) model to
predict the gender of nouns in German and Romanian to a high precision. It is
however well known that Romanian has phonological patterns that indicate
gender, while German is more similar to Swedish, in that there are a number of
rules that have exceptions. Both German and Romanian have three genders each,
while Swedish has only two.

Desai et al. (2015) used a morphological analyzer to extract features to train
a classifier for the grammatical gender in Konkani. Dinu et al. (2012) trained
machine (SVM) using ngram features and finds support for Romanian being a two-
gender language, rather than the normally considered three genders.

Cucerzan and Yarowsky (2003) trained trie models that depend on morphological
analysis (suffix cues) and contextual information from a corpus to determine
the gender of a word. The model is bootstrapped with a small list of tagged
words. The system was evaluated on French, Romanian, Slovene, Spanish, and
Swedish. They conclude that a one-character suffix is too short to be a good
predictor for Swedish. They end up achieving good prediction accuracy.
However, they learn the gender based on the context of nouns in a corpus,
which is a different task than the one we are solving.

Basirat and Tang (2018) use word embeddings to predict the grammatical gender
of Swedish nouns. Here, context of nouns is also used and they are able to
achieve a high classification accuracy of $93.7\%$.

In the current work, we are not depending on any such corpus; the proposed
model works solely using the raw character stream of the word. Thus, we use
much less information as compared to context based methods.

##  6 Discussion and conclusions

In this work, we have seen that a simple RNN-based model is able to predict
grammatical gender in Swedish nouns, a lexical property that is considered
difficult and does not adhere to rules other than in a few special cases. In
contrast, our results show that we can learn the necessary patterns only by
looking at the sequence of characters in a given word, without using any
context words.

## References

  * Basirat and Tang (2018) Ali Basirat and Marc Tang. 2018.  Linguistic information in word embeddings.  In _International Conference on Agents and Artificial Intelligence_ , pages 492–513. Springer.
  * Bengio et al. (1994) Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994.  Learning long-term dependencies with gradient descent is difficult.  _Neural Networks, IEEE Transactions on_ , 5(2):157–166.
  * Borin et al. (2013) Lars Borin, Markus Forsberg, and Lennart Lönngren. 2013.  SALDO: a touch of yin to WordNet’s yang.  _Language Resources and Evaluation_ , 47(4):1191–1211.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using rnn encoder-decoder for statistical machine translation. In _EMNLP_ , pages 1724–1734. ACL.
  * Cotterell et al. (2016) Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016.  The SIGMORPHON 2016 shared task – morphological reinflection.  In _Proceedings of the 14th Annual SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology_ , pages 10–22.
  * Cucerzan and Yarowsky (2003) Silviu Cucerzan and David Yarowsky. 2003.  Minimally supervised induction of grammatical gender.  In _Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1_ , pages 40–47. Association for Computational Linguistics.
  * Desai et al. (2015) Shilpa Desai, Jyoti Pawar, and Pushpak Bhattacharyya. 2015.  Staged approach for grammatical gender identification of nouns using association rule mining and classification.  _Research in Computing Science_ , 90:359–371.
  * Dinu et al. (2012) Liviu P Dinu, Vlad Niculae, and Octavia-Maria Sulea. 2012.  The romanian neuter examined through a two-gender n-gram classification system.  In _LREC_ , pages 907–910.
  * för språk och folkminnen (2019) Institutet för språk och folkminnen. 2019.  Språkrådgivning.
  * Hochreiter (1998) Sepp Hochreiter. 1998.  The vanishing gradient problem during learning recurrent neural nets and problem solutions.  _International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems_ , 6(02):107–116.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780.
  * Kann and Schütze (2016) Katharina Kann and Hinrich Schütze. 2016.  MED: The LMU system for the SIGMORPHON 2016 shared task on morphological reinflection.  In _Proceedings of the 14th Annual SIGMORPHON Workshop on Computational Research in Phonetics, Phonology, and Morphology_ , pages 62–70.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Mogren and Johansson (2017) Olof Mogren and Richard Johansson. 2017.  Character-based recurrent neural networks for morphological relational reasoning.  In _Proceedings of the First Workshop on Subword and Character Level Models in NLP_ , pages 57–63.
  * Nastase and Popescu (2009) Vivi Nastase and Marius Popescu. 2009.  What’s in a name?: in some languages, grammatical gender.  In _Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3-Volume 3_ , pages 1368–1377. Association for Computational Linguistics.
  * Rebbe-Gullberg-Ivan (1954) Rebbe-Gullberg-Ivan. 1954.  _Svensk språklära_.  Svenska Bokförlaget.
  * Teleman et al. (1999) Ulf Teleman, Staffan Hellberg, and Erik Andersson. 1999.  _Svenska akademiens grammatik_.  Svenska akademien.

##  Appendix A Appendix

Correctly predicted word | Correct label
---|---
aortaruptur | 1
utrikesfråga | 1
nappa | 1
djurvänlighet | 1
silverklang | 1
ukrainska | 1
förlöpare | 1
befästande | 0
förhydande | 0
projekterbarhet | 1
elektroakustik | 1
överhopande | 0
aschle | 0
utvecklingsstöd | 0
amoralitet | 1
fostrare | 1
catering | 1
omotsäglighet | 1
ovilja | 1
rättssal | 1
utpekning | 1
otrohetsskandal | 1
kyrkostadga | 1
dagboksskrivande | 0
överlackerbarhet | 1
pressveck | 0
vattenled | 1
urklipp | 0
rymd | 1
pressgurka | 1
altarkärl | 0
glav | 1
stjärngranne | 1
bordsskiva | 1
desinficeringsmedel | 0
svedenborgare | 1
krigsdans | 1
upphävande | 0
meningsmotståndare | 1
oapplåderbarhet | 1
opromulgerbarhet | 1
stoftpartikel | 1
ignorerbarhet | 1
sammetsblomster | 0
stångryttare | 1
rafräschissör | 1
byrå | 1
naturlighet | 1
porrtidning | 1
hjässa | 1
Table 5: Randomly selected words that the proposed model predicted correct. Incorrectly predicted word | Correct label
---|---
hut | 1
bomullstuss | 1
lyster | 1
drogsug | 0
nukleon | 1
zeugma | 0
självdisciplin | 1
tjänstenit | 0
mörkalv | 1
avföringstransplantat | 0
particip | 0
älgko | 1
guckusko | 1
vedlår | 1
decimalkomma | 0
hälsodryck | 1
luftled | 1
synkop | 1
finne | 1
näst | 0
underhållskolonn | 1
jättetyfon | 1
tågstopp | 0
stråt | 1
riboflavin | 1
maraton | 0
fortuna | 0
flusspat | 1
barr | 1
mattesnille | 0
berättarjag | 0
tussilago | 1
eurodollar | 1
perm | 0
kundnöjdhetsenkät | 1
chassi | 0
gen | 1
björnbindsle | 0
kartotek | 0
cyklop | 1
akantom | 0
dego | 1
cocktailparty | 0
sudd | 0
dödsur | 0
schapp | 0
mc-gäng | 0
stins | 1
bukfett | 0
arbetshandikapp | 0
Table 6: Randomly selected words that the proposed model predicted wrong.