# CRITERION OF REALITY OF ZEROS IN A POLYNOMIAL SEQUENCE SATISFYING A THREE-
TERM RECURRENCE RELATION

Innocent Ndikubwayo  Department of Mathematics, Stockholm University, SE-106
91 Stockholm, Sweden<EMAIL_ADDRESS>Department of Mathematics, Makerere
University, 7062 Kampala, Uganda<EMAIL_ADDRESS>
###### Abstract.

This paper establishes the necessary and sufficient conditions for the reality
of all the zeros in a polynomial sequence $\\{P_{i}\\}_{i=1}^{\infty}$
generated by a three-term recurrence relation
$P_{i}(x)+Q_{1}(x)P_{i-1}(x)+Q_{2}(x)P_{i-2}(x)=0$ with the standard initial
conditions $P_{0}(x)=1,P_{-1}(x)=0,$ where $Q_{1}(x)$ and $Q_{2}(x)$ are
arbitrary real polynomials.

###### Key words and phrases:

recurrence relation, polynomial sequence, support, real zeros

###### 2010 Mathematics Subject Classification:

Primary 12D10, Secondary 26C10, 30C15

##  1\. Introduction

Asymptotic root distributions for sequences of univariate polynomials has been
a topic of study in analysis for many decades [7]. In particular, sequences of
polynomials with all real zeros are important in many branches of mathematics.
Such polynomials possess several nice properties. For example, if a polynomial
$P(x)=\sum_{i=0}^{n}b_{i}x^{i}$ is real-rooted and has nonnegative
coefficients, then the sequence $\\{b_{i}\\}_{i=0}^{n}$ is log-concave, i.e
$b_{i}^{2}\geq b_{i+1}b_{i-1}$ for all $1\leq j<n$ [6]. This log-concavity
implies that $\\{b_{i}\\}_{i=0}^{n}$ is unimodal, whereby the sequence
increases to a greatest value (or possibly two consecutive equal values) and
then decreases [6]. In addition, polynomials with real zeros are closed with
respect to differentiation and the zeros of derivative interlace with the
zeros of the polynomial. A good source of information about real-rooted
polynomials is a book V.P. Kostov, “Topics on hyperbolic polynomials in one
variable” [10].

In this paper, we discuss some cases of the problem under which conditions
polynomials satisfying a finite linear recurrence relation have all real
roots. Our general set-up is as follows. Fix complex-valued polynomials
$Q_{1}(x),\ldots,Q_{k}(x)$ and consider a finite linear recurrence relation of
the form

$P_{i}(x)+Q_{1}(x)P_{i-1}(x)+Q_{2}(x)P_{i-2}(x)+\dots+Q_{k}(x)P_{i-k}(x)=0,~{}~{}~{}~{}~{}~{}~{}i=1,2,\dots$
(1.1)

with the standard initial conditions

$P_{0}(x)=1,P_{-1}(x)=P_{-2}(x)=\dots=P_{-k+1}(x)=0.$ (1.2)

The generating function for the polynomial sequence
$\\{P_{i}\\}_{i=1}^{\infty}$ of this recurrence is given by

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+Q_{1}(x)t+Q_{2}(x)t^{2}+\dots+Q_{k}(x)t^{k}}.$

One of the well-known results in this area is a description of the
accumulation set for the zeros of $P_{i}(x)$ when $i\to\infty$ provided by the
theorem by Beraha, Kahane and Weiss, see [8]. It asserts that the support of
the limiting root-counting measure coincides with the following set. Let
$Q_{1},\dots,Q_{k}$ be complex polynomials as given in Equation (1.1) above.
Define a curve $\Gamma_{Q}\subset\mathbb{C}$ consisting of all values of $x$
such that the characteristic equation

$1+Q_{1}(x)t+Q_{2}(x)t^{2}+\dots+Q_{k}(x)t^{k}=0$ (1.3)

has at least two roots $t_{1},t_{2}$ for which

(a) $|t_{1}|=|t_{2}|$;

(b) $|t_{1}|$ is the minimum among the absolute values of all roots.

######  Theorem 1.

[8] Suppose that $\\{P_{i}(x)\\}$ satisfies (1.1), (1.2) and (1.3). Suppose
further that $\\{P_{i}(x)\\}$ satisfies no recursion of order less than $k$
and that there is no constant $\omega\in\mathbb{C}$ of unit modulus for which
$t_{r}=\omega t_{s}$ for some $r\neq s.$ Then the zeros of $P_{i}(x)$
accumulate along the curve $\Gamma_{Q}$ as $i\to\infty$.

This result provides a description of the asymptotic behaviour of the roots of
$P_{i}(x)$. However recently Tran [2] has found a number of cases when the
zeros of $P_{i}(x)$ actually lie on the limiting curve $\Gamma_{Q}$ for all or
for all sufficiently large $i$. In particular, he has proven the following
results.

######  Theorem 2.

[2] Let $\\{P_{i}(x)\\}$ be a polynomial sequence whose generating function is

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+Q_{1}(x)t+Q_{2}(x)t^{2}},$

where $Q_{1}(x)$ and $Q_{2}(x)$ are polynomials in $x$ with complex
coefficients. All the zeros of every polynomial in the sequence
$\\{P_{i}(x)\\}$ which satisfy $Q_{2}(x)\neq 0$ lie on the curve $\Gamma_{Q}$
defined by

$\mbox{Im}\left(\frac{Q_{1}^{2}(x)}{Q_{2}(x)}\right)=0~{}~{}~{}~{}\mbox{and}~{}~{}~{}~{}0\leq\mbox{Re}\left(\frac{Q_{1}^{2}(x)}{Q_{2}(x)}\right)\leq
4.$ (1.4)

$Moreover,~{}these~{}~{}zeros~{}~{}become~{}~{}dense~{}~{}in~{}~{}\Gamma_{Q}~{}~{}when~{}~{}i\to\infty.$

Theorem $2$ covers the special case of (1.1) and (1.2) for the polynomials
generated by the recurrence

$P_{i}(x)+Q_{1}(x)P_{i-1}(x)+Q_{2}(x)P_{i-2}(x)=0$

with the standard initial conditions $P_{0}(x)=1$ and $P_{-1}(x)=0$.

A more general result of Tran is as follows.

######  Theorem 3.

[3] Let $\\{P_{i}(x)\\}$ be a polynomial sequence with generating function

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+Q_{1}(x)t+Q_{2}(x)t^{k}},$

where $Q_{1}(x)$ and $Q_{2}(x)$ are polynomials in $x$ with complex
coefficients. Then there exists a constant $C=C(k)$ such that for all $i>C$,
all the zeros of $P_{i}(x)$ which satisfy $Q_{2}(x)\neq 0$ lie on the curve
$\Gamma_{Q}$ defined by

$\mbox{Im}\left(\frac{Q_{1}^{k}(x)}{Q_{2}(x)}\right)=0~{}~{}~{}~{}\mbox{and}~{}~{}~{}~{}0\leq(-1)^{n}\mbox{Re}\left(\frac{Q_{1}^{k}(x)}{Q_{2}(x)}\right)\leq\frac{k^{k}}{(k-1)^{k-1}}.$

$Moreover,~{}these~{}~{}zeros~{}~{}become~{}~{}dense~{}~{}in~{}~{}\Gamma_{Q}~{}~{}when~{}~{}i\to\infty.$

In the present paper, we want to characterise a situation as above that gives
rise to polynomial sequences with only real zeros.

######  Problem 1.

In the above notation, consider the recurrence relation

$P_{i}(x)+Q_{1}(x)P_{i-1}(x)+Q_{2}(x)P_{i-2}(x)=0$

with the standard initial conditions,

$P_{0}(x)=1,P_{-1}(x)=0$

where $Q_{1}(x)$ and $Q_{2}(x)$ are arbitrary real polynomials. Give necessary
and sufficient conditions on $(Q_{1}(x),Q_{2}(x))$ guaranteeing that all the
zeros of $P_{i}(x)$ will be real for all $i$.

To formulate our main result, we need to look at the curve defined by the
first condition in $(\ref{im}).$ We shall view $\mathbb{C}P^{1}$ as
$\mathbb{C}\cup\\{\infty\\}$, the extended complex plane and $\mathbb{R}P^{1}$
as the union of the real line in $\mathbb{C}$ with $\\{\infty\\}$. Let
$f:\mathbb{C}P^{1}\to\mathbb{C}P^{1}$ be the rational function defined by
$f=\frac{Q_{1}^{2}(x)}{Q_{2}(x)}$ where $Q_{1}(x)$ and $Q_{2}(x)$ are real
polynomials. Denote by $\widetilde{\Gamma_{Q}}\subset\mathbb{C}P^{1}$ the
curve given by Im$(f)=0$, that is

$\widetilde{\Gamma_{Q}}=f^{-1}(\mathbb{R}P^{1}).$

We note that for real polynomials $Q_{1}(x)$ and $Q_{2}(x)$, the curve
$\widetilde{\Gamma_{Q}}$ contains $\Gamma_{Q}$ since
$[0,4]\subset\mathbb{R}P^{1}.$

######  Lemma 4.

The curve $\widetilde{\Gamma_{Q}}$ has the following properties:

(a) $\widetilde{\Gamma_{Q}}\supset\mathbb{R}P^{1};$

(b) $\widetilde{\Gamma_{Q}}$ is invariant under complex conjugation;

(c) except $\mathbb{R}P^{1}$, $\widetilde{\Gamma_{Q}}$ might contain ovals
disjoint from $\mathbb{R}P^{1}$ (which come in complex-conjugate pairs) and
ovals crossing $\mathbb{R}P^{1}$ which are mapped to themselves by complex
conjugation;

(d) the intersection points of the second type of ovals with $\mathbb{R}P^{1}$
are exactly the real critical points of $f$.

Figures $1$ and $2$ illustrate the properties of $\widetilde{\Gamma_{Q}}$
claimed in Lemma 4 $(a)-(d)$. The main result of the present paper is as
follows.

$\begin{array}[]{c}\includegraphics[height=170.71652pt,width=170.71652pt]{ino3}\end{array}$

Figure 1. The curve $\widetilde{\Gamma_{Q}}$ for
$f=\frac{Q_{1}^{2}(x)}{Q_{2}(x)}$ where $Q_{1}(x)=x^{2}+1$ and
$Q_{2}(x)=x^{2}+6$.

$\begin{array}[]{c}\includegraphics[height=170.71652pt,width=170.71652pt]{ok2}\end{array}$

Figure 2. The curve $\widetilde{\Gamma_{Q}}$ for
$f=\frac{Q_{1}^{2}(x)}{Q_{2}(x)}$ where $Q_{1}(x)=x^{2}+5x+3$ and
$Q_{2}(x)=5x^{2}-1.$ The green points are the real critical points of $f$.

######  Theorem 5.

Let $\\{P_{i}(x)\\}$ be a sequence of polynomials whose generating function is

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+Q_{1}(x)t+Q_{2}(x)t^{2}},$ (1.5)

where $Q_{1}(x)$ and $Q_{2}(x)$ are arbitrary coprime real polynomials in $x$.
Then, for all positive integers $i$, all the zeros of $P_{i}(x)$ are real if
and only if the following conditions are satisfied:

(a) The polynomial $Q_{1}(x)$ must have all real and simple zeros.

(b) No ovals $\gamma$ of $\widetilde{\Gamma_{Q}}$ disjoint from
$\mathbb{R}P^{1}$ should exist.

(c) All the zeros of the discriminant $D(x)$ of the characteristic polynomial
$1+Q_{1}(x)t+Q_{2}(x)t^{2}$ must be real.

(d) No real critical values of $f$ should belong to the interval $(0,4)$.

(e) The polynomial $Q_{2}(x)$ must be non-negative at the zeros of $Q_{1}(x).$

Remark. The situation when $Q_{1}(x)$ and $Q_{2}(x)$ have a common real zero
is not interesting to consider since from
$P_{i}(x)+Q_{1}(x)P_{i-1}(x)+Q_{2}(x)P_{i-2}(x)=0$, such a zero would
necessarily be a zero of $P_{i}(x)$ for all $i$.

Acknowledgements. I am sincerely grateful to my advisor Professor Boris
Shapiro who introduced me to the problem and for the many discussions and
guidance of the above problem. I also thank Professor Rikard Bøgvad and Dr.
Alex Samuel Bamunoba for the discussions and guidance. I acknowledge and
appreciate the financial and academic support from Sida Phase-IV bilateral
program with Makerere University 2015-2020 under project 316 ‘Capacity
building in mathematics and its applications’.

##  2\. PROOFS

Let us begin with the following definitions and remarks.

Definition 1. For a non-constant rational function $R(x)=\frac{P(x)}{Q(x)}$,
where $P(x)$ and $Q(x)$ are coprime polynomials, the degree of $R(x)$ is
defined as the maximum of the degrees of $P(x)$ and $Q(x)$.

Equivalently, the degree of $R(x)$ is the number of distinct preimages of any
generic point.

Definition 2. A point $x_{0}$ is called a critical point of $R(x)$ if $R(x)$
fails to be injective in a neighbourhood of $x_{0}$, that is,
$R^{\prime}(x_{0})=0$. A critical value of $R(x)$ is the image of a critical
point. The order of a critical point $x_{0}$ of $R(x)$ is the order of the
zero of $R^{\prime}(x)$ at $x_{0}$.

Now, if $d$ is the degree of $R(x)$ and if $w$ is not a critical value then
$R^{-1}(w)=\\{x_{1},x_{2},\dots,x_{d}\\}$ with $x_{i}\neq x_{j}$ for all
$i\neq j$. Since the points $x_{j}$ are non-critical there is a neighbourhood
of each of these points such that $R(x)$ is injective on that neigbourhood.
The function $R(x)$ has $2d-2$ critical points in $\mathbb{C}P^{1}$ counting
multiplicities. This follows from the fact that in the complex plane, deg
$(R^{\prime})=$ deg$(P^{\prime}Q-Q^{\prime}P)=$ deg$(P)+$ deg$(Q)-1$ while the
order of the critical point at infinity is $|$deg$(P)-$ deg$(Q)|-1,$ [4].

Definition 3. Given a pair $(P,Q)$ of polynomials, define their Wronskian as

$W(P,Q)=P^{\prime}Q-Q^{\prime}P.$

One interesting thing about the Wronskian is that if $P$ and $Q$ are coprime,
then the zeros of $W(P,Q)$ are exactly the critical points of the rational map
$R=P/Q$. In [9] we find that if $P$ and $Q$ have all real, simple and
interlacing zeros, then all zeros of $W(P,Q)$ are non-real. In addition, if we
know that $\alpha$ is a zero of $R$ of multiplicity $\geq 2$, then $\alpha$ is
also a (multiple) zero of the Wronskian. More information about the Wronski
map can be found in [1, 9].

###### Proof of Theorem 5.

(a) Substitution of the initial conditions $P_{0}(x)=1$ and $P_{-1}(x)=0$ in
the recurrence relation

$P_{i}(x)+Q_{1}(x)P_{i-1}(x)+Q_{2}(x)P_{i-2}(x)=0$

gives for $i=1$,

$P_{1}(x)+Q_{1}(x)P_{0}(x)+Q_{2}(x)P_{-1}(x)=0$

or

$P_{1}(x)=-Q_{1}(x).$

Therefore $Q_{1}(x)$ must have all its zeros real since we require all the
zeros of $P_{i}(x)$ to be real for all $i$ and in particular for $i=1$. These
zeros must be simple (see part $(e)$ for the justification).

(b) Suppose there exists an oval $\gamma$ of $\widetilde{\Gamma_{Q}}$ which
does not intersect $\mathbb{R}P^{1}$. From Lemma 4 $(c)$, $\gamma$ is the type
one oval contained in $\widetilde{\Gamma_{Q}}.$ We note that all the points on
$\gamma$ and its interior are of the form $z=x+iy$ where
$x,y\in\mathbb{R},y\neq 0$ and this is a connected component with $\gamma$ as
its boundary. This component is mapped by $f$ onto the half plane with degree
$\geq 1$ depending on the number of critical points of $f$ it strictly
contains. The boundary $\gamma$ of the component, is mapped onto
$\mathbb{R}P^{1}$ (the boundary of the half plane). In particular, the image
$f(\gamma)$ covers the interval $[0,4]\subset\mathbb{R}P^{1}$. Therefore
$\Gamma_{Q}=f^{-1}([0,4])$ must contain an arc of the boundary $\gamma$. From
Theorem 2 [2], all the zeros of $P_{i}(x)$ are contained in $\Gamma_{Q}$ for
all $i$ and are dense in $\Gamma_{Q}$. Now since we require all the zeros of
$P_{i}(x)$ to be real, it must be that $\Gamma_{Q}\subseteq\mathbb{R}P^{1}$.
This is not possible as we already have that $\Gamma_{Q}$ contains an arc of
$\gamma$ yet this arc is not contained in $\mathbb{R}P^{1}$, hence a
contradiction.

(c) It is known [5] that the endpoints of $\Gamma_{Q}$ are the points where
$t_{1}=t_{2}$ (see the notation in Theorem 2). In our case equation (1.3) has
degree $2$ in $t$. Therefore every $x$, for which the roots of (1.3) coincide
belongs to $\Gamma_{Q}$. These $x$ are exactly the zeros of
$D(x)=Q_{1}^{2}(x)-4Q_{2}(x).$ Since we require that
$\Gamma_{Q}\subset\mathbb{R}P^{1}$, all the zeros of $D(x)$ must be real.

(d) Suppose there exists a critical value $w\in(0,4)$. Then there must exist a
real critical point $x_{c}$ such that $f(x_{c})=w.$ Clearly, $x_{c}$ is a
point on $\Gamma_{Q}$. It is known [4] that a point $x_{c}\in\mathbb{C}P^{1}$
is a critical point of order $k$ for a rational function $R(x)$ if and only if
there are open sets $U$ containing $x_{c}$ and $V$ containing $w=R(x_{c})$
such that each $w_{0}\in V,w\neq w_{0}$ has exactly $k+1$ distinct preimages
in $U$.

In our scenario let $x_{c}$ be such a critical point of order $k$ for $f(x)$
and $V$ be the real interval $(w-\epsilon,w+\epsilon)$ for sufficiently small
$\epsilon>0$. Then $V\subset(0,4)$. Note that since $x_{c}$ is a critical
point of order $k$ for $f(x)$ and any point $z\in V,z\neq w$ has exactly $k+1$
distinct preimages in $U$, we have at $x_{c}$, locally $U=f^{-1}(V)$ the
preimage of $V$ consists of $k+1$ distinct curves (arcs) with a common
intersection only at $x_{c}$. One of these curves is a line segment on the
real line while the remaining $k$ curves are complex i.e apart from $x_{c}$,
points on these $k$ curves are of the form $z=x+iy$ where
$x,y\in\mathbb{R},y\neq 0$.

Now since complex arcs are formed in the preimage of $V$, then some of the
zeros of $P_{i}(x)$ will be contained in the complex arcs since all the zeros
of $P_{i}(x)$ are contained in ${\Gamma_{Q}}=f^{-1}([0,4]\supset V)$ and are
dense there as $i\to\infty$. This contradicts our requirement that
${\Gamma_{Q}}$ is contained in $\mathbb{R}P^{1}$. Therefore in order to have
all the real roots of $P_{i}(x)$ for all $i$, no real critical values of $f$
can be in the real interval $(0,4)$. Otherwise the condition that
${\Gamma_{Q}}\subset\mathbb{R}P^{1}$ cannot hold.

(e) Let $x_{0}$ be a zero of $Q_{1}(x)$, i.e $Q_{1}(x_{0})=0$. Note that
$Q_{2}(x)$ and $Q_{1}(x)$ do not have a common zero since they are coprime.
Therefore at the point $x_{0}$, we have $Q_{2}(x_{0})\neq 0.$ It remains to
show that $Q_{2}(x_{0})>0$. We note that all the zeros of $Q_{1}(x)$ are
critical points of $f$. In addition, they belong to $\Gamma_{Q}$ because at
$x_{0}$, we have that $f(x_{0})=Q_{1}^{2}(x_{0})/Q_{2}(x_{0})=0$; therefore
both the real and the imaginary part of $f$ vanish, hence satisfying (1.4) of
Theorem $2$. Suppose that $x_{0}$ is a simple critical point of $f$ and let
$Q_{2}>0$ at $x_{0}$. Then locally $f=Q_{1}^{2}/Q_{2}\geq 0$ on the interval
in $\mathbb{R}$ and if $Q_{2}<0$ at $x_{0}$, then locally
$f=Q_{1}^{2}/Q_{2}\geq 0$ on the complex arc, i.e there exists an interval
$I\subset[0,4]$ such that $f^{-1}(I)$ contains a complex arc. On the other
hand, if $x_{0}$ is a critical point of order $>1$, then locally
$f=Q_{1}^{2}/Q_{2}\geq 0$ on some complex arc irrespective of whether
$Q_{2}>0$ or $Q_{2}<0$. However, it is known that the zeros of $P_{i}(x)$ are
contained in $\Gamma_{Q}=f^{-1}[0,4]$ and are dense there as $i\to\infty$, and
so for the reality of all the zeros of $P_{i}(x)$ we require that
$\Gamma_{Q}\subset\mathbb{R}P^{1}$. This is not possible if $Q_{2}<0$ at
simple zeros of $Q_{1}(x)$ or when zeros of $Q_{1}$ have multiplicity $>1$
since in either case there will be some zeros of $P_{i}(x)$ on the complex arc
which is a contradiction. Therefore the polynomial $Q_{2}(x)$ must be non-
negative at the zeros of $Q_{1}(x)$ as a necessary condition for the reality
of all the zeros of $P_{i}(x)$ for all $i$. Furthermore all the zeros of
$Q_{1}(x)$ must be simple otherwise as explained above some zeros of
$P_{i}(x)$ would be on the complex arc. (This last part settles part $(a)$ of
the Theorem where we require all the zeros of $P_{i}(x)$ to be simple). ∎

Remark. Each of the conditions of Threorem 5 $(a)-(e)$ is only a necessary
(and not a sufficient) condition for the reality of all the zeros of
$P_{i}(x)$. To guarantee reality of all the zeros of $P_{i}(x)$ for all $i$,
all the five conditions must be satisfied simultaneously. Below are some of
the examples illustrating this claim.

Example 1. Consider the sequence of polynomials $\\{P_{i}(x)\\}$ generated by
the rational function

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+(-x^{2}+2x)t+(5x^{2}-1)t^{2}}.$

The corresponding $f$ is given by

$f(x)=\frac{(-x^{2}+2x)^{2}}{5x^{2}-1}.$

The zeros of $Q_{1}(x)$ are $0$ and $2$ which are real and simple (Theorem
$5(a)$). There are no ovals disjoint from $\mathbb{R}P^{1}$ (Theorem $5(b)$).
The discriminant $D(x)=x^{4}-4x^{3}-16x^{2}+4$ has only real zeros (Theorem
$5(c)$). These are $-2.39337,-0.54374,0.47570$ and $6.46141$ (corrected to $5$
dp and are indicated by red points in Figure 3). However as seen from Figure
$3$, not all the zeros of $P_{100}(x)$ are real. This shows that the above
three conditions satisfied are not each sufficient to guarantee that all the
zeros of $P_{i}(x)$ will be real for all $i$.

$\begin{array}[]{c}\includegraphics[height=156.49014pt,width=142.26378pt]{ino5}\end{array}$

Figure 3. The zeros of $P_{100}(x)$ for the generating function
$1/(1+(-x^{2}+2x)t+(5x^{2}-1)t^{2})$.

Example 2. Consider the sequence of polynomials $\\{P_{i}(x)\\}$ generated by
the rational function

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+(2x^{2}-8x+6)t+(-5x^{3}+37x^{2}-43x-21)t^{2}}.$

The corresponding $f$ is given by

$f(x)=\frac{(2x^{2}-8x+6)^{2}}{-5x^{3}+37x^{2}-43x-21}.$

$\begin{array}[]{c}\includegraphics[height=142.26378pt,width=142.26378pt]{trs}\end{array}$

Figure 4. The zeros of $P_{100}(x)$ for the generating function
$1/(1+(2x^{2}-8x+6)t+(-5x^{3}+37x^{2}-43x-21)t^{2})$.

The zeros of $Q_{1}(x)$ are $1$ and $3$ which are real and simple. Also there
are no ovals disjoint from $\mathbb{R}P^{1}$. The discriminant
$D(x)=x^{4}-3x^{3}-15x^{2}+19x+30$ has only real zeros, i.e $x=-3,x=-1,x=2$
and $x=5.$ However $f$ has a critical value of $3.50783\in(0,4)$ corresponding
to the critical point $-1.66437$ corrected to $5$ dp, hence the condition of
Theorem $5(d)$ is violated. Consequently, some of the zeros of $P_{100}$ are
not real (see Figure $4$). The first three conditions of Theorem 5 are
satisfied but not the fourth condition. Therefore having no critical value in
$(0,4)$ is indeed a necessary condition for the reality of all the zeros of
$P_{i}(x).$

Example 3. Consider the sequence of polynomials $\\{P_{i}(x)\\}$ generated by
the rational function

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+(2x^{2}-8x+6)t+(x^{4}-8x^{3}+21x^{2}-14x-16)t^{2}}.$

The corresponding $f$ is given by

$f(x)=\frac{(2x^{2}-8x+6)^{2}}{x^{4}-8x^{3}+21x^{2}-14x-16}.$

$\begin{array}[]{c}\includegraphics[height=142.26378pt,width=128.0374pt]{hmm}\end{array}$

Figure 5. The zeros of $P_{100}(x)$ for the generating function
$1/(1+(2x^{2}-8x+6)t+(x^{4}-8x^{3}+21x^{2}-14x-16)t^{2})$.

The zeros of $Q_{1}(x)$ are $1$ and $3$ which are real and simple. Also there
are no ovals disjoint from $\mathbb{R}P^{1}$. The discriminant
$D(x)=4x^{2}-40x+100$ has only real zeros, i.e $x=5.$ In addition $f$ has no
critical value in the real interval $(0,4)$. Thus conditions of Theorem 5
$(a)$ to $(d)$ are satisfied. Note that on the zeros of $Q_{1}$ we have
$Q_{2}(1)=-16\hbox to0.0pt{\kern 4.49997pt$|$\hss}>0$ and $Q_{2}(3)=-4\hbox
to0.0pt{\kern 4.49997pt$|$\hss}>0$. Thus condition $(e)$ of Theorem $5$ is
violated. Consequently some of the zeros of $P_{100}$ are not real as seen in
Figure $5$.

Example 4. Consider the sequence of polynomials $\\{P_{i}(x)\\}$ generated by
the rational function

$\sum_{i=0}^{\infty}P_{i}(x)t^{i}=\frac{1}{1+(x^{2}-2x-5)t+x^{2}t^{2}}.$

The corresponding $f$ is given by

$f(x)=\frac{(x^{2}-2x-5)^{2}}{x^{2}}.$

$\begin{array}[]{c}\includegraphics[height=142.26378pt,width=142.26378pt]{gd2}\end{array}$

Figure 6. The zeros of $P_{200}(x)$ when the generating function is
$1/(1+(x^{2}-2x-5)t+x^{2}t^{2})$.

In this case, all the five conditions of Theorem $5$ have been satisfied and
as seen in Figure $6$, all the zeros of $P_{200}(x)$ are real. We used
$i=200,$ but any arbitrary value of $i\in\mathbb{N}^{+}$ works. The red points
are the zeros of the discriminant and these are the endpoints of the intervals
where all the zeros of $P_{i}(x)$ for all $i$ are located, that is, all the
zeros of $P_{i}(x)$ for all $i$ are supported on the real axis and the support
is a union of two disjoint real intervals given by
$[-\sqrt{5},-1]\cup[\sqrt{5},5]\subset\mathbb{R}P^{1}$. The zeros of
$P_{i}(x)$ are dense in this support as $i\to\infty.$

##  3\. Final Remarks

Problem. Describe similar conditions guaranteeing reality of roots for all
polynomials in the context of Theorem 3.

## References

  * [1] K. Dilcher, K. B. Stolarsky: Zeros of the Wronskian of a Polynomial. Journal Of Mathematical Analysis And Applications 162 (1991), 430–451. MR1212730
  * [2] K. Tran: Connections between discriminants and the root distribution of polynomials with rational generating function. J. Math. Anal. Appl. 410 (2014), 330–340. MR3109843
  * [3] K. Tran: The root distribution of polynomials with a three-term recurrence. J. Math. Anal. Appl. 421 (2015), 878–892. MR3250512
  * [4] L. Carleson, T. W. Gamelin: Complex dynamics, Universitext: Tracts in Mathematics. Springer-Verlag, New York (1993). MR1230383
  * [5] N. L. Biggs: Equimodular curves. Discrete Math. 259 (2002), 37–57. MR1948772
  * [6] P. Brändén: Unimodality, log-concavity, real-rootedness and beyond. Handbook of enumerative combinatorics, Discrete Math. Appl. (2015), 437–483. MR3409348
  * [7] Q. I. Rahman, G. Schmeisser: Analytic Theory of Polynomials. London Math. Soc. Monogr. New Ser. 26., The Clarendon Press, Oxford University Press, Oxford (2002). MR1954841
  * [8] S. Beraha, J. Kahane, N. J. Weiss: Limits of zeros of recursively defined families of polynomials. Studies in Foundations and Combinatorics. Advances in Math., Supplementary Studies 1 (1978), 213–232.
  * [9] V. Kostov, B. Shapiro, M. Tyaglov: Maximal univalent disks of real rational functions and Hermite-Biehler polynomials. Proc. Amer. Math. Soc. 139(5), (2011), 1625–1635. MR2763752
  * [10] V. P. Kostov: Topics on hyperbolic polynomials in one variable. Panoramas et Synth$\grave{e}$ses 33 (2011), vi + 141 p. SMF. MR2952044


# TiNO-Edit: Timestep and Noise Optimization
for Robust Diffusion-Based Image Editing

Sherry X Chen Corresponding author email<EMAIL_ADDRESS>Yaron Vaxman
Elad Ben Baruch  David Asulin  Aviad Moreshet
Kuo-Chin Lien  Misha Sra  Pradeep Sen

###### Abstract

Despite many attempts to leverage pre-trained text-to-image models (T2I) like
Stable Diffusion (SD) [25] for controllable image editing, producing good
predictable results remains a challenge. Previous approaches have focused on
either fine-tuning pre-trained T2I models on specific datasets to generate
certain kinds of images (e.g., with a specific object or person), or on
optimizing the weights, text prompts, and/or learning features for each input
image in an attempt to coax the image generator to produce the desired result.
However, these approaches all have shortcomings and fail to produce good
results in a predictable and controllable manner. To address this problem, we
present TiNO-Edit, an SD-based method that focuses on optimizing the noise
patterns and diffusion timesteps during editing, something previously
unexplored in the literature. With this simple change, we are able to generate
results that both better align with the original images and reflect the
desired result. Furthermore, we propose a set of new loss functions that
operate in the latent domain of SD, greatly speeding up the optimization when
compared to prior losses, which operate in the pixel domain. Our method can be
easily applied to variations of SD including Textual Inversion [13] and
DreamBooth [27] that encode new concepts and incorporate them into the edited
results. We present a host of image-editing capabilities enabled by our
approach. Our code is publicly available at
https://github.com/SherryXTChen/TiNO-Edit.

##  1 Introduction

Pure text-guided & Reference-guided image editing Original Ref w/o ref w/ref
“sandwich” $\rightarrow$ “cake” $+$ “a magenta hat”

Stroke-guided image editing
---
Original | User input | Result | Original | User input | Result
|  |  |  |  |
$+$ “a sunflower” |  $+$ “curtains”

Image composition Original User input Result Original User input Result $+$ “a
scarecrow” “white” $\rightarrow$ “yellow plaid” Image editing with DreamBooth
(DB) or Textual Inversion (TI) Original DB Result Original TI Result “photo”
$\rightarrow$ $\langle concept\rangle$ “sky” $\rightarrow$ $\langle
concept\rangle$

Figure 1: Overview of capabilities enabled by TiNO-Edit. TiNO-Edit offers
various image-editing capabilities and can be run with DreamBooth (DB) [27] or
Textual Inversion (TI) [13]. By leveraging diffusion timestep and noise
optimization techniques, it can generate realistic and high quality outputs.

Computer-generated image synthesis has been studied for decades for its wide
range of applications including content creation, marketing and advertising,
visualization/simulation, entertainment, and storytelling. Recent advances in
diffusion-based text-to-image (T2I) generations [30, 10, 25, 23, 19] models
have allowed users to specify, modify, and enhance images in novel ways using
text-based prompts or other inputs [35]. Still, these often give undesired
results, so the main question is how to use these models in an artistic
workflow to enable controllable image editing that can produce the desired
results.

To this end, researchers have fine-tuned existing model backbones on new
datasets to accommodate specific use cases, including conditioning image
editing on instructions [3], using images with inpainting masks [34] or
bounding boxes [17], and including other visual information such as edges,
segmentation maps, and depth maps [35]. Another line of work has focused on
optimizing weights [36, 6], intermediate attention/feature layers [22, 31], or
inputs [21, 33] of existing T2I backbones with respect to each image the user
wants to edit to try and produce the desired result.

In this last thrust, optimizing text-prompt inputs has gained the most
attention. We have seen considerable work on prompt engineering [25] which
studies the effect of different (positive or negative) text prompts, including
optimizing the text prompt encoding in the CLIP domain [24] as well as
adjusting the weights of text prompts [33]. However, as shown in the examples
in this paper, these previous approaches still leave a lot to be desired when
it comes to high-quality image editing.

Despite previous work that attempts to optimize input parameters to the
diffusion model, we observe that there are two other kinds of inputs whose
optimization has not been thoroughly explored yet: the noise used to corrupt
the input images, and the diffusion timesteps. For noisy images, most work
either inverts/reconstructs them with respect to the noise scheduler they use
with the diffusion backbones [21, 32, 22, 31] or simply distorts the original
image based on a pre-chosen distortion level [8, 20, 1], which is determined
by the starting diffusion timestep. Although preliminary research has been
conducted to explore how this affects the similarity between the output and
the original image [8, 20, 1], these observations have yet to be incorporated
into the methods in an automated manner, so it still requires extensive trial-
and-error for users to find the optimal settings. Furthermore, the remaining
timestep values are fixed based on method hyper-parameters.

Given the important role noisy input images and timesteps play in the
diffusion-based image synthesis process, we argue that they should also be
optimized based on the image editing objective. To this end, we present TiNO-
Edit, an optimization-based method that is built up of one T2I backbone,
namely Stable Diffusion (SD) [25], to automatically find the best level of
noise that should be added and removed from the original image to achieve the
best results. We design a set of loss functions that operate in the latent
domain of SD to save computational time and resources. More importantly, TiNO-
Edit supports a line of image editing capabilities (Fig. 1), including pure
text-guided image editing, image editing guided by reference images, stroke-
based image editing, and image composition. It can also be applied with
variations of SD including Textual Inversion [13] and DreamBooth [27] that
encode new concepts in order to generate outputs with concepts that may
otherwise be hard to describe, which is a capability that has been largely
overseen.

In summary, our contributions include:

  * •

A novel SD algorithm for image editing that supports various image-editing
capabilities.

  * •

A new set of loss functions that save computational time and resources.

  * •

A novel image editing capability of incorporating new concepts from variations
of SD

##  2 Related Work

###  2.1 Text-to-image diffusion-based models

Diffusion models have shown unprecedented quality in image generation tasks
and become the backbone of many text-to-image (T2I) applications [3, 34, 35,
36, 21, 33, 32, 22, 31]. These models use text prompts as inputs to the
diffusion model to condition the reverse denoising process. Like the general
diffusion models, T2I diffusion-based models are able to take Gaussian noise
and denoise it using text prompts to generate images that align with those
text prompts.

###  2.2 Text-guided diffusion-based image editing with pre-trained models

T2I diffusion-based models have drawn a lot of attention with their high-
quality synthesis results. One line of work takes these pre-trained models as
backbones and fine-tunes them to perform specific image editing tasks. For
example, ControlNet [35] takes a copy of the U-Net [26] encoder component of
SD [25] and attaches it back to the SD backbone to accept visual information
including image edge maps, depth maps, segmentation maps, etc. This attached
encoder is then fine-tuned to encode that information and guide the generated
image output accordingly. Although ControlNet primarily focuses on T2I
applications, other work built on it supports text-guided image editing [14].

On the other hand, InstructPix2Pix [3] fine-tunes SD to accept instructions
via text prompts to perform image editing. They tailor a dataset of image
pairs and corresponding instructions that reflect changes in the image pairs,
using the T2I backbone and a large language model GPT-3 [4]. Similarly, Paint
by Example [34] is an exemplar-based image editing method via inpainting,
where they attach a CLIP-based classification model on SD to accept image
examples that will be used to fill the masked region of an input image. The
pre-trained SD backbone is then fine-tuned to be conditioned on these
examples.

Another approach centers on optimizing weights [36, 6], intermediate
attention/feature layers [22, 31], or inputs [21, 33] for each specific image.
Similar to the previously mentioned methods, these approaches also need a
specific dataset. The dataset may be formed from a single image, such as its
patches [36]. The model takes in the noisified patches as well as their
positions with respect to the original image and learns to denoise those
patches. During inference, both models (before and after fine-tuning) are used
to denoise the input. Subsequently, the outputs from each stage are combined
through diffusion (reverse denoising) steps to generate the final image.

Intermediate layers of pre-trained models can be used to guide image editing
processes. Existing work [31, 22, 11] has looked into various aspects
including cross and self-attention maps, as well as activation functions and
their effects on object location, shape, size, and appearance in the generated
results. Again, image inversion may be applied with respect to the original
image [31] to learn relevant information of intermediate layers, which can
then be injected in the denoising process when editing this image [31] or be
compared with the ones with respect to the edited results and optimize the
latter to be aligned with the former [22].

Finally, inputs of T2I models include text prompts, input noise, and
timesteps, all of which may be optimized for more desirable editing outcomes.
Image editing work that utilizes input optimization usually starts from image
inversion (reconstructing images by representing them in the domain of the
generative model), which involves finding the reverse denoising path that will
lead to the input image [21]. Among all inputs of T2I models, prompts have
gained a fair amount of attention as they have a lot of power over the
semantics of generative images and, thus are very relevant to image editing
results. In the context of SD, prompts can either be positive to describe what
the generated images should contain or negative, to describe undesirable
visuals that should be excluded from the outputs (e.g. “low quality, jpeg
artifacts, ugly…”). Optimization can be performed on both positive [33] or
negative prompts [21].

Our optimization-based method falls into the above category. In contrast to
prior work, our method maintains both positive and negative prompts while
focusing on optimizing input noise and timesteps instead to get the
corresponding edits entailed by the prompts.

##  3 Preliminaries

Figure 2: Effect of starting timestep and noise on image editing. Suppose we
want to change the cat in the left image to a dog, we can input this image and
the target prompt “a photo of a dog” to Stable Diffusion (SD) Img2Img [25,
15], along with random Gaussian noise $N$ and a starting time $T\in[0,1]$ to
produce results such as those shown in the grid on the right. Here, we vary
$T$ (fixed per column) and $N$ (fixed per row). As $T$ increases, the output
matches the target prompt better, but it also diverges more from the original
image in terms of composition and pose. Furthermore, different random noise
inputs can lead to different visual features.

In this section, we first provide an overview of Diffusion Denoising Implicit
Model (DDIM) [30] and Stable Diffusion (SD) [25] algorithms, which our method
is based on, followed by an experiment that gives us insight into the effect
of various components in them on edited results.

DDIMs are generative models that produce natural images from noise
iteratively. During training, DDIMs model a forward diffusion (FD) process
transforming images to noise in $S$ timesteps, denoted as
$\\{t_{k}=k\cdot\frac{1}{S},k\in[0,S]\\}$, where the corresponding noisified
image $I_{k}$ per timestep from a pristine image $I$ is

$\begin{split}I_{k}=\text{FD}_{k}(I,N)=\sqrt{\alpha(t_{k})}\cdot
I+\sqrt{1-\alpha(t_{k})}\cdot N,\end{split}$ (1)

where noise $N\sim\mathcal{N}(0,\mathbf{I})$ and $\alpha(\cdot)$ is a pre-
defined diffusion function. Starting from $\tilde{I}_{k}=I_{k}$, DDIMs then
model a reverse diffusion (RD) that remove noise iteratively and recover
intermediate images as

$\begin{split}\tilde{I}_{k-1}=&{\text{RD}}_{k,k-1}(\tilde{I}_{k},\tilde{N}_{k})\\\
=&\sqrt{\alpha(t_{k-1})}\left(\frac{\tilde{I}_{k}-\sqrt{1-\alpha(t_{k})}\cdot\tilde{N}_{k}}{\sqrt{\alpha(t_{k})}}\right)\\\
&+\sqrt{1-\alpha(t_{k-1})}\cdot\tilde{N}_{k},\end{split}$ (2)

, where $\tilde{N}_{k}={\text{UNet}_{\text{DDIM}}}(\tilde{I}_{k},t_{k})$ is
the predicted noise from a denoising UNet ${\text{UNet}_{\text{DDIM}}}$ [26].

The same can be applied to Stable Diffusion (SD) [25] with two key
differences. First, SD performs diffusion steps in a latent space of a
Variational Auto-Encoder (VAE) [12] with an encoder $\text{VAE}_{\text{enc}}$
and a decoder $\text{VAE}_{\text{dec}}$. The latent representation of $I$ is
$L=\text{VAE}_{\text{enc}}(I)$. Secondly, the denoising UNet in SD is
conditioned on an additional text prompt $p$, where
$\tilde{N}_{k}={\text{UNet}_{\text{SD}}}(\tilde{L}_{k},t_{k},p)$, which
enables its text-to-image synthesis functionality.

SD Img2Img variant [15] can edit images with text. Given an image $I$ and time
$T\in[0,1]$, $I$ is first noisified with respect to timestep $\max_{t_{k}\leq
T}t_{k}$ using Eq. 1 and then denoised. The idea behind this, first introduced
in SDEdit [20], is that the less we distort the input, the more likely we are
to generate a result similar to it.

However, this raises the question as to what the value of $T$ should be. To
give us insight into the problem, we can run a simple experiment, shown in
Fig. 2. As we can see, changes in $T$ and $N$ can lead to drastically
different results, which motivates our timestep and noise optimization
approach explained in the next section.

##  4 TiNO-Edit

Figure 3: Optimization parameters. We find optimization parameters by studying
the SD denoising process. The output $\tilde{L}_{0}$ is only affected by the
timesteps $t_{k}$ ($k\in[1,K]$) and the noisy latent image input
$\tilde{L}_{k}$ for each of the $K$ denoising steps. Note we are assuming that
the learning models are all fixed (denoted by the snowflake symbol) and that
the number of timesteps $K$ is a constant. $\tilde{L}_{k}$ can then be traced
back through $K$ iterations to the initial latent image input $\tilde{L}_{K}$
that is computed from starting timestep $t_{K}=T$ and the Gaussian noise $N$.
Hence, we can achieve our goal by simply optimizing $N$ and time steps $t_{k}$
for all $k\in[1,K]$.

Rather than finding the optimal values manually as in previous approaches,
TiNO-Edit aims to automate the process by optimizing the variable parameters
related to $N$ and $T$ so that they produce desired results. From the SD
denoising process (Fig. 3), we see that the output result $\tilde{L}_{0}$ is
only affected by the timesteps $t_{k}$ ($k\in[1,K]$) and the input Gaussian
noise $N$, assume all models are fixed and the number of steps $K$ is a
constant. We optimize them, starting from $t_{k}=k\cdot\frac{T}{K}$ and
$\mathcal{N}(0,\mathbf{I})$, with respect to our loss functions defined in
Sec. 4.2 via gradient descent with the Jacobian from
$\text{UNet}_{\text{SD}}$. We optimizes all $t_{k}$’s so they can be non-
uniformly spaced and is more flexible than only optimizing $T$ and/or $k$. The
updated $t_{k}$’s are used in the next optimization step as the timestep
inputs to the UNet for the reverse diffusion process. Note that we don’t
optimize $\alpha(t_{k})$ because doing so along with changing $t_{k}$ will
break the dependency between the two learnt by the models and may cause
generation quality degrade.

The inputs to our method include the input image $I$, the target prompt $p$,
the input image description/prompt $p_{O}$ and additional inputs
$\mathcal{A}=\\{I_{*},M_{a},*\in\\{r,s,c\\}\\}$ where $a,r,s,c$ denote the
additional image and mask input to the editing capabilities of adding objects,
reference-guided image editing, stroked-guided image editing, and image
composition if applicable.

###  4.1 Masking mechanism

During the editing process, it does not make sense to add noise or denoise
regions that we want to maintain. So we design a masking mechanism
$\text{MSK}(I,p_{O},p,\mathcal{A})$ to locate the editing region $M$ from
aforementioned inputs. To replace objects in pure text-guided image editing,
$M=\text{CLIPSeg}(I,o)$, where $\text{CLIPSeg}(\cdot,\cdot)$ [18] computes
regions in $I$ that correspond to objects $\\{o:o\in p_{O},o\notin p\\}$. To
perform style transfer, $M_{ij}=1$ for all pixel locations $(i,j)$. To add
objects for pure text-guided or reference-guided image editing,
$M=M_{a}\in\mathcal{A}$. For stroke-guided image editing or image composition,
$M_{ij}=1-\delta(I_{ij},(I_{*})_{ij})$ using the Kronecker delta function
($\delta(x,y)=1\text{ if }x=y\text{, otherwise }0$).

###  4.2 Optimization loss functions

Figure 4: Training LatentCLIP. Our LatentCLIP visual encoder
($\text{LatentCLIP}_{\text{vis}}$) is a copy of a pre-trained CLIP image
encoder ($\text{CLIP}_{\text{vis}}$) [24], except the first convolution layer
is replaced to accommodate for taking the latent vector
$\text{VAE}_{\text{enc}}(I)$[12] as input and output the image feature
$f_{L}$. The entire LatentCLIP visual encoder is unfrozen (indicated by the
fire symbol, as opposed to the snowflake symbol which means the model is
frozen) and is trained to minimize the cosine difference between $f_{L}$ and
$f$, which is the image feature of $I$ from the CLIP image encoder.

To achieve the desired output $\tilde{I}_{0}$ or its latent representation
$\tilde{L}_{0}$, we need to design a set of optimization functions that
captures key characteristics the output should have with respect to the
original image $I$ or its latent representation $L$.

First, we want to enforce semantic similarity between the two with a CLIP-
based[24] semantic loss function. But instead of operating it in the pixel
domain like previous methods [33, 9], we have designed our own CLIP visual
encoder that operates in the latent domain called LatentCLIP. This greatly
speed up the optimization as the latent domain is much more compacted than the
pixel domain.

LatentCLIP visual encoder $\text{LatentCLIP}_{\text{vis}}$ takes a latent
image and is trained to output the same feature as its pixel domain
counterpart from the original CLIP visual encoder $\text{CLIP}_{\text{vis}}$
(Fig. 4), where the model training objective is

$1-\cos(\text{LatentCLIP}_{\text{vis}}(L),\text{CLIP}_{\text{vis}}(I)).$ (3)

$\text{LatentCLIP}_{\text{vis}}$ is initialized from a pretrained
$\text{CLIP}_{\text{vis}}$, which the first convolution layer replaced to
accommodate for the dimension of latent images.

Algorithm 1 TiNO-Edit

1:function TiNO-Edit($I,p_{O},p,\mathcal{A}$)

2: $K\in\mathbb{Z}^{+}$, $T\in[0,1]$

3: $t_{k}\leftarrow k\cdot\frac{T}{K},k\in[0,K]$

4: $N\sim\mathcal{N}(0,\mathbf{I})$

5: $L\leftarrow\text{VAE}_{\text{enc}}(I)$

6: $M=\text{MSK}(I,p_{O},p,\mathcal{A})$

7: for $w\leftarrow 1$ to $W$ do

8: $\tilde{L}_{K}\leftarrow\text{FD}_{K}(L,N)$ $\triangleright$ Eq. 1

9: $\tilde{L}_{K}\leftarrow\tilde{L}_{K}\bigodot M+L\bigodot(1-M)$

10: for $k\leftarrow K$ to $1$ do

11: $\tilde{N}_{k}\leftarrow\text{UNet}_{\text{SD}}(\tilde{L}_{k},t_{k},p)$

12:
$\tilde{L}_{k-1}\leftarrow\text{RD}_{k,{k-1}}(\tilde{L}_{k},\tilde{N}_{k})$
$\triangleright$ Eq. 2

13: $\tilde{L}_{k-1}\leftarrow\tilde{L}_{k-1}\bigodot M+L\bigodot(1-M)$

14: end for

15: $\underset{N,t_{k}}{\text{argmin}}\
\mathcal{L}_{total}(L,\tilde{L}_{0},p_{O},p,\mathcal{A})$$\triangleright$ Eq.
8

16: end for

17: return $\text{VAE}_{\text{dec}}(\tilde{L}_{0})$

18:end function

To incorporate $\text{LatentCLIP}_{\text{vis}}$ and the CLIP text encoder
$\text{CLIP}_{\text{text}}(\cdot)$ [24, 7] to the semantic loss function, we
may calculate the cosine distance between the output feature and the prompt
feature
$\cos(\text{LatentCLIP}_{\text{vis}}(\tilde{L}_{0}),\text{CLIP}_{\text{text}}(p))$
[24], the distance between the original prompt and the target prompt in CLIP
text domain $\text{CLIP}_{\text{text}}(p_{O})-\text{CLIP}_{\text{text}}(p)$,
as well as the distance between the original image and the output in CLIP
image domain
($\text{LatentCLIP}_{\text{vis}}(L)-\text{LatentCLIP}_{\text{vis}}(\tilde{L}_{0})$)
[33] if we adopt directly from these prior methods.

However, all of them rely on the assumption that the image encoder and the
text encoder are from the same CLIP pre-trained model without any
modifications or fine-tuning with the same feature dimensions. This prevent
different variations of SD and new concepts learned through DreamBooth [27] or
Textual Inversion [13] outside what the original CLIP text encoder to be
incorporated in the loss. To this end, we design a new semantic loss function
that minimizes the difference between the cosine distance between
original/target images in the CLIP image domain and the one between
original/target prompt in the CLIP text domain as

$\begin{split}&\mathcal{L}_{sem}(L,\tilde{L}_{0},p_{O},p)\\\
=&\cos(\text{LatentCLIP}_{\text{vis}}(L),\text{LatentCLIP}_{\text{vis}}(\tilde{L}_{0}))\\\
&-\cos(\text{CLIP}_{\text{text}}(p_{O}),\text{CLIP}_{\text{text}}(p)),\end{split}$
(4)

where $\text{CLIP}_{\text{text}}$ may include new concepts learned through
DreamBooth [27] or Textual Inversion [13].

Similarly, to ensure semantic similarity to the latent representation of the
reference image $L_{r}=\text{VAE}_{\text{enc}}(I_{r})$ for reference-guided
image editing, we define

$\mathcal{L}_{ref}(L,L_{r})=\cos(\text{LatentCLIP}_{\text{vis}}(L),\text{LatentCLIP}_{\text{vis}}(L_{r}).$
(5)

Next, we want to enforce visual similarity between $L$ and $\tilde{L}_{0}$.
Following the same idea in LatentCLIP, we designed LatentVGG, a VGG encoder
[29] that operates in the latent domain, which is initialized and trained
along with a pre-trained VGG encoder. The training objective is

$\|\text{LatentVGG}(L),\text{VGG}(I)\|_{1}.$ (6)

After training, our perceptual loss is

$\mathcal{L}_{perc}(L,\tilde{L}_{0})=\|\text{LatentVGG}(L)-\text{LatentVGG}(\tilde{L}_{0})\|_{1}.$
(7)

To put everything together, our final loss function is

$\begin{split}&\mathcal{L}_{total}(L,\tilde{L}_{0},p_{O},p,\mathcal{A})\\\
=&\lambda_{sem}\cdot\mathcal{L}_{sem}(L,\tilde{L}_{0},p_{O},p)\\\
&+\lambda_{ref}\cdot\mathcal{L}_{ref}(L,L_{r})\\\
&+\lambda_{perc}\cdot\mathcal{L}_{perc}(L,\tilde{L}_{0}),\end{split}$ (8)

where $\lambda_{sem}=1$, $\lambda_{perc}=0.5$, and $\lambda_{ref}=1$ if
applicable.

In the end, TiNO-Edit (Alg. 1) optimize $N$ and all $t_{k}$’s for a pre-
defined $W$ number of optimization steps with respect to
$\mathcal{L}_{total}$, each of which consists of $K$ timesteps as one complete
denoising process.

Figure 5: Compounded image editing. We present a compounded image-editing
workflow by applying our method repeatedly on a single image. For each step,
the user can perform any of the supported editing operations. Any additional
information such as inputs including masks, reference images, user strokes,
user-composed images, and concept images used to train custom concepts are
shown next to the corresponding arrows.

##  5 Experiments

Original | T2L | DD | P2P0 | SINE | EDICT | IP2P | PnP | NTI | Ours
---|---|---|---|---|---|---|---|---|---
|  |  |  |  |  |  |  |  |
“a cat” $\rightarrow$ “a dog”
|  |  |  |  |  |  |  |  |
“roses” $\rightarrow$ “sunflowers”
|  |  |  |  |  |  |  |  |
$+$ “a bracelet”
|  |  |  |  |  |  |  |  |
$+$ “a magenta hat”
|  |  |  |  |  |  |  |  |
“oil painting ”$\rightarrow$ “watercolor”
|  |  |  |  |  |  |  |  |
“3d rendering” $\rightarrow$ “crochet”
Figure 6: Pure text-guided image editing comparison. We compare with the
following baselines: Text2LIVE (T2L) [2], DisentanglementDiffusion (DD) [33],
Pix2Pix-Zero (P2P0) [22], SINE [36], EDICT [32], InstructPix2Pix (IP2P) [3],
Plug-and-Play (PnP) [31], and Null-text Inversion (NTI) [21]. The user
indicates the desired editing via text, where the part that reflects the edit
is shown below each row.

TiNO-Edit uses SD v2.1 [25] with $K=10$ denoising steps, starting timestep
value $T=0.75$, and the number of optimization steps $W=50$. To use LatentCLIP
and LatentVGG for our loss function, each model has been trained on a subset
of LAION-5B [28] datasets for $10^{5}$ iterations with a batch size of $16$.
We use the AdamW [16] optimizer with a learning rate of $lr=\text{1e-5}$.

We use two AdamW [16] optimizers, one for $t_{k}$’s with $lr=1$ and the other
for $N$ with $lr=0.005$. Our method can also run on DreamBooth (DB) [27] or
Textual Inversion (TI) [13], where we use the same SD and noise scheduler
DB/TI is trained with to avoid performance degradation.

###  5.1 Compounded Image Editing

We present a compounded image editing workflow by applying our method
repeatedly on a single image with different operations(Fig. 5). For each step,
the user can perform any of the supported editing operations, which gives the
user creative control over how they want to edit images.

Original | Mask | Ref | VCT | GLIGEN | PbE | Ours
---|---|---|---|---|---|---
|  |  |  |  |  |
|  |  |  |  |  |
|  |  |  |  |  |
Figure 7: Comparison with reference-guided image editing baselines. We compare our method with Visual Concept Translator (VCT) [6], GLIGEN [17], and Paint-by-Example (PbE) [34], where the masks (Mask) indicate edit regions where the object in the reference image (Ref) should be included. Original | Strokes | BLD 0.5 | BLD 0.75 | SDE 0.5 | SDE 0.75 | Ours
---|---|---|---|---|---|---
|  |  |  |  |  |
$+$ “boots”
|  |  |  |  |  |
“chandelier” $\rightarrow$ “stained glass chandelier”
|  |  |  |  |  |
$+$ “an apple tree”
|  |  |  |  |  |
“hair” $\rightarrow$ “blue curly hair”
|  |  |  |  |  |
$+$ “a teddy bear”
|  |  |  |  |  |
$+$ “a gift box”
Figure 8: Stroke-based image editing and image composition baseline
comparison. We compare to Blended Latent Diffusion (BLD) [1] and SDEdit (SDE)
[20] for stroke-guided image editing (row 1-3) and image composition (row
4-6). BLD and SDEdit both accept a starting timestep $T$ as input, and we test
two values 0.5 and 0.75 (shown next to the method names).

###  5.2 Qualitative comparisons

Pure text-guided image editing. We compare TiNO-Edit with other text-guided
image editing baselines, including Text2LIVE (T2L) [2],
DisentanglementDiffusion (DD) [33], Pix2Pix-Zero (P2P0) [22], SINE [36], EDICT
[32], InstructPix2Pix (IP2P) [3], Plug-and-Play (PnP) [31], and Null-text
Inversion (NTI) [21] and test various use cases (Fig. 6). While some baselines
generate good results in sevaeral scenarios (e.g., EDICT rows 2, 4; NTI rows
2, 3, 5), many still pose various issues, including in-cohesive edit (T2L rows
3, 4, 5; DD row 4), failing to add new objects (P2P0, EDICT, PnP for row 3),
concept leaking (IP2P row 2: image becomes yellow; row 3: shirt is magenta-
colored; NTI row 4: man gets a hat texture), and altering other attributes
(P2P0, SINE, PnP row 4; NTI row 6). On the other hand, our method is robust
across use cases.

Reference-guided image editing. For reference-guided image editing, we compare
our method with Visual Concept Translator (VCT) [6], GLIGEN [17], and Paint-
by-Example (PbE) [34] (Fig. 7). our method is better at preserving details (a
dog with its tongue sticking out, white stripes on the hat) when maintaining
other image regions. In contrast, VCT failed to add objects, GLIGEN does not
compose edited objects to the original images (dog standing on the sofa) and
PbE generates objects that are similar to the references at a high level
without keeping their details.

Stroke-guided image editing. We compare to Blended Latent Diffusion (BLD) [1]
and SDEdit (SDE) [20] for stroke-guided image editing (row 1-3 of Fig. 8),
both of which take a starting timestep as input. BLD largely generates results
that do not align with the strokes and the same problem appears in SDEdit. On
the other hand, TiNO-Edit can generate objects from strokes that blend well
with the original.

Image composition. We also compare with BLD and SDEdit for image composition
use cases (Fig. 8 row 4-6), where TiNO-Edit refine user-composed images and
generate cohesive and nature-looking outputs (Fig. 8) such as making the teddy
bear sit on the sofa rather than floating in front of it. Again, BLD changes
the user input drastically. SDEdit with the starting timestep of 0.75 largely
omits the new object that should be added to the results. While SDEdit with
the starting timestep of 0.5 generates better results, the output may not look
cohesive.

###  5.3 Quantitative Comparisons

T2L DD P2P0 SINE EDICT IP2P PnP NTI Ours CLIP-T 0.299 0.316 0.271 0.314 0.327
0.315 0.321 0.316 0.328 CLIP-I 0.879 0.853 0.793 0.912 0.898 0.845 0.835 0.843
0.924 DINO-I 0.864 0.862 0.776 0.805 0.838 0.828 0.756 0.799 0.874

Table 1: Pure text-guided image editing qualitative evaluation. |  VCT |  GLIGEN |  PbE |  Ours
---|---|---|---|---
CLIP-T |  0.267 |  0.295 |  0.300 |  0.311
CLIP-I |  0.899 |  0.814 |  0.841 |  0.928
$\text{CLIP-I}_{r}$ |  0.537 |  0.616 |  0.570 |  0.674
DINO-I |  0.919 |  0.770 |  0.813 |  0.869
Table 2: Reference-guided image editing qualitative evaluation | BLD 0.5 | BLD 0.75 | SDEdit 0.5 | SDEdit 0.75 | Ours
---|---|---|---|---|---
CLIP-T | 0.298 | 0.287 | 0.295 | 0.295 | 0.298
CLIP-I | 0.930 | 0.924 | 0.897 | 0.953 | 0.959
$\text{CLIP-I}_{s}$ | 0.956 | 0.927 | 0.986 | 0.956 | 0.987
DINO-I | 0.953 | 0.959 | 0.961 | 0.973 | 0.978
Table 3: Stroke-guided image editing qualitative evaluation |  BLD 0.5 |  BLD 0.75 |  SDEdit 0.5 |  SDEdit 0.75 |  Ours
---|---|---|---|---|---
CLIP-T |  0.273 |  0.273 |  0.294 |  0.267 |  0.298
CLIP-I |  0.955 |  0.950 |  0.903 |  0.937 |  0.958
$\text{CLIP-I}_{c}$ |  0.943 |  0.945 |  0.965 |  0.964 |  0.978
DINO-I |  0.977 |  0.975 |  0.972 |  0.973 |  0.983
Table 4: Image composition qualitative evaluation

We created a test set of 200 samples in the form of
$(I,p_{O},p,I_{r},I_{s},I_{c})$, where we ran each method for each sample and
report the following popular metrics averaged across the test set ( Tabs. 1,
2, 3 and 4):

  * •

CLIP-T: $\cos(\text{CLIP}\text{vis}(\tilde{I}),\text{CLIP}\text{text}(p))$

  * •

CLIP-I:
$\cos(\text{CLIP}_{\text{vis}}(\tilde{I}),\text{CLIP}_{\text{vis}}(I))$

  * •

$\text{CLIP-I}_{*}$:
$\cos(\text{CLIP}_{\text{vis}}(\tilde{I}),\text{CLIP}_{\text{vis}}(I_{*}))$,
$I_{*}\in\\{I_{r},I_{s},I_{c}\\}$

  * •

DINO-I:
$\cos(\text{DINO}_{\text{vis}}(\tilde{I}),\text{DINO}_{\text{vis}}(I))$

where $\tilde{I}$ is the output, $\text{DINO}_{\text{vis}}$ refers to DINO ViT
[5] which can be used to evaluate image perceptual alignment. Our method
outperforms other methods in most metrics, including DINO-I, a non-CLIP-based
metric not used during training. In Tab. 2, VCT scores the highest in DINO-I
as its outputs are very similar to the original images without respecting the
target prompts/images (Fig. 7), which is also supported by the fact that it
gets the worst CLIP-T score.

###  5.4 Ablation studies

---
Figure 9: Changes in timesteps and noise through optimization. We plot the values of $t_{k}$’s alongside the minimum, maximum, and mean of $N$ across $W$ optimization steps. This indicates that $N$ remains within the noise distribution that SD has been learned, preventing potential degradation of the output image. |  |  |  |  |  |
---|---|---|---|---|---|---
const $t_{k}$ |  |  |  |  |  |
const $N$ |  |  |  |  |  |
$M=\textbf{1}$ |  |  |  |  |  |
| W=0 | W=10 | W=20 | W=30 | W=40 | W=50
Figure 10: Effect of optimizing timesteps, noise, and using the masking mechanism. Optimizing both $t_{k}$’s and $N$ leads to optimize results across different number of optimization steps. Original | w/ latent | w/ pixel | Concept | w/ latent | w/ pixel
---|---|---|---|---|---
|  |  |  |  |
“Audrey Hepburn” $\rightarrow$ “Marilyn Monroe” |  “Audrey Hepburn” $\rightarrow$ $\langle concept\rangle$
Figure 11: Effect of LatentCLIP and LatentVGG on image editing quality and
custom concepts. Compared with the original CLIP [24] and VGG [29] that
operate in pixel spaces (“w/ pixel”), using our LatentCLIP and LatentVGG (“w/
latent”) is not only more efficient but leads to better quality outputs.

Change in timesteps and noise throughout optimization. We look at how
$t_{k}$’s and N change across W optimization steps and average the values
across our test set in Fig. 9. Timesteps earlier in the diffusion process are
changed more compared to timesteps later in the diffusion process because the
former has a larger effect on the editing result. The range and the mean of
$N$ is quite stable over time because they are optimized with respect to a
lower learning rate and handle more local changes with respect to the changing
$t_{k}$’s. Stability in $N$ also means that we don’t need to worry about $N$
going outside the noise distribution SD has learnt and cause performance
degrade.

Effect of optimizing timesteps, noise, and using the masking mechanism. We
first study the effect of optimizing timesteps, noise, and using the masking
mechanism by generating results with different optimization configuration with
a fixed seed (Fig. 10). As we can see, TiNO-Edit is the most robust across
various $W$. Keeping $t_{k}$’s constant causes the outputs to degrade over
time because when the timestep values are sub-optimal, $N$ has to change more
drastically to over-compensate for our loss function, which pushes it away
from the Gaussian noise distribution that SD learns. Omitting optimizing $N$
has a smaller effect on the results with larger $W$ but is more prone to
artifacts (the shadow around the coat collar). Moreover, when $W$ is small, we
may get drastically different images compared to the original. Lastly, when
the editing region is not constrained by the mask $M$, the global structure of
the image may get changed entirely. This suggest that optimizing both
$t_{k}$’s and $N$ is necessary for optimal results.

Effect of LatentCLIP and LatentVGG. We study the effect of using LatentCLIP
and LatentVGG compared to using the original CLIP [24] and VGG [29] on a
NVIDIA A6000. In terms of image editing quality, using our LatentCLIP and
LatentVGG leads to high-quality image editing results with better alignment
with the original image and less artifacts (Fig. 11 row 1) while performing
better with custom concepts in Stable Diffusion variants such as Textual
Inversion [13] (Fig. 11 row 2).

##  6 Conclusion

In this paper, we have presented TiNO-Edit, an image editing method with pre-
trained Stable Diffusion models by optimizing diffusion timesteps and input
noise. We have designed a set of loss functions that greatly reduce the
computation resources required for optimization. Comparing TiNO-Edit with
various task-specifically baselines, our method is able to generalize across
specific image editing tasks and produce superior results.

## Appendix

##  Appendix A Additional Results

Here we show more results, including pure text-guided image editing (Fig. 12),
reference-guided image editing (Fig. 13), stroke-guided image editing (Fig.
14), and image composition (Fig. 15). Our method can be applied for any of the
above use cases with custom concepts in DreamBooth [27] and Textual Inversion
[13] (Fig. 16).

Original | Result
---|---
|
“winter” $\rightarrow$ “spring”

Original | Mask | Result
---|---|---
|  |
$+$ “an orange saddle”

Original | Result
---|---
|
“vector art” $\rightarrow$ “photo”

Figure 12: Pure text-guided image editing examples. We can use text to perform
various image editing operations including changing object attributes (row 1),
adding objects (with a mask indicating the location; row 2), or changing image
style (row 3). The desired editing is indicated via text prompts, where the
part that reflects the editing is shown below each sample.

Original | Reference | Result
---|---|---
|  |
“dolphin” $\rightarrow$ “fish”
|  |
“a sports car” $\rightarrow$ “a classic car”

Original | Mask | Reference | Result
---|---|---|---
|  |  |
$+$ “sunglasses”

Figure 13: Reference-guided image editing examples. Our method can take a reference image and either replace objects (row 1,2) or add objects in the image region indicated by the masks (row 3) to the corresponding objects in the reference image. The desired editing is indicated via text prompts, where the part that reflects the editing is shown below each sample. Original | User input | Result
---|---|---
|  |
“blouse” $\rightarrow$ “plaid blouse”
|  |
$+$ “a doll”
|  |
$+$ “a tower”
Figure 14: Stroke-guided image editing examples. Our method is able to take user strokes and edit the image accordingly, where the strokes will be converted based on the input text prompts, where the part that reflects the editing is shown below each sample. It can change existing objects in the image (row 1) as well as add new objects (rows 2-3) to the images. Original | User input | Result
---|---|---
|  |
“cartoon wheel” $\rightarrow$ “wheel”
|  |
$+$ “a bow tie”
|  |
$+$ “a dog”
Figure 15: Image composition examples. Our method is able to take a user-composed image and harmonize it. For example, we can adjust the position and the lighting of the composed car wheel (row 1) and the bow tie (row 2) automatically so they blend naturally as part of the image rather than floating on top of their respective images. Similarly, we can add a photo of a dog to a drawing and change its style to fit more closely to the drawing itself. Original | Concept (DB) | Result
---|---|---
|  |
woman $\rightarrow$ $\langle concept\rangle$
Original | Concept (TI) | Result
|  |
“photo” $\rightarrow$ $\langle concept\rangle$
Figure 16: Image editing with DreamBooth and Textual Inversion. Our method can
be applied with DreamBooth (DB) [27] or Textual Inversion (TI) [13] and
incorporate their corresponding custom concepts in the image editing
capabilities. The desired editing is indicated via text prompts with the token
corresponding to the concept, where the part that reflects the editing is
shown below each sample and the token is denoted as $\rightarrow$ $\langle
concept\rangle$.

##  Appendix B Additional Ablations

###  B.1 Effect of user inputs

Original | User input | Result
---|---|---
|  |
|  |
$+$ “a doll”
Figure 17: Effect of user inputs. We can see the effect of user inputs on editing results in a stroke-guided image editing use case. Here the user provides input strokes and intends to convert them into “a doll”. When the strokes contain a smiley face, it is converted into a realistic-looking face of a doll. When the smiley face strokes are omitted, our method interprets it as the side view of the face of the doll. Concept | w/ original NS | w/ DDIM
---|---|---
|  |
“a cat, $\langle concept\rangle$”
|  |
“a girl, $\langle concept\rangle$”
Figure 18: Effect of noise schedulers with custom concepts. When editing with
custom concepts in DreamBooth (DB) [27] and/or Textual Inversion (TI) [13], we
use the same noise scheduler they are trained with because the concepts can
get lost otherwise. Take a look at Stable Diffusion [12] T2I outputs (input
text prompts shown below each sample) using DB (row 1) or TI (row 2) with
their original noise schedulers (NS) vs. the DDIM [30] scheduler. The style of
outputs match more closely to the concepts using the original noise scheduler.

We explore the effect of user inputs on the editing results by looking at a
stroke-guided image editing use case (Fig. 17). Here the user provides input
strokes and intends to convert them into “a doll”. When the input strokes
contain a smiley face, our method successfully converts it into a realistic-
looking face of a doll. When the smiley face strokes are omitted in the input
strokes, our method interprets this as a side view of the doll’s face,
generating results accordingly.

###  B.2 Effect of noise schedulers

When editing with custom concepts in DreamBooth (DB) [27] or Textual Inversion
(TI) [13], we use the same noise scheduler as when these concepts are trained
with their respective models rather than using the default DDIM scheduler [30]
because the concepts can get degraded or lost with a different noise scheduler
as shown in Fig. 18.

###  B.3 Effect of LatentCLIP and LatentVGG

| w/ latent | w/ pixel
---|---|---
Average GPU RAM usage (GB) | 22 | 41
Average optimization time (s) | 63 | 150
Table 5: Effect of LatentCLIP and LatentVGG on computational requirement. We
study the computational requirement of our method using LatentCLIP and
LatentVGG (col “w/ latent”) as well as the original CLIP [24] and VGG [29]
(col “w/ pixel”) on NVIDIA A6000 (48GB), where we average the GPU memory usage
and optimization time across 50 trials. Our method with its LatentCLIP and
LatentVGG requires only around 50% of GPU memory and time compared to using
the original CLIP and VGG.

We study the computational requirement of our method using LatentCLIP and
LatentVGG compared to using the original CLIP [24] and VGG [29] by averaging
the GPU memory usage and optimization time of our method across 50 trials
(Tab. 5). Our method with its LatentCLIP and LatentVGG requires only around
50% of GPU memory and time compared to using the original CLIP and VGG, as the
latter operates in the image pixel domain which is much larger than the SD
latent domain.

## References

  * Avrahami et al. [2023] Omri Avrahami, Ohad Fried, and Dani Lischinski.  Blended latent diffusion.  _ACM Transactions on Graphics (TOG)_ , 42(4):1–11, 2023.
  * Bar-Tal et al. [2022] Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kasten, and Tali Dekel.  Text2LIVE: Text-driven layered image and video editing.  In _European Conference on Computer Vision (ECCV)_ , pages 707–723. Springer, 2022.
  * Brooks et al. [2023] Tim Brooks, Aleksander Holynski, and Alexei A Efros.  InstructPix2Pix: Learning to follow image editing instructions.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 18392–18402, 2023.
  * Brown et al. [2020] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in Neural Information Processing Systems (NeurIPS)_ , 33:1877–1901, 2020.
  * Caron et al. [2021] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  In _International Conference on Computer Vision (ICCV)_ , 2021.
  * Cheng et al. [2023] Bin Cheng, Zuhao Liu, Yunbo Peng, and Yue Lin.  General image-to-image translation with one-shot image guidance.  In _International Conference on Computer Vision (ICCV)_ , pages 22736–22746, 2023.
  * Cherti et al. [2023] Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev.  Reproducible scaling laws for contrastive language-image learning.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2818–2829, 2023.
  * Couairon et al. [2022] Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord.  DiffEdit: Diffusion-based semantic image editing with mask guidance.  _arXiv preprint arXiv:2210.11427_ , 2022.
  * Crowson et al. [2022] Katherine Crowson, Stella Biderman, Daniel Kornis, Dashiell Stander, Eric Hallahan, Louis Castricato, and Edward Raff.  VQGAN-CLIP: Open domain image generation and editing with natural language guidance.  In _European Conference on Computer Vision (ECCV)_ , pages 88–105. Springer, 2022.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Nichol.  Diffusion models beat GANs on image synthesis.  _Advances in Neural Information Processing Systems (NeurIPS)_ , 34:8780–8794, 2021.
  * Epstein et al. [2023] Dave Epstein, Allan Jabri, Ben Poole, Alexei A. Efros, and Aleksander Holynski.  Diffusion self-guidance for controllable image generation.  _arXiv preprint arXiv:2306.00986_ , 2023.
  * Esser et al. [2021] Patrick Esser, Robin Rombach, and Bjorn Ommer.  Taming transformers for high-resolution image synthesis.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 12873–12883, 2021.
  * Gal et al. [2022] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or.  An Image is Worth One Word: Personalizing text-to-image generation using textual inversion.  _arXiv preprint arXiv:2208.01618_ , 2022.
  * Gao et al. [2023] Shanghua Gao, Zhijie Lin, Xingyu Xie, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan.  EditAnything: Empowering unparalleled flexibility in image editing and generation.  In _ACM International Conference on Multimedia (ACMMM)_ , 2023.
  * HuggingFace [2023] HuggingFace.  HuggingFace Stable diffusion image-to-image, 2023.  https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/img2img.
  * Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Li et al. [2023] Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee.  GLIGEN: Open-set grounded text-to-image generation.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 22511–22521, 2023.
  * Lüddecke and Ecker [2022] Timo Lüddecke and Alexander Ecker.  Image segmentation using text and image prompts.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 7086–7096, 2022.
  * Luo et al. [2023] Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao.  Latent consistency models: Synthesizing high-resolution images with few-step inference.  _arXiv preprint arXiv:2310.04378_ , 2023.
  * Meng et al. [2022] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.  SDEdit: Guided image synthesis and editing with stochastic differential equations.  In _International Conference on Learning Representations (ICLR)_ , 2022.
  * Mokady et al. [2023] Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or.  Null-text inversion for editing real images using guided diffusion models.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 6038–6047, 2023.
  * Parmar et al. [2023] Gaurav Parmar, Krishna Kumar Singh, Richard Zhang, Yijun Li, Jingwan Lu, and Jun-Yan Zhu.  Zero-shot image-to-image translation.  In _SIGGRAPH_ , pages 1–11, 2023.
  * Podell et al. [2023] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach.  SDXL: Improving latent diffusion models for high-resolution image synthesis.  _arXiv preprint arXiv:2307.01952_ , 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International Conference on Machine Learning (ICML)_ , pages 8748–8763. PMLR, 2021.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 10684–10695, 2022.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-Net: Convolutional networks for biomedical image segmentation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_ , pages 234–241. Springer, 2015.
  * Ruiz et al. [2023] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.  DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 22500–22510, 2023.
  * Schuhmann et al. [2022] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al.  LAION-5B: An open large-scale dataset for training next generation image-text models.  _Advances in Neural Information Processing Systems (NeurIPS)_ , 35:25278–25294, 2022.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ , 2014.
  * Song et al. [2020] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  _arXiv preprint arXiv:2010.02502_ , 2020.
  * Tumanyan et al. [2023] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel.  Plug-and-play diffusion features for text-driven image-to-image translation.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 1921–1930, 2023.
  * Wallace et al. [2023] Bram Wallace, Akash Gokul, and Nikhil Naik.  EDICT: Exact diffusion inversion via coupled transformations.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 22532–22541, 2023.
  * Wu et al. [2023] Qiucheng Wu, Yujian Liu, Handong Zhao, Ajinkya Kale, Trung Bui, Tong Yu, Zhe Lin, Yang Zhang, and Shiyu Chang.  Uncovering the disentanglement capability in text-to-image diffusion models.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 1900–1910, 2023.
  * Yang et al. [2023] Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen.  Paint by Example: Exemplar-based image editing with diffusion models.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 18381–18391, 2023.
  * Zhang et al. [2023a] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.  Adding conditional control to text-to-image diffusion models.  In _International Conference on Computer Vision (ICCV)_ , pages 3836–3847, 2023a.
  * Zhang et al. [2023b] Zhixing Zhang, Ligong Han, Arnab Ghosh, Dimitris N Metaxas, and Jian Ren.  SINE: Single image editing with text-to-image diffusion models.  In _Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 6027–6037, 2023b.


# A Rigidity Result for minimal Rotation hypersurfaces of 5d spaces of
constant curvature

Aaron J. Tyrrell

(Date: April 2023)

###### Abstract.

In this paper we show that a particular extrinsic pointwise hypersurface
invariant is always non-positive on minimal hypersurfaces of constant
curvature spaces and vanishes identically if and only if the hypersurface is
rotational. We show this for hypersurfaces of 5-dimensional spaces of constant
curvature but we conjecture that this should generalize to a similar result in
other dimensions.

##  1\. introduction

Let $M^{5}(c)$ be a simply connected, complete Riemannian manifold with
constant curvature $c,$ where $c$ is a real number. Do Carmo and Dajczer [1]
studied rotational hypersurfaces of such manifolds and a consequence of
Corollary $4.4$ of their work is that a non-totally geodesic minimal
hypersurface of $M^{5}(c)$ is rotational if and only if it has exactly $3$
principal curvatures which are equivalent. Some other work on rotational
hypersurfaces can be found here [3], [2].

##  2\. Statement and proof of result

######  Theorem 2.1 (Rotational Rigidity).

Let $N$ be a minimal hypersurface of $M^{5}(c).$ Let $A$ be the second
fundamental form of $N.$ Then define

(2.1) $S(N):=|A^{2}|^{2}-\frac{7}{12}|A|^{4},$

then

$S(N)(p)\leq 0,\quad\forall\hskip 2.84526pt{p\in N}$

with

$S(N)\equiv 0$

if and only if $N$ is a rotational hypersurface.

######  Remark 2.2.

This gives another way to view minimal rotation hypersurfaces of $M^{5}(c).$
This is as minimizers amongst minimal hypersurfaces of the energy

(2.2) $E_{rot}[N]:=\int_{N}\frac{7}{12}|A|^{4}-|A^{2}|^{2}dA,$

where $dA$ is the area form of the induced metric on $N.$

###### Proof.

Given $p\in N,$ let $\lambda_{1},\lambda_{2},\lambda_{3},\lambda_{4}$ be the
principal curvatures of $N$ at $p.$ Then letting

(2.3) $\displaystyle SN(p)$
$\displaystyle=\lambda_{1}^{4}+\lambda_{2}^{4}+\lambda_{3}^{4}+\lambda_{4}^{4}-\frac{7}{12}(\lambda_{1}^{2}+\lambda_{2}^{2}+\lambda_{3}^{2}+\lambda_{4}^{2})^{2}$
$\displaystyle=:s(\lambda_{1},\lambda_{2},\lambda_{3},\lambda_{4}).$

We can apply Lagrange multipliers to $s(x,y,z,w)$ subject to the constraint
$x+y+z+w=0$ to get the system:

(2.4) $\displaystyle 4x^{3}-\frac{28}{12}||x||^{2}x$ $\displaystyle=\lambda$
(2.5) $\displaystyle 4y\textsterling^{3}-\frac{28}{12}||x||^{2}y$
$\displaystyle=\lambda$ (2.6) $\displaystyle 4z^{3}-\frac{28}{12}||x||^{2}z$
$\displaystyle=\lambda$ (2.7) $\displaystyle 4w^{3}-\frac{28}{12}||x||^{2}w$
$\displaystyle=\lambda$ (2.8) $\displaystyle x+y+z+w$ $\displaystyle=0,$

where $||x||^{2}:=x^{2}+y^{2}+z^{2}+w^{2}.$ We also define
$x_{1}:=x,x_{2}:=y,x_{3}:=z,x_{4}:=w.$

By the symmetry of these equations we see that the set of points satisfying
these equations simultaneously is preserved under reflections through the
hyperplanes $\\{x_{i}=x_{j}:\hskip 2.84526pti\neq j\\}$. A natural approach is
to set $x=y=z$ and try to solve. This yields $w=-3x,z=y,x=y$ and

$-108x^{3}+84x^{3}=\lambda,$ $x^{3}=-\frac{\lambda}{24},$

and so

$(x,y,z,w)=-\sqrt[3]{\lambda}\big{(}\frac{1}{\sqrt[3]{24}},\frac{1}{\sqrt[3]{24}},\frac{1}{\sqrt[3]{24}},-\frac{\sqrt[3]{9}}{\sqrt[3]{8}}\big{)}.$

By the scale invariance of $(2.4)-(2.8)$ we can let $\lambda=24\alpha^{3}$
where $\alpha\in\mathbb{R}$ and we get critical points

$\alpha(-1,-1,-1,3)\hskip 5.69054pt\forall\hskip
2.84526pt\alpha\in\mathbb{R}.$

Observe

$s(\alpha(-1,-1,-1,3))=0.$

Clearly we could have chosen any one of $x,y,z,w$ to not be equal to the
others, and therefore we get a critical point at $\alpha\sigma(-1,-1,-1,3)$
for any $\alpha\in\mathbb{R}$ and $\sigma\in S_{4}$ where $S_{4}$ is the
symmetric group on 4 letters. Now to see that $0$ is the largest critical
value, first consider that at a critical point $(x,y,z,w)$ subject to our
constraint with $x,y,z$ distinct, $(2.4)$ and $(2.5)$ are equal so we must
have

(2.9) $4x^{3}-\frac{28}{12}||x||^{2}x=4y^{3}-\frac{28}{12}||x||^{2}y,$

therefore

(2.10) $\displaystyle 4(x^{3}-y^{3})$
$\displaystyle=\frac{28}{12}||x||^{2}(x-y).$

Factoring the left hand-side of $(2.10)$ gives

$\displaystyle 4(x-y)(x^{2}+xy+y^{2})$
$\displaystyle=\frac{28}{12}||x||^{2}(x-y),$

recall $x\neq y$ so

(2.11) $(x^{2}+xy+y^{2})=\frac{7}{12}(x^{2}+y^{2}+z^{2}+w^{2}).$

The right hand-side of $(2.11)$ is independent of the choice of equations we
chose to set equal in $(2.9)$ and since $x\neq z$ we get

$x^{2}+xy+y^{2}=x^{2}+xz+z^{2}$

this implies

$x(y-z)+(y-z)(y+z)=0,$

and therefore

$(y-z)(x+y+z)=0$

thus, since $y\neq z$ we get

(2.12) $w=0.$

This implies that $\lambda=0,$ now if any two of $x,y,z$ is non-zero then it
follows that they differ by a sign. To see this take $(2.4)$ and see that

(2.13) $4x^{2}=\frac{28}{12}||x||^{2},$

since the right hand-side of $(2.13)$ is independent of which of $(2.4)$,
$(2.5),$ $(2.6)$ and $(2.7)$ we chose we get that $x_{i}^{2}=x_{j}^{2}$ if
$x_{i}\neq 0$ and $x_{j}\neq 0.$ The only possibility is that two of $x,y,z$
are additive inverses and the other is $0.$ WLOG assume $x=-y$ then

(2.14) $\displaystyle s(x,-x,0,0)$ $\displaystyle=2x^{4}-\frac{7}{3}x^{4}$
$\displaystyle=-\frac{1}{3}x^{4}.$

Therefore any constrained critical point with 3 distinct coordinates is not a
global maximum (we only considered the case where $x,y$ and $z$ are distinct
but by symmetry this covers every case where three of the coordinates are
distinct).

We are left to consider points of the form $\sigma(x,y,x,y)$ for $\sigma\in
S_{4}$. Now by $(2.8)$ we must have $x=-y$ which implies the point has the
form $(x,-x,x,-x).$ Clearly then

$\displaystyle s(x,-x,x,-x)$ $\displaystyle=4x^{4}-\frac{28}{3}x^{4}$
$\displaystyle=-\frac{26}{3}x^{4}$

we see that these cannot be global maximums either.

Now to be sure that $0$ is the global maximum value of $s(x,y,z,-x-y-z),$
consider the following

(2.15) $s(t(x,y,z,w))=t^{4}s(x,y,z,w).$

Therefore to prove that $0$ is the global maximum value it is enough to prove
that $0$ is the maximum on a neighborhood of the origin. We will consider the
closed unit ball in $\mathbb{R}^{4}$ intersected with our constraint
hyperplane: $x+y+z+w=0.$ By homogeneity, if there is point
$p\in\\{(x,y,z,w):x+y+z+w=0\\}$ such that $s(p)>0$ then $s$ will have to
achieve its maximum on the boundary of the ball. Assume for the sake of
contradiction that this maximum is achieved at
$\underline{x}\in\mathbb{S}^{4}\cap\\{(x,y,z,w):x+y+z+w=0\\}.$ By homogeneity
of the function we get

(2.16) $\displaystyle
4s(\underline{x})=\sum_{i}\partial_{i}s(\underline{x})x_{i}$

and by Lagrange multipliers we get that there exists
$\lambda,\mu\in\mathbb{R}$ such that

(2.17) $\partial_{i}s(\underline{x})=\lambda+2\mu x_{i}\quad\forall\hskip
2.84526pti\in\\{1,2,3,4\\}.$

Using $(2.16)$ and $(2.17)$ we can derive

(2.18) $4s(\underline{x})=2\mu.$

Suppose that $\underline{x}$ has one coordinate which is $0,$ wlog say $x=0.$
Then

$\displaystyle\partial_{x}s(x,y,z,w)$
$\displaystyle=4x^{3}-\frac{28}{12}||x||^{2}x$ $\displaystyle=0.$

Therefore by $(2.17)$

$\lambda=0.$

This implies that $(2.17)$ can be written as

(2.19) $\partial_{i}s(\underline{x})=2\mu x_{i}.$

Now if $\underline{x}$ has another coordinate equal to zero then the value of
$s(\underline{x})$ will be negative by $(2.14),$ so suppose $y,z,w$ are non-
zero. Now exactly two of the non-zero coordinates must have the same sign.
Assume $y\geq z>0.$

Next consider

$\displaystyle\partial_{z}s(x,y,z,w)$
$\displaystyle=4z^{3}-\frac{28}{12}||x||^{2}z$
$\displaystyle=4z^{3}-\frac{28}{12}(x^{2}+y^{2}+z^{2}+w^{2})z$
$\displaystyle\leq
4z^{3}-\frac{28}{12}z^{3}-\frac{28}{12}(y^{2}+z^{2}+x^{2})z$
$\displaystyle<0.$

Therefore by $(2.19)$

$2\mu z<0,$

this implies that

$\mu<0$

which is a contradiction to $(2.18).$ We can make a similar argument in the
case where two of $y,z,w$ are negative as follows, suppose $0>z\geq w,$ then

$\displaystyle\partial_{z}s(x,y,z,w)$
$\displaystyle=4z^{3}-\frac{28}{12}||x||^{2}z$
$\displaystyle=4z^{3}-\frac{28}{12}(x^{2}+y^{2}+z^{2}+w^{2})z$
$\displaystyle\geq
4z^{3}-\frac{28}{12}z^{3}-\frac{28}{12}(y^{2}+z^{2}+x^{2})z$
$\displaystyle>0.$

Therefore by $(2.19)$

(2.20) $2\mu z>0,$

this implies that

$\mu<0$

which is a contradiction to $(2.18).$ Therefore $\underline{x}$ can only have
non-zero coordinates.

Suppose that $\underline{x}$ has two positive coordinates and two negative
coordinates, wlog assume that the positive coordinates are $x$ and $y$ and
assume wlog $x\geq y$ and that the negative coordinates are $z,w$ with $z\geq
w.$ Then

$\displaystyle\partial_{y}s(x,y,z,w)$
$\displaystyle=4y^{3}-\frac{28}{12}||x||^{2}y$
$\displaystyle=4y^{3}-\frac{28}{12}(x^{2}+y^{2}+z^{2}+w^{2})y$
$\displaystyle\leq
4y^{3}-\frac{28}{12}y^{3}-\frac{28}{12}(y^{2}+z^{2}+w^{2})y$
$\displaystyle<0.$

Therefore

$\lambda+2\mu y<0,$

this implies that

$\lambda<0.$

Next consider

$\displaystyle\partial_{z}s(x,y,z,w)$
$\displaystyle=4z^{3}-\frac{28}{12}||x||^{2}z$
$\displaystyle=4z^{3}-\frac{28}{12}(x^{2}+y^{2}+z^{2}+w^{2})z$
$\displaystyle\geq
4z^{3}-\frac{28}{12}z^{3}-\frac{28}{12}(y^{2}+z^{2}+x^{2})z$
$\displaystyle>0.$

Therefore

$\lambda+2\mu z>0,$

this implies that

$\lambda>0.$

Therefore we have reached a contradiction and hence $\underline{x}$ cannot
have two positive coordinates and two negative coordinates.

Now we know that $\underline{x}$ must have $3$ coordinates of the same sign
and the $4th$ must be of the opposite. Suppose $x\geq y\geq z>0$ and
$w=-x-y-z.$ Then

(2.21) $\partial_{x}s(\underline{x})=4x^{3}-\frac{7}{3}x=\lambda+2\mu x$

which implies

(2.22) $4x^{3}-\frac{7}{3}x-2\mu x=\lambda$

the right hand-side of $(2.22)$ is independent of which variable we chose in
$(2.21)$ so we get

(2.23) $4x_{i}^{3}-\frac{7}{3}x_{i}-2\mu
x_{i}=4x_{j}^{3}-\frac{7}{3}x_{j}-2\mu x_{j}.$

It follows that

(2.24) $4(x_{i}^{3}-x_{j}^{3})-(\frac{7}{3}+2\mu)(x_{i}-x_{j})=0$

now if $x_{i}\neq x_{j}$ then dividing by $x_{i}-x_{j}$ gives

(2.25) $4(x_{i}^{2}+x_{i}x_{j}+x_{j}^{2})-(\frac{7}{3}+2\mu)=0$

therefore

(2.26) $4(x_{i}^{2}+x_{i}x_{j}+x_{j}^{2})=(\frac{7}{3}+2\mu)$

now since the right hand-side is independent of the choice of coordinates we
get that if $x,y$ and $z$ are distinct then

(2.27) $\displaystyle 4(x^{2}+xy+y^{2})$ $\displaystyle=\frac{7}{3}+2\mu$
(2.28) $\displaystyle 4(x^{2}+xw+w^{2})$ $\displaystyle=\frac{7}{3}+2\mu$

writing $w=-x-y-z$ and taking the average of $(2.27)$ and $(2.28)$ gives

$2(x^{2}-xz+w^{2}+y^{2})=\frac{7}{3}+2\mu.$

We could have swapped the roles of $x$ and $z$ and gotten

$2(z^{2}-xz+w^{2}+y^{2})=\frac{7}{3}+2\mu.$

Which implies

$x=z.$

Therefore $x,y$ and $z$ cannot be distinct.

Now suppose two of the coordinates are equal, wlog $x=z,$ then if $y\neq x$ we
get the following equations

(2.29) $\displaystyle 4(y^{2}+yz+z^{2})$ $\displaystyle=\frac{7}{3}+2\mu$
(2.30) $\displaystyle 4(z^{2}+zw+w^{2})$ $\displaystyle=\frac{7}{3}+2\mu$
(2.31) $\displaystyle 4(y^{2}+yw+w^{2})$ $\displaystyle=\frac{7}{3}+2\mu$

Writing $w=-x-y-z$ and taking the average of equations $(2.29)$ and $(2.30)$
gives

$2(y^{2}-zx+w^{2}+z^{2})=\frac{7}{3}+2\mu.$

Taking the average of $(2.29)$ and $(2.31)$ gives

$2(y^{2}-yx+z^{2}+w^{2})=\frac{7}{3}+2\mu.$

This implies that $y=z.$

Therefore $\underline{x}$ is of the form $\alpha\sigma(1,1,1,-3)$ for
$\alpha\in\mathbb{R_{+}}$ and $\sigma\in S_{4}$ and we get

$s(\underline{x})=0,$

which is a contradiction. If $x,y$ and $z$ were negative instead of positive
then we would still get a contradiction since the point $-\underline{x}$ would
also be a local maximum on which $s$ has the same value by symmetry but by
what we’ve just argued we would get $s(\underline{x})=0.$ Now by corollary
$4.4$ of [1] we get the result. ∎

## References

  * [1] Manfredo P Do Carmo, M do Carmo, and M Dajczer, _Rotation hypersurfaces in spaces of constant curvature_ , Springer, 2012.
  * [2] Gilbert Levitt and Harold Rosenberg, _Symmetry of constant mean curvature hypersurfaces in hyperbolic space_ , (1985).
  * [3] Biao Wang, _Simons’ equation and minimal hypersurfaces in space forms_ , Proceedings of the American Mathematical Society 146 (2018), no. 1, 369–383.