# Fibonacci sequence and Pythagorean triples in the composition of functions
for integer solutions from certain operators

Pablo José Vega Esparza
Banner Code:00336820

### Abstract

The following paper synthesizes research where theorems and their respective
proofs are postulated based on quadratic equations with special properties
given by Phytagorean triples and Fibonacci sequence and an environment in
number theory and its applications to calculus. It is important to read the
document carefully to build a clear and comprehensive understanding of what is
being done.

##

Integer solutions to an equation are of great importance in number theory. The
first question to ask is: What conditions must be met for a quadratic equation
to have two real integer solutions?

$\displaystyle ax^{2}+bx+c=0$

The following conditions are proposed to find two integer and real roots:

formula(1):

$\boxed{\begin{aligned} &a:\text{Leg of Pythagorean triple}\\\ \\\
&b=2a\psi,\quad\psi:\text{Hypotenuse of Pythagorean triple}\\\ \\\ &c=a^{3}\\\
\end{aligned}}$

$\displaystyle ax^{2}+bx+c=0,\quad x\in\mathbb{Z}$

Proof:

Let $a,b,c$ belong to integers.

If $x=\frac{{-b\pm\sqrt{b^{2}-4ac}}}{{2a}}$,

Then, $x=-\frac{b}{2a}\pm\frac{\sqrt{b^{2}-4ac}}{2a}$.

With $b$, it is deduced that it must have among its factors $a$, along with
$2$, so $b=2\cdot a\cdot\psi$, where $\psi$ is the factor by which $b$ will be
a multiple of $a$ along with 2, and therefore, proportional.

$\implies x=-\frac{2\cdot\psi\cdot a}{2\cdot
a}\pm\frac{1}{2}\sqrt{\frac{(2\cdot\psi\cdot a)^{2}-2^{2}\cdot a\cdot
c}{a^{2}}}$ $\implies x=-\frac{2\cdot\psi\cdot a}{2\cdot
a}\pm\sqrt{\psi^{2}-\frac{c}{a}}$ $\implies
x=-\psi\pm\sqrt{\psi^{2}-\frac{c}{a}}$

If $c=a^{3}$, notice that it forms a perfect square of $a$, therefore we get

$x=-\psi\pm\sqrt{\psi^{2}-a^{2}}\quad\textbf{formula (2)}$ (1)

For the expression to make sense $\sqrt{\psi^{2}-a^{2}}>0$ with
$\psi^{2}>a^{2}$. Note that if we choose the factor $\psi$ of $b$ as the
hypotenuse of a Pythagorean triple, along with $a$ as one of the legs
belonging to a Pythagorean triple, we have obtained two integer summands from
which two integer solutions are obtained.

$x_{1}\in\mathbb{Z},\quad x_{2}\in\mathbb{Z}\hfill\quad\blacksquare$

$\displaystyle a:\text{Leg of Pythagorean triple}$ $\displaystyle
b=2a\cdot\psi,\quad\psi:\text{Hypotenuse of Pythagorean triple}$
$\displaystyle c=a^{3}$

## Fibonacci Sequence in Pythagorean Triples

There are families of Pythagorean triples composed of elements from the
Fibonacci sequence, which yield special results given the nature of the
sequence.

###### Theorem 1.

: Let $\varphi_{i},\varphi_{i+1},\varphi_{i+2},\varphi_{i+3}\in\varphi_{n}$ be
terms of the Fibonacci sequence. Then:

formula(3)

$\begin{cases}\alpha=\varphi_{i}\cdot\varphi_{i+3}\\\
\beta=2\cdot\varphi_{i+1}\cdot\varphi_{i+2}\\\
\gamma=\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\end{cases}$

satisfy $\alpha^{2}+\beta^{2}=\gamma^{2}$, where
$\alpha,\beta,\gamma\in\mathbb{Z}$.

Proof: The Fibonacci sequence is defined recursively as follows:

$\begin{cases}\varphi_{0}=0\\\ \varphi_{1}=1\\\
\varphi_{n}=\varphi_{n-1}+\varphi_{n-2},\quad\text{for }n\geq 2\end{cases}$

where $\varphi_{n}$ represents the $n$-th term of the Fibonacci sequence.

The recursive definition implies that
$\varphi_{i+2}=\varphi_{i+1}+\varphi_{i}\quad\text{and}\quad\varphi_{i+3}=\varphi_{i+2}+\varphi_{i+1}$.

Thus, if we define $\alpha=\varphi_{i}\cdot\varphi_{i+3}$, we get
$\alpha=\varphi_{i}(\varphi_{i+2}+\varphi_{i+1})$. For
$\beta=2\cdot\varphi_{i+1}\cdot\varphi_{i+2}$, we have
$\beta=2\cdot\varphi_{i+1}\cdot(\varphi_{i}+\varphi_{i+1})$. Finally, for
$\gamma=\varphi_{i+1}^{2}+\varphi_{i+2}^{2}$, we obtain
$\gamma=\varphi_{i+1}^{2}+(\varphi_{i}+\varphi_{i+1})^{2}$.

Assume that $\alpha$, $\beta$, $\gamma$ do not satisfy
$mcd(\varphi_{i},\varphi_{i+1},\varphi_{i+2},\varphi_{i+3})\neq 1$.

$\alpha^{2}+\beta^{2}\neq\gamma^{2}\implies(\varphi_{i}(\varphi_{i+2}+\varphi_{i+1}))^{2}+(2\cdot\varphi_{i+1}\cdot(\varphi_{i}+\varphi_{i+1}))^{2}\neq(\varphi_{i+1}^{2}+(\varphi_{i}+\varphi_{i+1})^{2})^{2}$

Expanding the expressions, we get:

$\left(\scriptstyle{\varphi_{i}^{4}+4\varphi_{i}^{3}\varphi_{i+1}+4\varphi_{i}^{2}\varphi_{i+1}^{2}}\right)+\left(\scriptstyle{4\varphi_{i}^{2}\varphi_{i+1}^{2}+8\varphi_{i}\varphi_{i+1}^{3}+4\varphi_{i+1}^{4}}\right)\\\
\neq\left(\scriptstyle{\varphi_{i}^{4}+4\varphi_{i}^{2}\varphi_{i+1}^{2}+4\varphi_{i+1}^{4}+4\varphi_{i}^{3}\varphi_{i+1}+4\varphi_{i}^{2}\varphi_{i+1}^{2}+8\varphi_{i}\varphi_{i+1}^{3}}\right)$

Simplifying further:

$\varphi_{i}^{4}+4\varphi_{i+1}^{4}+4\varphi_{i}^{3}\varphi_{i+1}+8\varphi_{i}\varphi_{i+1}^{3}+8\varphi_{i}^{2}\varphi_{i+1}^{2}\\\
\neq\varphi_{i}^{4}+4\varphi_{i+1}^{4}+4\varphi_{i}^{3}\varphi_{i+1}+8\varphi_{i}\varphi_{i+1}^{3}+8\varphi_{i}^{2}\varphi_{i+1}^{2}$

This leads to a contradiction, so the assumption must be false. Therefore,

$\alpha^{2}+\beta^{2}=\gamma^{2}$

with $mcd(\varphi_{i},\varphi_{i+1},\varphi_{i+2},\varphi_{i+3})=1$.

Thus,

$(\varphi_{i}\cdot\varphi_{i+3})^{2}+(2\cdot\varphi_{i+1}\cdot\varphi_{i+2})^{2}=(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})^{2}$

forms a Pythagorean triple $\blacksquare$.

The connection between the formula determined in the first section and the
formula for generating Pythagorean triples from the Fibonacci sequence is
significant, as we can write formula (1) in the following way:

  1. 1.

$a=\alpha$, $b=2\cdot\alpha\cdot\gamma$, $c=\alpha^{3}$

  2. 2.

$a=\beta$, $b=2\cdot\beta\cdot\gamma$, $c=\beta^{3}$

Note that there are two expressions since the formula works with one of the
two legs of a Pythagorean triple, with $\alpha,\beta$ being legs of the same
triple.

Similarly, we can rewrite this expression according to formula (1), given the
previously proven equivalence with terms of the Fibonacci sequence:

  1. 1.

$a=\varphi_{i}\cdot\varphi_{i+3}$,
$b=2\cdot(\varphi_{i}\cdot\varphi_{i+3})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})$,
$c=(\varphi_{i}\cdot\varphi_{i+3})^{3}$

  2. 2.

$a=2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})$,
$b=2^{2}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})$,
$c=2^{3}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})^{3}$

In the next section, sequences characterizing the given formula will be
presented due to their connection with the Pythagorean triples generated by
the Fibonacci sequence.

## Primitive Pythagorean Triples and Consecutive Patterns

Definition 1: A Pythagorean triple $(a,b,c)$ is called primitive if its
components are relatively prime, that is, if $mcd(a,b,c)=1$.

Primitive Pythagorean triples serve as bases for others, considering triples
that differ by a positive integer from their base.

###### Theorem 2.

$:$ Let $a,b,c\in\mathbb{Z}$ such that
$a^{2}+b^{2}=c^{2}\quad\implies(ka)^{2}+(kb)^{2}=(kc)^{2}$, where
$k\in\mathbb{Z}$ forms a non-primitive Pythagorean triple.

Demonstration:

The demonstration is trivial since it results merely from scaling the
expression with an integer factor. It is directly related to the similarity of
triangles, which maintain the same properties. $\blacksquare$

Once the primitive and non-primitive Pythagorean triples are defined, we
proceed to "experiment with what we have."

Let $f:\mathbb{R}\rightarrow\mathbb{R}$ be defined as:

$f(x)=(3+3n)x^{2}+2(3+3n)(5+5n)x+(3+3n)^{3}$

where $n\in\mathbb{N}$.

Let $g:\mathbb{R}\rightarrow\mathbb{R}$ be defined as:

$g(x)=(4+4n)x^{2}+2(4+4n)(5+5n)x+(4+4n)^{3}$

where $n\in\mathbb{N}$.

Taking 4 terms of the Fibonacci sequence:

$\begin{cases}\varphi_{1}=1\\\ \varphi_{2}=1\\\ \varphi_{3}=2\\\
\varphi_{4}=3\\\ \end{cases}$

Determining the values that will form the primitive triple from the 4
consecutive terms of the sequence using formula (3):

$\displaystyle\alpha=\varphi_{1}\cdot\varphi_{4}=1\cdot 3=3\qquad$
$\displaystyle\beta=2\cdot\varphi_{3}\cdot\varphi_{2}=2\cdot 2\cdot 1=4\qquad$
$\displaystyle\gamma=\varphi_{2}^{2}+\varphi_{3}^{2}=1^{2}+2^{2}=5\qquad$

From this primitive triple, we can generate a family of Pythagorean triples by
scaling the base triple (3, 4, 5) with an integer. For each integer $n$
chosen, different sets $(a,b,c)$ of these triples are taken, but they belong
to the same family.

Note that the following expression is true according to Theorem 1.2:

$(3+3n)^{2}+(4+4n)^{2}=(5+5n)^{2}$

If the reader is still not convinced, here is the proof.

Proof:

Consider the following equality:

$(3+3n)^{2}+(4+4n)^{2}=(5+5n)^{2}$

Factoring out $(n+1)^{2}$:

$(n+1)^{2}(3^{2}+4^{2})=5^{2}(n+1)^{2}\implies 3^{2}+4^{2}=5^{2}$

Thus, it is demonstrated. $\blacksquare$

Now, note that the functions $f$ and $g$ comply with the values to form the
coefficients proposed in formula (1):

$\boxed{\begin{aligned} &a:\text{Leg of Pythagorean triple}\\\
&b=2a\psi,\quad\psi:\text{Hypotenuse of Pythagorean triple}\\\ &c=a^{3}\\\
\end{aligned}}$

We continue with the expression for $x$ to obtain integer roots:

$x=-\psi\pm\sqrt{\psi^{2}-a^{2}}$

Replacing the corresponding values in the expression for $f$ and $g$:

For $f$:

$x=-(5+5n)\pm\sqrt{(5+5n)^{2}-(3+3n)^{2}}$

For $g$:

$x=-(4+4n)\pm\sqrt{(5+5n)^{2}-(4+4n)^{2}}$

We can rewrite the roots for both $f$ and $g$:

For $f$:

$x=-(5+5n)\pm(4+4n)$

For $g$:

$x=-(5+5n)\pm(3+3n)$

This implies that the roots of $f$ are given by $x_{1}=-(n+1)$ and
$x_{2}=-9(n+1)$.

The roots of $g$ are given by $x_{1}=-2(n+1)$ and $x_{2}=-8(n+1)$.

The two roots for this family of Pythagorean triples result from subtracting
and adding the hypotenuse with the leg that was not chosen for the
coefficients of the function, and they all follow a pattern.

Continuing with the roots of the derivative with respect to each function,
which will give us the point in $\text{Dom}(f)$, $\text{Dom}(g)$ for which the
maximum or minimum of $\text{Ran}(f)$, $\text{Ran}(g)$ is found:

With respect to $f$:

$f^{\prime}(x)=2(3+3n)x+2(3+3n)(5+5n)$

Setting the derivative to zero and solving for $x$:

$2(3+3n)x+2(3+3n)(5+5n)=0$

Dividing both sides by $2(3+3n)$:

$x+(5+5n)=0$

Subtracting $(5+5n)$ from both sides:

$x=-(5+5n)$

Thus, the root of the derivative function is $x=-5(n+1)$.

With respect to $g$:

$g^{\prime}(x)=2(4+4n)x+2(4+4n)(5+5n)$

Setting the derivative to zero and solving for $x$:

$2(4+4n)x+2(4+4n)(5+5n)=0$

Dividing both sides by $2(4+4n)$:

$x+(5+5n)=0$

Subtracting $(5+5n)$ from both sides:

$x=-(5+5n)$

Since the roots for each function are integers, their maximum or minimum will
also be integers.

The critical points for each function, regardless of how the coefficients are
composed following formula (1), will be the same. Why do we talk about maxima
when clearly the functions $f$ and $g$ have minima? If we consider formula (1)
such that:

$\boxed{\begin{aligned} &\textbf{formula 1.1}\\\ &a:-\text{Leg of Pythagorean
triple}\\\ &b=-2a\psi,\quad\psi:\text{Hypotenuse of Pythagorean triple}\\\
&c=a^{3}\\\ \end{aligned}}$

There will get exactly the same results for $f$ and $g$ but with opposite
signs, both for the roots and the critical points.

The reader may think that the connection between the triples and the given
values for the quadratic equation coefficients is not closely related;
however, this is incorrect, as the value corresponding in the range of both
$f$ and $g$ is influenced by the nature of the Pythagorean triples.

With respect to $f$:

$(3+3n)\left(-(5+5n)\right)^{2}+2(3+3n)(5+5n)\left(-(5+5n)\right)+(3+3n)^{3}=-4\cdot
12(n+1)^{3}$

With respect to $g$:

$(4+4n)\left(-(5+5n)\right)^{2}+2(4+4n)(5+5n)\left(-(5+5n)\right)+(4+4n)^{3}=-3\cdot
12(n+1)^{3}$

It can be noted that the values corresponding to the maximum in
$\text{Ran}(f)$ and $\text{Ran}(g)$ contain within their factors the leg of
the primitive triple $(3,4,5)$ that was not chosen for the composition of the
function’s coefficients, along with a factor which is 12, resulting from the
sum of the three sides that make up the right triangle forming the triple.
(The calculation is omitted to simplify the document, as it is extensive).

What about the integral from root to root? Could the reader imagine that the
integral from one root to another follows a sequence and is also integer for
every set $(a,b,c)$ of triples in this family? It turns out to be true.

With respect to $f:$

$\left|\int_{-9(n+1)}^{-(n+1)}\left((3+3n)x^{2}+\left(2(3+3n)(5+5n)x+(3+3n)^{3}\right)\right)\,dx\right|=4^{4}(n+1)^{4}$
(2)

The proof is left as an exercise for the reader (the omission of the
calculation is made to simplify the document, as the calculation is
exhausting).

Now with respect to $g:$

$\left|\int_{-8(n+1)}^{-2(n+1)}\left((4+4n)x^{2}+(2(4+4n)(5+5n))x+(4+4n)^{3}\right),dx\right|=12^{2}(n+1)^{4}$
(3)

As mentioned earlier, if the factors for formula (1) are taken as negative due
to symmetry, the same roots, critical point, and integral with opposite sign
are maintained; in this case, positive. $\blacksquare$.

Figure 1: Representation of the integral with respect to the function $f$,
taking $n=0$, considering the symmetry given by formula 1.1. Figure 2:
Representation of the integral with respect to the function $g$, taking $n=0$,
considering the symmetry given by formula 1.1.

The results obtained above sound amazing, right? The reader could affirm that
mathematics couldn’t be more generous than he already has been. Family of
functions that have the same properties. It would be easy to search through
the factors of a quadratic equation and find the fact that it satisfies
formula (1) and know its roots, minimum or maximum, and value of the integral
from root to root, which coincidentally is a perfect square for each
Pythagorean triple selected in this family. The patterns found for different
results depend on how the functions are composed with the terms of the
Fibonacci sequence.
Could it be that these properties hold for every Pythagorean triple generated
by the Fibonacci sequence? together with formula (1)?

## Generalization and Theorem to be Presented

Note: It is important for the reader to have become sufficiently familiar with
formulas (1), (2), and (3). If not, please refer back to them on the previous
pages.

As already defined based on formula (1) and formula (3),

$*(1).$ $a=\varphi_{i}\cdot\varphi_{i+3}$,
$b=2\cdot(\varphi_{i}\cdot\varphi_{i+3})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})$,
$c=(\varphi_{i}\cdot\varphi_{i+3})^{3}$
$\implies
y_{1}=(\varphi_{i}\cdot\varphi_{i+3})x^{2}+2\cdot(\varphi_{i}\cdot\varphi_{i+3})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x+(\varphi_{i}\cdot\varphi_{i+3})^{3}$

$*(2).$ $a=2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})$,
$b=2^{2}\cdot((\varphi_{i+1}\cdot\varphi_{i+2}))\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})$,
$c=2^{3}\cdot((\varphi_{i+1}\cdot\varphi_{i+2}))^{3}$
$\implies
y_{2}=2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})x^{2}+2^{2}\cdot((\varphi_{i+1}\cdot\varphi_{i+2}))\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x+2^{3}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})^{3}$

Being $a$, $b$, $c$ coefficients of

$\displaystyle y=ax^{2}+bx+c,\quad$

with formula (2) one can obtain its roots.

With respect to *(1)

$\displaystyle
x=-(\varphi_{i+1}^{2}+\varphi_{2}^{2})\pm\sqrt{(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})^{2}-(\varphi_{i}\cdot\varphi_{i+3})^{2}}$
$\displaystyle\implies
x=-(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})\pm(2\cdot\varphi_{i+1}\cdot\varphi_{i+2})$
$\displaystyle x_{1}=-(\varphi_{i+1}-\varphi_{i+2})^{2}$ $\displaystyle
x_{2}=-(\varphi_{i+1}+\varphi_{i+2})^{2}$

Whit respect on *(2)

$\displaystyle
x=-(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})\pm\sqrt{(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})^{2}-(2\cdot\varphi_{i}\cdot\varphi_{i+2})^{2}}$
$\displaystyle\implies
x=(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})\pm(\varphi_{i}\cdot\varphi_{i+3})$
$\displaystyle
x_{1}=-(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})+(\varphi_{i}\cdot\varphi_{i+3})$
$\displaystyle
x_{2}=-(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})-(\varphi_{i}\cdot\varphi_{i+3})$

Once the logic behind the limits is understood, the theorem is presented.

###### Vega’s Theorem 0.

3
Let
$f:\mathbb{R}\rightarrow\mathbb{R}:f(x)=(\varphi_{i}\cdot\varphi_{i+3})x^{2}+2\cdot(\varphi_{i}\cdot\varphi_{i+3})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x+(\varphi_{i}\cdot\varphi_{i+3})^{3}\\\
\\\ $Let
$g:\mathbb{R}\rightarrow\mathbb{R}:g(x)=2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})x^{2}+2^{2}\cdot((\varphi_{i+1}\cdot\varphi_{i+2}))\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x+2^{3}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})^{3}$

With $\varphi_{i},\varphi_{i+1},\varphi_{i+2},\varphi_{i+3}\in\varphi_{n}$
(Fibonacci sequence)

$\theta_{1}=-(\varphi_{i+1}-\varphi_{i+2})^{2}$,
$\theta_{2}=-(\varphi_{i+1}+\varphi_{i+2})^{2}$, roots of $f$

$\phi_{1}=-(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})+(\varphi_{i}\cdot\varphi_{i+3})$,
$\phi_{2}=-(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})-(\varphi_{i}\cdot\varphi_{i+3})$,
roots of $g$

$\displaystyle\implies\int\limits_{\theta_{2}}^{\theta_{1}}f(x)\,dx\quad,\quad\int\limits_{\phi_{2}}^{\phi_{1}}g(x)\,dx\quad\in\mathbb{Z}$

Proof:

(1). Let $\varphi_{n}$ be the Fibonacci sequence, defined recursively as
$F_{0}=0$, $F_{1}=1$, and $F_{n}=F_{n-1}+F_{n-2}$ for $n\geq 2$. The general
formula for the $n$th term of the Fibonacci sequence, denoted as
$\varphi_{n}$, can be expressed as:

$\varphi_{n}=\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{n}-\left(\frac{1-\sqrt{5}}{2}\right)^{n}\right)$

The hypothesis to be stated is denoted as

$\displaystyle\varphi_{4n}=\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4n}-\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\right)\equiv
0\quad mod(3).\forall\quad n\in\mathbb{N}$

Proof(1)

By the principle of induction, let’s denote the base case

$\displaystyle
p(1)=\varphi_{4n}=\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4n}-\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\right)$
$\displaystyle\implies\varphi_{4}=\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4}-\left(\frac{1-\sqrt{5}}{2}\right)^{4}\right)$
$\displaystyle\implies=\frac{1}{\sqrt{5}\cdot
2^{4}}\left(\left({1+\sqrt{5}}\right)^{4}-\left({1-\sqrt{5}}\right)^{4}\right)$
$\displaystyle=\frac{1}{\sqrt{5}\cdot
2^{4}}\cdot\left(\left(1+\sqrt{5}\right)^{2}+\left(1-\sqrt{5}\right)^{2}\right)\cdot\left(\left(1+\sqrt{5}\right)^{2}-\left(1-\sqrt{5}\right)^{2}\right)$
$\displaystyle=\frac{1}{\sqrt{5}\cdot
2^{4}}\cdot\left(\left(1+2\sqrt{5}+5\right)+\left(1-2\sqrt{5}+5\right)\right)\cdot\left(\left(1+2\sqrt{5}+5\right)-\left(1-2\sqrt{5}+5\right)\right)$
$\displaystyle\implies\frac{1}{\sqrt{5}}\cdot\frac{4\sqrt{5}\cdot
12}{4^{2}}=3$

Clearly

$\displaystyle 3\equiv 0\quad mod(3)$

hence the base case is proven. Now let’s assume

$\displaystyle\varphi_{4n}=\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4n}-\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\right)\equiv
0\quad mod(3).\forall\quad n\in\mathbb{N}$

is true $\implies$

$\displaystyle\varphi_{4(n+1)}=\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4(n+1)}-\left(\frac{1-\sqrt{5}}{2}\right)^{4(n+1)}\right)\equiv
0\quad mod(3).\forall\quad n\in\mathbb{N}$

should be true.
Algebraically developing the expression for $\varphi_{4(n+1)}$

$\displaystyle\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1+\sqrt{5}}{2}\right)^{4}-\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1-\sqrt{5}}{2}\right)^{4}\right)$

$\displaystyle\frac{1}{\sqrt{5}}\left(\frac{1+\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1+\sqrt{5}}{2}\right)^{4}\quad-\frac{1}{\sqrt{5}}\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1-\sqrt{5}}{2}\right)^{4}$

Without modifying the expression, we proceed with

$\displaystyle\frac{1}{\sqrt{5}}\left(\frac{1+\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1+\sqrt{5}}{2}\right)^{4}-\frac{1}{\sqrt{5}}\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1+\sqrt{5}}{2}\right)^{4}$
$\displaystyle+\frac{1}{\sqrt{5}}\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1+\sqrt{5}}{2}\right)^{4}-\frac{1}{\sqrt{5}}\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\cdot\left(\frac{1-\sqrt{5}}{2}\right)^{4}$
$\displaystyle=\frac{1}{\sqrt{5}}\cdot\left(\frac{1+\sqrt{5}}{2}\right)^{4}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4n}-\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\right)\quad+\frac{1}{\sqrt{5}}\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\cdot$
$\displaystyle\left(\left(1+\frac{\sqrt{5}}{2}\right)^{4}-\left(\frac{1-\sqrt{5}}{2}\right)^{4}\right)$

By the already proven base case, the expression is rewritten.

$\displaystyle=\frac{1}{\sqrt{5}}\cdot\left(\frac{1+\sqrt{5}}{2}\right)^{4}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4n}-\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\right)\quad+3\left(\frac{1-\sqrt{5}}{2}\right)^{4n}$

$\implies$

By inductive hypothesis, the assertion holds for $\varphi_{4(n+1)}$

$\displaystyle\implies\varphi_{4n}=\frac{1}{\sqrt{5}}\left(\left(\frac{1+\sqrt{5}}{2}\right)^{4n}-\left(\frac{1-\sqrt{5}}{2}\right)^{4n}\right)\equiv
0\quad mod(3).\forall\quad n\in\mathbb{N}\blacksquare$

From what we have proven above, we can deduce that $\exists\quad
X:X=\\{\varphi_{i},\varphi_{i+1},\varphi_{i+2},\varphi_{i+3}\\}$ with
$X\subset\varphi_{n}$, $\quad\exists\quad\varphi_{k}\in X:\varphi_{k}\equiv
0\quad mod(3)\quad*(1)$
Continuing with the proof, regarding $f$

$\displaystyle\int\limits_{\theta_{2}}^{\theta_{1}}f(x)\,dx\quad=\int\limits_{\theta_{2}}^{\theta_{1}}(\varphi_{i}\cdot\varphi_{i+3})x^{2}+2\cdot(\varphi_{i}\cdot\varphi_{i+3})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x+(\varphi_{i}\cdot\varphi_{i+3})^{3}\,dx$

Let’s denote for ease of notation

$\displaystyle\int\limits_{\theta_{2}}^{\theta_{1}}(\varphi_{i}\cdot\varphi_{i+3})x^{2}\,dx+\int\limits_{\theta_{2}}^{\theta_{1}}2\cdot(\varphi_{i}\cdot\varphi_{i+3})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x\,dx+\int\limits_{\theta_{2}}^{\theta_{1}}(\varphi_{i}\cdot\varphi_{i+3})^{3}\,dx=P1+P2+P3$
$\displaystyle
P1+\left[(\varphi_{i}\cdot\varphi_{i+3})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x^{2}\right]_{\theta_{2}}^{\theta_{1}}+\left[(\varphi_{i}\cdot\varphi_{i+3})x\right]_{\theta_{2}}^{\theta_{1}}$

Clearly, both $P2$ and $P3$ are integers, integrating $P1$
$\implies$

$\displaystyle\int\limits_{\theta_{2}}^{\theta_{1}}(\varphi_{i}\cdot\varphi_{i+3})x^{2}\,dx=[\frac{1}{3}(\varphi_{i}\cdot\varphi_{i+3})x^{3}]_{\theta_{2}}^{\theta_{1}}$

$\displaystyle\frac{1}{3}(\varphi_{i}\cdot\varphi_{i+3})\left(\left(-(\varphi_{i+1}-\varphi_{i+2})^{6}\right)-\left(-(\varphi_{i+1}+\varphi_{i+2})^{6}\right)\right)$
$\displaystyle=\frac{1}{3}(\varphi_{i}\cdot\varphi_{i+3})\left((\varphi_{i+1}+\varphi_{i+2})^{6}-(\varphi_{i+1}-\varphi_{i+2})^{6}\right)$
$\displaystyle\implies\frac{1}{3}(\varphi_{i}\cdot\varphi_{i+3})\left(\left((\varphi_{i+1}+\varphi_{i+2})^{3}-(\varphi_{i+1}-\varphi_{i+2})^{3}\right)\cdot\left((\varphi_{i+1}+\varphi_{i+2})^{3}+(\varphi_{i+1}-\varphi_{i+2})^{3}\right)\right)$

using the difference and sum of cubes formulas

$\displaystyle\scriptstyle{\frac{1}{3}(\varphi_{i}\cdot\varphi_{i+3})\left(\left(\varphi_{i+1}+\varphi_{i+2}-\varphi_{i+1}+\varphi_{i+2}\right)\cdot[(\varphi_{i+1}+\varphi_{i+2})^{2}+(\varphi_{i+1}+\varphi_{i+2})\cdot(\varphi_{i+1}-\varphi_{i+2})+(\varphi_{i+1}+\varphi_{i+2})^{2}]\right)}$
$\displaystyle\scriptstyle{\cdot\left(\left(\varphi_{i+1}+\varphi_{i+2}+\varphi_{i+1}-\varphi_{i+2}\right)\cdot[(\varphi_{i+1}+\varphi_{i+2})^{2}-(\varphi_{i+1}+\varphi_{i+2})\cdot(\varphi_{i+1}-\varphi_{i+2})+(\varphi_{i+1}-\varphi_{i+2})^{2}]\right)}$

Simplifying

$\displaystyle\frac{2^{2}}{3}\left(\varphi_{i}\cdot\varphi_{i+3}\cdot\varphi_{i+2}\cdot\varphi_{i+1}\right)\cdot[(\varphi_{i+1}+\varphi_{i+2})^{2}+(\varphi_{i+1}+\varphi_{i+2})\cdot(\varphi_{i+1}-\varphi_{i+2})+(\varphi_{i}-\varphi_{i+2})^{2}]$
$\displaystyle\cdot[(\varphi_{i+1}+\varphi_{i+2})^{2}-(\varphi_{i+1}+\varphi_{i+2})\cdot(\varphi_{i+1}-\varphi_{i+2})+(\varphi_{i+1}-\varphi_{i+2})^{2}]$

The integral reduces to factors, of which, by the deduction made in *(1), some
of them must be multiples of 3. This proves that $P1$ is an integer. Now,
since it has been proven that $P1$, $P2$, $P3\in\mathbb{Z}$.

$\displaystyle\implies\int\limits_{\theta_{2}}^{\theta_{1}}f(x)\,dx\quad\in\mathbb{Z}$

Similarly for $g$

$\displaystyle=\int\limits_{\phi_{2}}^{\phi_{1}}g(x)\,dx$
$\displaystyle=\int\limits_{\phi_{2}}^{\phi_{1}}2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})x^{2}+2^{2}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x+2^{3}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})^{3}\,dx$

Separating the integral

$\displaystyle\int\limits_{\phi_{2}}^{\phi_{1}}2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})x^{2}\,dx+\int\limits_{\phi_{2}}^{\phi_{1}}2^{2}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x\,dx+\int\limits_{\phi_{2}}^{\phi_{1}}2^{3}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})^{3}\,dx$

Defining for ease of notation $T1+T2+T3$ as each integral in order, just like
in the previous proof, $T2$ and $T3$ turn out to be integers, since

$\displaystyle
T2=\left[2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})\cdot(\varphi_{i+1}^{2}+\varphi_{i+2}^{2})x^{2}\right]_{\phi_{2}}^{\phi_{1}}$
$\displaystyle
T3=\left[2^{3}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})^{3}x\right]_{\phi_{2}}^{\phi_{1}}$

Integrating $T1$:

$\displaystyle\int\limits_{\phi_{2}}^{\phi_{1}}2\cdot(\varphi_{i+1}\cdot\varphi_{i+2})x^{2}\,dx=\left[\frac{2}{3}\cdot(\varphi_{i+1}\cdot\varphi_{i+2})x^{3}\right]_{\phi_{2}}^{\phi_{1}}$

Applying the difference of cubes formula, we get:

$\displaystyle=\frac{2}{3}\cdot\left(\varphi_{i+1}\cdot\varphi_{i+2}\right)\cdot\left(\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)+\varphi_{i}\cdot\varphi_{i+3}\right)^{3}-\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)-\varphi_{i}\cdot\varphi_{i+3}\right)^{3}\right)$
$\displaystyle=\frac{2}{3}\cdot\left(\varphi_{i+1}\cdot\varphi_{i+2}\right)\cdot\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)+\varphi_{i}\cdot\varphi_{i+3}+\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)+\varphi_{i}\cdot\varphi_{i+3}\right)\cdot$
$\displaystyle\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)+\varphi_{i}\cdot\varphi_{i+3}\right)^{2}+\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)+\varphi_{i}\cdot\varphi_{i+3}\right)$

$\displaystyle\cdot\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)-\varphi_{i}\cdot\varphi_{i+3}\right)+\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)-\varphi_{i}\cdot\varphi_{i+3}\right)^{2}$

Simplifying the expression:

$\displaystyle=\frac{2^{2}}{3}\cdot\left(\varphi_{i+1}\cdot\varphi_{i+2}\cdot\varphi_{i}\
\cdot\varphi_{i+3}\right)\cdot$
$\displaystyle[\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)+\varphi_{i}\cdot\varphi_{i+3}\right)^{2}+\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)+\varphi_{i}\cdot\varphi_{i+3}\right)$

$\displaystyle\cdot\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)-\varphi_{i}\cdot\varphi_{i+3}\right)+\left(-\left(\varphi_{i+1}^{2}+\varphi_{i+2}^{2}\right)-\varphi_{i}\cdot\varphi_{i+3}\right)^{2}]$

Similarly, for $g$, by deduction *(1), it is concluded that both $T1$, $T2$,
and $T3$ have integer values, with $T1$ being an integer since some of its
factors are multiples of 3, thus demonstrating that.

$\displaystyle\implies\int\limits_{\theta_{2}}^{\theta_{1}}f(x)\,dx\quad,\quad\int\limits_{\phi_{2}}^{\phi_{1}}g(x)\,dx\quad\in\mathbb{Z}\quad\blacksquare$

## References

$[1]$ Renault, M. (1996). The Fibonacci Sequence Under Various Moduli (Wake
Forest University, Ed.) [Review of The Fibonacci Sequence Under Various
Moduli]. Fibthesis.pdf; Marc Renault.
https://webspace.ship.edu/msrenault/fibonacci/fibthesis.pdf

$[2]$ Aguilera, C. (2022). On Primitive Pythagorean Triples. ffhal-03247261v2.
https://hal.science/hal-03247261v2/file/Colorin_Primitive_Pythagoren_Triples.pdf

$[3]$ Mathcentre. (2009). Quadratic Equations. mc-TY-quadeqns-1. mathcentre.
https://www.mathcentre.ac.uk/resources/uploaded/mc-ty-quadeqns-1.pdf

$[4]$ Bicknell-Johnson, M. (n.d.). Pythagorean Triples Containing Fibonacci
Numbers: Solutions for $F*\pm F*=K^{2}$. A.C. Wilcox High School, Santa Clara,
CA 95051.
https://www.fq.math.ca/Scanned/17-1/bicknell.pdf


# Photoinduced enhancement of superconductivity in the plaquette Hubbard model

Yuxi Zhang  Department of Physics, University of California, Davis, CA
95616,USA  Rubem Mondaini  Beijing Computational Science Research Center,
Beijing 100084, China  Richard T. Scalettar  Department of Physics, University
of California, Davis, CA 95616,USA

###### Abstract

Real-time dynamics techniques have proven increasingly useful in understanding
strongly correlated systems both theoretically and experimentally. By
employing unbiased time-resolved exact diagonalization, we study pump dynamics
in the two-dimensional plaquette Hubbard model, where distinct hopping
integrals $t_{h}$ and $t_{h}^{\prime}$ are present within and between
plaquettes. In the intermediate coupling regime, a significant enhancement of
$d$-wave superconductivity is observed and compared with that obtained by
simple examination of expectation values with the eigenstates of the
Hamiltonian. Our work provides further understanding of superconductivity in
the Hubbard model, extends the description of the pairing amplitude to the
frequency-anisotropy plane, and offers a promising approach for experimentally
engineering emergent out-of-equilibrium states.

##  I Introduction

Out-of-equilibrium dynamics and photoinduced properties have been attracting
intense attention recently. In a single-particle setting, periodic drives
provide an additional “Floquet dimension”, and can result in phenomena such as
engineered Chern insulating behavior and Wannier-Stark localization [1]. These
techniques have also allowed considerable new insight into strongly correlated
systems, and many-body phases induced by either electron-electron or electron-
phonon interactions [2, 3], including studies of photoinduced $\eta$-pairing
[4, 5], bond-order wave (BOW) physics [6], dynamical topological engineering
[7, 8], and especially superconductivity [9, 10, 11, 12, 13, 14, 15]. A focus
has been on gaining new insight into cuprate systems, and the origin of
pairing in high-$T_{c}$ materials [16, 17, 18, 19].

One mechanism for the enhancement of superconductivity is the use of
photoexcitation to transfer charge between orbitals, thereby shifting the
doping away from 1/8 filling where the “stripe anomaly” leads to a suppressed
$T_{c}$. Here exact diagonalization studies have clarified whether the
emergence of superconductivity via photoexcitation seen experimentally results
from phonon or electronic mechanisms [13]. Indeed, superconductivity can also
be developed out of the 1/8 filling phase through photoexcitation of the Cu-O
stretch mode in the plane of the stripes [9]. This coupling to vibrational
modes offers advantages in that the lower energy photons have less likelihood
of creating high temperature electronic distributions and heating the lattice.

In calculations similar to the study to be described here, a theoretical
understanding of the enhancement of pairing is provided through the lifting of
the degeneracy between superconducting and charge density wave (CDW) ground
states in the attractive Hubbard Hamiltonian [12]. In the Hubbard-Holstein
model, inducing $d$-wave pairing has been shown to be especially effective at
the quantum boundary between the CDW and spin density wave (SDW) ground state
phases [13]. Theoretical work within a BCS model has also explored the role of
the symmetry of the pair wave function. The existence of nodes in a $d$-wave
phase allows for a more rapid decay, making the dynamics substantially faster
than the $s$-wave channel [10].

In parallel with these developments, theoretical studies have shown that
inhomogeneities such as stripe-like or plaquette decorations of two-
dimensional square lattices could enhance $d$-wave pairing in equilibrium [20,
21, 22, 23, 24, 25, 26]. This is an especially rich area of inquiry given the
complex entwining of spin and charge textures with pairing [27, 28, 29, 30,
31, 32]. Specifically, under some circumstances inhomogeneities clearly
interfere with superconductivity, while in others they appear to play a role
in its formation and stability.

In this paper we combine these two themes of out-of-equilibrium dynamics and
spatial inhomogeneity. In particular, we examine the effect of pumping on a
geometry built up from attractive $d$-wave binding centers formed by $2\times
2$ plaquettes. Our goal is to understand the interplay of these two effects
which are already known individually to result in enhanced superconductivity.
Our work will therefore examine the possibility that a greater signal of
pairing might be thereby engineered, a prospect made particularly intriguing
by the recent experimental emulation of fermionic models in $2\times 2$
plaquettes in platforms of quantum dots [33].

In the next section we introduce the plaquette Hubbard model and our
computational methodology. We then examine the degree of $d$-wave pairing in
the different low energy eigenstates. A central result here is a ‘phase
diagram’ which shows the degree of enhanced superconductivity in the plane of
hopping anisotropy $t_{h}^{\prime}/t_{h}$ and excitation energy
$E_{\alpha}-E_{0}$. We next examine time evolution, with the objective of
employing a quasi-periodic drive of an appropriate frequency $\omega_{0}$ to
target those optimal $E_{\alpha}-E_{0}$. The end result is a second ‘phase
diagram’ which indicates the achievable enhancement in the plane of
$\omega_{0}$ and drive amplitude $A_{0}$. Here we focus on the
$t_{h}^{\prime}/t_{h}$ which gives optimal pairing in the analysis of the
eigenstates. We conclude the paper with a summary of results and a discussion
of the general significance of our findings.

##  II Model and Methodology

The two dimensional Hubbard model is described by

$\displaystyle\mathcal{\hat{H}}=$
$\displaystyle-\sum_{\langle\mathbf{i},\mathbf{j}\rangle,\sigma}\big{(}t_{\mathbf{i},\mathbf{j}}^{\phantom{\dagger}}\,\hat{c}^{\dagger}_{\mathbf{i}\sigma}\hat{c}^{\phantom{\dagger}}_{\mathbf{j}\sigma}+{\rm
H.c.}\big{)}+U\sum_{\mathbf{i}}\hat{n}_{\mathbf{i},\uparrow}\hat{n}_{\mathbf{i},\downarrow}\,\,,$
(1)

where
$\hat{c}^{\phantom{\dagger}}_{\mathbf{i}\sigma}(\hat{c}^{\dagger}_{\mathbf{i}\sigma})$
is the annihilation (creation) operator for an electron on site $\mathbf{i}$
with spin $\sigma=\uparrow,\downarrow$ and
$\hat{n}_{\mathbf{i},\sigma}^{\phantom{\dagger}}=\hat{c}^{\dagger}_{\mathbf{i},\sigma}\hat{c}^{\phantom{\dagger}}_{\mathbf{i},\sigma}$
is the number operator.
$t_{\mathbf{i},\mathbf{j}}=t_{h}^{\phantom{{}^{\prime}}}(t_{h}^{\prime})$
gives the hopping integral within (between) plaquettes, and on-site electron-
electron interaction is tuned via $U$. We set $t_{h}=1$ as the energy scale,
and the time $t$ is measured in terms of $t_{h}^{-1}$ with $\hbar=1$.
Throughout this work, we use an $N=4\times 4$ lattice with periodic boundary
conditions (PBC), as illustrated in Fig. 1.

Figure 1: Schematic illustration of the plaquette Hubbard model on a $4\times
4$ lattice with PBCs, with relevant terms in the Hamiltonian annotated. Dark
(light) colored bonds represent the intra-(inter-) plaquette hopping
$t_{h}(t_{h}^{\prime})$.

The system is driven by a time-dependent external field, which is introduced
in the Hamiltonian via the Peierls’ substitution

$\displaystyle
t_{\mathbf{i},\mathbf{j}}^{\phantom{\dagger}}\,\hat{c}^{\dagger}_{\mathbf{i}\sigma}\hat{c}^{\phantom{\dagger}}_{\mathbf{j}\sigma}+{\rm
H.c.}\rightarrow
t_{\mathbf{i},\mathbf{j}}^{\phantom{\dagger}}\,e^{i\mathbf{A}(t)\cdot(\mathbf{r}_{\mathbf{i}}-\mathbf{r}_{\mathbf{j}})}\,\hat{c}^{\dagger}_{\mathbf{i}\sigma}\hat{c}^{\phantom{\dagger}}_{\mathbf{j}\sigma}+{\rm
H.c.}\,\,,$ (2)

where $\mathbf{A}(t)$ is the spatially uniform vector potential. We choose
$\mathbf{A}(t)$, mimicking an ultrafast photoirradiation, of the form

$\displaystyle\mathbf{A}(t)=A_{0}e^{-(t-t_{0})^{2}/2t_{d}^{2}}\,\cos[\omega_{0}(t-t_{0})+\varphi]\,\mathbf{e}_{\rm
pol}\,\,,$ (3)

i.e., an oscillatory Gaussian centered at $t_{0}$ and width $t_{d}$, with
amplitude $A_{0}$, frequency $\omega_{0}$ and polarization direction
$\mathbf{e}_{pol}=(\hat{\mathbf{x}}+\hat{\mathbf{y}})/\sqrt{2}$. Finally,
$\varphi$ is a time-phase of the pump pulse, allowing the possibility of
phase-averaging that eliminates particular fast coherent oscillations tied to
interaction magnitudes [2]. When applicable, results of the dynamics will
employ an average over 10 equidistant values of $\varphi$ in the range
$[0,2\pi)$. In what follows, we set $t_{0}=100\,t_{h}^{-1}$ and
$t_{d}=25\,t_{h}^{-1}$, with maximum time chosen as $t_{\rm
max}=200\,t_{h}^{-1}$, which is well after the pump still exhibits a relevant
amplitude.

Simulations are carried out with the time-dependent Lanczos method [34, 35],
where the time-evolution is obtained via
$|\Psi(t+dt)\rangle=e^{-i\mathcal{\hat{H}}(t)dt}|\Psi(t)\rangle$, with $dt$ a
sufficiently small time step. The initial state, $|\Psi(t\to-\infty)\rangle$,
is the ground state of (1), with $|\Psi(t)\rangle$ and associated measurements
being evaluated at each time step.

##  III Equilibrium Results

To map out where an out-of-equilibrium protocol would induce enhanced
superconductivity, we start by examining the equilibrium properties of the
plaquette Hubbard model across a range of
$(t_{h}^{\prime}/t_{h}^{\phantom{{}^{\prime}}},U,n_{e})$ parameters. Some of
these have already been explored at finite temperatures by means of quantum
Monte Carlo simulations [26] or at the ground-state [22] in lattice sizes as
in Fig. 1. Here, we go beyond that and investigate different equal-time
correlators computed over the low-lying eigenspectrum of (1) $[\hat{\cal
H}|\alpha\rangle=E_{\alpha}|\alpha\rangle]$. Among them, we calculate the
pairing structure factor at different $\gamma$-channels

$\displaystyle P_{\gamma}=\frac{1}{N}\sum_{{\bf i},{\bf
dr}}\langle\hat{\Delta}_{\bf i}^{(\gamma)\phantom{\dagger}}\hat{\Delta}_{\bf
i+dr}^{(\gamma)\dagger}\rangle\,\,,$ (4)

where $\hat{\Delta}_{\bf i}^{(\gamma)\phantom{\dagger}}=(\hat{c}_{{\bf
i+\hat{x}},\sigma}+f(\gamma)\hat{c}_{{\bf i+\hat{y}},\sigma})\hat{c}_{{\bf
i},\overline{\sigma}}$, with $f(\gamma)=-1(+1)$ for $d$-wave(extended
$s$-wave, $s^{*}$) pairing. In a similar fashion, the staggered spin and
charge correlations are probed as

$\displaystyle S_{x}=\frac{1}{N}\sum_{\bf
i,dr}\langle(-1)^{dx+dy}\hat{O}^{\dagger}_{\bf
i}\hat{O}^{\phantom{\dagger}}_{\bf i+dr}\rangle\,\,,$ (5)

where $\hat{O}_{\bf i}=\hat{n}_{\bf i\uparrow}\pm\hat{n}_{\bf i\uparrow}$, and
the $+$ sign defines charge ($x=$ CDW) and the $-$ sign spin ($x=$ SDW)
density wave order.

Over the large set of parameters available, we focus the investigation on both
the doped ($\rho\equiv n_{e}/N=0.875$) and undoped cases ($\rho=1$); the
latter described in the Appendix A. Due to geometric constraints in the
lattice size we tackle, a period-8 magnetic stripe formation is not supported
[36, 37, 30, 38]. The ground state properties of $\hat{\cal H}$, originally
reported in Ref. 22, indicated that a maximum binding energy occurs at
intermediate plaquette hopping energies
($t^{\prime}_{h}/t_{h}^{\phantom{{}^{\prime}}}\simeq 0.5$), and large
repulsive interactions $U\simeq 8\,t_{h}$. This serves as a paradigm to our
investigation, in which we are interested in the set of parameters that
maximize the expectation values of pairing correlators over states beyond that
of the ground state $|0\rangle$.

Figure 2: (a) $d$-wave, (b) extended $s$-wave, (c) CDW and (d) SDW structure
factor, in the ground state (dashed lines) and the first few excited states.
Different colors and symbols represent different quasi-momentum sectors.
Parameters used are $t_{h}^{\prime}/t_{h}^{\phantom{{}^{\prime}}}=0.8$,
$U=8\,t_{h}$ and $n_{e}=14$. We show the different momentum sectors ${\bf k}$
for completeness, since for this filling the ground state always resides at
${\bf k}=(0,0)$. Each sector displays various eigenstates $|\alpha\rangle$,
some of which may display degeneracies owing to other point group symmetries
not being resolved, e.g. mirror symmetries.

Figure 2 displays the eigenstate expectation values (EEV) of the previously
introduced correlators for $t_{h}^{\prime}/t_{h}^{\phantom{{}^{\prime}}}=0.8$,
$U=8\,t_{h}$. Most noticeably, a group of three excited states (two of them
degenerate) with energy gap $E_{\alpha}-E_{0}\simeq 0.25\,t_{h}$ (belonging to
the same momentum sector) has $d$-wave pairing $P_{d}\simeq 1.9$,
significantly higher than that of the ground state $P_{d}=1.66$. With the
exception of the staggered spin correlations, no similar substantial relative
increase is seen in other correlators, including the extended $s$-wave or CDW
channels. Equivalent simulations were performed for other
$t_{h}^{\prime}/t_{h}^{\phantom{{}^{\prime}}}$ values, and a compilation of
the results of the relative maximum enhancement over excited states with
respect to the ground-state, $\Delta{\cal
O}\equiv(\max\\{{\langle\alpha|\hat{\cal O}|\alpha\rangle\\}}-\langle
0|\hat{\cal O}|0\rangle)/\langle 0|\hat{\cal O}|0\rangle$, is given in Fig. 3.
Although this maximum increase for the $d$-wave pairing is seen right in the
vicinity of the homogeneous system ($t^{\prime}_{h}=t_{h}$), we focus on
$t_{h}^{\prime}/t_{h}=0.8$ to study the regime of robustly formed plaquettes.

##  IV Out-of-equilibrium results: Photoirradiation

Having established that the expectation values of correlators, in particular
$P_{d}$, measured in excited eigenstates of the Hamiltonian may possess larger
values in comparison to the ones at the ground state, we turn to the out-of-
equilibrium scenario, with an added time-dependent external field which can
cause transitions to these states. Its associated vector potential amplitude
is depicted at the top of Fig. 4. We follow a procedure that has been proven
feasible in other contexts of photoirradiation [6, 8]: by using a fine-tuned
temporal perturbation, whose frequency is set resonantly with the target state
one is aiming to excite ($\omega_{0}\equiv E_{\rm target}-E_{0}$), the
dynamics at long times thus exhibit a large overlap with such eigenstate.

Figure 3: Relative variation of the maximum eigenstate expectation values
$\Delta{\cal O}$, quantified over excited states, with respect to its ground-
state average (see text) for different correlators, when taking into account
at least 24 eigenstates in the low-lying spectrum for the ${\bf k}=(0,0)$
momentum sector. Parameters are the same as in Fig. 2: $U=8\,t_{h}$ and
$n_{e}=14$.

There are a number of constraints in this protocol. First, the initial and
final states need similar quantum numbers. In particular, total momentum, as
the vector potential used [Eq. (3)] is spatially uniform and does not break
translation invariance. Second, the pump needs to be time-constrained,
otherwise the system would indefinitely absorb energy from the drive, owing to
the thermalizing properties of non-integrable systems [39]. In such case,
heating towards an infinite temperature regime would ensue at sufficiently
long-times, exhibiting featureless correlators.

Following these requirements, we use a pump with parameters $A_{0}=0.1$,
$\omega_{0}=0.257\,t_{h}$ as to closely match that of the excited states with
stronger $d$-wave pairing signal [see top of Fig. 2(a)]. These parameters are
also reasonably aligned to the ones employed in recent experiments. Using
$t_{h}\simeq 0.3$ eV, which is most typically seen in most cuprates materials,
$\omega_{0}=0.257\,t_{h}$ corresponds to $0.08$ eV or 20 THz whereas
$A_{0}=0.1$ translates into a pump fluence of $\sim 0.02$ J/cm2, which are
comparable to that used in photo-induced experiments [9, 19]. A discussion of
the neighboring frequencies that can influence the dynamics and the reason why
we chose this specific $\omega_{0}$ is given in Appendix B. One can, however,
see from Fig. 2 that this $\omega_{0}$ at least roughly corresponds to the
excitation energy of a set of states with enhanced $P_{d}$. In particular, a
doublet of states ($|6\rangle$ and $|7\rangle$), energy-degenerate up to
$10^{-12}\ t_{h}$, possesses one of the largest expectation values of the
$d$-wave pairing $P_{d}$ ($\simeq 1.92$). Under these conditions, Fig. 4(a)
shows that a considerable enhancement in $d$-wave pairing is induced,
accompanied by an energy increase, such that the mean energy at long times,
$E=\langle\Psi(t=t_{\rm max})|\hat{\cal H}|\Psi(t=t_{\rm max})\rangle$, is
close to the eigenenergy of the target states. Simultaneously, a similar large
enhancement on the staggered spin correlations is also achieved [Fig. 4(b)].

To demonstrate that the dynamics is being influenced by the target and its
neighboring states at these time scales, we compute the overlaps of the time-
evolving wave functions and the eigenstates of the equilibrium Hamiltonian,
$|\langle\Psi(t)|\alpha\rangle|^{2}$. We report those in Fig. 4(c), where one
notices that the weight of the target doublet states rises up to 0.52 and 0.40
after the pump, for states $|7\rangle$ and $|6\rangle$, respectively,
explaining thus the enhancement of $d$-wave pairing under these conditions.
The participation of the ground state $|0\rangle$ decreases significantly,
whereas the contribution of other eigenstates is mostly negligible. In the
insets, the EEVs of $P_{d}$ and $S_{\rm sdw}$ are shown in this set of
parameters, $(U/t_{h},t^{\prime}_{h}/t_{h}^{\phantom{{}^{\prime}}})=(8,0.8)$,
together with the mean energy of the system $E(t_{\rm max})$.

A first account of which eigenstates can significantly contribute to the
dynamics at a given time is given by the width in energy of $|\Psi(t)\rangle$,
defined as
$\sigma_{E}=[\langle\Psi(t)|H^{2}|\Psi(t)\rangle-\langle\Psi(t)|H|\Psi(t)\rangle^{2}]^{1/2}$
[40, 41]. This is displayed as a shaded area surrounding $E(t_{\rm max})$ in
the insets of Fig. 4(a) and 4(b), where we notice that the eigenstates within
the window are the ones that may display a large overlap with
$|\Psi(t)\rangle$ in Fig. 4(c). While this window contains six eigenstates in
the ${\bf k}=(0,0)$ quasi-momentum sector, not all of them affect the
dynamics. Symmetry requirements related to how they couple to the current
operator $\hat{\cal
J}=\sum_{\langle\mathbf{i},\mathbf{j}\rangle,\sigma}\big{(}-it_{\mathbf{i},\mathbf{j}}^{\phantom{\dagger}}\,\hat{c}^{\dagger}_{\mathbf{i}\sigma}\hat{c}^{\phantom{\dagger}}_{\mathbf{j}\sigma}+{\rm
H.c.}\big{)}$ dictate which ones will eventually couple to the external field.
That is, besides energy requirements, overlaps $|\langle\alpha|\hat{\cal
J}|0\rangle|^{2}$ also classify, in first order, the states most influencing
the dynamics. Even if the pulse maintains the quasi-momentum of the time-
evolving wavefunction, the polarization employed is not sufficient to mix all
different mirror symmetry sectors, and as a result some eigenstates must not
participate in the dynamical evolution process.

Figure 4: Top panel: schematics of the pulse amplitude over time. (a)
Evolution of the phase-averaged $d$-wave pair correlation $P_{d}$ (red solid
curve) and mean energy $E$ (blue solid curve). The different curves in shades
of red and blue denote the values for each time-phase $\varphi$. (b) The same
for the staggered spin structure factor. (c) Dynamics of the overlaps
$|\langle\Psi(t)|\alpha\rangle|^{2}$ over the course of photoirradiation.
Darker color curves represent lower energy states; the degenerate doublet,
$|6\rangle$ and $7\rangle$, displays the largest contribution to
$|\Psi(t)\rangle$ as marked, whereas the participation of $|0\rangle$, the
initial state, diminishes significantly. The insets give the EEVs for both
quantities with the mean energy at $t=t_{\rm max}$ and within a window given
by $\sigma_{E}$ (see text). Pump parameters are $A_{0}=0.1$ and
$\omega_{0}=0.257\,t_{h}$.

Now that we have confirmed that an enhancement of $d$-wave pairing is
attainable for these pump parameters, we explore other combinations of
$(A_{0},\omega_{0})$, identifying the ones that give optimal results in the
enhancement of pair correlations at long times. Towards that end, we study the
variation of different post-pump observables as $dP_{d}\equiv P_{d}(t=t_{\rm
max})-P_{d}(0)$ [Fig. 5(a)] and $dS_{\rm sdw}\equiv S_{\rm sdw}(t=t_{\rm
max})-S_{\rm sdw}(0)$ [Fig. 5(b)], quantifying the dynamical variations of
$d$-wave pairing and staggered spin correlations. A relatively narrow band in
pump amplitudes $A_{0}$ results in increased pairing for resonant drives. The
reason can be seen by the map of injected energies $dE\equiv E(t=t_{\rm
max})-E_{0}$ in Fig. 5(c): Increasing $A_{0}$ leads to a systematically larger
absorbed energy by the system, thereby a larger contribution of states in the
bulk of the spectrum, not associated to an enhanced pairing amplitude, is
manifest.

We note that not only a specific resonant drive with frequency $\omega_{0}$,
in principle, is able to target a state, but also higher harmonic
contributions $2\,\omega_{0},3\,\omega_{0},\ldots$. A tuning of other
parameters of the pulse, as $A_{0}$ and $t_{d}$, is necessary in these
conditions in order to control the injected energy in the system, while
monitoring the typical width in energy of $|\Psi(t_{\rm max})\rangle$.
Adjusting these parameters, within physically reasonable limits, thus presents
as a systematic protocol to access the physics of competing states close to
the ground-state, guaranteeing that the long-time dynamics is largely
influenced by the (group of) target excited state(s). Such entwining of
different correlations is known to occur in doped Hubbard Hamiltonians and, as
we demonstrated, engineered time-dependent perturbations are key to pick one
(or more) type in detriment of others.

Figure 5: Contour plot of the enhancement of the $d$-wave pair [staggered
spin] structure factor (a) [(b)] at $t=t_{\rm max}$ for various pump parameter
$(A_{0},\omega_{0})$ values. (c) Similar results for the injected energy by
the photoirradiation. All results are obtained for $\varphi=0$. In all panels,
star markers identify the pump parameters used in Fig. 4.

##  V Summary and Discussion

We studied the plaquette Hubbard model under photoirradiation, with a goal of
dynamical enhancement of $d$-wave superconductivity. By focusing on a regime
away from half-filling ($\rho=0.875$), we observed a moderate increase of
pairing (15% for $t^{\prime}_{h}/t_{h}^{\phantom{{}^{\prime}}}=0.8$) at long
times in relation to the equilibrium ground state. This is accompanied by
enhanced staggered spin fluctuations. Such correlated behavior is not
unfamiliar to the physics of unconventional superconductors, where a close
competition or entwining of states displaying spin and pairing order is known
to occur. That we dynamically reach a simultaneous enhancement of both pair
and spin degrees of freedom provides a further, interesting window into the
interplay of the low-energy physics of the weakly doped Hubbard model.

A fundamental difference between our study and others that have numerically
investigated the increase of superconducting correlations in time is that, in
our case, the unperturbed Hamiltonian does not feature strong charge or spin
fluctuations associated to long-range order. In particular, the unperturbed
system is not a Mott insulator as in Ref. 4 nor a Peierls insulator as in Ref.
13. Consequently, the gaps separating the ground state and others excited
states are substantially smaller. This is one of the reasons behind our pump
pulses having an approximate one order of magnitude smaller frequencies.

Our protocol, on the other hand, is intrinsically tied to the specific details
of the Hamiltonian’s low-lying spectrum, and can be used more generally to
induce changes of the wave-function characteristics, surpassing the specific
focus on superconductivity [6, 8]. It is worth noticing that a requirement to
achieve an efficient targetting of states is that these are well separated in
energy from the bulk of the spectrum, otherwise the overlaps with non-desired
states is unavoidable. When approaching the thermodynamical limit (not
addressed here) the low-lying eigenspectrum is still discrete (that is, does
not form a continuous bulk), and identification of targeting protocol is still
feasible.

Nonetheless, a confirmation of these results in larger cluster sizes may shed
light on the understanding of its impact in explaining potential connections
to experimental results. We stress, however, that in dealing with small
clusters, finite-size effects for $d$-wave correlators do not monotonic
decrease with the system size, but are rather influenced by the number $z_{d}$
of independent neighboring $d$-wave plaquettes [42]. In the thermodynamic
limit $z_{d}=4$, and the smallest cluster size featuring such configuration
possess $N=20$. This sits at the verge of what is computationally accessible
within the scope of exact numerical methods. A potential venue of future
investigation is to understand the competition of charge textures (as stripes)
and pairing, now adding real-time dynamics to potentially enhance one degree
of freedom in detriment of the other. This builds prospects to connect to
recent experimental results in cuprates [16, 18, 43, 19].

A word of caution, however, is that realistic treatment of such materials
using model Hamiltonians might necessarily take into account either phonon
degrees of freedom [14] or the effects of multiple bands [44], such as the
distinct Cu and O orbitals. Still, the ‘bottom-up’ approach we develop to
first understand the physics of the low-lying spectrum in order to justify
dynamical modifications of suitable correlators should similarly work.

##  Appendix A The half-filled case

In the main text, we focus on the case of a hole-doped system ($n_{e}=14$),
guided by the enhanced superconducting properties in cuprate materials in
these conditions, and by the large pair binding energy observed in a similar
Hamiltonian [22]. Yet, the possibility that enhanced pairing can be obtained
even when the parent system is a robust insulator has also been proven before
[13, 4]. Thus, using our guiding protocol to infer the likelihood that a
photoirradiation perturbation, assessing first the properties of the low-lying
spectrum, we compute the largest relative enhancement of correlators at half-
filling ($n_{e}=16$), in analogy to Fig. 3.

Figure 6: The equivalent of Fig. 3 but for the half-filled system at
$U/t_{h}=8$. Within a span of 24 eigenstates of the low-lying spectrum at zero
quasi-momentum (the same where the ground-state resides) no siginificant
increase of matrix elements for pairing correlators in respect to its ground-
state average is observed.

We notice that other than potentially larger spin fluctuations, the remaining
observables we investigate do not possess large matrix elements in the low-
lying eigenspectrum in comparison to the ground-state. As a result, no
significant dynamical enhancement of pairing is expected in this scenario,
including, specifically, $d$-wave pairing.

##  Appendix B Enhancement of $d$-wave pairing at neighboring frequencies

In the main text we have used a specific pump frequency,
$\omega_{0}=0.257\,t_{h}$. Here we discuss the rationale behind this choice,
based on the results for the phase averaged enhancement of the $P_{d}$ pairing
correlators at different long times, as displayed in Fig. 7. The maximum
enhancement of the $d$-wave pairing at times following the pulse are obtained
for frequencies $\omega_{0}$ slightly below $E_{6,7}-E_{0}=0.2648t_{h}$. The
reason is that, despite of the target doublet states being mostly excited,
contributions from other states to the dynamics, as e.g. state $|1\rangle$,
lead to an optimal selection of pump parameters that are slightly below a
simple locked-in frequency analysis at $E_{\rm target}-E_{0}$.

Figure 7: The phase averaged value of the $d$-wave pairing correlator at
various late times as marked. Empty markers give the expectation values within
various low-lying eigenstates.

In other contexts, when initial and target states are related by a first order
phase transition in a slightly different regime of parameters of the
Hamiltonian, resonant-selection of the states is often highly effective [6,
8], since in such cases they are well separated from the bulk of the spectrum
[45]. This may explain recent observations of enhancement of $d$-wave pairing
in the vicinity of a phase boundary in systems where the interplay of electron
and phonon degrees of freedom is present [13, 14].

###### Acknowledgements.

The work of YXZ, and RTS was supported by the grant DE‐SC0014671 funded by the
U.S. Department of Energy, Office of Science. R.M. acknowledges support from
the National Natural Science Foundation of China (NSFC) Grants No. U1930402,
12050410263, 12111530010 and No. 11974039. Computations were performed on the
Tianhe-2JK at the Beijing Computational Science Research Center.

## References

  * Oka and Kitamura [2019] T. Oka and S. Kitamura, Floquet engineering of quantum materials, Annual Review of Condensed Matter Physics 10, 387 (2019).
  * Wang  _et al._ [2017] Y. Wang, M. Claassen, B. Moritz, and T. Devereaux, Producing coherent excitations in pumped Mott antiferromagnetic insulators, Physical Review B 96, 235142 (2017).
  * Wang  _et al._ [2018a] Y. Wang, M. Claassen, C. D. Pemmaraju, C. Jia, B. Moritz, and T. P. Devereaux, Theoretical understanding of photon spectroscopies in correlated materials in and out of equilibrium, Nature Reviews Materials 3, 312 (2018a).
  * Kaneko  _et al._ [2019] T. Kaneko, T. Shirakawa, S. Sorella, and S. Yunoki, Photoinduced $\eta$ pairing in the Hubbard model, Physical Review Letters 122, 077002 (2019).
  * Shirakawa  _et al._ [2020] T. Shirakawa, S. Miyakoshi, and S. Yunoki, Photoinduced $\eta$ pairing in the Kondo lattice model, Phys. Rev. B 101, 174307 (2020).
  * Shao  _et al._ [2019] C. Shao, H. Lu, H.-G. Luo, and R. Mondaini, Photoinduced enhancement of bond order in the one-dimensional extended Hubbard model, Physical Review B 100, 041114 (2019).
  * McIver  _et al._ [2020] J. McIver, B. Schulte, F.-U. Stein, T. Matsuyama, G. Jotzu, G. Meier, and A. Cavalleri, Light-induced anomalous Hall effect in graphene, Nature Physics 16, 38 (2020).
  * Shao  _et al._ [2021] C. Shao, P. D. Sacramento, and R. Mondaini, Photoinduced anomalous Hall effect in the interacting Haldane model: Targeting topological states with pump pulses, Phys. Rev. B 104, 125129 (2021).
  * Cavalleri [2018] A. Cavalleri, Photo-induced superconductivity, Contemporary Physics 59, 31 (2018).
  * Peronaci  _et al._ [2015] F. Peronaci, M. Schiró, and M. Capone, Transient dynamics of $d$-wave superconductors after a sudden excitation, Physical Review Letters 115, 257001 (2015).
  * Wang  _et al._ [2016] Y. Wang, B. Moritz, C.-C. Chen, C. Jia, M. van Veenendaal, and T. P. Devereaux, Using nonequilibrium dynamics to probe competing orders in a Mott-Peierls system, Physical Review Letters 116, 086401 (2016).
  * Sentef  _et al._ [2017] M. A. Sentef, A. Tokuno, A. Georges, and C. Kollath, Theory of laser-controlled competing superconducting and charge orders, Phys. Rev. Lett. 118, 087002 (2017).
  * Wang  _et al._ [2018b] Y. Wang, C.-C. Chen, B. Moritz, and T. Devereaux, Light-enhanced spin fluctuations and d-wave superconductivity at a phase boundary, Physical Review Letters 120, 246402 (2018b).
  * Wang  _et al._ [2021a] Y. Wang, T. Shi, and C.-C. Chen, Fluctuating nature of light-enhanced $d$-wave superconductivity: A time-dependent variational non-Gaussian exact diagonalization study, Phys. Rev. X 11, 041028 (2021a).
  * Tang  _et al._ [2021] T. Tang, Y. Wang, B. Moritz, and T. P. Devereaux, Orbitally selective resonant photodoping to enhance superconductivity, arXiv preprint arXiv:2106.04742 (2021).
  * Fausti  _et al._ [2011] D. Fausti, R. Tobey, N. Dean, S. Kaiser, A. Dienst, M. C. Hoffmann, S. Pyon, T. Takayama, H. Takagi, and A. Cavalleri, Light-induced superconductivity in a stripe-ordered cuprate, Science 331, 189 (2011).
  * Kaiser  _et al._ [2014] S. Kaiser, C. R. Hunt, D. Nicoletti, W. Hu, I. Gierz, H. Y. Liu, M. Le Tacon, T. Loew, D. Haug, B. Keimer, and A. Cavalleri, Optically induced coherent transport far above ${T}_{c}$ in underdoped YBa2C3O6+δ, Phys. Rev. B 89, 184516 (2014).
  * Nicoletti  _et al._ [2014] D. Nicoletti, E. Casandruc, Y. Laplace, V. Khanna, C. R. Hunt, S. Kaiser, S. Dhesi, G. Gu, J. Hill, and A. Cavalleri, Optically induced superconductivity in striped La2-xBaxCuO4 by polarization-selective excitation in the near infrared, Physical Review B 90, 100503 (2014).
  * Cremin  _et al._ [2019] K. A. Cremin, J. Zhang, C. C. Homes, G. D. Gu, Z. Sun, M. M. Fogler, A. J. Millis, D. N. Basov, and R. D. Averitt, Photoenhanced metastable c-axis electrodynamics in stripe-ordered cuprate La1.885Ba0.115CuO4, Proceedings of the National Academy of Sciences 116, 19875 (2019).
  * Tsai and Kivelson [2006] W.-F. Tsai and S. A. Kivelson, Superconductivity in inhomogeneous Hubbard models, Physical Review B 73, 214510 (2006).
  * Yao  _et al._ [2007] H. Yao, W.-F. Tsai, and S. A. Kivelson, Myriad phases of the checkerboard Hubbard model, Physical Review B 76, 161104 (2007).
  * Tsai  _et al._ [2008] W.-F. Tsai, H. Yao, A. Läuchli, and S. A. Kivelson, Optimal inhomogeneity for superconductivity: Finite-size studies, Physical Review B 77, 214502 (2008).
  * Doluweera  _et al._ [2008] D. Doluweera, A. Macridin, T. Maier, M. Jarrell, and T. Pruschke, Suppression of $d$-wave superconductivity in the checkerboard Hubbard model, Physical Review B 78, 020504 (2008).
  * Yao and Kivelson [2010] H. Yao and S. A. Kivelson, Fragile Mott insulators, Physical Review Letters 105, 166402 (2010).
  * Smith and Kennett [2013] P. M. Smith and M. P. Kennett, Disorder effects on superconducting tendencies in the checkerboard hubbard model, Phys. Rev. B 88, 214518 (2013).
  * Ying  _et al._ [2014] T. Ying, R. Mondaini, X. Sun, T. Paiva, R. Fye, and R. Scalettar, Determinant quantum Monte Carlo study of $d$-wave pairing in the plaquette Hubbard Hamiltonian, Physical Review B 90, 075121 (2014).
  * Kivelson  _et al._ [2003] S. A. Kivelson, I. P. Bindloss, E. Fradkin, V. Oganesyan, J. Tranquada, A. Kapitulnik, and C. Howald, How to detect fluctuating stripes in the high-temperature superconductors, Reviews of Modern Physics 75, 1201 (2003).
  * Berg  _et al._ [2009] E. Berg, E. Fradkin, S. A. Kivelson, and J. M. Tranquada, Striped superconductors: how spin, charge and superconducting orders intertwine in the cuprates, New Journal of Physics 11, 115004 (2009).
  * Vojta [2009] M. Vojta, Lattice symmetry breaking in cuprate superconductors: stripes, nematics, and superconductivity, Advances in Physics 58, 699 (2009).
  * Huang  _et al._ [2018] E. W. Huang, C. B. Mendl, H.-C. Jiang, B. Moritz, and T. P. Devereaux, Stripe order from the perspective of the Hubbard model, npj Quantum Materials 3, 1 (2018).
  * Agterberg  _et al._ [2020] D. F. Agterberg, J. S. Davis, S. D. Edkins, E. Fradkin, D. J. Van Harlingen, S. A. Kivelson, P. A. Lee, L. Radzihovsky, J. M. Tranquada, and Y. Wang, The physics of pair-density waves: cuprate superconductors and beyond, Annual Review of Condensed Matter Physics 11, 231 (2020).
  * Tranquada [2020] J. M. Tranquada, Cuprate superconductors as viewed through a striped lens, Advances in Physics 69, 437 (2020).
  * Dehollain  _et al._ [2020] J. P. Dehollain, U. Mukhopadhyay, V. P. Michal, Y. Wang, B. Wunsch, C. Reichl, W. Wegscheider, M. S. Rudner, E. Demler, and L. M. K. Vandersypen, Nagaoka ferromagnetism observed in a quantum dot plaquette, Nature 579, 528 (2020).
  * Manmana  _et al._ [2005] S. R. Manmana, A. Muramatsu, and R. M. Noack, Time evolution of one-dimensional quantum many body systems, AIP Conf. Proc. 789, 269 (2005).
  * Prelovšek and Bonča [2013] P. Prelovšek and J. Bonča, Ground state and finite temperature Lanczos methods, in  _Strongly Correlated Systems: Numerical Methods_, edited by A. Avella and F. Mancini (Springer Berlin Heidelberg, Berlin, Heidelberg, 2013) pp. 1–30.
  * Zheng  _et al._ [2017] B.-X. Zheng, C.-M. Chung, P. Corboz, G. Ehlers, M.-P. Qin, R. M. Noack, H. Shi, S. R. White, S. Zhang, and G. K.-L. Chan, Stripe order in the underdoped region of the two-dimensional Hubbard model, Science 358, 1155 (2017).
  * Huang  _et al._ [2017] E. W. Huang, C. B. Mendl, S. Liu, S. Johnston, H.-C. Jiang, B. Moritz, and T. P. Devereaux, Numerical evidence of fluctuating stripes in the normal state of high-${T}_{c}$ cuprate superconductors, Science 358, 1161 (2017).
  * Ponsioen  _et al._ [2019] B. Ponsioen, S. S. Chung, and P. Corboz, Period 4 stripe in the extended two-dimensional Hubbard model, Phys. Rev. B 100, 195141 (2019).
  * D’Alessio and Rigol [2014] L. D’Alessio and M. Rigol, Long-time behavior of isolated periodically driven interacting lattice systems, Phys. Rev. X 4, 041048 (2014).
  * Sorg  _et al._ [2014] S. Sorg, L. Vidmar, L. Pollet, and F. Heidrich-Meisner, Relaxation and thermalization in the one-dimensional Bose-Hubbard model: A case study for the interaction quantum quench from the atomic limit, Phys. Rev. A 90, 033606 (2014).
  * Bauer  _et al._ [2015] A. Bauer, F. Dorfner, and F. Heidrich-Meisner, Temporal decay of Néel order in the one-dimensional Fermi-Hubbard model, Phys. Rev. A 91, 053628 (2015).
  * Maier  _et al._ [2005] T. A. Maier, M. Jarrell, T. C. Schulthess, P. R. C. Kent, and J. B. White, Systematic study of $d$-wave superconductivity in the 2d repulsive Hubbard model, Phys. Rev. Lett. 95, 237001 (2005).
  * Först  _et al._ [2014] M. Först, R. I. Tobey, H. Bromberger, S. B. Wilkins, V. Khanna, A. D. Caviglia, Y.-D. Chuang, W. S. Lee, W. F. Schlotter, J. J. Turner, M. P. Minitti, O. Krupin, Z. J. Xu, J. S. Wen, G. D. Gu, S. S. Dhesi, A. Cavalleri, and J. P. Hill, Melting of charge stripes in vibrationally driven La1.875Ba0.125CuO4: Assessing the respective roles of electronic and lattice order in frustrated superconductors, Phys. Rev. Lett. 112, 157002 (2014).
  * Wang  _et al._ [2021b] Y. Wang, Y. Chen, T. P. Devereaux, B. Moritz, and M. Mitrano, X-ray scattering from light-driven spin fluctuations in a doped Mott insulator, Communications Physics 4, 212 (2021b).
  * Chen  _et al._ [2010] X. Chen, Z.-C. Gu, and X.-G. Wen, Local unitary transformation, long-range quantum entanglement, wave function renormalization, and topological order, Phys. Rev. B 82, 155138 (2010).


# Mask CycleGAN : Unpaired Multi-modal Domain Translation with Interpretable
Latent Variable

Minfa Wang
Stanford University
<EMAIL_ADDRESS>

###### Abstract

We propose Mask CycleGAN, a novel architecture for unpaired image domain
translation built based on CycleGAN [6], with an aim to address two issues: 1)
uni-modality in image translation and 2) lack of interpretability of latent
variables. Our innovation in the technical approach is comprised of three key
components: masking scheme, generator and objective. Experimental results
demonstrate that this architecture is capable of bringing variations to
generated images in a controllable manner and is reasonably robust to
different masks.

##  1 Introduction

CycleGAN [6] is a popular approach for unpaired image-to-image translation
between two domains. It has been proven to be effective in a wide variety of
domain translation tasks, including horse-zebra, apple-orange, summer-winter,
etc. While it keeps inspiring generative modelling community to build up more
and more applications and research ideas, CycleGAN has its limitations too.
One notable limitation is that the translation is deterministic and hence lack
of variation. People have discovered that achieving multi-modality through
CycleGAN is challenging [2, 7], largely due to the reason that the source
image is at high dimensionality and usually causes the generator network to
ignore other noise sampled from other distribution. A natural idea to enable
multi-modal image generation is by introducing additional latent variables,
which are often modeled as Gaussian distributions. However, samples from
Gaussian distributions are generally lack of interpretability.

Mask CycleGAN aims to address both issues above by using pixel mask as latent
variables. Figure 1 shows a high-level overview of our architecture and
comparison with other popular architectures. In later sections, we will
elaborate the architecture in details. We will show its formulation is a full
generalization of CycleGAN, and hence is at least equally expressive.
Moreover, the pixel mask offers great control of the image generation outcome.

Figure 1: Comparison of variants of CycleGAN.

##  2 Related Work

##### CycleGAN

[6] is a popular approach for unpaired image-to-image translation between two
domains. One of the most innovative ideas from this work is cycle consistency,
which encourages the mapping between two domains to be invertible, and
indirectly alleviate the problem of mode collapse. CycleGAN is capable of
generating visually appealing images.

##### Augmented CycleGAN

[2] brings multi-modality into CycleGAN by augmenting it with two latent
variables $Z_{a}$ and $Z_{b}$, and corresponding encoders $E_{a}$, $E_{b}$ and
discriminators $D_{Z_{a}}$, $D_{Z_{b}}$. The latent variables are optimized
through the minimax game between the encoder and the discriminator.

##### BicycleGAN

[7] is an architecture for paired multi-modal image-to-image translation. The
name Bicycle refers to the two cycles from 1) Conditional Variational
Autoencoder GAN (cVAE-GAN) and 2) Conditional Latent Regressor GAN (cLR-GAN).

##### Image inpainting

is a technique to reconstruct a image patch from a partially covered or
blurred image. The input of image inpainting can be thought as a clear source
image applied on a mask, which is similar to our setup. Some of the works [5]
in this field inspires the generator design of the Mask CycleGAN to impose
soft constraint on pixel invariance on certain image region during
transformation.

##### Attention

is technique to discover and make use of region of interests from the input by
assigning different weights to different parts of the input. When the input is
image, the weight is typically computed at pixel level, and the attention
weight map can be thought of as a soft mask. People [4] have attempted to use
unsupervised attention mask to improve generative modelling.

##  3 Problem statement

The task for CycleGAN-alike architectures is unpaired image-translation
between two domains. At inference time, an image from source domain will be
given as input, and the output will be an image from target domain, retaining
the basic features of the input image. The mathematical formulation is
introduced in the Notation section below.

###  3.1 Notation

We introduce the following terminologies to help elaborate the technical
approach. Same as CycleGAN, we have $a$ as a sample image from the source
domain, $b$ as a sample from the destination domain, and $G_{AB}$ as generator
mapping an image from source to destination. With mask $m$, we could derive
the following properties: $\tilde{b}=G_{AB}(a,m)$ as the fake image in
destination domain, $a^{\prime}=G_{BA}(\tilde{b},m)$ as the recovered image in
source domain. Other terms like $\tilde{a}$, $b^{\prime}$ and $G_{BA}$ are
defined symmetrically.

Regarding the mask, we call masked region to be the region where the
information of the pixel values of the image is kept, and the contextual
region to be the region outside the masked region.

##  4 Approach

Our architecture is based on CycleGAN. Please refer to [6] for more details of
its original architecture. The sections below elaborate the key modifications
we made to incorporate masks into the whole system.

###  4.1 Masking

####  4.1.1 Discussion: binary vs. continuous mask

##### Binary mask

a mask that has value 1 on its masked area, and 0 elsewhere. It is a kind of
hard mask, eliminating all the information from the contextual region of the
image.

##### Continuous mask

a mask that has float number between 0 and 1 for all its dimensions,
representing the weights of different pixels of the image. It is a soft mask
where the boundary between masked and contextual regions could be blurry, and
the information for contextual region will be partially retained after the
mask is applied on the image.

One of the reasons that it is challenging to introduce multi-modality in
CycleGAN is that when you feed a concatenation of image features and latent
variables as the input to the generator, the generator quickly learns that the
dimensions of the latent variables provides little additional values in
optimizing the overall objective, and hence zeros out those dimensions,
forfeiting multi-modality.

Thanks to the interpretability of the mask, we can re-design the interaction
between the mask and the input image to force the generator to respect the
effect of the mask. The revamped generator design is detailed in the following
Generator section. To avoid unexpected information leaking to the generator,
we choose binary masks in this work. Below is a list of masking schemes that
we have considered.

####  4.1.2 Centered-square masking scheme

The masked region always is centered in the image, has square shape, and has a
size of $(0.5|0.8|1.0)$ * image size. Figure 2(a) provides an visualization of
the masks generated from this scheme.

The centered-square masking scheme is simple to understand and fast to
evaluate. On the other hand, it is very limited on the amount variation it is
capable of producing.

####  4.1.3 Multi-rectangles masking scheme

Multi-rectangles masking scheme, elaborated in Algorithm 1, is a
generalization of the centered-square masking scheme. It allows more
variations in size, position and (compound) shape of the mask, encouraging the
generator to generalize better. The limitation of this masking scheme is that
it still produces rectangular edges. Figure 2(b) provides an visualization of
some samples generated from this masking scheme.

// Minimal number of rectangles to draw.

Set MinMaxNumRects = 5, MinNumRects = uniform(1, MinMaxNumRects) ;

// Minimal accumulative area relative to the whole image area.

Set MinSumRelArea = 0.15 ;

// Size of image; lower and upper bound of individual rectangle size.

Set Size = 128, MinRectSize = Size / 10, MaxRectSize = Size ;

// Initialize variables to keep track of status in the loop.

Set numRects = 0 ;

Set sumRelArea = 0.0 ;

Set mask = zeros(3, Size, Size) ;

while  _numRects < MinNumRects or sumRelArea < MinSumRelArea_ do

// Randomly generate the top left corner of the rectangle.

i0 = uniform(0, Size - MinRectSize) ;

j0 = uniform(0, Size - MinRectSize) ;

// Randomly generate the bottom right corner of the rectangle.

i1 = uniform(i0 + MinRectSize, min(i0 + MaxRectSize, Size)) ;

j1 = uniform(j0 + MinRectSize, min(j0 + MaxRectSize, Size)) ;

// Draw rectangle on the mask. Update intermediate states.

mask[:, i0:i1, j0:j1] = 1 ;

numRects += 1 ;

sumRelArea += ((i1 - i0) * (j1 - j0)) / (Size * Size) ;

end while

Algorithm 1 Multi-rectangles masking scheme

Some properties of this masking scheme:

  * •

When MinRectSize = 1 and MaxRectSize = 2, this masking scheme is equivalent to
sample individual pixels independently.

  * •

When MinSumRelArea = 1, this masking scheme will always generate the full
mask, making the whole algorithm become equivalent to CycleGAN.

  * •

The time complexity of this algorithm is O(max(MinNumRects, MinRelArea /
(MinRectSize / Size)2)).

####  4.1.4 Binary-attention masking scheme

If we have access to some pre-trained image attention network, which is able
to produce a pixel level attention map, then we could potentially convert it
to a binary mask by binarizing the attention weights on all pixels based on
some threshold.

Figure 2(c) provides an illustration of the binary-attention masking scheme.

Figure 2: Visualization and comparison of different Masking schemes. (c) is
adopted from an image in [1].

###  4.2 Generator

The generator design is one of the critical parts of the architecture. Figure
3(a) shows the plain-vanilla generator used in original CycleGAN that maps a
source image to a destination image. It is easy to see that the mapping is
deterministic with a fixed source image. In literature, when people try to
introduce the additional latent variable $z$, a common approach [7, 2] is
concatenates the latent variable vector to some intermediate vector
representation of the source image. The drawbacks of this approach are 1) the
influence of latent variable is hard to interpret and control, and 2)
empirically the generator often tends to ignore variations in the latent
variable.

Since our latent variable, $m$, is interpretable, we could design its
interaction with the source image in a way that enforces the masking behavior.
The interaction is defined through the mask encoder $E$, with its architecture
detailed in Figure 3(b). The full generator design is shown in Figure 3(c),
where complicated domain translation logic, captured in $G$, is required to
only depend on masked region of the source image.

Figure 3: Comparison of generators. (b) is the expanded view of the mask
encoder $E$ used in (c).

###  4.3 Losses

Similar to CycleGAN, we optimize the model by minimizing three pairs of
losses. We have made modifications on each pair to accommodate the
architecture change introduced by the mask.

##### GAN loss

For each triplet $(a,\tilde{a},m$), we will have two discriminators $D_{AF}$
and $D_{AM}$. The first generator $D_{AF}$ is functionally equivalent to the
discriminator $D_{A}$ in original CycleGAN, which is trying to distinguish a
true image from a generated one for the source distribution, and implicitly
encourages the generator to produce overall coherent image inside and outside
the masked area. The second discriminator $D_{AM}$ is responsible for the same
task for the masked image pair. We have the final GAN loss as the normalized
weighted sum of the two discriminators’ losses. Formally:

$\displaystyle\mathcal{L}_{GAN}^{AF}$ $\displaystyle=\mathbb{E}_{a\sim A}[\log
D_{AF}(a)]+\mathbb{E}_{\tilde{a}\sim\tilde{A}}[\log(1-D_{AF}(\tilde{a}))]$
$\displaystyle\mathcal{L}_{GAN}^{AM}$ $\displaystyle=\mathbb{E}_{a\sim A}[\log
D_{AM}(a\odot
m)]+\mathbb{E}_{\tilde{a}\sim\tilde{A}}[\log(1-D_{AM}(\tilde{a}\odot m))]$
$\displaystyle\mathcal{L}_{GAN}^{A}$
$\displaystyle=\lambda_{GAN}^{M}\mathcal{L}_{GAN}^{AM}+(1-\lambda_{GAN}^{M})\mathcal{L}_{GAN}^{AF}$

There are two remarks. First, we need to introduce the second discriminator
$D_{AM}$ because despite the fact that generator tries to produce coherent
images, there would generally exists some discontinuity around the mask
boundary, making the task relatively easy for $D_{A}$, and in turn causes
gradient vanishing problems to the generator. Even if the generator manages to
learn, without $D_{AM}$, the generator would tend to ignore the mask and
suffer from mode collapse. Second, when $\lambda_{GAN}^{M}L_{GAN}^{AM}=0$,
this objective falls back to the GAN objective in original CycleGAN.

##### Cycle loss

facilitates cycle-consistency for translation between two domains. The high-
level idea is that when a image goes through a forward mapping and then a
backward mapping, it should recover to itself, which requires the forward
mapping to preserve the information of the source image. This is an effective
method to prevent mode collapse. Our loss objective is the normalized weighted
sum of the cycle losses for the masked and context areas.

$\displaystyle\mathcal{L}_{CYC}^{A}=\lambda_{CYC}^{M}||(a-a^{\prime})\odot
m||_{1}+(1-\lambda_{CYC}^{M})||(a-a^{\prime})\odot(1-m)||_{1}$

Different weights are applied for masked and context areas, and the weights
could be adjusted to control how strict we hope the generator to keep the
pixels in the context area intact. When $\lambda_{CYC}^{M}=0.5$, this loss
falls back to the cycle loss in original CycleGAN (with a scale factor).

##### Identity loss

states that when you try to map an image from the destination domain to the
destination domain, it should map to itself. Formally:

$\displaystyle\mathcal{L}_{IDT}^{A}=||a-G_{BA}(a,m)||_{1}$

Note this formulation is identical to the identity loss in original CycleGAN.
We don’t need special treatment for the masked area because we intentionally
want this specific mapping to be invariant of the mask.

###  4.4 Full objective

The full objective is

$\mathcal{L}=(\mathcal{L}_{GAN}^{A}+\mathcal{L}_{GAN}^{B})+\lambda_{CYC}(\mathcal{L}_{CYC}^{A}+\mathcal{L}_{CYC}^{B})+\lambda_{IDT}(\mathcal{L}_{IDT}^{A}+\mathcal{L}_{IDT}^{B})$

###  4.5 Algorithm

initialize weights and optimizers ;

initialize $G_{AB}$, $G_{BA}$, $D_{AF}$, $D_{BF}$, $D_{AM}$, $D_{BM}$ ;

while  _True_ do

Read next batch $a$, $b$ ;

Sample $m$ from $M$ ;

$\tilde{b}$ = $G_{AB}(a,m)$, $\tilde{a}$ = $G_{BA}(b,m)$ ;

$a^{\prime}$ = $G_{BA}(\tilde{b},m)$, $b^{\prime}$ = $G_{AB}(\tilde{a},m)$ ;

// \--- optimize generator below ---

l_cyc_a = $\lambda_{CYC}^{M}$ * cyc_loss(a * m, $a^{\prime}$ * m) + (1 -
$\lambda_{CYC}^{M}$) * cyc_loss(a * (1 - m), $a^{\prime}$ * (1 - m)) ;

l_cyc_b = $\lambda_{CYC}^{M}$ * cyc_loss(b * m, $b^{\prime}$ * m) + (1 -
$\lambda_{CYC}^{M}$) * cyc_loss(b * (1 - m), $b^{\prime}$ * (1 - m)) ;

l_idt = idt_loss(a, $G_{BA}$(a, m)) + idt_loss(b, $G_{AB}$(b, m)) ;

l_gan_a = (1 - $\lambda_{GAN}^{M}$) * gan_loss($D_{AF}$($\tilde{a}$)) +
$\lambda_{GAN}^{M}$ * gan_loss($D_{AM}$($\tilde{a}$ * m))) ;

l_gan_b = (1 - $\lambda_{GAN}^{M}$) * gan_loss($D_{BF}$($\tilde{b}$)) +
$\lambda_{GAN}^{M}$ * gan_loss($D_{BM}$($\tilde{b}$ * m))) ;

l_g = (l_gan_a + l_gan_b) + $\lambda_{CYC}$ * (l_cyc_a + l_cyc_b) +
$\lambda_{IDT}$ * l_idt ;

l_g.backward() ;

optimizer_g.step(l_g) ;

// \--- optimize discriminator below ---

l_da = (1 - $\lambda_{GAN}^{M}$) * d_loss($D_{AF}$, a, $\tilde{a}$) +
$\lambda_{GAN}^{M}$ * d_loss($D_{AM}$, a * m, $\tilde{a}$ * m) ;

l_db = (1 - $\lambda_{GAN}^{M}$) * d_loss($D_{BF}$, b, $\tilde{b}$) +
$\lambda_{GAN}^{M}$ * d_loss($D_{BM}$, b * m, $\tilde{b}$ * m) ;

l_d = l_da + l_db ;

l_d.backward() ;

optimizer_d.step(l_d) ;

end while

Algorithm 2 Masked CycleGAN

##  5 Experiments

###  5.1 Setup

We evaluated the model on the following datasets: MNIST-SVHN, Horse-Zebra,
Monet-Photo, Vangogh-Photo. The image resolution for all datasets is 128x128,
except for MNIST-SVHN, which is 32x32. For all the experiments, we used the
same hyper-parameter settings as follow: $\lambda_{GAN}^{M}=0.7$,
$\lambda_{CYC}^{M}=0.3$, $\lambda_{CYC}=10$ and $\lambda_{IDT}=5$.

We experimented with the centered-square and multi-rectangles masking schemes,
and evaluated the algorithm both qualitatively and quantitatively. For
quantitative analysis, we used the FID score of test set and the set generated
with full mask as baseline.

###  5.2 Quantitative Results

##### Frechet Inception Distance (FID)

[3] is a method to measure the performance of a generative model by computing
the following distance on the inception feature representation of the
generated data distribution and true data distribution fitted by multivariate
Gaussian. We use FID as the main quantitative metric for our model.

####  5.2.1 Train with centered-square masking scheme

Figure 4: FID matrix of Mask CycleGAN trained with centered-square masking
scheme on Horse-Zebra dataset. (a) shows FID scores among real and generated
A:horse datasets. (b) shows FID scores among real and generated B:Zebra
datasets.

We computed FID scores for various pairs of datasets derived from the Horse-
Zebra dataset. The results are represented in two matrices shown in Figure 4,
where (a) shows FID scores for various horse datasets, and (b) shows FID
scores for various zebra datasets.

There are several conclusions we could draw from the matrices:

  * •

$FID_{*}(scale=*,train)<FID_{*}(scale=*,test)$. The generator was trained on
the training dataset, and hence would be able to emulate the training data
distribution better.

  * •

$FID_{A}(train,test)>FID_{B}(train,test)$. According to FID, the horse dataset
has more variations in style. This characteristic matches with our human-eye
judgement.

  * •

$FID_{A}(scale=*,test)$ > $FID_{B}(scale=*,test)$. The generator performs
worse in emulating the horse distribution. This could be attributed to the
intrinsic difficulty of horse dataset.

  * •

$\boldsymbol{FID_{B}(scale=0.8,test)<FID_{B}(scale=1.0,test)}$. This is
probably the most interesting finding. Usually, the larger the mask scale, the
more information is exposed to the generator, and hence generator has more
expressive power to fit a distribution. However, for the horse dataset, we see
that the model performs better with the comparably smaller mask. One
explanation is that typically the object of interest is presented around the
center of the image, and a smaller mask actually filters out some noise and
produces some mild regularization.

####  5.2.2 Train with multi-rectangles masking scheme

We also conducted FID evaluation on the model with multi-rectangles masking
scheme. The result is shown in Figure 5.

Figure 5: FID matrix for model with multi-rectangles masking scheme, evaluated
on horse-zebra dataset.

From the matrices, we could draw the same conclusions as the previous
experiment. In addition, what is worth noting is that regardless of masking
schemes in training, we used centered-square masking scheme in inference.
Nonetheless, on $FID_{B}(scale=0.8,test)$, the model trained with multi-
rectangles masking scheme outperforms the model trained with centered-square
masking scheme: 84.83 vs. 101.17. This is a promising sign that more
variations in masks in training could help the model generalize better in
inference.

###  5.3 Qualitative Results

We examine the qualitative results of the model through a output grid shown in
Figure 6. The model used in the grid was trained only on centered-square
masks, but was evaluated on both centered-square and multi-rectangles masks.
It is shown from the output that Mask CycleGAN is robust in translation across
many image domains, and is able to generalize to work with mask it has never
seen during training.

Figure 6: Outputs grid of Mask CycleGAN evaluated on several datasets with
different masks. First row shows different masks used on different columns.
First column shows the source images and the related tasks. Other cells show
the generator output with the corresponding source image and mask as inputs.

####  5.3.1 Generalization to round mask

One interesting question is that if only trained with rectangular masks, will
the model be able to generalize to round masks? We did the analysis by feeding
the generator with round masks of different scales and the outputs are shown
in Figure 7. Interestingly, model trained with centered-square masking scheme
appears to generalize better to round masks.

Figure 7: Models evaluated on round masks.

##  6 Conclusion

In this work, we proposed a novel generative modelling algorithm called Mask
CycleGAN. We introduced the motivation of the idea, the mathematical
formulation, and quantitative and qualitative evaluations of the algorithm. We
illustrate that Mask CycleGAN is capable of bringing variations in CycleGAN-
alike generators in a controllable manner. We believe this architecture will
open doors for a series of interesting applications.

In the future, we plan to further improve the robustness of the algorithm
along two directions: 1) design of generator, where we may experiment with
more sophisticated architecture applied on the context region of the image; 2)
explore different masking schemes like binary-attention to help the model
generalize better to more variations of masks during inference.

##  7 Acknowledgement

Special thanks to Kristy Choi for her feedback and advice offered in the
development of the project, and Sonya Chen for proofreading.

The code for this project is available at
https://www.github.com/minfawang/mask-cgan.

## References

  * Agarwal et al. [2013] C. Agarwal, A. Bose, S. Maiti, N. Islam, and S. Sarkar.  Enhanced data hiding method using dwt based on saliency model.  pages 1–6, 09 2013.  ISBN 978-1-4673-6188-0.  doi: 10.1109/ISPCC.2013.6663414.
  * Almahairi et al. [2018] A. Almahairi, S. Rajeswar, A. Sordoni, P. Bachman, and A. C. Courville.  Augmented cyclegan: Learning many-to-many mappings from unpaired data.  _CoRR_ , abs/1802.10151, 2018.  URL http://arxiv.org/abs/1802.10151.
  * Heusel et al. [2017] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, G. Klambauer, and S. Hochreiter.  Gans trained by a two time-scale update rule converge to a nash equilibrium.  _CoRR_ , abs/1706.08500, 2017.  URL http://arxiv.org/abs/1706.08500.
  * Mejjati et al. [2018] Y. A. Mejjati, C. Richardt, J. Tompkin, D. Cosker, and K. I. Kim.  Unsupervised attention-guided image to image translation.  _CoRR_ , abs/1806.02311, 2018.  URL http://arxiv.org/abs/1806.02311.
  * Pathak et al. [2016] D. Pathak, P. Krähenbühl, J. Donahue, T. Darrell, and A. A. Efros.  Context encoders: Feature learning by inpainting.  _CoRR_ , abs/1604.07379, 2016.  URL http://arxiv.org/abs/1604.07379.
  * Zhu et al. [2017a] J. Zhu, T. Park, P. Isola, and A. A. Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  _CoRR_ , abs/1703.10593, 2017a.  URL http://arxiv.org/abs/1703.10593.
  * Zhu et al. [2017b] J. Zhu, R. Zhang, D. Pathak, T. Darrell, A. A. Efros, O. Wang, and E. Shechtman.  Toward multimodal image-to-image translation.  _CoRR_ , abs/1711.11586, 2017b.  URL http://arxiv.org/abs/1711.11586.

##  8 Appendix

Figure 8: Outputs of Mask CycleGAN. (a) shows the results from MNIST-SVHN
translation. (b) shows horse-zebra translation. Please zoom in to inspect the
details.