# ColorwAI: Generative Colorways of Textiles through GAN and Diffusion
Disentanglement

Ludovica Schaerf  0000-0001-9460-702X Max Planck Society – University of
ZurichCenter for Digital Visual Studies -Culmannstrasse 1ZurichSwitzerland8006
<EMAIL_ADDRESS>,  Andrea Alfarano  Max Planck Society – University of
ZurichZurichSwitzerland<EMAIL_ADDRESS>and  Eric Postma  Cognitive
Science & AI, Tilburg University5037 ABTilburgNetherlands
<EMAIL_ADDRESS>
(2024)

###### Abstract.

Abstract. Colorway creation is the task of generating textile samples in
alternate color variations maintaining an underlying pattern. The
individuation of a suitable color palette for a colorway is a complex creative
task, responding to client and market needs, stylistic and cultural
specifications, and mood. We introduce a modification of this task, the
”generative colorway” creation, that includes minimal shape modifications, and
propose a framework, ”ColorwAI”, to tackle this task using color
disentanglement on StyleGAN and Diffusion. We introduce a variation of the
InterfaceGAN method for supervised disentanglement, ShapleyVec. We use Shapley
values to subselect a few dimensions of the detected latent direction.
Moreover, we introduce a general framework to adopt common disentanglement
methods on any architecture with a semantic latent space and test it on
Diffusion and GANs. We interpret the color representations within the models’
latent space. We find StyleGAN’s W space to be the most aligned with human
notions of color. Finally, we suggest that disentanglement can solicit a
creative system for colorway creation, and evaluate it through expert
questionnaires and creativity theory.

Colorways, Disentanglement, Textiles, Diffusion, GANs

††copyright: acmcopyright††journalyear: 2024††doi: XXXXXXX.XXXXXXX††ccs:
Applied computing Arts and humanities††ccs: Applied computing Media arts††ccs:
Computing methodologies Computer vision

##  1\. Introduction

Color profoundly impacts the general sentiment expressed by a textile,
influencing its perceptual structure and depth. Its importance is mirrored in
the design brief preceding everyday clothing and furniture production.
Alongside target specifications, technical requirements, and suggested moods,
a crucial part of this brief indicates the desired colorways (Briggs-Goode and
Townsend, 2011; Dickinson, 2011), alternate color variations of the same
pattern. The market typically demands around 6-8 colorways to meet diverse
societal needs.

These colorways can range from slight base-color variations that maintain the
original mood to completely different colorations that alter the feel, as in
Figure 1. The creation of textile colorways is complex and subjective,
influenced by historical and cultural contexts, technical discoveries, trends,
seasonal requirements, and individual preferences (Dickinson, 2011).

Figure 1. Example of a colorway in cold and warm tones. Pattern generated
using StyleGAN2-ADA and colorway generated using InterfaceGAN.

Example of a colorway in cold and warm tones. Pattern generated using
StyleGAN2-ADA and colorway generated using InterfaceGAN.

With traditional graphics programs and existing AI re-colorization methods, to
change the color scheme of the graphic design, a color palette must be
predetermined by the designer. With ColorwAI, the framework proposed in the
paper, the designer is invited to ”brainstorm with the machine about the
desired result”, tackling the ideation moment of the colorway. The designer
proposes a color and ColorwAI suggests the pattern in a novel color scheme
and, so on, through a repeating dialogue. In this way, the model prompts the
designer to consider original possibilities, sometimes adjusting the design’s
shapes. The model, because of the knowledge gained from learning historical
textiles, adjusts the pattern with slightly different lines and shapes, to be
more suitable to the new color scheme identified. It changes the shape of the
pattern to the minimum, in a more harmonious way in the model’s lens.

The purpose of the model is to suggest a ”generative colorway”, a colorway
that generates both chromatic and slight pattern variation, to stimulate the
designer’s creativity. This is intended to accompany the creative process of
the textile expert, who is traditionally given minimal specifications from the
design brief and must present colorways that are stylistically and
conceptually fitting the original design, but novel and original.

We propose to tackle the task of ”generative colorway” creation with our
framework ColorwAI, by relying on disentanglement. In deep neural networks,
acquired representations are high-dimensional latent spaces encompassing a
large variety of features. Disentanglement refers to the extraction of an
interpretable feature such as color from the high dimensional latent space.
Disentanglement learning is the task of learning a meaningful traversal
direction in the latent space corresponding to a semantic variation of the
output (Choi et al., 2021; Bengio, 2013). A direction that only influences one
semantic factor is considered disentangled from all other factors of
variation. In the context of machine learning and computer vision,
disentanglement is a well-established method for image editing and
interpretation of the hidden representations of generative neural networks.

Because of its nature, weakly supervised disentanglement can determine the
representation of color as a dimension within the latent space of generative
neural networks (Locatello et al., 2020). The color can be edited by
perturbing along the latent direction, thus creating a ”generative colorway”
around that base color. Because generative neural networks such as VAEs, GANs,
and Diffusion feature at least one coherently and semantically arranged latent
space (Karras et al., 2019; Kwon et al., 2022; Higgins et al., 2016), we
assume that a disentangled color direction provides visually plausible color
modifications, responding to the need for a suitable palette for the textile.
Whenever a perturbation from a starting pattern along the latent direction of
a color incurs into a lesser defined, warped, area of the latent space, the
modifications are more substantial, incurring structural modifications. This
may occur for a textile pattern when a certain chromaticity, e.g. blue, is
lacking in the training data. In such cases, disentanglement creates slight
pattern modifications, adjusting the original textile pattern into a similar
one for which a blue example does exist in the training data.

An additional advantage of disentanglement is that it allows understanding how
colors are represented within these models in the context of textiles. Given
the wealth of cultural, societal, and technical factors behind the use of
color in textiles, as expanded in Section 2.1, we wish to complement the
practical framework with insights into how the methods that generate these
colorways represent colors.

In this paper, we present ColorwAI, a framework for ”generative colorway”
creation that can be flexibly applied to any generative architecture with a
semantic latent space. We identify four stages in the disentanglement
pipeline: the coupling of latent codes with image annotations, the
disentanglement of the latent space, the modifications of the latent code, and
the inference with modified latent code. We show disentanglement results for
StyleGAN and DDPM (Karras et al., 2019; Ho et al., 2020), maintaining distinct
coupling and inference steps, but common disentanglement methods. This
approach to Diffusion is a generalization of BoundaryDiffusion (Zhu et al.,
2024). We test InterfaceGAN and StyleSpace disentanglement methods and
introduce a modification of InterfaceGAN, that we call ShapleyVec. The latter
uses a subset of latent space neurons for disentanglement (Wu et al., 2021;
Shen et al., 2020). We demonstrate the superiority of the latter method in
terms of quantitative performance and interpretability.

To summarize, the contribution of this work is as follows:

  1. (1)

We introduce a new task motivated by practical utility, the ”generative
colorway” creation, that extends existing work on color re-colorization with
creative controls;

  2. (2)

We present ColorwAI, a framework for color disentanglement that can be applied
agnostically to all generative architectures presenting a semantic latent
space, including GANs and Diffusion;

  3. (3)

We enrich the methods that deal with disentanglement to favor better color
disentanglement while maintaining interpretability and novelty through our
modification of InterfaceGAN, ShapleyVec. We interpret the color
representations within these models.

  4. (4)

We justify the creativity of the framework in the context of Computational
Creativity literature, conducting an expert survey.

##  2\. Context

####  2.0.1. Textiles and Colors

Colors and textures create complex interactions influencing human perception
and artistic evolution. Textiles, originally referring to woven (from Latin
textilis) fabrics, have accompanied humans since the beginning of
civilization, becoming a distinctive factor of humanity. In the West European-
Mediterranean sphere, its history interlaced with that of color already 5000
years ago in Egypt (Whiston, 1948). The colors came from the fibers and dyes.
Their pigments were extracted from elements of nature, as in the ancient
tradition of the madder root, creating shades of red and orange, and the
expensive carmine-yielding cochineal insects.

Pigment availability influenced cultural identities. Christian and Copts111The
Copts are an indigenous Christian ethnoreligious community in Egypt. colors
are distinctively subdued in line with religious and social reasons, as
opposed to the bright pigments of Assyrians and Babylonians, mirroring their
active and strenuous civilizations but also the availability of the vibrant
Tyrian purple (Whiston, 1948). In more recent centuries, color developments
instantiated trends that characterized their time, such as mauveine and
analine purple in the 1850s and turquoise a century later (Dickinson, 2011).

Despite the current availability of plentiful synthetic dyes, colors continue
to carry rich and layered meaning.

The use of color in textiles is influenced by its perception. Studies
initiated from Chevreul’s ’Laws of Contrast of Colour’ have shown that the co-
location of colors into several schemes creates a harmonious product
(Burchett, 2002), becoming the de-facto standard in color design. Bauhaus
artists such as Johannes Itten and Josef Albers, influenced by Goethe’s
’Theory of Colors’, explored color relationships affecting human perception.
Josef Albers (1963) investigated color interaction. He demonstrated how
perception of two identical colors on different backgrounds yields
perceptually distinct colors (Albers, 2013). Rules and costumes were
introduced after the Great Exhibition in London of 1851, including the
’Grammar of Ornaments’ by Owen Jones (Jones, 2016). Jones mentions, in the 37
principles for textile design, the importance of using a pondered distribution
of hues on the surface.

Using digital synthetic textures, (Wang et al., 2020) demonstrates the
influence of shape on the perception of the color composition in patterns,
showing how more elongated shapes lead to less prominent chromatic
impressions. The finding was already experimentally clear to Albers and Jones.
Our perception of color similarity and arrangement is merely functional.
Particularly, color patterns are very modestly understood by our vision
compared to texture, contrast, and color (Mojsilovic et al., 2000).

####  2.0.2. Texture Generation and Re-colorization

Relevant research in Computer Vision has focused on the generation of textures
(Portilla and Simoncelli, 2000; Karagoz et al., 2023; Dai et al., 2014; Lin et
al., 2023). By texture synthesis, we mean the generation of images with
structural components, spatially homogeneous and repeating, often fabrics or
materials samples. Early works adopt parametric statistical models such as
Portilla et al. (Portilla and Simoncelli, 2000), using complex wavelet
coefficients. More recently, papers use deep learning-based generative models
such as AE, GANs, and Diffusion. Bergmann et al. (Bergmann et al., 2018)
introduce a structural similarity loss to detect textural defects using
Autoencoders. Lin et al. (Lin et al., 2023) exploits StyleGAN to generate
realistic visual textures, obtaining high-quality samples and inversion
(Karras et al., 2019). Lastly, Karagoz et al. (Karagoz et al., 2023) generate
textile patterns using Stable Diffusion conditioned with a number of keywords
from textile metadata (Rombach et al., 2022).

Given the success of the last two models and the proven semantic arrangement
of their latent space (Karras et al., 2019; Kwon et al., 2022), we generate
textile samples and colorways using StyleGAN and Diffusion.

The generation of colorways is closely linked with the task of image re-
colorization. In this context, Xu et al. (Xu et al., 2019) developed a re-
colorization algorithm on fabrics based on parts segmentation and variation
based on given color themes. Cho et al. (Cho et al., 2017) introduced the
first deep neural network for content-aware image re-colorization. Yang et al.
(Yang et al., 2021), among other factors, disentangle color mood within scene
depictions. The color control is only partially addressed in (Yang et al.,
2021). In the context of conditioning methods, Li et al. (Li et al., 2020) use
text conditioning to vary the color of semantic image segments using natural
language.

Re-colorization algorithms are time-consuming and constrained by requiring the
selection of the predetermined palette, disentanglement methods are more
flexible because they change the entire palette based on a coarse input, but
have not comprehensively addressed the aspect of color.

With ColorwAI, we wish to disentangle coarse color directions, that allow
harmonious modifications based on a predominant guiding color. The choice of
general controls is aimed at preserving the creative aspect of the framework.

####  2.0.3. Disentanglement

Recent breakthroughs in deep neural networks for image generation have
significantly impacted the artistic and design industries (Kingma and Welling,
2013; Goodfellow et al., 2020; Rombach et al., 2022), enhancing image editing
and manipulation through disentanglement and text-conditioning techniques
(Gatys et al., 2016; Karras et al., 2019; Patashnik et al., 2021; Wang et al.,
2023). This study examines the creative and interpretative aspects of these
advancements.

Methodologies for determining and interpreting the latent factors of variation
have been developed starting from variations of VAE and GAN models in
supervised and unsupervised settings (Karras et al., 2019; Higgins et al.,
2016). Harkonen et al. (Härkönen et al., 2020), for instance, discover
interpretable image controls using Principal Component Analysis (PCA) on the
latent space of the model. Using supervision, Shen et al. (Shen et al., 2020)
and Yang et al. (Yang et al., 2021) train a machine learning regression to
find the separation boundary between different semantics in the latent space.
Methods such as Wu et al. (Wu et al., 2021) focus, instead, on the
identification of specific dimensions and channels that disentangle the
selected feature.

Recently, attention has been dedicated to disentanglement in Diffusion models.
This task is more complex than the disentanglement of earlier methods as
Diffusion models do not present a unique deterministic latent space. Some
works focus on the UNet bottleneck $h-space$ or latent representation
$\epsilon-space$ using deterministic variations of the scheduler and inversion
(Kwon et al., 2022; Zhu et al., 2024), while others disentangle the text
conditioning from CLIP (Wu et al., 2023).

##  3\. ColorwAI

This section introduces the ”generative colorways” framework, detailing the
StyleGAN and DDPM models, their latent space properties, and disentanglement
techniques. We describe methods for discovering latent traversal directions,
including InterfaceGAN, StyleSpace, and our modified ShapleyVec (Fig. 2).

Figure 2. Schematization of the ColorwAI framework. We show the
disentanglement steps for the two architectures, their latent spaces, and the
inference procedure. Steps 1 and 4 are different for GANs and Diffusion, while
2 and 3 are common.

Schematization of the ColorwAI framework. We show the disentanglement steps
for the two architectures, their latent spaces, and the inference procedure.
Steps 1 and 4 are different for GANs and Diffusion, while 2 and 3 are common.

###  3.1. Latent Spaces

####  3.1.1. StyleGAN

StyleGAN2-ADA is a style-based GAN generative neural network belonging to the
family of StyleGANs (Karras et al., 2019). The ADA version is more suitable
for training with smaller data than the original and versions 2 and 3 of
StyleGAN (Karras et al., 2021).

The generative neural network comprises two parts: a mapping $F$ and a
synthesis network $G$. The mapping $F:\mathcal{Z}\rightarrow\mathcal{W}$
receives as input a random code
$\mathbf{z}\in\mathcal{Z},\mathbf{z}\sim\mathcal{N}(0,1)$ and maps the vector
through a series of fully connected layers to a latent vector
$\mathbf{w}\in\mathcal{W}$. $\mathbf{w}$ is used by the synthesis network $G$
to generate the resulting image $\mathbf{x}=G(\mathbf{w})\in\mathcal{X}$. Both
$\mathcal{Z}$ and $\mathcal{W}$ are in $\mathcal{R}^{512}$.

The mapping network generates a space, $\mathcal{W}$, that is more
disentangled than the original latent space because it applies a non-linear
transformation that helps separate and refine the individual factors of
variation in the data. This transformation allows $\mathcal{W}$ to achieve a
more semantically meaningful representation where distinct features can be
independently controlled and manipulated.

####  3.1.2. Diffusion Denoising Probabilistic Models

In this section we outline the core principles of Diffusion models (Ho et al.,
2020), leaving in the appendix more details.

Diffusion models (Ho et al., 2020) generate data by learning how to reverse a
forward diffusion process that gradually adds noise to the data. Starting from
pure noise, the reverse process progressively refines the image to the desired
outcome.

The forward process adds Gaussian noise to data points at each step:

(1)
$q(\mathbf{x}t|\mathbf{x}{t-1})=\mathcal{N}(\mathbf{x}t;\sqrt{1-\beta_{t}}\mathbf{x}{t-1},\beta_{t}\mathbf{I})$

The reverse process, parameterized by a neural network, aims to denoise these
latent variables:

(2)
$p_{\theta}(\mathbf{x}{t-1}|\mathbf{x}t)=\mathcal{N}(\mathbf{x}{t-1};\mu\theta(\mathbf{x}t,t),\Sigma\theta(\mathbf{x}_{t},t))$

DiffusionCLIP integrates diffusion models with CLIP to guide the generation
process using text embeddings, by conditioning the reverse process (Kim et
al., 2022; Radford et al., 2021).

Asyrp propose to modify the generative reverse process to use the deepest
feature maps, the h-space, of a pre-trained diffusion model as semantic latent
space for image manipulation, using a modified deterministic DDIM denoising
schedule.

Boundary Diffusion segments the latent space into distinct regions for precise
attribute control without additional parameters (Zhu et al., 2023). This
method modifies the reverse process to include boundaries during the mixing
step, which is the denoising step where the final image starts to shape up.
This method finds the disentangled direction of a feature by determining its
semantic boundary using SVM. The latent direction is the orthonormal direction
exiting this boundary, similar to the InterfaceGAN method used for GANs (Shen
et al., 2020). This indicates that other GAN disentanglement methods, such as
StyleSpace and ShapleyVec, can also be applied within the Boundary Diffusion
framework.

###  3.2. Disentanglement Methods

Given a well-trained model, disentanglement methods assume that the generated
image $\mathbf{x}$ can be described by a set of semantics $\mathbf{s}$ and
that there is a mapping from the $G:\mathcal{X}\rightarrow{}\mathcal{S}$. In
particular, as we aim to generate colorways, we describe an image $\mathbf{x}$
by its predominant color, $\mathbf{s}$. Since there is a mapping between the
latent space $\mathcal{Z}$ and the image space $\mathcal{X}$, there is also a
mapping between the $\mathcal{Z}$ and the semantic space $\mathcal{S}$.

In disentanglement, research has empirically shown that given two semantics,
there is a hyperplane in the latent space that divides the two factors (Shen
et al., 2020). As such, there is a direction $\mathbf{n_{s}}$ in the latent
space that manipulates a certain semantic. Given that direction, an image can
be manipulated using
$\mathbf{z_{1}}\rightarrow\mathbf{z}+\alpha\mathbf{n_{s}}$ to obtain a new
image with that direction enhanced. In our case, if we generate an image
$\mathbf{x}$ using $\mathbf{z}$ and we wish to make the image bluer, we need
to find the direction that disentangles the factor blue $\mathbf{n_{blue}}$
and move positively towards that direction.

To find the traversal directions, the method of InterfaceGAN trains a linear
SVM to find the hyperplane that separates each semantic and uses the
orthonormal vector exiting the hyperplane as the disentangled direction
$\mathbf{n}$ (Shen et al., 2020). Conversely, StyleSpace proposes a method to
find the dimensions that deviate the most from the population mean, showing
that the variation only present in those dimensions changes the desired
semantic (Wu et al., 2021).

####  3.2.1. ShapleyVec

A problem with InterfaceGAN is that the direction $\mathbf{n}$ found in latent
space $\mathbf{z}$ utilizes all the dimensions of $\mathcal{Z}$ for the
manipulation. For an ideal disentanglement, it should be possible both to
manipulate the desired semantic without modifying any other aspect of the
output (disentanglement) and to represent the semantic into only one dimension
of the latent code (completeness). While the latter is often impossible,
finding a direction that uses all dimensions will likely adopt nodes that do
not encode that semantic, and therefore incur partial entanglements.
Differently, dimensions that exhibit the largest variance, like in StyleSpace,
may not take into account joint influences between dimensions and therefore do
not provide a comprehensive direction.

In ShapleyVec, we build upon these two methods, maintaining their vantage
points. First, we use a linear classifier like InterfaceGAN to find the
optimal separation boundary between the two classes (images with a given
primary color and images without that primary color). Second, we use Shapley
values222The method was coined by Lloyd Shapley in 1953 as a method for
collaborative game theory, that assigns payouts to players based on their
contribution to the total payout. The method is widely used in machine
learning interpretability, as it provides a quantitative and comparable value
for feature importance.  to analyze the contribution of each dimension to the
final prediction (Lundberg and Lee, 2017). Inspired by StyleSpace, we use
Shapley values to filter out the dimensions that do not contain enough
information about the desired feature. The filtering works as follows: we sort
the dimensions by their Shapley values in descending order and maintain the
first dimensions whose sum of the Shapley values reaches $0<E<1$. Shapley
values indicate the marginal contribution of each feature, with the total
contribution summing to one. Using only the filtered dimensions, we train
another linear classifier. We find the final direction $\mathbf{n}$ in a
manifold space of $\mathcal{Z}$, where the dimensions that have been filtered
out are masked using $0$. The orthonormal direction found using the second
classifier is used.

##  4\. Experimental Setup

####  4.0.1. Dataset and Processing

We train our models on the textile collection of the IMET dataset from the
Metropolitan Museum of Art, NY, including 20,000 textiles spanning from the
sixth century BC to the 21st century across various civilizations.
Additionally, we insert 10,000 textile samples from the Victoria and Albert
Museum collections. After filtering out black-and-white images, we used 12,918
images for training, 4,324 for validation, and 4,318 for testing. To
standardize the images, we remove backgrounds using the language-conditioned
Segment Anything Mask (Liu et al., 2023; Kirillov et al., 2023) with the text
’textile’, extracting the largest spanning square within the mask.

####  4.0.2. Color Annotation

To enable harmonious modifications based on a predominant guiding color, we
annotate the main color of each image. Given a latent code $\mathbf{z}$ that
generates an image $\mathbf{x}$, we annotate the color $\mathbf{s}$ of
$\mathbf{x}$. We rely on CIELab and HSV color encodings, for their alignment
with human vision and intuitive representation (Mojsilovic et al., 2000). We
extract the color palette from each image, select a suitable color codebook,
and quantize the main color in the palette. We empirically find that an
adaptation of Bhatia’s method (Bhatia et al., 2004) using CIELab encoding
provides the most pleasing and varied palette. We save the palettes and
perform adaptive clustering to derive a set of $19$ predominant colors,
forming the basis of the disentangled directions. Each image is assigned one
of the codebook colors based on HSV similarity weighted $80\%$ on Hue, $10\%$
on Saturation, and $10\%$ on Value, to ensure that the annotations respect the
original hue. For more information on the color annotation pipeline refer to
Appendix 2.

####  4.0.3. Disentanglement Evaluation

Inspired by the verification method developed by (Ramesh et al., 2022), we
evaluate the disentanglement of each of the color classes using a pseudo-
accuracy score. We compare the performance of ShapleyVec to InterfaceGAN and
StyleSpace, in StyleGAN and DDPM.

The verification score is not suitable for multi-class disentanglement as
there is no gradual increase to the desired semantic. Therefore, we first find
the ideal intensity of modification $\alpha_{optimal}$ that allows changing
the color of the image while maintaining most of its original structure. To do
so, we test the disentanglement of each color at different $\alpha$ and
register the SSIM (Structural Similarity Index Measure) to the original image.
We find the optimal alpha for each $\mathbf{n}$ as:

(3) $\alpha_{optimal}=\frac{1}{n}\sum_{i=1}^{N}\alpha*$

where $N$ is the number of samples and $\alpha*$ is:

(4) $\alpha*=max_{\alpha}[SSIM(\mathbf{z_{0}},\mathbf{z_{\alpha}})\geq
0.75\times SSIM(\mathbf{z_{0}},\mathbf{z_{0}})]$

$\mathbf{z_{0}}$ is the original image, and $\mathbf{z_{\alpha}}$ is the image
modified in a direction by a factor $\alpha$. We choose to use the SSIM as it
represents the perceived structural modification.

Once we find $\alpha_{optimal}$, we sample $m$ latent codes at random and
annotate their main color $\mathbf{s}$. If an image is already in a given
color, we remove it from the evaluation of that color. We then compute the
pseudo accuracy ($p-acc$) on the remaining images. We calculate the percentage
of images $\mathbf{x}$ generated by moving in the direction $\mathbf{n_{c}}$
of color $c$ by $\alpha_{optimal}$ whose main color is $c$. To take into
account that some colors are more similar to others in the color wheel, we
also propose a relaxed version of the accuracy ($relaxed-acc$), where the
modification is $1$ if the final hue sits within the range of the desired
color $c$, and $max(0,1-abs(d/100))$ where $d$ is the distance to the closest
border.

###  4.1. Experiments

To find the disentangled direction $\mathbf{n_{c}}$ for each color $c$ in
GANs, we annotate $1000$ latent codes $\mathbf{w}\in\mathcal{W}$ which we
sample randomly from $\mathcal{N}(0,1)$. We focus on the latent space of
StyleGAN2-ADA $\mathcal{W}$. For DDPM, we invert $1000$ random images, which
were previously annotated, using deterministic DDIM. We focus on the latent
spaces $\mathcal{Z_{t}}$, the UNet input space, and $\mathcal{h_{t}}$, the
UNet bottleneck layer, at mixing timestep $t=350$. We experiment with
$100<t<500$ and empirically find $350$ to be the best tradeoff between
adherence to the original pattern and color variation.

####  4.1.1. Quantitative Results

We test the performance of InterfaceGAN using several hyperparameters. We
train both Logistic Regression (LR) and Support Vector Machines (SVM), the
former with different regularization, $C\in[0.1,0.01,0.001]$. For StyleSpace
we test extracting $10$, $20$, or $40$ dimensions. Lastly, for ShapleyVec, we
assess if maintaining $0.25$ and $0.50$ of the explanation $E$ yields better
results. We evaluate all methods using both accuracy $p-acc$ and relaxed
accuracy $relaxed-acc$ sampling $100$ latent codes $\mathbf{w}$. We find that
generally, LR yields slightly superior results, with $C=0.1$, $40$ dimensions
for StyleSpace, and $E=0.5$ for ShapleyVec.

Model | Method | p-acc. (std) | relaxed-acc. (std)
---|---|---|---
StyleGAN | InterfaceGAN | 0.243 (0.14) | 0.86 (0.08)
StyleGAN | ShapleyVec |  0.287 (0.18) | 0.89 (0.09)
StyleGAN | StyleSpace | 0.20 (0.177) | 0.874 (0.11)
DDPM | InterfaceGAN | 0.253 (0.41) |  0.901 (0.15)
DDPM | ShapleyVec | 0.194 (0.36) | 0.883 (0.17)
DDPM | StyleSpace | 0.247 (0.43) | 0.89 (0.16)
Table 1. Results overview.

The results of the evaluation are shown in Table 1. We observe that the
standard deviations are generally high due to the great variability in
performance among colors, especially for DDPM. Moreover, we see a slight
superiority of both ShapleyVec and InterFaceGAN to StyleSpace on StyleGAN.
This is to be expected for GANs as StyleSpace is mostly intended to be used on
the higher dimensional $\mathcal{W}+$ or $\mathcal{S}$ space of StyleGAN.
Furthermore, ShapleyVec marginally surpasses InterFaceGAN in terms of accuracy
and relaxed evaluation in StyleGAN, and vice-versa for DDPM. Particularly,
ShapleyVec has the best overall p-accuracy, obtaining more precise
modifications, InterfaceGAN achieves the best relaxed accuracy, better
approximating a general color mood.

Figure 3. Colorways in three colors for three patterns using InterfaceGAN,
ShapleyVec, and StyleSpace on StyleGAN and DDPM. The examples are chosen
because of their representativity of general observed trends.

Colorways in three colors for three patterns using InterfaceGAN, ShapleyVec,
and StyleSpace on StyleGAN and DDPM. The examples are chosen because of their
representativity of general observed trends.

Some visual examples are shown in Figure 3. We observe an astonishing
difference between the type of color modification achieved by StyleGAN and by
DDPM. The first features more vivid shape and chromatic modifications. The
shift to a neighboring pattern, visible in the first and second patterns from
the top, appears seamless and visually pleasant, while the attempted
transition in the last image of DDPM towards yellow stripes manifests as
drastic and incomplete. Furthermore, DDPM maintains more precisely the shape
of the original pattern, inserting chromatic modifications in the details. In
the first example on the right, we interestingly see that DDPM substitutes the
contours with colored flowers in order to change the chromatic profile of the
image. This difference may be due to the different setting of the
disentanglement: while the modification in StyleGAN takes place prior to any
decoding, the modified latent space is inserted in Diffusion only at the
mixing step, where the skeleton of the image is already visible.

In terms of visual differences between the methods, the examples show that
InterfaceGAN is able to change the color with the smallest shape alterations.
In the context of StyleGAN, however, the tonality of the final image is most
faithfully achieved by ShapleyVec. For DDPM, we observe a subtle small
variation across methods.

####  4.1.2. Color Representation

An advantage of disentanglement is the creation of an explicit feature
representation within the latent space of a model for each semantic. We
leverage this property to explore the representation of each color and its
features. We also show the confusion matrix of the disentangled directions
following the quantitative evaluation. These experiments are displayed in
Figure 4. We show the results for ShapleyVec, as it is the only method with
meaningful representations in terms of vector similarities and that uses a
subset of neurons of the latent space. For clarity, we only show results for a
subset of color directions, which cover the spectrum of colors available,
sacrificing some shades of brown and blue.

Figure 4. Relationship between color representations for StyleGAN and DDPM
using ShapleyVec. The relationships are shown in terms of similarity between
the disentangled color vectors, in terms of which dimensions of the latent
space encode which color(s), and of how often the modifications using such
color directions result in other colors.

Relationship between color representations for StyleGAN and DDPM using
ShapleyVec. The relationships are shown in terms of similarity between the
disentangled color vectors, in terms of which dimensions of the latent space
encode which color(s), and of how often the modifications using such color
directions result in other colors.

The leftmost column pictures the mutual cosine similarities between the
feature representations. We observe that the representations within the $W$
space of StyleGAN broadly mirror our perceptual color similarity. In fact,
beiges appear close to yellows, in turn, similar to oranges and themselves to
browns. Similarly, blues and green are closely related. On the other side, the
similarities within DDPM’s $h-space$ are not naturalistic, with the largest
similarity appearing between Alpha Blue and Orange-Yellow tones.

The central column figures which neurons in the latent spaces $W$ and
$h-space$ are used in the color representations, and the overlap of the
neurons across all pairs of colors. Like DDPM’s vectorial representation, the
overlaps between colors are not naturalistic. Particularly, the number of
overlapping neurons in DDPM mirrors the vector similarities, due to the
sparsity of the vectors. In fact, DDPM’s $h-space$ has 18’000 neurons, out of
which ShapleyVec uses around 5000 to encode each color. In StyleGAN, we find
two colors, Fuji Snow, a white, and Firebird Tail Lights, an orange, that have
many common nodes to all other colors, as if they were more general ’color’
representations. In this context, the white could coincide with a black-and-
white direction and the orange with a saturation direction.

Lastly, the rightmost column shows the confusion matrices of the modifications
using StyleGAN and DDPM. While the confusion in StyleGAN is mostly with
neighboring colors in digital spaces, DDPM displays a clear preference towards
mutations towards certain colors.

Overall, it appears that the representations within StyleGAN are coherent with
human representations, due both to the lower dimensionality of the space
($512$ vs $18^{\prime}000$) and the uniqueness of the space as opposed to the
denoising timesteps and skip connections present in diffusion.

##  5\. Discussion on Computational Creativity

####  5.0.1. Evaluation

The premise of this framework is to create a stimulating interaction that
boosts creativity. It is subjective to assess whether the system provides
creative outputs and its influence on the user. While the quantitative
evaluation addresses the ’quality’ of how well the re-colorization is
achieved, as discussed by Anna Jordanous, SOTA comparisons should accompany
but not substitute qualitative evaluation (Jordanous, 2022). The field of
Computational Creativity equips us with tools to better frame this question
(Ritchie, 2019). Using Ritchie’s terminology, we can distinguish a creative
system in a loose sense, thus as carrying out activities considered by society
as generally ’creative’, and creativity in a strict sense, an attribute of a
specific agent, action, or outcome based on achievement of a certain quality.
The colorway creation, pertaining to the design sphere, can be coarsely
considered as $creative_{L}$, in the loose sense. What we wish to assess is
whether its outputs can be deemed $creative_{S}$, strict sense. To better
understand the creativity of the framework, we conducted an expert qualitative
evaluation of the framework and its results. This is referred to in the
literature as an external evaluation with a symptomatic approach (Ritchie,
2019). That is to say: we consider creativity in the eye of the beholder and
inquire into which symptoms of creativity an expert recognizes or not in the
system.

The questionnaire is framed within Boden’s creativity evaluation axes:
novelty, surprise, acceptability, and quality (Boden, 2004). We assess novelty
in P-creativity terms - novelty for the user in question; and surprise in
terms of combinatorial, exploratory, and transformational creativity, further
explained later (Boden, 2004; Ritchie, 2019). To assess the quality we focus
on the aesthetic value of the output and, lastly, we evaluate acceptability
with respect to how much the system can be used in practice and, therefore,
’accepted’ by the community.

The questionnaire was filled out by 4 design and/or textile experts and
contains 5 sections with open-ended questions accompanied by an indicative
5-point Likert scale question. The results indicate that designers find the
color variations mildly novel, with cases where the color is completely
flipped being considered the most innovative. The results were mostly not
indicated as surprising. The acceptability, on the other hand, was strongly
favored, with the major drawback being that the material creation process is
completely separated from the motif, unlike in real-world textile design. The
quality was also very positively evaluated, and the aesthetics of the color
modifications were mostly appreciated for the sharpness and contrast of the
result. The major drawback is the seldom loss of contrast between colors,
which makes the shapes less legible.

Interestingly, the evaluators all displayed a preference for contemporary-
looking designs, suggesting a preference for abstract shapes, and saturated
and contrasting colors. This highlights a major limitation of the current
approach, which is trained on historical textiles and uses nude tones as
disentangled directions. The perception of limited novelty and surprise is
also influenced by the historical style of the results.

####  5.0.2. Type of Creativity

Margaret Boden (Boden, 2010) identifies three forms of creativity:
combinatorial, exploratory, and transformational creativity. The first is the
ability to generate new elements through the combination of existing ones; the
second is the generation of new forms through the exploration of conceptual
spaces, while the last refers to an abstract transformation.

Disentangled semantic directions allow exploring the potential states of the
model, the colors that are ’plausible’ for certain textiles. We expect the
model to change a design to a color only to the extent that it makes sense
within its ’conceptual’ representation of the data. Following this reasoning,
the disentangled color directions, which we originally claimed to stimulate
creativity, would allow exploratory creativity, as they would enable using a
basic concept to explore modifications that go beyond that concept itself and
into the cultural framework of the model.

Within this type of creativity, historicity becomes a value, a framework aimed
at being surpassed. The framework allows generating characteristic examples
and mutating the colors according to what has been historically seen, whilst
allowing for variations. The addition of novel colors, shapes, and other forms
of art can further strengthen the ability to learn from the rule and go beyond
it. In this sense, following a direction of the latent space posits this type
of generation in an advanced stage of Dan Ventura’s Mere Generation steps
(Ventura, 2016). In fact, such a process can be considered a filtering step
using visual perception, leaving only the inception of a knowledge base as the
last missing step towards creativity and away from the highly critiqued ’mere
generation’.

##  6\. Conclusion

In conclusion, we have explored the task of ”generative colorway” creation
using disentanglement learning. We proposed ShapleyVec, which marginally
outperforms other disentanglement methods on StyleGAN and allows the
interpretation of the color representation in three ways. We find StyleGAN’s
$W$ space to be most aligned with human color perception and to yield the most
seamless modifications. We compare the results with disentanglement on DDPM,
establishing a shared framework, and identifying a key difference in the
editing. While StyleGAN changes ’semantically’ the output, DDPM
disentanglement based on the mixing step only allows to ’redirect’ the output
toward the new color. Finally, we propose that disentanglement can serve as a
method to leverage the exploratory creativity of latent spaces using Margaret
Boden’s terminology.

We acknowledge the limitations of this paper which open avenues for future
research. Further tests and evaluations should be carried out to be able to
assert the superiority ShapleyVec over InterfaceGAN. To enrich the comparison
between GAN and Diffusion, more disentanglement methods on DDPM not using the
mixing step, such as Asyrp, should be tested. Lastly, to enrich the novelty
and surprise of the output, evolutionary art methods can be included in future
work.

## References

  * (1)
  * Albers (2013) Josef Albers. 2013.  _Interaction of color_.  Yale University Press, .
  * Bengio (2013) Yoshua Bengio. 2013.  Deep learning of representations: Looking forward. In _International conference on statistical language and speech processing_. Springer, ., ., 1–37.
  * Bergmann et al. (2018) Paul Bergmann, Sindy Löwe, Michael Fauser, David Sattlegger, and Carsten Steger. 2018.  Improving unsupervised defect segmentation by applying structural similarity to autoencoders.  _arXiv preprint arXiv:1807.02011_ ., . (2018), .
  * Bhatia et al. (2004) Sanjiv K Bhatia et al. 2004\.  Adaptive K-Means Clustering.. In _FLAIRS_. ., ., 695–699.
  * Boden (2004) Margaret A Boden. 2004.  _The creative mind: Myths and mechanisms_.  Routledge, .
  * Boden (2010) Margaret A Boden. 2010.  _Creativity and art: Three roads to surprise_.  Oxford University Press, .
  * Briggs-Goode and Townsend (2011) Amanda Briggs-Goode and Katherine Townsend. 2011.  Textile design: principles, advances and applications.  _._ ., . (2011), .
  * Burchett (2002) Kenneth E Burchett. 2002.  Color harmony.  _Color Research & Application: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur_ 27, 1 (2002), 28–31.
  * Cho et al. (2017) Junho Cho, Sangdoo Yun, Kyoung Mu Lee, and Jin Young Choi. 2017.  Palettenet: Image recolorization with given color palette. In _Proceedings of the ieee conference on computer vision and pattern recognition workshops_. ., ., 62–70.
  * Choi et al. (2021) Jaewoong Choi, Junho Lee, Changyeon Yoon, Jung Ho Park, Geonho Hwang, and Myungjoo Kang. 2021.  Do not escape from the manifold: Discovering the local coordinates on the latent space of GANs.  _arXiv preprint arXiv:2106.06959_ ., . (2021), .
  * Dai et al. (2014) Dengxin Dai, Hayko Riemenschneider, and Luc Van Gool. 2014.  The synthesizability of texture examples. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. ., ., 3027–3034.
  * Dickinson (2011) K Dickinson. 2011.  The use of colour in textile design.  In _Textile Design_. Elsevier, ., 171–191e.
  * Gatys et al. (2016) Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016.  Image style transfer using convolutional neural networks. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. ., ., 2414–2423.
  * Goodfellow et al. (2020) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020.  Generative adversarial networks.  _Commun. ACM_ 63, 11 (2020), 139–144.
  * Härkönen et al. (2020) Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. 2020.  Ganspace: Discovering interpretable gan controls.  _Advances in neural information processing systems_ 33 (2020), 9841–9850.
  * Higgins et al. (2016) Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2016.  beta-vae: Learning basic visual concepts with a constrained variational framework. In _International conference on learning representations_. ., ., .
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising diffusion probabilistic models.  _Advances in neural information processing systems_ 33 (2020), 6840–6851.
  * Jones (2016) Owen Jones. 2016.  _The Grammar of Ornament: A Visual Reference of Form and Colour in Architecture and the Decorative Arts-The complete and unabridged full-color edition_.  Princeton University Press, .
  * Jordanous (2022) Anna Jordanous. 2022.  Should we pursue SOTA in Computational Creativity?  _._ ., . (2022), .
  * Karagoz et al. (2023) Halil Faruk Karagoz, Gulcin Baykal, Irem Arikan Eksi, and Gozde Unal. 2023.  Textile Pattern Generation Using Diffusion Models.  _arXiv preprint arXiv:2304.00520_ ., . (2023), .
  * Karras et al. (2021) Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2021.  Alias-free generative adversarial networks.  _Advances in Neural Information Processing Systems_ 34 (2021), 852–863.
  * Karras et al. (2019) Tero Karras, Samuli Laine, and Timo Aila. 2019.  A style-based generator architecture for generative adversarial networks. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. ., ., 4401–4410.
  * Kim et al. (2022) Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. 2022.  Diffusionclip: Text-guided diffusion models for robust image manipulation. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. ., ., 2426–2435.
  * Kingma and Welling (2013) Diederik P Kingma and Max Welling. 2013.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ ., . (2013), .
  * Kirillov et al. (2023) Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. 2023.  Segment Anything.  _arXiv:2304.02643_ ., . (2023), .
  * Kwon et al. (2022) Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. 2022.  Diffusion models already have a semantic latent space.  _arXiv preprint arXiv:2210.10960_ ., . (2022), .
  * Li et al. (2020) Bowen Li, Xiaojuan Qi, Thomas Lukasiewicz, and Philip HS Torr. 2020.  Manigan: Text-guided image manipulation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. ., ., 7880–7889.
  * Lin et al. (2023) Jue Lin, Zhiwei Xu, Gaurav Sharma, and Thrasyvoulos N Pappas. 2023.  Texture representation via analysis and synthesis with generative adversarial networks.  _e-Prime-Advances in Electrical Engineering, Electronics and Energy_ 6 (2023), 100286.
  * Liu et al. (2023) Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. 2023\.  Grounding dino: Marrying dino with grounded pre-training for open-set object detection.  _arXiv preprint arXiv:2303.05499_ ., . (2023), .
  * Locatello et al. (2020) Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. 2020.  Weakly-supervised disentanglement without compromises. In _International Conference on Machine Learning_. PMLR, ., ., 6348–6359.
  * Lundberg and Lee (2017) Scott M Lundberg and Su-In Lee. 2017.  A unified approach to interpreting model predictions.  _Advances in neural information processing systems_ 30, . (2017), .
  * Mojsilovic et al. (2000) Aleksandra Mojsilovic, Jelena Kovacevic, Jianying Hu, Robert J Safranek, and S Kicha Ganapathy. 2000.  Matching and retrieval based on the vocabulary and grammar of color patterns.  _IEEE transactions on image processing_ 9, 1 (2000), 38–54.
  * Pappas (1989) Thrasyvoulos N Pappas. 1989.  An Adaptive Clustering Algorithm for Image.  _._ ., . (1989), .
  * Patashnik et al. (2021) Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. 2021.  Styleclip: Text-driven manipulation of stylegan imagery. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. ., ., 2085–2094.
  * Portilla and Simoncelli (2000) Javier Portilla and Eero P Simoncelli. 2000.  A parametric texture model based on joint statistics of complex wavelet coefficients.  _International journal of computer vision_ 40 (2000), 49–70.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021\.  Learning transferable visual models from natural language supervision. In _International conference on machine learning_. PMLR, ., ., 8748–8763.
  * Ramesh et al. (2022) Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ 1, 2 (2022), 3.
  * Ritchie (2019) Graeme Ritchie. 2019.  The evaluation of creative systems.  _Computational creativity: The philosophy and engineering of autonomously creative systems_ ., . (2019), 159–194.
  * Rombach et al. (2022) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022.  High-resolution image synthesis with latent diffusion models. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. ., ., 10684–10695.
  * Shen et al. (2020) Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. 2020.  Interfacegan: Interpreting the disentangled face representation learned by gans.  _IEEE transactions on pattern analysis and machine intelligence_ 44, 4 (2020), 2004–2018.
  * Song et al. (2020) Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020.  Denoising diffusion implicit models.  _arXiv preprint arXiv:2010.02502_ ., . (2020), .
  * Ventura (2016) Dan Ventura. 2016.  Mere generation: Essential barometer or dated concept. In _Proceedings of the Seventh International Conference on Computational Creativity_. Sony CSL, ., ., 17–24.
  * Wang et al. (2020) Jing Wang, Jana Zujovic, June Choi, Basabdutta Chakraborty, Rene van Egmond, Huib de Ridder, and Thrasyvoulos N Pappas. 2020.  Influence of Texture Structure on the Perception of Color Composition.  _J. Percept. Imaging_ 3, 1 (2020), 10401–1.
  * Wang et al. (2023) Zhizhong Wang, Lei Zhao, and Wei Xing. 2023.  Stylediffusion: Controllable disentangled style transfer via diffusion models. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. ., ., 7677–7689.
  * Whiston (1948) P Whiston. 1948.  Design and Colour in textiles.  _Journal of the Textile Institute Proceedings_ 39, 6 (1948), P193–P199.
  * Wu et al. (2023) Qiucheng Wu, Yujian Liu, Handong Zhao, Ajinkya Kale, Trung Bui, Tong Yu, Zhe Lin, Yang Zhang, and Shiyu Chang. 2023.  Uncovering the disentanglement capability in text-to-image diffusion models. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. ., ., 1900–1910.
  * Wu et al. (2021) Zongze Wu, Dani Lischinski, and Eli Shechtman. 2021.  Stylespace analysis: Disentangled controls for stylegan image generation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. ., ., 12863–12872.
  * Xu et al. (2019) Chen Xu, Yu Han, George Baciu, and Min Li. 2019.  Fabric image recolorization based on intrinsic image decomposition.  _Textile Research Journal_ 89, 17 (2019), 3617–3631.
  * Yang et al. (2021) Ceyuan Yang, Yujun Shen, and Bolei Zhou. 2021.  Semantic hierarchy emerges in deep generative representations for scene synthesis.  _International Journal of Computer Vision_ 129 (2021), 1451–1466.
  * Zhu et al. (2023) Ye Zhu, Yu Wu, Zhiwei Deng, Olga Russakovsky, and Yan Yan. 2023.  Boundary Guided Learning-Free Semantic Control with Diffusion Models. In _Conference on Neural Information Processing Systems (NeurIPS)_ , Vol. . ., ., .
  * Zhu et al. (2024) Ye Zhu, Yu Wu, Zhiwei Deng, Olga Russakovsky, and Yan Yan. 2024.  Boundary guided learning-free semantic control with diffusion models.  _Advances in Neural Information Processing Systems_ 36 (2024), .

##  Appendix A Diffusion Denoising Probabilistic Models

Diffusion models (Ho et al., 2020) generate data by simulating a reverse
diffusion process. The core idea is to learn how to reverse a forward
diffusion process that gradually adds noise to the data. Starting from pure
noise, the reverse process progressively refines the image to the desired
outcome.

Let $\mathbf{x}_{0}$ be a data point from the true data distribution. The
forward process defines a sequence of latent variables
$\\{\mathbf{x}_{t}\\}_{t=0}^{T}$ by adding Gaussian noise at each step:

(5)
$q(\mathbf{x}_{t}|\mathbf{x}_{t-1})=\mathcal{N}(\mathbf{x}_{t};\sqrt{1-\beta_{t}}\mathbf{x}_{t-1},\beta_{t}\mathbf{I})$

where $\beta_{t}$ are the variance schedule parameters controlling the amount
of noise added at each step. The reverse process, parameterized by a neural
network, aims to denoise the latent variables step by step:

(6)
$p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t})=\mathcal{N}(\mathbf{x}_{t-1};\mu_{\theta}(\mathbf{x}_{t},t),\Sigma_{\theta}(\mathbf{x}_{t},t))$

The neural network $\mu_{\theta}$ is trained to approximate the true reverse
conditional distribution, allowing the model to progressively denoise from
$\mathbf{x}_{T}$ (pure noise) back to $\mathbf{x}_{0}$ (data sample).

Diffusion models are currently state-of-the-art in image generation but the
presence of a suitable semantic latent space to control the generation process
is still under discussion. They iteratively transform a random noise vector to
produce a high-resolution image, but there is no single semantic mapping
between the vector and the image’s features due to a lack of structured
hierarchy in the models, making it difficult to edit specific attributes of
the generated image.

DiffusionCLIP (Kim et al., 2022) integrates diffusion models with Contrastive
Language-Image Pre-training (CLIP) (Radford et al., 2021) to guide the
diffusion process, enabling the generation of images that are semantically
aligned with provided textual descriptions. Let $\mathbf{y}$ be the textual
description and $\mathbf{e}_{y}$ be the corresponding text embedding obtained
from CLIP. The diffusion model’s reverse process is then conditioned on this
text embedding $\mathbf{e}_{y}$:

(7)
$p_{\theta}(\mathbf{x}_{t-1}|\mathbf{x}_{t},\mathbf{e}_{y})=\mathcal{N}(\mathbf{x}_{t-1};\mu_{\theta}(\mathbf{x}_{t},t,\mathbf{e}_{y}),\Sigma_{\theta}(\mathbf{x}_{t},t,\mathbf{e}_{y}))$

Asyrp (Kwon et al., 2022) propose to modify the generative reverse process to
use the deepest feature maps, the h-space, of a pre-trained diffusion model as
semantic latent space for image manipulation. DDIM (Song et al., 2020)
redefines Eq. 1 as

(8)
$q_{\sigma}(x_{t-1}|x_{t},x_{0})=\mathcal{N}\left(\sqrt{\alpha_{t-1}}x_{0}+\sqrt{1-\alpha_{t-1}-\sigma_{t}^{2}}\frac{x_{t}-\sqrt{\alpha_{t}}x_{0}}{\sqrt{1-\alpha_{t}}},\sigma_{t}^{2}\mathbf{I}\right)$

which can be rewritten as:

(9) $x_{t-1}=\sqrt{\alpha_{t-1}}P_{t}(\epsilon_{\theta}(x_{t}|\Delta
h_{t}))+D_{t}+\sigma_{t}z_{t},$

where $P_{t}$ is the predicted image, $D_{t}$ is the direction pointing to
$x_{t}$, $\sigma_{t}$ is the variance of the reverse process, and
$z_{t}\sim\mathcal{N}(0,I)$ is a noise term. In Asyrp, only $P_{t}$ is
modified while $D_{t}$ remains unchanged. The shift $\Delta h_{t}$ is
optimized to achieve the desired $P_{t}$ using a directional CLIP loss:

(10)
$L_{\text{direction}}(x_{\text{edit}},y_{\text{target}};x_{\text{source}},y_{\text{source}})=1-\frac{\Delta
I\cdot\Delta T}{\|\Delta I\|\|\Delta T\|},$

where $\Delta T=E_{T}(y_{\text{target}})-E_{T}(y_{\text{source}})$ and $\Delta
I=E_{I}(x_{\text{edit}})-E_{I}(x_{\text{source}})$, with $E_{T}$ and $E_{I}$
being the CLIP text and image encoders, respectively.

Boundary Diffusion (Zhu et al., 2023) enables precise attribute control by
directly segmenting the latent space into distinct regions without adding any
extra parameters to the base diffusion model. This is in contrast to Asyrp,
which requires training auxiliary editing neural networks, and DiffusionCLIP,
which fine-tunes the pre-trained diffusion model. Boundary Diffusion
introduces a modified reverse process of eq. 9 to include boundaries in the
latent space:

(11) $x_{t-1}=\sqrt{\alpha_{t-1}}P_{t}(\epsilon_{\theta}(x_{t}|B_{t}(\Delta
h_{t})))+D_{t}+\sigma_{t}z_{t}$

where $\epsilon_{\theta}$ is the noise predictor conditioned on the modified
feature maps $h_{t}$, and $B_{t}(\Delta h_{t})$ represents the boundary
function applied to the shift in the latent variables. Furthermore, the
boundary is introduced in the latent space during the mixing step, the
denoising step at which the final image becomes clear.

BoundaryDiffusion finds the disentangled direction of a feature by obtaining
the semantic boundary of that feature using SVM. The latent direction is the
orthonormal direction exiting the semantic boundary. The method to find the
disentangled direction is equivalent to the InterfaceGAN proposed for GANs
(Shen et al., 2020). The applicability of this method on the latent space of
Diffusion indicates that, within the BoundaryDiffusion framework, also the
other GAN disentanglement methods, StyleSpace and ShapleyVec, can be applied.

##  Appendix B Color annotation

####  B.0.1. Color Annotation

As we wish to disentangle coarse color directions, that would allow harmonious
modifications based on a predominant guiding color, we annotate the
predominant color of each image. Given a latent code $\mathbf{z}$ that
generates an image $\mathbf{x}$, we annotate the main color $\mathbf{s}$ of
$\mathbf{x}$. Digitally, color can be represented in different encodings,
notably, RGB, HSV, CIELab, XYZ, each enforcing an inherent concept of
similarity and gamuts. Particularly, the encoding of CIELab is most aligned to
human vision, while HSV (hue, saturation, value), is the most understandable
and intuitive color representation (Mojsilovic et al., 2000).

We wish to annotate an image with the color that most completely represents
the perception it stimulates, while concurrently reducing the number of
possible color directions to a sufficiently varied but manageable set. We
tackle this in three steps: the color palette extraction of each image, the
selection of a suitable color codebook for quantization, and the final
quantization of the main color in the palette. The predominant color
annotation process can be found in Figure 5.

Several methods have been developed to extract palettes from an image, often
in the domains of image compression or retrieval. The most simple method
relies on k-means to identify a set of $K$ clusters from the $RGB$ or $Lab$
colors of an image and take the cluster centroids to be the palette. While
this method works well, it fails to extract contrasting colors, which stand
out the most to the human eye. Modifications have been proposed, including
adaptive clustering methods (Pappas, 1989; Bhatia et al., 2004), and codebook-
based quantization methods (Mojsilovic et al., 2000). We test the different
methods on our data and find empirically that an adaptation of (Bhatia et al.,
2004)’s method using $CIELab$ encoding results in the most pleasing and varied
color palette.

Successively, we save the extracted color palettes for all images in the
training set and reiterate the adaptive clustering to obtain a data-driven set
of predominant colors present in textiles. The resulting set of 19 colors
becomes the basis of the disentangled directions.

Lastly, to assign each image to one of the codebook colors, we retain the
first color of the palette and quantize it based predominantly on the Hue
similarity. In fact, as we wish to ensure that the annotations respect the hue
of the original image, and are only partly concerned with saturation and
brightness, we base encode the colors in $HSV$ and weigh each channel based on
empirical performance. We create a similarity metric using $80\%$ Hue, $10\%$
Saturation, and $10\%$ Value.

Figure 5. Predominant color annotation. From left to right: palette
extraction, codebook identification, color quantization. The final annotated
color is the lower box of each rightmost image.

Predominant color annotation. From left to right: palette extraction, codebook
identification, color quantization. The final annotated color is the lower box
of each rightmost image.


11institutetext: P. Rozmej 22institutetext: Faculty of Physics and Astronomy,
University of Zielona Góra, Szafrana 4a, 65-246 Zielona Góra, Poland
22email<EMAIL_ADDRESS>33institutetext: A. Karczewska
44institutetext: Faculty of Mathematics, Computer Science and Econometrics,
University of Zielona Góra, Szafrana 4a, 65-246 Zielona Góra, Poland
44email<EMAIL_ADDRESS>
# Comment on “Two-dimensional third- and fifth-order nonlinear evolution
equations for shallow water waves with surface tension” [Nonlinear Dyn,
doi:10.1007/s11071-017-3938-7]

Piotr Rozmej  Anna Karczewska

(Received: / Accepted: )

###### Abstract

The authors of the paper “Two-dimensional third- and fifth-order nonlinear
evolution equations for shallow water waves with surface tension” Fok  claim
that they derived the equation which generalizes the KdV equation to two space
dimensions both in first and second order in small parameters. Moreover, they
claim to obtain soliton solution to the derived first order (2+1)-dimensional
equation. The equation has been obtained by applying the perturbation method
burde  for small parameters of the same order. The results, if correct, would
be significant. In this comment, it is shown that the derivation presented in
Fok  is inconsistent because it violates fundamental properties of the
velocity potential. Therefore, the results, particularly the new evolution
equation and the dynamics that it describes, bear no relation to the problem
under consideration.

###### Keywords:

Shallow water waves Boussinesq system of equations soliton solutions

###### pacs:

02.30.Jr 05.45.-a 47.35.Bb

##  1 Outline of the metod

The authors of Fok  consider the model of inviscid and incompressible fluid
which motion is irrotational in a container with a flat bottom. In dimensional
variables, the set of hydrodynamical equations has the following form:

$\displaystyle\phi_{xx}+\phi_{yy}+\phi_{zz}$ $\displaystyle=$ $\displaystyle
0,\quad\mbox{in~{}the~{}whole~{}volume},$ (1)
$\displaystyle\phi_{z}-(\eta_{x}\phi_{x}+\eta_{y}\phi_{y}+\eta_{t})$
$\displaystyle=$ $\displaystyle 0,\quad\mbox{at~{}the~{}surface},$ (2)
$\displaystyle\phi_{t}+\frac{1}{2}(\phi_{x}^{2}+\phi_{y}^{2}+\phi_{z}^{2})+g\eta$
$\displaystyle=$ $\displaystyle 0,\quad\mbox{at~{}the~{}surface},$ (3)
$\displaystyle\phi_{z}$ $\displaystyle=$ $\displaystyle
0\;\quad\mbox{at~{}the~{}bottom}.$ (4)

Here $\phi(x,y,z,t)$ denotes the velocity potential, $\eta(x,y,t)$ denotes the
surface function and $g$ is the gravitational acceleration. Indexes denote
partial derivatives, i.e. $\phi_{xx}\equiv\frac{\partial^{2}\phi}{\partial
x^{2}}$, and so on. The authors take into account surface tension terms, as
well. In this note, we neglect these terms since their presence or absence
does not change the source of errors made in Fok .

Next, the authors introduce a standard scaling to dimensionless variables
(different in $x$\- and $y$-direction)

$\displaystyle\tilde{x}=$ $\displaystyle
x/L,\quad\tilde{y}=y/y_{0},\quad\tilde{z}=z/h_{0},\quad\tilde{t}=t/t_{0},$ (5)
$\displaystyle\quad\tilde{\eta}=$
$\displaystyle\eta/A,\quad\tilde{\phi}=\phi/(L\frac{A}{h_{0}}\sqrt{gh_{0}}),$
(6)

where $A$ is the amplitude of surface distortions from equilibrium shape (flat
surface), $h_{0}$ is average fluid depth, $L$ is the average wavelength (in
$x$-direction), and $y_{0}$ is a wavelength in $y$-direction. In general,
$y_{0}$ should be of the same order as $L$, but not necessarily equal.
Notation $t_{0}=L/\sqrt{gh_{0}}$ is not explained in the paper.

Then the set (1)-(4) takes in scaled variables the following form (tildas are
now dropped):

$\displaystyle\beta\phi_{xx}+\gamma\phi_{yy}+\phi_{zz}$ $\displaystyle=$
$\displaystyle 0,$ (7)
$\displaystyle\eta_{t}+\alpha(\eta_{x}\phi_{x}+\frac{\gamma}{\beta}\eta_{y}\phi_{y})-\frac{1}{\beta}\phi_{z}$
$\displaystyle=$ $\displaystyle 0,~{}\mbox{for}~{}z=1+\alpha\eta,$ (8)
$\displaystyle\phi_{t}+\frac{1}{2}\alpha\left(\phi_{x}^{2}+\frac{\gamma}{\beta}\phi_{y}^{2}+\frac{1}{\beta}\phi_{z}^{2}\right)+\eta$
$\displaystyle=$ $\displaystyle 0,~{}\mbox{for}~{}z=1+\alpha\eta,$ (9)
$\displaystyle\phi_{z}$ $\displaystyle=$ $\displaystyle
0\;\,~{}\mbox{for}~{}z=0.$ (10)

As usual, small parameters are defined as follows:

$\alpha=A/h_{0},\quad\beta=(h_{0}/l)^{2}\quad\mbox{and}\quad\gamma=(h_{0}/y_{0})^{2}.$

##  2 Details of calculations in Fok  limited to first order

Next, the authors assume $\gamma=\beta$ and write erroneous formula (Fok, ,
Eq. (11)) for the velocity potential

$\phi=\sum_{m=0}^{N}\sum_{n=0}^{N}\frac{(-1)^{m+n}\beta^{m+n}z^{2(m+n)}}{(2m)!(2n)!}\frac{\partial^{2(m+n)}}{\partial
x^{2m}\partial y^{2n}}f(x,y,t).$ (11)

In the following, the authors limit their considerations to the Boussinesq
equations up to second order in small parameters. Then it is enough to use the
explicit form of the potential (11) up to third order (due to terms
$\frac{1}{\beta}\phi_{z}$ in (8) and $\frac{1}{\beta}\phi_{z}^{2}$ in (9) the
velocity potential should be valid up to one order higher than the Boussinesq
equations)

$\displaystyle\phi=f$
$\displaystyle-\frac{\beta\,z^{2}}{2}\left(f_{xx}+f_{yy}\right)+\frac{\beta^{2}z^{4}}{24}\left(f_{xxxx}+6f_{xxyy}+f_{yyyy}\right)$
$\displaystyle-\frac{\beta^{3}z^{6}}{720}\left(f_{6x}+15f_{4x2y}+15f_{2x4y}+f_{6y}\right).$
(12)

_The fact that formulas ( 11)-(2) are wrong is easy to check by a direct
substitution to the Laplace equation (7)._ With the above form of the velocity
potential, the authors obtained the set of Boussinesq’s equations in the form
(here surface tension is neglected and only terms of first order are retained)

$\displaystyle\eta_{t}+f_{xx}+f_{yy}$
$\displaystyle+\alpha\left(f_{x}\eta_{x}+f_{y}\eta_{y}+\eta(f_{2x}+f_{2y})\right)$
$\displaystyle-\frac{1}{6}\beta\left(f_{xxxx}+6f_{xxyy}+f_{yyyy}\right)=0,$
(13) $\displaystyle\eta_{x}+f_{xt}$
$\displaystyle+\alpha\left(f_{x}f_{xx}+f_{y}f_{xy}\right)-\frac{1}{2}\beta\left(f_{xyyt}+f_{xxxt}\right)=0.$
(14)

Next, the authors insert the velocity potential into equations (8) and (9)
retaining terms up to second order in $\alpha,\varepsilon=\beta$. They
introduce the following notation

$u=f_{x},\qquad v=f_{y}.$ (15)

With the above notation Boussinesq’s equations (2)-(14) can be formulated as
(Fok, , Eq. (12))-(Fok, , Eq. (13)), that is as

$\displaystyle\eta_{t}+u_{x}+v_{y}$
$\displaystyle+\alpha\left(u\eta_{x}+v\eta_{y}+\eta(u_{x}+v_{y})\right)$ (16)
$\displaystyle-\frac{1}{6}\varepsilon\left(u_{xxx}+3u_{xyy}+3v_{xxy}+v_{yyy}\right)=0,$
$\displaystyle\eta_{x}+u_{t}$
$\displaystyle+\alpha\left(uu_{x}+vv_{x}\right)-\frac{1}{2}\varepsilon\left(v_{xyt}+u_{xxt}\right)=0.$
(17)

In the following, the authors apply the perturbative approach described in
detail by Burde and Sergyeyev burde  and next extended in KRcnsns  to more
complicated cases. In this method, one begins from zeroth-order solutions,
then uses their properties in the calculation of corrections of the first
order, and so on. In zeroth order eqs. (Fok, , Eq. (12))-(Fok, , Eq. (13))
reduce to

$\eta_{t}+u_{x}+v_{y}=0,\qquad\mbox{and}\qquad\eta_{x}+u_{t}=0,$ (18)

which have solutions $u=\eta$, $v=a(x,t)$ with
$\eta_{t}=-\eta_{x},~{}u_{t}=-u_{x}$. The authors of Fok  take constant $v=a$,
which is a particular solution to $v_{y}=0$.

Looking for the solution in first-order approximation, one introduces
corrections of the first order to the equations (18) and requires that the
Boussinesq’s equations become compatible in this order (what means that these
two equations become equivalent). The authors look for first-order solutions
in the form

$\displaystyle u$ $\displaystyle=$
$\displaystyle\eta+\alpha\,B(x,y,t)+\varepsilon\,C(x,y,t),$ (19)
$\displaystyle v$ $\displaystyle=$ $\displaystyle a\qquad\mbox{(constant )}.$
(20)

Inserting first order corrections (19)-(20) into (16)-(17), retaining only
terms up to first order, and using properties $B_{t}=-B_{x},~{}C_{t}=-C_{x}$
the authors obtained Here, we cite the formulas from Fok . More detailed
derivation, with the correct velocity potential formula, is presented in
section 3. These equations, after integration give the formulas (Fok, , Eq.
(23))-(Fok, , Eq. (24)). These formulas read as

$B=-\frac{1}{4}\eta^{2}-\frac{1}{2}a\int\eta_{y}\,dy,\qquad
C=\frac{1}{4}\eta_{yy}+\frac{1}{3}\eta_{xx}.$ (21)

So, in first order approximation $u$ becomes (in (Fok, , Eq. (25)) terms at
$\alpha$ and $\varepsilon$ are incorrectly positioned)

$u=\eta+\alpha\left(-\frac{1}{4}\eta^{2}-\frac{1}{2}a\int\eta_{y}\,dy\right)+\varepsilon\left(\frac{1}{4}\eta_{yy}+\frac{1}{3}\eta_{xx}\right).$

Insertion of $u$, given by the above expression into (Fok, , Eq. (12)) and
(Fok, , Eq. (13)) leads in both cases to the same equation (Fok, , Eq. (26)),
that is

$\eta_{t}+\eta_{x}+\alpha\left(\frac{3}{2}\eta\eta_{x}+\frac{1}{2}a\eta_{y}\right)+\varepsilon\left(\frac{1}{6}\eta_{xxx}-\frac{1}{4}\eta_{xyy}\right)=0.$
(22)

In other words, equation (Fok, , Eq. (25)) supply compatibility of
Boussinesq’s equations and therefore may supply the correct (2+1)-dimensional
evolution equation in the first order. On this basis, the authors extend the
derivations to second order in small parameters and claim to obtain the
(2+1)-dimensional fifth-order evolution equation (Fok, , Eq. (32)). They also
claim to find the analytic solution to the equation (Fok, , Eq. (26)) and
discuss its properties.

We do not comment here the derivation of second-order equation nor the
authors’ solutions to the first order equation because already first order
wave equation is incorrect. The details of the argumentation that the equation
(22), that is, (Fok, , Eq. (26)), is wrong are presented in section 4.

##  3 Details of correct calculations in first order perturbation approach

The correct formula for the velocity potential fulfilling (1) has the
following form

$\displaystyle\phi(x,z,t)$
$\displaystyle=\sum_{m=0}^{\infty}\frac{(-1)^{m}}{(2m)!}z^{2m}\,(\beta\partial_{xx}+\gamma\partial_{yy})^{m}f(x,y,t).$
(23)

For $\gamma=\beta$, the explicit form of this velocity potential up to third
order in small parameters reads as

$\displaystyle\phi=f$
$\displaystyle-\frac{\beta\,z^{2}}{2}(f_{xx}+f_{xy})+\frac{\beta^{2}z^{4}}{24}(f_{xxxx}+2f_{xxyy}+f_{yyyy})$
$\displaystyle-\frac{\beta^{3}z^{6}}{720}(f_{6x}+3f_{4x2y}+3f_{2x4y}+f_{6y}).$
(24)

The correct formula for the velocity potential (23) and (3) differs from that
used by the authors ((11) and (2)) by values of coefficients in front of mixed
$xy$-derivatives.

With the correct formula (3), one obtains the following Boussinesq’s
equations. From (8), limiting to first order, one gets

$\displaystyle\eta_{t}+f_{xx}+f_{yy}$
$\displaystyle+\alpha\left(f_{x}\eta_{x}+f_{y}\eta_{y}+\eta(f_{2x}+f_{2y})\right)$
$\displaystyle-\frac{1}{6}\epsilon\left(f_{xxxx}+2f_{xxyy}+f_{yyyy}\right)=0.$
(25)

The correct result from (9) in first order (after differentiation over $x$) is

$\displaystyle\eta_{x}+f_{xt}$
$\displaystyle+\alpha\left(f_{y}f_{xy}+f_{x}f_{2x}\right)-\frac{1}{2}\epsilon\left(f_{xyyt}+f_{xxxt}\right)=0.$
(26)

In variables $u,v$ (15), equations (3)-(26) become

$\displaystyle\eta_{t}+u_{x}+v_{y}$
$\displaystyle+\alpha\left(\eta_{x}u+\eta_{y}v+\eta(u_{x}+v_{y})\right)$ (27)
$\displaystyle-\frac{1}{6}\epsilon\left(u_{xxx}+2u_{xyy}+v_{yyy}\right)=0,$
$\displaystyle\eta_{x}+u_{t}$
$\displaystyle+\alpha\left(uu_{x}+vv_{x}\right)-\frac{1}{2}\epsilon\left(u_{yyt}+u_{xxt}\right)=0.$
(28)

Equations (3) and consequently (27) differ from equations (2) and (14)
obtained from incorrect velocity potential (11) by the factor in front of the
term $f_{xxyy}$.

Let us follow the authors’ approach with the correct equations (27)-(28). In
zeroth order we have (18). In first order assume the correction functions in
the same form (19)-(20).

Substitution to (27) and limitation to first order terms gives

$\eta_{t}+\eta_{x}+\alpha\left(a\,\eta_{y}+B_{x}+2\eta\eta_{x}\right)+\varepsilon\\!\left(\\!C_{x}-\frac{1}{6}\eta_{3x}-\frac{1}{3}\eta_{x2y}\\!\right)=0.$
(29)

Substitution to (28) and retention of terms up to first order gives

$\eta_{t}+\eta_{x}+\alpha\left(B_{t}+\eta\eta_{x}\right)+\varepsilon\left(C_{t}-\frac{1}{2}\eta_{2yt}-\frac{1}{2}\eta_{2xt}\right)=0.$
(30)

Substraction of (30) from (29) gives

$\displaystyle\alpha$ $\displaystyle(B_{x}-B_{t}+\eta\eta_{x}+a\eta_{y})$ (31)
$\displaystyle+\varepsilon$
$\displaystyle(C_{x}-C_{t}-\frac{1}{6}\eta_{3x}-\frac{1}{3}\eta_{x2y}+\frac{1}{2}\eta_{2yt}+\frac{1}{2}\eta_{2xt})=0.$

In (31), we use the properties
$B_{t}=-B_{x},~{}C_{t}=-C_{x},~{}\eta_{2yt}=-\eta_{2yx},~{}\eta_{2xt}=-\eta_{3x}$
valid in zeroth order. Since expressions in (31) are already in first order it
is sufficient. Recall that the general form, e.g., $B_{t}=-B_{x}+\alpha
Ba+\varepsilon Be$, and so on, do not change the further results since after
insertion into (31) second order terms have to be rejected. These properties
and freedom of $\alpha,\varepsilon$ allow us to obtain simple differential
equations for $B$ and $C$ in the form

$2B_{x}+\eta\eta_{x}+a\eta_{y}=0,\quad
2C_{x}-\frac{2}{3}\eta_{3x}-\frac{5}{6}\eta_{xyy}=0.$

Integrating above equations one obtains

$B=-\frac{1}{4}\eta^{2}-\frac{1}{2}a\\!\int\\!\eta_{y}\,dx,\quad\mbox{and}\quad
C=\frac{1}{3}\eta_{xx}+\frac{5}{12}\eta_{yy}.$ (32)

Then, the function $u$ becomes

$u=\eta+\alpha\\!\left(\\!-\frac{1}{4}\eta^{2}-\frac{1}{2}a\\!\int\\!\eta_{y}\,dx\\!\right)+\varepsilon\left(\\!\frac{1}{3}\eta_{xx}+\frac{5}{12}\eta_{yy}\\!\right).$
(33)

With $u$ in the form (33), both Boussinesq’s equations should reduce to the
same wave equation. Indeed, this is the case, and the final first order wave
equation receives the following form

$\eta_{t}+\eta_{x}+\alpha\left(\frac{3}{2}\eta\eta_{x}+\frac{1}{2}a\,\eta_{y}\right)+\varepsilon\left(\frac{1}{6}\eta_{xxx}+\frac{1}{12}\eta_{xyy}\right)=0.$
(34)

Although the steps from assumed form of first order functions (19)-(20) to
equations (22) (Fok, , Eq. (26)) or (34) are mathematically correct, the final
equations are incorrect because the assumptions (19)-(20) violate the
properties of the velocity potential. Deatils ere explained in the next
section.

##  4 Critics

The authors treat $u,v$ as independent functions, ignoring that, in fact,
$u,v$ are partial derivatives of the same function $f$. Since $u=f_{x}$ and
$v=f_{y}$, and these functions and their partial derivatives should be
continuous, then the fundamental condition has to be fulfilled

$u_{y}=v_{x}\equiv f_{xy}.$ (35)

This condition is not fulfilled by the functions $u,v$ in equations (19)-(20)
which are necessary for the authors to derive new (2+1)-dimensional equation.
The requirement that condition (35) is fulfilled by (19)-(20) leads from one
side to

$f_{xy}\equiv u_{y}=\eta_{y}+\alpha B_{y}+\varepsilon C_{y}$

and from the other side to

$f_{xy}\equiv v_{x}=0.$

So, the condition $u_{y}=v_{x}$ (35) can be fulfilled only when
$\eta_{y}+\alpha B_{y}+\varepsilon C_{y}=0$ which, due to freedom of
$\alpha,\varepsilon$, imply $B_{y}=C_{y}=\eta_{y}=0$ ! This means that the
surface profile does not dependent on $y$ (there is translation symmetry in
$y$-coordinate). In other words, searching for the solution in the form
(19)-(20) reduces the problem to be only (1+1)-dimensional (with $\eta_{y}=0$
the final equation reduces to the usual KdV equation). Therefore, the derived
first order (2+1)-dimensional evolution equation cannot be the solution to the
Boussinesq set (2)-(14). In the same way the equation (34) cannot be the
solution to the correct Boussinesq’s set (3)-(26).

Can we extend the form of corrections (19)-(20) in such a way that they
fulfill the condition (35)? We assume the most general form of first order
functions $u,v$, that is

$\displaystyle u$ $\displaystyle=$
$\displaystyle\eta+\alpha\,B(x,y,t)+\varepsilon\,C(x,y,t),$ (36)
$\displaystyle v$ $\displaystyle=$ $\displaystyle
a(x,t)+\alpha\,R(x,y,t)+\varepsilon\,S(x,y,t).$ (37)

Inserting (36)-(37) into (27)-(28) and neglecting terms of second order one
obtains

$\displaystyle\eta_{t}+\eta_{x}$
$\displaystyle+\alpha\left(a\,\eta_{y}+B_{x}+2\eta\eta_{x}+R_{y}\right)$ (38)
$\displaystyle+\varepsilon\left(C_{x}-\frac{1}{6}\eta_{xxx}-\frac{1}{6}\eta_{xyy}+S_{y}\right)=0$
and $\displaystyle\eta_{t}+\eta_{x}$
$\displaystyle+\alpha\left(B_{t}+\eta\eta_{x}+a\,a_{x}\right)+\varepsilon\left(C_{t}-\frac{1}{2}\eta_{yyt}-\frac{1}{2}\eta_{xxt}\right)=0.$
(39)

Substraction of (39) from (38) and use the properties
$B_{t}=-B_{x},~{}~{}C_{t}=-C_{x}$ allows us to obtain two equations

$\displaystyle 2B_{x}$
$\displaystyle+\eta\,\eta_{x}-a\,a_{x}+a\,\eta_{y}+R_{y}=0,\quad\mbox{and}$
(40) $\displaystyle 2C_{x}$
$\displaystyle-\frac{2}{3}\left(\eta_{xxx}+\eta_{xyy}\right)+S_{y}=0.$ (41)

Integration over $x$ gives

$\displaystyle B$
$\displaystyle=-\frac{1}{4}\eta^{2}+\frac{1}{4}a^{2}-\frac{1}{2}a\\!\int\\!\eta_{y}\,dx-\frac{1}{2}\\!\int\\!R_{y}\,dx,$
(42) $\displaystyle C$
$\displaystyle=\frac{1}{3}(\eta_{xx}+\eta_{yy})-\frac{1}{2}\\!\int\\!S_{y}\,dx.$

So, in principle, the functions

$\displaystyle u$
$\displaystyle=\eta+\alpha\left(-\frac{1}{4}\eta^{2}+\frac{1}{4}a^{2}-\frac{1}{2}a\\!\int\\!\eta_{y}\,dx-\frac{1}{2}\\!\int\\!R_{y}\,dx\right)$
$\displaystyle\hskip
19.37494pt+\varepsilon\left(\frac{1}{3}(\eta_{xx}+\eta_{yy})-\frac{1}{2}\\!\int\\!S_{y}\,dx\right),\quad\mbox{and}$
(43) $\displaystyle v$
$\displaystyle=a(x,t)+\alpha\,R(x,y,t)+\varepsilon\,S(x,y,t),$ (44)

can make Boussinesq’s equations compatible. But, is it possibile to find such
functions $a,R,S$ which ensure fulfilling the condition (35), that is ensure
$u_{y}=v_{x}$? Because

$\displaystyle u_{y}$
$\displaystyle=\eta_{y}+\alpha\left(-\frac{1}{2}\eta\eta_{y}-\frac{1}{2}a\\!\int\\!\eta_{yy}\,dx-\frac{1}{2}\\!\int\\!R_{yy}\,dx\right)$
$\displaystyle\hskip
19.37494pt+\varepsilon\left(\frac{1}{3}(\eta_{xxy}+\eta_{yyy})-\frac{1}{2}\\!\int\\!S_{yy}\,dx\right)\hskip
21.52771pt$ (45) $\displaystyle v_{x}$
$\displaystyle=a_{x}+\alpha\,R_{x}+\varepsilon\,S_{x},$ (46)

this task seem to be hopeless.

##  5 Another possibile approach

The above considerations imply that in a shallow water problem, it is
practically impossible to obtain (2+1)-dimensional evolution equation for the
profile of surface wave $\eta(x,y,t)$, even in first order approximation.
However, this model can supply differential equations of any order for the
function $f(x,y,t)\equiv\phi^{(0)}(x,y,t)$. Below, we present such an equation
in first order.

Recall, that from (8) and the velocity potential (3) one obtains the first
Boussinesq’s equation in the form (3). The corresponding Boussinesq’s equation
resulting from (9) is

$\eta+f_{t}+\frac{1}{2}\alpha\left(f_{x}^{2}+f_{y}^{2}\right)-\frac{1}{2}\varepsilon\left(f_{xxt}+f_{yyt}\right)=0.$
(47)

Now, inserting
$\eta=-\left(f_{t}+\frac{1}{2}\alpha\left(f_{x}^{2}+f_{y}^{2}\right)-\frac{1}{2}\varepsilon\left(f_{xxt}+f_{yyt}\right)\right)$
from (47) into (3) and neglecting second order terms one obtains first order
evolution equation for the function $f(x,y,t)$

$\displaystyle f_{xx}$
$\displaystyle+f_{yy}-f_{tt}+\alpha\left[-f_{t}\left(f_{xx}+f_{yy}\right)-2\left(f_{x}f_{xt}+f_{y}f_{yt}\right)\right]$
(48)
$\displaystyle+\varepsilon\left[\frac{1}{2}\left(f_{xxtt}+f_{yytt}\right)-\frac{1}{6}\left(f_{xxxx}+2f_{xxyy}+f_{yyyy}\right)\right]=0.$

It can be further simplified utilizing zeroth-order solution
$f_{tt}=f_{xx}+f_{yy}$. Since the term
$\varepsilon\frac{1}{2}\left(f_{xxtt}+f_{yytt}\right)$ is already first order,
we can use the equalities

$f_{xxtt}=(f_{tt})_{xx}=(f_{xx}+f_{yy})_{xx}=f_{xxxx}+f_{xxyy},$
$f_{yytt}=(f_{tt})_{yy}=(f_{xx}+f_{yy})_{yy}=f_{xxyy}+f_{yyyy}$

to replace it by
$\varepsilon\frac{1}{2}\left(f_{xxxx}+2f_{xxyy}+f_{yyyy}\right).$ Therefore an
equivalent, simpler form of (48), still valid up to first order, is

$\displaystyle f_{xx}$
$\displaystyle+f_{yy}-f_{tt}+\alpha\left[-f_{t}\left(f_{xx}+f_{yy}\right)-2\left(f_{x}f_{xt}+f_{y}f_{yt}\right)\right]$
(49)
$\displaystyle+\varepsilon\left(\frac{1}{3}\left(f_{xxxx}+2f_{xxyy}+f_{yyyy}\right)\right)=0.$

If the solution to (48) or (49) is known, the equation (47) supplies the
surface profile function $\eta(x,y,t)$.

In general, for a flat bottom, one can extend this procedure to arbitrary
order. However, even in first-order approximation partial differential
equation (49) corresponding to (2+1)-dimensional shallow water problem is
highly complicated. There is little hope to find either a solution to (49) or
a (2+1)-dimensional wave equation for wave profile $\eta(x,y,t)$ without
further significant simplifications.

##  6 Conclusions

We have proved that the (2+1)-dimensional KdV-type equation (Fok, , Eq. (26))
has been inconsistently obtained by the authors and therefore cannot describe
(2+1)-dimensional surface waves. Moreover, we have shown that when assumptions
for first order functions (Fok, , Eqs. (14)-(15)) are used consistently with
the properties of the velocity potential, then the solution reduces to the
usual KdV equation. Additionally, we have demonstrated that even a consistent
extension of the authors’ method Fok  gives no hope for obtaining appropriate
first order (2+1)-dimensional evolution equation for shallow water problem.

## References

  * (1) Fokou, M., Kofane, T.C., Mohamadou, A., Yomba, E.: Two- dimensional third-and fifth-order nonlinear evolution equations for shallow water waves with surface tension. Nonlinear Dynamics 91, 1177–1189 (2018)
  * (2) Burde, G.I., Sergyeyev, A.: Ordering of two small parameters in the shallow water wave problem. J Phys. A Math Theor. 46, 075501 (2013)
  * (3) Karczewska, A., Rozmej, P.: Can simple KdV-type equations be derived for shallow water problem with bottom bathymetry? Commun Nonlinear Sci Numer Simulat 82 105073 (2020)
  * (4) Rozmej, P., Karczewska, A.: Comment on "Two-dimensional third- and fifth- order nonlinear evolution equations for shallow water waves with surface tension" [Nonlinear Dyn, doi:10.1007/s11071-017-3938-7] https://arxiv.org/abs/2105.08519 (2021). [Preprint]