# Intelli-Paint: Towards Developing Human-like Painting Agents

Jaskirat Singh1,2, Cameron Smith1, Jose Echevarria1, Liang Zheng2,
1Adobe Research, 2Australian National University
<EMAIL_ADDRESS>{casmith<EMAIL_ADDRESS>
###### Abstract

The generation of well-designed artwork is often quite time-consuming and
assumes a high degree of proficiency on part of the human painter. In order to
facilitate the human painting process, substantial research efforts have been
made on teaching machines how to “paint like a human”, and then using the
trained agent as a painting assistant tool for human users. However, current
research in this direction is often reliant on a progressive grid-based
division strategy wherein the agent divides the overall image into
successively finer grids, and then proceeds to paint each of them in parallel.
This inevitably leads to artificial painting sequences which are not easily
intelligible to human users. To address this, we propose a novel painting
approach which learns to generate output canvases while exhibiting a more
human-like painting style. The proposed painting pipeline Intelli-Paint
consists of 1) a progressive layering strategy which allows the agent to first
paint a natural background scene representation before adding in each of the
foreground objects in a progressive fashion. 2) We also introduce a novel
sequential brushstroke guidance strategy which helps the painting agent to
shift its attention between different image regions in a semantic-aware
manner. 3) Finally, we propose a brushstroke regularization strategy which
allows for $\sim$60-80% reduction in the total number of required brushstrokes
without any perceivable differences in the quality of the generated canvases.
Through both quantitative and qualitative results, we show that the resulting
agents not only show enhanced efficiency in output canvas generation but also
exhibit a more natural-looking painting style which would better assist human
users express their ideas through digital artwork.

Figure 1: Developing human-like painting style. _(Left)_ Painting sequence
visualization which demonstrates that our method exhibits significantly higher
resemblance with the human painting style as opposed to previous state of the
art. _(Right)_ This resemblance is achieved through 1) a progressive layering
strategy which allows for a more human-like evolution of the canvas, 2) a
sequential attention mechanism which focuses on different image regions in a
coarse-to-fine fashion and 3) a brushstroke regularization formulation which
allows our method to obtain detailed results while using significantly fewer
brushstrokes ($\sim$1/20 as compared to Paint Transformer [21] in above).

##  1 Introduction

Paintings form a key medium through which humans express their ideas and
emotions. Nevertheless, the ability with which such ideas can be expressed is
often limited by the prowess of the human painter. Moreover, the creation of
finer-quality art is often quite challenging and requires a considerable
amount of time on part of the human painter.

One of the ways to address this problem is to develop autonomous painting
agents which can assist human painters to better express their ideas in a
quick and concise fashion. To this end, there is a growing research interest
[12, 22, 6, 35, 16, 31, 8, 36, 21, 26, 23, 3, 4] in teaching machines _“how to
paint”_ , in a manner similar to a human painter. For instance, Huang _et al_.
[12] use deep reinforcement learning in order to learn an unsupervised
brushstroke decomposition for the creation of non-photorealistic imagery. Zou
_et al_. [36] use gradient descent in order to optimize over the brushstrokes
parameters for the entire painting trajectory. Similarly, Liu _et al_. [21]
propose a novel Paint Transformer which formulates the learning to paint
problem as a feed-forward set prediction problem. Despite their efficacy, the
current works often lack semantic understanding of the image contents and are
invariably reliant on a progressive grid-based division strategy, wherein the
painting agent divides the overall image into successively finer grids, and
then proceeds to paint each of them in parallel. This inevitably leads to
hierarchically bottom-up painting sequences which are quite mechanical and
thus not applicable for human users.

In this paper, we propose a novel painting pipeline _Intelli-Paint_ which
tries to address the need for semantic-aware painting sequences, by mimicking
the human painting style in three main ways. First, we propose a progressive
layering strategy which, much like a human, allows the painting agent to
successively draw a given scene in multiple layers. That is, instead of
starting to paint the entire scene at once, our method learns to first paint a
realistic background scene representation before adding in each of the
foreground objects in a progressive layerwise fashion.

Second, the human painting process is characterized by a localized spatial
attention span. For instance, a potential artist would focus on different
local image areas while painting distinct parts of the final canvas [34]. This
is in sharp contrast with previous works, which either focus on the entire
image or several predefined grid blocks [21, 36]. To better mimic the human
style, we introduce a sequential brushstroke guidance approach which allows
the painting agent to shift its attention between different image areas
through a sequence of localized attention windows. The spatial dimensions and
position of the localized attention window are progressively adjusted during
the painting process so as to paint a given scene in a coarse-to-fine fashion.

Third, we note that prior works often use a fixed brushstroke budget which not
only leads to wasteful / overlapping brushstroke patterns (refer Fig. 4) but
also imparts an artificial painting style to the final agent. To this end, we
propose an inference-time brushstroke regularization formulation which removes
brushstroke redundancies by regularizing the total number of brushstrokes
required for painting a given canvas. Our experiments reveal that this not
only leads to a $\sim$60-80% enhancement in the brushstroke decomposition
efficiency but also leads to more natural looking painting sequences which are
easily intelligible by a human painter. In summary, towards developing human-
like painting agents, this paper makes the following contributions.

  * •

We introduce a progressive layering approach, which much like a human, allows
the painting agent to draw a given scene in multiple successive layers.

  * •

We propose a sequential brushstroke guidance strategy which enables the
painting agent to focus on different image regions through a learned sequence
of coarse-to-fine localized attention windows.

  * •

Finally, we introduce an inference time brushstroke regularization procedure
which results in a $\sim$60-80% enhancement in the brushstroke decomposition
efficiency and leads to more natural painting sequences which are easily
intelligible by a human user.

##  2 Related Work

Figure 2: _Method Overview._ Given a target image $I$, the _Intelli-Paint_
agent first learns to paint a realistic background scene on the canvas. Once
the background scene has been painted, the agent then proceeds to
progressively add each of the foreground objects using a sequential
brushstroke guidance procedure. To do this, the painting agent first uses the
convex combination formulation from Eq. 7 to select the foreground object it
would like to paint (indicated by object window $\mathcal{G}_{t}$). The
features within each object region are then painted in a coarse-to-fine
fashion through a sequence of localized attention windows $\mathcal{W}_{t}$.
Finally, the brushstroke sequence is fed into a regularization procedure which
removes brushstroke redundancies / overlaps to output the most efficient
painting sequence for each test image.

The problem of teaching machines _“how to paint”_ has been extensively studied
in the context of stroke-based rendering (SBR), which focuses on the
recreation of non-photorealistic imagery through appropriate positioning and
selection of discrete elements such as paint strokes or stipples [11, 33].
Traditional SBR algorithms often require greedy search [10, 19], optimization
over devised heuristics [29], or user input for selecting the position and
nature of each brushstroke [9, 28]. More recent solutions [8, 7] adopt the use
of recurrent neural networks for computing optimal brushstroke decomposition.
However, these methods require access to dense human brushstroke annotations,
which limits their applicability to most real world problems. In another work,
Zhao _et al_. [34] use a conditional variational autoencoder framework for
synthesising time-lapse videos depicting the recreation of a given target
image. However, this requires access to painting time-lapse videos from real
artists for training. Furthermore, the time-lapse outputs are generated at
very low-resolution as compared to the high-resolution output sequences
generated using our approach.

In recent years, there has been an increased focus on learning an unsupervised
brushstroke decomposition without requiring access to dense human brushstroke
annotations. For instance, recent works [31, 13, 6, 22, 12, 26] use deep
reinforcement learning and an adversarial training approach for learning an
efficient brushstroke decomposition. Optimization-based methods [36] directly
search for the optimal brushstroke parameters by performing gradient descent
over a novel optimal-transport-based loss function. In another recent work,
Liu _et al_. [21] propose a novel Paint Transformer which formulates the
learning to paint problem as a feed-forward stroke set prediction problem.

While the above works show high proficiency in painting quality output
canvases, the generation of the same invariably depends on a progressive grid-
based division strategy. In this setting, the agent divides the overall image
into successively finer grids, and then proceeds to paint each of them in
parallel. Experimental analysis reveals that this not only reduces the
efficiency of the final agent, but also leads to mechanical (grid-based)
painting sequences which are not easily intelligible to human users.

##  3 Our Method

The Intelli-Paint framework (refer Fig. 2) is based on a two-stage hybrid
optimization strategy which consists of two modules: _Sequential Planner_ and
_Stroke Regularizer_. In the first stage, the _Sequential Planner_ learns to
predict a coarse but human-like initialization for the brushstroke sequence
${\mathbf{s}}_{init}$. The coarse brushstroke sequence initializations are
then fed into a gradient descent based _stroke regularization_ procedure,
which removes redundant brushstroke patterns and refines the original
brushstroke parameters to output the most efficient stroke decomposition
${\mathbf{s}}_{pred}$ for each test image. This two-stage process is
formulated as,

$\displaystyle{\mathbf{s}}_{init}=Planner(C_{init},I_{target}),$ (1)
$\displaystyle{\mathbf{s}}_{pred}=StrokeReg({\mathbf{s}}_{init},I_{target}),$
(2)

where $C_{init}$ signifies the blank canvas initialization and $I$ is the
target image. In the following sections, we discuss each of the above modules
in full detail.

###  3.1 Sequential Planner

####  3.1.1 Reinforcement Learning Formulation

The _Sequential Planner_ is modelled as a deep reinforcement learning agent
which learns a painting policy $\pi$ predicting vectorized brushstroke
parameters $\mathbf{a}_{t}$ from the current agent state $s_{t}$. The agent
state $s_{t}$ at any timestep $t$ is modeled as the tuple
$(C_{t},I,t,\mathcal{G}_{t},\mathcal{W}_{t},\mathcal{S}_{I},l)$, where $C_{t}$
is the canvas state, $I$ is the target image, $\mathcal{S}_{I}$ signifies the
target-image saliency map, $l$ is the current painting layer (Sec. 3.1.2), and
$(\mathcal{G}_{t},\mathcal{W}_{t})$ represent the coarse and fine local
attention windows for the painting agent respectively (Sec. C.1).

The canvas state $C_{t}$ is updated using a differentiable neural renderer
module, which rasterizes the predicted brushstroke parameters $\mathbf{a}_{t}$
to output a brushstroke alpha map $S_{\alpha}(\mathbf{a}_{t})$ and its colored
rendering $S_{color}(\mathbf{a}_{t})$. The canvas updates at each timestep $t$
are then computed as follows,

$\displaystyle
C_{t+1}=C_{t}\odot(1-S_{\alpha}(\mathbf{a}_{t}))+S_{color}(\mathbf{a}_{t}).$
(3)

We next discuss further details regarding the above formulation which allows
the painting agent to generate canvases while exhibiting a human-like painting
process.

####  3.1.2 Progressive Layering

The human painting process is often progressive and multi-layered [34, 25].
That is, instead of painting everything on the canvas at once, humans often
first paint a basic background layer before progressively adding each of the
foreground objects on top of it (refer Fig. 1). However, such a strategy is
hard to learn using previous works which directly minimize the pixel wise
distance the generated canvas $C_{t}$ and the target image $I$.

To this end, we propose a progressive layering strategy, which much like a
human artist, allows the painted canvas to evolve in multiple successive
layers. The objective of the painting agent in the first layer, is to paint a
realistic background scene by trying to only focus on the non-salient
(background) image areas. In doing so, the salient image regions are painted
so as to maximize the efficiency of painting the background contents (_e.g._
salient region corresponding to a bird sitting on a tree would be painted
while focusing on tree leaves and branches, as in Fig. 4). Once the background
layer is drawn, the painting agent in the successive layer then proceeds to
add different foreground objects in a decreasing order of saliency. An
illustration of a two layer painting process is shown in Fig. 2. The painting
agent first draws a realistic background scene (by focusing only on background
image contents like ground, grass, _etc_.), before adding in the foreground
objects in the second layer.

In order to achieve this layering process, we first divide the painting
episode into multiple layers as follows,

$\displaystyle C_{out}=\sum_{l=0}^{L-1}\
\sum_{t=1}^{T/L}C^{l}_{t}\odot(1-S_{\alpha}(\mathbf{a}^{l}_{t}))+S_{color}(\mathbf{a}^{l}_{t}),$
(4)

where $L=2$ is the number of layers111For simplicity, we primarily use $L=2$
in the main paper. Details on extending progressive layering to $L>2$ are
provided in supp. material., $T$ is the episode length, $C_{0}^{l=0}$
signifies an empty initial canvas, and $C_{0}^{l=1}$ is initialized as the
canvas output $C^{l=0}_{T/L}$ from the last layer.

Given canvas state $C_{t}$, input image $I$ and foreground saliency map
$\mathcal{S}_{I}$, the layerwise painting style can then achieved be achieved
by optimizing the following layered reward objective for each layer $l$,

$\displaystyle r^{layer}_{t}(l)=$ $\displaystyle
D(I\odot\mathcal{M}_{I}(l),C_{t+1}\odot\mathcal{M}_{I}(l))$ (5)
$\displaystyle-$ $\displaystyle
D(I\odot\mathcal{M}_{I}(l),C_{t}\odot\mathcal{M}_{I}(l)),$

where $D(I,C_{t})$ is the joint conditional Wasserstein GAN [1] discriminator
score for image $I$ and canvas $C_{t}$ [12], and the layered-mask
$\mathcal{M}_{I}(l)$ is defined as,

$\displaystyle\mathcal{M}_{I}(l)=1-\mathcal{S}_{I}\odot(1-l).$ (6)

####  3.1.3 Sequential Brushstroke Guidance

Human painters often exhibit a localized spatial attention span while focusing
on distinct image areas [34]. This is in stark contrast with previous works
which either compute stroke decomposition globally over the entire canvas or
over a set of predefined grid regions [12, 21, 36]. To this end, we propose a
sequential brushstroke guidance strategy, which allows the reinforcement
learning agent to shift its attention between different image regions through
a sequence of coarse-to-fine attention windows
$\\{\mathcal{W}_{0},\mathcal{W}_{1}\dots\mathcal{W}_{T}\\}$. The computation
of the localized attention window $\mathcal{W}_{t}$ at any timestep $t$ is
done in the following broad steps,

Foreground object selection. The RL agent first selects the in-focus
foreground object by predicting coordinates of a coarse global attention
window $\mathcal{G}_{t}$. Given an input image $I$ with $N$ foreground
objects, we model
$\mathcal{G}_{t}=x^{\mathcal{G}}_{t},y^{\mathcal{G}}_{t},w^{\mathcal{G}}_{t},h^{\mathcal{G}}_{t}$
as a convex combination of each of in-image object bounding box detections
$\mathcal{B}_{i}\in\mathbb{R}^{4},i\in[1,N]$.

$\displaystyle\mathcal{G}_{t}=\sum_{i=0}^{N}\alpha^{t}_{i}\
\mathcal{B}_{i},\quad s.t.\ \ \forall t\ \
\sum_{i}\alpha^{t}_{i}=1,\quad\alpha^{t}_{i}\geq 0.$ (7)

where $\bm{\alpha}^{t}=\\{\alpha^{t}_{1},\ \dots\
\alpha^{t}_{N}\\}\in\mathbb{R}^{N}$ are the spatial attention parameters
predicted by the RL agent at timestep $t$. $\mathcal{B}_{0}$ represents an
attention window over the entire canvas and is used while switching focus to
background image areas.

Local attention window selection. Within each object window $\mathcal{G}_{t}$,
the agent further learns to sequentially shift its focus on different in-
object features through a sequence of coarse-to-fine local attention windows
$\mathcal{W}_{t}$. In particular, given the coarse object window coordinates
$x^{\mathcal{G}}_{t},y^{\mathcal{G}}_{t},w^{\mathcal{G}}_{t},h^{\mathcal{G}}_{t}$,
the coordinates
$\mathcal{W}_{t}=x^{\mathcal{L}}_{t},y^{\mathcal{L}}_{t},w^{\mathcal{L}}_{t},h^{\mathcal{L}}_{t}$
for the finer localized attention windows are computed in a Markovian fashion
as,

$\displaystyle
x^{\mathcal{L}}_{t+1}=x^{\mathcal{G}}_{t+1}+(x^{\mathcal{L}}_{t}+\Delta
x_{t})\ w^{\mathcal{G}}_{t+1},$ (8) $\displaystyle
y^{\mathcal{L}}_{t+1}=y^{\mathcal{G}}_{t+1}+(y^{\mathcal{L}}_{t}+\Delta
y_{t})\ h^{\mathcal{G}}_{t+1},$ (9) $\displaystyle
w^{\mathcal{L}}_{t+1}=(max(1-\tilde{t},w_{min})+\Delta w_{t})\
w^{\mathcal{G}}_{t+1},$ (10) $\displaystyle
h^{\mathcal{L}}_{t+1}=(max(1-\tilde{t},h_{min})+\Delta h_{t})\
h^{\mathcal{G}}_{t+1},$ (11)

where $\tilde{t}\in[0,1]$ is the normalized episode timestep,
$(w_{min},h_{min})$ are the minimum attention window dimensions and
$(\Delta\mathcal{W}_{t}=\Delta x_{t},\Delta y_{t},\Delta w_{t},\Delta
h_{t})\in\mathbb{R}^{4}$ are successive Markovian [5] updates predicted by the
RL agent.

Brushstroke parameter adjustment. Finally, the coordinates of local attention
window are used to modify the predicted brushstroke parameters
$\mathbf{a}^{l}_{t}$, so as constrain the painting agent to only draw within
the localized attention window. This procedure can be expressed as,

$\displaystyle\mathbf{a}^{l}_{t}\leftarrow
ParamAdjustment(\mathbf{a}^{l}_{t},\mathcal{W}_{t}).$ (12)

Further details regarding the implementation of the adjustment function are
provided in the supplementary material.

####  3.1.4 Human-Consistency Penalties

Human artists inherently try to focus on spatially close image areas and try
to avoid unnecessary spatial oscillations when painting a given image [34]. In
this regard, while the Markovian adjustment procedure introduced in Sec. C.1
ensures the spatial closeness of two consecutive local attention windows
$\mathcal{W}_{t}$, unnecessary movements may still arise due to oscillations
between different coarse attention windows $\mathcal{G}_{t}$. In order to
prevent learning such stroke decompositions we introduce the following spatial
penalty reward,

$\displaystyle r^{spatial}_{t}=-\|\mathcal{G}_{t+1}-\mathcal{G}_{t}\|_{F},$
(13)

where $\|.\|_{F}$ represents the Frobenius norm.

Similarly, human painting sequences are also characterized by the use of same
/ similar color patterns at consecutive timesteps [34]. Thus, in order to
mimic this behaviour we propose the following color transition penalty
$r^{color}_{t}$,

$\displaystyle r^{color}_{t}=-\|(R,G,B)_{t+1}-(R,G,B)_{t}\|_{F},$ (14)

where $(R,G,B)_{t}$ represents the brushstroke color prediction vector at
timestep $t$.

###  3.2 Brushstroke Regularization

The current works on autonomous painting systems are often limited to using
(an almost) fixed brush stroke budget irrespective of the complexity of the
target image. Experiments reveal that this not reduces the efficiency of the
generated painting sequence but also results in redundant / overlapping
brushstroke patterns (refer Fig. 4) which impart an unnatural painting style
to the final agent.

To address this, we propose an inference-time brushstroke regularization
strategy which refines and removes redundancies from the initial brushstroke
sequence predictions $\mathbf{s}_{init}$ to output the most efficient stroke
decomposition $\mathbf{s}_{pred}$ for each test image. To do this, we first
associate each brushstroke with the corresponding importance vector
$\beta^{l}_{t}\in[0,1]$ by modifying the stroke rendering process as,

$\displaystyle C_{out}=\sum_{l=0}^{L-1}\
\sum_{t=1}^{T/L}C^{l}_{t}\odot(1-\beta^{l}_{t}\
S_{\alpha}(\mathbf{a}^{l}_{t}))+\beta^{l}_{t}\ S_{color}(\mathbf{a}^{l}_{t}),$

where $\beta^{l}_{t}=Sign(x^{l}_{t})$ and
$x^{l}_{t}\sim\mathcal{N}(0,10^{-3})$ is randomly initialized from a normal
distribution.

We then use gradient descent in order to optimize the following loss function
over both brushstroke parameters $\mathbf{a}^{l}_{t}$ and importance vectors
$\beta^{l}_{t}$ (through $x^{l}_{t}$):

$\displaystyle\mathcal{L}_{total}(\mathbf{a}^{l}_{t},x^{l}_{t})=\mathcal{L}_{2}(I,C_{out})+\gamma\
\sum_{l=0}^{L-1}\sum_{t=1}^{T/L}\|\beta^{l}_{t}\|_{1},$ (15)

where the backpropagation gradients $\partial\beta^{l}_{t}/{\partial
x^{l}_{t}}$ are computed as $\sigma(x^{l}_{t})(1-\sigma(x^{l}_{t}))$,
$\sigma(.)$ is the sigmoid function and $\gamma$ balances the weightage
between brushstroke refinement and the need to use as few brushstrokes as
possible.

Figure 3: _Qualitative method comparison w.r.t painting efficiency._ Comparing
final canvas outputs while using $\sim$ 300 brushstrokes for (b) Ours, (c)
Paint Transformer [21], (d) Optim [36], (e) RL [12] and (f) Semantic-RL [26].
We observe that our approach results in more accurate depiction of the fine-
grain features in the target image while using a low brushstroke count. Please
zoom in for better comparison.

##  4 Implementation Details

Neural renderer. In this paper, we primarily adopt the _PixelShuffleNet_
architecture from Huang _et al_. [12] while designing the neural
differentiable renderer. While our approach is not limited to a particular
rendering mechanism, we find that as opposed to the opaque brushstroke models
used in [36, 21], the use of a more naturally blending brushstroke
representation from [12], allows our method to mimic the human painting style
in a more closer fashion.

Layered training. The use of progressive layering module requires
conditionally training the painting agent policy at each layer while
initializing the canvas state with the output from the last layer. In order to
save computation time during training, we train the successive layer policies
in consecutive batches while using the canvas output from the last layer.
Furthermore, we only use $L=2$ layers at the training time. At inference time,
the trained progressive layering policy can then be applied for $L>2$ layers
by appropriately modifying the target image saliency maps. Please refer
supplementary material for further details.

Saliency and bounding box predictions. A key component of the Intelli-Paint
pipeline is the sequential brushstroke guidance strategy which relies on the
computation of object saliency and bounding box predictions. In this work, we
use a pretrained U-2-Net model [24] model in order to compute foreground
saliency predictions. The bounding box predictions are then computed as the
union over bounding box outputs from pretrained Yolo-v5 [14] and the overall
bounding box for the saliency prediction output.

Overall training. The RL-based _Sequential Planner_ agent is trained using the
model-based DDPG algorithm [12] with the following overall reward function for
each layer $l$,

$\displaystyle r_{t}^{overall}(l)=r^{layer}_{t}(l)+\mu\ r^{gbp}_{t}+\eta\
r^{spatial}_{t}+\gamma\ r^{color}_{t},$

where $r^{gbp}_{t}$ is the guided backpropagation based focus reward from
[26]. The hyperparameter $\mu$ is set to 10 for our experiments, while the
human consistency penalty coefficients $\eta,\gamma$ are raised from $10^{-4}$
to $0.1$ in a linear schedule during the training process. The final RL agent
is trained for a total of 5M iterations with a batch size of 128.

Figure 4: _Qualitative method comparison w.r.t resemblance with the human
painting style._ We compare different methods (b-f). All painting sequences
are generated using a different brushstroke count (indicated in the boxes), so
as to ensure similar pixel-wise reconstruction loss with the target image. The
corresponding frames for each sequence are computed after $\sim$ 10%, 40%, 60%
and 100% of the overall painting episode. We clearly see that our method
offers higher resemblance with the human painting style as compared to
previous works.

##  5 Comparison with the state of the art

In this section, we provide extensive qualitative and quantitative results
comparing our method with recent state of the art neural painting methods [12,
36, 26, 21]. First, in Sec. 5.1, we demonstrate the improved painting
efficiency of our method in generating detailed paintings when using limited
number of brushstrokes. Second, we show that our method leads to painting
sequences with increased resemblance with the human painting style (refer Sec.
5.2). Finally, we provide a discussion of some assumptions that our method
makes and analyse the robustness of our approach to potential violations of
the same (refer Sec. 5.3).

###  5.1 Painting Efficiency

_Qualitative Comparison._ Fig. 3 shows a qualitative comparison between the
generated canvases using a low budget of 300 brushstrokes per canvas. Note
that due to grid-wise formulation for Paint Transformer [21] and Optim [36],
the corresponding results are reported after $\sim$360 and 330 brushstrokes
respectively. We clearly see that our method results in more accurate
depictions of target image (_e.g._ fine-grain features for car, hut, and birds
in row 1-3 from Fig. 3) when using limited number of brushstrokes. In
contrast, previous methods often lack an intelligent mechanism for efficient
brushstroke distribution across the canvas which leads to poor performance
when using a limited brushstroke budget. Interestingly, we also note that
while Paint Transformer [21] exhibits improved inference speeds, it performs
worse than previous methods like Optim [36] when using a small number of
brushstrokes.

_Quantitative Comparison._ Table 1 shows quantitative results on the quality
of the finally generated canvases using $\sim$ 300 brushstrokes per canvas.
The final results are reported in terms of both pixel wise $l_{2}$ distance
$\mathcal{L}_{pixel}$ and perceptual similarity loss $\mathcal{L}_{pcpt}$ [15]
between the final canvas and the target image. The quantitative values show
that our method helps in significantly lowering the distance metrics between
the painted canvas and the target image as compared to previous works. In
particular, we note that for CUB-Birds dataset [30], our approach leads to a
reduction of 47.1%, 25.6% 24.9% and 38.2% in the $\mathcal{L}_{pixel}$
distance metric as compared to RL[12], Semantic-RL[26], Optim [36] and Paint
Transformer [20], respectively.

Method | Stanford Cars[17] | CUB-Birds[30]
---|---|---
$\mathcal{L}_{pixel}$ | $\mathcal{L}_{pcpt}$ | $\mathcal{L}_{pixel}$ | $\mathcal{L}_{pcpt}$
RL [12] | 97.85 | 0.67 | 96.14 | 0.76
Semantic-RL [26] | 79.98 | 0.55 | 68.46 | 0.55
Optim [36] | 76.52 | 0.54 | 67.90 | 0.53
Transformer [21] | 87.78 | 0.57 | 82.43 | 0.56
Ours | 56.92 | 0.44 | 50.94 | 0.45
Table 1: _Method comparison w.r.t painting efficiency._ Our method results in
much more accurate depictions of the target image while using a limited
brushstroke budget.

###  5.2 Similarity with Human Painting Style

_Qualitative Comparison._ We demonstrate the practical applicability of our
method to actual human users by qualitatively comparing the painting sequences
generated by our method with those drawn by actual human artists. Results are
provided in Fig. 4. We observe that our method bears high resemblance with the
human painting style in terms of both layerwise painting evolution and
localized attention. In contrast, previous state of the art methods often try
to directly minimize the pixel wise distance between the painted canvas and
the target image, thereby leading to intermediate canvas states which are less
intelligible for a human user.

For instance, in the first example from Fig. 4, much like a human painter, our
method first paints a realistic background representation (consisting of the
sky, mountains, river and the ground) before drawing in the foreground car in
a coarse-to-fine fashion. This results in a more human-like evolution of the
painted canvas which can be easily relatable to actual human artists. In
contrast, methods like Paint Transformer[21], Optim [36] and RL [12] directly
make brushstrokes based on low-level image features (_e.g._ red brushstrokes
for the car in row-1 and head of the bird in row-5 from Fig. 4). This leads to
more bottom-up painting sequences which are different from the human style.
Meanwhile, Semantic-RL [26] tries to paint both foreground and background
regions in parallel, thereby lacking the semantic painting evolution exhibited
by human users.

_Quantitative Comparison._ We also report quantitative results demonstrating
the human-likeliness of our approach as compared to previous works. To this
end, we devise a human user study wherein each human participant is shown a
series of paired painting sequences comparing our method with previous works.
For each pair, the human subject is then asked to select the painting sequence
which best resembles the human painting style. The user study was conducted
across 50 unique Amazon Mechanical Turk subjects and a set of randomly chosen
100 painting sequences from the CelebA [18] and CUB-Birds [30] datasets.
Results are reported in Table 2. We clearly see that our method results in
painting sequences which are perceived as a closer match for the human
painting style by majority of human users.

Method | Intelli-Paint Preference
---|---
RL [12] | 83.11 %
Semantic-RL [26] | 69.09 %
Optim [36] | 75.41 %
Transformer [21] | 86.50 %
Table 2: _User-Study:_ Showing % of painting samples for which human users
prefer Intelli-Paint sequences over previous works. In all the four studies,
more users chose Intelli-Paint over the other.

###  5.3 Robustness to Limitations / Assumptions

While our work results in significant improvements in both painting efficiency
and human-likeliness of the generated painting sequence, it does build on some
inherent assumptions. _First_ , we note that our method only mimics some
_general_ aspects (progressive layering, coarse to fine localized attention)
of the human painting process, and, thus does not claim to be calibrated to
the fine-grain variations in the painting styles of each human artist.
Nevertheless, as demonstrated in Table 2, we find that our painting style is
considered more relatable by majority of human users.

_Second_ , our method relies on the computation of image saliency masks for
allowing a human-like evolution of the painted canvas. Thus limitations of the
pretrained U2-Net [24] model become our limitations. Nevertheless, we note
that failure to detect a particular salient object would simply lead to
painting the corresponding region in the background layer, and thus does not
affect the quality of the final canvas.

_Finally,_ to learn a human-like painting style, our method requires self-
supervised training on a dataset of _real_ images. This is in contrast with
Paint transformer [21] which performs self-training on an _artificial_
dataset, and Optim [36] which does not require any training. That said, once
trained we find that our method is able to generalize across a range of
domains at inference time. For instance, we note that all results in Fig. 3, 4
were generated using an Intelli-paint model trained only on the CUB-Birds [30]
dataset.

##  6 Conclusion

In this paper, we emphasize that the practical merits of an autonomous
painting system should be evaluated not only by the quality of generated
canvas but also by the interpretability of the corresponding painting sequence
by actual human artists. To this end, we propose a novel _Intelli-Paint_
pipeline which uses progressive layering to allow for a more human-like
evolution of the painted canvas. The painting agent focuses on different image
areas through a sequence of coarse-to-fine localized attention windows and is
able to paint detailed scenes while using a limited number of brushstrokes.
Experiments reveal that in comparison with previous state-of-the-art methods,
our approach not only shows improved painting efficiency but also exhibits a
painting style which is much more relatable to actual human users.

## References

  * [1] Martin Arjovsky, Soumith Chintala, and Léon Bottou.  Wasserstein gan.  arXiv preprint arXiv:1701.07875, 2017.
  * [2] Kevin Crowston.  Amazon mechanical turk: A research tool for organizations and information systems scholars.  In Shaping the future of ict research. methods and approaches, pages 210–221. Springer, 2012.
  * [3] Kevin Frans and Chin-Yi Cheng.  Unsupervised image to sequence translation with canvas-drawer networks.  arXiv preprint arXiv:1809.08340, 2018.
  * [4] Kevin Frans, LB Soros, and Olaf Witkowski.  Clipdraw: exploring text-to-drawing synthesis through language-image encoders.  arXiv preprint arXiv:2106.14843, 2021.
  * [5] Paul A Gagniuc.  Markov chains: from theory to implementation and experimentation.  John Wiley & Sons, 2017.
  * [6] Yaroslav Ganin, Tejas Kulkarni, Igor Babuschkin, SM Eslami, and Oriol Vinyals.  Synthesizing programs for images using reinforced adversarial learning.  arXiv preprint arXiv:1804.01118, 2018.
  * [7] Alex Graves.  Generating sequences with recurrent neural networks.  arXiv preprint arXiv:1308.0850, 2013.
  * [8] David Ha and Douglas Eck.  A neural representation of sketch drawings.  arXiv preprint arXiv:1704.03477, 2017.
  * [9] Paul Haeberli.  Paint by numbers: Abstract image representations.  In Proceedings of the 17th annual conference on Computer graphics and interactive techniques, pages 207–214, 1990.
  * [10] Aaron Hertzmann.  Painterly rendering with curved brush strokes of multiple sizes.  In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pages 453–460, 1998.
  * [11] Aaron Hertzmann.  A survey of stroke-based rendering.  Institute of Electrical and Electronics Engineers, 2003.
  * [12] Zhewei Huang, Wen Heng, and Shuchang Zhou.  Learning to paint with model-based deep reinforcement learning.  In Proceedings of the IEEE International Conference on Computer Vision, pages 8709–8718, 2019.
  * [13] Biao Jia, Jonathan Brandt, Radomír Mech, Byungmoon Kim, and Dinesh Manocha.  Lpaintb: Learning to paint from self-supervision.  arXiv preprint arXiv:1906.06841, 2019.
  * [14] Glenn Jocher, Alex Stoken, Ayush Chaurasia, Jirka Borovec, NanoCode012, TaoXie, Yonghye Kwon, Kalen Michael, Liu Changyu, Jiacong Fang, Abhiram V, Laughing, tkianai, yxNONG, Piotr Skalski, Adam Hogan, Jebastin Nadar, imyhxy, Lorenzo Mammana, AlexWang1900, Cristi Fati, Diego Montes, Jan Hajek, Laurentiu Diaconu, Mai Thanh Minh, Marc, albinxavi, fatih, oleg, and wanghaoyang0106.  ultralytics/yolov5: v6.0 - YOLOv5n ’Nano’ models, Roboflow integration, TensorFlow export, OpenCV DNN support, Oct. 2021.
  * [15] Justin Johnson, Alexandre Alahi, and Li Fei-Fei.  Perceptual losses for real-time style transfer and super-resolution.  In European conference on computer vision, pages 694–711. Springer, 2016.
  * [16] Dmytro Kotovenko, Matthias Wright, Arthur Heimbrecht, and Björn Ommer.  Rethinking style transfer: From pixels to parameterized brushstrokes.  arXiv preprint arXiv:2103.17185, 2021.
  * [17] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei.  3d object representations for fine-grained categorization.  In 4th International IEEE Workshop on 3D Representation and Recognition (3dRR-13), Sydney, Australia, 2013.
  * [18] Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo.  Maskgan: Towards diverse and interactive facial image manipulation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  * [19] Peter Litwinowicz.  Processing images and video for an impressionist effect.  In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 407–414, 1997.
  * [20] Beyang Liu, Stephen Gould, and Daphne Koller.  Single image depth estimation from predicted semantic labels.  In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1253–1260. IEEE, 2010.
  * [21] Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Ruifeng Deng, Xin Li, Errui Ding, and Hao Wang.  Paint transformer: Feed forward neural painting with stroke prediction.  arXiv preprint arXiv:2108.03798, 2021.
  * [22] John FJ Mellor, Eunbyung Park, Yaroslav Ganin, Igor Babuschkin, Tejas Kulkarni, Dan Rosenbaum, Andy Ballard, Theophane Weber, Oriol Vinyals, and SM Eslami.  Unsupervised doodling and painting with improved spiral.  arXiv preprint arXiv:1910.01007, 2019.
  * [23] Reiichiro Nakano.  Neural painters: A learned differentiable constraint for generating brushstroke paintings.  arXiv preprint arXiv:1904.08410, 2019.
  * [24] Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan, Osmar Zaiane, and Martin Jagersand.  U2-net: Going deeper with nested u-structure for salient object detection.  volume 106, page 107404, 2020.
  * [25] Nancy Reyner.  How to paint with layers - in acrylic & oil, Dec 2017.
  * [26] Jaskirat Singh and Liang Zheng.  Combining semantic guidance and deep reinforcement learning for generating human level paintings.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16387–16396, 2021.
  * [27] Avishek Siris, Jianbo Jiao, Gary K.L. Tam, Xianghua Xie, and Rynson W.H. Lau.  Inferring attention shift ranks of objects for image saliency.  In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
  * [28] Daniel Teece.  3d painting for non-photorealistic rendering.  In ACM SIGGRAPH 98 Conference abstracts and applications, page 248, 1998.
  * [29] Greg Turk and David Banks.  Image-guided streamline placement.  In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 453–460, 1996.
  * [30] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie.  The Caltech-UCSD Birds-200-2011 Dataset.  Technical Report CNS-TR-2011-001, California Institute of Technology, 2011\.
  * [31] Ning Xie, Hirotaka Hachiya, and Masashi Sugiyama.  Artist agent: A reinforcement learning approach to automatic stroke generation in oriental ink painting.  IEICE TRANSACTIONS on Information and Systems, 96(5):1134–1144, 2013.
  * [32] Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S Huang.  Generative image inpainting with contextual attention.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5505–5514, 2018.
  * [33] Kun Zeng, Mingtian Zhao, Caiming Xiong, and Song Chun Zhu.  From image parsing to painterly rendering.  ACM Trans. Graph., 29(1):2–1, 2009.
  * [34] Amy Zhao, Guha Balakrishnan, Kathleen M Lewis, Frédo Durand, John V Guttag, and Adrian V Dalca.  Painting many pasts: Synthesizing time lapse videos of paintings.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8435–8445, 2020.
  * [35] Ningyuan Zheng, Yifan Jiang, and Dingjiang Huang.  Strokenet: A neural painting environment.  In International Conference on Learning Representations, 2018.
  * [36] Zhengxia Zou, Tianyang Shi, Shuang Qiu, Yi Yuan, and Zhenwei Shi.  Stylized neural painting.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15689–15698, 2021.

Supplementary Material
Intelli-Paint: Towards Developing Human-like Painting Agents

Figure 5: Artistic Painting Evolution. In contrast with previous works which
directly try to minimize the $\mathcal{L}_{pixel}$ distance between the
painted canvas and the original image (d), our method allows for a more human-
like evolution of the painted canvas across different domains. For instance,
for faces it first begins with a rough facial outline shown in (a). It then
refines this outline further to paint the highly artistic human-like
intermediate representation in (b), before finally adding all facial details
(eyes, nose, lips) in the final painting (c).

##  Appendix A Analysis: Progressive Layering

The progressive layering module forms one of the key components of the
_Intelli-paint_ pipeline. In this section, we further discuss different
aspects of the progressive layering module. First, we demonstrate the human-
like artistic painting evolution exhibited by our method for the facial domain
in Appendix A.1. We then analyse the use of the progressive layering module as
an effective alternative to performing foreground object removal for non-
standard object masks (refer Appendix A.2). Finally, we provide a formulation
for extending the proposed layering mechanism for $L>2$ layers (refer Appendix
A.3).

###  A.1 Artistic Painting Evolution

By allowing the final painting agent to draw a given scene in multiple
successive layers, the progressive layering strategy facilitates for a more
human-like evolution of the painted canvas [25]. In this section, we analyse
some intermediate canvas representations painted by our method, and compare
its resemblance with the human-like ideation process for the facial domain.

Figure 6: Intelli-paint for foreground object removal. Comparing foreground
object removal using (b) progressive layering vs GAN-based image inpainting
from (c) Yu _et al_. [32]. Our method performs image inpainting using
_learning to paint_ paradigm itself and results in much more natural
background scene representations as opposed to GAN based image inpainting
methods for non-standard object masks.

Results are shown in Fig. 5. We observe that instead of directly trying to
minimize the pixel-wise distance between the painted canvas and the target
image (as is done in previous works), our method takes a much more human-like
approach to portrait generation. For instance, consider the second example
(row-2) for the painting sequence in the right column. Instead of directly
painting based on low-level features (_e.g._ white brushstrokes for the mouth
region), our method first draws a rough outline for the facial shape. It then
refines this outline whilst indicating (but not drawing) the potential
locations of important focalpoints (eyes, nose, lips) through appropriate
facial shading. Only after the intermediate sketch has been set up, does it
then proceed to add the fine-grain details on the exact facial features (eyes,
nose, mouth, _etc_).

###  A.2 Application: Foreground Removal

The progressive layering module of the _Intelli-Paint_ pipeline also provides
an effective alternative for performing foreground object removal using non-
standard object masks. As compared to previous GAN based methods, our method
performs image in-painting over the foreground regions using the _learning to
paint_ paradigm itself. Results are shown in Fig. 6. We clearly see that the
intermediate canvases generated using our method result in much more natural
looking background scene representations. In contrast, GAN based image-
inpainting methods show poor generalizability over large non-standard object
masks and clearly reveal the presence of the foreground object in the final
image predictions.

###  A.3 Extending Progressive Layering

The progressive layering formulation discussed in the main paper primarily
divides the painting process into two broad layers (background and
foreground). In this section, we show that the original two layered
progressive formulation can be easily extended to $L>2$ layers by using ranked
saliency maps [27] for the target image $I$.

In particular, consider a ranked saliency map ${\mathcal{S}_{I}}$, such that
$\\{\mathcal{S}_{I}[k]\\}_{k=1}^{L}$ indicate the salient regions for the
target image $I$ ranked in increasing order of saliency. The progressive
layering strategy can then be trivially extended to $L>2$ layers by using the
following layered-mask $\mathcal{M}_{I}(l)$ (refer Eq. 5, 6 in the main paper)
for each layer $l$,

$\displaystyle\mathcal{M}_{I}(l)=1-\bigcup_{k=1}^{L-l}\mathcal{S}_{I}[k].$
(16)

Figure 7: Progressive Layering for $L>2$ layers. The agent first begins by
painting a realistic background scene. Once the background layer is drawn, the
painting agent in each successive layer then proceeds to add different
foreground objects in decreasing order of saliency.

Results for progressive painting sequences while using $L>2$ layers are shown
in Fig. 7. The painting process is divided in three layers. We observe that in
the first layer, the painting agent proceeds to draw a realistic background
scene on the canvas. In the second layer, it then proceeds to add the most
salient object (white car) on to the canvas. Finally, in the third layer it
shifts its attention to the least salient object (cow) in order to complete
the final painting.

##  Appendix B Ablation Studies

The _Intelli-paint_ pipeline utilizes three main modules for mimicking the
human painting style: 1) progressive layering, 2) sequential brushstroke
guidance and 3) brushstroke regularization. In this section, we aim to
understand the contribution of each of these modules in the development of a
human-like painting process. However, since the main focus of our method is
the recreation of the human artistic creation process, we argue that the
effect of each module is best understood by visualizing the corresponding
painting sequences. We thus refer the readers to the official project webpage
for detailed visualizations on the importance of each module in the human-like
painting process.

##  Appendix C Algorithm Details

In this section, we further elaborate on some algorithm details which could
not be included / fully explained due to space constraints in the main paper.

###  C.1 Sequential Brushstroke Guidance

The sequential brushstroke guidance strategy (discussed in Sec. 3.1.3 of main
paper), allows the reinforcement learning based sequential planner agent to
shift its attention between different image regions through a sequence of
coarse-to-fine attention windows
$\\{\mathcal{W}_{0},\mathcal{W}_{1}\dots\mathcal{W}_{T}\\}$. The computation
of the localized attention window $\mathcal{W}_{t}$ at any timestep $t$ is
done in the following broad steps.

Foreground object selection. The RL agent first selects the in-focus
foreground object by predicting coordinates of a coarse global attention
window $\mathcal{G}_{t}$. Given an input image $I$ with $N$ foreground
objects, the computation of the coarse attention window $\mathcal{G}_{t}$ can
be expressed through Algorithm 1.

Algorithm 1 Foreground Object Selection

Input: Foreground selection convex coefficients $\bm{\alpha}^{t}$; In-image
bounding box detections $\mathcal{B}_{i}\in\mathbb{R}^{4},i\in[1,N]$.
Output: Coarse object attention window $\mathcal{G}_{t}$.
Defaults: Bounding box over the entire image $\mathcal{B}_{0}$.

1:function
ObjectSelect($\bm{\alpha}^{t},\\{\mathcal{B}_{0},\dots\mathcal{B}_{N}\\}$)

2: $\\{\alpha^{t}_{0},\ \dots\
\alpha^{t}_{N}\\}=\bm{\alpha}^{t}\in\mathbb{R}^{N+1}$;

3: $\mathcal{G}_{t}=\sum_{i=0}^{N}\alpha^{t}_{i}\ \mathcal{B}_{i}$;

4: return $\mathcal{G}_{t}$.

5:end function

Local attention window selection. Within each object window $\mathcal{G}_{t}$,
the agent further learns to sequentially shift its focus on different in-
object features through a sequence of coarse-to-fine local attention windows
$\mathcal{W}_{t}$. In particular, given the coarse object window coordinates
$x^{\mathcal{G}}_{t},y^{\mathcal{G}}_{t},w^{\mathcal{G}}_{t},h^{\mathcal{G}}_{t}$,
the coordinates
$\mathcal{W}_{t}=x^{\mathcal{L}}_{t},y^{\mathcal{L}}_{t},w^{\mathcal{L}}_{t},h^{\mathcal{L}}_{t}$
for the finer localized attention windows are computed using the Markov update
function in Algorithm 2.

Algorithm 2 Markov Updates for Local Attention Window

Input: Current coarse, local attention windows
$(\mathcal{G}_{t},\mathcal{W}_{t})$; Markovian bounding box refinements
$\Delta\mathcal{W}_{t}$.
Output: Updated local attention window $\mathcal{W}_{t+1}$.
Defaults: $w_{min}=h_{min}=0.2$.

1:function
MarkovUpdate($\mathcal{W}_{t},\mathcal{G}_{t},\Delta\mathcal{W}_{t}$)

2:

3:
$x^{\mathcal{G}}_{t},y^{\mathcal{G}}_{t},w^{\mathcal{G}}_{t},h^{\mathcal{G}}_{t}=\mathcal{G}_{t}$;

4:
$x^{\mathcal{L}}_{t},y^{\mathcal{L}}_{t},w^{\mathcal{L}}_{t},h^{\mathcal{L}}_{t}=\mathcal{W}_{t}$;

5: $\Delta x_{t},\Delta y_{t},\Delta w_{t},\Delta
h_{t}=\Delta\mathcal{W}_{t}$;

6:

7: $x^{\mathcal{L}}_{t+1}=x^{\mathcal{G}}_{t+1}+(x^{\mathcal{L}}_{t}+\Delta
x_{t})\ w^{\mathcal{G}}_{t+1}$;

8: $y^{\mathcal{L}}_{t+1}=y^{\mathcal{G}}_{t+1}+(y^{\mathcal{L}}_{t}+\Delta
y_{t})\ h^{\mathcal{G}}_{t+1}$;

9: $w^{\mathcal{L}}_{t+1}=(max(1-\tilde{t},w_{min})+\Delta w_{t})\
w^{\mathcal{G}}_{t+1}$;

10: $h^{\mathcal{L}}_{t+1}=(max(1-\tilde{t},h_{min})+\Delta h_{t})\
h^{\mathcal{G}}_{t+1}$;

11:

12:
$\mathcal{W}_{t+1}=x^{\mathcal{L}}_{t+1},y^{\mathcal{L}}_{t+1},w^{\mathcal{L}}_{t+1},h^{\mathcal{L}}_{t+1}$;

13: return $\mathcal{W}_{t+1}$.

14:

15:end function

Brushstroke parameter adjustment. Finally, the coordinates of local attention
window are used to modify the predicted brushstroke parameters
$\mathbf{a}^{l}_{t}$, so as constrain the painting agent to only draw within
the localized attention window. This procedure can be expressed as,

$\displaystyle\mathbf{a}^{l}_{t}\leftarrow\textsc{ParamAdjust}(\mathbf{a}^{l}_{t},\mathcal{W}_{t}).$
(17)

Assuming that $\mathbf{a}^{l}_{t}$ at each timestep $t$, depicts the
parameters of a quadratic Bézier curve [12, 26] as follows,

$\displaystyle\mathbf{a}^{l}_{t}=(x_{0},y_{0},x_{1},y_{1},x_{2},y_{2},z_{0},z_{2},w_{0},w_{2},r,g,b),$
(18)

where the first 10 parameters depict stroke position, shape and transparency,
while the last 3 parameters form the _rgb_ representation for the stroke
color. The parameter adjustment function is then implemented as per Algorithm
3.

Algorithm 3 Parameter Adjustment Function

Input: Initial vectorized brushstroke prediction $\mathbf{a}^{l}_{t}$; current
local attention window coordinates $\mathcal{W}_{t}$.
Output: Modified brushstroke prediction vector $\mathbf{a}^{l}_{t}$.

1:function ParamAdjust($\mathbf{a}^{l}_{t},\mathcal{W}_{t}$)

2:

3:
$x_{0},y_{0},x_{1},y_{1},x_{2},y_{2},z_{0},z_{2},w_{0},w_{2},r,g,b=\mathbf{a}^{l}_{t}$;

4:
$x^{\mathcal{L}}_{t},y^{\mathcal{L}}_{t},w^{\mathcal{L}}_{t},h^{\mathcal{L}}_{t}=\mathcal{W}_{t}$;

5:

6: $x_{0}=x^{\mathcal{L}}_{t}+x_{0}\cdot w^{\mathcal{L}}_{t}$;

7: $y_{0}=y^{\mathcal{L}}_{t}+y_{0}\cdot h^{\mathcal{L}}_{t}$;

8: $x_{2}=x^{\mathcal{L}}_{t}+x_{2}\cdot w^{\mathcal{L}}_{t}$;

9: $y_{2}=x^{\mathcal{L}}_{t}+y_{2}\cdot h^{\mathcal{L}}_{t}$;

10: $w_{0}=w_{0}\cdot avg(w^{\mathcal{L}}_{t},h^{\mathcal{L}}_{t})$;

11: $w_{2}=w_{2}\cdot avg(w^{\mathcal{L}}_{t},h^{\mathcal{L}}_{t})$;

12:

13:
$\mathbf{a}^{l}_{t}=x_{0},y_{0},x_{1},y_{1},x_{2},y_{2},z_{0},z_{2},w_{0},w_{2},r,g,b$;

14: return $\mathbf{a}^{l}_{t}$.

15:

16:end function

###  C.2 Brushstroke Regularization

The current works on autonomous painting systems are often limited to using
(an almost) fixed brush stroke budget irrespective of the complexity of the
target image. Experiments reveal that this not only reduces the efficiency of
the generated painting sequence but also results in redundant / overlapping
brushstroke patterns (refer main paper) which impart an unnatural painting
style to the final agent.

To address this, we propose an inference-time brushstroke regularization
strategy which refines and removes redundancies from the initial brushstroke
sequence predictions $\mathbf{s}_{init}$ to output the most efficient stroke
decomposition $\mathbf{s}_{pred}$ for each test image. This regularization
procedure can be summarized through Algorithm 4.

Algorithm 4 Brushstroke Regularization Function

Input: A target image $I$; initial brushstroke sequence
$\mathbf{s}_{init}=\\{\mathbf{a}^{l}_{t}\mid 0\leq l\leq L-1,0\leq t\leq
T/L\\}$.
Output: Refined brushstroke sequence $\mathbf{s}_{pred}$.
Defaults: Number of layers $L$; episode length $T$; number of iterations $M$,
$C_{init}=C_{0}^{l=0}=\textsc{BlankCanvas}$;

1:function StrokeReg($\mathbf{s}_{init},I$):

2: $\\{\mathbf{a}^{l}_{t}\mid 0\leq l\leq L-1,0\leq t\leq
T/L\\}=\mathbf{s}_{init}$;

3: $x^{l}_{t}\sim\mathcal{N}(0,10^{-3})\quad\forall l,\forall t$;

4: for $0\leq i\leq M$ do

5: $\beta^{l}_{t}=\textsc{Sign}(x^{l}_{t})\quad\forall t,l$;

6: []  $\begin{multlined}C_{out}=\sum_{l=0}^{L-1}\
\sum_{t=1}^{T/L}C^{l}_{t}\odot(1-\beta^{l}_{t}\
S_{\alpha}(\mathbf{a}^{l}_{t}))\\\ +\ \beta^{l}_{t}\
S_{color}(\mathbf{a}^{l}_{t});\end{multlined}C_{out}=\sum_{l=0}^{L-1}\
\sum_{t=1}^{T/L}C^{l}_{t}\odot(1-\beta^{l}_{t}\
S_{\alpha}(\mathbf{a}^{l}_{t}))\\\ +\ \beta^{l}_{t}\
S_{color}(\mathbf{a}^{l}_{t});$

7: $\mathcal{L}_{total}=\mathcal{L}_{2}(I,C_{out})+\gamma\
\sum_{l=0}^{L-1}\sum_{t=1}^{T/L}\|\beta^{l}_{t}\|_{1}$;

8: []  $a^{l}_{t}\leftarrow
a^{l}_{t}-\frac{\partial\mathcal{L}_{total}}{\partial a^{l}_{t}}\quad\forall
t,l;$

9: []  $x^{l}_{t}\leftarrow
x^{l}_{t}-\frac{\partial\mathcal{L}_{total}}{\partial x^{l}_{t}}\quad\forall
t,l;$

10: end for

11: $\mathbf{s}_{pred}=\\{\beta^{l}_{t}\cdot\mathbf{a}^{l}_{t}\mid 0\leq l\leq
L-1,0\leq t\leq T/L\\}$;

12: return $\mathbf{s}_{pred}$.

13:end function

###  C.3 Overall Inference Algorithm

The overall inference algorithm for the _Intelli-paint_ pipeline can now be
summarized as per Algorithm 5.

Algorithm 5 Inference Algorithm for Intelli-Paint

Input: A target image $I$; image-saliency map $\mathcal{S}_{I}$; number of
layers $L$; painting episode length $T$.
Required: RL-based sequential planner POLICY;

1:$\mathcal{W}_{0}=x^{\mathcal{L}}_{0},y^{\mathcal{L}}_{0},w^{\mathcal{L}}_{0},h^{\mathcal{L}}_{0}=(0,0,1,1)$;

2:$C_{init}=C_{0}^{l=0}=\textsc{BlankCanvas}$;

3:for $0\leq l\leq L-1$ do

4: for $0\leq t\leq T/L$ do

5: $s_{t}=(I,C^{l}_{t},\mathcal{G}_{t},\mathcal{W}_{t},\mathcal{S}_{I},l)$;

6:
$\mathbf{a}^{l}_{t},\bm{\alpha^{t}},\Delta\mathcal{W}_{t}=\textsc{Policy}(s_{t})$;

7:
$\mathcal{G}_{t}=\textsc{ObjectSelect}(\bm{\alpha}^{t},\\{\mathcal{B}_{0},\dots\mathcal{B}_{N}\\})$;

8:
$\mathcal{W}_{t}=\textsc{MarkovUpdate}(\mathcal{G}_{t},\mathcal{W}_{t-1},\Delta\mathcal{W}_{t})$;

9:
$\mathbf{a}^{l}_{t}\leftarrow\textsc{ParamAdjust}(\mathbf{a}^{l}_{t},\mathcal{W}_{t})$;

10:
$C^{l}_{t+1}=C^{l}_{t}\odot(1-S_{\alpha}(\mathbf{a}^{l}_{t}))+S_{color}(\mathbf{a}^{l}_{t})$;

11: end for

12:end for

13:$\mathbf{s}_{init}=\\{\mathbf{a}^{l}_{t}\mid 0\leq l\leq L-1,0\leq t\leq
T/L\\}$;

14:${\mathbf{s}}_{pred}=\textsc{StrokeReg}({\mathbf{s}}_{init},I_{target})$;

15:return $\mathbf{s}_{pred}$.

##  Appendix D Human User Study

Figure 8: Human User Study Interface. Each user is shown two painting
sequences (_gif_ format) and asked to select the one which would be most
interpretable by actual human artists.

In this section, we provide details about the human data collection process
which is used to quantitatively demonstrate the human-likeliness of our
approach as compared to previous state of the art [12, 26, 36, 21].

Basic Setup. The user-study was conducted across 50 unique Amazon Mechanical
Turk [2] subjects wherein each human participant is shown a series of paired
painting sequences comparing our method with previous works. For each pair,
the human subject is then asked to select the painting sequence which best
resembles the human painting style. Each painting sequence is presented as a
_gif_ image with a total duration of 10 seconds. Fig. 8 illustrates the basic
interface setup for the user-study.

Filtering process. Since the choice on human-likeliness of a painting sequence
is subjective, we found that unfiltered collection of data from Amazon M-turk
can lead to quite noisy responses, wherein many users simply select the
responses at random in order to quickly collect the assignment reward. To
address this, we take the following measures for avoiding data collection
noise. First, we limit the data collection to human subjects with a HIT rate
[2] greater than 90%. In order to further refine the quality of collected
data, we also limit the responses to users having a bachelors degree or
equivalent. Furthermore, we intentionally use a repeated seed comparison for
each sequence of paired comparisons shown to a particular user. Responses of
users who respond differently to this repeated comparison are discarded while
reporting the final results in the main paper.

##  Appendix E Inference Time Analysis

While the ability to closely mimic the human artistic creation process
determines the practical merits of a stroke based rendering approach [12, 36,
21] over pixel-based image generation methods, the time taken to generate the
same can help us better understand the practical usability of a method. In
this section, we provide an analysis comparing the inference time required for
our method with previous works [12, 36, 26, 21]. In order to facilitate a
better understanding for inference-time requirements of our approach we define
two different variations of our method.

Ours (with StrokeReg). This indicates the default Intelli-paint pipeline
consisting of a _Sequential Planner_ for human-like painting sequence
initialization and _Stroke Regularizer_ for predicting the most efficient
brushstroke decomposition for each test image $I$.

Ours (w/o StrokeReg). We note that since the stroke regularization process is
based on performing gradient descent across the entire painting trajectory, it
is considerably slower than the _Sequential Planner_ agent. Also, since
performing brushstroke compression may not be necessary for each potential
application, we define a variant of _Intelli-paint_ which only relies on
sequential predictions from the _Sequential Planner_ to learn a human-like
painting process.

Results comparing inference time between different methods are shown in Table
3. All values are computed using the official implementations from respective
authors [12, 26, 36, 21] on a single Nvidia V100 GPU. We observe that since
our method requires significantly fewer brushstrokes than Paint Transformer
[21], RL[12] and Semantic-RL [26], it performs faster than the above when not
using stroke regularization. Similarly, while both _Ours (with StrokeReg)_ and
Optim [36] require gradient descent based optimization, our method is
considerably faster since it relies on high-quality initializations from the
_sequential planner_ agent. In contrast, Optim [36] typically begins with a
random or heuristic-based brushstroke initialization which takes the gradient
descent optimization longer to converge.

Method | Inference Time (_s_)
---|---
RL [12] | 2.317 s
Semantic-RL [26] | 2.631 s
Optim [36] | 416.7 s
Transformer [21] | 1.154 s
Ours (_with StrokeReg_) | 72.21 s
Ours (_w/o StrokeReg_) |  0.948 s
Table 3: Method comparison w.r.t. inference time. We observe that since our
method requires significantly fewer brushstrokes than Paint Transformer [21],
RL[12] and Semantic-RL [26], it performs faster than the above when not using
stroke regularization. Similarly, while both _Ours (with StrokeReg)_ and Optim
[36] require gradient descent based optimization, our method is considerably
faster since it relies on high-quality initializations from the _sequential
planner_ agent. In contrast, Optim [36] typically begins with a random or
heuristic-based brushstroke initialization which takes the gradient descent
optimization longer to converge.