# Hyper-Realist Rendering: A Theoretical Framework

Ergun Akleman
Visual Computing & Computational Media, School of PVFA
Computer Science and Engineering, College of Engineering
Texas A&M University, College Station, TX, 77831
<EMAIL_ADDRESS>
Murat Kurt
International Computer Institute
Ege University
Izmir, Türkiye
<EMAIL_ADDRESS>
Derya Akleman
Department of Statistics
Texas A&M University, College Station, TX, 77831
<EMAIL_ADDRESS>
Gary Bruins
Visualization Program, Department of Architecture,
Texas A&M University, College Station, TX, 77831
<EMAIL_ADDRESS>
Sitong Deng
Department of Visualization
Texas A&M University, College Station, TX, 77831
<EMAIL_ADDRESS>
Meena Subramanian
Department of Visualization
Texas A&M University, College Station, TX, 77831
<EMAIL_ADDRESS>
Joint with Computer Science and Engineering Department.

###### Abstract

This is the first paper in a series on hyper-realist rendering. In this paper,
we introduce the concept of hyper-realist rendering and present a theoretical
framework to obtain hyper-realist images. We are using the term Hyper-realism
as an umbrella word that captures all types of visual artifacts that can evoke
an impression of reality. The hyper-realist artifacts are visual
representations that are not necessarily created by following logical and
physical principles and can still be perceived as representations of reality.
This idea stems from the principles of representational arts, which attain
visually acceptable renderings of scenes without implementing strict physical
laws of optics and materials. The objective of this work is to demonstrate
that it is possible to obtain visually acceptable illusions of reality by
employing such artistic approaches. With representational art methods, we can
even obtain an alternate illusion of reality that looks more real even when it
is not real. This paper demonstrates that it is common to create illusions of
reality in visual arts with examples of paintings by representational artists.
We propose an approach to obtain expressive local and global illuminations to
obtain these stylistic illusions with a set of well-defined and formal
methods. Our formalism is based on four principles: (1) colors and textures
must be represented by algebraically complete structures, (2) we should allow
impossible and inconsistent shapes in scene concepts, (3) we have to decompose
illumination and shading, and (4) we must build our shading functions using
statistic-based methods instead of physics-based methods. We claim that these
principles allow us to develop simple methods since we can avoid algebraic
inconsistencies. Using such simple methods, it can be possible to
significantly speed up modeling and rendering algorithms by focusing only on
essential elements that can create illusions of reality.

_Keywords_ Expressive Rendering, Global Illumination, Computer Graphics

##  1 Introduction and Motivation

The objective of Hyper-Realist Rendering is to study artistic approaches to
identify the perceptual and cognitive requirements to obtain visually
acceptable illusions of reality. Our idea stems from the principles of
representational arts that attain visually acceptable renderings of scenes
without implementing strict physical laws of optics and materials. With
representational arts, we can even obtain an alternate illusion of reality
that looks more real even when it is not real, which is usually called hyper-
realism Taylor and Head (2009); Baudrillard (1988); Wolny (2017). Our goal is
to formalize the methods of artists by turning them into a set of well-defined
rules that provide the perceptual and cognitive requirements for hyper-
realism. To reach this goal, one needs to test the validity of a set of
hypotheses that we put forward based on the methods of representational
artists. By formally identifying the requirements to obtain hyper-realistic
results, it is possible to significantly speed up modeling and rendering
algorithms for visualization by focusing only on essential elements that can
create illusions of reality.

According to Gombrich Gombrich (1960), the processes of developing artistic
methods demonstrate substantial similarity to the scientific processes, as
presented by Popper’s theory of science Popper (1995); Richmond (1994). This
similarity suggests that artistic methods can be used effectively to solve
some difficult problems Marsden et al. (1993); Edens and Potter (2007). The
advantage of artistic methods is that they provide simplified models that are
particularly suitable for solving some specific problems, similar to
scientific models. For vision-related problems, artistic methods are
particularly appropriate, since representational painting, illustration, and
sculpting all focus on the creation of an illusion of reality using intuitive
and straightforward approaches.

The integral component of hyper-realist rendering will involve the formal
identification of information that is essential for the creation of an
illusion of reality. We already have a reasonable estimate of the information
that is essential to capture based on the experience of representational
artists. Our goal in this paper is to provide a general strategy for the
development of specific hypotheses for any given artist or artistic style.
These experience-based hypotheses and models will be developed using visual
data-based statistical analysis. These hypotheses can also be evaluated with
rigorous user studies to identify the quality of mathematıcal models.

(a)  Original Painting.

(b)  Frame 15 of our animation.

(c)  Frame 110 of our animation.

Figure 1: An example of hyper-realistic emulation: An animated still life
painting with dynamically changing shadows, specular highlights, and caustics
using a moving area light source that emulates a painting shown in Figure
1(a). The goal of such emulations is to provide direct artistic control for
obtaining the desired look and feel.

The global illumination effects are the most important elements in obtaining
hyper-realism. The most well-known global illumination effects for hyper-
realism are reflections, refractions, and caustics (see figures 1 and 2). The
main problem with these global effects is that obtaining these effects is also
the hardest part of rendering computation. The difficulty comes from the fact
that none of the global illumination effects can be created by using only
local shader functions. Global illumination effects directly emerge from all
the inter-reflections of lights in a given scene. To simulate global
illumination effects, we should consider the relative positions and
orientations of all objects (including air and water) in the given scene. The
objects in a scene should usually include not only opaque solids, but also
semi-transparent solids, liquids, and gases such as flesh, water, smoke, and
even air. We also consider shadows as a part of global illumination effects,
since they cannot be computed only by local shader functions. Similarly,
scattering and subsurface scattering are also considered global effects, since
all require more than local information to calculate the final image. In all
of these cases, the interrelations between objects play an important role.

###  1.1 Basis and Rationale

Although the term hyper-realism is not commonly used, the goal of rendering in
computer graphics practice has always been to obtain visual illusions of
reality. The image generation process is well understood in current computer
graphics practice: It consists of three main stages: (1) Representation; (2)
Rendering, and (3) Compositing Hughes et al. (2014); Kahrs et al. (1996);
Foley et al. (1994); Brinkmann (2008). The representation stage involves the
creation or reconstruction of (1) camera parameters Hemayed (2003), (2) light
positions and orientations Debevec et al. (1998); Debevec (2002), (3)
orientations, positions and shapes of all objects in the scene Zhang et al.
(1999); Slabaugh et al. (2001); Hartley and Zisserman (2003); Stoykova et al.
(2007), and (4) material properties of all visible points Filip and Haindl
(2008); Kurt and Edwards (2009); Kurt (2018, 2019).

With hyper-realism, our first conceptual contribution is to demonstrate that
it is possible to significantly simplify the representations of objects and
materials by using rough approximations that can provide the same illusions.
An important implication of this simplification is that we can significantly
speed up the representation and rendering processes. Our second conceptual
contribution with the hyper-realistic rendering approach is to demonstrate
that it is possible to split rendering into two parts, illumination and
shading, by using algebraically consistent structures. An important
implication of this split is that we can move shading to the compositing stage
and directly control look-and-feel in post-processing, i.e. in the compositing
stage. The hyper-realist approach can be used successfully in two of the key
computer graphics applications: emulation and augmentation.

(a)  A Photograph used as the Background.

(b)  A typical augmentation of the photograph with a virtual rolling pin.

(c)  A hyper-realistic augmentation of the photograph with a virtual rolling
pin.

Figure 2:  A hyper-realistic augmentation example that demonstrates
demonstrates how global illumination improves overall quality. In this
particular case, the shadow and reflection of the rolling pin are added in
2(c). Reflection, in particular, becomes increasingly important during motion,
such as moving the camera or object positions in AR applications. The
augmented image in 2(c) was created by Gary Bruins in a computer animation /
digital compositing course in 2001 taught by Ergun Akleman.

  * •

Hyper-Realistic Emulations: These can be defined as a process of rendering
hyper-realistic images by simulating scenes using virtual cameras, virtual
lights, virtual geometry, and virtual materials that represent the scene. This
is a standard computer graphics rendering process that includes all virtual
reality applications. Figure 1 demonstrates the importance of global
illumination in emulation Subramanian and Akleman (2020). The original still-
life painting by Mena Subramanian contains a glass object, which can have
reflection and refraction with Fresnel effect and grapes, which can have
reflection and subsurface scattering. Therefore, an emulation must include all
global illumination effects from shadows, and subsurface scattering, to
reflections, refractions, and even caustics. For a successful emulation, we
also need to obtain painterly versions of these global effects. As shown in
this example, once we have a good estimate of scene description that can mimic
the original information reasonably well, we can use this information even for
stylistic emulation.

  * •

Hyper-Realistic Augmentations: These can be defined as a process of rendering
hyper-realistic images by adding the impact of the new objects in the scene.
The original scene is represented using proxy cameras, proxy lights, proxy
geometry, and proxy materials. The new objects are represented by virtual
geometry and virtual materials. The impact of the virtual objects on the proxy
scene (e.g. their shadows, their reflections, and their refractions) is
computed as a new image. This new image is combined with the original
background image to produce a single composite image Abad et al. (2003);
Brinkmann (2008); Mitjà et al. (2010). The applications of augmentation are
movie post-production and augmented reality. Figure 2 demonstrates the
importance of global illumination in image augmentation. In this example, a
background image (see Figure 2(a)) is combined with a foreground image of a
rolling pin Figure 2(b). The simple combination of two images that are
rendered with the same camera parameters, as shown in Figure 2(b), does not
produce a believable perceptive image. Figure 2(c) shows a composite image
with global illumination, which is more appealing. As shown in the visual
comparison of these two examples, creating appropriate shadow, reflection,
refraction, and caustic images and combining them into a background image is
critical for achieving realistic compositing.

(a)  Determining position and orientation of the tabletop.

(b)  Representation of Shapes and Positions of Objects.

(c)  Representation of Shapes and Positions of Lights.

Figure 3:  To create shadow and reflection in the composited images shown in
Figure 2, there is a need to reconstruct the proxy shapes of the countertop
3(a) and the lid 3(b); and the positions and light intensities 3(c).

###  1.2 Context & Motivation

Consider the augmentation case shown in Figure 2. Note that the process
consists of a series of at least three complicated operations: (1) production
of the shadow of the computer-generated object (s), (2) production of
reflection of the computer-generated rolling pin, and (3) production of an
appropriate image of the rolling pin. Each of these operations requires an
appropriate representation of the positions, shapes, and material properties
of all objects and lights in the background scene. This is a relatively simple
case. The real scenes usually have multiple transparent objects and there can
be multiple distinctly colored lights. Unfortunately, the inclusion of these
effects for hyper-realistic illusions is prohibitively expensive in the
current process for applications that require real-time interaction such as
Augmented Reality(AR) or Virtual Reality(VR).

The main problem with current practice for real-time applications is that the
process requires a reasonably good representation of real objects and
implementation of fundamental physical laws of optics, both of which are
prohibitively difficult and expensive. For example, the true 3D representation
process demonstrated in Figure 3(b) is not trivial, since real objects are
almost always imperfect. When we use proxy cameras, proxy lights, proxy
geometry, and proxy materials, there will always be some relatively minor
reconstruction errors that can cause significant disturbances in visual
perception. Moreover, we need information for the whole, as demonstrated in
Figure 3(c), to use image-based lighting, which may not always be possible.
Also, physically-based realistic rendering with global illumination can take a
significant amount of time. In other words, although the creation of hyper-
realist illusions is common _especially in the movie industry_ , it is still a
laborious process that requires a significant amount of computing and human
resources. This resource problem cannot be solved simply using brute-force
solutions by using faster computers and better GPUs. _There is a need for a
significant paradigm shift to provide hyper-realism in real-time applications
such as VR or AR._

##  2 Previous Work

Hyper-realist rendering can be considered to be related to non-photorealistic
rendering (NPR) in computer graphics. NPR emerged as a subfield of computer
graphics during the 1990s to produce computer-generated images that invoke the
appearance of being created "by hand" Haeberli (1990); Saito and Takahashi
(1990); Strothotte and Schlechtweg (2002); Gooch and Gooch (2001) by emulating
broad artistic styles such as outlines and silhouettes Hertzmann (1999);
Northrup and Markosian (2000); Isenberg et al. (2002), clip arts Stroila et
al. (2007), technical illustrations Gooch et al. (1998); Diepstraten et al.
(2002), pen and ink drawings Winkenbach and Salesin (1994); Salisbury et al.
(1997); Markosian et al. (1997); Deussen and Strothotte (2000); Salisbury et
al. (2023), impressionist Meier (1996); Litwinowicz (1997), implicit painting
Akleman (1998a, b), and cubist paintings Meadows and Akleman (2000); Smith et
al. (2004); Morrison and Akleman (2020); Akleman et al. (2023a), Chinese
painting Chan et al. (2002); Xu et al. (2003); Wang and Wang (2007); Li and Li
(2020), charcoals Majumder and Gopi (2002); Du and Akleman (2016), and
stippling Lu et al. (2002); Secord (2002); Martín et al. (2017); Galea et al.
(2016); as well as artistic tools and mediums such as brush strokes
Vanderhaeghe and Collomosse (2013); Yeh and Ouhyoung (2002); Hertzmann (1998);
Lin et al. (2012), watercolor Curtis et al. (1997). Convolution neural
networks are also effective for style transfer Gatys et al. (2016); Selim et
al. (2016). On the other hand, the inclusion of Global Illumination Effects in
hyper-realist and painterly emulations has not been very common in computer
graphics research.

Barycentric shading has been useful for obtaining global effects in a wide
variety of painterly styles Akleman et al. (2016). Liu created reflection
effects for a dynamic and 3D version of a Jiangnan water country painting by
the contemporary Chinese artist Yang Ming-Yi Liu and Akleman (2015). Zhao
created subsurface effects on an ocean surface for an Ayvazovski painting Yan
(2015). Deng also obtained the reflection effect on animated water in Chinese
paintings using post-processing operations Deng and Akleman (2023).
Subramanian created painterly reflection, refraction, and caustics with a
classical still-life painting of wine and glass Subramanian and Akleman
(2020). Du obtained global illumination effects with charcoal and cross-
hatching Du and Akleman (2016, 2017). Clifford created painterly reflections
with moving water in a dynamic painting emulating a Venice painting of Anne
Garney Clifford and Akleman (2020). Ross controlled the color in the shadow
regions with Bezier surfaces in a shader emulating the style of Georgia
O’Keeffe Ross and Akleman (2021). Castenada obtained the "painterly"
subsurface scattering effect for human faces Castaneda and Akleman (2017).
Global illumination effects can also be obtained with limited information only
with vector fields and images Youyou (2014a); Wang et al. (2014); Akleman et
al. (2017). Using these methods, it is also possible to obtain interactive
global illumination effects in real-time with the desired artistic styles
Akleman et al. (2023b).

##  3 Theoretical Framework

Our approach to obtaining hyper-realism is based on the physical limitations
of artists in collecting a complete knowledge of a scene. Artists do not have
complete 3D shape information for all objects in the scene. They have only
partial knowledge of geometry from a limited set of points of view. Artists
also do not know the complete reflectance distribution functions. Again, they
have only partial knowledge of reflectance distribution functions from a
limited set of points of view.

Based on a straightforward interpretation of these limitations, painting or
illustration can appear to be post-production processes that are applied to a
photograph or a rendering.111In fact, many non-photorealistic approaches, such
as Haeberli (1990); Saito and Takahashi (1990), are essentially post-
processing applications. However, painters or illustrators do not view the
scenes from only one point of view. Although they do not access full 3D shapes
of all the objects in the scene, they can still move their head to get a
better idea about the shapes. Moving their heads, they can also identify
different global effects, they can understand the shapes of the silhouettes,
they can understand the depth with parallax, etc.

In other words, although painting and illustration are essentially post-
production processes, they are based on not only "photographs" but also a
deeper knowledge of the scenes that can be dynamically observed. In some
sense, the visible world can conceptually be decomposed into regions by
artists. Each region is processed differently based on its material
properties. On the basis of this observation, we present a new conceptual
framework to produce hyper-realist results. This conceptual framework allows
us to decompose the hyper-realist rendering process into "illumination" and
"shading". In other words, we view the rendering process in two stages called
"Illumination" and "Shading". We consider illumination as a process of
building realistic and non-realistic images, such as photographic images or
paintings. These images can be relatively simple, missing rich details. On the
other hand, we view shading as a process to convert these simple or
uninteresting images into hyper-realistic, complex, and interesting images.

(a) A shape sculpted by wood.

(b) Perspective view from right vantage point creates an illusion of
impossible object.

Figure 4: This figure shows how anamorphism can be used to create illusion of
impossible objects. It is important to note that there exist infinitely many
solutions that can produce the same impossible object Elber (2011).

A key property of our framework is to use non-realistic geometry with basic
materials to compute illumination and shading to support impossible,
inconsistent, and incoherent shapes Akleman et al. (2024a, b) (see Figure 4).
In other words, these geometric shapes and basic materials are allowed to form
non-Riemannian manifolds Bujack et al. (2022); Akleman et al. (2023c). This is
a very critical property to obtain artistic results, since artistic creations
often include impossible, inconsistent, and incoherent elements. An additional
advantage of our framework is that we can use simpler geometry and materials,
since geometry and materials do not have to satisfy the Riemannian property
Gromov et al. (1999); Bujack et al. (2022).

We need to point out a caveat in this way of thinking. Almost all shapes used
in computer graphics are not Riemannian manifolds, since they are not smooth,
that is, they are not infinitely differentiable Lee (2018). For example,
triangular meshes are not really differentiable in their edges and vertices.
We need to clarify that by the Riemannian property, we mean that the property
of the distance between two points on the manifold is the length of the
shortest path that connects them. This can be uniquely defined by adding small
differences in any given geodesic Gromov et al. (1999); Bujack et al. (2022)..
This property suggests that if we travel along a path, we can return to the
original position, and only some of the small differences become zero. This
property can hold even when the shapes are not Riemannian manifolds. However,
the property must not hold in impossible, inconsistent, or incoherent shapes.
For example, consider the never-ending staircase in M. C. Escher’s famous
Ascending and Descending lithograph Schattschneider (2010). In that
lithograph, two groups of identically dressed men are continuously ascending
and descending the never-ending staircase Escher (1960). Since we cannot
return the same height by continuously climbing, this is an impossible shape
and obviously does not satisfy the Riemannian property. Note that a good
example of such structures is non-conservative vector fields Akleman et al.
(2024a). Gradient fields, on the other hand, can satisfy the Riemannian
property Akleman et al. (2024a).

Non-conservative vector fields are useful for representing shapes that do not
satisfy the Riemannian property Akleman et al. (2024a, b). Fortunately, vector
fields are not the only ones. There is another representation that can support
the non-Riemannian property. Consider the impossible object in Figure 4(b).
The shape in Figure 4(a) is just a normal shape that can be considered a
Riemannian 2-manifold if we assume that it is infinitely differentiable at the
sharp corners. On the other hand, it appears to be an impossible object 4(b)
from a vantage point. Note that this shape from the vantage point can be
considered as a height field. This height field is discontinuous in one region
and, therefore, is not-differentiable. In other words, we can also produce
impossible shapes with height fields by introducing discontinuities from a
vantage point. This corresponds to an anamorphic representation of height
fields (see Figure 5). As a result, this gives us two types of representations
that can be used for hyper-realistic illumination.

Figure 5: Anamorphism is obtained by simply projecting a 3D-looking image of
an object or scene from a vantage point to a random 3D geometry. Viewing the
projected image from the same vantage point creates the illusion of a 3D
object or scene.

Based on this viewpoint, we can start by computing illumination using such
relatively simple non-Riemannian geometries to obtain relatively simple
images. Then, by converting these relatively simple images with stylistic
reflectance distribution functions, we obtain a final hyper-realist rendering.
The process is conceptually simple if there is no constraint, i.e. if we
create our artistic images. On the other hand, if we use this process for
emulations or augmentations, the problem is an inverse problem. We need to be
careful, especially with registration problems.

For illumination computation, we need (1) simplified non-Riemannian geometric
representations of all objects that appear to be correct only from a limited
set of points of view; and (2) reflectance distribution functions that are
roughly acceptable for most invisible points. On the other hand, for shading
computation, we need some type of stylistic reflectance distribution maps
(functions) for all visible points that can convert the original illumination
images into new and more stylistic images. Fortunately, there are already some
methodologies to represent such simplified models. In the rest of the paper,
we give details of these illumination and shading stages.

(a) The top view of a simple image.

(b) Perspective view from right vantage point creates an illusion of a square
hole.

(c) The top view of an exaggerated drawing.

(d) Perspective view from right vantage point creates an illusion of a 3D cube
.

Figure 6: Examples of Anamorphic images that are obtained as screen-captures
of video instructions by Jon Harris Harris (2017a, b), who demonstrates how
easy it is to create anamorphic art even for children.

###  3.1 Illumination Stage: Rendering with Simplified non-Riemannian Geometry
& Basic Materials as Virtual and Proxy Objects

As described in the previous section, in the hyper-realist illumination
process, we represent both virtual and proxy objects with relatively simple
non-Riemannian geometries and basic materials to support impossible,
inconsistent, and incoherent structures. In this section, we briefly describe
how to obtain and use such structures. First, we point out that, in computer
graphics, all geometries and materials are essentially simplified
representations that do not necessarily support the Riemannian property.
However, the omission of the Riemannian property is usually practical and is
not necessarily intentional. For example, we do not check normal maps if they
are gradient fields or not. Most likely, many of the normal maps do not
satisfy the Riemannian property, but since they somewhat work, no one enforces
theoretical consistency. In hyper-realism, we theoretically embrace the non-
Riemannian property. In this section, we briefly discuss how to build simple
non-Riemannian geometries and materials that can support hyper-realist
emulations and augmentations.

The simplest geometries that can be used as proxy and virtual object shapes
are the Mock-3D shapes, which appear to be 3D but are essentially 2D Youyou
(2014b). In mock-3D shapes are essentially a combination of two art
approaches: anamorphic art Schulman (2020); Odeith (2024); Beever (2024) and
bas-reliefs Belhumeur et al. (1999); Weyrich et al. (2007). The advantage of
both types of representation is that they can directly support impossible,
incoherent, and inconsistent structures. In anamorphic art, the perspective
information is embedded into the picture by projecting the picture into a 3D
scene. Therefore, the anamorphic images appear to be 3D from a vantage point.
Figure 5 shows the main approach to obtain anamorphic art.

Bas-reliefs, on the other hand, are scaled-down geometries Belhumeur et al.
(1999); Weyrich et al. (2007) that are qualitatively similar to actual
geometry (see Youyou (2014b) for a discussion). Anamorphisms are often
included with bas-reliefs even in early cave art to create the illusion of 3D
Akleman et al. (2024a). We call these Mock-3D shapes anamorphic bas-reliefs.
Note that anamorphism and bas-reliefs are exactly how we can create 3D
impossible shapes in practice Elber (2011).

(a)  A flatland scene. The red circle is a light. The cyan rectangle and
yellow ellipsoid represent objects in the scene. The eye is the camera. The
line is the 1D image.

(b)  Expected shadow of a 2D CG object in Flatland scene. The CG object is the
blue circle.

(c)  1D Proxy shapes that are used to reconstruct the flatland scene. The
original light is replaced by a directional light. The original objects are
replaced by qualitative similar curves that are given as a height function.
The perspective is embedded in shapes by using a parallel projection. Note
that the image is still the same.

(d) The shadow of a CG object on proxy objects. Note that the shadow is
similar to the original shadow mainly because the proxy objects are
qualitatively similar to the original objects.

Figure 7: A 2D Flatland example that demonstrates an anamorphic bas-relief
that represents a complicated scene with qualitatively similar proxy shapes
that embed perspective transformation can provide qualitatively acceptable
global illumination effect. This example visually shows that it is possible to
obtain shadows that are reasonably close to original shadows. We do not even
reconstruct light positions accurately.

Note that bas-reliefs do not have to be represented by 3D geometry. We can
even use normal maps Johnston (2002); Okabe et al. (2006); Shao et al. (2012).
One problem with normal maps is that they are supposed to be gradient fields
and therefore they cannot theoretically represent impossible, incoherent, and
inconsistent shapes such as those used by artists such as Pablo Picasso of M.
C. Escher Akleman et al. (2024a). This is not really a major problem. Normal
maps can support non-conservative vector fields and can be used directly to
represent impossible, incoherent, and inconsistent shapes Wang et al. (2014);
Youyou (2014b). The main problem is that normal maps do not have thickness
information, which can create problems with thin objects or representing
transparent objects Akleman et al. (2024b). The thickness information can also
be embedded in normal map images, in an extended form of normal maps, called
shape maps Akleman et al. (2024a, b).

Figure 8: Compositing operation with a manipulated $W(u,v,t)$ image.

Anamorphic bas-reliefs are mock3D shapes with associated materials. An
additional advantage of anamorphism is that it makes the assignment of
materials with texture mapping simple, just a perspective projection. Such
objects can essentially be formulated as a layer that is represented as a
parametric surface that is given as $\mathbf{P}(u,v)$ where $(u,v)$ are
parametric coordinates in $[0,1]^{2}$. The simplest of such layers is a planar
rectangle, and in most cases, it is sufficient to use a planar rectangle. We
can also use tensor product B-spline surfaces or simply a wrinkled polygonal
mesh. Since these are always functions of a square domain $[0,1]^{2}$ to the
3D space $\mathbf{P}=(x,y,z)$, it is easy to texture map them. We can simply
use the parametric coordinates $(u,v)$ as texture coordinates. Without loss of
generality in the rest of the paper, we assume that it is a unit square in
$z=0$ as $\mathbf{P}(u,v)=(u,v,c)$ where $(u,v)\in[0,1]^{2}$ and $c$ is a
constant, which is usually zero.

Figure 9: Compositing operation with a manipulated $W(u,v,t)$ image. In this
case, we blurred $W(u,v,t)$ to obtain show that the result is robust. Note
that the term that corresponds to the classical (non-barycentric) diffuse
formula $T_{1}W_{3}$ is just a blurred image. On the other hand, the
barycentric formula still provides clean results.

Each of these rectangular texture layers can consist of several components,
each of which is a texture projected onto the rectangular surface. We call
these components “channels” consistent with the standard terminology used in
image manipulation. However, in this case, the term channel will refer to
entities that are more general than simple color channels. One channel
provides proxy shape information, which we call Mock3D shapes, which are
either height fields or vector fields. Height fields are represented by images
called depth maps, and vector fields are represented by images called normal
maps. The additional channels are control images that provide material
properties of the objects.

An important property of using anamorphic bas-reliefs as virtual and proxy
shapes is that embedding perspective transformations does not hurt computation
of global illumination effects. We want to demonstrate how the concept works
in a lower dimension shown in Figure 7(a). In this particular case, we provide
a scene from Flatland, that is, a 2D world Abbott (2006). A flatlander’s view
of the world is 1D, i.e. one dimension lower, as shown in this figure. Using
the same analogy, note that our view of the world is just 2D, again one
dimension lower. If we keep the boundaries the same, we can obtain the same 1D
image even using a parallel projection as shown in Figure 7(c).

To support more general scenes, mock-3D scenes can be constructed by
projecting each set of images onto 2-manifold or 2-complex meshes Akleman and
Chen (1999); Akleman et al. (2000, 2003, 2024a). We can freely deform these
shapes in the z direction to represent different versions of the same
impossible object. This is particularly useful since we can deform objects
without changing the visible parts Elber (2011). Deformations at the depth of
Z can also provide local layering McCann and Pollard (2009). Using 2-manifold
meshes is standard computer graphics practice. for 2-complexes, see Akleman et
al. (2024a).

(a) A figurative painting by Jonathan Kiker.

(b) An augmentation of the painting with glasses.

Figure 10: An example of hyper-realist augmentation of a figurative painting
Kiker (2010).

An important property of these 2-manifold meshes and 2-complexes is that they
are not unique. As long as the edges of the silhouette match, the shapes are
acceptable, such as the one shown in Figure 4(a). This property makes the
modeling process very simple. Another important property is that the textures
that describe materials can be very simple, even a simple color. In the
process of illumination, we only need some color bleeding. This even affects
the color of the shadows, which is never constant. It is also important to
estimate the approximate luminance and hue of the occluded regions Akleman et
al. (2023c).

We point out that this type of representation is useful only from a vantage
point of view. In hyper-realist rendering, we are philosophically against
spending time to obtain very precise 3D scenes. If we have a high-quality 3D
representation, they can work perfectly in this framework. On the other hand,
many of the infinitely many potential anamorphic bas-reliefs can work to
obtain basic illumination. We call this basic illumination, but, in the
illumination stage, we can compute all global illumination effects. We simply
ignore perceptually unimportant secondary effects. For instance, consider a
wall made of slightly different colors of bricks, which works as a large
reflector and illuminates the scene. In this stage, we do not need to consider
all individual bricks. Their average color simply contributes to the color
bleeding in the scene. For global illumination, we mainly need effects such as
color bleeding in the scene. On the other hand, we need local shape
information to compute local shadows Wang et al. (2014); Youyou (2014b);
Akleman et al. (2017, 2023d).

An important detail of the illumination stage is that the final rendered
image, denoted by $W$ (see $W_{0}$ and $W_{1}$ Figures 8 and 9), is created as
the total illumination of all visible points. Ignoring material properties of
visible points is necessary since $W$ images will later be used to construct
Hyper-realist images through the shading stage. If we include shader effects
in the illumination stage, the shaders will be applied twice, which is
incorrect. Ignoring material properties of visible points is straightforward
for most of the “backward” rendering algorithms, such as path tracing, since
they start with visible points. The only change in the algorithm is to ignore
the first shader effect. It is also better to separate diffuse and specular
illumination since shaders can act differently for these two 222note that
$W_{0}$ and $W_{1}$ Figures 8 and 9 include both specular and diffuse parts..

An additional computation in the illumination stage is done to compute dynamic
shaders that will be used in the shading stage. A critical idea is that, in
the illumination stage, we also need to identify shader information at any
given point. For example, consider the reflection of a brick wall we mentioned
earlier. In the shading stage, we need texture information to see individual
bricks on a mirror surface. Therefore, not only the illumination image $W$,
but also the high-quality shading parameters $T_{0}$ and $T_{1}$ in Figures 8
and 9 are computed as texture images during the illumination stage.

###  3.2 Shading Stage: Stylistic Materials in Screen Space

The concept of creating complex styles with images, which can be considered a
high level of shader, is introduced in Akleman et al. (2024b) and Akleman et
al. (2023a). In this paper, we position those ideas into a bigger framework
under the umbrella of hyper-realism. Figures 8 and 9 show the simplest use of
this shading approach. In these cases, we use the shading stage with only
diffuse materials. In this particular case. we use barycentric diffuse shaders
Akleman et al. (2016) to compute the final images. The shading stage is
critical to obtaining desired styles for hyper-realist rendering.

Figures 8 and 9 demonstrate that the quality of illumination is not that
critical to obtaining the desired style. Insofar as illumination provides an
appropriate average color in any given region, it is sufficient to use it.
These are simple weighted averages of the texture images $T_{0}$ and $T_{1}$
using the illumination image $W$. The main advantage of barycentric shading is
its robustness, i.e. the resulting rendering image is not ruined by the
problems in illumination computation. In the particular case in Figure 8
reflection on the surface of the water is calculated by assigning the
appropriate texture information to the surface of the water (see Deng and
Akleman (2023) for details). Many simple cases that only include mirror-like
water can be implemented in this way, such as a 3D version of a Jiangnan water
country painting by the contemporary Chinese artist Yang Ming-Yi as a primary
visual reference Liu and Akleman (2015). The painting mainly uses diffuse
surfaces such as Edgar Payne’s landscapes can also be obtained by similar -
i.e. relatively simple- diffuse barycentric shading Justice and Akleman
(2018).

(a) A figurative painting by Dick Davison.

(b) An augmentation of the painting with dogs.

Figure 11: Another example of hyper-realist augmentation of a figurative
painting Kiker (2010).

On the other hand, if we want to have more complex effects such as reflection
and refraction, nonlinear diffuse color change, subsurface scattering, or
caustics, bush stroke effects need more complicated barycentric shading
formulations. For example, Ross used high-dimensional Bezier functions to
emulate the style of Georgia O’Keeffe that requires non-linear color changes
in diffuse illumination Ross and Akleman (2021). Another set of difficult
examples of landscape paintings. For example, the sea paintings of Ivan
Aivazovsky include a high-quality subsurface scattering with water that
requires a more complicated barycentric shader Yan (2015). The classical still
life paintings require a barycentric shader that can support painterly
reflection, refraction, Fresnel effects and caustics Subramanian and Akleman
(2020). Moreover, if we want to obtain charcoal, hatching, or cross-hatching,
we need to use a barycentric shader with a zero degree B-spline Du and Akleman
(2017, 2016).

Regardless of complexity, shaders should support an algebraic structure to
obtain all the colors that are necessary to obtain the desired style.
Barycentric shaders are desired since they provide such an algebraic structure
Akleman et al. (2016). However, we observe that barycentric methods are not
the only ones that obtain algebraic structures. For example, it is also
possible to represent colors with complex numbers to support an algebraically
consistent structure Akleman et al. (2023c).

##  4 Conclusion and Discussion

Despite significant technical advances in real-time applications, obtaining
high-quality visuals is still not practical in most real-time scenarios. Most
of the current interactive real-time experiences present visuals that lack the
basic elements necessary to obtain a realistic perception of a scene (similar
to the example shown in Figure 2(b)). Our goal is to eventually achieve hyper-
realism in real-time by presenting visuals that are of a quality comparable to
the image shown in Figure 2(c), including all global illumination effects from
shadows to reflection, refraction, and caustics with art-directable visuals.
These effects are critical to improving user cognition and performance while
providing non-ambiguous visual perceptions of reality, as even a small detail
such as the position of a shadow can alter our perception of the environment.

We claim that representational art-based approaches provide the type of
simplified methods to obtain non-ambiguous visual perception in real-time
applications. In current practice, the predominant use of hyper-realism is in
the motion pictures industry and is achieved through a laborious and time-
consuming post-production process. These hyper-realist methods can still keep
humans in the loop and provide a rich toolbox for developers, designers, and
artists to fulfill distinct visual goals. This toolbox will enable users to
(1) re-purpose the background scene to improve an AR experience for individual
needs, and (2) achieve effects that are essential for visual perception,
including critical global illumination components such as shadows, reflection,
refraction, and caustics.

Our primary achievement in this paper is to provide a theoretical framework to
simplify shapes and materials to support hyper-realism in real-time
interactive applications such as VR or AR. Our secondary goal is to understand
how to tailor these shapes and material representations to satisfy any given
visual goal, such as providing informative visualizations to medical
researchers. In this paper, we presented such a formal framework for obtaining
hyper-realistic visual illusions. Quantitative and qualitative insights based
on this framework can be extremely useful for the construction of targeted
visual goals. Our framework emulates the experience of artists. It goes beyond
photography, but we still base our framework on the limitations of humans in
capturing the 3D world. Our observation is that artists despite this
limitation create impressive work. Therefore, we want to leverage this
limitation by representing shapes and material parameters in a very liberal
way to obtain acceptable hyper-realist results.

Figure 12: Augmenting the rendering of a complicated object using a mix use of
styles and removing some details Stanley (2005).

##  5 Future Work

The main advantage of our approach comes from the fact that we do not need to
have very precise representations to obtain a wide variety of styles. Since
our approach is not physically based333except computation of reflections
during illumination stage, which we can completely ignore for shape and
material estimation., we can view the creation of shapes and materials as an
inverse problem using only statistical or mathematical methods Öztürk et al.
(2006); Kurt et al. (2010); Kurt and Edwards (2009); Bilgili et al. (2011);
Ward et al. (2014); Kurt and Cinsdikici (2008); Öztürk et al. (2010); Szécsi
et al. (2010); Bilgili et al. (2012); Ergun et al. (2012); Kurt et al. (2013);
Güngör and Kurt (2014); Kurt (2017); Tongbuasirilai et al. (2017); Kurt (2018,
2019, 2020); Tongbuasirilai et al. (2020); Kurt (2021); Mir et al. (2022); Gök
et al. (2023).

Having algebraic frameworks also simplifies the inverse problems we face for
emulations and augmentations. For example, similar to Guernera et al. Guarnera
et al. (2020), we can estimate the material parameters of figurative
paintings. This estimation of material parameters can be done by computing
method difference images between reference figurative paintings and rendered
images. By optimizing material parameters, we try to minimize the Peak signal-
to-noise ratio (PSNR) value, which is computed between reference figurative
paintings and rendered images.

###  5.1 Hyper-realist Emulations

The main application of hyper-realist emulations is to turn figurative
paintings into dynamic structures. We are planning to prepare a paper on
obtaining dynamic figurative paintings since figurative paintings are the
motivation for hyper-realist renderings. Painters create figurative paintings
mainly based on visual observations of reality. Their methods do not
necessarily come from physical reality. It is important to note that long
before understanding the physical rules, artists were able to capture the
illusions of reality. Understanding and representing a wide range of
figurative paintings can help develop practical implementations of the
approach presented in this paper.

Hyper-realist emulations can also be used to emulate photographs. These
emulations can be useful to create virtual versions of reality in which we can
create alternate realities by changing the light positions. Anamorphic bas-
reliefs can be acceptable beyond vantage points. However, they are still
acceptable from a limited point of view. If we need to have significant
displacement from a single vantage point, it is good to have multiple vantage
points and interpolate between them. Since the shapes are essentially 2D, it
can be possible to use 2D interpolation Sederberg and Greenwood (1992).

###  5.2 Hyper-Realist Augmentations

Figurative paintings can also be augmented using the presented method. In this
case, we start by emulating a figurative painting by creating proxy geometry
and proxy materials. Then, we can add any new object to the scene using the
same framework Kiker (2010); Stanley (2005). Figures 11 and 10(a) show two
examples of hyper-realist augmentations of figurative paintings.

Another application of hyper-realistic augmentations is the creation of
medical or scientific illustrations. In such illustrations, the main goal is
to provide easily understandable visual stories through images that look
realistic. Scientists and doctors often realize, for example, that photographs
or realistic illustrations do not tell the story. These photographs and/or
illustrations need to be turned into storied visuals through hyper-realist
augmentation. The main problem in these applications is to remove unnecessary
details to control gaze as shown in Figure 12 Stanley (2005). Note that our
approach can also work with complete 3D data. Therefore, for medical or
scientific illustrations, we use existing shape data in form of 3D models,
magnetic resonance imaging (MRI) or X-ray form.

An additional application of hyper-realistic augmentations is real-time
augmentations in historical sites by adding virtual shapes to replace missing
objects such as sculptures, columns, and buildings. Using sunlight directions,
it is possible to add virtual shadows of virtual objects that match existing
ones. In such applications, hyper-realism will also be essential. We will also
present a paper discussing the details of this hyper-realist augmentation
approach.

###  5.3 Validations

This formal framework can be validated with rigorous user studies and careful
statistical analysis. This paper provided the theoretical framework with some
examples of proof-of-idea. We will continue to build the algorithmic
foundations for this theoretical framework. We are not planning to conduct
user studies ourselves, but we are interested in supporting other researchers
in evaluating these algorithms through controlled studies of human subjects
using static, dynamic, and interactive visualizations. We expect that there
will be static and dynamic experiments that employ side-by-side comparisons of
images and animations that are created using physically accurate versus our
proposed simplified algorithmic processes. Based on our methods, it is also
possible to develop interactive experiments to test the importance of global
illumination for task completion. Fortunately, current ray-tracing-based
technologies can support some simple global illumination effects, such as
shadows, simple reflection, and refraction, in real-time virtual reality
settings. The availability of simple effects in real-time virtual reality (VR)
is an opportunity to design interactive experiments to study the perceptual
effects of simple global illumination. However, we expect these simple effects
to be still out of reach for AR applications. Our findings will be useful in
the future to obtain interactive AR in the future. It will also be useful to
improve the performance of virtual reality applications. Moreover, more
complex effects, such as radiosity or caustics, are beyond the reach of even
VR applications. We will study the perceptual effects of more complex global
illumination effects only through images and animations.

## References

  * Taylor and Head [2009] John Russell Taylor and Clive Head.  _Exactitude: hyperrealist art today_.  Thames & Hudson, 2009.
  * Baudrillard [1988] Jean Baudrillard.  The hyper-realism of simulation.  _Jean Baudrillard: selected writings_ , pages 143–147, 1988.
  * Wolny [2017] Ryszard W Wolny.  Hyperreality and simulacrum: Jean baudrillard and european postmodernism.  _European Journal of Interdisciplinary Studies_ , 3(3):76–80, 2017.
  * Gombrich [1960] Ernst Hans Gombrich.  _Art and Illusion: A Study in the Psychology of Pictorial Representation_.  Phaïdon Press, 1960.
  * Popper [1995] Karl Popper.  _Karl Popper: philosophy and problems_ , volume 39.  Cambridge University Press, 1995.
  * Richmond [1994] Sheldon Saul Richmond.  _Aesthetic criteria: Gombrich and the philosophies of science of Popper and Polanyi_ , volume 6.  Rodopi, 1994.
  * Marsden et al. [1993] SS Antman JE Marsden, L Sirovich S Wiggins, L Glass, RV Kohn, and SS Sastry.  _Interdisciplinary Applied Mathematics_ , volume 3.  Springer, 1993.
  * Edens and Potter [2007] Kellah Edens and Ellen Potter.  The relationship of drawing and mathematical problem solving: Draw for math tasks.  _Studies in Art Education_ , 48(3):282–298, 2007.
  * Hughes et al. [2014] John F Hughes, Andries Van Dam, James D Foley, Morgan McGuire, Steven K Feiner, David F Sklar, and Kurt Akeley.  _Computer graphics: principles and practice_.  Pearson Education, 2014.
  * Kahrs et al. [1996] John Kahrs, Sharon Calahan, Dave Carson, and Stephen Poster.  Pixel cinematography: A lighting approach for computer graphics.  _SIGGRAPH course notes_ , 1996.
  * Foley et al. [1994] James D Foley, Andries Van Dam, Steven K Feiner, John F Hughes, and Richard L Phillips.  _Introduction to computer graphics_ , volume 55.  Addison-Wesley Reading, 1994.
  * Brinkmann [2008] Ron Brinkmann.  _The art and science of digital compositing: techniques for visual effects, animation and motion graphics_.  Morgan Kaufmann, 2008.
  * Hemayed [2003] Elsayed E Hemayed.  A survey of camera self-calibration.  In _Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003._ , pages 351–357. IEEE, 2003.
  * Debevec et al. [1998] Paul Debevec, Steven Gortler, Leonard McMillan, Richard Szeliski, and Chris Bregler.  Image-based modeling and rendering.  _SIGGRAPH 98 Course Notes for Course_ , 15, 1998.
  * Debevec [2002] Paul Debevec.  Image-based lighting.  _IEEE Computer Graphics and Applications_ , 22(2):26–34, 2002.
  * Zhang et al. [1999] Ruo Zhang, Ping-Sing Tsai, James Edwin Cryer, and Mubarak Shah.  Shape-from-shading: a survey.  _IEEE transactions on pattern analysis and machine intelligence_ , 21(8):690–706, 1999.
  * Slabaugh et al. [2001] Greg Slabaugh, Ron Schafer, Tom Malzbender, and Bruce Culbertson.  A survey of methods for volumetric scene reconstruction from photographs.  In _Volume Graphics 2001_ , pages 81–100. Springer, 2001.
  * Hartley and Zisserman [2003] Richard Hartley and Andrew Zisserman.  _Multiple view geometry in computer vision_.  Cambridge university press, 2003.
  * Stoykova et al. [2007] Elena Stoykova, A Ayd, Philip Benzie, Nikos Grammalidis, Sotiris Malassiotis, Joern Ostermann, Sergej Piekh, Ventseslav Sainov, Christian Theobalt, and Thangavel Thevar.  3-d time-varying scene capture technologies—a survey.  _IEEE Transactions on Circuits and Systems for Video Technology_ , 17(11):1568–1586, 2007.
  * Filip and Haindl [2008] Jiři Filip and Michal Haindl.  Bidirectional texture function modeling: A state of the art survey.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 31(11):1921–1940, 2008.
  * Kurt and Edwards [2009] Murat Kurt and Dave Edwards.  A survey of brdf models for computer graphics.  _ACM SIGGRAPH Computer Graphics_ , 43(2):1–7, 2009.
  * Kurt [2018] Murat Kurt.  A survey of bsdf measurements and representations.  _Journal of Science and Engineering_ , 20(58):87–102, January 2018.  ISSN 1302-9304.
  * Kurt [2019] Murat Kurt.  Real-time shading with phong brdf model.  _Journal of Science and Engineering_ , 21(63), September 2019.  ISSN 1302-9304.
  * Subramanian and Akleman [2020] Meena Subramanian and Ergun Akleman.  A painterly rendering approach to create still-life paintings with dynamic lighting.  In _ACM SIGGRAPH 2020 Posters_ , pages 1–2. ACM SIGGRAPH, 2020.
  * Abad et al. [2003] Francisco Abad, Emilio Camahort, and Roberto Vivó.  Integrating synthetic objects into real scenes.  _Computers & Graphics_, 27(1):5–17, 2003.
  * Mitjà et al. [2010] Carles Mitjà, Toni Bover, Miquel Bigas, and Jaume Escofet.  Matching rendered and real world images by digital image processing.  In _Optics, Photonics, and Digital Technologies for Multimedia Applications_ , volume 7723, page 77230D. International Society for Optics and Photonics, 2010.
  * Haeberli [1990] Paul Haeberli.  Paint by numbers: Abstract image representations.  In _Proceedings of the 17th annual conference on Computer graphics and interactive techniques_ , pages 207–214, 1990.
  * Saito and Takahashi [1990] Takafumi Saito and Tokiichiro Takahashi.  Comprehensible rendering of 3-d shapes.  In _Proceedings of the 17th annual conference on Computer graphics and interactive techniques_ , pages 197–206, 1990.
  * Strothotte and Schlechtweg [2002] Thomas Strothotte and Stefan Schlechtweg.  _Non-Photorealistic Computer Graphics: Modeling, Rendering, and Animation_.  Morgan Kaufmann, 2002.
  * Gooch and Gooch [2001] Bruce Gooch and Amy Gooch.  _Non-photorealistic rendering_.  AK Peters, Ltd., 2001.
  * Hertzmann [1999] Aaron Hertzmann.  Introduction to 3D non-photorealistic rendering: Silhouettes and outlines.  _Non-Photorealistic Rendering. SIGGRAPH 99 Course Notes_ , 1999.
  * Northrup and Markosian [2000] JD Northrup and Lee Markosian.  Artistic silhouettes: A hybrid approach.  In _Proceedings of the 1st international symposium on Non-photorealistic animation and rendering_ , pages 31–37, 2000.
  * Isenberg et al. [2002] Tobias Isenberg, Nick Halper, and Thomas Strothotte.  Stylizing silhouettes at interactive rates: From silhouette edges to silhouette strokes.  _Computer Graphics Forum_ , 21(3):249–258, 2002.
  * Stroila et al. [2007] Matei Stroila, Elmar Eisemann, and John Hart.  Clip art rendering of smooth isosurfaces.  _IEEE Transactions on Visualization and Computer Graphics_ , 14(1):135–145, 2007.
  * Gooch et al. [1998] Amy Gooch, Bruce Gooch, Peter Shirley, and Elaine Cohen.  A non-photorealistic lighting model for automatic technical illustration.  In _Proceedings of the 25th annual conference on Computer graphics and interactive techniques_ , pages 447–452. ACM, 1998.
  * Diepstraten et al. [2002] Joachim Diepstraten, Daniel Weiskopf, and Thomas Ertl.  Transparency in interactive technical illustrations.  _Computer Graphics Forum_ , 21(3):317–325, 2002.
  * Winkenbach and Salesin [1994] Georges Winkenbach and David H Salesin.  Computer-generated pen-and-ink illustration.  In _Proceedings of the 21st annual conference on Computer graphics and interactive techniques_ , pages 91–100. ACM, 1994.
  * Salisbury et al. [1997] Michael P Salisbury, Michael T Wong, John F Hughes, and David H Salesin.  Orientable textures for image-based pen-and-ink illustration.  In _Proceedings of the 24th annual conference on Computer graphics and interactive techniques_ , pages 401–406, 1997.
  * Markosian et al. [1997] Lee Markosian, Michael A Kowalski, Daniel Goldstein, Samuel J Trychin, John F Hughes, and Lubomir D Bourdev.  Real-time nonphotorealistic rendering.  In _Proceedings of the 24th annual conference on Computer graphics and interactive techniques_ , pages 415–420. ACM Press/Addison-Wesley Publishing Co., 1997.
  * Deussen and Strothotte [2000] Oliver Deussen and Thomas Strothotte.  Computer-generated pen-and-ink illustration of trees.  In _Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques_ , pages 13–18. ACM Press/Addison-Wesley Publishing Co., 2000.
  * Salisbury et al. [2023] Michaet P Salisbury, Sean E Anderson, Ronen Barzel, and David H Satesin.  Interactive pen-and-ink illustration.  In _Seminal Graphics Papers: Pushing the Boundaries, Volume 2_ , pages 393–400. ACM SIGGRAPH, 2023.
  * Meier [1996] Barbara J Meier.  Painterly rendering for animation.  In _Proceedings of the 23rd annual conference on Computer graphics and interactive techniques_ , pages 477–484. ACM, 1996.
  * Litwinowicz [1997] Peter Litwinowicz.  Processing images and video for an impressionist effect.  In _Proceedings of the 24th annual conference on Computer graphics and interactive techniques_ , pages 407–414. ACM Press/Addison-Wesley Publishing Co., 1997.
  * Akleman [1998a] Ergun Akleman.  Implicit painting of CSG solids.  In _Proc. of CSG_ , volume 98, pages 99–113, 1998a.
  * Akleman [1998b] Ergun Akleman.  Implicit surface painting.  In _Proc. of Implicit Surfaces’ 98_ , pages 63–68, 1998b.
  * Meadows and Akleman [2000] S. Meadows and E. Akleman.  Abstract digital paintings created with painting camera technique.  _Proc. D’ART 2000 / Information Visualization 2000_ , 2000.
  * Smith et al. [2004] Jeffrey Smith, Ergun Akleman, Richard Davison, John Keyser, et al.  Multicam: A system for interactive rendering of abstract digital images.  In _Bridges: Mathematical Connections in Art, Music, and Science_ , pages 265–272. Bridges Conference, 2004.
  * Morrison and Akleman [2020] Chris Morrison and Ergun Akleman.  Remote empathetic viewpoint: A novel approach to extending cubism.  In _ACM SIGGRAPH 2020 Posters_ , pages 1–2. ACM SIGGRAPH, 2020.
  * Akleman et al. [2023a] Ergun Akleman, Cassie Mullins, Christopher Morrison, and David Oh.  Recursive camera painting: A method for real-time painterly renderings of 3d scenes, 2023a.
  * Chan et al. [2002] Ching Chan, Ergun Akleman, and Jianer Chen.  Two methods for creating chinese painting.  In _10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings._ , pages 403–412. IEEE, 2002.
  * Xu et al. [2003] Songhua Xu, Francis CM Lau, Feng Tang, and Yunhe Pan.  Advanced design for a realistic virtual brush.  _Computer Graphics Forum_ , 22(3):533–542, 2003.
  * Wang and Wang [2007] Chung-Ming Wang and Ren-Jie Wang.  Image-based color ink diffusion rendering.  _IEEE Transactions on Visualization and Computer Graphics_ , 13(2):235–246, 2007.
  * Li and Li [2020] Fengjiao Li and Shaobin Li.  real-time rendering of 3d animal models in chinese ink painting style.  In _2020 International Conference on Culture-oriented Science & Technology (ICCST)_, pages 284–287, New York, NW, 2020. IEEE, IEEE.
  * Majumder and Gopi [2002] Aditi Majumder and M Gopi.  Real time charcoal rendering using contrast enhancement operators.  In _Proceedings of Symposium of Non Photorealistic Animation and Rendering_ , 2002.
  * Du and Akleman [2016] Yuxiao Du and Ergun Akleman.  Charcoal rendering and shading with reflections.  In _ACM SIGGRAPH 2016 Posters_ , SIGGRAPH ’16, pages 32:1–32:2. ACM, 2016.  ISBN 978-1-4503-4371-8.  doi:10.1145/2945078.2945110.  URL http://doi.acm.org/10.1145/2945078.2945110.
  * Lu et al. [2002] Aidong Lu, Christopher J Morris, David S Ebert, Penny Rheingans, and Charles Hansen.  Non-photorealistic volume rendering using stippling techniques.  In _Proceedings of the Conference on Visualization ’02_ , pages 211–218. IEEE Computer Society, 2002.
  * Secord [2002] Adrian Secord.  Weighted voronoi stippling.  In _Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering_ , pages 37–43, 2002.
  * Martín et al. [2017] Domingo Martín, Germán Arroyo, Alejandro Rodríguez, and Tobias Isenberg.  A survey of digital stippling.  _Computers & Graphics_, 67:24–44, 2017.
  * Galea et al. [2016] Brendan Galea, Ehsan Kia, Nicholas Aird, and Paul G Kry.  Stippling with aerial robots.  In _Proceedings of the Joint Symposium on Computational Aesthetics and Sketch Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering_ , pages 125–134, 2016.
  * Vanderhaeghe and Collomosse [2013] David Vanderhaeghe and John Collomosse.  Stroke based painterly rendering.  In _Image and Video-Based Artistic Stylisation_ , pages 3–21. Springer, 2013.
  * Yeh and Ouhyoung [2002] Jun-Wei Yeh and Ming Ouhyoung.  Non-photorealistic rendering in Chinese painting of animals.  _Journal of System Simulation_ , 14(6):1220–1224, 2002.
  * Hertzmann [1998] Aaron Hertzmann.  Painterly rendering with curved brush strokes of multiple sizes.  In _Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques_ , pages 453–460. ACM, 1998.
  * Lin et al. [2012] Liang Lin, Kun Zeng, Yizhou Wang, Ying-Qing Xu, and Song-Chun Zhu.  Video stylization: Painterly rendering and optimization with content extraction.  _IEEE Transactions on Circuits and Systems for Video Technology_ , 23(4):577–590, 2012.
  * Curtis et al. [1997] Cassidy J Curtis, Sean E Anderson, Joshua E Seims, Kurt W Fleischer, and David H Salesin.  Computer-generated watercolor.  In _Proceedings of the 24th annual conference on Computer graphics and interactive techniques_ , pages 421–430. ACM Press/Addison-Wesley Publishing Co., 1997.
  * Gatys et al. [2016] Leon A Gatys, Alexander S Ecker, and Matthias Bethge.  Image style transfer using convolutional neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2414–2423, 2016.
  * Selim et al. [2016] Ahmed Selim, Mohamed Elgharib, and Linda Doyle.  Painting style transfer for head portraits using convolutional neural networks.  _ACM Transactions on Graphics (ToG)_ , 35(4):1–18, 2016.
  * Akleman et al. [2016] Ergun Akleman, S Liu, and Donald House.  Barycentric shaders: Art directed shading using control images.  In _Proceedings of the Joint Symposium on Computational Aesthetics and Sketch Based Interfaces and Modeling and Non-Photorealistic Animation and Rendering_ , pages 39–49, Geneva, Switzerland, 2016. Eurographics Association.
  * Liu and Akleman [2015] Siran Liu and Ergun Akleman.  Chinese ink and brush painting with reflections.  In _SIGGRAPH 2015: Studio_ , SIGGRAPH ’15, pages 8:1–8:1, New York, NY, USA, 2015. ACM.  ISBN 978-1-4503-3637-6.  doi:10.1145/2785585.2792525.  URL http://doi.acm.org/10.1145/2785585.2792525.
  * Yan [2015] Zhao Yan.  Painterly shading ocean surface.  Master’s thesis, Texas A&M University, College Station, TX, 2015.  Retrieved from http://oaktrust.library.tamu.edu/handle/1969.1/15638.
  * Deng and Akleman [2023] Sitong Deng and Ergun Akleman.  A digital compositing approach to obtain animated chinese still-life paintings with global effects, 2023.
  * Du and Akleman [2017] Yuxiao Du and Ergun Akleman.  Designing look-and-feel using generalized crosshatching.  In _ACM SIGGRAPH 2017 Talks_ , page 40. ACM, 2017.
  * Clifford and Akleman [2020] Jack Clifford and Ergun Akleman.  A barycentric shading process to create dynamic paintings in contemporary fauvist-expressionist style with reflection.  In _Proceedings of Eurasia Graphics 2021_ , pages 32–39, Ankara, Turkey, 2020. Eurasia Graphics.
  * Ross and Akleman [2021] Jessica Ross and Ergun Akleman.  Georgia o’keeffe shader: An approach to obtain shaders to simulate a given painterly style.  In _Proceedings of Eurasia Graphics 2022_ , pages 27–34, Ankara, Turkey, 2021. Eurasia Graphics.
  * Castaneda and Akleman [2017] Saif Castaneda and Ergun Akleman.  Shading with painterly filtered layers: A technique to obtain painterly portrait animations.  In _Proceedings of the Symposium on Non-Photorealistic Animation and Rendering_ , page 8, New York City, NW, 2017. ACM.
  * Youyou [2014a] Wang Youyou.  _Qualitative Global Illumination of Mock-3D Scenes_.  PhD thesis, Texas A&M University, College Station, TX, 2014a.  Retrieved from http://oaktrust.library.tamu.edu/handle/1969.1/157921.
  * Wang et al. [2014] Youyou Wang, Ozgur Gonen, and Ergun Akleman.  Global illumination for 2d artworks with vector field rendering.  In _ACM SIGGRAPH 2014 Posters_ , page 95. ACM, 2014.
  * Akleman et al. [2017] Ergun Akleman, Fermi Perumal, and Youyou Wang.  Cos $\theta$ shadows: an integrated model for direct illumination, subsurface scattering and shadow computation.  In _ACM SIGGRAPH 2017 Posters_ , page 38. ACM, 2017.
  * Akleman et al. [2023b] Ergun Akleman, Youyou wang, Yinan Xiong, Anusha Shanker, Fermi Perumal, Ozgur Gonen, and Motahareh Fard.  Web-based dynamic paintings: Real-time interactive artworks in web using a 2.5d pipeline, 2023b.
  * Elber [2011] Gershon Elber.  Smi 2011: Full paper: Modeling (seemingly) impossible models.  _Comput. Graph._ , 35(3):632–638, June 2011.
  * Akleman et al. [2024a] Ergun Akleman, Youyou Wang, and Ozgur Gonen.  Representing and modeling inconsistent, impossible, and incoherent shapes and scenes with 2d non-conservative vector fields mapped on 2-complexes, 2024a.
  * Akleman et al. [2024b] Ergun Akleman, Youyou Wang, and Ozgur Gonen.  Compositing with 2d vector fields by using shape maps that can represent inconsistent, impossible, and incoherent shapes, 2024b.
  * Bujack et al. [2022] Roxana Bujack, Emily Teti, Jonah Miller, Elektra Caffrey, and Terece L Turton.  The non-riemannian nature of perceptual color space.  _Proceedings of the National Academy of Sciences_ , 119(18):e2119753119, 2022.
  * Akleman et al. [2023c] Ergun Akleman, Shubham Agarwall, Donald H. House, and Tolga Talha Yildiz.  Circular average filtering and circular linear interpolation in complex color spaces, 2023c.
  * Gromov et al. [1999] Mikhael Gromov, Misha Katz, Pierre Pansu, and Stephen Semmes.  _Metric structures for Riemannian and non-Riemannian spaces_ , volume 152.  Springer, 1999.
  * Lee [2018] John M Lee.  _Introduction to Riemannian manifolds_ , volume 2.  Springer, 2018.
  * Schattschneider [2010] Doris Schattschneider.  The mathematical side of mc escher.  _Notices of the AMS_ , 57(6):706–718, 2010.
  * Escher [1960] Maurits Cornelis Escher.  Ascending and descending.  https://en.wikipedia.org/wiki/Ascending_and_Descending, 1960.
  * Harris [2017a] Jon Harris.  Drawing a 3d anamorphic cube - easy trick art optical illusion.  https://www.youtube.com/watch?app=desktop&v=3-VqyETSD9o, 2017a.
  * Harris [2017b] Jon Harris.  How to draw an easy anamorphic hole for kids - trick art on paper.  https://www.youtube.com/watch?app=desktop&v=B9Ke7GvPCVc, 2017b.
  * Youyou [2014b] Wang Youyou.  _Qualitative Global Illumination of Mock-3D Scenes_.  PhD thesis, Texas A&M University, College Station, TX, 2014b.  Retrieved from http://oaktrust.library.tamu.edu/handle/1969.1/157921.
  * Schulman [2020] Sandra Schulman.  Anamorphic murals elevate the street art of hector diaz.  https://www.wpbmagazine.com/anamorphic-murals-elevate-street-art-hector-diaz/ and www.artistdiaz.com, 2020.
  * Odeith [2024] Sergio Odeith.  Anamorphic street art by sergio odeith.  https://www.odeith.com/walls/, 2024.
  * Beever [2024] Julian Beever.  Pavement art – 3d illusions.  https://www.julianbeever.net/pavement-art-3d-illusions/, 2024.
  * Belhumeur et al. [1999] Peter N Belhumeur, David J Kriegman, and Alan L Yuille.  The bas-relief ambiguity.  _International journal of computer vision_ , 35(1):33–44, 1999.
  * Weyrich et al. [2007] Tim Weyrich, Jia Deng, Connelly Barnes, Szymon Rusinkiewicz, and Adam Finkelstein.  Digital bas-relief from 3d scenes.  _ACM transactions on graphics (TOG)_ , 26(3):32–es, 2007.
  * Johnston [2002] Scott F. Johnston.  Lumo: illumination for cel animation.  In _Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering_ , NPAR ’02, pages 45–52, 2002.
  * Okabe et al. [2006] Makoto Okabe, Gang Zeng, Yasuyuki Matsushita, Takeo Igarashi, Long Quan, and Heung-Yeung Shum.  Single-view relighting with normal map painting.  _Proceedings of Pacific Graphics_ , pages 27–34, 2006.
  * Shao et al. [2012] Cloud Shao, Adrien Bousseau, Alla Sheffer, and Karan Singh.  Crossshade: shading concept sketches using cross-section curves.  _ACM Trans. Graph._ , 31(4):45:1–45:11, 2012.
  * Abbott [2006] Edwin A Abbott.  _Flatland: A romance of many dimensions_.  OUP Oxford, 2006.
  * Akleman and Chen [1999] Ergun Akleman and Jianer Chen.  Guaranteeing the 2-manifold property for meshes with doubly linked face list.  _International Journal of Shape Modeling_ , 5(02):159–177, 1999.
  * Akleman et al. [2000] Ergun Akleman, Jianer Chen, and Vinod Srinivasan.  A new paradigm for changing topology of 2-manifold polygonal meshes.  In _Pacific Graphics_ , volume 2000, pages 192–201, 2000.
  * Akleman et al. [2003] Ergun Akleman, Jianer Chen, and Vinod Srinivasan.  A minimal and complete set of operators for the development of robust manifold mesh modelers.  _Graphical models_ , 65(5):286–304, 2003.
  * McCann and Pollard [2009] James McCann and Nancy Pollard.  Local layering.  _ACM Transactions on Graphics (TOG)_ , 28(3):1–7, 2009.
  * Kiker [2010] Jonathan Kiker.  _Using local information for compositing CG into traditional art_.  PhD thesis, Texas A & M University, 2010.
  * Akleman et al. [2023d] Ergun Akleman, Youyou wang, Yinan Xiong, Anusha Shanker, Fermi Perumal, Ozgur Gonen, and Motahareh Fard.  Web-based dynamic paintings: Real-time interactive artworks in web using a 2.5d pipeline, 2023d.
  * Justice and Akleman [2018] Matthew Justice and Ergun Akleman.  A process to create dynamic landscape paintings using barycentric shading with control paintings.  In _ACM SIGGRAPH 2018 Posters_ , page 30, New York City, NW, 2018. ACM.
  * Stanley [2005] Michael Leighton Stanley.  _Digital compositing with traditional artwork_.  PhD thesis, Texas A&M University, 2005.
  * Öztürk et al. [2006] Aydin Öztürk, Ahmet Bilgili, and Murat Kurt.  Polynomial approximation of blinn-phong model.  In Louise M. Lever and Mary McDerby, editors, _Proceedings of the 4th Theory and Practice of Computer Graphics_ , TPCG ’06, pages 55–61, Middlesbrough, United Kingdom, 2006. Eurographics Association.  ISBN 3-905673-59-2.  doi:10.2312/LocalChapterEvents/TPCG/TPCG06/055-061.  URL http://www.eg.org/EG/DL/LocalChapterEvents/TPCG/TPCG06/055-061.pdf.
  * Kurt et al. [2010] Murat Kurt, László Szirmay-Kalos, and Jaroslav Křivánek.  An anisotropic brdf model for fitting and monte carlo rendering.  _ACM SIGGRAPH Computer Graphics_ , 44(1):1–15, 2010.
  * Bilgili et al. [2011] Ahmet Bilgili, Aydn Öztürk, and Murat Kurt.  A general brdf representation based on tensor decomposition.  _Computer Graphics Forum_ , 30(8):2427–2439, 2011.
  * Ward et al. [2014] Greg Ward, Murat Kurt, and Nicolas Bonneel.  Reducing anisotropic bsdf measurement to common practice.  In _Material Appearance Modeling_ , pages 5–8, 2014.
  * Kurt and Cinsdikici [2008] Murat Kurt and Muhammed Gökhan Cinsdikici.  Representing brdfs using soms and mans.  _ACM SIGGRAPH Computer Graphics_ , 42(3):1–18, 2008.
  * Öztürk et al. [2010] Aydın Öztürk, Murat Kurt, and Ahmet Bilgili.  A copula-based brdf model.  _Computer Graphics Forum_ , 29(6):1795–1806, September 2010.
  * Szécsi et al. [2010] László Szécsi, László Szirmay-Kalos, Murat Kurt, and Balázs Csébfalvi.  Adaptive sampling for environment mapping.  In _Proceedings of the 26th Spring Conference on Computer Graphics_ , SCCG ’10, pages 69–76, New York, NY, USA, 2010. ACM.  ISBN 978-1-4503-0558-7.  doi:http://doi.acm.org/10.1145/1925059.1925073.  URL http://doi.acm.org/10.1145/1925059.1925073.
  * Bilgili et al. [2012] Ahmet Bilgili, Aydın Öztürk, and Murat Kurt.  Representing brdf by wavelet transformation of pair-copula constructions.  In Roman Ďurikovič and Carol O’Sullivan, editors, _Proceedings of Spring Conference on Computer Graphics_ , pages 71–77, Budmerice, Szlovákia, 2012. Comenius University Press.
  * Ergun et al. [2012] Serkan Ergun, Murat Kurt, and Aydın Öztürk.  Real-time kd-tree based importance sampling of environment maps.  In Roman Ďurikovič and Carol O’Sullivan, editors, _Proceedings of Spring Conference on Computer Graphics_ , pages 84–91, Budmerice, Szlovákia, 2012. Comenius University Press.
  * Kurt et al. [2013] Murat Kurt, Aydın Öztürk, and Pieter Peers.  A compact tucker-based factorization model for heterogeneous subsurface scattering.  In Silvester Czanner and Wen Tang, editors, _Proceedings of the 11th Theory and Practice of Computer Graphics_ , TPCG ’13, pages 85–92, Bath, United Kingdom, 2013. Eurographics Association.  ISBN 978-3-905673-98-2.  doi:10.2312/LocalChapterEvents.TPCG.TPCG13.085-092.  URL http://diglib.eg.org/EG/DL/LocalChapterEvents/TPCG/TPCG13/085-092.pdf.
  * Güngör and Kurt [2014] Cengiz Güngör and Murat Kurt.  Improving visual perception of augmented reality on mobile devices with 3d red-cyan glasses.  In _Proceedings of the IEEE 22nd Signal Processing and Communications Applications Conference_ , SIU ’14, pages 1706–1709, Trabzon, Turkey, April 2014. IEEE.
  * Kurt [2017] Murat Kurt.  Experimental Analysis of BSDF Models.  In Reinhard Klein and Holly Rushmeier, editors, _Proceedings of the 5th Eurographics Workshop on Material Appearance Modeling: Issues and Acquisition_ , MAM ’17, pages 35–39, Helsinki, Finland, 2017. The Eurographics Association.  ISBN 978-3-03868-035-2.  doi:10.2312/mam.20171330.  URL https://diglib.eg.org:443/handle/10.2312/mam20171330.
  * Tongbuasirilai et al. [2017] Tanaboon Tongbuasirilai, Jonas Unger, and Murat Kurt.  Efficient BRDF sampling using projected deviation vector parameterization.  In _Proceedings of the IEEE International Conference on Computer Vision Workshops_ , ICCVW ’17, pages 153–158, Venice, Italy, October 2017. IEEE Computer Society.  ISBN 978-1-5386-1034-3.  doi:10.1109/ICCVW.2017.26.  URL http://doi.ieeecomputersociety.org/10.1109/ICCVW.2017.26.
  * Kurt [2020] Murat Kurt.  A Genetic Algorithm Based Heterogeneous Subsurface Scattering Representation.  In Reinhard Klein and Holly Rushmeier, editors, _Proceedings of the 8th Eurographics Workshop on Material Appearance Modeling: Issues and Acquisition_ , MAM ’20, pages 13–16, London, UK, 2020. The Eurographics Association.  ISBN 978-3-03868-108-3.  doi:10.2312/mam.20201140.  URL https://diglib.eg.org/handle/10.2312/mam20201140.
  * Tongbuasirilai et al. [2020] Tanaboon Tongbuasirilai, Jonas Unger, Joel Kronander, and Murat Kurt.  Compact and intuitive data-driven brdf models.  _The Visual Computer_ , 36(4):855–872, April 2020.  doi:10.1007/s00371-019-01664-z.  URL https://doi.org/10.1007/s00371-019-01664-z.
  * Kurt [2021] Murat Kurt.  Gensss: a genetic algorithm for measured subsurface scattering representation.  _The Visual Computer_ , 37(2):307–323, February 2021.  doi:10.1007/s00371-020-01800-0.  URL https://doi.org/10.1007/s00371-020-01800-0.
  * Mir et al. [2022] Sermet Mir, Barış Yıldırım, and Murat Kurt.  An analysis of goniochromatic and sparkle effects on multi-layered materials.  _Journal of Science and Engineering_ , 24(72):737–746, September 2022.  ISSN 1302-9304.
  * Gök et al. [2023] Gökçe Gök, Selin Küçük, Murat Kurt, and Ergin Tarı.  A u-net based segmentation and classification approach over orthophoto maps of archaeological sites.  In _Proceedings of the IEEE 31st Signal Processing and Communications Applications Conference_ , SIU ’23, pages 1–4, Istanbul, Turkey, July 2023. IEEE.
  * Guarnera et al. [2020] Dar’ya Guarnera, Giuseppe Claudio Guarnera, Matteo Toscani, Mashhuda Glencross, Baihua Li, Jon Yngve Hardeberg, and Karl R. Gegenfurtner.  Perceptually validated cross-renderer analytical BRDF parameter remapping.  _IEEE Trans. Vis. Comput. Graph._ , 26(6):2258–2272, 2020.  doi:10.1109/TVCG.2018.2886877.  URL https://doi.org/10.1109/TVCG.2018.2886877.
  * Sederberg and Greenwood [1992] Thomas W Sederberg and Eugene Greenwood.  A physically based approach to 2–d shape blending.  In _Proceedings of the 19th annual conference on Computer graphics and interactive techniques_ , pages 25–34, 1992.