# Coefficient invariances of Convex functions

Derek K. Thomas  Derek K. Thomas, Department of Mathematics, Swansea
University Bay Campus, Swansea, SA1 8EN, United Kingdom.
<EMAIL_ADDRESS>
# coefficient invariances for convex functions

Derek K. Thomas  Derek K. Thomas, Department of Mathematics, Swansea
University Bay Campus, Swansea, SA1 8EN, United Kingdom.
<EMAIL_ADDRESS>
###### Abstract.

We summarise known sharp bounds for coefficient invariances for convex
functions, and suggest some significant open problems

###### Abstract.

For convex univalent functions we give instances where the sharp bound for
various coefficient functionals are identical to those for the corresponding
bound for the inverse function . We give instances where the sharp bounds
differ and also suggest some significant open problems.

###### Key words and phrases:

Univalent, Convex functions, Carathéodory functions, Inverse, coefficients
functions

###### 1991 Mathematics Subject Classification:

30C45

##  1\. Introduction

Let ${\mathcal{A}}$ denote the class of analytic functions $f$ in the unit
disk $\mathbb{D}=\\{z\in\mathbb{C}:|z|<1\\}$ normalized by
$f(0)=0=f^{\prime}(0)-1$. Then for $z\in\mathbb{D}$, $f\in{\mathcal{A}}$ has
the following representation

$f(z)=z+\sum_{n=2}^{\infty}a_{n}z^{n}.$

Let ${\mathcal{S}}$ denote the subclass of all univalent (i.e., one-to-one)
functions in ${\mathcal{A}}$.

The most significant subclasses of $\mathcal{S}$ are the classes
$\mathcal{S}^{*}$ of Starlike and $\mathcal{C}$ of Convex functions, with
$\mathcal{C}$ defined as follows.

$f\in\mathcal{C}$ if, and only if, $f\in\mathcal{A}$ and

$\Re\left(1+\dfrac{zf^{\prime\prime}(z)}{f^{\prime}(z)}\right)>0,\quad
z\in\mathbb{D,}$

A classical result using the Carathéodory functions shows that if
$f\in\mathcal{C}$, then $|a_{n}|\leq 1$ for $n\geq 2$, see e.g [6].

##  2\. Inverse coefficients

Each $f\in\mathcal{S}$ possess an inverse function $f^{-1}$ given by

$f^{-1}(w)=w+\sum_{n=2}^{\infty}A_{n}w^{n},$

valid in some set $|w|<r_{0}(f)$

Although it took from 1916 to 1985 to solve the Bieberbach conjecture, in 1923
Lowner found the sharp upper bound for $|A_{n}|$ for all $n\geq 2,$ by showing
that

$|A_{n}|\leq\dfrac{\Gamma[2n+1]}{\Gamma[n+1]\Gamma[n+1]}.$

Since the Koebe function $k(z)$ is also starlike, Lowner’s bound is also sharp
for the class $\mathcal{S}^{*}$.

However since the Koebe function does not belong to $\mathcal{C}$, other sharp
bounds must hold for $|A_{n}|$ when $f\in\mathcal{C}$, and a complete solution
to this problem appears difficult to find.

The first indication that this was not a straightforward, but interesting
problem occurs in a paper in 1979 by Kirwan and Schober [2], who showed that
there exists a function in $\mathcal{C}$ such that $|A_{n}|>1$ for $n\geq 10$.

Then in 1982, Libera and Zlotkiewicz [5] proved the following.

Let $f\in\mathcal{C}$ and have inverse function given by

$f^{-1}(w)=w+\sum_{n=2}^{\infty}A_{n}w^{n},$

then for $2\leq n\leq 7$, the following sharp inequalities hold.

$|A_{n}|\leq 1.$

Subsequently in 1984, Campschroer [1] showed that if $f\in\mathcal{C}$ and has
inverse function given by (2), then $|A_{8}|\leq 1,$ and the inequality is
sharp.

Thus the first questions that arise are, what is the sharp bound for
$|A_{9}|$, and what are the sharp bounds for $|A_{n}|$ when $n\geq 10$?

The next obvious question is to ask if there are any other invariance
properties amongst coefficient functionals in $a_{n}$ and $A_{n}$?

If it turns out that there are other invariant properties, then why is this
so?

We will see that there are instances where sharp bounds can be found for
functionals concerning $A_{n}$ ($|A_{n}|$ for instance) which are different
for the sharp bound for the corresponding functional concerning $a_{n}$
($|a_{n}|$ for instance)?

Perhaps the most natural generalisation to the class $\mathcal{C}$ of convex
functions is the class of convex functions of order $\alpha$ defined as
follows.

For $0\leq\alpha<1$, denote by ${\mathcal{C}}(\alpha)$ the subclass of
${\mathcal{C}}$ consisting of convex functions of order $\alpha$ i.e.,
$f\in{\mathcal{C}}(\alpha)$ if, and only if, for $z\in\mathbb{D}$

$\operatorname{Re}\left\\{1+\frac{zf^{\prime\prime}(z)}{f^{\prime}(z)}\right\\}>\alpha.$

The first obvious question to consider is to look for invariance amongst the
initial coefficients, where we at once encounter non-invariance between
$|a_{3}|$ and $|A_{3}|$.

Using elementary techniques, it is a simple exercise using well-known tools to
prove the follow inequalities, all of which are sharp

If $f\in\mathcal{C}(\alpha),$ then

$|a_{2}|,|A_{2}|\leq
1-\alpha,\quad\text{and}\quad|a_{3}|\leq\dfrac{(3-2\alpha)(1-\alpha)}{3}.$

$|A_{3}|\leq\left\\{\begin{array}[]{ll}\dfrac{(3-4\alpha)(1-\alpha)}{3},&\hbox{$0\leq\alpha\leq\dfrac{1}{2}$,}\\\
&\\\
\dfrac{1-\alpha}{3},&\hbox{$\dfrac{1}{2}\leq\alpha<1$.}\end{array}\right.$

In 2016, Thomas and Verma [7] demonstrated some invariance properties amongst
the class of strongly convex functions defined as follows.

For $0<\beta\leq 1$, denote by ${\mathcal{C}}^{\beta}$ the subclass of
${\mathcal{C}}$ consisting of strongly convex functions i.e.,
$f\in{\mathcal{C}}^{\beta}$ if, and only if, for $z\in\mathbb{D}$

$\Big{|}\arg\Big{(}1+\frac{zf^{\prime\prime}(z)}{f^{\prime}(z)}\Big{)}|\leq\dfrac{\beta\pi}{2}.$

We give first some invariant properties amongst the initial coefficients
proved for $f\in\mathcal{C}^{\beta}$ in [7]

Let $f\in\mathcal{C}^{\beta}$ , then

$|a_{2}|,|A_{2}|\leq\beta,\quad\quad\quad|a_{3}|,|A_{3}|\leq\left\\{\begin{array}[]{ll}\dfrac{\beta}{3},&\hbox{$0<\beta\leq\dfrac{1}{3}$,}\\\
&\\\ {\beta}^{2},&\hbox{$\dfrac{1}{3}\leq\beta\leq 1$.}\end{array}\right.$

$|a_{4}|,|A_{4}|\leq\left\\{\begin{array}[]{ll}\dfrac{\beta}{6},&\hbox{$0<\beta\leq\sqrt{\dfrac{2}{17}}$,}\\\
&\\\
\dfrac{\beta}{18}(1+17{\beta}^{2}),&\hbox{$\sqrt{\dfrac{2}{17}}\leq\beta\leq
1$.}\end{array}\right.$

All the inequalities are sharp.

If $f\in\mathcal{C}^{\beta}$, then for any complex number $\nu$,

$\left|a_{3}-\nu{a_{2}^{2}}\right|,\left|A_{3}-\nu{A_{2}^{2}}\right|\leq\max\left\\{\frac{\beta}{3},\
{\beta}^{2}\left|1-\nu\right|\right\\}.$

Both inequalities are sharp.

For $f\in\mathcal{A}$, the logarithmic coefficients $\gamma_{n}$ of $f(z)$ are
defined by

$\log\dfrac{f(z)}{z}=2\sum_{n=1}^{\infty}\gamma_{n}z^{n}.$

They play a central role in the theory of univalent functions, and formed the
basis of de Brange’s proof of the Bieberbach conjecture.

We make the following definition.

Let $f\in\mathcal{C}^{\beta}$, and $\log\dfrac{f^{-1}(\omega)}{\omega}$ be
given by

$\log\dfrac{f^{-1}(\omega)}{\omega}=2\sum_{n=1}^{\infty}c_{n}{\omega}^{n}.$

It was also shown in [7] that if $f\in\mathcal{C}^{\beta}$, then

$|\gamma_{1}|,|{c}_{1}|\leq\dfrac{\beta}{2},\quad\quad\quad|\gamma_{2}|,|{c}_{2}|\leq\left\\{\begin{array}[]{ll}\dfrac{\beta}{6},&\hbox{$0<\beta\leq\dfrac{2}{3}$,}\\\
&\\\ \dfrac{{\beta}^{2}}{4},&\hbox{$\dfrac{2}{3}\leq\beta\leq
1$,}\end{array}\right.$

$|\gamma_{3}|,|{c}_{3}|\leq\left\\{\begin{array}[]{ll}\dfrac{\beta}{12},&\hbox{$0<\beta\leq\sqrt{\dfrac{2}{5}}$,}\\\
&\\\
\dfrac{\beta}{36}(1+5{\beta}^{2}),&\hbox{$\sqrt{\dfrac{2}{5}}\leq\beta\leq
1$.}\end{array}\right.$

All the inequalities are sharp.

Clearly the more complicated the coefficient functional, the more difficult
will be the analysis, and finding invariance.

We consider next the second Hankel determinants $H(2,2)(f)$ and
$H(2,2)(f^{-1})$, defined by

$H(2,2)(f)=a_{2}a_{4}-a_{3}^{2},$

and

$H(2,2)(f^{-1})=A_{2}A_{4}-A_{3}^{2}$

It was further shown by Thomas and Verma [7] that if
$f\in\mathcal{C}^{\beta}$, then

$|H(2,2)(f)|,|H(2,2)(f^{-1})|\leq\begin{cases}\dfrac{\beta^{2}}{9},&\mbox{\
}0<\beta\leq\dfrac{1}{3},\\\ &\\\
\dfrac{\beta(1+\beta)(1+17\beta)}{72(3+\beta)},&\mbox{\
}\dfrac{1}{3}\leq\beta\leq 1\\\ &\end{cases}$

who claimed that all the inequalities are sharp.

Although the proofs of the positive results are correct, the claim that the
second inequality is sharp is false.

Note however that when $\beta=1,$ i.e., $f\in\mathcal{C}$,
$|H(2,2)(f)|,|H(2,2)(f^{-1})|\leq\dfrac{1}{8},$ and these inequalities are
sharp.

Although the methods used in the proofs of the above invariance are correct,
they are not strong enough to give sharp bounds for the second inequality. The
following correction was subsequently given by Lecko, Sim and Thomas [10].

If $f\in\mathcal{C}^{\beta}$, then the following sharp inequalities hold.

$|H(2,2)(f)|,|H(2,2)(f^{-1})|\leq\begin{cases}\dfrac{\beta^{2}}{9},&\mbox{\
}0<\beta\leq\dfrac{1}{3},\\\ &\\\
\dfrac{\beta^{2}(1+\beta)(17+\beta)}{72(2+3\beta-\beta^{2})},&\mbox{\
}\dfrac{1}{3}\leq\beta\leq 1.\\\ &\end{cases}$

Note again that when $\beta=1,$ i.e., $f\in\mathcal{C}$,
$|H(2,2)(f)|,|H(2,2)(f^{-1})|\leq\dfrac{1}{8.}$

The problem of finding sharp bounds for the difference of successive
coefficients $|a_{n+1}|-|a_{n}|$ for functions in $\mathcal{S}$ represents one
of the most difficult areas of study in univalent function theory, with the
only sharp bound known so far is when $n=2$, where Duren prove the rather
curious sharp inequality

$-1\leq|a_{3}|-|a_{2}|\leq\frac{3}{4}+e^{-\lambda_{0}}(2e^{-\lambda_{0}}-1)=1.029\cdots,$

where $\lambda_{0}$ is the unique value of $\lambda$ in $0<\lambda<1$,
satisfying the equation $4\lambda=e^{\lambda}$.

Although Leung found the complete solution when $f$ is starlike by showing
that $||a_{n+1}|-|a_{n}||\leq 1$, the problem for convex functions remains
mostly open.

In 2016, Ming and Sugawa [4] made the first advance in this problem by finding
the following sharp bounds when $n\geq 2$

$|a_{n+1}|-|a_{n}|\leq\dfrac{1}{n+1},$

and also proved the sharp lower bounds

$|a_{3}|-|a_{2}|\geq-\dfrac{1}{2},\quad\text{and}\quad|a_{4}|-|a_{3}|\geq-\dfrac{1}{3}$

Thus in particular we have the sharp bounds

$-\dfrac{1}{2}\leq|a_{3}|-|a_{2}|\leq\dfrac{1}{3}\quad\text{and}\quad-\dfrac{1}{3}\leq|a_{4}|-|a_{3}|\leq\dfrac{1}{4}.$

Sim and Thomas in 2020 [11] proved the following inequalities hold for the
inverse coefficients, thus demonstrating another example of invariance.

$-\dfrac{1}{2}\leq|A_{3}|-|A_{2}|\leq\dfrac{1}{3},$

and have recently shown, (the proof of which requires much more complicated
analysis), that the following sharp inequalities hold

$-\dfrac{1}{3}\leq|A_{4}|-|A_{3}|\leq\dfrac{1}{4}.$

Any advances when $n\geq 4$ would require deeper methods of proof.

It is clear from the above, that the more complicated the functional and class
of convex functions considered, the less likely is it that there will be
invariance. Also functionals containing coefficients $a_{n}$ and $A_{n}$ for
$n\geq 3$ will similarly be difficult to deal with.

Sim and Thomas ([12], Proposition 1) have recently given a general lemma
concerning functions of positive real part, which provides a tool enabling
coefficient differences to be found when $n=2$, which can be applied to many
subclasses of univalent functions . This can also be used to consider
coefficient differences of the inverse function. However the lemma only
applies when $n=2$.

In recent years it has become fashionable to consider subclasses of convex
(and starlike) functions where the function $p(z)$ is specified, often having
a range with some interesting geometrical property.

Examples of recently discussed subclasses of convex functions, where some
initial invariance properties have been found are as follows.

A most natural class of convex functions related to the exponential function
is the class $\mathcal{C}_{E}$ defined as follows

$\mathcal{C}_{E}=\left\\{f\in\mathcal{A}:1+\frac{zf^{\prime\prime}(z)}{f^{\prime}(z)}\prec
e^{z}\right\\}.$

Some initial coefficients results for the class $\mathcal{C}_{E}$ were given
by Zaprawa [14], and similar analysis shows that the following invariance
properties hold.

$|a_{2}|,|A_{2}|\leq\dfrac{1}{2}\quad\text{and}\quad|a_{3}|,|A_{3}|\leq\dfrac{1}{4}\quad\text{and}\quad|a_{4}|,|A_{4}|\leq\dfrac{17}{144}.$

All the inequalities are sharp.

Using well-known methods it is also possible to prove the following
inequalities, both of which are sharp.

$|a_{2}a_{3}-a_{4}|,|A_{2}A_{3}-A_{4}|\leq\dfrac{1}{12}.$

Similarly the following sharp invariances hold for the second Hankel
determinants for $f\in\mathcal{C}_{E}$ [14],

$|H(2,2)(f)|,|H(2,2)(f^{-1})|\leq\dfrac{73}{2592}.$

A class $\mathcal{C}_{SG}$ of convex functions which exhibits some interesting
invariance properties is related to a modified sigmoid function and is defined
by

$\mathcal{C}_{SG}=\left\\{f\in\mathcal{A}:1+\frac{zf^{\prime\prime}(z)}{f^{\prime}(z)}\prec\frac{2}{1+e^{-z}}\right\\}.$

Here the function $2/(1+e^{-z})$ is a modified sigmoid function which maps
$\mathbb{D}$ onto the domain
$\Delta_{SG}=\left\\{w\in\mathbb{C}\right\\}:\left|\log\left(w/\left(2-w\right)\right)\right|<1\\}.$

The class $\mathcal{C}_{SG}$ was first discussed in [8] where some invariance
properties were found. In particular the following invariance properties hold.

$|a_{2}|,|A_{2}|\leq\dfrac{1}{4}\quad\text{and}\quad|a_{3}|,|A_{3}|\leq\dfrac{1}{12}\quad\text{and}\quad|a_{4}|,|A_{4}|\leq\dfrac{1}{24}.$

All the inequalities are sharp.

Further invariance properties for functions in $\mathcal{C}_{SG}$ also hold
(see [9]), where the following inequality for $|a_{2}a_{3}-a_{4}|$ was proved,
and the inequality for $|A_{2}A_{3}-A_{4}|$ follows using similar methods,

$|a_{2}a_{3}-a_{4}|,|A_{2}A_{3}-A_{4}|\leq\dfrac{1}{24}.$

Both inequalities are sharp.

As already mentioned, finding sharp bounds for the differences of coefficients
can present difficulties and next we give an example of proved non-invariance
for $\mathcal{C}_{SG}$, noting that all the inequalities are sharp.

$-\dfrac{5}{24}\leq|a_{3}|-|a_{2}|\leq\dfrac{1}{12},\quad\text{and}\quad-\dfrac{1}{4}\leq|A_{3}|-|A_{2}|\leq\dfrac{1}{12}.$

As mentioned above other choices of $p(z)$ have been made primarily to define
some kind of interesting geometric property of the range of $p(z)$, for
example

(i) $p(z)=1+\dfrac{4}{3}z+\dfrac{2}{3}z^{2}$ gives a cardioid domain.

(ii) $p(z)=1+\dfrac{4}{5}z+\dfrac{1}{5}z^{4}$ gives a 3-leaf petal shaped
domain.

It is very likely that similar invariances hold for some functionals in these
classes.

Finally we note a recent interesting example of non-invariance.

It was shown by Sim, Zaprawa and Thomas in 2021 [13] that for
$f\in\mathcal{C}(\alpha),$

$|H(2,2)(f)|\leq\dfrac{(1-\alpha)^{2}(6+5\alpha)}{48(1+\alpha)},$

and that this inequality is sharp, thus solving a long-standing problem.

In the same paper it was shown that for $f\in\mathcal{C}(\alpha),$

$|H(2,2)(f^{-1})|\leq\begin{cases}\dfrac{1}{96}(12-28\alpha+19\alpha^{2}),&\mbox{\
}\alpha\in[0,\dfrac{2}{5}],\\\ &\\\ \dfrac{1}{9}{(1-\alpha)^{2}},&\mbox{\
}\alpha\in[\dfrac{2}{5},\dfrac{4}{5}],\\\ &\\\
\dfrac{\alpha(1-\alpha)^{2}(19\alpha-8)}{48(1+\alpha)(2\alpha-1)},&\mbox{\
}\alpha\in[\dfrac{4}{5},1].\end{cases}$

and that all the inequalities are sharp.

Thus unless $\alpha=0$, we have non-invariance.

We note here that when $f\in\mathcal{C}^{(}\beta),$ there is invariance
between $|H(2,2)(f)|$ and $|H(2,2)(f^{-1})|$ for all $\beta\in(0,1]$, which is
curious since the definition of the class $\mathcal{C}^{(}\beta)$ involves the
power $p(z)^{\beta}.$

Checking for invariances is a matter of applying available well-known tools to
functionals.

But the real problem is to discover WHY invariances occur in the classes and
functionals considered.

The answer is probably that invariances are both class and functional
dependant, and that there is no simple rule.

Recall that in 2016, Ming and Sugawa [4] proved that if $f\in\mathcal{C}$,
then when $n\geq 2$

$|a_{n+1}|-|a_{n}|\leq\dfrac{1}{n+1},$

and that the bounds are sharp.

In the same paper Ming and Sugawa [4] further proved that when $n\geq 4$

$-\dfrac{1}{n}<|a_{n+1}|-|a_{n}|\leq\dfrac{1}{n+1},$

thus

$||a_{n+1}|-|a_{n}||=\mathcal{O}\big{(}\dfrac{1}{n}\big{)},\ \ \text{as}\
n\to\infty.$

Is it true that if $f\in\mathcal{C}$, then

$||A_{n+1}|-|A_{n}||=\mathcal{O}\big{(}\dfrac{1}{n}\big{)},\ \ \text{as}\
n\to\infty?$

Next note that Kowalczyk, Lecko and Sim [3] have shown that if
$f\in\mathcal{C}$, then the third Hankel determinant

$|H_{3,1}(f)|=|a_{3}(a_{2}a_{4}-a_{3}^{2})-a_{4}(a_{4}-a_{2}a_{3})+a_{5}(a_{3}-a_{2}^{2})|\leq\dfrac{4}{135},$

and that this inequality is sharp.

It is true that the following sharp inequality holds?

$|H_{3,1}(f^{-1})|=|A_{3}(A_{2}A_{4}-A_{3}^{2})-A_{4}(A_{4}-A_{2}A_{3})+A_{5}(A_{3}-A_{2}^{2})|\leq\dfrac{4}{135}.$

## References

  * [1] J. T. P. Campschroer, Growth of the coefficients of the inverse of a convex function. Indag. Math. 87 (1), (1984), 1–6.
  * [2] W. E. Kirwan and G. Schober, Extremal problems for meromorphic univalent functions. J. Analyse Math. 30, (1976), 330–348.
  * [3] B. Kowalczyk, A. Lecko and Y. J. Sim. The sharp bound for the Hankel determinant of the third kind for convex functions. Bull. Aust. Math. Soc. 97 (3), (2018), 435–445.
  * [4] L. Ming and T. Sugawa, A note on successive coefficients of convex functions. Comput. Methods Funct. Theory 17(2), (2017), 179–193
  * [5] R. J. Libera and E. J. Zlotkiewicz, Coefficient bounds for the inverse of a function with derivative in P. Proc. Amer. Math. Soc. 87(2), (1983), 251–257.
  * [6] Ch. Pommerenke, On the Hankel determinants of univalent functions. Mathematika (London) 14 (1), (1967), 108–112.
  * [7] D. K. Thomas and S. Verma, Invariance of the coefficients of strongly convex functions. Bull. Aust. Math. Soc. 95 (3), (2017), 436–445.
  * [8] A. Riaz, M. Raza and D. K. Thomas, Hankel determinants for starlike and convex functions associated with sigmoid functions. Forum Math. 34 (1), (2022), 137–156..
  * [9] A. Riaz, M. Raza and D. K. Thomas, Coefficient estimates for starlike and convex functions related to the sigmoid functions. Ukraine Math. J., accepted for publication.
  * [10] Y. J. Sim, A. Lecko and D. K. Thomas, The second Hankel determinant for strongly convex and Ozaki close-to-convex functions. Ann. Mat. Pura. Appl. 200 (6), (2021), 2515–2533.
  * [11] Y. J. Sim and D. K. Thomas, On the difference of inverse coefficients of convex functions. J. Anal. 30 (2), (2022), 875–893
  * [12] Y. J. Sim and D. K. Thomas, On the difference of inverse coefficients of univalent functions. Symmetry 12 (12), (2020), 2040.
  * [13] Y. J. Sim, D. K. Thomas and P. Zaprawa, The second Hankel determinant for starlike and convex functions of order alpha. Complex Var. Elliptic Equ. 67 (10), (2022), 2423–2443
  * [14] P. Zaprawa, Hankel determinants for univalent functions related to the exponential function. Symmetry. 11 (10), (2019), 1211.


0 Research

# A Design Process of Visual Analytics Application using Conceptual Graph

Lei Shi
Beihang University

###### Abstract

State-of-the-art visual analytics techniques in application domains are often
designed by VA professionals over qualitative requirement collected from end
users. These VA techniques may not leverage users’ domain knowledge about how
to achieve their analytical goals. In this position paper, we propose a user-
driven design process of VA applications centered around a new concept called
analytical representation (AR). AR features a formal abstraction of user
requirement and their desired analytical trails for certain VA application,
and is independent of the actual visualization design. A conceptual graph
schema is introduced to define the AR abstraction, which can be created
manually or constructed by semi-automated tools. Designing VA applications
with AR provides a shared opportunity for both optimal analysis blueprint from
the perspective of end users and optimal visualization/algorithm from the
perspective of VA designers. We demonstrate the usage of the design process in
two case studies.

Introduction

###  0.1 Background

Visual analytics (VA) is coined as “the science of analytical reasoning
facilitated by interactive visual interfaces” [32], or more specifically,
“combines automated analysis techniques with interactive visualizations” [20].
In the new century, VA emerges as a promising methodology in the field of
visualization and data science. Numerous successful VA applications have been
presented in the annual VAST conference [1], contributing to key domains such
as finance [12], urban computing [6], machine learning [36], and many others.
Yet, state-of-the-art VA applications generally have indirect domain user
involvement in their design process. Among six example papers on VA
application & system category provided in the VIS’20 website [4], five papers
reported certain kind of user requirement or task analysis, the other paper
mentioned little from the end user’s perspective. In most of these works, raw
input from end users is first summarized by VA experts (normally the authors)
into list-based requirement/task, and then referenced in the VA design
process. Domain users do not directly contribute to the VA framework
constructed for their applications. Notably, only one example paper planned
the detailed analytical reasoning process of their VA solution in the design
phase, for which the authors themselves were domain experts on the application
of scientific literature analysis [8].

Promoting domain user involvement in the VA design process to a level beyond
qualitative requirement analysis adopted in classical visualization research
can be beneficial. First, compared with visualization problems mostly dealing
with fixed data types, VA applications are often built over more complex
customer data where not all data input in their raw form are applied in
solving the customer problem. Picking the right data source, selecting the
most appropriate analysis direction all demand thorough understanding of user
requirement and deep knowledge about the domain. This calls for closer
involvement of end users in the VA design process. Second, existing
requirement/task analyses in VA applications result in unstructured, textual
lists as design guidelines. These guidelines normally vary a lot across VA
applications. The final VA pipeline constructed in each application, though
carefully follows these specific guidelines, is often seen as an one-time
case-by-case design. It is difficult to reuse successful VA applications to
automate the design in future. Third, from industry’s perspective, the
deficiency of domain user involvement in VA applications potentially slows
down the adoption of the technology in the mass. By contrast, the growing
popularity of software such as Tableau [3] and Microsoft Power BI [2]
witnesses the democratization of information visualization (InfoVis)
techniques. Commodity users can freely drag-and-drop data items to gain much
control of the visualization design, which may be one important reason of its
success. Meanwhile, the widest spontaneous usage of VA technology outside the
visualization community is seen in the deep learning field where the black-box
models are analyzed and visualized for interpretability. Interestingly, many
of these VA applications are constructed by machine learning experts who can
gain full control of the design process as domain users. A similar call to
embrace customers in the visualization community has been made many years
before by Lorensen [23].

###  0.2 Related Work

Figure 1: (a) Conceptual graph, (b) its adaptation for AR, and (c) an example
of AR schema and (d)(e) visualization instances.

A central topic of visualization and VA research has been its design and
implementation process. Card et al. proposed the InfoVis reference model [11],
a standard pipeline to create interactive visualization from source data. Keim
et al. illustrated a VA paradigm to integrate visualization with automatic
data analysis methods for interactive decision support [20]. The visual
information seeking mantra by Shneiderman suggested a golden rule in designing
the exploration process with visualization [30]. Mckenna et al. presented a
design activity framework composed of four critical activities for designing
visualizations: understand, ideate, make, and deploy [24]. All these high-
level theories for visualization design might have been followed by many
designers, but most of them do not consider the problem of getting direct
involvement of end users in the VA design process. The theories help to
modularize the implementation of each visualization stage, but provide little
guidance in reusing VA solutions for customer problems. On implementing
visualization software, a number of design patterns have been summarized in
the literature [17][14]. Nevertheless, our work focuses on the VA design
process, which is generally independent of its implementation.

More relevant to our study, the dataflow network used in scientific
visualization (SciVis) [16] provides low-level abstraction of data analysis
and visualization pipeline for each application. By integrating with software
toolkits [27] and user interfaces such as VisTrails [7], the dataflow model
allows greater user involvement in the visualization design. End users could
fulfill their requirement and evaluate the visualization performance in the
same interactive interface. Typically, SciVis experts behave as both end users
and visualization designers in these applications. Compared with the dataflow
model, we consider a different scenario in VA applications where domain users
are normally not visualization and algorithm experts. These nonprofessionals
may integrate their demand into the analytical pipeline of VA application
through the dataflow-like model, but probably can not assemble the optimal
visualization and algorithm choice, because of their lack of expertise in the
VA field. In the InfoVis community, Chi and Riedl also proposed a data state
model for visualization applications [15], which can be seen as a dual form of
the dataflow model. The data state model is mainly used to characterize
visualization techniques with a focus on their interaction aspect. In
contrast, we study the abstraction of user requirement/task in VA
applications, which can be independent of their detailed visualization and
interaction design.

Related to our work, the user tasks accomplished in visualization were
intensively discussed in the literature. Shneiderman proposed a task by data
type taxonomy which includes seven data types (1D, 2D, etc.) and seven
visualization tasks (overview, zoom, etc.) [30]. Stasko summarized ten low-
level analysis tasks in understanding data with visualization [5], which were
compiled over existing low-level task taxonomies [35]. Brehmer and Munzner
studied both low-level and high-level visualization tasks and proposed a
multi-level typology to orchestrate the task design space [10]. Similar
researches have also been conducted on the taxonomy and design space of
visualization tasks with the multidimensional approach [28], in cubic views
[26], from the perspective of user interaction [18], and for certain data
types [22][33]. These existing studies on classifying visualization tasks are
orthogonal to our goal of promoting user involvement to fulfill their tasks in
VA applications. In fact, we borrowed some of the high-level task taxonomy in
defining the task node in the representation of user requirement for visual
analytics. Notably, our work is also motivated by Munzner’s nested model for
visualization design [25], which decouples domain problem characterization
(user requirement), data/operation abstraction (analysis and visualization
task), and visualization design (user performance). The VA design process
introduced here partially fulfills Munzner’s nested model in the first and
second levels of abstraction.

###  0.3 Two-Stage VA Design Process

Getting user involvement in VA design brings the immediate challenge of having
unprofessional visualizations. The main idea of this work is to allow
structured input of domain requirement/knowledge from end users, while
imposing the separation of satisfying user requirement from optimizing
visualization performance in VA applications. To achieve this idea, a two-
stage VA design process is proposed. In the first stage, an abstraction of end
user’s visual analysis trail is described in the form of data analysis and
visualization tasks. The abstraction is independent of the visualization
design so that end users can get involved in this stage to the largest extent
without caring about the genre and performance of final visualization design.
In the second stage, the first-stage abstraction is translated by VA designers
into the classical VA pipeline, through optimally selecting data analysis
algorithms and visualizations, e.g. by the design activity framework of
Mckenna et al. [24]. The optimal VA design can not only be used in the current
application, but also recorded in VA knowledge base indexed by structured
requirements, which facilitates reuse/automation of VA applications.

Provisioning the two-stage VA design process is more difficult than
envisioning it. How to isolate user requirement from their performance when
the two are often entangled in visual analytics? How to design the abstraction
in the first stage to minimize the gap for end user’s direct involvement while
ensuring formality for faithful translation into optimal VA designs in the
second stage? We make two contributions to address these challenges.

  * •

The new concept of analytical representation (AR) is introduced as the
abstraction of user requirement and analytical trails in VA applications. We
propose a formal method using conceptual graph to describe the intension of
AR, and using complementary visualization and algorithm instances to
illustrate its extensions. End users could precisely integrate their desired
visual analytics tasks into AR and offer VA designers a detailed blueprint for
constructing VA applications.

  * •

The VA design process with AR is presented and evaluated. Case study results
show that by the new design process, visual analytics in our sample
applications are made both adaptive and dedicated to solving end user’s domain
problems. In the cases, the design process generates VA pipelines that are
compatible to existing reference models.

##  1 Analytical Representation

###  1.1 Definition

Analytical representation is the visualization-independent abstraction of
relevant data and processing for certain visual analytics application.
Primarily, AR is defined for and used in VA scenarios where domain problems
are solved by combining automated data analysis and human-in-the-loop
visualization. The main body of AR is the abstraction of required trails of
data analysis and visualization from the view of end users. Most importantly,
the abstraction is independent of the actual visualization design and data
analysis algorithms. In a sense, AR could be the mental map of end users on
how to solve a domain problem using the combination of data analysis and
visualization, though he may not know and need not know the detailed method in
each step of visual analysis. Figure 1(c) gives an example of AR schema for
citation influence analysis by linked high-level task abstractions on data
analysis and visualization.

Empowered by the concept and tool to create AR, end users could focus on the
requirement and domain knowledge for VA applications, without considering the
performance using particular visualization design. The AR abstraction can be
exemplified by automatically generated visualization instances, which help end
users early evaluate the effectiveness of VA applications at the design stage.
End users could iterate AR blueprint using evaluation result as feedback. AR
itself also serves as a quantitative index of knowledge base for VA best
practices. Building such knowledge base could push forward the reuse,
automation, and finally wider adoption of VA technology.

###  1.2 AR Schema by Conceptual Graph

To achieve formalism in analytical representation, we propose to use
conceptual graph (CG) to define the schema of AR. Historically, CG is first
invented to represent database schema and natural language (see an example in
Figure 1(a) ) [31], and later extended to support a wider scope of knowledge
representation [13]. AR, defined as the representation of end user’s
requirement and domain knowledge for certain VA application, is also a kind of
knowledge representation. This is the main reason we apply CG as the schema of
AR. Additionally, the expressive power of labeled graph notation in CG allows
the level of abstraction in AR definition.

There are necessary adaptations from the initial definition of CG to the
schema of AR, which is illustrated in Figure 1(b). CG is originally composed
of the concept nodes and the relation nodes. The concept node is altered to
the data node in AR (boxes in Figure 1(c)). Each data node has a data type and
a data instance. The relation node is altered to two classes of nodes: the
analysis task node (diamonds in Figure 1(c)) and the visualization task node
(circles in Figure 1(c)). Each task node has a task type and a task target.
Directed arcs connecting nodes in AR represent data flows.

The key concept of AR is the typed node representation. Each class of nodes
can define its own type ontology, i.e., a multi-level category (lattice) of
types. For data nodes, we use the taxonomy by Shneiderman as the top-level
category for data types [30]: 1D, 2D, 3D, nD, temporal, tree, network. A
different taxonomy can also fit in depending on the usage scenario of AR. For
visualization task nodes, we advocate the use of high-level task taxonomy as
its type category, as low-level task nodes can make the abstraction of AR
overly detailed and less reusable. In this work, we apply the high-level task
typology by Brehmer and Munzner [10]: present, produce, explore, identify,
etc. Again, using a different high-level visualization task taxonomy is
possible. Analysis task nodes are not our focus, any ontology of data analysis
algorithms will work.

In the example of Figure 1(c), a visual analysis trail on the citation
influence of a scientific paper is represented by the adapted CG form of AR
schema. As end users expect, a full citation influence graph of the paper
should be extracted and summarized. Then s/he wants to visualize and
interactively explore the influence graph summarization (T2 visualization
task). Also, the user wants to be presented a text summarization of paper
clusters in the graph to understand the citation influence content (T1
visualization task). Inherited from CG notations, the schema of AR can also be
represented in a textual format known as Conceptual Graph Interchange Format
(CGIF). The bottom line of Figure 1(c) gives part of this format for the
example. The CGIF of AR can be analyzed by symbolic computation, similar to
knowledge reasoning and inference on CGs.

CG is an abstract schema of AR. Beyond the abstraction, end users also need a
preview and early evaluation of the VA design over the AR schema. We propose
to link each visualization task node in AR with at least one instance. The
instances can be retrieved from the knowledge base of VA designs indexed by AR
graphs and nodes. With sufficient VA applications input to the knowledge base,
the instance can be automatically recommended as the design of the best-
matched or most-representative task node in the knowledge base for the current
node. Figure 1(d)(e) presents artificial instances for the two visualization
task nodes in Figure 1(c). A standard tag cloud visualization is mapped to T1
visualization task node [34]. A specialized visualization for clustered
citation graph [29] is selected for T2 task node. If necessary, algorithm
instances of the analysis task node can also be delivered to end users for
evaluation.

###  1.3 Comparison to Other Concepts

AR is related to many existing concepts, but there are clear distinctions.
Data representation is the class of methods primarily used to prepare data for
machine computing and is not designed for visualization purpose. Visual
representation, on the other hand, describes the mapping from data to
visualization, but does not consider domain-specific VA tasks. AR can be seen
as a concept between data and visual representation with the focus to abstract
high-level data analysis and visualization tasks for certain VA application.

In the knowledge representation community, ontology and knowledge graph are
widely adopted. They are used for the representation of subjects and entities
in one field, while AR is used as the representation of VA tasks in certain
application. Graphical models such as neural networks are designed to
represent data distributions, which are special types of data representation.
They are not at the same level as AR in the pipeline of visual data analysis.

##  2 Designing Visual Analytics with AR

Figure 2: The design process of VA applications with AR.

Figure 2 suggests a reference model for designing VA applications with AR. In
the first half of the model before AR, user requirement is analyzed to
characterize VA tasks separately on data analysis and visualization. These
tasks and available data input for the application are connected according to
end user’s desired visual analytics trail of the application. We note that end
users can compose an AR of the application solely by themselves, but the
participation of VA designers in this stage is highly recommended. Users may
not be familiar with the category system of VA tasks and the capability of
related visualizations and algorithms. An alternative AR design approach is to
automatically extract AR blueprint from visualization sketches of the end
user. The automated approach can be highly challenging and is not the focus of
this work.

In the second half of the model in Figure 2, VA designers and algorithm
designers create the actual visualizations used in the VA application and
select appropriate algorithms for the data analysis. These visualizations and
algorithms should cover all task nodes specified in the AR blueprint. The AR-
based VA design process exhibits a clear advantage in that designers may pick
visualizations and algorithms according to their experience and expertise, and
can also leverage the AR knowledge base storing the design of qualified VA
applications. The knowledge base can suggest or even automate the design of
certain AR task nodes using existing best practices. Another advantage of the
proposed VA design process lies in that the output is still a standard VA
pipeline. Every stage is already decided in the design process and can be
implemented by VA system programmers in the same way with previous VA
applications.

##  3 Case Study

Figure 3: VLSI chip production case: (a) AR schema; (b)(c)(d) visualization
instances. Citation analysis case: (e) AR; (f)(g) visualization instances.

We present two case studies to demonstrate the usage of AR-based design
process for VA applications. In the first case study, we show how the
integration of domain knowledge and user requirement in AR can allow clearer
path towards user’s analytical goal. The background is the well-known
application of parallel coordinates visualization (PCP) by Inselberg in
analyzing the production dataset of VLSI chips [19]. The dataset is composed
of 16 variables: $X_{1}$ and $X_{2}$ are yield and quality of each production
batch; $X_{3}\sim X_{12}$ are the number of defects in ten types; $X_{13}\sim
X_{16}$ are physical parameters of each batch. Through iterative navigation of
the dataset with PCP, several data patterns are found to help maintain both
high batch yield and quality, including a small range of $X_{3}$, $X_{6}$, and
$X_{15}$.

While we applaud for the detective ingenuity of the author, there is also
doubt about whether ordinary users can discover the same pattern with PCP. In
an alterative approach using visual analytics and AR, users start from their
requirement, i.e., to obtain high yield and quality in a batch. By VLSI
literature [21], it is also common sense for domain users that yield and
quality ($X_{1}$,$X_{2}$) can be modeled by #defects ($X_{3}\sim X_{12}$),
which are further determined by physical parameters ($X_{13}\sim X_{16}$).
With this domain knowledge, we believe end users can achieve the AR design of
Figure 3(a), albeit certain variations among users. In their design, $X_{1}$
and $X_{2}$ are first visually correlated to clarify the analytical goal: what
is the desired high range of yield and quality (see the red circle in the
scatterplot instance of Figure 3(b))? To answer which defects affect the most
on high yield/quality, predictive models can be built and visually analyzed.
In simplified instances by the regression model and visreg visualization [9]
(Figure 3(c)), users could easily identify $X_{3}$ and $X_{6}$ as key types of
defects for chip yield and quality where $X_{1}$ ($X_{2}$) changes
significantly with $X_{3}$ ($X_{6}$) outside the selected high range in red.
Adding these dimensions to lookup the desired physical parameters ($X_{13}\sim
X_{16}$), the PCP instance in Figure 3(d) suggests a small range of $X_{15}$,
the same pattern detected by Inselberg.

The second case study shows that the VA design process with AR helps to
distinguish between different user requirement for the same domain problem. In
our previous work, we have developed a VA system for citation influence
analysis, whose AR schema has been used as the example in Figure 1(c). During
the research cycle, we also received from reviewers and other academic users
about different opinions of our work. Notably, they consider the citation
influence of a paper as the direct 1-hop citation network from the original
paper, instead of the full citation network measuring the influence on a
field. In addition, they also want to understand the temporal dynamics of the
influence, which is not considered by us. By the AR-based design, the new user
requirement can be satisfied by simply revising the AR blueprint. As shown in
Figure 3(e), the extract task node is changed to analyze 1-hop citation
influence. The summarize influence graph node is removed as there may not be
many nodes in the 1-hop citation influence graph. The time dimension is
incorporated in most task nodes and a new visualization task node is added to
identify evolution patterns from citation influence (T2). The visualization
instances in Figure 3(f)(g) clearly reflect the different user requirement in
comparison to the previous instances (Figure 1(d)(e)). Using AR-based VA
design process, both similar and divergent user requirement can be represented
and executed, which effectively enhances the reusability of VA applications.

##  4 Discussion

As a preliminary framework, the current AR design process does have
limitations. First, AR specifies analytical trails in VA applications, but it
is not designed for explorative visual analytics in which end users have
little knowledge about the domain and/or do not have clear analytical goal.
The deficiency can be alleviated by adding a feedback mechanism to the AR
design process. An initial AR can be constructed from general-purpose analysis
tasks. After gaining new knowledge and potential analytical goals in the
domain, users can update the AR design by translating the new knowledge/goal
into visual analytics trails. Repeating the design process helps to solve
domain problems with VA techniques.

Second, there are threats in constructing AR. End users may not have the
capability to summarize their requirement into appropriate task nodes and an
accurate AR schema. While VA designers can certainly provide support through
direct communication, we believe that the end users designing VA applications
with AR should have moderate skills in data analytics, though they may not be
visualization experts. Unlike automatic data analysis applications such as
data mining, visual analytics inherently require end users to understand and
manipulate the data analysis process. For these users, composing AR blueprints
should be possible with training. In addition, semi-automated AR construction
tools providing taxonomies and instances of visualization/analysis tasks will
further reduce the barrier for the proposed VA design process.

Third, as a position paper, many remaining work need to be carried out along
the direction. The existing taxonomies of data types, analysis and
visualization tasks should be studied for the best fit in AR design. The
design automation tool for AR will also expedite the usage of AR-based VA
applications. The theory and implementation of AR knowledge base is another
promising future work towards democratization and reusability of VA
applications.

##  5 Conclusion

Synthesizing end user’s requirement and their domain knowledge is challenging
in existing VA applications. In this position paper, we show that it is
possible to formally abstract this critical information by a new concept
called analytical representation. AR adopts the conceptual graph as its schema
and can be evaluated through extended visualization instances. Designing VA
applications with AR enjoys three advantages. First, AR could be used as a
blueprint for solving specific domain problem following the analytical trails
recommended by end users. Second, the visualization-independent nature of AR
allows the optimization of visualization and algorithm designs in the
application through separate effort of VA/algorithm designers. Third, the
abstraction enables quantification of VA designs and the establishment of VA
knowledge base, so that design automation and reuse of VA applications can be
expected. We also present two case studies to demonstrate the desired
advantages of the new design process.

## References

  * [1] IEEE Conference on Visual Analytics Science and Technology, 2006–2020.  http://vis.computer.org/visweek2010/vast/.
  * [2] Microsoft powerbi, 2020.  https://powerbi.microsoft.com/.
  * [3] Tableau, 2020.  https://www.tableau.com/.
  * [4] Vast paper types, 2020.  http://ieeevis.org/year/2020/info/call-participation/vast-paper-types.
  * [5] R. Amar, J. Eagan, and J. Stasko.  Low-level components of analytic activity in information visualization.  In Proceedings of IEEE Symposium on Information Visualization, pp. 111–117, 2005.
  * [6] G. Andrienko and N. Andrienko.  Spatio-temporal aggregation for visual analysis of movements.  In Proceedings of IEEE Symposium on Visual Analytics Science and Technology, pp. 51–58, 2008.
  * [7] L. Bavoil, S. P. Callahan, P. J. Crossno, J. Freire, C. E. Scheidegger, C. T. Silva, and H. T. Vo.  Vistrails: Enabling interactive multiple-view visualizations.  In Proceedings of IEEE Visualization, pp. 135–142, 2005.
  * [8] F. Beck, S. Koch, and D. Weiskopf.  Visual analysis and dissemination of scientific literature collections with survis.  IEEE Transactions on Visualization and Computer Graphics, 22(1):180–189, 2015.
  * [9] P. Breheny and W. Burchett.  Visualization of regression models using visreg.  The R Journal, 9(2):56–71, 2017.
  * [10] M. Brehmer and T. Munzner.  A multi-level typology of abstract visualization tasks.  IEEE Transactions on Visualization and Computer Graphics, 19(12):2376–2385, 2013.
  * [11] S. K. Card, J. D. Mackinlay, and B. Shneiderman.  Readings in Information Visualization: Using Vision to Think.  Academic Press, 1999.
  * [12] R. Chang, M. Ghoniem, R. Kosara, W. Ribarsky, J. Yang, E. Suma, C. Ziemkiewicz, D. Kern, and A. Sudjianto.  Wirevis: Visualization of categorical, time-varying data from financial transactions.  In Proceedings of IEEE Symposium on Visual Analytics Science and Technology, pp. 155–162, 2007.
  * [13] M. Chein and M.-L. Mugnier.  Graph-based knowledge representation: computational foundations of conceptual graphs.  Springer Science & Business Media, 2009.
  * [14] H. Chen.  Toward design patterns for dynamic analytical data visualization.  In Proceedings of SPIE Visualization and Data Analysis, vol. 5295, pp. 75–86, 2004.
  * [15] E. H.-h. Chi and J. T. Riedl.  An operator interaction framework for visualization systems.  In Proceedings of IEEE Symposium on Information Visualization, pp. 63–70, 1998.
  * [16] D. S. Dyer.  A dataflow toolkit for visualization.  IEEE Computer Graphics and Applications, 10(4):60–69, 1990.
  * [17] J. Heer and M. Agrawala.  Software design patterns for information visualization.  IEEE Transactions on Visualization and Computer Graphics, 12(5):853–860, 2006.
  * [18] J. Heer and B. Shneiderman.  Interactive dynamics for visual analysis.  Communications of the ACM, 55(4):45–54, 2012.
  * [19] A. Inselberg.  Multidimensional detective.  In Proceedings of IEEE Symposium on Information Visualization, pp. 100–107, 1997.
  * [20] D. Keim, G. Andrienko, J.-D. Fekete, C. Görg, J. Kohlhammer, and G. Melançon.  Visual analytics: Definition, process, and challenges.  In Information Visualization, pp. 154–175. Springer, 2008.
  * [21] I. Koren and A. D. Singh.  Fault tolerance in vlsi circuits.  Computer, 23(7):73–83, 1990.
  * [22] B. Lee, C. Plaisant, C. S. Parr, J.-D. Fekete, and N. Henry.  Task taxonomy for graph visualization.  In Proceedings of the AVI workshop on beyond time and errors: novel evaluation methods for information visualization, pp. 1–5, 2006.
  * [23] B. Lorensen.  On the death of visualization.  In NIH/NSF Proc. Fall 2004 Workshop Visualization Research Challenges, vol. 1, 2004.
  * [24] S. McKenna, D. Mazur, J. Agutter, and M. Meyer.  Design activity framework for visualization design.  IEEE Transactions on Visualization and Computer Graphics, 20(12):2191–2200, 2014.
  * [25] T. Munzner.  A nested model for visualization design and validation.  IEEE Transactions on Visualization and Computer Graphics, 15(6):921–928, 2009.
  * [26] A. Rind, W. Aigner, M. Wagner, S. Miksch, and T. Lammarsch.  Task cube: A three-dimensional conceptual space of user tasks in visualization design and evaluation.  Information Visualization, 15(4):288–300, 2016.
  * [27] W. J. Schroeder, L. S. Avila, and W. Hoffman.  Visualizing with vtk: a tutorial.  IEEE Computer Graphics and Applications, 20(5):20–27, 2000.
  * [28] H.-J. Schulz, T. Nocke, M. Heitzler, and H. Schumann.  A design space of visualization tasks.  IEEE Transactions on Visualization and Computer Graphics, 19(12):2366–2375, 2013.
  * [29] L. Shi, H. Tong, J. Tang, and C. Lin.  Vegas: Visual influence graph summarization on citation networks.  IEEE Transactions on Knowledge and Data Engineering, 27(12):3417–3431, 2015.
  * [30] B. Shneiderman.  The eyes have it: A task by data type taxonomy for information visualizations.  In Proceedings of IEEE Symposium on Visual Languages, pp. 336–343, 1996.
  * [31] J. F. Sowa.  Conceptual structures: information processing in mind and machine.  1983\.
  * [32] J. J. Thomas and K. A. Cook.  Illuminating the Path: The Research and Development Agenda for Visual Analytics.  National Visualization and Analytics Center, 2005.
  * [33] E. R. Valiati, M. S. Pimenta, and C. M. Freitas.  A taxonomy of tasks for guiding the evaluation of multidimensional visualizations.  In Proceedings of the 2006 AVI workshop on beyond time and errors: novel evaluation methods for information visualization, pp. 1–6, 2006\.
  * [34] F. B. Viegas, M. Wattenberg, and J. Feinberg.  Participatory visualization with wordle.  IEEE Transactions on Visualization and Computer Graphics, 15(6):1137–1144, 2009.
  * [35] S. Wehrend and C. Lewis.  A problem-oriented classification of visualization techniques.  In Proceedings of IEEE Visualization, pp. 139–143, 1990.
  * [36] K. Wongsuphasawat, D. Smilkov, J. Wexler, J. Wilson, D. Mane, D. Fritz, D. Krishnan, F. B. Viégas, and M. Wattenberg.  Visualizing dataflow graphs of deep learning models in tensorflow.  IEEE Transactions on Visualization and Computer Graphics, 24(1):1–12, 2017.