# Computationally Efficient Reinforcement Learning:
Targeted Exploration leveraging Simple Rules

Loris Di Natale, Bratislav Svetozarevic, Philipp Heer, and Colin N. Jones This
research was supported by the Swiss National Science Foundation under NCCR
Automation, grant agreement 51NF40_180545, and in part by the Swiss Data
Science Center, grant agreement C20-13.L. Di Natale, B. Svetozarevic, and P.
Heer are with the Urban Energy Systems Lab, Empa, Dübendorf, Switzerland.
{loris.dinatale, bratislav.svetozarevic<EMAIL_ADDRESS>Di Natale
and C. N. Jones are with the Laboratoire d’Automatique, EPFL, Lausanne,
Switzerland<EMAIL_ADDRESS>
###### Abstract

Model-free Reinforcement Learning (RL) generally suffers from poor sample
complexity, mostly due to the need to exhaustively explore the state-action
space to find well-performing policies. On the other hand, we postulate that
expert knowledge of the system often allows us to design simple rules we
expect good policies to follow at all times. In this work, we hence propose a
simple yet effective modification of continuous actor-critic frameworks to
incorporate such rules and avoid regions of the state-action space that are
known to be suboptimal, thereby significantly accelerating the convergence of
RL agents. Concretely, we saturate the actions chosen by the agent if they do
not comply with our intuition and, critically, modify the gradient update step
of the policy to ensure the learning process is not affected by the saturation
step. On a room temperature control case study, it allows agents to converge
to well-performing policies up to $6-7\times$ faster than classical agents
without computational overhead and while retaining good final performance.

##  I Introduction

Despite its success in many applications [1, 2], model-free Reinforcement
Learning (RL), and in particular deep RL (DRL), usually suffer from data
inefficiency, i.e., they require a significant number of interactions with the
environment to converge [3, 4]. This stems from the necessity to explore the
state-action space to find optimal policies and leads to significant
computational costs. It also limits the deployment of DRL methods on physical
systems without pretraining in simulation: learning a building temperature
control policy from scratch can for example take years of data [5, 6].

To speed up the training of DRL agents, researchers have for example
investigated how to leverage expert demonstrations [7, 8], but this requires
access to an expert policy which is not always available in practice. Instead,
we postulate that prior knowledge of physical systems often allows us to
design simple rules that agents should follow a priori, such as “Do not heat
the room if it is already $26\text{\,}\mathrm{\SIUnitSymbolCelsius}$”; we
indeed know this action will always be suboptimal in that state, there is no
need for agents to explore its consequences.

In this paper, we hence propose modifications of actor-critic algorithms to
encode simple rules in RL agents, introducing state-dependent constraints on
the agents’ actions to restrict exploration to interesting regions of the
state-action space. In other words, the key idea is to avoid visiting state-
action pairs that are known to be suboptimal by the expert to accelerate the
convergence towards meaningful solutions and thus increase the efficiency of
(D)RL. Note that while state-dependent bounds were concurrently introduced in
[9], they are enforced a posteriori in the environment instead of directly
modifying the agent’s behavior and do not necessarily improve convergence. In
another line of work, prior knowledge successfully accelerated learning in
[10], but relying on fuzzy rather than direct rule integration.

###  I-A Constraining RL agents

To bound the decisions taken by RL agents, one typically defines some
constrained set of actions and either project the actions of the agents on
this set at each time step or switch to a fallback controller when needed [11,
12, 13, 14]. The main challenge with these operations is that they are usually
not differentiable and hence cannot be learned by RL agents, with the notable
exceptions of [15, 16, 17], who leveraged differentiable optimization layers
[15], modified the policy updates to account for projections [16], or derived
a closed-form solution of the projection step [17]. However, these methods
either entail additional computational burden [15, 16] or rely on a learned
linearization of the constraints [17]. Alternatively, one could also apply
tools from the safe RL literature [18, 19, 20], typically relying on
constrained policy optimization [21, 22]. However, this would again introduce
both engineering and computational overhead.

The complexity of the methods discussed above often stems from the fact that
they are designed to impose state constraints on DRL agents, which is a more
challenging problem in general since it leads to complex action bounds for the
agent at each step. Here, however, we argue that prior knowledge can
straightforwardly be used to accelerate the training of DRL agents through
simple state-dependent box constraints on their actions, which allows us to
leverage less computationally expensive tools.

To alleviate the issue of non-differentiability without increasing either the
engineering or the computational burden, Reward Shaping (RS) heuristics might
be used in various forms to penalize agents when constraints are violated, let
them know when a fallback controller was used or they were saturated, or
introduce prior knowledge about the task to solve [14, 23, 24]. While such
methods might accelerate the learning process to some extent, they are however
indirect, i.e., they only influence the learned policies through the reward
function that the agent will learn to optimize over time. Moreover, shaping
the reward function simultaneously impacts the learning process of both the
actor and the critic.

###  I-B Contribution

In this work, we propose to clip DRL agents’ actions according to simple
expert-designed state-dependent bounds and then modify the gradient update
step of the actor to let agents learn from their mistakes and accelerate their
convergence to expected actions. To explain the effectiveness of the gradient
modification, we provide an intuitive analytical analysis of its impact on the
learning of DRL agents. Importantly, contrary to RS, our modifications only
affect the actor, allowing the critic to learn the true Q-values.

Remarkably, our method bypasses the need for complex projection steps and does
not require access to a fallback controller or an expert policy. Moreover and
critically, the proposed modifications do not impact the computational
complexity of the algorithm, are straightforward to design and implement, and
can be coupled with any actor-critic algorithms. Note that, in contrast with
[10], where the expert knowledge is potentially overridden by the policy, our
method enforces the wanted behaviors on agents at all times.

The effectiveness of the proposed Efficient Agents (EAs) is demonstrated in
simulation on a room temperature control case study, where they converge up to
$6$ – $7$ times faster than classical ones and $2$ – $3$ times faster than RS-
based agents while retaining good final performance. This hints at how the
proposed modifications can provide a simple yet effective and computationally
inexpensive mean to leverage expert knowledge to accelerate DRL algorithms.

##  II Preliminaries

###  II-A Reinforcement Learning

At each time step $t$, given an observation $s_{t}$ of the state of the
environment, an RL agent chooses an action $a_{t}$. The environment then
transitions to $s_{t+1}$ according to the transition probabilities
$P(s_{t},a_{t})$ and sends the new state and the reward signal
$r(s_{t},a_{t})$ to the agent. The objective of any deterministic RL
algorithm111While the presented analyses deal with deterministic actor-critic
agents for clarity, the results can easily be extended to the stochastic case.
is to find a policy $\pi(s_{t})$ that maximizes the expected discounted
cumulative returns:

$\displaystyle J(\pi)=\mathbb{E}_{a_{t}\sim\pi(s_{t}),s_{t+1}\sim
P(s_{t},a_{t})}\left[\sum_{t=0}^{\infty}{\gamma^{t}r(s_{t},a_{t})}\right],$
(1)

where $\gamma$ is the discount factor trading off near- and long-term rewards,
and the initial state $s_{0}\sim\rho$ is sampled from the corresponding
initial distribution. With a slight abuse of notation on the expectation for
clarity, we can define the Q-function of any state-action pair $(s,a)$:

$\displaystyle
Q^{\pi}(s,a)=\mathbb{E}_{\pi}\left[\sum_{t=0}^{\infty}{\gamma^{t}r(s_{t},a_{t})|s_{0}=s,a_{0}=a}\right],$
(2)

which captures the expected returns when action $a$ is chosen in state $s$ and
the policy $\pi$ is followed thereafter.

In our simulations, we let agents explore the environment with the
$\epsilon$-greedy exploration strategy, which means we apply the following
action to the environment:

$a(s)=\text{clip}(\pi(s)+\epsilon,a^{\textit{low}},a^{\textit{up}}),\qquad\epsilon\sim\mathcal{N}(0,\sigma),$
(3)

where the noisy actions are clipped elementwise between $a^{\textit{low}}$ and
$a^{\textit{up}}$, the predefined action bounds from the environment, and
$\epsilon$ is the Gaussian exploration noise with a standard deviation of
$\sigma$. All the transition tuples $(s,a,r,s^{\prime})$ observed by the agent
are stored in a replay buffer.

###  II-B Actor-critic algorithms

In practice, policies and Q-functions are often parametrized with Neural
Networks (NNs) as $\pi_{\theta}$ and $Q_{\phi}$, respectively, leading to DRL,
and numerous algorithms have been developed to maximize (1) [25]. In this
work, we are interested in deterministic actor-critic methods stemming from
[26], where both the actor $\pi_{\theta}$ (also referred to as the policy) and
the critic $Q_{\phi}$ are optimized in parallel leveraging gradient descent.
While different flavors exist, most algorithms compute the gradient of the
critic using the Temporal Difference (TD) loss [26]:

$\displaystyle\hat{\nabla}_{\phi}Q_{\phi}$
$\displaystyle=\nabla_{\phi}\left[\frac{1}{|B|}\sum_{b=(s,a,r,s^{\prime})\in
B}{\left(Q_{\phi}(s,a)-y(b)\right)^{2}}\right],$ (4)

with
$y(b)=\left(r+\gamma\max_{a^{\prime}}Q_{\phi}(s^{\prime},a^{\prime})\right)$
and where a batch $B$ of past transitions is sampled from the replay buffer
and used to estimate expectations. Leveraging the policy gradient theorem
[27], one can similarly use the critic to estimate the actor gradient as:

$\displaystyle{\color[rgb]{0,0,0}\hat{\nabla}_{\theta}\pi_{\theta}}$
$\displaystyle=-{\color[rgb]{0,0,0}\nabla_{\theta}\left[\frac{1}{|B|}\sum_{s\in
B}{Q_{\phi}(s,\pi_{\theta}(s))}\right].}$ (5)

Note that these gradients are easily computed using automatic differentiation
when the actor and the critic are parametrized with NNs.

In this paper, we rely on the Twin Delayed Deep Deterministic (TD3) policy
gradient algorithm, which introduces a few modifications to limit the well-
known overestimation bias of Q-functions plaguing vanilla actor-critic
algorithms [28]. Remarkably, however, these adjustments do not impact the
actor gradient in (5), allowing us to seamlessly integrate the proposed
modifications detailed in Section III.

##  III Methods

###  III-A State-dependent action saturation

In many cases, prior knowledge allows us to design state-dependent upper and
lower bounds $a^{\textit{max}}(s)$ and $a^{\textit{min}}(s)$, respectively, on
the actions we expect well-performing control policies to take in a given
state $s$, with:

$\displaystyle\quad a^{\textit{low}}\leq a^{\textit{min}}(s)\leq
a^{\textit{max}}(s)\leq a^{\textit{up}}.$ (6)

To limit the exploration of known suboptimal state-action pairs, we can then
modify (3) accordingly to:

$\displaystyle a(s)$
$\displaystyle=\text{clip}(\pi_{\theta}(s)+\epsilon,a^{\textit{min}}(s),a^{\textit{max}}(s)).$
(7)

Note that these bounds stemming from prior knowledge are also enforced at test
time when $\epsilon=0$ to ensure an agent would never turn left if whenever
there is a wall in that direction, for example, neither during the training
nor the deployment phase.

###  III-B Actor gradient modification

The major problem with the clipping operation in (7) is its non-
differentiability. Worse yet, its subdifferentials go to zero whenever agents
are saturated (see (9) in Section III-C), making any backward flow of
information on the overriding process impossible. As a countermeasure, to let
agents learn from their mistakes, we also modify the actor gradient (5) to:

$\displaystyle\hat{\nabla}^{\textit{EA}}_{\theta}\pi_{\theta}=-\nabla_{\theta}\Bigg{(}$
$\displaystyle\frac{1}{|B|}\sum_{(s,a)\in
B}\bigg{[}{Q_{\phi}(s,\pi_{\theta}(s))}$
$\displaystyle-\frac{\lambda}{2}\left(\pi_{\theta}(s)-a(s)\right)^{2}\bigg{]}\Bigg{)},$
(8)

where $\lambda$ is a hyperparameter. The last term in (8) penalizes actions
chosen by the policy $\pi_{\theta}(s)$ if they deviate from the constrained
action $a(s)$ that was applied to the environment, thus steering the agent’s
decisions towards expected actions.222In another line of work, this penalty
was also used in [15] to improve the robustness of differentiable layer-based
RL for state-constrained problems.

Alternatively, we note here that one could instead modify the reward function
to include this penalty (RS) and then maximize (1). Remarkably, however, the
latter also impacts the learning process of the critic in (4) when applied to
actor-critic frameworks, contrary to our method. We will show empirical
benefits of the proposed modification (8) over RS-based penalties in terms of
convergence speed in Section V.

###  III-C Implications of the modified gradients

Let $C(s)=\left\\{a\in\mathbb{R}:a^{\textit{min}}(s)\leq a\leq
a^{\textit{max}}(s)\right\\}$ for any given state $s$.333Without loss of
generality, we assume that $a\in\mathbb{R}$ in this section for clarity. This
assumption can easily be lifted for multi-dimensional problems. Grouping all
the parameters $\theta$ in a vector and recalling the definition of the action
$a(s)$ applied to the environment in state $s$ from (7), we can define its
subgradient $\nabla_{\theta}a(s)$ as:

$\displaystyle a(s)$ $\displaystyle=\begin{cases}a^{min}(s),&\text{if
}\pi(s)<a^{min}(s),\\\ \pi_{\theta}(s)+\epsilon,&\text{if }\pi_{\theta}(s)\in
C(s),\\\ a^{max}(s),&\text{if }\pi(s)>a^{max}(s).\end{cases}$
$\displaystyle\implies\nabla_{\theta}a(s)$
$\displaystyle=\begin{cases}\nabla_{\theta}\pi_{\theta}(s),&\text{if
}\pi_{\theta}(s)\in C(s),\\\ 0,&\text{else,}\end{cases}$ (9)

where $\nabla_{\theta}\pi_{\theta}(s)$ is the actor gradient. We can then
rewrite the gradient of EAs (8) as:

$\displaystyle\hat{\nabla}^{\textit{EA}}_{\theta}\pi_{\theta}$
$\displaystyle=-\frac{1}{|B|}\sum_{(s,a)\in
B}\bigg{[}{\nabla_{\theta}Q_{\phi}(s,\pi_{\theta}(s))}$
$\displaystyle\qquad\quad-\nabla_{\theta}\left(\frac{\lambda}{2}\left(\pi_{\theta}(s)-a(s)\right)^{2}\right)\bigg{]}$
$\displaystyle=-\frac{1}{|B|}\sum_{(s,a)\in
B}\bigg{[}\nabla_{\theta}Q_{\phi}(s,\pi_{\theta}(s))$
$\displaystyle\qquad\quad-\left(\lambda\left(\nabla_{\theta}\pi_{\theta}(s)-\nabla_{\theta}a(s)\right)^{\top}\left(e_{\theta}(s)\right)\right)\bigg{]},$

where we introduce the error term $e_{\theta}(s)=\pi_{\theta}(s)-a(s)$. We
hence get the following modified actor gradient, where we omit $(s,a)\in B$
for clarity:

$\displaystyle\hat{\nabla}^{\textit{EA}}_{\theta}\pi_{\theta}$
$\displaystyle=\begin{cases}-\frac{1}{|B|}\sum_{B}\Big{[}\nabla_{\theta}Q_{\phi}(s,\pi_{\theta}(s))\Big{]},\
\text{if }\pi_{\theta}(s)\in C(s),\\\
-\frac{1}{|B|}\sum_{B}\Big{[}\nabla_{\theta}Q_{\phi}(s,\pi_{\theta}(s))\\\
\qquad\quad\
-\lambda\nabla_{\theta}\pi_{\theta}(s)^{\top}e_{\theta}(s)\Big{]},\
\text{else.}\end{cases}$

Remarkably, the additional penalty term in (8) hence allows us to solve the
issue of the subdifferentials of the clipping operator being zero when actions
are saturated, modifying the gradients only when the constraints are not met.
Indeed, as long as the action chosen by the agent respects the constraints
provided by the expert, the classical gradient (5) is used. On the other hand,
as soon as the constraints are not met, the gradient is modified in the
direction $e_{\theta}(s)$ to accelerate the convergence of $\pi_{\theta}(s)$
to $C{\color[rgb]{0,0,0}(s)}$ despite the subdifferential of the clipped
action being zero, confirming the graphical intuition from [15, Fig. 2]. This
allows EAs to learn from their mistakes and — we hypothesize — helps them
rapidly converge to meaningful policies.

##  IV Room temperature control case study

To assess the effectiveness of the proposed method, we apply it to a
temperature control case study, where the objective is to minimize the energy
consumption of a room while maintaining the comfort of the occupants,
represented by predefined temperature bounds that should not be exceeded.

###  IV-A Reinforcement Learning framework

The continuous action space of the agents corresponds to how much heating or
cooling power, should be applied at each time step, normalized between
$a^{\textit{low}}=-1$ and $a^{\textit{up}}=1$. During the heating season,
$a^{\textit{low}}$ corresponds to the heating being turned off and
$a^{\textit{up}}$ to full heating, and the contrary in the cooling case.
Physically Consistent Neural Networks (PCNNs) [29] are used to simulate one
bedroom in the NEST building [30] and $s_{t}$ gathers time, weather,
temperature, and comfort bound information (see [6] for details). The reward
function is defined as the negative weighted sum of energy consumption $E_{t}$
and comfort violations, i.e. how far from the designed bounds the temperature
inside the room is:

$\displaystyle r(s_{t},a_{t})$
$\displaystyle=-\max{\\{L_{t}-T_{t},T_{t}-U_{t},0\\}}-\alpha E_{t},$ (10)
$\displaystyle E_{t}$ $\displaystyle=\begin{cases}\frac{a_{t}+1}{2}\
E^{\textit{max}}_{\textit{heat}},&\text{in the heating season},\\\
\frac{1-a_{t}}{2}\ E^{\textit{max}}_{\textit{cool}},&\text{in the cooling
season}.\end{cases}$

where $L_{t}$ and $U_{t}$ represent the lower and upper comfort bounds on the
temperature $T_{t}$ at time $t$, respectively, $\alpha$ is a weighting factor,
and $E^{\textit{max}}_{\textit{heat}}$ and $E^{\textit{max}}_{\textit{cool}}$
stand for the maximal heating and cooling power, respectively.

###  IV-B Design of the saturation rules

In the context of room temperature control, we intuitively know that an
optimal policy should gradually stop heating when the temperature reaches the
upper comfort bound and gradually start heating as soon as the lower bound is
not met (and vice versa for cooling). To encode these simple rules, we design
state-dependent action bounds as follows:

$\displaystyle a^{\textit{min}}(s_{t})$
$\displaystyle=\text{clip}\left(\frac{(L_{t}-m)-T_{t}}{n-m},\ 0,\
1\right)^{2}*2-1$ (11) $\displaystyle a^{\textit{max}}(s_{t})$
$\displaystyle=1-2*\text{clip}\left(\frac{T_{t}-(U_{t}+m)}{n-m},\ 0,\
1\right)^{2},$ (12)

with $n\geq m\geq 0$ representing design parameters to leave more or less
freedom to the agents. In words, we start constraining the action of the
agents as soon as the temperature deviates from the bounds for more than $m$
degrees and then quadratically increase the constraint until $n$ degrees have
been reached, where the agent is forced to use the maximum or minimum power,
as pictured in Fig. 1.

Figure 1: Representation of the action bounds used in this work.

##  V Results

To investigate the influence of $m$ and $n$, which measure how much prior
knowledge is transmitted to DRL agents, we train different EAs (EA $m$ / $n$).
For comparison purposes, we also train agents with the classical actor
gradient (5), introducing the additional squared penalty
$\frac{\lambda}{2}\left(\pi_{\theta}(s)-a(s)\right)^{2}$ in the reward
function instead as another computationally inexpensive means to incorporate
prior knowledge in agents (RS $m$ / $n$). Finally, we also analyze two
classical DRL agents with different random seeds (Classical $1$ and
$2$).444The code and data are available on https://gitlab.nccr-
automation.ch/loris.dinatale/efficient-drl.

###  V-A Final performance

All the agents were trained on up to three-day-long episodes randomly sampled
from three years of data. They were evaluated after each $96$ steps of
$15\text{\,}\mathrm{min}$, i.e. one day’s worth of data, hereafter also
referred to as an epoch, on a testing set of $50$ unseen sequences of three
days. They all use the same hyperparameters as in [6]. We manually set
$\lambda=100$ for EAs to ensure the constraints are enforced as fast as
possible and $\lambda=10$ for RSs since larger penalties led to instability.
While we empirically observed more robust performance of EAs with respect to
$\lambda$ compared to RSs, a complete sensitivity analysis is left for future
work.

The best reward obtained by all the trained agents over the first $500$ epochs
can be found in Table I, and the corresponding trade-off between energy
consumption and comfort violations is plotted in Fig. 2. These results
illustrate how tighter parameters $m$ and $n$, i.e., higher levels of prior
knowledge, allow EAs and RSs to converge to better solutions in this limited
training regime. In particular, it allows EAs to reduce the amount of comfort
violations without significantly increasing the energy consumption. Classical
DRL agents on the other hand usually use less energy at the cost of additional
comfort violations in this early phase of learning before converging to near-
optimal solutions after longer training times [6].

TABLE I: Best reward obtained by each agent on the test set over the first $500$ epochs. Agent | Rew. | Agent | Rew. | Agent | Rew.
---|---|---|---|---|---
Classical 1 | -2.64 | Classical 2 | -2.75 |  |
RS $0$ / $1$ | -2.58 | EA $0$ / $1$ | -2.85 | EA $0.5$ / $1$ | -2.83
RS $0$ / $0.5$ | -2.44 | EA $0$ / $0.5$ | -2.58 | EA $0.25$ / $0.5$ | -2.74
RS $0$ / $0.25$ | -2.37 | EA $0$ / $0.25$ | -2.51 | EA $0.2$ / $0.25$ | -2.62
RS $0$ / $0.1$ | -3.69 | EA $0$ / $0.1$ | -2.46 | EA $0.075$ / $0.1$ | -2.46

Figure 2: Average energy consumption (En.) and comfort violations (Vio.) over
the test set corresponding to each agent in Table I. The performance of two
industrial baselines, of an agent trained for 125,000 epochs (Best Agent), and
the optimal performance achievable (Optimum), all computed as in [6], are
reported in gray.

###  V-B Visualization of the impact of prior knowledge

Figure 3: Behavior of a classical agent and EAs with various $m$ and $n$
parameters (ours) minimizing the heating power consumption (bottom) while
maintaining the temperature in the grey dotted bounds (top). Left: Performance
before training, where EAs are saturated once they exceed the bounds by $n$
degrees. Right: Performance after training, showing how all agents converged
to similar solutions (Table II).

To intuitively understand the effect of action saturation, we visualize its
impact on some EAs in Fig. 3, where the behavior of all agents is plotted
before training on the left, and after on the right, for the same three days
during the heating season. Focusing on the left plot, we see the untrained
classical DRL agent in black letting the temperature diverge to an
uncomfortably high range (out of the bounds of the plot) as it starts
exploring the state space using roughly constant heating power. On the other
hand, all the EAs are forced to stop heating once they are $n$ degrees out of
bounds. Consequently, even before training, such agents will not overheat the
room and keep it at acceptable temperatures for the occupants, corresponding
to what we expect from good controllers. However, note that EAs can present
control input oscillations due to the impact of external disturbances, mainly
the solar gains around noon, triggering the saturation mechanism on and off.

On the right plot, after training, one can observe that all EAs generally take
comparable decisions — still being sometimes saturated, which ensures
compliance with prior expert knowledge — leading to similar temperature
patterns. On the other hand, the classical agent presents a slightly different
behavior, with smoother decision patterns. Interestingly, this agent is the
only one heating in the early afternoon, while the EAs wait until the end of
the afternoon to heat the room with high power and meet the comfort bound
tightening at $8$pm. This allows the classical agent to use less energy than
EAs over these three days but can incur additional comfort violations (Table
II), as expected from Fig. 2.

TABLE II: Reward, sum of comfort violations (Vio.), and energy consumption (En.) of each agent over the three days depicted on the right of Fig. 3. Agent | Classical 1 | EA (ours)
---|---|---
$m$ / $n$ | - |  $0$ / $1$ |  $0$ / $0.5$ |  $0$ / $0.25$
Reward | -0.68 | -0.69 | -0.89 | -0.60
Vio. [Kh] | 1.28 | 1.18 | 2.23 | 0.65
En. [kWh] | 5.03 | 5.38 | 5.54 | 5.46

###  V-C Data efficiency of the proposed gradient modification

Figure 4: Convergence speed of various EAs with different $m$ and $n$
parameters (ours), compared to agents using RS and two classical agents in
black (one in the top plots, one in the bottom ones). The vertical lines and
annotations specify the number of days of data required to obtain a reward of
$-2.95$ for each agent, which corresponds to the performance of two industrial
rule-based baselines.

A comparison of the convergence speed of various agents over the first $300$
epochs is plotted in Fig. 4, where the vertical lines and annotations
illustrate the number of days required to attain performance on par with rule-
based on-off industrial baselines from [6]. In general, we observe that all
the EAs attain returns on par with the baselines significantly earlier than
classical DRL agents, in as little as $29$ days instead of roughly $200$, an
improvement of almost an order of magnitude. In particular, the smaller $n$ is
chosen (from left to right in Fig. 4), the faster the convergence of the EAs
in green and blue. Intuitively, this makes sense, as tighter constraints
introduce more prior knowledge to the EAs, thereby allowing them to find
interesting solutions faster, without losing time exploring suboptimal state-
action pairs. On the other hand, the influence of $m$ is less marked, with
$m\neq 0$ (blue) and $m=0$ (green) leading to very similar convergence
patterns in the bottom row of plots in Fig. 4.

Remarkably, RS does not seem to drastically speed the training up in this case
study (red). While RS $0$ / $0.25$ does converge twice as fast as the
classical DRL agents, RS $0$ / $0.1$ does not converge at all, hinting at the
fragility of RS in general. Even when they converge, RSs are still two to
three times slower than their EA counterparts. On the other hand, RS seems to
lead to more consistent performance than classical agents and EAs after a few
hundred epochs, which is confirmed by their good final performance in Fig. 2.

Overall, these results support our claim that, as long as the rules provided
to the agents are well-defined and correspond to expected behaviors, the
proposed modifications can indeed greatly accelerate the convergence of DRL
agents. Critically, this does not significantly impact the quality of the
final solution (Sec. V-A). Interestingly, incorporating more specific expert
knowledge in EAs — through smaller $m$ and $n$ — further accelerates their
convergence. This corresponds to our intuition: better-defined rules help
agents more. Remarkably, the modifications proposed in Sec. III provide the
desired speedup for a wide variety of parameters $m$ and $n$, contrary to RS,
hinting at the robustness of the proposed scheme.

##  VI Conclusion

Starting from the postulate that prior expert knowledge often gives us an
intuition of how good control policies should behave, we presented a scheme to
encode it in actor-critic frameworks through simple rules to accelerate
learning and decrease the associated computational load. These rules take the
form of bounds on the agent’s actions that can directly be enforced during
training and online operations. To ensure agents learn from their mistakes, we
also modified the actor gradients to steer control policies towards expected
behaviors, limiting the exploration of known suboptimal state-action pairs.
Critically, both these operations are computationally inexpensive, ensuring
the gains in sample complexity positively impact the training time of the
agents.

On a room temperature control case study, this scheme allowed us to accelerate
the convergence speed of DRL agents by up to $6$ – $7\times$. Furthermore,
modifying actor gradients proved to be $2$ – $3$ times more effective and more
robust than the widespread reward shaping method. Remarkably, this was done
without suffering from a significant drop in the final performance of the
control policies, illustrating how prior knowledge can help alleviate the
computational burden of DRL. This represents an interesting first step towards
efficient agents that can be deployed and learned from scratch on physical
systems, potentially bypassing the need for complex simulators. In future
work, it would be interesting to investigate annealing strategies on $\lambda$
or leverage primal-dual optimization tools to adaptively tune the influence of
the additional penalty in the actor gradient and let agents learn more
expressive policies after the initial exploration phase.

## References

  * [1] A. Coronato, M. Naeem, G. De Pietro, and G. Paragliola, “Reinforcement learning for intelligent healthcare applications: A survey,” _Artificial Intelligence in Medicine_ , vol. 109, p. 101964, 2020.
  * [2] Z. Zhang, D. Zhang, and R. C. Qiu, “Deep reinforcement learning for power system applications: An overview,” _CSEE Journal of Power and Energy Systems_ , vol. 6, no. 1, pp. 213–225, 2019.
  * [3] Y. Duan, J. Schulman, X. Chen, P. L. Bartlett, I. Sutskever, and P. Abbeel, “RL2: Fast reinforcement learning via slow reinforcement learning,” _arXiv preprint arXiv:1611.02779_ , 2016.
  * [4] M. Schwarzer, N. Rajkumar, M. Noukhovitch, A. Anand, L. Charlin, R. D. Hjelm, P. Bachman, and A. C. Courville, “Pretraining representations for data-efficient reinforcement learning,” _Advances in Neural Information Processing Systems_ , vol. 34, 2021.
  * [5] Z. Wang and T. Hong, “Reinforcement learning for building controls: The opportunities and challenges,” _Applied Energy_ , vol. 269, p. 115036, 2020\.
  * [6] L. Di Natale, B. Svetozarevic, P. Heer, and C. N. Jones, “Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control,” _arXiv preprint arXiv:2203.05434_ , 2022.
  * [7] A. Nair, B. McGrew, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Overcoming exploration in reinforcement learning with demonstrations,” in _2018 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2018, pp. 6292–6299.
  * [8] T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, I. Osband _et al._ , “Deep q-learning from demonstrations,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 32, no. 1, 2018.
  * [9] B. De Cooman, J. Suykens, and A. Ortseifen, “Enforcing Hard State-Dependent Action Bounds on Deep Reinforcement Learning Policies,” in _Machine Learning, Optimization, and Data Science: 8th International Workshop, LOD 2022, Certosa di Pontignano, Italy, September 19–22, 2022, Revised Selected Papers, Part II_. Springer, 2023, pp. 193–218.
  * [10] P. Zhang, J. Hao, W. Wang, H. Tang, Y. Ma, Y. Duan, and Y. Zheng, “KoGuN: accelerating deep reinforcement learning via integrating human suboptimal knowledge,” _arXiv preprint arXiv:2002.07418_ , 2020.
  * [11] R. Wang, X. Zhang, X. Zhou, Y. Wen, and R. Tan, “Toward Physics-Guided Safe Deep Reinforcement Learning for Green Data Center Cooling Control,” in _2022 ACM/IEEE 13th International Conference on Cyber-Physical Systems (ICCPS)_. IEEE, 2022, pp. 159–169.
  * [12] H. Mao, M. Schwarzkopf, H. He, and M. Alizadeh, “Towards safe online reinforcement learning in computer systems,” in _33rd conference on neural information processing systems (NeurIPS 2019)_ , 2019.
  * [13] R. Bautista-Montesano, R. Galluzzi, K. Ruan, Y. Fu, and X. Di, “Autonomous navigation at unsignalized intersections: A coupled reinforcement learning and model predictive control approach,” _Transportation Research Part C: Emerging Technologies_ , vol. 139, p. 103662, 2022.
  * [14] H. H. Goh, Y. Huang, C. S. Lim, D. Zhang, H. Liu, W. Dai, T. A. Kurniawan, and S. Rahman, “An Assessment of Multi-Stage Reward Function Design for Deep Reinforcement Learning-Based Microgrid Energy Management,” _IEEE Transactions on Smart Grid_ , 2022.
  * [15] B. Chen, P. L. Donti, K. Baker, J. Z. Kolter, and M. Bergés, “Enforcing policy feasibility constraints through differentiable projection for energy optimization,” in _Proceedings of the Twelfth ACM International Conference on Future Energy Systems_ , 2021, pp. 199–210.
  * [16] S. Gros, M. Zanon, and A. Bemporad, “Safe reinforcement learning via projection on a safe set: How to achieve optimality?” _IFAC-PapersOnLine_ , vol. 53, no. 2, pp. 8076–8081, 2020.
  * [17] G. Dalal, K. Dvijotham, M. Vecerik, T. Hester, C. Paduraru, and Y. Tassa, “Safe exploration in continuous action spaces,” _arXiv preprint arXiv:1801.08757_ , 2018.
  * [18] P. Osinenko, D. Dobriborsci, and W. Aumer, “Reinforcement learning with guarantees: a review,” _IFAC-PapersOnLine_ , vol. 55, no. 15, pp. 123–128, 2022.
  * [19] S. Gu, L. Yang, Y. Du, G. Chen, F. Walter, J. Wang, Y. Yang, and A. Knoll, “A Review of Safe Reinforcement Learning: Methods, Theory and Applications,” _arXiv preprint arXiv:2205.10330_ , 2022.
  * [20] J. Garcıa and F. Fernández, “A comprehensive survey on safe reinforcement learning,” _Journal of Machine Learning Research_ , vol. 16, no. 1, pp. 1437–1480, 2015.
  * [21] T. D. Simão, N. Jansen, and M. T. Spaan, “AlwaysSafe: Reinforcement learning without safety constraint violations during training,” in _Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems_. International Foundation for Autonomous Agents and Multiagent Systems, 2021.
  * [22] J. Achiam, D. Held, A. Tamar, and P. Abbeel, “Constrained policy optimization,” in _International conference on machine learning_. PMLR, 2017, pp. 22–31.
  * [23] M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu, “Safe reinforcement learning via shielding,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 32, no. 1, 2018.
  * [24] Y. Hu, W. Wang, H. Jia, Y. Wang, Y. Chen, J. Hao, F. Wu, and C. Fan, “Learning to utilize shaping rewards: A new approach of reward shaping,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 15 931–15 941, 2020.
  * [25] OpenAI, “Part 2: Kinds of RL Algorithms,” 2018, accessed: 25.11.2022. [Online]. Available: https://spinningup.openai.com/en/latest/spinningup/rl_intro2.html#a-taxonomy-of-rl-algorithms
  * [26] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” _arXiv preprint arXiv:1509.02971_ , 2015.
  * [27] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” in _International conference on machine learning_. PMLR, 2014, pp. 387–395.
  * [28] S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in _International Conference on Machine Learning_. PMLR, 2018, pp. 1587–1596.
  * [29] L. Di Natale, B. Svetozarevic, P. Heer, and C. N. Jones, “Physically consistent neural networks for building thermal modeling: theory and analysis,” _Applied Energy_ , vol. 325, p. 119806, 2022.
  * [30] Empa, “NEST,” 2022, accessed: 25.11.2022. [Online]. Available: https://www.empa.ch/web/nest/overview


# Du Bois Wrapped Bar Chart: Visualizing categorical data with
disproportionate values

Alireza Karduni1, Ryan Wesslen1, Isaac Cho1,2, Wenwen Dou1

1University of North Carolina at Charlotte, $<$akaduni, rwesslen,
<EMAIL_ADDRESS>2North Carolina A&T State University<EMAIL_ADDRESS>
###### Abstract

We propose a visualization technique, Du Bois wrapped bar chart, inspired by
work of W.E.B Du Bois. Du Bois wrapped bar charts enable better large-to-small
bar comparison by wrapping large bars over a certain threshold. We first
present two crowdsourcing experiments comparing wrapped and standard bar
charts to evaluate (1) the benefit of wrapped bars in helping participants
identify and compare values; (2) the characteristics of data most suitable for
wrapped bars. In the first study (n=98) using real-world datasets, we find
that wrapped bar charts lead to higher accuracy in identifying and estimating
ratios between bars. In a follow-up study (n=190) with 13 simulated datasets,
we find participants were consistently more accurate with wrapped bar charts
when certain category values are disproportionate as measured by entropy and
H-spread. Finally, in an in-lab study, we investigate participants’ experience
and strategies, leading to guidelines for when and how to use wrapped bar
charts.

###### keywords:

bar chart; graphical perception; user study; evaluation; Mechanical Turk;
crowdsourcing; Information visualization

<ccs2012> <concept> <concept_id>10003120.10003145.10011769</concept_id>
<concept_desc>Human-centered computing Empirical studies in
visualization</concept_desc> <concept_significance>500</concept_significance>
</concept> <concept>
<concept_id>10003120.10003145.10003147.10010923</concept_id>
<concept_desc>Human-centered computing Information
visualization</concept_desc> <concept_significance>300</concept_significance>
</concept> </ccs2012>

[500]Human-centered computing Empirical studies in visualization [300]Human-
centered computing Information visualization

##  1 Introduction

The concept of a bar chart was first introduced by French scientist Nicole
Oresme in the 14th century [5, 16]. However, many sources attribute the wide
adoption of bar charts to the 1786 seminal work by William Playfair on
“Exports and Imports of Scotland to and from different parts for one Year from
Christmas 1780 to Christmas 1781” [27, 32].

The modern definition of a bar chart is “a chart that presents categorical
data with rectangular bars with heights or lengths proportional to the values
that they represent” [30]. Bar charts are now considered one of the most
popular and prolific visualization techniques for communicating categorical
values [28, 36]. However, certain data characteristics such as
disproportionally large and small values make performing certain tasks with
bar charts challenging. Figure 1 shows three examples we identified from news
articles [22, 8, 10] that use bar charts to visualize categorical values111We
recreated the charts from the original data due to copyright issues.. All
three examples are characterized by one disproportionately large value, which
induces a significant white space-to-data ratio in the chart’s plane.
Estimating the smallest values and the ratio of largest to smallest values is
challenging with charts like these. To address these challenges, we propose a
new technique called a “wrapped bar chart”.

Figure 1: Three recreated examples of bar charts based on data and charts in
news articles [22, 8, 10]. All three charts are characterized by one extremely
disproportionate value, leaving the smaller values difficult to estimate and
compare.

Our work is inspired by the work of William Edward Burghardt “W. E. B.” Du
Bois, a sociologist, historian, activist, and author who was also a prolific
graphic designer [2]. His work for the 1900 Exposition Universelle in Paris
utilized hand-drawn data visualizations to focus on the African American
population in the 1890’s South [4]. Du Bois explored many different types of
visualization techniques including using bar charts to highlight inequities
for African Americans in Georgia. For example, African Americans worked
disproportionately in agriculture occupations such as laborers, farmers, and
planters but rarely in manufacturing and professional occupations like
engineers, masons, merchants, and barbers. This extreme difference made
drawing African-American occupation counts with a standard bar chart to be
ineffective, especially when comparing the largest occupation (bar) to the
smallest occupation (bar). To address this problem, he introduced an
innovative solution to his bar charts: to wrap the tallest bars around
themselves, allowing more room to show smaller bars. Figure 2 presents three
charts by Du Bois that include a “wrapped” bar for its largest data value.

Figure 2: Three examples of wrapped bar charts designed by Du Bois [2]. Images
assessed from Library of Congress. Images courtesy of Library of Congress,
Prints & Photographs Division, LC-USZ62-1234

Present-day visualization practitioners attempt to deal with the problem of
disproportionate categorical data in bar charts through ad hoc solutions that
have known issues. For example, one common solution is “breaking the axis” by
interrupting the vertical axis of a chart at a specific point through a break.
Similarly, cut-off bars limit the upper bound of the axis. However, these
techniques are known to mislead users [9]. While the two aforementioned
techniques use a linear scale, other studies have analyzed scientific data
with a non-linear scale (e.g. logarithmic) when data values cover a large
range of magnitudes [7]. However, to our knowledge Du Bois’ wrapping technique
has not been studied by the visualization community to evaluate its
effectiveness. Our work is to explore the potential and usages of Du Bois’
intriguing innovation. To this aim, we first introduce a technique for
developing wrapped bar charts in web-based visualizations. Second, we conduct
three experiments to explore the limitations, potentials and benefits of
wrapped bar charts in helping users conduct data analysis tasks in datasets
with large variance between data values. Our paper makes the following
contributions:

  * •

We design and develop the Du Bois wrapped bar chart222The similar term
“wrapped bar graph” has been used for a different visualization technique.
Stephen Few [18] introduced this term for a chart that splits the sorted bars
in a horizontal bar chart into multiple columns to eliminate the need for
scrolling [25]. In contrast, the Du Bois wrapped bar chart wraps
disproportionately large bars so that small values are discernible.  inspired
by Du Bois’ work for web-based visualizations using D3.js.

  * •

We conduct two online crowd-sourced experiments for comparing participants’
performance with wrapped bar charts versus standard bar charts in
identification and ratio estimation tasks, using accuracy and time spent to
complete each task as metrics of performance.

  * •

We conduct an in-lab focus group study that focuses on gathering information
about participants’ strategies when using wrapped bar chart, as well as
collecting suggestions to improve the wrapped bar design.

In the following sections, we review related work, design of wrapped bar
charts, experiment design, discussion of results, and conclude with
limitations and future works.

##  2 Related Work

###  2.1 Graphical Perception Studies of Bar Charts

Many empirical studies have been conducted on understanding users’ perception
of the visual encoding of bar charts [14, 20]. Findings from these experiments
provide design guidelines and considerations for applying bar charts to
various tasks such as difference comparison and proportion estimation. The bar
chart experiment by Cleveland & McGill studied participants’ estimate of the
proportion of the lengths of two bars [14]. The experiment included 10
different pairs of bars with proportion values ranging from 17.8% to 82.5%.
Participants’ responses were measured by the average log absolute error - the
difference between a participant’s answer and the true percentage value. The
analysis results revealed that the accuracy was affected by multiple factors:
the responses were more inaccurate with the stacked bar charts than standard
bar charts; the responses were most inaccurate for true percents around 60% -
80%; and the accuracy decreased with increasing space between the two bars.

As an effort to study crowdsourcing for perceptual experiments, Heer & Bostock
conducted an approximate replication of Cleveland & McGill’s bar chart study
with slight modifications on the true percents [20]. The new results are
consistent with the findings from the original study although the log absolute
error results are better in the new study, possibly due to the true percents
being rounded to whole numbers instead of numbers with decimals. More
recently, through a series of follow-up experiments, Talbot et al. [31] aimed
to explore the results from Cleveland & McGill’s experiment in order to
explain the sources of bar chart interpretation error. Their findings –
including evidence that shorter bars are more difficult to compare – suggest
that additional studies are needed to provide a more complete understanding of
what impacts (bar) chart perception. Talbot et al. also found that the
separation (gap) between bars increase bar comparison difficulty but the
effect of intervening distractor bars are small. More recently, Zhao et al.
[36] conducted experiments to evaluate how the perception of a bar changes
based on the heights of its neighboring bars and found the neighborhood effect
does exist.

Our study is inspired by the design and findings from the aforementioned
graphical perception experiments. More specifically, the tasks in our study
include the ratio estimate and identification tasks that require comparison of
bars appearing both similar and orders-and-magnitude different in heights. A
notable difference between the ratio estimation task in this study and
previous studies on bar charts is that we chose to ask participants to
calculate the ratio of largest/smallest bars as opposed to the percentage of
smallest/largest bars. While the smallest percentage (small/large) evaluated
by Cleveland et al. and Heer et al. was 17.8%, our study would involve
estimating percentages as small as 0.2% and 0.24% if framed as small/large
estimation seen in the previous studies.

###  2.2 Evaluating Alternative Bar Chart Designs

Multiple studies have evaluated alternative bar chart designs. Skau et al.
[26] introduced different illustrative embellishments which are design
alternatives of a (single-series) bar chart. Skau et al.’s alternative bar
chart designs drew inspiration from visual embellishments [6] which provide
non-linguistic rhetorical figures that are used frequently in the
visual/performing arts, advertisements, graphical user interfaces, etc.
Srinivasan et al. [29] introduced design variants of the multi-series bar
chart. Both studies evaluated the performance on the task of comparing values
between bars. Both studies added additional information to single- and multi-
series bars such as embellishments [26] and difference overlay [29]. However,
such additions do not affect how the bars are plotted.

Y-axis Distortion. In comparison, another suite of alternative bar chart
designs distort the Y axis in a bar chart thus affects how the length of the
bars are determined and visually presented. Cut-off bars, scale break [12],
and logarithmic scale are methods that were proposed to address problems
classic linear bar charts may have in displaying disproportionately large
values. These methods distort the Y-axis with a either non-linear scale or
omitting a value range on the Y-axis. Cleveland discussed some of the
techniques involving Y-axis distortion and called for experimentation for
improving graphical communication in science [13]. Hlawatsch et al. noted that
these distortions introduce a “lie factor” [34] when there is discrepancy of
effect size between the data and its representation [21]. The distortions will
result in misleading scenarios for quantitative comparisons, which is the most
common task for classic linear bar charts [9]. More recently, Borgo et al. [7]
proposed and evaluated an alternative bar chart design called Order of
Magnitude Marker (OOMM) to facilitate the task of large magnitude number
detection. The OOMM technique used a normalized scientific notation $A\times
B^{10}$, with B determining the Y-axis scale and interval. The authors
reported an empirical study (N=21) that demonstrates OOMM outperformed linear
and log-scale bar charts design in identification and ration tasks. Note that
the study did require participants to have basic knowledge of calculus and
familiarity with concepts such as graphs and logarithmic scale. Therefore, we
contend that OOMM and other techniques that distort the Y-axis may be better
suited for communicating scientific data but may not be as effective for
general audience [13, 12]. In contrast, the wrapped bar chart was originally
invented for exhibitions and aimed to communicate discrepancy between values
to general audience. For these reasons, our experiments evaluated the proposed
wrapped bar chart technique against standard linear bar chart, without
considering techniques that distort the Y-axis. Since the wrapped bar chart
employ the same linear scale, we refer to standard linear bar chart as
standard bar chart when reporting the experiments.

##  3 Design of Du Bois Wrapped Bar Charts

To develop wrapped bar charts inspired by Du Bois, we define two threshold
variables that determine where and how to wrap a bar. First, we define a
threshold $t1$ that determines where on the axis to wrap a bar for the first
time. For a value exceeding $t1$, the bar representing this value will be
wrapped. In terms of the wrapping direction, we decided to wrap down-to-up
(first example in Figure 2). This design choice allows room for wrapping
multiple times and ease of estimating the length of the wrapped portions.

Figure 3: Top: Standard bar chart. Middle: Wrapped bar chart at full length.
Bottom: Wrapped bar chart at half length. Color choices for the bars and
background are adopted from Du Bois’ work in Figure 2.

Essentially, $t1$ limits the numerical axis by a value and wraps any bar that
exceeds that value. In Figure 3, $t1$ is set to $1000$. As a results, the
largest bar with a value of 8500 has 8 full wraps (8 times $t1=1000$ equals
8000) with the tail reaching half of the numerical axis (equaling 500).

When designing the wrapped bars, we made multiple design decisions. First, in
addition to a one-to-one ratio between the width of bars and the gaps between
categories, we also determined on half of a bar-width gap for separating bars
in the wrapped portion. Second, in our design, the value of a wrapped bar is
the sum of only the vertical lines in a vertical wrapped bar charts and the
wrapping portions in the horizontal axis do not contribute to the total value.
Similarly, the vertical wrapping portions in a horizontal wrapped bar chart
would not count toward the total length. Therefore, our wrapped bar chart
maintains a linear relationship between values and vertical height of bars.
Such design decision allows for easy estimate of the overall value of a
wrapped bar. For example, a bar with a value of 5,500 in a wrapped bar chart
with $t1=1000$ and $t2=1$, wraps a total of 5 times with 500 as the tail of
the wrapped bar. These charts are developed using D3.js and an interactive
version of the wrapped bar chart can be viewed at https://wrapped-
barchart.herokuapp.com/.

Note that the wrapped bar chart prototype was first developed for Study 1
(shown in Figure 4). We improved the design of the wrapped bar chart based on
feedback collected through Study 1. The changes were mostly cosmetic,
including changing the bar and background color to resemble Du Bois’ original
work, and adjusting the bar width and gaps as detailed above. The final bar
charts design used in Study 2 are illustrated in Figure 3 (top and middle
charts).

##  4 Study 1: Pilot

###  4.1 Motivation and Hypotheses

Revisiting Du Bois’ wrapped bar chart on African American students’ enrolled
courses (Figure 2-right), we can see that the number of students enrolled in
industrial courses has orders of magnitude larger than ones enrolled in
business courses. From this example, we hypothesize that wrapped bar charts
may bring two advantages when presenting datasets with high variance 1) the
ability to maintain a fixed scale in a bar chart that allows for better
visibility for bars with smaller values and 2) enabling more precise
comparison/estimation of differences between larger and smaller values.
However, we recognize that wrapping would discount the preattentiveness of the
length of the bars [33], thus tasks of identifying the highest bar and
estimating the value may take longer.

Based on these factors, we develop three hypotheses about wrapped bar charts
in comparison to standard bar charts. Study 1 is conducted on two common types
of tasks for a bar chart, namely identification and ratio estimation [7, 29,
14, 20, 31]. For these two types of tasks, we hypothesize that:

  * •

H1: Participants will achieve higher accuracy with wrapped bar charts in
identifying smallest values.

  * •

H2: Participants will be more accurate with wrapped bar charts in estimating
ratios involving smallest values.

  * •

H3: Participants will take longer to complete of the tasks with wrapped bar
charts, e.g., identifying the bar with the largest value, estimating the ratio
of largest to smallest.

In addition to these hypotheses, we also want to measure the time performance
of each participant per dataset to ensure that any gains in better task
accuracy are not at the expense of additional time to complete. Therefore, we
also report the time to complete the full set of tasks per individual per
dataset.

###  4.2 Experiment Design and Procedure

To compare participants’ performance between wrapped bar charts and standard
bar charts, we developed a web-based application for Study 1 The application
recorded participants’ responses and response time. The experiment design
involves two factors: 2 datasets $\times$ 2 chart types. We selected two real-
world categorical datasets that exhibits very high variances between their
largest and smallest categorical values (See Figure 4). The differences in
these data values make the task of identifying and comparing lowest bars very
difficult with standard bar charts. The first dataset is about number of
Facebook ads by US presidential candidates for the 2020 election (values
ranging from 78 to 43,000), while the second dataset is about number of
resignation of the members of the US Congress by decade (data ranging from 1
to 122).

Figure 4: Two datasets used for Study 1. Top: Number of ad spending per
presidential candidate, Bottom: Number of Congressional resignations per
decade.

The experiment has a between-subject design as each participant sees either a
wrapped or standard bar chart for each dataset. As shown in Figure 10, each
participant viewed both datasets but with varying bar chart types by being
randomly assigned to either group A or B. Bars in Study 1 were plotted with a
fixed bar width and gap width calculated to evenly distribute the bars in the
charting area. (see Figure 4). To catch participants that answered randomly,
we added two simple bar charts (3-4 bars) before participants perform tasks on
each dataset.

Each participant is asked to complete six tasks for each bar chart presented
to them during the study:

  * •

T1: Identify the bar with the largest value.

  * •

T2: Identify the bar with the smallest value.

  * •

T3: How many times is the bar with the largest value to the bar with the
smallest value?

  * •

T4: Identify the bar with the second largest value.

  * •

T5: Identify the bar with the second smallest value.

  * •

T6: How many times is the value of the second smallest bar to the smallest
bar?

For the identification tasks (T1, T2, T4 and T5), the participants used a
mouse click to select a bar and then click on the "submit" button. For the
ratio estimation tasks (T3 and T6), the participants enter a number and submit
their answer.

Figure 5: The general flow of Study 1. Each participant is assigned to one of
the two (dataset*chart type) group after tutorial.

###  4.3 Experiment Results

We deployed the study on Amazon Mechanical Turk. 98 participants completed the
study with an average completion time of 5.5 minutes.333One participant was
dropped for not completing and a second participant was dropped for repeating
one of the sections. Each participant was compensated $0.50 for their time. In
total, 56 participants were randomly assigned to Group A (standard bar first)
and 42 to Group B (wrapped bar chart first).

We considered two different metrics to analyze the results. For T1, T2, T4 and
T5 that resulted in categorical selection, we measured performance by user
accuracy in identifying the correct bar value (e.g., largest, smallest). We
define user identification accuracy as the percentage of times they identify a
small or large bar correctly and report the estimate differences in percentage
points. For T3 and T6, we used log absolute error similar to Cleveland and
McGill [14].

To analyze and report our experiment data, we use a non-null hypothesis
statistical testing (non-NHST) approach focusing on sample means and
bootstrapped 95% confidence intervals (CIs) [17]. To enable comparison between
our experiment results and other studies, we adopt the method by Talbot et al.
to report estimates of simple effect sizes and associated CIs [31], i.e.
differences between estimated outcomes of experiment conditions followed by
95% CIs in square brackets [11]. Narrow CIs compared to the estimated
differences show strong evidence, while CIs that include zero imply more
uncertainty about the sign of effect. As highlighted by Kim et al., reporting
mean differences as simple effect sizes has limitations as variations around
these averages are not considered [23]. To address such limitations, we also
report Cohen’s d, which is a measure of _standardized_ effect size [15]. For
consistency on the direction of reporting, we will report effect sizes and CIs
of wrapped versus standard bar charts throughout this paper.

As an overview, Figure 6-left shows that participants achieved higher accuracy
on the identification tasks by 10.07 [5.86,14.28] (d=0.32) percentage points.
Figure 6-right shows that participants achieved higher accuracy using wrapped
bar charts across two datasets on ratio estimation tasks as indicated by
estimated log absolute error difference of -2.84 [-3.91, -1.78] (d=-0.52).

####  4.3.1 Identification Accuracy Wrapped vs. Standard Bar Chart

Figure 7 shows the participants’ accuracy on the identification tasks
performed on both datasets. Consistent with H1, our results demonstrate the
advantage of wrapped bar chart design in helping user identify bars with small
values. Participants are more accurate with identifying the smallest value
(T2) with wrapped bar chart design for both datasets.  For the Facebook Ads
dataset, participants were more accurate on average by 27.38 [13.69, 41.07]
(d=0.71) percentage points using wrapped bar charts. Similarly, for the
Congress dataset, participants were generally more accurate by 22.61 [8.92,
36.30] (d=0.73) percentage points. With the Facebook Ad dataset, participants
are also more accurate at identifying the second largest value with wrapped
bar chart design on average by 33.03 [17.26, 48.80] (d=0.78) percentage
points. However, for the Congress dataset, we did not observe noticeable
improvements for identifying second smallest 2.67 [-8.33, 13.69] (d=0.08). The
analysis did not reveal noticeable differences in identifying largest and
second largest bars in both datasets across the two conditions.

Figure 6: Study 1 mean and 95% individual-level bootstrapped CI’s (n = 98) for
identification accuracy (left) and log absolute error (right) for participants
using wrapped bar chart vs. standard bar chart. Figure 7: Study 1 mean
accuracy and 95% bootstrapped CI’s (n = 98) for identification task accuracy
(T1, T4, T2, T5) using a wrapped vs. standard bar chart. Figure 8: Study 1
mean and 95% bootstrapped CI’s (n = 98) for log absolute error (T3, T6) of
participants using wrapped bar chart vs. standard bar chart across tasks and
datasets. Figure 9: Study 1 mean completion time (in seconds) and 95%
bootstrapped CI’s (n = 98) per participant for all six tasks comparing wrapped
bar chart vs. standard bar chart.

####  4.3.2 Ratio Estimation Accuracy Wrapped vs. Standard

Figure 8 shows the participant’s performance on the ratio estimation tasks (T3
and T6). The results for estimating the ratio of the largest to the smallest
value (T3) is consistent with H2 that participants with the wrapped bar chart
performed much better.  Specifically, the estimated mean difference in log
absolute error is -4.30 [-5.36, -3.23] (d=-1.52) for the Facebook ads dataset
and -3.01 [-4.38, -1.65] (d=-0.8) for the Congress dataset. Inconsistent with
H2, no significant difference is observed between accuracy using wrapped and
standard bar chart to estimate the ratio of second lowest to the lowest value.

####  4.3.3 Trial Completion Time

In Figure 9, we provide the mean completion time and 95% bootstrapped CI’s per
participant for each trial (i.e., dataset/bar chart combination with six
tasks). We did not find evidence supporting H3 as participants took nearly the
same average time with wrapped bar chart as the standard bar chart for both
datasets. However, one issue with only measuring the overall trial completion
time is no visibility for the time taken for each individual task (e.g.,
largest bar identification, large / small ratio test). This issue is addressed
in the second study.

####  4.3.4 Discussion

The results from the pilot demonstrated that wrapped bar chart design can lead
to improved accuracy for identification and ratio estimation task. However,
what kind of dataset could benefit from being visualized in a wrapped bar
chart remains unclear. As a result, we conducted another study detailed in the
next section. The design of Study 2 is also informed by lessons learned
through the first study design, implementation, and analysis. For example, the
issue raised when analyzing the trial completion time motivates an improved
data collection for Study 2 to better track time duration by individual task.

##  5 Study 2: Investigating when to use wrapped bar charts

The primary goal of Study 2 is to investigate when it is advantageous to
present data in a wrapped design instead of a standard bar chart. Study 2 is
conducted to explicitly evaluate user performance on datasets of different
characteristics.

###  5.1 Improved Study Design and Hypotheses

First, we highlight design changes from Study 2. Most importantly, to evaluate
when to present data in a wrapped bar chart design, we leverage theoretical
metrics (normalized entropy and H-spread) as heuristics to determine what data
characteristics made them ideal candidates for wrapped bar charts.

Given our Study 1 results, we improved our study design and implementation in
the following ways:

  * •

Within-subjects study design to better measure the difference between standard
versus wrapped bar charts.

  * •

Remove identification and ratio test tasks of 2nd largest / smallest (T4, T5,
and T6) to simplify study design and allow more repeated trials.

  * •

Remove information about the dataset (such as number of Facebook ads in Study
1) to avoid distracting participants from focusing on the tasks.

  * •

Enhance time tracking to measure task-level duration rather than only trial-
level duration.

Figure 10: Study 2 Experiment Procedure

Study 2 is a within-subjects, repeated trials experiment that measures the
efficacy of a wrapped bar chart relative to a standard bar chart. The goal of
Study 2 is to provide guidance for visualization designers when to consider
using wrapped bar chart based on the characteristics of the data to be
visualized. In particular, we propose two data metrics to quantify what data
characteristics render wrapped bar charts useful. Entropy serves as the
primary metric for characterizing data distribution. However, as shown in the
bar chart simulations (Fig. 11), each entropy bin can exhibit different top
category concentration. We thus used H-spread as a secondary measure for how
"far out" the highest bar is as defined by Tukey [35].

Primary Data Metric: Information entropy to characterize the concentration of
values. In information theory, entropy measures the average (expected) amount
of information from an event [19]. For a more certain event, there is less
information contained in that event and implies a lower entropy. Conversely,
an event that is equally likely across all possible outcomes (e.g., a uniform
distribution) will have high entropy. Essentially, entropy is inversely
related to concentration. We hypothesize that participants will perform better
with wrapped bar charts for our tasks on discrete categorical datasets with
low entropy because such datasets exhibit a disproportionate concentration of
values within a few categories. Conversely, such performance gains with
wrapped bar charts will diminish for discrete categorical with higher entropy
as their values are more evenly spread across categories. We define entropy
as:

$\textrm{Entropy}=-\sum^{N}_{i=1}{p_{i}*\log_{2}p_{i}}$

where $p_{i}$ is the percent of values for a category $i$ and $N$ is the
number of categories. One issue with entropy is that it increases with the
number of categories, limiting the comparison across categorical datasets with
a different number of categories. Therefore, we normalized entropy by dividing
it by the log (base 2) of the number of categories ($N$). This normalization
converts entropy into a range of values between 0 (all values concentrated
into one bar) to 1 (uniform distribution).

$\textrm{Normalized Entropy}=\textrm{Entropy}/\log_{2}N$

Secondary Data Metric: H-Spread to characterize disproportional values. In his
book, Exploratory Data Analysis, John Tukey describes a method called ‘fences’
for identifying values that are “straying out far beyond the others” in a
dataset [35]. His heuristic, which is widely used in boxplots to show
disproportionate values, denotes the distance of a value away from the hinges
(upper and lower quantiles) divided by the H-spread (the difference between
the values of the hinges). He describes values larger than 1.5 times the
H-Spread away from the hinges as ‘outside values’. Similarly, he names values
that are 3 times the H-Spread outside the hinges as ‘far out’ values.
Borrowing Tukey’s heuristic, we formally define the “H-Spread” metric for a
categorical dataset with $X$ values as:

$\textrm{H-Spread}=(max(X)-Q3(X))/(Q3(X)-Q1(X))$

Based on our results from Study 1 and our new theoretical metrics, we develop
four hypotheses to test in Study 2.

  * •

H4: Participants will achieve a higher identification accuracy for the
smallest values with wrapped bar charts for datasets with low normalized
entropy.

  * •

H5: Participants will achieve a higher identification accuracy for the
smallest values with wrapped bar charts for datasets with high H-spread.

  * •

H6: Participants will have better large-to-small accuracy (i.e., lower log
absolute error) in estimating ratios between largest and smallest bars with
the wrapped bar charts than standard bar charts.

  * •

H7: Participants will spend more time with wrapped bar charts when estimating
largest-smallest bars ratio.

###  5.2 Data Simulation

To select our datasets for Study 2, we simulated 10,000 datasets based on
10,000 random draws from a fixed number of categories (e.g., 15
categories).444 Our simulator tool is a deployed R shiny app at
https://ryanwesslen.shinyapps.io/wrapped_bar_sim/ . Next, we categorized each
distribution into four fixed ranges (bins) for both normalized entropy and
H-spread, creating a 4 $\times$ 4 grid of 16 possible combinations. We then
randomly drew one dataset per normalized entropy and H-Spread bin combination
to use in our experiment. Given we used only a finite number of simulations,
we were only able to generate datasets for 13 of the 16 because datasets with
a low normalized entropy (less than 0.60) and high H-Spread (more than 4.50)
did not occur in our 10,000 dataset simulation. Figure 11 provides the 13
sampled datasets used in Study 2 by each of the bin combinations. Although
each dataset is visualized as a sorted bar chart in the data simulation app,
the bars appear in random order in the simulated dataset for Study 2.

Figure 11: 13 simulated datasets used in Study 2 that were randomly sampled
from each Normalized Entropy (Columns) and H-Spread (Rows) bin combinations
for a 10,000 dataset simulation.

###  5.3 Experiment Design and Participants

For Study 2, we designed a within-subject Amazon Mechanical Turk experiment in
which participants were provided both the standard and wrapped bar charts in
random order for each simulated datasets. We improved the design of standard
and wrapped bars to maintain a 1 to 1 ratio between bar width and gaps. Given
13 datasets, each participant completed 26 total trials consisting of three
tasks: (T1) identify the largest bar, (T2) identify the smallest bar, and (T3)
ratio estimation of the largest-to-the-smallest bar value. We randomized the
order of the dataset-chart combinations (trials) to reduce related confounding
factors.

203 participants completed our study, each receiving $2.00 reward for
completion. On average, participants took 29 minutes and 42 seconds. We
evaluated individual level performance on identification tasks to identify
participants who performed unreasonably. After investigating the number of
correct answers on the easiest task (i.e., T1), we dropped 13 participants who
incorrectly identified the largest value for both standard and wrapped bar
charts for more than 10 of 26 datasets, leaving 190 participants for Study 2.

###  5.4 Experiment Results

For reporting our results, we used individual-level bootstrapped means since
participants completed repeated trials across each experiment factor. To
calculate, we first averaged performance on a participant basis per factor
(e.g., Entropy bin) and then bootstrapped on user level performance. This
enabled us to control for heterogeneity between participant performance, which
is commonly found in crowdsourced visualization experiments [3]. Since Study 2
is a within-subjects study, we report paired Cohen’s d for within-subject
paired samples [24].

Figure 12: Study 2 large-small bar identification accuracy (T1 and T2) and 95%
bootstrapped confidence intervals (n = 190) by Normalized Entropy bin (A) and
H-Spread bin (B). We provide an example of a dataset from each bin as a visual
cue.

First, consistent with H4 and H5, we find participants in general had similar
or better small bar identification accuracy with a wrapped bar chart compared
to a standard bar chart. Figure 12 provides identification accuracy for Study
2 identification tasks (T1 and T2) by normalized entropy and H-spread bin. The
largest difference was in identifying the smallest value bars in datasets with
normalized entropy below 0.75. We observed a mean difference of 35.78 [28.42,
43.15] (d=0.70) percentage points for wrapped bar charts in datasets with
normalized entropy between 0.45-0.6 normalized entropy bin; and a difference
of 16.88 [12.31, 21.44] (d=0.68) percentage points for 0.6-0.75 normalized
entropy. For the entropy bins of 0.75-0.9 and 0.9-1, the identification
accuracy mean differences are small, respectively 1.06 [-1.01 , 3.2] (d=0.08)
and 2.01 [0.05 , 4.26] (d=0.2).

In addition, we find that participants’ had higher accuracy on small bar
identification with wrapped bar charts than standard bar charts for datasets
with H-spread larger than 1.5. We observe the largest effect with the highest
H-spread category of 4.5+, with accuracy difference of 16.53 [13.36. 19.71]
(d=0.80) percentage points. For H-spread of 3-4.5 we observe a difference of
5.92 [2.06 , 9.78] (d=0.28) and for 1.5-3.0, a difference of 8.28 [4.64,
11.92] (d=0.45). We did not observe noticeable differences in datasets in
H-Spread bin of 0-1.5.

Interestingly, we find wrapped bar charts yield worse accuracy in largest bar
identification for datasets with high H-Spread. For datasets with H-Spreads of
3.00-4.5, participants were on average less accurate by -5.28 [-7.67, -2.89]
(d=-0.31) and for datasets with H-Spreads of 4.5+, a similar worse accuracy of
-5.65 [-8.15, -3.15] (d=-0.34) percentage points when using wrapped bar
charts. For normalized entropy, we observe small or no effect within different
value ranges. For normalized entropy values between 0.45 and 0.6, we did not
observe significant differences (d=0.05) in large bar identification accuracy
between wrapped and standard bar chart. For normalized entropy between
0.6-0.75, we observe on average a worse accuracy of -7.72 [-11.02, -4.42]
(d=-0.36) percentage points with wrapped bar charts; for normalized entropy
0.75-0.9, we observe a mean difference of -3.26 [5.21, 1.31] (d=-0.22), and
for 0.9-1.0 we observe a mean difference of -2.85 [-4.81, -0.89] (d=-0.22). We
suspect that this may occur when multiple bars are wrapped, especially
multiple times, leading to higher cognitive load in counting the number of
wraps. We explore qualitative feedback to understand this hypothesis in our
in-lab Study 3.

Figure 13: Study 2 the mean log absolute error and 95% bootstrapped confidence
intervals (n = 190) for large-small ratio estimation (T3) by Normalized
Entropy bin (A) and H-Spread bin (B). We provide an example of a dataset from
each bin as a visual cue.

Second, we observe evidence for H6 that participants consistently have better
large/small ratio accuracy with wrapped bar charts compared to standard bar
charts. Figure 13 provides the mean log absolute errors and 95% bootstrapped
CI’s by either normalized entropy or H-spread bins. For normalized entropy
bins, we find the largest effect on datasets within the 0.45-0.6 range with a
log absolute error mean difference of -1.47 [-2.01, -0.9] (d=-0.62). For
normalized entropy within 0.6-0.75, we observe a mean difference of -0.58
[-0.97, -0.18] (d=-0.54). For normalized entropy of 0.75-0.9, we observe a
mean difference of -0.89 [-1.54, -0.25] (d=-0.57). And we observe a mean
difference of 0.67 [-1.26, -0.087] (d=-0.56) for normalized entropy values
between 0.9-1.0.

For H-Spread bins, we find the highest mean difference of -0.97 [-1.49, -0.46]
(d=-0.69) in the H-Spread 4.5+ datasets. We observe a similar mean difference
of -0.91 [-1.47, -0.34] (d=-0.55) for datasets with H-Spread in the 3-4.5
range. We observe a smaller effect of -0.67 [-1.21, -0.14] (d=-0.51) for
H-Spread 1.5-3 datasets. In summary, we observe that participants achieved
higher accuracy in ration estimation in low normalized entropy and high
H-spread datasets.

Figure 14: Study 2 mean completion time (n = 190) and 95% bootstrapped
confidence intervals for each task.

Third, we find some evidence in support of H7 that on average participants
tend to take 7.31 [4.98, 9.64] (d=0.42) seconds longer to complete the ratio
task (T3) with a wrapped bar chart as compared to a standard bar chart. We
find a smaller mean difference in time for the wrapped bar chart in
identifying the largest bar T1 of 3.04 [0.64, 5.44] (d=0.18) additional
seconds However, we find no difference in completion time for identification
of the smallest bar T2. Figure 14 provides the mean time to complete each task
along with 95% CI’s. While we anticipated some increase in task time with the
wrapped bar charts, overall, we do not find burdensome time effects that
offset the benefits in identification and ratio accuracy.

##  6 Study 3: Qualitative Evaluation

###  6.1 Motivation and study design

After obtaining the quantitative results in Study 1&2, We design a qualitative
study to gain a better understanding of users’ individual experience with
wrapped bar charts. We conducted in-lab focus group studies with a total of 24
participants. Our participants were graduate students enrolled in data science
or architecture degree programs. None of these students had prior experience
with wrapped bar charts. The procedure for Study 3 followed these steps:

  1. 1.

Participants were given a brief overview of the study purpose (e.g., informed
consent and instructions).

  2. 2.

Participants completed an online study similar to Study 2. However, to control
for time, participants were asked to complete tasks on 14 of the 26 datasets.

  3. 3.

Participants completed an open-ended post-questionnaire individually to
provide feedback on the study.

  4. 4.

Participants take part in a focus group session on user experience,
challenges, and suggested improvements.

The open-ended questions in the post questionnaire asked participants for
their strategies on calculating ratios and their feedback on the usefulness of
wrapped bar charts. After all participants in a session completed the study,
we held group discussion about wrapped bars to further understand
participants’ feedback. We collected participants’ written comments,
transcribed audio recordings from the open discussion sessions. Two coders
independently thematically analyzed users comments and feedback. We identified
four themes from participants’ written comments and discussions:

Benefits of wrapped bar charts: The most prevalent themes from both focus
group studies include the intuitiveness of the wrapped bar chart design, when
they could be beneficial, and participants’ strategies to interpret wrapped
bars. Out of 24 participants, 21 mentioned they believe wrapped bar charts are
useful for specific cases when some values in a dataset are disproportionate.
One participant’s comment summarizes this theme: “[Wrapped bar charts are
useful] when there is an obvious disparate effect from one attribute over the
others. If the axis scales were calibrated effectively, wrapped bar charts
could more concisely convey differences in orders of magnitude”. Another
participant understood the benefit of wrapped bars but highlighted the
importance of the tasks that chart designer want to explicitly support:
“[Wrapped bar charts are useful] when it is hard to read minimums and when the
minimums are crucial.”.

Usability of wrapped bar charts: Participants highlighted two important points
on the usability of wrapped bar charts. First, they understood that wrapped
bars were useful for specific cases when largest values render a number of
small values illegible. However, 16 participants argued that at some point the
task of estimating values of the wrapped bars can be cumbersome, especially
when there are many wraps. One participant commented: “wrapped chart is easier
if there are only couple of wraps. but if number of wraps increase, it gets
harder”. A major source of annoyance for five participants was when they had
to estimate the tail end value of wrapped bars stems from the opposite
direction of the y axis (for the wrapper portion going top to bottom), as one
participant mentioned: “… You need to do additional subtractions when the bar
is coming downwards”.

Strategies for reading wrapped bar charts: Another emerging theme in our
discussions with participants was the different strategies they took to read
wrapped bars, and calculate ratios between largest and smallest values.
Majority of users simply tried to estimate the value of largest and smallest,
and do a division task as described by two participants: “I assume a number
for minimum between the range. Then, another number for the max. After that, I
just divide and make the number round if possible.” and “count the tall bar
then the short bar and divide. for the tall bars I counted the number of full
bars and multiplied by the max number then added the end bar up.” However,
three participants came up with another novel strategy to read wrapped bars:
“I started estimating values of each of the largest and shortest columns
[bars] and then divided them. But after a few iterations I found a better
method. I estimated if stacking the shortest [bar] on itself X number of times
would fill up to the first grid line and then counted how many lines the
longest [bar] took up.”

Suggestions for improving wrapped bar charts: In the focus-group sessions,
participants also discussed how to improve the design of wrapped bar charts.
One suggestion was including an ‘inverse axis’ to the right side of a wrapped
bar chart to help estimate the value of the tail end of wrapped bars: “Use two
y scales on both left and right side of the chart but in a reverse way”.
Moreover, participants had multiple suggestions on ways of simplify the
estimation of value of largest bars by including count of wraps, using colors,
and increasing gaps between wrapped bars.

##  7 Discussion and Future Work

In our two online experiments, we find specific cases (low entropy and high
H-Spread) where users gain accuracy in identification and ratio tasks while
using wrapped bar charts. However, we also identify subtleties and drawbacks
in using wrapped bars in these specific cases. First, both quantitative and
qualitative findings including participants’ higher time spent on identifying
the largest bar and mentions of annoyance with too many wraps provide evidence
that wrapped bar charts are less preattentive than standard linear bar charts.
Our participants recognized that for estimating values of wrapped bars, they
need to go through a mathematical process of addition or multiplication of
values. Based on the feedback in the focus-group study, we believe that this
effect might be negligible for a small number of wraps. Adding information
such as providing a label for the number of wraps is needed for larger number
of wraps. To understand the trade-off between cognitive load and accuracy in
wrapped bar charts, we plan to conduct a future experiment studying the
optimum threshold for wrapping bar charts and the limit to the number of times
a bar can be wrapped before it becomes ineffective.

To understand the source of error w.r.t the ration estimation task, we further
explored whether identification accuracy impacts users’ ratio estimation
accuracy. Out of 4,949 valid trial responses collected in Study 2, 12.5% (617)
inaccurately identified either the largest or smallest bar. We ran a mixed
effects model on only the trials with correct identification responses and
found similar effects on log absolute error, with both means decreasing by
about 0.1. This exploration suggests an improvement for future study design,
i.e. to explicitly highlight the correct highest and smallest bars for the
ratio estimation task.

In the future, there are several design improvements we plan to make and
evaluate with wrapped bar charts in order to reduce the cognitive load
required to interpret wrapped bars. First, as mentioned in the focus-group
studies, we plan to introduce an inverse y-axis to the right side of a
horizontal bar chart to help read the tail end of wrapped bars (when going
from top to bottom). Moreover, we also plan to experiment different gap sizes
between wrap bar charts to help with the task of counting the number of wraps.
Third, we plan to study the effects of changing the second threshold of
wrapped bars (see Figure 3 bottom) on participant performance. Finally, one
solution to the trade-off between cognitive load and benefits of wrapped bars
could be to introduce a new interaction technique for interactive bar charts.
‘Wrapping’ would allow users to quickly move from standard bar charts to
wrapped bar charts and observe the values of small bars. These design
solutions require a comprehensive and systematic study to estimate and
understand the benefits of these changes.

From an application perspective, there are multiple extensions we can test
with wrapped bar charts. First, in order to better understand the potentials
of wrapped bar charts, we need to collect more data from users including mouse
positions, user confidence level, and user satisfaction. Second, we need to
compare wrapped bar charts with other types that are built to deal with
datasets with values of high variance, such as broken axis and logarithmic
scale. Third, during our development of wrapped bar charts, we developed a new
kind of visual interaction by allowing users to interactively change the
thresholds of wrapped bar charts ($t1\&t2$) and explore high and low data
points. In a future study, we plan to thoroughly study this interaction
technique for conducting similar visual tasks and learn about user preference.
Last, we suspect that zoomed-out wrapped bar charts could work well with many
more categories ($N$ > 50), like a power law distribution (e.g., Zipf’s law
for word counts) that is carefully ordered (e.g., by word-topic theme).

Finally, as a general suggestion for visualization designers to employ wrapped
bar chart design within their work we provide the following recommendation
resulting from our quantitative and qualitative evaluations: Using entropy and
H-Spread as heuristics to measure a dataset and determine whether it is
beneficial to use wrapped bar charts. We recommend using wrapped bar charts to
visualize datasets with normalized entropy less than 0.75 as we see
performance gain in identification and ratio tasks (Figures 12 and 13).
Moreover, in line with Tukey’s suggestion, when the H-Spread threshold is
larger than 4.5 that includes far out values), wrapped bar charts could be a
good design option.

##  8 Conclusion

In this paper, we designed, developed, and evaluated wrapped bar charts
originally introduced by W.E.B. Du Bois to highlight the benefits of his
visual innovation added to the standard linear bar charts presentation. We
developed an implementation of wrapped bar charts for the web. We hypothesized
that wrapped bar charts can outperform standard bar charts for datasets with
disproportionate values. Using a web-based interface we conducted two online
experiments. We found that for cases where data values are disproportionate as
measured by normalized entropy and H-spread, wrapped bar charts allow
participants to achieve higher accuracy on tasks of identification and ratio
estimation but sometimes at the expense of more time spent and potentially
more cognitive load. Finally, resulting from our focus group study, we develop
a list of potential design improvements for such charts. Our findings, can
serve as guidelines for visualizing datasets with disproportionate values
using wrapped bar charts.

## References

  * [1]
  * [2] 2016\.  W. E. B. Du Bois’ Hand-Drawn Infographics of African-American Life (1900) The Public Domain Review.  https://publicdomainreview.org/collections/w-e-b-du-bois-hand-drawn-infographics-of-african-american-life-1900. (2016).  (Accessed on 06/12/2019).
  * [3] Alfie Abdul-Rahman, Karl J Proctor, Brian Duffy, and Min Chen. 2014.  Repeated measures design in crowdsourcing-based experiments for visualization. In Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization. ACM, 95–102.
  * [4] Witney Battle-Baptiste, Britt Rusert, and others. 2018.  WEB Du Bois’s Data Portraits: Visualizing Black America.  Chronicle Books.
  * [5] James R Beniger and Dorothy L Robyn. 1978.  Quantitative graphics in statistics: A brief history.  The American Statistician 32, 1 (1978), 1–11.
  * [6] Rita Borgo, Alfie Abdul-Rahman, Farhan Mohamed, Philip W Grant, Irene Reppa, Luciano Floridi, and Min Chen. 2012.  An empirical study on using visual embellishments in visualization.  IEEE Transactions on Visualization and Computer Graphics 18 (2012), 2759–2768.
  * [7] Rita Borgo, Joel Dearden, and Mark W Jones. 2014.  Order of magnitude markers: An empirical study on large magnitude number detection.  IEEE transactions on visualization and computer graphics 20, 12 (2014), 2261–2270.
  * [8] Eliot Brown. 2019.  This Is Not the Way Everybody Behaves. How Adam Neumann’s Over-the-Top Style Built WeWork.  https://www.wsj.com/articles/this-is-not-the-way-everybody-behaves-how-adam- neumanns-over-the-top-style-built-wework-11568823827. (Sep 2019).  (Accessed on 09/19/2019).
  * [9] John Bryan. 1995.  Seven types of distortion: A taxonomy of manipulative techniques used in charts and graphs.  Journal of technical writing and communication 25, 2 (1995), 127–179.
  * [10] Aaron Bycoffe, Sarah Frostenson, and Julia Wolfe. 2019.  A Final Look At Who Won The Third Democratic Debate.  https://projects.fivethirtyeight.com/democratic-debate-september-poll/. (Sep 2019).  (Accessed on 09/16/2019).
  * [11] Guillaume Calmettes, Gordon B Drummond, and Sarah L Vowler. 2012.  Making do with what we have: use your bootstraps.  Advances in physiology education 36, 3 (2012), 177–180.
  * [12] Clark R Carvalho and Michael D McMillan. 1992.  Graphic representation in managerial decision making: The effect of scale break on the dependent axis.  Technical Report. AIR FORCE INST OF TECH WRIGHT-PATTERSON AFB OH.
  * [13] William S Cleveland. 1984.  Graphical methods for data presentation: Full scale breaks, dot charts, and multibased logging.  The American Statistician 38, 4 (1984), 270–280.
  * [14] William S Cleveland and Robert McGill. 1984.  Graphical perception: Theory, experimentation, and application to the development of graphical methods.  Journal of the American statistical association 79, 387 (1984), 531–554.
  * [15] Jacob Cohen. 1988.  Statistical power analysis for the social sciences.  (1988).
  * [16] Geoff Der and Brian S Everitt. 2014.  A handbook of statistical graphics using SAS ODS.  Chapman and Hall/CRC.
  * [17] Pierre Dragicevic. 2016.  Fair statistical communication in HCI.  In Modern Statistical Methods for HCI. Springer, 291–330.
  * [18] Stephen Few. 2013.  Wrapping Graphs to Extend Their Limits.  https://www.perceptualedge.com/articles/visual_business_intelligence/wrapping_graphs_to_extend_their_limits.pdf. (2013).
  * [19] Robert M Gray. 2011.  Entropy and information theory.  Springer Science & Business Media.
  * [20] Jeffrey Heer and Michael Bostock. 2010.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 203–212.
  * [21] Marcel Hlawatsch, Filip Sadlo, Michael Burch, and Daniel Weiskopf. 2013\.  Scale-Stack Bar Charts. In Computer Graphics Forum, Vol. 32. Wiley Online Library, 181–190.
  * [22] Thomas Kaplan and Sarah Almukhtar. 2019.  How Trump Is Outspending Every 2020 Democrat on Facebook.  https://www.nytimes.com/interactive/2019/05/21/us/politics/trump-2020-facebook-ads.html. (May 2019).  (Accessed on 09/13/2019).
  * [23] Yea-Seul Kim, Jake M. Hofman, and Daniel G. Goldstein.  Effectively Communicating Effect Sizes. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). Association for Computing Machinery, New York, NY, USA.
  * [24] Daniël Lakens. 2013.  Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs.  Frontiers in psychology 4 (2013), 863.
  * [25] Pranathi Mylavarapu, Adil Yalcin, Xan Gregg, and Niklas Elmqvist. 2019\.  Ranked-List Visualization: A Graphical Perception Study. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Computing Machinery, New York, NY, USA, Article Paper 192, 12 pages.  DOI:http://dx.doi.org/10.1145/3290605.3300422
  * [26] Drew Skau, Lane Harrison, and Robert Kosara. 2015.  An evaluation of the impact of visual embellishments in bar charts. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 221–230.
  * [27] Ian Spence. 2006.  William Playfair and the psychology of graphs. In Proceedings of the American Statistical Association, Section on Statistical Graphics. 2426–2436.
  * [28] Ian Spence and Stephan Lewandowsky. 1991.  Displaying proportions and percentages.  Applied Cognitive Psychology 5, 1 (1991), 61–77.
  * [29] Arjun Srinivasan, Matthew Brehmer, Bongshin Lee, and Steven M Drucker. 2018.  What’s the Difference?: Evaluating Variations of Multi-Series Bar Charts for Visual Comparison Tasks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 304.
  * [30] Ben Starr. 2014.  How to Design Bar Charts.  https://visage.co/data-visualization-101-bar-charts/. (2014).  (Accessed on 06/13/2019).
  * [31] Justin Talbot, Vidya Setlur, and Anushka Anand. 2014.  Four experiments on the perception of bar charts.  IEEE transactions on visualization and computer graphics 20, 12 (2014), 2152–2160.
  * [32] AnyChart Team. 2015.  First Ever Bar Charts Created in 14th Century.  https://www.anychart.com/blog/2015/06/02/first-bar-chart-in-history/. (June 2015).  (Accessed on 06/13/2019).
  * [33] Anne Treisman and Janet Souther. 1985.  Search asymmetry: A diagnostic for preattentive processing of separable features.  Journal of Experimental Psychology: General 114, 3 (1985), 285\.
  * [34] ER Tufte and P Graves-Morris. 1983.  Vol. 2.  The Visual Display of Quantitative Information. CT: Graphics press Cheshire (1983).
  * [35] John W Tukey. 1976.  Exploratory data analysis.  Massachusetts: Addison-Wesley (1976).
  * [36] Mingqian Zhao, Huamin Qu, and Michael Sedlmair. 2019.  Neighborhood Perception in Bar Charts. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). ACM, Article 232, 12 pages.