# Chart4Blind: An Intelligent Interface for Chart Accessibility Conversion

Morris Baumgarten-Egemole<EMAIL_ADDRESS>ACCESS@KIT, Karlsruhe Institute of TechnologyAdenauerring (Geb.
50.28)KarlsruheBaden-WürttembergGermany76131 ,  Omar Moured
<EMAIL_ADDRESS>0000-0003-4227-8417 CVHCI & ACCESS@KIT, Karlsruhe
Institute of TechnologyAdenauerring (Geb. 50.28)KarlsruheBaden-
WürttembergGermany76131 ,  Karin Müller  0000-0003-4309-1822
<EMAIL_ADDRESS>ACCESS@KIT, Karlsruhe Institute of
TechnologyAdenauerring (Geb. 50.28)KarlsruheBaden-WürttembergGermany76131 ,
Alina Roitberg<EMAIL_ADDRESS>0000-0003-4724-9164
Institut für Künstliche Intelligenz, Universität StuttgartAdenauerring (Geb.
50.28)KarlsruheBaden-WürttembergGermany76131 ,  Thorsten Schwarz
0000-0002-7346-5744<EMAIL_ADDRESS>ACCESS@KIT, Karlsruhe Institute
of TechnologyAdenauerring (Geb. 50.28)KarlsruheBaden-WürttembergGermany76131
and  Rainer Stiefelhagen<EMAIL_ADDRESS>0000-0001-8046-4945
CVHCI & ACCESS@KIT, Karlsruhe Institute of TechnologyAdenauerring (Geb.
50.28)KarlsruheBaden-WürttembergGermany76131

(2018; 2024)

###### Abstract.

In a world driven by data visualization, ensuring the inclusive accessibility
of charts for Blind and Visually Impaired (BVI) individuals remains a
significant challenge. Charts are usually presented as raster graphics without
textual and visual metadata needed for an equivalent exploration experience
for BVI people. Additionally, converting these charts into accessible formats
requires considerable effort from sighted individuals. Digitizing charts with
metadata extraction is just one aspect of the issue; transforming it into
accessible modalities, such as tactile graphics, presents another difficulty.
To address these disparities, we propose Chart4Blind, an intelligent user
interface that converts bitmap image representations of line charts into
universally accessible formats. Chart4Blind achieves this transformation by
generating Scalable Vector Graphics (SVG), Comma-Separated Values (CSV), and
alternative text exports, all comply with established accessibility standards.
Through interviews and a formal user study, we demonstrate that even
inexperienced sighted users can make charts accessible in an average of $4$
minutes using Chart4Blind, achieving a System Usability Scale rating of
$90\%$. In comparison to existing approaches, Chart4Blind provides a
comprehensive solution, generating end-to-end accessible SVGs suitable for
assistive technologies such as embossed prints (papers and laser cut), 2D
tactile displays, and screen readers. For additional information, including
open-source codes and demos, please visit our project page
https://moured.github.io/chart4blind/.

Data Visualization, Chart, Digitization, Accessible Charts, Embossed Paper

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; March 18–21, 2024; Greenville,
SC††booktitle: IUI ’24: ACM Conference on Intelligent User Interfaces (ACM
IUI), March 18–21, 2024, Greenville, SC††price: 15.00††isbn:
978-1-4503-XXXX-X/18/06††journalyear: 2024††copyright:
acmlicensed††conference: 29th International Conference on Intelligent User
Interfaces; March 18–21, 2024; Greenville, SC, USA††booktitle: 29th
International Conference on Intelligent User Interfaces (IUI ’24), March
18–21, 2024, Greenville, SC, USA††doi: 10.1145/3640543.3645175††isbn:
979-8-4007-0508-3/24/03††ccs: Human-centered computing Accessibility systems
and tools

Figure 1. Using the Chart4Blind interface, a sighted person can upload a
bitmap image, which is automatically analyzed, and accessibility relevant
information is inserted. The interface shown in (a) allows manual adaptations
to improve accessibility. The export function translates the recognized text
to Braille fonts and lines to lines, that can be embossed and distinguished
tactilely (b). A sample outcome of the export is an embossed tactile graphic
with a short textual description for finger exploration by people with visual
impairments (c).

The picture depicts three subfigures. In (a), the Chart4Blind interface is
illustrated, highlighting three AI tools and the metadata fields. A simple
line chart with three lines is also visible in the UI. (b) displays the same
line chart as in (a), but in print-accessible mode. (c) showcases the printed
version of the chart, indicating the complete flow of the chart in the
conversion process.

##  1\. Introduction

Humans heavily rely on graphs and charts to analyze trends, make comparisons,
and extract meaningful insights in both academic and commercial areas. In many
professions, an effective understanding of graphical data is a crucial
prerequisite for achieving success. The rapidly growing amount of information
available online (Taylor, 2023) has shifted the focus of research from
gathering data to ensuring it is accessible and understandable to all. Line
graphics are a widespread way of presenting information, e.g., when teaching
students, presenting a new idea to business partners, or exchanging knowledge
in general.

However, most existing chart representation methods are predominantly visual,
posing significant accessibility challenges for people with visual
impairments. Despite the advancements in data visualization tools, there
remains a significant gap in ensuring that these tools are inclusive and
accessible for all. Line charts used in books, learning materials, and on
slides are a common type of graph. Typically, these documents lack textual and
visual metadata, which prevents people with visual impairments from exploring
the graphics. Moreover, the process of converting these charts into accessible
formats such as textual descriptions or tactile graphics is often labor-
intensive and requires significant effort from sighted individuals (Sheppard
and Aldrich, 2001). Although tools to automatically describe graphs textually
have been more explored in recent years, (Lundgard and Satyanarayan, 2022;
Sharif et al., 2022; Tang et al., 2023), approaches to automatically extract
information from graphs and generate tactile graphics have hardly been
addressed in research. While a handful of studies have approached this from a
computer vision standpoint (Singh et al., 2023; Moured et al., 2023a, b), the
user experience aspect of people who create accessible graphics has been
largely overlooked and is the main motivation of this work.

Our main goal is to create a tool that helps sighted people make graphs
accessible to people with visual impairments. To address this goal, we
introduce Chart4Blind – an intelligent user interface designed to transform
bitmap image representations of line charts into universally accessible
formats (Nagao and Hatanaka, 2005; Edman, 1992), including SVG, CSV files, and
alternative text. These formats can be used to emboss a tactile graphic on
paper or to read it on a 2D haptic Braille display. While our design
principles primarily target the visually impaired and blind individuals, our
framework has strong potential for saving time of a sighted user e.g.
assistants, friends, or family members of blind persons by supporting
automatic digitization and information extraction from line graphs.
Chart4Blind ensures that the converted charts align with established
accessibility standards (Initiative, 2023) and supports the process of
transfer of line charts. We started with a series of need-finding interviews
to guide our design choices. The tool design emphasizes clear progress
indicators during conversion, consistency in the conversion steps, intuitive
data input methods, and integration of AI tools for efficient data extraction.
To validate our tool, we conducted a usability study with 10 sighted
participants aged 22-34 years. The study revealed that the Chart4Blind tool is
valuable for converting line charts into accessible formats and helps sighted
users understand accessibility guidelines with an average System Usability
Scale rating of 90%. To validate the accessibility of the produced graphic for
BVI in particular, we conducted a follow-up study with 3 blind individuals,
asking open-ended questions regarding the output of our interface and
collecting valuable feedback for future improvement.

This paper introduces Chart4Blind, an intelligent user interface designed to
convert bitmap image representations of line charts into universally
accessible formats, and has the following major contributions:

  * •

Chart4Blind System: Through a series of interviews and feedback, we designed
an intuitive tool for converting bitmap line charts into multiple universally
accessible formats. It also supports collaborative efforts where multiple
people can work together and assist in the conversion process. Chart4Blind has
a user-friendly interface and can be utilized by individuals with varying
levels of experience.

  * •

Integration of Intelligent Features: The tool incorporates intelligent deep
learning models to ensure a seamless conversion process, particularly OCR and
segmentation models. Users can interact with the model predictions through
simplified actions, such as drag-and-drop.

  * •

Usability Study: A thorough usability study involving sighted people aged
between 22-34 years to validate the effectiveness of the Chart4Blind system.
Furthermore, another user study was conducted specifically with blind
participants to assess the output quality and accessibility. The tool achieved
an average System Usability Scale rating of $90\%$.

##  2\. Related Work

Our research builds upon the fundamental elements of earlier studies, which
include: (1) accessible charts, (2) chart deconstruction, and (3) chart
summarization.

###  2.1. Accessible Representation of Charts

People with visual impairments often struggle with bitmap images of charts
(Bigham et al., 2006; Singh et al., 2023). To address this, the Web Content
Accessibility Guidelines recommend offering a textual description of the chart
alongside its graphical representation as alternative text(Consortium et al.,
1999). Yet, such descriptions are rarely created by authors (Bigham et al.,
2006). To reduce the manual effort in this task, some works automate
alternative text generation (Balaji et al., 2018; Kantharaj et al., 2022;
Farahani et al., 2023). Unfortunately, such tools have several issues, such as
producing irrelevant information and hallucinations (Tang et al., 2023;
Kantharaj et al., 2022).

While the alternative text is useful for describing line charts, it is not
always enough for those with visual impairments (Altmanninger and Wöß, 2006;
Watanabe et al., 2012). There are three alternative ways to present graphical
information for visually impaired people: (1) Tactile graphics, using relief
elements for haptic perception (Serrano et al., 2021), (2) Alternative Text,
describing graphical content in words (Devine et al., 2011), or with screen
readers (Zou and Treviranus, 2015; Blanco et al., 2022) and (3) Sonification,
mapping raw data values to a diverse range of sounds, varying pitch,
frequency, and tone to enable easy distinction between line trends (Holloway
et al., 2022; Bru et al., 2023; Hermann et al., 2011).

When it comes to tactile graphics, they can be in different formats: embossed
paper, swell paper, thermoform, laser cut, and 3D printed (Butler et al.,
2021). These formats typically provide better dot resolutions, utilizing
various pin height levels to represent more information, and they’re cost-
effective and portable for individuals with visual impairments (Li et al.,
2019; Engel and Weber, 2018). However, they lack the capability for advanced
interaction. The second format is digital tactile displays, which can be
refreshed and offer additional features such as zooming, interaction buttons,
and audio output (Moured et al., 2023a). Nonetheless, this option tends to be
more expensive and offers lower pin resolution. In our work, we ensure that
the converted charts meet the accessibility needs of both digital and printed
formats.

Having access to a chart’s raw data, which includes plot data, titles, axis
labels, and descriptions, is highly beneficial. For example, some works use
this data to create tactile graphics like Audio-Tactile charts, which combine
touch interaction with audio feedback (Araki et al., 2014; Engel et al.,
2019). Similarly, Sonification and alternative text descriptions benefit from
raw data access. In terms of raw data extraction methods, they can be
categorized as: (1) Manual, requiring human intervention without automated
tools, like Data Thief (Tummers, 2006), (2) Semi-automatic, combining
automatic features with some human intervention (Rohatgi, 2014; Jung et al.,
2017; Vaingast, 2014; Masson et al., 2023a; Huwaldt and Steinhorst, 2023;
Tummers, 2006), and (3) Fully automatic, with no manual intervention (Lal et
al., 2023; Choi et al., 2019), though they have limitations in the conversion
accuracy.

Given the importance of raw data, vector graphics, such as SVGs, offer a
promising alternative representation (Godfrey and Loots, 2014). SVGs, created
using a chart’s raw data, have features that make them more accessible than
bitmap images (Ferraiolo et al., 2000). Their structure supports tactile
printable graphics creation (Krufka and Barner, 2005). For instance, Braille
printers can emboss raised dots to mark outlines (Krufka and Barner, 2005).
Additionally, methods like LineSpace (Swaminathan et al., 2016), which use 3D
printer filament to print SVG file elements, show the potential of SVGs. In
conclusion, SVGs can complement traditional alternative text descriptions for
line charts.

###  2.2. Image to Vector Graphic Conversion

Having determined SVG as a suitable format for an alternative representation,
we now describe techniques for converting bitmap images to vector graphics.
Tools such as Libre Office Draw can be used to convert images to a Libre
Office vector graphic format (Foundation, 2020), but this method is time-
consuming (Prescher et al., 2014). To address these challenges, Jayant et al.
introduced an automated solution that converts bitmap images into Adobe
Illustrator vector graphics (Jayant et al., 2007). However, this tool requires
manual training on similar charts and directly translates charts into
printable graphics, which is not ideal for tactile charts due to braille
embosser constraints (Engel and Weber, 2017). Several guidelines have been
introduced to govern the conversion process (Gale, 2005; on Information Access
for People with Print Disabilities, 2022). Goncu et al. proposed a tool that
converts pie and bar chart data into SVG (Goncu and Marriott, 2008), arguing
against a one-size-fits-all approach for accessible chart representation.
Offering multiple output options can diminish barriers between BVIP and
sighted users (Goncu and Marriott, 2008), making the original and tactile
charts more alike, potentially facilitating BVI’s interpretation of chart data
(Engel and Weber, 2018).

###  2.3. User Experience Analysis

Usability is vital when designing interactive user interfaces (Unger and
Chandler, 2012). Recent applications emphasize intuitive design. Tools such as
ChartDetective (Masson et al., 2023a) and PlotDigitizer (Huwaldt and
Steinhorst, 2023) are web-apps that adhere to design best practices,
incorporating Visual Information-Seeking Mantra principles (Shneiderman,
1996). However, the former tool, ChartDetective, accepts only vector graphic
charts in PDF format, and neither tool supports accessible output formats.

Enhancing UX also involves offering a magnified view around the mouse pointer,
improving application accuracy. While tools like im2graph (Vaingast, 2014) and
WebPlotDigitizer (Rohatgi, 2014) emphasize such explorations.

From 2006 to 2023, across various chart analysis tools (Tummers, 2006; Huwaldt
and Steinhorst, 2023), manual calibration of chart axes, involving setting
four calibration points to map pixel values to the x-y plane, has remained a
consistent feature. Despite the potential benefits of automation, its
implementation is limited by current algorithms, which only achieve 61.7%
accuracy in axis detection (Moured et al., 2023b). Thus, manual methods,
either by prompt-based clicking (Tummers, 2006; Rohatgi, 2014) or drag-and-
drop (Huwaldt and Steinhorst, 2023), are preferable. Semi-automation also aids
in text value entry. Im2Graph (Vaingast, 2014) and ChartDetective (Masson et
al., 2023a) employ OCR to recognize image text (Chaudhuri et al., 2017; Nagy
et al., 1999). After calibration, various techniques exist for extracting plot
data, often resulting in CSV files (Rohatgi, 2014; Jung et al., 2017;
Vaingast, 2014; Masson et al., 2023a; Huwaldt and Steinhorst, 2023; Tummers,
2006; Huwaldt and Steinhorst, 2023). WebPlotDigitzer and PlotDigitizer offer
semi-automatic extraction, letting users edit detected data markers. Im2graph
employs a color-based line detection approach. Data markers, which can mimic
curves and match exported values, are commonly used due to their predictable
results.

##  3\. NEED-FINDING INTERVIEWS

To gain a comprehensive understanding of how sighted individuals approach the
creation of accessible line charts for BVI individuals, and to identify how
future tools can efficiently support their efforts, we conducted a series of
exploratory need-finding semi-structured interviews. While recent research has
focused on the challenges involved in extracting chart metadata (Cheng et al.,
2023; Lal et al., 2023; Hassan et al., 2023), it has not thoroughly addressed
the specific requirements of end users, particularly the blind and visually
impaired. Furthermore, there has been a gap in exploring the essential
prerequisites for an online tool to streamline this conversion process.
Therefore, our need-finding interviews were carried out with the primary
objective of uncovering these specific design requirements. Our inquiries
primarily aimed to:

  * •

Understand the limitations of common interfaces and user practices for
converting charts into accessible formats.

  * •

Pinpoint the features that users find important in such a tool.

  * •

Gain insights into how AI tools are explored in prior research and could
facilitate the creation of accessible charts.

###  3.1. Participants

We conducted a semi-structured interview with four sighted participants (P1-4,
2 female, and 2 male, age range 25-40 years) who would use the Chart4Blind
user interface. We collaborated with a local non-profit accessibility service
center ACCESS@KIT111https://www.access.kit.edu/, to identify experts who
transfer literature for blind users. All sighted individuals exhibited
proficiency in working with line charts, familiarity with accessibility
guidelines, and practical experience in converting chart materials. Three of
the participants occasionally converted charts for BVI students, while one
participant undertook this task on a daily basis. Table 1 illustrates the
working domains and experience level of our participants in more detail. The
participants received no compensation. The study was part of a series of
studies which were approved by the ethical review committee of Karlsruhe
Institute of Technology.

Table 1. Participants’ overview: Experience in BVI chart conversion and approximate conversion frequency. Participant | Gender | Field | Experience | Frequency | Tool used
---|---|---|---|---|---
P1 | F | Assistive Technologies | +5 years | Daily basis | LibreOffice Draw (Foundation, 2023)
P2 | F | Computer Science | +2 years | once every 2 weeks | LibreOffice Draw
P3 | M | Computer Science | +10 years | once a month | Inkscape (Conservancy, 2023)
P4 | M | Physics | +2 years | once a month | WebPlotDigitizer (Rohatgi, 2014)

###  3.2. Procedure

During our need-finding study, we asked participants open-ended questions
about their experiences, challenges, and requirements for the chart conversion
process. We asked the following questions:

       (1) Describe your workflow for converting charts into an accessible format.
       (2) What challenges did you encounter during this process?
       (3) What specific computer-based tools do you utilize for the conversion process?

We followed up by inquiring about the steps they found most challenging and
time-consuming. Depending on the interview, we additionally asked:

        (4) Could you estimate the time required for chart conversion?
        (5) Which features of your current tools do you find most useful?

These one-to-one interviews were conducted face-to-face, with the interviewer
asking questions and taking written notes, and they typically lasted around
40-45 minutes. After reviewing the interview notes for mentioned topics, we
established the design principles and requirements. This led to the
development of the Chart4Blind prototype using Figma222https://www.figma.com/,
a popular design tool widely used by user interface designers. During this
phase, participants P1 and P3 assisted in refining minor aspects of
Chart4Blind such as color schemes, font styles, and the tutorial script.

###  3.3. Findings

In this section, we present the primary themes derived from our analysis, as
well as the identified user requirements (UR) that will guide our Chart4Blind
prototype design.

####  3.3.1. Needs for streamlined process

Our participants employed distinct tools for the conversion task. P1-3
utilized drawing tools (Conservancy, 2023; Foundation, 2023) to overlay
components onto chart images, replicating the original chart’s elements. On
the other hand, P4 employed WebPlotDigitizer (Rohatgi, 2014) to extract
metadata and then used data visualization tools to regenerate an accessible
chart. When asked about the average time required for a single chart
conversion, participants agreed that it depends on the chart’s complexity.
None reported taking less than 10 minutes, even for easy charts, as the
process requires manual drawing, adding text labels, and writing a chart
description.

Despite the diverse tools used, we identified common steps in the conversion
process, highlighting an opportunity to streamline this task into a unified
tool. However, participants encountered challenges involving inexperienced
individuals for assistance, as the existing tools were not designed for this
purpose. Extensive tutorials were necessary to prepare individuals for the
task. Additionally, the process lacked the potential for parallelization due
to limitations in (real-time) data-sharing mechanisms within the current
tools. P1 emphasized how these challenges prevented effective crowdsourcing
efforts. In response to these findings, we propose developing a user interface
that caters to both experts and non-experts, presenting clear and consistent
steps (UR-1). Furthermore, the tool facilitates a more parallelizable process
to encourage concurrent contributions (UR-2).

####  3.3.2. Needs for intelligent features

The analysis of question 4 highlighted two time-consuming tasks: text
extraction and line segmentation, aligning with charts being both visual and
textual. P4 mentioned that many charts have numerous axis ticks, often
requiring manual rewriting. P1-2 discussed line segmentation challenges,
especially with more lines or complex trends like sinusoidal patterns. P3,
using WebPlotDigitizer, noted the tool’s reliance on color filters for
segmentation, requiring parameter tuning. These insights highlight the need
for a robust OCR (UR-3) and a deep learning-based line annotation tool (UR-4).
Recognizing diverse metadata needs for various charts, we opted for a semi-
automatic approach over an end-to-end solution.

####  3.3.3. Needs for accessible output

Another aspect missing in other tools is ensuring an accessible export. Most
participants (P1-3) prefer drawing tools due to their SVG export capability.
However, two challenges were highlighted. Firstly, P3 pointed out that the
drawing approach did not include line metadata in the export, containing only
the line drawing position relative to canvas dimensions and not the real
coordinates. Secondly, P1 mentioned the need for further processing on the
export, especially for complex charts that require simplification for BVI
individuals such as reducing the number of included axis labels to fit the
embossed paper size or optimizing the SVG for screen readers by removing
unnecessary tags and adding descriptive ones.

To our knowledge, no tool currently enables metadata extraction and produces a
printable export suitable for BVI individuals. Given varying needs, sometimes
leaning towards metadata or visual elements, we introduced the requirement to
support accessible extraction in both CSV and SVG formats (UR-5). The decision
was informed by the support of other BVI assistive technologies for these
formats (Engel et al., 2019; Thompson et al., 2023).

###  3.4. Design Principles

Based on our interview findings, we have identified key design principles that
guide our Chart4Blind system:

  * •

Clear Progress Indication: The tool should provide users with a clear view of
their current progress during chart conversion, indicating what has been
completed and what remains.

  * •

Maintain Consistent Steps: The tool should ensure that the steps in the
conversion process remain consistent regardless of chart complexity, allowing
for a more parallelized process.

  * •

Intuitive Data Input: The tool should allow for an intuitive approach to make
sure that all necessary information needed for an accessible chart is added.
In addition, the tool should also support intuitive interactions such as drag-
and-drop for textual content and drop-down lists for axis label formats (e.g.,
linear, logarithmic).

  * •

Automated Selection Tools: To reduce the efforts spent extracting metadata,
the tool should integrate an automated solution to ease the task of element
selection (e.g., lines, texts). In our case, we choose AI-driven tools for
line segmentation, text extraction, and chart description when needed.

  * •

Ready-to-Use Accessible Exports: The tool should provide users with accessible
output for various modalities such as tactile printing and digital displays
used with a screen reader.

##  4\. The Chart4Blind System

In this section, we describe the design of our Chart4Blind system, which
assists sighted people in making charts accessible for people with visual
impairments by following the design principles we derived from our need-
finding interviews. Our system comprises two main modules: (1) Data Extraction
Module and (2) Rendering Module.

Figure 2 shows the pipeline of Chart4Blind system. When a user uploads a line
chart image, a toolbar appears that is linked to our AI-based Data Extraction
Module, which is responsible for extracting the chart’s textual and visual
data. After calibrating four axis points, this intelligent module converts
pixel values into the raw data domain. Subsequently, our SVG viewing mode,
connected to the Rendering Module, generates the SVG view in real-time as
users update field information. All session information is stored upon consent
acceptance, and a unique Token is assigned to facilitate collaborative work.
The final step includes the export of the visual data to a printable tactile
graphic format (SVG) and to a textual description (CSV). The following
paragraphs describe the two main modules in more detail.

Figure 2. The pipeline of Chart4Blind consists of the input of a bitmap line
chart, followed by the Data Extraction Module, which includes an AI-based line
segmentation and optical character recognition step, and a manual correction
step by a sighted user (a). The Rendering Module updates the information in
real-time and ensures an accessible representation (c). The system allows the
export of the information to an SVG and a CSV format. The SVG can be accessed
with a screen reader or printed as a tactile graphic (d). The metadata can be
exported as accessible CSV as well.

The picture illustrates the pipeline of Chart4Blind. On the far left, a sample
input line chart with three lines enters the first stage (a), which involves
AI-based data extraction. Here, the user can interact with the model
predictions (c). The user also has the option to view progress using different
view modes through the rendering view options (d). Finally, the outputs can be
exported, as shown, to SVG or CSV, and a sample output is depicted on the far
right.

###  4.1. Data Extraction Module

Charts typically encompass two types of information: textual and visual. For
each type, we incorporate an intelligent tool to facilitate extraction with
minimal user interactions. In order to accomplish this, we conducted a
comprehensive review of the state-of-the-art in this field (Bajić and Job,
2023; Masson et al., 2023b; Ying et al., 2023) and opted to utilize a
segmentation model for line data extraction and OCR for the textual content.
The reason behind our selection can be summarized as follows:

  * •

A light model capable of running offline on standard consumer hardware
avoiding external server infrastructure and minimizing backend computational
demands.

  * •

A model that generates results that could be adjusted and validated by the
user in order to simplify the interaction.

####  4.1.1. Instance Segmentation Feature

A line chart primarily exists in the pixel domain. The image itself does not
contain the original table data used for its creation. Hence, a crucial step
involves converting from the pixel domain to the raw domain. This conversion
task requires three specific conditions. Firstly, segmentation of the line of
interest. Secondly, two calibration points on each axis to map pixels, given
that the axes are straight lines. Lastly, the data format of labels (e.g.,
linear, logarithmic, date, etc.).

To address the initial challenge with minimal interactions, we conducted
experiments utilizing instance segmentation models. This approach enables the
segmentation of each line into a distinct mask, providing the flexibility to
trace each line. Users gain control over selecting which line to segment.
Motivated by Ying et al. work (Lal et al., 2023), we trained a Mask2Former
(Cheng et al., 2022) on the LG dataset (Moured et al., 2023b). Unlike
LineFormer, the LG dataset we used comprises real charts with human
annotations, which we found to be more robust for the charts users might
upload. The model achieved a score of 62.12 IOU (Intersection over Union) on
the test set. From the segmentation masks, we sampled key points, equidistant
from each other, as demonstrated in Figure 4. Overlaying the masks on the
chart empowers the user to choose which line to process and allows control
over increasing or decreasing the number of key points extracted.

Regarding the calibration points, we offer users four initialized points
positioned close to the Y and X axes. Users are then guided to position these
four calibration points onto the axis ticks, input the corresponding label
values, and choose the data format from a drop-down menu, including time,
logarithmic, and linear data types. The conversion to the raw domain is then
carried out following the same linear approach as (Rohatgi, 2014) and (Huwaldt
and Steinhorst, 2023).

####  4.1.2. OCR Feature

To categorize the textual content in the line chart, we followed the analysis
outlined in (Tang et al., 2023) and (Moured et al., 2023b). The chosen
categories are shown in Table 2. For each category, a corresponding text field
has been added within the metadata tab in the interface. We utilize the
Tesseract OCR system (Kay, 2007), operating on the user’s browser even when
offline. This particular model has demonstrated a good performance in the
ICDAR chart text understanding challenge (Yao et al., 2015). We employed a
template-based approach (Alam et al., 2023) to auto-fill chart descriptions.
This equipped the chart with a summary of the encoded elements and descriptive
statistics (e.g., extremes, outliers, etc.), corresponding to Levels 1 and 2,
as proven preferable by many BVI individuals (Lundgard and Satyanarayan,
2021).

Table 2. Text fields present in the Chart4Blind interface. Property | Fields
---|---
Calibration | Axes calibration points
Chart Information | Plot title
Axes titles
Axes labels
Additional Information | Chart description
Data point description

###  4.2. Rendering Module

To address UR-1 and UR-5 findings, we incorporated a real-time view to track
conversion progress. Users can switch between modes to visualize textual or
line drawings. Our rendering module displays a real-time SVG view using D3.js
and exports results at the end of the session. D3.js was chosen for its memory
efficiency and rich feature set, surpassing other DOM manipulation methods
(Benbba, 2021). It also complies with W3C (Initiative, 2023) standards and
enables interactive chart creation, beneficial for future audio integration to
test screen reader accessibility.

Considering the space and size constraints of printed charts on embossed
paper, adherence to several print guidelines for tactile illustrations (of
North America, 2010; Nagao and Hatanaka, 2005; Edman, 1992) is essential.
Related requirements are summarized as follows:

  * •

Lines should be capable of being distinguished by touch, either by using
different thicknesses or different types of symbols such as a dotted or dashed
line.

  * •

Lines of ¡ 0.4 mm thick should not be used, as it can be difficult to obtain a
sufficient bump on capsule paper.

  * •

Text in tactile illustrations should be written in Braille and oriented
horizontally. A margin of at least 3.0 mm should be left around the Braille
characters.

Our rendering module offers an additionally accessible visualization mode,
allowing users to export SVGs that align with printing guidelines for tactile
graphics (of North America, 2010). This includes a reduced number of axes
labels, typically limited to 3 to 5 labels depending on the page size due to
the size of Braille script. The (default) digitally accessible SVG mode is
more suitable for 2-D tactile displays and screen readers (Figure 3-(b)),
enabling the embedding of more visual content with corresponding description
tags (e.g. ¡desc¿). Figure 3 illustrates a sample rendering outputs for both
digital and print-accessible SVG modes.

Although line charts come in various styles and layouts, our rendering is
well-suited for univariate charts with only one type of data and a single set
of x and y coordinates. Additionally, since our segmentation model doesn’t
automatically predict line encodings, users have the option to set them
manually as needed.

Figure 3. Rendering Module view for a bitmap line chart. (a) displays the
digitally accessible SVG view, ideal for screen readers, and refreshable
tactile devices. (b) shows the print-accessible SVG view, suitable for print
modalities such as embossed papers or laser cut.

The image exhibits two screenshots from the UI Rendering Module. On the left,
the SVG is finely detailed and suited for screen readers or technologies that
provide enhanced interaction modalities. On the right, a more coarse
representation of the chart to accommodate the necessary information in the
printed version, such as larger text.

###  4.3. Chart4Blind Interface

Informed by insights obtained from need-finding interviews with our
participants and guided by our design principles, we developed the Chart4Blind
interface. This interface allows users to upload a chart image and facilitates
the generation of an accessible version with text descriptions in diverse
formats. In the course of our design process, we engaged two participants from
the need-finding interviews to gather expert feedback. We carefully considered
this feedback to enhance the seamless conversion process within the interface.
The interface components are illustrated in Figure 4.

####  4.3.1. Intuitive Selection Tools

In line with our objective to minimize the number of clicks required in the
process of chart conversion, we provide two options for text extraction. The
first option enables users to one-click on the OCR icon from the toolbar for
text extraction, prompting the localization and prediction of all textual
content in the chart, Figure 4 (b). Users can then drag and drop the text into
the appropriate fields. The second option allows users to select a specific
text of interest. Both options support batch selection for axes labels, which
are automatically sorted in the corresponding fields, eliminating the need to
select each label separately.

Segmented points on each line are also modifiable. Users have the capability
to shift a point, delete, or add points with right and left mouse clicks
respectively. To improve accuracy, we incorporated a zoom window that enlarges
the surrounding area around the mouse cursor following the PlotDigitizer
approach (Huwaldt and Steinhorst, 2023). Given the potential for errors during
this stage, we log a history of labeling actions, enabling users to undo or
redo actions within the interface.

Figure 4. An overview of Chart4Blind interface sections: (a) Home menu for
actions like upload, undo, redo, and tutorials. (b) AI toolbar with OCR and
segmentation models. (c) Canvas for the uploaded chart, allowing interaction
for calibration points and predicted line adjustments. (d) Rendering Module
for real-time SVG visualization before export. (e) Metadata section for
visualizing extracted line data and seamlessly drag-and-drop of textual
content.

An overview of Chart4Blind interface sections: (a) Home menu for actions like
upload, undo, redo, and tutorials. (b) AI toolbar with OCR and segmentation
models. (c) Canvas for the uploaded chart, allowing interaction for
calibration points and predicted line adjustments. (d) Rendering Module for
real-time SVG visualization before export. (e) Metadata section for
visualizing extracted line data and seamlessly drag-and-drop of textual
content.

####  4.3.2. Interactive Tutorials and Accessibility Guidelines

As we expect users who may not be familiar with chart conversion or
accessibility guidelines to use the app, we have incorporated two tutorials.
These tutorials serve as a reference to help users understand how to utilize
the tool effectively. The first tutorial is interactive and is presented when
opening the app for the first time. A sample chart is loaded, with pop-up
short animated GIFs and text descriptions. users can interact with tutorials
and experiment with the tools at the same time.

The second tutorial is designed to inform users about the accessibility
guidelines. This tutorial showcases an accessible SVG chart within the
rendering module. Similar to the previous tutorial, components are highlighted
with corresponding descriptions. Notably, the accessibility tutorial has
played a crucial role in educating users about the importance of chart
descriptions, leading to an increase in the number of charts with accessible
descriptions, as will be discussed in our user study.

##  5\. Usability Study

We conducted a user study to see if the current implementation of Chart4Blind
fulfils our design principles in terms of supporting user requirements
discussed in the findings section 3.3 to semi-automatically support the
creation of accessible charts according to current standards (of North
America, 2010). We also conducted a follow-up study measuring the
accessibility of our exports involving BVI individuals.

###  5.1. Participant

We invited a total of 10 sighted participants (T1-10, age range 22-34 years).
Similar to the previous study, we collaborated with ACCESS@KIT to recruit
participants via email lists. They did not receive any compensation. We
screened participants for basic knowledge of charts: all participants were
familiar with reading line charts and frequently worked with them. While their
educational backgrounds varied, none of the participants had prior experience
with chart accessibility or the conversion process. The study was part of a
series of studies which were approved by the ethical review committee of
Karlsruhe Institute of Technology.

###  5.2. Study Design

We prepared three line charts from our test set, for the conversion process
and one simple chart for the tutorial session to help users become familiar
with the tool. We consider three levels of complexity of charts to experiment
with the conversion process, which we define as follows:

  * •

Simple: Few lines with few data points and labels.

  * •

Compound: Two or more lines with different label formats.

  * •

Dense: Complex lines such as long sinusoidal waves or overlapping trends with
text annotations.

We randomly selected line charts meeting the established criteria to ensure
diversity (see Figure 5). These charts were collected from the recent LG
dataset (Moured et al., 2023b), featuring real charts from public documents
across 5 distinct fields (e.g., social science, natural science, etc.). The
charts are provided in PNG format, accompanied by the ground-truth
hierarchical segmentation masks for all visual and textual elements.

Each participant completed a total of 3 sessions, progressing from Simpler to
Denser charts. For each session, we recorded task completion time (measured in
seconds), mouse clicks, SVG, and CSV exports. Heatmaps were generated using
the recorded mouse clicks. Line point quality was measured using the Frechet
Distance (Alt and Godau, 1995), and the SVGs were utilized in the follow-up
study, as discussed later.

Figure 5. Four line charts with different complexities utilized for the user
study: Simple charts (a) and (b) each contain one simple line trend for the
tutorial and main session respectively. (c) A compound chart with additional
lines overlapped, and visible axes. (d) A dense chart featuring relatively
complex trends, point annotations, and less visible axes.

Four line charts with different complexities utilized for the user study:
Simple charts (a) and (b) each contain one simple line trend. (c) A compound
chart with additional lines overlapped, and visible axes. (d) A dense chart
featuring relatively complex trends, point annotations, and less visible axes.

###  5.3. Procedure

After completing the consent form, participants were invited to a face-to-face
user study. They went through a tutorial session to familiarize themselves
with the Chart4Blind tool. During the tutorial, we presented the Chart4Blind
tool using a sample chart (Figure 5 (a)) that was not used in the subsequent
sessions. Additionally, participants were guided to an accessibility tutorial
to understand the expected results. In order to get comfortable with the tool,
we allowed the participant to interact with it for 15-20 minutes.

In the main sessions, each of the 10 participants was provided with three
charts (see Figure 5 (c-d)) and asked to upload them and start the conversion
process. Participants were informed that they were free to choose their
preferred approach, whether utilizing the integrated AI tools or performing
manual metadata labelling. They were also informed that they could revisit
both the tool and accessibility tutorials at any time without losing progress
if they had any questions.

After completing all the trials, the participants were presented with the SUS
survey (Jordan et al., 1996), followed by open-ended questions to express
their opinions and thoughts. The entire experiment took approximately 90
minutes to complete. We developed an offline experimental system to facilitate
the study, setting up Chart4Blind on a local laptop, with the tool accessible
through the Chrome browser.

###  5.4. Results and Discussion

####  5.4.1. Task Performance

We measured the average time in seconds spent on each chart as illustrated in
Figure 5. For the initial tutorial part, users took, on average, 9min and 5sec
(STD: 3min and 45sec), with differences among users—some were interested in
understanding more about accessibility guidelines, while others chose to go
through the tutorial via GIF animations only.

For the main session, we observed that participants completed the conversion
process in an average of 4min and 36sec (STD: 2min 55sec). We informally asked
the most experienced participant about how long it takes on average to create
a chart with the tools she used (LibreOffic Draw). She reported a 10 minutes
duration.

Figure 6. On the left, average conversion task completion time in
minutes:seconds. On the right, a radar chart depicting the number of clicks
for the top 5 sections interacted with in Chart4Blinds.

On the left, average conversion task completion time in minutes:seconds. On
the right, a radar chart depicting the number of clicks for the top 5 sections
interacted with in Chart4Blinds.

####  5.4.2. Interaction Patterns

To conduct a more comprehensive analysis of the time spent, we closely
examined user click patterns and the duration they spent on various parts of
the interface. In Figure 5-b, a radar chart demonstrates the average number of
clicks on the top 5 interacted components of the UI. The analysis suggests an
influence on completion time when incorporating the segmentation model. All
users started with initial predictions, later making slight adjustments to the
misaligned points, specifically focusing on points situated around the corners
of the curved lines (see Figure 5-c). This trend indicates a decline in
segmentation performance in these specific areas, suggesting a need for
improvement in future work. However, in contrast to annotating the entire line
manually, this discrepancy in performance is minimal compared to the overall
number of points. In the case of the Dense chart, users edited $10.0\%$ of the
overall points.

Further investigation revealed that the time spent varied based on the
elements within the charts. For instance, in charts with more textual
components like the Compound chart, users tended to spend more time engaging
with the text fields. On the other side, in the Dense chart, users engaged
more with the rendering module, likely due to a few misaligned points caused
by the curved lines. As users gained more experience and knowledge of
accessibility guidelines, we observed a decrease in interactions with the
menubar in each subsequent session.

####  5.4.3. Post-study Feedback

User satisfaction was assessed through a SUS survey (Jordan et al., 1996)
conducted after the interview. The study yielded an average score of $90$. All
participants either agreed or strongly agreed that the Chart4Blind tool is
valuable for converting line charts into accessible formats and helps to
understand the accessibility guidelines. Additionally, they agreed that they
would recommend this tool to other colleagues.

We conducted a follow-up discussion to gather participant insights, covering
the following aspects:

  * •

Describe your experience with the tool.

  * •

What did you find most intuitive and why?

  * •

What parts do you find difficult or confusing and why?

  * •

What suggestions or functions you would like to see or improve in the tool?

All participants expressed a positive experience with the tutorials, finding
them informative enough to create an accessible chart. They felt confident
following the conversion process through the rendering view, even without
prior experience. T8 remarked, ”I didn’t feel the difference in converting
Dense and Simple charts; they both took similar effort,” indicating that the
tool was seamless to utilize regardless of chart complexity.

6 participants found the line segmentation feature valuable as it
significantly reduced the effort required to trace lines manually. A few
participants also valued the OCR and drag-and-drop features equally alongside
the segmentation model.

Interviews further revealed that while Chart4Blind generally met their needs,
some participants suggested the following improvements: Three participants
mentioned that automating the calibration step would be helpful, expressing an
occasional lack of confidence when placing calibration points. T5 explained,
”This chart [Dense] has a transparent axis with light gridlines; maybe other
charts have no axis. I can’t locate efficiently even with the zoom window.”

Our sighted participants found the template-based summaries good but expressed
a preference for more contextual summarizations reflecting domain knowledge
presented in the chart. T10 suggested further integration of natural language
models to enhance chart summarization.

##  6\. Exploratory Evaluation of Output Accessibility

As an exploratory effort to evaluate output accessibility with assistive
technologies, we conducted a remote study. We recruited three visually
impaired individuals via the ACCESS@KIT center email list. We randomly
selected three converted charts from our test set, shown in Figure 7, and sent
them to the participants, including embossed prints via mail and SVG exports
via email. Please refer to our supplementary materials for the SVGs.
Participants were instructed to review the charts using a screen reader first
and then explore the corresponding embossed print. We utilized the SoSciSurvey
tool (Leiner, 2019) to create an online questionnaire accessible with screen
readers. The complete version of the survey is available in the supplementary
materials. Participants were guided to answer two questions:

  * •

Could you describe your experience in using a screen reader to access the
provided SVG? Were both the SVG and the accompanying chart description
digitally accessible to you?

  * •

Please discuss your interaction with the provided embossed print. Were you
able to access both the visual and textual elements effectively?

Figure 7. Three printed tactile charts sent to our BVI individuals. The top
row displays the original chart images, while the bottom row presents the
tactile versions. For better resolution please refer to our supplementary
materials.

Three printed tactile charts sent to our BVI individuals. The top row displays
the original chart images, while the bottom row presents the tactile versions.
For better resolution please refer to our supplementary materials.

All participants are familiar with the usage of screen readers, and with
reading charts on tactile prints provided by ACCESS@KIT due to their studies
in STEM subjects.

A few notable points were analyzed: two participants found the template
summarization informative, while one preferred more comprehensive descriptions
that also summarize the overall trends and patterns of the data. Furthermore,
participants reported a few missing elements: plot title in chart (a), as well
as missing legends in charts (b) and (c), highlighting potential errors made
by sighted users when converting the graphics to an accessible format.
Regarding the tactile prints, one participant mentioned that a few labels were
very close to the border of the text bounding box (see Figure 7-a and b),
making it slightly more difficult to interpret.

In response to these observations, we made updates to the SVG rendering
attributes to maintain a larger distance between the border and the inner text
(see Figure 7-c). Additionally, we are actively working on implementing a
status system that notifies the user if any information is still missing
before the export.

##  7\. Limitations and Future Work

The evaluation of Chart4Blind indicates that utilizing intelligent solutions
for chart accessibility enhanced the efficiency and effectiveness of the
conversion process for sighted individuals. The study also revealed important
lessons and helped us identify the limitations that we would like to lift in
future work.

Chart analysis limitations: Currently, Chart4Blind exclusively supports line
charts. We initially chose this type because it is commonly found in documents
and, unfortunately, due to dataset constraints, we could only train the
segmentation model on the LG dataset provided by Omar et al. (Moured et al.,
2023b). We considered synthesizing charts as an alternative, but it yielded
biased and unrealistic performance with the real charts. As the field of chart
analysis advances, we aim to support a broader range of chart types, including
bar charts, pie charts, and scatter plots. Nevertheless, the current instance
segmentation paradigm may not be adaptable to all chart types. For example,
segmentation models may struggle with very faint dots in scatter plots, as
they consist of very few pixels, potentially increasing the complexity of the
models. Future research may introduce additional modalities that users can
select from to extract metadata.

LLMs for chart summarization: Large Language Models (LLMs) have recently
demonstrated significant potential in achieving contextual understanding from
images. For example, Smith et al. highlighted the effectiveness of ByT5 model
in generating structured chart summarizations from images for BVI individuals
(Tang et al., 2023). Our user studies revealed that BVI participants favored
having more detailed descriptions of the trends and patterns observed in the
chart, while sighted participants aimed to minimize the time spent on
generating these descriptions. To address this, we are exploring more
collaborative authoring techniques using LLMs to streamline the chart
description generation process

Limitation of user study: Our initial evaluation of Chart4Blind was conducted
through user studies in a controlled environment, focusing solely on line
charts and within a restricted timeframe. Additionally, our interaction logs
and collected data are limited, affecting the depth of our analysis. To enable
more thorough analysis and contributions, we have made our system and models
publicly available, encouraging further exploration from both the
accessibility and chart analysis communities. We are currently conducting
additional studies with experts to address the aforementioned points.

##  8\. Conclusion

We have introduced Chart4Blind, an interactive web-based tool designed to
facilitate a semi-automated accessible chart conversion process using instance
segmentation and OCR models. The tool guides inexperienced users in processing
online charts and exporting accessible visualizations for blind and visually
impaired individuals, supporting various assistive modalities such as embossed
papers, screen readers, and tactile displays. We equip the digital exports
with chart descriptions generated through template-based summaries. Our tool
incorporates interactive features for text selection and annotating line
points. A user study involving inexperienced users confirms that this semi-
supervised approach, coupled with interactive features, simplifies the task of
chart conversion more efficiently and effectively. We anticipate that
Chart4Blind will significantly contribute to making information visualization
and analytics more accessible to everyone. Future work includes the
incorporation of other diagram types. To encourage further research in this
relatively unexplored area, Chart4Blind is accessible through the project page
https://moured.github.io/chart4blind/.

###### Acknowledgements.

This work was supported by the European Union’s Horizon $2020$ research and
innovation program under the Marie Sklodowska-Curie Grant No.$861166$. The AI
experiments were conducted in part on the HoreKa supercomputer, funded by the
MWK and the Federal Ministry of Education and Research.

## References

  * (1)
  * Alam et al. (2023) Md Zubair Ibne Alam, Shehnaz Islam, and Enamul Hoque. 2023.  SeeChart: Enabling Accessible Visualizations Through Interactive Natural Language Interface For People with Visual Impairments. In _Proceedings of the 28th International Conference on Intelligent User Interfaces_. 46–64.
  * Alt and Godau (1995) Helmut Alt and Michael Godau. 1995.  Computing the Fréchet distance between two polygonal curves.  _International Journal of Computational Geometry & Applications_ 5, 01n02 (1995), 75–91.
  * Altmanninger and Wöß (2006) Kerstin Altmanninger and Wolfram Wöß. 2006.  Dynamically generated scalable vector graphics (SVG) for barrier-free web-applications. In _Computers Helping People with Special Needs: 10th International Conference, ICCHP 2006, Linz, Austria, July 11-13, 2006. Proceedings 10_. Springer, 128–135.
  * Araki et al. (2014) Kosuke Araki, Tetsuya Watanabe, and Kazunori Minatani. 2014.  Development of Tactile Graph Generation Software Using the R Statistics Software Environment. In _Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility_ (Rochester, New York, USA) _(ASSETS ’14)_. Association for Computing Machinery, New York, NY, USA, 251–252.
  * Bajić and Job (2023) Filip Bajić and Josip Job. 2023.  Review of chart image detection and classification.  _International Journal on Document Analysis and Recognition (IJDAR)_ (2023), 1–22.
  * Balaji et al. (2018) Abhijit Balaji, Thuvaarakkesh Ramanathan, and Venkateshwarlu Sonathi. 2018.  Chart-text: A fully automated chart image descriptor.  _arXiv preprint arXiv:1812.10636_ (2018).
  * Benbba (2021) Safwane Benbba. 2021.  _Comparison of D3. js and Chart. js as visualisation tools_.  B.S. thesis.
  * Bigham et al. (2006) Jeffrey P. Bigham, Ryan S. Kaminsky, Richard E. Ladner, Oscar M. Danielsson, and Gordon L. Hempton. 2006.  WebInSight: Making Web Images Accessible. In _Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility_ (Portland, Oregon, USA) _(Assets ’06)_. Association for Computing Machinery, New York, NY, USA, 181–188.
  * Blanco et al. (2022) Matthew Blanco, Jonathan Zong, and Arvind Satyanarayan. 2022.  Olli: An extensible visualization library for screen reader accessibility.  _IEEE VIS Posters_ 6 (2022).
  * Bru et al. (2023) Egil Bru, Thomas Trautner, and Stefan Bruckner. 2023.  Line Harp: Importance-Driven Sonification for Dense Line Charts.  _arXiv preprint arXiv:2307.16589_ (2023).
  * Butler et al. (2021) Matthew Butler, Leona M Holloway, Samuel Reinders, Cagatay Goncu, and Kim Marriott. 2021.  Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020. In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ (Yokohama, Japan) _(CHI ’21)_. Association for Computing Machinery, New York, NY, USA, Article 278, 15 pages.
  * Chaudhuri et al. (2017) Arindam Chaudhuri, Krupa Mandaviya, Pratixa Badelia, Soumya K Ghosh, Arindam Chaudhuri, Krupa Mandaviya, Pratixa Badelia, and Soumya K Ghosh. 2017.  _Optical character recognition systems_.  Springer.
  * Cheng et al. (2022) Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexander Kirillov, and Rohit Girdhar. 2022.  Masked-attention Mask Transformer for Universal Image Segmentation.  _CVPR_.
  * Cheng et al. (2023) Zhi-Qi Cheng, Qi Dai, Siyao Li, Jingdong Sun, Teruko Mitamura, and Alexander G Hauptmann. 2023.  ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules.  _arXiv preprint arXiv:2304.02173_ (2023).
  * Choi et al. (2019) Jinho Choi, Sanghun Jung, Deok Gun Park, Jaegul Choo, and Niklas Elmqvist. 2019.  Visualizing for the non-visual: Enabling the visually impaired to use visualization. In _Computer Graphics Forum_ , Vol. 38. Wiley Online Library, 249–260.
  * Conservancy (2023) Software Freedom Conservancy. 2023.  Inkscape: Open Source Scalable Vector Graphics Editor.  Retrieved January 18, 2024 from https://inkscape.org
  * Consortium et al. (1999) World Wide Web Consortium et al. 1999\.  Web content accessibility guidelines 1.0.  (1999).
  * Devine et al. (2011) Heather Devine, Andres Gonzalez, and Matthew Hardy. 2011.  Making accessible PDF documents. In _Proceedings of the 11th ACM symposium on Document engineering_. 275–276.
  * Edman (1992) Polly Edman. 1992.  _Tactile graphics_.  American Foundation for the Blind.
  * Engel et al. (2019) Christin Engel, Emma Franziska Müller, and Gerhard Weber. 2019.  SVGPlott: an accessible tool to generate highly adaptable, accessible audio-tactile charts for and from blind and visually impaired people. In _Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments_. 186–195.
  * Engel and Weber (2017) Christin Engel and Gerhard Weber. 2017.  Improve the accessibility of tactile charts. In _Human-Computer Interaction-INTERACT 2017: 16th IFIP TC 13 International Conference, Mumbai, India, September 25–29, 2017, Proceedings, Part I 16_. Springer, 187–195.
  * Engel and Weber (2018) Christin Engel and Gerhard Weber. 2018.  A user study to evaluate tactile charts with blind and visually impaired people. In _Computers Helping People with Special Needs: 16th International Conference, ICCHP 2018, Linz, Austria, July 11-13, 2018, Proceedings, Part II 16_. Springer, 177–184.
  * Farahani et al. (2023) Ali Mazraeh Farahani, Peyman Adibi, Mohammad Saeed Ehsani, Hans-Peter Hutter, and Alireza Darvishy. 2023.  Automatic Chart Understanding: A Review.  _IEEE Access_ 11 (2023), 76202–76221.
  * Ferraiolo et al. (2000) Jon Ferraiolo, Fujisawa Jun, and Dean Jackson. 2000.  _Scalable vector graphics (SVG) 1.0 specification_.  iuniverse Bloomington.
  * Foundation (2020) The Document Foundation. 2020.  _LibreOffice Draw_.  Retrieved January 18, 2024 from https://www.libreoffice.org/discover/draw/
  * Foundation (2023) The Document Foundation. 2023.  _LibreOffice_.  Retrieved January 18, 2024 from https://www.libreoffice.org
  * Gale (2005) Gillian Gale. 2005.  Guidelines on Conveying Visual Information.  _Round Table on Information Access for People with Print Disabilities Inc. Available from http://printdisability. org/guidelines/guidelines-on-conveying-visual-information-2005_ (2005).
  * Godfrey and Loots (2014) A Jonathan R Godfrey and M Theodor Loots. 2014.  Statistical software (R, SAS, SPSS, and Minitab) for blind students and practitioners.  _Journal of Statistical Software_ 58 (2014), 1–25.
  * Goncu and Marriott (2008) Cagatay Goncu and Kim Marriott. 2008.  Tactile Chart Generation Tool. In _Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility_ (Halifax, Nova Scotia, Canada) _(Assets ’08)_. Association for Computing Machinery, New York, NY, USA, 255–256.
  * Hassan et al. (2023) Muhammad Yusuf Hassan, Mayank Singh, et al. 2023\.  LineEX: Data Extraction From Scientific Line Charts. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_. 6213–6221.
  * Hermann et al. (2011) Thomas Hermann, Andy Hunt, John G Neuhoff, et al. 2011\.  _The sonification handbook_. Vol. 1.  Logos Verlag Berlin.
  * Holloway et al. (2022) Leona M Holloway, Cagatay Goncu, Alon Ilsar, Matthew Butler, and Kim Marriott. 2022.  Infosonics: Accessible infographics for people who are blind using sonification and voice. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_. 1–13.
  * Huwaldt and Steinhorst (2023) Joseph A Huwaldt and S Steinhorst. 2023.  Plot digitizer.  Retrieved January 18, 2024 from http://plotdigitizer.sourceforge.net
  * Initiative (2023) Web Accessibility Initiative. 2023.  _W3C_.  Retrieved January 18, 2024 from https://www.w3.org/WAI/tutorials/images/
  * Jayant et al. (2007) Chandrika Jayant, Matt Renzelmann, Dana Wen, Satria Krisnandi, Richard Ladner, and Dan Comden. 2007.  Automated Tactile Graphics Translation: In the Field. In _Proceedings of the 9th International ACM SIGACCESS Conference on Computers and Accessibility_ (Tempe, Arizona, USA) _(Assets ’07)_. Association for Computing Machinery, New York, NY, USA, 75–82.
  * Jordan et al. (1996) Patrick W Jordan, Bruce Thomas, Ian Lyall McClelland, and Bernard Weerdmeester. 1996.  _Usability evaluation in industry_.  CRC Press.
  * Jung et al. (2017) Daekyoung Jung, Wonjae Kim, Hyunjoo Song, Jeong-in Hwang, Bongshin Lee, Bohyoung Kim, and Jinwook Seo. 2017.  ChartSense: Interactive Data Extraction from Chart Images. In _Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems_ (Denver, Colorado, USA) _(CHI ’17)_. Association for Computing Machinery, New York, NY, USA, 6706–6717.
  * Kantharaj et al. (2022) Shankar Kantharaj, Rixie Tiffany Ko Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. 2022.  Chart-to-Text: A Large-Scale Benchmark for Chart Summarization.  arXiv:2203.06486 [cs.CL]
  * Kay (2007) Anthony Kay. 2007.  Tesseract: an open-source optical character recognition engine.  _Linux Journal_ 2007, 159 (2007), 2.
  * Krufka and Barner (2005) Stephen E. Krufka and Kenneth E. Barner. 2005.  Automatic Production of Tactile Graphics from Scalable Vector Graphics. In _Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility_ (Baltimore, MD, USA) _(Assets ’05)_. Association for Computing Machinery, New York, NY, USA, 166–172.
  * Lal et al. (2023) Jay Lal, Aditya Mitkari, Mahesh Bhosale, and David Doermann. 2023.  LineFormer: Rethinking Line Chart Data Extraction as Instance Segmentation.  arXiv:2305.01837 [cs.CV]
  * Leiner (2019) D.J. Leiner. 2019.  _SoSci Survey – the Solution for Professional Online Questionnaires_.  Retrieved October 6, 2023 from https://www.soscisurvey.de/en/index
  * Li et al. (2019) Jingyi Li, Son Kim, Joshua A Miele, Maneesh Agrawala, and Sean Follmer. 2019.  Editing spatial layouts through tactile templates for people with visual impairments. In _Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems_. 1–11.
  * Lundgard and Satyanarayan (2021) Alan Lundgard and Arvind Satyanarayan. 2021.  Accessible visualization via natural language descriptions: A four-level model of semantic content.  _IEEE transactions on visualization and computer graphics_ 28 (2021), 1073–1083.
  * Lundgard and Satyanarayan (2022) Alan Lundgard and Arvind Satyanarayan. 2022.  Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content.  _IEEE Transactions on Visualization and Computer Graphics_ 28, 1 (2022), 1073–1083.
  * Masson et al. (2023a) Damien Masson, Sylvain Malacria, Daniel Vogel, Edward Lank, and Géry Casiez. 2023a.  ChartDetective: Easy and Accurate Interactive Data Extraction from Complex Vector Charts. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ (Hamburg, Germany) _(CHI ’23)_. Association for Computing Machinery, New York, NY, USA, Article 147, 17 pages.
  * Masson et al. (2023b) Damien Masson, Sylvain Malacria, Daniel Vogel, Edward Lank, and Géry Casiez. 2023b.  ChartDetective: Easy and Accurate Interactive Data Extraction from Complex Vector Charts. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–17.
  * Moured et al. (2023a) Omar Moured, Sara Alzalabny, Thorsten Schwarz, Bastian Rapp, and Rainer Stiefelhagen. 2023a.  Accessible Document Layout: An Interface for 2D Tactile Displays. In _Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments_. 265–271.
  * Moured et al. (2023b) Omar Moured, Jiaming Zhang, Alina Roitberg, Thorsten Schwarz, and Rainer Stiefelhagen. 2023b.  Line Graphics Digitization: A Step Towards Full Automation. In _International Conference on Document Analysis and Recognition_. Springer, 438–453.
  * Nagao and Hatanaka (2005) H Nagao and S Hatanaka. 2005.  Braille Books and Translation of Illustration by Using PCs.  _Dokusho Kobo, Tokyo_ (2005).
  * Nagy et al. (1999) George Nagy, Thomas A. Nartker, and Stephen V. Rice. 1999.  Optical character recognition: an illustrated guide to the frontier. In _Document Recognition and Retrieval VII_ , Daniel P. Lopresti and Jiangying Zhou (Eds.), Vol. 3967. International Society for Optics and Photonics, SPIE, 58 – 69.
  * of North America (2010) Braille Authority of North America. 2010.  Guidelines and Standards for Tactile Graphics.  Retrieved January 17, 2024 from https://www.brailleauthority.org/tg/web-manual/index.html
  * on Information Access for People with Print Disabilities (2022) Round Table on Information Access for People with Print Disabilities. 2022.  Guidelines for producing accessible graphics / Round Table on Information Access for People with Print Disabilities Inc.  (2022).  Retrieved January 18, 2024 from https://printdisability.org/guidelines/graphics-2022/
  * Prescher et al. (2014) Denise Prescher, Jens Bornschein, and Gerhard Weber. 2014.  Production of accessible tactile graphics. In _Computers Helping People with Special Needs: 14th International Conference, ICCHP 2014, Paris, France, July 9-11, 2014, Proceedings, Part II 14_. Springer, 26–33.
  * Rohatgi (2014) Ankit Rohatgi. 2014.  WebPlotDigitizer user manual version 3.4.  (2014), 1–18.  Retrieved January 18, 2024 from https://automeris.io/WebPlotDigitizer/userManual.pdf
  * Serrano et al. (2021) J Serrano, GM Bruscas, JV Abellán, and R Lázaro. 2021.  Study of additive manufacturing techniques to obtain tactile graphics. In _IOP Conference Series: Materials Science and Engineering_. IOP Publishing, 012117.
  * Sharif et al. (2022) Ather Sharif, Olivia H. Wang, Alida T. Muongchan, Katharina Reinecke, and Jacob O. Wobbrock. 2022.  VoxLens: Making Online Data Visualizations Accessible with an Interactive JavaScript Plug-In. In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ (New Orleans, LA, USA) _(CHI ’22)_. Association for Computing Machinery, New York, NY, USA, Article 478, 19 pages.
  * Sheppard and Aldrich (2001) Linda Sheppard and Frances K Aldrich. 2001.  Tactile graphics in school education: perspectives from teachers.  _British Journal of Visual Impairment_ 19, 3 (2001), 93–97.
  * Shneiderman (1996) Ben Shneiderman. 1996.  The eyes have it: A task by data type taxonomy for information visualizations. In _Proceedings 1996 IEEE symposium on visual languages_. IEEE, 336–343.
  * Singh et al. (2023) Mandhatya Singh, Muhammad Suhaib Kanroo, Hadia Showkat Kawoosa, and Puneet Goyal. 2023.  Towards accessible chart visualizations for the non-visuals: Research, applications and gaps.  _Computer Science Review_ 48 (2023), 100555.
  * Swaminathan et al. (2016) Saiganesh Swaminathan, Thijs Roumen, Robert Kovacs, David Stangl, Stefanie Mueller, and Patrick Baudisch. 2016.  Linespace: A Sensemaking Platform for the Blind. In _Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems_ (San Jose, California, USA) _(CHI ’16)_. Association for Computing Machinery, New York, NY, USA, 2175–2185.
  * Tang et al. (2023) Benny Tang, Angie Boggust, and Arvind Satyanarayan. 2023.  VisText: A Benchmark for Semantically Rich Chart Captioning. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Toronto, Canada, 7268–7298.
  * Taylor (2023) Petroc Taylor. 2023.  _Amount of data created, consumed, and stored 2010-2020, with forecasts to 2025_.  Retrieved January 18, 2024 from https://www.statista.com/statistics/871513/worldwide-data-created/
  * Thompson et al. (2023) John R Thompson, Jesse J Martinez, Alper Sarikaya, Edward Cutrell, and Bongshin Lee. 2023.  Chart Reader: Accessible Visualization Experiences Designed with Screen Reader Users. In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_. 1–18.
  * Tummers (2006) B. Tummers. 2006.  DataThief III.  Retrieved January 18, 2024 from https://datathief.org/
  * Unger and Chandler (2012) Russ Unger and Carolyn Chandler. 2012.  _A Project Guide to UX Design: For user experience designers in the field or in the making_.  New Riders.
  * Vaingast (2014) Shai Vaingast. 2014.  im2graph.  Retrieved January 18, 2024 from https://www.im2graph.co.il/
  * Watanabe et al. (2012) Tetsuya Watanabe, Toshimitsu Yamaguchi, and Masaki Nakagawa. 2012.  Development of software for automatic creation of embossed graphs: Comparison of non-visual data presentation methods and development up-to-date. In _Computers Helping People with Special Needs: 13th International Conference, ICCHP 2012, Linz, Austria, July 11-13, 2012, Proceedings, Part I 13_. Springer, 174–181.
  * Yao et al. (2015) Cong Yao, Jianan Wu, Xinyu Zhou, Chi Zhang, Shuchang Zhou, Zhimin Cao, and Qi Yin. 2015.  Incidental scene text understanding: Recent progresses on icdar 2015 robust reading competition challenge 4.  _arXiv preprint arXiv:1511.09207_ (2015).
  * Ying et al. (2023) Lu Ying, Yun Wang, Haotian Li, Shuguang Dou, Haidong Zhang, Xinyang Jiang, Huamin Qu, and Yingcai Wu. 2023.  Reviving Static Charts into Live Charts.  _arXiv preprint arXiv:2309.02967_ (2023).
  * Zou and Treviranus (2015) Hong Zou and Jutta Treviranus. 2015.  ChartMaster: A tool for interacting with stock market charts using a screen reader. In _Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility_. 107–116.