# Structure Editor for Building Software Models

Mohammad Nurullah Patwary  The University of Texas at ArlingtonArlington, TX,
USA<EMAIL_ADDRESS>,  Ana Jovanovic  The University of Texas at
ArlingtonArlington, TX, USA<EMAIL_ADDRESS>and  Allison Sullivan
The University of Texas at ArlingtonArlington, TX, USA
<EMAIL_ADDRESS>
###### Abstract.

Alloy is well known a declarative modeling language. A key strength of Alloy
is its scenario finding toolset, the Analyzer, which allows users to explore
all valid scenarios that adhere to the model’s constraints up to a user-
provided scope. Despite the Analyzer, Alloy is still difficult for novice
users to learn and use. A recent empirical study of over 93,000 new user
models reveals that users have trouble from the very start: nearly a third of
the models novices write fail to compile. We believe that the issue is that
Alloy’s grammar and type information is passively relayed to the user despite
this information outlining a narrow path for how to compose valid formulas. In
this paper, we outline a proof-of-concept for a structure editor for Alloy in
which user’s build their models using block based inputs, rather than free
typing, which by design prevents compilation errors.

##  1\. Introduction

Despite their ability to help developers produce correct software, software
models are not widely adopted. Unfortunately, modeling languages are
notoriously difficult to learn, which is compounded by development toolsets
that lag behind the state-of-the-art for integrated development environments
(IDE) (fmedu_tools, ; humanin_fm, ). Alloy is a well known modeling language
that has been used to help validate software designs (CD2Alloy, ; Margrave, ;
ChordAlloy, ; WickersonETAL2017, ; ChongETAL2018, ), to test and debug code
(DiniETALKoratAPI2018, ; MarinovKhurshid01TestEra, ; SamimiETALECOOP2010, ;
ZaeemKhurshidECOOP2010, ) and to provide security analysis of systems
(CheckMateMicro2019, ; websecurity, ; BagheriETAL2018, ). Alloy has a
reputation for being a more user friendly formal method, mainly due to its IDE
the Analyzer, which lets users explore their models by producing a collection
of satisfying instances that visually display allowed and prevented behavior.
Over the years, there have been several efforts to build introductory formal
methods courses centered around Alloy (alloystudy1, ; alloystudy2, ;
alloystudy3, ). Unfortunately, these studies have revealed that even with the
Analyzer, new users still feel overwhelmed trying to learn the language.

One reason for this is that several studies, including those done outside of
an educational setting (spincomparison, ; alloyformalise23study, ), have
revealed that some of the commonly sighted strengths of the Analyzer are not
actually helpful to new users. While one perceived strength, visualized
instances, has received a lot of attention (abstractalloy, ; Hawkeye, ; Reach,
; Amalgam, ; Aluminum, ; PorncharoenwaseETALCompoSat2018, ), a recent
empirical study exploring over 93,000 Alloy models written by novice users
points to another “strength” that can be burdensome: the Analyzer’s compiler-
based error reports (jovanovic2024empirically, ). The Analyzer uses Alloy’s
set of grammar rules as well as an internal type system to flag syntax and
type based compilation errors. If an error is detected, then the user is given
a summary message and problematic portions of the model’s text are highlighted
in red.

Unfortunately, the guidance provided by the compiler’s error reports can be
ambiguous at best, with the text report templates being too generic, and
misleading at worst, with the highlighted formulas compiling in isolation. The
lack of quality feedback provided by these error reports is a cascading
problem, as this study reveals that users struggle with the basics of writing
a model: 72.75% of all users make at least one compilation mistake and 29.28%
of the models overall fail to compile. Furthermore, users can get stuck in
long edit chains trying to make their model compile. 7.48% of the time users
need 5 or more edits to make their model compile, and users even give up
without making a compilable model 2.88% of the time. In order to help
developers write valid models faster and easier, a new feedback mechanism is
needed.

Our theory is that Alloy’s grammar and type rules can help users determine how
to compose their formulas, as these rules combine together to provide an
underlying rigid environment to build formulas in. However, presenting this
information at compile time is too passive and muddles the transfer of
knowledge. Our vision is to enable the user to proactively see the grammatical
and type implications of their choices by building models in a structure
editor. Structure editors enable users to build programs not by free typing in
an editor, but instead by interacting with a tree-structured representation,
eliminating the possibility of introducing syntactic errors. Unfortunately,
structure editors themselves are not widely adopted, as existing editors have
a difficult time balancing the ease of making edit actions with guaranteeing
the structural validity of the code (bahlke1992design, ; bau2017learnable, ;
holwerda2018usability, ). However, given that a user’s options on what to
write next to finish composing a formula are more restricted by Alloy’s
grammar and type system than other languages with structure editors, and that
new users struggle with the grammar, we believe that a structure editor for
Alloy will be beneficial.

1var sig File { var link : lone File }

2var sig Trash in File {}

3var sig Protected in File {}

4

5/* All unprotected files are deleted.*/

6pred inv5 { all x : File | x not in Protected => x in Trash }

7/*Chain of edits from one session from the Alloy4Fun dataset.*/

8inv5 S: all x : !in Protected | x in Trash

9inv5 S: all x : File | x !in Protected | x in Trash

10inv5 S: all x : File | all x !in Protected | x in Trash

11inv5 T: all x : File | x !in Protected -> x in Trash

12

13/* The protected status never changes.*/

14pred inv10 { always Protected = Protected’ }

15/*Chain of edits from Alloy4Fun Dataset.*/

16inv10 T: always Protected once Protected

17inv10 T: after Protected always Protected

18inv10 T: Protected since Protected

19inv10 T: always Protected since Protected

20inv10 T: always (Protected since Protected)

Figure 1. Alloy Model of a File System Trash Can

##  2\. Motivating Example

In this section, we outline key concepts of Alloy and the different types of
compilation errors.

###  2.1. Model of a File System Trash Can in Alloy

Figure 1 displays a model of a file system trash can from the Alloy4Fun
dataset (alloy4funbenchmark, ). Alloy4Fun is an online platform in which users
fill in empty predicates to match a given English description. Signature
paragraphs introduce named sets and can define relations, which outline
relationships between elements of sets. Line 1 introduces a named set File and
establishes that each File atom connects to zero or one (lone) File atoms
through the link relation. Lines 2 and 3 introduce the named sets Trash and
Protected as subsets (in) of File. All signatures and relations are mutable
(var), which means that the elements of these sets can vary across different
states.

Predicates (pred) introduce named formulas that can be invoked elsewhere.
Predicate inv5 uses universal quantification (all), subset (in), subset
exclusion (!in) and implication (=>) to express that if a file is not in the
set of protected files, then the file is in the trash. Predicate inv10 uses
the future temporal operator (always) and set equality (=) to express that the
set of protected files is always the same. To execute predicates, users write
Alloy commands. Commands have a scope that places an upper bound on the size
of all signature sets. The command “run inv5” searches for an assignment to
all sets in the model using up to 3 File atoms by default.

###  2.2. Non-compilable Alloy4Fun Submissions

To illustrate what compilation errors look like, Figure 1 includes two chains
of edits that represent back-to-back submissions made by a user through
Alloy4Fun. ‘S’ indicates a syntax error and ‘T’ a type error. The red
highlights mimics the highlights provided by the error report, which also
includes a written message.

####  2.2.1. Syntax Errors

The chain of edits for inv5 depict a user trying to properly structure a
quantified formula (lines 8-11). First, the user forgets to create the domain
(line 8). For syntactic errors, the text portion of the error report will list
expected tokens; however, this list can be too long to be effective. For line
8, the user is told there are 37 possible tokens expected in place of the
highlighted text. After adding a domain, the error report highlights the
misapplication of the vertical bar keyword on line 9, and informs the users
that there are 38 possible tokens that could apply here. Yet, it takes two
edits for the user to actually remove this incorrect syntax (lines 10-11). On
line 11, the user replaces it with the cross product operator (->) when the
user likely intended to use implication (=>). These two operators are visually
similar, but logically very different. The error report does not direct the
user to this mistake, but instead highlights the subset exclusion operator, as
the set “x” and the set formed by “Product -> x” are not of the same type.

####  2.2.2. Type Errors

The chain of edits for inv10 depicts a user repeatedly making the same high
level type error. Type based error reports provide varying degrees of
feedback. For line 11, the report gives the detailed context that “Left type =
this/File. Right type = this/File->this/File” in the error report.
Unfortunately, for lines 16-20, the report is minimal and only states that
“this expression failed to typecheck.” Fundamentally, the underlying mistake
is that Protected evaluates to a set, which is not inherently something that
is true or false. Meanwhile, the preceding temporal operators expect to reason
over a boolean formula. To illustrate, consider evaluating the following
formulas if Protected = File0:

always (Protected) $\equiv$ always ({File0}) $\equiv$ type erroralways (some
Protected) $\equiv$ always (some {File0}) $\equiv$ true

Since this hanging use of Protected is repeated across all the edits, the
error reports never help the user realize the mistake. Case in point, on line
20, the user resorts to trying to insert parenthesis to resolve the issue
before giving up.

Figure 2. Hello World in Scratch

##  3\. Design Overview

In this section, we outline a proof of concept for a structure editor for
Alloy. Figures 4 and 5 display snapshots of using a structure edit to
construct inv5 and inv10 respectfully, which we will use throughout this
section to highlight how our design can prevent compilation errors and even
guide development.

###  3.1. Blocked Based Editor

Block based editors, which are currently the most popular type of structure
editor, have users build programs by using a drag-n-drop interface to connect
blocks of code. Each block typically represents a syntactic form within the
language, e.g. an if statement. The visual design of blocks can provide
valuable context. For instance, the shape of blocks can indicate how blocks
interconnect. Figure 2 displays a hello world program in Scratch, one of the
most popular block based editor (maloney2010scratch, ). “Event” code blocks
are yellow and represent the start of a program, given by its shape which
allows blocks to be added to the bottom but not the top. The interlocked
purple block is a “Looks” code block that has a character say “hello.”

Scratch has been successfully used in introductory programming courses, even
at the K-12 level (su2022effect, ; duo2023analysis, ; montiel2021educational,
). This later point is of interest to us, as one of the main applications we
envision is to help users learn Alloy faster and easier than is currently
possible using the Analyzer. Therefore, our structure editor is inspired by
the best practices derived from Scratch’s success. As figures 4 and 5 show,
like Scratch, our structure editor is composed of three interconnected panels.
The “Formula Type” panel displays different categories of blocks and the
“Blocks” panel displays all selectable blocks of that type, which can be
dragged and dropped into the “Model” panel canvas to build a formula.

The use of blocks themselves is already helpful at preventing errors, as
blocks remove typos, which would prevent the type error on line 11 in Figure
1. The difference between the implication and the cross product operators is
two similar looking characters that are also neighboring keys on a keyboard,
increasing the chance a user might mix them up. However, in a structure editor
the user would instead select the implication operator and not type it, as
Figure 4 (d) illustrates. While this mistake throws an error, it is possible
for the user to make a typo that compiles, such as mixing up set intersection
and set difference, which are also neighboring keys. Moreover, if the user
actually intended to use the cross product operator, our structure editor will
prevent this placement, since as Figure 4 (c) illustrates, the operator would
not be selectable.

###  3.2. Definition of Blocks

We uses operators and basic sets as the selectable blocks for construction.
For operators, Alloy’s underlying grammar divides operators based on structure
only, e.g. unary or binary. However, we believe structure alone is too broad
of a category for user to easily find the desired operator. For instance, the
binary structure would intermix linear temporal operators (e.g. since) with
relational operators (e.g. +) and with propositional operators operators (e.g.
or). Instead, we plan to organize operators by type of mathematical logic, as
outlined in Figure 3, then by structure. This organization may change
depending on future user studies. For basic sets, the available basic sets for
a hole are all signatures, all relations, and any locally defined variables
within scope of the formula edit.

Scratch’s puzzle piece block shape design guides users to build sequential
steps, which is appropriate for an imperative language. However, Alloy is
declarative, which means there is no control flow. Therefore, our shape design
does not convey order of execution, but we do use our block shape to convey
whether a formula produces a set (rounded) or boolean (squared). The shapes
are used throughout the “Blocks” and “Model” panels, helping to convey the
underlying type structure for construction as well as already built formulas.

Relational Logic Operators:unOp::= ~ | * | ^ | ! | no | lone | some | one | setbinOp::= & | + | - | ++ | <: | :> | . | -> | in | = | < | > | =< | >=Propositional Logic Operators:binOp::= or | and | iff | =>First-order Logic Operators:quant::= all | no | some | lone | oneLinear Temporal Logic Operators:unOp::= always | eventually | after | before | historically | once | ’binOp::= since | triggered | until | releases | ;

Figure 3. Division of Operators for Structure Editor

(a) - prevents Figure 1 line 8 error

(b) - guides correction of Figure 1 line 8 error

(c) - prevents Figure 1 line 11 error

(d) - guides correction of Figure 1 line 11 error

Figure 4. Wireframe of structure editor steps for inv5

###  3.3. Block Based Construction

To construct a formula, users add blocks at open locations, which we will
refer to as holes, within the model. Holes are indicated by either (1) a
labeled bracket, e.g. [domain] in Figure 4 (a), or (2) circle or square icon,
e.g. $\circ$/$\square$, that represent extension points on existing formulas.
A model starts off as a single square extension point. To add blocks, we plan
to explore using (1) the traditional drag-and-drop interface and (2) a pop-up
menu, as seen in Figure 4 (b), that allows users to select a block at a
specific hole. As Section 3.3.2 will detail, not all type errors can be
prevented by distinguishing sets from booleans and we would need an
intractable number of shapes to account for all possible types that could be
introduced. Therefore, we plan to have the user select an active hole. This
enables us to efficiently make blocks that throw errors unselectable, which we
indicate as grayed out blocks, as is the reason we plan to explore a popup
menu interface. We could prevent the user from placing a block into a hole
after selection, but we believe a proactive indicator will help users more
easily determine the right block to match their intent and help educate the
user on how to compose formulas.

####  3.3.1. Preventing Syntax Errors

The design of our two holes ensure users build models that are syntactically
valid. The labeled bracket enables us to let the user know the structural
expectations for a formula. These labeled brackets are similar to Scratch’s
holes, as seen in Figure 2 where the Look block lets the user add “hello” and
“2” after placement. Figure 4 (a)’s labeled brackets conveys to the user that
they will need to define a variable, a domain and a subformula in order to
create a universally quantified formula. This directly prevents the syntax
error from line 8 in Figure 1 from being possible. The extension points enable
us to convey to the user all the locations in which the current formula can be
extended. For instance, Figure 4 (b) shows how the user can select the latter
extension point for File and build out a more intricate domain, such adding a
set intersection block to make “File & Trash.” Furthermore, these extension
points do not force the user to build the formula in a specific order. For
instance, in Figure 4 (b), the user can select the leading extension point and
build the domain “Trash & File.”

|
---|---
(a) - prevents Figure 1 lines 16-20 error | (b) - guides correction of Figure 1 lines 16-20 error
Figure 5. Wireframe of structure editor selection options for building off of
“always” in inv10

####  3.3.2. Preventing Type Errors

There are two classifications of type errors, primitive and violations of
Alloy’s type-checking rules. Primitive type errors occur when a user creates a
formula that produces a set when a boolean is expected or vice versa. Our
shape design handles these errors, as we will enforce that only rounded shapes
can be inserted into rounded holes. We also propagate this information to the
active hole substitution filtering, mentioned in Section 3.3. To illustrate,
when composing inv10, our structure editor prevents the user from directly
adding any of the basic sets, as seen in Figure 5 (a), as they produce sets
when a boolean is needed. This immediately addresses the high level error the
user was never able to correct in Figure 1 lines 16-20. Simply conveying the
primitive type information can have a big impact guiding users. To illustrate,
Figure 5 (b) shows that the user will learn that set-based relational
operators are unselectable but comparison-based relational operators are
selectable. Likewise, Figure 4 (b) and (c) also show examples of this.

However, shape alignment cannot prevent all type based errors. For instance,
for the type error on line 11 in Figure 1, “x” and “Protected->x” are both
sets, but they have different internal types. Sets in Alloy have internal
types related to their element’s relationship with the signatures and
relations of the model. Type-checking errors occur when sets of different
internal types are used within the same subformula. We can use constraint
checking to determine if inserting a given block into a hole will create a
type conflict. This enable us to provide nuanced but helpful information to
the user. For instance, consider the intermediate step “x !in [rhs]” to build
the formula “x !in Protected” from Figure 4 (c). When listing the Basic Set
blocks, we can prevent link from being selectable, as x is of type File and
link is of type File->File.

##  4\. Future Plans

To bring our structure editor development environment from a proof of-concept
to a reality, there are a few key problems to address.

Feel of Workflow. First, edit actions to compose a model need to be intuitive,
enabling the user to feel as if their construction of a model is not being
unnecessarily restricted. One issue we forsee is that for construction, our
extension points allow some flexibility, but this may not be enough. For
instance, returning to the previous example, we could let the user first build
“x !in File” in order to extend it into “x !in File.link,” which may feel more
natural than the user having to select the relational join first to create
“File.link.” Therefore, we plan to explore interface designs which allow users
to build intermediate formulas to then insert into holes. Besides
construction, we will also need to enable more edit actions, including
refactoring, replacements, and deletions.

For all edit actions, we will need to (1) ensure the edits happens without
noticeable lag and (2) conduct user studies to ensure that the workflow to
take these actions is not cumbersome. For the former, between the current
performance of Alloy’s compiler and our success at leveraging constraint
checking for several Alloy tasks such as fault localization and repair
(ARepair, ; AlloyFL, ; AUnit, ), we believe responsiveness is feasible. For
the latter, we anticipate using students as novice users and active members of
the Alloy discourse group as expert users, and designing students to explore
multiple design choices such as categorization and order of blocks, how blocks
can be added and how users select different edit actions.

Adoption. Second, while a block-based editor is likely to have a beneficial
impact for formal methods education, creating an editor that sees adoption
even among expert users is an open problem. One avenue we plan to explore is
to enable different edit modes, giving the user the option to choose how
guided the structure editor will be. We current plan to investigate two
options: (1) an active mode following the traditional structure editor
workflow and (2) an overlay mode in which the user free types and the
structure editor live compiles the user’s formulas and flags issues as they
appear. In the later, we will need to ensure we balance how often we attempt
to compile to ensure that we flag issues in a timely manner without nagging
the user as they are actively typing out an operator or set. Another avenue we
plan to explore is utilizing Alloy itself to add context to edit options. For
instance, if the user selects a refactoring edit action to swap “a in b” with
“b in a,” we can generate an instance that highlights the difference between
the two. The ability to explore the impact of an edit before selecting the
edit could motivate expert users to develop in our structure editor, as our
editor could save time spot-checking formulas.

##  5\. Related Work

Structure editors is an active research field (maloney2010scratch, ;
voelter2012language, ; teitelbaum1981cornell, ; mcnutt2023projectional, ;
omar2017hazelnut, ; asenov2017envision, ; Fructure, ), with the most popular
type being block based, in which the user authors programs by dragging and
dropping blocks together on a canvas. While block based editors have seen
success in educational settings, they are not widely used due how clunky
development can be (bahlke1992design, ; bau2017learnable, ;
holwerda2018usability, ). As a result, there is a whole body of work looking
to enable other forms of inputs mechanism largely centered around keystrokes
(kolling2010greenfoot, ; berger2016efficiency, ; cursorless, ; moon2022tylr, ;
moon2023gradual, ). These other iterations are avenues we will consider for
inputs if our user studies reveal blocks are not ideal. However, given the
rigid development options enforced by Alloy’s type and grammar system, we
believe that a block-based editor is a good starting point.

##  6\. Conclusion

A recent empirical study has revealed that users struggle from the get go to
write models: new users have a hard time producing models that successfully
compile. This paper presents the concept of a structure editor development
environment for Alloy that replaces the passive transfer of knowledge that
compilation error reports provide with an active guided development
environment.

## References

  * (1) Akhawe, D., Barth, A., Lam, P.E., Mitchell, J., Song, D.: Towards a formal foundation of web security. In: 2010 23rd IEEE Computer Security Foundations Symposium. pp. 290–304 (2010)
  * (2) Alloy4Fun Benchmark: https://zenodo.org/record/4676413 (2022)
  * (3) Almstrum, V.L., Dean, C.N., Goelman, D., Hilburn, T.B., Smith, J.: Support for teaching formal methods. In: Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education. p. 71–88. ITiCSE-WGR ’00, Association for Computing Machinery, New York, NY, USA (2001)
  * (4) Asenov, D.: Envision: reinventing the integrated development environment. Ph.D. thesis, ETH Zurich (2017)
  * (5) Askarpour, M., Bersani, M.M.: Teaching formal methods: An experience report. In: Bruel, J.M., Capozucca, A., Mazzara, M., Meyer, B., Naumchev, A., Sadovykh, A. (eds.) Frontiers in Software Engineering Education. pp. 3–18. Springer International Publishing (2020)
  * (6) Bagheri, H., Kang, E., Malek, S., Jackson, D.: A formal approach for detection of security flaws in the Android permission system. Formal Asp. Comput. (2018)
  * (7) Bahlke, R., Snelting, G.: Design and structure of a semantics-based programming environment. International Journal of Man-Machine Studies 37(4), 467–479 (1992)
  * (8) Bau, D., Gray, J., Kelleher, C., Sheldon, J., Turbak, F.: Learnable programming: blocks and beyond. Communications of the ACM 60(6), 72–80 (2017)
  * (9) Berger, T., Völter, M., Jensen, H.P., Dangprasert, T., Siegmund, J.: Efficiency of projectional editing: A controlled experiment. In: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. pp. 763–774 (2016)
  * (10) Boyatt, R., Sinclair, J.: Experiences of teaching a lightweight formal method (01 2008)
  * (11) Chong, N., Sorensen, T., Wickerson, J.: The semantics of transactions and weak memory in x86, Power, ARM, and C++. SIGPLAN Not. 53(4), 211–225 (2018)
  * (12) Cursorless GitHub: https://github.com/cursorless-dev/cursorless (2024)
  * (13) Dini, N., Yelen, C., Alrmaih, Z., Kulkarni, A., Khurshid, S.: Korat-API: A framework to enhance Korat to better support testing and reliability techniques. In: SAC (2018)
  * (14) Dúo-Terrón, P.: Analysis of scratch software in scientific production for 20 years: programming in education to develop computational thinking and steam disciplines. Education Sciences 13(4), 404 (2023)
  * (15) Fructure: A structured interaction engine in Racket GitHub: https://github.com/disconcision/fructure (2024)
  * (16) Holwerda, R., Hermans, F.: A usability analysis of blocks-based programming editors using cognitive dimensions. In: 2018 IEEE symposium on visual languages and human-centric computing (VL/HCC). pp. 217–225. IEEE (2018)
  * (17) Jovanovic, A., Sullivan, A.: Reach: Refining alloy scenarios by size (tools and artifact track). In: 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE). pp. 229–238 (2022)
  * (18) Jovanovic, A., Sullivan, A.: Empirically exploring how novices write software models in alloy. arXiv preprint arXiv:2402.06624 (2024)
  * (19) Kölling, M.: The greenfoot programming environment. ACM Transactions on Computing Education (TOCE) 10(4), 1–21 (2010)
  * (20) Krishnamurthi, S., Nelson, T.: The human in formal methods. In: Formal Methods - The Next 30 Years - Third World Congress, FM 2019, Porto, Portugal, October 7-11, 2019, Proceedings. Lecture Notes in Computer Science, vol. 11800, pp. 3–10. Springer (2019)
  * (21) Maloney, J., Resnick, M., Rusk, N., Silverman, B., Eastmond, E.: The scratch programming language and environment. ACM Transactions on Computing Education (TOCE) 10(4), 1–15 (2010)
  * (22) Mansoor, N., Bagheri, H., Kang, E., Sharif., B.: An empirical study assessing software modeling in alloy. In: International Conference on Formal Methods in Software Engineering. pp. 44–54 (2023)
  * (23) Maoz, S., Ringert, J.O., Rumpe, B.: CD2Alloy: Class diagrams analysis using Alloy revisited. In: MODELS (2011)
  * (24) Marinov, D., Khurshid, S.: TestEra: A novel framework for automated testing of Java programs. In: ASE (2001)
  * (25) McNutt, A., Chugh, R.: Projectional editors for json-based dsls. In: 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). pp. 60–70. IEEE (2023)
  * (26) Montiel, H., Gomez-Zermeño, M.G.: Educational challenges for computational thinking in k–12 education: A systematic literature review of “scratch” as an innovative programming tool. Computers 10(6), 69 (2021)
  * (27) Moon, D., Blinn, A., Omar, C.: Tylr: A tiny tile-based structure editor. In: Proceedings of the 7th ACM SIGPLAN International Workshop on Type-Driven Development. pp. 28–37 (2022)
  * (28) Moon, D., Blinn, A., Omar, C.: Gradual structure editing with obligations. In: 2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). pp. 71–81. IEEE (2023)
  * (29) Nelson, T., Danas, N., Dougherty, D.J., Krishnamurthi, S.: The power of "why" and "why not": Enriching scenario exploration with provenance. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (FSE). pp. 106–116 (2017)
  * (30) Nelson, T., Saghafi, S., Dougherty, D.J., Fisler, K., Krishnamurthi, S.: Aluminum: principled scenario exploration through minimality. In: 2013 35th International Conference on Software Engineering (ICSE). pp. 232–241. IEEE (2013)
  * (31) Nelson, T., Barratt, C., Dougherty, D.J., Fisler, K., Krishnamurthi, S.: The Margrave tool for firewall analysis. In: LISA. pp. 1–8 (2010)
  * (32) Omar, C., Voysey, I., Hilton, M., Aldrich, J., Hammer, M.A.: Hazelnut: a bidirectionally typed structure editor calculus. ACM SIGPLAN Notices 52(1), 86–99 (2017)
  * (33) Porncharoenwase, S., Nelson, T., Krishnamurthi, S.: Composat: specification-guided coverage for model finding. In: Formal Methods: 22nd International Symposium, FM 2018, Held as Part of the Federated Logic Conference, FloC 2018, Oxford, UK, July 15-17, 2018, Proceedings 22. pp. 568–587. Springer (2018)
  * (34) Ringert, J.O., Sullivan, A.: Abstract alloy instances. In: Formal Methods - 25th International Symposium, FM 2023, Lübeck, Germany, March 6-10, 2023, Proceedings. pp. 364–382 (2023)
  * (35) Samimi, H., Aung, E.D., Millstein, T.D.: Falling back on executable specifications. In: ECOOP. pp. 552–576 (2010)
  * (36) Simonot, M.: Teaching abstraction in mathematics and computer science - a computer-supported approach with alloy (01 2012)
  * (37) Su, Y.S., Shao, M., Zhao, L.: Effect of mind mapping on creative thinking of children in scratch visual programming education. Journal of Educational Computing Research 60(4), 906–929 (2022)
  * (38) Sullivan, A.: Hawkeye: User-guided enumeration of scenarios. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). pp. 569–578. IEEE (2021)
  * (39) Sullivan, A., Wang, K., Zaeem, R.N., Khurshid, S.: Automated test generation and mutation testing for Alloy. In: 2017 IEEE Conference on Software Testing, Validation and Verification (ICST). pp. 264–275 (2017)
  * (40) Teitelbaum, T., Reps, T.: The cornell program synthesizer: a syntax-directed programming environment. Communications of the ACM 24(9), 563–573 (1981)
  * (41) Trippel, C., Lustig, D., Martonosi, M.: Security verification via automatic hardware-aware exploit synthesis: The CheckMate approach. IEEE Micro (2019)
  * (42) Voelter, M., Pech, V.: Language modularity with the mps language workbench. In: 2012 34th International Conference on Software Engineering (ICSE). pp. 1449–1450. IEEE (2012)
  * (43) Wang, K., Sullivan, A., Khurshid, S.: Automated model repair for Alloy. In: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE). pp. 577–588 (2018). https://doi.org/10.1145/3238147.3238162
  * (44) Wang, K., Sullivan, A., Marinov, D., Khurshid, S.: Fault localization for declarative models in alloy. In: 31st IEEE International Symposium on Software Reliability Engineering, ISSRE 2020, Coimbra, Portugal, October 12-15, 2020. pp. 391–402 (2020)
  * (45) Wickerson, J., Batty, M., Sorensen, T., Constantinides, G.A.: Automatically comparing memory consistency models. In: POPL (2017)
  * (46) Zaeem, R.N., Khurshid, S.: Contract-based data structure repair using Alloy. In: ECOOP. pp. 577–598 (2010)
  * (47) Zave, P.: A practical comparison of alloy and spin. Formal Aspects of Computing 27(2), 239–253 (Mar 2015)
  * (48) Zave, P.: Using lightweight modeling to understand chord. SIGCOMM Comput. Commun. Rev. 42, 49–57 (2012)


# VISON: An Ontology-Based Approach for
Software Visualization Tool Discoverability

Leonel Merino1, Ekaterina Kozlova23, Oscar Nierstrasz3, Daniel Weiskopf1
1VISUS, University of Stuttgart, Germany  2National Research University Higher
School of Economics, Russia  3SCG, University of Bern, Switzerland

###### Abstract

Although many tools have been presented in the research literature of software
visualization, there is little evidence of their adoption. To choose a
suitable visualization tool, practitioners need to analyze various
characteristics of tools such as their supported software concerns and level
of maturity. Indeed, some tools can be prototypes for which the lifespan is
expected to be short, whereas others can be fairly mature products that are
maintained for a longer time. Although such characteristics are often
described in papers, we conjecture that practitioners willing to adopt
software visualizations require additional support to discover suitable
visualization tools. In this paper, we elaborate on our efforts to provide
such support. To this end, we systematically analyzed research papers in the
literature of software visualization and curated a catalog of 70 available
tools that employ various visualization techniques to support the analysis of
multiple software concerns. We further encapsulate these characteristics in an
ontology. _VISON_ , our software visualization ontology, captures these
semantics as concepts and relationships. We report on early results of usage
scenarios that demonstrate how the ontology can support  (i) developers to
find suitable tools for particular development concerns, and  (ii) researchers
who propose new software visualization tools to identify a baseline tool for a
controlled experiment.

##  I Introduction

Complex questions may arise during software development [1, 2, 3, 4]. Over the
last two decades, many software visualizations have been presented in the
research literature and shown to be suitable to address some of these
questions [5]. However, there is still little evidence of where and how
software visualizations are being discovered and adopted by practitioners. To
find a suitable tool, practitioners need to examine aspects such as the
development tasks supported by the tool, the required execution environment,
the level of maturity of the tool, and whether there is a maintenance plan for
future improvements and bug fixes. For example, practitioners can be reluctant
to adopt some prototypical visualization tools that often have a short
lifespan, and more open to adopt tools that belong to long-term projects and
are expected to be maintained for a fairly long time.

Figure 1: A Sankey diagram that presents our curated catalog of 70 available
visualization tools introduced in the literature of software visualization,
classified by publication year, software aspects, evaluation strategy, and
tool name. Figure 2: An overview of the concept hierarchy of the _VISON_
software visualization ontology using the OntoGraf visualization plug-in. Blue
edges denote subclass relationships and violet edges identify instances of a
class.

To address the gap between existing software visualizations and their
practical applications, we build on previous studies [6, 7] in which we
reviewed the literature of software visualization to collect their
characteristics. This article is based on results that were reported in the
doctoral thesis of the first author [8]. We updated the review and curated a
catalog of 70 publicly available software visualization tools. For each tool
in the catalog, we identify  (i) the tool’s name (_e.g._ , Jive),  (ii)
software aspects (_e.g._ , behavior, structure, evolution),  (iii) software
concerns (_e.g._ , execution traces of Java programs),  (iv) last update
(_e.g._ , 2017),  (v) execution environment (_e.g._ , Eclipse plug-in),  (vi)
employed visualization techniques (_e.g._ , node-link diagram),  (vii) display
medium (_e.g._ , standard computer screen (SCS), immersive virtual reality
(I3D)), and  (viii) evaluations (_e.g._ , controlled experiment).  Figure 1
shows a Sankey diagram of our catalog of visualization tools introduced in the
literature of software visualization. The ontology, which enables both textual
and visual search methods, then can be used by practitioners to find suitable
software visualizations as well as by researchers who can reflect on the
software visualization domain. The ontology can also enable higher-level
frameworks to support practitioners to search for software visualization
tools.

_Why a software visualization ontology?_ Ontologies are formal and explicit
descriptions of concepts in a domain [9]. Ontologies can help  (i) share a
common understanding of the structure of information among people or software
agents,  (ii) reuse domain knowledge,  (iii) enforce domain assumptions,  (iv)
separate domain knowledge from operational knowledge, and  (v) analyze domain
knowledge.

Figure 2 shows an overview of _VISON_ , our software visualization ontology
that encapsulates main characteristics of software visualizations. To the best
of our knowledge, VISON is the first ontology of software visualizations. We
elaborate on lessons learned from developing the ontology, and early results
of usability through usage scenarios.

To populate VISON, we built on a set of selected papers of previous surveys of
the software visualization literature [6, 7]. Specifically, we scanned each
classified design study paper to identify software visualization tools. For
each tool that we found, we checked whether the tool is publicly available on
the internet. In the end, we curated a catalog of 70 publicly available
software visualization tools that we used to populate our ontology.

The main contribution of this paper is twofold:  (i) a curated catalog of 70
available software visualization tools and,  (ii) a publicly available [10]
software visualization ontology.

The remainder of the paper is structured as follows: Section II describes
related work that focuses on practical applications of software visualizations
and catalogs of software visualization tools. Section III elaborates on the
main concepts of ontologies that are addressed in our study. Section IV
presents VISON, our software visualization ontology. We first elaborate on a
catalog of 70 available software visualization tools, and then we discuss
ontology implementation details. Section VI concludes and presents future
work.

##  II Related Work

We group related work into two main themes. We first discuss research that
proposes approaches to practical applications of software visualization tools.
Then, we elaborate on studies that present catalogs of software visualization
tools.

###  II-A Practical Applications of Software Visualizations

We observe that there are only a few studies that have been carried out to
fill the gap between existing software visualizations and their practical
applications. For instance, Hassaine _et al._ [11] elaborate on an approach
for generating visualizations to support software maintenance tasks. Sfayhi
and Sahraoui [12] describe how to generate interactive visualizations based on
descriptions of code analysis tasks. To this end, developers are required to
describe the task using a domain-specific language. Grammel _et al._ [13]
investigate how novices construct information visualizations. Based on the
analysis of the usage of simple visualizations such as charts and scatter
plots, they identify a user’s need for information visualization tools.
However, we observe that these visualizations provide limited support for the
analysis of development concerns. Three other studies [14, 15, 16] investigate
software development tasks for which visualization tools have been proposed,
however, we consider that the tasks in these studies are at a too high-level
for developers to find an appropriate visualization to their particular needs.
Merino _et al._ [17] introduce a meta-visualization approach of live
visualization example objects annotated with the type of development questions
that they can help investigate. In the visualization, developers can identify
suitable visualization examples by detecting the surrounding keywords in the
tag-iconic cloud-based visualization. Instead, we propose the use of an
ontology that can encapsulate the semantics of the characteristics of software
visualizations. As opposed to the described studies, our ontology-based
approach leverages existing software visualization tools by attempting to
provide practitioners a means for discovery.

###  II-B Catalogs of Software Visualization Tools

Some studies examine software visualization tools, in particular, to create
guidelines for designing and evaluating software visualizations. For example,
Storey  _et al._ [18] examine 12 software visualization tools and propose a
framework to evaluate software visualizations based on intent, information,
presentation, interaction, and effectiveness. Sensalire  _et al._ [19, 20]
classify the features users require in software visualization tools. To this
end, they elaborate on lessons learned from evaluating 20 software
visualization tools and identify dimensions that can help design an evaluation
and then analyze the results. In our investigation, we do not attempt to
provide a comprehensive catalog of software visualization tools, but we seek
to provide a means to boost software visualization discoverability.

Some other studies present taxonomies that characterize software visualization
tools. Myers [21] classifies software visualization tools based on whether
they focus on code, data, or algorithms; and whether they are implemented in a
static or dynamic fashion. Price _et al._ [22] present a taxonomy of software
visualization tools based on six dimensions: scope, content, form, method,
interaction, and effectiveness. Maletic _et al._ [23] propose a taxonomy of
five dimensions to classify software visualization tools: tasks, audience,
target, representation, and medium. Schots _et al._ [24] extend this taxonomy
by adding two dimensions: resource requirements of visualizations, and
evidence of their utility. Merino _et al._ [6] add _needs_ as a main
characteristic of software visualization tools. In their context, “needs”
refers to the set of questions that are supported by software visualization
tools. Although we consider these studies crucial for reflecting on the
software visualization domain, we think that practitioners may require a more
comprehensive support to identify a suitable tool. In particular, we believe
that the semantics of concepts and their relationships are often missing in
taxonomies and other classifications. The use of an ontology enforces the
analysis of these relationships, which can play an important role in
identifying a suitable visualization tools.

##  III Ontology Design Considerations

An ontology is a formalization of a model to describe what is essential in a
_domain_. That is, the ontology describes the _concepts_ in the domain that
can define various _properties_ and _restrictions_. Hence, an ontology
populated with a set of individual _instances_ of the concepts is usually
referred to as a knowledge base. However, defining what in the domain is
modeled as a concept or an instance is subjective. We opted to follow the
widely used guidelines proposed by Noy and McGuiness [25]. We now elaborate on
how we addressed their suggested steps to create our _software visualization
ontology_.

_Step 1. Determine the domain and scope of the ontology._

  * •

_What is the domain that the ontology will cover?_ Software visualizations.

  * •

_For what we are going to use the ontology?_ To allow 1) developers to find
suitable visualizations for their particular concerns and 2) researchers to
reflect on the software visualization domain.

  * •

_For what types of questions the information in the ontology should provide
answers?_ Questions that identify particular software visualizations that
fulfill the restrictions imposed by the context of the developers needs.

  * •

_Who will use and maintain the ontology?_ Software developers willing to adopt
visualizations, and who have used a visualization from the ontology and want
to add new supported questions to it. Also, researchers who want to add new
data to the ontology for a new or an existing indexed visualization approach.

_Step 2. Consider reusing existing ontologies._ To the best of our knowledge,
this is the first ontology of software visualizations.

_Step 3. Enumerate important terms in the ontology._ We include the
characteristics of software visualization and their evaluations, as well as
the classifications presented in previous studies [6, 7].

_Step 4. Define the concepts and the concept hierarchy._ We opt for a bottom-
up development process in which we start from instances of proposed software
visualizations. For each, we identify the various concepts involved in its
context (_e.g._ , tasks, media, environments, frameworks, questions,
evaluation strategies). We define a hierarchy of concepts following an “is-a”
relation. When defining the concepts, we avoid creating cycles and validate
that sibling concepts (_i.e._ , at the same level in the hierarchy) correspond
to the same level of generality.

_Step 5. Define the properties of concepts._ We characterize the concepts
based on their properties. For instance, for the concept _medium_ we define
the _dimensionality_ (_e.g._ , 2D/3D) property. Then, when we define
particular software visualizations as instances in the ontology, we can
specify a medium and its dimensionality. Thus, researchers can use the
ontology to investigate, for instance, the correlation between evaluation
strategies and visualizations that use visualization techniques of a higher
dimensionality displayed on a medium of a lower dimensionality.

_Step 6. Define the restrictions of the properties._ We only use restrictions
to define disjoint concepts.

_Step 7. Create instances._ We create instances in the ontology for each
proposed software visualization in our data set. Thus, visualization tools are
the materialization of a combination of property values of concepts.

##  IV VISON: Software Visualization Ontology

Certainly, an empty ontology that describes concepts and relationships but has
no instances cannot be useful for practitioners. Therefore, before we describe
an implementation of our ontology, we elaborate on the systematic approach
that we used to populate it. In the following, we describe the process
followed to collect a set of relevant software visualization tools and their
characteristics from the research literature.

###  IV-A Software Visualization Tools

We built on the data sets from the proposed software visualization approaches
(presented in previous studies [6, 7]). We reviewed the 387 software
visualization papers published in the VISSOFT/SOFTVIS conferences. Since the
goal of our investigation is to facilitate the discoverability of software
visualization tools, we included in our catalog only software visualization
tools that are:  (C1) identified with a name and  (C2) publicly available on
the internet.

We scanned each paper to identify a name for the proposed software
visualization approach. Then, we looked for a URL where the tool might be
available. In most cases (where we did not find a URL in the paper), we
searched the Web using the name of the tool (C1). When we did not find a
positive result, we added “visualization” to the search keywords. When we
found an available tool (C2), we checked the last time when the tool was
updated. Sometimes, we had to download the tool to look for the date in the
files. In the end, we found 70 software visualization tools that fulfill the
criteria and that we therefore included in our catalog.

To characterize a tool we first identified its name and whether it focuses on
the structure, behavior, or evolution of software systems [26]. Then, for the
tools in each category, we identified the development concern expressed by the
visualization. Instead of describing high-level tasks (_e.g._ , reverse-
engineering), we formulated descriptions with the main keywords of the
concerns (_e.g._ , “reports that summarize methods execution”), which we think
can help developers relate their particular context to the one envisioned by a
proposed visualization tool. We also classified the tools based on their
execution environment, the employed visualization technique, and the medium
used to display them. Finally, we reused the data presented in our previous
study [7] to highlight the maturity of tools that have proven effective to
support the target task through evaluations.

Table I: A curated catalog of 70 available software visualization tools. Tools are grouped by aspects: Behavior, Structure, Evolution, and E.-S.-B. (their combination). Asp. |  Tool’s Name |  Year | Software Concern |  Environment |  Technique |  Med. |  Evaluation
---|---|---|---|---|---|---|---
Behavior |  Clack |  2018 | Concepts for teaching networks in CS |  Java |  Node-link |  SCS |  Anecdotal
ToonTalk |  2018 | Concepts for teaching children to program |  Web |  Visual language |  SCS |  N/A
LTSView |  2017 | Transition systems |  Various |  3D node-link |  SCS |  N/A
Gzoltar |  2017 | Fault localization for debugging Java progs. |  Java;Ecli. |  Icicle; treemap |  SCS |  Experiment
SIFEI |  2017 | Spreadsheets formulas for testing |  Excel |  Visual language |  SCS |  Experiment
SwarmDebugging |  2017 | Reuse knowledge of debugging sessions |  Eclipse |  Node-link |  SCS |  Usage Scen.
MethodExecutionReports |  2017 | Summarization of methods execution |  Java |  Charts |  SCS |  Experiment
Jive |  2016 | Execution traces of Java programs |  Eclipse |  Node-L; aug.src. |  SCS |  Anecdotal
Cerebro |  2016 | Execution traces for feature identification |  Web |  Node-link |  SCS |  Usage Scen.
Jsvee; Kelmu |  2016 | Concepts for teaching programming in CS |  Web |  Aug. source code |  SCS |  Usage Scen.
Kayrebt |  2015 | Control and data flow of the Linux kernel |  Linux |  Node-link |  SCS |  Usage Scen.
TiledGrace |  2015 | Programming in the Grace language |  Web |  Visual language |  SCS |  Experiment
jGrasp |  2015 | Concepts for teaching programming in CS |  Various |  Aug. source code |  SCS |  Exp.; Survey
xViZiT |  2015 | Spreadsheets formulas for testing |  Java |  Aug. source code |  SCS |  Usage Scen.
Beat |  2014 | Execution traces of Java concurrent prog. |  Eclipse |  Aug. source code |  SCS |  N/A
regVIS |  2014 | Assembler control-flow of regular expr. |  Windows |  Visual language |  SCS |  Experiment
GraphWorks |  2013 | Concepts for teaching graph theory in CS |  Java |  Anim. node-link |  SCS |  N/A
Synchrovis |  2013 | Execution traces of Java concurrent prog. |  Java |  City |  SCS |  Usage Scen.
PlanAni |  2011 | Concepts for teaching programming in CS |  Various |  Aug. source code |  SCS |  Experiment
ProfVis |  2011 | Execution traces of Java programs |  Java |  Node-link |  SCS |  Experiment
GEM |  2011 | Dynamic verification of MPI programs |  Eclipse |  Aug. source code |  SCS |  N/A
Dyvise |  2009 | Java heap to detect memory problems |  Java |  Icicle |  SCS |  Anecdotal
Jive |  2007 | Execution traces of Java programs |  Java |  Charts |  SCS |  Usage Scen.
Jove |  2007 | Execution traces of Java programs |  Java |  Charts |  SCS |  N/A
VeldVisualizer |  2007 | Execution traces of Java programs |  Java |  Pixel |  SCS |  N/A
TraceVis |  2007 | Execution traces based on call graphs |  Java |  Node-link |  SCS |  Usage Scen.
ALVIS |  2006 | Concepts for teaching programming in CS |  Windows |  Visual language |  SCS |  N/A
Evolve |  2003 | Execution traces of Java programs |  Java |  Pixel |  SCS |  Usage Scen.
Structure |  PhysVis |  2018 | Software quality based on metric analysis |  VisualStudio |  3D node-link |  I3D |  Usage Scen.
SpartanRefactoring |  2018 | Automatic code refactoring for readability |  Eclipse |  Aug. source code |  SCS |  N/A
Softwarenaut |  2017 | Architecture and dependency analysis |  VisualWorks |  Node-L.; treemap |  SCS |  N/A
CodeMetropolis |  2017 | Software quality based on metric analysis |  Java |  City |  SCS |  N/A
Explen |  2017 | Slice-based techs. for large metamodels |  Eclipse |  UML |  SCS |  N/A
iTraceVis |  2017 | Eye movement data of code reading |  Eclipse |  Heatmap |  SCS |  Experiment
CityVR |  2017 | Architecture based on metrics in OOP |  Pharo; U. |  City |  I3D |  Experiment
ExplorViz |  2016 | Architecture based on metric analysis |  Web |  City |  S/I |  Experiment
MetaVis |  2016 | Annotated visualization example objects |  Pharo |  Node-L.; tag cloud |  SCS |  Usage Scen.
CodeCity |  2015 | Software quality based on code smells |  Pharo |  City |  SCS |  Usage Scen.
Explora |  2015 | Software quality based on metric analysis |  Pharo |  Polymetric views |  SCS |  Usage Scen.
OrionPlanning |  2015 | Arch. modularization and consistency |  Pharo |  Node-link |  SCS |  Usage Scen.
VariabilityBlueprint |  2015 | Decomposition of models in FOP |  Pharo |  Polymetric views |  SCS |  Usage Scen.
StenchBlossom |  2014 | Software quality based on code smells |  Eclipse |  Aug. source code |  SCS |  Experiment
Visuocode |  2014 | Navigation and composition of systems |  Mac |  Aug. source code |  SCS |  N/A
SolidSDD |  2014 | Software quality based on code clones |  Windows |  HEB |  SCS |  Usage Scen.
SolidFX |  2013 | Architecture, metric and dependencies |  Windows |  HEB; pixel |  SCS |  Experiment
SeeIT3D |  2013 | Software architecture of Java systems |  Eclipse |  City |  SCS |  Experiment
AspectMaps |  2013 | Architecture of aspect-oriented programs |  Pharo |  Iconic; pixel |  SCS |  Experiment
VisMOOS |  2010 | Software architecture of Java systems |  Eclipse |  Node-link |  SCS |  N/A
Rigi |  2009 | Architecture and dependency analysis |  Various |  Node-link |  SCS |  N/A
Barrio |  2009 | Architecture and dependency analysis |  Eclipse |  Node-link |  SCS |  Usage Scen.
Evolution |  FlaskDashboard |  2017 | Flask Python Web services performance |  Python |  Charts; heatmap |  SCS |  Usage Scen.
ObjectEvolutionBlueprint |  2016 | Object mutations |  Pharo |  Charts |  SCS |  Experiment
TypeV |  2016 | Abstract syntax trees of a system’s project |  Web |  Charts |  SCS |  Usage Scen.
SHriMP |  2015 | Hierarchical structures in OOP |  Eclipse |  Node-link |  SCS |  N/A
AGG |  2013 | Hierarchical structures in OOP |  Java |  Node-link |  SCS |  N/A
DEVis |  2013 | Technical documents |  Eclipse |  Spiral |  SCS |  Theoretical
ClonEvol |  2013 | Software quality based on code clones |  Windows |  HEB |  SCS |  Usage Scen.
SoftwareEvolutionStorylines |  2010 | Developers interactions in projects |  Processing |  StoryLines; charts |  SCS |  N/A
CVSgrab |  2009 | Interactions during debugging |  Windows |  Pixel |  SCS |  N/A
VisualCodeNavigator |  2007 | Source code changes |  Windows |  Aug. src.; pixel |  SCS |  Usage Scen.
CVSscan |  2007 | Source code changes |  Windows |  Pixel |  SCS |  Case Study
MetricView |  2006 | Hierarchical structures and metrics in OOP |  Windows |  3D UML |  SCS |  N/A
E.-S.-B. |  Mondrian |  2018 | Execution traces of feature dependencies |  Pharo |  Polymetric views |  SCS |  N/A
Getaviz |  2018 | Developing and evaluating software vis. |  Web |  City |  S/I |  N/A
CodeBubbles |  2018 | Debugging within CodeBubbles |  Ecli.; VS |  Visual language |  SCS |  N/A
CHIVE |  2015 | Feature location (reconnaissance) |  Eclipse |  3D node-link |  SCS |  N/A
Graph |  2015 | Code dependencies |  Pharo |  Node-link |  SCS |  Usage Scen.
SpiderSense |  2015 | Execution traces of Java programs |  Web |  Pixel; treemap |  SCS |  Usage Scen.
Vizz3D |  2013 | Software architecture and quality |  Java |  3D node-link |  SCS |  N/A
GEF3D |  2010 | Execution traces of Java programs |  Eclipse |  3D UML |  SCS |  Usage Scen.

Table I presents our curated catalog of 70 available software visualization
tools classified by the software’s _date_ of last update, _environment_
required to execute, employed visualization _technique_ , _medium_ , and
evidence of the visualization’s effectiveness through _evaluations_. Each
tool’s name is linked to a URL that contains instructions for downloading and
installation. Visualization tools are classified into software _aspects_ :
behavior, evolution, and structure.

####  IV-A1 Behavior

Several visualization tools support teaching various subjects in computer
science. _ToonTalk_ [27] comes with a visual language (similar to Scratch
[28]) that is to be used on the Web. The tool targets children as an audience.
We are not aware of any evaluation of ToonTalk. However, the tool has been
maintained over the last twelve years, which shows evidence of maturity.
Similarly, _Tiled Grace_ [29] offers a visual representation alternative to
the textual mode when programming in the Grace language. Another mature tool
is _Clack_ [30], which helps students of network courses understand the
behavior of routers. _GraphWorks_ [31] focuses on supporting students of graph
theory, although it has not been maintained in the last few years.

Figure 3: The _Jive_ visualization tool to support the analysis of behavior of
concurrent Java applications. Figure taken from the Web [32], and reused with
permission © 2005 Jayaraman.

Some other tools are available to deal with understanding the execution of
programs for testing. The Eclipse plug-in _Jive_ [33] (shown in Figure 3)
stands out since it has been maintained for the last eleven years, which is
congruent with anecdotal evidence of its adoption. Even though all of these
tools are available, almost none of them have been maintained lately. Amongst
them, _ProfVis_ [34] is the only one that has proven effective in an
experiment. A few others— _Jove_ [35] and _Veld Visualizer_ [36]—have been
presented only through usage scenarios. Other tools that, to our knowledge,
have not been evaluated are _Jive_ [37], _TraceVis_ [38], and _Evolve_ [39].
Two tools— _Beat_ [40] and _Synchrovis_ [41]—target the analysis of the
behavior of concurrent Java programs, whereas the tool _Cerebro_ [42] can be
used to identify software features from the runtime data.

Three visualization tools support debugging tasks based on the visualization
of program behavior. _Dyvise_ [43] supports the detection of memory problems
through the visualization of the Java heap. _GEM_ [44] (shown in Figure 4) is
a graphical explorer of MPI programs. _Gzoltar_ [45] has shown evidence of
effectiveness through an experiment. More recently, _SwarmDebugging_ [46] is
an Eclipse plug-in that aims to reuse the knowledge of previous debugging
sessions to recommend locations in the code to define breakpoints.

Four visualization tools support various aspects of teaching computer science
courses. _PlanAni_ [47] is a program animation system based on the concept of
the roles of variables for teaching programming. _ALVIS_ [48] enables
algorithm visualization to learning programming. _jGrasp_ [49] is an
integrated development environment with visualizations for teaching Java.
_Jsvee_ and _Kelmu_ [50] are visualization libraries to help instructors teach
aspects of the runtime of a program.

Other visualization tools deal with various particular concerns. _LTSView_
[51] is the oldest one, and it is still being maintained. It supports the
visualization of transition systems that model the behavior of a software.
_SIFEI_ [52] and _xViZiT_ [53] focus on the visualization of spreadsheets,
whereas _regVis_ [54] deals with the visualization of assembler control-flow
based on regular expressions. _Method Execution Reports_ [55] embed word-size
graphics in reports of method executions. _Kayrebt_ [56] provides support for
activity diagram extraction and a visualization toolset designed for the Linux
codebase.

All the twenty-eight listed tools that focus on the behavior of software
systems are displayed on the standard computer screen.

Figure 4: The _GEM_ graphical explorer of MPI programs. © 2010 IEEE.
Reprinted, with permission, from Humprey _et al._ [44].

####  IV-A2 Structure

Various other visualization tools focus on particular concerns. _MetaVis_ [17]
can be used to visualize annotated software visualization example objects.
_OrionPlanning_ [57] includes visualization for modularization and consistency
of software projects. _Explen_ [58] supports the visualization of large
metamodels. _iTraceVis_ [59] has shown evidence of being effective to
investigate how developers read code through the visualization of their eye
gazes. _Spartan Refactoring_ allows automatic code refactoring in the editor.
_Visuocode_ [60] supports the navigation and composition of software systems.

Some visualization tools are available for supporting architecture tasks such
as _SeeIT3D_ [61] and _VisMOOS_ [62]. _SolidFX_ [63], _Softwarenaut_ [64]
(shown in Figure 5), _Rigi_ [65], and _Barrio_ [66] are suitable for the
analysis of structures and dependencies in object-oriented software systems,
_AspectMaps_ [67] supports aspect-oriented programs, and _Variability
blueprint_ [68] does so for feature-oriented programs.

Three tools support the visualization of software systems quality based on the
analysis of code smells. _CodeCity_ [69] (shown in Figure 6) and
_CodeMetropolis_ [70] visualize software metrics based on the city metaphor.
_StenchBlossom_ [71] augments the Eclipse source code editor with ambient
visualizations. _SolidSDD_ [72] supports visual clone analysis. _Explora_ [73]
is a visualization tool for software quality based on the analysis of metrics.

Figure 5: The _Softwarenaut_ tool for visualization of hierarchical structures
to support architecture tasks. Figure taken from the Web [74], and reused with
permission © 2006 Lungu. Figure 6: The _CodeCity_ tool, which visualizes the
structure of software systems to support the analysis of code smells. Figure
taken from the Web [75], and reused with permission © Wettel.

Twenty of the listed tools that focus on the structure of software systems are
displayed using the standard computer screen. Only three use immersive virtual
reality: _PhysVis_ [76], in which users visualize software metrics shown as a
physical particle system, _ExplorViz_ [77], in which developers obtain an
overview of the architecture of a system represented as a city, and _CityVR_
[78], which adds interactions and visualization of software metrics and
smells.

####  IV-A3 Evolution

A few tools support the visualization of the evolution of hierarchical
structures in object-oriented programs such as _AGG_ [79]. _SHriMP_ [80] is
the oldest one and has been maintained for twelve years. _MetricView_ [81]
presents a UML class diagram in 3D that is augmented with software metrics.
Others deal with various concerns. _CVSgrab_ [82] supports the visualization
of the evolution of interactions of developers during debugging, whereas
_Visual Code Navigator_ [83] and _CVSscan_ [84] (shown in Figure 7) focus on
source code changes. _DEVis_ [85] is used to visualize the evolution of
technical documents. _Object Evolution Blueprint_ [86] deals with the
evolution of object mutations. _Flask dashboard_ [87] supports the
visualization of the performance evolution over versions of Web services
implemented using the Flask framework for Python. _TypeV_ [88] allows one to
analyze the evolution of a system through the visualization of abstract syntax
trees. _ClonEvol_ [89] (shown in Figure 8) visualizes the evolution of code
clones to improve the quality of systems. _Software Evolution Storylines_ [90]
supports the visualization of the interactions between developers during
software project evolution.

Figure 7: The _CVSscan_ visualization tool to support the analysis of
evolution for software maintenance. Figure taken from the Web [91], and reused
with permission © 2005 Telea. Figure 8: The _ClonEvol_ visualization tool,
which helps developers analyze the evolution of code clones. © 2013 IEEE.
Reprinted, with permission, from Hanjalić [89].

All the twelve listed tools that focus on the evolution of software systems
are displayed on the standard computer screen.

####  IV-A4 Behavior/Evolution/Structure

Eight approaches correspond to frameworks that can be used to visualize
multiple aspects of software systems. Four of them correspond to active
projects introduced several years ago. _Mondrian_ [92] is an engine for rapid
lightweight visualization, which is currently supported in the Roassal engine
[93]. _CodeBubbles_ [94] is an environment that encapsulates code snippets
into bubbles that can be reused through composition. _Vizz3D_ [95] is a
framework for online configuration of 3D information visualizations that was
originally available for Eclipse, and later made available for Visual Studio.
_CHIVE_ [96] is a framework for developing, in particular, 3D software
visualizations.

_GEF3D_ [97] is a framework for developing 2D/2.5D/3D graphical editors.
_Graph_ [98] is a domain-specific language for visualizing software
dependencies as a graph (shown in Figure 9). _Getaviz_ [99] and _SpiderSense_
[100] enable the design, implementation, and evaluation of software
visualizations.

The framework _Getaviz_ supports visualizations displayed in immersive virtual
reality, whereas the seven other frameworks are limited to the standard
computer screen.

Figure 9: The _Graph_ domain-specific language for agile prototyping of
visualization of graph structures. Figure taken from the Web [101], and reused
with permission © 2014 Bergel. Figure 10: The _classes_ view in Protégé
showing the hierarchy of concepts. We selected the name of the tools, which
are listed in the right pane.

The characterization presented in Table I contains only part of the content of
our data set described in previous publications [6, 7]. Various other
characteristics of software visualizations can help developers willing to
adopt visualization to find a suitable approach. We believe that an ontology
can be suitable to implement such richer model.

###  IV-B Implementation Details

We implement our ontology using _Protégé_ [102], a popular, free, and open-
source framework for the design and use of ontologies. In it, we define the
concepts (in the tool called _classes_), properties, restrictions, and
instances. Figure 10 shows the _classes_ view in Protégé with a detail of the
hierarchy of concepts. As the concept, we selected the name of the tools,
which are listed in the right pane.

Figure 2 shows an overview of our implementation of the concepts hierarchy
using the _OntoGraf_ visualization plug-in included in Protégé.

Table II: Metrics of the software visualization ontology. Property | Metric | Value
---|---|---
Metrics | Axiom | 3,290
Logical axiom count | 2,428
Declaration axiom count | 862
Class count | 150
Individual property count | 20
Class axioms | SubClassOf | 143
DisjointClasses | 32
Object property axioms | SubObjectPropertyOf | 1
ObjectPropertyDomain | 2
ObjectPropertyRange | 3
Individual axioms | ClassAssertion | 696
ObjectPropertyAssertion | 1,547
NegativeObjectPropertyAssertion | 4

We present the list of metrics available in the _Ontology metrics_ view of
Protégé in Table II. Although we are aware that many more individuals and
relationships must be added to the ontology to increase its usability, we
observe that our current implementation is not small according to a survey of
ontology metrics [103] that reported that ontologies on average contain 36.11
classes (standard deviation of 78.53) and 28.13 instances (standard deviation
of 97.59).

##  V Usage Scenarios

We now demonstrate the ontology through two usage scenarios.

_Scenario 1. Find suitable visualization tools that support the analysis of
performance issues at runtime._

Two concepts are defined in the specification of this need:  (1) the source of
the data is the _runtime_ and  (2) the problem dealt is the _performance_ of
the software system.  We translate this specification to the syntax specified
by the Ontology Web Language (OWL). Figure 11 shows the resulting query and
the suitable tools returned.

Figure 11: Scenario 1: Finding suitable visualization tools that support the
analysis of _performance_ issues at _runtime_.

_Scenario 2. Find visualization tools under a _free_ license that support the
analysis of source code._

Similarly, the specification of this need defines two concepts:  (1) the
license of the tool has to be _free_ and  (2) the source of the data must be
the _source code_ of the software system.  Figure 12 shows the translated
specification of the need in the OWL syntax and the suitable tools returned.

Figure 12: Scenario 2: Finding suitable free visualization tools that support
the visualization of source code.

### Threats to Validity

In our paper selection process, we might have overlooked papers from relevant
venues that describe important software visualization tools. We mitigated this
bias by selecting papers published in the two most frequently cited venues
dedicated to software visualization: SOFTVIS and VISSOFT. We selected software
visualization papers published between 2002 to 2018 in SOFTVIS and VISSOFT.
The excluded papers from other venues or published before 2002 may affect the
generalizability of our results. We mitigated bias in the data collection
procedure (which could obstruct reproducibility of our investigation) by
establishing a protocol to extract the data of each paper equally, and by
maintaining a spreadsheet to keep records, normalize terms, and identify
anomalies.

##  VI Conclusion

Although many software visualization approaches have been proposed to deal
with various software concerns, usually developers are not aware of tools they
can put into action. In this paper, we presented our attempts to fill the gap
between existing software visualizations and their practical applications:
(1) We presented a curated catalog of 70 available software visualization
tools that we linked to their repositories; we classified the tools into
various categories (_e.g._ , task, data, environment) to help developers who
look for suitable visualizations.  (2) We summarized our results in developing
VISON, our software visualization ontology.

The ontology offers a rich model to encapsulate the various characteristics of
software visualizations. We reported on our experience designing and
implementing our ontology of software visualizations in the Protégé tool. We
demonstrated how the ontology can be used through usage scenarios. Our
ontology is publicly available [10]. We expect the ontology will help
developers find suitable software visualizations and researchers to reflect on
the field. Users of the ontology will be able to contribute, for instance, by
adding characteristics of new visualizations, or by adding the results of
evaluations of existing visualizations. In the future, we plan to combine our
previous work on meta-visualization [17] with VISON.

## Acknowledgments

Merino and Weiskopf acknowledge funding by the Deutsche Forschungsgemeinschaft
(DFG, German Research Foundation) – Project-ID 251654672 – TRR 161. Nierstrasz
thanks the Swiss National Science Foundation for its financial support of
“Agile Software Assistance” (project 181973). The authors thank Craig Anslow
and Mircea Lungu for valuable comments on a previous version of the paper.

## References

  * [1] J. Sillito, G. C. Murphy, and K. De Volder, “Questions programmers ask during software evolution tasks,” in _Proceedings of FSE_. ACM, 2006, pp. 23–34.
  * [2] A. J. Ko, R. DeLine, and G. Venolia, “Information needs in collocated software development teams,” in _Proceedings of ICSE_. IEEE Computer Society, 2007, pp. 344–353.
  * [3] T. Fritz and G. C. Murphy, “Using information fragments to answer the questions developers ask,” in _Proceedings of ICSE_. ACM, 2010, pp. 175–184.
  * [4] T. D. LaToza and B. A. Myers, “Hard-to-answer questions about code,” in _Proceeedings of PLATEAU_. ACM, 2010, pp. 8:1–8:6.
  * [5] L. Merino, M. Ghafari, and O. Nierstrasz, “Towards actionable visualisation in software development,” in _Proceedings of VISSOFT_. IEEE, 2016.
  * [6] ——, “Towards actionable visualization for software developers,” _Journal of Software: Evolution and Process_ , vol. 30, no. 2, p. e1923, 2017\.
  * [7] L. Merino, M. Ghafari, C. Anslow, and O. Nierstrasz, “A systematic literature review of software visualization evaluation,” _Journal of Systems and Software_ , vol. 144, pp. 165–180, 2018.
  * [8] L. Merino, A. Bergel, and O. Nierstrasz, “Overcoming issues of 3D software visualization through immersive augmented reality,” in _Proceedings of VISSOFT_. IEEE, 2018, pp. 54–64.
  * [9] T. R. Gruber, “Toward principles for the design of ontologies used for knowledge sharing?” _International Journal of Human-Computer Studies_ , vol. 43, no. 5-6, pp. 907–928, 1995.
  * [10] L. Merino, E. Kozlova, O. Nierstrasz, and D. Weiskopf, “Artifact: VISON: An Ontology-Based Approach for Software Visualization Tool Discoverability,” Jul. 2019. [Online]. Available: https://doi.org/10.5281/zenodo.3268626
  * [11] S. Hassaine, K. Dhambri, H. Sahraoui, and P. Poulin, “Generating visualization-based analysis scenarios from maintenance task descriptions,” in _Proceedings of VISSOFT_. IEEE, 2009, pp. 41–44.
  * [12] A. Sfayhi and H. Sahraoui, “What you see is what you asked for: An effort-based transformation of code analysis tasks into interactive visualization scenarios,” in _Proceedings of SCAM_. IEEE, 2011, pp. 195–203.
  * [13] L. Grammel, M. Tory, and M.-A. Storey, “How information visualization novices construct visualizations,” _Transactions on Visualization and Computer Graphics_ , vol. 16, no. 6, pp. 943–952, 2010.
  * [14] K. Gallagher, A. Hatch, and M. Munro, “A framework for software architecture visualization assessment,” in _Proceedings of VISSOFT_. IEEE Computer Society, 2005, pp. 76–81.
  * [15] J. Paredes, C. Anslow, and F. Maurer, “Information visualization for agile software development,” in _Proceedings of VISSOFT_. IEEE, 2014, pp. 157–166.
  * [16] M. Shahin, P. Liang, and M. A. Babar, “A systematic review of software architecture visualization techniques,” _Journal of Systems and Software_ , vol. 94, pp. 161–185, 2014.
  * [17] L. Merino, M. Ghafari, O. Nierstrasz, A. Bergel, and J. Kubelka, “MetaVis: Exploring actionable visualization,” in _Proceedings of VISSOFT_. IEEE, 2016, pp. 151–155.
  * [18] M.-A. D. Storey, D. Čubranić, and D. M. German, “On the use of visualization to support awareness of human activities in software development: a survey and a framework,” in _Proceedings of SOFTVIS_. ACM, 2005, pp. 193–202.
  * [19] M. Sensalire, P. Ogao, and A. Telea, “Classifying desirable features of software visualization tools for corrective maintenance,” in _Proceedings of SOFTVIS_. ACM, 2008, pp. 87–90.
  * [20] ——, “Evaluation of software visualization tools: Lessons learned,” in _Proceedings of VISSOFT_. IEEE, 2009, pp. 19–26.
  * [21] B. Myers, D. Giuse, R. Dannenberg, B. Vander Zanden, D. Kosbie, E. Pervin, A. Mickish, and P. Marchal, “Garnet: Comprehensive support for graphical highly-interactive user interfaces,” _IEEE Computer_ , vol. 23, no. 11, pp. 71–85, 1990.
  * [22] B. A. Price, R. M. Baecker, and I. S. Small, “A principled taxonomy of software visualization,” _Journal of Visual Languages and Computing_ , vol. 4, no. 3, pp. 211–266, 1993.
  * [23] J. I. Maletic, A. Marcus, and M. Collard, “A task oriented view of software visualization,” in _Proceedings of VISSOFT_. IEEE, 2002, pp. 32–40.
  * [24] M. Schots and C. Werner, “Using a task-oriented framework to characterize visualization approaches,” in _Proceedings of VISSOFT_. IEEE, 2014, pp. 70–74.
  * [25] N. F. Noy, D. L. McGuinness _et al._ , “Ontology development 101: A guide to creating your first ontology,” 2001, Stanford Knowledge Systems Laboratory Technical Report KSL-01-05 and Stanford Medical Informatics Technical Report SMI-2001-0880.
  * [26] S. Diehl, _Software Visualization_. Berlin Heidelberg: Springer-Verlag, 2007.
  * [27] K. Kahn, “Time travelling animated program executions,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 185–186.
  * [28] J. Maloney, L. Burd, Y. Kafai, N. Rusk, B. Silverman, and M. Resnick, “Scratch: A sneak preview,” in _Proceedings of C5_. IEEE Computer Society, 2004, pp. 104–109.
  * [29] M. Homer and J. Noble, “A tile-based editor for a textual programming language,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–4.
  * [30] D. Wendlandt, M. Casado, P. Tarjan, and N. McKeown, “The clack graphical router: visualizing network software,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 7–15.
  * [31] D. Medani, G. Haggard, C. Bassett, P. Koch, N. Lampert, T. Medlock, S. Pierce, R. Smith, and A. Yehl, “Graph works-pilot graph theory visualization tool,” in _Proceedings of SOFTVIS_. ACM, 2010, pp. 205–206.
  * [32] B. Jayaraman, “JIVE,” Jul. 2019. [Online]. Available: https://cse.buffalo.edu/jive/images/home/JIVE-UI.png
  * [33] P. Gestwicki and B. Jayaraman, “Methodology and architecture of jive,” in _Proceedings of SOFTVIS_. ACM, 2005, pp. 95–104.
  * [34] S. Lin, F. Taïani, T. C. Ormerod, and L. J. Ball, “Towards anomaly comprehension: using structural compression to navigate profiling call-trees,” in _Proceedings of SOFTVIS_. ACM, 2010, pp. 103–112.
  * [35] S. P. Reiss, “The paradox of software visualization,” in _Proceedings of VISSOFT_. IEEE, 2005, pp. 59–63.
  * [36] ——, “Visualizing program execution using user abstractions,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 125–134.
  * [37] ——, “Visualizing Java in action,” in _Proceedings of SOFTVIS_. ACM, 2003, pp. 57–66.
  * [38] P. Deelen, F. van Ham, C. Huizing, and H. van de Watering, “Visualization of dynamic program aspects,” in _Proceedings of VISSOFT_ , 2007, pp. 39–46.
  * [39] Q. Wang, W. Wang, R. Brown, K. Driesen, B. Dufour, L. Hendfren, and C. Verbrugge, “EVolve: an open extensible software visualization framework,” in _Proceedings of ACM_ , 2003, pp. 37–49.
  * [40] P. Johnson and S. Marsland, “Beat: a tool for visualizing the execution of object orientated concurrent programs,” in _Proceedings of SOFTVIS_. ACM, 2010, pp. 225–226.
  * [41] J. Waller, C. Wulf, F. Fittkau, P. Döhring, and W. Hasselbring, “SynchroVis: 3D visualization of monitoring traces in the city metaphor for analyzing concurrency,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–4.
  * [42] V. K. Palepu and J. A. Jones, “Revealing runtime features and constituent behaviors within software,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 86–95.
  * [43] S. P. Reiss, “Visualizing the Java heap to detect memory problems,” in _Proceedings of VISSOFT_. IEEE, 2009, pp. 73–80.
  * [44] A. Humphrey, C. Derrick, G. Gopalakrishnan, and B. Tibbitts, “Gem: Graphical explorer of MPI programs,” in _Proceedings of ICPP_. IEEE, 2010, pp. 161–168.
  * [45] C. Gouveia, J. Campos, and R. Abreu, “Using HTML5 visualizations in software fault localization,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–10.
  * [46] F. Petrillo, G. Lacerda, M. Pimenta, and C. Freitas, “Visualizing interactive and shared debugging sessions,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 140–144.
  * [47] J. Sajaniemi and M. Kuittinen, “Program animation based on the roles of variables,” in _Proceedings of SOFTVIS_. ACM, 2003, pp. 7–16.
  * [48] C. D. Hundhausen, J. L. Brown, and S. Farley, “Adding procedures and pointers to the ALVIS algorithm visualization software: a preliminary design,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 155–156.
  * [49] J. H. Cross II and T. D. Hendrix, “jGRASP: an integrated development environment with visualizations for teaching Java in CS1, CS2, and beyond,” _Journal of Computing Sciences in Colleges_ , vol. 23, no. 2, pp. 170–172, 2007.
  * [50] T. Sirkiä, “Jsvee & Kelmu: Creating and tailoring program animations for computing education,” _Journal of Software: Evolution and Process_ , vol. 30, no. 2, p. e1924, 2018.
  * [51] B. Ploeger and C. Tankink, “Improving an interactive visualization of transition systems,” in _Proceedings of SOFTVIS_. ACM, 2008, pp. 115–124.
  * [52] D. Kulesz, J. Scheurich, and F. Beck, “Integrating anomaly diagnosis techniques into spreadsheet environments,” in _Proceedings of VISSOFT_. IEEE, 2014, pp. 11–19.
  * [53] K. Hodnigg and M. Pinzger, “XVIZIT: Visualizing cognitive units in spreadsheets,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 210–214.
  * [54] S. Toprak, A. Wichmann, and S. Schupp, “Lightweight structured visualization of assembler control flow based on regular expressions,” in _Proceedings of VISSOFT_. IEEE, 2014, pp. 97–106.
  * [55] F. Beck, H. A. Siddiqui, A. Bergel, and D. Weiskopf, “Method execution reports: Generating text and visualization to describe program behavior,” in _Proceedings of VISSOFT_. IEEE, 2017, pp. 1–10.
  * [56] L. Georget, F. Tronel, and V. V. T. Tong, “Kayrebt: An activity diagram extraction and visualization toolset designed for the Linux codebase,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 170–174.
  * [57] G. Santos, N. Anquetil, A. Etien, S. Ducasse, and M. T. Valente, “OrionPlanning: Improving modularization and checking consistency on software architecture,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 190–194.
  * [58] A. Blouin, N. Moha, B. Baudry, and H. Sahraoui, “Slicing-based techniques for visualizing large metamodels,” in _Proceedings of VISSOFT_. IEEE, 2014, pp. 25–29.
  * [59] B. Clark and B. Sharif, “iTraceVis: Visualizing eye movement data within Eclipse,” in _Proceedings of VISSOFT_. IEEE, 2017, pp. 22–32.
  * [60] D. R. Bradley and I. J. Hayes, “Visuocode: A software development environment that supports spatial navigation and composition.” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–4.
  * [61] B. Sharif, G. Jetty, J. Aponte, and E. Parra, “An empirical study assessing the effect of SeeIT 3D on comprehension,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–10.
  * [62] A. Fronk, A. Bruckhoff, and M. Kern, “3D visualisation of code structures in Java software systems,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 145–146.
  * [63] A. Telea and D. Auber, “Code flows: Visualizing structural evolution of source code,” _Computer Graphic Forum_ , vol. 27, no. 3, pp. 831–838, 2008.
  * [64] M. Lungu, M. Lanza, and T. Gîrba, “Package patterns for visual architecture recovery,” in _Proceedings of CSMR_. IEEE, 2006, pp. 185–196.
  * [65] H. M. Kienle and H. A. Muller, “Requirements of software visualization tools: A literature survey,” in _Proceedings of VISSOFT_. IEEE Computer Society, 2007, pp. 2–9.
  * [66] J. Dietrich, V. Yakovlev, C. McCartin, G. Jenson, and M. Duchrow, “Cluster analysis of Java dependency graphs,” in _Proceedings of SOFTVIS_. ACM, 2008, pp. 91–94.
  * [67] J. Fabry and A. Bergel, “Design decisions in AspectMaps,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–4.
  * [68] S. Urli, A. Bergel, M. Blay-Fornarino, P. Collet, and S. Mosser, “A visual support for decomposing complex feature models,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 76–85.
  * [69] R. Wettel and M. Lanza, “Program comprehension through software habitability,” in _Proceedings of ICPC_. IEEE, 2007, pp. 231–240.
  * [70] G. Balogh and Á. Beszédes, “CodeMetropolis – a Minecraft based collaboration tool for developers,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–4.
  * [71] E. Murphy-Hill and A. P. Black, “An interactive ambient visualization for code smells,” in _Proceedings of SOFTVIS_. ACM, 2010, pp. 5–14.
  * [72] L. Voinea and A. C. Telea, “Visual clone analysis with SolidSDD,” in _Proceedings of VISSOFT_. IEEE, 2014, pp. 79–82.
  * [73] L. Merino, M. Lungu, and O. Nierstrasz, “Explora: A visualisation tool for metric analysis of software corpora,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 195–199.
  * [74] M. Lungu, “Softwarenaut,” Jul. 2019. [Online]. Available: https://cloud.githubusercontent.com/assets/464519/21022349/9ec2f748-bd7c-11e6-87ad-29c5332caba9.png
  * [75] R. Wettel, “CodeCity,” Jul. 2019. [Online]. Available: https://wettel.github.io/pics/wof/jmol.png
  * [76] S. Scarle and N. Walkinshaw, “Visualising software as a particle system,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 66–75.
  * [77] F. Fittkau, J. Waller, C. Wulf, and W. Hasselbring, “Live trace visualization for comprehending large software landscapes: The ExplorViz approach,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–4.
  * [78] L. Merino, M. Ghafari, C. Anslow, and O. Nierstrasz, “CityVR: Gameful software visualization,” in _Proceedings of ICSME_. IEEE, 2017, pp. 633–637.
  * [79] S. Jucknath-John and D. Graf, “Icon graphs: visualizing the evolution of large class models,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 167–168.
  * [80] R. Lintern, J. Michaud, M.-A. Storey, and X. Wu, “Plugging-in visualization: experiences integrating a visualization tool with Eclipse,” in _Proceedings of SOFTVIS_. ACM, 2003, pp. 47–56.
  * [81] M. Termeer, C. F. Lange, A. Telea, and M. R. Chaudron, “Visual exploration of combined architectural and metric information,” in _Proceedings of VISSOFT_. IEEE, 2005, pp. 1–6.
  * [82] L. Voinea and A. Telea, “How do changes in buggy Mozilla files propagate?” in _Proceedings of SOFTVIS_ , vol. 4, no. 05, 2006, pp. 147–148.
  * [83] W. De Pauw, S. Krasikov, and J. F. Morar, “Execution patterns for visualizing web services,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 37–45.
  * [84] L. Voinea, A. Telea, and J. J. van Wijk, “CVSscan: visualization of code evolution,” in _Proceedings of SOFTVIS_ , 2005, pp. 47–56.
  * [85] J. Zhi and G. Ruhe, “DEVis: A tool for visualizing software document evolution,” in _Proceedings of VISSOFT_. IEEE, 2013, pp. 1–4.
  * [86] R. Schulz, F. Beck, J. W. C. Felipez, and A. Bergel, “Visually exploring object mutation,” in _Proceedings of VISSOFT_. IEEE, 2016, pp. 21–25.
  * [87] P. Vogel, T. Klooster, V. Andrikopoulos, and M. Lungu, “A low-effort analytics platform for visualizing evolving Flask-based Python web services,” in _Proceedings of VISSOFT_. IEEE, 2017, pp. 109–113.
  * [88] M. D. Feist, E. A. Santos, I. Watts, and A. Hindle, “Visualizing project evolution through abstract syntax tree analysis,” in _Proceedings of VISSOFT_. IEEE, 2016, pp. 11–20.
  * [89] A. Hanjalić, “ClonEvol: Visualizing software evolution with code clones,” in _Proceedings of VISSOFT_ , 2013, pp. 1–4.
  * [90] M. Ogawa and K.-L. Ma, “Software evolution storylines,” in _Proceedings of SOFTVIS_. ACM, 2010, pp. 35–42.
  * [91] A. Telea, “CVSscan,” Jul. 2019. [Online]. Available<EMAIL_ADDRESS>  * [92] M. Meyer, T. Gîrba, and M. Lungu, “Mondrian: An agile visualization framework,” in _Proceedings of SOFTVIS_. ACM, 2006, pp. 135–144.
  * [93] I. Fernandez, A. Bergel, J. P. S. Alcocer, A. Infante, and T. Gîrba, “Glyph-based software component identification,” in _Proceedings of ICPC_ , 2016, pp. 1–10.
  * [94] S. P. Reiss, “The challenge of helping the programmer during debugging,” in _Proceedings of VISSOFT_. IEEE, 2014, pp. 112–116.
  * [95] T. Panas, R. Lincke, and W. Löwe, “Online-configuration of software visualization with Vizz3D,” in _Proceedings of SOFTVIS_ , 2005, pp. 173–182.
  * [96] B. Cleary, A. Le Gear, C. Exton, and J. Buckley, “A combined software reconnaissance & static analysis Eclipse visualisation plug-in,” in _Proceedings of VISSOFT_. IEEE, 2005, pp. 1–2.
  * [97] J. von Pilgrim and K. Duske, “GEF3D: a framework for two-, two-and-a-half-, and three-dimensional graphical editors,” in _Proceedings of SOFTVIS_. ACM, 2008, pp. 95–104.
  * [98] A. Bergel, S. Maass, S. Ducasse, and T. Gîrba, “A domain-specific language for visualizing software dependencies as a graph,” in _Proceedings of VISSOFT_ , 2014, pp. 45–49.
  * [99] D. Baum, J. Schilbach, P. Kovacs, U. Eisenecker, and R. Müller, “GETAVIZ: generating structural, behavioral, and evolutionary views of software systems for empirical evaluation,” in _Proceedings of VISSOFT_. IEEE, 2017, pp. 114–118.
  * [100] N. H. Reddy, J. Kim, V. K. Palepu, and J. A. Jones, “SPIDER SENSE: Software-engineering, networked, system evaluation,” in _Proceedings of VISSOFT_. IEEE, 2015, pp. 205–209.
  * [101] A. Bergel, “GRAPH,” Jul. 2019. [Online]. Available: http://agilevisualization.com/img/circle.png
  * [102] M. A. Musen, “The Protégé project: a look back and a look forward,” _AI Matters_ , vol. 1, no. 4, pp. 4–12, 2015.
  * [103] M. Sicilia, D. Rodríguez, E. García-Barriocanal, and S. Sánchez-Alonso, “Empirical findings on ontology metrics,” _Expert Systems with Applications_ , vol. 39, no. 8, pp. 6706–6711, 2012\.