byHendrik Wolf

Detail Visualization for Live Coding

Bachelor’s Thesis at theMedia Computing GroupProf. Dr. Jan BorchersComputer Science DepartmentRWTH Aachen University

Thesis advisor:Prof. Dr. Jan Borchers

Second examiner:Prof. Dr. Bernhard Rumpe

Registration date: 08.07.2014Submission date: 30.09.2014

iii

I hereby declare that I have created this work completely onmy own and used no other sources or tools than the oneslisted, and that I have marked any citations accordingly.

Hiermit versichere ich, dass ich die vorliegende Arbeitselbstandig verfasst und keine anderen als die angegebe-nen Quellen und Hilfsmittel benutzt sowie Zitate kenntlichgemacht habe.

Aachen, September2014Hendrik Wolf

v

Contents

Abstract xi

Uberblick xiii

Acknowledgements xv

Conventions xvii

1 Introduction 1

2 Related work 5

3 Prototype 15

3.1 Motivation for our Enhanced Prototype . . . 15

3.1.1 Shortcomings of Kurz’ Prototype . . . 16

3.2 Initial Design Ideas . . . . . . . . . . . . . . . 18

3.2.1 Three Columns Layout . . . . . . . . . 19

3.2.2 Each Variable in Its Own Frame . . . 21

3.2.3 Linking the Three Columns . . . . . . 23

vi Contents

3.2.4 Grouping of Data Types and TheirDetail Visualizations . . . . . . . . . . 25

3.3 Preliminary User Study . . . . . . . . . . . . . 33

3.3.1 Setup and Tasks . . . . . . . . . . . . . 33

3.3.2 Results . . . . . . . . . . . . . . . . . . 35

3.4 Final Design and Applied Changes . . . . . . 36

3.4.1 Frame Design . . . . . . . . . . . . . . 36

3.4.2 Detail Visualizations . . . . . . . . . . 37

4 Implementation 41

4.1 Existing Foundation for Our Prototype . . . . 41

4.2 Newly Added Components . . . . . . . . . . 45

4.3 Frame Management . . . . . . . . . . . . . . . 47

4.3.1 Frame Creation . . . . . . . . . . . . . 47

4.4 Visualization Management . . . . . . . . . . . 48

4.4.1 Data Type Dependent Visualization . 48

5 Discussion of our Prototype 49

5.1 Capabilities and Advantages . . . . . . . . . 49

5.2 Limitations and Shortcomings . . . . . . . . . 50

6 Summary and Future Work 53

6.1 Summary and Contributions . . . . . . . . . 53

6.2 Future Work . . . . . . . . . . . . . . . . . . . 54

Contents vii

A Not Implemented Detail Visualizations beforeUser Study With New Frame 57

Bibliography 63

Index 67

ix

List of Figures

1.1 Three possible visualizations for a numbervariable . . . . . . . . . . . . . . . . . . . . . . 2

2.1 Rehearse - continuous inline feedback . . . . 7

2.2 Inventing on Principle - binary search example 8

2.3 Learnable Programming - two different visu-alizations . . . . . . . . . . . . . . . . . . . . . 9

2.4 Screenshot of the Light Table . . . . . . . . . 10

2.5 Screenshot of the IPython Notebook . . . . . 11

2.6 Screenshot of Swift Playgrounds . . . . . . . 12

3.1 The old version of the prototype by JoachimKurz . . . . . . . . . . . . . . . . . . . . . . . 16

3.2 Inline visualization of a variable in Light Table 19

3.3 Three column layout . . . . . . . . . . . . . . 21

3.4 One idea for the frame design and its menus 22

3.5 Illustration of the missing link to the detailview . . . . . . . . . . . . . . . . . . . . . . . . 23

3.6 Image of the hierarchical list visualization . . 27

x List of Figures

3.7 Plot visualization of a number variable . . . . 29

3.8 Hierarchical list visualization of an object . . 31

3.9 HTML Visualization of a HTML image string 32

3.10 Final frame and its menus . . . . . . . . . . . 37

3.11 Final number visualization . . . . . . . . . . . 38

3.12 Final object visualization . . . . . . . . . . . . 38

4.1 Server Architecture . . . . . . . . . . . . . . . 43

4.2 Newly added components . . . . . . . . . . . 46

xi

Abstract

Programming without receiving a feedback on the internal runtime state of a pro-gram demands much from the cognition of the programmer. He has to take overthe role of the computer and simulate the program flow continuously in his head.In contrast, live coding uses the capability of the computer for that continuous sim-ulation. The computer executes the program in the background and displays theresult after each applied change.

Several existing live coding tools explore detail visualizations with different levelsof abstraction and possible user interaction. These tools show the benefit of fur-ther extensions. One extension is the capability to regulate the amount of showninformation or filter out interesting parts. Furthermore, a majority of prototypesdoes not yet offer the possibility to choose between different representations. Thispossibility is essential since there is no visualization which is appropriate for eachsituation.

This thesis presents a live coding prototype in JavaScript as well as diverse detailvisualizations. The prototype enables the programmer to switch between differentrepresentations to find the most appropriate one. In case there exists no appropriatevisualization, our prototype offers the possibility to enhance it further by addingnew visualizations to the existing set.

xii Abstract

xiii

Uberblick

Wahrend des Programmierens erhalt der Programmierer keine Informationen ubermogliche Programmzustande zur Laufzeit. Er muss die Rolle des Computersubernehmen und die Ausfuhrung des Programms kontinuierlich in seinem Kopfsimulieren. Dies erfordert hohe Konzentration. Im Gegensatz dazu wird beim LiveCoding die Leistungsfahigkeit und Kapazitat des Computers fur diese Simulationbenutzt. Der Computer fuhrt das Programm im Hintergrund aus und zeigt dieErgebnisse nach jeder Anderung am Quelltext an.

Es existieren bereits diverse Live Coding Werkzeuge, die verschiedene Detail-Visualisierungen mit variablem Abstraktionslevel und moglicher Benutzerinter-aktion anbieten. Diese Werkzeuge zeigen das Potential von Live Coding. Einemogliche Erweiterung ist, dem Benutzer die Fahigkeit zu geben, die angezeigteDatenmenge selbststandig zu regulieren und interessante Teile herauszufiltern.Außerdem fehlt in den meisten Live Coding Werkzeugen und Prototypen bis jetztdie Moglichkeit, zwischen verschiedenen Visualisierungen zu wechseln. Das istwichtig, da es keine Visualisierung gibt, die in allen Situationen passend und hil-freich ist.

In dieser Arbeit prasentieren wir einen Live Coding Prototypen fur JavaScript,sowie verschiedene Detail Visualisierungen. Unser Prototyp ermoglicht dem Be-nutzer, zwischen verschiedenen angebotenen Visualisierungen zu wechseln. Weit-erhin hat der Benutzer die Moglichkeit eigene Visualisierungen zu implementierenund sie den bestehenden hinzuzufugen.

xv

Acknowledgements

I would like to thank Prof. Dr. Jan Borchers for making this thesis possible. I wouldalso like to thank my supervisor Jan-Peter Kraemer, for giving me this amazingtopic as well as his valuable and constructive suggestions.

Special thanks go to Moritz Wittenhangen, for taking over as my supervisor andgiving me frequently advice. Lastly, I would like to thank all people at i10 forhelping me whenever I had a problem or question.

Thanks for all the support!

xvii

Conventions

Throughout this thesis we use the following conventions.

Text conventions

Definitions of technical terms or short excursus are set offin colored boxes.

EXCURSUS:Excursus are detailed discussions of a particular point ina book, usually in an appendix, or digressions in a writ-ten text.

Definition:Excursus

Source code and implementation symbols are written intypewriter-style text.

myClass

The whole thesis is written in American English.

1

Chapter 1

Introduction

The original programming cycle is divided into fourphases: edit, compile, link and run. The programmer re-ceived no feedback while editing the source code. As a re- Programming without

feedback is strainingsult, in order to verify that the program works as intended,the programmer has to simulate the changes and their ef-fects in his head. But simulating the flow of a programstrains the limits of the human cognition [Snell, 1997]. Ad-ditionally, most people do not want to wait frequently fortheir program to compile, link and finally run so that theycan debug it. Thus, they implement as much as possiblewithout debugging the program. This leads to a larger time Longer code editing

periods increase therequired debuggingtime

gap between the editing of source code and the detection ofbugs, which increases the required debugging time. Moreparts of the program are affected by the bugs and as a con-sequence have to be adapted to fix them.[Saff and Ernst,2004a].

LIVE CODING/PROGRAMMING:A coding paradigm. The computer continuously exe-cutes the program in the background and provides im-mediate feedback of the program state after each appliedchange. The feedback can be displayed automatically oron demand.

Definition:LiveCoding/Programming

Recently, increased research on live programming fea-tures and tools has been conducted [Choi et al., 2008],

2 1 Introduction

[McDirmid, 2013], [Victor, 2012b], [Guo, 2013]. Live pro-Live programming isexpected to assist

programmers byproviding continuous

visual feedback

gramming intends to close the time gap between program-ming and debugging by providing immediate feedbackwhile programming. This allows to reduce the four phasesof the cycle to just one. Expectations on live programmingare, e.g., the minimization of the latency between the visualfeedback on a change and it effects on the program, reduc-tion of the strain on the human cognition and the simplifi-cation of debugging (faster and more accurate). However,to tap the full potential of live coding, appropriate visual-izations are required. Using live programming to providejust arbitrary immediate feedback is not necessarily useful.Victor [2011] states the following in his essay ”Ladder ofAbstraction”:

”The appropriate visualization varies, and often there aremultiple good visualizations, each offering a useful per-spective. We can look for metrics that summarize some as-pect of the behavior. We can also consider transformationsthat make it easier to visually compare multiple states ofthe system.”

Figure 1.1: Three possible visualizations for a variable of type number. On the left, next tovalue assignments of type number, the assigned value is shown. The results of assignmentsin conditions are only visualized if they evaluate to true. This provides a quick overview onthe program state. In the middle, a hierarchical list can be seen, which provides a view on allvalue changes applied to a number variable within its scope. On the right, a visualizationin form of a plot is shown, which also illustrates all value changes assigned to the variable.

For example, a number variable in JavaScript can be visu-alized in form of a hierarchical list consisting of all valuechanges over the whole program or as an exact value rightto each line consisting of an assignment. In other situa-

3

tions a plot could be the better choice, e.g., when the trend No appropriatevisualization for eachsituations

of a variable throughout a loop is of interest (see figure1.1). Usually, the user is capable of answering the questionin which situation, which visualization is the most useful.Therefore, it is important that the user can switch betweendifferent visualizations easily.

Recently, new live coding tools and prototypes, such as the Almost non tools andprototypes supportswitching betweendifferentvisualizations

Light Table 1 by Chris Granger or Swift Playgrounds 2 byApple, have been published, each with a slightly differentlayout and approach on how to visualize feedback. Theyenable a programmer to obtain live results in a more de-tailed way. To give an example, the Light Table enables theuser to look at the properties of an object in form of a hierar-chical list. The user can click trough the different hierarchylevels to open parts of interest for a more detailed view.But these new tools and prototypes do not provide func-tionality for switching between different visualizations.This prevents the programmer to find the appropriate visu-alization for each situation. The moment their tool choosesnot the appropriate one and shows the wrong data, or theright data in a wrong way, the visualized feedback becomesworthless for the programmer.

In this thesis we present a live coding prototype, which en- Present a prototypewhere differentvisualizations can beused and added

ables the programmer to open an interactive visualizationfor variables and to switch between different visualizationsto find the appropriate one for his needs. Furthermore, aprogrammer can implement his individual visualizationsfor different data types and add them to the selection. Thisbachelor thesis is structured as follows:

First, we present in chapter 2 “Related work” inspiring re-lated work and state-of-the-art developments, which weuse for comparisons with our own prototype, in the areaof live coding. In chapter 3 “Prototype” we present ourprototype. We start with a list of reasons why an enhance-ment of Kurz’ prototype is indispensable. Afterwards, weintroduce our initial ideas for our layout as well as detailvisualizations and illuminate our design process. Then wepresent our conducted preliminary user study which we

1 http://www.lighttable.com2https://developer.apple.com/swift/resources.com

4 1 Introduction

used to get some feedback on our concept and design. Inthe end, we reveal our final design and highlight the ap-plied changes which are based on the user study. Chap-ter 4 “Implementation” deals with the implementation ofour prototype. Since the prototype is an enhancement ofKurz’ prototype, we explain its architecture for a generalunderstanding of its manner of functioning. Then, we de-scribe our newly constructed components, their assignedtasks and the integration into Kurz’ prototype. We preciseboth in a more detailed way, the management of the framesand the management of the detail visualizations. In chap-ter 5 “Discussion of our Prototype” we discuss our proto-type with regard to its capabilities and limitations. Finally,we describe in chapter 6 “Summary and Future Work” pos-sibilities for further enhancements of our prototype and theneed for a future user study after it got refined.

5

Chapter 2

Related work

In this chapter we present all kinds of previous research We present pastresearch as well asup-to-datedevelopments

and state-of-the-art developments on the topic of live pro-gramming. We start with an introduction of the typicallyused action cycle while programming, its problems and at-tempts to solve these problems. We then present early re-search in the direction of live programming, followed bysome research on guidelines and taboos regarding the vi-sualization of live feedback. Afterwards, we present var-ious new developments and up-to-date software releases.Finally, we describe some research about the usefulness oflive programming.

The assumption that complex cognitive skills are a neces- Programming withoutfeedback anddebugging takes agreat strain onhuman cognition

sity for programmers exists at least since 1975. Gould [1975]describes in his paper ”Some Psychological Evidence onHow People Debug Computer programs” multiple factorswhy the error correction task (debugging) is exhaustingand cognitive skills are needed. To reduce the strain on theprogrammer, a high number of debugging tools has beenpublished. They are intended to simplify the debuggingprocess by illuminating the program, its runtime states andthe existing data at these states. An example for a debug-ging tool is the Whyline, a debugging prototype for Alice,a programming environment [Ko and Myers, 2004]. TheWhyline allows the programmer to ask predefined ques-tions about the program’s runtime failure in form of ”whydid” and ”why didn’t” questions. The set of available ques-

6 2 Related work

tions is generated automatically. The tool answers thesequestions by showing the runtime events which causedthe output. In comparison to the usually used approach,stepping through the code after implementing it, this toolsimplifies the search for the corresponding parts of codecausing the output. To sum up, debugging tools assist theprogrammer with the error correction task and reduce thestrain on the programmer.

However, there is also a strain on the cognition of a pro-grammer while editing source code. Snell [1997] explains,that this strain is caused by the necessity to simulate thewhole program flow unassisted while editing. He intro-duces a tool which already uses the idea of live coding.It merges the edit, evaluate and debug mode (similar tothe four phases introduced in the introduction) and pro-vides continuous feedback for the programmer. He callsthe method of getting feedback just after typing some codethe ”ahead of time debugging” feature. But the concept ofContinuous feedback

already introduced1985 by Henderson

and their VisiProgenvironment

continuous feedback was introduced even earlier by, e.g.,Henderson and Weiser [1985] and their VisiProg environ-ment. They describe in a futuristic way (they built no actualprototype) a system where a programmer receives contin-uously feedback for his input and illustrates the specifica-tions of their system.

The attempt of making programming easier to understandIntroduction of VIVAas a language forimage processing

by using not only textual feedback but also more graphi-cal visualizations, such as diagrams or flow charts, was al-ready introduced by visual programming languages. Oneexample is VIVA, a visual language for image processing[Tanimoto, 1990]. In VIVA the programmer can express al-gorithms by drawing flow diagrams (use of visualizations)instead of writing source code.

Over the last years, plenty of papers, prototypes and toolsIntroduction ofRehearse editor and

the use of inlinevisualizations for

feedback

concerning live coding have been published. One of theearlier tools is the Rehearse editor, introduced by Choi et al.[2008]. Using an inline visualization the programmer re-ceives direct feedback on the results of execution. In ad-dition, undone statements are kept visible to enable back-tracking to earlier states of the code (see figure 2.1).

7

Figure 2.1: The figure shows a function called ”stylize” and itsdefinition in the Rehearse editor. (1) shows the function name,parameters and values. Beneath each line of source code (2), theresult of execution is shown (3). Undone statements (4) are stillvisualized which simplifies backtracking. This screenshot is takenfrom Choi et al. [2008]

Another approach of the Rehearse development environ-ment was introduced by Br et al. [2010]. The environ-ment enables a developer to execute and afterwards inter-act with his application in an extra window. Triggered byeach user interaction, recently executed lines become high-lighted. The authors claim that using this visualizationsimplifies connecting results of interactions with the cor-responding source code.

A more recent development is the prototype implemented Probing in form ofinline feedback andtracing in form of asecond column formarked code

in the programming language YingYang and published byMcDirmid [2013]. His prototype uses a combination ofprobing (an inline visualization) and tracing. Probing en-ables the user to inspect all kind of variables and expres-sions by visualizing their current values directly in the next

8 2 Related work

Figure 2.2: This figure is a screenshot of Victor’s talk ”Inventing on Principle” [Victor,2012b]. On the left side the source code for a binary search method is illustrated. On theright side we can see the used example function parameters for the binary search function,as well as feedback for each time a variable is used. For each iteration of the used while loopa new column for all its variables is created.

line. Tracing enables the user to place print statements,which then visualize the chosen data in a second column, aso called trace pane. For example, different states of an ar-ray can be illustrated. Furthermore, each trace output canbe used as a navigation to the source code used to createthat line.

An influential person in the area of live coding and the de-Bret Victor publishedmultiple essays and

and held apresentation

regarding live codingand visualizations

sign of visualizations for feedback is Bret Victor. He madenumerous contributions in the field of live coding and heldsome talks about it such as ”Inventing on Principle” Victor[2012b]. He illustrates plenty of visualization techniques onwhat and how to visualize feedback. Moreover, he presentsin his essay ”Ladder of Abstraction” some design guide-lines, e.g., that depending on the situation a different vi-sualization with a higher or lower level of abstraction canbe appropriate [Victor, 2011]. In his essay ”Learnable Pro-gramming” Victor [2012a] enumerates multiple criteria aprogramming environment has to fulfill to make the pro-gramming more understandable for the user. Additionally,he names plenty of misconceptions in regard to the contentof feedback and its design. Almost all of his examples inhis essay ”Learnable Programming” refer to the online en-vironment Khan Academy1 , which was published for be-ginners to learn programming.

1https://www.khanacademy.org/computing/cs

9

Most of the examples in his essay refer to Khan Academy, Victor criticizes thelive codingenvironment by KhanAcademy andsuggestimprovements

since it was a response to this environment. They took hisprevious talks as an inspiration for it and Victor was notsatisfied with the result. Khan Academy’s environment of-fers a live coding feature in form of a two view layout. Onthe left side the user can write his code and on the rightside the result of the program (everything is rendered intoa canvas element) is permanently updated, without show-ing, e.g., intermediate results for variables.

Figure 2.3: This figure consists of two screenshots from a video used in Victor [2012a]. Asource code sample is on the left side of each screenshot. On the right side is the visualizedfeedback. This visualization is an example for the usage of different abstraction levels. For asmall part of the timeline the exact newly assigned value is shown in form of a table. Whenthe programmer zooms out of the timeline the illustration changes from a table to a plot. Bymoving the mouse over the plot for a variable its concrete values are shown.

Finally, we present three of the newest published program-ming environments with live coding features: Light Table2, Swift Playgrounds3 and IPython Notebook4 . All of themoffer visualizations of varying abstraction levels, which theother presented prototypes do not. It is interesting to seethat all of them actually chose a different layout and place-ment for their visual feedback.

The Light Table by Chris Granger, for JavaScript, combines Use of a secondcolumn and inlinevisualizations for thelive feedback

an arbitrary number of columns and inline layout for itslive feedback (see figure 2.4). Two columns are the standardfor the JavaScript version of the Light Table, but the user

2http://www.lighttable.com3https://developer.apple.com/swift4http://ipython.org/notebook.html

10 2 Related work

Figure 2.4: Shows the Light Table and parts of a JavaScript program. Two columns arecurrently used and an inline visualization for an object is open.

can add any desired number of columns. These columnscan contain websites as well as other files. Furthermore, aconsole can be opened at the bottom of the editor. On theleft side a programmer can constantly see his source code,on the right side a webpage where the JavaScript code isinduced.

Furthermore, to look at intermediate results, each variableSecond columnshows a website

where the JavaScriptcode is induced

can be inspected in a more detailed view. To give an exam-ple, for an object an inline visualization in form of a hierar-chical list, containing all its properties, can be opened. Theuser can open and close properties of interest for furtherinformation. Additional information and intermediate re-sults can be also perceived by opening a console at the bot-tom of the editor. When it is open, it uses the whole widthof the editor for feedback visualization. To add results tothe console the user has to use the console.log() com-mand.

11

Figure 2.5: The figure illustrates an IPython Notebook example. For each line of input,an output line directly beneath that is created whenever an Python object is returned by anexpression.

In IPython a total inline layout is used (see figure 2.5). Ev-ery output for an input line, which consists of an expres- Alternate

representations forfeedback can beselected

sion and returns a Python object, is displayed directly inthe next one or multiple lines. Commands, such as (lastoutput), can be used to access the results later on. Onefeature, which is similar to our concept, is the offered func-tionality to choose alternative representations for feedbacksuch as HTML, JSON, PNG or LaTeX.

The layout of the recently published Swift Playgrounds Use of a a sidebarand assistant editorfor feedback

(still in the beta phase) builds a strong contrast to that (seefigure 2.6). As a standard there are two columns. The leftcolumn (called Source Editor Window) contains the sourcecode, while the right one (called Sidebar) contains feedbackfor every line of source code directly next to it. Whenever auser wants more information, such as more context, a thirdcolumn named Assistant Editor is used. For each line offeedback in the sidebar, a new visualization in the AssistantEditor can be opened. The Assistant Editor is only visiblewhen there is at least one detail visualization open.For each type of data exists a predefined visualization. The Use of predefined

visualizationsdetail visualization, e.g., for a number variable of interest, isa plot which contains all the changes applied to the variableat the chosen line over the runtime. Furthermore, the SwiftPlaygrounds enable the visualization of a whole applica-tion. To sum up, the sidebar is primarily used for textualvisualizations with less context around it, whereas the As-sistant Editor provides more detailed visualizations, con-taining more information and possibly with a more graph-

12 2 Related work

Figure 2.6: The image shows an example of the Swift Playgrounds and their live codingfeatures. It is divided into three columns. The left column contains the source code, thecentral one short feedback for assignments, functions or loops and the third column showstwo detail visualizations. The top one illustrates the result after executing the whole ap-plication called ”Balloons”. Beneath that is a visualization in form of a plot, showing theflight height of a zeppelin over time. It is associated with the variable yOffsetForTime.

ical representation.

Numerous more prototypes exists, but since we want tolimit the number of presented prototypes, tools and con-cepts to the (in our opinion) most interesting ones we justenumerate a few of them and do not describe them in moredetail: Guo [2013] with Python Tutor, Pharo5, Lively Ker-nel6 and Edwards [2004] with the EG prototype.Finally, after looking at all these live coding tools and pro-totypes, the question arises how great the benefits of livecoding and its continuous visual feedback are.

A study about continuous visual feedback while debug-ging conducted by Wilcox et al. [1997] showed a neutralresult and they concluded, that it could be task and pro-grammer dependent whether live feedback actually is of

5http://pharo.org/6http://www.lively-kernel.org/

13

benefit or not. In their user study each participant was con-fronted with two Forms/3 programs to debug, one withlive feedback and one without. Since the participants were Usefulness of live

feedback probablytask and programmerdepend

divided into two groups, respectively half of the partici-pants worked on each problem with live feedback. For theiruser study they used three measurements: debugging ac-curacy, debugging behavior and debugging speed. Theirresults showed no overall improvement regarding the ac-curacy and speed. It was quite task dependent whether livefeedback improved the results or made them worse.In another study, conducted by Saff and Ernst [2004b] andpresented in their paper ”An experimental evaluation ofcontinuous testing during development”, the results wereslightly in favor of live feedback.To sum up, the positions and findings about live codingvary and it is still necessary to research in which situationsit provides a significant advantage. We believe that the re-sults of future users will be in favor for live coding andmore significant. Live coding needs to be researched fur-ther before we can tap its full potential. To give an exam-ple, there are still different approaches (two columns, threecolumns, two columns and inline,...) on how to optimallyintegrate visual live feedback into development environ-ments. It is still unclear which approaches are best suited.

15

Chapter 3

Prototype

In this chapter we present our prototype, which is an en- This chapter is splitinto the initial designprocess, apreliminary userstudy and our finaldesign

hanced version of the already existing prototype by Kurz[2013], Heinen [2012] and Belzmann [2013] for JavaScriptin Adobe Brackets, illuminate the design process and jus-tify our design decisions. We start with giving reasons forthe necessity of an enhancement of the current prototype.Then, we introduce our basic design concept, such as sepa-rating the editor in three columns, and detail visualizations.Next, we explain the purpose of our preliminary user studyand describe its setup. One reason for this user study wasto improve and refine our design. Finally, we present ourfinal design and talk about the applied changes, based onthe user study.

3.1 Motivation for our Enhanced Proto-type

Our ambition is to develop an interactive and easily ex-pandable software prototype, which enables the user tolook at specific variables of interest in detail and perceivetheir changes while programming. Mainly, as already men-tioned in the introduction, it is intended to help the userto keep track of the current state of the program and to fixbugs while programming, without the exhausting task of

16 3 Prototype

simulating the program flow and effects of changes to thecode in his head. The prototype is going to be used for fu-ture user studies after further refinement. One reason is totest the usefulness of our enhanced prototype for program-mers. A couple of ideas for possible user tasks will be listedin chapter 6 “Summary and Future Work”. Our prototypeis based on the Adobe Brackets plugin by Kurz [2013].

Figure 3.1: A cutout of the old prototype by Joachim Kurz. The source code is on the left,the live feedback on the right. The user can use the boxes in line 8 and 11 to step throughthe iterations of the loops.

3.1.1 Shortcomings of Kurz’ Prototype

The current live coding editor by Kurz [2013] already pro-Current visualizationprovides in some

cases not enoughinformation

vides a rough overview on the state of a program, by show-ing the immediate results for each assignment and termright next to the corresponding line of source code. There-fore, it is divided into two columns: one for the source codeand one for the feedback. We call the first column (sourcecode) from now on code view and the second one (visual-ized feedback) feedback view.In a majority of cases, the current live coding editor withKurz’ prototype plugin (see the cutout in figure 3.1) is suf-ficient to spot bugs and responsible parts of code. Butoccasionally more detailed, clearly laid out visualizations,placed in a larger space than one line, are necessary.

In Kurz’ prototype, we determine the following three major

3.1 Motivation for our Enhanced Prototype 17

problems we want to solve with our new prototype:

1. Lack of possible interaction with the visualization,to adapt the representation and amount of informa-tion

There is almost no interaction possible with the cur- Filtering ofinformation orswitching betweenvisualizations is notpossible with Kurz’prototype

rent prototype. It is not possible to switch betweendifferent visualizations, hide any undesired informa-tion or get additional one. For example, when con-sidering a nested object, initially reducing the showninformation to the top-level properties could preventfrom the confrontation with a large amount of un-necessary information. If required, the user can openproperties of interest by himself.

Using the current live editor, monitoring a couple ofselected variables and their changing values can be-come nerve-racking. The user has to look for all lineswhere value changes are applied to the monitoredvariable. That is time consuming and not efficient.

2. Partly unfitting illustration of information

In the current prototype each visualization for an ap- Often not enoughspace for anappropriatevisualization

plied assignment has the space of exactly one sourcecode line. More precisely, each visualization is about25 pixel high and infinitely wide. But for visualiza-tions with a width above the width of the feedbackview, a scrollbar shows up and only a part of thewhole content is visible at the same time. That spaceis sufficient for the visualization of a single number asa result of a term, but not for most of the other datatypes, such as an array or an object. In addition, datasuch as HTML elements, cannot be illustrated prop-erly by visualizations with a textual representation. Possibility of a less

abstract view ismissing

In these cases a more graphical representation is es-sential. One possibility is to take the HTML elementand render it.It becomes notable, that in some cases a visualizationneeds more space than one line, to allow a clear lay-out.

18 3 Prototype

3. Missing context and additional information

The current prototype provides for each line in theIt occurs that contextis missing code view exactly one line in the feedback view.

These lines are located right next to their correspond-ing lines in code view. Thus, a line of code, whichchanges the value of a variable, and its result are al-ways located next to each other. That can become aproblem when, e.g., changes to a variable are appliedat different locations in the program. The user has toscroll through the program to compare how its valuechanges.

Bret Victor states the following in his essay ”Learn-able Programming”:

”Data needs context. It is rarely enough to see a sin-gle data point in isolation. We understand data bycomparing it to other data.” Victor [2012a].

Therefore, the current illustration of loops and valuechanges during all iterations implies another prob-lem. While clicking through the iterations, the pro-grammer has to remember previous values of an iter-ation to compare them. Thus, it is difficult to observevalue changes. Also, additional information such asthe prototype of an object or the length of an array aremissing.

3.2 Initial Design Ideas

During our development process we had to make multipleSpace forvisualizations is

limited, thereforethoroughly planing ofthe layout is essential

design decisions concerning how and where to realize ourdetail visualizations. The limited space for the detail visu-alizations is a challenge, moreover since the code and feed-back view already take a great part of the available space.We looked at various already existing live coding proto-types and tools, collected ideas and finally came up withdesigns for our separate detail visualizations and our un-derlying layout.

3.2 Initial Design Ideas 19

3.2.1 Three Columns Layout

Since the available space in an integrated development en- Inline visualization isone possibility forplacing feedback

vironment (IDE) is quite limited, it is necessary to thinkabout where to place the visualizations for the live feedbackand how much space to provide for them. One possibilityis to use an inline approach, such as it is done in the LightTable 1, placing the visualizations in the same space as thesource code (see figure 3.2).

Figure 3.2: A screenshot from a video introducing the Light Ta-ble. It shows an inline visualization. The source code is pushedaway by the detail visualization of a variable named mesh.

We decided against this approach, because with each line1 http://www.lighttable.com/

20 3 Prototype

consumed by the visualization, the alignment of the sourcecode is altered further and potentially important contextInline approaches rip

apart the alignmentand less source code

is visible

disappears from sight. An inline approach could also beapplied in the feedback view, but the main idea of that viewis that each result is aligned right next to its associated linein the code view. An inline approach would destroy thatconcept.

Another possibility is to use some vertical space of editor,as it was done by the canvas view in the SuperGlue en-vironment presented by McDirmid [2007], where the visu-alizations are appended at the bottom in a separate field.However, todays monitors usually have an aspect ratio of16:9 or 16:10 and offer more horizontal than vertical space.That implies to rather use up the horizontal than the verti-cal space.

In Pharo2, an IDE wich provides immediate feedback, for aPharo creates foralmost each task a

new window, but thatprovokes disorderand conceals the

other windows

high number of tasks a new indivudal window is created.The windows can be placed wherever the user wants toplace them. We believe this solution is open for disordercaused by the user and not properly arranged by its own.Moreover, whenever a new additional window is created,it overlaps existing ones and hides their content.

Finally, we decided to use a three column layout, like it isWe use a threecolumn layout used in the newly published Swift Playgrounds3. We took

the old prototype and added a third column, which we calldetail view, for the detail visualizations. Starting from left,the code view is still located in the first column and thefeedback view from the old prototype in the second one.Using this layout, we have to think about how to split theavailable space between these three different views. Sincethe required space for each view can change depending onthe current situation, we made the columns resizable.

2http://pharo.org/3https://developer.apple.com/swift

3.2 Initial Design Ideas 21

Figure 3.3: The UI is divided into three columns. From left toright the different views: code, feedback and detail.

3.2.2 Each Variable in Its Own Frame

Our goal is enabling a detail visualization with possible in-teraction for each variable. In our prototype each visualiza- Each variable is

created in its ownpersonal frame withits own settings andoptions

tion is opened in an own “private” frame by clicking on aninteresting value in the feedback view. This frame is, whenopened for the first time, positioned at the same height asthe associated line, pushing aside overlapping frames. Incase the user wants to open an already existing frame, theexisting frame becomes repositioned to the position of theassociated line. We use this repositioning to point out thelink of the frame, which is currently of particular interest.

Each frame consists of a menu bar and an area for the ac- Menu bar offersdiversefunctionalities toadapt visualizationsand the showncontent

tual detail visualization underneath. The menu bar con-tains the menus shown in figure 3.4 and provides function-alities such as switching between different visualizations,filtering of specific values or closing the detail visualiza-tion.There are two main arguments for our ”independent frameconcept”: First, every detail visualization having its ownframe results in a clear visual separation of all visualiza-tions. In addition, establishing a link to the code view There is a clear

visual separation ofall visualizations andindividual adaption ispossible

as well as the feedback view becomes less difficult. Sec-ond, every detail visualization can be adapted indepen-dently from the rest. The visualization in one frame canbe changed without influencing the others.Furthermore, it is still possible to add global functionality,affecting all frames, without additional work. The disad-vantage of this solution is, that each frame and menu-bartakes up a bit of the limited space. Comparing the advan-tages and the disadvantages of this approach it seems to usthat the idea of separate frames is worth to be explored.

22 3 Prototype

Figure 3.4: One of the first ideas for the design of the frame. Onthe left side of the frame are the two buttons for the navigationand above them the associated variable name of the frame. Onthe right side we put the other buttons for the filter menu, optionmenu, minimization and closing. Right next to the frame youfind the designs for the filter and option menu. At the top is thefilter menu, designed like the search menu in the Brackets editor.Beneath that is the option menu, containing possible visualiza-tions and options.

All frames have further functionality. For example, to con-Each frame offersfurther functionality

such as dragging orresizing

form the height and position of each frame to the currentsituation, they can be resized and dragged. When drag-ging a frame to the position of an other frame, they are au-tomatically positioned side by side, which enables a bettercomparison between them.The menu bar provides one more feature: a navigationfunction for nested data types, similar to the navigation onwebsites. When a nested property of an object is opened,the navigation can be used to switch to the visualization ofthe parent object and backwards. This navigation function-ality has to be implemented for each visualization indepen-dently and for several ones it does not make any sense toimplement such functionality. Due to a lack of time we didnot implement this functionality but reserved space for itand kept the idea in our design, in order to add it later on.The set of possible interactions with an actual detail visual-

3.2 Initial Design Ideas 23

ization depends on its programmer.

Offering numerous features and possible interactions to theprogrammer represents a trade off. On the one hand, it canincrease the value of visualizations and, e.g., help the pro-grammer to find errors and bugs. On the other hand, theuser can become too distracted by the visualizations and asa result uses too much time to interact with them. Duringthis time, the programmer does not write any source code.We took that risk into consideration and still believe thatoffering features to adapt a visualization and the dis-played information provides more advantages than disad-vantages.

3.2.3 Linking the Three Columns

In his talk ”Inventing on Principle”, Victor [2012b] men- A link betweensource code and itsillustrated feedbackis important

tions the necessity of a connection between source code andits output. When using live programming to provide im-mediate visual feedback, it is also important that the usercan see the link between the visualizations and the useddata. Without that link, the feedback is of no use to theuser. He does not know on which data the visualization isbased.

Figure 3.5: The screenshot illustrates the missing link from the code and feedback viewto the detail view. The black arrows represent an established link between each line in thefeedback and code view. The red arrows mark the missing link from the feedback and codeview to the detail visualizations in the detail view.

In Kurz’ prototype this link could be established without

24 3 Prototype

effort, since the visualizations and their associated lines inthe code view are in a line-to-line relationship, as you can seeBy introducing the

detail view in a thirdcolumn the

establishment of alink between all

views becomes morecomplicated

in figure 3.5. Additionally, the corresponding line in thesource code is highlighted while the mouse is over an visu-alization.Unfortunately, it is not that simple when considering ourdetail view. One problem is the increased amount of spacenecessary for appropriate detail visualizations, in compar-ison to the ones in the feedback view. As a result, the line-to-line concept can not be applied for our detail view (seefigure 3.5). Even in case that a visualization is created at thesame line, the distance to its corresponding line in the codeview prevents the user from recognizing the link withoutdifficulties.

We came up with multiple solutions to make the link be-To create a link weuse highlighting,displaying of the

variable name andconnecting lines

such as used in theversion editor in

Xcode

tween the three views more transparent. First, we had theidea to use highlighting like it is done in the prototype byKurz. While the mouse is tracked over an element in oneview, the corresponding parts in the other views are high-lighted. Second, each frame contains the associated vari-able name. Unfortunately, this method connects only thecode and detail view and there can be a lot of independentvariables with the same name. Therefore, we planned inaddition to render lines in order to connect a line in thefeedback view with its associated frame in the detail view.We perceived this approach in the version editor of Xcode.

These lines are only visible while the focus of the program-mer lies on a concrete frame or line. Through the alreadyexisting link between the code view and the feedback view,the link between the code view and the detail view is alsoimproved. By combining these features we wanted to es-tablish a transparent link between all three views.

Additionally, another problem is caused by using an in-Synchronizedscrolling between the

detail view and theother views makes

no sense

creased amount of space for each detail visualization andenabling the dragging of them. The scrolling betweenthe code and feedback view can be synchronized becauseof their line-to-line relationship. In contrast, synchroniz-ing the detail view with the others is problematic. Afterscrolling down to another frame in the detail view, multiplelines of the code and feedback view are no longer visible.

3.2 Initial Design Ideas 25

The other way around, when scrolling in the code or feed-back view, the same detail visualization stays in the field ofvision far too long.Xcode has a similar problem in its version editor. For exam-ple, whenever the newer version of a function is as twice aslong as the old one, synchronized scrolling can cause anundesired result. The same function could be still visibleon one side, while being already out of the field of visionon the other. This problem is solved by a simple mecha-nism. When scrolling in the version with the larger part A solution is to adapt

the speed of scrollingrespectively for eachview

of code (our detail view), scrolling in their version withshorter parts (our code and feedback view) is comparableslower and vise versa. We included this method in our de-sign, but unfortunately had not enough time to implementthis feature. Furthermore, whenever all lines of code asso-ciated with a detail visualization are no longer visible, thedetail visualization should be moved out of the way auto-matically to open space for other detail visualizations.

3.2.4 Grouping of Data Types and Their Detail Vi-sualizations

Higher level program languages always have multiple data We perform agrouping and splittingof the JavaScriptdata types andassign each group astandardvisualization

types. For some data types using the same detail visual-izations is reasonable and they can be grouped together,for others it makes sense to divide them into differentsubtypes. Before considering to design detail visualiza-tions, splitting and grouping of data types is advantageous.Thereby, an individual set of visualizations can be createdfor each group. Additionally, one of the visualizations ofeach group has to be selected as the standard visualiza-tion, which is created when a user wants to open a detailvisualization for a variable of this group. The visualization,which is most likely useful in common cases, should be se-lected.

JavaScript basically incorporates the following data types: Arrays are split intoone, two andmultidimensionalarrays

object, array, number, string, boolean, undefinedand null. We decided to create detail visualiza-tions for these different data types but to do a fur-ther distinction between strings and HTML, as well

26 3 Prototype

as between one dimensional, two dimensional andmultidimensional arrays. We distinguish betweenUndefined and null

are grouped together these types of arrays, because of their different grade ofnesting. Thus, there are diverse possibilities to visualizethem. Furthermore, we group undefined and null togethersince we do not add any visualizations for both of them.

Of course there could be done further distinctions, in caseof too many visualizations. For example we could divideour number visualization group into groups such as: plots,textual representations, bar charts and so forth. However,we do not need such a distinction, since our target is notto optimize our prototype for switching between hundredsof visualizations, as it would be necessary for an actual livecoding tool. We want to research how much detail visual-izations and the ability to switch between them assists pro-grammers with their programming tasks. For that we donot need such a high number of visualizations.We came up with visualizations for each of our selectedgroups, except for undefined, null and boolean and explainour design decisions for the visualizations of data types weimplemented in detail: number, object and HTML.

Number Visualization As shown by Victor [2011] multi-A more graphicalrepresentation,

illustrating morecontext is necessary

ple visualizations, with diverse levels of abstractions, existsfor a variable of type number. One possibility is a textualrepresentation, visualizing an exact value as the result of aterm, like it is done by Kurz [2013]. This visualization pro-vides the information whether a term returns the expectedresult. It hides information such as the general trend of thevalue, previous values or intermediate results within theterm. A different visualization, which includes more infor-mation about the context, is necessary.

Unfortunately, including context, such as the value changeswithin a loop, can induce space problems. We came up withtwo designs, showing information with different levels ofabstraction:

• A hierarchical list which contains all value changes ap-plied within the scope of a number variable. This vi-

3.2 Initial Design Ideas 27

Figure 3.6: The figure shows a hierarchical list divided into thethree columns: Iteration, Value and Line, containing all changesof a variable over its whole scope.

sualization provides the user with the information inwhich line a new value was applied, the exact valueand in case it happens within a loop, the number ofthe corresponding iteration. Therefore, as shown in A hierarchical list

visualizations dividedinto the columns:iteration, value andline

figure 3.6, the list is divided into three columns: iter-ation, value and line. In the iteration column the usercan hide unessential information or look at (nested)loops in more detail, by closing and opening of itera-tions. Examples for another possible use are the com-parison of the exact results after each iteration of aloop or the identification of an unintentional use of aglobal variable.So, we identify three important design decisionswhen using a hierarchical list: the shown content ofthe initial hierarchical list, the effects of opening andclosing loops and the order of the applied changeswithin the scope of the associated variable.

When a number visualization is opened by clicking

28 3 Prototype

on a value in the feedback view, which is assignedwithin a (nested) loop, all loops and iterations con-taining the assignment are opened automatically. Weassume that the context of this specific change is of in-terest for the user, since it was used to open the detailvisualization. Thus, we anticipate less necessary in-Trade-off between

necessary number ofuser interactions and

amount of potentialunnecessary

displayed information

teractions till the required information are displayed.

There is a trade-off relationship between the neces-sary number of interactions, to find the informationof interest, and the amount of potentially unnecessarydisplayed information. For example, when opening aclosed loop, it is possible to increase the amount of in-formation in one step more or less.One option is to visualize all applied changes duringWe decided for

higher amount ofinformation

each iteration. Another option is to visualize all itera-tions as closed loops by showing just the last assignedvalue after each incrementation. When using the sec-ond method, the user has to open the closed loops ofinterest on his own, potentially leading to more nec-essary interactions. To reduce the number of neces-sary interactions, we decided for the first option andaccepted a larger amount of information. A futureuser study is necessary to show if users prefer rathera large amount of information and less interactions orvice versa.

The last design decision is about the order of the ap-Order of executionpath vs order of

appearanceplied changes within the scope. There are two pos-sibilities to sort the results: order of execution pathand order of appearance. In this visualization the or-der of appearance is used, which yields the advantagethat all calls of a specific function and the resultingchanges to a variable are grouped together. Again, itDecided for order of

appearance is not clear whether this is the better choice or not anda user study is required to determine the preferableoption.

• A plot which also displays all value changes appliedA graphicalillustration in form of

a plotwithin the scope of a variable (see figure 3.7). Thisvisualization provides a more graphical view by plot-ting the development of a variable and its changingvalues, sorted in the order of appearance. Throughthis view the detection of outliers is simplified andthe programmer can check if the general trend looks

3.2 Initial Design Ideas 29

right. Such a graphical view could be useful, e.g., tocheck the implemented flight path of an object (gal-lons example in swift playgrounds displays a plot toillustrate the flight path of a zeppelin). We did notimplement this visualization yet.

One problem of this visualization is that the scale is Zoom controlenables user tochange the level ofabstraction andconsider specificparts

determined by the highest values on the y-axis andthe number of changes (x-axis). That bears the risk toprevent the user from getting any concrete values onspecific parts of the program. Therefore, we enablea zoom in both dimensions at the same time at onepoint of interest. This can be also used to look at aspecific intervals, such as a loop. Furthermore, theplot provides more concrete information, such as theline number or the exact value, by moving the mouseover a specific point on the plot. This method limitsthe amount of unnecessary information displayed atthe same time and the visualization is still capable ofproviding exact values and information.

Figure 3.7: A plot which contains all value changes of a variable.The values are located on the y-axis, the number of the currentchange is located on the x-axis. Each point in the plot marks achange. The magnifiers can be used to zoom in or out.

This visualization is just one of many possibilities.Especially for quantitative data (in our case num-

30 3 Prototype

bers) exist plenty of graphical representations such asquantile plots, histograms or visualizations for multi-dimensional data spaces. An example of a tool for thevisualization of multidimensional data spaces is Par-allel Coordinates introduced by Inselberg and Dims-dale [1990]. It represents multidimensional data in atwo-dimensional visualization. Plenty of other use-ful visualizations are presented by Tufte [2001] in hisbook ”The Visual Display of Quantitative Informa-tion”.

Object Visualization An object in JavaScript can containObjects can havedeeply nested

propertiesan unlimited number of properties of each data type. Forinstance it can contain, multidimensional arrays or objects,which possibly contain an object element on their own.Since it might be just one or two properties which are ofinterest for the programmer, it is important to limit theamount of displayed information. In addition, all top-levelproperties and nested ones are also variables. A program-mer can access them, change their value or pass them to amethod.

We came up with a hierarchical list visualization whereObjects arevisualized in form of

a hierarchical listeach property of type object can be opened in the sameframe. Properties of other variable types are, when opened,created in a complete new frame with the standard detailvisualization of their group. Each property is visualizedin a private line, containing the name of the property andFor all properties

except the ones oftype object an extra

detail visualizationcan be opened

either an exact value or a preview. A preview is used forproperties, such as arrays or HTML elements, which areopened in a new detail frame and can not be compressedto a specific value. It displays information about a propertywithout creating an extra detail view. One benefit of our vi-sualization is the possibility to look up all offered functionsand accessible variables of an object, without the necessityto use a documentation. That matches one of the principlesused for the Light Table: ”You should never have to lookfor documentation” [Granger].

Finally, we added one additional detail information to ourdesign. The first property of an object is a list, consistingof all variables which have a pointer to this object. This is

3.2 Initial Design Ideas 31

Figure 3.8: Visualization of an object in a hierarchical list form.The first property contains all variables which have a pointer atthe visualized object.

used to show the variables which are directly influenced bychanging the selected object (see figure 3.8).

HTML Visualization JavaScript was primarily created towork with HTML elements, to create and alter them. It canaccess the DOM elements of a website with a list of selec-tors. To simplify the adaption of HTML elements, jQuery4

was created. It is a powerful library which assists with se-lecting as well as changing elements and provides some ad-ditional functionality. The result of these changes becomesvisible in form of a changed website. It contains new, al-tered elements or additional functionality.

The process of website creation can be divided into a cy- HTML visualizationsimplifies thecreation of websitesand the positioning ofelements

cle of three steps: positioning or appending an elementor functionality, reloading the page and finally checkingwhether it is the desired result. The visualization can beused to control the appearance of elements and their posi-tion before appending them to a website, which is intended

4http://jquery.com/

32 3 Prototype

Figure 3.9: The figure shows on the left a XML website request. The resulting HTMLstring is assigned to a variable. On the right a frame and the detail visualization for theHTML string is illustrated (the webpage).

to accelerate the creation of websites.

Our HTML visualization enables the user to look at one orHTML strings arerecognized and

renderedmultiple elements immediately after they were created, byrendering them into their detail visualization frame. Evena whole website can be displayed (see figure 3.9). Also pos-sible is the usage of a CSS document for changing the styleof HTML elements.

Summary of Other Visualizations First, our string visu-alization is identical to our number visualization, exceptthat it shows string values instead of number values. Next,our array visualizations. One dimensional arrays are illus-trated as a list, where each array element is initially closed(not whole content is visible) and can be opened by the user.We display a preview for each array element. For two di-mensional arrays we have two possible visualizations: a hi-erarchical list and a matrix. For the hierarchical list we alsodisplay previews of the content of the arrays. Furthermore,to illustrate the number of dimensions and to simplify ac-cessing the shown values, we display colored brackets andcolor their corresponding indices the same. The multi di-mensional array is visualized as the same as a two dimen-sional array. It is the same except for a deeper level of nest-ing. Images of the other visualizations can be found in ap-

3.3 Preliminary User Study 33

pendix A.

3.3 Preliminary User Study

In her paper ”A Nested Model for Visualization Design and Munzner presents amodel ofvisualization creation

Validation” Munzner [2009] presents a model of visualiza-tion creation. It contains the following four nested layersin a descending order: domain problem characterization,data/operation abstraction design, encoding/interactiontechnique design and algorithm design. Each of these lay-ers represents one category which threads the validity ofa design. In our case, the domain problem characteriza-tion is pretty clear and refers to the task of simultaneouslyprogramming and debugging. The next two layers are ofinterest for the user study. In our case, they address thechallenge of using the correct data type for the right dataand for these data, appropriate visualizations.Before we start with the implementation we wanted to as- Conduction of a user

study to get somefeedback and assurethe potential of ourvisualizations

sure that our detail visualizations are meeting the require-ments of Munzner [2009]. Furthermore, we want to getideas for further improvements, expose confusing partsand show that there are actual situations where our visual-izations help to understand source code or solve a problem.

3.3.1 Setup and Tasks

To acquire brief feedback and some opinions on our visual-izations we conducted a qualitative user study. Therefore,we presented five people our concept and visualizations inform of low fidelity paper prototypes. The persons wereon average between the age of 22 and 27 years old. All ofthem used JavaScript once before but were not experiencedJavaScript developers.

The process was roughly the same for each participant. Wepresented them an incorrect source code sample, explainedwhat the sample originally was supposed to do and testedif they could solve the problem, or at least notice whichresults were buggy through our visualization. We asked

34 3 Prototype

them if there are not self-explanatory parts of the visual-Process consists of:presentation of code

scenario as well asour visualizationsand interviewing

them

izations or frame. Furthermore we asked for suggestionsfor improvement and let each participant explain our ownframe and visualizations to us. We used these explanationsto control, if the participants really perceived the visualiza-tions as we supposed them to. In other words, if our mentalmodel, the system image and the users mental model match[Norman, 2002]. Additionally, a couple of design problemswere exposed.

Due to the fact that all visualizations utilize their potentialFor each type ofvisualization wecreated an own

scenario

in different situations we came up with a different setupfor each group of visualizations. An object visualization forexample is not useful when there is an error within a loopconsisting of numbers. However, we describe only the sce-narios of visualizations we also implemented.When creating scenarios for the different visualizations, itis a challenge to keep them simple enough to be understoodeasily and in an adequate amount of time, but to provideat the same time situations where more detailed visualiza-tions than in Kurz’ prototype become necessary. Hence,Difficult to find

meaningful scenariosof a similar length as

well as complexity

the setups have a variable level of difficulty and a variableneeded amount of time to explain them. These differencespose no problems for the user study, since we do not wantto compare the visualizations and its assessment. Ratherwe want to get a short review for each one individually.

HTML Visualization Scenario We chose a simple setupfor the HTML visualization. A HTML image element whichuses the wrong CSS class is assigned to a variable. Thus, aplaceholder image was rendered into a detail frame insteadof the intended image in form of the Google logo.

Object/1-dim Array Visualization Scenario For the ob-ject visualization we implemented a small online shop-alike structure where one could add products to a sales list.The error was, that the same product could be added to thelist of sales (in form of an array) multiple times.

3.3 Preliminary User Study 35

Number Visualization Scenario We used a small base64encoding algorithm for the number visualizations. The al-gorithm of a base64 encoding is quite short and in our mindeasy to understand. More important, no specific knowl-edge of JavaScript is necessary to understand it. We im-plemented an incorrect exit condition for one used whileloop, which led to a wrong result of the encoding.

3.3.2 Results

All participants stated that the visualizations have poten- Participants believethe visualizationslook promising

tial and could prove as useful in programming tasks. How-ever, at the same time they pointed out some shortcomingsof our visualizations and partly suggested possibilities toimprove them. The number of shortcomings were quitedifferent for each visualization. Whereas the HTML visu-alization was not criticized at all, four of the participantsfound multiple flaws in the number visualizations.

First, the illustration of the location (line) for each initial Illustration of the lineof the initial loopdeclaration isirritating

loop declaration in the hierarchical list visualization is con-fusing. Second, all of the participants criticized the designof an opened loop. The privation of a button to close theloop again at only the line of the first iteration and not at alliterations irritated them. Two of these participants also re-marked that they expect the visualization to automaticallyopen all loops which contain the line used to open the detailvisualization. This was fortunately exactly as we designedit but since we presented them only a paper prototype thatbehavior was not observable. When we presented our plotvisualization, most of them remarked the absence of exactvalues. They expected the visualization to provide in addi- Hierarchical list is the

preferred standardvisualization

tion to the value and the line, also the number of iteration(when the mouse is moved over a specific point in the plot).All of the participants preferred the list visualization as thestandard visualization. They mentioned that in their opin-ion the plot is less suitable for a majority of situations.

For our menu bar we also got some suggestions for im- The navigation iconsare notself-explanatory

provement. It was not obvious for the participants that thetwo arrows on the leftmost side of the menu-bar could be

36 3 Prototype

used as a navigation. None of them understood the use ofthese arrows at first sight.

The last point of criticism was about our design decisionParticipants want toopen a new frame

through interactionwith the code view

to initialize the creation of a visualization by clicking on aline of interest in the feedback view. Four of five partici-pants wanted to be able to open it somehow by interactingwith the code view. They did not find it obvious to use thefeedback view for that.

3.4 Final Design and Applied Changes

By using the results of our preliminary user study (see sub-section 3.3.2) we changed and improved our design of theframe, the object and both number visualizations.Most of the participants wanted to be able to use a line inAppending the

opening of a newframe to the code

view is possible byusing the right click

menu of the codeview or offering of an

extra method

the code view to open a detailed visualization. Moreover,they thought that there is no visual implication that open-ing a detailed visualization is possible by clicking on a linein the feedback view. To solve the problem of no visual im-plication, the image of the mouse cursor is changed, fromthe standard HTML cursor to a pointer cursor, wheneverthe mouse is moved over a line in the feedback view. Thereare multiple options where to add the functionality to openframes in the detail view. Opening a new frame wheneverthe user clicks on a line in the code view is not an option.Programmers click around source code frequently, to cor-rect or add something. This method would lead to numer-ous of not required open frames. We came up with two so-lutions: append it as an additional option to the right clickmenu of the code view or implement an extra method suchas openDetailView(varName, line). For now, we added thefunctionality just at the feedback view and eventually willadd it to the code view later on.

3.4.1 Frame Design

We did not change anything at the frame concept exceptfor the menu-bar. Additionally, we refined the style of the

3.4 Final Design and Applied Changes 37

Figure 3.10: The new frame and menu design. The filter menu on the right, the optionmenu on the left. The new option menu contains only the visualizations which are availablefor a specific data type.

frame. First, we replaced the icons for navigation through Frame conceptstayed the same butwe changed the styleof a frame andeliminated thevagueness

a history site navigation box as visible in figure 3.10. Thisprovides more information on the currently displayed con-tent as well as past one. This design should be more self-explanatory than the two arrow icons. Additionally, wechose to provide an individual option menu for each datatype group.

3.4.2 Detail Visualizations

As you can see in figure 3.11, we adapted the followingparts of our hierarchical list visualization for numbers: weadded possible interaction to all iterations of an openedloop, removed the number of the line which illustrates thebegin of loops and added a line coloring to simplify thedifferentiation of the different lines in the list as well astheir contained values.

38 3 Prototype

Figure 3.11: Hierarchical list after opening it by clicking on a number value of interestwithin a loop. All loops containing the line of value change are opened automatically.

Figure 3.12: Hierarchical list of an object. Each property has a preview of its value or thewhole value. Object properties are opened within this frame, changing the visualization.The other properties have a frame icon and can be clicked to open a detail view for them in anew frame

3.4 Final Design and Applied Changes 39

In the object visualization (see figure 3.12) we removed thelist of pointers till we find a better way to visualize thatwithout using to much additional space. Finally, we addedthe same line coloring to this visualization as applied forthe hierarchical list visualization for numbers.

40 3 Prototype

41

Chapter 4

Implementation

In this chapter we present the architecture of Kurz’ applica- We present ournewly addedcomponents and theexisting foundation ofthe prototype

tion, the architecture of our enhancement as well as its com-ponents and the integration into the old prototype. Sincewe enhance the prototype developed over multiple itera-tions by Heinen [2012], Belzmann [2013] and Kurz [2013],there are components we have to access in order to add ourdetail view and its visualizations. For example, we use theavailable structure which contains the feedback data fromthe server or the update function for the views. By explain-ing the architecture our prototype bases on, we get a gen-eral understanding of how the program works.After giving an overview about our general concept, itscomponents and their relation to each other, we give moredetails about the management of frames and our detail vi-sualizations.

4.1 Existing Foundation for Our Prototype

The first version of our live coding plugin for Bracketswas developed by Heinen [2012]. His prototype uses aloop in combination with the intern eval function ofJavaScript, which evaluates and executes a given argument,to create live feedback. Next, Belzmann [2013] improvesthe capability and performance of the system by discard-

42 4 Implementation

ing the idea of using the intern eval function of JavaScript.Instead, his plugin uses a client-server architecture. Fi-nally, Kurz [2013] modified the client by adding compo-nents, such as the LiveDataView, LiveDataController andLiveDataPane, implementing the feedback view and adapt-ing the backend to remove some limitations.

All these modifications led to the current architecture:

The basic concept is the division of the program into a clientA client-serverarchitecture is used and a server, called backend. The client continually sends

a whole JavaScript file to the server and takes care of thevisualization process.Therefore it consists of the major components: Main, Live-DataView and LiveDataController, as well as multiple subcomponents, such as LiveDataPane and LiveCodingEvalu-ator. Each component is exactly one JavaScript file. Mostof the components are plugged together by Main. It hasaccess to the major components, creates the LiveDataCon-troller and tells it, e.g., to use the LiveDataView as the dis-play.The LiveDataView and LiveDataController are followingKurz implemented

LiveDataView andLiveDataControlleras a Model-View-

Controllerpattern

the concept of the Model-View-Controller pattern. TheLiveDataController decides what content to display, whento update the content and talks to the LiveCodingEvalua-tor, which receives the messages from the server, in orderto obtain all necessary data. Afterwards, the LiveDataCon-troller forwards the received data to the LiveDataView.The LiveDataView defines for each piece of content its po-The

LiveDataControllermanages the contentto display and when

to update the contentwhile LiveDataViewmanages how and

where to display thecontent

sition and representation. It displays error messages, cre-ates the line-to-line layout and decides for each line thedisplayed content as well as its appearance. Furthermore,it attaches event handler for possible interactions and up-dates the view whenever the controller tells it about an oc-curred change. To sum up, every feedback and additionalinformation is visualized by the LiveDataView. The compo-nent LiveDataPane supports LiveDataView, by synchroniz-ing the scrolling between the two views as well as addingnew views (feedback view).

The server instruments and executes the code and finallysends the results (runtime state of the program) back to theclient. Whenever new code is received by the controller

4.1 Existing Foundation for Our Prototype 43

component, it communicates with the client, it spawns anew child process in form of an ExecutionManager. An The server uses child

processes to executenewly received code

ExecutionManager instruments the code, creates a sandboxand spawns two new child processes on his own. One childprocess generates a source map which maps from the in-strumented code to the original position in the source codeand vice versa. The source map is necessary because un-caught exceptions can falsify registered source code loca-tions. The other process executes the instrumented code.By creating a new child process, each time new code is re- One child process

generates a sourcemap and oneexecutes the code

ceived, the crash of the whole program can be prevented.Usually, in case the executed code has an infinite loop ordoes not stop for other reasons it would be necessary toshut down the whole program to fix it. However, sinceeach child process can be shut down independently, with-out influencing the client, the actual program remains re-sponsive.

Figure 4.1: The picture presents the server architecture of Kurz’ prototype. The controllercommunicates with the client through a WebSocket protocol and receives its messages. Incase of newly received source code it spawns a new process named ExecutionManager. TheExecutionManager also spawns new child processes, one for the generation of a source mapand one for the code execution.

The whole client-server communication, its message struc- Communication, inform of JSON strings,is established via aWebSocket protocol

ture and setup, is an important part of the architecture.Communication between the client and the server is estab-lished by using a WebSocket protocol. Most messages aresent in form of JSON strings and contain code, but there arealso control type messages to control the server.

44 4 Implementation

{type: "code",code: code,[options: options object],id: id

}

The messages send by the server are mostly messages aboutsource code changes. The messages contain informationsuch as the number of iterations of a loop, a result of aterm or the associated line and column in the source code.However, it can also send control messages such as errormessages. A more detailed description of the client-servercommunication, the content and structure of the messagesand diverse server tasks can be found in Belzmann [2013].Plenty of functionalities, such as the instrumentation ofsource code or usage of multiple processes, are realized bythird party modules. The most important ones are the fol-lowing: ws, escodegen, esprima, contextify and child process.All of them are modules of the platform Node.js, which canUse of Node.js and

its modules e.g. forthe client-server

architecture and childprocesses

be used for server-side as well as network applications andis built on Chrome’s JavaScript runtime. Ws (WebSocket)enables the communication to the client. Esprima is usedto parse source code into an AST (abstract syntax tree). Es-codegen can generate code out of the instrumented AST aswell as a source map. Contextify enables the execution ofsource code in a sandbox and child process to spawn newchild processes like it is done by the controller componentof the server.

These components build the foundation for our prototype.Most neededinformation for our

detail view arealready provided by

the foundation

They provide almost all important data we need for ourdetail view and its visualizations. The messages contain,e.g., the results of terms, the corresponding lines in the codeview or the number of iterations we need for our visualiza-tions. LiveDataView and LiveDataPane provide us with afoundation where we can attach our detail view and imple-ment functionality such as the scrolling or updating of ourvisualizations.

4.2 Newly Added Components 45

4.2 Newly Added Components

To realize our detail view and visualizations we added fur- Use of further thirdparty modules suchas escope.js forgetting all scopes ofan AST

ther third party modules: escope, estraverse and htmlTok-enizer.Estraverse is used in connection with esprima and escope.Through estraverse it is possible to move and search troughan AST. The AST is created by the esprima parser. By us-ing escope we acquire all scopes and can filter the scope ofspecific variables, which is necessary for our number vi-sualization. In addition, we believe that it could be prettyuseful for future visualizations and changes. Lastly, htmlTo-kenizer helps to detect errors in HTML elements. Since weadd a third view to the surface and are interested in visual-izing existing data, we mainly adapt, extend and make useof the components LiveDataView, LiveDataPane and Live-DataController.Furthermore, we implement our own components to createan architecture where it is possible to easily add an own vi-sualization and adapt the functionality of frames as well asvisualizations. We add the following components to the ex-isting foundation: FrameManager, FrameFunctionality, De-tailVisManager as well as the various visualizations: Num-berVisualization, ObjectVisualization and HTMLVisualiza-tion.

We create our third view and implement the scroll func- Adaption of theupdate and otherfunctions inLiveDataView for ourthird view

tions and resize functions for the widths of the views withinthe LiveDataPane.In the LiveDataView we implement all functions concernedwith the creation of a detail frame: mostly creating HTMLelements, adding classes to them and setting a CSS class).In short, almost everything that changes the appearanceof the interface. In this component we had to adapt somemethods, such as the update function, to enable the im-mediate update of our detail visualizations after each ap-plied change. LiveDataView

requires functionalityfrom FrameManagerandDetailVisManager

First, the position of the associated line in the code viewand feedback view can possibly have been changed ordeleted with each update. Henceforth, we check if the asso-ciated line to each detail frame is still at the same position.In case the position changed or the line was deleted, we

46 4 Implementation

search for our associated line in a certain radius. Second,the data of our line can possibly have changed. Thus, eachopen detail frame compares their current data with the newones, whenever the function is called. In case of a differ-ence, the visualization is adapted automatically.Additionally, LiveDataView is the link to the newly addedcomponents: FrameManager and DetailVisManager. By us-ing these components we separate large sections of the de-tail view functionality from the architecture of Kurz’ pro-totype. Thus, new functionality can be added more easilyand changing the behavior of the detail view and its ele-ments becomes easier.

Figure 4.2: The figure shows the LiveDataView as well as our newly added components andtheir relationship to each other. LiveDataView is our major interface to the already existingarchitecture. It requires DetailVisManager as well as the FrameManager and therefore hasaccess to all their exported methods. On the right sight the FrameManager is shown. Itrequires and exports all functionality provided by FrameFunctionality. The FrameFunc-tionality requires the FrameManager since it needs the global frame attributes and func-tionality. On the left side is the DetailVisManger, which requires all DetailVisualizationsand exports a method called createDetailVisualization. Each detail visualiza-tion has to require the FrameManager to keep it informed and to provide information suchas the used data for the visualization. Each DetailVisualization has the possibility to requirethe LiveDataView to use its offered functionalities which can assist with the creation of anew visualization.

We offer further functionality the programmer can useto create his own visualizations. First, as already men-tioned we use escope to get all existing scopes of the cur-rently opened JavaScript file. We then implemented a

4.3 Frame Management 47

function called getScopeOfVariable. This method iscalled with a line number and a variable name as pa-rameters. It determines the whole scope of the cho-sen variable. Second, we implemented a method calledgetVarValuesLinesDepth. This method provides in-formation on the changes of a variable within its wholescope. For example it contains all applied value changesand their corresponding lines in the code view. How-ever, this method does not work for variables which arecreated and incremented in the header of a loop declara-tion. In addition, there is other functionality such as themethod getLineDataForAllIterations, which pro-vides for all iterations the results at a certain line in thefeedback view, respectively code view.

4.3 Frame Management

The FrameManager contains all global attributes which are FrameManagercontains global frameattributes andaccesses andforwards allfunctionality fromFrameFunctionality

used to manage the frames, such as an unique identifierfor each frame. It has an up-to-date list of existing frames,checks for overlaps and has access to all functionality pro-vided by FrameFunctionality. Most of these functions arejust forwarded to the LiveDataView. Furthermore, he hasinformation about each frame such as the currently useddata and whether the visualizations refers to multiple linesof code or just one line.The component FrameFunctionality exists as a place forall additional frame functionality a programmer wants toadd. We did this separation of the FrameManager andthe FrameFunctionality to clearly separate the managementand global attributes from the set of additional function-ality. Additionally, it limits the amount of source code tocheck in case of an undesired behavior.

4.3.1 Frame Creation

The creation of a frame is initiated by clicking on a line in Creation of frames istriggered inLiveDataView

the feedback view. In LiveDataView all necessary HTMLelements for a new frame are created: the menu bar and its

48 4 Implementation

menus, as well as the extra space for the detail visualiza-tion. All methods and global attributes necessary for theframe creation, to guarantee uniqueness or to register to anevent handler, are provided by the FrameManager and in-directly by the FrameFunctionality. During the creation ofa frame, the LiveDataView checks the data type of the con-cerned variable. With that knowledge it orders the Detail-VisManager to create the detail visualization for the frame.Here we also distinguish between the data type groups wedefined in section 3.2.4.

4.4 Visualization Management

The DetailVisManager manages all defined data typeAll visualizations areregistered at the

DetailVisManagergroups and their visualizations. It knows of all detail vi-sualization files and can use them to create a new detail vi-sualization. The DetailVisManager also offers this function-ality to the LiveDataView. We created the DetailVisMangerwith the target to simplify the adding, deletion and changeof visualizations, without the need to apply considerablychanges in other components. It also offers the functional-ity to change the currently used standard visualization ofeach group.

4.4.1 Data Type Dependent Visualization

Each visualization is represented by an own componentwhich can consists of multiple JavaScript files. Yet, one file,which can be required by the DetailVisManager and is ableto control the whole visualization process, is necessary. Incase a programmer wants to add his own detail visualiza-tion, he can implement it and add it to its belonging group.The same visualization can be possibly added to multiplegroups. The DetailVisManager has an array variable foreach visualization group which contains all the visualiza-tions belonging to this group. If a detail visualization is re-quested by the LiveDataView, it searches for the concernedgroup, get its current standard visualization and starts thecreation of that visualization.

49

Chapter 5

Discussion of ourPrototype

5.1 Capabilities and Advantages

Our enhancements of the prototype by Joachim Kurz intro-duces new capabilities and advantages. First, in case theprovided feedback of the second column is not sufficient,the programmer has now the possibility to open a detail vi-sualization which is located in the third column. He canperceive feedback in other forms than textual representa-tions, which can illustrate additional and contextual infor-mation.By displaying each visualization in its individual frame, weenable the programmer to adapt each visualization with-out influencing the others. At the same time it ensures aclear separation between the open visualizations. In ad-dition, these frames can be dragged to other positions toenable a better comparison between visualizations and theprogrammer is able to adapt the size of each frame to hisrequirements.Another advantage of our prototype is that the user canchoose in each situation the visualization which fits thebest. For the case that no representation fulfills the require-ments, the foundation of our prototype offers a program-mer to implement his own visualizations and add them to

50 5 Discussion of our Prototype

the existing sets.We provide functionality to simplify the implementa-tion of new visualizations, such as a method calledgetScopeOfVariable which returns the whole scope ofa variable.Finally, we added the live programming feature to ourdetail visualizations. Whenever parts of source code arechanged, our detail visualizations check for differences andadapt accordingly. In addition, whenever the program-mer clicks through the iterations of a loop, the affectedvisualizations update their content. If the programmer isnot satisfied with the standard update behavior after codechanges or the iteration of a loop was changed, he can usehis own implemented update functions for his visualiza-tions.

5.2 Limitations and Shortcomings

Our enhanced prototype has a couple of limitations. Someof these limitations we took over from Kurz’ [2013] proto-type. First, the application can only capture console outputbut no input. Second, feedback is only provided for linesconsisting of an assignment. Furthermore, the plugin doesonly work with Node.js compatible programs. For moredetails about these see Kurz’ [2013] and Belzmann’s [2013]theses.

Our enhancements introduce some new limitations. First,Changing the typegrouping is

cumbersomethe programmer has to use our grouping of data types. Thegrouping can not be enhanced by just adding a new group.Changes to parts of the DetailVisManager, LiveDataViewas well as all detail visualizations, which use our currentgrouping, are inevitable.Second, the prototype has problems with multiple assign-Detail visualizations

for multipleassignments in oneline are not possible

ments in one line. Due to the concept that each line hasexactly one attached frame, detail visualizations are onlypossible for one variable of each line. Currently, when theuser initiates the creation of a frame, we check if a framealready exists for that line.To open multiple detail visualizations for one line in thefeedback view, we have to save the position of each value

5.2 Limitations and Shortcomings 51

in that line and adapt the identifiers accordingly.One shortcoming, already mentioned by the participants of User interaction to

open a detailvisualization is onlypossible in thefeedback view

our preliminary user study, is the absence of possibilitiesto open visualizations by interacting with lines in the codeview. This has nothing to do with our architecture and canbe added later on without necessary changes to it. We al-ready proposed possible solutions for that in section 3.4.Another limitation is the necessity to reload the extension,every time an user changes the currently shown JavaScriptfile. In the course of events our attached resize and draghandlers become partly inoperative.Further, our number visualization has some problems andin some cases it does not work appropriate.Finally, visualizing a representation which contains a hugeamount of data leads to performance issues. Visualizationssuch as our object and HTML ones typically pose no prob- Illustration of a large

amount of data andusage of many eventhandler leads toperformance issues

lems, cause they refer to the data of one assignment andnot to the data of the whole scope. In contrast, our num-ber visualization consists of all value changes within thewhole scope of a variable. There is a performance degra-dation due to the necessity to collect all data as well as bythe creation of the visualization itself. Moreover, each eventhandler attached to the visualizations decreases the perfor-mance. For example, the creation of a visualization for avariable within a nested loop of depth 2, where the outerloop has 100 and the inner one 10 iterations, already takesa few seconds and there is a small delay when interactingwith it. However, the performance of our prototype shouldbe sufficient for a user study. We describe a potential futureuser study in 6.2.

53

Chapter 6

Summary and FutureWork

In this chapter we shortly summarize our results and con-tributions. Afterwards, we present possible future work,possible improvements for the live coding editor and a po-tential future user study.

6.1 Summary and Contributions

First, by considering multiple live coding prototypes and Gathered ideas,designed our layoutas well as detailvisualizations andfinally implementedthe detail view andthree visualizations

tools, we gathered design ideas and guidelines for our visu-alizations. Moreover, we figured out some requirements forour detail view by looking at Kurz’ [2013] prototype. After-wards, we grouped and split the data types of JavaScript.We then came up with designs of detail visualizations forall these groups and a layout for our extension. We decidedto use a three column layout, where each visualization be-longs to its own unique frame.Furthermore, we came up with an use case for each groupand conducted a small qualitative user study by using thesescenarios. After we got feedback and improved our design,we enhanced the prototype by Kurz, implemented the de-tail view as well as three detail visualizations and extendedthe live coding functionality to our frames and their detail

54 6 Summary and Future Work

visualizations. We implemented our extensions in a struc-ture that enables a programmer to enhance the prototypewith his own visualizations and provide functionalities toassist him. In addition, the user can switch between visu-alizations of the same group to find a suitable one for eachsituation. Also interactions, such as filtering of information,are possible. Finally, we described limitations we inheritedfrom the previous prototype as well as limitations of ourextensions.

Our main contributions are:

• Determination of the requirements for our prototypeby looking at the old one, as well as collection of de-sign ideas by considering other live coding tools andprototypes.

• Creation of visualizations for all data type groups andconduction of a preliminary qualitative user study forthese designs.

• Implementation of the detail view, the live coding ca-pability for detail visualizations, our final detail vi-sualizations and the foundation for further visualiza-tions.

6.2 Future Work

There are still possible improvements and tasks for our pro-Diverse possibilitiesfor enhancement totype which have not been realized yet. The capabilities

and performance of the prototype can be improved further,issues need to be fixed and user studies are yet to conduct.First, numerous designs for visualizations are yet to imple-ment. Second, there are still parts of the application whichcan be enhanced.

The participants of the user study wished to be able toRest of designedvisualizations and

functionalities needto be implemented

initialize the creation of a detail visualization by interac-tion within the code view. Therefore, we already presentedsome design ideas in section 3.4. These ideas have not beenimplemented yet. There are other functionalities we in fact

6.2 Future Work 55

designed but still not integrated in the new version of theprototype: the navigation, dimension reduction of arrays,recognition of changes in the data type of a variable withinits scope, lines from the feedback view to a frame to em-phasize the link (see section 3.2.3) as well as the plannedscrolling behavior between the detail view and the otherviews.Furthermore, while observing the Light Table 1 we came up Adding of the real

documentation to theobject visualization

with another advancement of the object visualization. Wewant to add the available online documentation, containingfunction descriptions and variables, to the object visualiza-tion.

Another issue is the performance of the program. There Outsourcing ofresource-demandingtasks to the server

are diverse ways to improve the performance of our addedcomponents. First, functionality, such as the determinationof the scope, could be handled by the server. Thus, the ap-plication would stay responsive while child processes onthe server are occupied with these tasks. Another task,which is resource-demanding and could be done by theserver, is the filtering of one specific scope out of the wholeset of scopes.A further possibility to increase the performance is to refac- Reduction of the

number of jQueryoperations

tor the whole source code by minimizing the number ofnecessary traverses through the Document Object Model(DOM) as well as jQuery operations in general. These areresource-demanding.

Finally, it is essential to test our enhanced prototype aftermore refinements and implementing further visualizationsin a more defined user study than our preliminary one (see3.3 “Preliminary User Study”). For example, it is not veri-fied yet if there are situations where our visualizations im-prove the debugging time and accuracy. We already cameup with an idea for a potential user study. A combinationof a screen and a video recoding can be used to collect asmuch data as possible.Moreover, more data can be achieved by letting each partic-ipant fill out a questionnaire for each visualization. Ques-tions of interest are for example, how useful the partici-pants found the visualizations, the amount of illustratedinformation or what kind of functionality is still missing

1 http://www.lighttable.com/

56 6 Summary and Future Work

or not necessary. Parallel to the screen recording, we couldmeasure the task completion time and how frequently thedifferent UI elements are used. UI elements of interestcould be the offered functionalities of our frame and ourdetail visualizations. An additional idea is to measure thenumber of interactions with each detail visualization. Thatcould show whether the visualizations need too many userinteractions and whether we have to reduce or increase theamount of displayed information. As in the preliminaryqualitative user study with paper prototypes, a differentsetup for each group of visualizations is advisable. For ourimplemented visualizations we already have some ideas.In the scenario for the HTML visualization the participantscould create a prescribed website or adapt an existing one.Here we can test how much faster participants can placenew elements and change their representation with thehelp of our live coding tool in comparison to those with-out. For testing our object visualizations, the participantsUse for each

visualization anindividual scenario

could develop a small program by using an unfamiliar API.In this way we can test whether our object visualization as-sists with the task of understanding a new API and withthe usage of all its offered functionalities. In addition, it is acommon task to use new unfamiliar libraries and it enablesus to check whether the participants need to pay less atten-tion to the actual documentation.Finally, an idea for an implementation task for number vi-sualizations could be, e.g., a Base64 encoding. When im-plementing this, number variables needs to be changed atdiverse locations of the program and within a loop overmany iterations. Another suggestion is to use a task wherethe user has to simulate more graphical events such as themovement and curves of a roller coaster.

57

Appendix A

Not Implemented DetailVisualizations beforeUser Study With NewFrame

58 A Not Implemented Detail Visualizations before User Study With New Frame

Figure A.1: A list containing all elements of the array. We illustrate the number ofthe elements at the top. Each element can be opened on its own and a preview isdisplayed.

59

Figure A.2: A two dimensional array in form of a hierarchical list. The coloredbrackets show how to access the visible elements and the dimension of the array.Again, for each element a preview is displayed.

Figure A.3: An open two dimensional array in form of a hierarchical list.

60 A Not Implemented Detail Visualizations before User Study With New Frame

Figure A.4: A two dimensional array in form of a matrix. For the use of the bracketslook at figure A.2

Figure A.5: A multidimensional array. The form of representation is the same as infigure A.2

61

Figure A.6: A table which consists of the three columns: Iteration, Value and Line.Has the exact layout as the number visualization before we improved and refinedit.

62 A Not Implemented Detail Visualizations before User Study With New Frame

Figure A.7: Our already presented plot visualization with the new Frame.

63

Bibliography

Ewgenij Belzmann. Utilization and visualization of pro-gram state as input data in a live coding environment.Diploma thesis, RWTH Aachen University, Aachen,April 2013.

Joel Br, Vignan Pattamatta, William Choi, Ben Hsieh, andScott R. Klemmer. Rehearse: Helping programmersadapt examples by visualizing execution and highlight-ing related code, 2010.

William Choi, Joel Brandt, and Scott R. Klemmer. Rehearse:Coding interactively while prototyping, 2008.

Jonathan Edwards. Example centric programming. SIG-PLAN Not., 39(12):84–91, December 2004. ISSN 0362-1340. doi: 10.1145/1052883.1052894. URL http://doi.acm.org/10.1145/1052883.1052894.

John D. Gould. Some psychological evidence on how peo-ple debug computer programs. International Journal ofMan-Machine Studies, 7(2):151 – 182, 1975. ISSN 0020-7373. doi: http://dx.doi.org/10.1016/S0020-7373(75)80005-8. URL http://www.sciencedirect.com/science/article/pii/S0020737375800058.

Chris Granger. Light table. URL https://www.kickstarter.com/projects/ibdknox/light-table.

Philip J. Guo. Online Python Tutor: Embeddable web-based program visualization for CS education. In Pro-ceeding of the 44th ACM Technical Symposium on Com-puter Science Education, SIGCSE ’13, pages 579–584, NewYork, NY, USA, 2013. ACM. ISBN 978-1-4503-1868-6.

64 Bibliography

doi: 10.1145/2445196.2445368. URL http://doi.acm.org/10.1145/2445196.2445368.

Bjorn Heinen. A live coding editor. Bachelor’s thesis,RWTH Aachen University, Aachen, December 2012.

Peter Henderson and Mark Weiser. Continuous execution:The visiprog environment. In Proceedings of the 8th In-ternational Conference on Software Engineering, ICSE ’85,pages 68–74, Los Alamitos, CA, USA, 1985. IEEE Com-puter Society Press. ISBN 0-8186-0620-7. URL http://dl.acm.org/citation.cfm?id=319568.319582.

Alfred Inselberg and Bernard Dimsdale. Parallel coordi-nates: A tool for visualizing multi-dimensional geome-try. In Proceedings of the 1st Conference on Visualization’90, VIS ’90, pages 361–378, Los Alamitos, CA, USA,1990. IEEE Computer Society Press. ISBN 0-8186-2083-8. URL http://dl.acm.org/citation.cfm?id=949531.949588.

Andrew J. Ko and Brad A. Myers. Designing the why-line: A debugging interface for asking questions aboutprogram behavior. In Proceedings of the SIGCHI Confer-ence on Human Factors in Computing Systems, CHI ’04,pages 151–158, New York, NY, USA, 2004. ACM. ISBN1-58113-702-8. doi: 10.1145/985692.985712. URL http://doi.acm.org/10.1145/985692.985712.

Joachim Kurz. Evaluating developer strategies in a livecoding environment. Master’s thesis, RWTH AachenUniversity, Aachen, August 2013.

Sean McDirmid. Living it up with a live program-ming language. SIGPLAN Not., 42(10):623–638, Oc-tober 2007. ISSN 0362-1340. doi: 10.1145/1297105.1297073. URL http://doi.acm.org/10.1145/1297105.1297073.

Sean McDirmid. Usable live programming. In SPLASHOnward! ACM SIGPLAN, October 2013. URLhttp://research.microsoft.com/apps/pubs/default.aspx?id=189802. To appear.

Tamara Munzner. A nested model for visualization de-sign and validation. IEEE Transactions on Visualization

Bibliography 65

and Computer Graphics, 15(6):921–928, November 2009.ISSN 1077-2626. doi: 10.1109/TVCG.2009.111. URLhttp://dx.doi.org/10.1109/TVCG.2009.111.

Donald A. Norman. The Design of Everyday Things. BasicBooks, New York, reprint paperback edition, 2002. ISBN0-465-06710-7.

David Saff and Michael D. Ernst. An experimental eval-uation of continuous testing during development. SIG-SOFT Softw. Eng. Notes, 29(4):76–85, July 2004a. ISSN0163-5948. doi: 10.1145/1013886.1007523. URL http://doi.acm.org/10.1145/1013886.1007523.

David Saff and Michael D. Ernst. An experimental evalu-ation of continuous testing during development. In IS-STA 2004, Proceedings of the 2004 International Symposiumon Software Testing and Analysis, pages 76–85, Boston, MA,USA, July 12–14, 2004b.

James L. Snell. Ahead-of-time debugging, or programmingnot in the dark. In Proceedings of the 8th International Work-shop on Software Technology and Engineering Practice (STEP’97) (Including CASE ’97), STEP ’97, pages 288–, Wash-ington, DC, USA, 1997. IEEE Computer Society. ISBN0-8186-7840-2. URL http://dl.acm.org/citation.cfm?id=829539.831969.

Steven L. Tanimoto. Viva: A visual language forimage processing. Journal of Visual Languages andComputing, 1(2):127 – 139, 1990. ISSN 1045-926X.doi: http://dx.doi.org/10.1016/S1045-926X(05)80012-6.URL http://www.sciencedirect.com/science/article/pii/S1045926X05800126.

Edward R. Tufte. The Visual Display of Quantitative Infor-mation. Graphics Press, second edition, 2001. ISBN0961392142.

Bret Victor. Ladder of abstraction, 2011. URL http://worrydream.com/#!2/LadderOfAbstraction/.

Bret Victor. Learnable programming, 2012a. URL http://worrydream.com/#!/LearnableProgramming/.

Bret Victor. Inventing on principle, 2012b. URL http://vimeo.com/36579366/.

66 Bibliography

E. M. Wilcox, J. W. Atwood, M. M. Burnett, J. J. Cadiz, andC. R. Cook. Does continuous visual feedback aid debug-ging in direct-manipulation programming systems? InProceedings of the ACM SIGCHI Conference on Human Fac-tors in Computing Systems, CHI ’97, pages 258–265, NewYork, NY, USA, 1997. ACM. ISBN 0-89791-802-9. doi:10.1145/258549.258721. URL http://doi.acm.org/10.1145/258549.258721.

67

Index

Base64 Encoding, 35

Code View, 16Continuous Feedback, 6

Debugging, 5Detail View, 20

Feedback View, 16Future Work, 54–56

Independent Frame Concept, 21Inline Visualization, 6IPython Notebook, 10

jQuery, 31

Kurz Prototype - Architecture, 41–44- Backend, 42- Client-Server model, 42- Intern Eval Function, 41- Model-View-Controller, 42- Sandbox, 43- Source Map, 43

Level of Abstraction, 9Light Table, 9Line-to-Line Relationship, 24Live Coding, 1

Mental Model, 34Message structure, 43Model of Visualization Creation, 33

Order of Appearance, 28Order of Execution, 28Original Programming Cycle, 1

Paper prototype, 33Parallel Coordinates, 30

68 Index

Potential User Study, 55Probing, 7

Qualitative User Study, 33Quantitative Data, 29

Rehearse, 6–7- Backtracking, 6

Scope, 26Swift Playgrounds, 11–12

- Assistant Editor, 11- Sidebar, 11- Source Editor Window, 11

Third Party Modules, 44–45- AST, 44- Child process, 44- Contextify, 44- Escodegen, 44- Escope, 45- Esprima, 44- Estraverse, 45- HTMLTokenizer, 45- Ws (WebSocket), 44

Three Column Layout, 18Tracing, 7Type Dependent Visualizations, 25–26

- Grouping, 25- Splitting, 25

Visual Programming Language, 6- VIVA, 6

Whyline, 5

YingYang, 7

Typeset September 30, 2014