Preprint

Many Views Are Not Enough:Designing for SynopticInsights in Cultural Collections

F. Windhager, S. SalisuDanube University Krems

R. A. Leite, V. Filipov, S. MikschVienna University of Technology

G. Schreder, E. MayrDanube University Krems

Abstract—Cultural object collections attract and delight spectators since ancient times. Yet, theyalso easily overwhelm visitors due to their perceptual richness and associated information.Similarly, digitized collections appear as complex, multifaceted phenomena, which can bechallenging to grasp and navigate. Though visualizations can create various types of collectionoverviews for that matter, they do not easily assemble into a ”big picture” or lead to an integratedunderstanding. We introduce coherence techniques to maximize connections between multipleviews and apply them to the prototype PolyCube system of collection visualization: With map,set, and network visualizations it makes spatial, categorical, and relational collection aspectsvisible. For the essential temporal dimension, it offers four different views: superimposition,animation, juxtaposition, and space-time cube representations. A user study confirmed thatbetter integrated visualizations support synoptic, cross-dimensional insights. An outlook isdedicated to the system’s applicability within other arts and humanities data domains.

CULTURES COLLECT objects, achievements,and practices deemed worthy of preservation dueto their aesthetic, historic, scientific, or socialvalue. Modern societies thus incorporate a wholesystem of GLAM institutions (i.e., galleries, li-braries, archives, and museums), dedicated tothese preservation practices and to the purpose ofcollective learning and intergenerational transmis-sion. Recently, digitization is providing new ways

and means for this endeavor, but the basic cog-nitive challenge for visitors and spectators staysthe same: Cultural collections are immensely richobject assemblies of high perceptual complexity,which are further connected to large amountsof additional information and which are fre-quently condensed into relatively small exhibitionspaces. Computer screens—whether stationary ormobile—make no exception. Visitors to such

IEEE Computer Graphics and Applications Published by the IEEE Computer Society c© 2020 IEEE 1

Department Head

information spaces thus are easily overwhelmedby an abundance of details, and can swiftly feel”museum fatigue” due to the huge amount ofinformation [1], which is why various techniqueshave been developed to provide them with acollection overview in advance, which assistsorientation and exploration.

In this context, visualization plays an es-sential role for digital cultural collections. Inaddition to realistic object images, which pro-vide detail views for selected objects (such asphotographs or 3D scans), information visualiza-tion provides manifold options to create collec-tion overviews and browsable arrangements onscreens [2]. Amongst other techniques, maps, set,and network visualizations play a frequent role,to convey visitors with orientation about origins,attributes, and relations within large corpora ofobjects. Also historical information (e.g., pre-sented via timelines) plays an essential role incultural heritage visualization, and it has beenargued that a temporal perspective (i.e., a ”di-achronic view”) could be productively added toevery ”synchronic” (i.e., non-temporal) visualiza-tion technique [3].

The standard technique to actually drawtogether a combination of multiple perspectivesis the use of coordinated multiple views[4]. Multiple views can effectively connectdifferent perspectives into an ensemble ofcomplementary composites, but they also comewith specific downsides: They split the users’attention between different views, they increaseperceptual complexity, and their heterogeneousmapping techniques require the coordinationof heterogeneous sensemaking procedures.Consequently, the information provided bymultiple views cannot easily be cognitivelyintegrated [5]. Against this background, weargue, that many views are good to have,but not good enough, if users should gain abetter integrated, bigger picture of a complexphenomenon (see Box 1). With this paper, weintroduce a prototype visualization system forcultural collections doing just that: It unfoldsa variety of views, but also goes beyondthis essential standard technique to visualizeinformation in a more integrated manner, tofacilitate synoptic insights, and to foster cross-dimensional reasoning. In parallel, the underlying

PolyCube project seeks to develop synopticvisualizations, which help users to develop anintegrated understanding of multiple dimensionsin the collection (a mental model) rather thana poorly connected mashup of information(a cognitive collage).1 In the following, wezoom in on challenges arising from the use ofcoordinated multiple views and delineate designrationales and specific “coherence techniques”for creating more integrated visualizations. Thenwe introduce the PolyCube prototype, summarizethe results of an evaluation, and sketch out futurework.

BEYOND MULTIPLE VIEWSThe data structure of cultural objects can becomplex (see Figure 1): In addition to realisticobject images, collectors and curators commonlyassemble a whole range of object metadata—and various datafication procedures can extract awhole range of additional object features [2]. Dig-ital cultural collections thus commonly appear ascomplex, multidimensional datasets, where eachobject is connected to multiple metadata entries.

Given this informational richness, visualiza-tion designers commonly conclude that “one viewis not enough” [3]. Single visualization tech-niques can only make a certain selection of datadimensions visible. Multiple views, by contrast,multiply the analytical vistas or interpretive win-dows into the data and thus can help to maximizeinsight, balance the strengths and weaknesses ofindividual views, and avoid misinterpretation [8].Most interfaces to cultural collections thus makeuse of this design strategy (e.g., by combininga geographic map with a faceted timeline), tomake more than one data dimension visible atonce and to offer more than just one dimensionfor conceptual orientation [2].

Multiple views can also be a way to mul-tiply insights for data dimensions with specificrelevance. Time arguably is such an essentialdimension in cultural heritage and history dataand it has been said that it could be produc-tively connected with every other ‘synchronic’visualization technique [3]. Where time plays acentral role, systems thus can offer more than one

1Online: http://www.donau-uni.ac.at/en/polycube/

2 IEEE Computer Graphics and Applications

sets

graphs

maps

video

text

3D object

time

creator

feature

audio category

imagesplace

other

object metadata

digital cultural

object

relation to..

attribute

timevis

texts

images

artifacts

music

films

sculptures

digi

tizat

ion

visua

lizat

ion

Figure 1. Digital cultural objects (center) are frequently connected to multiple dimensions of metadata, whichcan be encoded into different aggregated views of a collection (right).

encoding for the ‘diachronic’ data dimension, sothat strengths and weaknesses of specific time-visualizations can mutually complement and bal-ance each other [8].

Problems with Multiple Views

However, coordinated multiple views also comewith notable downsides and cognitive overheads:They increase perceptual complexity and raise theefforts required to learn a system, while splittinguser’s attention during comparison of views andcontext switching [9]. Multiple views merge het-erogeneous visualization techniques within onelarger frame of display space, yet they cannot bedirectly integrated on a synoptic reasoning level(cf. the fable of the elephant and the blind men)[5]. Juxtaposed information visualizations do noteasily assemble into a “bigger picture” of thecultural collection, and do not necessarily lead toan integrated and synoptic understanding—exceptusers invest high mental effort to re-integratethem on a non-perceptual level into one mentalmodel (see Box 1, [6], [7]). Even then, thereis no natural interconnectivity of different typesof diagrams: They do not work like pieces of apuzzle or photographic perspectives, which canbe easily re-assembled to an integrated whole.If a “bigger picture” is intended, visualizationdesigners have to actually support its creation—and interface designers have to invest additionalefforts within the ‘gutter’ of multiple views [10].

Design Rationales

With specific regard to these challenges, we de-veloped the PolyCube framework to investigatebetter integrated and more coherent visualizationtechniques. With focus on cultural collections, theproject has been organized around a number ofdesign requirements, which aim to foster synopticinsights and cross-dimensional reasoning. In shortorder, the PolyCube framework provides:

• multiple synchronic collection overviews,• multiple types of diachronic overviews,• access to single objects (close-up views), and• flexible recombination of views, while• maximizing visual system integration by the

use of multiple ‘coherence techniques’.

Figure 2 shows conceptually how the Poly-Cube framework interconnects the four collectionvisualization techniques, which have been out-lined in Figure 1 (right-hand side).

Coherence Techniques

To develop a better integrated and coherent rep-resentation of the collection, we combine threedistinct design strategies, which we refer to as”coherence techniques” [7], i.e. techniques whichhelp users to connect perspectives and informa-tion for the construction of a better integratedmental model. (1) We utilize the standard co-herence technique of coordination of views [4],[9], which juxtaposes and interconnects a map, aset, and a network perspective via the interactiontechniques of brushing, linking, and coordinated

May/June 2020 3

Department Head

Box1: Some remarks on cognition

The design rationale of the presented system draws on basic cognitive assumptions. For a betterunderstanding, we summarize some of the guiding concepts deemed relevant for reasoning withmultiple views. For a more detailed discussion see [6], [7]:

Mental models. In cognitive terms, while users explore and observe an information space, theycontinuously construct an internal representation of this space and use this so-called mental modelto interpret the information and to explore it. Mental models are not fixed, but are dynamicrepresentations that develop over time, growing richer in structure and content.

Coherence. When confronted with a task or problem, users utilize and inspect their mental modelto find and evaluate possible solutions. In these cases, internal representations are more effectivewhich connect their different pieces of information in a coherent and consistent fashion. In contrastto such a well-connected mental model, distorted and loosely connected internal representationswere termed cognitive collages.

Integration. If complex conceptual entities and phenomena are visualized across multiple views,users will build up representations for each of these views, which are only loosely connected. Todevelop a coherent bigger picture—and to solve problems across multiple dia- and synchronicdata dimensions—these separate information pieces have to be (re-)connected. This syntheticaltask turns out to require high amounts of cognitive load, as cognitive science research shows [6].Therefore, to solve cross-dimensional problems with less effort, users require support to build upinternal representations, which combine and interconnect multiple data dimensions consistently.This is the background, against which the PolyCube project investigates design and coherencetechniques—to go beyond multiple views and help users build up more integrated mental models.

highlighting. (2) To go beyond multiple views,we resolve one stand-alone view—the one ontime (cf. [9]: Rule of Parsimony)—and system-atically integrate it into all other perspectives:While maps, sets, and networks commonly appearas synchronic (i.e., non-temporal or aggregated)views, we do not only enrich them with tempo-ral information (i.e., coherence technique No.2:”integrated encoding”), but offer four differentways to do so. As such, we ‘upgrade’ eachsynchronic view to four types of syn-diachronic(i.e., time-oriented) views: (i) a space-time cuberepresentation (STC), (ii) a color-coded superim-position view (SI), (iii) an animated view (ANI),and (iv) a juxtaposition view (JP). (3) Finally,to visually integrate these multiple diachronicviews, we mediate the transitions between themwith the coherence technique of “animated can-vas transitions”, which utilize dynamic space-time cube operations [11]. In joint, this packageof coherence techniques aims to go beyond thecommon, casual bricolage of multiple views, and

to provide a synoptic, but flexible system forcomplex collection visualization with a focuson the historical data dimension and visual andcognitive information integration.

Data, Users, Tasks

In order to further specify the design of our visu-alizations, we considered characteristics of data,users, and users’ tasks in the cultural collectioncontext.

Data. As we aimed to develop a visualizationframework applicable to a wide range of culturalcollections, we used two different datasets toinform and test our prototype design, which focuson photographs and films. The first dataset isa random selection of 2000 objects from thephotography collection of Charles W. Cushman.2

Cushman was a well-travelled American amateurphotographer, who documented his activities by

2Online: https://webapp1.dlib.indiana.edu/cushman/, preparedby http://miriamposner.com/blog/getting-started-with-palladio/.

4 IEEE Computer Graphics and Applications

Coherence technique No. 2: Integrated encoding of syn-chronic and diachronic data dimensions assyndiachronic views.

superimposition(+ color coding)

animation(slideshow)

juxtaposition(small multiples)

t2

t1

space time cube (stc)

sets

graphs

maps

synchronic views

diachronic views

animatedmaps

animatedsets

animatedgraphs

juxtaposedmaps

juxtaposedsets

juxtaposedgraphs

superimposedmaps

superimposedsets

superimposedgraphs

map timecube

set timecube

graph timecube

Coherence technique No. 3: Seamless Canvas Transitions

Coherence technique No. 1:Coordination of Views

timevis

*

*

Figure 2. Cross-tabulation of the available synchronic views (lines) and diachronic views (columns) of thePolyCube system, together with annotation of implemented visual “coherence techniques” (orange).

means of a travelogue, annotating all his pho-tographs with locations, dates, keywords, andshort descriptions. Each photographic object thushas categorical, temporal, and spatial metadataentries, and we additionally computed inter-objectrelations based on keyword and description sim-ilarity.

The second dataset has been extracted fromthe Internet Movie Database (IMDB) by Spitz andHorvat [12] for over 55,000 movies, comprisinginformation on genres, locations of production,year of release, and references to other movies(such as ‘is referenced’, ‘is featured’, ’is versionof’, ‘is a sequel’, ‘is a spin-off’). We reduced thedataset to keep only the most influential movies(i.e. which have been referenced at least 15 timesin other movies), which reduced the dataset toapprox. 2,000 movies. Our IMDB dataset thusalso features categorical, spatial, temporal, and re-lational metadata entries for each movie, togetherwith a short description and a movie poster,which we added from the Open Movie Database(OMDb).3

Users. Regarding the users of our system, weintended to support casual users, i.e. a hetero-geneous group of users with different levels ofvisual literacy, expert knowledge, and interest.As they explore cultural collections mainly forleisure purposes, they are not necessarily moti-vated to invest high amounts of cognitive load tofully process and interconnect all information andperspectives available. Therefore, casual users can

3Online: http://www.omdbapi.com/

benefit from visualization designs which activelyassist them in the construction of a mental model.

In addition, it is important to be aware thatcasual users will only persist in exploration aslong as it is rewarding for them (i.e., engaging,aesthetically pleasing, interesting or intriguing).Hermetic and complex system designs can easilyterminate the interaction at an early stage.Tasks. Casual users of cultural collections oftenhave no specific information needs, but lookaround and browse for something interesting,which they can access for details on demand.They do not pursue concrete tasks, but they arekeen on gaining an overview and exploring thedigital collection. When designing and develop-ing PolyCube, we attempted to clarify these rathervague tasks as follows: (1) gaining a (synoptic)overview and conceptual orientation regarding thedistribution of the major data dimensions of acultural collection (e.g., time, space, categories,relations), (2) finding single objects of personalinterest and inspecting their details, (3) browsingthrough objects (e.g., according to time, relations,geographic origins or shared categories). Thesetasks align with the task typology proposed byBrehmer and Munzner [13], as the users in thiscase are consumers of the visualization and wantto explore, browse, and enjoy.

SYSTEM DESIGNBy its name, the PolyCube system refers toits initial syn-diachronic setup, which combinesmultiple space-time cube representations as ‘co-

May/June 2020 5

Department Head

Figure 3. The PolyCube system in its initial setup, providing a map, a set, and a network visualization of theCushman photography collection from a space-time cube perspective, with one category filtered.

ordinated linked cubes’. Figure 3 depicts thisinitial setup, which offers a multi-dimensional,time-oriented overview for different dimensionsof a cultural collection. From left to right, itlinks a map-based view with a visualization ofset-typed information and a graph-based repre-sentation of the collection [Coherence techniqueNo. 1: coordination of views]. Instead of fol-lowing a common design practice and addinga coordinated timeline visualization as an extraview, STC representations encode diachronic andsynchronic data dimensions in a visually andlocally integrated fashion (coherence techniqueNo. 2: integrated encoding).

Coordinated Multiple CubesA map-based space-time cube visualizes thespatio-temporal origins of a cultural object col-lection as a three-dimensional point cloud, whereeach object can be selected to access detailedinformation and a preview. While the horizontaldata plane shows a geographic map, the verticalaxis represents time as an upward-pointing, spa-tial dimension. Temporal object positions can bevisualized in a continuous or aggregated fashion(according to the chosen temporal granularity,signified by the number of horizontal slices). Theresulting distribution of data points discloses thetemporal and spatial origins and extension of acollection, together with spatio-temporal clustersand outliers.

A set-based space-time cube makes categori-cal or set-typed collection information visible bythe means of circular set diagrams, in which datapoints, again, represent individual objects. Datapoints are horizontally arranged according to theirprimary set-affiliation (which can signify culturalstyles, movements, curatorial categories, or gen-res), horizontally in time layers, and within eachset-instance over time in a sunflower Phyllotaxislayout [14]. A visual hull structure can be acti-vated to connect different temporal instances ofeach set, making set dynamics within a collection(like emergence, growth, diminution, breaks, ordecline) visible and traceable over time [5].

Finally, a network-based space-time cube rep-resents relations between objects, based on aforce-directed network layout. To avoid visualclutter generated by too many ties, we de-cided to cap the maximum number of displayedrelations—or to hide links on demand. In caseof weighted ties (Cushman), only the top sim-ilarity relations between objects are displayedas links. Also for non-weighted networks, onlyone random relation is visualized at the moment.Relations between objects thus form a three-dimensional meshwork, disclosing node clustersand outliers, as well as bundles and structuresof relations. Structural node measures can bevisualized on demand as glyph size (total degreecentrality, as well as in-degree and out-degree),

6 IEEE Computer Graphics and Applications

Figure 4. Three alternative syn-diachronic views support the visual analysis of the temporal developments ofmaps, sets, and networks by the means of a) color coded superimposition, b) animation, and c) juxtaposition.

to easily identify the most important (e.g., mostreferenced) cultural objects.

To reduce the complexity for casual usersand to accommodate also for simpler datasets,each space-time cube can be hidden on demand.Thereby, the Polycube system offers multipleindependent, but coordinated views that still sumup into a bigger picture of juxtaposed componentswhen needed.

Further Syn-Diachronic ViewsFrom a comparative point of view, all visualiza-tion techniques have different analytical strengthsand weaknesses. This also applies for differentvisualizations of time [15], [16]—and it holdsalso true for space-time cube encodings, whichcan suffer from visual clutter and other short-comings (see more in the discussion). Due tothe relevance of time for archival and histori-cal collections, we thus provide multiple syn-diachronic views, to investigate the (geographical,set-typed, and relational) temporality of collec-tions with a plurality of encoding techniques.To preserve the users’ mental map during theswitching of syn-diachronic views and to helpthem to integrate new perspectives into their priorconstructed mental models, we utilize animatedspace-time-cube operations as canvas transitionsbetween the different syn-diachronic views (i.e.,coherence technique No. 3) [11].

A color-coded superimpostion view (SI) lit-erally flattens the STC perspective, to offer anaccumulated, top-down view on maps, sets, and

networks. In this view, time is encoded by the useof a Viridis color scale, which also appears as anannotated legend on the left side of the screen(see Figure 4a).

An animation function (ANI) allows users tolet a selected temporal brush traverse the datasetalong the time axis (see Figure 4b). This anima-tion function can be activated in the superimposi-tion view, resulting in a flat animation, but it canalso be activated in the two other syn-diachronicperspectives (STC and JP).

Finally, a juxtaposed view (JP) presents timein a ‘sliced’ manner, arranging flattened timesections as panels side by side, from top tobottom (see Figure 4c). The number of panelsis controlled by the users—and corresponds tothe number of temporal slices shown in the STCrepresentation.

Interaction DesignMany settings of the system can be modified andregulated by its users. Most fundamentally, usersdecide (via buttons at the bottom right) if theywant to see all three synchronic views (maps,sets, networks)—or only a selection thereof. Theyalso choose the syn-diachronic perspective oftheir liking (STC, SI, ANI, JP) and every chosenview can be zoomed and panned—and in case ofthe STC also rotated. When switching betweensyn-diachronic views (more specifically betweenSTC, SI, and JP), the animated canvas transitionscome into play. Figure 5 shows how these canvastransitions mediate all switching activities, so

May/June 2020 7

Department Head

juxtaposition

TIM

E

superimposition

GEO SET NET

TIMEGEO SET NET

space-time cubeTIM

E

animation

Figure 5. Animated canvas transitions between syn-diachronic views, based on space-time cube operations.

that the mental maps—which have already beencreated—can be preserved.

Object Preview: Across all syn-diachronicviews, clicking a data point opens a close-up-panel, presenting an object preview on the leftside. The panel also displays selected objectmetadata (object categories, date and place oforigin, degree centrality), similar objects, and abrowsing function, which allows to swipe chrono-logically through a cultural collection.

Temporal Filtering: A timeline widget at theleft side of the screen allows to filter and brushdynamically within the temporal scope of a col-lection. This functionality works across all syn-diachronic views in a coordinated fashion.

Glyph Color: The system provides three coloroptions for the data points: a uniform color (red),distinct categorical colors according to the set-assignment of objects (as indicated by a legendat the bottom of the screen), and a temporal color-coding, utilizing the Viridis color scale.

Noverlap Function: With increasing size andspatio-temporal collection density, the placementof data points on maps can generate significantoverlays, merging hundreds of objects into oneglyph. As a clutter management technique, a user-controlled “noverlap”-function adds positional jit-ter to object positions in all map-based views,thus spreading out glyphs around their collectivecenter point, which allows to disentangle co-locations and supports visual disambiguation.

Implementation Details

The PolyCube system is an open-source webapplication, and a demonstration version show-casing the data of the two case studies is availableat the following URL: https://danubevislab.github.io/polycube/cga2020/. The source code and im-plementation along with steps on how to set upthe system can be found at https://github.com/danubevislab/polycubeViews. In our implementa-tion we use D3.js (https://d3js.org/) to computethe layout of the set-based and network-basedspace time cubes, three.js (https://threejs.org/) tocreate a three-dimensional environment and plotour data points, and Angular (https://angular.io/)to manage the interactions, views, and navigation.For the map-based space time cube we also utilizeMapbox (https://www.mapbox.com/) to arrangeour data points according to their geographicposition.

EVALUATIONIn the course of the system development, weconducted several user studies (for geo-temporalviews [3], [17], for set-temporal views [14]) andregular heuristic inspections. For the summativeevaluation of the whole prototype system, weset up a qualitative study with ten casual users(under 30yrs: 2, 30-40yrs: 5, over 50yrs: 3 /university degree: 6 / female: 9). Their experiencewith visualization was mixed (5 low, 3 medium,2 high), as was their interest in the domains ofthe collections—photography (0 low, 2 medium,8 high) and films (1 low, 2 medium, 7 high). The

8 IEEE Computer Graphics and Applications

test procedure took between 21 and 33 minutes(M=28).

The aim of the evaluation was to understandhow the coherence techniques could support ca-sual users. More specifically, we focused on (1)how animated canvas transitions (coherence tech-nique No. 3) supported the users’ understandingof the different views, and (2) how the integratedencoding (coherence technique No. 2) and thecoordination of views (coherence technique No.1) affected users’ synoptic insight into the collec-tion. In addition we inspected the overall usabilityof the system, and asked for user preferencesregarding various syn-diachronic perspectives.

Procedure. Each participant tested the prototypeindividually on a 24” computer screen. In phaseI, the experimenter started with a guided walk-through of the Cushman photography collection,in which she explained all visualization and in-teraction options. By a mixture of guidance andonboarding techniques, we aimed (1) to transi-tion from familiar to unfamiliar views, (2) toincrease the visual complexity stepwise, and (3)to minimize coherence breaks. Figure 6 gives anoverview on the guided walk-through: We startedwith a conventional map (1) and showed thetemporal development of the collection with ananimation (2). Then a seamless canvas transitionguided the user to the geographic STC represen-tation (3). This view was amended stepwise witha categorical (4) and a relational STC (5). Af-terwards, the combined STCs were transformedto a SI (6) and a JP view (7) by means ofseamless canvas transitions. After each step, usersdescribed their understanding of the visualiza-tion and any misconceptions were cleared upimmediately. In addition the users were asked tocompare the different views, to describe perceivedstrengths and weaknesses, and to state their over-all preferences.

In phase II, the test participants could freelyexplore the IMDB movie collection with thesame prototype. During the exploration, they wereasked to think aloud and to share all observationson the data as well as on the prototype. Duringthe whole procedure, an observer took notes ofthe participants utterances and interactions. Theseprotocols were analyzed qualitatively.

1 2 3 4

5 6

7

Figure 6. Guided walk through the PolyCube proto-type: map (1), animation (2), transition to space-timecube (3), addition of set-time cube (4) and a net-timecube (5), transition to the superimposed view (6) andthe juxtaposed view (7). Arrows indicate animatedcanvas transitions.

Results

Animated Canvas Transitions. After a seamlesstransition from a 2D map to a geographic STC,eight users correctly understood the visualizationtechnique (P2: ”pictures are distributed accordingto time and space”), though three of them werenot completely confident. One user required anexplanation and another one mistook the yearlygeographic dispersion for the overall number ofpictures in this year.

The animated transition from the combinedSTC to the JP view, helped all users to understandit quite easily (P2: ”superimposed view, but nowsplit up by time”). In contrast, the animatedtransition from the combined STC to the SI view,was more difficult for the users: Only five userscould correctly describe the new view (P10: ”the time information was now extracted from the3D view, the color shows the chronology”), butsome needed to see the transition multiple times.Another four users had problems to understandthe new color-coding and frequently mistook itfor the categories.

When asked about their opinion on the canvastransitions, half of users regarded the transitionsbetween the views as helpful for their understand-ing. Three users found it only partly useful (P8:”it should be slower to be useful”) and two foundit not useful at all (P7: ”it is a nice effect, but nothelpful”).

Overall, the results of the evaluation con-firm that animated canvas transitions helped most

May/June 2020 9

Department Head

users to understand the STC and the JP view.The SI view turned out to be rather difficultdue to a coherence break of the data points’color-coding: The transition from categorical totemporal color coding was not included in theanimated transition and led to several difficultiesand misconceptions. One user suggested to keepthe categorical color coding in SI and use addi-tionally color saturation, for example, to encodetime.

Preferences. During the walk-through, users wereasked to state their opinion on each syn-diachronic view, as compared to the others (seeFigure 7) and to describe their individual prosand cons. Nine out of ten users found the STCview best (P1: ”offers always a reference point fororientation” / P3: ”good overview on the map andthe sets” / P8: ”most informative” / P10: ”cool,good for exploration, more substance”). The SIview was rated high regarding aesthetics, butlow for its information value (P4: ”aestheticallymore pleasing and extraordinary, but for analysisthe 3D view is better”). One exception was thenetwork view, which two users found more com-prehensible in SI (P2: ”The 2D network is morepleasant”). The JP view received mixed opinions.Most users found it useful for an analysis ofsingle time points, but too scattered for an overallpicture (P3: ”its too much, but good if you areinterested in a year only” / P6: ”if you wantto look at a year only, it is useful, but for anoverview other views are better”). ANI was founduseful as a starting point (P2: ”it’s good in thebeginning to see the genesis”).

Synoptic Insights. During the free exploration ofthe IMDB collection, we asked the users to thinkaloud and coded multidimensional insights intothe data: Eight users had geo-temporal insights(P2: ”The U.S. is active over all years, in Europethere is a drop during WW2, and Asia started lateronly”) and eight users reported a set-temporalinsight (P10: ”It is astonishing how early Sci-Fi films started.”). Only one user had relational-temporal insights, whereas the other users hadno insights at all in the network visualization oridentified only influential films or clusters withouta reflection on time.

Beyond these single views, we were interestedto see how many users integrated more data di-

STC SI JPTP1TP2 Geo Set Net Geo Set NetTP3 Geo Set NetTP4TP5TP6 Geo Set Net Geo Set NetTP7TP8TP9 Geo Set Net Geo Set NetTP10

best worst.

Figure 7. Preferences of different views.

mensions in their insights, for example, sets, time,and space. We found such higher-dimensionalinsights only in the think-aloud protocols of threetest users (P3: ”In the early 20th century mainlydramas were produced in Europe.”). An impor-tant and frequently used interaction techniquefor higher-dimensional insights (as well as thereduction of information density and complexity)was the option to filter all views according to thecategories, which was applied by seven test users.

These findings indicate on one hand that theintegrated encoding could successfully supportcasual users in gaining synoptic insights intogeographic space or categories over time. How-ever, we did observe hardly any network-relatedinsights over time. This might be due to thefact, that the network view turned out to be themost complex view for most users and some evenexpressed difficulties to image its usefulness ingeneral (P9: ”It might have some logic”).

On the other hand, our results do not indicatethat the coordination of views was perceived asan effective coherence technique in supportingsynoptic insights. While the juxtaposed and co-ordinated cubes or canvases of the system tech-nically allow for a ‘massive parallel glimpse’ onall time-oriented data dimensions, our users seem-ingly did not utilize this synoptic option duringfree exploration. Even though we observed somehigher-dimensional insights, these were gained byapplying a categorical filter on the geo-temporalview rather than by integrating patterns found inthe set- and in the geo-temporal view. However,in our evaluation procedure we did not explicitlyprompt the users to look for synoptic patternsintegrating all three perspectives. Given time,motivation, or more specific tasks, such insightsmight still occur.

10 IEEE Computer Graphics and Applications

DISCUSSIONThe outlined system design contains numerouschallenges for future work, and various aspects,which deserve further reflection:

Benefits and Costs of Coherence Techniques

From a cognitive science perspective, coherenceis an important factor influencing the quality ofinternal representations—especially for represen-tations of complex information spaces—and howeffectively reasoning operations across multipledata dimensions can be performed. The Poly-Cube system implements three different visualcoherence techniques, aiming to draw togetherdifferent aspects of a multi-view system (Fig. 2).Regarding the actual performance of these tech-niques, the results of our qualitative user studystrengthen the notion that animated canvas transi-tions successfully act as an onboarding technique,which helped users to develop an understandingof a novel view coming from an already familiarone. Our results suggest that the animated tran-sitions leads users to seamlessly integrate novelviews with the existing internal representationsof familiar views. The integrated encoding oftime and geographic space as well as time andcategories enabled the study participants to gainsynoptic insights into the collection—an indicatorof the integration of those data dimensions inthe internal representation as well. However, thejuxtaposition and coordination of views seemedto be less effective, as hardly any user connectedgeographic, categorical, and relational patterns. Ifthey did, they used the categorical color-codingand filtering within the geo-temporal view—afurther argument for integrated encoding. In ad-dition, inconsistent encoding of visual variables(in our case: color) was confirmed to be harm-ful for an overall understanding of a multi-viewsystem [9]. As this study is qualitative, we didnot systematically investigate the effects of thedifferent coherence techniques. Therefore, furtherresearch is required to assemble further visualcoherence techniques and explore their benefitsand limitations in a more systematic way.

While a strong motivation of this projectemerged from the fact that the cognitive costsof multiple views can accumulate into solid bar-riers for casual users, we are aware of various

overheads, that come along with coherent visu-alization design as well. Obviously, additionalefforts regarding a system’s design, implemen-tation, and testing are needed to better connectand combine a plurality of views. Future studieshave to corroborate our guiding hypothesis, andthe beneficial effects of coherence techniques willhave to be proven in a more systematic fashion,including novel evaluation methods [7].

Utilizing Three Dimensions

Three-dimensional designs are frequently crit-icized in an information visualization context:Amongst other reasons, 3D is known to generateocclusion of objects, depth ambiguity, perspectivedistortion of distances and angles, and to raiseinteraction costs for analysts trying to compen-sate these effects [18]. However, the resultingstandard recommendation (i.e. to avoid 3D anduse multiple views instead) also incurs significantcollateral costs. These costs (such as split atten-tion and steep increase of cognitive load duringpost-perceptual syntheses) frequently remain hid-den during evaluation procedures which focus onlocal and rather short-term analytical tasks, andwhich do not reflect on the role and relevance ofinformation integration [7]. By contrast, the useof STC representations in the PolyCube systemactually serves multiple integrative causes, andhas additional positive effects, which arguably arealso rarely heeded in the visualization context [5]:

• In line with SI, JP, and ANI, STC repre-sentations provide a visually integrated syn-diachronic view (coherence technique No. 2),thus helping to avoid the additional visualiza-tion of a linked timeline. However, STCs do soin a uniquely just and balanced manner: Theyutilize the most effective visual variable (i.e.positional encoding) equally for both sides: xand y-axis to a synchronic, and the z-axis tothe diachronic data dimension [cf. [16]].

• The morphology of the resulting point cloudscan be parsed and read by the highly trainedfaculties of 3D gestalt perception: Temporaldevelopments become visible as spatial pat-terns along the vertical dimension—as effec-tively illustrated by the temporal pattern lan-guage encoded into the hull structure of the set-time cube. Empirical studies on casual users

May/June 2020 11

Department Head

confirm that they can identify multidimen-sional patterns more quickly and more accu-rately with STC than with 2D visualizations([17]). Similarly, the present study observedthat they facilitate synoptic insights.

• As an additional feature, STC representationscan act as translational hubs: They can mediatebetween the various syn-diachronic views (SI,JP, ANI), and also translate from temporalto non-temporal perspectives, while supportingvisual analysts navigation by animated canvastransitions [11] (coherence technique No. 3).

• Studies show that STC representations havea certain attraction power and are considered‘cool’ (as summarized in [17]). Also in thepresent study, most casual users preferred theSTC view for the exploration of a culturalcollection. This kind of affinity should not bedismissed, given the importance of drawing ca-sual users into an in-depth exploration process.

• Finally, as parallel, coordinated ”PolyCubes”(coherence technique No.1), STCs provide aspatially adjacent “macro-shape” (so to say an“elephant”). This macro-shape can also helpto navigate multidimensional datasets, and totransform abstract analytical tasks into move-ments of perspective-taking, or of spatial ma-nipulation (cutting, flattening, probing, com-paring, etc.) of one or all cubes [11].

Drawing these arguments together, we considerSTC representations to provide a powerful andlargely untapped potential for visual synthesisand information integration. By offering them asone out of four syn-diachronic views—reconciledand mediated by STC operations—we do notrisk possible insights, but aim to maximize andconnect them.

Flexibility

An important aspect in the development of Poly-Cube, was the flexibility of the system. Depend-ing on the data, the users, and their intentions,the system can be flexibly adapted: (1) The mostrelevant and interesting synchronic views (map,sets, or network) can be activated or de-activated,as can be additional visual encodings (such as thecolors of data points, links, or hull structures).This allows users—who are easily overwhelmedby visual complexity—to reduce the information

load to an appropriate level. (2) Different syn-diachronic views (STC, SI, JP, ANI) can beselected for exploration. This is especially impor-tant for casual users, who differ in their individualpreferences (see figure 7) and are more likely toexplore a collection with views that meet theirpreferences. (3) Finally, PolyCube is intended tobe open for all kinds of cultural collection data,which include temporal and geographic, categori-cal, or relational data. To enable the visualizationof novel cultural datasets, the prototype pulls thedata from a Google spreadsheet.

Missing and Uncertain Data

Data from historical collections and archives fre-quently contains significant gaps, ambiguities andhigh amounts of uncertainty. For the many issuesconnected to such issues of heterogeneous dataquality, the PolyCube prototype currently offerstwo coping techniques: 1) For collections withmissing data dimensions (i.e., geo-information orrelations between objects), the number of viewscan be reduced (e.g., set-time-visualizations only,or geo-and-set-time combination only). 2) Foruncertain data, we have started to develop glyph-based and relation-based uncertainty visualizationtechniques, which is a non-trivial endeavor, whenconsistent encoding is sought for an already com-plex visualization system [19].

Limitations

Due to its computing-intensive character, we con-sider the STC view to be the system’s bottleneckperspective—especially with regard to the net-work view, where large numbers of edges haveto be visualized in addition to each object. Whileour two use cases (Cushman and IMDB) featureabout 2000 objects per cube, we have been ableto represent up to 20.000 objects per cube, whilemaintaining reasonable response times. However,we expect that analysts have to avoid the 3D view,when they want to investigate large collections,going significantly beyond this threshold.

Generalization and Future WorkWhat we stated at the beginning about culturalcollection data—rich, complex and dynamic—actually applies for a wealth of other phenomena,not only, but especially in the (computational)

12 IEEE Computer Graphics and Applications

social sciences and (digital) humanities realm.Novel visualization approaches and collabora-tions for these challenging subject matters aredeveloping [20]—and the availability of multiple“interpretive” perspectives is a constant demand,as indicated by terms like plurality, generos-ity, triangulation, and parallax [5]. From event-based data (historical events) to trajectory data(biographies, military campaigns), to relationaldata (historical networks), to infinite combina-tions of contextual visualizations of digital his-tory, humanities and cultural heritage studies—new types of visualization systems and visual-analytical environments are needed in the digitalhumanities. As an example, the PolyCube systemconcept has already been extended to representbiographical data in multiple information spaces,including non-geographic domains [16].

CONCLUSIONWith this paper we introduced the PolyCubevisualization system for rich, time-oriented col-lection data with the specific aim to maximizesynergies and connections between the encodingand representations of multiple data dimensions.While firmly embracing the use of multiple views,the system also aims to facilitate synoptic insightsand cross-dimensional reasoning with less cog-nitive effort. For that matter, our main designstrategies in the development of the prototypewere 1) assembly and coordination of comple-mentary visualizations, 2) integrated encoding oftime with multiple types of syn-diachronic views,as well as 3) mediating the switching betweenvarious views with animated canvas transitions.An evaluation with casual users confirmed thatespecially the latter two are efficient techniquesto create more integrated internal representations,while future work has to shed light on cognitivetop-level syntheses with more systematic studydesigns.

Concerning the future application of the out-lined system design, we consider an infiniteamount of subject matters in digital humani-ties and social science domains to be connectedto complex, dynamic systems or time-orientedphenomena. To convey research and teachingcommunities with appropriate visualizations, theprovision of many parallel views is not enough.

With this paper we make the case for the devel-opment of more considerate, “visual-synthetical”design strategies. While obviously providing allrelevant visual-analytical functions, novel designstrategies should also help users to integrate andconnect multiple perspectives to complex mentalmodels on a visual top-level of reasoning oper-ations. With one possible approach to this kindof interface design, we aim to give an impulse tocorresponding developments and discussions in avariety of related visualization domains.

ACKNOWLEDGMENT

The authors want to thank Emoke Agnes Horvatfor sharing the data of the study on cinematiccitations [12]. This work was supported by agrant from the Austrian Science Fund (FWF, No.P28363-24), and the NFB (No. SC16-032).

REFERENCES

1. S. Bitgood, “Museum Fatigue: A Critical Review,” Visitor

Studies., vol. 12, no. 2, pp. 93–111, 2009.

2. F. Windhager, P. Federico, G. Schreder, K. Glinka, M.

Dork, S. Miksch, and E. Mayr, “Visualization of Cultural

Heritage Collection Data: State of the Art and Future

Challenges,” TVCG., vol. 25, no. 6, pp. 2311–2330, 2019.

3. M. Dork, C. Pietsch, G. Credico, “One view is not enough.

High-level visualizations of a large cultural collection,“Inf.

Des. J., vol. 23, no. 1, pp. 39–47, 2017

4. J. C. Roberts, H. AlManeea, P. W. S. Butcher, R. Lew,

G. Rees, N. Sharma, A. FrankenbergGarcia, “Multiple

Views: Different meanings and collocated words,” Com-

puter Graphics Forum, vol. 38, no. 3, pp. 79–93, 2019

5. F. Windhager, S. Salisu, G. Schreder, and E. Mayr,

“Orchestrating Overviews:A Synoptic Approach to the

Visualization of Cultural Collections. Remaking Collec-

tions,” Open Library of the Humanities, SI on Remaking

Collections, 2018.

6. G. Schreder, F. Windhager, M. Smuc, E. Mayr, “A Mental

Models Perspective on Designing Information Visualiza-

tions for Political Communication” JeDEM-eJournal of

eDemocracy and Open Government. vol. 8, no. 3, pp.

80–99, 2016

7. F. Windhager, G. Schreder, & E. Mayr, “Designing for

a Bigger Picture: Towards a Macrosyntax for Infor-

mation Visualizations,” OSF Preprint. March 31, 2017,

doi:10.31219/osf.io/q9kdt.

May/June 2020 13

Department Head

8. N. Kerracher, J. Kennedy, & K. Chalmers, “The design

space of temporal graph visualisation,” Proc. of EuroVis

14, Vol. Short., 2014.

9. M. Q. W. Baldonado, A. Woodruff, & A. Kuchinsky,

“Guidelines for Using Multiple Views in Information Visu-

alization,“ Proc. of Conf. on AVI, pp. 110-119, 2000.

10. S. McCloud, Understanding comics: The invisible art

New York, NY: Harper Collins, pp. 60–93, 1994.

11. B. Bach, B. Dragicevic, D. Archambault, C. Hurter,

S. Carpendale, “A Descriptive Framework for Temporal

Data Visualizations Based on Generalized Space-Time

Cubes,” CGF., vol. 36, no. 6, pp. 36–61, 2017.

12. A. Spitz, E. A. Horvat, “Measuring long-term impact

based on network centrality: Unraveling cinematic cita-

tions,” PloS one vol. 9, no. 10, e108857, 2014.

13. M. Brehmer, T. Munzner, “A multi-level typology of

abstract visualization tasks,” TVCG vol. 19, no. 12, pp.

2376–2385, 2013.

14. Salisu, S., Mayr, E., Filipov, V., Leite, R., Miksch, S.,

Windhager, F. (2019). “Shapes of Time: Visualizing Set

Changes Over Time in Cultural Heritage Collections,”

Poster Comp. of EuroVis’19, pp. 45–47, 2019.

15. S. Kriglstein, M. Pohl, M. Smuc, “Pep Up Your Time

Machine,“ Handbook of Human Centric Visualization,

New York, NY: Springer, pp. 203–225, 2014.

16. E. Mayr, F. Windhager, “Once upon a Spacetime: Visual

Storytelling in Cognitive and Geotemporal Information

Spaces,” SPRS Int. J. Geo-Inf., vol. 7, no. 3, pp. 1–18,

2018.

17. E. Mayr, G. Schreder, S. Salisu, F. Windhager, “Inte-

grated Visualization of Space and Time: A Distributed

Cognition Perspective”. OSF Preprint. Dec. 12, 2018, doi:

10.31219/osf.io/agvhw

18. M. Sedlmair, T. Munzner, M. Tory, “Empirical guid-

ance on scatterplot and dimension reduction technique

choices.” TVCG, vol. 19, no. 12, pp. 2634-2643, 2013.

19. F. Windhager, S. Salisu, E. Mayr, “Exhibiting Uncer-

tainty: Visualizing Data Quality Indicators for Cultural

Collections,” MDPI Informatics., vol. 6, no. 3, pp. 1–16,

2019.

20. A. J. Bradley, M. El-Assady, K. Coles, et al., “Visualiza-

tion and the Digital Humanities: Moving Toward Stronger

Collaborations,” IEEE CG&A. vol. 38. no. 6, pp. 26-38,

2018.

Florian Windhager is a research associate at theDepartment for Knowledge & Communication Man-agement at Danube University Krems, Austria. Hisresearch interests are in information visualization,digital humanities, and new fields of application for vi-

sual reasoning. He holds a Master of Philosophy, andhis PhD is developing a visual analytics frameworkfor the area of art-historical and biographical data.Contact him at florian.windhager@donau-uni.ac.at.

Saminu Salisu is a research associate at the De-partment for Knowledge & Communication Manage-ment at Danube University Krems, Austria, and aPhD student at Vienna University of Technology,Austria. His research interests are information Visu-alisation, Visual Analytics, Uncertainty Visualisationand Spatio-Temporal Visualisation. Contact him ats.salisu@live.co.uk.

Roger A. Leite is a PhD student at the Institute ofVisual Computing and Human-Centered Technologyat the Vienna University of Technology (TU Wien). Hismain research interests include Information Visual-ization, Visual Analytics, time-oriented problems, andnetworks. Contact him at roger.leite@tuwien.ac.at.

Velitchko Filipov is a PhD student at the Institute ofVisual Computing and Human-Centered Technologyat the Vienna University of Technology (TU Wien).His research interests are Information Visualization,Visual Analytics, graphs, and networks. Contact himat velitchko.filipov@tuwien.ac.at.

Silvia Miksch is professor at the Institute of Vi-sual Computing and Human-Centered Technology,TU Wien. She served as paper co-chair of IEEE VASTand EuroVis conferences and on the editorial board ofIEEE TVCG. Her main research interests are Visu-alization (particularly focus+context and interaction)and time. Contact her at miksch@ifs.tuwien.ac.at.

Gunther Schreder is a research associate at theDepartment for Knowledge & Communication Man-agement, Danube University Krems, Austria. Hisresearch focuses on how technology can supportthe cognitive processes of individuals and groups.He holds a Master of Science in Psychology fromthe University of Vienna, Austria. Contact him atguenther.schreder@donau-uni.ac.at

Eva Mayr is a postdoctoral researcher at the Depart-ment for Knowledge & Communication Management,Danube University Krems, Austria. Her main researchinterests are the cognitive processes during inter-action with information visualizations, in particularin ”casual”, informal learning settings. Eva holds aPhD in applied cognitive and media Psychology fromthe University of Tubingen, Germany. Contact her ateva.mayr@donau-uni.ac.at.

14 IEEE Computer Graphics and Applications