Scientific visualization represents information asimages that let us explore, discover, analyze, and

validate large collections of data. Much research in thisarea is dedicated to designing effective visualizationsthat support specific analysis needs. Recently, though,we’ve considered visualizations from another angle.We’ve started asking, Are visualizations beautiful? Canwe consider visualizations works of art?

You might expect answers to these questions to varywidely depending on an individual’s interpretation ofwhat it means to be artistic. We believe that the issues ofeffectiveness and aesthetics may not be as independentas they seem initially. We can learn much from studyingtwo related disciplines—human psychophysics and arttheory and history. Human psychophysics teaches ushow we see the world around us. Art history shows ushow artistic masters capture our attention by designingworks that evoke an emotional response. The commoninterest in visual attention provides an important bridgebetween these domains. We’re using this bridge to pro-duce effective and engaging visualizations, and we’d liketo share some of the lessons we’ve learned along the way.

Multidimensional visualizationThrough our lab work, we’ve studied various issues in

scientific visualization for much of the last 10 years. Alarge part of our effort focused on multidimensional visu-alization, the need to visualize multiple layers of over-lapping information simultaneously in a common displayor image. We often divide this problem into two steps:

� the design of a data-feature mapping M, a functionthat defines visual features (such as color, texture, ormotion) to represent the data and

� an analysis of a viewer’s interpretation of the imagesM produces.

An effective M generates visualizations that let viewersrapidly, accurately, and effortlessly explore their data.

One promising technique we discovered is usingresults from human perception to predict the perfor-mance of a particular M. The low-level visual systemidentifies certain properties of what we see very quick-ly, often in only a few tenths of a second or less. Perhapsmore importantly, this ability is display-size insensitive,so visual tasks are completed in a fixed length of time

that’s independent of the amount of information beingdisplayed. Obviously, these findings are attractive in amultidimensional visualization context. We can com-bine different visual features to represent multiple dataattributes and pack large numbers of multidimension-al data elements into an image. A viewer then rapidlyanalyzes sequences of images in a movie-like fashion.

Figure 1 shows two example visualizations of multi-dimensional weather data. We constructed the firstimage by taking traditional visualizations of eachattribute, then compositing them. Hue represents tem-perature (yellow for hot, green for cold), luminance rep-resents pressure (bright for high, dark for low), directedcontours represent wind direction, and Doppler radartraces represent precipitation. We built the secondimage using simulated brush strokes that vary their per-ceptual color and texture properties to visualize thedata. Here, color represents temperature (bright pinkfor hot, dark green for cold), density represents pres-sure (denser for lower pressure), stroke orientation rep-resents wind direction, and size represents precipitation(larger strokes for more rainfall). Although viewersoften gravitate toward the first image because of itsfamiliarity, any attempt to perform real analysis tasksleads to a rapid appreciation of the careful selection ofcolors and textures in the second image. Our experi-ments showed that viewers prefer the second image forthe vast majority of the tasks we tested.

Using perceptual guidelines can dramatically increasethe amount of information we can visualize. We can’ttake advantage of these strengths with an ad-hoc choiceof M, however. Certain combinations of visual featuresactively mask information by interfering with our abil-ity to see an image’s important properties. A key goal,therefore, is to build guidelines on designing effectivevisualizations and to present these findings in a way thatmakes them accessible to other visualization researchersand practitioners.

During the last year, we asked ourselves, How can wemake our visualizations engaging or aesthetically pleas-ing? Although this issue has only recently received atten-tion in the visualization community, we feel it’s animportant factor worthy of study. An image regarded asinteresting or beautiful can encourage viewers to studyit in detail. We might use stylistic techniques that captureand focus attention on certain areas of a painting to

Christopher G.Healey

North CarolinaState University

James T. Enns

University ofBritish Columbia

0272-1716/02/$17.00 © 2002 IEEE

Perception and Painting: A Search for Effective, Engaging Visualizations ________

Visualization ViewpointsEditors: Theresa-Marie Rhyne andLloyd Trinish

10 March/April 2002

highlight important or unexpectedproperties in the data. We expect thelessons learned from studying thework of master painters to have asignificant impact on the quality ofvisualizations we produce.

Visual perceptionAt first glance, the areas of per-

ception and painting might appearcompletely independent of oneanother. However, we found animportant overlap between thebrush style properties in Impres-sionist painting and the fundamen-tal visual features detected by thelow-level visual system. This corre-spondence between low-level visu-al features and painterly styles iscritical for our work, because it letsus design effective and aestheticvisualizations.

One of the most importantlessons of the past 25 years ofresearch in psychophysics is thathuman vision doesn’t resemble therelatively faithful and largely passiveprocess of modern photography.The goal of human vision isn’t to cre-ate a replica or image of the seenworld in our heads. A much bettermetaphor for vision is that of adynamic and ongoing construction project, where theproducts being built are short-lived models of the exter-nal world that are specifically appropriate for the view-er’s current visually guided tasks. It would appear thathumans don’t have general-purpose vision. What we seewhen confronted with a new scene depends as much onour goals and expectations as it does on the array of lightthat bombards our eyes.

Among the research findings responsible for thisaltered view of seeing is a greater appreciation of thefollowing:

� Detailed form and color vision is only possible for atiny window of several degrees of arc surroundingyour current gaze location. Seeing beyond the singleglance therefore requires a time-consuming series ofeye movements.

� Eye movements required for seeing a whole scene—such as the 180-degree view we often assume wehave—are discrete. Our eyes must make many move-ments to see the detail in a large scene and we gainalmost no visual information during an eye move-ment itself.

� Memory for information in one glance to the next isextremely limited. At most, we can successfully mon-itor the details from only three or four objects betweenglances. Often perception is limited to only a singleobject at a time. What we see therefore depends crit-ically on which object(s) in a scene we’re looking forand attending to.

� Human vision is designed to capitalize on the assump-tion that the world is generally a quiet place, so weonly register differences. Objects that are differentfrom their surroundings, or that change or move, drawattention to themselves because of the difference sig-nals that emanate from these visual field locations.

� Few basic visual features exist that we can use to guideattention. These features include differences in thefirst order properties of luminance and hue and thesecond-order properties of orientation, texture, andmotion. Effective third-order properties are limitedto simple properties of shape such as length, area, andconvexity.

We can illustrate the reality of each of these findingsthrough the so-called change blindness that affects usall.1 It involves a task similar to a game that has amusedchildren reading comic strips for many years. Try to findthe difference between the two pictures in Figure 2 (nextpage). (Hint: look at the bushes immediately behind theSphinx.) Many viewers have difficulty seeing any dif-ference and often have to be coached to look carefully tofind it. Once they’ve found it, they realize that the dif-ference isn’t subtle. Change blindness isn’t a failure tosee because of limited visual acuity; it’s a failure basedon inappropriate attentional guidance. Some parts ofthe eye and the brain are clearly responding differentlyto the two pictures. Yet, this doesn’t become part of ourvisual experience until we focus our attention directlyon the objects that differ.

IEEE Computer Graphics and Applications 11

1 Two exam-ples of visualiz-ing weatherconditions: (a) traditionalvisualizationsfor eachattribute com-posited into asingle imageand (b) simulat-ed brush strokesthat vary theircolor and tex-ture to visualizethe data.

(a)

(b)

Harnessing human vision effectively for data visual-ization purposes therefore requires that we constructdisplays that draw attention to their important parts.Because the displays are typically novel, we can’t drawon the expectations that might accompany viewing afamiliar scene. Rather, we must rely on an effective map-ping between data values and features, so that differ-ences in the visual features draw the eyes—and moreimportantly, the mind—on their own. Luring the view-er’s eyes to a particular object or location in the scene isthe first step in having the viewer form a mental repre-sentation of that part of the scene that may outlast thenext glance or scene that comes into view.

Nonphotorealistic visualizationOur interest in artistic visualizations naturally led us

to explore in two directions: nonphotorealistic render-ing in computer graphics and art history and art theorydiscussions of known painterly styles.

Nonphtorealistic rendering converts pictures or 3Dgeometric scenes into nonphotorealistic images. Ratherthan trying to generate results that are indistinguish-able from photographs of an equivalent scene (for exam-ple, photorealistic rendering), nonphotorealistictechniques draw their inspiration from artistic works(see Figure 3) to produce pen-and-ink sketches; cartooncels; or paintings that simulate pencil and charcoal,watercolor, and oil-based brush strokes.2 The proper useof these strokes results in images that guide the eyemuch more effectively than realistic photographs, inpart because artists can make large differences in visu-

al properties coincide with scene locations that theywish to have the viewer spend the most time examin-ing. Some visualization researchers have already start-ed using these ideas as inspiration for painterlyvisualization techniques.3,4 We’re focusing on methodsthat use collections of simulated brush strokes to pro-duce a nonphotorealistic image. We plan to visualize amultidimensional data element with one or more brushstrokes, where the attribute values embedded in the ele-ment control the stroke’s visual properties.

Our study of painterly styles from art history was ini-tially restricted to the Impressionist movement. (Thisterm was attached to a small group of French artists—initially including Monet, Dégas, Manet, Renoir, andPissarro, and later Cézanne, Sisley, and Van Gogh,among others—who broke from the traditional schoolsof the time to approach painting from a new perspec-tive.) Our decision to study the Impressionists was moti-vated in part by a need to narrow our initial focus to asingle painting style; in part by the Impressionists’ care-ful study of color, light, and objects in their paintings;and in part by a personal appreciation of these artists.Properties of hue, luminance, and lighting were explic-itly controlled and even studied in a scientific fashionby some Impressionists.5 Other distinctive examples ofthe artists’ painterly styles include the path of the brushstrokes (such as straight or curved), their length anddensity, the brush used (which affected the strokes’coarseness), and the weight of paint applied to eachstroke.

We observed that many of the styles we discoveredhad a close correspondence to visual features from ourperceptual visualizations. For example, color and light-ing in Impressionism have a direct relationship to theuse of hue and luminance in visualization. Other styleslike path, density, and length have partners like orien-tation, contrast, and size in perception. Taking this intoaccount, we used the following strategy to produce aneffective and aesthetic visualization:

� Produce a data-feature mapping M that uses the per-ceptual color and texture patterns that best representa particular dataset and associated analysis tasks.

� Swap each visual feature in M with its correspondingpainterly style.

Visualization Viewpoints

12 March/April 2002

2 An exampleof changeblindness, theinability toquickly identifysignificantdifferencesacross separateviews of a com-mon scene. Tryto identify thedifferencebetween the two pho-tographs. C

ourt

esy

of R

on R

ensi

nk

3 A close-upview of an oilpainting thatdemonstratesvarious strokestyles, such ascolor, path, size,and density.

Cou

rtes

y of

Lau

ra T

ateo

sian

� M now defines a mapping fromdata to painterly styles that con-trols the visual appearance of com-puter-generated brush strokes;apply this mapping to produce apainted representation of theunderlying dataset.

This strategy successfully gener-ated painterly visualizations of mul-tidimensional data. However, wewere still left to wonder if our visual-izations were effective or aesthetic.

EffectivenessThe guidelines we use to build our perceptual visual-

izations come from psychophysical experiments6 thatmeasure the absolute performance of different visualfeatures, and the interactions that occur between them.We conducted a similar set of experiments but with sim-ulated brush strokes substituted for the original per-ceptual glyphs. This produced painterly images withstrokes that varied in their color, orientation, density,and regularity of placement.

Figure 4 shows examples from the experiments weran. Half of the displays contained a randomly locatedgroup of target strokes defined by a difference in a tar-get style (such as color in Figure 4a and orientation inFigure 4b). Some displays randomly varied a back-ground style (such as orientation during the search fora color target in Figure 4a, and color during the searchfor an orientation target in Figure 4b). This let us testfor visual interference, a situation where variations in abackground style inhibit a viewer’s ability to identify thetarget. We showed each display to a viewer for 200 mil-liseconds, after which we cleared the screen. We thenasked the viewer to answer whether a target group ofstrokes was present or absent in the display.

Our analysis of viewer accuracy mirrored the findingsfrom our original visual perception experiments (seeHealey et al.6 for a complete explanation of theseresults). Salient features in our original perceptual visu-alizations were salient in the painterly images.Interference patterns were also identical. This suggestswe can use our existing rules of perception to buildpainterly representations to effectively visualize valuesin an underlying dataset. It also suggests that any newguidelines we discover could be extended to our painter-ly environment.

AestheticsAlthough our initial experiments showed that our

painterly visualizations are effective, we still had no evi-dence of their aesthetic merit. We ran a new set of exper-iments designed to investigate this property. With theseexperiments, we asked three questions:

� How artistic do viewers judge our painterly visual-izations relative to paintings by artistic masters?

� Can we identify any fundamental emotional factorsthat predict when viewers will perceive an image asartistic?

� Can we categorize individual viewers as preferringdifferent types of art (for example, Realism orAbstractionism), and if so, how do these preferencesimpact the emotional responses that predict artisticrankings?

In our experiments, we asked viewers to order 28images on a scale from 1 (lowest) to 7 (highest). Wepresented seven images from four different categories:master Impressionists (impressionism), master Abstrac-tionists (abstractionism), nonphotorealistic renderings(nonphotorealism), and painterly visualizations (visu-alization).

Figure 5a shows an example of the painterly visualiza-tions we tested. Although the visualizations were show-ing real weather conditions (we represented temperatureby color, wind speed by coverage, pressure by size, andprecipitation by orientation), we provided the viewerswith no explanation about what they depict. We werecareful to zoom in to a point where viewers wouldn’t

IEEE Computer Graphics and Applications 13

4 Example displays from our effec-tiveness experiment: (a) greentarget in orange strokes, randombackground orientation, densepacking, and irregular placement;(b) 45-degree target in 30-degreestrokes, random background color,very dense packing, and irregularplacement.

(a) (b)

5 Exampledisplays fromthe aestheticjudgmentexperiment: (a) a painterlyvisualization ofweather condi-tions and (b) a nonphoto-realistic render-ing of aphotograph ofLake Moraine inBanff, Canada.

(a)

(b)

interpret the image as part of a map. We classified theseimages as abstract in nature, because they had no obviousrelationship to a real-world object or scene. We pairedthem against seven reproductions of real paintings bymaster Abstractionists: one painting each by de Kooning,Johns, Malevich, Mondrain, and Pollock, and two paint-ings by Kline.

Because we derived many of our original painterlystyles from the Impressionists, we wanted to includetheir works in our experiment. We picked seven paint-ings: one each by Cézanne, Monet, Morisot, Pissarro,Seurat, Sisley, and Van Gogh. We counterbalancedthese paintings with seven nonphotorealistic render-ings. Figure 5b shows one of the renderings—a pictureof a mountain lake. We based these images on under-lying photos and applied exactly the same brush strokesand painting scheme that we used to generate ourpainterly visualizations. This was easy to do because aphoto is also a dataset with three dimensions: red,green, and blue.

We asked viewers to rank the 28 images for five dif-ferent questions. The first asked about the images’artistic merit. We designed the other four to probe aviewer’s emotional responses. Each ranking was con-ducted in a similar fashion. For example, during theranking of artistic merit, we asked viewers to look atthe images this way:

As a first step, I would like you to look throughthis entire set of pictures in order to choose onepicture that you like the best. This is a picture thatyou would like to place as art somewhere in yourhouse or at your place of work. It’s the one youthink is the best example of good art.

Now look through the remaining pictures andchoose the one that you think is the worst exam-ple of art.

I would now like you to go through the rest ofthese pictures in the order in which they come upand assign each one a number from 1 to 7. If thepicture is as good as the one you chose to be best,then give it a 7. If it is as bad as the one you choseto be worst, give it a 1. If it is somewhere in-between, then choose an appropriate numberbetween 1 and 7. Remember, 7 represents the bestart. Please use the whole range of numbers to thebest of your ability.

The remaining four questions asked viewers to do anidentical ranking based on how emotionally pleasingthe images were (emotionality), how active they were(arousal), how much meaning they contained (mean-ing), and how complex they were (complexity). Weselected the first two questions using the emotional cir-cumplex, a theory that models eight basic human emo-tions around the two orthogonal dimensions of pleasureand arousal.7 We designed the second two questions tomeasure composition around the two dimensionsabstract–real and visual complexity.

Although our results are still preliminary, we havealready discovered several interesting and exciting find-ings. Of the 25 viewers we tested, 20 ranked the master

Impressionists as most artistic, followed by the nonpho-torealistic renderings, then the master Abstractionistsand painterly visualizations. These viewers consistentlypreferred realistic images over abstract ones. In fact, theyjudged the nonphotorealistic images as more artistic, onaverage, than the Abstractionist paintings. This wasn’tbecause the viewers felt that abstract images were com-pletely lacking in artistic merit (the abstract imagesreceived an average rank of 2.97). They simply preferredrealism to abstractionism. Emotionality and meaningpredicted 90 percent of the variance in viewer respons-es. If viewers who liked realism found an image pleasingand meaningful, they felt it was highly artistic.

Five of our viewers had a different set of rankings.They judged the painterly visualizations as most artistic,followed by the master Abstractionists, then the masterImpressionists and nonphotorealistic renderings. Theseviewers clearly preferred abstract images to realisticones, ranking the visualizations as more artistic thanthe Impressionist paintings. For these viewers, arousalwas the most important predictor of artistic merit. Whenthey found an image active, they tended to feel it washighly artistic.

One final point of interest is that our results show thatcomputer-generated images can be seen as highly artis-tic. Those who preferred realism felt the nonphotoreal-istic renderings were artistic. People who preferredabstractionism felt the painterly visualizations wereartistic. We were initially concerned that viewers mightbelieve a computer-generated image could never be seenas art. Our results suggest that this isn’t true.

Real-world visualizationsTo tie these ideas together, we present a final painter-

ly visualization from the application testbed we’ve usedthroughout much of this article: a dataset of monthlyenvironmental and weather conditions collected andrecorded by the Intergovernmental Panel on ClimateChange. This dataset contains mean monthly surfaceclimate readings in half-degree latitude and longitudesteps for the years 1961 through 1990 (for example,readings for January averaged over the years 1961through 1990, readings for February averaged over theyears 1961 through 1990, and so on).

We visualize temperature, wind speed, pressure, andprecipitation with a data-feature mapping M thatassigns brush stroke color, coverage, size, and orienta-tion, respectively. We show temperature with colorsselected uniformly from a perceptually balanced colorpath that runs from dark blues and greens (for cold tem-peratures) to bright pinks (for hot temperatures). Weshow wind speed with coverage (coverage is the amountof an element’s spatial region covered by its brushstrokes). Coverages range exponentially from very small(for little or no wind) to full (for strong winds). We showpressure with sizes ranging from small (for low pres-sure) to large (for high pressure). Finally, we show pre-cipitation by orientations ranging from 0 degrees orupright (for no precipitation) to 90 degrees or flat (forheavy precipitation).

Figure 6a shows a visualization of data for Februaryacross the eastern US. Figure 6b shows data for the same

Visualization Viewpoints

14 March/April 2002

month along the west coast. Although it’s unlikely thatanyone might mistake these images for realImpressionist paintings, we feel these images containimportant aesthetic qualities that make them stand outfrom traditional visualizations. Color and texture pat-terns representing different weather phenomena can beseen within these images. For example, Figure 6a showsthe expected cold to hot temperature gradient (darkblue strokes to bright pink strokes) running north tosouth, light rain and strong winds (upright strokes thatfully cover the background canvas) in the center of thecountry, and heavy rain and weak winds (tilted strokeswith low coverage) in the south and northeast of theGreat Lakes. Figure 6b highlights the warmer tempera-tures and heavy rainfall (tilted pink and red strokes) typ-ically found in the Pacific Northwest around Seattle andOlympia during the winter months.

The ideas discussed in this article represent the firststeps in our investigation of the roles of perception andaesthetics in scientific visualization. Our ongoing effortsinclude new experiments to study how changing animage along an emotional dimension affects a viewer.For example, we’re testing visualizations with sharp dis-continuities in the brush stroke properties (represent-ing an increase in visual complexity), and we’re alsostudying how an initial explanation of the visualizationschanges a viewer’s rankings (representing an increase inmeaning.) Results from these experiments will offer fur-ther evidence on how variations in an image and its con-text can affect a viewer’s aesthetic judgments.

We’re also pursuing other avenues of investigation.For example, we’re searching for new brush stroke prop-erties that might let us increase the expressiveness ofour nonphotorealistic visualizations. We’re discussinghow the dynamic properties of a computer screen canbe used to animate our paintings, hopefully in ways thatdraw attention to important information in the dataset.We’re optimistic that these ideas will promote furtherinterest in the relationship between effectiveness andaesthetics in scientific visualization. �

AcknowledgmentsWe thank Mark Rempel for his help with data collec-

tion and analysis and for his insightful comments on ourresults. We also thank Laura Tateosian for the landscapeportrait she painted to provide examples of brush strokeproperties. Finally, we thank Ron Rensink for his imagesof change blindness and for his suggestions about theuse of perception for scientific visualization. This work isfunded in part by the National Science and EngineeringResearch Council of Canada, and by US National ScienceFoundation grants IIS-9988507 and ACI-0083421.

References1. R.A. Rensink, “Seeing, Sensing, and Scrutinizing,” Vision

Research, vol. 40, nos. 10–12, June 2000, pp. 1469-1487.2. B. Gooch and A. Gooch, Non-Photorealistic Rendering, AK

Peters, Natick, Mass., 2001.3. D.H. Laidlaw, “Loose, Artistic ‘Textures’ for Visualization,”

IEEE Computer Graphics and Applications, vol. 21, no. 2,Mar.–Apr. 2001, pp. 6-9.

4. P. Rheingans and D. Ebert, “Volume Illustration: Nonpho-torealistic Rendering of Volume Models,” IEEE Trans. Visu-alizations and Computer Graphics, vol. 7, no. 3, July–Sept.2001, pp. 253-264.

5. M. Schapiro, Impressionism: Reflections and Perceptions,George Grazillier, New York, 1997.

6. C.G. Healey and J.T. Enns, “Large Datasets at a Glance:Combining Textures and Colors in Scientific Visualization,”IEEE Trans. Visualizations and Computer Graphics, vol. 5,no. 2, Apr.–June 1999, pp. 145-167.

7. L.F. Barrett and J.A. Russell, “The Structure of CurrentAffect: Controversies and Emerging Consensus,” CurrentDirections in Psychological Science, vol. 8, no. 1, Jan./Feb.1999, pp. 10-14.

Readers may contact the authors at the Dept. of ComputerScience, 452 EGRC #7534, North Carolina State Univ.,Raleigh, NC 27695-7534, email healey@csc.ncsu.edu.

IEEE Computer Graphics and Applications 15

(a) (b)

6 Painterlyvisualization ofweather condi-tions over thecontinental US:(a) weatherconditions forFebruary overthe eastern USand (b) weatherconditions forFebruary overthe west coast.