Special Section

CAA 2005: Hybridity: Arts, Sciences and

Cultural EffectsGuest Editors: Yvonne Spielmann

and Jay David Bolter

Presenting selected papers from a special section of the 93rd Annual Conference of the College Art Association, Atlanta, GA, 16–19 February 2005.

ARTISTIC PRACTICE IN TRANSITIONDigital media-arts practice can be considered a hybrid artform, with distinctive roots in a number of historically sepa-rate knowledge bases. It coalesces the aesthetic, plastic andformal methodologies of art with the physicality of electronichardware material, the procedural logical language of math-ematically defined symbolic code and the systematic processesof information management. Media arts is a discursive disci-pline that through aesthetic means reflects upon how tech-nology transforms culture. I can easily trace the influence ofthe development of digital technologies in my transition fromresearch into the syntax and visual impact of the photographicimage to my current artistic works, which rely extensively oninformation-science methodologies as they explore the col-lection, digitization, processing and visualization of data. Thetransition from the analogue to the digital has reformulatedconceptual and practical methods of artistic engagement withthe materials, tools and project directions, shifting greater em-phasis to the experiential and to the recalibration of culturalmeaning into information. My recent projects rely on mathe-matically generated organizational methods and visualizationsto explore how they might reveal unexpected and new formsof expression. Motion-sensing technologies have enhanced in-teractivity in my public installations by transforming the pas-sive viewer into an active contributor.

As my projects have increased in scale and technologicalcomplexity during the past 15 years, my artistic-productionmode transformed itself from individual to a team-basedmodel, with specialists and knowledgeable assistants con-tributing to the work. Under such circumstances, the role ofthe artist shifts from that of an individual developer to one ofproject designer and manager, focusing on conceptualization,planning and production supervision. This is a working ap-proach familiar to those in the music, architecture, film andtheater disciplines, but relatively new to the visual arts. Implicit

in such collaborative situations isthe understanding that no individ-ual can have all the necessary knowl-edge to fully realize a project. Thecollaborative approach leads to newinsights and unexpected results asthe work evolves; the challenge is tomaintain the integrity of the origi-nal concept. Given the flexibility ofsuch a situation, the artwork be-comes a research opportunity forthe participants as they explore howtheir knowledge and approaches can synthesize with those ofthe other contributors. The development of digital media art can comprise many highly diverse disciplines. In the caseof the Pockets Full of Memories project I realized in 2001 for the Centre Pompidou [1], the team of experts included a cognitive-science natural-language scientist from Helsinki, anindustrial/organization psychologist from Germany, a visual-communication design team from Germany, a productionteam led by a database and hardware design engineer fromBudapest, an interaction designer from Germany and a webengineer from the Center for Research in Electronic Art Tech-nology (CREATE) lab at the University of California at SantaBarbara (UCSB). The critical components for the success ofsuch projects are common ground and a shared understand-ing of the projects’ goals and how to realize them as quickly aspossible. Consequently, one of the implicit goals of the intro-ductory core courses in the UCSB Media Arts & Technology(MAT) graduate program is to learn to communicate betweendifferent areas of expertise.

UCSB’S MEDIA ARTS & TECHNOLOGY PROGRAMInterdisciplinarity has been at the core of the MAT programsince its inception in 2000. The chair reports to the Dean ofLetters and Science (Humanities) and the Dean of Engineer-ing. Students apply to one of three study areas: multimedia en-gineering, sound/music or visual/spatial arts; but all cyclethrough a set of courses that promote interdisciplinary learn-ing with the expectation that they will hybridize their research.Students enter the program with diverse backgrounds such asarchitecture, visual communication, computer science, elec-trical computing engineering, electronic sound research, sig-nal processing and contemporary music composition. Incontrast to a few years ago, recent applicants possess basic

©2006 ISAST LEONARDO, Vol. 39, No. 3, pp. 214–218, 2006 215

Perspectives on Collaborative Research and Education in Media Arts

George Legrady

George Legrady (media artist), Media Arts & Technology Graduate Program, University of California, Santa Barbara, CA 93106, U.S.A. E-mail: <legrady@arts.ucsb.edu>.Web site: <www.georgelegrady.com>.

Based on a paper presented at “Hybridity: Arts, Sciences and Cultural Effects,” a special section of the 93rd Annual Conference of the College Art Association, Atlanta, GA, 19–16February 2005.

A B S T R A C T

Digital arts is by nature ahybrid practice, integrating thepoetics, aesthetics and concep-tual strategies of art with thelogical, systematic methods oftechnological processes fromengineering and the sciences.This article reviews the develop-ment of interdisciplinary, collabo-rative arts-engineering researchand education at the Universityof California at Santa Barbara,focusing on the Media Arts &Technology graduate programfrom a visual/spatial artsperspective.

CAA

20

05

Article Frontispiece. The CNSI AlloSphere, a fully spherical, immersive audiovisual environment, is situated in MAT’s newresearch facilities in the California Nanosystems Institute Buildingat the University of California at Santa Barbara. (© UCSB)

computer programming competenceeven though many may not have themathematics fundamentals necessary for a graduate engineering degree. MostMAT faculty have joint appointmentswith departments that represent their re-search areas. These consist of computerscience, electrical computing engineer-ing (ECE), music, art and film studies. Allof the visual/spatial arts faculty (LisaJevbratt, Marcos Novak, Marko Peljhanand myself) hold joint appointments inthe Department of Art [2] and entertaina broad range of interests that includedata mapping and visualization, inter-active installations, telecommunicationsand locative media, wireless technolo-gies, database aesthetics, networks, inter-action design, digital 3D, virtual reality,transarchitecture, mixed-reality immer-sive environments, algorithmic aesthet-ics, biological systems and media theory.

Media Arts & Technology is closelyconnected to a number of research labs,such as CREATE [3], which was estab-lished in 1986 by former MAT depart-ment chair JoAnn Kuchera-Morin, whoholds a joint affiliation with the Depart-ment of Music. The associate director of

dia; four of the five co-principals are MATfaculty, with Manjunath being the pri-mary principal. The multimedia IGERTinitiative at UCSB is an interdisciplinary, collaborative research program fundingdoctoral student research in the areas of (a) multimedia systems (storage, trans-mission and programming of digital multimedia); (b) multimedia content(representation and analysis of multi-media information and advanced toolsfor creating, analyzing and manipulatingmultimedia content); and (c) interactiv-ity (presentation of, and human interac-tion with, multimedia information in thecontext of interactive multimedia appli-cations). The NSF IGERT program hasrecently been identified by the NationalAcademy of Sciences as an interdiscipli-nary success story in its report “Facilitat-ing Interdisciplinary Research” [7].

The MAT program is in the planningstage of moving into new research facili-ties at the California Nanosystems Insti-tute (CNSI) [8], which is to take place in2006. The facilities will include a numberof research-specific labs, such as a mo-tion-capture lab to track, model and rec-ognize human movement and gesture in

CREATE is Curtis Roads. CREATE servesas a productive environment available tostudents, researchers and media artistsfor the realization of music and multi-media works. The center also serves as a laboratory for research and develop-ment of a new generation of software andhardware tools to aid in media-basedcomposition. On the engineering side,MAT and Computer Science professorMatthew Turk, who is now chair of MAT,is co-director of the Four Eyes Lab [4],which focuses on research in imaging, in-teraction and innovative interfaces, a mixof computer vision, augmented reality,mobile and wearable computing, and hu-man-computer interaction. B.S. Manju-nath, a MAT/ECE professor, directs theVision Research Lab [5], which focuseson multimedia signal processing andanalysis, including applications in largemultimedia databases, bio-image infor-matics, digital libraries, steganography(data hiding) and image processing. TheMAT program is closely aligned with theNational Science Foundation–funded In-tegrated Graduate Education and Re-search Training (IGERT) program [6],focused on interactive digital multime-

216 Legrady, Collaboration in Media Arts

CAA

20

05

Fig. 1. Ben Ritter, Ashok Basawapatna and Uli Schmidts, The Shape of Discourse, Truncated Octahedron, 2004. (© Ben Ritter, Ashok Basawapatna and Uli Schmidts) The Shape of Discourse visualizes the degree of connectedness of a list of 64 architects in two ways: as a 3D,dynamically clustering assortment of cuboctahedrons and as a 2D map of interconnected lines [13].

various contexts, including office/work-station scenarios, performance/controlscenarios and everyday life scenarios; andan interactive installation space to createmultimedia engineering projects that testthe limits of visual, gestural and audio sys-tems, as well as digital visual art and mu-sic installations and audio immersive andtouch-sensitive artworks. The DistributedSystems, Robotics & Display I/O DeviceLab is a space for research on distributedmultimedia software environments andmultimedia communications and signalprocessing, configuring multimedia com-puter systems and integrating new de-vices (helmets, gloves and wands with six degrees of motion interaction) as well as constructing them. The lab will bea focal point for the MAT researchers’various expertise in distributed multi-media computing systems, multimedianetworking and communications, dis-tributed processing environments andmedia signal processing. The Pluri au-dio/video laboratories are production focused for combining multi-channel audio and multi-screen video. The Allo-Sphere (Article Frontispiece) is a 3-story-high, enclosed, fully immersive 360˚virtual visual and audio environmentused for advanced research and pro-duction applications. The plan for the AlloSphere is to allow creation of virtualsimulations that completely immerse theviewer. The CNSI at UCSB comprises faculty, students and researchers from 11 different academic departments. TheMAT facilities will be employed for arts-engineering projects and research. Therelationship of present to future media is of particular interest, especially in

domain representation and perceptualdistortion. My lectures address methodsof artistic-project development, the na-ture of narrative, formal aesthetics, time-space design and other specific aspectsof the production of an interactive in-stallation. Technological topics coveredfocus on visual perception and deliveryof real-time televisual signals, with stu-dent-driven implementation of 3D de-vices to control remote cameras and torecord viewers’ actions. Our future goalsare to develop projects to study and meas-ure audience responses to insertion ofcontrolled distortions in real-time video.

For the production phase of studentprojects, we are interested in two-personteams consisting of an engineer and an artist. Weekly lectures are followed by small project assignments in which students must demonstrate an under-standing of the week’s lectures througha project realized in the software appli-cation of their choice. During the secondhalf of the course, they are expected torealize a final project that synthesizes arts and engineering knowledge withresults meaningful to both perspectives(Figs 1,2).

The past two courses revealed that stu-dents approached the projects’ planningand development based on worldviewsand problem-solving approaches definedaccording to training in their individualdisciplines. Whereas the engineers wereinterested in technical problem-solvingopportunities, one psychology studentwas interested in visual-perception issues,and the arts students tended to focus onthe cultural aspects of media messages.The artists’ interests in the difficult-to-

the fields of nanotechnology, biotech-nology, new materials and new fabrica-tion methods.

MULTIDISCIPLINARITYIN THE CLASSROOM: SIGNAL PROCESSINGAND ART PRACTICEInterdisciplinarity is a challenging pros-pect, not only for students who have yetto fully master their discipline of choice,but also for the teaching faculty with theirspecialized areas of expertise. It is diffi-cult to cross disciplinary boundaries andachieve results, but it is also a great op-portunity to expand horizons and learnabout different ways to perceive, defineand address problems.

In the spirit of bringing arts-engi-neering interdisciplinary studies into theclassroom, my colleague Jerry Gibson ofMAT and ECE proposed a few years agothat he and I teach a graduate course thatwould combine our knowledge bases (en-gineering signal processing and media-arts project production) as a way to fosterengineering-level research in conjunc-tion with the experimental approach ofthe visual arts. We have now taught MAT256: Visual Design through Algorithmswith Explorations in Visual Perception[9] twice and plan to do so again in thecoming year.

The course consists of weekly lecturesby both faculty on subject matter specificto our individual areas of expertise. En-gineering signal-processing topics taughtby Gibson include sampling, Markovchains, mutual information, fundamen-tals of information theory, frequency-

Legrady, Collaboration in Media Arts 217

CAA

20

05

Fig. 2. August Black and Lance Putnam, Worm Walker, 2004. (© August Black and Lance Putnam) Visual examples of a result of Worm Walker,which uses frequency information from an image to control a separate stochastic walk based on an image’s pixel data [14].

measure noise of cultural messages andsubtexts stood in contrast to the engi-neering desire for a purer signal: the col-lection of reliable and measurable data.

EVALUATION ANDINTERPRETATION IN ANINTERDISCIPLINARY CONTEXTThe scientific literature in research jour-nals speaks of the necessity and wisdomof interdisciplinary collaboration and coordination in IT research. The NSF en-dorses this approach: “Today the bound-aries between all disciplines overlap andconverge at an accelerating pace. Prog-ress in one area seeds advances in an-other” [10]. Crossing interdisciplinaryboundaries in the sciences is challeng-ing enough. What, then, to make of artistic-engineering interactions? By whatmethod does one identify, define andperhaps measure value? In the classroom,I am continually reminded that my engi-neering colleague delivers informationthat is grounded in proven measurablefacts, whereas my observations as an artistare at best opinions, formed through ex-perience but without the authority of sci-entific testing.

It is difficult to convey to non-special-ists the rules by which works of art areevaluated, as they exist at the implicitlevel and are continuously shifting. Themeanings and evaluations of works of artdo not generally follow clear and prede-termined means of understanding andevaluation. Meaning in an artwork oper-ates at multiple levels, including those of metaphor, material form, contextualsituation, nature of production, institu-tional affiliation, market value, etc. Thesituation is quite slippery in relation to the quantifiable scientific paradigm.Artistic and related practices such as de-sign and architecture focus on contextualrelationships, where the meaning of thework may lie outside itself. Consider, forinstance, a particular building’s design as a response to its cultural situation defined through the collection of dataabout occupants, traffic fluctuations,noise levels, surrounding buildings, po-litical and economic implications, etc.Art-oriented production requires acuteunderstanding of such things as the con-text in which it is to be situated, how it is to be interpreted and what rules it fol-lows or twists inside out. A work of art is therefore always in conversation withthe conditions of its production and

and scientific research stays firmly fo-cused on its intended research goals.Artistic projects tend to be open-ended,having the luxury to deviate from theirintended course to arrive at an un-planned outcome in cases where the fi-nal results have not been committed to,as in public commissions. The artist ad-ditionally benefits from the flexibility todeviate to pursue a project that may notbe clear at all in its essence until it reachescompletion. In the short run, the free-form arts approach may not contributeto resolution of the research problem athand but may nonetheless have an im-pact down the road by opening up newvistas that might benefit collaborative re-search in the future.

References

1. Brigitte Steinheider and George Legrady, “Inter-disciplinary Collaboration in Digital Media Arts: APsychological Perspective on the ProductionProcess,” in Special Section “ISEA 2002 Selected Pa-pers,” Leonardo 37, No. 4, 315–321 (2004).

2. <www.arts.ucsb.edu>.

3. <www.create.ucsb.edu>.

4. <http://ilab.cs.ucsb.edu/>.

5. <http://vision.ece.ucsb.edu/>.

6. <http://media.igert.ucsb.edu>.

7. <www.nap.edu/catalog/11153.html>.

8. <www.cnsi.ucsb.edu/index.html>.

9. <www.mat.ucsb.edu/~g.legrady/academic/courses/05w256/05w256.html>.

10. Introductory quote by Rita Colwell, Director,NSF, “Crossing Boundaries—Interdisciplinary Science,” <www.nsf.gov/od/lpa/forum/colwell/rc030220aucklandnz.htm>.

11. C. van Bruggen, John Baldessari (New York: Riz-zoli, 1990) p. 33.

12. Jerry Gibson, e-mail, 31 January 2005.

13. <www.mat.ucsb.edu/~g.legrady/academic/courses/04w594/toh/index.html>.

14. <www.mat.ucsb.edu/~g.legrady/academic/courses/04w594/wormwalker/index.html>.

George Legrady is a professor of interactive me-dia in the Media Arts & Technology gradu-ate program at the University of California atSanta Barbara. He has previously held ap-pointments at the Merz Akademie, Stuttgart,San Francisco State University, the Universityof Southern California, the California Insti-tute of the Arts and the University of WesternOntario. His interactive installations havebeen exhibited internationally, most recentlyat the Kiasma Museum of Contemporary Art,Helsinki; Ars Electronica; DEAF Media Fes-tival, Rotterdam; the Centre Georges Pompi-dou; and the San Francisco Museum ofModern Art.

its institutional history. The artist JohnBaldessari aptly summed it up in the work“Painting for Kubler,” which consisted ofthe following sentences on canvas:

This painting owes its existence to priorpaintings. . . . Ideas of former paintinghad to be rethought in order to tran-scend former work. To like this paintingyou will have to understand prior work.Ultimately this work will amalgamatewith the existing body of knowledge [11].

Whereas the artist is by nature a gen-eralist, as his or her work often dependson the juxtaposition of sampled infor-mation from a wide spectrum of sources,the scientific research model operates atthe micro and specific level. Research ad-dresses a specific problem, and when asolution is achieved, it requires the ex-periment to be repeatable. In contrast,an artwork’s value lies in its unique ap-proach in addressing a problem, and toreduce it to its integral components tounderstand its inner workings may resultin the dissolution of its essential qualitiesin the process.

The interdisciplinary, collaborativeprocesses and interactions of our team-taught course became a topic of discus-sion itself. My engineer colleague Gibsonvoiced three pertinent concerns: (a) Theinteraction between the arts and engi-neering faculty members out of class is as important as what happens in class. (b) Having both faculty members pres-ent at all class sessions allows the facultyto interact and the students to observethis interaction and benefit. (c) The engineering faculty should feel free topresent very technical topics withoutoversimplification of the ideas involvedas long as there are visual and audiodemos to illustrate the concepts. Thenontechnical students are thus exposedto more rigorous definitions of topicssuch as complexity, information, noise,filtering, etc., although they cannot de-rive the result or perhaps write out theequations [12].

We have learned from each other thateach discipline accepts divergences in re-search directions at the early stages, butthere are differences along the way. Gib-son has described a recurring situationin basic research in which a proposed en-gineering research plan may open up unexpected insights leading to a refor-mulation or redirection of the research.Once a commitment to a research topicis agreed upon, however, engineering

218 Legrady, Collaboration in Media Arts

CAA

20

05