Jacobs University Bremen School of Humanities and Social Sciences

Handbook

PhD-Program Visual Communication & Expertise (VisComX)

29. September 2010

1

Content 1  Introduction to VisComX ....................................................................................... 2 

1.1  Concept ........................................................................................................ 2 

1.2  Degree .......................................................................................................... 3 

1.3  Target Audience ........................................................................................... 3 

2  Study Plan ............................................................................................................ 3 

3  Available Courses ................................................................................................ 6 

3.1  Involved Faculty ........................................................................................... 6 

3.2  Regular Course Offerings ............................................................................. 6 

3.2.1  VisComX in Digital Contexts: Intro-Workshop .......................................... 6 

3.2.2  VisComX Retreat ...................................................................................... 6 

3.2.3  VisComX Lecture Series .......................................................................... 7 

3.2.4  VisComX Research Colloquium ............................................................... 7 

3.2.5  VisComX - Disciplinary Foundations ........................................................ 7 

3.2.6  Advanced Visual Workshop ...................................................................... 7 

3.2.7  Transdisciplinary Visuality ........................................................................ 7 

3.3  Infrastructure for PhD-Research Projects ..................................................... 8 

3.3.1  PIAV (Political Iconographic Archive of Vision) ........................................ 8 

3.3.2  KiVi (Key Visuals Online Database) ....................................................... 10 

3.3.3  Eye Tracking Laboratory ........................................................................ 10 

3.3.4  Emotion Physiology Lab ......................................................................... 11 

3.3.5  VisBench ................................................................................................ 12 

3.3.6  Facilities at Universität Bremen and Hochschule Bremen ...................... 12 

4  Collaborations .................................................................................................... 13 

4.1  Duke University – Visual Studies Initiative (VSI) ........................................ 13 

2

1 I1.1 At theimpotradittelecointo ajourninflueembrtechnadvevisuacommProgr

Figurproce The t VisuaVisuaapprosciendistribmediaculturempibackgundeproce

IntroducConcept

e beginningrtance in allional publiommunicatioa “news phoalism. Visu

encing decisrace new menology into rtising, and

alization. Tmunication pram.

re 1. VisComXess areas that

two key term

al Communal communicoaches that nce and engbution, usagated and iral, economrical, social ground, visrstanding an

esses, as we

ction to t of the 21st areas of lifcation patton networksotographer”,uals elicit sion-makingedia, integraartwork tha

d user intehe investigprocesses in

X Visual Proct are investiga

ms of the Res

nication cation is an

range fromgineering. Tge, and recnterpersona

mic, historicscientific tra

ual commund explaininell as their im

VisCom

t century, vife on a globterns; digitas has the po, thereby crboth cogni

g and comating video, at in turn ierfaces. Sogation of n a global s

cess Model ilated by VisCo

search Cent

expanding fm the sociaThese approception procally communcal, politicaladition that

unication resng current vmplications fo

mX

isual commubal scale. Dial photograotential to tureating a netive and emunication digital art, nfluences toftware app

these comsetting is at

llustrating theomX members

ter are define

field of transl sciences t

oaches aim cesses, as nicated visul, and sociis nonethel

search is pisual phenoor the future

unication angital technoaphy coupurn every b

ew kind of memotional r

patterns. Chard- and sthe aesthetiplications implex inter

the focus o

e interdependers.

ed as follow

sdisciplinaryto the humaat explainin

well as theuals in conial contextsess based problem-orie

omena and ve.

nd expertiselogy has traled with uystander of

mobile multimreactions, oContemporasoftware, traics of enterncreasingly rrelations oof the VisCo

ent relationsh

ws:

y research, canities as wng visual p

e meanings ntemporary s. Influenceon a multidented and visual comm

is of key ansformed ubiquitous

an event media lay oftentimes ary artists nsforming rtainment,

rely on of visual omX PhD

hips of four

combining well as to roduction, of mass-aesthetic,

ed by an isciplinary aimed at

munication

3

Visual Expertise Visual expertise is understood as the capacity to see and perceive visual stimuli, to interpret their meaning and to form and maintain a visual representation. It reflects both an individual as well as a collective human capacity. The capacity is not static. Rather, the degree of visual expertise is determined by attributes of the visuals, such as their complexity, originality or reproductive surface, materiality, and importantly, it is modulated by aesthetic, cultural, organizational, physiological, psychological, and technological conditions and resources as well as their interaction. Visual expertise in this sense takes into account both the impact of new technologies and new media on arts, society, and the individual user as well as the reciprocal effects of users and user habits on subsequent technological developments. Visual expertise in the context of this Research Center is thus examined from the perspective of multiple disciplines, including but not restricted to art history, mass communication, psychology, as well as from the science and engineering perspective.

1.2 Degree Upon graduating in the VisComX PhD-program, Jacobs University Bremen awards the degree Doctor of Philosophy (PhD). Depending on the area of specialization of the dissertation, the PhD is awarded in one of the following disciplines: Communication Science, Psychology, Art History, Computer Science, or Visual Studies.

1.3 Target Audience The target audience of the VisComX PhD program are excellent graduate students who have completed their Master’s degree on a visual topic and would like to continue their education in an interdisciplinary research environment. Graduates of Jacobs University’s Global Visual Communication (GVC) Master program who have completed their MA-degree with an excellent grade will be eligible for continuing their education in the VisComX-PhD-track. Additionally, PhD-students enrolled at the Visual Studies Initiative (VSI) of Duke University (see section 4.1.) will participate in the VisComX PhD-program in the 3rd semester as exchange students.

2 Study Plan The PhD program is an integral part of the VisComX Research Centre (www.jacobs-university.de/viscomx). The Centre was founded in 2009 at Jacobs University with the aim to promote collaborative visual research and understanding of pressing problems in the contemporary world by uniting aesthetic, cultural, historical, psychological, social scientific and technological experts. Sharing its research agenda with the Centre, the PhD program stands at the forefront of cutting-edge transdisciplinary research in the field of visual communication. It is a structured 3-year program. For researching the heightened complexity of visual communication processes, the PhD program draws upon an integrated, transdisciplinary approach combining and interrelating four core disciplines: communication science (including media studies) as nexus discipline is connected to art history, experimental psychology as well as computer science (see Fig. 2). The four main research areas are built upon the expertise in these four disciplinary fields, thus capturing the full visual process: 1. Visualization, 2. Perception, 3. Interpretation, 4. Evaluation (see

4

Figure 1 as well as the multidisciplinary architecture of the VisComX PhD program in relation to research foci below). PhDs are offered in all of the contributing disciplines, but the area of study will always be cross-disciplinary. This transdisciplinary profile is directly inserted into the supervision structure, thereby guaranteeing innovative research in the individual PhD projects. Over the full period of three years, each dissertation project is co-supervised by two professors from different disciplines, ideally the core disciplines of the research program. Figure 2. VisComX Architecture Connecting Disciplines and Research Fields The program is structured by qualification modules that consist of different course offerings, including the regularly offered VisComX Research Colloquium, and varying academic exchange formats. The program consists of three years of PhD study. Major work includes the program-guided development, elaboration, and finalization of the research plan, resulting in a dissertation. Every student in the program develops a PhD proposal in cooperation with two PhD-advisors from two different disciplinary fields. This proposal is a written document and presented in public upon invitation by the Dean at most eight months after the qualifying exam or after entrance into the PhD Program. The Dissertation Committee approves the PhD proposal, which is required not to exceed 6000 words. In addition, active participation in all program activities is required. As part of their training, supervisees are expected to assist in teaching and in research activities and to participate in the preparation of further program events. The PhD program is tailor-made for qualifying and professionalizing graduate students to become leaders in academic and applied contexts of visual communication. We are offering a highly stimulating environment with excellent facilities for social and behavioral research, profound training in visual methods and theories, visualization and statistics and close contact to supervisors in multiple disciplines. The study plan comprises a three-year period, consisting of a taught component of a minimum of 3 and a maximum of 4 semesters, followed by a research-only period devoted to completing the PhD-thesis (see Table 1). The 4th semester is envisaged as an exchange semester where VisComX-students attend courses at our collaborating partner institution: VSI-Duke University (see section 4.1).

5

Semester Attendance Course Title 1st

Mandatory - Visual Communication and Expertise - Intro-Workshop - VisComX Retreat - VisComX Lecture Series - Advanced Visual Workshop (“Neuroscience and Art”) - VisComX Research Colloquium

Optional - General Information and Communication Technology I - Non-Western Influences in European Avantgarde Art - Introduction to Cognitive Psychology - Introduction to Social Psychology

2nd Mandatory - Disciplinary Foundations: Communication Science - Disciplinary Foundations: Art History - Transdisciplinary Visuality (USC) - VisComX Lecture Series - Advanced Visual Workshop (“The Art of Illustration”) - VisComX Research Colloquium

Optional - Advanced Quantitative Methods - General Information and Communication Technology II

3rd Mandatory - Disciplinary Foundations: Computer Science - Disciplinary Foundations: Psychology - VisComX Retreat - VisComX Lecture Series - Advanced Visual Workshops (“FACSGen”, “Eyetracking”, Webdesign”) - - Transdisciplinary Visuality (ISC) - VisComX Research Colloquium

Optional - Advanced Qualitative Methods

4th Optional Exchange Semester – Visual Studies Initiative Duke University/USA

5th Research-Only – Finalization of PhD-project - VisComX Research Colloquium

6th Research-Only – Completion of PhD-thesis, PhD-defense - VisComX Research Colloquium

Table 1. Structure of an Example of the VisComX PhD Study Plan. Specific Advanced Workshop titles might vary from semester to semester; placement of particular courses in respective semesters might vary. Upon completion of the 3-year-program graduating students of VisComX will have acquired an exceptional expertise and training in the four disciplinary pillars of the Research Center – communication science, art history, psychology and computer science – in addition to the specialized knowledge in their respective field in which they obtain the PhD-degree. The VisComX-PhD-program thus offers a worldwide unique transdisciplinary education with excellent career opportunities on an increasingly globalized academic labor market.

6

3 Available Courses 3.1 Involved Faculty The following faculty members from Jacobs University are involved in the program: Prof. Dr. Peter Baumann (Computer Science), SES Prof. Dr. Werner Bergholz (Electrical Engineering), SES Prof. Dr. Benjamin Godde (Neuroscience), JCLL Prof. Dr. Otthein Herzog (Visual Information Technologies), SHSS, Technologie-

Zentrum Informatik und Informationstechnik (TZI) Prof. Arvid Kappas, PhD (Psychology), SHSS Prof. Dr. Ing. Lars Linsen (Computional Science and Computer Science), SES Prof. Dr. Peter Ludes, PhD (USA) (Mass Communication), SHSS Prof. Dr. Birgit Mersmann (Non-Western and European Art), SHSS Prof. Dr. Marion G. Müller (Mass Communication), SHSS Prof. Bettina Olk, PhD (Psychology), SHSS Prof. Dr. Margrit Schreier (Empirical Methods in the Humanities and Social

Sciences), SHSS Prof. Dr. Welf Werner (International Economics), SHSS Prof. Dr. Adalbert Wilhelm (Statistics), SHSS Prof. Dr. Isabel Wünsche (Art and Art History), SHSS

3.2 Regular Course Offerings 3.2.1 VisComX in Digital Contexts: Intro-Workshop

Offered for every new cohort at the initial beginning of the PhD-program, providing an overview of the structures, goals and ongoing research projects in the Research Center. Course Description: Introductory high-intensity workshop on structures, programs and perspectives of the research center from disciplinary and transdisciplinary perspectives, introducing Communication/Media, Art History, Psychology and Computer Science and their approaches to visual communication and expertise as well as the areas of study involved. - Mandatory - Participants: First semester PhD-students - First semester (Fall)

3.2.2 VisComX Retreat

Course Description: Strategic planning of future VisComX curriculum, research program and research agenda, development of specific research projects, including a collaborative workshop defining areas of interest for VisComX and developing ideas for particular multidisciplinary proposals and international collaborations. The junior fellows will be familiarized with the planning stages of large-scale research proposals, the constraints and challenges of collaborative research in the national and international funding environment. The VisComX Retreat thus provides valuable insights and training in micro-processes of scholarly collaboration and communication. - Mandatory - Participants: All VisComX Fellows – all enrolled VisComX PhD-students - Once per year (typically in fall semester)

7

3.2.3 VisComX Lecture Series

Course Description: Annual series of invited guest lectures by prominent international scholars in the visual field representing a multiplicity of disciplines and connecting both VisComX and particularly the junior fellows/PhD-students to the international academic community. - Voluntary/Optional - PhD-VisComX, MA-GVC - Each semester

3.2.4 VisComX Research Colloquium

Course Description: The purpose of the VisComX Research Colloquium is to familiarize students with concrete problems in the actual practice of visual communication research. The attendance and active participation in the Research Colloquium is mandatory. - Mandatory - PhD-VisComX, MA-GVC - Each semester

3.2.5 VisComX - Disciplinary Foundations

Course Description: This course offers an introduction into each of the four disciplinary fields and the necessary disciplinary foundations for transdisciplinary collaboration in the VisComX Research Center: Communication/Media Studies, Art History (Bildwissenschaft), Experimental Psychology and Computer Science. The course format can vary from semester to semester (Seminar, Lecture, High-intensity workshop). Four disciplinary introduction courses will be offered, ideally during the first three semesters of the PhD-program, depending on curricular constraints. During the first two semesters the course “Visual Communication” [Course 900482] has to be taken as the disciplinary introduction into communication science/media studies. - Mandatory - 4 courses - PhD-VisComX - first two years of study

3.2.6 Advanced Visual Workshop

Course Description: The workshops are a specific offering to deepen the understanding of a specific interdisciplinary issue or disciplinary approach to visual communication and expertise. Internal or external experts provide access and training to advanced knowledge and practice, as a result enabling students to further develop their own projects. - Mandatory - PhD-VisComX - 3-6 workshops offered in the taught component, depending on demand and supply

3.2.7 Transdisciplinary Visuality

This course which may be offered in the form of a University Studies Course (USC) or as an Independent Studies Course (ISC) is a guided research course highlighting the connections and links between different disciplines in the field of visual

8

comm(USCparticcomp- M- P- O

3.3 3.3.1

AmonMülleand cartoanno2.000

Figur Basiccollabdevedigitizexpecorpodeve

munication aC or ISC) cularly recomponent of theMandatory PhD-VisComXOne VisComX

Infrastru PIAV (Po

ng the resoer’s corpus onewspaper ons as well tated accord

0 images are

re 3. “Zettelkä

c research boration of lop thesaurized visual arimental arcora of visualoping new a

and expertisduring theimmended ae PhD-progr

X X-USC offer

ucture forolitical Icono

ources availaof the visuaclippings –as print andding to a poe available in

ästen” Politica

conductedcommunica and metad

archives. Thchive index al material annotation to

se. Studentsr enrollmen

after the 1st

ram.

red annually

r PhD-Reographic Ar

able to the l archive PIA

– mainly pred art reproduolitical-iconon digitized fo

al Iconographi

d at the Rtion scholar

data categorhesauri like to test and like e.g., pools (see Fig

s are requirnt. Transdist semester,

(varying se

esearch Prchive of Vi

VisComX PAV with appess photogructions – collographic indormat, includ

ic Archive of

Research Crs with linguries for imprthe PIAV-Indevelop ne

press photoggure 4).

red to take sciplinary V

towards th

mesters), IS

Projects sion)

PhD-programproximately raphy, but alected over tex (see Tab

ding annotat

Vision (PIAV)

Center Visuists and coroved retrievndex (see Tew algorithmgraphy whic

at least onVisuality couhe end of th

SC upon dem

m is Prof. M15.000 newalso archivathe past 15 yble 2). Apprion (see Tab

)

ComX invoomputer scival and annTable 2) serms and analch is categ

ne course urses are he taught

mand

Marion G. ws images al photos, years and roximately ble 2).

olves the ientists to

notation of rve as an lyze large orized by

9

Tablecateg

FigurJerichRabincente

e 2: Excerpt gories based o

re 4: Segmeho-Agreemenn (left) and Pr, September

from the PIAon the iconog

ented press pt, signed at tPalestinian ler 1993.

AV-Index – a graphic motive

photograph “the White Hoeader Yassir

thesaurus coes of images,

“Handshake” ouse between

Arafat (righ

omprising momainly press

(PIAV-index n then Israeli t), US-Presid

ore than 250 s photographs

A.4.3.a) at Prime Minis

dent Bill Clin

annotation s.

the Gaza-ter Yitzhak

nton at the

10

3.3.2 KiVi (Key Visuals Online Database)

There are editing facilities for still and moving images, image retrieval/database software and hardware, including an Enterprise Media Asset Management Platform (Blue Order Media Archive ®), a video annotation tool (KIVI, Fig. 5), which allows the coding of video segments on different layers and the export and analysis of the code, as well as image synthesis possibilities. The multimedia services of Jacobs University’s Information Resource Center provide a professional Media Content Manager, which has been supplemented by a team from Mass Communication/Integrated Social Sciences (Prof. Peter Ludes, Jan Müller) and the Center for Computing Technologies of Bremen University, TZI (Prof. Otthein Herzog, Martin Stommel) by a special archiving and coding system for intercultural media content comparisons. More than 1.000 video tapes of US-American and German TV information programs since 1949 and more than 100 TV annual reviews from Brazil, China, India, Germany, and the United States provide a unique moving visual database.

Figure 5: Screenshot from KiVi database

3.3.3 Eye Tracking Laboratory

The laboratory consists of specialized and general purpose rooms, among those several rooms with PC workstations, including special software (Superlab, Medialab, Matlab, C++), to be used for behavioral experiments. The specialized eye tracking laboratory is equipped with an EyeLink II (SR Research, Canada) eye tracker (see Fig. 6), which consists of a headset with three cameras that track the movements of the pupils and the head with a very high temporal (500 Hz, recording every 2 ms) and spatial (typically 0.3 degrees) resolution. The recording of head position is crucial as it allows separating measurements of eye and head movements.

11

Figurrestin The rand sscanpwith s

Figurthe netime in

3.3.4

Lab fof emeCutehumaFigur

re 6: Two eyeg on the foreh

recorded datsaccades anpath of eachspecial softw

re 7: Exampleext. Yellow nn ms the part

Emotion

facilities at Jmotional reae) the teaman interactiore 8).

e cameras mhead, records

ta contain innd saccadic

h participant ware for prog

es of scanpanumbers indicticipant fixated

Physiology

Jacobs Univactions to v

m of Prof. Aon with visua

onitor eye mos the location

nformation rec reaction timvery accura

gramming an

aths. Yellow acate the sequd on the parti

y Lab

versity providvisual stimu

Arvid Kappasals (e.g., he

ovements. A of the head i

egarding themes and al

ately (see Fignd analyzing

arrows indicauence of fixaticular spot.

de ample opli. In two Es focuses oeart rate, ski

head camerain relation to t

e location anlow for the g. 7). The eyg experimen

te moving frotions and blu

pportunity toEU-projects on studying in conducta

a, hidden insithe environme

d duration oreconstructi

ye tracker is tal data.

om one fixatioue numbers in

o study the c(CyberEmovarious mence, facial E

ide the bar ent.

of fixations ion of the equipped

on point to ndicate the

complexity tions and

easures of EMG, see

12

Figur The objecphysirate.

3.3.5

An implanelumethe k

Figurvisitin

3.3.6

The cadd HochmediavideoFaciliinclud

re 8. The lab i

laboratory ictive behavioiological res

VisBench

mmersive ste of size 1,5n) in a passnow-how pro

re 9: Demonsng colleagues

Facilities

collaborationfurther spec

hschule Brema lab, comp

o lab, and usities at thede: multiple

is equipped w

is equippedoral paramesponses like

h

tereoscopic 0m x 2,66msive stereo sovided by P

stration of Vifrom VSI Duk

s at Univers

ns with VisCcialized facmen, sevenputer graphisability lab we Technologlabs in the

with an EMG f

d to study peters like reae EMG resp

workbench m each using

set-up and arof. Lars Lin

isBench at Jake University,

ität Bremen

ComX memilities. Spec

n specializedcs lab, broa

with video mogie-ZentrumDigital Medi

for measuring

parameters action timesponses, skin

with a verg 4 HDTV 3-an optical trsen’s team (

acobs Univery, December 2

n and Hoch

bers at Univcifically, at td labs are ad band anonitoring fac Informatik ia Group; a

g facial muscl

linked to eand facial

n conductan

rtical and a -chip DLP prracking syste(see Fig. 9).

rsity by Prof.2009.

schule Brem

versität and the section available fod communic

cilities and aund Infor

studio for di

le activities.

emotion, forresponses ace, respirat

horizontal rojectors (6.em can be

Lars Linsen

men

HochschuleMedieninfo

or VisComXcation lab, an eye trackemationstechigital photo

r example as well as ion, heart

projection 500 ANSI used with

n’s team to

e Bremen ormatik at X projects: audio lab, er system. hnik (TZI) and video

13

production equipped with lights and chroma keying backgrounds; rooms for usability studies and for post production of digital media (video, audio and animation). In addition facilities at the TZI available for VisComX research include a full body motion suite for motion tracking that can be used for analysis and media production.

4 Collaborations 4.1 Duke University – Visual Studies Initiative (VSI) The Visual Studies Initiative (VSI) at Duke University is the official collaboration partner of VisComX, planning to exchange PhD-students for a semester abroad at the respective partner institution, and complementing VisComX both in terms of faculty (see list below) and visual research focus. It is envisaged that Duke PhD-students participate in the Jacobs VisComX PhD-program in the 3rd semester, and both cohorts (Duke and Jacobs University PhD-students) spend the 4th semester together at Duke University (see Table 1). Duke University – VSI Faculty

Prof. Rachael B. Brady (Computer Science/Electrical Engineering) Prof. David J. Brady, PhD (Electrical & Computer Engineering) Prof. Caroline A. Bruzelius, Ph.D. (Art History) Prof. Sarah Cohen (Journalism) Prof. Sheila Dillon, Ph.D. (Art History) Prof. James T. Hamilton, Ph.D. (Public Policy) Prof. Mark Hansen, Ph.D. (Media Theory) Prof. Kevin S. Labar, Ph.D. (Neuroscience) Prof. Timothy W. Lenoir, Ph.D. (New Technologies/History of Science) Prof. Julian Lombardi, Ph.D. (Computer Science) Prof. Hans J. van Miegroet, Ph.D. (Art History/Economics) Prof. Mark J. Olson, Ph.D. (Communication Science) Prof. Michael L. Platt, Ph.D. (Neuroscience) Prof. Matthew S. Reynolds, PhD (Electrical & Computer Engineering) Prof. William Seaman, Ph.D. (New Media/Visual Studies) Prof. Kristine Stiles, Ph.D. (Art History/Visual Studies) Prof. Victoria E. Szabo, Ph.D. (Digital Media) Prof. Dr. Carlo Tomasi, Ph.D. (Computer Science)

Directed by Prof. Hans van Miegroet, VSI at Duke pursues a similar transdisciplinary research and study program, but with a more applied component which includes a 3-D virtual environment (DiVE = Duke immersive Virtual Environment) and a new type of online platform (Open Cobalt). Prof. van Miegroet is also a member of the Research Center VisComX’s advisory board. For further information on VSI at Duke University see: http://visualstudies.duke.edu/.