Post on 28-May-2015
description
Multimodal Interactional Analysis
Capturing Different Media
1. Sound2. Video3. Photo4. Observation (Field Notes)
MediaMedia ModesModesSoundSound Speech, music, laughter, background noiseSpeech, music, laughter, background noise
VideoVideo LightingLighting Ambient lighting in Ambient lighting in homes, daylight homes, daylight
Physical presencePhysical presence Posture, facial and Posture, facial and bodily gestures, eye bodily gestures, eye contact, pointingcontact, pointing
Production elementsProduction elements On screen effects, On screen effects, framing of shots, framing of shots, cutting between cutting between scenes, etc.scenes, etc.
Physical objectsPhysical objects Placing objects in field Placing objects in field of view, gesturing of view, gesturing with objects, with objects,
PhotographyPhotography Similar to video but without the Similar to video but without the movement/time aspectsmovement/time aspects
Observation Observation (field notes)(field notes)
Spoken language, physical behaviours, social Spoken language, physical behaviours, social dynamics, printed text/signs, etc. dynamics, printed text/signs, etc.
Multimodal Activity
Observe and record examples of how technologies influence the way that people relate to one another
Use one of the following media: 1.Sound Recording2.Observation (Field Notes)3.Still Photos4.Video Recording
Things to look out for?Reciprocity in communication
coordination of turn taking in conversation‘gift’ exchange
Ritualised behaviour openings and closings
Defining the ‘meaning’ of physical spaceWhat behaviours are
allowed/expected/normsReinforcement of these norms
Divided Consciousness/Absent PresenceDealing with immediate and distant social
contact simultaneously
Feedback SessionWhat modes have been captured?What different stories do they tell?Were there any issues with collecting/using
particular media?
1. Watch the videos/listen to recordings/view images
2. Identify themes◦ Observe and identify significant moments that highlight
multimodal interaction as an aspect of communication – note the time
◦ Refer to existing literature on multimodal aspects of communication to check what you see and hear – for example Goffman (1959)
3. Watch the videos4. Describe the multimodal aspects of the videos5. Roughly transcribe the video data to clarify significant
moments in communication and identify commonalities across videos
6. Watch the videos7. Final transcription in a suitable form to explain themes
and subthemes◦ Include standard annotation conventions
Developing Multimodal Themes
Multimodal transcription of video data (using my own coding scheme)
Multimodal Transcription
www.sigridnorris.com
Multimodal transcription – from conversational analysis:
(n) pause noted in seconds = joining of words : an extension of the preceding syllable; ; rising or falling intonation for subsequent
utterance underlined text indicates spoken with
particular vocal energy - a sudden cut off to an utterance; ? a rising tone . a falling tone , a shifting continuous tone bridging
utterances.
Multimodal Description
ReferencesClark, H. H. (1992). Arenas of language use. Chicago: The
University of Chicago Press.Clark, H. H., & Schaefer, E. F. (1989). Contributing to
discourse. Cognitive Science, 13, 259–294Kendon, A., Harris, R. M., & Ritchie Key, M. (1975).
Organization of behavior in faceto-face interaction. Paris: Mouton Publishers.
Kendon, A. (1990). Conducting interaction: Patterns of behavior in focused encounters.Cambridge: Cambridge University Press.
Goffman, E. (1963). Behavior in public places: Notes on the social organization of gatherings. New York: The Free Press.
Ten Have, P. (1999). Doing conversation analysis. A practical guide. London: Sage Publications.