Subjectivity and Whole Body Interaction

Subjectivity and Whole Body Interaction

Peter Wright

Art and Design Research Centre

C3RI

Sheffield Hallam University

Acknowledgements

EPSRC Leonardo Network Research at the intersection of art and HCI

EPSRC LTR ProjectTheory and method in experience-centred design

Design 21 My Exhibition ProjectAffective communication in product and exhibition design

Sheffield Health Trust2nd Life for the Third Age

Hester Reeve, Ben Heller, Jon Wheat, John McCarthy

Overview

The interesting case of Installation ArtIs it happening?

Theories of Embodiment and experiencePaul Dourish: McCarthy and Wright

Second Life for the Third Ageagency and coupling in virtual space

My ExhibitionPersonalisation and ambient interaction

Towards a critical framework for whole body interaction

Disclaimer!!

The interesting case of Claire Bishop on the History of Installation Art

The subject in the art work

“Installation differs from traditional media (sculpture, painting, photography and media) in that it addresses the viewer directly as a literal presence in the space.

Rather than imagining the viewer as pair of disembodied eyes that survey the work from a distance, installation art presupposes an embodied viewer whose senses of touch, smell and sound are as heightened as their sense of vision.

But more than this…

The viewer is not just ‘in the art work’ but they complete it

“The spectator is in some way regarded as integral to the completion of the work.” Reiss

They are then at once experiencing the art work and part of it.

The spectator is so integral that “…without having the experience of being in the piece, analysis … is difficult.” Reiss

The experiencing subject

If installation art is at pains to offer 1st-hand experience what kind of experience does it offer?

Four modalities of experience that installation art structures for the viewer, each with a different model of the subjective self:

–The Freudian self–The phenomenology of perception–Lacan and Barthes beyond the pleasure principle–The activated/politicised subject

In general HCI has been slower than Installation Art to escape the gravitational pull of Cartesianism

As a consequence it has under theorised experience and subjectivity

The disembodied mind

The dominance of the eye

and hand

Representation rather than participation

But things are beginning to change

Embodiment and Embodied Interaction

Paul Dourish

Dourish: Embodied interaction

Embodiment is both a physical and a social phenomenon

Meaning is constructed in practical acts of engagement with social and physical worlds

Objects and people in the world can be understood as affording different forms of engagement

Meaning is maintained by a coupling between the agent and the world

“Embodied phenomena are those which by their very definition occur in real time and real space”

“Embodiment is the property of our engagement with the world that allows us to make in meaningful.”

“Embodied interaction is the creation, manipulation and sharing of meaning through engaged interaction with artifacts”

Dourish: Space and place

Space is often characterised in Cartesian terms as a container for our action (an immutable backdrop)

But in fact space is constructed in response to human concerns

Place is characterised in terms of human habitation, adaptation, culture but also personal histories and emotional meanings

See also Fitzpatrick on Locales

Dourish Key Points

the key things to take from Dourish then are:

– embodiment as both a physical and social– participation rather than representation– meaning as constructed through physical/social interaction

in place– coupling between the embodied person and the environment

The theory orients towards the situated nature of our interactions with technology but under-theorises the ‘felt life’ of these interactions

But more than this…

With his concern for the social, Dourish under-theorises, the subject

While he acknowledges both the physical and the social aspects of embodiment, he does not explore how these inter-relate

We have tried to foreground felt life and the subjectivity of our experience with technology

The spectator is so integral that “…without having the experience of being in the piece, analysis … is difficult.” Reiss

What kinds of experience of experience does technology structure?

A holistic approach to “Felt Life”

where embodied, narrative, emotional and sensual aspects of felt life are treated as equal, interdependent constituents of experience

Continuous engagement and sense making

Wherein the self is the centre of experience, is already engaged in experience and bring to a situation a history of personal, social and cultural meanings, and anticipated futures that structure experience through acts of sense making

A dialogical approach to self

Wherein self, other, object and setting are actively constructed as multiple centres of value (multiple voices) and are always an unfinalised project

Just three themes

Compositional Emotional

SensualSpatio-temporal

The threads in experience Sensualour sensory engagement with a concrete situation- palpable visceral

Emotionalthe evaluative relations that unite needs and desires to the particular“that was satisfying, I’m proud of that”

Compositionalthe narrative structure of a situation, before during and beyond“what’s this about, what has happened, what should I do?”

Spatial & temporalThe quality of space and time, Place, pace, control,

A holistic approach

anticipatingexpectation as a continuous process in

experienceconnecting

immediate pre-linguistic sense of a situation

interpretingunderstanding the events in terms and

how they fit with expectationsreflecting

making judgements about the experience

appropriatingmaking something our own

recountingto self and others shapes our

understanding and fuels anticipation

Sense making

Making sense of the threads

Dourish, McCarthy and Wright: Summary

Our embodied interactions are shot through with sensations, emotions, values, ideals, and feelings that are subjectively felt and can be intersubjectively shared

We experience our interactions from a unique position…

…yet our sense of self is constituted in response to the voices (values) of others

Experiences do not come ready made they are actively finished by the subject

Second life for the Third Age

Or

“Nelly and the avatars”

Nelly and the Avatars

The peopleFall Pre-hab class, 6 people aged 86-91, No previous experience of computers, WIIs, Playstations or X-

Boxes

The technologyWeight sensors on weighing scales controlling a Line Dancing

avatar created though 3-D motion capture

The aimto explore the feasibility of using whole body interaction with such

an extreme user group

To assess whether the technology would be experienced positively by both client and therapist

Video Excerpt

Nelly and the Avatars

Initial impressionsFall Pre-hab class, 6 people aged 86-91, no previous experience of computers, WIIs, Playstations or X Boxes

The technologyWeight sensors on weighing scales controlling a Line Dancing avatar

created though 3-D motion capture

The aim– to explore the feasibility of using whole body interaction with such an

extreme user group– To assess whether the technology would be experienced positively by

both client and therapist

The visionto have a group of people some of who cannot walk, some of

whom cannot talk, some of whom cannot move their upper torso dancing together in virtual space.

Nelly and the Avatars: Initial Impressions

The emotional response

– What a remarkable bunch of elderly people!

– They loved it!

– They want to do it again

– They talked about dancing

Coupling issues

– The mapping between user and avatar movements didn’t appear to be self-evident

– Remembering the required movements was an issue

– The language of movement was confusing

– Multiple couplings between feet, weight and avatar movement were confusing

My Exhibition

Background Project aims

How can the visceral qualities of interactive images, sounds, lighting and other sensory factors, can be used to help people personalise experiences of exhibition content?

We are designing interactive "zones" in the exhibition which will help people attend to narratives and forms of engagement that will suit them.

Zones will recognise individual visitors, offer choices, record their decisions and actions to build up a record of the visit that will inform the next waypoint about that visitor's interests and guide them onward in their visit. The "ambient" communication, and unencumbered interaction

Background

The Froissart Chronicles

The exhibition is centred on a collection of illuminated manuscripts chronicling The Hundred Years War

Each written by the same author - John Froissart, but for different Patrons- French and English.

Background

Our Corridor

Our installation will be in the approach corridor and will comprise three zones

Each Zone will present a story –Sir John Chandos–Sieging the Castle –The writing of the Chronicles

When our part of the exhibition is open it will be the visitors first experience of the content

Background

Our Corridor

A physical mock-up of part of the corridor has been built in our interaction lab

Background Some themes

The stories offer a wealth of personal and historical perspectives which we have provisionally organised around themes:

Passion, History, Politics, Forensics, The corpse, The chronicler, The curator…

These themes offer a way of organising and potentially personalising content

The also offer the possibility of experiencing multiple points view and multiple voices that characterise the chronicles

Interaction Design

We are in the middle of this, we haveA digital infrastructure for controlling lights, video and soundA database to record personal details, develop personalisation

trajectories and to track movement through the space and interaction in it.

We have a interoperable toolkit for rapidly prototyping interactions including RFID, and gesture-based interaction

The system knows who a person is where they are where they have been and what they have seen.

The aim is personalisation of based on who a person is where they have been and what they have already seen

Towards a Critical Framework

So what would I like to see in a critical framework?

A framework that:Centres on the experience of interaction not the technology for

interaction

Takes as its starting point the richness of the embodied subject

Focuses on the design space but connects to use

Encompasses work, leisure, educational and cultural applications

Can be made meaningful to user researchers, artists, performers, designers and engineers

A possible starting point: Hornecker & Buur 2006

Many one-dimensional viewpoints already exist:

Data-centred viewpoint

Focus on body as a source of data and coupling between body and digital representations

Expressive-Movement Centred viewpoint

Focus on “sensory richness” of expressive movement

Space-Centred view

Focus on the installation and the interaction space (whole body multiple people

A possible starting point: Hornecker & Buur 2006

The framework themes:Tangible Manipulation (sensual interaction, objects, tactile qualities)

Can users grab, feel and manipulate the important elements

How easy is it to grasp action-effect mappings?

Expressive Representation (what is represented/manipulated,its legibility)Are representations meaningful and relevant?Are the representations self-evident in terms of use?

Spatial Interaction (relations of people and objects in space)Do people and objects meet in meaningful ways?Is movement meaningful to system and others?

Embodied Facilitation (relations in social space)Does the physical arrangement encourage/force

collaboration?Does ensemble of representations build on users’

experiences?

Looking around

Paul DourishWhere the action is (MIT Press 2001)

McCarthy and Wright Technology as Experience (MIT Press 2004)

Hornecker and BuurGetting a grip on tangible interaction (CHI 2006 Proceedings)