MotionComposer - the search for coherence in movement and sound
description
Transcript of MotionComposer - the search for coherence in movement and sound
![Page 1: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/1.jpg)
Robert Wechsler
Workshop/Roundtable
McGill University, Feb.29-Mar.2, 2012
MotionComposer- the search for coherence in movement and sound
www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 2: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/2.jpg)
Mercredi
1 – 6 workshop
Jeudi
1 – 6 workshop
Vendredi
2 – 3 presentation
3 – 5 roundtable
Überblick
www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 3: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/3.jpg)
introductions
demonstration
3 talks - 40 min ea.
discussion after each 20 – 40 min.
project design and execution
project presentation, de-briefing
Überblick - Workshop (both days)
www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 4: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/4.jpg)
who am i?
interactive performance
motion tracking
mapping and synaesthesia
the motioncomposer: an intuitive expression device for persons with disabilities (Bauhaus Universität)
Überblick – 3 talks over 2 days.
www.motioncomposer.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 5: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/5.jpg)
dancer, choreographer, developer
- New York, Nürnberg, Stuttgart, Weimar
- Palindrome Dance Company
- interactive since 1975
- developed EyeCon software with Frieder Weiss 1995-2005
- teach motion tracking
- MotionComposer project director, Bauhaus University
me
www.palindrome.de/ppt Part 1: me
![Page 6: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/6.jpg)
influences
New York City in the 1980's
www.palindrome.de/ppt Part 1: me
![Page 7: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/7.jpg)
John Cage Merce Cunningham
![Page 8: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/8.jpg)
![Page 9: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/9.jpg)
Interactive what
activity response
activity /response activity /response
www.palindrome.de/ppt Part 2: Interactive Performance
![Page 10: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/10.jpg)
interactive performance is nothing new
Cave drawings, Spain ca. 4000 BC
![Page 11: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/11.jpg)
Dionysis Thr. Athens - 300 BC
mechanical technology was used to empower performers
![Page 12: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/12.jpg)
Louis XIV - sun king and megalomanic
![Page 13: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/13.jpg)
Louis loved mechanical devices, toys, remote
acting devices (automation)
![Page 14: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/14.jpg)
He also loved cannons, the ultimate amplifier of gesture.
![Page 15: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/15.jpg)
Interactive devices were used by New York performance artists in the 60's and 70's
Robert Rauschenberg - "Pelican" 1963
Laurie Anderson - "Walking Solo for Tape Bow Violin" 1979
![Page 16: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/16.jpg)
Interactive Performance today
Interaction Design
Gesture Sonification
Motion Capture
Virtual Environments
Immersive Environments
Haptic responders
Diversified Performance (using internet)
and and and
www.palindrome.de/ppt Part 2: Interactive Performance
![Page 17: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/17.jpg)
Part 3: Motion Tracking
![Page 18: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/18.jpg)
motion tracking vs. motion capture
motion capture– highly accurate, expensive (ca. 1m €)– generally not realtime– used for data collection (research), making realistic human or animal animations
(films, games, etc.)
motion tracking– less data, less equipment, less cost (ca. 1k-2k €)– concerned with motion qualities like dynamic, direction of motion– real time
www.palindrome.de/ppt Part 3: Motion Tracking
![Page 19: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/19.jpg)
motion capture
– tracks location of fixed positions on body with reflective markers– 12-24 cameras, each lens is ringed with infrared lights
www.palindrome.de/ppt Part 3: Motion Tracking
![Page 20: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/20.jpg)
• Vicon is a leading company in motion capture
motion capture
![Page 21: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/21.jpg)
motion capture
![Page 22: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/22.jpg)
motion capture
![Page 23: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/23.jpg)
motion capture
![Page 24: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/24.jpg)
– performances with interactive animated characters: "No DNA" (http://www.nodna.com)
– dance pieces on stage
– Motione (2003-5) Arizona State U. (http://ame.asu.edu/motione)
(Trisha Brown, Kaiser, et.al.)
"22" (Bill T. Jones, Kaiser , et.al.)– Luc Vanier et al. at the interactive performance facility U of Wisconsin at
Milwaukee (http://www.isl.uiuc.edu/Publications/final%20dance1.pdf)
real-time mo-capture
www.palindrome.de/ppt Part 3: Motion Tracking
![Page 25: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/25.jpg)
motion tracking sensors
• body-oriented
• environment-oriented
www.palindrome.de/ppt Part 3: Motion Tracking
![Page 26: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/26.jpg)
motion tracking sensors
• body-oriented
– body-worn electrodes
• EEG
• ECG
• EMG
• Touching
– joint bend-sensors
– accelerometers, other body-worn devices
• environment-oriented
– video camera-based (eg. eyecon)
– infrared camera-based
– floor sensors, wall sensors, etc
www.palindrome.de/ppt Part 3: Motion Tracking
![Page 27: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/27.jpg)
Part 4: Mapping and Synaesthesia
![Page 28: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/28.jpg)
Synaesthesia
Kandinsky
Dostoyovsky
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 29: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/29.jpg)
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
Dance + Music = ?
![Page 30: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/30.jpg)
Where , how do they connect?
How can we build up this connection?
What is Mapping?
movement sound
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 31: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/31.jpg)
movement sound
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
What possibilities does the technology allow?
![Page 32: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/32.jpg)
analogue Digital
sensor device
d-to-aconversion
a-to-dconversion
processing output device
e.g. movemente.g. sound
analogue
![Page 33: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/33.jpg)
What is Mapping?
movement soundmakes
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 34: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/34.jpg)
What is Mapping?
dance musicmakes
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 35: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/35.jpg)
What is Mapping?
dance music
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 36: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/36.jpg)
What is Mapping?
dance music
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 37: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/37.jpg)
What is Mapping?
dance music
correlation?
![Page 38: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/38.jpg)
What is Mapping?
turns off
correlation
turns on
iterate
toggle
lower
raise
complexcontinuous variables
dance music
![Page 39: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/39.jpg)
What is Mapping?
turns off
iterate
toggle
lower
raise
correlation
turns on
dance music
complexcontinuous variables
![Page 40: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/40.jpg)
What is Mapping?
correlationdance music
![Page 41: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/41.jpg)
What is Mapping?
Number of mappings (M) = number of Input parameters (I) x number of Relationships (R) x number of Output parameters (O)
M = I x R x O
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 42: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/42.jpg)
There are thousands of possibilities.
Most are intelligible.
few are palpable.
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
![Page 43: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/43.jpg)
Coherent Mapping -- movement parameters
![Page 44: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/44.jpg)
Coherent Mapping -- movement parameters
spatial / temporal
![Page 45: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/45.jpg)
Coherent Mapping -- movement parameters / spatial
![Page 46: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/46.jpg)
Coherent Mapping -- movement parameters / spatial
![Page 47: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/47.jpg)
Coherent Mapping -- movement parameters / spatial
![Page 48: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/48.jpg)
Coherent Mapping -- music parameters / time and timing
1 discrete events
![Page 49: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/49.jpg)
1 discrete events
Accents Stops
Starts Other instaneous changes
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
Coherent Mapping -- music parameters / time and timing
![Page 50: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/50.jpg)
2 Tags
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
Coherent Mapping -- music parameters / time and timing
![Page 51: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/51.jpg)
3 Modulation
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
Coherent Mapping -- music parameters / time and timing
![Page 52: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/52.jpg)
4 Repetition, sequences and timing patterns
A, B, A, B, ... (bi-modal)
A, B, C, D, E, F, ... (scales and melodies)
A, A, A, A, A, A,A, ... (rhythms and randomness)
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
Coherent Mapping -- music parameters / time and timing
![Page 53: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/53.jpg)
5 Multi-dimensional „qualitative“ assocations
many evocative qualities arise through combinations of aural and kinesthetic experience.
www.palindrome.de/ppt Part 4: Mapping and Synaesthesia
Coherent Mapping – time and space
![Page 54: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/54.jpg)
dynamic activity, stillness, flow (direction) sensing
background substraction, „3d cams“ position, shape
intelligemt systems body part and gesture recognition
Motion Tracking technology - approaches
![Page 55: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/55.jpg)
Motion Tracking - Software
MT:EyeCon, Eyesweb, MotionComposer, MAX/Jitter (et al.
Cyclops), Isadora, VNS, Icarus .. Vicon (etc.)
Sound:EyeCon, Kontakt, Ableton (Max live), ...
DSP:Csound, Max/msp, Pure Data, Supercollider
![Page 56: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/56.jpg)
Hardware
Computers
Cameras
Framegrabbers
Motion Tracking - Hardware
![Page 57: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/57.jpg)
thank you.
questions?
Join MotionComposer...
www.motioncomposer.com
www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 58: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/58.jpg)
thank you.
questions? ideas?
www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 59: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/59.jpg)
thank you.
questions? ideas?
www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 60: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/60.jpg)
thank you.
questions? ideas?
www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 61: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/61.jpg)
thank you.
questions? ideas?
www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt
![Page 62: MotionComposer - the search for coherence in movement and sound](https://reader030.fdocuments.us/reader030/viewer/2022013011/56813f2b550346895da9caf6/html5/thumbnails/62.jpg)
thank you.
questions? ideas?
www.palindrome.de this power point may be downloaded from: www.palindrome.de/ppt