Celluloid An interactive media sequencing language.
-
Upload
mabel-hudson -
Category
Documents
-
view
218 -
download
0
Transcript of Celluloid An interactive media sequencing language.
![Page 1: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/1.jpg)
Celluloid
An interactive media sequencing language
![Page 2: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/2.jpg)
Today
•Overview of Celluloid•Sample Code•Syntax and Semantics•Translator Architecture•Framework Architecture•Development Environment•Testing•Conclusion
![Page 3: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/3.jpg)
Overview• Interactive Media sequencing language
•Designed to allow users to easily construct their own media systems and allow for real time interaction with the programmer to dynamically change programs on the fly allowing programming to be elevated to a performance art
•Programs are translated into Java SE which can then be compiled into Java bytecode and run on any JVM
![Page 4: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/4.jpg)
Motivation
•Celluloid was motivated by the growing role interactive media plays in our daily lives
•We’ve focused on trying to build a language that targets a wide audience and remains simple and intuitive
•With Celluloid, you can build the technical and artistic developments that will power the future of the media age
![Page 5: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/5.jpg)
Important Properties
•Reactive programming allows modified primitives to affect later primitives defined with them
•Celluloid is designed to be very human readable
•Provides facilities for abstracting away the hardware details of underlying input and output devices to increase portability
![Page 6: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/6.jpg)
Visualizing a Celluloid System
•Multiple devices are connected and abstracted through the device type.
•Enables software mixing that was previously not possible.
![Page 7: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/7.jpg)
Type System• Primitive Types: number, boolean, string, time.
• Timelines: Controls the playback of inputs in the order specified in the source program
• Devices: Encapsulate IO abstractions. Devices are responsible for providing the interface between external media and internal components
• Constraints: Enforce a set of allowed operationson a device as well as share this definition across devices.
• Events: Allows for simple message passing between devices and timelines.
![Page 8: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/8.jpg)
Language Properties•Compiled statically typed language.
•Compiled into Java code built on a custom framework.
•Allows for user defined functions and user defined types.
•Custom device, constraint, and event types are allowed.
![Page 9: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/9.jpg)
Sample Programtimeline audioinput audio1 = new AudioFile(“audiofile1.mp3”)input audio2 = new AudioFile(“audiofile2.mp3”)input audio3 = new AudioFile(“audiofile3.mp3”)output main = new AudioOutput(“Audio”)
in audio do play audio1 @start play audio2 @5s play audio3 @10send
in main do play audio @startend
![Page 10: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/10.jpg)
Translator Architecture
celluloid.gcelluloidLexer.g
celluloidParser.g
celluloid.java.stg
source_code.cld
target_code.java
org.celluloidLang.*
![Page 11: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/11.jpg)
Framework Architecture
• Java Media Framework
• Implementation of Timelines
• Implementation of Devices
• Implementation of Constraints
•Announcements and Events
•Reactive Programming
![Page 12: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/12.jpg)
Java Media Framework (JMF)
•Designed to manipulate “audio, video and other time-based media”
•Cross platform support
•Supports many popular formats▫Plugins available to support additional formats▫FOBS for JMF
![Page 13: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/13.jpg)
Implementation of Timelines
• Controls the playback of inputs in the order specified in the source program
• Timer object keeps track of time and executes the appropriate action at the appropriate time
• System utilitzes 100 millisecond time-slices
• HashMap containing when-based events and the corresponding action to execute when the event’s condition(s) is satisfied
![Page 14: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/14.jpg)
Implementation of Devices
•Two built-in devices for reading audio and video files are provided out-of-box: JMFVideo and JMFAudio
•Framework provides the ability to implement real-time and static media
•Examples of real-time media: midi devices, streaming media, instruments
![Page 15: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/15.jpg)
Implementation of Constraints• Constraint interfaces▫ Enforce allowed operations on device classes in
the framework
• Constraint functions▫ Keeps track of what should be executed and when
it should be executed▫ Anonymous inner class keeps track of what is
executed
![Page 16: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/16.jpg)
Sample Constraint Translationin timelineOne do play inputOne @0send
... would translate to ...
ConstraintFunction cf = new ConstraintFunction (inputOne, 0) { public void execute() { input.play(); //where (input == inputOne) }}
timelineOne.addConstraintFunction(cf);
![Page 17: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/17.jpg)
Reactive Programming
•Reactive programming involves the propagation of change among assignment expressions.
•Example:▫A=4;▫B=9;▫C = A + B▫Print C //displays 13▫A=10▫Print C //displays 19
![Page 18: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/18.jpg)
Announcements and Events
•Announcements and events allow for simple message passing between devices and timelines.
• Implemented as a global event broker (i.e. event listener) available to relevant types in the framework
•Example: when Audio5 audioGain > 7 do play Video4 end
![Page 19: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/19.jpg)
Development Environment
• IDE Devolopment▫IntelliJ▫Eclipse▫ANTLRWorks
•Unit Testing• JUnit
•Repository▫Git▫GitHub
![Page 20: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/20.jpg)
Testing
•Goal: Find faults and determine whether those faults are translation faults or target framework faults.
•Black Box•White Box•Unit Tests▫ Full unit test suite for both the parser and the
framework• Integration Tests▫Testing hand of off parser to the framework
![Page 21: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/21.jpg)
Conclusion
•What we learned:▫Learned tools for developing language▫Implement language
•Worked well:▫Repository, issue tracking, Google docs
•What to do different:• Start earlier
•Future development▫REPL- Read eval print loop
![Page 22: Celluloid An interactive media sequencing language.](https://reader031.fdocuments.us/reader031/viewer/2022020117/56649f4a5503460f94c6bba1/html5/thumbnails/22.jpg)
Questions?