Hacking the Kinect with GAFFTA Day 1

83
OPEN FRAMEWORKS + KINECT PART 1 @GAFFTA

description

First day of slides for @GAFFTA workshop http://www.gaffta.org/2012/07/24/hacking-the-kinect-with-openframeworks/ Part 1 of the live stream : http://www.youtube.com/watch?v=WXfy8Cuje-0&feature=plcp Part 2 of the live stream : http://www.youtube.com/watch?v=I80FsOlMPj8&feature=plcp

Transcript of Hacking the Kinect with GAFFTA Day 1

Page 1: Hacking the Kinect with GAFFTA Day 1

OPEN FRAMEWORKS + KINECT PART 1

@GAFFTA

Page 2: Hacking the Kinect with GAFFTA Day 1

Open Frameworks is an open source,creative coding platform.

Page 3: Hacking the Kinect with GAFFTA Day 1

Makes it easy to start c++ More power with less headache

Page 4: Hacking the Kinect with GAFFTA Day 1

OverscanSO SO Limited

http://www.creativeapplications.net/openframeworks/overscan-openframeworks/

Page 5: Hacking the Kinect with GAFFTA Day 1

Scramble SuitKyle McDonald + Arturo Castro + Others

http://www.creativeapplications.net/openframeworks/scramble-suit-face-tracking-openframeworks/

Page 6: Hacking the Kinect with GAFFTA Day 1

NodebeatJustin Windle + Seth Sandler

http://www.creativeapplications.net/iphone/nodebeat-iphone-ipad-of/

Page 7: Hacking the Kinect with GAFFTA Day 1

EntrailsBy Lukasz Karluk

http://www.creativeapplications.net/openframeworks/entrails-openframeworks/

Page 8: Hacking the Kinect with GAFFTA Day 1

University of Dayton Interactive WallBy Flight Phase

http://www.creativeapplications.net/openframeworks/interactive-wall-at-ud-openframeworks-kinect/

Page 9: Hacking the Kinect with GAFFTA Day 1

Starry Night Van Gough Interactiveby Petros Vrellis

http://www.creativeapplications.net/openframeworks/vincent-van-goghs-starry-night-interactive-by-petros-vrellis-openframeworks/

Page 10: Hacking the Kinect with GAFFTA Day 1

Bloom Skinby Elttob Tep issey Miyake

http://www.creativeapplications.net/openframeworks/bloom-skin-the-wave-installation-for-elttob-tep-issey-miyake/

Page 11: Hacking the Kinect with GAFFTA Day 1

Screen Lab #2Screen Lab + MediaCityUK

http://www.creativeapplications.net/environment/screenlab-2/

Page 12: Hacking the Kinect with GAFFTA Day 1

CLOUDS Interactive DocumentaryJames George + Jonathan Minard

http://www.creativeapplications.net/openframeworks/clouds-interactive-documentary-exploring-the-creativity-through-the-lens-of-code/

Page 13: Hacking the Kinect with GAFFTA Day 1

Mclaren P12 TrailerMarshmellow Laser Feast

http://www.creativeapplications.net/processing/mclaren-p12-teaser-mclaren-vs-aerodynamics-by-mlf/

Page 14: Hacking the Kinect with GAFFTA Day 1

Light FormMathieu Rivier ( ECAL )

http://www.creativeapplications.net/openframeworks/light-form-interactive-landscape-by-mathieu-rivier-ecal/

Page 15: Hacking the Kinect with GAFFTA Day 1

Saatchi and Saatchi New Directors ShowcaseMarshmellow Laser Feast

http://www.creativeapplications.net/openframeworks/light-form-interactive-landscape-by-mathieu-rivier-ecal/

Page 16: Hacking the Kinect with GAFFTA Day 1

Paik Times FiveBy Flightphase

http://www.creativeapplications.net/openframeworks/light-form-interactive-landscape-by-mathieu-rivier-ecal/

Page 17: Hacking the Kinect with GAFFTA Day 1

How do I use Open Frameworks ?

A few IDE options:

XCodemac

Code::blocksmac, windows , linux

Visual Studiowindows

Page 18: Hacking the Kinect with GAFFTA Day 1
Page 19: Hacking the Kinect with GAFFTA Day 1

Image + Video Utilities

Page 20: Hacking the Kinect with GAFFTA Day 1

Vector Graphics API

Page 21: Hacking the Kinect with GAFFTA Day 1

Sound in / out , panning, volume, play speed

Page 22: Hacking the Kinect with GAFFTA Day 1

Geometry Shader

Page 23: Hacking the Kinect with GAFFTA Day 1

GLSL Shader Example

Page 24: Hacking the Kinect with GAFFTA Day 1

Open GL 3D Utilities: Vector Classes , Geometry and Texture Utilities, Camera Control

Page 25: Hacking the Kinect with GAFFTA Day 1

String utilities, file system access

Page 26: Hacking the Kinect with GAFFTA Day 1

Core addons include...

Page 27: Hacking the Kinect with GAFFTA Day 1

XML Manipulation

Page 28: Hacking the Kinect with GAFFTA Day 1

Servers and Connections - TCP / UDP / OSC

Page 29: Hacking the Kinect with GAFFTA Day 1

OpenCV Wrapper

Page 30: Hacking the Kinect with GAFFTA Day 1

3D Model Animation loader

Page 31: Hacking the Kinect with GAFFTA Day 1

But the real power of Open Frameworks is in the community

Page 32: Hacking the Kinect with GAFFTA Day 1

Recently, www.ofxAddons.com was created which makes finding these great addons easy !

Page 33: Hacking the Kinect with GAFFTA Day 1

ofxBullet - Bullet Physics Wrappergithub.com/NickHardemann/ofxBullet

Page 34: Hacking the Kinect with GAFFTA Day 1

Multiple Augmented Reality LibrariesSURF , ARToolkit , Qualcomm

Page 35: Hacking the Kinect with GAFFTA Day 1

and all of it is open source

Page 36: Hacking the Kinect with GAFFTA Day 1
Page 37: Hacking the Kinect with GAFFTA Day 1
Page 38: Hacking the Kinect with GAFFTA Day 1
Page 39: Hacking the Kinect with GAFFTA Day 1

Download OF for your platform from :http://www.openframeworks.cc/download/

Copy the OF download to somewhere more permanent.

Everything you do for openFrameworks needs to be within this directory.

You may need to do some initial setup, check http://www.openframeworks.cc/setup/ for details

Your first App !

Literally do this.

Page 40: Hacking the Kinect with GAFFTA Day 1
Page 41: Hacking the Kinect with GAFFTA Day 1
Page 42: Hacking the Kinect with GAFFTA Day 1

Recently the community created a project genearator which makes it easier to get started.

Page 43: Hacking the Kinect with GAFFTA Day 1

You can also include any addons you want from the start to avoid messing with any compiler settings.

Page 44: Hacking the Kinect with GAFFTA Day 1

Main.cpp

Sets up the window size and openGL contextRuns application class until quit

Page 45: Hacking the Kinect with GAFFTA Day 1

The Application Cake.

Ingredients - .h filesInstructions - .cpp files

Page 46: Hacking the Kinect with GAFFTA Day 1

testApp.h

declaration of class, functions, and variables

Page 47: Hacking the Kinect with GAFFTA Day 1

testApp.h

implementation of class, functions, and variables

Page 48: Hacking the Kinect with GAFFTA Day 1

setup ( )

load assetsinitialize valuesinitialize addons or components

Page 49: Hacking the Kinect with GAFFTA Day 1

update( )

apply force to particlescalculationsincrement video frames

Page 50: Hacking the Kinect with GAFFTA Day 1

draw ( )

draw shapes/images/videosuse GLSL ShadersApply Blend ModesSave Pixels from screen

Page 51: Hacking the Kinect with GAFFTA Day 1

Look at the folder 00_basics for class outline

Page 52: Hacking the Kinect with GAFFTA Day 1

There is no var keyword.

#include is your new import

float = AS3 number

lots of similar of typesstring, int, uint

Page 53: Hacking the Kinect with GAFFTA Day 1

Functions look a little different, with the return type at the start of the declaration.

Multiple functions can share the same name, as long as they have different input parameters or return types.

Page 54: Hacking the Kinect with GAFFTA Day 1

Step 1Preprocessor runs through and combines all your code in one giant file. Target the preprocessor with the “#” symbol

Step 2Compiler parses code and make sure there are no errors. Your code is broken down into a lower level language: Assembly.

Step 3The Assembly code is turned into readable code by the computer inside object files.

Step 4The object files are linked together into an executable file.

C++ Compiler - Step 1

show person staring at a screen stereotypicallymonkey at a computer works too

First you write some code. You could do this in your IDE ( xcode )

Page 55: Hacking the Kinect with GAFFTA Day 1

Github

Github is awesome.

Go there. like now and sign up.

github.com

Page 56: Hacking the Kinect with GAFFTA Day 1

Github

Github is a social coding platform that allows you to host a git repository for free as long as it’s public and open source.

Github has great resources for getting started with githttp://help.github.com/

Page 57: Hacking the Kinect with GAFFTA Day 1

Github

Github is a great way to move your own files between machines.

It has built in issue tracker and and wiki capabilities.

Great way to collaborate and share code.

Page 58: Hacking the Kinect with GAFFTA Day 1

Git Quick Tip

.gitignore is a file specific to a repository that allows you to specify what is not tracked by git.

with c++ normally this is excessive IDE generated files, and build files.

Binary files don’t track super well on git because there are no changes to track except files size. but sometimes you need to include them anyway.

Page 59: Hacking the Kinect with GAFFTA Day 1

Translation with Transformation Matrices

Instead of moving an object on the screen,the entire screen is moved.

Think of it as a global registration point.

Page 60: Hacking the Kinect with GAFFTA Day 1

Simple Translate

to build off of the current space or to end a local spaceuse:

ofPushMatrix( )

and

ofPopMatrix( )

Page 61: Hacking the Kinect with GAFFTA Day 1

Simple Translate

Page 62: Hacking the Kinect with GAFFTA Day 1

Simple Rotation

Page 63: Hacking the Kinect with GAFFTA Day 1

Simple Scaling

Page 64: Hacking the Kinect with GAFFTA Day 1

Order Matters

Page 65: Hacking the Kinect with GAFFTA Day 1
Page 66: Hacking the Kinect with GAFFTA Day 1

01 Circles

Step 1Draw a circle with a random color where the mouse cursor is.

Page 67: Hacking the Kinect with GAFFTA Day 1

01 Circles

Step 2Create struct ColorPoint, structs are like classes except they cannot have methods.

A struct is useful for storing grouped data.

Store each color point in a vector<> which operates similar to a dynamically sized array

Page 68: Hacking the Kinect with GAFFTA Day 1

01 Circles

Step 3Draw screen into a Frame Buffer Object ( FBOs ) FBOs are called with begin() and end() Anything between those functions will be stored and be available for use later.

Mirror the FBO vertical / horizontal for symmetry

Page 69: Hacking the Kinect with GAFFTA Day 1

01 Circles

Step 4 - BONUSPull colors from a color palette.

Page 70: Hacking the Kinect with GAFFTA Day 1

02 Animator

Step 1 Create a looping sequence of FBOsDraw a circle into wherever the mouse is

Page 71: Hacking the Kinect with GAFFTA Day 1

02 Animator

Step 2 Scale the circle radius based on mouse speedAnd draw a random color

Page 72: Hacking the Kinect with GAFFTA Day 1

02 Animator

Step 3 Including an addonWe will use ofxUI to add some sliders to make our animator a little more fun.

Adding addon files to a projectAdding a RGB slider color ranges

Page 73: Hacking the Kinect with GAFFTA Day 1
Page 74: Hacking the Kinect with GAFFTA Day 1

03 Particles

Step 1Load an ImageCreate still particles from the raw pixels of the image

Page 75: Hacking the Kinect with GAFFTA Day 1

03 Particles

Step 2Add the repulse / attract modes to the system

Add particle alpha trails

Page 76: Hacking the Kinect with GAFFTA Day 1

03 Particles

Step 3Update particles colors from a movie

Loading a movie

Page 77: Hacking the Kinect with GAFFTA Day 1

Joining the Community !

Introduce yourself at :

http://forum.openframeworks.cc/index.php/board,11.0.html

The OF community is very welcoming.

Page 78: Hacking the Kinect with GAFFTA Day 1

How to be a good community member ?

Read your IDE’s in depth forum walkthrough it will save you a lot of hassle.

Search the forums for answers before making a post.

Ask questions and post your code.

DO NOT beg for code.

Page 79: Hacking the Kinect with GAFFTA Day 1

Additional Resources

There are now official OF tutorials !http://openframeworks.cc/tutorials

Roxlu has a wonderful collection of slides to explain some of the awesome features of OF 007http://roxlu.com/blog/entry/145/openframeworks-007-presentations

The new version of Programming Interactivity by Joshua Noble is the goto book for OFhttp://programminginteractivity.com/wordpress/

Page 80: Hacking the Kinect with GAFFTA Day 1

Additional Resources

Unofficial c++ reading listhttp://forum.openframeworks.cc/index.php/topic,9034.msg42670.html

Processing ‘s official site still has one of the best explanations of core conceptshttp://processing.org/learning/

The OF forums are a treasure trove of discovery and works in progress. Subscribing to the RSS feed will keep you very up to date.http://forum.openframeworks.cc/

Page 81: Hacking the Kinect with GAFFTA Day 1

Additional Resources

Jeffery Crouse has some really good tutorials and is a professor http://www.jeffcrouse.info/teaching/

Creative Applications does an amazing job showcasing and collecting installations and other creative apps.

http://www.creativeapplications.net/

Page 82: Hacking the Kinect with GAFFTA Day 1

Ben McChesneyLead Experience DeveloperHelios Interactive

@bendesigning on twitter

benmcchesney.com/benmcchesney.com/blog

github.com/benMcChesney

Page 83: Hacking the Kinect with GAFFTA Day 1

THANK YOU