Kinect for Windows Quickstart Series
-
Upload
roberto-reto -
Category
Technology
-
view
1.808 -
download
2
description
Transcript of Kinect for Windows Quickstart Series
![Page 1: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/1.jpg)
KINECT FOR WINDOWS
QUICKSTART SERIES
![Page 2: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/2.jpg)
KINECT FOR WINDOWS
INSTALLING &USING KINECT
![Page 3: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/3.jpg)
KINECT FOR WINDOWS
INTRODUCTION
• Designed to get you started• Assume you have some programming
experience• Managed API - Concepts work in both
VB/C#
![Page 4: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/4.jpg)
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
![Page 5: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/5.jpg)
KINECT FOR WINDOWS
KINECT SENSORSIR Emitter Color Sensor
IR Depth Sensor
Tilt Motor
Microphone Array
![Page 6: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/6.jpg)
KINECT FOR WINDOWS
GET STARTED
• http://kinectforwindows.org • Order Kinect Hardware• Download Kinect SDK
![Page 7: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/7.jpg)
KINECT FOR WINDOWS
INSTALLING AND USING KINECT
DEMO
![Page 8: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/8.jpg)
KINECT FOR WINDOWS
SETTING UP YOUR DEVELOPMENTENVIRONMENT
![Page 9: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/9.jpg)
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development
Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
![Page 10: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/10.jpg)
KINECT FOR WINDOWS
KINECT API BASICS
• Manage Kinect state • Connected• Enable Color, Depth, Skeleton• Start Kinect
• Get Data• Events - AllFramesReady• Polling – OpenNextFrame
![Page 11: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/11.jpg)
KINECT FOR WINDOWS
RESOURCES
• Install Kinect Explorer• KinectWpfViewers
• Coding4Fun Toolkit• Skeletal scaling
![Page 12: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/12.jpg)
KINECT FOR WINDOWS
SETTING UP YOUR DEV ENVIRONMENT
DEMO
![Page 13: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/13.jpg)
KINECT FOR WINDOWS
CAMERA FUNDAMENTALS
![Page 14: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/14.jpg)
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
![Page 15: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/15.jpg)
KINECT FOR WINDOWS
CAMERA DATA
• Events return ImageFrame• PixelDataLength• FrameNumber• Timestamp• Dimensions: Height, Width
• Use AllFramesReady event to synchronize
![Page 16: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/16.jpg)
KINECT FOR WINDOWS
RESOLUTIONS
• Color• 12 FPS: 1280X960 RGB• 15 FPS: Raw YUV 640x480• 30 FPS: 640x480
• Depth • 30 FPS: 80x60, 320x240, 640x480
![Page 17: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/17.jpg)
KINECT FOR WINDOWS
CAMERA FUNDAMENTALS
DEMO
![Page 18: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/18.jpg)
KINECT FOR WINDOWS
WORKING WITHDEPTH DATA
![Page 19: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/19.jpg)
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
![Page 20: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/20.jpg)
KINECT FOR WINDOWS
DEPTH DATA• Returns the distance and player for every
pixel• Ex: 320x240 = 76,800 pixels
• Distance• Distance in mm from Kinect ex: 2,000mm (6.56
feet)
• Player• 1-6 players
![Page 21: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/21.jpg)
KINECT FOR WINDOWS
MODE
Mode Depth & Player Center Hip Joint Other 19 Joints
Default Yes Yes Yes
Near Yes Yes No, for v1.0
![Page 22: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/22.jpg)
KINECT FOR WINDOWS
DISTANCES1.3’ 2.6’ 9.8’ 13.1’ 26.2’
.4 .8 3 4 8
DefaultMode
Near Mode
Feet
Meters
![Page 23: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/23.jpg)
KINECT FOR WINDOWS
FORMULAS
• Distance Formulaint depth = depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth;
• Player Formulaint player = depthPoint & DepthImageFrame.PlayerIndexBitmask;
![Page 24: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/24.jpg)
KINECT FOR WINDOWS
WORKING WITH DEPTH DATA
DEMO
![Page 25: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/25.jpg)
KINECT FOR WINDOWS
SKELETAL TRACKINGFUNDAMENTALS
![Page 26: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/26.jpg)
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
![Page 27: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/27.jpg)
KINECT FOR WINDOWS
SKELETON DATA
![Page 28: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/28.jpg)
KINECT FOR WINDOWS
SKELETAL JOINTS• Each player with set of <x, y, z> joints in meters• Each joint has associated state
• Tracked, Not tracked, or Inferred
• Inferred - Occluded, clipped, or low confidence joints• Use TransformSmoothParameters to smooth joint data to
reduce jitter
![Page 29: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/29.jpg)
KINECT FOR WINDOWS
SKELETAL TRACKING• Maximum two players tracked at once
• Six player proposals per Kinect
• Each skeleton has a unique identifier - TrackingID • SkeletonStream
• Set AppChoosesSkeletons to false• Use ChooseSkeletons method to choose which skeletons to
track
![Page 30: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/30.jpg)
KINECT FOR WINDOWS
SKELETAL TRACKING
DEMO
![Page 31: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/31.jpg)
KINECT FOR WINDOWS
AUDIOFUNDAMENTALS
![Page 32: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/32.jpg)
KINECT FOR WINDOWS
QUICKSTART SERIES
• Installing and Using the Kinect Sensor• Setting up your Development Environment• Camera Fundamentals• Working with Depth Data• Skeletal Tracking Fundamentals• Audio Fundamentals
![Page 33: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/33.jpg)
KINECT FOR WINDOWS
SOUND POSITION• Sound Source Angle – the angle and confidence level of
where audio is coming from• Beam Angle – The angle used to record audio that you can
set as a “directional microphone”
-50 0 +50
![Page 34: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/34.jpg)
KINECT FOR WINDOWS
RECORDING
• KinectAudioSource audio buffer• Use Echo Cancellation to not record PC
sound
![Page 35: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/35.jpg)
KINECT FOR WINDOWS
SPEECH RECOGNITION
• Grammar – What we are listening for• Code – GrammarBuilder, Choices
• Note: Ensure AutomaticGainControl = false
![Page 36: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/36.jpg)
KINECT FOR WINDOWS
GRAMMAR<!-- Confirmation_Yes._value: string ["Yes"] --><rule id="Confirmation_Yes" scope="public"> <example> yes </example> <example> yes please </example> <one-of> <item> yes </item> <item> yeah </item> <item> yep </item> <item> ok </item> </one-of> <item repeat="0-1"> please </item> <tag> out._value = "Yes";</tag>
var grammar = new Choices();grammar.Add(“yes please");grammar.Add(“yes");grammar.Add(“yeah");grammar.Add(“ok");
![Page 37: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/37.jpg)
KINECT FOR WINDOWS
MULTI-MODAL FEEDBACK• Engagement model–Mic indicator for speech-enabled menus– Keyword to engage
• Feedback and confirmation
![Page 38: Kinect for Windows Quickstart Series](https://reader036.fdocuments.us/reader036/viewer/2022062319/5556008cd8b42a3f168b4662/html5/thumbnails/38.jpg)
KINECT FOR WINDOWS
AUDIO
DEMO