Wiimote/Kinect Lab Midterm UpdateSenior Design December 2011, Group 16

Adviser: Dr. Tom Daniels

Brenton HankinsRick HantonHarsh GoelJeff Kramer

Project Goal and Concept• Provide software to wrap the Microsoft Kinect SDK functions

to provide an interface useful to freshmen in CprE 185• Fix current issues with WiiWrap2 – a program currently used

to interpret Wiimote input data in CprE 185

System DesignSample Student Code:

int main(int argc, char **argv) {

int kr = KINECTE_OK;

KINECT *handle; // Kinect handlekr = kinect_global_init();handle = kinect_init(); //initialize handleif(!handle){

kinect_global_cleanup();return 0;

}

//modify rgb values for funkinect_add_video_modifier(handle, &modify_rgb_values);

//modify depth videokinect_add_depth_modifier(handle, &clarify_far_objects);

//detect circular heads using agorithmkinect add_depth_observer(handle, &detect_heads);

//overlay video feed by user defined imagekinect_add_video_modifier(handle, &overlay_heads_with_image);..

Student defined callbacks

System Design (contd.)

Provides functions to analyze and modify frame data before displaying on the output GUI

Receives frames and provides it to the program efficiently after processing

Part of Microsoft Research Kinect SDK. Handles drivers and hardware issues.

Students write callbacks to process the incoming frames and modifying them if desired

Project Status - WiiWrap• Analyzed & solved current software problems• Implemented buffer flushing to prevent data loss in piping action• Switched IR reporting to raw values from IR sensor• Set up memory in WiiWrap to record IR position values between

visible instances (in case of temporary loss of signal)• Added code for better usability in WiiWrap GUI• Fixing user program vulnerability (user program failure would

shut down entire GUI)• Created Installer Package

Project Status - Kinect• Implementing code with SDK• Started implementation this semester using the Kinect SDK

rather than OpenKinect library, resolved system issues. • Observed Depth and Video data streams and made modifications

to the frames in an effort to understand student problem solving• Implemented Frame processing engine in C++ to receive frames

from the Kinect and send them to the C Interface• Partly done with the basic C interface to receive and modify

frames• Modifying open source GUI for displaying output and running

student code

Testing and Evaluation• We are using similar test plans for both the Wii and Kinect.• Initially we will use black box testing methods on individual

modules of the program.• Testing boundary inputs and random inputs

• The second phase of testing will be integration testing.• Testing the entire system as a whole single unit

• GUI testing can be done alongside the integration phase of the testing plan.

• Once the functionality is tested, we will need to do performance testing especially on the GUI.• Use Eclipse TPTP to test performance.• Test GUI input buffering, load tests, etc.

Pending Issues• Need to get GUI implementation completed• Need to work on implementing and documenting student

Kinect API to provide fuller functionality from the SDK• Propose ideas for future CprE 185 labs using Kinect• Do testing with current 185 students• Document WiiWrap2 functionality in simple terms so it can be

provided to middle school and high school teachers

• Waiting for CSG to provide computer with root or proper hardware/software to test code against 185 lab environment

Project Success Criteria

Full Success•Installer created and tested for improved version of WiiWrap2•Provide raw numerical Kinect data to the user•Provide Kinect SDK interface to the user for the user to modify graphical output in real-time•Provide a GUI for graphical output with options for various display outputs (RGB, Depth, Skeleton)•Create demonstration labs for 185Partial Success•Installer for WiiWrap2 created and tested•Kinect can provide raw output to the user•Allow user to modify graphical output in real-time (basic GUI)