Senior Project – Computer Science - 2009

Wii Punch: Tracking and Gesture Recognition for 3 Game Environments

Owen HeneghanAdvisor – Prof. Aaron Cass

Straight Tracking•Connect the Wiimote to the computer via Bluetooth•WiiR3d project takes Brian Peeks’s managed library for Wiimote and a dll to calculate how to track•Connect to XNA game which uses JibLibX physics•OnTrackData method from WiiR3d passes updated coordinate data via UDP•‘Fist’ moves accordingly

AbstractThe purpose of this study is to better understand playability and preference for controlling virtual environments. This study consists of users playing two similar games, with the only difference between them lying in how the subjects control the characters. Looking at the differences through a user study and in game scoring between these two scenarios, we want to find which control structure, straight tracking or gesture recognition is better and preferred for 3D game environments

Gestures•Uses same technology as Straight Tracking•When main game receives a call to the update of TrackData, it passes it to a GestureRecognition class, which keeps track of the last 10 positions•GestureRecognition is able to calculate relative speeds and accelerations. •Responds to main program to perform a punch if the acceleration and speed over time are over a certain threshold as well as having the current position of the arm be beyond some threshold in the Z-axis. •It then threads the movement of the ‘fist’ to predetermined points, with a certain rest between each point.

User StudyUsers either performed the game with straight tracking then gesture recognition or vice versa. At the end of both they completed a user evaluation survey.

ImplementationFor this project there are two main scenes. In the first we create a wall of red bricks, with one being colored black. We rate the user on the speed and accuracy with which they can knock over the black block. This scene tests the users ability to accurately hit the block needed. We grade the user on their speed to hit down the required block.In the second scene we have a ball falling from above the users field of view. This scene tests the users ability to react to the falling ball. We grade the user in their ability to knock the ball the farthest distance from the starting point.

Results•More users preferred straight tracking over gestures•Performance and surveys both coincided•User feedback indicated that predetermined movements needed most work•Under straight tracking users sometimes would ‘push’ rather than punch objects

• Faster times for knocking down the wall scenes

0

5

10

15

20

25

30

0 5 10 15 20 25

Participant

Ave

rag

e B

rick

Sco

re

Straight Tracking

Gesture Recognition

0

5

10

15

20

25

0 5 10 15 20 25

Participant

Ave

rag

e B

all D

rop

Sco

re

Straight Tracking

Gesture Recognition