Tennis Real Play: an Interactive Tennis Game with Models from Real Videos
description
Transcript of Tennis Real Play: an Interactive Tennis Game with Models from Real Videos
![Page 1: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/1.jpg)
TENNIS REAL PLAY
: AN
INTERACTIVE T
ENNIS
GAME WITH
MODELS
FROM REAL VIDEOS
J UI - H S I N
LA I E
T AL . ,
N T U
A C M MM 2
0 1 1
![Page 4: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/4.jpg)
THE CHALLENGES1.Player Database
- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
![Page 5: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/5.jpg)
THE CHALLENGES1.Player Database
- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
![Page 6: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/6.jpg)
PLAYER DB: FOREGROUND SEGMENTATION
PROBLEM: THE FIGURES ARE NOT IN THE SAME SIZE…
![Page 7: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/7.jpg)
SOLUTION: FIDUCIAL COORDINATEFiducial (adj.) 基準的
(m0~m7: homography parameters, can be determined by trigonometrics)
![Page 8: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/8.jpg)
PLAYER DB: HIT STATISTICSJui-Hsin Lai et al., “Tennis Video 2.0: A New Presentation of Sports Videos with Content Separation and Rendering”, JVCI 2011
1. Forehand volley2. Backhand volley3. Forehand stroke4. Backhand stroke5. Drop shot
![Page 9: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/9.jpg)
PLAYER DB: BEHAVIOR MODEL
• The final piece of the DB that combines other pieces
• Players speed, hit trajectory, reaction time…etc.
• Can create ‘virtual ’matches between players (like fantasy baseball!)
kcir:In other words, the gamer actually plays minor role in this ‘game’.Whether a player can catch up (speed/direction); how does one choose to hit (force/angle), these are pre-determined.
![Page 10: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/10.jpg)
THE CHALLENGES1.Player Database
- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
![Page 11: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/11.jpg)
PLAYER RENDERING: CLIP SELECTIONIf the player decides (how?) to go from point A to point B:
The system automatically choose the “suitable” clips and seam them together.
Suitable: 1. shortens the distance 2. runs along the path 3. min # of clips
![Page 12: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/12.jpg)
FROM “MOTION” TO “HIT”
Shape/texture similarity between the frame of the clip
The Player DB
kcir:So the swing of the gamer is meaningless.What only matters is the timing of swing.
![Page 13: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/13.jpg)
FROM “MOTION” TO “HIT”In order to look good:
They want to use “view morphing” in between the transitionPROBLEM: YOU NEED
‘LABELS’ BEFORE YOU CAN MORPHSolution: [JCV09] and [JCV04] on
Auto Labeling for morphing
![Page 14: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/14.jpg)
THE CHALLENGES1.Player Database
- Auto Video Analysis- Auto Clip Collection
2.Player Rendering- Realtime Clip Editing- Player Strategy Analysis
3.The Game System- Background/Foreground Rendering- Realtime Processing
![Page 15: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/15.jpg)
TO CONSTRUCT 3D FROM 2D
![Page 16: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/16.jpg)
3D GAMING (FUTURE WORK)
![Page 17: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/17.jpg)
EVALUATION• Computation Analysis• Prediction of Game Result
- Virtual Match• Subjective Evaluation
- Compares to Wii and xbox- Interactivity- Immersive Experience
![Page 18: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/18.jpg)
COMPUTATION ANALYSISIntel i7 2.6 GHz CPU with 4GB RAMRendering resolution: 720 x 480
=> Frame rate: 30 fps (standard mpeg1 requirement)
Basis
You can offline-computed these two
![Page 19: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/19.jpg)
PREDICTION OF GAME RESULT
![Page 20: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/20.jpg)
SUBJECTIVE EXPERIMENTS20 undergrads5-point scoring system
Q1. InteractionQ2. Immersive ExperienceQ3. InterestingQ4. Innovative application (huh?)Q5. Willing to play TRP after watching videos?
![Page 21: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/21.jpg)
SUBJECTIVE EXP: COMPARES TO OTHERSParticipants played TRP, Wii tennis, and TP3 xbox
Q6. Entertainment levelsQ7. Realism of visual effectsQ8. Interactiveness (didn't you just ask this in Q1?)Q9. Preference
![Page 22: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/22.jpg)
RESULTS
Using Wii as the comparison basis (score=3)
Q1. InteractionQ2. Immersive ExperienceQ3. InterestingQ4. Innovative application (huh?)Q5. Willing to play TRP after watching
videos?Q6. Entertainment levelsQ7. Realism of visual effectsQ8. InteractivenessQ9. Preference
![Page 23: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/23.jpg)
PUBLIC DEMO
People are asking: “How can I get this feature on my own TV?”
![Page 24: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/24.jpg)
CONCLUSION1. The first work to integrate video-based
rendering and interactive sports game running in realtime
2. Without motion capture system, construct game from video without much human assistance
3. Game results can reflect the match results in real world and predict match results
![Page 25: Tennis Real Play: an Interactive Tennis Game with Models from Real Videos](https://reader035.fdocuments.us/reader035/viewer/2022081604/5681658c550346895dd855b1/html5/thumbnails/25.jpg)
COMMENTS1. The value of this work is its auto process of video->game,
which includes:- video analysis- auto editing of the clips- realtime rendering- computational complexity to overcome with
2. The game itself, although put in the title, is just a bonus3. How can this be better than Wii !?!?!?
- gamer cannot control how they hit. The hitting result is determined by the player DB. (Williams always wins)- gamer don't have to chase the ball. The player will instantly know where to go. (this is cheating!)- You will always win if you know which player to pick….And you call this FUN? Poor consumers