Dongyan Xu, Daniel Aliaga, Scott Yost, Jonathan Deutsch

MRT Overview

Motivations

•To create a common plane for immersive demonstration and perception

•Allow users to focus on the tabletop instead of shifting their vision and attention back and forth between the computer screen and the physical objects

Advantages

•Cheaper than VR

•Works better than head mounting

•One focus (unlike video-conferencing)

•Can use computer-unfriendly objects

Conference XP

•Participants

•Venues

•RTP - Real-Time Protocol

DirectShow

•Insert source-filter to grab the image from the camera and do camera calibration

•Insert transform-filter for projector calibration

Raw Image

Camera Correction

• Camera is not perfectly aligned with deskop, has spherical distortion, etc.

• Create Tsai model for calibrated camera

• Sample Tsai model to warp the camera image into canonical space

Corrected Image

UI and Graphics

Raw Projector Output

Projector Calibration

• Projector is misaligned, projection area doesn’t perfectly overlap with camera field of view

• Project sequential pattern onto table, scan output via camera to detect projector “usable area”

• Can then warp images to projector such that they appear squared and aligned

Prewarped Output

Image Pipeline

CameraCorrection

UI graphics

Projector Correction

Projector

One more problem!

• Problem: Camera <-> projector feedback loop will saturate image

• Solution: Synchronize camera and projector

Synchronization!

• Immediately before the camera grabs an image, stop projecting onto the table

• Result: Camera sees only real objects, nothing from projector

• Accomplished through hardware synchronization pins on camera, software

Applications

• We provide image processing, networking code – you provide the app.

• Application #0: Digital Paper

Applications

• Focus: collaborative teaching

–young children: origami, other art

–school-age: chemistry, physics experiments

Applications

• Your ideas?

Interested?

Contact: Prof. Dongyan Xu (dxu@cs.purdue.edu)

Prof. Daniel Aliaga (aliaga@cs.purdue.edu)