CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality •...
Transcript of CS294-137: Theory and Applications of Virtual Reality and ... · Definition: Virtual Reality •...
CS294-137: Theory and Applications of Virtual Reality and Immersive Computing
Achin Bhowmik, James O’Brien, Allen Y. Yang
Fall, 2017
Lecturers
AchinBhowmik
CTO&EVP,StarkeyHearingTechnologies
FormerVP,PerceptualComputing,
Intel
Email:[email protected]
JamesO’Brien
Professor,EECS
ChiefScientist,Avametric
Email:[email protected]
AllenY.Yang
ChiefScientist,FungInstitute
FormerCTO,AtheerLabs
Email:[email protected]
Other Guest Lecturers: CAC.berkeley
Other Resources @ Berkeley
Other Resources @ Berkeley
l Founded in Spring 2015 l >200 members in each
semester enrollment
VR@Berkeley
Course ScheduleWeek1(8-23):IntroductionandCapstoneOptionsWeek2(8-30):HumanPerceptionintheContextofVRWeek3(9-6):BasicUnity3D/VRProgrammingWorkshopWeek4(9-13):CourseprojectproposalpresentationWeek5(9-20):ComputerGraphicsrelatedtopicsWeek6(9-27):3DVisionrelatedtopicsWeek7(10-4):OpticsandDisplaytechnologiesWeek8(10-11):LocalizationandMapping****************Week9:(10-18)VRFilmMaking(RichardHernandez)Week10(10-25):Gaming(JackMcCauley)Week11(11-1):Telemedicine(RuzenaBajcsy/GregorijKorillo)Week12(11-8):AR/VRinArts&Design(TedSelker)Week13(11-15):ComputationalImagingforVR(RenNg)Week14(11-22):NoclassWeek15(11-29):FinalprojectpresentationWeek16(12-6):Finalprojectpresentation
Grading PolicyYourfinalgradeswillbedeterminedbyfourfactors:1. Attendancebyyou(10%)
2. Interactionduringtheclassgivenbythelecturers(10%)
3. Course/capstoneprojectpresentation(40%)
4. Aresearchpaperdiscussingyourprojectanditsrelevantliteratureandcommercialapplications(40%)
Recommended Reading Material• Perception:SensationandPerception
byBruceGoldstein
• VirtualReality:VirtualRealityByStevenLaValle(andcheckouthisYouTubelectures)
• ComputerGraphics:FundamentalsofCGbyPeterShirley
• ComputerVision:AnInvitationto3-DVisionbyYiMa,etal.• Display:MobileDisplays
byAchinBhowmik,etal.
• AR/VRMarketResearch:Virtual&AugmentedReality,understandingtheraceforthenextcomputingplatformbyGoldmanSachs
Goal of the Course• UnderstandthefundamentaltheoriesthatenableVR/ARandIC
• UnderstandthemaintechnologydriversofVR/ARandICmarkets
• Becomeanexpertincriticizingthecurrentsoftwareandhardwaresolutions
• Beingabouttheinventnewsolutionsthataddressexistingneeds/painpointsofVR/ARandICapplications
• (Optional)Beingmotivatedtopursueacareerinrelevantresearchorentrepreneurialfields
AR/VR: A Hot Market in 2016
Holodeck: A VR Experience
Leia’s Hologram: An AR Experience
Brief History of VR/AR/IC
PhotographyGraphics Stereoscopy
Interaction3DAudio
Early Forms of Paintings and Arts
CavepaintingsSince35,000–40,000BP
Writingandlanguages,Since3100BC
Early Forms of Photography
CameraObscura,circa400BC Firstphotoonpaper,1800s Kodachrome,1935
3D Illusion with Perspective
AmesRoom,byAdelbertAmes,Jr.,1946
Seeing 3D from Stereo
Wheatstonemirrorstereoscope,1838 Holmesstereoscope,1861
Audio: From Mono to Spatial 3D
Edisoncylinderphonograph,1899
DolbyStereo,1977
Inventionofheadphones,1910s
DolbyAtmosforVR
Interaction Modalities
Keyboard,since1860s Mouse,DouglasEngelbart,1963
iPhone,2007
Hands as 3D Input Device
Sensorama: The First VR Prototype
MortonHeilig,1958
Virtuality: Dawn of VR Gaming
Tilt Brush in Virtual 3D
CES 2017: Samsung 4D VR Experience
IvanSutherland,1968
The First AR Prototype
Anatomy of an AR Device: HoloLens
Definition: Virtual Reality• VR is a computer technology that uses head mounted
displays, sometimes in combination with other sensory devices, to generate realistic images, sounds, and other sensations (touch, smell, motion, etc.) that simulate a user’s physical presence in a virtual environment.
27
Definition: Augmented Reality• AR is a computer technology that augments a
physical, real-world environment directly or through its indirect view computer-generated sensory information, including graphics, video, and sound. AR may alter a user’s view of reality, and may also enhance one’s perception of reality.
28
Enabling Technologies/ Open Research
• Near-EyeDisplaysandOptics
• 3DLocalization
• 3DContentCapturing
• NewHuman-ComputerInterface
Near-Eye Optical Module
HMD Stereo Display Challenge
AccommodationVergence
Localization via Beacons
Localization via Depth Perception
Google Tango Inside-Out Localization
360 VR Capturing: Photo Stitching
GoogleJumpVR
From 360 VR to Real-Time 3D VR
Lytro Immerge
New 3D Human-Computer Interface
New 3D Human-Computer Interface
Connecting AR/VR and Robotics
Course/Capstone Projects Examples
Berkeley OpenARK
• Lumus • PMD• Webcam• Motion Sensor• Microsoft Surface Pro• <$5K per unit
Beta released early 2017
2 Allow for breathing room
ISAACS: Immersive Semi-Autonomous Aerial Command System
2 Allow for breathing room
Drone Fleet Control (Tomlin)
2 Allow for breathing room
Drone Safety Control (Tomlin)
VR in Autonomous Driving (Borrelli)