160525 SMPTE Hollywood VR - philip...
66
1 Virtual Reality Overview & Update Philip Lelyveld USC Entertainment Technology Center [email protected]
Transcript of 160525 SMPTE Hollywood VR - philip...
1
Virtual RealityOverview & Update
Philip LelyveldUSC Entertainment Technology Center
Founded 1993 by George Lucas tobringtogethertechnologyandentertainmentvisionariestocollaborateonthefutureofentertainmenttechnology
Adamantlyneutral• Discussions• Collaborations
• Research• Coverage
CURRENT MEMBER COMPANIES
3
CURRENT MEMBER COMPANIES
4
In Augmented Reality, the world that you would normally see if you walked down the aisle without the screen is being augmented – enhanced, supplemented, added to - with additional data and experiences.
In Virtual Reality you are virtually – nearly, almost – but not really there in the grocery store aisle.
RealEnvironment
AugmentedReality
VirtualReality
6
TransitionalMixed Reality
Blended Reality
TheAR– Reality– VRContinuum
DifferentwaystohaveaVRexperience
MagicWindow
Cellphone-based viewer
HeadMountedDisplay(HMD)
GroupVR
8
VirtualRealityonCellphoneorTablet“MagicWindow”
Near$0costofentry
Swipe
Tilt and wave
TetheredversusUntethered
• BandwidthIssue• Compression• Latency
Sitting,Standing,WalkingHeadMountedDisplayswithPeripherals
12
GroupVRExperiencesLocationBasedEntertainment
TheVoidopensTimesSquareVRCenterFirstattraction:“Ghostbusters:Dimensions”OpensJuly1,2016,developedincooperationwiththemovie
TheLanguageofVR:Part1
Existingtoolsandskillsarebeingreworkedtocreatesomethingverydifferent
“Virtual Reality is not a media experience. When it is done well, it is an actual experience.” Professor Jeremy BailensonStanford Virtual Human Interaction Lab
16
Senseof‘Presence’
1.Moresensesaccurately engaged=increasedpresence
2.FidelityContract
“Ifeelhorribleaboutmakingthis,”…“Youactuallyfeelguilty.MymouthdroppedthefirsttimeIshotsomeonein
my GTA:V VRsetup.”
GrandTheftAutovideo-gamedesignerJosephDelgado,uponexperiencingtheVRversionofGTA:V
5/13/[email protected] 18
19
HeightenedSenseofEmpathy
3D facial expressions evoke stronger emotionsthan their 2D counterparts
due to the illusion of non-mediation
[email protected]: 11/24/15 research from Aalto Univ. & Univ. of Helsinki
You are in a snow stormYou must get warm in 5 minutes or you will die
Audio is a key ‘presence’ componentPlayers ‘feel’ colder as they playYour body moves slower as time goes on
Languagestory telling + engagement design elements
From “Storm” by Andrick [email protected] 23
NextVR3603DRig6RedPro6Kcameras60fpsSpatialaudiocapture
JauntJ1-24REarly201632K(16Kpereye)120fps 25
VRCameras
Captured 3D
Nokia OZO
Captures 360 stereoscopic 3D through:- 1950 capture with 50% overlap for 3D- 8 synchronized global shutter sensors- 8 integrated mikes for spatial audioJaunt committed to support Ozo for camera & production workflow
Surround 360 by Facebook
Not a product; an open source design for others to build on2048x2048 60 fps genlocked cameras – 14 around, 1 up, 2 down
Orah 4i live-streaming Prosumer VR camera from VideoStitch
[email protected] 284fisheyelenses,4mikes,4Koutput,30secdelay,~$3,600
VRCameras(consumermkt.)
Bublcam from Bubl ($799)
Kodak SP360 (2X$290)
Ricoh Theta ($350)
Giroptic ($499)[email protected] 29
Samsung 360Two 15MP cameras360-degree video in 3,840-by-1,920 resolutionGalaxy S7 and S7 Edge, Gear VR
Nikon KeyMission 360 ($?)Waterproof to 100’
32
Wild– TheExperience,VRshortfromFox(ReeseWitherspoon, LauraDern)
SoundAudiobreakdown• Livespatialaudiocapture• Miked audiowithposition-tracking• Audiowithnoposition(musicscore)
Stitching
StandardVRcamerarigscanhavebuilt-in stitchingsoftwareFromtalkbyAnrick Bregman
Editing tools
Skyboxiscreatinga‘directtoRift’ playerfortheAdobesuitethathangsthemenuswithintheHMDspace
37
UserInterface
Tomaintainpresence,UIfornavigationdesign andinformationcommunicationmustbeconsistentwiththerestoftheexperience
GameEngines
DigitalDistributionofVR
• YouTube360• AppleTV360• Hulu360• Amazon360• Netflix360
• Littlstar• Vrideo• WearVR• ….
[email protected] 39WearVR’s new home page 4/16/16
40
Technical ConsiderationsBandwidth and QoSCompressionLatencyProcessing powerLocal buffer/storage …
Option: Distribute a 4K frame, then client warps it into a sphere.
Capture at high-resolution, distribute at what client-processing, bandwidth, and display tech allow.
Google Fiber expressed interest in streaming VR
HeadMountedDisplayTechnicalTargets
4K-8Kresolutionpereye
>90framespersecondrendering (vscapture)
<20msec latency
2200 fieldofview(FOV)
Github VR standards discussions
Google put out RFComments for embedded spatial audio standards
FB camera released as open source to drive innovation and open standards
Google DayDream VR for Android N, including reference specs for phones and Gear-like HMDs
Standards?Standardization efforts.
44
Camera-capturedVRLockedtocamerapositionTheDirector’sJourney
ComputerRenderedVRFreetowander(Storyworld)
Subtlecoercion
Decision
MixedRealitySphericalcaptureliveandinsertintoCGI
“Volumetriccapturewillchangeeverything.”ChrisEdwards,[email protected] 45
Hybrid- OutsidetoInside‘VolumetricVR’
Replay-technologies.com8i.comMS Hololens Holoport [email protected] 46
Capture from many anglesCreate a model that can be explored
LytroCinemalight-fieldcameraCinemaframetoday,volumetricVRinthefuture
Everypixelhascolor,direction,x/y/zpositioninfoEveryobjectcanbeseparatedfromitssurroundings– greenscreen755RAWmegapixel40Kresolution,300FPS, 400Gbpsofdata
Deluxe Launches VR Unit to Facilitate Industrial-Scale Virtual Reality Productions
Deluxeassembledateam ofspecialistsfromboththeworldofvisualeffectsandvideogameproduction.
Variety, 4/11/[email protected] 48
TheCameraisYourAudience’sHead!!!
We don’t see shaky-cam when we run
The world doesn’t tilt when we tilt our heads
We see the world differently sitting and standing
VRputsyouinthesceneThereisnothirdwalltobreak
Maintainingpresence dependsonmaintainingexpectationsintheexperience
HIDE(orbeabletoremove)EVERYTHINGTHATDOESN’TBELONGTHERE
CrewLightsEquipmentShadowsofequipment…NewLEDlightformfactorscanbehiddeninthesceneOrShootinmultipledirections(ex.180then180)andblendinpostIndoorscenes;sampletheceiling,putlightsthere,paintthemoutlater
Real-timeremotemonitoringoftheshootisbeingtested(ex.Teradek Sphere,Ozo system)ViewdailiesinHMDforsenseoftheexperienceANDlookatindividualsensorfeedsforproblems
Solvingproblems
Greenscreenhelmetontoheadless torso.
Headbouncesdifferently frombody. FromtalkbyAnrick Bregman
Thefinalimagemustbeerror-freeImperfectionsstandoutglaringly,andbreakpresence
Stitchingisa3-passprocess;- algorithmicreal-timefirstpass- quickpassforeditingwork- compositingheavyworkflowwithclean-upstitching
54
Wild– TheExperience,VRshortfromFox(ReeseWitherspoon, LauraDern)
SoundAudioEngineering• Difficulttoproperlypositionfromaflatscreen• Toolsandroomsforspatialaudioarebeingdeveloped• Gameenginebasedaudioknowledge(Unity,Unreal)
August5th &6th 2016LosAngelesConvention
Center
SoCal VROctober 30, 2016Long Beach CC
For-Profitclasses
MonthlySaturday,May28thVRLASchoolfocusesonthetoolsandprocessesforeditingstories
Websites
Road to VR – www.roadtovr.com - latest newshttps://www.facebook.com/groups/Virtual.Realities/ - latest newshttps://www.facebook.com/RoadtoVR/ - latest newsWearVR, Littlstar, Vrideo – latest contentReddit VR – community discussion groups
UploadVR - http://uploadvr.comVR Reporter - https://virtualrealityreporter.comVR Scout - http://vrscout.comWomen in VR – www.wivr.netWomen in VR Los Angeles - http://www.meetup.com/Women-in-VR/All Things VR – allthingsvr.coKzero - http://www.kzero.co.uk/blog/
Podcasts
Voices of VR - dailyReal Virtual Show - weeklyhttp://a16z.com/?s=virtual+reality
http://www.apoip.org/home/events-calendar/http://allvirtualreality.com/events/list-virtual-reality-events-vr-expo-exhibitions-
conferences-2016.htmlhttps://www.jvrb.org/events
Lists of VR and AR conferences
Modern era consumer VR is <3 years old
New art form and communication tool
WeneedtobringcinematicvisualandaudiocraftspeopleintothisfieldParaphrasedfromconversationwithJacquelineBosnjak,CEO,QDepartment
Think about…
How to direct attention in an open space- Theme park and theme park ride design- Theatre in the round- Architecture and design
Existing skills and tools that port over to VR
Experimenting with a cheap VR camera
Learning Unreal, Unity, and other game engine UI tools
ImagefromNotesonBlindnessVRexperience
66
Thank YouPhilip Lelyveld
USC Entertainment Technology Center
www.etcentric.orgwww.PhilipLelyveld.com
