WebGL Meetup GDC 2012 - Khronos Group
Transcript of WebGL Meetup GDC 2012 - Khronos Group
© Copyright Khronos Group, 2012 - Page 1
WebGL Meetup GDC 2012
© Copyright Khronos Group, 2012 - Page 2
Khronos API Ecosystem Trends
Neil Trevett Vice President Mobile Content, NVIDIA
President, The Khronos Group
© Copyright Khronos Group, 2012 - Page 3
Mobile Gaming Innovation Vectors
Console-Class Performance,
Quality, Controllers and TV
connectivity
HTML5 and WebGL Web Apps that can be discovered on the Net and run on any platform
Vision Cameras as sensors, Gesture Processing, Augmented Reality
Sensor Fusion Devices become ‘magically’ context aware – location, usage, position
New platform capabilities being driven by SILICON and APIs
OpenVL
© Copyright Khronos Group, 2012 - Page 4
Khronos - Connecting Software to Silicon • Creating open, royalty-free API standards
- Focus on graphics, dynamic media, compute and sensor hardware
• Low-level - just above raw silicon
- “Foundation” functionality needed on every platform
• Safe forum for industry cooperation
- ‘By the industry for the industry’
- Open to any company to join
- IP framework to protect
members and industry
APIs enable software developers to turn silicon
functionality into rich end user experiences
© Copyright Khronos Group, 2012 - Page 5
Board of Promoters
Over 100 members – any company worldwide is welcome to join
Apple
© Copyright Khronos Group, 2012 - Page 6
HTML5 – Cross OS App Platform • Increasing diversity of devices
creates a demand for a true cross OS
programming platform
• BUT need more than “more HTML”
Rich Experiential Processing Multi-core CPUs
Rich 2D and 3D GPU GPU Computing
Multiple HD cameras Image and vision processing
Video encode/decode Audio encode/decode
Inertial and positional sensors
How can the Browser rapidly assimilate such diverse functionality?
Traditional Web-content
© Copyright Khronos Group, 2012 - Page 7
Leveraging Native API Investment into HTML5 • HTML5 evolving into cross-platform programming platform
- Gradually exposing complete system capabilities
• Opportunity to synergize Web and native APIs development
- Leverage native API investments, reduce developer learning cycles
• Khronos and W3C creating close liaison
Native APIs shipping or working group underway
JavaScript API shipping or working group underway
WebVL? Vision
Processing
WebAudio Advanced JavaScript
Audio
WebMAX? Camera
control and video
processing
Possible future JavaScript APIs
Device and Sensor APIs
Device Orientation
Working Groups
Native
JavaScript
OpenVL
© Copyright Khronos Group, 2012 - Page 8
WebGL Deployment • WebGL 1.0 Released at GDC March 2011
- Mozilla, Apple, Google and Opera working closely with GPU vendors
• Typed array 1.0 spec ratified by Khronos in May
- Supporting bulk data transfer between threads (workers)
- Many use cases - background mesh loading, generation, deformation, physics ...
• 1.0.1 release of WebGL spec and conformance suite imminent
- 100% robust stance on security
- Fixing bugs in 1.0.0 conformance suite
- Implementations will report getContext("webgl") (not experimental)
http://caniuse.com/#search=webgl WebGL is not enabled by default in Safari
© Copyright Khronos Group, 2012 - Page 9
WebCL – Parallel Computing for the Web • Khronos has launched new WebCL initiative
- First announced in March 2011
- API definition underway
• JavaScript binding to OpenCL
- Security is top priority
• Many use cases
- Physics engines to complement WebGL
- Image and video editing in browser
• Stay close to the OpenCL standard
- Maximum flexibility
- Foundation for higher-level middleware
© Copyright Khronos Group, 2012 - Page 10
OpenVL • Vision Hardware Acceleration Layer
- Enable hardware vendors to implement
accelerated imaging and vision algorithms
• OpenVL can be used by high-level
libraries or applications directly
- Primary focus on enabling mobile and
embedded systems
• OpenCV is widely used open source
library for vision projects
- Future version will leverage OpenVL
OpenCV open source library
Application
OpenVL
Open source sample implementation?
Hardware vendor implementations
Other higher-level CV libraries
© Copyright Khronos Group, 2012 - Page 11
StreamInput Connects Sensors to Apps
Advanced Sensors Everywhere Standard cameras, depth cameras
motion and position, touch, microphones wireless controllers
Apps Need Sophisticated Access to Sensor Data
Without coding to specific systems or sensor hardware
Apps request semantic sensor information StreamInput defines list of possible semantic requests
“Am I in an elevator?” “Give me gestures and face position”
Sensor graph created to provide sensor stream High-quality sensor data stream using optimized, sensor smart middleware
Apps gain ‘magical’ situational awareness – and portability
© Copyright Khronos Group, 2012 - Page 12
Current StreamInput Participants • Aiming for specification release in 2012
© Copyright Khronos Group, 2012 - Page 13
Possible Implementation of Vision Stack
Semantics and fusion of camera and positional sensors
Accelerated computer vision algorithms
Parallel computation
OpenVL
High-level computer vision library
Use OpenCL to implement CV HAL with
parallel execution
Accelerate OpenCV
library with CV HAL
Functions
Implement StreamInput vision sensor modules with
OpenCV
Data and event interop with CL /
GL / ES for display and compute processing
Camera input from OpenMAX AL or other camera subsystems
© Copyright Khronos Group, 2012 - Page 14
Khronos Ecosystem of Standards
Embedded and Mobile 3D
Cross platform desktop 3D
Plugin-free 3D Web Content
Heterogeneous Parallel
Computing Camera, Imaging and Streaming Media
Web Compute
Hardware accelerated vision functions
OpenVL
Unified Sensor and Input Processing
Graphics Sensors
Compute Imaging and vision
!
!
Na
tive
A
PIs
Web APIs
© Copyright Khronos Group, 2012 - Page 15
Current Khronos Liaisons • W3C
- HTML5/JavaScript bindings: WebGL, WebCL, sensors, video and camera
- Browser Acceleration: CSS shaders, composition, audio, SVG, Canvas
• Web3D
- Liaison agreement regarding X3D and COLLADA
- Declarative 3D community group at W3C - 3D scene graph in the DOM - WebGL
• ISO/IEC JTC1
- SC4/TC184 3D Visualization Specifications - Copyright License COLLADA executed
- SC24/WG9 – Augmented Reality working group – using Khronos APIs
• OMA and OGC
- Augmented reality cooperation - complementary activities
© Copyright Khronos Group, 2012 - Page 16
WebGL and Declarative 3D • Declarative 3D
- W3C Community Group – open to all
• Very synergistic
- WebGL can accelerate Declarative 3D
• Declarative 3D seems to be a major opportunity for X3D community
- By re-using existing HTML5 machinery – rather than duplicate – have a chance to
be very widely adopted
Canvas
2D 3D
Scenegraph
Immediate
© Copyright Khronos Group, 2012 - Page 17
Need for 3D Transmission Standard • Approaching chaos in data encodings used in WebGL browsers and apps
- We need to enable native decompression in browsers rather than JavaScript
• Possible key requirements
- Full-scene - Geometry, textures, materials, animations, physics etc.
- Compression of textures and geometry
- Streaming support with LOD flexibility
• Some formats in use – but no widespread consensus
- COLLADA, KML, MPEG-4, VRML, JSON, X3D binary, PowerVR POD, GZIP etc. etc.
• Is the industry ready to work on this? Is the need real?
Audio Video Images 3D
All above defined by MPEG – should 3D be an MPEG standard too? Must be royalty free! Leverage MPEG-4 Part 16 ‘AFX’, Part 25 – a lot of investment – but why no momentum?
MP3 H.264 PNG/JPEG ?
© Copyright Khronos Group, 2012 - Page 18
COLLADA WebGL Synergy • COLLADA synergy with WebGL authoring
- COLLADA import into common WebGL frameworks such as three.js?
- JSON as transmission format - robust COLLADA to JSON conversion?
• Lots of discussion needed around deployment formats
- Compression, streaming – use the Transmission Standard