Understanding Hardware Acceleration on Mobile Browsers
-
Upload
ariya-hidayat -
Category
Technology
-
view
8.236 -
download
3
Transcript of Understanding Hardware Acceleration on Mobile Browsers
![Page 1: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/1.jpg)
Understanding Hardware Acceleration on Mobile Browsers Ariya Hidayat
![Page 2: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/2.jpg)
![Page 3: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/3.jpg)
whoami
![Page 4: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/4.jpg)
Challenges
![Page 5: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/5.jpg)
Game vs Web
![Page 6: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/6.jpg)
Game
Animation
Physics
Textured objects
Text
Transformation
![Page 7: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/7.jpg)
Large Area, but Still Bounded
![Page 8: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/8.jpg)
Web Page
Images
Text
![Page 9: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/9.jpg)
From Wave to Pixels
Layout engine
Network stack
GraphicsJavaScript engine
User interface
![Page 10: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/10.jpg)
WebKit Components
WebKit Library
JavaScriptCore
WebCore
HTMLrendering
SVG
DOM CSS
![Page 11: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/11.jpg)
Platform Abstraction
Network Unicode Clipboard
Graphics Theme Events
Thread Geolocation Timer
![Page 12: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/12.jpg)
Graphics Abstraction
Mac Chromium Qt
CoreGraphicsSkia
QPainter
graphics stack
GraphicsContext
![Page 13: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/13.jpg)
Mobile Device
Touch interface
GPU
CPU
Radio
Power
![Page 14: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/14.jpg)
Graphics Processor
Divide and conquer
Very specific purpose
![Page 15: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/15.jpg)
Relationship
CPU GPU
![Page 16: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/16.jpg)
SoC = System-on-a-Chip
![Page 17: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/17.jpg)
Fundamental Physical Limitations
1.Available memory2.Bus bandwith3.Speed difference
![Page 18: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/18.jpg)
Optimized for Games
Transformation
Textured triangles
Fixed geometry
Parallelism
![Page 19: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/19.jpg)
Challenges
Predictable contents (assets) ✔Mostly text ✔Mostly images ✔
immediateInitial response
![Page 20: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/20.jpg)
http://www.trollquotes.org/619-super-spider-bat-troll-quote/
![Page 21: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/21.jpg)
Primitive Drawing
![Page 22: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/22.jpg)
Path is Everything
Path
Triangle Rectangle
PolygonEllipse
![Page 23: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/23.jpg)
Stroke = Outline
Solid Dashed Dotted Textured
![Page 24: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/24.jpg)
Brush = Fill
SolidNone Gradient Textured
![Page 25: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/25.jpg)
Smooth via Antialiasing
Multiple levels of transparency
Perfect for parallelism
![Page 26: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/26.jpg)
Path Approximation
Curves
Polygon
![Page 27: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/27.jpg)
Gradient: Expensive
CPU: sequential, tedious
GPU: parallel, still a lot of work
![Page 28: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/28.jpg)
Blur Shadow: Really Posh
Involved pixel “blending”
GPU: parallel, still tedious
![Page 29: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/29.jpg)
Transformation
Scaling
RotationPerspective
![Page 30: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/30.jpg)
Text Rasterization
![Page 31: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/31.jpg)
Font Atlas
Bye
Buffer
![Page 32: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/32.jpg)
Simple to Complex
Hello!
Simple shapeSolid color
CurvesGradient brushTextured stroke
Blur shadowSerif textRotated
Make it as an image
![Page 33: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/33.jpg)
Backing Store
![Page 34: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/34.jpg)
Maps
Tile
![Page 35: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/35.jpg)
Pinch to Zoom
when you pinch...
![Page 36: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/36.jpg)
Responsive Interface
Processor
Rendering
User interaction
decoupled
![Page 37: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/37.jpg)
Rendering vs Interaction
Web Page
Backing StoreScreen
Rendering
User interaction
![Page 38: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/38.jpg)
Checkerboard Pattern
![Page 39: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/39.jpg)
Progressive Rendering
Crisp but slow
Fast but blurry
![Page 40: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/40.jpg)
Tiling System
CPU GPU
........
Texture upload
![Page 41: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/41.jpg)
Tile Transformation
Panning = Translation Zooming = Scaling
![Page 42: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/42.jpg)
Backing Store Support
Texture-based
Vector-based
Honeycomb and later
![Page 43: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/43.jpg)
Y U NOposition:fixed
![Page 44: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/44.jpg)
Layer & Compositing
![Page 45: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/45.jpg)
Mechanics of Animation
Initialization
Animation step
“Hey, this is good stuff. Keep it around as texture #42.”
“Apply [operation] to texture #42.”
![Page 46: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/46.jpg)
Elements = Layer
![Page 47: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/47.jpg)
Typical Animation
opacity, movement, scaling, rotation, ...
![Page 48: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/48.jpg)
Immediate Mode
every animation tick
setInterval(function() { box.draw(x, y); x += 10;}, 20);
![Page 49: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/49.jpg)
Scene Graph
box.drawInto(layer);setInterval(function() { layer.translate(10, 0);}, 20);
Change transformation matrix
![Page 50: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/50.jpg)
Logical 3-D
![Page 51: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/51.jpg)
Keep the Textures
![Page 52: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/52.jpg)
Compositing Support
Honeycomb and later
![Page 53: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/53.jpg)
Well-known Trick
-webkit-transform: translateZ(0)
forcing 3-D transform
Not needed for CSS animation!
![Page 54: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/54.jpg)
Magical Advice
Use translate3d for hardware acceleration
Misleading (at best)
![Page 55: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/55.jpg)
Debugging in Safari
defaults write com.apple.Safari IncludeInternalDebugMenu 1
![Page 56: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/56.jpg)
Compositing Indicators
Composited layer
Container layer
No texture
Overflow
Texture (number = upload)
![Page 57: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/57.jpg)
Avoid Texture Reupload
No reupload Upload
OpacityPosition
SizeAnimation
Everything else!
“Hardware accelerated CSS”
![Page 58: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/58.jpg)
Examples
![Page 59: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/59.jpg)
Avoid Overcapacity
GPU = Limited silicon spaceLarge layer ➔ broken into “tiles”
![Page 60: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/60.jpg)
Prepare & Reuse
Viewport
Hide the layer offscreen
![Page 61: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/61.jpg)
Alpha Blending
Color change
![Page 62: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/62.jpg)
Alpha Blending: The Trick
Blended with
![Page 63: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/63.jpg)
Wrap-up
![Page 64: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/64.jpg)
Hardware Acceleration
Drawing primitives
Backing stores
Layer & compositing
![Page 65: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/65.jpg)
What Can We Learn?
Traditional graphics programming has been always full of tricks.
It will always be.
![Page 66: Understanding Hardware Acceleration on Mobile Browsers](https://reader031.fdocuments.us/reader031/viewer/2022020122/5561ef8bd8b42aa5068b56f2/html5/thumbnails/66.jpg)
There is No Silver Bullet