Scaling next generation video backend InfrastructureZafar Shahid FaceBook. Video Landscape •Color...
Transcript of Scaling next generation video backend InfrastructureZafar Shahid FaceBook. Video Landscape •Color...
Scaling next generation
video backend Infrastructure
Zafar Shahid
Video Landscape • Color Gamut
Video Landscape • VR 360 Video
Video Landscape • High Dynamic Range
• Too dark
• Too bright
• Display Ranges
Video Landscape • Resolution
• HD
• UHD
• 8K
• Frame Rate
• Interlace
• 30, 60,120 fps
• Bit Depth
• 8/10 bit
Video On Demand
• HLS Live Streaming
• Mpeg DASH
• RTMP
Live Streaming
Video Conferencing
Video Codecs
Video Processing @ Scale • Process millions of videos per day
• Peak load (New year Eve)
• Electricty Cost
Security • Transport Layer Security (TLS)
• DTLS (UDP)
• Certificate Exchange
Certificate
Exchange
Storage @ Scale • Each storage media has some Advanatages &
Limitations
• Cache
• DRAM
• Hard Drive
• SSD
• Tape Drives
File Abstraction
File Abstraction
With & Without Abstraction
File Transfer
“Not representative of actual server locations.”
File Transfer via Vcache
“Not representative of actual server locations.”
File write @Scale
Machine Learning@Scale
Machine learning@ Scale • What problems to solve
• Classification
• Ranking
• Video encoding decisions
• Content understanding
• Making decisions
• Training speed
• Training frequency
Machine Learning
ML @Scale - Challenges
ML @Scale – AI Eco System
ML @Scale – AI Eco System
ML @Scale – Web Tier
ML @Scale – Compute/Memory & Storage
Tier
ML @Scale – GPU Processing Tier
ML @Scale – Mapping Hardware to ML
What is @ Scale • Nothing is available off-the-shelf.
• Build your own
• What is next biggest problem @Scale
Questions?