Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
-
Upload
chris-adamson -
Category
Technology
-
view
231 -
download
0
Transcript of Building A Streaming Apple TV App (CocoaConf San Jose, Nov 2016)
Building a Streaming Apple TV app
Chris Adamson (@invalidname) CocoaConf San Jose • November, 2016
Slides available at slideshare.net/invalidname Code available at github.com/invalidstream
I can haz video app?
Client Backend
Login Authentication
Browse/search CMS
Playback Streaming
Content
Production
Encoding
Monetization
Digital Media
• Codecs — How we encode (represent) audio or video in a digital form
• AAC, Apple Lossless, H.264, ProRes
• Containers — How we deliver encoded media
• MP3, .aac, Shoutcast, .mov, .mp4
Codecs
• From “coder / decoder”
• Simplest: PCM (audio), M-JPEG (video)
• Modern: AAC, H.264/AVC
• Next gen: H.265 (HEVC), VP9
Containers: QuickTime
• Dates back to the early 1990s
• Originally used Mac dual-fork files
• Content agnostic: nearly any media content can be contained in a QuickTime file
• MPEG-4 adopted QuickTime as basis of its file format (1998)
QuickTime: OK for playback
• Resolve all references into a single file (flatten) or export (possibly re-encoding) to a file
• Dual-fork approach abandoned to support QuickTime for Windows (and later, Mac OS X)
QuickTime: Bad for Streaming
• Metadata at end of file meant you needed the whole file
• First solution: “Quick-Start”, which put metadata at beginning of file
• Later: QuickTime Streaming Server
• Also-ran to RealPlayer, Windows ASF, Flash
Streaming
• Different from progressive download in that the client does only wants the media for the current playback time
• Live sources are possible
Livestreaming• Essential to some content types: breaking news,
sports, one-of-a-kind events
• Chat rooms allow interaction with audience during the program
• It's just fun
• Twitch.tv often has over 1 million simultaneous viewers
Bad old streaming
• Real, ASF, Flash, QTSS – socket-based connections
• Hard to scale
• Bad for mobile
• Not on port 80; frequently blocked
Adaptive streaming
• Send “stream” as a series of small files over HTTP, expect client to reassemble them
• Easy to scale
• Good for mobile / inconsistent networks
• Runs on port 80; never blocked
HTTP Live Streaming
• Apple’s de facto standard for adaptive streaming
• Supported on iOS (iPhone, Apple TV) from day one
• Also: QuickTime, Safari, Microsoft Edge, Flash, Android, Roku, etc
• Standards-based competitor: MPEG-DASH
Streaming playback @IBAction func handlePlayButtonTapped(_ sender: AnyObject) { guard let demoItem = demoItem else { return }
self.playerVC?.player?.pause() let playerVC = AVPlayerViewController() playerVC.showsPlaybackControls = true let player = AVPlayer(url: demoItem.url) playerVC.player = player present(playerVC, animated: true) player.play() self.playerVC = playerVC }
This is identical to local file playback
HLS, how do you even?
• Source media is split into short (~10 second) segments, as MPEG-2 transport stream (.ts) files
• iOS 10 / macOS Sierra support “Fragmented MP4”, which is potentially MPEG-DASH compatible
• An .m3u8 playlist provides a manifest of files to be played back
#EXTM3U#EXT-X-TARGETDURATION:10#EXT-X-VERSION:3#EXT-X-MEDIA-SEQUENCE:0#EXT-X-PLAYLIST-TYPE:VOD#EXTINF:10.00000,fileSequence0.ts#EXTINF:10.00000,fileSequence1.ts#EXTINF:10.00000,fileSequence2.ts#EXTINF:10.00000,fileSequence3.ts#EXTINF:10.00000,fileSequence4.ts#EXTINF:10.00000,fileSequence5.ts#EXTINF:10.00000,fileSequence6.ts#EXTINF:10.00000,fileSequence7.ts#EXTINF:10.00000,fileSequence8.ts#EXTINF:10.00000,fileSequence9.ts#EXTINF:7.40000,fileSequence10.ts#EXT-X-ENDLIST
mediafilesegmenter
• Takes a source MP4 file and splits it into segment files and an .m3u8 playlist
• Resulting directory can be hosted on any http server to provide stream
Why is this better than a flat file?
• Easier to skip around
• Can be encrypted / DRM’ed
• Also, about that bitrate…
Variant Playlists
• Allow you to encode at multiple bitrates
• Client determines at runtime if it is keeping up, and whether to request a different variant for the next 10 sec segment
• Instead of “buffering” stalls, viewers just see degraded image/sound quality
variantplaylistcreator#EXTM3U#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=1269452,BANDWIDTH=1304420,CODECS="mp4a.40.2, avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONEwifi-high/prog_index.m3u8#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=5172486,BANDWIDTH=5330176,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONEbroadband-high/prog_index.m3u8#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=2599260,BANDWIDTH=2677271,CODECS="mp4a.40.2, avc1.4d4016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONEbroadband-low/prog_index.m3u8#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=757070,BANDWIDTH=782983,CODECS="mp4a.40.2, avc1.42e016",RESOLUTION=640x360,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONEwifi-low/prog_index.m3u8#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=474503,BANDWIDTH=507149,CODECS="mp4a.40.2, avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONEcellular-high/prog_index.m3u8#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=224956,BANDWIDTH=243197,CODECS="mp4a.40.2, avc1.42e015",RESOLUTION=428x240,FRAME-RATE=15.000,CLOSED-CAPTIONS=NONEcellular-low/prog_index.m3u8#EXT-X-STREAM-INF:AVERAGE-BANDWIDTH=33400,BANDWIDTH=33547,CODECS="mp4a.40.2",CLOSED-CAPTIONS=NONEaudio-only/prog_index.m3u8
Compressor’s HLS bitratesTable 1
Broadband high 5000Broadband low 2500Wi-fi high 1250Wi-fi low 735Cellular high 450Cellular low 225Audio only 32
HLS livestreams• Exactly the same as the VOD streams we’ve
seen before
• Support variant playlists and encryption
• Difference: the playlist doesn’t have a #EXT-X-ENDLIST
• Client knows to refresh the playlist periodically to fetch new segment files
mediastreamsegmenter
• Works just like mediafilesegmenter to create the segment files and a playlist
• Takes as input an MPEG-2 transport stream
• Nothing in Apple’s developer toolchain can actually create such a stream
Streaming servers
• Services that create the stream files from an input stream you provide
• Can also transcode to lower bitrates and provide a variant playlist
• Can also transmux to non-HLS formats
• Generally you send them RTMP
Wowza
• Top streaming service provider
• Wowza Streaming Engine — runs on your server (real or virtual)
• Wowza Streaming Cloud — streaming as a service
Wirecast
• Proprietary streaming production software
• Mixes shots, captures from cameras, iOS devices, or screen, highly customizable
• Prices start at $500
• Watch for a 1/3 off sale on Black Friday
Securing Streams
• With -k option, mediafilesegmenter and mediastreamsegmenter will AES-128 encrypt segments, using keys from the specified location
• FairPlay Streaming for HLS provides severe DRM.
• See WWDC 2015 session “Content Protection for HTTP Live Streaming”
Takeaways• Video streaming is the best thing the Apple TV
does
• HLS streams are just flat files on a web server
• Create them with mediafilesegmenter or a streaming server
• For lifestreams RTMP to the streaming server with OBS, Wirecast, etc.
Building a Streaming Apple TV app
Chris Adamson (@invalidname) CocoaConf San Jose • November, 2016
Slides are available at slideshare.net/invalidname Code is available at github.com/invalidstream