Video Streaming on iOS - SiliconIndia · Copyright © 2012-13 • TechJini Solutions Pvt Ltd •...

Copyright © �� -�� • TechJini Solutions Pvt Ltd • All rights reserved �

Video Streaming on iOS

Shravya Shridhar,JinilabsProduct division,TechJini Solutions Pvt Ltd

Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 2

Agenda

● What is streaming● Http live streaming● What do you need to set up HLS● How HLS work● Timed metadata in stream● Media encryption


● What is streaming● Multimedia is constantly received by and presented to an end-

user while being delivered by a streaming provider● Listening or watching content in ‘real time'● Content can be audio, video, plain text

● Types of streaming● Live streaming ex: Listening to radio● Video on demand ex: Watching a movie


Codec, bitstream, Transport,bit rate

● Video codec ex: H.264 or VP8

● Audio codec ex: MP3 or AAC

● Transport ex: MPEG-2

● quality of an audio or video file

● Protocols :

● HTTP/HTTP/RTSP/RTMP/RTP


HTTP Live Streaming

● Adaptive streaming communications protocol

● Supported devices: iOS, Apple TV devices , Macs running OSX in Snow Leopard or late, android 4.0 and above

● Any app store app delivering video greater than 5Mb should use HLS

● Supported in most of the streaming servers adobe,wowza,realnetworks,akmai etc


What do you need to implement HLS

● Server Components ● Web server (apache may be)● Content storage ● CDN● Apple provided tools (segmenter ,encoder ,validator)

● Protocol● HTTP

● Client Components● Apple default player or AVFoundation framework or html 5 player embedded

in mobile web page


HTTP Live streaming


Preparing videos for delivery● Current Apple recommendation:

– Video: H.264 Baseline Level 3.0,3.1and Main Level 3.1– Audio:

● HE-AAC or AAC-LC up to 48 kHz, stereo audio● MP3 (MPEG-1 Audio Layer 3) 8 kHz to 48 kHz, stereo

audio– Bit rate recommended :

● 64,150,240,440,640kbps for cellular● 1240,1840,2540,4540Kbps for wifi

– Frame rate● 10 fps < 200 kbps. ● 12 to 15 fps < 300 kbps. ● 29.97 > 300Kbps

● Each segment is a MPEG-2 transport stream with a .ts extension.

Copyright © �� -�� • TechJini Solutions Pvt Ltd • All rights reserved �

Video Segments and playlist structure:● mediastreamsegmenter -s 3 -D -f /Library/WebServer/Documents/stream 239.4.1.5:20103

● mediafilesegmenter -f VDO_DIR -l log/log.q1 -i avatar-q3.m3u8 -B q1- -t 10 -I src/avatar.ts

Output .m3u8 file structure


Creating variant playlist● Command:

variantplaylistcreator -resolution-tags -no-codecs-tags -o Avathar/avathar.m3u8 avathar-q1.m3u8 avathar/low.plist -iframe-url avathar-q2.m3u8 avathar/medium.plist avathar-q3.m3u8 avathar/high.plist

● Output:


Final m3u8 file for player


Validating streams● MediaStreamValidator -validate

http://devimages.apple.com/iphone/samples/bipbop/gear3/prog_index.m3u8

● Output:

http://devimages.apple.com/iphone/samples/bipbop/gear3/prog_index.m3u8


Client Player set up:

● Html tag

<video src="http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8"/>

● MPMediaViewController can be used as shown below:


Using AVFoundation framework


Using AVFoundation framework contd:

NSURL *videoUrl = [NSURL URLWithString:@"http://"];

/* Creating asset from the url */AVURLAsset *asset = [AVURLAsset URLAssetWithURL:newMovieURL options:nil]; /* Requesting tracks and playable key*/ NSArray *requestedKeys = [NSArray arrayWithObjects:kTracksKey, kPlayableKey, nil];

[asset loadValuesAsynchronouslyForKeys:requestedKeys completionHandler: ^{

dispatch_async( dispatch_get_main_queue(), ^{

/* If requested key present , then play else error*/[self playAsset:asset withKeys:requestedKeys];

}); }];


- (void) playAsset:(AVUrlAsset *)asset withKeys:(NSArray *)requestedKeys {

NSError *error = nil;AVKeyValueStatus keyStatus = [asset statusOfValueForKey:thisKey error:&error];

/* If there is error in fetching tracks */if (keyStatus == AVKeyValueStatusFailed){

[self assetFailedToPrepareForPlayback:error];return;

}

/* Creating AVPlayer Item*/AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset];

/* Creating AVPlayer Item from AVPlayerItem*/AVPlayer *player = [AVPlayer playerWithPlayerItem: playerItem];

player.allowsAirPlayVideo = YES; /* BY default YES*/

/* Creating view layer for AVPlayer*/AVPlayerLayer *playerLayer = [AVPlayerLayer playerLayerWithPlayer:player];

playerLayer.frame = [self getInitialFrameForPlayer];

/* Adding to the view layer */[self.view.layer addSublayer:playerLayer];

Copyright © �� -�� • TechJini Solutions Pvt Ltd • All rights reserved ��

/* Adding status key to observe the status change of player */ [[player currentItem] addObserver:self

forKeyPath:@”s t a t us ” options:NSKeyValueObservingOptionInitial | NSKeyValueObservingOptionNew context:MyStreamingMovieViewControllerPlayerItemStatusObserverContext];

/* Update the scrubber during normal playback. */timeObserver = [[player

addPeriodicTimeObserverForInterval:CMTimeMakeWithSeconds(1, NSEC_PER_SEC) queue:NULL usingBlock: ^(CMTime time) { [self updateSeekbar]; /* Here you can update seekbar , or fi nd out the buffer

level */

}] retain];

/* Notifi cation when player reaches end */[NSNotifi cationCenter defaultCenter] addObserver:self

selector:@selector(realPlayerItemDidReachEnd:) name:AVPlayerItemDidPlayToEndTimeNotifi cation object:[self.player currentItem]]; playerLayer.videoGravity = AVLayerVideoGravityResize;


Timed meta data in streams

● Adding song's artist name to audio streams, cricket score to live cricket match.

● ID3 format or an image file (JPEG or PNG)

● Apple client software automatically displays the still image with audio only file

● Adding observer to AVPlayer

/* Adding observer to read meta data */[self.player addObserver:self

forKeyPath:@”currentItem.timedMetadata” options:0

context:MyStreamingMovieViewControllerTimedMetadataObserverContext];


Reading the meta data from AVPlayer

- (void)observeValueForKeyPath:(NSString*) path ofObject:(id)object change:(NSDictionary*)change context:(void*)context {

if (context == MyStreamingMovieViewControllerTimedMetadataObserverContext) {

NSArray* array = [[player currentItem] timedMetadata]; /* get meta data object from player*/for (AVMetadataItem *metadataItem in array) {

if ([(NSString *)[timedMetadata key] isEqualToString: AVMetadataID3MetadataKeyGeneralEncapsulatedObject])

{

NSDictionary *propertyList = (NSDictionary *)[timedMetadata value];

/* Metadata payload could be the list of ads. */NSArray *newAdList = [propertyList objectForKey:@"ad-list"];

/* Or it might be an image url or cricket score*/NSString *imageURL = [propertyList objectForKey:@"cricket_scorel"];

}}}}


Media Encryption● Change of encryption key periodically through https● Supply the credentials in the didReceiveAuthenticationChallenge callback of NSURLConnection


Links for further help

● https://developer.apple.com/library/ios/#documentation/networkinginternet/conceptual/streamingmediaguide/Introduction/Introduction.html

● http://tools.ietf.org/html/draft-pantos-http-live-streaming-08

● http://www.streamingmedia.com/Articles/Editorial/What-Is-.../What-is-HLS-%28HTTP-Live-Streaming%29-78221.aspx


Questions ??


For further queries

[email protected]

mailto:[email protected]

Video Streaming on iOS - SiliconIndia · Copyright © 2012-13 • TechJini Solutions Pvt Ltd •...

Documents

Transcript of Video Streaming on iOS - SiliconIndia · Copyright © 2012-13 • TechJini Solutions Pvt Ltd •...