Multimedia

Audio,vedio and Images

• End user typically refer to vedio/audio using the respective file format• MP4 or AVI(audio vedio interleave)• Developer needs to understand what it is behind the scene.• An MP4 vedio is just like a zip file that can package files within it.• In multimedia this package format is referred to as container format.• A container need to package viedio elements,audio elements and

metadata that provide information about the title or cover art of the vedio.

Continue..

Keeping audio & vedio elements together is usually referred to as multiplexing.• MPEG-4(,mp4),Flash(.flv) & Ogg(.ogv) are example of popular

container format.• Audio & vedio in raw form very large & unsuitable for storage and

transport,so before packaging it in container format need to be compressed using compression algorithm.CODECS• H.264 AVC,Theora & VP8 are popular vedio codecs.• MP3,AAC & Vorbis are popular audio codecs.

• Android support several media codecs & container formats• An MPEG-4 container with encoded audio and vedio elements shown.

Encoded vedio(H.264 AVC)

Encoded audio(aac lc)

Metadata(title cover art)

• Once media is available in a container,it is ready to play.

• To play it first audio & vedio content is separated(de-multiplexed).

• The individual components are then decompressed back to the raw format using respective codec.

• Finally process by the smart device to play.

• Decompressing vedio elements result in displaying a series of images on the screen.• Decompressing the audio element result in a sound stream on the

speaker of the device• Both these elements are synchronized with the help of metadata

Multimedia API-playback

• Built-in app mechanism-requesting the pre-existing Android app to playback a media element from an app that require the multimrdia feature.• Developers perspective-send an intend to play the required media

using the built in Android App.• Rest is taken care by the built-in app itself

audio

• To play an audio using built-in app mechanism-use an implicit intent to start the intended music player app.

Intent i= new Intent(Intent.ACTION_VIEW);Uri uri= Uri.parse(“http://www.mobmusicstore.com/music.mp3”);i.setDataAndType(uri,”audio/*”);startActivity(i);

vedio

Playing a vedio using a built-in mechanism ia similar to playing an audio. Intent i= new Intent(Intent.ACTION_VIEW);Uri uri= Uri.parse(“http://www.mobmusicstore.com/vedio.mp4”);i.setDataAndType(uri,”vedio/*”);startActivity(i);

In-app Mechanism

• More control and flexibility to a developer to handle the media playback.• Does not require another app to play the desired media rather plays it

within app• Media player API is the engine

• MediaPlayer object behaves as a state machine & has its own lifecycle and states.• Media stream goes through the idle,initialized,prepared and started

states.• Idle state-MediaPlayer objects gets here as soon as created.• Initialized state-set the data source-a file path or an HTTP URL of the

media stream to play(setDataSource() method).• Prepare-media stream fetched and decoded to

playback(prepareAsync())

• Once it is prepared it can be start using start() followed by play() stop().• Remember to call release() once the mediaplayer object is not

required any more which takes it to end state.• MediaPlayer mediaPlayer = MediaPlayer.create(this, R.rap.song);

mediaPlayer.start(); mediaPlayer.pause();

• In order to start music from the beginning , you have to call reset() method. Its syntax is given below.• mediaPlayer.reset();

• isPlaying()This method just returns true/false indicating the song is playing or not• seekTo(position)This method takes an integer, and move song to that

particular second• getCurrentDuration()This method returns the current position of

song in milliseconds• getDuration()This method returns the total time duration of song in

milliseconds• setVolume(float leftVolume, float rightVolume)This method sets the

up down volume for this player

• setDataSource(FileDescriptor fd)This method sets the data source of audio/video file• selectTrack(int index)This method takes an integer, and select the

track from the list on that particular index • getTrackInfo()This method returns an array of track information