Developing Windows ® CE Multimedia Solutions Using Windows Media Rod Deyo Program Manager Windows...

Developing WindowsDeveloping Windows®® CE CE Multimedia Solutions Multimedia Solutions Using Windows MediaUsing Windows Media

Rod DeyoRod DeyoProgram ManagerProgram ManagerWindows CE PlatformsWindows CE PlatformsMicrosoft CorporationMicrosoft Corporation

8-3078-307

AgendaAgenda

What is Windows CE DirectXWhat is Windows CE DirectX®®?? Software availabilitySoftware availability DXPAK and Pocket PCDXPAK and Pocket PC Streaming multimediaStreaming multimedia Windows Media Player controlWindows Media Player control Windows Media technologiesWindows Media technologies DirectShowDirectShow®®

Windows CE DirectXWindows CE DirectX

Designed for OEMs to build DirectX Designed for OEMs to build DirectX and Windows Media enabled platforms and Windows Media enabled platforms using Windows CE using Windows CE

Adds to Windows CE Platform Builder Adds to Windows CE Platform Builder 2.12 and 3.02.12 and 3.0

Example ApplicationsExample Applications

Alpha-blended displays for advanced Alpha-blended displays for advanced set top boxes and automotive systemsset top boxes and automotive systems

Viewing Internet multimedia content Viewing Internet multimedia content with Web browser with Web browser

WM Audio and MP3 juke boxes and WM Audio and MP3 juke boxes and playersplayers

Video time-shiftVideo time-shift Advertising signageAdvertising signage

ProductsProducts

WebTVWebTV™™ Microsoft Microsoft®® TV (MSTV) client TV (MSTV) client MSNMSN™™ Web Companion Web Companion Microsoft AutoPC Automotive Microsoft AutoPC Automotive

Adaptation Kit (AAK)Adaptation Kit (AAK) Windows-based Terminal (WBT)Windows-based Terminal (WBT) Portable audio playerPortable audio player Digital music systemsDigital music systems Multimedia internet appliancesMultimedia internet appliances

Windows CE Multimedia Windows CE Multimedia ArchitectureArchitecture

Graphics and Audio DriversGraphics and Audio Drivers

DirectShow Filter GraphDirectShow Filter Graph

WM Player ControlsWM Player Controls

Application/BrowserApplication/Browser

DirectDraw/GDI DirectSound/WaveDevDirectDraw/GDI DirectSound/WaveDev

Windows Windows Media Media ComponentsComponents DirectX DirectX

ComponentsComponents

HardwareHardware

Sample Sample DriversDrivers

Content/HTML PageContent/HTML Page

Windows CE DirectXWindows CE DirectX DirectDrawDirectDraw®® Graphics Graphics

Hardware accelerationHardware acceleration Surface management, tear-free fast blts, Surface management, tear-free fast blts,

overlays, clippers, palettes, video portsoverlays, clippers, palettes, video ports DirectSoundDirectSound®® Audio Audio

Buffer management, software mixing, Buffer management, software mixing, rate and format conversion, capturerate and format conversion, capture

DirectShow MultimediaDirectShow Multimedia Flexible multimedia playbackFlexible multimedia playback Multiple file formats and codecsMultiple file formats and codecs

Windows CE MultimediaWindows CE Multimedia

Windows Media Player controlWindows Media Player control Run, pause, stop media controlRun, pause, stop media control Uses DirectShowUses DirectShow

Windows Media technologiesWindows Media technologies Streaming with DirectShow filtersStreaming with DirectShow filters Windows Media formatsWindows Media formats Audio, video, speech codecsAudio, video, speech codecs

Author and encode once, play Author and encode once, play anywhere!anywhere!

AgendaAgenda



DirectX Platform Adaptation Kit DirectX Platform Adaptation Kit (DXPAK) for Platform Builder(DXPAK) for Platform Builder

DXPAK 1.0 released 12-99DXPAK 1.0 released 12-99 Enables hardware accelerated multimediaEnables hardware accelerated multimedia

DirectDraw 6.1DirectDraw 6.1 DirectSound 6.1DirectSound 6.1 DirectShow 6.1DirectShow 6.1 Sample drivers – DDGPE, DSHAL, WaveDevSample drivers – DDGPE, DSHAL, WaveDev

Platform Builder 2.12Platform Builder 2.12 Supports x86, MIPS FP, and SH4 CPUsSupports x86, MIPS FP, and SH4 CPUs


DXPAK 1.1 released 5-00DXPAK 1.1 released 5-00 Enables Windows Media 4.1 clientEnables Windows Media 4.1 client

DirectShow WM streaming filterDirectShow WM streaming filter WM ASF/ASX FormatsWM ASF/ASX Formats Microsoft WMA and MPEG-4, Sipro ACELP.net, Microsoft WMA and MPEG-4, Sipro ACELP.net,

Fraunhofer MP3 codecsFraunhofer MP3 codecs

Microsoft Windows Media Player 6.4 Microsoft Windows Media Player 6.4 ActiveXActiveX®® Controls Controls

Platform Builder 2.12Platform Builder 2.12 Platform Builder 3.0 version soon!Platform Builder 3.0 version soon!

Additional CPUsAdditional CPUs Further performance optimizationsFurther performance optimizations

AgendaAgenda


DXPAK And Pocket PCDXPAK And Pocket PC

Pocket PCPocket PC Product SDKProduct SDK WM Player app, audio codecs onlyWM Player app, audio codecs only Local file-based playback onlyLocal file-based playback only

Windows CE DXPAKWindows CE DXPAK Platform Builder add-onPlatform Builder add-on WM Player controlWM Player control Audio, video, speech codecsAudio, video, speech codecs DirectShow-based Windows Media DirectShow-based Windows Media

streamingstreaming

DXPAK And Pocket PC DXPAK And Pocket PC FeatureFeature DXPAK 1.1DXPAK 1.1 Pocket PCPocket PC

Local File PlaybackLocal File Playback YesYes Yes*Yes*

WMT StreamingWMT Streaming YesYes NoNo

WMP applicationWMP application NoNo Yes*Yes*

WMP controlWMP control YesYes NoNo

DirectXDirectX YesYes NoNo

WMA codecWMA codec YesYes Yes*Yes*

MP3 codecMP3 codec YesYes Yes*Yes*

MPEG-4 codecMPEG-4 codec YesYes NoNo

Speech codecSpeech codec YesYes NoNo* As part of application* As part of application

AgendaAgenda


Effective StreamingEffective Streaming

Data is delivered only as neededData is delivered only as needed Audio and video stream display Audio and video stream display

synchronizationsynchronization Data compressionData compression

Audio (44.1 kHz 16 bit stereo) Audio (44.1 kHz 16 bit stereo) 1.41 mbps 1.41 mbps 64 kbps is 22:1 64 kbps is 22:1

Video (320x240 16 bpp 30 fps) Video (320x240 16 bpp 30 fps) 36.9 mbps 36.9 mbps 300 kbps is 122:1 300 kbps is 122:1

Effective StreamingEffective Streaming

Coordinated application, network, Coordinated application, network, and physical layer protocols and physical layer protocols Network conditions and bandwidth taken Network conditions and bandwidth taken

into considerationinto consideration Packet loss mitigatedPacket loss mitigated

Client and server work together Client and server work together automatically automatically

Simple application controlsSimple application controls

AgendaAgenda


WM Player ControlWM Player Control

Windows Media Player Control v6.4 for CEWindows Media Player Control v6.4 for CE Scriptable from JScriptScriptable from JScript™™

VCR-Like Controls: Play, Stop, Pause, FF, FRVCR-Like Controls: Play, Stop, Pause, FF, FR Audio Controls: Balance, Volume, MuteAudio Controls: Balance, Volume, Mute Information Retrieval: Title, Author, CopyrightInformation Retrieval: Title, Author, Copyright Monitoring: Stream status, network receptionMonitoring: Stream status, network reception Seamless stream switchingSeamless stream switching Playlists, URL flipping, MarkersPlaylists, URL flipping, Markers Componentized: Toolbar, Statbar, BkgnbmpComponentized: Toolbar, Statbar, Bkgnbmp

<Object ID="MediaPlayer"<Object ID="MediaPlayer"

CLASSID="CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95"CLASSID="CLSID:22d6f312-b0f6-11d0-94ab-0080c74c7e95"

TYPE="application/x-oleobject"TYPE="application/x-oleobject"

WIDTH="320"WIDTH="320"

HEIGHT="240"HEIGHT="240"

STYLE="position:absolute;STYLE="position:absolute;

left:50px;left:50px;

top:50px;“top:50px;“

>>

<PARAM NAME="FileName" <PARAM NAME="FileName" VALUE=http://example.microsoft.com/media/sample.asf>VALUE=http://example.microsoft.com/media/sample.asf>

<PARAM NAME="ShowStatusBar" VALUE="1"><PARAM NAME="ShowStatusBar" VALUE="1">

<PARAM NAME="ShowControls" VALUE="1"><PARAM NAME="ShowControls" VALUE="1">

</Object> </Object>

No CODEBASE=No CODEBASE=url url attribute – WMP control attribute – WMP control must be part of WinCE must be part of WinCE device imagedevice image

Make controls Make controls visiblevisible

WMP Control EmbeddingWMP Control Embedding

WMP Control HTML TagsWMP Control HTML Tags

Tags have identical behavior to desktop Tags have identical behavior to desktop <Object> Tag<Object> Tag

Provides full WMP control functionality in IEProvides full WMP control functionality in IE <Embed> Tag<Embed> Tag

Netscape compatibility, limited on IE Netscape compatibility, limited on IE <A> Tag<A> Tag

Not supported by DXPAK WMP controlNot supported by DXPAK WMP control Use Windows CE WMPHLPR sample applicationUse Windows CE WMPHLPR sample application

<IMAGE> Tag<IMAGE> Tag MPEG-1, AVI, and QT v2 formats onlyMPEG-1, AVI, and QT v2 formats only

WMP Control HostWMP Control Host

Hosted by OCX 96 compatible container Hosted by OCX 96 compatible container Internet Explorer compatible browserInternet Explorer compatible browser

Control can be eitherControl can be either WindowedWindowed WindowlessWindowless

WMP Control DifferencesWMP Control Differences

WMP control is NOT the WMP WMP control is NOT the WMP applicationapplication No app, no menus or dialog boxesNo app, no menus or dialog boxes

Subset of desktop WMP v6.4 controlSubset of desktop WMP v6.4 control Smaller, robust control for Smaller, robust control for

Embedded devicesEmbedded devices Compatibility with existing media contentCompatibility with existing media content

Features not supportedFeatures not supported PowerPointPowerPoint®® (PPT) streaming (PPT) streaming Hotspots ASF authoringHotspots ASF authoring

AgendaAgenda


WM TechnologiesWM Technologies

Windows Media Technologies v4.1 for CEWindows Media Technologies v4.1 for CE Componentized DirectShow source and transform Componentized DirectShow source and transform

filtersfilters WM Format container and metadata files WM Format container and metadata files

(ASF, ASX, NSC)(ASF, ASX, NSC) ASF streaming (UDP, TCP, HTTP)ASF streaming (UDP, TCP, HTTP) Codecs (WMA, MPEG-4, Sipro ACELP, MP3)Codecs (WMA, MPEG-4, Sipro ACELP, MP3) AuthenticationAuthentication Intelligent multiple bit-rate streamingIntelligent multiple bit-rate streaming Event driven stream switchingEvent driven stream switching

HeaderHeader

AudioStreamAudioStream

IndexIndex

Video Stream #1Video Stream #1

Video Stream #nVideo Stream #n

Script StreamScript Stream

TimeTime

WM Technology FormatsWM Technology Formats

Advanced Streaming Format (ASF)Advanced Streaming Format (ASF) Application-level transport and program streamApplication-level transport and program stream Stores media and metadata in one fileStores media and metadata in one file Contains multiple bit-rate video streams Contains multiple bit-rate video streams

synchronized with audio streamsynchronized with audio stream

WM Technology FormatsWM Technology Formats Advanced Streaming FormatAdvanced Streaming Format

Time based synchronizationTime based synchronization Interleaved data optimized for HTTP Interleaved data optimized for HTTP

packetspackets Advanced Stream Redirector (ASX) v3Advanced Stream Redirector (ASX) v3

Hyperlinks to ASF files Hyperlinks to ASF files Server and protocol rollover rulesServer and protocol rollover rules PlaylistsPlaylists

Windows Media Services Station (NSC)Windows Media Services Station (NSC) Multicast channelsMulticast channels

WM Technology ProtocolsWM Technology Protocols

MulticastMulticast One packet is broadcast to multiple clientsOne packet is broadcast to multiple clients Requires use of Windows Media Station files (NSC)Requires use of Windows Media Station files (NSC)

MMSMMS TCP connection for command, UDP or TCP for dataTCP connection for command, UDP or TCP for data Protocol rollover (UDP -> TCP -> HTTP) Protocol rollover (UDP -> TCP -> HTTP)

HTTPHTTP HTTP packets can be received through firewallsHTTP packets can be received through firewalls Does not support protocol rolloverDoes not support protocol rollover

Local FileLocal File

WM Technology CodecsWM Technology Codecs

Windows Media Audio v2Windows Media Audio v2 8 – 48 kHz stereo at 56 – 128 kbps8 – 48 kHz stereo at 56 – 128 kbps Near FM quality at 28.8 kbps, near CD quality Near FM quality at 28.8 kbps, near CD quality

at 64 kbpsat 64 kbps Microsoft MPEG-4 Video v2, v3 Microsoft MPEG-4 Video v2, v3

28.8 – 1000 kbps28.8 – 1000 kbps 320x240 resolution video 30 fps at 300 kbps320x240 resolution video 30 fps at 300 kbps

Sipro ACELP.net Sipro ACELP.net Speech quality 8-16 KHz mono at 5-16 kbpsSpeech quality 8-16 KHz mono at 5-16 kbps

Fraunhofer MPEG-1 Layer 3Fraunhofer MPEG-1 Layer 3 Near CD quality at 128 kbpsNear CD quality at 128 kbps

WMT ComponentizationWMT Componentization

WMT for CE is componentized into WMT for CE is componentized into the following modulesthe following modules MMS streamingMMS streaming HTTP streamingHTTP streaming File streamingFile streaming Broadcast and multi-bit rate streamingBroadcast and multi-bit rate streaming ASX ASX Individual codecsIndividual codecs

WMT DifferencesWMT Differences

ASX v3ASX v3 PREVIEWDURATION, BANNER, LOGO PREVIEWDURATION, BANNER, LOGO

elements not supportedelements not supported v1 and v2 formats not supportedv1 and v2 formats not supported

AgendaAgenda


DirectShowDirectShow Powerful, configurable connection architecture for Powerful, configurable connection architecture for

streaming mediastreaming media Automatically handles audio and video synchronization Automatically handles audio and video synchronization Source, transform, and rendering filtersSource, transform, and rendering filters Filter graph: a collection of connected filtersFilter graph: a collection of connected filters Every filter is pluggable/replaceableEvery filter is pluggable/replaceable WM ASF streamer, parser, and decoder filtersWM ASF streamer, parser, and decoder filters

ASF ASF StreamerStreamer

ASF ASF ParserParser

WMAWMAdecoderdecoder

MPEG-4MPEG-4decoderdecoder

AudioAudiorendererrenderer

VideoVideorendererrenderer

Typical Streaming Filter GraphTypical Streaming Filter Graph

DirectShow ArchitectureDirectShow Architecture

HardwareHardware

DirectDraw/DirectSoundDirectDraw/DirectSoundFile sysFile sys

ActiveXActiveXControlsControls

Filter graph managerFilter graph manager

Source Source filterfilter

Transform Transform filterfilter ClockClock

COM COM interfacesinterfaces

Browser/ApplicationBrowser/Application

RendererRendererfilterfilter

InternetInternet

DirectShow FiltersDirectShow Filters Allocate buffersAllocate buffers Negotiate media types and connections Negotiate media types and connections

to other filters to other filters Push data through the filter graphPush data through the filter graph Send events to the filter graph manager Send events to the filter graph manager Synchronize multiple threadsSynchronize multiple threads State - run, pause, stoppedState - run, pause, stopped

DirectShow InterfacesDirectShow Interfaces Filter Graph ManagerFilter Graph Manager

Create filter graph - IGraphBuilderCreate filter graph - IGraphBuilder Run, pause, stop - IMediaControlRun, pause, stop - IMediaControl Seek to position or time – IMediaSeekingSeek to position or time – IMediaSeeking Events (rendering complete) – IMediaEventsEvents (rendering complete) – IMediaEvents Audio and video control – IBasicAudio, Audio and video control – IBasicAudio,

IBasicVideoIBasicVideo Filters – usually hidden by FGMFilters – usually hidden by FGM

Control, pins, info – IBaseFilterControl, pins, info – IBaseFilter Media-related data – IMediaSampleMedia-related data – IMediaSample Position, seeking – IMediaSeekingPosition, seeking – IMediaSeeking

DirectShow Renderer FiltersDirectShow Renderer Filters AudioAudio

Volume and balance controls - IBasicAudioVolume and balance controls - IBasicAudio Uses DirectSound to allow multiple audio streams Uses DirectSound to allow multiple audio streams

to be played and mixed at onceto be played and mixed at once Fallback to WaveOut if DirectSound not presentFallback to WaveOut if DirectSound not present

VideoVideo Video window controls – IBasicVideoVideo window controls – IBasicVideo Uses DirectDraw for optimal video playback Uses DirectDraw for optimal video playback

performanceperformance Will use hardware overlay surfaces if supportedWill use hardware overlay surfaces if supported Uses hardware accelerated YUV to RGB color Uses hardware accelerated YUV to RGB color

space conversionspace conversion Fallback to GDI if DirectDraw not presentFallback to GDI if DirectDraw not present

DirectShow DifferencesDirectShow Differences

Not supportedNot supported WDM KS or CSAWDM KS or CSA IKsPropetySetIKsPropetySet Audio or video captureAudio or video capture WM encodingWM encoding

DirectShow Media Playback – DirectShow Media Playback – It’s Easy!It’s Easy!

Declare interface pointers and event code storageDeclare interface pointers and event code storage

// playmovie.cpp// playmovie.cpp

HRESULT PlayMovie(LPTSTR lpszmovie)HRESULT PlayMovie(LPTSTR lpszmovie)

{{

// interface pointers we’ll need// interface pointers we’ll need

IGraphBuilder *pGB = NULL;IGraphBuilder *pGB = NULL;

IMediaControl *pMC = NULL;IMediaControl *pMC = NULL;

IMediaEvent *pME – NULL;IMediaEvent *pME – NULL;

// something to hold event code // something to hold event code

LONG evCode; LONG evCode;

DirectShow PlaybackDirectShow Playback

CoCreateinstance of FilterGraph object and get CoCreateinstance of FilterGraph object and get IGraphBuilder interfaceIGraphBuilder interface

// instantiate a filter graph as in-proc server // instantiate a filter graph as in-proc server

if (FAIL(CoCreateInstance(CLSID_FilterGraph,if (FAIL(CoCreateInstance(CLSID_FilterGraph,

NULL,NULL,

CLSCTX_INPROC,CLSCTX_INPROC,

IID_GraphBuilder,IID_GraphBuilder,

(void**) &pGB)))(void**) &pGB)))

return E_FAIL;return E_FAIL;

No filter graph exists yet since media type and No filter graph exists yet since media type and format are unknown to GraphBuilderformat are unknown to GraphBuilder


Get IMediaControl to run filter graph and IMediaEvent Get IMediaControl to run filter graph and IMediaEvent to wait for end of stream to wait for end of stream

// get interface to control graph// get interface to control graphif(FAIL(pGB->QueryInterface(IID_MediaControl,if(FAIL(pGB->QueryInterface(IID_MediaControl,

(void **) &pMC)))(void **) &pMC)))

return return E_FAILE_FAIL;;

// get interface for done event// get interface for done event

if(FAIL(pMC->QueryInterface(IID_IMediaEvent,if(FAIL(pMC->QueryInterface(IID_IMediaEvent,

(void **) &pME)))(void **) &pME)))return return E_FAILE_FAIL;;

Build the filter graph from the media and formatBuild the filter graph from the media and format

// build the filter graph // build the filter graph

if(FAIL(pGB->RenderFile(lpszMovie, NULL)))if(FAIL(pGB->RenderFile(lpszMovie, NULL)))

return return E_FAILE_FAIL;;


Start playing the media and block application waiting Start playing the media and block application waiting for EC_COMPLETE from renderer filterfor EC_COMPLETE from renderer filter

// play the file// play the filepMC->Run()))pMC->Run()))

// and block indefinitely until completed// and block indefinitely until completed

pME->WaitForCompletion(INFINITE,&evCode)))pME->WaitForCompletion(INFINITE,&evCode)))

// release interfaces// release interfaces

pGB->Release(); pMC->Release(); pME->Release();pGB->Release(); pMC->Release(); pME->Release();

}}

Release interfacesRelease interfaces

SummarySummary

OEMs can now build DirectX OEMs can now build DirectX and Windows Media enabled platforms using and Windows Media enabled platforms using Windows CE and Platform BuilderWindows CE and Platform Builder

Componentized Windows CE versions of Componentized Windows CE versions of Windows Media Technologies v4.1Windows Media Technologies v4.1 Windows Media Player Control v6.4Windows Media Player Control v6.4 DirectShow 6.1DirectShow 6.1

Further InformationFurther Information

White papersWhite papers Multimedia Streaming on Windows CEMultimedia Streaming on Windows CE DirectX and Windows Media Technologies on CEDirectX and Windows Media Technologies on CE

Windows CE DX WebsiteWindows CE DX Website http://microsoft.com/windows/embedded/ce/guide/fhttp://microsoft.com/windows/embedded/ce/guide/f

eatures/directx.aspeatures/directx.asp MSDN Developer CenterMSDN Developer Center

http://msdn.microsoft.com/windowsmediahttp://msdn.microsoft.com/windowsmedia E-mail E-mail

[email protected]@microsoft.com

http://microsoft.com/windows/embedded/ce/guide/features/directx.asp

http://microsoft.com/windows/embedded/ce/guide/features/directx.asp

http://msdn.microsoft.com/windowsmedia

Questions?Questions?

Developing Windows ® CE Multimedia Solutions Using Windows Media Rod Deyo Program Manager Windows...

Documents

Transcript of Developing Windows ® CE Multimedia Solutions Using Windows Media Rod Deyo Program Manager Windows...