1Efficient multimedia query-by-content from mobile devices

2Efficient multimedia query-by-content from mobile devices

3Efficient multimedia query-by-content from mobile devices

4Efficient multimedia query-by-content from mobile devices

5Efficient multimedia query-by-content from mobile devices

.

6Efficient multimedia query-by-content from mobile devices

7Efficient multimedia query-by-content from mobile devices

8Efficient multimedia query-by-content from mobile devices

9Efficient multimedia query-by-content from mobile devices

10Efficient multimedia query-by-content from mobile devices

Example of a communication sequence

between a mobile device and a server using

query streaming.

11Efficient multimedia query-by-content from mobile devices

12Efficient multimedia query-by-content from mobile devices

13Efficient multimedia query-by-content from mobile devices

14Efficient multimedia query-by-content from mobile devices

The General query input format of MQF

15Efficient multimedia query-by-content from mobile devices

The General query output format of MQF

16Efficient multimedia query-by-content from mobile devices

Example instantiation of MQF with a simple query

17Efficient multimedia query-by-content from mobile devices

18Efficient multimedia query-by-content from mobile devices

19Efficient multimedia query-by-content from mobile devices

20Efficient multimedia query-by-content from mobile devices

<mqf>

<queryId>if_123</queryId>

<from>client.uri</from>

<query stream=“true”>

<replyType>image/jpeg</replyType>

<item queryLevel=“example”>

<mpeg7:MediaData64>

AaBbCc/==

</mpeg7:MediaData64>

</item>

</query>

</mqf>

Initial communication from the client using query streaming features of MQF.

<mqf>

<queryId>id_123</queryId>

<from>server.uri</from>

<replies>

<reply>

<item>http://server/image1.jpg</item>

<index>1</index>

</reply>

<reply>

<item>http://server/image2.jpg</item>

<index>2</index>

</reply>

</replies>

</mqf>

Server reply to the example

Server

Reply

Server

Reply

<query>

<item queryLevel=“exact”>

<dc:Creator>John Doe</dc:Creator>

</item>

</query>

Client refinement: adding a term to the query.

21Efficient multimedia query-by-content from mobile devices

22Efficient multimedia query-by-content from mobile devices

Efficient multimedia query-by-content from mobile devices 23

Transmission/Storage Medium

IP MP4Delivery

Layer

Multiplex

MPEG-2 ATM ...

Multiplexed Streams

DemultiplexMultiplex

Schema

streams

Description

streams

Compression

Layer

Elementary Streams

Multimedia

streams

Upstream

Data

Application

APIs

Defines Describe

Reconstruction

Description

Decoder

Schema

Decoder

BiM or Textual

Parsing

BiM or Textual

Decoding

MPEG-7 Architecture

24Efficient multimedia query-by-content from mobile devices

Block diagram showing FRUs and FUUs to retrieve fragments of XML.

25Efficient multimedia query-by-content from mobile devices

26Efficient multimedia query-by-content from mobile devices

27Efficient multimedia query-by-content from mobile devices

28Efficient multimedia query-by-content from mobile devices

Results of Table 4 show the total files size of the server replies is reduced to approximately 9.1% when compared to the results of Table 3 without BiM compression.

Efficient multimedia query-by-content from mobile devices 29

Audio Query Scenario

1. The user sends a sample of a song to the server.

2. The server replies that it has 100 possible matches for the song since the

example provided by the user was not of sufficient length or contains

noise.

3. The client asks the server what is the genres ( types ) contained in the

result set using an FRU query.

4. The server replies using FUU: rock, country and classical.

5. The user sends an additional query term using FRU: the genre is country.

6. The server finds exactly one match for the given criteria.

30Efficient multimedia query-by-content from mobile devices

The client sends an audio example.

The server replies that there are 100 possible matches found.

31Efficient multimedia query-by-content from mobile devices

The result set that is kept in the server.

The FRU request to retrieve the genres in the result set.

The FUU replies from the FRU

The additional term to be added to the initial query.

32Efficient multimedia query-by-content from mobile devices

33Efficient multimedia query-by-content from mobile devices

34Efficient multimedia query-by-content from mobile devices

Efficient multimedia query-by-content from mobile devices 35

Thank you for your kind attention