Distributed Multimedia Synchronization Based on Fuzzy Causal Relations

Luis Alberto Morales Rosales

Congreso Internacional de Informática Aplicada

Misantla, Veracruz, a 25 de Abril del 2012

2

Overview

• Practical Aspect (application area)– Distributed multimedia synchronization in real time

• Distributed sources• Heterogeneous data (discreet and continuous)• Loss of data• Transmission delay

• Theoretical Aspect– Development of a flexible causal relation for distributed systems

unlike the one proposed by Lamport (1978)• For applications where certain degradation of the system is allowed

is not necessary to assure a strict causal delivery, for example, multimedia, scheduling, cooperative work and planning

INAOE 3

Outline

1. Introduction

2. Related work

3. Problem description

4. Research proposal

5. Results

6. Conclusions

4

Introduction

• Distributed systems– Absence of a global clock– Concurrency– Fail tolerance – Examples: cooperative applications, mobile systems,

multimedia applications, etc.

• Distributed multimedia systems– The exchange of big volumes of multimedia data in a

communication network among a group of participants

5

Introduction

• Multimedia synchronization– The preservation of temporal dependencies

among the application data from the time of generation to the time of presentation

Multimedia

Text, slides, images, etc.

VideoAudio

Multimedia data

6

Introduction

A

A

A

P1 P2

P3

V

V

A

VV

VV

SS

A- Audio (Voice)V- VideoS- Slides

P1 – Participant 1P2 - Participant 2P3 - Participant 3

Example of a multimedia scenario

7

Introduction

Representation of a multimedia scenario

A

V

A

V

V

S

P1

P3

P2

A- Audio (Voice)V- VideoS- Slides

P1 – Participant 1

P2 - Participant 2

P3 - Participant 3

Time Line

Related Work

Synchronization

Synchronous Asynchronous

Common referencePhysical clock

Disadvantages BottlenecksIntroduction of random delays Not scalable

Logical dependencies

DisadvantagesIntroduction of random delays Discard of messagesHalt of the system

8

9

Related work: Synchronous

• Works that use fuzzy logic– Zhou et al. [6]

• Temporal model based on fuzzy petri networks to represent the multimedia synchronization

• Video on demand

– Ramaprabhu et al. [3]• Broadcast transmission of video on demand

• Consider the parameters: available bandwidth, network delay and buffer space

10

Related work: Synchronous (Cont.)

• Works that use fuzzy logic– Coelho et al. [1]

• Methodology for the high level specification and decentralized coordination of temporal interdependences among objects of multimedia documents previously stored

• Membership function to calculate the lifetime of the data

• Strict causal relation for event ordering

• Global time reference

11

Related work: AsynchronousCausal and -causal algorithms

• The causal and -causal algorithm use the causal relation proposed by Lamport– Main works: Baldoni [2] et al., Tomoya et al.[5] and Pomares et al. [4]

– Introduce delays and/or discard of messages (packets)

bcStrict delivery order -causal order

bc and c < max

( b, c)Infinite halt

( b, c )Deliver c and discard b

a b c

Time Line

12

Context problem

• Distributed multimedia synchronization in real time by considering:

• Asynchronous sources• Heterogeneous data• Communication network characteristics

– Loss of the data and transmission delay

• Without previous knowledge of the system behavior

How can we ensure the temporal dependencies among events in this kind of enviroments?

13

Research proposal

• As hypothesis we claim that for systems where some degradation of the system is allowed, it is not necessary ensuring a strict causal order of the data, which is associated with a binary value

• We propose the fuzzy causal relation and the fuzzy causal consistency to relax the order delivery

• We show that relaxing the causal delivery order of the data can improve the performance of the synchronization

14

• Fuzzy causal relation (FCR) denoted by “a b”– The FCR is based on a notion of “distance” among the events

– The distance can be established considering three main domains: spatial (RS), temporal (RT) and/or logical(RL)

– Using the notion of distance, the FCR establishes a cause-effect degree that indicates “how long ago” an event a happened before an event b

– One assumption considered for the FCR is that “closer” events have a stronger cause-effect relation, according to the addressed problem

ResultsFuzzy causal relation (FCR)

• The distance between events is determined by the fuzzy relation DR: E E [0, 1]

DR(a,b) = RS RT RL

Where:RS=(R1 R2... Rs)RT=(R1 R2... Rt)RL=(R1 R2... Rl)

Ri = membership function

• The fuzzy union operator chosen for intra and inter domains is the max operator

15

Results (Cont.)Fuzzy causal relation (FCR)

• Distance relation– The DR grows monotonically and it is directly proportional to the

spatial, temporal and/or logical distances between a pair of events

– DR(a,b) with a value tending to zero indicates that the events a and b are “closer”

– The DR cannot determine precedence dependencies among events, it only indicates certain distance among them

16

Results (Cont.)Fuzzy causal relation (FCR)

17

Results (Cont.)Fuzzy causal relation (FCR)

• The fuzzy causal relation “ ” on a set of events E satisfies the two following conditions:

1. a b If a b 0 DR(a,b) < 1

FCR between two events

• a c If b abc DR(a,b) DR(a,c) : DR(a,b), DR(a,c) < 1

Transitivity

•The fuzzy concurrency (FCNR) is defined as follows:

a b If (ab ba) ( (DR(a, b)= DR(b, a) )< 1)

The value of the fuzzy causal relation between a pair of events is represented as: FCR(a,b)

18

Results(Cont.)Example of the FCR and the FCNR

a e If b abe DR(a,b) DR(a,e) : DR(a,b), DR(a,e) ) < 1

FCR(a,e)

a

b

c

d

e

f1

2

Example of fuzzy precedence among causal messages FCR(a,e)

19

Results(Cont.)Fuzzy Causal Consistency

• The Fuzzy Causal Consistency (FCC) is based on the FCR

• The goal of the FCC is to indicate “how good” the performance of the system is in a certain time

• The meaning of the performance can be indicated according to the problem to solve

20

Results(Cont.)Fuzzy Causal Consistency

cd

w

FCRp (b,a)b

H(a)Fuzzy causal consistency

FCCp(a)

FCRp(c,a)FCRp(d,a)

FCRp( w,a)

)(

)(

)(

),()(

)(

aHb

aHbp

p bGP

abFCRbGP

aFCC

Calculation of the Fuzzy causal consistency

a

b

c

d Average Weight

FCRp( b,d)

FCRp( c,d) FCCp(a)

Example of calculation of the FCC

GP(b) is a weighting degree used to determine priorities or weight for every fuzzy causal relation when it is needed

Results (Cont.)Fuzzy causal delivery order

• The fuzzy causal delivery order must satisfy the following condition:

If send(m) send(m’) then

  p dests(m)dests(m’), FCCp(m’)≤ FCCmax

–deliveryp(m) deliveryp(m’) or

–deliveryp(m’) deliveryp(m)

21

22

Results Causal order vs fuzzy causal order

a+d -

Strict order delivery -causal order deliveryFuzzy causal order

delivery

a+d - y d < max a+ d-

( a+, d - )Infinite halt

( a+, d - )Deliver d and discard a

Deliver: (a+, d - ) or (d -, a+)

Synchronization Periodk =Part(Z)

Example of a fuzzy causal delivery

a-a+

a+ d- c-

a+

a- = x1 d - = y1 D

c+ = xn

b+ =yp

A C

B

Causal delivery order of events

i =Part(X)

d - Fuzzy causal deliverya+

j =Part(Y)

23

Temporal Distance

RN

Causal DistanceRD

FCCp(a)

System

Component of input variables

Current State ofthe System

Adjustment andSelective Discard

Weighting DegreeGP

Fuzzy Causal RelationsFuzzy CausalConsistency

Fuzzy Causal Component

Fuzzy Control System

Performance ofthe System

Scheme of the Distributed Multimedia Mechanism

FCRp(m) m H(a)

Fuzzy Control

SynchronizationModel

Network Conditions

NC

24

Synchronization model

i ≠ j

i =Part(X)

j =Part(Y)

Synchronization periods

F

A

BF’

E’E C

D

Fuzzy causal delivery

Formally a period is defined as: (e, f) (E x F), e f e f

BFEDCEFA IIII '')|||(

EF

25

Synchronization model

26

Results (Cont.)The FCR applied to the intermedia synchronization

• The FCR for the intermedia synchronization problem gives a qualitative measure of the temporal and logical dependencies between two events with regard to a partial view of a participant

• The value of the FCR is calculated by:

),max(),(

if),(),(

ND RRbaDR

babaDRbaFCR

Results (Cont.)The FCR applied to the intermedia synchronization

• The meaning of the values obtained by the FCR(a, b) is as follows:– When FCR(a, b) tends to zero indicates that the events

a and b are “closer”, which means that the network present low transmission delay and low loss of messages

– When FCR(a, b) tends to the unit indicates the opposite

27

28

Results (Cont.)FCC applied to the intermedia synchronization

• FCC gives a qualitative measure of the synchronization error, according to the temporal and logical dependencies in the whole system in a certain time, with regard to the partial view that every participant has

– When FCCp(a) tends to zero, this indicates that the performance of the system is good

– When FCCp (a) tends to the unit, this indicates that the system performance is regular or bad

• The actions that the control will be able to execute are: – the immediate deliver of the received message a or its delay, and – the determination whether a selective discard of the messages contained

in the causal history of the message a is carried out• The Mandami’s model is used as inference mechanism• Triangular membership functions are included as part of the fuzzy

control

Fuzzy Control System

= delivery timeCurrent State of the System

(CS)

Fuzzy Causal Consistency, FCCp(a)

Network ConditionsNC(a)

Diagram of the fuzzy control system

Results (Cont.)Fuzzy control system

29

• The NC determines the network condition– The NC consider: transsmision delays, jitter among messages, loss of

messages, network congestion and bandwith available– When the value of NC(a) is near to zero, this means that the network

conditions are good– When NC(a) tends to one, this represents that the network conditions

are bad

Results (Cont.)Test of a multimedia scenario

• The Host from W to Y send:– Audio– Video, and– Animation

• Host Z only listens• We simulated three

scenarios:• Soft, Medium and Hard

• 100,000 simulations• Fuzzy causal order vs. -

causal order

Host W

Host X

Host Z

Host Y

30

•When Lamport’s relation is used the system halts if a message is lost

35

Results (Cont.)Test: Fuzzy causal order vs. -causal

Hard case

-Causal Order

Fuzzy causal order

Classification of messages loss

Conclusions

• The definitions of FCR and FCC permits to establish a more asynchronous ordering for application where certain degradation of the system is allowed

• The FCO allows a more asynchronous delivery of events compared with the causal delivery order based on the happened-before relation introduced by Lamport

• The FCR and FCC allow a measure of the performance of the application by a participant at runtime without halt the system

36

Conclusions

• A novel fuzzy control for intermedia synchronization that works in a distributed manner was presented

• By using the fuzzy control and FCC the mechanism discard fewer messages than the -causal mechanism

38

References

1. André L. V. Coelho, Alberto B. Raposo, Ivan L. M. Ricarte. “Bringing Flexibility to the Specification and Coordination of Temporal Dependencias among Multimedia Components”. VII Simposio Brasileiro de Sistemas Multimídia e Hipermídia, Florianópolis, Brazil, SBC. 2001, pp. 37-52.

2. Roberto Baldoni, Achour Mostéfaoui, Michel Raynal. “Efficient Causally Ordered Communications for Multimedia Real-Time Applications”. The 4th International Symposium on High Performance Distributed Computing (HPDC '95), Washington, DC, USA, August 2-4, 1995, pp. 140-147

3. Ramaprabhu Janakiraman, Marcel Waldvogel and Lihao Xu. “Fuzzycast: Efficient Video-on-demand over Multicast”. Proceedings INFOCOM 2002, New York, NY, USA, June 2002.

4. Saul E. Pomares Hernandez, Luis A. Morales Rosales, Jorge Estudillo Ramirez, and Gustavo Rodriguez Gomez, “Logical Mapping: An Intermedia Synchronization Model for Multimedia Distributed Systems,” Journal of Multimedia, Eds. Academy Publisher, Vol. 3 No.5, 2008, ISSN: 1796-2048, pp. 33-41.

5. Tomoya Enokido, Sei-ichi Hatori, Takuya Tojo Makoto Takizawa. “Group Communication in Distributed Multimedia Objects”. Proceeding of The Eighth IEEE International Workshop on Object-Oriented Real-Time Dependable Systems (WORDS 2003), 2003, pp. 258.

6. Yi Zhou, Tadeo Murata. “Modeling and Analysis of Distributed Multimedia Synchronization by Extended Fuzzy-Timing Petri Nets”. Transactions of the Society for Desing and Process Science, Volume 5, Number 4, December 2001, pp. 23-37.

39

Thanks !

Questions?