Multicast Data Dissemination
description
Transcript of Multicast Data Dissemination
![Page 1: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/1.jpg)
Multicast Data DisseminationMulticast Data Dissemination
Wang LamSpecial University Oral Examination
7 July 2004
![Page 2: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/2.jpg)
2
ContentsContents
Current multicast networksContributions– Data scheduling– Network issues
Related and future workConclusion
![Page 3: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/3.jpg)
3
Current multicast networksCurrent multicast networks
Traditional data service: one-to-one
Multicast networks: one-to-many
IP: multicast group addresses (IPv4: 224.0.0.0 - 239.255.255.255; IPv6: FF00::/8)
Network bottlenecks
Client joins Unreliable delivery Datagrams (UDP)
![Page 4: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/4.jpg)
4
Multicast data disseminationMulticast data dissemination
Client joins
Unreliable delivery
Datagrams
Supports varying bandwidth clients
All requested data must arrive
Data arranged to optimize performance
![Page 5: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/5.jpg)
5
Principal contributionsPrincipal contributions
Data scheduling–Minimize delay for clients requesting many
items– Scheduling for subscribers and Scheduling for subscribers and
downloadersdownloaders 11Networking issues– Reliable deliveryReliable delivery
2– Splitting bandwidth into channels
![Page 6: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/6.jpg)
6
Principal contributionsPrincipal contributions
Data scheduling
– Scheduling for subscribers and Scheduling for subscribers and downloadersdownloaders
![Page 7: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/7.jpg)
7
Subscribers and downloadersSubscribers and downloaders
Data scheduling– Scheduling for subscribers and
downloaders• Distributing data for a Web repository• Metrics and techniques• Sample results
![Page 8: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/8.jpg)
8
The multicast sourceThe multicast source
Stanford WebBase
100+ million Web pages
Additional benefits of multicast
crawler
repository
multicastserver
indexingand
analysis
clients
WWW
![Page 9: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/9.jpg)
9
Multicast server
A multicast facilityA multicast facility
Clients issue requests to server
Clients listen to shared multicast
Server schedules data onto multicast
Downloaders and subscribers
clients
![Page 10: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/10.jpg)
10
Clients request multiple itemsClients request multiple items
Broadcast disks: one-item “response time”
Multicast: client delay is different
Subscribers: freshness and age
w x y z y w x z A • • A • • B • • B • • C • • C • • D • D • E • E •
F • F •
![Page 11: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/11.jpg)
11
Example scheduler: CircExample scheduler: Circ
Arbitrarily order data items
Send requested data
w x y z G • • • H • • I • J •
![Page 12: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/12.jpg)
12
Example scheduler: PopExample scheduler: Pop
Send most requested data
w x y z G • • • H • • I • J •
![Page 13: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/13.jpg)
13
Example scheduler: R/QExample scheduler: R/Q
Number of requesting clients
Smallest request size
w x y z G • • • H • • I • J •
clients 1 3 2 1 minreq 3 1 2 1
![Page 14: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/14.jpg)
14
Example scheduler: R/QExample scheduler: R/Q
Number of requesting clients
Smallest request size
w x y z G • • H • I J •
clients 1 0 2 1 minreq 2 0 1 1
![Page 15: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/15.jpg)
15
Some results for subscribersSome results for subscribers
Choice of scheduler depends on performance metric
Update frequency has little effect
![Page 16: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/16.jpg)
16
Downloaders and subscribersDownloaders and subscribers
Average download client delay
0
100
200
300
400
500
600
700
800
25 50 75 100 125 150 175 200
Number of downloaders
hou
rs
Circ
R/Q
RxC
Pop
![Page 17: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/17.jpg)
17
Downloaders and subscribersDownloaders and subscribers
Average freshness over clients
0.830.840.850.860.870.880.89
0.90.910.920.930.94
25 50 75 100 125 150 175 200
Number of downloaders
Circ
R/Q
RxC
Pop
![Page 18: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/18.jpg)
18
Downloaders and subscribersDownloaders and subscribers
Average download client delay
0
100
200
300
400
500
600
700
800
25 50 75 100 125 150 175 200
Number of downloaders
hour
s
Circ
R/Q
RxC
Pop
Average freshness over clients
0.830.840.850.860.870.880.89
0.90.910.920.930.94
25 50 75 100 125 150 175 200
Number of downloaders
![Page 19: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/19.jpg)
19
SummarySummary
Differences from broadcast disksDownloaders and subscribersStudied design tradeoffs for various
metrics and techniques
![Page 20: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/20.jpg)
20
Principal contributionsPrincipal contributions
Data scheduling–Minimize delay for clients requesting many
items– Scheduling for subscribers and Scheduling for subscribers and
downloadersdownloadersNetworking issues– Reliable deliveryReliable delivery
2– Splitting bandwidth into channels
![Page 21: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/21.jpg)
21
Principal contributionsPrincipal contributions
Networking issues– Reliable deliveryReliable delivery
![Page 22: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/22.jpg)
22
Principal contributionsPrincipal contributions
Networking issues– Reliable deliveryReliable delivery• Multicast server model• Reliability techniques• Sample results• Other challenges
![Page 23: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/23.jpg)
23
The multicast sourceThe multicast source
Stanford WebBase
100+ million Web pages
Network loss <5% to >20%
crawler
repository
multicastserver
indexingand
analysis
clients
WWW
![Page 24: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/24.jpg)
24
multicast server
A multicast facilityA multicast facility
Clients issue requests to server
Clients listen to shared multicast
Server schedules data onto multicast
Data channel unreliable
clients
![Page 25: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/25.jpg)
25
Forward Error CorrectionForward Error Correction
Compute fixed fraction of redundant data
Reconstruct from subset of bits
Vary padding by item
data FEC data FEC
requests requests
![Page 26: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/26.jpg)
26
Forward Error CorrectionForward Error Correction
Compute fixed fraction of redundant data
Reconstruct from subset of bits
Vary padding by item
![Page 27: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/27.jpg)
27
Forward Error CorrectionForward Error Correction
Compute fixed fraction of redundant data
Reconstruct from subset of bits
Vary padding by item
![Page 28: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/28.jpg)
28
Forward Error CorrectionForward Error Correction
Compute fixed fraction of redundant data
Reconstruct from subset of bits
Vary padding by item
![Page 29: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/29.jpg)
29
Forward Error CorrectionForward Error Correction
Compute fixed fraction of redundant data
Reconstruct from subset of bits
Vary padding by item
FEC(0.2R)
![Page 30: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/30.jpg)
30
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs requests requests
data data
![Page 31: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/31.jpg)
31
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs
![Page 32: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/32.jpg)
32
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs
NAK
NAK
![Page 33: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/33.jpg)
33
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs
![Page 34: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/34.jpg)
34
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs
![Page 35: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/35.jpg)
35
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs
![Page 36: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/36.jpg)
36
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs
![Page 37: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/37.jpg)
37
RetransmissionRetransmission
Wait for NAK Queue
retransmission of enough bits
Queue only on selected NAKs
NAK
R(1)
NAK
![Page 38: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/38.jpg)
38
ReschedulingRescheduling
Do nothing Rerequest data
item
Combine with prior reliability schemes
requests requests
data data
![Page 39: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/39.jpg)
39
ReschedulingRescheduling
Do nothing Rerequest data
item
Combine with prior reliability schemes
NAK
NAK
![Page 40: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/40.jpg)
40
Clients of Uniform Loss RatesClients of Uniform Loss Rates
Average download client delay
400
500
600
700
800
900
1000
0 2 4 6 8 10client loss (% packets)
dela
y (s
econ
ds)
FEC(0)+R(0)
FEC(0)+R(1)
FEC(0)+R(2)
FEC(0)+R(10)
FEC(0)+R(inf)
![Page 41: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/41.jpg)
41
Clients of Tiered Loss RatesClients of Tiered Loss Rates
Average download client delay
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 0.2 0.4 0.6 0.8 1fraction of clients having high loss
dela
y (s
econ
ds)
FEC(0.03R)+R(inf)
FEC(0.03R)+R(0)
FEC(0.1)+R(inf)
![Page 42: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/42.jpg)
42
Clients of Tiered Loss RatesClients of Tiered Loss Rates
Average download client delay
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
0 0.2 0.4 0.6 0.8 1fraction of clients having high loss
dela
y (s
econ
ds)
FEC(0.01R)+R(inf)FEC(0.03R)+R(inf)FEC(0.05R)+R(inf)FEC(0.01R)+R(0)FEC(0.03R)+R(0)FEC(0.05R)+R(0)FEC(0.1)+R(inf)
![Page 43: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/43.jpg)
43
Additional resultsAdditional results
Error-correcting packets help retransmissions
Variable FEC can outperform matched-rate FEC
Data-in-progress announcement can slightly help new clients
![Page 44: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/44.jpg)
44
SummarySummary
Multicast server scenario allows a variety of reliability techniques
Techniques form many combinationsStudied design tradeoffs
![Page 45: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/45.jpg)
45
Principal contributionsPrincipal contributions
Data scheduling–Minimize delay for clients requesting
many items– Scheduling for subscribers and Scheduling for subscribers and
downloadersdownloadersNetworking issues– Reliable deliveryReliable delivery– Splitting bandwidth into channels
![Page 46: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/46.jpg)
46
PublicationsPublications
W. Lam and H. Garcia-Molina, “Multicasting a Data Repository,” WebDB 2001
W. Lam and H. Garcia-Molina, “Multicasting a Changing Repository,” ICDE 2003
W. Lam and H. Garcia-Molina, “Reliably Networking a Multicast Repository,” SRDS 2003
W. Lam and H. Garcia-Molina, “Slicing Broadcast Disks,” submitted for publication
W. Lam and H. Garcia-Molina, “Implementing Multicast Data Dissemination,” technical report
![Page 47: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/47.jpg)
47
Publications (Stanford WebBase)Publications (Stanford WebBase)
J. Cho, T. Haveliwala, W. Lam, S. Raghavan, A. Paepcke, and H. Garcia-Molina, “Stanford WebBase Components and Applications”
http://www-diglib.stanford.edu/~testbed/doc2/WebBase/
Web crawler and client code:ftp://db.stanford.edu/pub/digital_library/
![Page 48: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/48.jpg)
48
Related workRelated work
Broadcast disksWeb cachingPublish/subscribe systemsVideo on demandReliable multicast protocolsLayered multicast protocols
![Page 49: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/49.jpg)
49
Next stepsNext steps
Other kinds of clients– On-the-fly processing– Partially ordered clients– Opportunistic clients
Distributed serversRequest mining
![Page 50: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/50.jpg)
50
http://www.cs.stanford.edu/~wlam/compsci/http://www.cs.stanford.edu/~wlam/compsci/
![Page 51: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/51.jpg)
51
![Page 52: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/52.jpg)
52
More challengesMore challenges
Different network loss ratesDifferent download speeds
Multicast server
clients
![Page 53: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/53.jpg)
53
More challengesMore challenges
Different download speeds– How many, how fast, how distributed?
Multicast server
clients
![Page 54: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/54.jpg)
54
Related workRelated work
Data schedulingAcharya, Franklin, Aksoy, Zdonik,et al.
Web cachingBestavros, Rodriguez, et al.
Multicast networkingFloyd, Van Jacobson, McCanne, Miller, Almeroth, et al.
![Page 55: Multicast Data Dissemination](https://reader036.fdocuments.us/reader036/viewer/2022062309/56814951550346895db69ff6/html5/thumbnails/55.jpg)
55
Related workRelated work
Multicast dissemination– Acharya, Franklin, Zdonik, Vaidya,
Hameed (broadcast disks)– Almeroth, Ammar, Fei (Web service)
Reliable multicast networking– Floyd, Van Jacobson, McCanne (SRM)–Miller, Robertson, et al. (MFTP)– Yajnik, Kurose, Towsley, et al. (Mbone)