Post on 17-Dec-2015
CS 414 - Spring 2011
CS 414 – Multimedia Systems Design Lecture 27 – Media Server (Part 3)
Klara Nahrstedt
Spring 2011
Administrative MP3 – posted today
CS 414 - Spring 2011
Some Interesting Facts
DBMS2.com Source (May 2009)Facebook had 400 terabytes of disks managed by
Hadoop/Hive with an approx. 6:1 compression ratioFacebook’s Hadoop/Hive system ingests 15
terabytes of new data per dayFacebook had 610 Hadoop nodes (in May 2009)
running in a single cluster and was heading for 1000 Yahoo had 2000 nodes (in May 2009) and was heading for
4000
CS 414 - Spring 2011
Some Interesting Facts Source: www.slideshare.net (March 2011) Current data sets:
NYSE: 8PB; Google > 12PB; Data Volumes:
NYSE: 1.5 TB daily; Facebook: 350 M users; 3.5B shared items/weekFacebook adds > 100K users, 55M ‘status’
updates, 80M photos daily
CS 414 - Spring 2011
Outline Disk Scheduling
SCAN-EDF Group Sweeping Mixed Scheduling
Admission Control File System Metadata/Indexing Block Size Issues
CS 414 - Spring 2011
Disk Scheduling Policies Goal of Scheduling in Traditional Disk
Management Reduce cost of seek time Achieve high throughput Provide fair disk access
Goal of Scheduling in Multimedia Disk Management Meet deadline of all time-critical tasks Keep necessary buffer requirements low Serve many streams concurrently Find balance between time constraints and efficiency
CS 414 - Spring 2011
EDF (Earliest Deadline First) Disk Scheduling Each disk block request is tagged with
deadline Policy:
Schedule disk block request with earliest deadline
Excessive seek time – high overheadPure EDF must be adapted or combined with
file system strategies
CS 414 - Spring 2011
EDF Example
CS 414 - Spring 2011
Note: Consider that block number Implicitly encapsulates the disk track number
SCAN-EDF Scheduling Algorithm Combination of SCAN and EDF algorithms Each disk block request tagged with
augmented deadline Add to each deadline perturbation
Policy: SCAN-EDF chooses the earliest deadline If requests with same deadline, then choose
request according to scan direction
CS 414 - Spring 2011
Implementation of SCAN-EDF Notation:
Di be deadline of disk block request ‘i’
Ni be track (block) position on disk
Nmax be maximum number of disk tracks
Deadline Modification: Di + f(Ni)
f(Ni) converts track number of ‘i’ into a small perturbation of deadline
Perturbation small enough so that Di + f(Ni) ≤ Dj + f(Nj) for Di ≤ Dj
Possible f(Ni) = Ni/Nmax
CS 414 - Spring 2011
SCAN EDF Example (Nmax = 100)
CS 414 - Spring 2011
Enhanced SCAN-EDF (1) Use more accurate perturbation of deadline Consider
Actual track position of disk head ‘N’ Nmax – max number of disk tracks
Ni – next track to be considered
CS 414 - Spring 2011
Head Moves Upwards
Enhanced SCAN-EDF (2) Algorithm:
If head moves upwards (towards Nmax), then
(a)
(b)
CS 414 - Spring 2011
maxmax )(,;
N
NNNfNNNN iiii
max
max)(,1;N
NNNfNNN iiii
Enhanced SCAN-EDF (3) If head moves downwards (towards 1), then
(a)
(b)
CS 414 - Spring 2011
maxmax )(:;
N
NNfNNNN iiii
max
)(:1;N
NNNfNNN iiii
Group Sweeping Algorithms
Policy: Each Request consists of (Deadline, Block Number
)Disk Block Requests served in cycles In one cycle, requests divided into groups
according to similar deadlinesWithin group use SCANAs we retrieve blocks, we may need smoothing
buffers to ensure continuity
CS 414 - Spring 2011
Group Sweeping Example
CS 414 - Spring 2011
Mixed Scheduling (uses SSTF – Shortest Seek Time First)
CS 414 - Spring 2011
Example of SSTF
Mixed Scheduling
CS 414 - Spring 2011
SSTF (Shortest Seek Time First) + Balanced Strategy
Admission Control
CS 414 - Spring 2011
Client 1 retrieves K1 blocks in oneround
Client 2 retrievesK2 blocks
Client 3 retrievesK3 blocks
Client 4 retrieves K4 blocks
Server
Admission Control Disk block requests are timed
Media server must determine admit a stream serve (schedule) a stream without having negative effect on
other streams already serviced.
Deterministic Guarantees Admission control considers worst case scenario when admitting
new stream Constrained Disk Placement Example: M - size of blocks, G –
size of gabs, rdt – data transfer of disk
CS 414 - Spring 2011
)/(sec
)(sec)(sec
storsr
torsGtorsMT
dtplay
Admission Control
CS 414 - Spring 2011
)/)((min)( ipl
iii
i
RK
α – overhead switching from one round (‘j-1’)to another round (j), and then transmitting the first block of the ‘j’ round
β – transmission time of (Ki-1) blocks in ‘j’ round, i=1,..4Ki – number of blocks retrieved by client ‘i’ ηi – Block granularity retrieved for client ‘i’ (e.g., in Bytes)Ri – playback rates of client ‘i’ (e.g., in Bytes per second)
Minimal Intra- Ki blocks delay Cost to switch and move Ki blocks
Admission Control
Statistical GuaranteesDeadlines are guaranteed with certain
probabilityAdmission control considers statistical
behavior of the disk system while admitting new stream (average performance)
Best effort ServiceNo guarantees
CS 414 - Spring 2011
Conclusion
The data placement, scheduling, are very important for any media server design and implementation.
Still need to consider multimedia file system and caching – next lecture
CS 414 - Spring 2011