1

Overview of Hyper-Proxy Project:High-Quality Streaming Media Delivery(an academia-industry collaboration)

Xiaodong Zhang

(Major Team Members: Songqing Chen, Bo Shen, Susie Wee )

Founded by NSF and HP Labs

2

Proxy Caching

Server Intermediary Client

Reduce response

time to client

Reduce network traffic

Reduce server’s load

3

Existing Internet Status

Servers Intermediaries Clients

Servers Intermediaries Clients

Media Objects

Very large sizes

Very rigorous real-time delivery

constrains: small startup

latency, continuous

delivery

A large numberof proxies with:disk, memory,and CPU cycles

Diverse clientaccess devices:

computers,PDAs,

cell-phones

A large numberof proxies with:disk, memory,and CPU cycles

4

A Missing Component in Proxy Caching

• Current streaming media delivery status (WWW’05)– Client-Server Model (poor scalability, heavy traffic)

Downloading (about 90%).Many media servers limits downloading to protect

copyrights.

– CDNs (very expensive, idle resources)Dedicated services.

• Why not leverage existing proxies in Internet?

5

Typical Engineering Solutions

• Examining existing proxy structure: Squid is an open source software.

• Redesign Squid: add the streaming function.

• Integrating segmentation methods: restructure the proxy to make it segmentation enable.

• Testing and evaluating the streaming proxy.

6

Three Aimed Impacts

• Academic Impact: influencing the research community via publications, and producing strong Ph.Ds.

• HP Product Impact: the research prototype is identified by product divisions for technology transfer.

• Industry Impact: a successful technology transfer or a product becomes demanding in industry or in the market.

7

Aimed Impacts from NSF• Intellectual merits and challenges:

– Knowledge Advancement and Discovery. – Originality and creativeness of the research. – Problem solving ability and qualifications.– Activity organization and resource availability.

• Broader impact: – Wider applications and deeper impact to society.– Education impact and diverse workforce training (e.g.,

gender, ethnicity, disability, geographic, etc. )– Dissemination of the tools and software in public.– Enhancement of infrastructure for research and education

8

Current Solutions for Proxy Streaming

• Dealing with large size– Object segmentation – partial caching

• Along the quality domain• Along the viewing time domain

• Reducing startup latency, increasing Byte Hit Ratio– Priority caching – caching beginnings popular objects.

• Guaranteeing continuous streaming– Little research done

9

The State-of-the-art of Segmentation & Priority Caching

• Prefix Caching (INFOCOM’99-’02)

• Uniform Segmentation (JSAC’02)

• Exponential Segmentation (WWW’01)

• Adaptive-Lazy Segmentation (NOSSDAV’03)

prefix suffix

…………………….

……………………..

Priority: low startup latency, bursty smoothing

Priority: load balance, low startup latency

Priority: low startup latency, quick replacement

Priority: high byte hit ratio

Existing strategies emphasize on byte hit ratio or startup latency.

ConflictingAutomatically provide

continuous streaming?

10

A Case of Conflicting Performance Interests

Evaluated on a typical workload: WEB

11

Problems in Streaming Proxy Design

• Independently pursuing highest byte hit ratio or minimizing startup latency– But they conflict with each other!

• Lack of serious considerations to guarantee the continuous streaming – But this is the most important QoS concern to viewers!– Providing continuous streaming also conflicts with

improving byte hit ratio!

12

Our Design Model

• An efficient design model should guarantee continuous delivery subject to low startup latency and high byte hit ratio.

• Challenges– How to reconcile guaranteeing continuous streaming

delivery with improving byte hit ratio?

– How to balance reducing startup latency with improving byte hit ratio?

13

Algorithms Design in Hyper Proxy(intellectual challenges)

• Active Prefetching Technique – Prefetching uncached segments to ensure continuous delivery.

• Lazy Segmentation Method– Segmenting objects adapting the access patterns.

• Priority-based Admission Policy– Continuous streaming > small startup latency > byte hit ratio

• Differentiated Replacement Policy– Keeping appropriate (not necessarily the most popular) segments

in the proxy

14

Our Contributions

• Addressed two pairs of conflicting interests

• Designed Hyper-Proxy following our efficient design model

• Implemented Hyper-Proxy and deployed it at Hewlett-Packard Company

• It is the first deployed segment-based streaming proxy

15

Outline

• Addressing two pairs of conflicting interests– Provide Continuous Streaming via Active Prefetching– Minimize Startup Latency via Performance Trade-off

• Hyper-Proxy Design

• Conclusion

16

Proxy Jitter: Minimization Objective

Client StreamingProxy

Internet

StreamingServer

? ? ?

YN !

Proxy Jitter: the delay of fetching uncached segments

Bs: encoding rate Bt: average bandwidth

Prefetching is done in a unicast channel

17

Window-based Prefetching • Principle: to fetch the next segment after a

client starts to access the current segmentStreaming speed = encoding rate, Bs

Prefetching speed = average network bandwidth, Bt

Streaming…

L

Prefetching

Bs ≤ 2BtL

18

Active Prefetching• Principle:prefetching as early as when a client starts to

access the object.

• Additional notation – Ns: the number of cached segments of an object

• Uniformly Segmented Object

– Ns < Bs/Bt – 1 (converted to number of segments) Some segments can not be prefetched in time

– Ns ≥ Bs/Bt – 1 All segments can be prefetched in time

Proxy Jitter!

19

Active Prefetching?

• Is it possible to guarantee an always-in-time prefetching?

• Yes! To increase Ns!– i.e., to cache more segments of those objects who

have difficulty to afford in-time prefetching

• How many segments of an object should be cached so that active prefetching always works?

20

Active Prefetching – A Step Further

• Minimum cached length of an object: – To ensure the prefetching of its uncached segments

to be in time. (Lobj: length of segment, L1: Length of 1st cached segment).

1

)1(

L

LBB

objs

t

1))1(

(log1

2

L

LBB

objs

t

Uniformly Segmented Object Exponentially Segmented Object

21

Active Prefetching – In Practice

High

Medium

Low

Cashed length by popularity

Medium

Byte Hit Ratio slightly decreases; Proxy Jitter is totally eliminated!

Same required minimum length for continuous streaming. But

Insufficient for popular files.

High

LowLow

Not all popular segments need to be cached, but appropriate ones!

Conflicting

Low popularity document needs additional caching space,

which will be provided by others.

22

Outline

• Addressing two pairs of conflicting interests– Guarantee Continuous Streaming via Active Prefetching

- Tradeoffs between Startup Latency and byte hit ratios

• Hyper-Proxy Design

• Conclusion and Future work

23

Modeling Assumptions

• Assumption 1: there is no premature termination.

• Assumption 2: objects are sequentially accessed.

• Assumption 3: Zipf-like distribution of object popularities (USITS’01, NOSSDAV’02)

• Assumption 4: Poisson distribution of request arrival intervals (USITS’01, NOSSDAV’02)

24

Modeling Results

• We get the relative changing rate of byte hit ratio to delayed start request ratio mathematically.

• We found byte hit ratio decreases much slower than delayed startup request ratio.

• We can use a small reduction of byte hit ratio to trade for a large decrease of delayed startup request ratio.

25

Outline

• Addressing two pairs of conflicting interests– Guarantee Continuous Streaming via Active Prefetching– Minimize Startup Latency via Performance Trade-off

• Hyper-Proxy Design

• Conclusion and Future work

27

Admission: fully admit the initially accessed objects.Replacement: find victim.Lazy Segmentation: triggered by a replacement.Replacement: kept partial segments, shift to next list.More replacements: adding partial segment sets. Prefetching: for shortage of segments, list shift.Admission: based on prefetching calculation.

Basic Hyper-Proxy Operations (Lazy Segmentation based)

Size: Full Size of a Media Object

CDL: cached data length

CDL = Size

its Prefetching Length<= CDL < Size

CDL < its Prefetching Length

32

Workload Summary

Workload #req #obj Size λ LengthEncoding

(Bs)Transfer

(Bt)Duration

WEB 15188 400 51 4 0.47 2-120 28~256 [0.5, 2] 1

PART 15188 400 51 4 0.47 2-120 28~256 [0.5, 2] 1

REAL 9000 403 20 - - - - - 10

λ : of Poisson, in second

: of Zipf-like, the skew factor

Length : of objects, in minute

Duration : of simulation time, in days

Size: of total objects, in GB

Encoding, Transfer: in Kbps

33

Hyper-Proxy Performance(comparing with Lazy-Segmentation, and Lazy-

Segmentation with prefetching)

34

Hyper-Proxy Performance (comparing with Lazy-Segmentation, and Lazy-

Segmentation with prefetching)

35

Hyper-Proxy Performance (comparing with Lazy-Segmentation, and Lazy-

Segmentation with prefetching)

39

Hyper-Proxy – Protocol

Servers Intermediaries Clients

Hyper-Proxy

HTTP; No Changes RTP/RTSP

Existing Web servers can provide “real” streaming service!

40

Hyper-Proxy – Architecture

Streaming Engine

Local Manager&Scheduler

SegmentationEnabled

Cache Engine

Disk

Client

Internet

Hyper-Proxy

RT

P/R

TS

PH

TT

P

ghost.mp4, please

1st segment, pleaseMeta data

Media data

Meta data

SynchronizationMedia data

Next segment, please

Perform streaming

Interfacing two engines

Object segmentation,Segment admission, replacement

Fast data path

42

Hyper-Proxy Evaluationin Global Intent Environment

• US West coast -- HP Labs, Palo Alto, CA– Server (apache 2.0.45): Pentium III 1 GHz, SuSE Linux 8.0– Hyper-Proxy: HP x4000 workstation 2 GHz– Many Clients: Pentium III 1GHz, SuSE Linux 8.0

• Asia – NTT, Takaido, Japan– Server (apache 2.0.45): Pentium III 1 GHz, SuSE Linux 8.0– Hyper-Proxy: HP x4000 workstation 2 GHz– Many Clients: Pentium III 1 GHz, SuSE Linux 8.0

• US East Cost – William and Mary, Williamsburg, VA– Hyper-Proxy: HP x4000 workstation 2 GHz– Many clients: diverse types.

48

Conclusions

• Hyper-Proxy design and implementation– Provide an efficient design model

– Enable streaming delivery service on Web servers

– Cause little playback jitter

– Produce a small startup latency

49

Current Status and Impacts

• Impact of Hyper-Proxy at HP (product impact)– Deployed at Printing Division for world-wide training– Deployed at Telecommunication Division– Used by Imaging and Signal Processing Team in HP

Labs

• Potential Industry Impact– Enterprise solution after the trial stage– Education solution for remote education (currently

several universities are using the prototypes).– Two patents pending.

50

Project 1: Mobility

Servers Intermediaries Clients

Device Hand-Off

Cooperation

51

Project 2: P2P Streaming

Servers Intermediaries Clients

Sharing contents and streaming

resources

52

Project 3: Diversity of Devices

Servers Intermediaries Clients

Different screen sizes, color

depths, et. al. transcoding

53

Project 4: Live Streaming

Servers Intermediaries Clients

The OscarAnnual Academy

Awards

cannot cache

54

Acknowledgement

• NSF and HP Labs for research grants

• Collaborators: Bo Shen, Susie Wee, Sumit Roy, Yong Yan, Sujoy Basu, Dan (Wai-tian) Tan, Ankcorn John, Mitch Trott, Zhichen Xu ; Zhen Xiao

• Members and alumni of HPCS Lab@WM

55

Related Publications (academic impact)1. S. Chen, B. Shen, Y. Yan, S. Basu, and X. Zhang. SRB: Shared Running

Buffers in Proxy to Exploit Memory Locality of Multiple Streaming Sessions. IEEE ICDCS 2004.

2. S. Chen, B. Shen, S. Wee, and X. Zhang. Designs of High Quality Streaming Proxy Systems. IEEE INFOCOM 2004.

3. S. Chen, B. Shen, S. Wee, and X. Zhang. Investigating Performance Insights of Segment-based Proxy Caching of Streaming Media Strategies. ACM/SPIE MMCN 2004.

4. S. Chen, B. Shen, S. Wee, and X. Zhang. Streaming Flow Analyses for Prefetching in Segment-based Proxy Caching Strategies to Improve Media Delivery Quality. WCW 2003.

5. S. Chen, B. Shen, S. Wee, and X. Zhang. Adaptive and Lazy Segmentation Based Proxy Caching for Streaming Media Delivery. ACM NOSSDVA 2003.

Songqing Chen received his Ph.D., has been placed in a research academic institution (George Mason U.)