1

Measurement Study of Netflix, Hulu, and a Tale ofThree CDNs

Vijay K Adhikari, Yang Guo, Fang Hao,Member, IEEE,Volker Hilt, Member, IEEE,Zhi-Li Zhang,Fellow, IEEE,Matteo Varvello,Member, IEEE,and Moritz Steiner

Abstract—Netflix and Hulu are leading OTT (Over-The-Top)content service providers in the US and Canada. Netflix aloneaccounts for 29.7% of the peak downstream traffic in US in2011. Understanding the system architectures and performanceof Netflix and Hulu can shed light on the design of such largescale video streaming platforms, and help improving the design offuture systems. In this paper, we perform extensive measurementstudy to uncover their architectures and service strategies. Netflixand Hulu bear many similarities. Both Netflix and Hulu videostreaming platforms rely heavily on the third-party infras truc-tures, with Netflix migrating majority of its functions to Am azoncloud, while Hulu hosting its services out of Akamai. Both serviceproviders employ the same set of three CDNs in delivering thevideo contents. Using active measurement study, we dissectseveralkey aspects of OTT streaming platforms of Netflix and Hulu,e.g., employed streaming protocols, CDN selection strategy, userexperience reporting, etc. We discover that both platformsassignthe CDN to a video request without considering the networkconditions and optimizing the user perceived video quality. Wefurther conduct the performance measurement studies of thethree CDNs employed by Netflix and Hulu. We show thatthe available bandwidth on all three CDNs vary significantlyover the time and over the geographic locations. We proposea measurement-based adaptive CDN selection strategy and amultiple-CDN-based video delivery strategy that can significantlyincrease users’ average available bandwidth.

Index Terms—Netflix, Hulu, video streaming, Content Distribu-tion Networks (CDN), OTT (Over-The-Top) content service, CDNselection strategy.

I. I NTRODUCTION

NETFLIX and Hulu are the leading subscription-basedvideo streaming service providers for movies and TV

shows. By April 2014, Netflix attracts more than 35 mil-lion subscribers in US alone, and about 48 millions worldwide [1]. It is the single largest source of Internet traffic,consuming29.7% of peak downstream traffic in 2011 [2]. LikeNetflix, Hulu also has a large viewer base, with 38 millioncasual viewers who watches Hulu at least once a year, and3 million paying subscribers. Both providers offer video atmultiple quality levels, capable of adapting to user’s availablebandwidth. Designing such large scale, fast growing videostreaming platforms with high availability and scalability is

Vijay K Adhikari is with Microsoft, Redmond, WA.Yang Guo, Fang Hao, Volker Hilt, and Matteo Varvello are withBell Labs,

Alcatel-Lucent.Zhi-Li Zhang is with University of Minnesota - Twin Cities.Moritz Steiner is with Akamai Technologies, San Francisco,CA 94103.

This work was done while Moritz was with Bell-Labs.

technically challenging. Because of their popularity and size,the design and traffic management decisions of these servicesalso have a profound impact on the Internet infrastructure.

In this paper we provide a detailed analysis of the Netflixand Hulu architectures, which are designed to serve massiveamount of content by combining multiple third party services.For instance, Netflix heavily utilizes Amazon cloud service[3],replacing in-house IT by Amazon Web Service (AWS), and us-ing Amazon SimpleDB, S3 and Cassandra for file storage [3].Microsoft Silverlight [4] is employed as the video playbackplatform for Netflix desktop users. Both Netflix and Huluuse Akamai, Limelight, and Level3 CDNs for video contentdelivery. Such third-party service based architecture canberegarded as a system design blueprint by future OTT contentservice providers.

Despite the popularity of Netflix and Hulu, surprisingly fewstudies have been looking into their streaming service plat-forms. In order to understand thearchitectural design issuesof such large-scale video streaming service platforms and theirimplications, we conducted extensive measurement studiesfrom June 2011 to October 2011, with initial results being pub-lished in [5], [6]. This journal version paper integrates/unifiesthe results from [5], [6], and offers a comprehensive treatmentof two most popular content distribution platforms. The issuescommon for both platforms are carefully compared, while theirdifferences are stressed. In particular, we have investigatedthe interactions between different components of such anarchitecture, and analyzed the strategies used by Netflix andHulu that provide the glue to piece together the overall system.We have also looked into the implications of these designdecisions on CDNs, underlying ISP networks as well as enduser quality-of experience (QoE).

To the best of our knowledge, we are the first to take asystematic look into the architecture of these video streamingplatforms together with an extensive measurement study ofthree CDNs they employ. Our results suggest the plausiblerole of business relationsbetween Netflix/Hulu and CDNs inconstraining how a content provider decides which CDN toselect to serve streaming videos to end users, and reveal thediffering CDN selection strategies used by Netflix and Hulu tomeet the business constraints. Furthermore, our measurementresults demonstrate that the CDN selection strategies employedby Netflix and Hulu do notnecessarilyprovide the bestpossibleQoE to end users, thus highlighting a keytrade-offinCDN selection decision making:business constraints vs. enduser QoE.To illustrate how this trade-off can be effectivelyexploited, we propose new video delivery strategies that can

2

significantly improve the user QoE by effectively utilizingmultiple CDNs, while still conforming to the business con-straints. The main contributions of this paper are summarizedas follows:• We dissect the basic architecture of the two popular

video streaming platforms by monitoring the commu-nications between the client-side player and variouscomponents of the two platforms.

• We analyze how Netflix and Hulu make use of multipleCDNs under changing bandwidth conditions. We findthat both Netflix and Hulu players stay with the sameCDN even if the other CDNs may offer better videoquality. In addition, Netflix tends to tie preferred CDNswith user accounts, while Hulu randomly selects thepreferred CDN for individual video playback followingan underlying CDN utilization distribution. Such CDNselection decisions are likely tied to – and constrainedby – the business relations between the content providersand CDNs.

• We perform an extensive bandwidth measurement studyof the three CDNs used by Netflix and Hulu. Theresults show that there are significant variations in CDNperformance across time and geo-locations. These resultsimply that the (static) CDN selections made by Netflixand Hulu do not necessarily provide the best QoE to endusers.

• Finally, we explore alternative strategies for improv-ing video delivery performance using multiple CDNswhile conforming to the business constraints. Our studyshows that selecting the best serving CDN based on asmall number of measurements at the beginning of eachvideo session can deliver more than 12% bandwidthimprovement over the static CDN assignment strategycurrently employed by Netflix. Furthermore, using mul-tiple CDNs simultaneously can achieve more than 50%improvement. Higher available bandwidth opens doorsfor supporting ever improving video quality (thus highervideo bit-rate), and new services such as 3D moviesand/or multiple concurrent movies in a single household.

The paper is organized as follows. Section II and Section IIIdescribe the architectures of Netflix and Hulu video streamingplatforms and their CDN selection strategy. Section IV presentsour measurement study of the three CDNs. Section V exploresthe alternative strategies for CDN assignment in order toimprove video delivery performance. Section VI discusses therelated work. Finally, Section VII concludes the paper anddiscusses the future work.

II. N ETFLIX VIDEO STREAMING PLATFORM

We start the section with the overview of Netflix videostreaming platform architecture. We dissect the architecture viatraffic monitoring, DNS resolutions, and WHOIS[7] lookup.We then present the timeline of serving a single Netflix clientas an example to illustrate the interplay between a Netflixplayer and various service components. We further collect alarge number of video streaming manifest files usingTamperData add-on[8], and analyze how geographic locations, client

TABLE I. K EY NETFLIX HOSTNAMES

Hostname Organizationwww.netflix.com Netflixsignup.netflix.com Amazonmovies.netflix.com Amazonagmoviecontrol.netflix.com Amazonnflx.i.87f50a04.x.lcdn.nflximg.com Level 3netflix-753.vo.llnwd.net Limelightnetflix753.as.nflximg.com.edgesuite.net Akamai

capabilities, and content types influence the streaming parame-ters. Finally, we focus on the Netflix CDN assignment strategy.Using dummynet[9] to strategically throttle individual CDN’sbandwidth, we discover how Netflix makes use of multipleCDNs in face of bandwidth fluctuation.

A. Overview of Netflix architecture

To observe the basic service behavior, we create a newuser account, login into the Netflix website and play a movie.We monitor the traffic during all of this activity and recordthe hostnames of the servers involved in the process. Wethen perform DNS resolutions to collect the canonical names(CNAMEs) and IP addresses of all the server names thatthe browser have contacted. We also perform WHOIS[7]lookups for the IP addresses to find out their owners. TableI summarizes the most relevant hostnames and their owners.Fig. 1 shows the basic architecture for Netflix video streamingplatform. It consists of four key components: Netflix datacenter, Amazon cloud, CDNs and players.

Fig. 1. Netflix architecture

Netflix data centers. Our analysis reveals that Net-flix uses its own IP address space for the hostnamewww.netflix.com. This server primarily handles two keyfunctions: (a) registration of new user accounts and cap-ture of payment information (credit card or Paypal ac-count), and (b) redirect users tomovies.netflix.com orsignup.netflix.com based on whether the user is loggedin or not respectively. This server does not interact with theclient during the movie playback, which is consistent with therecent presentation from Netflix team [10].

Amazon cloud. Except for www.netflix.comwhich is hosted by Netflix, most of the other Netflixservers such asagmoviecontrol.netflix.com andmovies.netflix.com are served off the Amazoncloud [11]. [10] indicates that Netflix uses various Amazon

3

50 100 150 200 250 300

Player (Amazon Cloud)

Manifest (Amazon Cloud)

Trickplay (CDN)

Video chunk (CDN)

Audio chunk (CDN)

Reporting (Amazon Cloud)

Number of seconds from start

Fig. 2. Timeline in serving a Netflix client

cloud services, ranging from EC2 and S3, to SDB andVPC [11]. Key functions, such as content ingestion, logrecording/analysis, DRM, CDN routing, user sign-in, andmobile device support, are all done in Amazon cloud.

Content Distribution Networks (CDNs). Netflix employsmultiple CDNs to deliver the video content to end users. Theencoded and DRM protected videos are sourced in Amazoncloud and copied to CDNs. Netflix employs three CDNs:Akamai, LimeLight, andLevel-3. For the same video with thesame quality level, the same encoded content is delivered fromall three CDNs. In Section II-D we study the Netflix strategyused to select these CDNs to serve videos.

Players. Netflix uses Silverlight to download, decode andplay Netflix movies on desktop web browsers. The run-timeenvironment for Silverlight is available as a plug-in for mostweb browsers. There are also players for mobile phones andother devices such as Wii, Roku, etc. This paper, however,focuses on Silverlight player running on desktop PCs.

Netflix uses the DASH (Dynamic Streaming over HTTP)protocol for streaming. In DASH, each video is encodedat several different quality levels, and is divided into small‘chunks’ - video segments of no more than a few seconds inlength. The client requests one video chunk at a time via HTTP.With each download, it measures the received bandwidth andruns arate determination algorithmto determine the qualityof the next chunk to request. DASH allows the player to freelyswitch between different quality levels at the chunk boundaries.

B. Servicing a Netflix client

We now take a closer look at the interaction between theclient web browser and various web servers involved in thevideo playback process. Fig. 2 shows the timeline along whichthe streaming service is provided to a desktop client, andindicates the involved server entities. The X-axis in this figureshows the time from the beginning of the experiment to 5minutes and the Y-axis lists different activities. The clientfirst downloads the Microsoft Silverlight application frommovies.netflix.com and authenticates the user. Afterauthentication, the player fetches the manifest file from thecontrol server atagmoviecontrol.netflix.com, basedon which it starts to download trickplay data and audio/videochunks from different CDNs. Client reports are sent back tothe control server periodically. We describe further details ofindividual activities below.

1) Silverlight player download and user authentication:Video playback on a desktop computer requires the MicrosoftSilverlight browser plug-in to be installed on the computer.When the user clicks on “Play Now” button, the browserdownloads the Silverlight application and then that applicationstarts downloading and playing the video content. This smallSilverlight application is downloaded for each video playback.

2) Netflix manifest file:Netflix video streaming are con-trolled by instructions in a manifest file that the Silverlightclient downloads. The Netflix manifest file provides the DASHplayer metadata to conduct the adaptive video streaming.The manifest files are client-specific, i.e., they are generatedaccording to each client’s playback capability. For instance,if the user player indicates it is capable of rendering h.264encoded video, h.264 format video is included in the manifestfile. If the player indicates that it can only play back .wmvformat, only .wmv format video is included.

The manifest file is delivered to end user via SSL connectionand hence the content of the file cannot be read over the wireusing packet capture tools such astcpdumpor wireshark. Weuse Firefox browser andTamper Dataplug-in to extract themanifest files. The extracted manifest file is in XML formatand contains several key pieces of information including thelist of the CDNs, location of the trickplay data, video/audiochunk URLs for multiple quality levels, and timing parameterssuch as time-out interval, polling interval and so on. Themanifest file also reveals interesting information on the Netflixsystem architecture. For instance, they show that Netflix usesthree CDNs to serve the videos. Different ranks are assignedto different CDNs to indicate to the clients which CDN ismore preferred than others. A section of one of the manifestfiles is shown in Fig. 3, where Level3 is listed as the mostpreferred CDN for this client. We will conduct more elaborateexperiments and discuss more details of the manifest files laterin this section.

3) Trickplay: Netflix Silverlight player supports simpletrickplay such as pause, rewind, forward and random seek.Trickplay is achieved by downloading a set of thumbnailimages for periodic snapshots. The thumbnail resolution, pixelaspect, trickplay interval, and CDN from where to downloadthe trickplay file are described in the manifest file. Thetrickplay interval for the desktop browser is 10 seconds, andmultiple resolutions and pixel aspects are provided.

4

<nccp:cdns><nccp:cdn>

<nccp:name>level3</nccp:name><nccp:cdnid>6</nccp:cdnid><nccp:rank>1</nccp:rank><nccp:weight>140</nccp:weight>

</nccp:cdn><nccp:cdn>

<nccp:name>limelight</nccp:name><nccp:cdnid>4</nccp:cdnid><nccp:rank>2</nccp:rank><nccp:weight>120</nccp:weight>

</nccp:cdn><nccp:cdn>

<nccp:name>akamai</nccp:name><nccp:cdnid>9</nccp:cdnid><nccp:rank>3</nccp:rank><nccp:weight>100</nccp:weight>

</nccp:cdn></nccp:cdns>

Fig. 3. CDN list in manifest file

4) Audio and video chunk downloading:As shown in Fig. 2,audio and video contents are downloaded in chunks. Downloadsessions are more frequent at the beginning so as to build upthe player buffer. Once the buffer is sufficiently filled, down-loads become periodic. The interval between the beginning oftwo consecutive downloads is approximately four seconds -the playback length of a typical chunk.

The manifest file contains multiple audio and video qualitylevels. For each quality level, it contains the URLs for indi-vidual CDNs, as shown in Fig. 4.

<nccp:bitrate>560</nccp:bitrate><nccp:videoprofile>

playready-h264mpl30-dash</nccp:videoprofile><nccp:resolution>

<nccp:width>512</nccp:width><nccp:height>384</nccp:height>

</nccp:resolution><nccp:pixelaspect>

<nccp:width>4</nccp:width><nccp:height>3</nccp:height>

</nccp:pixelaspect>v<nccp:downloadurls>

<nccp:downloadurl><nccp:expiration>131xxx</nccp:expiration><nccp:cdnid>6</nccp:cdnid><nccp:url>http://nflx.i.../...</nccp:url>

</nccp:downloadurl><nccp:downloadurl>

<nccp:expiration>131xxx</nccp:expiration><nccp:cdnid>4</nccp:cdnid><nccp:url>http://netflix.../...</nccp:url>

</nccp:downloadurl><nccp:downloadurl>

<nccp:expiration>131xxx</nccp:expiration><nccp:cdnid>9</nccp:cdnid><nccp:url>http://netflix.../...</nccp:url>

</nccp:downloadurl></nccp:downloadurls>

Fig. 4. Video downloadable for one quality level

5) User experience reporting:After the playback starts,Netflix player communicates periodically with the controlserveragmoviecontrol.netflix.com. Based upon thekeywords such as “/heartbeat” and “/logblob” in the requestURLs and the periodicity of the communication, we conjecturethat they are periodic keep alive messages and log updates.However, the actual messages that we have extracted by usingTamper Datado not appear to be in clear text and hence wecannot verify it further.

C. Manifest file analysis

A manifest file is delivered over the SSL connection. WeuseTamper Dataplug-in for Firefox browser to read the file.Since the manifest files contain a wealth of information andshed lights on the Netflix strategies, we conduct a large scaleexperiment by collecting and analyzing a number of manifestfiles. We are interested in understanding how geographiclocations, client capabilities, and content type (e.g., popular vsunpopular, movies vs TV shows) may impact the streamingparameters. We use six different user accounts, 25 moviesof varying popularity, age and type, four computers withMac and Windows systems at four different locations for thisexperiment. From each computer, we log into Netflix site usingeach of the user accounts and play all of the movies for fewminutes to collect the manifest files. In addition to using clientmachines located in different geographies, we also configurethose client browsers to use Squid proxy servers running onten PlanetLab nodes hosted by US universities in differentgeographic regions to collect additional manifest files.

1) CDN ranking and user accounts:Netflix manifest filesrank CDNs to indicate which CDNs are preferred. CDNranking determines from which CDN the client downloads thevideo and may affect user perceived video quality. We analyzethe collected manifest files to understand the factors that affectthe rankings of the CDNs. For this analysis, we build a tablethat lists CDN ranking for each combination of user account,client computer (or PlanetLab proxy), movie ID and time ofday for several days. Analysis of this table suggests that theCDN ranking is only based upon the user account. For a givenuser account, the CDN ranking in the manifest file remainsthe same irrespective of movie types, computers, time andlocations. Furthermore, for the same movie, computer, locationand around the same time, two different users may see differentCDN rankings. We also observe that the CDN ranking for eachuser account remains unchanged for at least several days. Suchassignment of ranking seems to be independent of availablebandwidth from each CDN as shown in the Section IV.

2) Audio/Video bit rates:Netflix serves videos in multipleformats and bit rates. When a Netflix client requests for themanifest file from Netflix, the client indicates the formatsof the content it can play. Netflix server then sends backa manifest file based upon the client request. For instance,Netflix client running on an older computer (Thinkpad T60with Windows XP) and a newer computer (Macbook Pro withSnow Leopard) have different capabilities and receive differentvideo downloading format and bit rates.

Based on the client capabilities, the server sends URLs forthe video and audio chunks in the returned manifest files.

5

In general, manifest files contain information about videochunks encoded in bit rates between 100Kbps to 1750Kbps(and 2350Kbps and 3600Kbps for videos available in HD) forthe manifest files sent to the newer computer. We see thatvideos available in HD can be served in up to 14 differentbit rates whereas non-HD content can be served in up to 12different bit rates. We also note that Netflix clients do not tryall possible available bit rates when trying to determine theoptimal playback rate.

D. CDN selection strategy

We have seen that a Netflix client can choose different videobit rates and different CDNs for video downloading. In thissection we conduct experiments to help understand how Netflixmake such choices when bandwidth is dynamic. We play asingle movie from the beginning. Once the playback starts, wegradually throttle the available bandwidth of the top rankedCDN in the manifest file. We usedummynetto throttle theinbound bandwidth to the client.

At the beginning, servers from each CDN are allowed tosend data at 3,900Kbps. After every minute, we reduce theavailable bandwidth for the current CDN by 100Kbps till itreaches 100Kbps. At that point we start throttling the nextCDN in the same way and so on. We plot our observationin Fig. 5. In this figure, the X-axis shows the time startingfrom the beginning of playback. The Y-axis shows both thethrottled bandwidth and the playback rate. In this instance,Level3, Limelight and Akamai CDNs are ranked first, secondand third respectively. The client starts downloading videochunks from the first CDN. In the beginning, it starts froma low bit rate and gradually improves the bit rate in a probingfashion. As we lower the available bandwidth for the first CDNwhile leaving the other CDNs intact, we notice somethinginteresting. Instead of switching to a different CDN, whichis not throttled, the client keeps lowering the bit rate and stayswith the first CDN. Only when it can no longer support eventhe very low quality level (i.e. when the available bandwidthfor the first CDN reaches 100Kbps), it switches to the secondCDN. It repeats almost the same behavior as we leave the firstCDN at 100Kbps and gradually lower the available bandwidthfor the second CDN while leaving the third CDN intact. Ingeneral, the Netflix clients stay with the same CDN as long aspossible even if it has to degrade the playback quality level.

III. H ULU VIDEO STREAMING PLATFORM

Besides Netflix, Hulu is another major OTT (Over-The-Top)video service provider that does not own extensive infras-tructure yet manages to support large scale video streamingservice. Examining the similarity and discrepancy of Netflixand Hulu’s respective video streaming platform and employedtechnologies shall shed light on the state-of-the-art of the videostreaming platform design. Unlike Netflix, Hulu offers bothsubscription based service (called HuluPlus) and free service.Free service is for desktop users, while HuluPlus supportsadditional platforms, e.g., set-top boxes, mobile devices, etc.,and offers HD video quality. Video advertisement is another

Fig. 6. High-level Hulu architecture

major component of Hulu, where short advertisement clips aretypically delivered to users prior to the main video content.

We employ methodologies similar to the ones used in theNetflix study in studying Hulu. We play multiple videos ondifferent web browsers from multiple locations with differentISPs, with or without firewalls and proxy servers. We alsocorroborate several of these observations using what has beenpublished on Hulu’s help pages and blogs. A high levelHulu architecture for desktop clients is shown in Figure 6.The client gets the HTML pages for the video from Hulu’sfront-end web server atwww.hulu.com. It then contactss.hulu.com to obtain a manifest file that describes theserver location, available bit-rates and other details. The clientuses the instruction in the manifest file to contact a videoserver to download the video. The client also periodically sendsits status report tot.hulu.com. The similarity between theNetflix architecture and the Hulu architecture is striking -bothplatforms utilize the third-party commercial data centersandmultiple CDNs to flexibly scale up/down to accommodate forthe changing user population.

Bandwidth Requirements. Hulu videos are streamed at480Kbps and 700Kbps. A few videos can also be streamedat 1000Kbps. HuluPlus subscribers can also access videos inHD quality when available. Clients can switch between bit-rates during playback based on available bandwidth, as wewill explain later.

CDNs. Hulu emply the same three CDNs as Netflix todeliver video contents to users. Based on manifest files, a Huluclient is first assigned a preferred CDN hostname, and thenuses DNS to select a server IP address.

Streaming Protocol. Hulu uses encrypted RTMP (RealTime Messaging Protocol) to deliver movies to desktopbrowsers. Hulu videos can be delivered over raw RTMP onport 1935 or RTMP tunneled over HTTP (RTMPT). Our exper-iments reveal that Level3 prefers raw RTMP whereas Akamaiand Limelight prefers RTMPT. All three CDNs use RTMPTwhen TCP port 1935 is blocked (by a firewall for instance).HuluPlus on a desktop browser uses the same technologiesand protocols. However, on mobile devices HuluPlus usesadaptive streaming over HTTP. For instance, oniPhone andiPad, HuluPlus content is delivered over HTTP LiveStream-ing technology[12]. Hulu advertisements are single .FLV files.These files are small (a few megabytes) and are downloadedover single HTTP transactions.

A. CDN Selection Strategy

Analyzing the captured packet traces, we find that Huluuses only one CDN server throughout the duration of a

6

0 1000 2000 3000 4000 5000 6000 7000 80000

500

1000

1500

2000

2500

3000

3500

4000

Number of seconds from start

Allo

wed

ban

dwid

th/O

bser

ved

bitr

ate

(in K

bps)

Bitrate − LimeLight Bitrate − Level3 Bitrate − Akamai Bandwidth − LimeLight Bandwidth − Level3 Bandwidth − Akamai

Fig. 5. CDN switching

video. But interestingly, it usually switches to a differentCDN for the next video. To further understand how networkconditions affect player behavior and CDN selection strategy,we conduct the same bandwidth throttling experiment as inNetflix measurement study. At the beginning, servers fromeach CDN are allowed to send data at1501 Kbps. At theend of every minute, we reduce the available bandwidth forthe current active CDN by100 Kbps till it reaches1 Kbps.As we lower the available bandwidth for the current CDNwhile leaving the other CDNs intact, we notice that insteadof switching to a different CDN, which is not throttled, theclient keeps lowering the bit-rate and stays with the originalCDN. This indicates that Hulu adapts to changing bandwidthby adjusting the bit-rates and continues to use the same CDNserver as long as possible. Only when the current CDN serveris unable to serve the lowest possible bit-rate, it switchesto adifferent CDN server. In summary, for a single video playback,Hulu’s CDN selection strategy is very similar to that of Netflix.

B. User Experience Reporting

From the packet trace, we find that the Hulu player sendsperiodic reports to a server that includes detailed informationabout the status of the client machine at that time, the CDNservers for video content and advertisements, and any problemsencountered in the recent past. These periodic status reportsare sent tot.hulu.com which maps to the same single IPaddress from all the locations in US. Using WHOIS[7] queries,we learn that the corresponding IP address, 208.91.157.68,isallocated to Hulu. Examples of detailed performance informa-tion contained in the periodic reports includes: video bit-rate,current video playback position, total amount of memory theclient is using, the current bandwidth at the client machine,number of buffer under-runs, and number of dropped frames.When the client adapts bit-rate due to changing networkconditions, the periodic reports also include details on why thebit-rate was changed. For instance, one of the messages reads“Move up since avg dropped FPS 0< 2 and bufferLength>10”, with FPS indicatingFrames Per Second. It appears thatHulu has sufficient user performance information for dynamicCDN selection if they choose to do so.

C. Manifest Files and CDN Usage AnalysisHulu clients follow the manifest files they receive from

the server to decide which CDN to use. Since Hulu encryptsthe manifest file sent to the client, the manifest files fromthe network traces are not readable. We collect the manifestfiles using a tool calledget-flash-videos[13]. A smallsection of an example Hulu manifest file is shown in Fig. 7.The last line in the figure shows Hulu’s CDN preference inthat manifest file. When we examine CDN preferences in afew manifest files, we observe that the preferred CDN variesfrom one manifest file to another. For instance, when we maketwo sequential requests for the same video, the preferred CDNsfor those two requests can be different.

’title’ => ’Soldier Girls’,’tp:Ad_Model’ => ’longform’,’tp:Frame_Rate’ => ’25’,’dur’ => ’4991138ms’,’tp:enableAdBlockerSlate’ => ’true’,’tp:Aspect_Ratio’ => ’4x3’,’tp:BugImageURL’ => ’’,’tp:researchProgram’ => ’’,’tp:comScoreId’ => ’’,’tp:hasBug’ => ’false’,’tp:defaultBitrate’ => ’650_h264’,’tp:primarySiteChannelNielsenChannelId’ => ’71’,’tp:CP_Promotional_Link’ => ’’,’tp:CPIdentifier’ => ’ContentFilm’,’tp:Primary_Category’ => ’Documentary and Biography’,’tp:adType’ => ’’,’tp:fingerPrint’ => ’csel3_prod_iad115’,’tp:CP_Promotional_Text’ => ’’,’tp:Segments’ => ’T:00:11:54;22,T:00:26:29;09,

T:00:38:49;27,T:00:57:37;18,T:01:03:15;02,T:01:17:12;03’,

’tp:adTypePlus’ => ’SponsoredFilm’,’tp:Tunein_Information’ => ’’,’tp:distributionPartnerComScoreId’ => ’3000007’,’tp:secondarySiteChannelNielsenId’ => ’38’,’tp:cdnPrefs’ => ’level3,akamai,limelight’,

Fig. 7. A section of Hulu manifest file

To better understand how Hulu selects different CDNs, werequest the manifest file every second for the same videofrom the same computer for 100 seconds. Figure 8 depicts

7

10 20 30 40 50 60 70 80 90 100

Akamai

Limelight

Level3

Time of request (in seconds)

Fig. 8. CDN preference change in a 100 seconds time interval

the preferred CDN along time, with ‘*’ indicating the selectedCDN for each request. Since the network conditions on thetested Hulu client is fairly stable during the experiment, theabove result indicates that Hulu CDN selection is not basedon instantaneous network conditions.

To further understand the impact of various factors suchas client location, video and time on CDN selection, we usethe get-flash-videos tool to collect manifest data for61 different videos of different genres, length, popularity andratings available on Hulu from 13 different locations acrossthe United States over multiple days (up to 24 days at oneof the locations). The client machines on these locations areconnected to residential broadband networks or business highspeed Internet services. They also cover a number of differentISPs including Comcast, AT&T, Verizon and CenturyLink.

For a given video at a given location and time, we downloadthe manifest file 100 times, with 1 second interval between twoconsecutive downloads. We call such 100 consecutive down-loads anexperiment. Each downloaded manifest file assignsone CDN as preferred CDN. We count the number of timeseach CDN is preferred for each experiment. We refer to thepercentage of times that a CDN is preferred in an experiment aspreference percentage. This preference percentage essentiallyreflects the likelihood for a CDN to be selected by Hulu.

Overall CDN preferenceFigure 9 shows the distribution ofpreference percentage for the three CDNs based on results forall videos, locations, and time. The three curves representingthe three CDNs are very close to Gaussian distributions. Themean preference percentage for Limelight, Akamai and Level3are 25, 28 and 47, respectively. Level3 is the preferred CDN47% of times, much more than the other two CDNs.

CDN preference over different locations.Figure 10 showsCDN preference observed from clients at different geographiclocations. These 13 locations span different cities acrosseightUS states. For this analysis, we combine data for all thevideos collected at the same location and calculate the averagepreference percentage for each location. We observe thatdifferent CDNs have different popularity but the popularitydo not change over different locations.

CDN preference for different videos. Figure 11 showsCDN preference for different videos. Here we aggregate theexperiments for each video across location and time andcalculate its average preference percentage. The small variationin preference percentage across different videos indicateCDNpreference is independent of which video is being served.

0 20 40 60 80 1000

0.02

0.04

0.06

0.08

0.1

0.12

CDN preference percentage

Pro

babi

lity

AkamaiLevel3Limelight

Fig. 9. Overall CDN preference distribution

2 4 6 8 10 120

10

20

30

40

50

Client location IDs

Ave

rage

pre

fere

nce

%

AkamaiLevel3Limelight

Fig. 10. CDN preference from geographic regions

CDN preference over time.Figure 12 shows CDN pref-erence change over different days at the same location. Thisresult is based on 24 days of experiments at a single location.Each data point represents the average preference percentageover all videos on each day for a given CDN. The results forother locations (not shown here) are similar. We observe thatthe CDN preferences do not change over time either.

In summary, we conclude that Hulu selects the preferredCDN randomly following a fixed latent distribution for eachof the playback requests. On average, one CDN (Level3) ispreferred more than others, but such selection preference doesnot seem to depend on instantaneous network conditions. Itis also evident that CDN selection is not affected by clientlocation at which the video is played. And the selectiondoes not change over the 24 days that we measured. We

8

10 20 30 40 50 600

10

20

30

40

50

Video IDs

Ave

rage

pre

fere

nce

%

AkamaiLevel3Limelight

Fig. 11. CDN preference for different videos

5 10 15 200

10

20

30

40

50

24 days between August 17 to September 9, 2011

Ave

rage

pre

fere

nce

%

AkamaiLevel3Limelight

Fig. 12. CDN preference over time

conjecture that such CDN preference is most likely basedon pricing and business arrangements and is not dependentupon instantaneous bandwidth or past performance history ofthe CDNs. Note that the above CDN usage analysis is notdoable for Netflix since Netflix ties the CDN preference tothe individual user account. It is impractical to create a largenumber of Netflix accounts to infer its CDN usage strategy.

IV. CDN PERFORMANCEMEASUREMENT

In the previous sections, we have shown that Netflix tiesthe CDN preference to user accounts, while Hulu choose thepreferred CDN for each video based on a latent distribution.In both cases, factors such as user geographic locations,network conditions, requested video contents do not trigger theCDN preference change. These observations suggest that theCDN preference and selection strategies employed by Netflixand Hulu are plausibly due tobusiness considerationssuchas business relations (e.g., pricing agreements) between thecontent providers and CDNs: while Netflix and Hulu employdifferent CDN selection strategies (one ties the preferredCDNto each user account and the other ties to each video), bothattempt to distribute and balance the video serving trafficamong the CDN in accordance with certainlatentdistribution.This raises a key designtrade-off in CDN selection decisionmaking: business constraints vs. end user QoE, and leads tothe following questions:• How does each CDN perform? Can the selected CDN

server consistently support the bandwidth needed for

high quality streaming?• How do different CDNs compare in terms of perfor-

mance? Is any CDN clearly better or worse than others?• How far is the current CDN selection strategy from “op-

timal”? Can the strategy be improved to support higherdelivery bandwidth while conforming to the businessconstraints?

In this section and the Section V, we attempt to addressthe above questions by conducting extensive measurementexperiments for the three CDNs used by Netflix from 95vantage points across the United States1.

We measure the bandwidth throughput between each van-tage point and a given CDN server by downloading multiplevideo chunks from the CDN server. Video file URLs arecollected for all three CDNs from Netflix manifest files. Herewe take advantage of the fact that the URLs in the manifestremain valid for several hours from the time the manifest fileisgenerated, and the validity of the URLs are not tied to clientIPaddress. Furthermore, the byte “range” of the download can beadjusted without affecting the URL validity. Once we extractthe URLs for the three CDNs, we “replay” the GET requestfrom all vantage points with byte range modified so that wedownload video chunks of the same size.

Similar to the actual Netflix video playback, when GETrequests are sent from a vantage point, the hostnames inthe URLs are resolved by DNS server, which returns the IPaddress of the edge server assigned by the CDN. To ensurethe measured bandwidth of three CDNs are comparable, wesend GET requests to three CDNs in round-robin order withina short duration. More specifically, measurement is repeatedin multiple “rounds”, with each round lasting 96 seconds. Around is further partitioned into four “slots”, with 24 secondsfor each slot. The first three slots of each round correspondto three CDNs, respectively, and we download video chunksof size1.8MByte. The last slot of each round is for a “joint”measurement for all CDNs, i.e., we send GET requests to thethree CDNs simultaneously, each requesting video chunks for0.6MByte data. We intend to find out how much total band-width one can get if all three CDNs are used simultaneously.We pick the size of the chunks and length of “slots” basedupon multiple trial measurements. In our trials, we find thatthese numbers make sure that different experiments do notinterfere with each other and chunk size is sufficiently largeso that we can have a good estimate of the bandwidth. We alsosend keep-alive messages to each server every second when nodata is transferred to make sure that the TCP session is aliveand sender window size does not drop.

The measurement is conducted for two hours between 8 to10pm CST, from June 8, 2011 to June 26, 2011. Based ondownloading time, we calculate the instantaneous bandwidth(i.e., throughput for each GET request), the one-day averagebandwidth (average bandwidth during the two hour period),and average bandwidth (over entire measurement study). These

1As Hulu shares the same set of CDN providers with Netflix, and sincethe RTMPE protocol used by Hulu is more difficult to work with than theDASH protocol used by Netflix, here we choose Netflix to conduct the CDNexperiments.

9

metrics allow us to examine CDN performance at multipletimescales. We conducted experiments from both residentialsites and PlanetLab nodes. There are 12 residential sites, 10in New Jersey, 1 in Minnesota, and 1 in California. Theresidential sites use 5 different service providers. To covera wider range of geographic locations, we also choose 83PlanetLab nodes spread across the United States as additionalvantage points. We ensure that all selected PlanetLab nodesare lightly loaded so that the nodes themselves do not becomethe bottleneck and the measurement results reflect the actualbandwidth that can be supported by the CDN server and thenetwork.

The rest of this section attempts to address the first twoquestions on CDN performance. We will further investigatethe other two questions on performance improvement in theSection V. We use CDNA, B, and C to denote the threeCDNs without particular order in the rest of the discussion.

A. Overall CDN performance

Fig. 13 shows the locations of all vantage points in ourexperiments as well as the CDN with highest average band-width at each vantage point during the measurement period.As the result indicates, no CDN clearly outperforms the others.In addition, Fig. 14 shows the CDF (Cumulative DistributionFunction) of average bandwidth at the PlanetLab nodes overthe entire measurement period. The available bandwidth atdifferent PlanetLab nodes varies significantly from location tolocation, ranging from 3Mbps to more than 200Mbps. TheCDF curves of three CDNs, however, are close to each other,indicating similar overall performance. Figures 15 and 16further show the average bandwidth at individual locationsfor PlanetLab nodes and residential sites, respectively. Thelocation index is sorted in the ascending order of CDNA’saverage bandwidth. CDN bandwidth measured at PlanetLabnodes appear to have much higher than that of residential sitesin general. This is because most PlanetLab nodes are locatedinuniversities, which typically have better access links. This alsoimplies that in most cases, the last mile is still the bottleneckfor streaming video. However, even the residential sites withrelatively low bandwidth, e.g. home 1 and 2 in Fig. 16, cansupport1.3Mbps on average, enough for standard definition(SD) videos.

Fig. 13. Best CDN at each vantage point

It is also interesting to note that home sites 4, 9, and 11 seesignificantly different average bandwidth from different CDNs.

In particular, CDN B outperforms all others considerably. Wefind that these three homes use the same ISP. It is conceivablethat CDN B has a better presence in this provider’s network.

B. Daily bandwidth variations

Next we examine the bandwidth variation at different sitesfrom the three CDNs over various timescales. We compute thecoefficient of variance (CoV) of the daily average bandwidthat all PlanetLab nodes by computing the ratio of the standarddeviation to the mean at each of the locations. Fig. 17shows the CoV for the one-day average bandwidth at variousPlanetLab nodes over multiple days. We indeed see high CoVat most nodes. The average CoV is 0.33, 0.30, and 0.30 forCDN A, B and C, respectively. At most locations, there aresignificant variations in daily bandwidth for all three CDNs.We show a few representative locations in Figures 18, 19and 20, which plot the one-day average bandwidth over themeasurement period at one PlanetLab node and two residentialsites, respectively. The results show significant variations ofaverage bandwidth on a daily basis.

Figures 18, 19 and 20 show that the performance rankingof the three CDNs also varies over time. Although the lowestCDN bandwidth across all three nodes is still above 3Mbps– sufficient to support standard definition (SD) levels, signif-icant variations in bandwidth and varying rankings of CDNsover time suggest that further improvement in CDN selectionstrategies is possible.

C. Variations in instantaneous bandwidth

We further investigate the instantaneous bandwidth variationduring two hours of video playing. This is important since aDASH player constantly monitors the available bandwidth todecide which quality level of video to download. The smalltime scale bandwidth may significantly impact the Netflixusers’ viewing experience as two hours is a typical length ofmovie. Figures 21, 22 and 23 show the comparison of threeCDNs for the same PlanetLab node and residential nodes.Although the variance is still significant, there is a “pattern”in the bandwidth change. For example, bandwidth for CDN Bin Fig. 21 alternates between two levels, one around 35 Mbpsand one around 20 Mbps. The average coefficient of variationfor two hour period is0.19, 0.21 and 0.18 respectively forCDNs A, B and C respectively for residential sites.

V. A LTERNATE V IDEO DELIVERY STRATEGIES

We have shown that Netflix and Hulu always prefer to useone CDN for video delivery, with other two CDNs serving asbackups: they are used only if current CDN cannot supporteven the lowest video quality. We have also shown that theavailable bandwidth on all three CDNs vary significantly overtime and over geographic locations. For instance, as shownin Fig. 13, out of 83 PlanetLab locations, CDNsA, B, andC perform best at 30, 28 and 25 locations, respectively. Themeasurement study of residential hosts shows similar results.Hence if users are given a bad CDN choice, their video viewingquality may suffer even though other CDNs can provide them

10

0 35.8 71.6 107.4 143.2 1790

20

40

60

80

100

Average bandwidth in Mbps

CD

F

CDN A CDN B CDN C

Fig. 14. CDF of average bandwidth at Planet-Lab nodes

20 40 60 80

20

40

60

80

100

120

140

160

PlanetLab node IDs

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 15. Average bandwidth at PlanetLab nodesover the entire period

1 2 3 4 5 6 7 8 9 10 11 120

5

10

15

20

Residential host IDs

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 16. Average bandwidth at residentialnetworks over the entire period

20 40 60 80

0.5

1

1.5

2

PlanetLab node IDs

Coe

ffici

ent o

f var

ianc

e

CDN A CDN B CDN C

Fig. 17. Coefficient of variance for the one-dayaverage at PlanetLab nodes

0 5 10 15 205

10

15

20

25

30

35

40

45

Time (in days)

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 18. One-day average bandwidth at aPlanetLab node over time

0 5 10 153

4

5

6

7

8

9

10

Time (in days)

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 19. One-day average bandwidth over timeat residential site 7

0 5 10 150

5

10

15

20

Time (in days)

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 20. One-day average bandwidth over timeat residential site 9

0 0.5 1 1.5 20

5

10

15

20

25

30

35

40

Time (in hours)

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 21. Instantaneous bandwidth at a PlanetLabnode

0 0.5 1 1.5 20

5

10

15

20

25

Time (in hours)

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 22. Instantaneous bandwidth at residentialsite 7

with more satisfying experience. In addition to improvingexperience for “unlucky” users, exploring potential ways ofincreasing video delivery bandwidth may also open doors fornew bandwidth-demanding services in the future, e.g., 3Dmovies or multiple concurrent movies in the same household.In this section, we first determine how much room availablefor further improvement. In other words, if we could have theoptimal CDN selection strategy in theory, how much betterit would be compared to current static assignment. We thenexplore two alternative CDN selection strategies that can easilybe deployed in practice, and demonstrate that we can indeedsignificantly increase the bandwidth for video delivery despite

the simplicity of such strategies.

A. Room for improvement

Given the instantaneous bandwidth trace, the optimal CDNselection strategy is to choose the top CDN at each point oftime. Although this cannot be done in practice as we do notknow the instantaneous bandwidth beforehand, this theoreticaloptimal strategy allows us to find out the highest bandwidtheach client can receive if the best CDN is used at any giventime. We refer to the average bandwidth achieved by theoptimal strategy as theupper bound average bandwidth.

11

0 0.5 1 1.5 22

3

4

5

6

7

8

9

10

Time (in hours)

Ave

rage

ban

dwid

th in

Mbp

s

CDN A CDN B CDN C

Fig. 23. Instantaneous bandwidth at residentialsite 9

2 4 6 8 10 12

2

4

6

8

10

12

14

16

18

Residential site IDs

Ave

rage

ban

dwid

th in

Mbp

s

Best CDN Average CDN Upper bound

Fig. 24. Average bandwidth and the upperbound at residential sites

20 40 60 80

50

100

150

200

PlanetLab node IDs

Ave

rage

ban

dwid

th in

Mbp

s

Best CDN Average CDN Upper bound

Fig. 25. Average bandwidth and the upperbound at PlanetLab nodes

0 2 4 6 8 1060

65

70

75

80

85

90

95

100

Number of measurements in the beginning

Per

cent

age

of th

e be

st C

DN

PlanetLab with measurementPlanetLab averageResidential with measurementResidential average

Fig. 26. Effect of number of measurements

2 4 6 8 10 12

5

10

15

20

25

30

Residential host IDs

Ave

rage

ban

dwid

th in

Mbp

s

Best CDN 3 CDNs combined

Fig. 27. Best CDN vs three combined CDNsfor residential hosts

20 40 60 80

50

100

150

200

250

300

350

400

PlanetLab node IDs

Ave

rage

ban

dwid

th in

Mbp

s

Best CDN 3 CDNs combined

Fig. 28. Best CDN vs three combined CDNsfor PlanetLab nodes

Fig. 24 and Fig. 25 show the average bandwidth of threeCDNs and the upper bound average bandwidth for residentialsites and PlanetLab nodes respectively. Here we use theaverage bandwidth over all three CDNs to reflect the staticassignment strategy. The actual assignment may of course bebetter or worse depending on which CDN gets selected, butthis gives the expected value. We also show the bandwidth ifone top CDN, i.e., the one with highest average bandwidthis selected. For the majority of the sites, the upper bound ismuch better than the average CDN case, and close to the topCDN case. In particular, the upper bound is 17% and 33%better than the average case for residential sites and PlanetLabnodes respectively, indicating there is significant room forimprovement. Assigning users to top CDN is only 6% to 7%worse than the theoretical optimal case. This indicates that ifwe can estimate which CDN is likely to perform best in nextcouple hours, we can achieve average bandwidth that is fairlyclose to theupper bound average bandwidth.

B. Measurement based CDN selection

Since selecting the top CDN for users gives good perfor-mance, we next study how to identify the top CDN effectively.We propose to have the player conduct the instantaneousbandwidth measurement multiple times at the beginning, andassign users the best-performing CDN for the rest of themovie. Fig. 26 shows the effect of number of measurementson performance. As reference, two straight lines show the ratio

of the CDN average bandwidth over top CDN bandwidth forall PlanetLab and residential nodes, respectively. In bothcaseswe calculate the average CDN bandwidth over all locations,time, and CDN providers, so they reflect the expected CDNperformance, assuming the three CDNs are equally likely tobe chosen in the static CDN assignment strategy. The othertwo curves are ratio of average bandwidth using measurementbased CDN selection strategy over that of using top CDNfor both PlanetLab nodes and residential sites. Using a smallnumber of measurements (≥ 2), the measurement basedstrategy delivers more than 12% improvement over the staticCDN assignment strategy. Although the average improvementis moderate, for certain users the improvement is significant,e.g., more than 100% for residential host 4. Given this methodis very straightforward and easy to implement, we believe thisis a favorable approach for improving video delivery.

C. Using multiple CDNs simultaneously

In previous sections, we have assumed that only one CDNcan be used at a time. However, since Silverlight playerdownloads video and audio content in chunks, it is possible touse all three CDNs simultaneously. For instance, the playercan download three different chunks in parallel from threedifferent CDNs to obtain larger bandwidth. Since the designof a HTTP adaptive streaming protocol that can best utilizemultiple CDNs is out of the scope of this paper, we try to see

12

if multiple CDNs can be used, whether they can offer higheraggregated throughput for end users.

Fig. 27 and Fig. 28 compare the average bandwidth usingtop CDN and the average bandwidth obtained by combiningthree CDNs for residential and PlanetLab nodes, respectively.We see that combining all three CDNs can significantlyimprove the average bandwidth. Specifically, the aggregatebandwidth obtained by combining all 3 CDNs is greater thanthe bandwidth of the single best CDN by 54% to 70% forresidential sites and PlanetLab nodes, respectively.

D. Considering business constraints

In the above client-side initiated CDN selection strategies,the business constraints are not considered. We intend toshow how much performance improvement can be achievedby allowing the end-users to freely choose the best CDN oruse multiple CDNs simultaneously. In practice, the contentproviders may well have business arrangement with CDNservice providers, in terms of pricing, the amount of trafficneed to be carried over a specific CDN, etc. How to integratethe business constraints with the client-side initiated CDNselection strategy to form a practical and high-performancevideo delivery framework is an interesting research challenge.Below we consider one type of the business constraints, wherethe overall traffic is shared by three CDNs based on a fixedsplit-ratio - the same business constraint as used by Hulu(see Section III-C). We explore aprobabilistic CDN profileassignment strategythat conforms with the aforementionedbusiness constraint yet provides end-users better QoE byallowing them to choose the best CDN from a list of candidateCDNs provided by the content provider.

Definea CDN profileto be a list of candidate CDNs fromwhich the end-user can choose the best one to retrieve thevideo. Denote byPi the i-th profile. For three CDNs, there areseven valid CDN profiles,P0 = [C0, C1, C2], P1 = [C0, C1],P2 = [C0, C2], P3 = [C1, C2], P4 = [C0], P5 = [C1], andP6 = [C2], whereCj denotes thej-th CDN, j = 1, 2, 3. NotethatP0 offers end-users all three CDNs and hence is the mostpreferred;P1, P2, andP3 offer two candidate CDNs whichallow users to avoid the worst performing CDN. For profilesP4, P5, andP6, the end-users have no choice but to use theassigned CDN, which is the current strategy used by Hulu.Upon the arrival of a request, the content provider assign thei-th profile to this request with the probabilitypi. The goal isto find the optimal{p∗i } so that the usage of profilesP0, P1,P2, andP3 is maximized while conforming with the businessconstraints. Denote byfj the targeted traffic fraction for CDNj, and bysi,j the probability that an end-user selects CDNj asthe best performing CDN given the profilei.

∑j si,j = 1, ∀i,

with si,j = 0 if CDN j is not included in the candidate listof profile i. The optimziation problem is formulated as thefollowing linear program:

max{pi}

(w · p0 +

3∑

i=1

pi), (1)

subject to ∑

i

si,j · pi = fj , ∀j (2)

∑

i

pi = 1, (3)

pi ≥ 0, ∀i (4)

wherew ≫ 1 is the weight to favor profile0. Constraint (2)ensures that the targeted traffic split is satisfied. Constraint(3) ensures{pi} is a valid probability distribution. The valuesof {si,j} can be measured using the following approach. Atthe beginning of the service, the content provider randomlyassignsN requests with profilei, ∀i, with N being a largenumber, e.g.,1000 requests. It then collectsNi,j , the numberof requests that select CDNj given profilei. The estimationsi,j = Ni,j/N can be improved further over time. Also, arequest’s video size does not explicitly appear in the model.Since each profile will be assigned to a large number ofrequests/sessions, the average session size approaches its meanthanks to Law of Large Numbers.

Next we use a numerical example to illustrate the benefitsof the proposed scheme. Suppose the target traffic split is[0.5 0.25 0.25]. The [si,j ] vectors for profiles0 to 6 are[0.3 0.3 0.4], [0.5 0.5 0], [0.45 0 0.55], [0 0.45 0.55], [1 0 0],[0 1 0], and[0 0 1], respectively. The value ofw is set to be 10.Solving the linear program using CPLEX, the optimal solutionis p∗

0= 0.625, p∗

1= 0.125, p∗

4= 0.25, and the rest are zeros. In

other words,62.5% of the users are able to select the best CDNfrom three CDNs,12.5% of the users are able to select a betterCDN between CDN0 and CDN1, and only25% of the users usethe assigned CDN0. The above example clearly demonstratethe possibility of integrating the business constraints into theCDN selection strategy. Note that the profiles are only usedfor limiting user’s choice during performance optimization; theCDNs that are not in the profile can still be used as backup.For example, users that are assignedP4 can still switch toCDN C1 or C2 whenC0 is not feasible.

VI. RELATED WORK

Several recent studies have been conducted in analyzingdifferent aspects of Netflix video streaming. Akhshabi etal. [14] have studied several video streaming players includingNetflix player and investigated how the streaming clientsreact to bandwidth changes. The measurement is done mostlyfrom one fixed location. Recently, Huang et al. conductedthe measurement study to examine the dynamic rate selectionalgorithm of Netflix, Hulu, and Vudu, and proposed techniqueto improve users’ perceived video quality [15]. Unlike theprevious work, we investigate a broader set of componentsin Netflix video delivery system, and focus on how the playerinteracts with different CDNs. To achieve this, we conductmore extensive measurement from multiple geo locations.

Recent work has also been done for other streaming plat-forms [16], [17]. Krishnappa et al. have studied Hulu streamingwith emphasis on improving performance using prefetchingand caching [16]. Adhikari et al. build a measurement infras-tructure by using PlanetLab nodes with the goal to understand

13

the YouTube system architecture [17]. Unlike our work, suchworks do not cover the behavior of multiple CDNs.

A large-scale video quality measurement study is conductedin [18] by examining a large number of video sessions from 91content providers during a one-week time period. The resultsshow that the existing video delivery fails to provide clientssatisfactory quality of experience. A centralized video controlplane framework is proposed to optimally select CDN for end-users and achieve significant QoE improvement. Unlike [18],our work focuses on two representative content providers,i.e., Netflix and Hulu, and dissects their architecture. Someof the measurement results presented in this paper, however,are consistent with the observations made in [18]. In addition,rather than a centralized control plane solution, a client-sideinitiated CDN selection mechanism is investigated in this work.

Measurement studies of CDNs such as Akamai, Limelightand YouTube [19], [20], [17] have also been conducted, mostof which focus on measurement of latency and do not cover thescenario where the client interacts with multiple CDNs. Manytechniques have been proposed to measure available bandwidthon a network path before, such as pathchar [21], pathload [22]and FabProbe [23]. However, they are not suitable for ourstudy for two reasons. First, both pathchar and pathload requirecontrol at the target machine of the measurement. Second,all such tools only measure the in-path bandwidth and theycannot capture possible bandwidth shaping at the server side.Additionally, using our method more accurately reflects thedownload speed over HTTP than other generic methods.

VII. SUMMARY

In this paper, we performed active and passive measurementsto uncover the overall architecture of Netflix and Hulu, and dis-sect several key components of these two streaming platforms.Since Netflix and Hulu use multiple Content Delivery Net-works (CDNs) to deliver videos to its subscribers, we measuredthe available bandwidth of employed CDNs, and investigatedits behavior at multiple time scales and at different geographiclocations. We observed that neither Netflix nor Hulu takesinto account the current network conditions when choosing aCDN. We found that conducting light-weighted measurementat the beginning of the video playback and choosing thebest-performing CDN can improve the average bandwidth bymore than 12% than static CDN assignment strategy, andusing all three CDNs simultaneously can improve the averagebandwidth by more than 50%. This can be very beneficial forsupporting future bandwidth-demanding services.

As Over-The-Top (OTT) continues to become one of the ma-jor means in delivering video content, the design of streamingplatform keeps evolving to be more scalable, more flexible,and provide better service. For instance, starting in June2012, Netflix initiated its own Content Delivery Networkcalled “Open Connect” so that ISPs can directly connect theirnetworks to Open Connect [24]. Open Connect CDN allowsISPs to peering with Netflix CDN for free at common Internetexchanges, or put Netflix storage appliances in or near ISPnetwork to save even more transit costs. The initiative hasattracted some ISPs to connect to Open Connect. However,

some major ISPs still refuse to connect with Open Connectdue to the business concerns. For the users belonging tothese un-connected ISPs, three CDNs are still used. Henceour results/conclusions remain valid for these ISPs and theirusers. As the user base of OTT continues to grow, the designof streaming platform must evolve and advance accordingly.Hence keeping up with the new design, understanding its prosand cons, and improving upon it remain to be an interestingresearch topic for the future.Acknowledgement.Vijay K. Adhikari and Zhi-Li Zhang weresupported in part by the US NSF grants CNS-1017647 andCNS-1117536, the DTRA grant HDTRA1- 09-1-0050, and aDoD ARO MURI Award W911NF-12-1-0385. Part of the workwas done while the first author was a summer intern at Alcatel-Lucent Bell Labs.

REFERENCES

[1] C. Smith, “By the Numbers: 20 Amaz-ing Netflix Statistics and Facts (May 2014),”http://expandedramblings.com/index.php/netflixstatistics-facts/#.U4D7My92XsI, May 2014.

[2] Sandvine, “Global Internet Phenomena Report, Spring 2011,” 2011.

[3] A. Cockroft, C. Hicks, and G. Orzell, “Lessons Netflix Learned fromthe AWS Outage,” Netflix Techblog, 2011.

[4] “Microsoft Silverlight,” http://www.microsoft.com/silverlight/.

[5] V. K. Adhikari, Y. Guo, F. Hao, M. Varvello, V. Hilt, M. Steiner, andZ.-L. Zhang, “Unreeling Netflix: Understanding and Improving Multi-CDN Movie Delivery,” in INFOCOM, 2012.

[6] V. K. Adhikari, Y. Guo, F. Hao, V. Hilt, and Z.-L. Zhang, “Atale ofthree CDNs: An active measurement study of Hulu and its CDNs,” in15th IEEE Global Internet Symposium, Orlando, FL, USA, 2012.

[7] L. Daigle, “WHOIS Protocol Specification,” 2004.

[8] “Tamper Data,” addons.mozilla.org/en-US/firefox/addon/tamper-data.

[9] “Dummynet,” http://info.iet.unipi.it/ luigi/dummynet/.

[10] A. Cockroft, “Netflix Cloud Architecture,” Velocity conference, 2011.

[11] “Amazon Web Services,” http://aws.amazon.com.

[12] R. Pantoset al., “http://tools.ietf.org/id/draft-pantos-http-live-streaming-06.txt.”

[13] “get-flash-videos, A command line program to download flash videos,”http://code.google.com/p/get-flash-videos/, 2011.

[14] S. Akhshabi, A. C. Begen, and C. Dovrolis, “An experimental evaluationof rate-adaptation algorithms in adaptive streaming over http,” inMMSys, 2011.

[15] T.-Y. Huang, N. Handigol, B. Heller, N. McKeown, and R. Johari,“Confused, timid, and unstable: picking a video streaming rate is hard,”in IMC, 2012.

[16] D. K. Krishnappa, S. Khemmarat, L. Gao, and M. Zink, “On the feasi-bility of prefetching and caching for online tv services: a measurementstudy on hulu,” inProceedings of the 12th international conference onPassive and active measurement, ser. PAM’11. Berlin, Heidelberg:Springer-Verlag, 2011, pp. 72–80.

[17] V. K. Adhikari, S. Jain, Y. Chen, and Z.-L. Zhang, “VivisectingYouTube: An Active Measurement Study,” inINFOCOM’12 Mini-conference, 2012.

[18] X. Liu, F. Dobrian, H. Milner, J. Jiang, V. Sekar, I. Stoica, andH. Zhang, “A case for a coordinated internet-scale video control plane,”in SIGCOMM, 2012, pp. 359–370.

[19] A.-J. Suet al., “Drafting behind akamai: inferring network conditionsbased on cdn redirections,”IEEE/ACM Trans. Netw., 2009.

[20] C. Huang, A. Wang, J. Li, and K. W. Ross, “Understanding hybrid cdn-p2p: why limelight needs its own red swoosh,” inNOSSDAV, 2008.

14

[21] A. Downey, “Using pathchar to estimate internet link characteristics,”in ACM SIGCOMM CCR, 1999.

[22] M. Jain and C. Dovrolis, “ Pathload: A Measurement Tool for End-to-End Available Bandwidth ,” inPAM’02, March 2002, pp. 14–25.

[23] D. Croce, T. En-Najjary, G. Urvoy-Keller, and E. W. Biersack, “Fastavailable bandwidth sampling for adsl links: Rethinking the estimationfor larger-scale measurements,” inPAM ’09, 2009, pp. 67–76.

[24] “Netflix Open Connect Content Delivery Network,”https://signup.netflix.com/openconnect.

Vijay Kumar Adhikari obtained his PhD in Com-puter Science from University of Minnesota in 2012.His primary research are is large-scale content distri-bution. He is currently a Program Manager at SQLDB team in Microsoft.

Yang Guo is a Member of Technical Staff at BellLabs Research (Crawford Hill, New Jersey). Prior tothe current position, he was a Principal Scientist atTechnicolor (formerly Thomson) Corporate Researchat Princeton, NJ. He obtained Ph.D. from Universityof Massachusetts at Amherst, and B.S. and M.S.from Shanghai Jiao Tong University. His researchinterests span broadly over the distributed systemsand networks, with a focus on Cloud computing,Software Defined Networks (SDN) and its security,and Internet content distribution systems.

Fang Hao received the B.S. degree in computer science from TsinghuaUniversity, Beijing, China, and the Ph.D. degree in computer science fromthe Georgia Institute of Technology, Atlanta, GA. He is currently a Memberof Technical Staff with Bell Labs Research, Alcatel-Lucent, Holmdel, NJ.His research interests include software defined networks, cloud computing,routing, traffic engineering, and network monitoring. He has published about30 papers and 20 patents for his work in networking systems, algorithms andprotocols.

Volker Hilt Dr. Volker Hilt is leading the IP Plat-forms Research group at Bell Labs/Alcatel-Lucent inStuttgart, Germany with a focus on cloud computingand IP communication.

Dr. Hilt received his M.S. in Commercial Infor-mation Technologies in 1996 and his Ph.D. in Com-puter Science in 2001 both from the University ofMannheim in Germany. He joined Bell Labs/Alcatel-Lucent in Holmdel, NJ in 2002 and moved toStuttgart, Germany in 2012.

His field of research is computer networkingwhere he has made contributions in the areas of cloud computing technologies,distributed multimedia systems, the Session Initiation Protocol (SIP), contentdistribution networks and peer-to-peer applications. Dr.Hilt is a contributor tothe Internet Engineering Task Force (IETF) and has published over 50 papersand co-authored more than 15 Internet drafts and RFCs.

Zhi-li Zhang received the B.S. degree in computerscience from Nanjing University, China, in 1986 andhis M.S. and Ph.D. degrees in computer sciencefrom the University of Massachusetts in 1992 and1997. In 1997 he joined the Computer Science andEngineering faculty at the University of Minnesota,where he is currently a Qwest Chair Professor andDistinguished McKnight University Professor. Dr.Zhang’s research interests lie broadly in computercommunication and networks, Internet technology,multimedia and emerging applications. He has co-

chaired several conferences/workshops including IEEE INFOCOM’06 andserved on the TPC of numerous conferences/workshops. Dr. Zhang is co-recipient of four Best Paper Awards and has received a numberof other awards.He is a member of ACM and a Fellow of IEEE.

Matteo Varvello received the Research Mastersdegree in network and distributed systems fromEURECOM, Sophia Antipolis, France, in 2006; theM.Sc. (cum laude) degree in networking engineeringfrom Polytechnic of Turin, Turin, Italy, in November2006; and the Ph.D. degree in computer sciencefrom Telecom Paris, Paris France, in December 2009.He has been a Member of Technical Staff at Bell-Labs (Holmdel, New Jersey) since January 2010. Hiscurrent research interests include software-definednetworking, content-centric networking and high-

speed packet classification.

Moritz Steiner received the M.S. degree (Diplom) incomputer science from the University of Mannheim,Germany in 2005, and the Ph.D. degree in computernetworks from Telecom ParisTech, France and theUniversity of Mannheim in 2008. His doctoral thesisfocused on P2P based network virtual environments,and large scale P2P network measurement study ofKad. Moritz was a Member of Technical Staff atBell Labs located in Murray Hill, NJ from 2009 to2013, with research interests in the areas of softwaredefined networks, peer-to-peer networks and cloud

computing. In August 2013 Moritz joined the Web Experience Foundry teamat Akamai, San Francisco, CA, the cutting edge arm of Web Experiencebusiness unit. He participates in the overall web experience product missionby exploring new and adjacent technology opportunities.