PerformanceComparison: IPv4andIPv6...IPv6ConnecQonFailureRatebyOriginAS AS...

Performance Comparison: IPv4 and IPv6

Lee Howard Geoff Huston

John Brzozowski Chris Donley

February 2014

Geoff Huston, APNIC

The Environment

We use Google Ads to deliver test scripts to a very broad cross-‐secQon of Internet Users

–  We use a script that requests dual-‐stack end clients to fetch unique V6 and a V4 URLs from our servers

(servers located in the US, Germany and Australia)

–  We have some 300,000 -‐ 400,000 ads delivered per day –  We packet dump all acQvity on the server

ConnecQon Reliability

Looking at the TCP handshake, what proporQon of IPv6 clients send us a SYN, but no following ACK?

SYN

SYN+ACK

ACK


IPv4 failure rate: 0.1% -‐ 0.3% IPv6 failure rate: 1.2 – 2.1% This appears to indicate that a visible proporQon of IPv6-‐capable end user devices are located behind firewall/filter setups that deny incoming IPv6 packets

IPv6 ConnecQon Failure Rate by Origin AS

AS Samples Failure Rate (%) AS Name 14210 22 95.45 EDGECAST-‐DCA -‐ EdgeCast Networks, Inc. US

132497 295 78.64 DNA-‐AS-‐AP SMARTLINK BROADBAND SERVICES PVT LTD IN

55680 53 75.47 KSI-‐UAJY-‐AS-‐ID Kantor Sistem Informasi Universitas Atma Jaya Yogyakarta ID

198864 341 75.37 QMW-‐AC-‐UK Queen Mary and Wesiield College, University of London GB

18106 45 73.33 VIEWQWEST-‐SG-‐AP Viewqwest Pte Ltd SG

25592 51 62.75 NETIS-‐AS NETIS TELECOM Inc. Yaroslavl region ISP provider Russia RU

2856 27 59.26 BT-‐UK-‐AS BTnet UK Regional network GB

17660 141 58.16 DRUKNET-‐AS DrukNet ISP BT

22773 33 57.58 ASN-‐CXA-‐ALL-‐CCI-‐22773-‐RDC -‐ Cox CommunicaQons Inc. US

17705 37 51.35 INSPIRENET-‐AS-‐AP InSPire Net Ltd NZ

55947 128 46.09 BBNL-‐IN Bangalore Broadband Network Pvt Ltd IN

4755 175 43.43 TATACOMM-‐AS TATA CommunicaQons formerly VSNL is Leading ISP IN

278 45 40.00 Universidad Nacional Autonoma de Mexico MX

53187 28 35.71 UNIVERSIDADE ESTADUAL DE CAMPINAS BR

23148 108 27.78 TERREMARK Terremark US

1930 4808 25.21 RCCN Rede Ciencia Tecnologia e Sociedade (RCTS) PT

10429 60 25.00 Telefonica Data S.A. BR

17832 73 23.29 SIXNGIX-‐AS-‐KR Korea Internet Security Agency KR

45133 39 23.08 SINGAPORE-‐POLYTECHNIC-‐AS-‐AP Singapore Polytechnic SG

21366 50 22.00 KYMP KYMP OY FI

This table looks at the origin AS’s with more than 20 sample points – these are the origin AS’s with the highest failure rates

IPv6 ConnecQon Failure Rate by Origin AS

AS Samples Failure Rate (%) AS Name 786 458,143 1.22 JANET JISC CollecQons And Janet Limited GB 15169 106,221 0.08 GOOGLE -‐ Google Inc. US 8708 75,393 5.70 RCS-‐RDS RCS & RDS SA RO 7922 57,659 1.84 COMCAST-‐7922 -‐ Comcast Cable CommunicaQons, Inc. US 7018 50,489 1.22 ATT-‐INTERNET4 -‐ AT&T Services, Inc. US 2516 50,358 0.16 KDDI KDDI CORPORATION JP 12322 38,125 5.02 PROXAD Free SAS FR 6147 32,512 1.75 Telefonica del Peru S.A.A. PE 22394 27,237 0.34 CELLCO -‐ Cellco Partnership DBA Verizon Wireless US 23910 23,372 2.98 CNGI-‐CERNET2-‐AS-‐AP China Next GeneraQon Internet CERNET2 CN 4739 19,398 0.77 INTERNODE-‐AS Internode Pty Ltd AU 3303 17,319 4.45 SWISSCOM Swisscom (Switzerland) Ltd CH 4773 13,963 0.57 MOBILEONELTD-‐AS-‐AP MobileOne Ltd. Internet Service Provider Singapore SG 19782 8,172 0.06 INDIANAGIGAPOP -‐ Indiana University US 7575 7,863 3.84 AARNET-‐AS-‐AP Australian Academic and Reasearch Network (AARNet) AU

This table looks at the origin AS’s where we have the highest number of samples

IPv6 ConnecQon Failure Rate for US + CA Nets

AS Samples Failure Rate (%) AS Name

14210 22 95.45 EDGECAST-‐DCA -‐ EdgeCast Networks, Inc. US 22773 33 57.58 ASN-‐CXA-‐ALL-‐CCI-‐22773-‐RDC -‐ Cox CommunicaQons Inc. US

23148 108 27.78 TERREMARK Terremark US 237 265 21.13 MERIT-‐AS-‐14 -‐ Merit Network Inc. US

5707 24 20.83 UTHSC-‐H -‐ The University of Texas Health Science Center at Houston US

812 1139 17.12 ROGERS-‐CABLE -‐ Rogers Cable CommunicaQons Inc. CA 11351 471 15.50 RR-‐NYSREGION-‐ASN-‐01 -‐ Time Warner Cable Internet LLC US

2055 325 14.46 LSU-‐1 -‐ Louisiana State University US

174 25 12.00 COGENT Cogent/PSI US 12271 1850 11.24 SCRR-‐12271 -‐ Time Warner Cable Internet LLC US

11426 209 9.09 SCRR-‐11426 -‐ Time Warner Cable Internet LLC US 1312 527 8.54 VA-‐TECH-‐AS -‐ Virginia Polytechnic InsQtute and State Univ. US

10796 669 7.77 SCRR-‐10796 -‐ Time Warner Cable Internet LLC US

1280 97 6.19 ISC-‐AS1280 Internet Systems ConsorQum, Inc. US 11427 2102 4.52 SCRR-‐11427 -‐ Time Warner Cable Internet LLC US

3356 316 4.43 LEVEL3 Level 3 CommunicaQons US 12222 24 4.17 AS12222 Akamai Technologies US

20001 4505 3.26 ROADRUNNER-‐WEST -‐ Time Warner Cable Internet LLC US

6939 4592 3.11 HURRICANE -‐ Hurricane Electric, Inc. US 33522 37 2.70 CPANEL-‐INC -‐ cPanel, Inc. US

RTT Measurements

Data performance is highly dependent on the RTT across the data connecQon Is IPv6 faster or slower than IPv4 in terms of an RTT comparison?

RTT EsQmate

Measuring the Qme interval for the compleQon of the TCP handshake

SYN

SYN+ACK

ACK

RTT Interval

Paired RTT DistribuQon

Take the TCP handshake and measure the elapsed Qme at the server between the iniQal SYN and the following ACK packet

–  this Qme value is an indicator of the RTT

Take the measurements where we have web log evidence that the IPv4 and IPv6 addresses correspond to a single experiment, and generate a raQo of the two RTT values


IPv6 is faster than IPv4 IPv4 is faster than IPv6

Then: July 2012

% of R

TT m

easuremen

ts

3x 3x 2x 2x 1.5x 1.5x =

RTT RaQo


IPv6 is faster than IPv4 IPv4 is faster than IPv6

Then: July 2012

Now: December 2013

% of R

TT m

easuremen

ts

3x 3x 2x 2x 1.5x 1.5x =

RTT RaQo

RTT DistribuQon

•  There is a slight change in the RTT distribuQons over the past 12 months favoring IPv6 being slightly faster than IPv4 –  This could be due to different network paths between IPv4 and IPv6 –  Or less deployment of port 80 trapping middleware in IPv6 as compared to IPv4

Lee Howard

Performance Dashboard

IPv4 Average 39.3ms IPv6 Average 33.4ms Difference 15%

Avg speed to site Avg speed from hub:

Green = IPv6 >30% faster Pink = IPv6 >30% slower

atl20-‐tbp1 chi10-‐tbp1 chi30-‐tbp1 clt01-‐tbp1 dca10-‐tbp1 dca20-‐tbp1 dfw10-‐tbp1

hou30-‐tbp1 lax00-‐tbp1 lax30-‐tbp1 nyc20-‐tbp1 nyc30-‐tbp1 sea20-‐tbp1 sjc10-‐tbp1 sjc30-‐tbp1 IPv4

Avg IPv6 Avg

A 14.2 12.3 17.7 8.7 17.4 7.9 7.9 7.7 1.8 3.6 2.4 4.2 9.2 0.7 14.9 4.9 1.2 7.8 5 9.2 3.1 9.2 2.6 8.5 27.1 38.1 9.2 19.6 12.1 21 9.72 10.89

B 11.9 12.6 29.7 27.1 11.1 7.8 0.8 1.1 5.9 1.7 7 11.2 20.1 14 9.4 7.3 8.6 6.8 1.1 2.5 3.9 10.95 6.22

C 14.8 15.6 179.9 181.5 176.1 183.4 16.1 16.5 9.6 10 10 10.8 47.9 35.8 48 39.8 83 30.6 82.8 32.5 15.6 15.3 14.9 15.3 98.8 53.7 82.8 22.8 83.4 23.3 64.25 45.79

D 14.9 15.4 179.7 181.8 176.2 183.6 16.2 16.6 9.9 9.9 10.2 10.6 47.9 36.2 47.9 39.3 83.1 30.6 82.7 32.5 15.8 16 15.2 15.2 98.8 53.7 82.8 22.8 83.4 23.3 64.31 45.83

E 7.8 6.4 32.6 38.2 32.5 37.8 15.8 15.4 12.2 13.7 10.1 38.8 17.1 18.1 21.1 21.5 0.6 0.8 2.9 3 10.3 6.1 5.7 6.7 19.7 25.6 11.1 11.6 12.1 12.3 14.11 17.07

F 15 15.9 41.7 35.6 35.9 31.5 8.5 9.4 1.7 3.1 2.5 4.1 47.6 30.8 47.9 34.7 66.8 9.1 66.5 16.3 15.8 16.1 15.1 16.5 93.1 61.1 77 17.4 77.4 18 40.83 21.31

G 15.1 16.3 177.1 182.4 173.8 183.9 10.7 11.8 4.3 5.6 4.9 6 43.4 24.8 43.5 30.9 66.8 8.9 66.4 16.5 11.3 11.4 9.7 11.2 90.9 65.5 74.9 15.8 75.4 17.6 57.88 40.57

H 16.3 15.1 48.3 44 48.3 40.2 10.7 9.9 7.5 2.9 4.3 22.2 83.1 63 85.7 68.2 60.8 39.5 43 38.7 10 8.5 9.5 8.9 67.8 56.7 48.9 37.8 50.5 38.9 39.65 32.97

I 15.1 16.3 28.5 27.5 25.2 25.2 10.4 10 3.7 1.5 6.6 6.1 50.2 47.8 47.9 47.3 80.7 65.5 79.9 79.8 10.9 10.8 9.7 7.8 82.1 80.7 77.1 75.8 76.6 74 40.31 38.41

J 39.3 71.4 41.5 39.2 43.1 38.6 35 61.8 27.4 58.5 27 65.9 57.7 59.8 55.6 64.4 34.6 69.5 36.3 84 16.7 52.7 17.7 52.9 11.3 61 27.9 73.9 30 63.4 33.41 61.13

K 14.9 15.1 177 182 173.9 183.5 8.4 16.5 1.7 9.9 2.5 11.1 43.5 36.9 43.5 38.9 66.8 30.7 66.4 32.7 8 15.6 7.6 15.5 90.9 53.9 74.9 22.8 75.4 23.3 57.03 45.89

Average 16.3 19.3 86.7 92.1 84.5 91.6 13.7 16.7 7.3 10.9 7.9 16.5 41.3 35.4 42.5 39.0 51.3 29.3 49.6 34.5 11.5 15.4 10.6 15.0 62.0 55.0 51.7 32.0 52.7 31.5 64.3 47.4

95th percen3le: 16.3 16.3 179.7

182.0 176.1 183.6 16.2 16.6 12.2 13.7 10.2 38.8 57.7 60.1 55.6 64.8 83.0 65.9 82.7 80.2 15.8 16.1 15.2 16.5 98.8 67.0 82.8 74.1 83.4 64.5 64.3 47.4

lax30-‐tbp1 nyc20-‐tbp1

IPv4 IPv6 IPv4 IPv6 A 5 9.2 3.1 9.2 B 14 9.4 7.3 C 82.8 32.5 15.6 15.3 D 82.7 32.5 15.8 16 E 2.9 3 10.3 6.1 F 66.5 16.3 15.8 16.1 G 66.4 16.5 11.3 11.4 H 43 38.7 10 8.5 I 79.9 79.8 10.9 10.8 J 36.3 84 16.7 52.7 K 66.4 32.7 8 15.6 Average 49.6 34.5 11.5 15.4 95th %ile 82.7 80.2 15.8 16.1

Performance Dashboard (extract)

Performance Dashboard: Hop Count

atl20-‐tbp1 chi10-‐tbp1 chi30-‐tbp1 clt01-‐tbp1 dca10-‐tbp1 dca20-‐tbp1 dfw10-‐tbp1 hou30-‐tbp1 lax00-‐tbp1 lax30-‐tbp1 nyc20-‐tbp1 nyc30-‐tbp1 sea20-‐tbp1 sjc10-‐tbp1 sjc30-‐tbp1

A 4.8 5 6.8 4.3 5 4.5 3.6 5 3.6 4 4 4 3.7 4 3.6 4 2.6 4.3 3.3 3.2 3.9 2.6 3.9 4 5.4 6.4 4.4 5.5 4.4 5.5

B 8 8 10 8 10 8.9 10 5.8 8 9.8 10 4.9 10 8 10 9.1 9.4 9.8 9 9.8 9 9 9 10.9 10 8 9 13.6 10 14.4

C 8 8 10 8 10 8.9 10 6.2 8 9.8 10 4.5 10 8 10 9.2 9.4 9.8 9 9.8 9 9 9 10.9 10 8.1 9 13.6 10 14.4

D 9.3 8.6 9 6.9 8 6.9 12.9 10.4 12.8 8.7 11 8.5 6 6 7 6.9 6 6 7.6 6.9 11.3 5.6 9.4 7.5 11 9 10 7 11 7.9

E 9 9.9 7.1 6.1 7.1 7 7.1 5.6 5 7.9 7 3.5 8.6 7.1 8.7 8 8.2 9.2 6.5 9.2 9.4 9 9.4 10.9 9.7 10.6 6.7 13.6 7.7 14.5

F 9 10 7.3 6.2 7.3 7 7.1 5.3 5 7.9 7 3.6 7 6 7 6.9 7.6 7.9 6 7.9 10 9 10 10.9 11 11 8 13.6 9 14.4

G 11 7.9 10 7.9 10 7.9 11.9 8.9 10 6.9 12 6.9 13 12 13 13 12.6 11 11 9.1 10 6 10 7.9 8 7 13.6 10 13.6 11

H 4 6 5 6.4 6 7.3 5 6 5 4 3 3.3 5 8.9 6 9.7 8.6 9.3 7 10.7 4 4.4 4 6.4 7 8.4 5 6.4 6 7.4

J 11 6.1 9.8 7.3 10.8 8.1 12 7.8 12 6.1 10 4.6 12.8 8.3 13.8 9.2 12.8 10.6 11.2 10 8 4 11 6 8.3 8 10.2 10 11.2 10.8

K 9 8 7 8 7 8.8 7.1 5.6 5 9.8 7 4.2 7 8 7 9.1 7.6 9.8 6 9.8 10 9 10 10.9 11 8 8 13.6 9 14.4

Average 8.3 7.8 8.2 6.9 8.1 7.5 8.7 6.7 7.4 7.5 8.1 4.8 8.3 7.6 8.6 8.5 8.5 8.8 7.7 8.6 8.5 6.8 8.6 8.6 9.1 8.5 8.4 10.7 9.2 11.5 95th percen3le:

11.0 9.9 10.0 8.0 10.1 8.9 12.1 9.1 12.1 9.8 11.1 7.1 12.8 9.2 13.1 10.0 12.6 10.6 11.0 10.1 10.1 9.0 10.1 10.9 11.0 10.6 10.5 13.6 11.4 14.4

IPv4 Average 8.38

IPv6 Average 7.54

Difference 10%

Dashboard History: AvgRTT over 1 week

Dashboard History: Speed Download over 1 month

Conclusions

• Of the sites we’re measuring, IPv6 is generally faster •  ExcepQons:

– Sites that tunnel IPv6 over IPv4 – Hubs that are not closest exit

•  No conclusions about variability—IPv4 and IPv6 performance both vary

• More research needed

John Brzozowski

IPv6 Performance

John Jason Brzozowski, Comcast February 11, 2014

Methodologies

•  Background •  IniQal focus to test, validate, and audit IPv6 •  Goal is to ensure ideal customer experience as IPv6

deployment advances and evolves •  Various approaches measure and test key elements in an

isolated and integrated manner •  Variety of applicaQons, tools, and protocols used to

perform tesQng and gather •  Third party and internally developed leveraging well

known protocols -‐ HTTP, ICMP, DNS •  iPerf •  Speed test

25

Approaches

Experience Index

Broadband Network

Core Network

Internet

ObservaQons •  IPv6 performance across broadband ecosystem appears to lead that of IPv4 in some cases, specifically for broadband data

•  Factors include the chipsets and implementaQons, specifically related to home networking

•  Broadband access implementaQons are likely more recent and with simpler deployment models

•  Core network performance is largely even across the core in typical scenarios

•  Core to Internet adds variability •  PotenQal opportuniQes exist to opQmize, larger MTU for IPv6

27

ObservaQons (conQnued)

•  Internet ecosystem introduces relevant variables •  DNS lookup Qme, connected Qme, and transfer Qme are key metrics

•  CDN readiness and availability specific to IPv6 •  Observed to result in inefficient and/or slow content distribuQon

•  Many CDNs, however, have exactly the same performance over IPv6 and IPv4 •  Some have greater performance over IPv6

•  ImplementaQon quality for non-‐CDN hosted content varies greatly, which also impacts performance

28

ObservaQons (conQnued)

•  And… •  Average speed tests indicate that IPv6 out performs IPv4 naQonwide

•  Large sample size, all speed test servers are dual stack enabled •  Comcast has ~30% of customers acQvely provisioned with dual stack enabled broadband

•  Even with highs and lows dropped average IPv6 speed test is faster than IPv4

29

Next steps

•  ConcatenaQon of all scenarios seems yield posiQve performance differences

•  Increase and improve measurements from the customer’s premises

•  Core network and Internet oriented measurements

•  Consider customer premises variables •  Wired versus wireless •  Wi-‐Fi radio (2.4Ghz versus 5Ghz) •  Other network acQvity within the customer’s premises

•  IdenQfy opQmizaQon opportuniQes 30

John Jason Brzozowski [email protected]

Chris Donley

Is IPv6 faster than IPv4?

•  CableLabs has been conducQng IPv6 interops since 2009 –  Observed subtle but persistent v6 performance improvements

•  Conducted lab tesQng to measure v4/v6 performance –  NaQve IPv6 –  IPv4 with one layer of NAT –  IPv4 with two layers of NAT

Slide 34

Methodology •  Conducted matched v4-‐v6 ping tests over the course of an IPv6 interop –  First sets: 2 NATS enabled –  Second sets: 1 NAT enabled

•  IdenQfy effect of NAT without v4-‐v6 path differences

•  Did NOT measure effects of traffic engineering or differenQated peering

Slide 35

Test Server

Switch

CMTS

CM

Low-‐end

Router

Home Router

Test PC

Configurable NAT

NAT Enabled

Results – One Layer of NAT

•  Could NAT be responsible for slower v4 performance?

•  IPv6 performed measurably be{er than IPv4 –  Lower mean, median, standard deviaQon, and minimum

–  StaQsQcally significant differences

Common Scenario Today

Slide 36

IPv4 IPv6

Mean 12.02

9.39

Median 9.42 8.75

Standard Dev.

5.76 3.51

Max 29.79

46.17

Min 7.68 7.38

Samples 122 129

Results – Two Layers of NAT

•  WHAT! •  One NAT showed differences, two show nearly idenQcal results –  81% chance that the means are really the same

–  V4 in both cases sQll shows higher variability –  V6 performed be{er half the Qme across 6 test runs

Possible CGN effect

Slide 37

IPv4 IPv6

Mean 10.00

10.06

Median 9.48 9.65

Standard Dev.

2.03 1.74

Max 17.79

15.99

Min 7.90 7.60

Samples 112 111

Conclusions (?)

•  IPv4 NAT IN SOME ROUTERS does affect performance •  Your mileage may vary – NAT performance seems to vary across different plaiorms

•  Other network effects such as tunnels, traffic engineering, network congesQon are likely to produce larger effects

Slide 38

Conclusions

PerformanceComparison: IPv4andIPv6...IPv6ConnecQonFailureRatebyOriginAS AS...

Documents

Transcript of PerformanceComparison: IPv4andIPv6...IPv6ConnecQonFailureRatebyOriginAS AS...

Performance*Comparison:* IPv4*and*IPv6*...IPv6*ConnecQon*Failure*Rate*by*Origin*AS* AS...

Documents

Transcript of Performance*Comparison:* IPv4*and*IPv6*...IPv6*ConnecQon*Failure*Rate*by*Origin*AS* AS...

PerformanceComparison: IPv4andIPv6...IPv6ConnecQonFailureRatebyOriginAS AS...

Transcript of PerformanceComparison: IPv4andIPv6...IPv6ConnecQonFailureRatebyOriginAS AS...