Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points...

47
Making Sense of Measurement November 2016 BEREC, Brussels Nick Hilliard CTO [email protected]

Transcript of Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points...

Page 1: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Making Sense of Measurement

November 2016 BEREC, Brussels

NickHilliardCTO

[email protected]

Page 2: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

IXP Overview

inex

Page 3: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

IXP Overview

inex

founded in 1996

90 peering organisations

140gbit/s traffic peaks

6 points of presence

dual infrastructure

Page 4: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Things We Do

interconnection enablement

Page 5: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Things We Do

interconnection enablement

IXP Manager

Page 6: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Things We Do

interconnection enablement

IXP Manager

automation

Page 7: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Things We Do

interconnection enablement

IXP Manager

automation

efficiency

Page 8: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Things We Do

interconnection enablement

no politics

IXP Manager

automation

efficiency

Page 9: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Things We Do

interconnection enablement

no politics

IXP Manager

automation

efficiency

outreach

Page 10: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Things We Do

interconnection enablement

no politics

IXP Manager

automation

efficiency

outreachgood of the internet

Page 11: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important

throughput

Page 12: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important

throughput

latency

Page 13: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important

throughput

latency

capacity

Page 14: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important

throughput

latency

capacity

packet loss

Page 15: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important

throughput

latency

capacity

packet loss

Page 16: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important to Consumers

“the internet is slow”

Page 17: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important to Consumers

“the internet is slow”“I’m not getting what I

paid for”

Page 18: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important to Consumers

“the internet is slow”“I’m not getting what I

paid for”“I’m not getting what I

think I paid for”

Page 19: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important to Consumers

Page 20: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Perception

confirmation:

“the internet is slow”

Page 21: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - The Provider Position

“there is a difference

between what you paid for

and what you’re getting”

Page 22: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - What’s Important to Consumers

Page 23: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Last Mile Problems

Page 24: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Last Mile Problems

Page 25: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Last Mile Problems

Page 26: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

entire household

watching netflix

Page 27: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

entire household

watching netflix

windows update

Page 28: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

trojans

entire household

watching netflix

windows update

Page 29: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

trojans

ddos

entire household

watching netflix

windows update

Page 30: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

trojansspam

ddos

entire household

watching netflix

windows update

Page 31: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

bittorrent

trojansspam

ddos

entire household

watching netflix

windows update

Page 32: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

bittorrent

trojansspam

ddos

entire household

watching netflix

windows update

ancient wifi cpes

Page 33: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Customer Difficulties

bittorrent

trojansspam

ddos

entire household

watching netflix

windows update

ancient wifi cpesdelivery problems

Page 34: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Difficulties

Page 35: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Difficulties

Page 36: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Reconciling the Gap

consumer side problems account for huge performance degradation

Page 37: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Reconciling the Gap

consumer side problems account for huge performance degradation

online measurement results often contaminated by consumer problems

Page 38: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Reconciling the Gap

consumer side problems account for huge performance degradation

fttx / cable / xdsl / mobile

online measurement results often contaminated by consumer problems

Page 39: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Reconciling the Gap

consumer side problems account for huge performance degradation

fttx / cable / xdsl / mobile

latency and packet loss are easy

online measurement results often contaminated by consumer problems

Page 40: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Reconciling the Gap

consumer side problems account for huge performance degradation

fttx / cable / xdsl / mobile

latency and packet loss are easy

online measurement results often contaminated by consumer problems

throughput is difficult

Page 41: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Reconciling the Gap

consumer side problems account for huge performance degradation

fttx / cable / xdsl / mobile

latency and packet loss are easy

online measurement results often contaminated by consumer problems

throughput is difficult

higher speed throughput is much

more difficult

Page 42: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Takeaways

there is a chasm between accurate measurement and customer perception

Page 43: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Takeaways

there is a chasm between accurate measurement and customer perception

measurement is hard but do-able

Page 44: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Takeaways

there is a chasm between accurate measurement and customer perception

measurement is hard but do-able

different technologies require different approaches

Page 45: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Takeaways

there is a chasm between accurate measurement and customer perception

measurement is hard but do-able

different technologies require different approaches

careful scope definition is critical

Page 46: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.

Measurement - Takeaways

there is a chasm between accurate measurement and customer perception

measurement is hard but do-able

different technologies require different approaches

careful scope definition is critical

consumer problems usually dwarf interconnection problems

Page 47: Making Sense of Measurement - BEREC · 90 peering organisations 140gbit/s traffic peaks 6 points of presence dual infrastructure. ... ddos entire household hing netflix ws update.