An Untold Story of Middleboxes in Cellular...

33
An Untold Story of Middleboxes in Cellular Networks Zhaoguang Wang 1 Zhiyun Qian 1 , Qiang Xu 1 , Z. Morley Mao 1 , Ming Zhang 2 1 University of Michigan 2 Microsoft Research

Transcript of An Untold Story of Middleboxes in Cellular...

Page 1: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

An Untold Story of Middleboxes in Cellular Networks

Zhaoguang Wang1

Zhiyun Qian1, Qiang Xu1, Z. Morley Mao1, Ming Zhang2

1University of Michigan 2Microsoft Research

Page 2: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Background on cellular network

2

Internet Cellular Core Network

An untold story of middleboxes in cellular networks 8/18/2011

Page 3: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Why carriers deploy middleboxes?

3

Internet Cellular Core Network

Private IP Public IP

IP address

An untold story of middleboxes in cellular networks 8/18/2011

Page 4: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Problems with middleboxes

4

Internet Cellular Core Network

An untold story of middleboxes in cellular networks 8/18/2011

Policies ?

Application performance

?

P2P ?

Smartphone energy cost

?

Page 5: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Challenges and solutions

• Policies can be complex and proprietary

√ Design a suite of end-to-end probes

• Cellular carriers are diverse

√ Publicly available client Android app

• Implications of policies are not obvious

√ Conduct controlled experiments

5 An untold story of middleboxes in cellular networks 8/18/2011

Page 6: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Related work

• Internet middleboxes study

– [Allman, IMC 03], [Medina, IMC 04]

• NAT characterization and traversal

– STUN[MacDonald et al.], [Guha and Francis, IMC 05]

• Cellular network security

– [Serror et al., WiSe 06], [Traynor et al., Usenix Security 07]

• Cellular data network measurement

– WindRider, [Huang et al., MobiSys 10]

6 An untold story of middleboxes in cellular networks 8/18/2011

Page 7: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Goals

• Develop a tool that accurately infers the NAT and firewall policies in cellular networks

• Understand the impact and implications

– Application performance

– Energy consumption

– Network security

7 An untold story of middleboxes in cellular networks 8/18/2011

Page 8: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

The NetPiculet measurement system

8

Internet Cellular Core Network

NetPiculet Server

NetPiculet Client

NetPiculet Client

NetPiculet Client

NetPiculet Client

Policies…

An untold story of middleboxes in cellular networks 8/18/2011

Page 9: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Target policies in NetPiculet

9

Firewall

IP spoofing

TCP connection timeout

Out-of-order packet buffering

NAT

NAT mapping type

Endpoint filtering

TCP state tracking

Filtering response

Packet mangling

An untold story of middleboxes in cellular networks 8/18/2011

Page 10: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Target policies in NetPiculet

10

Firewall

IP spoofing

TCP connection timeout

Out-of-order packet buffering

NAT

NAT mapping type

Endpoint filtering

TCP state tracking

Filtering response

Packet mangling

An untold story of middleboxes in cellular networks 8/18/2011

Page 11: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Key findings

8/18/2011 An untold story of middleboxes in cellular networks 11

Firewall

Some carriers allow IP spoofing Create network vulnerability

Some carriers time out idle connections aggressively Drain batteries of smartphones

Some firewalls buffer out-of-order packet Degrade TCP performance

NAT One NAT mapping linearly increases port # with time Classified as random in previous work

Page 12: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Diverse carriers studied

• NetPiculet released in Jan. 2011

– 393 users from 107 cellular carriers in two weeks

12 An untold story of middleboxes in cellular networks 8/18/2011

91%

9%

UMTS

EVDO

43%

24%

19%

10%

2% 2%

Europe

Asia

North America

South America

Australia

Africa

Technology Continent

Page 13: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Outline

13

1 • IP spoofing

2 • TCP connection timeout

3 • TCP out-of-order buffering

4 •NAT mapping

An untold story of middleboxes in cellular networks 8/18/2011

Page 14: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Outline

14

1 • IP spoofing

2 • TCP connection timeout

3 • TCP out-of-order buffering

4 •NAT mapping

An untold story of middleboxes in cellular networks 8/18/2011

Page 15: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Why allowing IP spoofing is bad?

15

Internet Cellular Core Network

10.9.9.101

10.9.9.202

SRC_IP = 10.9.9.101 …

DST_IP = 10.9.9.101 …

DST_IP = 10.9.9.101 …

DST_IP = 10.9.9.101 …

DST_IP = 10.9.9.101 …

An untold story of middleboxes in cellular networks 8/18/2011

Page 16: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Test whether IP spoofing is allowed

8/18/2011 An untold story of middleboxes in cellular networks 16

Internet Cellular Core Network

NetPiculet Server

NetPiculet Client

Allow IP spoofing!

10.9.9.101

SRC_IP = 10.9.9.202 PAYLOAD = 10.9.9.101

Page 17: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

4 out of 60 carriers allow IP spoofing

17 An untold story of middleboxes in cellular networks 8/18/2011

7%

93%

Allow

Disallow

IP spoofing should be disabled

Page 18: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Outline

18

1 • IP spoofing

2 • TCP connection timeout

3 • TCP out-of-order buffering

4 •NAT mapping

An untold story of middleboxes in cellular networks 8/18/2011

Page 19: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Why short TCP timeout timers are bad?

19

Internet Cellular Core Network

KEEP-ALIVE KEEP-ALIVE KEEP-ALIVE Terminate Idle TCP Connection

An untold story of middleboxes in cellular networks 8/18/2011

Page 20: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

5min < Timer

Measure the TCP timeout timer

8/18/2011 An untold story of middleboxes in cellular networks 20

Internet Cellular Core Network

NetPiculet Server

NetPiculet Client

5min < Timer <

10min

Time = 0 Time = 5 min Time = 10 min

Is alive?

Yes!

Is alive?

Page 21: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Short timers identified in a few carriers

< 5 min 5%

5 - 10 min 10%

10 -20 min 8%

20 - 30 min 11%

> 30 min 66%

21

4 carriers set timers less than 5 minutes

An untold story of middleboxes in cellular networks 8/18/2011

Page 22: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Short timers drain your batteries

• Assume a long-lived TCP connection, a battery of 1350mAh

• How much battery on keep-alive messages in one day?

22

20%

5 min

An untold story of middleboxes in cellular networks 8/18/2011

Page 23: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Outline

23

1 • IP spoofing

2 • TCP connection timeout

3 • TCP out-of-order buffering

4 •NAT mapping

An untold story of middleboxes in cellular networks 8/18/2011

Page 24: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

TCP out-of-order packet buffering

8/18/2011 An untold story of middleboxes in cellular networks 24

Internet Cellular Core Network

NetPiculet Server

NetPiculet Client

Buffering out-of-order

packets

Packet 1 Packet 2 Packet 3 Packet 4 Packet 5 Packet 6

Page 25: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Fast Retransmit cannot be triggered

25

1 2

Degrade TCP performance!

An untold story of middleboxes in cellular networks 8/18/2011

RTO

Page 26: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

TCP performance degradation

• Evaluation methodology – Emulate 3G environment using WiFi

– 400 ms RTT, loss rate 1%

26

+44%

Longer downloading

time

More energy consumption

8/18/2011

Page 27: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Outline

27

1 • IP spoofing

2 • TCP connection timeout

3 • TCP out-of-order buffering

4 •NAT mapping

An untold story of middleboxes in cellular networks 8/18/2011

Page 28: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

NAT mapping is critical for NAT traversal

28

A B

NAT 1 NAT 2 An untold story of middleboxes in cellular networks 8/18/2011

Use NAT mapping type for port prediction P2P

Page 29: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

What is NAT mapping type?

• NAT mapping type defines how the NAT assign external port to each connection

29 An untold story of middleboxes in cellular networks 8/18/2011

NAT

12 TCP connections

Page 30: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Behavior of a new NAT mapping type

30 An untold story of middleboxes in cellular networks 8/18/2011

• Creates TCP connections to the server with random intervals

• Record the observed source port on server

Treated as random by existing traversal techniques Thus impossible to predict port

NOT random! Port prediction is feasible

Page 31: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Lessons learned

8/18/2011 An untold story of middleboxes in cellular networks 31

Firewall

IP spoofing creates security vulnerability IP spoofing should be disabled

Small TCP timeout timers waste user device energy Timer should be longer than 30 minutes

Out-of-order packet buffering hurts TCP performance Consider interaction with application carefully

NAT One NAT mapping linearly increases port # with time Port prediction is feasible

Page 32: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

Conclusion

• We built NetPiculet, a tool that can accurately infer NAT and firewall policies in the cellular networks

• NetPiculet has been wildly deployed in hundreds of carriers around the world

• We demonstrated the negative impact of the network policies and make improvement suggestions

32 An untold story of middleboxes in cellular networks 8/18/2011

Page 33: An Untold Story of Middleboxes in Cellular Networksconferences.sigcomm.org/sigcomm/2011/slides/s374.pdf · 2011. 8. 22. · 8/18/2011 An untold story of middleboxes in cellular networks

33 An untold story of middleboxes in cellular networks 8/18/2011

[email protected]

http://mobiperf.com