optimizing heterogeneous networks

David ChuComputer Science DivisionEECS DepartmentUC Berkeley

Qualifying ExamUC Berkeley

16 October 2007

can a declarative approach help us cope with heterogeneous networks?

3

ken: approximate data collection(chu-icde06)

sdlib: sensor data library(chu-spots06)

dsn: declarative sensor networks(chu-sensys07)

optimizing heterogeneous networks

in the field: does it help?

optimizing for adaptability and competing incentives

completed futureongoing

X

X

4

outline

• declarative sensor networks

• motivation

• language & example programs

• feasibility assessment

• optimizing heterogeneous networks

• optimizing for adaptability and competing incentives

• deployment application

5

heterogeneous networks

• … are proliferating

• more demands on expert engineers

• sensornets as one example…

6

contextSensor Networks early experiences

7

motivation

programming sensor networks is difficult

building entire sensor systems is even harder

8

inspiration

data management network design

s e n s o r n e t w o r k s

9

inspiration : data management• declarative is widely used in data

management• relational databases• spreadsheets• abstract “what” from “how”

• (Sensor-Network-As-Database)• [MFH05], [YG02]

10

inspiration : network design• declarative is new idea in networking• compact• flexible• analyzable, optimizable• Internet Routing, Overlays built declaratively

• (the P2 project)

11

inspiration

data management network design

s e n s o r n e t w o r k s

( DSN )

12

what we did

• adapted declarative language

• wrote declarative examples

• built compiler & runtime for sensornets

13

brief language overview

Rule2:

Fact:

Rule1:

join don’t care

Built-ins:

implies

builtin(temperature, ‘TemperatureImplementingModule.c’).

bodyTwohead bodyOne

14

a full example : tree

D

S

C

D

Z

C2

SC1

15

working programs

geographic routing

tracking

localization

link estimator

multi-hop collection

tree routing

trickle, in-network data aggregation, beacon vector routing,pathDCS, convex hull fallback routing, right-hand rule fallback routing, …

evaluating gossip dissemination

source*borrowed picture

19

… from designer’s paper [LPC04] … DSN specification

10x6 topology

30x2 topology

evaluating tree-collection

2. collection messages flowtoward root

1. treeconstruction

*borrowed picture

21

evaluating tree-collection

messages sent

hop-counts

(similar performance)

22

lines of code

23

compiled size

TelosB mote code space = 48KB, data space = 10KB

24

live tracking demo (vldb06)

25

architectural flexibility

• dsn can…

• describe entire system stack• application + network + mac layers

• naturally expose abstractions

• freely mix and match with outside libraries

26

thoughts up to now…

• sensor networks→ data + communication

• several examples of functional programs

• feasible for today’s hardware platforms

• can explore variety of architectures…

27

[current work]

optimizing heterogeneous networks

28

outline• declarative sensor networks

• optimizing heterogeneous networks

• motivation

• dimensions for optimization

• execution space by example

• finding optimal

• optimizing for adaptability and competing incentives

• deployment application

29

the protocol problem

• above: workloads vary

• below: greater diversity of networks

• who is going to figure out the best protocol for your situation?

• protocol optimization has a long history [CDO97], [VLC88], [HA89],…

30

implications of declarative

• concise “what” programming

• 1 specification,N possible execution plans

ü

?

31

Planner

Environment- Workload- Network

characteristicsCost

criteria

Optimized execution strategy

Network protocol

specification

32

manually engineering solutions

• packet filtering• offload to resource-rich gateways

• content distribution networks• manual configuration of distribution points

• session state• explicitly choose stateless variants when critical

• quality of service• choose tradeoff between node state or packet state

• routing• domain-specific protocols e.g. DSR, 6lowpan

• distributed event detection• configure detector placement

designparameters:state,rendezvous?

34

toward automated solutions• identify possible executions of program

• search execution space for optimal plan

• but first some rough idea of what we’re after…

Planner

35

packet filter program

% forwarding rulemessage(@Next,Src,Dest,Load) :-

message(@Crt,Src,Dest,Load), nexthop(@Crt,Dest,Next).

% firewall rulereceive(@Crt,Src,Dest,Load) :- message(@Crt,Src,Dest,Load),

Crt == Dest, firewall(@Crt,Load).

% queryreceive(@Crt,Src,Dest,Load)?

recursive relation

base relation

non-recursive relation

36

d

f

ba

c

e

g

nexthopd b

nexthopd b

nexthopf d

nexthopb a

nexthopg c

nexthope d

base relation as table partitioned across nodes

37

d

f

ba

c

e

g

base relation as network graph

39

d

f

ba

c

e

g

firewall

message

receive

40M N

M N

M N

M

A

E

D

B

msg() facts

nexthop() facts

41

r

f

A

E

D

B

42

r

f

A

E

D

B

43

d

f

ba

c

e

g

8

9

10

1

29

firewall

message

receive

varying join selectivity shifts the optimal rendezvous

1: Meet-In-The-Middle (MiM) Rewrite

applications: packet filtering, event detection, CDNs

message

message

44

d

f

ba

c

e

g

firewall

messagemess age+ =

2: Query Scramble Rewrite

receive

HiResVideo EventDetectorFlag

applications: query scrambling, mitigating latency X bandwidth

45

d

f

ba

c

e

g

firewall

message

fire wall+=

receive

3: Semijoin Rewrite

application: semijoin

allowed options

allowedports

46

d

f

ba

c

e

g

firewall

message

receive

connections,

Two separate relations at endpoint.

3b: Traditional Join Placement

application: semijoin

47

d

f

ba

c

e

g

49

d

f

ba

c

e

g

4: Pullback Rewrite

application: alternate between DVR and SR

nexthopd b

nexthopd b

nexthopd bb a

nexthopd bb a

nexthopf d

nexthopb a

nexthopg c

nexthope d

nexthopg cd bb a

50

d

f

ba

c

e

g

4: Pullback Rewrite

message

application: alternate between DVR and SR

nexthopd bb a

nexthopg cd bb a

51

routing layer state proxying

Sensornet

Internetnexthop forwarding table

D

C A

B

source route to Ddistance vector routing

A: D via B

B: D via C

C: D via D C: D via D

4: Pullback Rewrite example

52

ABCL

M’

M

ABC

L’

M,L

M’,L’

ABCM

L

L’M’,L’

M,L

M’

M,L

M

5: SessionState Rewrite

regular

proxy

stateless

53

…

M

M

M

M’

M’

A

B

C

B

L

L’

5: SessionState Rewrite

application: pushing state into packets/proxies

54

…

A

B

C

B

M,L

M,L

M’,L’

M’,L’

M,L L0

5: SessionState Rewrite

application: pushing state into packets/proxies

55

…

M

M

M’

A

B

C

B

L

L’

M,L

M,L

M’,L’

5: SessionState Rewrite

application: pushing state into packets/proxies

prefetch read

defer write

56

summary of rewrite family

57

example from dsn paper:before rewrite% Sample temperature and initiate multihop sendtransmit (@Src, Temperature ) :− thermometer (@Src, Temperature ) , timer (@Src,

collectionTimer , Period ) .

% Prepare message for multihop transmissionmessage (@Src, Src , Dst , Data ) :− transmit (@Src, Data ) , nextHop (@Src, Dst , Next ,

Cost ) ˜ .

% Forward message to next hop parentmessage (@Next, Src , Dst , Data ) :− message (@Crt , Src , Dst , Data ) , nextHop (@Crt

, Dst , Next , Cost ) ˜ , Cr t != Dst .

% Receive when at destinationalert(@Crt , Src , Data ) :− message (@Crt , Src , Dst , Data ) , Crt==Dst, anomaly(@Crt,

Thresh), Data > Thresh.

2

1

3

4

recursive relation

base relation

non-recursive relation

58

message(),nexthop(),nexthop(),…, nexthop(),…, nexthop(),anomaly()

rendezvous()

alert()

59

example from DSN paper:after rewrite% Sample temperature and initiate multihop sendtransmit (@Src, Temperature ) :− thermometer (@Src, Temperature ) , timer (@Src, collectionTimer , Period ) .

% Prepare message for multihop transmissionmessage (@Src, Src , Dst , Data ) :− transmit (@Src, Data ) , nextHop (@Src, Dst , Next , Cost ) ˜ .

% Forward message to next hop parentmessage (@Next, Src , Dst , Data ) :− message (@Crt , Src , Dst , Data ) , nextHop (@Crt, Dst , Next , Cost ) ˜ , Cr t != Dst, -

rendezvous’(@Crt).

% Receive when at destination optimal (middle) pointalert’(@Crt , Crt_, Src , Data ) :− message (@Crt , Src , Dst , Data ) , Crt==Dst, anomaly’(@Crt, Crt_, Thresh_), Data > Thresh,

rendezvous (@Crt).

% Move anomaly’() backwardanomaly’(@Crt, Crt, Thresh) :- anomaly(@Crt,Thresh).anomaly’(@Prev, Crt, Thresh) :- anomaly(@Crt, Crt_, Thresh), nextHop(@Prev, Crt_, Crt, Cost), -rendezvous (@Crt).

% Auxilliary rules to move alert’() to alert()alert’(@Next, Crt_, Src, Data) :- alert’(Crt, Crt_, Src, Data), nextHop(@Crt, Crt_, Next, Cost).alert(@Crt, Src, Data) :- alert’(@Crt, Crt, Src, Data).

2

1

3

4

5

6

rendezvous

rendezvous rendezvou

s

60

finding optimal

• goal: generate optimal rendezvous facts

• get statistics for both workload and resources• synopses: well-understood database summarization

technique• relevant statistics: join selectivity, size of tables, nexthop

distribution, link costs, storage costs

• run optimization• currently custom optimizer algorithms for each rewrite

(rewrites are still general)• should use general purpose dynamic programming optimizer• in both cases, algorithms are exhaustive

61

pullback rewrite: early resultssimulation35 nodes1 sender30 receivers1000 e2e sends

utilize available resource

account for workload

DVR inactive until enough table space for

all receivers.

62

planned experiment 1MiM for event detection

• setup• 30 node sensor network• all nodes send samples via collection tree to

root• root maintains list of anomaly thresholds

• hypothesis• MiM optimization lowers total energy

consumption by identifying optimal tree depth to which to push down anomaly thresholds

63

planned experiment 2SessionState for web server

• setup• 1 server, X clients in P2• each client connects with server with stateful

protocol

• hypothesis• SessionState optimization increases the

number of clients that can connect at overall lower messaging cost by giving “stateful” priority to more frequently connecting clients

status of current work

• captured design/execution space:• rendezvous and state

• identified ways to expose possible executions• MiM Family of Rewrites

• implemented first cut planner

• todo• implement generic rewriter• implement generic optimizer• prove semantic invariance w.r.t. queried relation• evaluate planner on test scenarios

64

65

[future work]

optimizing for adaptability and competing incentives

66

outline• declarative sensor networks

• optimizing heterogeneous networks

• optimizing for adaptability and competing incentives

• why adaptability?

• potential optimization for adaptability

• why competing incentives?

• potential optimization for competing incentives

• deployment application

67

optimize for best case performance?• environment changes quickly

optimizer run infrequently

• partially observable environment e.g. imperfect synopses

• delayed visibility of environment

• adaptability as theme of DB research in last decade [DIR07], [KD98], [IFF+99], [AH00]

68

options for adaptability

• iterating traditional optimizer more often [KD98]• PRO: natural port of algorithms and semantics• CON: state may still be obscured• CON: synopsis collection overhead increases

• run-time adaptability [IFF+99], [AH00]• PRO: continuous distributed planning• CON: unclear what global semantics are achieved

69

adaptive rewrite for latency

% Sample temperature and initiate multihop sendtransmit (@Src, Temperature ) :− thermometer (@Src, Temperature ) , timer (@Src, collectionTimer , Period ) .

% Prepare message for multihop transmissionmessage (@Src, Src , Dst , Data ) :− transmit (@Src, Data ) , nextHop (@Src, Dst , Next , Cost ) ˜ .

% Forward message to next hop parentmessage (@Next, Src , Dst , Data ) :− message (@Crt , Src , Dst , Data ) , nextHop (@Crt, Dst , Next , Cost ) ˜ , Cr t != Dst, -

anomaly’(@Crt, Crt_, Thresh_).

% Receive when at destination optimal (middle) pointalert’(@Crt , Crt_, Src , Data ) :− message (@Crt , Src , Dst , Data ) , Crt==Dst, anomaly’(@Crt, Crt_, Thresh_), Data > Thresh.

% Move anomaly’() backwardanomaly’(@Crt, Crt, Thresh) :- anomaly(@Crt,Thresh).anomaly’(@Prev, Crt, Thresh) :- anomaly(@Crt, Crt_, Thresh), nextHop(@Prev, Crt_, Crt, Cost),

-message(@Crt, Src, Crt_, Data).

% Auxilliary rules to move alert’() to alert()alert’(@Next, Crt_, Src, Data) :- alert’(Crt, Crt_, Src, Data), nextHop(@Crt, Crt_, Next, Cost).alert(@Crt, Src, Data) :- alert’(@Crt, Crt, Src, Data).

2

1

3

4

5

6

dynamic rendezvous

dynamic rendezvous

70

global query plan

m

n

f

r

J1 J2 J3

71

derivation graph

m

n

f

r

Original derivationAugmented derivationDashed represents negation

no unique model(multiple possible outcomes)is well-known

72

potential experiment 1

• setup• 30 node event detection sensor network (similar to

previously planned experiment)• use MiM rewrite in both “best-case” and “adaptible”

mode to find optimal rendezvous

• hypothesis• if anomaly list does not change frequently, “best-

case” performs better• if anomaly list does change frequently, “adaptible”

performs better

73

potential experiment 2

• setup• X clients, 1 server (similar to previously

planned experiment)• relax synopsis accuracy assumptions e.g. most

frequent clients (table distribution) not accurately reported

• hypothesis• as accuracy of synopses decrease, “adaptible”

performs better than “best-case”

74

optimizing for competing incentives• assumed single administrative domain

• sensornets• corporate CDNs• individual ASs

• competing incentives• privately-owned servers & hosts• ISP peering relationships• multiple sensornet queries

• individuals have incentive to deviate from global optimal [MRW+04], [SDK+94]

75

d

f

ba

c

e

g

8

9

10

1

29

firewall

message

receive

3: Semijoin Rewrite

connections,

Two separate relations at endpoint.

PINRELATION(connections)

76

d

f

ba

c

e

g

8

9

10

1

29

messageDomain 1

Domain 2

private plan 1

private plan 2

77

[application]

large scale and fine-graineddebris flow monitoring

78

outline

• declarative sensor networks

• optimizing heterogeneous networks

• optimizing for adaptability and competing incentives

• deployment application

• geological study

• declarative’s role

79

[Left] La Conchita, California – a small seaside community along Highway 101 south of Santa Barbara. This landslide and debris flow occurred in the spring of 1995. A reoccurrence in 2005 claimed 4 lives and resulted in 29 missing persons. [Right] Chehalis, Washington - landslides and debris flows during the winter storms of February 1996. Photographs by R.L. Schuster, U.S. Geological Survey.

80

[Above] The locations of the 2005-2006 and 2006-2007 debris flow deployment sites.[Top Right] Smoke from the Day Fire. [Middle Right] Recently burned hillside in Burbank, CA was the site of two debris flows in 2005-2006 Winter season. [Bottom Right] Base of the channel after debris flow with remaining sediment. [Bottom Left] Burn-resilient vegetation is quickly recovering just a few months after the fires and debris flows.

Harvard Burn Site

Day Fire

82

[Above] Parshall flume used in conjunction with water level logger at the channel’s choke-point. [Top Right] Custom overland flow sensor for fine-grained detection of water runoff. [Bottom Right] Solar-powered base station for actuating and gathering data from the wireless sensor network, shown here connected to laptop during testing.

83

declarative sensornetsin context

• how easy is managing sensornets with declarative programming?

• are the network optimizations useful in helping out a real deployment?• event detection• network management

84

timeline• Fa07:

• complete current work on Optimizing Heterogeneous Networks• implement generic rewriter• implement generic optimizations (concurrent work on optimization framework)• prove rewrite correctness• fully evaluate net planner

• help with USGS 1st deployment• continue engineering and testing hardware and software for 1st round deployment• deploy in selected site, help with issues as they arise

• Sp08+Fa08:• start and complete future work on Optimizing for Adaptability/Competing Incentives

• understand related work, decide on which course to pursue• design and implement generic rewrites• decide which platform(s) to use• augment runtime for runtime decision making

• help with USGS 2nd deployment

• Sp09:• write thesis

85

thanks

collaborators

Joe Hellerstein, Scott Shenker, Ion Stoica,

Lucian Popa, Arsalan Tavakoli

Tsung-Te Lai

Phil Levis, Jung Woo Lee, Aby John,Kevin Klues

Daniel Malmon, Joel Johnson

references• [MFH05] Samuel Madden, Michael J. Franklin, Joseph M. Hellerstein, Wei Hong.

TinyDB: an acquisitional query processing system for sensor networks. ACM Transactions on Database Systems (TODS), Volume 30 , Issue 1, March 2005.

• [YG02] Yong Yao, J. E. Gehrke . The Cougar Approach to In-Network Query Processing in Sensor Networks. Sigmod Record, Volume 31, Number 3. September 2002.

• [LPC04] Philip Levis, Neil Patel, David Culler, and Scott Shenker. Trickle: A Self-Regulating Algorithm for Code Propagation and Maintenance in Wireless Sensor Networks. In Proceedings of the First USENIX/ACM Symposium on Networked Systems Design and Implementation, 2004.

• [DIR07] Amol Deshpande, Zachary Ives, Vijayshankar Raman. Adaptive Query Processing. Foundations and Trends in Databases: Vol. 1: No 1, pp 1-140, 2007.

• [CDO97] C. Castelluccia, W. Dabbous, and S. O’Malley. Generating efficient protocol code from an abstract specification. IEEE/ACM Trans. Netw., 5(4):514–524, 1997.

• [VLC88] S. T. Vuong, A. C. Lau, and R. I. Chan. Semiautomatic implementation of protocols using an estelle-c compiler. IEEE Trans. Softw. Eng., 14(3):384–393, 1988.

• [HA89] D. Hernek and D. P. Anderson. Efficient automated protocol implementation using rtag. Technical Report UCB/CSD-89-526, EECS Department, University of California, Berkeley, Aug 1989.

references (2)• [KD98] N. Kabra and D. J. DeWitt. Efficient mid-query re-optimization of suboptimal

query execution plans. In SIGMOD ’98: Proceedings of the 1998 ACM SIGMOD international conference on Management of data, (New York, NY, USA), pp. 106–117, ACM Press, 1998.

• [IFF+99] Z. G. Ives, D. Florescu, M. Friedman, A. Levy, and D. S. Weld, “An adaptive query execution system for data integration,” in SIGMOD ’99: Proceedings of the 1999 ACM SIGMOD international conference on Management of data, (New York, NY, USA), pp. 299–310, ACM Press, 1999.

• [AH00] R. Avnur and J. M. Hellerstein, “Eddies: continuously adaptive query processing,” in SIGMOD ’00: Proceedings of the 2000 ACM SIGMOD international conference on Management of data, (New York, NY, USA), pp. 261–272, ACM Press, 2000.

• [MRW+04] R. Mahajan, M. Rodrig, D. Wetherall and J. Zahorjan. Experiences Applying Game Theory to System Design. SIGCOMM 2004 Workshop on Practice and Theory of Incentives and Game Theory in Networked Systems (PINS), Portland, OR, August 2004.

• [SDK+94] Michael Stonebraker, Robert Devine, Marcel Kornacker, Witold Litwin, Avi Pfeffer, Adam Sah, and Carl Staelin. An Economic Paradigm for Query Processing and Data Migration in Mariposa, Sequoia 2000 Technical Report 94/49, University of California, Berkeley, CA, Apr. 1994.

88

backup slides

93

evaluation

95

a declarative architecture• why rethink the architecture?

• disparate application requirements

• breaking of traditional abstraction boundaries

• what are the implications?

• architectural flexibility is essential

• put resource management in user’s hands

96

resource management

• memory

• processor

• energy

97

session state program

• % include “forwarding rule”

• % respond to message• msg(@Crt,Dest,Src,NewLoad) :-

msg(@Crt,Src,Dest,Load), Crt == Dest, NewLoad = modify_load(Load).

• % local state update• local(@Crt,NewState) :- msg(@Crt,Dest,Load),

Crt == Dest, local(State), NewState = modify_state(State,Load).

5: SessionState Rewrite