5 PILLARS OF CLOUDINESS

wayne wallscloud evangelist

alex brandtsr developer

Wednesday, August 21, 13

FIVE PILLARS OF CLOUDINESS

1 2 3 4 5

Parallel Modular Horizontal Agile Secure

2

Wednesday, August 21, 13

Pillar #1:Parallel

Wednesday, August 21, 13

1MRecords

Answer

Node

4

Wednesday, August 21, 13

1MRecords

Answer

Node1-250K

Node251K-500K

Node501K-750K

Node751K-1M

5

Wednesday, August 21, 13

6

1MRecords

Answer

Node1-250K

Node251K-500K

Node501K-750K

Node751K-1M

Wednesday, August 21, 13

6

1MRecords

Answer

Node1-250K

Node251K-500K

Node501K-750K

Node751K-1M

Wednesday, August 21, 13

7

Wednesday, August 21, 13

DATA

CALCULATIONMAP

CALCULATIONREDUCE

DATA

CALC

D

C

R

D

D

C

R

D

D

C

R

D

SHUFFLE

DATA

7

Wednesday, August 21, 13

8

Wednesday, August 21, 13

Thread 1

Deadlock

9

Thread 2

Wednesday, August 21, 13

Thread 1

Deadlock

9

Thread 2

Wednesday, August 21, 13

File A

Thread 1

Deadlock

9

Thread 2

Wednesday, August 21, 13

File A

Thread 1

Deadlock

9

Thread 2

Wednesday, August 21, 13

File A File B

Thread 1

Deadlock

9

Thread 2

Wednesday, August 21, 13

File A File B

Thread 1

Deadlock

9

Thread 2

Wednesday, August 21, 13

File A File B

Thread 1

Deadlock

9

File B

Thread 2

Wednesday, August 21, 13

File A File B

Thread 1

Deadlock

9

File B

Thread 2

Wednesday, August 21, 13

File A File B

File A

Thread 1

Deadlock

9

File B

Thread 2

Wednesday, August 21, 13

File A File B

File A

Thread 1

Deadlock

9

File B

Thread 2

Wednesday, August 21, 13

File A File B

File A

Thread 1

Deadlock

9

X

File B

Thread 2

Wednesday, August 21, 13

File A File B

File A

Thread 1

Deadlock

9

X

File B

Thread 2

Wednesday, August 21, 13

File A File B

File A

Thread 1

Deadlock

9

X X

File B

Thread 2

Wednesday, August 21, 13

Solution? Scheduling

Y

XA

B

A

B

Y

X

1

2

3

4

10

Wednesday, August 21, 13

1M records

Answer

Node

1M records

Answer

Node1-250K

Node251K-500K

Node501K-750K

Node750K-1M

11

Wednesday, August 21, 13

1M records

Answer

Node

1M records

Answer

Node1-250K

Node251K-500K

Node501K-750K

Node750K-1M

11

Wednesday, August 21, 13

12

Wednesday, August 21, 13

13

Wednesday, August 21, 13

pillar #2: modular

Wednesday, August 21, 13

ONE PERSON BUILDS A CAR

15

Wednesday, August 21, 13

CHASSIS ENGINE BODY INTERIOR

16

Wednesday, August 21, 13

CHASSIS ENGINE BODY INTERIOR

17

Wednesday, August 21, 13

CHASSIS ENGINE BODY INTERIOR

18

Wednesday, August 21, 13

CHASSIS ENGINE BODY INTERIOR

SEATS FLOOR

DASHBOARD

GASAND

BRAKE PEDALS

18

Wednesday, August 21, 13

CHASSIS ENGINE BODY INTERIOR

19

Wednesday, August 21, 13

CHASSIS ENGINE BODY INTERIOR

20

Wednesday, August 21, 13

defect

CHASSIS ENGINE BODY INTERIOR

20

Wednesday, August 21, 13

defect defect

CHASSIS ENGINE BODY INTERIOR

20

Wednesday, August 21, 13

defect defect defect

CHASSIS ENGINE BODY INTERIOR

20

Wednesday, August 21, 13

defect defect defectdefectCHASSIS ENGINE BODY INTERIOR

20

Wednesday, August 21, 13

defect defect defectdefect LEMONCHASSIS ENGINE BODY INTERIOR

20

Wednesday, August 21, 13

Pillar #3:horizontal

Wednesday, August 21, 13

2 GB2 CORES

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

VERTICAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

VERTICAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

SHARED NOTHING THROUGH STATELESSNESS

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

SHARED NOTHING THROUGH STATELESSNESS

22

8 GB4 CORES

Wednesday, August 21, 13

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

2 GB2 CORES

VERTICAL

HORIZONTAL

SHARED NOTHING THROUGH STATELESSNESS

2 GB2 CORES

22

8 GB4 CORES

Wednesday, August 21, 13

23

Wednesday, August 21, 13

24

Wednesday, August 21, 13

ONE SERVER NEAR

CAPACITY

24

Wednesday, August 21, 13

ONE SERVER NEAR

CAPACITY

ADD ANOTHER SERVER

24

Wednesday, August 21, 13

ONE SERVER NEAR

CAPACITY

ADD ANOTHER SERVER

25

Wednesday, August 21, 13

1234 MESSAGE QUEUE

26

Wednesday, August 21, 13

1234MESSAGE QUEUE

27

Wednesday, August 21, 13

1

2

3

4

28

MESSAGE QUEUE

Wednesday, August 21, 13

1

2

3

4

28

56789101112 MESSAGE QUEUE

Wednesday, August 21, 13

29

5678 9101112MESSAGE QUEUE

1

2

3

4

Wednesday, August 21, 13

29

5678 9101112

ADD MORE SERVERSBASED ON

MESSAGE QUEUE

MESSAGE QUEUE

1

2

3

4

Wednesday, August 21, 13

MESSAGE QUEUE

30

5

6

7

8

9

10

11

12

13141516

ADD MORE SERVERSBASED ON

MESSAGE QUEUE

1

2

3

4

Wednesday, August 21, 13

31

5678 9101112

13141516

ADD MORE SERVERSBASED ON

MESSAGE QUEUE

1234

MESSAGE QUEUE

Wednesday, August 21, 13

32

5678 9101112

13141516

ADD MORE SERVERSBASED ON

MESSAGE QUEUE

1234

MESSAGE QUEUE

Wednesday, August 21, 13

33

5678 9101112

13141516

ADD MORE SERVERSBASED ON

MESSAGE QUEUE1234

MESSAGE QUEUE

Wednesday, August 21, 13

33

5678 9101112

13141516

ADD MORE SERVERSBASED ON

MESSAGE QUEUE1234

MESSAGE QUEUE

DECREASE SERVERSBASED ON

MESSAGE QUEUE

Wednesday, August 21, 13

MESSAGE QUEUE

34

5678 9101112

13

14

15

16

ADD MORE SERVERSBASED ON

MESSAGE QUEUE1234

DECREASE SERVERSBASED ON

MESSAGE QUEUE

Wednesday, August 21, 13

Pillar #4:agile

Wednesday, August 21, 13

36

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature A

Feature B

Feature C

37

Monday

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature A

Feature B

Feature C

38

Tuesday

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature A

Feature B

Feature C

Feature E

Feature D

38

Tuesday

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature AFeature BFeature C

Feature E

Feature G

39

Wednesday

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature AFeature BFeature C

Feature E

Feature D

Feature F

Feature G

39

Wednesday

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature A

Feature B

Feature C

Feature E

Feature D

Feature F

Feature G

40

Friday

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature A

Feature B

Feature C

Feature E

Feature D

Feature F

Feature G

Feature H

Feature I

40

Friday

Wednesday, August 21, 13

Features In Progress Blocked Tabled Testing Complete

Feature A

Feature B

Feature C

Feature EFeature D

Feature F

Feature G

Feature H

Feature I

41

Friday - WK 2

Wednesday, August 21, 13

57%

14%

29%

Failed Successful Challenged

49%

42%

9%

Waterfall Agile

Source: The CHAOS Manifesto, The Standish Group, 2012

42

Wednesday, August 21, 13

43

Continuous Integration/Continuous Delivery

Wednesday, August 21, 13

44

Quality

Wednesday, August 21, 13

45

Release Cycle

Wednesday, August 21, 13

IMAGE: blogs.bu.edu

46

Wednesday, August 21, 13

47

Lint Testing

Test Driven Development

Functional Testing

Black Box

Integration Testing

Unit Testing

Behavior Driven Development

White Box

Smoke Testing

Acceptance Testing

Wednesday, August 21, 13

ENSURES QUALITY

47

Lint Testing

Test Driven Development

Functional Testing

Black Box

Integration Testing

Unit Testing

Behavior Driven Development

White Box

Smoke Testing

Acceptance Testing

Wednesday, August 21, 13

48

Lint Testing

Test Driven Development

Functional Testing

Black Box

Integration Testing

Unit Testing

Behavior Driven Development

White Box

Smoke Testing

Acceptance Testing

Wednesday, August 21, 13

ENSURES CONFIDENCE

48

Lint Testing

Test Driven Development

Functional Testing

Black Box

Integration Testing

Unit Testing

Behavior Driven Development

White Box

Smoke Testing

Acceptance Testing

Wednesday, August 21, 13

Pillar #5:secure

Wednesday, August 21, 13

50

Wednesday, August 21, 13

51

PRESENTATION

APPLICATION

STORAGE

Wednesday, August 21, 13

51

PRESENTATION

APPLICATION

STORAGE

INTERFACESECURITYACROSS ALL LAYERS

Wednesday, August 21, 13

52

Wednesday, August 21, 13

53

Host-based Authentication

Wednesday, August 21, 13

54

User Authentication

Wednesday, August 21, 13

55

Hybrid Cloud

Wednesday, August 21, 13

56

Trustworthiness as an Organization

Wednesday, August 21, 13