Welcome

Cloud Architectural Patterns (using Microsoft Azure tools)

Pushkar Chivate

Agenda• Quick Look: Microsoft Azure as a platform

• Overview of patterns for cloud applications

• Specifically Data-Management, Resiliency and Messaging patterns

• Demo • Azure Tables,

• DocumentDB,

• Azure Service bus (Queues and Pub-Sub architecture)

What is the Cloud?

IT

Capacity

Time

Load

Forecast

Allocated

IT-capacity

Under capacity

Over

capacity

Actual

Load

How the Cloud Helps?

IT

Capacity

Time

Load

Forecast

Actual

Load

Capacity on

Demand

Cloud ModelsIaaS Infrastructure as a service(e.g. Azure VMs)

PaaS Platform as a service(e.g. Azure Storage, Service Bus etc.)

SaaS Software as a Service(e.g. Office 365)

Microsoft Azure is a cloud platform

● On-Demand

● Self-service

● Pay per use

● Highly automated

● Designed for Failure

● Designed for Scale

Microsoft AzureDesigned for Failure

Network load balancer

VM1 VM2 VM3 VM4 VM5 VM6

VM7 VM8

Fault Domain

1

Fault Domain

2Fault Domain

3UD1

UD2

Microsoft AzureDesigned for Scale

Scale out

Network load balancer

VM1 VM2 VM3 VM4

Azure VMs, Websites&

Rethinking AppsA look at various features

Azure offers…

Architectural Pattern

An architectural pattern is a general, reusable solution to a commonly occurring problem within a given

context.

Cloud PatternsData-Management Patterns

● Index Table & Sharding PatternCreate indexes over the fields in data stores that are frequently referenced by query criteria. This pattern can improve query performance by allowing applications to more quickly retrieve data from a data store. Divide a data store into a set of horizontal partitions shards

● Materialized View PatternGenerate prepopulated views over the data in one or more data stores when the data is formatted in a way that does not favor the required query operations. This pattern can help to support efficient querying and data extraction, and improve performance

● Command and Query Responsibility Segregation (CQRS) PatternSegregate operations that read data from operations that update data by using separate interfaces.

Cloud PatternsMessaging Patterns

● Pipes and Filter PatternDecompose a task that performs complex processing into a series of discrete elements that can be reused.

● Priority Queue PatternPrioritize requests sent to services so that requests with a higher priority are

received and processed more quickly than those of a lower priority

.

● Queue-based Load Leveling PatternUse a queue that acts as a buffer between a task and a service that it invokes in order to smooth intermittent heavy loads that may otherwise cause the service to fail or the task to timeout.

Data StorageVarious data storage techniques

● SQL Azure

● Azure Table

● Blob storage

● Document DB

Materialized View PatternThis pattern can be used to support efficient querying and data extraction, and improve

application performance

What is a materialized view?

Materialized view is a database object that contains results of a query

Materialized View

OrderID Account

1 A

2 B

ItemID Name Stock

100 Shirts 100

101 Pants 125

OrderID ItemID Qty Ordered

1 100 2

1 101 3

2 100 2

Materialized View

Database

tables

ItemID Name Total Qty Ordered

100 Shirts 4

101 Pants 3

Denormalized data

Data refreshes

periodically

Materialized Views Vs Regular Views

Materialized View● Stored as a physical

database object

● Indexes on the base tables are not used , can create indexes on materialized view

● Faster performance

● Needs to be updated

Regular View● Stored as a query

against the base object

● Indexes on the base tables are used

● No performance improvement

● Need not be updated

Materialized view

When underlying data is complex and difficult to query

directly

Simplify queries - expose data in a way that doesn’t

require knowledge of underlying data structure

Provide access to specific subset of data (security

reasons)

Bridging the disjoint when using different data stores

When to use materialized

views?

Materialized viewSnapshot

s

Database

1

Table

1

Materialized view

1

Database

2

Table

1

Materialized view

2

Snapsho

t

Materialized

view

Databases supporting Materialized views

Oracle

Called Indexed Views

IBM DB2SQL

Server

MySQL

No Native support

Can be implemented

using triggers

Called Materialized Query

Table

Called Materialized View

Materialized view

When not to use materialized views?● The source data is simple and easy to query.

● The source data changes very quickly, or can be accessed without using a view. The processing overhead of creating views may be avoidable in these cases.

● Consistency is a high priority. The views may not always be fully consistent with the original data

Azure Tables

What are Azure Tables?

● Non relational entities (not RDBMS).

● Tables are nothing but collection of entities

● Entities are comprised of properties

● Properties are name value pairs

Azure TablesAzure Storage

Account

Employees

(Table)

Entity

Property

Entity

Property

Property PropertyProperty

Azure Table EntitiesAzure Storage

Account

Employees

(Table) Entity

Property

<Name,Value

>

Property

<Name,Value

>

Entities● Partition Key

● Row Key

● Timestamp

● … other properties

● Each entity can have different number and type of properties

Partition Key + Row Key = Unique Identifier

(No support for secondary index in Azure Table)

Property

<Name,Value

>

Azure Tables Partitions

Employees (Table) Certifications (Table)

(Entity)

EmployeeName=…CertName=…BirthDate=…

CertNumber=…

EmployeeName=…CertName=…BirthDate=…

CertNumber=…

FavoriteTeam=…

(Entity)

CertName=…EmployeeName=…

CertNumber=…BirthDate=…

CertName=…EmployeeName=…

CertNumber=…BirthDate=…

PK = EmployeeName

RK = CertName

PK = CertName

RK = EmployeeName

How do you Partition the data?• EmployeeName or CertName?

Azure Tables Partitions

Employees (Table) Certifications (Table)

(PK) EmployeeName(E1), (RK) CertName (C1)(PK) EmployeeName(E1), (RK) CertName

(C2)

(PK) EmployeeName(E2), (RK) CertName

(C1)

(PK) EmployeeName(E2), (RK) CertName

(C3)

(PK) EmployeeName(E3), (RK) CertName

(C2)

(PK) EmployeeName(E3), (RK) CertName

(C3)

How do you Partition the data?• EmployeeName or CertName?

(PK) CertName (C1), (RK) EmployeeName(E1)

(PK) CertName (C1), (RK)

EmployeeName(E2)

(PK) CertName (C2), (RK)

EmployeeName(E1)

(PK) CertName (C2), (RK)

EmployeeName(E3)

(PK) CertName (C3), (RK)

EmployeeName(E2)

(PK) CertName (C3), (RK)

EmployeeName(E3)

O

R

Azure Tables

EmployeeName BirthDate FavoriteTeam

David Anderson 1/1/1970

Nancy Wilson 4/15/1965 Atlanta Falcons

John Doe April 1, 1989

No Fixed Schema for

entities

Employee table

Azure Table

Demo

Azure Tables

Summary

● No nice relational schema model that we get in RDBMS

● Easy to work with

● Built to scale

● Great pricing

DocumentDB• Azure Document DB - relatively new service

• It’s fully managed Document database as a service

• It stores the data in JSON

• The Azure Tables are extremely scalable and cheap, but if you start querying on any other attributes then you start running into problems.

• It’s massively scalable but fully query-able by all parts of JSON tree.

• Capacity units are 10GB in size and predictable reads and writes / second.

DocumentDB

DocumentDB

Demo

Azure Service Bus

● Secure messaging capabilities

● Enable loosely coupled solutions

● Achieve Publish-Subscribe scenarios

Tightly Coupled

Store Front

End

Store Back

End

Order

processing

Shipping

Loosely Coupled

Store Front

End

Store Back

End

Order

processing

Shipping

Order Queue

Loosely Coupled

Store Front

End

Store Back

End

Order

processingOrder Queue

Shipping

Service Bus – Queue

Demo

Publish - Subscribe

Store Front

End

Store Back

End

Order

processingOrder Queue

Order Printers

Subscription

1

Subscription

2

Shipping

Order

Papers

Service Bus – AMQP• Service Bus supports the Advanced Message Queueing Protocol (AMQP) 1.0.

• AMQP enables you to build cross-platform, hybrid applications using an open standard protocol

• Reliable: The AMQP 1.0 protocol allows messages to be exchanged with a range of reliability guarantees, from fire-and-forget to reliable, exactly-once acknowledged delivery.

• Using AMQP from .Net:Endpoint=sb://[namespace].servicebus.windows.net;SharedSecretIssuer=[issuer

name];SharedSecretValue=[issuer key];TransportType=Amqp

• Using AMQP from Java:# servicebus.properties - sample JNDI configuration# Register a ConnectionFactory in JNDI using the form:# connectionfactory.[jndi_name] = [ConnectionURL]connectionfactory.SBCF = amqps://[username]:[password]@[namespace].servicebus.windows.net

Service Bus – AMQP

Cloud PatternsResiliency Patterns

● Circuit Breaker PatternHandle faults that may take a variable amount of time to rectify when connecting to a remote service or resource. This pattern can improve the stability and resiliency of an application.

● Retry PatternEnable an application to handle temporary failures when connecting to a service or

network resource by transparently retrying the operation in the expectation that the failure is transient. This pattern can improve the stability of the application.

● Compensating Transaction PatternUndo the work performed by a series of steps, which together define an eventually consistent operation, if one or more of the operations fails. Operations that follow the eventual consistency model are commonly found in cloud-hosted applications that implement complex business processes and workflows.

SummaryWe Looked at…

- Azure as Infrastructure (VMs)

- Azure as Platform -Websites- Storage - Azure Tables, DocumentDB- Messaging – Service Bus, Queues, Topics, AMQP- Azure CDN

Summary

Network load balancer

VM1 VM2 VM3 VM4

SQL DatabaseAzure

TablesDocumentDB

Service Bus Queue

Subscription 1 Subscription 2

Worker Role 1 Worker Role 2

Cach

e

References

- MSDN (cloud architectural patterns)

- azure.microsoft.com (Learning videos and documentation)

- some images from www