virtual techdays

virtual techdaysINDIA │ 22-24 november 2010

Windows Azure Storage – Deep DiveGaurav Mantri │ Founder, Cerebrata Software

Overview of Windows Azure Storage Blob Containers & Blobs Tables & Entities Queues & Messages API & Tools Resources Q&A


S E S S I O N A G E N D A

Please provide feedback for the session. Again, the feedback console for this session will be live till 5:00pm today. Your feedback is considered in deciding the content for the next event.

In case you are facing any technical problems, please change your status on the top-right corner of your screen, and someone will assist you.

If you have questions in between the presentation, use the “Q&A panel” on Live Meeting console to ask your question. I will try to answer as many questions as possible.


Housekeeping


Windows Azure StorageStorage Options

Blob Storage Table Storage

Queue Storage SQL Azure

Think of it as file system in the cloud. Similar to Amazon S3. 2 Level hierarchy. No limit of number of blob containers & blobs. Size limited to the size of Storage Account (currently 100 TB).


Blob Storage – Containers & BlobsOverview

Think of it as “Folders” in Cloud file system. Similar to “Buckets” in Amazon S3. Scoped by storage account. No limit on number of blobs. Nested blob containers not supported. Supports metadata (up to 8K in size). Supports ACL & Signed URL. Typical URL: http://<myaccount>.blob.core.windows.net/<containername> “$root” blob container: http://<myaccount>.blob.core.windows.net Use “$root” for storing client access policy file.


Blob ContainersOverview

Create Delete Get/set metadata Get/set ACL & access policies List blobs


Blob ContainersOperations

The blob container name must be a valid DNS name, confirming to the following naming rules: Blob container names must start with a letter or number, and can contain only letters,

numbers, and the dash (-) character. Every dash (-) character must be immediately preceded and followed by a letter or

number e.g. my-blob-container. All letters in a container name must be lowercase e.g. siteimages. Blob container names must be from 3 through 63 characters long.

Valid blob container names: abc, myblobcontainer, videos-and-images Invalid blob container names: a, MyBlobContainer, videos_and_images


Blob ContainersNaming

ACL & Access Policies provide a mechanism to provide additional access security on blob containers and blobs.

Account owner will always have complete control over blob container. Provide 3 kinds of access control over blob container:

Full public read access (Container) Public read access for blobs only (Blob) No public read access (Private)

Up to 5 access policies can be defined for a blob container. Provide read/write/delete/list access on blob container and blobs without giving away

account credentials. Use “Signed URL” to provide time limited access to blob container and blobs.


Blob ContainersACL & Access Policies

virtual techdaysINDIA │ 18-20 august 2010

DEMO: Blob Container ACL

Ability to assign name/value pairs as blob container metadata. Metadata size is limited to 8K. Metadata name must be a valid C# identifier.


Blob ContainersMetadata

Think of them as “Files” in the cloud. URI accessible e.g. http://<myaccount>.blob.core.windows.net/<blobcontainer>/myblob.txt

Scoped by blob container. Flat hierarchy however an illusion of folder hierarchy is achievable. Size limited by blob type and storage account capacity. Supports metadata (up to 8K in size). Two kinds of blobs – Block blob & Page blob.


BlobsOverview

Create/Upload/Download Delete Copy Get/Set Metadata Snapshot Lease


BlobsOperations

Ideal for streaming purposes e.g. images, documents, videos etc. Each block blob can be up to 200GB in size. Consists of a sequence of blocks where each block can be of a maximum of

4 MB size.


BlobsBlock Blobs

Ideal for random read/write workloads. Each page blob can be up to 1TB in size. Consists of a sequence of pages where each page is of 512 bytes. Pay only for the pages used. Empty pages are not charged.


BlobsPage Blobs

Azure drive is a page blob mounted as an NTFS drive. Provides a durable NTFS volume for Windows Azure applications. Mounted by one VM at a time for read/write. Contents of an Azure drive is only visible to the VM which mounted it.


BlobsAzure Drive

Use it to store structured data in the cloud. NOT a relational database rather it’s a NO-SQL implementation by

Microsoft. Similar to Amazon SDB. Possible to store huge amounts of data. No limit of number of tables & entities. Size limited to the size of Storage Account (currently 100 TB).


Table Storage – Tables & EntitiesOverview

NOT a relational table. Think of it as a property bag where disparate data can be stored. For example, in one table you can store Customer and Order data. Data in a table is “Partitioned” to support massive storage.


TablesOverview

Create Delete Query entities Entity group transaction Insert Update – Merge/Replace


TablesOperations

The table name must be a valid DNS name, confirming to the following naming rules: Table names may contain only alphanumeric characters. A table name may not begin with a numeric character. Table names are case-insensitive. Table names must be from 3 through 63 characters long.

Valid table names: Abc, Customer, Account1 Invalid blob container names: A, 1Temp


TablesNaming

NOT a row in a table. Think of it as a property bag. A maximum of 255 properties per entity. Mandatory properties – PartitionKey, RowKey, and Timestamp (system

generated). Each entity can be of a maximum of 1MB in size. Each property can be of following data type


EntitiesOverview

PartitionKey & RowKey are of “String” type. Once created, PartitionKey & RowKey are not editable. An entity can be uniquely identified by it’s PartitionKey and RowKey value. Timestamp value is system generated and is always in UTC.


EntitiesPartitionKey, RowKey and Timestamp

Supports the query options defined by the ADO.NET Data Services Framework.

Not all query options supported. Querying table will return all attributes of matching entities i.e. you can’t

select attributes to return. For best performance, one must always include “PartitionKey” in query

criteria. Example (PartitionKey eq ‘MyPartitionKey’) and (RowKey eq ‘MyRowKey’)


TablesQuerying tables

virtual techdaysINDIA │ 18-20 august 2010

DEMO: Tables & Entities

Use it as reliable messaging infrastructure in the cloud. Similar to Amazon SQS. No limit of number of queues and messages. Set a limit for message expiration (currently a message can stay for a

maximum of 7 days).


Queues & MessagesOverview

The queue name must be a valid DNS name, confirming to the following naming rules: A queue name must start with a letter or number, and may contain only letters, numbers,

and the dash (-) character. The first and last letters in the queue name must be alphanumeric. The dash (-) character

may not be the first or last letter. All letters in a queue name must be lowercase. A queue name must be from 3 through 63 characters long.

Valid queue names: abc, myqueue, poison-messages Invalid queue names: Abc, MYQUEUE, -poison_messages


QueuesNaming

A message can be 8K in size. For larger messages use Blob or Table storage. No limit on a number of messages in a queue. Message expiration date can be specified. A message can stay in a queue

for a maximum of 7 days. Up to a maximum of 32 messages can be fetched at a time. Peek/Get message semantics.


MessagesOverview

Azure Storage is accessible via REST API. Microsoft has provided a .Net based storage client library which is built on

top of REST API. Open source libraries are available for Java & PHP support.


API & ToolsAPI

Storage can also be managed by a number of tools both from Microsoft & ISVs.

Tools range from open source to commercial. Partial list of available tools:

Cerebrata Cloud Storage Studio - $70.00 ClumsyLeaf Azure Explorer - $50.00 Microsoft MMC Plugin – Free Microsoft Visual Studio Plugin – Free Microsft MyAzureStorage.com – Free Azure Storage Explorer – Free

http://blogs.msdn.com/b/windowsazurestorage/archive/2010/04/17/windows-azure-storage-explorers.aspx


API & ToolsTools



Windows Azure Home Page: http://www.microsoft.com/azure

Windows Azure Storage Team Blog: http://blogs.msdn.com/b/windowsazurestorage/

Windows Azure MSDN Forums: http://social.msdn.microsoft.com/Forums/en-US/category/windowsazureplatform

Neil Mackenzie Blog: http://convective.wordpress.com

MSDN Documentation: http://msdn.microsoft.com/en-us/library/dd163896.aspx


ResourcesUseful links

http://www.microsoft.com/azure

http://blogs.msdn.com/b/windowsazurestorage/

http://social.msdn.microsoft.com/Forums/en-US/category/windowsazureplatform

http://convective.wordpress.com/

http://msdn.microsoft.com/en-us/library/dd163896.aspx

virtual techdaysTHANKS│22-24 november 2010

Questions??

virtual techdaysTHANKS│22-24 november 2010

[email protected] │ @gmantri │ www.cerebrata.com

mailto:[email protected]

http://www.cerebrata.com/

virtual techdays

Documents

Transcript of virtual techdays