Windows Azure Storage
description
Transcript of Windows Azure Storage
Windows Azure StorageAnton Boyko
Windows Azure StorageStorage in the CloudScalable, durable, and availableAnywhere at anytime accessOnly pay for what the service uses
Exposed via RESTful Web ServicesUse from Windows Azure ComputeUse from anywhere on the internet
Windows Azure Storage AccountUser specified globally unique account name
North Central US Northern Europe
Western Europe East Asia
South East Asia
US Europe Asia
Can choose geo-location to host storage account:
South Central US
West US East US
Windows Azure Storage AccountCan CDN Enable AccountBlobs delivered via 24 global CDN nodes
Can co-locate storage account with compute accountExplicitly or using affinity groups
Accounts have two independent 512 bit shared secret keys
100 TBs per account
New FeaturesGeo-ReplicationStorage AnalyticsLogs: Provide trace of executed requests for your storage accountsMetrics: Provide summary of key capacity and request statistics for Blobs, Tables, and Queues
Storage SecurityWindows Azure Storage provides simple security for calls to storage serviceHTTPS endpointDigitally sign requests for privileged operations
Two 512bit symmetric keys per storage accountCan be regenerated independently
More granular security via Shared Access Signatures
Windows Azure Storage Abstractions
TablesStructured storage. A table is a set of entities; an entity is
a set of properties.
QueuesReliable storage and delivery of messages for an application.
BlobsSimple named files along with metadata for the file.
DrivesDurable NTFS volumes for Windows Azure applications to use. Based on Blobs.
Blob Storage
Blob Storage Concepts
BlobContainerAccount
http://<account>.blob.core.windows.net/<container>/<blobname>
Pages/ Blocks
contoso
PIC01.JPG
Block/Page
Block/Page
PIC02.JPGimages
VID1.AVIvideos
Blob Details
Main Web Service
Operations
PutBlobGetBlobDeleteBlobCopyBlobSnapshotBlob LeaseBlob
Blob Details
Associate Metadata with Blob
Standard HTTP metadata/headers (Cache-Control, Content-Encoding, Content-Type, etc)
Metadata is <name, value> pairs, up to 8KB per blob
Either as part of PutBlob or independently
Blob Details
Blob always accessed by
name
Can include ‘/‘ or other delimeter in name e.g. /<container>/myblobs/blob.jpg
Blob ContainersMultiple Containers per AccountSpecial $root container
Blob ContainerA container holds a set of blobsSet access policies at the container level Associate Metadata with ContainerList the blobs in a containerIncluding Blob Metadata and MD5 NO search/query. i.e. no WHERE MetadataValue = ?
Blobs ThroughputEffectively in Partition of 1Target of 60MB/s per Blob
Two Types of Blobs Under the Hood
Block BlobTargeted at streaming workloads
Each blob consists of a sequence of blocksEach block is identified by a Block ID
Size limit 200GB per blob
Optimistic Concurrency via Etags
Page BlobTargeted at random read/write workloads
Each blob consists of an array of pages Each page is identified by its offset from the start of the blob
Size limit 1TB per blob
Optimistic or Pessimistic (locking) concurrency via leases
TheBlob.wmv
Uploading a Block BlobUploading a large blob
10 GB Movie
Bloc
k Id
1Bl
ock
Id 2
Bloc
k Id
3
Bloc
k Id
N
blobName = “TheBlob.wmv”;PutBlock(blobName, blockId1, block1Bits);PutBlock(blobName, blockId2, block2Bits);…………PutBlock(blobName, blockIdN, blockNBits);PutBlockList(blobName,
blockId1,…,blockIdN);
TheBlob.wmv
BenefitEfficient continuation and retryParallel and out of order upload of blocks
THE BLOB
Windows AzureStorage
Page Blob – Random Read/WriteCreate MyBlob
Specify Blob Size = 10 GbytesSparse storage - Only charged for pages with data stored in them
Fixed Page Size = 512 bytesRandom Access Operations
PutPage[512, 2048)PutPage[0, 1024)ClearPage[512, 1536)PutPage[2048,2560)
GetPageRange[0, 4096) returns valid data ranges:
[0,512) , [1536,2560)GetBlob[1000, 2048) returns
All 0 for first 536 bytesNext 512 bytes are data stored in [1536,2048)
0
10 GB
5121024153620482560
10 GB Address Space
Shared Access SignaturesFine grain access rights to blobs and containersSign URL with storage key – permit elevated rightsRevocationUse short time periods and re-issueUse container level policy that can be deleted
Two broad approachesAd-hocPolicy based
Content Delivery Network (CDN)High-bandwidth global blob content delivery24 locations globally (US, Europe, Asia, Australia and South America), and growingSame experience for users no matter how far they are from the geo-location where the storage account is hosted
Blob service URL vs. CDN URL:Windows Azure Blob URL: http://images.blob.core.windows.net/Windows Azure CDN URL: http://<id>.vo.msecnd.net/ Custom Domain Name for CDN: http://cdn.contoso.com/
pic1.jpg
Windows Azure CDN
To Enable CDN:Register for CDN via Dev PortalSet container images to public
pic1.jpg
GEThttp://guid01.vo.msecnd.net/images/pic.1jpg
http://sally.blob.core.windows.net/images/pic1.jpg
http://sally.blob.core.windows.net/ http://guid01.vo.msecnd.net/
pic1.jpg
404
TTL Content Delivery Network
Windows Azure Blob Service
EdgeLocation
EdgeLocation
EdgeLocation
Drives
Windows Azure DrivesDurable NTFS volume for Windows Azure InstancesUse existing NTFS APIs to access a network attached durable driveUse System.IO from .NET
BenefitsMove existing apps using NTFS more easily to the cloudDurability and survival of data on instance recycle Drives can be up to 1TB
A Windows Azure Drive is an NTFS VHD Page BlobMounts Page Blob over the network as an NTFS driveLocal cache on instance for read operationsAll flushed and unbuffered writes to drive are made durable to the Page Blob
Drive DetailsOperations performed via Drive API not REST CallsOperations on DrivesCreateDriveCreates a new NTFS formatted VHD in Blob storage
MountDrive/UnmountDriveMounts a drive into Instance at new drive letterUnmounts a drive freeing drive letter
Get Mounted DrivesList mounted drives; underlying blob and drive letter
Snapshot DriveCreate snapshot copy of the drive
VM
How Windows Azure Drives Works Drive is a formatted page blob stored in blob
serviceMount obtains a blob lease Mount specifies amount of local storage for cacheNTFS flushed/unbuffered writes commit to blob store before returning to appNTFS reads can be served from local cache or from blob store (cache miss)
DemoBlob
OS
Application
Drive X:
Windows Azure Blob Service
Local Cache
Failover with DrivesMust issue NTFS Flush command
to persist dataUse System.IO.Stream.Flush()
Read/Write Drives protected with leases1 Minute lease expiryMaintained by Windows Azure OS DriverUnmount on RoleEntryPoint.OnStop
On failureLease will timeout after 1 minuteRe-mount drive on new instance
Q & A