Post on 15-Nov-2014
description
• intended audience : – SQL DBA’s, SQL Server enthousiasts, non Azure Database developers
• goal : – Enlight the most valueable aspects on Windows Azure SQL Database and all related material
An evening in the cloud:
The old, the new and the big
Techmike2KX.wordpress.com
@Techmike2KX
mike.martin@csps.be
Me.About();
A quick intro to Windows Azure
SaaSPhysical Virtual
A Continuous Offering From Private To
Public Cloud
IaaS PaaS
Cloud ServicesPackaged Software
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
You
man
age
Infrastructure(as a Service)
Storage
Servers
Networking
O/S
Middleware
Virtualization
Data
Applications
Runtime
Managed by vendor
You
man
age
Platform(as a Service)
Managed by vendor
You
man
age
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
Software(as a Service)
Managed by vendor
Storage
Servers
Networking
O/S
Middleware
Virtualization
Applications
Runtime
Data
Cloud PlatformsPublic Cloud Platforms
Private Cloud Platforms Infrastructure as a service (IaaS)
Platform as a Service (PaaS)
For Scalable Web Apps For Data-Driven Business Apps
For Collaboration Apps
Microsoft
Others
Dynamic infrastructure Toolkit for System
Center (Hyper-V Cloud, AppFabric Server)
Dynamic DataCenter Toolkit for Hosters
(Hyper-V Cloud)
Hosters
Windows Azure Platform
VMWare vSphere (ESX, vCenter)
Hosters
VMWare vCloud Initiative (ESX, vCenter)
Amazon Web Services
GoogleAppEngine
Saleforce.comForce.com
GoGrid Cloud Hosting, RackSPace, others
Dynamics CRM / xRM Online Office 365
Windows Live
Google Docs
Amazon Web Services
Cloud services Web sites Virtual machines
Applicationbuilding blocks
StorageBig data
Caching
CDN
Database
Identity
Media
Messaging
Networking
Traffic
DBSQL databaseRelational SQL Server Engine in the CloudClustered for high availabilityFully Managed ServiceSQL Reporting support
A SQL Database Architecture in Azure
A Server Is Not A Machine
SQL Server
A Machine
SQL Database
Server
A TDS Endpoint
How It Works
ArchitectureClient Layer - Used by application to communicate directly with SQL Database.Services Layer – Gateway between Client layer and Platform layer.Platform Layer – Includes physical servicers and services that support the Services layer.Infrastructure Layer – IT administration of the physical HW and OS.
Infrastructure Layer
PHPWCF Data Services
SQL ServerApplications
and Tools
ODBC ADO.NET
Tabular Data Stream (TDS)
Client Layer
Services Layer
Provisioning
Billing and Metering
Connection Routing
Provisioning
Billing and Metering
Connection Routing
Provisioning
Billing and Metering
Connection Routing
…
…
…
TDS+SSL
Platform Layer
SQL Server
SQL Database
Fabric
Management Services
SQL Server
SQL Database
Fabric
Management Services
SQL Server
SQL Database
Fabric
Management Services
…
…
…
Database Replicas and Failover
Replica 1
Replica 2
Replica 3
DB
Single Database Three Replicas
Single Primary
Replica 4
!
Provisioning a SQLDatabase in Azure
Server ProvisioningServer DefinedService head that contains databasesConnect via automatically generated FQDN (xxx.database.windows.net)Initially contains only a master database
Provision Servers InteractivelyLog on to Windows Azure Management PortalCreate a SQL Database serverSpecify admin login credentialsAdd firewall rules and enable service access
Automate Server ProvisioningUse Windows Azure Platform PowerShell cmdlets (or use REST API directly)wappowershell.codeplex.com
Create Database…Use Familiar TechnologiesTransact-SQL
Languages.NET Framework (C#, Visual Basic, F#) via ADO.NET
C / C++ via ODBC
Java via Microsoft JDBC provider
PHP via Microsoft PHP provider
FrameworksOData, Entity Framework, WCF Data Services, NHibernate
ToolsSQL Server Management Studio (2008 R2 and later)
SQL Server command-line utilities (SQLCMD, BCP)
CA Erwin® Data Modeler
Embarcadero Technologies DBArtisan®
SQL Server ComparisonFocus on logical vs. physical administration
Database and log files automatically placed
Three high-availability replicas maintained for every database
Tables require a clustered index
Maximum database size is 150 Gb
Unsupported SQL Server FeaturesUse command, linked servers, distributed transactions, distributed views, distributed queries, four-part names
Service Broker
Common Language Runtime (CLR)
SQL Agent
Native Encryption
*Backup / Restore
Enhanced ToolingSQL Database Management PortalWeb designers for tables, views, stored procs
Interactive query editing and execution
SQL Server Data Tools (SSDT)Visual Studio IDE for database development
Includes modern designers and projects with declarative, model-driven development
Develop and test in both connected and disconnected states
Platform targeting for both SQL Server (2005 and above) and SQL Database
Get it free with Web PI, with SQL Server 2012 and with Visual Studio 11
Database Deployment
Data-Tier Application Framework (DAC Fx)Alternative to traditional script based approach
Dramatically simplifies deployment, migration and versioning of databases
Provides a single unit of deployment for schema (dacpac) or for schema + data (bacpac)
Supports automatic versioning of database schemas
Supports platform targeting for both SQL Server (2005 and above) and SQL Database
Build from scratch or extract from existing db
How To Get The Latest DAC FxWith SQL Server Data Tools
With SQL Server 2012 Management Studio
With SQL Database Import/Export Service
Via sqldacexamples.codeplex.com
Feature pack supportWhat’s Supported
ConstantsConstraintsCursorsExtension of spatial data types and methods through CLRIndex management and rebuilding indexesLocal temporary tablesReserved keywordsSpatial data and indexesStored proceduresStatistics managementTransactionsTriggersTables, joins, and table variablesTransact-SQL language elements such as
Create/drop databasesCreate/alter/drop tablesCreate/alter/drop users and logins
User-defined functionsViews
What’s not SupportedCommon Language Runtime (CLR)Database file placementDatabase mirroringDistributed queriesDistributed transactionsFilegroup managementGlobal temporary tablesSQL Server configuration optionsSQL Server Service BrokerSystem tablesTrace Flags
T-SQL• supported :
http://msdn.microsoft.com/en-us/library/ee336270.aspx
• partially supported : http://msdn.microsoft.com/en-us/library/ee336267.aspx
• unsupported : http://msdn.microsoft.com/en-us/library/ee336253.aspx
There Are Two Ways To Secure A Database:
Within The Database
On The Server
Server Benefits
Server identity and access controlSQL authentication supported (No Integrated authentication)
The Admin login is similar to sa
Connect to master to administer logins
loginmanager: Server-Level security role for creating logins
dbmanager: Server-Level security role for creating databases
Database BenefitsDatabase identity and access controlLogins require an associated user account
The Admin login is automatically associated with dbo
The dbo has full rights in the database
Manage users with CREATE / ALTER / DROP USER commands
Add users to roles via sp_add_rolemember to grant privileges
Utilize schemas where appropriate
SQL Database Firewall
Securing your dataIP Address-based access control for SQL Database
Rules can be defined at the server and database
No IP authorized by default
Configurable using the SQL Database Portal and REST API
Option to disable/enable access from applications hosted in Windows Azure
Services Layer
Services Layer
Internet
SQL Database Firewall
Application ConnectivityConnecting To SQL Database1. TDS (Tabular Data Stream) protocol over TCP/IP supported
2. SSL required
3. Use firewall rules to connect from outside Microsoft data center
ASP.NET EXAMPLE:
Considerations And Best Practices1. login: [login]@[server]
2. Idle connections
3. Long running transactions
4. DoS guard
5. Failover events
6. Throttling
7. Connection pooling and Retry logic
8. Latency introduced for updates
9. No cross-database dependencies
<connectionStrings><addname="AdventureWorks"connectionString=
"Data Source=[server].database.windows.net;Integrated Security=False;Initial Catalog=ProductsDb;User Id=[login];Password=[password];Trusted_Connection=False;Encrypt=true;"
providerName="System.Data.SqlClient"/></connectionStrings>
BI Capable SQLDatabase in Azure
SQL ReportingVisualize your dataSQL Server Reporting Services technology as a serviceIdeal for operational reporting against SQL Database dataEnterprise-ready with automatic support for HADesigned to scale elastically with demandRapid Provisioning
Architecture – What’s Under The Hood
Load Balancer
Gateway GatewayGateway
RS InstanceMonitor/Comm[SLA & Usage][Performance][Traces & Logs]
RS Engine
RS InstanceMonitor/Comm[SLA & Usage][Performance][Traces & Logs]
RS Engine
RS InstanceMonitor/Comm[SLA & Usage][Performance][Traces & Logs]
RS Engine
SQL Database
RS GPMTenant Data
Operations
Directory Services
Allocation + Service
Customer Service
SLA Conformation
Load Balancing
Billing/Usage
Monitoring[Alerting]
[Performance][Traces & Logs]
http://server.reporting.windows.net/reports/report.rdl
SSRS Azure SSRS
Tooling BIDS, Report Builder BIDS, Report Builder
Data Sources Assorted Data Sources SQL Database
Report Management Report Manager or SharePoint Windows Azure Developer Portal, URL Browsing
Developer Custom Data Sources, assemblies, etc.
No extensibility yet
Security Model Windows Authentication SQL Database Authentication
Feature Comparison
The cloud and Beyond
SQL Data Sync
Data SynchronizationMicrosoft Sync Framework technology as a service
No code setup and configuration
Synchronize datasets hosted in SQL Database or SQL Server
Customizable to fit your business needs
SQL Databas
e (US)
SQL Databas
e (Hub)
SQL Server
(OnPrem)
SQL Databas
e (WE)
SQL Data Sync
Components And Architecture
HTTPS
TDS
SQL Database
Windows Azure Blobs
Data Sync Agent
Windows Azure PortalAdmin UI
HTTPS
Service Data• Configuration - Servers, Sync
Groups, Schemas, Agents, etc.• Tasks queue - sync, get schema,
provision, etc.• Logging - task results
Service Data• Temporary batch files of
uploaded data changes
Batch Files• Temporary files containing
changed data to upload
Windows Service
UI
Admin UI Handler
Cloud Task Execution
Task Scheduler
- Get tasks- Task results
Browser
Agent Task Execution
TDS
User DatabasesAdded database objects:• Change Tracking tables• Sync meta-data tables• Triggers & sprocs
User DatabasesAdded database objects:• Change Tracking
tables• Sync meta-data
tables• Triggers & sprocs
Key:Windows Azure Web or worker roles
Syncing Between SQL Server And SQL Database
SQL Server
Application
On-Premises
SQL Databa
se
Application
Cloud
E.g. Migration periodE.g. Different apps sharing same dataE.g. DR
Multiple Locations (e.g. branch)Share database between locationsAggregate data in cloud
One-way sync to cloudOne-way sync from cloudTwo-way sync
Sync
Sync
SQL Server
Application
Syncing Between SQL Databases
SQL Databa
se
Application
SQL Databa
se
Application
SQL Databa
se
Application
SQL Databa
se
Application
Sync
Sync
Sync
Cloud
Scale-out via multiple copies of dataSeparate reporting & OLTP workloads
Geographically located web applications
Data Sync And Reporting
SQL Databa
se
Application
SQL Databa
se
Reporting
One-way Sync
Different Indexing
Filtered Subset of Data
Sync
SQL Data Sync And Windows Azure Traffic Manager
US
SQL Databa
se
Application
SQL Databa
se
Application
Sync
Europe
SQL Databa
se
Application
Asia
Sync
Traffic ManagerControl traffic distribution Azure-hosted
applications
DNS-based
Several Distribution Options
Behavior And Database ImpactSync ProvisioningChange Tracking via triggers and side-tables
Minimal DML overhead
No change to user tables
Stored Procedures and meta-data tablesTransactional Consistency“Net changes”; not “all changes”
Batches split into transactions to apply
PK/FK ordering preserved
Transaction boundaries at source not preservedE.g. Could get new Order applied in separate transaction to associated new OrderDetail
BatchingData change split into batches for transfer
Spring is in the Cloud
PaaS SaaSPhysical Virtual IaaS
A Continuous Offering From Private to
Public Cloud
Windows Azure SQL Database
SQL Server and Windows Azure VMSupported VersionsSQL Server 2012, 2008 R2, 2008
Supported FeaturesAll SQL Server features supported except failover clustering *
SQL Server ProvisioningCloud-first using stock images, bring your own server / VHD, capture cloud images
SQL Server LicensingPay by the hour or migrate your own license via Software Assurance ** * AlwaysOn Availability Groups not supported until GA
** Windows Azure Compute and Storage charges also apply
Virtual Machine Sizes and SQL Editions
Each Persistent Data Disk Can be up to 1 TB
VM Size CPU Cores Memory Bandwidth# Data Disks SQL Edition
Extra Small Shared 768 MB 5 (Mbps) 1 Express
Small 1 1.75 GB 100 (Mbps) 2 Standard
Medium 2 3.5 GB 200 (Mbps) 4 Standard
Large 4 7 GB 400 (Mbps) 8Standard / Enterprise
Extra Large 8 14 GB 800 (Mbps) 16Standard / Enterprise
Running SQL Server in a Windows Azure VMStorage RecommendationsDo not use write cachingAvoid using OS drive for large databasesConsider putting database and transaction log files on separate drivesConsider putting tempdb on the non-persistent cache disk (D:\)
Database RecommendationsConsider using database page compression to reduce I/O
High Availability RecommendationsConsider latency between primary and replica when choosing sync mode
SQL Server or Windows Azure SQL Db? SQL Server
(IaaS)Windows Azure SQL Database
(PaaS)
Development
Migrate Existing Apps Develop New Apps
Management
Full Control Managed Service
Compatibility
Full SQL Server Capabilities Based on SQL Server TechnologyShared TechnologyNetwork transport (Tabular Data Stream)
SQL dialect (Transact-SQL)Data access APIs (ADO.NET, ODBC, JDBC)
Development tools (SQL Server Data Tools)Management tools (SQL Server Management
Studio)
SQL Federation
Database ScalabilityScale to hundreds of nodes via database shardingMulti-tenancy via flexible repartitioningOnline split operations to minimize downtimeAutomatic data discovery regardless of changes in how data is partitioned
Database ScalabilityScale-upSingle database that houses all the data of an applicationHard to handle peak loadOK with exponential incremental cost
Scale-OutMultiple databases spread over multiple independent nodesCost effective, commodity class hardwareTypical patterns: Sharding and Horizontal Partitioning
User Traffic
Single Database
LB LBFront Tier Middle Tier
SQL Azure
SQL Federation
SQL Databas
eLarger User Traffic
FrontTierFrontTierFrontTierFrontTier
FrontTierFrontTierFrontTierMiddle
Tier
Database ElasticityExtend the scalability model to the database tierAdd and remove SQL Database nodes via database partitioning (Federations)Scale on demand to your traffic without any downtime
Why Use SQL Federation?Database ScalabilityCreate an elastic database tier that can expand and contract with your applications workload without downtime
Gain practically unlimited scale by harnessing 100s of SQL Database nodes
SQL Databas
e
Why Use SQL Federation?Simplified Multi-tenancyBuild Multi-tenant Solutions
Efficient management of tenant placement and re-placement
Multiple-tenants per database Single tenant per database Multiple databases per tenant
Simplified Development and AdministrationRobust programming and connectivity model with native tooling
Same great programming model using existing tools
Who Are SQL Federation for? A Few ExamplesWeb Scale DB Solutions
Multi-tenant Saas ISVs
Workloads with Spikes, Bursts, Peaks, etc…
NoSQL Applications
Windows Azure Connectivity Options
Data SynchronizationSQL Data Sync
Application-Layer Connectivity & Messaging
Service Bus
CLOUD ENTERPRISE
Secure Machine-to-Machine Network
ConnectivityWindows Azure Connect
Secure Site-to-Site Network Connectivity
Windows Azure Virtual Network
Secure Site-to-Site Network Connectivity
Windows Azure Virtual Network
Windows Azure Virtual Networks
Your “virtual” branch office / datacenter in the cloudEnables customers to extend their Enterprise Networks into Windows AzureNetworking on-ramp for migrating existing apps and services to Windows AzureEnables customers to run “hybrid” apps that span cloud and their premises
A protected private virtual network in the cloudEnables customers to setup secure private IPv4 networks fully contained within Windows AzureIP address persistenceInter-service DIP-to-DIP communication
Virtual Network ScenariosHybrid Public/Private Cloud
Enterprise app in Windows Azure requiring connectivity to on-premise resources
Enterprise Identity and Access Control
Manage identity and access control with on-premise resources (on-premises Active Directory)
Monitoring and Management
Remote monitoring and trouble-shooting of resources running in Windows Azure
Advanced Connectivity Requirements
Cloud deployments requiring persistent IP addresses and direct connectivity across services
DNS ScenariosWindows Azure DNS Scenarios Use your own DNS Scenarios
A. Client-server applications using persistent VMs
B. Stateless roles with persistent VMs
C. Hybrid connectivity with on-premise (DNS on-premise)
D. SharePoint with custom DNS (persistent VM)
Persistent VM Role
SQL Reporting Service
Persistent VM Role
SQL Analysis Service
Persistent VM Role
SQL Service
Internet
IIS Web ApplicationWeb Role 1
IIS Web ApplicationWeb Role 2
IIS Web ApplicationWeb Role 3
Persistent VM Role
SQL ServiceLB
On-Premises Machine
Active Directory
Active Directory
SQL ServiceDomain joined to On-
Premises Network
On-Premises Machine
Business Components &
Entities
On-Premises Machine
UI Process Components
Web Tier
Active Directory
Internet
Persistent VM Role
SharePoint FrontEnd
Persistent VM Role
SharePoint FrontEnd
Persistent VM Role
Search and Indes
SQL Service
Persistent VM Role
DC DNS
Persistent VM Role
Persistent VM Role
SQL
Persistent VM Role
SQL
Local DNS
SQ
L Mirro
ring
LB
Open User Access
(Website)
Warning! Hot and New!!
Released last week : new featuresLinked Server and Distributed
Queries Allow server linking like in On Premises' situations
1. Only link to Windows Azure SQL Database not vice versa!!
2. Run queries over the database from within your on premises situation
3. Virtually all DB ‘s can be linked (except for the MASTER DB)
Released last week : new featuresRecursive Triggers
As it states : triggers can recurse (as in Sql Server 2012 box product (see BOL for more info))
USAGE EXAMPLE:
ALTER DATABASE ADVENTUREWORKS2012 SET RECURSIVE_TRIGGERS ON|OFF;
Released last week : new featuresDBCC SHOW_STATISTICS
Displays displays current query optimization statistics for a table or indexed view.
USAGE EXAMPLE:
DBCC SHOW_STATISTICS ( table_or_indexed_view_name , target )
[ WITH [ NO_INFOMSGS ] < option > [ , n ] ]
< option > :: =
STAT_HEADER | DENSITY_VECTOR | HISTOGRAM | STATS_STREAM
Released last week : new featuresFIREWALL RULES ON
DATABASE LEVEL• Previously set only at the server level, either
through the management portal or via T-SQL commands
• Now, firewall rules can be additionally set at the more granular database level
• Different rules on different DB on the same logical server possible
Released last week : new featuresFIREWALL RULES ON DATABASE LEVEL
USAGE EXAMPLE / HOWTO:
sp_set_database_firewall_rule-- create new database-level firewall setting for only IP 0.0.0.4
exec sp_set_database_firewall_rule N'Example DB Setting 1','0.0.0.4','0.0.0.4'
-- update database-level firewall setting to also allow IP 0.0.0.5
exec sp_set_database_firewall_rule N'Example DB Setting 1','0.0.0.4','0.0.0.5'
New table available for storing these : sys.database_firewall_rules additionally to the sys.firewall_rules
Bigger, Better, Bolder!
An Exotic OverviewBIG DATA RELATED PRODUCTSHADOOP on Windows Azure
“Cloud Numerics”
“Data Explorer”
“Data Hub”
“Trust Services”
StreamInsight
Codename “Cloud Numerics”Microsoft Codename "Cloud Numerics“ is an HPC Based computing model making use of Windows Azure SQL Database with following capabilities:
• Parallel workloads• Scale-out enabled functions ranging from Descriptive Statistics, to Fourier
Transforms, to Linear Algebra and more• Improved IO enabling parallel reads and writes from a variety of data sources,
including Azure Blob storage, and formats
Call to Action
http://WindowsAzure.com• Guidance
• Training• Technical Docs• Pricing, Licensing
Resources
• www.windowsazure.com• Windows Azure Training Kit• MSDN WA SQL Database home : http://
msdn.microsoft.com/en-us/library/windowsazure/ee336279.aspx• BI :
https://www.windowsazure.com/en-us/home/features/business-analytics/
• SQL : https://www.windowsazure.com/en-us/manage/services/sql-databases/
• Pricing, Licensing : https://www.windowsazure.com/en-us/pricing/calculator
Thank you for your attention!Thank you for your attention!