Ulrich RoxburghServices2 Ltd.

Get Out, and Take Your Data With You!

SESSION CODE: ARC304

3

Agenda

Occasionally Connected ApplicationsMicrosoft Sync Framework V2Data Synchronisation FundamentalsSQL Server SynchronisationThe Cloud

4

Enterprise Application Model

ERP CRMDatabase

5

Smart Client ApplicationsDisadvantages

Network Requirements - The remote device requires constant connection to the corporate network while accessing their data Data Access Speeds - Remote workers are typically connected over slow, unreliable wired or wireless networks Single Point of Failure - All users are reliant on a single server. If that database becomes unavailable due to planned server downtime or from server failures, all remote workers will be disconnectedServer Scalability - As more workers work remotely, the performance of the corporate servers will be affected, leading to a need to add additional hardware

6

Occasionally Connected Applications (OCAs)

An OCA allows a remote worker to continue to access their data, but locally.OCA requires data synchronization capabilities. Data synchronization takes information stored in the client database and synchronizes changes with a server database. Advantage of a synchronization-based solution

Users do not require constant network connectionUsers are not limited by network speedCommon application codeOffloads processing requirements from the central database.

7

Occasionally Connected Scenarios

8

Challenges associated with Data Synchronisation

Change TrackingMaintaining Change DataConflict Detection and ResolutionPrioritizing Data ExchangeBackground SynchronizationMultiple Synchronization TopologiesCustom Client and Server DatabasesSecurity

9

Introducing Microsoft Sync Framework

Microsoft Sync Framework is a synchronization platform that enables collaboration and offline scenarios. Supports sync ecosystems that integrate any type of data, using any protocol over any network.Highlights

Add sync support to new and existing applications, services, and devicesEnable collaboration and offline capabilities for any applicationLeverage sync capabilities exposed in Microsoft technologies to create sync ecosystemsExtend the architecture to support custom data types including files

10

Capabilities

Any Store - Add synchronization to a wide range of applications, services, and devices, from enterprise-level services to simple USB storage devices.Any Data Type - Easily integrate new data types to sync using Microsoft Sync Framework’s metadata-based synchronization technologyAny Protocol - Synchronize data using existing architectures and protocols. Integrate sync into a variety of protocols. Enable web services to expose and synchronize data with any platform by creating a data feed based on the FeedSync open specification.Any Network Configuration - Sync-enable your applications, devices, and services in true peer-to- peer and hub/spoke topologies. Recover easily from network interruptions without losing data integrity. Reduce network traffic using efficient change enumeration.

11

Core Components

Database synchronization providersFile synchronization providersWeb feed synchronization componentsCustom providersMetadata storage servicesCore API and runtime

12

Developing with Sync Framework

Developer:The applicationThe data storeThe data transfer protocol

Sync Framework.Synchronization session, or managerThe synchronization runtime

Sync Framework, or the DeveloperThe sync providerThe metadata store

13

Synchronisation Flow

Synchronization Session initiated with destinationDestination prepares and sends knowledgeDestination knowledge used to determine changes to be sentChange versions and source knowledge sent to destinationDestination compares items against source version and knowledgeConflicts are detected and resolved or deferredDestination requests item data from sourceSource prepares and sends item data

14

Participants

Full Participants - Devices that allow developers to create applications and new data stores directly on the device. E.g. Windows Phone, laptopPartial Participants - Devices that have the ability to store data either in the existing data store or another data store on the device but do not have the ability to launch executables. E.g. thumb drives or SD Cards. Simple Participants - Devices that are only capable of providing information when requested. These devices cannot store or manipulate new data. E.g. RSS Feeds and web services.

15

Change Tracking

Change tracking provides a list of changes made from one point in time to another. Commonly implemented using rowversions and triggers, plus a “deleted” tableThe major disadvantages are:

Changes are required to the schema to add columns and tablesTriggers are fired for each change made, which has performance implications. Logic for maintaining proper rowversions and row deletions can get extremely complicated.

SQL Server 2008 has built-in change tracking, implemented without rowversions and triggersThe Sync Framework database synch providers take advantage of SQL Server 2008 change tracking and provide the following advantages :

No schema changes are requiredTriggers are not required for tracking changesAll of the logic for tracking changes is internal to the SQL Server engine

16

Conflict Resolution

Conflicts occur when two or more databases make a change to the same piece of dataA variety of ways to resolve these conflicts.

Last change to come in winsHighest priority user winsManual selection

Sync Framework provides conflict detection and resolution capabilities out of the boxSQL Server 2008 makes it easier to identify conflicts.

17

Sync Providers

Database sync providers - ADO.NET Sync providers and SQL Server Sync Providers. Can be used to synchronize databases for the following types of scenarios:

Collaborative scenarios.Offline scenarios

File synchronization provider - Can be used to synchronize files and folders in NTFS, FAT, or SMB file systems. The directories to synchronize can be local or remote; they do not have to be of the same file system. Web feed synchronization components - Can be used in two ways:

To write a provider that represents a FeedSync XML file as its replica. To synchronize the data of another type of replica (such as a file system) with an RSS or Atom feed.

Custom providers - Can be used to create synchronization providers for any type of data store.

18

Database Sync Providers

DbServerSyncProvider and SqlCeClientSyncProvider support offline scenarios.

Suitable for client-server topologiesLess complex but cannot participate in topologies with other providers

SqlSyncProvider, SqlCeSyncProvider, and DbSyncProvider support collaboration scenarios

Suitable for peer-to-peer and mixed topologies. More flexible. E.g. Synchronize one instance of SqlSyncProvider and one instance SqlCeSyncProviderUse Sync Framework metadata and can participate in topologies with other Sync Framework providers.

19

SQL and SQL CE Providers

SQL Server 2008 introduced new change tracking Sync Framework SQL sync providers take advantage of SQL Server 2008 change tracking:

No schema changes are requiredTriggers are not required for tracking changesLogic is internal to the SQL Server engine

SqlCeSyncProvider is used for SQL Server Compact databases.SqlSyncProvider for any other edition of SQL Server, including SQL Server Express.SqlSyncProvider can be used as a server or a client

20

SQL Server Data Scalability

SQL AzureSQL Server 2008 R2 DatacenterSQL Server 2008 R2 Parallel Data WarehouseSQL Server 2008 R2 EnterpriseSQL Server 2008 R2 StandardSQL Server 2008 R2 ExpressSQL Server 2008 R2 Compact

21

Other alternatives

Remote Data Access (RDA) - Used to sync SQL Server Compact with SQL Server

Does not support sync using services, incremental change tracking, or conflict detection and resolution

Merge replication - Used to synchronize different editions of SQL Server

Does not support sync using services, or heterogenous databases

22

SQL Sync Architecture: Two-Tier

Sync Provider

Sync Adapter

Sync Orchestrator

Sync Provider

Sync Adapter

Data Data

23

SQL Sync Architecture: n-Tier

Sync Provider Proxy

Sync Adapter

Sync Orchestrator

Sync Provider

Sync Adapter

Data Data

24

Implementing SQL SyncDefine and Provision Server// define new scope, get description for Products table, and add to scopeDbSyncScopeDescription scopeDesc = new DbSyncScopeDescription("ProductsScope");

DbSyncTableDescription tableDesc = SqlSyncDescriptionBuilder.GetDescriptionForTable("Products", serverConn);

scopeDesc.Tables.Add(tableDesc);

// Create a server scope provisioning object based on the Product Scope and provision // it, skipping creation of the table because it exists SqlSyncScopeProvisioning serverProvision = new SqlSyncScopeProvisioning(serverConn, scopeDesc);

serverProvision.SetCreateTableDefault(DbSyncCreationOption.Skip);serverProvision.Apply();

25

Implementing SQL SyncProvision SQL Express Client// get the description of ProductsScope from the SyncDB server databaseDbSyncScopeDescription scopeDesc = SqlSyncDescriptionBuilder.GetDescriptionForScope("ProductsScope", serverConn);

// create server provisioning object based on the ProductsScopeSqlSyncScopeProvisioning clientProvision = new SqlSyncScopeProvisioning(clientConn, scopeDesc);

// starts the provisioning processclientProvision.Apply();

26

Implementing SQL SyncExecuting Sync// create the sync orchestrator, and set local and remote providersSyncOrchestrator syncOrchestrator = new SyncOrchestrator();

syncOrchestrator.LocalProvider = new SqlSyncProvider("ProductsScope", clientConn);syncOrchestrator.RemoteProvider = new SqlSyncProvider("ProductsScope", serverConn);

// set the direction of sync session to Upload and DownloadsyncOrchestrator.Direction = SyncDirectionOrder.UploadAndDownload;

// execute the synchronization process SyncOperationStatistics syncStats = syncOrchestrator.Synchronize();

27

Implementing SQL SyncGet results// print statisticsConsole.WriteLine("Start Time: " + syncStats.SyncStartTime);Console.WriteLine("Total Changes Uploaded: " + syncStats.UploadChangesTotal);Console.WriteLine("Total Changes Downloaded: " + syncStats.DownloadChangesTotal);Console.WriteLine("Complete Time: " + syncStats.SyncEndTime);Console.WriteLine(String.Empty);

28

Implementing SQL SyncFiltered Scopes// create server provisioning objectSqlSyncScopeProvisioning serverProvision = new SqlSyncScopeProvisioning(serverConn, scopeDesc);

serverProvision.SetCreateTableDefault(DbSyncCreationOption.Skip);

// set the filter column to OriginState and filter value to 'NC'serverProvision.Tables["Orders"].AddFilterColumn("OriginState");serverProvision.Tables["Orders"].FilterClause = "[side].[OriginState] = 'NC'";

// start the provisioning processserverProvision.Apply();

29

The ‘Cloud’

2-tier architecture: Sync Framework runs on the local computer and uses a SqlSyncProvider object to connect to the SQL Azure databaseN-tier architecture: A Sync Framework database provider runs in a Windows Azure hosted service and communicates with a proxy provider that runs on the local computer.

30

2-Tier SQL Server ↔SQL Azure Sync// Create a scope named "customers" and add tables to it… Omitted for brevity

// Create a provisioning object for "customers" and apply it to the on-premise database.SqlSyncScopeProvisioning onPremiseConfig = new SqlSyncScopeProvisioning(onPremiseConn, customersScopeDesc);onPremiseConfig.Apply();

// Provision the SQL Azure database from the on-premise SQL Server database.SqlSyncScopeProvisioning azureCustomersConfig = new SqlSyncScopeProvisioning(azureConn, customersScopeDesc);azureCustomersConfig.Apply();

31

n-Tier SQL Server ↔SQL Azure Sync

Windows Azure hosted service is the middle tier, and communicates between the local proxy and SQL Azure.To implement…

Create a component that implements the service contract and calls a SqlSyncProvider object on the SQL AzureDeploy this component in a hosted service on Windows AzureCreate a proxy provider that implements KnowledgeSyncProvider, runs on your local computer, and makes calls to the service interface implemented by your Windows Azure hosted service.

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED

OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.