Embarquez les services d'intégration SQL Server
2005Romelard FabriceD311
Presentation MVP ASP.NET since 2003 French communities member
GUSS ASP-PHP Codes Sources
.NET developer, SQL Server DBA and Sys. Admin
Coworkers Ilem SA at Geneva (Suisse)
Survey SQL Server ? SQL Server Integration Services ? .NET Developer ?
Prerequisites
Knowledge of Integration Services Knowledge of Data Flow Functionality
Level 300Level 300
Objectives Introduction to SSIS programming model Learn how to integrate with dynamic
metadata Learn how to utilize data cleansing
functionality in your apps
Integration Services
SSIS Terminology Package
Tasks Precedence Constraints Connection Managers Containers
Data Flow Task Components – Source
Adapters, Transformations, Destination Adapters
Paths
Application Overview Get data from an Excel file Provide fuzzy cleansing for certain text
fields FirstName, LastName
Save cleaned data in another Excel file Look at finished application first, then go
through several iterations to build it
Application
SSIS is embeddable SQL Server uses SSIS
SMO Maintenance Plans
Other (non SQL) products in development are using SSIS Writing your own UI is possible
SSIS designer, Management Studio, Import/Export Wizard, Migration Wizard
Uses no secret APIs Enumerating/adding/removing/changing/listening/scheduling/…
Considering releasing Migration Wizard in Shared Source Digital signing enables tamper resistance Several customers doing metadata driven package
development
Pipeline Metadata Pipeline engine requires static metadata
Early design decision Buffers laid out during pre execute
Strict data types Cannot map columns during execution
Designer debugging expects design time metadata at execution time
Configured (dynamic) queries must resolve to design time metadata at runtime
Dynamic Metadata Scenarios
Source schema changes/not known until execution
Metadata driven ETL processes
Handling dynamic metadata Generate data flows dynamically
Creating Packages
Creating Packages From scratch through object model
Create all package elements from scratch Fast, small, efficient Harder to evolve the application
From template package Adjust only what needs adjusting after loading
the template package Need to embed potentially large template file Easier to evolve the application Digital signing detects user changes
Components Terminology Component
Input Input Columns (Only data referenced by component) Virtual Input Columns (All available data produced by upstream
components – used at design time for selecting input columns) External Metadata Columns (Schema snapshot)
Output Output Columns (Produced data) External Metadata Columns (Schema snapshot)
LineageID uniquely identifies a column Every output column gets a new Lineage ID
Column Mapping Sources: ExternalColumn<->OutputColumn Transforms: InputColumn<->OutputColumn Destinations: InputColumn<->ExternalColumn
Pipeline Programming Model ComponentMetadata
Provided for all components by the engine automatically
Manages metadata and persistence for the component
Contact information for unregistered components
Helps delay creation of components until necessary
Runtime Connection Collection Connection managers used
by the componentComponentMetaDataComponentMetaData
InputsInputs
OutputOutputss
ComponentComponent
RCCRCC
Configuring Data Flows
Using Fuzzy transforms
SSIS As A Source ETL processes typically
encode complex business rules
Reuse is important One version of the truth Updates in one place Leverage advantages of
SSIS: scalability, manageability, visual building of complex processes, etc.
SSIS Source Implementation Implements IDbConnection
ConnectionString is the command line args to dtexec.exe
Command CommandText is the name
of the DataReaderDest component in package
ExecuteReader runs the package when asked for data, returns IDataReader
Supports SchemaOnly also
DataReaderDest implements IDataReader
Gets the first buffer and waits for data request
Microsoft.SqlServer.Dts.DtsClientMicrosoft.SqlServer.Dts.DtsClient Data Reader Destination ComponentData Reader Destination Component
Putting it together
Summary
Programming SSIS is straightforward Several embedding options exist SSIS can handle flexible metadata SSIS provides rich functionality and high
performance
Resources Samples installed by setup Community site, run by MVPs
http://www.sqlis.com http://www.guss.fr
Interact with product team on MSDN Forums http://forums.microsoft.com/msdn/ShowForum.aspx?
ForumID=80 Webcasts, training, blog links, books, …
http://msdn.microsoft.com/SQL/sqlwarehouse/SSIS/default.aspx
NewsGroups SQL Server
• exclusive, special information on Windows Vista and the 2007 Microsoft Office release
• notifications upon availability of new beta versions of to download or order them as soon as possible
• product-specific bonus material (videos, samples, webcasts, gadgets, etc.)
Your benefits:
Top Related