SSIS Custom Components
description
Transcript of SSIS Custom Components
![Page 2: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/2.jpg)
Why ?• Provide new functionality not provided as
standard
![Page 3: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/3.jpg)
Why ?• Reusability
o Component is a DLLo Single code baseo Can be used multiple times in a single projecto Can be shared across multiple projectso Easy to test Component version
• Performanceo Faster than scripting
• Well documentedo Though not a how-to guide
![Page 4: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/4.jpg)
Types Of Component• Data Connections• Log Providers• For Each Loops• Control Flow Tasks• Data Flow Pipeline Component• Custom User Interface
![Page 5: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/5.jpg)
Pipeline Component Types
• Sources• Transforms• Destinations
![Page 6: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/6.jpg)
Design/Run Time• Design Time
o Work done in BIDSo attachments / detachmentso Validationo Column usage
• Run Timeo Metadata interrogationo DTEXECo Flow of data
![Page 7: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/7.jpg)
Demo 1• Reuse and Performance
![Page 8: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/8.jpg)
Performance Comparison
100000 500000 1000000 5000000 10000000 20000000 319996800%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
ScriptCustom
100,000
500,000
1,000,000
5,000,000
10,000,000
20,000,000
31,999,680
Custom 261 900 1,667 7,867 16,375 32,852 51,426Script 684 2,069 3,949 19,214 38,690 76,755 123,243
![Page 9: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/9.jpg)
Requirements• Visual Studio – BIDS is not enough• Or Visual Basic / C# Express• Client Tools SDK
![Page 10: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/10.jpg)
Starting Out• Target Framework 3.5 (Advanced compile options)• Sign the assembley• Add References(Program
file(x86)/<SqlServer>/100/sdk/Assemblies)o Microsoft.SqlServer.DTSPipelineWrapo Microsoft.SqlServer.DTSRuntimeWrapo Microsoft.SqlServer.ManagedDTSo Microsoft.SqlServer.PipeLine Host
![Page 11: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/11.jpg)
Class Creation• Inherits PipelineComponent• Uses attribute DtsPipelineComponent
![Page 12: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/12.jpg)
Post Build• Copy DLL to “C:\Program Files (x86)\Microsoft SQL
Server\100\DTS\PipelineComponents” • Register to Global Assembley Cache using
GACUTIL• Must Restart BIDS• For first use “Choose Items”,”SSIS Data Flow
Components” ,tick Component
![Page 13: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/13.jpg)
MetaData• IDTSComponentMetaData100• PipelineComponent.ComponentMetaData• Describes the Component to the engine• Inputs, Outputs• Custom data held within IDTSCustomProperty100
o Most level s of object
![Page 14: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/14.jpg)
MetaData• Inputs – IDTSInput100
o Exposed via InputCollection member in MetaDatao One instance for each attached inputo Contains virtual column collection
• Accessed with GetVirtualInput() member• View of the IDTSOutput100 of the Upstream component• IDTSVirtualInputColumn100
o Input Column Collection• Accessed with InputColumnCollection• Those that are used in the component• IDTSInputColumn100
o SetUsageType used to add the virtual column to the input column
![Page 15: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/15.jpg)
MetaData• Outputs – IDTSOutput100
o Exposed via OutputCollection member in MetaDatao One class for each outputo output Column Collection
• Accessed with OutputColumnCollection• IDTSOutputColumn100
• Dispositions – Errorso Set IsErrorOut on IDTSOutput100
![Page 16: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/16.jpg)
Icons• Size
• 16*16 For ToolBox• 32*32 For Design Surface• Order of “IconResources” is important
• Build action must be “Embedded Resource”
![Page 17: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/17.jpg)
Errors and warnings• FireError
o At design or run time
![Page 18: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/18.jpg)
Errors and warnings• FireWarning
![Page 19: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/19.jpg)
Design Time Methods• Methods
o ProvideComponentProperties• Define initial metadata of component
o Validate• Tests the metadata is correct
o ReinitializeMetaData• Fix the metadata
![Page 20: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/20.jpg)
Debug
![Page 21: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/21.jpg)
Demo 2• Build a simple component
![Page 22: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/22.jpg)
Run-Time Processing
Pre-Execute
PrimeOutput
ProcessInput
PostExecute
![Page 23: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/23.jpg)
PreExecute• Setup the runtime objects• Interrogate the Metadata and buffer manager• Find the colindex(s) in buffers based on metadata
o BufferManager.FindColumnByLineageID(InputId,InputCol.LineageId)
PrimeOutput
![Page 24: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/24.jpg)
Process Input• Loop on buffer.NextRow• If buffer.EndOfRowset is true set
outputBuffer.SetEndOfRowset()• MetaData functions are
not optimized for performance.
![Page 25: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/25.jpg)
PipelineBuffer• Used for both input and output buffer• Get<DataType> and Set <DataType>
o SetString / GetStringo SetInt32 / GetInt32
• AddRowo Insert and move to new row
• SetEndOfRowseto After final row has been poplulated
![Page 26: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/26.jpg)
Sync Or Async ?• Sync
o Add columns to existing data flowo SynchronousInputID of output = ID of input
• Asynco Create new data flow buffero SynchronousInputID =0
![Page 27: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/27.jpg)
Demo 3• RunTime execution
![Page 28: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/28.jpg)
User Interface
![Page 29: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/29.jpg)
User Interface
![Page 30: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/30.jpg)
User Interface• A Class that implements IDtsComponentUI• Registered to the component class with
UITypeName
• PublicKeyToken is found with GACUTIL
![Page 31: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/31.jpg)
User Interface
![Page 32: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/32.jpg)
User Interface• Demo 4
o User interface o UI Code Step Through
![Page 33: SSIS Custom Components](https://reader036.fdocuments.us/reader036/viewer/2022062310/56815ec9550346895dcd505e/html5/thumbnails/33.jpg)
Conclusion• Like SSIS , large learning curve• Reusability • Potentially Faster ?• .Net skills are required