IKnow Process Visibility Process Management iRule Nordic QualityStage rule sets Petter Enqvist...
-
Upload
roderick-maximilian-underwood -
Category
Documents
-
view
217 -
download
0
Transcript of IKnow Process Visibility Process Management iRule Nordic QualityStage rule sets Petter Enqvist...
iKnowProcess Visibility
Process Management
iRuleNordic QualityStage rule sets
Petter EnqvistSenior systems consultantIntelligent Communication AS
• iKnow provides tools for operating DataStage:
– Monitoring job progress and server health
– Evaluating job statistics to improve performance and optimize process flow
– Process flow design and execution
– Recovery and notifications
• iKnow adds visibility into data integration processes on the IBM InfoSphere Information Server platform
• iKnow is easy to use!
Introduction to iKnow
• Mitigate the risk of the “shrinking batch window” and ensure faster access to business critical information
• Optimize hardware resource usage• Provide process visibility to operators and
managers• Reduce skill requirements for operating
DataStage
Introduction - Benefits
• Real-time monitoring provides early indication of problems, reducing the effects of errors
• Reduced process complexity keeps the costs of managing the organisation’s data integration solutions down
• Add value to existing scheduling environments by extending monitoring and statistics capabilities
Introduction – Benefits cont.
iKnow is designed for DataStage customers with complex data integration requirements:
– Mission critical data integration processes
– Large number of jobs
– Complex interdependencies between jobs
– Enterprise wide scheduling needs: non-DataStage jobs
– Need to monitor and optimize their data integration environment
Introduction - Target audience
?
Process Visibility
3. How do DataStage job performance and hardware resource usage correlate?
iKnow Analytics
• Platform and requirements– Works with DataStage 8.0.1, 8.1 & 8.5
– Windows 2003 & 2008, Linux, AIX and Solaris supported
– Java 1.5 or higher required
– Apache Tomcat v6.0 or WebSphere v6.1 or higher required
– DB2 9.1 – 9.7, Oracle 10g and 11g supported
• Development resources– http://
publib.boulder.ibm.com/infocenter/iisinfsv/v8r5/topic/com.ibm.swg.im.iis.ds.cliapi.ref.doc/topics/c_dsvjbref_WebSphere_DataStage_Development_Kit_Job_Control_Inter.html
Technology
DataStage Engine
iKnow Server
Server
Database server
iKnow Web App
iKnow Client
iKnow Agent
iKnow Agent
iKnow Service
Provider
iKnow Service
ProviderDataStage
Engine
Architecture
iKnow Agent
iKnow Agent
• Main fetaures–Standardisation–Matching
• Data de-duplication• Reference matching
–Survive
• Supporting features– Investigate
QualityStage in a nutshell
• Step 1: Standardisation– Input:
–Output:
QualityStage in a nutshell
Name Address1 Address2 CountryLindkvist, Karl Anders Mannerheimvägen 15 02700 Grankulla FinlandKarl Anders Lindquist Mannerheimv. 15 B 02700 Grankulla FinlanfAnna Lindkvist Mannerheimvägen 15 01300 Helsingfors Finland
FName MName LName StreetName Type StreetComplete HNo HDet Zip City Country
Karl Lindkvist Mannerheim v Mannerheimvägen 15 02700 Grankulla Finland
Karl Anders Lindquist Mannerheim v Mannerheimvägen 15 B 02700 Grankulla Finland
Anna Lindkvist Mannerheim v Mannerheimvägen 15 01300 Helsingfors Finland
FName MName LName StreetName Type StreetComplete Country Set Weight Match
Karl Lindkvist Mannerheim v Mannerheimvägen Finland 1 25,96 MP
Karl Anders Lindquist Mannerheim v Mannerheimvägen Finland 1 16,19 DA
Anna Lindkvist Mannerheim v Mannerheimvägen Finland RA
• Step 2: Matching– Input:
–Output:
QualityStage in a nutshell
FName MName LName StreetName Type StreetComplete HNo HDet Zip City Country
Karl Lindkvist Mannerheim v Mannerheimvägen 15 02700 Grankulla Finland
Karl Anders Lindquist Mannerheim v Mannerheimvägen 15 B 02700 Grankulla Finland
Anna Lindkvist Mannerheim v Mannerheimvägen 15 01300 Helsingfors Finland
• Standard QualityStage structure– NORDICPREP– NORDICNAME– NORDICADDR– NORDICAREA
• Countries covered– Denmark– Finland– Norway– Sweden
iRule - Introduction
iRule - Introduction
Steps:1. Preprocessor
2. Investigate
3. Domain standardisation
4. Investigate
Matching
• Rule set development package– Standardization rules template
• Includes templates for domains: address, area, and name.
– Standardization rules development kit• Consists of a series of jobs that produce reports
to assist in custom rules development.
– Standardization quality assessment kit• Consists of a series of jobs that produce reports
to assist in evaluating standardization results.
QualityStage - What’s new in 8.5
QualityStage - What’s new in 8.5
24
• Summary reporting to understand how well standardisation is performing
• Sections
• Summary results for records assessed
• Percentage of records that populate each individual output category to determining anomalies
• Composition of standardized output patterns
Standardisation quality assessment
QualityStage - What’s new in 8.5
• Introduced a match specification report to capture the details of the match specification
• Includes a glossary and all the pertinent details for the specification and passes
• Enables the Business to review Match Rules more effectively
Match specification report
Intelligent Communication ASAkersgata 49
0180 Oslo
Norge
+ 47 22 89 20 90
http://www.intelcom.no
http://www.iknowsw.com/
Thank you!