a Designer

8/3/2019 a Designer

1/86

Mahindra Satyam 2009 1

1

INFORMATICAINFORMATICA

POWERCENTER 8XPOWERCENTER 8X DESIGNER

8/3/2019 a Designer

2/86


Module Objectives

In this module you will learn about

The Power Center Designer Components

Transformations

2

8/3/2019 a Designer

3/86


Designer Overview

Designer has tools to help you build mappings and mapplets so you can specifyhow to move and transform data between source and targets

The Designer has the following tools that we use to analyze sources, design

target schemas, and build source-to-target mappings

Source Analyzer Target Designer Transformation Developer Mapplet Designer

Mapping Designer

3

8/3/2019 a Designer

4/86


Demo: Creating an ODBC DSN

4

8/3/2019 a Designer

5/86


Demo: Creating a Source Definition

5

8/3/2019 a Designer

6/86


Demo: Creating a Target Definition

6

8/3/2019 a Designer

7/86


Designer Overview

7

8/3/2019 a Designer

8/86


Designer Components

Navigator Workspace Output

Status Bar Overview Instance data Target data

8/3/2019 a Designer

9/86


What is a Transformation??

A transformation is a repository object that generates, modifies, or passes data

Transformations in a mapping represent the operations the Informatica Serverperforms on the data

Data passes into and out of transformations through ports that are linked in a

mapping or mapplet

Ports can be one of the three types:

Input data is receivedOutput data is provided

Input/O

utput data passes directly through transformationunchanged

8/3/2019 a Designer

10/86


Designer Transformations

10

Aggregator Expression

Filter

Joiner

Lookup

Rank

Sequence Generator

Source Qualifier

Stored Procedure

Update Strategy

Router

Normalizer

8/3/2019 a Designer

11/86


Transformation Types

Active

Changes the number of rows passing through it

Passive

Do not change the number of rows passingthrough it

Connected

Connected to the transformations in the dataflow

Unconnected

Not connected to any transformations in the

8/3/2019 a Designer

12/86


Types Of Transformation

Active Transformation

Filter

Router

Update Strategy

Aggregator

SorterRank

Joiner

Normalizer

12

Passive transformations

Sequence Generator

Stored Procedure

Expression

Lookup

8/3/2019 a Designer

13/86


Aggregator Transformation

The Aggregator is an active transformation

The Aggregator transformation allow us to perform aggregate calculations,such as averages and sums

The Aggregator transformation is unlike the Expression transformation, inthat we can use the Aggregator transformation to perform calculations on

groups

The Expression transformation permit us to perform calculations on a row-by-row basis only

We can use conditional clauses to filter rows, providing more flexibility

thanS

QL

language

The Integration Services performs aggregate calculations as it reads, andstores necessary data group and row data in an aggregate cache

13

8/3/2019 a Designer

14/86


Components ofAggregator Transformation

Aggregate expression

Entered in an output port

Group by port

Indicates how to create groups

Sorted input

Used to improve session performance

Aggregate cache

Store group values in a index cache and row data in the data cache

14

8/3/2019 a Designer

15/86



15

Aggregation Statement

8/3/2019 a Designer

16/86



Conditional Clauses

We use conditional clauses in the aggregate expression to reduce thenumber of rows used in the aggregation

The conditional clause can be any clause that evaluates to TRUE orFALSE

Null Values in Aggregate Functions

When we configure the Integration Service, we can choose how we wantthe Integration Service to handle null values in aggregate functions

We can choose to treat null values in aggregate functions as

NULL

or zeroBy default, the Integration Service treats null values as NULL in aggregate

functions

16

8/3/2019 a Designer

17/86


Demo : Create an Aggregator Transformation

8/3/2019 a Designer

18/86


Expression Transformation

To calculate values in a single row before we write to the target

To test conditional statements

To perform calculations involving multiple rows, such as sums or averages

To perform any non-aggregate calculations

18

Expression Statement

8/3/2019 a Designer

19/86


Demo : Create an Expression Transformation

8/3/2019 a Designer

20/86


Filter Transformation

Used to filter rows in a Mapping

Active transformation

Specify a filter condition which returns TRUE orFALSE

Records that return TRUE are allowed to pass

Filtered records are lost

20

8/3/2019 a Designer

21/86


Filter Condition

Filter Condition is entered in the Expression Editor of the Filtertransformation

The transformation language is used to enter the filter condition expression

Filter Condition

8/3/2019 a Designer

22/86


Demo : Create an Filter Transformation

8/3/2019 a Designer

23/86


Joiner Transformation

An active transformation

Used to join source data from two related heterogeneous sources residing indifferent locations or file systems

We can also join data from the same source

Joins sources with at least one matching column

Uses a condition that matches one or more pairs of columns between the two sources

We can use the following sources with the joiner transformation

Two relational tables existing in separate databasesTwo flat files in potentially different file systems

Two differentO

DBC sourcesA relational table and an XML sourceA relational table and a flat file sourceTwo instances of the same XML source

23

8/3/2019 a Designer

24/86


Joiner Transformation

Settings for joiner

Master and detail source

One of the sources is

specified as the master source,and the other as the detail

source

Type of join

Normal (Default)

MasterOuter

Detail Outer

Full Outer

8/3/2019 a Designer

25/86


Joiner Transformation Tips

Perform joins in a database when possible

Join sorted data when possible

For an unsorted Joiner transformation, designate the source

with fewer rows as the master source

For a sorted Joiner transformation, designate the source

with fewer duplicate key values as the master source

25

8/3/2019 a Designer

26/86


Demo : Create a Joiner Transformation

a2

8/3/2019 a Designer

27/86

Slide 26

a2 I have done the modifications till hereAjithkumar Menon, 4/10/2008

8/3/2019 a Designer

28/86


Lookup Transformation

27

Used to look up data in a relational table, view, or synonym

Passive Transformation

We can import a lookup definition from any flat file or relational

database to which both the PowerCenter Client and Integration

Service can connect

The Integration Service queries the lookup source based on the lookup

ports in the transformation

Can be Connected/Unconnected

It compares Lookup transformation port values to lookup source

column values based on the lookup condition

8/3/2019 a Designer

29/86


Lookup Transformation

To configure a

LookupTransformation the following

components are defined:

Lookup table

Ports

PropertiesCondition

8/3/2019 a Designer

30/86


Lookup Transformation- Lookup Condition

Look up condition

8/3/2019 a Designer

31/86


Types ofLookup transformationTypes ofLookup transformation

The

Lookup Transformation can be configured in two ways, Connected and

Unconnected, to perform different types of lookups.

Connected

Unconnected

8/3/2019 a Designer

32/86


Connected Lookup Transformation

Input values directly from another transformation in the pipeline

For each input row, the PowerCenterServer queries the lookup source or

cache based on the lookup ports and the condition in the transformation

If the transformation is uncached or uses a static cache, the PowerCenter

Server returns values from the lookup query

If the transformation uses a dynamic cache

It inserts the row into the cache when the row is not found

It updates the row in the cache or leaves it unchanged

It flags the row as insert, update, or no change The PowerCenterServer passes return values from the query to the next

transformation

8/3/2019 a Designer

33/86


Unconnected Lookup Transformation

An unconnected Lookup transformation receives input values from the

result of a :LKP expression in another transformation, such as an Update

Strategy transformation

The PowerCenterServer queries the lookup source or cache based on the

lookup ports and condition in the transformation

The PowerCenter Server returns one value into the return port of the

Lookup transformation

The Lookup transformation passes the return value into the :LKP

expression

8/3/2019 a Designer

34/86


Connected Vs Unconnected Lookup Transformation

Connected Lookup

Receives input values directly

from the pipeline

Cache includes all lookup

columns used in the mapping

If there is no match for the

lookup condition, it returns the

default value for all output

ports

Pass multiple output values to

another transformation

Supports user-defined defaultvalues

More visible, shows where the

lookup values are used

33

Unconnected Lookup

Receives input values from other

transformation calling: LKP

expression

Cache includes all lookup/output ports

in the lookup condition If there is no match for the lookup

condition, returns null

Pass one output value to another

transformation

Does not support user-defined defaultvalues

Less visible, as the lookup is called

from an ex pression within another

transformation

8/3/2019 a Designer

35/86


Lookup Transformation Tips

34

Add an index to the columns used in a lookup condition

Place conditions with an equality operator (=) first

Cache small lookup tables

Join tables in the database

Use a persistent lookup cache for static lookups

Call unconnected Lookup transformations with the :LKP reference

qualifier

8/3/2019 a Designer

36/86


Lookup Caches

The Integration Service builds a cache in memory when it processes the

first row of data in a cached Lookup transformation

It allocates memory for the cache based on the amount we configure in the

transformation or session properties

The Integration Service stores condition values in the index cache and

output values in the data cache

The Integration Service queries the cache for each row that enters the

transformation

The Integration Service also creates cache files by default in the

$PMCacheDir

35

8/3/2019 a Designer

37/86


Demo : Create a Lookup Transformation

8/3/2019 a Designer

38/86


Rank Transformation

Used to select only the top or bottom rank of data

Active transformation

Generates the Rank Index

Grouping can be done

8/3/2019 a Designer

39/86


Demo : Creating a Rank Transformation

8/3/2019 a Designer

40/86


Sequence Generator Transformation

The Sequence Generator

Transformation is used to generatenumeric values It is a Passive Transformation It has two predefined output ports

CURRVALNE

XTVAL

The output ports cannot be changed No ports can be added Sequence Generator Transformation

can be used to generate numericvalues for the following:Create keysReplace missing valuesCycle through a sequential range

of numbers

8/3/2019 a Designer

41/86


Sequence Generator Transformation-Working

The Integration Service generates a value each time a row enters a connected

transformation, even if that value is not used

When NEXTVAL is connected to the input port of another transformation, theIntegration Service generates a sequence of numbers

When CURRVAL is connected to the input port of another transformation, theIntegration Service generates the NEXTVAL value plus one

We might reuse a Sequence Generator when we perform multiple loads to a singletarget

40

8/3/2019 a Designer

42/86



CURRVAL is the NEXTVAL value plus

one orNE

XTVAL

plus the Increment Byvalue.

Connecting CURRVAL and NEXTVALPorts to a Target

We configure the Sequence Generatortransformation as follows: CurrentValue = 1, Increment By = 1

When we run the workflow, theIntegration Service generates thefollowing values for NEXTVAL andCURRVAL.

If we connect the CURRVAL portwithout connecting the NEXTVALport, the Integration Service passes aconstant value for each row

NEXTVAL CURRVAL

1 2

2 33 4

4 5

5 6

41

8/3/2019 a Designer

43/86



Only the CURRVAL Port to aTarget

For example, we configurethe Sequence Generatortransformation as follows

OUTPUTCurrent Value = 1, Increment

By = 1When we run the workflow,

the Integration Servicegenerates the followingconstant values for CURRVAL

CURRVAL

1

1

1

1

1

42

8/3/2019 a Designer

44/86


Demonstration: Sequence Generator

8/3/2019 a Designer

45/86


Source Qualifier Transformation

A Source Qualifier is an active transformation

The Source Qualifier represents the rows that the Integration Service readswhen it executes a session

When we add a relational or a flat file source definition to a mapping sourceQualifier transformation automatically comes

44

8/3/2019 a Designer

46/86



We can use the Source Qualifier to perform the following tasks

Join data originating from the same source databaseFilter records when the Integration Service reads source dataSpecify an outer join rather than the default inner join Specify sorted ports

Select only distinct values from the source Create a custom query to issue a special SELECT statement for the

Integration Service to read source data

45

8/3/2019 a Designer

47/86



To view the default query:

From the Properties tab, select SQL QueryClick Generate SQLClick Cancel to exit

Example of source Qualifier Transformation

We might see all the orders for the month, including order number,order amount, and customer name

The ORDERS table includes the order number and amount of eachorder, but not the customer name. To include the customer name, weneed to join the ORDERS and CUSTOMERS tables.

46

8/3/2019 a Designer

48/86



SQL Query:

We can give query in the Source Qualifiertransformation.

Joining Source Data:`

We can use one Source Qualifier transformationto join data from multiple relational tables.These tables must be accessible from the sameinstance or database server

Sorted Ports:

The Integration Service adds the configured

number of columns to anOR

DER

BY clause,starting from the top of the Source Qualifiertransformation.

The source database sort order must correspondto the session.

47

8/3/2019 a Designer

49/86


Demonstration: Source Qualifier

8/3/2019 a Designer

50/86


Stored procedure Transformation

A Stored Procedure is a passive transformation

A Stored Procedure transformation is an important tool for populating andmaintaining databases. Database administrators create stored procedures toautomate tasks that are too complicated for standard SQL statements

Stored procedures run in either connected or unconnected mode

The mode we use depends on what the stored procedure does and how we planto use it in a session

We can configure connected and unconnected Stored Procedure transformationsin a mapping

49

8/3/2019 a Designer

51/86


Demonstration: Stored Procedure

8/3/2019 a Designer

52/86


Update Strategy

An Update Strategy is an active transformation

When we design a data warehouse, we need to decide what type of informationto store in targets. As part of the target table design, we need to determinewhether to maintain all the historic data or just the most recent changes

The model we choose determines how we handle changes to existing rows. In

PowerCenter, we set the update strategy at two different levels

Within a sessionWithin a mapping

51

Operation Constant Numeric Value

Insert DD_INSERT 0

Update DD_UPDATE 1

Delete DD_DELETE 2

Reject DD_REJECT 3

8/3/2019 a Designer

53/86


Update Strategy Expressions

Frequently, the update strategy expression uses the IIF or DECODE functionfrom the transformation language to test each row to see if it meets a particular

condition

If it does, you can then assign each row a numeric code to flag it for a

particular database operation

For example, the following IIF statement flags a row for reject if the entry

date is after the apply date. Otherwise, it flags the row for update:

IIF( ( ENTRY_DATE > APPLY_DATE), DD_REJECT, DD_UPDATE )

52

8/3/2019 a Designer

54/86


8/3/2019 a Designer

55/86


Update Strategy

Setting the Update Strategy

We use the following steps to define an update strategy

To control how rows are flagged for insert, update, delete, or reject within amapping, add an Update Strategy transformation to the mapping

Define how to flag rows when we configure a session

We can flag all rows for insert, delete, or update, or we can select the datadriven option, where the Integration Service follows instructions coded intoUpdate Strategy transformations within the session mapping

Define insert, update, and delete options for each target when we configurea session

54

8/3/2019 a Designer

56/86


Update Strategy

You can set the following update strategy

options:

Insert. Select this option to insert a rowinto a target table

Delete. Select this option to delete arow from a table

U pdate. You have the followingoptions in this situation:

U pdate as Update. U pdate each rowflagged for update if it exists in thetarget table

U pdate as Insert. Inset each rowflagged for update

Update else Insert. Update the row if itexists. Otherwise, insert it

Truncate table. Select this option totruncate the target table before loading

data 55

8/3/2019 a Designer

57/86


Update Strategy in Session Task

Enter an update strategy expression toflag rows as inserts, deletes, updates, orrejects.

Setting the Update Strategy for aSession

When we configure a session, we

have several options for handlingspecific database operations,including updates.

Specifying an Operation for AllRows

When we configure a session, wecan select a single databaseoperation for all rows using theTreat Source Rows As setting

Configure the Treat Source RowsAs session property 56

8/3/2019 a Designer

58/86


Demonstration:Update Strategy

8/3/2019 a Designer

59/86


Router Transformation

A Router transformation is an Active Transformation.

A Router transformation is similar to a Filter transformation because bothtransformations allow us to use a condition to test data

A Filter transformation tests data for one condition and drops the rows of datathat do not meet the condition. However, a Router transformation tests data forone or more conditions and gives us the option to route rows of data that do notmeet any of the conditions to a default output group

If we need to test the same input data based on multiple conditions, use a Routertransformation in a mapping instead of creating multiple Filter transformationsto perform the same task

58

R T f i

8/3/2019 a Designer

60/86



59

Group tab in Router Transformation

Click the Group Filter Condition field to open the Expression Editor.

Enter a group filter condition.

Click Validate to check the syntax of the conditions we entered.

ClickOK.

Connect group output ports to transformations or targets.

Choose Repository-Save

R t T f ti

8/3/2019 a Designer

61/86



Router Transformation Components

60

A Router transformation has the

following types of groups.

Input

Output

There are two types of output

groups.

User-defined groups

Default group

Working with Ports

A Router transformation has input ports

and output ports.

Input ports reside in the input group,and output ports reside in the outputgroups

We can create input ports by copying

them from another transformation or

Port tab in Router Transformation

R t T f ti

8/3/2019 a Designer

62/86



Connecting Router Transformations in a

Mapping

When we connect transformations to aRouter transformation in a mappingconsider the following rules

We can connect one group to one

transformation or target

Connect one port to Multiple Target

We can connect one output port in agroup to multiple transformations or

targets.

Connect Multiple out ports to Multiple Target

We can connect multiple output ports inone group to multiple transformations or

targets. 61

8/3/2019 a Designer

63/86


Demonstration: Router

R bl T f ti

8/3/2019 a Designer

64/86


Reusable Transformation

Reusable transformation is a transformation that can be used in multiplemappings

We can create most transformations as a non-reusable or reusable but onlycreate the External Procedure transformation as a reusable transformation

When we add a reusable transformation to a mapping, we add an instance of

the transformation. The definition of the transformation still ex

ists outsidethe mapping

Methods To Create Reusable Transformation:

Design it in the Transformation Developer

Promote a non-reusable transformation from the Mapping Designer

63

R bl T f ti

8/3/2019 a Designer

65/86


Reusable Transformation

Creating Reusable Transformation

Create in transformation developer

Promote an existing transformation tore-usable

Changes that can invalidate mapping

When we delete a port or multipleports in a transformation

When we change a port data type, youmake it impossible to map data from that port to another port using anincompatible data type

When we change a port name,expressions that refer to the port are nolonger valid

When we enter an invalid expressionin the reusable transformation, mappingsthat use the transformation are no longer 64

8/3/2019 a Designer

66/86


Demonstration: Reusable

WorkingWith Flat Files

8/3/2019 a Designer

67/86


WorkingWithFlat Files

We can import or create flat file sourcedefinitions in the Source Analyzer orcreate flat file target definitions in theTarget Designer or import flat fileslookups or use existing file definitions ina Lookup transformation

When we create a session with a filesource, we specify a source file locationdifferent from the location we use , whenwe import the file source definition

66

WorkingWith Flat Files

8/3/2019 a Designer

68/86


WorkingWithFlat Files

Editing a flat file definition:Table tab

Edit properties such as table name, business name, and flat fileproperties.

Columns tabEdit column information such ascolumn names, data types, precision,and formats.

Properties tabWe can edit the default numericand date time format properties inthe Source Analyzer and the Target

Designer.Metadata Extensions tab

We can extend the metadata storedin the repository by associatinginformation with repository objects,such as flat file definitions.

67

8/3/2019 a Designer

69/86


Demonstration: Flat Files

User Defined Functions

8/3/2019 a Designer

70/86



We can create user-defined functions using the PowerCenter transformationlanguage

Once you create user-defined functions, we can manage them from the User-Defined Function Browser dialog box. We can also use them as functions in theExpression Editor

We create a user-defined function in the Transformation Developer.Configure the following information when we create a user-defined function

NameTypeDescriptionArgumentsSyntax

69


8/3/2019 a Designer

71/86



70

Steps to Create User-Defined Functions

In the Transformation Developer, click Tools > User-Defined Functions. ClickNew

The Edit User-Defined Function dialog box appears

Enter a function name

Select a function type

If we create a public user-defined function, we cannot change the functionto private when we edit the function.


8/3/2019 a Designer

72/86



Optionally, enter a description of the user-defined function.

We can enter up to 2,000 characters.

Create arguments for the user-defined function. When we create arguments, configure the argument name, data type, precision,and scale. We can select transformation data types.

Click Launch Editor to create an expression that contains the arguments wedefined.

ClickOK The Designer assigns the data type of the data the expression returns. The datatypes have the precision and scale of transformation data types.

ClickOK The expression displays in the User-Defined Function Browser dialog box.

71

8/3/2019 a Designer

73/86


Demonstration: User Defined

Mapplet Designer

8/3/2019 a Designer

74/86


Mapplet Designer

A mapplet is a reusable object that we create in the Mapplet Designer. It containsa set of transformations and we reuse that transformation logic in multiplemappings

When we use a mapplet in a mapping, we use an instance of the mapplet. Like areusable transformation, any change made to the mapplet is inherited by allinstances of the mapplet

Usage ofMapplets

Use multiple source definitions and source qualifiers to provide source data for amapping

Accept data from sources in a mapping

If we want the mapplet to receive data from the mapping, we use an Inputtransformation to receive source data

A mapplet can contain as many transformations as you need

We can create a mapplet to feed data to multiple transformationsContain unused ports

73

MappletDesigner

8/3/2019 a Designer

75/86


MappletDesigner

Limitations ofMapplets

We cannot connect a single port in the Input transformation to multipletransformations in the mapplet

An input transformation must receive data from a single active source

A mapplet must contain at least one Input transformation or source definition

with at least one port connected to a transformation in the mapplet and sameapplies for output transformation

When a mapplet contains a source qualifier that has an override for the defaultSQL query, we must connect all of the source qualifier output ports to the nexttransformation within the mapplet

We cannot include PowerMart 3.5-style LOOKUP functions in a mappletWe cannot include the following objects : Normalizer transformations, Cobolsources, XML Source Qualifier transformations, XML sources and targets, Pre-and post- session stored procedures and other mapplets

74

8/3/2019 a Designer

76/86


Demonstration: Mapplet

Data Profiling

8/3/2019 a Designer

77/86


g

Data profiling is a technique used to analyze source data

PowerCenter Data Profiling can help us to evaluate source data and detectpatterns and exceptions. we can profile source data to suggest candidate keys, detectdata patterns and evaluate join criteria

Use Data Profiling to analyze source data in the following situations`

During mapping development .During production to maintain data quality.

We can create the following types of data profiles

Auto profile

Custom profile

76

Data Profiling

8/3/2019 a Designer

78/86


Steps To Create Auto Profile:

When we create an auto profile, we can profile groups or columns in the source. Or,we can profile the entire source. To create an auto profile. Select the source definition in the Source Analyzer or mapplet in the MappletDesigner you want to profile. Launch the Profile Wizard from the following Designer tools.

77

Data Profiling

8/3/2019 a Designer

79/86


78

We can create a custom profile from the following Designer tools

SourceAnalyzer. Click Sources > Profiling > Create Custom Profile.

Mapplet Designer. Click Mapplets > Profiling > Create Custom Profile.

Profile Manager. Click Profile > Create Custom.

Profile Manager

8/3/2019 a Designer

80/86


Profile Manager

Profile Manager is a tool that helps tomanage data profiles.

It is used to set default data profileoptions, work with data profiles in therepository, run profile sessions, view profile results, and view sources andmapplets with at least one profile definedfor them

When we launch the Profile Manager,we can access profile information for theopen folders in the repository.

There are two views in the ProfileManager

Profile View Source View

79

Profile View

Source View

Debugger Overview

8/3/2019 a Designer

81/86


gg O

We can debug a valid mapping to gain troubleshooting information about data

and error conditions.

Debugger is used in the following situations:

Before we run a session

After we run a session

Create breakpoints

Configure the Debugger

Run the Debugger

Monitor the Debugger

Modify data and breakpoints

80

Debugger Overview

8/3/2019 a Designer

82/86


gg

81

Create Breakpoints

Goto mapping

8/3/2019 a Designer

83/86


gg

Choose connections of

target and source and clicknext.Click on next

Debug Indicators

82

8/3/2019 a Designer

84/86


In this module, we covered Working with PowerCenter 8 Designer

83

SummaryReview

8/3/2019 a Designer

85/86


Review

Designer is used to create mappings that contain transformation instructions for the

Integration Service . (True/False)

A FilterTransformation is an -------------------------- Transformation.

User-defined functions are not available to other users in a repository.(True/False)

Explain briefly about connected and unconnected transformation

84

8/3/2019 a Designer

86/86

THE END

a Designer

Documents

Transcript of a Designer