602 - Intro to Informatica - dbmanagement.infodbmanagement.info/Books/MIX/OBIEE_602_Intro_to... ·...

BICG University

Course 602 Introduction to Informaticafor the BI Applications

Version 11.11.02.01

2011 BICG University2

Copyright

Copyright 2011, BI Consulting Group, LLC. All rights reserved. No part of this material may be reproduced, stored in a retrieval system or transmitted by any means, electronic, mechanical, photocopy, recording or otherwise, without written permission from the authors.

BICG training materials are intended for the use of students who have paid for and have attended BICG training. Please contact BICG for information on receiving additional training materials: http://www.bicguniversity.com


Course WelcomeCourse Welcome

BI Consulting Groups comprehensive training program reflects the best practices developed after more than 200 implementations of Oracle BI Enterprise Edition (formerly Siebel Analytics). Our curriculum is designed to not only provide a functional understanding of the variety of Oracle BI and data warehousing tools, it also provides a wealth of advice, tips, and tricks designed to fully take advantage of all aspects of OBIEE before, during and after a project.

Our classes operate in an immersion environment, designed to allow the student to become fully engaged in all aspects of the class curriculum through a series of real world exercises. Unlike other trainingorganizations that use trainers who have never actually implemented the software or built a data warehouse, BI Consulting Group taps into our wealth of experienced consultants to act as trainers and mentors. The best practices, tips and tricks developed by our team will allow you to push your data warehouse and Oracle BI Enterprise Edition implementations to the limit.

Mike Jelen[Vice President of BICG University][email protected]

Welcome to BI Consulting Groups Introduction to Informatica for BI Applications Day 1 class!


Agenda

TopicTopic PagePage

Connecting to the Virtual Machines 6

Lesson #1 Data Warehousing and ETL Processes 12

Lesson #2 Overview of ETL Processes with BI Applications 18

Lesson #3 Informatica Server Tools: Administration Console 33

Exercise #3.1 How to Use Administration Console 55

Lesson #4 Informatica Client Tools: Repository Manager 59

Exercise #4.1 How to Use Repository Manager 83

Lesson #5 Informatica Client Tools: Designer 86

Exercise #5.1 How to Use Designer 132

Lesson #6 Informatica Client Tools: Workflow Manager 136

Exercise #6.1 How to Use Workflow Manager 164

Lesson #7 Informatica Clent Tools: Workflow Monitor 166

Exercise #7.1 How to Use Workflow Monitor 175


Post-It Notes

Throughout this course, you will see yellow Post-It notes. These notes identify topics that we may discuss in class, or we may suggest that you review them in your own time. These notes are designed to further your knowledge about Oracle BI Enterprise Edition (OBIEE) and BI BestPractices. We also include our own tips and tricks that will help you when you are developing applications using OBIEE.

Light Bulb: These notes will provide advice, tips or tricks that weve developed over the years, or points to ponder.

Question Mark: These are questions that well be debating in class. Many of these will have no single correct answer, instead, theyre designed to expand your thinking about pertinent topics.

Drafting Table: These are exercises that well be working on in class. At BICG we believe in following standards and best practices, but we dont believe in limiting creativity. These exercises are your chance to flex your creative muscles!


Introduction to Informatica for BI Applications

Connecting to the BICG Virtual Machines

6


Connecting to the BICGU Virtual Connecting to the BICGU Virtual

MachinesMachines

1. Open a web browser and go to http://portal.bicguniversity.com. Enter the username and password you've been provided by your instructor. Confirm that "BICG" has been selected in the Domain field.

2. Click the Bookmark link for your virtual machine.



MachinesMachines

3. A second browser window will open to inform you that your browser is being checked for Java compatibility.

4. You may receive a pop-up informing you that the page you're viewing uses Java. Click OK.

5. If Java is not installed on your system, click the Get Java button to download and install it, then click Try Again.



MachinesMachines

6. If you receive the following certificate warning, click the Always trust content from this publisher checkbox, then click Yes to continue to the site.

7. You may also receive the following security warning when connecting to a virtual machine for the first time. Click the Always trust content from this publisher checkbox, then click Run to continue.



MachinesMachines

8. The first time you connect you will also receive an unknown publisher warning from the Remote Desktop application. Click the Dont ask me again checkbox, then click Connect.

9. If you receive Windows Security Alert regarding Java, click the Unblock button to allow all features of the program to run.



MachinesMachines

8. In most cases, your credentials will be entered automatically and you will be logged in and taken directly to your desktop. However, some combinations of operating systems and browsers will not automatically populate these fields. If this is the case, please enter your login credentials manually, using the username "Administrator" and thepassword "bicg". The domain field should be left blank.

9. At this point, you should see the desktop of the virtual machine set up for your coursework. If you are unable to get to this point, please notify your instructor as soon as possible so we can assist in troubleshooting any issues.

Youre ready to begin the course!

* To send a Ctrl+Alt+Del command to your virtual machine:

Windows: Ctrl+Alt+EndMac: Fn+Ctrl+Opt+Cmd+Del

** To toggle between normal and full screen modes:

Windows: Ctrl+Alt+BreakMac: Cmd+2


Customizations for the BI Applications

Lesson #1

Data Warehousing and ETL Processes

1.12

Lesson #1Lesson #1


Data Warehousing and ETL ProcessesData Warehousing and ETL Processes

Lesson Objectives:

To define terms and processes associated with ETL.

Content:

1. Goals of Data Warehousing

2. ETL Processes

3. ETL Terminology



Goals of Data Warehousing

The data warehouse must make an organizations information easily accessible.

The data warehouse must present the organizations information consistently.

The data warehouse must be adaptive and resilient to change.

The data warehouse must be a secure bastion that protects out information assets.

The data warehouse must serve as the foundation for improved decision making.

The business community must accept the data warehouse if it is to be deemed successful.

The goals identified on this page were taken from Ralph Kimballs The Data Warehouse Toolkit. We highly suggest reading through this book, if you have not done so already.



ETL Processes

ETL is an acronym for a three step process to extract data from source systems and load the data to a data warehouse. The three steps of the ETL process are:

1. Extract and load Staging Tables: Extracts and consolidates data from one or more source systems and loads into the data warehouse staging tables.

2. Transforms the data: Transforms data in the staging tables and computes calculated values in preparation for the load.

3. Load Dimension and Fact Tables: Generates and maintains data warehouse surrogate keys and loads target dimension and fact tables.


ETL processes are further refined to two types of mappings:

1. Source Dependent Extract (SDE) mappings extracts the data from the transactional systems and loads to the data warehouse staging tables. SDE mappings are designed with respect to the sources unique data model.

2. Source Independent Load (SIL) mappings extracts and transforms data from the staging tables and loads to the data warehouse target tables. SIL mappings are designed to be universal with any source.


Data Warehouse Staging Tables

Data Warehouse

Target Tables

ETL Staging Data

Staging Processes

Data Warehouse Staging Tables

ETL Source Data

Extraction Processes

Source System

Databases

There is third type mapping known as a Post Load Processing (PLP) mapping, which is used to load aggregate tables after target tables have been loaded. This mapping is designed to be source independent.


ETL Terminology

ETL the process by which data is extracted from source A and transformed, aggregated and loaded into source B. ETL can be implemented by such mediums as PL/SQL or various ETL development tools.

Full Load the process of extracting all required data from the source and loading it into target tables. A full load will truncate all the target tables.

Incremental Load subsequent loads after the full load, extracting source data deltas.

Star Schema a denormalized schema consisting of a centralized fact and one or more dimensions.

Snowflake Schema a normalized schema with multiple facts at different levels of grain and dimensions containing foreign key relationships to other dimensions.




Check for Understanding

1.What does ETL stand for?

2.How many loads are involved in the ETL process?

3.What is a source?

4.Explain the difference between a star and snowflake schema?


Customizations for the BI Applications

Lesson #2

Overview of ETL Processes with BI Applications

1.19

Lesson #2Lesson #2


Lesson Objectives:

To define terms and processes associated with BI Applications.

Content:

1. BI Applications Terminology

2. Review of ETL Processes Model for BI Applications

Overview of ETL Processes w/ BI Apps.Overview of ETL Processes w/ BI Apps.



BI Applications Terminology

BI Applications Oracle provided pre-built Informatica and DAC repositories that are configured to extract data from different transactional systems and load data into a single data warehouse.

Informatica a 3rd party ETL tool integrated with the BI Applications. SDE and SILmappings are designed and executed using the Informatica Clients and Servers.

Data Warehouse Console (DAC) controls the design and execution of workflows in Informatica via PMCMD commands to the Integration Service.

Metadata any data used to maintain the operations of a system or warehouse. (Ex: logic)

Repository a collection of meta data. Repositories can exists as files or as objects in a database.

Client Graphic User Interface (GUI) tools for Informatica and DAC.

Server data processing machine of either Informatica or DAC. Instructions for the processing are sent from the client tools.

PMCMD Power Mart Command Prompt. A utility used to issue commands to the Informatica Service


Below is a high level diagram of all the BI Application components and their relationships with other BI Application components.



Outlined below are the standard OBIEE components.

The BI Applications package comes with a pre-built metadata repository(RPD) and web catalog(webcat).




Outlined below are the standard ETL components.

The BI Applications package comes with a pre-built Informatica and DAC repositories.


Remember that all of our source data are located in transactional databases.

Examples of transactional systems include Oracle E Business Suite, Peoplesoft, and JD Edwards.



Pending the version of Informatica, the Informatica Server is also known as the Integration Service

The Informatica Server extracts the source data and loads it into the target tables based on the mappings metadata.

DAC issues the start of this process through PMCMD commands that are sent to the Informatica Server.



The pre-built Informatica repository stores workflow, session and mapping meta data.



The Informatica Client Tools are GUI front-end tools responsible for maintaining the Informatica Repository. The tools include Repository Manager, Designer, Workflow Manager, and Workflow Monitor.



The DAC Server is used for issuing PMCMD commands to the Informatica Server.



The DAC Repository maintains the target data warehouses table and index definitions. It also manages Informatica Workflow source and target table dependencies.



The DAC Client is the GUI front-end tool for managing the DAC repository.

The remainder of this course will focus on the programming and functioning of the DAC application.




Check for Understanding

1.What is included with the purchase of a BI Applications package?

2.How does the BI Applications package simplify the ETL process?



Lesson #3Lesson #3

Informatica Server Tool: Administration Console

1.33

Lesson #3Lesson #3


Lesson Objectives:

To understand and execute the functionalities of the InformaticaAdministration Console.

Content:

1. What is the Administration Console?

2. Who uses the Administration Console?

3. Administration Console Terminology and Icons

4. Functions of the Administration Console

5. BI Applications and the Administration Console

6. Exercise 3.1: How to Use the Administration Console

Server Tool: Administration Console Server Tool: Administration Console



What is the Informatica Administration Console?

The Informatica Administration Console is a server tool that manages the domains properties, integration services, repository services, and user security.

Who uses the Informatica Administration Console?

The Informatica Administration Console is used primarily by system administrators.



Administration Console Terminology and Icons

Domain A collection of nodes and services that defines the PowerCenterenvironment. The domain controls the administration of shared services and system resources across a collection of machines.

Node A logical representation of a machine in a domain. A domain can have more than one node and each can be configured to run one or more the application services.

Integration Service An application service contained within the domain that processes, manages and monitors workflows of the PowerCenter.

Repository Service An application service contained within the domain that managesconnections to the repository.

License Key Identifies the terms and conditions of a clients Informatica environment.

Versioning The process of checking out, editing, and checking in repository objects. Only checked-out objects can be modified.



Administration Console Terminology and Icons

Domain Tab - View and edit properties of the domain and objects within the domain.

Logs Tab - View log events for the domain and services within the domain.

Administration Tab - Manage domain users in the Administration tab. This tab appears if you have permission on the domain.

Upgrade Tab Upgrade Repositories and Integration Services. This tab appears if you have permission on the domain.

Manage Accounts Tab Manage your domain user profile. On this tab, you can change your domain user password andupdate your user profile.



Functions of the Informatica Administration Console

1.To stop and start repository services

2.To stop and start integration services

3.To manage and create domain objects

4.To view log events

5.To view and edit domain object properties

6.To manage security

7.To upgrade repositories

8.To create backup repositories



To stop and start repository services:

1.Click on the repository service in the navigator to open the repository service workspace.

2.Locate the Enable button near the top right-hand side of you screen. Click Enable to start the repository services.

1

2

1

2



To stop and start repository services:

3.Click on the Domain to validate the service has started.

4.To stop the repository services, you will click the Disable button. Choose either option in the pop-up widow and click OK.

3

3

4

4


To stop and start integration services:

1.Click on Integration Service in the navigator to open the integration service workspace.

2.Locate the Enable button near the top right-hand side of you screen. Click Enable to start the integration services.


1

2

1

2



To stop and start integration services:

3.Click on the Domain to validate the service has started.

4.To stop the integration services, you will click the Disable button. Choose the option you would prefer in the pop-up and click OK.

3

3

4

4



To manage and create domain objects:

1.Click on the Create drop down in the upper left-hand corner of your screen to display the option available in the domain.

1

1



To view log events:

1.Click on the Logs tab in the upper left-hand corner of your screen. The log view will display all events that have occurred in your domain.

2.Click on the drop-down arrow to the view log filter options and change selection criteria for a services particular log.

3.Change filter criteria and click OK.

You can filter for any one particular service or all of them.

1

2

3

1

2

3



To view and edit domain object properties:

1.Choose the domain object that you would like to edit from the left hand navigator.

2.Click the tabs in the objects workspace to view the properties and configurations for the object selected.

1

2

1

2



To view and edit domain object properties:

3. Click the Edit button in the upper right-hand corner of the properties window that you would like to edit.

Commonly edited domain object properties:

Domain => Properties => Database Properties Node => Properties => Backup Directory Integration Service => Properties => Codepage Integration Service => Processes => Directories Integration Service => Associated Repository Repository => Properties => Database Properties

3

3



To create backup repositories:

1.Click on the Action tab in the repository service.

2.Choose Back Up Contents from the dropdown menu.

1

2

1

2




3.Enter the a repository username (preferably the Administrators), password, and filename for the backup. Click OK.

3

3




4.Validate that Back Up Contents task has succeeded.

5.Double check the file in the backup directory.

REMEMBER: You can change the location of the backup directory by editing the node properties tab.

4

4

5

5



To configure security:

1.Click on the Manage Accounts tab in the upper right-hand corner of the Administration Console.

2.To create a new user, click on the Create User button.

3.Fill in the required prompts for the new user and click OK.

You can also Add and Edit Groups and Roles.

1

2

3

1

3



To configure security:

4.To edit privileges, click on the user in the navigator view of the page.

5.Choose the Privileges tab for the user.

6.Click on the button to edit the users privileges.

NOTE: The Administrator User and Roles privileges cannot be modified.

5

6

5

6

4

4



BI Applications and Administration Console

License Key restrictions

Versioned Repository



NOTE: The license key does not include Informaticas pushdown optimization and parallel processing functions.

License Key restrictions:

The Informatica license key provided by Oracle for the BI Applications should only be used for BI Application objects.



NOTE: This can be overcome if you migrate the BI Applications folders into a non-versioned repository.

Versioned Repository:

The out-of-the-box BI Applications Repository comes with versioning enabled. This setting is listed in the Administration Console and cannot be modified.


Exercise Objectives:

Prepare the Informatica environment for use.

Exercises:

1. Start the repository and Integration Services.

2. Create an administrative user.

3. Create a developer user.

4. Create a user with limited access.

Exercise #3.1: Administration ConsoleExercise #3.1: Administration Console


1. Log in to Administration Console Start All Programs Informatica Power Center

Services Administration Console Username: Administrator Password: Administrator


2. Confirm that $PMRootDir value exists on the VM.


3. Validate the Integration Service $PMSourceFileDir is $PMRootDir/SrcFiles View under Integration Services - Processes

(Why do we do this? The Informatica Workflow Manager reads parameter files from this directory. We must verify its location because it is essential to the success of other Informatica Tools.)


4. Start the Repository and Integration Services.


5. Add a new administrative user Username: BICG-adm Password: welcome1


6. Sign out as the Administrator and sign in as BICG-adm


7. Add a developer user who does not have access to the domain. Username: BICG-dev Password: welcome2(HINT: sign out as BICG-adm and sign back in as BICG-dev.

If you cannot access the Administration Console, you have given BICG-dev the correct access.)


8. Add another developer who does not have access to the domain. Username: OTHER-dev Password: welcome3


Questions & AnswersQuestions & Answers

Introduction to Informatica for the BI Introduction to Informatica for the BI

ApplicationsApplications



Lesson #4Lesson #4

Informatica Client Tools:

Repository Manager

1.61

Lesson #4Lesson #4


Client Tools: Repository ManagerClient Tools: Repository Manager

Lesson Objectives:

Understand and execute the functionalities of the Informatica Repository Manager.

Content:

1. What is the Repository Manager?

2. Who uses the Repository Manager?

3. Terminology and Icons of the Repository Manager

4. Functions of the Repository Manager

5. BI Applications and the Repository Manager

6. Exercise 4.1: How to Use the Repository Manager


What is the Informatica Repository Manager?

The Informatica Repository Manager is a client tool that manages the repository objects.

Who uses the Informatica Repository Manager?

Informatica Repository Manager is used by system administrators and developers, alike.



Repository Manager Terminology


Repository objects Any object found in the repository.

Folder A collection of repository objects that are related by source or target.

Sources Objects that contain source table definitions. Objects can be relational tables, views, XML, flat files, etc

Targets Objects that contain target table definitions.

Transformations Reusable objects used to apply logic within mappings.

Mapplets Reusable objects that contain a set of transformations that can be used in multiple mappings.

Mappings A source to target logic. Mappings do not store data.

Tasks Used in the Workflow Manager to help design workflows.


Repository Manager Terminology


Sessions A special type of task used in the Workflow Manager to define mapping connection information.

Workflows A collection of sessions and tasks.

Configurations A predefined set of parameters that are applied to sessions.

Export/Import XML A way to migrate objects between folders and repositories.

Drag and Drop A way to migrate objects between folders and repositories.


Functions of the Informatica Repository Manager

1.To configure connections domains and add repositories

2.To view the Informatica repository objects and dependencies

3.To create new folder structures

4.To manage folder structures and security

5.To migrate Repository objects

6.To Check Out/In Repository objects



To configure domains connections and add repositories:

1.Click on the Repository tab. Choose Configure Domains


1

1


To configure domains connections and add repositories:

2.Click on the icon to add a new domain. Enter the domain name, gateway host, gateway port. Click OK.

3.In the list of repositories, highlight the repositories you would like to add.


2

3

2

3



To view the Informatica Repository objects and dependencies:

1.Double click on the icon to open the repository.

1

1



2.Click on a folder to open and view the contents.


The folders seen in the left hand navigator are the BI Prebuilt Adaptors for the 7.9.6.1 BI Applications.

2

2



3.Right click on an object and choose View Dependencies


3

3



4.Check objects to compare. Click OK.

5.Objects listed are the dependent objects within the folder.


5

5

4

4


To create new folder structures:

1.Highlight repository to create the new folder.

2.Click Folder tab. Choose the Create option in the drop down menu.

3.Give the new folder a name and description. Click OK.


1

2

3

1

2

3


To manage folder structures and security:

1.Highlight the folder that you would like to change the permissions on.

2.Click Folder tab. Choose Edit from the drop down menu.

3.Click on the Permissions tab. You can change the permissions of any user here by clicking on Add and modifying the users properties.


3

1

2

3


To migrate Repository objects:

1.Select and highlight object to be exported.

2.Click on the Repository tab and choose Export objects... from the drop down menu.


Make a mental note of where you have placed your export. You will need to retrieve it from this location during the import.

1

2

1

2



3.Highlight the folder in the left-hand navigator where you would like your imported objects to go.

4.Click on Repository and choose Import objects


3

3

4

4



5.Locate your exported XML.

6.Highlight the objects to import and click on the button. Click Next.


6

5

6



7.Choose destination folder by clicking on current destination folder. Highlight desired folder and click OK.

8.Resolve conflicts, if necessary.

9.Click Import.


7

8

9

7


To Check Out/In Repository Objects:

1.In the Repository Navigator, highlight the object you want to modify.

2.Click on the Versioning tab and select Check Out


1

2

1

2



3.Enter check-out comments. Click OK.


A check-out comment notes what you intend to modify during the check out. Although not required, it is best practice to complete this step.

To undo a Check Out, click on the Versioning tab and choose Undo Checkout.

3

3



4.To Check In a repository object, high light the object you want to Check In.

5.Enter Check In comments. Click OK.


Though not the only Client Tool capable of checking out/in objects, the Repository Manager allows the user to check out/in objects, from Sources to Workflows.

5

5

4

4


BI Applications and Repository Manager

Create Custom folders for out-of-the-box object modifications



All customized objects should be migrated to this folder. NO EXCEPTIONS


Create Custom folders for out-of-the-box object modifications:

It is best practice to create Custom_ folders when customizing out-of-the-box mappings and workflows. This will allow you and other developers to easily identify customizations made to the out-of-the-box objects.


If you migrate workflows using the export/import method, all dependencies will be included.


Create Custom folders for out-of-the-box object modifications:

Migrating objects to the Custom_ folders will require the migration off all dependent objects, including workflows and sessions. This may result in the migration of dozens of objects.



Prepare the Informatica environment for use.

Exercises:

1. Create a new folder in the repository

2. Allow an administrator to access a folder

3. Deny a user access to a folder

4. Migrate repository objects into a folder

Exercise #4.1: Repository ManagerExercise #4.1: Repository Manager


1. Log in to Repository Manager Start All Programs Informatica Power Center

Client Repository Manager Username: Administrator Password: Administrator


2. Create a new folder under BICG_BI_DW. Title: Custom_BICG_602


3. Add BICG-adm to the user list under folder permissions. Validate access to ALL permissions.


4. Add OTHER-dev to the user list under the folder permissions. Validate that the user has access to NO permissions.

(When there are multiple projects working on the same repository, this gives us the ability to control who has access to the specific folders. It also allows us to create read only users for the production repository.)


5. Using the export/import method, migrate HZ_PARTIES table from the SDE_ORA11510_Adaptor Sources folder to Custom_BICG_602.




Lesson #5Lesson #5

Informatica Client Tools: Designer

1.90

Lesson #5Lesson #5


Client Tools: DesignerClient Tools: Designer

Lesson Objectives:

Understand and execute the functionalities of the Informatica Designer.

Content:

1. What is Designer?

2. Who uses Designer?

3. Terminology and Icons of Designer

4. Functions of Designer

5. BI Applications and the Designer

6. Exercise 5.1: How to Use Designer


What is the Informatica Designer?

The Informatica Designer is a client tool used to create instructions for the Integration Service. The instructions consist of source and target definitions, along with transformation objects used to transform the data.

Who uses the Informatica Designer?

Informatica Designer is used by system administrators and developers, alike.




Source Analyzer Used create, import, and view source objects.

Target Designer Used to create, import, and view target objects. The target designer can also be used to create tables in the database.

Transformation Developer Used to create, edit and validate reusable transformations.

Mapplet Designer Used to create, edit, and validate reusable mapplets.

Mapping Designer Used to create, edit, and validate mappings the Integration Service will use to extract, transform and load the data.

Workspace Window Used to view or edit objects. Only one workspace can be open at a time.

Navigator Window Used to view objects in repositories. Navigator objects are dragged into the workspace.

Designer Terminology and Icons


Designer Terminology and Icons


Primary key Unique key used to maintain referential integrity and insert updates and deletes.

Port A link that connects source, transformation, and target data together.

Transformation Types

Source Qualifier generates and queries the sources tables for data extraction.

Expression applies logic that transforms data. These transformations are flexible in their abilities.

Sequence Generator generates a nd assigns unique key values.

Aggregator performs aggregate functions such as average, sum, maximum, minimum, etc

Lookup performs queries on data outside of the source qualifier.

Filter filters data that meet specific criteria.

Router directs data that meet group conditions to different transformation or targets.

Update Strategy contains logic that determines how inserted source data is applied to the target tables.

Mapplet Input Gateway objects for ports in and out of mapplets.

Mapplet Output Gateway objects for ports in and out of mapplets.


Functions of Informatica Designer

1.To view and import sources

2.To view and import targets

3.To create mapplets

4.To write or generate source extract query

5.To create mappings

6.To transform the data

7.To cleanse the data

8.To pass mapping parameters



To view and import sources:

1. To view a source object, enter the Source Analyzer tool.

2. Drag and drop the source table from the Sources folder into the Source Analyzerworkspace.

3. Double-click on the source to view source properties.


1

2

3

1

2

3



4. Click on the Columns tab to view source columns and definitions.


Allowed to be modified by hand, it is a best practice to import source objects directly from the source system

4

4



5. To import object from source, enter the Source Analyzer tool.

6. Click on the Sources tab. Choose the correct import type.


5

6

5

6



7. To Import from Database..., enter the ODBC data source, username, and password.

8. Highlight tables and click OK.


7

8

7

8


To view and import targets:

1. To view a target object, enter the Target Designer tool.

2. Drag and drop the target table from the Targets folder into the Target Designerworkspace.


Notice that the drop down options at the top of the screen have changed. These drop downs change as you move through the different tools in Designer.

1

2

1

2



3. Double-click on the target to view target properties.

4. Click on the Columns tab to view target columns and definitions.


3

4

4



5. To import object from target, enter the Target Designer tool.

6. Click on the Targets tab. Choose the correct import type.


6

5

6



7. To Import from Database..., enter the ODBC data source, username, and password.

8. Highlight tables and click OK.


7

8

7

8


To create mapplets:

1. Enter the Mapplet Designer tool.

2. Click on the Mapplet tab, and select Create from the drop down menu.

3. Name your mapplet with prefix mplt_.


1

2

3

2

3


To create mapplets:

4. Drag and drop your source tables from the Sources folder in the navigator. The sources will appear green in your Mapplet Designer.


Notice a Source Qualifier transformation is also created in the mapplet as the source is introduced.

4

4


To create mapplets:

5. Add an Mapplet Output transformation by clicking on the Output icon and then clicking in the Mapplet Designer workspace.


5

5


To create mapplets:

6. Connect source qualifier ports to output transformation.


The ports connected to the output transformation will be the input ports into the mapping.

6

6


To write or generate source extract query:

1. Double-click on the source qualifier box (yellow) in the Mapplet Designer workspace.

2. Click on the Properties tab.


1

2

1

2



3. Click on the icon in the row labeled Sql Query.

4. Click Generate SQL to view the SQL that will be executed at run time. Click OK.


Notice ONLY output ports connected from the Source Qualifier transformation to the Output transformation are generated.

3

3

4

4




Other Properties Fields used to help generate extract query: User Defined Join Joins multiple source objects Source Filter Applies filter conditions in the form of a WHEREclause Select Distinct Returns distinct records


To create mappings:

1. Enter the Mapping Designer tool.

2. Click on the Mapping tab, and select Create from the drop down menu. Name your mapping.

3. Drag and drop your mapplet from the Mapplet folder in the navigator. It will appear light blue in your Mapping Designer workspace.


1

2

3

1

2

3


To create mappings:

4. Add a non-reuseable expression transformation to the mapping by clicking on the icon and then clicking in the Mapping Designer workspace.

5. Connect the ports from the mapplet to the expression transformation.


5

5

4

4


To create mappings:

6. Add a non-reuseable filter transformation to the mapping by clicking on the icon and then clicking in the Mapping Designer workspace

7. Connect the ports from the expression transformation to the filter transformation.


6

7

6

7


To create mappings:

8. Drag and drop your target table from the Target folder into the Mapping Designer workspace.

9. Connect your filter transformation ports to your target table.


8

9

8

9


To transform the data:

1. In the mapping designer workspace, double-click on the expression transformation object. Select the Ports tab.


The three types of ports in expression transformations: I = Input a port connection by a previous transformation. O = Output a port connection to be used in a future transformation. V = Variable an internal port used to store data.

1

1



2. Click the port you would like to transform. Click on the icon in the upper right-hand corner.


Common Icons in Transformations:

- Used to add a new port

- Used to delete a port

- Used to copy a port

- Used to paste a port

- Used to move ports up and down the list

2

2



3. Click on the icon to paste the port. Rename the new port with OUT_ as a prefix.

4. Uncheck O for the original port, and uncheck I for the new port.

5. Click on the icon in the Expression column.


Notice how the text is no longer shadowed and expression column has pop-up icon.

5

3

5

3

4

4



6. Highlight and delete the contents in the Formula: box.

7. Under the Functions tab, double-click on the function you want to apply to the port.


The highlighted function will be defined and its outcome described in the gray area below the list of functions.

67

6

7



8. Under the Ports tab, double-click on the input port you want to transform. Click OK.


Click on the Validate button to test the syntax of the formula.

8

8



9. The new output port will appear as a new row of the transformation. Reconnect this new port to its associated port in the next transformation.


Only ports with a can be linked to other ports.

9

9


To cleanse the data:

1. In the Mapping Designer tool, double-click on the filter transformation object.

2. Select the Properties tab.

3. Click on the icon to open the formula box.


The default Filter Condition will always appear as True. This means that no records are filtered.

2

3

1

2

3


To cleanse the data:

4. In the Formula: box, enter the function and/or filtering condition on a port you would like to apply to your mapping. Click OK.


Your filtering conditions do not have to include a function or a true or false statement. Ex: STATE = CA is a valid filter condition.

4

4



To pass mapping parameters:

1. Enter the mapping designer tool.

2. Click on the Mappings tab and select Parameters and Variables

2

1

2



3. Click on the icon to add a new parameter.


Parameters are values that are used to make port values dynamic.Ex: ETL_PROC_WID is a unique value assigned to each run of the mapping by the integration service.

3

3



4. Define the parameter name, datatype and precision. Also define the type as Parameteror Variable. Click OK.


Mapping level parameters are defined with $$. Session level parameters are defined with $.

4

4



5. In the mapping designer workspace, double click on the expression transformation. Click on the Ports tab.

6. Add new port by clicking on Add icon. Define port name, datatype and precision. Make an Output port by deselecting Input checkbox


6

5

6



7. Click on popup icon and select Variable tab. Open Mapping parameters folder and double click desired parameter. Click OK.


7

7



8. Connect parameterized output port from expression transformation to the target table through the filter transformation .


8

8


BI Applications and Designer

Mapplets generally contain the Source Qualifier transformation

Source Qualifiers have Sql Overrides

Common SDE/SIL Parameters

Dimension Lookups in Fact Table SILOS

System Target Table Columns

Update Logic



Mapplets generally contain the Source Qualifier transformation:

SDE are most commonly developed by Oracle with a Source Qualifier transformation inside a mapplet.

This creates a clean appearance and allows for the source extract to be reusable.




Source Qualifiers have SQL Overrides:

Almost all SDE mappings will contain SQL overrides written in the Source Qualifier transformation.

This allows the SDE mappings to efficiently push table joins to the source and help spread resource consumption across the environments.



Common SDE/SIL Parameters:

Most mappings will contain at least two parameters within their transformations.

SILOS:$$ETL_PROC_WID Unique identifier for a ETL Process$$TYPE2_FLG Type 2 Slowly Changing Dimension Indicator$$INSERT_ONLY_FLAG Bypasses Update Logic

SDE:$$DATASOURCE_NUM_ID Unique identifier for a source$$TENENT Used to help isolate data$$LAST_EXTRACT_DATE Last ETL Process date



Dimensional Lookups in Fact Table SILOS:

Fact table SIL mappings will contain most dimensional table lookup transformations in mapplets. This also helps create cleaner mappings, which are easier to trace and debug.



System Target Table Columns:

All target tables contain 10 to 15 system columns. These columns are used for, but not limited to, maintaining insert/update history, update transformation references, delete status and source identifier.



Update Logic in SILOS:

SILOS mappings contain complex insert/update calculations. The logic varies by mappings, but generally the calculation compares the source with the target and calculates if there is a difference (Ex. SYSTEMS_COLS_DIFF) or if it should be soft deleted.

Update Flags:I - InsertU UpdateD Soft DeleteB Insert and Soft DeleteX Ignore/Reject



Execute a design plan within Informatica Designer.

Exercises:

1. Import a database object into Informatica Designer

2. Create a mapplet

3. Create a mapping

4. Create a transformation

5. Connect source, mapplet, mapping, and target

Exercise #5.1: DesignerExercise #5.1: Designer


1. Log in to Designer Start All Programs Informatica Power Center

Client Designer Username: Administrator Password: Administrator


2. Open Custom_BICG_602


3. Open Target Designer view in the workspace.

4. Import WC_PARTY_DS from database into Target Designer using the BICGU_602_DW ODBC Connection. Username: BICGU_602_DW Password: bicgu_602_dw



5. In Mapplet Designer, create a mapplet using HZ_PARTIES as the source. Name: MPLT_PartyDimension. From the Source Qualifier: map ports PARTY_ID,

PARTY_NUM, PARTY_NAME, CREATION_DATE, and STATE to the Mapplet output expression.

6. In Mapping Designer, create a new mapping with the mapplet MPLT_PartyDimension. Title: SDE_PartyDimension.



7. Create an expression transformation in the mapping. Create an output port. Name the port

OUT_PARTY_NAME. Write an Initcap function using the PARTY_NAME

input port.

8. Create a filter transformation in the mapping. Write a filter condition on the state port that filters on

STATE = MN.



9. Connect the expression transformation ports to the filter transformation ports.

10. Drag and drop the target table from the Target folder into the Mapping Designer workspace.



11. Connect the filter transformation ports to the target table ports. Save your work!!!




Lesson #6Lesson #6


Workflow Manager

1.144

Lesson #6Lesson #6


Client Tools: Workflow ManagerClient Tools: Workflow Manager

Lesson Objectives:

Understand and execute the functionalities of the Informatica Workflow Manager.

Content:

1. What is Workflow Manager?

2. Who uses Workflow Manager?

3. Functions of Workflow Manager

4. BI Applications and the Workflow Manager

5. Exercise 6.1: How to Use Workflow Manager


What is the Informatica Workflow Manager?

The Informatica Workflow Manager is a client tool that executes workflows for the mappings that have been developed in Designer.

Who uses the Informatica Workflow Manager?

Informatica Workflow Manager is used by system administrators and developers, alike.




Task Developer Used to create and view sessions, commands, and emails executed in a workflow.

Worklet Designer Used to create and view worklet

Workflow Designer Used to create, view and schedule workflows by connecting together tasks and worklets.

Task A session, command or email used in workflows

Worklet An object that groups sets of tasks.

Workflow A set of instructions that tells the Integration Service how and when to run tasks such as sessions, email notifications, and shell commands.

Connection An object that defines a connection in the repository. Connections can be databases, ftp sites or applications

Workflow Manager Terminology and Icons




$source / $target Parameterized source and target connections defined within a session, respectively. They are only set once per session.

Parameter File A list of values read by the Integration Service during runtime.

Unoverride A button used to revert back to an original value in a workflow or session.

Normal Load A load that invokes database transactional logging. Default load for any session that contains an update transformation.

Bulk Load A load that bypasses database transactional logging.

Full Load An initial load of data from the source containing historic data.

Incremental Load Subsequent loads that bring in new or modified data since the previous ETL process.




Tasks

Assignment assigns a value to a workflow variable,

Command executes a shell command during the workflow.

Control stops or aborts the workflow.

Decision specifies a condition to evaluate.

Email sends emails during workflow.

Event-wait waits for an event to occur.

Event-raised notifies the event wait task that an event has occurred.

Session runs a mapping that has been created in Designer.

Timer waits for a timed event to occur.

These Tasks are available only in the Workflow Designer.


Functions of Informatica Workflow Manager:

1. To define source and target connection information

2. To create tasks

3. To create workflows

4. To alert successes and failures

5. To override mapping level properties


Uses of Workflow Manager not applicable to BI Applications: To define the order of an execution plan To create worklets


To define source and target connection information:

1. Click on the Connections tab. Choose Relational to create a database connection object.

2. Choose New to create a new connection in the Relational Connection Browser.

3. In the Select Subtype window, choose the database platform to which you are connecting.


1

1

2

3


To define source and target connection information:

4. Name your Connection. Enter your username, password, and connection string.


A connection string for an Oracle connection object is the entry in the tnsnames.ora file. You can find this file in the Oracle Client /network/admin directory.

4

4


To create tasks:

1. Click on Task Developer tool.

2. Click on the Task tab. Choose Create to create a new Task.


1

2

1

2


To create tasks:

3. Choose Session from the dropdown menu. Name your session and click Create.

4. Choose the mapping for your session from the popup menu. Click OK.

5. Click Done to create the task in the workspace.


3

5

3

4

4


To create tasks:

6. Double-click on your session in the Task Developer workspace.

7. Click on the Mapping tab.

8. Click on the icon in the Sources folder to open the designer mappings source qualifier transformation.


All of the mappings transformations are listed in the left-hand navigation list.

7

8

6

7

8


To create tasks:

9. Click on the icon under the Connections Value column and choose your source connection.

10. Click on the icon in the Targets folder to open the designer mappings target table.

11. Click on the icon under the Connections Value column and choose your target connection.


In the Properties section of the target table view, you can change the insert, update, and delete properties and truncate options.

9

11

9

10

11


To create tasks:

12. Click on Files, Directories, and Commands in the left-hand column.

13. Under the Parameter Filename, write $PMSourceFileDir\ and the parameter filename.

14. Click OK to close the Session task.


REMEMBER: $PMSourceFileDir is a value that is validated in the Administrative Console. It identifies where parameter files are located on the Integration Services Node.

13

12

13

14


To create workflows:

1. Click on the Workflow Designer tool.

2. Click on the Workflow tab at the top of the screen. Choose Create to create a new workflow.


1

2

1

2


To create workflows:

3. Name the workflow and click OK.

4. Drag and drop your session into the Workflow Designer workspace.

5. Connect the icon to the session using the task link tool ( ).


5

5

3

4

4


To alert successes and failures:

1. Double-click on the session in the Workflow Designer workspace.

2. Under the General tab, check box Fail parent if this task fails and Fail parent if this task does not run. Click OK.


Failure alerts on the session tasks can only be set in the Workflow Designer.

1

2

2


To override mapping level properties:

1.Click on the Task Developer tool.

2.Double-click on the session. Click on the Mappings tab.

3.Click on the source in the Source folder.

4.Scroll down and click in the Value column of the attribute that you would like to override in the Properties section.


Common attributes to override: SQL Query Source Filter transformationsConnections

1

2

3

4

4


To override mapping level properties:

5.Click on the icon to open the formula box. This is where you can override the mapping level property.

6.After entering your new formula, click OK to view the changes in the Values column.

7.To revert your changes, click on the Revert button that appears in the Unoverride column.


The BI Applications use SQL Query overrides to create unique full and incremental load sessions.

5

6 7

5

6

7


BI Applications and Workflow Manager

One Session per Workflow

Full and Incremental Workflows

Connections by Parameters

Parameter Files




The Workflow Managers flexibility allows for dynamic and complex workflow to be designed and executed. These workflows contain dozens of decision and event tasks to help guide the Integration Service through the execution order of sessions and worklets. The downside of this approach is that development takes extreme care and may takes days or weeks to fine tune.

To our luck, the BI Applications take the guess work out of the execution plan development by providing a tool called the Data Administration Console a.k.a. DAC.


NOTE: Take our 603 Introduction to DAC course!



In order for DAC to build an execution order based on session (mapping) dependencies, there must be a one to one relationship between workflows and sessions.


One of the few exceptions to the rule are found in Dimensional Full Load workflows.



To keep mapping maintenance at a minimum, the BI Applications created two sessions for each mapping, Full and Incremental. These session will be identical with the exception of the Sql Override.


If you make a customization to a mapping, you will need to migrate both Full and Incremental workflows to the Custom folder.



Full Load Session:

Incremental Load Session:




The sessions source and target connections are managed by $DBConnection_OLTP and $DBConnection_OLAP parameters. Parameterized connection values allow connection objects referenced by the session to change without the need for modifying the connection objects.


NOTE: Lookup transformations connections can be even further parameterized by $Source/$target.



Connection values passed via parameters must match the connections objects established in the Connections tab.



Parameter Files

Parameter values are read by the Integration Service and are passed down into session, mappings or mapplets. Parameter values are sourced from .txt files located on the Integration Services $PMSourceFileDir\ directory.

The parameter file name is specified in the sessions Properties tab under the Parameter Filename attribute.


Parameter files are generated by the DAC Server for each workflow at run time and copied to the Integration Service.


Parameter Files

Sample Parameter file.


Common Parameters:$DBConnection_OLTP Operational System$DBConnection_OLAP Target Database$$DATASOURCE_NUM_ID Source Identifier$$ETL_PROC_WID ETL Identifier



Create and initiate a successful workflow in Informatica Workflow Manager.

Exercises:

1. Create a new session in Workflow Manager.

2. Create a new workflow.

3. Monitor alerts in Workflow Manager.

4. Start a workflow.

Exercise #6.1: Workflow ManagerExercise #6.1: Workflow Manager


1. Log in to Workflow Monager Start All Programs Informatica Power Center

Client Workflow Manager Username: Administrator Password: Administrator

2. Open Custom_BICG_602



3. Open Task Designer view in the workspace.

4. Create a new session and title it s_SDE_PartyDimension.



5. Assign a connection value of ORA_11_5_10 to the source qualifier.

6. Assign a connection value of DataWarehouse to the target table.



7. In the Workflow Designer view, create a workflow that is titled wf_SDE_PartyDimension.

8. Add and link s_SDE_PartyDimension to your workflow.



9. Check alert successes and failures for s_SDE_PartyDimension.

10. Right-click in Workflow Designer workspace and choose Start Workflow.




Lesson #7Lesson #7


Workflow Monitor

1.179

Lesson #7Lesson #7


Client Tools: Workflow MonitorClient Tools: Workflow Monitor

Lesson Objectives:

Understand and execute the functionalities of the Informatica Workflow Monitor.

Content:

1. What is Workflow Monitor?

2. Who uses Workflow Monitor?

3. Functions of Workflow Monitor

4. Exercise 7.1: How to Use Workflow Monitor


What is the Informatica Workflow Monitor?

The Informatica Workflow Monitor is a client tool that monitors workflows and records workflow activity.

Who uses the Informatica Workflow Monitor?

Informatica Workflow Monitor is used by system administrators and developers, alike.



Functions of Informatica Workflow Monitor:

1.To read workflow logs

2.To read session logs

3.To report workflow and session properties.



To read workflow logs:

1. Click on the workflow you would like to view.

2. Right-click on the workflow and choose Get Workflow Log from the dropdown menu.


1

2

1

2


To read workflow logs:

3. All workflow activity is listed in the popup table.


Noteworthy Workflow Activity: Session names Parameter file values Duration

3

3


To read session logs:

1. Click on the session you would like to view.

2. Right-click on the session and choose Get Session Log from the dropdown menu.


1

2

1

2


To read session logs:

3. All workflow activity is listed in the popup table.


Noteworthy Session Activity: Connection information SQL executed Transformation properties Number of records extracted Number of records loaded Runtime statistics

3

3


BI Applications and Workflow Monitor

N/A




Monitor a workflow within the Workflow Monitor.

Exercises:

1. Connect to the integration service.

2. Open a workflow log.

3. Monitor a workflow log.

Exercise #7.1: Workflow MonitorExercise #7.1: Workflow Monitor


1. Log in to Workflow Monitor Start All Programs Informatica Power Center

Client Workflow Monitor Username: Administrator Password: Administrator

2. Connect to the Integration Service.



3. Open Custom_BICG_602.4. Open the workflow log for wf_SDE_PartyDimension.



5. Open session log for s_SDE_PartyDimension. Validate there were no Errors and find the number of

successful target records loaded.



Thank You

Thank you for taking the Introduction to Informatica

for the BI Applications course!

Please visit the BICG University site at http://www.bicguniversity.com for information on additional courses.


About BI Consulting GroupAbout BI Consulting Group

Contact Information:

For more info on BICG please visit http://www.biconsultinggroup.com

Oracles premier BI consulting partner for Oracle BI EE

Collectively completed more than 280 Oracle BI Enterprise Edition projects

End-to-end Oracle BI provider with implementation, education, support, and product-based services.

Average consultant has 4+ years of Oracle BI Enterprise Edition expertise, and more than 10 years of BI/DW experience.

Experts at Oracles Pre-Built Analytic Applications including Financial, HR, Supply Chain, Sales, Service and Marketing Analytics.

Participant Oracle Partner Advisory Council


Notable BICG ReferencesDeployed Oracle RMW

for Service Analytics

after failed deployment

by other Systems

Integrator.

Deploying Service

Analytics and a custom

data warehouse

across international

divisions.

Multiple projects

deployed across

multiple divisions,

including a Price

Scenario

application allowing

what-if

forecasting.

Sales Analytics

application

against existing

EDW.

Deployment of

Oracles Pharma

Analytics, after failed

implementation by

another Systems

Integrator.

Deployment of Order

Analytics against a

new, custom EDW.

Redeveloped dozens of

existing Dashboards using

BICGs best practices

methodology; and

developed several new

custom data warehouses.

Integrating

multiple source

systems together

in a unified front-

end Dashboard.

Other Notable Customers:

American Express

Fidelity

Cbeyond

Shopzilla

Bayer

Medtronic Emergency

Response

RBC Dain

UBC

Pharmaceutical

Life Fitness

Benderson

Development

602 - Intro to Informatica - dbmanagement.infodbmanagement.info/Books/MIX/OBIEE_602_Intro_to... ·...

Documents

Transcript of 602 - Intro to Informatica - dbmanagement.infodbmanagement.info/Books/MIX/OBIEE_602_Intro_to... ·...