ABC Practice

8/20/2019 ABC Practice

1/246

Informatica PowerCenter Express(Version 9.6.0)

Transformation uide


2/246

Informatica PowerCenter Express Transformation Guide

Version 9.6.0January 2014

Copyright (c) 2009-2014 Informatica Corporation. All rights reserved.

This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on useand disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted inany form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software may be protected by U.S.and/or international Patents and other Patents Pending.

Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and asprovided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013©(1)(ii) (OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14

(ALT III), as applicable.

The information in this product or documentation is subject to change without notice. If you find any problems in this product or documentation, please report them to usin writing.

Informatica, Informatica Platform, Informatica Data Services, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange,PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data Explorer, Informatica B2B Data Transformation, Informatica B2B Data Exchange InformaticaOn Demand, Informatica Identity Resolution, Informatica Application Information Lifecycle Management, Informatica Complex Event Processing, Ultra Messaging andInformatica Master Data Management are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world.

All other company and product names may be trade names or trademarks of their respective owners.

Portions of this software and/or documentation are subject to copyright held by third parties, including without limitation: Copyright DataDirect Technologies. All rightsreserved. Copyright © Sun Microsystems. All rights reserved. Copyright © RSA Security Inc. All Rights Reserved. Copyright © Ordinal Technology Corp. All rightsreserved.Copyright© Aandacht c.v. All rights reserved. Copyright Genivia, Inc. All rights reserved. Copyright Isomorphic Software. All rights reserved. Copyright © MetaIntegration Technology, Inc. All rights reserved. Copyright © Intalio. All rights reserved. Copyright © Oracle. All rights reserved. Copyright © Adobe SystemsIncorporated. All rights reserved. Copyright © DataArt, Inc. All rights reserved. Copyright © ComponentSource. All rights reserved. Copyright © Microsoft Corporation. Allrights reserved. Copyright © Rogue Wave Software, Inc. All rights reserved. Copyright © Teradata Corporation. All rights reserved. Copyright © Yahoo! Inc. All rightsreserved. Copyright © Glyph & Cog, LLC. All r ights reserved. Copyright © Thinkmap, Inc. All rights reserved. Copyright © Clearpace Software Limited. All rightsreserved. Copyright © Information Builders, Inc. All rights reserved. Copyright © OSS Nokalva, Inc. All rights reserved. Copyright Edifecs, Inc. All rights reserved.Copyright Cleo Communications, Inc. All rights reserved. Copyright © International Organization for Standardization 1986. All rights reserved. Copyright © ej-

technologies GmbH. All rights reserved. Copyright © Jaspersoft Corporation. All rights reserved. Copyright © is International Business Machines Corporation. All rightsreserved. Copyright © yWorks GmbH. All rights reserved. Copyright © Lucent Technologies. All rights reserved. Copyright (c) University of Toronto. All rights reserved.Copyright © Daniel Veillard. All rights reserved. Copyright © Unicode, Inc. Copyright IBM Corp. All rights reserved. Copyright © MicroQuill Software Publishing, Inc. Allrights reserved. Copyright © PassMark Software Pty Ltd. All rights reserved. Copyright © LogiXML, Inc. All rights reserved. Copyright © 2003-2010 Lorenzi Davide, Allrights reserved. Copyright © Red Hat, Inc. All rights reserved. Copyright © The Board of Trustees of the Leland Stanford Junior University. All rights reserved. Copyright© EMC Corporation. All r ights reserved. Copyright © Flexera Software. All rights reserved. Copyright © Jinfonet Software. All rights reserved. Copyright © Apple Inc. Allrights reserved. Copyright © Telerik Inc. All rights reserved. Copyright © BEA Systems. All rights reserved. Copyright © PDFlib GmbH. All rights reserved. Copyright ©

Orientation in Objects GmbH. All rights reserved. Copyright © Tanuki Software, Ltd. All rights reserved. Copyright © Ricebridge. All rights reserved. Copyright © Sencha,Inc. All rights reserved.

This product includes software developed by the Apache Software Foundation (http://www.apache.org/), and/or other software which is licensed under various versionsof the Apache License (the "License"). You may obtain a copy of these Licenses at http://www.apache.org/licenses/. Unless required by applicable law or agreed to inwriting, software distributed under these Licenses is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express orimplied. See the Licenses for the specific language governing permissions and limitations under the Licenses.

This product includes software which was developed by Mozilla (http://www.mozilla.org/), software copyright The JBoss Group, LLC, all rights reserved; softwarecopyright © 1999-2006 by Bruno Lowagie and Paulo Soares and other software which is licensed under various versions of the GNU Lesser General Public License

Agreement, which may be found at http:// www.gnu.org/licenses/lgpl.html. The materials are provided free of charge by Informatica, "as-is", without warranty of anykind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose.

The product includes ACE(TM) and TAO(TM) software copyrighted by Douglas C. Schmidt and his research group at Washington University, University of California,Irvine, and Vanderbilt University, Copyright (©) 1993-2006, all rights reserved.

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (copyright The OpenSSL Project. All Rights Reserved) andredistribution of this software is subject to terms available at http://www.openssl.org and http://www.openssl.org/source/license.html.

This product includes Curl software which is Copyright 1996-2013, Daniel Stenberg, . All Rights Reserved. Permissions and limitations regarding thissoftware are subject to terms available at http://curl.haxx.se/docs/copyright.html. Permission to use, copy, modify, and distribute this software for any purpose with orwithout fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

The product includes software copyright 2001-2005 (©) MetaStuff, Ltd. All Rights Reserved. Permissions and limitations regarding this software are subject to termsavailable at http://www.dom4j.org/ license.html.

The product includes software copyright © 2004-2007, The Dojo Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject toterms available at http://dojotoolkit.org/license.

This product includes ICU software which is copyright International Business Machines Corporation and others. All rights reserved. Permissions and limitationsregarding this software are subject to terms available at http://source.icu-project.org/repos/icu/icu/trunk/license.html.

This product includes software copyright © 1996-2006 Per Bothner. All rights reserved. Your right to use such materials is set forth in the license which may be found at

http:// www.gnu.org/software/ kawa/Software-License.html.

This product includes OSSP UUID software which is Copyright © 2002 Ralf S. Engelschall, Copyright © 2002 The OSSP Project Copyright © 2002 Cable & WirelessDeutschland. Permissions and limitations regarding this software are subject to terms available at http://www.opensource.org/licenses/mit-license.php.

This product includes software developed by Boost (http://www.boost.org/) or under the Boost software license. Permissions and limitations regarding this software aresubject to terms available at http:/ /www.boost.org/LICENSE_1_0.txt.

This product includes software copyright © 1997-2007 University of Cambridge. Permissions and limitations regarding this software are subject to terms available athttp:// www.pcre.org/license.txt.

This product includes software copyright © 2007 The Eclipse Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to termsavailable at http:// www.eclipse.org/org/documents/epl-v10.php and at http://www.eclipse.org/org/documents/edl-v10.php.

This product includes software licensed under the terms at http://www.tcl.tk/software/tcltk/license.html, http://www.bosrup.com/web/overlib/?License, http://www.stlport.org/doc/ license.html, http:// asm.ow2.org/license.html, http://www.cryptix.org/LICENSE.TXT, http://hsqldb.org/web/hsqlLicense.html, http://httpunit.sourceforge.net/doc/ license.html, http://jung.sourceforge.net/license.txt , http://www.gzip.org/zlib/zlib_license.html, http://www.openldap.org/software/release/


3/246

license.html, http://www.libssh2.org, http://slf4j.org/license.html, http://www.sente.ch/software/OpenSourceLicense.html, http://fusesource.com/downloads/license-agreements/fuse-message-broker-v-5-3- license-agreement; http://antlr.org/license.html; http://aopalliance.sourceforge.net/; http://www.bouncycastle.org/licence.html;http://www.jgraph.com/jgraphdownload.html; http://www.jcraft.com/jsch/LICENSE.txt; http://jotm.objectweb.org/bsd_license.html; . http://www.w3.org/Consortium/Legal/2002/copyright-software-20021231; http://www.slf4j.org/license.html; http://nanoxml.sourceforge.net/orig/copyright.html; http://www.json.org/license.html; http://forge.ow2.org/projects/javaservice/, http://www.postgresql.org/about/licence.html, http://www.sqlite.org/copyright.html, http://www.tcl.tk/software/tcltk/license.html, http://www.jaxen.org/faq.html, http://www.jdom.org/docs/faq.html, http://www.slf4j.org/license.html; http://www.iodbc.org/dataspace/iodbc/wiki/iODBC/License; http://www.keplerproject.org/md5/license.html; http://www.toedter.com/en/jcalendar/license.html; http://www.edankert.com/bounce/index.html; http://www.net-snmp.org/about/license.html; http://www.openmdx.org/#FAQ; http://www.php.net/license/3_01.txt; http://srp.stanford.edu/license.txt; http://www.schneier.com/blowfish.html; http://www.jmock.org/license.html; http://xsom.java.net; http://benalman.com/about/license/; https://github.com/CreateJS/EaselJS/blob/master/src/easeljs/display/Bitmap.js;http://www.h2database.com/html/license.html#summary; http://jsoncpp.sourceforge.net/LICENSE; http://jdbc.postgresql.org/license.html; http://protobuf.googlecode.com/svn/trunk/src/google/protobuf/descriptor.proto; https://github.com/rantav/hector/blob/master/LICENSE; http://web.mit.edu/Kerberos/krb5-current/doc/mitK5license.html. and http://jibx.sourceforge.net/jibx-license.html.

This product includes software licensed under the Academic Free License (http://www.opensource.org/licenses/afl-3.0.php), the Common Development and DistributionLicense (http://www.opensource.org/licenses/cddl1.php) the Common Public License (http://www.opensource.org/licenses/cpl1.0.php), the Sun Binary Code License

Agreement Supplemental License Terms, the BSD License (http:// www.opensource.org/licenses/bsd-license.php), the new BSD License (http://opensource.org/licenses/BSD-3-Clause), the MIT License (http://www.opensource.org/licenses/mit-license.php), the Artistic License (http://www.opensource.org/licenses/artistic-license-1.0) and the Initial Developer’s Public License Version 1.0 (http://www.firebirdsql.org/en/initial-developer-s-public-license-version-1-0/).

This product includes software copyright © 2003-2006 Joe WaInes, 2006-2007 XStream Committers. All rights reserved. Permissions and limitations regarding thissoftware are subject to terms available at http://xstream.codehaus.org/license.html. This product includes software developed by the Indiana University Extreme! Lab.For further information please visit http://www.extreme.indiana.edu/.

This product includes software Copyright (c) 2013 Frank Balluffi and Markus Moeller. All rights reserved. Permissions and limitations regarding this software are subjectto terms of the MIT license.

This Software is protected by U.S. Patent Numbers 5,794,246; 6,014,670; 6,016,501; 6,029,178; 6,032,158; 6,035,307; 6,044,374; 6,092,086; 6,208,990; 6,339,775;6,640,226; 6,789,096; 6,823,373; 6,850,947; 6,895,471; 7,117,215; 7,162,643; 7,243,110; 7,254,590; 7,281,001; 7,421,458; 7,496,588; 7,523,121; 7,584,422;7,676,516; 7,720,842; 7,721,270; 7,774,791; 8,065,266; 8,150,803; 8,166,048; 8,166,071; 8,200,622; 8,224,873; 8,271,477; 8,327,419; 8,386,435; 8,392,460;8,453,159; 8,458,230; and RE44,478, International Patents and other Patents Pending.

DISCLAIMER: Informatica Corporation provides this documentation "as is" without warranty of any kind, either express or implied, including, but not limited to, theimplied warranties of noninfringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or documentation is

error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The information in this software anddocumentation is subject to change at any time without notice.

NOTICES

This Informatica product (the "Software") includes certain drivers (the "DataDirect Drivers") from DataDirect Technologies, an operating company of Progress SoftwareCorporation ("DataDirect") which are subject to the following terms and conditions:

1.THE DATADIRECT DRIVERS ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT

LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.

2.IN NO EVENT WILL DATADIRECT OR ITS THIRD PARTY SUPPLIERS BE LIABLE TO THE END-USER CUSTOMER FOR ANY DIRECT, INDIRECT,

INCIDENTAL, SPECIAL, CONSEQUENTIAL OR OTHER DAMAGES ARISING OUT OF THE USE OF THE ODBC DRIVERS, WHETHER OR NOT

INFORMED OF THE POSSIBILITIES OF DAMAGES IN ADVANCE. THESE LIMITATIONS APPLY TO ALL CAUSES OF ACTION, INCLUDING, WITHOUT

LIMITATION, BREACH OF CONTRACT, BREACH OF WARRANTY, NEGLIGENCE, STRICT LIABILITY, MISREPRESENTATION AND OTHER TORTS.

Part Number: PCX-DUG-96000-0001


4/246

Table of Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Informatica Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Informatica My Support Portal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Informatica Documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Informatica Web Site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Informatica Support YouTube Channel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Informatica Marketplace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Informatica Velocity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Chapter 1: Introduction to Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Introduction to Transformations Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Active Tr ansformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Passive Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Unconnected Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Transfor mation Descriptions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Developing a Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Reusable Transfor mations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Reusable Transformation Instances and Inherited Changes. . . . . . . . . . . . . . . . . . . . . . . . 4

Editing a Reusable Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Expressions in Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

The Expr ession Editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

Port Names in an Expression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Adding an Expression to a Port. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Comments in an Expression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Expression Validation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Creating a Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Tracing Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Chapter 2: Aggregator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Aggregator Tr ansformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Developing an Aggregator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Aggregator Transformation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Aggregator Transf ormation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Aggregate Caches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Aggregate Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Aggregate Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Nested Aggregate Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Conditional Clauses in Aggregate Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Group By Por ts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Non-Aggregate Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Table of Contents i


5/246

Default Values of Group By Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Sorted Input for an Aggregator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Sorted Input Conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Sorting Data in an Aggregator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Creating a Reusable Aggregator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Creating a Non-Reusable Aggregator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Tips for Aggregator Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Troubleshooting Aggregator Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Chapter 3: Expression Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Expression Transf ormation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Developing an Expression Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Expression Transf ormation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Expression Transf ormation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Chapter 4: Filter Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Filter Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Filter Transfor mation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Filter Condition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Filtering Rows with Null Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Filter Transformation Performance Tips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Chapter 5: Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Java Transfor mation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Reusable and Non-Reusable Java Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Active and Passive Java Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Datatype Conversion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Designing a Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Java Transfor mation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Creating Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Setting Default Port Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Java Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Configuring the Classpath for the Developer Tool Client. . . . . . . . . . . . . . . . . . . . . . . . . . 29

Configuring the Classpath for the Data Integration Service. . . . . . . . . . . . . . . . . . . . . . . . 29

Developing Java Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Creating Java Code Snippets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Importing Java Packages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Defining Helper Code. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Java Transfor mation Java Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Imports Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Helpers Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

On Input Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

At End Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

ii Table of Contents


6/246

Functions Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Full Code Tab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Filter Optimization with the Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Early Selection Optimization with the Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . 36

Push-Into Optimization with the Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Creating a Java Tr ansformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Creating a Reusable Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38Creating a Non-Reusable Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Compiling a Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Troubleshooting a Java Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Finding the Source of Compilation Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

Identifying the Source of Compilation Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Chapter 6: Java Transformation API Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Java Transformation API Methods Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

defineJExpression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

failSession. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

generateRow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

getInRowType. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

getMetadata. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

incrementErrorCount. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

invokeJExpression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

isNull. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

logError. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

logInfo. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

resetNotification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

setNull. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

storeMetadata. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Chapter 7: Java Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Java Expressions Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Expression Function Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Using the Def ine Function Dialog Box to Define an Expression. . . . . . . . . . . . . . . . . . . . . . . . 53

Step 1. Configure the Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Step 2. Create and Validate the Expression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Step 3. Gener ate Java Code for the Expression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

Creating an Expression and Generating Java Code by Using the Define Function Dialog Box. . 54

Java Expression Templates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Working with the Simple Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

invokeJExpression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Simple Interface Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Working with the Advanced Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Invoking an Expression with the Advanced Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Table of Contents iii


7/246

Rules and Guidelines for Working with the Advanced Interface. . . . . . . . . . . . . . . . . . . . . 57

EDataType Class. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

JExprParamMetadata Class. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

defineJExpression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

JExpression Class. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Advanced Interface Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

JExpression Class API Reference. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61getBytes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

getDouble. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

getInt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

getLong. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

getResultDataType. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

getResultMetadata. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

getStringBuffer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

invoke. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

isResultNull. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Chapter 8: Joine r T r a n s f o r m a t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4

Joiner Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

Joiner Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Joiner Transformation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Defining a Join Condition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Simple Condition Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Advanced Condition Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Join Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Normal Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Master Outer Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

Detail Outer Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Full Outer Join. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

Sorted Input for a Joiner Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Configuring the Sort Order. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Adding Transf ormations to the Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Rules and Guidelines for Join Conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Example of a Join Condition and Sort Order. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Joining Data from the Same Source. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Joining Two Branches of the Same Pipeline. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Joining Two Instances of the Same Source. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Guidelines for Joining Data from the Same Source. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Blocking the Source Pipelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Unsorted Joiner Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Sorted Joiner Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Joiner Transformation Performance Tips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Rules and Guidelines for a Joiner Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

iv Table of Contents


8/246

Chapter 9: Lookup Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Lookup Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Connected and Unconnected Lookups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Connected Lookups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Unconnected Lookups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Developing a Lookup Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80Lookup Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Default Lookup Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

SQL Override for a Lookup Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

SQL Override for an Uncached Lookup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

Reserved Words. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Guidelines for Overriding the Lookup Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Overriding the Lookup Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Lookup Source Filter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Filtering Source Rows in a Lookup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Lookup Condition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

Rules and Guidelines for Lookup Transformation Conditions. . . . . . . . . . . . . . . . . . . . . . . 84

Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Query Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Run-time Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Creating a Reusable Lookup Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

Creating a Non-Reusable Lookup Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

Creating an Unconnected Lookup Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Unconnected Lookup Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Chapter 10: Lookup Caches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Lookup Caches Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

Lookup Cache Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Uncached Lookup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Static Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Persistent Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Rules and Guidelines for Rebuilding a Persistent Lookup Cache. . . . . . . . . . . . . . . . . . . . 95

Dynamic Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Shared Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Rules and Guidelines for Sharing a Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Cache Comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Cache Partitioning for Lookups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Tips for Lookup Caches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Chapter 11: Dynamic Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Dynamic Lookup Cache Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Table of Contents v


9/246

Uses for a Dynamic Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Dynamic Lookup Cache Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

Dynamic Lookup Cache and Output Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Lookup Transformation Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

Lookup Transformation Values Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

SQL Override and Dynamic Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

Mapping Configur ation for a Dynamic Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105Insert Else Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Update Else Insert. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

Dynamic Lookup Cache and Target Synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Conditional Dynamic Lookup Cache Updates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Conditional Dynamic Lookup Cache Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Configuring a Conditional Dynamic Lookup Cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

Dynamic Cache Update with Expression Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Null Expr ession Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Expression Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

Configuring an Expression for Dynamic Cache Updates. . . . . . . . . . . . . . . . . . . . . . . . . 110

Dynamic Lookup Cache Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

Rules and Guidelines for Dynamic Lookup Caches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Chapter 12: No rma l i z e r T ra n s fo rma t io n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 3

Normalizer Tr ansformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

Normalizer Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Multiple-Occurring Fields. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Multiple-Occurring Records and Subrecords. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

Normalizer Tr ansformation Groups and Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Normalizer Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Developing a Normalizer Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Key Generation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

Creating a Normalizer Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Merging Fields. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Normalizer Mapping Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

Normalizer Example Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Normalizer Example Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Normalizer Example Input and Output Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Normalizer Example Mapping Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

Chapter 13: Rank Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Rank Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Ranking String Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

Rank Caches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Rank Transformation Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Rank Transformation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

vi Table of Contents


10/246

Rank Port. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Rank Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

Defining Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Rank Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Chapter 14: Router Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

Router Transf ormation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127Working with Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Input Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Output Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Using Gr oup Filter Conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

Adding Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Working with Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

Connecting Router Transformations in a Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Router Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

Chapter 15: Sequence Generator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133Sequence Generator Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Create Keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Replace Missing Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Cycle Through a Range of Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Multiple Mappings for the Same Target. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Sequence Generator Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

Pass-Thr ough Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

NEXTVAL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Sequence Data Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Sequence Data Object Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Sequence Generator Transformation Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

Start Value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

Increment Value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

End Value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Reset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Cycle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Creating a Reusable Sequence Generator Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . 139

Creating a Nonreusable Sequence Generator Transformation. . . . . . . . . . . . . . . . . . . . . . . . 140

Frequently Asked Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

Chapter 16: Sorter Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Sorter Transf ormation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

Developing a Sorter Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Sorter Transf ormation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

Sorter Cache Size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Sorter Transf ormation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

Table of Contents vii


11/246

Creating a Sorter Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Creating a Reusable Sorter Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Creating a Non-Reusable Sorter Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Sorter Transf ormation Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145

Chapter 17: SQL Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

SQL Transfor mation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147SQL Transfor mation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Input Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

Output Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

Pass-through Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

SQLError Por t. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

Number of Rows Affected. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

SQL Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

SQL Transfor mation Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Define the SQL Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

Input Row to Output Row Cardinality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Query Statement Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Port Configur ation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Maximum Output Row Count. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Error Rows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

Continue on SQL Error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Filter Optimization with the SQL Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

Early Selection Optimization with the SQL Transformation. . . . . . . . . . . . . . . . . . . . . . . 159

Push-Into Optimization with the SQL Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . 160

SQL Transformation Example with an SQL Query. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Logical Data Object Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Salary Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

Employee Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

SQL Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

Stored Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

SQL Transformation Ports for Stored Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

Stored Procedure Result Sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Stored Procedure Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Manually Creating an SQL Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

Creating an SQL Transformation from a Stored Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . 169

Chapter 18: U n i o n T r a n s f o r m a t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7 1

Union Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

Groups and Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

Union Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

Union Transformation Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

viii Table of Contents


12/246

Creating a Union Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Creating a Reusable Union Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Creating a Non-Reusable Union Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

Chapter 19: Update Strategy Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

Update Strategy Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

Setting the Update Strategy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175Flagging Rows Within a Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Update Strategy Expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Update Strategy Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . . . 176

Aggregator and Update Strategy Transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Specifying Update Options for Individual Targets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

Chapter 20: Web Service Consumer Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . 178

Web Service Consumer Transformation Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

SOAP Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

WSDL Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

Web Ser vice Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

WSDL Selection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

Web Service Consumer Transformation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

HTTP Header Input Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

Other Input Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

Web Service Consumer Transformation Input Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182

Rules and Guidelines to Map Input Ports to Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

Customize View Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Mapping Input Ports to the Operation Input. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

Web Service Consumer Transformation Output Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . 185

Rules and Guidelines to Map Nodes to Output Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Mapping the SOAP Message as XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Customize View Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Mapping the Operation Output to Output Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

Web Service Consumer Transformation Advanced Properties. . . . . . . . . . . . . . . . . . . . . . . . 188

Web Service Error Handling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Message Compression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

Concurrency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

Filter Optimizations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Enabling Early Selection Optimization with the Web Service Consumer Transformation. . . . . 193

Push-Into Optimization with the Web Service Consumer Transformation. . . . . . . . . . . . . . 193

Creating a Web Service Consumer Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

Web Service Consumer Transformation Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

Input File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

Logical Data Object Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Table of Contents ix


13/246

Logical Data Object Mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Web Service Consumer Transformation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

Chapter 21: Parsing Web Service SOAP Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Parsing Web Service SOAP Message Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

Transformation User Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

Multiple-Occurring Output Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202Normalized Relational Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Generated Keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Denormalized Relational Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

Pivoted Relational Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

Parsing anyType Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

Parsing Derived Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Parsing QName Elements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

Parsing Substitution Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

Parsing XML Constructs in SOAP Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206

Choice Element. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

List Element. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Union Element. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Chapter 22: Generating Web Service SOAP Messages. . . . . . . . . . . . . . . . . . . . . . . . 208

Generating Web Service SOAP Messages Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208

Transformation User Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Input Por ts Area. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Operation Area. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Port and Hier archy Level Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

Keys. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211

Map Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

Map a Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213

Map a Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Map Multiple Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Pivoting Multiple-Occurring Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214

Map Denormalized Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

Derived Types and Element Substitution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Generating Derived Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

Generating anyType Elements and Attributes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

Generating Substitution Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

Generating XML Constructs in SOAP Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

Choice Element. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

List Element. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

Union Element. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

x Table of Contents


14/246

Appendix A: Transformation Delimiters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Transformation Delimiters Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

Table of Contents xi


15/246

Preface

The Transformation Guide contains information about common functionality in the Developer tool. It is written

for developers. This guide assumes that you have an understanding of data quality concepts, flat file and

relational database concepts, and the database engines in your environment.

Informatica Resources

Informatica My Support Portal

As an Informatica customer, you can access the Informatica My Support Por tal at

http://mysupport.informatica.com.

The site contains product information, user group information, newsletters, access to the Informatica How-To

Library, the Informatica Knowledge Base, Informatica Product Documentation, and access to the Informatica

user community.

Informatica DocumentationThe Informatica Documentation team takes every effort to create accurate, usable documentation. If you

have questions, comments, or ideas about this documentation, contact the Informatica Documentation team

through email at [email protected]. We will use your feedback to improve our

documentation. Let us know if we can contact you regarding your comments.

The Documentation team updates documentation as needed. To get the latest documentation for your

product, navigate to Product Documentation from http://mysupport.informatica.com.

Informatica Web Site

You can access the Informatica corporate web site at http://www.informatica.com. The site contains

information about Informatica, its background, upcoming events, and sales offices. You will also find productand partner information. The services area of the site includes important information about technical support,

training and education, and implementation services.

Informatica Support YouTube Channel

You can access the Informatica Support YouTube channel at http://www.youtube.com/user/INFASupport. The

Informatica Support YouTube channel includes videos about solutions that guide you through performing

specific tasks. If you have questions, comments, or ideas about the Informatica Support YouTube channel,

xii

mailto:[email protected]://www.youtube.com/user/INFASupporthttp://www.informatica.com/http://mysupport.informatica.com/mailto:[email protected]://mysupport.informatica.com/


16/246

contact the Support YouTube team through email at [email protected] or send a tweet to

@INFASupport.

Informatica Marketplace

The Informatica Marketplace is a forum where developers and partners can share solutions that augment,extend, or enhance data integration implementations. By leveraging any of the hundreds of solutions

available on the Marketplace, you can improve your productivity and speed up time to implementation on

your projects. You can access Informatica Marketplace at http://www.informaticamarketplace.com.

Informatica Velocity

You can access Informatica Velocity at http://mysupport.informatica.com. Developed from the real-world

experience of hundreds of data management projects, Informatica Velocity represents the collective

knowledge of our consultants who have worked with organizations from around the world to plan, develop,

deploy, and maintain successful data management solutions. If you have questions, comments, or ideas

about Informatica Velocity, contact Informatica Professional Services at [email protected].

Preface xiii

http://[email protected]/http://mysupport.informatica.com/http://www.informaticamarketplace.com/http://[email protected]/


17/246

xiv


18/246

C H A P T E R 1

Introduction to Transformations

This chapter includes the following topics:

• Introduction to Transformations Overview, 1

• Developing a Transformation, 3

• Reusable Transformations, 3

•

Expressions in Transformations, 4• Creating a Transformation, 7

• Tracing Levels, 7

Introduction to Transformations Overview

A transformat ion is an object that generates, modifies, or passes data.

Informatica Developer provides a set of transformations that perform specific functions. For example, an

Aggregator t ransformation performs calculations on groups of data.

Transformations in a mapping represent the operations that the Data Integration Service performs on the

data. Data passes through transformation ports that you link in a mapping or mapplet.

Transformations can be active or passive. Transformations can be connected to the data flow, or they can be

unconnected.

Active Transf ormations

An active transformation changes the number of rows that pass through a transformation. Or, it changes the

row type.

For example, the Filter transformation is active, because it removes rows that do not meet the filter condition.

The Update Strategy transformation is active, because it flags rows for insert, delete, update, or reject.

You cannot connect multiple active transformations or an active and a passive transformation to the same

downstream transformation or transformation input group, because the Data Integration Service might not be

able to concatenate the rows passed by active transformations.

For example, one branch in a mapping contains an Update Strategy transformation that flags a row for

delete. Another branch contains an Update Strategy transformation that flags a row for insert. If you connect

these transformations to a single transformation input group, the Data Integration Service cannot combine the

delete and insert operations for the row.

1


19/246

Passive Transformations

A passive transformat ion does not change the number of rows that pass through the transformation, and i t

maintains the row type.

You can connect multiple transformations to the same downstream transformation or transformation input

group if all transformations in the upstream branches are passive. The transformation that originates thebranch can be active or passive.

Unconnected Transformations

Transformations can be connected to the data flow, or they can be unconnected. An unconnected

transformation is not connected to other transformations in the mapping. An unconnected transformation is

called within another transformation, and returns a value to that transformation.

Transformation Descriptions

The Developer tool contains transformations that perform different data integration functions.

The following table describes each transformation:

Transformation Type Description

Aggregator Active/

Connected

Performs aggregate calculations.

Expression Passive/

Connected

Calculates a value.

Filter Active/

Connected

Filters data.

Java Active or Passive/

Connected

Executes user logic coded in Java. The byte code for the

user logic is stored in the repository.

Joiner Active/

Connected

Joins data from different databases or flat file systems.

Lookup Active or Passive/

Connected or

Unconnected

Look up and return data from a flat file, logical data object,

reference table, relational table, view, or synonym.

Normalizer Active/

Connected

Processes source rows that contain multiple-occurring data

and returns a target row for each instance of the multiple-occurring data.

Output Passive/

Connected

Defines mapplet output rows.

Rank Active/

Connected

Limits records to a top or bottom range.

Router Active/

Connected

Routes data into multiple transformations based on groupconditions.

2 Chapter 1: Introduction to Transformations


20/246

Transformation Type Description

Sequence Generator Passive/

Connected

Generates a numeric sequence of values.

Sorter Active/

Connected

Sorts data based on a sort key.

SQL Active or Passive/

Connected

Executes SQL queries against a database.

Union Active/

Connected

Merges data from different databases or flat file systems.

Update Strategy Active/

Connected

Determines whether to insert, delete, update, or reject rows.

Web ServiceConsumer Active/ConnectedConnects to a web service as a web service client to accessor transform data.

Developing a Transformation

When you build a mapping, you add transformations and configure them to handle data according to a

business purpose.

Complete the following tasks to develop a transformation and incorporate it into a mapping:

1. Add a nonreusable transformation to a mapping or mapplet. Or, create a reusable transformation thatyou can add to multiple mappings or mapplets.

2. Configure the transformation. Each type of transformation has a unique set of options that you can

configure.

3. If the transformation is reusable, add it to the mapping or mapplet.

4. Link the transformation to other objects in the mapping or mapplet.

You drag ports from upstream objects to the transformation input ports. You drag output ports from the

transformation to ports on downstream objects. Some transformations use predefined ports that you can

select.

Note: If you create a reusable transformation, you add the input and output ports you need before you link

the transformation to other objects. You cannot add ports to the transformation instance on the mapplet or

mapping canvas. To update the ports on a reusable transformation, open the transformation object from therepository project and add the ports.

Reusable Transformations

Reusable transformations are transformations that you can use in multiple mappings or mapplets.

Developing a Transformation 3


21/246

For example, you might create an Expression transformation that calculates value-added tax for sales in

Canada to analyze the cost of doing business in that country. Rather than perform the same work every time,

you can create a reusable transformation. When you need to incorporate this transformation into a mapping,

you add an instance of it to the mapping. If you change the definition of the transformation, all instances of it

inherit the changes.

The Developer tool stores each reusable transformation as metadata separate from any mapping or mappletthat uses the transformation. It stores reusable transformations in a project or folder.

When you add instances of a reusable transformation to mappings, changes you make to the transformation

might invalidate the mapping or generate unexpected data.

Reusable Transformation Instances and Inherited Changes

When you add a reusable transformation to a mapping or mapplet, you add an instance of the transformation.

The definition of the transformation still exists outside the mapping or mapplet, while an instance of the

transformation appears within the mapping or mapplet.

When you change the transformation, instances of the transformation reflect these changes. Instead of

updating the same transformation in every mapping that uses it, you can update the reusable transformationone time, and all instances of the transformation inherit the change. Instances inherit changes to ports,

expressions, properties, and the name of the transformation.

Editing a Reusable Transformation

When you edit a reusable transformation, all instances of that transformation inherit the changes. Some

changes might invalidate the mappings that use the reusable transformation.

You can open the transformation in the editor to edit a reusable transformation. You cannot edit an instance

of the transformation in a mapping. However, you can edit the transformation runtime properties.

If you make any of the following changes to a reusable transformation, mappings that use instances of it

might not be valid:

• When you delete one or more ports in a transformation, you disconnect the instance from part or all of the

data flow through the mapping.

• When you change a port datatype, you make it impossible to map data from that port to another port that

uses an incompatible datatype.

• When you change a port name, expressions that refer to the port are no longer valid.

• When you enter an expression that is not valid in the reusable transformation, mappings that use the

transformation are no longer valid. The Data Integration Service cannot run mappings that are not valid.

Expressions in TransformationsYou can enter expressions in the Expression Editor in some transformations. Expressions modify data or

test whether data matches conditions.

Create expressions that use transformation language functions. Transformation language functions are SQL-

like functions that transform data.

Enter an expression in a port that uses the value of data from an input or input/output port. For example, you

have a transformation with an input port IN_SALARY that contains the salaries of all the employees. You

might use the values from the IN_SALARY column later in the mapping. You might also use the



22/246

transformation to calculate the total and average salaries. The Developer tool requires you to create a

separate output port for each calculated value.

The following table lists the transformations in which you can enter expressions:

Transformation Expression Return Value

Aggregator Performs an aggregate calculation based on all

data passed through the transformation. Alte rnatively, you can speci fy a filt er fo r records in

the aggregate calculation to exclude certain kinds

of records. For example, you can find the totalnumber and average salary of all employees in a

branch office using this transformation.

Result of an aggregate calculation

for a port.

Expression Per forms a calcu lat ion based on values within a

single row. For example, based on the price andquantity of a particular item, you can calculate the

total purchase price for that line item in an order.

Result of a row-level calculation for

a port.

Filter Specifies a condition used to filter rows passedthrough this transformation. For example, if youwant to write customer data to the BAD_DEBT table

for customers with outstanding balances, you could

use the Filter transformation to filter customer data.

TRUE or FALSE, based on whethera row meets the specified condition.The Data Integration Service passes

rows that return TRUE through this

transformation. The transformation

applies this value to each row thatpasses through it.

Joiner Specifies an advanced condition used to join

unsorted source data. For example, you can

concatenate first name and last name master portsand then match them with the full name detail port.

TRUE or FALSE, based on whether

the row meets the specified

condition. Depending on the type of join selec ted, the Data Integra tion

Service either adds the row to the

result set or discards the row.

Rank Sets the conditions for rows included in a rank. Forexample, you can rank the top 10 salespeople who

are employed with the organization.

Result of a condition or calculationfor a port.

Router Routes data into multiple transformations based on

a group expression. For example, use thistransformation to compare the salaries of

employees at three different pay levels. You can do

this by creating three groups in the Router

transformation. For example, create one groupexpression for each salary range.

TRUE or FALSE, based on whether

a row meets the specified groupexpression. The Data Integration

Service passes rows that return

TRUE through each user-defined

group in this transformation. Rowsthat return FALSE pass through the

default group.

Update Strategy Flags a row for update, insert, delete, or reject. You

use this transformation when you want to control

updates to a target, based on some condition youapply. For example, you might use the Update

Strategy transformation to flag all customer rows forupdate when the mailing address has changed. Or,

you might flag all employee rows for reject for

people who no longer work for the organization.

Numeric code for update, insert,

delete, or reject. The transformation

applies this value to each rowpassed through it.

The Expression Editor

Use the Expression Editor to build SQL-like statements.

Expressions in Transformations 5


23/246

You can enter an expression manually or use the point-and-click method. Select functions, ports, variables,

and operators from the point-and-click interface to minimize errors when you build expressions. The

maximum number of characters you can include in an expression is 32,767.

Port Names in an Expression

You can enter transformation port names in an expression.

For connected transformations, if you use port names in an expression, the Developer tool updates that

expression when you change port names in the transformation. For example, you write an expression that

determines the difference between two dates, Date_Promised and Date_Delivered. If you change the

Date_Promised port name to Due_Date, the Developer tool changes the Date_Promised port name to

Due_Date in the expression.

Note: You can propagate the name Due_Date to other non-reusable transformations that depend on this port

in the mapping.

Adding an Expression to a Port

You can add an expression to an output port.

1. In the transformation, select the port and open the Expression Editor .

2. Enter the expression.

Use the Functions and Ports tabs and the operator keys.

3. Optionally, add comments to the expression.

Use comment indicators -- or //.

4. Click the Validate button to validate the expression.

5. Click OK.

6. If the expression is not valid, fix the validation errors and validate the expression again.

7. When the expression is valid, click OK to close the Expression Editor .

Comments in an Expression

You can add comments to an expression to describe the expression or to specify a valid URL to access

business documentation about the expression.

To add comments within the expression, use -- or // comment indicators.

Expression Validation

You need to validate an expression to run a mapping or preview mapplet output.

Use the Validate button in the Expression Editor to validate an expression. If you do not validate an

expression, the Developer tool validates it when you close the Expression Editor . If the expression is

invalid, the Developer tool displays a warning. You can save the invalid expression or modify it.



24/246

Creating a Transformation

You can create a reusable transformation to reuse in multiple mappings or mapplets. Or, you can create a

non-reusable transformation to use one time in a mapping or mapplet.

To create a reusable transformation, click File > New > Transformation and complete the wizard.

To create a non-reusable transformation in a mapping or mapplet, select a transformation from the

Transformation palette and drag the transformation to the editor.

Certain transformations require you to choose a mode or perform additional configuration when you create

the transformation. For example, the Parser transformation requires that you choose either token parsing

mode or pattern parsing mode when you create the transformation.

After you create a t ransformation, it appears in the editor. Some transformations contain predefined ports and

groups. Other transformations are empty.

Tracing LevelsWhen you configure a transformation, you can set the amount of detail that the Data Integration Service

writes in the log.

By default, the tracing level for every transformation is Normal. Change the tracing level to a Verbose setting

when you need to troubleshoot a transformation that is not behaving as expected. Set the tracing level to

Terse when you want the minimum amount of detail to appear in the log.

Configure the following property on the Advanced tab:

Tracing Level

Amount of deta il that appears in the log for a t ransformation.

The following table describes the tracing levels:

Tracing Level Description

Terse Logs initialization information and error messages and notification of rejected

data.

Normal Logs initialization and status information, errors encountered, and skippedrows due to transformation row errors. Summarizes mapping results, but not

at the level of indiv

ABC Practice

Documents

Transcript of ABC Practice