Data integration extravaganza

Oracle EPM (Hyperion) Suite Data Integration Extravaganza!

Applications and Service Offerings

AdvancedEPM has successfully completed 100+ EPM/BI Implementations

Financial Consolidation

Budgeting & Planning

Business Intelligence & Analytics

Profitability Management Solutions

Data Warehousing

Infrastructure

Performance Tuning & Optimization

Business Process and Project Management

Organizational Change Management

AdvancedEPM Service Offerings

Full Lifecycle Product Implementations

Upgrades & Migrations

Managed Services

Training & Support

Center of Excellence Development

Project Management

Change Management

Gap Assessments / Health Checks

Road Mapping

Business Process Improvement

AdvancedEPM has a series of service offerings within our core focus of Oracle Hyperion Enterprise Performance Management and Business Intelligence.

Thought LeadershipWe are committed to bringing our clients the latest thinking and education around EPM & BI.

Webcasts Speaking Events Training

Join us and expand your skillset!

AdvancedEPM is proud to be offering 5-10 FREE webcasts each month focused on Oracle Enterprise Performance Management and MDM solutions.

Register today!www.advancedepm.com

AdvancedEPM is recognized by the Oracle Hyperion community and is frequently invited to speak at conferences on various EPM topics.

A few 2013 Speaking Events• Collaborate 2013• Kaleidoscope• CFO Magazine – Corporate

Performance Management Event

Not all training is created equally!

AdvancedEPM recognizes that every Oracle Hyperion training class has unique requirements. We evaluate your needs and deliver Hyperion training courses that are tailored to your learning requirements.

Technologies• Hyperion Application Link (HAL)• Data Integration Management (DIM) • Oracle Data Integrator (ODI)• Enterprise Performance Management Architect (EPMA)• Financial Data Quality Management (FDM)• Financial Data Quality Management Enterprise Edition (FDMEE)• ERP Integrator (ERPi)• Data Relationship Management (DRM)• Essbase Integration Services (EIS)• Essbase Studio (EAS)• Outline Load Utility• DATAEXPORT Command• MaxL• Windows Batch Scripting• Essbase SQL Loader• Essbase Analytics Link (EAL) for HFM

And More!!

Informatica, Oracle Data Integrator, EPMA, OutlineLoad Utility, Load Rules, Report Scripts, Calculation Scripts, Shell Scripts, Windows Batch, VBScript, @XREFS, @XWRITES, Database Partitioning, ERP Integrator, Web Services, etc., etc.

Be Happy – This Stuff is Fun

HAL• Background

– Hyperion tool for loading data and metadata to Hyperion Essbase, Financial Management, and Planning

– No longer supported by Oracle– Vignette Business Integration Studio (VBIS)

• Common Uses– Loading data (text, relational) to Essbase, Planning– Extracting outline information

• Moped vs. Sport Bike?

HAL• Use Case

– Company wants to move GL data from a relational data source or flat file and use it to update the outline in Planning

– Extract Data From Essbase/Planning

DIM• Data Integration Management• Based on Informatica PowerCenter• ETL tool similar to ODI

DIM• Can load data and metadata to Essbase,

Planning, HFM, Performance Scorecard, etc.• Can process large volumes of data and

metadata with complex transformations• Not Oracle’s strategic direction

DIM• Use Case(s):

– IT driven process passing large volumes of data at high speeds

– Complex mapping logic is required– Multiple workflows – Robust Process Monitoring– Web Service(s) Enablement (WSDL, SOAP, UDDI)

ODI• Oracle Data Integrator• Very flexible tool• Knowledge modules for interacting with

systems (Essbase, Planning, SQL Server, Oracle, Flat file, etc.)

ODI• Can load data and metadata to Essbase,

Planning, HFM, etc.• Can load data warehouses, ERP systems and

most databases• Strategic data integration tool from Oracle

ODI• Use Case

– Upstream data is in relational databases accessible via JDBC driver, wants to combine data from a file coming in via FTP and use that to load to a Hyperion cube, and send an email out on failure.

EPMA• Can manage metadata and applications (*)

within a graphical interface• Dimension Library centralizes hierarchies for

different dimensions• Can move data with Data Synchronization• Interface Tables • Application Library

EPMA• Manage Essbase, Planning, HFM in one place• Mostly deals with metadata• Provides validation functionality• Provides limited data mapping / integration

Define properties for all products for each member

Dimension Library

Graphical designer for Oracle EPM ETL processes

Data Synchronization

EPMA• Use Case

– Administrator already uses EPMA, deploys metadata to various applications

– Ad hoc data sync across environments– Quickly build a new application for a specific

purpose by using metadata and data from existing applications

AD-HOC CAGE MATCHEPMA vs. DRM!!

What’s the difference?Feature DRM EPMA

Property inheritance

Redeploy Oracle EPM applications

Customized property value validation

Customized structural validation

Member formula validation

Metadata versioning

Rollback to point-in-time

Share members across dimensions

Enforce custom business rules

= Limited Support

= Full/Advanced Support

= Not Supported

What’s the difference?Feature DRM EPMA

Interface with non-Oracle EPM applications

Customizable user interface

Custom property creation/definition

Workflow/approvals

Flexible output format

Transaction history

= Limited Support

= Full/Advanced Support

= Not Supported

What’s the difference?

Same Master Data Management Capabilities?

EPMAWatercolor (Kindergarten)

DRMStarry Night (Van Gogh)

Better Together – We Can All Get Along!!

DRM&

EPMA

How Can They Work Together?

Data Relationship Management

(1)Manage Enterprise

hierarchies and attributes

EPM Architect

HFM / HP / Essbase

(2)Packaged

Integration updates EPM hierarchies and

attributesin EPMA

(3)Manage EPM

applications and share dimensions across

applications

(4)Validate and deploy

dimensions to applications

FDM• Financial Data Quality Management• Load HFM, Essbase & Planning from flat files• SQL based, business driven• Highly customizable with VB Script• Self-service or automated data loading• Allows drill back from target systems

FDM

• Use Case– Enable SME’s to map their own data– Real time validation & verification – SOX Compliance & Certification– Industrial strength ETL not required– Sarbanes Oxley Certs

ERPi• Introduced in 11.1.1.3• Pulls directly from source ERPs• Load metadata to HFM, Planning and Essbase• Combined with FDM, loads data to HFM,

Planning or Essbase• Facilitates user drill back and write-back to

source ERP from HFR or Smart View

ERPi

• Use Case– Enhance functionality of existing FDM– With supported source ERPs, create robust 2-way

data flow for reporting and analysis. Provide access to detail with out replicating across applications.

– Push downstream data to all systems

FDMEE - Key Features

FDMEE - Mapping

RDBMS JDBC – Find Out What it Means to Me

ConectionString="DRIVER=oracle.jdbc.OracleDriver;URL=jdbc:oracle:thin:@<mc-name>:<port-no>:<DbName>;uid=<uid>;pwd=<pwd>"

set con = CreateObject("ADODB.Connection")con.open(ConectionString) set RS = CreateObject("ADODB.Recordset")RS.Open "Select * FROM EBS", con, 3, 3

HEY!!!!! I can do anything with VBScript

Lifecycle Management (LCM)– Install Shared Services and the product and the LCM

functionality is available without any additional services– Execute lights-out processing from the command line to

integrate with third-party schedulers– Navigate through a migration using the Migration

Wizard– Set up a migration definition file that can be reused and

scheduled– Review one of several reports - Migration Status,

Comparison, and Audit - to manage your migrations

DRM— Data Relationship Management (formerly MDM)

— Enterprise master data management solution

— Source/target agnostic

— Synchronize, blend and compare hierarchies

— Customizable scripting and business rule validations

— Enforce enterprise master data policies

— Change tracking

— Roll back to point-in-time Hierachies Only?Wrong!

Internal Stakeholders

Culture

Stewardship Program

Delivery MechanismsBackend Applications

Infrastructure & Technology

DRM Governance, Stewardship & Change Management

Technology, Infrastructure & Delivery

Culture and Alignment

Constituency Analysis

Shared Services

External Stakeholders

Governance Program

Management

AlignmentDRM

DRM

• Very robust metadata management tool• Can manage and deploy data for Hyperion and

non-Hyperion systems• Does not process data• Formerly known as MDM (Razza)

DRM• Use Cases

– Hierarchies for dimensions in cube(s) are sourced from external system

– Multiple cubes need same hierarchies– Keep all Hyperion products in sync– Centralize and reduce metadata maintenance– Segregate metadata management from system

administration– FDM Mappings, ANY Mappings!

DRM - API

DRM -> EPMA Integration

Export Profiles display the Hierarchies that are to be exported based on the Dimension Type identified via Hierarchy Group naming

Custom designed System Exports are created as part of the Application Template loading. These Export profiles allow exporting of DRM Versions and Hierarchies into EPMA and specific EPM Application Views

New Fields added as part of the EPMA Integration to specify Target EPMA / EPM Application and its Dimension Type

Introducing Data Relationship Governance

• Governance Work list• Two types of workflows:

– Change requests– Remediation requests

• Single & multi-item requests• Pre-validate requests to

ensure quality on data entry• Calculate derived / inherited

attributes• Comments & attachments• Notifications & alerts• Request activity history

Enable change management and remediation workflows natively

Four steps to configuration

DRG Configuration

Workflow Model

Workflow Tasks

Workflow Stages

Workflow Node Access Groups

Model Filters

Essbase Integration Services

• Used to model cubes from relational data sources

• Can be used to build/update outline and load data

• Supports complex SQL if needed• Very fast if used properly


• Pros– Very flexible– Rich dimension building capabilities without

having to mess around with load rules• Cons

– Once model is setup, you’re “married to it”– You need to really think to get it right


• Use Case– Data and metadata already exist in a SQL

database, and now the business would like to build multiple cubes to provide analysis capabilities and possibly drill-thru


Essbase Studio

Essbase Studio• Used to design cubes from relational data

sources, OBIEE, EPMA apps & text file• Used for light-weight data modeling• Evolution of EIS• Common Metadata Repository• Lineage Tracking• Limited EAS functionality provided

Essbase Studio

Essbase Studio• Pros

– Very flexible– Enables true artifact reusability– Introspection is very cool– Drill through (RDBMS, URL, HFM, Java, etc.)

• Cons– Not your father’s Application Manager– Not your father’s EAS

Outline Load Utility• Can load data and metadata• Create a load file in text (CSV)• Complex command line optionsOutlineLoad [-f:passwordFile][/S:server] /A:application /U:userName [/M] [/E:outputFileName] [/I:inputFileName/D[U]:loadDimensionName|/DA:attributeDimensionName:baseDimensionName] [/TR] [/N] [[/R] [/U]] [/C] [/F] [/K] [/X:exceptionFileName] [L:logFileName] [/DX:HSP_Rates][/DS:HSP_SMARTLISTS] [/8][/?]

Outline Load Utility• Pros

– No ETL required– Perform batch outline updates to Planning– Can load data as well

• Cons– Arcane syntax– Practical limitations (Volume)

Outline Load Utility

Outline Load Utility

• Use Case– Company uses Planning to do yearly forecast of

employee salaries. HR provides a spreadsheet with employee data and salary basis data

– Outline load utility can be used periodically to update the employee members in the Planning outline and load up their base data

• Zero Level• Upper Level• All Data• Very fast• Column Format

Essbase Native Exports

• Easy to Use• Fairly Flexible• Can build data & Metadata• Very Modest “ETL”• Can be fast

Essbase Load Rules

@DATAEXPORT• Historically, an Essbase CDF (custom defined

function) used to export data to text file or relational data source

• Is now native functionality (no need to setup CDF, which can be tricky)

• Useful for calling from MaxL (in place of report script)

• Can provide export capability to users by giving them access to a calc script

@DATAEXPORT

• Pros– Can be easier and more flexible than report scripts

• Cons– Can only export data (no metadata)

• Use Case– Exporting data to be used to load another cube

@DATAEXPORT

• Use Case– Data exists in cube but not necessarily easily

accessible from other SQL sources, and wish to generate a text file with a certain subset of data (addressable with FIX)

@DATAEXPORT

@XREF | @XWRITE/*Enables a database calculation to write values to another Essbase database, or to the same database*/FIX (East, Actual, Budget, Sales) "100-30" ( @XWRITE("100-30", sam2basic, "100-20"); ) ENDFIX

/* Enables a database calculation to incorporate values from another Essbase database */Units = Units * @XREF(InflatDB,Inflation,US);

HFM Extended AnalyticsExtended Analytics enables you to export data and metadata from Financial Management applications to a flat file. The data and metadata are exported in a star schema format. Each star schema consists of a fact table and a number of dimension tables. Each record in the fact table of the star schema represents a cell or dimension intersection in the Financial Management data structure.

Essbase APIDeclare Function EsbLoadDatabase Lib "ESBAPIN" (ByVal hCtx As Long, ByVal AppName As String, ByVal DbName As String) As Long

Sub ESB_LoadDatabase () Dim sts As Long Dim AppName As String Dim DbName As String AppName = "Sample" DbName = "Basic" '************** ' Load Database '************** sts = EsbLoadDatabase (hCtx, AppName, DbName)

End Sub

MaxL

• Used to automate Essbase administrative tasks

• Replaced harder to read “ESSCMD”• Can do almost everything to a

server/application/cube/user that can be done in EAS

MaxL

• Commonly used to automate repetitive tasks and processes on an Essbase server

• Comes with Essbase (“free”)• Can import data (load rule), export data

(report script, calc)

ASO MDX Allocations

MaxLset MY_NAME = "ProcessCube";login $ESSUSER identified by $ESSPW on $ESSSERVER; spool stdout on to "$LOGPATH\\$CONTEXT.$MY_NAME.$1.stdout.txt";spool stderr on to "$LOGPATH\\$CONTEXT.$MY_NAME.$1.stderr.txt"; alter database $1.$2 reset data; import database $1.$2 data

connect as $SQLUSER identified by $SQLPW using server rules_file 'LdQVSOS'

on error write to "$LOGPATH\\$CONTEXT.$MY_NAME.$1.error.txt"; execute aggregate process on database $1.$2

stopping when total_size exceeds $3; spool off;logout;exit;

MaxL

• Use Case– Data/metadata available in text file or relational

data source, want to automate on the server or work in to existing automation

– Perl Module

Batch Scripting

Batch Scripting

• Simple batch files used to perform a sequence of tasks

• Lends itself very easily to automating and scheduling for recurring tasks

• Can glue together other data integration tools: MaxL, ODI, outline load, etc.

Batch Scripting• Pros

– Very easy to implement– Not limited to Hyperion products

• Cons– Large scripts can become unwieldy and hard to

test– Many scripts are not good at dealing with an error

in the middle of processing– Custom code not supported– People my laugh at you

Batch Scripting

• Use case– Weekly close process consists of a series of

repetitive steps• Generate text file from another system• Copy text file• Load data to Essbase• Calc cube

– A simple batch file and MaxL script can be created and scheduled to automate this

Essbase SQL Loader• Used as the data source in a load rule• Primarily used for data loading• Also useful for dimension building• Very useful if you want to “skip” putting data

in a text file• Can serve as a stepping stone to using EIS,

ODI, Studio, etc.• Useful in conjunction with automation

EAL for HFM

HFMUser

EssbaseUser

EAL for HFM

EAL HEA

Extracts Data Directly to Essbase Yes No

Extracts Data to Relational Tables Yes Yes

Extracts Metadata to Relational Tables Yes Yes

Extracts Metadata Directly to Essbase Yes No

Does not require Data Manipulation Yes No

Does not require Data Load Rules Yes No

Does not require AIS Yes No

Real Time Data Synchronization Yes No

Manage Process from Essbase Yes No

EAL vs. Extended Analytics

EAL vs. OFMAEAL OFMA

Extracts Data Directly to Essbase Yes No

Extracts Data to Relational Tables Yes No

Extracts Metadata to Relational Tables Yes No

Extracts Metadata Directly to Essbase Yes No

Does not require Data Manipulation Yes Yes

Does not require Data Load Rules Yes Yes

Does require OBIEE No Yes

Real Time Data Synchronization Yes Yes

Manage Process from Essbase Yes No

Automatically builds OBIEE .rpd file No Yes

Automatically builds OBIEE Executive Dashboards No Yes

MediatorMain

Orchestration Composite

HumanWorkflow

Close Manager

Close Web Application Server (ADF)

Mediator

Synchronous Service

EventAdapters

End User Link

SOA

External System

Asynchronous Service

FCM – Data Flow (Integrated)

• System Automated tasks call an external webservice to perform their action– Webservice must support asynchronous calls– WSDL should contain separate porttypes for request and

response– Web service accepts WS-Addressing based reply information.

The WS-Addressing headers should be used by the Web service to direct responses to the correct callback service

– A client proxy is generated dynamically based on the definition– Any parameters are passed to the webservice using the

parameter code/value pair

FCM – Web Service(s) Integration

Account Reconciliation Manager (ARM)

Demo’s and POC’s

Lunch & Learns Client Success Stories & Ref’s

[email protected]

Data integration extravaganza

Technology

Transcript of Data integration extravaganza