Best Practices for Bobj ion With BW

Best Practices for ImplementingSAP BusinessObjects BIIntegrated with SAP BI

Lionel Jeanson, Stephane Thibault & Tobias KaufmannSAP Intelligence Platform & NetWeaver RIG

March 2010

mailto:[email protected]



I. Introduction1) Product positioning, quick reminder2) Architecture3) Data Federator / OLAP Universes positioning

II. Web IntelligenceIII. Data FederatorIV. OLAP UniversesV. BWVI. Conclusion

Agenda

Guided

Search &Exploration

OLAPAnalysis Ad-Hoc QRA Dashboarding &

VisualizationEnterpriseReporting

Interactive ExperienceFree

ProfessionallyInformed

TechnicallyCapable

InformationConsumers

Executives &Managers

BusinessAnalysts

Mapping The Right Tool to The Right ScenarioSupporting an Enterprise BI Strategy – SAP BW context

WebIntelligence

WebIntelligence

BExAnalyzer

(Voyager innon SAP BW

context)

BExAnalyzer

(Voyager innon SAP BW

context)

XcelsiusXcelsiusCrystalReportCrystalReport

Explorer(BusinessObjects

Explorer)

Explorer(BusinessObjects

Explorer)

© SAP 2010 /

ArchitectureThe Big Picture

SAP Business Objects platform

Web Intelligence

NetWeaver BI

SAP NetWeaver – Enterprise Data Warehouse

SAP NetWeaver – BW OLAP Engine

MDX Queries

OLAP Universes(business objectsdefined using MDX

SQL Queries

Relational Universes(business objects defined using SQL)

Data Federator

© SAP 2010 /

BWDF Fascade

Data Federator

RDBMS Analytic Engine

OLAP BAPI ODBOXMLA

UniversesQT ODA

SQL Access MDX Access

WebIntelligence

CrystalReports

XcelsiusQaaWS

Cells

Rows

Rows

rows

Cells

BIA RowsRows

© SAP 2010 /

ArchitectureData Transformations

Data Federator Universe or OLAP Universe?MDX (OLAP Universe) versus SQL (Data Federator Universe)

= fully supported© SAP 2010 /

BW Server Feature MDX SQL

BW Hierarchies

Restricted and Calculated Key Figures

BEx Queries

BW Variables

Currency and Unit Conversion

Exceptions, Conditions

Security

AVG, COUNT, SUM, MIN, MAX Aggregations

Navigational Attributes

Mass Data Enabled

Ad-hoc Reporting

Federation (e.g. BW – RDBMS)

Non Cumulative Key Figures

= limited support / workaround

BW Server Feature Workaround

Restricted & Calculated Key Figures,Formulas Define calculated measures in Universes / WebI

BW Variables Define @prompt in Universes or WebI Query Panel; does notwork for exit variables or variables filled through authorizations

Currency and Unit Conversion Simulate via joins in Data Federator

Exception Aggregations SQL standard aggregations (AVG, COUNT DISTINCT) availablevia Data Federator processing

Display Attributes Define joins (in DF) from InfoProvider to master dataInfoProvider tables

Conditions Use filters on measures

Exception Use Alerters in Web Intelligence

BW Hierarchies No workaround

Non-Cumulative Key Figures No workaround

© SAP 2009 /

Data Federator Universe or OLAP Universe?MDX versus SQL - Workarounds

Question 1Are BEx queries already inplace and should be usedas basis for Universes?

Question 2Do you require specific BWOLAP features not covered

in the Web IntelligenceQuery Panel or Universe

Designer?

Implement connectivity throughData Federator

Check BEx query and WebIntelligence query design andapply latest patches

Evaluate effort / risk to replaceBEx queries by Data Federator.If feasible, do so. If not, see A.

A

B

C

D

YES

NO

YES

YES

NO

NO

Carefully review and redesignBEx and Web Intelligencequeries for performance andapply latest patches.

Web Intelligence and BW InteroperabilityDecision Help for Web Intelligence / BW projects

© SAP 2010 /

I. IntroductionII. Web Intelligence

1) Query design2) Performances hints

III. Data FederatorIV. OLAP UniversesV. BWVI. Conclusion

Agenda


Web Intelligence(uses for analysis and reporting – relational metaphor)

NetWeaver BI



MDX Queries SQL Queries


Data Federator


Query designQuery Stripping

The Query Panel Definition iswhat defines the “Microcube”and thus the MDX or SQL Querywhich is executed on backend

When refreshing the data inthe Web Intelligence report, theentire Microcube is refreshed

© SAP 2010 /

© SAP 2010 /

This feature is only supported for OLAP Universes.

Query designEnabling Query Stripping – New feature in XI 3.1 SP3

Query designSplit your query to extract more constant master data (1/3)

For each row, the number of cells will be 17:2 for Country (index + value)

2 for Order Date (index + value)2 for Order ID (index + value)

1 for Delivered Value1 for Order Quantity,

9 for Customer(1 for the value, 1 for the index, 7 detail cells)

Let’s assume there are 1000 rowsTotal number of cells = 17 x 1,000 = 17,000


For each row, the number of cells will be 10:2 for Country (index + value)

2 for Order Date (index + value)2 for Order ID (index + value)

1 for Delivered Value1 for Order Quantity,

2 for Customer (index + value)

Total number of cells = 10 x 1000 = 10,000

The total number of master data will be 9:9 for Customer

-(1 for the value, 1 for the index, 7 detail cells)

© SAP 2010 /

Total Number of cells= 10,000 + 9 = 10,009


Use the Web Intelligencefeatures (Merged Dimension)

© SAP 2010 /

Web Intelligence, performance hintsReport size, scheduling, server groups

© SAP 2010 /

BusinessObjects Enterprise cluster tuningMany tuning possibilities…

© SAP 2010 /

http load balancing and httpstatic content front end servers

Web application serverclustering and tuning

Business Objects Enterprise(BOE) servers : CMS, job

servers, CMS and AuditingDB…Beware no more than 3 CMS, may lead to desynchronization of the BOE cluster.

I. IntroductionII. Web IntelligenceIII. Data Federator

1) Key Concepts2) Optimization3) Future

IV. OLAP UniversesV. BWVI. Conclusion

Agenda



NetWeaver BI





Data Federator


Runtime

Design

Data FederatorKey Concepts

Flexible data modelingUnified view (Design Multi-Source Data Foundation)Connect to your existing information systemsTransform data on the fly (function, mapping rules…)

Query any sources uniformlyMulti-Source SQL Query engine– Including multi-source join operators– Pull required data at runtime (access information on demand)– Parallel query execution

Query optimizer– Generate an optimal global execution plan according to the underlying sources– Delegate dynamically computations to sources

Rewrite SQL statements– Generate native queries (SQL, …)– Can generate non relational statements (program and function calls…)

© SAP 2010 /

Data FederatorHow it works – Design time

© SAP 2010 /

Data FederatorHow it works – Run time

© SAP 2010 /

Data Federator OptimizationsConnector

Important Connector Settings

- CallbackServerProgramID- Package Size- ForcedCapabilities- DebugReportID- useBinaryXML- maxTablePathLength

© SAP 2010 /

Data Federator OptimizationsConfiguration (1/3)

You can tune the performance of your queries.

1. system parametersMemory used by the JVM

core.bufferManager.executorMemory

core.bufferManager.executorStaticMemory

core.bufferManager.maxConcurrentQueries

core.bufferManager.maxConcurrentOperatorsPerQuery

© SAP 2010 /



1. system parameters



2. statistics

© SAP 2010 /


You can tune the performance of your queries


2. statistics

3. bind join operator

© SAP 2010 /

leselect.core.optimizer.bindJoin.minCardinalityleselect.core.optimizer.bindJoin.reductionFactor

leselect.core.optimizer.bindJoin.maxQueriesleselect.core.optimizer.bindJoin.useIndexOnly

Data Federator Enhancements (1/2)Specific to SAP data source

Copy source tablesThis option lets you copy the source table schema as target tables

© SAP 2010 /

Data Federator Enhancements (2/2)Specific to SAP data source

Copy source tablesThis option lets you copy the source table schema as target tables

© SAP 2010 /

Filter / Search SAP BW InfoProvider

Automatic SAP BW callback registrationThe SAP BW callback registration is simplified starting XI 3.1 SP2.The declarative SP1 approach is still available but by default Data Federatorconfigures itself automatically and no manual callback registration is required.

I. IntroductionII. Web IntelligenceIII. Data FederatorIV. OLAP Universes

1) Bex query & Universes2) Sizing3) Future

V. BWVI. Conclusion

Agenda



NetWeaver BI





Data Federator


OLAP UniverseConcepts / Definitions

DefinitionMapping of OLAP data sources

ObjectiveCommon user experience for all query & reporting needs whatever the data source is

In SAP BusinessObjects XI 3.1 SP2Use the current universe query specification and Web Intelligence query panelMetadata catalog integrated with connection serverUse Voyager driversUse MDX as query language

© SAP 2010 /

1 Create connection

OLAP UniverseAccessing OLAP Data

2 Select cube

3 Automatic creationof universe

No representation oftables as we havewithin a RDBMS

universe.

© SAP 2010 /

OLAP UniverseSAP NetWeaver BW Metadata Mapping

Class = Dimensions grouping

Detail = Attribute

Object = Characteristic level or Hierarchy level

Sub-class = Characteristic or Hierarchy

Measure = Key Figure

Filter = SAP variable or predefined filter

© SAP 2010 / © SAP 2010 / Page 30

© SAP 2010 /

OLAP UniversesSizing WIReportServers

Size Web Intelligence as normal with Sizing Guide/Sizing Spreadsheet: thenFor each Web Intelligence Report Server CPU count 2 instancesEach Web Intelligence Report Server should have 2 GB RAM, so 1 CPU Web Intelligence is equivalent to 4GB RAMSet Number of Connections to 14-15, then test

This will control your load balancing‘sweet spot’ may depend on report sizes, etc.Observe tests and adjust further if necessary if load is spread unequally

Looking for relatively busy CPU (after all, tests should run for max. load)Looking for equal/close to equal memory consumption for all Web Intelligence Report ServersKeep memory thresholds setting high (default):

we don’t want to trigger ‘Memory Analysis’ at all, if we can avoid it

Will lead to similar CPU estimates, but higher memory requirementsAssumption: nobody will be concerned if memory requirements come out higher at initial sizing. Much harderto adjust once the sizing has already been established and hardware purchasedSince only memory related, has no licensing impact

© SAP 2010 /

A most successful configuration

Note that loadbalancing WebI instances is supposed to be weighted round-robin. In practice, servers“early” in the server list are preferred, and receive more requests. When instances are set to unlimitedconnections, or the default 25-40, server instances may reach memory threshold well before other serversreach capacity, or are even loaded fully. Number of connections has to be configured appropriately

Observed behaviorMost current patch level for memory fixes (LAFix 0.2.22, FP 1.9, FP 2.1)Higher memory consumption than expectedFewer reports can be run per server instanceLow CPU usage, so extra capacity thereLook for configuration that allows for better load balancing between WebI instances

Most successful configurationMore instances of Web Intelligence Report Server / CPU (generally 2)2GB RAM per WebI instance as normalReduced number of connections per WebI Report Server instance to force better load balancing: 12-16 (14-15)

Filter Editor

This editor applies to OLAP universes.It provides a User Interface guided workflow to build the predefined definition

The User Interface offers all necessary controls for data and integrity checksIt will also allow to call the prompt editorIt will provide access to universe metadata

OLAP UniverseNew features - BOE XI 3.1 SP3 (1/2)

© SAP 2010 /

SDK improvements to support all features for OLAP universesGenerate and update OLAP universesPossibility of OLAP universes refresh scheduling or mass universes generation

List of Values manageable through SDKUniverse refresh: more generation and update parametersMetadata parsing improvements

Other features

© SAP 2010 /

OLAP UniverseNew features - BOE XI 3.1 SP3 (2/2)

I. IntroductionII. Web IntelligenceIII. Data FederatorIV. OLAP UniversesV. BWVI. Conclusion

Agenda



SAP NetWeaver BI





Data Federator


Web Intelligence and BW Interoperability

Data Manager

Relational Universe

Web IntelligenceWeb IntelligenceBEx Web / BEx Analyzer

MicrocubeMicrocube

MDX Interface

BEx Queries

BW InfoProviders

MultidimensionalUniverse

BEx Queries

BW InfoProvidersBW InfoProviders

Classic BW / BEx setup Web Intelligence based onBEx Queries

Web Intelligence based onData Federator

Data FederatorBEx

BW

BOE

BW

BOE

BW

BEx Queriesdesigned for

OLAP navigation

Advanced OLAPnavigation on BW

BEx Queries asdata basis for Web

Intelligence andMicrocubes

MDX is not amass datainterface

Data Manager is amass data

Make use of BW AnalyticEngine but query design

according to WebIntelligence use case

Highly intuitivenavigation for

business users

BW Analytic Enginefunctionality not

available

© SAP 2010 /

Web Intelligence Report and BEx Query (2/2)Design Time and Runtime

Query Stripping

XI 3.1 SP 3

Wrong Design

Manual changerequired

SAP NetWeaverBW 7.02

© SAP 2010 /

BEx QueryDesign Recommendations

Reduce Usage of Navigational Attributes as much as possibleAlso, if simply displaying a Characteristic’s Attribute, DO NOT use the NavigationalAttribute – rather utilize the Characteristic Attribute for display in the reportThis avoids unneeded JOINS, and also reduces total number of rows transferred to WebI

Reduce Usage of Restricted Key Figures (RKF) as much as possibleRKF “column restrictions” are carried out in the OLAP processing – this results in highervolumes of data being transferred from DB / BIA to the OLAP engineConsider persisting the calculated Key Figures in the InfoProvider if possible

Using the Selection of Structure Elements if usefulUse Selection of Structure Elements option can be selected for any query; however, it isonly useful in some casesCheck FEMS number in Query / Technical Information / StatisticIf you do not select the Use Selection of Structure Elements option, these dynamic(structure element) selections are not transferred to the databaseNormally, the data for the entire structure or for both structures is then read from thedatabase and stored in cache

Simplify as much as possible

© SAP 2010 /

BEx QueryParts of Execution Time

Query generation

Determinating variable

default values

Exp/Imp buffer toosmall

Complex customer logic

for variables

…

User restriction onvariables

n.a.

Preparing hierarchyprocessing

Reading query data

Missing aggregates

SQL execution plan

Hierarchy buffer

Uncompressed data

Missing indexes…

© SAP 2010 /

BEx QueryParts of Execution Time

Calculation ofkey figures

Complex queries

Latest performance

optimizations inactive

…

http://www.sdn.sap.com/irj/sdn/bi-performance-tuning

Reading texts andattributes

'List queries'

Fragmented indexes

…

Transferring result setto frontend

Formatting

Network problems

Frontend hardware

Large result sets

Complex formatting

…

© SAP 2010 /

http://www.sdn.sap.com/irj/sdn/bi-performance-tuning

I. IntroductionII. Web IntelligenceIII. Data FederatorIV. OLAP UniversesV. BWVI. Conclusion

Agenda

Data Federator Performance StrategiesPush Processing To Data

GoalsImprove query response timeAvoid unnecessary data transferReduce memory consumption (increase scalability)

Optimized queriesRe-write queries (Parameterized queries…)Multi-source join operator based on data pattern (bind join)Intelligent data connectionLeverage database indexes

ScalabilityPipeline architecturePredictable pipelined memory managerOperators have been designed for optimal and low memory consumption

© SAP 2010 /

PartitioningDatabase instancesFunctional partitions

Load balancingDispatch workload between query servers

© SAP 2010 /

Data Federator Performance Strategies

OLAP UniversesDesign of Universes / BEx Queries

m : n

n > 1

1 : 1

Several WebI Reportsshould use one Universe

Several WebI Reportsshould use one Universe

Do not puteverything inone Universe*

Do not puteverything inone Universe*

One BExQuery for one

Universe

One BExQuery for one

Universe

1 < a

Cluster BExQueries withsimilar query

elements

Cluster BExQueries withsimilar query

elements

Do not add elements to BEx Querythat are unused in Universe

Do not add elements to BEx Querythat are unused in Universe

© SAP 2010 /

1 : 1

a : b

BEx QuerySet Query Properties using Query Monitor

Transaction Code RSRT

© SAP 2008 /

1. EnterQuery

2. Press Properties

3. Click checkbox

Transaction Code RSRT

BEx QueryView FEMS Number in Statistics

© SAP 2008 /

1. EnterQuery

2. Press TechnicalInformation

3. Click Statistics

4. View FEMSNumber

Disclaimer

This presentation outlines our general product direction and should not be reliedon in making a purchase decision. This presentation is not subject to your licenseagreement or any other agreement with SAP. SAP has no obligation to pursueany course of business outlined in this presentation or to develop or release anyfunctionality mentioned in this presentation. This presentation and SAP's strategyand possible future developments are subject to change and may be changed bySAP at any time for any reason without notice. This document is provided withouta warranty of any kind, either express or implied, including but not limited to, theimplied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in thisdocument, except if such damages were caused by SAP intentionally or grosslynegligent.

© SAP 2010 /

Best Practices for Bobj ion With BW

Documents

Transcript of Best Practices for Bobj ion With BW