Business Intelligence in SAP Environments: Understanding the value of complementary BI Solutions

Inforte - A Business & Decision CompanyDavid Dixon

BUSINESS INTELLIGENCE IN SAP® ENVIRONMENTS:Understanding the value of complementary BI solutions

Copyright © 2008 Inforte Corp. All Rights Reserved. www.inforte.com

WHITE PAPER - Business Intelligence in SAP Environments: Understanding the value of complementary BI solutions

TABLE OF CONTENTS

1. UNDERSTANDING THE VALUE OF COMPLEMENTARY BI SOLUTIONS .....................................1

1.1 White Paper Objective ..............................................................................................................1

1.2 Executive Summary ...................................................................................................................1

1.3 Definition of Terms ....................................................................................................................2

2. “WHY NOT SAP?” .....................................................................................................................5

2.1 SAP Usability Considerations .....................................................................................................5

2.2 SAP Openness Considerations ...................................................................................................7

3. WHY CONSIDER HYBRIDS? .......................................................................................................9

3.1 “One-Stop-Shop” Misnomer .......................................................................................................9

3.2 “Best-of-Breed” Misnomer.........................................................................................................11

4. APPROACHES FOR SAP INTEGRATION ...................................................................................11

4.1 Data Quality Management in SAP ............................................................................................12

4.2 Interface-Based Access Approaches ..........................................................................................13

4.2.1 Open Analysis Interfaces Data Access....................................................................................14

4.2.2 Operations-Based Data Access..............................................................................................15

4.3 Extraction-Based Data Access Approaches.................................................................................15

4.3.1 SAP Business Suite Data Access.............................................................................................16

4.3.2 SAP BW Data Access...........................................................................................................17

5. EVALUATING RETURN ON INVESTMENT .................................................................................17

6. CONCLUSION............................................................................................................................20

7 APPENDICES..............................................................................................................................21

7.1 Specialist BI Selection Criteria ..................................................................................................21

7.2 SAP Platform Considerations ....................................................................................................21

7.3 SAP Front-End History..............................................................................................................22

7.4 Query Design for Open Analysis Interfaces ...............................................................................24

7.5 Open Analysis Interfaces .........................................................................................................25

7.6 ABAP™-Based Data Access .......................................................................................................27

7.7 SAP BW Metadata .................................................................................................................28

7.8 SAP Delta Change Capture for Extraction..................................................................................30

7.9 Open Hub Services.................................................................................................................32

1.1 WHITE PAPER OBJECTIVEThis white paper discusses how complementary Specialist Business Intelligence (BI) solutions can fit within a hybrid architecture strategy, whetheryour organization is an “SAP shop,” running the majority of their processes on SAP, or has “SAP in its shop,” with a heterogeneous enterpriseapplication environment that includes SAP solutions.

Because of growing IT complexity and the higher stakes involved with enterprise solutions, decision makers increasingly rely on specialists andconstituents for support. When considering such support, decision makers must take into account both immediate, tactical, project-basedconcerns and longer-term strategic implications.

Building consensus on support decisions not only distributes political risk and promotes inclusiveness, but also promotes objectivity and thetechnical accuracy of recommendations. But if consensus is split, as is often the case in BI environments consisting of SAP and non-SAPpractitioners, then consensus-building becomes a bottleneck. The explanations proposed in this document can serve as a communication tool tohelp expedite the arbitration of differences.

Project teams that face tight timelines to perform high-impact BI evaluations or assessments that need to incorporate how to position SAP andSpecialist BI solution capabilities will also benefit from this white paper.

1.2 EXECUTIVE SUMMARYA 2007 Information Week survey found that decision makers ranked “BI vendors that can integrate with existing operational applications”at the top of their strategic-planning wish lists.1

Ironically, however, when it comes time to evaluate BI solutions, organizations often divide themselves into product-aligned camps. They debateideological differences that preclude the other camp, when in fact Specialist BI solutions can complement SAP irrespective of footprint size.

SAP shops typically put into place governance rules that categorically ask “Why not use SAP?” failing to understand the hybrid options withinSAP solutions as well as across vendors. Recent SAP acquisitions and tighter partnerships are opening up new hybrid options such as the recentSAP offer to acquire Business Objects.2 This paper shows that there is indeed value that complementary Specialist BI solutions add to customersthat run SAP solutions. The optimal solution isn’t an either/or choice, but rather one which incorporates both elements. Assumptions that SAP isa good enough “one-stop-shop”—or on the other hand, that SAP complicates “best-of-breed” BI environments—are no longer relevant.

One serious perceptual problem involves the misuse of oversimplified labels. Both “one-stop-shop” and “best-of-breed” are inappropriatebecause of rapidly-changing industry dynamics. In the first place, SAP “one-stop-shop” architectures are falling short by sweeping unmetneeds under the carpet. In addition, “best-of-breed” solutions from Specialist BI providers now offer “one-stop-shop” BI in their own rightthat can complement rather than complicate SAP deployments. Using inaccurate labels can create impressions that lead decision-makingdown the wrong path. Strategically leveraging the respective strengths of all solution providers make “best-of-both-worlds” or “best-of-hybrid” capabilities possible.

To be clear, “best-of-both-worlds” does not mean BI nirvana, but rather an environment that is better than having to pick a vendor-alignedextreme. Achieving a more balanced and optimal state with hybrid architectures has less to do with buying into the the hype around thelatest buzzword technologies and more to do with focusing on established interfaces, maturing technologies and best practice techniquesthat are coming of age. But perhaps the tipping point is not so much technology-driven as knowledge- and experience-driven. After yearsof investment in studying SAP concepts, data and technology nuances, Specialist BI providers have developed proven solutions capableof leveraging preexisting investments in SAP transactional systems. The stabilization of SAP integration touch points, as well as pre-delivered best practices, promises to turn hybrid BI architectures from a legacy landscape problem into a strategic infrastructure asset.


WHITE PAPER - Business Intelligence in SAP Environments: Understanding the value of complementary BI solutions 1

Hybrid architectures are a reality that is not going away any time soon, no matter how big the SAP footprint. Rather than factionalize oruniversalize SAP architectures and programs, IT organizations should realize that the ever-growing scope and complexity of their enterprisearchitecture, as well as the need to yield better business value, necessitates a multi-product and customization strategy.

The questions you should weigh are:

• What “best-of-hybrid” approach gives the most capability and flexibility?

• Which “best-of-hybrid” approach keeps options open and minimizes lock-in risks?

Addressing these questions can be done in a more pragmatic and incremental way through iterative transitions, rather than attempting a multi-year, “Big Bang” approach whose business benefits will invariably decline as the world changes. The key to defining the optimal transition statearchitecture is to understand each product’s present-day strengths and weaknesses. Not all BI providers that have SAP solutions are createdequal, and it is important to consider criteria such as a solution providers’ experience working with SAP customers and the maturity of theirsolutions. (See Appendix 7.1 for a detailed list).

SAP data-centric organizations that presume “SAP-speak” is the lingua franca of BI risk alienating others, who in turn often craft workaroundsthat unravel the spirit of SAP centralization. By taking an SAP BW “one-stop-shop” approach, organizations inadvertently create walls that canforce isolated groups to choose preferred “best-of-breed” tools without the benefits of a single version of truth, shared support and transparency.

Rather than letting hidden costs mushroom, SAP shops should consider expanding their architectural vision to address unmet needs by leveragingcomplementary, Specialist BI solutions. Not only can Specialist BI tools reach larger audiences, but core constituents can also benefit frombroader and deeper BI capabilities. Otherwise, an ambitious SAP initiative may atrophy. The need for fast and flexible non-SAP data integrationand end-user time-to-value should not be underestimated.

Whereas the decision to standardize on a single ERP vendor is often made with a view toward gaining efficiencies, making total cost ofownership (TCO) the prevalent financial driver, the application of BI inhabits a far more strategic plane. If properly applied, BI goes beyondsimple reporting, and can drive fundamental business decision-making at many levels throughout the organization. For this reason alone, it isimportant to understand what the business is trying to accomplish when choosing amongst solution providers. While TCO may be king for ERP,ROI is the applicable approach for BI.

Even so, many organizations still need to consider TCO. When evaluating the various approaches, the enhancement costs of a single-productapproach must be weighed against the integration costs of a hybrid approach. TCO also comes into play when information consumers evolveinto their own producers and eliminate the middleman (i.e. BI developers). Business is changing faster than IT can support. As a result, backlogsare burgeoning and lengthy projects lose value. Faster time-to-value not only positively impacts business opportunity costs, but also resourceallocation and operational efficiencies.

As automation shifts tasks into higher-brain-function knowledge work, empowering the knowledge worker with usable tools becomes anincreasingly important—if not the most important—driver in business value creation in the information revolution.

1.3 DEFINITION OF TERMSBefore reading this document, you should be familiar with a number of acronyms, terms and concepts in order to avoid getting lost in words ormissing important connotations. This section outlines the vocabulary used in this white paper.

The objective of this section is to capture the distinctions between terms influenced by SAP but reconciled with the rest of the industry so thatyou can better appreciate “buzzword” nuances.



Note that other sources, some of which are cited below, also have comprehensive and slightly varying definitions for the same terms; you areencouraged to reference them for further background.

Data Warehouse, also more broadly referred to as Enterprise Data Warehouse (EDW), is a term and architectural approach that writerBill Inmon is largely credited with originating and popularizing. Inmon has a very specific definition for data warehousing, and has writtenmultiple books on the subject. For the sake of this white paper, the important nuance to grasp is that an EDW is: 1. Not the same thingas BI, 2. Should not be confused with data marts, and 3. By definition is a separate system from transactional ones. SAP positions theEDW as an IT scenario that can be built with the functions of its SAP NetWeaver™ technology platform. This paper uses the term EDW torepresent the back-end part of BI that processes, manufactures and prepares data for the front–end, and consists of activities such asextraction, transformation and loading (known as “ETL”), data transfer scheduling, data modeling and data management (such as aggregatebuilds and indexing).

Data Mart is a term that has numerous varying interpretations and nuances, a problem exacerbated by SAP’s own technical and specificuse of the term—that is, for transferring SAP BW data to another SAP system. This paper uses this term to either represent a subset of datathat sources its data either from the EDW or the underlying transactional systems, thereby by-passing the EDW. Data marts are usually theunderlying data structures for reporting, querying and analysis, while the EDW is more of a central data repository. As a result, data martsare generally faster and simpler to implement but are done so in isolation.

Furthermore, they are typically (but not necessarily) dimensional in nature in order to better support OLAP (online analytical processing) formsof analysis. SAP dimensional data structures are called InfoCubes, and in this case SAP refers to Ralph Kimball’s body of work for data design.Beyond contributions to dimensional data design, Kimball proposes an alternate approach to EDW architecture. Kimball’s “bus” architecturereaps the benefits of fast implementation without the isolation tradeoff. The author anticipates that as SAP NetWeaver Master Data Managementgains adoption as a data repository this enterprise “bus” approach will gain popularity. However, in the meantime, EDW in the SAP worldusually means taking the Inmon approach, and Kimball’s main contribution is reflected in InfoCubes.

OLAP is an acronym for Online Analytical Processing, and was coined in 1993 by E.F. Codd, popularly known for his academiccontributions to relational database theory and design. This paper will not consider OLAP’s academic aspects, but rather its significanceas it relates to SAP BW. Namely, all the SAP front-end BI tools known as the SAP Business Explorer (SAP BEx) Suite, as well as the majorityof third-party BI tools, access SAP BW data universally through SAP’s native and proprietary OLAP processor. SAP’s OLAP, or analyticalengine, is the heart of SAP BW data access no matter whether the data is flat or dimensional, and is the underlying interface to integratedplanning, data mining and the SAP NetWeaver Visual Composer (explained in Appendix 7.3 in more detail).

Business Intelligence (BI) refers to the front-end layer (e.g. OLAP and presentation tools) that sits on top of the data layer (i.e. martsand warehouse), and is also used as an all-encompassing umbrella term. So as not to confuse the two usages, the front-end “out-of-the-box” application capabilities will be more specifically referred to as part of a “BI Suite,” while the overall development infrastructurecapabilities will be referred to as part of a “BI Platform.”

Suite refers to a family of applications or tools, connoting integration across the family of software and functionality that is pre-delivered“off-the-shelf” or “out-of-the-box” rather than requiring custom programming. A suite represents the full breadth of software capabilitiesunder the same brand. SAP ERP, SAP Customer Relationship Management (SAP CRM) and SAP Supply Chain Management (SAP SCM)are examples of SAP Business Suite applications. This paper will on occasion refer to the SAP Business Suite as “SAP transactional systems”in order to distinguish between its online transaction processing (OLTP) and OLAP capabilities.



Platform is an environment that is designed to support third parties (customers or other vendors) in developing their own applications. The platformitself may consist of applications, frameworks, reusable services, interfaces, tools and an integrated development environment (IDE). Some of the better-known platform examples are .NET from Microsoft and WebSphere from IBM. SAP NetWeaver is SAP’s technical platform.

SAP NetWeaver is the underlying application development infrastructure for SAP’s own SAP Business Suite applications and industry-specificapplications. The SAP NetWeaver technical platform also enables composite applications where third-party vendors or customers can buildcustom business processes by integrating SAP applications with non-SAP applications. Such integrations occur via reusable enterprise servicesusing SAP NetWeaver not only as a technical platform, but also as a business process platform (BPP). SAP brands these composite applicationsas “xApps,” like SAP xApp™ Resource and Portfolio Management (SAP xRPM), which loads SAP and non-SAP projects for resource andportfolio management. (Appendix 7.2 gives a historical background to SAP NetWeaver and the implications it has on SAP BW).

EA is an acronym for Enterprise Architecture. EA represents a rapidly evolving and expanding discipline that has its roots in recognizing

and holistically addressing the growing complexity of IT, as well as its corresponding difficulties in meeting business needs. It is a discipline

that traces back twenty years to the thought leadership of John Zachman, but has now broadened to comprise many different competing

methodologies and frameworks. EA has enjoyed significant private sector adoption and legally-mandated adoption in the public sector.

For the sake of this white paper, the use of the term “EA” is meant to suggest that SAP cannot represent a holistic architecture and plan

for the entire enterprise, as EA represents a much larger concept.

SAP BEx is SAP’s branded BI suite. The SAP BEx consists of a grouping of BI tools such as SAP BEx Query Designer, SAP BEx ReportDesigner, SAP BEx Web Application Designer, SAP BEx Analyzer and SAP BEx Broadcaster. Using these tools, dashboards, reports and ad-hoc analysis can be developed on the web or in Excel without custom code. The SAP BEx also includes the OLAP processor and onlyworks with SAP BW.

SAP BW stands for SAP Business Information Warehouse, and is SAP’s branded BI platform that includes SAP BEx. The term “SAP BW”was dropped by SAP after SAP BW release 3.5 (also known as SAP NetWeaver BI 2004). The subsequent release was named SAPNetWeaver BI 7.0 (formerly known as NetWeaver BI 2004s). Because SAP brand and release names have been subject to frequentchange, most customers still refer to SAP NetWeaver BI as SAP BW particularly in reference to the back-end such as the data warehouseor data models. As a result, this paper will continue to use this term for readability.

ERP Reporting will be used in this paper to represent the reporting and analysis capabilities of SAP Business Suite applications. As earlyreferences to SAP CRM, SAP SCM and other enterprise applications used terms like “extended ERP” or “ERP II”, this paper will use “ERPreporting” as an umbrella term to encompass these enterprise applications as well. SAP ERP information systems like Logistics (LIS), HumanResources (HRIS) or Financials (FIS), as well as tools such as Report Painter and Query Painter, are examples of ERP reporting. SAP solutionmaps refer to ERP reporting as part of their “Analytics” capabilities.

Specialist BI is a term used in this paper to refer to SAP third-party BI vendor suites sometimes traditionally referred to as “non-SAP” (inthe SAP context), “pure play” or “best-of-breed” BI vendors that have comprehensive capabilities, both in terms of breadth and depth. Assome of these BI vendors have expanded their offering to become “one-stop-shops” in their own right and in the case of Business Objectsare in the process of being acquired, the Specialist BI term is being used in place of the traditional terms. Furthermore, a distinction mustbe made between Specialist BI vendors like Business Objects, Cognos or SAS and Niche BI vendors, as described below.

Niche BI refers to vendors that work within a specific product category in BI. The category not only includes Niche BI reporting andanalysis tools such as Panorama or XLCubed, but other BI product categories such as ETL, vendors like Informatica or Ab Initio, or datawarehousing vendors such as Teradata and Kalido.



2. “WHY NOT SAP?”No matter the size of the SAP footprint, organizations are increasingly asking, “Why not SAP?” especially in areas that are already paid forin their license agreement, such as SAP BW. The business case for SAP BW is that it is part of the SAP NetWeaver technical platform and henceintegrated (or integrateable) with the rest of SAP’s Solutions.

What follows is some historical context around the evolution of SAP BW to separate myth from fact. In addition, this paper will make the casefor employing Specialist BI tools to complement present-day SAP BW (i.e. SAP NetWeaver BI 7.0) capabilities. As IT investments in both SAP BW and Specialist BI soltuions are growing, it makes pragmatic business sense to leverage both rather than push one out, even in an EA-disciplined organization.

First, this paper will discuss SAP front-end usability, followed by a consideration of back-end openness from a BI and ERP reporting perspective.The Information Week survey noted earlier indicated that integration and compatibility problems, followed by ease-of-use issues, are the toptwo roadblocks standing in the way of enterprise-wide adoption of BI.3 For an understanding of how the SAP NetWeaver technical platformunderlies usability and openness refer to Appendix 7.2.

2.1 SAP USABILITY CONSIDERATIONSSupreme Court Justice Potter Stewart’s famous quote defining obscenity—“I know it when I see it”—applies equally well to usability. Forcustomers, usability is a product feature that is more cost-effective to consensus-score rather than empirically test and measure as part ofa scientific work study. This white paper will not go into depth describing usability features, but rather delve into center-of-design concepts.As it relates to SAP tools, a distinction is made between developer usability and end-user usability. Historically, SAP report developmentand usage were clearly divided into two roles: the developer and the user. As an ultimate result, production design became developer-centric and centralized to avoid expensive rework. In contrast, Specialist BI typically blurred the distinction by putting development intothe hands of the end-user, enabling autonomy. In fact, some BI products require zero report development, relying on “on-the-fly” andbookmarking capabilities.

Usability goes beyond just graphical capabilities in the user interface. Appendix 7.3 provides a personal perspective that illustrates how SAP’scenter-of-design for report development was historically centered on developers. SAP applied software engineering principles to reportdevelopment so as to eliminate redundancies and promote reuse through a building-blocks approach. This design approach to separatedevelopment into logical and separated units of work carried through to SAP BW. In contrast, Specialist BI historically centered on end–users,empowering them to build reports and analysis from scratch with a high degree of visual what-you-see-is-what-you-get (WYSIWYG) designcontrol. More details are provided in Appendix 7.3.

But the centers-of-design for both users and developers are converging as the toolsfor each become more sophisticated and easier to use. The potential to build hybridsolutions by “mashing up” SAP with Specialist BI capabilities is around the corner.Presently, SAP pilot projects are testing next-generation SAP NetWeaver APIs, calledBI Consumer Services, that not only enable universal data access but support accessto functions such as exception alerts, list rankings and document attachments.4 With

the integration of SAP NetWeaver Visual Composer into SAP NetWeaver Composition Environment (SAP’s Java-based development platform) it ispossible that front-end tools from third-party vendors can be mashed up into a composite application. Specialist BI solutions already provide thiscapability, typically as part of dashboard designer tools. SAP may follow suit. Currently, it is not possible to mix and match SAP front-end tools likeSAP BEx Report Designer, SAP BEx Web Application Designer and SAP NetWeaver Visual Composer without programming or separating theminto portal iViews (SAP portlets). At some point, either Specialist BI or SAP BW dashboards will become ‘mash-boards’—a mixed-and-matchedcomposite of user interface components and web services. Some Specialist BI tools already directly support seamlessly integration with SAPNetWeaver Portal via iViews, and can interact across iViews and auto-generate reports into easy-to-use web services.



The potential to build hybrid solutions by“mashing up” SAP with Specialist BI capabilitiesis around the corner.

Figure 1 depicts the complementary capabilities of SAP BW versus Specialist BI tools. The power of SAP BW is that it can be used like die-castmolding—the output is predefined and standardized for mass consumption. Die-casts take more pre-defined design and planning, but once inplace are reusable over and over. The power of Specialist BI tools is that it can be used like molding by hand, which is more flexible, easier topick up, on-the-fly and WYSIWYG. Having both options for molding information gives organizations the power to support both centralized anddecentralized solutions in a controlled EA. Top-down developer-centric solutions can be balanced with bottom-up user-centric solutions, ratherthan stir conflict in the middle.



In conclusion, Specialist BI tools can still offer complementary capabilities to the latest release of SAP BW tools by providing users tools thatare self-service driven without undermining the strength and capabilities of SAP BW.

2.2 SAP OPENNESS CONSIDERATIONSThe author’s first customer assignment at SAP was to write an interface to load data from a flat file that would eventually roll up into the financialconsolidation ledger. The program was not that flexible; the file was fixed length and had to be in a specific file format.

About a year later, SAP came out with a financial consolidations-based tool called “flexible upload”—“flexible” in that the structure and fileformat were configurable—that retired the development.

There is now many standard and stable APIs available in SAP. In particular, profitability analysis (CO-PA) and LIS have long supportedexternal data interfaces with customizable structures and mapping rules. Even from the earliest days of SAP BW, flat file data loadingwas supported.

These interfaces require files in a file structure that is accessible via the SAP application server. Later, SAP supported direct loading frommultiple external database systems; this was done by opening extra database connections on the application server in addition to thedefault. The constraint in this case was that the database system had to be one of the limited set that SAP itself runs on. SAP called this

Technology Platform for Programmer Developers

Application Self-Servicefor Business Users

SPECIALIST BI

SAP

IBM

Figure 1 - Developer versus User Spectrum

type of SAP BW data source DB Connect. Then, when the SAP application server supported a Java stack, SAP introduced Universal Data(UD) Connect, designed to connect directly to anything Java can. Java Database Connectivity (JDBC) alone has over 220 drivers for allmajor relational database management systems.

But, technically speaking, connecting to an external data source and translating that data source into SAP semantics is much like placinga call to China and then speaking Chinese. SAP’s recent acquisition activities for Specialist BI and OEM agreement with Niche BI(Informatica) will help SAP customers build those translations without having to program them in ABAP. From a licensing perspective, thesedeals are more than just a replacement to the reseller agreement SAP had with Ascential before that company was bought by IBM. Thesedeals imply SAP recognizes the need for seamlessly integrated outside ETL capabilities to complement their own. Specialist BI suites offerETL capabilities but also can go beyond, from the OEM perspective, with enterprise information integration (EII) as well as data qualityand cleansing capabilities.

EII, also known as data federation, is the equivalent of ETL, minus the data replication. In fact, SAP BW has data federation capabilities byincorporating its ETL engine; no data has to be stored in SAP BW, but rather can point to SAP BW representations of extraction structures calledDataSources. Of course, there are performance implications with querying structures that ETL the data on-the-fly each time it is referenced.Specialist BI suites and Niche BI vendors like MetaMatrix have tools that are more established in this domain for more robust needs.

Data quality and cleansing is a prerequisite to effective data transformation, whether it be ETL or EII. Imagine how much harder conversingin Chinese would be if several rustic dialects and non-native-speaker versions were on the other end of the phone rather than one clearStandard Mandarin voice.

Simply standardizing and automating processes on SAP Solutions does noteradicate data defects or the need for data quality management. In fact, SAPNetWeaver MDM, designed for master data consolidation, cleansing, andquality control, would not exist if that were the case. Here again, Specialist BIcan complement SAP capabilities, especially in the area of customer dataintegration (CDI). Furthermore, Specialist BI suites can provide solutions for thoseSAP customers that have not yet taken the architectural plunge in to SAP NetWeaver MDM, but still need data quality and cleansing capabilities.

Although SAP BW has opened its architecture from external connectivity perspective, there still remains a significant value gap between SAPand non-SAP sourced data unless Specialist BI or Niche BI is leveraged. This gap exists both because of the level of programming effort neededin its native ETL engine, and also because of the availability of SAP’s pre-delivered information models. The SAP information models, calledBusiness Content, are predictably slanted towards SAP applications. SAP BW projects tend to be SAP-centric not because of a lack of SAP BWopenness, but because of the lack of automation and standardization of non-SAP scenarios. Although SAP BW does have some Business Contentfor external data sources, including Dun and Bradstreet and Oracle Financials, there is not enough pre-delivered content and native ETLcapabilities to make the same business case for accelerated implementations on non-SAP data as there is for SAP data.

Due to their “Switzerland neutrality” in BI, Specialist BI and Niche BI vendors can bring much more pre-delivered content and connectorsfor diverse enterprise applications external to SAP such as Salesforce.com, Oracle and pre-Oracle applications (i.e. JD Edwards, Siebel,Hyperion, Peoplesoft, et al) as well as SAP itself. Furthermore, on-demand BI has pushed the envelope. Pre-packaged reporting andanalysis on external market data or public government records for market research, competitive analysis, and forecasting or industrybenchmarking are now available as subscription-based services. Diverse sources such as eBay market data, Bureau of Labor Statistics,Bureau of Economic Analysis, Dun & Bradstreet, Hoovers, and Thomson Financial can be readily accessed and used in a web-based BItool from an online store.



Here again, Specialist BI can complement SAPcapabilities, especially in the area of customerdata integration (CDI).

While SAP does have partnerships with syndicated data sources and doesstand to gain by supporting other enterprise applications, it does not seemSAP will be expending much effort delivering Business Content for thesesources, and vice versa. Specialist BI and Niche BI vendors, on the otherhand, have a much more vested interest. For organizations with a small SAPfootprint, the multi-vendor pre-delivered content that Specialist BI and NicheBI vendors bring may be more relevant. Specialist BI and Niche BI vendors donot cover all business applications or external sources, and those they docover are done at varying degrees, especially in regards to SAP data, socustomers must account for this in their evaluation.

In SAP BW, all new data sources must flow through its ETL engine for integration, and subsequently must be information-modeled intoInfoProviders, in order to be available for SAP BEx. As discussed in Appendix 7.3, in SAP this information design is done with the DWWorkbench tool as a developer task. An alternative is to use SAP NetWeaver Visual Composer, which leverages the same underlyingarchitectural components as UD Connect to connect to any SAP and non-SAP system, as well as ABAP via RFC and web services.

SAP NetWeaver Visual Composer is touted as a business process expert tool. However, it is not yet a candidate for deployment as a self-servicereporting tool for business users. SAP NetWeaver Visual Composer has not yet demonstrated widespread adoption or acceptance by end-usercommunities commensurate to Specialist BI tools. Specialist BI tools have the benefit of an established history and rapport with businesscommunities (end-users and experts).

Both Specialist BI products and SAP NetWeaver Visual Composer can connect heterogeneous data sets in their tools. What this effectively meansis that users can directly access outside data sources (or raw exports) and integrate them into their reports and analysis without having to waitfor IT specialists to build the information model for them.

However, as explained in Appendix 7.3, SAP NetWeaver Visual Composer is not as strong an information-modeling tool. In Specialist BI andNiche BI products, users have the ability to do their own information design and integration work. In contrast, SAP BW takes a structured andarchitected approach to BI that most IT organizations would commend for governance and control reasons. However, ad-hoc self-service analysisshould not be discounted, since inexorable business demands and specialized needs are creating mounting IT backlogs.

Specialist BI tools provide a quick alternative solution to access outside data sources without the need for a data warehouse or even data mart(SAP or non-SAP). For SAP shops, this enables the bridging of data gaps caused by data sources deemed too trivial or difficult to cost-justify thebuild in SAP BW. For non-SAP shops, this enables reporting on SAP data (whether in an SAP transactional system or in SAP BW) without loadingthe data into another environment. Furthermore, many SAP satellites need more immediate visibility to, and integrated reconciliation with, SAPdata before they can effectively replace or stabilize interfaces that may break down after SAP retires legacy systems.

From an EA and roadmap perspective, although SAP BW may deliver more flexible ad-hoc data integration options, it is better to focus EAefforts on present-day realities; the IT industry is so volatile that prediction even one year out is difficult. EA should be focused on balancingstrategy and stability through continuous transition-state architectures rather than attempting a multi-year leap from the “as-is” current state to the“to-be” end state. As many environments already have both SAP and Specialist BI suites in their portfolio, it is a matter of rationalizing thoseassets but then also finding ways to create synergies with those stabilized investments.

The main challenge is not technological, but political and organizational, as IT and even business fiefdoms will clash around vendor-aligneddogma. Ironically, organizations pushed BI vendors hard to interoperate with each other, and now that SAP integration has reached maturity,it is the organizations themselves that are more closed than the technologies.



Due to their “Switzerland neutrality” in BI,Specialist BI and Niche BI vendors can bring muchmore pre-delivered content and connectors fordiverse enterprise applications external to SAPsuch as Salesforce.com, Oracle and pre-Oracleapplications (i.e. JD Edwards, Siebel, Hyperion,Peoplesoft, et al) as well as SAP itself.

3. WHY CONSIDER HYBRIDS?The world of IT has its share of industry-specific dichotomies and trade-offs. But over time, innovations and improvements born of industry debateand market needs give rise to new alternatives that do not necessarily require compromise. Both SAP and Specialist BI suites are now touting“best-of-both-worlds” solutions with lower overall TCO by leveraging maturing interfaces, the benefits of better architectural design, enablingtechnologies and acquisitions. Furthermore, established BI providers have rapidly grown into a comprehensive suite of BI tools and applicationsthat are as much “one-stop-shop” as they are “best-of-breed” in BI.

The promise of “best-of-both-worlds” offerings is taking the place of “best-of-breed” versus “one-stop-shop” trade-offs. The premise of this white paper is thatyou do not have to pick vendor sides and then wait for your vendor to deliver“best-of-both-worlds” capabilities. Instead, you can start to pragmatically “spreadyour bets” with a composite approach capitalizing on well-established interfacesor extraction techniques.

In fact, BI vendor interoperability is exactly what the majority of the market has been clamoring for. An Information Week survey in March 2007noted that 78% of respondents indicated that what they wanted from BI vendors was the “ability to integrate with existing applications,” puttingthat desire at the top of the survey wish list.5

The ability to customize and extend vendor solutions further obscures the “make” versus “buy” dichotomy, embodied in the difference between buildingsolutions out of “best-of-breed” versus buying a “one-stop-shop” solution, respectively. The power of “out-of-the-box” capabilities in an application suitecan be combined with the development and self-service capabilities of a platform or tool. This enables organizations to build malleable designs andarchitecture on an enterprise scale that is unique to their needs as well as enable “prosumer” (consumer as producer) activities.6

As vendors make acquisitions, embrace openness and create separate licensingagreements, hybrid architectures have more potential for synergy thancannibalism. For enterprise architects, the new challenge is to determine theoptimal hybrid architecture. Hybrid architectures historically have posedproblems, since proprietary architectures lock out other vendors, creating defacto isolation. As vendors adopt open standards and existing interfaces mature,

hybrid landscapes can become more of a strategy than a legacy. Considering the large data volumes and heavy infrastructure investmentsalready made in enterprise BI solutions, putting these investments together instead of throwing some away makes good sense. The increase inacquisitions, partnerships, semi-packaged applications and composite solutions are testaments to the value of the hybrid approach.

Perhaps a different set of dichotomies in lieu of the traditional “best-of-breed” versus “one-stop-shop” debate is necessary. The traditionaldichotomy has been an architectural debate that is vendor aligned and, as a result, polarizes approaches. The traditional dichotomy suggeststhat one vendor group precludes the use of the other instead of considering both. The perception is that organizations must force-fit either SAPBW or a competing BI product rather than finding the right balance.

Given the recent trends in BI, perhaps alternate dichotomies are necessary to assess overall best-of-hybrid architectural approaches thatholistically assess depth-versus-breadth capabilities and open-versus-proprietary features. “To SAP or not to SAP” is no longer a clear-cutquestion, and indeed, is now a bit myopic. Instead, a bigger-picture question is how to blend different BI product capabilities in order tomaximize the enterprise value of information assets when SAP is in the mix.

3.1 “ONE-STOP-SHOP” MISNOMERA decision to standardize on SAP BW for all reporting, query and analysis might be flawed without consideration of all user requirements, presentand future. Very often an organization will wait for transactional systems to be standardized before rolling out BI. Consequently, the organization is



The premise of this white paper is that you donot have to pick vendor sides and then waitfor your vendor to deliver “best-of-both-worlds” capabilities.

As vendors adopt open standards and existinginterfaces mature, hybrid landscapes canbecome more of a strategy than a legacy.

quite immature with respect to BI, and thus its understanding of how user requirements may develop is very limited. It makes sense to investigatewhat analysis organizations similar to your own perform, how long it took for them to get there, and what technology they finally selected.

Many large-footprint SAP BW implementations could benefit from enhancing their solutions with Specialist BI capabilities, but given theirmandates, budget and program scope simply do not look beyond the SAP walls.

Even in wall-to-wall SAP implementations, typically there is still BI demand for some non-SAP data sources that are deemed too small (such asa spreadsheet), too interim (like a retiring legacy system), or too unwieldy (like syndicated data) to be brought into the SAP BW solution. Inaddition, there may be BI demand from satellite consumers that want to bring SAP access into their local solution, but that demand gets eitherignored or addressed with a one-size-fits-all data dump.

In many of these “exception” cases, under-served or ignored groups find local or un-standardized workarounds that undermine the spirit of EAand the move to become an SAP shop. It is not uncommon behavior in large-footprint SAP BW implementations to find users routinely exportingdetailed data from reports that were supposed to serve users’ needs in and of themselves. By taking a “one-stop-shop” approach with SAP BW,organizations may inadvertently create an SAP fiefdom that is surrounded by satellites with the autonomy to choose their own BI tools but withoutthe benefits of centralized truth, support and visibility.

Rather than letting hidden costs proliferate, SAP shops should considerexpanding their architectural vision to address unmet outside needs byleveraging Specialist BI tools. Approaches for doing so will be discussed laterin this document. Not only can Specialist BI tools reach larger audiences, butcore constituents can also benefit from broadened and deepened BIcapabilities. Otherwise, an ambitious SAP EDW initiative may atrophy into anSAP data mart. The need for fast and flexible non-SAP data integration should not be underestimated.

Besides providing easier access to non-SAP data, Specialist BI solutions can also potentially better fulfill implementation-specific requirements,such as more dynamic or functional semi-additive measures, pixel-based formatting, specialized interactive visualization, advanced publishingand distribution options or self-service usability, while at the same time integrating with SAP outputs in SAP NetWeaver Portal or on the web.

In broader EA terms, no one vendor can cover all enterprise needs. In the early days of SAP, this notion was lost on some SAP shops (andperhaps still is). When SAP ERP first hit the market, there were few software implementations quite like it in magnitude. It was so monolithic andcovered so much business scope that organizations assumed they had gotten their EA in one vendor box.

But as customer IT investments continued to grow and landscapes became more complex, the belief that SAP ERP was an all-encompassing island shiftedwith the advent of other monolithic enterprise applications such as SAP CRM and SAP SRM. EA-aware advocates realized ERP investments were likepoured concrete, and SAP eventually responded with what evolved into the SAP NetWeaver BPP to address interoperability and flexibility concerns.

Multi-vendor landscapes and hybrid architectures are a present and future reality. Asa result, customers must realize that they own their EA; not the vendors. Rather thanfactionalize or universalize SAP architectures and programs, IT organizations shouldrealize that the ever-growing scope and complexity of their EA and the need forbusiness autonomy necessitate a multi-product and customization strategy. Even SAPhas implicitly recognized and acknowledged the need for hybrid solutions byfostering a healthy partner ecosystem around its core solutions. The SAP and

Microsoft joint venture product called Duet is one such manifestation and further blurs the vendor lines to promote hybrid architectures. Furthermore,mergers and acquisitions as well as trends towards business networks and cross-enterprise collaboration virtually guarantee that “one-stop-shops” willeventually be unattainable. EA disciplines around managing and controlling hybrid architectures are a growing, not shrinking, need.



SAP shops should consider expanding theirarchitectural vision to address unmet outsideneeds by leveraging Specialist BI tools.

IT organizations should realize that the ever-growing scope and complexity of their EA andthe need for business autonomy necessitate amulti-product and customization strategy.

3.2 “BEST-OF-BREED” MISNOMERSpecialist BI Suites that were traditionally dubbed as “best-of-breed” are increasingly harder to distinguish from “one-stop-shop” brandedenterprise application vendors, at least from a BI perspective. For example, Specialist BI offerings in Enterprise Performance Management (EPM)are as comprehensive and integrated, if not more so, than SAP solution offerings in that category.

As a result, alternate methods to differentiate and complement solution providers are needed. Two criteria for comparison are the breadth anddepth of functionality and features in contrast to the flexibility and openness of the architecture on which it is built (i.e. business applicationversus infrastructure platform).

Many former “best-of-breed” functions and features have since been commoditized and hence, no longer qualify as “best-of-breed.” “Slice-and-dice,” ranking lists, exception alerts and web-based reporting and analysis are now non-differentiating functions. In contrast, having a single,integrated and consistent BI front-end for all BI needs is now a differentiator for Specialist BI providers to address.

Furthermore, Niche BI vendors with true “best-of-breed” capabilities are not viable over the long-term unless they can interoperate as part of abest-of-hybrid solution. In other words, Niche BI vendors with proprietary and closed architectures are increasingly hard to justify asorganizations move towards enterprise solutions. Isolated “best-of-breed” solutions can persist in fragmented and departmentalizedorganizations, but as those walls come down, so will those products because of the EA headaches they produce. Many times, there are moreopen and competing alternatives that are better cost-justified. For Niche BI vendors to add long-term value to their short-term business value,they need to invest in interoperability in order to function in best-of-hybrid environments.

4. APPROACHES FOR SAP INTEGRATIONSpecialist BI providers have three fundamental ways to achieve hybrid information integration with SAP:

• Presentation output integration via a Portal.

• Data quality integration via master data maintenance.

• Data access and information design integration.

Both SAP and Specialist BI providers have the ability to integrate into SAP NetWeaver Portal via iViews to achieve integration to a web-basedcentral access point. Reporting and analysis output is simply published through an iView. Not only can SAP and Specialist BI iViews be arrangedtogether on the same portal, but SAP drag-and-relate and portal-eventing technology also enables data integration and interactivity betweeniViews. Standard portal capabilities enable roles-based composite applications that leverage both SAP and Specialist BI output.

At the other end of the information value chain, Specialist BI data quality capabilities can directly embed into SAP transactional systems forspecialist features such as postal validation and duplicate matching for maintenance by business partners such as customers and vendors. Nomatter the size of your SAP footprint and whether you are using SAP NetWeaver MDM, Specialist BI data quality management capabilities canoffer functions and features that native SAP capabilities cannot.

Perhaps the most controversial BI integration topic involves SAP data access because there are two different approaches with significant ITstrategy and architectural implications. The two basic ways to access and integrate SAP data in question are:

• Reporting on the data directly from SAP via an interface

• Extracting the data out of SAP



Additionally, each data access approach has two options:

• Access data from SAP BW

• Access data from the SAP transactional systems

If you are a large SAP shop, you may keep data in SAP as your source of truthrather than have it extracted. As not all reporting needs are typically addressedby SAP BW (such as real-time reporting due to customization or performanceimpacts), Specialist BI tools add the most value by being able to merge SAP BWand SAP transactional data as well as serve as a consistent tool across both.Having a consistent tool not only lowers TCO as an enterprise standard, but alsosaves change management costs, especially when underlying systems are goingthrough upgrades or migrations on different release schedules. All back-end change volatility and complexity can then be shielded from the user.Alternatively, Specialist BI tools may be more appropriate for certain user groups that deem the SAP BW user interface less than optimal. Thedirect access methods have few technical variations, while the positioning of front-end tools has far greater variability.

If you have SAP in your shop, then you are likely to have your “source of truth” in another system, and may want to extract data from SAP. Akey architectural decision to make is whether to extract from the SAP BW or from the SAP transactional systems. Theoretically, it makes moresense to extract from the SAP transactional systems, but pragmatically most organizations use SAP BW as a proxy to the transactional systemsin order to leverage all the pre-delivered Business Content extractors. However, the latter option has a licensing impact. (The impacts of eacharchitectural approach are discussed in more detail later.) For ERP real-time reporting needs, the data access approach needs to directly readsome type of interface instead of replicating data via extraction. Whether you consider your environment an SAP or non-SAP shop, the dataaccess approach for ERP reporting should be handled the same.

4.1 DATA QUALITY MANAGEMENT IN SAPEven if a business had only a single instance of SAP ERP, owning no other applications outside of it, data defects can still proliferate. Data entryis prone to human errors, and is perhaps the most common source of bad data in the form of non-conforming conventions, typographical errors,duplicates, obsolete entries and missing details. The likelihood of process breakdowns is high due to the cross-departmental nature of ERP masterdata maintenance. For example, both finance and sales share the customer master, which may be updated at different times for different reasonsthat can in turn lead to duplication. Material master records are even more enterprise-crosscutting; with finance, sales, procurement,manufacturing and materials management all possessing their own views of these records. Even if well-regimented master data workflows aredefined, status and tracking is not a failsafe against data entry errors.

In reality, most environments have more than one SAP ERP system running, along with many other applications sharing similar master data.Even an enterprise CRM system has a separate customer database than an ERP, irrespective if both are SAP. Furthermore, many largerenvironments may have multiple instances of SAP as well as other vendor packages, all on different versions and configurations. Suchheterogeneous environments create additional data consistency issues due to differences in formats, structures and versions of truth.

Add spider-web interfaces and massive data conversions (where sometimes validations are turned off for performance reasons) to the scenarioand the need for data quality control rigor mounts. Poor quality in customer data alone costs corporate America $611 billion a year.7

Although data quality management solutions typically support data profiling, mass cleansing and quality monitoring capabilities on largeexported data sets, best practices in data quality are the same as in manufacturing: scrap and rework costs more in the long run than addressingproblems at their source. The basic manufacturing-based quality concepts and tenets are the same: preventing problems upstream savesinspection and repair downstream.



Specialist BI tools add the most value by beingable to merge SAP BW and SAP transactionaldata as well as serve as a consistent toolacross both.

SAP NetWeaver MDM has validations and matching strategies to correct errors and eliminate duplicates, but as yet this is not where most SAPenvironments first enter their master data. In many SAP environments, master data maintenance is still done in a local SAP ERP system. At best,a master client designed for central maintenance distributes updates to other SAP clients via Application Link Enabling (ALE). However, in mostcases, master data maintenance is still distributed. As a result, these systems still need a central way to validate and quality-control data enteredinto those distributed systems.

For checking documents posted into the ERP-based financial modules, SAP has validation and substitution rules that can be applied via Booleanlogic or custom-programmed enhancements. But these business rules are system-defined (i.e. are local), as are other application-specific checksand validations SAP delivers in the form of Business Add-Ins (BADIs).

In contrast, however, there exist two notable BADIs designed for a well-integrated set of functions for managing address data on auniversal basis, a capability which SAP calls Business Address Services (BAS). Almost anything in SAP that references an address isintegrated with BAS. A limited number of Specialist BI and Niche BI products with CDI capabilities offer direct SAP integration throughthe following two BAS BADIs in question:

• Postal Validation (Online and in Batch)

• Duplicate Check, Error Tolerant Search Interface

Before SAP-based applications, some early data quality applications centered on names and addresses stored in mainframes for marketingcampaign mailings. These applications realized significant savings by cross-referencing postal information that government agencies madeavailable such as NCOALink (National Change Of Address). Postal standardization of names and addresses has probably more obviousreference for a single source of truth, and it can be applied in other countries as well. Business Objects (formerly First Logic) provides globalpostal validation coverage to over 190 different countries.

There are other standards for other master data such as UPC (Universal Product Code), EAN (European Article Number) and GTIN (GlobalTrade Item Number) for products. Some industries also maintain data pools for product master data, such as 1SYNC for global datasynchronization (GDS) in retail using globally referenceable GTIN. SAP NetWeaver MDM provides direct support for GDS and 1SYNC.

Having access to up-to-date postal data not only supports validation (catching address changes, omissions or errors) but de-duplicationand error-tolerant searches as well. For example, Business Objects (formerly First Logic) performs fuzzy or phonetic logic search thatprompts data entry clerks with alternate matches if duplicates or errors are found even if entered data is ambiguous. The same logic canbe used for error-tolerant search.

This direct integration with SAP master data maintenance enables bad dataprevention at the source so that no further scrubbing is necessary after extraction.Whether you choose to leverage SAP BW or not, your SAP solution stands tobenefit from postal validation, duplicate check and error tolerant search thatcomplementary solutions can provide using government agency databases.

4.2 INTERFACE-BASED ACCESS APPROACHESIf you are an SAP shop, SAP BW may be the best candidate for housing all the data for your reporting and analysis needs as it relieves thedata and performance strain these activities have on your SAP transactional systems.



Whether you chose to leverage SAP BW or not,your SAP solution stands to benefit from postalvalidation, duplicate check and error tolerantsearch that complementary solutions canprovide using government agency databases.

From a data management perspective, SAP BW generally has better capabilities than the information systems found in the SAP transactionalsystems for data design and integration. Some examples:

• Data realignments in ERP modules such as CO-PA were either system intensive or unavailable. In SAP BW, data design concepts such as time-dependent master data (i.e. changes tracked outside of the transactions) and navigational attributes (i.e. joining master data with transactiondata in reports) reduce the impact of changes in reporting structures. In addition, these data design concepts provide more functionality forhistorical versus any-point-in-time versions of truth.

• There are limited options to improve reporting performance in SAP Business Suite applications. Pre-calculation, query cache, databasepartitioning, logical partitioning and aggregates are all SAP BW standard functions not generally available in the SAP Business Suite.More notably, in-memory indexing technology, called BI Accelerator and powered by partner hardware appliances, offers improvedperformance on SAP BW.

• Keeping transactional history in SAP BW longer is easier with near-line storage capabilities through partnerships like SAND technologies.

• SAP BW has ETL capabilities, whereas SAP transactional systems do not. However, in some cases the configuration-based data integrationand validation capabilities in ERP modules such as CO-PA and the new general ledger are stronger. However, these modules are designedfor local SAP data, while the ETL capabilities of SAP BW can be more generally applied across SAP and non-SAP data.

In addition, SAP BW has several techniques for fresher data: delta-change captures (Business Content pre-delivers application-specific andsophisticated mechanisms), trickle feeds (called real-time data acquisition or RDA) and federated data access (Remote Cubes). Remote Cubesactually diminish the SAP BW value propositions for offloading data and performance and, as a result, should be restricted to uses such as tie-back reconciliation.

From reporting, query and analysis perspectives, SAP BW allows you to leverage Business Content, BI analysis authorizations and drill-back totransactional systems as well as its native OLAP capabilities.

The standard way SAP BW handles all data requests is to pass them through the SAP proprietary OLAP processor, whether it is SAP or third-party. Put differently, SAP BW data access is SAP BEx query-centric. SAP refers to the set of APIs that access their OLAP processor through SAPBEx queries, as Open Analysis Interfaces. From an overall modeling perspective, if Specialist BI tools are being used to push down informationdesign (data unions and joins), query design (information subsets, OLAP definitions and calculations) and presentation design (formatting andlayout) then the question of how to design a BEx query for external access becomes a strategic design decision. Using Specialist BI tools as asimple presentation design tool on top of query design limits the value proposition of the solution. On the other hand, uncontrolled accesspotentially undermines your SAP BW system instead of augmenting it. Appendix 7.4 reviews some best practice query design principles to makethe most of the hybrid design.

4.2.1 OPEN ANALYSIS INTERFACES DATA ACCESSAs noted earlier, almost all direct access to SAP BW data passes through the OLAP processor based off of SAP BEx queries. The Open AnalysisInterfaces are designed to allow complementary BI solutions direct access to the OLAP engine. These interfaces are based on a long-establishedand proven Microsoft standard on which many BI companies and products are built. SAP provides practically full coverage of the standard,and yet some customers remain skeptical of what Specialist BI or Niche BI providers can yield from Open Analysis Interfaces—not, it must benoted, without reason (specific details are covered at length in Appendix 7.5).

However, the first Open Analysis Interface has been around almost as long as the Microsoft standard and has reached full maturity insome BI products support for the interface. The current situation has improved significantly from the early adopters’ experiences, as solutionproviders that committed to SAP BW integration have ironed out the issues and closed the gaps. Nevertheless, not all solution providersare created equal in this respect. Established solution providers that have iterated multiple generations of certified SAP offerings and that have



a long working histories with SAP customers are in a much better position tofully capitalize on Open Analysis Interfaces than vendors new to SAP.

Specialist BI firms with an established history of integration with SAP BWwill have worked out the kinks, while those vendors new to SAP will haveto lay out the development dollars and commitment to work through theseoften-obscure issues. For SAP shops wanting to keep the data in SAP BW,establishing integration with SAP BW as one of the first criteria in avendor selection will quickly create a short list.

4.2.2. OPERATIONS-BASED DATA ACCESSSAP BW was first originally designed for analytics on historical and summarized data rather than detailed operational data. As a result, thebuild of the environment was more architected and OLAP-driven than reporting-driven. To deliver enterprise reporting, SAP partnered withCrystal Decisions (now Business Objects) to deliver form-based formatted reporting capabilities for SAP BW. Joint development integratedCrystal Reports directly into the SAP BW metadata repository and against the native OLAP processor (for better performance and flatteningoperations instead of an Open Analysis Interface). Later, SAP recently introduced the SAP BEx Report Designer for formatted reporting in thelatest version. However, SAP BEx Report Designer only works against SAP BW data versus SAP Business Suite data, while Crystal Reports notonly works against both but can integrate non-SAP data as well.

In fact, Crystal Decisions was a popular enterprise reporting tool against SAP ERP data well before the partnership deal for SAP BW integration.Crystal Reports shows its maturity and stabilization after seven generations with SAP integration, pixel-based visual control, expert modes andaccelerators (such as wizards and templates) that in the eyes of users raise the bar high for other enterprise reporting tools.

In order for Specialist BI or Niche BI products to read SAP Business Suite data, they must interoperate with ABAP via an RFC connection.However, establishing the technical connection to SAP environments is only the tip of the iceberg. Understanding where to go for the data andhow to access it is a far more challenging endeavor that only vendors that have an established history can credibly deliver.

Appendix 7.6 covers these technical and semantic complexities with ABAP-based data access in more detail.

4.3 EXTRACTION-BASED DATA ACCESS APPROACHESIf SAP solutions are in your shop, then you are probably faced with one of the following two perspectives:

• You cannot invest in SAP BW for strategy or budget reasons.

• You have invested in SAP BW, but now must figure out how to position it vis-à-vis your other BI solutions.

In either case, the fundamental question is, what is the best way to extract data from SAP Solutions? More specifically, do you pull from the SAPBusiness Suite directly or use SAP BW, either as a pass-through or part of a federated data warehouse?

Those enterprises that are looking for SAP BW alternatives, either because of budget constraints or their level of investment in non-SAP BW dataarchitectures, can skip to the next section on data extraction considerations.

Those enterprises that are looking to incorporate SAP BW into their existing landscape must understand that the SAP BW metadata management,design practices and tools are fundamentally different than traditional BI. In such environments, traditional BI architects and data modelers may



Established solution providers that have iteratedmultiple generations of certified SAP offeringsand that have a long working histories with SAPcustomers are in a much better position to fullycapitalize on Open Analysis Interfaces thanvendors new to SAP.

find SAP has a steep learning curve. Data modelers may be challenged to quickly understand SAP Business Suite's sheer volume of tables andin-depth data logic. It may also take users some time to understand the SAP BW system’s data repository, and the graphical renderingconventions and the tools.

Appendix 7.7 goes into more detail about the paradigm differences in order to explain why attempts to incorporate SAP BW into non-SAP BWlandscapes are often met with ideological resistance. It is inherently rooted in how the EDW is modeled and how BI metadata is managed.

In addition, perhaps one of the most hotly-contested capabilities of SAP BW is its scalability for very large data warehouses. SAP credibility asan EDW is growing as SAP BW multi-terabyte testimonies increase. IBM recently issued a rigorous proof-of-concept study that endorsed SAPBW as a solution for managing 20 terabytes or greater.8 Additionally, for large volumes, some of the same Niche BI vendors that provide non-SAP near-line storage capabilities can be leveraged for SAP BW environments.

However, for some environments with retail-based financial transactions, point-of-sale records or RFID tracking, the sizing needs will push wellbeyond 100 terabytes. For those sizes, there are no references or empirical evidence that SAP BW can effectively scale. But for hybrid federateddata warehousing strategies, does SAP need to? If SAP BW is just used for SAP data, is 20 terabytes enough? In most cases, the answer is“yes,” which then drives the question of how central sources of truth are ensured and how to best architect SAP BW. Is SAP BW a simple pass-through proxy for SAP transactions? Should the environment also handle multi-layered staging and incorporate a reporting layer? Should EDWtransactional history be federated?

These paradigms and shades of gray can spur philosophical BI debates that create cultural barriers around technological differences. The goodnews is that Specialist BI suites can stitch together the results from both environments. The bad news is that many organizations are not preparedfor that from a cultural perspective.

4.3.1 SAP BUSINESS SUITE DATA ACCESSBefore deciding on where to extract data from SAP Solutions, due diligence should be performed to determine what is involved with extractionfrom the various sources, what SAP BW capabilities are provided to facilitate extraction and what percentage of extracted data is non-SAPsourced. In the end, there are only two best-practice options for getting data out of SAP:

• From the SAP Business Suite applications directly.

• From data loaded into SAP BW.

This section will focus on the first method. The next section will focus on the second.

Some large Specialist BI suites provide SAP quick-start offerings (i.e. pre-packaged and even fixed-price content that includes consulting)that are positioned as low-cost data mart alternatives to implementing SAP BW. Of course, the content (i.e. pre-defined mappings andinformation models) in these bundles can also be leveraged to bridge SAP into larger-scale data warehouses as well. The content thatSpecialist BI suites offer usually covers all of the core modules in SAP ERP. The value proposition is a savings in implementation time byleveraging a ready-to-run template. The added advantage in using a Specialist BI suites is that the products will have similar templates forother business suite vendors that SAP Business Content will not cover. In addition, Specialist BI suite leverage their integrated capabilities, suchas a data lineage from queries or report output, back to the raw SAP source that does not have an equivalent in SAP Business Content. Havingpre-delivered content at your disposal is important in order to eliminate data complexities, but even SAP Business Content cannot cover allscenarios and requirements. As a result, in some cases the data challenges cannot be avoided, and therefore must be addressed.



Extracting data from SAP transactional systems faces some of the same challenges as reporting directly from those systems (i.e. knowing whereto go for the data and how to transform it into something meaningful) with the additional technical challenges of streaming the data and handlingdelta change capture (i.e. the incremental difference from the last data pull). Both are important for data load performance. Each ETL solutionprovider (or Specialist BI suite with ETL capabilities), as well as SAP BW, has a mechanism for handling data streams; handling delta changecapture is much more data source-driven and hence, SAP-centric topic.

The details behind SAP BW delta change capture techniques for extraction are covered in Appendix 7.8. It is intended for extractiondevelopers that need to understand how SAP BW extraction works for incremental data pulls in order to optimize the performance of theirown development work.

For implementations whose requirements can be met by a Specialist BI quick start offering, the details covered in Appendix 7.8 are probablymore than you need to know. However, the larger and more specialized (i.e. non-core modules) the SAP footprint, the more likely that thesedetails will be relevant to an architect and extraction developer, and also make SAP BW Business Content more attractive.

4.3.2 SAP BW DATA ACCESSMost of the value of Business Content lies in the pre-delivered extractors and metadata (atomic data elements or fields called InfoObjects). Thedownstream information models (such as InfoProviders and queries) serve more as templates for reference and reconciliation. Most customersend up building their own downstream information models from scratch while leveraging the existing InfoObjects and extractors.

As explained in Appendix 7.8, all of the SAP extractors leverage a core interface called the BI service API, which gives Business Contentextractors privileged access to the delta queue. In turn, the delta queue has widespread coverage that helps support faster refreshes of data.Any external access to the BI service API is not SAP-supported. Nevertheless, some organizations found the Business Content extractorscompelling enough to warrant customizing them in order to feed third-party ETL tools. These isolated approaches have yet to meet withwidespread success, and are, of course, discouraged by SAP.

Instead, most organizations turn to SAP BW and Business Content to get data out of SAP transactional systems and use the SAP-supported OpenHub Services to distribute data from SAP BW. Open Hub Services export data from SAP BW InfoProviders to file or table which third-party toolscan then read from. Appendix 7.9 explains in more detail.

For SAP BW inbound considerations, where there are requests to pull data out of a legacy EDW in order to integrate the data with SAP data,there are alternatives to consider beyond data replication. For those environments concerned about data duplication and establishing a universalsource of truth in a separate environment, SAP has EII or federation capabilities. Master data can point to DataSources directly or ABAP codethat reads an outside data source instead of a native SAP BW-generated ABAP table. Similarly, transaction data can be loaded into a RemoteCube via a DataSource or ABAP code instead of an SAP BW-generated extended star schema.

5. EVALUATING RETURN ON INVESTMENTWhen evaluating the cost of building new capabilities with either a hybrid approach or a single product approach, the following generalformulas compare the two alternatives (making the sweeping assumption that the benefits can be realized by both approaches within the sametimeframe for comparability purposes):

• Single Product Approach = Cost of Implementing Product Capability + Enhancement Costs + Operating Costs.

• Hybrid Approach = Cost of Implementing Multiple Products’ Capabilities + Cost of Integrating Products + Operating Costs.

In other words, is it cheaper to extend a single product design by filling the gaps with enhancements, or is it cheaper to cover the gaps withanother application and integrate it?



The cost of implementing a product capability is comprised of tangible expenses such as hardware, software and labor, and intangibleexpenses, such as business opportunity costs and burned-out resources. Time-to-value is perhaps the largest hidden cost, though difficult tomeasure and interrelated with the expected business benefits. Labor is probably the most expensive tangible cost, especially in SAP BWimplementations, and gets much more expensive if customization is necessary. Similarly, with Specialist BI suites offering on-demand (i.e.software-as-a-service or SaaS) alternatives, this is also the case on a much smaller scale and controlled basis; hardware and software costs arereplaced by ongoing subscription fees as part of operating costs, and labor is shifted to prosumers.

The enhancement cost in SAP BW usually means ABAP for gaps in OLAP capabilities, JavaScript for web-based formatting gaps, or Visual Basicfor workbook-based formatting gaps. Because of the SAP knowledge required for all three types of developments, the labor costs and time tofulfill requests can be extensive.

In contrast, many times there is an alternative solution that can directly address agap, but the limitation has been the cost of integrating with SAP BW. However,this cost has been effectively absorbed by SAP-committed solution providers, likeSpecialist BI suites, who provide comprehensive MDX and SAP support.Furthermore, Specialist BI suites that provide robust Open Analysis Interfaceintegration also provide broad and deep BI capabilities that can address mostgaps without going to another Specialist BI firm. In essence, when comprehensiveOpen Analysis Integration is covered and SAP BEx query performance can

sustain a requirement, then the only real cost to consider (over and above standard product-related skills) is the hybrid skills necessary fororchestrating the design and implementation. Because Specialist BI tools typically are designed for less-skilled developers and informationworkers, the availability of labor for standard product skills should be easier to secure than the hybrid skills necessary to pull it together. Similarto EA, the customer should develop and own these cross-product skills in their BI competency center rather than looking to market in order toreduce TCO, as few consultancies have these truly hybrid skills.

For ERP reporting not covered by BI solutions, the cost comparison for the single product approach is the cost of either developing in one of theSAP native reporting tools (such as SAP Query or Report Painter) or a custom ABAP report. If the reporting needs to be done in ABAP, then allthe complicated business logic and data design must be done regardless, and can just as easily be encapsulated in a RFC-enabled functionmodule and passed to a Specialist BI tool in a hybrid approach. After the function module, the incremental cost of integration is low as theSpecialist BI can already readily connect to RFC data; all that remains is some metadata mapping, layout development and formatting. If acustom ABAP report has requirements for numerous custom layout and formatting options that makes a better case to pass the work to anempowered user in a Specialist BI tool.

When SAP transactional applications need to be brought into a non-SAP BW environment, then the single-product approach in this case mayconsist of leveraging a Specialist BI suite to extract directly from SAP into their environment. If one of their pre-packaged offerings meets therequirements, then the costs are usually fixed and rapid to implement. However, broader-scoped implementations are likely to run into significantenhancement costs if Business Content extractors need to be reverse-engineered and duplicated.

In contrast, the hybrid approach would be the cost of implementing both SAP BW and Specialist BI capabilities leveraging the Business Contentextractors and Open Hub Services. In this case, the cost of integration would be low, since relatively less labor is required to prepare Business Contentand Open Hub Services. The only potential integration cost is if the third-party scheduler does not already support the new Open Hub Service APIs.In earlier releases, the Open Hub Services model is similar, except InfoProviders need to be loaded as well, and similar APIs would need to be custom-developed. Note that the non-trivial semantic integration costs are shared by both extraction options, and therefore are left out in this comparison.

TCO is also directly impacted when information consumers evolve into prosumers and disintermediate BI developers, a proposition that makeson-demand BI quite promising. A well-known quandary in the IT industry is that business is changing at a pace faster than IT can match. As a



Specialist BI suites that provide robust OpenAnalysis Interface integration also provide broadand deep BI capabilities that can address mostgaps without going to another Specialist BI firm.

result, backlogs of projects pile up and even worse, projects that run too long become obsolete or lose value. Time-to-value is not only importantto minimize business opportunity costs, but also for effective resource allocation and operational efficiencies.

Of course, return on investment analysis is more than just evaluating TCO; it also includes estimating business benefits. Evaluating businessbenefits is much more difficult to quantify or estimate than costs, especially if it is a question of usability or the value of more accessibleinformation. The International Organization for Standardization (ISO) defines usability as:

“The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in aspecified context of use.”9

Ironically, the difficulty in measuring such intangible value is one of the strongest underlying business drivers for BI. The growingimportance of knowledge workers in value creation activities is a generally-acknowledged trend but hard to empirically measure andengage. Yet there are visible trends and anecdotes for shift in work that is commanding higher-brain-function activities. Perhaps one of themore convincing scientific studies that demonstrates the shift from routine and manual tasks to non-routine cognitive tasks (analytic andinteractive) is depicted in Figure 2 below.10



Non-routine analytic, interactive and manual tasks represent non-repetitive quantitative and reasoning skills, communication and coordination skills,and ‘hands-on’ or dexterity-related skills, respectively. Routine cognitive and manual tasks are repetitive ‘thinking’ and ‘doing’ tasks, respectively.

Ever-expanding enterprise applications are continuously automating routine manual and even routine cognitive tasks (through operational BI), drivingworkers toward non-routine tasks to handle exceptions. Celebrated management guru Peter Drucker noted that amid the information revolution, workers

1960 1970 1980 1990 2000

40

42

44

46

48

50

52

54

56

58

60

62

38

Nonroutine analytic

Routine cognitive

Nonroutine interactive

Routine manual

Nonroutine manual

Mea

n T

ask

Input

in P

erce

ntile

s of

1960 T

ask

Dis

trib

ution

Figure 2 - Skill Content of Technological Change (Autor, Levy and Murane)11

are divided into service workers (performing routine cognitive tasks) and higher value-adding knowledge workers (performing non-routine analytic andnon-routine interactive tasks). For non-routine work, software cannot easily replace people, as in manual or routine work, but rather must support andenable them “to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” as in the ISO definition for usability.

In this information revolution context, empowering the knowledge worker is perhaps the most important driver for business value creation.

Hybrid architecture strategy forces the need for interoperable solutions. By achieving more open EA, IT environments will become more flexibleand agile in responding to business needs. By leveraging solutions that enable the IT specialist as well as empower the business user,organizations can then orchestrate their work in a more balanced and collaborative way for maximum business gain.

6. CONCLUSIONTwo fundamental ways Specialist BI and Niche BI can complement SAP capabilities are in the areas of end-user usability and data integration(especially non-SAP data and data quality management).

For those customers who are looking for complementary capabilities to their SAP BW and SAP ERP Reporting, it is important to remember thatnot all solution providers know how to effectively bolt-on or bolt-in to SAP, even if they are SAP NetWeaver-certified. There are differing levelsof breadth and depth in complementing and integrating with SAP that can make or break hybrid architectures. Only Specialist BI or Niche BIproviders with a detailed understanding of, long demonstrated history with, and high degree of investment commitment in SAP can effectivelymake hybrid solutions work. The integration touch points and considerations covered include:

• Data quality management – The best solutions address root causes of defects at the source: during manual data entry.

• SAP BW enterprise reporting, query and analysis – The level of SAP BEx support beyond SAP NetWeaver certification should be aggressively evaluated.

• ERP reporting – Deep knowledge of application-specific data access and technical complexities is essential. The ability to merge SAP BWdata with ERP Reporting is also an important Specialist BI value-add capability.

• SAP extractions – The same data complexity considerations as ERP Reporting but with additional factors related to Business Content evaluation.

• Open Hub Services – Instead of direct SAP extractions, leveraging Business Content and SAP BW as a proxy to the SAP transactional systems.

All these SAP data access techniques are maturing and are enabling the feasibility of hybrid architectures. Specialist BI and Niche BI solutionproviders that integrate with SAP create best-of-both-worlds value propositions that improve the value of their own solutions as well as SAP.

ABOUT THE AUTHORDavid Dixon is a Vice President at Inforte, a Business & Decision Company, and co-author of the book “Mastering the SAP Business InformationWarehouse.” He joined Inforte in March 2004 via the acquisition of COMPENDIT, an SAP BI consulting firm. David was a founding teammember of COMPENDIT. Prior to that, he was a Platinum Consultant with SAP. David has worked with SAP’s SEM development team onnumerous occasions in support of Business Consolidations and Business Planning and Simulation, and has also continuously collaborated withthe SAP NetWeaver RIG Americas. He started his career in 1995 as a Financials and Controlling (FI/CO) consultant with SAP, specializing inall of the SAP reporting and analysis applications and tools. David has extensive project experience in implementing complicated globalsolutions for Fortune 100 companies and has presented at various SAP and BI forums such as SAP TechEd, TDWI and ASUG.



ABOUT INFORTEInforte, a Business & Decision Company, helps its clients acquire, develop and retain profitable customers with a unique combination of strategy, BusinessIntelligence, and CRM consulting services. Our approach enables clients to improve their understanding of customer behavior; successfully apply this insight tocustomer interactions; and continually analyze and fine-tune their strategies and tactics. Founded in 1993, Inforte is headquartered in Chicago with offices inNorth America, Germany, India and the United Kingdom. For more information, call 800.340.0200 or visit www.inforte.com

7.2 SAP PLATFORM CONSIDERATIONSThe bane of SAP’s existence has been usability and openness.Although SAP has come a long way on both fronts, it still has yet todefinitively shed its reputation in BI. Perhaps the recent SAP move toacquire Business Objects will help wipe this stigma clean.

Understanding where and why SAP struggles, and its historical context,lends insight to how and when to complement its capabilities with othersolutions or approaches. With a fast-growing and evolving partnerecosystem on the SAP NetWeaver technical platform, understandinghow third-party solutions complement SAP is increasingly important.Whereas some third parties such as Specialist BI firms found a distinctniche in the early SAP market providing more open and graphicaltools, that differentiating gap is becoming less obvious.

Aside from cultural and competitive reasons, the fundamentaltechnological legacy that constrained openness and usability hasstrong roots related to a four-letter acronym: ABAP. ABAP standsfor Advanced Business Application Programming and started asa report generation programming language for SAP’smainframe-based solution known as SAP R/2.® ABAP thenevolved into a full- blown IDE when SAP migrated SAP R/2 to aclient-server based solution called SAP R/3.®

ABAP is an SAP-proprietary programming language, so sinceinception it has not been portable to any environment other than SAP.Ironically, this made ABAP even more “open” than most otherapplications, since all of SAP’s source code is exposed to anyone thatknows how to read ABAP; it just wasn’t as technically interoperable as



7. APPENDICES

7.1 SPECIALIST BI SELECTION CRITERIAThe table below can be used as a guideline to evaluate different Specialist BI providers as a BI solution for your SAP environment or as astandard for all enterprise information (SAP and non-SAP).

Criteria for Judgment Rationale

Number of SAP customers Experienced with numerous customers and different customer deployments.

SAP partnership history Any unique integration co-developed with SAP.

Breadth and depth of SAP Solution providers should provide complete solutions that cover information management (data integration, complementary capabilities data quality, data federation, data marts etc) through BI tools and performance management applications (planning, budgeting, consolidation, profitability analysis, compliance etc.) giving the customer the option to standardize on the BI Specialist suite.

Maturity of SAP solution Multiple iterations and generations of a solution improve robustness of offering.

Size of organization Alleviates vendor viability concerns.

SAP integration touch points Number of “Powered By SAP NetWeaver” certified designations. How solution leverages the SAP NetWeaver technical platform such as integration with SAP security model and ABAP.

SAP data warehouse and Native drivers that provide the ability to create reports directly off SAP BW SAP transitional systemstransactional reporting capabilities like SAP ERP.

Pre-delivered content and Third-party equivalent to SAP Business Content that helps accelerate projects with pre-packaged templates for SAP Solutions developments that address SAP-specific complexities or template offerings for SAP-specific scenarios.

it is today. This enabled programmers to dive into the ABAP IDE inorder to enhance and even modify—against recommendations—SAPapplications. But getting SAP to talk to outside languages required anSAP proprietary library and files to be installed that varied byplatform, if supported, as well as other technology restrictions.

ABAP started as a fourth-generation language (4GL) that was firstdeveloped in an era where many vendors were looking for ways toabstract and improve on COBOL (considered 3GL). Even today, anestimated three-fourths of all computer transactions are still inCOBOL, with an even greater footprint for financial transactions.For business applications, COBOL has substantial credibility andhistory, and correspondingly, so does ABAP.

When the author first started coding ABAP in SAP R/3 (about threeyears after SAP R/3 was officially launched in 1992), he found thesimilarities to COBOL striking, as did others who dubbed it“German COBOL.” Just as COBOL has evolved to become moreobject-oriented and interoperable, so has ABAP; but to give ABAPcredit, perhaps it is even more advanced than COBOL. However,just as very few developers would position COBOL for high-endgraphical user interface (GUI) developments, ABAP has suffered asimilar fate. Although SAP has ABAP-based graphical capabilitiesmuch stronger than anything that evolved for character-basedCOBOL, SAP has long recognized that it needed an alternativetechnology to build GUIs.

Well before the launch of the SAP NetWeaver offering, SAP hadplans to incorporate Java into its platform in order to exploit themarket-leveling qualities of Java. As early as December 1998, SAPrevealed to the market its intention to move towards Java in orderto address usability and openness issues. Almost a decade later, aJava stack embedded in SAP’s application server platform is anactive reality with support for the latest release (Java PlatformEnterprise Edition 5).

With Java-based capabilities, SAP has developed rich Internetapplications (RIA) such as its SAP NetWeaver Visual Composer toolthat can be modeled and deployed from the Web for buildingdashboards or analytical applications. SAP is also well poised totake advantage of the latest Web 2.0 innovations and leveragepartners with Java-based strengths such as their fully embeddedAdobe Macromedia Flex capabilities (via OEM).

In addition to browser-based user interfaces enabled by its Javastack, SAP has redesigned its desktop client leveraging Microsoft

technology and entitled it NetWeaver Business Client (NWBC).SAP has also partnered with Microsoft to create a joint-development offering called Duet, which integrates SAP intoMicrosoft Office. In addition, SAP has partnered with Adobe tocreate yet another user interface alternative called AdobeInteractive Forms. Furthermore, SAP NetWeaver Mobile is a wholeplatform offering around developing mobile device applications.

SAP is decoupling the user interface from the ABAP-driven back-end, thereby opening the business case for third-party front-endtools wider than even before. New value in SAP can be created bygiving it a new face but relying on its application logic backbonethrough enterprise services.

But usability is not just about the chosen programming language,nor is openness just about technical interoperability or portability;both are just low-level constraints or enablers. Intuitive design in thefront end and semantic integration in the back end are much higher-level and higher-impact factors.

7.3 SAP FRONT-END HISTORYThe author made an early career out of being an SAP financialreporting specialist due to the level of mastery needed to efficientlydevelop SAP R/3-based reports. The early application-specific (likethe Legal Consolidations Report Writer) and generic report writers(such as Report Writer and ABAP Query) shared a similar designparadigm: development was logic and form-driven rather thanvisual and layout-driven. Similar to the rest of SAP R/3, reportdevelopment was transaction-driven and consisted of form-based,screen-by-screen sequential steps.

Since report development was highly conceptual and configuration-driven, this high-level step-by-step description of the SAP ReportWriter conveys the gist of “developer-centric design.” The work wasbroken down into logical units:

• You first start with the report creation transaction where a form-based input screen prompts for a library name (which representsthe reporting structure) and the report name, which effectivelymakes the assignment.

• In the next form-based screen you are prompted for header-leveldetails such as report text and authorization group assignment(obviously nothing a user would do).



• On the next screen, you define row sections by entering texts,and then control their formatting by configuring indicators foroptions like zero handling, indentations, alignment justification,page breaks and borders.

• Then within each row section (in another drill-down screen) youenter row blocks, which are either a reference to hierarchies,formulas or metrics. Again, on each row you control formattingoptions by setting configuration switches. In either row section orrow block there are also “business intelligence” settings such assubtotaling logic, sorting, thresholds and traffic light colors.

• Then column blocks are defined on another screen, and are are,once again, either references to hierarchies, formulas or metricsbelonging to the library with associated configuration switchesfor formatting.

• Then on another screen, the general data selection criteria aredefined which act as filters or variables to the report.

• A final optional screen can be configured with printing optionssuch as headers, footers, title page and last page.

Up until this point, the developer must conceptualize what theoutput looks like until the report is actually generated and executed.What ensued for novice report writers was an iterative trial-and-error process until the output looked correct.

It was not long before Report Writer was displaced by ReportPainter, which was layout-driven and mostly defined on one screen.However, it was still an abstraction for the actual output, althoughmore visual. ABAP query development (presently known as InfoSetquery) has become even more graphical with a query painteroption and the ability to define table joins as in MS Access.However, these tools are still ABAP-based, and most practitionerswill agree that the SAP’s BEx front-end is far more advanced inusability for both the developers and user perspective. In fact, therehave been ERP implementations where users threatened to stop go-live unless given alternatives to the native SAP ERP reporting. Thisdifference in capabilities drives SAP Business Suite implementationsto SAP BW, which is perhaps a strategic outcome desired by SAP.This difference also gives a business cause to employ a SpecialistBI tool without the need for a data warehouse or even a data mart.

The design paradigm of logically separating design into differentstages has carried through to the SAP BW to some extent. There aredeveloper advantages to taking a methodical “building blocks”

approach (namely, abstraction and reuse), but this approachnormally confounds typical users that expect a more direct method.More explicitly, SAP BW logically separates information design(data unions and joins), query design (information subsets, OLAPdefinitions and calculations) and presentation design (formattingand layout) from each other.

The SAP BW information design is done by a back-end tool calledthe Data Warehouse (DW) Workbench. Data unions (to combinedata sets) and joins (to link data sets) are done by structures SAPcalls Multi-Providers and InfoSets, respectively. These structures(generically referred to as InfoProviders) are centrally defined andcreated by developer specialists.

In contrast, the query design is developed with a front-end toolcalled the SAP BEx Query Designer. SAP BEx Query Designer isconceptually similar to the Report Writer in terms of how it logicallyseparates the components of a query. However, instead of flippingthrough form-based screens, the general data selection (calledfilters), rows and columns are all presented in a layout logicallyseparated into separate tabs or panes. Furthermore, the standalonetool is in a much more user-friendly .NET-based environment. Theequivalent of the library (i.e. the InfoProvider) is also displayed andits data elements can be dragged and dropped into the differentpanes to subset the data and parameterized to control the output.

Although a conceptual tabular rendering is displayed duringdevelopment, the SAP BEx Query Designer does not have an actualpicture or preview of what the output will look like until the query isexecuted. The “look-and-feel” of the output also depends on the defaulttemplate or pattern, which applies styling, formatting and functionsavailable that are separately defined (another reusable abstraction).

In addition, there are optional standalone presentation design tools(such as the SAP BEx Web Application Designer for webapplications and SAP BEx Analyzer for MS Excel workbooks) thatuse graphical layout-based designer tools, where presentationitems can be dragged and dropped (like buttons, tabular datagrids, graphs, charts, and so forth). Each of these items isrepresented by an icon and a true picture of the output filled withdata is not seen until actual time of execution (e.g. the layout givesyou a general idea of where a radio button group or tabular gridmight drop into a cockpit, but you don’t see the number of radiobutton values that will display or the row and column structure untilthe cockpit is filled with data at runtime).



Many of the OLAP and formatting functions and features that canbe predefined at design-time in the SAP BEx Query Designer are alsoavailable to the user at runtime besides just drag-and-drop slicing-and-dicing. This led to a stand-alone tool called SAP BEx Web Analyzer.The SAP BEx Web Analyzer tool has many of the same runtimecapabilities as its developer-centric counterparts (i.e. SAP BEx QueryDesigner and to a lesser degree the SAP BEx Web ApplicationDesigner); the difference is that it does not logically separate designtime from runtime (i.e. they are the one and the same). “Development”or the navigation state is saved as a bookmark for reuse or reference.Furthermore, no pre-defined queries are necessary, as this can bedone ad-hoc via the web as well (following similar logicaldevelopment steps as in SAP BEx Query Designer).

However, the SAP BEx Web Analyzer was just introduced in the mostcurrent release of SAP BW, and is still far from reaching parity with theSAP BEx design-time tools. Similarly, SAP NetWeaver’s VisualComposer has not yet replaced the SAP BEx Suite in reportingcapabilities, nor is it yet clear that it will do so. Furthermore, back-endInfoProviders still need to be defined for all sources of SAP BW data,whether SAP NetWeaver Visual Composer, SAP BEx Web Analyzer,the SAP BEx Suite or a Specialist BI tool is used.

As the name suggests, SAP NetWeaver Visual Composer is agraphical and visual design tool for developing SAP NetWeaverPortal content. It is a relatively new, rapidly evolving product thatis clearly born of Java, suffering none of the graphical andopenness challenges of ABAP world but with the advantage ofbeing able to interoperate with ABAP through remote functioncalls (RFC). As a result, however, it requires the use of SAPNetWeaver Portal and is not available as a standalone tool fordesktop integrations or offline capabilities.

SAP NetWeaver Visual Composer is a much broader tool forreporting or even dashboard deployments, and has beenintegrated into the recent and more holistic SAP NetWeaverComposition Environment platform for Java-based developments.SAP NetWeaver Visual Composer is engineered for model-drivendevelopments that are perhaps more process design-centric (i.e.flow diagramming) than information–design-centric (i.e. entity-relationship diagramming). For example, it lacks informationdesign capabilities such as inner and left-outer joins, as well asnative OLAP capabilities, such as support for drilldowns andhierarchies. Nonetheless, SAP NetWeaver Visual Composer doeshave a graphical layout tab that gives a better visual clue of outputthen the SAP BEx designer tools. SAP NetWeaver Visual Composeralso makes the sharp distinction between design-time and run-time

activities, but both are done via the SAP NetWeaver Portal insteadof separate environments or standalone tools.

In contrast, established Specialist BI tools typically do not separateinformation design, query design and presentation design intostandalone tools or reusable development objects. Instead, SpecialistBI tools combine them all together in one report development or adhoc analysis experience in order to give end-users quicker time tovalue. Furthermore, the distinction between design time and runtimeare blurred through true WYSIWYG capabilities. Users or less-skilleddevelopers are empowered to do a lot more on their own through ahigh degree of graphical control and visualization, therebydisintermediating the developer specialists. Such autonomy and self-service capabilities do raise governance and reusability issues thatshould be addressed by holistic EA strategy (i.e. without the SAPblinders) and organizational design.

7.4 QUERY DESIGN FOR OPEN ANALYSIS INTERFACESDesign separate and controlled SAP BEx queries specifically forSpecialist BI or Niche BI access that are separate from BEx queriesused for SAP BW reporting, query and analysis.

There should be no more than one external access query per datastructure in SAP BW (i.e. InfoCubes or Multi-Providers fordimensional structures and DataStore objects or master data forrelational structures). The SAP BEx query then becomes the proxyfor the data structure. Whether you allow access to the EDW layeris a case-by-case strategic design decision. But generally, basingthese queries off of integrated data marts (InfoCubes or Multi-Providers) proves better for performance and saves the reportingand analysis users design work on which they can build.

The queries should have all the detailed fields needed for reportingand analysis defined in the free characteristics with optionalvariables (which makes them available but does not immediatelypull all data, and also makes them all filterable).

Global metrics, calculations or columns should be defined asrestricted key figures, calculated key figures or structures,respectively (these are defined in the query but makes them part ofinformation design for Specialist BI queries). Local metrics,calculations or columns can be pushed down to Specialist BI toolusers. These local definitions should be reviewed and governed toidentify designs that are better pulled back into the global designfor reuse and standardization.



7.5 OPEN ANALYSIS INTERFACESIn order to enable a SAP BEx query for external data access, it must be released by flagging an indicator for OLE DB for OLAP (ODBO)in the query properties. ODBO is a widely supported Microsoft standard that extends its Object Linking and Embedding Database (OLEDB) specification to OLAP. As ODBO is a Microsoft standard using the COM (Component Object Model) platform, it requires someMicrosoft files to be installed on the client system. Alternatively, the same interface (called the OLAP BAPI) can be called via an SAPconnection, but in this case SAP files need to be installed on the client in order to establish the RFC connection. In order to avoid theseclient files and be desktop independent, there is a newer third option called XML for Analysis (XMLA), which leverages the web standardsXML (Extensible Markup Language), HTTP (Hypertext Transfer Protocol) and SOAP (Simple Object Access Protocol). Just to review thestandards: XML is to encode documents with tags to give it structure, HTTP is the communications protocol for passing information throughthe web and SOAP is a protocol for exchanging XML-based messages.

SAP refers to all three APIs as their Open Analysis Interfaces, and they all work off of the same ODBO-released queries. All Open AnalysisInterfaces use the industry-adopted OLAP language created by Microsoft called MDX (Multidimensional Expressions). MDX was firstintroduced as part of the ODBO specification in 1997 and stabilized by 1999 (latest version update of ODBO). Think of MDX as thequery language to multi-dimensional data in the same way that SQL (structured query language) is the query language to relational data.A version of Table 1 can be found on SAP help but is summarized here for quick reference on the comparisons.



OLE DB for OLAP (ODBO) OLAP BAPI XML/A

Communication layer COM (Component Object Model) RFC (Remote Function Call) XML using SOAP over HTTP

Prerequisites for the client Driver installed on client system Only one library needs to None (registration of ODBO Components be installed and registered. consisting of a few of DLLs) Access library available from all SAP platforms

Query language MDX MDX MDX

Platforms Windows SAP platforms Platform independent

Availability As of SAP BW 1.2B As of SAP BW 2.0A As of SAP BW 3.0A

Unicode Capability Fully Unicode-enabled The caller specifies the Fully Unicode-enabled, by required Code Page when default UTF-8 (Unicode establishing the RFC connection. Transformation Format, 8 bit display).

Table 1 - Comparison of the SAP BW Open Analysis Interfaces

What all these acronyms mean is that all third-party BI tools have a standard way of accessing SAP BW data through the SAP OLAP processor.However, only the SAP BEx Suite works directly against the native OLAP processor using ABAP as the language to access SAP BW data. Inorder to translate to MDX (for Open Analysis Interfaces), SAP developed an MDX processor on top of the native OLAP processor and contracted

After ODBO was first introduced close to a decade ago, SpecialistBI suites and MDX-focused Niche BI firms have either closed ornarrowed the gap from rocky starts.

Fortunately, MDX is a proven standard with a lot of industryadoption even though it is Microsoft standard (with healthyexpansion to the web-based XMLA from ODBO roots). Practitionerswill swear by it, and there are vendors other than Microsoft thathave built their products and company around it.

As SAP contracted support for its ODBO provider from Simba, OpenAnalysis Interfaces had always provided close to full coverage forMDX (at least currently, there only a few minor exceptions such assome metadata search and the ability to combine MDX cubes that can

be done via Multi-Providers in SAP BW). In addition, SAP slightlyextended MDX so that it could support its SAP BEx variables.

In contrast, Specialist BI provider support for Open AnalysisInterfaces has varied greatly and is at different levels of maturity.For SAP shops considering adding a specialty BI tool to thecomplement SAP BW, this level of integration with open analysisinterfaces one of the most important solution criteria to evaluate.

Simba Technologies indirectly tested all of the Open AnalysisInterfaces against each other and the native SAP OLAP processorby selecting a number of Specialist BI tools and SAP BEx Analyzerto run four series of tests on SAP demo content.12 What Simba wasfound was that for the same data and similar outputs the



Simba Technologies to develop the ODBO provider in parallel. The work completed in 1998 as part of SAP BW 1.2B. Since then, the OLAPBAPI and XMLA were released to round out the Open Analysis Interfaces as part of SAP BW 2.0A and SAP BW 3.0A, respectively. Thearchitecture of Open Analysis Interfaces reproduced from SAP help documentation is illustrated in Figure 3.

Figure 3 - Architecture of SAP BW Open Analysis Interfaces

CLIENTRich Client

InfoCube

Web Browser

Web Service

DataStore Master Data

Meta DataOLAP Processor

MDX Processor

OLAP BAPIs

SAP BEx

XML for Analysis

OLE DB for OLAP

COM

RFCHTTP SOAPBI SERVER

performance times greatly varied depending on the solutionprovider tool. The tests analyzed three potential factors:

• The type of Open Analysis Interface that was being called.

• The efficiency of the MDX statement that was being generated bythe application.

• The efficiency of the application after it received the data.

What Simba’s tests and conclusions highlight is that even if asolution provider is certified in one of the MDX-based interfacesdoesn’t mean there is parity with other certified solutions from anSAP integration perspective.

Furthermore, what the tests did not cover is how comprehensiveor rudimentary a solution provider support is for MDX from afunctionality perspective. There is a difference betweenmandated MDX for the Microsoft specification and optional MDXthat can have an impact on performance.

However, the level of MDX sophistication is trivial in comparisonto other factors impacting SAP BW integration. Deliberateengineering around SAP-specific concepts, conventions andmetadata is the far greater challenge. The fact that SAP adopteda Microsoft standard for querying its data does not abstractaway SAP complexity; in fact the situation is quite the opposite.Vendors integrating with SAP BW via open analysis interfacesmust be intimately aware of how SAP BEx works besides simplyunderstanding how to call the interface.

Some common problems that novice third-party vendor integrationsencounter with Open Analysis Interfaces stem from failing to take intoaccount the full range of SAP BEx capabilities. Common problemareas with Open Analysis Interfaces include (but are not limited to)how complementary tools handle the following scenarios:

• Metadata translation and synchronization of SAP BW and thelocal third party BI environment. SAP maps SAP BEx queries tothe MDX schema which in turn maps to third party metadataconventions; the translation of which is not always naturalcausing potential confusion.

• In particular, the use of both row and column structures in SAPBEx Queries may create a confusing mapping to thedimensions and facts of third party metadata; a scenario that

novice vendors may fail to account for by treating bothstructures the same way.

• Handling all the different types of SAP BEx variable inputs andincorporating them into queries—a uniquely SAP concept.Each variable is assigned one of five variable types(characteristic value, hierarchy, hierarchy node, text andformula) and one of five processing types (manual or defaultentry, replacement path, customer exit, SAP exit orauthorization exit) that drive a diverse set of behaviors.

• When third parties use the interface to replicate data for localOLAP analysis, not only is data duplicated but also SAP BWOLAP definitions may be lost. For example, SAP-definedexception aggregation (or semi-additive) behaviors are lostwhen the connection is dropped.

• SAP-relevant query output considerations such as currencies,unit of measures and hierarchy-awareness.

• The use of SAP BW time-dependent master data, hierarchiesand hierarchy structures including capabilities to set thecorrect reference date (commonly used for reorganizationsand financial restatements).

7.6 ABAP-BASED DATA ACCESSEarly SAP users discovered that it was better to read from theSAP application server layer than its database layer because ofSAP’s three-tiered client-server architecture. The SAP applicationserver has logical objects and data that the database does not,and is only accessible via ABAP. For example, the applicationserver has its own buffering and locking that stores data beforeit commits to the database. Furthermore, how data is stored inthe database may be very different than how it is represented inABAP, and in turn how it is presented to the user. Significantdata transformation can occur as information passes through thethree-tiered architecture.

For example, in the early days of SAP R/3 the application hadmore tables than the database systems it supported couldmanage. So SAP created what it called pooled tables, whichgrouped several logical tables into one physical database tableusing a particular convention that saves data as strings. Pooledtables (which are one-to-many ABAP to database tables) appearthe same as transparent tables (which are one-to-one ABAP todatabase tables) in the ABAP dictionary. The SAP financial lineitems table is an example of a pooled table.



But the data transformation journey does not stop at the ABAPdata dictionary layer; there can be further application-specificconventions applied to ABAP dictionary tables to make the dataintelligible. Similar to pooled tables, SAP has another table type,called cluster tables, that allows for complex data objects inABAP memory to be persistently stored without flatteningoperations. However, these structures have a separateconvention and data structure that requires special application-specific logic and ABAP syntax to read (using IMPORT instead ofSELECT command statements). The HR payroll databases areexamples of cluster tables.

SAP also has a concept it calls logical databases, whichrepresents a hierarchy of reporting structures that are populatedwith values via ABAP programming. The structures in thehierarchy may or may not represent actual tables. Logicaldatabases are popular for reporting because they encapsulatecomplicated data models into a standard object for reuse.However, ABAP reports that use logical databases have to followa specific programming structure and convention.

An easier use of logical databases is to use them in InfoSetsdefined by the SAP Query reporting tool (similar in concept butnot to be confused with SAP BW InfoSets—the ERP reportingstructures are sometimes referred to as “classic InfoSets” todistinguish the difference). Furthermore, SAP query can be usedto list report against transparent tables and graphically definedjoin logic defined in classic InfoSets. Similarly, Report Painterreports can be defined against simple transparent tablestructures via LIS, or against Report-Writer specific logical tablesthat are filled via specialized application-specific programs. Insome cases, this logic is also encapsulated and abstracted via aBAPI that performs functions such as reading the complex datastructures of the Controlling (CO) module for managementaccounting.

All of these SAP-specific considerations make it impractical for anySpecialist BI tool to report directly out of the SAP database.Furthermore, when shopping for a Specialist BI provider to do SAPERP reporting, the product should not only account for the SAP-specific technical complications but have pre-delivered content inorder to accelerate development. Established solution providerwith

SAP expertise will have explicit support for reading againsttransparent tables, HR Infotypes, classic InfoSets (which can alsocover logical databases) and BAPIs, as well as example templatescenarios. Anything else needs to be coded as custom RFCs thatcan replace customary ABAP report programs. Some solutionprovider also provide integration to well-known RFC-enabled APIsfor SAP currency translations and unit-of-measure conversions.

Interestingly, these same RFC-based data access techniques canbe used to read SAP BW data without the need to go through theSAP-endorsed open analysis interfaces. The SAP Data Manager(called by both the OLAP processor and the DW Workbenchtransaction to display InfoProvider data) can be directly invokedvia an API. This API is not RFC-enabled, so a “wrapper” APIneeds to be programmed in order to make it generally available.SAP has documented this API as the Data Mart interface.Similarly, master data has a documented API that is already RFC-enabled. For DataStore objects in particular, there is an SAP-endorsed BAPI that some products already explicitly support,which reads the flat structure that is commonly used as an EDWor staging layer in best practice SAP BW environments.

7.7 SAP BW METADATAWhat follows is a comparison of SAP BW to traditional BIconcepts. It is intended for those organizations that haveimplemented SAP BW and are struggling with making it fit withthe rest of their BI landscape. The basic premise is that paradigmdifferences between SAP BW and traditional BI can causechange management clashes that need to be reconciled beforethe technological differences can be sorted out. Strategicquestions also must be raised as to which paradigm needs to beembraced, extended or extinguished.

The way logical data models are designed and developed inSAP BW is typically different than in traditional EDW or datamart environments. Figure 3 below compares how SAP BWmodeling works vis-à-vis a traditional approach with EA anddata modeling tools, and compares them to the ObjectManagement Group (OMG) consortium’s Model-DrivenArchitecture (MDA) specification standard.



First, separate standalone metadata design tools are not employedto build logical and physical data models in SAP BW; everything isdeveloped and administered in the DW Workbench at the logicallayer only. Second, the build of the logical models are moreabstract and configuration-driven than data-diagrammed.Information design is not done by logical entity-relationshipdiagramming (ERD), but rather by defining SAP BW data objectsthat have an underlying implicit ERD built in.

More explicitly, SAP BW has two fundamental data objects to choosefrom for storing transactional data: dimensional structures calledInfoCubes and flat structures called DataStore objects. Users“configure” InfoCubes by graphically dragging and dropping fields(called InfoObjects) into dimensional groupings or a fact grouping.After saving and activation, the SAP BW system then automaticallygenerates what SAP calls an extended star schema. For BIpractitioners, that is a star schema that is snow-flaked or normalized

one level. In other words, InfoCubes are structures that join masterdata with transaction data directly. Users “configure” DataStoreobjects similarly by graphically dragging and dropping InfoObjectsinto either a key field grouping (one that defines the primary key) ordata field grouping. After saving and activating a standard DataStoreobject the system generates a group of three tables (one for updating,one for activating and a change log for tracking before and afterimages). These physical tables are generated in the ABAP dictionary,which would normally map to physical data models in a standalonemetadata management tool. Whereas the SAP application server thenautomatically activates and adjust the database, the metadatamanagement tool would generate a Data Definition Language (DDL)script that executes SQL to physically create or modify tables in thedatabase system.

The SAP BW system generates its “best practice” schemas thatcannot be altered (i.e. further normalized or de-normalized) at the


WHITE PAPER - Business Intelligence in SAP Environments: Understanding the value of complementary BI solutions 29Ite

rate

Enterprise Architecture Models and Tools

DBMS DBMS

SAP ABAP RepositoryPhysical Data Models

SAP BW Metadata RepositoryLogical Data Models

DDL Script DB Connect

DW WorkbenchTranslator

XMI Import/ExportMetaData Interface

COD

E SP

ACE

MO

DEL

ING

SPA

CE

Source: http://www.omg.org/mof/

Base Level: UML Platform-Independent

Model of Business Functionality and Behavior

Intermediate Level:UML Platform-Specific Model(s)

on reflected platforms generated from PIM

Automated Transformation

Automated Transformation

Top Level:Implementation(s) generated

from PSMs

Modeling in a technology-independent UML profile allows

a precise representation of business process/rules.

Executed by MDA tool which follows OMG-standard mapping. Recruiting PSM may need some

hand adjustments based on infrastructure decisions.

Modeled in a technology-specific UML profile. Represents every aspect of a coded application,

but still as a model.

Executed by MDA tool. Many tools on the market execute this step very well.

Generated code and auxiliary files ready for completion, linking with legacy or other

code and deployment.

Figure 4 - SAP BW modeling concepts

physical data model layer without serious modificationrepercussions. This logical layer can be read by any third-partymetadata tool that accepts XML Metadata Interchange (XMI)imports, which is an abstraction layer higher than data modelersimmersed in the relational world are used to playing in. It is aUnified Modeling Language (UML) standard set by the OMGconsortium for sharing models across platforms. SAP BW also hasan XMI import option so these models can be externally developedand imported, a practice which has yet to become commonplace.

Perhaps because SAP applications were born out a configuration-driven, three-tier, client-server model, SAP BW developmentabstracts the database details that most traditional EDW or datamart developers live and breathe. This thereby creates a changemanagement impact that begs the strategic question of whether thislevel of control is more effectively wielded by a system (SAP BW)or experts (traditional BI data modelers).

Furthermore, EDW purists may find a rigidity and lack of datadesigner control in order to address what might be perceived asgaps in the SAP BW delivered schemas. For example, DataStoreobjects are not designed to capture all historical changes, but aredesigned to overwrite latest changes. Some SAP BW customers thatperceived this design as an EDW requirement gap addressed it byadding a timestamp (or something similar) to the key of a DataStoreobject, or saving off the change log details. Both approaches havea customized ABAP programming impact that may have beenavoided by simply adopting the SAP BW convention or leveragingan existing legacy EDW that supports this capability. Anotherpossibility is that this customization was effective in order to beused in federated warehouses but also comply with EDWstandards. Regardless, EDW corporate philosophy is an importantdiscussion point when incorporating SAP BW.

As another EDW example, because InfoCube extended star schemascannot be flexibly defined (i.e. further normalized or linked to anytable), scenarios are created where transactional data is alsoredundantly stored as master data in order to achieve better flexibility(e.g. standard change run realignments) and performance (i.e. BIAccelerator and aggregates only support InfoCubes). As a specificexample, some environments have modeled purchase orders as masterdata in order to better handle volatile changes in the orders rather thanhistorically saving it as dimensional attributes with an InfoCube. But thisapproach creates new challenges, such as performance and thehandling of ‘attributes of attributes’ (what SAP calls transitive attributes)which fall outside the extended star schema (such as a geographical

attributes of a vendor on a purchase order modeled as master data).Transitive attributes need to be redundantly brought into the extendedstar schema as navigational attributes for reporting purposes.

In addition, data engineers may find the SAP BW metadatarepository inflexible for a number of reasons. First, it is not astandalone repository, but rather tightly coupled with SAP BW.Second, definition history or versioning is not stored anywhere;there is only business content, modified and active versions as wellas transaction event logs. Third, multiple InfoObjects cannot be sub-typed or super-typed (i.e. applying inheritance principles) but rathercreated as completely separate InfoObjects (creating significantmaster data redundancy) or with reference to each other (withupdates restricted to only the parent).

Lastly, non-SAP BW practitioners may find SAP BW data lineagecapabilities lacking. One of the criticisms of Business Contentextractors is that although it abstracts a lot of complexity, it is alsoa data lineage “black box.” The argument it that as long asBusiness Content can be used out-of-the-box then it is acceptable,but once additional fields need to be added, mapped and sourced,as well as reconciled back to underlying source tables, then datalineage documentation is needed. Furthermore, once the data isloaded in SAP BW there is no data lineage capability that can takea data element in a report and trace it all the way back to theextraction source, as can be found in some Specialist BI suites.

7.8 SAP DELTA CHANGE CAPTUREFOR EXTRACTIONCapturing changes on all records can be a complicated affair that evenSAP Business Content has missed (but then corrected) in the past. Sincetransactional data is normalized (or split) in many different tables, anynormalization (or bringing together) of those tables can create datarealignment scenarios where one table change requires a refresh of alltables. For example, financial documents are generally comprised oftwo tables: a header table and a line item details table. Both tables areextracted together as a financial document transaction by SAP BusinessContent. Each table has text fields associated with them that can bechanged after a document is posted. If either the header text or the lineitem text is changed, the whole document must be re-extracted. In morecomplicated extractors, or even BI data models where multiple sourcesof data are integrated (typically the case when marrying logistics withfinancials), there can be more complicated technical scenarios such asdata volatility, absence of an event trigger and update timingdifferences (such as batch scheduling).



SAP BW has two fundamental ways for identifying delta changes:timestamps (or datestamps) or delta queues.

The timestamp approach is the most popular and straightforwardapproach when it is available. The mechanism is similar to a fullload extraction, but is restricted to a date range of the most recenttransactions. Some SAP extractors also add what is referred to asa “safety delta,” where the date range is expanded to capture time-overlapping records for each delta pull in order to ensure recordsare not missed. For frequent updates, this might happen if there isa memory backlog that has not yet been committed to database orif a transaction is asynchronous (i.e. passed off to another workprocess and queued for update). The downstream overlaps thencan be resolved by comparing the last load with current load andreconciling the difference. In SAP BW, this is essentially done withDataStore objects and their change log and activation functions;third-party ETL products typically handle this approach as well.

The delta queue approach is more sophisticated with technicalvariations. Conceptually, delta queues are separate storage areasthat are written to after an event change (record creation, changeor deletion) and emptied after extraction. This eliminates the needfor any safety delta and downstream reconciliation of redundancy.It is more of a fail-safe mechanism that can conserve space bystoring pointers to the real data. SAP BW has a generic delta queueas a shared service for many extractors. The same delta queue isalso used in SAP BW in order to accept pushed records from SAPExchange Infrastructure (XI) or third-party Enterprise ApplicationIntegration (EAI) product. The SAP delta queue actually tracks, in acompressed format, “before” and “after” images of all changeddata records, allowing identification and tracking of every singlefield value change accessible and administered via a check tool.The way transactional systems update the delta queue can varyfrom application-specific logic to general frameworks. One popularframework is what SAP calls business transaction events (BTE),which are heavily used by the SAP Advanced Planner Optimizer(APO) product but also leveraged for SAP BW extractions. BTEs areinterfaces in the logic for document posting that can be hooked intoby outside applications like SAP APO, SAP BW or even thirdparties. Another framework for LIS extractions asynchronouslyupdates the delta queue directly via a special periodic batch job.

One advantage the delta queue approach has over the timestampapproach is that the delta queue tracks all changes betweenextractions, while the timestamp approach only grabs the latest

changes. For example, if a document is changed several times inthe same day (datestamp stays the same) or several times since thelast extraction (datestamp updated), the timestamp approach willmiss those changes even if the time of change is also recorded,unless new records are generated. Because the delta queue isevent-driven, all the change events should be registered. Oneadvantage the timestamp approach has is the ability to repeat daterange extractions. The SAP delta queue does have a repeat modewhere data from the last extraction is still kept as a fail-safe.Although the delta queue is engineered to support multiple targetsystems, third parties do not typically integrate with this mechanism,as it is part of the SAP proprietary BI service API (the interface usedby all SAP Business Content extractors).

Conceptually, another type of delta queue approach is to trackchanges in a separate log and then read the log duringextraction in order to propagate changes. In SAP, ALE changepointer technology takes this approach. ALE is SAP middlewaretechnology to integrate SAP systems with other systems (SAP ornon-SAP including SAP BW). ALE uses the standard SAP formatfor electronic data interchange, called IDocs (IntermediateDocument), to pass messages and data across networks. ALEchange pointers were used to identify the master data changesthat needed to be replicated in a distributed SAP landscape. TheBI service API leverages this infrastructure for some of its masterdata extractors for delta change tracking. ALE change pointerstrack the keys of all changed data records in a separate tableand are designed only to deliver the latest version of an entiredata record versus field level changes.

Third-party ETL products typically support ALE change pointers aswell. They also support Idocs, which should not be confused withALE change pointers. To read from ALE change pointer tables,developers do not necessarily need to use IDocs or ALE, whichare downstream mechanisms to store the changes in a messagestructure and to communicate to the other systems, respectively.Similarly, developers do not need ALE change pointers to createIDocs. A case in point is the SAP BW extractors that read thedelta queue in order to generate IDocs to pass to SAP BW viaALE. IDocs are the standard structures used to pass SAPinformation over networks to other systems. They are typicallypurged from the system as part of operations, but can beleveraged by ETL vendors as another delta change capturemechanism.



7.9 OPEN HUB SERVICESIn the latest release of SAP BW, SAP supports the direct loading ofa component called Open Hub Destination from any SAP BW DataSource without having to go through an InfoCube or DataStoreobject (whereas in previous releases it did). Both an Open HubDestination and Data Source have persistent objects associatedwith them. An Open Hub Destination can be either a file or tablewhile a Data Source has what is called a Persistent Staging Area(PSA); both can be emptied when loading is completed. As a result,a fairly direct path can be made to load an outside system throughSAP BW via the Open Hub Service such as a delta update.Furthermore, the current release supports a series of Open HubService APIs for notifications, status updates, metadata search andOpen Hub Service data extraction (instead of reading the file ortable directly) so that the process can be effectively managed andmonitored by a third-party scheduling tool.

From a customer perspective, using Open Hub Services as a directpass-through mechanism to access SAP Business Content extractors

has only two downsides in comparison to extracting directly fromthe SAP transactional system (besides not leveraging the rest of SAPBW or having another environment to maintain):

• Outbound SAP BW data has a licensing impact (in fact any dataexported from SAP BW has a licensing impact, Open HubService or not).

• Data has to be persistently stored at least twice before reachingits target: the PSA and the Open Hub Destination file or table inthe BW application server.

Although the additional persistent objects in SAP BW mightincrease latency, they gain access to SAP capabilities like deltaloading and RDA trickle feeds that speed it up which may netpositive over a custom direct access approach.

For organizations where there is already a significant SAPenvironment already, then data is already in SAP BW and only theOpen Hub Service needs to be configured to pull data out.

Copyright © 2008 Inforte Corp. All Rights Reserved. www.inforte.com2.21.08


1 According to Information Week: “When evaluating a vendor, nearly 80% of respondents are interested in the company's ability to integrate its offerings with existing applicationsand provide service and support.” Weier, Mary Hayes and Lisa Smith. “Many Companies Plan To Increase BI Spending.” Information Week, 19 March 2007.http://www.informationweek.com, Path: Many Companies Plan To Increase BI Spending.

2 The majority of this content was prepared prior to the announcement that SAP has tendered an offer to acquire Business Objects. This white paper should provide significant andpertinent context to the potential synergies this deal creates.

3 Information Week reports that 51% and 48% of those surveyed cited integration and compatibility problems and ease-of-use issues as enterprise BI deterrents, respectively. Weier,Information Week.

4 Wood, Brian. “Where Will Enterprise SOA Matter Most In Your Organization?” SAP White Paper, http://www.sap.com/community/pub/flash/kw31_07_story_2.epx.

5 Weier, Information Week.

6 Futurist Alvin Toffler coined the term “prosumer,” a combination of “producer” and “consumer,” in his book The Third Wave.

7 According to a TDWI estimate based on reported cost savings by survey respondents after data quality implementations extrapolated by Dun and Bradstreet counts of businesses byemployee size. Cost savings were realized in the way of excessive spending such as unneeded postage, printing and staff overhead. Eckerson, Wayne W. “Data WarehousingSpecial Report: Data Quality and the bottom line.” Application Development Trends, 1 May 2002. http://www.adtmag.com, Path: “Data Warehousing Special Report.

8 Davis, Carol. “Extreme Business Warehousing: Experiences With Netweaver BI at 20 Terabytes.” IBM Case Study, Document WP101012. http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP101012.

9 ISO 9241-11 (1998) Guidance on Usability

10 Autor, David H., Frank Levy, and Richard J. Murnane. “The Skill Content of Recent Technological Change: An Empirical Exploration.” Quarterly Journal of Economics, 118(4),November 2003. http://web.mit.edu/flevy/www/ALM.pdf.

11 A median value of 70% means that 50% of the task type is needed when only 30% was needed in 1960 (vertical axis normalized around the 50th percentile median measure).

12 Rajan, Amyn, George Chow and Darryl Eckstein. “ODBO, BAPI and XMLA: It’s All MDX To Me.” Simba Technologies White Paper. http://www.simba.com/docs/ODBO-BAPI-and-XMLA-Its-All-MDX-to-Me.pdf.

500 N. Dearborn Street, Suite 1200Chicago, IL 60610312.540.0900www.inforte.com

Business Intelligence in SAP Environments: Understanding the value of complementary BI Solutions

Technology

Transcript of Business Intelligence in SAP Environments: Understanding the value of complementary BI Solutions