Final Dissertation

An Exploratory Study on Synchronization of Heterogeneous Reporting Environments

By 10030241011

Khushboo Agarwal2010-12

Symbiosis Centre for Information Technology(a constituent member of SIU Established under section 3 of the UGC Act 1956

vide notification No. F.9-12/2001-U.3 of the Government of India)

Acknowledgement

Successful completion of any task would be incomplete without expression of appreciation of

simple gratitude to the people who made it possible, because success is an epitome of hard

work cogency for filling the machine indefatigable, perseverance and most of all encouraging

guidance and steering, so with veneration and honour. I acknowledge all those whose

guidance and encouragement has made it successful in completing this research. I avail this

opportunity to express deep sense of gratitude and sincere thanks to Dr. R. Raman, Director,

SCIT.

The kind gesture and technical acumen of my guide Mr. S V K Bharathi , held me out of

perplexities and predicaments. His contribution to my research is difficult to restrict in words.

I express my sincere thanks to him.

I would like to take this opportunity to say thanks to all the respondents who have helped me

in getting a deeper understanding of the topic and its usage in the industry. One of the person

who has helped me a lot in the completion of this research is Mr. Arvinder Singh (Manager in

Bcone) who was my manager at the time of my internship.

Lastly I would like to thank my friends for their valuable support without which it have been

very difficult to complete the research. They not only helped me in my work, but also

providing me the moral support to finish our project work successfully.

Thanking you,

Khushboo Agarwal

An Exploratory Study on Synchronization of Heterogeneous Reporting Environments 2

Table of ContentsCHAPTER 1 Introduction......................................................................................................................................................4

1.1 Summary of Abstract............................................................................................................................................................4

1.1.1 Need..............................................................................................................................................................................4

1.1.2 What is MDM?..............................................................................................................................................................5

1.2 Objectives:............................................................................................................................................................................5

1.3 Scope:...................................................................................................................................................................................6

1.4 Methodology:........................................................................................................................................................................6

1.4.1 How can MDM help?....................................................................................................................................................6

CHAPTER 2:................................................................................................................................................................................. 7

2.1 Literature Review:..............................................................................................................................................................7

2.1.1 Types of Data in the Enterprise.....................................................................................................................................9

2.1.2 Why research is carried out?........................................................................................................................................10

Chapter 3 Analysis of Work Done...........................................................................................................................................11

3.1 Analysis of Problem under Research.................................................................................................................................11

3.2 Identifying The Challenges and the solution suggested:.....................................................................................................11

3.2.1 MDM Lifecycle...........................................................................................................................................................12

3.2.2 Strengths of Master Data Management:.......................................................................................................................13

3.2.3 Advantages of Master Data Management....................................................................................................................13

3.2.4 Limitations of MDM:..................................................................................................................................................14

3.2.5 Where does MDM fit into business strategy?..............................................................................................................14

3.2.6 MDM business value drivers and benefits...................................................................................................................15

3.2.7 MDM Workflow..........................................................................................................................................................15

3.3. Proposed Solution:.............................................................................................................................................................16

3.4 Technical Justification of the Solution...............................................................................................................................17

3.4.1 MDM CAPABILITIES...............................................................................................................................................17

3.4.2 How MDM helps in different scenarios:......................................................................................................................21

3.5 Technical Environment & Technical Details......................................................................................................................23

3.6 Possible Applications in the Industry..............................................................................................................................26

3.6.1 Where MDM can be used?..........................................................................................................................................26

CHAPTER 4.................................................................................................................................................................................29

4.1 Findings :....................................................................................................................................................................29

4.2 Recommendations.......................................................................................................................................................30

4.3 Conclusion :........................................................................................................................................................................31

References and Bibliography:........................................................................................................................................................32


CHAPTER 1 Introduction

1.1 Summary of Abstract With the ever increasing demand for cost optimization, faster product launches, more efficient compliances with regulations, one of the biggest pain areas for enterprises is achieving consistent data quality. Companies across many industries face business challenges that affect their master data—the high-value, business-critical information about customers, suppliers, products and accounts—and the ability of IT to deliver on the requirements of a dynamic business. This critical business information is replicated and fragmented across business units, geographic branches and applications. Enterprises now recognize that these symptoms indicate a lack of effective and complete management of master data.

1.1.1 Need: There is a need in the market for integrating various heterogenous reporting environments. A company which has SAP as their database acquired a company which has its presence in different countries. CEO of the company wants a single report as what is the sale of a particular product, it becomes very difficult to extract data from different sources with different databases having different names of a single product, so there is a need for having a single reporting tool which can integrate all the different databases for a common reporting purpose.

The evolution of master data management solutions In general, master data management (MDM) solutions should offer the following: • Consolidate data locked within the native systems and applications • Manage common data and common data processes independently with functionality for use in business processes • Trigger business processes that originate from data change • Provide a single understanding of the domain—customer, product, account, location— for the enterprise

What is heterogeneous environment? Heterogeneous environment is a combination of different reporting environments in an organization. Consider an example, a company has Oracle as their database, but it has acquire some companies which have different databases like SAP, People soft, EBS etc. then it becomes difficult for that organization to consolidate data and extract data from different databases.The higher management needs only a single consolidated report which is not possible to extract.There were a lot of problems being faced by companies without having a common reporting mechanism:

1. Inconsistent and duplicate business-critical data—such as customers, products, partners, and suppliers—stored in different formats in disparate systems across the enterprise can impede strategic business imperatives.

2. No consolidated and reliable data3. Difficult to acquire and retain customers4. Tough to leverage operational efficiency as a competitive differentiator,5. Support informed decision-making across the enterprise.

The solution to these problems is MDM.


1.1.2 What is MDM?Master Data Management is an essential discipline for any company considering Information as a strategic asset or planning to do so. A set of disciplines, processes and technologies, for ensuring the accuracy, completeness, timeliness and consistency of multiple domains of enterprise data - across applications, systems and databases, and across multiple business processes, functional areas, organizations, geographies and channels. Master data management (MDM) provides a means to link data from various structured data sources and to generate one master record for each entity. For example, MDM delivers an integrated view of key information about entities such as customers, products, and suppliers."An MDM layer enables companies to realize internal efficiencies by reducing the cost and complexity of processes that use master data (through fewer code clashes, less data duplication, better control over business processes, and so on). It reduces manual translation and analysis to improve repeatability and speed to insight. Other MDM benefits include increased revenue (for example, from providing more accurate and comprehensive information to the right customers at point of sale) and regulatory compliance."

Consider the following example. A customer purchases a product, which creates or changes data within the MDM customer domain application and also in the separate MDM product domain application. Each MDM application persists the information most applicable to its own domain focus, but that information is not integrated with the other domain, in effect creating silos of master data.

Take, for example, a client’s location. While a customer relationship management (CRM) system may display one address, an accounting package may show another. Yet a third address may be included in an electronic document, such as a purchase order, transferred during the course of a business-to-business transaction. The goal of Master Data Management (MDM) is to enable this ideal world. Through a combination of architecture, technology, and business processes, MDM provides an approach to incrementally reducing the amount of redundantly managed information and providing information consumers throughout an enterprise with authoritative master data. MDM Systems that focus exclusively on managing information about customers are often called Customer Data Integration (CDI) systems. MDM Systems that focus exclusively on managing the descriptions of products are called Product Information Management (PIM) systems. MDM Systems that enable multiple domains of master data, and that support multiple implementation styles and methods of use, are sometimes also called Multi-Form MDM Systems.

1.2 Objectives: The main aim of this dissertation project is to study the integration of different reporting heterogeneous environments to generate a single report which helps the higher level management in taking decisions and why it is needed.


1.3 Scope: The scope of this dissertation will include how the data from different databases will be extracted , how it will be integrated, and how will the problem of integrating data will be solved. How the problems of increasing revenues, increasing profits, improving efficiency will be solved through this integration.There will be a common reporting tool which can integrate heterogeneous environments to a single report which can solve the problem of organizations of integrating different data from different sources.

1.4 Methodology: It includes the study of different case studies where the companies face these type of problems, where they have integration and reporting problem and why they have installed MDM. How the problems are solved after having Master data Management solution.

1.4.1 How can MDM help?MDM is a deliberate initiative comprising of a set of methodologies, strategies, disciplines, and technologies that enable organizations to acquire, cleanse, enrich, consolidate, federate and govern data across many disparate systems.

1.4.1.1 Master Data Management (MDM) : An Approach for ImplementationFour MDM business objectives shared as part of a lengthy, technically-advanced look into MDM. These are as follows: 1) Improve an organization's ability to adjust to rapidly changing business requirements,2) Improve operational efficiency (Streamline business processes, Improving data quality),3) Improving information management efficiency • Enable broader and more complex data integration,

• Eliminate redundant data management activities,• Eliminate redundant integration activities

4) Improve decision making• Enable data quality improvements,• Simplify data integration

1.4.1.2 Master Data Management Quotes by Analysts "The prime reason for MDM's emerging popularity, and the principal failure of data quality strategies and solutions to date, has been the development and deployment of enterprise applications that have remained siloed. Enterprise applications -- whether it's ERP, CRM, or SCM -- they've all become silos of information within the enterprise. As a result, the data has been duplicated over and over again." MDM’s Monumental Momentum Andrew White, Gartner"MDM's time has come. For years, enterprises have been product-driven businesses. Now, the shift is under way to become more customer-centric, and that's going to require MDM. " Ranjay Gulati, Distinguished Professor of Strategy and Organizations, The Kellogg School of Management at Northwestern University


CHAPTER 2:

2.1 Literature Review :

The pain that organizations are experiencing around consistent reporting, regulatory compliance has prompted a great deal of interest in Master Data Management (MDM). This research explains what MDM is, why it is important, and how to manage it, while identifying some of the key MDM management patterns and best practices that are emerging. [The What, Why, and How of Master Data Management.htm]

The paper includes the solutions to the problems which are solved by MDM. Understanding the fundamental problem that MDM addresses helps answer the following three questions. In a nutshell, the modern IT application landscape is fragmented. This leaves critical data domains such as customer, product, site, and supplier created and managed differently within each of the many applications. Initiatives to increase the quality of the data within each application do nothing for cross application consistency. These data inconsistencies break enterprise business processes and are propagated into the data warehouse and analytical systems. Data quality initiatives for the data warehouse try to fix problems in a snapshot representation of application activity, with no way to deal with the root cause inconsistencies across the applications themselves. Garbage keeps flowing in, and we all know what that does for the reports that flow out. [blogs.oracle.com/mdm/entry/why_mdm_why_now_why_oracle]

A key characteristic of an MDM solution is to enable sustainability - this isn't a one time activity but rather a process and service that remains as a key strategic and enabling core competency going forward. The benefits to getting MDM right are significant - in highly regulated industries for example, MDM can mean decreasing risk and financial impact associated with regulatory compliance, fines, recalls and lawsuits. Common across all domains, MDM excellence enables driving out of costs and inefficiencies imbedded within processes, decreasing cycle times, improving agility, time to market and increased ability to drive growth and add customer value. [ www.itbusinessedge.com/cm/blogs/lawson/why-mdm-is-a-strategic-discipline]

An MDM project plan will be influenced by requirements, priorities, resource availability, time frame, and the size of the problem. Most MDM projects include at least these phases:

1. Identify sources of master data. This step is usually a very revealing exercise. Some companies find they have dozens of databases containing customer data that the IT department did not know existed.

2. Identify the producers and consumers of the master data. Which applications produce the master data identified in the first step, and—generally more difficult to determine—which applications use the master data. Depending on the approach you use for maintaining the master data, this step might not be necessary. For example, if all changes


http://blogs.oracle.com/mdm/entry/why_mdm_why_now_why_oracle

are detected and handled at the database level, it probably does not matter where the changes come from.

3. Collect and analyze metadata about for your master data. For all the sources identified in step one, what are the entities and attributes of the data, and what do they mean? This should include attribute name, datatype, allowed values, constraints, default values, dependencies, and who owns the definition and maintenance of the data. The owner is the most important and often the hardest to determine. If you have a repository loaded with all your metadata, this step is an easy one. If you have to start from database tables and source code, this could be a significant effort.

4. Appoint data stewards. These should be the people with the knowledge of the current source data and the ability to determine how to transform the source into the master-data format. In general, stewards should be appointed from the owners of each master-data source, the architects responsible for the MDM systems, and representatives from the business users of the master data.

5. Implement a data-governance program and data-governance council. This group must have the knowledge and authority to make decisions on how the master data is maintained, what it contains, how long it is kept, and how changes are authorized and audited. Hundreds of decisions must be made in the course of a master-data project, and if there is not a well-defined decision-making body and process, the project can fail, because the politics prevent effective decision making.

6. Develop the master-data model. Decide what the master records look like: what attributes are included, what size and datatype they are, what values are allowed, and so forth. This step should also include the mapping between the master-data model and the current data sources. This is normally both the most important and most difficult step in the process. If you try to make everybody happy by including all the source attributes in the master entity, you often end up with master data that is too complex and cumbersome to be useful. For example, if you cannot decide whether weight should be in pounds or kilograms, one approach would be to include both (WeightLb and WeightKg). While this might make people happy, you are wasting megabytes of storage for numbers that can be calculated in microseconds, as well as running the risk of creating inconsistent data (WeightLb = 5 and WeightKg = 5).

7. Choose a toolset. You will need to buy or build tools to create the master lists by cleaning, transforming, and merging the source data. You will also need an infrastructure to use and maintain the master list.


2.1.1 Types of Data in the Enterprise

What Kind ofInformation?

Examples How Is ItUsed?

How Is ItManaged?

Metadata

Descriptiveinformation

XML schemas,database catalogs,WSDLdescriptionsData lineageinformationImpact analysisData Quality

Wide variety ofuses in toolingand runtimes

Metadata repositories,by tools,within runtimes

ReferenceData

Commonlyused values

State codes,country codes,accountingcodes

Consistentdomain of valuesfor commonobjects

Multiple strategies

Master Data Key businessobjects usedacross anorganization

Customer dataProductdefinitions

Collaborative,Operational,and Analyticalusages

Master DataManagementSystem

TransactionalData

Detailed informationaboutindividualbusinesstransactions

Sales receipts,invoices,inventorydata

Operationaltransactions inapplicationssuch as ERP orPoint of Sales

Managed by applicationSystems

HistoricalData

Historical informationaboutboth businesstransactionsand master data

Data warehouses,DataMarts, OLAPsystems

Used for analysis,planning,and decisionmaking

Managed by informationintegrationand analytical tools


2.1.2 Why research is carried out?The research is carried out to understand the pain areas of different organizations and how they have solved those problems and in which way.Most organizations and analyst agree that the basic reason the reporting is wrong is that the operational data feeding the analytical engines is filled with errors, duplications and inconsistencies. If the poor quality reporting is to be fixed, it has to be fixed at its source – poor quality data under the applications that run the business. This is the Master Data. The solution to this overarching problem is Master Data Management. MDM is the glue that ties analytical systems to what is actually happening on the operational side of the business. [ Better Information through Master Data Management – MDM as a Foundation for BI An Oracle White Paper September 2011]

How do you get from a thousand points of data entry to a single view of the business? This is the challenge that has faced companies for many years. These data quality problems continue to impact operational efficiency and reporting accuracy. Master Data Management is the key. It fixes the data quality problem on the operational side of the business and augments and operationalizes the data warehouse on the analytical side of the business. In this paper, we will explore the central role of MDM as part of a complete information management solution. [Master Data Management An Oracle White Paper November 2007]

Why Wipro has implemented MDM solutions in their organization, how the problems solved by implementing MDM. The implementation is aimed to enable better and faster business decisions across the customer organization, standardize business processes and improve collaboration, meet strict compliance requirements and reduce costs through enhanced process efficiency. This was the objective of the MDM implementation, with a focus on business intelligence. The solution provides simultaneous multi domain MDM implementation, including data governance to enable single organizational view of master data. [news/wipro-implements-mdm-solution-at-lexmark/618298/]

Why TCS implemented MDM?Quality data, consistent across the enterprise, can drive significant business benefits. Efficient processes and effective governance are critical to achieving this goal. TCS offers Master Data Management strategies and solutions to give the insight and tools to reap benefits through a single and consistent 360-degree view of data across your enterprise. TCS' focused Master Data Management Solutions are supported by dedicated Centers of Excellence with a multidisciplinary resource pool. In coalition with best-in-class technology vendors and strategic alliance partners,MDM Solution capabilities span:

Consulting Data Strategy Architecture Implementation Support

[ http://www.tcs.com/offerings/consulting/it_consulting ]


http://www.tcs.com/offerings/consulting/it_consulting

Chapter 3 Analysis of Work Done

Many companies still don’t have a true view of their customers, products, and suppliers, much less their inventory and financials. While they invest in new, sophisticated enterprise applications to handle business processes, the data they generate is not centrally managed. In fact, these systems often generate inconsistent and conflicting information. While efficiency may improve in specific operational functions, an overall view of the enterprise actually becomes harder to achieve. Every move a company makes depends on the data that’s circulating through the operational systems. When it’s unreliable, this data will affect decision making, and runs the risk of:

Slowing new product introductions Creating supply chain inefficiency Increasing the cost of compliance Hiding revenue Reducing sales efficiency Misguiding marketing efforts Losing customer loyalty

3.1 Analysis of Problem under Research There are many reasons why organizations are addressing a need for integrating data now, even though they may not have paid attention to it in the past. Although the specific way that each enterprise uses information may vary by size, type of business, number of touch points, and other critical variables:

• Information is now more pervasive within most enterprises • There are more regulatory requirements now than ever• There is a greater need to communicate more efficiently with business partners, suppliers,

and customers• Many enterprises have disparate technology systems between business units and regions • Many organizations are trying to manage multiple transaction or operational systems

performing similar functions• Data generated in different contexts needs to be rolled up or combined to provide a

broader, analytic view of the enterprise

3.2 Identifying The Challenges and the solution suggested:Regardless of the catalyst that makes effective data management increasingly important, there are clearly issues that create costly and time-consuming challenges in enterprise information management. Common challenges that every company faces today are: data quality, creating “a single version of the truth,” and information flow. These challenges are perpetuated by manual processes to merge, transform, and translate data.

• Security• Staleness of data• Data quality• Accessibility• Standardization• Human error


The result of a non-integrated and unplanned data management program is operational inefficiencies that are costly and time-consuming. The solution to these problems are MDM. Now explaining the cycle of MDM, how it works, how it integrates different types of data.

Figure (MDM_Productsheet_US.pdf)

3.2.1 MDM LifecycleAn effective MDM implementation involves more than just creating and running the required applications. Once the applications are in place, the MDM Suite continues to cleanse and deduplicate data and makes the updated information available to external sources. The MDM Suite organizes the MDM lifecycle into three phases: Creation, Synchronization, and Syndication.

Creation - This phase begins with analyzing the structure of the reference data and then designing and building the master index application based on that analysis. Once the master index application is configured, the data quality tools can be generated in order to profile, cleanse, match, and load the legacy data from external systems that are part of the MDM system.

Synchronization - The MDM application can propagate any reference data updates to external systems that are configured to accept such information. Once MDM services are implemented as either passive or active services, the project can be configured to actively deliver MDM services to external systems. Synchronization keeps data in all systems current, and is an ongoing process.

Syndication - Once the MDM application is running, you can create and manage virtual views on the reference data, defining who in your organization can see what information and how that information is presented. All access to information is available as services implemented by the MDM Suite in different views.

For example, your accounting department might need a different set of data than the sale department requires. Syndication removes the complexity of obtaining information from multiple sources and provides a single point of access.


3.2.2 S trengths of Master Data Management:

1) Generic Data modelling: The data modelling tool is flexible enough to cater for a wide variety of master data that exists within an organization. In comparison other products may focus on either customer or products data management.2) Compliments Hub-Spoke architecture: MDS is a very open master data management platform. With the ability to be accessed via the Database layer or the range of Webservice API’s that are available. This makes for a very flexible solution in which MDS can serve as a HUB for obtaining and providing data from upstream and downstream systems using the preferred design pattern.

3) Integration with existing Microsoft technologies: In conjunction with the above, MDS integrates and leverages with existing Microsoft technologies such as SSIS, Biztalk, and Sharepoint 2010 for workflows, which is great for those organizations that have also invested into those technologies

3.2.3 Advantages of Master Data ManagementThe goal of master data management (MDM) is to enable this ideal world. Through a combination of architecture, technology, and business processes, MDM is an approach to reducing the amount of redundantly managed information incrementally, providing information consumers throughout an enterprise with authoritative master data.

A single source of master data represents three important capabilities:

I. Lower operational costs Using MDM software to bring together isolated data helps automate manual business processes and reduce errors. MDM can also help rationalize systems and applications after a merger or acquisition, as well as indicate when duplicate customer or product data exists in disparate systems. As a result, it helps companies eliminate redundancies such as the double mailings of statements, promotional offers or product catalogs.

II. Improved agility By quickly consolidating customer information from different systems, companies can more easily identify sales opportunities or underserved regions, enabling them to tackle potential new markets or business channels. This also helps companies bring customers on board faster and serve them better by delivering customized offers based on client profiles and preferences.

III. Increased compliance and reduced risk MDM can help reduce fraud. When business processes are consistent and use accurate data, crucial steps are less likely to be skipped, and more likely to adhere to government regulations, industry standards or even corporate service-level agreements. This enables prompt and accurate audit reporting. MDM software also helps ensure that companies have a complete view of where data resides, so they can always control access to it.

IV. Ability to use the information in a consistent way: A critical element in this ideal world of master data is flexibility. As business requirements, regulations, and


implementation technologies change, we often find that the definition and use of the master data needs to evolve as well. In our ideal world, this type of change would not be a disruptive to the environment. For example, if a retailer decides to open stores in a new country, it should be easy to extend the definition of its products to accommodate additional information, such as new currencies and new regulatory requirements. At the same time, we don't want this simple data model change to break existing applications

V. Ability to evolve the master data and the management of the master data to accommodate changing business needs:

VI. Accelerate time-to market through process automation and information sharing for new product introduction processes and supply chain collaboration.

VII. Increase customer satisfaction through a multichannel unified customer experience.

VIII. Reduce shipment inaccuracies and invoice errors with trading partners.

IX. Be able to analyze and optimize procurement through a single view of vendor.

X. Get a global view of risk across counterparties.

3.2.4 Limitations of MDM:

1. Omission of deduplication engine: One of the main processes of MDM is the detection and removal or duplicates, which is completely omitted within the MDS platform. Yes it can be facilitated via SSIS or the API’s however this would mean a separate area for reporting on these items which is outside of the MDS platform. However it has been noted that it will be available in the next version of MDS.

2. Stewardship UI: The UI that is shipped with the product is functional, but it is by no means up to the standards in which it is expected from microsoft especially in comparison to Sharepoint 2010. It good for the one-off updates but will be combersome for those that are heavy users which might be use to excel. However there is currently a 3rd party product called Master Data Maestro which fills this gap and provides additional functionality such as merging versions, and cut and paste to child and sybling hierarchies which is currently not present within the web UI. I would recommend taking a look at this product to address any issues with regards to functionality of the existing interface.

3.2.5 Where does MDM fit into business strategy?

Master data is the information about customers, products, materials, accounts and other entities that is critical to the operation of the business. But companies hold pieces of master data in many different applications, such as enterprise resource planning (ERP) and customer relationship management (CRM) systems. Each of those source systems creates and holds the data in its own


unique way. As a result, information does not match from one system to the next. Critical data elements may be missing,duplicated or inconsistent. Further, each department can only operate from within its own compartmentalized view.

3.2.6 MDM business value drivers and benefits1. Growth

Determine and execute on opportunities for cross-sell and up-sell Provide a consistent experience across all customer touch points Identify high-value customers for priority service

2. Agility• Speed time to market for new product introductions• Enable faster and more accurate customer on-boarding• Deliver customized offers based on client profiles and preferences

3. Spending • Eliminate multiple mailings to members of the same household• Consolidate duplicate customer records• Rationalize systems and applications from acquired companies

4. Compliance • Support customer opt-in and privacy preference programs• Enable prompt and accurate audit reporting• Synchronize with third-party data sources such as government “watch lists”

3.2.7 MDM Workflow

The following steps describe the general workflow for implementing the MDM Suite solution once you create the master index application and generate the data quality tools. These steps correspond to the diagram below.

Extract data from existing systems (Data Integrator). Configure standardization, cleansing, and analysis rules, and then cleanse and profile the

extracted data (Data Quality). Match and load standardized data (Master Index and Data Integrator). Deploy the MDM application to perform ongoing cleansing and deduplication (Master

Index Server). Monitor and maintain data using the data stewardship application (Master Index Web

Application).


Source: MDM Workflow Diagram (http://docs.oracle.com/cd/E19509-01/820-5699/ref_mdm-primer-process_c/index.html)

1. Perform a preliminary analysis of the data you plan to store in the master index application to determine the fields to include in the object structure and their attributes.

2. Create and configure the Master Index application, defining the object structure, standardization and match logic, queries, runtime characteristics, and any custom processing logic.

3. Create the database that will store the reference data.4. Define security for the MIDM and any web services you will expose.5. Generate the profiling, cleansing, and bulk match and load tools.6. Extract the data from external systems that will be profiled, cleansed, and loaded into the

master index database.7. Analyze and cleanse the extracted data. Adjust the application configuration based on the

results.8. Perform a match analysis using the IBML tool. Adjust the matching logic based on the

results.9. Load the matched records to the master index database.10. Build and deploy the MDM project.11. Define connectivity to external systems using a combination of adapters, business

processes, web services, Java, and JMS Topics.12. Create any necessary presentation layer views.

3.3. Proposed Solution:Solution to these problems are MDM (master data Management).

Key processes for any MDM system.:


http://docs.oracle.com/cd/E19509-01/820-5699/ref_mdm-primer-process_c/index.html

• Profile the master data. Understand all possible sources and the current state of data quality in each source.• Consolidate the master data into a central repository and link it to all participating applications.• Govern the master data. Clean it up, deduplicate it, and enrich it with information from 3rd party systems. Manage it according to business rules.• Share it. Synchronize the central master data with the connected applications. Insure that data stays in sync across the IT landscape.• Leverage the fact that a single version of the truth exists for all master data objects.

Profile: The first step in any MDM implementation is to profile the data. This means that for each master data business entity to be managed centrally in a master data repository, all existing systems that create or update the master data must be assessed as to their data quality. Deviations from a desired data quality goal must be analyzed. Examples include: the completeness of the data; the distribution of occurrence of values; the acceptable ranges of value; etc

Consolidate: Consolidation is the key to managing master data. Without consolidating all the master data attributes, key management capabilities such as the creation of blended records from multiple trusted sources is not possible.. Share: Clean augmented quality master data in its own silo does not bring the potential advantages to the organization.

Consolidation is the key to managing master data, and a logical and physical model that canhold the master data is a prerequisite to true consolidation.

Leverage MDM creates a single version of the truth about every master data entity across the enterprise. This data feeds all operational and analytical systems across the enterprise.

3.4 Technical Justification of the Solution

3.4.1 MDM CAPABILITIES The following sections highlight the key MDM capabilities supporting BI.

1. Data Model The MDM data model is unique in that it represents a superset of all ways master data has been defined by all attached applications. It has the flexibility to accommodate organization and industry specific extensions. The model is tailored to map to the way organizations do business. It holds all necessary hierarchical information, all attributes needed for duplicate identification, removal and prevention, as well as cross-reference information for all attached operational systems.


Example, the single master schema holds customer data in both business-to-business (Old Navy, Banana Republic) and business-to-consumer (Mary Smith, Mary Evans) formats. In addition, it holds the master supplier data (Acme, Inc, AI Corp) and retail product data (VN-Sweater, RF-Sweater). The names and all needed attributes are maintained.

2. Change Management In order to deal with real time changes to master data, such as the marriage of Mary Smith to Mr. Evans, Oracle‟s MDM solution utilizes an Event-Driven Architecture. Any change to master data attributes triggers a business event that in turn invokes a workflow process. The workflow process builds appropriate XML payload packages and executes the configured steps for the particular data change.

Example, the introduction of Mary Evans triggered a „New Customer‟ event. This kicked off a workflow to populate Mary‟s record with all available information. For example, it may have requested address validation from Trillium (or other postal address verification vendor) to insure that all addresses are mailable. Standardized addresses also aid in duplicate identification. The workflow may have requested data augmentation for credit ratings, or obtained an AbiliTec ID from Acxiom to assist with duplicate identification. This is done in real time.

3. Person Duplicate Identification Oracle‟s MDM solution for customer data is Customer Hub. It comes with a set of Oracle Enterprise Data Quality servers for finding duplicate customer records. A primary technique is to configure a rules engine to find potential matches using a large number of customer attributes.

Example, Old Navy has entered Mary Smith as a customer. Her master ID is 551. The Customer Hub manages Old Navy as a source system (ID = ON) and records Mary Smith‟s ID in that system as 1234. Mary Evans is similarly managed. This is the base for the MDM cross-reference. MDM utilizes all available attributes to determine if these are duplicates. Typical match rules will examine addresses, phone numbers, e-mail addresses etc. Additionally, 3rd party data such as an AbiliTec ID from Acxiom may be used. In our example, the system fines that Mary Smith and Mary Evans are indeed duplicates in spite of the different name.


4. Company Duplicate Identification With Oracle‟s Supplier Hub, company duplicate identification uses the same general rules engines as the Person duplicate identification. The key difference is that the number and type of attributes available for a company are different. For example, companies can have a DUNs number provided by D&B.

Example, a search on AI Corp produces a match with Acme Inc.

Alias information was used by out-of-the-box duplicate identification rules.

5. Duplicate Elimination & Cross-reference Once the Customer Hub identifies Mary Smith and Mary Evans as duplicates, it eliminates the duplicates by merging the multiple records into one. The cross reference is maintained. Where before the merge, there were two customer records each pointing back to one source system, we now have one customer record pointing back to two source systems.

6. Hierarchy Management Hierarchy information is critical for proper aggregation and roll-ups. Oracle‟s Customer Hub utilizes Oracle Data Relationship Management (DRM) to maintain any number of simultaneous hierarchies used by the operational applications. These include Dunn & Bradstreet hierarchies with out-of-the-box connectivity to D&B for both batch and real time information access.


Example, D&B provides the hierarchy information for Old Navy and Banana Republic. It turns out that they are both subsidiaries of The Gap.

7. Product Standardization Oracle‟s MDM solution for product data is the Product Hub with a set of Oracle Enterprise Product Data Quality (PDQ) servers for product data standardization. This standardization enables rapid and parameterized searching and accurate duplicate identification. In our example, Old Navy uses the string: VN PO 50 Blue W 24W 36B 22A. Banana Republic‟s sweater is identified by: B Wool V Neck Pllver S:36. These records are loaded into the Product Hub schema through PDQ‟s Data Lens. Attributes such as style, color, and size are populated as well as catalog codes. An English description is generated as well as other appropriate languages as

needed.

8. Updated Star Schema MDM has identified the customer duplicates; maintained the cross reference back to the sourcing systems across a merge; developed the single golden customer record utilizing survivorship rules; found the two products to be identical; learned that the two retailers belong to one corporate hierarchy; and found through good duplicate identification techniques that Acme, Inc. and AI Corp are in fact two names for the same vendor. If we deliver this updated cross reference and dimension data to the data warehouse, we get the following star schema.


3.4.2 How MDM helps in different scenarios:

Analytics perspective: Organizations can employ quality data for reporting and compliance purposes and to optimize and enhance partner and channel engagement.

Operations Perspective:Organizations can build a centralized hub representing the best version of the truth, providing an accurate, consistent, and secure copy of master data to all systems and business users across LOBs.

Scenario 1:

Information consolidation Companies in such industries as banking, insurance and telecommunications often grow their business through mergers and acquisitions. Newly merged entities are left with complex IT infrastructures; that is, many duplicate applications, databases and warehouses. In response, these companies may embark on infrastructure rationalization and modernization projects to consolidate sources of customer data, product data and other master data elements. It uses MDM to build a physical master repository of data for customers, accounts, products or other domains. MDM creates what is essentially a “golden record” for each master data domain, a single source of authority that enables accurate analysis and decision making.

Billing

MDMTech Support

In-store Sales and Marketing

Online sales and marketing

Scenario 2:

Secure information sharingNot all organizations have control over all phases of customer data. For example, a single corporate entity might have distinct lines of business or geographic locations that maintain independent information systems. Other institutions may collaborate among affiliated entities or trading partners. These types of organizations need to aggregate, share or distribute relevant information, but security and privacy concerns present an ongoing challenge.


Grey Janice65, James street

10/1/10 |Lvl1| Resolved

5/23/11 | $235 | Software

[email protected]

CustomerJanice Grey65, James [email protected]

History10/1/10 |Support| Lvl1| Resolved5/23/11 |Purchase in-store| $235 | Software

mailto:[email protected]

Consider the healthcare industry, in which patient data is maintained by multiple, unaffiliated entities—hospitals, physicians’ offices, clinics, laboratories and pharmacies. There is an increasing need in healthcare not only to keep costs down, but because of government regulations, to keep patient data secure. How can healthcare partners safely share and analyze patient data to improve care and optimize service delivery?

Department of Motor Vehicles (DMV)

MDMChild Welfare and Support (CWS)

Women and Infant Children (WIC)

Health Services

=virtual

Scenario 3: Collaborative authoringProduct development is one of the areas where MDM is most valuable. For organizations to quickly bring new offers to market, they need to centralize the creation and maintenance of master information about products.

Organizations that employ MDM collaborative authoring capabilities for PIM can realize considerable benefits:

• Optimized customer, partner, supplier and employee relationships • Fewer product information errors, leading to improved sales and reduced losses • Multichannel initiatives built upon a common, trusted source of product information • Published materials created from reliable, current product information • Accurate, up-to-date e-commerce initiatives • Improved operational efficiency enabled by leveraging accurate product information

For example, product managers might enter product descriptions and bills of materials, while advertising agencies update product images and engineers input components and packaging data.The result: a single, consistent and accurate view of product information that enables accurate decision making. MDM can help organize the complex and chaotic process of product development.


Grey Janice65, James street

10/1/10 | Case worker home visit

5/23/11 | Distribution | $500.00

7/7/08 | Referral to specialist

Customer Janice Gray 65 Main Street

History 7/7/08 | Health Services | Referral to… 10/1/10 | CWS | Case worker home… 5/23/11| WIC | $500.00

Supplier data

MDM

ERP Data

Marketing Data

Print Catalog Data

3.5 Technical Environment & Technical Details

MDM methods of useMDM requires the ability to collaborate, define, and publish master data, operational processes to manage and maintain master data throughout its transactional stages, and analytical capabilities to provide better insight and leverage embedded information. Multi-Form MDM is a term used to address the fact that MDM supports multiple styles of use for master data (collaborative, operational, and analytical) and spans multiple data domains, such as customer and product. It is not uncommon for multiple methods of use to be applied even to the same data domain within a large enterprise environment.i) Collaborative style

The collaborative style of MDM supports the definition, creation, and synchronization of master data. This style is often associated with the creation, augmenting, or altering of master data to support processes, such as the new product introduction and definition process or data stewardship. There are always business processes associated with maintaining master information, whether it's setting up new products to be sold, hiring new employees, or managing suppliers. The MDM system participates in such processes, either driving the entire process or it can be called by another system.


Name: Flat Screen TV Product ID: FT64-10 Contrast Ratio: 30,000:1

Name: Flat Screen TV Product ID: FT-011

Name: Flat TV Product ID: FT70-10 Description: Black & Silver new model

Name: Flat TV Product ID: FT85-43 Contrast Ratio: Unavailable

PIM Data

Product ID: FT64-10 Supplier ID: BGR-1448 Short Name: Flat TV Long Name: Flat Screen TV Print Description: Flat TV Promotional Description: Black & Silver new model Contrast Ratio: 30,000:1

Collaborative MDM

Source: (http://msdn.microsoft.com/en-us/library/bb410798.aspx)

ii) Operational styleThe operational style of MDM supports the consumption of master data by operational systems to perform transactions, and the MDM repository is considered the authoritative source of master data. Furthermore, in operational mode, master data is leveraged by applications through services, where services provide control over master data creation, management, quality, and access. For example, as part of a process to add a new customer, a Line of Business (LOB) system would consume an MDM service to validate if this customer is a unique customer or an existing customer. The MDM service would cleanse and standardize the new customer information and perform matching logic against the MDM repository to determine if the customer already exists within the LOB system or within the enterprise.If it is determined that the customer is a new customer for that LOB, the LOB system could commit the new customer information to its transactional database.

Operational MDM

Source: (http://msdn.microsoft.com/en-us/library/bb410798.aspx)For example, registry information in the MDM repository can be used to consume a federated query service to create a virtual record consisting of structured and unstructured


http://msdn.microsoft.com/en-us/library/bb410798.aspx


data that spans heterogeneous systems, and return the results to an authorized user, application, or process.

Example New Account Opening process.

A good example of Operational MDM usage is a New Account Opening business process. In this process, a person or organization wants to open a new account—perhaps a bank account, a cable TV account, or any other kind of account. If the customer isn’t already known, then the new customer is added to the MDM System and a new account is created (presuming that the new customer meets the appropriate requirements).

iii) Analytical styleIn analytical MDM, master data from the MDM system is used as the accurate, clean source for master data to provide the dimensional source for analytical environments, and addresses the need to augment MDM operational services with in-line decision support analytics.

Analytical MDM

Source: (http://msdn.microsoft.com/en-us/library/bb410798.aspx)

Analytical MDM also enables accurate business intelligence, and allows accurate objects and structures to be automatically synchronized with data warehouses and analytic applications. Historically, data warehousing initiatives attempted to address data quality problems downstream from applications. Data warehousing does not fix the business processes that create inaccurate master data in the applications, nor does it correct the master data back in the applications. MDM gives businesses a way to correct bad data and the processes that create bad data at the source.


RECORDArrangementRequest

ANALYZECustomerRelationship

ANALYZEArrangementRequest

APPLYProductPolicy

APPLY CreditRating Scale

FORECASTArrangementRisk

OFFERArrangement


3.6 Possible Applications in the Industry

3.6.1 Where MDM can be used?First, let’s consider the distribution of unmanaged master data throughout a typical enterprise. Why are there multiple copies of master data? Are these copies redundant? This distribution may be viewed along any number of dimensions, including by line of business (LOB), by mergers and acquisitions, and by the introduction of packaged software.

1. A Cross-LOB PerspectiveLines of business (LOB) are the natural segmentations of responsibilities that form within an organization, especially where the organization carries a broad portfolio of products or services. By their nature, lines of business often have unique perspectives on core business.

Investment

Loans

Deposits

Lines of business often maintain their own business information

information, such as products (the products that this LOB offers), customer (the type of customer information that is important), and account (the nature of an ongoing relationship with a customer based on one or more products). For example, a line of business within a financial institution that is focused on deposits will usually carry different product information than a line of business focusing on investments. Similarly, the customer information that is important to a mortgage department is often different from the information that is important in the support of checking accounts. In reality, neighbouring lines of business experience varying levels of cohesion—some lines of business share a great deal of business information, while others share a good deal less. Within large organizations, lines of business frequently act as very different sub-organizations.

2. A Cross-Channel PerspectiveEach line of business may also have a number of distinct (distribution) channels to market. While these channels are often very similar, the resulting treatment of business information is often very different. For example, within a single line of business there are frequently entirely different solutions in place for attended channels (such as a branch office) and unattended channels (such as the Internet). These differences in customer interaction patternsacross channels often drive a perception that the problem space differs sufficiently to merit an entirely different solution. In other cases, increased complexity is caused by


evolution, with emerging channels adopting solutions that were simply not available when support for existing channels was defined.

Investments

Loans

Deposits

Partner Internet Branch Channel variance further scatters business information

There are, of course, valid differences in business information across channels—Internet only product offers, in-branch deals, and so on. The location of master data can also be influenced by the realization that core information can be shared across lines of business, particularly where the adoption of a new channel acts as a catalyst driving a common view of master data across that channel but not across related channels. The result can be a unification of master data across lines of business for some channels but not for others. For example, many enterprises strive to provide a single point of entry for customer self-service over the Internet. Even if a customer has five different kinds of accounts at a bank (managed by five different systems), the organization will still want to present a unified view of that customer relationship through this channel.

3. Mergers and Acquisitions: Mergers and acquisitions serve to dramatically accelerate the replication of business information within an enterprise. For example, consider a case where organization A is to merge with organization B. Both organizations maintain LOB-specific stores of master data however,

Organization A

Investments

Loans

Deposits Branch Internet


Organization B

Investments

Loans

Deposits

Branch Internet Two organizations with different patterns of data distribution.

Organization A shares information across lines of business within the Internet channel, while organization B has all but eliminated channel-specific perspectives on master data, and each line of business operates on the same data, regardless of the channel concerned. Merging these organizations yields a very different picture, however. For example, an organization may, after several years of geographic growth, realize that each region has independently created localized systems that contain partially replicated and overlapping sets of data, which has led to an incomplete and inconsistent view of its customers, suppliers, and products. Addressing these business problems can be viewed as a merger of the different geographically based organizations and systems.

Investments

Loans

Deposits

Branch Internet

The resulting merger often suffers from the worst of all inputs.


CHAPTER 4

4.1 Findings : We have found through this research as how MDM can be helpful in different scenarios and what are the different industries where MDM can be applied.These benefits are categorized using the same three principles we discussed earlier, that is, that an MDM System:

• Provides a consistent understanding and trust of master data entities• Provides mechanisms for consistent use of master data across the organization• Is designed to accommodate and manage change

We have found out a lot of ways how MDM can be implemented in industries,What are the advantages, strengths and limitations of implementing MDM.It describes the fundamental concepts of master data and MDM. We describe the key characteristics of a Master Data Management System and how the MDM System’s ability to manage master data provides benefits to the enterprise. We also introduce the multipleMDM methods and implementation styles. “MDM Architecture Patterns” provides an overview of architecture patterns often encountered in MDM deployments. We describe in detail the architecture patterns that helped to shape the MDM Reference Architecture. The architecture patterns encountered were either new architecture patterns, variations of existing architecture patterns, or known architecture patterns that were applied in the area of Master Data Management.

Chapter 1: We have selected a topic and how we have selected this, what is the need of this research, why organizations are going towards MDM, what are the objectives, what is the scope of this research and what is the methodology to go ahead with this research.

Chapter 2: This includes all the literature review which we have studied during the research progress and documented that.

Chapter 3: This chapter includes the Analysis of Problem under Research, what are the Alternative Solutions and their advantages & disadvantages and limitations, what is the Proposed Solution , what are the Technical Justification of the Solution which is suggested , what is the Technical Environment & Technical Details of the solution and where the solution can be applied , what are the Possible Applications in the Industry.

4.1.1 Limitations of this study:

This research does not cover many of the technical aspects of Master Data Management. It doesnot provide project planning management methodology or an MDM Solution Architecture based on software products. This does not involve “MDM as an SOA Enabler” which describes the relationship between MDM and Service-Oriented Architectures. This also does not demonstrate how MDM and SOA work together to help in the achievement of business and IT goals related to managing master data, and explain why we view MDM as an enabler for any SOA-style solution.


4.2 Recommendations These are the following recommendations before implementing MDM in an organization which needs to be taken care of otherwise instead of solving the problems, it can create havoc in the organization in the form of investing a lot of money and not getting the benefits which were intended by implementing MDM.

1. Start with the problem not the solution - Make sure MDM solves a key business problem and explain it as a business solution. Don't use the words MDM with business people!

2. Do an organizational change management assessment - Many techies scoff at this notion. This is critical to the success of large transformational projects. I highly recommend using a consultant because they can see things that those deeply rooted in the culture cannot.

3. Secure strong executive sponsorship - Find a champion on the business side or leverage your C-Level IT folks if they have the integrity and respect of the leadership team. They need push this initiative down through the ranks and remove all obstacles that interfere with the success of the initiative.

4. Don't cut out steps to save money - When I say don't do it on the cheap I am not saying you have to buy expensive tools. There are some great open source tools that may work for you. What I am saying is don't cut out steps, don't refrain from bringing in experts because they are expensive, don't skip architecture and planning steps, etc.

5. Partner with experts, acquire resources with the proper skills - Why reinvent the wheel? Hire an expert who has implemented MDM many times and leverage his/her expertise. Leverage some experienced consultants to work side by side with your people to get real on the job training. Hire some full time employees with MDM experience. If your budget does not allow for new hires, get rid of the dead weight or those resistant to change and replace them with people who have the necessary skills. Don't put all of your eggs in the consulting basket!

6. Put your best project manager(s) on the project - It still comes down to controlling scope, getting the right requirements, mitigating risks, and communicating effectively. This is one series of projects that you cannot afford to have fail.

7. Think of MDM as an evolution, not a project - Define the future state and break up the initiative into a series of manageable deliverables. I recommend an architectural assessment and well defined strategy as an early deliverable. Why does the first deliverable always have to be production ready code? Get a plan in place first!

8. Don't underestimate the complexity - Take time up front to fully grasp what you are about to undertake. There is nothing more embarassing than telling your users that you


need more time because you didn't anticipate numerous tasks. In many cases they won't care and will hold you to your original dates. Then you start to cut out steps (see #4).

9. Implement and adhere to governance - There is nothing worse than devoting huge amounts of time building out a great architectural solution, only to see its value degrade over time due to lack of control and standards. If you are not going to govern your MDM, why bother go through the effort?

10. Don't let vendors drive the architecture - There are plenty of wonderful tools in the Magic Quadrants and in the Wave, but that doesn't mean you need to buy them. You may already have everything you need. You may not need all of the features in commercial products and can find a reliable open source solution. Whatever you do, make sure you come to that decision by analyzing your business and technical requirements, evaluating tools based on these requirements, and performing a proof of concept before committing your precious capital. Don't let the Power Point slides blind you! Anybody can make a nice Power Point deck. Only a few tools will actually meet your needs at your price!

4.3 Conc lusion : High quality, integrated master data is critical to the success of many types of information systems today. However, implementing a master data environment that effectively supports business needs is cost effective and delivers value in a reasonable period of time poses significant challenges.MDM is a broad subject that touches on many of the concepts of enterprise informationarchitecture. MDM strives to untangle and simplify the complex systems that have evolvedto manage core business information by logically consolidating this information into managedyet flexible MDM Systems. Acting as either a system of record or a system of reference,MDM Systems can provide authoritative data to all enterprise applications.As we have described throughout the chapters, successful MDM Systems:• Provide a consistent understanding and trust of master data entities• Provide mechanisms for consistent use of master data across the organization• Are designed to accommodate and manage changeThese are the key principles of MDM.The business drivers behind MDM are compelling—from regulatory compliance to improvingthe responsiveness of an organization to change. By providing authoritative informationas a set of services, MDM is also a key enabler for broader enterprise strategies, such as SOA.


References and Bibliography:

1. Master Data Management Is Managing Your Enterprise Data Like Comparing

Apples to Oranges? (Dell Enterprises pdf)2. MDM_Productsheet_US.pdf3. www.informit.com/articles/article.aspx?p=12184214. msdn.microsoft.com/en-us/library/bb410798.aspx5. Master Data Management An Oracle White Paper November 20076. IBM Multiform Master Data Management: The evolution of MDM applications

(pdf)7. Master data management vision and value: Part 1 by IBM8. Master data management vision and value: Part 2 by IBM9. IBM Multiform Master Data Management: The evolution of MDM applications

pdf10. White paper on Best practices for a successful MDM implementation by Infosys11. Enterprise Master Data Management by IBM press pdf12. TIBCO’s Master Data Management Solution : Consistent Information

Everywhere13. Better Information through Master Data Management – MDM as a Foundation for

BI An Oracle White Paper September 201114. Master Data Management An Oracle White Paper November 200715. www.mdmsource.com/master-data-management-benefits16. www.mdmsource.com/master-data-management-quotes-by-analysts.html17. aminemekkaoui.typepad.com/business_intelligence/2007/09/four-ways-to-en.html


Final Dissertation

Documents

Transcript of Final Dissertation