Meta data driven edw development with information server at ericsson some reflection from the real...
-
Upload
ibm-sverige -
Category
Data & Analytics
-
view
307 -
download
0
description
Transcript of Meta data driven edw development with information server at ericsson some reflection from the real...
![Page 1: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/1.jpg)
Meta Data Driven EDW Development with Information Server at Ericsson
- Some Reflection from the Real World
Anders Bergdahl
![Page 2: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/2.jpg)
Affecto, Leading EIM in the Nordics
Affecto has1100 Employees, 160 in Sweden.
133 MEUR revenue, 10,5 MEUR in profit
Four competence areas:
• DM/IM
• DW/BA
• CPM
• CBI EIM
2
![Page 3: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/3.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 3
› BI & Analytics is a set of capabilities that enable and support business processes with operational, tactical and strategic insights.
› BI & Analytics provides current, historical, predictive and prescriptive views of business operations.
Ericsson BI & Analytics Description
![Page 4: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/4.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 4
EDW 2.0
1. EDW 2.0 is the Next Generation DataWarehouse for Business Intelligence and Analytics at Ericsson.
2. EDW 2.0 follows LEDW(Logical Enterprise DataWarehouse) architecture pattern. It combines the strengths of traditional repository warehouse(EBW) with data virtualization, in-memory data processing and distributed processing.
3. Main features for EDW 2.0 are agility and lower cost/demand, improved reuse, data quality measurement, better data quality delivered faster, Reduced data latency, alignment with Ericsson global process and Enterprise Information model.
4. EDW 2.0 enables all information delivery and use scenarios of Standard reporting, ad-hoc reporting, visual exploration, advanced analysis and collaboration.
![Page 5: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/5.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 5
EDW2.0 Features
1. Harmonized and consistent definitions for measures, KPIs and dimensions.
2. Alignment of analytics data modeling with enterprise business process models and enterprise information models.
3. Established ownership of analytics products.4. Agreed process for handling cross dependencies
between different analytics products.5. Agile development process.6. Data quality measurements and reporting.
![Page 6: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/6.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 6
EDW2.0 Features
7. Information lifecycle management and classification of data as hot warm or cold.
8. Seamless transition from consolidated cross process analysis to localized detailed reporting on operational data.
9. Access by business process roles – improved information delivery channels.
10.Support for different data latency needs of near real time, several times a day, daily, weekly, monthly and yearly snapshots.
![Page 7: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/7.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 7
EDW2.0 Features
11.Search on metadata to find the reports for a given set of measures and dimensions, documentation, usage instructions, roles required, contacts for further instructions or change requests.
12.Usage monitoring of reports and data to identify unused content and usage patterns.
13.Test environment with production like data.
![Page 8: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/8.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 8
EDW2.0 Design Principles
› Extract once and deploy many.– Reduce load on operational systems and keep data synchronized.
› Reuse.– Reuse data, definitions, business rules, programs.
› Data near processing.– Move algorithms to data and not data to algorithms
› Read less, move less– Design to ensure less data is read and less data is moved between
processing units and layers.
![Page 9: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/9.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 9
EDW2.0 Reference architecture
![Page 10: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/10.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 10
Access Control
Data Integration (Structure, Unstructure, Mutiple S ource, Log Processing etc.)
Com
mon Inform
ation Model (C
IM) &
Business G
lossaryITK Web
Logs
Business Process/Applications (Tools & Sources Syte ms)
ONE HRMS
ITM
PRIM
GSSO
CRM360 Verdi SRM
SMA
BarTrack
Test net CRM COPAELIS
IBMS Feature Store
SMS
MHWeb
PQATCloud
Web Clients, Mobile, Desktop Clients,
Planning & Simulation Statistical AnalysisCollaboration BI
Standard Reporting
Ad Hoc Reporting
Self Service Federation/VirtualizationBroadcastingDATA PRESENTA
TION +VISUALIZA
TION
USER PROVISION
ING + ACCESS CONTROL
ENTERPRISE
DW
INTEGRATION
DATA SOURCES (EXAMPLES
)
DATA MANAGEME
NT+
DATA ARCHITECT
URE
FINANCE SUPPLY and LOGISTICS
Common Dimension / Master Data (3600 View)
Cross Data Analytics (E2E View)
EDW2.0 Capabilities MapA
ligned Processes &
Data
ILM D
efinitions
Definitions &
D
escriptions
Cross F
unctional Mapping
OPCM
Alignment Acquisition Real Time Change Data Capture Quality Governance
Profiling Cleansing Transformation Enrichment
MARKETING and SALESHR
PRODUCT MGMT PRODUCT DEV SERVICE DELIVERY IT & COMMON FUNCTIONS
![Page 11: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/11.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 11
Functional Capability AREAS
Product Development
Marketing & Sales
Supply & Logistics
Resource Management
Communications
Sourcing
Product Management
HR & O
Finance
IT
Security
Service Delivery
Operational Development
Real Estate
Technology and Research
![Page 12: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/12.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 12
LINKs to the Analytics ROADMAPS for CROSS PROESS VALUE FLOWS
O2C – Order to Cash
T2C – Tender to Cash
I2R – Incident to Resolution
Q2C – Quote to Cash
N2S – Need to Solution
O2S – Opportunity to Solution
O2F – Order to Fulfillment
![Page 13: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/13.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 13
Technical Capability AREAs
Technology and Data Platform
Development Frameworks
COE
Operations Support
![Page 14: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/14.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 14
HANASAPBW
IBM Neteeza
Microsoft SQL Server
(PDW)Hadoop
EDW2.0 – multiple technology environment
ITK
Business Process/Applications (Sources Sytems)
ONE HRMS
ITM
PRIM
GSSO
CRM360
OPCM
Verdi MDM
SMA
BarTrack
Test net ELIS
IBMS Feature Store
SMS
MHWeb
PQAT Manual
BI Semantic & Information Delivery
Logical Enterprise Data Warehouse
Data virtualization for low volume joins
Physical data movement from sources
Physical data movement across DW platforms
CDW Part Int Sol
Sandbox CDW Part Int Sol
SandboxCDW Part Int Sol
Sandbox
![Page 15: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/15.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 15
LEDW
› LEDW stands for “Logical Enterprise Data Warehouse” and is an enabler for an common information layer that can service the whole enterprise
LEDW
Common Data Warehouse (CDW)
An enterprise wide repository of reusable information objects,
integrated across business processes & sources, stored
in a unified schema
Partially Integrated Solutions
Can consume data from CDW but will not share data to CDW.
Sandbox
Temporary workspace for pilots / poc
![Page 16: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/16.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 16
Data Management Platforms
SAPBW
• Maintainence of existing solutions• Not to be used for new solution areas
HANA
• Primary platform for EDW2.0• Data originating from SAP• Data which should be combined with SAP data• Complex R&A Support• Operational Reporting• Master data hub• Platform for high performance in memory
data discovery exploration and visualization
![Page 17: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/17.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 17
Data Management Platforms
Netezza
MS PDW
• Very high volume data• Platform for advanced analytic usage • Use case driven usage
• Very high volume data• Use Case driven usage
Hadoop
• High volume, velocity and variety of data processing • More then 100 TB / year of data volume needed• Use Case driven usage
SQL Server
• Low volume data• Stand alone application usage
![Page 18: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/18.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 18
content
› LEDW introduction› LEDW development process
![Page 19: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/19.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 19
› Decided on IIS late 2012
› Started to develop a framework spring 2013– Support from IBM and IBM partners– Need to adapt to internal standards and ”traditions”– Big company, many views..
› Continuous work, first iteration available for use…– Still waiting for first solution in full production
The task.. In brief
![Page 20: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/20.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 20
EDW development - aligned to the Need 2 solution WORK flow
DescribedSolution
Realized Benefit
AcceptedDelivery
ClarifiedNeed
Phases where EDW solutions is defined, developed, tested and deployed
![Page 21: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/21.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 21
Business representative
UnderstandInformation model
Described solution WORK flow
Define Described Solution deliverables
Clarified Need
Describe Solution activity plan
Understandprocesses model
Model Logical Solution
Evaluate and DescribeSolution
ProposedSolution
SpecifyRequirements
PerformBusinessFulfillment Analysis
Business Case
N2S Decision
Point
N2S Decision
PointDescribed Solution
N2S Decision Points (as part of the value flow)
SODRequirements defined
Process models
LogicalInformationModels
Data quality analysis
Affected Capabilities
Prototype
Logical Solution
Source discovery & data quality assessmment
GAP analysis
DescribedSolution
Realized Benefit
AcceptedDelivery
ClarifiedNeed
TPM
Solution Area functional expert
Data modeler
SOD
Update business glossary
![Page 22: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/22.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 22
Accepted deliverywork flow
Develop Solution
DescribedSolution
Plan Delivery
DeliveryAssignments
UATApproval
ApprovedCut-Over
Plan
Solution In Operation
N2S Decision
Point
Approved Solution
Develop Business Solution
Define Realize Solution
Deliverables
Realize Solution
Activity Plan
N2S Decision Points
Agreed Plan for Realize Solution
DescribedSolution
Execute on Business Fulfillment
Business Case Report
Business Preparedfor New Solution
Updated Benefit Realization
Plan
N2S Decision
PointAcceptedDelivery
Implement Solution in Business
Solution In BusinessOperation
DescribedSolution
Realized Benefit
AcceptedDelivery
ClarifiedNeed
Develop Solution
PrepareRelease
Deploy Solution
LEDW development flow
Step1
Step 2
Step 3
Step 4
![Page 23: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/23.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 23
1. Plan delivery› Select delivery approach
– Development Project Delivery Model (see IT Project Framework following ePROPS)
– Established Capability Delivery Model (ECDM)– CDW development carried out via Establised capability model– Non integrated solution can be delivered via Development project model
› Set up a plan for the delivery– Project delivery Model (IT Project framework)– Established capability DM
› Backlog prioritization› Sprint planning
› Plan the releases– Monthly & weekly releases
Weekly Weekly Weekly Weekly Weekly WeeklyMonthly Monthly
![Page 24: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/24.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 24
2. Develop solution
Backlog
Sprint 1
Backlog
Sprint 2
Backlog
Sprint 3
Sprint 1
Sprint 2
Sprint 3
Production deploymentRelease R14-01
Production deploymentRelease R14-02
QA
QA
QA
QA
Production deploymentRelease R14-03
Sprint 4
Production deploymentRelease R14-04
Backlog
Sprint 4
QA Executed by LEDW release team
SprintExecuted by LEDW development project or established capability
![Page 25: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/25.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 25
› Detailed requirements› Design & prototype solution› Build solution & unit test› System integration test› Performance test› User acceptance test› Document solution
2. Develop solutionETLData model
SecurityInformation delivery (BI)
![Page 26: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/26.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 26
Accepted deliveryWORK flow
Develop Solution
DescribedSolution
Plan Delivery
DeliveryAssignments
UATApproval
ApprovedCut-Over
Plan
Solution In Operation
N2S Decision
Point
Approved Solution
Develop Business Solution
Define Realize Solution
Deliverables
Realize Solution
Activity Plan
N2S Decision Points
Agreed Plan for Realize Solution
DescribedSolution
Execute on Business Fulfillment
Business Case Report
Business Preparedfor New Solution
Updated Benefit Realization
Plan
N2S Decision
PointAcceptedDelivery
Implement Solution in Business
Solution In BusinessOperation
DescribedSolution
Realized Benefit
AcceptedDelivery
ClarifiedNeed
Develop Solution
PrepareRelease
Deploy Solution
LEDW development flow
1 2 3 4
![Page 27: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/27.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 27
3. Prepare release
PROD
Test Domain Prod Domain
NZ HANA PDWNZ HANA PDW
Data integration Data integration
System runnning production
Copy of production(reduced fact tables)Regression testUAT
Controlled by LEDW release team only
System testUser testSIT
Accessible by project and established capability resources
DevelopementEnvironmentUnit test
DEV TEST QA
ReleaseHandover
Landing area Landing area
Data integration
DEV TEST DEV TESTDEV TEST QA PROD QA PROD QA PROD
![Page 28: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/28.jpg)
EDW2.0 Summary | Ericsson Internal | , Rev | 2014-03-10 | Page 28
4. Deploy solution
NZ PDW
Data integration
Releases deployed by LEDW release team only
QA Prod QA Prod
Source Source Source Source Source
Landing area
PRODQA
Release R14-01 Release R14-01
Prod Domain
Data integration
![Page 29: Meta data driven edw development with information server at ericsson some reflection from the real world](https://reader036.fdocuments.us/reader036/viewer/2022081403/554dcc47b4c905c2488b54ef/html5/thumbnails/29.jpg)