Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line...
Transcript of Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line...
![Page 1: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/1.jpg)
Steps Towards Business Intelligence
Ahsan Kabir ,MVPChapter Leader “techforum PASS”
![Page 2: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/2.jpg)
What is BI
Business Intelligence is an umbrella term thatincludes the applications, infrastructure andtools, and best practices that enable decisionmakers to make proper decisions.
![Page 3: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/3.jpg)
• What happened?
• What is happening?
• Why did it happen?
• What will happen?
Past
Present
Future
“Understand the pulse of the
Organization”
Why BI
![Page 4: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/4.jpg)
How BI ?
![Page 5: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/5.jpg)
Microsoft BI Technologies
![Page 6: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/6.jpg)
What is DW?
“…is designed specifically to be a central repository for all data in a company separated from transactional systems.”
“…is designed to be the source of analysis and reports.”
“But it’s not a copy of a source database.”
![Page 7: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/7.jpg)
Why DW
• Central Repository
• Reduce extra load
• sources unaffected
• Empower Business Users
• Improve data quality
• Single version of the truth
![Page 8: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/8.jpg)
1) Data volumes,
2) Real-time data,
3) New sources and types of data, and
4) Cloud-born data
But …..
![Page 9: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/9.jpg)
Now
• The data warehouse is unable to keep up with explosive volumes.
• The data warehouse is falling behind the velocity of real-time performance requirements.
• The data warehouse is slower than desired in adopting a variety of new data sources, slowing time–to-value
• The platform costs more, while performance lags.
![Page 10: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/10.jpg)
Planning
1. Analytical and Report requirement
2. Business Process
3. Prioritization
4. Identify Source Data
5. Dimensional Model
6. Documentation
7. Design Data Warehouse
![Page 11: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/11.jpg)
Data Warehouse vs.Data Mart
Data Warehouse
Enterprise-wide
Union of all data marts
Data Mart
Departmental or Business line
Single business process
![Page 12: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/12.jpg)
Kimball
Bottom-Up
Data marts
Logical data warehouse
Decentralized
Quick results, iterative approach
Inmon
Top-Down:
Enterprise data model
Centralized
Later create data marts
More upfront work but less redo
Kimball vs. InmonMethodology
![Page 13: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/13.jpg)
Data Model
OLAP cube /Multidimensional modeling :
“…is based on the OLAP cube and is fitted with measures and dimensions”
In-memory tabular model:
“…is based on a new In-memory engine for tables “
![Page 14: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/14.jpg)
OLAP cube/Multidimensional modeling
Fact or measure
“… are numeric and additive values “
Foreign keys
Dimension
“…Descriptive information”
Surrogate key.
Business key.
![Page 15: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/15.jpg)
SSAS Loaded into in-Memory engine called xVelocity in-memory.
Tabular modeling allows you to create a table-based model from existing data in Warehouse and and create a relationship between models.
Data Analysis Expression (DAX) is an expression language for SSAS Tabular, which helps you create calculations and measures based on existing columns and relationships.
Tabular Model
![Page 16: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/16.jpg)
Schema Design
The layout indicate the relationship between facts and dimensions is called a schema.
Star Schema :
For each fact entity join with single level of dimension entities.
Snowflake Schema :
If there are dimensions with large numbers of attributes, it might be necessary to break the dimensions down into sub dimension entities
Star Schema
Snowflake Schema
![Page 17: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/17.jpg)
DEMO
![Page 18: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/18.jpg)
Analysis Services (SSIS)
1. Develop Cubes and 2. Create dimensions and measures.3. Creating hierarchies4. MDX queries will be compiled,
parsed, and executed in the SSAS engine
![Page 19: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/19.jpg)
ETL
“…is a program that periodically runs.”
Extract
Fetching data from the source
relational databases, web services, and
SharePoint lists.
Transform
“..Cleansing the data and converting to a
OLAP-friendly data model”
Load
“..loading data into the data warehouse
as fact and dimensions”
![Page 20: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/20.jpg)
Demo
![Page 21: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/21.jpg)
Data is kept in a
specific business
line wise.
Before enter into warehouse
Data is processed
(cleansed and transformed)
Warehouse Data Marts
Users query
the data
warehouse
“…staging area is an area where we fetch data from different sources exactly as it is into our integrated database. “
Staging
![Page 22: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/22.jpg)
Data Quality Services
Data quality issues can be divided into the following categories:
Uniqueness
Validity
Accuracy
Standardization
Completeness
Name Address City House No DoB State Country
Ahsan CDA Avenue CTG 181/1 05/11/1978 BD
Kabir RB Avn CTG 41/6 23/04/1991 DHK Bangladesh
Before
After
Accuracy Consistency Completeness Conformity
Name Address City House
No
DoB Stat
e
Country
Ahsan CDA Avenue CTG 181/1 05/11/1978 CT Bangladesh
Kabir RB Avenue DHK 41/6 23/04/1991 DHK Bangladesh
![Page 23: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/23.jpg)
Start DQS
Knowledge Base Management
Knowledge Base Management is where you can create and manage Knowledge Base, domains, and domain rules
Data quality projects
projects apply the Knowledge Base and matching rules on an existing dataset and provide results.
Administration
Configuration and administration tasks can be performed here
![Page 24: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/24.jpg)
Components in DQS
1. Cleansing,
Cleansing is about cleaning data based on a Knowledge Base and domains.
2. Matching,
Matching would match data based on the similarity rules and threshold defined in a Knowledge Base.
3. Monitoring
Monitoring will show the status of records during the cleansing and matching projects.
4. Profiling.
Profiling will help in creating business rules or changing the domain rules and Knowledge Base from what the existing data profiling results are.
![Page 25: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/25.jpg)
Demo
![Page 26: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/26.jpg)
Technology
SSDT
“…is the integrated IDE for SSIS, SSRS, and SSAS. SSDT was formerly known as Business Intelligence Development Studio (BIDS). “
SSIS
SSIS was released with this name for the first time in 2005, but prior to that, it was named Data Transformation Services (DTS). DTS was available even in SQL Server 2000
![Page 27: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/27.jpg)
SSRS
“is a data Visualization tools for
developing and publishing reports”
ReportServer DB
Report definition,
Snapshot,
Execution log etc.
ReportServer TempDB
Session and
Cached information.
Report Server web application
Report Manager web application
Reporting Services Configuration Manager.
![Page 28: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/28.jpg)
Master Data Service (MDS)
“…is data shared across computer systems in the enterprise.”
“… is the dimension or hierarchy data in data warehouses and transactional systems”
“… is core business objects shared by applications across an enterprise
-The processes and technology to produce and maintain a single clean copy of master data
Customer
ABC
PQR
XYZ
Country
Europe
Norway
Sweden
![Page 29: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/29.jpg)
Features
Domain management
models, entities, attributes, and hierarchies.
Business rules
Data validation is also provided.
Import and export master data
Data cleanup
![Page 30: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/30.jpg)
Architecture
SQL Server database for storing data and metadata.
MDS engine read and write information to that database by : WebUI and Excel Add-ins.
MDS uses subscription views to export information from MDS to other systems
Staging mechanism to import data from other systems, which is called entity-based staging.
![Page 31: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/31.jpg)
Demo
![Page 32: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/32.jpg)
Resources:Data Warehouse Architecture – Kimball and Inmon methodologies: http://bit.ly/SrzNHy
SQL Server 2012: Multidimensional vs tabular: http://bit.ly/SrzX1x
Data Warehouse vs Data Mart: http://bit.ly/SrAi4p
Fast Track Data Warehouse Reference Guide for SQL Server 2012: http://bit.ly/SrAwsj
Complex reporting off a SSAS cube: http://bit.ly/SrAEYw
Surrogate Keys: http://bit.ly/SrAIrp
Normalizing Your Database: http://bit.ly/SrAHnc
Difference between ETL and ELT: http://bit.ly/SrAKQa
Microsoft’s Data Warehouse offerings: http://bit.ly/xAZy9h
Microsoft SQL Server Reference Architecture and Appliances: http://bit.ly/y7bXY5
Methods for populating a data warehouse: http://bit.ly/SrARuZ
Great white paper: Microsoft EDW Architecture, Guidance and Deployment Best Practices:
http://bit.ly/SrAZug
End-User Microsoft BI Tools – Clearing up the confusion: http://bit.ly/SrBMLT
Microsoft Appliances: http://bit.ly/YQIXzM
![Page 33: Step towards Business Intelligence › Steps-Towards... · Data is kept in a specific business line wise. Before enter into warehouse Data is processed (cleansed and transformed)](https://reader035.fdocuments.us/reader035/viewer/2022081611/5f038c3d7e708231d409983f/html5/thumbnails/33.jpg)
Thanks