Post on 11-Jun-2020
© 2003 Kimball Group All rights reserved.
GatheringRequirements for
EffectiveDimensional Modeling
GatheringRequirements for
EffectiveDimensional Modeling
Margy RossDec 17, 2003
© 2003 Kimball Group All rights reserved.
Kimball Group
ConsultingProject assessments and
strategyRequirements analysis
Dimensional modeling anddesign reviews
Technical architecture
margy@ralphkimball.com952.974.0404
EducationPublic and on-site classes
Dimensional ModelingData Warehousing
Lifecycle
www.kimballuniversity.com
“Definitive Source for the Dimensional Approach”
© 2003 Kimball Group All rights reserved.
Reference Materials
r Course materials adapted from...§ The Data Warehouse Lifecycle Toolkitè R. Kimball, L. Reeves, M. Ross,
W. Thornthwaite (Wiley 1998)
§ The Data Warehouse Toolkit, 2nd Editionè R. Kimball, M. Ross (Wiley 2002)
§ Kimball Universityè Design Tipsè Intelligent Enterprise articlesè www.KimballUniversity.com
© 2003 Kimball Group All rights reserved.
Session Agenda and Goals
r Agenda:§ Introductions§ Lifecycle Approach§ Dimensional Model Overview§ Gathering Requirements for Dimensional
Modeling
r Today’s Goals:§ Learn proven approach for gathering data
warehouse business requirements§ Learn specific concepts and practical
techniques
© 2003 Kimball Group All rights reserved.
Top 10 ReasonsData Warehouses Miss the Mark
q Focus on technology and data rather thanbusiness requirements and goals.
q Fail to secure business sponsorship.
q Pursue all-or-nothing approach rather thancompelling iterations.
q Spend time and energy building normalizeddata foundation, with nothing left forpresentation.
q Concentrate on operational performance anddevelopment rather than query performanceand ease of use.
© 2003 Kimball Group All rights reserved.
Top 10 ReasonsData Warehouses Miss the Mark
q Make the user view of the data too complex.
q Build isolated data silos (marts orwarehouses).
q Only load summarized data into the userpresentation area.
q Presume business requirements are static.
q Fail to recognize that the data warehouse’ssuccess is tied directly to user acceptance.
© 2003 Kimball Group All rights reserved.
BusinessDimensional
Lifecycle Approach
BusinessDimensional
Lifecycle Approach
© 2003 Kimball Group All rights reserved.
Business Dimensional Lifecycle
TechnicalArchitecture
Design
TechnicalArchitecture
Design
ProductSelection &Installation
ProductSelection &Installation
AnalyticApplication
Specification
AnalyticApplication
Specification
AnalyticApplication
Development
AnalyticApplication
Development
ProjectPlanningProject
Planning
Business
Requirement
Definition
Business
Requirement
Definition
DeploymentDeploymentMaintenance
andGrowth
Maintenanceand
Growth
Project ManagementProject Management
DimensionalModeling
DimensionalModeling
PhysicalDesign
PhysicalDesign
Data StagingDesign &
Development
Data StagingDesign &
Development
© 2003 Kimball Group All rights reserved.
DimensionalModeling Concepts
DimensionalModeling Concepts
© 2003 Kimball Group All rights reserved.
SimplifiedElements of Data Warehouse
Data StagingArea
Data Warehouse Bus:Conformed facts anddimensions
Data Warehouse Bus:Conformed facts anddimensions
Services:Transform from
source-to-targetMaintain conform
dimensionsNo user query
support
Data Store:Flat files or
relational tables
Design Goals:Staging
throughputQuality /
consistency
Services:Transform from
source-to-targetMaintain conform
dimensionsNo user query
support
Data Store:Flat files or
relational tables
Design Goals:Staging
throughputQuality /
consistency
PresentationArea
Ad Hoc Query &Analysis Tools
Standard Reports
AnalyticApplications:
ModelingForecastingScoringData mining
Ad Hoc Query &Analysis Tools
Standard Reports
AnalyticApplications:
ModelingForecastingScoringData mining
Data AccessTools
SourceSystems
Extract
Load
Extract
Extract
DimensionalAtomic AND
summary dataBusiness-process
centric
Design Goals:Ease-of-useQuery
performance
DimensionalAtomic AND
summary dataBusiness-process
centric
Design Goals:Ease-of-useQuery
performance
Access
© 2003 Kimball Group All rights reserved.
PRODUCT KEY
Product Desc.SKU #SizeBrand Desc.Class Desc.
Terminology: Dimensions
r Set of attributes (columns) related to asubject/object§ Who, what, when, where, why, how§ Product, Customer, Date, Patient, Vendor,
Facility, …r Each dimension row is a unique
occurrence§ One row per product, customer, day, …
r Dimension attributes:§ Report labels and query constraints§ “By” words and “where” clauses§ Verbose descriptive attributes, in addition to
codes§ Hierarchical relationships
© 2003 Kimball Group All rights reserved.
DATE KEYPRODUCT KEYSTORE KEYPROMOTION KEY
$ SalesUnit Sales
Terminology: Facts
r Result from a businessprocess or businessevent§ Facts are usually numeric
and additive
r Granularity/grain§ Identifies the fact level of
detail§ One row per sale,
one row per service call,one row per claim, …
§ Atomic grain is mostflexible
© 2003 Kimball Group All rights reserved.
Terminology: Dimensional Modelor Star Schema
r Fact table perbusiness process /event, plus relevantdimensions
r Benefits:
§ Easier to understand
§ Better performancefrom fewer joins
§ Extensible to handlechange
StoreAttributes
PromoAttributes
ProductAttributes
Product KEY Store KEY
Promo KEYFacts
Product KEYDate KEYStore KEYPromo KEY
DateAttributes
Date KEY
DimensionTables
DimensionTables
FactTable
© 2003 Kimball Group All rights reserved.
“Dimensions”“Dimensions”Report, row and column headings
“Facts”“Facts”Numeric report values
Sales Rep Performance ReportCentral Region
Jan 2003 Feb 2003Dollars Dollars
Chicago District Adams 990 999
Brown 900 999
Frederickson 990 999
Minneapolis District Andersen 950 999
Smith 950 999
Central Region Total 4,780 4,995
Sample Report Translationof Dimensional Model
© 2003 Kimball Group All rights reserved.
Terminology:Conformed Dimensionsr One row for each instance of a business
subject or object (product, customer, etc.)r All fact tables use same standard dimensions
§ Established via Bus Matrix, enforced in ETL
r Consistent – apples to apples acrossprocesses
ProductAttributes
Product KEY
Shipment Facts
Product KEY…
Shipment Qty
Order Facts
Product KEY…
Inventory Qty
Inventory FactsProduct KEY…
Order Qty
© 2003 Kimball Group All rights reserved.
Terminology: Enterprise DataWarehouse Bus Architecture
Store Inventory
Store Sales
Date Store Promo Distr.Center
Shipper VendorProduct
Purchase Orders
© 2003 Kimball Group All rights reserved.
Terminology:Data Warehouse Bus Matrix
r Rows = Business processesr Columns = Conformed dimensions
Date Product Store Promo Dist Ctr Shipper VendorStore SalesStore InventoryStore DeliveriesDist Ctr InventoryDist Ctr DeliveryPurchase Orders
© 2003 Kimball Group All rights reserved.
Creating Conformed Dimensions
r Assign surrogate key to every dimension row§ Buffer the DW from operational system changes§ Integrate data from multiple sources§ Query performance advantages§ Prerequisite for slowly changing dimension
r Combine all attributes intoMaster dimension table
r Use the Master tomap surrogatekeys to fact rows
Product CodeDescriptionBrandCategoryHeightWidthWeightStandard Cost
Product KEY
Marketing
Logistics
Cost Acctg.
© 2003 Kimball Group All rights reserved.
Dimensional Modeling Processr Develop the Data Warehouse Bus matrixr Follow the 4-step method
§ Step 1: Identify the business process (matrixrow)
r Research data sources, including direct dataexploration
r Complete the 4-step method§ Step 2: Declare the grain§ Step 3: Identify the dimensions§ Step 4: Identify the facts
r Diagram the dimensional model
© 2003 Kimball Group All rights reserved.
Key Input to Dimensional Model
DimensionalModel:
Business ProcessGrain
DimensionsFacts
BusinessRequirements
DataRealities
© 2003 Kimball Group All rights reserved.
Gathering BusinessRequirements
Gathering BusinessRequirements
© 2003 Kimball Group All rights reserved.
ProjectScope
BusinessRequirements
Maintenance andGrowth
ApplicationSpecification
TechnicalArchitecture
Deployment Plan
PhysicalDesign
DimensionalModel
Data StagingDesign
Importance ofBusiness Requirements
© 2003 Kimball Group All rights reserved.
Recommended Approach forUncovering Requirements
r Start with business users to understand...§ Business objectives§ Information/analysis themes§ Decision-making process
r Interweave data “reality” meetings withsource system experts or DBAs§ Ability to support user themes§ Data gotchas
© 2003 Kimball Group All rights reserved.
Techniques forUncovering Requirements
r Interviews: All voices heard: Less interviewee time required
r Facilitated Group Sessions: Less elapsed time, but more participant time0 Limited # participants: Brainstorming, consensus and issue resolution
r Interviews AND Facilitated Groups
© 2003 Kimball Group All rights reserved.
Preparation:Research and Team Roles/Tools
r Don’t presume you already know it all§ Research available resourcesè Annual report, marketing literature, web content
§ Organization chart§ Previous warehousing initiatives
r Identify interview team roles§ Lead interviewer, scribe and observers
r Prepare interview questionnaire§ Specific to interviewee level and function§ 1-page fallback device, not script
© 2003 Kimball Group All rights reserved.
Preparation:Select Interviewees
r Horizontal user representation (breadth)§ Understand common vocabulary and data across
core functions§ Critical to Data Warehouse Bus framework
r Vertical user representation (depth)§ Vision à tactics; full perspective in target area§ Executives, managers and analysts
r Preliminary IT representatives§ Source experts, DBAs, IT liaisons and IT mgmt
© 2003 Kimball Group All rights reserved.
Preparation:Schedule/Prepare Intervieweesr Scheduling logistics:
§ Solicit administrative help§ Individual sessions for execs§ Individual or small groups for othersè Max 2 - 3 people; homogeneous; < 2 levels
§ Maximum 3 - 4 interviews per dayè One hour for individuals; 1 1/2 hour for groupsè Minimum 1/2 hour between interviews
r Prepare users:§ Conduct User Kick-off Meeting - “ownership”§ Follow-up with pre-interview letterè Bring copies of key analyses
© 2003 Kimball Group All rights reserved.
Interview Flow:Introduce the Interview
r Set the stage…§ Review project and interview objectives§ Introduce team players and roles§ Confirm time
r Establish overall tone§ Practice in advance§ No technical jargon; No phones or pagers
r Get users to talk about what they do§ Job responsibilities and organizational fit?
© 2003 Kimball Group All rights reserved.
Business ExecutiveInterview Contentr Big-picture understanding and vision
§ Objectives of your organization? What are youtrying to accomplish?
§ How do you measure success? How do you knowdoing well? How often measured?
è Identify key business processes and facts
§ Key business issues? Vision for organization?
§ Opportunities to better leverage information withinorganization? Impact on business?
è Identify expectations and business benefits
© 2003 Kimball Group All rights reserved.
Business Manager/AnalystInterview Contentr Similar to Business Exec, but more detailed
§ Objectives of your department? How measuresuccess? Key metrics? How often?è Identify key business processes and facts
§ How distinguish between products? Natural waycategorize products? How narrow list of products?è Identify dimension attributes and hierarchies
§ Types of routine/ad hoc analysis performed? Howoften? Improvements to current methods?è Identify data access tool requirements &
application templates
§ Opportunity to impact business with improvedaccess to information? Financial impact?
© 2003 Kimball Group All rights reserved.
Business AnalystReport / Spreadsheet Review
r Understand current analyses andopportunities for improvement§ What data on the report is important?§ How do they use the report today?§ If report were dynamic, what’s different?
r Remember: Designing analyticenvironment, not reporting system§ Resist temptation to focus on top five reports
© 2003 Kimball Group All rights reserved.
IT Data AuditInterview Contentr Feasibility of supporting requirements
§ Overview key operational source system
§ Update frequency? Availability of historical data?§ Known data caveats or quality issues?è Identify data availability and gotchas
r Dig until confident that core data exists§ More detailed data analysis will follow for
dimensional modeling
§ Beware of classic “profitability” data trap
© 2003 Kimball Group All rights reserved.
Interview Flow:Wrap-Up
r Ask about project success criteria§ What is the #1 thing the project must accomplish
to be deemed successful?è Identify measurable success criteria
r Review next steps§ General disclaimer to manage expectations§ Deliverables and next feedback opportunity
r Thank for participation
© 2003 Kimball Group All rights reserved.
r Remember interview role§ Do LISTEN; don’t defend, sell, try to impress, etc.
r Establish peer basis§ Use their vocabulary§ Don’t dive too quickly
r Strive for conversational flow
r Verify communication§ Capture terminology precisely
r Maintain interview schedule flexibility§ Can add new folks, but avoid interview burn-out
Interview Flow:Ground Rules
© 2003 Kimball Group All rights reserved.
Post-Interview:Review Interview Results
r Informally debrief with team§ Common themes§ Do-ability§ Areas requiring clarification§ User analytical / technical sophistication
r Fill-in rough interview notes§ Highlight key points and vocabulary
© 2003 Kimball Group All rights reserved.
Post-Interview:Publish Deliverables
r Don’t overlook formal documentation§ Validation§ Reference material
r Individual interview write-ups§ Summary, not transcriptè Business Objectivesè Analytic and Info Requirementsè Project Success Criteria
r Consolidated findings document
© 2003 Kimball Group All rights reserved.
Post-Interview:Publish Deliverables cont’d
Consolidated Requirements Findings
l Executive Overview
l Project Overview
l Business Requirementsl Business Process Opportunities (matrix rows)
q Description
q Typical questions
q Data feasibility
l Preliminary Data Warehouse Bus Matrix
l Success Criteria
© 2003 Kimball Group All rights reserved.
Low
High
High
Low Feasibility
PotentialBusiness
Impact
C A
DE
Post-Interview:Facilitation for Next Stepsr Session with Business
and IT management§ Confirm findings§ Get commitment§ Prioritize
r For each requirement§ Business impact / value§ Feasibility
r Outcomes:§ “Right” opportunities§ Consensus§ Ownership§ Roadmap for growth
B
© 2003 Kimball Group All rights reserved.
Linking Data Warehouse BusMatrix and Prioritization Grid
PotentialBusiness
Impact
Low
High
High
Low Feasibility
C A
DE
B
Dim1 Dim2 Dim3 Dim4 Dim5Biz Proc ABiz Proc BBiz Proc CBiz Proc DBiz Proc E
Data Warehouse Bus Matrix
© 2003 Kimball Group All rights reserved.
Tips for Handling Obstacles
“Abused User” è Review documentation“Validate” earlier inputUse alternative forum
“Overbooked User” è Get sponsor’s helpDefer, if prevalent
“Comatose User” è Don’t prolong agonyFind replacement
“Overzealous User” è Assess homogeneityReschedule sessions
“Non-Existent User” è Typically fatal - STOP!
© 2003 Kimball Group All rights reserved.
Business RequirementsWarning Signs§ “We already understand the requirements -- better than
the users do.”
§ “We’ll send out a survey.”
§ “We’ll just look at their current reports and data files tounderstand users’ requirements.”
§ “Since each interview lasts one hour, we’ve scheduled2 days of interviews with 8 interviews per day.”
§ “We don’t have time to document the interviews.”
© 2003 Kimball Group All rights reserved.
Business RequirementsSummary
r Understanding business requirements isCRITICAL to successful data warehousing
r Don’t overlook the up-front preparation
r Focus on listening
r Document what you’ve heard
r Close the loop following requirements