Decision Suppot System
-
Upload
paul-seiler -
Category
Technology
-
view
570 -
download
0
description
Transcript of Decision Suppot System
“Not everything that counts !can be counted.!And not everything that !can be counted - counts.”
Albert Einstein
Keep in Perspective
“Not everything that counts !can be counted.!And not everything that !can be counted - counts.”
Keep in Perspective
How important is a DSS?
How important is a DSS?
53,800 employees
Imagine the CEO of a large enterprise with:
How important is a DSS?
53,800 employees2,600 branch offices
Imagine the CEO of a large enterprise with:
How important is a DSS?
53,800 employees2,600 branch offices$6.4 billion annual budget
Imagine the CEO of a large enterprise with:
How important is a DSS?
53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers
Imagine the CEO of a large enterprise with:
How important is a DSS?
53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers
Imagine the CEO of a large enterprise with:
What strategic information might this CEO expect to be available?
How important is a DSS?
53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers
Imagine the CEO of a large enterprise with:
What strategic information might this CEO expect to be available?
Would an 18 month delay in finding out how many employees
left the company be OK?
How important is a DSS?
53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers
Imagine the CEO of a large enterprise with:
What strategic information might this CEO expect to be available?
Finding out how a customer performed on an evaluation -
six months later?
How important is a DSS?
53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers
Imagine the CEO of a large enterprise with:
What strategic information might this CEO expect to be available?
Not knowing the location or the age of the technology in your
branch offices?
How important is a DSS?
53,800 employees2,600 branch offices$6.4 billion annual budget758,000 customers
Imagine the CEO of a large enterprise with:
What strategic information might this CEO expect to be available?
We find ourselves in the Information Age with an aging
information system
What are the needs of the educational community?
Current Education Data Sets
Student
Performance
Personal
Finance
Infrastructure
Foreign
Current Education Data Sets
Student
Performance
Personal
Finance
Infrastructure
Foreign
Current Education Data Sets
Student
Performance
Personal
Finance
Infrastructure
Foreign
Promotes narrow decisions based on information extracted from one or two functional data-sets
(finance and assessment)
Current Education Data Sets
Student
Performance
Personal
Finance
Infrastructure
Foreign
Redundant data entry is commonDisconnected data increases resources needed
Collections become costly and inefficient
Educational Data WarehousingPerformance TID (10 digit)NCLB mathNCLB read. . . . . . . . AP score
PSAT math. . . . . . . . . . . . . . . . ACT enroll
Student TID (10 digit)Teacher SS#LEA Number. . . . . . . .
Stud genderStu Grade lvl
Stu FTE. . . . . . . .
Admin Unit No.
School Infrastructure Admin Unit No.
. . . . . . . . Technology
Crime/Safety. . . . . . . . . . . . . . . . Bld Age
. . . . . . . . Title I
Supply IHE Unit No,. . . . . . . .
SS#. . . . . . . .
IHE Endorsement
Finance LEA Number. . . . . . . . Per PupilTotal Rev. . . . . . . . Avg Salary
Operating Bdgt. . . . . . . .
Gov Data Admin Unit No.
. . . . . . . . Live BirthsGPS systemNum ArrestsCong Dist
Foreign Data Admin Unit No.
. . . . . . . . Employment
NCESUniversity
PersonnelAdmin Unit No.
. . . . . . . . Teacher SS#
Teacher Assign. . . . . . . .
Type of Cert. . . . . . . .
Cert Exp Date
Data Partnerships
Educational Data WarehousingPerformance TID (10 digit)NCLB mathNCLB read. . . . . . . . AP score
PSAT math. . . . . . . . . . . . . . . . ACT enroll
Student TID (10 digit)Teacher SS#LEA Number. . . . . . . .
Stud genderStu Grade lvl
Stu FTE. . . . . . . .
Admin Unit No.
School Infrastructure Admin Unit No.
. . . . . . . . Technology
Crime/Safety. . . . . . . . . . . . . . . . Bld Age
. . . . . . . . Title I
Supply IHE Unit No,. . . . . . . .
SS#. . . . . . . .
IHE Endorsement
Finance LEA Number. . . . . . . . Per PupilTotal Rev. . . . . . . . Avg Salary
Operating Bdgt. . . . . . . .
Gov Data Admin Unit No.
. . . . . . . . Live BirthsGPS systemNum ArrestsCong Dist
Foreign Data Admin Unit No.
. . . . . . . . Employment
NCESUniversity
PersonnelAdmin Unit No.
. . . . . . . . Teacher SS#
Teacher Assign. . . . . . . .
Type of Cert. . . . . . . .
Cert Exp Date
Data Partnerships
andProvides for a common set of definitionsBecomes the sole source of reusable dataImproves timeliness and utility of reports
a location that:Integrates information from disparate
systems into a total view and a common foundation for understanding student performance and school improvement
What is a Data Warehouse?
What is a Data Warehouse?DW is not just storage but the
tools to query, analyze and present information on the web.
What is a Data Warehouse?DWs have many definitions - with these similarities:
Subject oriented - gives information about a person instead of operations.
Integrated - a variety of sources are merged into a whole.
Non-volatile - provides users with a consistent picture over specified time periods.
Robust architecture - that allows concurrent access by a multiple number of users with frequent queries.
Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.
What is a Data Warehouse?DWs have many definitions - with these similarities:
Subject oriented - gives information about a person instead of operations.
Integrated - a variety of sources are merged into a whole.
Non-volatile - provides users with a consistent picture over specified time periods.
Robust architecture - that allows concurrent access by a multiple number of users with frequent queries.
Quality data - valid and reliable data that promotes confidence in DW and forms the nucleus of information used by the educational community.
What is a Decision Support System?
DSS is a process used by the educational community (with support
of the data warehouse) that transforms data into a knowledgebase
that will support decision-making.
What is a Decision Support System?
DSS starts with a:Problem +
administration =
Data
What is a Decision Support System?
DSS starts with a:Problem +
administration =
Data + dissemination =
Information
What is a Decision Support System?
DSS starts with a:Problem +
administration =
Data + dissemination =
Information+ social
discussion =
Knowledge
What is a Decision Support System?
DSS starts with a:Problem +
administration =
Data + dissemination =
Information+ social
discussion =
Knowledge+ community response =
Policy/Action
What is a Decision Support System?
DSS starts with a:Problem +
administration =
Data + dissemination =
Information+ social
discussion =
Knowledge+ community response =
Policy/Action+ wisdom to ask a more
complex question =
12 Steps to Creating the DSS
These steps are a combination of buying and building that depend on
time and money
12 Steps to Creating the DSS
These steps are a combination of buying and building that depend on
time and money
Education CommunityInvolvement
12 Steps to Creating the DSS
These steps are a combination of buying and building that depend on
time and money
Conceptual AgreementDRA/DBA Staffing
Meta Data
Security/ConfidentialityUnique ID#
Edit check/ETL
Dup Res
BI tool
Data WarehouseData Mining
TrainingDecision Support System
Education CommunityInvolvement
Time from Operation to
Analysis
DRA
DBA
Information Democracy
12 Steps to Creating the DSS
Conceptual AgreementDRA/DBA Staffing
Meta Data
Security/ConfidentialityUnique ID#
Edit check/ETL
Dup Res
BI tool
Data WarehouseData Mining
TrainingDecision Support System
Education CommunityInvolvement
Time from Operation to
Analysis
DRA
DBALooks linear - is multidimensional
Information Democracy
12 Steps to Creating the DSS
Conceptual AgreementDRA/DBA Staffing
Meta Data
Security/ConfidentialityUnique ID#
Edit check/ETL
Dup Res
BI tool
Data WarehouseData Mining
TrainingDecision Support System
Education CommunityInvolvement
Time from Operation to
Analysis
DRA
DBA
Information Democracy
Factor in the fatigue-fizzle function
12 Steps to Creating the DSS
Conceptual AgreementDRA/DBA Staffing
Meta Data
Security/ConfidentialityUnique ID#
Edit check/ETL
Dup Res
BI tool
Data WarehouseData Mining
TrainingDecision Support System
Education CommunityInvolvement
Time from Operation to
Analysis
DRA
DBA
Information Democracy
Factor in the fatigue-fizzle function
Escape velocity
DS
S 12 Steps
Step #1Concept Formation
Initiation Phase
Project Planning
Execution/ ControlCloseout
DS
S 12 Steps
Building the Framework for the DW and DSS
Building the Framework for the DW and DSS
Must have conceptual agreement on the:
Building the Framework for the DW and DSS
Must have conceptual agreement on the:
Design of support system
Building the Framework for the DW and DSS
Must have conceptual agreement on the:
Design of support system Policy to protect data
Building the Framework for the DW and DSS
Must have conceptual agreement on the:
Design of support system Policy to protect data Advisory committee(s)
Building the Framework for the DW and DSS
Must have conceptual agreement on the:
Design of support system Policy to protect data Advisory committee(s) Buy or build and
Building the Framework for the DW and DSS
Must have conceptual agreement on the:
Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding
Building the Framework for the DW and DSS
Must have conceptual agreement on the:
Design of support system Policy to protect data Advisory committee(s) Buy or build and Funding
DSS Design: Best Practice
DSS Design: Best Practice
Who? What?
When?Data Warehouse
School Codes
Whom? Where? With?
DSS Design: Best Practice
Who? What?
When?Data Warehouse
Decision Support Users
Used for:OperationManagementPolicy MakersInstructionResearch
Decision Support Tools
Used how:Data miningAnalysisAd-hoc queryOff-line manipulation
School Codes
Foreign datai.e. Employment, Higher Ed
Whom? Where? With?
DSS Design: Best Practice
Who? What?
When?Data Warehouse
Decision Support Users
Used for:OperationManagementPolicy MakersInstructionResearch
Decision Support Tools
Used how:Data miningAnalysisAd-hoc queryOff-line manipulation
School Codes
Data Democracy Web Interface
Foreign datai.e. Employment, Higher Ed
Whom? Where? With?
DSS Design: Best Practice
As professionals, we need to make informed decisions, anticipate their impact on
education and design appropriate policy.
Who? What?
When?Data Warehouse
Decision Support Users
Used for:OperationManagementPolicy MakersInstructionResearch
Decision Support Tools
Used how:Data miningAnalysisAd-hoc queryOff-line manipulation
School Codes
Data Democracy Web Interface
Foreign datai.e. Employment, Higher Ed
Whom? Where? With?
Why
Steering Committee Oversight to design of the DSS
Steering Committee Oversight to design of the DSS Local district policy concerns
Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification
Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and
Steering Committee Oversight to design of the DSS Local district policy concerns Meta Data modification Standard reports, and Long term funding
2001 2002 2003 2004 2005 2006
Cost Savings?(OCIO-USED)
Current costs (paper and mail)
Warehouse/DSS initiative
Break even
2001 2002 2003 2004 2005 2006
Cost Savings?(OCIO-USED)
“We spend a lot of resources on an existing data edifice that isn’t very useful”
Current costs (paper and mail)
Warehouse/DSS initiative
Break even
DS
S 12 Steps
Step #2DRA & DBA
DS
S 12 Steps
Partnership on Both Sides of the Keyboard
Partnership on Both Sides of the Keyboard
DBA: technical implementation of the data warehouse
environment - chairs IT group
DRA: modifies and enforces standards that sustain the
DSS environment - chairs data
managers group
DRA and DBA CollaborationUser
requirements
Time18 Mo 30 Mo
Feature expectation (DRA)
IT Development Cycle (DBA)
Critical Divergence
DRA and DBA CollaborationUser
requirements
Time18 Mo 30 Mo
Feature expectation (DRA)
IT Development Cycle (DBA)
Critical Divergence
18 Mo
DRA and DBA CollaborationUser
requirements
Time18 Mo 30 Mo
Feature expectation (DRA)
IT Development Cycle (DBA)
Critical Divergence
18 Mo 30 Mo
DRA and DBA CollaborationUser
requirements
Time18 Mo 30 Mo
Feature expectation (DRA)
IT Development Cycle (DBA)
Critical Divergence
Outcome of building the DW within time frame:Data Warehouse will run 12-15 years - whereasCurrent apps last 6-7 years (with patches)
18 Mo 30 Mo
DS
S 12 Steps
Step #3Define the Data
DS
S 12 Steps
Meta DataData about the Database in the Data Warehouse
Meta DataData about the Database in the Data Warehouse
School Meta Data
Manual
Finance Meta Data Manual
Meta Data promotes the -• common understanding by users • data interchange with other agencies
Student Meta Data Manual
to define Meta Data break task into logical support
groupsPersonnel Meta Data Manual
Performance Meta Data Manual
Pupil Personnel
Human Resources
Finance Office
Test Company
Facilities Manager
Meta Data Online ManualsStudent
Personnel
Fina
nce
School Infrastructure
Performance
Meta Data Online ManualsStudent
Personnel
Fina
nce
School Infrastructure
Performance
Employment
Higher Education
Name of Field
Technical Information
Number of characters: (length)
Record position: (35-39)
Field type: (alpha, numeric, character)
SIF name:
XML tag: < >
Field Number
Warehouse name:R/Ecode
Warehouse type: VCAR
Blanks: (not accepted, null)
Progam Information
Date Information
Submission:
Code format:
Definition:
Elements (variables):
Revised:
Effective:
Discontinued:
Reporting Period:
?
Edits
Error traps:
Cross field edits:
Fatal Error:
Warning:
Historical Information
Form number replaced:
Statutory requirement:
Used for:
Report number:
Meta Data Online ManualsStudent
Personnel
Fina
nce
School Infrastructure
Performance
Employment
Higher Education
Step #4Maintaining Security and Confidentiality
DS
S 12 Steps
Protection is both sides of the keyboard
Protection is both sides of the keyboard
System Security (DBA)Identification (confident of who)
Authentication (confident of source)Authorization (grant access rights)
Access control (user profiling)Administration (security procedures)Auditing (monitoring and detection)
Protection is both sides of the keyboard
Confidentiality (DRA)
System Security (DBA)Identification (confident of who)
Authentication (confident of source)Authorization (grant access rights)
Access control (user profiling)Administration (security procedures)Auditing (monitoring and detection)
Established FERPA policyUnique NSN w/check sumStatistical disclosure (<6)
Human subject review policyPurge and destruction
Set levels of access & audit
Step #5Unique Testing ID (NSN)
DS
S 12 Steps
Test Identification Number: Production
Record layout
Warehouse layout
First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………
TID (10 digit)First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………
Test Identification Number: Production
Record layout
Warehouse layout
• Only assigned to one student (is unique).• Number and name can be confirmed as
being correct (verified via check sum).• Meets criteria as an identifier (is valid).• Has no intrinsic meaning (is nominal).• Can be substituted for a student’s name
(is not personally identifiable).• Permanent over the life-cycle of the
student (0-21 for special education).• Is returned and used by all local
education agencies (is ubiquitous).• Issued only by the SEA (is restricted).• Accessible by selected SEA employees
only (is confidential).
First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………
TID (10 digit)First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………
TID rules:
Test Identification Number: Problems
10 digit Check Sum
Constant
Variables
First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………ID#
Admin Unit #
Test Identification Number: Problems
10 digit Check Sum
Constant
Variables Variables change
First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………ID#
Admin Unit #
First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………ID#
Admin Unit #
Moves
Test Identification Number: Problems
10 digit Check Sum
Need other constant:Date of ImmunizationPlace of BirthBirth Cert Number
Constant
Variables Variables change
First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………ID#
Admin Unit #
First nameLast name
Date of BirthGender………FTE………Grade
Race/Ethnic………………ID#
Admin Unit #
Moves
42 states use a unique student identifier (DQC)
How constructed (NCES)
How issued (NCES)
Combination of fields (5)
Other (9)
Soc Sec Number (8)
SSN plus algorithm (1)
Random number (8)
SEA (20)
Other (4)
School (2)
LEA (9)
ISD (1)
Crossing over from aggregate to single record
Data reliability and validity
Time
Aggregate collection
Crossing over from aggregate to single record
Data reliability and validity
Time
Aggregate collection
Single record collection
Crossing over from aggregate to single record
Data reliability and validity
Time
Aggregate collection
Single record collection
Crossing over from aggregate to single record
Data reliability and validity
Time
Aggregate collection
Single record collection
Single record collection
Classrooms: District AClass size! Reading! 10! 8.04! 8! 6.95! 13! 7.58! 9! 8.81! 11! 8.33! 14! 9.96! 6! 7.24! 4! 4.26! 12! 10.84! 7! 4.82! 5! 5.68
Classrooms: District BClass size! Reading! 14! ! 8.1! 6! ! 6.13! 4! ! 3.1! 12! ! 9.13! 7! ! 7.26! 5! ! 4.74 ! 10! ! 9.14! 8! ! 8.14! 13! ! 8.74! 9! ! 8.77! 11! ! 9.26
Classrooms: District CClass size! Reading! 10! 7.46! 8! 6.77! 13! 12.74! 9! 7.11! 11! 7.81! 14! 8.84! 6! 6.08! 4! 5.39! 12! 8.15! 7! 6.42! 5! 5.73
Classrooms: District DClass size!Reading! 8! 6.58! 8! 5.76! 8! 7.71! 8! 8.84! 8! 8.47! 8! 7.04! 8! 5.25! 19! 12.5! 8! 5.56! 8! 7.91! 8! 6.89
Aggregated Data can be Misleading
Classrooms: District AClass size! Reading! 10! 8.04! 8! 6.95! 13! 7.58! 9! 8.81! 11! 8.33! 14! 9.96! 6! 7.24! 4! 4.26! 12! 10.84! 7! 4.82! 5! 5.68
Classrooms: District BClass size! Reading! 14! ! 8.1! 6! ! 6.13! 4! ! 3.1! 12! ! 9.13! 7! ! 7.26! 5! ! 4.74 ! 10! ! 9.14! 8! ! 8.14! 13! ! 8.74! 9! ! 8.77! 11! ! 9.26
Classrooms: District CClass size! Reading! 10! 7.46! 8! 6.77! 13! 12.74! 9! 7.11! 11! 7.81! 14! 8.84! 6! 6.08! 4! 5.39! 12! 8.15! 7! 6.42! 5! 5.73
Classrooms: District DClass size!Reading! 8! 6.58! 8! 5.76! 8! 7.71! 8! 8.84! 8! 8.47! 8! 7.04! 8! 5.25! 19! 12.5! 8! 5.56! 8! 7.91! 8! 6.89
Aggregated Data can be Misleading
! Avg. classrooms != 11! Avg. class size != 9.0! Avg. reading score != 7.5
Four districts are similar
District C
5
10
10 20
District D
5
10
10 20
District A
5
10
10 20
District B
5
10
10 20
Reports using Disaggregated Data
Individual reading scores
Four districts are very different
Cleaning the DataStep #6
DS
S 12 Steps
Quality Data
Quality DataReasons for poor quality of data:Absence of definitionsUnclear definitionsLack of human resourcesInconsistent collections cycles (not ongoing)Insufficient timeInadequate training on entry and data trapsLack of data integrationFear of 'punishment' (look bad syndrome)
Quality DataThe key elements that improve the quality of what is being collected include:
• Consistency. Data fields must have a standardized definition so that each entity can be collected from each district in a systematic manner.
• Timeliness. There is no efficiency in gathering statewide data that reflects a one-time need or an unusual piece of information. Do a survey.
• Reliability. The data set should reflect a dependable measurement of every entity from one collection cycle to another (i.e., data has accuracy regardless of who enters it.)
• Validity. A data element must reflect a logical and meaningful description of an entity and should not be subject to interpretation (i.e., data has utility to answer the question being asked.)
The key elements that improve the quality of what is being collected include:
• Consistency. Data fields must have a standardized definition so that each entity can be collected from each district in a systematic manner.
• Timeliness. There is no efficiency in gathering statewide data that reflects a one-time need or an unusual piece of information. Do a survey.
• Reliability. The data set should reflect a dependable measurement of every entity from one collection cycle to another (i.e., data has accuracy regardless of who enters it.)
• Validity. A data element must reflect a logical and meaningful description of an entity and should not be subject to interpretation (i.e., data has utility to answer the question being asked.)
Step #7Resolving Duplicates
DS
S 12 Steps
Thresholds and Assigning ID numbersTrue False True
Non-match
Match
Match is true - are the same student (assign same ID#)
Thresholds and Assigning ID numbers
Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60
True False True
Non-match
Match
Match is true - are the same student (assign same ID#)
Non-match is true - are different students
(assign different ID#s)
Thresholds and Assigning ID numbers
Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60
Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60
True False True
Non-match
Match
Match is true - are the same student (assign same ID#)
Match is false - are different students (assign same ID#)
Non-match is true - are different students
(assign different ID#s)
Thresholds and Assigning ID numbers
Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60
Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60
Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60
True False True
Non-match
Match
Match is true - are the same student (assign same ID#)
Match is false - are different students (assign same ID#)
Non-match is false - are the same student (assign different ID#s)
Non-match is true - are different students
(assign different ID#s)
Thresholds and Assigning ID numbers
Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60
Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60
Pat ! Smith! M! 1/19/60Patrick! Smith! ! 1/19/60Pat ! ! Smyth! M! 1/19/60
Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60
True False True
Non-match
Match
Match is true - are the same student (assign same ID#)
Match is false - are different students (assign same ID#)
Non-match is false - are the same student (assign different ID#s)
Non-match is true - are different students
(assign different ID#s)
Thresholds and Assigning ID numbers
Pat ! Smith! M! 1/19/60Pat! T! Smith! M! 1/19/60
Patricia! Smith! F! 1/19/60Pat! ! Smith! ! 1/19/60
Pat ! Smith! M! 1/19/60Patrick! Smith! ! 1/19/60Pat ! ! Smyth! M! 1/19/60
Pat ! Smith! F! 1/19/60Pat! T! Smith! ! 1/19/61Patrick ! Smith ! M! 1/19/60
True False True
Non-match
Match
Error Creep
Step #8Select a BI Tool
DS
S 12 Steps
Task #1:Create Model
Software & Hardware
Task #1:Create Model
On scalable, normalized, symmetric multiprocessing
architecture
Software & Hardware
Task #2: Set up a ‘road map’
Task #3: Choose a BI tool
Task #3: Choose a BI tool
Step #9Data Warehouse
DS
S 12 Steps
Benefits of DW:
Benefits of DW:Reduction of paper formsSavings from data duplicationBest use of technologySole source of reusable dataCommon set of definitionsIntegrated environment of core dataBreaks cycle of low quality dataAnswers that took months take daysReports that took days take minutes
Reduction of paper formsSavings from data duplicationBest use of technologySole source of reusable dataCommon set of definitionsIntegrated environment of core dataBreaks cycle of low quality dataAnswers that took months take daysReports that took days take minutes
Data Democracy for the Educational CommunityRe
port
s
QuerySimple - one time
Sophisticated - ongoing
Ad-hoc
Pre-defined
Data Democracy for the Educational CommunityRe
port
s
QuerySimple - one time
Sophisticated - ongoing
Ad-hoc
Pre-defined
General Public
Finance Officers
Audit
ors
Reporters
ResearchersLegislative Aides
Data Democracy for the Educational CommunityRe
port
s
QuerySimple - one time
Sophisticated - ongoing
Ad-hoc
Pre-defined
General Public
Finance Officers
Audit
ors
Reporters
ResearchersLegislative Aides
Push
Pull
As system is used one will find a need to store data not being captured
Push example: one time - pre defined
School report card• School Size: small vs. large schools• Spending: percent of budget on staff salary• Safety: rate of expulsions and degree of crime• Technology: ratio of pc's to students & connectivity• Class Size: teacher-student ratio, average size• Staff Turnover: rate and attendance• Advanced Placement: number passing test• Test Scores: gaps in State performance test• College Acceptance Rate: percent taking ACT, PSAT• Graduation/Dropout Rates: number taking GED• Satisfaction: teachers, parents and students
Push example: one time - pre defined
Significant Usable
Pull example: ongoing - Ad hoc
The largest class size in high school is the 9th grade Not really No
Significant Usable
Pull example: ongoing - Ad hoc
The largest class size in high school is the 9th grade Not really No
Some 9th grades have a disproportionate number of Hispanics Possibly No
Significant Usable
Pull example: ongoing - Ad hoc
The largest class size in high school is the 9th grade Not really No
Some 9th grades have a disproportionate number of Hispanics Possibly No
Many female Hispanics in the 9th grade are retained due to poor science skills Possibly Yes
Significant Usable
Pull example: ongoing - Ad hoc
The largest class size in high school is the 9th grade Not really No
Some 9th grades have a disproportionate number of Hispanics Possibly No
Many female Hispanics in the 9th grade are retained due to poor science skills Possibly Yes
Hispanics in the 8th grade had fewer computers in science classrooms and more teachers who do not have a teaching major in science
Yes Yes
Significant Usable
Pull example: ongoing - Ad hoc
The DW Backbone:
NCLB
The Sole Authority for the Educational Community
School Accreditation
Crime/Safety
Quality Workforce
AYP State Report Card
IDEA
Title II (IHE)
Fiscal Trends
The DW Backbone:
NCLB
The Sole Authority for the Educational Community
School Accreditation
Crime/Safety
Quality Workforce
AYP State Report Card
IDEA
Title II (IHE)
Fiscal Trends
Step #10Data Mining
DS
S 12 Steps
Data re-construction
Data re-constructionUndirected and exploratory
knowledge discovery
Data re-constructionUndirected and exploratory
knowledge discovery
Sequencing: order of patterns or groups
Data re-constructionUndirected and exploratory
knowledge discovery
Sequencing: order of patterns or groups
Framing: using past data to predict trend
Data re-constructionUndirected and exploratory
knowledge discovery
Sequencing: order of patterns or groups
Framing: using past data to predict trend
Clustering: assembling unforeseen groups
Data re-constructionUndirected and exploratory
knowledge discovery
Sequencing: order of patterns or groups
Framing: using past data to predict trend
Clustering: assembling unforeseen groups
Drilling: interactive discovery
Multidimensional Ad-hoc Analysis
Student
Technology Infrastructure
Performance
Multidimensional Ad-hoc Analysis
Student
Technology Infrastructure
Performance
Single Parent Homes
Live Births
Millages Passed
Multidimensional Ad-hoc Analysis
Student
Technology Infrastructure
Performance
Single Parent Homes
Live Births
Millages Passed
Ethnic change and growth by
enrollment
Multidimensional Ad-hoc Analysis
Student
Technology Infrastructure
Performance
Single Parent Homes
Live Births
Millages Passed
Ethnic change and growth by
enrollment
Performance by gender by PCs
Multidimensional Ad-hoc Analysis
Student
Technology Infrastructure
Performance
Single Parent Homes
Live Births
Millages Passed
Ethnic change and growth by
enrollment
Performance by gender by PCs
Trends and Projections
Multidimensional Ad-hoc Analysis
Student
Technology Infrastructure
Performance
Single Parent Homes
Live Births
Millages Passed
Ethnic change and growth by
enrollment
Performance by gender by PCs
Trends and Projections
Similar districtsthat passed bonds
by monthover past 3 yrs
by ethnicityby buildingby grade
Step #11Conduct Training
DS
S 12 Steps
The ultimate goal of training is to have everyone who touches the data at every level know what is expected of them, so that the data that are submitted will be
the valid and reliable.
TrainingTraining must also include detailed procedures, for example:
Training
Who gets notified when an error is discovered and how is the notification done?
Training must also include detailed procedures, for example:
Training
Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)?
Training must also include detailed procedures, for example:
Training
Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data?
Training must also include detailed procedures, for example:
Training
Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user?
Training must also include detailed procedures, for example:
Training
Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing?
Training must also include detailed procedures, for example:
Training
Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified?
Training must also include detailed procedures, for example:
Training
Who gets notified when an error is discovered and how is the notification done? What is the procedure for making corrections of data within an agency (i.e., who actually makes them and retransmits the new error-free data)? Who reviews, verifies or signs off on the cleaned data? Who provides technical assistance to the end user? What is the procedure to ensure a new copy of the data is retained for auditing? Who receives confirmation that the file has been received as specified? Who secures the data and maintains confidentiality?
Training must also include detailed procedures, for example:
Reallocation of Resources
Data collection, error checks, and clean-up
Have multiple collections -
use once disregard
Analysis Reporting Decision support and shared data
Have
Reallocation of Resources
Data collection, error checks, and clean-up
Have multiple collections -
use once disregard
Analysis Reporting Decision support and shared data
WantHave
Staff training - shifts from front to back end
Step #12The DSS
DS
S 12 Steps
Step #12The DSS
DS
S 12 Steps
Providing access to critical information for driving, managing, tracking, and measuring
institutional policies and goals.
The first decision of the DSS is to make a decision
Transactional Cyclical
The first decision of the DSS is to make a decision
Transactional CyclicalRealtime Points in time
Day to day operations HistoricalUpdates daily/weekly Updates quarterly
7X24 6X18Read/write Read only
Short term data retention Long-term (longitudinal)Mission critical queries Strategic-analytical queriesMore open access paths More restricted accessStandardized reports Adhoc reports
Server based Warehouse technology
DSS: Helps Anticipate Issues
Policy Repercussion
Policy Forecasting
Problem Anticipation
Problem Reaction
DSS: Helps Anticipate Issues
Policy Repercussion
Policy Forecasting
Problem Anticipation
Problem Reaction
Current
DSS: Helps Anticipate Issues
Policy Repercussion
Policy Forecasting
Problem Anticipation
Problem Reaction
Need to be
DSS: Helps Anticipate Issues
Policy Repercussion
Policy Forecasting
Problem Anticipation
Problem Reaction
Need to be
Cannot anticipate with only ‘required’
data
Help Anticipate Impact of Policy:Class Size
Help Anticipate Impact of Policy:Class Size
Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?
Help Anticipate Impact of Policy:Class Size
Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?
Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?
Help Anticipate Impact of Policy:Class Size
Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?
Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?
Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?
Help Anticipate Impact of Policy:Class Size
Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?
Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?
Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?
Infrastructure Issues - Do buildings have the space for additional classrooms?
Help Anticipate Impact of Policy:Class Size
Achievement Issues - Does reducing the class size below grade 4 improve achievement on the state performance test?
Teacher Supply/Demand Issues - Will the current production of teachers by IHEs meet the increased need for staff at the LEAs?
Fiscal Issues - Using average salary, will there be sufficient funds for LEAs to hire additional staff?
Infrastructure Issues - Do buildings have the space for additional classrooms?
Trend Issues - Will improved achievement impact employment, graduation or adult life roles?
Impact on State Standards
Impact on State Standards
Efficiency of SystemInputs Process Outputs
Impact on State Standards
Efficiency of System
Input issues:fiscal resourcesteacher supply
building structuretechnologypoverty
Inputs Process Outputs
Impact on State Standards
Efficiency of System
Input issues:fiscal resourcesteacher supply
building structuretechnologypoverty Process issues:
crime and safetyprof development
attendanceteacher experiencestudent performance
Inputs Process Outputs
Impact on State Standards
Efficiency of System
Input issues:fiscal resourcesteacher supply
building structuretechnologypoverty Process issues:
crime and safetyprof development
attendanceteacher experiencestudent performance
Output issues:college entrance
graduate numbersretention ratesemployment
Inputs Process Outputs
Impact on State StandardsEffectiveness of System
Efficiency of System
Input issues:fiscal resourcesteacher supply
building structuretechnologypoverty Process issues:
crime and safetyprof development
attendanceteacher experiencestudent performance
Output issues:college entrance
graduate numbersretention ratesemployment
Inputs Process OutcomesOutputs
Impact Policy
Outcome issues:works with others
acquires informationunderstands inter-relationships
allocates resources works w/variety of tech
Impact on State StandardsEffectiveness of System
Efficiency of System
Output issues:college entrance
graduate numbersretention ratesemployment
Inputs Process OutcomesOutputs
Impact PolicyWill no
t impact
policy with on
ly
‘required’
data
Outcome issues:works with others
acquires informationunderstands inter-relationships
allocates resources works w/variety of tech
Will not impact
policy with on
ly
‘required’
data
Finding the BalanceRequired
Data
MandatoryMeasurement in volume
(amounts, avg., ranks, percents) Realistic
Social IntegrationVocational Orientation
Use of TimeDaily Living Skills
MobilityUse of Environmental Ques
Desired Data
Finding the BalanceRequired
Data
MandatoryMeasurement in volume
(amounts, avg., ranks, percents) Realistic
Social IntegrationVocational Orientation
Use of TimeDaily Living Skills
MobilityUse of Environmental Ques
The DSS must help policy makers find a comfortable balance between acceptable risks and benefits.
Desired Data
InputIssues
ProcessIssues
OutputIssues
OutcomesIssues
General Public
Parents
Teachers
Support Staff
Admin/Boards
State Legislators
Others
Helps in Data Discovery
Standards moves from efficiency to effectiveness
One Last Time
District UsersUpload data formatsCorrect duplicate dataFERPA requests
Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard
Web Front End
Error reports
One Last Time
File ETL: • Student • Assessment • Finance • Professional
Deve
lope
r Ap
plica
tion
s
Error reports
Check Sum
Student IDsMatch & Merge
Audit (FERPA)
SecurityDistrict UsersUpload data formatsCorrect duplicate dataFERPA requests
Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard
Web Front End
Error reports
One Last Time
File ETL: • Student • Assessment • Finance • Professional
Deve
lope
r Ap
plica
tion
s
WAREHOUSE
Error reports
Check Sum
Student IDsMatch & Merge
Audit (FERPA)
SecurityDistrict UsersUpload data formatsCorrect duplicate dataFERPA requests
Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard
Web Front End
Error reports
School Codes
Meta Data
GPS
Reliable/Valid
One Last Time
File ETL: • Student • Assessment • Finance • Professional
Deve
lope
r Ap
plica
tion
s
WAREHOUSE
Error reports
Check Sum
Data Mart
Student IDsMatch & Merge
Audit (FERPA)
Security
DoE UsersGenerate Report CardFederal: EDEN, NCLB, IDEASkopusIssue Assessment IDs
District UsersUpload data formatsCorrect duplicate dataFERPA requests
Public Portal AccessPredefined Report CardsLimited queriesDashboard/Scorecard
Web Front End
Error reports
School Codes
Meta Data
GPS
Reliable/Valid
Current problem:data rich and information poor
Current problem:data rich and information poor
Department
Data Silos
Current problem:data rich and information poor
Department Educational Community
Gap:Lack of confidence No trust in systemHave a low ROI
Data Silos
Solution
Department Educational Community
Data Democracy
Secure ScalableFlexible
Data Warehouse
Student
Meta Data
Manual
Performance Meta Data Manual
Personnel
Meta Data Manual
School Meta Data Manual
Finance Meta Data Manual
Apply information and facilitate
decision-making
Solution
Department Educational Community
Data Democracy
Secure ScalableFlexible
Data Warehouse
Student
Meta Data
Manual
Performance Meta Data Manual
Personnel
Meta Data Manual
School Meta Data Manual
Finance Meta Data Manual
Apply information and facilitate
decision-making
Without Data
You’re Just Another Person With an Opinion
We find ourselves in an
Information Agewith an aging information
system
Decisionsbegin with good data
Most of the fun using the DSS is not finding the
answer to your question - it’s finding the new
questions you don’t have the answers to.