Closing the Gap: From Data to Decision Making
description
Transcript of Closing the Gap: From Data to Decision Making
![Page 1: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/1.jpg)
Closing the Gap:From Data to Decision Making
Stefinee Pinnegar, Ph.D.Pat Esplin, Ph.D.
Brigham Young University
academy.byu.edu
![Page 2: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/2.jpg)
Closing the gapfrom data to decision making
Intractable Problems •Our world is confronted with problems that occur over and over and are difficult to address•Providing solutions at a particular place and a particular time can inform others
Overarching Questions •Organizational goals can be catalysts for identifying and responding to the intractable problems within your setting•There is power in translating your goals into questions that guide assessment and action.
Assessment Strategies •Identifying available data embedded within normal activities of the organization provides a rich source of information•Using an assessment cycle transforms data to actionable information
Decision Making •Organizations become more strategic when they base actions on meaningful data•Learning organizations seek improvement by gathering information about the effectiveness of their actions in an assessment cycle.
![Page 3: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/3.jpg)
1. _____________________________________2. _____________________________________3. _____________________________________4. _____________________________________5. _____________________________________6. _____________________________________
Intractable Problems
•What are some intractable human issues that always underlie problem solving in education?
![Page 4: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/4.jpg)
1. Literacy2. Economic health/poverty3. Diversity4. Health5. Family6. Power
Intractable Problems
•Our (student & society) intractable problems
![Page 5: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/5.jpg)
Overarching Questions
•How can we effectively support student success here?
• Underlying questions–Who?–What?–When?–Where?–Why?—Mission statement– How?
![Page 6: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/6.jpg)
The existing knowledge, understanding, abilities, motives
and conceptions/ styles of learning students already have
Quality of learning achieved
How studentsapproach learning
and studying
How course material is selected, organized, presented, assessed
University teachers’ ways of thinking about teaching
How students perceive the teaching-learning
environment
Conceptual framework indicating influences on student learning
How a teaching-learning environment is designed
and implemented
Noel Entwistle, ESRC/TLRP ETL report 3, March 2003
What students are expected to learn and understand
Constructive alignment
![Page 7: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/7.jpg)
1. _____________________________________2. _____________________________________3. _____________________________________4. _____________________________________5. _____________________________________6. _____________________________________
Overarching Questions
•What are your institution’s/program’s basic or overarching questions?
![Page 8: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/8.jpg)
Assessment Strategies
•Levels of on-going assessment
Classroom
Program/ Department
Institution
![Page 9: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/9.jpg)
Assessment Cycle
Your institutionMission/Values
Assessment criteria (How will we know if we
have done this well?)
Learning – So what?(What did we learn? What does it mean?)
Outcomes(What do we want
to achieve?)
Improvement(Now what? Policy, resource changes?)
Assessment results (What happened?)
Assessment activities (Design, carry out)
![Page 10: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/10.jpg)
InternalBYUDeveloped
Employers of BYU Graduates
Survey
Graduate Student
Exit Survey
External
BYU Freshman Surveys
• Pre-arrival• End-of-year
Suite of BYU Instruments
Timeline
NSSE
Non-Returning Student Survey
Freshman Sophomore Junior Senior Grad Std Post-college
NSSEBCSSE
AQ2 (10 year)
Alumni Questionnaire
Senior Survey
Internal /In use
Internal / In DevelopmentKey External
Aims of a BYU EducationFoundation
Internal / Planned
![Page 11: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/11.jpg)
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
Assessment Strategies
•What kinds of data do you already have?•What kinds of data do you need?
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
Data you have Data you need
![Page 12: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/12.jpg)
Overarching Questions• How can we effectively support
student success here?
Assessment Strategies• What data do we already have?• What data do we need?
• Note: each word in the question directs our efforts:
• How• Can• We• Effectively• Support• Student success• Here
![Page 13: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/13.jpg)
Assessment Strategies
•Who matters at my institution?
Freshman Academy
![Page 14: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/14.jpg)
1. _____________________________________2. _____________________________________3. _____________________________________4. _____________________________________5. _____________________________________6. _____________________________________
Assessment Strategies
•Who matters at my institution?
![Page 15: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/15.jpg)
Assessment Strategies
•What matters to us?
Freshman Academy (FA) is a one-semester program that helps first-year students make important connections within the university during their first semester at BYU. By creating the best teaching-learning environment possible, FA strives to provide an academic environment that facilitates the development of well-rounded students and bridges the gap between high school and college. FA does this by helping students connect with each other, faculty members, academic majors, the University Core, The Aims of a BYU Education, and other university resources.
Freshman Academy Mission Statement
![Page 16: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/16.jpg)
1. _____________________________________2. _____________________________________3. _____________________________________4. _____________________________________5. _____________________________________6. _____________________________________
Assessment Strategies
•What matters at my institution?
![Page 17: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/17.jpg)
StudentPeer
mentor
Other studentsFaculty
Academic Experience
Development of the total Person
Aims of BYU Education• Spiritually strengthening• Intellectually enlarging• Character building• Life-long learning & service
Academic majors
University resources
Bridging the GapFreshman Academy Mission
![Page 18: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/18.jpg)
What matters to stakeholders?
Cost Effectiveness
High impact Low impact
High cost
Low cost
Student retention
Student performance (grades)
Student learning
![Page 19: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/19.jpg)
Time spent at the university
Expe
ctati
ons
1 yr 4 yr3 yr2 yr
When do things matter?
![Page 20: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/20.jpg)
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
Assessment Strategies
•When do you do things?
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
What things do you do when? What do you know about them?
![Page 21: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/21.jpg)
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
Assessment Strategies
•What do you know about student success at your institution?
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
What things do you know? How do you know them?
![Page 22: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/22.jpg)
Analysis of Existing Data
Points in time
Criteria Demographics
BackgroundExpectations Engagement
ConnectionsLearning Habits Performance
Pre-semester •BCSSE•BCSSE•Learning Goals Inventory•Boyer Model
•High school engagement•FA interactions•Housing placement
•Approaches to study•Time spent in high school
•ACT•GPA/EGPA analysis•Summer reading reflection
Week 3 •Time logs•PM “How well do you know”•Meals with faculty•Time logs
•Weekly surveys•Classroom observations•Time logs
Mid-term •Socio-grams•Meals with faculty
•Weekly surveys•Classroom observations
End of Semester
•End of semester survey•Student reflections•Peer mentor reflections summaries
•Peer mentor reflection summaries
•Final learning papers•Student reflections•Peer mentor reflection summaries
•Student reflections•Peer mentor reflection summaries
Follow-up 2nd Semester
•Modified Temple survey•Use of campus resources•NSSE•Peer mentor story cycles
•Use of time•(ETL)•Peer mentor story cycles
![Page 23: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/23.jpg)
Assessment Strategies• What do you know about: How can
we effectively support student success here?
Decision Making • What do we do now?
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
What do you know? What can you do?
![Page 24: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/24.jpg)
High ACT
Low ACT
High HS GPA
Low HS GPA
Significant time preparing for class
Low level Engagement
Limited time preparing for class
High level Engagement
Assessment Strategies
•Students who matter
![Page 25: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/25.jpg)
Assessment Strategies• What do you know about: How
can we effectively support student success here?
Decision Making • What do we do now?
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
1. ________________2. ________________3. ________________4. ________________5. ________________6. ________________
What do you know? What can you do?
![Page 26: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/26.jpg)
![Page 27: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/27.jpg)
![Page 28: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/28.jpg)
Before semester• Application• Registration• Autobiogra
phies• Surveys• Contact
First 3 weeks• Interviews• How well
do you know?
Mid-term• Faculty
meetings• Student
engagement
End of Semester• Interviews• Transition
ratings• Student
engagement
Using typical activities and timelines
![Page 29: Closing the Gap: From Data to Decision Making](https://reader035.fdocuments.us/reader035/viewer/2022062411/56816694550346895dda785b/html5/thumbnails/29.jpg)
Closing the gapfrom data to decision making
Intractable Problems •Our world is confronted with problems that occur over and over and are difficult to address•Providing solutions at a particular place and a particular time can inform others
Overarching Questions •Organizational goals can be catalysts for identifying and responding to the intractable problems within your setting•There is power in translating your goals into questions that guide assessment and action.
Assessment Strategies •Identifying available data embedded within normal activities of the organization provides a rich source of information•Using an assessment cycle transforms data to actionable information
Decision Making •Organizations become more strategic when they base actions on meaningful data•Learning organizations seek improvement by gathering information about the effectiveness of their actions in an assessment cycle.