How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process...
-
Upload
austin-quentin-osborne -
Category
Documents
-
view
214 -
download
0
Transcript of How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process...
![Page 1: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/1.jpg)
How we let them drive:Developing dashboards to deliver multiple
years of NSSE/FSSE dataOur Process
Southern Association of Institutional Research2010 Conference - New Orleans
Monday, September 27th4:15 p.m. - 5:00 p.m.
Robin Logan, Director of Institutional ResearchEarl Harmsen, Faculty H-E-B School of Business
Michael Schulte, Graduate Research Assistant & Doctoral Candidate
![Page 2: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/2.jpg)
4 year, Private, Catholic
Hispanic Serving Institution
Established 1881 by the Sisters of Charity of the Incarnate Word
Carnegie: Master’s L
San Antonio: 7th largest city in the US
![Page 3: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/3.jpg)
UGUG GRGR PRPR INTLINTL TOTTOT
5,4645,464 1,2411,241 509509 494494 7,7087,708
MainMain ADCaPADCaP VUVU INTLINTL TOTTOT
69%69% 15%15% 10%10% 6%6% 100%100%
HISPHISP MINMIN WHIWHI INTLINTL UNKUNK TOTTOT
49%49% 11%11% 21%21% 11%11% 8%8% 100%100%
UIW Demographics
![Page 4: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/4.jpg)
Geri Hockfield Malandra. (2008) Accountability and learning assessment in the future of higher education. On the Horizon, 16(2), 57-71. Retrieved September 10, 2010, from proQuest Education Journals. (Document ID: 1484475591).
Accountability and assessment have become prominent topics of institutional and public discussion
![Page 5: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/5.jpg)
Gonyea, R. M., & Kuh, G. D. (Eds.). (2009). Editor’s note: Using NSSE in institutional research. New Directions for Institutional Research, 141, 1-4. DOI: 10.1002/ir.282
...“increased visibility and importance of institutional research offices staffed by highly skilled and competent professionals who can provide campus leaders with objective, trustworthy data about student and institutional performance”
![Page 6: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/6.jpg)
multiple years of data
NSSE 3 years + 2010
FSSE 2 years + 2010
"still in the binders"
various committees and assessment teams needed “meaningful” information
too much to comprehend all at one time
![Page 7: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/7.jpg)
the challenge
• multi-campus institution
• 20 questions referenced in the QEP
• QEP population for NSSE required filters to present the correct views with FSSE data
• colleges requesting data specific to them
![Page 8: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/8.jpg)
evaluate what had already been done
narratives...
lots of words!
lots of actions!
unclearmethodologies
data quality
![Page 9: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/9.jpg)
NSSE Multi-Year Data Analysis Guide. (2010). Indiana University Center for Postsecondary Research. Retrieved September 23, 2010 from http://nsse.iub.edu/2010_Institutional_Report/pdf/NSSE%20MYDAG.pdf
1. Identify and focus on specific questions
2. Select and employ appropriate methods of analysis
3. Attend to data quality for each year in the analysis
4. Take into account changes in NSSE items and reports across years
5. Merge multiple years of data
![Page 10: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/10.jpg)
how does IR...?
• deliver what multiplestakeholders need
• move towards “plug-and-play”
• keep it simple
• keep our sanity
![Page 11: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/11.jpg)
Hughes, C. (2009). Framing the activities of institutions and academic development units in support of assessment. Innovations in Education and Teaching International, 46(2), 123-133. Retrieved September 10, 2010, from Research Library. (Document ID: 1796476001).
• Universities demonstrate their commitment to enhancing and assuring the quality of assessment through the employment of a diverse array of initiatives.
![Page 12: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/12.jpg)
NSSE ALL
NSSE UNIT
Tableau - known!
Interactive - for UNIT - known!
Data source created in MSExcel
One file or multiple files?
FSSE ALL
QEP Combined File
![Page 13: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/13.jpg)
NSSE ALL
stacked data for multi-year comparison
check alignment of questions
add identifier columns (YR, FY/SY, GROUPS)
decision about experimental items from prior years
![Page 14: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/14.jpg)
![Page 15: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/15.jpg)
• Means and Frequencies
•Means ... well-intentioned, but not enough to present to the community
•Frequency distributions... referenced for all but one benchmark in the QEP
XLSX: NSSE ALL DATA
![Page 16: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/16.jpg)
But what about question response rate?
But what about knowing which was used in the QEP?
But what about ...
![Page 17: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/17.jpg)
XLSX: NSSE ALL DATA
![Page 18: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/18.jpg)
NSSE UNIT
• much easier
• same basic steps
• data for all responses - requires CALCS!
• match to campus data for campus and college **not necessary with 2010 because it was institutional column
![Page 19: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/19.jpg)
• What do we include?
filters on the dashboard
QEP notations
benchmark notations
XLSX: NSSE UNIT LEVEL DATA
![Page 20: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/20.jpg)
• What do we include?
means and frequencies
response rate specifically for the question
XLSX: NSSE UNIT LEVEL DATA
![Page 21: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/21.jpg)
• What do we include?
• static tables from NSSE ALL
•comparison to NSSE, Master’s, and Peer Institutions
XLSX: NSSE BENCHMARK DATA
![Page 22: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/22.jpg)
![Page 23: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/23.jpg)
FSSE ALL
• Three concepts to consider:
those questions chosen for QEP
benchmark comparisons to NSSE
with
without
![Page 24: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/24.jpg)
• Going beyond the frequencies
• *for those with NSSE crosswalks
XLSX: FSSE DATA
![Page 25: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/25.jpg)
Incorporating the QEP goals
Crosswalk comparison
• Methodology
• FSSE: Main Campus faculty only
• NSSE: Data filtered to Main Campus respondents
XLSX: FSSE CROSSWALK
XLSX: NSSE AGG QEP CALCS
XLSX: FSSE DATA
![Page 26: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/26.jpg)
QEP - Should it be its own workbook?
YES!
![Page 27: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/27.jpg)
Respondent CharacteristicsXLSX: NSSE RESPONDENTS XLSX: PEER INSTITUTIONS
![Page 28: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/28.jpg)
End Result
• NSSE ALL
• almost too big
• NSSE UNIT
• split into three interactive workbooks because of size
FSSE
split into four workbooks because of size
QEP
limited to 20 questions
![Page 29: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/29.jpg)
End Result
Data source: MSExcel
started with multiple files
transferred to a single multi-tabbed file
Worksheets:
NSSE Benchmarks
NSSE ALL Data
NSSE UNIT Data
Peer Institutions
NSSE Respondents
FSSE Crosswalk
FSSE Data
![Page 30: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/30.jpg)
Moore, J. (2005). Seven recommendations for creating sustainability education at the university level: A guide for change agents. International Journal of Sustainability in Higher Education 6(4), 326-339. Retrieved September 10, 2010, from Research Library (Document ID: 929185721).
Recommendation: integration of university plans, decision-making structures and evaluative measures and the integration of the research, service and teaching components of the university
![Page 31: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/31.jpg)
Gonyea, R. M., & Kuh, G. D. (Eds.). (2009). Editor’s note: Using NSSE in institutional research. New Directions for Institutional Research, 141, 1-4. DOI: 10.1002/ir.282
...“ascendance of student engagement and other process indicators that serve as both proxy measures for institutional quality and actionable information to inform improvement efforts.”
![Page 32: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/32.jpg)
Robin Logan, Director of Institutional ResearchEarl Harmsen, Faculty H-E-B School of Business
Michael Schulte, Graduate Research Assistant & Doctoral Candidate
![Page 33: How we let them drive: Developing dashboards to deliver multiple years of NSSE/FSSE data Our Process Southern Association of Institutional Research 2010.](https://reader030.fdocuments.us/reader030/viewer/2022032805/56649ee85503460f94bf9b25/html5/thumbnails/33.jpg)