Post on 14-Dec-2015
September 30, 2014
Dawn Cullity
Bernadette Ferro
Kent Phillippe
VFA Year Two: Show & Tell
Then
VFA Overview
Now
The Voluntary Framework of Accountability is the first national system of accountability
SPECIFICALLY
for community collegesby community colleges.
User Experience
AACC’s Objectives for the VFA Improve the user experience
Increase utility (so we can meet all the goals listed here)
• Value-added metrics for accountability (using data to measure performance and make better decisions)
• National benchmarking for good practices
• Opportunity to streamline reporting / relieve IR capacity
• Ability to tell a better, more robust story
• Better data to inform public policy and advocacy
Utility
feedback
Objective: Improve Data Input
Feedback on the user set-up and data collection process/data tool
Stability of, and access to, the online collection and display (dashboards)
Tool is clunky, especially the online forms, and needs better validation processes
Workflow is not intuitive and progress bars does not tell us where we are in data collection process
improvements
Cohort Bulk Upload Online Data Entry
Raw Data File Upload
Not Reporting
2008 Cohort (6 year) O O O O
2012 Cohort (2 year) O O O O
CTE O O Not Available O
ABE O O Not Available O
Submit
Workflow: Data Configuration
Workflow: Managing Data CollectionWORKFLOW ACTIONS STATUS / NOTES
Configure Data “Data last configured on…”– with a list data configured (6 year, 2 year, CTE, and/or ABE) or
“Data needs to be configured”
Upload Data File(s) If no files to be uploaded show nothing.
If files have been uploaded “data last uploaded on .../../..” and lists each file type that has been configured and the date uploaded
Edit/View Data Dimmed if data not configured – “Data needs to be configured”Dimmed if data upload files are in the queue – “Data in the queue to be uploaded”If all data is Processed – allow user to select.
Data Validation Checks
Mark Data for Review If Data configured – allow for the user to click Mark for Review – ‘Display the last notes stored by the users of the system for review’
Publish Data
Dashboard and Exports Takes user to Exports page
Public Outcomes Reports Dimmed if data has not been made public yet
VFA Manual Data Entry
VFA Cohorts Disaggregated
feedback
Objective: Increase Utility of VFA Exports
Feedback on the use of the VFA and its data
Dashboards need improvement to be useful and understandable
Colleges need to be able to download final values including data and calculated percentages
Benchmarking parameters and policies
Dashboards need a lot of interpreting before passing to senior leadership/board improvements
VFA Export: Outcomes Dashboard
VFA Export: Tabular Data
VFA Benchmarking Dashboard: Benchmarking Selectors
VFA Benchmarking Dashboard: Six-Year Outcomes
VFA Export: Public Dashboard Report
feedback
Objective: Improve Data Collection Resources
Feedback on the resources (instructions)
Clarity: make the Metrics Manual user friendly, understandable
Consistency
Authority: be definitive with methodology and how metrics are to be reported
improvements
Cohorts
Reporting VFA
Measures
Six Year Cohort
Two Year Cohort
Main
Credential Seeking
First Time In College
Developmental Education Progress
Measures
Two-Year Progress Measures
Six-Year Outcomes Measures
Career & Technical Education Measures
Adult Basic Education Measures
CTE Cohort
ABE Cohort
VFA Metrics Manual
VFA Metrics Manual (cont.)
VFA Overview & Training Webinars
http://vfa.aacc.nche.edu/webinars/Pages/default.aspx
Y2 , Y3, Y4…
Close-Out What do you think should be our priorities going forward?
Questions & Feedback
Contact VFA at VFA@aacc.nche.edu
principle accountability
system
Reminders Sign-ups open October 1st
Data collection opens November 3rd
Questions During the Webinar
Data Dashboards Have the dashboards been tested with IE, Firefox, and Chrome?
The VFA dashboards are supported on the following Internet browsers: Internet Explorer (IE 8 – IE 10) Firefox (16.x – 22.x) Google Chrome (22.x – 28.x)
Please clarify public vs private dashboards. Upon publishing your VFA data, the VFA makes a link available on your College Profile page that takes the public user
to your Outcomes data. The Outcomes data are presented to the user in a packaged PDF report, again made available on a link from your College Profile page. All information that is contained in your Outcomes Dashboard is packaged into this public PDF report. The Outcomes data includes the “All Students” (not disaggregated) counts of the following: Developmental Education Progress, Two-Year Progress, and Six-Year Outcomes Measures; CTE Profile & Completer/Leaver Outcomes, and ABE Cohort Outcomes. Raw data, disaggregated data, and the benchmarking tool and data will not be available to the public. We will send the user group a sample in the next few weeks.
Will the new dashboards show data from previous years’ data collections? Yes. The new Outcomes and Benchmarking dashboards will render all data from Year One (2013-14 data collection)
onward. • The Public Dashboard Report – a PDF report showing the graphs or your published Outcomes data –is only
available for VFA Year Two (2014-15 data collection) ONWARD. To the public, the link to this report will always be for the most recent year.
Can we view the names of benchmarking institutions selected? Currently, the tool does not return a list of the colleges that are in the benchmarking group. AACC is working with
the VFA Oversight Board and the dashboard developers on both the policy and technical changes needed to make this happen. We have receive this request and are working toward fulfilling it.
• Please note that the benchmarking tool will always use selector criterion to establish the colleges in the benchmarking group; colleges will not be able to pick individual colleges against which to benchmark.
Questions &
Answers
Questions During the Webinar (cont.)
Raw File Upload Process
More info about the "raw data" uploadThe Raw Data upload process requires the college to upload 5 files:
1. Student demographics, 2. Student developmental education placement, 3. Student course data, 4. Completions at two years, and 5. Completions at six years.
The data will be extracted, translated and loaded into the VFA database. After the ETL is completed, the data from the five files will be permanently
deleted from the system. The VFA does not keep student-unit record data.
If the raw data upload method will become the "method of choice," will the bulk upload method eventually be abandoned?
The VFA has no plan to abandon any of the three data input methods at this time. We anticipate that many users will move toward the raw upload method because it is a process where data extraction is streamlined and methodology is preserved. Like all data initiatives, though, we understand that users work differently and have preferences for how to enter data.
Questions &
Answers
What data can be downloaded to Excel? All data that you entered for your college can be downloaded to an Excel document,
so that the college may manipulate their own data. Downloading benchmarking data requires more development and is not available for Year Two.
How is the VFA handling the "English/Reading" combination topic going forward? For Year One, the VFA allowed colleges who offered English and reading under a
combined topic to report the progress and outcomes of these students under an English/Reading combination subject. There was confusion and errors in reporting, so for Year Two we ask all colleges who offer the combined subject to report it under the English subject. For these colleges, the reading subject area will be null.
When will the new versions of the VFA Metrics Manual and User Guide be released? There are aspects of the Metrics Manual and User Guide that require all other data
collection development to be completed. Therefore, the VFA Metrics Manual will be released on November 3rd, when data collection opens. The User Guide will be released soon after.
Is there an easy way to report SAM (Student Achievement Measure) using VFA? The data reported for SAM’s associate/certificate model is a roll-up of the VFA’s Six-
Year Outcomes for the full-time and part-time, credential-seeking cohort. If a college reports the VFA, it can easily report SAM. The SAM folks have been working with their developers on a file that helps with getting VFA data into SAM. We will update you on this post-October 20 when SAM launches. See the Resources section of the SAM website for more information: http://www.studentachievementmeasure.org/resources#
Questions &
Answers
Questions During the Webinar (cont.)
Misc.