Welcome Workshop Goals: Understand data sources Learn the basic components of an effective...
-
Upload
amber-maldonado -
Category
Documents
-
view
214 -
download
0
Transcript of Welcome Workshop Goals: Understand data sources Learn the basic components of an effective...
Welcome
Workshop Goals:Understand data sources
Learn the basic components of an effective assessment
Analyze and effectively use data
Overview of NOCTI
Over 40 years serving the CTE community Non-profit organization
Owned and operated by the 50 states
Works with both secondary and post-secondary populations
The Whitener Group (a NOCTI subsidiary) provides services to industry
NOCTI Customer Breakdown
Industry Connections
NOCTI Products
Supplemental Products
Over 80 Job Ready tests offered
30 Pathway tests in 10 clusters available
Over 90 private industry employment and/or advancement tests delivered
20 teacher-level assessments
Assessments Study guides for all Job Ready
and Pathway assessments Countdown Kits offered for 40
titles Successful Schools-Successful
Programs Guide
NOCTI Job Ready Assessments
Assess technical skills at the occupation level (e.g., Accounting, Carpentry, Pre-Engineering)
Measure aspects of occupational competence such as factual and theoretical knowledge
Target secondary and post-secondary program completers
Assessment Blueprints
Blueprints include:
Competency List
Content Distribution
Sample Items and Jobs
NOCTI Job Ready Assessment Blueprint
Number of Questions: 222 Areas Covered:
AccountingAccounting
Assessment Blueprints to Prepare
How can I prepare my students for the NOCTI assessment?Assessment Blueprints!!!
Ideas for using blueprints in the classroom:
Use throughout the school year as a syllabus and guide for review
Involve the students: have them calculate the number of questions in each content area
Send a copy home with students to share with parent
Assessment Results
NOCTI’s Standard Score Reporting Package: Individual Score Reports Group Score Reports Comparative Data
Other Reports Available: Task-Link Reports Integrated Academic Reports
NOCTI Standard Score Reports
Individual Score Individual Score Reports Reports
•Provided for each test taker•Scores by duty and total•Pre-/post-test comparisons•Comparative data
Individual Score Individual Score Reports Reports
•Provided for each test taker•Scores by duty and total•Pre-/post-test comparisons•Comparative data
NOCTI Standard Score Reports
Group Score ReportGroup Score Report
•Provided for the testing group•Duty and total scores for each participant
Group Score ReportGroup Score Report
•Provided for the testing group•Duty and total scores for each participant
NOCTI Standard Score Reports
Analysis of Scores Analysis of Scores ReportReport
•Provided for the testing group•Comparative data•Criterion-referenced cut score•Other statistical information
Analysis of Scores Analysis of Scores ReportReport
•Provided for the testing group•Comparative data•Criterion-referenced cut score•Other statistical information
Using Data to Improve Instruction
Using Data to Improve Instruction
Collect Data:Become familiar with
your data
Understanding Your Data
School Program Teacher Class
Student Standards
National State District/System
What the Data Reveal
Achievement gapsDifferences between subgroupsTrends Progress
Assessment
Includes process of collecting, analyzing, and interpreting information
Determines the degree to which students are becoming competent and meeting standards
Occurs at many levels (e.g., classroom and program assessments to state and national assessments)
Good Assessments
Valid Test items measure what they are supposed to
measure: the important content or concepts that were taught and that the student must know
Test is relevant and based on current standards Subject matter experts develop and evaluate
test items
Good Assessments
Reliable Test and its items produce a similar
score over time Scores are consistent, not contradictory Scores are not affected by chance
Student Assessment
On-going with frequent opportunities for feedback Use a variety of different types of assessment (e.g.,
written and performance, formative and summative) during the year
Summative Assessment
Pretests (covering all major content areas) are used to identify entry skills and measure gain
End-of-program or annual test Broader in scope, for comparison of groups
Formative Assessment
Continuous correction from quizzes, homework, projects, participation
Use practice items for instructional revision before summative assessment
Building an Assessment
Test specification matrix or “Blueprint” National Standards Duties and Tasks Written/Performance Weighting Panel of Experts
Step 2
Analyze Data:Draw conclusions from data
Classroom Example
A Carpentry Class
Areas on a NOCTI Carpentry Test
Safety
Tools and Accessories
Blueprint Reading and Estimation
Foundations, Forms and Concrete
Rough Framing
Exterior Finish
Interior Systems Installation
Interior Finish
Carpentry Related Mathematics
Individual Carpentry Pre-Test
Carpentry Pre-test Areas Pre-test % CorrectStudent 1
Post-test % CorrectStudent 1
Safety 54.6 --
Tools and Accessories 30.0 --
Blueprint Reading and Estimation 41.1 --
Foundations, Forms, and Concrete 36.4 --
Rough Framing 53.8 --
Exterior Finish 36.4 --
Interior Systems Installation 40.0 --
Interior Finish 27.3 --
Carpentry Related Mathematics 35.9 --
Total 40.0
Carpentry Class Pre-Test
Carpentry Pre-test Areas
Student 1 % Correct
Student 2 % Correct
Student 3 % Correct
Student 4 % Correct
Student 5 %
Correct
Student 6 %
Correct
Class Average % Correct
Safety 54.6 45.5 36.4 27.3 27.3 54.5 40.9
Tools and Accessories
30.0 20.0 10.0 20.0 10.0 30.0 20.0
Blueprint Reading and Estimation
41.1 17.6 23.5 23.5 29.4 35.3 28.4
Foundations, Forms, and Concrete
36.4 54.5 45.5 45.5 36.4 54.5 45.5
Rough Framing 53.8 38.5 30.8 38.5 30.8 53.8 41.0
Exterior Finish 36.4 36.4 27.3 36.4 36.4 45.5 36.4
Interior Systems Installation
40.0 60.0 40.0 60.0 60.0 80.0 56.7
Interior Finish 27.3 9.1 18.2 27.3 9.1 27.3 19.7
Carpentry Related Mathematics
35.9 45.5 27.3 36.4 36.4 54.5 39.4
Total 40.0 34.0 28.0 33.0 29.0 46.0 35.0
Carpentry Class Post-Test
Carpentry Post-test Areas
Participant Group Site State Nation
Safety 100.0 86.4 79.5 72.5 75.5
Tools and Accessories 91.7 79.2 78.2 70.7 70.6
Blueprint Reading and Estimation
100.0 95.0 83.8 64.3 66.4
Foundations, Forms, and Concrete
92.3 69.2 65.8 51.6 52.9
Rough Framing 97.4 71.8 69.8 63.1 64.1
Exterior Finish 91.7 64.6 67.6 58.1 57.8
Interior Systems Installation 87.5 53.1 61.9 55.6 56.5
Interior Finish 87.5 79.7 75.6 66.6 67.0
Carpentry Related Mathematics
100.0 56.3 68.2 66.1 67.0
Total 94.3 72.0 71.4 62.7 63.4
Thank You!
Questions?
www.nocti.org