Innovations in Career-Tech Assessment

33
Innovations in Career-Tech Assessment Presented by: Sandra Pritz, Senior Consultant National Occupational Competency Testing Institute

description

Innovations in Career-Tech Assessment. Presented by: Sandra Pritz, Senior Consultant National Occupational Competency Testing Institute. Why should career-technical programs assess academic skills?. State/national academic standards demand attention. - PowerPoint PPT Presentation

Transcript of Innovations in Career-Tech Assessment

Page 1: Innovations in  Career-Tech Assessment

Innovations in Career-Tech Assessment

Presented by:Sandra Pritz, Senior Consultant

National Occupational Competency Testing Institute

Page 2: Innovations in  Career-Tech Assessment

Why should career-technical programs assess academic

skills?• State/national academic standards

demand attention.• High school students must pass

academic tests for graduation.• Workplace demands for technical

literacy are increasing.• Academics in context tend to be

student-friendly.

Page 3: Innovations in  Career-Tech Assessment

How Should Career-Technical Programs Assess Academic Skills?

• In the context of the students’ career-technical program

• In alignment with state and national academic standards

• In ways that help students translate to other tests they will need to take

• Embedded In their career-technical tests, such as end-of-course or program

Page 4: Innovations in  Career-Tech Assessment

Alternative assessments

• Ask students to perform, create, produce something

• Elicit applications of higher-level thinking and problem-solving skills

• Connect with tasks that are meaningful in terms of the related instruction

• Engage students in authentic real-world tasks

Adapted from A Practical Guide to Alternative Assessment (Herman,Aschbacher, and Winters, ASCD,1992)

Page 5: Innovations in  Career-Tech Assessment

Portfolio Assessment

• Involves assessing a collection of student work

• Establishes a purpose, criteria (for what gets put in, when, and by whom), and how the collection or individual pieces will be evaluated

• Engages students in assessment through gathering evidence of their progress and achievements that meet the criteria

Page 6: Innovations in  Career-Tech Assessment

Value Added to Career-Technical Programs by Tests

• External and effective• Critical competencies

benchmarked to industry standards

• Correlated to state and national academic standards

Page 7: Innovations in  Career-Tech Assessment

Effective Tests-

• are based on standards• provide transparent links between

standards, learning objectives, and learning activities

• help translate what is required into terms the learner can address

• complement projects, portfolios, alternative assessments

Page 8: Innovations in  Career-Tech Assessment

Critical Competencies Are Central

• Students should recognize them as learning objectives with standards, move toward self-assessment.

• Teachers should teach to them in learning activities.

• They should be assessed—in multiple ways—with the results used to help focus instruction, for the group and for individuals.

Page 9: Innovations in  Career-Tech Assessment

Process to Use Multiple Tests to Improve Instruction

• Written tests—practice at all levels by including in assignments; mirror types of items on HSTW tests, include academics embedded in the technical industry/program.

Page 10: Innovations in  Career-Tech Assessment

Electronic Technology WrittenStudents must be able to comprehend the following:

•Read diagrams

•Recognize symbols

•Apply Ohm’s LawMath Calculations needed:

•Solve parallel and series circuits.

•Solve algebra equations

•Manage inverse relationships

Page 11: Innovations in  Career-Tech Assessment

Performance Tests

- Practice by including scenarios in assignments

- Mirror real job tasks, time, tool selection, procedures

- Open-ended, can apply a rubric, and can easily assess at higher levels

- Some aspects yes/no, some scaled, some qualitative judgment by expert

Page 12: Innovations in  Career-Tech Assessment

Precision Machining Performance

 1.You will receive a piece of cold rolled steel

1.125” diameter x 6.62 long.

2.Following industrial safety rules, machine the part on the lathe according to the specifications given on the drawing provided. Check tools for condition and sharpness.

Requires student to: -evaluate drawing to develop a work plan -select speeds and feeds - read blueprints

Page 13: Innovations in  Career-Tech Assessment

Coupling Written and Performance Tests

• Reflects reality better than one or the other alone

• Provides for multiple forms of assessment• Gives all students a chance to succeed • Enables mutual reinforcement of learning• States and schools using NOCTI tests

have documented improvement in instruction from using analyses of test results.

Page 14: Innovations in  Career-Tech Assessment

Career and Technical Skills

AcademicSkills

Academic Skills

+ Career Technical Skills

Comprehensive Assessment

Academic Skills Measured in Context

Page 15: Innovations in  Career-Tech Assessment

Assessing Academic Skills in Context

• NOCTI tests of occupational competencies also test some embedded academics.

• Math, science, and language arts are required to complete most occupational items and performance tasks.

Page 16: Innovations in  Career-Tech Assessment

Needs in Reading

• Comprehend and interpret the materials of the occupational field

• Compare, analyze, synthesize technical materials into a new set of information

Page 17: Innovations in  Career-Tech Assessment

Needs in Math and Science

• Use analysis techniques• Solve problems that require

integration of more than one math concept and/or multiple steps

• Estimate and check answers for reasonableness

• Solve open-ended problems

Page 18: Innovations in  Career-Tech Assessment

Science -- Carpentry

• 4% of the written test items and 14% of the jobs of the performance test match NSES science standards at the 9-12 grade level.

• Example question: Concrete wall forms must be made __________

(A. strongest at the top, B. strongest in the middle, C. strongest at the bottom, D. uniform throughout).

Page 19: Innovations in  Career-Tech Assessment

NOCTI Academics Correlation Project

• 7 NOCTI job-ready tests correlated to academic standards for math, science, and language arts- National standards (NCTM, NSES,

NCTE)- State standards (NY, CA, FL)

• Several business/industry tests correlated to national standards

Page 20: Innovations in  Career-Tech Assessment

Academics Identified

• Written tests contain embedded academics that match standards in all three disciplines.

• Performance tests contain a higher percentage of embedded academics than written tests, and the items tend to be at higher levels of proficiency.

Page 21: Innovations in  Career-Tech Assessment

Employability Skills

• Competency lists like SCANS can be used to embed matching employability skills in tests

• Bloom’s taxonomy* can be used to embed higher order skills in tests

* Bloom’s taxonomy progresses from knowledge to comprehension,application, analysis, synthesis, and evaluation

Page 22: Innovations in  Career-Tech Assessment

Bloom’s Taxonomy -- Carpentry

Written Performance

Knowledge 37% 14%

Comprehension 25% 0%

Application 27% 14%

Analysis 10% 0%

Synthesis 1% 72%

Evaluation 0% 0%

Page 23: Innovations in  Career-Tech Assessment

SCANS Skills -- Carpentry

Written46%

Performance86%(1 or

more)Resources 6%

Information 8%

Systems 5% 50%

Technology 27% 86%

Page 24: Innovations in  Career-Tech Assessment

Sample of Carpentry Written Test Item with Technical and Academic Content

at Proficient Level

If a benchmark is established at 100’-0” and the bottom of the foundation is at 96’-4”,the bottom of the foundation is _______ the benchmark. (A. 6’-4” above, B. 3’-8” above, C. 6’-4”below, D. 3’-8” below)

Math: proficient, 2 national standard matchesReading: basic, 1 national standard matchBloom’s: analysis SCANS: acquires and

evaluates information

Page 25: Innovations in  Career-Tech Assessment

Sample Carpentry Performance Job At Advanced Technical Level

with Integrated Academics

Using the appropriate tools, equipment, and material, install an interior window frame in the rough opening provided. Case the top and part of both sides of the frame using mitered corners. Use a table saw or power miter box to cut trim.

Math: proficient; 4 national standard matchesReading: advanced; 1 national standard matchBloom’s: synthesis SCANS: selects technology

Page 26: Innovations in  Career-Tech Assessment

Ways NOCTI can help

• By providing performance as well as written tests, NOCTI can help assess academics at higher levels.

• NOCTI can respond to the need to reinforce academic and employability instruction through assessment, can customize tests and test reports to isolate academics scores.

• NOCTI can provide states the service of cross-correlating its tests to their state standards.

Page 27: Innovations in  Career-Tech Assessment

Ways NOCTI can help (cont.):Workplace Readiness Test

• NOCTI Workplace Readiness Test measuring employability skills is being up-scaled

• A large and stellar Subject Matter Expert Team has rated the critical competencies

• Career Cluster Foundation Skills will be assessed

Page 28: Innovations in  Career-Tech Assessment

Ways NOCTI can help (cont.):Workplace Readiness Test

• Employability skills are focus of NOCTI best-selling Workplace Readiness Test since 1996.

• Up-scaled test in process to meet changed needs of business & industry– Research and synthesis of standards literature (e.g.

MSSC, SCANS, EFF)– Review by >30 leaders, 3 focus groups– New assessment to be developed by fall– Will be formatted to align with career clusters

Page 29: Innovations in  Career-Tech Assessment

For Tech Prep Programs

• NOCTI’s critical competency lists can be used to document curriculum alignment for secondary and post-secondary articulation.

• NOCTI can validate student mastery of competencies at the secondary level for transition to and possible credit at the post-secondary level.

Page 30: Innovations in  Career-Tech Assessment

States Career Cluster Initiative Moves to Assessment Stage

• NASDCTEc has named NOCTI a developer and the provider of its cluster, pathways, foundation tests.

• NOCTI is administering the test developed for the National Health Sciences cluster.

• NOCTI is developing 3 pathways tests in construction for PA, to be used nationally.

• NOCTI is developing a Workplace Readiness foundation assessment.

Page 31: Innovations in  Career-Tech Assessment

What Do I Receive from NOCTI?

• Score Reports– Coordinator’s Report– Individual Student Reports– Customized Reporting Available—

e.g. integrated academics

• Statistical Information– Group Data– Site Data– State Data– National Data

• Student Certificates

• Personalized Contact Assistance

All results are broken down into

duty, category and total scores.

Note:

Page 32: Innovations in  Career-Tech Assessment

Job Ready Welding Written Assessment

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Ther

mal

Cut

ting

Proc

esse

s

Hea

tTr

eat/M

etal

lurg

y/D

isto

rtion

Con

trol

Wel

ding

Ter

ms/

Blue

prin

tR

eadi

ng

Wel

ding

Insp

ectio

n

Wel

ding

Con

sum

able

s

Safe

ty

Pow

er S

ourc

e Pr

inci

ples

Oxy

fuel

Wel

ding

&Br

azin

g

Shie

lded

Met

al A

rcW

eldi

ng

Gas

Met

al A

rc &

Flu

xC

ored

Arc

Wel

ding

Gas

Tun

gste

n Ar

cW

eldi

ng

Mea

surin

g In

stru

men

ts

Wel

ding

Qua

lifica

tions

&Pr

oced

ures

Wel

ding

Cos

ts

Tota

l

Pre-Test Scores Post-Test Scores

Pre and Post Test Gain Analysis

Page 33: Innovations in  Career-Tech Assessment

Contact NOCTI for more information and

to discuss your interests---

www.nocti.org or 1-800-334-6283

Thank You