TCAP Achievement TCAP Achievement TransitionTransition

Laura AtkinsLaura Atkins

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

Questions to Explore

• When the new TCAP achievement test is administered in 2010?

• Will it be a criterion-referenced assessment or a norm-referenced assessment?

• Will the test administered in 2010 have a close correlation to the state’s new curriculum standards?

Questions to Explore

• Will the test administered in 2010 be NAEP like?

• Will roll-outs be conducted during the spring of 2009 to prepare leadership teams in school systems for working with teachers as assessment processes change?

Questions to Explore

Would you consider holding a state testing conference or regional roll-outs for this unique transitional period resulting from implementation of new curriculum standards and a new assessment process in 2009-2010?

Questions to Explore

How can TVASS information be used most effectively to develop the instructional program to meet student needs as curriculum standards change and added rigor is applied?

Questions to Explore

• What plans (if any) are in place to change the K-2 assessment process?

• What plans (if any) are in place to change the assessment process for ELL and SPED students?

• Discuss security procedures.

New Achievement New Achievement Contract & Contract & TimelineTimeline

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

New Achievement Contract

• Grades K-2 (Optional NRT) • Grades 3-8 (Mandated CRT)• Modified Academic Achievement

Standards Assessment (MAAS)• English Linguistically Simplified

Assessment (ELSA)

New Vendors

•Pearson Educational Measurement (PEM)

ReportingLogisticsPearson AccessK-2 testing

•Educational Testing Service (ETS)Write custom items for TN grades 3-8

New Achievement Contract

• October 1, 2007: New content standards to Assessment

• December 2007: Release of Achievement RFP

• January 2008: SBE approval of content standards

• January to March 2008: Creation and review of New Items by CTB

New Achievement Contract

•March 2008: New Achievement Contract begins

•March to August 2008: Continued creation and review of New Items by Pearson/ETS (Vendor Transition including Item Bank transfer)

•June 2008: Review of New Passages (ETS)

New Achievement Contract

•August 2008: Review of New Items

(ETS)

•December 2008: Passage Review

(ETS)

•February 2009: Item Review (ETS)

New Achievement Contract

• TCAP T: Spring 2009 (Operational items assess current SPIs)

• TCAP A: Spring 2010 New Operational Assessment (Assess revised SPIs for RLA, Math, & Science)

• TBA: New Cut Scores Established (Standard Setting Required)

ResourcesResources

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

TCAP T Time Limits

Grade 3SUBTEST TEST ITEMS TIME LIMIT

Science, Part 1 39 55

Science, Part 2 38 55

Social Studies, Part 1

39 55

Social Studies, Part 2

38 55Grades 4-8SUBTEST TEST ITEMS TIME LIMIT

Science, Part 1 39 50

Science, Part 2 38 50

Social Studies, Part 1

39 49

Social Studies, Part 2

38 48

Same as

Form S

(Spring

2008)

TCAP Achievement (3-8)

State Performance Indicator Chart

Spring 2009http://www.state.tn.us/education/assessment/tsachrcspi.shtml

TCAP Achievement (3-8)

Third Grade Pilot

Pilot results are in… Results are positive!ALL students grades 3 – 8 will now use a separate answer document.ALL students (3 – 12) may mark in test booklets.

TCAP Achievement (3-8)

Practice Answer DocumentPractice Answer Document •Practice answer document created as a resource for teachers of grades 3-8.

•Designed for practice in filling in bubbles on a document similar to the TCAP Achievement answer document. Available online at:Available online at:

http://www.state.tn.us/education/assessment/doc/Grade3_Ansdoc.pdfhttp://www.state.tn.us/education/assessment/doc/Grade3_Ansdoc.pdf

TCAP Achievement (3-8)

TCAP Achievement (3-8)

Unique Identification NumbersUnique Identification Numbers Beginning with the Summer 2008 Gateway/EOC test administration, both the student social security number and the State unique identification number (EIS) will be captured in an effort to phase out the use of social security numbers.

TCAP Achievement (3-8)

For information about Curriculum roll-out

sessions contact the Office of Curriculum and

Instruction.

615/532-6300

TCAP Achievement

The Office of Curriculum and Instruction (TDOE) is developing companion documents to the new curriculum standards. User’s Guide to the Tennessee Mathematics Curriculum FrameworkScott Eddins 615/741-3073scott.eddins@state.tn.us

TCAP Achievement

•Overview •General Organizational Format•Webb Levels and the TN Curriculum •Vision of Standards Based Science,•Technology Engineering & Mathematics Education

•Implementation

User’s Guide to the Tennessee Mathematics Curriculum Framework

TCAP Achievement

•Formative and Summative Assessment Mapping

•Webb Depth of Knowledge Levels•Alignment with ACT & NAEP•Comparison of Former & New Standards

•Vocabulary in Focus •Content in Focus

User’s Guide to the Tennessee Mathematics Curriculum Framework

State Performance Indicator Chart

Spring 2009

http://www.state.tn.us/education/assessment/tsachrcspi.shtml

TCAP Achievement (3-8)

Online Resources

• Item Samplers & Practice Tests (Current)

• Educators Guide to Test Interpretation (will be revised)

• Teacher Directions (will be revised)

• Frequently Asked Questions (will be revised)http://www.state.tn.us/education/assessment/

tsachhome.shtml

K-2 TestingK-2 Testing

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

Grades K-2 •“Shelf Test” (Norm-Referenced)•Optional •No longer use TerraNova •Use Stanford Achievement Test Spring 2009

K-2 TestingNew Stanford 10

Levels•SESAT 2 (Kindergarten)•Primary 1 (First Grade)•Primary 2 (Second Grade)

K-2 TestingNew Stanford 10

Subtests•Reading•Mathematics•Language•Spelling •Listening•Science•Social Studies

K-2 TestingNew Stanford 10

•Flexible time limits

•Content reflects national standards

•Full-color

•Easy-to-navigate

•Normed in 2002

•Must order before Thanksgiving!

K-2 TestingNew Stanford 10

New AssessmentsNew Assessments

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

New Assessments

•Modified Academic Achievement Standards (MAAS)

•English Linguistically Simplified Assessment (ELSA)

Achievement - ELSA

English Linguistically Simplified Assessment (ELSA)

• Available for ACH - Spring 2010• Intended for Limited English

Proficient (LEP) students • ACH test in “simplified” English

Achievement – ELSA

Characteristics of simplified language assessments

• Reduce wordiness• Use common words• Avoid words with multiple

meanings• Use concrete words • Reduce complexity of sentence

structure• Use the simplest verb forms• Use less complex context

•Reduce linguistic barriers to measurement of the intended knowledge or skill

•The knowledge or skill being assessed must not be changed

•Challenging vocabulary should be used only when it is relevant to the knowledge or skill being assessed

•Same ACH test (e.g., large print and Braille versions)

Achievement– ELSA

Achievement– MAAS

April 2007, federal regulations finalized an option for an alternate assessment based on modified academic achievement standards

Resource:National Center on Educational Outcomes (NCEO)http://cehd.umn.edu/AAMAS.html

Achievement– MAAS

•TDOE adopts Modified Academic Achievement Standards (MAAS) Assessment

•Pearson/ETS contracted to field test MAAS items in Spring 2009

•MAAS to be operational in Spring 2010

•MAAS becomes one component of the NCLB Adequate Yearly Progress (AYP)

Achievement– MAAS

•Available for all 3-8 content areas (Reading and Language Arts, Mathematics, Science, and Social Studies)

Achievement– MAAS

Assess same content standards as ACH• Modify test format• Modify test questions• Modify achievement standards

Achievement– MAAS

•Referred to as the 2% solution•Students for whom the alternate assessment is too easy

•Students who cannot demonstrate achievement on TN’s ACH

•Design an assessment with more valid score interpretation for the 2% population

Achievement– MAAS

•Develop modified achievement standards (performance levels)

•For on-grade-level content standards

•Challenging for eligible students•Assess a student’s mastery for grade-level content

•But at a less difficult level than grade-level achievement standards (performance levels)

•EligibilityGuidance from State

Department of Education (Special Education Division)

IEP Team•Content Standards (same as ACH)•Achievement Standards (specific to MAAS)

Achievement– MAAS

•State must establish clear and appropriate guidelines for IEP teams to determine eligibility.

•Students may be assessed on the modified assessment in one or more subjects.

•Eligible students may be from any of the disability categories listed in the IDEA.

Achievement– MAAS

Achievement– MAAS

•State must ensure students’ IEPs include goals for content standards for the grade in which the student is enrolled.

•Students must have access to the curriculum and instruction for the grade in which the student is enrolled.

Achievement– MAAS

•Students cannot be precluded from attempting to complete the requirements for a regular high school diploma.

•“A student’s IEP Team continues to determine how the student will participate in State and district-wide assessments.” (OESE)

Achievement– MAAS

• Achievement expectations may be modified to provide a better method for assessing growth over time for the 2% population.

• Achievement standards should promote challenging expectations for this population.

•After performance level descriptors for each achievement level are obtained, appropriate cut scores can be set for each proficiency level.

•Educator committees will be convened to recommend cut scores for this population on this new assessment.

Achievement– MAAS

Test design changes include:• Three answer option choices• Shorter reading passages• Reduced reading load• Simplified language• Decreased cognitive

complexity

Achievement– MAAS

Format changes include: • Print styles simplified (modified

use of italics, underlining, bold-facing)

• Charts, graphs, and tables simplified

• Print type size enlarged• Essential information reinforced• Fewer items on each page

TCAP Achievement– MAAS

Pearson AccessPearson Access

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

November/December 2008 • TDOE will upload student

data to PEMSolutions (pre-id)

Pearson Access

Spring 2009 - Testing Coordinators will use the PEMSolutions tool of Pearson Access to:

•manage student data (before and after testing)•access reports

November/December 2008 • TDOE will upload student

data to PEMSolutions (pre-id)

Last name, First name, MIStudent ID numberBirthdateGenderEthnicityCode A/BGrade

Pearson Access

Visit the Computer Lab for a demonstration

• Login/Logout & Reset Password (use same passwords as order entry)

• Administrative Management• Student Demographic & Enrollment

Data• Test Results• Support• FAQ & much more

PearsonAccess PEMSolutions

RANDA SolutionsRANDA Solutions

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

TCAP Achievement ProgramTransition

•TurnLeaf (CTB) available through June 2009

•Our online applications site is down for upgrades:

https://www.eval.k-12.state.tn.us/login

https://tnors.turnleaf.com

TCAP Achievement ProgramTransition

•New Tech Vendor (upgraded & enhanced)Achievement OrdersRIsSGLsTraining Videos

https://tdoe.randasolutions.com

TCAP Achievement (3-8)

Achievement Orders:•Please verify all quantities (check Braille and LP).•For late orders or changes contact:•Freddie Summers 615/532-1249freddie.summers@state.tn.us •Order #s and EIS upload information will be used to decide how many tests to order for each school. Systems must continuously update EIS

Test WindowTest Window

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

Spring 2009 Test Window

•For the school year 2008 - 2009, school systems can now select 6 consecutive school days to administer the TCAP Achievement Test.

•This change has been made to accommodate school systems with large ELL and SPED populations.

•The six consecutive school days must fall within the state testing window.

Spring 2010 Test Window

Guidance from the U.S. Department of Education concerning the reporting of AYP information states that beginning with the school year 2009-2010, the State Department of Education will be required to report AYP information no later than 14 days before the start of the school year.

This new regulation will require reporting AYP information at least two weeks earlier than the current AYP reporting date.

Since some Tennessee schools open as early as July 14, we must adjust the TCAP Achievement Test dates in order to comply with the new directive.

Spring 2010 Test Window

We have drafted two possible solutions that will have the least impact on the Spring 2010 administration of the TCAP Achievement Test:

Option 1: March 29 - April 9, 2010 (two-week window)

Option 2: March 22 - April 9, 2010 (three-week window)

Spring 2010 Test Window

Easter Sunday – April 4, 2010Good Friday – April 2, 2010

Option 1: March 29 - April 9, 2010 (two-week window)

Option 2: March 22 - April 9, 2010 (three-week window)

Spring 2010 Test Window

March21

March22

March23

March24

March25

March26

March27

March28

March29

March30

March31

April 1

April 2

Good Friday

April 3

April 4

Easter Sunda

y

April 5

April 6

April 7

April 8

April 9

April 10

Sun Mon Tues Wed Thurs Fri Sat

2008-2009 Achievement Window2008-2009 Achievement Window

March 30 – April 24March 30 – April 24

Spring 2010 Test Window

Test SecurityTest Security

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

Test Security

T.C.A. 49-1-607 states:T.C.A. 49-1-607 states:Any person found not to have Any person found not to have followed security guidelines…followed security guidelines…

•placed on immediate placed on immediate suspensionsuspension•grounds for dismissalgrounds for dismissal•includes dismissal of includes dismissal of tenured employeestenured employees•grounds for revocation of grounds for revocation of state licensestate license

Test Security

1) Adopt a locally monitored test security policy that incorporates, at a minimum, these State Test Security Guidelines. This policy should include a Testing Code of Ethics for personnel to sign and leave at the district office for documentation.

State Test Security GuidelinesThe Public School Systems, State Special, and Non-Public Schools MUST:

Test Security

2) Train all personnel involved in the testing process on State Test Security Law, Security Guidelines, local policy, and test administration procedures; retain training documentation for system records.

3) Implement check-in, check-out, and quantity verification procedures for all test materials at the system level, at the school level, and for each test session.

Test Security

4) Restrict handling of test materials to authorized personnel at all times.

5) Implement policies and procedures to prohibit all personnel from obtaining knowledge of test items or passage content before, during, and after testing.

Test Security

6) Return test materials immediately after each test session and when the entire administration is completed. Store test materials in a centrally located locked room that is inaccessible to unauthorized persons.

7) Create a secure, yet positive, environment for testing.

Test Security

8) Conceal or remove all instructional or reference materials in the test setting that are related to the content area being assessed, such as maps, posters, student samples, bulletin board items, familiar study aids such as graphic organizers, models, or number lines that relate to subject content.

Test Security

Any information regarding the content Any information regarding the content being measured on the test being measured on the test MUSTMUST be be concealed during the test administration. concealed during the test administration. Including but not limited to the following:Including but not limited to the following:•Maps •Charts•Posters •Student Work •Bulletin Board Items•Writing Formulas •Word Lists•Parts of Speech

•Graphic Organizers•Number Lines•Definitions•Word Walls•Displays•Models

Test Security

9) Turn off all electronic communication devices (cell phones, pagers, PDAs, etc.) in the test setting. Ensure proper calculator use as outlined in the Test Administration Manual, making sure that calculators are cleared before and after administration of each test.

Test Security

10) Confirm each student is the person named on the answer document for every testing session. A photo ID may be required if administrators are not responsible for normal classroom instruction.11) Require Test Administrators and Proctors to carefully adhere to all test administration and accommodation instructions, following appropriate schedules and time limits, outlined in all test directions.

Test Security

12) Require Test Administrators and Proctors to remain with the students and be observant and nondisruptive throughout the testing session.

13) Prohibit coaching students in any way during State assessments.

14) Prohibit reading test items and passages by anyone other than those reading to students being tested through read aloud, unless indicated in test instructions or accommodations.

Test Security

15) Ensure students respond to test without assistance from anyone.

16) Ensure that test items are not reproduced, duplicated, or paraphrased in any way, for any reason, by any person.

17) Maintain confidentiality of student-specific accountability demographic information and test results at all times.

Test Security

18) Document test security concerns, including missing materials, on the Breach of Testing Security Report form.

19) Failure to report a breach of security compromises the integrity of the testing process and should be treated as a breach of testing security.

Item ReviewItem Review

Office ofOffice ofAssessment, Assessment, Evaluation, & Evaluation, & ResearchResearch

Item Review

Purpose:• Obtain Tennessee educator input• Support the content validity of the

Achievement TestTwo sets of committees

• content reviews• bias and sensitivity reviews

Item Review

Item Content Review CommitteesGrades 3-4 (4 content areas)Grades 5-6 (4 content areas)Grades 7-8 (4 content areas)

Item Bias Review CommitteesReading and Language Arts Grades 3-8Mathematics Grades 3-8Social Studies Grades 3-8Science Grades 3-8

Item Development Cycle

Items selectedfor test

Items field-tested

Items developed by ETS

Items reviewed by TN educators Field-test results analyzed

Item Review

• ETS brings a range of items developed for all performance indicators.

• Following a structured processeducators read all items

individuallythe committee discusses items

• Educators mark their individual comment form with input on items

•Each committee reviews all ACH items and all MAAS items for a grade span (3-4; 5-6; 7-8).

•Each committee is comprised of teachers of regular and special education students, as well as teachers of limited English proficient (LEP/ELL) students.

Item Review

Item Review

For each item, educators will recommend:

• accept “as is”• accept with edits• reject

Item Review

Guiding Questions• Does the item measure the SPI?• Does the item reflect sound

instructional practice at the grade level?

• Is the item clearly and concisely stated?

• Does the item have one, and only one, clearly correct answer?

Item Review

Guiding Questions• Does the item measure the SPI?• Does the item reflect sound

instructional practice at the grade level?

• Is the item clearly and concisely stated?

• Does the item have one, and only one, clearly correct answer?

Item Review

Guiding Questions• Are the other answer options

clearly incorrect? • Are all stimuli and/or graphics

clear and understandable, and important to answering the item?

• Is the item appropriate for all groups of students?

Questions???

Contact Information

Laura Atkins – Achievement(615) 253-5208 laura.atkins@state.tn.us

Questions???

ContactsContactsLaura Atkins - AchievementUma Natarajan – NAEPLori Nixon – Special Ed. Assessment

(portfolio/MAAS)Steven Nixon – ELDA/ELSA/ELL AssessmentMarcy Tidwell – Assessment LiteracyFreddie Summers – Achievement OrdersMary Taylor – GWY/EOC/ACT/Plan/ExploreCharlotte Woehler – Writing

615/741-0720