Central Florida Assessment Collaborative

Item Writer & Reviewer Training

www.cpalms.org

Steps for Assessment Development

Item Writing• Uses the course description and item

specifications document to create items• Items are written to cognitive complexity

levels that the benchmark/standard warrants• Items are written to align with the item

specifications• Items are written according to ‘good item

writing rules’

Steps for Assessment Development

Item Review• Were the Item Writing Guidelines followed?

• Is the item free of bias and sensitivity issues?• Does the item appear to measure what’s specified in the

item spec?• Is item truly appropriate for the students taking the

course?• Is the language complexity appropriate for this course?• Has correct grammar and punctuation been used in the

construction?• Is accurate scoring information included?• Is there alignment with the complexity of the standard?

Where to Find Graphics Free for

UseSymbol Stix

Advanced Google Image Search

Pixabay

• N2Y (The company that makes SymbolStix) has given permission to use the symbols in our test questions.

• SymbolStix is included in a UNIQUE or News-2-You subscription.

• You must include the following phrase in the keywords tab on Eduphoria:

SymbolStix used with permission. © SymbolStix LLC 2003-2014 All rights reserved

Creative Commons vs Public Domain

• Creative Commons refers to images and passages that may be used for commercial purposes. The original owner must be attributed using APA. (www.noodletools.com)

• Public Domain refers to images and passages that may be used for commercial purposes. No APA attribution is needed but should still be noted that the resource is considered public domain.

• Copyrighted images and passages need previous, written permission to use that material commercially. The original owner must be attributed using APA.

• Student work may be used with permission from both the student and the parent. (If the student is a minor.) No attribution needed.

How to Find Public Domain Graphics: Google Images

• Google Images allows you to do an advance image search where you can choose to only see graphics that are public domain.

• http://www.google.com/advanced_image_search

Scroll to the bottom of the page on the

google image search. The last box is where you can choose image

rights. Choose “free to use, share

or modify, even commercially”.

The column on the side of the screen gives information about each

search option.

http://www.google.com/advanced_image_search

How to Find Creative Commons Graphics:Pixabay

• Pixabay allows you to search a database of images. Each image has its own requirements for use, however all are free for use.

• Many images require and attribution to the original creator using APA format.

• www.pixabay.com

Pixabay Home Screen

This section will let you know exactly if you should give the original creator attributions or not. In this case, it is public domain, so no APA attribution is needed.

How to Create Attributions in APA

Basic Format for an Electronic Image

• Author (Role of Author). (Year image was created). Title of work [Type of work], Retrieved Month Day, Year, from: URL (address of website)

Basic Format for an Electronic Image (No Author)

• Title of work [Type of work]. (Year image was created). Retrieved Month Day, Year, from: URL (address of website)

Basic Format for an Electronic Image (No Author, No Title, No Date)

• Many images found on the Web are of this category, but you should still look for this missing information: try clicking on the image, and/or looking at the bottom of the image.

• [Subject and type of work]. Retrieved Month Day, Year, from: URL (address of website)

Specific Image Examples

• Kulbis, M. (Photographer). (2006). Men pray [Photograph], Retrieved April 12, 2006, from: http://accuweather.ap.org/cgi-bin/aplaunch.pl

• Japanese Geisha [Photograph]. (2006). Retrieved April 10, 2006, from: http://www.bergoiata.org/fe/divers28/10.htm

•  [Untitled photograph of a baby chimpanzee]. Retrieved April 12, 2006, from: http://perso.wanadoo.fr/jdtr/struc/chimp3.htm

APA Citation Style Guide. (2014, January 1). Retrieved December 16, 2014, from http://www.landmark.edu/library/citation-guides/landmark-college-citation-guides/apa-citation-style-guide/#Images

Writing a Question in Publisher

Step 1 Choose the Grade/Course/Standard

• 1st Grade

• Access Math

• MAFS.1.OA.1.AP.1b• Solve addition and subtraction word problems within 20.

Step 2Decide which template to use

Step 3Open the template in Microsoft Publisher

Step 4Write your question

(in this example -the word problem)

Step 5Find the graphics you will be using

Step 6Insert your graphics into Microsoft Publisher and

design your visual aid for the question

Step 7Create your answer choices

Step 8Save your question as a picture

Select all the items on the page.

Then right click them. A pop-up menu will

appear.

Choose “Save as Picture.”

Name and save the picture to your

computer.

Step 9The rubric page

In Microsoft Publisher, open the rubric

template and edit it for the question.

- Change the correct answer on the rubric.

- Write the scripting for the question.

- Insert the picture into the rubric page.

Step 10Save the rubric page as a picture

Select all the items on the page.

Then right click them, a pop-up menu will

appear.

Choose “Save as picture.”

Name and save the picture to your

computer.

Step 11

Save

Make sure both the rubric and the question are saved in Microsoft Publisher. If

changes need to be made at a later time, you will need to edit them, re-save as a

picture, and then put them on Eduphoria.

The Grading Rubric

The grading rubric is modeled on the

Florida Alternate Assessment.

The areas that are highlighted are the

areas that have to be changed for each

question. The rest stays the same.

For each question you need to write

scripting for what the test administrator should say and do.

Insert the jpeg of the template question.

Make sure you change the correct

answer in the rubric.

Bias and Sensitivity

Bias & Sensitivity

“A bias and sensitivity review helps to ensure that items and stimuli are free of bias and do not exclude any group of children from demonstrating what they know and are able to do.”

33

Bias

Students should not be at an advantage or disadvantage based on their personal characteristics.

Personal characteristics: Gender

Race/Ethnicity

Religion

SES

Geographic Region

Experience

34

Bias: Examples

Portrayal of leisure activities that are not accessible to all children

Balance of gender and ethnicities in names

Stereotyping any groups

Over-inclusion or under-inclusion of any group

Sensitivity

Is the subject matter acceptable to Florida students, parents, and other community members?

NOT related to student success; more tuned in to student/parent reaction to content and context.

Moralistic issues, political positions, sexuality, etc.

36

Violence Criminal Activities Drugs (illegal, prescription, alcohol,

tobacco) and individuals associated with drugs or controlled substances

Junk Food Height/Weight Dieting, other concerns about self-

image

Child Abuse/Neglect Guns/Gun Control Suicide Animals, Animal Bones, Animal Rights Creationism Family Conflict Running Away

Sensitivity Topics List (Partial)

Uploading Your Question in Eduphoria

Login to Eduphoria

Log into https://sac.schoolobjects.com

Please keep in mind that your Eduphoria account will not be created until you have completed all training requirements.

Login to Eduphoria

Click on “aware”

Inserting the Question

In the upper left of your screen, make sure you have the “Item Bank” tab chosen click on your Content Area’s folder.

You should choose the ESE Access folder.

Inserting the Question

Upon clicking on your Content Area’s folder, a list will show which courses are available to choose from. In our case, we are going to choose Access Mathematics – Grade 1.

Inserting the Question

Find the course (Access Mathematics 1st Grade) in the

course list.

Select the standard from the list of standards. If standards are not listed for your course OR if

the standard you are looking for is not listed, you should not write any questions. In this

case, please contact your district representative. You are

not allowed to submit a question that does not have

a standard attached.

Click on “add question.”

Choose “constructed response” from the drop down menu.

Inserting the Question

Highlight “new rubric question.”

Click the button to add a picture.

Insert the picture of the question; this

should be the file you created in Publisher.

Inserting the Rubric

Highlight “enter scoring guide.”

Click the button to “Insert Image.”

Insert the picture of the rubric page.

Adding Keywords

In the keywords tab add:

Level of Question-Participatory LevelSupported Level

Independent Level

SymbolStix used with permission. © SymbolStix LLC 2003-2014 All rights

reserved

**You must click the Green Plus Sign each

time you enter something in the

Keywords (as indicated by the purple arrow).

Submit the Item for Review

Once you have written your item and everything in the previous slides has

been completed, you are ready to submit your item for review!

In order to do this, click “Publish” in the bottom right.

At this point, your item will need to go through two levels of review before being approved. It will go through a level 1 content review and a level 2

editing/proofreading review.

Reviewing Questions in Eduphoria

Login to Eduphoria

Log into https://sac.schoolobjects.com

Please keep in mind that your Eduphoria account will not be created until you have completed all training requirements.

Login to Eduphoria

Click on “aware”

Reviewing the Question

In the upper left of your screen, make sure you have the “Manage” tab chosen.

Reviewing the Question

In the bottom left of your screen, click “Approve Questions.”

Reviewing the Question

Next, select the course you plan to review items for.

Reviewing the Question

When you hover where the red arrow is, you’ll see a green pencil appear. Click this to open the item for review.

Reviewing the Question

Reviewing the Question