Critiquing Assessments Within & For Web-Based Curriculum EDT 560 – May 2, 2006 Applying Technology...

Critiquing Assessments Within & For Web-Based Curriculum

EDT 560 – May 2, 2006Applying Technology to Assessment in EducationFor Gail Garthwait ~ By Betsy Bass

May 2, 2006 EDT 560: Critiquing Assessments Within and For Web-Based Curriculum

2

i. Table of Contentsii. Presentation Site Map I. IntroductionII. What are Assessments and How do You Know

if they are Good Enough – or Right – for You?

III. Critiquing the Assessments Within and For Web-Based Curriculum – List of Process & Product Assessment Tools

Index Page: to 4-Types of Web-Based Curriculum Projects

A. WebQuestsB. On-Line Collaborative ProjectsC. Student PublicationD. Other Web-Based Curriculum Projects

IV. Discussion

Table of

Contents

Contents-p.2Contents-p.2


3

A. Build-in Assessment Strategies VS Stand-Alone Assessment Tools

B. Rubrics1. What are Rubrics2. Where do you find them?3. When should I use rubrics?4. Should a rubric be used for everything

taught?5. Might Rubric use encourage kids to NOT do

their best?6. How would an educator build a rubric?

C. What makes a good assessment?V. Conclusion (VI. References & VII. Appendix A. – ISTE

Standards MetaRubric are in paper, not PowerPoint presentation.)

ContentsContents


4

PowerPoint

Presentation

Site Map

(Inspiration)

ContentsContents


5

“Traditionally, assessment has been viewed as evaluation, although assessment can provide more information about students than just grades. Assessment becomes evaluation when judgments are made, judgments that determine, pass, or certify (Eisner, 1994 in Shambaugh & Magliaro, 2006, p. 120).”

Where might we find curriculum and assessment tools today?

Introduction

ContentsContents


6

Curriculum & Assessment Tools Sources Text Books Educational Books, Magazines, and other

Publications Teacher-Created Material ~ Shared School Districts The World-Wide Web or Internet

This Presentation will review Assessments Within and For Web-Based Curriculum

ContentsContents


7

4 Types of Web-Based

Curriculum Review of four types of web-based curriculum ~ discussed in

Critical Basics: Web-Based Curriculum Projects: Examples & their built-in and created for assessments at

http://www.mediaworkshop.org/lifesciences/ls_workshops/webbased.html

They are: web quests, online collaborative projects, student publications, and other web-based curriculum projects.

Teachers who do not have pre-planned evaluations: Can use Web-based built-in assessments - if they exist Can find many relevant and appropriate ones on the web But whatever they choose, they must critique the tool for its quality

first.

ContentsContents


8

Is the Assessment Tool that You Chose -

The Best One for your Needs, Goals, & Students? The next question -

Is the assessment tool of good quality? And who decides that? Critiques and reviews of company-developed

assessments are on the market and many of the web-based curriculum has peer review that helps an educator make decisions.

Often however ~ the decision is left with the teacher and his or her colleagues to determine what is best for one’s own particular goals, needs, and students.

ContentsContents


9

What are Assessments Exactly? & What are some of the Assessment Tools? The Glossary Definition of assessment in

Shambaugh and Magliaro (2006, p.307) states that: “Assessment is “from the Latin word, assidere,

meaning “to sit beside someone.” Assessment serves multiple purposes, providing teachers with a judgment on how students have preformed, are performing, or are likely to perform. Includes a family of methods to provide information on students, including placement, diagnostic, formative, and summative information (Linn & Gronlund, 2000).”

ContentsContents


10

What are some of the tools used for formative and summative assessments? Many of the tools in the “family of methods”

can be found in textbooks, educational material or on the web.

A table -shown on the next slide – lists a variety of Process and Product Assessment Tools

Source: http://eduscapes.com/sessions/ptools/assess.htm Article: Engaging Technology Tools Projects by Annette Lamb

(7/99. Updated 10/99).

ContentsContents


11ContentsContents


12

4 Types of Web-Based Curriculum

Review of four types of web-based curriculum ~ discussed in Critical Basics: Web-Based Curriculum Projects: Examples

& their built-in and created for assessments

They are: webquests online collaborative projects student publications other web-based curriculum projects Discussion

ContentsContents


13

Web-Based Curriculum Projects & Their Assessments WebQuests - What are they?“An inquiry-oriented activity in which most or all of the

information used by students is online. By providing links necessary to complete the quest, the student is able to focus on the material rather than spend time looking for it. The five-part WebQuest (Introduction, Task, Resources, Process, Evaluation, and Conclusion) promotes critical thinking at the levels of analysis, synthesis and evaluation. (4/19/2006) www.netnet.org/students/student%20glossary.htm.”

ContentsContents


14

Webquests One of the best WebQuests sites is: the WebQuest

Portal by Bernie Dodge (http://webquest.org/). Each WebQuest is developed with the Evaluation as part

of the curriculum activity. The WebQuest Portal lists peer review of each

WebQuest: from one to five stars. Will discuss: Evaluation Section of a WQ Evaluation Rubric Review 5-Star/5 WebQuest 1 Review 5-Star/5 WebQuest 2 Review 2-Star/5 WebQuest 3 Review 1-Star/5 WebQuest 4 Discussion

ContentsContents


15

El Nino or El No-no (5 stars/5) by Keith Nuthall http://www.powayschools.com/projects/elnino/

This WQ included a grading rubric and was aligned to content standards.

The point range was Exemplary, Accomplished, Developing, and Beginning.

The topics covered were Focus/Organization, Support, and Conventions.

ContentsContents


16

Mrs. Christensen's Food Chain Gangs (5 stars/5) by Mrs. Christensen http://teacherweb.com/IN/PNC/chris/

The Food Chain WQ did include a rubric and the topics it reviewed were Graphics/Originality, Required Elements, Knowledge Gained, Content Accuracy, and Attractiveness.

Points ranged from 4 (best) to 1. Aligned to standards.

ContentsContents


17

Journey Through Time (2 stars/5) by Amy

Schneider http://www.longwood.k12.ny.us/lhs/teach/webq/aschneidertime/

Journey Through Time did not include a rubric in the same format as the two above. The task was to develop a travel brochure of a geologic period that they researched – discussing plants, animals and the environment. The evaluation rates the brochure as A- Excellent, B-Good Job, C and D – with two sentences per grade discussing what is required. For example A: is exemplary, contains accurate information, and wonderful illustrations and D is “Your brochure is missing some information. It has few or no illustrations.”

The Teacher Page included alignment to NYS Science Standards.

ContentsContents


18

Bay Quest (1 star out of 5) by Laurie Rosenberg and Lori Sheridan http://cte.jhu.edu/techacademy/web/2000/col

e/BayQuest/ Even though this WQ rated lower on the

average than #3 – it included an actual rubric with 0-36 points for Title, Link Construction, Link Evaluation, and Style (different value for different tasks).

However, it did not appear to be aligned with standards.

ContentsContents


19

Evaluation Section of Kathy Schrock’s Webquest Evaluation Rubric: http://webquest.sdsu.edu/webquestrubric.html

Let’s use this criteria to critique the 4 WebQuests!

ContentsContents


20

WebQuest

Discussion I think, given Kathy Schrock’s definition of

what a good evaluation should include, the star rating of the WebQuests by the peer evaluators were fairly reasonable.

The good ones were detailed and clearly expressed what the students were required to do, whereas the poorer-rated ones did not.

4-Types4-Types

ContentsContents


21

Online collaborative projects

This category represents any project where a student, class, or group of people works with others on a project. It may be as simple as all sampling water temperature on one day and comparing it – or being part of a major educational and/or environmental project.

In reviewing the projects discussed below under the online collaborative projects section of the Critical Basics paper, I realized that most online collaborative projects do not come with evaluations built-in.

It is the teacher’s responsibility to choose an appropriate evaluation for these endeavors.

4-Types4-Types

ContentsContents


22

Collaboration Projects that I reviewed from the Critical Basics site are:1. West Logotree: An alphabetical listing of

online projects across all grade level ~ Join an On Line Project at http://www.siec.k12.in.us/%7Ewest/online/join.htm

I reviewed several of these projects, but no individual project came with a built-in rubric or evaluation suggestion.

West Logotree did offer: Evaluations: Three rubrics for assessing webpages by students.

ContentsContents


23

Online collaborative projects 2. “Global Schoolhouse: Find online projects in a variety of content areas across all grade levels.” Once again these collaboration projects seem open-ended for teachers to decide how they want to evaluate their students’ progress.

3. QESN Projects Around the World: Online projects around the world for students of all ages. Offered in English and French.

There is much help to find a project and get involved ~ but the evaluation sections again seem to be left to the individual teachers.

ContentsContents


24

An example is: “Down the drain: How much water do you use? NEW! (Gives description of project.)

http://www.k12science.org/curriculum/drainproj/information.html

Many projects list the standards with which the project aligns, but does not go as far as offering assessment options. Down the Drain offers the following, but not evaluation ideas.

“Material Available on this Site Specific instructions for how to collect, submit, and analyze

the data are given in the Student Activities section which is linked in the left-hand menu. Data is also available to be submitted and viewed through the Project Data section but please be sure to read the instructions first. Students and teachers are welcome to submit work for publication in the Student Area. The Teacher Area contains a teacher's guide, a list of curriculum standards that are satisfied by the project, hands-on activities and other related project information. The Reference Material section has a lot of links related to water usage and conservation, and the Online Help section contains a list of online experts you can contact with water related questions as well as contact information for the CIESE project leader.”

ContentsContents


25

Student publications The web is used for many educational uses

in the classroom now, including publishing long- and short-term projects, portfolios, writing blogs, publishing magazines, or showcasing work, such as the Maine Lakes Conservancy Institute’s Student Portal, where students write under eight categories about their lake and its watershed during an educational lake ecology program. This can be viewed at: http://www.mlci.org/Students/default.aspx .

Since each participating school has their own program within their school, each teacher devises their own assessment methods.

4-Types4-Types

ContentsContents


26ContentsContents


27

Student publications I reviewed these two forms of student

publications under the list for Critical Basics –III. Student Publications: The Middle Ages: 7th Grade students report about

what they learned Digital Portfolios: High School student use the

web to organize their best work, (including teacher's curriculum)

ContentsContents


28

Student publications:

The Middle Ages The Middle Ages Curriculum had 3 projects. All 3 projects listed evaluations processes –

How the educator would assess the students. Tools ranged from essays, oral presentations,

research papers, and unit tests. However - once again, for the three projects,

no rubrics or assessment tools that would guide the students to do their best or to suggest tools for other teachers, were listed.

ContentsContents


29

Student publications:

Digital Portfolios

Digital Portfolios are a form of showcasing student’s work and their growth overtime ~ that is gaining popularity.

Many rubrics & basic checklists to get students started ~ can be easily found

The digital portfolios linked above (Digital Portfolios) are of a high school class creating their first portfolio and the use of animation and bright colors is predominant - maybe too predominant!

ContentsContents


30

Student publications: Digital Portfolios Digital or electronic (e-) portfolios usually don’t have rubrics built-in. Using a rubric developed by someone else - or reviewing several and adapting it for one’s own use, seems to be a normal process.

The Meta-Rubric for ISTE Standards is a good one to use

A rubric for evaluating electronic portfolios from the University of Wisconsin (A+ Rubric) can be found at: http://www.uwstout.edu/soe/profdev/eportfoliorubric.html

4-Types4-Types

ContentsContents


31

Other web-based curriculum projects

IV. Other Web-based Curriculum Projects

Other web-based curriculum projects include curriculum of all topics by teachers posted on the web and multimedia projects – which may include video and hypermedia projects. The authors of Critical Basics suggest that teachers watch for two main things: Convention and Content Considerations when critiquing on-line curriculum projects.

Here are their suggestions >>>

Convention Considerations: Design, navigation and layout conventions Does the site clearly state the appropriate grade level Is there a means to contact the person who made the site Is the site current? When was it last updated?

Content Considerations:

Does the content match the appropriate grade level that it targets? Is it engaging and interesting for the students? Is it well organized and easy to follow and understand?

4-Types4-Types

ContentsContents


32

Other web-based Curriculum projects Once again, the assessment tools appear

to be up to the educator to choose. Some of the rubrics that I found

that look useful are: 1. Web Based Learning Environments

Assessment Rubric http://school.discovery.com/schrockguide/

pdf/web_based_learning_rubric.pdf 2. Multimedia Project Rubric

http://www.ncsu.edu/midlink/rub.mmproj.htm

ContentsContents


33

Other web-based Curriculum projects

3. Alaska Department of Education & Early Development:

A Collection of Assessment Strategies

http://www.educ.state.ak.us/tls/frameworks/mathsci/ms5_2as1.htm

4. Scoring Power Points by Jamie McKenzie

(About good Power Point Presentations – includes a rubric) http://fno.org/sept00/powerpoints.htmlContentsContents


34

ScienceMysteries – for kids to solve! I also found a site that contained Science Mysteries

for teachers to engage their students – especially those who do not usually like science!

Although they do not have assessments attached – they are a wonderful web-based educational source.

Once again, an educator must choose, develop, or adapt their own assessment tool. Look at the Writer Guy, Ken Eklund’s Website – for wonderful ways to get your students involved in solving a scientific mystery! (But create or find your own assessments!)

http://www.writerguy.com/sciencemystery/teachers.htm

4-Types4-Types

ContentsContents


35

Discussion Some projects, such as WebQuests, include evaluation (rubrics often) as part of their development process, But the majority of the projects reviewed

contained lists of evaluation possibilities of varying quality or no assessment tool suggestions at all.

Collaboration Projects and Digital Portfolios rarely contained evaluation tools and educators depended on stand-alone tools for assessment.

On-line lesson plans and projects, created by teachers for their students, usually included lists of evaluation strategies – but no scoring guidelines such as rubrics.

4-Types4-Types

ContentsContents Contents-p.2Contents-p.2


36

Discussion: Built-In Assessment Strategies VS Stand-Alone Assessment Tools

A built-in assessment strategy is one where the curriculum product developer has created an assessment specifically for that product.

A good one includes formative and summative forms of evaluation along with some numerical or content guide such as a rubric or point value to a lab checklist.

If a built-in assessment doesn’t exist - an educator must use a “stand-alone” assessment tool - one of their own, a colleague’s, find one in a published book or manuscript, or download one from the web.

The educator may adapt one or develop a new one consisting of ideas from several that were reviewed.



37

Discussion - Rubrics1. What are Rubrics?2. Where do you find them?3. When should I use rubrics?4. Should a rubric be used for

everything taught?5. Might Rubric use encourage

kids to NOT do their best?6. How would an educator build

a rubric?

Contents-p.2Contents-p.2ContentsContents


38

Rubrics - 1. What are Rubrics? “Web definitions for Rubric In general a rubric is a scoring guide used in

subjective assessments. A rubric implies that a rule defining the criteria of an assessment system is followed in evaluation. A rubric can be an explicit description of performance characteristics corresponding to a point on a rating scale. A scoring rubric makes explicit expected qualities of performance on a rating scale or the definition of a single scoring point on a scale.serc.carleton.edu/introgeo/assessment/glossary.html - Definition in context”

The elements of a rubric, as defined in Kathy Schrock’s Rubric to Evaluate WebQuests, are: Overall Aesthetics, Introduction, Task, Process, Resources, and Evaluation – They work together.


RubricsRubrics


39

Rubrics - 2. Where do you find them?

Built-in to a lesson plan or curriculum project by the developer - who creates it specifically for project or

As a Stand-Alone Assessment Tool ~ which can be found in books, educational reference material, from other teachers, on the Web, or can be developed or adapted from many sources

Teachers can easily save rubrics to be reused and/or adapted for other lesson plans or projects later.

Rubrics can be developed to automatically calculate grades.

Rubrics are fairly versatile and can be easily adapted to fit an educator’s needs and clearly reflect his or her goals.


RubricsRubrics


40

Rubrics - 3. When should I use rubrics?

It is often helpful to use rubrics when: it is important for students to be clear about your expectations

concerning a particular project, lesson, assignment. when engaging in project where the underlying concepts or tasks

may be particularly complex or elusive to students; rubrics help to define the important components and the expectations for learning for each component.

students are asked to assess their own work or the work of others. talking with students about their work; rubrics give both the teacher

and the student a common language and place to think about how the work might be improved.

evaluating a final project or assignment; the more information a student receives about the level of his or her work, the more he she will learn from it.

when it is important to be able to communicate student progress not only to students, but also to parents, administrators, and the community.”

Source: http://www.thinkinggear.com/tools/using.cfm?t_id=1&ui=4


RubricsRubrics


41

Rubrics - 4. Should a rubric be used for everything taught? No –many different tools exist on paper

and on the web to assess formative and summative deficiencies and accomplishments.

Some fit the needs, goals, and students’ abilities better than others.

An educator needs to make these choices.


RubricsRubrics


42

Rubrics - 5. Might Rubric use encourage Kids to NOT do their best? (Mike’s Comment) Although most students seem to appreciate the

guidance that rubrics provide, educators may find some students who feel that passing is good enough and that a rubric gives them the criteria to know what is needed - to just slide by.

However, the author’s opinion is that the students that may use a rubric to help them achieve the minimum and squeeze by with a just-passing grade, would probably have found another means to reach the same result.

These are my thoughts – what are others?!


RubricsRubrics


43

Rubrics - 6. How would an educator build a rubric?

Published workbook instruction, teacher-created work, or on Websites such as Creating a Rubric and Writing Rubrics (University of Wisconsin, Teacher Created Rubrics for

Assessment, 2006). Rubrics should include both qualitative

and quantitative descriptors – Why? How would a rubric developer decide on

what to weight the most heavily - content versus technical items & why? Comments?


RubricsRubrics


44

Discussion: What Makes A Good Assessment? How do you know if it is a good one? Who decides this and For whom are they analyzing the assessment? 1. Research the credibility of the source 2. Read peer-reviews, published reviews, or

talk to colleagues to learn how other people or organizations rate the product in question

3. Review rubrics about evaluation of the projects, read magazines on technology and credibility of sources and where to find the best ones

4. Use own experience to create the best material for his or her own use



45

Discussion: What Makes A Good Assessment? Know before looking for and critiquing

assessments for what types of tools and for what functions, one is looking!

The teacher must be the ultimate judge of what is the best tool for his or her curriculum project and the information gleaned from the teacher’s own experience, that of colleagues’ and from reviews by peers and organization will be the supporting material in his or her final decision.



46

Conclusion … In the end, teachers must adapt web-based curriculum and their assessments…

To meet the needs of their own course and lesson plans, &

With the special needs of their own students in mind during the structuring of formative and summative evaluations.

Although rubrics may appear to be a panacea for all evaluations, educators must use whatever assessment format best fits their needs-



47

Conclusion Simple checklists can be a good

guide for some things

The educator has The responsibility of knowing his or

her students well enough and his or her own goals for the project or unit well enough, to be able to choose, develop, or adapt appropriate assessments &

To be able to critique them to be able to choose, develop, or adapt the best ones.


Critiquing Assessments Within & For Web-Based Curriculum EDT 560 – May 2, 2006 Applying Technology...

Documents

Transcript of Critiquing Assessments Within & For Web-Based Curriculum EDT 560 – May 2, 2006 Applying Technology...