1 A new resource for the identification and utilization of E lectronic L earning A ssessment R...

1

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Assessment

TechnologyInstructionELARELAR

A new resource for the

identification and utilization of

Electronic Learning Assessment Resources

Overview for CTAP Staff

September 21, 2005

Presented by

John Cradler

Ruthmary Cradler

Chris York

2



Assessment


What is an ELAR?

• A technology application that simplifies delivery, aggregation, and disaggregation of assessment data • It provides information that allows teachers and administrators to make informed decisions and to prepare instructional plans based on student data

3



Assessment


1. ConductAssessments

2. ObtainRelevant Data

3. AnalyzeData

4. DetermineConclusions

5. PlanInstruction

6. ImplementInstruction

An ELAR should assisteducators in conducting data-driven instructional planning

4



Assessment


Why do we need an ELAR information service?

• Increased State and Federal instructional accountability requirements for reporting student outcomes.

• Increased emphasis on using student assessmentdata to inform district, school, and classroom instructional planning decisions.

• Technology applications or ELARs are rapidly emerging as a potential tool to enable data-driven decisions

• Educators need information on how to selectand use ELARs that meet their local needs

5



Assessment


How did the ELAR information service evolve?

• California Legislature was interested in enabling educators to make more meaningful and effective use of State testing data

• The legislature recognized that technology has the potential to contribute to more effective use of student testing data to plan instruction

• State Senator Jack Scott authored SB 1384 which authorized development of criteria, a review process, and a strategy for informing educators about how to select and utilize ELARs

6



Assessment


How is the ELAR project collaboratively implemented?

• CLRN: Provides for the online database results of ELARreviews, coordinates the reviews, and is the host SETS project for the ELAR information service.

• TICAL: Provided input on the design of the criteria and will assist in designing and conducting professional development and posting information relevant to ELARs on the TICAL website.

• TechSETS: Assists in the technical review of ELARS and provides input as an ELAR advising entity.

• Learning Resources: A contractor to CLRN who provides overall leadership and management for the design and implementation of the ELAR information resources.

7



Assessment


Who are the other important collaborators?

• CTAP: The 11 CTAP Regions along with regional assessment staff will disseminate information and training on the use of the ELAR information resource.

• CDE Educational Technology Office: The contracting and overseeing agent for the ELAR component of CLRN is the CDE.

• CDE Office of Assessment: Played a major role in developing ELAR review criteria and is an active participant in the review of ELARs.

• ACSA and CUE: Both were supporters of SB 1384 and will be featuring conference workshops on how to use the ELAR information.

• CARET: The Center for Applied Research in Educational Technology provides research on the use of ELARs.

8



Assessment


How were the Review Criteria Developed?

• Research was conducted to inform development of criteria.

• Draft ELAR Criteria were developed.

• ELAR Advisors and TICAL Cadre members provided Input.

• CDE Offices of Assessment and Technology provided Input.

• Draft criteria were field tested, revised, and finalized.

• CDE Approved the ELAR Review Criteria and related

guidelines by the legislated date of 12-31-04.

9



Assessment


The ELAR Review Criteria:

Three categories of minimum requirements for all ELAR applications have been established to cover the range of types of ELARs:

Type I: Configured to import, process, and enable analysis of California State testing data.

Type II: Not yet configured for California but can be modified toprocess and analyze California testing data.

Type III: Does not use state testing data but can enable locally collected test and other data to inform local instructional planning.

All ELARs for each of the three types must meet eight general requirements ranging from having correct spelling to enabling instructional planning.

10



Assessment


The ELAR Review Criteria (continued):

A. Sources of data: this can range from student grades to the CST

B. Relevant background: data that could have on impact on student achievement

C. Reporting & analysis of data: this ranges from the ELAR being able to enable analysis of data from the district to the classroom level, the same students over time, disaggregating by a variety of sub-groups, customization of reports for particular groups, charting of data, etc.

D. Standards to guide ELAR design: this includes attention to both content standards as well as technical standards

E. Customization of the ELAR: ranges from local modification of reporting to purchasing only parts of the ELAR needed

11



Assessment


The ELAR Review Criteria (continued):

F. General or other features: this ranges from inclusion of instructional planning templates to links to research-based strategies to address achievement gaps identified by the ELAR

G. Support for using the ELAR: this addresses the availability of assistance from the publisher ranging from training to an online ‘help-desk’.

H. Technical requirements: this includes bandwidth, hardware, needed technical support, security requirements etc.

I. Development and research: a description of any research or evaluation conducted on the cost vs. the benefits or impact of the program on instructional planning

12



Assessment


The ELAR Application and Review Process:

A. The publisher matches the ELAR features and to the ELAR criteria as the major component of the ELAR Application.

B. CLRN/ELAR staff arrange a review date, time and location.

C. The review process begins with a publisher presentation of the resource to the reviewers.

D. Reviewers try out the resource and test the features.

E. Reviewers evaluate the publishers match of the ELAR to the criteria and look for areas of agreement/disagreement.

F. When the review is completed the resource information is posted on the CLRN/ELAR website.

13



Assessment


After the review, what is provided?

A. ELAR Product Profile: A profile of each of the ELARs that were reviewed describing the target users of the resource, technical requirements, and specific training needs to effectively use the resource.

B. Narrative Summary: A summary of the salient features of the program based on the results of the review as well as information on appropriate uses of the resource.

C. ELAR Consumer Report Matrix: A matrix allowing for quick and detailed comparison of features identified in the review as a major or minor focus of the ELAR.

D. ELAR Complete review: The detailed profile description and analysis of the ELAR including ratings and a narrative summary.

14



Assessment


The ELAR Consumer Report Matrix:

G. Support for using the ELAR 1. Online ‘help-desk’ is included as part of the ELAR package. x w l l l w l 2. Telephone ‘help-desk’ is included as part of the ELAR package. w l l l l l l 3. Availability of tech support, hours, days, etc., is described. l l l l l l l 4. Training on how to use the ELAR is part of the package. l l l l l l l 5. Streaming video tutorials are available on using the ELAR. x w l x x x x 6. User prerequisite skills needed to use the ELAR are described. l w x x l x l 7. User prerequisite knowledge of assessments is described. x w x x x x l 8. Sample reports generated by the ELAR are provided with explanations. w w l w l l l 9. Information regarding cost(s) and process for obtaining ELAR upgrades

is described. l w l l l w l

10. User feedback on the ELAR is available. l l l l l l l 11. Resources for obtaining professional development for using the ELAR

are listed l l l l l w l

ELAR Title and Publisher Electronic Learning Assessment

Resources Review Criteria

Rating Key: Ratings reflect the Review Criteria focus for each ELAR reviewed as of August 2005.

Major Focus ……… l

Minor Focus ……… w

Not Applicable ……x

1. Aperture Suite; Intelligent

Solutionz, Inc.

2. Assessa; Eyecues, Inc.

3. Edusoft Assessment

System; Edusoft

4. Instructional Data Management

System; ETS Pulliam

5. Instructional Mgt. Solutions;

SchoolNet, Inc.

6. MEASURES Software Suite;

Datawise, Inc.

7. STARS; SchoolCity, Inc.

SampleELAR

ComparisonMatrix

15



Assessment


From ELAR criteria development to ELAR utilization

4. Design &conduct ELAR

review training

1. Reviewresearch & existing

ELAR criteria

7. Maintain & Expand the

ELAR database

9. DisseminateELAR informationand training

10. Educatorsaccess, select & utilize ELARs

11. Evaluate ELAR Program development, implementation & impact & revise as needed

2. Establish &approve reviewCriteria (Dec 04)

3. Adapt CLRN& TICAL Websitesfor ELAR access

5. Publisherssubmit ELARs for review

8. Design ELARProfessionalDevelopment

6. Conductreviews of

ELARs

16



Assessment


In summary, the ELAR information resource offers:

• A one-stop web-based location to review data management systems based on approved criteria.

• State-approved criteria enabling educators to compare/ evaluate their current instructional management systems.

• Research and information links on effective use of data management systems in instructional decision making.

• Professional development through TICAL and CTAP on how to use the ELAR website, to utilize ELARs to inform data-driven decisions, and apply the ELAR review criteria to guide local analysis and selection of ELARs.

17



Assessment


What are the information features still under construction?

• As soon as there are sufficient ELARs posted an ELAR ‘search feature’ will allow users to match ELARs with their own profile as is now done with CLRN ELR searches.

• In collaboration with the Center for Applied Research in Educational Technology (CARET), ELAR related research findings will be linked to the ELAR information site.

• Online video streamed demonstrations of how to use ELARs is planned for the ELAR component of the CLRN site.

• User feedback will be collected for each of the ELARs posted and then summarized and edited for inclusion on the ELAR database.

18



Assessment


ELAR project partner contact information:

Learning Resources

QuickTime™ and a decompressor




The ELAR Project is a service of the California Learning Resource Network (CLRN) funded by the CDE through a contract with the Stanislaus County Office of Education

Contact:Ellis Vance

[email protected]

Contact:Michael Simkins

Contact:John [email protected] Bob Beuthel

[email protected]

ContactBill Simpson:[email protected] [email protected]

mailto:[email protected]





1 A new resource for the identification and utilization of E lectronic L earning A ssessment R...

Documents

Transcript of 1 A new resource for the identification and utilization of E lectronic L earning A ssessment R...