Connecting Implementation and Technology:

The Importance of Open Educational Resources

Portland, ORFebruary 8, 2011

Doug Sovde, Achieve

2

The Promise of Open Educational Resources

Freely (or nearly so) available content which benefits from the wisdom of the massesAccess to teachers, students, and parents wherever they are A variety of content to meet a variety of needs

So what’s missing?

Measures and Assurances of Quality

3

Examples of OERs

“Type I”: Coherent materials primarily for student use Monterey Institute of Technology (http://www.hippocampus.org)

“Type II”: Searchable library of materials Curriki (http://www.curriki.og)

“Type III”: Build your own textbookCk12 (http://www.ck12.org)

4

Adoption Efforts

Maine’s one child-one laptop program

California and supplemental content (CORE)

Virginia and Texas

Washington State’s Open Course Library

Beaverton School District

5

The Promise of Open Educational Resources

Why Isn’t It More?

Measures and Assurances of Quality

6

Achieve’s Role In the Open Educational Resources Initiative

Leverage experiences with ADP Network and the Common Core State Standards Initiative to support the development of OERs that have external alignment to the Common Core State Standards and internal alignment among Standards, Objectives, Assessments, and Activities.

To determine the characteristics of quality in the OER environment.

To determine to what extent the following Achieve tools can provide information about alignment and quality and if/how they should be revised or replaced with new tools:

Quality Review Criteria (QR)Assessment To Standards (ATS) AlignmentQuality Indicator Tool (QI)

7

The Achieve Quality OER Project

Nine month Hewlett-funded grant to define quality and develop and refine tools and processes to judge quality

Intended to capitalize on the possibilities associated with the common standards initiative

Based on Achieve’s background in alignment services to states

Standard to Standard Alignment

Assessment to Standard Alignment

In partnership with OER developers and state- and district-level users

8

Process

Piloted two tools- the Quality Review Protocol and the Assessment to Standards (ATS) Alignment Protocol on one OER to judge the effectiveness of the tools.

Based on definition of quality as alignment of OERs to Common Core State Standards

Revised both protocols based on first pilot, created third tool to judge other aspects of quality (inclusion of practice problems, leveraging of technology, etc…) and piloted on two additional OERs to judge effectiveness of tools.

Sought feedback of OER advisory group on tools and definitions

Revised and created new tools and created multi-user approach to judging quality

9

Findings

Alignment of an OER’s assets to the Common Core State Standards continues to be the most important aspect of quality

For adoption and advocacy purposes

Quality Review and ATS protocols significantly adjusted to meet the needs of the OER environment in this regard

Other aspects of quality are important to the user

Engage learners through multimedia, interaction, and simulation

Allow for flexible use and control over content by users to meet range of instructional approaches and needs of learners

10

Process for Evaluating OERs

Developer submits responses to Achieve questionnaire

Rate degree of alignment using Rubric I, with a five point scale: Exceptional, Strong, Some value, Needs improvement, N/A (the OER does not include this element

Rate additional criteria (using Rubrics II-VI), with a five point scale (highest being Superior rather than Exceptional)

• Quality of Explanation of Content (Rubric II)• Quality of Materials for Instructional Purposes (Rubric III)• Quality of Embedded Assessment (Rubric IV)• Quality of Practice Problems (Rubric V)• Deeper Learning (Rubric VI)• Interactivity (Rubric VII)

11

Rubrics I - III

Degree of Alignment to Common Core State Standards (Rubric I):Does the OER cover all of the content in a grade band or a subject area and do the included objects have excellent alignment with the CCSS?

Quality of Explanation of Content (Rubric II): Does the OER provide enough information, experience, and practice to allow the self-directed learner to master the topic(s) identified by the developer?

Quality of Materials for Instructional Purposes (Rubric III): Does the OER offer well-designed, comprehensive, easy to understand and easy to use sets of objects or learning experiences that completely cover a topic and are designed to be used to teach others?

12

Rubrics IV - VI

Quality of Embedded Assessment (Rubric IV): Does the OER embed assessments before, during, and after learning and do the content in the assessments map clearly and directly to the important content from the objects in the OER?

Quality of Practice Problems (Rubric V): Does the OER offer an extensive number of practice problems for clearly identified content and are the problems well written and supported by accurate answer keys?

Deeper Learning (Rubric VI): Does the OER offer students the opportunity to engage in critical thinking, problem solving, collaboration, and communication?

Interactivity (Rubric VII): Does the OER take advantage of the technology in which it’s based?

13

Beta Test & Results

Based on the draft process and rubrics, two OERs were evaluated. Below is a sample side-by-side analysis of CCSS and OER

• Columns 1 & 2 contain the CCSS and relevant content from the OER

• Column 3 contains the rating (3=excellent alignment; 2=good alignment; 1=weak alignment; 0=no alignment possible; a * indicates that the OER content goes beyond the CCSS in some manner).

Core Concepts Aligned OER objects Rating

B4.A. A rate of change can be expressed as a rate of change (eg: people per year) or a relative rate of change (eg: percent per year).

70.1 Unit ties exponential functions to quadratic functions of the form of y = x2 and reviews vocabulary of “base” and “power.” 1

B4.B. Constant relative rate of change leads to exponential functions.

70.2 Offers examples of exponential functions with the of the form y= bx; y = a*bx; and y = a * bx + c. 70.3 Identifies other families of exponential functions [y = 10x; y = ex; y = (1/2)x; y = 3 * (1/2)x + c] that will not be examined in this lesson 70.4 Explains “a key feature of exponential functions is that each time x increases by 1, y changes by a factor of the base” using a table of values for y = 3 * 2x. Examples of exponential functions shown.

2*

B4.C. Repeated percentage growth (at a constant relative rate) generates graphs that “bend upward."

70.4 Explains “a key feature of exponential functions is that each time x increases by 1, y changes by a factor of the base” using a table of values for y=3*2x to illustrate. Examples of exponential functions graphs shown although the positive direction of the graphs is not made explicit.

2 or 3

14

Beta Results

Sample Completed Rubric:

• The sample OER was first evaluated using Degree of Alignment rubric (Rubric I).

• Rubrics II – V were then utilized, as applicable.

OER BDegree of Alignment to Common Core Standards: Some Value

OER B content is well aligned to an internal set of Algebra I objectives. It does not include all of the content the CCSS associated with this topic.

Quality of Explanation of Content: StrongOER B provides enough information, experience, and practice to allow the self-directed learner to master the topic(s).

Quality of Materials for Instructional Purposes: NAQuality of Embedded Assessments: Strong

The OER includes both on-going (formative) and summative assessments.The content in the assessments map directly to the important content from the objects in the OER. Both skills and knowledge are assessed. While the rating is “strong”, it should be noted that the assessments are quite routine and low in cognitive demand: the degree to which knowledge is assessed is border line. There is an adaptive component to this online site, however, which supports the rating of “strong”.

Quality of Practice Problems: Some ValueOER B offers sound practice problems for identified content; there are only one or two problems types available; there are only one or two format types.

15

Next Steps

Test rubrics and process with focus groups beyond the OAG, and create additional rubrics to determine:

• Whether and/or how well the OER– Engages learners through multimedia (in print, online, audio,

video) and interaction/simulation;• Whether and/or how frequently the OER is updated to reflect new

developments in the content areas and consistent with the development of new standards and assessments

User engagement with live OERs: Involve users more directly, possibly by making the tools themselves “open,” and by providing tool-use professional development activities

Mount the tools electronically, so they become Open

16

In the End…

Open Educational Resources hold the promise of high quality, freely available, and regularly updated materials to support differentiated instruction.

They hold the promise of massive reallocations of funds away from static, rigid resources to professional development and instructional leadership

They hold the promise of making education relevant, mobile, and dynamic, allowing students, parents, and teacher to engage in on-demand learning.

Open Educational Resources hold the promise of equity

Open Educational Resource Initiativehttp://www.hewlett.org/oer

Questions?dsovde@achieve.orgmsaxby@achieve.org