1

MODULE

Introduction

whatworks.ed.gov

2

MODULE Introduction

This module provides an introduction to the What Works Clearinghouse (WWC) Group Designs training.

The objectives of this module are to:

Clarify the purpose of this training and explain the certification options Provide background information on the WWC Describe logistics related to training Identify resources to support learning the WWC Group Design

Standards Provide an overview of the topics covered in the training

Training Purpose The WWC offers this training to inform the public about our design

standards, increase transparency, and promote the use of rigorous research.

The purpose of this training is to: • Describe the WWC Group Design Standards• Explain how to review a study using the WWC Group Design Standards• Prepare people for the optional group design certification activities

This training is not a comprehensive training on designs or methods.

3

Who Should Take the WWC Group Design Training?

Those interested in assessing the credibility of research evidence and interpreting research findings.• People trying to make evidence-based decisions. • Those interested in developing rigorous studies to meet WWC Group Design

Standards.

People who want to receive a certificate of completion for viewing the first six modules.

People who want to pursue one of two certification options through the WWC:1) Becoming certified in the WWC Group Design Standards.2) Becoming a certified reviewer for the WWC.

4

What Works Clearinghouse: Who, What, How, and Why

Who we are: An initiative of the Institute of

Education Sciences (IES) at the U.S. Department of Education

What we do: Evaluate and summarize the rigorous

research on education interventions

How we do it: Using a common framework:

The WWC Group Design Standards

Why we do it: Provide educators with

the information to make evidence-based decisions

5

Examples of WWC Products

6

Intervention reportssummarize all the

rigorous evidence on a specific intervention to

support evidence-baseddecision making.

Practice guides help educators address classroom challenges by

offering concrete suggestions and

summarizing the evidence base associated with the

suggested practices.

Quick reviewscontain the WWC’s

assessment of a single publication that is receiving media

attention.

Training Logistics

The online training is open to the public and accessible at all times.

You do not have to complete the online training in one sitting.

You can return to the training and start any module at any time.

You can work through training modules at your own pace.

You may print each module to view offline.

You should proceed through the modules sequentially, as later modules may revisit and build upon the concepts from earlier modules.

7

Training Logistics

Use the button in the middle of the screen to continue moving through the training or use the menu on the left to navigate through the modules.

Use the buttons on the top right to share the training with others or save your progress.

When saving your progress, you will obtain a unique passcode (PIN), which you can use to resume the training when you return.

You must complete all of the modules in the training series (and view them through the WWC website) to obtain your certificate of completion.

The “print certificate” link will appear after you complete the final module.

8

Trainee ResourcesTrainees can access several resources to supplement their learning during this training:

Procedures and Standards Handbooks: The WWC Procedures Handbook and the WWC Standards Handbook document all the procedures and standards in this training and include information about how the WWC reviews, documents, and reports on studies.

Review protocols: Review protocols describe the scope and parameters of each review.

Glossary: Glossary terms are underlined and bolded in red. You can access the full glossary through the Help Desk menu tab on the WWC homepage or from the Resources menu on the training page.

FAQ: You can access a list of questions and answers through the FAQ link in the Resources menu.

Help: You can submit technical questions about the training environment or questions about the content covered in the training through the WWC help desk.

Access these resources and more through the WWC website: whatworks.ed.gov.

9

What Is the Value of the WWC Group Design Standards?

• The WWC developed and implemented a set of standards for reviewing and synthesizing education research because:

There is a substantial amount of research on education interventions. The quality of that research varies widely. There were no common rules or guidelines for developing research studies.

• The WWC standards provide a set of common guidelines to help the public:

Decipher the information from different studies with different findings. Identify key elements in understanding quality research.

10

What Are the WWC Group Design Standards? The standards provide a common framework

to assess the credibility of research evidence on the effectiveness of interventions.

The WWC standards are sets of criteria used to assess research and assign a rating that reflects a study’s internal validity.

Upon reviewing an eligible study, the WWC assigns one of three ratings to it.

After reviewing studies using the standards, the WWC summarizes its findings in different publications and products.

11

Does Not Meet WWC Group

Design Standards

Meets WWC Group Design

Standards Without

Reservations

Design Standards

Meets WWC Group Design

Standards With

Reservations

Group Design Standards Framework

12

Training Module Content

Each module includes:

A summary of the objectives for the module.

Detailed information on the module topic, including examples.

“Knowledge Checks” to reinforce content and test what you’ve learned.

13

Training Module Overview

In addition to this introductory module, the WWC Group Design Standards Training contains several other modules:

14

Required Modules forCertificate of Completion

Additional Training Modules

ReportingCluster-Level AssignmentStudy Review Guide

MODULE

MODULE

8Group DesignsAttritionBaseline EquivalenceConfounding FactorsOutcome MeasuresSystematic Review Process

MODULEMODULEMODULEMODULEMODULE

12345

MODULE 6

MODULE 9

7

Module 1 Overview: Group Designs

The Group Designs module will answer:

What types of research designs can we review using the WWC Group Design Standards? (randomized controlled trials [RCTs] and quasi-experimental designs [QEDs])

What counts as an eligible RCT and an eligible QED study?

What does the WWC consider well-executed randomization?

15

Module 2 Overview: Attrition

The Attrition module will answer:

What is attrition and how does the WWC define attrition?

How does the WWC assess attrition for RCTs?

Why is attrition important?

How does the WWC calculate attrition rates?

How does the WWC apply the attrition standards to RCTs?

How can we recognize what the WWC does and does not consider to be attrition?

16

Module 3 Overview: Baseline Equivalence

The Baseline Equivalence module will answer:

How does the WWC define baseline equivalence?

When and how to apply the WWC equivalence standard?

How does the WWC determine the characteristics that it uses to assess baseline equivalence?

17

Module 4 Overview: Confounding Factors

The Confounding Factors module will answer:

How does the WWC define a confounding factor?

What are examples of confounding factors?

How do confounding factors affect the WWC’s review of a study?

18

Module 5 Overview: Outcome Measures

The Outcome Measures module will answer:

How does the WWC define an outcome measure?

Why does the WWC evaluate the outcome measures used to demonstrate impacts in studies?

What is the WWC standard for outcome measures and what requirementsmust a study’s outcome measures meet?

How does the WWC apply the outcome measure standard?

19

Module 6 Overview: Systematic Review Process

The Systematic Review Process module will answer:

What are the steps of the systematic review process?

How does the WWC use the systematic review process to evaluate education research?

How does the WWC use review protocols to guide the parameters of each review?

How does the WWC identify and define a study?

20

Module 7 Overview: Reporting

The Reporting module will answer:

How does the WWC report on the effectiveness of an intervention?

How does the WWC define and calculate effect size and the improvement index?

What are the statistical adjustments the WWC makes during a study review?

How does the WWC characterize study findings based on the effect size and statistical significance of effects for a domain?

21

Module 8 Overview: Cluster-Level Assignment

The Cluster-Level Assignment module will answer:

How does the WWC identify eligible cluster designs?

How do compositional changes affect the rating of cluster RCTs?

What is the standards framework the WWC uses to review cluster studies?

22

Module 9 Overview: Study Review Guide

The Study Review Guide module will answer:

What is the purpose of the study review guide?

How do reviewers access the study review guide on the WWC website?

What are the major components of the study review guide?

How do reviewers use the study review guide to enter information from a study?

23

24

INTRO MODULE

Conclusion

25

MODULE Introduction

You can access all the resources mentioned in this module through the WWC website, whatworks.ed.gov.

The full slide deck for this module is available on the WWC website. To receive a certificate of completion for viewing these training modules, you

must view the videos on the WWC website.

Thank you!