Statistical Randomization Techniques Statistical Randomization Techniques for Single-Case Intervention Datafor Single-Case Intervention Data

Joel R. LevinUniversity of Arizona

Purpose of This Presentation (Dejà Vu?)Purpose of This Presentation (Dejà Vu?)

To broaden your minds by introducing you to new and exciting, scientifically credible single-case intervention design-and-analysis possibilities.

To let you know that these procedures are becoming increasingly acceptable to single-case intervention researchers and are beginning to appear in the SCD research literature.

Whether YOU ever choose to adopt them in your own SCD research (after this week!) is entirely up to you.

A Permutation-Test Primer (Based on Levin, 2007A Permutation-Test Primer (Based on Levin, 2007))• Rationale and assumptions• Samples and populations• Individuals and groups• Scores and ranks

– exact probabilities and sampling approximations for scores

Levin, J. R. (2007). Randomization tests: Statistical tools for assessing the effects of educational interventions when resources are scarce. In S. Sawilowsky (Ed.), Real data analysis (pp. 115-123). Greenwich, CT: Information Age.

Example 1: Very Small SamplesExample 1: Very Small Samples• From a total of 6 elementary school classrooms, 3 are

randomly assigned to receive an instructional intervention that is designed to boost students’ academic performance (intervention classrooms), while students in the other 3 classrooms continue to receive their regular instruction (control classrooms).

• Following the instructional phase of the experiment, students in all classrooms are administered a 50-point achievement test and the average test performance within each classroom is calculated.

• Of interest in this study is the mean achievement-test difference between the 3 intervention classrooms and the 3 control classrooms.

Example 1: Data-Analysis RationaleExample 1: Data-Analysis Rationale

The obtained mean difference is examined in the context of the distribution of all possible mean differences that can be generated by assigning the 6 obtained classroom means to two instructional conditions, assuming that 3 classroom means must be assigned to each condition.

A statistical test is then conducted by addressing the question: How (un)likely or (im)probable is what actually occurred (i.e., the obtained intervention-control mean difference) in relation to everything that could have occurred (i.e., the distribution of all possible intervention-control mean differences, given the study's design structure and the set of means produced)?

Should the result of the foregoing test be deemed statistically improbable (e.g., p < .05), then the researcher would conclude that the two instructional methods differ with respect to students' average achievement-test performance.

In how many different ways can 6 scores be assigned to 2 groups, if 3 scores must end up in each group? That is the same thing as asking how many different combinations of 3 objects are there if selected from a total of 6 objects.

So as not to waste time attempting to express that quantity symbolically here: The answer, my friends, boils down to 6!/3!3! = 20. For example, consider the following 6 scores:

1 2 3 4 5 6

Let us systematically count the specific ways that 3 scores could be assigned to Group 1. (Note: The order in which the 3 scores are listed is not important.)

1. 1 2 3 2. 1 2 4 3. 1 2 5 4. 1 2 6 5. 1 3 4 6. 1 3 5 7. 1 3 6 8. 1 4 5 9. 1 4 6 10. 1 5 6 11. 2 3 4 12. 2 3 513. 2 3 6 14. 2 4 5 15. 2 4 6 16. 2 5 617. 3 4 5 18. 3 4 6 19. 3 5 6 20. 4 5 6

Example 1: Classroom Means and All Possible Assignments of Example 1: Classroom Means and All Possible Assignments of Them to the Two Conditions (Them to the Two Conditions (NN11 = = NN22 = 3) = 3)

Condition 1 Condition 2 M Difference (C1-C2) 1. 42.6, 40.1, 39.6 (122.3, 40.8) 37.6, 36.7, 36.3 (110.6, 36.9) 3.9* p1 = 1/20 = .05

2. 42.6, 40.1, 37.6 (120.3, 40.1) 39.6, 36.7, 36.3 (112.6, 37.5) 2.6 p2 = 2/20 = .10 3. 42.6, 40.1, 36.7 (119.4, 39.8) 39.6, 37.6, 36.3 (113.5, 37.8) 2.0 4. 42.6, 40.1, 36.3 (119.0, 39.7) 39.6, 37.6, 36.7 (113.9, 38.0) 1.7 5. 42.6, 39.6, 37.6 (119.8, 39.9) 40.1, 36.7, 36.3 (113.1, 37.7) 2.2 6. 42.6, 39.6, 36.7 (118.9, 39.6) 40.1, 37.6, 36.3 (114.0, 38.0) 1.6 7. 42.6, 39.6, 36.3 (118.5, 39.5) 40.1, 37.6, 36.7 (114.4, 38.1) 1.4 8. 42.6, 37.6, 36.7 (116.9, 39.0) 40.1, 39.6, 36.3 (116.0, 38.7) 0.3 9. 42.6, 37.6, 36.3 (116.5, 38.83) 40.1, 39.6, 36.7 (116.4, 38.80) 0.0310. 42.6, 36.7, 36.3 (115.6, 38.5) 40.1, 39.6, 37.6 (117.3, 39.1) -0.611. 40.1, 39.6, 37.6 (117.3, 39.1) 42.6, 36.7, 36.3 (115.6, 38.5) 0.612. 40.1, 39.6, 36.7 (116.4, 38.80) 42.6, 37.6, 36.3 (116.5, 38.83) -0.0313. 40.1, 39.6, 36.3 (116.0, 38.7) 42.6, 37.6, 36.7 (116.9, 39.0) -0.314. 40.1, 37.6, 36.7 (114.4, 38.1) 42.6, 39.6, 36.3 (118.5, 39.5) -1.415. 40.1, 37.6, 36.3 (114.0, 38.0) 42.6, 39.6, 36.7 (118.9, 39.6) -1.616. 40.1, 36.7, 36.3 (113.1, 37.7) 42.6, 39.6, 37.6 (119.8, 39.9) -2.2 17. 39.6, 37.6, 36.7 (113.9, 38.0) 42.6, 40.1, 36.3 (119.0, 39.7) -1.718. 39.6, 37.6, 36.3 (113.5, 37.8) 42.6, 40.1, 36.7 (119.4, 39.8) -2.019. 39.6, 36.7, 36.3 (112.6, 37.5) 42.6, 40.1, 37.6 (120.3, 40.1) -2.620. 37.6, 36.7, 36.3 (110.6, 36.9) 42.6, 40.1, 39.6 (122.3, 40.8) -3.9

What Summary Measure(s) Can/Should Be What Summary Measure(s) Can/Should Be Analyzed by The To-Be-Presented Techniques?Analyzed by The To-Be-Presented Techniques?

• Means (Levels) of the phases

Medians

Truncated/Censored data based on a priori rules

Even randomly selected observations?

• Slopes (Trends) of the phases

• Variances of the phases

• Any “Predicted Pattern” within or between phases

• Special considerations when groups are the units of intervention administration

Randomized Adaptations of Randomized Adaptations of Traditional Single-Case DesignsTraditional Single-Case Designs

• Phase/Intervention Randomization (Within Cases)– the order of the A and B phases/interventions is randomly

determined for each case (e.g., participant, pair, group, classroom)

• Intervention Randomization (Between Cases)‒ cases are randomly assigned to interventions

• Intervention Start-Point Randomization– the transition point from one phase to the next is

randomly determined for each case• Case Randomization

– cases are randomly assigned to positions within the design

Why Randomization Statistical Tests?• Conceptually and computationally straightforward

– easy to explain to others– parsimonious

• Logically consistent connection to the design’s randomization components– implications for Type I error control

• Statistical power characteristics

• Underlying statistical assumptions

• Versatility and adaptability relative to other single-case statistical approaches

BackgroundBackground• Eugene Edgington’s randomization-test contributions

• Randomized phase/intervention designs– Basic design (AB) and return to baseline design (ABA), including

when A and B consist of two different interventions– Reversal (or withdrawal or “operant”) design (ABAB…AB) and

alternating treatment design– Multiple-baseline design

• Randomized intervention start-point designs– Basic design (AB) and return-to-baseline design (ABA), including

when A and B consist of two different interventions– Reversal (or withdrawal or “operant”) design (ABAB…AB) and

alternating treatment design– Multiple-baseline design

• Replications (i.e., multiple units) and extensions of these

Three Sample Statistical Simulation StudiesThree Sample Statistical Simulation Studies

• ABAB…AB and Alternating Treatment Design (Levin, Ferron, & Kratochwill, 2012)

• Variety of AB Designs (Levin, Ferron, & Gafurov, 2014)

• Multiple-Baseline Design (Levin, Ferron, & Gafurov, 2015)

Top 10 List of Distinctions Between Actual and Top 10 List of Distinctions Between Actual and Simulated Single-Case Intervention StudiesSimulated Single-Case Intervention Studies

(With thanks to David Letterman, wherever you are!)(With thanks to David Letterman, wherever you are!)

10. Participants

9. Planning and Design

8. Methods and Procedures

7. Institutional Review Boards

6. Investigator Personality Characteristics

Top 10 Distinctions (Continued)Top 10 Distinctions (Continued)

5. Time and Hassle

6. Data Analysis

3. Rationalizations and Rationale for Future Research

2. Follow-Up

1. Publication Outlets and Impact Issues

Bonus Question: Why did I ever bother?

Systematic vs. Randomized ABAB…AB Systematic vs. Randomized ABAB…AB and Alternating-Treatment Designsand Alternating-Treatment Designs

• A few selected results from a Monte Carlo simulation study by Levin, Ferron, and Kratochwill (2012)

Five 24-Observation Designs Five 24-Observation Designs (Individual Observations)(Individual Observations)

ABABABABABABABABABABABAB 24 Phase

AABBAABBAABBAABBAABBAABB 12 Phase

AAAABBBBAAAABBBBAAAABBBB 6 Phase

AAAAAABBBBBBAAAAAABBBBBB 4 Phase

AAAAAAAAAAAABBBBBBBBBBBB 2 Phase

Type I Error Probability as a Function of Autocorrelation Type I Error Probability as a Function of Autocorrelation in the 24-Observation Design (Individual Observations)in the 24-Observation Design (Individual Observations)

Five 24-Observation ABAB…AB Designs Five 24-Observation ABAB…AB Designs (Phase Means)(Phase Means)

ABABABABABABABABABABABAB Block=1(24 phases)

BAABABABBAABBAABABBABABA Random Pair(13-24 phases)

AABBAABBAABBAABBAABBAABB Block=2(12 phases)

AAABBBAAABBBAAABBBAAABBB Block=3 (8 phases)

AAAABBBBAAAABBBBAAAABBBB Block=4(6 phases)

Type I Error Probability as a Function of Autocorrelation Type I Error Probability as a Function of Autocorrelation in the 24-Observation Design (Phase Means)in the 24-Observation Design (Phase Means)

Effect-Size Alert for Single-Case ResearchEffect-Size Alert for Single-Case ResearchOutcomes, or Don’t “Dis” Large Effect Sizes Here!Outcomes, or Don’t “Dis” Large Effect Sizes Here!

Marquis et al. (2000) noted in their meta-analysis of positive behavior support that “[t]he smallest [conventional effect-size measure] for outcomes was 1.5, which would be considered quite large in a group study context” (p. 165); and that their effect-size estimates “ranged from 1.5 standardized units to 3.1 units” (p. 167).

Rogers and Graham (2008, p. 885) indicated that “[W]hen we have used [the conventional method of effect-size calculation in meta-analyses of] single subject design studies in writing, the effect sizes are typically 3.0 and higher.”

In a single-case enuresis-treatment study conducted by Miller (1973), the conventional effect sizes calculated for the two participants were 5.98 and 6.41 (Busk & Serlin, 1992, p. 201-202).

ReferencesReferencesBusk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case

research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 187-212). Hillsdale, NJ: Erlbaum.

Marquis, J. G., Horner, R. H., Carr, E. G., Turnbull, A. P., Thompson, M., Behrens, G. A., et al. (2000). A meta-analysis of positive behavior support. In R. Gersten, E. P. Schiller, S. Vaughn, (Eds.), Contemporary special education research: Syntheses of knowledge base on critical instructional issues (pp. 137-178). Mahwah, NJ: Erlbaum.

Miller, P. M. (1973). An experimental analysis of retention control training in the treatment of nocturnal enuresis in two institutional adolescents. Behavior Therapy, 4, 288-294.

Rogers, L. A., & Graham, S. (2008). A meta-analysis of single subject design writing intervention research. Journal of Educational Psychology, 100, 879-906.

Power for Power for dd = 1.00 as a Function of Autocorrelation in the = 1.00 as a Function of Autocorrelation in the 24-Observation Design (Phase Means)24-Observation Design (Phase Means)

Major ConclusionsMajor Conclusions 1. Claims that systematic ABAB…AB designs produce

inflated Type I error probabilities in series with positively autocorrelated observations are grossly “inflated.”

2. A systematic design consisting of alternating A and B individual observations yields very respectable power for detecting larger effect sizes. From a methodological perspective, an even more appealing option is the randomized pairs ─ or, in certain situations, the randomized paired doubles ─ design, perhaps with one or more mandatory initial A' observations in situations where a true baseline is desired (Kratochwill & Levin, 2010). The latter is not an issue if A and B represent two alternative interventions.

Adapted from Levin, J. R., & Wampold, B. E. (1999). Generalized single‑case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14, 59‑93.

Replicated AB Design With Three Cases (“Units”), Two Within-Series Intervention Conditions, 20 Time Periods, and 13 Potential Intervention Points for Each Case

Marascuilo, L. A., & Busk, P. L. (1988). Combining statistics for multiple-baseline AB and replicated ABAB designs across subjects. Behavioral Assessment, 10, 1-28.

Levin & Wampold’s (1999) Simultaneous Levin & Wampold’s (1999) Simultaneous Start-Point ModelStart-Point Model

Time Period 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Pair 1X A A A A A A A A A A A │ B B B B B B B B B

Pair 1Y A A A A A A A A A A A │ B B B B B B B B B

Note: Potential intervention start points are between Time Periods 5

and 17 inclusive.

*Randomly selected intervention start point for the pair of units

Levin, J. R., & Wampold, B. E. (1999). Generalized single-case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14, 59–93.

A Two-Intervention (Between Cases) ExampleA Two-Intervention (Between Cases) Example

From Levin, J. R., & Wampold, B. E. (1997, July) Single-case randomization tests for a variety of situations. Paper presented at the 10th European Meeting of the Psychometric Society, Santiago de Compostela, Spain.

Means and Mean Differences Associated With Each of the Means and Mean Differences Associated With Each of the 9 Potential Intervention Startpoints, 9 Potential Intervention Startpoints,

By Reinforcer TypeBy Reinforcer Type

Start Point Social Token T-S

A B B-A A B B-A Diff Rank

3 13.0 13.9 0.9 12.5 16.3 3.8 2.9 9

4 13.3 13.9 0.6 12.7 16.7 4.0 3.4 8

5 13.5 13.9 0.0 12.5 17.3 4.8 4.8 7

6 13.8 13.7 -0.1 12.6 17.9 5.3 5.4 3

*7 13.7 13.8 0.0 13.2 18.2 5.0 5.0 4*

8 14.0 13.4 -0.6 14.0 18.0 4.0 4.6 5.5

9 14.1 13.0 -1.1 14.5 18.0 3.5 4.6 5.5

10 14.4 11.7 -2.7 14.7 18.7 4.0 6.7 2

11 14.5 10.0 -4.5 15.1 18.5 3.4 7.9 1

Levin & Wampold’s (1999) Replicated Levin & Wampold’s (1999) Replicated Simultaneous Start-Point ModelSimultaneous Start-Point Model

Time Period 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Pair 1X A A A A A A A A A A A* B B B B B B B B B Pair 1Y A A A A A A A A A A A* B B B B B B B B B

Pair 2X A A A A A A A A A* B B B B B B B B B B B Pair 2Y A A A A A A A A A* B B B B B B B B B B B

Note: Potential intervention start points are between Time Periods 5 and 17 inclusive.

*Randomly selected intervention start point for each pair of units

Levin, J. R., & Wampold, B. E. (1999). Generalized single-case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14, 59–93

Three Sample Statistical Simulation StudiesThree Sample Statistical Simulation Studies

• ABAB…AB and Alternating Treatment Design (Levin, Ferron, & Kratochwill, 2012)

• Variety of AB Designs (Levin, Ferron, & Gafurov, 2014)

Proposed Single-Case InterventionProposed Single-Case InterventionDesign/Analysis Schemes*Design/Analysis Schemes*

1. AB Randomized Phase-Order Design With Intervention Start-Point Randomization

*Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2014). Improved randomization tests for a class of single-case intervention designs. Journal of Modern Applied Statistical Methods, 13(2), 2-52; retrievable from http://digitalcommons.wayne.edu/jmasm/vol13/iss2/2.

Onghena, P., Vlaeyen, J. W. S., & de Jong, J. (2007). Randomized replicated single-case experiments: Treatment of pain-related fear by graded exposure in vivo. In S. Sawilowsky (Ed.), Real data analysis (pp. 387-396). Greenwich, CT: Information Age.

Suppose that in a 16-observation design, A and B are either a baseline phase and an intervention phase or two different interventions that a single case is to receive. The case is assigned randomly to one of the two phase orders (AB or BA). [With a true baseline-intervention design this can be accomplished by including one or more mandatory baseline/adaptation observations (A') for both phase orders.]

The random assignment of phase orders is required for the subsequent AB randomization test (modified Edgington test) to be valid.

In addition, the case receives a randomly selected intervention start point, with an a priori specification of 10 potential start points, from Observations 4 through 13 inclusive.

AB Randomized Phase-Order Design (With AB Randomized Phase-Order Design (With Mandatory Initial AMandatory Initial A' ' Baseline Phase)Baseline Phase)

With the original Edgington (1975) model, the study can be diagrammed as:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A' A' A' A A A B B B* B B B B B B B B B B

However, with the revised Edgington model, the opposite “pretend” ordering of As and Bs was also possible and therefore can be included in the randomization distribution:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A' A' A' B B B A A A* A A A A A A A A A A

AB Randomized Phase-Order Design (With AB Randomized Phase-Order Design (With Mandatory Initial AMandatory Initial A'' Baseline Phase) Baseline Phase)

For Edgington’s model, with 10 potential intervention points, even if the observed outcome were the most extreme the lowest one-tailed significance probability would be p = 1/10 = .10.

In contrast, for the revised model with both orders taken into account, even with only 10 potential intervention points it would be possible to attain a one-tailed significance probability of p = 1/20 = .05.

Moreover, our simulation results indicate that the revised procedure: (1) maintains satisfactory Type I error control; and (2) exhibits power that is generally about .20-.40 points higher (and in some situations, even more) than that of the original Edgington procedure.

Comparison (α = .05, one-tailed) of randomization tests for a one-case (N = 1) AB randomized intervention start-point design (Single) and the randomized intervention start-point plus randomized intervention-order design (Dual), where the start point was randomly selected between the 6th through the 25th observations inclusive in a 30-observation study.

Comparison (α = .05, one-tailed) of randomization tests for the Single and Dual basic AB randomized designs replicated across N cases. The rejection rate of the null hypothesis is shown as a function of effect size and N, for a 15 observations design with 5 potential intervention start points designated from between the 6th and 10th observations inclusive and an autocorrelation of .3.

Selected Single- Versus Dual-Randomization Power Selected Single- Versus Dual-Randomization Power Comparisons of Longer and Shorter Series Simulations Comparisons of Longer and Shorter Series Simulations

(SL = Series Length, NSP = Number of Potential (SL = Series Length, NSP = Number of Potential Intervention Start Points)Intervention Start Points)

N d r Size (SL/NSP) Single Dual Difference

2 2.0 .30 Longer (15/5) .44 .85 .41

      Shorter (9/5) .42 .80 .38

3 1.5 .30 Longer (15/5) .49 .90 .41

      Shorter (7/3) .28 .73 .45

5 1.0 .30 Longer (15/5) .45 .89 .44

      Shorter (8/2) .15 .71 .56

Multiple-Baseline DesignMultiple-Baseline Design

• Revusky’s (1967) statistical procedure

Revusky, S. H. (1967). Some statistical treatments compatible with individual organism methodology. Journal of the Experimental Analysis of Behavior, 10, 319-330.

• Wampold and Worsham’s (1986) improved permutation procedure

Wampold, B. E., & Worsham, N. L. (1986). Randomization tests for multiple-baseline designs. Behavioral Assessment, 8, 135-143.

• Type I error and power assessments have been made by Ferron and Sentovich (2002) of the Wampold-Worsham (1986), Marascuilo-Busk (1988), and Koehler-Levin (1998) procedures

Ferron, J., & Sentovich, C. (2002). Statistical power of randomization tests used with multiple-baseline designs. Journal of Experimental Education, 70, 165-178.

Three Sample Statistical Simulation StudiesThree Sample Statistical Simulation Studies

• ABAB…AB and Alternating Treatment Design (Levin, Ferron, & Kratochwill, 2012)

• Variety of AB Designs (Levin, Ferron, & Gafurov, 2014)

• Multiple-Baseline Design (Levin, Ferron, & Gafurov, 2015)

Levin, J. R., Ferron, J. F., & Gafurov, B. S. (2015). Comparison of randomization-test procedures for single-case multiple-baseline designs. Unpublished manuscript. University of Arizona, Tucson.

Proposed Single-Case InterventionProposed Single-Case InterventionDesign/Analysis SchemesDesign/Analysis Schemes

1. AB Randomized Phase Design With Intervention Start-Point Randomization

2. AB Crossover Design With Intervention Start-Point Randomization

3. “Uber Supernova” Multiple-Baseline Comparative-Treatment Design With Intervention Start-Point Randomization

Multiple-Baseline Comparative-Treatment DesignMultiple-Baseline Comparative-Treatment Design(Modified Levin-Wampold & Koehler-Levin Designs)(Modified Levin-Wampold & Koehler-Levin Designs)

Time Period

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Pair 3X A A A A B* B B B B B B B B B B B

Pair 3Y A A A A B* B B B B B B B B B B B

Pair 1Y A A A A A A B* B B B B B B B B B

Pair 1X A A A A A A B* B B B B B B B B B

Pair 4Y A A A A A A A A A A B* B B B B B

Pair 4X A A A A A A A A A A B* B B B B B

Pair 2X A A A A A A A A A A A A A A B* B

Pair 2Y A A A A A A A A A A A A A A B* B

Note: Pairs are randomly assigned to stagger positions. Interventions X and Y are randomly assigned within each pair. The bolded letters represent the potential start points for each pair, with the asterisked B indicating the actual start point randomly selected for each pair.

ReferencesReferencesLevin, J. R., Ferron, J., M., & Kratochwill, T. R. (2012).

Nonparametric statistical tests for single-case systematic and randomized ABAB…AB and alternating treatment designs: New developments, new directions. Journal of School Psychology, 50, 599-624.

Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2014). Improved randomization tests for a class of single-case intervention designs. Journal of Modern Applied Statistical Methods, 13(2), 2-52; retrievable from http://digitalcommons.wayne.edu/jmasm/vol13/iss2/2.

Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2015). Comparison of randomization-test procedures for single-case multiple-baseline designs. Unpublished manuscript, University of Arizona, Tucson.

Some Randomization-Test SoftwareSome Randomization-Test Software

Randomized Phase DesignsEdgington & Onghena’s (2007) SCRT (Single-Case Randomization Tests) program, in their bookLevin, Ferron, & Kratochwill’s (2012) SAS software for various ABAB…AB and alternating treatment designs

Randomized Intervention Start-Point DesignsEdgington & Onghena’s (2007) SCRT program, in their book (also Bulté & Onghena, 2008)Koehler’s (2012) program for the Koehler-Levin (1998) multiple-baseline procedure (http://mkoehler.educ.msu.edu/regrand/)Gafurov & Levin’s ExPRT (Excel Package of Randomization Tests) Version 2.0, August 2015; downloadable at http://code.google.com/p/exprt

OtherBorckhardt et al.’s (2008) Simulation Modeling Analysis (SMA) program

Current and Projected Features of Gafurov & Current and Projected Features of Gafurov & Levin’s Levin’s ExPRTExPRT Statistical Software Statistical Software

Features of the programs include:• exact nonparametric statistical analyses based on some type of randomization• options for either random or fixed intervention start points and intervention orders• designs based on either one or multiple cases (replications)• an unlimited number of total observations for up to 15 cases• either individual or paired cases• analyses conducted with either raw or standardized data

Current and Projected Features of Current and Projected Features of the the ExPRTExPRT Statistical Software Statistical Software

Features of the programs include:• user-defined α levels (one- or two-tailed tests)• statistical decisions (reject, do not reject) and significance probabilites (p-values)• statistical tests based on either mean (level) or slope (trend) • output distribution of all possible outcomes • graph of the outcomes for each case • case-by-case and across-case summary measures and effect-size estimates• a randomizing routine for planned studies

AB DesignAB DesignBasic time-series designBaseline/Control (A) vs. Intervention (B) Intervention A vs. Intervention BIntervention start-point randomization procedure (Edgington model; Marascuilo-Busk model; Levin-Wampold simultaneous start-point model for two different matched interventions: comparative-effectiveness and general-effectiveness tests)Levin et al.’s (2014) randomized intervention-order option; single-case crossover design: intervention-effect and time-effect.

Additional Additional ExPRTExPRT Randomization Tests Randomization TestsABA DesignABA DesignIntervention start-point randomization (Onghena model)Overall test and separate two-phase testsReversal (ABAB) DesignReversal (ABAB) DesignIntervention start-point randomization (Onghena model)Overall test and separate two-phase testsMultiple-Baseline DesignMultiple-Baseline DesignWithin-case comparisons (Wampold-Worsham model)Within-case comparisons; intervention start-point

randomization (Koehler-Levin and restricted Marascuilo-Busk models)

Between-case comparisons (Revusky model)

Additional ReferencesAdditional References

Bulté, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40, 467–478.

Edgington, E. S., & Onghena, P. (2007). Randomization tests (4th ed.) Boca Raton, FL: Chapman & Hall/CRC.

Koehler, M. J., & Levin, J. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple‑baseline design. Psychological Methods, 3, 206‑217.

Levin, J. R., & Ferron, J. M. (2014). Review of Dugard, File, and Todman’s Single-case and small-n designs: A practical guide to randomization tests (2nd ed.). American Statistician, 68, 217-218.

Levin, J. R., Lall, V. F., & Kratochwill, T. R. (2011). Extensions of a versatile randomization test for assessing single-case intervention effects. Journal of School Psychology, 49, 55-79.

Levin, J. R., Marascuilo, L. A., & Hubert, L. J. (1978). N = nonparametric randomization tests. In T. R. Kratochwill (Ed.), Single subject research: Strategies for evaluating change (pp. 167–196). New York, NY: Academic Press.