Chapter 3:The Nature and Value of

Empirically Validated Interventions

Heather M. PaneCaldwell College

ABA 553

Crighton ‘Bud’ Newsom◦ Director of Psychology: Southwest Ohio

Developmental Center ◦ Advocated for high quality, science-based autism

treatment

Christine A. Hovanitz ◦ University of Cincinnati◦ Interests: Psychological mechanisms underlying

regulation and dysregulation of physiological states

Authors

Professionals must provide ‘competent’ standards.

What is competence?

Competence is often linked to scientific knowledge.

Ethical Standards

APA Ethical Standards state: “Psychologists’ work is based on established scientific and professional knowledge of the discipline”(American Psychological Association [APA], 2002. p5).

AMA states: “Physicians are expected to continue to study, apply, and advance scientific knowledge” (American Medical Association, 2001, p. 1).

What is ‘competence’?

Professional organizations typically avoid an explicit requirement that practitioner only use scientifically valid treatments.

Most of professionals working with people with developmental disabilities tolerate dubious therapies.

Authors point…

BACB Guidelines for Responsible Conduct 1.01: Reliance on Scientific Knowledge 1.02: Competence

◦ BA’s provide services, teach, and conduct research only within the boundaries of their competence, based on their education, training, supervised experience, or appropriate professional experience.

◦ new areas and techniques: only after first undertaking appropriate study, training, supervision, and/or consultation from persons who are competent in those areas or techniques.

Behavior Analyst Certification Board (BACB). (2001-2010). Guidelines for responsible conduct for behavior analysts. Retrieved February 2, 2012 from http://www.bacb.com/index.php?page=57

What is ‘competence?”

2.10: Treatment Efficacy

◦ (a): Must recommend scientifically supported most effective treatment procedures.

◦ (b): Clients have the right to effective treatment.

◦ (c): BA’s must review and appraise likely effects of alternative treatments.

BACB Treatment Efficacy

Formal standards in society for judging treatments (Beutler, 1998).

From the health care system and legal system

◦ Cost effectiveness: Based on number of people served and the cost.

◦ Standard of common practice: Based on frequency of use in the community.

◦ Doctrine of the respectable minority: treatment is based on an explicit theory/method of delivering and a ‘respectable’ minority of professionals endorse it.

What are they NOT considering?

Acceptability of Treatment

What are they NOT considering?

The OUTCOMEClient

Effectiveness Objective Success of the

Treatment

Acceptability of Treatment

Order: Discoverable regularities in relationships between events◦ Energy, mass, speed of light, reinforcement and

percentage of correct responses on academic tasks.

Expressed mathematically or verbally◦ Objective◦ Testable◦ Replicable

Elements of Science

Statements are Objective: Key terms stated so explicitly and unambiguously that a knowledgeable listeners can agree on the meaning.

Operationalizing: minimizes ambiguity by defining terms of the measurement operations required to make the observation of interest (e.g., frequency, duration).

Reliability: Independent observer make same measurement at same time.

Elements of Science

“Operational definitions make phenomena available for public

evaluation, verification, and replication, which are

impossible when events are described in terms of personal

perception and intuition.”

Green, 1996

Speculating that the problem originated in Mario’s ability to process and integrate tactile inputs within the central nervous system, we conclude that a treatment program designed to include enriched tactile experiences derived from participation in activities that Mario enjoys and finds meaningful will increase the likelihood that he will take in, process, and integrate tactile inputs as a basis for planning motor actions. While we can not directly observe if this occurs, we can observe if Mario’s motor behavior improves (Fisher & Murray, 1991).

Jacobson, J. W., Foxx, R.M., & Mulick, J. A. (2010). Controversial therapies for developmental disabilities: fad, fashion, and science in professional practice. New York, NY, Routledge.

Consider the statement…

Speculating that the problem originated in Mario’s ability to process and integrate tactile inputs within the central nervous system, we conclude that a treatment program designed to include enriched tactile experiences derived from participation in activities that Mario enjoys and finds meaningful will increase the likelihood that he will take in, process, and integrate tactile inputs as a basis for planning motor actions. While we can not directly observe if this occurs, we can observe if Mario’s motor behavior improves. (Fisher & Murray, 1991).

Jacobson, J. W., Foxx, R.M., & Mulick, J. A. (2010). Controversial therapies for developmental disabilities: fad, fashion, and science in professional practice. New York, NY, Routledge.

Consider the statement…

Statements are Testable: ◦ extent to which statements can be verified or

falsified by undertaking certain operations and manipulations- EXPERIMENT.

◦ Measuring the effects of the manipulated variables on the behavior being studied.

‘Testing the hypothesis out’ distinguishes science from other disciplines.

Elements of Science

Simplification of the conditions: Eliminating factors that are not being studied.

Change only one factor at a time.◦ Why?◦ Baseline-Treatment◦ Repeating introductions and withdrawals than we

can attribute causal effect Group Designs/Statistical Analysis: Average

results across a number of participants.◦ Randomized control design- tx and control groups

Elements of Science ‘Rules of Evidence’

Statements are replicable: When independent investigators can repeat the original operations and make the same observations.

Elements of Science

Most critical and self-correcting kind of knowing.

Asks “How do you know?” A commitment to a science approach

increases knowledge. Discriminates valid from invalid

interventions. Core principles of science are evident in

basic and applied research. Wealth of empirically validated treatments

for developmental disabilities.

Science

Ranking: Ask professionals in field to rank treatments.◦ American Association on Mental Retardation:

formed two expert groups to evaluate psychological interventions and pharmacological interventions for problem behavior (Rush & Frances, 2000).

◦ Scale: 9- extremely appropriate: treatment of choice 1- extremely inappropriate: treatment you would

never use. ◦ Then ranked as treatment of choice.

Evaluating Science Evidence

New York State Department of Health (1999).◦ Screened literature◦ Reviewed relevant articles◦ Had to meet high standards of quality

Adequate information concerning the intervention methods.

Participants were in the relevant age group. Controlled experimental design. Evaluation of functional outcomes.

◦ Summarized each and noted strength of the evidence.

Evaluating Science Evidence

Traditional Approach: unsystematic observations, reflect on own experience, textbooks, ask an expert.

Evidence-based Approach: Supplemented with observations recorded systematically in a standardized way to facilitate analysis. ◦ Define patients problems◦ Focused literature reviews◦ Utilize “Method” and “Results” sections

Developments in Medicine as a Model

Agency for Healthcare Research and Quality (AHRQ).

Twelve evidence-based practice centers.

◦ Evidence reports and technology assessments based of medical literature.

◦ Recent project: Identify and describe systems for assessing quality of scientific articles and rating strength of a body of evidence.

◦ Identified 19 grading systems for rating quality of published studies and 7 for rating strength of evidence.

Developments in Medicine as a Model

Strength of evidence: ◦ Quality: the extent to which their design, content,

and analysis has minimized selection, measurement, and confounding biases.

◦ Quantity: the magnitude of treatment effect, the number of studies, and sample size across all studies.

◦ Consistency: the extent to which similar findings are reported from both similar and different and experimental designs.

EXAMPLE of system used to evaluate strength of evidence TABLE 3.1 PAGE 37

Developments in Medicine as a Model

Slow to emphasize that services must be empirically validated.

McFall(1991) “Manifesto for a Science of Clinical Psychology”.

◦ Scientific Clinical Psychology was the only acceptable form of clinical psychology

Organized Psychology

‘Manifesto…’1) Services only provided when:a) Nature of service is describedb) Claimed benefit is statedc) Benefits must be validated scientificallyd) Negative side effects that outweigh benefits

must be ruled out. 2) Primary objective of doctoral training

programs in clinical psychology must be to produce the most competent clinical scientists.

Organized Psychology

1993 Task Force to validate the treatments used in clinical psychology.

Criteria were established:◦ “Well-established treatments” and “probably

efficacious treatments”.◦ Well established must be significantly more

efficacious than a comparison treatment or placebo in at least two group-design studies or series of controlled single-subject studies, by at least two independent research teams following a written treatment manual.

Organized Psychology

1993 Task Force continued…◦ ABAB design, multiple-baseline design: intervention

is considered:

‘probably efficacious’ if shown to benefit at least three participants.

‘well-established’ if it has at least three replications, with three or more participants, by at least two independent research groups, and absence of conflicting results.

EXAMPLE of Criteria for Empirically Supported Psychological Interventions TABLE 3.2 PAGE 38

Organized Psychology

Generalization to ‘real life’.◦ From laboratory, clinics, clinical trials◦ Lower level of training in ‘real life’

Restrictive list of treatments that will get reimbursed.

Vulnerable to malpractice suits.

Concerns with Identifying Empirically Supported Treatment

Make sound clinical judgment with more consistency.

Avoid the possibility of eliminating psychological interventions by the health care delivery system.

Proponents of Identifying Empirically Supported Treatment

What do you do when encountered with a problem for which no clear evidence

exists favoring one course of action over another?

QUESTION

Behave Scientifically!

ANSWER

1) Assume fallibility (weakness, shortcomings, unreliability).

◦ We may be wrong. Errors that occur in treatment provision:

(O’Donohue, 1997)

◦ False descriptive statements◦ False casual statements◦ False ontological statements◦ False relational claims◦ False predictions◦ False professional ethical claims

Behave Scientifically

2) Have an awareness that all treatments and service offers are based on knowledge claims which should be examined with respect to the quality of their source.

3) Realize that errors in treatment and service delivery can cause serious harm.◦ At very least, takes away from effective treatment

time.

Behave Scientifically

4) Be informed:◦ Accept that we may be wrong.◦ Find out if beliefs are consistent with the scientific

literature.◦ Seek criticism from peers and listen non-

defensively.◦ Conduct clinical practice in such a way to gain

frequent feedback about our decisions and actions: Peer reviews, satisfactory reports, social validity

interviews, program evaluations, client follow up.◦ Give criticism to others.

Behave Scientifically

Increasing demand for accountability (consumers, government, general public).

Increase in threats to the scientific tenets that have fueled treatment development.◦ ungrounded theories and treatments

Threats from postmodernist philosophies.

Summary

Respond by assessing alternative interventions and

by offering better alternatives of evidence-

based knowledge and empirically validated

interventions.

Final Thought…

Questions/Comments

Thank you!

American Medical Association (AMA). (2001). Principles of medical ethics [On-line]. Retrieved May 25, 2012 from http://www.ama-assn.org/ama/pub/physician-resources/medical-ethics/code-medical-ethics/principles-medical-ethics.page

American Psychological Association. (APA). (2010). Ethical Principles of Psychologists and Code of Conduct. Received February 2, 2012 from http://www.apa.org/ethics/code/index.aspx

Behavior Analyst Certification Board (BACB). (2001-2010). Guidelines for responsible conduct for behavior analysts. Retrieved February 2, 2012 from http://www.bacb.com/index.php?page=57

Beutler, l. E. (1998). Identifying empirically supported treatments: What if we didn’t? Journal of Consulting and Clinical Psychology, 66, 113-120.

Green, G. (1996). Evaluating claims about treatments for autism. In C. Maurice, G. Green, & S. C. Luce (Eds.), Behavioral intervention for young children with autism (pp. 15-28). Austin, TX: Pro-Ed.

Jacobson, J. W., Foxx, R.M., & Mulick, J. A. (2010). Controversial therapies for developmental disabilities: fad, fashion, and science in professional practice. New York, NY, Routledge.

McFall, R. M (1991). Manifesto for a science of clinical psychology. The Clinical Psychologist [On-Line]. 44 75-88 Retrieved May 28, 2012 from http://horan.asu.edu/ced522readings/mcfall/manifesto/manifest.htm

New York State Department of Health (1999). Clinical Practice guide: Report of the recommendations. Autism/pervasive developmental disorders, assessment and intervention four young children (age 0-3 years)

( Publication No. 4215) Albany, NY.

O’Donohue,W. (1997, Spring). On behaving scientifically: fallibilism, criticism, and epistemic duties. Clinical Science Newsletter [On-Line], pp2-7. Retrieved on May 26, 2012 from http://sites.google.com/site/sscpwebsite/newsletters-1

References