Using equivalence-based instruction to teach the visual ...
Transcript of Using equivalence-based instruction to teach the visual ...
Received: 9 August 2018 Revised: 27 March 2019 Accepted: 28 March 2019
DOI: 10.1002/bin.1669
R E S E A R CH AR T I C L E
Using equivalence‐based instruction to teach thevisual analysis of graphs
Bryan J. Blair1 | Jonathan Tarbox2 | Leif Albright3 |
Jacquelyn M. MacDonald4 | Lesley A. Shawler1 | Samantha R. Russo1 |
Michael F. Dorsey5
1The Van Loan School, Institute for Behavioral
Studies, Endicott College, Beverly,
Massachusetts
2College of Letters, Arts, and Sciences,
Department of Psychology, University of
Southern California, Los Angeles, California
3Department of Applied Behavior Analysis,
Caldwell University, Caldwell, New Jersey
4School of Health Sciences, Regis College,
Weston, Massachusetts
5Amego, Inc, Attleboro, Massachusetts
Correspondence
Bryan J. Blair, Ph.D., BCBA‐D, Library Learning
Center, Room 228, 1 University Plaza,
Brooklyn, NY 11201.
Email: [email protected]
This manuscript was submitted to the Institute for Behavior
the seventh author in partial fulfillment of the requirements
ical and insightful feedback. Additionally, the authors thank
their assistance. The first author is now an Assistant Profe
Behavioral Interventions. 2019;1–14. wiley
Previous research has demonstrated low rater agreement
when visually inspecting trends in single‐subject design
graphs (see Wolfe, Seaman, & Drasgow, 2016). Didactic
instruction of visual analysis of practicing behavior analysts
has resulted in generally poor and unreliable performances
(Danov & Symons, 2008; Diller, Barry, & Gelino, 2016).
Therefore, a refined instructional technology to improve
the reliability among behavior analysts is warranted. Devel-
oping research has focused on the application of
equivalence‐based instruction (Brodsky & Fienup, 2018;
Rehfeldt, 2011) for a variety of complex human behaviors.
In the current study, equivalence‐based instruction was
used to train four participants to identify functional rela-
tions displayed in five different classes of graphs. Training
resulted in the formation of five equivalence classes by all
participants consisting of three members (graph, functional
relation rule, and functional relation statement). In addition,
the skills were maintained for up to 2 weeks and general-
ized to novel graphs.
KEYWORDS
match‐to‐sample, multiple‐exemplar training, stimulus equivalence,
visual analysis
al Studies in the Van Loan School at Endicott College by the first author under the supervision of
for the doctoral degree in applied behavior analysis. The authors thank Caio Miguel for his crit-
Mark Silberman, Elizabeth Kirby, staff at Hopeful Journeys, and faculty at Caldwell University for
ssor at Long Island University ‐ Brooklyn.
© 2019 John Wiley & Sons, Ltd.onlinelibrary.com/journal/bin 1
2 BLAIR ET AL.
1 | INTRODUCTION
One of the fundamental qualities of behavior analysis is its use of visual analysis of data. Unlike other fields (e.g., psy-
chology) that rely on statistical comparisons and the standardization of collected data, behavior analysts rely almost
exclusively on the visual inspection of summarized and plotted data over time. An extensive line of research has sug-
gested that teaching visual analysis of graphs using standard didactic approaches and traditional supervisory methods
might not lead to the development of fluent and reliable visual analytic repertoires in practicing behavior analysts
(Danov & Symons, 2008; DeProspero & Cohen, 1979; Diller et al., 2016; Ninci, Vannest, Willson, & Zhang, 2015;
Ottenbacher, 1993; Wolfe et al., 2016). However, a few studies have shown that when participants are directly
taught specific visual analysis rules, agreement across participants improved (Hagopian et al., 1997; Roane, Fisher,
Kelley, Mevers, & Bouxsein, 2013).
The essential skill for behavior analysts, specifically, is to identify whether graphically presented data demonstrate
a functional relation between the independent and dependent variables. This skill is necessary to determine whether
interventions are effective, and in some cases, to identify the specific components of interventions that are control-
ling the target behavior(s). However, few studies have investigated different instructional methods for teaching stu-
dents and practitioners to reliably visually analyze graphs to determine functional relations between independent and
dependent variables in more common behavior analytic experimental designs (Wolfe et al., 2016).
One promising instructional framework that focuses on the direct teaching of a few specific skills and the emer-
gence of novel responding is the application of derived stimulus relations methodologies (Hayes, Barnes‐Holmes, &
Roche, 2001) in general, of which stimulus equivalence (Sidman, 1994) appears to be an example. In several studies,
participants were exposed to direct teaching methods with multiple exemplars, resulting in the emergence of rela-
tional responding across an array of stimulus parameters, participants, training settings, and instructional protocols
(Hayes et al., 2001; Sidman, 1994).
When stimuli that share no topographical similarities become part of the same class of stimuli, an equivalence
class has formed (Green & Saunders, 1998; Sidman, 1994; Sidman & Tailby, 1982). In order to demonstrate that
an equivalence class has emerged, the properties of reflexivity (A = A), symmetry (if A = B then B = A), and transitivity
(if A = B and B = C then A = C) must be demonstrated among the stimuli that form the class. For an equivalence class
to form, a learner must consistently select a member of an equivalence class when presented with other members of
the same class. Conditional discrimination training, specifically match‐to‐sample (MTS) training, is a common method
of establishing equivalence classes. There have been a number of published studies on the establishment of stimulus
equivalence (see Sidman, 1994; Stromer, Mackay, & Stoddard, 1992, for thorough reviews). Additionally, a growing
number of investigations have been conducted using equivalence‐based instruction (EBI) to teach standard
college‐level content to typically developing adults (Brodsky & Fienup, 2018; Rehfeldt, 2011).
In a series of studies (Critchfield & Fienup, 2010; Fields et al., 2009; Fienup, Covey, & Critchfield, 2010; Fienup &
Critchfield, 2010), EBI was used to teach college student complex skills such as those required for interpreting statis-
tics, and for relating facts about brain anatomy. The studies used computer‐based training to teach and test the rela-
tions involved in the equivalence classes. In all studies, derived relational responses emerged and generalized to novel
exemplars. In another study that used EBI procedures, Walker and Rehfeldt (2012) taught graduate students in a
distance‐learning program to identify single‐subject design graphs via a computer‐based system. Participants learned
to select the definitions of research designs and graphs when presented with the names, and to select vignettes when
presented with definitions of designs. After training, participants demonstrated several emergent selection‐based and
topographical responses including selecting and writing the names of designs when presented with graphs and
vignettes, and selecting the definitions of designs when presented with vignettes. The results of the study suggested
that with direct training of specific discriminative responses, derived relational responses emerged. The authors con-
cluded that EBI was an efficient method for teaching higher‐level concepts to college students with minimal instruc-
tion (Walker & Rehfeldt, 2012). However, one limitation noted was that the training did not result in acceptable
generalization to novel stimuli for all participants.
BLAIR ET AL. 3
Two studies that used EBI to teach academic skills were recently published (Albright, Reeve, Reeve, & Kisamore,
2015; Albright, Schnell, Reeve, & Sidener, 2016). In both studies, the investigators used EBI to teach complex skills
such as the concepts of statistical variability and the interpretation of the results of functional analysis graphs to col-
lege students. Custom computer software deliveredMTS trials in the experimental sessions in a pretest‐train‐posttest‐
maintenance design. They incorporated multiple‐exemplar training (MET) in order to promote generalization. In the
first study, participants learned to select statistical category definitions, sample numbers, and standard deviation def-
initions when presented with category names. In the second study, participants learned to select graph descriptions
and graphs when presented with the function and to select a vignette when presented with a graph description. Both
studies resulted in substantial increases in scores from pretest to posttest conditions and equivalence classes formed
for all participants. Results also suggested that skills were maintained at 1 week, and generalized to novel stimuli.
The purpose of the current study was to systematically replicate and extend studies on EBI (e.g., Albright et al.,
2016; Walker & Rehfeldt, 2012) with an eminently applied skill for behavior analysts. In recent studies, EBI has been
shown to be effective in teaching relevant academic skills to college students and the current study further extends
this line of research. The specific purpose of the current study was to investigate whether a visual–visual MTS pro-
cedure with MET, in which conditional discriminations between graphs and analytic rules, and between analytic rules
and functional relation statements, were taught, resulted in the formation of equivalence classes based on the level of
functional relation, including the emergence of derived relational responding in the form of selecting a functional rela-
tion statement when presented with an ABAB design graph.
2 | METHOD
2.1 | Participants
Two undergraduate students and two direct care staff who worked at an agency that provides services to children
diagnosed with autism spectrum disorder, participated in the study. We recruited participants via direct email, posts
on an online course site, and posts on social media. The four participants (one man and three women) ranged in age
from 19‐to 25‐years‐old. A screening tool was used to determine the participants' level of experience with, and expo-
sure to, the visual analysis of graphs. No participants had any experience with studying or implementing single‐
subject designs. The Endicott College Institutional Review Board approved this research project. Prior to participa-
tion, all participants signed a consent form and they received a gift card for participating in the study, regardless of
whether the participant completed all parts of the training and testing.
2.2 | Setting and materials
Experimental sessions were conducted in a small student lounge or a conference room with desktop or laptop com-
puters. For all sessions, we used the same custom web‐based software that Albright et al. (2016) developed specif-
ically for use in EBI research to design the EBI system and to deliver and control the MTS training and testing
sessions (the software is not commercially available). The developers of the EBI training software designed the sys-
tem to automatically present visual–visual MTS training and testing trials and conditions to the participant via a web
browser based on the mastery criteria and in the order that was programmed by the researcher. In addition, the soft-
ware recorded participants' responses as data in spreadsheet table form.
2.3 | Experimental stimuli, training protocol, and training structure
The visual stimuli (see Table 1 and Figure 1) used were graphs (denoted as A), written visual analysis rule statements
(denoted as B), and written functional relation statements (denoted as C), displayed on a computer screen. Five
TABLE
1Exp
erim
entalstim
uli
Class
Labe
lNotation
12
34
5
Graph
Ax
Ay
Az
Rule
BThe
reisaclea
rch
ange
in
leve
land
tren
d
betw
eenba
selin
ean
d
trea
tmen
tco
nditions
The
reisach
ange
inleve
l
betw
eenba
selin
ean
d
trea
tmen
tco
nditions
andthe
tren
disvariab
leacross
cond
itions
The
tren
dsin
allc
ond
itions
are
toovariab
leto
makea
determ
inationofwhe
ther
thereisafunc
tiona
lrelation
Thereisaslightch
ange
inleve
l
betwee
nbaselinean
dtrea
tmen
t
conditionsan
dthetren
dis
variab
leacross
conditions
Thereisnoch
ange
in
leve
lortren
d
betwee
nbaseline
andtrea
tmen
t
conditions
Statem
ent
CFun
ctiona
lrelation
Proba
blefunc
tiona
lrelation
Unc
lear
Probab
lynofunctional
relation
Nofunctional
relation
4 BLAIR ET AL.
FIGURE 1 Trained (solid arrow) and tested (dotted arrow) relations
BLAIR ET AL. 5
equivalence classes (1. Functional Relation, 2. Probable Functional Relation, 3. Unclear, 4. Probably No Functional
Relation, and 5. No Functional Relation) were established. The graphs were 30 ABAB figures with varying behaviors,
levels, trends, and numbers of data points per condition and were created for this study and did not represent actual
clinical or research data. Fifteen graphs were used during training, and 15 novel exemplars were used for the gener-
alization probes. We chose ABAB because, “they are the most straightforward and powerful within‐subject design for
demonstrating functional relations between environmental manipulations and a behavior” (Cooper, Heron, & Heward,
2007, p. 177). To program for generalization, we included three exemplars of each graph (denoted as Ax Ay, and Az)
for each equivalence class. Prior to the study, three PhD level BCBAs (BCBA‐Ds) reliably identified the graphs by
class. For the functional relation rule (B) and functional relation statement (C) stimuli, one stimulus was included in
each class. During all sessions, all stimuli were presented as black‐and‐white images on a computer screen. The size
of the stimuli that the software displayed depended on the computer monitor size and resolution; however, for this
study, all stimuli were approximately 7.5 cm × 7.5 cm.
We used a simple‐to‐complex training protocol and linear training structure (Table 2 and Figure 2) for this exper-
iment (Arntzen, 2012; Saunders & Green, 1999). We designed the system to teach the AB baseline discriminations,
then present the BA symmetry test, then teach the BC baseline discriminations, then present the CB symmetry test.
Finally, the system presented the AC transitivity test followed by the CA equivalence test. The system automatically
presented training and testing conditions based on the programmed mastery criteria for each condition.
2.4 | Experimental design and dependent variables
We used a single‐session pretest‐posttest design, embedded in a two‐tier nonconcurrent multiple baseline across
participants design (Carr, 2005; Christ, Cooper, Heron, & Heward, 2007; Watson & Workman, 1981) to examine
experimental control and assess the effects of conditional discrimination training in the form of MTS on the forma-
tion of five three‐member classes. We exposed Participants 2 and 4 to two pretests, conducted in two successive
conditions in a single session, to control for testing effects (Griffith, Ramos, Hill, & Miguel, 2018). Finally, we con-
ducted maintenance and generalization probes for all participants. The primary dependent variable was the percent-
age of correct trials for all relations, particularly transitivity and equivalence relations, during testing conditions. We
also calculated the trials‐to‐criterion for all participants for AB and BC training conditions, and training and testing
durations.
TABLE 2 Training and testing sequence and parameters
Condition Relations trained or tested Trials per block Mastery criterion
Interface Training N/A N/A N/A
Pretest AB, BA, AC, CA 60 N/A
BC, CB 10 N/A
AB Train AB 15 15/15 in one block
BA Test BA 15 14/15 in one block
BC Train BC 5 5/5 in one block
CB Test CB 5 5/5 in one block
AC Test AC 15 14/15 in one block
CA Test CA 15 14/15 in one block
Posttest AB, BA, AC, CA 60 N/A
BC, CB 10 N/A
Generalization AC 15 N/A
Maintenance AB, BA, AC, CA 60 N/A
BC, CB 10 N/A
FIGURE 2 Training protocol (simple‐to‐complex) and structure (linear)
6 BLAIR ET AL.
2.5 | Procedural integrity and interobserver agreement
The researcher used a written checklist to ensure that the correct conditions were conducted with the correct
participants and the researcher read instructions to ensure that all participants were provided with the appropriate
BLAIR ET AL. 7
instructions for each condition. Primary and secondary observers assessed procedural integrity via recorded video
in 33% of experimental sessions. According to the primary observer, the researcher completed 100% of the steps
correctly. We calculated interobserver agreement for procedural integrity by dividing the total number of steps
with observer agreement by the total number of possible steps and multiplying by 100. Interobserver agreement
was 100%.
2.6 | Procedure
2.6.1 | Visual analysis social validity
Prior to the start of the study, 12 Master's level Board Certified Behavior Analysts® (BCBAs) enrolled in a PhD pro-
gram in ABA were asked to assess the level of functional relation for 30 ABAB graphs.
2.6.2 | Visual–visual MTS training and testing system
Prior to the start of a block of trials the system presented the following instructions to participants, “Select the stim-
ulus that matches the sample stimulus. During this block you will be immediately notified if your selection was correct
or incorrect” or, “During this block you will not be notified if your selection was correct or incorrect.” After clicking
“OK,” the system presented the first trial.
A trial consisted of the presentation of a visual sample stimulus at the top of the computer screen, the display of
the total number of trials in the training/testing block, the number of trials remaining, and a progress indicator. When
the participant clicked the sample stimulus (e.g., A1) with a computer mouse, the system presented five visual com-
parison stimuli on the screen (one correct member (e.g., B1) of the class and four incorrect members, one from each
of the other classes (e.g., B2, B3, B4, and B5) in an array (randomly located) at the bottom of the screen. Trial‐and‐
error instruction was used for the MTS training conditions. During training trials, clicking on the comparison stimulus
resulted in immediate feedback (“Correct” or “Incorrect”). If a participant selected the correct stimulus, it was identi-
fied as correct, and when the participant clicked anywhere on the screen the system presented the next trial; how-
ever, if a participant selected an incorrect stimulus, the correct stimulus was not identified and the next trial was
presented. During testing trials, clicking on any comparison stimulus resulted in no feedback and the next trial was
immediately presented. Prior to the start of sessions, participants completed a simple computer‐based visual–visual
MTS task (color name‐to‐color) in order to familiarize themselves with the computer software and matching task.
All participants responded correctly for all interface training trials.
2.6.3 | Pretest
Immediately following the interface training, the researcher initiated the pretest condition. The pretest condition ran-
domly presented all possible conditional discriminations and exemplars (graph‐to‐functional relation rule [AB], func-
tional relation rule‐to‐functional relation statement [BC], functional relation rule‐to‐graph [BA], functional relation
statement‐to‐functional relation rule [CB], graph‐to‐functional relation statement [AC], and functional relation
statement‐to‐graph [CA]) as individual trials one time in a single block of trials. At the end of the pretest, the
researcher immediately started the MTS training and testing session.
2.6.4 | MTS training and symmetry tests
During the baseline training conditions (AB and BC), the system trained participants to select the correct comparison
stimulus from an array when presented with a sample stimulus. Trials were presented in blocks until participants met
the mastery criterion (100% correct for one block). Immediately following a successful baseline training condition, the
8 BLAIR ET AL.
system automatically presented participants with the corresponding symmetry (BA or CB) test. If a participant failed a
symmetry test, the corresponding baseline relation training condition was presented again.
2.6.5 | Transitivity and equivalence probes
Immediately following successful symmetry tests, the system presented participants with transitivity (AC) and equiv-
alence (CA) probes. If a participant failed either these tests, the AB training condition was presented again and the
participant proceeded through the training and testing system until mastery criteria for all conditions were met
(Figure 2).
2.6.6 | Posttest
Immediately following a successful equivalence (CA) test, the researcher initiated the posttest. The posttest condition
was identical to the pretest condition.
2.6.7 | Generalization
We initially conducted generalization probes with BAB reversal graphs with Participants 1 and 3 immediately follow-
ing the posttest (data not shown). However, due to a lower than expected performance for Participant 3, we also
conducted generalization probes with ABAB reversal graphs with all four participants. Due to this change in method
during the study, we conducted generalization probes after maintenance probes for Participants 1 and 3 and prior to
maintenance probes for Participants 2 and 4. Probes were identical to the AC test condition except with novel
graphs.
2.6.8 | Maintenance
All participants completed maintenance probes an average of 15 days (range, 14 to 17) after the training sessions.
The maintenance test was identical to the pretest and posttest conditions.
2.6.9 | Satisfaction survey
At the conclusion of the study, all participants voluntarily completed a satisfaction survey (see Table 3).
TABLE 3 Satisfaction survey results
Question Mean Range (1–5)
Prior to this training, how confident were you in your ability to visually analyze graphs? 1.5 1–2
After this training, how confident are you in your ability to visually analyze graphs? 3.5 3–4
The computer‐based training helped me learn about visually analyzing graphs. 4.0 4–4
I felt successful during the training and testing. 4.0 3–5
Compared to other instructional methods (e.g., lecture, reading a textbook,
and study guide questions), rate your preference for this type of instruction.
2.8 2–4
The computer‐based training frustrated me. 3.3 2–4
I would use this type of instruction to learn other concepts and to help me master other material. 3.3 3–4
I would recommend this type of instruction to others. 3.3 2–4
BLAIR ET AL. 9
3 | RESULTS
The mean score for the visual analysis social validity test that was completed by 12 Master's level BCBAs enrolled in
a PhD program in ABA was 73% (range, 60% to 90%). Participant scores for the pretests were below 60% (range,
35% to 57%) for all four participants (see Figures 3 and 4). Both Participants 2 and 4 showed slight increases in over-
all scores from pretest 1 to pretest 2; however, the performance for CA relations for Participant 2 decreased. Imme-
diately after completing the training conditions, Participants 1, 3, and 4 passed symmetry (BA, CB), transitivity (AC),
and equivalence (CA) tests, whereas Participant 2 failed symmetry and transitivity tests and required remedial training
to meet the mastery criteria. The mean total number of trials (Table 4) to complete training and testing was 234
(range, 150 to 440). The mean duration of MTS training and emergent relation testing sessions was 49 min (range,
13 min to 2 hr 16 min). The posttest scores for all relations and participants showed a substantial improvement in
performance when compared to pretest scores, with little variability across participants. All participants scored
89% or higher on the posttest (range, 89% to 97%) for all relations.
There was some variation in participant responding during the ABAB generalization probes. The mean score for all
participants was 80% (range, 73% to 87%). The maintenance scores for all relations and participants showed that
trained and emergent relational responding maintained for 2 weeks for three out of four participants (the score for
Participant 3 dropped to 77% during the maintenance probe).
Table 3 shows the results of the satisfaction survey that all participants completed at the end of the research. The
results suggest that all participants were generally satisfied with and accepting of the training procedures.
4 | DISCUSSION
Research in the area of stimulus equivalence and derived relational responding (Hayes et al., 2001; Sidman, 1994) has
focused on the acquisition of basic discriminative and emergent responses in structured settings. However, a recent
FIGURE 3 Results for Participants 1 and 2
FIGURE 4 Results for Participants 3 and 4
TABLE 4 Total number of experimental trials and duration of training and testing required for mastery criterion, byparticipant
Trials to mastery criterion
Participant AB BA BC CB AC CA Remedial Duration
1 105 15 5 5 15 15 N 13 min
2 270 45 60 20 30 15 Y 2 hr 16 min
3 135 15 10 5 15 15 N 23 min
4 90 15 10 5 15 15 N 26 min
10 BLAIR ET AL.
focus on using EBI with MET has yielded promising results with a wide range of learners and across an array of com-
plex skills (Brodsky & Fienup, 2018; Rehfeldt, 2011). When compared to more traditional didactic teaching methods,
this methodology of instruction appears to be an effective alternative, especially when the formation of equivalence
classes, as opposed to simple stimulus generalization, is the goal of the instruction.
In the current study, we used a web‐based EBI system to teach visual analysis of graphical data to undergraduate
students and direct care staff who had no experience with the analysis of single‐subject data graphs. For all partici-
pants, five three‐member classes of arbitrarily related stimuli were formed using automated visual–visual MTS and
MET training of baseline discriminations (AB and BC). In addition, the skills maintained for at least 2 weeks and were
applied to several novel exemplars. All participants who completed the study learned the directly trained baseline dis-
criminations, derived relational responding immediately emerged for three out of four participants, and all partici-
pants passed the tests for the emergence of symmetrical relations (BA, CB), and transitivity and equivalence
relations (AC, CA) by the end of training and testing. These results replicate and extend recent studies (Albright
et al., 2015; Albright et al., 2016; Fienup & Critchfield, 2010; Walker & Rehfeldt, 2012) where the current study
BLAIR ET AL. 11
better controlled for testing effects when compared to previous studies, and with novel complex stimuli that are
socially relevant to behavior analysts.
The current study also replicated and extended the results fromWalker and Rehfeldt (2012) by using EBI to teach
participants to visually analyze data in single‐subject research designs. However, where the researchers trained par-
ticipants to identify and define single‐subject research designs using a stimulus equivalence paradigm and MET, the
current study extended the identification of research designs to the selection of functional relation statements
related to ABAB graphs using a similar training procedure. In contrast to some previous studies, the data from the
current study suggest that not only did equivalence classes form and participants learned to discriminate functional
relations, but the skills also generalized to novel stimuli.
In the current study, 12 BCBAs enrolled in a PhD program correctly matched a graph to a functional relation
statement in only 73% of opportunities. By comparison, the four participants who completed EBI training in the cur-
rent study accurately selected a functional relation statement when presented with a graph in 93% (range, 85% to
100%) of opportunities immediately following training, and in 81% (range, 73% to 90%) of opportunities 2 weeks
after training. Participants in the current study also accurately selected a functional relation statement when pre-
sented with a novel graph in 80% (range, 73% to 87%) of opportunities. In addition, three out of four participants
required less than 30 min of training to demonstrate this ability, which is considerably less time than is usually
required to teach this skill using more traditional methods. The results of the satisfaction survey add to previously
published literature that showed that participants are generally receptive to this type of instruction and prefer it to
other types of instruction (Albright et al., 2015; Albright et al., 2016). The results of the satisfaction survey for this
study confirmed that participants had very little confidence in their ability to visually analyze graphs prior to the train-
ing and felt fairly confident in their skills after training.
The primary limitation of this study is the use of a pretest/posttest design where only two participants were
exposed to multiple pretest probes. The design significantly limits the conclusions of the study, and future research
should choose a stronger design. Another limitation to this study was the results for generalization. After training,
one participant did not generalize the identification of functional relation statements to BAB graphs, which prompted
us to include generalization probes with ABAB reversal graphs for all participants. When presented with novel ABAB
graphs all four participants correctly identified the functional relation statement at relatively high percentages. Given
the limited number of exemplars used in the current study, future studies should investigate how reliably the rule
statements can be applied to a variety of graphs of different research design types or even actual clinical data. In
addition, future research should test how the rules learned in this study apply to clinical decision‐making behavior.
Future research should also investigate the parameters of the exemplars used in training in order to ensure the gen-
eralization of responding to novel stimuli without the need for further training.
A third limitation of the study was the high number of training trials needed to meet training criteria and pass the
emergent relations (AC, CA) tests for Participant 2. Compared to other studies on EBI (e.g., Albright et al., 2016), the
results for this participant appear to be somewhat atypical. Given that the visual analysis of graphs is a complex, pos-
sibly rule‐governed behavior that might include unobserved covert behavior (e.g., tacts and intraverbals), the slow
acquisition might be due in part to the topography of the behavior itself. The slower acquisition of the trained con-
ditional discriminations, and lack of immediate emergence of untrained responding, might have also been related to
the inherent complexity of the compound stimuli (written phrases and graphs). Anecdotal reports from participants
revealed that many had never seen single‐subject graphs before and as a result struggled with the initial AB training
blocks. However, after a participant “got it,” the participant appeared to progress through the training more quickly,
and the results of the study, particularly the immediate emergence of derived relations following training for three
out of four participants, support these self‐assessments. Future research should investigate prerequisite skills of par-
ticipants, conditional discrimination training in EBI with complex/compound visual and written stimuli, fluency with
the terms and possible technical jargon in the stimuli, and pre‐training instructions given to participants.
A fourth limitation was that it is possible that participants were exposed to further instruction or self‐initiated
study in the area of visual analysis. However, given the somewhat arbitrary nature of the stimuli included in the
12 BLAIR ET AL.
current study, the fact that participants were all full‐time undergraduate students or worked full‐time, and the fact
that they likely did not have many opportunities to be exposed to the skill of visually analyzing graphs in the
2 weeks between posttest and maintenance, it appears unlikely that any other variables influenced the perfor-
mance during maintenance.
A final limitation of the study was related to the skill itself that was taught. A functional relation is not a binary
phenomenon—there are degrees to the amount of control that an independent variable has on a dependent variable,
especially in nonlaboratory settings (e.g., classrooms, home, and community). However, the stimuli used in this study
were admittedly somewhat arbitrarily created by behavior analysts according to simplified generally accepted rules.
Future studies in the area of visual analysis should use systematic criteria and rules for classifying graphs (e.g., over-
lapping data points and slopes of trends) and not simply rely on interrater agreement for the establishment of cate-
gories of graphs.
For the current study, we probed for the maintenance of skills only once following training. Although the results
of the study suggest that participants mastered the skills, equivalence classes formed, and responses maintained for
several weeks for most participants, the long‐term effects of a single training session of EBI are unknown. Future
research should focus on the effects of EBI on long‐term maintenance as well as the effects of shorter‐duration
refresher/remedial trainings on maintenance and generalization.
Future research should also investigate using EBI to teach other applied behavior analytic skills associated with
clinical activities. For example, some applied skills might include selecting appropriate function‐based treatment,
assessment procedures, and data collection methods. The current study used EBI to teach the visual analysis of ABAB
reversal graphs; however, future studies could investigate the efficacy of EBI for teaching visual analysis skills for
other graphs, specifically multiple baseline graphs given their widespread use in ABA research.
The results of the current study confirm that typically developing adults can discriminate topographically dis-
similar visual stimuli (pictures‐to‐words) and that the trained responses, with the incorporation of MET, can lead
to the emergence of derived relational responses, or discriminations, and that the responding can generalize to
novel stimuli. However, it is possible that a participant in the current study learned the rule that describes differ-
ent types of graphs and then used that rule to identify the extent of functional relation demonstrated by other
graphs. Given the fact that the graph‐to‐functional relation statement discrimination emerged for all participants
and generalized to novel stimuli, it is reasonable to question whether some part of that performance was due
to the emergence of a relation not only among the three stimuli in the functional relation statement class, but also
among the verbal stimuli in the rule itself (Barnes‐Holmes et al., 2002; Tarbox, Tarbox, & O'Hora, 2009; Tarbox,
Zuckerman, Bishop, Olive, & O'Hora, 2011). The slight increases in pretest scores for Participants 2 and 4 might
also be explained by the use of verbal problem‐solving skills and/or the participants' history with written
selection‐based assessments (i.e., multiple choice tests). However, future research in this area, specifically with
EBI for written visual stimuli in applied settings, is warranted.
Although the results of the current study demonstrated that EBI was used to teach a fairly complex behavior ana-
lytic skill, the skill itself is not exactly the behavior that clinicians engage in when they are analyzing data graphs. Ana-
lyzing data in actual clinical settings is much more complicated and nuanced than the analysis of the graphs that was
demonstrated in the current study. Additionally, the actual verbal behavior (either covert or overt) that a participant
engaged in during the analysis was not measured or assessed. For example, at the conclusion of the training, a par-
ticipant could have been asked to describe the graph, and the participant's response could have been compared to
the rule that was directly taught. Future research should assess a participant's verbal behavior when engaging in ana-
lytic activities, not just visually analyzing graphs. In addition, future studies should test for the emergence of
untrained topographical responses (Reyes‐Giordano & Fienup, 2015) including written and vocal tacts of particular
aspects of the graph (i.e., identifying level, trend, and variability) and tacts of the level of functional relation itself. This
line of research would provide further distinction between the establishment of equivalence classes as a result of
conditional discrimination performances without verbal behavior and performances with the use of verbal behavior
(Jennings & Miguel, 2017; Petursdottir, Carp, Peterson, & Lepper, 2015).
BLAIR ET AL. 13
Complex human behavior has been under both theoretical and experimental investigation since the dawn of man-
kind. Traditional approaches tend to be based on a priori assumptions and appeals to mentalistic constructs that leave
both the scientist and practitioner with an empirically unsupported foundation upon which to build a set of teaching
principles and protocols. However, data from studies like the current investigation allow both scientist and practi-
tioner to better understand and apply tools such as EBI and MET when training complex human behavior. However,
this area is admittedly open for conceptual analysis, and the field of behavior analysis is strongly encouraged to
engage in this conversation. More research needs to be conducted, and much like the training procedures them-
selves, only with multiple exemplars of research studies will a clear and concise conceptual understanding of complex
human behavior, along with practical and empirically supported technologies to train behavior analysts, ultimately
emerge.
ORCID
Bryan J. Blair https://orcid.org/0000-0002-1787-1676
REFERENCES
Albright, L., Reeve, K. F., Reeve, S. A., & Kisamore, A. N. (2015). Teaching statistical variability with equivalence‐basedinstruction. Journal of Applied Behavior Analysis, 48(4), 883–894. https://doi.org/10.1002/jaba.249
Albright, L., Schnell, L., Reeve, K. F., & Sidener, T. M. (2016). Using stimulus equivalence‐based instruction to teach graduate
students in applied behavior analysis to interpret operant functions of behavior. Journal of Behavioral Education, 25(3),
1–20. https://doi.org/10.1007/s10864‐016‐9249‐0
Arntzen, E. (2012). Training and testing parameters in formation of stimulus equivalence: Methodological issues. European
Journal of Behavior Analysis, 13(1), 123–135. https://doi.org/10.1080/15021149.2012.11434412
Barnes‐Holmes, D., Denis, O., Roche, B., Hayes, S. C., Bissett, R. T., & Lyddy, F. (2002). Understanding and verbal regulation.
In Relational frame theory (pp. 103–117). Springer US. https://doi.org/10.1007/0‐306‐47638‐X_6
Brodsky, J., & Fienup, D. M. (2018). Sidman goes to college: A meta‐analysis of equivalence‐based instruction in higher
education. Perspectives on Behavior Science, 41(1), 1–25. https://doi.org/10.1007/s40614‐018‐0150‐0
Carr, J. E. (2005). Recommendations for reporting multiple‐baseline designs across participants. Behavioral Interventions,
20(3), 219–224. https://doi.org/10.1002/bin.191
Christ, T. J., Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Experimental control and threats to internal validity of con-
current and nonconcurrent multiple baseline designs. Psychology in the Schools, 44(5), 451–459. https://doi.org/10.1002/pits.20237
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis. Upper Saddle River, NJ: Pearson.
Critchfield, T. S., & Fienup, D. M. (2010). Using stimulus equivalence technology to teach statistical inference in a group
setting. Journal of Applied Behavior Analysis, 43(4), 763–768. https://doi.org/10.1901/jaba.2010.43‐763
Danov, S. E., & Symons, F. J. (2008). A survey evaluation of the reliability of visual inspection and functional analysis graphs.
Behavior Modification, 32(6), 828–839. https://doi.org/10.1177/0145445508318606
DeProspero, A., & Cohen, S. (1979). Inconsistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis,
12(4), 573–579. https://doi.org/10.1901/jaba.1979.12‐573
Diller, J. W., Barry, R. J., & Gelino, B. W. (2016). Visual analysis of data in a multielement design. Journal of Applied Behavior
Analysis, 49(4), 980–985. https://doi.org/10.1002/jaba.325
Fields, L., Travis, R., Roy, D., Yadlovker, E., Aguiar‐Rocha, L. D., & Sturmey, P. (2009). Equivalence class formation: A method
for teaching statistical interactions. Journal of Applied Behavior Analysis, 42(3), 575–593. https://doi.org/10.1901/
jaba.2009.42‐575
Fienup, D. M., Covey, D. P., & Critchfield, T. S. (2010). Teaching brain‐behavior relations economically with stimulus
equivalence technology. Journal of Applied Behavior Analysis, 43(1), 19–33. https://doi.org/10.1901/jaba.2010.43‐19
Fienup, D. M., & Critchfield, T. S. (2010). Efficiently establishing concepts of inferential statistics and hypothesis decision
making through contextually controlled equivalence classes. Journal of Applied Behavior Analysis, 43(3), 437–462.https://doi.org/10.1901/jaba.2010.43‐437
Green, G., & Saunders, R. R. (1998). Stimulus equivalence. In K. A. Lattal, & M. Perone (Eds.), Handbook of research methods in
human operant behavior (pp. 229–262). New York: Plenum Press.
14 BLAIR ET AL.
Griffith, K. R., Ramos, A. L., Hill, K. E., & Miguel, C. F. (2018). Using equivalence‐based instruction to teach piano skills to
college students. Journal of Applied Behavior Analysis, 51(2), 207–219. https://doi.org/10.1002/jaba.438
Hagopian, L. P., Fisher, W. W., Thompson, R. H., Owen‐DeSchryver, J., Iwata, B. A., & Wacker, D. P. (1997). Toward the
development of structured criteria for interpretation of functional analysis. Journal of Applied Behavior Analysis, 30(2),
313–326. https://doi.org/10.1901/jaba.1997.30‐313
Hayes, S. C., Barnes‐Holmes, D., & Roche, B. (2001). Relational frame theory: A post‐Skinnerian account of human language and
cognitionSpringer Science & Business Media.
Jennings, A. M., & Miguel, C. F. (2017). Training intraverbal bidirectional naming to establish generalized equivalence class
performances. Journal of the Experimental Analysis of Behavior, 108(2), 269–289. https://doi.org/10.1002/jeab.277
Ninci, J., Vannest, K. J., Willson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single‐case data: A
meta‐analysis. Behavior Modification, 39(4), 510–541. https://doi.org/10.1177/0145445515581327F
Ottenbacher, K. J. (1993). Interrater agreement of visual analysis in single‐subject decisions: Quantitative review and
analysis. American Journal on Mental Retardation, 98(1), 135–142.
Petursdottir, A. I., Carp, C. L., Peterson, S. P., & Lepper, T. L. (2015). Emergence of visual–visual conditional discriminations.
Journal of the Experimental Analysis of Behavior, 103(2), 332–348. https://doi.org/10.1002/jeab.136
Rehfeldt, R. A. (2011). Toward a technology of derived stimulus relations: An analysis of articles published in the Journal of
Applied Behavior Analysis, 1992‐2009. Journal of Applied Behavior Analysis, 44(1), 109–119. https://doi.org/10.1901/jaba.2011.44‐109. https://doi.org/10.1901/jaba.2011.44‐109
Reyes‐Giordano, K., & Fienup, D. M. (2015). Emergence of topographical responding following equivalence‐basedneuroanatomy instruction. The Psychological Record, 65(3), 495–507. https://doi.org/10.1007/s40732‐015‐0125‐4
Roane, H. S., Fisher, W. W., Kelley, M. E., Mevers, J. L., & Bouxsein, K. J. (2013). Using modified visual‐inspection criteria to
interpret functional analysis outcomes. Journal of Applied Behavior Analysis, 46(1), 130–146. https://doi.org/10.1002/jaba.13
Saunders, R. R., & Green, G. (1999). A discrimination analysis of training‐structure effects on stimulus equivalence outcomes.
Journal of the Experimental Analysis of Behavior, 72(1), 117–137. https://doi.org/10.1901/jeab.1999.72‐117
Sidman, M. (1994). Equivalence relations and behavior: A research story. Boston: Authors Cooperative.
Sidman, M., & Tailby, W. (1982). Conditional discrimination vs. matching to sample: An expansion of the testing paradigm.
Journal of the Experimental Analysis of Behavior, 37(1), 5–22. https://doi.org/10.1901/jeab.1982.37‐5
Stromer, R., Mackay, H. A., & Stoddard, L. T. (1992). Classroom applications of stimulus equivalence technology. Journal of
Behavioral Education, 2(3), 225–256. https://doi.org/10.1007/BF00948817
Tarbox, J., Tarbox, R., & O'Hora, D. (2009). Nonrelational and relational instructional control. Derived relational responding.
In R. A. Rehfeldt, & Y. Barnes‐Holmes (Eds.), Derived relational responding: Applications for learners with autism and other
developmental disabilities: A progressive guide to change (pp. 111–127). New Harbinger Publications.
Tarbox, J., Zuckerman, C. K., Bishop, M. R., Olive, M. L., & O'Hora, D. P. (2011). Rule‐governed behavior: Teaching a
preliminary repertoire of rule‐following to children with autism. The Analysis of Verbal Behavior, 27(1), 125–139.https://doi.org/10.1007/BF03393096
Walker, B. D., & Rehfeldt, R. A. (2012). An evaluation of the stimulus equivalence paradigm to teach single‐subject design to
distance education students via blackboard. Journal of Applied Behavior Analysis, 45(2), 329–344. https://doi.org/
10.1901/jaba.2012.45‐329
Watson, P. J., & Workman, E. A. (1981). The non‐concurrent multiple baseline across‐individuals design: An extension of the
traditional multiple baseline design. Journal of Behavior Therapy and Experimental Psychiatry, 12(3), 257–259. https://doi.org/10.1016/0005‐7916(81)90055‐0
Wolfe, K., Seaman, M. A., & Drasgow, E. (2016). Interrater agreement on the visual analysis of individual tiers and functional
relations in multiple baseline designs. Behavior Modification, 40, 1–22. https://doi.org/10.1177/0145445516644699
How to cite this article: Blair BJ, Tarbox J, Albright L, et al. Using equivalence‐based instruction to teach the
visual analysis of graphs. Behavioral Interventions. 2019;1–14. https://doi.org/10.1002/bin.1669