PenberthyMillarDisseminationArticle

P1: LHE/hgi P2: GKZ

Innovative Higher Education PH112-370188 June 10, 2002 10:47 Style file version June 4th, 2002

Innovative Higher Education, Vol. 26, No. 4, Summer 2002 ( C© 2002)

The “Hand-off” as a Flawed Approachto Disseminating Innovation: Lessonsfrom Chemistry

Debra L. Penberthy and Susan B. Millar

ABSTRACT: Drawing on studies of active learning methods in an undergraduate chem-istry course at a research institution, we first present two case studies exploring thechange processes and outcomes of the faculty member who designed the course andthe one who adapted it. We then explore the nature of the problems experienced dur-ing the adaptation process. We conclude with recommendations for successfully diffusinginnovations: adapting faculty members should choose innovations that genuinely interestthem and are aligned with their goals, should experiment with innovations in a gradualway, and should receive support throughout the process.

KEY WORDS: dissemination; chemistry reform; faculty development.

In recent years many professors and faculty developers have been in-volved in promoting the use of active and cooperative learning methods.They are motivated by both personal experience with the innovationsand a growing body of literature indicating that small group (Brown,Collins, & Duguid, 1989; Bruer, 1954; Chickering & Gamson, 1987; Fox,1998; Lave & Wenger, 1991; Rogoff, 1990; Springer, 1998; Springer,Stanne, & Donnovan, 1997) and other active learning strategies areeffective at fostering knowledge acquisition and use (Bruffee, 1992;Kurfiss & Boice, 1990; Vygotsky, 1978; Weimer, 1990; Wertsch, 1991).Simultaneously, faculty development programs are gaining acceptancein colleges and universities across the country (Gaff & Simpson, 1994).

Debra L. Penberthy, B.A., Florida State University (1990), is an educational evalua-tor at the Office of Educational Assessment at the University of Washington. Priorto this she served as an evaluator at the University of Wisconsin’s Learning throughEvaluation, Adaptation and Dissemination (LEAD) Center. Her research interests in-clude faculty development, K-16 systemic reform, and alternative models for evalu-ation. She is currently a graduate student at the University of Washington. SusanMillar, Ph.D. Cornell University (1981), a cultural anthropologist, currently directs theUW-Madison (LEAD) Center (http://www.engr.wisc.edu/∼lead). She has expertise in eval-uation of education reform and change processes in higher education, with a focus on thephysical sciences. She also is the “lead fellow” for the Institute on Learning Technol-ogy, a project of the National Institute for Science Education College Level One team.(http://www.wcer.wisc.edu/nise/cl1).

251 C© 2002 Human Sciences Press, Inc.

P1: LHE/hgi P2: GKZ


252 INNOVATIVE HIGHER EDUCATION

Regardless of the type of reform, faculty and faculty developers areincreasingly interested in effective ways to promote the adaptation ofinnovations and to support colleagues who are interested in trying newmethods. This article seeks to improve existing processes of faculty de-velopment by presenting the experiences of (1) a full professor of chem-istry at a major research university who had strong personal evidenceand evaluation data that active learning methods helped him achievehis student learning goals in an analytical chemistry course for ad-vanced first-year students, and (2) his departmental colleague (anotherfull professor) whom he enlisted to use these active learning methodsto teach the same course.

We begin with an exploration of the change processes and outcomesfor the faculty member who designed the course (referred to by thepseudonym “Ted”) and the one who adapted the course (“Peter”). Wethen explore the nature of the problems Peter experienced during theadaptation process. We conclude with lessons intended for faculty“change agents,” that is, faculty who are interested in promoting changein educational practice among their colleagues and for professionalfaculty developers.

In short, this article argues that successful dissemination is unlikelyif a faculty change agent or professional developer tries to modify acolleague’s practice by promoting his or her own approach and sim-ply telling a potential adapter how to implement the changes, leavingthe colleague to reproduce the strategies on his or her own. To pro-mote the use of innovative teaching strategies among more faculty, itis essential to foster a situation in which adapting faculty memberschoose to adapt innovations based on their own interest and excite-ment, select innovations to address student learning problems thatimmediately concern them, experiment with innovations in a gradualway, and receive support throughout the process. Implementation ofthese four guidelines should enable adapting professors to proceed inthe methodical, experimental fashion needed to acquire the necessaryskill with, and faith in, new pedagogical methods, their students, andthemselves.

The Study

In 1995, the National Science Foundation’s Division of Undergradu-ate Education funded a project entitled “New Traditions: Revitalizing

P1: LHE/hgi P2: GKZ


Lessons on Disseminating Innovation 253

the Curriculum” initiative.1 New Traditions (NT) is a coalition of chem-istry faculty from various research and undergraduate institutions inthe Midwest who are engaging in and studying curricular innovationbased on the premise that students learn chemistry in more meaningfuland effective ways when instructors use active learning techniques andthe curriculum is context-rich (New Traditions Project Leaders, 1998).The primary goals of the NT group are, first, to determine whether ornot their methods result in more effective learning and, second, to fos-ter adaptation of proven reforms among faculty around the country. Inservice of the first goal, the authors, third-party evaluators contractedby the NT group, evaluated the reforms using qualitative and quantita-tive methods. As a first step in achieving the second goal, the NT grouphas undertaken experiments involving the adoption and adaptation ofNT reforms by other members of the coalition. One such experiment isthe focus of this article.

The information presented here is based on a two-phase researchstudy. In the first phase, we conducted an in-depth investigation ofTed’s course, which incorporated small-group and other active learn-ing strategies. In addition to documenting Ted’s process of designingthe course and his goals and strategies, we investigated the students’learning processes and outcomes and the experiences of the gradu-ate teaching assistants for the course. We found that the course washighly effective at fostering Ted’s learning goals (Millar, S.B., Pasch,J., Penberthy, D.L., & Kosciuk, S.A., 1995; Wright, J.C., Millar, S.B.,Kosciuk, S.A., Penberthy, D.L., Williams, P.H., & Wambold, B.E., 1998).

During this first phase study we also learned that the NT faculty atTed’s institution, despite efforts to engage other science faculty col-leagues on campus in their efforts to reform the teaching of lowerdivision courses, largely worked in isolation. This lack of collegial in-teraction about teaching is typical at major research institutions.

Perhaps somewhat less typical is the paucity of campus resources de-voted to faculty development. While teaching improvement resourceshad been available for several years from the information technologydivision, the only formal faculty development resource offered to allinstructors on campus had just been made available. However, partici-pation was limited to faculty who made a commitment to participate in

1“New Traditions” was headquartered at the University of Wisconsin-Madison andfunded from 1995–2000 to develop, evaluate, and then disseminate ways to improvestudent learning in introductory chemistry courses. Dissemination activity is continu-ing from 2000–2003 under an additional NSF grant.

P1: LHE/hgi P2: GKZ



a two-hour meeting each week. In addition, essentially no formal fac-ulty development resources were provided by the college in which thechemistry department is located. It was not a surprise, therefore, thatit did not occur to the NT faculty to seek support for their educationreform efforts from faculty developers on campus.

Some non-“NT” members of Ted’s department, although interestedin the conclusions of our evaluation, questioned whether the studentoutcomes were more due to Ted’s personality and his innate teachingabilities than to his inventive teaching methods. These faculty mem-bers thought it unlikely that others could have the same success as Ted,simply through changing their teaching approaches. Ted was confidentthat this was not the case and wanted to demonstrate that anotherfaculty member could indeed do what he had done. Ted and other mem-bers of the NT coalition hoped to show that others could obtain the samechanges in student learning and skills and to document the process bywhich other faculty learned to use the new strategies. To this end, Teddecided to “hand-off” the course to another faculty member who mightbe willing to teach it repeatedly. He acted on his decision by enlistingPeter to implement the course in the spring of 1996. Peter’s agreementto take over the course enabled the second phase of the authors’ re-search, a study of the course adapter’s experience in implementing Ted’scourse. This phase is called the faculty change processes study to dis-tinguish it from the original course evaluation. We had three primaryresearch questions:

• To what degree did the “hand-off” of this course work?• What factors affected the success or failure of the hand-off?• How, if at all, did Ted’s change process differ from Peter’s? Based

upon the answers to the above, what lessons can be offered to in-form other efforts involving the promotion or adaption of innova-tive pedagogical techniques by higher education faculty?

For the faculty change processes study our primary data collectionmethod was structured, open-ended interviews with the instructors.(Some of these interviews were originally conducted as part of the eval-uation of Ted’s course.) The instructors included the course designer(Ted) and the course adapter (Peter), Peter’s graduate student teachingassistants (TAs), and Ted’s TAs.

Ted was interviewed by one of the authors twice in 1995, as part ofthe original study of his course. The other author interviewed Peterfive times over the period just prior to the start of the 1996 springsemester through the final examination week. These interviews were

P1: LHE/hgi P2: GKZ



conducted at two- to three-week intervals in order to obtain “real-time”reactions and reflections from the professor about his experience. Allsix of Peter’s TAs were interviewed prior to the middle of the semester,and three were interviewed again at the end of the semester. All six ofTed’s TAs were interviewed in the middle and at the end of the semesterin which they taught the course. In addition, Ted and Peter were inter-viewed together immediately after the semester ended. Peter also wasinterviewed once in the spring of 1998, 21 months after he taught thecourse, and again in spring 2000 to ascertain the longer term effectsof his spring 1996 course adaptation experience. In addition to inter-views, we conducted observations of both instructors’ labs and lecturesand surveyed Peter’s students to provide a context for interview analy-sis. (For the original study of Ted’s course we conducted many in-depthinterviews and multiple surveys with students.) Finally, both professorscarefully reviewed, commented on, and agreed with the researchers onthe interpretation of the data presented here.

The Case of the Course Designer (Ted)

Motivations for Change, Course Goals, and Strategies

Since 1973, Ted had been teaching a large lecture (100 students),introductory analytical chemistry course intended for advanced, first-year science, math, and engineering majors. The pre-requisite for thecourse was an accelerated one-semester course in general chemistry.The course involved the use of teaching assistants who led discussionsections (one hour per week) and laboratories (two four-hour sessionseach week) with groups of approximately 20 students. Teaching assis-tants for this and all other large introductory chemistry courses aregenerally early graduate students who have yet to receive a researchappointment. They are not required to have a demonstrated interest inteaching; and, at the time of this study, the TA training consisted of oneweek with very little time spent on pedagogical issues.

Over the years, Ted had noticed that even the students who performedat the top of the class on exams were unable to apply the material. Hebegan to wonder how well the other students comprehended it. Wishingto improve these outcomes, he began to talk with his students, primar-ily in the laboratory, to get their input on why they were understandingless than he had hoped and how he might address this problem. Concur-rently, he was discussing small group learning with a senior colleague

P1: LHE/hgi P2: GKZ



who had moved to a liberal arts college and was experienced with thatapproach. Of note, he did not work with other faculty or with facultydevelopers on campus during this period of course development. We be-lieve this was both a function of being in an individualistic academicenvironment and also of teaching a specialized course that serves afairly small number of introductory students.

Ted believed that it was possible to improve students’ critical think-ing and comprehension. Starting in 1991, he approached his searchfor the necessary teaching and learning strategies with the scientificmethod in mind, stating in an interview with one of the authors thateducational reform is “an experimental science just like any other re-search . . . You try lots of different things . . . and it’s important to findout both whether they’re working or not and what you need to do . . . tomake it work better.” He began to understand that his primary goal wasto use the course as “a vehicle to train students how to do science andthink like scientists.” For him, acquisition of content knowledge anddevelopment of lab skills were secondary goals. His main concern wasthat the students truly engage in the discovery process he designed forthem, for he believed that they would develop the requisite knowledgeand skills if they did.

Thus it was that, over a period of several years, and working pri-marily on his own, Ted tried one or more innovations each semester.For example, one year he introduced the use of cooperative exams, an-other year he required students to read and analyze research papersin small groups, and then the next year asked students to work insmall groups to tackle difficult open-ended lab problems. Although hefound that each innovation addressed the student learning problem tosome extent, he was still unsatisfied with the outcomes. He decided thatthe best approach would be to use several innovations simultaneouslyand strategically to foster synergism among the innovations. He taughtthis integrated version of the course, which we call structured-active-learning (SAL), for the first time in 1991. From 1991–1995 he made sub-tle changes to the SAL course, based on what he learned each semesterabout what worked best. By 1995, the essential course elements were:

• TAs teaching lab and discussion sections using a “guide-on-the-side” approach,

• small group, open-ended laboratory projects with the gradingbased on written reports and oral exams,

• small groups of students reading and analyzing research papers,• interactive techniques incorporated into a more traditional lecture,

P1: LHE/hgi P2: GKZ



• an absolute grading scale to encourage cooperation,• cooperative exams as well as traditional exams,• spreadsheet programs for homework and laboratory problems,• a volunteer student board of directors that advised the professor

on the course,• and integration of all course activities and assessments.

Ted strongly emphasized the role of the course reward structure—that is, the types of assignments and tests, and the basis upon whichthey were graded—in fostering the course goal of promoting an ability to“think like a scientist.” He stated,

The essential thing that I’m after is giving students opportunities to cre-atively explore projects that have defined goals, and in the process theyimprove their skills. It’s then really neat to talk to them [in the oral ex-ams] . . . If you have given them something for a project that does nothave an answer . . . [or] a “right way,” . . . and they’re convinced that thereisn’t a right way, then there is opportunity for them to be creative. And ifthey can get into that project so that they enjoy what they’re doing, theyconsider it significant and worth their time, [and] they get ownership ofwhat’s going on, then the emphasis for them is not on the grade thatthey’re going to get, but on the pride that they take in doing it, as well asthe experience of working together.

He emphasized that his reward structure was designed to promote cre-ativity and discourage memorization, and interview statements madeit clear that his reward structure values process over specific learningoutcomes. In fact, Ted often discussed the fact that the oral exams on theopen-ended lab projects were as much an opportunity for the instruc-tor to learn about the students’ comprehension levels as for studentsto develop deeper understanding of what they had done and how tointerpret it.

Ted stated that the philosophy informing his structured-active-learning (SAL) method is that the great majority of students will suc-ceed in taking responsibility for their own learning and become engagedin a scientific discovery process if an instructor provides them withenough support. The SAL method is designed to challenge students toachieve difficult and sophisticated goals while providing them with thenecessary resources, motivation, and support structures.

Outcomes

Ted felt that his SAL course was highly successful at helping himachieve his goals. In addition, an in-depth study of Ted’s course that

P1: LHE/hgi P2: GKZ



explored the nature of students’ learning processes and assessed stu-dent performance outcomes confirmed his perceptions of student out-comes (Millar, S.B., Pasch, J., Penberthy, D.L., & Kosciuk, S.A., 1995;Wright, J.C., Millar, S.B., Kosciuk, S.A., Penberthy, D.L., Williams, P.H.,& Wambold, B.E., 1998). The course was highly effective both in termsof creating a “learner-centered” environment and in fostering criticalthinking skills and understanding of the course material. Based onthese conclusions, and motivated by the NT group’s goal to test whethermethods proven to work for one professor could be “handed off” to an-other, Ted felt that the next step was to demonstrate that others couldachieve the same results. Thus, he engaged a colleague to teach thiscourse using the SAL methods.

The Case of the Course Adapter (Peter)

Motivations for Change, Course Goals, and Strategies

Peter, a long-time colleague of Ted, had been teaching upper-divisionundergraduate and graduate courses for over 20 years. His primaryundergraduate course was a medium-sized (50–100 students) sopho-more/junior level chemistry course that included lab sections taught byTAs. Although he used a fairly standard lecture for this course, he in-corporated active and small-group learning in the laboratories: he hadstudents work in small groups on open-ended, real-world projects insubject areas specified by him, within which students chose a specificarea.

Peter emphasized that although the lab sections were taught by TAs,he was present at all times during them. He worked closely with thestudents, providing them ongoing guidance through informal conver-sations. He believed that the best model for education was the OxfordUniversity model of one-on-one interactions. He was highly invested inteaching and had some of the same criticisms of the existing paradigmin chemistry education as did Ted.

[Ted] and I both think that the teaching is really important. I’ve felt fora long time that undergraduate teaching just does not accomplish propergoals a lot of times. I don’t think that an undergraduate is any differentfrom a graduate student except in terms of the maturity, the total amountof material they’ve seen. When they go through a course, I would like thestudents to be able to deal with the subject matter in a way that’s verysimilar to a graduate student’s approach . . . But most of our studentscan’t. . . .

P1: LHE/hgi P2: GKZ



Because Ted and Peter were friends and seemed to share the sameviews on undergraduate education, Ted asked Peter to teach the SALanalytical chemistry course for first-year students. Peter agreed, butfor several reasons he was somewhat unsure of his chances for successin the course. First, he had never taught first-year students. Second, hewas uncomfortable with allowing the TAs to play a significant role inadvising the students on the open-ended group work. In part, he wasconcerned because only one of his six TAs had experience using SALmethods. Also, he felt that some of the TAs’ lab skills and knowledgeof the subject were insufficient. Third, he was concerned that studentswould develop misconceptions about course content and achieve lowerlevels of skill in experimentation and rigor in the lab if they received toolittle guidance. While convinced that the course fostered creativity andthe ability to cope with new situations, he was concerned that studentsmight emerge from the course with weaker content knowledge and lab-oratory skills and that this might hinder their success in upper-divisioncourses requiring chemistry.

Peter’s reservations about the SAL methods were implicit in how hedescribed his goals for the course:

[As a result of this course] I would like to see students be competent interms of solution chemistry, chemical equilibrium problems, . . . becausethat’s the background they should get from this course that carries on tothe next. Since most of them are going into some type of biology or themedical field, it’s really important that they can deal with those. By thesame token, I think it’s very important that they get certain lab skills thatother departments expect them to have. And that’s one thing that I think[Ted] did not emphasize perhaps as much as probably is necessary. Butit’s a trade off. If we’re going to do these lab projects, it’s a kind of balancethat we have to work out. So, it depends on the kinds of problems that wetry to solve. I think those are the main [goals]. In addition, if we’re reallygoing to be able to deal with any of this correctly, [the students] have tobecome problem solvers in a fairly sophisticated way.

Peter felt that students could develop their problem-solving abilitiesif given the chance to experience what it was like to think for them-selves through open-ended projects. However, because of the need tobuild students’ knowledge base for subsequent courses, he also felt itwas important to insure that the course exposed them to the correctideas, even if this circumscribed the students’ freedom in the discoveryprocess. The following interview excerpt illustrates this point.

Interviewer: It seems that you have a larger set of primary goals than[Ted] had. It seems that you place a greater emphasis on the lab skillsand on the applied understanding of the content.

P1: LHE/hgi P2: GKZ



Peter: I think both of them have to be there . . . because when you moveinto [higher level courses], those other departments really do depend onthese [lab skills] . . . and the content. [Ted’s] feeling is that most of the[students in the course] are not going to be . . . using [these lab skills andthis content] all that much. The ability to think critically . . . and [use the]scientific method is [his] main thing. And that’s probably true for maybehalf of this class, but I think there’s a good half of the class for which thatisn’t all of it.

Peter’s goals were evident in the way he defined success in the course.For the open-ended laboratory projects, it was very important to himthat the majority of the students arrive at one of many possible correctsolutions the majority of the time. Moreover, his primary goal was forstudents eventually to be able to solve problems efficiently, that is, with-out going through a lengthy trial-and-error process. This contrasted sig-nificantly with Ted’s primary goal of getting students to experience andunderstand the scientific process by confronting successes and failures.For Ted, success on the open-ended projects was less important thanmaking sure that students experienced the messiness of real science.

Once Peter began planning his course, he encountered another diffi-culty: he had little specific information from Ted on the details of how hehad taught the course. Due to the nature of the research environmentin which Ted, the course designer, had been working, Ted had spentlittle time documenting what he had done. So, although Peter had anunderstanding of the nature of the assignments and the course materi-als, he had few actual models from the original course. For example, hehad no detailed descriptions of the lab experiments because the labora-tory instructions stated only the goals for the experiments. Though hewas adopting a full set of integrated teaching strategies, in some wayshe felt that he had to reinvent the wheel. In his first interview, he ex-pressed his fear that he would fail to create successful course activitiesbased on Ted’s basic ideas, stating:

My main concern at this point in time is just the logistics of carrying allthis out and getting everything put together . . . I don’t know very muchabout what [Ted] did. [I have talked with him], but he doesn’t have verymuch documentation for many of the things that they did . . . The realquestions that I have are about the nitty gritty details, which are ab-solutely crucial in this kind of thing. In the sciences you must have thatnailed down very well. Otherwise the whole thing’s a failure.

Given these factors, Peter’s primary personal goal in this experi-ment was to see how the course as a whole and the individual inno-vations would work. He saw himself as a participant in an educational

P1: LHE/hgi P2: GKZ



experiment in which he was truly unsure of the outcome: “There wereseveral things within the course that were questions for me . . . MostlyI was just trying to get a feeling for what would happen. Because [Ted]talking about it is one thing, and really experiencing it is quite dif-ferent.” Understandably, Peter felt considerable stress from the begin-ning of the experiment, remarking that a primary goal for him was to“survive.”

Peter tried to “adopt” the SAL course, that is, to teach the courseusing all of the major course features that Ted had used. However,he found it necessary to “adapt” the course for the following reasons:1) he placed a greater emphasis on lab skills and content outcomesthan did Ted, 2) he lacked detailed information on the innovations,and 3) he wanted to give the TAs less responsibility for guiding thestudents.

First, Peter used a more traditional, non-interactive lecture formatthan Ted had. He felt that the students’ need for background materialfor the open-ended lab projects was too great to justify using lecturetime for significant student interaction. Second, he produced numeroussupplementary reading packets to provide students with informationthey might need during the experiments. Third, to foster acquisition oflab skills, he increased the number of “closed-ended” lab experiments.Fourth, in contrast to Ted, who was available for consultation for onlyparts of the lab periods, Peter maintained high levels of student contactduring labs. As in his upper-division course, he was available in a roomadjacent to the laboratories and spent the entire time fielding questionsfrom students. Although this practice was in line with his approach toother courses that used TAs, he felt it was even more important inthis situation, due to his concerns about the teaching assistants’ labskills and content knowledge. By contrast, Ted expected the TAs to bethe primary guide for the students’ lab experimentation process. Thisdifference in the two professors’ views of the TAs may be attributable totwo inter-related factors. First, it may be that Ted had TAs with greaterteaching skill and experience. Secondly, Peter and Ted seemed to holddiffering views on the level of responsibility that was appropriate to givea TA. This difference paralleled the differences in the way they viewedthe undergraduate students—with Ted giving more responsibility tothe TAs and students than Peter.

Another contrast was in the reward structure. Whereas Ted’s ap-proach clearly rewarded students for demonstrating that they had en-gaged in the scientific process and had learned about the process andthe content through doing so, Peter’s approach placed greater emphasis

P1: LHE/hgi P2: GKZ



on students showing that they had achieved a working solution to thelab experiments. Thus, while Ted used oral exams in the labs primarilyto explore the process that students had developed to solve an open-ended problem, Peter used them to assess their learning process and todetermine whether they had achieved a satisfactory result.

In addition, Ted and Peter differed in the way that they respondedto students who seemed to resist the active learning approach. Ted ex-plained that he and his TAs consistently discouraged students fromseeking spoon-fed answers by pushing them to think creatively andnot grading them primarily on whether or not they obtained a work-ing answer. By contrast, Peter, strongly believing that students shouldarrive at a working answer, would eventually help a student who re-sisted the effort of thinking independently by suggesting a specific pos-sible solution.

Outcomes

Peter indicated that his level of uncertainty about how well thingswould go never subsided during the experiment and that he constantlyfelt overwhelmed by the magnitude of the changes he was making. Hefrequently emphasized that the context—first-year students and someof the course content—was completely new to him and that he, the TAs,and the students were inexperienced with many of the active learningstrategies used in the course. He explained that, as a result of theirlack of experience with the new approach, they all spent much moretime than expected and generally found the course stressful. Moreover,Peter’s research program was preempted for at least one semester, dueto the time demands of this course.

In contrast to the student feedback we gathered for Ted’s course, wehave very limited information on the students’ point of view or studentlearning outcomes for Peter’s course. We did not conduct an in-depthstudy on the experiences of the students in Peter’s course for threereasons: 1) we wanted to give the course adopter a chance to experimentwith and become comfortable with the teaching innovations prior todelving into the students’ learning experiences and outcomes; 2) wehad limited resources; and 3) we were most interested in exploringthe faculty change process. We do know that the student responsesto a written survey on their beliefs about the value of the course andthe teaching methods were bimodal. A sizable group of the studentsreported positive learning experiences; for others, the experience wasnegative and left them feeling frustrated.

P1: LHE/hgi P2: GKZ



At the end of the semester, Peter indicated that he was not willingto teach the analytical chemistry course again in the same way. Bothhe and the TAs conveyed that their experience had been difficult andthat they were unsure the extra time and energy were commensuratewith what they felt the students had gained. Thus it appears that thehand-off was not successful.

However, we learned from Peter in spring 2000 that he has come tofeel more positive about his experience with the course and its impacton the students. He explained that he had decided to teach the courseagain because he now had a much better understanding of what Ted wasattempting and of what he himself wants to achieve and how. Lengthyconversations with Ted had left him with a better understanding ofwhat Ted expected of the students and of what Ted actually did in thecourse. Peter stated that he lacked much of this knowledge in spring1996, and was thus forced to operate on the basis of many assumptionsthat, in fact, were not accurate. He explained that another factor in hisdecision to teach this course again is the time he had had to reflect onand understand his experiences in 1996 and to plan out his own SALversion of the course.

He proceeded to teach the course again in spring 2001, adapting someof the SAL methods to suit his own goals and style. In response to ourinquiry about this second experience with the course, he wrote to usthat, “It was the best class I’ve ever seen, and it ended up with anastonishingly high class average–no curve was used. These are verybright young people!”

Conclusions

One of the primary factors in the failure of the hand-off was thatPeter lacked personal motivation to teach the SAL course. He indicatedthat his main reason for taking on this challenge was to help Ted testhis and the NT group’s premise that the course could be adapted. Peterknew that it was important that Ted find someone to engage in thiseducational experiment, and he considered this a personal favor to hislong-time colleague.

In addition, at Ted’s suggestion, Peter agreed to recreate fully theoriginal SAL course. Peter did not seek out individual course compo-nents that interested him and then conduct his own experiment withthem. Had this been the case, it is more likely that his experimentwould have been successful. By contrast, Ted’s growing interest in theuse of active learning had led over the years to a passionate exploration

P1: LHE/hgi P2: GKZ



of these methods. During interviews with Ted, it was clear that he hada great deal of fun trying out these new teaching methods. We believethis is an important reason for his success with them. The literatureabout faculty culture makes it clear that personal motivation to changeis a critical factor in successful reform (Candy & Borthwick, 1994;Scott & Weeks, 1996).

Another major factor was that Peter was on unfamiliar ground andundertook too much change at once. For example, he had not previouslytaught some of the course material or first-year students. Moreover,except for open-ended lab projects, he had no prior experience with thespecific active-learning teaching strategies.

Still another important determinant in the outcome of this experi-ment was that Peter had little support from Ted, as there was scantdocumentation of the details of implementation. This lack of supportand documentation arose in part due to the circumstances under whichTed created the course. While articulating his new goals for studentlearning and designing and testing a set of new course elements tohelp achieve these goals, Ted worked for the most part in isolation. Inthis regard, his behavior was entirely consistent with the highly indi-vidualistic culture of his institution and department. This culture wasexpressed, in part, by the absence of either a central or college facultydevelopment office to which he might turn for ideas and advice. In ad-dition, Ted had no release time to develop the course or to mentor hiscolleague in adapting some of his methods. Thus, when it came time forPeter to adapt the course, he was, in large part, on his own.

Finally, Peter’s efforts were less successful than he had hoped be-cause there were important differences between his and Ted’s learninggoals. As it may not be clear why this might cause the problems thatPeter encountered, we will explain in greater detail. This conclusionis supported by related findings from Lutterodt (1980), who studiedfactors relevant to successful adaptation of science education curricu-lum materials. Lutterodt explains that, when the goals of the designerand potential adapter are not in concert, adaptation requires funda-mental changes that “affect the essential coherence of a curriculumso that . . . adaptation is no longer the most appropriate developmentstrategy” (p. 132). In the case studies at hand, although both profes-sors (Ted and Peter) sought to foster critical thinking skills, Peter wasparticularly concerned with insuring that the students understood thecourse material and developed the necessary laboratory skills. This dif-ference was crucial because promoting mastery of course material wasonly a secondary goal of Ted’s course strategies: he designed the course

P1: LHE/hgi P2: GKZ



activities (including the grading structure) primarily to force studentsto engage in a discovery process by which their critical thinking skillswould be further developed—even if that process resulted in some mis-conceptions. When Peter used Ted’s strategies, he unintentionally dis-rupted the essential coherence of the original course.

Three examples will suffice to illustrate the problems arising fromthe differences in the goals of the two professors. One example is Peter’sdecision to produce handouts and spend eight hours each week inter-acting with the students during lab time in order to ensure that theylearned the material correctly. This resulted in an excessive demand onhis time. Another example is that, while requiring approximately thesame number of assignments geared toward promoting critical thinkingas Ted did, Peter added a number of “canned” labs designed to teachskills. According to TA feedback, the students felt continually pressedfor time and overwhelmed by the amount of work they had to do and thebody of material they were expected to comprehend. Also, it is likely thatthe students had insufficient uninterrupted time to focus on the open-ended projects. A third example is that Peter unintentionally confusedthe students about his expectations of them. According to some of theTAs, the students interpreted some of his actions to mean that a goodgrade depended more on achieving a working solution, although he toldthe students that their goal was to work creatively and independentlyon the open-ended projects. In part, this situation arose because Peterfelt it necessary to act as the primary resource for some lab sections,having a stronger need than Ted to control the learning environmentand having been assigned a group of TAs that he felt were not very wellprepared. In this role, he apparently led some students to believe thathe had a pre-determined “right answer.” Once this idea began circu-lating through the student grapevine, the open-ended character of theproblems—essential to their effectiveness in developing independentcritical thinking—was subverted.

In the end, the difference between the two professors’ goals and theresulting changes Peter made may have resulted in the latter feelingthat his efforts were not successful. He felt he achieved his goals for toofew students to justify all the hard work of everyone involved.

Lessons on Disseminating Innovation

The contrasts between the change processes experienced by Ted andPeter provide a basis for lessons that might guide faculty and faculty

P1: LHE/hgi P2: GKZ



developers seeking to promote and support faculty innovation. Belowwe present four such lessons.

Change Should be Motivated by Personal Interestin Particular Innovations

We recommend that faculty undertake innovations only as they aremotivated by personal interest. This suggestion is based not only onour study, but on the literature on best practice in faculty development(Apps, 1985; Candy & Borthwick, 1994; Hutchinson & Huberman, 1993;Louckes-Horseley & Stiegelbauer, 1991; Scott & Weeks, 1996).

As Ted observed the way his students were learning prior to changinghis course and began discussing active learning methods with a moreexperienced colleague, he developed a growing interest in experiment-ing with new pedagogical approaches. In particular, he was drawn tothe use of groupwork; open-ended lab experiments; take-home, coopera-tive exams; and reading research papers. When he began to incorporatesome of these teaching strategies into the SAL course, he felt excitedand believed that he would have success with the methods. As it turnsout, he found some strategies more effective than others and modifiedhis course accordingly over a period of years. But on the whole, his pre-diction that he would improve student learning through structuring thelearning environment in a new way was borne out in reality. In partic-ular, he found the students to be more conversant with the materialand more willing and able to engage in active discovery than he hadpreviously imagined.

Peter approached the SAL course with an entirely different attitude.As stated previously, he became involved in the course as a favor tothe course developer. He did not choose the innovations he used in thecourse out of his own enthusiasm for the methods. For example, al-though he believed in and had experience with open-ended lab projects,he expressed no particular interest in the use of cooperative exams orin having students read research papers. He approached those ideasand other course aspects with skepticism, and this was apparent tohis teaching assistants and students. In a situation where all involvedare experimenting with new methods, it was essential that the leader,the course professor, believe his or her efforts would result in impor-tant payoffs for the students. In this case, Peter had little such faith.This, according to some of the TAs, created doubt among both TAs andstudents, which in turn undermined the efforts of the instructors andweakened the students’ confidence in the instructors. Based on research

P1: LHE/hgi P2: GKZ



cited by Angelo (1993), we speculate that these doubts about the coursemethods weakened students’ willingness to assume the higher level ofresponsibility for their own learning that is essential to the effective-ness of Ted’s SAL approach.

It is of interest that, as noted above, Peter eventually taught thecourse again, but only in his own way. In particular, he emphasizedthe small-group, open-ended laboratory projects and scaled back otheractive learning activities. These modifications are consistent with hisinitial views about the high value of exploratory laboratory experiencesand his skepticism about the feasibility and effectiveness of some of theother innovations.

Innovations Should be Matched to the Instructor’s Goalsfor Student Learning

Consistent with the relevant literature (Angelo, 1993), our study sug-gests that it is critical that the strategies being adapted are matched tothe adapting professor’s student learning goals. Ted felt frustrated thathis students were not developing the kinds of scientific thinking skillsthat he wanted them to have. This frustration led him to change hiscourse through the use of teaching innovations that not only attractedhim but that, on the basis of discussions with students and an experi-enced colleague, he believed would help him achieve his student learn-ing goals. Peter also was dissatisfied with undergraduate education andfelt it was necessary to provide students with more “real-world” scienceexperiences. However, he used the SAL innovations primarily becauseTed had presented them as integral to the course, without consideringbeforehand if they would help him achieve his own goals for studentlearning. As illustrated above, this led to difficulties for all involved.

Change Should be Undertaken in a Gradual Way

We recommend that faculty attempt only those innovations for whichthey feel ready, try only one or two innovations at a time, and workwithin a familiar context. The importance of this recommendation isillustrated by this study and supported by the literature on best prac-tice in educational innovation (Louckes-Horsley & Stiegelbauer, 1991;Kozma, 1985).

Ted was clearly ready for the changes he made. He not only imple-mented innovations in the context of a course he had taught for almost

P1: LHE/hgi P2: GKZ



two decades, but also experimented with several different innovationsprior to attempting a fully integrated SAL version of his course. Heundertook an ambitious, even risky, set of innovations by using a me-thodical, experimental process. Because he was deeply interested ineach innovation, he observed its effects carefully. By introducing onlyone or two new “variables” at a time, he was able to explore the effectsof each innovation. In addition, because he was working in a familiarcontext, with students for whom he had clearly articulated learninggoals, he had the confidence to eventually combine several of the inno-vations and see how they worked together. By contrast, Peter attemptedto implement several innovative strategies, for many of which he didnot feel ready, in a course he was teaching for the first time. Also, be-cause he was inexperienced with almost all of the strategies, he wasunable to understand how each one worked individually, let alone howthe various strategies worked in combination with one another. HadPeter attempted to incorporate his own variations of one or two of Ted’sinnovations in the upper-division course that he had taught for manyyears, we believe he would not have experienced such discomfort andwould have had greater success.

Innovation Needs to be Supported

Finally, we believe that support and training prior to and duringthe use of an innovation, possibly including release time to prepare forand implement the changes, may be critical for successful adaptationof reforms. This study and the literature about best practice in inno-vation, including Kozma’s 1985 piece, support this recommendation.For two reasons, Ted was not fully able to support Peter. First, duringthe development of his own course, he did not have time to reflect onhis practice and document the details of his strategies. Thus, when itcame time for him to help a colleague take over the course, he was lessaware of his own process and had little documentation to share. Havingto develop instructional materials while also learning how to use newteaching strategies created a highly stressful situation for Peter, whospent an inordinate amount of time creating materials and figuring outhow to implement the strategies. Second, Ted’s teaching and researchload did not permit him to provide Peter with the ongoing support heneeded during the experiment. For example, when Peter’s students be-gan floundering during the open-ended laboratory projects, his intuitiontold him to give students more and more help, to the point that some stu-dents felt they were given advice toward a right solution. Had the two

P1: LHE/hgi P2: GKZ



professors been in closer contact, Ted could have shared his experiencethat this desire to help is a natural tendency but one to be resisted inorder to maintain the integrity of the projects. Further, a faculty devel-oper, had one been available, might have alerted him to this problem.

In conclusion, we recommend the use of existing models of facultydevelopment that integrate the above lessons. One such program is“collaborative staff development,” where working groups of faculty whoare involved or interested in the same types of curricular or pedagog-ical changes support each other (Scott & Weeks, 1996). Another suchprogram is based on the idea of “the ally within.” In this program, fac-ulty change agents are selected by departmental nominations basedon their past teaching accomplishments and funded to support changefrom within their departments (Candy & Borthwick, 1994).

Based on what we know about faculty culture, we believe that thosewho are interested in promoting reform need to foster a process in whichfaculty (1) try innovations that interest them, (2) pursue methods thatare aligned with their goals, (3) undertake change in a gradual way,and (4) are supported by colleagues and resources. Finally, we suggestthat “adoption,” a term that implies taking a fully formed innovationand simply plugging it into an existing course or program, is not viable.In so saying, we reaffirm a principle articulated by Angelo (1993): “Theoperating maxim is: Adapt, don’t adopt” (p. 4).

Acknowledgments

This work was sponsored by the National Science Foundation un-der grant DUE-9455928. We would like to express our gratitude to thecooperating faculty members and teaching assistants whose generousdonation of time made this study possible.

References

Angelo, T. A. (1993). A “teacher’s dozen”: Fourteen general, research-based principles forimproving higher learning in our classrooms. AAHE Bulletin, 45 (8), 3–7, 10.

Apps, B. K. (1985). Improving practice in continuing education. San Francisco: Jossey-Bass.

Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learn-ing. Educational Researcher, 18 (1), 32–42.

Bruer, J. T. (1954). Schools for thought: A science of learning in the classroom. Cambridge,MA: Massachusetts Institute of Technology Press.

Bruffee, K. A. (1992, September/October). Science in a postmodern world. Change, 24 (5),18–25.

P1: LHE/hgi P2: GKZ



Candy, P., & Borthwick, J. (1994). The ally within: An innovatory approach to networkingand staff development. Innovative Higher Education, 18, 189–204.

Chickering, A. W., & Gamson, Z. (1987). Seven principles for good practice in undergrad-uate education. AAHE Bulletin, 39 (7), 3–7.

Fox, M. A. (1998). Improving undergraduate science education – national initiatives, localimplications. Journal of College Science Teaching, 27, 373–375.

Gaff, J. G., & Simpson, R. D. (1994). Faculty development in the United States. InnovativeHigher Education, 18, 167–176.

Hutchison, J. H., & Huberman, M. (1993). Knowledge dissemination and use in sci-ence and mathematics education: A literature review. (NSF Report CB2649X-00-0)Arlingon, VA: National Science Foundation.

Kozma, R. B. (1985). A grounded theory of instructional innovation in higher education.Journal of Higher Education, 56, 300–319.

Kurfiss, J., & Boice, R. (1990). Current and desired faculty development practices amongPOD members. In L. Hilsen (Ed.). To Improve the Academy: Resources for student,faculty, & institutional development 9, (pp. 73–82). Stillwater, OK: New Forums.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation.Cambridge, UK: Cambridge University Press.

Louckes-Horsley, S., & Stiegelbauer, S. (1991). Using knowledge of change to guide staffdevelopment. In: A. Lieberman & L. Miller (Eds.). Staff development for educationin the ‘90s: New demands, new realities, new perspectives (pp. 15–36). New York:Teachers College Press.

Lutterodt, S. A. (1980). The adaptation of science curricula–an exploratory analysis ofsome relevant decisions. European Journal of Science Education, 2, 121–138.

Millar, S. B., Pasch, J. E., Penberthy, D. L., & Kosciuk, S. A. (1995). Formative feedback re-port #3, Chemistry 110, Spring 1995. Madison, WI: University of Wisconsin-Madison,LEAD Center.

New Traditions Project Leaders (1998). Aims and goals of the New Traditions Project:The third annual report to the national visiting committee. Madison, WI: Universityof Wisconsin-Madison Chemistry Department.

Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in social context.New York: Oxford University Press.

Scott, C. D., & Weeks, P. A. (1996). Collaborative staff development. Innovative HigherEducation, 21, 101–111.

Springer, L. (1998). Research on cooperative learning in college science, mathematics,engineering, and technology. Cooperative Learning and College Teaching, 8, 2–4.

Springer, L. S., Stanne, M. E., & Donnovan, S. S. (1997). Effects of small-group learningon undergraduates in science, mathematics, engineering, and technology: A meta-analysis. National Institute for Science Education Monograph No. 11. Madison, WI:University of Wisconsin-Madison.

Vygotsky, L. S. (1978). Mind in society: the development of higher psychological processes.M. Cole, V. John-Steiner, S. Scribner, and E. Souberman (Eds.) Cambridge, MA:Harvard University Press.

Weimer, M. (1990). Improving college teaching: Strategies for developing instructionaleffectiveness. San Francisco: Jossey-Bass.

Wertsch, J. V. (1991). Voices of the mind, a sociocultural approach to mediated action.Cambridge, MA: Harvard University Press.

Wright, J. C., Millar, S. B., Kosciuk, S. A., Penberthy, D. L., Williams, P. H., & Wampold,B. E. (1998). A novel strategy for assessing the effects of curriculum reform on studentcompetence. Journal of Chemistry Education, 75, 986–992.

PenberthyMillarDisseminationArticle

Documents

Transcript of PenberthyMillarDisseminationArticle