Composing a Further Life: The Age of Active Wisdom Research...

Hand to Hand Association of Children’s Museums

14 Connecting Visitors with

Research through Living Laboratory®

I

A S S O C I A T I O N O F C H I L D R E N ’ S M U S E U M S

1 Composing a Children’s

Museum Field: Research

Agendas and More

3 What Do We Need to Know: The Children’s

Museum Research Agenda Project

4 The Evolving Role of Research in

Museums

6 Contributing to What We Know

about Museum Visitors:

Participating in the Visitor Studies

Continuum

10 Using Research to Make Learning through

Play Visible

Composing a Children’s Museum Field: Research Agendas and MoreAl DeSena, PhD National Science Foundation

n science and in everyday life, time is rela-tive. The ten years that have elapsed since

While re-reading “Establishing a Re-search Agenda for the Children’s Museum Field,” my lead article to the winter 2004 is-sue of Hand to Hand, I was struck with how many of the messages I tried to convey then still resonate with me. In particular, I still am an advocate of thinking and working systemically and systematically to develop a research framework that contributes to a stronger knowledge base but one that is also integrated deliberately with our collective creativity and wisdom. I also quoted Mary Catherine Bateson throughout the piece, feeling compelled, with her thoughts as a guide, to help mold a more inclusive context that shapes and contains a research agenda in ways that are practical, realistic, and that matter dearly to us. Some of you may recall that she also gave a very well received key-

I like to think of men and women as artists of their own lives, working with what comes to hand through accident or talent to compose and

re-compose a pattern in time that expresses who they are and what they believe in—making meaning even as they are studying and working and

raising children, creating and re-creating themselves...lives are composed somewhat like works of art, partly planned and partly improvised…

Like the faces of wise and loving elders, lives so composed may be beautiful.

Mary Catherine Bateson

Composing a Further Life:

The Age of Active Wisdom

the two 2004/2005 issues of Hand to Hand about the potential benefits and imple-mentation of a children’s museum research agenda may seem long or short depending on how one thinks about the scope of the task and on one’s frame of reference. Ten years, one tenth of a century, one hundredth of a millennium…What degree of progress is reasonable to expect? How do we apply the carefully considered and vetted 2015 research agenda now to make a difference in the lives of children, their families, and communities? Are we taking the long view for the long haul? How long do we as insti-tutions and as a field expect to exist, change form in minor or major ways, make a dif-ference? Jo

anna

Mor

riss

ey

Volume 28 Number 3 Fall 2014Revving Up Research


ExEcutivE DirEctor Laura Huerta Migus

© 2015 Association of Children’s Museums. All rights reserved. Reproduction without permission prohibited.

call for authors Hand to Hand is written by volunteer authors including: museum practitioners from all levels, educators, researchers and other professionals. Visit www.ChildrensMuseums.org and search “Editorial Guidelines” to learn how to prepare an article for print.call for sponsors Hand to Hand sponsorship opportunities begin at $1,500. Contact ACM for details.

EDitor & DEsignEr Mary Maher 908 East High StreetCharlottesville, Virginia 22902 Phone (434) 295-7603 Fax (434) 295-5045 email [email protected]

association of chilDrEn’s MusEuMs 2711 Jefferson Davis Highway, Suite 600 Arlington, VA 22202 email [email protected] www.ChildrensMuseums.org

Hand to Hand, a journal for children’s museum profes-sionals and others interested in children, families and informal learning, is published on a quarterly basis by the Association of Children’s Museums (ACM). Subscription Information: ACM Members: one free copy per issue; U.S. Subscribers: $30; International Subscribers: $50. Opinions expressed in this journal are those of the authors and do not necessarily reflect the position of ACM.

BOARD MEMBERS Leslie BusharaChildren’s Museum of Manhattan (NY)

Susan Garrard Mississippi Children’s Museum (Jackson, MS)

Joe Hastings Explora! (Albuquerque, NM)

Al Najjar Clay Center for the Arts & Sciences of West Virginia (Charleston, WV)

Jeri Robinson Boston Children’s Museum (MA)

Michael ShanklinKidspace Children’s Museum (Pasadena, CA)

Charles Trautmann Sciencenter (Ithaca, NY)

Mark Thorne National Children’s Museum (National Harbor, MD)

PRESIDENT Marilee Jennings Children’s Discovery Museum of San Jose (CA)

VICE PRESIDENTS Jennifer Farrington Chicago Children’s Museum (IL)

Rhonda Kiest Stepping Stones Museum for Children (Norwalk, CT)

Mike Yankovich Children’s Museum of Denver (CO)

TREASURER Catherine Wilson Horne Discovery Place, Inc. (Charlotte, NC)

SECRETARY Tanya AndrewsChildren’s Museum of Tacoma (WA)

PAST PRESIDENT Jane WernerChildren’s Museum of Pittsburgh (PA)

BoarD of DirEctors

note presentation at the 2005 InterActiv-ity conference, which is documented in the summer 2005 issue of Hand to Hand—and very much worth revisiting.

And, thus, I applaud the Association of Children’s Museums (ACM), the Institute of Museum and Library Services (IMLS), consultants, and ev-eryone involved in the field for their con-siderable, thought-ful, and systematic efforts reflected in today’s “final prod-uct,” knowing that the research agenda document and the Phase 2 steps are still but milestones along a path that will con-tinue to germinate, enlighten, and morph over time. Also, as you can see from the introductory quote from Mary Catherine Bateson’s work, I am again drawing here upon the ever-flowing stream of inspiring ideas presented in Composing a Further Life as well as in her many other books.

In addition to the voluminous literature in sociology, social psychology, organization studies, etc. on the development of organi-zations, professions, and fields like children’s museums, one could derive meaningful in-sights into where we find ourselves today and where we may be heading by expand-ing Bateson’s metaphor about “composing a life” to encompass our full range of interests —to the children, families, and communi-ties we serve, to our staff and collaborators in organizations around the globe dedicated to improving lives, and, indeed, to our as-sociation, profession, and field.

I am certain that from Composing a Fur-ther Life you could derive many more appli-cations beyond her original intent than what I have space to emphasize here, where I have selected six ideas that I find to be especially salient.

Composing a Children’s Museum Field

I would propose that one could easily exchange Bateson’s “men and women” with a full set of stakeholders who are part of the system within which we think and act: visi-

tors, staff, museums as organizations, net-works of organizations, communities served, and professional associations. All are en-gaged in the lively interplay of pattern-mak-ing, meaning-making, studying, planning, and improvising, and, I would suggest, are guided by either explicit or implicit aesthet-

ic criteria—creating beauty. Even research and research agendas should aspire to be beautiful—there are the scientific founda-tions, but there is also the art of it all.

Another consid-eration about com-posing a field is the systems question of the scope of the re-search agenda, what is manageable, how to prioritize, and espe-cially, how to connect that work with other forms of knowledge-building. In his recent article, “Measuring Museum Impact and Performance,” mu-seum planner John

Jacobsen provides a useful perspective on this need where he suggests that we develop a theory of action that is attentive to public values of society as a whole; private values of businesses; personal values of individuals, families, and households; and our own insti-tutional values. For each of these entities he then provides examples of key benefits. For example, for public values, benefits include broadening participation, preserving heri-tage, strengthening social capital, enhancing public knowledge, serving education, ad-vancing social change, and communicating public identity and image.

Interdependence

Bateson makes a very strong case that American society is hyper-individualistic, when in fact human life—indeed, all of life and earth’s systems—are interdependent. “The reality of all life is interdependence. We need to compose our lives in such a way that we both give and receive, learning to do both with grace, seeing both as parts of a single pattern rather than antithetical al-ternatives.” (p. 7) One implication of this truth for me is that we should continue to

continues on page 12

My observation is that it is often much

easier for proposers to make a case

for institutional advancement, but

much harder for field-wide. Part of

the reason is that it has been

difficult to know and synthesize all

that is pertinent at the field-wide level

to then consider how to position one’s

initiative in that broader context.

Efforts such as this Children’s Museum

Research Agenda initiative serve both

to help focus the field on its shared

interests—a statement to the world

about significance and identity—and

to develop over time a knowledge

base that can better inform important

decision-making activities.


hildren’s museums contribute to the lives of children, families, and commu- What Do We Need

to Know? The Children’s Museum Research Agenda Project

Jessica J. Luke, PhDUniversity of Washington

and Victoria Garvin

Association of Children’s Museums

to participate in one of four webinars that provided an overview of the project and its key activities, a synopsis of the symposium process and proceedings, and a discussion of the categories for research. Following the webinars, the project team took the nine sets of questions and organized them into three themes. Below, we share highlights from each theme and its questions.

Research Agenda Themes

These questions investigate the unique and long-term impacts of children’s muse-ums, the learning environment itself, and institutional culture—from beliefs about learning to the role of research in help-ing children’s museums evolve and inno-vate. Research in these areas can be used to strengthen the mission of children’s muse-ums, identify best practices by demonstrat-ing how specific elements in the children’s museum environment support acquisition of specific skills, and gain increased support from funders and other stakeholders.

The Value and Impacts of Children’s Museums

• How are children’s museum impacts distinct from other influences on chil-dren?• What are the long-term benefits/impacts of children’s museum visits on children/families?

• What do community members (e.g., parents, teachers, children, stakehold-ers) perceive the benefit/value of chil-dren’s museums to be?

Learning Environments and Strategies

• What children’s museum activi-ties promote creativity and problem-solving, cognitive processes, executive function, etc.? And why is any of this important?• What factors (parent-child interac-tions, design, staff interactions) in-fluence learning (critical thinking, creativity, innovation) in children’s museums?• What is a “high-quality” children’s museum experience, and how do we measure it?

Children’s Museums as Learners

• How do beliefs about learning (from project participants, museum staff, parents, teachers, board members, stakeholders, and funders) impact how children’s museums support children’s learning and how we understand the role of children’s museums?• What is the role of research in a children’s museum? How can it help children’s museums to adapt, evolve, innovate?• How do we grow a culture of re-search in our museums, in our part-nerships, and in our communities?

Audience-focused research questions can tease apart the process of learning in chil-dren’s museums. This information can be used to improve practice to further expand museum learning, to strengthen the learning ecosystem, and to elevate the stature of chil-dren’s museums. This research may lead to the development of a shared language to talk about learning in children’s museums as well as the discovery of new types of learning.

continues on page 13

ACharacteristics of Children’s Museums

BAudience

Cnities in significant and meaningful ways. In a recent public opinion poll, 95 percent of parents strongly agreed/agreed that chil-dren’s museums are valuable learning insti-tutions. In the past four years, 30 percent of museums awarded the National Service Award—the highest U.S. honor conferred on museums and libraries that make a dif-ference for individuals, families, and com-munities—were children’s museums. While there is research ongoing to document the impacts of children’s museum, much re-mains unknown about how learning is sup-ported and facilitated in these institutions.

Funded by the Institute of Museum and Library Services, the Association of Chil-dren’s Museums (ACM) and the University of Washington’s Museology Graduate Pro-gram (UW) undertook the Learning Value of Children’s Museum Research Agenda Project in December 2012. The project goal: to generate a field-wide research agen-da for children’s museums that identifies and prioritizes the evidence most needed by the field to articulate and demonstrate the distinct learning impacts of children’s muse-ums. The full research agenda can be down-loaded from ACM’s website. This article provides an overview of the process used to build the agenda, and highlights some sig-nificant areas of research identified within it.

Building the Research Agenda

On September 10–11, 2013, more than 110 museum staff, academic and indepen-dent researchers and evaluators, and policy-makers worked collaboratively to identify the most pressing research questions that needed to be answered in the children’s mu-seum field. Inspired by calls to action from several prominent researchers, participants engaged first in small group discussions to generate questions, and then in large group discussions to clarify and prioritize the ques-tions. A list of nine sets of questions emerged to guide research and evaluation studies. Af-ter the symposium, ACM invited the field

The Association of Children’s Museums is pleased to circulate the Learning Value of Children’s Museums Research Agenda. While this document is

called a research agenda, if you prefer to think of it as roadmap or a framework for research, please do so. What it’s called is of less importance than

what it represents: the summary of a field-wide process to identify and prioritize what needs to be known about children’s museums, the learning that

takes place in them, and the impact they have on the lives of the children, families, and communities they serve. This research agenda also represents

a pivotal milestone in the children’s museum field as it marks the starting point for developing a collective, evidence-based body of knowledge that

can be used across institutions to improve practice, build theories of learning, and demonstrate the unique value of children’s museums.

—Introduction, Learning Value of Children’s Museums Research Agenda

3Hand to Hand Association of Children’s Museums

Hand to Hand Association of Children’s Museums4

oted writer, researcher, and museum education theorist George Hein began

Democracy (2012), Hein explores the work of philosopher and educational reformer John Dewey and its relevance for museums. Through the lens of Progressivism and its focus on improving people’s lives through socio-political and educational reform, Hein discusses the long history and current state of progressive education in museums.

Hein is currently Professor Emeritus in the Graduate School of Arts and Social Sci-ences at Lesley University in Cambridge, Massachusetts, and Senior Research Associ-ate at the Program Evaluation and Research Group at Endicott College, Beverly, Massa-chusetts.

—Ed.

What do we mean by “using research” in the museum?

The term “research” has been part of museum activity as long as there have been museums. Traditionally, research was carried out by curators to find out something about

the museum’s collection-—for example, the provenance of a painting, the background information for developing an exhibition, or the historical validity of an interpretation. But applying the term to studying visitors in a systematic way is a relatively recent phe-nomenon, roughly 120 years old.

With the advent of public museums, which dates back to the 18th century, mu-seum professionals have been curious about who their visitors were, what they did in mu-seums, and what the consequences of those visits were for them. Even though there were no children’s museums before the Brooklyn Children’s Museum opened in 1899, chil-dren visited museums, so speculation about what they might learn and whether the museums were appropriate for them goes back farther. For example, George Brown Goode, a champion of museums as educa-tional institutions, wrote in 1888, “I would not organize the museums primarily for the use of people in their larval or school-going stage of existence.” In contrast, H. H. Hig-gins, who published the first visitor study in 1884, argued that even casual visitors including “Little parties of children [who] sometimes found the table-cases convenient for racing around” should be taken seriously since they and their parents, mainly casual visitors, “had brought themselves for the time into contact with sources of improve-ment,” and that their large numbers “made them after all, the most interesting class of visitors.”

Is there a more compelling need for re-search now than in previous years, and if so, how does this change affect the direction of research in the field?

The need for visitor research is as old as the recognition that museums had a public educational function, but the recognition that visitor research is important derived originally from the awareness, sometime in the late 19th century, that social science as a discipline might be of use to society.

The systematic study of nature as pri-marily an experimental effort, rather than mainly an intellectual one, is associated with shifts in thinking in the 16th and 17th cen-turies; we think of “modern” science and modern scientific methods as activities that evolved over the past 400 years or so. The application of these general ways of thinking to social issues—human development, so-

Work to democratize education,

to improve the opportunities for all children,

and to provide rich learning experiences

cannot succeed without simultaneously

addressing other impediments

to achieving a just society.

—George Hein, Boston Stories

The Evolving Role of Research in MuseumsAn Interview with George Hein, PhD, Lesley University

Nhis professional life as a chemist. His sub-sequent immersion in the social activism of anti-war movement of the ’60s led him to question his involvement with biochemical research. At the same time, following his own children’s progress through a subur-ban Boston-area school system, he became more interested in improving education, specifically science education, and applied his academic training to new work on the Elementary Science Study Project at the Educational Development Center (EDC) in Newtown, Massachusetts. EDC’s original goal of improving science education in the United States contributed to a revolution in science teaching, evolving away from rigid, skills-based curricula to hands-on, materi-als-rich and theme-based units offering stu-dents open-ended exploration.

Joining the Lesley University faculty in 1975, Hein founded its first PhD pro-gram in educational studies. With colleague Brenda Engel, Hein cofounded the Program Evaluation and Research Group in 1976 to evaluate the educational work of twenty-five Boston area museums and arts organi-zations. Since that time, Hein has been a leader in museum education and evaluation, specializing in qualitative in-depth program evaluation and visitor research.

From 2006 to 2007, Hein was president of Technical Education Research Centers, a Cambridge, Massachusetts-based nonprofit educational research and development or-ganization focused on innovation in STEM learning. Over the years, Hein has been a Visiting Scholar at The J. Paul Getty Mu-seum; Visiting Faculty, Fu-Jen University (Taipei); Visiting Faculty in Museum Stud-ies at University of Technology, Sydney (Australia); Howard Hughes Medical Insti-tute Visiting Scholar, California Institute of Technology; Visiting Faculty Museum Stud-ies Program, Leicester University; Fulbright Research Fellow, King’s College, London; and a Research Associate, Museum of Sci-ence (Boston.)

In addition, Hein is the author of nu-merous books and articles, including Learn-ing in the Museum (1998), and, with co-author Mary Alexander, Museums: Places of Learning (1998). In his most recent book, Progressive Museum Practice: John Dewey and


cial conditions, public health, family life—came later, and has had enormous impact on society, for better or worse, starting in the late 19th Century. Child development, one of the great intellectual achievements of the 20th century, is an example of research that has provided enormous benefits to edu-cation, child rearing, and children’s health. Phrenology and Social Darwinism are ex-amples of social science concepts that were not particularly beneficial for society.

Progressivism, with its faith in the ratio-nal understanding of social issues, was ap-plied not only to topics such as public health and safety, but also to social organization. It validated studying how people develop and how they learn, which naturally led to re-search in informal settings. That broader research area has been explored by anthro-pologists, sociologists, and ethnographers for decades and has been extended in recent years to learning in museums.

But there is another major factor that has made visitor studies more significant today than in the past: the post-modern critique of a growing number of institutions and practices. Just about every form of human interaction—the family, marriage, schools, cultural institutions, forms of government and museums—have come under scrutiny as to their structures, values, and usefulness to society. Whether the critique comes from the political left—arguing that traditional forms of institutions are outdated, undemo-cratic or oppressive—or from the political right—arguing that traditional structures need to be maintained in their classic forms to preserve civilization—museums need to justify their existence more than ever. If we accept that museums are educational insti-tutions, then we need to demonstrate that they accomplish this mission and we need to provide evidence for however we define that responsibility.

Based on your long experience in the field, can you think of any earlier, perhaps over-looked avenues of research pertinent to chil-dren’s museums that should be revisited?

A major obstacle to researching children’s development and learning in museums as opposed to other settings is the difficulty of finding a stable group of subjects to study over a reasonable period of time. Learn-ing and development take time. Children’s mental and physical development have been studied extensively by observing and inter-acting with the children over long periods; learning (or any behavior) in schools has been studied by documenting the activities

of individual children or groups of children over semesters or even several years. But, museum visitors on the whole are ephem-eral, and it takes considerable effort just to identify and know enough about their back-ground and social situations to study them over a reasonable time as individuals. Mu-seum visits are both short and infrequent.

There are a few studies that have at-tempted to follow museum visitors over long periods of time (interviews a year af-ter a museum visit, for example) but noth-ing has been done with the consistency and breadth of scope to match, for example, epidemiological public health research (the Harvard Nurses Health Study has followed hundreds of thousands of nurses since the original cohort in 1976) or the classic child development studies that followed children over months and years. Most relevant to children’s museums may be the High-Scope Perry Preschool Study that followed children for many years during and after their Head Start experience and demonstrated that their early education exposure not only increased their IQ scores, but also improved their lives as adults in measurable ways.

In 1956, a London study of children visiting the Children’s Gallery at the (Lon-don) Science Museum reported that most children spent very little time at individual exhibits and “tend to flit from one thing to another, stopping to press buttons or turn handles, treating the Gallery more as an amusement arcade than as a source of sci-entific information.” But among the same children there were some “habitués” who had come multiple times and who went to “considerable trouble and expense” to come to the museum, even from “more distant parts of London by means of buses, tubes or both.” I wish the researchers had followed these children over time.

What methods and/or questions yield the richest and most actionable information?

We need longitudinal studies (over many years) of children who spend considerable time in museums. We also need more stud-ies that take into account the cultural and social context of children’s experiences in museums. Fortunately, the field has moved

away from strictly behaviorist research that measured only time spent overall and at specific exhibits and simple pre/post in-terviews or questionnaires (popular in the 1920s and 1930s) to more detailed analyses of children’s experiences with references to the social and cultural issues that influence museum experiences.

Are research and evaluation fundamen-tally different? If so, are they different enough to be viewed as separate functions or applied differently by museums?

The goals of evaluation and research in any field may be different, but the methods used are the same. They both apply a set of research tools (observations, interviews, ask-ing people to perform a specific action or analyzing the products of human activity). Evaluation, by definition, is the study of what a specific activity or program has (or has not) accomplished. Research is an effort to understand something about the world. In order to carry out experimental research (as distinct from theoretical research or li-brary research) the researcher(s) will most likely develop or follow some activity that happens in the real world. They may choose to study naturally occurring practices or specifically devise a situation to provide evi-dence relevant to the researcher’s questions. For example, in his pioneering behavior-ist research in the 1930s, Arthur Melton studied the effect of varying the number of paintings on the wall of a museum gallery on the time visitors spent in the gallery. He was not evaluating visitor response to a par-ticular exhibition, but trying to gain experi-entially based information on the effect of the number of paintings (they are not even identified in the research report) on visitor time in the gallery.

The term “evaluation” traditionally re-fers to the examination of a program or ex-hibition that has been initiated for some rea-son other than to expand general knowledge about how people behave and learn. The goal of a program may be to achieve some educational outcome, increase visitorship, or continue the museum’s service agenda. Evaluation then asks the research question, did the activity do what it intended and what other consequences were there? The purpose is to inform the activity’s sponsors of a program’s intended and unintended outcomes and to provide information to a program’s creators about how to improve future actions. An exhibition, for example, is not conceived to further our knowledge

continues on page 8

We need longitudinal studies (over

many years) of children who spend

considerable time in museums. We also

need more studies that take into

account the cultural and social context

of children’s experiences in museums.



f you visit The Children’s Museum of In-dianapolis during the summer months,

institutional knowledge of what works (and what doesn’t). The resulting reports con-tribute to a long-term record of evaluation findings, but the evaluation results are most powerful when shared with project teams and integrated into how they approach their work. Helping other staff members to ap-ply evaluation study findings and growing the evaluation capacity of non-evaluators on staff are some of the primary tasks of the research and evaluation department. While the museum’s size and resources may allow for a great volume of evaluation work to take place, the range and nature of visitor studies that are conducted fall along a continuum. Many of the study types on this continuum are feasible for small and medium-sized mu-seums as well.

The field of visitor studies is often iden-tified with evaluation but more accurately it is a continuum of “quick and dirty” evalua-tion at one end to basic research at the other. Research is the systematic study of a topic with the goal of producing new knowledge. Basic research can be thought of as building a field’s theories but not necessarily apply-ing the knowledge in real-world situations. Evaluation is a type of applied research—taking the results from a systematic study and putting them to work to solve practical problems. Museum staff members who are not professional evaluators conduct much of the evaluation along the visitor studies continuum; this is true at The Children’s Museum of Indianapolis even though there

are now two professional, full-time evalua-tors on staff. By walking though the types of evaluation on the visitor studies continuum, staff at even small museums may realize that they are participating in this field-wide movement.

One end of the visitor studies continu-um is anchored by the quick-turn around, improvement-oriented evaluation so many staff members do on a regular basis. At The Children’s Museum of Indianapolis this of-ten takes the form of prototyping, mechani-cal remediation of gallery elements, or it-erative improvements to regularly-delivered programs. This type of evaluation is initi-ated and led by exhibit developers and pro-gram staff. Often called “quick and dirty” evaluation, it uses a small number of inter-actions with visitors to identify an issue. For example, staff have taken rough prototypes of exhibit elements out on the floor to see if visitors understand how they work or if additional labels are needed. The staff mem-ber often draws upon her intuition of what works and her knowledge of best practices to identify the problem and develop solutions. While some may call this type of evalua-tion “informal,” this degrades its very real value and the intentionality with which it is conducted. Evaluation is nothing more than having a question, collecting data (through observation, conversations, surveys, etc.), finding a pattern in the data, and using that pattern to assess a situation. If a problem is approached with this mind-set of intention-ality, then it is evaluation and it does not need to be labeled as “informal” to excuse its rough-and-ready appearance.

Iyour curiosity might draw you to a tempo-rary gallery with the inviting name Try It Out! At its entrance, a staff member greets you and explains that your family is wel-come to participate in the activity inside and provide feedback. Intrigued by the array of collections objects laid out on the tables and by the idea that the museum’s staff wants to learn from you and your children, your fam-ily enters the space and is greeted by another staff member. This person introduces herself as one of the museum’s curators and guides your family through an activity in which you read creative object labels and rate the perceived importance of five objects. After spending about ten minutes completing the activity and answering some of the curator’s follow-up questions, your family is ready to go visit the dinosaurs, and you leave the space feeling excited that you got a glimpse behind the scenes of the museum. You prob-ably would never realize that you had just experienced one piece of the continuum of visitor studies that exists at The Children’s Museum of Indianapolis.

Many museums, children’s museum included, use visitor studies to learn more about their audiences and to expand the field’s knowledge of issues critical to muse-um work. The Children’s Museum of India-napolis has a longstanding tradition of con-ducting evaluation to improve projects and processes in part because there has been an evaluator on staff for the last twenty years. With a staff member dedicated to evalu-ation, the museum has been able to build

F r o M E vA l u At i o n to r E s E A r C H : t H E v i s i to r s t u d i E s C o n t i n u u M

From “quick and dirty” on-the-floor prototpying to longitudinal, university-affiliated studies, there is a considerable amount of overlap among the types of research conducted in museums. But there is value to all well-designed, well-managed evaluation and research. Much of what we know about our visitors and what works in informal learning settings is documented through evaluation as well as research. non-evaluators and evaluators alike can

participate in growing our understanding by conducting evaluation and research along the visitor studies continuum

t i M E + C o M P l E X i t Y

Small front-end studies or formative evaluations

Medium-sized front end studies and summative evaluations

Large-scale, long-term research projects and community impact studies

P r o F E s s i o n A l C A PA C i t Y r E Q u i r E d

Professional evaluators/researchers (in-house staff or outside consultants)

Experienced non-evaluator staff

Non-evaluator staff, some experience

c c

Contributing to What We Know about Museum Visitors: Participating in the Visitor Studies Continuum

Susan Foutz and Claire ThomaThe Children’s Museum

of Indianapolis

6


Moving along the continuum, anoth-er form of evaluation are relatively small studies that include more visitors than the quick-and-dirty variety and usually use a form or data collection “instrument” to re-cord the data. This can include front-end studies, such as title-testing or gathering families’ thoughts on a particular concept, or formative evaluations. Formative evalua-tion includes testing elements or programs that are not finalized and still have room for substantial changes. Prototype-testing is one type of formative evaluation. At the muse-um, these types of studies are also regularly conducted by non-evaluator staff members. The museum’s Try It Out! space is a prime example of this type of evaluation.

In 2014 a formative study conducted in Try It Out! was led by the museum’s mar-keting team. They wanted to know how to advertise a winter-themed exhibit, Jolly Days. Previous TV spots had used animat-ed characters in an animated winter land-scape. Marketing staff wondered if families who had never visited the museum before would prefer the animated commercial or a live-action commercial that showcased the actual exhibit. So they set up two laptops, and asked first-time visitors to watch the an-imated commercial and a rough commercial made from old footage that showed families playing in the real exhibit. After watching both commercials they asked families which commercial they preferred and why. The re-sults were clear to the team—if your goal is to attract new visitors to Jolly Days, the live-action commercial was more powerful since it showed exactly what there was to do in the exhibit.

An example of a front-end study from Try It Out! involved asking visitors to vote on which potential exhibit they would visit. Visitors were given descriptions on cards of a dozen exhibits and asked to sort them into three piles: would not visit, would visit if we were here but wouldn’t make a special trip for it, would make a special trip for it. The families were instructed to decide as a group how to categorize the exhibits. As they did this, the exhibit development team took notes on their conversations and how they sorted the cards. This allowed the staff to see which exhibits had intergenerational appeal and rule out some exhibit ideas.

Museum evaluation staff are available for (and actively encourage) consultation, but other museum staff are responsible for developing a question that can be answered through visitor interactions, designing the data collection form, collecting the data,

discovering the patterns in the data, and reporting it to their project teams. The key to making evaluation manageable for a non-evaluator staff member is limiting the study to just one evaluation question with immediate applicability. Again, this type of evaluation is common at museums of all sizes, and is suitable to be carried out by staff without extensive evaluation experience.

Keeping in mind that the boundaries on the continuum of visitor studies are fuzzy, the next few types of evaluations are usually undertaken by the the museum’s full-time evaluation staff or staff with more extensive evaluation experience. These include small-to-medium studies that may take multiple methods or data sources to fully address the evaluation question. For example, the exhibit developer for the Take Me There: China gallery was interested in exploring what attracted visitors to a case of objects representing diverse religious traditions in China and if, after seeing the objects and reading the labels, they had unanswered questions. She designed a two-phase study. From the first phase, observing visitor be-havior and analyzing that data, she found that the majority of visitors who stopped at the case were adults, and most of them had moved through the gallery in such a way that they had seen the case head-on. Visi-tors who approached from either side were much less likely to stop, and children very rarely stopped at all. In the next phase, she interviewed visitors who had stopped at the case, and asked them what drew them to the case, what objects they looked at, which la-bels they read, and whether there was any-

thing they still had questions about. With her existing experience in evaluation, this staff member was able to make sense of the information she collected primarily because she designed a study that was within her ca-pabilities to carry out. For staff who do not have the same level of experience, the mu-seum’s evaluators would lead the study with the assistance of key project staff. The col-laboration helps to grow the project staff ’s abilities to think evaluatively.

In another evaluation example, the ex-hibit team identified a series of questions about a temporary exhibit that would help them plan for future exhibits. Two of their primary questions were about how long visi-tors stop at the main elements in the gallery and whether they read the labels. Although the exhibit developers had the experience necessary to carry out the studies them-selves, the evaluation staff took the lead due to the amount of time required to collect the various types of data. In this instance, the evaluation staff relied on the help of trained interns and volunteers to collect and analyze the data. For small museums that do not have as much staff experience with visi-tor studies, this point on the continuum is where the assistance of a consultant or exter-nal evaluator may become necessary.

Farther along the visitor studies con-tinuum are summative evaluations of gal-leries, multi-year programs, or grant-funded projects. At The Children’s Museum, stud-ies of this size are designed and managed by the evaluation staff and typically have more evaluation questions, require a larger pool of respondents, and take more experi-ence to design and carry out. The museum conducts in-depth summative evaluations of all permanent galleries in the year follow-ing their opening. A recent example of this type of study is the summative evaluation of Playscape, a newly redesigned permanent gallery. The study included five overarching evaluation questions, and the methods in-cluded full gallery timing and tracking, exit interviews, longitudinal online surveys, and focus groups. The analysis and reporting of the quantity of data collected took expertise and time that most non-evaluators do not have. The museum typically does only one large-scale exhibit evaluation of this type per year. The data is used in a number of ways: to inform the board and the executive team of the degree to which the exhibit met its goals, as baseline information in case chang-es to the gallery are made in years to come, and most importantly, to inform the exhibit

continues on page 9

One end of the visitor studies continuum is

anchored by the quick-turn around, improve-

ment-oriented evaluation so many staff

members do on a regular basis....

While some may call this type of

evaluation “informal,” this degrades its very

real value and the intentionality with which it

is conducted. Evaluation is

nothing more than having a question,

collecting data (through observation,

conversations, surveys, etc.), finding a

pattern in the data, and using that pattern to

assess a situation. If a problem is approached

with this mind-set of intentionality, then it is

evaluation and it does not need to be

labeled as “informal” to excuse its

rough-and-ready appearance.



of visitors; it is intended to educate, amuse, increase attendance, mollify a donor, satisfy the whim of a director, or any number of other reasons. But funding agencies want to know what was accomplished due to their investment. The “pure” researchers, as dis-tinct from the evaluators, want to find out how people learn, or what conditions influ-ence their behavior, so their agenda may or may not coincide with the museum’s educa-tional or social agenda.

Evaluation is sometimes considered more objective if conducted by outside profession-als. Is this true of research as well?

I believe that it doesn’t matter if research or evaluation is conducted by museum staff or outside consultants. But both categories of study are likely to be more effective, more efficient, and more valid if carried out by people with experience in such work than if these responsibilities are added on to the workload of people who have little train-ing, less time and (perhaps) less incentive—whether these are museum staff or outside consultants. Expertise helps! There are many examples of excellent evaluation and research studies carried out by museum staff or by external consultants. Unfortunately, there are examples of sloppy evaluation and research studies carried out by both catego-ries of evaluators or researchers. Hiring an “independent” consultant rather than hav-ing work done by a museum employee is no guarantee that the external researcher or evaluator will be more “objective” than a museum employee. Political and personal qualities enter into any relationship and can influence the performance of the work. For example, an employee may feel pressured to modify data or overlook certain factors, so that a conclusion is less negative, but similar pressures may influence outside consultants who rely on future contracts and good rec-ommendations for their professional surviv-al. In working with external researchers, it’s important for museum staff to be sure that the outsider’s approach to evaluation or his or her research agenda matches the politi-cal and educational goals of the museum. In working with staff as evaluators the political and social tone of that particular institution may influence what the staff member takes for granted or fails to notice.

Just because a research finding is pub-lished or is hyped in the press does not mean that it is either valid or relevant—and most

children’s museum employees are not profes-sional researchers. What are some ways that children’s museum staff can become appro-priately critical consumers of research?

There is no simple answer to this ques-tion. Just think about the controversies that surround issues involving empirical data on topics such as evolution or climate change. There are quite a few professional research-ers and evaluators active in the field of mu-seum studies today. There are abundant pro-fessional organizations, discussions, training sessions, and conferences. Most researchers interested in museum studies are willing to discuss professional work with museum staff on either a formal or informal basis. Any publicly discussed research or evaluation report is likely to have some limits to its ap-plicability to a specific situation. But there is also likely to be someone nearby—and ev-eryone in the field is “nearby” today through email, blogs and discussion groups!—who would be willing to comment on a topic, especially if it has been “hyped by the press.”

Research questions, especially brain re-search and more rigorous lab-based psychol-ogy research, are often extremely narrowly defined. How can museums identify relevant and useful applications of broader research?

The applicability of brain research find-ings to practical day-to-day activities of children’s museums is difficult to evaluate, but my sense is that its relevance is minor at best. There was a period when some brain research groups claimed that because there was little brain growth during adolescence this finding made efforts to teach teenag-ers concepts at that age unlikely to succeed. More recently, brain development research-ers are more cautious about applying their anatomical and physiological findings di-rectly to education advice. The ability of humans to adjust to actual situations is in-fluenced by an enormous range of factors. Even with current technology that allows various forms of brain imaging while sub-jects perform some activity, our knowledge of the correlation between the results of such studies and human capabilities in everyday life is still limited.

How transferable is any research—or even evaluation—when children’s museums often have unique environments? How applicable are the findings of studies not conducted in “real” museum environments?

Again, this is not an easy question to answer—it depends. Piaget studied the de-velopment of object permanence in his own children, noting how babies gradually real-

ized that an object out of sight might still exist. (Every parent knows about this devel-opmental stage.) American behaviorist psy-chologists criticized him, claiming that his were not “ordinary” children and he had a small sample size, only three. But his work is irrefutable and relevant to our knowledge of child development. Doing research in labs or with special equipment does not make the results irrelevant for other settings. Mu-seums have set up their own experimental rooms to test a mock up of an exhibition or to ask visitors to try a hands-on activity. Exhibit developers have found such research useful, even if the experimental situation can only provide partial information about how an exhibit will work when incorporated into the public museum setting. It’s impor-tant to ask about the applicability of any re-search results to different settings, but the answer will vary depending on the particular research and the particular settings that are being compared.

If museums follow John Dewey’s philoso-phy of Progressive Education, they engage visitors in learning by doing and work toward (roughly paraphrased from Progressive Muse-um Practice, p. 38) “improving economic and social conditions…and building a stronger democratic society.” As museums work to in-crease accessibility and promote diversity in its broadest possible definition, what research exists—or could be done—to demonstrate the strengths of this approach?

If museums are serious about accept-ing the challenge of promoting democ-racy and social justice, then I believe they should incorporate that responsibility into their research activities. The list of poten-tial research questions provided in the re-cent Learning Value of Children’s Museums Research Agenda, implies that “improving economic and social conditions and build-ing a stronger democratic society” might be addressed within the agenda, but in my view the proposed research agenda could include more direct questions on this topic. Besides proposing “How do children’s museums help us understand cultural variations?” We might ask, “What can children’s museums do to improve tolerance towards the cul-tural variations in our society?” Or, based on knowledge of local social issues, museum staff could initiate research or evaluation ac-tivities deliberately designed to contribute hard evidence of practices that might ame-liorate discriminatory situations.

Some years ago, Jeri Robinson at Boston Children’s Museum added a component to Countdown to Kindergarten, their program

The Evolving Role of Researchcontinued from page 5

8


that prepares children and families to enter the public schools. After a family registers for the program, informational postcards are mailed to their home address. One mailing includes information about taking the card to a nearby public library to get a library card. A piece of mail addressed to a person is a requirement for obtaining a library card as a Boston resident in the Boston Public Library system. I would love to know what difference (if any) it made in the lives of the children who received such a card and acted on it, in comparison to children who didn’t follow up on this opportunity. That, to me, is social science research focused on an is-sue of social justice. Even earlier, in 1976, at the same museum, the late exhibit developer Janet Kamien launched the landmark exhi-bition What If You Couldn’t? that exposed visitors to various forms of disabilities. An evaluation question for that exhibition or

development team. The evaluation staff work with the team to identify the lessons learned and how they could be applied to future projects. In-depth evaluation studies of programs are usually done for grant-fund projects or a high-profile program, like the museum’s preschool. Similarly the evalu-ation staff work with the project team to identify questions, then plan and carry out the evaluation with minimal help from the team.

The next level of study may straddle the line between evaluation and research: stud-ies that are designed to measure the impact of the institution and its mission on visit-ing families and on the community. While staff evaluators at The Children’s Museum conducts these studies, most small and medium-sized museums would likely need outside help. These projects are often large, relatively exploratory, and may not result in immediately applicable findings. In-stead, an institution would have to reflect on and absorb the findings, using them to inform long-term planning or institutional change. An example of such a study was the museum’s large-scale visitor study to deter-mine the rate at which families experienced

various indicators of family learning during their visit and to explore whether any cor-relations exist between demographic factors and frequencies of family learning behav-iors. The study included the development of a multi-item summative or Likert scale that required multiple phases of pilot testing to create a reliable measure that can be used in future studies.

Finally, the continuum ends with research projects. Even at The Children’s Museum, research projects are not often conducted by internal staff (except for some grant-funded projects). For the museum and many other museums, including research projects in the museum’s visitor studies portfolio means finding outside partners such as local college or university professors or graduate students who need to conduct research for their de-gree program. In a recent example of this type of partnership at the museum, a for-mer evaluation intern approached the mu-seum about collecting dissertation data in Playscape, the museum’s gallery for children ages five and under. Internal evaluation staff acted as a liaison for the student, assisting her in scheduling data collection, discussing best practices in observing and approaching visitors, and sending an email to museum members on her behalf. While her research

Contributing to What We Knowcontinued from page 7

may not directly benefit the museum, it will assist the children’s museum field as a whole by providing information on how parents scaffold their pre-school-aged children in hands-on galleries.

As the Association of Children’s Mu-seums looks to build a field-wide research agenda, it is important to remember that many institutions with various levels of evaluation capacity can and do participate in growing the field’s body of knowledge. Much of what we know about our visitors and what works in informal learning set-tings is documented through evaluation as well as research. Non-evaluators and evalua-tors alike can participate in growing our un-derstanding by conducting evaluation and research along the visitor studies continuum and sharing our results inside and outside our museums.

Susan Foutz is the director of research and eval-uation at The Children’s Museum of Indianapolis. She holds a master’s degree in museum studies and began her career as an evaluator in 2003.

Claire Thoma is the evaluation and research coordinator at The Children’s Museum of India-napolis. She earned her master’s degree in museum studies from Indiana University-Perdue University Indianapolis in 2012 and has been at The Chil-dren’s Museum since 2010.

others that focus on differences in cultures, abilities, or backgrounds might be, “does ex-hibition X promote productive interaction among children of different cultures, abili-ties, or backgrounds after one or more vis-its?” Research questions designed to capture very specific data are the most effective way to understand and demonstrate the value of what museums do.

If children’s museums could agree on pur-suing one or two major questions or mea-sures across museums nationwide, what do you think they should be?

For me, they would be about how mu-seums could support the issues discussed in the previous question.

Questions to Dr. Hein were developed by Cyn-thia Mark-Hummel, Lorrie Beaumont, and Nicole Rivera.

Cynthia Mark-Hummel, former director of early learning research and education at the DuPage Children’s Museum (Naperville, IL), has more than thirty-five years of experience with exhibit and pro-gram development, administration, and construc-tion management in museums including The Field Museum, Cranbrook Institute of Science, the Mu-seum of Science (Boston), and the Adler Planetarium.

Lorrie J. Beaumont, EdD, has over twenty years of experience as an educator and evaluator for mu-seum exhibits and programs. Prior to becoming di-rector of Evergreene Research and Evaluation, LLC, she worked for Selinda Research Associates in Chi-cago. Beaumont is now director of programs at the Ann Arbor Hands-On Museum (MI).

Nicole Rivera, EdD, is an educational psycholo-gist and faculty member at North Central College in Naperville, IL. Through her Informal Learn-ing Research Team, Rivera and her students work on research and evaluation projects at the DuPage Children’s Museum and other local institutions. She is the current chair of the Chicago Cultural Organi-zations Research Network.

If museums are serious about accepting the challenge of promoting democracy and social justice, then I believe they should incorporate that

responsibility into their research activities. The list of potential research questions provided in the recent Learning Value of Children’s Museums Research

Agenda, implies that “improving economic and social conditions and building a stronger democratic society” might be addressed within the agenda, but in

my view the proposed research agenda could include more direct questions on this topic. Besides proposing “How do children’s museums help us

understand cultural variations?” We might ask, “What can children’s museums do to improve tolerance towards the cultural variations in our society?”



ince its beginnings in the late 1970s, Providence Children’s Museum (PCM)

1) Learning is experiential, dynamic, and shaped by the physical, social, and cultural environment. 2) Museum experiences support learn-ing by providing opportunities for children to explore with their senses, to make their own decisions, to expe-rience challenges, to learn with and from other people, and to reflect on their own ideas. Based on these ideas, the museum can

provide learners with • time and space to learn in their own ways; • respectful and inspiring learning en-vironments containing multisensory materials and loose parts; • freedom to choose, to take risks, to fail and try again; and• encouragement and support from peers and caring adults.The museum recognizes the connections

between active exploration, play, and learn-ing and that the processes share many char-acteristics.

Creating new “learning frameworks” to replace the old “educational philosophy” required staff to reconcile their individual beliefs with evidence from research in fields from early childhood education to develop-mental psychology to anthropology. And while staff have always built practices based on such literature, the group aimed to cre-ate a shared resource for staff and volunteers that summarizes the museum’s perspective in a single document that supports its edu-cational decisions. For example, research and educational theory support the value of working through challenges for children’s learning and development—and so PCM emphasizes strategies that encourage chil-dren to set goals, challenge themselves, and take risks.

By defining what the museum can pro-vide to foster play and learning, the new frameworks also outline a structure for evaluating exhibits and programs and iden-tifying opportunities for improvement. By grounding museum practices more firmly within the research literature, staff can cre-

ate a common vocabulary to discuss play and learning, in order to voice and achieve the museum’s mission more effectively.

The museum has traditionally drawn on existing research rather than generating its own, but a recent grant from the Na-tional Science Foundation (award number 1223777) to Brown University for a collab-orative project has allowed the museum to advance its research and evaluation efforts. The project has two major lines of work. On the academic side, Brown’s Causality and Mind Lab is conducting research on the development of scientific thinking and children’s perceptions of their own learn-ing (see sidebar). At the museum, the proj-ect team, which includes both authors and other members of the exhibits department, is conducting research and evaluation to better understand what caregivers, children, and informal educators think about learning through play and exploration in exhibits. The museum’s project researcher is jointly appointed as a visiting scholar at Brown, bridging the interests and needs of the two institutions.

Early in the project, the PCM project team evaluated existing exhibit materi-als and interviewed caregivers about their perceptions of children’s play and learning. Most caregivers agreed with the messages about play and learning presented in the exhibits, and they all agreed that children learned through play in general. However, many were less certain about whether their children learned through play during their time at the museum. Some caregivers felt children didn’t retain much about what they learned during museum visits or that play alone would not lead to “learning.” Others believed that children were learning at the museum but they weren’t sure how to rec-ognize or describe this kind of learning. To the project team, these responses indicated a need to communicate more effectively about the various ways that open-ended play and exploration contribute to children’s learning and development, and the range of supporting roles that adults can play—en-couraging, scaffolding, observing and/or let-ting children play independently. But, rec-ognizing that families visit the museum for different reasons and interact with one an-other in different ways, the team wanted to emphasize that families don’t have to change their agendas or their behavior in order for children to benefit from museum visits.

Using Research to Make Learning

through Play VisibleSusan Letourneau, PhD, and

Robin Meisner, PhDProvidence Children’s Museum

Shas been guided by an educational philoso-phy, articulated in an internal document, that includes the educational beliefs ground-ed in research and practice that inform all museum exhibits, programs, and communi-cations. In 2014, education and exhibit staff challenged themselves to not only review and renew this document, but to tackle a significant issue: in response to the decline in societal opportunities for free play, the museum has made an institution-wide com-mitment to advocating for children’s play.

In pursuit of that goal, the museum has convened dozens of play-focused com-munity conversations; communicated the importance of play through articles, inter-views, and social media; and created Play Power, an exhibit that communicates the importance of play for children’s develop-ment. The message that learning occurs through play and exploration is an impor-tant part of the visitor experience, and staff strives to make that learning more visible, more noticed and appreciated by caregivers and by children themselves. Although there is ample evidence to support the importance of play for children’s learning and develop-ment, the terms “play” and “learning” can be ambiguous. Simply stating that children learn through play does not make clear how this happens or what it means in the context of the museum. In order to communicate a cohesive message, museum staff needed to come together to more clearly define their shared beliefs about play and its connection to learning. In addition, the museum’s edu-cational philosophy needed to better articu-late not only what staff believe but why—what evidence supports the staff ’s shared understanding of play and learning?

Inspired by the writings of museum planner Jeanne Vergeront, staff discussed these questions: “Who are the museum’s learners? How do they learn? Why do we think that learning happens this way? And what kinds of experiences does the museum provide to support learning?” The group named learning theorists (including Dewey, Vygotsky, and many others) and bodies of research (including research on cognitive de-velopment, motivation and emotion, social interactions, cultural contexts, etc.) that in-fluenced their perspectives on learning and identified the following primary common-alities among them.

10


The team is using lessons learned from these caregiver conversations to prototype new activities, labels, and print materials that aim to broaden families’ perceptions about what counts as learning during their visits. Based on existing research and input from PCM staff and volunteers, the team identified observable behaviors that relate to children’s developing thinking skills—and in particular, instances when children might be noticing or sharing their thoughts and ideas. When this happens, children may begin to reflect on their own learning, especially if caregivers and educators recognize and sup-port these thought processes. For example, children share ideas by thinking out loud or talking about their discoveries. They also make their thinking visible through nonver-bal behaviors, such as expressing frustration if their plans weren’t working out, focusing deeply on their goals, or using trial and er-ror to test and improve something they were working on. Building on this knowledge, the team tested a play observation activity for caregivers that included a list of “think-ing behaviors” and a brief description of how each relates to children’s learning, prompting caregivers to watch their chil-dren at play and see what they noticed. New versions of exhibit labels also made more explicit the connections between behaviors that caregivers might observe at the museum and children’s underlying thought processes and developing cognitive skills (e.g., “Kids do the same thing over and over to practice

what they’re learning”). In both approaches, caregivers said they appreciated having new strategies for noticing their children’s learn-ing and having the language to talk about it.

The museum’s newly renovated Mind Lab, designed for learning about learning, serves as the site for ongoing research stud-ies at PCM and offers a self-guided activity that encourages exploration and experimen-tation. Labels communicate not only how children learn through experience, explora-tion, and play, but offer research evidence to back up these claims. Illustrated summaries share the findings and implications of indi-vidual studies on relevant topics (e.g., how children learn from one another or through sensory experience), including studies com-pleted in Mind Lab by academic partners and by the museum’s internal researcher. Caregivers report learning something new from these synopses and seeing the relevance of the research findings in their own lives.

Perhaps not surprisingly, the project team found that caregivers enjoy helping the museum think about children’s learn-ing and are very curious to understand more about this work, making PCM’s research and evaluation efforts a powerful commu-nication strategy in itself. Through conver-sations with families, the team has shared not only PCM’s message about the connec-tions between play and learning, but also the depth of thinking that goes into the learning experiences the museum provides, and the research base that supports its efforts. Staff

have also discovered that the mere presence of researchers in an exhibit can convey that children’s learning at the museum is worthy of study, often leading visitors to observe children’s play with greater attention and interest.

The project team has committed to com-municating the process and findings from their work with other PCM staff and visi-tors. The team shares regular updates about results and prototyping plans with the pub-lic through the museum’s behind-the-scenes blog and its quarterly newsletter, and with PCM staff, volunteers, and board through presentations and informal brown bag dis-cussions. This consistent research presence has contributed to a growing culture of observation at the museum. The research-based implementation of the new learning frameworks will ideally give museum staff across departments more structure for re-flecting together about their work, building on their excitement to strengthen an institu-tion-wide culture of reflective practice.

Susan Letourneau, project researcher at Provi-dence Children’s Museum and the Causality and Mind Lab at Brown University, received a PhDin cognitive neuroscience from Brandeis University.

Robin Meisner, director of exhibits at Provi-dence Children’s Museum, has worked in museums since high school and has a PhD in education re-search from King’s College London.

This material is based upon work supported by the

National Science Foundation under the grant numbers listed in the above text.

For more than a decade, Providence Chil-dren’s Museum has partnered with researchers who study children’s learning and development, including the following:

• dr. david sobel, professor of cognitive science and psychology at Brown university, and members of his research team at the Cau-sality and Mind lab study the ways children learn about how things work (cause and effect) and children’s understanding of how people think and learn.

• dr. Jennifer van reet, associate profes-sor of psychology at Providence College, and members of her research team at the college’s Kid think lab study how different types of play develop in early childhood and how play is helpful to children’s overall development.

similar to the living laboratory® model (see article on page 14), researchers from both labs invite families to participate in active re-search studies in the museum’s Mind lab space and share information about their work with

caregivers through one-on-one conversations.the museum’s partnership with Brown

university also includes two research proj-ects funded by the national science Founda-tion. From 2012-2015, one project (award number 1223777) focuses on the develop-ment of metacognition and scientific thinking skills, and includes research and evaluation in museum exhibits alongside research studies conducted in Mind lab. Beginning in 2015, a second project (award number 1420548), in collaboration with paired university/children’s museum research teams from university of California, santa Cruz/Children’s discovery Museum in san Jose, and university of texas at Austin/the thinkery will investigate how open-ended exploration and parents’ explanations affect children’s causal learning in each of the three museums, and how museum exhibit de-sign and facilitation can influence parent-child interactions as well as children’s exploration and learning.

A university / Museum research PartnershipTHE MIND LAB



strive, as many have, to know and to act in ways where we recognize that all parts of the system, from children to organizations, communities, and professional societies are learners, teachers, developers, and research-ers—and creators of research agendas. She poignantly reminds us that it is children who teach adults how to be parents. Many children’s museums have children’s advisory committees. Some, like the Living Labs®

Network presented elsewhere in this issue, mix and match the learning, the researching, and the program developing among visiting children, adult caregiv-ers, museum staff, and univer-sity researchers. How much more invested such stakehold-ers would be and what greater significant benefits would be realized (what Bateson calls “reciprocal benefits”) if we con-tinue to consciously elaborate a fully developed concept of how such role-exchanges can be implemented! Todd Siler, who for many years has been a prime leader in the development of the world of interdis-ciplinary “Artscience” and who also creates cartoon “words of wisdom,” captures the essence of this notion in what he calls a Truizm—“In the world of life, everyone is a student and a teacher.”

Identity

Bateson’s book and much of her life’s work draw on the theoretical framework of Erik Erikson and his eight “ages” from the oral/sensory stage of infants to maturity in adults. Each stage is characterized both by strengths (virtues) and potential patholo-gies. For example, during puberty/adoles-cence the tension is between identity versus role confusion. In the age of maturity, the conflict is between ego integrity versus de-spair, and the strength of that stage he calls wisdom. Various formulations of “identity” are currently plentiful in research literature in education and in the social sciences. Tak-ing a systems approach, how might one apply such notions to identity at all levels of our system? And what is the role of a re-search agenda in creating and sustaining an identity—this is who we are, this is what we know, this is what we believe in, this is how we want to be understood and valued? How do we negotiate our way through various

identity crises that may beset our organiza-tions and profession?

Over these past ten years I have had the privilege of working at the National Science Foundation and processing many hundreds of proposals. As with IMLS, we promote both institutional and field-wide advance-ment. My observation is that it is often much easier for proposers to make a case for institutional advancement, but much harder for field-wide. Part of the reason is that it has been difficult to know and synthesize

all that is pertinent at the field-wide level to then consider how to position one’s initia-tive in that broader context. Efforts such as this Children’s Museum Research Agenda initiative serve both to help focus the field on its shared interests—a statement to the world about significance and identity—and to develop over time a knowledge base that can better inform important decision-mak-ing activities.

Legacy

Particularly insightful and relevant to my points here is Bateson’s chapter, “What We Pass On.” She elaborates: “Each new genera-tion of adults inherits a way of being from the previous generation and passes on to the next, with changes large and small occurring at every step. The handing on of accrued learning and experience takes place in many different forms, not only in relation to bio-logical children but in relation to students and others whom we teach and guide, some-times specifically preparing them to replace us.” (p. 182)

Again, considering the full system within which we act, and looking specifically at the research agenda and its systematic enact-ment via Phase 2 and beyond, i.e. the re-search agenda’s formulation and use, how might we draw on the concept of “legacy” to

help direct our efforts in ways that have util-ity and durability? How do we learn from prior and current efforts (indeed, struggles) to integrate research and practice and then connect that knowledge with the set of val-ues and benefits Jacobsen has called on us to address? Are there roles for brokers who can help forge meaningful collaborations of “legacy builders”? How might our visitors contribute to the legacy, “what-we-pass-on” process?

Active Wisdom

The central concept in Bateson’s book highlights the age of active wisdom that, she explains, is a new phenomenon in human history and evolution, i.e. the time of life where many more of us have already lived full lives but remain vital and can continue to contribute in signifi-cant measure to our families, our communities—our museums! At this individual level, the interview with George Hein in this issue, as well as his recent book, Progressive Museum Practice: John Dewey and

Democracy, are excellent examples of the ap-plication of “active wisdom” in the museum field.

It takes time, energy, and dedication to be a potential instrument of “active wis-dom,” but it also requires a perspective from on high that can see the forest—and even beyond—as well as the trees. For me this implies the valuing of museum philosophy (our core conceptual foundations) and his-tory (the stories we tell about ourselves) as well as the empirical research literature, which, after all is ultimately embedded in either explicit or implicit philosophical positions and within a historical context. This perspective is critical in helping us all be alert and responsive to the currently pervasive and tense dynamics in all sectors of society of the interplay between knowl-edge, values, and power. We need to discern how these dynamics inform what different individuals and groups, within their vari-ous frames of reference, are likely to count as important outcomes, and then how they decide what kind of evidence about those outcomes is convincing. Some knowledge is based on research-derived “evidence,” but, as we in the field all know, and as Bateson af-firms (p. 74), our understanding also grows out of the active exercise of honed skills of empathy and imagination. And, not surpris-ingly, I would suggest that we should apply

Composing a Research Agendacontinued from page 2

12


this wisdom to all levels of the system.Since the founding of the Brooklyn Chil-

dren’s Museum in 1899 has us approaching 120 years, is the children’s museum field in an age of active wisdom, or is it still in its infancy? Might the development of the re-search agenda and process be a sign of grow-ing up, that we are now in a “school-age” stage, and our strength is competence but the potential pathology could be a tempta-tion to yield to inertia?

Wholeness

Finally, Bateson’s book has a chapter called “A Time for Wholeness.” In reading it and thinking about the children’s museum research agenda process, I was reminded that we often express our goals as nurturing the whole child. Shouldn’t this concept of wholeness be extended to the wholeness of families, staff, communities, and our profes-sion/field? Isn’t this wholeness, deep down, what we really yearn for?

As we move forward with the next steps, I think we need to position the research agenda’s use and merits within a broader context—one that fosters connectivity. Bateson ends the chapter, and I this piece, with this thought: “We compose our lives [organizations, communities, profession] in time, improvising and responding in con-text, yet weaving threads of continuity and connecting the whole as we move back and forth in memory.” (p. 181)

Al DeSena is a program director at the National Science Foundation working primarily in the Di-vision of Research on Learning and its Advancing Informal STEM Learning program. He was found-ing director of the Carnegie Science Center in Pitts-burgh, PA, and of Exploration Place in Wichita, KS. He served on the ACM board from 2001 to 2004.

Works Cited

Bateson, Mary Catherine. 2010. Com-posing a Further Life: The Age of Active Wis-dom. New York: Alfred Knopf (Reprint New York: Vintage Books, 2011).

Hein, George. 2012. Progressive Museum Practice: John Dewey and Democracy. Walnut Creek, CA: Left Coast Press.

Jacobsen, John W. 2014. “Measuring Museum Impact and Performance,” White Oak Free Library http://www.whiteoakas-soc.com/presentations-publications.html.

Any opinions, findings, conclusions or recom-mendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

Children’s Learning

• What kinds of learning are effective-ly facilitated and supported in chil-dren’s museums (e.g., cognitive learn-ing, emotional growth, social skills, mastery of the physical environment, attitude formation)? • What foundational knowledge, skills, and experiences for various do-mains (art, science, etc.) are important and necessary for success at age five, ten, fifteen? • What types of early learning experi-ences lead to foundational knowledge and skills needed for success?

Adult/Child Learning

• What is the role of strong adult/child relationships in children’s develop-ment and which aspects of these rela-tionships are children’s museums best poised to support?• What is the impact of children’s mu-seums on parents/caregivers and the family as a whole? • What are the most effective strate-gies for helping parents understand their roles?

Ecosystem of Learners

• Who are the learners in children’s museums: children of different ages, caregivers/parents, teachers, staff, oth-ers?• What relationships between chil-dren’s museums and other learning environments create a healthy learning ecosystem? What is the nature of those relationships?• In what ways are children’s museums essential to the learning ecosystem?

These questions explore how children’s museums prepare children and families to be culturally competent and full participants in their local communities and our global soci-ety. They also explore the nature of healthy communities and what allows children’s museums to become community anchors. Researching these questions can strengthen community ties—especially with schools and universities—further the work of chil-dren’s museums as change agents, provide grounding for new partnerships with com-

munity service organizations, and enhance the profile of children’s museums with local governments and stakeholders.

Children’s Museums and Cultural/Social Issues

• How does a children’s museum serve as a cultural broker?• What is the role of children’s muse-ums in negotiating answers to major questions in society (e.g., technology, nature, cultural responsiveness)?• How do children’s museums help us understand cultural variations (e.g., race, ethnicity, language, class, ability) in learning and development?

The Role of Children’s Museums in Communities

• What is a children’s museum’s im-pact on a community? What makes a children’s museum a successful com-munity anchor? • How do stakeholders value children’s museums in terms of school readiness, family dynamics, and community health?• Over the long term, what impacts can children’s museums have on civ-ics, culture, economics, partnerships, community health, and building bridges and networks?

Call to Action

Through evidence-generating research, children’s museums can demonstrate their impact and increase trust with other organi-zations in the learning ecosystem. The future of education is shaped daily, and children’s museums have a very important role to play in what is to come. Not only is this research agenda intended to initiate new research and evaluation studies and new ways of report-ing research and evaluation findings, but it is intended to encourage public and private funders to underwrite specific research stud-ies in this field, laying the groundwork for policy changes to support larger roles for children’s museums in the learning ecosys-tem and in community capacity-building.

Jessica J. Luke, PhD, is associate director, re-search, Museology Graduate Program at the Uni-versity of Washington. Victoria Garvin is the deputy director of core programs at the Association of Chil-dren’s Museums.

This project was made possible in part by the Institute of Museum and Library Services through a National Leadership Grant, award number LG-64-12-0655-12.

CLearning Landscape

What Do We Need to Know?continued from page 3



amilies visiting Madison Children’s Museum (MCM)

programs at MCM (with UW as a part-ner), at Maryland Science Center and at the Oregon Museum of Science & Industry (partnering with Johns Hopkins University and Lewis and Clark College, respectively). These institutions now serve as initiative hub sites, helping others forge similar part-nerships. To date, more than two dozen chil-dren’s museums and science centers across the country have established collaborations with local child development scientists that integrate Living Laboratory® into their early childhood offerings.

Kia Karlen, director of education at the Madison Children’s Museum, oversees their Living Laboratory® program and serves as Midwest/Southern hub leader for the NLL. “Living Laboratory® fulfilled many goals shared by MCM’s leadership team when we opened our current facility in 2010,” says Karlen. “We wanted to strengthen part-nerships with the University of Wisconsin, provide new opportunities for parent en-gagement and adult education, and create opportunities for staff and volunteers to ac-

cess information and training related to child development. Building from the existing and proven Living Laboratory®

model made it much easier to forge research partnerships than had we started from scratch.”

For the past three years, Karlen has worked closely with Dr. Kristin Shutts, assistant professor of psychology at the University of Wisconsin-Mad-ison. Twice a week for three hours, Dr. Shutts and her team of graduate and undergradu-ate students come onto the Wildernest exhibit floor and invite families to participate in quick, fun studies that explore children’s learning and develop-

ment. “We are interested in understanding how young children sort out all the com-plexities of the social world,” says Shutts. “What factors guide children’s social actions toward others such as decisions about whom to help, befriend, and trust? And how do they figure out how other people are con-nected to one another—friends or foes, bosses or subordinates, or members of dif-ferent social groups? We probe their under-standing of relationships by showing simple displays to them and asking them different questions.”

Living Laboratory® works at the inter-section of, and directly benefits, three dis-tinct audiences: museum visitors, museum educators, and scientists. Museum visitors learn about current research happening in their community through participation in active studies and conversations with local scientists. Museum educators and scientists learn from each other through ongoing mutual professional development opportu-nities, in which scientists learn methods to more effectively communicate their research to the public and museum educators learn about the most cutting edge research in the field of child development.

Living Laboratory® studies are con-ducted out in the open, as part of a dynamic exhibit environment, as opposed to a pri-vate area away from the museum floor. The goal is to increase the visibility of and ac-cess to research experiences in an effort to

Fin Madison, Wisconsin, have dozens of engaging, hands-on exhibits and programs to choose from in this free-choice learning environment. Visitors can explore and invent their way through Possible-opolis, il-luminate shapes in the Shadow Room of the Art Studio, or play a fun game about body lan-guage with a developmental scientist from the University of Wisconsin-Madison (UW) in Wildernest, the museum’s exhi-bition for young children. The UW presence in Wildernest is part of MCM’s integration of Living Laboratory®, a collabor-ative model for educating families about cur-rent research in child development through active participation in research studies and one-on-one conversation with scientists.

Living Laboratory®, established in 2005 at the Museum of Science (MOS) in Boston, Massachusetts, is a scalable, flexible model for bringing scientists who study children’s learning and development onto the exhibit floor to educate the public about the “sci-ence of kids.” The program began in the Discovery Center, MOS’s early childhood ex-hibition, as an effort to actively engage adult caregivers in scientific thinking through a topic that dominates their thoughts, time and energy: their children. Child develop-ment researchers from local institutions (including Harvard University, Boston Col-lege, Boston University, Tufts University, and Boston Children’s Hospital) collaborate with museum staff by bringing studies nor-mally run in their on-campus labs directly onto the exhibit floor. The result: more than 30,000 families have participated in real re-search as part of their Museum of Science experience.

The National Living Laboratory initia-tive (NLL) was created in response to re-quests for help from colleagues in bringing the “science of kids” to their own institu-tions. In 2011, support from the National Science Foundation allowed MOS to begin broad implementation of the Living Labo-ratory® model by establishing collaborative

Connecting Visitors with Research through Living Laboratory®

Marta Biarnes and Becki KiplingMuseum of Science, Boston

Dr. Craig Smith, post-doctoral research fellow at the University of Michigan, plays a research game with a young visitor at the

Ann Arbor Hands-On Museum.

Living Laboratory®, established in 2005

at the Museum of Science in Boston,

Massachusetts, is a scalable, flexible model

for bringing scientists who study children’s

learning and development onto the

exhibit floor to educate the public about

the “science of kids.”

14


break down barriers between scientists and the public. The challenge for scientists is creating research studies that are quick and interactive enough to captivate the interest of child and adult visitors alike, while still addressing the scientific question elegant-ly. Through her time collaborating with MCM, Dr. Shutts has designed dozens of studies that fit well within the museum en-vironment. “In a recent study completed at MCM,” Shutts says, “we showed children a brief video of a conversation between one person who acted in a way that suggested a high position of social power (e.g., chest puffed out, shoulders back) and another person who acted in a way that suggested a lower position of power (e.g., head down). Then, we asked children to guess which per-son was ‘in charge.’ We found that children as young as five years of age were surprising-ly good at using body language to figure out which person was higher in power.” The re-

sults of this research were recently published in the journal Child Development.

In some cases, Living Laboratory® stud-ies are a small part of a larger research ques-tion which may span the museum, universi-ty lab, preschools, parks, etc. In other cases, they are pilot studies designed to be con-ducted only at the museum. Study topics, methodology, and research questions are de-termined by the scientists and then submit-ted to the museum for approval. To conduct a study through a Living Laboratory® site, researchers must submit study proposals to the museum that include evidence of ethical approval from their university’s Institutional Review Board, museum-specific consent forms for parents and guardians, and any educational materials they plan to distrib-ute. Once the study has been approved, all researchers who will be interacting with the public go through a thorough orienta-tion before they set up a study any exhibit. Ongoing—typically weekly—mutual pro-fessional development opportunities occur among researchers and museum educators at the beginning of each research shift.

Engaging adult audiences in children’s exhibits is a challenge for many children’s

museums and science centers. Living Labo-ratory® provides an immersive experience for families that focuses on adult education but is still fun for children. “Living Lab pro-vides ongoing opportunities for parents and caregivers to interact with guest ‘experts,’ and children and adults alike are excited to help with research studies,” says Karlen.

Scientists and museum educators learn to improve their practices through Living Laboratory’s® mutual professional develop-ment opportunities. Karlen: “The presence of Living Lab® researchers and program-ming in Wildernest has been tremendously beneficial to volunteers and staff, many of whom are more experienced working with school-age children than with very young children. Involvement with Living Lab®, researchers and use of research toys pro-vide ongoing opportunities for volunteers and staff to become more knowledgeable in child development topics and more confi-

dent interacting with visitors in Wildernest. The result is a more engaged staff presence in the museum, and more opportunities for staff and volunteers to work as educators while on the floor.”

Scientists have also found a variety of benefits to collaborating with museums in-cluding increasing dissemination of their re-search beyond the academic community to the general public, gaining access to a more diverse pool of participants, and improving undergraduate and graduate student com-munication practices through the mutual professional development program. “We have the opportunity to interact and connect with families who have never participated in research before either because they have nev-er heard about it, or because they don’t have a lot of spare time to bring their child into our lab for studies,” says Dr. Shutts. “Addi-tionally, both graduate and undergraduate students have the chance to practice explain-ing their research to the public. They learn to articulate exactly how and why their work is important; that skill is really useful when it comes to writing grants and recruiting par-ticipants for future research.”

Museum professionals interested in col-

laborating with scientists to bring cutting edge child development research to their exhibit floors can join the Living Labora-tory® community. Community members can access the NLL website (www.livinglab.org) to find an academic collaborator, learn about NLL events, download professional development resources, or create their own “research toys” (fun, educator-facilitated, hands-on activities based on completed re-search studies). To date, the online commu-nity includes over 400 museum profession-als and academics from around the world. “The benefits of participating in the nation-al community have had a ripple effect in our museum,” Karlen says. “The relationships built with museums and colleagues across the country have not only strengthened our Living Laboratory® program, but have enabled our team to build stronger connec-tions with our museum partners, and par-ticularly with science museums.”

As with any museum program, sustain-ability is definitely a challenge. The National Living Laboratory Project Team encourages conversations about future sustainability be-tween professors and museum staff early and often. It is hoped that Living Laboratory®

collaborations may open new funding op-portunities that weren’t considered before.

In October 2014, eighteen NLL com-munity members, including twelve chil-dren’s museums, were awarded Living Labo-ratory® stipends of up to $3,000 to either integrate Living Laboratory® for the first time at their institution, or enhance the educational impact of an already established Living Laboratory® program. Please Touch Museum® (Philadelphia, Pennsylvania) re-ceived a stipend to establish a new collabo-ration with the Monell Chemical Senses Center, an independent research institution. “After learning about the program, we were fascinated by the model of connecting mu-seum visitors and working scientists,” says Alice Gonglewski, associate director of fam-ily learning. “As our mission in a nutshell is learning through play, we like the play-based research activities. Children enjoy the inter-actions and are easily engaged in the tasks.

Living Laboratory® works at the intersection of, and directly benefits, three distinct audiences: museum visitors, museum educators, and scientists.

Museum visitors learn about current research happening in their community through participation in active studies and conversations with

local scientists. Museum educators and scientists learn from each other through on-going mutual professional development opportunities,

in which scientists learn methods to more effectively communicate their research to the public and museum educators learn about the

most cutting edge research in the field of child development.



lifelong learning, making them a natural fit for the Living Lab® model. It is easy to make a case for starting a Living Lab® part-nership to museum leadership because it provides opportuni-ties such as parent education and professional development for staff that many children’s muse-ums want but struggle to fund. The National Living Laboratory initiative provides myriad re-sources for museums of any size, and the professional network makes it easy to seek and share advice.”

To learn more about Living Laboratory®, or to join the com-munity, please visit www.livinglab.org.

Marta Biarnes is the professional development associate, Museum of Sci-ence, Boston and co-principal investi-gator, National Living Laboratory. Becki Kipling is the Discovery Center program manager, Museum of Science, Boston, and principal investigator, National Living Laboratory.

Adult caregivers as well as mu-seum staff are interested in stud-ies that offer opportunities for them to ask questions and learn from the researchers directly. This attractive partnership model of-fers a good balance of benefits for both parties. The museum supports scientists by provid-ing access to larger numbers of study participants and also helps researchers hone their delivery and interaction skills. Museum visitors and staff get a front row seat to the process of scientific in-quiry. Living Lab® is a great way to make science meaningful and accessible to visitors and support our STEAM learning initiatives. Our museum strives to connect with Philadelphia’s rich com-munity of academic institutions and working scientists, and Living Lab® gives us an opportunity to do that in a very mission-driven way.”

Karlen says it best: “Children’s museums are grounded in child development pedagogy and spark

16

Association of Children’s Museums2711 Jefferson Davis Highway

Suite 600Arlington, VA 22202

Nonprofit Org.U. S. Postage PAID

Permit No. 123 Arlington, VA

Che

ryl D

eWel

t

Libbie Brey, graduate student researcher from the UW Social Kids Lab, conducts research on children’s ability to read body language.

Living Laboratory® studies are conducted out in the open,

as part of a dynamic exhibit environment, as opposed to a private area

away from the museum floor. The goal is to increase the visibility of and

access to research experiences in an effort to break down

barriers between scientists and the public.

Revving Up Research

Composing a Further Life: The Age of Active Wisdom Research...

Documents

Transcript of Composing a Further Life: The Age of Active Wisdom Research...