Engineering Summer Programs: A Strategic Model€¦ · Engineering Summer Programs: A Strategic...
Transcript of Engineering Summer Programs: A Strategic Model€¦ · Engineering Summer Programs: A Strategic...
Paper ID #11443
Engineering Summer Programs: A Strategic Model
Dr. Laura Bottomley, North Carolina State University
Dr. Laura Bottomley, ASEE Fellow, is the Director of Women in Engineering and The Engineering Placefor K-20 Outreach and a Teaching Associate Professor in the Colleges of Engineering and Educationat NC State University. She teaches an Introduction to Engineering class for incoming freshmen in theCollege and Children Design, Invent, Create, a course for elementary education students that introducesthem to engineering design and technology as well as various electrical engineering classes.
In 2009 Dr. Bottomley was selected for a Presidential Award for Excellence in Mathematics, Scienceand Engineering Mentoring by the White House Office of Science and Technology Policy and by theEducational Activities Board of the IEEE for an Informal Education Award. She was also inducted intothe YWCA Academy of Women in 2008 for her contributions to eliminating racism and empoweringwomen and was selected as the 2011 Woman of the Year by the RTP chapter of Women in Transportation.In 2013 she was named one of 125 Transformational Women by NC State University.
In her role as director of The Engineering Place at NC State, Dr. Bottomley and her colleagues reach morethan 10,000 students, 2000 teachers and 1500 parents each year. The programs she leads include sum-mer camps for K-12 students; programs that send undergraduates and graduate students into schools towork with elementary and middle school students; training sessions for NC State engineering alumni whowant to be volunteer teachers in their communities; and professional development and classroom supportfor K-12 teachers who want to introduce engineering concepts to their young students. In addition, sheco-authored statewide engineering standards for K-12 and delivers teacher professional development inintegrated STEM. Bottomley also directs NC State’s Women in Engineering program, which works toboost the number of women engineers in academia and industry. The NC State Women in EngineeringProgram was selected as the outstanding program for 2008 by WEPAN, the Women in Engineering Pro-gram Advocates Network for the progress made in recruiting and retaining women students in engineeringat NC State University. In addition to her roles at the University, Dr. Bottomley has taught fifth gradescience as a volunteer consultant, helped schools reinvent themselves as engineering magnet schools andacted as a consultant to the N.C. Dept. of Public Instruction and Wake County Public Schools. She servedon a national team for the National Assessment of Educational Progress developing an assessment forengineering and technological literacy, works with IEEE and the National Academy of Engineering onthe Engineering Equity Extension Project and served as a curriculum consultant on a National ScienceFoundation Gender Equity grant. She also co-authored the Engineering Connections to STEM documentpublished by the North Carolina Department of Public Instruction. She is currently serving on a commit-tee with the National Academy of Engineering, Guiding the Implementation of K-12 Engineering.
Dr. Jerome P. Lavelle, North Carolina State University
Jerome P. Lavelle is Associate Dean of Academic Affairs in the College of Engineering at North CarolinaState University. His teaching and research interests are in the areas of engineering economic analysis,decision analysis, project management, leadership, engineering management and engineering education.
Mrs. Susan Beth D’Amico, NC State University
Susan B. D’Amico Coordinator of Engineering K-12 Outreach Extension The Engineering Place Collegeof Engineering NC State University
Susan earned a B.S in Industrial Engineering from NC State and has worked in the
Telecom and Contract Manufacturing Industries for over 25 years as an Industrial Engineer, Process Engi-neer, Manufacturing Engineer, Project Manager, Business Cost Manager and Program Manager. Inspiredby coursework she developed and presented as an engineer, her professional path made a turn towardseducation by completing coursework for lateral entry teaching.
Susan now works for The Engineering Place, the K-12 outreach arm for NC State University’s College ofEngineering, as a coordinator for Outreach. Her main responsibility is to manage the week long Day and
c©American Society for Engineering Education, 2015
Page 26.644.1
Paper ID #11443
Residential Summer Engineering Camps for rising 3rd through 12th graders in Raleigh and throughoutthe growing number of partner locations throughout the state of North Carolina. Over 1,700 children willbe attending one of her engineering camps during the summer of 2015.
Mr. Landon Drew LaPorte, North Carolina State University
Graduate Research Assistant at the Friday Institute, North Carolina State University
c©American Society for Engineering Education, 2015
Page 26.644.2
Engineering Summer Programs: A Strategic Model (Evaluation)
The Engineering Place is the umbrella program for all engineering K-12 outreach, extension and
engagement activities at North Carolina State University. Operating under the Office of
Academic Affairs this unit last year had over 10,000 touches with K-12 students, parents and
teachers across North Carolina. From the website at www.engr.ncsu.edu/theengineeringplace/:
The Engineering Place is NC State’s K–20 education and resource headquarters for exploring
engineering. Through hands-on summer camps, in-school mentoring, dynamic volunteer
programs, topical workshops and much more, The Engineering Place builds excitement around
engineering for students and teachers.
Engineering summer camps have been offered at NC State University for almost 20 years. Over
time the focus, purpose and strategy associated with planning and executing the camps has
matured to support the current 41 weeks of camp per summer. In the most recent summer these
camps engaged over 1,700 students in grades 2-12 at various locations across the state. Several
design elements of The Engineering Place summer camps are particularly unique, including the
staff mix. In our camps we assemble a combination of engineering educators, K-12 educators,
engineering undergraduate students, and high school students using a tiered mentoring
arrangement. This model was developed as part of our NSF GK-12 grant and has been shown to
have positive and long-term impact on all of the participants. The camps themselves are
designed to be financially self-supporting, including provision for at least five percent
scholarships/aid for those families needing financial assistance. The camp curriculum is linked to
cutting edge research activities in the College, with specific attention to the tenets put forward in
the NAE document, Changing the Conversation5. Attendance at the camps averages 30-40%
female and 35-40% underrepresented ethnic minorities with no specific targeted recruiting.
This paper describes the details of the design of the summer programs and provides assessment
results from more than fifteen years of camps within the College.
Introduction:
The Engineering Place began offering engineering camps almost 20 years ago with a middle
school camp for 30 students and 6 teachers. Since then the program has grown to serve over
1,700 K-12 students per year across the state of North Carolina. This growth has necessitated the
development of a systematic organized approach to planning and implementation, as the team
feels very strongly that the unique flavor of the camps must be maintained. Therefore, the team
has identified distinct measureable goals to which all aspects of the camp are tied: approach,
activities, advertising, application process, assessment and budget.
The mission statement for Engineering Summer Camps is: To provide an enlightening
educational, hands-on experience for elementary, middle and high school students and teachers
that introduces, broadens perspectives and enhances experiences in the disciplines of
engineering and to attract a diverse population to the field of engineering by providing initial or
reinforcing positive experiences to all populations.
Page 26.644.3
The goals are:
Goal 1: Attract a diverse population to summer programs.
Goal 2: Provide an overview of engineering and its many areas of expertise, always highlighting
the true nature of engineering through the Habits of Mind6.
Goal 3: Improve students’ attitudes towards learning in STEM disciplines.
Goal 4: Improve teacher’s attitudes towards teaching in STEM disciplines.
Goal 5: Recruit future engineering students to engineering, with a preference for NC State
College of Engineering.
Goal 2 is particularly important in supporting each of the other goals. Many programs, whether
summer camp in school or out of school, may focus on very narrow aspects of engineering, or
may even be teaching, what the authors would refer to as, technology. To date, therefore, the
camps at The Engineering Place have avoided robotics and Legos to the extent possible to ensure
that participants get a broad view of practical engineering.
This paper is organized to deliberately integrate the mechanics of the camp operation and the
underlying philosophy of those same mechanics. These camps differ from others in the literature
in many ways. The philosophical basis for the camps seems to be completely unique. A
sampling of the literature concerning summer camps yields a variety of publications about
engineering camps for middle and high school students10,11
. Reference 11, in particular, contains
a discussion of the types of camps available and their purposes. Many camps are focused on
robotics. Some are single gender. None of the references discovered mentioned an elementary
engineering camp, and the typical numbers of attendees was under thirty. The camps at The
Engineering Place have some essential differences. The goals are unique. The longevity of the
programs is unique, and the number of attendees, sixty for elementary camps and ninety for 9th
and 10th
grade camps, are significantly different. In addition, there is no other program that
offers a continuum of camps designed on a common platform for students in grades 2-12.
None of the literature discusses staff training, and none of the camps appear to have a tiered
mentoring structure like that found in these camps. For these reasons, the details of these
elements are included in this paper, rather than just assessment results.
The selection process for camp does not give priority to the children whose parents are able to
apply quickly. Giving first-come-first-serve preference was shown to be biased toward more
wealthy parents, which was not intended. The application period runs from the first Monday in
January until April 1st. For the day camps, the selection process is determined by the level of
interest in an applicant’s answers to the questions: “Why do you want to attend camp?” and
“What have you recently learned that excited you?” Upper division high school applicants have a
more rigorous application process for the simple reason that admission to the College of
Engineering is increasingly competitive, and these applicants are within a year of applying. The
goals of the camp have shifted slightly at this level to recruit more directly to the College. The
rubric used for evaluating these applications combines grade point average (40%), course rigor
(30%), personal statement essay (20%), and class rank/standardized test scores (10%).
Page 26.644.4
Camp budgets consist largely of salary expenses for teachers and undergraduate and high school
students. The combined planned budget for the 2015 camps is $482K, comprising 37% for day-
camps and 63% for overnight (residential)-camps. Below are budget percentages:
Residential Camps Budget Breakdown:
Salaries (Full and Part Time Staff) 31%
Meals 18%
Housing 16%
Materials 14% Admin Materials 1%
Misc. Expenses 20%
Day Camps Breakdown:
Salaries (Full and Part Time Staff) 48%
Meals 16%
Materials 6%
Admin Materials 6%
Misc. Expenses 24%
To give a framework for how the camps are organized, sample weekly agendas are shown in
Figure 1. The agenda is designed to be fast-paced, while giving adequate time for each activity
to meet its goals. For example, activities during the beginning of the week need to concentrate
on teaching how to use the design process and on the Engineering Habits of Mind of optimism—
sticking with it until success is achieved—or teamwork. Later in the week, activities may be
more designed to teach a particular scientific concept; however, each activity is multifunctional
and multidisciplinary. Later in the week, the design process becomes a habit in itself, and the
students are functioning better as a team. Activities frequently become more challenging and
more open-ended.
Note that Friday afternoons are dedicated to parent showcases for all students, including student
presentations, and fun design competitions that are assigned Friday morning for younger
students. The design competitions are chosen to incorporate lessons learned during previously
completed activities and may even be a complete redesign of the same activity. Students are not
bored by this repeat; rather they embrace the chance to improve on what they have learned.
An additional important philosophy of the camp is to teach both the attendees AND the
participating teachers about the true and broad nature of engineering. For example, the camps
avoid incorporating robotics. This is not because robots are not a part of engineering, but many
schools use robotics as a substitute for engineering. The fraction of engineers who actually
design robots is terribly small. A more likely subject would be shoe design or food-related
applications of engineering.
Page 26.644.5
Monday Tuesday Wednesday Thursday Friday
9:00 - 9:15Team Assignments/
Badges/Journals/Pre-survey
9:15 - 9:30
9:30 - 9:45
9:45 - 10:00
10:00 - 10:15
10:15 - 10:30
10:30 - 10:45
10:45 - 11:00
11:00 - 11:15
11:15 - 11:30
11:30 - 11:45
11:45 - 12:00
12:00 - 12:15
12:15 - 12:30
12:30 - 12:45
12:45 - 1:00
1:00 - 1:15 Closing Remarks
1:15 - 1:30 Travel to Competition Venue
1:30 - 1:45
1:45 - 2:00
2:00 - 2:15
2:15 - 2:30
2:30 - 2:45
2:45 - 3:00
3:00 - 3:15 Project Ranking
3:15 - 3:30 Dismissal Dismissal Dismissal Dismissal
Gallery WalkGallery Walk of Projects
Awards/ Closing/ Dismissal
Overview of Week
Activity: Ionic Printing of
Hydrogels
Mentor: Daniel Morales
Activity: Medical Systems
Mentor: Julie Ivy, PhD
Activity: Biomymicry -
Mechanical Engineering
Mentor: Leyf Starling
Activity: Airplane Challenge
Mentor: Leyf Starling
Activity: Engineering World
Health
Mentor: Carlos Amaral, PhD
Campers are placed in their
project teams and assigned
new breakout rooms.
Challenges are presented.
Project Dependent Breakout
Time: Campers work on an
engineering project of their
choosing.
Welcome/ Camp Overview/
Presentation to Parents
Lunch Lunch Lunch Lunch Lunch
Activity: Civil Engineering
Project
Mentor: Emily Berglund,
PhD
Prepare for Gallery Walk
Clean-up Breakout Room and
Set Up for Exhibitions
NC STATE COLLEGE OF ENGINEERING K-12 OUTREACH PROGRAM
2014 RALEIGH HIGH SCHOOL SUMMER ENGINEERING DAY CAMP AGENDA
Research/ Project Work
* Finish Projects
* Complete Week Long Activity
* Complete Post Camp Surveys
Welcome/ Intros/ Overview/
Rules and Expectations/ Grand
Challeges
Activity: Double Egg Drop
Figure 1: Sample Camp Agendas
Page 26.644.6
Strategic Camp Design Process
Over the course of many years designing, offering, assessing, re-designing, re-assessing, etc. our
summer camps we have developed a strategic design process approach for existing and new
camps. The simple steps are: Make it Easy, Make it Fun, Make it Work, Work the Mechanics,
and Assessment and Reflection. Each of these is described in more detail below:
Making it Easy
After the camp leadership team has planned the agendas and teachers have vetted them, an
activity journal is created for each camper (see Figure 2). These become the “engineering
notebook” for the week. Space is allowed for each stage of the design process, all activities are
outlined, and a place is provided for student reflection. Not only does this make teaching easier
in the camp, it give the campers a great tool to take home, share with parents and come back to
any activity in the future.
Figure 2: Example student activity journals
While the activity journals describe each design challenge, the undergraduate students are
responsible for working together to develop and implement the testing protocols for each design.
Camp managers must develop a sense of the potential success of a particular test before
implementing at the camp—this is part of the camp protocol so that these undergraduates gain
ownership of that process.
Page 26.644.7
Making it Fun
The camp schedule is designed so that the majority of the day is spent in hands-on work.
Further, the projects are chosen so that 90% of them require creative design (e.g., design a water
squirter that you can use to knock down a paper wall), 8% of them are strictly analytical (e.g.,
learning to do root cause analysis cases), and 2% are procedural (e.g., put together something
from instructions). The focus is not on what the product looks like. In fact, for most of the
projects at camp, it is impossible to predict what the resultant designs will look like, and that is
the point. The projects have carefully constructed constraints and a testing protocol, but the
students generate their own ideas. Examples of outcomes are never shown, so as not to cause the
students to take a particular tack or to limit their creative intent.
Teachers and undergraduates are trained to facilitate the design process to reduce any frustration,
while not overly directing the outcome. Figures 3 and 4 show examples of PowerPoint slides
used in the training sessions for camp staff. Most of the students who attend camps need to be
taught how to engage in the design process and in the engineering habits of mind6, so the weekly
schedule is carefully crafted with this in mind. Shorter, simpler projects with fewer degrees of
freedom are scheduled early in the week with multiple opportunities for iteration. Some of these
shorter projects will be built upon by more complex design opportunities later in the week,
helping to provide scaffolding that increases the probability of success and learning.
Figure 3: Slide for teacher/undergraduate student training P
age 26.644.8
Figure 4: Slide for teacher/undergraduate training
Communications among and within the groups are maintained dynamically. For example, an
“engineering change management” memo can be originated from the undergraduates and passed
to the camp leaders for posting and communicating to the teams. (This most often happens with
new projects that have not been through a camp cycle before.) This might originate because a
participant asked a question that led to a need to make a call to allow an innovation in the design
or to not allow such. The undergraduate students, as a group, make the decision. (For example,
if masking tape is provided, but not duct tape, and a participant team asks for duct tape, will it be
allowed?) Projects are sometimes refined by participants asking unanticipated questions. If a
teacher receives such a question, they communicate with the undergraduates who then determine
whether to make an engineering change or not. An example of such a change would be if a
catapult for marshmallows is being designed that requires both accuracy and distance, and a
participating student group realizes that the rubric will score a catapult that significantly
overshoots the target more than one that just misses it, the teacher may want to make a change in
the way the project is described to disallow this solution…or they may not, depending on what
they want to teach. (This is a real example from camp.)
Making it Work
The camp staffing plan is structured with tiered mentoring and bridging across age groups.
Some team pictures are shown in Figure 5.
Page 26.644.9
Figure 5: Sample team pictures from middle and elementary camp
Using the numbers from an elementary engineering camp, seventy-two elementary students (just
finished grade 2 to just finished grade 4) are divided into six teams of twelve. Each team of
twelve is assigned to a classroom with another team. Each of those teams has a team lead, who
is a teacher with experience teaching the relevant age group, and an undergraduate counselor.
There are also sufficient high school assistants to distribute two to each room. Mentoring takes
place between each of the “levels” of staffing, as well as with each “level” and the student
attendees, as indicated in figure 3. This means that all of the elements are in place to complete a
bridge from K-12 to engineering!
When the camp agendas are designed, function is not a uniting factor, meaning that camps are
not defined as “electrical engineering camp,” or the like. The uniting factor is the engineering
design process, Figure 6. The depiction of the process that is used is based on the five steps
identified by the Museum of Science, Boston4.
Figure 6: The Engineering Design Process used in the NC State University Engineering camps
Page 26.644.10
Because the design process applies to every discipline, every Grand Challenge, etc., it makes an
optimal unifying factor. The camps are also designed to reflect the NAE Grand Challenges for
Engineering7. The camp workbooks, camp tee shirts and the open and closing presentations refer
to the Challenges and connect what goes on in the camp to the broader world of engineering.
Figure 7: Camp tee shirt and workbook content design
New and different camp activities are always included in the agendas as a result of collaboration
with active researchers in the engineering departments. For example, a recent favorite activity
involved designing a hydrogel that can be ionically imprinted from a penny using electricity.
Students learned about the current research being used for hydrogels from the researcher and his
graduate students and were able to create their own. Another favorite activity is a water resource
distribution and model building activity designed by an environmental engineering graduate
student which is based on her research topic.
Working the Mechanics
Each year, the number of camps and partner sites may vary for reasons usually associated with
personnel (e.g., if a departmental contact takes leave and is unable to identify another contact),
but most years the number and variety of camps increase as we grow across the state. Partnering
sites are important because we are able to bring the program and content to areas throughout the
state that don’t have similar local opportunities. The key factor towards a successful partnering
site is to balance providing guidance and direction with encouraging a partner to make it their
own locally flavored camp. This helps integrate relevance into the activities, making the camper
experience more meaningful. Table 1 lists the camps offered in summer 2014.
Page 26.644.11
Table 1: 2014 Engineering Camps by the Numbers: Residential & Day
Day Camps Number of weeks
1,153 Campers
3rd
– 10th
Grade
8 Raleigh
480
4 Charlotte
288
3 Rocky Mount
144
5 Hickory
162
1 Havelock
60
1 Wilson
19
Residential
393
11th/ 12th Grade
6 Raleigh - Week 1
124
6 Raleigh - Week 2
133
6 Raleigh – Week 3
136
Weeks 40 Total Attending 2014 Camps
1,546
Figure 8: Residential & Day Camp Attendance Numbers: 2010 - 2014
Page 26.644.12
As indicated in Figure 8, student attendance at the camps has increased dramatically over the
years since their inception. The demographics of the camps by gender and ethnicity have also
changed. Targeted advertising (through organizations like Girl Scouts and Girls and Boys Club)
is used to attract a diverse group, with some success. The demographics for 2014 are illustrated
in Figures 9 and 10 below.
Figure 9: 2014 Raleigh Day Camp Demographics by Gender
Figure 10: 2014 Raleigh Day Camp Demographics by Ethnicity
Assessment and Reflection: Since their beginnings approximately 15 years ago, the engineering
camps have undergone many changes. One of the most important changes has been the
continuous improvement of the camp assessment system, which in itself leads to changes in the
Page 26.644.13
design and activities of the camps. The assessments have always been chosen to address the
camp goals, as stated in the Introduction, but they have not always been well designed to do so!
It is important to realize that this is not intended to be a research project. The assessments are
designed to match how well the camps meet the camp goals, which can be modified if desired by
the team. Research can be (and has been) overlaid. A paper about a project done by our College
of Education partners has been submitted to this conference. In addition, the camps frequently
encompass the broader impacts portions of grants belonging to fellow researchers by
incorporating research-based activities. The camps make an excellent test bed for research, but
this paper does not incorporate those results.
The first type of assessment done was simply a Likert-style survey of whether the participants
enjoyed various aspects of camp. The assessment plan still includes a survey of this type for
formative assessment. Statistical analysis is not done on this data, as it is deemed uninstructive.
If a certain number of participants do not like an activity, it is modified. Sample results for the
camps are shown in Figure 11 below.
Figure 11: 2014 Elementary Day Camps combined. Number of respondents choosing 1-5 for,” How did you like each of the activities you did at camp”
Sample conclusions from this data would be that students enjoyed the theme park ride design and
the marble wall run design and that the artificial hear/hydraulic arm needs to be revised or
replaced. The activities with the highest ratings are ones that we are likely to select for our
partnering camp locations in the following years. These data are also used to form the process
for creating new activities. For example, the theme park ride is a very open-ended design
project. The attendees clearly preferred this activity to the artificial heart/hydraulic arm activity,
which is very procedural. Future activities will be more open-ended.
The second type of question asks the students to describe their feelings about various aspects of
camps (see Figure 12).
Page 26.644.14
Figure 12: 2014 Middle School Day Camps combined. Number of respondents choosing 1-5 for,
“Select the option that best describes your feelings”
A sample conclusion from this chart would be that students want more time for the activities.
This is a tough concept that is best addressed early in the week so that improvements in time
management are realized as the week progresses. Next year’s staff training sessions and intro
activities will incorporate more of this important concept.
Although ratings for the elementary, middle and early high school (9th
and 10th
grade) day camps
are compiled as above, ratings are separated for each of the late high school (11th
and 12th
grade)
workshops, as they are taught by a different set of staff in the departments. The Camps Director
then meets with each department to go over the analysis and suggest changes to make in
activities, structure, etc.
The second level of assessment asks attendees to self-rate on characteristics that match with the
camp goals (as stated in the introduction). Figure 13 is an example of such a rating from the high
school day camp, which has campers select a research topic to pursue after the first day of camp.
Page 26.644.15
Figure 13: 2014 High School Day Camp Assessment Data, Number of respondents choosing 1-5
From the output above, the camp administrator concluded that the attendees felt comfortable
working in the team setting, which is an important collaborative element to achieve. The
administrator also found evidence for a need to reevaluate the incorporation of the engineering
design process and whether all staff were sufficiently prepared to use it appropriately. A deeper
level of evaluation of the goals of the camp is measured as in Figure 14.
Figure 14: 2014 High School Day Camp Assessment Data, Number of respondents choosing 1-5
Campers felt unsure about coming up with ideas and developing creative solutions for their
group project. The training for the next year camp will incorporate more mentoring techniques in
staff training sessions to promote creativity and resulted in the administration designing an
opening activity that introduces creativity.
Starting in 2006, camp assessments also included collection of data using a survey standardized
by the Burroughs Wellcome Fund. Sample results are given in Table 2. Several problems
existed with using this type of assessment, including confusion between how engineering and
science relate. This effect could have contributed to a lack of certainly of how to interpret the
data. In addition, one cannot interpret an answer that the program will not encourage the student
Page 26.644.16
to take more science classes as negative, if it only confirmed their decision to do so.
Nonetheless, the funder required that we use this assessment, to which we added our own
formative assessment questions. Table 2 shows sample results from this assessment.
Table 2: 2006 Summer Camp Attitude Assessment Strongly
disagree Disagree Uncertain Agree Half
way (4.5)
Strongly agree
No response
1a. This program helped me understand science better.
2.0% 4.0% 16.0% 50.0% 0.0% 26.0% 2.0%
1b. Because of this program, I feel better about being able to learn science.
2.0% 8.0% 18.0% 46.0% 0.0% 24.0% 2.0%
1c. I learned some things in this program that I can use in science class at school.
2.0% 6.0% 10.0% 46.0% 0.0% 34.0% 2.0%
1d. Because of this program, I think I am more aware of the importance of science in everyday living.
2.0% 2.0% 14.0% 54.0% 0.0% 26.0% 2.0%
1e. I tell my family or friends about the things we do in this program.
2.0% 6.0% 10.0% 34.0% 0.0% 44.0% 2.0%
1f. Because of this program, I am more excited about science.
4.0% 10.0% 16.0% 38.0% 2.0% 24.0% 6.0%
1g. Because of this program, I think I have a better understanding of what scientists do.
4.0% 6.0% 6.0% 40.0% 2.0% 40.0% 2.0%
In 2011 The Engineering Place was offered the opportunity to participate in a deeper level of
assessment by the MISO (Maximizing the Impact of STEM Outreach) project, an NSF funded I3
project3. The MISO project was created to unify the evaluation of STEM outreach projects at our
university and to track participants longitudinally. The survey collects attitudinal data about
math attitudes, science attitudes, engineering and technology attitudes and 21st century learning,
using a five point Likert scale. The attitude tests were devised and validated by the MISO staff9.
A snapshot of the MISO attitudes report from the 2014 camps is shown in Figure 15.
Page 26.644.17
Figure 15: MISO Attitude Survey Sample Results from 2014 Camps
The attitude surveys are administered in the morning on the first day of camp and again at the
end of camp. This leads to the ability to do paired sample analysis. Here are findings for
engineering camps in the summer of 2014. Constructs are formed from multiple questions in
Page 26.644.18
four areas of interest, math, science, engineering and technology and 21st Century Skills. All
attitudes tests are paired sample t-tests from pre and post measures of individuals.
Elementary
The S-STEM assessment created by the MISO project3 has four sections, math, science,
Engineering/Tech, 21st century skills. Students are asked questions designed to elicit their
attitudes in these areas. A construct composite is made by averaging responses to questions in
each of the four sections. Here answers are averaged for each participant and then pre scores
and compared to post scores.
For elementary cohort A, students showed significant attitudinal gains in all areas. For
elementary cohort B, students showed significant attitudinal gains in Math, Science, and 21st
century learning showing significant, as indicated in Table 3. Engineering attitudes did not show
significant improvement, although the pre-mean rating was already 4.2 out of 5 on the Likert
Scale.
Table 3: Elementary Engineering Camp Post vs. Pre Attitude Analysis
Paired Samples Test Difference in Means,
Post-Pre-test
constructs,
Elementary
Paired Differences Sig. (2-tailed) Mean Std Dev N 95% Confidence
Interval of the Difference
Lower Upper
Math, cohort A .0777 .3519 113 .0121 .1433 .021
Science, cohort A .2286 .4283 112 .1485 .3089 .000
21st Century Skills, cohort
A .1315 .3355
109 .0678 .1952 .000
Engr/Tech, cohortA .0958 .4061 109 .0187 .1729 .015
Math, cohort B .1136 .3638 110 .0449 .1824 .001 Science, cohort B .2253 .4461 108 .1402 .3104 .000 21
st Century Skills, cohort
B .1633 .3570 103 .0935 .2330 .000
Engr/Tech, cohort B .0279 .4598 108 -.0598 .1156 .530
Middle and High School Camps, Grouped
All attitude scores improved significantly from Pre to Post for both cohorts C and D, as indicated
in Tables 4 and 5.
Page 26.644.19
Table 4: Cohort C Middle and High School Engineering Camp Post vs. Pre Attitude
Analysis Paired Samples Test
Difference in Means,
Post-Pre-test constructs,
Middle and High
Paired Differences Sig. (2-
tailed) Mean Std.
Deviatio
n
N 95% Confidence
Interval of the
Difference
Lower Upper
Math .0661 .3001 475 .0391 .0932 .000
Science .1073 .3709 472 .0738 .1409 .000
Engr/Tech .0642 .3446 469 .0329 .0955 .000
21st Century Skills .0798 .3604 468 .0471 .1126 .000
Table 5: Cohort D Middle and High School Engineering Camp Post vs. Pre Attitude
Analysis Paired Samples Test
Difference in Means,
Post-Pre-test constructs,
Middle and High
Paired Differences Sig. (2-
tailed) Mean Std.
Deviatio
n
N 95% Confidence
Interval of the
Difference
Lower Upper
Math .0517 .28058 580 .0288 .0746 .000
Science .0993 .3583 578 .0700 .1286 .000
Engr/Tech .0299 .3849 576 -.0016 .0614 .063
21st Century Skills .0482 .3782 573 .0172 .0793 .002
Some of the most interesting data are coming from the longitudinal analysis that is beginning to
come out of the MISO project utilizing data from the National Student Clearinghouse8.
Engineering Place camp participants are just beginning to be old enough to have been surveyed
by MISO and graduate. The numbers are small, as yet, but they will continue to increase. Some
sample results are outlined in Table 6.
Table 6: Sample Longitudinal Tracking Results from Independent Samples t-tests
Persistence (Fall 2012-Fall 2013)
105 students matriculated in Fall 2012 and 94.29% continued in Fall 2013
This was significantly higher than the National Average of 68.7% persistence (t = 11.24, df =
104, p < .01)
Extended Persistence (Entered College Before Fall 2013)
Of the 317 students that matriculated to college from the Engineering Place, 95.90% persisted by
enrolling in consecutive Fall semesters
This was significantly higher than the National Average of 68.7% (t = 4.38, df = 316, p < .01)
On-track in College
Engineering Place average of 89% of students remained on-track during college (students did not
miss consecutive Fall or Spring semester enrollment, did not withdraw from enrollment during
the semester, and did not enroll less than full-time)
Page 26.644.20
This was significantly higher than the National Average of 84.7% (t = 2.59, df = 346, p = 0.01)
Time to Graduate from Earliest College Enrollment
Engineering Place average time to graduate was 3.73 years for 27 students that did graduate from
4-year institutions
This was largely significant lower than the National Average of 4.41 years (t = -8.05, df = 26, p <
.01) -
Two-Year Graduates
Engineering Place had 27 four-year college graduates and 10 two-year college graduates
Note that the metrics, although they are statistically significant where indicated, do not take into
account any self-selection bias for students who choose to come to an engineering camp.
Additionally, numbers are still too low to do gender and ethnicity analysis, but these results will
be interesting, when available. The sample sizes for time to graduate and two-year graduate
data are much smaller than other samples, but the findings remain valid. The MISO data and the
longitudinal analysis of it will enable the staff to monitor the long-term effects on camp
attendees. These will, in-turn, allow the camps to be changed, if necessary. Initial results show
very positive long-term impact and do not suggest changes.
Conclusions
After almost 20 years of offering summer camp, The Engineering Place has accumulated a great
deal of information about what works and doesn’t work. One of the most important lessons
learned has been to be sure that everyone on the planning and implementation teams understands
and commits to the overall mission and goals for the camps. This dedication has allowed The
Engineering Place to post very effective outcomes for the camp participants. One of the
hallmarks of work originating with The Engineering Place is to share results, materials, lesson
plans and stories with any interested party, with the only requirement being for
acknowledgement. Others are encouraged to contact the authors for more information.
This paper outlines the camp mechanics and some assessment results focused on students.
Additional outcomes are sought for the influence the camps have on participating teachers.
Assessment of these effects will be the subject of future work, although anechdotal evidence is
very encouraging.
Acknowledgements
Portions of this material are based upon work supported by the National Science Foundation
under Grant No. DUE-1038154 – any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and do not necessarily reflect the views of
the National Science Foundation.
Page 26.644.21
References
1. Bottomley, Laura and Elizabeth Parry, (2002). K 12 Redux: Sending College Students Back (In)To Schools.
ASEE Annual Conference Proceedings, 2002.
2. LaPorte, L. & Wiebe, E. (April, 2015). Operationalizing Metrics of Persistence and On-Track in the STEM
Pipeline Using the National Clearinghouse Database. Accepted for presentation at the NARST Annual
Meeting, Chicago, IL.
3. MISO, Maximizing the Impact of STEM Outreach. (2014). MISO Project Homepage. Accessed January,
2015 at http://miso.ncsu.edu.
4. Museum of Science, Boston, (2014) EIE Homepage, Accessed January, 2015 at http://www.eie.org/.
5. National Academy of Engineering. (2008) Changing the Conversation: Messages for Improving Public
Understanding of Engineering. Washington, DC: The National Academies Press, 2008.
6. National Academy of Engineering and the National Research Council , (2009) Engineering in K–12
Education: Understanding the Status and Improving the Prospects Washington, DC: The National
Academies Press, 2009.
7. National Academy of Engineering, (2015) Grand Challenges for Engineering Homepage, Accessed
January, 2015 at http://www.engineeringchallenges.org/.
8. NSC, National Student Clearinghouse (2015). Homepage, Accessed January, 2015 at http://
http://www.studentclearinghouse.org/
9. Unfried, A., Faber, M., Stanhope, D. & Wiebe, E. (in press, 2014). The development and validation of a
measure of student attitudes toward science, technology, mathematics, and engineering. Journal of
Psychoeducational Assessment.
10. Barger Marilyn, Richard Gilbert, and Marie A. Boyette, (2011) Best Practices for Student Robotic Camps,
ASEE Annual Conference Proceedings, 2011.
11. Yilmaz, M.; Jianhong Ren; Custer, S.; Coleman, J., "Hands-On Summer Camp to Attract K–12 Students to
Engineering Fields," Education, IEEE Transactions on , vol.53, no.1, pp.144,151, Feb. 2010
Page 26.644.22