The Blue Waters Student Internship Program: Promoting Competence and Confidence for Next Generation Researchers in High-Performance Computing

Patricia Jacobs, Phillip List, Mobeen Ludin, Aaron Weeden, Robert M. Panoff, PhD Shodor Education Foundation, Inc.

Durham, NC pjacobs@shodor.org, bplist@shodor.org, mludin@shodor.org, aweeden@shodor.org, rpanoff@shodor.org

Abstract—The Blue Waters Student Internship Program (BWSIP), a year-long program funded for three years by the National Science Foundation, motivates and trains the next generation of supercomputing researchers. A community engagement partnership of the Blue Waters Petascale Computing Facility at the National Center for Supercomputing Applications (NCSA) and Shodor, the BWSIP has developed, demonstrated, and evaluated novel lessons involving hands-on, interactive, and collaborative methodologies to teach parallel and distributed computing (PDC) and high-performance computing (HPC) topics. Students participating in the program gain experience in the application of high-performance computing to real-world problems in science, mathematics, and engineering through a year-long internship. By engaging undergraduate and graduate students in Petascale computing research and development projects, students build confidence and competence in PDC and HPC.

The BWSIP recruited a large and diverse applicant pool from across the US from which 21 research interns reflecting that diversity were selected and each matched with a mentor and a project for the year-long internship. Students, many having only introductory programming experience, began their internship by attending the two-week Petascale Institute –each day including 6.5 hours of directed, inquiry-based learning and 2 hours of open lab using the run-modify-write paradigm– during which they were trained in PDC and HPC tools, techniques, and technologies using Blue Waters at NCSA, and by analogy other XSEDE HPC resources. Students then continued working all summer on their home campuses, or were hosted by their mentor, with on-going work expected to be continued during both Fall and Spring semesters.

The project engaged an external evaluator to conduct formative and summative assessments of the program. BWSIP Interns participated in pre- and post-surveys, daily reflections/evaluation questions, as well as in a focus group during their training. Even with significant differences in background, knowledge, and with varying projects, participants stated that the two-week institute was an essential element to help them learn conceptual thinking and how to program using parallel computing. It is proposed that the curriculum and approach for the Institute could be adapted

for a semester course at the undergraduate or graduate level.

Keywords—Petascale education, Interactive, Templates, Inquiry-based Learning, Experience, Evaluation, Curriculum, Pedagogy, Parallel Computing, High-Performance Computing, Distributed Computing, Supercomputing, and Programming

I. INTRODUCTION Blue Waters, supported by the National Science Foundation,

the National Center for Supercomputing Applications (NCSA), and the University of Illinois, is one of the world’s most powerful supercomputers and the fastest on a university campus. To answer the challenge of developing a next-generation community of practitioners and users for multi-core/many-core supercomputers [1], Shodor and the National Computational Science Institute, in collaboration with NCSA and Blue Waters, developed the Blue Waters Student Internship Program (BWSIP) whose features are described in this paper along with preliminary results showing substantial learning among the interns and instructors. The BWSIP experience provides guidance for both improving the effort for subsequent cadres of interns while informing the development of curriculum for undergraduate or graduate courses in parallel and distributed computing (PDC) and high-performance computing (HPC).

In Spring 2014, 21 research interns were selected to participate in the BWSIP through an application process from across the United States. The program provides each student with an intensive two-week training workshop in PDC/HPC tools, techniques and technologies, a $5000 stipend, a research mentor for a full year (summer, fall, and spring), and the opportunity to provide a presentation or poster on the results of their research activity at the 2015 Blue Waters Symposium.

The Blue Waters Petascale Institute successfully trained the BWSIP interns alongside four XSEDE Student Research Experience (SRE) students. The institute was held at NCSA at the University of Illinois at Urbana-Champaign, IL, from May 25, 2014 to June 6, 2014. Instructors were from Shodor and NCSA. Interns participated in inquiry-based explorations, hands-on training, and evening labs to learn about PDC and HPC.

II. SELECTION PROCESS Shodor and the Blue Waters Project selected 21 undergraduate

students to participate in the Blue Waters Student Internship Program for 2014-2015. The program participants were drawn from a much larger pool of 62 student applicants and 30 mentors who submitted 34 project descriptions for internship positions.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

The selection process for interns, mentors, and projects was based on three main criteria: 1) likelihood of success, 2) creating a diverse workforce, and 3) promoting excellence.

The likelihood of a student’s success was measured in terms of his/her experience, education, and interest in HPC. In addition, students who took the initiative to arrange an internship with a specific mentor were evaluated as being more likely to be successful. The likelihood of success for each proposed project was measured in terms of its use of PDC and HPC in teaching or in STEM research, the ability to match a student with it, and its use of the Blue Waters supercomputer itself. Each project proposal was given a rank in terms of how well it was planned, how it fit in the domain of sciences, engineering, or mathematics, and its scope in terms of duration and amount of work required to complete.

One of the foremost challenges in high-performance computing is promoting involvement of historically under-represented undergraduate students. BWSIP has been facing the challenge head-on while supporting undergraduate internship experiences that involve the application of HPC to problems in the sciences, engineering, or mathematics [1]. Since the BWSIP seeks to promote a diverse workforce in HPC, based on past experience with an earlier internship effort [2], recruitment and participation among under-represented communities was proactive and encouraged, including women, minorities, and students from a wide range of academic institutions, including predominantly undergraduate institutions (PUI) and minority serving institutions (MSI). For 2013-2014, the recruiting effort resulted in a diverse, qualified applicant pool for BWSIP. The applicant pool described themselves as 30% Caucasian, 23% Asian or Pacific Islander, 22% African Americans, 7% Hispanic, 7% Middle Eastern, and 12% Not Specified. In addition, 23% of the applicants were women. These recruitment efforts led to a diverse pool of applicants, where gender and ethnicity were not primary criteria for selecting a diverse group of interns. This resulted in the selected group of interns consisting of 57% females and non-Caucasian males.

The criterion of “promoting excellence” was a qualitative expectation of how much a student could grow in his/her knowledge of PDC/HPC. A student who already had a strong background in PDC/HPC would seemingly not benefit as much from the BWSIP as one with only a less-than-expert level of knowledge or experience.

Each project description and intern application was diligently reviewed to ensure that the right projects were identified, the right interns were identified, and that the right interns were matched with the right project and mentor to lead to a high probability of success.

Figure 1. 2014 Petascale Institute Participants and Instructors

III. TRAINING THE TRAINERS The institute instructors consisted of Shodor staff members

with guest speakers from NCSA and the University of Illinois. Shodor staff had successfully delivered similar training/curriculum in the past through the National Computational Science Institute workshops on parallel and cluster computing as well as parts of the curriculum at different conferences such as XSEDE, SC, and Tapia [2,3]. To prepare Shodor instructors to plan, deliver, and evaluate effective training on the Blue Waters system, Shodor instructors themselves attended an intensive four-day training at NCSA in January 2014. The training was lead by NCSA and Blue Waters staff members.

During this training, the instructors were introduced to the Blue Waters architecture, login access, programming environments, and other tools to aid in areas such as visualization, debugging, and optimization. The instructors ported and ran on Blue Waters existing code from scientific modules used in previous Shodor workshops for training, testing, and scaling purposes. The Shodor staff members were also taught different methods for running parallel programs on the Blue Waters system that could take advantage of its unique hardware resources, exploring different examples that could be used in the institute.

As part of the training, Shodor instructors and the NCSA team shared ideas on the curriculum for the institute. Topics were chosen to engage students in PDC and HPC tools and techniques over the two weeks of the Petascale Institute and discussed best practices for teaching them. To ensure that the most qualified and experienced instructors were selected for specific topics such as CUDA and matters relating to Blue Waters architecture, the NCSA team offered to teach or provide guest speakers for specific units. In addition, the Shodor instructors met with the I-STEM evaluation team to discuss evaluation methodologies that could help improve the institute on a daily basis within the institute and for coming years. Following the training at NCSA as a test of their newly-acquired skills –since one learns best that which one is required to teach to another– the Shodor instructors prepared lessons and trained other Shodor staff on accessing, using, and optimizing the use of Blue Waters.

IV. THE PETASCALE INSTITUTE

A. Early Access to the System All BWSIP interns attended the Petascale Institute, along with

the additional four students from the XSEDE Summer Research Experience (SRE) who would benefit from training in preparation for their SRE project. One of the main goals of the institute was for all of the students to access and log into the Blue Waters system as soon as possible and become comfortable with navigating and using the supercomputer. On the first day of the institute, students were given a brief hands-on reminder of basic Linux/Unix, command line interfaces, and remote access tools. By the end of the afternoon session, all of the students were able to successfully copy files such as project code to and from Blue Waters. All students had training accounts on Blue Waters, but some difficulties arose, such as not being able to use the Globus Online files transfer from their own laptops or from their home campuses. Therefore, to help smooth the process, the students were also provided with accounts on Shodor servers so they could easily use tools such as “SCP” to copy files to and from Blue Waters.

Another goal for early access to and success in running the Blue Waters system was to enable students to run and observe real

scientific codes deployed on multiple nodes. Working examples were taken from a range of science domains and demonstrated different types of scientific computing with a range of communication and computation patterns. These modules included GalaxSee (an n-body simulation), Pandemic (a spread of disease model), and Game of Life (a cellular automata simulation of Conway’s Game of Life). The students were able to run these example modules on multiple Blue Waters nodes using different problem sizes and numbers of nodes that gave them an opportunity to observe and discuss different methods for scaling, such as strong and weak scaling. Also, the students were able to realize and discuss the needs and importance of parallel and distributed computing.

B. Interactive, Inquiry-based Lessons The lessons at BWSIP were interactive both through consistent

instructor-student interaction and in-lesson examples for the students.

During the first days of the institute, to ensure students comprehended the material of a lesson, the instructors would ask the students as a whole if they understood what had just been explained. This was usually met with a few nods and no raised hands, and it failed to communicate adequately the students’ comprehension. The students told the instructors through daily surveys that a better way to detect confusion was to ask the group pointedly, “Is there any confusion?” They also suggested a change of the typical pacing question from, “Is everyone ready to move on?” to, “Is there anyone who still needs help or more time?” This allowed individual students to make their voices better heard. In itself, this phrasing is not a new pedagogical technique, but it is a lesson learned and an example of instructor-student interaction helping the learning process.

Additionally, the Shodor instructors who were not teaching a given lesson would help in a method called “shoulder-surfing.” The students were told at the beginning of the institute that they could always seek one-on-one assistance from any of the instructors, even in the midst of a lesson. When the teaching instructor was presenting, the other instructors would be on the periphery of the room, scanning the students, looking for those in need; such a need might include troubleshooting a technical difficulty or explaining something the student felt uncomfortable asking in front of the whole class. When students were working on in-lesson examples or inter-lesson exercises, all of the instructors would continuously walk around the class, checking on students’ progress and helping when needed. This shoulder-surfing, with four or more instructors involved simultaneously, helped students with questions sooner and removed much of the students’ fear of slowing down the progress of others by asking a question.

Figure 2. Instructors and students working together.

The instructors told the students to approach any experiment or test with a shortened form of the scientific method: expectation, observation, and reflection. This method could be used even on the small scale of in-lesson examples, and could quickly reveal to a student any misunderstanding about the subject, which could then be addressed by an instructor or a shoulder-surfer before the lesson moved on to new material. While this method relied on the student’s initiative to seek assistance, other methods (e.g., exercises, labs, and surveys) allowed instructors to actively see where assistance was needed.

Most lessons at the institute included interactive examples. These examples were intended to keep the students engaged, demonstrate how to apply the principle/technology at hand to a problem, and, in many cases, be modified by the students for better understanding. Some of these were not code examples, but were physical examples by analogy to which the students could relate. Where feasible, the code examples were programs that the students would run and modify from their own accounts. In other cases, the instructor would have an active discussion with the class, which could involve modifying code “on-screen” at the suggestion of the students.

One such interactive example was an active discussion used in a presentation on cache performance to explain fetching, misses, latency, and overall cache use. In this discussion, students were guided by the instructor in determining how a librarian could best keep certain books at certain levels of convenience, fetching them when certain requests came in, and storing them later. With this explicit analogy, the students developed an understanding of the concepts and could then ask newly-formed questions with this understanding. Due to time constraints or abstractness of the subject matter, a few lessons lacked interactive examples; in both the daily and end-of-institute surveys, participants rated these lessons among the lowest for helping them learn, and the comments made it clear: “It's hard to say that I learned anything, when there were hardly any examples.” [4]

C. Exploration Exercises Throughout the institute, students participated in exploration

exercises designed to help them immediately expand upon what they had just heard about in presentations or practiced in hands-on examples. These exercises were developed in such a way as to encourage students to make discoveries by modifying working examples, seeing how they work, and extending them to run faster, while making better use of the software tools, or more effectively utilizing the Blue Waters system. Students modified examples that already worked, building understanding by running and modifying the code. By contrast, students reported previous classes had forced them to finish non-working examples themselves before being able to explore and make modifications to them, or –even worse– having to type in a whole example code just to have something to test. As an example, when students learned visualization techniques in the second week of the institute, exercises expanded upon already-working examples that students had run, modified, and built in the previous days of the workshop, as opposed to students having to generate working code first and then create a visualization of it. Students responded favorably to the exercises in daily surveys and frequently requested more of them. Students seemed particularly appreciative of exercises with

clear instructions, working code, and opportunities to work together and help each other.

Figure 3. Time to Science v. Number of Cores.

A notable example of this kind of exercise was the institute session with the highest mean value (4.35 out of 5) according to daily surveys [4], the so-called “Time to Science Exploration.” Before the exercise, the students had listened to several presentations, done hands-on examples about supercomputer architecture and why supercomputers are used to do science, and taken a tour of the Blue Waters Petascale Computing Facility. Then they were given an opportunity to explore a real science example running on the same Blue Waters system whose machine room they had just seen in person during the tour.

The exercise utilized the GalaxSee n-body model. Students were tasked with discovering what the “Time to Science” curves for this model would be; that is, what the shape of the graph would look like if the number of cores were plotted on the x-axis and the runtime of the parallel program were plotted on the y-axis. Students divided into groups, with each group responsible for a certain problem size (number of stars in the galaxy) and with each person in the group responsible for a certain number of cores –2, 4, 8, 16, 32, 64, etc.– (a physical demonstration of domain decomposition). Students were also tasked with writing down their results and walking them up to the instructor at the front of the room, who plugged them into an interactive graphing tool (a physical manifestation of both message passing and functional decomposition).

Students applied what they had learned earlier in the day to submit jobs to the Blue Waters system. As the results came back, students were encouraged to pay attention to patterns in the various runtimes throughout the room as other students delivered their results to the instructor. For some small problem sizes and core counts, students received their results quickly and were the first to report their results; other students took longer to receive the results; and in the meantime, the instructor entered in the data. Once all results had been submitted and entered, the students were asked what they expected the curves to look like. Many students thought the runtimes of the program would continue to decrease steadily as the number of cores increased. The instructor then revealed the curves and asked the students whether what they observed in the curves matched what they were expecting. For

many students, the observation did not match the expectation; the speedup of the program actually diminished as the number of cores increased and even became a slowdown for large problem sizes and core counts.

Students were tasked with using what they had learned during the day to reflect on the results and come up with an explanation for why the speedup was mitigated and a slowdown had occurred. Many students were able to deduce that having more cores implies having more communication, and that for algorithms with many communications, having more cores can have a significant effect on the communication overhead incurred during runtime. Students worked together to come to this conclusion, and as previously mentioned, this exercise scored highly in the daily survey. Many students commented that it was a good opportunity to have hands-on practice using the system, work together and help each other, and make a discovery about modeling a real problem. The exercise was an exemplar of the kind of approach the instructors sought to make during the institute, which was successfully received and had a positive effect on student learning.

D. Evening Labs Evening labs provided students with an opportunity to explore

and solidify their understanding by engaging in focused, hands-on summative activities that brought together topics from the lessons of the day. At two hours in length, the evening labs were designed to focus on a given learning objective while providing students time to work together and to think more deeply about the overall concepts and applications. An example of an evening lab had the students practice their skills using OpenMP by modifying an existing serial code to parallelize it using a task-based approach. The lab began with an instructor providing a summary of the model and its code, explaining the task, and encouraging collaboration and a planning-first approach. Students then began working: some individually, but most in pairs or in groups. Though some students were not able to complete a solution by the end of the lab, according to the daily survey, most rated the experience as “Valuable” or “Very Valuable” (15 of 25), and rated their understanding as increasing somewhere between “Moderately” and “Significantly.” Instructors provided a sample solution at the end of the lab session so students could, on their own time, review their approaches and solidify their understanding.

In other labs instructors provided more detailed instruction at the beginning and reduced the scope of the suggested exercises allowing more students to come to their own working solutions; these labs were all ranked higher by the students in terms of value. For the Day 4 lab, one student remarked in the daily survey that, “The evening lab was extremely helpful. All the instructors were willing to sit down and help and work through anything with us.” [4] Instructors also noticed that during lab time, discussions between students were more frequent and collaborative, and one student mentioned in an exit survey that, “It would have been nice if sometime during the institute, like lab, we broke off into groups based on what would be used in our projects and go more into depth on those topics.” [4] Throughout the institute, labs served a role of encouraging group collaboration and problem solving with structured activities designed to solidify understanding.

Figure 4. Time to Science v. Number of Cores

V. PROJECTS As part of the BWSIP program, students have been working

this past summer on what will be year-long projects matched with faculty mentors to gain real experience involving hands-on use of HPC resources to solve problems in the sciences. Faculty mentors were encouraged to apply for education allocations on the Blue Waters supercomputer to enable students to utilize a compute resource with many cores per node, many nodes, large memory, GPGPUs, and with a familiar system and workflow used during the two-week institute. The research projects span the sciences in content, including, but not limited to, biology (virtual screening of disease-curing molecules), physics (acoustic simulation), astronomy (reducing and analyzing astronomical data sets), computer science (minimizing HPC network allocation fragmentation), and petascale education (fixing and updating petascale curriculum modules). Students worked full-time during the summer months following the institute and were awarded a summer stipend; they will continue to work for two subsequent semesters at roughly eight hours per week, with additional stipends, and are expected to produce publications of their work at the project’s conclusion along with a poster for the Blue Waters research Symposium in Spring 2015.

Preliminary evidence suggests that topics covered during the two-week institute have helped students be successful during their projects. Most (75%) of the students responded to a “snap” set of questions sent by the instructors at or near the end of the summer portion of their internship, and many listed the core topics of MPI, OpenMP, CUDA and computational thinking as being particularly useful to them. Also listed among the most helpful lessons were parallel thinking, optimization, and visualization. Some students who responded to the questions indicated that there were topics not covered during the institute that they needed to know to help them in their projects, but none of the them indicated they were unable to learn these topics during the course of their internship or did not know them beforehand. In subsequent years, BWSIP staff will evaluate projects in advance to ensure that needed skills are addressed during the institute, or that a learning plan is developed for each student.

As an example of a project experience, four of the students returned with the Shodor instructors at the end of the two-week institute to begin projects at Shodor involving verification and

validation of parallel code, fixing bugs, modifying code to be optimized for Blue Waters, and scaling curriculum modules used during the institute along with modules developed for parallel systems in general, but not specifically Blue Waters [5]. The students spent the summer working collaboratively and successfully applying knowledge gained during the institute to achieve the goals of the projects. For example, the students were able to utilize knowledge of MPI and OpenMP to find and fix bugs in the curriculum modules’ parallel codes. As another example, the students applied what they had learned about batch job submission to run the codes on three different supercomputers (Blue Waters and XSEDE’s Gordon and Stampede clusters), and they employed skills in measuring performance to generate “Time to Science” curves for each module for different problem sizes and core counts. Although the Shodor staff were on hand to assist the interns, the students were able to accomplish most of their work by helping each other and applying techniques and collaborative thinking and analysis skills learned and practiced during the two-week institute.

VI. FORMATIVE/SUMMATIVE ASSESSMENT EVALUATION SUMMARY

Throughout the year, student interns will work with their mentors on their home campuses or at other campuses and are expected to present their projects and results at the Blue Waters Symposium 2015. In addition, each month, students are required to submit a monthly progress report on their projects stating their accomplishments, issues and upcoming milestones.

The evaluation of the Blue Waters Student Internship Program is an ongoing process embedded into all aspects of the program directed towards acquiring PDC and HPC skills and knowledge. For the evaluation, the Shodor instructors worked closely with the evaluation group to perform both formative and summative assessments of the program. Evaluation methods required each intern to complete pre- and post-institute surveys and daily reflections/evaluation questions as well as participate in a focus group. The daily reflections/evaluation process helped the program instructors continually improve the effectiveness of the program’s structure and curriculum during the institute. Both the interns and participating faculty mentors were given an online-based pre-survey about their prior experience and expectations regarding the internship program. The students were also invited daily via email to participate in online-based daily surveys in order to complete session reviews and to provide feedback. During the second week, the students were divided into three different groups for 90-minute focus group interviews to address and share their overall experience about the institute. On the last day of the institute, students responded to the exit survey that provided them an opportunity to share their perception of overall results, program impact, and future concerns about the institute.

The following are summaries of the findings of the two-week institute from “The Evaluation of the Blue Waters Community Engagement Program” [4]:

1. Respondents generally reported high levels of overall satisfaction with the institute experience. All (100%) respondents agreed or strongly agreed in the exit survey that, “My goals for attending the two week training institute were achieved,” and, “Overall, I would rate this experience as successful.” In addition, each daily session survey showed high levels of overall satisfaction with each day.

2. Respondents generally reported high levels of

satisfaction with the communication between instructors and the participants. Overall, the majority of respondents were satisfied with the communication between instructors. Also, in the open-ended questions and other comments at the focus group interviews, respondents mentioned that because they felt that the instructors tried to listen to them and help them, they were able to provide more feedback.

3. The majority of the participants agreed that the institute

helped them in developing their technical skills. In the pre survey, 100% of respondents agreed or strongly agreed with, “I expect to apply the knowledge/skills I gain during this internship to my work/research.” Also, 95% of the respondents said the part to which they were most ‘looking forward’ during this internship was learning technical skills. In the exit survey, most participants also reported that this institute helped them to develop their technical skills.

4. Participants noted that the most useful thing they had

learned in the institute was the overall concept of parallel computing. Over 60% of the respondents noted that the most interesting and useful things that they learned during the institute were the conceptual thinking and actual programming of parallel computing. Even with differences in background knowledge and with varying projects, participants indicated the overall concept of parallel computing helped them to understand of the need for supercomputing.

5. All respondents agreed or strongly agreed that they had a

better understanding of both supercomputing and Blue Waters as a result of the institute experience. In the pre survey, 90% of the respondents agreed or strongly agreed that they wanted to learn more about supercomputing, and in the post survey 100% of respondents reported they had a better understanding of both Blue Waters and supercomputing as a result of the experience.

The post-institute survey asked Blue Waters and SRE students

to “indicate the degree to which [they] agree or disagree with the following statements,” some of which appear in Table I. Every Blue Waters Intern and some of the SRE students responded. These questions were designed to measure the institute’s effectiveness in developing students’ technical skills and preparing them for HPC research.

To a great extent, the evaluation data demonstrates the value

added and the measurability of the institute as it assessed the extent to which the appropriate structure and meaningful content effectively developed students skills and knowledge in the areas of PDC and HPC. The I-STEM group at the University of Illinois will continue to evaluate and improve the program’s structure and curriculum.

TABLE I. SELECT POST-INSTITUTE SURVEY QUESTIONS [4]

Statement Strongly Disagree Disagree Neutral Agree Strongly

Agree

a 0 0 0 12 11

b 0 0 0 8 15

c 0 0 1 12 10

d 0 0 1 11 11

e 0 0 1 5 17

f 0 0 0 7 16

a. “My goals for attending the two week training institute were achieved.”

b. “I am satisfied with the interaction with instructors during this institute.” c. “I know the next steps for me to proceed with my project.” d. “I know the next steps for me to build on what I learned.” e. “This institute helped me to develop my technical skills.”

f. “Overall, I would rate this experience as successful.”

VII. FUTURE IMPROVEMENTS AND EXTENSIONS There are many suggestions that were made during the exit

survey and focus groups that need to be considered for future years. One of the main suggestions given by many students was to have the institute last longer than two weeks with shorter teaching days. Taking this approach would allow students to have more time to do evening readings and practice on topics that were taught during the day. The Shodor instructors have since discussed the issue and noted there are many logistical, financial, and technical concerns that need to be addressed and resolved in order to increase the duration of the institute. With a longer institute, the interns would have less time working on their projects with their mentors over the summer, and the cost for room and board would be higher. However, some topics could be addressed in other modes to provide more student-student interaction.

The surveys have helped the instructors create better documentation, tutorials, and teaching curriculum for the future institutes. The Shodor instructors are working on a clear list of prerequisites for next year’s institute, such as pre-installation of necessary software, Linux/Unix experience, and C programming experience. The team is also creating tutorials, documentation, and web-based resources in order to enhance student-learning experiences before the institute.

Another major change the instructors are considering is to develop new teaching materials, examples, exercises, and solutions prior to the institute that are more specific to the projects each year that students will work on. This way there will be more time to test all the examples and exercises that are appropriate for the particular topic, make sure they are working correctly, and perhaps provide serial versions of code to the interns prior to the institute. Providing the serial version of code before institute could help students understand the algorithm and problem in advance, so during the institute the limited time could be used to focus more on the actual parallel and distributed computing as it applies to the model.

A number of students made clear that they had gotten more out of the two-weeks spent exploring PDC/HPC in the context of petascale computing enabled by Blue Waters than out of some semester-long courses in parallel computing or even their introduction to programming classes. Consequently, working with

both Blue Waters and XSEDE funding, Shodor will be looking to provide all institute materials in such a form on-line that faculty from any college or university could use the content and approach as components of their own courses. Individual students, if motivated, could use these same curricular materials self-paced instruction and parallel computing exploration.

VIII. CONCLUSION The Blue Waters Student Internship Program provided a

structured, engaging and hands-on training environment for teaching a wide range of topics in PDC and HPC. During the Petascale Institute, students learned skills such as OpenMP, MPI, CUDA, optimization, visualization and Parallel Thinking. The structure of the Petascale institute included dynamic lectures, hands-on activities and lab explorations.

Students who participated in the BWSIP training acquired skills and knowledge that enable them to have competence and confidence in PDC and HPC, with preliminary evidence from the end of their summer work period showing the value of the Petascale Institute content and approach.

ACKNOWLEDGMENT The authors would like to thank the staffs of the Blue Waters

Project, the National Petascale Computing Facility, and the National Center for Supercomputing Applications, all of the University of Illinois at Urbana-Champaign, without whose help and support this work would not have been possible.

This research is part of community engagement for the Blue Waters sustained-petascale computing project, which is supported by the National Science Foundation (awards OCI-0725070 and

ACI-1238993) and the state of Illinois. Blue Waters is a joint effort of the University of Illinois at Urbana-Champaign and its National Center for Supercomputing Applications. This work also used the eXtreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1053575.

REFERENCES [1] S. C. Glotzer, B. Panoff, and S. Lathrop, “Challenges and

opportunities in preparing students for petascale computational science and engineering,” Computing in Science and Engineering, vol. 11, issue 5, Sep. 2009, pp. 22-27, doi:10.1109/MCSE.2009.134.

[2] J. K. Houchins, J. D. Krause, S. A. Lathrop, and R. M. Panoff, “A multi-cultural success story for achieving diversity in multi-core/many-core internships,” Proc. 2011 TeraGrid Conf.: Extreme Digital Discovery (TG '11). ACM, 2011, Article 51, 1 page, doi:10.1145/2016741.2016796.

[3] A. F. Gibbon, D. A. Joiner, H. Neeman, C. Peck, and S. Thompson, “Teaching high performance computing to undergraduate faculty and undergraduate students,” Proc. 2010 TeraGrid Conf. (TG '10). ACM, 2010, Article 7, 7 pages, doi:10.1145/1838574.1838581.

[4] L. DeStefano, J. S. Sung, “The evaluation of the Blue Waters Community Engagement Program: Petascale Institute report,” I-STEM Education Initiative, University of Illinois at Urbana-Champaign, unpublished.

[5] “BWSIP: undergraduate petascale modules,” 2012, retrieved Aug. 27, 2014, from: http://shodor.org/petascale/materials/modules/.