CALIFORNIA STATE UNIVERSITYMONTEREY BAY

PIKEMINNOW REWARD EDUCATION PROJECT

CAPSTONE Report

Submitted in partial satisfaction of requirements of the degree of

MASTER OF SCIENCE in

Instructional Science and Technology

Michael J. Anderson

December 2, 2017

Capstone Approvals: (At least one advisor and capstone instructor should approve)

_________________________  ___________________________ _____________Advisor Name                                 Signature                                       Date

_________________________  ___________________________ _____________Capstone Instructor Name              Signature                                       Date

1

Table of ContentsExecutive Summary ……………………………………………………………………………... 4

Introduction ……………………………………………………………………………………… 5

Background …………………………………………………………………………………... 6

Problem ……………………………………………………………………………................. 6

Learner Analysis ……………………………………………………………………………... 8

Performance Levels ……………………………………………………………………......… 9

Desired Outcomes Interventions ….…………………………………….……………......…. 10

Environmental Scan ……………………………………………………………………….... 10

Solution Description ………………………………………………………………………….... 12

Proposed Solutions ……………………………………………………………………….… 12

Goals and Objectives ……………………………………………………………….………. 12

Major Instructional Strategies ………………………………………………………………. 14

Content Analysis …………….……………………………………………………………… 17

Task Analysis …………………………………………………………………………….…. 21

Media Components …………………………………………………………………………. 22

Challenges and Breakthroughs ……………………………………………………………... 23

Methods and Procedure ……………………………………………………………………….... 24

Major Deliverables ……………………………………………………………………….…. 24

Remaining Process for Completing PREP ……………………………………………….…. 28

Resources …………………………………………………….……………………………….... 28

Timeline …………………………………………………………………………………...…… 30

Evaluation and Testing ………………………………………………………………………… 31

Conclusion ……………………………………………………………………………………... 34

References ……………………………………………………………………………………... 36

2

Appendices, Figures, and TablesFigure 1: General Checklist .…………………………………………………………………… 17

Figure 2: Daily Fishing Checklist ….…………………………………………………..………. 18

Figure 3: Voucher Payment Checklist …………………………………………………………. 19

Figure 4: Special-Tag Vouchers Checklist …………………………………………………….. 20

Figure 5: Slide Description ………………………………………………………….…….... 24-27

Table 1: Timeline Milestones ………………………………………………………………. 29-30

Figure 6: Revision Request Document ……………………………………………………….... 30

Figure 7: Narration Draft Revisions …………………………………………………………… 31

Appendix A: Questions for Usability Test ………………………………………………… 37-38

Appendix B: Questions for Learning Test ……………………………………………….…….. 39

Appendix C: Responses from the Learning Test …………………………………………… 40-41

Appendix D: Descriptive Statistics …………………………………………………………….. 42

Appendix E: t-Test Results ………………………………………………………………..…… 43

3

Executive SummaryThe Pikeminnow Reward Education Project (PREP) was conceived to enhance the

efficacy of the Northern Pikeminnow Sport Reward Program (NPSRP). Specifically, PREP

aimed to make the NPSRP content more accessible and effective at preparing new participants to

begin participating in the program at a critical confidence level. PREP was charged with

reorganizing the content to be more concise, easy to follow, and interactive than existing content

on pikeminnow.org. The PREP Team decided that an interactive online Participant Guide (PG)

would best support the roughly 500 new annual participants in the recent history of the NPSRP,

since the vast majority of participant errors are made by new and under-informed participants.

PREP aims to increase participation in the NPSRP by making the orientation process

easier and clearer for new participants who wish to join the sport reward fishery. The PREP PG

is designed to address participation errors that have persisted over the history of the NPSRP.

Four job-aids in the form of checklists were designed to provide a more manageable set of

displays for participant orientation. PREP presents the priority information, displayed in the

checklists in the PG, as a vehicle to help prospective participants more efficiently and effectively

reach the critical confidence level for successful participation during the NPSRP season.

The PREP team collaborated mostly asynchronously during the development process.

Once the team agreed-upon a need, individual contributions were made to the shared drive by the

team member best positioned to provide the content. All contributions were visible for review

and each task was given a window of operation during which input was considered. After a task

was proposed to organize the content provided by the team, a timeline was provided. Over the

course of PREP, the evaluation cycle occurred regularly. Revision requests were managed via

shared documents in the form of interactive tables. A stakeholder evaluation and learner test

were conducted. The results from the learner test showed evidence that the PG will likely lead to

new NPSRP participants reaching the critical confidence-level desired by the PREP Team.

Increased participation and replacement of sporadic participants remains critical to the

operational objectives of the NRSRP. Importantly, more successful participation by new

participants is equally essential to the future viability and consistency of the NPSRP. Continued

revisions of pikeminnow.org and the public outreach efforts shall remain a priority component of

the NPSRP into the future. The new season of the NPSRP opens on May 1, 2018 and PREP aims

to have reached its audience before then and into the season as new participants join over time.

4

Introduction and BackgroundBackground:

Over the history of the NPSRP, engagement with the public has had to evolve. Initially,

the program was commonly known as a “bounty” for “Squawfish.” Nine years into the NPSRP,

the name “Squawfish” was formally replaced with Northern Pikeminnow due to cultural

sensitivity concern over the use of “squaw” in conjunction with the fish at the request of the

Yakima Native American Tribe. Additionally, recent years have seen the NPSRP focus on the

“sport-reward” term to describe the management of Northern Pikeminnow populations on the

main-stem Columbia River in lieu of “bounty.” The term “bounty” gave many the incorrect

impression that the NPSRP aimed to eradicate the species from the Columbia River. As a native

species, the Northern Pikeminnow is welcome in the Columbia River. However, the NPSRP is

charged with keeping the Northern Pikeminnow populations from ballooning out of control due

to the environmental advantages provided by the hydro-electric dams and consequent changes in

river characteristics (such as drastically more “slack water” in which Pikeminnow prefer to live)

on the Columbia and Snake Rivers.

The Bonneville Power Administration has accepted financial sponsorship of the NPSRP

as an ongoing measure to mitigate the effects of the hydro-electric dams on the salmonid runs in

the Columbia River system. The Pacific States Marine Fisheries Commission (PSMFC) which

operates fisheries programs in five western states (Alaska, Washington, Idaho, Oregon, and

California), manages the NPSRP and pikeminnow.org. The Oregon Department of Fish and

Wildlife (ODFW) partners with the PSMFC as part of the NPSRP to monitor Northern

Pikeminnow population statistics overall and as a result of the NPSRP. The Washington

Department of Fish and Wildlife (WDFW) partners with the PSMFC as part of the NPSRP to do

the actual collection and processing of the Northern Pikeminnow submitted during the season,

including distribution of payment vouchers directly to the anglers.

The NPSRP is a rare collaboration between a large regional stewardship commission,

neighboring state departments of fish and wildlife, and a large inter-state public utility.

Significantly unique among the characteristics of the NPSRP is that it depends upon well-

informed and motivated private citizens to catch the fish according to the regulations. While a

fishing license is required for participants to submit Pikeminnow for the sport reward, the

prospect of earning back the cost of the fishing license, supplies, incidentals, and more is the

5

primary motivation for participation in the program. Successful NPSRP anglers are classified

according to performance levels over the course of a season. Tier 1 anglers submit between one

and 25 fish for the season, earning $5.00 per qualifying fish. Tier 2 anglers submit between 26

and 200 fish, earning $6.00 per fish. Tier 3 anglers must submit at least 201 qualifying fish and

earn $8.00 per fish. Seasoned participants, classified as Tier 2 and Tier 3 in the NPSRP, tend to

perform much better and consequently earn more money than new participants (Williams,

Winther, & Barr, 2016, p. 27). If one of the top performers takes a year off or retires, it could

take scores, if not hundreds of new participants to take up the slack. Keeping participation within

range of the target of an estimated 10-20% reduction in Pikeminnow population in the Columbia

River system each year is essential to give a significant enough population of juvenile salmonids

a chance to reach the Pacific Ocean. That 10-20% reduction in Pikeminnow population each year

is estimated to prevent loss of juvenile salmonids by 35-40% each year (Williams, Winther, &

Barr, 2016, p. 7). Therefore, having enough participation to reach the Pikeminnow reductions are

essential to achieving the desired population support for the native salmonid runs.

Problem:Like so many life-long Pacific Northwest residents, I had heard about the ‘Squawfish

bounty.’ In an attempt to learn more about how to earn the ‘bounty,’ I encountered

pikeminnow.org from a search engine result list. At this point, my experience authentically

mirrors a typical visitor to pikeminnow.org as a prospective participant. Initially, I felt confused

that search results for ‘squawfish bounty’ yielded results that included both ‘squawfish’ and

‘pikeminnow’ in the abstracts. As recently as 2015, news articles describing the NPSRP still

included ‘squawfish’ in the story, though they usually also mention ‘pikeminnow’ as well.

Furthermore, the continued term ‘bounty’ has persisted in published news reports up to and

including 2017 (Nesbit, 2015). The PREP Team made it clear early and often that they need

exclusive use of ‘pikeminnow’ to identify the fish and ‘sport reward’ to describe the money that

anglers make as a part of the NPSRP.

When I finally reached pikeminnow.org, I tried navigating through the menu items and

explored the links hosted on the site to find-out more details on the program. A number of

problems became obvious as I navigated through the content on pikeminnow.org. Firstly, it took

hours to review the information available not including the studies and ecological reports that are

published and chronicled on pikeminnow.org. Secondly, a series of videos appeared on different

6

tabs and sub-menu items and had playtimes that amounted to more hours of passive viewing.

Passive viewing is a problem because a prospective participant has no way to engage in an

interactive experience on pikeminnow.org. Additionally, there was no opportunity for someone

to check understanding with confirmative or corrective feedback and no indication of when a

baseline level of understanding had been achieved. Thirdly, there was no one place to go on

pikeminnow.org that would prepare prospective participants to effectively and efficiently

become actual participants.

Upon sharing these experiences with the NPSRP program manager at PSMFC, he began

the process of assembling a team of stakeholders from the partner organizations. At this point, I

was able to encounter the ‘problems’ that the employees were having with uninformed or

misinformed participants in the NPSRP. The PREP Team member representing the WDFW

indicated that anglers at the registration stations: still attempt to submit fish that are not Northern

Pikeminnow, try to submit Pikeminnow from outside of the program area, try to submit fish at a

registration station in which no evidence of prior registration was present, attempt to submit fish

that are not in acceptable condition, and attempt to submit Pikeminnow that are smaller than the

minimum size. The stakeholders from the PSMFC who are responsible for overseeing payment

at the end of the NPSRP season, indicated problems related to compliance and understanding of

the payment process. The first problem is that a flood of calls usually come into the hotline

around the end of the season and for some the calls come too late to have their payment

documentation processed at all. The second problem is that many new participants presume that

the voucher submission is due according to postmark, end of calendar year, or end of the tax-

preparation season leading to frustrated participants who sometimes are not paid due to delaying

submission of their payment documentation. After the 2016 NPSRP season, 26.7% of the anglers

in the NPSRP who received payment vouchers failed to successfully process their vouchers,

resulting in no payment for the sport-reward (Williams, Winther, & Barr, 2016, p. 27).

The PREP Team decided to weigh the options for how to go about improving the

educational outreach of the NPSRP aimed at improving participant performance. One option was

to redo the videos with much shorter playtimes resulting from concise storyboarding. A second

option was to create an interactive online learning module (in which all PREP Team members

had experienced as a part of their job) and host the PG on pikeminnow.org. The consensus of the

PREP Team was to create an online module in the form of a PG to walk prospective participants

7

through the process of becoming a productive NPSRP participant. The PREP Team valued the

interactive element as the difference in effectively addressing the performance gaps of some of

the participants. The PREP Team hypothesized that most of the tier-2 and tier-3 participants

were brought into the program by word of mouth, whereby a person coached the prospective

participant through the processes associated with becoming a paid participant. These word of

mouth participants were likely to have the misconceptions straightened out by a veteran (tier-2 or

tier-3) NPSRP participant or an NPSRP staff member. The format of an online, interactive PG

would serve the same role as a veteran NPSRP angler walking new participants through the vast

information available on pikeminnow.org for the purpose of reaching a critical confidence-level.

Learner Analysis:The learners for the NPSRP PG are primarily local residents of communities along the

course of the Columbia and lower Snake Rivers. The metropolitan area of Portland, Oregon

provides many participants at several registration station locations near Portland, Oregon and

Vancouver, Washington. The only other significant population center in the NPSRP zone is the

Tri-Cities of Richland, Kennewick, and Pasco in southern Washington State. Pikeminnow are

caught in large numbers in the Columbia River Gorge, which includes towns in Oregon and

Washington that are primarily tourist- and/or timber-based economies. Lewiston, Idaho and its

neighboring city (Clarkston, WA) across the Snake River in Washington State represent the only

population center along the lower Snake River that is inside the NPSRP zone. The remainder of

the NPSRP zone consists of very small towns, farmland, and other rural areas. For Tier 2 and

Tier 3 anglers, the seasonal NPSRP earnings can be a significant portion of total annual income.

The potential users of the NPSRP is any person with access to a fishing pole and

proximity to one of the registration stations. The ages of the participants could be children young

enough to be exempt from purchasing a fishing license: under 14 years of age in the State of

Washington and under 11 years of age in the State of Oregon. While children are unlikely to earn

significant sport reward money from the NPSRP, some of the adult participants enjoy teaching

their children and/or grandchildren how to fish while participating in the sport-reward fishery.

The most successful NPSRP anglers fish from their own boat. The vast majority of

Pikeminnow submitted for the sport-reward were caught from a boat as opposed to from shore or

some other permanent structure such as a pier, dam, or dock. News reports indicate that these

most successful Pikeminnow anglers spend long days and even nights on their boat during some

8

of the NSPRP season (May through September) and consider it a job. In 2016, the top ten earners

made over $30,000 during the five-month season, each reeling in 4,000 Pikeminnow or more per

angler. These top-earning anglers are not the target audience for the NPSRP PG. However, the

NPSRP administrators believe that supporting new participants to becoming more proficient and

prolific in their Pikeminnow catch is in the best-interests of the stability of the program.

Performance Levels:In order to fully understand the learners, it is important to understand the payment

structure for NPSRP participation. Participants are categorized into three groups: Tier 1, Tier 2,

and Tier 3. The tiered system was first implemented in 1995 and updated for the 2014 NPSRP

season. On average, around 60% of the NPSRP anglers are Tier 1 and earn about 5% of the

sport-reward excluding the special-tag vouchers, which are worth $500.00 per qualifying fish.

On average, since the recent revision to the tiered system for the sport-reward, around 25% of the

NPSRP anglers are in the Tier 2 category and earn about 20% of the sport-reward. That leaves

15% of NPSRP anglers that reach Tier 3 and earn about 75% of the sport-reward excluding

special-tag vouchers (Williams, Winther, & Barr, 2016, p. 44).

It is unlikely for new participants to reach Tier 2 during their first NPSRP season and

even less likely to achieve Tier 3 status. Therefore, it is a process to build a larger base of Tier 2

and Tier 3 NPSRP anglers. If over 25% of NPSRP participants from 2016 alone failed to

successfully receive payment for the fish that were indicated on payment vouchers distributed at

the registration stations, then it could indicate that many of the new participants (Tier 1) did not

reach a critical confidence-level during their preparations to participate in the program. However,

the exit interviews do not inquire as to why so many qualifying participants did not successfully

complete the payment requirements.

Without question though, when 25% of the participants who received a payment voucher

each year fail to collect the sport-reward in which they are entitled, it represents a problem. It

could be that some of the participants did not consider it worth their time to send in a voucher for

one or just a few fish. In these instances, then perhaps these participants need more effective

educational support to become more prolific Pikeminnow anglers. It could be that some of the

participants simply did not dedicate enough time to catch enough fish to motivate them to

process the payment. In this case, these anglers will likely never reach the desired Tier 2 or Tier

3 levels because of availability or motivation. If a bad experience with the payment process

9

(which clearly affects many participants each year) causes participants to turn away from the

NPSRP, then it could have a significant negative impact on participants’ progression toward the

preferred Tier 2 and Tier 3 proficiency levels.

Desired Outcome Interventions:The NPSRP PG was designed to take the information that exists as content hosted on

pikeminnow.org including links that lead to information about fish and game laws and provide a

more concise pathway to reaching a critical confidence level for prospective participants.

Additionally, the NPSRP PG was designed to provide a measure of accountability in the form of

interactive multimedia that target the participation errors that arise either at the registration

stations or as part of the payment process. Since the vast majority of participation errors occur

among Tier 1 (including prospective participants) anglers, the NPSRP PG was designed

primarily to take learners from minimal knowledge of the participation guidelines to a critical

confidence-level. Results of the NPSRP PG effectiveness will not be known until the end of the

2018 sport-reward season when observations from the technicians at the registration stations and

voucher payment success rates have been made and reported to the PSMFC.

At every step along the way, the PREP team remained accountable by consensus to

isolate only the information that a prospective participant would need to know in order to reach a

critical confidence-level for successful participation. With guidance from The Checklist

Manifesto: How to Get Things Done Right (Gawande, 2009), these critical facts, practices, and

understandings were translated into four checklists. These checklists serve as job-aids and meet

the first objective of making the content as concise as possible and more portable that a website

that requires browser navigation. Since these checklists formed the framework for the interactive,

online PG, the multimedia principle of “Segmenting” further required that the checklisted

content fall into more manageable categories. The result was a “General Checklist,” a “Daily

Checklist,” a “Payment Process Checklist,” and a “Special Tag Voucher Checklist.”

Furthermore, the “Pre-Training Principle” informed the decision to post the checklists above the

button for the NPSRP PG and instructions to download and review them before beginning the

PG. Further description of how the multimedia principles were applied will be discussed in the

Solution Description section of this report.

Environmental Scan:

10

The most obvious place to become familiar with the NPSRP is pikeminnow.org. Though

many organizations partner with the PSMFC, which administers pikeminnow.org, most of the

content has been written by biologists and technicians. In the 2016 Annual Report (posted on

pikeminnow.org with the other annual reports) “Recommendations” section, the second

recommendation alluded to the need for the content on the site to respond more effectively to

educational concerns: “d) Investigate use of internet and social media for advertising NPSRF

[Fishery] and for angler recruitment and education” (p. 34). Also, the recommendation section

mentioned educational ambitions via new video efforts: “c) Continue to develop video content

for use in improving angler education, NPMP awareness” (p. 34). While updated video content

should increase awareness of the NPSRP, the priority is to improve the education of prospective

participants so that they can become effective participants in the future of the NPSRP.

Since the nature of recruitment and training of NPSRP participants is necessarily de-

localized, it makes sense that the solutions would be web-based, easily accessible, and

theoretically sound. Therefore, the 12 Principles of Multimedia Learning (n.d.) were honored

while developing the PG. Albert Bandura’s Social Learning Theory (n.d.) also gave purpose to

the PG as a guide, whereby desired behaviors are modeled as authentically and clearly as

possible. For its structure though, the PG has to be able to improve performance of the

prospective participants as they prepare to work through the Tier 1 phase. Robert Gagné’s Nine

Events of Instruction (2017) were employed to ensure that the critical competencies designated

by the PREP Team were given the best chance to translate into desired behavior as a result of

completing the PG. Merrill’s First Principles of Instruction (2002) were also considered,

primarily by way of encouraging the learners to first preview the four checklists before

beginning the PG, which will reference the checklist content. All of the questions and

comprehension checks within the PG were framed as going through problem-solving processes,

which apply to new participants as they ritualize the actions and mindsets that lead to successful

participation in the NPSRP.

Apart from the theoreticians previously mentioned, published news articles on the

NPSRP provided insight into how the program had historically been advertised and presented to

the public in the Columbia River region. Some of the articles used words like “bounty” and

“squawfish” that the NPSRP has stopped using. Usually, the articles will mention the money as

the motivating factor for participants in the NPSRP. Still other articles discussed the purpose of

11

the program (improving native salmonid recovery efforts) in concert with the extrinsic financial

motivation of prolific Pikeminnow anglers. There really is not another program quite like the

NPSRP because the Northern Pikeminnow is a native species and the goal is not to eradicate the

species from the Columbia River system; most other ‘pay for submitting fish’ programs are

aimed at combatting invasive species. As a result, the lesson learned from investigating the

published articles, is that it is critical to represent all new educational and outreach products in a

way that clearly represents the values and mission of the NPSRP so as to focus the message on

the realities and not the myths.

Solution DescriptionProposed Solutions:

The overall solution for meeting the educational and outreach needs of the NPSRP go

beyond the PG. However, the PG is the best place to start because it includes the participant job-

aids that focus attention on the essential program regulations and guidelines. Specifically, the PG

takes prospective participants from a presumed unconfident-level of understanding about

program details and leads the learners through a pathway that supports the concepts of

participating in the NPSRP by focusing on the behaviors that are required of responsible and

successful participation.

The information that participants need has always been available on the website, within

the brochures, and discussed in conversations and presentations at fishing shows. The solution

proposed by the PREP Team in the form of the NPSRP PG is to make that information

accessible as briefly (concisely) and effectively as possible. The fact that this program is entirely

voluntary, means that the learners are going to lose interest and/or focus if the educational

component does not employ the principles of multimedia learning in as orderly a manner as

possible. The PG audio-visual presentation and the checklists that drive the content of the PG

respect the voluntary nature of participation in the program. The interactivity aspects of the PG

work to both maintain the interest of the learners and provide confirmation/support for proving

that the learners are clear on the essential program expectations as designated by the PREP

Team.

Goals and Objectives:

12

The goals for PREP include improved perceptions of the NPSRP by prospective

participants and error reductions by active participants resulting from educational outreach

methods that create better-informed participants. It is important for prospective participants in

the NPSRP, whether they actually catch Pikeminnow or not, to understand that the purpose of

the program is to mitigate salmonid vulnerabilities and not to eradicate the Northern Pikeminnow

from the Columbia River system. The PG was designed to clearly establish that the primary goal

is salmonid restoration. In the future, the media outreach to the public and prospective

participants will be able to build upon the tone of the PG as they recommend interested persons

to access the PG when visiting pikeminnow.org.

A more concrete goal of PREP is to impact behaviors of the participants. Firstly, the

NPSRP would like more participants to engage with the sport-reward fishery. Secondly, the

NPSRP would like to report reduced numbers of participants attempting to submit fish that do

not meet the qualifications of: physical condition, stream of origin, size, and species. Thirdly,

the PSMFC and ODFW aims to have fewer errors in payment voucher processing and other

required documents. While no numerical goal was set by PSMFC, a significant reduction in the

number and/or rate of payment vouchers that go unclaimed is a goal after implementation of the

NPSRP PG. A second part to the third goal is that the payment hotline will receive fewer

questions and complaints about the voucher payment process because the PG and participant job-

aids would have sufficiently promoted compliance behaviors prior to any processing errors

occurring.

The learning objectives of the NPSRP PG are:

1) To guide NPSRP participants through the major program requirements.

2) To guide NPSRP participants through a typical day of fishing during the season.

3) To ensure that NPSRP participants can distinguish the Northern Pikeminnow from other

similar fish species. And,

4) To guide NPSRP participants through the process of applying for and receiving payment

for participation in the program.

Objectives 1, 2, and 4 are directly supported by the checklist job aids. Objectives 1 and 2 each

have their own job aid and objective 4 has two partially redundant but distinct checklists because

there are two different kinds of payment vouchers that are processed differently from one

13

another. All of the four objectives are subject to the multimedia principles of learning within the

PG including comprehension checks, feedback, and confirmation of correct behavior.

Major Instructional Strategies:Since the majority of instructional design decisions were responsive to the format of the

PG along with its component checklists, Richard E. Mayer’s 12 Principles of Multimedia

Learning (n.d.) was closely considered. The table below shows how each principle was

addressed in the NPSRP PG:

Name of Multimedia Principle Application Description within the NPSRP PG

1) Coherence Principle The Checklists were as concise as possible in number and wording. To establish an expectation that the entire checklist is not to be read every time that it appears, the agent was superimposed over a portion of the writing. Additionally, no sounds or extraneous imagery was included in the slides.

2) Signaling Principle Highlight boxes used on their own or with greyed-out areas focused the learner. Also, zoom areas were consistently employed to draw the eye.

3) Redundancy Principle Again, the checklists in their entirety are not meant to be the focus of the eye during the PG; instead, the narration was meant to support the imagery and inflection designed to match visual signaling.

4) Spatial Contiguity

Principle

Learners will notice that the checklist image is in a consistent spot and that the images directly supporting the narration will appear to the right side of the slide and proportions are made consistent as well.

5) Temporal Contiguity

Principle

A left-side big-picture side of the slide is consistent from slide to slide and the supporting visuals appear on the right side of the slides that contain descriptions and/or explanations.

6) Segmenting Principle Each checklist has its own focus and is addressed one at a time, beginning with the most general and ending with the specific details of the payment process. Comprehension checks were deployed during the PG between segmented sections.

7) Pre-Training Principle The checklists are intended to be downloaded, printed-out (recommended), and read before going through the PG. One of the checklists is meant to be carried with the participant during fishing days and will be made available at registration stations.

8) Modality Principle Apart from a few instances of zooming and the reveal of learning objectives, there are no

14

distracting animations. Instead, the content is meant to be presented as images (graphics/schematics where available) supported by synchronized narration.

9) Multimedia Principle From the beginning, the pattern among the slides is to have narration synchronized with the images that most clearly represent the content from the checklists as the slides progress.

10) Personalization Principle The agent was selected for his casual yet presentable appearance and the narration criteria was: clear, friendly, and supportive in tone.

11) Voice Principle The narration was delivered in a voice that could have been one of the participants in a friendly tone and inflection reminiscent of a supportive coach.

12) Image Principle The agent is present to indicate expressions of reactions to ‘self-talk’ included in the PG, to point-out certain portions of an image, and to indicate that documentation is expected during a specific portion of a process described on the slide. All images are static and given enough time on screen.

The checklists were inspired by Atul Gawande’s book The Checklist Manifesto: How to

Get Things Done Right (2009). In the research for the book, Gawande discovered that when

surgeons use checklists to prepare for surgery, physician errors that lead to significant injury and

death almost completely disappear. One might presume that a surgeon would know everything

that he or she would need to know to perform the procedures that they have done many times

before. Yet the number of preventable deaths from improperly followed procedures by surgeons

were high for each hospital studied, until the checklists were accepted and practiced. Similarly,

one might presume that a functional adult would know to mail-in their payment vouchers

provided by NPSRP technicians at the registration stations, often accompanied by a self-

addressed envelope, within an allowable timeframe. Obviously, the 26% failure rate for standard

voucher payment is far too high. Therefore, a checklist for each of the payment-vouchers

processing-requirements was drafted and supported within the PG. Participants should replace

the factors that led to past failures with confirmative support while completing the PG.

Albert Bandura’s Social Learning Theory (n.d.) also informed the design of the NPSRP

PG. New participants to the sport-reward fishery want to be successful. Therefore, the tone of the

PG, as represented visually by the agent, models a successful Tier 3 angler. Although all three of

the principles that make up the theory were present in the PG, the second and third played the

15

most informative role in imagining the learner experience. Bandura’s second principle of social

learning is that “Individuals are more likely to adopt a modeled behavior if it results in outcomes

they value” (Social learning theory, n.d.). The most commonly cited reason for participating in

the NPSRP is to earn the sport-reward payments. For this reason, focus on successful payment

behaviors are highlighted for the learner as they progress through the PG. Bandura’s third

principle of social learning states that “Individuals are more likely to adopt a modeled behavior if

the model is similar to the observer and has admired status and the behavior has functional

value” (Social learning theory, n.d.). The agent in the PG looks as though he could step onto his

boat and begin fishing even though he is not dressed in gloves, lures, and carrying equipment.

The agent implies a Tier 3 NPSRP angler, which is what the prospective participants would aim

to be after becoming an active participant.

Finally, Robert Gagné’s Nine Events of Instruction (2017) were followed to create the

storyboard for the NPSRP PG. To gain attention (event 1), the PG welcomes the learners and

shows the impressive sponsorship of the NPSRP using .png icons of the organizations. To inform

learners of the objectives (event 2), the PG shows each of the four objectives appearing on a slide

one at a time and pauses until navigation control is acted upon by the learner. To stimulate recall

of prior learning (event 3), the PG maintains an image of the checklist being presented, which

should have been reviewed before beginning the PG. To present the content (event 4), only the

portion of the checklist focused upon at the moment is highlighted and described by the

narration. To provide learning guidance (event 5), the imagery is matched to the point on the

checklist highlighted and synced to the narration. To elicit performance (event 6), the learners

are posed with six true/false questions, two drag-and-drop exercises, and one multiple-choice

question overall, but only after the section in which the content was posed and before presenting

new points. To provide feedback (event 7), the multiple-choice and true/false questions provide

specific confirmative and corrective comments, depending upon their response. The drag-and-

drop exercises do not move on until completed successfully with confirmative comment support.

To assess performance (event 8), the learners must successfully complete the drag-and-drop

assessments before moving on to confirm understanding of size and species. The payment

process is assessed with true/false questions designed to highlight stumbling points and elucidate

correct behaviors. Finally, to enhance retention and transfer [on the job] (event 9), the learner is

16

encouraged to print-out the daily fishing checklist and all checklists will be made available at the

registration stations, so that they can be at-the-ready.

Content Analysis:Since the tasks are in part dependent-upon the content in the checklists, each of the

checklists will be analyzed in this section. Below each checklist, a content analysis will follow.

Overall, the checklists were constructed from a process whereby each of the PREP Team

members provided important content for prospective participants with which to understand and

comply while participating. After an exhaustive list of critical content was brainstormed, the

PREP Team challenged each of the items to ensure concision and pare down the list. After the

critical content was capped, the instructional designer applied the Segmenting Principal to make

the total quantity of critical content pieces within each like-group manageable for learners with

varied abilities and education levels. The result was one checklist for the “General NPSRP,”

another for the requirements that an NPSRP anger would encounter on a “Daily Fishing” trip,

and two checklists for the “Voucher Payment Process” since there is a distinct process and

requirements for the “Normal Vouchers” and the “Special Tag Vouchers.”

Figure 1: General Checklist

17

This “General Checklist” serves to clearly outline several key aspects of the NPSRP.

Many of these items show that there are specific restrictions to participate and most of them are

in direct response to participant errors observed by NPSRP staff. The first five items were

selected to go first to set the tone that the participants are responsible for an earnest standard of

compliance. Whereas the last three items, including the asterisked item, were listed last so as to

avoid distracting from the first five compliance items. Providing the payment figures first was

deemed a risk to maintaining the focus of the checklist, which is to prevent recurring participant

errors in the future. The information on the checklists was already available on the website, but

this checklist managed to arrange these eight priority items onto a single, concise checklist that

reduces the time that a prospective participant would need to encounter all of these items to a

few minutes from what might have taken over an hour navigating on pikeminnow.org.

Furthermore, this checklist provides an opportunity for the prospective participants to gain

confidence and frame a benchmark-level of understanding for active participation.

Figure 2: Daily Fishing Checklist

18

This “Daily Checklist” was focused to isolate the critical items that may involve a

personal interaction on an NPSRP fishing trip. The second item could have been assigned to the

“General Checklist” but was placed on the “Daily Checklist” because the technicians at the

registration stations who collect and process the fish would look for evidence of how the fish was

caught. Additionally, law enforcement can approach anglers and check for compliance at any

time. Worth noting on the Columbia River is that since it serves as a border between Oregon and

Washington over a long distance, “law enforcement” could include any of the following: US

Coast Guard, OR and WA State Police, the local county Sheriff, and potentially city police

departments. Additionally, since the dams on the Columbia and Snake Rivers are built and

maintained by the Army Corp of Engineers, military and Homeland Security personnel may

check on the anglers in certain areas.

Another theme of the “Daily Checklist” is that the items on the list represent efforts to

reduce frustration on the part of NPSRP participants who might not realize these nuanced

guidelines. This is the one checklist that NPSRP anglers are encouraged to keep on-hand on their

boat or otherwise in their possession while fishing as a reference. Since the registration stations

have such varied and limited hours of operation, the very first checklist item is for the angler to

pencil-in the hours for the one registration station in which the participant will interact for that

day. Misconception management was a priority for the PREP Team and the “Daily Checklist”

provides focused support to limit errors and avoid frustration from participants.

Figure 3: Voucher Checklist

19

Figure 4: Special-Tag Voucher Checklist

The information on these last two checklists (Figures 3 & 4) was historically available in

a particular area on pikeminnow.org. Here, the arrangement of items on two distinct checklists

provides confidence to NPSRP participants that they will be paid for their efforts during the

season. Given that in 2016, over a quarter of all vouchers provided to participants failed to yield

payment, these voucher payment checklists intend to guide participants through the payment

process(es) as clearly and concisely as possible. Collectively, these checklist items were arranged

in a logical sequence and phrased as though guided by coaching to build confidence that they

payment process is manageable.

Task Analysis:The tasks that are a part of the PG sequence take three different forms. The first task is

for participants to examine the image of registration station hours and respond to a question

which calls for interpretation of a hypothetical scenario that has caused problems for some

participants in the past. The question poses the situation and asks if the hypothetical scenario

falls within the constraints listed on the registration station hours image. If a learner responds

incorrectly, then corrective feedback is provided. If the learner responds correctly, then

confirmative feedback aims to build confidence. Whether the learner responds correctly or

incorrectly, this task highlights the reason that the first item on the “Daily Checklist” has a space

to pencil-in the operating hours for the registration station for the day of fishing, when a fishing

trip is planned.

20

The second form of task in the PG appears as true/false of multiple-choice questions that

check understanding of items on the checklist after those items have been narrated with

corresponding support images in the PG sequence. These six questions focus on checklist items

in which an interaction was deemed necessary by the PREP Team. All six of these questions

were set to have a correct response before continuing. If the learner responds incorrectly, then

corrective feedback will appear. Learners will then have an opportunity to select the correct

response. When the correct response is submitted, the confirmative feedback appears. The result

is a multi-level support response for the learners that requires interaction.

The third form of task in the PG includes two drag-and-drop exercises. These two tasks

use images of actual Northern Pikeminnow. In the case of the Species Identification drag-and-

drop activity, images of the species most commonly mistaken for Northern Pikeminnow are also

visible. In the first drag-and-drop, learners have to designate whether or not the fish shown are

large enough to qualify for the sport-reward. A ruler is provided and recollection of the minimum

nine-inch total length regulation is needed to measure each fish and drop into the “reward” box

or the “no reward” box. If the learner responds incorrectly, then the fish all return to their

original positions and the learner will have to try again until the sorting has been completed

successfully before moving on. No hints were provided as to how many, if any of the fish shown,

would qualify for the reward. The learner has to actually practice carefully measuring the fish

before dropping them in the correct box. This activity provides an important message that only

certain specimens will be accepted at the registration stations.

The last interaction is a drag-and-drop in which learners must apply the hints and

information from the previous slides on the similar species to Northern Pikeminnow and

distinguish the target species from the commonly mistaken species. Since bringing-in specimens

for submission that are not within NPSRP guidelines causes frustration and lowered-morale,

showing the learners how to build confidence in identifying the species as rapidly as possible

enhances motivation. The learners have to identify the Northern Pikeminnow specimens, drag-

and-drop them into the “reward” box and drag all of the non-Pikeminnow into the “no reward”

box in order to continue. If any mistakes are made, then the fish will all return to their original

positions and the learner will have to keep trying until they have correctly identified the species.

Upon successful completion of the species identification drag-and-drop, a confirmative feedback

comment provides additional support and builds confidence within the learner.

21

Media Components:In an effort to maintain consistency with pikeminnow.org and serve as an enhancement to

what already exists, all images used in the NPSRP PG were taken from pikeminnow.org except

for the dollar sign icon used in the two drag-and-drop exercises, a calendar that emphasizes

November 15th with a push pin, and a drawing of the word “October” along with fallen leaves

and pumpkins. Pikeminnow.org has all of the information that a participant would need to reach

Tier 2 or Tier 3 performance, provided that those individuals had sufficient opportunity to fish

during the season. The photo imagery was present already. The PG used those images and

supported them with in one of three ways. 1) the images were matched to the checklist item side-

by-side on the slides. 2) Enhanced with narration to point-out the correlation between the image

and the checklist item, sometimes with the aid of the agent. And, 3) Zooming, highlighting,

cropping, or other Photoshop functions were designed to signal the learner to the particular

portion of the image that is most significant.

In keeping with the 12 principles of multimedia learning, this PG intentionally excluded

animations, sound effects, and extraneous words and images. Conversely, a friendly-looking

agent was selected as a realistic portrayal of an actual NPSRP Tier 2 or Tier 3 angler. The

narration was selected to sound natural, clear, and friendly in a manner that suspends disbelief

that the narrator is not actually the man posing as the agent in the images.

The delivery format serves as an enhancement as well. The PG can be reviewed more

than once if learners want a review since it is easily accessible online. In the past, if learners

went to an informational session at an outdoor show or fishing store, then it would be difficult to

replicate that experience; especially when compared to simply clicking the PG button on

pikeminnow.org from any location at any time. Additionally, the PG provides an interactive

element that the website offerings simply did not accomplish, whether the textual or archived

video sections were visited. The interaction is key to providing an accountability that the learners

understand the critical program criteria, especially the information about the day of fishing and

payment process.

Challenges and Breakthroughs:The major challenge posing the PREP Team during the conception, design, and

development of the NPSRP PG was that the instructional designer had never personally met any

of the other PREP Team members before or during the process of development. Of course, this

22

challenge did not need to become an impediment. With a clear understanding that contact would

be limited between PREP Team members, communication and asynchronous participation

became critically important. To account for this challenge, the members of the PREP Team were

empowered to access the shared team Google drive at prep4pikeminnow and respond to specific

requests for contributions within a given time period. If clarification via groupware using the

shared documents did not suffice, then members could contact one another by phone

appointment and in one case with a scheduled virtual conference.

The challenge of limited access turned-out to be an asset if not a breakthrough. In fact,

the essential problem with pikeminnow.org as an educational tool is that there is simply too

much information available, in paragraph form, involving excess navigation. Given that we

aimed to make the process of building critical confidence among prospective participants as

concise and organized as possible via the NPSRP PG, it ended up helping that our

communication as a team also had to be concise and organized. We could have spent hours and

days discussing various aspects of the program to the delight of ourselves, but not to someone

just trying to earn some extra cash fishing for Pikeminnow. Instead, we had a mantra of “does

this absolutely need to be included?” and “does this require an interactive element in order to

build confidence among the prospective participants?” In the end, this challenge in the form of

time and access constraints made the path to building a useful product easier to discover.

Methods and ProceduresMajor Deliverables:

The first type of major deliverables are the checklists. Each of the checklists in described

in the “Content Analysis” section and shown as: Figure 1, Figure 2, Figure 3, and Figure 4.

These deliverables will be posted on pikeminnow.org above the button to enter the NPSRP PG.

Additionally, these checklists may be distributed on the pages of pikeminnow.org that pertain to

each one. The checklists will be posted as downloadable links, so that the learners can view them

full-size and print them out as resource guides as a participant during the season. The intent for

and recommendation of the learners interested in accessing the PG is to first download and

review the checklists before opening the PG.

The content and justification for the slides is indicated below:

Figure 5: Slide Description

23

Slide # and Title Contents of the Slide Purpose

1 - Title Page Icons of the sponsors of the NPSRP along with a narrated welcome.

To welcome the learners and set a tone.

2 - Agent introduction Agent appears, introduces himself and invites the learners to enter their name in the space provided to begin.

Initiates learner actions and continues assimilation of the tone.

3 - Objectives All four NPSRP PG objectives are written, entering in sequence.

Let the learners know what the purpose of the PG is and to allow for readiness to proceed.

4 - NPSRP Program Checklist

An image of the General Checklist appears with the agent in front of the right portion and narration reminding learners to download their own copy of the Checklist.

Establish that the learner should have already downloaded and viewed the checklist(s) so as to allow for activation of prior knowledge.

5 - Program Checklist fishing license

Highlight that a fishing license is required from the checklist.

Establish the pattern of walking through the content with the agent.

6 - Priest Rapids Dam Zoomed-in map of the origin of the Columbia River zone.

Show visual evidence of the domain of the NPSRP along the Columbia River.

7 - Hells Canyon Dam Zoomed-in map of the origin of the Snake River zone.

Show visual evidence of the domain of the NPSRP along the Snake River.

8 - Pikeminnow Size Pikeminnow drawing with a ruler and a highlight box showing how to measure a minimum sized Pikeminnow.

Minimum allowable Pikeminnow size is a critical content on the checklists.

9 - Condition of Fish Pikeminnow in a cooler on ice. All Pikeminnow must be kept in fresh condition until submitted.

10 - Drag and Drop Fish Size

Five different sized Pikeminnow to be sorted according to NPSRP regulations.

Accountability for knowing how to measure and identify sport-reward eligible sized Pikeminnow.

11 - Overview of Day of Checklist

Daily Checklist for Pikeminnow Fishing.

Que transition from General Requirements to Day of Fishing guidelines.

24

12 - Station Hours Image of the registration station hours and a visual of the agent posing a self-talk question while pointing at the image.

Model of desired behavior by asking self-talk question and referencing the printed material.

13 - Quiz Q 1 T/F station hours

Problem solving question requiring a return to the previous slide.

Accountability for understanding that there are rules applied to registration station hours and exclusivity.

14 - Hook and Line Only

Highlighted point on checklist regarding method of catching Pikeminnow and image of preferred bait options.

Reinforcing the particular way in which fish submitted as part of the sport-reward must be caught for compliance.

15 - When & Where Highlighted actions on the Daily Checklist regarding registration station exclusivity and agent engaging in self-talk leading to frustration.

Letting the learners know that the regulations are fair and firm; there will be no negotiating if procedure is not followed correctly.

16 - Completed and Signed Vouchers

Highlighted actions regarding collecting vouchers with completed information and agent indicating documentation at this step.

Emphasizing the exclusivity guidelines for registration stations.

17 - Make Sure That You Are Paid

First point on Payment Process Checklist regarding IRS form W-9 and an actual image of a W-9 along with agent indicating documentation.

Point out that the money from the sport-reward is taxable and will be reported to the IRS.

18 - What to Mail Highlighted points on the Payment Process Checklist that indicate the items that need to be mailed-in and the destination (mailing address) for the standard vouchers.

Empowering the learners to imagine the voucher submission process including the contents of and actual envelope provided by request.

19 - Quiz Q 2 T/F Technician Mails Documents

Knowledge check about who is responsible for mailing in the documents.

Making it clear that it is the participant’s own responsibility to mail in the voucher documents.

20 - Quiz Q 3 T/F W-9 Form

Knowledge check regarding ISR form W-9.

Reminding participants that a W-9 is required one time only.

21 - Quiz Q 4 T/F Pre-Addressed Envelope

Knowledge check regarding payment process assistance from registration station technicians.

Remind participants to pick up a self-addressed envelope for easier processing.

22 - After Mailing the Vouchers

Highlighted portion of the Payment Process Checklist that indicate the

First reminder that November 15 is the day that all

25

deadline and troubleshooting options. documents need to be received.

23 - Specially Tagged Fish

Highlighted portion of the Special Tag Voucher Checklist regarding “spaghetti tags” and a photo of an actual Pikeminnow with a special tag.

Visual confirmation of what a spaghetti tag looks like and that further action is required in order to receive payment.

24 - Special Tag Voucher Payment

Highlighted portion of Special Tag Voucher Checklist indicating mailing destination and deadline for receipt and a series of images showing October and clearly citing November 15th as the “received by” date.

Another reminder of the critical date. Specifically, that November 15 is the “received by” date and not a postmark date. Also, point-out that documentation must be accurate and complete.

25 - Quiz Q 5 MC Received by Date

Multiple choice question asking to select the critical documentation receipt date for payment.

Accountability for the learner to internalize the importance of November 15th.

26 - Quiz Q 6 T/F regular vs special tag voucher payment

Knowledge check for participants who have submitted both standard and special tag vouchers.

Establish reasonable expectations and reminding the participant that the NPSRP is a bureaucracy.

27 - Quiz Q 7 T/F Calendar year mail-out

Knowledge check framed differently from before including a caution of possible nullification.

Extra support for participants to comply with the required timeline in an effort to reduce non-payments in the future.

28 - What is a Northern Pikeminnow?

Agent emoting to cue that the PG is transitioning from documentation to species identification for the Northern Pikeminnow.

Visible re-engagement of the theme that the PG will coach the learners through the details to come.

29 - Species Identification Guide

Agent engaging in self-thought and labeled images of the three species under examination.

Reinforcing modeled behavior of our agent.

30 - Peamouth Drawing and photo of Peamouth along with points and highlights.

Visual guide to the quick ID of a Peamouth including photo.

31 - Chiselmouth Drawing and photo of Chiselmouth along with points, zoom, and highlights.

Visual guide to the quick ID of a Chiselmouth including an actual photo of the head.

32 - Pikeminnow Drawing and photo of Northern Pikeminnow along with points and highlight boxes.

Visual guide to the quick ID of a Pikeminnow including an actual photo of the head.

33 - Species ID Drag Multiple specimens of each of the Accountability that the

26

and Drop three species under examination for identification, a “reward eligible” box, and a “NO Reward” box for the drag-and-drop activity.

learners can distinguish the target species from commonly mistaken species to build confidence.

34 - Conclusion (new) Five bullet-points that bring together the important aspects of the NPSRP using empowerment language.

To recognize completion of the NPSRP PG while maintaining the coaching tone from the agent.

35 - Certificate Widget for Course Completion. Can be printed-out and produced upon request from NPSRP staff.

36 - References/Credits

https://api.icons8.comicons8-bank.png

Compliance.

Remaining Process for Completing PREP:As of now, the NPSRP PG has met its designated objectives. However, further action has

been requested by the PREP Team and a few more steps are still ahead before the product can go

live on pikeminnow.org. Since the PG will be hosted on pikeminnow.org, some changes to the

current content and layout are still agenda items. Several recommendations have been shared

with the PREP Team to make navigation more intuitive, reorganize existing content, and

prominently post new enhancements including the PG and checklists. However, due to the

seasonal nature of the NPSRP, some of these agenda items cannot be addressed until early 2018.

Additionally, any changes and/or enhancements to pikeminnow.org must go through an approval

process which proceeds slowly. Regarding the PG itself, as long as the feedback loop remains

open from the PREP Team, which is composed of Subject-Matter Experts (SMEs), the requests

for revisions on the style and usability enhancements will continue. The NPSRP season does not

open again until May 2018; but the changes to pikeminnow.org will have to be set by February

2018 at the latest. Remaining progress by committee will remain tabled until January 2018, when

the action plan leading to approved content going live on pikeminnow.org can be reactivated

within the PREP Team structure.

ResourcesTo complete the PREP process, resources were required in the following forms: human,

published scientific reports, the program website, published news articles, a New York Times

27

‘bestseller’ book, and instructional design theory. The process started and ended (will end) with

human input. Conversations that explored the state of the NPSRP and its ambitions revealed a

need for technologically-delivered educational support for visitors to pikeminnow.org and for

prospective participants in particular. The first person to engage with this conversation is the

Program Manager of the NPSRP, Steve Williams. Steve enlisted the collaboration of experts

who have been working with the NPSRP for years and in some cases decades. Joining the

conversation and eventually forming the PREP Team: Program Manager for the WDFW aspect

of the NPSRP Eric Winther as the supervisor for the registration station technicians, Pikeminnow

researcher for the NPSRP with the ODFW Mac Barr as the biologist familiar with population

estimates and tagging (including payment process for special tag vouchers) of both Northern

Pikeminnow and juvenile salmonids, PSMFC and pikeminnow.org webmaster Craig Miller as

the web technician expert and person familiar with the voucher payment process, and recently

retired administrator David Roberts, responsible for authorizing funding of NPSRP activities by

the Bonneville Power Administration (BPA).

Once the team had been assembled and needs ascertained, the learning and instructional

design resources were considered. Since the product selected by the PREP Team to pursue was

an interactive online module, Richard E. Mayer’s 12 Principles of Multimedia Learning were

immediately brought into the design decisions. All 12 of these principles were honored in

development of the NPSRP PG whereby a tone of respect for the learner and the learning

objectives were prioritized. Since the PG is essentially a teaching device, Robert Gagné’s Nine

Events of Learning were also considered in storyboarding the PG. The revolutionary guidebook

for advocating checklists for procedural learning The Checklist Manifesto by Atul Gawande

informed the decision to include checklists as job-aids and guidance on how to create effective

checklists. These checklists also became the structure for the PG, which facilitated a coherent

presentation of the PG content and sequencing, while honoring a commitment to concision. Also

employed in the design and development of the PG was the Social Learning Theory by Albert

Bandura whereby the learners would be motivated by engaging with modeled behavior in the PG

by an agent who represents participants in the NPSRP respectfully and authentically.

Since a constraining aspect to the PREP Team member access agreement meant that

SME time should be recognized as voluntary, the published work of the SMEs became an

invaluable resource. Obviously, pikeminnow.org contained information from the NPSRP experts

28

including direct citations of the PREP Team members. However, the NPSRP Annual Reports

chronicled on pikeminnow.org (especially the most recently published annual report from 2016)

was particularly helpful to paint a picture of how the NPSRP has performed. The 2016 report,

which was literally authored by Steve, Eric, and Mac is 148 pages in its entirety. While all of the

research, observations, and recommendations within the report was informative and pertinent to

the NPSRP, the first 48 pages provided data indicating program performance. The 2016 Annual

Report revealed the number of participants, including their proficiency classification. Data on:

day to day catch rates, hours spent overall and by individuals fishing for Pikeminnow, correlation

on catch and geographic location, and detailed earnings of the participants were also charted in

the annual report. The discoveries from the annual report that might apply to the NPSRP PG

were discussed and prioritized during the development and primarily design process. Finally, the

2016 Annual Report contained recommendations for educational outreach that ended-up

supporting the design and development decisions of the PREP Team.

All of these resources were integrated into the process and are evident in the NPSRP PG

along within the evaluation process that remains ongoing. Additional resources include: Adobe

Captivate 9, Adobe Photoshop, Google Drive, GoToMeeting.com, and versatilecompany.com for

project management document templates.

TimelineTable 1: Timeline Milestones

Milestone Description Completion Date

(estimate)

Assemble Team Project Sponsor Steve Williams assembles team of SMEs to provide support in the form of the PREP Team and team members agree to join.

9/8/2017

Establish Communication System

Contact was made with each SME both as a group and later individually to confirm that the team drive created to host groupware documents is accessible and discuss secondary communication preferences.

9/15/2017

Multimedia Deliverable Mode Consensus

The PREP team agreed to pursue an online interactive module (PG) instead of revised instructional videos or some combination thereof.

9/25/2017

Storyboard of PG Delivered for Revision Process

A storyboard in the form of an Adobe Captivate Project that contains a visual display and sequence from which to add narration and navigation.

9/31/2017

Working Version of A fully navigable version of the PG was provided 10/11/2017

29

PG provided for first post-storyboard revision

for a revision cycle to PREP Team.

Narration Draft Submitted for Revision

A draft of the narration added to the slide description of the current revised PG shared within the team drive for revision.

10/27/2017

Narration Script Finalized

Finalized script for the PG is approved for recording.

11/14/2017

Finalized Checklists Approved

After revising the checklists during the revision process of the narration script, the finalized versions of the checklists were updated in the PG.

11/20/2017

Stakeholder Evaluation

Stakeholder evaluation form completed and results discussed with PREP Team to inform revisions.

11/28/2017

Final Report Delivered

Final Report delivered to MIST and PREP Team via Capstone Festival.

(12/12/2017)

Recommendations for Website Revisions Consensus

A series of accompanying revisions and enhancement to pikeminnow.org are agreed-upon along with the deliverables provided via the PREP Team.

(1/12/2018)

All Deliverables from PREP Team Live on website

The NPSRP PG, checklists, and pikeminnow.org revisions are live for availability during the Capstone Festival.

(1/26/2018)

Evaluation and TestingFormative Evaluation & Usability Testing:Figure 6: PREP Team Revision Plan

30

The formative evaluation took place primarily through asynchronous involvement along a

suggested timeline with groupware, including the groupware document in the Team Google

Drive show in Figure 6, called the PREP Team Revision Plan. Revision requests were made in

the column on the left, by any PREP Team member after a link to the most recently published

version of the PG was uploaded to the university server. Developer commentary was posted in

the second column for the original requester to indicate a proposed path toward resolution or to

pose a clarification. The final decision is shown in the third column either by the project sponsor

or by proxy from the original requester. The column on the right communicates the progress of

each revision request usually from a salmon colored box, which indicates further input from the

revision requester, through a yellow color indicating that progress is underway, to the eventual

green color indicating that the item had been resolved in the PG and would be observable upon

publication and sharing of the next version of the PG link. The red colored box was a

hypothetical designation for infeasible requests, but all of the requests were resolvable. Figure 6

does not show an exhaustive record of the revision requests, it shows the structure of the process

which was used over the course of development and would have taken several pages if shown in

its entirety.

For the usability test questions, see Appendix A.

Figure 7: PREP Participation Guide Narration Guide

31

Similar to Figure 6, Figure 7 provides a look at the structure that was employed by the

PREP Team to provide revision notes the narration before offering a narrated version of the PG

for review. Figure 7, chronicles the script for the PG where each slide represented a scene unto

itself. The middle column represents the most concise available phrasing to compliment the

imagery of the PG drafts. The column on the right was available for the PREP Team to access

asynchronously in the Team Google Drive within a suggested time frame and provide

improvements or in some cases nearly complete rephrasing by the member of the PREP Team

who claimed expertise for the content represented on that particular slide. In some instances,

members of the PREP Team partnered remotely to address the revisions for the narration. Other

times, the script revealed that the source information on pikeminnow.org was inconsistent,

unclear, or in need of alteration. The results of these discoveries were translated into the

recommendations for revisions to the website content and layout previously described. Once the

final revisions were made, a new document with only the slide number and narration was shared

in the Team Google Drive to represent the final script.

Initially, the evaluation shown in Appendix A was intended to be given to members of

the PREP Team and also the technicians who staff the registration stations during the NPSRP

season. However, this process will have to wait for a more convenient time in the future given

the seasonal nature of the NPSRP. Instead, PREP Team members who were available for a

conference call agreed to dialog regarding the group submission that was provided. The dialog

did not match the results of the form responses. This inconsistency was revealed in the dialog to

have resulted from unclear instruction or expectations for responses. On the other hand, the

respondents were happy to have the survey experience upon which to draw during the dialog and

the results of the conversation yielded a constructive understanding of how the PG was received

by important stakeholders. Had the responses on the survey reflected the phrasing and rating

scale as stated, the responses would have provided confirmative evidence that the PG is

structurally and educationally sound. The dialog also revealed that the look of the PG is expected

to reflect the image of the PSMFC in a way that the designer could not have known. For this

reason, the PSMFC has elected to take responsibility for stylistic changes to the PG if such

changes are to take place at all. As far as meeting the program objectives of educating

prospective participants in a manner that is reflective of the stated objectives, the consensus is

that the PG has achieved those ambitions.

32

Questions for the learning effectiveness test (identical for the pre- and post-tests) are

shown in Appendix B.

Appendix C reveals the results from the learning effectiveness tests. Links were sent to

five people of varying age and three distinct geographic regions. The learning effectiveness

testers were given instructions to take and submit the pre-test, go immediately to the link for the

PG and follow the sequence all the way to the end, then take and submit the post-test. The pre-

test and the post-test are the identical six questions in the same sequence.

The pre-test results were predictably inconsistent and responses were far more likely to

be incorrect than correct. In fact, four of the five testers only responded correctly to one of the

six pre-test questions and the fifth tester provided only incorrect responses. For the post-test, the

only tester who did not correctly respond to all of the six questions was the same tester who had

responded incorrectly to all six of the pre-test questions. Obviously, the pattern is that the PG

made a maximally positive impact on learning for the testers. The ‘Descriptive Statistics” for the

pre-and post-tests are shown in Appendix D. Standing-out is that the mean score on the pre-test

(14%) improved to 90% as a mean on the post test. Also notable among the post-test results is

that both the median and mode for the testers was 100%; in fact, 4 out of 5 earned 100%.

Since the five testers were tracked from their pre-test to their post-test performance, a

paired t-test was used to analyze the learning effectiveness. Four out of the five testers earned a

pre-test score of 17% correct. All four of those five testers went on to earn a 100% on the post-

test. The one tester who did not score a 100%, earned a score of 50% on the post-test, but

submitted a pre-test with 0% correct. Therefore, all of the testers improved by at least 50% and

four out of five of the testers improved by 83% on the post-test. The one tester who gained the

least, appears to be an outlier. However, even with this one outlier, the hypothesis that the PG

would result in significantly higher scores remains valid by a comfortable margin.

As long as the absolute value of the t Stat is greater than the t Critical two-tail value, then

the null hypothesis that no significant leaning took place can be rejected. Since |-7.23| = 7.23 and

7.23 > 2.31 (t Critical two-tail) then statistically significant gains can be claimed and by a

significant margin. The null hypothesis can be rejected if the p Value is lower than the 0.05 (or

5% threshold). Given that the p Value is 0.0003 (or 0.03%), the null hypothesis can be

comfortably rejected and the learning effectiveness test does indeed show significant learning.

The t-Test results are available in Appendix E.

33

ConclusionOver the course of PREP, I have grown to appreciate the value of remaining objective

when working on a solution. The NPSRP has considered ways to make the orientation process

for participants more interactive and consistent. In reality, the NPSRP PG is a first attempt at

engaging the prospective participants and challenging them to be benchmark-prepared to

participate from day one. I am sure that the PG will evolve over time to meet the needs of the

program and the learners. I look forward to discovering how the availability of the PG will

impact the error rate among participants who have valid payment vouchers and seek payment.

PREP challenged me to approach organizational problem-solving with balance,

creativity, and stewardship. In previous projects, I had a single client communicating her

message to an entire team of instructional designers (of which I was one of them) or just myself

serving all of the instructional design roles. During PREP, there were several clients doubling as

experts and I was the only instructional designer. The PREP Team demanded that I respect their

time but also made it clear that they expect a product that they would be proud to include in

formal enhancements to NPSRP outreach. I had to devise efficient methods for managing

communication, content, and timelines. I feel that the communication system worked within the

constraints placed-upon me. I am grateful for access to the Project Management course

concurrent to PREP. The process of analyzing and documenting the project management

resources aided in conceptualizing the tasks and timeline that I would need to carry-out for

PREP. If I had greater access to the PREP Team during the design and development phases, then

I might have shared the project management aspect with them. I wonder if it would have helped

PREP Team members to have been more aware of the justification for design decisions that

informed interactions between team members.

After delivery of the NPSRP PG to the PSMFC, I still need to follow-up with

organizational and content recommendations that resulted from conceptualizing the full

implementation of PREP on pikeminnow.org. The PG cannot really be separated from the

checklists that were the first product delivered by PREP. I look forward to finding-out how the

implementation plan meets the realities of “going live” for the 2018 NPSRP season.

I discovered that environmental preservation and stewardship projects such as the NPSRP

are legitimate professional challenges that are important and rewarding. I would certainly not

shy-away from continuing to pursue projects that make a real and positive impact on the natural

34

environment. I can think of no more redeeming purpose than employing my expertise as an

instructional designer to projects that serve the public good. Selling soap has its place, as long as

the soap doesn’t negatively impact the rivers, oceans, and other natural settings that we should

continue to protect.

35

References

12 principles of multimedia learning. (n.d.). University of Hartford-Faculty Center for Learning

Development. Retired from

http://hartford.edu/academics/faculty/fcld/data/documentation/technology/presentation/

powerpoint/12_principles_multimedia.pdf

Gagné, R. (2017). 9 levels of instruction. Center for Instructional Technology and Learning.

Retrieved from http://citt.ufl.edu/tools/gagnes-9-events-of-instruction/

Gawande, A. 2009. The checklist manifesto: How to get things done right. New York. NY.

Henry Holt and Company, LLC

Williams, S., Winther, E., & Barr, C.M. (2016). 2016 Annual Report: Report on the predation

index, predator control fisheries, and program evaluation for the Columbia River Basin

Northern Pikeminnow Sport Reward Program. Retrieved from

http://www.pikeminnow.org/wp-content/uploads/2017/03/2016-Pikeminnow-AR.pdf

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and

Development, 50(3), 43-59. Retrieved from

http://mdavidmerrill.com/Papers/firstprinciplesbymerrill.pdf

Nesbit, M. (2015, June 4). Anglers can earn money fishing for Northern Pikeminnow. Tri-City

Herald. Retrieved from

http://www.tri-cityherald.com/sports/outdoors/article32227557.html

Social learning theory. (n.d.). Retrieved from

http://www.instructionaldesign.org/theories/social-learning.html

Appendix A: Questions for the usability test

36

Item # Question text along with rating scale

1

2

3

4

5

37

6

7

8

9

10

11

38

Appendix B: Questions for the Learning Test

Question # Pre-test Question Text Post-test Question Text

1 What is the minimum fish size requirement for NPSRP payment?

What is the minimum fish size requirement for NPSRP payment?

2 In order to receive payment for participation in the NPSRP, how must the payment voucher(s) be submitted:

In order to receive payment for participation in the NPSRP, how must the payment voucher(s) be submitted:

3 When do the registration voucher(s) have to be processed in order to receive payment for participation in the NPSRP?

When do the registration voucher(s) have to be processed in order to receive payment for participation in the NPSRP?

4 How much are the spaghetti tags (fish tagged with a valid spaghetti tag) worth if they are verified?

How much are the spaghetti tags (fish tagged with a valid spaghetti tag) worth if they are verified?

5 Which species is most likely to be confused with a Northern Pikeminnow by NPSRP anglers?

Which species is most likely to be confused with a Northern Pikeminnow by NPSRP anglers?

6 Do participants in the NPSRP need to have a W-9 form on file in order to receive payment?

Do participants in the NPSRP need to have a W-9 form on file in order to receive payment?

39

Appendix C: The Pre-test and Post-test for users unaffiliated with the NPSRP

40

41

Pre-test Post-Test

Appendix D: Descriptive Statistics for the Pre- and Post-Tests

42

Appendix E: t-Test

43

t-Test: Two-Sample Assuming Equal Variances

Pre-test score Post-Test ScoreMean 0.136 0.9Variance 0.00578 0.05Observations 5 5Pooled Variance 0.02789Hypothesized Mean Difference 0Df 8t Stat -7.23334377P(T<=t) one-tail 4.47461E-05t Critical one-tail 1.859548038P(T<=t) two-tail 8.94922E-05t Critical two-tail 2.306004135p Value = 0.000320278

44