Deployment Performance Review of the 2016 North Pacific ... · Deployment Performance Review of the...

Deployment Performance Review of the 2016 North Pacific Observer Program

2016 Observer Science Committee

Presented by Fishery Monitoring and Analysis Division, Alaska Fisheries Science Center, Seattle

North Pacific Fishery Management CouncilObserver Advisory CommitteeMay, 2017 Seattle, WA

The Analytical TeamAnalyses were performed by the Fisheries Monitoring and Analysis Division in consultation with experts with practical knowledge of observer data. The Division convenes its Observer Science Committee annually. This years members included:

• Craig Faunce (AFSC/FMA) • Jane Sullivan (Alaska Sea Grant Fellow, AKRO/SF)• Steve Barbeaux (AFSC/REFM)• Jennifer Cahalan (PSMFC)• Jason Gasper (AKRO/SF)• Sandra Lowe (AFSC/REFM)• Ray Webster (IPHC)

This review is intended to inform the Council and the public of how well various aspects of the program are working and lead to recommendations for improvement (based on the data). OSC recommendations do not need to equate to official NMFS recommendations or actions for future ADPs.

Efficiencyis focused on inputs: how well is a task performed?

Effectivenessis focused on outputs:How meaningful is the product?

http://bitbar.com/effective-mobile-devops-strategy-and-typical-goals/https://blog.versionone.com/words-mean-things-efficient-and-effective/

Die Slowly

Survive Thrive

Die quickly

http://bitbar.com/effective-mobile-devops-strategy-and-typical-goals/

https://imgs.xkcd.com/comics/efficiency.png

Why so Random?

Chapter 3 is focused on the efficiency and effectiveness of observer and EM deployment.

In order to evaluate observer and EM deployments, We must know how the trips or vessels were selected so that we can determine the probability of the outcome.

The mathematical theorems which justify most frequentist statistical procedures apply only to random samples.

Nonrandom samples have unknown probabilities associated with them and hence are useless for purposes of statistical inferences.

Changes in Methods:

• NEW! Trip definitions for full coverage reverted back to 2013 & 2014 methods.

• (Trip definitions from quota monitoring and do not accurately reflect fishing trips). Not comparable to 2015 values.

• Updated spatial coverage maps

• Visual summaries of vessel-selection strata (Electronic Monitoring)

• Development of Annual Report in a fully reproducible research project in R Markdown

• Increase efficiency

• Reduce errors

Observer Deployment 2016

Partial Coverage Two Year Comparison: Effort Predictions

2015 2016

Observer days used 5,318 4,677

Observer days predicted (ADP) 5,517 4,900

Effort prediction accuracy (%)(Days predicted in APD vs. days used) -3.6 -8.4

Section 3.5.1

15 strata to evaluate in 2016

U.S. Department of Commerce | National Oceanic and Atmospheric Administration | NOAA Fisheries |

Full coverage

1) Regulatory 2) Voluntary 3) HAL15.4%

4) POT15.2%

5) TRW28.3%

Partial coverage

Zero-selection

6) EM Research 7) Zero-selection

EM Voluntary1

EM Vol 100%EM Vol 30%

8) Jan-Feb 9) Mar-Jun10) Jul-Oct 11) Nov-Dec

12) Jan-Feb 13) Mar-Jun14) Jul-Oct 15) Nov-Dec

1 EM was in pre-implementation in 2016; data were not used for Catch Accounting. EM systems deployed using vessel-selection.

Partial Coverage Two Year Comparison: Coverage Rates

2015 2016

t T Zero All Zero HAL POT TRW EM All

Total trips 2,148 4,676 2,093 8,917 2,109 2,655 1,261 2,738 227 8,990

Observed trips 241 1,094 2 1,337 0 398 185 767 76 1,426

% Observed 11.2 23.4 0.0 15.0 0.0 15.0 14.7 28.0 33.4 15.91

% Expected 12.0 24.0 0.0 0.0 15.4 15.2 28.3

Meets Expectations? Yes Yes Yes Yes Yes Yes Yes

Yes &

No

1 The % Observed for all partial coverage categories would be 15.0% if EM is excluded.Section 3.6.1, Table 3-5


2015 2016


Total trips 2,148 4,676 2,093 8,917 2,109 2,655 1,261 2,738 227 8,990

Observed trips 241 1,094 2 1,337 0 398 185 767 76 1,426

% Observed 11.2 23.4 0.0 15.0 0.0 15.0 14.7 28.0 33.4 15.91

% Expected 12.0 24.0 0.0 0.0 15.4 15.2 28.3


Yes &

No


EM Voluntary

Jan-Feb Mar-Jun

Jul-Oct Nov-Dec

Jan-Feb Mar-Jun

Jul-Oct Nov-Dec

EM Voluntary 100%EM Voluntary 30%

35.7 – 50.0% 0.0 – 57.1%


2015 2016


Total trips 2,148 4,676 2,093 8,917 2,109 2,655 1,261 2,738 227 8,990

Observed trips 241 1,094 2 1,337 0 398 185 767 76 1,426

% Observed 11.2 23.4 0.0 15.0 0.0 15.0 14.7 28.0 33.4 15.91

% Expected 12.0 24.0 0.0 0.0 15.4 15.2 28.3


Yes &

No



Evaluation of EM Vessel-selection: Anticipating Effort

Figure 3-3


Evaluation of EM Vessel-selection: Coverage Rates

Figure 3-4

Partial Coverage Two Year comparison: ODDS


2015 2016

t T HAL POT TRW

Total trips logged 2,147 4,368 2,846 1,331 2,825

Programmed Selection % 12.0 24.0 15.41 15.24 28.31

Initial Selection Rate1 11.6 23.8 15.9 14.3 28.4

Final Selection Rate2 12.6 25.4 17.7 14.4 29.6

Cancellation % (Selected Trips) 23.7 13.2 23.9 25.3 15.8

Selection rate as programmed? Yes Yes No Yes Yes

Are initial and final selection rates similar over time? No No No No No

Tables 3-2 to 3-4, Fig. 3-21 Random number only. 2 Includes cancellations, waivers, and inherits.

Partial Coverage Two Year comparison: ODDS


2015 2016

t T HAL POT TRW

Total trips logged 2,147 4,368 2,846 1,331 2,825

Programmed Selection % 12.0 24.0 15.41 15.24 28.31

Initial Selection Rate1 11.6 23.8 15.9 14.3 28.4

Final Selection Rate2 12.6 25.4 17.7 14.4 29.6

Cancellation % (Selected Trips) 23.7 13.2 23.9 25.3 15.8

Selection rate as programmed? Yes Yes No Yes Yes

Are initial and final selection rates similar over time? No No No No No

1 Random number only. 2 Includes cancellations, waivers, and inherits.

Tables 3-2 to 3-4, Fig. 3-2

Partial Coverage Two Year comparison: Temporal and Spatial Bias


2015 2016

t T HAL POT TRW

Dockside deployment rates as expected? (Table 3-7)

No(King Cove)

No(King Cove)

Temporal observation rates as expected? (Fig. 3-5)

Yes (0%)

Yes(0.6%)

Yes (0%)

Yes(0%)

Yes(0.06%)

Spatial observation rates as expected? (Figs. 3-6 to 3-12) No Maybe Yes Yes Yes

Partial Coverage Two Year comparison: Temporal and Spatial Bias


2015 2016

t T HAL POT TRW

Dockside deployment rates as expected? (Table 3-7)

No(King Cove)

No(King Cove)

Temporal observation rates as expected? (Fig. 3-5)

Yes (0%)

Yes(0.6%)

Yes (0%)

Yes(0%)

Yes(0.06%)

Spatial observation rates as expected? (Figs. 3-6 to 3-12) No Maybe Yes Yes Yes

Tendering continues to affect genetic sampling and salmon bycatch estimation within the Pollock trawl fleet

Trip-selection + higher coverage reduced temporal bias

Minimal bias: HAL: SE AK (10 trips > expected),

eastern AI (6 trips < expected)POT: eastern AI (1 trip > expected)

TRW: central GOA (19 trips > expected)

Partial Coverage Two Year comparison: Trip Metrics


2015 2016

t T HAL TRW POT

Observed trips same asunobserved? (Tables 3-8 & 3-9)

No No

YesObserved trips 8-14% shorter

Observed trips 6-13% shorter,Landed catch weighed 9-10% less

1% less diverse

Retained 8% more species

Retained 16% fewer species

t t T HAL TRW POT

Tendered trips same asnon-tendered?(Tables 3-10 & 3-11)

No Omitted, low sample size No

Partial Coverage Two Year comparison: Trip Metrics


2015 2016

t T HAL TRW POT

Observed trips same asunobserved? (Tables 3-8 & 3-9)

No No

YesObserved trips 8-14% shorter

Observed trips 6-13% shorter,Landed catch weighed 9-10% less

1% less diverse

Retained 8% more species

Retained 16% fewer species

t t T HAL TRW POT

Tendered trips same asnon-tendered?(Tables 3-10 & 3-11)

No Omitted, low sample size No

Observer effects at 11%, 15%, 23% and 28% coverage

Tendering effects in Pot and Trawl

Partial Coverage Two Year comparison: Trip Metrics2015 2016

t T HAL TRW POT

Tendered Only: Observed trips same as unobserved?(Tables 3-12 & 3-13)

Maybe

Omitted, low sample

size

No

Yes33–101% less catch

when observed188% shorter

trips,16% fewer species,

70% less catch25% fewer

species

51% shorter

trips

1 Although this result was not associated with a p-value of < 0.05, large effect sizes were found..

t T HAL TRW POT

Non-tendered Only:Observed trips same asunobserved?(Tables 3-14 & 3-15)

No No

Yes5-13% shorter trips when

observed Vessels 2-3% longer

3.4% fewer areas

1.2% lessdiverse

6% shorter trips, 8%

more species

14% fewer species

Adequacy of sample size:The observer at sea is providing NMFS with at-sea discard rates on catcher vessels that are applied to landed catch to produce total catch. The goal is to apply discard rates from observed trips to unobserved trips with similar traits (you wouldn’t want to apply discard from a BSAI trawler to a GOA trawler for example).

For this reason, it is important that for each NMFS Area there is at the very least one observed trip. We can evaluate the likelihood of “missing” an area from 2016 data.

The likelihood of missing goes down as you: • Increase the number of trips in an area • Increase the sampling rate

Areas and gears with low amounts of effort activity will require higher selection rates to observe than areas and gears with large amounts of effort.

Small deployment “boxes” require higher rates of selection.


2014

2015

Prob

abilit

y of N

o Obs

erve

d Sam

ple U

nits

Number of Sample Units

2016

Figure 3-15

10.6 %

24.0 %

11.2 %

23.4 %

14.7 %

15.0 %28.0 %

OSC Recommendations

OSC Recommendations 2016

2013, 2014, 2015, & 2016:

1. The OSC reiterates its three-year recommendation that the NMFS improve the linkages between ODDS and eLandings.


2014, 2015, & 2016:

2. The OSC reiterates its two-year recommendation that the NMFS explore ways to reduce the impact of cancellations on the number

of trips selected for observer coverage in the ODDS.


3. The OSC recommends an alternative model of monitoring salmon bycatch be explored in the partial coverage fleet.


4. Three recommendations concerning future at-sea coverage rates:

a. Sampling rates should be high enough in each stratum to maximize the probability of achieving three observed trips in each

of the NMFS Areas (under funding constraints).


b. Future ADPs should include in each proposed sampling design sample allocation that is proportional to fishing effort

(equal rates among strata).


c. The OSC recommends that the SSC and Council request NMFS HQ reinstate its funding for observer deployment in the North

Pacific at levels necessary to ensure a minimum of 15% coverage among all strata in upcoming ADPs. If the critical 15% coverage

rate is surpassed among all strata combined, then sampling days afforded in excess of this amount may be allocated among strata

according to an optimization algorithm.

See More At: https://alaskafisheries.noaa.gov/fisheries/observer-program


Differential behavior caused by salmon.

Deliver to tender.No offload monitoring.

Zero salmon count extrapolated to trip & fleet.

Deliver dockside.Full offload monitoring.

Full trip salmon count extrapolated to fleet.

How long is this trip going to take?

How long is an unobserved tendered trip?

Figure 3-14

HALHook-and-line

POTPot gear

TRWTrawl

Deployment Performance Review of the 2016 North Pacific ... · Deployment Performance Review of the...

Documents

Transcript of Deployment Performance Review of the 2016 North Pacific ... · Deployment Performance Review of the...