Pride Scorecard Training December 2013

Pride Scorecard TrainingDecember 2013

Agenda• Big picture (10 min)• Overview of the new measures

- Ariela on capacity (45 min)- Brian on constituency (20 min)

• Recap the scoring system (15 min) • Highlight upcoming events (5 min)

Outcomes• Understand the new measures• Understand how the PSC can inform next steps for your

campaign• Understand how to assess a campaign based on the criteria

and assigning a color score

Agenda & Outcomes

• Big picture• Overview of the new measures

- Ariela & Team on Capacity- Brian on Constituency

• Review the scoring system• Highlight upcoming events

Agenda

(Succinct) Timeline of Pride Scorecard

~Feb20122011 2012 -

TodayJuly

2013Dec. 13

Feb14

Laun

ch o

f the

New

C

ampa

ign

Trac

ker

Ref

ine

Que

stio

n on

Goo

gle

Doc

s

Beg

an s

olic

iting

form

al fe

edba

ck

from

TM

T, S

ocia

l Mar

ketin

g, M

MT

•U

sed

vers

ion

on G

oogl

e D

ocs

•C

ount

less

twea

ks

Form

al a

ppro

val b

y R

egio

nal

VP,

Dal

e

Pilo

ted

in 3

regi

ons

for 3

mon

ths

Last

dra

ft re

view

ed b

y TM

T, S

ocia

l m

arke

ting,

MM

T

~Jan2012

Commitment is to create at least one opportunity per year to make substantive changes.

What the Pride Scorecard Measures

The Pride Scorecard measures the progress of a campaign based on the objectives of the 3Cs using a set of action-oriented indicators. Using the 3Cs as the framework allows us drive improvements in the campaigns and to assess the likelihood of continued success after the campaign has ended.

Objectives of the 3Cs: • Build the Capacity of the Rare Fellow(s) and IP to have the

willingness and ability to continue the campaign successfully on their own.

• Develop a Constituency for the campaign that ensures the community's commitment to conservation, their adoption of the desired BC as a social norm and ownership of threats.

• Achieve the campaign's Conservation goals via new behavior adoption, threat reduction and successfully monitoring impact.

1. Improve Campaigns – Action oriented indicators are designed to drive decisions to maximize the impact of our campaigns

2. Guidance - Support staff in identifying what the key issues are for their campaigns

3. Transparency - Create a standard indicator for global conversations about campaigns to foster a global conversation

4. Accountability – we all should know our key roles in driving campaign improvement

Why We have a Pride Scorecard

Who Uses the Pride Scorecard and WhyWho WhyDale • Part of Dale’s monthly reporting pack to assess the progress of each cohort, where to

invest resources, etc.• Liz sends a Status Report with PSC scores included to Cohort Director at same time as to the

team member who assembles the full reporting pack• Cohort director adds 3 to 5 bullet points to the report and sends them to VP

RVP • To assess cohort progress and how to allocate resources across campaigns• To understand the needs of the team

Cohort Director

• To assess the progress and impact of their cohort, where it is on track and where it needs support

• To identify what actions are needed to ensure each campaign achieves its long-term objectives

• To make recommendations about investing or cancelling campaigns

Board • To get updated on Rare’s programmatic progress and impact

Rare • To measure our performance and impact using the BSC

Regions • To measure regional performance and impact using the DBSCs

PPMs • To systematically document successes and achievements of their campaigns• To identify specific steps and actions needed to improve their campaigns• To keep an eye on upcoming actions and steps that need to occur next to stay on

track

TMT • To assess areas where training needs to be re-enforced

MMT • To track the progress of their monitoring efforts, and adapt accordingly

QMI • To share campaign and cohort progress in reports to other teams• To flag challenges early, to collect and share bright spots• To ensure the global minimum standard of quality for campaign progress is be met

Where the Scorecard Fits In

Pride Scorecard

Weekly Flash Reports

Regions have own management

processes

Quarterly BSC Reports

Select BSC Measures are in

your IBSC

Monthly Status Reports




Agenda

Refining Rare’s ‘Capacity’ Metrics

Define what we want to measure

Deliverables or skills/capacities?

Pride Skills Assessment

TMT Meetings and 1:1

Conversations

Training Department Strategic Planning

Final 5 Capacities

Inventory of Existing Tools

Scorecard, Alumni Surveys, etc.

Questions

Succ

ess

Indi

cato

rs

Input from TMT & Global Programs

Training Management Team Meeting

Validate Success Indicators and Draft

Tools with Alumni

Roadmap for Developing the 5 Capacities, Success Indicators, and Metrics

TMT Feedback on Learning Outcomes

Revise Success Indicators with PPM

and TMT Input

Monitoring Team Meeting

Input from Monitoring Team on Success Indicators

Share Metrics Framework with LT

LT Meeting

RareWorld Communication

Email Metrics Framework to

RareWorld

5 Key Leadership Capacities

Alignment of Metrics Tools to Kirkpatrick’s Model of Training Evaluation

During the course of the campaign, what are some of the key Strategic Planning skills that the Rare Fellow(s) has displayed?

Rare Fellow(s) is consulting qualitative and/or quantitative research to inform campaign decisions.

Rare Fellow(s) is using SMART Objectives of the site-level Theory of Change to drive campaign strategies and/or work plan.

Rare Fellow(s) is adapting strategies and/or Research Plan and/or work plan(s) to account for new information and/or unforeseen circumstances.

Capturing ‘Behavior’ in the Scorecard: “Presence” or “Absence”




Agenda

Stepped BC Metrics

What percentage of the campaign’s SMART Objective for the TA’s enabling step in BC has been achieved so far? Sustainable Fisheries campaigns: Fishers regularly attend and participate in meetings, trainings and workshops about fisheries.ARA campaigns: Targeted members of the Upstream TA are taking steps towards signing an ARA.

Campaign is achieving at least 20% to 39% of this BC SMART Objective

Campaign is achieving 40% to 59% of this BC SMART Objective Campaign is achieving 60% to 79% of this BC SMART Objective Campaign is achieving 80% to 99% of this BC SMART Objective Campaign is achieving 100% of this BC SMART Objective

Stepped BC Metrics

What percentage of the campaign’s SMART Objective for the TA’s intermediate step in BC has been achieved so far? Sustainable Fisheries campaigns: Fishers register their boat and fishing gear.ARA campaigns: Targeted members of the Upstream TA have signed an ARA with a minimum duration of 5 years.



Stepped BC Metrics

What percentage of the campaign’s SMART Objective for the TA’s ultimate step in BC has been achieved so far? Sustainable Fisheries campaigns: Fishers acquire fishing licenses.ARA campaigns: Members of the Upstream TA who have signed ARAs, isolate the agreed upon conservation objectives in their properties for conservation and got a minimum and renewable period of 5 years






Agenda

Recap: Scoring SystemOld New

• Pride Scorecard scores were an average of the scores for each measure

• Color assignment was automatic based on the 5 pt. scale

• Scores were sometimes artificially low or high due to averaging

• Pride Scorecard scores are the summed total points of each measure

• PPMs assign a color to each Pride Scorecard score, which is approved by Cohort Director

• Campaign progress is portrayed more accurately throughout the cohort timeline

Indicative Pride Scorecard Dashboard

Indicative target for cohort at completion

Period

Key Points to Keep in Mind for Scoring Baseline

• Created by hypothetically scoring a TOP campaign using each measure for each month

• This baseline is a starting off point for us to test and improve• Anything in the scoring baseline being presented can change

and be negotiated as needed

Points Possible Over TimeMonth Possible Points Descriptions

Month 1 23 The end of the first month Rare Fellow(s) has been in the field after 1st Uni Phase

Month 4 40 4 months after Rare Fellow(s) has been in the field, midpoint of the Planning Phase

Month 9 63 The end of the first month Rare Fellow(s) have been back in the field after 2nd Uni Phase

Month 12 84 4 months after Rare Fellow(s) has been in the field after 2nd Uni Phase, near or at public launch time

Month 22 103 Final months of Implementation Phase, all essential SM, BC, BR, TR, CR actions in place

Color Assignment GuidanceEach region can set their own standards for green, yellow, red

Month Possible Points

Green Range(80% or more of points possible)

Yellow Range(50% to 79% of points possible)

Red Range(49% or less of points possible)

Month 1 23 18 – 23 12 – 17 0 -11

Month 4 40 32 – 40 20 – 31 0 – 19

Month 9 63 50 – 63 32 – 49 0 – 31

Month 12 84 67 - 84 42 – 66 0 – 41

Month 22 103 82 - 103 52 - 81 0 - 51

The most important thing is that if you need management assistance, flag the campaign as red

Green = Progress is on track to achieve objectivesYellow = Progress is lower than expected but does not require immediate senior manager actionRed = Progress is off track and/or requires senior management attention to resolve critical issues

Submission & Approval Process

1. PPM, Training Manager/Director, Monitoring Specialist and BR Specialist score measures

2. PPM assigns color before submitting to Cohort Director3. Cohort Director reviews and approves Pride Scorecards as

before, and approves PPM’s color assignment4. Cohort Director assigns a rank to each campaign




Agenda

Next StepsComplete December Pride Scorecards (due 12/20/13) in Google with updated measures and new scoring system• Share feedback on measures, month ranges for points and point ranges asap• Feedback will be incorporated into January Pride Scorecards

Complete January Pride Scorecards in Google• Share feedback again on updates made asap• Feedback will be incorporated but not by the February Pride Scorecard due to

Campaign Tracker launch

Get training on the new Campaign Tracker (late January)• Learn how to upload and approve milestones• Learn how to blog and upload multimedia files• Complete the Pride Scorecard using the CampaignTracker (February)

Appendix: Questions about the CampaignTracker TransitionWhat’s in the CampaignTracker• Milestones page• Prompts on upcoming milestones• For Rare/EDF/UCSC staff

• Weekly flash report• Pride Scorecard• Campaign administration

What’s NOT in the Campaign Tracker• No more groups• No more personal pages

What happens to content that is there• Active groups are being migrated to new online tools• Campaign content will be ported into the Campaign Tracker automatically• Personal pages and blogs will be archived

When will it happen?The transition will begin in mid January and be largely complete by the end of February

Who has access to the Campaign Tracker• Unlike RarePlanet the site will ONLY be accessible to Rare/EDF/UCSC staff and Implementation partners

How does this link to the upcoming relaunch of Rare.org• Campaign blogs will link to Rare.org• Select campaign information to provide an overview and some deliverables will be linked• People will learn about programs, cohorts and campaigns via Rare.org

Appendix: Status ToolsWeekly Flash

ReportPride

ScorecardCohort

RankingD-BSC

Purpose Shows the progress the campaign is making against this month’s goals in the project plan.

Helps to track progress in achieving campaign goalsHelps to provide guidance about how to achieve those goals

Ranks the intrinsic importance of a site

Tracks progress on achieving annual objectives

Primary User PPM & Cohort Director

Cohort Team & Cohort Director

Cohort Director, VP, Dale

All

Indicative actions

• Quickly escalate at risk campaigns

• Highlight key issues

Highlight key opportunities and challenges

Ensure that we invest in the most important campaigns

Allocate regional priorities to achieve regional (not just cohort) objectives

Frequency Weekly Monthly Monthly Quarterly

During the course of the campaign, what are some of the key Strategic Planning skills that the Rare Fellow(s) has displayed?

Rare Fellow(s) is consulting qualitative and/or quantitative research to inform campaign decisions. Rare Fellow(s) is using SMART Objectives of the site-level Theory of Change to drive campaign strategies and/or work

plan. Rare Fellow(s) is adapting strategies and/or Research Plan and/or work plan(s) to account for new information and/or

unforeseen circumstances.

During the course of the campaign, what are some of the key skills associated with Behavior Change Interventions that the Rare Fellow(s) has displayed?

Rare Fellow(s) is accounting for audience’s perceived benefits and barriers in the design and/or implementation of their Marketing Strategy.

Rare Fellow(s) is aligning Social Marketing and Barrier Removal strategies to accelerate the adoption of the new behavior.

Rare Fellow(s) is selecting and/or implementing activities and materials to progress the target audience through the Stages of Behavior Change (as aligned to K, A, IC, and BC).

Appendix: All Updated Capacity Measures

During the course of the campaign, what are some of the key Effective Communication skills that the Rare Fellow(s) has displayed?

Rare Fellow(s) is communicating the goals and/or strategies of the campaign in terms that are appropriate for the intended audience.

Rare Fellow(s) is using Brand, messages, activities, and/or materials to communicate the Benefits Exchange. Rare Fellow(s) is motivating key influencers and/or groups to take concrete actions to support the campaign.

During the course of the campaign, what are some of the key skills related to Conservation Science and Management that the Rare Fellow(s) has displayed?

Rare Fellow(s) is addressing site-specific threats to the conservation target through the campaign strategies and/or work plans.

Rare Fellow(s) is designing and/or implementing a Barrier Removal strategy that addresses site-specific threats. Rare Fellow(s) is tracking the measurement and monitoring of campaign impact (TR and CR).

During the course of the campaign, what are some of the key elements of Skills Transference that the Rare Fellow(s) has displayed?

Rare Fellow(s) is sharing lessons learned within and/or across cohorts. Rare Fellow(s) is communicating and/or sharing key capacities (Strategic Planning, Behavior Change Interventions,

Effective Communications, and Conservation Science and Management) with implementing partners and/or local agencies.

Rare Fellow(s) is taking actions and/or considering decisions that will contribute to the sustainability and/or continuation of the campaign.

Appendix: All Updated Capacity Measures

Pride Scorecard Training December 2013

Documents

Transcript of Pride Scorecard Training December 2013