Commonwealth of VirginiaState Homeland Security ProgramPUBLIC SAFETY STAKEHOLDER WORKSHOP

FEBRUARY 20, 2020

Peer Review and SME Evaluation

2

• Participate in the peer review by signing up at the VDEM website

https://www.vaemergency.gov/emergency-management-community/grants/

• Participate as a SME by contacting your chief regional coordinator

• Region 1 – Donna Pletch - donna.pletch@vdem.virginia.gov

• Region 2 – Mark Stone - mark.stone@vdem.virginia.gov

• Region 3 – Gene Stewart - gene.stewart@vdem.virginia.gov

• Region 4 – Tim Estes - tim.estes@vdem.virginia.gov

• Region 5 – Bruce Sterling - bruce.sterling@vdem.virginia.gov

• Region 6 – Mike Guzo - mike.guzo@vdem.virginia.gov

• Region 7 – Andy John - andy.john@vdem.virginia.gov

• Interoperability or Cyber Security - Tom Crabbs – thomas.crabbs@governor.virginia.gov

3

◦ Morning Session: (9:15-12:00)

◦ Welcome and Opening Remarks

◦ Curtis Brown, Chief Deputy State Coordinator/Chief Diversity and Inclusion Officer, VDEM

◦ Dr. Jeffrey Stern, State Coordinator, VDEM

◦ Shawn Talmadge, Assistant Secretary, Office of Public Safety and Homeland Security

◦ Program Updates

◦ Special Operations/Swift Water Rescue - Daniel Brewer

◦ Interoperability – Tom Crabbs

◦ Complex Coordinated Terrorist Attack Program-Shannon Daniel

◦ Fusion Center – John Janssen

◦ Lunch 12:00-1:00

◦ Afternoon Session (1:00 – 3:30)

◦ State Homeland Security Program – Barry Ezell

◦ Group Discussion

◦ Final Discussion & Closing Remarks

Agenda

2020 FEMA HSGP Target Investments

4

Target investments (Baseline Allocation 80% - $7.07M)

◦ Planning

◦ Training

◦ Exercises

◦ Equipment

FEMA Priority Investments (20% increase to base allocation - $8.8M)

◦ Enhancing Cyber Security (including election security) – 5%

◦ Enhancing protection of soft targets/crowded places – 5%

◦ Enhancing information and intelligence sharing – 5%

◦ Addressing emergent threats (unmanned aerial systems, WMDs, etc.) – 5%

Virginia can also apply for an additional 15% above the base and priority allocation ($10.1M)

Special Operations SME Group

Virginia Fire Chiefs Association recommended that a separate SME review team be stood up to review project applications that are submitted by localities that are not a part of the special operations teams.

◦ Fire Chiefs

◦ VEMA

◦ VDEM

5

State Homeland Security Program

6

Objectives

Review:

• Grant process

• Historic funding

• Grant guidelines & ground round rules

• 2019 grant

Discuss:

• 2020 priorities and recommendations

• 2020 evaluation criteria

7

Grant Cycle Process

8

1. Convene a stakeholder workshop to discuss grant topics and application evaluation criteria

2. Submission of proposals by applicants

3. Screening of projects by VDEM

4. Conducting peer reviews

5. Performing model calculations

6. Conducting analysis

7. Making funding decisions

8. Submitting selected projects to FEMA

2,8

56

,43

7

19

,24

2,2

73

14

,90

6,5

99

12

,56

5,5

69

5,0

28

,67

7

3,0

43

,26

3

4,4

08

,10

0

4,8

82

,76

8

2,8

20

,19

0

2,4

93

,32

0

2,5

22

,09

9

2,2

79

,96

4

2,4

83

,38

9

27 133 145 167 64 72 84 79 58 64 65 49 54

2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019

Award # Projects

9

Competitive Funds Awarded 2007-2019

• The available funds for competitive projects have

continually declined over the last decade.

• Since 2007, approximately $80M has been

awarded to over 1,000 projects.

2020 FEMA HSGP Target Allocation

•Law Enforcement Requirement (25%)

•Projected project performance period is October 1, 2020 - June 30, 2022

10

FY 2020

Virginia Base

Allocation (80%)

FEMA Priorities

(20%)

Competitive

(15% Additional)

Total $7,076,800 $8,846,000 $10,172,900

State Projects 20% $1,415,360 $1,769,200 $2,034,580

SOT & Fusion $2,500,000 $2,500,000 $2,500,000

Exercise Programs $900,000 $900,000 $900,000

Local Competitive $2,261,440 $3,676,800 $4,738,320

FY19 - $9.2M

FY 18 - $7.1M

2020 SHSP FEMA Funding Allocation

11

$0

$500,000

$1,000,000

$1,500,000

$2,000,000

$2,500,000

$3,000,000

$3,500,000

$4,000,000

$4,500,000

$5,000,000

State Projects

20%

SOT & Fusion Exercise

Programs

Local

Competitive

2020 Projected Allocation Scenarios

Virginia Base Allocation (80%) FEMA Priorities (20%)

Competitive (Additional 15%)

Ground Rules

12

• Projects will be funded at a MINIMUM of $10K, and a MAXIMUM of $250K

• Prime movers (pickup trucks) are not allowed.

• A $500K cap will be placed on all special mission (armored type) vehicles

• A $250K cap of competitive grant funds available will be placed on all Community Emergency Response Team (CERT) project awards.

• Projects that are divided into multiple proposals are not allowable. Applicants must include all project components in one proposal.

• State agencies are eligible to partner with local or regional fiduciary agencies on projects but must not function as the fiduciary agent.

• For local competitive projects, the bottom 40% of lowest scoring projects will be eliminatedprior to benefit-cost ranking.

13

2019 Projects # Cost

Proposals Submitted 126 $13.2M

Proposals Evaluated by Peer Reviewers and SMEs

(3 projects were ineligible; 2 projects were reprogrammed)

121 $12.6M

Eligible Proposals Scoring above 40% 72 $6.3M

Projects Selected for Submission to FEMA 54 $2.5M

2019 Proposal Submissions

$7

,56

4,1

37

$3

,59

0,9

30

$8

48

,77

8

$5

50

,02

3

$3

8,0

00

76 16 13 14 2

RESPONSE PROTECTION PREVENTION MITIGATION RECOVERY

$1

0,5

33

,09

2

$1

,12

4,0

51

$9

34

,72

6

93 11 17

COMPETITIVE COMPETITIVE

INTEROPERABILITY

COMPETITIVE CERT

14

Mission AreaType

Proposal type and primary mission areas are consistent with historical submissions.

Most CERT projects

were submitted by

Region 5.

Most interoperability

projects were

submitted by Region 3.

Response has historically been the

primary mission area addressed by

most proposals.

2019 - 121 Proposals Evaluated

15

121 Proposals Evaluated by Core Capability$2

,60

4,9

37

$2

,40

4,0

75

$1

,66

4,5

92

$1

,12

3,8

70

$9

61

,52

0

$5

31

,84

0

$4

39

,62

7

$4

11

,24

8

$4

04

,33

1

$3

34

,62

5

$3

12

,60

6

$3

01

,39

7

$2

24

,58

4

$1

50

,00

0

$1

50

,00

0

$1

00

,51

5

$9

0,0

00

$8

6,3

10

$8

5,1

38

$8

2,2

88

$5

8,0

00

$4

5,8

50

$2

4,5

17

5 15 14 12 11 12 10 7 3 7 3 5 3 2 1 2 2 1 2 1 1 1 1

• Operational communications has historically been the core capability primarily

addressed by proposals

• Physical protective measures includes a $2.1M dollar project for barricades

• On-scene security, protection & law enforcement includes numerous vehicles

over $200K

• Operational coordination includes a $650k request for a mobile command post

& a $460K request that includes a $370K vehicle.

2019 - 121

Projects

Evaluated

Core

Capabilities

16

2019 - 121 Proposals Evaluated by Region

$3

,73

6,8

68

$2

,48

3,9

80

$1

,59

6,4

97

$1

,49

9,1

03

$1

,14

9,6

26

$1

,13

8,1

52

$9

87

,64

3

29 19 25 11 12 10 15

REGION 5 -

TIDEWATER

REGION 6 -

ROANOKE AREA

REGION 1 -

RICHMOND

REGION 7 -

NORTHERN

VIRGINIA

REGION 4 -

SOUTHWEST

REGION 3 -

CENTRAL

VIRGINIA

REGION 2 -

CULPEPER

Historically Region 5 submits the most proposals, typically followed by

Region 1. Region 6 stands out for a marked increase in number of

proposals and dollars requested. Contributing to the dollar request is a

$650K mobile command post.

17

2019 Stakeholder Participation181 stakeholders participated in the annual workshop, peer review, or SME review.

• Workshop: 65

• SMEs including 1 LE per SME group: 48

• Peer reviewers: 68 – The majority of peer reviewers have historically come from Region 5

17

12 119

8 7

4

68 Peer Reviewers by Region

32

25

74

EM FIRE EMS LE HHS

68 Peer Reviewers by Discipline

Stakeholder Participants:

2017 – 173; 2018 – 185

0%

20%

40%

60%

80%

100%

$0 $2,000 $4,000 $6,000 $8,000

Thousands

Cost Benefit versus Benefit Only

Benefit Cost Benefit Only Cutline

18

• Projects were ranked in two ways, by

benefit score only, and by benefit

cost ratio. Historically benefit/cost

ranking of projects has

outperformed benefit only.

Benefit/Cost has allowed more

projects and greater overall benefit

to be achieved than by benefit

alone.

• Benefit Cost ranking funds 54 of the

projects and achieves 76% of the

cumulative benefit value.

• Benefit only ranking funds 27

projects and achieves 44% of the

cumulative benefit value.

2019 Results – 72 Proposals for $6.3M Ranked by Cumulative Benefit versus Benefit Cost

54 “In” 18 “Out”

27 “In” 45 “Out”

19

2019 Observations• Criteria that preformed the best to worst in 2019

• Vetted Regionalism

• Scope

• Project Management

• Risk

• Benefit

• Resource sharing

• Sustainment

• Capability Linkage

• Whole of Community

• Performance Measurement Plan

• Project Replication

• Evaluators tend to comment on non-existent or

vague responses to questions and no supporting

documentation

• Historically projects over $100K have less of a

chance of getting funded.

17%15%

14%12%

11%9%

8%6%

5%3%

2%

SHSP 2019 Scoring Criteria Weights

Non-Regional Criteria

Regional Criteria

20

2019 Scoring Criteria (1-7 of 11)

No

n -

Re

gio

na

l Cri

teri

a

Scope - The proposal identifies specific jurisdiction(s), the impact to the jurisdiction(s) & how the interaction

occurs.

Capability Linkage - The project links to core capabilities and preparedness goals with a nexus to terrorism.

Sustainment - Consider whether this project sustains or enhances a current project or if it is a new project. If

the project is new, the proposal describes how any equipment, licenses, training and other features will be

maintained and upgraded after the period of performance? If this is sustainment of a current project, the

proposal describes the outcomes of the previous period(s) of performance and ongoing sustainment plan.

Whole of Community - The project addresses for example – Public-Private Partnerships, State-Local

Partnerships, access, and functional needs?

Project Management - The proposal explains how the project will be managed; how contracts will be

managed; how accountability to timelines and grant rules will be monitored and deficiencies corrected. A

timeline is provided from grant award to completion. The proposal indicates if SHSP projects funded in the

past three years were completed and provides an explanation if the project was not completed.

Performance Measurement Plan - The proposal identifies how the project's results will be evaluated and

who will evaluate them? The proposal describes the overall results that the project is expected to accomplish

in qualitative and/or quantitative terms. Some descriptions could include performance measures, national

standards, and core capabilities.

Project Replication - Can the project be easily replicated beyond the initial scope or area of initial concern

(e.g. training, planning or some other document asset that can be implemented elsewhere)?

21

2019 Scoring Criteria (8-11 of 11)

Re

gio

na

l Cri

teri

a

Risk - The project links to known/emerging risk with a nexus to terrorism. The project fits state, region, local,

and tribal priorities and/or established documented risk. Documentation is provided as appropriate. (e.g.

THIRA, Risk Assessments, Emergency Operations Plan, etc.). The project addresses risk in terms of threat,

vulnerability and consequence.

Benefit - The project benefits the jurisdiction, region and state. The project benefits the community, region,

staff and other stakeholders.

Vetted Regionalism – 1) The proposal identifies a capability gap, 2) the project addresses the gap, and 3) the

project includes letters of concurrence from jurisdictions identified in scope. Y/N (All three must apply to

achieve 100 points). 50 points may be given to projects not completely satisfying the #3 requirement if the

project aligns with regional priorities.

Resource Sharing - The project has a credible plan to share resources. The plan has documentation (e.g.

MOUs, contracts, etc.) that demonstrates collaboration and/or agreement from multiple jurisdictions, and/or

multiple regions, and/or multiple disciplines (e.g. fire, police, emergency medical services).

2019 Scoring Criteria

22

17%15%

14%12%

11%9%

8%6%

5%3%

2%

SHSP 2019 Scoring Criteria Weights

Non-Regional Criteria

Regional Criteria

0 5

3040

50

90100

0

100

0

50

100

0 1 2 3 4 5 6

Sco

re V

alu

e

Score

Criteria Score Values

All Other Criteria Vetted Regionalism

Total score is the weighted average of score values

Group DiscussionGRANT TOPICS

EVALUATION CRITERIA

23

Group Discussion• Assign a group leader to report out

• Assign a scribe to take notes on flip chart

• You will be guided through open discussions in which you will make recommendationsfor the 2020 grant cycle

Recommendations that can be finalized today or finalized in near future – Will be

Any recommendations that will need additional research – Will be documented and addressed in next grant cycle as appropriateaddresin FY20 cycle

24

Pre-submitted stakeholder Questions?

25

• There should be a conversation on several recurring "sustainment" projects, like firefighting

foam and training, that keep coming up under the competitive process each year, with

consideration on moving these items to non-competitive funding. There should also be a

requirement for regional socialization of projects across existing applicable regional

commissions/committees. I look forward to discussing these ideas at the workshop.

• Explanation of the Bumping of projects last year to allow for the Law enforcement percentage to

be met.

• Others?

Scoring CriteriaOpen Discussion

• Topic 1: Do we have the correct criteria?

• Topic 2: Are the criteria defined satisfactorily?

• Topic 3: Do we want to keep the same weights?

Activity

• Weight criteria by Individually ranking the criterion importance 1-n (1 is most important).

Note: More than one criterion can be ranked the same. For example, if you think two of the criteria are number 1 in importance then rank them both 1. Then score the criterion of next importance 2.

https://odu.co1.qualtrics.com/jfe/form/SV_dgSlZv4XDXz6zyZ

26

Next Steps

27

Activity Start End

Proposal Submission February 25 April 24

Peer Review Sign-up December 16 April 24

Regional Meetings March 9 March 26

Peer/SME Review May 1 May 15

Awards announced October 1

Grant period start date October 1

Adjournment

28

BACKUP

29

Work Groups Needed

• Non-Profit Grant Review – individuals identified

• Investment Justifications – work group needed

• Contract special ops – Response Programs – Special Ops

• Non-contract special ops – SME group needed

• Interoperability review – established, need to communicate with

• Regional SME review group – established by CRC

• Cyber security investment review – Tom Crabbs

Feb April May June July

FEMA releases

grant guidance

mid April.

Stakeholder Meeting held in

Richmond and across the

Commonwealth 4/11-4/25

Project reviews and

approvals

Investments are

prepared and

submitted to FEMA

Project development

period. Proposal due

May 15 – (tentative)

FEMA approval of the

Grant and Awards

made for projects

September

Aug - Sept

PROJECTED

Timeline 2019

Marc

h

Project

Submission

timeline will be

announced

through the

grants list

serve and

regional offices

Grant Reporting

Grant Awarding Process

◦ After award is made there is a 60 day period for submitting budgets and final documents

◦ 25% are received within 60 days

◦ 50% – 75% - received within 120 days

◦ 75% - 95% - received within 6 months

◦ 96% - and above – over one year

Quarterly Progress Reports

2013 Quarterly Progress Reports were made mandatory

Currently less than 60% are being submitted on time

32

Funds Returned

$6,472,244

$3,297,807$4,157,368

$4,931,200

$838,345

$645,343$140,523

$396,000

$0

$1,000,000

$2,000,000

$3,000,000

$4,000,000

$5,000,000

$6,000,000

$7,000,000

$8,000,000

2011 2012 2013 2014 (P)

Returned

Awarded

33

34

Projects Eliminated Low Scoring, Low BCR

Projects Set Aside Funds Unavailable

Model Calculation

Eligibility Screening

AdjudicationFunding Decision

Proposal Submission

Stakeholder Workshop

Projects Submitted to FEMA for Funding

Local Competitive

PEER/SME Review

State Share Alternative Process

FEMA NOFO

AnalysisHigh Rank, Funds Available

35

2020 Criteria Ranking Sheet

Enter the importance of each criterion from 1...n, 1 being the most important. More than one

criterion can be ranked the same. For example, if you think two of the criteria are number 1 in

importance then rank them both 1. Then score the criterion of next importance 2.

Stakeholder Name:

Rank Number Criteria Name

Risk

Benefit

Vetted Regionalism

Resources Sharing

Scope

Capability Linkage

Sustainment

Whole of Community

Project Management

Performance Measurement Plan

Project Replication

Gra

nt

Pro

cess

20

12

-Pre

sen

t

36

1) Grant guidance issued; emphasis on sustainment of current capabilities

•2012-2014 - by senior leadership

•2015-Present - by PS stakeholders at workshop

2) Scoring criteria and criteria weights determined

3) Statewide project proposals submitted via web

•2012-2013 – SMEs made recommendations, and project managers presented proposals to senior leadership. Senior leadership then scored proposals.

•2014 - Public safety stakeholders individually scored proposals that were randomly assigned

•2015-Present - Public safety stakeholders individually scored non-regional criteria. SMEs scored regional criteria for their region. In 2017 interoperability projects regional criteria were scored by SIEC

4) Projects evaluated

5) Analysis and benefit-cost ranked results presented to senior leadership

•2012-2013 - Senior leadership made the final funding decision

•2014-Present - Funding decisions were made based on project benefit-cost ranking

6) Final Adjudication

Criteria weights impact model output

37

Non-regional weighted more Regional weighted more

Example of Cost by Groups & Weighting Cost

38

Group # of Projects $ Amount(s)

I 1 $250K-$500K

II 15 $75K-$249K

III 5 $20K - $74K

Benefit 100 100 100 100

Cost $100 $75 $50 $25

Ratio 1.0 1.3 2.0 4.0