IMPROVING WATER QUALITY - World Resources Institute · IMPROVING WATER QUALITY: ... methods to...

WORKING PAPER | January 2014 | 1

Working Paper

IMPROVING WATER QUALITY:A REVIEW OF THE MISSISSIPPI RIVER BASIN HEALTHY WATERSHEDS INITIATIVE (MRBI) TO TARGET U.S. FARM CONSERVATION FUNDS

MICHELLE PEREZ AND SARA WALKER

CONTENTSSummary ...........................................................................1

Introduction........................................................................3

Background ........................................................................5

Methods .............................................................................6

Findings and Discussion....................................................8

Conclusion and Recommendations ..................................24

Endnotes ..........................................................................30

References ........................................................................32

Appendix: Assessment Factors, Criteria, and Ratings .......34

Disclaimer: Working Papers contain preliminary research, analysis, findings, and recommendations. They are circulated to stimulate timely discussion and critical feedback and to influence ongoing debate on emerging issues. Most working papers are eventually published in another form and their content may be revised.

SUMMARYHistorically, federal conservation programs have focused on solving environmental and natural resource problems on individual farms. While improvements have been made in water quality and wildlife habitat at the farm scale, landscape-scale environmental benefits in streams, lakes, and bays, for example, are less commonly documented. Excess nutrients (nitrogen, N, and phosphorus, P) con-tinue to impair thousands of waterways, and eutrophica-tion leads to hypoxia (low oxygen levels that harm aquatic life) or dead zones in water bodies around the country.

Currently, approximately1 10 percent of the U.S. Depart-ment of Agriculture’s (USDA) Natural Resource Conser-vation Service’s (NRCS) conservation budget is spent on targeting conservation efforts in high priority areas to achieve environmental outcomes at the landscape scale (i.e., across a geographic region facing similar water quality issues such as a watershed). However, focusing more conservation efforts in this manner, as opposed to the predominant approach, which disperses rather than concentrates funds across farms in each state, has the potential to achieve greater environmental improvements per dollar spent. In 2009, NRCS launched the Land-scape Conservation Initiatives to more effectively address priority environmental and natural resource concerns by focusing on the most important geographic areas. These initiatives hold great promise for cost-effectively achieving significant outcomes at the landscape scale.

The World Resources Institute (WRI) reviewed the Missis-sippi River Basin Healthy Watersheds Initiative (MRBI), one of NRCS’s largest water quality-focused Landscape

Suggested Citation: Perez, Michelle, and Sara Walker. 2014. “Improving Water Quality: A Review of the Mississippi River Basin Healthy Watersheds Initiative (MRBI) To Target U.S. Farm Conservation Funds.” Working Paper. Washington, DC: World Resources Institute. Available online at: wri.org/publication/MRBI

2 |

Conservation Initiatives to determine how well it was designed to achieve measurable improvements in water quality. WRI selected MRBI because of its focus on local water bodies in the Mississippi River Basin that drain into the Gulf of Mexico, the country’s largest water body suffer-ing from hypoxia.

MRBI is a Cooperative Conservation Partnerships Initia-tive (CCPI)2 that strives to accelerate adoption of con-servation practices to achieve landscape-scale outcomes in high priority locations by leveraging financial and technical resources from project partners. Such partners include state and local agricultural and environmental agencies, universities, soil and water conservation districts (SWCDs), and nongovernmental organizations (NGOs). Given that it is too soon to evaluate the achievement of measurable improvements in water quality, WRI focused its analysis on the design of MRBI and the awarded pro-posals from 2010 and 2011.

WRI reviewed the literature and interviewed experts3 to produce six factors deemed to be indicators of effective targeting: stakeholder and producer buy-in; presence of specific, measurable, achievable, results-oriented, time-bound, and quantitative (SMART-Q) goals; geographic targeting; monitoring and evaluation; cost effectiveness; and adaptive management. WRI then developed between one and four criteria for each of the six targeting factors to assess the degree of fulfillment of each factor. Most, but not all, of the targeting factors and their criteria reflect what NRCS requires or encourages in its request for pro-posals (RFPs) for MRBI projects. Cost effectiveness was not reflected in the RFPs, and adaptive management was mentioned indirectly.

WRI reviewed MRBI at the programmatic level and project level for the presence of these targeting factors and their associated criteria. WRI assigned the following ratings based on the degree to which MRBI, as a whole, addressed the factor’s criteria, where—

5 = Exemplary. All criteria for the factor are met.

4 = Good. Most criteria for the factor are met.

3 = Fair. Some criteria for the factor are met.

2 = Poor. Few criteria for the factor are met.

1 = Very Poor. No criteria for the factor are met.

To assign the ratings, WRI reviewed MRBI’s available literature (Web sites, press releases, annual reports,

and RFPs), evaluated 60 percent of the 2010 and 2011 project proposals, and interviewed NRCS staff associated with the initiative.

To assess how effectively MRBI is designed to target available funds to achieve landscape-scale water quality outcomes, WRI defined effective targeting. WRI con-cluded that effective targeting, in general, and also specifi-cally by MRBI, would result in achievement of improved landscape-scale water quality outcomes associated with reduced nutrient and sediment pollution, such as reduc-tions in in-stream N, P, and sediment concentrations. Given that press releases and RFPs for MRBI state that the initiative was developed to address local and Gulf of Mexico water quality problems and that measurable conservation results would be evaluated on a watershed basis, WRI believes this vision of effective targeting is also shared by NRCS.4

WRI commends NRCS’s efforts to begin focusing conser-vation funds to maximize water quality outcomes through this partnership-based, targeted watershed project approach. Overall, MRBI received an average rating of “fair” across the six factors we used to rate how likely the program’s design is to achieve improved landscape-scale water quality outcomes for nutrient and sediment pollu-tion. MRBI excelled at geographically targeting conserva-tion activities, receiving a “good rating.” It received “fair” ratings for including stakeholders and producers in MRBI, for setting SMART-Q goals, for measuring and evaluating progress, and for reflecting principles of adaptive manage-ment. It received a “poor” rating for cost effectiveness. These ratings are described in more detail throughout this report.

Based on these findings, WRI has identified specific recommendations for MRBI that may assist NRCS and its project partners in achieving measurable improvements in landscape-scale water quality outcomes.

RecommendationsStakeholder and Producer Buy-in1. Clarify which stakeholders are involved in what aspects

of MRBI. 2. Enable agricultural producers and rural landowners

to participate in the development of MRBI and the projects.

3. Prioritize awards to future projects that leverage and formalize significant resources from non-USDA sources.

A Review of the Mississippi River Basin Healthy Watersheds Initiative


SMART-Q Goals4. Lead by example and write a clear and SMART-Q goal

statement for MRBI that aims to achieve landscape-scale outcomes and require projects to do the same.

5. Prioritize future MRBI funds to those projects that aim to achieve already existing landscape-scale policy objectives.

Geographic Targeting6. Provide “targeting narratives” for the MRBI-designat-

ed focus areas and the MRBI project watersheds.

Monitoring and Evaluation7. Improve leadership and accountability for landscape-

scale outcomes. 8. Establish advisory teams for water quality monitoring,

metrics, and modeling. 9. Prioritize projects with already existing baseline

monitoring data or that propose to use a paired watershed approach.

10. Consider requiring watershed-based planning to help ensure attainment of improved landscape-scale water quality outcomes.

Cost Effectiveness11. Require projects to provide at least a narrative dis-

cussion about the cost effectiveness of their targeted watershed projects, and accelerate improvement of methods to quantitatively estimate cost effectiveness of practices and projects.

Adaptive Management 12. Develop a formal framework on adaptive management

to more effectively implement MRBI, and require projects to include plans for adaptive management in their proposals.

INTRODUCTIONFor more than three decades,5 federal conservation pro-grams have helped provide solutions to environmental and natural resource problems associated with individual fields and farms. Farmers have adopted scores of conser-vation practices with financial and technical assistance from federal, state, and local conservation programs, university extension programs, and nongovernmental organizations (NGOs). These practices have addressed soil erosion, manure management, nutrient management, pesticide management, wildlife habitat, and other envi-ronmental and natural resource concerns. On individual farms, there is evidence that on-site environmental prob-lems are being solved, and cleaner water and more wildlife habitat are being provided (see Figure 1).6

Figure 1 | Before and After Conservation Practice Implementation

BEFORE AFTER

The business-as-usual approach to federal conservation programs works by solving individual water quality problems on individual farms. On a field in Stark County, Ohio, the “before” photo indicates that the farm’s dairy cows were walking outside of the access road, tearing up the field, causing gully erosion and surface water runoff. In the “after” photo, severalconservation practices were installed to solve the livestock, erosion, and runoff problems, including a dairy cow access road, an electric fence, and a grassed waterway.

Source and photo credit: NRCS

4 |

However, water pollution, particularly eutrophication, the condition wherein excess nitrogen (N) and phospho-rus (P) result in algae blooms that create hypoxic areas or dead zones with insufficient oxygen to support aquatic life, remains one of the most widespread environmental problems in the country. More than 15,000 water bod-ies remain “impaired” for nutrients,7 and for the majority of these, agriculture is listed as the first or one of the top sources of these impairments.8

Historically, U.S. Department of Agriculture (USDA) has worked primarily on solving water quality problems within individual farm boundaries, and therefore the focus has not been on generating or assessing landscape-scale environmental outcomes (e.g., delisting an impaired water body). Working at the landscape scale to improve water quality involves prioritizing watersheds and specifi-cally, subsections of watersheds from which the majority of pollutants are delivered and are thus in need of the greatest conservation. By targeting funds to these areas, the collective efforts of individual farm conservation efforts are more likely to lead to in-stream water quality improvements.9

USDA’s primary approach to aid individual farms often results in conservation practices being implemented on farms that are dispersed across the landscape, an approach that helps address individual resource concerns but does not ensure that those individual efforts will col-lectively amount to solving priority water quality prob-lems. By concentrating more resources in high priority sections of a watershed, better environmental outcomes can occur for the same or lower costs (see Figure 2).

The business-as-usual approach to conservation funding depicted in the left panel in Figure 2 disperses conserva-tion practices to a few fields located in various places on the landscape, both inside and outside of a watershed that is known to have an impaired river segment (depicted by the fish symbol and red stream coloring). Thus, the busi-ness-as-usual approach is solving environmental prob-lems on individual farms and fields, but it is not directly addressing the impaired river segment.

In contrast, a targeted approach to conservation funds, as depicted in the right panel in Figure 2, indicates that a decision has been made to concentrate conservation funds

Figure 2 | Business as Usual vs. Targeting Conservation

BUSINESS AS USUAL Solves individual farm water quality problems

TARGETING CRITICAL SUB-AREASSolves landscape-scale water quality problems

BUSINESS AS USUALSolves individual farm water quality problems

The business-as-usual approach (left panel) disperses conservation projects across the landscape, while a targeting approach (right panel) concentrates conservation projects upstream from the impaired water body to improve water quality.

Source: WRI

BUSINESS AS USUAL Solves individual farm water quality problems





directly above the impaired stream segment. By imple-menting “the right practices, in the right places, at the right scale, and at the right time,”10 the impaired stream segment can become cleaner over time and able to host many more fish and other aquatic organisms.

After 30 years, only a handful of measurable landscape-scale environmental outcomes have been documented in the United States, largely due to a lack of large-scale agri-cultural conservation targeting efforts. For example, the U.S. Environmental Protection Agency (EPA) Nonpoint Source Control Program that leads targeted watershed projects to clean up impaired water bodies has credited USDA financial and technical assistance to farmers as key factors in about one-third of the EPA’s 443 success stories of partially or fully restored nutrient-impaired water bod-ies.11 In addition, several university studies have docu-mented increased populations of several endangered or threatened bird species due to concentrating funds from federal conservation programs in priority wildlife areas.12

In 2009, USDA’s Natural Resources Conservation Service (NRCS) launched a series of partnership-based, land-scape-level targeting initiatives to address water, wildlife, and ecosystem resource concerns.13 This effort to better target conservation funds to achieve environmental out-comes could improve the effectiveness of federal conserva-tion programs that seek to concentrate funds in particu-lar watersheds or other geographically defined areas to achieve environmental outcomes at the landscape scale.

The purpose of this working paper is to assess how well the Mississippi River Basin Healthy Watersheds Initiative (MRBI), one of more than a dozen NRCS Land-scape Conservation Initiatives,14 is designed to achieve landscape-scale water quality outcomes.

The World Resources Institute (WRI) focused on MRBI for two reasons: First, it is a large, landscape-scale con-servation effort focused on improving water quality that is impaired due largely to nutrient and sediment pollu-tion from agricultural sources. Second, MRBI is associ-ated with the Gulf of Mexico, the largest water body in the United States that is suffering from hypoxia. Decades of data and modeling analysis attribute 70 to 80 percent of the nitrogen and phosphorus in the Gulf to agricultural activities in the Mississippi River Basin.15 Thus, WRI’s analysis of MRBI is restricted to the effect that the initia-tive can have on water quality (rather than also looking at its impact on wetlands or wildlife habitat, the other two initiative concerns).

WRI assessed MRBI against six indicators that were identified as being critical factors to achieving environ-mental improvements: stakeholder buy-in, goal state-ments, geographic targeting efforts, monitoring and evaluation efforts, demonstration of cost effectiveness, and adaptive management principles. This review pro-vides a snapshot of how well-designed the initiative and its projects are, as it is too early to evaluate environmen-tal outcomes. Nevertheless, the analyses will be useful to NRCS’s initiative leaders, the targeted watershed project leaders, and policymakers who want federal conservation programs to help achieve landscape-scale water quality improvements. We hope such efforts will help demon-strate that scarce taxpayer funds are being spent in the most cost-effective manner.

BACKGROUNDNRCS launched MRBI in 2009 under the Cooperative Conservation Partnerships Initiative (CCPI), which was authorized by the 2008 Farm Bill.16 WRI was particu-larly interested in the CCPI projects due to their primary focus on water quality rather than wetland protection and wildlife habitat. CCPI is a 2008 initiative that “is designed specifically to foster coordination with other partners, and to encourage additional non-federal investment in natural resource conservation through the use of in-kind services or matching funds in a geographic area, such as a watershed.” In so doing, CCPI projects aim to “acceler-ate conservation assistance to improve water quality, protect, restore, and enhance wetlands, enhance wildlife habitat;….”17 By statute, CCPI is allocated about 6 percent of NRCS’s funds for some of its working lands conserva-tion programs18—the Environmental Quality Incentives Program (EQIP), Wildlife Habitat Improvement Program (WHIP), and Conservation Stewardship Program (CSP)19—which has been estimated to translate roughly to more than US$100 million per year.20 CCPI provides this money to farmers and landowners in the form of financial assis-tance to implement and maintain conservation practices.

CCPI does not provide any technical assistance funding to the project partners (e.g., soil and water conservation districts, SWCDs, and nonprofit conservation groups) to implement the project or to measure outcomes. Instead, CCPI relies on project partners to cover their own staff time by leveraging existing or new funding from other sources like state conservation and environmental pro-grams, charitable foundations, or other sources of funding from farm and conservation NGOs.

6 |

MRBI has committed about $80 million per year over four years21 for voluntary CCPI, Wetlands Reserve Enhance-ment Program (WREP), and Conservation Innovation Grant (CIG) projects in priority watersheds located in 13 key states: Arkansas, Illinois, Indiana, Iowa, Kentucky, Louisiana, Minnesota, Mississippi, Missouri, Ohio, Ten-nessee, South Dakota (beginning in 2011), and Wiscon-sin.22 In 2009, NRCS state conservationists, state technical committees,23 and state water quality agencies selected 43 designated focus areas (8-digit HUC watersheds)24 within the participating states, and partners submitted targeted watershed project proposals for sub-watersheds (12-digit HUC)25 within the designated focus areas. Fifty-eight MRBI-CCPI projects were funded in 2010 and 17 in 2011. Additional projects were awarded in 2012 but were not reviewed by WRI.

MRBI is exceptional in many regards but particularly in its focus on environmental outcomes. NRCS boldly stated in its 2010 request for proposals (RFP) that “…the Missis-sippi River carries an average of 436,000 tons of sedi-ment each day. Nutrients, like nitrogen and phosphorus, contribute to both local water quality problems and the hypoxic zone in the Gulf of Mexico. To help solve these water quality problems, NRCS developed MRBI.”26 The press release in 2009 emphasized that “The initiative is performance oriented, which means that measurable con-servation results are required in order to participate.” In addition, “Assessment of progress in implementing MRBI will be critical, as will evaluation of outcomes at the field scale/edge of field and on the watershed basis.”27

METHODSTo assess how effectively MRBI is designed to target available funds to achieve landscape-scale water quality outcomes, WRI defined effective targeting. Specifically, for MRBI, effective targeting would result in achievement of improved landscape-scale water quality outcomes associ-ated with reduced nutrient and sediment pollution, such as reductions in in-stream N, P, and sediment concentra-tions. Given that MRBI literature states that the initiative was developed to address local and Gulf of Mexico water quality problems and that measurable conservation results would be evaluated on a watershed basis, WRI believes this vision of effective targeting for achieving landscape-scale water quality outcomes is also shared by NRCS for MRBI.

Hence, the success of MRBI in achieving these landscape-scale water quality outcomes depends largely on the suc-cess of its targeted watershed projects. But for the projects to succeed and for their collective success to be evaluated and communicated to the public, there must be effective leadership at what WRI will refer to as the program level. The program level of MRBI refers to the NRCS staff that designed and continues to oversee MRBI. Among other things, the NRCS staff provides guidance to the targeted watershed projects through the RFP process. Conversely, efforts by the individual MRBI project leaders and part-ners of the 45 reviewed targeted watersheds will be referred to as being at the project level. WRI’s assessment reflects MRBI in its entirety, from the program level down to the project level.

Evaluation FactorsBecause at the time of this assessment, MRBI had only been under way for two years, it is too soon to assess out-comes, such as improvements in water quality, achieved at the landscape scale.28 Due to natural lag times in surface and subsurface flow from fields to streams to water-shed outlets, it could take years to observe water quality improvements at a watershed scale.29 Therefore, WRI identified factors related to the initiative’s design and the awarded project proposals, as a proxy for how likely they are to result in such targeting success in the future. These factors were culled from interviews with agricultural and water quality professionals30 as well as literature on topics relevant to targeting and water quality outcomes, including scientific, economic, and policy journal articles; governmental and nongovernmental reports; and news articles.

From this information, WRI identified six factors important to success in achieving landscape-scale water quality outcomes:

1. involving stakeholders and agricultural producers and achieving their buy-in;

2. setting policy-oriented goals that are SMART-Q (specific, measurable, achievable, results-oriented, time-bound, and quantitative);

3. employing geographic targeting of conservation efforts;

4. establishing monitoring and evaluation systems for assessing progress toward meeting goals;

5. being cost effective;

6. managing adaptively.



To assess MRBI’s programmatic efforts against the six factors, WRI reviewed MRBI Web sites and the available annual program performance reports. WRI also inter-viewed NRCS staff representing MRBI, the Landscape Conservation Initiatives, and EQIP, which is one of the largest program sources of funding for MRBI. In addition, WRI reviewed the 2010 and 2011 RFPs to determine what guidance and requirements the agency specified for its partners who implement the projects.

Rating SystemWRI then developed a review and rating system for the MRBI program and its projects, based on the six factors. For each factor, WRI identified important rating criteria that should be present in the program and/or project pro-posals. Checking for these criteria in initiative materials and project proposals ensured that MRBI would be rated as objectively as possible. This assessment uses five-circle symbols to illustrate the degree to which individual crite-ria were fulfilled by the initiative, where—

●●●●● = Exemplary. Criteria met.

●●●● = Good. Criteria mostly met.

●●● = Fair. Criteria met to some degree.

●● = Poor. Criteria largely not met.

● = Very Poor. Criteria not met.

Then, these criteria ratings were averaged to assign an overall rating per factor. The ratings for each factor were then again averaged to rate MRBI as a whole across all six factors, where—

5 = Exemplary. All criteria for the factor are met.

4 = Good. Most criteria for the factor are met.

3 = Fair. Some criteria for the factor are met.

2 = Poor. Few criteria for the factor are met.

1 = Very Poor. No criteria for the factor are met.

To calculate the overall rating for each of the six targeting factors, WRI took the mathematical average of each fac-tor’s criterion scores. WRI used standard rounding rules to round to the nearest whole number. Even though this paper uses the six factors to evaluate the success of MRBI, it should be noted that NRCS has not, thus far, clearly indicated that it shares the same vision of success for MRBI as defined by our six factors. Although

the factors largely represent the MRBI RFP requirements, WRI included additional elements that we believe are important for targeting success. Most notably, the RFPs did not require any discussion or analysis of cost effec-tiveness in the project proposals, nor did they require an adaptive management plan. As a result, our assessment holds the MRBI program and projects to higher standards than those currently set by NRCS.

Assessment MethodsTo assess the individual projects under MRBI, WRI ran-domly selected 60 percent of the awarded MRBI-CCPI project proposals from 2010 and 2011 (see Table 1 and Figure 3). WRI also reviewed the project performance reports that NRCS had available at the time of this study. Because project effectiveness could only be assessed based on how the proposals were written, WRI interviewed 13 leaders from five projects about implementation in the first year or two of the projects to gain some understand-ing of how implementation matched or diverged from what was intended and stated in the proposal. These interviews also shed light on obstacles faced by the projects. These findings are discussed in detail in a follow-up WRI paper on the barriers to targeting and how to overcome them.

Because MRBI is still a young program, the assessment is largely based on NRCS documentation. In some instances, it was unclear whether there were flaws in the design of the program or simply a lack of program and project docu-mentation. While we recognize that a lack of transparency does not equate to a lack of action, our assessment of MRBI is based on the readily available information. We hope that the results of this research and associated rec-ommendations may help improve how the MRBI program and project leaders design, implement, and evaluate the initiative and the projects.

NUMBER OF AWARDED PROJECTS WRI’S REVIEW

2010 2011 TOTAL # REVIEWED % REVIEWED

MRBI Projects 58 17 75 45 60%

Table 1 | Projects Reviewed for this Assessment

8 |

FINDINGS AND DISCUSSIONIn this section, findings for the MRBI program and its projects for each of the six targeting factors are presented and discussed.

1. Stakeholder and Producer Buy-inAddressing large, landscape-scale environmental prob-lems, such as water bodies impaired for drinking, swim-ming, or fishing, calls for flexible and transparent decision making by a diversity of stakeholders. By engaging stake-holders broadly, solutions to these problems are likely to be more sound and durable.32 Targeting works best when it is locally driven by landowners and other stakeholders who work together toward a common outcome and believe that they have ownership over the effort.33 Moreover, when farm producers are involved in identifying the envi-ronmental problem and planning the project to address it, the project is likely to see greater farmer participation.34 Finally, partnership approaches to conservation that lever-

age nonfederal funds are increasingly being recognized as important, due to their ability to empower local partners with decision-making abilities, accountability, and tools.35 By committing their own funds, project partners signal their support for the targeted watershed project solution to water quality problems. Based on this understanding, the MRBI program and projects were evaluated using the list of criteria in Box 1 for how well they attempt to engage stakeholders and producers.

FindingsWRI assigned a rating of “fair” for stakeholder and producer involvement. NRCS and its project partners have made commendable efforts to involve a diversity of stakeholders and leverage additional funding. MRBI could be improved by clarifying the affiliation and function of stakeholders in the development of the MRBI program and individual projects. MRBI could also make a greater effort to involve producers in the project planning pro-cess to gain their buy-in and ownership of the project and pursue formal relationships with entities able to provide

DEGREE TO WHICH EACH CRITERION WAS MET:

1a. The initiative does a good job involving relevant stakeholders. ••••

1b. Producers could be more involved in MRBI efforts. ••

1c. MRBI makes some effort to leverage nonfederal funds. •••

Overall rating: Fair

Table 2 | Stakeholder and Producer Buy-in Rating

1a. Relevant stakeholders were identified, analyzed, and represented systematically for all stages of the project and empowered to participate in decision making.

1b. Producers were involved in identifying the environmental problem and planning the project.

1c. Funding was leveraged from nonfederal partners.36

Box 1 | Criteria for Assessing MRBI’s Stakeholder and Producer Buy-in Efforts

Figure 3 | Number of MRBI Proposals Reviewed by WRI Out of the Number of Proposals Awarded Per State and Their Geographic Distribution31

WIMN

IA IL IN

MO

AR

LA

MS

TN

KY

OH

1/2

6/11

1/2

6/10

2/3

2/3

2/3

5/7

3/34/7

3/4

10/18

Source: WRI

Portions of MRBI states within the Mississippi River Basin

Portions of MRBI states outside the Mississippi River Basin



nonfederal funds to help ensure the success of the targeted watershed projects. Table 2 displays the degree to which the various criteria were fulfilled that led to the overall rating on this factor. Each criterion is discussed in detail in Table 2.

1a. Relevant stakeholders were identified, analyzed, and represented systematically for all stages of the project and empowered to participate in decision makingThe available information provided about the program indicates that several key stakeholders were involved in designing MRBI and in selecting the 43 priority focus areas, including staff from NRCS headquarters and state conservationists with input from the state technical com-mittees and state water quality agencies.37 Note that each state has varied membership on its state technical com-mittee, but generally the committee can include represen-tatives from federal, state, and local agricultural and water quality agencies; farm groups; environmental or conserva-tion groups; and university extension staff. If these indi-viduals are also members of each of the 13 state technical committees who participated in this process, then it is likely that many relevant stakeholders were involved in development of MRBI program.

However, the program has not provided a description of the stakeholders involved in each of the 13 states and the extent to which they were involved in the design of MRBI. Doing so would likely enable analysis of whether the right stakeholders were or are involved in the program and might encourage invitation of additional stakeholders that should be involved if obvious omissions are observed. Again, this might be occurring, but there is a lack of docu-mentation; or it may not be occurring.

Given the focus on improving water quality, it is com-mendable that each of the 13 state conservationists appear to have sought input from their state’s water quality agen-cies to select the designated focus areas for MRBI. How-ever, it is unclear if, during initial design of MRBI, NRCS staff reached out to counterparts at EPA or the U.S. Geo-logical Survey (USGS)—agencies specializing in similar efforts of watershed protection, restoration, and monitor-ing efforts—for feedback, guidance, or requests for assis-tance. As will be described in later sections, NRCS staff reviewed programmatic data from the EPA 319 nonpoint source program and scientific data from USGS regarding the Gulf of Mexico as inputs into the selection of the 43 focus areas, but additional interagency collaboration has not been described at the program level.

This ambiguity about stakeholder participation and defined roles in designing MRBI leads to questions about which individuals are serving which functions for the initiative and within each state (i.e., providing leadership, oversight and accountability, measurement, evaluation, reporting, and coordination). During the research and writing of this paper, NRCS hired a permanent MRBI coordinator, which seemed to put an end to the rotation of interim coordinators during the initiative’s first two years. However, NRCS did not announce which of the aforemen-tioned functions this permanent coordinator was empow-ered to provide. As a result, WRI is uncertain how much oversight and guidance the coordinator provides.

At the project level, both the 2010 and 2011 RFPs required project applicants to describe who their project partners are and what roles, responsibilities, and capabilities they are providing. WRI’s review of 45 project proposals sug-gests that nearly all projects listed a large and impressive array of stakeholders partnering to implement the water-shed project. The median number of partners per project was 9.5 with a range from 2 to 30. Partners included fed-eral, state, and local government institutions in both the agricultural and environmental sectors; universities; and farm or environmental/conservation NGOs. This diver-sity of stakeholders is key to ensuring that MRBI projects are designed and carried out by people with the skills and knowledge necessary to be successful. For example, 98 percent of proposals had partners with connections to producers, 99 percent had partners with water qual-ity monitoring skills, and all proposals had partners with conservation technical assistance skills.

Source: WRI

Figure 4 | Distribution of MRBI Project Leaders

2%

58%

4%

10%

20%

(N = 45 PROJECT PROPOSALS)

SWCD

NGO

State Agency

University

Other

10 |

to more effectively identify (a) where the resource con-cerns are, (b) what the likely causes and possible solutions are, (c) where the project efforts should be targeted, and (d) how the outcomes of the project should be monitored and measured—all key elements of effective conservation targeting.39

1c. Funding was leveraged from nonfederal partnersBoth 2010 and 2011 RFPs announced that MRBI would prioritize proposals that significantly leverage nonfederal financial and technical resources. The 2010 RFP stated that it would prioritize projects in the selection process that provided additional funding for conservation prac-tice payments (in addition to the federal funds provided by EQIP, CSP, or WHIP). The 2011 RFP went further to specify that such nonfederal resources could also provide for water quality monitoring and evaluation of conserva-tion practices and for coordination with other local, state, or federal efforts.40

Despite this encouragement, less than half of the reviewed projects had statements indicating that they would leverage nonfederal conservation funds from a variety of sources. Of the projects that did leverage additional funds, the median amount of matching funds represented 57 percent of the projects’ total budgets. This number is surprisingly low and indicates underreporting. Given that MRBI funding goes to producers (i.e., not to the project partners), projects with nonfederal partners have to lever-age nonfederal funds at least in the way of in-kind staff time. In reality, it is possible that this percentage is higher, but project leaders did not consistently describe or itemize these funds in their proposals.

Considering that federal conservation programs are not flush with funds, leveraging nonfederal funds is not only necessary to ensure that projects have adequate fund-ing but is also necessary to gain the buy-in needed by the many stakeholders critical to a targeted watershed proj-ect’s success. Some projects included plans to use extra funds to pay for additional in-stream water quality moni-toring stations or supplement conservation payments to install edge-of-field monitoring stations. Because NRCS MRBI funding does not provide funds to project partners for these tasks, other projects included statements that some leveraged funds would cover the staff time for stake-holders to implement the project by conducting outreach, providing technical assistance to producers, and providing monitoring or modeling services to measure and report on the outcomes of the project.

Regarding project leadership, just over half of the MRBI projects were led by local SWCDs (see Figure 4); then by farm or environmental NGOs; state agricultural, conserva-tion, or water quality agencies; or universities.38 SWCDs are important leaders of conservation efforts, as they have a long history of working one-on-one with farmers.

Despite projects having a plethora and diversity of stake-holders on board, less than half of the proposals specified that their project would involve stakeholders in decision making at the outset, through implementation of the proj-ect, and during the evaluation stage. This lack of specificity about stakeholder engagement is likely due to the fact that the RFPs did not specifically call for proposals to explain how stakeholders would be involved in decision making throughout the project life spans. This expectation could be included in future RFPs, and NRCS could inform cur-rent MRBI project leaders that they expect this issue to be discussed in the annual reports.

1b. Producers were involved in identifying the environmental problem and planning the projectThere is no evidence that NRCS engaged producers and landowners in the development or implementation of MRBI at the programmatic level, except to the extent that members of the state technical committees include producers and landowners. NRCS did require the projects in both RFPs to describe how project partners will encour-age producer participation to increase the likelihood of project success. But neither RFP called for producers to be involved in the project design process.

An impressive majority (87 percent) of reviewed projects did describe an outreach and marketing strategy to gain producer and landowner participation in the project. Many projects mentioned using radio and newspaper announcements, direct mailings, meetings, workshops, farm field days, and engagement through relationships with certified crop advisers, agricultural cooperatives, etc., to reach out to producers. However, less than half (44 percent) of the reviewed projects called for producer involvement in the design of the project and problem identification.

Involving producers directly—or indirectly through their farm trade associations—in problem definition, project planning, and decision making often leads to greater buy-in and participation from the key constituents in the targeted watershed project. Ensuring that the producers have seats at the decision-making table would help MRBI



2. Policy-Oriented Smart-Q GoalsWRI used SMART-Q criteria to assess the quality of the program and project goals. Guidelines for setting spe-cific, measurable, achievable, results-oriented,41 and time-bound (SMART) goals were introduced over two decades ago.42 WRI believes these indicators for success-ful objectives43 are appropriate for evaluating the goals set by MRBI. SMART goals help individuals, programs, and projects establish a clear direction, identify results, and perform at a higher level than would otherwise be achieved.44

WRI added a sixth letter, Q for quantitative, to the SMART mnemonic, given that modern water quality policy goals are increasingly quantitative (e.g., total maximum daily load (TMDL) allocations, numeric nutrient criteria, thresholds to remove water bodies from impaired waters lists). A quantitative goal statement places higher stan-dards on the measurable component of SMART goals. Although a goal to “improve water quality” is measurable, it could technically be met with only the slightest of water quality improvements.

WRI also assessed whether the MRBI program and proj-ects mentioned the presence of any water quality policy drivers, either in the form of water quality objectives (e.g., TMDLs, impaired water bodies lists, and the scientific

recommendation of a 45 percent nutrient reduction goal for the Gulf of Mexico).45 WRI noted if the program or the projects mentioned being designed to achieve some or all of the specific nutrient or sediment load reductions. The MRBI program and projects were evaluated using the list of criteria in Box 2 to assess how policy-oriented and SMART-Q MRBI’s goals are.

FindingsMRBI received a “fair” rating for having policy-oriented SMART-Q goals (see Table 3). MRBI lacked a clear and thorough goal statement and had different requirements in the two RFPs for SMART-Q goals. However, two-thirds of the reviewed projects wrote ambitious, outcome-ori-ented goals that contained all or most SMART-Q compo-nents (see Table 3).

It is important for NRCS to set an overarching goal for MRBI under which partners can design projects that work to address the initiative’s overall mission. More transpar-ent goals that foster better management and enable evalu-ation will include most or all of the SMART-Q elements. And ideally, if water quality policies or recommendations already exist in the targeted watersheds, projects should set goals that complement the existing water quality improvement efforts.

2a. Goals are SMART-QFor the MRBI program, although WRI discovered various objectives stated in a variety of press releases, Web sites, and other program literature, no single clear goal state-ment emerged that encapsulated the purpose of MRBI or the overarching goal under which the projects should be


2a. Both the program’s and the projects’ stated goals are somewhat SMART-Q. ••••

2b. MRBI could more systematically link its efforts to overarching water quality policies and recommendations. ••


Table 3 | Policy-Oriented Smart-Q Goal Rating

2a. Goals are SMART-Q.46

Specific: Goals state who is involved, what will be accomplished, where, and by when.

Measurable: Goals provide criteria to assess progress toward specified goal(s).

Achievable: The necessary skills, finances, and abilities are in place, and the project has a clear plan of action.

Results-oriented: Goals focus on the desired end result(s) of the activities (i.e., the outcomes), not on the activities that will be implemented.

Time-bound: Goals include a time frame for completion.

Quantitative: Goals have a numeric target that the project is aiming to achieve.

2b. Goals aim to address already existing water quality policies or recommendations.

Box 2 | Criteria for Assessing MRBI’s Policy-Oriented SMART-Q Goals

12 |

degree to which the goal statement was results-oriented, or outcome-based. In our assessment, we considered four types of goals, which are summarized below and explained in more detail in Box 3.

Output goals aim to implement conservation practices

Interim outcome goals aim to achieve in-field or edge-of-field environmental improvements

Environmental outcome goals aim to achieve in-stream water quality improvements

Ecological balance outcome goals aim to achieve in-stream water quality improvements beyond a specified threshold needed for achieving ecological restoration

Because of MRBI’s stated intention “to improve the health of watersheds,” WRI categorized this goal as being an environmental outcome goal. The remainder of the state-ment, however, focuses more on the necessary outputs (e.g., implemented practices) and interim outcomes (e.g., avoidance, control, and trapping of nutrient runoff) than a desired environmental outcome at the landscape scale. In WRI’s view, the initiative should enumerate a specific,

designed. The following statement from the 2010 MRBI RFP was selected to serve as the initiative’s goal statement because WRI believed it best represented MRBI:

“To improve the health of the watersheds within the Mississippi River Basin, NRCS and its partners will help producers to voluntarily implement conservation practices that avoid, control, and trap nutrient runoff; improve wildlife habitat; restore wetlands; and maintain agricultural productivity.”47

MRBI’s goal statement had half of the desired SMART-Q characteristics—“specific” because it focuses on a subset of water quality concerns: nutrient pollution; “measurable” because it is arguably possible to measure reductions in nutrient runoff; and “results-oriented” because it seeks “to improve the health of the watersheds” through attainment of interim and environmental outcomes (reduce nutrient runoff, improve wildlife habitat, restore wetlands).

Given MRBI’s progressive design to achieve outcomes, rather than simply outputs (e.g., acres of conservation practices, contracts signed), WRI further analyzed the

The current business-as-usual measures of success associated with NRCS conservation programs (e.g., EQIP, CSP, WHIP) focus on administrative metrics such as the acres of installed practices or the number of signed contracts with producers. While these administrative metrics are important stepping-stone outputs, they are not the same as actual environmental and natural resource outcomes. Because a key purpose of MRBI is to achieve water quality outcomes at the landscape scale, the goals and corresponding metrics should reflect that ambition at both the programmatic level and the project level.

WRI built on definitions found in the scientific literature and in NRCS’s Landscape Conservation Initiatives literature to define four levels of goals based on the degree to which they focus on achieving long-term, water quality outcomes.

1. Output goals aim to achieve implementation of in-field and edge-of-field conservation activities (e.g., livestock exclusion, planting of cover crops) as measured by administrative metrics, such as numbers of acres receiving conservation treatment, or dollars spent.

2. Interim outcome goals aim to achieve in-field and edge-of-field improvements (e.g., reduction in manure application rates, reduction in sedi-ment runoff from edge of field) as measured by farm-site agricultural practice performance metrics. Interim outcome goals are short-term goals and can be tracked via modeling or monitoring.

3. Environmental outcome goals aim to achieve in-stream improvements (e.g., reductions in nutrient concentrations in streams, improved water quality) as measured by off-farm-site natural resource or environmental performance metrics. Environmental outcome goals are intermediate-term goals and can be tracked via modeling or monitoring. At this level, the focus of conservation effort is on changing behavior and outcomes on many agricultural fields with sufficient density and intensity of effort to result in improved water quality indicators in area streams.

4. Ecological balance outcome goals aim to achieve major thresholds of ecosystem health in water bodies (e.g., restoration of native species to stream, removal of water body from impaired waters list) as measured by off-farm-site natural resource or environmental performance metrics. Eco-logical balance goals are long-term outcome goals and can be tracked via modeling or monitoring. To achieve ecological balance goals, conserva-tion activities not only need to be dense and intense, they also need to be tied to specific thresholds for quantitative improvements in specific water quality indicators that must be met for the water body to be restored to ecological health.

Box 3 | Goals to Drive Water Quality Outcomes



amounts employed by each project, and because appli-cants must implement conservation practices in order to realize environmental outcomes, all projects understand-ably had output-related goals. Surprisingly, most (90 percent) of the projects also had environmental or interim outcome goals. This is a significant improvement in the approach to using conservation funds by NRCS and its partners. In the future, projects should also aim to achieve ecological balance goals for individual project watersheds.

The areas where projects consistently fell short, as would be expected, were the same areas where the RFP fell short in its guidance: lack of adequate timelines and quantitative targets.

2b. Goals aim to address already existing water quality policies or recommendationsAlthough MRBI’s objectives extend beyond water qual-ity (e.g., they also include improving wildlife habitat and maintaining agricultural productivity), WRI focused primarily on water quality in this review. The initiative’s goal is not currently tied to any water quality policies, such as local nutrient- or sediment-related TMDLs, nor is it associated with the Gulf of Mexico nutrient reduction goal recommended by the 2007 EPA Science Advisory Board.49 However, WRI believes that both these local policies and regional voluntary guidance should be priority factors for consideration when selecting designated focus areas or specific 12-digit HUC watersheds for targeted watershed projects and that the projects should aim to achieve such policy goals. Given that the Mississippi River Basin drains into the Gulf of Mexico, which suffers from hypoxia, this omission of directly relating MRBI to local and/or Gulf-impaired water goals is a significant missed opportunity for prioritizing use of targeted conservation funds to directly addressing these highest priority water quality issues. Although the RFPs did not ask for discussion of these local or regional water quality policies or guidance goals, surprisingly, the majority of proposals (80 percent) did mention the presence of a policy driver in their project watersheds. However, only a few (33 percent) of the proj-ects planned to directly address the policy goal through their conservation activities, and even fewer (16 percent) mentioned the quantitative nutrient or sediment load reductions called for in the policy driver. MRBI projects could significantly improve their use of taxpayer dollars if the projects aimed to achieve existing water quality improvement policies or regional recommendations.

quantitative, water quality-related, ecological balance goal for the impaired local streams that feed into the Missis-sippi River Basin and Gulf of Mexico.

However, the initiative’s current goal statement lacks achievable, time-bound, and quantitative elements. Given the ambiguity about what it means “to improve the health of the watersheds,” it is unclear if the goal is achievable. There is no mention of a time frame for accomplishing the goal, and there is no quantitative target associated with this goal that enables all observers to recognize when the goal has been reached.

In the 2010 RFP, NRCS required projects to describe SMART objectives while the 2011 RFP only required SMAR objectives (dropping the “time for completion” criterion, although, later in the RFP, requiring projects to state an implementation schedule when different objec-tives and conservation practices will be completed). In both RFPs, NRCS required that project duration could not exceed five (2010 RFP) or four (2011 RFP) years in length. However, neither RFP discussed the need to maintain the installed conservation practices and to monitor water quality for the desired improvements beyond the four to five years of project implementation. While the project implementation was time-bound, the goals were not. And project life spans are likely not long enough to real-ize outcome goals. Future RFPs could clarify this timing distinction between changes in land management and water quality responses. More discussion on this topic is provided in the Monitoring and Evaluation section under criteria 4.c.

Though neither RFP required quantitative goals, the 2010 RFP gave greater priority to proposals that “Provide evidence of a watershed planning process which identifies quantifiable project goals for field-scale nutrient manage-ment and watershed-scale nutrient load reduction.”48 However, this element was missing from the 2011 RFP.

At the project level, about half of the project plans speci-fied quantitative interim outcomes or environmental out-comes (see Box 3 for definitions). Overall, WRI found that a third of projects met all of the SMART-Q characteristics, a third met most of the SMART-Q characteristics, and a third had only some of the SMART-Q characteristics.

WRI also analyzed the degree to which the project goals focused on outcomes as part of the results-oriented component of SMART-Q (see Box 3). Because the RFPs required descriptions of the priority practices and funding

14 |

3. Geographic Targeting Inherently, some farmland may be better suited to reduc-ing nutrient runoff than other farmland due to hydrologic and/or management factors. According to conservation experts, in some watersheds, these “hydrologically sensi-tive areas” may make up only 20 percent of a watershed but contribute up to 80 percent of the pollution in that watershed.50 By geographically concentrating practices in a watershed’s land area that is the most hydrologically sensitive, conservation funds and technical assistance can be strategically deployed to achieve specific, land-scape-scale water quality improvements that could not be achieved by randomly dispersing the funds across the entire watershed (see Figure 2).

Once these high priority areas are identified, it is then critical that conservation efforts occur with “the right practices, at the right scale, and at the right time,”51 meaning that the location and timing of the practice(s) is equally as important as the practice itself. Because NCRS has honed this mantra at the field and farm level by work-ing one-on-one with farmers through its conservation planning process, WRI was particularly interested if MRBI

facilitated application of this philosophy to the landscape scale. That is, does MRBI encourage selection of the right locations for the right practices at the right time to col-lectively achieve landscape-scale water quality outcomes? The MRBI program and projects were evaluated using the list of criteria in Box 4 for how well designed MRBI is to geographically target conservation efforts.

FindingsMRBI received a “good” rating for the geographic target-ing factor (see Table 4). The program provided evidence that important water quality datasets as well as institu-tional capacity factors were reviewed when selecting the 43 designated focus areas, and the majority of the projects made the case for selecting a specific watershed for the targeting project. Most projects mentioned specific pol-lutant sources and the conservation practices appropriate for dealing with them. Many projects also took the oppor-tunity to suggest new ranking criteria and use participa-tion incentives to gain involvement of the most effective producers and practices in the highest priority locations.To strengthen MRBI’s targeting efforts even more, the ini-tiative should provide to the public the rationale for each of the 43 designated priority areas and the specific water-sheds selected for targeted conservation projects. Also, MRBI projects could maximize water quality results if partners better identified critical sub-areas within project watersheds that can achieve the largest specific environ-mental benefits to the water body of concern.


3a. MRBI geographically targets conservation to priority areas though the targeting rationale could be more transparent. ••••

3b. Sub-watershed targeting by projects could be improved. •••

3c. Ranking criteria suggested by most partners. ••••Overall rating: Good

Table 4 | Geographic Targeting Rating

3a. Scientific evidence for targeting was provided to explain why project areas were selected. Possible reasons include—

Land rationale: Land can yield large environmental benefits (either remedial or preventive) and/or land has a severe environmental risk.

Water rationale: The water body is impaired and/or the water body has high resource value.

Capacity rationale: There is availability of resources and staff as well as agency and stakeholder willingness.

Achievability rationale: Water quality problem is deemed resolvable by the project.

3b. High priority areas within project watersheds were identified based on specific sources of pollutants, and appropriate controls were suggested.

3c. Eligibility requirements, participation incentives, or enroll-ment screens were proposed in order to encourage participa-tion by landowners who can be the most environmentally effective (due to location on the landscape, production practices, and conservation opportunities). 52

Box 4 | Criteria for Assessing MRBI’s Geographic Targeting Efforts



3a. Scientific evidence for targeting was provided to explain why project areas were selectedIn consultation with state conservationists, state techni-cal committees, and state water quality agencies, MRBI tapped into many resources to identify high priority water-sheds in which to carry out the initiative throughout the 13 participating states. For example, NRCS and the states used many sources of information related to nutrient issues and reviewed institutional capacity factors to select 43 eight-digit priority watersheds called “designated focus areas.” These data sources included—

NRCS Conservation Effects Assessment Project (CEAP) data;

USGS Spatially Referenced Regression on Watershed Attributes (SPARROW) model;

state-level nutrient reduction strategies and priorities; and

monitoring and modeling data on nutrient levels, as available.53

This bulleted list suggests that appropriate datasets were reviewed, and relevant factors such as monitoring capabil-ities and landowner buy-in were appropriately considered. Other useful data sources that were not mentioned but would strengthen the selection process include consider-ation of N, P, or sediment-related TMDLs. Although the above list refers generally to all of the 43 designated focus areas, WRI did not find specific justification for selecting each of the 43 watersheds individually. Because no expla-nations were given on why each focus area was selected, we could only assess the MRBI program based on the general rationale provided, not on whether the particular watersheds selected are most deserving.

Via the RFPs, MRBI asks potential partners to identify smaller, 12-digit HUC watersheds where the partners would focus their work within the initiative’s 43 larger (eight-digit HUC) focus area watersheds. MBRI also asks these potential partners to provide criteria and a rationale for choosing these specific 12-digit project watersheds. Most of the partners demonstrated a thoughtful approach to selecting their project watersheds, though some propos-als could be improved. For example, the vast majority (93 percent) of reviewed projects included scientific evidence, typically concerning high nutrient and/or sediment loads, to justify why the project watersheds were selected. Three-quarters (76 percent) indicated that partners chose their

project watershed because of the ability to yield large environmental benefits. Over half (60 percent) empha-sized the availability of staff and resources to participate in the project; while half mentioned the resource value of the water body to the public (49 percent) and the severity of risk (51 percent).

3b. High priority areas within project watersheds were identified based on specific sources of pollutants, and appropriate controls were suggestedMRBI applicants were required to describe the natural resource concerns that would be addressed and the con-servation practices that would be used to address these resource concerns.54 Furthermore, the 2010 RFP “encour-aged,” while the 2011 RFP “required” proposals to identify the sources of nutrients causing water quality problems. The 2011 RFP included stringent requirements that proj-ects submit detailed project area maps displaying specific locations in need of conservation treatment.

More than three-quarters of the reviewed projects identi-fied specific pollutant sources in their project watershed and described the conservation practices that offered the best controls for those pollutant sources. However, less than half of the partners provided evidence that their sub-watershed targeting was linked to the critical areas generating the majority of the pollutant(s). This finding is important because some studies suggest that as little as 15 percent of farms may contribute the majority of agricultural nutrient pollution.55 Because project water-sheds can range from 10,000 acres to 40,000 acres, a project’s efforts should be tailored to the appropriate scale for targeting the areas where the most severe resource concerns originate. Otherwise, project practices may not be “dense” enough (i.e., concentrated above the impaired water body) and “intense” enough (i.e., have sufficient numbers or acres of appropriate practices) to address the water quality impact (see Figure 2).

It is important to note, however, that a watershed that is completely homogeneous in terms of hydrology (e.g., the entire area may be tile-drained) and crop rotation (e.g., corn and soybeans only) may not warrant targeting to critical sub-areas. The challenge for these types of water-sheds will be to achieve significant and sufficient adoption of the appropriate conservation practice systems to yield water quality improvements that can be measured at the watershed outlet.

16 |

The RFPs encouraged adoption of the “systems-based approach to conservation” recently deployed by NRCS to identify the most effective suite of practices for a treated field to “avoid, control, and trap” nutrient and sediment runoff. This approach is new for NRCS as it comprehen-sively addresses nutrient pollution by treating pollutants at various points throughout their use and transit. Previ-ously, it was not uncommon for single-practice conserva-tion contracts to address only one aspect of nutrient pollu-tion. A large majority (80 percent) of the reviewed projects did identify the specific sources of pollutants and specific practice controls that would be targeted and implemented, respectively, through a systems-based approach.

3c. Eligibility requirements, participation incentives, or enrollment screens were proposed in order to encourage participation by landowners who can be the most environmentally effectiveFor targeted watershed project conservation contracts to gain higher priority, the RFPs required proposals to sug-gest new ranking criteria in line with MRBI’s objectives—in addition to those already used by the various funding programs (e.g., EQIP, CSP, and WHIP). About two-thirds (62 percent) of MRBI proposals took the opportunity to do so by tailoring ranking criteria (e.g., enrollment screens) to ensure selection of the best contracts in the highest priority locations.56 In addition, 73 percent of reviewed projects used incentives, such as higher cost share rates for priority practices, to encourage participation and entice producers in those high priority areas to adopt a systems-based conservation approach. This use of ranking criteria and extra incentives encourages participation and selection of producers who are most effective at generating environmental benefits.

4. Monitoring and Evaluation Conservation experts agree that a monitoring and evalua-tion (M&E) plan is crucial to ensuring good management of on-the-ground projects. M&E goes hand-in-hand with implementation to provide accountability for the effective-ness of conservation activities. By having an effective M&E system in place, partners can demonstrate results, and better decisions can be made about the future of the project based on the impacts achieved. 57 (Adaptive management as a response to M&E is further addressed in section 6.)

Experiences with landscape-scale water quality improve-ments have underscored the importance of watershed management planning. A review of the 1980 to 1990

Rural Clean Water Program (a joint USDA-EPA effort),58 which included 21 targeted watershed projects, concluded that watershed-based planning is a critical prerequisite for improving water quality conditions in local streams, rivers, or lakes.59 EPA’s 319 Nonpoint Source Control Program requires a specific watershed planning process in order to protect and improve water quality.

Although targeted watershed projects that do not have watershed management plans may be able to improve stream water quality, history suggests that they are more successful if a watershed plan is first developed, and then conservation strategies are implemented and continually evaluated and adjusted. Based on this understanding, the MRBI program and projects were evaluated using the list of criteria in Box 5 for MRBI’s M&E efforts.

FindingsMRBI received a “fair” rating for the M&E criteria. Although valiant efforts are being made to better monitor and assess progress, such as the new edge-of-field moni-toring protocols, satisfying the M&E criteria is still a work in progress (see Table 5).

4a. The program and the projects planned to collect and evaluate data along the way to determine if activities are helping to meet the goal(s). Alternatively, they planned to conduct an ex-post performance assessment after project completion to determine the impact of the activities, unrelated to any goals.

4b. Projects planned to collect data to establish either a mini-mum of two years of baseline information (i.e., pretreatment water quality conditions) from which to assess progress, or use a paired watershed approach.

4c. Projects planned to collect data to determine if the conserva-tion activities were installed appropriately (i.e., in accordance with standards and specifications) and maintained over the life of the project or contract.

4d. Projects planned to conduct watershed-based planning to increase the likelihood that landscape-scale water quality outcomes will be achieved.60

Box 5 | Criteria for Assessing MRBI’s Monitoring and Evaluation Efforts



There are several opportunities to improve the M&E efforts related to MRBI. The initiative should clarify which institutions (e.g., NRCS, other federal agencies such as USGS or EPA, state water quality agencies, universities, project partners) will provide accountability, leadership, and oversight for tracking progress at the edge of field, in stream, and watershed outlet. MRBI should develop advisory teams to further refine the monitoring protocols and evaluation procedures, metrics development, and modeling approaches to tracking progress. The MRBI pro-gram leaders and project partners should redouble their efforts to ensure that projects are establishing adequate baselines for all three tiers of monitoring. NRCS should consider requiring all future awarded MRBI projects to either carry out already existing watershed-based plans or to develop new watershed plans for priority watersheds. Short of requiring watershed plans, the initiative should require new project proposals to demonstrate how they will ensure achievement of “intentional conservation practice implementation” and “intentional water quality monitoring” and allow for longer project time frames to account for the natural delay in water quality responses to land use changes.

4a. The initiative and the projects planned to collect and evaluate data along the way to determine if activities are helping to meet the goal(s). Alternatively, they planned to conduct an ex-post performance assessment after project completion to determine the impact of the activities, unrelated to any goalsWith the development of MRBI, NRCS is now attempting to account for the effects of conservation practices on the landscape, in addition to on the farm.

Both RFPs asked for MRBI proposals to include a descrip-tion of a plan for “water quality monitoring, evaluation, and reporting on progress.”61 However, the RFP did not directly require projects to submit those plans or submit annual performance reports that highlight progress.62 Upon WRI’s request for the performance reports, NRCS supplied what it had received: just six project reports and two state-level summaries (Arkansas and Kentucky). Thus, without submission of annual performance information, NRCS may not be able to evaluate the collective, land-scape-scale results of the projects. Furthermore, WRI’s review of the available reports indicated that the projects were solely reporting administrative output metrics, such as the number of contracts and dollars spent, rather than reporting on monitoring data that had been or was being collected to evaluate water quality outcomes.

In reviewing the collective effort of NRCS and its partners, at both the programmatic level and the project level, to evaluate MRBI’s progress toward achieving landscape-scale outcomes, WRI was particularly interested in (a) the level of monitoring, (b) the water quality indicators, and (c) the modeling evaluations that were required or encour-aged and actually being implemented by projects.

MANY PROJECTS PROMISE MONITORING, BUT INITIATIVE HAD ROCKY START TO OVERSIGHT

MRBI signaled its commitment to achieving landscape-scale water quality improvements by stating in a press release that the “assessment of the progress in implement-ing MRBI will be critical, as will evaluation of outcomes at the field scale/edge of field and on the watershed basis.”63 This stance is reflected in both the 2010 and 2011 RFPs, which give higher priority to proposals that use a three-tier approach (i.e., plans designed to monitor and assess progress at the edge of field, in stream, and at the outlet of the 12-digit watershed). This comprehensive approach


4a. Process to track and report on progress remains unclear and incomplete although various monitoring efforts are under way and being improved.

•••4b. Due to NRCS oversight, MRBI projects are

now likely poised to collect effective and sufficient baseline information to enable progress tracking.

••••4c. Few projects are committed to checking long-

term maintenance of practices. •••4d. Less than half the MRBI projects are using

watershed management plans. •••Overall rating: Fair

Table 5 | Monitoring and Evaluation Rating

18 |

to water quality monitoring is ideal as it allows for prog-ress to be assessed at the landscape scale, versus solely edge-of-field (EOF) monitoring, which can only assess the effectiveness of treating runoff on one field.

WRI’s review of 45 projects indicates that about half the project proposals ambitiously aimed to accomplish all three tiers of monitoring (see Figure 5).

Although NRCS stated that “the MRBI is adopting a three-tiered monitoring and evaluation approach designed to assess outcomes at the field, 12-digit, and 8-digit water-shed scales,”64 the agency has restricted its involvement and oversight to only the first tier (EOF monitoring). This situation is understandable given NRCS’s focus on work-ing with producers at the farm scale, but it poses a number of challenges because NRCS has not designated another institution to be accountable for the in-stream and water-shed-outlet monitoring of MRBI projects. Without such a designee, there remains a lack of coordination on how and what the project partners should be monitoring and how NRCS will aggregate, analyze, and translate that data in order to communicate the landscape-scale outcomes being achieved by MRBI. While EOF monitoring is valuable at the farm scale for a variety of reasons (see Box 6), it does not provide evidence that taxpayer funding is helping to achieve cleaner water or meet policy goals in streams, riv-ers, and downstream water bodies.

During the preparation of this paper in 2011, NRCS did begin to collaborate with USGS and EPA to review MRBI projects for the quality of their water quality monitor-ing efforts. The agencies identified 15 12-digit project watersheds in which they would collaborate to oversee the quantification and demonstration of water quality improvements. Project proposals from the states in which these watersheds were located were reviewed. This pilot effort is a good start toward evaluating progress in

Figure 5 | Proportion of Reviewed Projects with Individual or Combined Tiers of Water Quality Monitoring

1%2%

58%

13%

11%

9%

4%

Although EOF monitoring by MRBI projects cannot help demonstrate if in-stream or watershed-outlet water quality improvements are ensuing from the project, there are several benefits to conducting such monitoring. The EOF data may provide evidence that the installed suites of practices do achieve cleaner water leaving individual producers’ fields. EOF water quality outcomes data could help improve the scientific literature on the nutrient and sedi-ment reduction efficiencies of individual or systems of practices. Finally, EOF data can be helpful to calibrate field-scale nutrient or sediment reduction estimation models and tools.

In March 2012, NRCS decided additional guidance was needed for the MRBI EOF monitoring activities due to concerns that some projects lacked adequate procedures to ensure that credible and usable data were being collected. NRCS placed a one-year moratorium on the use of conservation funds to pay farmers to install EOF monitoring stations. During this moratorium, NRCS reviewed relevant guidance on field-scale water monitoring and engaged many different water quality monitoring experts. In May 2013, NRCS introduced a new, more robust EOF water quality monitoring protocol to ensure that the MRBI stations and data being collected met technical specifications.*

This moratorium was an important adaptive management activity for NRCS to engage in to help ensure that credible data are being collected. On the other hand, the moratorium has delayed deployment of EOF monitoring stations and pushed back the amount of time it will take for these stations to establish adequate baseline data before conservation practices are installed. For the minority of reviewed projects that committed to conducting EOF monitoring as their only form of monitoring and evaluation, it remains to be seen how they will be able to track progress for some time to come.

*See USDA NRCS (2012a) and USDA NRCS (2012b).

Box 6 | NRCS Oversight of Edge-of-Field Monitoring

Source: WRI

Three-Tiered

In-Stream & Watershed

Edge-of-Field & Watershed

Watershed

In-Stream

Edge-of-Field

Edge-of-Field & In-Stream

(N = 45 PROJECT PROPOSALS)



achieving environmental outcomes. For the remaining majority of MRBI projects outside these special 15, there still remains a lack of oversight by a single institution to coordinate and communicate the results of the monitoring efforts.

MRBI PROJECTS PROMISE TO MONITOR MANY WATER QUALITY INDICATORS, BUT AGGREGATION OF RESULTS ACROSS THE INITIATIVE IS UNCERTAIN

In the RFPs, NRCS also signaled to the targeted water-shed projects that it would prioritize projects that “utilize environmental indicators to monitor water quality and evaluate effects of conservation practices…implemented through the project on a field or edge of field as well as selected downstream monitoring points.” Impressively, nearly 90 percent of proposals specified the water qual-ity indicators that would be monitored, despite the fact that NRCS did not require, but only encouraged, propos-als to include this information. The most commonly cited indicators were phosphorus (64 percent) and nitrogen (56 percent) at either the first, second, or third tiers of moni-toring (see Figure 6). Surprisingly, given MRBI’s focus on water quality at the landscape scale, very few projects included statements that they would monitor for common indicators of in-stream water quality such as dissolved oxygen and chlorophyll a (which are indicative of exces-sive algae).

The variety of water quality indicators that the projects are promising to monitor raises a concern about consis-tency in the MRBI project M&E protocols. Without NRCS oversight of the in-stream and watershed-outlet tiers of monitoring, there may be significant heterogeneity in both monitoring design and the chemical variation of pollutants being monitored. WRI observed numerous chemical vari-ations within each of the major categories of water quality indicators listed in Figure 6.65 This variation may make it difficult to aggregate the detected changes in water quality across projects in order to summarize initiative results and communicate the initiative-wide improvements.

In addition, given the natural lag in time between land-use changes and water quality responses, few projects (31 percent) mentioned taking the opportunity to measure in-field metrics of success such as changes in commercial or manure application rates. In addition, only a handful of reviewed projects mentioned measuring in-stream water quality through macro-invertebrate counts in streams, which is a metric that is relatively easy to train landown-

ers how to use and therefore may be a good alternative or complementary monitoring option to the use of more technical, chemical indicators.

FEW PROJECTS TAKE ADVANTAGE OF MODELING APPROACHES TO M&E

Although NRCS specified that an M&E plan was required to report on progress, the RFPs did not specify whether water quality models should also be employed to provide additional predictive estimates of success, as well as to inform planning efforts. Nevertheless, just over a quarter of the proposals stated that the project would use models to evaluate outcomes, either in conjunction with or instead of conducting water quality monitoring. Thirteen models were identified, ranging in scope, scale, and sophistica-tion, including local soil erosion spreadsheet calculators, state phosphorus indices for in-field P measurement, and internationally recognized watershed-scale hydrologic models.66

It is important for each watershed to identify and take advantage of models that have been either developed for or calibrated to the project watershed’s land use and hydrologic conditions. However, the diversity of models also raises concerns about the challenge of comparing

Figure 6 | Numbers of Reviewed MRBI Projects Stating Which Water Quality Indicators They Will Be Monitoring

Toxics2

Ph2

P29

N 25

Nitrite/Nitrate

18

Solids/TSS 13

Sediment 13

Nutrients

7

Ammonia5

Physical Parameters

5E. Coli

3

DO3

Algae2

Biological2

Organics 2

Turbidity1

Soil Loss1

Source: WRI

20 |

results across projects if outcomes are being estimated by different models at different scales and if there is no NRCS oversight of these modeling analyses. This diversity of models underscores the need for accelerated deployment of universally applicable models and tools calibrated for use in priority watersheds to be able to predict the nutri-ent and sediment reduction benefits achieved at both the field and watershed scales.

4b. Projects planned to collect data to establish either a minimum of two years of baseline information (i.e., pre-treatment water quality conditions) from which to assess progress, or use a paired watershed approach Fortunately, given NRCS’s development of the new EOF water quality monitoring protocol, there is now a require-ment that at least two years of baseline data be collected in order to establish the “control” status before the “treat-ment” or installation of practices occurs.67 However, NRCS provides no guidance for establishing baseline data for in-stream or watershed-outlet monitoring, so it is unclear

how many projects are using already established baseline data or are following credible protocols to establish new baseline data.

Prior to the release of the new EOF monitoring protocols, only half of the awarded MRBI proposals reviewed by WRI mentioned establishing a two-year baseline condition for their monitoring plans. Furthermore, none of the propos-als mentioned using a “paired stream or paired watershed approach” to establishing a baseline condition for the in-stream or watershed-outlet monitoring data, although it is believed to be one of the most effective means for detect-ing water quality changes attributable to land treatment.68

4c. Projects planned to collect data to determine if the conservation activities are installed appropriately (i.e., in accordance with standards and specifications) and maintained over the life of the project or contractStandard operating procedure for NRCS conservation contracts is that on-site verifications of 10 percent of all contracts will be conducted to ensure that practices are

EPA* outlines six main steps to effective watershed management and nine minimum elements (a through i) for projects that are funded through the 319 NPS Program for threatened or impaired waters.

Step 1. Build Partnerships Identify key stakeholders; identify issues of concern; set preliminary goals; develop preliminary indicators; conduct public outreach

Step 2. Characterize the Watershed Gather existing data and create a watershed inventory; analyze data; identify causes and sources of pollution (Element a); identify data gaps and collect additional data if necessary; quantify pollutant loads

Step 3. Set Goals and Identify Solutions Set overall goals and management objectives; develop indicators/targets; determine load reductions needed (Element b); identify critical areas; identify management practices to achieve goals (Element c)

Step 4. Design an Implementation Program Develop an implementation schedule (Element f); develop interim milestones to track implementation of management measures (Element g); develop criteria to measure progress toward meeting watershed goals (Element h); develop monitoring program (Element i); develop information/education component (Element e); develop evaluation process; identify technical and financial assistance needed to implement plan (Element d) (e.g., character-ization & analysis tools like GIS, monitoring, models, databases, load calculations); assign responsibility for reviewing and revising the plan

Step 5: Implement the Watershed Plan Prepare work plans; implement management strategies; conduct monitoring; analyze data; conduct information/education activities; share results

Step 6. Measure Progress and Make Adjustments Review, evaluate information; prepare annual work plans; report back to stakeholders and others; make adjustments

*See EPA (2008).

Box 7 | Watershed-Based Planning



properly installed (structural), implemented correctly (management), and well-maintained over the life of the contract. MRBI’s RFPs did not signal to its staff or to the project partners that any additional verification of prac-tices was necessary.

However, since the MRBI projects are attempting to achieve landscape-scale outcomes, it is even more impor-tant for project leaders to ensure that the many new prac-tices are properly installed in the right locations. Further-more, repeating annual practices each year or maintaining structural multiyear practices for water quality response purposes requires significantly longer time frames than those called for in most annual or short-term conserva-tion contracts.69 Therefore, it may be unrealistic to expect projects with short time frames to demonstrate water quality outcomes by the end of the project time frame.70 This potential mismatch between short-term conserva-tion contracts and long-term MRBI outcome expectations should be reevaluated, given the many years in natural lag time that occurs between land-use changes and water quality responses.71

4d. Projects planned to conduct watershed-based planning to increase the likelihood that landscape-scale water quality outcomes will be achievedThe 2011 RFP did not mention watershed management plans, while the 2010 RFP stated that priority would be given to projects that “provide evidence of a watershed-planning process” and listed seven elements

that constituted that evidence. Since it was only encour-aged but not required in the first year, only 40 percent of the reviewed projects mentioned that their MRBI projects were based on an existing watershed management plan.

Many of the core elements required by EPA in its 319 Non-point Source Pollution Control Program watershed plans (see Box 7) are also required by the MRBI RFPs. However, the RFP is missing several critical elements.

We also assessed MRBI against a recent study72 that evalu-ated 13 recent USDA National Institute of Food and Agri-culture Conservation Effects Assessment Project (NIFA-CEAP) targeted watershed projects (see Box 8). We found that the MRBI projects may be falling short of achieving what the NIFA-CEAP study describes as “intentional conservation” and “intentional water quality monitoring.” That is, without project planners knowing (a) where in the watershed poor land-management activities or effec-tive conservation implementation is already occurring or (b) whether existing or new conservation practices are well-maintained, it will be difficult for them to “intention-ally” install new practices in the most effective locations or “intentionally” establish the right monitoring systems in the right locations to detect the desired water quality improvements and attribute the changes appropriately.

The findings from the 13 USDA NIFA-CEAP projects* indicate that conducting watershed-based planning helps targeted watershed projects achieve measurable in-stream water quality improvements. The MRBI RFPs already require two important watershed planning elements: (1) identify pollutants of concern and their sources in the watershed and (2) select and prioritize new conservation practices for installation.

However, the NIFA-CEAP review also found that watershed projects must also go further to (1) conduct spatially explicit surveys of existing land use, management, and conservation practices within the watershed and (2) keep track of practice implementation and land-management activities over time, neither of which was called for in the MRBI RFPs.

Furthermore, the NIFA-CEAP evaluation found that certain institutional challenges complicate the development of watershed plans:

“To link water quality response to land treatment changes, conservation practices must be tracked as intensively as water quality monitoring, and at the same temporal and spatial scales…..Knowledge of land use, management, and conservation practices is essential to understanding the effectiveness of conservation programs. Data on conservation practices or land management are often unavailable due to confidentiality agreements or are incomplete.”

For more on this challenge, see WRI’s forthcoming publication in this series, which discusses barriers to targeting.**

*See Osmond et al. (2012).**See Walker and Perez (2014).

Box 8 | Recent Lessons Learned about Watershed Planning

22 |

5. Cost Effectiveness In an era of tight fiscal budgets, it is increasingly impor-tant that public funds be spent to achieve the greatest possible water quality benefits at the least cost. Consider-ing cost effectiveness in awarding conservation funding results in selecting applicants that propose to accomplish more per dollar awarded.73 This concept is known as pay for performance, as it awards conservation funds based on the quality of the practice effectiveness (i.e., the environ-mental benefits generated) rather than on the practice itself.74 Box 9 describes the criterion used to assess cost effectiveness.

FindingsOverall, MRBI ranked “very poor” for how well it addresses cost effectiveness (see Table 6). There is no cost effectiveness requirement in the RFP, and as a result, proposals largely lacked discussions of the relationship between the magnitude of the desired water quality out-come and the cost to attain that outcome.

WRI acknowledges that there is a lack of readily avail-able and standardized scientific and economic tools to evaluate cost effectiveness, making precise quantification of cost effectiveness challenging. However, MRBI could be greatly improved if NRCS invests in the development of cost effectiveness measurement tools. This capability would allow NRCS to better manage its resources so that conservation funds generate the greatest water quality outcomes possible per public and/or private dollar spent. In the meantime, partners should be asked to demonstrate cost effectiveness and pay for performance to the greatest extent possible with available information.

5. MRBI planned to maximize environmental cost effectiveness (i.e., attaining an environmental goal at the lowest possible cost to society), measured as the environmental benefits per dollar spentWRI found very little evidence that MRBI was designed to ensure cost effectiveness, as defined in the criterion. Although geographically targeting priority areas should ensure that the greatest possible environmental benefits are achieved, it is no guarantee that the maximum environmental benefits are achieved at the lowest possible cost.

MRBI tries to more cost-effectively use federal dollars by giving greater weight to proposals that leverage nonfederal dollars, which is a good step. However, encouraging pri-vate spending on conservation results in the cost-effective use of only federal dollars; it does not ensure that the greatest conservation outcomes are achieved at the lowest possible overall cost.

The MRBI RFPs did not require projects to discuss how they would strive for environmental cost effectiveness or to measure project cost effectiveness. In addition, although cost effectiveness is one minor component of EQIP’s ranking system for evaluating proposals, actual estimates of cost effectiveness (e.g., $/lb. N reduced) in each contract are not generated by the application ranking system. Twenty percent of MRBI project proposals at least mentioned that costs were considered in some aspect of the project. However, none of the projects explicitly aimed to pay for performance, nor did they adequately describe plans to achieve, implement, or assess cost effectiveness in any way for the entire project.

5. MRBI planned to maximize environmental cost effectiveness (i.e., attaining an environmental goal at the lowest possible cost to society), measured as the environmental benefits per dollar spent.75

Box 9 | Criteria for Assessing MRBI’s Efforts to Be Cost Effective


5. Cost effectiveness goes largely unaddressed. •Overall rating: Very Poor

Table 6 | Cost Effectiveness Rating



6. Adaptive Management Adaptive management is the process of decision making amid uncertainty, adapting project management based on information gleaned from project activities. Essen-tially, adaptive management “emphasizes learning while doing.”76 As NRCS continues down the uncharted path of implementing its landscape-scale initiatives and measur-ing their environmental outcomes, it is extremely impor-tant that NRCS learn from these new experiences and adapt its initiatives based on lessons learned. NRCS needs to assess if the initiative’s actions are achieving the desired conservation goals and, if not, reevaluate strategies in order to demonstrate that taxpayer-funded conservation programs are a good investment for achieving landscape-scale outcomes.77 Likewise, project partners should include adaptive management plans in their proposals so that the effectiveness of their projects can be evaluated and the necessary changes can be made over the course of the project period. The criteria used for assessing MRBI’s adaptive management efforts can be found in Box 10.

FindingsMRBI received a “fair” rating for the adaptive manage-ment factor (see Table 7). NRCS is practicing many prin-ciples of adaptive management but does not have a formal process in place yet for collecting or analyzing outcome-based information in order to adapt its overall program strategy or refine project requirements. Less than half the reviewed projects indicate that comprehensive adaptive management is being implemented, and less than half include outreach strategies to share lessons learned with appropriate audiences.

Formalizing an adaptive strategy for the initiative as a whole and developing a strategy to bring lessons learned down to the project level would significantly improve the effectiveness of MRBI. WRI recommends that the strategy include collecting annual performance reports, analyz-ing the available output or outcome metrics such as water quality monitoring data, and then using this information to analyze program effectiveness, share results, and adapt where necessary to improve impact. Many of these activi-ties will require NRCS to ask more from its partners, both in terms of reporting data and progress and adaptively managing their projects.

6a. MRBI planned to manage and analyze data as they become available throughout implementation; and, based on findings about project results, planned to adapt work plans as necessary MRBI does not have a formal adaptive management framework in place. However, NRCS has been continu-ously adjusting the initiatives based on changes in cir-cumstances. Priority areas have changed and likely will continue to change as will the primary resource concerns and priority practices. As discussed in the M&E section, due to concerns raised by project leaders and state con-servationists about the quality of the EOF water quality monitoring efforts, NRCS went through a comprehensive review process that successfully culminated in new rigor-ous M&E protocols.79

While NRCS has been reevaluating its strategy for MRBI, there has been no evidence that NRCS has adapted its strategy based on findings related to actual environmen-tal outcomes. Although it may be too soon to expect this, given the natural lag times, it is important to establish a framework to be followed. Without a formal process in

6a. MRBI planned to manage and analyze data as they become available throughout implementation; and, based on findings about project results, planned to adapt work plans as necessary.

6b. MRBI planned to share lessons learned and project products with key internal and external audiences.78

Box 10 | Criteria for Assessing MRBI’s Adaptive Management Efforts


6a. MRBI is adapting as necessary although formal adaptive management protocols could be established. •••

6b. MRBI is communicating frequently with stakeholders but lacks formal outreach strategy. •••


Table 7 | Adaptive Management Rating

24 |

place that creates a feedback loop with the projects and other stakeholders, MRBI may not be ready to implement the crux of adaptive management: to analyze data on program effectiveness and adapt when necessary to maximize impact.

Both MRBI RFPs stated that they would prioritize propos-als that discussed adaptive management strategies but did not require that partners had formal plans in place. Proj-ect life spans may be too short for robust adaptive man-agement, but an annual reevaluation based on progress would be beneficial to ensure that the multiyear project is proceeding as planned. Just under half (42 percent) of the proposals recognized the benefits of adaptive management and took advantage of the potential for a higher ranking by proposing to analyze data and adapt their work plans, as necessary.

6b. MRBI planned to share lessons learned and project products with key internal and external audiencesNRCS does not have a formal outreach strategy for sharing results. However, the agency does communicate initiative activities frequently through NGO conservation commu-nity partner meetings and presentations with other agen-cies or stakeholder groups. Although annual reports are a great way to share the initiative’s activities and progress, they currently lack details necessary for the agency or the stakeholders to use as a meaningful basis for adaptive management changes.

As stated earlier, the RFPs encouraged proposals to include discussion of adaptive management. In addition, the 2010 RFP prioritized proposals that outlined water-shed planning processes, including “an information and education component.” Though no more specifics were provided, this component likely represents an outreach strategy. Nearly half (42 percent) of the reviewed 2010 and 2011 proposals discussed plans to share results and lessons learned with landowners and stakeholders.

CONCLUSION AND RECOMMENDATIONSWRI commends NRCS for focusing conservation funding through its partnership-based targeted watershed project approach. Overall, MRBI received an average rating of “fair” across the six assessment factors, indicating that while the initiative is off to a good start, there is room for improvement in the MRBI program and targeted conser-vation projects. MRBI was most effective at geographically targeting conservation activities, receiving a “good rat-ing.” It received “fair” ratings for including stakeholders and producers in MRBI, for setting SMART-Q goals, for monitoring and evaluating progress, and for reflecting principles of adaptive management. It received a “very poor” rating on the cost-effectiveness evaluation factor. (See Figure 7 and the appendix for a table of all of the factors and criteria used in the assessment and the ratings for each.)

WRI identified the following 12 recommendations for NRCS and its state and local partners and project leaders to consider as they strive to make MRBI more effective at targeting scarce conservation funds to achieve greater water quality outcomes. Many of these recommendations may also apply to NRCS’s other landscape initiatives and conservation programs, as well as to other government agency or private conservation programs that aim to make the most of limited conservation or technical assistance funds.

Stakeholder and Producer Buy-In Recommendations1. Clarify which stakeholders are involved in which

aspects of MRBI. NRCS should provide the public with descriptions of the stakeholders involved at the program level, how they were involved during the development of the initiative, and what roles they are currently playing. NRCS should also clarify what functions (e.g., leadership, oversight and accountability, measurement, evaluation, reporting, and coordination) the new permanent MRBI coordinator is providing and who else may be taking the lead on other functions. Within each of the 13 participating states, simi-lar descriptions of stakeholders and functions should be provided, and official lines of communications and coor-dination should be established between the headquarters representative and the state representative for each function.



Additionally, MRBI will benefit from increased collabora-tion with EPA and USGS to leverage their areas of exper-tise, and it should clarify which functions these agencies will provide. (See the recommendations sections for the M&E factors for suggestions to involve other appropriate stakeholders at the programmatic level.) In future RFPs, MRBI should require more specificity on which project-level stakeholders will be coordinating which efforts with the newly clarified functional leaders at the programmatic level.

2. Enable agricultural producers and rural landowners to participate in the development of MRBI and the projects.

To ensure greater buy-in from the producers and land-owners that the projects seek to engage, the initiative should allow for more time during the proposal devel-opment process to garner their support. The initiative could also award more points for project applications that provide letters of support from farmers or farm groups or

other evidence that buy-in is being achieved. In so doing, producers and landowners can participate in designing the targeted watershed project—helping to define the problem that the project will tackle, setting outcome-based water quality goals to accomplish, analyzing and prioritizing geographic targeting opportunities, and helping to set up M&E efforts.

Gaining more up-front producer buy-in helps ensure pro-ducer engagement over the life of the project and extend the reach of the program through farmer-to-farmer encouragement. If such collective participation is concen-trated in the right locations with the adoption of the right practices, it increases the likelihood of achieving mea-surable water quality outcomes. Moreover, encouraging project leaders, stakeholders, and producers to continue engaging throughout the life of the project by sharing lessons learned will likely make all projects more effective and could provide insights for improving the conven-tional conservation programs.

Source: WRI

Figure 7 | Overall Rating for MRBI

Exemplary

Good

Fair

Poor

Very Poor

Stakeholder & SMART-Q Goals Geographic Targeting Monitoring & Cost Effectiveness Adaptive Producer Buy-in Evaluation Management

WRI targeting factors that were also called for in the MRBI RFPs

WRI targeting factors that were not called for or were indirectly called for in the MRBI RFPs

26 |

3. Prioritize awards to future projects that leverage and formalize significant resources from non-USDA sources.

Given the reality that the federal conservation programs are likely to receive less funding in coming years, suc-cessful projects will be those that can leverage non-USDA sources. Furthermore, given that NRCS limits its over-sight of the measurement of water quality outcomes to the edge of the field and that it is relying on other stake-holders to measure outcomes of its targeted watershed projects in stream and at the watershed outlet, NRCS should formalize relationships with key monitoring and funding stakeholders.

For example, project partners could leverage significant resources, technical assistance, and monitoring exper-tise from (a) federal, state, and university water quality monitoring institutions, (b) charitable foundations that fund environmental and conservation groups, and (c) farm trade association groups that fund environmental programs. NRCS should allow for more time than the two-month turnaround period to submit proposals after the RFP is released during the proposal development stage for project leaders to secure these partners, leverage existing resources, and even submit proposals for new sources of non-USDA funds.

Although other agencies and organizations would be lead-ing components of MRBI, USDA must maintain oversight of all of MRBI’s coordinated efforts to ensure that the environmental outcomes associated with its conservation programs are achieved. In so doing, NRCS should commit resources to its own staff and provide technical assistance funds to its many partners to cover the significant amount of time and expertise that it takes to design and imple-ment targeted watershed projects.

Policy-Oriented SMART-Q Goals Recommendations4. Lead by example and write a clear and SMART-Q

goal statement that aims to achieve landscape-scale outcomes and require projects to do the same.

Given the urgency of solving landscape-scale water qual-ity problems, the initiative’s SMART-Q program goal could read like this:

The goal of MRBI is to achieve both local water quality objectives (e.g., specific nutrient and sediment reductions called for in TMDLs) and regional water quality goals (e.g., a 45 percent reduction in delivered-to-the-Gulf nutrient loads) by 2020 in priority project watersheds. Improving water quality will be accomplished by work-ing in partnership with producers and key stakeholders on a sub-watershed basis to implement targeted nutrient and sediment reduction conservation system strategies which also achieve farm economic viability. MRBI will also achieve these goals by measuring changes in appro-priate environmental and natural resource indicators in fields, at the edge of fields, in priority streams and at watershed outlets.

SMART-Q goals should also be written at the project level where the actual on-the-ground work is conducted and tied to the overarching landscape-scale goal of the MRBI program. Designing projects around these clear goal statements helps to ensure that the desired outcomes are achieved on schedule.

5. Prioritize future MRBI funds to those projects that aim to achieve already existing landscape-scale policy objectives.

Given that funds are limited, NRCS should consider that the best use of scarce taxpayer resources for conservation is to solve the existing policy priorities for water quality: removing streams from the impaired waters list, achiev-ing some or all of the nutrient or sediment or other agri-cultural pollutant reductions called for in TMDL plans, making progress toward shrinking the hypoxic zone in the Gulf of Mexico, etc. NRCS could prioritize future proj-ect applications that commit to achieving existing policy drivers that will restore ecosystem health and improve water quality in targeted landscapes.



Geographic Targeting Recommendations6. Provide “targeting narratives” for the MRBI-designated

focus areas and the MRBI project watersheds.These narratives should include information such as why each of the 43 designated focus areas were selected, what kinds of data were used, what decisions were made based on that data, and what specific water quality outcomes are hoped for at which scales. A similar narrative should be developed for each targeted watershed project within those areas, including, but not limited to—

how and why critical sub-areas within each project watershed were selected and prioritized,

what water quality goals and objectives were estab-lished for the project, and

which eligibility and ranking criteria were developed to ensure that the most effective practices and producers are selected.

By providing this information to the public, NRCS will be helping to establish the narrative for each watershed’s “targeting story.” Such narratives will help each of the participating stakeholders and producers, NRCS, and the public understand each project’s priority problems and solutions and foster accountability.

Monitoring and Evaluation Recommendations7. Improve leadership and accountability for landscape-

scale outcomes. There are a plethora of stakeholders involved in measure-ment of many different kinds of outcomes at the MRBI project level. NRCS should work with the institutions (e.g., NRCS, USGS, EPA, state water quality agencies, universities, and project partners) to clarify which will provide accountability, leadership, and oversight for monitoring and reporting on water quality progress at the edge of field, in stream, and at the watershed outlet for each project, for each state, and for the initiative as a whole.

NRCS should be the coordinator of all of these efforts, providing the necessary leadership, oversight, and accountability for the initiative to collect and synthesize these outcomes and report them to the public on both a project-by-project basis and through aggregation, col-lectively as an MRBI-wide success story. Serving in this

role will require a commitment from NRCS to maintain a permanent MRBI coordinator. This permanent MRBI coordinator or an additional MRBI outcomes coordina-tor could be empowered to provide a liaison function with the project leaders and with the potential exter-nal review panel, and possibly also, an evaluation and reporting function. For example, as the point person for MRBI M&E information, this person could aggregate data across projects, analyze the data collectively, and report on the overall initiative outcomes. In addition, the coor-dinator could help relay monitoring results to assist in updating practice performance effectiveness datasets or to calibrate models and tools. NRCS does not necessarily have to do everything in house. Instead, the agency could convene an external review panel to help carry out these functions under NRCS leadership.

In addition, each state should consider establishing and empowering a state MRBI outcomes coordinator for the MRBI projects. Responsibilities for these 13 individuals could include coordinating meetings with the projects’ water quality monitoring stakeholders to discuss prog-ress and the need for adaptive management; collecting, aggregating, and analyzing output and outcome metrics; producing initiative-level annual reports for each state; and sharing findings with the MRBI outcomes coordina-tor and the public.

8. Establish advisory teams for water quality monitoring, metrics, and modeling.

NRCS should establish advisory teams that bring in appropriate experts from USDA’s Agricultural Research Service, USGS, EPA, state water quality agencies, univer-sities, and NGOs to build the capacity of key stakeholders at NRCS headquarters, the states, and the project leaders. This helps bolster the limited experience of some NRCS partnership stakeholders with setting outcome-based goals and quantitative objectives, developing outcome-based metrics, and monitoring or modeling water quality at various scales. For example, a water quality monitor-ing advisory team could conduct a review and prioritiza-tion of the most important water quality indicators that MRBI projects should be monitoring. This team could also establish the protocols needed to ensure that cred-ible data are being collected, aggregated, analyzed, and reported for the in-stream and watershed-outlet monitor-ing stations. The monitoring team should also establish formal ties with interested parties to update the state of the science on EOF conservation practice efficiencies.

28 |

This team should also establish formal protocols for help-ing the modeling communities to calibrate and validate the many different field-level and watershed-level models with the new EOF monitoring data.

Given the long time it takes for land-use changes to result in measurable landscape-scale water quality outcomes, a metrics advisory team could develop additional interim metrics for reporting by the projects and the initiative. For example, new in-field or EOF interim metrics could include reporting on the changes in fertilizer or manure applications due to nutrient management planning and potential reduction in soil loss as measured through RUSLE-2. In-stream metrics could include biological water quality indicators such as macroinvertebrate sur-veys, Secchi disk readings for water clarity, odor ratings, or algae identification.

A modeling advisory team could ensure that models and tools being used by the MRBI projects are effectively calibrated and validated for use in current and future project watersheds. This team could also provide training to project leaders who may be interested in using avail-able and appropriate models and tools to estimate their project’s outcomes. Although NRCS is currently develop-ing a modeling and field-based planning tool (through its Conservation Streamlining Delivery Initiative) that will estimate nutrient and sediment reductions from conser-vation practices, it should also pursue development of a corresponding economic analysis tool. By incorporating not only costs of implementing and maintaining practices but also possible net benefits from reduced fertilizer bills or increased crop yield associated with some practices, conservation planners will be able to help producers factor in cost effectiveness into conservation decision making.

9. Prioritize projects with already existing baseline monitoring data or that propose to use a paired watershed approach.

Water quality monitoring is an expensive and challeng-ing effort, and unless it is conducted properly, it may not produce adequate results to be worth the effort involved. However, it is important that water quality benefits be demonstrated where possible. Therefore, consistent with the findings of the recent NIFA-CEAP analysis, WRI recommends that the initiative prioritize acceptance of future project proposals to those that are located in watersheds with already established baseline data or those that propose to use a paired watershed approach. In addition, the initiative should award additional conservation practice funds to these two types of projects because they are more likely to be able to mea-sure water quality improvements if sufficient land-use change is accomplished.

10. Consider requiring watershed-based planning to help ensure attainment of improved landscape-scale water quality outcomes.

To increase the likelihood that landscape-scale water quality improvements can be achieved and measured, MRBI should consider requiring watershed-based plan-ning for all future projects. NRCS could adopt EPA’s watershed planning protocols and conduct training pro-grams for all future project applicants. In addition, MRBI and its watershed planning partners should determine if there are specific watershed-based planning tasks that could be undertaken by the ongoing projects to improve the likelihood of attaining real-world results. Short of requiring watershed plans, the initiative should require new project proposals to justify how the project will ensure achievement of “intentional conservation practice implementation” and “intentional water quality monitor-ing” and allow for longer project time frames to account for the natural delay in water quality responses to land-use changes.



Cost Effectiveness Recommendations11. Require projects to provide at least a narrative

discussion about the cost effectiveness of their targeted watershed projects and accelerate improvement of methods to quantitatively estimate cost effectiveness of practices and projects in order to ultimately structure conservation payments to reflect performance.

While WRI recognizes the current technical challenges of estimating cost effectiveness, it is important that projects make a case, albeit a qualitative one, for the cost effec-tiveness of the proposed practices to ensure that funded projects are likely to use taxpayer funds efficiently. For example, projects should describe how the proposed core and supporting practices prioritized for the project are the most appropriate for addressing water quality issues in the project areas and how they might yield the largest environmental benefits for the dollars expended. Priority could be given to proposals that are able to quantitatively demonstrate cost effectiveness, and cost effectiveness could be better used in ranking criteria.

Qualitative descriptions of cost effectiveness would be one step forward; but it is crucial that, as soon as pos-sible, MRBI and its projects estimate and evaluate the cost effectiveness of their efforts to achieve environmen-tal outcomes. NRCS could enter into formal partnerships with USDA’s Economic Research Service, university agri-cultural economists, and/or other economic and policy specialists to develop the models, tools, and approaches to estimate the cost effectiveness of conservation prac-tices and targeted watershed projects and to create a framework to pay for performance.

Adaptive Management Recommendations12. Develop a formal framework on adaptive

management to more effectively implement and require projects to include plans for adaptive management in their proposals.

While MRBI leadership at headquarters is clearly practic-ing adaptive management to some extent, it is important that a formal process be developed to ensure that all prin-ciples of adaptive management are followed, and that the initiatives are formally reevaluated on a regular basis.MRBI should require, rather than encourage, projects to have adaptive management plans. If comprehensive watershed planning is required, adaptive management planning is already accounted for. But where watershed planning is not occurring, stand-alone adaptive manage-ment plans should include annual performance reports that are submitted to NRCS throughout the duration of the projects. The reports should summarize progress, challenges and successes, and any requests for changes in project implementation due to the lessons learned. In addition, both the MRBI program and its projects should be sharing the lessons learned as the program and the projects are adaptively managing with the public on a timely basis.

30 |

ENDNOTES1. Personal communication with staff at USDA NRCS. December 2, 2013.2. Note that MRBI also receives funding from the Wetlands Reserve

Enhancement Program and the Conservation Innovation Grant Program. WRI focused solely on the proposals associated with CCPI.

3. See the acknowledgments section for a list the professionals WRI interviewed.

4. USDA Commodity Credit Corporation (March 2010) and USDA (2009a).5. Although the federal government has been collaborating with farmers

since the Soil Conservation Service (NRCS’s predecessor agency) was founded in 1935, the 1985 Farm Bill is regarded as the beginning of the current voluntary conservation program financial and technical assistance approach.

6. White (2012).7. Hall et al. (2012).8. Hall et al. (2012). 9. Walter et al. (2007) and Hansen, R. and D. Hellerstein (2006).10. Soil and Water Conservation Society (SWCS) and Environmental

Defense Fund (EDF) (2007) and Minnesota Department of Agriculture (MDA) Web site on precision conservation.

11. U.S. Environmental Protection Agency (EPA) Section 319 Nonpoint Source Success Stories Web site (2013).

12. USDA Farm Service Agency CRP M&E Web site (2013).13. USDA Land Conservation Initiatives (LCI) Web site (2013) and

USDA Cooperative Conservation Partnership Initiative (CCPI) Web site (2012).

14. USDA LCI Web site (2013).15. Alexander et al. (2008).16. USDA CCPI Web site (2012).17. USDA Commodity Credit Corporation (March 2010).18. CCPI pulls funds from the Environmental Quality Incentives Program

(EQIP), the Wildlife Habitat Incentives Program (WHIP) and the Con-servation Stewardship Program (CSP) while MRBI pulls in additional funding from the Conservation Innovation Grants (CIG) program and the Wetlands Reserve Enhancement Program (WREP).

19. Food Security Act of 1985 [As Amended Through P.L. 110-246, Effective May 22, 2008], H.R. 6124, 110th Cong. (2008).

20. Personal communication with staff at American Farmland Trust. November 11, 2011.

21. USDA (2009a).22. USDA Mississippi River Basin Healthy Watersheds Initiative Web site

(2013). Note that NRCS did not provide WRI with any awarded proposals for South Dakota to review.

23. Membership in each state’s state technical committees varies but can include representatives from agricultural and environmental federal, state, and local agencies as well as NGOs.

24. The hydrologic unit code (HUC) is a hierarchical classification system for watersheds based on size. Eight-digit HUC watersheds average about 700 square miles.

25. 12-digit HUC watersheds average about 40 square miles.26. USDA Commodity Credit Corporation (March 2010). 27. USDA (2009a).28. Due to natural lag times in subsurface flow from a field to a stream to a

watershed outlet, it could take years to observe water quality improve-ments at a watershed scale.

29. Meals et al. (2010).30. See the acknowledgments section for the list of professionals

WRI interviewed. 31. Note that NRCS did not provide WRI with any awarded proposals

from South Dakota in 2011 although South Dakota was a participating MRBI state in 2011.

32. Reed (2008).33. SWCS and EDF (2007).34. Gale et al. (1993) and Ingels and Rodecap 2009. WRI recognizes

that some producers may not have time or interest in participating in many meetings or reviewing documents; but their engagement should still be sought, and they can still be represented by the various farm trade associations.

35. American Farmland Trust (2011).36. American Farmland Trust (2011); Scarlett (2011); Gale et al. (1993). 37. USDA Commodity Credit Corporation (March 2010).38. The “Other” category includes entities such as irrigation districts

and planning commissions that were categorized as separate entities from SWCDs.

39. Osmond et al. (2012).40. The MRBI RFP for 2011 did not mention encouraging or prioritizing

projects that leveraged non-USDA funds except in the section of the RFP dealing with projects that request access to Wetlands Reserve Enhancement Project (WREP) funds.

41. WRI replaced the original “realistic” for another commonly used “R” keyword: results-oriented.

42. Doran (1981).43. WRI uses the terms goals and objectives interchangeably in the goals

discussion. In practice, some may prefer to keep the two terms separat-ed, relegating the term goal to reflect a generic, overarching description of the ambition of the water quality effort (e.g. “the goal is to improve water quality in a particular stream”) and using the term objective to indicate a specific, often numeric limitation or level of pollutant (e.g., a concentration) or at least a narrative statement to reflect protection of the beneficial uses of a water body.

44. Florida International University (2012).45. EPA (2007). 46. Meyer (2003).47. USDA Commodity Credit Corporation (November 2010).48. USDA Commodity Credit Corporation (March 2010).49. EPA (2007).50. SWCS and EDF (2007).51. SWCS and EDF (2007) and MDA Web site on precision

conservation (2013).52. SWCS and EDF (2007); USEPA (1993); Walter et al. (2007).53. USDA (2009a).54. The 2010 RFP prioritizes projects that “Provide evidence of a watershed

planning process that identifies nutrients (N or P) causing water quality problems and identifies their sources enabling implementation to be prioritized.” The 2011 RFP requires proposals to include “The descrip-tion should include information related to land use types, vegetation, soils, hydrology, potential sources of water quality impairments, occurrences of at-risk species, proximity to other protected areas, and a summary of resource concerns.”



55. Carey, J. (2013).56. Because NRCS uses its own screening mechanism for participants,

these results include only those proposals that went above and beyond the NRCS standard.

57. Stem et al. (2005).58. Gale et al. (1993). 59. EPA (2013). 60. Osmond et al. (2012); Gale et al. (1993); Stem et al. (2005);

USDA NRCS (2010).61. USDA Commodity Credit Corporation (March 2010).62. During development of approved project contract agreements,

NRCS could require such reports.63. USDA (2009a).64. USDA Commodity Credit Corporation (March 2010).65. WRI grouped the wide variety of water quality indicators that the

proposals promised to monitor into major indicator categories. The organics category includes soluble organic matter and particulate or-ganics. Biological includes habitat and total biovolume. Algae includes algae and algal community structure. Nutrients include nutrients, detri-mental nutrients, N&P total, and N&P total and dissolved. Phosphorus includes P, orthophosphate, particulate and reactive P, and total P. Sediment includes sediment, sediment concentrations, sediment from runoff, and suspended sediment. Nitrogen includes N, infield only N, reactive N, total N, and TKN. Ammonia includes ammonia and ammo-nium. Nitrite/nitrate includes NO

2, NO

3, NO

3-N, and nitrite. Solids/TSS

include chloride, dissolved solids, suspended solids, total solids, and TSS. Physical parameters include physical parameters, discharge, and flow. Toxics include herbicides and sediment absorbed contaminants.

66. For example, the SWAT model, which is the Soil and Water Assessment Tool.

67. USDA NRCS (2012a).68. Spooner and Line (1993); Osmond et al. (2012).69. Osmond et al. (2012). 70. The 2010 RFP required projects to not exceed five years while the 2010

RFP required projects to not exceed four years. 71. Gale et al. (1993); Osmond et al. (2012).72. Osmond et al. (2012).73. SWCS and EDF (2007). 74. Greenhalgh, S., M. Selman, and J. Guiling. (2006).75. Cattaneo et al. (2005); Scarlett (2011).76. U.S. Department of the Interior (2008). 77. The Conservation Measures Partnership (2007).78. The Conservation Measures Partnership (2007). Note that the Con-

servation Measures Partnership included three additional steps, but WRI believed these three steps had been adequately covered in other targeting factors.

79. USDA NRCS (2012a); USDA NRCS (2012b).

32 |

REFERENCESAlexander, R.B., R.A. Smith, G.E. Schwarz, E.W. Boyer, J.V. Nolan, and J.W. Brakebill. 2008. “Differences in Phosphorus and Nitrogen Delivery to the Gulf of Mexico from the Mississippi River Basin.” Environmental Science & Technology 42: 822─830 http://pubs.acs.org/doi/pdf/10.1021/es0716103

American Farmland Trust. 2011. National Agricultural Landscapes Forum: Executive Summary http://www.nrcs.usda.gov/Internet/FSE_DOCUMENTS/stelprdb1045163.pdf.

Carey, J. 2013. “Precision Conservation.” Conservation Magazine, September 2013. Available at http://conservationmagazine.org/2013/09/precision-conservation/

Cattaneo, A., R. Claassen, R. Johansson, and M. Weinberg. 2005. Flexible Conservation Measures on Working Lands: What Challenges Lie Ahead? USDA ERS Report Number 5 http://www.ers.usda.gov/publications/err5/err5.pdf

Christensen, T. 2011. “Mississippi River Basin Healthy Watersheds Initiative Partner Update”. 2011. (Accessed August 9, 2011). http://www.nrcs.usda.gov/Internet/FSE_DOCUMENTS/stelprdb1046715.pdf

Claassen R., V. Breneman, S. Bucholtz, A. Cattaneo, R. Johansson, and M. Morehart. 2004. Environmental Compliance in Agricultural Policy: Past Performance and Future Potential, AER-832. Washington, DC: U.S. Department of Agriculture, Economic Research Service, p. 50 http://www.ers.usda.gov/publications/aer-agricultural-economic-report/aer832.aspx#.UcNGRT55Mt1

The Conservation Measures Partnership. 2007. “Open Standards for the Practice of Conservation.” Available at http://www.conservationmeasures.org/wp-content/uploads/2010/04/CMP_Open_Standards_Version_2.0.pdf

Doran, G.T. 1981. “There’s a S.M.A.R.T. Way to Write Management’s Goals and Objectives.” Management Review 70: 35─36.

Florida International University. 2012. “Expectations & Goals.” (Accessed January 24, 2013). http://hr.fiu.edu/index.php?name=expectations_goals

Gale, J.A., D.E. Line, D.L. Osmond, S.W. Coffey, J. Spooner, J.A. Arnold, T.J. Hoban, and R.C. Wimberley. 1993. Evaluation of the Experimental Rural Clean Water Program. National Water Quality Evaluation Project (NWQEP). NSCU Water Quality Group, Biological and Agricultural Engineering Department, NC State University, Raleigh, NC: U.S. Environmental Protection Agency: EPA-841-R-93-005. http://nepis.epa.gov/Exe/ZyNET.exe/20013B06.TXT?ZyActionD=ZyDocument&Client=EPA&Index=1991+Thru+1994&Docs=&Query=&Time=&EndTime=&SearchMethod=1&TocRestrict=n&Toc=&TocEntry=&QField=&QFieldYear=&QFieldMonth=&QFieldDay=&IntQFieldOp=0&ExtQFieldOp=0&XmlQuery=&File=D%3A%5Czyfiles%5CIndex%20Data%5C91thru94%5CTxt%5C00000017%5C20013B06.txt&User=ANONYMOUS&Password=anonymous&SortMethod=h%7C-&MaximumDocuments=1&FuzzyDegree=0&ImageQuality=r75g8/r75g8/x150y150g16/i425&Display=p%7Cf&DefSeekPage=x&SearchBack=ZyActionL&Back=ZyActionS&BackDesc=Results%20page&MaximumPages=1&ZyEntry=1&SeekPage=x&ZyPURL

Greenhalgh, S., M. Selman, and J. Guiling. 2006. Paying for Environmental Performance: Investing in Farmers and the Environment. Washington, DC: World Resources Institute. Accessible at http://pdf.wri.org/pn_envmkts_paying_for_performance.pdf.

Hall, L., T. Christensen, J. Bramblett, S. Hopkins, and J. Hubbert. 2012. USDA’s National Water Quality Initiative: A Watershed Academy Webcast, July 2012. Presented on the U.S. Environmental Protection Agency’s Watershed Academy Webcast. http://water.epa.gov/learn/training/wacademy/upload/2012_07_10-slides.pdf

Hansen, L., and D. Hellerstein. 2006. Better Targeting, Better Outcomes. Conservation Program Design. United States Department of Agriculture Economic Research Service. Available at http://www.ers.usda.gov/publications/EB2/EB2.pdf.

Ingels, C. and J. Rodecap. 2009. Developing and Implementing Cost-effective, Performance-based Incentives in NE Iowa Watersheds. Iowa State University, University Extension. Available at http://www.bae.ncsu.edu/programs/extension/wqg/nmp_conf/2009_new_orleans/tuesday/mini_workshop_2_ag_nutrient_mgt/1_ingles.pdf.

Meals, D.W., S.A. Dressing, and T.E. Davenport. 2010. “Lag Time in Water Quality Response to Best Management Practices: A Review.” Journal of Environmental Quality 39: 85─96. Available at http://lshs.tamu.edu/docs/lshs/end-notes/lag%20time%20in%20water%20quality%20response%20to%20best%20management%20practices_%20a%20-1256684817/lag%20time%20in%20water%20quality%20response%20to%20best%20management%20practices_%20a%20review.pdf/

Meyer, Paul J. 2003. “Attitude is Everything: What would you do if you knew you couldn’t fail? Creating S.M.A.R.T. Goals.” Attitude is Everything: If You Want to Succeed and Go Above and Beyond. Meyer Resource Group, Incorporated. http://www.vanderbilt.edu/greek_life/SMART%20Goals.pdf

Minnesota Department of Agriculture. 2013. Precision Conservation: Targeting Clean Water Implementation. http://www.legacy.leg.mn/projects/precision-conservation-targeting-clean-water-implementation

Osmond, D.L., D.W. Meals, D. L.K. Hoag, and M. Arabi, eds. 2012. “How to Build Better Agricultural Conservation Programs to Protect Water Quality: The National Institute of Food and Agriculture–Conservation Effects Assessment Project Experience.” Ankeny, IA: Soil and Water Conservation Society. http://www.swcs.org/en/publications/building_better_agricultural_conservation_programs/

Perez, M., and C. Cox. 2009. “Seizing a Watershed Moment: Making EQIP Work for Water Quality in 10 Mississippi River Border States.” Environmental Working Group. http://static.ewg.org/reports/2009/seizing_a_watershed_moment/09EWG-EQIP-Summary-Report.pdf.

Public Law 110-246, 110th Congress, 2nd sess. (May 22, 2008), Food Security Act of 1985, as Amended.

Reed, M.S. 2008. “Stakeholder Participation for Environmental Management: A Literature Review. Biological Conservation 141: 2417─2431. http://bit.ly/oz492J

Scarlett, L., 2011. “America’s Working Lands: Farm Bill Programs and Landscape-Scale Conservation.” September 2011. Lincoln Institute of Land Policy. https://www.lincolninst.edu/pubs/dl/1958_1279_Scarlett%20final.pdf



Soil and Water Conservation Society and Environmental Defense Fund. 2007. “Environmental Quality Incentives Program: Program Assessment, March 2007. http://www.swcs.org/documents/filelibrary/EQIP_assessment.pdf

Spooner, J. and D.E. Line. 1993. “Effective Monitoring Strategies for Demonstrating Water Quality Changes from Nonpoint Source Controls on a Watershed Scale.” Water Science & Technology 28: 143–148.

Stem, C., R. Margoluis, N. Salafsky, and M. Brown. 2005. “Monitoring and Evaluation in Conservation: A Review of Trends and Approaches.” Conservation Biology 19: 295─309. http://www.fosonline.org/wordpress/wp-content/uploads/2010/06/Stem-etal.2005MEinConservation-ReviewTrendsApproaches.pdf

U.S. Department of Agriculture (USDA). 2009a. “Agricultural Secretary Vilsack Announces Major Initiative to Improve Health of Mississippi River Basin.” September 24, 2009. USDA News Release. http://www.usda.gov/wps/portal/usda/usdamediafb?contentid=2009/09/0463.xml&printable=true&contentidonly=true

USDA. 2009b. “Secretary Vilsack Announces 41 Watersheds to take part in Mississippi River Basin Initiative.” November 23, 2009. USDA News Release. http://www.usda.gov/wps/portal/usda/!ut/p/c5/04_SB8K8xLLM9MSSzPy8xBz9CP0os_gAC9-wMJ8QY0MDpxBDA09nXw9DFxcXQcAA_1wkA5kFaGuQBXeASbmnu4uBgbe5hB5AxzA0UDfzyM_N1W_IDs7zdFRUREAZXAypA!!/dl3/d3/L2dJQSEvUUt3QS9ZQnZ3LzZfUDhNVlZMVDMxMEJUMTBJQ01IMURERDFTODU!/?printable=true&contentidonly=true&contentid=2009%2f11%2f0586.xml

USDA Commodity Credit Corporation. 2010. Federal Register, Vol. 75, No. 40, Page 9380, Cooperative Conservation Partnership Initiative and Wetland Reserve Enhancement Program, Notice of Request for Proposals. March 2, 2010. Washington, DC: USDA.

USDA Commodity Credit Corporation. 2010. FR 73027, Cooperative Conservation Partnership Initiative and Wetland Reserve Enhancement Program, Notice of Request for Proposals. November 29, 2010. Washington, DC: USDA. http://www.nrcs.usda.gov/Internet/FSE_DOCUMENTS/nrcsdev11_023944.pdf

USDA Cooperative Conservation Partnership Initiative Web site. 2012. (Accessed January 24, 2012) http://www.nrcs.usda.gov/wps/portal/nrcs/main/national/programs/financial/ccpi/.

USDA Farm Service Agency. 2013. CRP Monitoring, Assessment and Evaluation Project (MAE) Web site. (Accessed January 15, 2013). http://www.fsa.usda.gov/FSA/webapp?area=home&subject=ecpa&topic=nra

USDA Landscape Conservation Initiatives Web site. 2013. (Accessed January 15, 2013) http://www.nrcs.usda.gov/wps/portal/nrcs/main/national/programs/farmbill/initiatives/

USDA Mississippi River Basin Healthy Watersheds Initiative Web site. 2013. (Accessed January 15, 2013) http://www.nrcs.usda.gov/wps/portal/nrcs/detailfull/national/programs/farmbill/initiatives/?cid=stelprdb1048200

USDA Natural Resources Conservation Service (NRCS). 2010. Interim Conservation Practice Standard: Monitoring and Evaluation, Code 799. February 2010. http://www.srwdmn.org/pdf/Interim_799_Monitoring_and_Evaluation_Standard.pdf

USDA NRCS. 2012a. Edge-Of-Field Water Quality Monitoring Data Collection and Evaluation Conservation Activity, Code 201, October 1, 2012.

USDA NRCS. 2012b. Edge-Of-Field Water Quality Monitoring System Installation Conservation Activity, Code 202, October 1, 2012. http://www.nrcs.usda.gov/Internet/FSE_DOCUMENTS/stelprdb1097618.pdf

U.S. Department of the Interior. 2008. Department Manual: Environmental Quality Programs. Chapter 1: Adaptive Management Implementation Policy. http://www.doi.gov/initiatives/AdaptiveManagement/documents/DOImanual3786.pdf

U.S. Environmental Protection Agency (EPA). 2007. Hypoxia in the Northern Gulf of Mexico. Washington, DC: U.S. Environmental Protection Agency. http://yosemite.epa.gov/sab/sabproduct.nsf/c3d2f27094e03f90852573b800601d93/$file/epa-sab-08- 003complete.unsigned.pdf.

U.S. EPA. 2008. Handbook for Developing Watershed Plans to Restore and Protect Our Waters. EPA 841-B-08-002, March 2008. Web site with link to handbook, quick guide, factsheets, and watershed planner tool: http://water.epa.gov/polwaste/nps/handbook_index.cfm#contents

U.S. EPA. 2013. Nonpoint Source Program and Grants Guidelines for States and Territories, April 12, 2013. http://water.epa.gov/polwaste/nps/upload/319-guidelines-fy14.pdf

U.S. EPA. 2013. Section 319 Nonpoint Success Stories. Web site. http://water.epa.gov/polwaste/nps/success319/U.S. EPA Office of Water. 1993. Geographic Targeting: Selected States and Examples. http://nepis.epa.gov/Exe/ZyNET.exe/20004LAA.PDF?ZyActionP=PDF&Client=EPA&Index=1991%20Thru%201994&File=D:\ZYFILES\INDEX%20DATA\91THRU94\TXT\00000008\20004LAA.txt&Query=841B93001%20or%20epa%20or%20geographic%20or%20targeting%20or%20selected%20or%20state%20or%20examples&SearchMethod=1&FuzzyDegree=0&User=ANONYMOUS&Password=anonymous&QField=pubnumber^”841B93001”&UseQField=pubnumber&IntQFieldOp=1&ExtQFieldOp=1&Docs=

Walter, T., M. Dosskey, M. Khanna, J. Miller, M. Tomer, and J. Wiens. 2007. “The Science of Targeting within Landscapes and Watersheds to Improve Conservation Effectiveness. “In Managing Agricultural Landscapes for Environmental Quality: Strengthening the Science Base, ed. M. Schnepf and C. Cox, pp. 63─89.

White, D. 2011. “Mississippi River Basin Healthy Watersheds Initiative Partner Update.” A PowerPoint presentation. August 9, 2011.

White, D. 2012. “Strengthening Conservation through the 2012 Farm Bill.” Conservation Slide Show for the Senate Agriculture Committee, February 28, 2012. http://www.ag.senate.gov/download/conservati16on-slideshow-by-chief-white

34 |

APPENDIX: ASSESSMENT FACTORS, CRITERIA, AND RATINGS

TARGETING FACTOR EVALUATION CRITERIA CRITERIA

RATINGFACTOR RATING

Stakeholder and Producer Buy-In

Relevant stakeholders were identified, analyzed, and represented systematically for all stages of the project and empowered to participate in decision making. ••••

FairProducers were involved in identifying the environmental problem and planning the project. ••Funding was leveraged from nonfederal partners. •••

Smart-Q Goals

Goals are specific, measurable, achievable, results-oriented, time-bound, and quantitative. ••••Fair

Goals aim to address already existing water quality policies or recommendations. ••

Geographic Targeting

Scientific evidence for targeting was provided to explain why project areas were selected. ••••Good

High priority areas within project watersheds were identified based on specific sources of pollutants, and appropriate controls were suggested. •••Eligibility requirements, participation incentives, or enrollment screens were proposed in order to encourage participation by producers who can be the most environmentally effective. ••••

Monitoring and Evaluation

The Initiative and the projects planned to collect and evaluate data along the way to determine if activities are helping to meet the goal(s). Alternatively, they planned to conduct an ex-post performance assessment after project completion to determine the impact of the activities, unrelated to any goals.

•••

Fair

Projects planned to collect data to establish either a minimum of two years of baseline infor-mation (i.e., pretreatment water quality conditions) from which to assess progress, or use a paired watershed approach. ••••Projects planned to collect data to determine if the conservation activities were installed appropriately (i.e., in accordance with standards and specifications) and maintained over the life of the project or contract. •••Projects planned to conduct watershed-based planning to increase the likelihood that landscape-scale water quality outcomes will be achieved. •••

Cost Effectiveness

MRBI planned to maximize environmental cost effectiveness (i.e., attaining an environmental goal at the lowest possible cost to society), measured as the environmental benefits per dollar spent. • Very

Poor

Adaptive Management

MRBI planned to manage and analyze data as they become available throughout implementation; and, based on findings about project results, planned to adapt work plans as necessary. •••

Fair

MRBI planned to share lessons learned and project products with key internal and external audiences. •••

Overall Rating: Fair



ACKNOWLEDGMENTSThis project was made possible by grants from the Packard Foundation, the Walton Family Foundation, and the McKnight Foundation. Michelle and Sara would like to thank NRCS for agreeing to a Memorandum of Understanding that provided WRI with access to the MRBI project proposals and performance reports. Thank you also to the many NRCS staff who met with us during the course of preparing this report: Wayne Honeycutt, Tom Christensen, Martin Lowenfish, Chris Hartley, Shaun McKinney, Karma Anderson, Craig Goodwin, Deena Wheby, Mark Rose, and Meghan Wilson. The authors also had the op-portunity to interview 13 MRBI project leaders from five MRBI projects during the preparation of this report. A complete list of those project leaders is pro-vided in WRI’s upcoming “Barriers to Targeting” Issue Brief as their experiences implementing MRBI are primarily reflected in that paper rather than in this one.

The authors also owe thanks to several agricultural targeting and water quality improvement professional who provided great feedback and food for thought during the project: David DeGeus (The Nature Conservancy), Deanna Osmond (North Carolina State University), Keith Admire (NRCS National Water Management Center), Todd Sutphin (Iowa Soybean Association’s Environmental Services Program), Katie Flahive and Stuart Lehman (EPA), Ferd Hoefner and Greg Fogel (National Sustainable Agriculture Coalition), and Jimmy Daukus (American Farmland Trust).

The authors would like to thank WRI’s Mindy Selman for her guidance on the review and WRI interns Lara Bryant, Elizabeth Tully, and Rachel Booher for their research, analysis, and design assistance. Finally, the authors thank the following individuals for reviewing this report: Robert Winterbottom and Francis Gassert with WRI, Martin Lowenfish (NRCS), Deanna Osmond (North Carolina State University), Rich Duesterhaus and Sarah Wessel (National Association of Conservation Districts), and Dave White (United Policy, LLC, Ecosystem Services Exchange, and former chief of NRCS). All errors of fact or interpreta-tion belong to the authors.

ABOUT THE AUTHORSMichelle Perez is a senior associate at WRI whose work focuses on farm conservation targeting, farm- and watershed-scale nutrient reduction tools, and market-based approaches to improving water quality.

Contact: [email protected]

Sara Walker is an associate at WRI whose works focuses on NutrientNet, WRI’s online water quality trading tool, and cost-effective and market-based approaches to reducing nutrient pollution.

Contact: [email protected]

OTHER PAPERS IN THIS SERIESWalker, S. and M. Perez. 2014 (forthcoming). “Improving Water Quality: Overcoming Barriers to Better Targeting of U.S. Farm Conservation Funds.” Issue Brief. Washington, DC: World Resources Institute.

Perez, M., K. Reytar, S. Walker, and M. Selman. 2014 (forthcoming). “Improving Water Quality: A National Modeling Analysis on Increasing Cost Effectiveness through Better Targeting of U.S. Farm Conservation Funds.” Report. Washington, DC: World Resources Institute.

36 |

Copyright 2014 World Resources Institute. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivative Works 3.0 License. To view a copy of the license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/

10 G Street, NE | Washington, DC 20002 | www.WRI.org

ABOUT WRI WRI is a global research organization that works closely with leaders to turn big ideas into action to sustain a healthy environment—the foundation of economic opportunity and human well-being. Our ChallengeNatural resources are at the foundation of economic opportunity and human well-being. But today, we are depleting Earth’s resources at rates that are not sustainable, endangering economies and people’s lives. People depend on clean water, fertile land, healthy forests, and a stable climate. Livable cities and clean energy are essential for a sustainable planet. We must address these urgent, global challenges this decade. Our VisionWe envision an equitable and prosperous planet driven by the wise manage-ment of natural resources. We aspire to create a world where the actions of government, business, and communities combine to eliminate poverty and sustain the natural environment for all people. Our ApproachCOUNT ITWe start with data. We conduct independent research and draw on the latest technology to develop new insights and recommendations. Our rigorous analysis identifies risks, unveils opportunities, and informs smart strategies. We focus our efforts on influential and emerging economies where the future of sustainability will be determined. CHANGE ITWe use our research to influence government policies, business strategies, and civil society action. We test projects with communities, companies, and government agencies to build a strong evidence base. Then, we work with partners to deliver change on the ground that alleviates poverty and strength-ens society. We hold ourselves accountable to ensure our outcomes will be bold and enduring. SCALE ITWe don’t think small. Once tested, we work with partners to adopt and expand our efforts regionally and globally. We engage with decision-makers to carry out our ideas and elevate our impact. We measure success through government and business actions that improve people’s lives and sustain a healthy environment.

IMPROVING WATER QUALITY - World Resources Institute · IMPROVING WATER QUALITY: ... methods to...

Documents

Transcript of IMPROVING WATER QUALITY - World Resources Institute · IMPROVING WATER QUALITY: ... methods to...