1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The primary goal of Grande Ronde Model Watershed program (GRMW) is the restoration of habitat critical to the survival of native anadromous and resident fish populations. The GRMW coordinates watershed planning activities and funds habitat enhancement projects within the Grande Ronde and Imnaha subbasins. The focus of the program currently is in the Upper Grande Ronde and Catherine Creek.

The GRMW encourages and supports sound land and water management, the local economy, and multiple land uses consistent with sound ecosystem management. Collectively, the GRMW plays a central role in coordinating the actions of numerous regional programs conducted by Tribes, agencies, counties, and landowners. The effort to coordinate local habitat restoration activities and to engage public support more broadly are commendable and consistent with the landscape approach advocated by the ISAB and others. The technical aspects of the project are strong. The GRMW has a long history of accomplishment, trained and experienced staff, and a programmatic network that can maintain adaptive capacity.

The program is significant to regional programs and is consistent with numerous recovery plans directed at habitat protection and recovery including the Grande Ronde and Imnaha Subbasin Plans, the FCRPS Biological Opinion, the Oregon Plan, and the Oregon Watershed Enhancement Board and more recent planning documents including the Atlas Process. The Atlas Process should be very useful to the GRMW program in its project planning and prioritization.

GMRW deserves credit for being proactive in expanding the scope of habitat restoration projects based on past experience, and for seeking to prioritize projects based on feasibility and biological benefits, for example the Stepwise project selection process and the Atlas Process. However, details were lacking on how feasibility and biological benefits are judged.

The Objectives and background are well described. The Objectives presented in the proposal, for example restore habitat connectivity and enhance floodplain connectivity, represent the desired outcomes of the suite of habitat enhancement projects funded through the GRMW program. The proposal includes adequate background information on the nature of the habitat problems being addressed, and extensive summaries of past activities with links to detailed results at individual sites. The results, in terms of individual projects, are impressive.

The Objectives are clearly related to the overall goal of improving native fish populations. The proposal, however, does not provide a compelling overview of progress towards achieving the program's Objectives, especially whether progress is being made in improving freshwater survival and growth of native fishes. Determining whether the GRMW is accomplishing its goals of habitat enhancement and improved freshwater fish productivity requires effectiveness monitoring, as emphasized by the ISRP in its previous review of this project. Effectiveness monitoring traditionally has not been a central component of the activities. The project has been in place since 1992, but it appears that effectiveness monitoring was only recently implemented.

With regard to this point, the sponsors make an important observation on p. 3: “Both the U.S. District Court in Oregon and the Court of Appeals for the Ninth Circuit have held that the ESA standard of jeopardy requires NOAA Fisheries to consider not only whether the species will survive but how the prospective actions (including habitat improvement projects) will affect the species’ prospects for recovery.” The ISRP interprets this as meaning that the results of restoration actions need to be quantified via effectiveness monitoring or the use of quantitatively based models to predict outcomes. It does not appear that this is being done at a scale and scope which will meet this criterion.

The GRMW recognizes the importance of effectiveness monitoring but states that it is constrained by lack of funding. The GRMW has approached the issue in at least three ways. First, for each objective they propose metrics or measures to evaluate project success. The metrics, for example miles of fencing and acres of riparian planting, pertain mostly to implementation with the assumption that they are having the desired outcome of improving habitat conditions and fish abundance. This is a complex and uncertain assumption but, in lieu of M&E, it is understandable from a practical perspective, depending on whether there is a direct relationship between the metrics and the desired outcomes of habitat improvement such as restoration of habitat connectivity and enhancement of floodplain connectivity. This assumption may be generally true, but it does not provide a quantitative assessment of actual habitat improvements. For example, are riparian plantings and other efforts to enhance riparian areas stabilizing banks, providing shade, and reducing water temperature? Perhaps most importantly, are these actions benefitting fish? This can only be demonstrated though M&E.

Second, the sponsors state that they will rely on ODFW and CRITFC monitoring projects to provide “overall watershed habitat status." It would be helpful if the sponsors had provided more detail regarding the way that these projects will satisfy the need for effectiveness monitoring of GRMW projects.

Third, the GRMW has made an effort to incorporate more site specific monitoring in the individual projects funded through their program. This is a positive step, and the ISRP recommends that this effort continue and expand in the future. The effectiveness of the GRMW program ultimately depends on the cumulative success of the individual projects in improving habitat. It would be helpful if the sponsors had provided more detailed information about this effort, including the responsibility of the GRMW in planning and design of the monitoring process, as part of its coordination role.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

The GRMW has successfully implemented an impressive number of projects. The Stepwise procedure developed for project selection, prioritization, and funding is a formalized process directly involving cooperators and includes technical review of each proposed project. This process helps ensure that individual projects share a common goal, that they are working in defined priority areas within the subbasins, and that closer cooperation among projects is fostered.

The Stepwise process, although useful, has limitations. It does not establish landscape scale priorities; rather, it assists the sponsors in developing and implementing individual projects. This limitation is important to recognize; how are priorities set at the landscape scale and the project class in terms of having a positive effect on fish survival? Further, how is the “biological benefit score” established and what are the components used to develop this quantitative score?

Results would be more meaningful if the sponsors presented at least a quantitative summary of how the projects it funds, taken together, have improved riparian and stream processes and freshwater survival of fish. If monitoring continues, as it should, the sponsors should develop an effective way of synthesizing results of individual projects to provide a “big picture” view of the success of the GMRW project as a whole.

Program management appears to have adapted appropriately to experience gained over 20 years, but this adaptation seems to have been passive rather than active. Adaptive management, as originally intended, requires deliberate experimentation to acquire the knowledge to reduce key uncertainties, with the goal of improving future decisions, and long-term benefits. Monitoring and evaluation are critical to such an adaptive management approach. Linking local monitoring of site specific projects to CHaMP methods used at watershed scales seems like an appropriate strategy given limited funding.

While learning is occurring at the program scale and at the scale of individual projects, the effectiveness of the adaptive management process could be vastly improved with the use of quantitative hypotheses or goals and the judicious use of reference sites for single actions or a group of actions. This would allow timely evaluation of effectiveness, and possibly the discovery of underlying mechanisms, and thereby improve learning.

Evaluation of Results

The purpose of the GRMW is to select, review, prioritize and fund habitat protection and restoration projects intended to benefit ESA-listed salmon and other fish species in the Grande Ronde and Imnaha Subbasins. The GRMW is a well-established and successful program that appears to have established stable and deeply rooted relationships with cooperators in the Grande Ronde subbasin. Its accomplishments since its inception in 1992 are impressive. The GRMW appears to be a well-managed program and, with the development of the Stepwise process, has improved its procedure for selection, review, and prioritization of projects. A strength of this program is its close working relationship with state and local governments, Tribes, conservation groups, private landowners, and other local public interests to coordinate habitat restoration projects on state and public lands.

The Grande Ronde Model Watershed Foundation, established by the GRMW, contracts directly with BPA and other funding sources to fund and implement restoration projects. Working through the Foundation, the GRMW with its cooperators is able to consolidate and coordinate habitat restoration planning at a subbasin scale and, through a formalized, structured process for project selection, helps ensure that projects address limiting factors in priority watersheds identified in the subbasin and other plans (and in the upcoming Atlas). Because the GRMW provides funding for projects it can exercise considerable influence on project selection and implementation. The existence of a single entity, such as the GRMW, responsible for planning and project selection within a subbasin should be considered in other subbasins where coordination among habitat restoration projects appears to be more loosely defined.

Determining whether the GRMW is accomplishing its goals of improving habitat and freshwater productivity of fish requires a robust effectiveness monitoring program. At present, monitoring is not sufficient to clearly demonstrate positive impacts of habitat improvement actions on fish. The GRMW should develop an effectiveness monitoring program that is capable of demonstrating quantitatively progress toward achieving the objectives of the individual projects funded through the GRMW and of the GRMW as a whole.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The success of this project requires close coordination with agencies, tribes, and the public. The GRMW seems to have been very successful in developing and maintaining these relationships and enfranchising a wide range of stakeholders.

The sponsors recognize climate change, non-native plants, and toxic chemicals as emerging limiting factors. In reality, these are not emerging limiting factors but ones that are already present at significant levels. As such, they should be addressed directly by program actions. An additional “emerging limiting factor” may be increasing agricultural demands for water, and this could be examined through scenarios, at a minimum, or the use of quantitative models/trend analyses. Flow restoration will need to operate in cooperation with agricultural demands and climate change. The project needs to have a strong understanding as to how these factors may impact future water supplies and timing.

Administration and overhead are 34% of the budget. This seems high compared to other similar projects. Is there justification for this high rate? If so, a detailed justification should be provided, especially so in that rent/utilities are a line item in the budget; these items are usually covered under overhead except in exceptional circumstances.

4. Deliverables, Work Elements, Metrics, and Methods

The project Deliverables are clearly linked to methods and individual restoration Objectives and should help meet the stated Objectives. Most of the Deliverables are classes of enhancement actions that will be undertaken by projects funded through GRMW. The specific projects that will be recommended for funding are given for each Deliverable. The ISRP assumes that these projects have already passed the Stepwise review process. A Deliverable as well as an Objective addressing M&E should be included. This Deliverable should specify the procedures the GRMW program will use to allocate funding for M&E. Will the GRMW propose guidelines for M&E for individual projects and will these guidelines or requirement be integrated into the Stepwise process for project selection? A more formalized process for M&E that applies to all projects funded through the GRMW is needed.

It was refreshing to see that the Deliverables were quantitative in terms of actions to be completed. The sociological results and benefits were highlighted in the Executive Summary but only lightly touched upon in the text. This is a highly important aspect central to the overall success in meeting programmatic goals. It should be directly addressed in the text, especially in the Work Elements and Deliverables.

Specific comments on protocols and methods described in MonitoringMethods.org

No comments.

===========QUALIFICATIONS FOLLOW================

The Grande Ronde Model Watershed is a strong, well organized program, and has had considerable success in implementing a large number of habitat enhancement projects. However, it needs to improve effectiveness monitoring and the adaptive management process to incorporate climate change, toxic chemicals, and non-native plants into the active program, and set priorities at the landscape scale. Results should be judged in terms of improvements to freshwater survival and productivity of fish.

Analysis of monitoring data often lags behind data collection. The sponsors should consider enlisting the assistance of NOAA-Fisheries early in the process to assist with the design of monitoring actions and with data analysis.

An Objective and Deliverable pertaining to M&E should be included in future proposals. An M&E Objective signifies a commitment to monitoring, especially effectiveness monitoring.

In future proposals quantitative details should be provided on how past and current actions are influencing survival and growth of native fishes. This should include monitoring results and how the results have altered actions through the adaptive management process.

Develop plans and actions to fully integrate climate change, toxic chemicals, non-native species, and agricultural water demands into an effective program.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The primary goal of Grande Ronde Model Watershed program (GRMW) is the restoration of habitat critical to the survival of native anadromous and resident fish populations. The GRMW coordinates watershed planning activities and funds habitat enhancement projects within the Grande Ronde and Imnaha subbasins. The focus of the program currently is in the Upper Grande Ronde and Catherine Creek.

The GRMW encourages and supports sound land and water management, the local economy, and multiple land uses consistent with sound ecosystem management. Collectively, the GRMW plays a central role in coordinating the actions of numerous regional programs conducted by Tribes, agencies, counties, and landowners. The effort to coordinate local habitat restoration activities and to engage public support more broadly are commendable and consistent with the landscape approach advocated by the ISAB and others. The technical aspects of the project are strong. The GRMW has a long history of accomplishment, trained and experienced staff, and a programmatic network that can maintain adaptive capacity.

The program is significant to regional programs and is consistent with numerous recovery plans directed at habitat protection and recovery including the Grande Ronde and Imnaha Subbasin Plans, the FCRPS Biological Opinion, the Oregon Plan, and the Oregon Watershed Enhancement Board and more recent planning documents including the Atlas Process. The Atlas Process should be very useful to the GRMW program in its project planning and prioritization.

GMRW deserves credit for being proactive in expanding the scope of habitat restoration projects based on past experience, and for seeking to prioritize projects based on feasibility and biological benefits, for example the Stepwise project selection process and the Atlas Process. However, details were lacking on how feasibility and biological benefits are judged.

The Objectives and background are well described. The Objectives presented in the proposal, for example restore habitat connectivity and enhance floodplain connectivity, represent the desired outcomes of the suite of habitat enhancement projects funded through the GRMW program. The proposal includes adequate background information on the nature of the habitat problems being addressed, and extensive summaries of past activities with links to detailed results at individual sites. The results, in terms of individual projects, are impressive.

The Objectives are clearly related to the overall goal of improving native fish populations. The proposal, however, does not provide a compelling overview of progress towards achieving the program's Objectives, especially whether progress is being made in improving freshwater survival and growth of native fishes. Determining whether the GRMW is accomplishing its goals of habitat enhancement and improved freshwater fish productivity requires effectiveness monitoring, as emphasized by the ISRP in its previous review of this project. Effectiveness monitoring traditionally has not been a central component of the activities. The project has been in place since 1992, but it appears that effectiveness monitoring was only recently implemented.

With regard to this point, the sponsors make an important observation on p. 3: “Both the U.S. District Court in Oregon and the Court of Appeals for the Ninth Circuit have held that the ESA standard of jeopardy requires NOAA Fisheries to consider not only whether the species will survive but how the prospective actions (including habitat improvement projects) will affect the species’ prospects for recovery.” The ISRP interprets this as meaning that the results of restoration actions need to be quantified via effectiveness monitoring or the use of quantitatively based models to predict outcomes. It does not appear that this is being done at a scale and scope which will meet this criterion.

The GRMW recognizes the importance of effectiveness monitoring but states that it is constrained by lack of funding. The GRMW has approached the issue in at least three ways. First, for each objective they propose metrics or measures to evaluate project success. The metrics, for example miles of fencing and acres of riparian planting, pertain mostly to implementation with the assumption that they are having the desired outcome of improving habitat conditions and fish abundance. This is a complex and uncertain assumption but, in lieu of M&E, it is understandable from a practical perspective, depending on whether there is a direct relationship between the metrics and the desired outcomes of habitat improvement such as restoration of habitat connectivity and enhancement of floodplain connectivity. This assumption may be generally true, but it does not provide a quantitative assessment of actual habitat improvements. For example, are riparian plantings and other efforts to enhance riparian areas stabilizing banks, providing shade, and reducing water temperature? Perhaps most importantly, are these actions benefitting fish? This can only be demonstrated though M&E.

Second, the sponsors state that they will rely on ODFW and CRITFC monitoring projects to provide “overall watershed habitat status." It would be helpful if the sponsors had provided more detail regarding the way that these projects will satisfy the need for effectiveness monitoring of GRMW projects.

Third, the GRMW has made an effort to incorporate more site specific monitoring in the individual projects funded through their program. This is a positive step, and the ISRP recommends that this effort continue and expand in the future. The effectiveness of the GRMW program ultimately depends on the cumulative success of the individual projects in improving habitat. It would be helpful if the sponsors had provided more detailed information about this effort, including the responsibility of the GRMW in planning and design of the monitoring process, as part of its coordination role.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

The GRMW has successfully implemented an impressive number of projects. The Stepwise procedure developed for project selection, prioritization, and funding is a formalized process directly involving cooperators and includes technical review of each proposed project. This process helps ensure that individual projects share a common goal, that they are working in defined priority areas within the subbasins, and that closer cooperation among projects is fostered.

The Stepwise process, although useful, has limitations. It does not establish landscape scale priorities; rather, it assists the sponsors in developing and implementing individual projects. This limitation is important to recognize; how are priorities set at the landscape scale and the project class in terms of having a positive effect on fish survival? Further, how is the “biological benefit score” established and what are the components used to develop this quantitative score?

Results would be more meaningful if the sponsors presented at least a quantitative summary of how the projects it funds, taken together, have improved riparian and stream processes and freshwater survival of fish. If monitoring continues, as it should, the sponsors should develop an effective way of synthesizing results of individual projects to provide a “big picture” view of the success of the GMRW project as a whole.

Program management appears to have adapted appropriately to experience gained over 20 years, but this adaptation seems to have been passive rather than active. Adaptive management, as originally intended, requires deliberate experimentation to acquire the knowledge to reduce key uncertainties, with the goal of improving future decisions, and long-term benefits. Monitoring and evaluation are critical to such an adaptive management approach. Linking local monitoring of site specific projects to CHaMP methods used at watershed scales seems like an appropriate strategy given limited funding.

While learning is occurring at the program scale and at the scale of individual projects, the effectiveness of the adaptive management process could be vastly improved with the use of quantitative hypotheses or goals and the judicious use of reference sites for single actions or a group of actions. This would allow timely evaluation of effectiveness, and possibly the discovery of underlying mechanisms, and thereby improve learning.

Evaluation of Results

The purpose of the GRMW is to select, review, prioritize and fund habitat protection and restoration projects intended to benefit ESA-listed salmon and other fish species in the Grande Ronde and Imnaha Subbasins. The GRMW is a well-established and successful program that appears to have established stable and deeply rooted relationships with cooperators in the Grande Ronde subbasin. Its accomplishments since its inception in 1992 are impressive. The GRMW appears to be a well-managed program and, with the development of the Stepwise process, has improved its procedure for selection, review, and prioritization of projects. A strength of this program is its close working relationship with state and local governments, Tribes, conservation groups, private landowners, and other local public interests to coordinate habitat restoration projects on state and public lands.

The Grande Ronde Model Watershed Foundation, established by the GRMW, contracts directly with BPA and other funding sources to fund and implement restoration projects. Working through the Foundation, the GRMW with its cooperators is able to consolidate and coordinate habitat restoration planning at a subbasin scale and, through a formalized, structured process for project selection, helps ensure that projects address limiting factors in priority watersheds identified in the subbasin and other plans (and in the upcoming Atlas). Because the GRMW provides funding for projects it can exercise considerable influence on project selection and implementation. The existence of a single entity, such as the GRMW, responsible for planning and project selection within a subbasin should be considered in other subbasins where coordination among habitat restoration projects appears to be more loosely defined.

Determining whether the GRMW is accomplishing its goals of improving habitat and freshwater productivity of fish requires a robust effectiveness monitoring program. At present, monitoring is not sufficient to clearly demonstrate positive impacts of habitat improvement actions on fish. The GRMW should develop an effectiveness monitoring program that is capable of demonstrating quantitatively progress toward achieving the objectives of the individual projects funded through the GRMW and of the GRMW as a whole.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The success of this project requires close coordination with agencies, tribes, and the public. The GRMW seems to have been very successful in developing and maintaining these relationships and enfranchising a wide range of stakeholders.

The sponsors recognize climate change, non-native plants, and toxic chemicals as emerging limiting factors. In reality, these are not emerging limiting factors but ones that are already present at significant levels. As such, they should be addressed directly by program actions. An additional “emerging limiting factor” may be increasing agricultural demands for water, and this could be examined through scenarios, at a minimum, or the use of quantitative models/trend analyses. Flow restoration will need to operate in cooperation with agricultural demands and climate change. The project needs to have a strong understanding as to how these factors may impact future water supplies and timing.

Administration and overhead are 34% of the budget. This seems high compared to other similar projects. Is there justification for this high rate? If so, a detailed justification should be provided, especially so in that rent/utilities are a line item in the budget; these items are usually covered under overhead except in exceptional circumstances.

4. Deliverables, Work Elements, Metrics, and Methods

The project Deliverables are clearly linked to methods and individual restoration Objectives and should help meet the stated Objectives. Most of the Deliverables are classes of enhancement actions that will be undertaken by projects funded through GRMW. The specific projects that will be recommended for funding are given for each Deliverable. The ISRP assumes that these projects have already passed the Stepwise review process. A Deliverable as well as an Objective addressing M&E should be included. This Deliverable should specify the procedures the GRMW program will use to allocate funding for M&E. Will the GRMW propose guidelines for M&E for individual projects and will these guidelines or requirement be integrated into the Stepwise process for project selection? A more formalized process for M&E that applies to all projects funded through the GRMW is needed.

It was refreshing to see that the Deliverables were quantitative in terms of actions to be completed. The sociological results and benefits were highlighted in the Executive Summary but only lightly touched upon in the text. This is a highly important aspect central to the overall success in meeting programmatic goals. It should be directly addressed in the text, especially in the Work Elements and Deliverables.

Specific comments on protocols and methods described in MonitoringMethods.org

No comments.

===========QUALIFICATIONS FOLLOW================

The Grande Ronde Model Watershed is a strong, well organized program, and has had considerable success in implementing a large number of habitat enhancement projects. However, it needs to improve effectiveness monitoring and the adaptive management process to incorporate climate change, toxic chemicals, and non-native plants into the active program, and set priorities at the landscape scale. Results should be judged in terms of improvements to freshwater survival and productivity of fish.

Analysis of monitoring data often lags behind data collection. The sponsors should consider enlisting the assistance of NOAA-Fisheries early in the process to assist with the design of monitoring actions and with data analysis.

Modified by Dal Marsters on 6/11/2013 1:03:39 PM.

The sponsors have satisfactorily addressed the ISRP's concerns and we thank them for clarifying several important issues regarding the operation of the Grand Ronde Model Watershed Program (GRMWP). The ISRP reiterates that the GRMWP has been highly successful in implementing projects and has an outstanding record of cooperative work among government and private entities. A central ISRP concern about the GRMWP was that the proposal did not provide an adequate summary of project effectiveness and monitoring. The sponsors make the point that compiling the results of 150 projects would yield benefits but is precluded due to fiscal limitations related to the 5% budget limitation imposed by BPA. The ISRP appreciates the sponsor's willingness to undertake this assessment, which apparently would largely require compilation of existing records, and encourages the NPCC and BPA to provide funds for this effort. This expenditure would be appropriate because the GRMWP is the largest program of its type in the basin -- truly a "model" as the name implies -- and the assessment would allow a better evaluation of the success of the program.

Qualification: The sponsors should develop a report presenting quantitative and qualitative results to date pertaining to the effectiveness of the projects under their domain, a general summary and conclusions about overall project effectiveness, and the application of the results to management. The sponsors should report positive results as well as results from projects that to date may not yet have produced significant effects. This effort should be funded by BPA and reviewed by the ISRP in FY07. The response of the sponsors of project # 199608300 may provide some guidance for preparation of the report.