The ISRP appreciates the leadership and positive impact of the Grande Ronde Model Watershed (GRMW) on habitat restoration in the Grande Ronde and Imnaha rivers, the time they invest in responses and dialogue with us, and the importance of this group of subbasin projects for the Fish and Wildlife Program. The response addresses most issues raised in the preliminary review; however, one significant issue remains.

The ISRP requests the proponents to provide information on the following condition:

• Synthesis Report. The proponents should submit the completed Synthesis Report to the Council and BPA for ISRP review by May 1, 2022.

In our preliminary review, we requested responses on the following topics:

1. Synthesis report. The GRMW confirmed that it is committed to providing a revised Synthesis Report to include the ISRP’s request for a “comprehensive empirical evaluation of the effectiveness of restoration actions on fish populations and demonstrated progress at the landscape level” by May 1, 2022. The ISRP acknowledges that this is an ambitious task but believes it is very important. The agreement by BPA to allow the project to use BPA funds to complete this task is essential to its completion and success. We encourage BPA to ensure that adequate funds are provided for completing the synthesis, a product that will be valuable for other projects in the Columbia River Basin.

The GRMW outlined an approach for completing the revision of the Synthesis Report, which includes much of the original information on the history of the project, details on restoration projects completed by the GRMW, the M&E matrix summary requested by the ISRP in this response loop, the GRMW’s updated adaptive management process, and an integrated implementation and evaluation approach. The implementation and evaluation approach will include information on physical and biological responses to their restoration actions and results from a life cycle model (depending on timing of availability). They will use LIDAR for much of this analysis, which will limit their evaluation to the period from 2009 to the present. The ISRP understands this limitation, but we encourage the proponents to analyze their available data to the extent possible and summarize the results.

The brief description of the proponents’ plans for completing the Synthesis Report appears to address the original ISRP review of the Synthesis Report. We refer the proponents to the 2018 ISRP review of the Synthesis Report (ISRP 2018-11) for elements needed in the report. The GRMW has made substantial progress on many of the recommendations from this review, especially related to development of an adaptive management process and development of life cycles models, which the ISRP sees as major accomplishments of the GRMW and its collaborators.

The GRMW Response asked for an explanation of how the ISRP and Council will use this report. First, the ISRP emphasizes that the primary purpose of our past recommendations to develop the Synthesis Report is to guide the GRMW in their efforts to understand the degree to which it has accomplished its ecological objectives, identify major programmatic needs, and identify high priority actions for the future. The Atlas, life cycle models, and analyses of specific fish populations and habitats all provide valuable information, but they do not provide the integrated landscape strategy and assessment of benefits to fish and wildlife that this 29-yr project potentially produced. Readers will want to know if the current restoration strategy is working, and what targeted steps will be taken to address areas that require improvement. Second, the GRMW has made significant progress in developing an effective adaptive management process, and the description of their process for evaluation and adjustment could serve as a model for other projects. Third, the Synthesis Report will provide critical information on effective methods and landscape-level strategy for regional conservation efforts in the upper Columbia River and Snake River region. The ISRP reviews many projects with far fewer technical resources than the GRMW, and such projects would greatly benefit from seeing how the GRMW strategy has been developed, implemented, and evaluated. Furthermore, the ISRP anticipates that the information on quantitative responses to past restoration actions and lessons learned over the duration of the project will be valuable for the NPCC Fish and Wildlife Program and individual projects and cooperators in the basin.

The ISRP greatly values the dialogue we have had with the GRMW in recent years. We have gained a much better understanding of the project and the challenges and limitations the proponents have faced. More importantly, we have learned much about the successes and leadership of the GRMW in the region. We continue to be available to discuss the proponents’ plans for completing the Synthesis Report and to clarify any scientific issues raised in our reviews.

2. SMART objectives. The proponents provided the additional information needed to specify the SMART aspects of their objectives. The GRMW provided objectives for physical and biological processes, implementation, and social processes by river basin and by sub-watershed. We appreciate the inclusion of explicit programmatic management goals, which provide information for tracking management effectiveness, and all objectives include metrics for measurement.

We requested additional information about how the proponents ensure that collaborators and other sponsors develop SMART objectives for their joint efforts. The response from the GRMW indicates that it requests all sponsors working with the GRMW to provide SMART objectives in step 3 of their Stepwise Process. The proponents’ response provides an example of how the online proposal format developed by the GRMW is designed to guide collaborating sponsors in developing SMART objectives. In the example provided in the SMART objectives spreadsheet, the sequence of worksheets for the specific Atlas project, limiting factors, restoration actions, and indicators provides a framework for SMART objectives that is more informative than most approaches we have seen. We encourage the proponents to ask sponsors to explain how the indicators will be measured, who will conduct the measurements, and who will compare the data to the specific desired outcomes. Such information often is lacking or overly brief.

3. M&E matrix – lead. The GRMW provided an Excel spreadsheet that identifies the biological and physical M&E efforts related to 90 implementation projects in the Grande Ronde/Imnaha subbasins. The spreadsheet includes information on seven types of biological responses (parr abundance, benthic macroinvertebrates, mussels, redd surveys, prespawn mortality, smolt abundance, smolt survival) and seven types of physical responses (habitat survey, water temperature, toxics, flow, riparian condition, groundwater, floodplain condition). The spreadsheet indicates the evaluation design for both project monitoring and basin-scale monitoring. The spreadsheet also identifies the project that was responsible for the monitoring, the watershed and biologically significant reach, and the project initiation or completion years. Seven BPA-funded projects provided information for the matrix. The GRMW did not request information for AEM sites in these two subbasins. The Nez Perce Tribes did not participate in developing the summary or providing information on their M&E efforts. In summary, the proponents provided initial information on all aspects of M&E that the ISRP requested in the response loop, though it did not include a narrative summary of monitoring efforts or maps of the locations. The proponents intend to provide a more complete version of the M&E summary in their Synthesis Report and include information on monitoring efforts of AEM, USFS, SWCD, BOR, and others in the subbasins. They plan to provide maps, and overall summary of the M&E efforts, and description of the linkages between projects in the final version in the Synthesis Report. With this additional information, the summary will serve as an excellent example of the cooperation between projects and collaborative identification of monitoring and evaluation in a geographic area that the ISRP envisioned in our request for M&E matrices.

The Council and Council staff have stated their support for developing summaries and matrices of the types and locations of monitoring efforts across projects in major geographic areas. The ISRP has provided additional information on the summary of monitoring and evaluation for geographic areas in the Programmatic Comments of this report. We anticipate that the Fish and Wildlife Program will identify the specific elements and formats for these RM&E summaries and matrices in the near future. The proponents of this project should coordinate with Council staff to align their effort with future M&E summaries for geographic areas.

Preliminary ISRP report comments: response requested (Provided for context. The proponents responded to the ISRP’s questions; see response link and final review above.)

Response request comment:

The Grande Ronde Model Watershed (GRMW) has developed a comprehensive watershed management approach and collaborates successfully with partners. The proponents have completed three Atlases, developed and implemented a structured decision-making tool for project identification, prioritization, and design, and obtained a Focused Investment Partnership from the Oregon Watershed Investment Board to provide $7 million in additional funding for restoration. They also have completed several major restoration projects, investigated potential toxic substances in the lower river, formed a place-based integrated water resources management plan with Union County, developed technical capability for remote aerial habitat surveys, produced the first draft of the 25-Year Synthesis Report, and partnered with more than 15 agencies, programs, and organizations. In addition to its achievements in recent years, the proponents have responded positively and constructively to recommendations from the ISRP and Council.

The ISRP requests the proponents to address the following issues in a point-by-point response to assist our review of the proposal:

1. Synthesis report. The ISRP requests a description of the plans to revise the 25-Year Synthesis Report, including planned analyses, summary information, links to future project prioritization and planning, and the anticipated timeline for completion of the different elements and final report. The proponents recognize that more analysis and synthesis are needed. They indicate that they are committed to creating a comprehensive synthesis of the program’s benefits for fish and wildlife, as well as how the project has addressed limiting factors for key life stages. The ISRP strongly encourages the Council and BPA to allow the GRMW to use BPA funds to produce this fundamental programmatic element of a large-scale, long-term restoration program.
2. SMART objectives. The project coordinates restoration actions in the Grande Ronde basin, and the proposal identifies the limiting factors and proposed actions for each project. No specific objectives are identified. Please explain how this project ensures that their partners develop SMART objectives for each project and whether SMART objectives are required in the Stepwise process.
3. M&E matrix - lead. One of the challenges for ISRP reviewers is understanding the specific monitoring that is being conducted for multiple implementation projects. Habitat restoration projects or hatchery projects implement actions that are intended to address limiting factors and benefit fish and wildlife. Most of these projects do not directly monitor habitat conditions or biological outcomes, but most identify other projects in the basin that monitor aspects of physical habitat or focal fish species. The monitoring project(s) in the basin provides essential monitoring data for habitat, juvenile salmonid abundance and distribution, outmigration, survival, and adult returns for salmon and steelhead. Some monitoring projects focus on status and trends in basins, while others focus on habitat relationships and responses to local actions. It is unclear what monitoring the monitoring project(s) conducts for each implementation project.

Given the regional leadership responsibilities of this programmatic project, the ISRP requests the Grande Ronde Model Watershed Project (199202601) to summarize the linkages between implementation and monitoring projects in the Grande Ronde and Imnaha geographic area. The summary should provide a table or matrix to identify what is being monitored for each implementation project and where and when the monitoring occurs. The summary also should explain how the projects are working together to evaluate progress toward addressing limiting factors and identify future actions. A map or maps could help identify the locations of monitoring actions. The monitoring information should clearly explain whether the biological monitoring is local information for the specific implementation site or basin scale monitoring of status and trends or fish in/fish out. We are asking implementation and other monitoring projects to assist your project in producing this summary.

The ISRP recognizes that this task may require more than two months to complete, but we will appreciate any progress that can be made, as well as updates on plans for their completion.

Q1: Clearly defined objectives and outcomes

The Grande Ronde Model Watershed (GRMW) is one of the longest running habitat restoration projects funded by the Council’s Fish and Wildlife Program. It has developed a comprehensive watershed management approach, collaborating successfully with partners including Oregon Department of Fish & Wildlife (ODFW), Columbia River Intertribal Council (CRITFC), U.S. Forest Service (USFS), and Bonneville Power Administration (BPA). Substantial improvements in its program since the 2017 Umbrella Review are evidenced in project outcomes, including:

• Completion of three Atlases— spatially explicit landscape databases for Catherine Creek, Upper Grande Ronde, and Wallowa/Imnaha — to track resource conditions and identify limiting factors, critical life stages, data gaps, and priorities for habitat protection and restoration

• Implementation of Stepwise, a structured decision-making tool, for project identification, prioritization, and design

• Acquisition of a Focused Investment Partnership from the Oregon Watershed Investment Board to provide $7 million in additional funding for restoration

• Completion of several major restoration projects, including fish passage for Beaver Creek and the Lostine River, and a large-scale restoration of the Catherine Creek in cooperation with the CTUIR

• Formation of a place-based integrated water resources management plan with Union County

• Development of a remote aerial survey system

• Preparation of the first draft of the 25-Year Synthesis Report.

The proposal presents a thorough description of the program’s background history and location, which reflects the information assembled in the initial version of the 2018 Synthesis Report. The proposal includes an informative history of key developments and implementation, as well as coordination of more than 280 projects by the GRMW since 1992. Based on Council recommendations in the 2013 Categorical Review, the GRMW focuses on coordination of restoration efforts rather than implementing projects. They have developed the expertise and infrastructure to conduct mapping and resource assessment, remote aerial surveys, and habitat suitability modeling.

The proposal includes quantitative information on the GRMW’s progress for major subbasin plan objectives: fish passage improvement, protection of high-quality habitat, watershed processes, channel conditions, riparian function, sediment reduction, and flow improvement. In addition, the proposal documents their achievements in bull trout recovery, project assistance, and public education and outreach. Objectives related to outreach and engagement (“Social goals and objectives”) are an important strength of this proposal that can serve as a model for other umbrella and Council projects. Achievements in all aspects have been significant and demonstrate the program’s effectiveness. The effectiveness of their collaboration and assistance is supported by their contribution to peer-reviewed publications with CRITFC and ODFW (Favrot and Jonasson 2020, Favrot, Jonasson, and Peterson 2018, Justice et al. 2017, Crump et al. 2019, White et al. 2021). Several of these peer review publications provide publicly available information sources for critical analyses and resource assessments that are related to the project.

The proposal provides SMART objectives for physical and biological processes, implementation, and social processes by river basin and by sub-watershed. The proposal also includes explicit management goals, which provide information for tracking management effectiveness. All objectives include metrics for measurement. While objectives are presented by the basin scale, SMART objectives specific to individual projects are not provided. The ISRP encourages the GRMW to ensure that associated implementation projects develop SMART objectives and document them in proposals and annual reports.

The proposal identifies several major confounding factors, including toxic substances, locations of major fish mortality in the lower watershed, human population growth, and climate change. The ISRP commends the project’s identification of approaches to address these potential factors, as well as application of their data and analyses in aiding future project selection, prioritization, and design processes.

Q2: Methods

The GRMW no longer implements restoration projects and now coordinates projects with partners, providing analytical resources (Atlases), project prioritization, selection and design (Stepwise), and data management (Atlases). The proposal provides brief descriptions of the databases and decision-making process, which are described in greater detail in Appendix B and on the GRMW website (https://www.grmw.org/). The data layers in the Atlases provide extensive information on historical distribution, habitat conditions, biological data, water quality, and social attributes.

Projects anticipated for 2021-2027 (44 titles) are listed in Appendix A, including information on major fish population group, priority population, priority watershed, limiting life stages, limiting habitat conditions, prioritized habitat action types, project titles, proponent organizations, exact location, and proposed year of implementation. While the limiting factors and proposed actions are described for each project, specific SMART objectives are not provided for each. Do the partners develop SMART objectives for each project? Is that a requirement in this Umbrella Project’s Stepwise process?

The proposal includes an overall project timeline from 2022 to 2027 and a Gantt chart for specific work elements by quarter.

Q3: Provisions for M&E

The GRMW proposal notes the funding for monitoring was reduced for all partners, not just the GRMW. They are both using data and information from the three Action Effectiveness Monitoring sites (AEM) and applying the AEM findings and conclusions in project selection and design. The GRMW has a long history of collaborating effectively with ODFW and CRITFC to obtain fish and habitat data, results of landscape modeling, and results from the life cycle models for Catherine Creek and the Upper Grande Ronde River. They have developed a model of habitat suitability that provides spatially continuous, reach-specific information on habitat conditions and suitability for Chinook and steelhead. They are using the life cycle models for projecting juvenile Chinook responses to restoration actions and the contribution of the projects to improving viable salmon population parameters and capacity.

The GRMW has responded positively and effectively to past ISRP recommendations to develop adaptive management processes. The proposal and Appendix B describe their adaptive management plan in detail. They have used the State-of-the-Science annual meetings and their collaboration with CRITFC and ODFW to create an ongoing process linking every phase of the planning, implementation, evaluation, and decision making. Much of this approach is captured in a recent peer-reviewed publication (White et al. 2021), which is valuable resource to all projects supported under the Fish and Wildlife Program. Nevertheless, the description of the adaptive management plan raises questions about how, by whom, and when are the final recommendations made and recorded after conclusions are reached at the State-of-the-Science meeting, and when are the recommendations presented to the Board, TAC, and IT?

The ISRP commends the proponents for continuing to pursue efforts to understand limits to recovery in the basin. The two-year surface water quality assessment of the Grande Ronde River and Catherine Creek is critical to understanding whether poor surface water quality plays a role in historically high levels of outmigrant Chinook salmon smolt mortality observed between February and May. They detected elevated concentrations of copper at four locations, which may be related to the ongoing smolt mortality in the lower reach around La Grande. This has been a major uncertainty and limit on the success of their program for several decades. It will be important to continue working with state and federal agencies to better understand and possibly address the serious water quality issues.

As an Umbrella project dedicated to coordinating restoration and conservation efforts in the Grande Ronde, Wallowa, and Imnaha basins, the GRMW has developed effective working relationships with many partners, including the CTUIR, Nez Perce Tribe, CRITFC, ODFW, NOAA Fisheries, AEM, Lower Snake River Compensation Plan, USFS, USFWS, NRCS (SWCD), OWEB, Freshwater Trust, Trout Unlimited, Wallowa Resources, and counties.

Q4: Results – benefits to fish and wildlife

The proponents thoroughly describe the accomplishments of their past actions in terms of numbers and types of projects, as well as amounts of habitat restored or conserved. The GRMW produced an initial 25-Year Synthesis Report in response to a qualification from the ISRP, even though BPA would not allow BPA funds to be used to develop the report. The ISRP reviewed the report and commended them for completing the draft in spite of the difficulties faced.

However, the ISRP also found that a comprehensive empirical evaluation of the effectiveness of restoration actions on fish populations and demonstrated progress at the landscape level remains to be completed. The ISRP requested the proponents to provide a comprehensive empirical evaluation of the effectiveness of restoration actions on fish populations and demonstrated progress at the landscape level. This proposal starts to provide some of the information on the benefits of their actions on fish and wildlife resources. The project tracks four biological indicators and six habitat indicators as part of its SMART objectives and adaptive management plan. The ODFW Grande Ronde Salmonid Life Cycle Monitoring Project provides annual estimates of Chinook and steelhead growth and survival. The proponents recognize that more analysis and synthesis is needed. They indicate that they are committed to creating a comprehensive synthesis of the program’s benefits for fish and wildlife, and the ISRP encourages the Council and BPA to allow them to use BPA funds to produce this fundamental programmatic element of a large scale, long-term restoration program.

References

Crump, C., L. Naylor, A. Van Sickle, Z. Mathias, and G. Shippentower. 2019. Monitoring and Evaluation of Supplemented Spring Chinook Salmon and Life Histories of Wild Summer Steelhead in the Grande Ronde Basin. Island City: CTUIR.

Favrot, S.D. and B.J. Jonasson. 2020. Fall and Winter Movement Dynamics of Naturally Produced Spring Chinook Salmon Parr in Two Neighboring Interior Pacific Northwest Natal Rivers. Transactions of the American Fisheries Society 149:532-551.

Favrot, S.D., B.J. Jonasson, and J.T. Peterson. 2018. Fall and Winter Microhabitat Use and Suitability for Spring Chinook Salmon Parr in a U.S. Pacific Northwest River. Transactions of the American Fisheries Society 147:151-170

Justice, C., S.M. White, D.A. McCullough, D.S. Graves, and M.R. Blanchard. 2017. "Can stream and riparian restoration offset climate change impacts to salmon populations?" Journal of Environmental Management 188: 212-227.

White, S.M., S. Brandy, C. Justice, K.A. Morinaga, L. Naylor, J. Ruzycki, E.R. Sedell, J. Steele, A. Towne James, G. Webster, and I. Wilson. Progress towards a comprehensive approach for habitat restoration in the Columbia Basin: Case study in the Grande Ronde River. Fisheries, December 4, 2020, fsh.10562. https://doi.org/10.1002/fsh.10562

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The primary goal of Grande Ronde Model Watershed program (GRMW) is the restoration of habitat critical to the survival of native anadromous and resident fish populations. The GRMW coordinates watershed planning activities and funds habitat enhancement projects within the Grande Ronde and Imnaha subbasins. The focus of the program currently is in the Upper Grande Ronde and Catherine Creek.

The GRMW encourages and supports sound land and water management, the local economy, and multiple land uses consistent with sound ecosystem management. Collectively, the GRMW plays a central role in coordinating the actions of numerous regional programs conducted by Tribes, agencies, counties, and landowners. The effort to coordinate local habitat restoration activities and to engage public support more broadly are commendable and consistent with the landscape approach advocated by the ISAB and others. The technical aspects of the project are strong. The GRMW has a long history of accomplishment, trained and experienced staff, and a programmatic network that can maintain adaptive capacity.

The program is significant to regional programs and is consistent with numerous recovery plans directed at habitat protection and recovery including the Grande Ronde and Imnaha Subbasin Plans, the FCRPS Biological Opinion, the Oregon Plan, and the Oregon Watershed Enhancement Board and more recent planning documents including the Atlas Process. The Atlas Process should be very useful to the GRMW program in its project planning and prioritization.

GMRW deserves credit for being proactive in expanding the scope of habitat restoration projects based on past experience, and for seeking to prioritize projects based on feasibility and biological benefits, for example the Stepwise project selection process and the Atlas Process. However, details were lacking on how feasibility and biological benefits are judged.

The Objectives and background are well described. The Objectives presented in the proposal, for example restore habitat connectivity and enhance floodplain connectivity, represent the desired outcomes of the suite of habitat enhancement projects funded through the GRMW program. The proposal includes adequate background information on the nature of the habitat problems being addressed, and extensive summaries of past activities with links to detailed results at individual sites. The results, in terms of individual projects, are impressive.

The Objectives are clearly related to the overall goal of improving native fish populations. The proposal, however, does not provide a compelling overview of progress towards achieving the program's Objectives, especially whether progress is being made in improving freshwater survival and growth of native fishes. Determining whether the GRMW is accomplishing its goals of habitat enhancement and improved freshwater fish productivity requires effectiveness monitoring, as emphasized by the ISRP in its previous review of this project. Effectiveness monitoring traditionally has not been a central component of the activities. The project has been in place since 1992, but it appears that effectiveness monitoring was only recently implemented.

With regard to this point, the sponsors make an important observation on p. 3: “Both the U.S. District Court in Oregon and the Court of Appeals for the Ninth Circuit have held that the ESA standard of jeopardy requires NOAA Fisheries to consider not only whether the species will survive but how the prospective actions (including habitat improvement projects) will affect the species’ prospects for recovery.” The ISRP interprets this as meaning that the results of restoration actions need to be quantified via effectiveness monitoring or the use of quantitatively based models to predict outcomes. It does not appear that this is being done at a scale and scope which will meet this criterion.

The GRMW recognizes the importance of effectiveness monitoring but states that it is constrained by lack of funding. The GRMW has approached the issue in at least three ways. First, for each objective they propose metrics or measures to evaluate project success. The metrics, for example miles of fencing and acres of riparian planting, pertain mostly to implementation with the assumption that they are having the desired outcome of improving habitat conditions and fish abundance. This is a complex and uncertain assumption but, in lieu of M&E, it is understandable from a practical perspective, depending on whether there is a direct relationship between the metrics and the desired outcomes of habitat improvement such as restoration of habitat connectivity and enhancement of floodplain connectivity. This assumption may be generally true, but it does not provide a quantitative assessment of actual habitat improvements. For example, are riparian plantings and other efforts to enhance riparian areas stabilizing banks, providing shade, and reducing water temperature? Perhaps most importantly, are these actions benefitting fish? This can only be demonstrated though M&E.

Second, the sponsors state that they will rely on ODFW and CRITFC monitoring projects to provide “overall watershed habitat status." It would be helpful if the sponsors had provided more detail regarding the way that these projects will satisfy the need for effectiveness monitoring of GRMW projects.

Third, the GRMW has made an effort to incorporate more site specific monitoring in the individual projects funded through their program. This is a positive step, and the ISRP recommends that this effort continue and expand in the future. The effectiveness of the GRMW program ultimately depends on the cumulative success of the individual projects in improving habitat. It would be helpful if the sponsors had provided more detailed information about this effort, including the responsibility of the GRMW in planning and design of the monitoring process, as part of its coordination role.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

The GRMW has successfully implemented an impressive number of projects. The Stepwise procedure developed for project selection, prioritization, and funding is a formalized process directly involving cooperators and includes technical review of each proposed project. This process helps ensure that individual projects share a common goal, that they are working in defined priority areas within the subbasins, and that closer cooperation among projects is fostered.

The Stepwise process, although useful, has limitations. It does not establish landscape scale priorities; rather, it assists the sponsors in developing and implementing individual projects. This limitation is important to recognize; how are priorities set at the landscape scale and the project class in terms of having a positive effect on fish survival? Further, how is the “biological benefit score” established and what are the components used to develop this quantitative score?

Results would be more meaningful if the sponsors presented at least a quantitative summary of how the projects it funds, taken together, have improved riparian and stream processes and freshwater survival of fish. If monitoring continues, as it should, the sponsors should develop an effective way of synthesizing results of individual projects to provide a “big picture” view of the success of the GMRW project as a whole.

Program management appears to have adapted appropriately to experience gained over 20 years, but this adaptation seems to have been passive rather than active. Adaptive management, as originally intended, requires deliberate experimentation to acquire the knowledge to reduce key uncertainties, with the goal of improving future decisions, and long-term benefits. Monitoring and evaluation are critical to such an adaptive management approach. Linking local monitoring of site specific projects to CHaMP methods used at watershed scales seems like an appropriate strategy given limited funding.

While learning is occurring at the program scale and at the scale of individual projects, the effectiveness of the adaptive management process could be vastly improved with the use of quantitative hypotheses or goals and the judicious use of reference sites for single actions or a group of actions. This would allow timely evaluation of effectiveness, and possibly the discovery of underlying mechanisms, and thereby improve learning.

Evaluation of Results

The purpose of the GRMW is to select, review, prioritize and fund habitat protection and restoration projects intended to benefit ESA-listed salmon and other fish species in the Grande Ronde and Imnaha Subbasins. The GRMW is a well-established and successful program that appears to have established stable and deeply rooted relationships with cooperators in the Grande Ronde subbasin. Its accomplishments since its inception in 1992 are impressive. The GRMW appears to be a well-managed program and, with the development of the Stepwise process, has improved its procedure for selection, review, and prioritization of projects. A strength of this program is its close working relationship with state and local governments, Tribes, conservation groups, private landowners, and other local public interests to coordinate habitat restoration projects on state and public lands.

The Grande Ronde Model Watershed Foundation, established by the GRMW, contracts directly with BPA and other funding sources to fund and implement restoration projects. Working through the Foundation, the GRMW with its cooperators is able to consolidate and coordinate habitat restoration planning at a subbasin scale and, through a formalized, structured process for project selection, helps ensure that projects address limiting factors in priority watersheds identified in the subbasin and other plans (and in the upcoming Atlas). Because the GRMW provides funding for projects it can exercise considerable influence on project selection and implementation. The existence of a single entity, such as the GRMW, responsible for planning and project selection within a subbasin should be considered in other subbasins where coordination among habitat restoration projects appears to be more loosely defined.

Determining whether the GRMW is accomplishing its goals of improving habitat and freshwater productivity of fish requires a robust effectiveness monitoring program. At present, monitoring is not sufficient to clearly demonstrate positive impacts of habitat improvement actions on fish. The GRMW should develop an effectiveness monitoring program that is capable of demonstrating quantitatively progress toward achieving the objectives of the individual projects funded through the GRMW and of the GRMW as a whole.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The success of this project requires close coordination with agencies, tribes, and the public. The GRMW seems to have been very successful in developing and maintaining these relationships and enfranchising a wide range of stakeholders.

The sponsors recognize climate change, non-native plants, and toxic chemicals as emerging limiting factors. In reality, these are not emerging limiting factors but ones that are already present at significant levels. As such, they should be addressed directly by program actions. An additional “emerging limiting factor” may be increasing agricultural demands for water, and this could be examined through scenarios, at a minimum, or the use of quantitative models/trend analyses. Flow restoration will need to operate in cooperation with agricultural demands and climate change. The project needs to have a strong understanding as to how these factors may impact future water supplies and timing.

Administration and overhead are 34% of the budget. This seems high compared to other similar projects. Is there justification for this high rate? If so, a detailed justification should be provided, especially so in that rent/utilities are a line item in the budget; these items are usually covered under overhead except in exceptional circumstances.

4. Deliverables, Work Elements, Metrics, and Methods

The project Deliverables are clearly linked to methods and individual restoration Objectives and should help meet the stated Objectives. Most of the Deliverables are classes of enhancement actions that will be undertaken by projects funded through GRMW. The specific projects that will be recommended for funding are given for each Deliverable. The ISRP assumes that these projects have already passed the Stepwise review process. A Deliverable as well as an Objective addressing M&E should be included. This Deliverable should specify the procedures the GRMW program will use to allocate funding for M&E. Will the GRMW propose guidelines for M&E for individual projects and will these guidelines or requirement be integrated into the Stepwise process for project selection? A more formalized process for M&E that applies to all projects funded through the GRMW is needed.

It was refreshing to see that the Deliverables were quantitative in terms of actions to be completed. The sociological results and benefits were highlighted in the Executive Summary but only lightly touched upon in the text. This is a highly important aspect central to the overall success in meeting programmatic goals. It should be directly addressed in the text, especially in the Work Elements and Deliverables.

Specific comments on protocols and methods described in MonitoringMethods.org

No comments.

===========QUALIFICATIONS FOLLOW================

The Grande Ronde Model Watershed is a strong, well organized program, and has had considerable success in implementing a large number of habitat enhancement projects. However, it needs to improve effectiveness monitoring and the adaptive management process to incorporate climate change, toxic chemicals, and non-native plants into the active program, and set priorities at the landscape scale. Results should be judged in terms of improvements to freshwater survival and productivity of fish.

Analysis of monitoring data often lags behind data collection. The sponsors should consider enlisting the assistance of NOAA-Fisheries early in the process to assist with the design of monitoring actions and with data analysis.

An Objective and Deliverable pertaining to M&E should be included in future proposals. An M&E Objective signifies a commitment to monitoring, especially effectiveness monitoring.

In future proposals quantitative details should be provided on how past and current actions are influencing survival and growth of native fishes. This should include monitoring results and how the results have altered actions through the adaptive management process.

Develop plans and actions to fully integrate climate change, toxic chemicals, non-native species, and agricultural water demands into an effective program.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The primary goal of Grande Ronde Model Watershed program (GRMW) is the restoration of habitat critical to the survival of native anadromous and resident fish populations. The GRMW coordinates watershed planning activities and funds habitat enhancement projects within the Grande Ronde and Imnaha subbasins. The focus of the program currently is in the Upper Grande Ronde and Catherine Creek.

The GRMW encourages and supports sound land and water management, the local economy, and multiple land uses consistent with sound ecosystem management. Collectively, the GRMW plays a central role in coordinating the actions of numerous regional programs conducted by Tribes, agencies, counties, and landowners. The effort to coordinate local habitat restoration activities and to engage public support more broadly are commendable and consistent with the landscape approach advocated by the ISAB and others. The technical aspects of the project are strong. The GRMW has a long history of accomplishment, trained and experienced staff, and a programmatic network that can maintain adaptive capacity.

The program is significant to regional programs and is consistent with numerous recovery plans directed at habitat protection and recovery including the Grande Ronde and Imnaha Subbasin Plans, the FCRPS Biological Opinion, the Oregon Plan, and the Oregon Watershed Enhancement Board and more recent planning documents including the Atlas Process. The Atlas Process should be very useful to the GRMW program in its project planning and prioritization.

GMRW deserves credit for being proactive in expanding the scope of habitat restoration projects based on past experience, and for seeking to prioritize projects based on feasibility and biological benefits, for example the Stepwise project selection process and the Atlas Process. However, details were lacking on how feasibility and biological benefits are judged.

The Objectives and background are well described. The Objectives presented in the proposal, for example restore habitat connectivity and enhance floodplain connectivity, represent the desired outcomes of the suite of habitat enhancement projects funded through the GRMW program. The proposal includes adequate background information on the nature of the habitat problems being addressed, and extensive summaries of past activities with links to detailed results at individual sites. The results, in terms of individual projects, are impressive.

The Objectives are clearly related to the overall goal of improving native fish populations. The proposal, however, does not provide a compelling overview of progress towards achieving the program's Objectives, especially whether progress is being made in improving freshwater survival and growth of native fishes. Determining whether the GRMW is accomplishing its goals of habitat enhancement and improved freshwater fish productivity requires effectiveness monitoring, as emphasized by the ISRP in its previous review of this project. Effectiveness monitoring traditionally has not been a central component of the activities. The project has been in place since 1992, but it appears that effectiveness monitoring was only recently implemented.

With regard to this point, the sponsors make an important observation on p. 3: “Both the U.S. District Court in Oregon and the Court of Appeals for the Ninth Circuit have held that the ESA standard of jeopardy requires NOAA Fisheries to consider not only whether the species will survive but how the prospective actions (including habitat improvement projects) will affect the species’ prospects for recovery.” The ISRP interprets this as meaning that the results of restoration actions need to be quantified via effectiveness monitoring or the use of quantitatively based models to predict outcomes. It does not appear that this is being done at a scale and scope which will meet this criterion.

The GRMW recognizes the importance of effectiveness monitoring but states that it is constrained by lack of funding. The GRMW has approached the issue in at least three ways. First, for each objective they propose metrics or measures to evaluate project success. The metrics, for example miles of fencing and acres of riparian planting, pertain mostly to implementation with the assumption that they are having the desired outcome of improving habitat conditions and fish abundance. This is a complex and uncertain assumption but, in lieu of M&E, it is understandable from a practical perspective, depending on whether there is a direct relationship between the metrics and the desired outcomes of habitat improvement such as restoration of habitat connectivity and enhancement of floodplain connectivity. This assumption may be generally true, but it does not provide a quantitative assessment of actual habitat improvements. For example, are riparian plantings and other efforts to enhance riparian areas stabilizing banks, providing shade, and reducing water temperature? Perhaps most importantly, are these actions benefitting fish? This can only be demonstrated though M&E.

Second, the sponsors state that they will rely on ODFW and CRITFC monitoring projects to provide “overall watershed habitat status." It would be helpful if the sponsors had provided more detail regarding the way that these projects will satisfy the need for effectiveness monitoring of GRMW projects.

Third, the GRMW has made an effort to incorporate more site specific monitoring in the individual projects funded through their program. This is a positive step, and the ISRP recommends that this effort continue and expand in the future. The effectiveness of the GRMW program ultimately depends on the cumulative success of the individual projects in improving habitat. It would be helpful if the sponsors had provided more detailed information about this effort, including the responsibility of the GRMW in planning and design of the monitoring process, as part of its coordination role.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

The GRMW has successfully implemented an impressive number of projects. The Stepwise procedure developed for project selection, prioritization, and funding is a formalized process directly involving cooperators and includes technical review of each proposed project. This process helps ensure that individual projects share a common goal, that they are working in defined priority areas within the subbasins, and that closer cooperation among projects is fostered.

The Stepwise process, although useful, has limitations. It does not establish landscape scale priorities; rather, it assists the sponsors in developing and implementing individual projects. This limitation is important to recognize; how are priorities set at the landscape scale and the project class in terms of having a positive effect on fish survival? Further, how is the “biological benefit score” established and what are the components used to develop this quantitative score?

Results would be more meaningful if the sponsors presented at least a quantitative summary of how the projects it funds, taken together, have improved riparian and stream processes and freshwater survival of fish. If monitoring continues, as it should, the sponsors should develop an effective way of synthesizing results of individual projects to provide a “big picture” view of the success of the GMRW project as a whole.

Program management appears to have adapted appropriately to experience gained over 20 years, but this adaptation seems to have been passive rather than active. Adaptive management, as originally intended, requires deliberate experimentation to acquire the knowledge to reduce key uncertainties, with the goal of improving future decisions, and long-term benefits. Monitoring and evaluation are critical to such an adaptive management approach. Linking local monitoring of site specific projects to CHaMP methods used at watershed scales seems like an appropriate strategy given limited funding.

While learning is occurring at the program scale and at the scale of individual projects, the effectiveness of the adaptive management process could be vastly improved with the use of quantitative hypotheses or goals and the judicious use of reference sites for single actions or a group of actions. This would allow timely evaluation of effectiveness, and possibly the discovery of underlying mechanisms, and thereby improve learning.

Evaluation of Results

The purpose of the GRMW is to select, review, prioritize and fund habitat protection and restoration projects intended to benefit ESA-listed salmon and other fish species in the Grande Ronde and Imnaha Subbasins. The GRMW is a well-established and successful program that appears to have established stable and deeply rooted relationships with cooperators in the Grande Ronde subbasin. Its accomplishments since its inception in 1992 are impressive. The GRMW appears to be a well-managed program and, with the development of the Stepwise process, has improved its procedure for selection, review, and prioritization of projects. A strength of this program is its close working relationship with state and local governments, Tribes, conservation groups, private landowners, and other local public interests to coordinate habitat restoration projects on state and public lands.

The Grande Ronde Model Watershed Foundation, established by the GRMW, contracts directly with BPA and other funding sources to fund and implement restoration projects. Working through the Foundation, the GRMW with its cooperators is able to consolidate and coordinate habitat restoration planning at a subbasin scale and, through a formalized, structured process for project selection, helps ensure that projects address limiting factors in priority watersheds identified in the subbasin and other plans (and in the upcoming Atlas). Because the GRMW provides funding for projects it can exercise considerable influence on project selection and implementation. The existence of a single entity, such as the GRMW, responsible for planning and project selection within a subbasin should be considered in other subbasins where coordination among habitat restoration projects appears to be more loosely defined.

Determining whether the GRMW is accomplishing its goals of improving habitat and freshwater productivity of fish requires a robust effectiveness monitoring program. At present, monitoring is not sufficient to clearly demonstrate positive impacts of habitat improvement actions on fish. The GRMW should develop an effectiveness monitoring program that is capable of demonstrating quantitatively progress toward achieving the objectives of the individual projects funded through the GRMW and of the GRMW as a whole.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The success of this project requires close coordination with agencies, tribes, and the public. The GRMW seems to have been very successful in developing and maintaining these relationships and enfranchising a wide range of stakeholders.

The sponsors recognize climate change, non-native plants, and toxic chemicals as emerging limiting factors. In reality, these are not emerging limiting factors but ones that are already present at significant levels. As such, they should be addressed directly by program actions. An additional “emerging limiting factor” may be increasing agricultural demands for water, and this could be examined through scenarios, at a minimum, or the use of quantitative models/trend analyses. Flow restoration will need to operate in cooperation with agricultural demands and climate change. The project needs to have a strong understanding as to how these factors may impact future water supplies and timing.

Administration and overhead are 34% of the budget. This seems high compared to other similar projects. Is there justification for this high rate? If so, a detailed justification should be provided, especially so in that rent/utilities are a line item in the budget; these items are usually covered under overhead except in exceptional circumstances.

4. Deliverables, Work Elements, Metrics, and Methods

The project Deliverables are clearly linked to methods and individual restoration Objectives and should help meet the stated Objectives. Most of the Deliverables are classes of enhancement actions that will be undertaken by projects funded through GRMW. The specific projects that will be recommended for funding are given for each Deliverable. The ISRP assumes that these projects have already passed the Stepwise review process. A Deliverable as well as an Objective addressing M&E should be included. This Deliverable should specify the procedures the GRMW program will use to allocate funding for M&E. Will the GRMW propose guidelines for M&E for individual projects and will these guidelines or requirement be integrated into the Stepwise process for project selection? A more formalized process for M&E that applies to all projects funded through the GRMW is needed.

It was refreshing to see that the Deliverables were quantitative in terms of actions to be completed. The sociological results and benefits were highlighted in the Executive Summary but only lightly touched upon in the text. This is a highly important aspect central to the overall success in meeting programmatic goals. It should be directly addressed in the text, especially in the Work Elements and Deliverables.

Specific comments on protocols and methods described in MonitoringMethods.org

No comments.

===========QUALIFICATIONS FOLLOW================

The Grande Ronde Model Watershed is a strong, well organized program, and has had considerable success in implementing a large number of habitat enhancement projects. However, it needs to improve effectiveness monitoring and the adaptive management process to incorporate climate change, toxic chemicals, and non-native plants into the active program, and set priorities at the landscape scale. Results should be judged in terms of improvements to freshwater survival and productivity of fish.

Analysis of monitoring data often lags behind data collection. The sponsors should consider enlisting the assistance of NOAA-Fisheries early in the process to assist with the design of monitoring actions and with data analysis.

Modified by Dal Marsters on 6/11/2013 1:03:39 PM.

The sponsors have satisfactorily addressed the ISRP's concerns and we thank them for clarifying several important issues regarding the operation of the Grand Ronde Model Watershed Program (GRMWP). The ISRP reiterates that the GRMWP has been highly successful in implementing projects and has an outstanding record of cooperative work among government and private entities. A central ISRP concern about the GRMWP was that the proposal did not provide an adequate summary of project effectiveness and monitoring. The sponsors make the point that compiling the results of 150 projects would yield benefits but is precluded due to fiscal limitations related to the 5% budget limitation imposed by BPA. The ISRP appreciates the sponsor's willingness to undertake this assessment, which apparently would largely require compilation of existing records, and encourages the NPCC and BPA to provide funds for this effort. This expenditure would be appropriate because the GRMWP is the largest program of its type in the basin -- truly a "model" as the name implies -- and the assessment would allow a better evaluation of the success of the program.

Qualification: The sponsors should develop a report presenting quantitative and qualitative results to date pertaining to the effectiveness of the projects under their domain, a general summary and conclusions about overall project effectiveness, and the application of the results to management. The sponsors should report positive results as well as results from projects that to date may not yet have produced significant effects. This effort should be funded by BPA and reviewed by the ISRP in FY07. The response of the sponsors of project # 199608300 may provide some guidance for preparation of the report.