The ISRP was pleased that the proponents oriented their project around the First Foods concept, especially as a framework for the objectives and the monitoring activities. The CTUIR not only use the concept for project guidance and governance but also include it in the restoration objectives for the North Fork of the John Day basin in Appendix 3. This is a valuable example of how integration of the Tribe’s traditional knowledge with regional scientific knowledge provides strength to both and, at the same time, increases the cultural and social relevance of the overall effort. The ISRP encourages the CTUIR to continue to highlight the benefits of recognizing diverse knowledge sources in order to achieve goals for fish and wildlife in the Columbia River Basin.

The ISRP highlights four suggestions to improve the project and its future evaluation:

1. SMART objectives. The SMART objectives should be in the main proposal narrative in Section 3. Incorporating these would greatly improve the ability of reviewers to evaluate the program accurately.
2. M&E matrix - support. As habitat projects and monitoring projects are not presented as part of an integrated proposal or plan, the need for a crosswalk to identify the linkages between implementation and monitoring is extremely important for basins or geographic areas. The ISRP is requesting a response from the Escapement and Productivity of Spring Chinook and Steelhead Project (199801600) to summarize the linkages between implementation and monitoring projects in the basin. During the response loop (September 24 to November 22, 2021), we ask your project to assist them in creating the summary and provide information about what is being monitored for your implementation project and about where and when the monitoring occurs. A map or maps of locations of monitoring actions would be helpful in this regard.
3. Examples of restoration outcomes. In a previous review, the ISRP asked that monitoring results be emphasized to a greater degree. The proposal does a great job of providing links to other reports, but a few examples showing changes in key properties over time would have been helpful.
4. Benefits to fish and wildlife. Since project activities are starting to be reflected in benefits to fish and to environmental quality, the ISRP recommends that the project develop plans for a comprehensive analysis and a description of the collective benefits of activities over the last two decades. This synthesis would be timely and useful.

Q1: Clearly defined objectives and outcomes

This is a well-functioning and comprehensive project, one that has greatly matured in its approaches and outcomes over the past decade. The objectives, in a general sense, are on target even though the longer-term ecological outcomes may be difficult to predict at this time.

The proponents have internally developed SMART objectives for the funding period, but the details are buried in Appendix 3. The ISRP emphasizes that the SMART objectives should appear as part of the main proposal narrative in Section 3 (Goals and Objectives). This will facilitate future evaluations of progress toward expected outcomes. Additionally, the project could provide more specific outcomes for some objectives. For instance, one objective is to “increase and reestablish in-stream thermal diversity throughout the year.” What metric will be used to quantify thermal diversity and what level of thermal diversity is the desired outcome?

Q2: Methods

The methods are based largely on established best management practices and are reasonable for the actions being proposed.

Table 2 lists examples of outreach efforts associated with certain project objectives, and it is noted later in the Potential Confounding Factors section that landowner priorities sometimes shift. Given the rich scope of this project, reviewers wonder if more outreach opportunities exist, and if they may not be of growing importance given the environmental challenges facing the basin (i.e., invasive species, changing climate). The ISRP also sees great value in communicating project results with others outside the basin, thereby encouraging even broader engagement/outreach efforts.

Q3: Provisions for M&E

The proponents have developed an effective Adaptive Management process, especially with the sustained involvement of additional collaborators. While it is not always clear who will be doing monitoring for specific project outcomes (there are numerous organizations involved), it is important that the data are appropriate for the activity and that they are readily available, which appears to be the case.

The proposal notes that the status of beaver populations is low, but that no formal census is available. Based on their importance to channel maintenance, is a status evaluation warranted? Two additional alternatives would be for the project to develop a few beaver reintroduction projects in areas where beaver could enhance existing efforts and to adopt a beaver-tolerance policy.

Given the emphasis on climate change in the confounding factors section, it is not clear whether monitoring is evaluating the success of efforts to preserve cooler water temperatures. Are these efforts working? In the Confounding Factors section, modeling water temperatures is mentioned, and while a model is good for broadly assessing vulnerability to changes in climate, such large-scale modeling typically does not account for the type of site-specific management actions occurring throughout the basin. Therefore, it would be helpful for future work to investigate water temperatures above and below restoration locations to evaluate the effectiveness of specific actions.

Q4: Results – benefits to fish and wildlife

Project activities are starting to be reflected in benefits to fish as well as to environmental quality. Considering the longevity of the project, a more thorough evaluation of the benefits would be timely. The ISRP suggests that the project develop plans for a comprehensive analysis and description of the collective benefits of activities that have taken place over the last two decades for fish and wildlife.

There are potentially many good aspects to the proposal, but the proposal’s narrative and the responses to several of the ISRP’s qualifications are unclear. The main qualifications are that the sponsors need to better explain the proposal rationale and to enlist the collaboration of specialists to aid in project implementation and evaluation. It is also important that relevant RM&E efforts outside this project are well coordinated with project activities listed in this proposal. More specifically, the sponsors need to address several issues that arose from their responses to the ISRP’s questions on the original proposal: see qualifications.

Responses to these qualifications should be submitted for ISRP review by the end of 2013.

The goal is much broader than the stated objectives of the project. The goal should be revised to reflect a balance with the objectives, or vice versa. As presented the objectives are not comprehensive enough to attain the goal.

The objectives should be stated in quantitative terms and time lines provided. As stated, the Deliverables are generally fine, but since the Objectives are not stated quantitatively, these need to be so. Quantitatively recasting the deliverables as environmental benefits or improvements expected to be realized after the individual projects are completed is essential to evaluate the project success.

The ISRP would still like to see the monitoring results collected since 2007. Please provide appropriate metrics and data to show that the restoration actions are making progress.

What is being done to control or eradicate non-native fishes? If this is an important issue with respect to the recovery of native salmonids, as it seems to be, it should be a program component.

The response does not address the question about how fish monitoring data are used to evaluate the effectiveness of habitat projects and only partially addresses specific collaborations between projects. The ISRP needs more fully developed responses to these questions in order to evaluate the proposal. As well, the ISRP expects that considerable ongoing collaboration will occur among the various programs.

The ISRP would appreciate clarification to the following issue: The sponsors state that they will reconcile their monitoring plan with other habitat monitoring plans such as CTUIR's Fisheries Habitat Monitoring Plan, CHaMP and ISEMP, but more description is needed on what will be done.

The ISRP is unsure what the response to No. 11 means. Please provide a revised response for consideration by the ISRP.

The details of cost-sharing, who will do the work among the partners, and timelines for completion, are not provided under Response No. 3. Please provide them.

Issues of data management, as requested in the ISRP qualification, are not addressed under Response No. 5. They should be described in a response.

This is a very ambitious project that should proceed in prioritized stages or program phases. CTUIR should prioritize actions and implement them as a means to develop expertise and better achieve success. Further, the sponsors should consider establishing a scientific advisory committee to assist with the staging of project phases and prioritization of activities.

The sponsors need to address the following issues in a response:

The status and direction of the RME program needs clarification. Are the sponsors modifying the program and, if so, how? What changes will be made and why? What is the status of data analysis? Is data analysis ongoing and, if so, when can results be expected? What is the relationship between this project's RME and CTUIR's Biomonitoring Plan and Fisheries Habitat Management Plan? How is the RME for this project similar to and different from these plans? If the sponsors are modifying their RME, what specific elements of the two plans will be incorporated?

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The project is consistent with a number of regional plans including the John Day Subbasin Plan, the Mid-Columbia Steelhead Recovery Plan, the FCRPS BiOp, and the Fish Accords Agreement. The North Fork John Day supports the largest populations of ESA-listed spring Chinook salmon and summer steelhead, and maintains some of the highest quality habitat in the subbasin. In general, the technical background adequately reviews limiting factors and fish population abundances in the study area.

Nevertheless, this was a difficult proposal to understand and evaluate, for three main reasons:

The proposal was poorly written in terms of clarity of ideas and actions, extensive use of vague words to describe outcomes, and use of acronyms without definition. Proofreading was also needed. The document should be carefully edited. In a number of instances statements in the same paragraph appear to contradict each other.

The goal of the project was not clearly stated. For example, in the Executive Summary the goal/purpose of the project varies in three separate paragraphs. In the first instance, it is stated as “This project protects, enhances, and restores functional floodplain, channel and watershed processes to provide sustainable and healthy habitat and water quality for aquatic species in the John Day River Subbasin.” In the second instance it is stated as “The purpose of this project is to protect and enhance habitat for improved natural production of indigenous, Mid-Columbia River (MCR) Evolutionary Significant Unit (ESU) summer steelhead (Oncorhynchus mykiss), listed as threatened under the Federal Endangered Species Act (ESA), and spring Chinook salmon (Oncorhynchus tshawytscha) within the North Fork of the John Day River Basin.” In the third instance it is stated as “The goal of the CTUIR North Fork John Day Habitat Enhancement Project (the Project) is to protect, enhance, and restore channel, riparian, and floodplain function and function relating these locations to upland adjacent upland areas using a ‘ridge top to ridge top’ approach to provide sustainable and healthy habitat and water quality for aquatic species in the North Fork John Day River Subbasin.” Although related in spirit, these are not the same. As such, it was very difficult to equate objectives and evaluate activities in the proposal to the stated goal.

Ten Objectives are provided but, for several, there are no deliverables (OBJ 4, 7, 8, 9, and 10). The topics related to the Objectives are discussed in the text, and they are listed as important concerns, but it is not clear how they will be addressed.

There is no overarching model or form of Structured Decision Making to guide the activities or set priorities, and this hampers taking a comprehensive approach to restoration. The activities, while individually important, are not treated as an integrated network of sites and actions chosen for their effectiveness at meeting clearly stated goals. Further, many sites are not monitored to determine if the actions have been effective, thereby undermining the Adaptive Management process.

A coherent discussion of the strategy for selecting and prioritizing restoration sites would have improved the proposal. In particular, since the NFJD supports significant areas of high quality habitat, it would be helpful to know how the project sites are located relative to these habitats and whether the location of these areas is considered in site selection.

The objectives appear sufficient to address the major limiting factors in the North Fork John Day.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

A number of habitat enhancement projects have been initiated in the North Fork John Day since the project’s inception. While the sponsors summarized habitat enhancement actions for a number of projects, discussed the outcomes of these actions to date, and provided pre-and post-project photographs, few quantitative results were presented. Has the monitoring data been analyzed and, if not, what are the plans for data analysis? The project needs to establish a comprehensive model or institute Structured Decision Making, as well as monitoring, to guide actions and evaluate outcomes.

During the 12-year project history, the CTUIR has helped administer and implement 29 efforts, developed 26 stock water sites to help protect 9.7 miles of stream channel, and entered 1600 acres of riparian, floodplain, and upland areas into Conservation Agreements. Additional cooperative work constructed 24.75 miles of riparian exclusion fence outside of the Conservation Agreements, replaced 5 passage barriers, provided weed control on over 300 acres, and redistributed 276,640 cubic yards of mine tailings. Additional work to develop efforts which did not mature included a fence construction, a watershed analysis, and a range inventory in the Desolation GA, aspen plantings with associated fencing along Upper Owens Creek (Lower Camas Creek GA), guzzler development above Rudio Creek (Lower NFJD GA), and road stabilization above Ukiah, Oregon (Lower Camas Creek GA) where landowners backed off of cooperative efforts, and one boundary fence and culvert replacement in the Desolation Creek GA dropped due to a shortfall in available funding. This equates to ~2.4 efforts annually, ~3 miles of stream protection, and ~133 acres of conservation.

Due to the lack of consistent monitoring, it is not clear that the individual or collective actions are having positive effects on focal species or environmental concerns. Further, without a general model or Structured Decision Making, it is not clear that the efforts are targeted at sites with a strong potential for aiding species’ recovery or ameliorating environmental concerns.

Several topics, which the ISRP suspects are locally important, are only lightly touched upon in the proposal. These are invasive non-native plants in riparian areas, impacts of non-native fishes on native populations, use of agricultural chemicals (toxics), browsing by native ungulates in restored areas, and strategies concerning beaver. These should be addressed in a substantive way in the proposed actions.

The ISRP is surprised and concerned that climate change models and scenarios are not consulted when planning activities. After all, on-the-ground activities need to be resilient to future environmental changes; there are several “user friendly” techniques available.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The sponsors could have provided a more detailed discussion of the relationship between their project and others that are ongoing in the North Fork John Day. For example, how are fish monitoring data collected by Project 1998-016-00: "Escapement and Productivity of Spring Chinook and Steelhead" used to evaluate the effectiveness of the habitat projects? In addition, the sponsors could have discussed in more detail the coordination with ODFW’s John Day Habitat Enhancement project (1984-021-00). For example, what sort of collaboration between the projects is occurring? Are sites being selected in a complementary way so as to optimize the potential benefits of habitat enhancement actions?

The status of the RM&E program, especially of effectiveness monitoring, and whether there are plans to modify the program, as the proposal implies, needs clarification. A concise overview of the M&E plan would be helpful including whether data collection at project sites and data analysis is currently being undertaken. The sponsors state that they will “reconcile” their monitoring plan with other habitat monitoring plans such as CTUIR’s Fisheries Habitat Monitoring Plan, CHaMP and ISEMP but it is not clear what they mean by “reconcile.” The sponsors present a lengthy discussion of CTUIR’s Biomonitoring Plan. How does this Plan relate to current project monitoring? Will elements of the Biomonitoring Plan be incorporated into a revised M&E plan for this project? Clarification of the status and direction of the project's monitoring program is needed.

The sponsors recognize climate, non-native plants, predation, and toxic chemicals as emerging limiting factors – and this is good to see. However, in reality, these are not emerging limiting factors but ones that are already present at significant levels. As such, they should be addressed directly by program actions.

There are ongoing program relationships with landowners, the U.S. Forest Service, local counties, and others. It is a small community, and the ISRP suspects there is ongoing communication at several levels. Our deeper concern is at a larger spatial scale. There are several other entities in the region proposing similar restoration actions. Efforts should be made by all entities, and coordinated by the Council, BPA and other funding agencies, to see that working relationships are established at the larger spatial scale. This will encourage local learning and build regional adaptive capacity.

4. Deliverables, Work Elements, Metrics, and Methods

Although the sponsors refer to monitoring methods and metrics in MonitoringMethods.org, the extent of this project’s monitoring in the North Fork, especially effectiveness monitoring, is unclear.

There are no deliverables for 5 of the 10 Objectives; this is mentioned above. The Objectives need to be recast as quantitative statements to identify specifically and quantitatively what will be achieved and provide realistic timelines. The deliverables need to reflect this quantitative approach.

Many of the deliverables, as stated, are generally fine. However, there are numerous specific questions about details of cost-sharing, who will do the work among the partners, and timelines for completion.

Data management: It appears that there is some in-house data management and perhaps some cooperation with partners, but the levels of sophistication and analyses are far from clear. These aspects should be fully articulated in a revision to this proposal.

Specific comments on protocols and methods described in MonitoringMethods.org

No comments at this time.

Modified by Dal Marsters on 9/26/2013 2:02:02 PM.

Chinook Mid-Columbia ESU steelhead, Mid-Columbia ESU bull trout and interior redband trout should all realize long-term benefits from the habitat improvements proposed. This project is well planned, and the objectives and methods have been thought through. Clear ties are made to the Fish and Wildlife Program, the BPA Watershed Management Program, the BiOp RPAs, Wy-Kan-Ush-Mi Wa-Kish-Wit, and the Subbasin Plan. There are many complementarities between this project and others in the subbasin, with clear descriptions of who does what, how they are related, and presentation of the role of CTUIR in the communities and watershed council.

This project proposes tributary habitat improvements in priority areas identified in the Subbasin Plan and tied to EDT results. Habitat limiting factors are linked with strategies and restoration activities. Detailed descriptions of habitat problems and activities to date are provided by geographic area. There is a clear description of project history and actions, but little evaluation of project outcomes and impacts. A table lays out the rationale for proposed actions. Objectives are specific to location, expressed in measurable units and relate actions to time lags for discerning measurable effects. Work elements are similarly specific, with milestones and dates. M&E will be done through collection of well-described, pre- and-post implementation data on channel hydrology and vegetative response. No direct monitoring of fish use of habitat. The sponsors should coordinate with ODFW so that fish monitoring occurs and can be tied to habitat improvements. Information transfer is accomplished through outreach and education activities, watershed council participation, landowner collaborations, and periodic reporting.