The proponents did not explicitly address the six topics in our response request, but instead they responded to our text for the four ISRP comments that followed the response requests. We reviewed the information provided by the proponents and attempted to evaluate the response in terms of the six topics we asked them to address.

The ISRP requests the proponents to provide information on the following three conditions in the next annual report and future workplans. Because of the importance of the proposal as a guiding document for the project, we encourage the proponents to also revise their proposal to reflect these additions.

1. SMART objectives. The proponents should provide SMART objectives with explicit quantitative definitions of the project’s expected overall ecological outcomes. They should explain the level of ecological response that they would consider meets the objective. 
   
   
2. SMART objectives. The proponents should describe their adaptive management process, including the regular process of project planning, assessment and evaluation, decision making, and the mechanism for determining and recording adaptive actions. They should identify the participants expected to contribute in each step of the adaptive management process. 
   
   
3. SMART objectives. By the end of the project funding period, the proponents should provide an update to their previous comprehensive assessment in 2008. This synthesis should quantify the overall contribution of their 37 years of protection and restoration of riparian habitats in the John Day River basin and the contributions to overall watershed conditions. The synthesis should identify the project’s benefits to fish and wildlife and the degree to which the project has improved limiting factors within the John Day subbasin.

In our preliminary review, we requested a response on the topics listed below. Our final comments based on the response are provided after each topic:

1. SMART objectives. The proponent emphasizes that they are not contractually obligated to monitor their actions to the level necessary to quantify the degree to which they achieved the objective’s targets. They indicate that “objectives and methods that are quantifiable, can be measured, and have been proven to achieve our stated goals referenced by several peer reviewed publications provided.”

However, the overall project outcomes provided in the proponents’ response are stated as increases or decreases without any quantifiable target outcomes (i.e., miles of river with exclosures per year, xx more days of floodplain connectivity, etc.). Without clear measurable objectives that describe expected ecological outcomes, it is unclear what level of increase or decrease would be viewed as successful and the basis for continued implementation.

For example, if restoration of riparian vegetation is one of the primary goals of the overall project, the SMART objective should identify the expected quantitative outcomes and timeframes for their attainment. An example of a SMART objective for the overall outcomes for riparian revegetation might be:

“Increase length of riparian vegetation protected by stream fencing by XX miles from 2022 to 2026, which would restore XX% of the potential riparian protection that has been identified as needed in the subbasin and increase the total miles of riparian vegetation protected by this project since 1984 by XX%.”

If the proponents have determined the proportion of the riparian area with either woody riparian vegetation, grasses, or wetland vegetation and has estimated the extent and rate of recovery anticipated, they could develop even more informative SMART objectives based on the intended outcome for the vegetation and the timeframe over which that would be achieved.

The proponent listed the potential projects for 2022 to 2026. For some of the projects, they indicated the planned number of riparian plants, miles of riparian fence, and number of BDAs. This information would serve as a start for developing overall SMART implementation objectives. The objectives could state the amount of habitat restoration planned to be implemented. This information could be related directly to overall assessments of habitat and limiting factors in the basin, such as the Atlas and the FIP Strategy, and the proponents could express the actions as a proportion of the total identified need for restoration in the subbasin.

We encourage the proponents to contact us if they have additional questions or issues they would like to discuss.

2. Overall plan. The proponent provides a list of roughly 25 projects they anticipate will be implemented from 2022-2027 and briefly indicates what will be implemented (e.g., miles of fencing, numbers of trees planted, numbers of BDAs). The response states that the projects for each year are selected by a core group of ODFW staff primarily from the Fish Habitat Program and the District and Assistant Fish Biologists. It does not indicate whether these meetings are ad hoc meetings held whenever the project director feels they are needed or whether they are part of a regular systematic prioritization process. It also does not indicate whether the project coordinates with other cooperators in the John Day River subbasin, such as the John Day FIP.

3. Implementation monitoring. The proponents indicate that they plan to evaluate the Murderers Creek Habitat Improvement Project. The response also states that other projects would be monitored by the ODFW monitoring project, but it did not specify which projects. It would be useful to include this information in the next annual report.

The proponents’ plan to evaluate all BDA constructed projects based on photopoints and evaluation of fish passage and ecological structure and function. These will be assessed at low and high flow 1 to 2 years after implementation and again in 5, 10, and 15 years. The response does not indicate how fish passage or ecological function will be assessed. This should be described. The revision states that “adaptive management will occur when structure function or fish passage is not acceptable or has failed” but it does not describe what determines acceptability or failure and does not explain the adaptive management process. Addressing these elements will improve the overall quality of monitoring and adaptive management.

4. M&E matrix – support. The proponents assisted the John Day River Salmonid Monitoring to Inform Recovery Project (199801600) in developing the RM&E Matrix for the John Day area, which was a strong summary of the monitoring and evaluation strategy and addressed a number of questions for the ISRP. The ISRP appreciates the collective efforts to develop the RM&E Matrix and Summary for the John Day subbasin and finds it very useful for reviewing this project. After establishing targets for this project’s objectives, the proponents are encouraged to continue this collaboration with the lead project to ensure its objectives are being achieved.

5. Project evaluation and adjustment. The Response does not identify an explicit adaptive management approach that includes a regular process of project planning, assessment and evaluation, decision making, and a process for determining and recording adaptive actions. The response summarizes how the project uses “a core group of ODFW staff, primarily from the Fish Habitat Program and the District and Assistant Fish Biologists” to select projects. It appears that many or all of these projects are identified through the Atlas Scoping and Mapping Process of the John Day basin Partnership and ODFW is one of the partners. This process appears to provide some of the elements of an adaptive management approach, relying on the Atlas planning process and the Fish Research Program to assess benefits. However, the project does not describe a strategic evaluation process to understand when and how the objectives and activities of the overall project need to be revisited and/or modified.

6. Synthesis. In our Response Request, we asked the proponent to describe the overall contribution to riparian habitats in the John Day River basin and the degree to which the project has influenced limiting factors within the basin. We also asked for information on any documentation of this in previous publications, technical reports, or annual reports.

In the response, the proponents use information on steelhead habitat and steelhead densities to determine the proportion of the steelhead population that exists within stream reaches that have been fenced by this project in the five major subbasins of the John Day basin. They report that 5 to 17% of the steelhead in the John Day basin potentially benefit from their fencing projects. They also describe habitat benefits that have been documented for fencing projects in Thirtymile Creek. We commend the proponents for bringing in these types of data and analyses to document benefits of the habitat projects and encourage them to build on the analyses of benefits they reported in their 2008 Comprehensive Project Review.

The ISRP requested information on the overall contribution of the project and the degree to which the project has influenced limiting factors. The proponents produced a previous Synthesis Report in 2008 (Comprehensive Project Review [1984-2007], Mainstem, Middle Fork, John Day Rivers Fish Habitat Enhancement Project), but this report was not mentioned in the proponents’ response. The ISRP review of the Comprehensive Project Review noted that the project has many strengths, but found that, while the restoration practices were recognized as scientifically justifiable practices, the project did not demonstrate the degree to which the project met objectives of the subbasin plan or provided meaningful benefits to fish and wildlife.

After 37 years of restoration implementation, the proponent should update the 2008 synthesis report and provide the information requested in the previous reviews. The assessment provided in the project’s response should be expanded to all relevant species and directly address known limiting factors.

Preliminary ISRP report comments: response requested

Response request comment:

Over the last 36 years, the John Day Habitat Enhancement project has focused primarily on protecting and restoring riparian and instream habitat through fencing and planting to contribute to the recovery of Mid-Columbia summer steelhead and spring Chinook salmon. The proponents have a long record of working with landowners and partners to protect riparian areas in the five major subbasins of the John Day River basin. This is a long running project with demonstrated success in working with private landowners to protect riparian habitat.

Several key elements of an effective project are not included in the proposal. The ISRP expected the proponents to have target accomplishments for each type of action; for instance, miles of new fencing, miles of fences repaired/maintained, number of off-channel improvements (e.g., beaver analog devices/sites), and these should be reflected in the SMART objectives. More complete detail on the project methods, including prioritization of subbasins, a data-informed process for identifying when passive restoration is acceptable, provisions for monitoring of project metrics that directly reflect the intended benefits (e.g., groundwater storage), and justification or elimination of streambank stabilization for protecting and enhancing anadromous habitat.

Analysis of existing data is needed to evaluate the efforts from the prior project phase have provided benefits or if alternate strategies should be employed. In addition, the proposal needs to describe a strategic monitoring approach that uses data from project partners and efficiently focuses limited monitoring resources on detecting intended benefits. Furthermore, the confounding factors section should explain how the project is addressing the identified factors in planning, monitoring, and evaluation. Finally, the proposal needs to describe how the proponents (1) evaluate individual actions and the project as a whole and (2) make adjustments when it is determined that project objectives are not being achieved.

The ISRP asks the proponents to provide information about the following in a response. The response can directly answer each topic requested or provide a revised proposal with a brief point-by-point response to the ISRP referencing where, and summarizing how, the issues were addressed in the revised proposal:

1. SMART objectives. Provide SMART objectives (see proposal instructions) for riparian habitat, channel conditions, water temperature, or fish populations and indicate how they were identified based on the Atlas Prioritization process. 
   
   
2. Overall plan. The proponents should describe the overall plan for the project for the full length of the next project period. Have specific projects been selected and scheduled for implementation? If so, what is the timeline for those project components? 
   
   
3. Implementation monitoring. What specific aspects of specific projects will be monitored, including implementation monitoring, photopoints, or collaborations with other monitoring projects? Have the methods for monitoring been documented? 
   
   
4. M&E matrix - support. The ISRP is requesting a response from the Escapement and Productivity of Spring Chinook and Steelhead project (199801600) to summarize linkages between implementation and all monitoring projects in the basin. We are asking your project to assist them in creating the summary by providing information to them about what is being monitored for your implementation project, as well as where, when, and by whom the monitoring occurs. A map or maps of locations of monitoring actions would be helpful in this regard. 
   
   
5. Project evaluation and adjustment. What is the structured and scheduled process used for adaptive management of the project? The description needs to include planning, prioritization, project selections, implementation, assessment, evaluation, decision making, and recording of decisions. 
   
   
6. Synthesis. After 37 years, what has been the overall contribution to riparian habitats in the John Day River basin and to overall watershed conditions (e.g., area or length protected, proportion of unshaded stream miles that have been revegetated)? To what degree has the project influenced limiting factors within the basin? Has this been documented in previous reports or Annual Reports? Note that this response should not require detailed monitoring data or complex landscape models. The Atlas process should provide the necessary context and data for such an analysis. This type of analysis also was recommended in the 2008 and 2014 ISRP reviews.

Q1: Clearly defined objectives and outcomes

The proposal identifies two major goals: restoring riparian zones with livestock exposures and restoring floodplain connectivity through constructed beaver dam analogs (BDA) and large wood. All objectives for these goals are implementation objectives related to planning, implementation metrics, inspections, and maintenance, and total amounts of fencing or structures installed.

No SMART objectives are developed for habitat conditions or biological responses. The objectives primarily are a list of tasks with limited detail about what is expected to be accomplished, even in terms of implementation. Without quantitative objectives, what are the criteria for evaluation of performance or project success? In addition, clearer justification is needed that the objectives provide the expected outcome of improving ecological function. The 2013 Geographic Review called for the project to develop quantitative objectives with time frames for expected responses, but the project has not responded to the recommendation.

Q2: Methods

The Methods section of the proposal largely describes the general processes used in site selection, design, cooperative agreements, and construction. The project has expanded its methods to include off-channel water developments (such as Beaver Analog Devices and other approaches) to improve local riparian conditions. The proponents also have switched to metal posts for fencing to improve longevity (fire resistant).

The Methods section does not provide detailed descriptions of the methods or citations to references where those details could be obtained. The Methods section presents a task list but does not provide content on how the tasks would be completed.

Several aspects of the methods require additional description in the proposal:

• Photo points are used to document temporal changes, but the photo frequency as well as the metrics quantified from the photos are not identified. No information is provided showing the utility of the photo points toward achieving fish and wildlife goals and objectives.
• The ISRP supports the project’s transition to focusing efforts within priority subbasins (page 13 of the proposal), but the proposal does not describe how prioritization occurs or which basins are the highest priority. We assume that it uses the John Day Atlas, but the proposal does not make that clear.
• The proposal identifies that riparian vegetation planting may be necessary for stream reaches that are not recovering passively. How do proponents determine that recovery is not adequate, and planting is needed? Given the lack of appreciable change demonstrated in the LiDAR imagery shown in the Appendix, a data-based process for evaluating recovery and identifying where more intensive planting is required seems critical.
• The metrics proposed for the LT-PBR sites (page 18 in the proposal) are not likely to document the benefits being sought, especially groundwater storage. Additional monitoring will be needed to demonstrate the benefit, particularly given the wide range of results reported in the literature on the effectiveness of BDAs in storing groundwater and recent publications on hydrologic limitations of wet meadow restoration of streamflow (Nash et al. 2018).
• The proposal mentions (page 18) implementing streambank stabilization projects where needed. This is not included in the goals or objectives, and it is not clear how this serves the priority species. It is not clear what habitat benefits stable streambanks provide. Is it intended to reduce sediment loads to the river? Much clearer justification for the habitat benefits of streambank stabilization, based on data and/or peer-reviewed literature, is needed or else this activity should be eliminated from the scope of work.
• The proposal contains a Gantt chart of general monthly activities for 2023 and 2024. The chart does not include specific projects or types of projects and does not include activities beyond years 2023 and 2024.

Q3: Provisions for M&E

The proponents note that BPA funds for monitoring are limited to 5% of total project cost and, as well, monitoring largely occurs through cooperation with the ODFW fish monitoring project (199801600). The ISRP recognizes that monitoring of effectiveness for an implementation project like this one will come from other projects or monitoring programs. Nevertheless, the types, locations, objectives, and outcomes of monitoring occurring for the John Day Habitat Enhancement project were not described in the proposal. Separately, the ISRP is asking the ODFW fish monitoring project to develop a matrix of monitoring activities and their specific connections to habitat and hatchery implementation projects in the John Day River basin. The ISRP is requesting that this habitat project should assist the lead project in developing this overview of implementation and monitoring in the basin.

The proposal points out that habitat projects such as BDAs also can be evaluated over time and indicated a few metrics for habitat and fish passage that can be used for BDA projects, but it did not indicate if those would actually be measured. While some content was provided regarding when BDAs are considered successful (“when stream flows are extended or become perennial, floodplains remain connected for longer durations, stream beds show evidence of sediment aggradation, and fish passage is maintained during 5% through 95% exceedance levels.”), there was no indication in the proposal that those things would be monitored, either by this project or cooperating projects. This is especially critical since the literature indicates the BDAs do not always produce the benefits expected by the proponents.

The proposal describes the proponents’ use of the Atlas Scoping and Mapping Process to prioritize projects at a landscape scale. The proposal indicates that ODFW will utilize this to the greatest extent possible. This seems to imply that there are factors that limit the use of the Atlas process, but it does not provide an explanation of what factors would limit its application. Clarification is needed about the prioritization process.

The project indicated in the 2013 Geographic Review that it is developing relationships between freshwater productivity and habitat variables at the watershed scale. This proposal provided no indication that such analysis is occurring. Both of the two previous ISRP reviews (2008 and 2014) raised concerns that habitat results were not adequately reported and that little monitoring and evaluation was occurring. That concern remains. In the past, the proponents have maintained that they are implementing actions recognized to be effective in the region and specific evaluation within the John Day River basin is not necessary. They describe monitoring conducted by others in Bridge Creek and Middle Fork John Day IMW. Independent of detailed monitoring at every project site, the proponents could develop an overall assessment of the potential biological benefits contributed by this project based on known relationships from other studies.

The proponents provide no explanation of a structured, systematic evaluation process, such as an adaptive management plan. The proposal clearly indicates that they collaborate with partners and participate in a number of planning groups, but they do not describe their own sequence of planning, design, implementation, evaluation, and decision-making. The proponents do not describe a scheduled process or regular meeting to evaluate outcomes, make decisions, and record the outcomes and decisions.

The confounding factors identified in the proposal are limited essentially to climate change or its influence on stream temperature. The proponents indicate that efforts to maintain riparian shade and floodplain connectivity would be beneficial in the face a warming trends within the region. The proposal mentions that warming trends exacerbate the effects of invasive smallmouth bass, which is a well-documented problem in the John Day River basin. It does not describe activities to coordinate project implementation with other actions to control nonnative predators, such as smallmouth bass. In the face of climate change and expansion of the distribution of smallmouth bass, it would be strategic for the project to directly consider (e.g., risk analysis) how expanding predation should impact the project prioritization and selection process. The proposal does not address other potential confounding factors in the John Day River basin, such as upslope conditions, irrigation and water withdrawal, human population growth, environmental contaminants, or other factors that may influence the success of their actions.

Q4: Results – benefits to fish and wildlife

Since it began in 1984, the project has protected 283 miles of streams and 10,154 acres of riparian habitat, installed 450 miles of fencing, completed 175 individual projects, planted over 74,000 riparian plants, and applied 3,745 pounds of native grass seed. Currently, 6,535 riparian acres are actively managed and maintained through the program. Since the last ISRP review, the proponents have constructed 141 miles of fencing with 52 individual landowners, installed 6 off-channel watering sites, constructed 39 instream habitat features, and constructed 23 BDA structures. Figure 2 in the proposal illustrates the number of miles protected by year. The graph indicates that the miles protected have decreased in recent years, but the apparent reduction is not discussed or explained. The proposal also describes changes in their approach to be more effective, which include obtaining longer cooperative riparian area agreements, increasing width of riparian exposures, and including instream habitat improvements.

The proposal describes the overall approach and how it is linked to documented limiting factors. It presents findings of past research on the effectiveness of livestock exclosures in both the John Day River basin and other areas of Northeast Oregon. Monitoring and evaluation by others show that a portion of the riparian fencing sites have provided some benefits, with the exception of temperature, which is not surprising given the limited spatial dimensions of the individual actions. While significant improvements in habitat conditions have been observed for livestock exclosures, most responses of fish populations have not been statistically significant. Reasons suggested to explain this include the limited size of exclosures, overall effects of reach-level and basin conditions water quality and habitat, maintenance of exclosures, and other factors. Additional analysis is needed to demonstrate where and what types of benefits for fish and wildlife are produced by the projects’ restoration actions. Given that some of the project’s restoration actions are monitored as part of ODFW monitoring Project 199801600, the proposal should provide information on the results of past monitoring to evaluate project benefits.

After 37 years, the project should be able to describe the overall contribution to riparian habitats in the John Day River basin and to overall watershed conditions (e.g., area or length protected, proportion of unshaded stream miles that have been revegetated, potential benefits to fish and wildlife). To what degree has the project influenced limiting factors within the basin? Addressing this concern does not require detailed monitoring data or complex landscape models. It could be based on simple relationships between riparian habitat conditions and abundance of fish and wildlife obtained from scientific literature. The Atlas process should provide the necessary context and data for such an analysis. It is a mature project in close collaboration with ODFW monitoring and assessment projects and, as such, a comprehensive analysis of the benefits of the project to fish and wildlife is well within the collective capacity of the proponents and their partners. This type of analysis also was recommended in the 2008 and 2014 ISRP reviews.

References

Nash, C.S., J.S. Selker, G.E. Grant, S.L. Lewis, and P. Noël. 2018. A physical framework for evaluating net effects of wet meadow restoration on late-summer streamflow. Ecohydrology. 11: e1953-. https://doi.org/10.1002/eco.1953.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The goal of this project is to enhance riparian and instream habitat through fencing, planting in riparian areas, non-native plant eradication, and improvements in passage. The work is intended to benefit recovery of ESA listed summer steelhead and spring Chinook salmon. The sponsors provided a thorough review of current knowledge of the effects of grazing on habitat and fish in the John Day basin. They also provided a program summary in response to the 2008 ISRP review which had a lengthy discussion of projects and accomplishments. The report was informative and provided evidence of ecological responses to riparian restoration.

The project appears to be well-coordinated with an array of other programs and projects in the John Day subbasin. It is consistent with the FCRPS BiOp, the Oregon Plan for Salmon and Watersheds, and the Grande Ronde Subbasin Plan.

The sponsors plan to develop relationships between a measure of freshwater productivity (the residuals of the relationship between smolts/redd or smolts/ number of redds) and habitat variables at the watershed scale to determine which variables are having the greatest and least impact on fish productivity. The sponsors assert that using residuals filters out the effects of density-dependence. The results of this analysis are intended to guide future habitat enhancement actions.

The sponsors are moving in the right direction in trying to sort out the effects of habitat variables and density-dependence on freshwater fish productivity. Using the residuals from the regression of smolts/redd (or spawner) against number of redds will remove the linear effect of number of redds. Another approach the sponsors should consider, which would provide more information, would be to use multiple regression with smolts /redd regressed against number of redds and habitat variables in the same model. The advantage in this approach is that it would determine how much correlation there is among the habitat variables and number of redds. Another consideration is that the sponsor’s analytical approach is not able to determine which life stage, for example egg-fry or fry-parr survival, is being benefitted most by habitat enhancement. They should consider using life stage specific measures of productivity and survival.

The goal of the project is clear. It is intended to improve ecological river function on private lands. The sponsors present two rather general objectives, the first of which largely restates the project goal in slightly more specific terms. The objectives are stated in qualitative terms and lack a time frame for expected response. These objectives should be re-stated in more quantitative terms.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

A large number of enhancement projects have been implemented since the inception of this project. The program has a long history and reports "restoring" nearly 200 miles of stream. A little surprising is the fact that, although response times of habitat to enhancement actions are slow (15-30 years), current agreements with landowners expire after 15 years. It would be beneficial for habitat recovery if there was a movement toward longer-term agreements with landowners, given the investments made and longer expected response times for habitat recovery. Additionally, there was no mention of how enhancement sites were selected, and if and how that process has been improved over time to be more strategic and effective. It would seem that with the large number of potential stream miles needing treatment (about 50% based on rough estimates provided by ODFW) and the limits to the actual miles that can be treated, a strategic approach for treatment area selection would be critical.

Although not quantitative, a series of more than 250 photo points for 42 projects is discussed and examples provided. These show good visual characterization of riparian response with a reported 87% positive response. There was no mention if there had been an effort to stratify sites by channel and valley type or even stream order to see if there might have been informative relationships. Additionally temperature, bank stability, neotropical bird counts, and channel transects have been used and, although quantitative results appear limited, those reported in the proposal indicate some promising trends. It appears that currently only channel transects and bank cover measurements are being used, but there is no discussion of why other channel morphology parameters were not reported.

In their latest review of this project (ISRP 2008-8) the ISRP expressed serious concern that habitat results were not adequately reported and wondered if an M&E plan was even in place. To a degree this concern still exists. Although some fish data were analyzed quantitatively at the watershed scale, presumably as part of the Middle Fork John Day River Intensively Monitored Watershed (MFJDR IMW) project, little analysis and interpretation of habitat data are presented, and no RM&E protocols are identified in the proposal.

The sponsors provide a rationale implying that there may be little need for them to conduct comprehensive site specific M&E in the future. Apparently the effectiveness of the suite of habitat enhancement projects implemented in the upper Middle Fork, including those executed by this project, will be inferred from work being conducted by the MFJDR IMW. The IMW will determine freshwater productivity (smolts/redd) at a location downstream from multiple sites where habitat enhancement actions have been implemented. The IMW project thus is intended to evaluate the cumulative effectiveness of all upstream habitat enhancement actions. This type of watershed or “population” scale evaluation is appropriate and could yield valuable information on freshwater productivity of focal species. It would have been useful if the sponsors stated what proportions of these sites were implemented by their project.

The sponsors contend that site specific M&E can be compromised by fish movement into the treated sites from areas outside them. Fish movement has been considered as a confounding factor in several studies evaluating fish response to habitat enhancement. Fish movement, however, could be viewed as a positive outcome of a restoration action. For example, if juvenile fish move into a newly created pool or into a fenced reach where habitat is recovering, they may have moved to these areas from less favorable habitat. The inference is that as a consequence of movement from less to more favorable habitat, fish survival and growth and subsequently freshwater productivity may improve. Movement, then, is a benefit to fitness and so can be viewed not as a confounding factor in evaluating effectiveness of a habitat project but as a positive outcome of the project.

In lieu of conducting site level effectiveness monitoring of habitat actions, the sponsors are relying on work being conducted by the Bridge Creek and Middle Fork John Day IMW to establish a “cause-effect relationship between habitat restoration and fish response” at the site or reach scale. Supposedly, the benefits for fish of particular kinds of site specific habitat enhancement actions, for example fencing and beaver dams, will be determined by this work. From the IMW results, the sponsors will infer effectiveness of their projects so alleviating their need to monitor each of them.

There is some justification for this approach as one of the purposes of ISEMP is to evaluate effectiveness of individual kinds of habitat restoration actions, recognizing that there is neither time, nor funds to evaluate individually the hundreds of habitat enhancement projects in the Columbia Basin. The sponsors, however, do not discuss whether ISEMP has reached the stage where their results can be applied to this project and when those results are expected. Nor is it clear that IMW results will be generalizable to all sites. There was no mention of future coordination or involvement with the Action Effectiveness Monitoring program. A base level implementation and compliance monitoring for each treatment type should be considered to augment IMW monitoring.

A variety of lessons learned and how they have changed project design and implementation is provided.

Evaluation of Results

This project began in 1984, and during this time has implemented an impressive number of habitat enhancement projects on private lands. The project appears to be well-managed and continues to produce expected products. It appears to enjoy good relationships with local landowners which should ensure its continued success.

Since its inception, the project has almost exclusively been dedicated to implementation of enhancement projects and has conducted only very limited effectiveness monitoring. Very little quantitative data pertaining to effectiveness monitoring was presented in the proposal and in its annual reports. Although M&E has been was mentioned in all annual reports, the only monitoring information that was reported consisted primarily of informative photo points and limited data on stream temperature and channel morphology at selected restoration sites. The sponsor’s rationale for lack of M&E is that watershed-scale effectiveness monitoring and site specific evaluation of particular kinds of enhancement actions is the purview of ISEMP’s IMW projects thus alleviating the need for M&E by this project. Whether M&E should be the responsibility of the IMW’s or the individual projects continues to be a concern for many habitat enhancement actions in the interior Columbia basin.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

Program activities appear to be well coordinated with tribal, state, federal agencies, and private landowners. Of particular importance is the apparent positive coordination with land management agencies that are responsible for about 1/3 of the upper watershed but a disproportionate share of higher quality habitat. This project closely coordinates with ISEMP and will rely on results of ISEMP’s work in Intensively Monitored Watersheds in the John Day to determine whether particular enhancement actions produce positive benefits for habitat and fish.

The sponsors provided a good review of possible climate change effects on fish in the Mid-Columbia. They maintain that the enhancement actions they are undertaking may help to detect and ameliorate these changes. There was a detailed discussion regarding the role of healthy riparian ecosystems as a hedge against climate change. There was no mention of improved connectivity as a similar important consideration. Additionally, there was no mention of issues like minimum stream flows or long term forest health, land use, and resource management changes that are, or could likely occur.

4. Deliverables, Work Elements, Metrics, and Methods

The Deliverables seem straightforward. There are no deliverables for M&E. Again, it appears that a very limited M&E program mostly involving photo points is in place.

Specific comments on protocols and methods described in MonitoringMethods.org

A variety of past monitoring activities were discussed and some results from them were discussed. Examples of periodic before and after photos for multiple years were provided. These examples showed clear visual tracking of vegetation response to habitat enhancement. It appears that monitoring activities have consisted of photo points and bank cover and channel transect monitoring at a few sites. There was no discussion of why these particular measures were selected. There should be more emphasis on monitoring to understand why some plantings and other activities work or do not work in particular situations.

===========QUALIFICATIONS FOLLOW================

The ISRP's concerns, questions, and comments can be dealt with in contracting, project document development, and future reviews.

The sponsors should clarify the role of ISEMP's IMW projects in M&E. If the sponsors are relying on the IMWs to provide site and reach scale evaluations of the effectiveness of various kinds of habitat actions, they should be certain that the IMW project has reached the point where its results can be extrapolated to the sponsor's project within the current project period and, if not, when this can be expected to occur

This project appears to be a solid program that is continuing important work. Some additional work to refine the program through strategic site selection, longer term agreements with landowners, and a more clearly stated plan for long term monitoring would improve effectiveness and efficiency and should be considered.

The sponsors should ensure that their project is closely coordinated with the Confederated Tribes of the Warms Springs' project, including identifying priority locations for restoration and division of activities. They should identify plans to cooperate with BPA's Action Effectiveness Monitoring program.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The goal of this project is to enhance riparian and instream habitat through fencing, planting in riparian areas, non-native plant eradication, and improvements in passage. The work is intended to benefit recovery of ESA listed summer steelhead and spring Chinook salmon. The sponsors provided a thorough review of current knowledge of the effects of grazing on habitat and fish in the John Day basin. They also provided a program summary in response to the 2008 ISRP review which had a lengthy discussion of projects and accomplishments. The report was informative and provided evidence of ecological responses to riparian restoration.

The project appears to be well-coordinated with an array of other programs and projects in the John Day subbasin. It is consistent with the FCRPS BiOp, the Oregon Plan for Salmon and Watersheds, and the Grande Ronde Subbasin Plan.

The sponsors plan to develop relationships between a measure of freshwater productivity (the residuals of the relationship between smolts/redd or smolts/ number of redds) and habitat variables at the watershed scale to determine which variables are having the greatest and least impact on fish productivity. The sponsors assert that using residuals filters out the effects of density-dependence. The results of this analysis are intended to guide future habitat enhancement actions.

The sponsors are moving in the right direction in trying to sort out the effects of habitat variables and density-dependence on freshwater fish productivity. Using the residuals from the regression of smolts/redd (or spawner) against number of redds will remove the linear effect of number of redds. Another approach the sponsors should consider, which would provide more information, would be to use multiple regression with smolts /redd regressed against number of redds and habitat variables in the same model. The advantage in this approach is that it would determine how much correlation there is among the habitat variables and number of redds. Another consideration is that the sponsor’s analytical approach is not able to determine which life stage, for example egg-fry or fry-parr survival, is being benefitted most by habitat enhancement. They should consider using life stage specific measures of productivity and survival.

The goal of the project is clear. It is intended to improve ecological river function on private lands. The sponsors present two rather general objectives, the first of which largely restates the project goal in slightly more specific terms. The objectives are stated in qualitative terms and lack a time frame for expected response. These objectives should be re-stated in more quantitative terms.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

A large number of enhancement projects have been implemented since the inception of this project. The program has a long history and reports "restoring" nearly 200 miles of stream. A little surprising is the fact that, although response times of habitat to enhancement actions are slow (15-30 years), current agreements with landowners expire after 15 years. It would be beneficial for habitat recovery if there was a movement toward longer-term agreements with landowners, given the investments made and longer expected response times for habitat recovery. Additionally, there was no mention of how enhancement sites were selected, and if and how that process has been improved over time to be more strategic and effective. It would seem that with the large number of potential stream miles needing treatment (about 50% based on rough estimates provided by ODFW) and the limits to the actual miles that can be treated, a strategic approach for treatment area selection would be critical.

Although not quantitative, a series of more than 250 photo points for 42 projects is discussed and examples provided. These show good visual characterization of riparian response with a reported 87% positive response. There was no mention if there had been an effort to stratify sites by channel and valley type or even stream order to see if there might have been informative relationships. Additionally temperature, bank stability, neotropical bird counts, and channel transects have been used and, although quantitative results appear limited, those reported in the proposal indicate some promising trends. It appears that currently only channel transects and bank cover measurements are being used, but there is no discussion of why other channel morphology parameters were not reported.

In their latest review of this project (ISRP 2008-8) the ISRP expressed serious concern that habitat results were not adequately reported and wondered if an M&E plan was even in place. To a degree this concern still exists. Although some fish data were analyzed quantitatively at the watershed scale, presumably as part of the Middle Fork John Day River Intensively Monitored Watershed (MFJDR IMW) project, little analysis and interpretation of habitat data are presented, and no RM&E protocols are identified in the proposal.

The sponsors provide a rationale implying that there may be little need for them to conduct comprehensive site specific M&E in the future. Apparently the effectiveness of the suite of habitat enhancement projects implemented in the upper Middle Fork, including those executed by this project, will be inferred from work being conducted by the MFJDR IMW. The IMW will determine freshwater productivity (smolts/redd) at a location downstream from multiple sites where habitat enhancement actions have been implemented. The IMW project thus is intended to evaluate the cumulative effectiveness of all upstream habitat enhancement actions. This type of watershed or “population” scale evaluation is appropriate and could yield valuable information on freshwater productivity of focal species. It would have been useful if the sponsors stated what proportions of these sites were implemented by their project.

The sponsors contend that site specific M&E can be compromised by fish movement into the treated sites from areas outside them. Fish movement has been considered as a confounding factor in several studies evaluating fish response to habitat enhancement. Fish movement, however, could be viewed as a positive outcome of a restoration action. For example, if juvenile fish move into a newly created pool or into a fenced reach where habitat is recovering, they may have moved to these areas from less favorable habitat. The inference is that as a consequence of movement from less to more favorable habitat, fish survival and growth and subsequently freshwater productivity may improve. Movement, then, is a benefit to fitness and so can be viewed not as a confounding factor in evaluating effectiveness of a habitat project but as a positive outcome of the project.

In lieu of conducting site level effectiveness monitoring of habitat actions, the sponsors are relying on work being conducted by the Bridge Creek and Middle Fork John Day IMW to establish a “cause-effect relationship between habitat restoration and fish response” at the site or reach scale. Supposedly, the benefits for fish of particular kinds of site specific habitat enhancement actions, for example fencing and beaver dams, will be determined by this work. From the IMW results, the sponsors will infer effectiveness of their projects so alleviating their need to monitor each of them.

There is some justification for this approach as one of the purposes of ISEMP is to evaluate effectiveness of individual kinds of habitat restoration actions, recognizing that there is neither time, nor funds to evaluate individually the hundreds of habitat enhancement projects in the Columbia Basin. The sponsors, however, do not discuss whether ISEMP has reached the stage where their results can be applied to this project and when those results are expected. Nor is it clear that IMW results will be generalizable to all sites. There was no mention of future coordination or involvement with the Action Effectiveness Monitoring program. A base level implementation and compliance monitoring for each treatment type should be considered to augment IMW monitoring.

A variety of lessons learned and how they have changed project design and implementation is provided.

Evaluation of Results

This project began in 1984, and during this time has implemented an impressive number of habitat enhancement projects on private lands. The project appears to be well-managed and continues to produce expected products. It appears to enjoy good relationships with local landowners which should ensure its continued success.

Since its inception, the project has almost exclusively been dedicated to implementation of enhancement projects and has conducted only very limited effectiveness monitoring. Very little quantitative data pertaining to effectiveness monitoring was presented in the proposal and in its annual reports. Although M&E has been was mentioned in all annual reports, the only monitoring information that was reported consisted primarily of informative photo points and limited data on stream temperature and channel morphology at selected restoration sites. The sponsor’s rationale for lack of M&E is that watershed-scale effectiveness monitoring and site specific evaluation of particular kinds of enhancement actions is the purview of ISEMP’s IMW projects thus alleviating the need for M&E by this project. Whether M&E should be the responsibility of the IMW’s or the individual projects continues to be a concern for many habitat enhancement actions in the interior Columbia basin.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

Program activities appear to be well coordinated with tribal, state, federal agencies, and private landowners. Of particular importance is the apparent positive coordination with land management agencies that are responsible for about 1/3 of the upper watershed but a disproportionate share of higher quality habitat. This project closely coordinates with ISEMP and will rely on results of ISEMP’s work in Intensively Monitored Watersheds in the John Day to determine whether particular enhancement actions produce positive benefits for habitat and fish.

The sponsors provided a good review of possible climate change effects on fish in the Mid-Columbia. They maintain that the enhancement actions they are undertaking may help to detect and ameliorate these changes. There was a detailed discussion regarding the role of healthy riparian ecosystems as a hedge against climate change. There was no mention of improved connectivity as a similar important consideration. Additionally, there was no mention of issues like minimum stream flows or long term forest health, land use, and resource management changes that are, or could likely occur.

4. Deliverables, Work Elements, Metrics, and Methods

The Deliverables seem straightforward. There are no deliverables for M&E. Again, it appears that a very limited M&E program mostly involving photo points is in place.

Specific comments on protocols and methods described in MonitoringMethods.org

A variety of past monitoring activities were discussed and some results from them were discussed. Examples of periodic before and after photos for multiple years were provided. These examples showed clear visual tracking of vegetation response to habitat enhancement. It appears that monitoring activities have consisted of photo points and bank cover and channel transect monitoring at a few sites. There was no discussion of why these particular measures were selected. There should be more emphasis on monitoring to understand why some plantings and other activities work or do not work in particular situations.

===========QUALIFICATIONS FOLLOW================

The ISRP's concerns, questions, and comments can be dealt with in contracting, project document development, and future reviews.

Modified by Dal Marsters on 6/11/2013 1:20:01 PM.

Three previous reviews have emphasized that future funding would be contingent upon providing analysis of project results based on quantitative monitoring of biological outcomes, specifically, habitat characteristics and presence of target species. The sponsor's response included excerpts from both a project-specific review from 1991 and a more general study from 2002, along with some example photopoint comparisons.

The sponsors have obviously conducted a locally popular program with results in re-vegetating of riparian corridors, as evidenced by the photopoint monitoring described in the response. After 22 years, the project should be showing changes in characteristics such as abundance of fishes, bank stability, and stream width-depth relationships. It is doubtful that before/after photopoint comparisons alone would be adequate for assessing some of the parameters listed in the proposal.

The 1991 and 2002 citations support continued fencing, but it is noted that sites studied by Kauffman et al. 2004, may not all be John Day sites and impacts on fish summarized from that paper are inconclusive. Citing preliminary analysis from project #199801600 might suggest that it would be wise to review project plans in terms of these more specific goals. Are current project proposals and priorities in line with these goals? Several project specific measurements are cited but not in the context of the watershed as a whole.

Another question is, overall, how much progress has been made toward project implementation goals? For example, what percentage of streambank miles needing rehabilitation have been rehabilitated to what extent? What changes have occurred in the watershed outside these projects that contribute to the cumulative effects of this project, both positive and negative? Project results must be assessed so that inferences can be drawn about changes observed in the John Day in the context of changes occurring in the larger region. Project 200301700, Integrated Status and Effectiveness Monitoring Program, includes a John Day pilot program that should be helpful in this, but is just getting organized. Close cooperation with the M&E project and sharing of results and experiences from this long-running project will maximize the benefits from both.

It is time for a comprehensive review of this project's biological results. One year of funding should provide time for this while continuing ongoing field projects. Future funding should be contingent on completion of a satisfactory document.