This detailed proposal provides a substantial amount of information. The proponents are to be congratulated on the time and effort that went into its development. It is clear that this is a well-organized project and one that has made substantial progress toward the landscape-scale restoration of reservation watersheds. The efforts are guided by an over-arching restoration strategy and prioritization matrix that provides strategic guidance for project development and planning. It is clear that project personnel are strongly committed to the continued effectiveness of their watershed and habitat protection and restoration work and community outreach efforts.

In the next annual report and future work plans, the proponents need to provide information to address the following Conditions:

1. SMART objectives. The proponents should provide SMART effectiveness objectives (see proposal instructions) for each of the major treatment types describing expected outcomes from project work, including public outreach. A primary goal of the project is to increase the amount of spawning and rearing habitat. A focus of the objectives needs to address metrics that are useful in gauging the amount and general quality of any of these added habitats. Outcomes for restoration treatments are likely best framed for a treatment reach or for a priority watershed. The effectiveness objectives need to meet SMART criteria (see proposal instructions). Some potential examples of outcomes for restoration work at a project or reach scale could include:

• • By_20xx__, ensure that stream surface shading is at least 60% and that the average number of days where summer maximum stream temperature exceeds 68 F is less than ____. (Protection of water temperature)
  • By_20xx__, increase the frequency of large primary pools to at least ____per mile. (Restore habitat complexity)

And for public outreach:

• • By_20xx__, increase annual participation in community outreach activities to at least ____people reached. (Community outreach)

Where treatments are designed to create thermal refuge and protect water temperatures, it would be helpful to identify the number of refuges to be established and the approximate river miles estimated to be affected.

The examples above can serve as a template for continued development of project scale, outcome objectives for all major work activities covered under the current proposal.

2. Linkages between projects. The proponents should provide a more detailed description of the linkage between this project and the sister projects. These projects are described as providing the “bulk of the field monitoring and data collection.” This information should include: A) identification of specific activities and timing for M&E of projects and treatment types, B) incorporation of project-scale outcome/effectiveness objectives, C) development of a general timeline and process for evaluation of data and summary of major findings/lessons learned, and D) incorporation of more detail on the gathering and evaluation of fish response data, including that for bull trout and lamprey, and its use in adjusting restoration treatments and locations.

While the proposal is well crafted and provides the information necessary to judge its scientific merit, the proposal is very long (41 pages without references or appendices), and future proposal could be reduced in length.

Q1: Clearly defined objectives and outcomes

The proposal provides a generally complete description of the range of problems that are to be addressed for whole watershed restoration. A good deal of survey and assessment data are provided, and numerous references are included to support identified problems, their source, and likely impacts on watershed conditions, habitat, and target fish species. From a review perspective, however, it would have been helpful to have the specific issues presented more succinctly (the recommendation is two pages or less). The rich information that the authors do provide, however, could form the basis of a review article, which would then only require a brief reference in the problem statement.

One area that should have additional discussion is the issue of toxic materials entering streams from adjacent roads and highways. Potential inputs from OR Highway-26 should receive additional discussion, especially given the recent research results on the effects of pollutants to fish and other aquatic life.

The proposal provides a good deal of documentation on the criteria that will be used to develop project-level implementation and effectiveness objectives. Although no discrete implementation objectives were provided for project implementation, maintenance, or monitoring/evaluation activities, presumably these are contained in detailed plans for project implementation. This was not made clear in the proposal.

To describe desired outcomes, five major limiting factors are identified, and seven primary "objectives" are identified. Other than the objective for sediment (achieve =20% sediment =6.4 mm in diameter in streambed substrate), the other four objectives are qualitative descriptions and have no time frames for expected achievement. For example, these include, “restore habitat complexity to historic, baseline conditions; Increase of thermal refugia and protection of water temperatures against global climate change; Restoration of watershed hydrologic function, including an increase of floodplain and wetland habitat areas; Protection of critical habitat and Conduct outreach with the Tribal community.” Where the need to increase thermal refugia and protect water temperatures is described, it would be helpful to identify the number of refuges to be established or perhaps the river miles to be affected as part of specific projects. This would serve as a benchmark against which progress could be measured. Related to this, a metric for measuring the restoration of watershed hydrologic function would be very useful.

The ISRP commends the project for including restoration activities that address upslope factors, particularly vegetation impacting summer stream flows and road conditions resulting in accelerated sediment delivery to streams. Community outreach activities are also included in the primary objectives. Unfortunately, the reviewers could find no examples of effectiveness objectives that met SMART criteria to describe desired outcomes for restoration projects, project maintenance, monitoring evaluation, or community outreach activities. It appears that most of the information and detail are in place to establish a core set of these objectives for the various project components. Examples of objectives for selected project activities are needed, especially those describing desired/expected outcomes and the associated time frames for accomplishment.

Q2: Methods

The proposal does a commendable job of fully describing methods for project planning, implementation and for various categories of monitoring. A good deal of data from a variety of past assessments are supported by maps and references to describe methods. These methods link directly to qualitative objectives that are not time bound. Linking methods to a core set of implementation and effectiveness objectives that meet SMART criteria will significantly improve the proposal.

The proposal provides a detailed description of a Warm Springs Reservation Restoration Strategy developed by the CTWSRO Fish Habitat Program. This strategy is designed to set the course for watershed scale protection and restoration for the next 20 years. Also use of a “prioritization matrix” is presented. This is used to help prioritize locations for protection and restoration activities.

For the three projects to be completed in the next five-year time period (Log Springs Meadow Restoration Project, Shitike Creek, and Warm Springs River, and Middle Beaver Creek), details of the projects could be included in a table similar to Table 5 and Table 8. This would be a good way to organize limiting factors to be addressed, species that will benefit, and proposed actions to be taken. When the projects are complete, tables could be updated to show progress.

Finally, a quite comprehensive description of three major confounding factors is provided. They include climate change and effects to stream temperatures and natural hydrologic processes; cumulative impacts of past and ongoing timber harvesting operations on Reservation sub-watersheds and streams; and the impact of stormwater runoff from highways and roads into stream channels influencing the survival of adult and juvenile salmonids.

Q3: Provisions for M&E

The proposal provides a good deal of general guidance information for project-level monitoring and evaluation activities. It is stated, “it is expected that the bulk of the field monitoring and data collection will be carried out by its sister projects,” and “The CTWSRO Fish Habitat program employs a range of M&E methods that are tailored to each individual project, depending on its objectives, design features, location, and the limiting factors being addressed.” There is limited discussion describing the actual linkage between the monitoring of sister projects and this proposal. The proposal provides a good amount of detail regarding approach and criteria for monitoring activities, but it does not provide any examples of an actual monitoring plan for a given project or set of activities. Although details of accomplishments are provided for past projects, discussion could be improved regarding how well the activities met intended outcomes or time frames for restoration response. Extensive use of a strong BACI approach to evaluate habitat and fish response to restoration efforts was described. Also, the proponent provided a good deal of information that addressed weaknesses identified in past ISRP reviews.

Since summer stream temperatures in many reservation streams are a key concern for the project, a general point to keep in mind is that modeled estimates of future water temperatures with changes in climate typically do not account for unique local characteristics or specific management actions intended to lead to reductions in water temperatures. These estimates are useful for guiding project planning, but documenting actual changes in water temperatures occurring with changing climate and effects of restoration actions will be critical for guiding efforts in the future.

The proposal provides a number of examples of lessons learned and describes how they have been used to adjust program activities and management. Additionally, there is some very complete documentation of project activities and accomplishments in annual reports. However, there is no direct description of an actual process that is used to periodically review and evaluate M&E findings, field observations, and work experiences, nor a description of how to link what is learned to adjust goals, activities and management decision making. This should be provided in future reports and proposals.

Q4: Results – benefits to fish and wildlife

The project provides a quite detailed description of both quantitative and qualitative results of past project work. There is a listing of past projects, accomplishments, and general responses for two of the streams with past restoration projects including fish population data from the AEM program. Summary tables are provided for each treatment type listing past project accomplishments. There is also a summary of six lessons learned, mostly focusing on restoration techniques. Additionally, recent annual reports provide additional detail for types of various project accomplishments (riparian protection, sediment reduction, community outreach). This information is provided in separate sections of the report. An overall summary describing major results would be particularly useful. Evidence included in the proposal indicates potential benefits to fish and wildlife, but a more explicit characterization would be helpful. Table 6 shows changes in steelhead, Chinook salmon, and coho salmon for one project in particular (Mill Creek at Potter’s Ponds), and that was encouraging, but no specific results were presented for bull trout or lamprey, which were also species of concern.

A major shortcoming is the lack of quantitative, time-bound (SMART) objectives describing desired outcomes for various project treatments and activity types. Although the proposal information makes it clear that a good deal has been accomplished, it is difficult to determine the degree to which the actual results to date match those that were originally planned.

Warm Springs River Wood Placement - Response Requested

The portion of this proposal package that deals with the plans for restoration of the Warm Springs River (WSR) was well done for many elements but incomplete for others. The process used to identify the project location was very complete. The method used to determine habitat limitations and design habitat actions to address these deficiencies also was very well done. However, the proposal does not include a description of work elements. Presumably, most of these would be associated with the implementation of the restoration design and establishment of the monitoring program. But they need to be included in the proposal to complete the review.

The ISRP understands that the Council recommended that RM&E needs for the Warm Springs River wood placement project be met through BPA's new Action Effectiveness Monitoring Plan (AEM). This AEM process is in its infancy. The ISRP recommends that it review the pilot study design once it is drafted. This applies to the full suite of Warm Springs’ projects that the ISRP has reviewed - Mill, Beaver, and the Large Woody Debris projects. It would be preferable to do this through the response loop time period, but if this is not feasible the ISRP will review the study design when a detailed draft is prepared.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

This proposal does a very good job of explaining the significance of this project to the regional effort to increase fish populations in this area of the Deschutes River watershed. The technical background for the project is complete. Information on current habitat conditions throughout the Warm Springs River watershed is provided. The discussion of wood delivery and routing and how this understanding was used to select sites and design of LWD treatments was very thorough. A discussion of WSR hydrology and incorporation into project design was included but there was minimal discussion of how long it will take for expected flows to scour the habitats that are anticipated to develop at project sites. Table 2 does a nice job of projecting expected habitat responses to treatment and focal species response to the new habitat. Also, it was mentioned that sediment is limiting factor but no discussion on dominant sources. If upland sources are dominant, additional information on priority locations and treatments is needed. If bank erosion is a major sediment source, the potential for LWD projects to accelerate local bank erosion should have been evaluated.

Increases in fine sediment and elevated water temperatures are both listed as limiting factors. There is no discussion of complementary treatments to LWD additions such as riparian reforestation and/or silvicultural treatments to increase stream shading and enhance long-term LWD recruitment and/or road decommissioning or improvement to reduce erosion and sediment delivery. Given that LWD recruitment is described as occurring locally through fire and windstorm disturbance events, it would seem that identification and treatment of riparian areas that are understocked with trees (future LWD) would be beneficial. Although this is not a requirement, it would be useful to help understand the entire suite of projects envisioned for restoring conditions in the Warm Springs River.

Information on spawner distribution and some data on juvenile salmonid abundance also are provided. These data are used to justify the priority reaches selected for restoration and to identify the appropriate restoration approaches and designs. Although there is no explicit statement of objectives in the proposal, the description of the current habitat conditions clearly indicates that the objective is to increase spawning and rearing habitat for salmonid fishes and lamprey in a reach of the Warm Springs River where these actions have the potential to have the greatest possible benefit. The appropriateness of this objective is well supported by the information provide in the proposal.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

This project is new and, therefore, there were no past accomplishments to include in the proposal. However, the process described in the proposal for the identification of the restoration reach and the development of a series of LWD structures to achieve different habitat objectives clearly indicates that this project already has been employing certain elements of the adaptive management process. Project site selection was based on a thorough assessment of current habitat conditions in the Warm Springs River augmented with information of fish distribution and abundance. These data, in conjunction with published information from unmanaged watersheds with climate and vegetation similar to the Warm Springs River, were used to identify the reach within the Warm Springs River where habitat was degraded but with a high potential for response by the focal species. Designs for wood structures were, in part, based on observations of the architecture of wood accumulations in the unmanaged, headwaters reach of the WSR. The project sponsors also sought design advice from BPA engineers. An additional resource could be restoration practitioners on the Mt. Hood and Deschutes National Forests. Both of these National Forests have a long history of LWD placement and monitoring of physical response. Specific locations for wood structures in the restoration reach were determined using a LiDAR-based DEM coupled with on-the-ground verification. Finally, an estimate of the potential gain in abundance of spawning steelhead and Chinook salmon based on the predicted increase in gravel availability and a prediction of increased juvenile parr density based on increases in pool area and cover were provided. These estimates could form the basis of a monitoring program to assess the effectiveness of this project (more on RME below). At each stage of the project development the sponsors used available information or collected new data to improve the design of the project.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The relationship between this project and the other ongoing habitat restoration and fish monitoring projects on the Warm Springs Reservation was not described. It seems likely that some of the fish data presented in the proposal were collected by the fish monitoring program on the reservation. If so, there is a link between this project and the monitoring program, and this linkage could be very productive in the development of a monitoring plan for this project. The habitat RME effort associated with this project is only briefly mentioned. This project will be one of the projects evaluated under the new Action Effectiveness Monitoring Program. The ISRP recently reviewed the proposal for this monitoring program, and this project would be very appropriate for inclusion, given the availability of existing data and an ongoing program collecting fish population data. However, it is not possible to judge the technical adequacy of the monitoring effort that will be associated with this project until the monitoring design and methods are developed. The ISRP should review a revised version of the proposal that includes the details of the AEM effort and clearly describes the linkages between this project-scale monitoring and the fish population monitoring that is already occurring.

The proposal did include a discussion of probable, local flow responses to climate change. Models developed for drainages with similar hydro-geomorphic characteristics were used to predict potential changes in flow with expected changes in precipitation patterns and temperature. However, it is not entirely clear how this analysis is being incorporated into conservation and restoration planning for the area. Climate change also may affect general forest health and increase the frequency of fires. Some consideration of this factor in project design also would be worthwhile.

4. Deliverables, Work Elements, Metrics, and Methods

The process used for designing the project was very complete, and the methods employed to collect the information used in the design phase were appropriate. The work elements to be completed as part of this project were not completely specified in the proposal. Presumably, most of the work elements would be related to the placement of the wood structures. Some information on the timing and logistical details of implementing the project should have been included in the proposal.

A description of methods that will be employed in executing the RM&E component of this project should be included in a revised proposal (or a link to a description of AEM project that will assess this project). This project will be a pilot for the new AEM program. The proposal indicates that habitat monitoring will be conducted in years 1, 3, 5, 7 and 10 after treatment. The most recent stream surveys were completed in 2000, but it is not clear if additional pre-project data will be gathered. In addition to the planned surveys, evaluations should be conducted after any flood events with a return interval equal to or greater than 5 to 10 years. The sponsors also may consider continuing habitat monitoring for longer than 10 years given dependence of the treatments on flows sufficient to scour stream bed materials. If no flows of sufficient magnitude occur within the planned 10-year monitoring period, the effectiveness of the treatments could not be completely assessed. There is no mention of fish population monitoring response to restoration treatments. If information useful for this purpose will be collected by other monitoring programs, a description of this activity should be included in the proposal.

Specific comments on protocols and methods described in MonitoringMethods.org

The proposal did not reference MonitoringMethods.org protocols.

===========QUALIFICATIONS FOLLOW================

Response to Qualifications on Beaver and Mill Creeks

The ISRP review of the Beaver and Mill creek restoration proposal, completed in December 2012, indicated that two areas of the plan still required some additional attention:

Warm Springs River Wood Placement - Response Requested

The portion of this proposal package that deals with the plans for restoration of the Warm Springs River (WSR) was well done for many elements but incomplete for others. The process used to identify the project location was very complete. The method used to determine habitat limitations and design habitat actions to address these deficiencies also was very well done. However, the proposal does not include a description of work elements. Presumably, most of these would be associated with the implementation of the restoration design and establishment of the monitoring program. But they need to be included in the proposal to complete the review.

The ISRP understands that the Council recommended that RM&E needs for the Warm Springs River wood placement project be met through BPA's new Action Effectiveness Monitoring Plan (AEM). This AEM process is in its infancy. The ISRP recommends that it review the pilot study design once it is drafted. This applies to the full suite of Warm Springs’ projects that the ISRP has reviewed - Mill, Beaver, and the Large Woody Debris projects. It would be preferable to do this through the response loop time period, but if this is not feasible the ISRP will review the study design when a detailed draft is prepared.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

This proposal does a very good job of explaining the significance of this project to the regional effort to increase fish populations in this area of the Deschutes River watershed. The technical background for the project is complete. Information on current habitat conditions throughout the Warm Springs River watershed is provided. The discussion of wood delivery and routing and how this understanding was used to select sites and design of LWD treatments was very thorough. A discussion of WSR hydrology and incorporation into project design was included but there was minimal discussion of how long it will take for expected flows to scour the habitats that are anticipated to develop at project sites. Table 2 does a nice job of projecting expected habitat responses to treatment and focal species response to the new habitat. Also, it was mentioned that sediment is limiting factor but no discussion on dominant sources. If upland sources are dominant, additional information on priority locations and treatments is needed. If bank erosion is a major sediment source, the potential for LWD projects to accelerate local bank erosion should have been evaluated.

Increases in fine sediment and elevated water temperatures are both listed as limiting factors. There is no discussion of complementary treatments to LWD additions such as riparian reforestation and/or silvicultural treatments to increase stream shading and enhance long-term LWD recruitment and/or road decommissioning or improvement to reduce erosion and sediment delivery. Given that LWD recruitment is described as occurring locally through fire and windstorm disturbance events, it would seem that identification and treatment of riparian areas that are understocked with trees (future LWD) would be beneficial. Although this is not a requirement, it would be useful to help understand the entire suite of projects envisioned for restoring conditions in the Warm Springs River.

Information on spawner distribution and some data on juvenile salmonid abundance also are provided. These data are used to justify the priority reaches selected for restoration and to identify the appropriate restoration approaches and designs. Although there is no explicit statement of objectives in the proposal, the description of the current habitat conditions clearly indicates that the objective is to increase spawning and rearing habitat for salmonid fishes and lamprey in a reach of the Warm Springs River where these actions have the potential to have the greatest possible benefit. The appropriateness of this objective is well supported by the information provide in the proposal.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

This project is new and, therefore, there were no past accomplishments to include in the proposal. However, the process described in the proposal for the identification of the restoration reach and the development of a series of LWD structures to achieve different habitat objectives clearly indicates that this project already has been employing certain elements of the adaptive management process. Project site selection was based on a thorough assessment of current habitat conditions in the Warm Springs River augmented with information of fish distribution and abundance. These data, in conjunction with published information from unmanaged watersheds with climate and vegetation similar to the Warm Springs River, were used to identify the reach within the Warm Springs River where habitat was degraded but with a high potential for response by the focal species. Designs for wood structures were, in part, based on observations of the architecture of wood accumulations in the unmanaged, headwaters reach of the WSR. The project sponsors also sought design advice from BPA engineers. An additional resource could be restoration practitioners on the Mt. Hood and Deschutes National Forests. Both of these National Forests have a long history of LWD placement and monitoring of physical response. Specific locations for wood structures in the restoration reach were determined using a LiDAR-based DEM coupled with on-the-ground verification. Finally, an estimate of the potential gain in abundance of spawning steelhead and Chinook salmon based on the predicted increase in gravel availability and a prediction of increased juvenile parr density based on increases in pool area and cover were provided. These estimates could form the basis of a monitoring program to assess the effectiveness of this project (more on RME below). At each stage of the project development the sponsors used available information or collected new data to improve the design of the project.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The relationship between this project and the other ongoing habitat restoration and fish monitoring projects on the Warm Springs Reservation was not described. It seems likely that some of the fish data presented in the proposal were collected by the fish monitoring program on the reservation. If so, there is a link between this project and the monitoring program, and this linkage could be very productive in the development of a monitoring plan for this project. The habitat RME effort associated with this project is only briefly mentioned. This project will be one of the projects evaluated under the new Action Effectiveness Monitoring Program. The ISRP recently reviewed the proposal for this monitoring program, and this project would be very appropriate for inclusion, given the availability of existing data and an ongoing program collecting fish population data. However, it is not possible to judge the technical adequacy of the monitoring effort that will be associated with this project until the monitoring design and methods are developed. The ISRP should review a revised version of the proposal that includes the details of the AEM effort and clearly describes the linkages between this project-scale monitoring and the fish population monitoring that is already occurring.

The proposal did include a discussion of probable, local flow responses to climate change. Models developed for drainages with similar hydro-geomorphic characteristics were used to predict potential changes in flow with expected changes in precipitation patterns and temperature. However, it is not entirely clear how this analysis is being incorporated into conservation and restoration planning for the area. Climate change also may affect general forest health and increase the frequency of fires. Some consideration of this factor in project design also would be worthwhile.

4. Deliverables, Work Elements, Metrics, and Methods

The process used for designing the project was very complete, and the methods employed to collect the information used in the design phase were appropriate. The work elements to be completed as part of this project were not completely specified in the proposal. Presumably, most of the work elements would be related to the placement of the wood structures. Some information on the timing and logistical details of implementing the project should have been included in the proposal.

A description of methods that will be employed in executing the RM&E component of this project should be included in a revised proposal (or a link to a description of AEM project that will assess this project). This project will be a pilot for the new AEM program. The proposal indicates that habitat monitoring will be conducted in years 1, 3, 5, 7 and 10 after treatment. The most recent stream surveys were completed in 2000, but it is not clear if additional pre-project data will be gathered. In addition to the planned surveys, evaluations should be conducted after any flood events with a return interval equal to or greater than 5 to 10 years. The sponsors also may consider continuing habitat monitoring for longer than 10 years given dependence of the treatments on flows sufficient to scour stream bed materials. If no flows of sufficient magnitude occur within the planned 10-year monitoring period, the effectiveness of the treatments could not be completely assessed. There is no mention of fish population monitoring response to restoration treatments. If information useful for this purpose will be collected by other monitoring programs, a description of this activity should be included in the proposal.

Specific comments on protocols and methods described in MonitoringMethods.org

The proposal did not reference MonitoringMethods.org protocols.

===========QUALIFICATIONS FOLLOW================

Response to Qualifications on Beaver and Mill Creeks

The ISRP review of the Beaver and Mill creek restoration proposal, completed in December 2012, indicated that two areas of the plan still required some additional attention:

The proposal was insufficient for review. The project description was overly general and lacked the specificity needed to provide the basis for a scientific evaluation of its merits. It should be re-drafted to provide this specificity. A revised proposal should build on habitat inventories and limiting factor analyses already completed for the Deschutes subbasin to demonstrate that proposed actions are appropriate and likely to succeed in the area of interest. Using a relatively new approach such as simulated beaver ponds holds promise, but such a restoration action should be set up as an experiment with suitable control sites for comparison