1. Objectives

The integrated report states that the single objective for both programs was to develop RME methods to assess status and trends, restoration opportunities, and the benefits of habitat restoration actions on salmon and steelhead and their habitat in the tributary environment. The larger spatial framework of these two programs is a critical context for evaluating whether they were scientifically sound. “The goal of ISEMP and CHaMP was to support the development of more realistic, spatially explicit, tributary habitat restoration scenarios and to quantify the degree to which these restoration actions might benefit salmonid populations in the interior Columbia River Basin." In addition, the integrated report explicitly states eight major questions the two programs were designed to answer. The objectives, methods, and analytical techniques are adequately described.

The two projects were terminated in late 2017 and early 2018.

2. Methods

The methods used in the ISEMP and CHaMP programs are thoroughly documented, and the ISRP evaluated the projects’ scientific soundness in multiple review processes from their beginning through 2013.¹ In addition, the ISAB and ISRP evaluated the projects’ contributions toward the Program’s critical uncertainties in 2015-2016 (ISAB/ISRP 2016-1). The ISRP found the projects’ methods generally to be scientifically sound and offered recommendations for improvements. Methods and results are also published in top peer-reviewed journals in fisheries, geomorphology, and ecology, and thus the methods underwent peer review by experts in those fields.

A review of the utility of CHaMP measurements and their protocols was contracted by BPA and completed by Wildlands Hydrology. Neither the ISRP nor ISAB have reviewed Wildland Hydrology’s assessment, which raised concerns, particularly about channel bedform classifications and identification of bankfull channel. In response to questions during their public presentation, leaders of Wildland Hydrology’s assessment team indicated that the CHaMP measurements of several of the major factors in habitat quality in the Columbia River Basin (e.g., riparian vegetation, large wood, temperature, and sediment particle size) are scientifically sound.

One of the fundamental characteristics of the ISEMP and CHaMP programs that differentiates them from AEM is the spatial and temporal scales for which they were designed. These two programs are designed to "assess status and trends, restoration opportunities, and the benefits of habitat restoration actions on salmon and steelhead and their habitat in the tributary environment." They are designed to identify these properties at the scale of entire watersheds or basins. AEM is designed primarily to assess the effectiveness of specific types of restoration actions at the site and project scale rather than entire watersheds or basins. Measurements in these different programs have the potential to be complementary rather than mutually exclusive if the necessary cross walks and data consistency for the individual site data are developed.

3. Results

The ISEMP/CHaMP project broke new ground at the cutting edge of (a) measuring and modeling stream habitat across a wide range of spatial scales, and (b) relating this to salmonid abundance and production. The integrated ISEMP/CHaMP report provided summaries of three intensively monitored watersheds, 10 studies of fish and habitat status and trends, 23 tools for restoration assessment and planning, 6 life-cycle models, and 4 data support projects. CHaMP sampled more than 800 sites in 8 years, which will provide critical information and a foundation for monitoring for years to come. Data/models and source code are readily available through cloud-based data warehouses and GitHub sites. Unlike most monitoring of local reaches, the data and analysis from ISEMP/CHaMP can be extrapolated from study reaches to the Basin. It is not an overstatement that many of the methods and results will be useful for decades to come among researchers and managers in the Columbia River Basin and worldwide.

Of the 46 subprojects summarized in the report, several have contributed substantially to our understanding of status and trends in major basins, design of future monitoring, and decision making in the Fish and Wildlife Program. We highlight several of these to illustrate the value of the results from these two programs:

• ISEMP/CHaMP developed methods to estimate juvenile salmon and steelhead density throughout a river network based on measures of the primary production of benthic algae, the ultimate source of food for invertebrates that feed fish communities. About half of the variation in fish abundance is explained by these algae growing on the streambed. In turn, gross primary production (GPP) can be predicted from models of stream temperature, conductivity, and solar energy, allowing researchers to calculate estimates of fish abundance from maps of these three physical variables.

• ISEMP/CHaMP adapted the Barker Model to analyzing mark-recapture data where fish are marked at one place and time, such as with PIT tags, but then captured (or detected by PIT-tag antennas) at distant locations over a wide range of times. This analytical technique is a key to measuring effects of habitat restoration on migratory fish like salmon and steelhead, which have smolts that emigrate. Without it, analysis of the abundance of fish with these complex life cycles is very difficult. The model will improve estimates of salmonid survival as they migrate through the river network and hydrosystem

• To relate fish abundance to habitat, Quantile Regression Forest models were developed. These models can address the thorny problem that habitat is not always the sole limiting factor for fish abundance and so data are often highly variable. This statistical method can deal with this problem.

• ISEMP/CHaMP created six life cycle models, which represent an extremely important tool for future analyses and decision making throughout the Basin. In recent reviews, the ISAB concluded these life cycle models provide holistic analyses of potential limiting factors over the full life cycle and geographic range of anadromous salmon and steelhead (ISAB 2017-1, ISAB 2018-1). These models will assist landscape-level assessment of status and trends by the Fish and Wildlife Program in the future and will greatly assist decision makers in the region.

• The Intensively Monitored Watersheds revealed several major findings, illustrating the value in this approach. The studies revealed both positive responses to restoration actions in one basin and no response in another basin. The balanced and unbiased analysis of the research data should be a model for other projects, because sometimes the desire or commitment to provide benefits to fish populations and habitats colors the interpretation of project results.

• The status and trends subprojects provided a scientifically strong foundation for analysis of future trends. As examples, the temperature subproject and the survival estimation subprojects identified critical issues about habitat conditions and population status while providing valuable analytical tools. The temperature estimation approach incorporated riparian condition, floodplain conditions, and discharge, which are significant predictors and elements of many restoration actions within the Basin. The visualization techniques for temperature will contribute to future temperature studies in the Northwest.

• ISEMP and CHaMP developed more than 23 restoration tools to assist restoration practitioners and assist in the analysis of ISEMP and CHaMP data. Several of these have been used by other projects and have substantially contributed to the synthesis of ISEMP and CHaMP data. Examples of major contributions are the Quantile Regression Forest models for fish habitat relationships, geomorphic network analysis tool box, riparian condition assessment tool, wood recruitment assessment tool, and river classification and geomorphic condition tool. These analytical tools will continue to assist and inform restoration planning and design.

Many of the ISEMP projects created GIS mapping tools to estimate variables needed to drive other models. For example, abundance and productivity of salmonids are different in valleys of different types (i.e., wide valleys with low gradient where the stream meanders vs. narrow and steep valleys where the stream is mainly cascades). To address this relationship in life-cycle models, they developed a tool to measure valley bottom width from digital maps using GIS.

A key finding from these projects is the need to design monitoring studies of restoration actions at appropriate scales of space and time for expected resource outcomes. Projects were able to detect local improvements but often failed to detect population-wide responses (e.g., Entiat IMW). Not unexpectedly, population metrics suffer from spatial mis-matches (e.g., population spatial scale is much larger than restoration spatial scale), from high variability in responses, from influences by many factors over and above restoration, and a longer-time lag for restoration actions to become effective. Detection of population scale responses to local restoration actions likely will need much longer time scales than are typically included in most monitoring programs.

The tests of the effects of habitat restoration carried out and published by the ISEMP/CHaMP program are among the best conducted to date, worldwide. An example is the research on the effects of Beaver-Dam analogs on steelhead in the Bridge Creek watershed of the John DayRiver (Bouwes et al. 2016²). As a result, this approach that requires less heavy equipment and site disturbance is being used increasingly in other programs throughout the Basin.

The program also broke new ground in determining how to measure habitat across a wide range of spatial scales from local sites to whole sub-watersheds, and where best to focus sampling efforts in such a daunting task. For example, a key result was that measuring more sites at intermediate intervals (e.g., 3 years) is more important than measuring fewer sites every year, because variation in habitat is larger across sites than among years within a given site.

Another key finding was that some of the most striking improvements in habitat and fish numbers have been achieved from lower cost projects that focus on natural-processes, such as small-scale structures and beaver input, not just from larger and more expensive projects.

Key project findings were shared widely in the refereed literature. The list of 53 peer-reviewed publications from the project, many of which appeared just in the last few years, is very impressive.

Lessons Learned and Areas for Improvement
The CHaMP protocol has been used by more than 10 state and Federal agencies and tribes to monitor fish habitat and restoration projects. Researchers and managers will continue to reap benefits from the tools developed and the papers published by the ISEMP/CHaMP program for decades to come. Despite the body of information developed by ISEMP and CHaMP, the lack of effective outreach and information sharing was a major weakness of the programs.

The programs did not provide frequent and timely syntheses of their measurements for use by regional projects and leaders of the Council and BPA. The programs were not successful in showing managers and policy makers why the results they were producing were useful. Examples of practical information for managers and decision makers in clear language free of jargon were needed. Instead, many of the reports were difficult to read and interpret, often because the writing was suitable for statisticians and GIS experts but not the intended audience. Most of the audience is lost by jargon, such as “site level sample inclusion probabilities,” “riparian vegetation departure products,” and “user-defined non-zero probability” in the Summary Report. Great results cannot be used if the audience who is supposed to use them cannot easily understand them.

One area that did not receive much attention in the project was the linkage of upslope/upstream processes and management to conditions in the downstream, valley bottom and fish bearing sections of watersheds. Although the need to provide tools or links for including these considerations was identified in past reviews, it does not appear to be included in the summary reports. Certainly, this would have added complexity to the work, but it is an important component for ultimately understanding, managing and restoring aquatic habitat and fish populations at watershed scale.

A final goal for ISEMP/CHaMP would be to show managers and decision makers results they can really use. The projects have ended, but NOAA Fisheries and collaborators are continuing to analyze, synthesize, and present their findings. It will be beneficial for the Fish and Wildlife Program to track and publicize when and where the findings and data are available into the future.

Design of Future Monitoring Programs
Although the projects are now closed, some observations looking back on the multi-year effort may be useful in designing and implementing future, large scale projects, particularly those that address similar research questions. The ISRP recommends that design of future monitoring should incorporate many of the 54 monitoring protocols and 800 CHaMP monitoring sites to take advantage of the 7-yr database for future trend analysis. The results of ISEMP and CHaMP can be used both for designing M&E projects and as prior long-term data for future monitoring sites. The investment in ISEMP and CHaMP has produced a substantial body of habitat data, methodological advances, analytical tools, and life cycle models, which provide a foundation on which to build future monitoring programs. This legacy should be carefully considered when developing the new tributary habitat RME strategy co-led by BPA, Council, and NOAA.

The analysis of measurement noise and overall variance for 54 different habitat measurements provides important insights for future monitoring. This analysis identifies the influence of different sources of variance for the habitat metrics used in the monitoring program. For example, most of the variance in estimates of coniferous riparian forest cover was related to either watershed or site components of variance, and variance in large wood fish cover was related primarily to site components. Interannual differences in their estimates were small, and they suggested greater information could be obtained by sampling a greater number of sites less frequently. They also found that crew-to-crew differences had little effect on the variance of their metrics. These results of the ISEMP/CHaMP programs should inform future design of monitoring programs in the Basin.

One of the most critical challenges for RME projects is providing information that can be directly applied to management issues throughout the life of the project, not just at the end. This requires agreement at the start of the project between the funders and the investigators about the degree of uncertainty that decision makers/management are willing to accept and the format for reporting. This also would require all users to understand that interim information be updated or revised in the future. More discussion and agreement on this issue between funders and project leaders at the outset of this long-term project would have been useful.

One of the innovative organizational aspects of the integrated final report is the concise and informative summaries of the individual sub-projects. The format provides a diagrammatic road map for the 46 sub-projects, graphical description of the relationships between the projects, concise nontechnical descriptions of the subprojects, concisely stated major findings and uses, contact information, and URL links to products, publications, web pages for tools or computer code, locations of data, and supporting files. The Council and BPA should consider using similar formats for other reports and products in the future.

4. 2017 Research Plan uncertainties validation

The summary report did not explicitly identify linkages to the 2017 Research Plan, but the 2017 Research Plan critical uncertainties database identifies direct or indirect linkages with 18 critical uncertainties for the ISEMP and CHaMP projects. The proponents did not directly crosswalk against the database. A quick review of the uncertainties database showed that most of them were covered by the projects except perhaps (a) resiliency to climate change, (b) additional habitat actions that are needed, and (c) estuary and plume actions and monitoring.

____________________________________

¹ ISRP 2013-2 includes a comprehensive history of ISRP and ISAB reviews of the projects beginning on page 5.

² Bouwes, N, N Weber, CE Jordan, WC Saunders, IA Tattam, C Volk, JM Wheaton and MM Pollock. 2016. Ecosystem experiment reveals benefits of natural and simulated beaver dams to a threatened population of steelhead (Oncorhynchus mykiss). Sci. Rep 6:28581. 10.1038/srep28581. http://www.ncbi.nlm.nih.gov/ pubmed/27373190. https://isemp.egnyte.com/dl/i3zz8KMhML.

 

Additional justification for the fast track elements (PIT tag arrays) is requested as a response during the fast track response loop. A written response to ISRP questions and presentation on the core ISEMP project is requested before or during the categorical RME review during the summer of 2010. This is an ambitious, broad-scale project that is producing useful information for managers on the status and trends of habitat and fish populations in the mid- and upper-Columbia. Additionally, the work is addressing general issues of basic importance, such as how many sites are needed in a watershed to track habitat improvements over time, and habitat restoration actions resulting in increased focal species populations. The ISRP recommends that the fast track components of this project be completed during this fast track review, but we find the overall proposal requires a comprehensive evaluation before or during the RM&E Categorical Review, after the ISRP has seen the larger RME Regional Plan. A response to justify the PIT tag arrays should provide evidence that they will provide data for estimating the intended response variables. PIT tag arrays may not provide the key response variable in habitat evaluations. Smolt recruitment per spawner as a function of the number of spawners is the key variable, pre and post, and compared to external controls. Will arrays provide this? If they do, will they provide this information with the accuracy required? How many sites are required? Are more needed? Does the data to date suggest the recruitment limitation is elsewhere? What does the data collected to date say about these questions? The comprehensive review and evaluation of ISEMP should include a written response and presentation to the ISRP. The written response and presentation should include analyses of data collected to date on status and trends of fish and habitat, and effectiveness of restoration actions in the subbasins they have been studying; the locations of the proposed PIT tag array should be more completely explained and justified. Other objectives, methods, procedures, and results to date need to be more completely clarified. In presenting these results they should demonstrate, as thoroughly as possible, how the monitoring methods and analyses they are employing are superior to other, more conventional methods. This will be especially important for the IMW sites. 1. Technical Justification, Program Significance and Consistency, and Project Relationships The project description adequately justifies the various elements of the work. A strong case is made for each of the efforts in the John Day, Salmon, Entiat, and Methow subbasins. Detailed descriptions of some of the approaches are repeated in Section F, making for a long proposal, but in general this project is well justified, important to the mid- and upper-Columbia, and well tied in (albeit complexly) with other restoration M&E efforts. The current proposal requests funding for installation of PIT-tag detection antenna arrays in Snake River tributaries and initiation of reach scale habitat monitoring in the Methow and Entiat River. The reach scale work is justified, but better justification for installation of the PIT-tag arrays is needed, as explained below. The proposal addresses several RPAs in the BiOp. ISEMP is linked to many projects in the Columbia River Basin. It makes use of an extensive amount of data collected by agencies and tribes and works in close cooperation with co-managers in the Basin. 2. Project History and Results Project results are summarized in a series of matrices for each fiscal year at each major watershed, including major accomplishments and links to progress reports. Although the ISRP prefers not to be referred to hyperlinked reports, the broad scope of this project and the large number of reports and papers justifies this approach. Most of the material is available on the ISEMP website. In general, the project has succeeded in achieving the majority of its objectives to date. The accomplishments listed in the proposal pertain primarily to monitoring protocol development and testing as well as other work related to ISEMP’s goals. A summary should have been included. A significant amount of data has been collected on status and trends of fish and habitat, and effectiveness of restoration actions in the course of protocol development and testing. Analysis of these data would be interesting and informative, and should be presented. One test of the effectiveness of ISEMP is whether its methods and means of data analysis prove superior to current methods for status and trend monitoring employed in the Basin with the understanding that there is overlap between the two. In addition to presenting results and analysis of data collected to this point, the proponents should compare their results to those obtained by other, more traditional, methods. For example, are ISEMP's escapement estimates for a particular river, for example using PIT-tag detector arrays, significantly more accurate than estimates obtained from weir or redd count data in the same river? 3. Objectives, Work Elements, and Methods With regard to the expanded sites in the Entiat and Methow subbasins, the GRTS sites have not yet been selected. The approach to site selection is a proven one and should be successful; however, the proposal does not state what will happen if there are landowner access issues. The ISRP will be interested in the interpretation of the macroinvertebrate sampling results, as macroinvertebrates tend to be quite variable and difficult to relate to experimental restoration. Many of the field techniques are described in various ISEMP sampling protocol reports and were not repeated in the proposal. The first objective pertains to installation of PIT-tag detector arrays in several tributaries of the Snake River. These arrays are in addition to numerous other arrays already present in Snake River tributaries. The proponents present a strong argument that arrays have many advantages in terms of accuracy and precision relative to more conventional techniques for assessing escapement and other adult parameters, and can be used to test many hypotheses and assumptions of interest to co-managers. The question is whether the proposed set of arrays is necessary and that depends on the purpose of the installations. If the purpose is to further test arrays and improve methods for analysis and dissemination of array data, why aren't the currently operational arrays sufficient to accomplish these tasks? If, on the other hand, the purpose is to install arrays just to monitor MPGs or some other specific management function, then this should be better justified in relation to stated ISEMP goals which appear to be directed primarily toward establishing and testing monitoring methodologies. Although this is a fast-track proposal, it seems unlikely that the arrays will be in place and operational to monitor the 2010 run. The second objective pertains to initiation of habitat status and trend monitoring in the Methow River and habitat and fish population status and trend monitoring in the Entiat River. The work in the Entiat would extend effort already ongoing in the IMW. The Methow work is well justified. It will determine whether the methods and metrics developed in the Entiat and Wenatchee Rivers are transferable to other rivers in the region. The effort to examine transferability is certainly worthwhile. 4. M&E Because this is entirely an M&E project, the objectives, methods, and analytical techniques were adequately described in Section F, previously discussed. In Table 1 on page 11, A core list of physical/environmental indicator variables to be monitored within subbasins in the Upper Columbia Basin is presented. There is a footnote indicating under water quality that other indicators can be measured, e.g., various metals and pollutants, herbicides and pesticides. It is gratifying to see this point made, but we wonder how and who makes these decisions and what are they based on? How were such indicators used in the past with respect to crop lands (certain pesticides, depending upon a particular crop), wastewater treatment plants (flame retardants, pharmaceuticals and personal care products), mining activity (selected heavy metals), or urban areas with low flow streams? Contaminants should be treated as a "wild card" that can confuse any salmon-habitat relationships, even on a large scale, including smolt survival many miles from the contaminant source. This overall project and the second objective in this proposal (reach scale habitat monitoring in the Entiat and Methow) should directly contribute to improvements in protocols and methods of data analysis and dissemination for status and trend monitoring in the Columbia River Basin. Without additional justification, however, it was difficult to envision how the proposed additional sets of PIT-tag arrays would further the specific objectives of the ISEMP program at a broader scale.

The Columbia Habitat Monitoring Program (CHaMP) monitoring program described in this proposal is very comprehensive and ambitious. The fact that this effort builds on the success of the ISEMP project provides increased confidence that the program can succeed. As the program is implemented, participants should consider developing formal mechanisms for communicating results to mangers. The ISEMP project holds a great deal of promise for answering the questions: “What is the current status of fish habitat in the Columbia River Basin?” and “Are restoration actions currently being undertaken in the Columbia River Basin having the desired effects on both habitat condition and biological response?” We were pleased to see that the current proposal includes a number of new watersheds that will expand the geographical scope and completeness of ISEMP. We were also pleased that the strong emphasis in standardized data collection and spatially balanced and randomized sampling is intended to bring more consistency to monitoring efforts in the Columbia River Basin. In general, the proposal meets scientific criteria, with one qualification. The ISRP also offers additional suggestions for project proponents to consider.

Qualification: The ISRP recommends that ISEMP organize a one-day workshop to discuss the CHaMP approach with the ISRP/ISAB and others. A draft of CHaMP should be circulated to the ISRP/ISAB before the workshop. Specific issues at the workshop should include how previously collected data can be or has been incorporated into CHaMP databases. It would also be useful to summarize how ISEMP priorities have evolved over the years, as well as a publication strategy. This qualification was discussed with the ISEMP project lead on September 13 and was determined to be agreeable. The workshop will be tentatively scheduled for late 2010 or early 2011.

1. Purpose, Significance to Regional Programs, Technical Background, and Objectives

In the Columbia River Basin, there are a wide variety of RM&E projects that often differ in sampling design, methods, metrics, and quality and extent of data analysis, and often do not lead to definitive evaluations of habitat restoration effectiveness. ISEMP is a far reaching project that is based on the reasonable proposition that a standardized set of protocols, procedures, and data collection and analytical methods that can be adopted basinwide is needed to improve data collection precision and accuracy, provide comparability of results within and between subbasins, and so improve the capability of determining the effectiveness of habitat restoration projects. This proposition has been tested in a limited number of watersheds. In this proposal ISEMP is expanding its scope to include several additional watersheds to further evaluate its protocols for study design, data collection, analytical methods, and information dissemination. This effort is worthwhile in that improvements in habitat effectiveness evaluation are badly needed within the Basin.

The technical background is very complete in the proposal. This project is clearly significant to regional restoration programs. A coordinated program for the collection, compilation, and archiving of data on fish populations and habitat condition has been identified in numerous ISRP and ISAB documents as a critical need for the Columbia Basin. Explanation of the significance of CHaMP to regional programs was very thorough. One question had to do with data archiving and database sharing. What will the interface between ISEMP databases and BPA’s Taurus project tracking system be? Will CHaMP data be linked to Taurus in such a way that interested parties can access habitat or population status and trend data (e.g., the CHaMP metadata library) directly, or will these databases be housed separately by ISEMP outside Taurus?

It was gratifying to see that CHaMP will be testing novel remote sensing techniques for assessing habitat condition over large areas, e.g., using green LiDAR or multispectral sensing. This project has also demonstrated leadership in tracking population and life history attributes using PIT-tag arrays and other landscape-based methods.

2. History: Accomplishments, Results, and Adaptive Management

The links provided to reports produced through ISEMP, in the aggregate, provide a very complete picture of the results generated by this project and reflects well on the prospects for success of the CHaMP program. The manner in which the CHaMP data will be used adaptively to modify future monitoring efforts is clearly described, and some examples are provided. However, the link with managers and policy decision makers in the basin is less clearly described. The proposal indicates that a process will be established specifically to utilize the data generated through CHaMP to produce new analysis tools, which will be used to generate the type of information required to determine future direction of restoration efforts and to support fisheries management decisions. But there is another step required to make this process maximally effective; a formal process for communicating the output from the data and analytical tools to non-technical audiences. ISEMP has used periodic newsletters as one mechanism for addressing this function. This approach also would be a reasonable option to consider for CHaMP. But the CHaMP project leaders should devote some effort to developing a consistent process for broadly disseminating program results.

The ISEMP project has expanded in scope perhaps more than any other habitat restoration-related project funded by BPA since its inception. There are now, according to the proposal, ISEMP studies taking place in 26 watersheds in the Columbia River Basin, all of which contain anadromous salmonids. That organizers have succeeded in growing this project in such an impressive fashion reflects well on the willingness of a wide variety of stakeholders (federal and state agencies, tribes, local conservation districts) to enter into cooperative arrangements with the ISEMP project to address large scale restoration status and effectiveness questions. ISEMP has grown to such an extent that many of its component parts could be treated as separate projects.

It was interesting to see how the proposal described past results. There were abundant maps and lists of activities taking place in ISEMP watersheds, but there were relatively few graphs or tables showing how target species have responded to habitat restoration. We were hoping for a little more in the way of biological response findings, since some restoration locations have now been monitored by ISEMP for seven years. The proposal suggests that the results of habitat restoration often require extended monitoring periods (i.e., often decades) in order for their effects to be assessed. We concur, but including a few highlights of some of the most informative results to date would have made the proposal more interesting.

The proposal does an excellent job of describing the formation and evolution of the CHaMP effort, which is in effect an important type of adaptive management, i.e., the development of standardized habitat survey protocols in order to facilitate data analyses and inter-watershed comparisons. Overall, the description of other activities was thorough and informative. One adaptive management question is: have any restoration actions changed as a result of ISEMP findings? In particular, we are interested in knowing if anything is being done differently because evidence is starting to suggest that current approaches are not working as anticipated. Perhaps, as the proposal points out, it is premature to make judgments but if there are any good examples of restoration practitioners learning from past mistakes, they would be worth knowing.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging)

Generally, the relationships of this program to other habitat and fish monitoring projects in the Columbia Basin are well described in the proposal. The one relationship that could have benefited from a more thorough discussion was the association between CHaMP and PNAMP. Several of the deliverables in the proposal will be co-developed with PNAMP. But the role of each organization in producing these deliverables was not clearly described. A paragraph in the introduction that outlines this relationship and some indication under the shared deliverables of roles and responsibilities would have helped to clarify the division of labor.

The monitoring work includes tagging, other types of fish population assessment, experimental habitat restoration, long-term habitat trend monitoring, habitat protocol standardization, food web studies, and management of very large datasets. The project does not emphasize identifying limiting factors; rather, it is aimed more toward evaluating the responses of aquatic habitat and fish populations to restoration actions at large spatial scales. Overall, the proposal does a good job of relating ISEMP-sponsored monitoring to other restoration and monitoring projects. The restoration questions being addressed in each of the watersheds are appropriate to the issues believed to be limiting to salmon production. The new CHaMP rotating panel (GRTS) monitoring design appears to hold considerable promise in characterizing habitat status and trends. The project is consistent with the call for expanded RM&E in many subbasin plans and regional programs. This work is relevant to most RM&E efforts basinwide and provides a means by which RM&E programs in diverse subbasins can be unified under a common set of protocols and procedures.

The ISRP suggests that pollutants not be overlooked as potentially limiting factors in certain locations.

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables, work elements, metrics and methods are adequately described. Some of the details specific to different work elements could be explained more completely (e.g., in what types of habitat will benthic macroinvertebrates be sampled, and why?), but given the very broad scope of the project and the need to cover all the work elements this is understandable.

Journal publications are listed in many of the deliverables for different objectives, but to date there have not been very many papers published from the ISEMP work. Hopefully this will change in the near future.

What was the rationale for including 25 sample sites in each basin for habitat status and trend monitoring? Did the ISEMP data suggest that this number of sites would provide adequate statistical power? Some support for this number of sites should be provided. It would seem that the number of sample sites required to adequately represent the range of channel types within a basin would vary based on watershed size, variation in topography, geology, land use and other factors. If this is the case, a variable sample size might be more appropriate.

The assessment of stable isotope analysis to characterize trophic aspects of habitat condition in the John Day River is a novel element of this habitat monitoring program and could lead to the development of a very valuable tool. Limiting factors have almost exclusively been restricted to physical habitat or water quality attributes, largely because there was no efficient method for assessing food web conditions. One suggestion about the proposed methodology for this work element: the terrestrial invertebrates should not be lumped together for stable isotope analysis. As with the aquatic invertebrates, the terrestrial insects should be grouped on the basis of functional group (detritivores, herbivores, or predators).

The deliverable that addresses status and trend monitoring in the Wenatchee actually discusses this activity in the Entiat. In fact, the information for the Entiat is repeated in the second deliverable, which actually does address the Entiat. This discrepancy seems like an inadvertent cut-and-paste error, but the Wenatchee information should be added to the first deliverable.

ISEMP suggests, where appropriate, an “experimental approach” where habitat restoration actions in “treatment” streams are compared to reference “untreated” streams. Given the physiographic and biological variability within reaches or tributaries within a subbasin, let alone differences between subbasins, selection of appropriate references and treatments could prove challenging. It probably would be helpful if ISEMP provided guidelines and/or assistance to subbasin investigators for selecting both reference and treatment sites. The same could be said for data analyses. With the large amount of data that will be collected, investigators may need some assistance in data analyses. It is our understanding that ISEMP is planning to provide analytical assistance where needed.

The Columbia Habitat Monitoring Program (CHaMP) monitoring program described in this proposal is very comprehensive and ambitious. The fact that this effort builds on the success of the ISEMP project provides increased confidence that the program can succeed. As the program is implemented, participants should consider developing formal mechanisms for communicating results to mangers. The ISEMP project holds a great deal of promise for answering the questions: “What is the current status of fish habitat in the Columbia River Basin?” and “Are restoration actions currently being undertaken in the Columbia River Basin having the desired effects on both habitat condition and biological response?” We were pleased to see that the current proposal includes a number of new watersheds that will expand the geographical scope and completeness of ISEMP. We were also pleased that the strong emphasis in standardized data collection and spatially balanced and randomized sampling is intended to bring more consistency to monitoring efforts in the Columbia River Basin. In general, the proposal meets scientific criteria, with one qualification. The ISRP also offers additional suggestions for project proponents to consider. Qualification: The ISRP recommends that ISEMP organize a one-day workshop to discuss the CHaMP approach with the ISRP/ISAB and others. A draft of CHaMP should be circulated to the ISRP/ISAB before the workshop. Specific issues at the workshop should include how previously collected data can be or has been incorporated into CHaMP databases. It would also be useful to summarize how ISEMP priorities have evolved over the years, as well as a publication strategy. This qualification was discussed with the ISEMP project lead on September 13 and was determined to be agreeable. The workshop will be tentatively scheduled for late 2010 or early 2011. 1. Purpose, Significance to Regional Programs, Technical Background, and Objectives In the Columbia River Basin, there are a wide variety of RM&E projects that often differ in sampling design, methods, metrics, and quality and extent of data analysis, and often do not lead to definitive evaluations of habitat restoration effectiveness. ISEMP is a far reaching project that is based on the reasonable proposition that a standardized set of protocols, procedures, and data collection and analytical methods that can be adopted basinwide is needed to improve data collection precision and accuracy, provide comparability of results within and between subbasins, and so improve the capability of determining the effectiveness of habitat restoration projects. This proposition has been tested in a limited number of watersheds. In this proposal ISEMP is expanding its scope to include several additional watersheds to further evaluate its protocols for study design, data collection, analytical methods, and information dissemination. This effort is worthwhile in that improvements in habitat effectiveness evaluation are badly needed within the Basin. The technical background is very complete in the proposal. This project is clearly significant to regional restoration programs. A coordinated program for the collection, compilation, and archiving of data on fish populations and habitat condition has been identified in numerous ISRP and ISAB documents as a critical need for the Columbia Basin. Explanation of the significance of CHaMP to regional programs was very thorough. One question had to do with data archiving and database sharing. What will the interface between ISEMP databases and BPA’s Taurus project tracking system be? Will CHaMP data be linked to Taurus in such a way that interested parties can access habitat or population status and trend data (e.g., the CHaMP metadata library) directly, or will these databases be housed separately by ISEMP outside Taurus? It was gratifying to see that CHaMP will be testing novel remote sensing techniques for assessing habitat condition over large areas, e.g., using green LiDAR or multispectral sensing. This project has also demonstrated leadership in tracking population and life history attributes using PIT-tag arrays and other landscape-based methods. 2. History: Accomplishments, Results, and Adaptive Management The links provided to reports produced through ISEMP, in the aggregate, provide a very complete picture of the results generated by this project and reflects well on the prospects for success of the CHaMP program. The manner in which the CHaMP data will be used adaptively to modify future monitoring efforts is clearly described, and some examples are provided. However, the link with managers and policy decision makers in the basin is less clearly described. The proposal indicates that a process will be established specifically to utilize the data generated through CHaMP to produce new analysis tools, which will be used to generate the type of information required to determine future direction of restoration efforts and to support fisheries management decisions. But there is another step required to make this process maximally effective; a formal process for communicating the output from the data and analytical tools to non-technical audiences. ISEMP has used periodic newsletters as one mechanism for addressing this function. This approach also would be a reasonable option to consider for CHaMP. But the CHaMP project leaders should devote some effort to developing a consistent process for broadly disseminating program results. The ISEMP project has expanded in scope perhaps more than any other habitat restoration-related project funded by BPA since its inception. There are now, according to the proposal, ISEMP studies taking place in 26 watersheds in the Columbia River Basin, all of which contain anadromous salmonids. That organizers have succeeded in growing this project in such an impressive fashion reflects well on the willingness of a wide variety of stakeholders (federal and state agencies, tribes, local conservation districts) to enter into cooperative arrangements with the ISEMP project to address large scale restoration status and effectiveness questions. ISEMP has grown to such an extent that many of its component parts could be treated as separate projects. It was interesting to see how the proposal described past results. There were abundant maps and lists of activities taking place in ISEMP watersheds, but there were relatively few graphs or tables showing how target species have responded to habitat restoration. We were hoping for a little more in the way of biological response findings, since some restoration locations have now been monitored by ISEMP for seven years. The proposal suggests that the results of habitat restoration often require extended monitoring periods (i.e., often decades) in order for their effects to be assessed. We concur, but including a few highlights of some of the most informative results to date would have made the proposal more interesting. The proposal does an excellent job of describing the formation and evolution of the CHaMP effort, which is in effect an important type of adaptive management, i.e., the development of standardized habitat survey protocols in order to facilitate data analyses and inter-watershed comparisons. Overall, the description of other activities was thorough and informative. One adaptive management question is: have any restoration actions changed as a result of ISEMP findings? In particular, we are interested in knowing if anything is being done differently because evidence is starting to suggest that current approaches are not working as anticipated. Perhaps, as the proposal points out, it is premature to make judgments but if there are any good examples of restoration practitioners learning from past mistakes, they would be worth knowing. 3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging) Generally, the relationships of this program to other habitat and fish monitoring projects in the Columbia Basin are well described in the proposal. The one relationship that could have benefited from a more thorough discussion was the association between CHaMP and PNAMP. Several of the deliverables in the proposal will be co-developed with PNAMP. But the role of each organization in producing these deliverables was not clearly described. A paragraph in the introduction that outlines this relationship and some indication under the shared deliverables of roles and responsibilities would have helped to clarify the division of labor. The monitoring work includes tagging, other types of fish population assessment, experimental habitat restoration, long-term habitat trend monitoring, habitat protocol standardization, food web studies, and management of very large datasets. The project does not emphasize identifying limiting factors; rather, it is aimed more toward evaluating the responses of aquatic habitat and fish populations to restoration actions at large spatial scales. Overall, the proposal does a good job of relating ISEMP-sponsored monitoring to other restoration and monitoring projects. The restoration questions being addressed in each of the watersheds are appropriate to the issues believed to be limiting to salmon production. The new CHaMP rotating panel (GRTS) monitoring design appears to hold considerable promise in characterizing habitat status and trends. The project is consistent with the call for expanded RM&E in many subbasin plans and regional programs. This work is relevant to most RM&E efforts basinwide and provides a means by which RM&E programs in diverse subbasins can be unified under a common set of protocols and procedures. The ISRP suggests that pollutants not be overlooked as potentially limiting factors in certain locations. 4. Deliverables, Work Elements, Metrics, and Methods Deliverables, work elements, metrics and methods are adequately described. Some of the details specific to different work elements could be explained more completely (e.g., in what types of habitat will benthic macroinvertebrates be sampled, and why?), but given the very broad scope of the project and the need to cover all the work elements this is understandable. Journal publications are listed in many of the deliverables for different objectives, but to date there have not been very many papers published from the ISEMP work. Hopefully this will change in the near future. What was the rationale for including 25 sample sites in each basin for habitat status and trend monitoring? Did the ISEMP data suggest that this number of sites would provide adequate statistical power? Some support for this number of sites should be provided. It would seem that the number of sample sites required to adequately represent the range of channel types within a basin would vary based on watershed size, variation in topography, geology, land use and other factors. If this is the case, a variable sample size might be more appropriate. The assessment of stable isotope analysis to characterize trophic aspects of habitat condition in the John Day River is a novel element of this habitat monitoring program and could lead to the development of a very valuable tool. Limiting factors have almost exclusively been restricted to physical habitat or water quality attributes, largely because there was no efficient method for assessing food web conditions. One suggestion about the proposed methodology for this work element: the terrestrial invertebrates should not be lumped together for stable isotope analysis. As with the aquatic invertebrates, the terrestrial insects should be grouped on the basis of functional group (detritivores, herbivores, or predators). The deliverable that addresses status and trend monitoring in the Wenatchee actually discusses this activity in the Entiat. In fact, the information for the Entiat is repeated in the second deliverable, which actually does address the Entiat. This discrepancy seems like an inadvertent cut-and-paste error, but the Wenatchee information should be added to the first deliverable. ISEMP suggests, where appropriate, an “experimental approach” where habitat restoration actions in “treatment” streams are compared to reference “untreated” streams. Given the physiographic and biological variability within reaches or tributaries within a subbasin, let alone differences between subbasins, selection of appropriate references and treatments could prove challenging. It probably would be helpful if ISEMP provided guidelines and/or assistance to subbasin investigators for selecting both reference and treatment sites. The same could be said for data analyses. With the large amount of data that will be collected, investigators may need some assistance in data analyses. It is our understanding that ISEMP is planning to provide analytical assistance where needed.

BPA has questions regarding the action effectiveness study for Salmon Subbasin watershed model implementation in the South Fork Salmon River and Lemhi River.

 

ISEMP staff would be happy to meet with the BPA FCRPS Biop workgroup to address any questions regarding the watershed model being implemented in the SF Salmon and Lemhi Rivers.  The modeling approach was reviewd by ISRP as a NPCC condition for funding in 2006, and has not changed appreciably.  The implemetnation of on the ground fish and habitat monitoring in the SF Salmon and Lemhi Rivers over the past two years is to paramterize the watershed model.  The model will be used to predict fish population responses to restoration actions planned and underway in the Lemhi River.

This is a good proposal overall. The large scale, basinwide approach is good. A project such as this one is clearly needed in the Columbia Basin to integrate M&E efforts and provide consistency among diverse M&E projects. The technical background and rationale are discussed and clearly establish the need for an integrated monitoring program for the Columbia Basin that could contribute to development of an adaptive management plan for the basin. If the process proposed in this project succeeds in bringing together a wide variety of large environmental data sets in a new and integrated fashion, it will represent a major breakthrough in describing and managing tributary restoration efforts.

The proposal is very complex. How are all of the separate parts of the proposal going to be integrated? Some questions related to the technical background of the project and its objectives need to be addressed:
1) What is the role of the sponsors in this project? Coordination? Data collection? Data analysis? Will the sponsors have some involvement in each objective?
2) What does monitoring at the subbasin scale mean?
3) What are some examples of metrics that represent subbasin-scale performance?
4) How will the information generated by the projects be integrated and analyzed to accomplish overall project objectives such as determining limiting factors and evaluation of basinwide project effectiveness?

The project history is clearly described, with a good justification of why the work should be continued and why the suggested pilot-scale sites were chosen. The list of accomplishments is impressive. The three-year history of the project shows how it has grown in both scope and linkages over time. The project is linked to numerous state and federal projects within each of the targeted subbasins.

The objectives are very broad in most cases and involve continuing work begun in 2003. The Wenatchee and John Day projects do a good job of relating objectives of each individual project to the overall project objectives. The objectives for the South Fork Salmon River and Lemhi are not as clear. How do the objectives for this work relate to overall project objectives (item 3.0)? The South Fork Salmon and Lemhi projects should use the same format as the Wenatchee and John Day.

Methods were clearly explained, and the approach will involve innovative techniques. This proposal builds on using many of the best available long-term population status and habitat inventory datasets in the region. The proposal notes that it will take a long time to determine the success of the integrated status and effectiveness monitoring program, but the provisions for long-term monitoring and the choice of monitoring sites were clearly thought out.