In our preliminary review, we asked the proponents to lead the development of an M&E Matrix for the John Day River basin. The ISRP appreciates the proponents’ leadership, constructive response, and thorough summary of monitoring activities in the John Day River subbasin, for both their project and other collaborating projects. In many ways, the summary of monitoring and evaluation, matrix of monitoring activities, and maps of monitoring locations provided for the John Day River basin is an excellent example of cooperation among projects and identification of collaborative monitoring and evaluation in a geographic area that the ISRP envisioned in our request for M&E matrices.

In the revised proposal, the proponents provide an initial map and table illustrating the relationships between implementation actions and monitoring. The revised proposal thoroughly describes the project’s monitoring activities, includes maps of the locations of all monitoring actions, and provides a matrix of the structured implementation and monitoring plan for three John Day River steelhead populations. The proponents collaboratively developed a table of implementation projects associated with the Oregon Watershed Enhancement Board Focused Investment Partnership (Table 8). For each project, the table identifies the restoration strategy, monitoring tier, type of monitoring (implementation, effectiveness), potential metrics, use of drone monitoring, and project responsible for monitoring. They plan to incorporate the figure and table in ongoing annual reports to BPA. The revised proposal includes a preliminary map of implementation projects and associated monitoring in the John Day River basin. The map distinguishes the type of monitoring as implementation, effectiveness, or fish-in and fish-out for fish population productivity. The proposal also summarizes linkages between implementation projects and monitoring efforts in the North Fork John Day and Middle Fork John Day rivers. Basically, the proponents provided initial information on all aspects of M&E that the ISRP requested in the Response Loop. Their M&E matrix and summary could be used as an example for other geographic areas.

The proponents responded to the ISRP’s question about quantifying the effects of both flow and temperature and representing these factors in the model for the Middle Fork spring Chinook salmon population and indicated they agreed that it would be beneficial to update and extend these analyses. Given the importance of both flow and stream temperature as limiting factors in the John Day River basin, the ISRP encourages the project to continue their efforts to understand and model the effects of flow and temperature on spawner recruitment.

The ISRP repeats from its preliminary review that this is an exemplary project and is a model for other M&E projects.

Preliminary ISRP report comments: response requested

Response request comment:

The John Day River Salmonid Monitoring project is a status and trends study that has received positive reviews from the ISRP since its inception. The John Day River basin is one of the few basins in the interior Columbia region that has had no recent hatchery releases; however, straying from Snake River populations is a major concern. The project steadily has improved its experimental design and refined its field methods and analyses. Its data are critical for regional management, and the project is closely integrated with key management plans and habitat restoration projects. The proposal provides a thorough literature review, with much appreciated hyperlinks to key papers and reports. This is an exemplary project and is a model for other M&E projects.

The ISRP requests a response from this project to provide the following information:

• M&E matrix - lead. Provide a summary of linkages between the monitoring conducted and the implementation projects in the John Day River basin. One of the challenges for ISRP reviewers is understanding the specific monitoring that is being conducted for multiple implementation projects. Habitat restoration projects or hatchery projects implement actions that are intended to address limiting factors and benefit fish and wildlife. Most of these projects do not directly monitor habitat conditions or biological outcomes, but most identify other projects in the basin that monitor aspects of physical habitat or focal fish species. The monitoring project(s) in the basin provides essential monitoring data for habitat, juvenile salmonid abundance and distribution, outmigration, survival, and adult returns for salmon and steelhead. Some monitoring projects focus on status and trends in basins, while others focus on habitat relationships and responses to local actions. It is unclear what monitoring the monitoring project(s) conducts for each implementation project.

The ISRP is requesting a response from this project to summarize the linkages between implementation and monitoring projects in the basin. The summary should provide a table or matrix that specifically identifies what is being monitored for each implementation project, as well as where and when the monitoring occurs. The summary also should explain how the projects are working together to evaluate progress toward addressing limiting factors and identify future actions. A map or maps of locations of monitoring actions would be helpful in this regard. The monitoring information should clearly explain whether the biological monitoring is local information for the specific implementation site or basin scale monitoring of status and trends or fish in/fish out. We are asking all relevant implementation projects to assist your project in producing this summary. In particular, the John Day Partnership recently hired Nick Bouwes to help map and coordinate their M&E efforts, and they will be asked to assist.

Q1: Clearly defined objectives and outcomes

The proposal clearly identifies four major goals for the monitoring and evaluation, and identifies key management questions addressed by the monitoring and analyses. Quantitative implementation objectives are identified for each major goal. There are no SMART objectives for physical or biological outcomes, but these are not directly relevant for an M&E project designed to provide critical data and analyses for the region and for habitat projects therein. The project generates information used by multiple projects in the John Day River basin and the region, and the activities appear to be well coordinated.

Q2: Methods

The ISRP has reviewed the proponents’ methods previously and has found them to be scientifically rigorous and excellent examples of Tier 2 statistical monitoring at the subbasin scale. The project continues to refine and strengthen their methods. They combined probabilistic sampling for steelhead populations and census sampling for adult Chinook populations. Their data are stored in regional databases and are incorporated into regional planning and evaluation. When BPA budget cuts prevented them from continuing their monitoring for certain population and habitat parameters, they obtained external funding for some measurements. Most notably, they obtained alternative funding for Chinook escapement assessments, which allowed them to continue to estimate SARs, a critical need for the Fish and Wildlife Program. They have used a spatial model (Falke et al. 2013) to provide quantitative predictions of redd occurrence probability and probability of spawning at a landscape scale. This information provides an important context for planning and prioritizing restoration actions.

The proposal thoroughly describes study designs, field and laboratory methods, and analytical procedures. Methods, data, sampling locations, and metadata are located in NRIMP, MonitoringResources.org, and Annual Reports. The ISRP appreciates the thorough descriptions of the methods.

The proposal includes a Gantt chart to describe the general operations of the project monthly for 2023 through 2025. While this helps reviewers understand general distribution of the project efforts through time, it does not provide information on the subcomponents of the project. This information should be provided in annual reports and work plans.

In the Survival, Age Structure, and Productivity portion of the Methods section, the proponents mention that environmental covariates and restoration metrics can be incorporated into the models with more time series data. Given the changes in temperature and flow occurring throughout the basin and the tremendous effort going into restoration activities throughout the basin, this is especially critical. Quantifying effects of these factors on fish will be key to manage for future conditions and ensure that restoration is effective. Could the authors attempt to run models that include these factors now, even if results are preliminary? If not, when do the authors anticipate that enough data will be available?

Q3: Provisions for M&E

The proposal describes a thorough adaptive management process based on regularly scheduled internal meetings, annual reports, and consultation with BPA. They also describe a formal process of meeting with the John Day Basin Partnership to inform habitat restoration planning and implementation. Their explanation is much more complete than that provided by other related projects in the Partnership.

It is less clear what triggers modifications to the program. How are changes to the project proposed and implemented? For example, the proposal mentions the need for more information on bull trout migration, possible challenges due to increasing numbers of invasive smallmouth bass, and changing climate, including asynchronous hydrology between the Columbia River and the John Day River. What prompts a redirection of the project to better account for these factors? Does each component have a “threshold of probable concern” that would result in changes to the activities? If so, what are they? As well, when is each action deemed successful, and when is it not?

The proposal also describes potential confounding factors related to climate change and expansion of nonnative predators. Rather than simply identifying and discussing the nature of these confounding factors, they present studies they are currently implementing to address these confounding factors and develop methods to implement and assess future management actions. They also have developed an approach for identifying coldwater refuges that could serve as the basis for habitat protection and restoration in the face of climate change. Equally importantly, they are conducting experiments to determine whether smallmouth bass predation is additive or compensatory, a complex and important question in the Columbia River Basin at this point.

The proponents provide an excellent description of their responses to past Council recommendations and ISRP reviews. They have responded positively and creatively to past suggestions, making improvements that exceed recommendations from the ISRP. This constructive communication between the proponents and the ISRP is an example of how the scientific review process is intended to function.

Q4: Results – benefits to fish and wildlife

The proposal provided 13 pages with informative tables and figures of past progress and outcomes to benefit fish and wildlife from the project since 2000. The proponents identified the importance of this information for management decisions within the Columbia River Basin and groups that are using their information for conservation and restoration actions. They clearly have identified lessons learned from their results and how they are being used for management. The indirect benefits of his project are large as this project strongly contributes to status and trend monitoring for steelhead and Chinook salmon, life-cycle models, regional actions, and management of the hydrosystem.

This project has a good track record and is managed by qualified scientists. It will contribute useful information for managing adaptively within the John Day subbasin. 1. Technical Justification, Program Significance and Consistency, and Project Relationships The technical justification and scientific background was reasonably thorough. There appear to be two major topic areas in this work. First, there is evidence that straying of hatchery Chinook salmon and steelhead into the John Day River system is adding to the difficulty of determining the status and trends of naturally produced fish in this (primarily) wild fish production subbasin, and there is the possibility that interbreeding of wild and hatchery fish may be contributing to a loss of fitness among naturally spawning and rearing stocks. This project proposes to extend the existing genetic monitoring program to further assess the significance of straying of Chinook and steelhead from hatcheries in other subbasins into the John Day River's natural production areas. Second, the project proposes to expand the aquatic and riparian habitat status and trend monitoring efforts. This is of interest because of the strong emphasis on habitat restoration in the John Day subbasin. The project is consistent with a number of RPAs in the BiOp and with other programmatic emphasis areas in the region. The approach is clearly spelled out, and the ISRP appreciates that the project proponents have responded directly to our suggestions from previous reviews and incorporated them into this plan. The proposal references specific recommendations from the RPA workshop and identifies the RPAs in the BiOp that the work will address. Additional linkages are given to the John Day subbasin plan and to the Council's Fish & Wildlife Program. The 2008 FCRPS BiOp identified the Upper John Day as a priority subbasin for recovery of the Mid-Columbia steelhead DPS. The John Day River is an important reference subbasin for comparisons to other anadromous stocks in more highly impacted subbasins of the Columbia River. Further, the John Day is unique among Columbia River subbasins because its Chinook and steelhead populations have had little influence from hatchery introductions. The project, initiated in 1998, is one of the more extensive monitoring and evaluation programs in the Columbia River Basin. The proposed work is consistent with the monitoring needs identified by the Columbia River Basin Fish and Wildlife Program, the BiOp, and the Oregon Plan for Salmon and Watersheds. The project also directly addresses several important needs specified in the John Day Subbasin Plan. 2. Project History and Results The project’s history is thoroughly described, including tables summarizing results of the previous nine years of work in this project. The maps were very helpful, although most of the material referred to existing redd count, adult holding areas, and genetic sampling. It would also have been useful to have included maps or tables describing the locations of significant habitat restoration projects (perhaps by category, if available) and sites where there are records of juvenile salmonid presence-absence or abundance, because these are included in this project's objectives. The project has generated much information on smolt production and adult escapement. The discussion of results could have been improved if the proponents would have discussed how the results to date have influenced management decisions. For example, what useful information was derived from the stock-recruitment relationship? 3. Objectives, Work Elements, and Methods The objectives and work elements are generally complete and appropriate. Several questions did arise in reviewing this section: 1) Obj. 1 Are 4000 PIT tags enough to estimate SAR given the relatively few adults that will be produced from this number of smolts? 2) Obj. 1 Is a 100% detection of PIT tagged adults at the FCRPS PIT tag facilities a reasonable assumption? 3) Obj. 2 How does one obtain cost-free flight time for redd surveys? 4) Obj. 4 If habitat assessments are conducted only on channels 5th order and smaller, will some areas potentially used by Chinook for spawning or rearing be excluded? 5) Obj. 4 Ignoring fast-water habitat when snorkeling could lead to some error in estimating abundance if a substantial proportion of the populations using fat-water habitat. This is not likely to be a problem for Chinook but could be for steelhead. Could several fast-water habitats in each sample area be electro-fished to address this problem? The two elements for which Fast-Track review were requested (Chinook portion of Obj. 2 and Obj. 6) were very complete. Three of the five work objectives apply to ongoing efforts by the project's proponents, and two objectives (habitat status and trends, juvenile salmonid monitoring) have been called for by the ISRP but not yet funded. The objective of operating the rotary screw traps from October 1 to May 31 is admirable, but mechanical problems, weather-related issues, and other unforeseen problems will inevitably occur. It would be useful to state what back-up equipment and procedures are in place to deal with such events. The spawner escapement estimation procedures were thoroughly described, and the project proponents have a lot of experience in this aspect of the work effort. It was nice to see that the proposed 2010 surveys will include some randomly selected reaches to check for Chinook salmon spawning range expansions. The ISRP wonders if any research has been done to determine the error rate for identifying hatchery fish based on adipose fin clipping in the John Day or nearby subbasins. What is the probability of mis-identifying a hatchery fish with an imperfect adipose clip, or a wild fish that has lost part of its adipose fin? The location of the habitat survey sites will be determined by a GRTS (EMAP-type) randomized design. While there will be some inevitable compromises due to landowner access issues, we wonder if there is any value in intentionally locating some of the habitat survey sites at (1) the same locations as the spawning index sites or the juvenile survey reaches, or (2) within or near riparian or channel restoration projects. We agree that the GRTS approach is appropriate for assessing overall habitat status and trends within the subbasin, but additional and very useful information might be gained by co-locating the habitat survey sites with sites where fish will be censused either as spawners or juveniles. Within the list of habitat attributes, we recommend expanding the surveys of large trees (>0.5 m DBH) to all species, not just conifers. Owing to past logging practices and wildfire history there are very few conifers of this size within 30 m of the stream channel throughout much of the John Day subbasin, but there are other species (in particular, black cottonwood) that meet this size criterion. While it is true that cottonwood will not persist as long in the streams, it is currently the best candidate for LWD recruitment. To estimate escapement of Chinook and steelhead from redd counts, the proponents propose to use data on fish/redd from another subbasin. This approach seems reasonable given that weirs are not present in the John Day to enumerate the number of adults potentially spawning above the weirs. Why are there no weirs in John Day tributaries? Steelhead redds will be enumerated in five tributaries thought to support independent populations. From this information subbasin escapement will be estimated. Why not estimate redd densities and escapement for each of the five tributaries (populations) rather than just for the subbasin as a whole? This approach could provide greater resolution and information about redd and escapement levels especially if the tributaries differ significantly in quality and quantity of spawning habitat. For Objective 3, pertaining to habitat status and trend monitoring, are any of the sample sites likely to be in reaches where restoration activities have taken place? How will data analysis deal with information from both sites where habitat enhancement has been implemented and sites that have not been treated? Will there be an attempt to distinguish between the two in analyses? Water temperature and thermal refugia are not being monitored. Is availability of cool water during summer not a limiting factor in the John Day basin? For Objective 4, will juvenile sampling be concurrent with habitat sampling? Estimation of juvenile abundance will be based upon snorkel counts of juveniles in pools. Pools will be ranked according to a visibility rating. Pools ranked 0 or 1 (poor visibility, high amount of hiding cover) will not be used in data analysis if the reach also contains pools ranked 2 and/or 3 (moderate to little hiding cover, good visibility). However, if all pools within a reach are ranked < 2, then the pools will be electrofished rather than snorkeled. This could introduce bias because for some reaches pools ranked 0 or 1 (probably the best fish habitat) will be excluded from analysis while in others (electrofished reaches) they will be included and, further, the reach data may not be comparable because two different methods of estimating fish density were employed (snorkel and electrofish). Density estimates of electrofished reaches may be greater than those of snorkeled reaches both because better quality habitat is being sampled and electrofishing may be a more effective way of counting fish. Why not electrofish all reaches and eliminate the potential problem? 4. M&E The M&E program proposed by this project has been carefully planned and well conducted to this point. The proposed additions to the ongoing M&E effort (Chinook and steelhead escapement, habitat and juvenile monitoring, genetic characterizations) would round out the program and provide much needed additional information. As in the past, the ISRP supports both juvenile and habitat monitoring to identify productive rearing habitats, establish quantitative relationships between habitat quantity and quality and juvenile abundance and distribution, quantify limiting factors for juvenile survival, and guide habitat enhancement actions.

We are in agreement and are willing to cooperate closely with the PNAMP data workgroup to make our data more avaialble. We are in the process of hiring a database position for our program to support database development for all of our BPA contracts.