The fast track element for Method #8 to install a second screw trap on Marsh Creek is sufficiently justified. For the remainder of the proposal, including the fast-track element to validate AHA modeling, a response is requested in the form of a revised narrative. The response should provide the following: 1. A table that outlines the ESU, MPG, Independent Populations, and streams in the Snake River system and identify which are potential high-precision and low-precision sites for RME. 2. A summary explanation of what process is underway (if any) to decide which component streams are part of the intensive and extensive sampling. 3. Greater detail of explanation for the precision/sampling intervals for intensive and extensive sites. 4. An overview-to-date of trends in ISS spring/summer Chinook abundance by location for all treatment and control (reference) streams. Include a summary table of the data collected for each of the sites by the ISS (or cooperators) since the last ISRP review. 5. Comparison of the precision and sampling intervals in the ISS streams with that desired by the BiOp RME and ISS statistical analysis. 6. Statements for the ISRP about any events or problems encountered since the last review that may compromise the analysis of the ISS. 7. Elaborate on the methods and approach to validate the assumptions and uncertainties in the AHA model In concluding, the ISRP suggests that the final report should be independently peer reviewed when in draft form, much like what was done with the 10-year retrospective for the comparative survival studies (ISAB/ISRP 2007-6). 1. Technical Justification, Program Significance and Consistency, and Project Relationships Justification for this project is well documented in the proposal and in the Council's current and past program language. The project has many links and relations—and large implications—for many other projects in the basin. The proposal clearly identifies relationships to the 2008 BiOp, TRTs recovery plan, Columbia Basin regional RM&E strategy, NPCC Fish and Wildlife Program, and WY-KAN-USH-MI WA-KISH-WIT. As the proposal states, the Idaho Supplementation Studies (ISS) study design was intensively scrutinized and updated prior to the last funding cycle. This proposal "represents the ongoing efforts of the cooperating agencies in the ISS program to take that study design to completion without change" which is a significant and necessary pledge. The objectives of the program are to evaluate the effects of supplementation on juvenile and adult Chinook abundance; evaluate changes in natural productivity after supplementation is stopped; evaluate various supplementation strategies; and develop supplementation recommendations. These objectives seem important, relevant and consistent with policies and with biological and policy needs. The key feature of this project is that it uses supplementation as a research treatment. Monitoring production and productivity responses for supplemented populations and unsupplemented reference populations could provide important insights into the effects of supplementation. Continued monitoring after supplementation ceases could contribute to a better understanding of the long-term effects of supplementation. At the completion of the project, status and trend monitoring currently provided by ISS will need to be incorporated into new or ongoing programs. The program has large potential significance to the region. It is supported by the region’s technical community as long as it reaches its objectives as planned and the results are unambiguous. If so, it should answer a number of long-standing and contentious issues about the impacts and efficacy of supplementation as a method of sustaining and providing long-term increases in depressed salmon populations' productivity. Data to allow such insights should start to come in over the next few years, thereby justifying this 20+ year investment. 2. Project History and Results The proposal provides a lengthy and good review of the project's 20-year history. The ISS study design was first proposed in 1990, and has been executed within this framework with challenges owing to the unavailability of fish for stocking during the mid-1990s and owing to policy/political decisions by the Nez Perce Tribe to begin a supplementation program on Johnson Creek, one of the control (reference) streams. Because of these difficulties, the ISRP urged a review of the experimental and analysis design in 2001 to determine whether the effort had the potential to produce data that could be analyzed and used to answer uncertainties about supplementation. The ISRP has reviewed updated study designs in 2005 and again in 2006 during the 2007/2009 project solicitation. Because of this history of review and the depth of detail in the design, it is probably not necessary to revisit these topics. However, the proposal is insufficient to fully glean this background from the proposal. The project history presentation also describes the extensive resources (funds) dedicated to the project for the past two decades. Treatment phase of this long-standing project ended in 2007, and the evaluation phase started in 2000. Thus limited data are available to represent the evaluation phase. However, virtually no results or data were presented in the project proposal other than the brief summary of Pahsimeroi Chinook genetic parentage studies. The lack of results, even at the minimum level of general trends and observations, was disappointing. The proponents should have provided more results on project performance to date. The proposal lists results as the development of protocols and the performance of annual data gathering (adult weirs, redd count and carcass surveys, emigrant traps, and snorkel estimates). While these steps were surely necessary to the project’s implementation and M&E, they do not constitute data or results—they are process. Only one peer-reviewed paper appears to have been produced through the project, and it is not focused the on primary objectives of project. There is a five-year-old masters thesis on parentage analysis in a study stream (Pahsimeroi) that has not been published in the open literature. This omission should be rectified, particularly given the reliance by the proposal proponents on the results of that study. Consequently, it should be validated through the peer review process. As written, the proposal suggests that data analysis will only begin after the last fish has been processed in 2014 or thereabout. It is reasonable (at least from the ISRP’s perspective) to expect that some preliminary analysis of data from the evaluation phase of this effort that began in 2008 start immediately. Finally, as a result of actions taken in response to the 2001 ISRP Provincial Review, the project has been modified to better provide relevant data. Unfortunately, the important work proposed in Genetic Evaluation of Salmon Supplementation in Idaho Rivers 200725000 was not funded. 3. Objectives, Work Elements, and Methods The proposal clearly states the objectives as research goals: 1) Assess the use of hatchery Chinook salmon to increase natural populations of spring and summer Chinook in the Salmon and Clearwater River drainages; and 2) Evaluate the genetic and ecological impacts of hatchery Chinook salmon on naturally reproducing Chinook populations. The proposal identifies the contribution that each task will make to program needs, though methods are not described by objective or work element Methods were not included in the proposal, though references were made to methods and analysis procedures that are well documented elsewhere, often as a result of discussion or reviews by the ISRP. A reviewer new to this project would likely find the proposal inadequate in these areas, if they were not familiar with the earlier documents and discussion. For example, a brief explanation and justification of the standardized index of adult escapement and natural production used to compare across treatment and control (reference?) streams should be provided in the proposal. Fortunately, the project appears to be remaining squarely on course with its schedule and methods as previously worked out with the ISRP through the extensive and intensive reviews done in 2001 and 2005. Two new objectives appear to be the basis for the Fast Track proposal review: 1) Method #8 would install a second screw trap on Marsh Creek in order to provide additional data on juvenile outmigration from the Marsh Creek drainage; and 2) Method #9 would "verify AHA and AHSWG model assumptions and predictions using ISS data." Installation of the screw trap at Marsh Creek appears reasonable and justified. The second Fast Track element, AHA model validation, is not currently justified and does not look time sensitive (i.e., fast track), and therefore more justification is needed. The other project work elements seem well established now and relatively routine. The proposed new task of using historical ISS juvenile release, survival, and adult return data to evaluate the AHA model by comparing actual adult returns to those predicted by AHA and AHSWG models is a useful component of the ongoing project; however, several issues are raised by this new task. First, ISS field activities are scheduled to end during this funding cycle, and appropriate plans will need to be developed if ISS infrastructure is to be maintained for intensive and extensive VSP status and trend monitoring (Appendix A). After 2012, ISS cooperators will no longer be evaluating adult returns to study streams (i.e., redd counts and carcass surveys), and juvenile migration evaluation will be complete after smolt trapping in 2014 (brood year 2012). In order to maintain current levels of intensive and extensive status and trend monitoring, ISS infrastructure and sampling duties will need to be incorporated into new or existing programs. Second, select personnel from the ISS cooperating agencies will need to be funded for a period of time after 2014 to complete data analysis, a completion report, peer reviewed publications, and ensure ISS findings are communicated to and incorporated into other regional supplementation programs. As the project nears completion and personnel are shifted within the project and agency, we encourage key personnel to remain engaged in the project through its completion, in order to best realize the project’s original objectives. Finally, AHA has been reviewed by the Puget Sound Technical Recovery Team (Review of the All-H analyzer model, March 18, 2005) and by the Recovery Implementation Science Team (April 9, 2009). There is no discussion of the past reviews of this model, how the ISS data will be used to address which assumptions and uncertainties, and how the analysis and assumption validation will be conducted. 4. M&E This is one of the project's strong suits, though not enough detail is provided in this specific proposal for a new reviewer to determine that (or possibly to support the project). The strength of the methods comes from the work done by University of Idaho (at the request of the ISRP) to identify a statistical procedure that could answer the supplementation questions posed by the project and by the proponents commitment as stated in this proposal to maintain the study design unchanged through the sunset date for the project. It is still not clear whether the ISS will yield data that can be analyzed to answer questions about the efficacy of supplementation. The primary challenge is the quality (precision) of adult abundance data that can be derived from redd count and carcass inspection in study streams that lack interrogation weirs. Even on streams that have weirs, estimating weir efficiency and adjusting data may be necessary. In the last review, the ISRP was critical of the carcass data that was collected in conjunction with redd counts, which would be used to assign total adult counts (based on redds) proportionately to wild, supplementation, and conventional hatchery (strays) production. For several sample streams the carcass information came from limited sampling and had no associated estimates of precision. The ISRP recommended that the redd counting and carcass inspection be more rigorous. The proposal cites that the goals and strategies for monitoring and evaluation of the status of Snake River Chinook salmon and steelhead identified in the fall 2009 RM&E workshop guide expansion of the ISS (Table 1, page 7). More information is needed for evaluation of whether the increased effort meets the RM&E goals. In particular, one goal is to "obtain high precision status and trend data for at least one population per adult life-history type per MPG (fish in, fish out monitoring).” One of the open questions is the selection of populations for this monitoring. The ISS proposal suggests that they may be collecting this information; however, the population is not yet selected. A succinct summary of the MPGs and independent populations established by the TRT, which have high precision data, and which are associated with the ISS needs to be included in the proposal. The high precision data type is not clear. The citation is to Crawford and Rumsey (2009) and reference is to data with a CV of 15% or less. CV (coefficient of variation) is not usually associated with precision of data, but with the variation associated with a state of nature. That is, salmon abundance across years has a CV, fall steelhead parr length has a CV. These are descriptions of the state of variation. They are not appropriate to determine confidence intervals. Crawford and Rumsey (2009) reference Carlile et al. (2008), which makes recommendations for coefficients of variation for estimates of total spawning escapement. The reference is to standard error of the estimate, not to variation in the population. More importantly, the statistical and biological basis for the recommendation in Carlile et al. (2008) has not been reviewed. The justification that the standard represents a realistic goal for planning because it corresponds to an acceptable risk (one year of one stock in six) of failing to label a stock of concern when warranted appears to be arbitrary. The observation that the standard has proven to be attainable for many escapement estimation studies does not mean that this is the appropriate data standard. Finally, the project proponents should continue to enlist the services of a statistician, such as they did with K. Steinhorst from the University of Idaho. It is hard to know from the information provided whether the final analysis after 2014 will be meaningful. As part of this review process, a solid reporting of the data already collected would help demonstrate project proponent’s ability to conduct the analysis. It would also be good for the ISRP/ISAB to review a draft report before the final report is released, similar to independent reviews of the Select Area Fisheries Evaluation (ISRP/IEAB 2007-3), Comparative Survival Studies 10-year retrospective (ISAB/ISRP 2007-6), and Captive Propagation projects (ISRP 2004-14). This is a very important (and expensive) long running project which heightens the need to make certain the program comes to fruition successfully.

Additional sponsor reply to ISRP's question:

https://pisces.bpa.gov/release/documents/DocumentViewer.aspx?doc=P119648&session=a8b6a70f-a0dd-42d1-845c-6ba34186c92e