In our preliminary review, we requested responses on the following six topics. Our final review comments, including suggestions to improve the project, are provided under each of the topics:

1. SMART objectives. The proponents provided the additional information needed to specify the SMART aspects of their objectives.

2. Specific thresholds for analysis and performance. Specific thresholds were provided for the desired coefficients of variation of each VSP parameter or other population metric, as well as the performance standards for operation of the Instream PIT tag Detection System (IPTDS).

3. Clarify methods and timelines. Additional details provided in the response were sufficient for the frequency of estimating screw-trap efficiency.

Results of work under a previous objective indicated that a substantial number of spring Chinook salmon redds were not counted (p. 13 of revised proposal). Proponents report that other monitoring programs conduct comprehensive weekly redd surveys that account for observer efficiency to estimate unbiased and precise estimates of hatchery and wild spawners (p. 18 and Table 3). However, the revised proposal also states (p. 23) that WDFW does not perform redd surveys in the Entiat River. Hence, it is unclear whether the unbiased redd counts needed to estimate the desired quantities will be available. The ISRP recommends the proponents clarify this issue in the next annual report.

4. Clarify sources of habitat data. Additional details provided were sufficient concerning the sources of habitat data as covariates for analysis of pre-spawn mortality for the future telemetry study.

5. Project evaluation and adjustment. The ISRP appreciates that this project provides long-term monitoring and does not address adaptive management of salmon or steelhead. The proponents adequately described how the decision will be made to terminate the PIT-tag based study (when stable relationships between unbiased redd counts and female escapement are achieved), and other decisions required to maintain a working and efficient IPTDS system.

The proponents did not describe adaptations needed if climate change greatly alters the hydrology of these subbasins (e.g., producing large channel-reorganizing floods), or the returns of salmon and steelhead (e.g., by extreme events that cause high mortality). For example, might this require contingency plans to change locations of the IPTDS antennas or habitat surveys, or change the collaboration with other projects because redds are concentrated in different locations? The ISRP requests that in the next annual report the proponents address what decisions will need to be made to anticipate such changes.

6. M&E matrix – support. The lead project (Upper Columbia River Programmatic Habitat Project 201000100) declined to prepare the matrix requested. The project addressed here provides critical data for management of salmon and steelhead in the upper Columbia River, so there is no question that they are, and will be, providing needed information. One example is that their future work may reveal habitat characteristics that explain pre-spawn mortality, leading to options for habitat restoration to reduce it.

The ISRP has provided additional information on the summary of monitoring and evaluation for geographic areas in the Programmatic Comments of this report. As the Fish and Wildlife Program develops efforts to identify monitoring activities and coordination between projects in major subbasins, the ISRP encourages this project to contribute its expertise and resources to help create an effective summary of RM&E efforts in the Upper Columbia River.

Preliminary ISRP report comments: response requested (Provided for context. The proponents responded to the ISRP’s questions; see response link and final review above.)

Response request comment:

This is a core project among those in the Upper Columbia River Basin. It benefits fish and wildlife by providing information critical for assessing status and trends of anadromous salmonids and making management decisions. The ISRP commends the project proponents on a history of conducting high-quality research and publishing the results in refereed journals.

The proponents generally have a clear idea of the work to be done in the next five-year phase of the project. However, there are several areas in which the proposal could be improved to refine the goals and objectives and provide a clear process for adaptive adjustments during the project period.

The ISRP requests that the proponents provide this information in a point-by-point response in a separate document, or in a revised proposal with a brief point-by-point description of where and how the issues are addressed in that document. In either case, the ISRP encourages the proponents to revise their proposal as a record for future project participants and a framework for ongoing evaluation of their project.

1. SMART objectives. Revise objectives to include all the elements of the SMART format (see proposal instructions). Objectives need to be Specific, Measurable, Achievable, Relevant, and Time Bound
2. Specific thresholds for analysis and performance. Revise objectives to include specific thresholds for statistical requirements for analysis and model performance, and operational performance objectives for the instream PIT tag detection system (IPTDS).
3. Clarify methods and timelines. Clarify methods and timelines for estimating screw-trap efficiency and correcting redd counts for observer bias.
4. Clarify sources of habitat data. If habitat relationships will be analyzed to assess causes of pre-spawning mortality, clarify what habitat attributes will be measured, or obtained from others, and who will make the measurements.
5. Project evaluation and adjustment. Develop and describe a formal adaptive management process that addresses how the project will be managed and decisions made, and what adaptations will be made in the face of climate change to ensure accurate and relevant information is collected on which to base management decisions. As part of the description of the adaptive management process, clarify how the decision will be made to begin the second phase of the study to measure fine-scale movement and distribution of spawners using radio telemetry, and to analyze reach-scale covariates to address causes of pre-spawning mortality.
6. M&E matrix - support. As habitat projects and monitoring projects are not presented as part of an integrated proposal or plan, the need for a crosswalk to identify the linkages between implementation and monitoring is extremely important for basins or geographic areas. The ISRP is requesting a response from the Upper Columbia River Programmatic Habitat Project (201000100) to summarize the linkages among implementation and monitoring projects in the Wenatchee, Entiat, Methow, and Okanogan basins. As a key M&E project and partner in the basin, we ask your project to assist them in creating the summary and provide information to them about what, where, and when your monitoring occurs and what is being monitored for and shared with implementation projects in the basin. A map or maps of locations of monitoring actions would be helpful in this regard.

Q1: Clearly defined objectives and outcomes

The proponents propose to develop a PIT-tag based Spring Chinook Salmon Escapement model for the Upper Columbia River basin, using the same framework as the model developed for steelhead, and to maintain and improve the instream PIT tag detection system (IPTDS). They also will use screw traps to estimate abundance of emigrant Chinook and steelhead from the Entiat River.

The ISRP has several concerns about the objectives:

Two additional objectives include 1) a radio-telemetry study of adult spring Chinook salmon spawners to identify patterns in movement and habitat use that can be related to pre-spawning mortality, and 2) relating pre-spawning mortality to habitat and frequency of recreation in specific reaches to evaluate factors causing this mortality. However, it is unclear whether either objective is achievable within the five-year time frame of this project. ISRP reviewers learned only in the budget narrative at the end of the proposal that the proponents require estimates of escapement and pre-spawning survival to meet a given precision (CV <10%) before pursuing these additional objectives. This needs to be made clear early in the proposal, stated in the objectives, and included in the adaptive management plan.

The objectives are clearly stated but are not fully specified in SMART format. For example, objectives under Goal 1 do not specify the subbasins in which the work will be done, nor the time period. Likewise, objectives under Goal 3 do not specify which life stages and species will be measured, nor what metrics of life history will be measured over what time period.

Given that the project conducts monitoring and analysis, rather than assessing biological performance of the fish per se, the implementation objectives could be improved by specifying the necessary or desired statistical requirements for the analysis (e.g., levels of accuracy and precision to be achieved for run escapement and composition, and pre-spawning mortality), and the operational performance to be achieved for the IPTDS (e.g., proportion of sampling period during which sites are operating, tagging rates, detection efficiencies for PIT tag arrays and smolt traps). Several of these are reported in Methods and should be explicitly stated in objectives.

Q2: Methods

Overall, the methods are based on sound science, and the ISRP commends the proponents on publishing key methods and analyses in peer-reviewed journals. However, certain details need to be specified:

• How often will estimates of rotary screw trap efficiency be made? Will the redd counts of Chinook salmon in the Entiat River be corrected for observer bias, as described earlier in the proposal for steelhead?
• If pre-spawning mortality will be related to habitat characteristics during this proposal period, what habitat measurements will be made by the proponents, and what habitat data will be provided by others?

Q3: Provisions for M&E

The proponents state that the project does not require an adaptive management process, but the ISRP found that a structured process is needed to address several points:

As described above, the decision process for moving to a second phase and conducting a study of fine-scale movement and distribution using radio telemetry, and relating pre-spawning mortality to reach-scale habitat features, is not described clearly and needs to be specified as part of the project evaluation process.

The proponents are asked to explain the overall management of the project activities on an annual basis, the schedule for such evaluations and decisions, the decision-making process and who is responsible for final decisions, and how these decisions are recorded.

Extreme events and other exigencies caused by climate change have potential to confound many projects addressing fish and wildlife in the Columbia Basin. The ISRP requests the proponents to define how they plan to adjust their monitoring and analysis to address such events and produce the most accurate and relevant information on which to base management decisions.

Q4: Results – benefits to fish and wildlife

This is a core project in the upper Columbia River Basin, which provides valuable information for assessing status and trends and making sound management decisions.

This proposal describes work that will be very useful in characterizing the population status of upper Columbia Chinook and steelhead. It seems well integrated into the ISEMP M&E effort, and it will also help describe the status of wild production and hatchery survival. The work is consistent with efforts of the UCRTT to improve RM&E methodologies and outputs. A few more details about the quantitative and statistical methods for some work elements would have been helpful, but overall the project is scientifically justified. The needs as outlined in the existing objectives are well articulated in clear language by the proponents. Many aspects of this project, however, seem to require well-developed quantitative skills, statistical skills, and other specialized skills related to PIT tagging arrays. Overall, the resumes of the personnel do not seem to indicate the necessary quantitative and statistical backgrounds to adequately implement, or even necessarily evaluate efforts on, some of the more quantitative objectives. The enlisting or a well-qualified statistician and other quantitative expertise as well as other experts in PIT tag applications will be necessary. It did not seem that this expertise had yet been identified. It might have been beneficial to have such expertise in on the proposal itself. This expertise should be enlisted at the earliest possible time, e.g., to critically evaluate past studies that lead to the need for this effort and to complete a design for locating the arrays. Is the statistician a WDFW statistician or an outside contract person? The proposal is unclear on this point. Similarly, the roles of NMFS and other specialist in the development of an “automated analytical tool for data collected at in-stream PIT tag interrogation systems” (Objective 3) are not clear. There does not seem to be the expertise listed in the proposal to deal with this objective. Should the PIT tag analytical tool be developed by PITAGIS, i.e., at a more centralized site (than in this regional proposal) and coordinated more broadly in the basin. The objective related to spawning ground surveys will fill an identified gap in the Okanogan Basin. The proposal is directed toward several specific objectives necessary for improving precision and accuracy of estimates. However the proponents are also aiming at RPAs 56.1, 57.1, 57.4 - "tributary or in some cases reach specific migration patterns, survival rates, residence time, and limiting factors could be assessed in relationship to habitat restoration actions." The proposal would be improved by more coordination with habitat restoration projects in the subbasins they propose to work in. The installation of all this technology could be helpful to habitat restoration evaluation. 1. Technical Justification, Program Significance and Consistency, and Project Relationships The proposal is technically well justified, adequately linked to the BiOp's RPAs for the upper Columbia, subbasin plans, and well integrated into existing Chinook and steelhead monitoring efforts in the eastern Cascade Mountains. The close connection to ISEMP (some parts of this project were actually designed by ISEMP staff) is beneficial. The proposed work is significant because of the proponents' need to understand the variance of smolt trap population estimates, improve spawning ground population estimates and document migration routes of steelhead and Chinook. Working relationships are stated to be good, and the proponents claim there is good coordination already. The Colville Confederated Tribes will be the lead agency in the Okanogan. Does that mean that WDFW will be the lead agency for the other three basins—the Wenatchee, Entiat, and Methow? There does seem to be overlap between this project and 201002800 (Rotating Panel) and possibly others in this package. It would be helpful to clarify that such overlaps do not exist. Is there adequate communication and working relationships among the various proponents in various sub-basins testing the use of “rotating panels” (i.e., a design of trapping smaller tributaries on a rotational basis) to improve steelhead escapement estimates? This is a new project, but it obviously builds on previous work. 2. Objectives, Work Elements, and Methods The objectives and work elements seemed reasonable and none involved untried technologies or untested field methods. Work elements for each of the seven objectives are summarized in a series of tables. While these tables and the supporting text do a good job of describing what would be done, in some cases few details were given regarding how the work would be done, by whom, and over what periods in the field. The tasks in support of Objective 5 (estimate observer efficiency...) were adequately described, however. Objective 1 Develop variance estimates for smolt abundance estimates This objective is narrowly defined around variance estimates for individual abundance estimates. This is a valid, important objective to meet. Although there is no component of this study aimed at assessing whether the smolt sampling arrays in the various rivers adequately cover the basins, this is also an important aspect affecting the overall abundance estimates and the usefulness of the data. It might be, as a minimum, worthwhile for the statistician to look at how adequacies or inadequacies in the overall array might affect overall abundance estimates in the Upper Columbia in relation to variance of individual estimates investigated in this proposed work. The proposal would be improved by more details on the existing smolt abundance estimates and how they are determined and compared. What is "our current method" (3rd line from bottom, page 2)? "The current method to calculate variance produces high variance estimates, despite good results for (e.g., r2 > 0.7) estimates of trap efficiency" (mid page 2). What is being correlated? More background information on the specifics of statistical adequacies and inadequacies would make the proposal more convincing. Objective 2 Estimate the proportion of hatchery steelhead on the spawning grounds The abstract mentions that this project will build on recent advances in PIT tag technology. The proposal would be improved by inclusion of details on what these advances are. Additional details on where the "additional PIT temporary and permanent arrays" are to be located would also be helpful. How many arrays are needed to complete coverage? Is this a statistical question or is it simply a matter of having an array at the mouth of every tributary where steelhead spawn? Or are they the designated minor spawning aggregations, as mentioned in 201002800? On page 3, relative to this objective, the following statement is made: "While radio telemetry may provide information on distribution and origin composition in the short term (RPA 50.4), it can also be used to verify PIT tag estimates will be used in the long term." This sentence should be re-worded for clarity. The proposed work should assist resolution of the major issue of multiple counts of the same spawner, i.e. "As a result, both wild and hatchery steelhead may be counted in one of more non-natal tributaries prior to spawning which would introduce a positive bias in abundance estimates." On page 13, the text should presumably read “in one or more non-natal…” Objective 3 Develop an automated analytical tool for data collected at in-stream PIT tag interrogation systems. It is not clear why this tool is not being developed by PTAGIS directly as it clearly has application beyond the boundaries and scope of the present proponents. There may be some definite advantages in having this objective addressed at a more centralized site and coordinated broadly in the basin rather than in this proposal. It would be worthwhile for the proponents to explain the rationale for retaining this particular objective in this proposal. Objective 4. Estimate the accuracy of steelhead spawning grounds surveys in the Upper Columbia Basin There is significant overlap between this proposal and 201002800 (Rotating panel on small tributaries of the Tucannon and Snake Rivers.) Has there been communication between the proponents? Or is the work so habitat specific that it has to be done independently in the various sub-basins? Objectives 5 Estimate observer efficiency in conducting redd surveys and calculate a variance estimate for selected tributaries Again, there is significant overlap between this proposal and 201002800 (Rotating panel on small tributaries of the Tucannon and Snake Rivers). Has there been communication between the proponents? Or is the work so habitat specific that it has to be done independently in the various sub-basins? Objective 7 Estimate the annual abundance and age composition of wild and hatchery steelhead upstream of Priest Rapids Dam In the proposal, “systematic sampling of the steelhead migrating pass Priest Rapids Dam would be conducted on a weekly basis (Tuesday and Thursday) throughout the run (July – October). The results would then be expanded. Overall, this proposal deals with issues of accuracy and precision. However, this sampling design (two days in seven, and only one day apart (Tuesday and Thursday)) for this last objective gives the appearance of being designed more for sampling convenience than for accuracy and precision. What information is available that the proposed sampling design is the best choice and that it will yield the desired accuracy and precision? Results will feed into ISEMP and other regional data bases. The proponents do not promise annual reporting or journal papers but presumably data obtained in the study will be generally accessible.