ISRP review is pending submittal of a proposal.

Modified by Thomas Ono on 4/17/2023 8:11:56 AM.

ISRP review is pending submittal of a proposal.

The sponsors provided more adequate details of their sampling methods and protocols in the response (www.monitoringmethods.org). The best methods to be used may relate specifically to hypotheses developed. That is, the information needed to evaluate these hypotheses, for example, age structure of fish in snorkel counts. The data collection approach itself seems to be acceptable, but after more than 10 years of data collection, significant problems are arising in the interpretation and actual understanding of the data because limitations have arisen for the data that were not clearly foreseen, for example, resident versus migratory life histories and the need for age structured life history information in snorkel counts.

The key aspect of the proposal for which a response was requested but not adequately addressed was a clear development of hypotheses to guide the bull trout investigation. The sponsors noted that funding limitations and staffing issues due to a shortage of lead scientists have limited hypothesis development. The sponsors stated that “The lack of qualified personnel that could dedicate time to this project has effectively arrested development of the scientific understanding that should have been realized, which by now would have resulted in development of hypotheses to be tested.” They also suggest that future efforts will address hypothesis development, but no hypotheses are forwarded and no details are provided as to how hypothesis development will occur. The sponsors seem reticent to develop hypotheses until they have more data, although they did mention some potential hypotheses on pages 8 and 9 of their response. Many of their responses suggest difficulties in interpreting and understanding data that they collected in the past, suggesting a significant lack of staff continuity and loss of institutional memory regarding the details of the data collection. They did note, however, that some outside scientists would be consulted regarding analysis and interpretation of existing data. For example, the analysis of Budy “will indicate, given the current monitoring study design, what precision and with what power that declining trends in bull trout populations can be detected. ”

This lack of hypothesis development and testing has had consequences on the direction and focus of the project since 1998. The sponsors noted that "In September 2011, a report that reviewed and synthesized data from 1998 to 2009 was completed (CTWSRO Natural Resources Branch Fisheries Research Dept. 2011). Through this effort and preparation of this categorical review, problems that prevent thorough analyses and interpretation of data collected were realized."One of the main "problems" was the inability to distinguish resident from fluvial bull trout, confounding attempts to assess status of the two population segments. Evidently, even after more than 10 years of investigations, this issue of two main life history components was not fully recognized or addressed. In the sponsors’ words, “Apparently, an initial assumption of the original monitoring plan was that only fluvial bull trout were present in the study area. This is believed to be erroneous and will be addressed by using half-duplex PIT tag technology to determine home range of resident forms and migration timing and spatial patterns for fluvial forms.” This difficulty of identifying the fish in each life history type has clouded the interpretation of the time series collected over the past decade. The proposed work with half-duplex PIT tags is thus designed to address this limitation, although the details of how the life histories will be, as the sponsors state, “teased out” remains unclear.

In trying to understand the resident versus fluvial life history components, it may be useful to think about exactly what kinds of data need to be collected from fish besides PIT tag data, for example telemetry data, scale pattern analysis, reproductive periodicity data, to identify the life histories and how many fish are contributing to each pattern. It would seem that radio telemetry might be an effective method for addressing this issue. In addition, the relation between native bull trout and introduced brook trout is confusing. As the sponsors state, “brook trout are sympatric with bull trout in index reaches therefore, redds from brook trout and resident bull trout may be indistinguishable. ” Other issues regarding interpretation, for example the data depicted in Figures 3-5 in the response, seem to be a result of not clearly having hypotheses to guide the exact sampling methods, resulting in difficulties in interpretation when such interpretation is attempted. For example, snorkel counts may need age estimates with them to be useful to interpret against redd surveys and having a hypothesis up front to guide the sampling will ensure that the data are being collected in the format needed to test a given hypothesis. The sponsors thus have more than a decade of data, but the interpretation remains a challenge. The project may benefit from assistance and collaboration with other scientists and specialists in the region with expertise in data management and model development.

In the response, insufficient information was also provided onhow management actions and habitat restoration will be evaluated.

The sponsors need to more appropriately frame their work and all future annual reporting into a bull trout life history framework, including hypotheses and how the data are to be used in hypothesis testing.

The ISRP recommends that the sponsors seek assistance with the data analysis and model development, using this long term and valuable bull trout data base, from Dr. John Skalski who is under contract to BPA or a scientist with similar expertise.

The ISRP also recommends that the sponsors develop a plan to assess bull trout response to habitat restoration and other management actions.

In addition, the ISRP suggests that the sponsors collaborate to a greater degree with other researchers in the Pacific Northwest, including academics and agencies. Such collaboration might include the development of their data sets for publication in refereed journals.

The ISRP requests a response to these issues:

1. Relationships to and coordination with other projects was not described. Please provide.
   
   
2. Methods for each of the objectives have not been described with adequate detail in either the proposal or in MonitoringMethods.org. Please provide. 
   
   
3. More effort needs to go into not just performing the monitoring tasks, but framing the tasks, in more meaningful scientific, management, and adaptive management contexts. What management actions will this inform? The sponsors need to take the next step and describe how the data could be applied and further explored. Hypothesis testing is in order. Some improvement in this area would help in this proposal, and especially in the reporting documents.
   
   
4. Describe the management actions that will be addressed given the several years of monitoring that have been well reported. How is habitat work expected to have an effect on bull trout? How might these proposed management actions be tested?

Address comments from the ISRP's previous review.

In a follow up to the 2007-09 ISRP review requesting a response, the sponsors provided mostly adequate responses to the ISRP questions. The proposal has dropped genetic evaluation of hybrids and PIT and radio-telemetry investigation of fish movement. The annual enumeration of bull trout adults and juveniles remains in the proposal, as well as testing the census model. In future proposal cycles, justification for annual census needs to be based on a statistical design and analysis, not just the bull trout recovery plan. The ISRP poses the question of how often must bull trout be sampled to obtain data for determining the trend in population abundance. No answer to this question has been received. 

Completion of the census model or permutation analysis is overdue, and testing of the model should have been completed by now. What is the status? The ISRP also asked if the model has been peer reviewed, but no response was provided.

While this project is listed as new in 2007, it has actually been ongoing since 1998 and by now status and trends of bull trout in this system should be understood. Application of project results for recovery actions should already be underway. It would therefore be essential for those proposing this work to frame the project in a broader context of bull trout ecology and management actions.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

This proposed project is involved with the collection of diverse life history and ecological data on bull trout from the Warm Springs River and Shitike Creek. The proposal provides information responsive to a number of regional plans including MERR Plan, the Deschutes River Subbasin Plan, the NPPC Research Plan (2006), the Accords, and the USFWS Bull Trout Recovery Plan. The role of this proposal in supporting information needs under those plans is clearly described. The technical background provided in the proposal gives adequate detail regarding the basic histories of bull trout on reservation lands and the region.

Each of the objectives, if achieved, will produce measurable results. The work proposed to be conducted is all relevant. All 11 objectives are important activities. However, the overall perspective on the goals, objectives, and hypotheses to be tested is lacking. All of these "objectives" other than No. 10 might more accurately be described as sampling “tasks” to be performed. With adequately designed protocols, many of them are typical, fairly routine fisheries work. The described results and indicated use of the data seldom go beyond basic monitoring, with the broader significance not discussed. Perhaps this site-specific data collection is the primary intent of the 20-year effort. However, the objectives of this study can be expanded to include acquisition of much more general knowledge and hypotheses testing for bull trout. Many opportunities exist in this work to test various hypotheses related to bull trout relevant to this site and other sites. As one example, the importance of groundwater to bull trout, mentioned in the text, may be framed into a hypothesis. Sampling could be designed to test hypotheses of interest to other bull trout investigators such as migratory patterns in relation to resource availability. It is intriguing that one population is adfluvial and another resident. The significance and reasons for the difference could be investigated and modeled, with the results leading to a useful publication on bull trout life histories. Similarly, the use of half-duplex tags and an additional screw trap are proposed without well-defined hypotheses identified. The proposal is therefore too focused on simply monitoring the situation as it changes, perhaps over concerns of deterioration, rather than conducting a scientific investigation. The monitoring work should be done, but it is also important to ask why the observed ecological situation for bull trout exists. The region could thereby gain basinwide applicable knowledge, including the potential for habitat improvements, effects of climate change, and reasons for observed life history patterns. The proposal appears to be written more as a handbook for technicians to implement, more than as a scientific proposal for scientists to conduct and learn from. By identifying higher-level objectives and hypotheses and collecting the data under the 11 current objectives, a more valuable outcome will result.

Some of the listed objectives could be combined under scientific objectives and hypotheses to be evaluated.

 2. History: Accomplishments, Results, and Adaptive Management (ISRP Review of Results)

After more than ten years of work, the sponsors do not provide accomplishments or results directly in the proposal but indicate that accomplishments and results may be found in two reports which are listed in documents. One report is called a retrospective and covers the period of 1998-2009. The other is just for one year (2009-2010). These are straightforward monitoring results with little or no interpretation or discussion. This lack of interpretation is consistent with the lack of higher level perspective and broader objectives noted above.

What have we learned of general and specific scientific value during this time that allows for more effective management of the fish or their habitat? How is it changing our approach from what it would have been in 1998? It would be good to describe results to date in this context. This part of the proposal is not developed in adequate detail.

Regarding adaptive management, little indication of its use is indicated other than a statement that no management actions have occurred regarding bull trout except that no fishing for bull trout can go on during steelhead and salmon fishing seasons. Has the Warm Springs Tribe started any actions regarding management, control, or eradication of brook trout? Brook trout certainly appear to be limiting and competing with bull trout in several places on the reservation, for example Mill Creek. This would be a possible adaptive management action. 

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (hatchery, RME, tagging)

No relationships with other projects are described. 

Climate change is briefly mentioned as an emerging limiting factor that they will track through their sampling. More thought needs to go into how results to date and planned work will address limiting factors. Some hypotheses would be useful to guide the sponsors’ thinking.

Responses to tagging questions were adequate.

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables/work elements are detailed in section 1 by objective. Most of the deliverables are data delivery that will need some serious scientific interpretation. It is not clarified if any interpretation and synthesis are part of this proposal.

 4a. Specific comments on protocols and methods described in MonitoringMethods.org

The methods described in MonitoringMehtods.org are incomplete, consisting mostly of general protocols. Some of the methods were not described beyond merely saying what would be done. It is unclear in some cases if the methodologies have been clearly worked out. Methods were listed, but it was indicated that they will be entered once they "receive a qualified rating from the ISRP." The sponsors need to provide methods in reasonable detail in a response before the ISRP can complete a review of the proposal.

Modified by Dal Marsters on 4/17/2012 12:47:15 PM.

Background
At the Council’s request, the ISRP reviewed a November 2011 response and revised proposal for the Confederated Tribes of the Warm Springs Reservation’s Fish Accord project, Monitoring Wild Populations of Spring Chinook Salmon (Oncorhynchus tshawytscha) and Summer Steelhead (O. mykiss) in Tributaries of the Lower Deschutes River within the Boundaries of the Confederated Tribes of the Warm Springs Reservation of Oregon (2008-311-00). This project was previously titled Natural Production Monitoring and Management.

This project is designed to monitor production of naturally produced spring Chinook salmon and steelhead in Reservation streams. Objectives include verification of species distribution, adult escapement, spawning surveys, juvenile outmigration and abundance and development of management strategies and goals. The ISRP reviewed earlier versions of this proposal: one in 2008, one in 2010, and two in 2011. See the most recent past review from May 2011 at ISRP 2011-13..

Recommendation
Meets Scientific Review Criteria (Qualified)
In general the project proponents provide sufficient justification for collecting and analyzing information on juvenile abundance to assess stream capacity, outmigration abundance, and migration timing for juveniles and adults. Data collected will also allow a better description of spring Chinook diversity, adult abundance, and pre-spawning mortality.

Although the ISRP does not need to review any additional responses, the project should address the following ISRP suggestions in development of a final statement of work and implementation of the project:

• Existing data presented in Appendix A deserve further analyses to assist development of the decision framework and proposed activities.

• The proponents should identify hatchery and natural adults in areas upstream of the smolt traps and incorporate downstream harvests of their fish into the recruitment analysis.

• Justification for sampling 50 juveniles of each species each week should be provided. Why is sampling 50 fish sufficient, but not excessive?

• Rationale for the target goal for estimating trap efficiency presented on page 40 is not provided. Justification is necessary, including why 5% precision in Table 7 is the target.

• The proponents should develop a set of criteria for establishing when improvements in juvenile outmigration can be clearly linked to habitat restoration efforts.

• Although snorkel surveys have been described in good detail and methodological concerns with redd surveys have been addressed, it would be helpful to establish visibility criteria based on turbidity measurements that would be used to determine when snorkeling and redd surveys would be suspended.

More details on these and additional suggestions are provided in the ISRP comments below.

Summary Comments
This proposal includes many details in support of the proposed activities and provides a fairly comprehensive narrative. The proposal’s eight listed objectives (although essentially tasks) identify reasonable activities in support of the project’s overall objectives that are described in narrative form. Inclusion of additional staff is a step in the right direction.

The project proponents have done a good job of describing field methods and techniques to assure quality control. The ISRP appreciates the details given for field crew training – something often lacking in other proposals. The sampling methods, for example the modified Hankin-Reeves snorkel surveys, have been carefully considered and are appropriate to the project’s setting and objectives. The proponents should identify hatchery and natural adults in areas upstream of the smolt traps and incorporate downstream harvests of their fish into the recruitment analysis.

The proponents have constructively used the ISRP comments, sought statistical advice, and modified sampling schemes to address the precision and bias of PIT tag assessments and sampling designs. Nevertheless, more information on the multiple regression analytical techniques proposed to evaluate the strength of fish abundance-habitat relationships would have been helpful, as well as a better description of how data would be archived and eventually made available to others involved in similar restoration projects.

The management application of the data is clearer in this iteration of the proposal. The information will be essential to the ongoing habitat restoration under the Warm Springs Fish Accord proposal #2008-301-00 (see ISRP 2011-27), for both assessment of effectiveness and for developing restoration strategies. Objective 8 provides a framework for assessing monitoring data to guide management and is a valuable component of the project indicating that project results will have an impact.

The appendices summarizing past data were very illuminating, but the data already collected deserve further analyses to assist development of the decision framework and proposed activities. The data and analyses presented in Appendix A suggest that a closer look and further analysis is warranted. For example:

1. It is not clear if the data in the Figures includes both wild and hatchery fish – this must be clarified, and a separate analysis applied to each.
2. Figure A7 on adult Chinook and juveniles should be re-analyzed as Beverton-Holt recruitment curves with a focus on regimes, for example stratified by PDO shifts. If the relationship still appears linear this suggests the system is under-seeded, that is, not at capacity.
3. Figure A8 appears non-linear as it should if Beverton-Holt recruitment applies. This figure suggests capacity is ~ 1000 adults. A re-analyses into regimes of productivity could be informative. Fig A9 is a function of the number of spawners, and appears to show adult returns in the regime shifts as ‘77 to ‘89, ‘90 to ‘99, 2000-2004. The same pattern may exist for outmigrants. These data may already inform an estimate of allowable harvest (see Ricker 1975 Appendices) as well as assist in defining the limiting life stage for both species.

 

ISRP General Comments
The questions below were asked in our earlier reviews and the proponents have responded, demonstrating progress in all iterations. Our comments regarding the most recent response are provided below after each question.

1. What management decisions will these data inform?

Management objectives have not been entirely clarified in this iteration of the proposal. The proponents explain the escapement goal for wild spring Chinook of 1,377 fish was derived by the USFWS (Appendix C) and further state there is no escapement goal for steelhead. The proposal states that it is current Tribal policy that wild steelhead will not be harvested, but that this policy could change if the overall health of the steelhead population reaches a point where some harvest could be sustained. Have numerical thresholds for population abundance been established which will allow for some Tribal catch of wild steelhead?

2. Will the data, including PIT-tag data, be sufficiently precise to adequately manage risk and provide confidence in decisions made? Evidence of data adequacy should be provided.

Appendix B provides some statistical rationale for number of PIT-tagged fish released. The data will be used for juvenile survival rates both within and outside the Deschutes subbasin and should be useful additions to databases on this topic.

It is not clear what the ultimate value is for the effort at qualitative documentation of the species assemblage in one pool and one riffle section in each of the five lower reaches in the Warm Springs River. The justification that this will, “allow a comparison of the distribution richness of assemblages of fishes and may be useful in detecting presence of non-native species” is not compelling.

Justification for sampling 50 juveniles of each species each week should be provided. Why is sampling 50 fish sufficient, but not excessive?
The target goal presented on page 40 for estimating trap efficiency is not clear. Justification is necessary, including why 5% precision in Table 7 is the target.

3. Will the GRTS-based sampling design be adequate given the physical constraints in the study area?

The proponents did an excellent job of describing how they arrived at a method for sampling in the canyons, and the ISRP is comfortable with the technique that was selected. The proposal states that sampling will occur from June to September, and quite likely this sampling window will experience a significant decline in streamflow over summer. Hopefully fish visibility will not change so much that early summer surveys underestimate juvenile abundance, but with the quality assurance controls in place the visual technique seems quite sound.

ISRP Comments and Recommendation Specific to Each Objective
For continuity of the review discussion across numerous reviews, organization of the objectives below is based on earlier proposals and responses.

Project Goal 1. Continue and improve annual life stage monitoring of wild spring Chinook salmon and steelhead in the Warm Springs River Basin and Shitike Creek.

Objective A) Juvenile Outmigration Monitoring
ISRP 2011-13: Response Requested

The ISRP encourages project proponents to develop a set of criteria for establishing when improvements in juvenile outmigration can be clearly linked to habitat restoration efforts. It will be important to develop a protocol to partition restoration effects from other factors such as cyclic weather changes (PDO regime shifts, El Niño/La Niña cycles) in order to measure restoration effectiveness.

PIT tagging and juvenile outmigration data will support life history and growth rate studies and include out-of-subbasin sampling. It is not entirely clear how the data will be used in cohort-run reconstruction for harvest management considerations.

Some details are missing in the length at age verification task. It is not clear how collection of scales will be randomized or why 50 fish of each species will be collected. The selected number of scales to be collected is not random; rather it is unknown until proportion of scale samples in each length group is known. It is not clear that the intense effort at age verification is warranted. Are the benefits worth the effort?

Objective B) Collect tissue samples for genetic analysis of O. mykiss in the Warm Springs River drainage
ISRP 2011-11: This objective was dropped from the project.

Objective C) Summer rearing snorkel surveys
ISRP 2011-13 - Response Requested

Snorkel and electrofishing surveys have been described in good detail. It would be helpful to establish visibility criteria based on turbidity measurements that would be used to determine when snorkeling surveys would be suspended.

Objective D) Spawning ground (redd) surveys
ISRP 2011-13: Response Requested

Methodological concerns were addressed, and a better description of the work was provided. Redd surveys have been expanded to the canyon reaches, and a method of comparing surveys in non-canyon reaches was presented. A method of comparing data from kayak and foot surveys was also developed. As with the snorkel surveys, it would be helpful to establish visibility criteria based on turbidity measurements that would be used to determine when redd surveys would be suspended.

Using a rotating panel design to identify redd distribution in multiple reaches is a good approach. Also, efforts at quality control of data collection are a positive feature of the redd enumeration effort.

Objective E) Enumerate adult escapement into Shitike Creek and the Warm Springs River
ISRP 2011-11: Meets Criteria

Objective F) Estimate harvest of Chinook salmon and Steelhead in the Deschutes Basin
ISRP 2011-11: Not Applicable

Project Goal 2. Provide management and co-management direction of the fisheries resources in the Deschutes River Basin

Objective A) Cooperate in Deschutes River Basin Fisheries Management Activities
ISRP 2011-11: Meets Criteria

Objective B) Provide co-management and assistance with fish handling at the Warm Springs National Fish Hatchery
ISRP 2011-11: Not Applicable

Modified by Dal Marsters on 4/30/2012 10:44:59 AM.

The proposal was insufficient for review. The title of the proposal is misleading in that the establishment of biologically based escapement goals is never addressed. Rather, the proposal is to improve the procedures for enumerating adult fall Chinook salmon in the Deschutes River. A more appropriate title would be something like “Develop methods to improve escapement estimates for Deschutes River fall Chinook.” If the project sponsors choose to focus the goal as implied by this revised title, then the ISRP recommends that the justification for the determination of genetic composition of spawners needs more detail and data as described below (Section F). The objective to examine the feasibility of installing a PIT tag detector/array in the lower Deschutes River is an excellent idea but also needs more details (e.g. design and size of array? specific potential location(s)? power availability? etc.) to be fully justified. If the project sponsors do wish to include the establishment of biologically based escapement goals, then they would need to significantly expand the proposal by adding two other objectives (with detailed study designs) to the proposal. One objective would be to examine potential hatchery fish effects of wild fall Chinook juveniles competing with hatchery and wild spring Chinook juveniles. Another objective would be to examine the carrying capacity of the river system (e.g. quantity and quality of available spawning habitat, food, rearing habitat, etc.) to estimate the potential population size of fall Chinook that may be supported.

The proposal is insufficient for technical review. The proposal identifies several monitoring, evaluation, and production topics for investigation of future tasks and actions. The present proposal does not present sufficient detail for an evaluation of the proposed work. The data obtained from the monitoring elements, and the derived metrics estimated from the data appear to be appropriate for management decisions but the explanation and justification for these tasks is not adequate. This appears to be a proposal to do a proposal by identifying feasibility studies. Basic details should be provided to better justify and explain the proposed approach and expected outcomes. The current level of description is inadequate to determine what is being proposed and why. The culture and release of fish for testing supplementation appear to be of sufficient scale to warrant a Three-Step Review, compliance with Northwest Power and Conservation Council Artificial Production Review policies, and would likely require an HGMP. A comment in the steelhead project 2008-311-00b is also germane to the spring Chinook. The need for projects such as this is clear due to the required BiOp mitigation for hydro losses by doing offsite actions in the tributaries. In fact the 2000 BiOp was partially invalidated because the offsite actions were not certain to occur. The Accord Agreements are designed to make the actions reasonably certain to occur.

The preliminary ISRP review requested that the sponsors clarify the basis for asserting that the Warm Springs River and Shitike Creek populations of bull trout warrant delineation as separate core areas; what was meant by "relative juvenile abundance and adult escapement indicate that Shitike Ck is robust while the Warm Springs R. population is less healthy than believed"; a better explanation of the analysis and purpose of the evaluation of bull and brook trout hybridization; and, the reasoning that more data is needed to complete the task of evaluating the census model for bull trout abundance.

The sponsors provided mostly adequate responses to the ISRP questions. The proposal has dropped genetic evaluation of hybrids and PIT and radio-telemetry investigation of fish movement. The annual enumeration of bull trout adults and juveniles remains in the proposal, as well as testing the census model. In future proposal cycles, justification for annual census needs to be based on statistical design of analysis, not just the bull trout recovery plan. The ISRP poses the question of how often must bull trout be sampled to obtain data for determining the trend in population abundance.

Completion of the census model is over-due, and testing of the model should be completed in this solicitation cycle. The ISRP also asked if the model has been peer reviewed, but no response was provided.

While this project is listed as new, it has actually been ongoing for several years and by now status and trends of bull trout in this system should be well understood. Application of project results for recovery actions should already be underway.

It would still be valuable to have those proposing this work frame the project in a broader context of bull trout ecology and management.