The sponsors provided more adequate details of their sampling methods and protocols in the response (www.monitoringmethods.org). The best methods to be used may relate specifically to hypotheses developed. That is, the information needed to evaluate these hypotheses, for example, age structure of fish in snorkel counts. The data collection approach itself seems to be acceptable, but after more than 10 years of data collection, significant problems are arising in the interpretation and actual understanding of the data because limitations have arisen for the data that were not clearly foreseen, for example, resident versus migratory life histories and the need for age structured life history information in snorkel counts.

The key aspect of the proposal for which a response was requested but not adequately addressed was a clear development of hypotheses to guide the bull trout investigation. The sponsors noted that funding limitations and staffing issues due to a shortage of lead scientists have limited hypothesis development. The sponsors stated that “The lack of qualified personnel that could dedicate time to this project has effectively arrested development of the scientific understanding that should have been realized, which by now would have resulted in development of hypotheses to be tested.” They also suggest that future efforts will address hypothesis development, but no hypotheses are forwarded and no details are provided as to how hypothesis development will occur. The sponsors seem reticent to develop hypotheses until they have more data, although they did mention some potential hypotheses on pages 8 and 9 of their response. Many of their responses suggest difficulties in interpreting and understanding data that they collected in the past, suggesting a significant lack of staff continuity and loss of institutional memory regarding the details of the data collection. They did note, however, that some outside scientists would be consulted regarding analysis and interpretation of existing data. For example, the analysis of Budy “will indicate, given the current monitoring study design, what precision and with what power that declining trends in bull trout populations can be detected. ”

This lack of hypothesis development and testing has had consequences on the direction and focus of the project since 1998. The sponsors noted that "In September 2011, a report that reviewed and synthesized data from 1998 to 2009 was completed (CTWSRO Natural Resources Branch Fisheries Research Dept. 2011). Through this effort and preparation of this categorical review, problems that prevent thorough analyses and interpretation of data collected were realized."One of the main "problems" was the inability to distinguish resident from fluvial bull trout, confounding attempts to assess status of the two population segments. Evidently, even after more than 10 years of investigations, this issue of two main life history components was not fully recognized or addressed. In the sponsors’ words, “Apparently, an initial assumption of the original monitoring plan was that only fluvial bull trout were present in the study area. This is believed to be erroneous and will be addressed by using half-duplex PIT tag technology to determine home range of resident forms and migration timing and spatial patterns for fluvial forms.” This difficulty of identifying the fish in each life history type has clouded the interpretation of the time series collected over the past decade. The proposed work with half-duplex PIT tags is thus designed to address this limitation, although the details of how the life histories will be, as the sponsors state, “teased out” remains unclear.

In trying to understand the resident versus fluvial life history components, it may be useful to think about exactly what kinds of data need to be collected from fish besides PIT tag data, for example telemetry data, scale pattern analysis, reproductive periodicity data, to identify the life histories and how many fish are contributing to each pattern. It would seem that radio telemetry might be an effective method for addressing this issue. In addition, the relation between native bull trout and introduced brook trout is confusing. As the sponsors state, “brook trout are sympatric with bull trout in index reaches therefore, redds from brook trout and resident bull trout may be indistinguishable. ” Other issues regarding interpretation, for example the data depicted in Figures 3-5 in the response, seem to be a result of not clearly having hypotheses to guide the exact sampling methods, resulting in difficulties in interpretation when such interpretation is attempted. For example, snorkel counts may need age estimates with them to be useful to interpret against redd surveys and having a hypothesis up front to guide the sampling will ensure that the data are being collected in the format needed to test a given hypothesis. The sponsors thus have more than a decade of data, but the interpretation remains a challenge. The project may benefit from assistance and collaboration with other scientists and specialists in the region with expertise in data management and model development.

In the response, insufficient information was also provided onhow management actions and habitat restoration will be evaluated.

The sponsors need to more appropriately frame their work and all future annual reporting into a bull trout life history framework, including hypotheses and how the data are to be used in hypothesis testing.

The ISRP recommends that the sponsors seek assistance with the data analysis and model development, using this long term and valuable bull trout data base, from Dr. John Skalski who is under contract to BPA or a scientist with similar expertise.

The ISRP also recommends that the sponsors develop a plan to assess bull trout response to habitat restoration and other management actions.

In addition, the ISRP suggests that the sponsors collaborate to a greater degree with other researchers in the Pacific Northwest, including academics and agencies. Such collaboration might include the development of their data sets for publication in refereed journals.

The ISRP requests a response to these issues:

1. Relationships to and coordination with other projects was not described. Please provide.
   
   
2. Methods for each of the objectives have not been described with adequate detail in either the proposal or in MonitoringMethods.org. Please provide. 
   
   
3. More effort needs to go into not just performing the monitoring tasks, but framing the tasks, in more meaningful scientific, management, and adaptive management contexts. What management actions will this inform? The sponsors need to take the next step and describe how the data could be applied and further explored. Hypothesis testing is in order. Some improvement in this area would help in this proposal, and especially in the reporting documents.
   
   
4. Describe the management actions that will be addressed given the several years of monitoring that have been well reported. How is habitat work expected to have an effect on bull trout? How might these proposed management actions be tested?

Address comments from the ISRP's previous review.

In a follow up to the 2007-09 ISRP review requesting a response, the sponsors provided mostly adequate responses to the ISRP questions. The proposal has dropped genetic evaluation of hybrids and PIT and radio-telemetry investigation of fish movement. The annual enumeration of bull trout adults and juveniles remains in the proposal, as well as testing the census model. In future proposal cycles, justification for annual census needs to be based on a statistical design and analysis, not just the bull trout recovery plan. The ISRP poses the question of how often must bull trout be sampled to obtain data for determining the trend in population abundance. No answer to this question has been received. 

Completion of the census model or permutation analysis is overdue, and testing of the model should have been completed by now. What is the status? The ISRP also asked if the model has been peer reviewed, but no response was provided.

While this project is listed as new in 2007, it has actually been ongoing since 1998 and by now status and trends of bull trout in this system should be understood. Application of project results for recovery actions should already be underway. It would therefore be essential for those proposing this work to frame the project in a broader context of bull trout ecology and management actions.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

This proposed project is involved with the collection of diverse life history and ecological data on bull trout from the Warm Springs River and Shitike Creek. The proposal provides information responsive to a number of regional plans including MERR Plan, the Deschutes River Subbasin Plan, the NPPC Research Plan (2006), the Accords, and the USFWS Bull Trout Recovery Plan. The role of this proposal in supporting information needs under those plans is clearly described. The technical background provided in the proposal gives adequate detail regarding the basic histories of bull trout on reservation lands and the region.

Each of the objectives, if achieved, will produce measurable results. The work proposed to be conducted is all relevant. All 11 objectives are important activities. However, the overall perspective on the goals, objectives, and hypotheses to be tested is lacking. All of these "objectives" other than No. 10 might more accurately be described as sampling “tasks” to be performed. With adequately designed protocols, many of them are typical, fairly routine fisheries work. The described results and indicated use of the data seldom go beyond basic monitoring, with the broader significance not discussed. Perhaps this site-specific data collection is the primary intent of the 20-year effort. However, the objectives of this study can be expanded to include acquisition of much more general knowledge and hypotheses testing for bull trout. Many opportunities exist in this work to test various hypotheses related to bull trout relevant to this site and other sites. As one example, the importance of groundwater to bull trout, mentioned in the text, may be framed into a hypothesis. Sampling could be designed to test hypotheses of interest to other bull trout investigators such as migratory patterns in relation to resource availability. It is intriguing that one population is adfluvial and another resident. The significance and reasons for the difference could be investigated and modeled, with the results leading to a useful publication on bull trout life histories. Similarly, the use of half-duplex tags and an additional screw trap are proposed without well-defined hypotheses identified. The proposal is therefore too focused on simply monitoring the situation as it changes, perhaps over concerns of deterioration, rather than conducting a scientific investigation. The monitoring work should be done, but it is also important to ask why the observed ecological situation for bull trout exists. The region could thereby gain basinwide applicable knowledge, including the potential for habitat improvements, effects of climate change, and reasons for observed life history patterns. The proposal appears to be written more as a handbook for technicians to implement, more than as a scientific proposal for scientists to conduct and learn from. By identifying higher-level objectives and hypotheses and collecting the data under the 11 current objectives, a more valuable outcome will result.

Some of the listed objectives could be combined under scientific objectives and hypotheses to be evaluated.

 2. History: Accomplishments, Results, and Adaptive Management (ISRP Review of Results)

After more than ten years of work, the sponsors do not provide accomplishments or results directly in the proposal but indicate that accomplishments and results may be found in two reports which are listed in documents. One report is called a retrospective and covers the period of 1998-2009. The other is just for one year (2009-2010). These are straightforward monitoring results with little or no interpretation or discussion. This lack of interpretation is consistent with the lack of higher level perspective and broader objectives noted above.

What have we learned of general and specific scientific value during this time that allows for more effective management of the fish or their habitat? How is it changing our approach from what it would have been in 1998? It would be good to describe results to date in this context. This part of the proposal is not developed in adequate detail.

Regarding adaptive management, little indication of its use is indicated other than a statement that no management actions have occurred regarding bull trout except that no fishing for bull trout can go on during steelhead and salmon fishing seasons. Has the Warm Springs Tribe started any actions regarding management, control, or eradication of brook trout? Brook trout certainly appear to be limiting and competing with bull trout in several places on the reservation, for example Mill Creek. This would be a possible adaptive management action. 

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (hatchery, RME, tagging)

No relationships with other projects are described. 

Climate change is briefly mentioned as an emerging limiting factor that they will track through their sampling. More thought needs to go into how results to date and planned work will address limiting factors. Some hypotheses would be useful to guide the sponsors’ thinking.

Responses to tagging questions were adequate.

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables/work elements are detailed in section 1 by objective. Most of the deliverables are data delivery that will need some serious scientific interpretation. It is not clarified if any interpretation and synthesis are part of this proposal.

 4a. Specific comments on protocols and methods described in MonitoringMethods.org

The methods described in MonitoringMehtods.org are incomplete, consisting mostly of general protocols. Some of the methods were not described beyond merely saying what would be done. It is unclear in some cases if the methodologies have been clearly worked out. Methods were listed, but it was indicated that they will be entered once they "receive a qualified rating from the ISRP." The sponsors need to provide methods in reasonable detail in a response before the ISRP can complete a review of the proposal.

Modified by Dal Marsters on 4/17/2012 12:47:15 PM.

The preliminary ISRP review requested that the sponsors clarify the basis for asserting that the Warm Springs River and Shitike Creek populations of bull trout warrant delineation as separate core areas; what was meant by "relative juvenile abundance and adult escapement indicate that Shitike Ck is robust while the Warm Springs R. population is less healthy than believed"; a better explanation of the analysis and purpose of the evaluation of bull and brook trout hybridization; and, the reasoning that more data is needed to complete the task of evaluating the census model for bull trout abundance.

The sponsors provided mostly adequate responses to the ISRP questions. The proposal has dropped genetic evaluation of hybrids and PIT and radio-telemetry investigation of fish movement. The annual enumeration of bull trout adults and juveniles remains in the proposal, as well as testing the census model. In future proposal cycles, justification for annual census needs to be based on statistical design of analysis, not just the bull trout recovery plan. The ISRP poses the question of how often must bull trout be sampled to obtain data for determining the trend in population abundance.

Completion of the census model is over-due, and testing of the model should be completed in this solicitation cycle. The ISRP also asked if the model has been peer reviewed, but no response was provided.

While this project is listed as new, it has actually been ongoing for several years and by now status and trends of bull trout in this system should be well understood. Application of project results for recovery actions should already be underway.

It would still be valuable to have those proposing this work frame the project in a broader context of bull trout ecology and management.