As part of this review, the ISRP also considered the proponents’ 2017-2019 Annual Report (Baker et al. 2020) and several earlier ISRP and ISRP/Independent Economic Analysis Board (IEAB) reviews. All earlier reviews were informative and gave the project high marks for providing fishery opportunities in the lower river. The ISRP/IEAB review (2007-3) was based on the project compilation covering 1994-2005 (North et al. 2006). The ISRP continues to believe that the SAFE project contributes significantly to lower-river fisheries while monitoring and considering upper river ESA-listed stocks. While we believe that the project meets most scientific criteria, the proposal itself was overly long and repetitious and did not follow the proposal guidance. For example, all the elements of SMART objectives are presented, but they are not clearly or explicitly stated.

In the next annual report and in a future synthesis report, the ISRP requests the proponents to provide information to address the following Conditions:

1. SMART objectives. Provide a single, consistently described set of SMART objectives (see proposal instructions) as part of the next annual report. 
   
   
2. Methods. In the next annual report, clearly describe methods that answer the four questions listed below in the Methods section of this review. 
   
   
3. Synthesis. By the end of this funding cycle, provide a synthesis report for ISRP review that evaluates accomplishments since the 2005 report (reviewed by the ISRP and IEAB in 2007) and includes:

• a cost/benefit analysis

• a clear description of how success of the project is determined

• an analysis of impacts to upper- and lower-river fish and fisheries, and

• a description of how the proponents detect small impacts on ESA-listed stocks, given the large number of fish released.

Q1: Clearly defined objectives and outcomes

The SAFE project receives funds from several state and local agencies as well as BPA to raise and release several salmon species (spring Chinook, tule fall Chinook, select area bright [SAB] fall Chinook, and coho) into off-channel hatchery and net pen sites in the lower Columbia River. Adult returns are intended to enhance non-Tribal commercial and recreational fisheries. The SAFE fisheries are monitored closely in-season to ensure minimal impacts on ESA-listed stocks. The project has been very successful in accomplishing these goals.

When the project was initiated in 1993 with a goal of determining the feasibility of the endeavor, there were nine objectives (see proposal pages 14-15). In 2011, the project objectives changed and are now stated as (proposal page 15):

• adaptively manage Select Area production and fisheries for increased benefit • monitor impact of Select Area fisheries

• monitor impact of Select Area production

• provide supplemental production for regional fisheries

• provide outlets for basin-wide reprogramming of hatchery production that reduces impacts.

These objectives are clearly stated but are not in the SMART format (see proposal instructions). Beginning on page 19, the proponents restate the objectives and provide extensive data and justification for the program based on the number of smolts released and the number, distribution and value of adults harvested. The proponents restate objectives in the proposal Goals and Objectives section. However, Figure 1 (perhaps 3-1, page 38) states four objectives that differ from earlier stated objectives, but these are approaching SMART objectives and subsequent text describes means of measuring success, how decisions are made (i.e., adaptive management), and the data collected. In the Methods section, the proponents provide numbers (measurable) and months (time-bound) for the various smolt releases. These data are also presented in the Timeline section (Table 7-1). The ISRP believes that all the elements for SMART objectives are presented in the proposal, but they are scattered and are not clearly and explicitly stated. A single, consistently described set of objectives is critical.

Q2: Methods

Most of the methods are based on sound science principles, are well documented in monitoringresources.org, and listed on page 44. The ISRP does, however, have some questions about the methods for the proponents to consider moving forward and when developing the synthesis report:

• How did the proponents determine that coded-wire tags in about 10% of the released fish was adequate? Was a power analysis conducted?

• More detail is required on how visual stock identification (VSI) is conducted.

• In various places in the proposal, the proponents briefly describe meetings with staff and public, but it is not clear how much of those meetings involve “outreach.” More description on how outreach/engagement with the public occurs and its importance to the program would be helpful.

• Given the fact that collecting and managing data seems to be one of the primary objectives of the project, what is the process for identifying questions that should be asked using the data? Is a formal mechanism available to ensure that the data collected are suitable for the questions to be answered?

Although it is not in the Methods section, the proponents want to explore past SARs in relation to as many response variables as can be documented. The description of variables, data and analysis sounds similar to what the Comparative Survival Study (CSS; BPA project 19960200) has been doing for many years to monitor Snake River and upper Columbia River stocks. The ISRP encourages the proponents to connect with that project. Much of the work needed to explore SARs may have already been done by the CSS Project. The proponents also mention hazing cormorants that threaten hatchery production. The proponents are collaborating with NOAA Fisheries to compare predation in the lower river to that of upriver stocks. The ISRP suggests that the collaboration examine whether or not the birds hazed away from SAFE hatcheries and net pens contribute to increased predation on ESA-listed smolts as they emigrate through the lower river.

Q3: Provisions for M&E

The project is well coordinated with the funders and cooperators. A flowchart for decision making is provided on page 48. The proponents provide an extensive listing of lessons learned as part of the Progress to Date section, and they also address potential confounding factors such as climate change and predation. Given the fact that the SARs of these fish do not seem to differ from those of upriver fish, the ISRP wonders why uncertainty in factors that affect ocean survival was not presented as a potential confounding factor to this project. The proponents also state that they have moved out of the project adjustment phase. The ISRP disagrees with the proponents’ assertion that project adjustment is a phase because project evaluation and adjustment should go on continuously.

Q4: Results – benefits to fish and wildlife

The project has clear benefits to fish and fisheries, as well as providing economic benefits to the communities in the lower Columbia River. The proponents have documented the costs of the project and the returns on the investment in providing fisheries that are regulated to protect ESA-listed species. However, how will the success of the project be evaluated? For example, is the project’s success based on the harvest opportunities provided, or the number and value of fish landed, or some measure of public satisfaction with the project as a whole? Or all of the above?

The proponents have provided a thorough and detailed response to each of the ISRP information requests.

Methodology is described in good detail, including a differentiation of the collection of data by other agencies (e.g., WDFW and ODFW) from the synthesis and summarization of those data by SAFE personnel. They also discuss how internal resources will be allocated away from field staff to project biologists to support coordination and data compilation and to assist with analysis and reporting. This approach reduces the potential for duplication of effort. Fishery sampling is described in detail by fishery, with citations provided for further information. Stream escapement sampling, which is largely conducted by other agencies, is sufficiently described, including issues related to the identification of hatchery fish.

A good description of how the project uses adaptive management is provided. Since project constraints (based on allowable impacts to ESA-listed stocks) change by year and within year, decisions are made on how the season will be structured and how harvest will be monitored. A flowchart is provided to show how project decision processes take account of and adapt to information within-season and between seasons. The process of monitoring impact rates on upriver spring Chinook is well described. The spring fisheries are differentiated from the fall fisheries in terms of difficulty and urgency of monitoring impact rates on non-local stocks.

The explanation regarding "efficient harvest" as a project deliverable states that the term "efficient" is relative and then describes how the term is used. However "efficient" is a term with technical meaning which is different from what the proponents describe. To avoid confusion it would be better to avoid "efficiency" altogether and use instead "harvest effectiveness" which is what they are describing.

To address the four questions regarding the effects of the SAFE program on harvests of local (natural origin) and non-local stocks, detailed tables are provided. These tables show some data gaps, which are recognized by the proponents.

The response adequately identifies problems with identifying local natural origin fish in harvest, acknowledges the importance of being able to do this, and states the intent to apply greater effort in determining these harvest rates. The response is similarly detailed in identifying problems with differentiating SAFE hatchery fish from other hatchery fish in order to estimate the proportion of SAFE fish on local spawning grounds. This problem arises because only a portion of the recovered hatchery strays (fin clip) have coded-wire tags. Exceptionally high levels of stray fall Chinook and coho salmon are observed in some watersheds, and this observation is recognized by the management agencies.

The response describes the Oregon and Washington recovery plans for salmon and SAFE's role in them. It notes the compromise between harvest opportunities and persistence of local populations, while identifying wild fish only areas upstream of hatchery weirs.

The response related to deliverables suggests that, as with other proposals, some confusion exists and therefore clarification is needed before the next round of proposals on how projects should list deliverables on the proposal form. There appear to be some difficulties with how Taurus structures objectives and deliverables leading to a hierarchy in the proposal that is confusing to reviewers. The explanation of the project's deliverables is adequate. The proponent notes that a project report will be prepared every three years and the latest report was uploaded to the web page one week ago.

The SAFE project provides an important approach for providing fishing opportunities in the lower Columbia River while attempting to minimize impacts on non-local stocks, including protected wild stocks. Nevertheless, the project should provide additional evidence that the fishery is not adversely affecting non-local and local natural-origin stocks. The ISRP recommends that the project prepare a comprehensive analysis of the project and a report at least every five years. The report should include a detailed project description, methods used to evaluate the project, project benefits, project costs, and project effects on natural-origin local and non-local stocks. Some key questions are listed below: 1. How many and what percentage of non-local stock populations are harvested and what is the stock composition of the non-local harvest? 2. How many local, natural-origin salmon are harvested? 3. What percentage of the local spawning escapement is represented by SAFE fish that escaped the fishery? 4. How will the SAFE project coexist with attempts to rebuild local natural origin fish? The proposal did not provide information on the methodology and the key monitoring questions noted above. The ISRP requests that the proponent provide a response with the following information: 1. Please describe the methodology that will be used to achieve each objective. • When monitoring the fishery, the methods should describe frequency of sampling, numbers of fish sampled, methods for stock identification, and methods for estimating catch of each stock. • Methodology used to sample streams for stray SAFE salmon should be documented. How many streams will be sampled, what area, and how frequent? How are stray stocks identified? • How does the program adaptively manage SAFE production and fisheries? How are adjustments made during the course of the season? Does the program have specific goals that it strives to achieve? 2. Please identify specific deliverables that are linked to each objective in the proposal (see comment below). 3. Please provide available information on the four key questions listed above for natural-origin local and non-local stocks associated with the SAFE fisheries. 1. Purpose, Significance to Regional Programs, Technical Background, and Objectives The proposal provides an adequate statement of purpose for the project, placing it well within the context of regional efforts to maintain commercial and recreational fisheries in the lower Columbia River while minimizing incidental impacts on listed ESUs. The significance of the project to regional programs is adequately described. The proposal lists four project goals: 1) mitigation of harvest opportunities lost through actions taken to recover listed ESUs, 2) protection of endangered species, 3) minimizing negative environmental impacts of SAFE hatcheries and rearing pens, and 4) minimizing the straying of hatchery fish to wild spawning grounds by maximizing harvest rates. The project has five objectives that are consistent with the specified goals: 1) adaptively manage select area production and fisheries, 2) monitor impact of select area fisheries, 3) monitor impact of select area production, 4) provide supplemental production for regional fisheries, and 5) provide outlet for basin-wide reprogramming of hatchery production. 2. History: Accomplishments, Results, and Adaptive Management A summary list of accomplishments and charts is provided. This information provided evidence that the project was addressing most of the goals and objectives. However, the ISRP identified additional information that should have been provided or discussed more thoroughly. For example, the proposal states that the SAFE fishery comprises 91% local stock for the winter, spring, and summer fisheries, and 87% Chinook and 80% coho local stock for the fall fishery. The project should attempt to identify non-local stocks that comprise the remaining 9%, 13%, and 20% in these fisheries and whether those percentages have a harmful impact on protected wild stocks. Did the local stocks harvested in the fishery include some natural-origin fish? Habitat restoration projects are underway in watersheds adjacent to SAFE fisheries (e.g., Young’s Bay) in order to recover depleted or extirpated local stocks; therefore, the project should evaluate how it might coexist with rebuilding of local natural-origin stocks. Migration timing might be one factor to consider. The proposal mentioned that some data on straying of SAFE fish to local spawning grounds have been collected, but the findings were not described. The ISRP encourages the SAFE project to collaborate with local ODFW and WDFW biologists in order to more accurately estimate numbers of SAFE fish straying to the local spawning areas. This is important to local stock rebuilding efforts because stocks used in the SAFE project are produced by using segregated hatchery practices and because the SAFE stocks may not be derived from nearby stocks. One performance element noted is the ex-vessel value of SAFE production. The proposal notes that ex-vessel value is a minimum indicator of economic value since it does not capture any multiplier effect. This is not fully correct. It is true that noting the value at point of first sale (ex-vessel value) does not account for local multiplier effects within the economy, but it is also the case that ex-vessel value represents gross revenues rather than net revenues (accounting for costs) and so overstate first-round benefits. An estimate of costs was given in the 2006 economic analysis, as was an analysis (based on predicted return rates, revenues and estimated costs) of economic impacts. Economic impacts were found to be positive for the two counties of the Astoria/Ilwaco area but less clear for the larger Oregon/Washington region. Earlier ISRP recommendations for employing a statistician for data analysis appear to still be relevant; a detailed statistical analysis of project outcomes or impacts was not included. A comprehensive analysis of the benefits, costs, and effects on natural-origin local and non-local stocks should be performed and reported at least every five years. The proposal presented some evidence of adaptive management, such as eliminating the use of stocks that had high stray rates. The proposal indicates the potential for increasing SAFE releases and harvest opportunities. If this occurs, will production from other hatcheries decrease to the same extent? 3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging) A long list of related projects is provided. An important emerging factor is the effort to recover locals stocks through habitat restoration projects in watersheds adjacent to SAFE fisheries. The SAFE project should more accurately estimate strays to spawning grounds (as proposed through new collaborations with ODFW and WDFW field crews), estimate harvest rates on local natural-origin salmon, and evaluate approaches to minimize harvests of these stocks. 4. Deliverables, Work Elements, Metrics, and Methods The same deliverable is listed for each of the five objectives. The deliverable is “efficient harvest of hatchery salmon while contributing to the recovery of listed stocks.” “Efficient” is not defined. This deliverable is not specific to any of the objectives, and its details are a brief history and justification for the project, not a description of deliverables and how they will be accomplished. Methods of the project are not described.

The very good response to the ISRP review was detailed and informative, clearly indicating areas where success, improvement and collaboration may be possible or desired. Opportunities for partnerships in tagging and estuarine and plume studies should continue to be encouraged and supported. At a future review, an independent monitoring and assessment would serve to further substantiate the positive claims in the response such as in North et al. (2006). The reporting of results has been commendable and informative in recent years, with signs of adaptive management.