Columbia Basin Fish and Wildlife Program Columbia Basin Fish and Wildlife Program
RSS Feed for updates to Project 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP) Follow this via RSS feed. Help setting up RSS feeds?

Project Summary

Project 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP)
Project Number:
2003-017-00
Title:
Integrated Status and Effectiveness Monitoring Program (ISEMP)
Summary:
This project seeks to develop two novel monitoring and evaluation programs: (i) subbasin-scale pilot status and trend monitoring efforts for anadromous salmonids and their habitat in the Wenatchee, John Day and Salmon River basins, and (ii) effectiveness monitoring for suites of habitat restoration projects in selected watersheds within the three target subbasins. This work builds on current status and trend monitoring programs within each of these basins; however, the proposed work differs structurally from much of the ongoing status and trend monitoring work as it focuses on the explicit development and testing of the sampling protocols and methodologies required for generating habitat and population monitoring data of known spatio-temporal resolution, accuracy and precision. In addition, the proposed work expands on the utility of status monitoring data to explicitly address watershed-scale questions of habitat restoration action effectiveness.

(i) The status and trend monitoring program for anadromous salmonids and habitat in the Wenatchee, John Day and Salmon River basins will serve three major data collection efforts:
--At the scale of the subbasin, assess on an annual basis the status of adult populations of anadromous salmonids.
--At the scale of the subbasin, assess on an annual basis the population status or productivity of juvenile anadromous salmonids.
--At the scale of the subbasin, assess on an annual basis the status of salmonid habitat.

Data from the status and trend monitoring program will be used for a variety of resource management purposes. The primary utility of the information will be the annual assessment of status and resulting trend over time for these fishes and their habitat. However, this program will also support restoration action planning and assessment by serving as the baseline information used for action siting, and the baseline against which actions' biological impact could be measured.

(ii) The effectiveness monitoring program for assessing the watershed-scale impact of restoration actions in selected portion of the Wenatchee, John Day and Salmon River basins will explicitly address questions of action efficacy:
--At the scale of a watershed, what is the biological benefit to anadromous salmonid populations of the implementation of ongoing habitat restoration actions?
--Within and between target watershed in a single subbasin, what is the distribution of physical/environmental habitat condition as a function of ongoing habitat restoration actions?

While the genesis of this project was initially strictly status and trend monitoring of populations and habitat condition, a natural extension of these data collection programs is a watershed scale assessment of habitat action efficacy. Habitat restoration actions are generally implemented on a reach or habitat unit scale and can be assessed for effectiveness at that scale. However, when needing to determine the population level response to restoration actions, the actions' cumulative impact must be assessed on the scale of the demographic unit as a whole. At this scale, determining the effect of multiple simultaneous actions is more an issue of differences in population growth rates (alternatively stage specific survivals, or productivity expressed as juveniles per adult) than an elucidation of the mechanism by which a particular action or class of actions alters the population processes of these fishes. Therefore, assessments of watershed scale population trajectories so closely resembles status monitoring that their combination is a natural pairing.

Finally, in order to efficiently and effectively assess the status, trends and effects of habitat management actions, a process to manage the flow of data and information from data generation to data analysis is needed. To meet this need, this project will develop a framework of tools that assist in data management at the scale of the data generators while also standardizing the form and communication of data sets to a regional data management system. The data management tools and products developed will be designed to integrate with on going regional efforts to standardize protocols and meta-data and develop distributed data management systems.
Proposer:
None
Proponent Orgs:
National Oceanic and Atmospheric Administration (Govt - Federal)
Starting FY:
2003
Ending FY:
2019
BPA PM:
Stage:
Implementation - Project Status Report
Area:
Province Subbasin %
Basinwide - 100.00%
Purpose:
Programmatic
Emphasis:
RM and E
Focal Species:
All Anadromous Fish
All Anadromous Salmonids
Bass, Smallmouth
Burbot
Carp, Common
Catfish
Chinook - All Populations
Chinook - Deschutes River Summer/Fall ESU
Chinook - Lower Columbia River ESU
Chinook - Mid-Columbia River Spring ESU
Chinook - Snake River Fall ESU
Chinook - Snake River Spring/Summer
Chinook - Snake River Spring/Summer ESU
Chinook - Upper Columbia River Spring ESU
Chinook - Upper Columbia River Summer/Fall ESU
Chinook - Upper Willamette River ESU
Coho - Lower Columbia River ESU
Coho - Unspecified Population
Crappie, Black
Crappie, White
Cutthroat Trout, Coastal - All Anadromous Populations
Cutthroat Trout, Coastal - Resident Populations
Cutthroat Trout, Coastal - Southwest Washington/Columbia River ESU
Cutthroat Trout, Westslope
Freshwater Mussels
Kokanee
Lamprey, Pacific
Other Resident
Perch, Yellow
Pike, Northern
Pikeminnow, Northern
Sockeye - All Populations
Sockeye - Lake Wenatchee ESU
Sockeye - Snake River ESU
Steelhead - All Populations
Steelhead - Lower Columbia River DPS
Steelhead - Middle Columbia River DPS
Steelhead - Snake River DPS
Steelhead - Upper Columbia River DPS
Steelhead - Upper Willamette River DPS
Trout, Brook
Trout, Brown
Trout, Bull
Trout, Interior Redband
Trout, Lake
Trout, Rainbow
Whitefish, Mountain
Species Benefit:
Anadromous: 100.0%   Resident: 0.0%   Wildlife: 0.0%
Special:
None

Steelhead spawning distribution in the Nason Creek Basin in 2011.

Figure Name: Figure 1

Document ID: P125379

Document: The Integrated Status and Effectiveness Monitoring Program; 3/11 - 2/12

Page Number: 17

Project: 2003-017-00

Contract: 52152

Steelhead spawning distribution in the Chiwawa River Basin in 2011.

Figure Name: Figure 2

Document ID: P125379

Document: The Integrated Status and Effectiveness Monitoring Program; 3/11 - 2/12

Page Number: 18

Project: 2003-017-00

Contract: 52152

Steelhead spawning distribution in the Wenatchee River and Icicle Creek in 2011.

Figure Name: Figure 3

Document ID: P125379

Document: The Integrated Status and Effectiveness Monitoring Program; 3/11 - 2/12

Page Number: 19

Project: 2003-017-00

Contract: 52152

Steelhead spawning distribution in the Peshastin Creek Basin in 2011.

Figure Name: Figure 4

Document ID: P125379

Document: The Integrated Status and Effectiveness Monitoring Program; 3/11 - 2/12

Page Number: 20

Project: 2003-017-00

Contract: 52152


Summary of Budgets

To view all expenditures for all fiscal years, click "Project Exp. by FY"

Expense SOY Budget Working Budget Contracted Amount Modified Contract Amount Expenditures *
FY2018 (Previous) $0 $1,209,413 $1,052,964 $2,229,978

BiOp FCRPS 2008 (non-Accord) $0 $1,209,413 $1,052,964 $2,229,978
FY2019 (Current) $0 $0 $156,449 $46,036

BiOp FCRPS 2008 (non-Accord) $0 $0 $156,449 $46,036
FY2020 (Next) $0 $0 $0 $0

* Expenditures data includes accruals and are based on data through 31-Oct-2018

Decided Budget Transfers  (FY2018 - FY2020)

Acct FY Acct Type Amount Fund Budget Decision Date
FY2019 Expense $0 From: BiOp FCRPS 2008 (non-Accord) Nov 30th SOY Transfers 12/07/2018

Pending Budget Decision?  No


Actual Project Cost Share

Current Fiscal Year — 2019
Cost Share Partner Total Proposed Contribution Total Confirmed Contribution
There are no project cost share contributions to show.
Previous Fiscal Years
Fiscal Year Total Contributions % of Budget
2016 $1,955,000 (Draft) 28 % (Draft)
2015 $2,145,913 30 %
2014 $2,009,388 29 %
2013 $2,435,219 33 %
2012 $2,671,108 31 %
2011 $2,789,698 30 %
2010
2009 $2,642,481 37 %
2008 $2,523,142 39 %
2007 $2,557,908 54 %

Contracts

The table below contains contracts with the following statuses: Active, Complete, History, Issued.
Expense Contracts:
Number Contractor Name Title Status Contracted Amount Dates
BPA-003092 Bonneville Power Administration PIT Tags - ISEMP Active $61,500 10/1/2005 - 9/30/2006
BPA-003090 Bonneville Power Administration PIT Tags - ISEMP Active $98,803 10/1/2006 - 9/30/2007
BPA-003434 Bonneville Power Administration Pit Tags - Integrated Status/Effect Progr Active $108,987 10/1/2007 - 9/30/2008
BPA-003989 Bonneville Power Administration PIT Tags - Integrated Status/Effect Progr Active $200,665 10/1/2008 - 9/30/2009
BPA-004994 Bonneville Power Administration PIT Tags - ISEMP Active $146,071 10/1/2009 - 9/30/2010
BPA-005654 Bonneville Power Administration PIT Tags - ISEMP Active $255,648 10/1/2010 - 9/30/2011
BPA-006388 Bonneville Power Administration PIT Tags - Integrated Status/Effect Program Active $173,550 10/1/2011 - 9/30/2012
BPA-007052 Bonneville Power Administration PIT Tags - ISEMP Active $109,070 10/1/2012 - 9/30/2013
BPA-007736 Bonneville Power Administration PIT Tags - ISEMP Active $147,149 10/1/2013 - 9/30/2014
BPA-008398 Bonneville Power Administration PIT Tags - ISEMP Active $166,593 10/1/2014 - 9/30/2015
BPA-008817 Bonneville Power Administration PIT Tags - ISEMP Active $93,391 10/1/2015 - 9/30/2016
BPA-009574 Bonneville Power Administration PIT Tags - ISEMP Active $69,645 10/1/2016 - 9/30/2017
75098 SOW US Fish and Wildlife Service (USFWS) 2003-017-00 EXP ISEMP - USFWS ENTIAT R FISHERIES EFFECT M&E Issued $373,870 2/1/2017 - 1/31/2018
46273 REL 134 SOW National Oceanic and Atmospheric Administration 2003-017-00 EXP ISEMP NOAA FISHERIES COORDINATION - PROJ DESIGN Issued $152,391 7/1/2017 - 6/30/2018
76143 SOW Terraqua, Inc. 2003-017-00 EXP ISEMP - TERRAQUA COORDINATION - UC & PROGRAMMATIC Issued $651,364 7/1/2017 - 6/30/2018
77840 SOW Biomark, Inc. 2003-017-00 EXP ISEMP - SALMON R. QUANTITATIVE CONSULTANTS Issued $782,246 12/1/2017 - 2/28/2019
78217 SOW US Fish and Wildlife Service (USFWS) 2003-017-00 EXP ISEMP - USFWS ENTIAT R FISHERIES EFFECT M&E Issued $275,000 2/1/2018 - 1/31/2019
78367 SOW Cascadia Conservation District 2003-017-00 EXP ISEMP - CASCADIA CONS DIST/COORD Issued $5,000 3/1/2018 - 2/28/2019



Annual Progress Reports
Expected (since FY2004):190
Completed:167
On time:164
Status Reports
Completed:680
On time:240
Avg Days Late:25

Earliest Subsequent           Accepted Count of Contract Deliverables
Contract Contract(s) Title Contractor Start End Status Reports Complete Green Yellow Red Total % Green and Complete Canceled
14880 REL 1 14880 REL 2, 14880 REL 4, 27480 REL 1, 27480 REL 2, 27480 REL 3, 27480 REL 4, 27480 REL 5, 27480 REL 6, 57863, 61818, 65674, 69875, 73971, 76143 2003-017-00 COORDINATION FOR WENATCHEE EFFECTIVENESS MONITORING Terraqua, Inc. 07/2003 07/2003 Issued 95 160 0 0 15 175 91.43% 0
16394 25153, 30032, 35962, 41715, 46273 REL 8, 46273 REL 28, 46273 REL 48, 46273 REL 69, 46273 REL 88, 46273 REL 104, 46273 REL 119, 46273 REL 134 2003-017-00 INTEGRATED STATUS & EFFECTIVENESS MONITORING National Oceanic and Atmospheric Administration 10/2003 10/2003 Issued 53 151 1 0 18 170 89.41% 2
17492 26211, 31579, 37149, 41811, 47099 2003-017-00 INTEGRATED STATUS & EFFECTIVENESS MONITORING PROGRAM US Forest Service (USFS) 03/2004 03/2004 Closed 24 35 0 0 0 35 100.00% 0
17856 2003 017 00 INTEGRATED STATUS & EFFECTIVENESS MONITORING US Forest Service (USFS) 05/2004 05/2004 Closed 2 6 0 0 1 7 85.71% 0
17904 PI 2003-017-00 FSL-HEADWATER NUTRIENTS University of Alaska Fairbanks 06/2004 06/2004 Closed 1 6 0 0 1 7 85.71% 0
18223 26351, 32097, 36719, 41723, 46464, 52089, 56392, 60348, 64301, 68300, 71785, 75288, 78367 2003-017-00 INTEGRATED STATUS & EFFECTIVENESS MONITORING Cascadia Conservation District 06/2004 06/2004 Issued 64 75 2 0 3 80 96.25% 0
19370 21174, 27178, 33338, 38410, 43058, 45992, 51637, 56338, 56662 REL 13, 56662 REL 48 2003-017-00 EXPANSION OF EXISTING SMOLT TRAPPING PROG IN NASON CR Yakama Confederated Tribes 09/2004 09/2004 Closed 54 58 0 0 3 61 95.08% 0
20997 REL 1 PI 2003-017-00 ECO LOGIC REL. 1: JOHN DAY PLANNING Eco Logical Research 02/2005 02/2005 Closed 3 4 1 0 0 5 100.00% 0
21698 26285, 31661, 37080, 41349, 46344, 52152, 56540, 60621, 64997, 68619, 71851, 75383 2003-017-00 EXPAND SMOLT TRAPPING & STEELHEAD SPAWNER SURVEYS Washington Department of Fish and Wildlife (WDFW) 03/2005 03/2005 Closed 56 65 0 0 0 65 100.00% 0
22456 200301700 CHAR CHANNEL & RIPARIAN HABITAT QUALITY - WENATCHEE SUB Washington Department of Ecology 03/2005 03/2005 Closed 7 3 1 0 0 4 100.00% 0
14880 REL 3 200301700 EXP UPPER COL STRATEGY PNAMP SIDE-BY-SIDE PROTOCOL TEST Terraqua, Inc. 07/2005 07/2005 Closed 10 4 1 0 0 5 100.00% 0
23744 200301700 EXP INTEGRATED STATUS & EFFECTIVENESS MONITORING ENTIAT US Fish and Wildlife Service (USFWS) 07/2005 07/2005 Closed 4 4 0 0 0 4 100.00% 0
14744 REL 3 200301700 EXP WENATCHEE RM&E PILOT - BIOANALYSTS, INC. Bioanalysts, Inc. 09/2005 09/2005 Closed 5 6 0 0 0 6 100.00% 0
BPA-003092 PIT Tags - ISEMP Bonneville Power Administration 10/2005 10/2005 Active 0 0 0 0 0 0 0
24727 200301700 EXP WENATCHEE RM&E PILOT - UNIV OF ALASKA, FAIRBANKS University of Alaska Fairbanks 10/2005 10/2005 Closed 4 6 0 0 0 6 100.00% 1
24725 32338, 37005, 41806, 47100, 52983, 57497 2003 017 00 WENATCHEE R M & E PILOT US Forest Service (USFS) 10/2005 10/2005 Closed 31 48 0 0 3 51 94.12% 0
26434 31865, 36839, 41578, 46221 200301700 CHAR CHANNEL & RIPARIAN HABITAT QUAL - WENATCHEE WDOE Washington Department of Ecology 03/2006 03/2006 Closed 28 19 0 0 1 20 95.00% 0
28029 33329, 36687, 41045, 45993, 51487, 56336, 59870, 64187, 67971, 71541, 75098, 78217 200301700 EXP INTEGRATED STATUS/EFFECTIVE MONITORING - USFWS US Fish and Wildlife Service (USFWS) 07/2006 07/2006 Issued 52 105 1 0 4 110 96.36% 5
BPA-003090 PIT Tags - ISEMP Bonneville Power Administration 10/2006 10/2006 Active 0 0 0 0 0 0 0
32166 200301700 EXP WENATCHEE RM&E PILOT - UNIV OF ALASKA, FAIRBANKS University of Alaska Fairbanks 04/2007 04/2007 Closed 4 7 0 0 0 7 100.00% 0
34059 2003-017-00 EXP ISEMP - UPPER COL SALMON RECOVERY BOARD DATA MGT Upper Columbia Salmon Recovery Board 08/2007 08/2007 Closed 4 1 0 0 3 4 25.00% 0
BPA-003434 Pit Tags - Integrated Status/Effect Progr Bonneville Power Administration 10/2007 10/2007 Active 0 0 0 0 0 0 0
40060 47762 2003-017-00 EXP ISEMP-USFS RMRS/BOISE SALMON R LIDAR US Forest Service (USFS) 09/2008 09/2008 Closed 16 6 0 0 4 10 60.00% 0
39498 48304, 53350 2003-017-00 EXP ISEMP BIO MARK SALMON R PIT ANT/INSTAL Biomark, Inc. 09/2008 09/2008 Closed 19 25 0 0 0 25 100.00% 0
BPA-003989 PIT Tags - Integrated Status/Effect Progr Bonneville Power Administration 10/2008 10/2008 Active 0 0 0 0 0 0 0
40672 45568, 50733 2003-017-00 EXP ISEMP-ENVIRONMENTAL DATA SVCS Environmental Data Services 12/2008 12/2008 Closed 12 31 0 0 0 31 100.00% 1
40673 45546, 50900, 55969, 59513, 64152, 67443, 71132, 74634 2003-017-00 EXP ISEMP-VOLK CONSULTING South Fork Research, Inc. 12/2008 12/2008 Closed 36 148 0 0 5 153 96.73% 0
40950 45428, 50585, 55487, 59729, 64146, 67442, 71131, 74633, 77840 2003-017-00 EXP ISEMP - SALMON RIVER - QUANTITATIVE CONSULTANTS Biomark, Inc. 12/2008 12/2008 Issued 39 257 15 0 9 281 96.80% 2
41151 45470, 50685, 55970, 59730, 64527, 67511, 71203, 74647 2003-017-00 EXP ISEMP ECO LOGICAL RES - JOHN DAY Eco Logical Research 12/2008 12/2008 Closed 36 141 0 0 12 153 92.16% 0
BPA-004994 PIT Tags - ISEMP Bonneville Power Administration 10/2009 10/2009 Active 0 0 0 0 0 0 0
BPA-005654 PIT Tags - ISEMP Bonneville Power Administration 10/2010 10/2010 Active 0 0 0 0 0 0 0
BPA-006388 PIT Tags - Integrated Status/Effect Program Bonneville Power Administration 10/2011 10/2011 Active 0 0 0 0 0 0 0
56399 60276, 64580 2003-017-00 EXP ISEMP - USFS/ ENTIAT RS - I M W SEDIMENT MONITOR US Forest Service (USFS) 03/2012 03/2012 Closed 12 13 0 0 0 13 100.00% 0
BPA-007052 PIT Tags - ISEMP Bonneville Power Administration 10/2012 10/2012 Active 0 0 0 0 0 0 0
BPA-007736 PIT Tags - ISEMP Bonneville Power Administration 10/2013 10/2013 Active 0 0 0 0 0 0 0
BPA-008398 PIT Tags - ISEMP Bonneville Power Administration 10/2014 10/2014 Active 0 0 0 0 0 0 0
69146 2003-017-00 EXP TERRAQUA ENTIAT SCREW TRAP OPERATION Terraqua, Inc. 04/2015 04/2015 Closed 5 5 0 0 0 5 100.00% 0
BPA-008817 PIT Tags - ISEMP Bonneville Power Administration 10/2015 10/2015 Active 0 0 0 0 0 0 0
BPA-009574 PIT Tags - ISEMP Bonneville Power Administration 10/2016 10/2016 Active 0 0 0 0 0 0 0
76140 2003-017-00 EXP CHAMP DATA ASSESSMENT AND ACQUISITION - PHASE I HDR Engineering, Inc. 05/2017 05/2017 Closed 0 0 0 0 0 0 0
74314 REL 22 2003-017-00 EXP QRF WINTER HABITAT - WDFW Washington Department of Fish and Wildlife (WDFW) 11/2017 11/2017 Closed 3 7 0 0 0 7 100.00% 0
Project Totals 679 1396 22 0 82 1500 94.53% 11


The table content is updated frequently and thus contains more recent information than what was in the original proposal reviewed by ISRP and Council.

Review: 2018 Research Project Status Review

Independent Scientific Review Panel Assessment

Assessment Number: 2003-017-00-ISRP-20181115
Project: 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP)
Review: 2018 Research Project Status Review
Completed Date: 11/15/2018
Final Round ISRP Date: 9/28/2018
Final Round ISRP Rating: Not Applicable
Final Round ISRP Comment:

1. Objectives


The integrated report states that the single objective for both programs was to develop RME methods to assess status and trends, restoration opportunities, and the benefits of habitat restoration actions on salmon and steelhead and their habitat in the tributary environment. The larger spatial framework of these two programs is a critical context for evaluating whether they were scientifically sound. “The goal of ISEMP and CHaMP was to support the development of more realistic, spatially explicit, tributary habitat restoration scenarios and to quantify the degree to which these restoration actions might benefit salmonid populations in the interior Columbia River Basin." In addition, the integrated report explicitly states eight major questions the two programs were designed to answer. The objectives, methods, and analytical techniques are adequately described.


The two projects were terminated in late 2017 and early 2018.

2. Methods


The methods used in the ISEMP and CHaMP programs are thoroughly documented, and the ISRP evaluated the projects’ scientific soundness in multiple review processes from their beginning through 2013.1 In addition, the ISAB and ISRP evaluated the projects’ contributions toward the Program’s critical uncertainties in 2015-2016 (ISAB/ISRP 2016-1). The ISRP found the projects’ methods generally to be scientifically sound and offered recommendations for improvements. Methods and results are also published in top peer-reviewed journals in fisheries, geomorphology, and ecology, and thus the methods underwent peer review by experts in those fields.

A review of the utility of CHaMP measurements and their protocols was contracted by BPA and completed by Wildlands Hydrology. Neither the ISRP nor ISAB have reviewed Wildland Hydrology’s assessment, which raised concerns, particularly about channel bedform classifications and identification of bankfull channel. In response to questions during their public presentation, leaders of Wildland Hydrology’s assessment team indicated that the CHaMP measurements of several of the major factors in habitat quality in the Columbia River Basin (e.g., riparian vegetation, large wood, temperature, and sediment particle size) are scientifically sound.


One of the fundamental characteristics of the ISEMP and CHaMP programs that differentiates them from AEM is the spatial and temporal scales for which they were designed. These two programs are designed to "assess status and trends, restoration opportunities, and the benefits of habitat restoration actions on salmon and steelhead and their habitat in the tributary environment." They are designed to identify these properties at the scale of entire watersheds or basins. AEM is designed primarily to assess the effectiveness of specific types of restoration actions at the site and project scale rather than entire watersheds or basins. Measurements in these different programs have the potential to be complementary rather than mutually exclusive if the necessary cross walks and data consistency for the individual site data are developed.

3. Results


The ISEMP/CHaMP project broke new ground at the cutting edge of (a) measuring and modeling stream habitat across a wide range of spatial scales, and (b) relating this to salmonid abundance and production. The integrated ISEMP/CHaMP report provided summaries of three intensively monitored watersheds, 10 studies of fish and habitat status and trends, 23 tools for restoration assessment and planning, 6 life-cycle models, and 4 data support projects. CHaMP sampled more than 800 sites in 8 years, which will provide critical information and a foundation for monitoring for years to come. Data/models and source code are readily available through cloud-based data warehouses and GitHub sites. Unlike most monitoring of local reaches, the data and analysis from ISEMP/CHaMP can be extrapolated from study reaches to the Basin. It is not an overstatement that many of the methods and results will be useful for decades to come among researchers and managers in the Columbia River Basin and worldwide.


Of the 46 subprojects summarized in the report, several have contributed substantially to our understanding of status and trends in major basins, design of future monitoring, and decision making in the Fish and Wildlife Program. We highlight several of these to illustrate the value of the results from these two programs:

• ISEMP/CHaMP developed methods to estimate juvenile salmon and steelhead density throughout a river network based on measures of the primary production of benthic algae, the ultimate source of food for invertebrates that feed fish communities. About half of the variation in fish abundance is explained by these algae growing on the streambed. In turn, gross primary production (GPP) can be predicted from models of stream temperature, conductivity, and solar energy, allowing researchers to calculate estimates of fish abundance from maps of these three physical variables.


• ISEMP/CHaMP adapted the Barker Model to analyzing mark-recapture data where fish are marked at one place and time, such as with PIT tags, but then captured (or detected by PIT-tag antennas) at distant locations over a wide range of times. This analytical technique is a key to measuring effects of habitat restoration on migratory fish like salmon and steelhead, which have smolts that emigrate. Without it, analysis of the abundance of fish with these complex life cycles is very difficult. The model will improve estimates of salmonid survival as they migrate through the river network and hydrosystem

• To relate fish abundance to habitat, Quantile Regression Forest models were developed. These models can address the thorny problem that habitat is not always the sole limiting factor for fish abundance and so data are often highly variable. This statistical method can deal with this problem.


• ISEMP/CHaMP created six life cycle models, which represent an extremely important tool for future analyses and decision making throughout the Basin. In recent reviews, the ISAB concluded these life cycle models provide holistic analyses of potential limiting factors over the full life cycle and geographic range of anadromous salmon and steelhead (ISAB 2017-1, ISAB 2018-1). These models will assist landscape-level assessment of status and trends by the Fish and Wildlife Program in the future and will greatly assist decision makers in the region.


• The Intensively Monitored Watersheds revealed several major findings, illustrating the value in this approach. The studies revealed both positive responses to restoration actions in one basin and no response in another basin. The balanced and unbiased analysis of the research data should be a model for other projects, because sometimes the desire or commitment to provide benefits to fish populations and habitats colors the interpretation of project results.


• The status and trends subprojects provided a scientifically strong foundation for analysis of future trends. As examples, the temperature subproject and the survival estimation subprojects identified critical issues about habitat conditions and population status while providing valuable analytical tools. The temperature estimation approach incorporated riparian condition, floodplain conditions, and discharge, which are significant predictors and elements of many restoration actions within the Basin. The visualization techniques for temperature will contribute to future temperature studies in the Northwest.

• ISEMP and CHaMP developed more than 23 restoration tools to assist restoration practitioners and assist in the analysis of ISEMP and CHaMP data. Several of these have been used by other projects and have substantially contributed to the synthesis of ISEMP and CHaMP data. Examples of major contributions are the Quantile Regression Forest models for fish habitat relationships, geomorphic network analysis tool box, riparian condition assessment tool, wood recruitment assessment tool, and river classification and geomorphic condition tool. These analytical tools will continue to assist and inform restoration planning and design.


Many of the ISEMP projects created GIS mapping tools to estimate variables needed to drive other models. For example, abundance and productivity of salmonids are different in valleys of different types (i.e., wide valleys with low gradient where the stream meanders vs. narrow and steep valleys where the stream is mainly cascades). To address this relationship in life-cycle models, they developed a tool to measure valley bottom width from digital maps using GIS.

A key finding from these projects is the need to design monitoring studies of restoration actions at appropriate scales of space and time for expected resource outcomes. Projects were able to detect local improvements but often failed to detect population-wide responses (e.g., Entiat IMW). Not unexpectedly, population metrics suffer from spatial mis-matches (e.g., population spatial scale is much larger than restoration spatial scale), from high variability in responses, from influences by many factors over and above restoration, and a longer-time lag for restoration actions to become effective. Detection of population scale responses to local restoration actions likely will need much longer time scales than are typically included in most monitoring programs.


The tests of the effects of habitat restoration carried out and published by the ISEMP/CHaMP program are among the best conducted to date, worldwide. An example is the research on the effects of Beaver-Dam analogs on steelhead in the Bridge Creek watershed of the John DayRiver (Bouwes et al. 20162). As a result, this approach that requires less heavy equipment and site disturbance is being used increasingly in other programs throughout the Basin.


The program also broke new ground in determining how to measure habitat across a wide range of spatial scales from local sites to whole sub-watersheds, and where best to focus sampling efforts in such a daunting task. For example, a key result was that measuring more sites at intermediate intervals (e.g., 3 years) is more important than measuring fewer sites every year, because variation in habitat is larger across sites than among years within a given site.


Another key finding was that some of the most striking improvements in habitat and fish numbers have been achieved from lower cost projects that focus on natural-processes, such as small-scale structures and beaver input, not just from larger and more expensive projects.

Key project findings were shared widely in the refereed literature. The list of 53 peer-reviewed publications from the project, many of which appeared just in the last few years, is very impressive.


Lessons Learned and Areas for Improvement
The CHaMP protocol has been used by more than 10 state and Federal agencies and tribes to monitor fish habitat and restoration projects. Researchers and managers will continue to reap benefits from the tools developed and the papers published by the ISEMP/CHaMP program for decades to come. Despite the body of information developed by ISEMP and CHaMP, the lack of effective outreach and information sharing was a major weakness of the programs.

The programs did not provide frequent and timely syntheses of their measurements for use by regional projects and leaders of the Council and BPA. The programs were not successful in showing managers and policy makers why the results they were producing were useful. Examples of practical information for managers and decision makers in clear language free of jargon were needed. Instead, many of the reports were difficult to read and interpret, often because the writing was suitable for statisticians and GIS experts but not the intended audience. Most of the audience is lost by jargon, such as “site level sample inclusion probabilities,” “riparian vegetation departure products,” and “user-defined non-zero probability” in the Summary Report. Great results cannot be used if the audience who is supposed to use them cannot easily understand them.

One area that did not receive much attention in the project was the linkage of upslope/upstream processes and management to conditions in the downstream, valley bottom and fish bearing sections of watersheds. Although the need to provide tools or links for including these considerations was identified in past reviews, it does not appear to be included in the summary reports. Certainly, this would have added complexity to the work, but it is an important component for ultimately understanding, managing and restoring aquatic habitat and fish populations at watershed scale.


A final goal for ISEMP/CHaMP would be to show managers and decision makers results they can really use. The projects have ended, but NOAA Fisheries and collaborators are continuing to analyze, synthesize, and present their findings. It will be beneficial for the Fish and Wildlife Program to track and publicize when and where the findings and data are available into the future.

Design of Future Monitoring Programs
Although the projects are now closed, some observations looking back on the multi-year effort may be useful in designing and implementing future, large scale projects, particularly those that address similar research questions. The ISRP recommends that design of future monitoring should incorporate many of the 54 monitoring protocols and 800 CHaMP monitoring sites to take advantage of the 7-yr database for future trend analysis. The results of ISEMP and CHaMP can be used both for designing M&E projects and as prior long-term data for future monitoring sites. The investment in ISEMP and CHaMP has produced a substantial body of habitat data, methodological advances, analytical tools, and life cycle models, which provide a foundation on which to build future monitoring programs. This legacy should be carefully considered when developing the new tributary habitat RME strategy co-led by BPA, Council, and NOAA.

The analysis of measurement noise and overall variance for 54 different habitat measurements provides important insights for future monitoring. This analysis identifies the influence of different sources of variance for the habitat metrics used in the monitoring program. For example, most of the variance in estimates of coniferous riparian forest cover was related to either watershed or site components of variance, and variance in large wood fish cover was related primarily to site components. Interannual differences in their estimates were small, and they suggested greater information could be obtained by sampling a greater number of sites less frequently. They also found that crew-to-crew differences had little effect on the variance of their metrics. These results of the ISEMP/CHaMP programs should inform future design of monitoring programs in the Basin.


One of the most critical challenges for RME projects is providing information that can be directly applied to management issues throughout the life of the project, not just at the end. This requires agreement at the start of the project between the funders and the investigators about the degree of uncertainty that decision makers/management are willing to accept and the format for reporting. This also would require all users to understand that interim information be updated or revised in the future. More discussion and agreement on this issue between funders and project leaders at the outset of this long-term project would have been useful.


One of the innovative organizational aspects of the integrated final report is the concise and informative summaries of the individual sub-projects. The format provides a diagrammatic road map for the 46 sub-projects, graphical description of the relationships between the projects, concise nontechnical descriptions of the subprojects, concisely stated major findings and uses, contact information, and URL links to products, publications, web pages for tools or computer code, locations of data, and supporting files. The Council and BPA should consider using similar formats for other reports and products in the future.

4. 2017 Research Plan uncertainties validation


The summary report did not explicitly identify linkages to the 2017 Research Plan, but the 2017 Research Plan critical uncertainties database identifies direct or indirect linkages with 18 critical uncertainties for the ISEMP and CHaMP projects. The proponents did not directly crosswalk against the database. A quick review of the uncertainties database showed that most of them were covered by the projects except perhaps (a) resiliency to climate change, (b) additional habitat actions that are needed, and (c) estuary and plume actions and monitoring.

____________________________________

ISRP 2013-2 includes a comprehensive history of ISRP and ISAB reviews of the projects beginning on page 5.

2 Bouwes, N, N Weber, CE Jordan, WC Saunders, IA Tattam, C Volk, JM Wheaton and MM Pollock. 2016. Ecosystem experiment reveals benefits of natural and simulated beaver dams to a threatened population of steelhead (Oncorhynchus mykiss). Sci. Rep 6:28581. 10.1038/srep28581. http://www.ncbi.nlm.nih.gov/ pubmed/27373190. https://isemp.egnyte.com/dl/i3zz8KMhML.

 

Documentation Links:
Review: Fast Track ISRP Review 2010

Independent Scientific Review Panel Assessment

Assessment Number: 2003-017-00-ISRP-20100623
Project: 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP)
Review: Fast Track ISRP Review 2010
Completed Date: None
First Round ISRP Date: 2/24/2010
First Round ISRP Rating: Response Requested
First Round ISRP Comment:
Additional justification for the fast track elements (PIT tag arrays) is requested as a response during the fast track response loop. A written response to ISRP questions and presentation on the core ISEMP project is requested before or during the categorical RME review during the summer of 2010.

This is an ambitious, broad-scale project that is producing useful information for managers on the status and trends of habitat and fish populations in the mid- and upper-Columbia. Additionally, the work is addressing general issues of basic importance, such as how many sites are needed in a watershed to track habitat improvements over time, and habitat restoration actions resulting in increased focal species populations.

The ISRP recommends that the fast track components of this project be completed during this fast track review, but we find the overall proposal requires a comprehensive evaluation before or during the RM&E Categorical Review, after the ISRP has seen the larger RME Regional Plan.

A response to justify the PIT tag arrays should provide evidence that they will provide data for estimating the intended response variables. PIT tag arrays may not provide the key response variable in habitat evaluations. Smolt recruitment per spawner as a function of the number of spawners is the key variable, pre and post, and compared to external controls. Will arrays provide this? If they do, will they provide this information with the accuracy required? How many sites are required? Are more needed? Does the data to date suggest the recruitment limitation is elsewhere? What does the data collected to date say about these questions?

The comprehensive review and evaluation of ISEMP should include a written response and presentation to the ISRP. The written response and presentation should include analyses of data collected to date on status and trends of fish and habitat, and effectiveness of restoration actions in the subbasins they have been studying; the locations of the proposed PIT tag array should be more completely explained and justified. Other objectives, methods, procedures, and results to date need to be more completely clarified. In presenting these results they should demonstrate, as thoroughly as possible, how the monitoring methods and analyses they are employing are superior to other, more conventional methods. This will be especially important for the IMW sites.

1. Technical Justification, Program Significance and Consistency, and Project Relationships
The project description adequately justifies the various elements of the work. A strong case is made for each of the efforts in the John Day, Salmon, Entiat, and Methow subbasins. Detailed descriptions of some of the approaches are repeated in Section F, making for a long proposal, but in general this project is well justified, important to the mid- and upper-Columbia, and well tied in (albeit complexly) with other restoration M&E efforts. The current proposal requests funding for installation of PIT-tag detection antenna arrays in Snake River tributaries and initiation of reach scale habitat monitoring in the Methow and Entiat River. The reach scale work is justified, but better justification for installation of the PIT-tag arrays is needed, as explained below.

The proposal addresses several RPAs in the BiOp. ISEMP is linked to many projects in the Columbia River Basin. It makes use of an extensive amount of data collected by agencies and tribes and works in close cooperation with co-managers in the Basin.

2. Project History and Results
Project results are summarized in a series of matrices for each fiscal year at each major watershed, including major accomplishments and links to progress reports. Although the ISRP prefers not to be referred to hyperlinked reports, the broad scope of this project and the large number of reports and papers justifies this approach. Most of the material is available on the ISEMP website. In general, the project has succeeded in achieving the majority of its objectives to date. The accomplishments listed in the proposal pertain primarily to monitoring protocol development and testing as well as other work related to ISEMP’s goals. A summary should have been included.

A significant amount of data has been collected on status and trends of fish and habitat, and effectiveness of restoration actions in the course of protocol development and testing. Analysis of these data would be interesting and informative, and should be presented. One test of the effectiveness of ISEMP is whether its methods and means of data analysis prove superior to current methods for status and trend monitoring employed in the Basin with the understanding that there is overlap between the two. In addition to presenting results and analysis of data collected to this point, the proponents should compare their results to those obtained by other, more traditional, methods. For example, are ISEMP's escapement estimates for a particular river, for example using PIT-tag detector arrays, significantly more accurate than estimates obtained from weir or redd count data in the same river?

3. Objectives, Work Elements, and Methods
With regard to the expanded sites in the Entiat and Methow subbasins, the GRTS sites have not yet been selected. The approach to site selection is a proven one and should be successful; however, the proposal does not state what will happen if there are landowner access issues. The ISRP will be interested in the interpretation of the macroinvertebrate sampling results, as macroinvertebrates tend to be quite variable and difficult to relate to experimental restoration. Many of the field techniques are described in various ISEMP sampling protocol reports and were not repeated in the proposal.

The first objective pertains to installation of PIT-tag detector arrays in several tributaries of the Snake River. These arrays are in addition to numerous other arrays already present in Snake River tributaries. The proponents present a strong argument that arrays have many advantages in terms of accuracy and precision relative to more conventional techniques for assessing escapement and other adult parameters, and can be used to test many hypotheses and assumptions of interest to co-managers.

The question is whether the proposed set of arrays is necessary and that depends on the purpose of the installations. If the purpose is to further test arrays and improve methods for analysis and dissemination of array data, why aren't the currently operational arrays sufficient to accomplish these tasks? If, on the other hand, the purpose is to install arrays just to monitor MPGs or some other specific management function, then this should be better justified in relation to stated ISEMP goals which appear to be directed primarily toward establishing and testing monitoring methodologies. Although this is a fast-track proposal, it seems unlikely that the arrays will be in place and operational to monitor the 2010 run.

The second objective pertains to initiation of habitat status and trend monitoring in the Methow River and habitat and fish population status and trend monitoring in the Entiat River. The work in the Entiat would extend effort already ongoing in the IMW. The Methow work is well justified. It will determine whether the methods and metrics developed in the Entiat and Wenatchee Rivers are transferable to other rivers in the region. The effort to examine transferability is certainly worthwhile.

4. M&E
Because this is entirely an M&E project, the objectives, methods, and analytical techniques were adequately described in Section F, previously discussed.

In Table 1 on page 11, A core list of physical/environmental indicator variables to be monitored within subbasins in the Upper Columbia Basin is presented. There is a footnote indicating under water quality that other indicators can be measured, e.g., various metals and pollutants, herbicides and pesticides. It is gratifying to see this point made, but we wonder how and who makes these decisions and what are they based on? How were such indicators used in the past with respect to crop lands (certain pesticides, depending upon a particular crop), wastewater treatment plants (flame retardants, pharmaceuticals and personal care products), mining activity (selected heavy metals), or urban areas with low flow streams? Contaminants should be treated as a "wild card" that can confuse any salmon-habitat relationships, even on a large scale, including smolt survival many miles from the contaminant source.

This overall project and the second objective in this proposal (reach scale habitat monitoring in the Entiat and Methow) should directly contribute to improvements in protocols and methods of data analysis and dissemination for status and trend monitoring in the Columbia River Basin. Without additional justification, however, it was difficult to envision how the proposed additional sets of PIT-tag arrays would further the specific objectives of the ISEMP program at a broader scale.
Documentation Links:
Review: RME / AP Category Review

Council Recommendation

Assessment Number: 2003-017-00-NPCC-20101011
Project: 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP)
Review: RME / AP Category Review
Proposal: RMECAT-2003-017-00
Proposal State: Pending BPA Response
Approved Date: 6/10/2011
Recommendation: Fund (Qualified)
Comments: See Programmatic issue #2.
Conditions:
Council Condition #1 Programmatic Issue: RMECAT #2 Habitat effectiveness monitoring and evaluation—.

Independent Scientific Review Panel Assessment

Assessment Number: 2003-017-00-ISRP-20101015
Project: 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP)
Review: RME / AP Category Review
Proposal Number: RMECAT-2003-017-00
Completed Date: 12/17/2010
Final Round ISRP Date: 12/17/2010
Final Round ISRP Rating: Meets Scientific Review Criteria (Qualified)
Final Round ISRP Comment:
The Columbia Habitat Monitoring Program (CHaMP) monitoring program described in this proposal is very comprehensive and ambitious. The fact that this effort builds on the success of the ISEMP project provides increased confidence that the program can succeed. As the program is implemented, participants should consider developing formal mechanisms for communicating results to mangers. The ISEMP project holds a great deal of promise for answering the questions: “What is the current status of fish habitat in the Columbia River Basin?” and “Are restoration actions currently being undertaken in the Columbia River Basin having the desired effects on both habitat condition and biological response?” We were pleased to see that the current proposal includes a number of new watersheds that will expand the geographical scope and completeness of ISEMP. We were also pleased that the strong emphasis in standardized data collection and spatially balanced and randomized sampling is intended to bring more consistency to monitoring efforts in the Columbia River Basin. In general, the proposal meets scientific criteria, with one qualification. The ISRP also offers additional suggestions for project proponents to consider.

Qualification: The ISRP recommends that ISEMP organize a one-day workshop to discuss the CHaMP approach with the ISRP/ISAB and others. A draft of CHaMP should be circulated to the ISRP/ISAB before the workshop. Specific issues at the workshop should include how previously collected data can be or has been incorporated into CHaMP databases. It would also be useful to summarize how ISEMP priorities have evolved over the years, as well as a publication strategy. This qualification was discussed with the ISEMP project lead on September 13 and was determined to be agreeable. The workshop will be tentatively scheduled for late 2010 or early 2011.

1. Purpose, Significance to Regional Programs, Technical Background, and Objectives

In the Columbia River Basin, there are a wide variety of RM&E projects that often differ in sampling design, methods, metrics, and quality and extent of data analysis, and often do not lead to definitive evaluations of habitat restoration effectiveness. ISEMP is a far reaching project that is based on the reasonable proposition that a standardized set of protocols, procedures, and data collection and analytical methods that can be adopted basinwide is needed to improve data collection precision and accuracy, provide comparability of results within and between subbasins, and so improve the capability of determining the effectiveness of habitat restoration projects. This proposition has been tested in a limited number of watersheds. In this proposal ISEMP is expanding its scope to include several additional watersheds to further evaluate its protocols for study design, data collection, analytical methods, and information dissemination. This effort is worthwhile in that improvements in habitat effectiveness evaluation are badly needed within the Basin.

The technical background is very complete in the proposal. This project is clearly significant to regional restoration programs. A coordinated program for the collection, compilation, and archiving of data on fish populations and habitat condition has been identified in numerous ISRP and ISAB documents as a critical need for the Columbia Basin. Explanation of the significance of CHaMP to regional programs was very thorough. One question had to do with data archiving and database sharing. What will the interface between ISEMP databases and BPA’s Taurus project tracking system be? Will CHaMP data be linked to Taurus in such a way that interested parties can access habitat or population status and trend data (e.g., the CHaMP metadata library) directly, or will these databases be housed separately by ISEMP outside Taurus?

It was gratifying to see that CHaMP will be testing novel remote sensing techniques for assessing habitat condition over large areas, e.g., using green LiDAR or multispectral sensing. This project has also demonstrated leadership in tracking population and life history attributes using PIT-tag arrays and other landscape-based methods.

2. History: Accomplishments, Results, and Adaptive Management

The links provided to reports produced through ISEMP, in the aggregate, provide a very complete picture of the results generated by this project and reflects well on the prospects for success of the CHaMP program. The manner in which the CHaMP data will be used adaptively to modify future monitoring efforts is clearly described, and some examples are provided. However, the link with managers and policy decision makers in the basin is less clearly described. The proposal indicates that a process will be established specifically to utilize the data generated through CHaMP to produce new analysis tools, which will be used to generate the type of information required to determine future direction of restoration efforts and to support fisheries management decisions. But there is another step required to make this process maximally effective; a formal process for communicating the output from the data and analytical tools to non-technical audiences. ISEMP has used periodic newsletters as one mechanism for addressing this function. This approach also would be a reasonable option to consider for CHaMP. But the CHaMP project leaders should devote some effort to developing a consistent process for broadly disseminating program results.

The ISEMP project has expanded in scope perhaps more than any other habitat restoration-related project funded by BPA since its inception. There are now, according to the proposal, ISEMP studies taking place in 26 watersheds in the Columbia River Basin, all of which contain anadromous salmonids. That organizers have succeeded in growing this project in such an impressive fashion reflects well on the willingness of a wide variety of stakeholders (federal and state agencies, tribes, local conservation districts) to enter into cooperative arrangements with the ISEMP project to address large scale restoration status and effectiveness questions. ISEMP has grown to such an extent that many of its component parts could be treated as separate projects.

It was interesting to see how the proposal described past results. There were abundant maps and lists of activities taking place in ISEMP watersheds, but there were relatively few graphs or tables showing how target species have responded to habitat restoration. We were hoping for a little more in the way of biological response findings, since some restoration locations have now been monitored by ISEMP for seven years. The proposal suggests that the results of habitat restoration often require extended monitoring periods (i.e., often decades) in order for their effects to be assessed. We concur, but including a few highlights of some of the most informative results to date would have made the proposal more interesting.

The proposal does an excellent job of describing the formation and evolution of the CHaMP effort, which is in effect an important type of adaptive management, i.e., the development of standardized habitat survey protocols in order to facilitate data analyses and inter-watershed comparisons. Overall, the description of other activities was thorough and informative. One adaptive management question is: have any restoration actions changed as a result of ISEMP findings? In particular, we are interested in knowing if anything is being done differently because evidence is starting to suggest that current approaches are not working as anticipated. Perhaps, as the proposal points out, it is premature to make judgments but if there are any good examples of restoration practitioners learning from past mistakes, they would be worth knowing.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging)

Generally, the relationships of this program to other habitat and fish monitoring projects in the Columbia Basin are well described in the proposal. The one relationship that could have benefited from a more thorough discussion was the association between CHaMP and PNAMP. Several of the deliverables in the proposal will be co-developed with PNAMP. But the role of each organization in producing these deliverables was not clearly described. A paragraph in the introduction that outlines this relationship and some indication under the shared deliverables of roles and responsibilities would have helped to clarify the division of labor.

The monitoring work includes tagging, other types of fish population assessment, experimental habitat restoration, long-term habitat trend monitoring, habitat protocol standardization, food web studies, and management of very large datasets. The project does not emphasize identifying limiting factors; rather, it is aimed more toward evaluating the responses of aquatic habitat and fish populations to restoration actions at large spatial scales. Overall, the proposal does a good job of relating ISEMP-sponsored monitoring to other restoration and monitoring projects. The restoration questions being addressed in each of the watersheds are appropriate to the issues believed to be limiting to salmon production. The new CHaMP rotating panel (GRTS) monitoring design appears to hold considerable promise in characterizing habitat status and trends. The project is consistent with the call for expanded RM&E in many subbasin plans and regional programs. This work is relevant to most RM&E efforts basinwide and provides a means by which RM&E programs in diverse subbasins can be unified under a common set of protocols and procedures.

The ISRP suggests that pollutants not be overlooked as potentially limiting factors in certain locations.

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables, work elements, metrics and methods are adequately described. Some of the details specific to different work elements could be explained more completely (e.g., in what types of habitat will benthic macroinvertebrates be sampled, and why?), but given the very broad scope of the project and the need to cover all the work elements this is understandable.

Journal publications are listed in many of the deliverables for different objectives, but to date there have not been very many papers published from the ISEMP work. Hopefully this will change in the near future.

What was the rationale for including 25 sample sites in each basin for habitat status and trend monitoring? Did the ISEMP data suggest that this number of sites would provide adequate statistical power? Some support for this number of sites should be provided. It would seem that the number of sample sites required to adequately represent the range of channel types within a basin would vary based on watershed size, variation in topography, geology, land use and other factors. If this is the case, a variable sample size might be more appropriate.

The assessment of stable isotope analysis to characterize trophic aspects of habitat condition in the John Day River is a novel element of this habitat monitoring program and could lead to the development of a very valuable tool. Limiting factors have almost exclusively been restricted to physical habitat or water quality attributes, largely because there was no efficient method for assessing food web conditions. One suggestion about the proposed methodology for this work element: the terrestrial invertebrates should not be lumped together for stable isotope analysis. As with the aquatic invertebrates, the terrestrial insects should be grouped on the basis of functional group (detritivores, herbivores, or predators).

The deliverable that addresses status and trend monitoring in the Wenatchee actually discusses this activity in the Entiat. In fact, the information for the Entiat is repeated in the second deliverable, which actually does address the Entiat. This discrepancy seems like an inadvertent cut-and-paste error, but the Wenatchee information should be added to the first deliverable.

ISEMP suggests, where appropriate, an “experimental approach” where habitat restoration actions in “treatment” streams are compared to reference “untreated” streams. Given the physiographic and biological variability within reaches or tributaries within a subbasin, let alone differences between subbasins, selection of appropriate references and treatments could prove challenging. It probably would be helpful if ISEMP provided guidelines and/or assistance to subbasin investigators for selecting both reference and treatment sites. The same could be said for data analyses. With the large amount of data that will be collected, investigators may need some assistance in data analyses. It is our understanding that ISEMP is planning to provide analytical assistance where needed.
First Round ISRP Date: 10/18/2010
First Round ISRP Rating: Meets Scientific Review Criteria (Qualified)
First Round ISRP Comment:
The Columbia Habitat Monitoring Program (CHaMP) monitoring program described in this proposal is very comprehensive and ambitious. The fact that this effort builds on the success of the ISEMP project provides increased confidence that the program can succeed. As the program is implemented, participants should consider developing formal mechanisms for communicating results to mangers. The ISEMP project holds a great deal of promise for answering the questions: “What is the current status of fish habitat in the Columbia River Basin?” and “Are restoration actions currently being undertaken in the Columbia River Basin having the desired effects on both habitat condition and biological response?” We were pleased to see that the current proposal includes a number of new watersheds that will expand the geographical scope and completeness of ISEMP. We were also pleased that the strong emphasis in standardized data collection and spatially balanced and randomized sampling is intended to bring more consistency to monitoring efforts in the Columbia River Basin. In general, the proposal meets scientific criteria, with one qualification. The ISRP also offers additional suggestions for project proponents to consider.

Qualification: The ISRP recommends that ISEMP organize a one-day workshop to discuss the CHaMP approach with the ISRP/ISAB and others. A draft of CHaMP should be circulated to the ISRP/ISAB before the workshop. Specific issues at the workshop should include how previously collected data can be or has been incorporated into CHaMP databases. It would also be useful to summarize how ISEMP priorities have evolved over the years, as well as a publication strategy. This qualification was discussed with the ISEMP project lead on September 13 and was determined to be agreeable. The workshop will be tentatively scheduled for late 2010 or early 2011.

1. Purpose, Significance to Regional Programs, Technical Background, and Objectives

In the Columbia River Basin, there are a wide variety of RM&E projects that often differ in sampling design, methods, metrics, and quality and extent of data analysis, and often do not lead to definitive evaluations of habitat restoration effectiveness. ISEMP is a far reaching project that is based on the reasonable proposition that a standardized set of protocols, procedures, and data collection and analytical methods that can be adopted basinwide is needed to improve data collection precision and accuracy, provide comparability of results within and between subbasins, and so improve the capability of determining the effectiveness of habitat restoration projects. This proposition has been tested in a limited number of watersheds. In this proposal ISEMP is expanding its scope to include several additional watersheds to further evaluate its protocols for study design, data collection, analytical methods, and information dissemination. This effort is worthwhile in that improvements in habitat effectiveness evaluation are badly needed within the Basin.

The technical background is very complete in the proposal. This project is clearly significant to regional restoration programs. A coordinated program for the collection, compilation, and archiving of data on fish populations and habitat condition has been identified in numerous ISRP and ISAB documents as a critical need for the Columbia Basin. Explanation of the significance of CHaMP to regional programs was very thorough. One question had to do with data archiving and database sharing. What will the interface between ISEMP databases and BPA’s Taurus project tracking system be? Will CHaMP data be linked to Taurus in such a way that interested parties can access habitat or population status and trend data (e.g., the CHaMP metadata library) directly, or will these databases be housed separately by ISEMP outside Taurus?

It was gratifying to see that CHaMP will be testing novel remote sensing techniques for assessing habitat condition over large areas, e.g., using green LiDAR or multispectral sensing. This project has also demonstrated leadership in tracking population and life history attributes using PIT-tag arrays and other landscape-based methods.

2. History: Accomplishments, Results, and Adaptive Management

The links provided to reports produced through ISEMP, in the aggregate, provide a very complete picture of the results generated by this project and reflects well on the prospects for success of the CHaMP program. The manner in which the CHaMP data will be used adaptively to modify future monitoring efforts is clearly described, and some examples are provided. However, the link with managers and policy decision makers in the basin is less clearly described. The proposal indicates that a process will be established specifically to utilize the data generated through CHaMP to produce new analysis tools, which will be used to generate the type of information required to determine future direction of restoration efforts and to support fisheries management decisions. But there is another step required to make this process maximally effective; a formal process for communicating the output from the data and analytical tools to non-technical audiences. ISEMP has used periodic newsletters as one mechanism for addressing this function. This approach also would be a reasonable option to consider for CHaMP. But the CHaMP project leaders should devote some effort to developing a consistent process for broadly disseminating program results.

The ISEMP project has expanded in scope perhaps more than any other habitat restoration-related project funded by BPA since its inception. There are now, according to the proposal, ISEMP studies taking place in 26 watersheds in the Columbia River Basin, all of which contain anadromous salmonids. That organizers have succeeded in growing this project in such an impressive fashion reflects well on the willingness of a wide variety of stakeholders (federal and state agencies, tribes, local conservation districts) to enter into cooperative arrangements with the ISEMP project to address large scale restoration status and effectiveness questions. ISEMP has grown to such an extent that many of its component parts could be treated as separate projects.

It was interesting to see how the proposal described past results. There were abundant maps and lists of activities taking place in ISEMP watersheds, but there were relatively few graphs or tables showing how target species have responded to habitat restoration. We were hoping for a little more in the way of biological response findings, since some restoration locations have now been monitored by ISEMP for seven years. The proposal suggests that the results of habitat restoration often require extended monitoring periods (i.e., often decades) in order for their effects to be assessed. We concur, but including a few highlights of some of the most informative results to date would have made the proposal more interesting.

The proposal does an excellent job of describing the formation and evolution of the CHaMP effort, which is in effect an important type of adaptive management, i.e., the development of standardized habitat survey protocols in order to facilitate data analyses and inter-watershed comparisons. Overall, the description of other activities was thorough and informative. One adaptive management question is: have any restoration actions changed as a result of ISEMP findings? In particular, we are interested in knowing if anything is being done differently because evidence is starting to suggest that current approaches are not working as anticipated. Perhaps, as the proposal points out, it is premature to make judgments but if there are any good examples of restoration practitioners learning from past mistakes, they would be worth knowing.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging)

Generally, the relationships of this program to other habitat and fish monitoring projects in the Columbia Basin are well described in the proposal. The one relationship that could have benefited from a more thorough discussion was the association between CHaMP and PNAMP. Several of the deliverables in the proposal will be co-developed with PNAMP. But the role of each organization in producing these deliverables was not clearly described. A paragraph in the introduction that outlines this relationship and some indication under the shared deliverables of roles and responsibilities would have helped to clarify the division of labor.

The monitoring work includes tagging, other types of fish population assessment, experimental habitat restoration, long-term habitat trend monitoring, habitat protocol standardization, food web studies, and management of very large datasets. The project does not emphasize identifying limiting factors; rather, it is aimed more toward evaluating the responses of aquatic habitat and fish populations to restoration actions at large spatial scales. Overall, the proposal does a good job of relating ISEMP-sponsored monitoring to other restoration and monitoring projects. The restoration questions being addressed in each of the watersheds are appropriate to the issues believed to be limiting to salmon production. The new CHaMP rotating panel (GRTS) monitoring design appears to hold considerable promise in characterizing habitat status and trends. The project is consistent with the call for expanded RM&E in many subbasin plans and regional programs. This work is relevant to most RM&E efforts basinwide and provides a means by which RM&E programs in diverse subbasins can be unified under a common set of protocols and procedures.

The ISRP suggests that pollutants not be overlooked as potentially limiting factors in certain locations.

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables, work elements, metrics and methods are adequately described. Some of the details specific to different work elements could be explained more completely (e.g., in what types of habitat will benthic macroinvertebrates be sampled, and why?), but given the very broad scope of the project and the need to cover all the work elements this is understandable.

Journal publications are listed in many of the deliverables for different objectives, but to date there have not been very many papers published from the ISEMP work. Hopefully this will change in the near future.

What was the rationale for including 25 sample sites in each basin for habitat status and trend monitoring? Did the ISEMP data suggest that this number of sites would provide adequate statistical power? Some support for this number of sites should be provided. It would seem that the number of sample sites required to adequately represent the range of channel types within a basin would vary based on watershed size, variation in topography, geology, land use and other factors. If this is the case, a variable sample size might be more appropriate.

The assessment of stable isotope analysis to characterize trophic aspects of habitat condition in the John Day River is a novel element of this habitat monitoring program and could lead to the development of a very valuable tool. Limiting factors have almost exclusively been restricted to physical habitat or water quality attributes, largely because there was no efficient method for assessing food web conditions. One suggestion about the proposed methodology for this work element: the terrestrial invertebrates should not be lumped together for stable isotope analysis. As with the aquatic invertebrates, the terrestrial insects should be grouped on the basis of functional group (detritivores, herbivores, or predators).

The deliverable that addresses status and trend monitoring in the Wenatchee actually discusses this activity in the Entiat. In fact, the information for the Entiat is repeated in the second deliverable, which actually does address the Entiat. This discrepancy seems like an inadvertent cut-and-paste error, but the Wenatchee information should be added to the first deliverable.

ISEMP suggests, where appropriate, an “experimental approach” where habitat restoration actions in “treatment” streams are compared to reference “untreated” streams. Given the physiographic and biological variability within reaches or tributaries within a subbasin, let alone differences between subbasins, selection of appropriate references and treatments could prove challenging. It probably would be helpful if ISEMP provided guidelines and/or assistance to subbasin investigators for selecting both reference and treatment sites. The same could be said for data analyses. With the large amount of data that will be collected, investigators may need some assistance in data analyses. It is our understanding that ISEMP is planning to provide analytical assistance where needed.
Documentation Links:

2008 FCRPS BiOp Workgroup Assessment

Assessment Number: 2003-017-00-BIOP-20101105
Project Number: 2003-017-00
Review: RME / AP Category Review
Proposal Number: RMECAT-2003-017-00
Completed Date: None
2008 FCRPS BiOp Workgroup Rating: Response Requested
Comments: BiOp Workgroup Comments: BPA has questions regarding the action effectiveness study for Salmon Subbasin watershed model implementation in the South Fork Salmon River and Lemhi River.

The BiOp RM&E Workgroups made the following determinations regarding the proposal's ability or need to support BiOp Research, Monitoring and Evaluation (RME) RPAs. If you have questions regarding these RPA association conclusions, please contact your BPA COTR and they will help clarify, or they will arrange further discussion with the appropriate RM&E Workgroup Leads. BiOp RPA associations for the proposed work are: (35.1 35.2 35.3 50.4 50.5 50.6 56.1 56.2 56.3 57.1 57.2 57.3 57.4 71.4 71.5 72.1)
All Questionable RPA Associations ( ) and
All Deleted RPA Associations ( )
Proponent Response:

BPA has questions regarding the action effectiveness study for Salmon Subbasin watershed model implementation in the South Fork Salmon River and Lemhi River.

 

ISEMP staff would be happy to meet with the BPA FCRPS Biop workgroup to address any questions regarding the watershed model being implemented in the SF Salmon and Lemhi Rivers.  The modeling approach was reviewd by ISRP as a NPCC condition for funding in 2006, and has not changed appreciably.  The implemetnation of on the ground fish and habitat monitoring in the SF Salmon and Lemhi Rivers over the past two years is to paramterize the watershed model.  The model will be used to predict fish population responses to restoration actions planned and underway in the Lemhi River.

Review: FY07-09 Solicitation Review

Council Recommendation

Assessment Number: 2003-017-00-NPCC-20090924
Project: 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP)
Review: FY07-09 Solicitation Review
Approved Date: 10/23/2006
Recommendation: Fund
Comments: Interim funding pending further Council consideration of regional monitoring and evaluation framework. ISRP fundable (qualified): address ISRP concerns during further consideration.

Independent Scientific Review Panel Assessment

Assessment Number: 2003-017-00-ISRP-20060831
Project: 2003-017-00 - Integrated Status and Effectiveness Monitoring Program (ISEMP)
Review: FY07-09 Solicitation Review
Completed Date: 8/31/2006
Final Round ISRP Date: None
Final Round ISRP Rating: Meets Scientific Review Criteria (Qualified)
Final Round ISRP Comment:
This is a good proposal overall. The large scale, basinwide approach is good. A project such as this one is clearly needed in the Columbia Basin to integrate M&E efforts and provide consistency among diverse M&E projects. The technical background and rationale are discussed and clearly establish the need for an integrated monitoring program for the Columbia Basin that could contribute to development of an adaptive management plan for the basin. If the process proposed in this project succeeds in bringing together a wide variety of large environmental data sets in a new and integrated fashion, it will represent a major breakthrough in describing and managing tributary restoration efforts.

The proposal is very complex. How are all of the separate parts of the proposal going to be integrated? Some questions related to the technical background of the project and its objectives need to be addressed:
1) What is the role of the sponsors in this project? Coordination? Data collection? Data analysis? Will the sponsors have some involvement in each objective?
2) What does monitoring at the subbasin scale mean?
3) What are some examples of metrics that represent subbasin-scale performance?
4) How will the information generated by the projects be integrated and analyzed to accomplish overall project objectives such as determining limiting factors and evaluation of basinwide project effectiveness?

The project history is clearly described, with a good justification of why the work should be continued and why the suggested pilot-scale sites were chosen. The list of accomplishments is impressive. The three-year history of the project shows how it has grown in both scope and linkages over time. The project is linked to numerous state and federal projects within each of the targeted subbasins.

The objectives are very broad in most cases and involve continuing work begun in 2003. The Wenatchee and John Day projects do a good job of relating objectives of each individual project to the overall project objectives. The objectives for the South Fork Salmon River and Lemhi are not as clear. How do the objectives for this work relate to overall project objectives (item 3.0)? The South Fork Salmon and Lemhi projects should use the same format as the Wenatchee and John Day.

Methods were clearly explained, and the approach will involve innovative techniques. This proposal builds on using many of the best available long-term population status and habitat inventory datasets in the region. The proposal notes that it will take a long time to determine the success of the integrated status and effectiveness monitoring program, but the provisions for long-term monitoring and the choice of monitoring sites were clearly thought out.
Documentation Links:

Legal Assessment (In-Lieu)

Assessment Number: 2003-017-00-INLIEU-20090521
Project Number: 2003-017-00
Review: FY07-09 Solicitation Review
Completed Date: 10/6/2006
In Lieu Rating: Problems May Exist
Cost Share Rating: 2 - May be reasonable
Comment: M&E for status/trend of salmonid/steelhead; multiple other entities authorized/required (e.g., fishery managers, others); need confirmation that cost-share is reasonable. Note: rating changed on final review from a 2.3 to a 2.2 because internal BPA review indicated that a significant part of the project relates to assessing offsite improvements for FCRPS mitigation.

Capital Assessment

Assessment Number: 2003-017-00-CAPITAL-20090618
Project Number: 2003-017-00
Review: FY07-09 Solicitation Review
Completed Date: 2/27/2007
Capital Rating: Does Not Qualify for Capital Funding
Capital Asset Category: None
Comment: None

Project Relationships: None

Name Role Organization
Chris Jordan Project Lead National Oceanic and Atmospheric Administration
Mike Ward Project Lead Terraqua, Inc.
Peter Lofy Interested Party Bonneville Power Administration
Benjamin Zelinsky Interested Party Bonneville Power Administration
Luca De Stefanis Env. Compliance Lead Bonneville Power Administration
Sarah (Terraqua) Walker (Inactive) Technical Contact Terraqua, Inc.
Jody Lando Project Manager Bonneville Power Administration
Kristen Jule Supervisor Bonneville Power Administration