Columbia Basin Fish and Wildlife Program Columbia Basin Fish and Wildlife Program
RSS Feed for updates to Project 1988-108-04 - StreamNet Follow this via RSS feed. Help setting up RSS feeds?

Project Summary

Project 1988-108-04 - StreamNet
Project Number:
1988-108-04
Title:
StreamNet
Summary:
StreamNet Project - formerly/AKA - Coordinated Information System (CIS)/ Northwest Environmental Database (NED)
BPA Project No.: 198810804

The Pacific States Marine Fisheries Commission (PSMFC) hosts StreamNet, which provides access to regional fish data by maintaining a coordinated, standardized, web-based distributed information network. Partner agencies funding through this project are; the Confederated Colville Tribes (CCT), Idaho Department of Fish & Game (IDFG), Montana Fish, Wildlife, and Parks (MFWP), Oregon Department of Fish & Wildlife (ODFW), and Washington Department of Fish & Wildlife (WDFW). The need for regionally coordinated and readily accessible data has been identified by the Northwest Power and Conservation Council (NPCC), the Bonneville Power Administration (BPA), and the National Oceanic and Atmospheric Administration Fisheries Program (NOAA). StreamNet works cooperatively with the agencies that create the data through StreamNet supported technical staff inside these agencies and by leading or coordinating a number of initiatives to aid in assuring a regional approach to data management, and provides this data to users such as NPCC, BPA, and NOAA.

StreamNet supports data flow and management within the data source agencies through assistance in development of database systems and approaches for improving data management efficiency and data dissemination. StreamNet currently handles data sharing for some agencies and supports development of agency capacities for others. StreamNet employees and subcontractors locate data, standardize data reporting through the cooperative development of protocols, complete QA/QC of data, and then assure the flow of data from state, tribal, or agency repositories to and through StreamNet. StreamNet participates in or leads a variety of teams of data management professionals from states, tribes, and agencies that coordinate regional data sharing. Data flow has been/is being streamlined through the implementation of application programming interfaces (APIs) for various data types.

StreamNet’s current priority is to help lead implementation of the Coordinated Assessments (CA) project, in partnership with the Pacific Northwest Aquatic Monitoring Partnership (PNAMP). CA focuses on salmon and steelhead populations in the Columbia Basin. The primary data types contained in and disseminated through the CA project that relate to abundance and Viable Salmon Population (VSP) parameters are five VSP indicators including population scale estimates of natural spawner abundance, smolt to adult ratio, adult recruits per spawner (spawner to spawner ratio) smolt outmigrants and presmolt abundance. In addition to high level indicator data, related data (aka Trends) is also curated by the StreamNet project, including spawner counts, juvenile counts, redd counts and dam and weir counts. These are raw data that relate to population estimates and estimates of VSP parameters, summarized to annual totals.

The Coordinated Assessments (CA) Project is an effort to develop efficient, consistent, and transparent data-sharing among the co-managers (fish and wildlife agencies and Tribes) and regulatory/funding agencies (BPA and NOAA) of the Columbia River Basin (CRB) for anadromous fish related data. The project has been coordinated by PNAMP and the PSMFC StreamNet project since its inception in 2010 (see: https://www.pnamp.org/project/coordinated-assessments-for-salmon-and-steelhead). A 5-year Plan for the CA project is reviewed and approved annually by the StreamNet Executive Committee. The Executive Committee is made up of fisheries professionals at the policy level from all member organizations, as well as CRITFC, NOAA-Fisheries, USFWS, PSMFC, BPA, PNAMP and the NPCC. CRITFC member tribes have been invited to participate on both the Executive Committee and Steering Committee and sometimes attend. The tribes have made it clear that lack of funding restricts their ability to participate. The 5-year Plan prioritizes data for contribution from partners.

CA is focused on the key indicators and metrics that have been identified as priorities for reporting progress on implementation of the Federal Columbia River Power System (FCRPS) Biological Opinion (BiOp). At the end of 2015, BPA identified a need to gather as much data as possible for populations they determined were Tier 1 and Tier 2 priorities and efforts continued to be focused on these 69 populations. Updates of certain standard data “trends”, such as those related to the CA project or feeding the NPCC dashboards (including for resident fish), are also a priority.

In addition to CA, StreamNet maintains the Data Store archive service, which allows agencies and projects that don’t have the capability to host and maintain data sets online to upload and preserve their data in a secure data repository, making them widely available. StreamNet also maintains the Council’s HEP and Protected Areas archives, as well as substantial historical data on fish populations, habitat, hatcheries and harvest. This data is publicly available and searchable with an integrated query system.


Proposer:
None
Proponent Orgs:
Pacific States Marine Fisheries Commission (Govt - State)
Starting FY:
1989
Ending FY:
2032
BPA PM:
Stage:
Implementation - Project Status Report
Area:
Province Subbasin %
Basinwide - 100.00%
Purpose:
Programmatic
Emphasis:
Data Management
Focal Species:
All Anadromous Fish
All Anadromous Salmonids
Bass, Largemouth
Bass, Smallmouth
Burbot
Carp, Common
Catfish
Chinook - All Populations
Chinook - Deschutes River Summer/Fall ESU
Chinook - Lower Columbia River ESU
Chinook - Mid-Columbia River Spring ESU
Chinook - Snake River Fall ESU
Chinook - Snake River Spring/Summer ESU
Chinook - Upper Columbia River Spring ESU
Chinook - Upper Columbia River Summer/Fall ESU
Chinook - Upper Willamette River ESU
Chub, Oregon
Chum - Columbia River ESU
Coho - Lower Columbia River ESU
Coho - Unspecified Population
Crappie, Black
Crappie, White
Cutthroat Trout, Coastal - All Anadromous Populations
Cutthroat Trout, Coastal - Resident Populations
Cutthroat Trout, Lahontan
Cutthroat Trout, Westslope
Cutthroat Trout, Yellowstone
Freshwater Mussels
Kokanee
Lamprey, Pacific
Lamprey, River
Lamprey, Western Brook
Other Anadromous
Other Resident
Perch, Yellow
Pike, Northern
Pikeminnow, Northern
Shad, American
Sockeye - All Populations
Sockeye - Snake River ESU
Steelhead - All Populations
Steelhead - Lower Columbia River DPS
Steelhead - Middle Columbia River DPS
Steelhead - Snake River DPS
Steelhead - Upper Columbia River DPS
Steelhead - Upper Willamette River DPS
Sturgeon, Green
Sturgeon, White - All Populations except Kootenai R. DPS
Sturgeon, White - Kootenai River DPS
Sturgeon, White - Lower Columbia River
Trout, Brook
Trout, Brown
Trout, Bull
Trout, Interior Redband
Trout, Lake
Trout, Rainbow
Walleye
Whitefish, Mountain
Wildlife
Species Benefit:
Anadromous: 90.0%   Resident: 10.0%   Wildlife: 0.0%
Special:
None

No photos have been uploaded yet for this project.

Summary of Budgets

To view all expenditures for all fiscal years, click "Project Exp. by FY"

Expense SOY Budget Working Budget Contracted Amount Modified Contract Amount Expenditures *
FY2018 (Previous) $2,145,483 $2,145,483 $4,088,003 $2,056,794 $1,748,152

BiOp FCRPS 2008 (non-Accord) $2,145,483 $4,088,003 $2,056,794 $1,748,152
FY2019 (Current) $2,002,970 $2,002,970 $0 $2,044,002 $1,523,497

BiOp FCRPS 2008 (non-Accord) $2,002,970 $0 $2,044,002 $1,523,497
FY2020 (Next) $2,002,970 $2,002,970 $4,005,940 $2,002,970 $0

BiOp FCRPS 2008 (non-Accord) $2,002,970 $4,005,940 $2,002,970 $0

* Expenditures data includes accruals and are based on data through 30-Jun-2019

Decided Budget Transfers  (FY2018 - FY2020)

Acct FY Acct Type Amount Fund Budget Decision Date
FY2018 Expense $2,145,483 From: BiOp FCRPS 2008 (non-Accord) FY18 SOY Budgets 07/17/2017
FY2019 Expense $2,002,970 From: BiOp FCRPS 2008 (non-Accord) Q1 FY19 Budgets 09/17/2018
FY2020 Expense $2,002,970 From: BiOp FCRPS 2008 (non-Accord) FY20 SOY 06/05/2019

Pending Budget Decision?  No


Actual Project Cost Share

Current Fiscal Year — 2019   DRAFT
Cost Share Partner Total Proposed Contribution Total Confirmed Contribution
Montana Fish, Wildlife and Parks (MFWP) $61,860
Pacific States Marine Fisheries Commission $12,000
Oregon Department of Fish and Wildlife $21,048
Oregon Department of Fish and Wildlife $20,403
Washington Department of Fish and Wildlife (WDFW) $127,626
Idaho Department of Fish and Game (IDFG) $17,589
Idaho Department of Fish and Game (IDFG) $119,950
Total $0 $380,476
Previous Fiscal Years
Fiscal Year Total Contributions % of Budget
2018 $376,742 15 %
2017 $132,166 6 %
2016 $231,017 10 %
2015 $211,513 9 %
2014 $483,006 19 %
2013 $525,927 28 %
2012 $447,339 18 %
2011 $821,604 26 %
2010 $572,709 23 %
2009 $565,252 22 %
2008 $529,086 20 %
2007 $424,557 15 %

Contracts

The table below contains contracts with the following statuses: Active, Complete, History, Issued.
* "Total Contracted Amount" column includes contracted amount from both capital and expense components of the contract.
Expense Contracts:
Number Contractor Name Title Status Total Contracted Amount Dates
293 REL 1 SOW Pacific States Marine Fisheries Commission 198810804 STREAMNET History $902,658 9/28/1995 - 9/30/2000
400 REL 1 SOW Synergy Consulting, Inc. 1988-108-04 PACIFIC NW HYDROPOWER DATA BASE & ANALYSYS SYSTEM Terminated $59,681 11/25/1999 - 12/31/2001
2996 REL 1 SOW Pacific States Marine Fisheries Commission 1988-108-04 STREAM NET Terminated $2,071,025 10/1/2000 - 9/30/2001
77051 SOW Pacific States Marine Fisheries Commission 1988-108-04 EXP STREAMNET Issued $4,088,003 10/1/2017 - 9/30/2019
CR-333118 SOW Pacific States Marine Fisheries Commission 1988-108-04 EXP STREAMNET Pending $4,005,940 10/1/2019 - 9/30/2021



Annual Progress Reports
Expected (since FY2004):16
Completed:14
On time:14
Status Reports
Completed:56
On time:29
Avg Days Late:5

Earliest Subsequent           Accepted Count of Contract Deliverables
Contract Contract(s) Title Contractor Start End Status Reports Complete Green Yellow Red Total % Green and Complete Canceled
293 REL 1 4053, 20518, 28958, 34428, 38615, 43664, 49053, 54765, 60412, 63021, 66435, 77051 198810804 STREAMNET Pacific States Marine Fisheries Commission 09/1995 09/1995 Pending 56 278 11 0 5 294 98.30% 0
4341 20649, 47658 1988-108-04 NW HYDROPOWER DATABASE DEVELOPMENT Synergy Consulting, Inc. 11/1997 11/1997 Closed 0 0 0 0 0 0 0
Project Totals 56 278 11 0 5 294 98.30% 0


The table content is updated frequently and thus contains more recent information than what was in the original proposal reviewed by ISRP and Council.

Review: 2019-2021 Mainstem/Program Support

Independent Scientific Review Panel Assessment

Assessment Number: 1988-108-04-ISRP-20190404
Project: 1988-108-04 - StreamNet
Review: 2019-2021 Mainstem/Program Support
Proposal Number: NPCC19-1988-108-04
Completed Date: None
First Round ISRP Date: 4/4/2019
First Round ISRP Rating: Response Requested
First Round ISRP Comment:

Response requested comment:

StreamNet is a long-standing project that has evolved into a key component of the Basin's anadromous fish management and reporting program. It has provided data management and dissemination for more than 20 years. Although there appears to be room for continued improvement and efficiency, the project is well organized and effective at meeting an increasing array of goods and services to its users. With continued focus on reducing expenditures for most projects and given the importance of the project to basinwide management and reporting, it appears careful review of future work plans and approaches will be needed to effectively address increasing demands for data management and dissemination.

The proponents are asked to respond to the following issues as well as additional issues identified in the review comments.

1.      The ISRP requests that the proponents develop an adaptive management (AM) plan, one that encompasses both internal and external AM. This plan was also requested in past ISRP project reviews. The ISRP notes that while an internal AM process is not described in the proposal, the proponents do provide internal goals and objectives. However, these are only part of an AM process. Please present the internal AM process for ISRP review.

2.      As described in the 2006 ISRP Review, "The project should have in place a system for monitoring and evaluating its performance. The program still needs to develop more in-depth measures of monitoring effectiveness and assess its impact in terms of user satisfaction. Use of the services should be documented, and more focus should be placed on outputs rather than inputs. A systematic way of evaluating effectiveness is needed. Who are the users? Were these users satisfied? Is tracking software used (e.g., Web Trends)? The sponsors should provide some evaluative performance information to address these questions."

The proponents still need to respond to a previous qualification from the 2012 ISRP review, specifically:

3.      That the proponents: "Provide a report describing in detail the data quality assurance and quality control (QA/QC) procedures used by StreamNet." If this is in another StreamNet publication, please provide the link. Otherwise, please provide a response detailing the QA/QC procedures.

Comment:

1. Objectives, Significance to Regional Programs, and Technical Background

There are three primary objectives listed in the 2019 proposal that address priority work items identified in the Five-Year Plan for Coordinated Assessments. These objectives are qualitative and do not lend themselves to tracking accomplishments with given timelines. The ISRP believes that the StreamNet project needs to establish quantitative objectives and timelines as well as interim milestones for meeting them.

The ISRP recognizes that the StreamNet Project is an important component among regional agencies and programs, especially its role in providing web-based, standardized, and comprehensive information for anadromous fish in the Basin. The need for regionally coordinated and readily accessible data has been consistently identified by the Northwest Power and Conservation Council (NPCC), the Bonneville Power Administration (BPA), and the National Oceanic and Atmospheric Administration Fisheries Program (NOAA). StreamNet began in 1995 and is now the primary data source for a number of agencies. A five-year Strategic Plan is in place and the current priority is identified as the Coordinated Assessments Project. StreamNet helps lead implementation of the Coordinated Assessments Project, in partnership with the Pacific Northwest Aquatic Monitoring Partnership (PNAMP).

The technical background of personnel involved appears to be strong.

2. Results and Adaptive Management

The project has been very effective in developing a data management and dissemination system focused primarily on anadromous fish. It provides consistent and transparent data-sharing among formal members as well as other cooperators and partners. Due to past and current work priorities and limited funding, StreamNet does not provide much organization or support for resident fish or wildlife data.

The use of supported technical staff located and working within member agencies is a useful way of developing ownership and technical expertise. However, the ISRP acknowledges that supervision and oversight of quality and work timelines may be difficult with a dispersed work group. According to the proponents, these technical staff work to develop the data management infrastructure within specific agencies to first satisfy the internal need for this information, and secondly to standardize and coordinate sharing of this information across state and agency boundaries.

The ISRP is impressed by the proponent's contributions to the development of database systems and approaches for improving the efficiency of data management and dissemination, their work enhancing agencies' capacities for data management, and their participation in a variety of teams of data management professionals from states, tribes, and agencies that coordinate regional data sharing.

There is no formal plan or discussion of adaptive management (AM). It is noted that a new organizational structure was put in place in 2014 to help improve project direction and performance. An Executive Committee was formed to direct the project, and it is comprised of policy level members of the StreamNet group as well as representatives of regional management agencies that fund the program and/or use the information. However, how the Executive Committee addresses AM issues is not discussed, and the ISRP would like more information on the mechanics of how AM is implemented.

An activity that helped drive adaptive changes to the project was the development and implementation of an online survey involving fisheries data management professionals in spring 2015. The survey asked respondents for opinions on data management questions. Discussions were then held with the StreamNet Steering Committee, BPA, NPCC, and others. One result was adoption of a five-year plan to guide activities. The plan was reviewed and updated by the Committee again on September 2017 and November 2018.

Some specific comments:

1.      The recommendation on pages 4-5 of the latest annual report represented a thoughtful level of synthesis and strategic thinking.

2.      The ISRP notes that many projects generate data in diverse formats and StreamNet brings them together nicely. This is a vitally important aspect of the project.

3.      The ISRP was surprised that photographs are no-longer stored. For habitat restoration work, photographs provide visual evidence of changes over time. Are these superseded by other formats? StreamNet seems like a logical place to store images from photo-points recorded over time.

3. Methods: Project Relationships, Work Types, and Deliverables

Well-developed database methods are used by StreamNet. There is a comprehensive description of project relationships, work types, and deliverables. Given declining funding it appears that maintaining the current mix of work types and level of activity will be increasingly challenging. The StreamNet Steering Committee and other partners will need to engage in a careful review of future work plans and approaches to ensure that the project is able to maintain its important basin-wide management and reporting responsibilities.

The proponents rightfully recognize that "Threats and limiting factors for StreamNet are primarily related to leadership, direction, and the data management capabilities within the data source agencies, rather than biological or ecological factors." Do the proponents have suggestions on how the Fish and Wildlife Program could assist in addressing these important threats and limiting factors?

Documentation Links:
Review: Resident Fish, Regional Coordination, and Data Management Category Review

Council Recommendation

Assessment Number: 1988-108-04-NPCC-20120313
Project: 1988-108-04 - StreamNet
Review: Resident Fish, Regional Coordination, and Data Management Category Review
Proposal: RESCAT-1988-108-04
Proposal State: Pending BPA Response
Approved Date: 3/5/2014
Recommendation: Implement with Conditions
Comments: Council recommendation:
Fund as proposed with the following supplemental recommendations through FY 2013:

- Data access under this work should continue to evolve towards a more accessible platform for various users and optimize dynamic web-services to facilitate coordinated data-sharing and data depiction.

- As feasible, this work should expand to include additional managers (and data collecting entities3) that currently cannot easily provide access to their data, whether raw or synthesized, to improve accessibility to their data.

- StreamNet should strive to be a comprehensive data portal (e.g. linking to and depicting data from other sources etc.) for locating fish data needed to inform Program implementation and broad Program evaluation, emphasizing on using web-services. With respect to salmonid fish data, data collectors could provide their data directly to StreamNet while non-salmonid fish data could be made accessible to StreamNet through web-services from resident fish databases or a resident fish data portal.

- Data stored and accessed through StreamNet should include synthesized information, e.g. population estimates, needed for informing Program implementation and broad Program evaluation.

- Data made accessible through StreamNet should focus on data funded by Bonneville and priority data for the program. Identification of Bonneville funded projects that collect fish data should be based on project information available at cbfish.org.

- As necessary, prioritization of Bonneville funded data should be informed by Bonneville and Council’s evaluation and reporting needs for the program (e.g., ISRP retrospective reports, Report to Congress, and HLI reports), and Bonneville FCRPS BiOp reports. Furthermore, if the PERC moves forward, it would be expected that the council recommendations based on the guidance from this committee would be incorporated in this work.

- Sponsor to participate on the PERC as requested by the Council to assist in developing recommendations of the PERC.

Independent Scientific Review Panel Assessment

Assessment Number: 1988-108-04-ISRP-20120215
Project: 1988-108-04 - StreamNet
Review: Resident Fish, Regional Coordination, and Data Management Category Review
Proposal Number: RESCAT-1988-108-04
Completed Date: 4/13/2012
Final Round ISRP Date: 4/3/2012
Final Round ISRP Rating: Meets Scientific Review Criteria (Qualified)
Final Round ISRP Comment:
Qualification #1 - Qualification #1 - Resolve issues concerning Deliverable #2
Resolve issues concerning Deliverable #2 (update existing StreamNet datasets), as follows: (1) StreamNet proposes to stop updating or to provide only opportunistic updating of some of its primary datasets for an unspecified number of years until data collection activities for the Coordinated Assessment (CA) project are completed. The sponsors need to clarify how this will this impact the Council's Fish and Wildlife Program and other projects and programs that require updated StreamNet datasets to complete their work; (2) A regional discussion on which (if any) data types should be permanently dropped from StreamNet needs to be held; and (3) The sponsors need to clarify whether derived value data being collected for the Coordinated Assessments project meet the needs for reporting High Level Indicators (HLIs) for viable salmonid population (VSP) parameters.
Qualification #2 - Qualification #2 - Design and implement a plan for internal and external effectiveness monitoring
Design and implement a plan for internal and external effectiveness monitoring. Previous ISRP reviews cited "Lack of clarity of who uses StreamNet, site use, and user satisfaction." The sponsors responded that "Site usage and use by agency is reported annually in our annual reports" and that it is difficult to assess satisfaction because it is used over the internet. A very strong rationale for any project is that it is achieving its objectives, and it is important to assess how well StreamNet is meeting the needs of agencies, tribes, and other users. The ISRP suggests that the sponsors provide two letters of reference from each agency working with StreamNet, one from the administrative level and the other from the staff level, outlining progress, improvements, limitations and shortcomings of the approach, and whether alternative forums or approaches might better meet agency needs.
Qualification #3 - Qualification #3 - Provide a report describing (QA/QC) procedures
Provide a report describing in detail the data quality assurance and quality control (QA/QC) procedures used by StreamNet. In the FY 2007-09 review, the ISRP encouraged the sponsors "to complete the draft document describing QA/QC procedures soon." In this proposal, the sponsors state, "We hope to develop a report describing the entire QA/QC process more fully in the future." The lack of well-documented QA/QC procedures reduces confidence in the quality of StreamNet datasets and data management systems.
First Round ISRP Date: 2/8/2012
First Round ISRP Rating: Meets Scientific Review Criteria (Qualified)
First Round ISRP Comment:

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The StreamNet project has clearly played an important role in providing information to regional programs and meeting regional objectives. However, the proposal would be improved by inclusion of a description and evaluation of project significance with respect to regional coordination of data management under the Council's Draft Monitoring Evaluation Research and Reporting Plan, Subbasin Plan objectives, Council's 2006 Research Plan, and other regional plans.

The problem statement focuses on the difficulty that StreamNet has in acquiring data from the Basin's management agencies. If regional data networks develop, however, the need for a central facility like StreamNet may decrease. A long-term goal is to move regional data dissemination toward a distributed “Exchange Network” model. The proposal would be improved by a more detailed description of this model. As those capabilities are developed, StreamNet is working with the Coordinated Assessment project to provide critical metrics, and the ISRP supports this effort. 

The primary objective of StreamNet, to provide easily assessable regional data for agencies and others, is highly important. StreamNet appears to be succeeding in this objective, but it could have provided statistics on numbers of users of the database from various agencies (see Qualification #2). Although these data are reportedly located in annual reports, these data should have been summarized in the proposal. 

The StreamNet website encourages submission of datasets that may be of interest to others. Although the proposal mentioned a number of other dataset projects, it was not clear to what extent StreamNet datasets might overlap with other datasets that are made available online. The proposal would be improved by inclusion of a dataflow chart showing how all of the datasets and database organizations integrate among themselves and avoid duplication.

2. History: Accomplishments, Results, and Adaptive Management (ISRP Review of Results)

The sponsors note that Pisces has some errors in their accounting of project reports, and that they have had difficulty in meeting deadlines for the annual reports and quarterly status reports because "input has to be obtained and consolidated from six subcontracting agencies." 

The proposal lists a number of accomplishments, and it describes how it has responded to previous deficiencies. They collaborate with agencies to help standardize data collection so that the data can be stored more readily. StreamNet has responded to most past ISRP comments (see Qualification #3), and they are attempting to fix problem areas as described in detail in the proposal. 

It would be useful to obtain information from active agency participants on StreamNet about the percentages of data from their agencies that find their way into the program each year, as well as their perceptions of how successful and useful the project is and what can be done to increase the quantity and quality of data entered (see Qualification #2).

StreamNet has focused on maintaining and updating a set of fish related data over many years and seems to have done a good job with those data and the information technology available. Better results could have been obtained if data storage in StreamNet had been viewed as higher priority among the states and tribes.

One important negative result relevant to the ISRP’s ongoing resident fish review is that provision of resident fish data to StreamNet is still not an agency priority. The sponsors state, "We are unable to change this situation, given that even when the NPCC Chair requested that we include more resident fish data, he also stated outright that there would be no additional funding to support the effort."

StreamNet devised an internet-based approach to disseminating data that are standardized and georeferenced across agency lines. StreamNet is, however, labor intensive, and with current staffing they have to focus on updating existing data sets, and have little time available to work to locate and standardize additional types of data.

StreamNet is evolving in response to input from agencies and user groups, demonstrating adaptive management for example by working with the Coordinated Assessments Project. However, this will result in delays by three agencies (WDFW, IDFG, and ODFW) in the updating of primary StreamNet databases (see Qualification #1). The ISRP concurs with the sponsors’ statements that a regional discussion is needed on which datasets, if any, to completely eliminate.

About three fourths of the project consists of sub-projects with states and tribes to develop data and databases and make these data available via StreamNet. StreamNet is proposing to expand the project to include the Colville, Shoshone-Bannock and CRITFC member tribes to the project. However, CRITFC’s StreamNet budget was moved to Columbia Basin Fish Accords. The current relationship with CRITFC is unclear. Has CRITFC data not been stored on StreamNet in the past? The proposal would be improved by addressing these issues.

Retrospective Evaluation of Results

StreamNet’s primary past contribution to the Fish and Wildlife Program has been to provide access to summarized, interoperable fisheries datasets collected by the Basin’s fisheries agencies. Data are provide via the project's website (www.streamnet.org) through an online data query system and interactive map applications. The primary data sets include:

  • Anadromous fish distribution (generalized)
  • Resident fish distribution (generalized)
  • Adult abundance in the wild, redd counts
  • Adult abundance in the wild, spawner counts
  • Adult abundance in the wild, dam/weir counts
  • Adult abundance in the wild, estimates of spawner population
  • Hatchery returns (anadromous).

StreamNet’s goal of providing updated data within a year of data collection in the field has not always been met because of delays from internal reviews and in release of data by the agencies that collected the data.

StreamNet also develops and disseminates a variety of other data types including stream network hydrography, fish barriers, protected areas, hatcheries, dams, and other structural facilities, and fish age data. StreamNet also disseminates independent data sets that do not fit the StreamNet data exchange format and are archived in the Data Store (www.streamnet.org/datastore_search.cfm), where they are searchable and downloadable, along with metadata and functions as a data archive, as suggested by the ISRP (ISRP 2000-3). They also provide source documents for all data contained in the StreamNet database to the StreamNet Library.

Additional past contributions include:

  • Initiating development of internal database systems in some partner agencies;
  • Responding to data-related requests from participants in the Fish and Wildlife Program, for example their lead role in developing an initial draft Data Exchange Template for the Coordinated Assessments project and hiring specialists to assist agencies in describing data management gaps and needs, which was used as a template to help the state and tribal agencies determine their capacity to locate and provide the specified indicators and metrics (www.pnamp.org/sites/default/files/ca-lessons_learned_report-2011-05-17.pdf);
  • Redesign and ongoing implementation of an online data query application for the StreamNet database (http://test.streamnet.org/); and
  • A guide on data sharing, Considerations for Regional Data Collection, Sharing and Exchange (Schmidt and the StreamNet Steering Committee (2009), and a condensed ‘top ten list’ format (StreamNet, 2010). Many of the concepts discussed in the data sharing guide were adopted by the Coordinated Assessments project and in new data-related requirements for BPA contracting.

StreamNet results can be evaluated in part by review of their responses to issues raised in the past at workshops and by groups like NED, PNAMP, CBFWA and the ISRP, as follows:

  • Timeliness of data updates: Addressed in part by encouraging and supporting agencies to develop internal database systems and initiate agency-wide approaches to data management.
  • Lack of data from some tribes: Addressed in part by initiating work with the CRITFIC member and other tribes.
  • Need for additional types of data: Addressed in part by assisting agencies with developing internal database capacity to allow more efficient data management and sharing.
  • Lack of derived data: Addressed in part by assisting agencies in development of a data exchange network approach, whereby agencies other than StreamNet provide derived estimates and supporting data via web services.
  • Lack of standardized field sampling: Addressed in part by collaborating with management agencies and regional scale entities to coordinate what is monitored and how, e.g., the Coordinated Assessments project.
  • Not enough resident fish data: Not addressed due to a lack of funding to support the effort.
  • Lack of standardization in data collected, collection methods, and data standards: Addressed in part because data collection issues were not addressed, but data standards were addressed through the Data Exchange Format (www.streamnet.org/reports_pubs.cfm.
  • Unclear priorities for the types of data provided through StreamNet: Addressed in part by organizing a workshop with CBFWA in fall 2006 and prioritizing abundance data to support the Status of the Resource report, but regional consensus recommendations were not developed.
  • Lack of clarity of who uses StreamNet, site use, and user satisfaction: Addressed in part by reporting site usage in annual reports, but user satisfaction could not be determined via an online user survey.
  • Lack of description of QA/QC procedures: Addressed only by a brief description in the 2012 proposal.
  • Lack of adequate metadata: Addressed in part by working through the PNAMP Metadata Work Group, but limited primarily to general descriptions due to lack of original metadata with the data submitted by data-collection agencies.
  • Justification of the amount of staff and infrastructure: Addressed in the 2012 proposal.
  • Description of the project interface: Addressed in StreamNet's Web Query System User’s Guide (www.streamnet.org/wqs_guide.html) and user guide for the map interfaces (map.streamnet.org/website/bluesnetmapper/HelpFile.htm).

The StreamNet strategic plan (Schmidt 2009) emphasizes developing internal database capabilities within the data source agencies and a distributed network for dissemination of data. As the distributed network develops, the need for a central location for data management should be evaluated by the agencies and entities collecting, disseminating, and using the data, as centralized coordination and management of such a distributed network will require considerable resources. 

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (hatchery, RME, tagging)

The primary relationships described include those with PNAMP, CBFWA, and CRITFC, and there appears to be good coordination and collaboration with these entities. This section of the proposal would be improved by more information on BPA-funded projects and project numbers, including data collection projects, that are necessary for successful completion of proposed StreamNet objectives and deliverables. The sponsors note that agencies are the major limiting factor for StreamNet, that is, the efficiency with which data can be located and accessed within agencies and converted to regional standards. Clearly, provision of data to StreamNet is not always a high priority for management agencies.

The ISRP supports the StreamNet strategic plan (Schmidt, 2009) that emphasizes providing more support for developing internal database capabilities within the data source agencies. However, progress has been slow in some agencies, because it is not viewed as high priority. Although not discussed in this proposal, the Tribal Data Network proposal (#200850700) appears to be the companion proposal to facilitate this move.

The ISRP supports the increased focus on providing derived estimates and assisting data source agencies with development of internal data systems for storage and dissemination of data.

The sponsors describe important emerging limiting factors with respect to regional-scale data coordination in the proposal. Concerns are being addressed to at least some extent by the Coordinated Assessments project. Nevertheless, the ISRP concludes that there is a need for improved coordination of data management at the regional scale that will necessarily involve discussions with the Council and BPA, as well as agreement and support from the states, tribes, and other agencies and entities involved in providing data to StreamNet.

4. Deliverables, Work Elements, Metrics, and Methods

The ISRP supports StreamNet’s proposed new work in several areas: (1) to help develop indicator and metric data for the Coordinated Assessment project; (2) to collaboratively establish data needs and priorities, agree on standardized formats and definitions for sharing, and initiate sharing of the selected data as routine operations; and (3) to revise the data query system to improve user friendliness, increasing speed, and linking tabular and GIS data.

The ISRP is concerned that some data currently collected (Deliverable #2) will be put on hold until a distributed network can be established (see Qualification #1). Because the primary focus of regional data coordination is the Coordinated Assessment project, it is not likely that Deliverable #2 can be met within the period covered by this proposal. Although this is reflected in a reduction in funding for this deliverable, as discussed earlier in the proposal there is a need for regional consensus on this issue and whether some Streamnet datasets should be completely eliminated.

Among the three work elements listed for this project, none has metrics. The guidance given on the proposal submission site emphasizes an “emphasis on outcomes,” discussion of hypotheses, quantitative (and qualitative) measures and metrics, summary tables and graphs, and trends. Data management activities are amenable to scientific analysis. Key questions, hypotheses, relationships, data gathering and analysis, reporting of results, and revisions based on what is learned are expected. Greater emphasis on trying to measure outcomes and include in the proposal an adaptive management framework for designing, implementing, evaluating, and revising data management activities is recommended.

A log-in system to StreamNet might allow the program to more effectively evaluate public usage.

4a. Specific comments on protocols and methods described in MonitoringMethods.org

The project is 100% RM&E and Data Management with three work elements:

None of these work elements is associated with metrics in MonitoringMethods.org; however, it would be useful for retrospective evaluation of project results to develop quantifiable metrics or these work elements that could be used to track trends in data management project results.

Qualifications

In summary, StreamNet’s objectives are clearly stated, and most deliverables appear to be on track to meet the objectives (see Qualification #1). The project is of benefit to the Fish and Wildlife Program. However, an effectiveness monitoring plan needs to be developed and implemented (Qualification #2), and QA/QC methods are not documented in sufficient detail (Qualification #3).

The ISRP supports the project’s shift in focus to increased emphasis on derived estimates, such as indicators and metrics to support regional scale reporting under the ESA, as per the Coordinated Assessment (CA) project (www.pnamp.org/project/3129). Acquiring data from the tribes is a major step forward. The ISRP supports the strategic plan (Schmidt, 2009) that emphasizes developing internal database capabilities within the data source agencies and a distributed network for dissemination of data. However, the need of a central location for data should be revaluated as a distributed network system is developed. Coordination and management of such a distributed network will require considerable resources. The ISRP concludes from information in the proposal that there is substantial room for improvement in regional coordination of data management in the Columbia River Basin. This will necessarily involve discussions with the Council and BPA, as well as agreement and support from the states, tribes, and other management agencies and entities involved in collecting and providing data to StreamNet.

Modified by Dal Marsters on 4/13/2012 12:19:03 PM.
Documentation Links:
Review: FY07-09 Solicitation Review

Council Recommendation

Assessment Number: 1988-108-04-NPCC-20090924
Project: 1988-108-04 - StreamNet
Review: FY07-09 Solicitation Review
Approved Date: 10/23/2006
Recommendation: Fund
Comments: Interim funding pending Council review of data priorities. Council draft recommendation is to hold to FY 2006 level. ISRP fundable (qualified): address in programmatic issue in the decision document.

Independent Scientific Review Panel Assessment

Assessment Number: 1988-108-04-ISRP-20060831
Project: 1988-108-04 - StreamNet
Review: FY07-09 Solicitation Review
Completed Date: 8/31/2006
Final Round ISRP Date: None
Final Round ISRP Rating: Meets Scientific Review Criteria (Qualified)
Final Round ISRP Comment:
The need to standardize data protocols continues. The central role of BPA in funding data collection in the basin should provide a mechanism to require standardization of data reporting, protocols, and methods. However, the sponsor's response provided reasonable explanation of the position of StreamNet relative to the data standards issue and the difficulties of moving the issue forward based on voluntary agency agreement. The proposal describes past, present, and future features of StreamNet in a subdued manner. Nevertheless, the importance of having data development and dissemination activities in the basin is clear.

StreamNet is complex, and the staff is attempting to meet the needs of a diverse audience. As we learned in the 2000 review, this is not easy. Workshops to establish priority needs, better mechanisms to track use and effectiveness, documented QA/QC procedures, moves toward standardization without offending clients (both users and suppliers of data), specific data development on hatchery releases, and other topics that the ISRP questioned appear to be underway and in the right direction. For example, the ISRP encourages the sponsors to complete the draft document describing QA/QC procedures soon.

The base program is fundable and serves an important role in the Basin. The ISRP strongly supports expanding the tasks and objectives of StreamNet to provide the most utility to the basin. The "Fundable" recommendation is qualified, however, because the program needs to develop measures of effectiveness and assess its impact in terms of those measures. The project should have in place a system for monitoring and evaluating its performance. The program still needs to develop more in-depth measures of monitoring effectiveness and assess its impact in terms of user satisfaction. Use of the services should be documented, and more focus should be placed on outputs rather than inputs. A systematic way of evaluating effectiveness is needed. Who are the users? Were these users satisfied? Is tracking software used (e.g., Web Trends)? The sponsors should provide some evaluative performance information to address these questions.

The ISRP recommends that the project receive an independent project review on the quality of its service delivery soon. The sponsors are receptive to the idea of an independent performance review.
Documentation Links:

Legal Assessment (In-Lieu)

Assessment Number: 1988-108-04-INLIEU-20090521
Project Number: 1988-108-04
Review: FY07-09 Solicitation Review
Completed Date: 10/6/2006
In Lieu Rating: Problems May Exist
Cost Share Rating: 3 - Does not appear reasonable
Comment: Data needs/coordination are authorized/require cost share appears low.

Capital Assessment

Assessment Number: 1988-108-04-CAPITAL-20090618
Project Number: 1988-108-04
Review: FY07-09 Solicitation Review
Completed Date: 2/27/2007
Capital Rating: Does Not Qualify for Capital Funding
Capital Asset Category: None
Comment: None

Project Relationships: None

Name Role Organization
Pam Kahut Administrative Contact Pacific States Marine Fisheries Commission
Stan Allen Supervisor Pacific States Marine Fisheries Commission
Mike Banach Interested Party Pacific States Marine Fisheries Commission
Chris Wheaton Project Lead Pacific States Marine Fisheries Commission
Thomas Pansky Project Manager Bonneville Power Administration
Zachary Penney Interested Party Columbia River Inter-Tribal Fish Commission (CRITFC)
Zachary Gustafson Env. Compliance Lead Bonneville Power Administration