Comment:

The ISRP believes the hiring of a professional librarian is a positive development. Ms. Wilkerson demonstrated a comprehensive knowledge of library resources during her presentation as well as during the follow-up Q&A with the ISRP.

The proponents need a multi-year plan as soon as possible, and the ISRP is looking forward to being able to comment on it.

1. Objectives, Significance to Regional Programs, and Technical Background

This proposal requests continued funding of a 23-year-old, regional project that stores, manages, organizes, and provides access to fish and wildlife literature on the Columbia River Basin and region. Such a library is vitally important for researchers and technicians, particularly for those not having access to the diverse informational resources provided by academic/university libraries.

A particularly useful aspect is the focus on grey literature (i.e., academic theses and dissertations, consultant reports, government documents, conference and meeting proceedings, working papers, and organizational development documents) which is rarely published and often not readily accessible to most users. The library is also an important regional resource providing ready access to a wide range of publications.

The project has a comprehensive set of five objectives. These are described in detail but are qualitative, with no projected completion dates, making tracking of implementation and effectiveness difficult. The ISRP strongly urges the proponents to develop quantitative, time-bound objectives as part of the multi-year plan.

2. Results and Adaptive Management

The ISRP was surprised to read that "The greatest stumbling blocks in the first three months has been the disorganization of the physical and virtual spaces. While there is a wealth of materials, the systems to receive and prioritize them remain somewhat of a mystery. The staff have been forced to reconstruct and reintegrate the current actions with past knowledge and systems." From the presentation to the ISRP, progress appears to have been made on this issue, though details on the strategy were not discussed. Could a steering committee be assembled to assist with these efforts moving forward?

The ISRP assumes that the proponents are aware that the need to store grey literature seems to be diminishing as many organizations now routinely publish such documents digitally. This was a general issue identified in the 2012 ISRP Review—the need for increased coordination to minimize duplication of efforts with other data and information management projects. The ISRP acknowledges that there has been an effort by the proponents to improve coordination, which should continue to ensure the library is prioritizing and making the best use of declining resources.

The project has generally met all work items and deliverables in a timely and competent manner. Nevertheless, as with many other data and information sharing projects, there is no adaptive management (AM) plan and the elements needed to support it. This was a qualification in the 2012 ISRP Review (ISRP 2012-6), "A greater project emphasis on scientific components, measurement of outcomes, and development of an adaptive management framework for designing, implementing, evaluating, and revising data management activities would help to resolve such issues and to identify scientific components of planned future growth." There is little evidence that this is being meaningfully addressed as the project moves forward.

Some recent progress on project evaluation and adaptation is underway. In 2018, the project began collecting and analyzing baseline metrics to examine how the Library collection is used. These metrics will drive future library services. Additionally, internal and external assessments by the library team have identified a number of needed changes and developed initial elements of an outreach strategy. The proposal notes, "The Library will embark on a robust User Assessment in 2019 and 2020 to better define user groups, field and research needs, technological and access concerns, and gather input on direct improvement." The ISRP looks forward to learning about the survey's outcomes and about the resulting changes to library operations.

However, with the development of a multi-year plan, the ISRP expects to see a formal AM process that is responsive to both internal and external issues.

3. Methods: Project Relationships, Work Types, and Deliverables

The library's professional staff appears to take pride in a well-run operation and to proactively search for ways to improve the scope and quality of services. Despite growing demands for expanded services, it appears that potential budget constraints may limit future operations, which would not be helpful to the Fish and Wildlife Program. The ISRP continues to urge researchers and managers to publish project results in the professional literature; having a vibrant library is central to this goal.

Finally, the ISRP supports the movement toward evaluating the feasibility and cost-effectiveness of moving the Library catalog into the Cloud. Cloud computing would allow a reduction in hardware and IT expertise necessary to "hold the collection details." Please keep the ISRP informed of progress as well as any issues that might arise.

1. Purpose: Significance to Regional Programs, Technical Background, and Objective

The sponsors explain project significance primarily with respect to BiOp RPAs. This section would be improved by a more detailed explanation of significance to the Council's Draft Monitoring Evaluation Research and Reporting Plan, Subbasin Plan objectives, Council's 2006 Research Plan, and other regional plans. This might simply be a matter of reorganization of proposal content, for example the significance of this project to the Council's 2006 Research Plan is described in the Problem Statement.

The project serves an important role in providing library services to customers throughout the Pacific Northwest. The proposal would be improved by inclusion of information on total number of customers and a breakdown into different categories, for example public, state agencies, and tribes.

In the problem statement the sponsors state, "A small percentage of the total number of documents produced by participating agencies are submitted as source documents by the data compilers employed directly or indirectly by the PSMFC StreamNet Project." This is surprising to the ISRP given that this is the primary scientific component of the Streamnet Library. This issue needs to be addressed during contracting (see Qualification #1).

With respect to Objective #2 (Support development of document repositories to improve efficiency of reporting and tracking research in the basin), the sponsors state, "The ability of the Assistant Librarian to locate documents would be significantly enhanced by the development of digital document repositories in participating agencies." The ISRP agrees, however, this would likely diminish the need for costly centralized library services. An independent evaluation of current and future needs for centralized library services would be useful.

2. History: Accomplishments, Results, and Adaptive Management (ISRP Review of Results)

According to the sponsors, the most productive accomplishment is locating and providing documents and other information to patrons in the Pacific Northwest with an average of 3000 requests annually, over 12 requests per business day, and many requests fulfilled with on-demand digitization of documents. The proposal would be improved by a breakdown into different categories of users both inside and outside the Basin.

The library plays a significant role in storing metadata for StreamNet and their other target customers, in providing a depository for materials from some state agencies, and in general making grey literature available. However, identification of scientific components to the project and a greater project emphasis on measurement of outcomes and development of an adaptive management framework for designing, implementing, evaluating, and revising data management activities is needed (see Qualification #1).

The library has a website which is easy to use and has provided access to journal literature via their journal collection, subscriptions to electronic journals, and full-text research databases; all necessary services for researchers in the Columbia Basin who do not have access to a research university library. The Library is expanding their collection of electronically available documents. However, the project might benefit from improved coordination with other projects and entities providing similar services (see Qualification 2).

ISRP Retrospective Evaluation of Results

The StreamNet Library is a cooperative, regional project that provides access to current and historical literature related to fish, wildlife and related habitat in the Columbia River Basin and Pacific Northwest. The Library provides research services including bibliography development, literature searches, document location and digitization, and metadata development assistance. As part of the StreamNet Project, the library serves as the repository for the StreamNet database source documents.

The project has succeeded in making documents and reports from BPA-funded projects accessible and has cataloged over 5300 electronically available documents. They have developed bibliographies of historical documents and organized current documents so that all species information is collated and can be related geographically. The Library has averaged 3000 requests annually during the past few years with over 12 requests on average per business day for locating and providing documents and other information to patrons in the Pacific Northwest. Many of these requests are fulfilled with on-demand digitization of documents. Several sets of significant research have been made available electronically:

• Northwest Fish Culture Conference
• Columbia River Thermal Effects Study
• 1990, 2001 and 2004 Subbasin plans and assessments
• Survey of the Columbia River and Minor Tributaries
• Inventory & Monitoring of Salmonid Habitat in the Pacific Northwest
• Bibliography on Vancouver Lake Watershed

The Library has also provided significant support for other libraries. While these results support the conclusion that the project’s objectives are being met, stronger emphasis on scientific component(s) of the project and coordination with other projects and entities providing similar services may be needed. Acquisitions and associated services are desirable, but coordination could help decide where collections are being duplicated or the value of having duplicate items. StreamNet Library has other partners that are doing similar activities, and it is likely there are coordination synergies that can be obtained with PSMFC, PNAMP, university, government, and historical archives. Information on projected trends in the rates of growth of paper and digital documents, number and type of users, types of user requests, and percent use of facilities could assist in planning for growth.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (hatchery, RME, tagging)

The identification of scientific components of the project is needed, especially regarding projected growth of the library (see Qualification #1). The proposal would be improved by more specific responses to proposal guidelines. A major limiting factor is that BPA-funded projects, including the StreamNet project, do not routinely provide their reports and publications to the Streamnet Library.

The StreamNet Library's goal of organizing and making available “the entire body of work of the biological research community in the Columbia River Basin” is ambitious. To plan for this growth trends in the rates of growth of paper and digital documents, number and type of users, types of user requests, and percent use of facilities could assist in planning for growth. StreamNet collects considerable data some of which is in the Cataloging Statistics and Reference Statistics Reports. The 5300 digital documents, which are about 17% of the collection and 3000 inquiries per year are a good start. But having trend data would be very valuable for StreamNet planning purposes, as well as retrospective evaluation of project results. Are these data summarized and discussed with cooperating partners? Can more background be given on the "customer" base? What is the rate of growth in physical and electronic users? From what entities do users come? At what rate has the library been collecting paper and digital documents? How are the documents distributed by subject? What subject areas are most requested by users? What is the rate of change in documents loaned or accessed digitally per year?

Mention is made of the increased use of Twitter and Facebook for providing data. What is the driver of this demand for service? Is it the social media aspect of these sites, more related to mobile need to access data, a way of expanding the user base for StreamNet, all or none of these? More background on the data management benefits and costs of this trend would be valuable for planning future commitments in this project.

The proposal says, “We provide access to full-text research databases that are accessible in the library. In FY2012, we will be expanding access to selected locations.” What are the locations that need this service? How was the need measure? What is the potential benefit from localizing the service? Why can the searching not be accomplished through the SteamNet web site and online catalog?

A move is contemplated and more space is needed for future acquisitions. Can the number of documents held by StreamNet be measured in physical characteristics and in terms of amounts of digital storage required? With digital conversion of documents, do the physical documents have to be archived at the StreamNet library, or could they be more effectively and adequately stored off site for archival purposes?

Publication practices and dissemination policies of some agencies appear to limit the ability of the library to acquire information for the broader community. 

Transfer speeds of the website appear to limit the ability of optical character recognition (OCR) reproductions of documents; however, images of documents are available. There is some potential for overlap with the Northwest Habitat Institute (NHI) proposal (#200307200) to implement a GIS spatial library and repository for habitat data and metadata.

4. Deliverables, Work Elements, Metrics, and Method

Deliverables seem quite consistent with the project’s overall objectives. A unique service to the region is their development of historical data sets by searching relevant reports and literature to pass to data compilers.

Further justification for continued expansion of the Library to special collections and materials from outside the Basin would be useful, given the limited funding and need to improve regional coordination within the Basin (see Qualifications # 1 and #2).

Of the five work elements listed for this project, only no. 99 (outreach and education) has metrics, but the work elements are more inputs rather than outcomes. Work element (119. Manage and Administer Projects) is also mentioned in the introductory materials but is not developed elsewhere. More development of the work elements, research methods, metrics to observe outcomes, and reporting goals would strengthen the scientific dimensions of the proposal.

As a data management proposal a scientific component could be added that has an adaptive focus to generate improved guidance, protocols, metrics, and measures as more experience is gained, particularly in analyzing the growth path being charted by StreamNet (see Qualification #1). The guidance given on the proposal submission site places “emphasis on outcomes” that would come from discussion of hypotheses, quantitative (and qualitative) measures and metrics, summary tables and graphs, and trends. The data management process is amenable to scientific analysis. Key questions, hypotheses, relationships, data gathering and analysis, reporting of results, and revisions based on what is learned are expected. Greater emphasis is recommended on trying to measure outcomes and include in the proposal an adaptive management framework for designing, implementing, evaluating, and revising data management activities. This is a project that would appear to benefit greatly from coordination between a wide range of database developers and users. The cost-effectiveness of coordination could be very high.

While this is a data management proposal, this project could substantially benefit from adopting a coordination focus as well (see Qualification #2). The Work Elements are common to those selected in the 19 coordination proposals. The Digital Library Collections plan includes decisions that reflect cost considerations, such as digital scanning resolution and inclusion of color. Would coordination provide benefits in making these decisions? StreamNet has other partners that are doing similar activities. Would coordination with PSMFC, PNAMP, university, government, and historical archives increase effectiveness and provide efficiencies. A private consultant, the Portland Audubon Society, and the Johnson Creek Watershed Council have added materials to the StreamNet collections. Acquisitions are desirable, but coordination could help decide where collections are being duplicated or the value of having duplicate items. Are there coordination synergies that can be obtained with PSMFC, PNAMP, university, government, and historical archives?

In general, the proposal would be improved by more detailed descriptions of specific methods, for example in response to the question, "Please describe the sources from which you are compiling data, as well as what proportion of data is from the primary source versus secondary or other sources?"  The sponsors answered, "We are compiling data from many different sources and include primary as well as secondary sources. The majority of our materials would be considered secondary sources." A specific answer would list individual sources by name or at least general categories of sources with some examples. What are the major secondary sources that provide the majority of the materials in the library? Why does the Streamnet Library website have special pages for only a few collections, for example the Vancouver Lake Bibliography when the Library's "Journals of interest" page lists only a few fisheries journals?

Some of information on the Library’s website is very out of date, for example on the "Suggested Readings & Background Information" page the most recent document listed was published in 1998. The page on "stock definitions" states "Found in the files, dated September 15, 1993, attributed to Larry Everson." If this page is necessary, it could be updated to provide information such as genetic population structure and listed ESUs.

4a. Specific comments on protocols and methods described in MonitoringMethods.org 

The protocols for the three RM&E work elements are published but do not provide adequate guidance on the methods and metrics. The best guidance available is from the ISRP (2007-14:2) “Evaluating the performance of coordination projects is conceptually the same as any other type of project. What is the goal of a coordination project? How will it contribute to the Fish and Wildlife Program? What are the specific objectives of the coordination project and the activities that accompany those objectives? What metrics will be used to measure the contribution of activities toward meeting the project objectives? That is, what are the indicators of success?” The project sponsors can identify metrics that work for the questions and hypotheses included in the proposal.

Qualifications

The StreamNet Library plays an important role in providing library services to customers throughout the Pacific Northwest. The Library’s goal to assemble grey literature, digitize reports and data, maintain journal subscriptions, subscribe to scientific literature search capabilities is very ambitious for a small library, and is proposed to increase substantially in the future. While the quality of services available supports the ISRP’s conclusion that the project’s objectives are being met, stronger emphasis on scientific component(s) of the project (see Qualification #1) and coordination with other projects and entities providing similar services (see Qualification #2) is needed.