This is a large, exciting, and important project in the Willamette Valley that has focused primarily on land acquisition and project initiation, possibly to the detriment of higher level (subbasin) planning and the establishment of a coordinated monitoring and evaluation plan for the subbasin.

Qualifications:
1. Sponsors should follow The Nature Conservancy (TNC) lead on organizing and documenting partnerships, progress, and development of a coordinated monitoring and evaluation approach. This can be done in-house with ODFW or by contracting to qualified partners.

2. Sponsors should plan to provide the ISRP with additional information on the progress of the Willamette Subbasin projects over the next several (3-5) years. This report should come in two forms:
a. Provide the ISRP with an Annual Summary Report. This should include a short summary section for the entire Willamette Basin project that succinctly captures work accomplished for the year and describes significant milestone events, or progress toward milestone events. In addition to the brief Summary section, the Annual Report should include 3-5 page reports for each property/project that summarize accomplishments for the year in both narrative and summary tables for that specific project. Summary results can be presented in tables and graphs, but should not include large tables of raw data. The ISRP is primarily interested in biological results and a synthesis of the M&E results for each property/project.

b. Sponsors should plan to present a day-long update to the ISRP on the Willamette Basin projects after two more field seasons of progress (perhaps scheduled for the fall of 2010 or winter of 2011). The presentation should summarize accomplishments and lessons learned to date and what has been learned by the development and implementation of the coordinated monitoring and evaluation system described in Qualification #1.

These steps will help document the success or failures of this important process and project (or more appropriately, set of projects). The Willamette basin project is an important pilot of landscape scale restoration and could serve as an example for others, if documented appropriately and reviewed at regular intervals.

Summary
In their response, the sponsors provided detailed information about each project that provided a more comprehensive view of its accomplishments. There is no doubt that the collaborations they have developed are key to what they have been able to accomplish. With respect to monitoring, there is a basic level of accountability wherever NPCC invests O&M funds; however, that need not require comprehensive effectiveness monitoring on every parcel. At this point, it appears that management of many parcels is not in the hands of the sponsors, which makes monitoring issues appear secondary to the urgency of securing expensive properties in the face of rapid urbanization. In this circumstance, contracting monitoring might be a useful approach. Following The Nature Conservancy approach may adequately resolve this issue. Ongoing work and partnerships with the NHI (Northwest Habitat Institute) may also prove fruitful.

Reviewers acknowledge that the project, in spite of a start date 15 years ago, has in fact only been operating with appropriate funds for the last five years or so. Efforts during that time have focused on acquisitions; consequently, the time is now ripe for development of a coordinated monitoring and evaluation plan for the various components in this Willamette basin landscape restoration project.

Management planning usually includes specifying a desired future condition, which becomes the basis for monitoring comparisons. Monitoring needs will vary depending on the condition of the site at the time of acquisition, and it is key to have an assessment of conditions at acquisition; not, as many suggest as a baseline, but as a point in time from which to plan and evaluate future management. The actual assessment and monitoring need not be by the sponsor as long as the sponsor is involved in the decision process and accepts accountability. The current suggestion to delay monitoring works against the desperately needed accumulation of knowledge of the functions of these ecosystems and means to sustain or restore them.

These considerations suggest separating acquisition from O&M and from M&E as three separate projects. Other sponsors have done this successfully with multiple partners and properties and working under similar development pressure. The acquisition component is scientifically sound, but the O&M requests, in light of the monitoring record, need further work.

Similarly, sponsors should look to the Oregon Conservation Plan for examples of meaningful monitoring and for a model of an adaptive management approach. A regional approach for monitoring and evaluation of habitat (and wildlife responses), such as work presently being conducted by the Upper Columbia United Tribes (project #200800700), is being developed A comparable program seems warranted for the Willamette.

Finally, the ISRP appreciated the Project’s "on the ground' summaries (p.6 - 44) and the project histories RM&E (p. 44 - 53) and the list of the proposed new projects (starting on page 53). The ISRP was not clear on why Camp Adair is a target as a new project; the argument that Camp Adair provides potentially critical linkage appears weak ecologically unless Camp Adair is a center of biological diversity, which was not indicated.

The project needs to have a more detailed M&E plan and summary of M&E results and adaptive management to date. The project is 15+ years old and does not have a well-documented set of results that reflect systematic accomplishments. This is a critical deficiency. The lack of monitoring, (or monitoring results) precludes our ability to determine if management is scientifically sound. This is a stark contrast to the obvious scientific rigor applied to developing the acquisition strategy and criteria. It does not appear that there are plans to remedy the situation, leaving the ISRP in the same situation as 2007, supporting the acquisition component of the project, but not the O&M, until a credible monitoring report and future monitoring strategy and plan are submitted. In its last review the Council reiterated this concern in their funding decision: "At the same time, the recommended expense funding for operation and maintenance should be considered interim, with a final recommendation pending the outcome of the wildlife land operation and maintenance review. In addition, the project sponsor should make special efforts in its annual reports and in the next project review cycle to report anything that it possibly can as to the benefits to fish and wildlife actually realized by the project that can be gleaned from the ongoing monitoring and evaluation efforts in the Willamette basin, as well as some assessments of what it would take to know more about the benefits to fish, assuming it is even possible." No monitoring results are reported in this proposal, even for the earliest acquisitions, although monitoring is listed as an activity in several project histories and management plans. For example, the South Pasture Plan was created in 2001 and included 5 years of monitoring by the subcontractors. Was this done? Could it be reported here? While the project is complimented for its many effective collaborations, this may also complicate accountability, perhaps being part of the monitoring issue. The sponsors are encouraged to consider how their collaborations might be an asset in the task of summarizing biological effectiveness to date, and ensuring adequate monitoring of biological results of Program investments going forward. This is a large, complex project, growing larger and more complex over time. Given the continuing differences in scientific merit between the acquisition and O&M components, it might be useful to split this into two separate projects that could be funded separately. The monitoring issue might then be more easily addressed. The sponsors might consider contracting out the monitoring, or adding a monitoring specialist to their team. This project is a priceless opportunity to apply adaptive management to unique and rare landscapes, the results of which will greatly leverage the investment in acquiring and managing these lands for the benefit of fish and wildlife. The Willamette subbasin is 94% privately owned (surface area), housing a majority of Oregon’s human population. This project may be a model for addressing fish and wildlife goals associated with a large river in a human dominated landscape, offering important lessons for the future of the entire region. 1. Technical Justification, Program Significance and Consistency, and Project Relationships The Willamette Wildlife Project is a network of over a dozen terrestrial habitat projects in the subbasin. The subbasin is 94% privately owned (by surface area coverage) and harbors a majority of the Oregon human population. The primary goal and outcome will be to secure, conserve, and restore habitats within the subbasin. The project provides excellent background with lots of justification with references for need of habitat restoration in the Willamette Valley Subbasin and addresses goals in the Willamette Subbasin Plan and the Oregon Conservation Strategy. Relationships and linkage to regional programs and plans are also described in good detail including the 2000 Fish and Wildlife Program, the 2004 Willamette Valley Subbasin Plan, the Oregon Conservation Strategy, the BiOp, and others. The sponsors list what appear to be all vertebrate communities in the subbasin as "primary" or "other" species (i.e., all salmon, steelhead, lamprey, resident chub, and all wildlife). This is probably true at some level if the ecosystem functions as it should. Consequently, there is a lengthy list of "objectives" and work elements outlined in the project. Most are broadly stated goals (e.g., "Connect Diverse Habitats", "Increase Interaction between Rivers and Floodplain) rather than measurable objectives and include work elements with coarse-filter activity targets (such as acres or stream miles treated). The Willamette Wildlife Project is a network of terrestrial habitat projects in the Willamette subbasin. The projects are loosely related and function more as a “program” than a project making it difficult for the ISRP to review it effectively. The project is large and diffuse. Reviewers were left to wonder how the larger project all fit together. The Willamette Subbasin Plan was notable for its extensive analyses of floodplain habitats (historic and present) and for its attempt to prioritize actions and opportunities. That approach is evident in the Willamette Wildlife Project; however, the reviewers had a hard time understanding the scope and details. Questions included: What is the strategic planning needed for? How does the project fit together? A great strength of this project is the numerous collaborations that sponsors have developed. The cost sharing is one significant result, as are the number of management activities contracted out. Integration of this program with the Oregon Conservation Strategy is another synergy. The flip side of this, however, is sorting out which activities serve which objectives, when the objectives of programs are not completely overlapping. This complicates accountability, which may be part of the monitoring issue. The sponsors are encouraged to consider how their collaborations might be an asset in the task of summarizing biological effectiveness to data, and ensuring adequate monitoring of biological results of Program investments going forward. 2. Project History and Results This section is incomplete – plans, ongoing accomplishments (mostly from various Subcontractor's status, accomplishments, and planning reports) are described in some detail, but documentation/reporting of results very weak. Detailed project histories and "results" are presented as actions, rather than outcomes of actions. This is suitable to finite tasks such as acquisitions, but problematic for management activities. See M&E below. The project has been ongoing since 1992. Monitoring and results reporting is a critical element for a project of this duration. After more than 10 years, there should be a substantial reporting of results to date. These should include both coarse-filter changes to the habitats (acres treated), as well as a fine-filter accounting of focal wildlife species responses. We searched Bonneville’s annual reports on the web and found only four Willamette reports done since initiation of the project in 1992, including a very detailed monograph on western painted turtle status (good to see but hardly covering all of the other habitat restoration activities). The 2005-06 progress report to BPA was the only one that included useful information on recent results of habitat restoration work and recent acquisitions. Why wasn't this report cited or used in the proposal? 3. Objectives, Work Elements, and Methods Objectives are only stated as broad goals (Willamette Subbasin Plan Goals?). Revising work elements into a series of objectives would make more sense. Methods and metrics were almost non-existent for Objectives 1 and 2, and Objective 3 mostly describes the revised HEP related plans and development activities. Objectives and work elements are lengthy and diverse, which reflect the patchwork of habitats included in the project. Objectives are broad and general, for example "pursue acquisitions," rather than something like "secure acquisitions sufficient to add X HU/year to program." There is a broad, un-measurable monitoring objective and a work element for a database, but no work elements to collect, analyze or evaluate monitoring data. For sites with management plans, the plans include monitoring. No work elements are in sufficient detail to offer methods where the scientific basis might be evaluated. There is a $30,000 work element for acquisitions, but this does not appear in the actual budget. Later a note states: "Approximately $800,000 for both FY2010 and FY2011 for the expense side of the budget (Personal Services, Supplies, and restoration subcontracts. Approximately $3,000,000/yr for easements and acquisitions.)" Are no acquisitions planned until the out years? 4. M&E Monitoring and results reporting is a critical element for a project of this duration. After more than 10 years, there should be a substantial reporting of results to date. These should include both coarse-filter changes to the habitats (acres treated) as well as a fine-filter accounting of focal wildlife species responses. Broad general plans for future monitoring are presented but not linked to any particular projects or practices. Compared to the level of detail presented in most sections, these plans are not credible. Meanwhile, restoration techniques appear to be being replicated from site-to-site, with no validation as to their success in restoring habitat to benefit fish and wildlife. Further, no scientific justification for selection or use of any restoration strategy is offered. This region is under-mitigated, and this project is an ambitious effort to remedy that situation. Not monitoring the results of management practices in these unique habitats means there is limited opportunity to validate and improve restoration practices as the project continues. Mistakes will likely be repeated, and opportunities will be missed. This is an economic as well as scientific weakness. It appears that M&E is to be planned, developed, and coordinated in upcoming workshops. The outline for this looks good and hopefully will get done. Until then, this is a failing point of this project. Consequently, the ISRP requests a more complete description of how progress will be monitored. Measurable objectives are not always listed and hinder future monitoring. For instance, the objective “remove exotic vegetation,” where, when and how, what species? Work elements are often stated in terms of amount of habitat obtained or restored rather than in terms of fish and wildlife outcomes. They need a region-wide reporting program. Defenders of Wildlife have an on-line program that they are encouraging reporting to, but no project results were found when it was searched. There are frequent references to HEP and two work elements related to "improving" HEP. While not presented as monitoring, there seems to be more commitment to HEP than to effectiveness monitoring. Sponsors really do need both, one for accounting and the other for scientific validity. Appendix #6 discusses the HEP issue in the Willamette Valley, primarily its history, calculation and problems. It is not really clear why this is included, except perhaps in support of Appendix 7: Wildlife Monitoring and Evaluation (Wildlife Advisory Committee-CBFWA) 2008, which appears to be a recommendation for the Program Amendment. None of this is relevant to the problem of effectiveness monitoring. The ISRP requests that the authors address fish and wildlife responses. Appendices include #5 which is a discussion of monitoring associated with the Oregon Conservation Strategy (OCS). While there is logical overlap and apparent efficiency in combining these efforts, it appears that the OCS team is years away from actually being able to monitor on the ground at the resolution currently needed for this project. This need not constrain current monitoring expected by the Fish and Wildlife Program. What the Program needs is considerably more explicit and site specific than what may be needed by the OCS.

The proposal is for a large-scale effort in habitat acquisition, enhancement, restoration and management. Consistent with the Subbasin Plan, State plans, and conservation NGO strategies, this proposal appears to be part of a well-coordinated, regional effort. This project (really a program of many interrelated projects) has been going on for 13 years. There are 13-14 ongoing projects that include routine restoration or maintenance activities, but rarely monitoring. New projects expand the scope of the program along the same trajectory.

The land acquisition portion of this proposal is fundable. It defines the problem of land acquisition where only small parcels are available and the methods for prioritizing, selecting, and acquiring properties are sound.

Beyond the pending acquisitions, the remainder of the proposal is fundable in part. The ISRP recommends funding for FY07 only to allow an assessment of past work. Future funding of the active management component of the budget should be contingent upon a meaningful quantitative and qualitative analysis of project accomplishments to date, in terms of benefits to fish and wildlife. Formal monitoring and informal observational results should be synthesized and analyzed in terms of lessons learned and future modifications needed in management and /or monitoring procedures.

The preliminary review requested a response to concerns about monitoring and evaluation: "The project history provides some, but not sufficient, assessment of progress that the ISRP requested last year [referring to the ISRP's review of this project in the Provincial and FY00 reviews]. The numerous objectives in the proposal will require significant administration to track progress of overall project. Timelines are not clear, nor are metrics for future assessment of accomplishments. The ISRP requests a more complete description of how progress will be monitored. Measurable objectives are not always listed. For instance, the objective ‘remove exotic vegetation' may not be achievable by any currently known means. Work elements are often stated in terms of amount of habitat obtained or restored rather than in terms of fish and wildlife outcomes. The ISRP requests that authors address fish and wildlife responses. The ISRP believes that management plans have been completed for some sites and would like to see a description of monitoring methods. Procedures have been available during this project that should now be driving a feedback loop that is not apparent. The ISRP requests a description of how this loop functions.

The proposal identifies some M&E efforts as part of work elements, but does not provide enough details to evaluate. Even implementation monitoring would be difficult given the information provided. Many proposals do not include metrics, and it appears monitoring is just now being addressed with initial development of reference sites and procedures. Objectives of the analysis, the sample design, and data to be collected should be clearly described in advance of projects. In addition to quarterly reports, strategies for sharing successes and lessons learned with others involved in similar mitigation activities is recommended. The response should describe the data to be generated, stored, or analyzed."

Monitoring is a critical element for a project of this duration. The response provided indications that the sponsors are aware of the need for effective monitoring and evaluation in terms of benefits to fish and wildlife but that they feel constrained by logistics, other reporting requirements, and by the difficulty of detecting changes to fish and wildlife during the early stages of projects and/or on small scattered parcels. The ISRP is not convinced by their argument as many other wildlife projects have set up monitoring that is consistent with what is requested by the ISRP. The Albeni Falls Monitoring Plan is one that has been reviewed by the ISRP and found exemplary.

Sponsors are currently involved in development of "a revised HEP/Habitat value method that is based on structural and compositional values (ecological components) of mitigation sites and an evaluation of risk factors (such as presence and abundance of exotic invasives) to those sites." The use of HEP (revised or not) may be required for accounting purposes, but it is not seen by the ISRP as an effective monitoring tool

The monitoring plans included in the response are monitoring objectives in general terms: "species population surveys or estimates will be completed each year for ...(species)." No methods or goals for a population are included or even if particular life stages or seasons are targeted for monitoring. In the case of South Meadow, monitoring includes elements such as acres treated, relative cover of weed species, but not who will do this when, where the data will go, when it will be evaluated, by whom or what will trip an adaptive management response. In the future, it may be difficult to use these data to improve restoration approaches.

The sponsors are no doubt doing a difficult job with many uncertainties. The ISRP encourages the sponsors to view monitoring as a means of documenting project success and learning from projects in order to improve the success of future efforts. Reference is made to informal information sharing among project participants as an informal feedback loop. While useful, this does little to institutionalize or document lessons learned in an environment where the authors note that little validated restoration technology exists. Synthesis and publication of monitoring and evaluation data will maximize the value of project investments in terms of future benefits to fish and wildlife.