Qualifications: HEP is acceptable for accounting/crediting purposes. Although it is not perfect, it is consistent. It's a policy decision to keep using it, not a scientific decision. Additional sites are being surveyed. Lacking habitat suitability indices for most species in the Willamette, CHAP is being used there. In part qualification: There is no scientific basis for expanding HAB and CHAP beyond the Willamette subbasin. Provision should be made that the HAB and CHAP procedures developed under this project be in the public domain.

The response is somewhat helpful, but deficiencies remain that should be addressed in the next proposal submission:

Section B. Problem statement: Technical and/or scientific background is only two sentences – identifying that HEP was developed by the USFWS and that CHAP is being developed by NHI. This section should provide some history and explanation of (1) the loss assessments for wildlife species owing to dam construction and inundation of the terrestrial habitats by reservoirs, (2) how wildlife loss is reflected in Habitat Units (HUs), and (3) how habitat acquisitions are assessed using HEP. This should contain a brief explanation of HEP including how surveys are conducted and how HUs are ultimately determined.

Section F. Biological Objectives. Objective 1 is to increase the number of HEPs by 30%. There should at least be one objective that identifies they are going to be estimating HUs for habitat crediting accounting.

In the methods subsection some mention is made of surveys to initiate HEP, but there is no description of what happens after the field data is collected. The HEP Sampling Design and Measurement Protocols (Ashley 2006) are helpful. Documentation does not explain how CHAP gets to actual numbers for a particular species. What assumptions and calculations are employed? The sponsor is encouraged to look at the Kootenai Operational Loss Assessment as an example of a modeling project where the sponsors present the formulas and assumptions within the "black box", participate in scientific conferences and publish their work in the refereed literature.

Section G. Monitoring and evaluation. This section only states that HEP is not an M&E tool and cannot provide wildlife species response data. There is no discussion in this section on CHAP or HAB. There should be monitoring and evaluation to validate HEP as an accounting tool. There seems to be general agreement between sponsors, the ISRP, and the Council that HEP is not a diagnostic tool for wildlife management and is inappropriate as an effectiveness monitoring index. HEP is monitoring in the sense that it is used as a follow-up to assess HUs since acquisition, but this is implementation monitoring.

Accounting does not substitute for habitat or species monitoring. While the sponsors acknowledge the distinction, many other sponsors need education on the limits of HEP and why it is not effectiveness monitoring.

Section H. Facilities and equipment: The sponsor reported NA (not applicable). They use data loggers, and computers, run analyses and write reports. Where and how is this accomplished? NHI has facilities and equipment that will be used for the CHAP work. This should be briefly described.

Many of HEP’s problems likely could be corrected or negotiated at less expense than further investment in CHAP for use in only one subbasin. When it comes to methodology, the response is slightly more detailed than before, but no more compelling in its promotion of CHAP over HEP. The reply makes HAB and CHAP no more transparent than previously, and does not clearly state that IBIS, which is a necessary element in CHAP, is another NHI proprietary product. HEP is in the public domain.

There is a simplification that goes into HAB before getting into CHAP. The matrix in HAB involves putting 1s in cells and adding them up. With weeds, as one example, you have variable threats, but this gets simplified to where the detail may be lost. Less detailed information is recorded in the HEP approach, e.g., percent tree canopy closure vs. tree species and multiple layers of closure if present (fine feature habitat elements). Apparently, the two approaches theoretically cannot be directly compared because CHAP compares values against an observed reference, while HEP compares against an "ideal." Both models are used to indirectly "predict wildlife responses." Modeling experts should review the CHAP procedure to verify its utility.

The appendix “Habitat Accounting and Appraisal Method (HAB)” was confusing and full of jargon that made it hard to interpret. Specifically, we question the statement (page 1): “Unlike many previous efforts to calculate the wildlife habitat value on a piece of land, the HAB approach does not rely on predictive models of species population or population response to derive intrinsic value. Rather, it is based first and foremost on standardized field inventory of existing conditions within a framework of a robust geographic information system.” The habitat value calculation is derived from two sets of matrices that characterize species-function and habitat-function. Why are the species-function and habitat-function matrices not considered as predictive models of wildlife response?

The Council has incorporated HUs as the measure of mitigation accounting and HEP as the method to estimate HUs. So the argument that the sponsor needs to provide a better method is probably not appropriate.

For both HEP and HAB/CHAP there has been little validation to establish whether they actually do a reasonable job of acting as a surrogate for counting animals or measuring or inventorying specific habitat elements and estimating the capacity and productivity of the unit for specific wildlife species. There are internal assumptions about habitat to wildlife relationships that need validation. Under either HEP or CHAP, habitat scoring can be high for a species without having that species present.

The sponsors did not follow the narrative instructions. Please resubmit in requested format, with attention to recommended page limits. Budget calculations are not needed in the narrative. A number of appendices were submitted as part of the proposal but are never mentioned in the narrative. If they are relevant, link to or quote significant passages and omit extraneous appendices. Some of the appendices go beyond HEP, raising other scientific issues and contradictions. This proposal involved both scientific and administrative issues that will be discussed separately: Science: The HEP/HSI method is scientifically outdated, but it can be used consistently, to make a rough estimate of habitat conditions. There is tremendous value in the HEP survey in creating baseline information. The field component also provides collateral information that can be used for many planning purposes. The problem in the Willamette: choosing to mitigate elk habitat with habitat for another species such as a butterfly, is primarily a policy decision, which HEP may be able to inform as well as any other approximation. One advantage of HEP promoted in the proposal is its flexibility to be used on different ecosystems, but the CHAP discussion seems to imply that HEP cannot do this? Appendix A, the HAB primer pages 42-46, shows how to convert between HEP HU and “HAB currency” used in CHAP. This conversion suggests that the methods are interchangeable. The CHAP issue has been recently addressed in detail by the ISRP, and we remain doubtful of its scientific superiority. That said, the use of GIS and reference sites are both scientifically valid tools for a more precise HEP. If use of CHAP is scientifically justified, the ISRP contends there needs to be a comparison of HEP and CHAP together with actual wildlife survey data and habitat inventories. Perhaps data from existing projects that have monitoring results could be the basis for a comparative evaluation. If this is done, it should be through an RFP and conducted by a neutral party. Policy/administrative: The CHAP component accounts for approximately 24% of the proposed budget. Is this reflective of its priority of the project? Additional comments for each of the proposal sections are provided below: 1. Technical Justification, Program Significance and Consistency, and Project Relationships The requested format was not used, thus these categories must be inferred. The reasoning in the narrative seems to be, we have to do accounting with HEP/CHAP, so here is what it will cost. Use of HEP is a policy decision. The sponsors began their presentation emphasizing this point: HEP is for accounting, not monitoring. There is no explanation of the "stacking" or the "out of site/out of kind" issues used later to justify promotion of CHAP methodology. 2. Project History and Results This section provided only a very general list of "clients" served each year. The ISRP requests information about the number of sites, acres, or proportion of baseline to repeat evaluations. 3. Objectives, Work Elements, and Methods Objectives and work elements appear straightforward, with questions of methods omitted (perhaps in appendices that are not cited?). Operating two HEP crews seems justified. On many snow-free sites, field work could continue for more extended periods, perhaps even year-round in some cases. Why not? Unless the sponsors have completed significant enhancements or reported significant changes in site conditions, is HEP needed every five years? For example if at four years out, sponsor just planted several acres of trees, their survival and habitat contributions would not yet be apparent. Some sponsors include HEP in their budgets, and the HEP team also operates with its own budget. It is not clear if these are independent HEP efforts or why this duplication exists. 4. M&E This is not relevant to an accounting procedure. Sponsor and Council should continue to emphasize to other project sponsors that HEP is in no way an effectiveness monitoring method. Terminology such as “accounting” and “effectiveness monitoring” as separate activities would help in this regard as many sponsors still appear to be unclear on this distinction.

Overall the ISRP viewed the use of HEP as a policy decision. HEP has played and can continue to play a role in the Council's program by establishing mitigation credits against the initial baseline losses that were agreed to be reasonably indexed by habitat units (HUs) derived from HEP. However, HEP is no longer considered to be a good method for evaluation of value of land to wildlife, as there have been significant improvements in both analytical methods and available data that underlie estimation of the relationships of wildlife species and assemblages to habitat. Further, HEP is not a sufficiently direct measure to support the purposes of monitoring and evaluation. Far better monitoring approaches and metrics are now available, and use of more direct approaches is required for effective evaluation of benefits to wildlife. In sum, HEP alone does not provide adequate biological M&E, and direct biological M&E is not improved by continuing HEP.

If the Council continues to use HEP as the basis for initial determination of mitigation value, then a consistent approach to evaluation is desirable and a standardized HEP approach could help to achieve such consistency. In this case, the proposed project should present more clear explanation of methods to be used, including the timing of sampling and what specific HEP models would be used to evaluate the structural characteristics of habitat, and any additional needed details to allow evaluation of sampling methods.

The reviewers found the CHAP portion of the proposal Not Fundable. The proposal did not provide convincing evidence that the approach of NWI would be a significant improvement over the HEP-derived habitat unit metric now in place. In particular, the methods used to determine habitat value (HV) were not clearly presented. It would have been useful for the proposal to include a more clear explanation of the calculation and use of habitat value, with an example from a subbasin of how to use the metric, habitat value, as a measure of progress towards mitigation. It seems likely that direct biological M&E will almost always be more convincing, more interpretable, and thus more useful for evaluation and application to management decision-making than would be a less direct, HEP-type measure. The proposal did not convince the ISRP that the NWI efforts to improve HEP would be as good as direct biological M&E.

The ISRP also noted that actual evaluation of wildlife projects was rarely provided in proposals. The use of HEP or CHAP would imply that habitat was an adequate proxy for value to wildlife, but this proposal does not articulate habitat goals or how and when progress towards goals would be measured. The use of HEP to provide monitoring and evaluation is not considered scientifically well advised. The relationships of HEP- or CHAP- derived metrics to focal species identified in subbasin plans or to non-focal species were not defined.