1. Objectives and outcomes 

The Summary Report provides a useful, information-rich overview for a long-term project. The Summary Report clearly describes project goals and provides sufficient details to indicate how management strategies and approaches could be expected to provide benefits to wildlife. The proponents identify three management goals for the project. Quantitative objectives and timelines to reach the objectives are needed so that monitoring can be used to evaluate progress toward the goals and quantitative objectives. 

2. Scientific principles and methods 

The Summary Report identifies activities taken to achieve results in a well-organized manner. For example, activities related to monitoring of high-priority species, improving water quality, enhancing habitat types, controlling noxious weeds, protecting springs and seeps, and managing grazing are clearly summarized. Rationales for use, or alteration, of methods are justified. 

The ISRP commends the proponents for presenting monitoring results in considerable detail in figures and tables. This presentation helped us to better understand the challenges the proponents are facing, to identify possible problems with the experimental design, and to focus potential concerns with the monitoring program. 

The proponents also state in a number of places that various surveys are intended “to detect possible benefits or consequences of land management actions” on small mammal, amphibian populations, and vegetation cover and composition. However, an experimental design with proper controls is needed to determine whether changes in metrics computed from surveys can be attributed to management actions rather than to other uncontrolled factors. BACI (Before After Control Impact) design are often used for this kind of ecological study. 

Many of the statements about declining or increasing trends seem to be based on the slopes of polynomial curves fitted to data in Figures 3-11. These claims are questionable as they are not supported by statistical analyses. Moreover, the curves in Figures 9 and 11 seem to be fitted incorrectly, or perhaps are affected by data or weighting schemes not shown in the figures. The curves inappropriately extend beyond the range of observations prior to 2015. 

A number of graphs showing species abundance are used to describe general population trends from about 2006 to present. References were provided for the protocols used to collect data, but methods specific to this effort were not described. Additional methods were provided in the 2004 Wildlife Mitigation Plan, but without reference to this plan it is difficult to determine the extent to which these methods were applicable to efforts described in the Summary Report. For example, how many samples were taken each year in each location and each month? Do error bars in the graphs show standard deviations or some other measure of variability? 

3. Monitoring and evaluation 

Much of the Summary Report is devoted to describing M&E activities. The summary used to describe monitoring and evaluation by activity including migratory bird surveys, brood surveys, small mammal surveys, amphibian surveys, vegetation assessment, and stream photo points provides an effective overview for understanding the project. This information is helpful, but difficult to fully evaluate because information on methodology and quantitative objectives is not provided. 

The presentation of monitoring and research findings should link directly to quantitative objectives so that progress in achieving the objectives can be assessed. However, quantitative objectives were not developed, and the results do not seem to address all of the nonquantitative objectives described earlier. For example, there was little description of the degree to which noxious weeds were controlled and what actions were taken to improve water quality. 

Data provided in the Summary Report shows that a lack of precision in abundance indices will make it difficult to detect trends. The cause of this variability is not clear. It could be due to differences among observers or other sources of variability in detectability or abundance. A statistical power analysis should be performed to estimate the sampling rates needed to gain sufficient precision to detect trends, especially for the amphibian and small mammal surveys. Sampling efforts to date have generated estimates of variability, which can now be used to conduct the power analysis. 

Population data were provided for birds and small mammals. How do these values compare with values in other areas where habitat is relatively undisturbed, or with expected values for “healthy” habitat? To what extent can population and vegetation trends be linked to actions taken on the wildlife area? 

The Summary Report describes two research components in terms of objectives that could easily be converted to testable hypotheses. 

Other minor points:

The wording “frequency of invasive plants occurring in all quadrats” in the captions for Figures 19 and 20 is confusing (and potentially misleading); presumably, the y-axis is the percent of quadrats containing the species based on all quadrats examined. 

In Table 6, why is n/N = 0.003 (instead of 0.15) for the western harvest mouse? With such small numbers, the Simpson index should be calculated with the “sampling without replacement” formula. 

In Table 7, the column headings (species names) are missing. Also, as described in the text, relative species cover is no longer a percentage measure. It is a dimensionless index (the ratio of two percentage measures). 

Appendix A provides a number of photo points comparing riparian and stream bank habitat changes from 2007 to 2016. Photos can be very useful to document changes. Were the before and after pictures taken during the same month?

4. Results: benefits to fish and wildlife and adaptive management

The project summary presents quantitative and qualitative results. Lessons learned are well described and are used to motivate suggested changes in management and monitoring as described in a section titled, Adaptive Management. These suggestions do not address the major issues of whether current treatments have been or will be useful for achieving desired outcomes. The ISRP is concerned that this project cannot implement adaptive management because its objectives are not quantitative, sampling rates and monitoring are insufficient to detect trends in the face of natural variability, and the experimental design is inadequate to attribute changes to treatments. 

Adaptive management in this wildlife area is described as correcting mistakes and recognizing patterns, rather than active decision-making stemming from a series of anticipated outcomes. Most of the concerns in this section involve methodology rather than achieving desired habitat conditions. Ideally, adaptive management should stem from quantitative objectives and timelines, followed by monitoring and evaluation that shows progress toward those objectives. Adaptive management should describe alternative actions taken to better achieve project goals and objectives. The lack of quantitative objectives inhibits implementation of adaptive management. 

Past annual reports described project activities to enhance wildlife habitat by haying and grazing, controlling noxious weeds, and managing water flow. The 2017 report refers to these activities in the Executive Summary, but it does not describe them under Section II (Results: Reporting, Accomplishments, Impact, and Adaptive Management). Consequently, it is not clear whether these activities were continued through 2016. Instead, the 2017 Summary Reports focuses exclusively on monitoring and evaluation, and research. Given the issues with experimental design and statistical evaluation described above, it is unclear how much benefit the project has provided to fish and wildlife.

Quantitative objectives and timelines to reach the objectives are needed in the next management plan so that monitoring can be used to evaluate progress toward these objectives. A formal adaptive management plan should be included in the next Management Plan. Adaptive management should stem from the quantitative objectives and timelines, followed by monitoring and evaluation that shows progress toward those objectives. The adaptive management plan should describe alternative actions that could be taken, if needed, to better achieve goals and objectives. The project proponents should consider questions such as the following when responding to the qualified recommendation for this project. This list is not exhaustive but is presented to aid in identifying quantitative objectives. 1. What measurable metric(s) for habitat conditions and abundance/diversity of wildlife can be used to describe the viability of focal species? 2. How many acres of upland, wetland, floodplain meadow and riparian habitats are to be enhanced? 3. To what extent will density of noxious weeds be reduced? 4. How many springs and seeps (or acres) will be protected? 5. What metric best defines whether or not grazing practices are managed to meet wildlife objectives? 6. What is the quantitative objective that defines success for the number of annual access and hunting permits issued to the public? 7. To what extent should deer/vehicle collisions be reduced?

The sponsors have provided most of the detail to support approval of the scientific merit of the proposal. The material provides a better understanding of the management objectives of the Burns Paiute Tribe for this land parcel. Consistency in staffing will contribute to effective adaptive management in this challenging environment. The ISRP qualifies the recommendation with the suggestion that in two to three years there is a review of monitoring progress.

For the most part, the response was helpful in framing the various work elements. One answer that is lacking is to the question concerning what proportion of funds is for O&M and what is for future acquisition.

The four-year rotation is an excellent plan, but it is also important to consider when in the year grazing occurs and for how long. To shift plant populations and create patchiness with grazing will probably result in overuse of key perennial forage species and may also be detrimental to livestock gain. If possible, yearlings would do a better job of this than pairs. Small burns could give these same results faster. With off-stream water and salt, most cattle should be discouraged from loafing in riparian areas unless the weather is hot and this is the only shade.

Wet meadows and hayfields would be easier and less costly to manage if they are in native grasses that do not require regular haying or periodic replanting. Grazing will keep these plants in a vegetative stage that produces good quality winter forage.

Vegetation data taken during the HEP survey can be statistically analyzed independently from the HEP analysis and used to monitor vegetation changes, if enough sites are sampled and they are measured more frequently than the five-year HEP interval. Line intercepts and plots are preferred over step point sampling for less common species. These are quick methods amenable to statistical analysis, as is the ISRP recommendation for more photo-points that could be accomplished with very minor budget increases.

The ISRP suggests that a review occurs in the next two to three years to evaluate monitoring progress.

The project has good potential for benefiting fish and wildlife but does not adequately describe activities or present a convincing case that objectives can be accomplished. The relationship between O&M and acquisition should be stated. The response should better organize objectives and justify and explain work element and methods. Using livestock grazing and haying to improve fish and wildlife habitat should be explained in more detail. Monitoring efforts are commendable, although some suggestions are offered below. Lessons learned from events such as “failed” seedings should be related, so that others might benefit from your experience. The ISRP requests a response that includes more data summaries and descriptions of activities to evaluate mitigation activities (e.g., weed management). HEP is appropriate for crediting, but the science is outdated for effectiveness monitoring. Metrics for M&E should be provided and justified in a response. 1. Technical Justification, Program Significance and Consistency, and Project Relationships As noted in a previous review, the logic for the project is that multiple fish and wildlife species could benefit from these restoration activities. The proposal includes cultural justification that complements the biological justification. The project has good potential for benefiting fish and wildlife but does not adequately describe activities or present a convincing case that objectives can be accomplished. A response that better justifies work elements and monitoring and evaluation is needed. More details are provided below. The proposal states that this is an ongoing project to manage 6385 deeded acres of various habitats along the Malheur River. Later the proposal states that this is an acquisition project. The sponsors should clearly state what type of project is proposed and if both, make that connection obvious. 2. Project History and Results The proposal provides an interesting chronological history summarizing work activities and cost for the restoration actions that have taken place over the last eight years. The results are explained in qualitative terms. It would be helpful to explain which efforts have been perceived as most successful and why, and which have required the most maintenance or adaptive management. It was useful to see the results of the 2006-2008 wildlife surveys, although there are insufficient results yet to be able to determine whether the restoration actions are having the desired effects. Evaluations of lessons learned from past events such as plantings failing, haying not completed, and no amphibians trapped despite activities such as irrigating to keep wetlands full, should be provided. Some objectives are repeated in number or in repeated wording. It is not clear that using salt to attract cattle away from riparian areas without fences will be adequate to protect the riparian habitat. Opportunities for off-stream watering should be explored. Using livestock grazing and haying as management tools where the goal is to improve fish and wildlife habitat should be explained in more detail in a response. Establishment of a long-term grazing policy as part of a management plan should be a priority. It was not clear why only nine photopoints were selected for long-term photo documentation of stream and riparian condition. Justification should be provided in a response. The 5-year aquatic habitat surveys in "critical streams" and 10-year surveys in secondary priority streams are probably too infrequent to document restoration-associated changes. The ISRP recommends at least 3-year survey intervals or more frequent surveys in case a large natural disturbance event (wildfire, flood, or multi-year drought) occurs. The annual temperature monitoring and fish survey plans appear adequate. The wildlife surveys, also conducted yearly, are well described, and project personnel are qualified for the work. We applaud the use of the Weed Information Management System as part of the weed control activity. The ISRP discourages use of HEP to determine vegetation trends or to evaluate if habitat needs of each target species are improving. HEP is appropriate for crediting, but the science is outdated for effectiveness monitoring. Metrics for M&E should be provided and justified in a response.

The project is well described, justified, described, planned, and documented. The location of this project makes it likely that fish and wildlife will benefit from the actions taken. The monitoring and evaluation plan is good. The sponsors are encouraged to effectively summarize their quantitative and qualitative results to provide further evidence of the value of the project. The long-term grazing allotment and grazing issues will be important issues affecting the landscape. Institutional policy decisions concerning grazing should be clearly articulated.

1. Technical Justification, Program Significance and Consistency, and Project Relationships
The technical justification of the project is written clearly and is compelling. This ongoing work is in a good geographic position to improve water and riparian conditions downstream as well as in the immediate area. The logical need for the project is clearly explained. The restoration activities are likely to benefit several fish and wildlife species. The proposal identifies the importance of the project to the Malheur subbasin. Focal species were identified and related to both the Malheur subbasin plan and to ODFW's wildlife species of concern. Relationships to other restoration efforts in the region were given in general terms.

2. Project History and Results
The history of the project is clearly reported and provides context that includes cultural justification to complement the biological justification. Partial results from 2006-2008 are presented in the proposal, but the short time series and lack of effective interpretation make evaluation of project success difficult. It would have been helpful to include monitoring data, even though preliminary, on upland and riparian vegetative recovery since grazing exclusion.

3. Objectives, Work Elements, and Methods
The objectives, work elements, and methods are described in a coherent manner. Several work elements involving alternative strategies are used to address the same objective. It is likely that such a strategy will result in useful information to share with others concerning successes and lessons learned. A map of the area showing where different actions would occur (fencing, controlled burning, and noxious weed control) for each biological objective would help. In general the work description and methods were adequately detailed.

4. M&E
Similar to the ISRP FY 2007-09 review, the monitoring and evaluation includes vegetative monitoring through four different methods: photo monitoring, noxious weed monitoring, HEP and forest inventory monitoring. The sponsors note that HEP provides an additional source of information that can be used to assess vegetative changes specifically for the focal species used in the baseline HEP, even though the ISRP does not recommend HEP as a monitoring tool. We note once again that field observations complement photography in understanding mechanisms involved and in developing any needed modifications or replicating success. More information should have been provided on how the data would be analyzed and archived.

The project is well described, justified, described, planned, and documented. The location of this project makes it likely that fish and wildlife will benefit from the actions taken. The monitoring and evaluation plan is good. The sponsors are encouraged to effectively summarize their quantitative and qualitative results to provide further evidence of the value of the project. The long-term grazing allotment and grazing issues will be important issues affecting the landscape. Institutional policy decisions concerning grazing should be clearly articulated. 1. Technical Justification, Program Significance and Consistency, and Project Relationships The technical justification of the project is written clearly and is compelling. This ongoing work is in a good geographic position to improve water and riparian conditions downstream as well as in the immediate area. The logical need for the project is clearly explained. The restoration activities are likely to benefit several fish and wildlife species. The proposal identifies the importance of the project to the Malheur subbasin. Focal species were identified and related to both the Malheur subbasin plan and to ODFW's wildlife species of concern. Relationships to other restoration efforts in the region were given in general terms. 2. Project History and Results The history of the project is clearly reported and provides context that includes cultural justification to complement the biological justification. Partial results from 2006-2008 are presented in the proposal, but the short time series and lack of effective interpretation make evaluation of project success difficult. It would have been helpful to include monitoring data, even though preliminary, on upland and riparian vegetative recovery since grazing exclusion. 3. Objectives, Work Elements, and Methods The objectives, work elements, and methods are described in a coherent manner. Several work elements involving alternative strategies are used to address the same objective. It is likely that such a strategy will result in useful information to share with others concerning successes and lessons learned. A map of the area showing where different actions would occur (fencing, controlled burning, noxious weed control) for each biological objective would help. In general the work description and methods were adequately detailed. 4. M&E Similar to the ISRP FY 2007-09 review, the monitoring and evaluation includes vegetative monitoring through four different methods: photo monitoring, noxious weed monitoring, HEP and forest inventory monitoring. The sponsors note that HEP provides an additional source of information that can be used to assess vegetative changes specifically for the focal species used in the baseline HEP, even though the ISRP does not recommend HEP as a monitoring tool. We note once again that field observations complement photography in understanding mechanisms involved and in developing any needed modifications or replicating success. More information should have been provided on how the data would be analyzed and archived.

The logical need for the project is explained. Multiple fish and wildlife species could benefit from these restoration activities. The proposal demonstrates significance of the project to the Malheur subbasin and includes cultural justification that complements the biological justification. The project history is clearly recounted.

The sponsors provided a response to the ISRP review that better explains provisions for monitoring and evaluation. This continues to be an issue identified in past ISRP reviews. The ISRP review asked for more details concerning monitoring and evaluation including: 1) benefits to fish and wildlife including an evaluation of how persistent the benefits will be, 2) possible adverse effects on non-focal species, 3) short and long-term success of habitat manipulation. The ISRP encourages more adaptive management as the project proceeds. The sponsors have provided additional information that responds to ISRP questions and concerns in a very effective manner. It is clear that extensive monitoring is planned and personnel are available to effectively evaluate the project.

Relationship and collaboration with other projects are noted as well as outreach and educational activities. However, some methods to share successes and lessons learned with others involved in similar monitoring and restoration activities should be identified.

Most objectives seem appropriate given the detail presented. The presentation of work elements was not very detailed in the proposal, but the response effectively provided justification for the methods chosen.

The facilities, equipment, and personnel are reasonable, and their description is well written. Personnel appear quite adequate now that additional resource personnel have been identified to assist with setting up and evaluating the monitoring program.

The logical need for the project is explained in the proposal. Multiple fish and wildlife species could possibly benefit from these restoration activities. The proposal demonstrates the significance of the project to the Malheur subbasin. The history of the project is clearly reported. The context includes cultural justification that complements the biological justification. This project is in a good topographic position to influence water and riparian conditions downstream as well the proximate area.

The monitoring and evaluation includes vegetative monitoring through four different methods: aerial photos, willow monitoring, vegetation transects and HEP. The sponsors note that the ISRP does not recommend HEP as a vegetation monitoring tool but assert that it is a source of additional information that can be used to assess vegetative changes specifically for the focal species used in the baseline HEP. Statements of desired conditions are very useful starting points. A few comments on vegetation monitoring: aerial photos will provide useful information on overstory species change, but will tell little about the reasons, e.g., recruitment or development of existing plants. Some field observation to complement photography will aid in understanding mechanisms involved and in developing any needed modifications or replicating success. It is noted that elk browsing appears to be limiting willow recovery. Is any management change indicated to assure meeting project objectives? It appears that vegetation frequency data may be incorrectly understood as more than just occurrence of a species in a proportion of plots examined. The sponsors should verify that this will give them what they are seeking. As far as transects: 1 per vegetation type will not allow very robust interpretation, regardless of the number of subsamples associated with the location. The ISRP appreciates that wildlife monitoring is also described in the response.

Relationship and collaboration with other projects are noted. Publications and other methods of sharing results were identified in the response. Methods to share successes and lessons learned with others involved in similar monitoring and restoration activities should be utilized.

Most objectives seem appropriate given the detail presented. The ISRP hopes to see more adaptive management as the project proceeds. Adaptive management means a systematic evaluation of monitoring results by the team to be used to verify successes, identify unanticipated opportunities, and change management when needed.

The description of facilities, equipment, and personnel is well written. The facilities, equipment, and personnel are generally appropriate. The sponsors have identified a consulting biometrician to provide statistical support as necessary.