Dr Julie Everett-Hincks, a legal and scientific researcher at the University of Otago, questioned whether it was right to burden farmers with regulatory compliance when the tool being used cannot reasonably measure nutrient losses.
“In its current form and governance structure, Overseer is not fit to be a regulatory tool.”
The report by the Parliamentary Commissioner for the Environment (PCE), Simon Upton, found significant gaps and shortcomings in Overseer that he says undermined confidence in its use as a regulatory tool.
So long as councils are aware of its limitation, Upton said Overseer can still be a useful tool, but he urged issues he has identified be urgently addressed.
Federated Farmers environment spokesman Chris Allen said the PCE findings of inaccuracy and gaps in Overseer had been raised by farmers.
“The significant inherent inaccuracies in the Overseer model means that is very unfair when the model is used to regulate farming activity central to farmers’ livelihoods, and even more importantly to mount prosecutions.
“How would an ordinary New Zealander feel if he or she was prosecuted for speeding in their car based on the model they are driving and when the radar or speed camera had an accuracy range of plus or minus 50%?”
Overseer’s inaccuracy means it should not be used to assign absolute limits to farm discharges that can then be traded.
“It is more useful for guidance about relative change or ‘direction of travel’ in terms of reducing nutrient losses, or a comparison of changes to farming systems rather than assigning absolute numbers.”
Overseer is a commercial entity owned by fertiliser co-operatives Ravensdown and Ballance, the Ministry for Primary Industries and AgResearch, and as such has not been open to outside analysis, something Upton said needs to happen to build confidence.
Six councils – Bay of Plenty, Canterbury, Hawkes Bay, Horizons, Otago and Waikato use Overseer to regulate nitrogen runoff, and three others require farmers to use it to predict that future nitrogen limits will be met.
In his report, Upton found some of the uncertainty with Overseer was due to it using a “typical” animal for calculating energy requirements, a risk from poor or inaccurate record keeping and the lack of geographically widespread field data to calibrate the model and account for soil and climatic differences.
The report stated that a 2001 analysis for predicting nitrogen losses revealed a variation of between 25 and 30%.
Overseer developers believe that variation applied to the model’s current version.
The absence of widespread trials to allow calibration to various conditions meant Overseer was not able to accurate generate results that matched soil, climate and management conditions.
“Some parts of New Zealand have not been covered by trials, meaning uncertainty in some regions may be greater than in others,” the report says.
For farms outside the model’s calibration the range of predicted nitrogen loss could exceed 50% and for Canterbury dairy farms on light or poorly-drained soils it could differ from 40% below the actual leaching rate to 60% above.
Upton recommended a comprehensive evaluation of Overseer, including an independent peer review, and subject it to sensitivity and uncertainty analysis.
Upton also recommended greater transparency on how Overseer works and setting up a working group to provide guidance on how councils can use it.
Local Government NZ regional sector chairman Doug Leeder said that with right investment, Overseer can be used to promote better water quality but at present it was used within its limitations.
“If we continue to use Overseer to inform regulation, it’s important that we apply our knowledge and gain investment to ensure it is fit for purpose, and in doing so get buy-in from farmers to make the most of the benefits it can provide to our waterways.”
Environment Canterbury said in a statement it was encouraged by the push for continuous improvement and had changed the way it used Overseer.
“We have been using Overseer for around 10 years. Our latest update to the Canterbury Land and Water Regional Plan, Plan Change 5, eliminates reliance on a fixed Overseer estimate and replaces it with a threshold based on irrigated area or the use of winter grazing of stock.”
Professor Richard McDowell, the chief scientist, Our Land and Water National Science Challenge, believed Overseer should be linked to farm environment plans.
He says Overseer needed to evolve to a point where it can map and target critical areas of farm nutrient loss, measure the impact of day to day decisions and help landowners identify suitable land use.
