×

Title:

Development of guidelines for quality assurance of Australian fisheries research and science information

Project Number:

2014-009

Organisation:

Pisces Australis Pty Ltd

Principal Investigator:

Andrew Penney

Project Status:

Completed

FRDC Expenditure:

$100,800.00

Program(s):

Environment

Development of guidelines for quality assurance of Australian fisheries research and science information

Final Report
ISBN:978-0-9577587-0-4
ISSN:
Author(s):Andrew J. Penney
Date Published:December 2016

Keywords: fisheries science, quality assurance, peer review,

There has been a long history of international development of increasingly detailed guidelines and standards for quality assurance of scientific information used to inform government policy and management decisions. The need for such guidelines has resulted largely from various crises of confidence in government decisions relating to human health, and to sustainable management of natural resources, most notably the bovine spongiform encephalitis (mad cow disease) outbreak in the United Kingdom in the mid-1990s. This led to the publication by the UK Government Chief Scientific Advisor of the May Report in 1997, which established key principles for the use of scientific evidence in policy making. This was followed by UK government Guidelines on Scientific Analysis in Policy Making in 2005, a Code of Practice for Scientific Advisory Committees in 2007, Principles of Scientific Advice to Government in 2010 and an updated and more detailed Code of Practice for Scientific Advisory Committees in 2011.

Similar developments in the European Union were prompted by numerous environmental and health concerns, relating to acid rain, synthetic hormones, foot and mouth disease, pesticides and the dioxin crisis in Belgium in 1999, when dangerously high levels of dioxin were found in poultry and eggs, due to widespread use of PCB insecticides. The European Governance White Paper of 2001 emphasised the importance of quality of, and trust in, scientific advice, and led to the Use of Expertise by the Commission: Principles and Guidelines in 2002. Various reviews of the credibility of science used to inform EU decision making led to the European Peer Review Guide and the European Code of Conduct for Research Integrity in 2011.

Development of guidelines for ensuring quality of science in Canada was directly related to one of the most influential fisheries crises experienced: the collapse of the Grand Banks and Newfoundland cod stocks, despite scientific advice warning of overfishing. Guidelines for Science Advice for Government Effectiveness (the SAGE principles) were adopted in 1999, supported by a Framework for Science and Technology Advice in 2000. In response, the Canadian Department of Fisheries and Oceans developed a detailed Science Advisory Process, emphasising the role of peer review and providing guidance on choosing the most appropriate peer review process under different circumstances, supported by a set of Principles and Guidelines for Participation in Peer Review Processes.

Development of quality assurance standards relating to fisheries science in the United States stem directly from incorporation of National Standard 2 in the Magnuson-Stevens Fishery Conservation and Management Act (MSA), requiring that “(2) Conservation and management measures shall be based upon the best scientific information available”. Parallel requirements to ensure the quality of scientific information, particularly relating to scientific advice regarding pesticides and environmental concerns, resulted from the 'Data Quality Act' in 2001, actually a short rider inserted into the Consolidated Appropriations (federal budget) Act in 2001, requiring all federal agencies to “issue guidelines ensuring and maximizing the quality, objectivity, utility and integrity of information (including statistical information) disseminated by the agency”.

In response, the US Office of Management and Budget published Guidelines on Information Quality in 2002 and an Information Bulletin for Peer Review in 2004. In response to both National Standard 2 and the OMB Guidelines, the National Oceanic and Atmospheric Administration published comprehensive Information Quality Guidelines in 2006, informed by 2004 guidelines produced by the National Research Council (NRC) on Improving Use of Best Scientific Information. In further response to NRC recommendations, NOAA published detailed Guidelines for Best Scientific Information Available in 2014, as a rule under the MSA, prescribing requirements for improved implementation of National Standard 2. This rule currently provides the most detailed international guidance on quality assurance and peer review of fisheries-related scientific information.

Key components of the above guidelines, codes of practice, standards and rules were identified and consolidated to provide a distillation and summary of key principles and best practices for science information quality assurance. These international key principles and guidelines for effective peer review were used as the basis for Research and Science Information Guidelines for Australian Fisheries, intended to be applicable to the quality assurance of all research or scientific information intended or likely to inform fisheries policy or management decisions. The Guidelines cover the following aspects of science information quality assurance:

  • Key Principles for Scientific Information Quality: Peer Review, Reliability, Integrity, Objectivity and Relevance (the PRIOR principles).

  • Responsibilities of Research Purchasers and Research Providers: relating to requiring, supporting or implementing processes to ensure the quality of research and scientific information, as defined by the Key Principles.

  • Criteria for Effective Peer Review: balance of expertise, independence, inclusiveness, transparency and openness, timeliness, impartiality, management of conflicts of interest, reporting of uncertainty and risk and staged technical guidance.

  • Stages and Forms of Peer Review: options ranging from single expert review to appointed panels, established working groups, technical review workshops or independent expert review, to ensure that peer review is cost-effective and appropriate to the complexity, contentiousness and likely influence of the information.

  • Data Retention and Management: to ensure that data underpinning science used to inform fisheries decisions is retained, securely stored and potentially available to further analysis, subject to applicable confidentiality and privacy requirements.

  • Documentation and Communication: to ensure that research results and scientific information are clearly reported, and that the integrity of information is protected in such communication.

  • Implementation and Reporting: providing guidance on development of organisation-specific plans for the implementation of science quality assurance processes

  • Definition of Terms: providing definitions for key terms used, to aid with the interpretation and implementation of the guidelines.

The guidelines are non-prescriptive and provide for high flexibility in implementation, to ensure they are relevant across the wide range of research activities informing policy and management decisions for Australian wild capture fisheries and their impact on the marine environment. It is expected that implementation will be achieved by means of agency-specific implementation plans, tailored to the requirements, capabilities and current processes of each agency. An example draft implementation plan is provided for Commonwealth fisheries, prepared by the Australian Fisheries Management Authority.

Research and Science Information Guidelines for Australian Fisheries

Final Report
ISBN:978-0-9577587-1-1
ISSN:
Author(s):Andrew J. Penney
Date Published:December 2016

Principal Investigator: Andrew Penney

Summary:

PURPOSE
Fisheries research and scientific information is used to inform fisheries management decisions and the formulation of related environmental policy. Government Ministers and decision-makers, stakeholders and the public need to have confidence and trust in research and scientific information used to inform fisheries management. To help achieve this, key principles for ensuring quality of science need to be adhered to, and effective science quality assurance processes need to be put in place, to:

  • ensure the quality and integrity of research and scientific information, irrespective of the source of that information;
  • require research providers, relevant advisory committees and advisory processes to meet sufficient Guidelines for ensuring the quality of scientific information; and
  • ensure that peer review processes, the primary mechanism for ensuring the quality of scientific information, are cost-effective and efficient.
These Guidelines provide guidance for high quality and reliable scientific information, and best practice quality assurance of research and scientific information to inform fishery management decisions, regardless of the source of that information. The Guidelines set out principles for research and scientific information quality, identify responsibilities, and describe requirements for getting third party peer reviews, evaluation of scientific information quality, storage and management of data and documentation and communication of results.
 
As well as the Guidelines is another document which makes up this study titled Development of guidelines for quality assurance of Australian fisheries research and science information (https://frdc.com.au/research/Final_Reports/2014-009-DLD%20B.pdf) which gives the overview of the development of the Guidelines themselves. 
 
SCOPE 
Scientific methods strive to produce objective and reliable information, and document how that information has been derived, such that the results can be validated and reproduced if required.
 
Scientific methods and quality assurance processes can be applied to any research project. Much of the research and scientific information used to inform fisheries management relates to fisheries characterisations, biological studies, stock assessments, resource abundance surveys and evaluations of impacts on associated or dependent species. Other disciplines using scientific methods also produce information that is used in fisheries management, including broader ecosystem, social science and economic studies. The principles and quality assurance processes in these Guidelines can be applied to any such information, if scientific methods have been used.
The provisions of these Guidelines are intended to be applicable to:
  • Fisheries research projects contracted or conducted by research purchasers and/or providers such as the Fisheries Research and Development Corporation (FRDC), Australian Fisheries Management Authority (AFMA) and Commonwealth or State government departments purchasing or providing research intended or likely to inform fisheries management decisions relating to wild capture fisheries and their impact on the marine environment.
  • Fisheries research projects conducted or contracted by the seafood industry or other stakeholder organisations as part of established research programmes intended to inform fisheries management decisions relating to wild capture fisheries and their impact on the marine environment.
  • Any other research conducted or contracted by other organisations, if that research is intended or likely to inform fisheries management decisions relating to wild capture fisheries and their impact on the marine environment. ny other research conducted or contracted by other organisations, if that research is intended or likely to inform fisheries management decisions relating to wild capture fisheries and their impact on the marine environment.

Objectives

1. Review recent national and international developments on science quality assurance principles, implementation guidelines and quality assurance processes relevant to Australian fisheries characteristics, management processes and requirements.

2. Prepare draft standard and guidelines for quality assurance of Australian research and science information intended or likely to inform fisheries policy and management decisions, including key principles for science quality, implementation guidelines and performance monitoring for science quality assurance processes.

3. Consult with fisheries agencies in other jurisdictions, as well as other relevant stakeholders, to ensure that the proposed science quality assurance guidelines are appropriate and implementable for all Australian fisheries, and potentially implementable for other science fields

4. Prepare an agency-specific plan for implementation of the science quality assurance key principles and quality assurance processes for AFMA, compatible with AFMA and Commonwealth fisheries requirements, capabilities and science procurement processes.