The current information landscape in Australian Government agencies

From the Freedom of Information Act to the Prime Minister’s public data policy statement, the notion that the information gathered, generated and funded by the Australian Government is a significant public asset is well established.
But while statements of intent are an important starting point, the work of delivering valuable open data to the community requires long term commitment and resourcing if the promised benefits are to be realised. The information policy and regulation landscape in Australia is complex, and, it could be argued, lacks a strong centralised focus on delivery. That could be set to change, though, given the recommendations in the Productivity Commission’s wide-ranging report of its inquiry into data use and availability. The following post provides an overview of the agencies that currently have an interest in data management by Australian Government entities and considers some of the Productivity Commission’s recommendations for delivering improvements in access to data.

Digital Transformation Agency

The main remit of the Digital Transformation Agency (DTA) is the delivery of government services online, although it also has a broader interest in ICT strategy and procurement. The DTA has published the Digital Service Standard, which applies to ‘new and redesigned government services, information and transactional and all high volume transactional services […] existing or being designed/redesigned’. Of relevance to the open data agenda is Standard Seven, ‘Use open standards and common platforms’. Although the open data is not strictly delivering a service, these standards give an idea of the direction and expectations being set by the DTA. The emphasis on open standards dovetails with guidance associated with, which also notes the importance of open standards for publishing data to be a useful exercise. The DTA’s Design Principles, which include ‘build digital services, not websites’ and ‘make things open: it makes it better’, give a cue to agencies to prioritise API access and putting ‘open’ in all its flavours at the heart of new undertakings.

Prime Minister and Cabinet

Following the Prime Minister’s Public Data policy statement, which called for public data to be open by default, the Australian Government has prioritised maximising the value of public data by making it more available to researchers and business. The Department of Prime Minster and Cabinet (PMC) currently has a Public Data branch that includes ownership of, which previously sat within the Department of Finance. The Public Data branch has a number of focus areas, including the Open Data initiative, which has oversight of projects such as DataStart, a public-private-partnership to support businesses using public data, the Open Data 500 survey, looking at uptake of public data and the Open Data Toolkit that supports data custodians wishing to publish data, especially to

The toolkit provides a range of resources for data custodians wishing to publish their data, including overviews of appropriate formats, relevant metadata, and other issues to consider. These resources are generally quite focussed on the work of making getting data ready to appear on a platform such as When it comes to the thorny issues of privacy and de-identification, it merely refers readers to the Office of the Australian Information Commissioner and does not provide any concrete guidance.

Office of the Australian Information Commissioner

The Office of the Australian Information Commissioner (OAIC) has responsibility for privacy, freedom of information and information policy. In the latter two roles, the OAIC emphasises the importance of information as a public good; it released its Principles on Open Public Sector Information in 2011 and has since produced resources for agencies wishing to publish data. Given its role as the privacy regulator, the OAIC also has an interest in ensuring that data publication does not lead to breaches of privacy. It has published a resource for agencies on data de-identification of data, which covers issues of when data should be de-identified and possible approaches to de-identification. The guide refers agencies to the National Statistical Service (NSS) and Australian National Data Service (ANDS) for more specific guidance on working with sensitive data.

National Statistical Service

The NSS is a community of agencies led by the Australian Bureau of Statistics (ABS) that supports the development and use of statistics. The NSS also provides governance for data integration projects involving Commonwealth data for research. These governance arrangements were put in place in part to ensure that data integration projects do not have adverse privacy consequences. Resources provided by the NSS include some information sheets on managing the privacy implications of releasing microdata and basic principles on how to confidentialise data.

Australian Signals Directorate

The Australian Signals Directorate (ASD) is an intelligence agency within the Department of Defence that provides foreign signals intelligence to the Australian Defence force and Government and information security advice and assistance to government agencies. The ASD is therefore concerned with security rather than the open data agenda, but keeping the right data safe is a core component of information management. The ASD produces the Australian Government Information Security Manual (ISM) as well as other guidance on mitigating cyber security risks.

Australian National Data Service

The Australian National Data Service (ANDS) supports better management and reuse of Australian research data, although it does not have any regulatory authority. Increasingly, the research sector is being expected to find ways to share data in recognition of the public funding that makes much of it possible. The National Health and Medical Research Council (NHMRC) has mandated that all data generated from research that it funds must be made available. At the same time, the push towards other ‘open’ practices in research, such as open access publishing, means that the research sector is set for some significant and exciting changes in the ways in which research data and outputs are consumed. Given the potential commercial value of some of this data and the privacy and ethics concerns for human research, the management of research data promises to be complex and challenging but has the potential to unlock significant value.

The Productivity Commission and beyond

On 31 March 2017, the Productivity Commission tabled its report into data availability and use. The report tackles the problem of fragmentation in the public data space by proposing a new statutory officer, the National Data Custodian (NDC), to provide guidance on data usage, including the ethical issues that arise from large volumes of digitised data, and a new Commonwealth law, the Data Sharing and Release Act. To facilitate the release of data, the NDC will accredit a number of Accredited Release Authorities (ARAs), whose duties would include approving trusted users for access to sensitive data and determining whether datasets should be shared or released. The report anticipates that ARAs would largely be existing Australian Government agencies but is explicit that they should be funded to take on this additional work.

Recognising the need to prioritise the release of high-value data, the Productivity Commission recommends identifying National Interest Datasets (NIDs) for prioritised release. This access process should be managed by an ARA and overseen by the National Data Custodian. The report stresses that designating datasets as NIDs should lead to practical national benefit, not merely a curated list, as new access arrangements are implemented.

In fact, the Data Sharing and Release Act, as envisaged, would significantly alter access to data in Australia. The report envisages that, as well as legislating that all non-sensitive government data should be made available, it would override many of the existing secrecy provisions that prevent data sharing or publication. Additionally, the Australian Government’s Protective Security Framework should also be amended to make it easier to share de-identified data. Furthermore, the report proposes a comprehensive consumer right to enable individuals to access the data that corporations hold about them (Recommendation 5.1) and that the Australian Competition and Consumer Commission (ACCC) should be resourced to support people to exercise this right. This definition of ‘consumer data’ would include not only the personal information that is currently governed by the Privacy Act, but data that is collected about consumers, for instance, based on their social media activities or online browsing histories. The report justifies this major step by arguing that allowing individuals to participate meaningfully in the use of their data will improve outcomes for consumers (for example, by allowing them to receive more targeted offers of services) but also that granting greater access and control will maintain a social licence for data use by government and corporations.

Whether such ambitious changes can be achieved remains to be seen. Many organisations, in the public, private, and not-for-profit sectors, would find it challenging to grant the sorts of access envisaged by the report due to weaknesses in internal information governance. With sustained attention and resourcing, however, a significant shift in the way that organisations handle and value data could unlock significant value for the Australian community, both as individuals and collectively.

Productivity Commission 2017, Data Availability and Use: Overview & Recommendations, Report No. 82, Canberra p 25

Recommendations 8.1 – 8.4 address these legislative reforms.

Data Availability and Use: Overview & Recommendations p 57

Want to stay up-to-date?

Stay on trend and in the know when you sign up for our latest content