Australian Digital Observatory

The Challenge

Social media is a rich source of data for researchers from many disciplines. However, researchers face multiple barriers incorporating these data into their research, around ensuring data quality, technical data pre-processing, and accessing advanced analytical tools. 

Social media datasets are also dynamic with new content arriving constantly, making storage and analysis complex.

The Response

For a number of years, many groups across Australia have independently collected significant social media holdings to serve their local research community. 

The Australian Digital Observatory (ADO) project seeks to sustainably align and connect these established platforms into a national infrastructure that supports a diverse array of research communities. 

Researchers will be able to access and analyse dynamic digital data including existing collections of national interest across Twitter, FlickR, YouTube, Reddit, Instagram and gaming platforms such as Steam.

A databank will be built from the existing partner infrastructures for collection of dynamic digital data (including but not limited to social media data), improving accessibility to Australian researchers. 

The ADO will develop and support an ecosystem of hosted platforms, tools, methods, and workflows. This ecosystem intends to both enable researchers to access databank data and support researchers’ own collections and analytical pipelines. 

The ADO will offer targeted data collection capabilities and mediated access to our curated data collection. The ADO will also offer data modelling, as well as data analysis and visualisation such as topic modelling and sentiment analysis. 

Underpinning this ecosystem will be the FAIR data principles, data governance, open source software principles, and the collective knowledge of a community of practice.

Who Will Benefit

Researchers and research organisations will benefit from the project’s core features:

• Dynamic data ecosystem – The Australian Digital Observatory will become a hub of knowledge, tools, data, and services for a variety of dynamic digital data research needs.
• Community of practice – The Australian Digital Observatory will bring together researchers, projects, and platforms to share knowledge and collaborate.

The Partners

Our partners are:

• QUT
• University of Melbourne 
• University of New South Wales
• The University of Queensland 
• Queensland Cyber Infrastructure Foundation
• Google Cloud

Target Outcomes

The ADO will become a hub of knowledge, tools, data, and services for a variety of dynamic digital data research needs across a wide variety of disciplines, enabling innovative research.

The project will result in an interdisciplinary team of data science researchers and support staff at key locations around Australia who will offer data-related services, training, and research support in data collection, storage, tidying, pre-processing, analytics, and visualisation to support academics’ research data pipelines. 

These tasks can be time-consuming and often involve a steep learning curve for specific technical skills. The ADO can reduce or eliminate this burden so that researchers can focus on analysis and interpretation.