Machine Observation Data Processing Infrastructure

The Challenge

Environmental scientists are faced with a mismatch between the ever-growing volume of raw data from camera traps, acoustic recorders and drones and the ability to store, process and analyse the videos, images, and audio recordings at a national scale.

Efforts to develop pipelines and infrastructure to process the machine observation data are siloed and duplicated, combined with few, if any, standards to ensure data interoperability. This hinders the merging of different sensor data sources and thereby achieving continental-scale monitoring and trend analysis.

The Response

The Machine Observation Data Processing Infrastructure (MODs) program will support the establishment of national shared infrastructure, services and standards to enable processing and reuse of automated observation and sensor data. The program is part of the Planet Research Data Commons, which is establishing national-scale data infrastructure for earth and environmental decision making.

The initial program focus is on processing image, acoustics and video data. National-scale infrastructure is needed to manage and maximise the impact of the large amount of data coming from deployments of camera traps, acoustic recorders and drones. These data need to be made analysis-ready to enable the goal of national-scale continuous monitoring. 

In collaboration with research institutions, government, NGOs and national research infrastructures, this program will coordinate and align technical capability across multiple partners to enhance continental-scale observation. The data outputs will be made available in analysis-ready formats that can be easily integrated into environmental research infrastructures and repositories. This will include implementation of persistent identifiers for observation data and associated instrumentation. 

Consultation is underway to establish a national capability that supports and synthesises data from machine-based observations that’s supported by dedicated storage and a high throughput computing pipeline.

Who Will Benefit

Researchers, government decision makers, non-governmental organisations (NGOs).

Program Design and Consultation

The Planet RDC is following a community co-design process to deliver its programs. The Planet RDC Program Description outlines the national consultation process and outcomes. 

Through our 2022 national consultation and policy analysis we identified major national stakeholders and national priorities. Further consultations with the community have identified data from drones, camera traps and ecoacoustics recorders to be the focus of data challenges for this program. A co-design workshop with over 50 stakeholders was held in February 2024 in Brisbane – read about the workshop and read the workshop report. 

Building digital infrastructure to process machine observation data could include provisioning a  combination of data storage, compute (cloud) resources, HPC, software tools for Extract Transform and Load (ETL), integration and visualisation, machine learning frameworks, data processing libraries, containerisation and orchestration, cybersecurity, documentation tools, and data governance frameworks.

We have published program guidelines for potential project partners, which include the program strategy and intended outcomes, along with clear guidance around expectations for projects. View the MODs program guidelines.

To learn more about collaborating with us on this program, please get in touch.

Target Outcomes

Overall this program will:

• enhance the interoperability of data across domains, disciplines and sectors to support multidisciplinary research
• create more efficient workflows that enable researchers to focus on research rather than data wrangling
• enable continental-scale monitoring on a national scale through improved capability and capacity.

Key Resources

• Read about the co-design workshop for the program, Workshop Addresses Wildlife Camera Trap, Ecoacoustic and Drone Data Challenges
• Read the National Shared Machine Observation Data Processing Infrastructure Workshop Report
• View the MODs program guidelines
• Learn more about the Planet Research Data Commons
• Read the program description for the establishment phase of the Planet Research Data Commons.