Environments to Accelerate Machine Learning-Based Discovery

The Challenge

The confluence of big data, machine learning (ML) techniques and parallel computing is making artificial intelligence (AI) useful across a range of research areas. Increasing sophistication, insight and accuracy is driving a strong and growing appetite across research groups for access to ML capacity, services, libraries, expertise and training.

The Response

This project brought together ML tools, libraries, and access to data to provide a consolidated platform for ML-based research, deployed on high-end graphics processing units (GPUs) at Monash University and The University of Queensland.

The project focused on improving our knowledge about deploying and configuring the most widely used ML tools and libraries (including Tensorflow, PyTorch, Keras) and data preprocessing tools (such as Pandas, Matlab and Moa). This enabled quick and efficient customisation, deployment and support of ML tools and libraries while leveraging the limited hardware available.

The environment developed for ML integrates with software development environments and supports core ML tools for preprocessing, annotating, training and validation. It was accompanied by a training program to increase researcher knowledge of ML tools. 

The project established a national community for researchers using ML, and produced advanced training materials that can be reused. 

Who Will Benefit

This project benefits researchers who require access to machine learning tools and computing and are interested in upskilling their machine learning expertise. It’s suitable for researchers across a wide range of domains including life sciences, engineering, computer science and, increasingly, the humanities and social sciences.

The Partners

• Monash University
• National Computational Infrastructure
• Pawsey Supercomputing Centre
• QCIF
• The University of Queensland

Target Outcomes

Fundamental and applied ML researchers have integrated access to low and high end GPUs, ML tools and libraries, data duration tools and reference datasets. 

This project has created Strudel2 an interactive environment to test ML ideas, demonstrate success and to ‘fail fast’. This can be accessed via a CVL desktop, or on MASSIVE.

The Machine Learning Community of Practice for Australia established under this project is facilitating knowledge sharing, best practice frameworks and advanced training materials. 

Other resources for making ML tools more efficient, including benchmarking tools and job resource monitoring scripts, can be found on ML4AU.