Welcome to AODN Cloud Optimised’s documentation!

This project is hosted on GitHub: aodn_cloud_optimised.

The AODN Cloud Optimised project converts IMOS datasets from source formats (NetCDF, CSV) to cloud-optimized formats (Parquet, Zarr) for efficient cloud-based access and analysis. It includes an optional MCP server for AI integration.

Documentation:

• Getting Started
  • Installation
  • Recommended: Data Processing (Core Only)
  • Jupyter Notebooks (Data Analysis)
  • Contributing to the Project
  • Direct Installation (pip / Poetry)
  • What’s Next?
  • Quick Start
• How-To Guides
  • Querying Cloud-Optimised Data
  • Dataset Configuration
  • Adding the dataset to pyproject.toml
  • Configuration Validation
  • Cluster Configuration & Distributed Processing
  • Notebooks
  • MCP Server
  • Update All Metadata Script
• Development
  • Development Setup
  • Debugging
  • Debug Unittests
  • Debug a cloud optimised pipeline
  • Testing
  • Building Documentation
  • Creating a Release
• Module Overview
  • Generic Cloud Optimised Creation
  • Create Dataset Configuration (semi-automatic)
  • Create AWS Registry dataset entry
  • Common Handler
  • Parquet Handlers
  • Zarr Handler
• Usage
  • Generic Cloud Optimised Creation Script
  • As a python module

Quick Links:

• Installation
• Recommended: Data Processing (Core Only)
• Jupyter Notebooks (Data Analysis)
• Contributing to the Project
• Direct Installation (pip / Poetry)
• What’s Next?
• Quick Start

Choose your path:

• 👤 New user? Start with Getting Started

• 🔧 Want to process data? See Quick Start or Dataset Configuration

• 📓 Write notebooks? Check Notebooks

• 🚀 Using MCP server? Go to MCP Server

• 👨‍💻 Contributing? Visit Development

• 📚 API reference? See Module Overview

Indices and tables

• Index

• Module Index

• Search Page