Application Ready Climate Projections - Developer Guide

Introduction

This document describes how to download the Application Ready Climate Data purchased through the CSIRO AgDataShop.

For details of the scientific methodology and quality control processes please refer to the technical report (doi.org/10.25919/88wm-mw53).

The AgDataShop provides a simple Python module to assist in the download of purchased datasets to enable easier integration with customer data pipelines. Advanced users (including programmer not familar with Python) may wish to download the datasets using the HTTP API, details can be found at the end of the document.

Dataset Structure

Application Ready Climate Data is made available in NetCDF (compliant with CF conventions) format as smaller tiles or a national dataset. The AgDataShop product page describes the tile variants and their spatial extent. 

Below is the the full list of purchasable tiles:

• SAMPLE

• AUS

• NSW1, NSW2, NSW3

• NT1, NT2,

• QLD1, QLD2, QLD3,

• SA1, SA2,

• TAS1,

• VIC1, VIC2,

• WA1, WA2, WA3

NOTE: You will only be permitted to download data for tile/s you have purchased.

Each tile includes a history dataset and a set of projection datasets for every combination of model, variable, period and scenario.

Available models include:

• ACCESS1-0,

• CanESM2,

• CESM1-CAM5,

• CNRM-CM5,

• GFDL-ESM2M,

• HadGEM2-CC,

• MIROC5,

• NorESM1-M

Available variables include:

• pr (Precipitation)

• tasmax (Daily Maximum Near-Surface Air Temperature)

• tasmin (Daily Minimum Near-Surface Air Temperature)

• rhs (Daily Relative humidity at surface)

• rsds (Daily Surface solar radiation downwards)

Available scenarios include:

• rcp26

• rcp45

• rcp85

Available periods include:

• 2016-2045

• 2036-2065

• 2056-2085

Available frequencies include:

• daily

• monthly

• seasonal

• annual

Available temporal contexts include:

• historical

• projection

NOTE: Further details about the scientific methodology for the datasets available in this product can be found in the technical report at doi.org/10.25919/88wm-mw53

Authentication Credentials

After purchasing access to data through the Data Shop, you will be allocated unique security credentials for accessing each product. To find these credentials, follow the steps below:

1. Sign into your Data Shop account.

2. Navigate to the My Account tab.

3. Click on the Orders menu item.

4. Under Recurring payments, select View order for the relevant order.

5. On the Order information page, check for the Note(s) heading. Here, you will find your client_id and client_secret.

Python Example to download data

The follow example provides a simple recipe for downloading and visualising the sample data (available as a free purchase via the AgData Shop [TODO- link]) using popular Python scientific libraries.

Prerequisites

Ensure a Python (tested on 3.10+) environment is available, we recommend using virtual environments to help isolate environments for different project dependencies.

Install the AgDataShop python client library using pip from a command line prompt:

Dataset Downloads

Import the Application Ready Climate Data class called CIHP1:

Instantiate the CIHP1 using your client id and secret provided when you purchased the product:

In this example the CIHP1 class will handle the OAuth2 authentication. This removes the need to handle the OAuth authentication process directly. When you purchase a product your order note will contain a client id and secret, these are required to complete the authentication (see Authentication Credentials section above). 

You can now specify filter criteria and start the download of the purchases datasets. The download_dataset function will return a list of paths to the files downloaded.

By default the download_dataset() method will display a progress bar, disable this by adding the argument progress_bar=False 

Example visualisations of downloaded data

Install the required dependencies :

Map Visualisation

The following example renders a map of one file from the sample dataset downloaded above.

Open

Time series point visualisation

In the below example we use the data selection features of x-array to extract and plot a time series for a location given by longitude and latitude coordinates.

Open

Jupyter Notebook example

A python Jupyter notebook example implementing the above example is available here: 

Alternative: HTTP API dataset downloads

The following guidance is for clients wishing to implement dataset downloads without using the Python module explained above. This can be useful for integration into bespoke analysis tools or used as a guide for access through other programming languages.

All download requests must authenticate using a OAuth2 Client Credentials Flow described here, Note most programming languages will have libraries to support OAuth2 authentication. Once the access token has been obtained, the token is used to request data downloads from HTTP download URLs. The access token must be provided as a HTTP Authorization header. 

HTTP download URLs can be determined using the following URL templates, substituting TILE, MODEL, VARIABLE, FREQUENCY, PERIOD and SCENARIO with values described at the top of this page, and BASELINE using the table below.

For all daily datasets:

For monthly, seasonal and annual datasets:

For all historical datasets:

The BASELINE dataset can be determined from the variable:

Contact Information

Pease Contact Ian.Macadam@csiro.au for questions on this product or agdatashop@csiro.au for general questions about shop products.