Your Seasonal Forecast API Guide

Summary:

This document explains how to access Your Seasonal Forecast data via the Eratos Senaps data platform. We make the Boorowa location available as part of the trial product as an example. This guide shows how to use the Sensor Data API provided by the Eratos Senaps platform to extract results.

Prerequisites:

Authorisation and license agreement. You can purchase a subscription through our AgData Shop.
An understanding of how Your Seasonal Forecast data is structured. Please see the Your Seasonal Forecast User Guide document for details.

Please refer to the shop Terms and Conditions for specific terms relating to accessing Your Seasonal Forecast data.

Introduction

The your Seasonal Forecast prediction outputs are supplied in a few arrays of data, and the confidence information is available as JSON metadata. Senaps provides API's for accessing these arrays of data (referred to as vectors in Senaps) and confidence metadata. These instructions describe how to access those vectors and metadata. A prior understanding of the contents of those vectors is assumed. Please see the Your Seasonal Forecast User Guide document for details on the product and how to interpret the contents of the data vectors.

Authentication (AgDataShop client ID)

If you were provided with a client_id and secret in your order email, use the process described in Authentication for AgDataShop product APIs. After this process you will have obtained an Access Token which must be applied to every API request as a HTTP Authorization Bearer token. Please note tokens expire as per standard OAuth2 Client Credentials flow.

Authorization: Bearer <INSERT TOKEN>

Authentication (Eratos Senaps API key)

If you have an Eratos Senaps API key then please insert this key as an API header with the name apikey on all API requests.

Trial data Object identifiers:

Object Type	Trial product identifiers

Object Type

Trial product identifiers

Location

cscf-r.csiro-trial.boorowa

cscf-r.csiro-trial.narrogin

cscf-r.csiro-trial.toowoomba

Categorised Forecast data stream

cscf-r.csiro-trial.boorowa.categorised_forecast

cscf-r.csiro-trial.narrogin.categorised_forecast

cscf-r.csiro-trial.toowoomba.categorised_forecast

Monthly rain to date data stream

cscf-r.csiro-trial.boorowa.monthly_rain_to_date

cscf-r.csiro-trial.narrogin.monthly_rain_to_date

cscf-r.csiro-trial.toowoomba.monthly_rain_to_date

The confidence information is contained within metadata of the location object, the rainfall data and predictions are accessed through the data streams objects.

Accessing Categorical Forecast Data - Example using interactive API-docs

As an example, we can use the Senaps interactive API Documentation to access data for the Boorowa trial site.

The Sensor Data API operations are documented here: https://senaps.eratos.com/api-docs/#/ , under the default definition of "Sensor Data API".

To authenticate API calls you must click the 'Authorize' button and enter your Access Token in the bearAuth field or your API key in the headerKey field.

To see details of the Boorowa Forecast stream scroll down and expand the GET /streams/{id} section by clicking on that 'GET' button. You can now enter the parameters to get the details of the data stream.

Click "Try it out" and paste "cscf-r.csiro-trial.boorowa.categorised_forecast" without the quotes into the ID field then click the "Execute" button.

If your stream is valid and authentication token is valid will see a response similar to this:

The full Response body is similar to:

{
  "_links": {
    "self": {
      "href": "https://senaps.eratos.com/api/sensor/v2/streams/cscf-r.csiro-trial.boorowa.categorised_forecast"
    },
    "observations": [
      {
        "href": "https://senaps.eratos.com/api/sensor/v2/observations?streamid=cscf-r.csiro-trial.boorowa.categorised_forecast"
      }
    ]
  },
  "id": "cscf-r.csiro-trial.boorowa.categorised_forecast",
  "reportingPeriod": "P1M",
  "resultsSummary": {
    "count": 3,
    "first": {
      "t": "2020-05-15T00:00:00.000Z",
      "v": {
        "v": [
          1,
          4.4,
          29.8,
          50.5,
          99.9,
          174.7,
          0,
          0.303,
          0.697,
          3,
          8.696,
          37.3664,
          71.8932,
          109.106
        ]
      }
    },
    "last": {
      "t": "2020-07-15T00:00:00.000Z",
      "v": {
        "v": [
          32.3,
          69.5,
          113.9,
          163.6,
          258.6,
          303.3,
          0,
          0.0606,
          0.9394,
          3,
          120.196,
          186.2804,
          228.58,
          296.72
        ]
      }
    }
  },
  "resulttype": "vectorvalue",
  "samplePeriod": "P3M",
  "usermetadata": {
    "version": [
      1
    ],
    "mappings": [
      {
        "i": 0,
        "id": "obs.p0",
        "label": "Observed Rainfall at 0th percentile",
        "group": "observed",
        "percentile": 0,
        "categories": null
      },
      {
        "i": 1,
        "id": "obs.p05",
        "label": "Observed Rainfall at 5th percentile",
        "group": "observed",
        "percentile": 0.05,
        "categories": null
      },
      {
        "i": 2,
        "id": "obs.p33",
        "label": "Observed Rainfall at 33rd percentile",
        "group": "observed",
        "percentile": 0.33,
        "categories": null
      },
      {
        "i": 3,
        "id": "obs.p66",
        "label": "Observed Rainfall at 66th percentile",
        "group": "observed",
        "percentile": 0.66,
        "categories": null
      },
      {
        "i": 4,
        "id": "obs.p95",
        "label": "Observed Rainfall at 95th percentile",
        "group": "observed",
        "percentile": 0.95,
        "categories": null
      },
      {
        "i": 5,
        "id": "obs.p100",
        "label": "Observed Rainfall at 100th percentile",
        "group": "observed",
        "percentile": 1,
        "categories": null
      },
      {
        "i": 6,
        "id": "frcst.p.t1",
        "label": "Proportion of ensembles in tercile 1",
        "group": "proportion_ensemble",
        "categories": null
      },
      {
        "i": 7,
        "id": "frcst.p.t2",
        "label": "Proportion of ensembles in tercile 2",
        "group": "proportion_ensemble",
        "categories": null
      },
      {
        "i": 8,
        "id": "frcst.p.t3",
        "label": "Proportion of ensembles in tercile 3",
        "group": "proportion_ensemble",
        "categories": null
      },
      {
        "i": 9,
        "id": "frcst_cat",
        "label": "Forecast Category",
        "group": "forecast_category",
        "categories": [
          {
            "value": 0,
            "label": "Inconclusive"
          },
          {
            "value": 1,
            "label": "Tercile 1"
          },
          {
            "value": 2,
            "label": "Tercile 2"
          },
          {
            "value": 3,
            "label": "Tercile 3"
          }
        ]
      },
      {
        "i": 10,
        "id": "frcst.p05",
        "label": "Forecast Rainfall at 5th percentile",
        "group": "forecast",
        "percentile": 0.05,
        "categories": null
      },
      {
        "i": 11,
        "id": "frcst.p33",
        "label": "Forecast Rainfall at 33rd percentile",
        "group": "forecast",
        "percentile": 0.33,
        "categories": null
      },
      {
        "i": 12,
        "id": "frcst.p66",
        "label": "Forecast Rainfall at 66th percentile",
        "group": "forecast",
        "percentile": 0.66,
        "categories": null
      },
      {
        "i": 13,
        "id": "frcst.p95",
        "label": "Forecast Rainfall at 95th percentile",
        "group": "forecast",
        "percentile": 0.95,
        "categories": null
      }
    ]
  },
  "_embedded": {
    "metadata": [
      {
        "length": 14,
        "type": ".VectorStreamMetaData"
      }
    ],
    "organisation": [
      {
        "_links": {
          "self": {
            "href": "https://senaps.eratos.com/api/sensor/v2/organisations/csiro"
          }
        },
        "id": "csiro"
      }
    ],
    "groups": [
      {
        "_links": {
          "self": {
            "href": "https://senaps.eratos.com/api/sensor/v2/groups/cscf-r.csiro"
          }
        },
        "id": "cscf-r.csiro"
      }
    ],
    "location": [
      {
        "_links": {
          "self": {
            "href": "https://senaps.eratos.com/api/sensor/v2/locations/cscf-r.csiro-trial.boorowa"
          }
        },
        "id": "cscf-r.csiro-trial.boorowa"
      }
    ]
  }
}

This response contains the metadata described in the User Guide as well as the earliest and most recent results. The results are the arrays of numbers - see the Your Seasonal Forecast API Guide to interpret the individual values.

Note with this forecast product only the most recent forecast is stored and it will contain 3 sets of values (predictions for 1, 2 and 3 months out). Note that the GET /Streams/{id} call above only returns the first and last predictions. To get all three forecasts the GET /observations API call should be used. As before click "Try it out", enter "cscf-r.csiro-trial.boorowa.categorised_forecast" without the quotes into the streamid parameter and click the "Execute" button. All other parameters can be left blank.

The result will be similar to below, it contains values for future months, in this case the forecast was produced on May 15 so results (predictions) exist for the periods mid May to mid June (timestamped 15 May), mid June to mid July (timestamped 15 June) and mid July to mid August (timestamped 15 July).

{
  "_links": {
    "self": {
      "href": "https://senaps.eratos.com/api/sensor/v2/observations"
    }
  },
  "_embedded": {
    "stream": {
      "_links": {
        "self": {
          "href": "https://senaps.eratos.com/api/sensor/v2/streams?id=cscf-r.csiro-trial.boorowa.categorised_forecast",
          "id": "cscf-r.csiro-trial.boorowa.categorised_forecast"
        }
      }
    }
  },
  "results": [
    {
      "t": "2020-05-15T00:00:00.000Z",
      "v": {
        "v": [
          1,
          4.4,
          29.8,
          50.5,
          99.9,
          174.7,
          0,
          0.303,
          0.697,
          3,
          8.696,
          37.3664,
          71.8932,
          109.106
        ]
      }
    },
    {
      "t": "2020-06-15T00:00:00.000Z",
      "v": {
        "v": [
          21,
          45.1,
          63.2,
          103,
          183.3,
          217.9,
          0.0303,
          0,
          0.9697,
          3,
          65.648,
          110.8092,
          145.2588,
          179.714
        ]
      }
    },
    {
      "t": "2020-07-15T00:00:00.000Z",
      "v": {
        "v": [
          32.3,
          69.5,
          113.9,
          163.6,
          258.6,
          303.3,
          0,
          0.0606,
          0.9394,
          3,
          120.196,
          186.2804,
          228.58,
          296.72
        ]
      }
    }
  ],
  "count": 3
}

Accessing Categorical Forecast Data

Accessing the rain to date data uses the same process as above, but with the stream id set to 'cscf-r.csiro-trial.boorowa.monthly_rain_to_date'.

Accessing Confidence Data - Example using interactive API docs

The confidence level/skill information is attached as metadata to the location object in Senaps. It can be obtained by making a GET /locations/{id} API Call. The location id for the Boorowa trial site is 'cscf-r.csiro-trial.boorowa'

after clicking 'Execute' the body of the return value contains the metadata as indicated below at the JSON path usermetadata.

{
  "_links": {
    "self": {
      "href": "https://senaps.eratos.com/api/sensor/v2/locations/cscf-r.csiro-trial.boorowa"
    }
  },
  "description": "cscf-r.csiro-trial_station",
  "geojson": {
    "type": "Point",
    "coordinates": [
      148.716,
      -34.433
    ]
  },
  "id": "cscf-r.csiro-trial.boorowa",
  "usermetadata": {
    "skill_data": {
      "score_by_month": {
        "lead_time": {
          "month1": [
            2,
            2,
            2,
            2,
            2,
            1,
            2,
            3,
            2,
            2,
            1,
            1
          ],
          "month2": [
            2,
            2,
            2,
            2,
            1,
            2,
            2,
            2,
            1,
            2,
            1,
            2
          ],
          "month3": [
            1,
            2,
            2,
            2,
            1,
            1,
            2,
            2,
            2,
            3,
            1,
            1
          ]
        }
      },
      "score_by_month_legend": [
        {
          "value": 1,
          "label": "Low"
        },
        {
          "value": 2,
          "label": "Medium"
        },
        {
          "value": 3,
          "label": "High"
        }
      ],
      "correct_by_month": {
        "lead_time": {
          "month1": [
            52.1739,
            43.4783,
            13.0435,
            26.087,
            34.7826,
            26.087,
            26.087,
            39.1304,
            34.7826,
            56.5217,
            26.087,
            30.4348
          ],
          "month2": [
            43.4783,
            21.7391,
            21.7391,
            21.7391,
            30.4348,
            34.7826,
            30.4348,
            39.1304,
            30.4348,
            34.7826,
            17.3913,
            34.7826
          ],
          "month3": [
            17.3913,
            43.4783,
            26.087,
            21.7391,
            39.1304,
            21.7391,
            39.1304,
            30.4348,
            21.7391,
            34.7826,
            26.087,
            8.6957
          ]
        }
      },
      "correct_by_month_legend": [
        [
          "value: percentage"
        ]
      ],
      "incorrect_by_month": {
        "lead_time": {
          "month1": [
            13.0435,
            4.3478,
            4.3478,
            8.6957,
            17.3913,
            17.3913,
            4.3478,
            0,
            13.0435,
            4.3478,
            17.3913,
            21.7391
          ],
          "month2": [
            13.0435,
            8.6957,
            8.6957,
            13.0435,
            17.3913,
            8.6957,
            4.3478,
            4.3478,
            13.0435,
            13.0435,
            13.0435,
            8.6957
          ],
          "month3": [
            13.0435,
            4.3478,
            13.0435,
            13.0435,
            17.3913,
            17.3913,
            13.0435,
            8.6957,
            13.0435,
            0,
            21.7391,
            13.0435
          ]
        }
      },
      "incorrect_by_month_legend": [
        [
          "value: percentage"
        ]
      ]
    },
    "skill_version": [
      "Skill_Score_v03"
    ],
    "skill_notes1": [
      "Skill scores defined using proportion of hindcasts incorrect and correct minus inconclusive hindcasts. If incorrect > 1 in 4.5 years; skill score = low. If incorrect < 1 in 4.5 years and > 1 in 20 years; skill score = med. If incorrect < 1 in 20 years; skill score = high. If incorrect score  > 1 in 4.5 years & correct > 1 in 1.5 years, skill score = med."
    ],
    "skill_datasource": {
      "hindcasts": [
        "Hindcasts: ACCESS-S1 v03. Calibrated to 5km x 5km using quantile-quantile matching approach."
      ],
      "obs": [
        "SILO grid 5km x 5km"
      ],
      "skill_table": [
        "0001_boorowa_skill_summary.csv"
      ]
    },
    "skill_updated_on": [
      "2020-05-15 17:26:31"
    ]
  },
  "_embedded": {
    "streams": [
      {
        "_links": {
          "self": {
            "href": "https://senaps.eratos.com/api/sensor/v2/streams/cscf-r.csiro-trial.boorowa.categorised_forecast"
          }
        },
        "id": "cscf-r.csiro-trial.boorowa.categorised_forecast"
      },
      {
        "_links": {
          "self": {
            "href": "https://senaps.eratos.com/api/sensor/v2/streams/cscf-r.csiro-trial.boorowa.monthly_rain_to_date"
          }
        },
        "id": "cscf-r.csiro-trial.boorowa.monthly_rain_to_date"
      }
    ],
    "organisation": [
      {
        "_links": {
          "self": {
            "href": "https://senaps.eratos.com/api/sensor/v2/organisations/csiro"
          }
        },
        "id": "csiro"
      }
    ],
    "groups": [
      {
        "_links": {
          "self": {
            "href": "https://senaps.eratos.com/api/sensor/v2/groups/cscf-r.csiro"
          }
        },
        "id": "cscf-r.csiro"
      }
    ]
  }
}

Programmatic access using the API endpoint:

(Most of the following information can be deduced from the API Docs examples above.)

a. The base path for the Sensor Data API is: https://senaps.eratos.com/api/sensor/v2/

b. The path for each type of request is show in the following table

Object class	Query path	Example full url

Object class	Query path	Example full url
Location	/locations	https://senaps.eratos.com/api/sensor/v2/locations
Stream	/streams	https://senaps.eratos.com/api/sensor/v2/streams
Observation	/observations	https://senaps.eratos.com/api/sensor/v2/observations

c. The request method full all requests is GET
d. Every request needs an authentication token. The process for obtaining a token is described above, the Access Token must be added as a HTTP Bearer token header.
e. Example parameters can be seen in the api-docs and the above examples.

The equivalent calls to above are:

^{https://senaps.eratos.com/api/sensor/v2/observations?streamid=cscf-r.csiro-trial.boorowa.categorised_forecast}

^{https://senaps.eratos.com/api/sensor/v2/locations/cscf-r.csiro-trial.boorowa}

The 'Try it out' functionality in the API Docs also provides examples of making those same API calls using the common command line tool curl.