Data Product Revisions and Releases

In this section, we introduce the concepts of provisional data, data releases, and data product revisions.

Currently, NEON data are published as data products, and 1) all of the data in a product is comparable from the first data collection date to the last, and 2) all data in a product is provisional, meaning that it can be changed if needed. In the near future, NEON will be providing 1) data revisions, where a instrument or protocol has significantly changed to the extent that users should be aware of potential issues with incompatibility, and 2) data releases, which will ensure reproducibility of analyses over the lifetime of the Observatory.

Provisional Data

Until the first data Release is published at the beginning of 2021 (see below), all data are provisional. Provisional means that data are dynamic and can be updated at any time, without guarantee of reproducibility. After the first data Release, all data collected since the most recent Release are provisional. Provisional data allow for near-immediate availability of data on the Data Portal, while retaining the ability to make corrections or additions as they are identified. To determine if provisional data may have changed since a previous download, check the timestamp at the end of the filename. The timestamp indicates the date and time the files were generated, and if corrections have been made, a new timestamp will be applied. Currently all available data are provisional.

Data Product Revisions

A data product revision may be created when a change to a sensor or processing pipeline is so significant that data from different revisions of the same data product are not directly comparable and should be used with caution when combining for use or analysis. Upon a data product revision, the REV field of the data product identifier will be incremented. The data product identifier takes the form DPL.PRNUM.REV, where DPL is the data product level, PRNUM is the product number, and REV is the product revision. For example, the data product identifier for the first revision of the 2D Wind Speed and Direction data product is DP1.00001.001. The full revision history for each data product can be found in the Data Product Catalog which will also include short summaries of the changes made between revisions.

Data Releases

Data release timeline

NEON data will be released once per year, with a time lag of six to 18 months depending on the time needed for data processing. Provisional data will continue to be published until the next year's release.

A data release is a set of data files that is static (unchanging), always available to end users, and citable. NEON plans to generate data releases annually, beginning 2021. One release will be generated for each data product annually. Each year's release will include all data collected for each data product up to a subsystem-specific provisional period prior to the release date. The provisional period allows sufficient time for all downstream lab analyses and quality control processes to be completed. For IS, AOP, and OS data products, the respective provisional periods are 6, 6, and 18 months. For instance, NEON's first release, planned for January 2021, is expected to include IS and AOP data through June 2019 and OS data through June 2018. The provisional period of OS data is longer due to latencies that are expected for data that need to be returned from external labs.

NEON data releases will remain stable and accessible throughout the lifetime of the observatory, and will be assigned Digital Object Identifiers (DOIs). DOI URLs will always resolve back to the dataset, and are thus ideal for citing NEON data in publications and applications.

The data in a release will never change. Any necessary data corrections will be reflected in a subsequent release.

Data that are not included in any NEON release are considered provisional and may be updated at any time, without guarantee of reproducibility. Although all NEON data are research-grade and undergo quality checks prior to publishing, additional data or quality information may become available at a later date. Publication of provisional data enables rapid data availability, while allowing for the possibility that additions and corrections may need to be made in the subsequent months. Users should plan to archive provisional data used in publication in an appropriate repository, as NEON will not assign a DOI until the data are included in an official release.

The default download from the NEON Data Portal will include the most recent release plus all available provisional data. Metadata included with the downloaded data files will denote the release identifier. The full release history for each data product will be available from the Data Product Catalog.