Data Availability

NEON end users commonly ask how quickly new data will be published after it is collected, and whether and when data will be available in the future. NEON began full operations in May 2019. Data collection during construction began at different times for every data product and every site, and data pipelines were built in parallel with early data collection. As a consequence, current data availability varies by site and by data product.

Data availability can depend on the expected sampling frequency and timing for any given data product throughout a year, as well as how long it might take for data to go from the point of collection through the data processing pipelines and finally to the portal for use. In general, for automated instrument data, this process will take less time than for observational sampling. Below, we explain some of the processes and limiting factors that affect data availability for each data collection system. 

  • Current availability of data by product or by site can be explored through our interactive Data Availability Charts.
  • For detailed information about current availability and general processing latencies of a given data product, visit each data product's detail page through the links found in Explore Data Products.
  • For information about the quality of available data, see Data Quality.
  • To learn more about data that are sent to external repositories for hosting, see Externally Hosted Data.

    Observation System (OS) Data

    Collection Frequency

    Frequency of data collection varies widely across observational data products. Information about collection frequency can be found in the data product detail pages in the Explore Data Products page and the data product readme files that are packaged with downloaded data; more detailed collection information can be found in protocol documents.

    Overall, OS data collection frequency varies from three times per week to once every five years. For products collected every five years, one-fifth of all sites are collected each year. Due to this schedule, data may not yet be available for some sites.

    Data Latency

    Observational data are collected by human observers, and are either directly observed or derived from analyses on physical samples collected in the field. The collection and analysis workflows result in four general categories of data latency for OS data:

    1. Data collected directly in the field
    2. Data from samples analyzed in NEON lab facilities
    3. Data from samples shipped to contracted analytical facilities on a rolling basis - i.e., samples are shipped as soon as they are collected or in small batches throughout the year
    4. Data from samples shipped to contracted analytical facilities on an annual basis - i.e., samples are shipped in bulk at the end of the field season

    With a few exceptions, the latency duration increases from category 1 to 4. Field-collected data may be available on the NEON Data Portal within a month of collection, while samples analyzed on a rolling basis are typically available between 90 days and nine months after collection. Samples analyzed in bulk annually are a small proportion of OS data and have a latency of slightly more than one year.

    Many OS data products include multiple data tables, each of which may fall into a different one of the categories above. For example, in the data product Plant foliar physical and chemical properties (DP1.10026.001), the cfc_fieldData table contains records of leaf collection in the field, the cfc_LMA table contains leaf mass per area measurements collected in the NEON lab, and the cfc_chlorophyll, cfc_elements, and cfc_lignin tables contain analyses performed by contracted facilities. Each data table is published to the data portal as it becomes available, so two months after leaf collection, the cfc_fieldData and cfc_LMA data may be available, while the chemical analyses are still pending. If downloaded data do not include all expected tables, check back later!

    Instrument System (IS) Data

    Collection Frequency

    Raw sensor measurement frequency varies by data product and ranges from 40 Hz to one measurement every five minutes. Averaging is commonly applied during the process of transforming raw measurements into the data products served on the NEON Data Portal. Most IS data products are provided at two aggregation intervals, commonly 1 and 30 minutes. The Explore Data Products and associated data product detail pages provide information on raw sensor measurement frequency and aggregation intervals, in addition to other details about each data product.

    Most often, sensors operate continuously after install at a site, barring local power or other unplanned issues. However, some sensor locations at some sites are removed seasonally due to adverse or inappropriate measurement conditions. For example, some aquatic sensors are removed during periods in which the lake or stream is dry or frozen, and the lowest 2D wind sensor is removed from towers that experience snow. Currently OKSR is the only site where the instrumented system is shut down entirely during winter due to lack of power supply. No data are processed during this period (no files available). We are working towards pausing the processing during any period that a sensor is seasonally removed. However, during NEON’s construction phase and in some current cases, processing is or was not paused for these scenarios, resulting in downloadable files with only quality flags populated (no sensor values). 

    Although there is a high degree of consistency in data product collection across sites, not all terrestrial/aquatic instrument data products are collected at all terrestrial/aquatic sites. To learn more about measurement collection across site types, visit the pages for Meteorology, Phenocam, Soil, Surface Water, and Groundwater sensor measurements. 

    Data Latency

    Instrumented data collected during any given month are published to the NEON Data Portal typically in the second week of the following month. All sites are online and streaming instrumented data, and most instrumented data products are continuously made available according to this monthly schedule. 

    All terrestrial instrumented sites are producing surface-atmosphere exchange data (DP4.00200.001: Bundled data products – eddy covariance) and the processed data availability for all sites is near-current (following the publishing schedule above). However, the code base for this data suite is still being updated frequently to improve quality flagging routines and output data coverage. 

    Airborne Observation Platform (AOP) Data

    Collection Frequency and Data Latency

    The AOP collects airborne remote sensing data at a subset of the NEON sites annually. The schedule for each flight season is published on the NEON website several months prior to commencement of collections (see this page for the latest updates on AOP’s flight season schedule), and provides the time period that the AOP will be at each domain. Precise collection times for a site within the domain’s scheduled time period is not predetermined, as it is primarily driven by weather conditions encountered upon arrival. The total collection time required for each site will vary based on the size of the area covered as well as weather conditions. Under good weather conditions, the smallest NEON sites can be collected in a single day, while the largest could take up to a week. If weather conditions prove unsuitable for collections during the entire scheduled time at a domain, portions of sites or entire sites may not be collected. The AOP data will typically be published on the NEON data portal in less than 60 days after the final collection day at a site.