Release 2023
Release Tag: RELEASE-2023
Release Date: January 27, 2023
Contents: JSON manifest
neonUtilities: use v2.2.0 or higher
NEON’s third data Release, RELEASE-2023, includes 161 data products annotated by 129 DOIs.
- RELEASE-2023 includes Instrumented Systems (IS) data through June 2022, and Observational Systems (OS) and Airborne Observation Platform (AOP) data through December 2021.
- Data products that are exclusively hosted by an external repository are excluded. These are 1) the Phenocam data products Phenology images (DP1.00033.001), Land-water interface images (DP1.20002.001), and Snow depth and understory phenology images (DP1.00042.001), and 2) the AERONET product Spectral sun photometer - calibrated sky radiances (DP1.00043.001).
- Sub-products that are exclusively downloadable via a parent product are associated with the same DOI as their parent.
- Some data products or subsets of data products (certain date ranges) have been excluded. Data may be available in excluded products and time periods; these data are provisional, and subject to change. Please refer to the exclusion table below.
- For all OS data products, data quality improvement and error correction are continuous and ongoing processes. Routine data entry errors (such as transcription errors, duplicate records, or adjusted spatial coordinates) have been corrected where possible. Due to the large volume of data (>20 million data records), these corrections are not described individually. The table below describes significant changes to data products between releases.
- For all IS data products, data were reprocessed to incorporate minor and/or isolated corrections to quality control thresholds, sensor installation periods, geolocation data, and manual quality flags. More specific changes to data products between this release and the previous release have been described in a table below. Please refer to this table as well as the issue logs for each data product for more details.
- AOP data products, due to their high volumes, will be exclusively available in this release until January 2024. The DOIs for the AOP data products in previous releases have been tombstoned.
Data Exclusions
| Data Product ID | Data Product Name | What is excluded besides 2022 | Reason for exclusion |
|---|---|---|---|
| DP1.00041.001 | Soil temperature | DCFS 2022-03 to 2022-06 | Science review flags were not applied to the data |
| DP1.10020.001 | Ground beetle sequences DNA barcode | 2021 | Lab analyses pending for 2021 samples |
| DP1.10022.001 | Ground beetles sampled from pitfall traps | 2021 | Lab analyses pending for 2021 samples |
| DP1.10023.001 | Herbaceous clip harvest | GRSM 2021 | Data incomplete at time of release |
| DP1.10038.001 | Mosquito sequences DNA barcode | 2021-11 and 2021-12 | Lab analyses pending for samples from late 2021 |
| DP1.10041.001 | Mosquito-borne pathogen status | 2021-11 and 2021-12 | Lab analyses pending for samples from late 2021 |
| DP1.10043.001 | Mosquitoes sampled from CO2 traps | 2021-11 and 2021-12 | Lab analyses pending for samples from late 2021 |
| DP1.10081.001 | Soil microbe community composition | 2019, 2020, and 2021 | Data processing pending for 2019 samples; lab analyses pending for 2020 and 2021 samples |
| DP1.10104.001 | Soil microbe biomass | 2021-10 to 2021-12 | Lab analyses pending for samples from late 2021 |
| DP1.10107.001 | Soil microbe metagenome sequences | 2020 and 2021 | Lab analyses incomplete for 2020 and 2021 samples |
| DP1.10108.001 | Soil microbe marker gene sequences | 2019, 2020, and 2021 | ITS sequencing pending for 2019; all sequencing pending for 2020 and 2021 |
| DP1.20063.001 | Aquatic plant bryophyte chemical properties | 2021 | Lab analyses pending for some 2021 samples |
| DP1.20086.001 | Benthic microbe community composition | 2019, 2020, and 2021 | Data processing pending for 2019 samples; lab analyses pending for 2020 and 2021 samples |
| DP1.20092.001 | Chemical properties of groundwater | 2020 and 2021 | Lab analyses pending for some 2020 and 2021 samples |
| DP1.20093.001 | Chemical properties of surface water | 2020 and 2021 | Lab analyses pending for some 2020 and 2021 samples |
| DP1.20141.001 | Surface water microbe community composition | 2019, 2020, and 2021 | Data processing pending for 2019 samples; lab analyses pending for 2020 and 2021 samples |
| DP1.20163.001 | Periphyton, seston, and phytoplankton chemical properties | 2020 and 2021 | Lab analyses pending for some 2020 and 2021 samples |
| DP1.20166.001 | Periphyton, seston, and phytoplankton collection | 2020 and 2021 | Lab analyses pending for some 2020 and 2021 samples |
| DP1.20206.001 | Stable isotopes in surface water | 2021 | Lab analyses pending for some 2021 samples |
| DP1.20267.001 | Gauge height | 2021-10 to 2021-12 | Discharge products are released by water year; RELEASE-2023 includes WY2021 |
| DP1.20277.001 | Benthic microbe group abundances | 2020 and 2021 | Lab analyses incomplete for 2020 and 2021 samples |
| DP1.20278.001 | Surface water microbe group abundances | 2020 and 2021 | Lab analyses incomplete for 2020 and 2021 samples |
| DP1.20279.001 | Benthic microbe metagenome sequences | 2020 and 2021 | Lab analyses incomplete for 2020 and 2021 samples |
| DP1.20280.001 | Benthic microbe marker gene sequences | 2019, 2020, and 2021 | ITS sequencing pending for 2019; all sequencing pending for 2020 and 2021 |
| DP1.20281.001 | Surface water microbe metagenome sequences | 2020 and 2021 | Lab analyses incomplete for 2020 and 2021 samples |
| DP1.20282.001 | Surface water microbe marker gene sequences | 2019, 2020, and 2021 | ITS sequencing pending for 2019; all sequencing pending for 2020 and 2021 |
| DP1.20288.001 | Water quality | BARC 2022-05 to 2022-06 | Null data populating due to calibration issue, resolution in-process |
| DP1.20288.001 | Water quality | PRLA 2022-04 to 2022-06 | Reprocessing to address delayed data ingest in progress |
| DP*.300**.001 | All AOP Data Products | ORNL - All Years (2015-2018) | Allowing for community assessment and exploration prior to inclusion in the release |
| DP*.300**.001 | All AOP Data Products | STEI/TREE 2017-08 | Data publishing issue at time of release |
| DP1.30001.001 | LiDAR slant range waveform | STEI/TREE 2016-09 | Data publishing issue at time of release |
| DP3.30015.001 | Ecosystem structure | STEI/TREE 2019-06 | Data publishing issue at time of release |
| DP4.00130.001 | Continuous discharge | 2021-10 to 2022-06 | Discharge products are released by water year; RELEASE-2023 includes WY2021 |
| DP4.00130.001 | Continuous discharge | MCRA, OKSR, KING, LECO, and TECR WY 2021 (2020-10 to 2021-09) MCRA and OKSR WY 2020 (2019-10 to 2020-09) POSE 2018-10 to 2019-04 | Reprocessing to address delayed data ingest in progress |
| DP4.00133.001 | Stage-discharge rating curves | 2021-10 to 2021-12 | Discharge products are released by water year; RELEASE-2023 includes WY2021 |
| DP4.00200.001 | Bundled data products - Eddy covariance | BARR 2017-05 to 2017-08 BART 2016-12 to 2017-01 BONA 2017-10 CLBJ 2017-09 to 2017-10 CPER 2017-11 DELA 2017-02 DSNY 2017-02 JORN 2017-03 to 2017-08 LAJA 2017-08 to 2018-04 NOGP 2017-09 ORNL 2017-03 to 2017-06 SERC 2017-03 to 2017-05 TEAK 2018-06 UNDE 2017-02 WREF 2017-09 to 2018-02 YELL 2017-08 |
Site start dates have been consistently defined in this release. The excluded site-months are prior to a site passing initial operation review or prior to installation of the 3D sonic anemometer. |
Changes between RELEASE-2022 and RELEASE-2023
| Data Product ID | Data Product Name | Summary of changes between RELEASE-2022 and RELEASE-2023 |
|---|---|---|
| Generic OS Statement | All observational data products | For all OS data products, data quality improvement and error correction are continuous and ongoing processes. Routine data entry errors (such as transcription errors, duplicate records, or adjusted spatial coordinates) have been corrected where possible. Due to the large volume of data (>20 million data records), these corrections are not described individually. The table below describes significant changes to data products between releases. |
| Generic IS Statement | All instrumented systems products | Data were reprocessed to incorporate minor and/or isolated corrections to quality control thresholds, sensor installation periods, geolocation data, and manual quality flags. All sensor data products now include a separate table of manual quality flags applicable to the data in each download package. |
| DP1.00001.001 | 2D wind speed and direction | Corrections were made to the quality control algorithm that checks for flow distortion by tower infrastructure. Only the distorted flow quality metrics and final quality flag for wind direction were impacted. See data product issue log for affected sites and time periods. |
| DP1.00004.001 | Barometric pressure | Barometric pressure at sea level data were reprocessed following a change in the quality control algorithm for upstream dependency relative humidity. |
| DP1.00013.001 | Wet deposition chemical analysis | "chemSubsampleID" information was incorrectly routing to the "sampleID" field in the wdp_collectionChem table. To link wdp_collectionChem to other tables in the data product (wdp_collection and wdp_chemLab), sampleID related to chemSubsampleID. This is no longer the case with Release 2023 and provisional data. |
| DP1.00094.001 | Soil water content and water salinity | Data were reprocessed to remove the quality metric indicating use of the manufacturer's default calibration for soil moisture (calDefaultQM) from impacting the final quality flag (VSWCFinalQF). Previously, the final quality flag was raised when the manufacturer's default calibration was used rather than a soil-specific calibration (calNEONQM). The quality metrics indicating which calibration was used (calDefaultQM and calNEONQM) continue to be published in the expanded download package as informational metrics. |
| DP1.00095.001 | Soil CO2 concentration | Assessment of final data quality was improved by removing the independent temperature comparison test from computation of the beta quality metric. The independent temperature test often cannot be run due to failure of the independent temperature sensor rather than the soil CO2 sensor, and this caused a portion of data to be quality flagged when there was no problem with the soil CO2 sensor. Please see the issue log for the data product for additional details. |
| DP1.00096.001 | Soil physical and chemical properties, Megapit | Standard spatial information (e.g., slope, aspect, nlcdClass) that were previously absent have been added. NLCD class is available upon data product download, other variables can be accessed with the geoNEON R package (e.g., mgp <- getLocByName(mgp_permegapit, 'pitNamedLocation', locOnly=T)) |
| DP1.00098.001 | Relative humidity | Data were updated to retain readings that fail the range test rather than remove them prior to averaging. Data failing the range test remains quality flagged, but the values are not redacted since these data are still useful to provide an indication of saturated water vapor conditions. |
| DP1.10017.001 | Digital hemispheric photos of plot vegetation | aopCollectDate field added and backfilled for distributed plots where possible, to facilitate easier integration with remotely sensed AOP data. |
| DP1.10023.001 | Herbaceous clip harvest | To reduce the time associated with sorting bryophytes from the current-year herbaceous material, a 25% subsample is sorted at HEAL, BONA, DEJU, and a 50% subsample is sorted at BARR and TOOL. Mass ratios are then calculated to deliver end users calculated estimates of dried, current-year mass. |
| DP1.10026.001 | Plant foliar traits | aopCollectDate added for samples associated with field spectral measurements (DP1.30012.001). Previously this information could not be associated with these records, but the database issue that prevented this has been fixed. |
| DP1.10043.001 | Mosquitoes sampled from CO2 traps | A new column called proportionIdentified was added to indicate how much of a sample was identified. This replaces the need for four previous weight columns which will no longer be published including: subsampleWeight, bycatchWeight, totalWeight, and weightBelowDetection. |
| DP1.10058.001 | Plant presence and percent cover | The 'eventID' field format was changed to better facilitate grouping of plots collected at a site in a single bout. The new structure is siteCode.boutNumber.year, e.g., SJER.1.2022. Previously this field included the plot number which was not conducive to grouping for common analytical procedures. |
| DP1.10066.001 | Root biomass and chemistry, Megapit | Standard spatial information (e.g., slope, aspect, nlcdClass) that were previously absent have been added. NLCD class is available upon data product download, other variables can be accessed with the geoNEON R package (e.g., mgp <- getLocByName(mgp_permegapit, 'pitNamedLocation', locOnly=T)) |
| DP1.10098.001 | Vegetation structure | Non-herbaceous perennial vegetation structure data product (DP1.10045.001) has been merged with DP1.10098.001 for RELEASE-2023. A number of fields identifying specific subplots where growth forms were verified not present, have been removed from the vst_perplotperyear table. |
| DP1.20066.001 | Aquatic plant bryophyte macroalgae clip harvest |
In apl_clipHarvest, boutNumber and plantColonizableDepth were added. In apl_biomass, initialSubsample was added. In apl_biomass and apl_taxonomyRaw, taxonRank was added. Improbable records were updated where dryMass equaled wetMassSubsample. |
| DP1.20120.001 | Macroinvertebrate collection | In inv_fieldData at TOOK, littoral locations were updated. In inv_taxonomyProcessed and inv_taxonomyRaw, duplicates were removed and primaryKey was updated. |
| DP1.20126.001 | Macroinvertebrate metabarcoding | The inv_metabarcodeTaxonomyStandard table has now been published. In inv_metabarcodeTaxonomy, F230 records were correctly labeled. TOOK littoral locations were updated. |
| DP1.20166.001 | Periphyton, seston, and phytoplankton collection | In alg_taxonomyProcessed and alg_taxonomyRaw, algalAnalysisMethod was added, rounding for algalParameterValue was updated to prevent 0, and calculations were updated for cell density. In alg_fieldData, boutNumber and taxonID were added for epiphyton samples. TOOK littoral locations were updated. |
| DP1.20194.001 | Sediment chemical and physical properties | Surrogate compounds in the organic subsample (spike compounds used in quality assurance of GC and GC/MS methods) were removed from asc_externalLabData. |
| DP1.20219.001 | Zooplankton collection | In zoo_fieldData, TOOK littoral locations were updated. |
| DP1.20221.001 | Zooplankton metabarcoding | The inv_metabarcodeTaxonomyStandard table is now published. In zoo_metabarcodeTaxonomy, the F230 records are now correctly labeled. TOOK littoral locations were updated. |
| DP1.20252.001 | Secchi depth | In dep_secchi, sampling impractical records were added, remarks were separated from the depth profile, and fields equipCondition and samplerType were added. |
| DP1.20254.001 | Depth profile at specific depths | In dep_profileHeader sampling impractical records were added, remarks were separated from secchi, and the equipCondition field was added. |
| DP1.20279.001 | Benthic microbe metagenome sequences | sampleClasses was updated to allow for more accurate publication of records. markerGenes records were removed from the metagenomics table. |
| DP1.20280.001 | Benthic microbe marker gene sequences | amb_fieldParent samplingImpractical records were updated. |
| DP1.20281.001 | Surface water microbe metagenome sequences | sampleClasses was updated to allow for more accurate publication of records. markerGenes records were removed from the metagenomics table. |
| DP4.00130.001 | Continuous discharge | Sensor data that receives a Science review quality flag in the Elevation of surface water data product (DP1.20016.001) is now excluded from the development of a gauge height to water column height relationship, thus improving regression coefficients and reducing uncertainty in the continuous stage and discharge data. |
| DP4.00200.001 | Bundled data products - eddy covariance | The turbulent CO2 concentration (DP1.00034.001) and CO2 flux (DP4.00067.001) field calibration was improved by refining validation evaluation criteria and thresholds to maximize data quality and availability. Instrument flush time was increased for the Atmospheric CO2 Isotope data product (DP1.00036.001) to reduce residual memory from the previous measurement level by excluding an additional three minutes of data from the beginning of each temporal average. The quality flagging of the storage flux component of data products DP4.00002.001, DP4.00067.001, and DP4.00137.001 was updated to allow data from one tower level to be missing during storage flux calculations without raising the final quality flag. |
| DP1.20093.001 | Chemical properties of surface water | The belowDetectionQF has been added to the swc_externalLabDataByAnalyte table to indicate when values are not detected (i.e., below 0) or below the method detection limit (i.e., 0 or above, but below the MDL). All negative values have been removed from the analyteConcentration field and ND and BDL flags have been added to historic data, where possible. pH values for .RAW samples have also been removed due to questionable data quality related to transport and storage times. |
| DP1.20092.001 | Chemical properties of groundwater | The belowDetectionQF has been added to the gwc_externalLabDataByAnalyte table to indicate when values are not detected (i.e., below 0) or below the method detection limit (i.e., 0 or above, but below the MDL). All negative values have been removed from the analyteConcentration field and ND and BDL flags have been added to historic data, where possible. pH values for .RAW samples have also been removed due to questionable data quality related to transport and storage times. |
| DP1.10072.001 | Small mammal box trapping | An eventID field was created and filled in for all records in the mam_perplotnight table. The eventID can be used to group records by sampling bout. |
| DP1.20107.001 | Fish Sampling | Morpho species names were corrected to fit NEON's data naming protocol. |
| DP1.20107.001 | Fish Sampling | BLUE 2017 data that had been left out of the previous releases were added. |
| DP1.10108.001 | Soil microbe marker gene sequences | A new field, nucleicAcidRange, was added to DNA extraction MIC ingest, to indicate when DNA concentration measures were above or below the threshold of the instrument measuring. |
| DP1.20280.001 | Benthic microbe marker gene sequences | A new field, nucleicAcidRange, was added to DNA extraction MIC ingest, to indicate when DNA concentration measures were above or below the threshold of the instrument measuring. |
| DP1.20282.001 | Surface water microbe marker gene sequences | A new field, nucleicAcidRange, was added to DNA extraction MIC ingest, to indicate when DNA concentration measures were above or below the threshold of the instrument measuring. |
| DP1.30003.001 | Discrete return LiDAR point cloud | Filenames were corrected for a number of data and metadata files between 2018-2021. Missing metadata reports were added for a number of sites collected in 2018. |
| DP3.30015.001 | Ecosystem structure | Data were updated for the 2020 CPER collection to fix an incorrect UTM zone. Missing metadata reports were added for a number of sites collected in 2018. |
| DP3.30024.001 | Elevation - LiDAR | Missing metadata reports were added for a number of sites collected in 2018. |
| DP3.30025.001 | Slope and aspect - LiDAR | Missing metadata reports were added for a number of sites collected in 2018. |
| DP1.30006.001 | Spectrometer orthorectified surface directional reflectance - flightline | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP2.30019.001 | Canopy water indices - flightline | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP2.30026.001 | Vegetation indices - spectrometer - flightline | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP2.30011.001 | Albedo - spectrometer - flightline | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP2.30012.001 | LAI - spectrometer - flightline | NO DATA values were fixed to remove small errors caused by computing precision. Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP2.30014.001 | fPAR - Spectrometer - flightline | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Filenames were updated for sites collected between 2018-2021 to the accepted naming convention. |
| DP3.30006.001 | Spectrometer orthorectified surface directional reflectance - mosaic | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP3.30011.001 | Albedo - spectrometer - mosaic | Data were updated for the 2017 DEJU collection to correct a geolocation issue. A mosaicked data file was deleted for sites collected between 2018-2021. Metadata filenames were updated for a number of sites. |
| DP3.30012.001 | LAI - spectrometer - mosaic | NO DATA values were fixed to remove small errors caused by computing precision. Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP3.30014.001 | fPAR - spectrometer - mosaic | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP3.30019.001 | Canopy water indices - mosaic | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |
| DP3.30026.001 | Vegetation indices - spectrometer - mosaic | Data were updated for the 2017 DEJU collection to correct a geolocation issue. Metadata filenames were updated for a number of sites. |