Incorrect formatting in legacy metagenomic sequencing files

It was discovered recently that many older NEON metagenomic samples that were sequenced in 2018 – 2019 were not formatted correctly. This issue affects multiple samples collected from 2014-2018. The fastq files from these sequencing runs are not in error, but the reads in the paired end files are in a different order to each other. The variable formatting prevents automated bioinformatics workflows from interleaving paired reads. Here is the link to a table that lists all possible samples affected. We used a conservative approach to evaluate the legacy files, such that, if a sequencing run showed at least some samples out of order, we have assumed all samples in that run need to be reformatted.

NEON’s metagenomic data sharing partner, the National Microbiome Data Collaborative, have retracted those affected sequencing records and the downstream analyses and are actively processing these new files and making records for the corrected data. We will provide an updated notification when these updated records become available. The NEON metagenomic data shared through the National Center for Biotechnology Information (NCBI) includes some of the affected files, so the effort to remove those files is also in progress.

These files will be repaired and made available from the NEON website in the 2025 data release; by design, the improperly formatted files will remain available through prior data releases. In the interim, we have created a GitHub repository (https://github.com/NEONScience/legacy_metagenome) that details how a user can fix the issue. We can also provide access to the repaired files on request. During the impacted time period, the laboratory sequenced all samples in a sequencing run across all four lanes of the Illumina NextSeq machine, which resulted in four files for each sample for each strand (R1/R2). These files were then concatenated into a single file for each strand for release. The affected files were concatenated in variable orders, so the R1 was often concatenated in a different order than the R2. Therefore, the sequence files are not in order in and of themselves, only out of sync (order) with its corresponding paired read. The repair will sort both files the same way for all affected samples.

Please use the Contact Us form for further inquiries and include the term “INC0075559” in the body of your request.