At the NEON Program, Data Quality Starts with Quality Training

Every year, the NEON program hires between 230-250 seasonal field technicians. These seasonal workers, often recent college graduates, are tasked with collecting field observations and physical samples at NEON field sites. How do we make sure they are conducting observations in exactly the same way? It starts with training led by experienced NEON field staff. A new internal Online Training Center makes it easier to standardize training across the NEON Domains and track training progress for new recruits.

Ensuring Data Quality Across the Observatory

The NEON program depends on the collection of high-quality, standardized data on both spatial and temporal scales. To ensure that data are comparable across sites dispersed from Puerto Rico to Alaska to Hawaii, and across the 30-year life of the Observatory, observations must be collected in the same way at every site, every year. If we collect ticks one way in Alabama and another way in California, we introduce variability that reduces the capacity for researchers to compare and draw conclusions from the data.

That's why the NEON program has created detailed written protocols for all field sampling and observations. Samples and field observations for the terrestrial and aquatic observational systems are conducted according to the same protocols at all NEON field sites, which are publicly available.

To ensure all staff understand and follow the protocols, seasonal technicians undergo rigorous in-person and online training before they begin collecting samples. They are trained by experienced full-time NEON staff members, who are themselves trained through a train-the-trainer model to ensure training quality and consistency.

"Each field site poses different challenges in how to implement the sampling protocol," explains Liz Knapp, Senior Curriculum Developer for the NEON program, "so it is important that everyone understands what it is we are trying to accomplish, what the methods are for, and why we do it in the way we do. We are trying to reduce variability based on who is doing the sampling. The only way to achieve that is through standardized protocols and standardized training delivered by experienced trainers."

From the Classroom to the Field: How NEON Seasonal Technicians Are Trained

Trainees start with an introduction to each of the protocols they will implement, followed by reading the publicly available protocol document (such as for mosquitoes) and hands-on practice with a trainer. Typically, a seasonal field technician will be trained in four to eight different protocols, depending on their area of specialty and Domain. Technicians usually specialize in one of three areas: terrestrial flora, terrestrial fauna, or aquatic. Someone specializing in terrestrial flora or fauna may also be trained in soil protocols.

Full-time field staff guide technicians through their training, which is overseen by Domain managers. Trainees learn both the basic protocols and any specific information about implementing the protocol in their Domain. They also receive extensive safety training. The entire training process – online and hands-on – for each protocol may take six to ten hours over several days. Dr. Rhea Esposito, the second member of the NEON Curriculum Development team, says, "We always start with the basics—how to navigate a plot, how to use a compass, what safety protocols you need to follow. We don't assume anything."

Each protocol course is delivered close to the time that it will be carried out in the field, so a trainee may complete a plant phenology course at the very beginning of the training season and complete a plant diversity course closer to "peak greenness" for their Domain. Timing is Domain-specific; the earliest courses start in mid-March.

It's also important the trainees undergo field training with full-time NEON field staff before they are released to implement the protocols on their own. Knapp says, "We focus on a train-the-trainer model that is decentralized, so you are really getting training in how to implement the protocol in the ecosystem in which you are working. Having hands-on training at the sites where you will implement the protocols makes it more effective."

Training for the Plant Diversity protocol at PUUM, D20 (2019).

Melissa Benedict, a full-time Domain staff in Domain 03, says, "As a trainer, I'm given all the materials I need to train these people that will be working alongside me, and it really makes the process run smoothly and efficiently. The training plans clearly outline every item that technicians need to know to do each protocol as well as the most common challenges."

Going Digital: The NEON Online Training Center

In 2021, the NEON program introduced a new internal Online Training Center to make it easier to track and standardize training across the Domains. Online courseware was designed in accordance with best practices in adult learning with plenty of interactivity and hands-on application. Online training is paired with Domain-specific field and lab training for a comprehensive learning experience.

The online courses were created by Knapp and Esposito with assistance from field staff and a final technical review by protocol authors. The courses are designed with the flexibility to be instructor-led or self-guided. Each learner has a personal dashboard where they can see the courses they are enrolled in. Learners must complete both online quizzes and performance-based assessments overseen by the Domain-based trainers in order to be certified as ready to implement any particular protocol. Trainers receive yearly in-person training so they know how to lead training at their Domains and what specifically to look for in completing performance-based assessments.

"The Online Training Center provides a degree of accountability that we didn't have before," says Knapp. "We can ensure that the most recent and updated training materials are being used across the Observatory, so we have consistent quality in the training in all Domains. We can also use analytics from the quizzes and performance-based assessments to see where people are struggling and improve our training materials to address common misunderstandings. It really is a quality control mechanism for us."

Image

Protocol training pipeline

The Online Training Center also helps with scheduling in the field, by providing a centralized database of who has completed each course. For example, if a field technician is going to be unavailable on a day they are scheduled to complete a small mammal protocol, it is easy to see who else in the Domain has already completed training for that protocol and can fill in.

The online courses have been especially valuable during continued COVID-19-related travel and meeting restrictions for the 2021 training season. Trainees are able to complete much of the introductory training remotely, limiting the need to gather face-to-face for classroom-based training. The Online Training Center also facilitates cross-Domain training.

An Ongoing Commitment to Data Quality and Scientific Rigor

The NEON seasonal training program not only ensures standardized data collection across the Observatory but also prepares field technicians to implement rigorous field ecology programs wherever their careers take them. For most first-time NEON technicians, this is the first experience they have had in conducting field science to these exacting specifications.

Knapp says, "We don't just want our temporary staff to be able to do the work we need done. We want them to understand why and how the data they are collecting will be used by the science community. When they work with us, they are part of a larger network, and they have to think about how the decisions they make in the field will impact the community as a whole. It is very different from other field experiences most of them have had in the past."

Jack Lindauer, now a full-time Lead Field Technician for the NEON program, says, "I am immensely grateful for the thoroughness of the training that we go through as new technicians. I feel confident I am trained to excel in my role thanks to the instruction from peers across the observatory. Additionally, I appreciate the high expectations of our work and how it demands that I further develop the skills and knowledge I gained while in college."

Dryland practice – training for electrofishing at D09. Photo credit: Tammy Otto

Full-time staff receive ongoing training, too. They come together annually in "training-paloozas" for each observational system. These annual cross-Domain sessions provide an opportunity for trainers to learn from the curriculum developers, protocol authors, and each other towards the goal of continuously improving the NEON training program. Outside of the paloozas, trainers and Domain managers stay in touch throughout the year to share challenges, best practices, and solutions across the field offices.

Esposito says, "It really is all about data quality. If one of the main goals of the NEON program is to create standardized, comparable data across the continent, that means the training has to be high-quality and consistent enough to support that. Our training programs for full-time and temporary field staff and the new Online Training Center are tools that help us reach those goals."