Using an API Token when Accessing NEON Data with neonUtilities
Last Updated: Nov 23, 2020
NEON data can be downloaded from either the NEON Data Portal or the NEON API. When downloading from the Data Portal, you can create a user account. Read about the benefits of an account on the User Account page. You can also use your account to create a token for using the API. Your token is unique to your account, so don't share it.
Using a token is optional! You can download data without a token, and without a user account. Using a token when downloading data via the API, including when using the neonUtilities package, links your downloads to your user account, as well as enabling faster download speeds. For more information about token usage and benefits, see the NEON API documentation page.
For now, in addition to faster downloads, using a token helps NEON to track data downloads. Using anonymized user information, we can then calculate data access statistics, such as which data products are downloaded most frequently, which data products are downloaded in groups by the same users, and how many users in total are downloading data. This information helps NEON to evaluate the growth and reach of the observatory, and to advocate for training activities, workshops, and software development.
Tokens can be used whenever you use the NEON API. In this tutorial, we'll focus on using tokens with the neonUtilities R package.
After completing this activity, you will be able to:
- Create a NEON API token
- Use your token when downloading data with neonUtilities
Things You’ll Need To Complete This Tutorial
You will need a version of R (3.4.1 or higher) and, preferably,
loaded on your computer to complete this tutorial.
Install R Packages
If you've never downloaded NEON data using the neonUtilities package before, we recommend starting with the Download and Explore tutorial before proceeding with this tutorial.
In the next sections, we'll get an API token from the NEON Data Portal, and then use it in neonUtilities when downloading data.
Get a NEON API Token
The first step is create a NEON user account, if you don't have one. Follow the instructions on the Data Portal User Accounts page. If you do already have an account, go to the NEON Data Portal, sign in, and go to your My Account profile page.
Once you have an account, you can create an API token for yourself. At the bottom of the My Account page, you should see this bar:
Click the 'GET API TOKEN' button. After a moment, you should see this:
Click on the Copy button to copy your API token to the clipboard:
Use API token in neonUtilities
In the next section, we'll walk through saving your token somewhere secure but accessible to your code. But first let's try out using the token the easy way.
First, we need to load the
neonUtilities package and set the working
# install neonUtilities - can skip if already installed, but # API tokens are only enabled in neonUtilities v1.3.4 and higher # if your version number is lower, re-install install.packages("neonUtilities") # load neonUtilities library(neonUtilities) # set working directory wd <- "~/data" # this will depend on your local machine setwd(wd)
NEON API tokens are very long, so it would be annoying to keep pasting the entire text string into functions. Assign your token an object name:
NEON_TOKEN <- "PASTE YOUR TOKEN HERE"
Now we'll use the
loadByProduct() function to download data. Your
API token is entered as the optional
token input parameter. For
this example, we'll download Plant foliar traits (DP1.10026.001).
foliar <- loadByProduct(dpID="DP1.10026.001", site="all", package="expanded", check.size=F, token=NEON_TOKEN)
You should now have data saved in the
foliar object; the API
silently used your token. If you've downloaded data without a
token before, you may notice this is faster!
This format applies to all
neonUtilities functions that involve
downloading data or otherwise accessing the API; you can use the
token input with all of them. For example, when downloading
remote sensing data:
chm <- byTileAOP(dpID="DP3.30015.001", site="WREF", year=2017, check.size=F, easting=c(571000,578000), northing=c(5079000,5080000), savepath=wd, token=NEON_TOKEN)
Token management for open code
Your API token is unique to your account, so don't share it!
If you're writing code that will be shared with colleagues or available
publicly, such as in a GitHub repository or supplemental materials of a
published paper, you can't include the line of code above where we assigned
your token to
NEON_TOKEN, since your token is fully visible in the code
there. Instead, you'll need to save your token locally on your computer,
and pull it into your code without displaying it. There are a few ways to
do this, we'll show two options here.
Option 1: Save the token in a local file, and
source()that file at the start of every script. This is fairly simple but requires a line of code in every script.
Option 2: Add the token to a
.Renvironfile to create an environment variable that gets loaded when you open R. This is a little harder to set up initially, but once it's done, it's done globally, and it will work in every script you run.
Option 1: Save token in a local file
Open a new, empty R script (.R). Put a single line of code in the script:
NEON_TOKEN <- "PASTE YOUR TOKEN HERE"
Save this file in a logical place on your machine, somewhere that won't be
visible publicly. Here, let's call the file
save it to the working directory. Then, at the start of
every script where you're going to use the NEON API, you would run this line
Then you'll be able to use
token=NEON_TOKEN when you run
functions, and you can share your code without accidentally sharing your
Option 2: Save token to the R environment
To create a persistent environment variable, we use a
Before creating a file, check which directory R is using as your home
# For Windows: Sys.getenv("R_USER") # For Mac/Linux: Sys.getenv("HOME")
Check the home directory to see if you already have a
.Renviron file, using
the file browse pane in RStudio, or using another file browse method with
hidden files shown. Files that begin with
. are hidden by default, but
RStudio recognizes files that begin with
.R and displays them.
If you already have a
.Renviron file, open it and follow the instructions
below to add to it. If you don't have one, create one using File -> New File
-> Text File in the RStudio menus.
Add one line to the text file. In this option, there are no quotes around the token value.
NEON_TOKEN=PASTE YOUR TOKEN HERE
Save the file as
.Renviron, in the RStudio home directory identified above.
Double check the spelling, this will not work if you have a typo. Re-start
R to load the environment.
Once your token is assigned to an environment variable, use the function
Sys.getenv() to access it. For example, in
foliar <- loadByProduct(dpID="DP1.10026.001", site="all", package="expanded", check.size=F, token=Sys.getenv("NEON_TOKEN"))
Get Lesson Code
If you have questions or comments on this content, please contact us.Contact Us