In this tutorial you will learn how to open a .tiff file in Jupyter Notebook and learn about kernels.

The goal of the activity is simply to ensure that you have basic familiarity with Jupyter Notebooks and that the environment, especially the gdal package is correctly set up before you pursue more programming tutorials. If you already are familiar with Jupyter Notebooks using Python, you may be able to complete the assignment without working through the instructions.

Assignment: Open a Tiff File in Jupyter Notebook

Set up Environment

First, we will set up the environment as you would need for each of the live coding sections of the Data Institute. The following directions are copied over from the Data Institute Set up Materials.

In your terminal application, navigate to the directory (cd) that where you want the Jupyter Notebooks to be saved (or where they already exist).

We need to create a new Jupyter kernel for the Python 3.8 conda environment (py38) that Jupyter Notebooks will use.

In your Command Prompt/Terminal, type:

python -m ipykernel install --user --name py34 --display-name "Python 3.8 NEON-RSDI"

In your Command Prompt/Terminal, navigate to the directory (cd) that you created last week in the GitHub materials. This is where the Jupyter Notebook will be saved and the easiest way to access existing notebooks.

###Open Jupyter Notebook Open Jupyter Notebook by typing into a command terminal:

jupyter notebook

Once the notebook is open, check which version of Python you are in.

 # Check what version of Python.  Should be 3.8. 
 import sys
 sys.version

To ensure that the correct kernel will operate, navigate to Kernel in the menu, select Kernel/Restart Kernel And Clear All Outputs.

You should now be able to work in the notebook.

#Download the digital terrain model (GEOTIFF file) Download the NEON GeoTiFF file of a digital terrain model (dtm) of the San Joaquin Experimental Range. Click this link to download dtm data: https://ndownloader.figshare.com/articles/2009586/versions/10. This will download a zippped full of data originally from a NEON data carpentry tutorial (https://datacarpentry.org/geospatial-workshop/data/).

Once downloaded, navigate through the folder to C:NEON-DS-Airborne-Remote-Sensing.zip\NEON-DS-Airborne-Remote-Sensing\SJER\DTM and save this file onto your own personal working directory. .

###Open GEOTIFF file in Jupyter Notebooks using gdal

The gdal package that occasionally has problems with some versions of Python. Therefore test out loading it using:

import gdal.

If you have trouble, ensure that 'gdal' is installed on your current environment.

Establish your directory

Place the downloaded dtm file in a repository of your choice (or your current working directory). Navigate to that directory. wd= '/your-file-path-here' #Input the directory to where you saved the .tif file

Import the TIFF

Import the NEON GeoTiFF file of the digital terrain model (DTM) from San Joaquin Experimental Range. Open the file using the gdal.Open command.Determine the size of the raster and (optional) plot the raster.

#Use GDAL to open GEOTIFF file stored in your directory SJER_DTM = gdal.Open(wd + 'SJER_dtmCrop.tif')>

#Determine the raster size.

  SJER_DTM.RasterXSize

Add in both code chunks and text (markdown) chunks to fully explain what is done. If you would like to also plot the file, feel free to do so.

Push .ipynb to GitHub.

When finished, save as a .ipynb file.

Setting up Jupyter Notebooks

You can set up your notebook in several ways. Here we present the Anaconda Python distribution method so as to follow the Data Institute set up instructions.

Browser

First, make sure you have an updated browser on which to run the app. Both Mozilla Firefox and Google Chrome work well.

Installation

Data Institute participants should have already installed Jupyter Notebooks through the Anaconda installation during the Data Institute set up instructions.

If you install Python using pip you can install the Jupyter package with the following code.

 
# Python2
pip install jupyter
# Python 3
pip3 install jupyter

Set up Environment

We need to set up the Python environment that we will be working in for the Notebook. This allows us to have different Python environments for different projects. The following directions pertain directly to the set up for the 2018 Data Institute on Remote Sensing with Reproducible Workflows, however, you can adapt them to the specific Python version and packages you wish to work with.

If you haven't yet created a Python 3.8 environment (released October 2019), you'll need to do that now. You can use the single line provided below, or refer back to the Python section of the installation instructions, for more details. To create this Python 3.8 environment, you must first install Anaconda Navigator onto your computer, then open the Anaconda Prompt application (or your terminal) and type the following into the prompt window:

conda create -n p38 python=3.8 anaconda

And activate the Python 3.8 environment:

On Mac:

source activate p38

On Windows:

activate p38

In the terminal application, navigate to the directory (cd) where you want the Jupyter Notebooks to be saved (or where they already exist).

Once here, we want to create a new Jupyter kernel for the Python 3.8 conda environment (p38) that we'll be using with Jupyter Notebooks.

With the p38 environment activated, in your Command Prompt/Terminal, type:

python -m ipykernel install --user --name p38 --display-name "Python 3.8 NEON-RSDI"

This command tells Python to create a new ipy (aka Jupyter Notebook) kernel using the Python environment we set up and called "p38". Then we tell it to use the display name for this new kernel as "Python 3.8 NEON-RSDI". You will use this name to identify the specific kernel you want to work with in the Notebook space, so name it descriptively, especially if you think you'll be using several different kernels.

Using Jupyter Notebooks

Launching the Application

To launch the application either launch it from the Anaconda Navigator or by typing jupyter notebook into your terminal or command window.

 
# Launch Jupyter
jupyter notebook

More information can be found in the Read the Docs Running the Jupyter Notebook.

Navigating the Jupyter Python Interface

The following information is adapted from Griffin Chure's Tutorial 0b: Using Jupyter Notebooks

If everything launched correctly, you should be able to see a screen which looks something like this. Note that the home directory will be whatever directory you have navigated to in your terminal before launching Jupyter Notebooks.

To start a new Python notebook, click on the right-hand side of the application window and select New (the expanded menu is shown in the screen shot above). This will give you several options for new notebook kernels depending on what is installed on your computer. In the above screenshot, there are two available Python kernels and one Matlab kernel. When starting a notebook, you should choose Python 3 if it is available or conda(root) .

Once you start a new notebook, you will be brought to the following screen.

Welcome to your first look at a Jupyter notebook!

There are many available buttons for you to click. The three most important components of the notebook are highlighted in colored boxes.

• In blue is the name of the notebook. By clicking this, you can rename the notebook.
• In red is the cell formatting assignment. By default, it is registered as code, but it can also be set to markdown as described later.
• In purple, is the code cell. In this cell, you can type an execute Python code as well as text that will be formatted in a nicely readable format.

Selecting a Kernel

A kernel is a server that enables you to run commands within Jupyter Notebook. It is visible via a prompt window that logs all your actions in the notebook, making it helpful to refer to when encountering errors. You'll be prompted to select a kernel when you open a new notebook, however, if you are opening an existing notebook you will want to ensure that you are using the correct kernel. The commands for selecting and changing kernels are in the Kernel menu.

When you select or switch a kernel, you may want to use the navigate to Kernel in the menu, select Restart/ClearOutlook. The Restart/ClearOutlook option ensures that the correct kernel will operate.

You can always check what version of Python you are running by typing the following into a code cell.

 # Check what version of Python.  Should be 3.5. 
 import sys
 sys.version

Writing & running code

The following information is adapted from Griffin Chure's Tutorial 0b: Using Jupyter Notebooks

All code you write in the notebook will be in the code cell. You can write single lines, to entire loops, to complete functions. As an example, we can write and evaluate a print statement in a code cell, as is shown below.

If you would like to write several lines of code, hit Enter to continue entering code into another line. To execute the code, we can simply hit Shift + Enter while our cursor is in the code cell.

 # This is a comment and is not read by Python
 print('Hello! This is the print function. Python will print this line below')

 Hello! This is the print function. Python will print this line below

We can also write a 'for' loop as an example of executing multiple lines of code at once.

 # Write a basic for loop. In this case a range of numbers 0-4.
 for i in range(5):
# Multiply the value of i by two and assign it to a variable. 
temp_variable = 2 * i

# Print the value of the temp variable.
print(temp_variable)

 0
 2
 4
 6
 8

There are two other useful keyboard shortcuts for running code:

• Alt + Enter runs the current cell and inserts a new one below
• Ctrl + Enter run the current cell and enters command mode.

For more keyboard shortcuts, check out weidadeyue's Shortcut cheatsheet.

If you would like more details on running code in Jupyter Notebooks, please go through the following short tutorial by Running Code by contributors to the Jupyter project. This tutorial touches on start and stopping the kernel and using multiple kernels (e.g., Python and R) in one notebook.

Writing Text

The following information is adapted from Griffin Chure's Tutorial 0b: Using Jupyter Notebooks

Arguably the most useful component of the Jupyter Notebook is the ability to interweave code and explanatory text into a single, coherent document. Through out the Data Institute (and one's everyday workflow), we encourage all code and plots should be accompanied with explanatory text.

Each cell in a notebook can exist either as a code cell or as a text-formatting cell called a markdown cell. Markdown is a mark-up language that very easily converts to other type-setting formats such as HTML and PDF.

Whenever you make a new cell, its default assignment will be a code cell. This means when you want to write text, you will need to specifically change it to a markdown cell. You can do this by clicking on the drop-down menu that reads code' (highlighted in red in the second figure of this page) and selecting 'Markdown'. You can then type in the code cell and all Python syntax highlighting will be removed.

Resources for Learning Markdown

• Review the NEON tutorial Git 04: Markdown Files
• Adam Pritchard's Markdown Cheatsheet

Saving & Quitting

The following information is adapted from Griffin Chure's Tutorial 0b: Using Jupyter Notebooks

Jupyter notebooks are set up to autosave your work every 15 or so minutes. However, you should not rely on the autosave feature! Save your work frequently by clicking on the floppy disk icon located in the upper left-hand corner of the toolbar.

To navigate back to the root of your Jupyter notebook server, you can click on the Jupyter logo at any time.

To quit your Jupyter notebook, you can simply close the browser window and the Jupyter notebook server running in your terminal.

Converting to HTML and PDF

In addition to sharing notebooks in the.ipynb format, it may useful to convert these notebooks to highly-portable formats such as HTML and PDF.

To convert, you can either use the dropdown menu option

File -> download as -> ...

or via the command line by using the following lines:

 jupyter nbconvert --to pdf notebook_name.ipynb 

Where "notebook_name.ipynb" matches the name of the notebook you want to convert. Prior to converting the notebook, you must be in the same working directory as your notebook or use the correct file path from your current working directory.

Converting to PDF requires both Pandoc and LaTeX to be installed. You can find out more in the ReadTheDoc for nbconvert.

If you prefer to convert to a different format, like HTML, you simply change the file type. jupyter nbconvert --to html notebook_name.ipynb Read more on what formats you can convert to and more about the nbconvert package .

Additional Resources

Using Jupyter Notebooks

• Jupyter Documentation on ReadTheDocs
• Griffin Chure's multi-part course on Using Jupyter Notebooks for Scientific Computing . Much of the material above is adapted from Tutorial 0b: Using Jupyter Notebooks .
• Jupyter Project's Running Code

Using Python

• Software Carpentry's Programming with Python workshop
• Data Carpentry's Python for Ecologists workshop
• Many, many others that a simple web search will bring up...

In this tutorial we learn how to effectively and efficiently document and publish our workflows online.

Documentation Is Important

As we read in the Reproducible Science overview, the four facets of reproducible science are:

• Documentation
• Organization
• Automation and
• Dissemination.

This week we will learn about the Jupyter Notebook as a tool to document and publish (disseminate) your code and code output.

View Slideshow: Share, Publish & Archive - from the Reproducible Science Curriculum

Jupyter Notebook

“The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more." -- Jupyter Notebook documentation.

We use markdown syntax in Notebook documents to document workflows and to share data processing, analysis and visualization outputs. We can also use it to create documents that combine code in your language of choice, output and text.

The Jupyter Notebooks grew out of iPython. Jupyter is a close acronym meaning Julia, Python, and R, which were the first languages outside Python that the Jupyter application was designed for. Jupyter Notebooks now supports over 40 coding languages. You may still find some references to iPython in materials related to Jupyter Notebooks. This series will focus on using Jupyter Notebooks with Python, but the information presented can apply to other languages as well.

The Jupyter Notebooks application is a browser-based application. Therefore, you need an updated browser (the Jupyter programmers recommend Mozilla Firefox or Google Chrome, but not Microsoft Explorer). When installed on your computer, you can always access the app even without internet access. You can also use Jupyter installed on a remote server. For example, Jupyter runs a training (temporary) server based version.

Why Jupyter Notebooks?

There are many advantages to using Jupyter Notebooks in your work:

• Human readable syntax.
• Simple syntax - it can be learned quickly.
• All components of your work are clearly documented. You don't have to remember what steps, assumptions, tests were used.
• You can easily extend or refine analyses by modifying existing or adding new code blocks.
• Analysis results can be disseminated in various formats including HTML, PDF, slideshows and more.
• Code and data can be shared with a colleague to replicate the workflow.

Explore Examples of Notebooks

Before we jump into how to work with notebooks, check out a few shared notebooks. As you look at these different notebooks, what aspects of the layout do you like, what don't you like? Is there a place in your current workflow that these notebooks would be useful?

• Jupyter's GitHub Wiki: A gallery of interesting Jupyter Notebooks. Not only is this a great collection of example notebooks, but also it is a valuable resource to learn other skills associated with using Python and Jupyter Notebooks.
• Fabian Pedregosa's Notebook Gallery

In the next tutorial, Introduction to using Jupyter Notebooks, we will learn more about working with Jupyter Notebooks.

Remote Sensing Indices

There are many different indices you might want in your research. NEON provides several indices as data products that have already been calculated and can will be available for download from the NEON data portal.

NEON Remote Sensing Vegetation Indices, Data Products, and Uncertainty

In this 20 minute video David Hulslander describes NEON Data Products including several remote sensing vegetation indices.

Activity Steps

1. Choose an index of interest. You may want to check out Verena Henrich & Katharina Brüser's Index Database for ideas: www.indexdatabase.de/ .

2. Work with your small group to create a script to calculate this index from the NEON data. Be sure to add comments so that the script is useful to others.

3. Add your script to the GitHub Repo: DI-NEON-participants to share with your colleagues. Save scripts to the DI-NEON-participants/2018-RemoteSensing/rs-indices.
   Be sure to provide a clear file name reflecting the contents. If you are comfortable, we recommend you put you names in the script as others may want to contact you about it.

This page outlines the tools and resources that you will need to install Git, Bash and Python applications onto your computer as the first step of our Python skills tutorial series.

Checklist

Detailed directions to accomplish each objective are below.

• Install Bash shell (or shell of preference)
• Install Git
• Install Python 3.x

Bash/Shell Setup

Install Bash for Windows

1. Download the Git for Windows installer.
2. Run the installer and follow the steps bellow:
   1. Welcome to the Git Setup Wizard: Click on "Next".
   2. Information: Click on "Next".
   3. Select Destination Location: Click on "Next".
   4. Select Components: Click on "Next".
   5. Select Start Menu Folder: Click on "Next".
   6. Adjusting your PATH environment: Select "Use Git from the Windows Command Prompt" and click on "Next". If you forgot to do this programs that you need for the event will not work properly. If this happens rerun the installer and select the appropriate option.
   7. Configuring the line ending conversions: Click on "Next". Keep "Checkout Windows-style, commit Unix-style line endings" selected.
   8. Configuring the terminal emulator to use with Git Bash: Select "Use Windows' default console window" and click on "Next".
   9. Configuring experimental performance tweaks: Click on "Next".
   10. Completing the Git Setup Wizard: Click on "Finish".

This will provide you with both Git and Bash in the Git Bash program.

Install Bash for Mac OS X

The default shell in all versions of Mac OS X is bash, so no need to install anything. You access bash from the Terminal (found in /Applications/Utilities). You may want to keep Terminal in your dock for this workshop.

Install Bash for Linux

The default shell is usually Bash, but if your machine is set up differently you can run it by opening a terminal and typing bash. There is no need to install anything.

Git Setup

Git is a version control system that lets you track who made changes to what when and has options for easily updating a shared or public version of your code on GitHub. You will need a supported web browser (current versions of Chrome, Firefox or Safari, or Internet Explorer version 9 or above).

Git installation instructions borrowed and modified from Software Carpentry.

Git for Windows

Git should be installed on your computer as part of your Bash install.

Git on Mac OS X

Video Tutorial

Install Git on Macs by downloading and running the most recent installer for "mavericks" if you are using OS X 10.9 and higher -or- if using an earlier OS X, choose the most recent "snow leopard" installer, from this list. After installing Git, there will not be anything in your /Applications folder, as Git is a command line program.

Git on Linux

If Git is not already available on your machine you can try to install it via your distro's package manager. For Debian/Ubuntu run sudo apt-get install git and for Fedora run sudo yum install git.

Setting Up Python

Python is a popular language for scientific computing and data science, as well as being a great for general-purpose programming. Installing all of the scientific packages individually can be a bit difficult, so we recommend using an all-in-one installer, like Anaconda.

Regardless of how you choose to install it, **please make sure your environment is set up with Python version 3.7 (at the time of writing, the gdal package did not work with the newest Python version 3.6). Python 2.x is quite different from Python 3.x so you do need to install 3.x and set up with the 3.7 environment.

We will teach using Python in the Jupyter Notebook environment, a programming environment that runs in a web browser. For this to work you will need a reasonably up-to-date browser. The current versions of the Chrome, Safari and Firefox browsers are all supported (some older browsers, including Internet Explorer version 9 and below, are not). You can choose to not use notebooks in the course, however, we do recommend you download and install the library so that you can explore this tool.

Windows

Download and install Anaconda. Download the default Python 3 installer (3.7). Use all of the defaults for installation except make sure to check Make Anaconda the default Python.

Mac OS X

Download and install Anaconda. Download the Python 3.x installer, choosing either the graphical installer or the command-line installer (3.7). For the graphical installer, use all of the defaults for installation. For the command-line installer open Terminal, navigate to the directory with the download then enter:

bash Anaconda3-2020.11-MacOSX-x86_64.sh (or whatever you file name is)

Linux

Download and install Anaconda. Download the installer that matches your operating system and save it in your home folder. Download the default Python 3 installer.

Open a terminal window and navigate to your downloads folder. Type

bash Anaconda3-2020.11-Linux-ppc64le.sh

and then press tab. The name of the file you just downloaded should appear.

Press enter. You will follow the text-only prompts. When there is a colon at the bottom of the screen press the down arrow to move down through the text. Type yes and press enter to approve the license. Press enter to approve the default location for the files. Type yes and press enter to prepend Anaconda to your PATH (this makes the Anaconda distribution the default Python).

Install Python packages

We need to install several packages to the Python environment to be able to work with the remote sensing data

• gdal
• h5py

If you are new to working with command line you may wish to complete the next setup instructions which provides and intro to command line (bash) prior to completing these package installation instructions.

Windows

Create a new Python 3.7 environment by opening Windows Command Prompt and typing

conda create –n py37 python=3.7 anaconda

When prompted, activate the py37 environment in Command Prompt by typing

activate py37

You should see (py37) at the beginning of the command line. You can also test that you are using the correct version by typing python --version.

Install Python package(s):

• gdal: conda install gdal
• h5py: conda install h5py

Note: You may need to only install gdal as the others may be included in the default.

Mac OS X

Create a new Python 3.7 environment by opening Terminal and typing

conda create –n py37 python=3.7 anaconda

This may take a minute or two.

When prompted, activate the py37 environment in Command Prompt by typing

source activate py37

You should see (py37) at the beginning of the command line. You can also test that you are using the correct version by typing python --version.

Install Python package(s):

• gdal: conda install gdal
• h5py: conda install h5py

Linux

Open default terminal application (on Ubuntu that will be gnome-terminal).

Launch Python.

Install Python package(s):

• gdal: conda install gdal
• h5py: conda install h5py

Set up Jupyter Notebook Environment

In your terminal application, navigate to the directory (cd) that where you want the Jupyter Notebooks to be saved (or where they already exist).

Open Jupyter Notebook with

jupyter notebook

Once the notebook is open, check which version of Python you are in by using the prompts

# check what version of Python you are using.
import sys
sys.version

You should now be able to work in the notebook.

The gdal package that occasionally has problems with some versions of Python. Therefore test out loading it using

import gdal.

Additional Resources

• Setting up the Python Environment section from the Python Bootcamp
• Conda Help: setting up an environment
• iPython documentation: Kernals

DUE: 21 June 2018

During the NEON Data Institute, you will share the code that you create daily with everyone on the NEONScience/DI-NEON-participants repo.

Through this week’s tutorials, you have learned the basic skills needed to successfully share your work at the Institute including how to:

• Create your own GitHub user account,
• Set up Git on your computer (please do this on the computer you will be bringing to the Institute), and
• Create a Markdown file with a biography of yourself and the project you are interested in working on at the Institute. This biography was shared with the group via the Data Institute’s GitHub repo.

Checklist for this week’s Assignment:

You should have completed the following after Pre-institute week 2:

• Fork & clone the NEON-DataSkills/DI-NEON-participants repo.
• Create a .md file in the participants/2018-RemoteSensing/pre-institute2-git directory of the repo. Name the document LastName-FirstName.md.
• Write a biography that introduces yourself to the other participants. Please provide basic information including:
  • name,
  • domain of interest,
  • one goal for the course,
  • an updated version of your Capstone Project idea,
  • and the list of data (NEON or other) to support the project that you created during last week’s materials.
• Push the document from your local computer to your GithHub repo.
• Created a Pull Request to merge this document back into the NEON-DataSkills/DI-NEON-participants repo.

NOTE: The Data Institute repository is a public repository, so all members of the Institute, as well as anyone in the general public who stumbles on the repo, can see the information. If you prefer not to share this information publicly, please submit the same document but use a pseudonym (cartoon character names would work well) and email us with the pseudonym so that we can connect the submitted document to you.

---

Have questions? No problem. Leave your question in the comment box below. It's likely some of your colleagues have the same question, too! And also likely someone else knows the answer.

This tutorial covers how create and format Markdown files.

What is the .md Format?

Markdown is a human readable syntax for formatting text documents. Markdown can be used to produce nicely formatted documents including pdfs, web pages and more. In fact, this web page that you are reading right now is generated from a markdown document!

In this tutorial, we will create a markdown file that documents both who you are and also the project that you might want to work on at the NEON Data Institute.

Markdown Formatting

Markdown is simple plain text, that is styled using symbols, including:

• #: a header element
• **: bold text
• *: italic text
• `: code blocks

Let's review some basic markdown syntax.

Plain Text

Plain text will appear as text in a Markdown document. You can format that text in different ways.

For example, if we want to highlight a function or some code within a plain text paragraph, we can use one backtick on each side of the text ( ), like this: Here is some code. This is the backtick, or grave; not an apostrophe (on most US keyboards it is on the same key as the tilde).

To add emphasis to other text you can use bold or italics.

Have a look at the markdown below:

  The use of the highlight ( `text` ) will be reserved for denoting code.
To add emphasis to other text use **bold** or *italics*.

Notice that this sentence uses a code highlight "``", bold and italics. As a rendered markdown chunk, it looks like this:

The use of the highlight ( text ) will be reserve for denoting code when used in text. To add emphasis to other text use bold or italics.

Horizontal Lines (rules)

Create a rule:

  ***

Below is the rule rendered:

---

Section Headings

You can create a heading using the pound (#) sign. For the headers to render properly there must be a space between the # and the header text. Heading one is 1 pound sign, heading two is 2 pound signs, etc as follows:

Heading two

## Heading two

Heading three

### Heading three

Heading four

#### Heading four

For a more thorough list of markdown syntax, please read this GitHub Guide on Markdown.

---

Got questions? No problem. Leave your question in the comment box below. It's likely some of your colleagues have the same question, too! And also likely someone else knows the answer.

This tutorial covers adding new edits or contents from your forked repo on github.com and a central repo.

We now have done the following:

1. We've forked (made an individual copy of) the NEONScience/DI-NEON-participants repo to our github.com account.
2. We've cloned the forked repo - making a copy of it on our local computers.
3. We've added files and content to our local copy of the repo and committed the changes.
4. We've pushed those changes back up to our forked repo on github.com.

Once you've forked and cloned a repo, you are all setup to work on your project. You won't need to repeat those steps.

In this tutorial, we will learn how to transfer changes from our forked repo in our github.com account to the central NEON Data Institute repo. Adding information from your forked repo to the central repo in GitHub is done using a pull request.

Adding to a Repo Using Pull Requests

Pull Requests in GitHub

Step 1 - Start Pull Request

To start a pull request, click the pull request button on the main repo page.

Alternatively, you can click the Pull requests tab, then on this new page click the "New pull request" button.

Step 2 - Choose Repos to Update

Select your fork to compare with NEON central repo. When you begin a pull request, the head and base will auto-populate as follows:

• base fork: NEONScience/DI-NEON-participants
• head fork: YOUR-USER-NAME/DI-NEON-participants

The above pull request configuration tells Git to sync (or update) the NEON repo with contents from your repo.

Head vs Base

• Base: the repo that will be updated, the changes will be added to this repo.
• Head: the repo from which the changes come.

One way to remember this is that the “head” is always ahead of the base, so we must add from the head to the base.

Step 3 - Verify Changes

When you compare two repos in a pull request page, git will provide an overview of the differences (diffs) between the files (if the file is a binary file, like code. Non-binary files will just show up as a fully new file if it had any changes). Look over the changes and make sure nothing looks surprising.

Step 4 - Create Pull Request

Click the green Create Pull Request button to create the pull request.

Step 5 - Title Pull Request

Give your pull request a title and write a brief description of your changes. When you’re done with your message, click Create pull request!

Check out the repo name up at the top (in your repo and in screenshot above) When creating the pull request you will be automatically transferred to the base repo. Since the central repo was the base, github will automatically transfer you to the central repo landing page.

Step 6 - Merge Pull Request

In this final step, it’s time to merge your changes in the NEONScience/DI-NEON-participants repo.

NOTE 1: You are only able to merge a pull request in a repo that you have permissions to!

NOTE 2: When collaborating, it is generally poor form to merge your own Pull Request, better to tag (@username) a collaborator in the comments so they know you want them to look at it. They can then review and, if acceptable, merge it.

To merge, your (or someone else's PR click the green "Merge Pull Request" button to "accept" or merge the updated commits in the central repo into your repo. Then click Confirm Merge.

We now synced our forked repo with the central NEON Repo. The next step in working in a GitHub workflow is to transfer any changes in the central repository into your local repo so you can work with them.

After completing the pull request to upload your bio markdown file, be sure to continue on to Git 07: Updating Your Repo by Setting Up a Remote to learn how to update your local fork and really begin the cycle of working with Git & GitHub in a collaborative manner.

Workflow Summary

Add updates to Central Repo with Pull Request

On github.com

• Button: Create New Pull Request

• Set base: central Institute repo, set head: your Fork

• Make sure changes are what you want to sync

• Button: Create Pull Request

• Add Pull Request title & comments

• Button: Create Pull Request

• Button: Merge Pull Request - if working collaboratively, poor style to merge your own PR, and you only can if you have contributor permissions

  ---

  Have questions? No problem. Leave your question in the comment box below. It's likely some of your colleagues have the same question, too! And also likely someone else knows the answer.

This tutorial reviews how to add and commit changes to a Git repo.

In the previous lesson, we created a markdown (.md) file in our forked version of the DI-NEON-participants central repo. In order for Git to recognize this new file and track it, we need to:

1. Add the file to the repository using git add.
2. Commit the file to the repository as a set of changes to the repo (in this case, a new document with some text content) using git commit.
3. Push or sync the changes we've made locally with our forked repo hosted on github.com using git push.

Check Repository Status -- git status

Let's first run through some basic commands to get going with Git at the command line. First, it's always a good idea to check the status of your repository. This allows us to see any changes that have occurred.

Do the following:

1. Open bash if it's not already open.
2. Navigate to the DI-NEON-participants repository in bash.
3. Type: git status.

The commands that you type into bash should look like the code below:

# Change directory
# The directory containing the git repo that you wish to work in.
$ cd ~/Documents/GitHub/neon-data-repository-2016

# check the status of the repo
$ git status

Output:

On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
 (use "git add <file>..." to update what will be committed)
 (use "git checkout -- <file>..." to discard changes in working directory)

Untracked files:
 (use "git add <file>..." to include in what will be committed)

_posts/ExampleFile.md

Let's make sense of the output of the git status command.

• On branch master: This tells us that we are on the master branch of the repo. Don't worry too much about branches just yet. We will work on the master branch throughout the Data Institute.
• Changes not staged for commit: This lists any file(s) that is/are currently being tracked by Git but have new changes that need to be added for Git to track.
• Untracked file: These are all new files that have never been added to or tracked by Git.

Use git status anytime to view any untracked changes that have occurred, what is being tracked and what is not currently being tracked.

Add a File - git add

Next, let's add the Markdown file containing our bio and short project summary using the command git add FileName.md. Replace FileName.md with the name of your markdown file.

# add a file, so that changes are tracked
$ git add ExampleBioFile.md

# check status again
$ git status

On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
 (use "git reset HEAD <file>..." to unstage)

	new file:  _posts/ExampleBioFile.md

Understand the output:

• Changes to be committed: This lists the new files or files with changes that have been added to the Git tracking system but need to be committed as actual changes in the git repository history.

Commit Changes - git commit

When we add a file in the command line, we are telling Git to recognize that a change has occurred. The file moves to a "staging" area where Git recognizes a change has happened but the change has not yet been formally documented. When we want to permanently document those changes, we commit the change. A single commit will work for all files that are currently added to and in the Git staging area (anything in green when we check the status).

Commit Messages

When we commit a change to the Git version control system, we need to add a commit message. This message describes the changes made in the commit. This commit message is helpful to us when we review commit history to see what has changed over time and when those changes occurred. Be sure that your message covers the change.

# commit changes with message
$ git commit -m “new example file for demonstration”

[master e3cd622] new example file for demonstration
 1 file changed, 56 insertions(+), 4 deletions(-)
 create mode 100644 _posts/ExampleFile.md

Understand the output: Each commit will look slightly different but the important parts include:

• master xxxxxxx this is the unique identifier for this set of changes or this commit. You will always be able to track this specific commit (this specific set of changes) using this identifier.
• _ file change, _ insertions(+), _ deletion (-) this tells us how many files have changed and the number of type of changes made to the files including: insertions, and deletions.

Why Add, then Commit?

To understand what is going on with git add and git commit it is important to understand that Git has a staging area that we add items to with git add. Changes are not actually documented and permanently tracked until we commit them. This allows us to commit specific groups of files at the same time if we wish. For instance, we may decide to add and commit all R scripts together. And Markdown files in another, separate commit.

Transfer Changes (Commits) from a Local Repo to a GitHub Repo - git push

When we are done editing our files and have committed the changes locally, we are ready to transfer or sync these changes to our forked repo on github.com. To do this we need to push our changes from the local Git version control to the remote GitHub repo.

To sync local changes with github.com, we can do the following:

1. Check the status of our repo using git status. Are all of the changes added and committed to the repo?
2. Use git push origin master. origin tells Git to push the files to the originating repo which in this case - is our fork on github.com which we originally cloned to our local computer. master is the repo branch that you are currently working on.

Let's push the changes that we made to the local version of our Git repo to our fork, in our github.com account.

# check the repo status
$ git status

On branch master
Your branch is ahead of 'origin/master' by 1 commit.
  (use "git push" to publish your local commits)

# transfer committed changes to the forked repo
git push origin master

Counting objects: 1, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 1.51 KiB | 0 bytes/s, done.
Total 6 (delta 4), reused 0 (delta 0)
To https://github.com/mjones01/DI-NEON-participants.git
   5022aca..e3cd622  master -> master

NOTE: You may be asked for your username and password! This is your github.com username and password.

Understand the output:

• Pay attention to the repository URL - the "origin" is the repository that the commit was pushed to, here https://github.com/mjones01/DI-NEON-participants.git. Note that because this repo is a fork, your URL will have your GitHub username in it instead of "mjones01".

View Commits in GitHub

Let’s view our recent commit in our forked repo on GitHub.

1. Go to github.com and navigate to your forked Data Institute repo - DI-NEON-participants.
2. Click on the commits link at the top of the page.
3. Look at the commits - do you see your recent commit message that you typed into bash on your computer?
4. Next, click on the <>CODE link which is ABOVE the commits link in github.
5. Is the Markdown file that you added and committed locally at the command line on your computer, there in the same directory (participants/pre-institute2-git) that you saved it on your laptop?

Is Your File in the NEON Central Repo Yet?

Next, do the following:

1. Navigate to the NEON central NEONScience/DI-NEON-participants repo. (The easiest method to do this is to click the link at the top of the page under your repo name).
2. Look for your file in the same directory. Is your new file there? If not, why?

Remember the structure of our workflow.

We’ve added changes from our local repo on our computer and pushed them to our fork on github.com. But this fork is in our individual user account, not NEONS. This fork is separate from the central repo. Changes to a fork in our github.com account do not automatically transfer to the central repo. We need to sync them! We will learn how to sync these two repos in the next tutorial Git 06: Syncing GitHub Repos with Pull Requests .

Summary Workflow - Committing Changes

On your computer, within your local copy of the Git repo:

• Create a new markdown file and edit it in your favorite text editor.

On your computer, in shell (at the command line):

• git status
• git add FileName
• git status - make sure everything is added and ready for commit
• `git commit -m “messageHere”
• git push origin master

On the github.com website:

• Check to make sure commit is added.
• Check to see if the file that you added is visible online in your Git repo.

---

Have questions? No problem. Leave your question in the comment box below. It's likely some of your colleagues have the same question, too! And also likely someone else knows the answer.