ESPM 4295 -GIS in Environmental Science and Management

Week 1 - Introduction and Course Mechanics

Read and understand the Student Code of Conduct and Academic Integrity

Read the Semester Project Description, to get an idea of what we're heading for over the next 15 weeks, and look at the Rainfall Mitigation description and the Met. Council Summary for additional motivating information.

Before the end of class on Wednesday, sign up for one of the project areas on the St. Paul Campus (2018 assignment sheet)

Create a practice geodatabase in NAD83(CORS96) UTM Zone 15 coordinates. The purpose is to dust off elementry digitizing and geodatabase creation skills, in case you haven't used them for a while. We also want you to get familiar with the image data on the class L drive (see a description in the rightmost column), and your study area.

Create a feature data set and two feature classes, as described in the following paragraphs.

The first feature class should contains a point data set. Open ArcMap and display an WMS of campus and digitize points on the St. Paul Campus for

1) the water tower behind Green Hall,

2) St Paul student center, and

3) The old power plant smoke stack, near the south end of campus.

The second feature class should be a polygon layer that defines the boundary of your chosen study area.  Use the images in the handout as a guide for the approximate boundaries of your study area. Most boundaries consist of roads. Load a WMS image (see resources at right), and create a polygon that follows these roads, including at least the middle of the road on the outside edge of your study area. Where it is a field edge or other non-road features, follow the "natural" boundaries, e.g., fence line, ditch, or parking lot edges. Consult Paul or Andy if you have any questions on boundary location. Do this boundary carefully, you'll be using it later in the semester.

Create a map with an image background, your digitized points and study area boundary, your name, a title, scale bar, and north arrow, and export as a pdf.

Next week, before Monday's class you must submit the map of your data, and the GDB file as a .zip archive, or similar common archive (e.g., .tar). If you do not know how to do this, see "How to Zip/Unzip" in the Resource column to the right.

If you cannot create a geodatabase, or are a bit fuzzy on digitizing in ArcMap from when you last learned it, or creating a map, then you should review the intro digitizing materials, in the resources column at right. We'll expect you to be able to create geodatabase layers and digitize into them quickly and easily for next week's exercise. 

If you're still lost after you've reviewed digitizing videos, and attempted a few times, ask Andy or Paul for help.

Generally, deadlines for the current week's work are described in this column.  Turn assignments in on the Course Canvas Site, unless specified otherwise.  Some activities are due in the same week as assigned, but typically assignments are due a week after being introduced.

All assignments due on Moodle, unless specified otherwise

By end of Wednesday's class,  our first meeting, you must:
Sign up for Study Area

and

Prior to class Next Monday, before the start of class, turn in your:

1. Geodatabase with a feature class for the three digitized campus points, and a feature class for the study area boundary, and 
2. Your pdf map.

Rules to follow when turning things in:

• Unless otherwise specified, submit all data as geodatabases (GDB extension), compressed in  .zip or similar formats. 
• Submit maps as pdfs
• Include your initials at the start of your files, and a W# where # is the week at the end of the filename.  For example, I might name my map "pvb_practicemap_W1.pdf", my geodatabase "pvb_practice_data_W1.gdb" and my zip file "pvb_practice_W1.zip".

Failure to follow this naming convention for compressed and component files may result in a deduction.

Next Monday before class is a hard deadline, you should turn in what you have. This class moves at a fast pace, and you can't be neglecting new work.  Once the deadline has passed, you will receive a 50% deduction from your graded score, and after the next class meeting, zero points. Unless noted otherwise, this will be our policy for assignments. If you're traveling out of town for another committment, pre-clear tardy submissions.

Course files are on the L:\ drive accessible from the computers in 35 Skok Hall, under the CFANS\LABS\ESPM4295W directory. Below we'll refer to the subdirectory as CLD for "Course Lab Drive"

There are is an ESPM4295W\CampusImages directory on the L:\drive. Three are various high-resolution, drone-based images of the St Paul Campus. You'll be using these extensively over the course of the semester.

Each student is also assigned a L:\home\{your id} directory.
This directory is for all your class work.  The Course Lab Drives are "read only" so you will copy all needed material to L:\home\{your id} for class work.

If you are going to work outside the lab (you have access to ArcGIS at home or in another office), you may wish to copy the high resolution image files to a USB or similar portable drive before leaving today. While an alternative is to use the instructions on how to access files over a VPN, in last week's resources, to copy over the internet, this may be unbearably slow if you don't have a very fast internet connection, with large files prone to hiccups and restarts during download. 

Introduction Slides

Citrix Virtual Computer Quick Start - access a MSWindows virtual desktop via browser, run ArcGIS software and access classdata)

How to Zip/Unzip

Review materials:

• Intro digitizing videos, from FRNM3131

Week 2 - NSSDA accuracy exercise and report

Display and view the photos available for this class on the class L drive, ESPM4295W directory, in the CampusImages folder. Your goal is to formally assess the geometric accuracy  of the 2017 leaf-off image ONLY, using the NSSDA specification as described in lecture this week.  You will also  visually assess the error in some of the other images, and write a brief report. 

The process involves extracting coordinates for well-defined, visible points on the image, and comparing them to "true" GNSS coordiantes collected for those same points. The location differences for the same point will define an error, and we'll do several points to get error statistics.

Follow the guidlines linked here.   

By next Wednesday, before the start of class, turn in: 

1. The NSSDA spreadsheet, 
2. A geodatabase with the layer you digitized for your 15-20 test point locations, and
3. Image accuracy assessment report.

The geodatabase must be zipped/archived. Remember to follow the naming convention, i.e., include your initials at the end of the component files, and week designators in the file name, as described in last week's deadline section. For example, I might name my accuracy spreadsheet something like pvb_NSSDA_W2.xls, and the zip file something like pvb_AccGDB_W2.zip.

Accuracy Assessment Slides

Links: Overview map, individual point descriptions,  and shapefile of GPS points for NSSDA accuracy assessment

Template Spreadsheet in CLD\NSSDA, and downloadble here.

NSSDA handbook, downloadable here.

Review materials:

• Extracting XY from shapefile or feature class points (for NSSDA calculations)

Additional information for your report: 

Read the "General Guidelines for Writing" and the "Writing Hints" pdfs in the CLD\Writing directory, as you'll be submitting several reports in this class. 

Additional tools that might be useful

• MS Windows Snip image from screen for figures
• ArcMap Export image from data frame for figures
• Using Draw.io to create a flowchart of your workflow

Week 3 - Project Geodatabase Creation, Digitizing, Topology 

Create a Geodatabase for your project area, containing the layers listed below. Review topology editing/digitizing in ArcMap as needed (videos on right), and begin digitizing for your project area, interpreting features from the aerial images, using the 2017 leaf-off as your main source, and supplementing with the other images as they appear helpful.

Your geodatabase should include:

1) A "Landcover" polygon layer that will hold sidewalks, plazas, grassy, forest, and shrub/flower bed areas, basically all road, non-building areas in your project area. Include a text attributes for this layer named material specifying asphalt, concrete, grass/forb, shrub, soil, or other. 

2) A "Building" polygon layer to hold all building footprints, with an attribute for the name (text), height (float, height in meters, with room for at least 1 decimal place), and roof type (categorical, with flat, low pitch, and steep pitch as categories).

3) A "Storm Sewer" point layer of all storm sewer grates. You can start with the data provided at right in Resources column, importing into your geodatabase the indicated layer, and then modifying as needed. It should have a column named verified (1 character text, holds either Y or N) that records if it has been field visited and verified (we'll do this verification in Week 6). 

4) A "Canopy" polygon layer of your tree canopy, with attributes for tree type (text) as conifer or broad-leaved, height (float, at least 1 decimal), and if the type has been verified (1 character text, Y/N).

5) If you have any, a layer that includes polygons of stormwater ponds, lakes, raingardens, or other infiltration areas, with a text attribute for type.

6) A "Boundary" layer that includes your study area boundary - you should import the one you digitized in week 1.

7) Layer topology, with, topological restrictions as:

• Land Cover and Buildings must not overlap,
• Land Cover polygons must not have gaps (except the holes created by Buildings) 
• Storm Sewer Grates must not be contained in Buildings
• Buildings and Land Cover must not overlap with Stormwater Ponds
• All layers must be contained within your Study Area Boundary

Digitize a point layer of  all the storm sewer drains for your study area, based primarily on what you can see in the 2017 leaf-off image. 

Start with the 2017 leaf-off image, but you may also use the 2015, 2016, and 2018 images to look in areas where buildings, trees, or other features obscure the ground in the 2017 leaf-off image. Beware of construction and changes in roads and storm drain  locations in the earlier images.

Optional early start: Digitize All Layers in a Test Area. If you have time, you should pick a sub area, about 10% of your study area, to digitize ALL layers, and test the topology rules for that digitized area. Pick a test area with trees and buildings, as some of the study areas have no trees or buildings over a large portion of their extent. 

You need to turn in:
Before the start of class on Wednesday of this week:
A geodatabase with:

• your completed study area polygon,
• your completed sewer grates for your study area,
• empty feature data classes (layers) for the remaining listed layers in the assignment to the left, 
• your first set of topological rules.

This is due before the start of class on this Wednesday.  You need to create the geodatabase and create the topology so that we can answer questions and start digitizing on Wednesday.

By next Monday before the start of class, turn in:

Your geodatabase with your completed storm sewer layer, your study area boundary layer, and empty feature classes for the other 5 layers, and completely specified topology rules.

You should also turn in two pdf maps, 

1) of the completed storm sewer point layer, with your study area boundary, an image background, and all the usual map elements (title, name, legend, scale, north arrow), and

2) a pdf map of your 10% test area, with the usual map elements.

Remember to use our standard naming convention (initials on the front, week on the back of the name), and zip up your data into an archive.

WEEKLY SLIDES

Identifying storm sewer grates 

Week 4 - Complete All Layers for your 10% area, Edit to "Clean/Remove" all Topological Errors, and Digitize Canopy for your Entire Study Area

Complete digitizing your sample area (remember, about 10% of your study area), using the digitizing strategies discussed this week. You should verify your topology, and practice fixing errors. Your data does not have to be completely "clean," and there may be "real" errors that need to be fixed, but make sure you understand how to find and fix errors using the various topological editing tools, as you will apply them next week, as well as doing.

Digitize all polygons for your canopy data layer (you don't need to complete the height attribute) over your entire study area. Use primarily the 2017 images, but as need include the 2015 through 2018 leaf on and leaf off images to locate crown boundaries as best you can. Digitize individual tree crowns, even when they touch or overlap - don't digitize groups of trees as one big blob. You need to assign heights to each tree, so in as much as you can, identify single trees when they are part of groups. It is difficult to determine what is an individual crown, but we'll go through examples in class, and do the best visual interpretation you can.

Create a map that contains only your 10% sample area, showing your data layers, and display the topology, making sure you are displaying any errors. Be sure to include the topology  and participating layers in your PDF map. It is o.k. if you still have topological errors, we'll be fixing those next week.

Create a second map that shows the tree canopy polygons.

Next Monday, before the start of class turn in:

1) Your completed geodatabase for your 10% sample area, all layers (Land Cover, Buildings, Storm Sewer Grates, Canopy, Study Area Bound, Ponds and other Collecting Areas, if you have them). Note that you your tree canopy polygon layer should be completely digitized, although the attributes may not be complete.  Also turn in:

2) the PDF map showing only your 10% sample area (not your entire study area) illustrating the same.

3) a PDF map of the tree canopy polygons for the entire study area.

Observe the usual requirements for the map elements, all data maps in one archive, and archive naming and submission.

Remember to use our standard naming convention (initials on the front, week on the back of the name), and zip up your data into an archive.

Weekly Slides

Notes on Topology

Editing admonition 

Strategies for digitizing  - NEVER digitize the same line twice, beware overlapping areas

Editing, see the intro videos provided in Week 1, and: 

• Intermediate digitizing videos, from FNRM3131

 More videos on topology, a bit repetitive from what's been provided, but perhaps may help: 

-Topology concepts, 
-Create topology,
-Applying topology, identifying topological errors,
-Correcting errors example 1,
-Fixing topological errors example 2

Finally, a general way to find tools that you know exist, but can't seem to find in the ArcMap toolbox
-Search for Tools

Week 5 - Edit to "Clean/Remove" all Topological Errors, Download Soils Data, and Begin Digitizing Your Entire Study Area

Complete cleaning the topology for your sample area (remember, about 10% of your study area), and verify and fix topology, keeping the data logically consistent across layers. Your data should be topologically "clean," with all "real" errors fixed, before you turn it in, but if it isn't, turn in progress by the deadline.

Complete the exercise : Soils data download 

Read the instructions and download soils data for your study area, and condition it so that it is useful for your analysis.

Continue working on the remaining layers in your geodatabase for your entire study area, particularly the Buildings layer

Create a map that contains only your 10% sample area, showing your data layers, and display the "clean" topology, that is, after topology checks and fixing any errors. Be sure to include the topology  and participating layers in your PDF map.

Create a map that shows the tree canopy polygons

Next Monday, before the start of class turn in:

• Your geodatabase, with the soils data layer and your completed layers, including landcover, buildings, and canopy, and grates

Create two pdf maps of vector data

1. a map with your soils data, displaying the soils data symbolized by type.  Include your study area boundary, and an image layer background.
2. a second map with your completed 10% area digitized, including all layers, and your clean topology, with all errors fixed.

Slides for Soil Download

Week 6 - Digitizing Buildings and Landcover Layer, 

Digitize Buildings for your entire study area
Complete the Buildings data layer for your entire study area - including all attributes, except for the height attribute, 

Complete your Landcover layer.

Next Monday, before class,
turn in:

A geodatabase with

1. a completed buildings layer, with all building attributes completed (name, roof type)
2. Your landcover data layer

Also turn in a pdf map that displays your buildings, landcover, and topology for your entire study area.

Include an image background, and the usual other map elements.

Week 7 - Watershed Definition

Calculate Watersheds for your Study Area
Create watersheds for all sewer grates that drain more than 5,000 square meters in your study area.

Creating watersheds was covered in FNRM3131/5131, taken by most folks in this course, so we won't provide specific step-by-step instructions. There are videos and pdfs in this week's resouces section that cover the steps.

Prior to filling the DEMS, you should run the SINK function in the ArcToolbox->Spatial Analyst->Hydrology tools in ArcMap. This will show the general extent of sinks, and you may then use the inquire cursor over the sink regions to identify the depth of most sinks. This will be useful in setting the depth for the fill process.

You will use the sewer grates as your pour points/outlets for your watersheds. Note that because the DEM contains some error, the flowpaths won't intersect many of your sewer grates, but only run near.  You should create a copy of your true sewer grate locations layer, and modify this copy to move the grates to fall on the appropriate nearest flowpath.  Make sure you work on the copy.

Include these watersheds and the corresponding pourpoint layer as part of your geodatabase you turn in next Monday before class.

Create and turn in a PDF map with an image background that shows your study area boundary, the pourpoints, watersheds as semi-transparent overlays, along with all the title, name, and other basic map elements. 

Next Monday, before class, turn in:

your completed geodatabase, all layers, including watersheds
your watershed map

Note that you will be graded on both the completeness and accuracy of the data in your geodatabase. Your topology should be included in your geodatabase. You must have all data digitized, and should have very few topological errors

• Your completed, edited, topologically correct geodatabase
• A total of two maps, both with image backgrounds and your study area boundary, that depict:

1. A hydrology map with watershed outlines (not filled, so we can see the air photo below), pourpoints/sewer grates, and vector flowlines from the classified/recoded flow accumulation layer with a stream threshold of 800 square meters
2. landcover and buildings, and each landcover type with a distinct color, and canopy colored by a continuous green height gradient.

Monday Slides 

Watersheds

DEM fill, sinks, flow direction, flow accumulation

Snap pour points (grates), Watersheds

Excerpt from Hydro Processing lab in 3131/5131

1st student Hydro Processing 1 Report

2nd student Hydro Processing Report

Week 8 - Project Roadmap

Monday in Class - we'll discuss the overall project, analysis, and output. Participation IS Mandatory

Carefully re-read the semester project description provided in the first week, and begin developing your flowchart of analysis. You need to think about  the spatial operations and order in which they'll be applied, represented by a box and arrows diagram you'll apply. 

Re-read the project description before class, and come prepared to ask questions about anything that's not clear.

We will provide you with campus wide data very similar to those you have been developing. All work from here forward will be using these campus-wide data, and NOT those that you have developed.  The data are available on the class lab drive, and via a zip file to the right, in Resources.

You may have to modify the data layers a bit for your analysis, e.g., re-calculate the watersheds, move the drain pour-points, and modify a few attributes, but these are relatively minor.

Your task is to estimate the runoff as described in the semester project document, for various storms under current landscape conditions, and after modifying tree canopy, landcover, buildings, and underground storage to mitigate rainstorms.

We will discuss an example flowchart, and provide a refresher on the raster calculator, raster to vector conversion, and some background for the specific problem.

You will have to add details to the flowchart, or create an alternative detailed one to estimate base runoff estimates for various storms.

Complete the Flowcharting exercise.

Before the start of class on Wednesday, this week,

turn in your flowchart exercise - first draft of your flowchart

Week's Slides

Example flowchart

Flowchart description

Make your analysis easier: Add a custom toolbox in ArcMap

Week 9 - Various Tools  

. 

By next Monday, before class, turn in:

• .

 Raster-vector conversion

Week 10 - More Tools

B

Week 11 - 

Work on Model Application to the Study Watersheds

Complete a detailed flowchart of your workflow which fills in missing steps and specific details (all arcgis tools/functions, names of intermediate data sets) so that it may serve as a roadmap to your analysis.

.

Before class on Wednesday, this week, turn in your latest flowchart - added details, ready for you to begin analysis in earnest.  This flowchart should have enough detail to demonstrate that you've started your analysis, and have experimented with the various steps and determined the specific arcgis tools, and specific parameters for each tool, you'll be using for each operation

ModelBuilder description and assignment (optional, extra credit) 
read and understand, optional for undergraduates to turn in the assignment, but using model builder could greatly ease the subsequent repeat analysis

Week 12 - Complete First Analysis of Current Conditions

Report draft 1 requirements - This first draft should include
1) the runoff volume and cost tables for your the 2.5 cm rainfall condition, in the format specified in the project description, and
2) a base map showing the watersheds on a campus image

Remember that the target audience is the set of administrators in charge of the University physical environment and stormwater management. They don't know anything about GIS software, or GIS theory, or any jargon on geospatial analysis terms. You need to provide enough detail so that an intelligent non-expert will understand what you did, and be convinced that your methods are sound and your results valid.

Turn in next Monday, before class:

• Report Draft 1 and data, including runoff volume calculations for at least 1 grate/watershed in table

You will most likely find it easiest to calculate a runoff "credit" for each of your rain gardens/surface infiltration areas, and underground storage features.  This credit is equal to the rain garden area in square meters, times 0.5 (maximum depth of rain garden, in meters), to calculate storage cubic meters of water. Your sum for the containing watershed should be reduced by this amount.  An underground storage feature is simply a point placed on a flowpath as near the outlet pourpoint as feasible, with an assigned maximum storage volume.

Note that the effective placement of rain gardens and underground storage requires they be on or near (a few meters) an existing flowpath.  Since flowpaths depend on the specific DEM, which in turn depends on the conditioning parameters, the DEM used in this analysis, along with corresponding flow accumulation, flow direction, and flowpaths are included in the finaldata directory on the L: drive. The flow1000 shapefile contains flowlines that drain 1000 1m cells or more. The ecDEM, ecfac, and ecfdir contain the DEM, flow accumulation, and flow direction rasters as tiffs, should you need them.

Also note that the rain garden must be downslope from a runoff generating area (impervious, building, saturated soil) in your watersh

Week 13 -  Project work, modifications and analysis to mitigate runoff for prescribed rainfall amounts

Examples of professionally written reports - good examples of proper tone, grammer, and level of detail:

1: Browns Creek
2: NPS Pollinator Survey
3: NPS Kettle Ponds
4: Dall Sheep Survey
5: Grand Sable Netting
 

Week 14 - Report Draft, and Continue Analysis Under Modifications

Report Draft 2 should include
1) your introduction (about 2 pages of text and not counting figures, 1.5 line spacing, 11 pt font or smaller) that briefly introduces the project, with a problem statement, location, goals, and basic approach, and supporting figures/maps,

2) Your methods (about 2-3 pages of text, not counting figures). You should describe the data layers and basic methods for developing them. Do not provide a sequential list or narrative of data development (first we did this, then we did that), and you only need to identify the software or specific software tools once, in the general introduction of methods.  Include important parameters that affect data quality, e.g., snap tolerances in digitizing. Include at least one figure of the output layers, and your flowchart of analysis. 

3) a written description of results for your 2.5 cm (1")  and and 5 cm (2") rainfalls under current conditions, including the completed tables for both of these "as is" conditions, including a discussion and supporting figures/maps that identify areas that generate large amounts of runoff, and potential approaches for mitigation for these areas.

4) a map of at least two layers that have been modified to mitigate runoff, e.g., of canopy, or rain garden locations.  These may be "in progress", that is, the layers may only cover a subset of target watersheds, and be further modified to improve your runoff mitigation, but you must show substantial progress at this point.

5) Include an appendiz with a first draft of maps of each input data layer.

No class final; all assignments due by Wednesday, December 21, 1:30 pm

You should turn in three final geodatabases:

1) A geodatabase with the word "Current" in the name that contains data used for calculating your "current condition" runoff for 2.5 cm and 5 cm storms. These are the base final data, as provided, with any modifications to the main data layers.  The geodatabase should also include your final "surplus water" layer, that is, the layer that you summed by watershed to get the total runoff volume for the study areas. 

2) A geodatabase with "2p5cm" in the name, that includes the modified layers used to obtain your 2.5 cm mitigation of runoff. You should include both the original and modified layers for a 1" storm only, that is, the original canopy layer and the canopy layer with added trees for a 1" storm, and NOT the canopy layers  or any other layers for a 2" storm.  If you did not need to modify a layer to reach the 1" threshold (e.g., no green roofs required for buildings), then just include the original layer (e.g., the original buildings).
Include any of the modified layers from: a) canopy, b) rain gardens/surface storage, c)perviousness, d) buildings (with green roofs attributed), e) subsurface storage entry points.

3) Same as 2, except for the 5 cm rainfall, with "5cm" in the name of the geodatabase. DO NOT include any of the modified layers from the 2.5 cm analysis, but DO include your original data layers and any modified data layers needed to meet the 5 cm requirements.

Your final report should include the information provided in previous drafts, plus a description/discussion of the changes for reducing runoff to zero under the different rainfall amounts. This should include appropriate maps/figures.

Thursday Dec 18, by 11:55 p.m., turn in your final report, via Canvas, and your copy your final data to the 4295Share\Z_FinalData\yourname subdirectory