ESPM 4295 -GIS in Environmental Science and Management

The week's activities are listed in this column

Mandatory meeting, everyone must join in the whole-class Zoom meeting on Wed., Sept. 9, link provided here:  Class Zoom Meeting Room

Week 1 - Introduction, Course Mechanics, ArcGIS Set Up and Use
Our primary goals this week are to cover course mechanics, for you to set up and verify ArcGIS Pro and class data access from home, refresh basic ArcGIS use and digitizing, and introduce our semester-long problem spatial analysis project. Whew.

This course is to develop mid-level skills in combining GIS tools in spatial analysis. The course may be taken entirely online, and the majority of the learning is asynchronous, meaning you can do it on your own. That said, there are a few, compulsory, synchronous, online, whole-class meetings, early on to describe our overarching project, and intermittently to cover key topics. We will also provide online, individual consultation. Andy and I have been teaching an online section of our introductory class for almost a decade.  Students may schedule individualized instruction in the teaching lab in 35 Skok Hall, depending on the trajectory of Covid-19 and subject to the reduced lab capacities. Masks covering the nose and mouth will be required at all times while in the lab room.  

This course is split into two periods. We develop skills during the first six or so weeks, focusing primarily on two practice areas in the northern part of the St Paul Campus (map here). The remainder of the semester focuses on an analysis for two project watersheds for an expanded area (map here).

ArcGIS Software Access
Most of us will use ArcGIS Pro via the web, with an app that gives access to a virtual computer. We will use a software called Citrix for this. Most students need only install Citrix for ArcGIS access, and all students should attempt to install it. Instructions are provided in the X295 Software Setup document, linked in the resources column, and the To Do section below.

All students should also install VPN software. This is needed to connect to a CFANS Lab drive that we'll use to distribute data. Instruction on installing the VPN software and on mapping a network drive are in the X295 Software Setup document.

Although Citrix will be the best method of ArcGIS access for most students, as an addition, or an alternative, you may also download/install ArcGIS on your own computer if it meets the technical requirements. 

Finally, depending on the course of Covid-19 and campus opening, you can come to the 35 Skok Hall labs for help/to work. 

Using the VPN and Accessing Data
As noted above, we will distribute data via a CFANS Lab drive. This drive is accessed via UMN distributed VPN software and drive mapping. Although the drive may have another letter designator when mapped to your computer, we will call it the L drive, or the CLD (class lab drive), as L this is the letter assigned when mapping in Skok 35. Our source data will be in the ESPM4295W\Data directory, and images in the ESPM4295W\CampusImages directory. 

The L:\ drive is accessible by mapping the drive from your home computer, as described in the  X295 Software Setup, linked in the To do section below, and in the resources column at right. You will start a VPN, map the L Drive, and copy the source data to your local computer drive.  Most of you will start a CFANS desktop/virtual computer through Citrix, and then copy the files you need to the virtual computer desktop for work in ArcGIS. When done, you will copy any needed files, e.g., your digitized layers,  from the virtual/Citrix computer back to your local computer, physical drive.  You should also periodically save project work to your Google drive, USB drive, or other "permanent" drive for backup. 

Note that you should NOT directly load files from the class data drive (L: drive) into an ArcGIS map; this means don't add data from the L: directly into an ArcGIS Map when using a virtual machine, or when working on your home computer, or in the Skok Hall Lab. 

ALWAYS copy your data files to the virtual machine desktop if using ArcGIS through Citrix, or copy files to your desktop or another local drive if using ArcGIS installed on a home computer, then add the files to ArcGIS from the local drive. At times you may get away with using the L drive directly, but your project is more likely to freeze, crash, or lose data, and at times it will be excruciatingly slow - particularly during peak computer use on campus. Do yourself a favor and get in the habit of copying data to your desktop when you start, and back to the L drive or a local drive when you finish.

Each student is also assigned a L:\home\{your id} directory.
This directory can be used as a backup for classwork, such that you can copy data you produce to L:\home\{your id}. You can also copy data to your google drive, or another location, but you should save your work at least daily, and at least once a week you should make a complete copy of all your data, time-stamped, so that if something goes wrong, e.g., a system crashes, taking your data with it, it won't be catastrophic.

One final word about ArcGIS access. Most of you will use Citrix, and access resources through an internet connection. A wired connection is often many times faster and more reliable than WiFi connection to your router.  If you have a clear path from your internet router to your computer, and your computer has an ethernet port, buy a CAT-5 or CAT-6 ethernet cable and connect directly to your router. If your computer does not have an ethernet port, consider purchasing a USB to ethernet plug (about $25), and using it with an ethernet cable. If you cannot connect with a wire, then avoid shared WiFi as much as possible, by using a separate node, or working during hours where there isn't much traffic/use. 

To do this week:
A) Read the Semester Project Description linked here, to get an idea of what we're heading for over the next 15 weeks, and look at the Rainfall Mitigation description and the Minnesota Pollution Control Agency guidelines, and this Minute Earth video on the coming "poopocalypse" for additional motivating information.

B) Set up access to ArcGIS Pro via a virtual machine/web browser, OR install ArcGIS Pro on a home machine, using the instructions found here:  X295 Software Setup. You should use ArcGIS Pro, even if you learned ArcMap in a previous course. You may have trouble moving back and forth between Map and Pro versions of Arc, and ArcMap will be unsupported in the near future, so you'll have to switch eventually. 

C) Create a practice geodatabase in NAD83(CORS96/2011) UTM Zone 15 coordinates.  Refer to the materials in the resources column, far right, if you need a refresher in getting started in ArcGIS Pro. If after viewing you still have questions, talk to Paul and Andy. We'll expect you to be able to create geodatabase layers and digitize into them quickly and easily for subsequent exercises. 

Start ArcGIS Pro and create a feature data set and two feature classes in your new geodatabase, shown on the practice map here, and as described below:  a line feature class for a trees study area, and a polygon feature class for a landcover study area.

Second, either copy the NorthCampus_leafoff2017.tif image, found in the L: drive, under the 4295\CampusImages directory, or using a high-resolution image from the MNGeo WMS, as described in the column to the right. Unless you've already done so, to copy from the L drive you will need to set up a VPN and map the network drive, as described towards the end of the X295 Software Setup document, in part A, above.  You should figure this out this week, if possible, because you will have to use this mapping in most weeks. If you can't map a network drive, and you've read the instructions carefully and tried a few times, contact Andy Jenks or I to troubleshoot and solve the problem. You will need to access the L drive during the course of this semester.  

Third,  digitize the boundaries on the St. Paul Campus for:

a) a "Trees Practice Area" outline into a polygon feature class, as shown on the practice map, here, and

b) The "Landcover" Practice Area boundary, a polygon feature class, shown on the practice map, here.

You'll only be digitizing the boundaries now.  You'll digitize the trees, landcover, and buildings in future assignments.

D) Create a map with an image background, your two digitized layers, your name, a title, scale bar, and north arrow, and export as a pdf. Remember, you can get an image background from the ArcGIS standard backgrounds, or from the class images, described in the Resources column for this week, at the far right.

One final note for this week, in this and all assignments and conduct, you're expected to understand and follow the Student Code of Conduct and Academic Integrity. Read it and let the instructors know if you don't understand any parts.

Deadlines for each week's work are listed in this column 

Turn in assignments on the Course Canvas Site, unless specified otherwise. Some activities are due  the same week as assigned, but typically assignments are due the week after assignment.

Assignments are submitted via Canvas, unless specified otherwise.

Prior to class Next Wednesday, before 10:40 a.m. (the start of class):

• Learn how to start an ArcGIS virtual machine through a browser (much preferred), or install an at-home copy, for ArcGIS Pro
• Create and turn in a pdf map of the digitized features described in the left column, plus an image background, name, title, scalebar, north arrow, and legend.

Next Wednesday before class is a hard deadline, you should turn in what you have completed by then. This class moves at a fast pace, and you can't be neglecting new work.  Once the deadline has passed, you will receive a 30% deduction from your graded score, and after two weeks, zero points. Unless noted otherwise, this will be our policy for assignments. If you're traveling out of town for another commitment, pre-clear tardy submissions.

Rules when turning things in:

• Submit maps as pdfs
• Submit data as geodatabases (GDB extension), compressed in  .zip or Mac archive formats. 
• Submit written documents as MS Word files or Apple Pages files. All Word or Pages files should have track changes turned on every time you write and edit. Without track changes, I will deduct up to 25% - this course is writing-intensive, track changes helps me evaluate your efforts at thoughtful editing and improvement.
• Include your initials at the start of your files, and a W# where # is the week at the end of the filename.  For example, I might name my map "pvb_practicemap_W1.pdf", my geodatabase "pvb_practice_data_W1.gdb" and my zip file "pvb_practice_W1.zip".

Failure to follow this naming convention for compressed and component files may result in a deduction, and/or a return to you for re-submission. The naming and week identifiers greatly help me organize submissions and grading.

When you turn in geodatabases, they must be compressed, and they should not contain copies of layers or extraneous intermediate layers.

When you turn in vector data layers, make sure you don't have extraneous columns in the tables.  The data may come with "extra" columns, or you may be generating intermediate columns through processing. Keep only those needed for the primary purpose of your data.

Resources for the week are listed in this column

Introductory Slides as pdf, or video

Semester Project "Big Picture" Overview Slides (refer to this throughout the semester) as pdf, or video

You'll need ArcGIS Pro software access, follow these instructions: X295 Software Setup

A video on accessing ArcGIS through the Citrix Workspace software 

You must compress your geodatabases before turning them in, here's a video on file compression:
How to Zip/Unzip

Review materials from FRNM3131 (note, data referred to below are in the 3131 course files, but you shouldn't need them for understanding the mechanics):

ArcGIS Pro Refresher295

• Review this Video, if need be, for starting ArcGIS Pro, creating a new project
• Here is a short written excerpt from Lab 1 in the 3131 course, on creating a Geodatabase, and importing data into it - it is some pretty detailed text, may take a bit to wade through, but it may be a good reference if you've forgotten or didn't learn the startup process
• Review video here on creating a geodatabase
• Review video on creating a feature dataset
• Review video here on digitizing polygons
• Accessing the MNGeo WMS server in ArcGIS Pro, to specify an image background

There is a Personal Zoom Room for Paul and Personal Zoom Room for Andy. 

Email us if you wish to meet outside of class hours and we'll arrange a time to meet in this room.

Mandatory meeting on Monday, Class Zoom Meeting Room

Week 2 - Digitize trees and write the introduction of your final report

Task 1 - Digitize the tree canopy for the "Trees" practice area

Look in the L: drive, 4295W\Data folder, for the geodatabase named "PracticeAreas.gdb."  Copy this gdb to your virtual computer desktop (browser/Citrix ArcGIS) or local disk drive (home full install of Arc). There are two data layers in the gdb, "Trees_Dig_Area" and "Land_Build_Area", that you'll be using as boundaries for digitizing. They correspond to the areas you digitized last week, but we provide them here so that we all start on the same page.

Download the "NorthCampus_leafoff2017" and "NorthCampus_leafon2017"  photos to your "local" drive. These source files are on the class L drive, ESPM4295W directory, in the CampusImages folder. Note that there are other years' images for larger areas, you may access these also. 

Your goal is to digitize each tree's canopy outline based on the images. You only have to digitize trees within the boundary defined by the "Trees_Dig_Area" in the PracticeAreas geodatabase.

Manually interpret individual tree crowns based on the various leaf-on and leaf-off images. It is best to use the 2017 images as a base, but you may use others for some areas. You can use leaf-off images to somewhat identify the crowns and to separate evergreen from deciduous trees, but leaf-on images provide the best source for crown edges.

You should do your best job of interpreting the individual trees, even where they grow in clumps. Do not be too meticulous in digitizing tree crowns. Something like 8 to 12 vertices should be acceptable for most normal, "rounded" crown boundaries. The point here is to refresh your digitizing knowledge and practice the skills, not to spend hours digitizing. Use autocomplete digitizing or split polygon digitizing as appropriate on clumps (see resources at right). Remember that individual crowns cannot overlap with other crowns, you must have "planar" topology in this layer.

Also note that the tree crowns can overlap with buildings for this initial data set, but later we will remove the overlap.  It is easiest to digitize crowns rather coarsely where they overlap buildings but don't worry now about matching building edges, as we will later be digitizing building outlines, and can use building polygons with the Erase function to remove the overlapping tree portions. There are digitizing options that allow you to auto-complete along existing boundaries, so you could digitize the buildings first and then and then match these edges, but it is easier to Erase in this case. 

You should create a text attribute named something like TreeType with the values "conifer" or "deciduous" assigned as appropriate for each polygon.

Task 2 - Begin your semester-long report by writing a polished draft of the introduction. This draft should be two to five pages at 1.5 line spacing, not counting figures. It should describe:

• the problem you're addressing,  
• why we're interested in doing this, 
• a description and map/figure of the study area (remember, the two large watersheds, NOT the practice areas), and
• the general approach you'll be using. 

Look at the information on rainwater and runoff provided in the first week for motivation, and the example reports in this week's resources columns for example introductions. 

Your description should be written as if you were working over the entire, two watershed analysis area of the semester project (map here), and not just your practice areas.  We want to build the report in pieces over the course of the semester. If your graphics and text refer to the practice areas as your study area, you'll cause yourself more work, in that you'll have to re-write sections and create new figures for your final report. You can include graphic examples from the Trees and Landcover practice/skills areas you're digitizing, but be strategic so that you don't show the practice area boundaries in the figures. You should be clear to describe images of the digitized tree, building, or landcover layers as examples from a sub-area of your overall study area.

You should write in the present or past tense when describing general problems or conditions (e.g., "managing rainfall runoff is one of the most common and expensive problems in developed areas,"  and the past tense when you're describing the data development and analysis of your project (e.g., "we used several images as data sources..."). Write as if you've already completed your data development and analysis, even though you haven't started yet. You'll slowly build the report, an assignment at a time, so you'll revise the introduction and add methods, results and recommendations in later labs. 

I'll expect you will have gone through the write-read-revise cycle several times, to make sure it is clear, concise, complete, well organized, and grammatically and factually correct. Read the materials in this week's resources column on general writing guidelines.  You must turn in two drafts: First, an iinitial draft which is fairly complete, and a second draft, with track changes on, in which you've tried to improve on clarity, grammar, spelling, and formatting. You must have track changes on between drafts.

If there is a tradeoff between finising the digitizing, and skimping on the report introduction, polish the introduction first.

By next Monday, before the start of class, turn in: 

1. A pdf map of the tree crowns, colored by TreeType, with an image background, and your trees practice area boundary, and the usual map elements (title, name, legend, scalebar, etc.). It is o.k. on this map if your trees overlap with buildings.
2. A compressed (zipped or archived) geodatabase containing your trees layer.
3. Two versions of a draft introduction for your final report, and initial version, and a revised version, with track changes on.

• You should use a 11 or 12 point font,
• Have 1" margins all around,
• Two to five pages, not counting figures, 1.5 line spacing, section and subsection titles on their own line (e.g., "Introduction"  or "Study Area") and you should bold, italicise, and increase font size to set off titles if you wish. 
• All figures should be numbered sequentially, e.g., Figure 1, Figure 2, have a descriptive caption below the figure, and be referenced in the narrative. You should at least have a study area figure, with the analysis watershed outlines and an image background.
• All tables should also be numbered sequentially, and have descriptive headings or captions. See the reports in the resources columns for examples to emulate. 
• Maps should have north arrows, scale bars, a descriptive title, and legend, in fonts that are no smaller than 9 point when displayed on a letter-sized page.

Week 2 Slides, digitizing,  writing examples

Video here on using the course data - Do not access the course L: drive from within ArcGIS. ALWAYS copy data to the virtual computer (desktop is easiest) and then work with the data on the virtual machine local drive. Then copy any new or changed data back to your directory on the L: drive, or to your own computer, or a USB drive or other "permanent" storage before you exit the virtual Citrix computer. If you load directly from the L drive into ArcGIS, you will likely lose data, time, and work.

Editing, see the intro videos provided in Week 1 on creating a geodatabase and digitizing points and polygons, and these below, excerpted and modified from FNRM3131 for more efficient digitizing:

• polygon snapping 
• split polygons
• merge polygons
• autocomplete polygons
• clip digitizing
• using transparent and hatch fills
• adding a field to a table
• manually selecting and assigning a field value into a table
• deleting fields

Additional information for your report: 

Read the "Guidelines for Your Report" and the "Writing Hints" pdfs. 

Additional tools that might be useful for your report:

• MS Windows Snip image from screen for figures 
• ArcGIS Pro Export image from data frame for figures
• Using Draw.io to create a flowchart of your workflow

Examples of professionally written reports - good examples of proper tone, grammer, and level of detail:

1: Browns Creek
2: NPS Pollinator Survey
3: NPS Kettle Ponds
4: Dall Sheep Survey
5: Grand Sable Netting
 

You should create a study area figure in your draft introduction, using the full watershed outlines in the Practice Areas Geodatabase provided in the L: drive, in the Data subdirectory.

Monday mandatory Class Zoom Meeting Room
Week 3 - Project Geodatabase Creation, Digitizing, Topology 

Create a Geodatabase with topology for the north campus practice area, containing the layers listed below. Review topology editing/digitizing in ArcGIS pro as needed (videos on right). 

Import the LandBuild_Dig_Area from the Practice_Areas geodatabase you used last week, into your new geodatabase.

Your geodatabase should have a feature data set that includes:
1) A "Landcover" polygon layer recording surface cover, with a text column named "material" for the surface type that contains two categories, either 1) impervious (e.g., roads, sidewalks, parking lots, paved plazas) or 2) pervious (grass, flower beds, non-compacted dirt, forested areas, shrubs). 

2) A "Buildings" polygon layer to hold all building footprints for the landcover practice area, with an attribute for the name (text), and roof type (with flat or pitched values).

3) A "Tree Canopy" data layer, imported from last week's digitizing.  Import those data into your geodatabase if you digitized them as a shapefile, or from last week's geodatabase.

4) The LandBuild_Dig_Area layer encompassing your landcover/building "practice" study area - this is imported from the Practice_Areas gdb.

5) Layer topology, with, topological restrictions as:

• Land Cover and Buildings must not overlap,
• Land Cover polygons must not have gaps (except the holes created by Buildings) 
• Trees and buildings must not overlap - remove the tree canopy where it overlaps buildings
• All layers must be contained within your Study Area Boundary

Start Digitizing Landcover in the "Landcover/Buildings" Practice Area. Digitize landcover for at least 50% of your practice area, and all buildings for that same 50% area, and test the topology rules. Use primarily the 2017 images, but use other images as needed (e.g., WMS images, other years' campus images, as available)  to locate building edges as best you can.

By next Monday before the start of class, turn in:

1) Your geodatabase with your digitized trees layer, the empty layers listed in the assignments column, and completely specified topology rules.

Remember to use our standard naming convention (initials on the front, week on the back of the name), and zip up your data into an archive.

2) You should also turn in a pdf map that shows at least 50% your landcover and 50% of your buildings digitized, with the usual map elements.

As noted above, you should create maps in a form that will be easily integrated into your written report. Think about the size and shape of the map when used as a figure, and font sizes, symbology, and the arrangement of map elements that will be readable when scaled to a page. Generally, you want fonts that are no smaller than 10 points in the figures, so if you resize the image, you need to take this into account. The most common mistake is shrinking an image to fit on a page and thereby reducing fonts to a 4 or 5 point size, which renders them useless.

Week 3 Slides, geodatabases, Catalog tips, digitizing and topology 

Video here on adding databases and folders to the ArcGIS Catalog to speed your work

Below are more videos on topology, the first a bit repetitive from what's been provided, but 2nd and third videos below help with the actual mechanics of creating topology and fixing topological errors in ArcGIS: 

-Topology concepts, 
-Create topology, 
-Fixing topological errors 

Your rules specify that the buildings layer must not overlap with trees. You can enforce this with the Erase tool, by first digitizing all the buildings, then erasing the trees with the building footprints. 

See this video and read the documentation on the Erase tool.

Monday Optional Class Zoom Meeting Room 
Week 4 - Finish Digitizing, Cleaning, and Adding Attributes To Your Geodatabase;  Start Methods Section of Your Report

Complete buildings and landcover  - finish digitizing the landcover and building layers.

Complete cleaning the topology for the practice areas, verify and fix topology for landcover/buildings, keeping the data logically consistent across layers, and complete the building roof type and landcover type and maximum infiltration attributes. Your data should be topologically "clean," with all "real" errors fixed, before you turn it in, but if it isn't, turn in progress by the deadline. 

Create a map showing your landcover data layer, and display the "clean" topology, after checking for and fixing any topological errors. Be sure to include the topology and participating data layers in your PDF map, and that only the "faux" errors appear in your topology.

Create a map of your canopy and buildings for the "landcover" practice area, with an image background. The canopy should be colored by type (conifers one color, deciduous trees another), and buildings should be colored by roof type (flat roofs one color, pitched roofs another). 

Note that the canopy may not overlap buildings, so you should use the ArcGIS Erase tool or other methods to remove any parts of trees that overlap buildings. Color your trees and building layers with semi-transparent fills so that it is apparent they don't overlap.

Start on the Methods section for your final report,  describing data creation.  This should be at least two pages on digitizing/topology creation, and at least two pages and up to six pages on analysis you will be doing (but write in the past tense, as if you've already done it). The page estimates assume a line spacing or 1.5, and do not count figures, which will add four or five pages. You should have a fairly concrete, clear description of the data development you've completed (the vector layers and topology).  Your description of the data as yet to be completed (soils and watersheds) will be somewhat vague, and you will improve it in the next draft.

In addition, you should include figures containing digitized layers, one for the landcover, one for soils, one combining buildings and canopy, and one for topology. You may use the maps you've already produced for previous assignments, as appropriate, but your methods section should describe these are examples; don't refer to your practice area as the entire study area, otherwise you will have to rewrite these sections or redo these figures later (remember, we'll be doing an analysis for a larger, two-watershed study area as the focus of our report, and want to be able to use these graphics in that final report).

Note there is a second part of the methods required here, the analysis part you've yet to do.  I want you to think about how you'll do this, and write at least two pages, and hopefully more, of your initial thoughts. In order to write this section, you will have to refer back to the project description provided in the first week, and the data development activities over the next two weeks so that you can describe the overall project methods.

You need to include a general description of the analysis, how you will put together the data to reach your goals. This will be perhaps a bit difficult now, in that you may not know what process you will follow, but you have to include some description of your overall analysis to arrive at an answer, even if it is in the broadest of language and not described in great detail. The point is to get you to review the project goals now, and think about how you will process data to reach them.  Re-read the project description, and include a written and/or graphic description of your general processing steps for your analysis, that is, the guide you'll follow after your data are complete.

Next Monday, before the start of class turn in:

1. Your completed geodatabase, that includes buildings, tree canopy, landcover, and topology. Do not include the images in the geodatabase, and remember to zip it or create a Mac archive before submission.
2.  A pdf map with landcover and your clean topology, with all errors fixed. Include the topology in the legend and on the map.

1.  A PDF map of the tree canopy and buildings.

Observe the usual map requirements.

You should also start on the methods section of your report, due in two weeks (Monday of week 6). 

Methods, Data Development & Analysis Overview slides.

Short lecture on Workflow, to help with writing methods section, and a brief description of a "big picture" flowchart

Using Draw.io to create a flowchart

Monday Mandatory Class Zoom Meeting Room

Week 5 - Complete 2nd Report Draft, Create Watersheds, Flowlines

2nd report draft:

• Edit/improve your introduction according to feedback you received on your last draft, and 
• Finish your draft of the Methods section for your report. Your methods section should include a description of your data development methods, and a broad, generalized description, and include a flowchart, of how you are going to do your analysis. You don't need too many details in this broad description, but I want you to start thinking about this now, so that you can quickly begin work when we start the analysis.

As with the introduction, you need to turn in both an initial draft and a polished draft

Create watersheds and Flowlines
Your goal is to create watersheds for our study area, and flowpaths, the surface paths along which water is expected to flow based on topography. Your watershed boundaries should look approximately like the map linked here. Pourpoints and a DEM are in the "Sheds" geodatabase in the class L drive, in the Data directory.  Retrieve this, and calculate the watersheds with the two vector pourpoints as outlets, using the provided DEM. The DEM and pourpoints are in the Sheds.gdb geodatabase in the Data directory on the class L drive. Note that the watersheds go outside the bounds of the previous PracticeAreas polygon, do not clip your results to the LandBuild_Dig or Trees_Dig areas, we're transitioning to a larger area. Your output will be vector polygons of the watersheds, and vector flowlines from the flow accumulation layer, with somewhere between a 75 and 300 sq-m threshold. 

We provide a figure of the approximate flowchart here. Note that important steps and branches are missing, but you may use this as a starting point.

YOU DON'T NEED TO DO THE DEM FILL STEP WHILE CALCULATING WATERSHEDS in this exercise, the DEM has already been conditioned for you.

Creating watersheds was covered in FNRM3131/5131, but many of you will not have taken that course, or it has been a while, so we provide links to instructions. See the videos and pdfs in this week's resources section, to the far right.

The watershed tool sometimes returns incomplete watersheds. The D8 algorithm only allows water to drain to one of eight directions, and in nearly flat terrain you can get divergent flow where you shouldn't. Sometimes this results in a cell near your flow channel with a slightly off direction, creating a second watershed upstream that doesn't drain into your pourpoint. 

Your watersheds should approximately match that of the map shown linked a few paragraphs up. If not, perhaps use a different pourpoint distance, or manually move the pourpoints to be closer to the initial flowlines.

The flowpaths (also called flowlines) might not intersect the provided pourpoints, but only run near.  You should use the SNAP POURPOINT function, with a snap distance of less than 5 meters, for this exercise.  

Create a pdf map of your watersheds, with a suitable image background, flowlines, and the usual legend, north arrow, etc.  Your watersheds should have an extent similar to those in the figure here.

Create a detailed flowchart that describes your specific watershed workflow. This flowchart should include each step, the names of the specific ArcGIS tools you used, and the descriptive names of the output layers.  Any key parameters should also be noted on the flowchart (e.g., snap pour point distance). Export as a pdf, and turn in on Canvas.

Next Monday, before the start of class turn in:

1. A pdf map of watersheds, with flowlines,
2. A flowchart of your process, and,
3. Your report 2nd draft.

Important! As noted in the more detailed materials provided earlier, DO NOT write your methods as an overtly sequential description, and DO NOT frequently refer to specific ArcGIS Pro tools. As in the example reports provided in the Week 2 resources column,  professional reports rarely include this level of detail or form in their methods. 

Watershed Slides

Most of you should have had exposure to the concept of watershed delineation based on a raster DEM.  Here is an excerpt from the "GIS Fundamentals" textbook explaining the general idea, for review: Watershed Background

Below are a video and instructions excerpted from FNRM3131/5131.  The students in that class are provided a "well-behaved" DEM, that has been pre-processed to work. You'll need to substitute your different input files and other parameters, but these provide an example of the workflow in ArcGIS Pro.

Creating watershed instructions, excerpted from 3131/5131, here using the tools:

• Flow direction
• Flow accumulation
• Snap pour points (snap distance less than 5 m)
• Watersheds

You DON'T NEED TO USE the FILL function on your DEM, included in some of the descriptive materials. The DEM has already been filled for you.  

Video for Watershed Creation

Video for creating vector flowlines from flow accumulation layer. This may require use of the tools

• Raster Reclassify
• Raster to polylines

Mandatory meeting on Monday, Class Zoom Meeting Room

Week 6 -   Work Flows and Model Builder
Introduction to Model Builder
This week we'll also redo the watershed delineation exercise, but this time to more explicitly demonstrate the use of Model Builder, a tool for codifying workflows. Model builder perhaps isn't worth the bother for small spatial problems you'll be doing only once, but is useful for complex spatial workflows, and for simpler or complex workflows you'll be repeating with different data sets or parameters.

Look at the example video, the Model Builder Quick Tour, Getting Started, and other resources from ESRI, linked in the right column.

Start a new project, copy in the campus DEM and the pourpoint file you used last week, found in the course L drive, data directory, Sheds.gdb. Build a processing model for watershed delineation using the ArcGIS model builder tool. You basically do the steps you followed for last week's exercise, but write them into a model that can conduct all the steps in sequence.

Note that you should make the input and output data layers, and the snap pourpoint distance as parameters, entered at run time.

The model is saved links to your project, so you need to turn in both your project and your geodatabase.  You can do this by exporting a Project Package, as shown in the video to the right. If you have problems creating a package and the deadline is nearing, use the snipping tool to capture the graphic of your Model Builder model, embed that in a Word or similar document, convert to PDF, and submit that as a place holder, and compress the geodatabase and submit that with the model figure. Contact one of the instructors to work out why you couldn't create the Project Package, as you'll need to do that in future exercises.

NOTE: The Project Package tool seems to have stopped working, or at least be working intermittently, perhaps due to a software update. It worked during testing in August/early September, but at least three students have not been able to create a package, with various error messages.

As an alternative, you may zip up the containing folder, as shown in the video "Zipping all project files" at the bottom of this week's resources column.

Next Monday, before class,
turn in:

1. A pdf map of your watersheds, that includes the watershed boundaries, stream lines, and an image background,
   
   
2. Your ArcGIS Model Builder model and data,
   a) either by zipping the project files (a .gdb, a .tbx, a .aprx, and an index, all within the enclosing directory automatically created by Arc when you start a new project), or
   b) exported as a project package, which hasn't worked for most students yet. Make sure to include your flow direction, flow accumulation, output watersheds, flowlines, and the model builder model.

Model Builder Introductory Slides.

A simple Model Builder example video

Model Builder resources from ESRI:

-Quick Tour
-Getting Started Building Models
-Model Builder 101
-Model Builder 202 (this is for an earlier Arc version, but much still applies)

If you have time, you can investigate the ESRI Geoprocessing courses, starting here.
Note that you have to sign in through the UMN account, using your X500 name and login, for access.

Finally, since your model builder lives inside a project, and you should turn in both your data and the model.

The preferred way to share your work is by creating something called an export package as shown in the video below, but there is currently a problem as noted above in the assignment:

Exporting a Project Package to submit to Canvas

Unfortunately, it appears this option is now working intermittently, so another option which should work involves zipping the containing folder, including the toolbox files (.tbx), index, geodatabase, and project file:

Zipping all project files to submit to Canvas

Mandatory meeting on Monday and Wednesday this week, Class Zoom Meeting Room
Week 7 - Project Roadmap, Flowcharting, Project-Wide Data

Monday and Wednesday in Zoom (Class) - we'll discuss the overall project, analysis, and output. 

1) Re-read the project description, review the general flowchart, create a detailed flowchart for interception sub-workflow.
Carefully re-read the semester project description provided in the first week, and develop a draft flowchart of your analysis. This will likely be much more complete and refined than what you included in your methods draft. You need to think about the spatial operations and order in which they'll be applied, represented by a box and arrows diagram you'll apply. Come to our meeting on Monday prepared to ask questions about anything that's not clear.

We provide a basic flowchart in the resources column to the right, also embedded within the flowcharting exercise below. We will discuss an example flowchart, and on Wednesday provide a refresher on some analysis tools that you'll likely use to solve our problem. 

This flowchart is missing some details. This week you will provide the details for the canopy interception sub-workflow, a small branch of the overall flowchart, you will expand and add details, and verify your sub-workflow is valid by applying it.  Follow the instructions in the Canopy Interception/Net Rainfall Flowcharting exercise here

2) Get Acquainted and Condition the Project-Wide Data
We've provided campus-wide data very similar to those you have been developing, plus some more. All work from here forward will be using these campus-wide data, and NOT those that you have developed for the North Campus Practice area, earlier.

The data are available on the class L drive, in the Data directory, in the WholeProjectData geodatabase. You will have to modify the data layers a bit for your analysis, e.g., modify or add attributes to change units from cm to meters in the soils data, or add a maximum canopy interception attribute, but these are relatively minor.

Layers and important attributes are:
Soils: Type, a text description of the dominant texture, and MaxInfl, the maximum infiltration rate in centimeters (not meters, remember 100 cm to a meter)
SurfacePermeability: Type, indicating if the landcover surface is permeable or impermeable
TreeCanopy: Type, value Dec indicating deciduous trees, value Con indicating coniferous; CanopyHeig, indicating short or tall trees. You will have to add maximum interception/absorption rates for each height/type combination.
Buildings: Roof, values pitched or FLAT (not green roofs on pitched surfaces)
Project_Study_Area: a bounding area, no attributes of note
Pourpoint2020: ppid, numeric identifiers for the watershed outlets 
ProjectDEM: a digital elevation model, meter elevation, 1 m cell size, covering the study area

Note that there is not a watersheds layer, you will have to create that, either manually or using your Model Builder tool.

There are also no rainfall layers, which you'll have to create.

Wednesday - Analysis and Tools
We'll review these tools in class, and describe how we might use them in a workflow to solve your primary analysis:

• Clip function - for data prep
• Union (in addition to last week)
• Dissolve, often used after Union
• Polygon to raster
• Raster calculator

Before the start of class on Monday, next week

• Turn in your flowchart exercise, the first draft of your detailed flowchart, as a pdf.
• Turn in a net interception map, CInt in the example flowchart.  Note that you must show areas where there are no trees with a value of zero interception.

• Turn in a net rainfall map, that is, a map of rainfall minus canopy interception. This is the layer labeled RAS that result from applying your sub-workflow in the flowcharting exercise. Display/color it by net rainfall values, in meters, for the extent of the study area.

Week's slides 

Tool Review slides

Example flowchart graphic (vector analysis, then raster)

Selecting and calculating fields for vector layers

Video refreshers on common tools (from FNRM3131)
-Clip & Intersect tools
-Dissolve function
-Union note that the union function often creates multipart shapes, so you'll need to do the multi-part to single part to separate them before further analysis
-Multipart to singlepart
-Raster Calculator 

You should use the search function as described in the Finding tools in ArcGIS Pro video and search the online ArcGIS Pro documentation, to understand and then apply the:

-Polygon to Raster tool, to convert vector features to rasters, prior to using the raster calculator

Mandatory meeting on Monday, Class Zoom Meeting Room

Week 8 -  Maximum surface absorption/infiltration layer creation

Surface absorption layer: Calculate maximum surface absorption, and create a flowchart of your process that combines the landcover, buildings, and soils layer to create this maximum surface absorption layer. This layer will have a value of zero for the impervious landcovers (roads, sidewalks) and a zero for buildings, and the maximum soil infiltration layer for the remaining locations. 

Students in 4295 ARE NOT REQUIRED to make a Model Builder model for this workflow. Students in 5295 DO have to eventually make a Model Builder model for this workflow. Don't be confused. Optionally, for extra credit, 4295 students can make a model builder model that calculates your maximum surface absorption layer.

Turn in a pdf map of the maximum infiltration layer, and a flowchart of your sub-workflow to calculate maximum absorption/infiltration, as described in the deadline column to the right

By next Monday, before class, turn in:

1. Map of the maximum surface absorption layer you created that infiltration across the surface, colored by m absorption per sq-m surface area, with the usual title, legend, scale bar, and other map elements.
2. Flowchart of your maximum infiltration workflow, as a pdf.

Week's slides:  Calculate Maximum Infiltration Layer Manually, and Net Rainfall in Model Builder (MB)

Videos:
Select table records based on values, manually, or in Model Builder 

An example video of using a code block to shorten the number of steps calculate a field value for a feature, based on other column values for that feature.

Another brief example on else-if python code block, an alternative to multi-step selection, but more complicated

Short video on a strategy to use when MB doesn't display an input file in a subsequent tool in a workflow.

Class Zoom Room

Week 9 - More Tools, Calculating Net Runoff for 2.5 cm Rainfall

We'll introduce the general steps needed to create a net runoff layer. This will involve combining the three branches of analysis you've already conducted - net rainfall (after interception), maximum surface absorption, and watersheds. Most of the work will be with tools you already know, but we will introduce a few new tools, needed to aggregate over the watersheds:

• Zonal statistics for Raster
• Summary Statistics for Vector

 You should apply the combined workflow, and calculate net runoff for your two watersheds, based on current surface conditions and a 2.5 cm storm.

 By next Monday, before class, turn in two maps and a spreadsheet:

1. Map that includes which shows the net runoff layer for a 2.5 cm rainfall event for each watershed.  The net rainfall is the layer that comes from rainfall, minus interception, minus surface absorption, with negative values set to zero. Include watershed boundaries on the map. 
2. A spreadsheet or pdf table with a row for each watershed, with columns of your average interception per watershed in cm, average maximum infiltration in cm, and summed aggregate runoff, in cubic meters, for each of the target watersheds.

Week's slides, Reclassification and Summarize by Zones to table, here

Example here on how to summarize or calculate other zonal statistics across a  layer to a table.

2nd example here of how to summarize across a vector polygon layer/table.

Long(ish) video here with a general hint at how to structure your geodatabases for easier debugging in Model Builder (MB),  and I walk through a MB model that roughly follows the earlier provided general flowchart.

Video here on stepping through a MB model, running a single tool at a time, while debugging.

Class Zoom Room

Week 10 - Calculate Runoff for 5 cm Rainfall, start Report Draft 3

Apply your analysis workflow to estimate runoff for the 5 cm rainfall event, analogous to what you produced last week, just at a higher rain level. Create maps of 1) interception and 2) runoff, and a table, produce pdfs, and turn these in.

Start report draft 3. Your report draft 3 should include revisions of your introduction and methods, and a new results section describing your net runoff results for the 2.5cm and 5cm rainfall-runoff for conditions as they are now. The results section should be 2 to 4 pages, not counting figures.

Note that the full report draft 3 is due in 2 weeks, but we have you turn in parts each of the next week. 

Next week you turn in maps of the 5 cm net interception and of net runoff.  You'll need these for your full analysis. Also, turn in a table showing net runoff summed for your watersheds - these will all be included in your report draft.

Report draft 3 is due in two weeks, and should include:

1) A revised introduction and methods sections, based on previous grading/comments. You should include  two flowcharts,  
- a detailed flowchart of your current-condition analysis workflow (before landscape modification/mitigation)  in an appendix, not in your methods section, and
- a  generalized flowchart of your workflow above, in your methods section, showing the main branches of analysis.

The detailed flowchart is each step, with the names of the ArcGIS tools and all your intermediate layers. Be sure to keep the fonts large enough to easily read. You will likely have to break the flowchart up, into several sections, on separate pages.

The generalized flowchart is the main branches, combined to their general topics and actions, e.g., a branch to develop net rainfall inputs, a branch to develop surface permeabilities, a branch for watersheds, etc. This will fit on one page at most, and is appropriate for the body of the report, in your methods section. It should be even more generalized than the example flowchart we provided for you earlier.

2)  A results section describing your findings for the base case rainfall for both 2.5 and 5 cm rainfall events.  The results section should include the runoff volumes in the format specified in the project description.

Your results section should also include maps of your estimated runoff under 2.5 and 5 cm rainfall.

Remember that the target audience is the set of administrators in charge of the University's physical environment and stormwater management. They don't know anything about GIS software, or GIS theory, or any jargon on geospatial analysis terms. You need to provide enough detail so that an intelligent non-expert will understand what you did, and be convinced that your methods are sound and your recommendations are valid.

By Monday, turn in, as pdfs: 

1) Map of 5 cm net rainfall (note, this is a late change, if you've already turned in net interception, or planned on it, either will be acceptable),

2) Map of net 5 cm storm runoff; maps should be appropriately symbolized across the range of the main layer,

and

3a) Either a table showing net runoff, in cubic meters of water, by watershed for each of the 2.5 and 5 cm storms,
or,

3b) annotation on the runoff map showing the 2.5 and 5 cm runoff amounts, clearly labeled.

The 5 cm analysis is just a repeat of the workflow you did with the 2.5 cm rainfall event, except now you are using a 5 cm rainfall event.

Mandatory meeting on Monday, Class Zoom Meeting Room

Week 11 -  Discuss and implement strategies for mitigation, create "new" forest canopy, your first version of the analysis, and results section of your report third draft.

We can mitigate by adding forest canopy, changing impervious surfaces to pervious surfaces, adding sinks, converting roofs to green roofs, and adding underground storage.  We will discuss general strategies for adding mitigation, and walk through one workflow for canopy addition. Your tasks will be:

1) Create a new forest canopy layer. Digitize points, buffer, clip/union with the original trees layer combine to create a modified canopy layer that is part of your 2.5 cm mitigation recommendations, and use this to calculate both a new net rainfall interception layer, and the cost of adding new canopy. 

Remember, your new canopy needs to be combined with the existing canopy, and cannot overlap with buildings, so you'll have to union/combine your canopy generated by buffering points with the existing digitized canopy, and clip the canopy layer with buildings.  

You need to calculate costs for the new canopy, and you have to pay for the total new canopy planted, even if it overlaps with the existing canopy. Hence your cost is based on the total new canopy area from your buffering, before you union it with existing canopy or clip it with buildings.

Create a pdf map that shows the combined canopy layer (old and new), highlighting what you added/changed compared to the old layer (you can do this by displaying the old layer on top of the new combined layer, using different shades). Include text listing the original canopy area, the new canopy area, and the new canopy cost, by watershed, somewhere on the map.

2) First version of results section for report draft 3 - you should turn in your first draft of the results for the "as is" conditions, 2.5 cm and 5 cm rainfall (no mitigations changes).  This is your initial draft, you will turn in a self-revised draft next week (no feedback by instructors between these two drafts). Note that this version DOES NOT HAVE TO INCLUDE revisions of the Introduction and Methods sections, but the version you turn in next week SHOULD BE COMPLETE, AND INCLUDE REVISIONS TO ALL PARTS.

Before class on Monday next week, turn in:

1. A pdf map showing your old and new canopy layers towards mitigation of a 2.5 cm rainfall events.
2. Your first attempt at a results section for your third report draft - this is your first attempt, you will self-revise and turn in a second copy next week, and you won't get feedback on this first submission before next week. My purpose here is to spur you to write a draft of the results, then seriously revise your own draft after writing the first.

Week's Slides, Buffering and strategies for "Creating" new canopy, and calculating rainfall mitigation needs.

Video here of one workflow and MB model to add tree canopy

Document describing strategies and examples for mitigation planning

Week 12 - Thanksgiving, and begin mitigation rain gardens and impervious surface

There are four primary ways besides increase canopy we'll use to mitigate runoff: convert impervious to pervious surfaces, rain gardens, green roofs, and underground storage. You should start work on rain gardens and pervious pavement before leaving for Thanksgiving, and continue after returning. Note that the rain gardens and impervious surface layers and calculations should be completed by the Wednesday after Thanksgiving, so you should work on some early this week and early the week after Thanksgiving.

1) Adding rain gardens. These should be placed near flowlines, in parking lots or grassy/flat areas, but subject to restrictions noted in the detailed project description document (e.g., near existing flowlines, not on roads or sidewalks, not consuming too much of parking lots, and others).  You should examine and place rain gardens, starting with large flat grassy areas, parking lots, and plazas. 

There are two complications with rain gardens.  First, they need to be near flowlines, so you need to place them accordingly. Second, they are limited to a 0.5 meter depth, so you need to make sure you don't over or under size them, and give them too much or too little credit for water mitigation. If you make the rain garden too big for the upstream area, it won't fill up for the 2.5 cm storm, and so will be too costly.  If it is too small, it won't absorb all the upstream water. Rather than go through many analysis iterations, your best approach is to estimate runoff based on average runoff conditions and upstream area, and make a rain garden large enough to capture that amount, assuming a 0.5m deep rain garden.  You can then verify the amount captured during a run, and adjust the calculations as need be.

2) Converting impervious surface to pervious surface. You'll have an estimate of how much rain gardens take up from the above analysis, so have an idea of how much additional volume you'll need to reduce, by subtracting the rain garden storage from your "as is" conditions. This will give you an estimate of the amount of additional water you need to process on the landscape. You can then calculate how much area in impervous surface this will correspond to, given the average soil maximum infiltration rates.  You then can make of copy of your original impervious surface, and split new polygons from existing features  (e.g., roads, sidewalks, plazas) and make them pervious. 

You will have to calculate the costs of impervious to pervious conversion, by first calculating the area of converted polygons, and multiplying by the cost per area listed in the project description.

Turn in the Wednesday after Thanksgiving, before Class: 

Turn in the following maps for the 2.5 cm mitigations: 

1) a map of rain gardens, overlain by the flowpath vectors, a hollow fill watershed layer, and with an image background, and

2) a map of converted pavement, from impervious to pervious, overlain by the flowpath vectors, an outline watershed layer, and with an image background.

You don't need to calculate areas, costs, or other attributes, I just want to see some progress on this mitigation work. 

3) Your final report draft three, with revised introductions, methods, and a results sections.

Week's slides here

Video on strategies for adding rain gardens, balancing capacity to runoff, and downstream accumulation of overflow from upstream rain gardens.

Video on strategies for converting impervious surfaces to pervious surfaces

Converting impervious surfaces to pervious surfaces will likely require splitting polygons, often splitting pieces and then merging, covered in general videos here:

• split polygons
• merge polygons

Remember to assign new table values reflecting the switch from impervious to pervious

Class Zoom Room

Week 13 -  Final mitigations and Final Runs

Add green roofs. You may still have positive runoff after adding rain gardens and pervious surface, so you may need to add green roofs.  Any flat roof can be converted to a green roof, they will absorb all the rainfall of a 2.5" storm.  This is an expensive way to reduce runoff, and is limited to just the roofs, but is a less expensive alternative than underground storage. 

The amount of runoff reduction is easy to calculate - the area of the roof in square meters, multiplied by 2.5 cm (or 0.025 meters), for cubic meters of outflow reduction.  You don't need to do another full application of your geoprocessing workflow, because of our assumptions of no absorption of overland flow. You can calculate the rainfall volume that falls on any converted roof, and subtract that amount from the runoff calculated with your rain gardens and pervious surface present.

You must calculate the cost, on a per surface area basis for converted rooftops.

Underground Storage. After subtracting the reduction in runoff due to green roofs, you may still have a positive outflow. If that is the case, then you can simply designate an amount of underground storage.  Again, you don't need to re-run your workflow, you can simple designate the amount of underground storage as that left over after everything else, and calculate your 

Turn in by Monday, next week, before class, at the start of class:

1) map showing the rain gardens, with flowlines and watershed boundaries (not filled), and an image background, with a table showing the surface area of the rain garden in sq. m., the volume of water stored for each rain garden in cubic m., and the cost of the rain garden

2) a map showing new pervious surfaces, with watershed boundaries (not filled) and an image background, and an annotation or table showing the total area converted to pervious surface, and the total cost.

3) Data tables for your reports, as specified in the report guidelines and problem statement. You should be able to nearly completely fill these out, but whatever your progress, complete them as much as you can, and turn them in as a pdf.

Class Zoom Room

Week 14 - Continue Analysis Under Modifications

Project work, modifications and analysis to mitigate runoff for prescribed rainfall amounts

• Not a dang thing

Class Zoom Room

Week 15 - Work on final analysis and final report

No class final; all assignments due by Wednesday, December 23, 11:59 pm

You should put your final geodatabases on the L:drive, the 4294 share directory, in a directory with your name and the word "final" as part of the directory name. You should also submit a note to the class Canvas site for the final data, letting me know you have transferred the data to the L drive.

You should turn in your final report on the class Canvas site, in the final report section.

You should turn in three final geodatabases to the L drive:

1) A geodatabase with the word "Current" in the name that contains data used for calculating your "current condition" runoff for 2.5 cm and 5 cm storms. These are the data you used at the start of your workflow, that is, after you added columns or modified the base data we gave you, but before you started your runoff workflow.  Don't include intermediate layers. The geodatabase should include:

• your final "runoff" layers, that is, the layers that you summed by watershed to get the total runoff volume for the study areas, 
•  watersheds,
• flowpath layer
• canopy
• impervious/pervious surface
• buildings
• rainfall layers 

2) A geodatabase with "2p5cm" in the name, which includes the modified layers used to obtain your 2.5 cm mitigation of runoff. You should include the modified layers for a 2.5 cm storm only, not the original layers (these are in the database above). Turn in the following:

• your final "runoff" layers, that is, the layers that you summed by watershed to get the total runoff volume for the study areas, 
•  watersheds, modified with your new rain garden pour points
• flowpath layer
• modified canopy layer
• modified impervious/pervious surface
• modified buildings, with roof attribute values changed to identify buildings with green roofs
• rain garden layer

3) a geodatabase with the "5cm" in the name, including all layers similar to those above, but for your 5 cm rainfall mitigations.

Your final report should include the information provided in previous drafts, plus a description/discussion of the changes for reducing runoff to zero under the different rainfall amounts. This should include appropriate maps/figures.

Wednesday, Dec. 23rd by 11:59 p.m., turn in your final report, via Canvas, and your copy your final data to the 4295Share\yournamefinaldata subdirectory