ESPM4295

Read and understand the Student Code of Conduct and Academic Integrity

In class, sign up for one of the project areas on the St. Paul Campus (sheet will be passed around during class period)

Visit your study site,  reconoiter the boundaries and interior, and identify a few stormwater grates, 

Activate the MNGEO web mapping service (WMS) in ArcMap (see video "Using WMS" in the Resources column, to right, inspect campus area for NGS points on Larpenteur, and the Campus project area you were assigned. The address for the WMS is http://www.mngeo.state.mn.us/chouse/wms/wms_image_server_specs.html, but you should read the general MNGEO WMS specification and site for a description of the images. We're in Ramsey County.

Read the Semester Project Description, to get an idea of what we're heading for over the next 15 weeks, and look at the Rainfall Mitigation website, for additional motivating information.

All assignments due on Moodle, unless specified otherwise

In class on Sept 8
1) Sign up for Study Area

Course files are on the L:\ drive accessible from the computers in 35 Skok Hall, under the CFANS\LABS\ESPM4295W directory. Below we'll refer to the subdirectory as CLD for "Course Lab Drive"

How to access lab files via VPN

How to Zip/Unzip

Using WMS

2017 Area Assignment Map

Measureing Image Accuracy

NSSDA Overview Slides

Do the assignment linked here: NSSDA accuracy exercise

We'll be collecting data by digitizing from aerial photographs. Your goal is to assess the geometric accuracy of source images.

Display and view the two sets of photos available for this class on the class L drive, in the CampusImages folder. There are two UAV/drone based photos, CampusNov2015 and CampusNov2016, and a HiRez_## set of images, from 2006. We provide all of these because buildings lean in different directions on different photos, and the November images still have some trees with leaf on, so different sets of ground features are obscured on the different images. You have a better chance of seeing all features if you look across several images. Beware, the surface may have changed between 2006 and 2017, so interpret carefully.

As described in the NSSDA instructions linked above, we've provided a shapefile with a set of points that have been accurately measured with GPS.  You need to match the GPS coordinates to the image locations of these well-defined features, extract the image coordinate values, and enter these into the NSSDA spreadsheet. You'll assume the GPS are true, and calculate the accuracy for each feature.

After collecting 15-20 points, fill in the NSSDA spreadsheets (blank downloadable here, and handbook here, and on lab drive under Skills/GPS/NSSDA),to turn in next week.  You will have three spreadsheets, one for each set of images. NOTE that you should check the calculated RMSE values, because of the way the spreadsheet is written, the calculations may be incorrect depending on how you enter your data. This is a good example of needing to clearly understand the downloaded utility. If your RMSE is calculated incorrectly, you will not receive full score for this exercise.

You should also read the "General Guidelines for Writing" and the "Writing Hints" pdfs in the CLD\Writing directory, as you'll be submitting several reports in this class. Your first report will include your NSSDA accuracy exercise, along with the stormsewer grate inventory you'll be doing next week.

Nothing due this week 

Link: Description and shapefile of GPS points for NSSDA accuracy assessment

Template Spreadsheet in CLD\NSSDA

CLD\CampusImages contains 5 to 15cm resolution aerial photographs

Review materials:

• Digitzing Refresher,
• Extracting XY from shapefile points (for NSSDA calculations)

Integrating Image and GPS collected data

Image Interpretation: Create Storm Drain Data Layer

You often have access to orthographic aerial photographs across a range of resolutions and dates. For example, the USGS provides high resolution aerial photography for built-up areas, and the USDA takes annual 1m resolution color and sometimes color infrared photographs for most of the country. This week you'll collect data for your on campus study area from aerial photographs.

You need to digitize a point layer of the storm sewer drains for your study area. 

You should review the digitizing refresher and videos, in the Resources column to the right, if you're at all rusty.

You should start in the lab, digitizing on-screen, and then verify your sewer grates by visiting all or most of them in the field.  

Note that the HiRez_## photographs you will use do not have prj files, and are in the Minnesota County Coordinate System for Ramsey County. All of your output and most other data, including the CampusNov201x images are in UTM zone 15 NAD83(CORS96) coordinates. You will need to proect the HiRez images for your study area to the CORS96 coordinates.  If you don't remember how, see the projection videos to the right.

You will create a map of your project areas with one of the image as a background, and your point layes, and an approriate legend, title, and other map elements. Turn this in on Moodle before class next Monday, along with your digitized points.

Monday 10:30 a.m. turn in before the start of class:
The three NSSDA spreadsheets, one for each of the images

Storm Sewer Catch Basins and lines, file for download here

CLD\CampusImages contains 5 to 15cm resolution aerial photographs

CLD\Skills\DigitTopology

Digitizing: Basic Editing

Data Design
GeoDatabase

Notes on Topology

Geodatabase creation, Digitizing, Topology 

Create a Geodatabase for your project area. Review topology editing/digitizing in ArcMap if you need to (videos on right), and begin digitizing the layers listed below for your project area, interpreting features from the CampusNov201x and HiRez06 images.

1) A "Walking Areas" polygon layer of sidewalks, plazas, grassy, forest, and shrub/flower bed areas, basically all non-road, non-building areas in your project area. Include a text attributes for this layer named "MATERIAL," specifying asphalt, concrete, grass/forb, shrub, soil, or other.

2) A polygon layer of all buildings, with an attribute for the name (text).

3) A point layer of all storm sewer grates. You can start with your work from last week, importing into your geodatabase the points you digitized. 

4) A polygon layer of your tree canopy, attributed (text) as conifer or broad-leaved.

5) If you have any, a layer that includes polygons of stormwater ponds, lakes, raingardens, or other infiltration areas, with a text attribute for type.

Topological restrictions are such that Walking Areas and Buildings must not overlap, and Storm Grates must not be contained in Buildings, and Buildings and Walking Areas must not overlap with Stormwater Ponds.

Begin digitizing this week, making sure to assign all attributes required for the layers. Turn in your geodatabase and topology template on Wednesday at the start of class. Don't worry that you don't have all the data digitized, this is just to ensure that you've created the geodatabase and started digitizing.

Monday10:30 a.m. before the start of class, turn in

• your pdf map,
• first draft of you stormwater grates data layer 

CLD\Skills\DigitTopology
&
CLD\Skills\GeoDB

Editing:
-Snapping, Editing tips, Cut & Split Polygons

Topology:
-Topology concepts, 
Create topology,
-Applying topology, identifying topological errors,
-Correcting errors example 1,
-Fixing topological errors example 2

Complete Mapping

Complete digitizing your sample area, and verify and fix topology, keeping the data logically consistent across layers. 

Print a map showing your data layers, and display the topology for the buildings and walking areas, such that is shows your "clean" topology. That is, run the topology check, and display the results in a map you print.

Monday,
before the start of class, turn a geodatabase that contains progress to date of your Walking areas/buildings/sewer grates Geodatabase

Data Dictionary

Weekly Slides

Drone Pre-Demo

Drone Post-Demo

Skills - Data Dictionary Exercise

Modern GIS data collection includes electronic verification and logging. This week you'll use electronic forms and verify/attribute data. We'll take our database into the field, and verify the surface type and canopy type layers

Wednesday - Attendance mandatory: Review/troubleshoot field data verification, and observe Drone Data Collection Demonstration

• geodtabase containing all digitized layers for your project area, and
• appropriate map(s) displaying the layers

Files in CLD\Skills\DataDictionary

Video intro for Data Dictionaries

LiDAR - Point cloud
reclassify & use 

LiDAR Intro

Skills - Introduction to LiDAR Exercise

LiDAR reclassification and height calculation for your project area

Backup date for drone data collection, if first date doesn't work.

Monday, before class,
turn in 

• geodatabase, and
• a map of your verified data layers

Videos:

• MnTOPO data download
• LasTools in Command Prompt
• Calculate Heights
• Point cloud to raster of canopy extent (sorry, done quickly on request)

This week's tasks require your to do a Week's : Soils data download 

Read the instructions and download soils data for your study area, and condition it so that it is useful for your analysis.

Your second task is to review the flowchart on the right, and fill in missing steps and specific details so that it may serve as a roadmap to your analysis, i.e., the actual name of the ArcMap tools you'll use, and the input data for each tool. Re-read the project description provided in the first week, and then complete the Standard Geodatabase and Flowcharting exercise. This is due the week after next, but don't put it off, it will take quite a bit of thought to complete the flowchart.

You've already developed much of your data, but most, if not all of the remaining date can be created from the high-resolution LiDAR DEMs, augmented by the Campus Photos we've used in this class already.

Monday, before class:
turn in two maps from LiDAR assignment

Background, stormwater mitigation: 
Met. Council Summary

Video: Using Draw.io to create a flowchart

Task - Calculate Watersheds for your Study Areas
Creating watersheds was covered in FNRM3131/5131, taken by most folks in this course, so we won't provide specific step-by-step instructions. There are three files that cover the steps, one an excerpt from the FNRM3131/5131 lab on this, and the other two reports from past student technical assignments, in which they described processing steps.

You might want to run the SINK function in the ArcToolbox->Spatial Analyst->Hydrology tools in ArcMap. This will show the general extent of sinks, and you may then use the inquire cursor over the sink regions to identify the depth of most sinks. This will be useful in setting the depth for the fill process.

You will use the sewer grates as your pour points/outlets for your watersheds. Note that because the DEM contains some error, the flowpaths won't intersect many of your sewer grates, but only run near.  You should create a copy of your true sewer grate locations layer, and modify this layer so the grates fall on the appropriate flowpath.  Make sure you work on a copy.

You'll include these as part of your geodatabase you turn in next Monday before class.

Monday, before class:
Turn in your LiDAR Assignment

Watersheds

1) DEM fill, sinks, flow direction, flow accumulation
2) Snap pour points (grates), Watersheds

Excerpt from Hydro Processing lab in 3131/5131

Proposed Template Geodatabase & Analysis Flowchart

Week's Slides

Monday in Class - we'll discuss the overall project, analysis, and output. 

Begin developing your flowchart of analysis. You need to think about  the spatial operations and order in which they'll be applied, represented by a box and arrows diagram you'll apply. 

Re-read the project description before class, and come prepared to ask questions about anything that's not clear.

NOTE:  We will not be using the data you developed for the analysis, but rather a similar data set for a larger study area.  This saves time in that we don't have to stitch data together, and it allows students that haven't completed their data do start at the same stage going forward, so as to not fall further behind.

Download/access data for entire study area, familiarize yourself with it, and evaluate it for requirements/modifications needed for your analysis. 

Begin needed processing, e.g., you'll have to review watershed boundaries, pour point locations, attributes for the various layers, etc., to make sure they exist for your analysis, or include modifications in your flowchart so that the needed steps are included.

Monday, start of class: turn in completed geodatabase 
Submit the following complete layers: 1) Walking areas, 2) Buildings, 3) Sewer Grates, 4) Tree Canopy, with heights, 5) Soils, 6) Watersheds, 7) Flowapths, and any 8) sinks, ponds, wetlands. 

You should also turn in a PDF map of each layer, with 1) Walking areas, Buildings, and Tree canopy on one map, 2) Soils and on a second map, and 3) Tree canopy, colored by height, with one of the image layers as background, and 4) Watersheds, Flowpaths, and Modified Stormwater Grates.  All maps should have an appropriate legend, scale bar, north arrow, title, and your name.

Note that you will be graded on both the completeness and accuracy of the data in your geodatabase, and your maps.

Make your analysis easier: Add a custom toolbox in ArcMap

Example flowchart

Flowchart description

We'll discuss Report Draft 1 - Introduction, Methods, and Overview figures

Next Wednesday you'll turn in a report draft, according to: Report Draft 1 Requirements

We'll also discuss Model Builder, an ArcGIS tool for automating processing

ModelBuilder description and assignment (read and understand, optional for undergraduates to turn in the assignment)

Complete all data development, modification  (including topology/data sources) and be starting the first stage of your analysis.

Next Monday before the start of class, you should turn in your database complete, and ready for analysis

A refresher on the raster calculator, raster to vector conversion, and some background for the specific problem

Analysis

Work on Report Draft 2 - this should include a revision/improvements of the introduction and data/methods sections including all figures for those sections, a completed, polished section describing your analysis methods for your project, including a graphic of your flowchart, and at least an outline of your results and conclusions. Next Monday, before class, you should turn in a first draft and a revised version, named ???_draft2a and ???_draft2b, where ??? are your name or initials.

Analysis

No class final; all assignments due by Wednesday, December 21, 1:30 pm

Your final geodatabase should include the following layers: 1) watershed boundaries corresponding to each assigned stormwater, and sinks for  areas that don't drain to grates, and the modified versions for your 1/4", 1", and 2" that include proposed canopy, walkable surface, road, building, rain garden, and underground storage location.

Wednesday Dec 21, by 1:30 p.m., turn in your final report, via Moodle, and your copy your final data to the 4295Share\Z_FinalData\yourname subdirectory