Raster Processing with Scipy.ndimage (Dev Meet Up II)
-
Upload
jhasthorpe -
Category
Documents
-
view
1.186 -
download
3
Transcript of Raster Processing with Scipy.ndimage (Dev Meet Up II)
Raster processing with scipy.ndimageHenry [email protected] @om_hennersN
ggggggggggggggiiiiiiiiisss....ssttttttaaccccccccccccccckkkkkkkkkkkkkkeeexxxxxcccccccccccchhhhaaaaaaannnnngggggggggggggggggg
eeee..ccccoooo
mmmmmmmmmmmmmmmmmm
cccccccccccchhhhhhhhhhhhxxxxxxxxxxxxxx aaaaaaaaaaeeeeeee nnnnnnnnnnnkkkkkkkkkkkkkkkkkkkkkk gggggggggggggggggggggggggggggggggccccccccccccccccccccccccaaaaaaaaa eeeeeeettttttttttttt ....sssssss cccccccccc.... oooooooooossssssssiiiiiiiiiiiii mmmmmmmmmmmmmmmmmiiiiigggggggggggggg
Getting set up
arcpy for ArcGIS 10 requires numpy 1.3.0
This means we’re restricted to scipy 0.7.1, matplotlib 1.0.1 and PIL 1.1.7 (which is still the latest version)
If you’re not using ArcGIS, or you’re using virtualenv get the latest versions!
All code for this talk can be downloaded from github:https://github.com/om-henners/ndimage_talk.git
Why process with scipy?
Open Source scientific algorithms
Easy to set up for large concurrent processing on local PCs and in the cloud (see PiCloud)
It’s free!*
*Well, aside from development cost
Getting data in and out
We’ll be using the arcpy.RasterToNumPyArray and arcpy.NumPyArrayToRaster functions
Alternatives include GDAL Python bindings, scipy image read functions, and many others
See getting_data_in.py and getting_data_out.py
Calculating median filter
Once you’ve got an array it’s one line of code to perform a basic filter function
scipy.ndimage.filters.median_filter(a, size=9)
Getting more complex
n-dimensional processing
We can supply a size or filter shape, but now we have to be even more aware of edge effects
Still the same one line of code
Generic filters in scipy
Most Spatial Analyst operations are in ndimage
For everything else there’s generic_filter
Flattens the target region and passes through to a callback function
Can be used to handle null data as ndimage can’t handle masked arrays (yet)
Calculating slope
We can implement a simple slope calculation using generic filter over a 3x3 footprint
We can use the form as described by the ESRI documentation:
slope = e is atan ( sqrt( [dz/dx]**2 + [dz/dy]**2 ) ) where[dz/dx] = ((c + 2f + i) - (a + 2d + g) / (8 * x_cellsize)[dz/dy] = ((g + 2h + i) - (a + 2b + c)) / (8 * y_cellsize))
Slower than a standard ndimage filter, but faster thanarcpy!
Calculating a variable focal maximum
Calculate the focal maximum for every point in an array, with the focal annulus defined by a radius from another raster
Don’t use a generic filter (there’s no need)
Don’t use ArcGIS Spatial analyst (it’s really slow)
Method 100x100 random raster
generic filter 4.7 secs
arcpy Spatial Analyst 6.7 secs
Do use inbuilt functions
It’s faster to calculate the focal sum using ndimage.maximum_filter for every possible buffer value than to roll your own function
100x100 array? 0.009 seconds
Last words on scipy.ndimage
It’s awesome, and free and threadsafe
It ties into any number of other packages (scikits-learn, scikits-image)
You can do just about anything in ndimage that you can in Spatial Analyst
If you have time to code it
And code it right
John Hunter (1968 - 2012)
http://numfocus.org/johnhunter/