Downloaded 12 times
![Linear Spectral Unmixing
The Process
Soil
Vegetation
Non-photosynthetic vegetation
Shadows
Shadow: [70.05629, 27.24081, 25.31275, 24.17432, 31.77904, 17.82422]
Veg: [65.46086, 30.09995, 26.27376, 117.45741, 76.96012, 26.25062]
NPV: [74.74029, 32.06931, 35.57350, 32.66032, 73.63062, 60.51104]
Soil: [143.65580, 79.30271, 102.82176, 93.60246, 176.57705, 117.49280]
Linear
Unmixing](https://image.slidesharecdn.com/pythonrasterfunction-devsummmit2015-150312141619-conversion-gate01/75/Python-Raster-Function-Esri-Developer-Conference-2015-12-2048.jpg)
![Linear Spectral Unmixing
The Process
Soil
Vegetation
Non-photosynthetic vegetation
Shadows
Shadow: [70.05629, 27.24081, 25.31275, 24.17432, 31.77904, 17.82422]
Veg: [65.46086, 30.09995, 26.27376, 117.45741, 76.96012, 26.25062]
NPV: [74.74029, 32.06931, 35.57350, 32.66032, 73.63062, 60.51104]
Soil: [143.65580, 79.30271, 102.82176, 93.60246, 176.57705, 117.49280]
Linear
Unmixing](https://crownmelresort.com/image.slidesharecdn.com/pythonrasterfunction-devsummmit2015-150312141619-conversion-gate01/75/Python-Raster-Function-Esri-Developer-Conference-2015-12-2048.jpg)
More Related Content
Similar to Python Raster Function - Esri Developer Conference - 2015
Python Raster Function - Esri Developer Conference - 2015
- 1. Python Raster Function: CustomOn-the-fly Analysis Feroz Abdul Kadar Jamie Drisdelle
- 2.
- 3. What’s a RasterFunction? • Mapping of one raster to another. • Loosely • Transient: on the fly. • Different from a geoprocessing tool. … a transformation of one raster into another. The Fundamentals
- 4. What’s a RasterFunction Chain? The Basic Model Function Raster Dataset Mosaic Dataset Item Image Service Mosaic Dataset A raster function “chain” encapsulate your algorithm as a tree of functions.
- 5. What’s a RasterFunction Chain? The Basic Model
- 6.
- 7. What’s a PythonRaster Function? • Natural evolution of the raster function’s COM API—usually implemented in .NET • Implement a raster function from the comfort of your Python module. • Architecture: Module loaded by an adapter—Python-aware and a first-class participant in the function chain. Your Python module—assisted by ArcGIS—is a raster function. The Extended Model
- 8. What’s a PythonRaster Function? The Extended Model Function Raster Dataset Mosaic Dataset Item Image Service Mosaic Dataset
- 9. Motivation • Extend ArcGIS—participatein a raster function chain. • Primary pipeline for image data in ArcGIS—processing, analyzing, and visualizing… on the fly. • Portable. Reusable. Dynamic. Fast. Scalable. • Why Python? - Friendly & easy to learn. “readability first”. “batteries included”. - Huge collection of libraries. Vibrant community of Pythonistas and Pythoneers. - “…de facto superglue language for modern scientific computing.” - “…tools for almost every aspect of scientific computing are readily available in Python.” Why Raster Functions in Python?
- 10. An Example Raster Functionin Python: Linear Spectral Unmixing Courtesy: Jacob Wasilkowski. @jwasil
- 11. Linear Spectral Unmixing •Determines abundance of certain material in a particular pixel. • Typically performed on lower resolution imagery (like Landsat) • Input training endmember spectra required for each class Introducing a Sample Use Case
- 12. Linear Spectral Unmixing TheProcess Soil Vegetation Non-photosynthetic vegetation Shadows Shadow: [70.05629, 27.24081, 25.31275, 24.17432, 31.77904, 17.82422] Veg: [65.46086, 30.09995, 26.27376, 117.45741, 76.96012, 26.25062] NPV: [74.74029, 32.06931, 35.57350, 32.66032, 73.63062, 60.51104] Soil: [143.65580, 79.30271, 102.82176, 93.60246, 176.57705, 117.49280] Linear Unmixing
- 13. Demo A Python RasterFunction in Action: Linear Spectral Unmixing Courtesy: Jacob Wasilkowski. @jwasil
- 14. The API • Howdoes ArcGIS Desktop or Server interact with my raster function? • Get started—step-by-step guide • Real-world or reference implementations—excellent springboard • Well-documented API reference • What additional libraries are needed? How complicated is it? • Lightweight design—no external dependencies outside of NumPy to begin with. • ArcGIS’ adapter provides assistance—opt out to take control of specific aspects. Create a new raster function using simple and familiar Pythonesque constructs. How do I create Raster Function in Python?
- 15.
- 16. The API • customizeour function object—a specific instance of our class—as soon as it's created. • Define raster function name & description. __init__
- 17. The API • Defineall input parameters to the function. • For each parameter, define: - Name (Identifier) - Display Name - Long Description - Data Type - Default Value - Required vs Optional - … getParameterInfo()
- 18. The API • Howare input rasters read—Padding, Mask, …? • How’s the output raster constructed—inherit NoData, Metadata, …? • Given: Nothing. • Returns: dictionary containing configuration attribute values. getConfiguration()
- 19. The API • Definea subset of input rasters. • Pixels read from selected rasters. • Given: properties of the requested pixel block, all scalar parameter values. • Returns: names of selected rasters. selectRasters()
- 20. The API • Definesthe output raster. • Invoked each time a dataset containing the Python raster function is initialized. • Given: Raster info associated with all input rasters. • Returns: Raster Info of the output raster. updateRasterInfo()
- 21. The API • Workhorseof the raster function. Process Pixels. • Given: • Expected pixel-block size+location • output raster properties (map space) • pixels+mask of selected input rasters • Returns: Pixels+mask of requested pixel block. updatePixels()
- 22. The API • Createor update dataset- or band-level metadata. • Given: • property names • band index • current key metadata values • Returns: updated values of given properties updateKeyMetadata()
- 23. The API • Given: •info on parent product, • context of execution. • Returns: • OK to Run (Boolean)—Licensed to execute or not? • Expected product level and extension. isLicensed()
- 24.
- 25.
- 26. What’s a RasterFunction Template? A raster function template anonymizes select raster and scalar inputs as Variables. The Basic Model Template Input Raster Variables Input Scalar Variables Output Raster
- 27. Raster Function Templates •Create new function templates - Via Raster Function Template Editor - Layer > Symbology > Export As Raster Function Template - Function Raster Dataset > Functions > Save as • Function Raster Layer in a Map • Image Analysis Window • Raster Functions Pane in Pro • Layer > Properties > Functions tab A raster function template makes your processing or analysis portable. Basic Workflows
- 28. Raster Function Templates •On a Mosaic Dataset - Populating a mosaic using the Add Rasters tool - Mosaic dataset items - Batch Edit Raster Functions or - Edit Raster Function Geoprocessing Tool. - As Processing Templates • On an Image Service—for server-side processing Learn more at https://github.com/Esri/raster-functions#raster-function-templates. Advanced Workflows
- 29. Raster Function Templates •Name & Description • Type: Item, Group, or Mosaic. • Definition Query • Fields that control grouping. Properties
- 30. Demo Processing Templates ona Mosaic Dataset
- 31.
- 32.
- 33. Performance • Vectorize. • UseNumPy and SciPy. • Use Cython: For production-grade performance. • Bridge to C/C++ implementations via ctypes or Boost.Python. Leverage well-known options to optimize time-critical parts of your raster function.
- 34. Publishing & Deployment •Python version • Desktop vs. Pro: Python 2.7 vs. 3.4 • Desktop vs. Server: 32- vs. 64-bit Python 2.7 • Package dependencies • Binary deployment • CPython bytecode. Cython compiled binary. • isLicensed method to restrict usage.
- 35. GitHub • https://github.com/Esri/raster-functions • Curatedcollection of raster functions and templates. • Go ahead: - Browse samples to learn more. - Fork the repo. Experiment. - Log defects or enhancement requests as issues. - Send pull requests to contribute code. GitHub enables collaboration. Where do functions and templates live?
- 36. Wiki • https://github.com/Esri/raster-functions/wiki • Detailsof interaction between ArcGIS and your Python raster function. • Recommendations and techniques for writing effective raster functions Where do I find the story?
- 37. The Road Ahead •Tools for testing your raster functions—profiling or debugging. • Streamline usability of Python raster functions in ArcGIS Pro. • Integration with Portal workflows. • Expand and maintain the wiki: API, recommendations, resources. …for Pythonistas and Pythoneers with a passion for scientific computing.
- 38. In Closing • Powerfulpipeline for processing, analysis, and visualization. - Raster Functions. Chains. Templates. • Using simple constructs in Python: participate and exploit. • Code and docs are on GitHub. • We’ll help you along the way—@jdrisdelle and @akferoz. Wake up! We are done.
- 39. Rate This Session www.esri.com/RateMyDevSummitSession Sessionson related topics: Python: Working with Raster Data—Wednesday at 4:00 Python: Working with Scientific Data—Wednesday at 5:30 http://www.esri.com/events/devsummit/agenda
Editor's Notes
- #10 Python (NumPy/SciPy) for numerical computing; Python (NumPy/SciPy/pandas/statsmodels) for statistical analysis; Python (MatPlotLib) for plotting and visualization Python (scikit-learn) for machine learning; “Python is becoming the de facto superglue language for modern scientific computing.” --http://profjsb.github.io/python-seminar/ “These days, tools for almost every aspect of scientific computing are readily available in Python.” --http://www.talyarkoni.org/blog/2013/11/18/the-homogenization-of-scientific-computing-or-why-python-is-steadily-eating-other-languages-lunch/
- #12 “Spectral imaging sensors often record scenes in which numerous disparate material substances contribute to the spectrum measured from a single pixel. Given such mixed pixels, we want to identify the individual constituent materials present in the mixture, as well as the proportions in which they appear. Spectral unmixing is the procedure by which the measured spectrum of a mixed pixel is decomposed into a collection of constituent spectra, or endmembers, and a set of corresponding fractions, or abundances, that indicate the proportion of each endmember present in the pixel. Endmembers normally correspond to familiar macroscopic objects in the scene, such as water, soil, metal, or any natural or man-made material. Unmixing provides a capability that is important in numerous tactical scenarios in which subpixel detail is valuable.” – Nirmal Keshava
- #16 import numpy as np class Addition(): def __init__(self): self.name = "Addition Function" def getParameterInfo(self): return [{ 'name': 'r', 'dataType': 'raster', }] def updatePixels(self, tlc, shape, props, **pixelBlocks): r = pixelBlocks['r_pixels'] + 10 pixelBlocks['output_pixels'] = r.astype(props['pixelType']) return pixelBlocks
- #28 10.3.1: . http://desktop.arcgis.com/en/desktop/latest/manage-data/raster-and-images/editing-function-chains-in-md.htm . http://desktop.arcgis.com/en/desktop/latest/tools/data-management-toolbox/edit-raster-function.htm Pro: . http://pro.arcgis.com/en/pro-app/help/data/imagery/apply-functions-to-a-dataset.htm
- #34 Cython extends Python syntax so that you can conveniently build C extensions, either to speed up critical code, or to integrate with C/C++ libraries.
- #35 Learn more: http://desktop.arcgis.com/en/desktop/latest/analyze/sharing-workflows/deploying-custom-python-packages.htm